2024-03-19T00:07:35.164 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-19T00:07:35.165 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/kernel_untar_build}} --archive /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245 --name pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245/orig.config.yaml 2024-03-19T00:07:35.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-19T00:07:35.320 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245 branch: wip-batrick-testing-20240318.181317 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/kernel_untar_build}} email: pdonnell@redhat.com first_in_suite: false job_id: '7609245' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240318.181317 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - PG_AVAILABILITY - PG_DEGRADED - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240318.181317 sha1: e48b1f6c0608b2615e1890cc46286c3877d6e910 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3005 sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 sleep_before_teardown: 0 subset: 433/512 suite: fs suite_branch: wip-batrick-testing-20240318.181317 suite_path: /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: e48b1f6c0608b2615e1890cc46286c3877d6e910 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGSTS8WjOSgm25/wh31ZgyFJookYsiny1sF8uKwrPy8zX0WPys5NAlc6S7eWESl8+Cc5hGCNUuy/06arND/TEmg= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFmS9BN+w4hdh7UG5GLfF38yw5sMZ3EGVaDjQ7r/TUz/28kmdUjUYE0Pv+4n1b6H4hRBGlSKvSTZHY5igkRmjE4= smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKxq9gaGaFTQoAjNVdlBop7VwVaMDGxPMpyddTH+j+v06Ke7MxGbUQu30X9yjirPsvbJd8U/rv4PKhkcvil8d+0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - kernel_untar_build.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-18_21:05:49 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-19T00:07:35.320 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa; will attempt to use it 2024-03-19T00:07:35.321 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks 2024-03-19T00:07:35.321 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-19T00:07:35.323 INFO:teuthology.task.internal:Checking packages... 2024-03-19T00:07:35.346 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b6ae24918b03b1b8a19c9263a50655bba80bb810' 2024-03-19T00:07:35.346 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-19T00:07:35.346 INFO:teuthology.packaging:ref: None 2024-03-19T00:07:35.346 INFO:teuthology.packaging:tag: None 2024-03-19T00:07:35.346 INFO:teuthology.packaging:branch: wip-batrick-testing-20240318.181317 2024-03-19T00:07:35.347 INFO:teuthology.packaging:sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:07:35.347 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240318.181317 2024-03-19T00:07:35.579 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2206.gb6ae2491 2024-03-19T00:07:35.581 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-19T00:07:35.590 INFO:teuthology.task.internal:no buildpackages task found 2024-03-19T00:07:35.590 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-19T00:07:35.603 INFO:teuthology.task.internal:Saving configuration 2024-03-19T00:07:35.623 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-19T00:07:35.634 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-19T00:07:35.659 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:01:51.146747', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGSTS8WjOSgm25/wh31ZgyFJookYsiny1sF8uKwrPy8zX0WPys5NAlc6S7eWESl8+Cc5hGCNUuy/06arND/TEmg='} 2024-03-19T00:07:35.681 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:01:51.150374', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFmS9BN+w4hdh7UG5GLfF38yw5sMZ3EGVaDjQ7r/TUz/28kmdUjUYE0Pv+4n1b6H4hRBGlSKvSTZHY5igkRmjE4='} 2024-03-19T00:07:35.706 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:01:51.148665', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKxq9gaGaFTQoAjNVdlBop7VwVaMDGxPMpyddTH+j+v06Ke7MxGbUQu30X9yjirPsvbJd8U/rv4PKhkcvil8d+0='} 2024-03-19T00:07:35.706 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-19T00:07:35.718 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-19T00:07:35.718 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-19T00:07:35.718 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-19T00:07:35.719 INFO:teuthology.run_tasks:Running task console_log... 2024-03-19T00:07:35.838 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f121641d160>, signals=[15]) 2024-03-19T00:07:35.838 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-19T00:07:35.845 INFO:teuthology.task.internal:Opening connections... 2024-03-19T00:07:35.845 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2024-03-19T00:07:35.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:07:35.938 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-03-19T00:07:35.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:07:36.029 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2024-03-19T00:07:36.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:07:36.118 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-19T00:07:36.126 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2024-03-19T00:07:36.143 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2024-03-19T00:07:36.144 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-03-19T00:07:36.200 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-03-19T00:07:36.200 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-03-19T00:07:36.200 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-03-19T00:07:36.200 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-03-19T00:07:36.200 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-03-19T00:07:36.201 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-03-19T00:07:36.201 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-19T00:07:36.201 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-03-19T00:07:36.201 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-03-19T00:07:36.201 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-19T00:07:36.201 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-03-19T00:07:36.201 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-19T00:07:36.201 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-19T00:07:36.201 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-19T00:07:36.202 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-03-19T00:07:36.229 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-03-19T00:07:36.247 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-03-19T00:07:36.248 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-03-19T00:07:36.304 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-03-19T00:07:36.304 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-03-19T00:07:36.304 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-03-19T00:07:36.304 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-03-19T00:07:36.304 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-03-19T00:07:36.305 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-03-19T00:07:36.305 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-19T00:07:36.305 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-03-19T00:07:36.305 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-03-19T00:07:36.305 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-19T00:07:36.305 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-03-19T00:07:36.305 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-19T00:07:36.305 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-19T00:07:36.305 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-19T00:07:36.305 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-03-19T00:07:36.329 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2024-03-19T00:07:36.347 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2024-03-19T00:07:36.348 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-03-19T00:07:36.404 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream" 2024-03-19T00:07:36.404 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9" 2024-03-19T00:07:36.404 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos" 2024-03-19T00:07:36.404 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora" 2024-03-19T00:07:36.405 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9" 2024-03-19T00:07:36.405 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9" 2024-03-19T00:07:36.405 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-19T00:07:36.405 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2024-03-19T00:07:36.405 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon" 2024-03-19T00:07:36.405 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-19T00:07:36.405 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/" 2024-03-19T00:07:36.405 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-19T00:07:36.405 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-19T00:07:36.405 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-19T00:07:36.405 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-03-19T00:07:36.429 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-19T00:07:36.441 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-19T00:07:36.463 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-19T00:07:36.463 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2024-03-19T00:07:36.466 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-03-19T00:07:36.468 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2024-03-19T00:07:36.485 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-19T00:07:36.493 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-19T00:07:36.493 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2024-03-19T00:07:36.522 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-03-19T00:07:36.525 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2024-03-19T00:07:36.546 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-19T00:07:36.629 INFO:teuthology.run_tasks:Running task kernel... 2024-03-19T00:07:36.645 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-19T00:07:36.645 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-19T00:07:36.645 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-19T00:07:36.646 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-19T00:07:36.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:07:36.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-19T00:07:36.894 DEBUG:teuthology.task.kernel:sha1 for client.0 is 73069ad93affa1ccfbc13c95e629925210a73624 2024-03-19T00:07:36.894 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "73069ad93affa1ccfbc13c95e629925210a73624"... 2024-03-19T00:07:36.894 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-03-19T00:07:36.911 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-361.el9.x86_64 2024-03-19T00:07:36.911 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 73069ad93affa1ccfbc13c95e629925210a73624 2024-03-19T00:07:36.912 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-19T00:07:36.912 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum upgrade -y linux-firmware 2024-03-19T00:07:39.119 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-19T00:07:40.510 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-03-19T00:18:38.497 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 13 kB/s | 8.0 MB 10:57 2024-03-19T00:18:42.822 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 19 MB 00:03 2024-03-19T00:18:47.656 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.1 MB 00:01 2024-03-19T00:18:49.457 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-03-19T00:18:50.592 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-03-19T00:18:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-19T00:18:57.760 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-19T00:18:57.771 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-19T00:18:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-03-19T00:18:57.771 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-19T00:18:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2024-03-19T00:18:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-19T00:18:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:18:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-19T00:18:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-19T00:18:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 1 Package 2024-03-19T00:18:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:18:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 375 M 2024-03-19T00:18:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-19T00:19:05.117 INFO:teuthology.orchestra.run.smithi069.stdout:linux-firmware-20240219-143.el9.noarch.rpm 51 MB/s | 375 MB 00:07 2024-03-19T00:19:05.118 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-19T00:19:05.118 INFO:teuthology.orchestra.run.smithi069.stdout:Total 51 MB/s | 375 MB 00:07 2024-03-19T00:19:05.119 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-19T00:19:05.200 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-19T00:19:05.201 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-19T00:19:07.647 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-19T00:19:07.648 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-19T00:19:14.021 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-19T00:19:14.244 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-19T00:19:14.316 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-19T00:19:15.689 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-19T00:19:15.689 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-19T00:19:16.095 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-19T00:19:16.095 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:19:16.095 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2024-03-19T00:19:16.095 INFO:teuthology.orchestra.run.smithi069.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-19T00:19:16.095 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:19:16.095 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-19T00:19:16.421 INFO:teuthology.task.kernel:Downloading kernel 73069ad93affa1ccfbc13c95e629925210a73624 on client.0... 2024-03-19T00:19:16.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=73069ad93affa1ccfbc13c95e629925210a73624 2024-03-19T00:19:16.636 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/73069ad93affa1ccfbc13c95e629925210a73624/centos/9/flavors/default/x86_64/ 2024-03-19T00:19:16.636 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g73069ad93aff-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/73069ad93affa1ccfbc13c95e629925210a73624/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-19T00:19:16.639 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi069.front.sepia.ceph.com to complete... 2024-03-19T00:19:28.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19 00:19:28 URL:https://3.chacra.ceph.com/r/kernel/testing/73069ad93affa1ccfbc13c95e629925210a73624/centos/9/flavors/default/x86_64/kernel-6.8.0_g73069ad93aff-1.x86_64.rpm [336343082/336343082] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-19T00:19:28.240 INFO:teuthology.orchestra.run.smithi069.stderr:FINISHED --2024-03-19 00:19:28-- 2024-03-19T00:19:28.240 INFO:teuthology.orchestra.run.smithi069.stderr:Total wall clock time: 12s 2024-03-19T00:19:28.240 INFO:teuthology.orchestra.run.smithi069.stderr:Downloaded: 1 files, 321M in 11s (28.5 MB/s) 2024-03-19T00:19:28.242 INFO:teuthology.task.kernel:Installing kernel 73069ad93affa1ccfbc13c95e629925210a73624 on client.0... 2024-03-19T00:19:28.243 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-19T00:19:30.171 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying... ######################################## 2024-03-19T00:19:30.299 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing... ######################################## 2024-03-19T00:19:30.378 INFO:teuthology.orchestra.run.smithi069.stdout:Updating / installing... 2024-03-19T00:19:37.913 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-6.8.0_g73069ad93aff-1 ######################################## 2024-03-19T00:19:53.710 INFO:teuthology.orchestra.run.smithi069.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-19T00:20:16.266 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi069.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-19T00:20:16.266 DEBUG:teuthology.orchestra.run.smithi069:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-19T00:20:16.337 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff 2024-03-19T00:20:16.337 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/System.map 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/config 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-19T00:20:16.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/kernel 2024-03-19T00:20:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/kernel/cpu 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/kernel/cpu/mce 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/kvm 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/arch/x86/kvm/kvm.ko 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/ansi_cprng.ko 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/anubis.ko 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/arc4.ko 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/async_tx 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/async_tx/async_memcpy.ko 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/async_tx/async_pq.ko 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-19T00:20:16.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/async_tx/async_tx.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/async_tx/async_xor.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/async_tx/raid6test.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/authenc.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/authencesn.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/blake2b_generic.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/blowfish_common.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/blowfish_generic.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/camellia_generic.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/cast5_generic.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/cast6_generic.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/cast_common.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/ccm.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/cmac.ko 2024-03-19T00:20:16.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/crc32_generic.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/crypto_engine.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/crypto_user.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/cts.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/des_generic.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/dh_generic.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/echainiv.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/essiv.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/fcrypt.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/khazad.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/lz4.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/lz4hc.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/md4.ko 2024-03-19T00:20:16.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/michael_mic.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/pcbc.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/pcrypt.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/rmd160.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/seed.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/serpent_generic.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/tcrypt.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/tea.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/twofish_common.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/twofish_generic.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/vmac.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/wp512.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/xcbc.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/xor.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/crypto/xxhash_generic.ko 2024-03-19T00:20:16.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/acpi 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/acpi/acpi_pad.ko 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/acpi/custom_method.ko 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/acpi/ec_sys.ko 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/acpi/platform_profile.ko 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/acpi/sbs.ko 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/acpi/sbshc.ko 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/acpi/video.ko 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/acard-ahci.ko 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/ahci_platform.ko 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/ata_generic.ko 2024-03-19T00:20:16.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/libahci_platform.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_acpi.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_ali.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_amd.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_artop.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_atiixp.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_atp867x.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_cmd640.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_cmd64x.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_cypress.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_efar.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_hpt366.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_hpt37x.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-19T00:20:16.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_it8213.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_it821x.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_jmicron.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_marvell.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_mpiix.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_netcell.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_ninja32.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_ns87410.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_ns87415.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_oldpiix.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_opti.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_optidma.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_pcmcia.ko 2024-03-19T00:20:16.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_piccolo.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_rdc.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_sch.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_serverworks.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_sil680.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_sis.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_sl82c105.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_triflex.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pata_via.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/pdc_adma.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_inic162x.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_mv.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_nv.ko 2024-03-19T00:20:16.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_promise.ko 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_qstor.ko 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_sil.ko 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_sil24.ko 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_sis.ko 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_svw.ko 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_sx4.ko 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_uli.ko 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_via.ko 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ata/sata_vsc.ko 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/auxdisplay 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/auxdisplay/ks0108.ko 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/base 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/base/regmap 2024-03-19T00:20:16.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/bcma 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/bcma/bcma.ko 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/aoe 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/aoe/aoe.ko 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/brd.ko 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/drbd 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/drbd/drbd.ko 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/floppy.ko 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/loop.ko 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/mtip32xx 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-19T00:20:16.349 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/nbd.ko 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/null_blk 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/null_blk/null_blk.ko 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/pktcdvd.ko 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/rbd.ko 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/virtio_blk.ko 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/xen-blkback 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/xen-blkfront.ko 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/zram 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/block/zram/zram.ko 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/hangcheck-timer.ko 2024-03-19T00:20:16.350 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/hw_random 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/hw_random/via-rng.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/ipmi 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/lp.ko 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/mwave 2024-03-19T00:20:16.351 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/mwave/mwave.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/ppdev.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/tlclk.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/tpm 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/tpm/tpm.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/uv_mmtimer.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/char/virtio_console.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/cpufreq 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-19T00:20:16.352 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/ccp 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/ccp/ccp.ko 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/intel 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/intel/qat 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/intel/qat/qat_common 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-19T00:20:16.353 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/padlock-aes.ko 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/padlock-sha.ko 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/virtio 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/dca 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/dca/dca.ko 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/devfreq 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/dma 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/dma/dw 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/dma/ioat 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-19T00:20:16.354 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/amd64_edac.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/e752x_edac.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/edac_mce_amd.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/i3000_edac.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/i3200_edac.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/i5100_edac.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/i5400_edac.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/i7300_edac.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/i7core_edac.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/i82975x_edac.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/ie31200_edac.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/sb_edac.ko 2024-03-19T00:20:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/edac/x38_edac.ko 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/firmware 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/firmware/edd.ko 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/gpio 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/gpio/gpio-ich.ko 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-appleir.ko 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-aureal.ko 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-axff.ko 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-dr.ko 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-elecom.ko 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-elo.ko 2024-03-19T00:20:16.356 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-emsff.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-gaff.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-gt683r.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-gyration.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-holtekff.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-hyperv.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-icade.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-keytouch.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-kye.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-lcpower.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-led.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-lenovo.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-19T00:20:16.357 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-multitouch.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-ortek.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-penmount.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-petalynx.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-picolcd.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-pl.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-plantronics.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-primax.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-rmi.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat-common.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-19T00:20:16.358 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-roccat.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-saitek.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-samsung.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-sjoy.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-sony.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-speedlink.ko 2024-03-19T00:20:16.359 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-steelseries.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-sunplus.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-tivo.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-tmff.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-topseed.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-twinhan.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-uclogic.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-waltop.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-wiimote.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-xinmo.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-zpff.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/hid-zydacron.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/uhid.ko 2024-03-19T00:20:16.360 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hid/wacom.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hv 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hv/hv_balloon.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hv/hv_utils.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hv/hv_vmbus.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/abituguru.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/abituguru3.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ad7414.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ad7418.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adc128d818.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adm1025.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adm1026.ko 2024-03-19T00:20:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adm1029.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adm1031.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adm9240.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ads7828.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adt7410.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adt7411.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adt7462.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adt7470.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adt7475.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/adt7x10.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/amc6821.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/applesmc.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/asb100.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/asc7621.ko 2024-03-19T00:20:16.362 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/atxp1.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/coretemp.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/dme1737.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ds1621.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ds620.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/emc1403.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/emc6w201.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/f71805f.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/f71882fg.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/f75375s.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/fam15h_power.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/fschmd.ko 2024-03-19T00:20:16.363 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/g760a.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/g762.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/gl518sm.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/gl520sm.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/i5500_temp.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/i5k_amb.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ibmaem.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ibmpex.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ina209.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ina2xx.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/it87.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/k10temp.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/k8temp.ko 2024-03-19T00:20:16.364 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lineage-pem.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm63.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm73.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm75.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm77.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm78.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm80.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm83.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm85.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm87.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm90.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm92.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm93.ko 2024-03-19T00:20:16.365 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm95234.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm95241.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/lm95245.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ltc2945.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ltc4151.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ltc4215.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ltc4222.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ltc4245.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ltc4260.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/ltc4261.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/max16065.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/max1619.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/max1668.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/max197.ko 2024-03-19T00:20:16.366 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/max6639.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/max6650.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/max6697.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/mcp3021.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/nct6683.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/nct6775-core.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/nct6775.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/nct7802.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pc87360.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pc87427.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pcf8591.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-19T00:20:16.367 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/powr1220.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/sch5627.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/sch5636.ko 2024-03-19T00:20:16.368 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/sht15.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/sht21.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/shtc1.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/sis5595.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/smsc47b397.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/smsc47m1.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/smsc47m192.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/thmc50.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/tmp102.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/tmp103.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/tmp401.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/tmp421.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/via-cputemp.ko 2024-03-19T00:20:16.369 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/via686a.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/vt1211.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/vt8231.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/w83627ehf.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/w83627hf.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/w83781d.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/w83791d.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/w83792d.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/w83793.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/w83795.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/w83l785ts.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/hwmon/w83l786ng.ko 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c 2024-03-19T00:20:16.370 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/algos 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-19T00:20:16.371 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-19T00:20:16.372 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-19T00:20:16.372 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-19T00:20:16.372 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-19T00:20:16.372 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-19T00:20:16.372 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-19T00:20:16.374 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-19T00:20:16.374 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-19T00:20:16.374 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-19T00:20:16.374 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-19T00:20:16.374 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-19T00:20:16.374 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-19T00:20:16.374 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/i2c-dev.ko 2024-03-19T00:20:16.374 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/i2c-mux.ko 2024-03-19T00:20:16.374 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-19T00:20:16.374 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/i2c-smbus.ko 2024-03-19T00:20:16.374 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/i2c/i2c-stub.ko 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/core 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/core/ib_core.ko 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/hw 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/hw/cxgb4 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/hw/mlx4 2024-03-19T00:20:16.375 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/hw/mlx5 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/hw/mthca 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/ulp 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/ulp/ipoib 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/ulp/iser 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/ulp/isert 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/ulp/srp 2024-03-19T00:20:16.376 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/ulp/srpt 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/gameport 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/gameport/gameport.ko 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/joydev.ko 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/keyboard 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/mouse 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/mouse/appletouch.ko 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/mouse/bcm5974.ko 2024-03-19T00:20:16.377 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/mouse/cyapatp.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/mouse/sermouse.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/rmi4 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/serio 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/serio/altera_ps2.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/serio/arc_ps2.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/serio/serio_raw.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/input/sparse-keymap.ko 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds 2024-03-19T00:20:16.378 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/leds-blinkm.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/leds-lm3530.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/leds-lp3944.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/leds-ss4200.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/trigger 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-19T00:20:16.379 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/bcache 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/bcache/bcache.ko 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-bio-prison.ko 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-cache-smq.ko 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-cache.ko 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-crypt.ko 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-delay.ko 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-flakey.ko 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-log-userspace.ko 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-multipath.ko 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-queue-length.ko 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-raid.ko 2024-03-19T00:20:16.380 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-round-robin.ko 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-service-time.ko 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-switch.ko 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-thin-pool.ko 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/dm-verity.ko 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/persistent-data 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/raid0.ko 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/raid1.ko 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/raid10.ko 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/md/raid456.ko 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/message 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/message/fusion 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/message/fusion/mptbase.ko 2024-03-19T00:20:16.381 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/message/fusion/mptctl.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/message/fusion/mptfc.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/message/fusion/mptlan.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/message/fusion/mptsas.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/message/fusion/mptscsih.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/message/fusion/mptspi.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/mfd 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/mfd/lpc_ich.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/mfd/lpc_sch.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/mfd/mfd-core.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/mfd/sm501.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/mfd/viperboard.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/mfd/vx855.ko 2024-03-19T00:20:16.382 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/mfd/wl1273-core.ko 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/altera-stapl 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/apds9802als.ko 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/apds990x.ko 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/bh1770glc.ko 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/cb710 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/cb710/cb710.ko 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/eeprom 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/eeprom/at24.ko 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/eeprom/max6875.ko 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/enclosure.ko 2024-03-19T00:20:16.383 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/hpilo.ko 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/isl29003.ko 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/isl29020.ko 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/lis3lv02d 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/mei 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/mei/mei-me.ko 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/mei/mei-txe.ko 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/mei/mei.ko 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/sgi-gru 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/sgi-xp 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-19T00:20:16.384 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/tifm_7xx1.ko 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/tifm_core.ko 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/tsl2550.ko 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/vmw_balloon.ko 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/vmw_vmci 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/bonding 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/bonding/bonding.ko 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/dsa 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/dsa/b53 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-19T00:20:16.385 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/dsa/mv88e6xxx 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/dummy.ko 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/eql.ko 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/3com 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/8390 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-19T00:20:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/adaptec 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/agere 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/alteon 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/altera 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/amd 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-19T00:20:16.387 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/atheros 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/atheros/alx 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/atheros/atlx 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/broadcom 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-19T00:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/broadcom/genet 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/brocade 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/brocade/bna 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/chelsio 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-19T00:20:16.389 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/cisco 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/cisco/enic 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dec 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dec/tulip 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-19T00:20:16.390 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dlink 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/dnet.ko 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/emulex 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/emulex/benet 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/ethoc.ko 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/fealnx.ko 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel 2024-03-19T00:20:16.391 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/e1000 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/e1000e 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/fm10k 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/i40e 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/iavf 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/igb 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/igbvf 2024-03-19T00:20:16.392 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/jme.ko 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/marvell 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/mellanox 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-19T00:20:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-19T00:20:16.394 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-19T00:20:16.394 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/micrel 2024-03-19T00:20:16.394 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-19T00:20:16.394 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/myricom 2024-03-19T00:20:16.394 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-19T00:20:16.394 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-19T00:20:16.394 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/natsemi 2024-03-19T00:20:16.394 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/neterion 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/nvidia 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/packetengines 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/qlogic 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-19T00:20:16.395 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/rdc 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/realtek 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/rocker 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/sfc 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/silan 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/sis 2024-03-19T00:20:16.396 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/smsc 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/stmicro 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/sun 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-19T00:20:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/tehuti 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/ti 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/via 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/wiznet 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/xircom 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-19T00:20:16.398 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/geneve.ko 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/hyperv 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ieee802154 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ifb.ko 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ipvlan 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/macvlan.ko 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/macvtap.ko 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/mdio 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/mdio.ko 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-19T00:20:16.399 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/mii.ko 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/net_failover.ko 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/netconsole.ko 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/nlmon.ko 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/pcs 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/amd.ko 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-19T00:20:16.400 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/bcm87xx.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/broadcom.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/cicada.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/davicom.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/dp83640.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/et1011c.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/icplus.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/lxt.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/marvell.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/micrel.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/national.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/phylink.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/qsemi.ko 2024-03-19T00:20:16.401 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/realtek.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/smsc.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/ste10Xp.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/phy/vitesse.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ppp 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ppp/ppp_async.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ppp/pppoe.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ppp/pppox.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/ppp/pptp.ko 2024-03-19T00:20:16.402 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/slip 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/slip/slhc.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/slip/slip.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/sungem_phy.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/tap.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/team 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/team/team.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/team/team_mode_random.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/tun.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/veth.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/virtio_net.ko 2024-03-19T00:20:16.403 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/vmxnet3 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/vxlan 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/vxlan/vxlan.ko 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/xen-netback 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/net/xen-netfront.ko 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ntb 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ntb/ntb.ko 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/nvme 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/nvme/host 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/nvme/host/nvme-core.ko 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/nvme/host/nvme.ko 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/parport 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/parport/parport.ko 2024-03-19T00:20:16.404 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/parport/parport_cs.ko 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/parport/parport_pc.ko 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/parport/parport_serial.ko 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pci 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pci/hotplug 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pci/pcie 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pci/xen-pcifront.ko 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pcmcia 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pcmcia/i82092.ko 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pcmcia/pd6729.ko 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pinctrl 2024-03-19T00:20:16.405 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pinctrl/intel 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/acerhdf.ko 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-19T00:20:16.406 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/hdaps.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/intel 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/intel_ips.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/platform/x86/wmi.ko 2024-03-19T00:20:16.407 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/powercap 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pps 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pps/clients 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pps/clients/pps_parport.ko 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/pps/pps_core.ko 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ptp 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ptp/ptp.ko 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ptp/ptp_kvm.ko 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-19T00:20:16.408 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-em3027.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-19T00:20:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-max6900.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-19T00:20:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/rtc/rtc-x1205.ko 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/3w-9xxx.ko 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/3w-sas.ko 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/3w-xxxx.ko 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/BusLogic.ko 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/a100u2w.ko 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/aacraid 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/advansys.ko 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/aic7xxx 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-19T00:20:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/am53c974.ko 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/arcmsr 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/atp870u.ko 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/be2iscsi 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/bfa 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/bfa/bfa.ko 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/bnx2fc 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/bnx2i 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/ch.ko 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/csiostor 2024-03-19T00:20:16.412 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/cxgbi 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/dc395x.ko 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/device_handler 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/dmx3191d.ko 2024-03-19T00:20:16.413 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/esas2r 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/esp_scsi.ko 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/fcoe 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/fnic 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/fnic/fnic.ko 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/hpsa.ko 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/hptiop.ko 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/hv_storvsc.ko 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/initio.ko 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/ipr.ko 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/ips.ko 2024-03-19T00:20:16.414 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/isci 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/isci/isci.ko 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/libfc 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/libfc/libfc.ko 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/libiscsi.ko 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/libsas 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/libsas/libsas.ko 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/lpfc 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/megaraid 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/megaraid.ko 2024-03-19T00:20:16.415 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/mpt3sas 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/mvsas 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/mvumi.ko 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/pm8001 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/pmcraid.ko 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/qla1280.ko 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/qla2xxx 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-19T00:20:16.416 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/qla4xxx 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/raid_class.ko 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/scsi_debug.ko 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/ses.ko 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/st.ko 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/stex.ko 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/sym53c8xx_2 2024-03-19T00:20:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/virtio_scsi.ko 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/scsi/wd719x.ko 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ssb 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ssb/ssb.ko 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target/iscsi 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target/loopback 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target/target_core_file.ko 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target/target_core_iblock.ko 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target/target_core_mod.ko 2024-03-19T00:20:16.418 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target/target_core_pscsi.ko 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target/target_core_user.ko 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target/tcm_fc 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-19T00:20:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty/ipwireless 2024-03-19T00:20:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty/n_gsm.ko 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty/n_hdlc.ko 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty/nozomi.ko 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty/serial 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty/serial/8250 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty/serial/arc_uart.ko 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty/serial/jsm 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ufs 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ufs/core 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ufs/host 2024-03-19T00:20:16.421 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/uio 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/uio/uio.ko 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/uio/uio_aec.ko 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/uio/uio_cif.ko 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/uio/uio_pci_generic.ko 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/uio/uio_sercos3.ko 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/class 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/class/cdc-acm.ko 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/class/usblp.ko 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/class/usbtmc.ko 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/host 2024-03-19T00:20:16.422 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/image 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/image/mdc800.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/image/microtek.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/adutux.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/appledisplay.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/emi26.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/emi62.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/ezusb.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/idmouse.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/iowarrior.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/ldusb.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/legousbtower.ko 2024-03-19T00:20:16.423 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/sisusbvga 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/usb3503.ko 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/usblcd.ko 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/uss720.ko 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/misc/yurex.ko 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/phy 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/phy/phy-generic.ko 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/aircable.ko 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/ark3116.ko 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-19T00:20:16.424 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/ch341.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/cp210x.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/cyberjack.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/empeg.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/io_ti.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/ipaq.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/ipw.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/ir-usb.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/keyspan.ko 2024-03-19T00:20:16.425 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/mct_u232.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/mos7720.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/mos7840.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/navman.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/omninet.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/opticon.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/option.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/oti6858.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/pl2303.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/qcaux.ko 2024-03-19T00:20:16.426 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/qcserial.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/quatech2.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/safe_serial.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/sierra.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/ssu100.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/symbolserial.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/usb_debug.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/visor.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/whiteheat.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage 2024-03-19T00:20:16.427 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/uas.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-karma.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-19T00:20:16.428 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/storage/usb-storage.ko 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/usbip 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/vfio 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/vfio/pci 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/vfio/vfio.ko 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/vhost 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/vhost/vhost.ko 2024-03-19T00:20:16.429 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/vhost/vhost_net.ko 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/vhost/vhost_scsi.ko 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/video 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/video/backlight 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/video/backlight/apple_bl.ko 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/video/backlight/lcd.ko 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/virtio 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/virtio/virtio.ko 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/virtio/virtio_balloon.ko 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/virtio/virtio_mmio.ko 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/virtio/virtio_pci.ko 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-19T00:20:16.430 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/virtio/virtio_ring.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/masters 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/masters/ds2482.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/masters/ds2490.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-19T00:20:16.431 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/w1/wire.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/hpwdt.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/i6300esb.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/ib700wdt.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/ibmasr.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-19T00:20:16.432 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/it87_wdt.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/machzwd.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/nv_tco.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/softdog.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/via_wdt.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/wdt_pci.ko 2024-03-19T00:20:16.433 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/watchdog/xen_wdt.ko 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/xen 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/xen/xen-evtchn.ko 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/xen/xen-gntalloc.ko 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/xen/xen-gntdev.ko 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/xen/xen-pciback 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/xen/xen-privcmd.ko 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/xen/xenfs 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/9p 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/9p/9p.ko 2024-03-19T00:20:16.434 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/affs 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/affs/affs.ko 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/befs 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/befs/befs.ko 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/binfmt_misc.ko 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/btrfs 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/btrfs/btrfs.ko 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/cachefiles 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/cachefiles/cachefiles.ko 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ceph 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ceph/ceph.ko 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/coda 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/coda/coda.ko 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/cramfs 2024-03-19T00:20:16.435 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/cramfs/cramfs.ko 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/dlm 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/dlm/dlm.ko 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ecryptfs 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/f2fs 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/f2fs/f2fs.ko 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/fat 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/fat/fat.ko 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/fat/msdos.ko 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/fat/vfat.ko 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/fuse 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/fuse/cuse.ko 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/fuse/fuse.ko 2024-03-19T00:20:16.436 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/gfs2 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/gfs2/gfs2.ko 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/hfs 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/hfs/hfs.ko 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/hfsplus 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/hfsplus/hfsplus.ko 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/isofs 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/isofs/isofs.ko 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/jfs 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/jfs/jfs.ko 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/lockd 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/lockd/lockd.ko 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/minix 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/minix/minix.ko 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/netfs 2024-03-19T00:20:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/netfs/netfs.ko 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs/blocklayout 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs/filelayout 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs/flexfilelayout 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs/nfs.ko 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs/nfsv3.ko 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs/nfsv4.ko 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs_common 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs_common/grace.ko 2024-03-19T00:20:16.438 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfs_common/nfs_acl.ko 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfsd 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nfsd/nfsd.ko 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nilfs2 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nilfs2/nilfs2.ko 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/mac-celtic.ko 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/mac-centeuro.ko 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/mac-croatian.ko 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/mac-cyrillic.ko 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/mac-gaelic.ko 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/mac-greek.ko 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/mac-iceland.ko 2024-03-19T00:20:16.439 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/mac-inuit.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/mac-roman.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/mac-romanian.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/mac-turkish.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp1250.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp1251.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp1255.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp737.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp775.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp850.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp852.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp855.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp857.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp860.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp861.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp862.ko 2024-03-19T00:20:16.440 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp863.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp864.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp865.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp866.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp869.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp874.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp932.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp936.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp949.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_cp950.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_euc-jp.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_iso8859-1.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_iso8859-13.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_iso8859-14.ko 2024-03-19T00:20:16.441 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_iso8859-15.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_iso8859-2.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_iso8859-3.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_iso8859-4.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_iso8859-5.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_iso8859-6.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_iso8859-7.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_iso8859-9.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_koi8-r.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_koi8-ru.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_koi8-u.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/nls/nls_utf8.ko 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ocfs2 2024-03-19T00:20:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ocfs2/cluster 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ocfs2/dlm 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ocfs2/dlmfs 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ocfs2/ocfs2.ko 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/overlayfs 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/overlayfs/overlay.ko 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/pstore 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/pstore/ramoops.ko 2024-03-19T00:20:16.443 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/reiserfs 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/reiserfs/reiserfs.ko 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/romfs 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/romfs/romfs.ko 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/smb 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/smb/client 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/smb/client/cifs.ko 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/smb/common 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/smb/common/cifs_arc4.ko 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/smb/common/cifs_md4.ko 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/squashfs 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/squashfs/squashfs.ko 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/sysv 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/sysv/sysv.ko 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/udf 2024-03-19T00:20:16.444 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/udf/udf.ko 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ufs 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/ufs/ufs.ko 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/xfs 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/fs/xfs/xfs.ko 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/kernel 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/kernel/rcu 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/kernel/rcu/rcutorture.ko 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/kernel/torture.ko 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/kernel/trace 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/asn1_encoder.ko 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/crc-itu-t.ko 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/crc8.ko 2024-03-19T00:20:16.445 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/crypto 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/crypto/libarc4.ko 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/crypto/libdes.ko 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/libcrc32c.ko 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/lru_cache.ko 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/lz4 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/lz4/lz4_compress.ko 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/lz4/lz4hc_compress.ko 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/math 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/math/cordic.ko 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/raid6 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/raid6/raid6_pq.ko 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/reed_solomon 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-19T00:20:16.446 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/ts_bm.ko 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/ts_fsm.ko 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/ts_kmp.ko 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/zstd 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/lib/zstd/zstd_compress.ko 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/mm 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/mm/hwpoison-inject.ko 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/6lowpan 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/6lowpan/6lowpan.ko 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/6lowpan/nhc_dest.ko 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/6lowpan/nhc_fragment.ko 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/6lowpan/nhc_hop.ko 2024-03-19T00:20:16.447 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/6lowpan/nhc_mobility.ko 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/6lowpan/nhc_routing.ko 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/6lowpan/nhc_udp.ko 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/802 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/802/garp.ko 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/802/mrp.ko 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/802/p8022.ko 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/802/psnap.ko 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/802/stp.ko 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/8021q 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/8021q/8021q.ko 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/9p 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/9p/9pnet.ko 2024-03-19T00:20:16.448 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/9p/9pnet_fd.ko 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/9p/9pnet_rdma.ko 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/9p/9pnet_virtio.ko 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/batman-adv 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/batman-adv/batman-adv.ko 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/br_netfilter.ko 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/bridge.ko 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-19T00:20:16.449 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-19T00:20:16.450 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/ebtables.ko 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ceph 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ceph/libceph.ko 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/core 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/core/failover.ko 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/core/pktgen.ko 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dccp 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dccp/dccp.ko 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dccp/dccp_diag.ko 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dccp/dccp_ipv4.ko 2024-03-19T00:20:16.451 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dccp/dccp_ipv6.ko 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dns_resolver 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dns_resolver/dns_resolver.ko 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dsa 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dsa/dsa_core.ko 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dsa/tag_brcm.ko 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dsa/tag_dsa.ko 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dsa/tag_none.ko 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/dsa/tag_trailer.ko 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ieee802154 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ieee802154/6lowpan 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ieee802154/ieee802154.ko 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4 2024-03-19T00:20:16.452 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/ah4.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/esp4.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/fou.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/gre.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/inet_diag.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/ip_gre.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/ip_tunnel.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/ip_vti.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/ipcomp.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/ipip.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-19T00:20:16.453 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-19T00:20:16.454 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_bic.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_dctcp.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_diag.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_highspeed.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_htcp.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_hybla.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_illinois.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_lp.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_scalable.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_vegas.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_veno.ko 2024-03-19T00:20:16.455 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_westwood.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tcp_yeah.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/tunnel4.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/udp_diag.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/udp_tunnel.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/ah6.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/esp6.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/fou6.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/ip6_tunnel.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/ip6_vti.ko 2024-03-19T00:20:16.456 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/ipcomp6.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-19T00:20:16.457 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/sit.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/tunnel6.ko 2024-03-19T00:20:16.458 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/key 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/key/af_key.ko 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/l2tp 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/l2tp/l2tp_core.ko 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/l2tp/l2tp_eth.ko 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/l2tp/l2tp_ip.ko 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/l2tp/l2tp_ip6.ko 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/l2tp/l2tp_netlink.ko 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/l2tp/l2tp_ppp.ko 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/llc 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/llc/llc.ko 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/mac802154 2024-03-19T00:20:16.459 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/mac802154/mac802154.ko 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/mpls 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/mpls/mpls_gso.ko 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set.ko 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-19T00:20:16.460 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-19T00:20:16.461 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conncount.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-19T00:20:16.462 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_flow_table.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_log_syslog.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_nat.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-19T00:20:16.463 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_nat_irc.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_nat_sip.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nf_tables.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nfnetlink.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nfnetlink_log.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_chain_nat.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_compat.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_ct.ko 2024-03-19T00:20:16.464 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_fib.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_hash.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_limit.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_log.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_masq.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_nat.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_numgen.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_queue.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_quota.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_redir.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_reject.ko 2024-03-19T00:20:16.465 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_reject_inet.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_socket.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/nft_tunnel.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_AUDIT.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_CT.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_DSCP.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_HL.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_HMARK.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_LED.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_LOG.ko 2024-03-19T00:20:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_NETMAP.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_NFLOG.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_RATEEST.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_SECMARK.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_TEE.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_TPROXY.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_TRACE.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_addrtype.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_bpf.ko 2024-03-19T00:20:16.467 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_cgroup.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_cluster.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_comment.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_connbytes.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_connlabel.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_connlimit.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_connmark.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_conntrack.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_cpu.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_dccp.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_devgroup.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_dscp.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_ecn.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_esp.ko 2024-03-19T00:20:16.468 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_hashlimit.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_helper.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_hl.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_ipcomp.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_iprange.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_ipvs.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_l2tp.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_length.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_limit.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_mac.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_mark.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_multiport.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_nat.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_nfacct.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_osf.ko 2024-03-19T00:20:16.469 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_owner.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_physdev.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_pkttype.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_policy.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_quota.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_rateest.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_realm.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_recent.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_sctp.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_set.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_socket.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_state.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_statistic.ko 2024-03-19T00:20:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_string.ko 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_tcpmss.ko 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_time.ko 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netfilter/xt_u32.ko 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netlink 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/netlink/netlink_diag.ko 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/nsh 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/nsh/nsh.ko 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/openvswitch 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/openvswitch/openvswitch.ko 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/openvswitch/vport-geneve.ko 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/openvswitch/vport-gre.ko 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/openvswitch/vport-vxlan.ko 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/packet 2024-03-19T00:20:16.471 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/packet/af_packet_diag.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/rds 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/rds/rds.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/rds/rds_rdma.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/rds/rds_tcp.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/act_csum.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/act_gact.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/act_mirred.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/act_nat.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/act_pedit.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/act_police.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/act_simple.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/act_skbedit.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/act_vlan.ko 2024-03-19T00:20:16.472 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/cls_basic.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/cls_bpf.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/cls_flow.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/cls_fw.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/cls_route.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/cls_u32.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/em_cmp.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/em_ipset.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/em_meta.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/em_nbyte.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/em_text.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/em_u32.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_choke.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_codel.ko 2024-03-19T00:20:16.473 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_drr.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_fq.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_gred.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_hfsc.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_hhf.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_htb.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_ingress.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_mqprio.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_mqprio_lib.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_multiq.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_netem.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_pie.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_plug.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_prio.ko 2024-03-19T00:20:16.474 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_qfq.ko 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_red.ko 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_sfb.ko 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_sfq.ko 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_tbf.ko 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sched/sch_teql.ko 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sctp 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sctp/sctp.ko 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sctp/sctp_diag.ko 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sunrpc 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sunrpc/auth_gss 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sunrpc/sunrpc.ko 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sunrpc/xprtrdma 2024-03-19T00:20:16.475 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/tipc 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/tipc/diag.ko 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/tipc/tipc.ko 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/unix 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/unix/unix_diag.ko 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/vmw_vsock 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/vmw_vsock/vsock.ko 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/xfrm 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/security 2024-03-19T00:20:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/security/keys 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/security/keys/encrypted-keys 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/security/keys/trusted-keys 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/security/keys/trusted-keys/trusted.ko 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/virt 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/virt/lib 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/kernel/virt/lib/irqbypass.ko 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.alias 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.alias.bin 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.builtin 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.builtin.alias.bin 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.builtin.bin 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.builtin.modinfo 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.dep 2024-03-19T00:20:16.477 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.dep.bin 2024-03-19T00:20:16.478 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.devname 2024-03-19T00:20:16.478 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.order 2024-03-19T00:20:16.478 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.softdep 2024-03-19T00:20:16.478 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.symbols 2024-03-19T00:20:16.478 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/modules.symbols.bin 2024-03-19T00:20:16.478 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.8.0-g73069ad93aff/vmlinuz 2024-03-19T00:20:16.482 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g73069ad93aff 2024-03-19T00:20:16.482 DEBUG:teuthology.orchestra.run.smithi069:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-19T00:20:16.505 INFO:teuthology.orchestra.run.smithi069.stdout:postinstall scriptlet (using /bin/sh): 2024-03-19T00:20:16.505 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-19T00:20:16.505 INFO:teuthology.orchestra.run.smithi069.stdout: /usr/bin/kernel-install add 6.8.0-g73069ad93aff /lib/modules/6.8.0-g73069ad93aff/vmlinuz 2024-03-19T00:20:16.505 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2024-03-19T00:20:16.505 INFO:teuthology.orchestra.run.smithi069.stdout:for file in vmlinuz System.map config; do 2024-03-19T00:20:16.505 INFO:teuthology.orchestra.run.smithi069.stdout: if ! cmp --silent "/lib/modules/6.8.0-g73069ad93aff/${file}" "/boot/${file}-6.8.0-g73069ad93aff"; then 2024-03-19T00:20:16.505 INFO:teuthology.orchestra.run.smithi069.stdout: cp "/lib/modules/6.8.0-g73069ad93aff/${file}" "/boot/${file}-6.8.0-g73069ad93aff" 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout: fi 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout:done 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout:new-kernel-pkg --remove 6.8.0-g73069ad93aff --rminitrd --initrdfile=/boot/initramfs-6.8.0-g73069ad93aff.img 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-install remove 6.8.0-g73069ad93aff 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/update-bootloader --remove 6.8.0-g73069ad93aff 2024-03-19T00:20:16.506 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2024-03-19T00:20:16.507 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -qi grub2-tools 2024-03-19T00:20:16.594 INFO:teuthology.orchestra.run.smithi069.stdout:Name : grub2-tools 2024-03-19T00:20:16.594 INFO:teuthology.orchestra.run.smithi069.stdout:Epoch : 1 2024-03-19T00:20:16.594 INFO:teuthology.orchestra.run.smithi069.stdout:Version : 2.06 2024-03-19T00:20:16.594 INFO:teuthology.orchestra.run.smithi069.stdout:Release : 68.el9 2024-03-19T00:20:16.594 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: x86_64 2024-03-19T00:20:16.594 INFO:teuthology.orchestra.run.smithi069.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-19T00:20:16.594 INFO:teuthology.orchestra.run.smithi069.stdout:Group : Unspecified 2024-03-19T00:20:16.594 INFO:teuthology.orchestra.run.smithi069.stdout:Size : 8264254 2024-03-19T00:20:16.594 INFO:teuthology.orchestra.run.smithi069.stdout:License : GPLv3+ 2024-03-19T00:20:16.594 INFO:teuthology.orchestra.run.smithi069.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-19T00:20:16.595 INFO:teuthology.orchestra.run.smithi069.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-19T00:20:16.595 INFO:teuthology.orchestra.run.smithi069.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-19T00:20:16.595 INFO:teuthology.orchestra.run.smithi069.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-19T00:20:16.595 INFO:teuthology.orchestra.run.smithi069.stdout:Packager : builder@centos.org 2024-03-19T00:20:16.595 INFO:teuthology.orchestra.run.smithi069.stdout:Vendor : CentOS 2024-03-19T00:20:16.596 INFO:teuthology.orchestra.run.smithi069.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-19T00:20:16.596 INFO:teuthology.orchestra.run.smithi069.stdout:Summary : Support tools for GRUB. 2024-03-19T00:20:16.596 INFO:teuthology.orchestra.run.smithi069.stdout:Description : 2024-03-19T00:20:16.596 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:20:16.596 INFO:teuthology.orchestra.run.smithi069.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-19T00:20:16.596 INFO:teuthology.orchestra.run.smithi069.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-19T00:20:16.596 INFO:teuthology.orchestra.run.smithi069.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-19T00:20:16.596 INFO:teuthology.orchestra.run.smithi069.stdout:hardware devices. 2024-03-19T00:20:16.596 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:20:16.596 INFO:teuthology.orchestra.run.smithi069.stdout:This subpackage provides tools for support of all platforms. 2024-03-19T00:20:16.597 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-19T00:20:16.597 INFO:teuthology.task.kernel:Updating grub on smithi069 to boot 6.8.0-g73069ad93aff 2024-03-19T00:20:16.597 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-19T00:20:17.163 INFO:teuthology.orchestra.run.smithi069.stderr:Generating grub configuration file ... 2024-03-19T00:20:18.653 INFO:teuthology.orchestra.run.smithi069.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-19T00:20:18.687 INFO:teuthology.orchestra.run.smithi069.stderr:done 2024-03-19T00:20:18.690 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2024-03-19T00:20:18.705 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.btXrIZRcWm 2024-03-19T00:20:18.705 DEBUG:teuthology.orchestra.run.smithi069:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.btXrIZRcWm 2024-03-19T00:20:18.779 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.btXrIZRcWm 2024-03-19T00:20:18.940 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.btXrIZRcWm is 6KB 2024-03-19T00:20:18.988 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.btXrIZRcWm 2024-03-19T00:20:19.002 DEBUG:teuthology.orchestra.run.smithi069:> sudo /bin/ls /boot/loader/entries || true 2024-03-19T00:20:19.067 INFO:teuthology.orchestra.run.smithi069.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-19T00:20:19.068 INFO:teuthology.orchestra.run.smithi069.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-19T00:20:19.068 INFO:teuthology.orchestra.run.smithi069.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-19T00:20:19.068 INFO:teuthology.orchestra.run.smithi069.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g73069ad93aff.conf 2024-03-19T00:20:19.069 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g73069ad93aff 2024-03-19T00:20:19.225 DEBUG:teuthology.orchestra.run.smithi069:> sudo shutdown -r now 2024-03-19T00:20:49.229 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-19T00:20:49.230 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-19T00:20:49.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:21:07.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-03-19T00:21:16.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-19T00:21:16.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:21:19.806 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-03-19T00:21:31.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-19T00:21:31.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:21:50.239 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-03-19T00:22:05.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-19T00:22:05.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:22:06.895 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-03-19T00:22:07.513 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-19T00:22:07.513 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g73069ad93aff"... 2024-03-19T00:22:07.513 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-03-19T00:22:07.552 INFO:teuthology.orchestra.run.smithi069.stdout:6.8.0-g73069ad93aff 2024-03-19T00:22:07.552 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g73069ad93aff vs 6.8.0-g73069ad93aff 2024-03-19T00:22:07.552 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-19T00:22:07.553 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-19T00:22:08.554 DEBUG:teuthology.parallel:result is None 2024-03-19T00:22:08.554 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-19T00:22:08.562 INFO:teuthology.task.internal:Creating test directory... 2024-03-19T00:22:08.563 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-19T00:22:08.570 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-19T00:22:08.574 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-19T00:22:08.617 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-19T00:22:08.624 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-19T00:22:08.635 INFO:teuthology.task.internal:Creating archive directory... 2024-03-19T00:22:08.635 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-19T00:22:08.640 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-19T00:22:08.643 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-19T00:22:08.751 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-19T00:22:08.758 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-19T00:22:08.758 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-19T00:22:08.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:22:08.775 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-19T00:22:08.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:22:08.792 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-19T00:22:08.798 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-19T00:22:08.818 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-19T00:22:08.856 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T00:22:08.872 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T00:22:08.874 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T00:22:08.896 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T00:22:08.951 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T00:22:09.032 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T00:22:09.036 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-19T00:22:09.044 INFO:teuthology.task.internal:Configuring sudo... 2024-03-19T00:22:09.044 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-19T00:22:09.048 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-19T00:22:09.051 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-19T00:22:09.115 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-19T00:22:09.124 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-19T00:22:09.124 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-19T00:22:09.128 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-19T00:22:09.130 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-19T00:22:09.167 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T00:22:09.282 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T00:22:09.354 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T00:22:09.423 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T00:22:09.501 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T00:22:09.502 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-19T00:22:09.620 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T00:22:09.663 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T00:22:09.741 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T00:22:09.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T00:22:09.831 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-19T00:22:09.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-19T00:22:09.900 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T00:22:09.944 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T00:22:10.002 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T00:22:10.088 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T00:22:10.131 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-19T00:22:10.131 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-19T00:22:10.201 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2024-03-19T00:22:10.208 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-03-19T00:22:10.210 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2024-03-19T00:22:10.249 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T00:22:10.279 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T00:22:10.316 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T00:22:10.586 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-19T00:22:10.595 INFO:teuthology.task.internal:Starting timer... 2024-03-19T00:22:10.596 INFO:teuthology.run_tasks:Running task pcp... 2024-03-19T00:22:10.616 INFO:teuthology.run_tasks:Running task selinux... 2024-03-19T00:22:10.632 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-19T00:22:10.632 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2024-03-19T00:22:10.765 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2024-03-19T00:22:10.770 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-03-19T00:22:10.845 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-03-19T00:22:10.846 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2024-03-19T00:22:10.905 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: 2024-03-19T00:22:10.907 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-19T00:22:10.907 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2024-03-19T00:22:10.958 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2024-03-19T00:22:10.958 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-03-19T00:22:10.993 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-03-19T00:22:10.994 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2024-03-19T00:22:11.022 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2024-03-19T00:22:11.023 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi069.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi163.front.sepia.ceph.com': 'permissive'} 2024-03-19T00:22:11.023 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T00:22:11.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:22:11.105 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T00:22:11.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:22:11.134 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T00:22:11.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:22:11.162 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-19T00:22:11.162 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2024-03-19T00:22:11.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-03-19T00:22:11.287 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2024-03-19T00:22:11.324 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-19T00:22:11.333 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-19T00:22:11.340 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-19T00:22:11.491 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-19T00:22:11.528 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-19T00:22:11.530 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi069.front.sepia.ceph.com,smithi086.front.sepia.ceph.com,smithi163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-19T00:39:52.117 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com')] 2024-03-19T00:39:52.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-19T00:39:52.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:39:52.263 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-03-19T00:39:52.417 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-19T00:39:52.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-19T00:39:52.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:39:52.506 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-03-19T00:39:52.577 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-19T00:39:52.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-19T00:39:52.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:39:52.663 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-03-19T00:39:52.734 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-19T00:39:52.734 INFO:teuthology.run_tasks:Running task clock... 2024-03-19T00:39:52.747 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-19T00:39:52.747 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-19T00:39:52.747 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T00:39:52.755 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-19T00:39:52.755 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T00:39:52.757 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-19T00:39:52.757 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T00:39:52.787 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-19T00:39:52.805 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-19T00:39:52.809 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-19T00:39:52.827 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-19T00:39:52.834 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-03-19T00:39:52.846 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-19T00:39:52.848 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-03-19T00:39:52.857 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2024-03-19T00:39:52.865 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-19T00:39:52.872 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2024-03-19T00:39:52.881 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-19T00:39:52.888 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-19T00:39:52.899 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-19T00:39:52.905 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-19T00:39:52.925 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-03-19T00:39:52.928 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T00:39:52.928 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-03-19T00:39:52.928 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:52.928 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:52.929 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:52.929 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:52.949 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-03-19T00:39:52.952 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T00:39:52.952 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-03-19T00:39:52.952 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:52.952 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:52.952 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:52.952 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:52.984 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2024-03-19T00:39:53.024 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2024-03-19T00:39:53.076 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-19T00:39:53.131 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-19T00:39:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-03-19T00:39:53.272 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T00:39:53.272 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-03-19T00:39:53.272 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:53.273 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:53.273 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:53.273 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T00:39:53.274 INFO:teuthology.run_tasks:Running task install... 2024-03-19T00:39:53.286 DEBUG:teuthology.task.install:project ceph 2024-03-19T00:39:53.286 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810'}} 2024-03-19T00:39:53.286 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810'} 2024-03-19T00:39:53.287 INFO:teuthology.task.install:Using flavor: default 2024-03-19T00:39:53.292 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-19T00:39:53.293 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-19T00:39:53.293 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-19T00:39:53.294 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:39:53.296 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-19T00:39:53.296 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:39:53.297 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-19T00:39:53.297 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:39:53.439 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-19T00:39:53.439 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-19T00:39:53.512 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-19T00:39:53.512 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-19T00:39:53.516 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-19T00:39:53.516 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-19T00:39:53.612 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-19T00:39:53.612 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-19T00:39:53.612 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-19T00:39:53.625 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-19T00:39:53.625 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-19T00:39:53.625 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-19T00:39:53.640 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-19T00:39:53.641 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-19T00:39:53.655 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-19T00:39:53.655 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-19T00:39:53.680 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-19T00:39:53.680 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T00:39:53.680 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-19T00:39:53.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-19T00:39:53.725 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-19T00:39:53.777 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-19T00:39:53.777 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-19T00:39:53.789 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-19T00:39:53.807 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-19T00:39:53.854 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-03-19T00:39:53.856 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-03-19T00:39:53.875 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2024-03-19T00:39:53.878 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2024-03-19T00:39:53.916 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-19T00:39:54.047 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-19T00:39:54.090 INFO:teuthology.orchestra.run.smithi086.stdout:74 files removed 2024-03-19T00:39:54.115 INFO:teuthology.orchestra.run.smithi163.stdout:74 files removed 2024-03-19T00:39:54.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-19T00:39:54.135 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2024-03-19T00:39:54.139 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2024-03-19T00:39:54.142 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-19T00:39:54.515 INFO:teuthology.orchestra.run.smithi069.stdout:74 files removed 2024-03-19T00:39:54.558 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-19T00:39:54.822 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-03-19T00:39:54.823 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 215 kB/s | 83 kB 00:00 2024-03-19T00:39:55.124 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 39 kB/s | 10 kB 00:00 2024-03-19T00:39:55.239 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-19T00:39:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 157 kB/s | 83 kB 00:00 2024-03-19T00:39:55.591 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-03-19T00:39:55.640 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-03-19T00:39:56.043 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-19T00:39:56.167 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-19T00:39:56.214 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 19 kB/s | 10 kB 00:00 2024-03-19T00:39:56.731 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-03-19T00:39:57.351 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-19T00:39:57.502 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.0 MB 00:00 2024-03-19T00:39:57.559 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-19T00:39:59.035 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS-9 - BaseOS 5.3 MB/s | 2.0 MB 00:00 2024-03-19T00:39:59.508 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-03-19T00:40:01.122 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-03-19T00:40:02.142 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.0 MB 00:04 2024-03-19T00:40:03.467 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 19 MB 00:02 2024-03-19T00:51:01.169 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 9.6 kB/s | 6.1 MB 10:54 2024-03-19T00:53:09.163 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 25 kB/s | 19 MB 13:05 2024-03-19T00:53:14.162 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.1 MB 00:01 2024-03-19T00:53:15.796 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-03-19T00:53:17.665 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-19T00:53:22.598 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-19T00:53:24.928 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:53:24.929 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:53:24.943 INFO:teuthology.orchestra.run.smithi163.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-19T00:53:24.945 INFO:teuthology.orchestra.run.smithi163.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-19T00:53:24.945 INFO:teuthology.orchestra.run.smithi163.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-19T00:53:24.947 INFO:teuthology.orchestra.run.smithi163.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-19T00:53:24.949 INFO:teuthology.orchestra.run.smithi163.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-19T00:53:24.952 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-19T00:53:24.953 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-19T00:53:25.033 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================================ 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================================ 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-19T00:53:25.051 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-19T00:53:25.052 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-19T00:53:25.053 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-19T00:53:25.054 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-19T00:53:25.055 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-19T00:53:25.056 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-19T00:53:25.056 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-19T00:53:25.056 INFO:teuthology.orchestra.run.smithi163.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-19T00:53:25.056 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-19T00:53:25.056 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-03-19T00:53:25.056 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-19T00:53:25.056 INFO:teuthology.orchestra.run.smithi163.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-19T00:53:25.056 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-19T00:53:25.056 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-19T00:53:25.056 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-19T00:53:25.057 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-19T00:53:25.058 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-19T00:53:25.059 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-19T00:53:25.060 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-19T00:53:25.061 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-19T00:53:25.062 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-19T00:53:25.063 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-19T00:53:25.064 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-19T00:53:25.065 INFO:teuthology.orchestra.run.smithi163.stdout:Installing weak dependencies: 2024-03-19T00:53:25.065 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-19T00:53:25.065 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:53:25.065 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-03-19T00:53:25.065 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================================ 2024-03-19T00:53:25.065 INFO:teuthology.orchestra.run.smithi163.stdout:Install 150 Packages 2024-03-19T00:53:25.065 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 24 Packages 2024-03-19T00:53:25.065 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:53:25.066 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 224 M 2024-03-19T00:53:25.066 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-03-19T00:53:25.683 INFO:teuthology.orchestra.run.smithi163.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 44 kB/s | 6.6 kB 00:00 2024-03-19T00:53:25.858 INFO:teuthology.orchestra.run.smithi163.stdout:(2/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 5.0 MB/s | 891 kB 00:00 2024-03-19T00:53:25.908 INFO:teuthology.orchestra.run.smithi163.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 2.8 MB/s | 145 kB 00:00 2024-03-19T00:53:26.184 INFO:teuthology.orchestra.run.smithi163.stdout:(4/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-03-19T00:53:26.359 INFO:teuthology.orchestra.run.smithi163.stdout:(5/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 8.5 MB/s | 1.5 MB 00:00 2024-03-19T00:53:26.434 INFO:teuthology.orchestra.run.smithi163.stdout:(6/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 5.9 MB/s | 5.4 MB 00:00 2024-03-19T00:53:26.818 INFO:teuthology.orchestra.run.smithi163.stdout:(7/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 10 MB/s | 4.7 MB 00:00 2024-03-19T00:53:27.760 INFO:teuthology.orchestra.run.smithi163.stdout:(8/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el9 10 MB/s | 9.8 MB 00:00 2024-03-19T00:53:27.843 INFO:teuthology.orchestra.run.smithi163.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 302 kB/s | 25 kB 00:00 2024-03-19T00:53:28.185 INFO:teuthology.orchestra.run.smithi163.stdout:(10/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x8 9.6 MB/s | 17 MB 00:01 2024-03-19T00:53:28.286 INFO:teuthology.orchestra.run.smithi163.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 2.2 MB/s | 221 kB 00:00 2024-03-19T00:53:28.378 INFO:teuthology.orchestra.run.smithi163.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 295 kB/s | 27 kB 00:00 2024-03-19T00:53:28.470 INFO:teuthology.orchestra.run.smithi163.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 345 kB/s | 32 kB 00:00 2024-03-19T00:53:28.703 INFO:teuthology.orchestra.run.smithi163.stdout:(14/174): ceph-common-19.0.0-2206.gb6ae2491.el9 6.5 MB/s | 21 MB 00:03 2024-03-19T00:53:28.737 INFO:teuthology.orchestra.run.smithi163.stdout:(15/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 2.7 MB/s | 746 kB 00:00 2024-03-19T00:53:28.787 INFO:teuthology.orchestra.run.smithi163.stdout:(16/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 1.9 MB/s | 163 kB 00:00 2024-03-19T00:53:28.838 INFO:teuthology.orchestra.run.smithi163.stdout:(17/174): librados-devel-19.0.0-2206.gb6ae2491. 1.2 MB/s | 127 kB 00:00 2024-03-19T00:53:28.963 INFO:teuthology.orchestra.run.smithi163.stdout:(18/174): libradosstriper1-19.0.0-2206.gb6ae249 2.8 MB/s | 501 kB 00:00 2024-03-19T00:53:29.055 INFO:teuthology.orchestra.run.smithi163.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 491 kB/s | 45 kB 00:00 2024-03-19T00:53:29.163 INFO:teuthology.orchestra.run.smithi163.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 1.2 MB/s | 131 kB 00:00 2024-03-19T00:53:29.280 INFO:teuthology.orchestra.run.smithi163.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 1.4 MB/s | 162 kB 00:00 2024-03-19T00:53:29.423 INFO:teuthology.orchestra.run.smithi163.stdout:(22/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 8.8 MB/s | 5.1 MB 00:00 2024-03-19T00:53:29.448 INFO:teuthology.orchestra.run.smithi163.stdout:(23/174): python3-rados-19.0.0-2206.gb6ae2491.e 1.9 MB/s | 321 kB 00:00 2024-03-19T00:53:29.515 INFO:teuthology.orchestra.run.smithi163.stdout:(24/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 3.2 MB/s | 298 kB 00:00 2024-03-19T00:53:29.540 INFO:teuthology.orchestra.run.smithi163.stdout:(25/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.1 MB/s | 99 kB 00:00 2024-03-19T00:53:29.615 INFO:teuthology.orchestra.run.smithi163.stdout:(26/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 852 kB/s | 85 kB 00:00 2024-03-19T00:53:29.707 INFO:teuthology.orchestra.run.smithi163.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-19T00:53:29.799 INFO:teuthology.orchestra.run.smithi163.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 370 kB/s | 34 kB 00:00 2024-03-19T00:53:29.891 INFO:teuthology.orchestra.run.smithi163.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 1.5 MB/s | 139 kB 00:00 2024-03-19T00:53:29.958 INFO:teuthology.orchestra.run.smithi163.stdout:(30/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 7.4 MB/s | 3.1 MB 00:00 2024-03-19T00:53:30.467 INFO:teuthology.orchestra.run.smithi163.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 3.1 MB/s | 1.8 MB 00:00 2024-03-19T00:53:30.942 INFO:teuthology.orchestra.run.smithi163.stdout:(32/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 15 MB/s | 48 MB 00:03 2024-03-19T00:53:30.967 INFO:teuthology.orchestra.run.smithi163.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2206.gb6 497 kB/s | 248 kB 00:00 2024-03-19T00:53:30.984 INFO:teuthology.orchestra.run.smithi163.stdout:(34/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 1.2 MB/s | 49 kB 00:00 2024-03-19T00:53:31.093 INFO:teuthology.orchestra.run.smithi163.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-03-19T00:53:31.110 INFO:teuthology.orchestra.run.smithi163.stdout:(36/174): ceph-prometheus-alerts-19.0.0-2206.gb 115 kB/s | 16 kB 00:00 2024-03-19T00:53:31.135 INFO:teuthology.orchestra.run.smithi163.stdout:(37/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 1.7 MB/s | 264 kB 00:00 2024-03-19T00:53:31.194 INFO:teuthology.orchestra.run.smithi163.stdout:(38/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 219 kB/s | 18 kB 00:00 2024-03-19T00:53:31.236 INFO:teuthology.orchestra.run.smithi163.stdout:(39/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 5.1 MB/s | 749 kB 00:00 2024-03-19T00:53:31.345 INFO:teuthology.orchestra.run.smithi163.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2024-03-19T00:53:31.370 INFO:teuthology.orchestra.run.smithi163.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 793 kB/s | 106 kB 00:00 2024-03-19T00:53:31.412 INFO:teuthology.orchestra.run.smithi163.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 920 kB 00:00 2024-03-19T00:53:31.437 INFO:teuthology.orchestra.run.smithi163.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-19T00:53:31.462 INFO:teuthology.orchestra.run.smithi163.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 346 kB/s | 32 kB 00:00 2024-03-19T00:53:31.487 INFO:teuthology.orchestra.run.smithi163.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 343 kB/s | 17 kB 00:00 2024-03-19T00:53:31.571 INFO:teuthology.orchestra.run.smithi163.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-03-19T00:53:31.596 INFO:teuthology.orchestra.run.smithi163.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 75 kB/s | 10 kB 00:00 2024-03-19T00:53:31.622 INFO:teuthology.orchestra.run.smithi163.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2024-03-19T00:53:31.647 INFO:teuthology.orchestra.run.smithi163.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 888 kB/s | 44 kB 00:00 2024-03-19T00:53:31.672 INFO:teuthology.orchestra.run.smithi163.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-19T00:53:31.697 INFO:teuthology.orchestra.run.smithi163.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 578 kB 00:00 2024-03-19T00:53:31.723 INFO:teuthology.orchestra.run.smithi163.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 46 kB 00:00 2024-03-19T00:53:31.748 INFO:teuthology.orchestra.run.smithi163.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-19T00:53:31.773 INFO:teuthology.orchestra.run.smithi163.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-19T00:53:31.798 INFO:teuthology.orchestra.run.smithi163.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-19T00:53:31.824 INFO:teuthology.orchestra.run.smithi163.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 456 kB/s | 34 kB 00:00 2024-03-19T00:53:31.916 INFO:teuthology.orchestra.run.smithi163.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-03-19T00:53:31.949 INFO:teuthology.orchestra.run.smithi163.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 6.5 MB/s | 811 kB 00:00 2024-03-19T00:53:31.983 INFO:teuthology.orchestra.run.smithi163.stdout:(59/174): postgresql-13.11-1.el9.x86_64.rpm 9.7 MB/s | 1.5 MB 00:00 2024-03-19T00:53:32.008 INFO:teuthology.orchestra.run.smithi163.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.4 MB/s | 134 kB 00:00 2024-03-19T00:53:32.108 INFO:teuthology.orchestra.run.smithi163.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-03-19T00:53:32.133 INFO:teuthology.orchestra.run.smithi163.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 633 kB/s | 78 kB 00:00 2024-03-19T00:53:32.167 INFO:teuthology.orchestra.run.smithi163.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-03-19T00:53:32.268 INFO:teuthology.orchestra.run.smithi163.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 5.8 MB 00:00 2024-03-19T00:53:32.284 INFO:teuthology.orchestra.run.smithi163.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 166 kB/s | 24 kB 00:00 2024-03-19T00:53:32.301 INFO:teuthology.orchestra.run.smithi163.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 138 kB/s | 18 kB 00:00 2024-03-19T00:53:32.326 INFO:teuthology.orchestra.run.smithi163.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-19T00:53:32.352 INFO:teuthology.orchestra.run.smithi163.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 631 kB/s | 32 kB 00:00 2024-03-19T00:53:32.435 INFO:teuthology.orchestra.run.smithi163.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-19T00:53:32.469 INFO:teuthology.orchestra.run.smithi163.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-19T00:53:32.495 INFO:teuthology.orchestra.run.smithi163.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-19T00:53:32.520 INFO:teuthology.orchestra.run.smithi163.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 825 kB/s | 69 kB 00:00 2024-03-19T00:53:32.545 INFO:teuthology.orchestra.run.smithi163.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.7 MB/s | 238 kB 00:00 2024-03-19T00:53:32.562 INFO:teuthology.orchestra.run.smithi163.stdout:(74/174): python3-prettytable-0.7.2-27.el9.noar 616 kB/s | 41 kB 00:00 2024-03-19T00:53:32.587 INFO:teuthology.orchestra.run.smithi163.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-19T00:53:32.612 INFO:teuthology.orchestra.run.smithi163.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-19T00:53:32.629 INFO:teuthology.orchestra.run.smithi163.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-03-19T00:53:32.646 INFO:teuthology.orchestra.run.smithi163.stdout:(78/174): python3-pytz-2021.1-5.el9.noarch.rpm 812 kB/s | 47 kB 00:00 2024-03-19T00:53:32.813 INFO:teuthology.orchestra.run.smithi163.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 210 kB/s | 39 kB 00:00 2024-03-19T00:53:33.064 INFO:teuthology.orchestra.run.smithi163.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-19T00:53:33.080 INFO:teuthology.orchestra.run.smithi163.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 95 kB/s | 41 kB 00:00 2024-03-19T00:53:33.106 INFO:teuthology.orchestra.run.smithi163.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2024-03-19T00:53:33.122 INFO:teuthology.orchestra.run.smithi163.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 972 kB/s | 56 kB 00:00 2024-03-19T00:53:33.140 INFO:teuthology.orchestra.run.smithi163.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-19T00:53:33.156 INFO:teuthology.orchestra.run.smithi163.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 876 kB/s | 44 kB 00:00 2024-03-19T00:53:33.182 INFO:teuthology.orchestra.run.smithi163.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-19T00:53:33.207 INFO:teuthology.orchestra.run.smithi163.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-19T00:53:33.249 INFO:teuthology.orchestra.run.smithi163.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-19T00:53:33.274 INFO:teuthology.orchestra.run.smithi163.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-19T00:53:33.299 INFO:teuthology.orchestra.run.smithi163.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-19T00:53:33.325 INFO:teuthology.orchestra.run.smithi163.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-19T00:53:33.808 INFO:teuthology.orchestra.run.smithi163.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 351 kB/s | 187 kB 00:00 2024-03-19T00:53:33.834 INFO:teuthology.orchestra.run.smithi163.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 384 kB/s | 195 kB 00:00 2024-03-19T00:53:33.917 INFO:teuthology.orchestra.run.smithi163.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-19T00:53:33.968 INFO:teuthology.orchestra.run.smithi163.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-03-19T00:53:34.032 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 24 kB/s | 19 MB 13:31 2024-03-19T00:53:38.908 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.1 MB 00:01 2024-03-19T00:53:41.544 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-03-19T00:53:42.577 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-03-19T00:53:48.340 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-03-19T00:53:50.957 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:53:50.959 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:53:50.983 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-19T00:53:50.984 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-19T00:53:50.985 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-19T00:53:50.987 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-19T00:53:50.990 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-19T00:53:50.994 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-19T00:53:50.995 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-19T00:53:51.084 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-19T00:53:51.129 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================================ 2024-03-19T00:53:51.129 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-03-19T00:53:51.129 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================================ 2024-03-19T00:53:51.129 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-19T00:53:51.129 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-19T00:53:51.129 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-19T00:53:51.129 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-19T00:53:51.129 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-19T00:53:51.130 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-19T00:53:51.131 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-19T00:53:51.132 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-19T00:53:51.133 INFO:teuthology.orchestra.run.smithi069.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-19T00:53:51.134 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-19T00:53:51.135 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-19T00:53:51.136 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-19T00:53:51.137 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-19T00:53:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-19T00:53:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-19T00:53:51.140 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout:Installing weak dependencies: 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-19T00:53:51.141 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:53:51.142 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-19T00:53:51.142 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================================ 2024-03-19T00:53:51.142 INFO:teuthology.orchestra.run.smithi069.stdout:Install 150 Packages 2024-03-19T00:53:51.142 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 24 Packages 2024-03-19T00:53:51.142 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:53:51.142 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 224 M 2024-03-19T00:53:51.143 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-19T00:53:52.353 INFO:teuthology.orchestra.run.smithi069.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-03-19T00:53:52.878 INFO:teuthology.orchestra.run.smithi069.stdout:(2/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 6.2 MB/s | 5.4 MB 00:00 2024-03-19T00:53:52.928 INFO:teuthology.orchestra.run.smithi069.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 2.9 MB/s | 145 kB 00:00 2024-03-19T00:53:53.228 INFO:teuthology.orchestra.run.smithi069.stdout:(4/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 7.9 MB/s | 2.4 MB 00:00 2024-03-19T00:53:53.278 INFO:teuthology.orchestra.run.smithi069.stdout:(5/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 963 kB/s | 891 kB 00:00 2024-03-19T00:53:53.470 INFO:teuthology.orchestra.run.smithi069.stdout:(6/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 6.2 MB/s | 1.5 MB 00:00 2024-03-19T00:53:53.746 INFO:teuthology.orchestra.run.smithi069.stdout:(7/174): ceph-common-19.0.0-2206.gb6ae2491.el9. 12 MB/s | 21 MB 00:01 2024-03-19T00:53:54.437 INFO:teuthology.orchestra.run.smithi069.stdout:(8/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el9 14 MB/s | 9.8 MB 00:00 2024-03-19T00:53:54.488 INFO:teuthology.orchestra.run.smithi069.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 515 kB/s | 25 kB 00:00 2024-03-19T00:53:55.338 INFO:teuthology.orchestra.run.smithi069.stdout:(10/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x8 9.0 MB/s | 17 MB 00:01 2024-03-19T00:53:55.396 INFO:teuthology.orchestra.run.smithi069.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 3.8 MB/s | 221 kB 00:00 2024-03-19T00:53:55.438 INFO:teuthology.orchestra.run.smithi069.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 661 kB/s | 27 kB 00:00 2024-03-19T00:53:55.497 INFO:teuthology.orchestra.run.smithi069.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 552 kB/s | 32 kB 00:00 2024-03-19T00:53:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:(14/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 5.2 MB/s | 746 kB 00:00 2024-03-19T00:53:55.706 INFO:teuthology.orchestra.run.smithi069.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 2.4 MB/s | 163 kB 00:00 2024-03-19T00:53:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:(16/174): librados-devel-19.0.0-2206.gb6ae2491. 1.7 MB/s | 127 kB 00:00 2024-03-19T00:53:55.881 INFO:teuthology.orchestra.run.smithi069.stdout:(17/174): libradosstriper1-19.0.0-2206.gb6ae249 4.9 MB/s | 501 kB 00:00 2024-03-19T00:53:56.065 INFO:teuthology.orchestra.run.smithi069.stdout:(18/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x8 1.7 MB/s | 4.7 MB 00:02 2024-03-19T00:53:56.173 INFO:teuthology.orchestra.run.smithi069.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 419 kB/s | 45 kB 00:00 2024-03-19T00:53:56.365 INFO:teuthology.orchestra.run.smithi069.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 688 kB/s | 131 kB 00:00 2024-03-19T00:53:56.516 INFO:teuthology.orchestra.run.smithi069.stdout:(21/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 8.1 MB/s | 5.1 MB 00:00 2024-03-19T00:53:56.549 INFO:teuthology.orchestra.run.smithi069.stdout:(22/174): python3-cephfs-19.0.0-2206.gb6ae2491. 886 kB/s | 162 kB 00:00 2024-03-19T00:53:56.600 INFO:teuthology.orchestra.run.smithi069.stdout:(23/174): python3-rados-19.0.0-2206.gb6ae2491.e 3.8 MB/s | 321 kB 00:00 2024-03-19T00:53:56.666 INFO:teuthology.orchestra.run.smithi069.stdout:(24/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.5 MB/s | 99 kB 00:00 2024-03-19T00:53:56.717 INFO:teuthology.orchestra.run.smithi069.stdout:(25/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-19T00:53:56.742 INFO:teuthology.orchestra.run.smithi069.stdout:(26/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 1.5 MB/s | 298 kB 00:00 2024-03-19T00:53:57.359 INFO:teuthology.orchestra.run.smithi069.stdout:(27/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 17 MB/s | 48 MB 00:02 2024-03-19T00:53:57.392 INFO:teuthology.orchestra.run.smithi069.stdout:(28/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 262 kB/s | 170 kB 00:00 2024-03-19T00:53:57.418 INFO:teuthology.orchestra.run.smithi069.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2206.g 591 kB/s | 34 kB 00:00 2024-03-19T00:53:57.527 INFO:teuthology.orchestra.run.smithi069.stdout:(30/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 3.8 MB/s | 3.1 MB 00:00 2024-03-19T00:53:57.585 INFO:teuthology.orchestra.run.smithi069.stdout:(31/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 725 kB/s | 139 kB 00:00 2024-03-19T00:53:57.644 INFO:teuthology.orchestra.run.smithi069.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 8.0 MB/s | 1.8 MB 00:00 2024-03-19T00:53:57.702 INFO:teuthology.orchestra.run.smithi069.stdout:(33/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 860 kB/s | 49 kB 00:00 2024-03-19T00:53:57.769 INFO:teuthology.orchestra.run.smithi069.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2206.gb 250 kB/s | 16 kB 00:00 2024-03-19T00:53:57.794 INFO:teuthology.orchestra.run.smithi069.stdout:(35/174): ceph-mgr-modules-core-19.0.0-2206.gb6 1.2 MB/s | 248 kB 00:00 2024-03-19T00:53:57.886 INFO:teuthology.orchestra.run.smithi069.stdout:(36/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 8.1 MB/s | 749 kB 00:00 2024-03-19T00:53:57.953 INFO:teuthology.orchestra.run.smithi069.stdout:(37/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 274 kB/s | 18 kB 00:00 2024-03-19T00:53:58.062 INFO:teuthology.orchestra.run.smithi069.stdout:(38/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 905 kB/s | 264 kB 00:00 2024-03-19T00:53:58.195 INFO:teuthology.orchestra.run.smithi069.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-03-19T00:53:58.445 INFO:teuthology.orchestra.run.smithi069.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 758 kB/s | 106 kB 00:00 2024-03-19T00:53:58.496 INFO:teuthology.orchestra.run.smithi069.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 920 kB 00:00 2024-03-19T00:53:58.538 INFO:teuthology.orchestra.run.smithi069.stdout:(42/174): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 662 kB 00:00 2024-03-19T00:53:58.613 INFO:teuthology.orchestra.run.smithi069.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 307 kB 00:00 2024-03-19T00:53:58.638 INFO:teuthology.orchestra.run.smithi069.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 229 kB/s | 32 kB 00:00 2024-03-19T00:53:58.855 INFO:teuthology.orchestra.run.smithi069.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.5 MB/s | 3.0 MB 00:00 2024-03-19T00:53:58.872 INFO:teuthology.orchestra.run.smithi069.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 45 kB/s | 10 kB 00:00 2024-03-19T00:53:58.889 INFO:teuthology.orchestra.run.smithi069.stdout:(47/174): flexiblas-openblas-openmp-3.0.4-8.el9 71 kB/s | 17 kB 00:00 2024-03-19T00:53:58.914 INFO:teuthology.orchestra.run.smithi069.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-03-19T00:53:58.948 INFO:teuthology.orchestra.run.smithi069.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-19T00:53:58.990 INFO:teuthology.orchestra.run.smithi069.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 661 kB 00:00 2024-03-19T00:53:59.023 INFO:teuthology.orchestra.run.smithi069.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.5 MB/s | 578 kB 00:00 2024-03-19T00:53:59.048 INFO:teuthology.orchestra.run.smithi069.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 493 kB/s | 46 kB 00:00 2024-03-19T00:53:59.082 INFO:teuthology.orchestra.run.smithi069.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 282 kB 00:00 2024-03-19T00:53:59.124 INFO:teuthology.orchestra.run.smithi069.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2024-03-19T00:53:59.158 INFO:teuthology.orchestra.run.smithi069.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.0 MB/s | 217 kB 00:00 2024-03-19T00:53:59.183 INFO:teuthology.orchestra.run.smithi069.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 341 kB/s | 34 kB 00:00 2024-03-19T00:53:59.525 INFO:teuthology.orchestra.run.smithi069.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-03-19T00:53:59.567 INFO:teuthology.orchestra.run.smithi069.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 2.1 MB/s | 811 kB 00:00 2024-03-19T00:53:59.617 INFO:teuthology.orchestra.run.smithi069.stdout:(59/174): postgresql-13.11-1.el9.x86_64.rpm 3.6 MB/s | 1.5 MB 00:00 2024-03-19T00:53:59.642 INFO:teuthology.orchestra.run.smithi069.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2024-03-19T00:53:59.909 INFO:teuthology.orchestra.run.smithi069.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 17 MB/s | 5.7 MB 00:00 2024-03-19T00:53:59.934 INFO:teuthology.orchestra.run.smithi069.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 280 kB/s | 78 kB 00:00 2024-03-19T00:53:59.993 INFO:teuthology.orchestra.run.smithi069.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-03-19T00:54:00.185 INFO:teuthology.orchestra.run.smithi069.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-03-19T00:54:00.210 INFO:teuthology.orchestra.run.smithi069.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 94 kB/s | 24 kB 00:00 2024-03-19T00:54:00.235 INFO:teuthology.orchestra.run.smithi069.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 79 kB/s | 18 kB 00:00 2024-03-19T00:54:00.260 INFO:teuthology.orchestra.run.smithi069.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-19T00:54:00.294 INFO:teuthology.orchestra.run.smithi069.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 561 kB/s | 32 kB 00:00 2024-03-19T00:54:00.461 INFO:teuthology.orchestra.run.smithi069.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-03-19T00:54:00.520 INFO:teuthology.orchestra.run.smithi069.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 843 kB/s | 217 kB 00:00 2024-03-19T00:54:00.545 INFO:teuthology.orchestra.run.smithi069.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 740 kB/s | 174 kB 00:00 2024-03-19T00:54:00.571 INFO:teuthology.orchestra.run.smithi069.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 637 kB/s | 69 kB 00:00 2024-03-19T00:54:00.596 INFO:teuthology.orchestra.run.smithi069.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 831 kB/s | 41 kB 00:00 2024-03-19T00:54:00.621 INFO:teuthology.orchestra.run.smithi069.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-19T00:54:00.646 INFO:teuthology.orchestra.run.smithi069.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 132 kB 00:00 2024-03-19T00:54:00.681 INFO:teuthology.orchestra.run.smithi069.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-19T00:54:00.705 INFO:teuthology.orchestra.run.smithi069.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 580 kB/s | 47 kB 00:00 2024-03-19T00:54:00.731 INFO:teuthology.orchestra.run.smithi069.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 525 kB/s | 43 kB 00:00 2024-03-19T00:54:00.756 INFO:teuthology.orchestra.run.smithi069.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 789 kB/s | 39 kB 00:00 2024-03-19T00:54:00.898 INFO:teuthology.orchestra.run.smithi069.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 247 kB/s | 41 kB 00:00 2024-03-19T00:54:01.023 INFO:teuthology.orchestra.run.smithi069.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 300 kB 00:00 2024-03-19T00:54:01.240 INFO:teuthology.orchestra.run.smithi069.stdout:(82/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-03-19T00:54:01.265 INFO:teuthology.orchestra.run.smithi069.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 160 kB/s | 56 kB 00:00 2024-03-19T00:54:01.291 INFO:teuthology.orchestra.run.smithi069.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 244 kB/s | 63 kB 00:00 2024-03-19T00:54:01.316 INFO:teuthology.orchestra.run.smithi069.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 908 kB/s | 44 kB 00:00 2024-03-19T00:54:01.341 INFO:teuthology.orchestra.run.smithi069.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-19T00:54:01.375 INFO:teuthology.orchestra.run.smithi069.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2024-03-19T00:54:01.400 INFO:teuthology.orchestra.run.smithi069.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-19T00:54:01.450 INFO:teuthology.orchestra.run.smithi069.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-19T00:54:01.475 INFO:teuthology.orchestra.run.smithi069.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-19T00:54:01.501 INFO:teuthology.orchestra.run.smithi069.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-19T00:54:02.009 INFO:teuthology.orchestra.run.smithi069.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 353 kB/s | 187 kB 00:00 2024-03-19T00:54:02.035 INFO:teuthology.orchestra.run.smithi069.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 367 kB/s | 195 kB 00:00 2024-03-19T00:54:02.135 INFO:teuthology.orchestra.run.smithi069.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-03-19T00:54:02.160 INFO:teuthology.orchestra.run.smithi069.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 173 kB/s | 21 kB 00:00 2024-03-19T00:54:02.260 INFO:teuthology.orchestra.run.smithi069.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 204 kB/s | 25 kB 00:00 2024-03-19T00:54:02.336 INFO:teuthology.orchestra.run.smithi069.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 937 kB/s | 803 kB 00:00 2024-03-19T00:54:02.461 INFO:teuthology.orchestra.run.smithi069.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-03-19T00:54:02.561 INFO:teuthology.orchestra.run.smithi069.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.1 MB/s | 216 kB 00:00 2024-03-19T00:54:02.612 INFO:teuthology.orchestra.run.smithi069.stdout:(100/174): python3-jinja2-2.11.3-5.el9.noarch.r 903 kB/s | 248 kB 00:00 2024-03-19T00:54:02.637 INFO:teuthology.orchestra.run.smithi069.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 468 kB/s | 35 kB 00:00 2024-03-19T00:54:02.687 INFO:teuthology.orchestra.run.smithi069.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 598 kB/s | 44 kB 00:00 2024-03-19T00:54:02.712 INFO:teuthology.orchestra.run.smithi069.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-03-19T00:54:02.813 INFO:teuthology.orchestra.run.smithi069.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2024-03-19T00:54:03.080 INFO:teuthology.orchestra.run.smithi069.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-03-19T00:54:03.188 INFO:teuthology.orchestra.run.smithi069.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.0 MB/s | 4.1 MB 00:01 2024-03-19T00:54:03.538 INFO:teuthology.orchestra.run.smithi069.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 757 kB/s | 548 kB 00:00 2024-03-19T00:54:03.563 INFO:teuthology.orchestra.run.smithi069.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 63 kB/s | 19 kB 00:00 2024-03-19T00:54:03.694 INFO:teuthology.orchestra.run.smithi069.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 200 kB/s | 26 kB 00:00 2024-03-19T00:54:03.731 INFO:teuthology.orchestra.run.smithi069.stdout:(110/174): gperftools-libs-2.9.1-2.el9.x86_64.r 476 kB/s | 309 kB 00:00 2024-03-19T00:54:03.814 INFO:teuthology.orchestra.run.smithi069.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 817 kB/s | 67 kB 00:00 2024-03-19T00:54:03.840 INFO:teuthology.orchestra.run.smithi069.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 336 kB/s | 49 kB 00:00 2024-03-19T00:54:03.965 INFO:teuthology.orchestra.run.smithi069.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-03-19T00:54:03.990 INFO:teuthology.orchestra.run.smithi069.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 865 kB/s | 151 kB 00:00 2024-03-19T00:54:04.057 INFO:teuthology.orchestra.run.smithi069.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-03-19T00:54:04.082 INFO:teuthology.orchestra.run.smithi069.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 476 kB/s | 43 kB 00:00 2024-03-19T00:54:04.133 INFO:teuthology.orchestra.run.smithi069.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 431 kB/s | 32 kB 00:00 2024-03-19T00:54:04.158 INFO:teuthology.orchestra.run.smithi069.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2024-03-19T00:54:04.192 INFO:teuthology.orchestra.run.smithi069.stdout:(119/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2024-03-19T00:54:04.225 INFO:teuthology.orchestra.run.smithi069.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-03-19T00:54:04.275 INFO:teuthology.orchestra.run.smithi069.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-03-19T00:54:04.301 INFO:teuthology.orchestra.run.smithi069.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 473 kB/s | 35 kB 00:00 2024-03-19T00:54:04.368 INFO:teuthology.orchestra.run.smithi069.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-03-19T00:54:04.401 INFO:teuthology.orchestra.run.smithi069.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-19T00:54:04.451 INFO:teuthology.orchestra.run.smithi069.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-03-19T00:54:04.477 INFO:teuthology.orchestra.run.smithi069.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 143 kB/s | 11 kB 00:00 2024-03-19T00:54:04.543 INFO:teuthology.orchestra.run.smithi069.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-03-19T00:54:04.569 INFO:teuthology.orchestra.run.smithi069.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-03-19T00:54:04.594 INFO:teuthology.orchestra.run.smithi069.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2024-03-19T00:54:04.619 INFO:teuthology.orchestra.run.smithi069.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 261 kB/s | 19 kB 00:00 2024-03-19T00:54:04.669 INFO:teuthology.orchestra.run.smithi069.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 265 kB/s | 20 kB 00:00 2024-03-19T00:54:04.695 INFO:teuthology.orchestra.run.smithi069.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 620 kB/s | 46 kB 00:00 2024-03-19T00:54:04.753 INFO:teuthology.orchestra.run.smithi069.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-03-19T00:54:04.779 INFO:teuthology.orchestra.run.smithi069.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 694 kB/s | 58 kB 00:00 2024-03-19T00:54:04.804 INFO:teuthology.orchestra.run.smithi069.stdout:(135/174): python3-more-itertools-8.12.0-2.el9. 591 kB/s | 79 kB 00:00 2024-03-19T00:54:04.837 INFO:teuthology.orchestra.run.smithi069.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-19T00:54:04.863 INFO:teuthology.orchestra.run.smithi069.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-03-19T00:54:04.888 INFO:teuthology.orchestra.run.smithi069.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-03-19T00:54:04.913 INFO:teuthology.orchestra.run.smithi069.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-03-19T00:54:04.938 INFO:teuthology.orchestra.run.smithi069.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 411 kB/s | 31 kB 00:00 2024-03-19T00:54:05.005 INFO:teuthology.orchestra.run.smithi069.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-03-19T00:54:05.031 INFO:teuthology.orchestra.run.smithi069.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 392 kB/s | 36 kB 00:00 2024-03-19T00:54:05.048 INFO:teuthology.orchestra.run.smithi069.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2024-03-19T00:54:05.073 INFO:teuthology.orchestra.run.smithi069.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 770 kB/s | 51 kB 00:00 2024-03-19T00:54:05.115 INFO:teuthology.orchestra.run.smithi069.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-19T00:54:05.140 INFO:teuthology.orchestra.run.smithi069.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 979 kB/s | 90 kB 00:00 2024-03-19T00:54:05.190 INFO:teuthology.orchestra.run.smithi069.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-19T00:54:05.215 INFO:teuthology.orchestra.run.smithi069.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 201 kB/s | 20 kB 00:00 2024-03-19T00:54:05.249 INFO:teuthology.orchestra.run.smithi069.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-03-19T00:54:05.416 INFO:teuthology.orchestra.run.smithi069.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-03-19T00:54:05.491 INFO:teuthology.orchestra.run.smithi069.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.3 MB/s | 321 kB 00:00 2024-03-19T00:54:05.808 INFO:teuthology.orchestra.run.smithi069.stdout:(152/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-03-19T00:54:06.084 INFO:teuthology.orchestra.run.smithi069.stdout:(153/174): librados2-19.0.0-2206.gb6ae2491.el9. 3.9 MB/s | 3.4 MB 00:00 2024-03-19T00:54:06.367 INFO:teuthology.orchestra.run.smithi069.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 149 kB/s | 42 kB 00:00 2024-03-19T00:54:06.442 INFO:teuthology.orchestra.run.smithi069.stdout:(155/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.4 MB/s | 2.3 MB 00:00 2024-03-19T00:54:06.559 INFO:teuthology.orchestra.run.smithi069.stdout:(156/174): NetworkManager-libnm-1.47.2-1.el9.x8 2.4 MB/s | 1.8 MB 00:00 2024-03-19T00:54:06.576 INFO:teuthology.orchestra.run.smithi069.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 535 kB/s | 71 kB 00:00 2024-03-19T00:54:06.676 INFO:teuthology.orchestra.run.smithi069.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 313 kB/s | 36 kB 00:00 2024-03-19T00:54:06.710 INFO:teuthology.orchestra.run.smithi069.stdout:(159/174): NetworkManager-tui-1.47.2-1.el9.x86_ 726 kB/s | 248 kB 00:00 2024-03-19T00:54:06.735 INFO:teuthology.orchestra.run.smithi069.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.2 MB/s | 192 kB 00:00 2024-03-19T00:54:06.785 INFO:teuthology.orchestra.run.smithi069.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-19T00:54:06.819 INFO:teuthology.orchestra.run.smithi069.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 213 kB/s | 23 kB 00:00 2024-03-19T00:54:06.844 INFO:teuthology.orchestra.run.smithi069.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 997 kB/s | 108 kB 00:00 2024-03-19T00:54:06.920 INFO:teuthology.orchestra.run.smithi069.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 155 kB 00:00 2024-03-19T00:54:06.945 INFO:teuthology.orchestra.run.smithi069.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-03-19T00:54:06.970 INFO:teuthology.orchestra.run.smithi069.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 510 kB/s | 64 kB 00:00 2024-03-19T00:54:07.029 INFO:teuthology.orchestra.run.smithi069.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 265 kB/s | 29 kB 00:00 2024-03-19T00:54:07.329 INFO:teuthology.orchestra.run.smithi069.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.2 MB/s | 1.2 MB 00:00 2024-03-19T00:54:07.404 INFO:teuthology.orchestra.run.smithi069.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.1 MB/s | 2.3 MB 00:00 2024-03-19T00:54:07.446 INFO:teuthology.orchestra.run.smithi069.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.9 MB/s | 465 kB 00:00 2024-03-19T00:54:07.555 INFO:teuthology.orchestra.run.smithi069.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 217 kB/s | 31 kB 00:00 2024-03-19T00:54:07.639 INFO:teuthology.orchestra.run.smithi069.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 901 kB/s | 74 kB 00:00 2024-03-19T00:54:07.680 INFO:teuthology.orchestra.run.smithi069.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.6 MB/s | 1.1 MB 00:00 2024-03-19T00:54:07.797 INFO:teuthology.orchestra.run.smithi069.stdout:(174/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.6 MB/s | 2.2 MB 00:00 2024-03-19T00:54:07.805 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-19T00:54:07.806 INFO:teuthology.orchestra.run.smithi069.stdout:Total 13 MB/s | 224 MB 00:16 2024-03-19T00:54:09.228 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-19T00:54:09.607 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-19T00:54:09.608 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-19T00:54:11.998 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-19T00:54:11.998 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-19T00:54:19.304 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-19T00:54:19.422 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-19T00:54:19.470 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-19T00:54:19.507 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:54:19.551 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:54:19.638 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-19T00:54:19.769 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-19T00:54:19.820 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-19T00:54:19.855 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-19T00:54:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-19T00:54:20.346 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-19T00:54:20.412 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:54:20.564 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:54:20.588 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:54:20.701 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:54:20.744 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:54:20.805 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-19T00:54:21.081 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-19T00:54:21.150 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-19T00:54:21.176 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:54:21.319 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:54:21.352 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:54:21.433 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:54:21.487 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-19T00:54:21.512 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:54:21.574 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:54:21.869 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:54:21.921 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:54:21.963 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-19T00:54:22.279 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-19T00:54:22.469 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:54:22.566 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:54:22.727 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:54:22.789 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:54:23.132 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:54:23.311 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-19T00:54:23.435 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-19T00:54:23.478 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-19T00:54:23.540 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-19T00:54:23.613 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-19T00:54:23.732 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-19T00:54:23.794 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-19T00:54:23.836 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-19T00:54:23.927 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-19T00:54:23.969 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-19T00:54:24.028 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-19T00:54:24.075 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-19T00:54:24.122 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-19T00:54:24.269 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-19T00:54:24.318 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-19T00:54:24.371 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-19T00:54:24.408 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-19T00:54:24.531 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-19T00:54:24.587 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-19T00:54:24.805 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-19T00:54:24.990 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-19T00:54:25.056 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-19T00:54:25.746 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-19T00:54:25.843 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-19T00:54:25.974 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-19T00:54:26.054 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:54:26.086 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:54:26.168 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-19T00:54:26.246 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-19T00:54:26.300 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-19T00:54:26.346 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-19T00:54:26.750 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-19T00:54:26.984 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-19T00:54:27.066 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-19T00:54:27.138 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-19T00:54:27.358 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-19T00:54:27.429 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-19T00:54:27.508 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-19T00:54:27.599 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-19T00:54:27.637 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-19T00:54:27.678 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-19T00:54:27.745 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-19T00:54:27.785 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-19T00:54:27.846 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-19T00:54:27.905 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-19T00:54:28.091 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-19T00:54:28.150 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-19T00:54:28.188 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-19T00:54:28.235 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-19T00:54:28.365 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-19T00:54:28.412 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-19T00:54:28.458 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-19T00:54:28.504 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-19T00:54:28.544 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-19T00:54:28.725 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-19T00:54:28.897 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-19T00:54:29.026 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-19T00:54:29.071 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-19T00:54:29.134 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-19T00:54:29.218 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-19T00:54:29.467 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-19T00:54:29.526 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-19T00:54:29.708 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-19T00:54:29.764 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-19T00:54:29.805 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-19T00:54:29.843 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-19T00:54:29.884 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-19T00:54:29.956 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-19T00:54:30.025 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-19T00:54:30.085 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-19T00:54:30.131 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-19T00:54:30.266 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-19T00:54:30.301 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-19T00:54:30.334 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-19T00:54:30.387 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-19T00:54:30.429 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-19T00:54:30.571 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-19T00:54:30.692 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-19T00:54:30.744 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-19T00:54:30.779 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-19T00:54:30.836 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-19T00:54:30.982 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-19T00:54:31.045 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-19T00:54:31.996 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-19T00:54:32.053 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-19T00:54:32.086 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-19T00:54:32.142 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-19T00:54:32.254 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-19T00:54:32.295 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-19T00:54:32.526 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-19T00:54:32.558 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-19T00:54:32.650 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-19T00:54:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-19T00:54:33.589 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-19T00:54:35.126 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-19T00:54:35.221 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-19T00:54:35.260 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-19T00:54:35.290 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-19T00:54:35.330 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-19T00:54:35.704 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-19T00:54:35.737 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:54:35.859 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:54:35.893 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-19T00:54:35.949 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-19T00:54:36.492 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-19T00:54:36.515 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:54:36.598 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:54:36.634 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-19T00:54:39.299 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:54:39.338 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:54:39.967 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:54:40.017 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:54:40.171 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:54:40.171 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-19T00:54:40.171 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-19T00:54:40.171 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:54:40.218 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:54:53.086 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:54:53.086 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2024-03-19T00:54:53.086 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2024-03-19T00:54:53.087 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2024-03-19T00:54:53.087 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2024-03-19T00:54:53.087 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2024-03-19T00:54:53.087 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /root 2024-03-19T00:54:53.087 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2024-03-19T00:54:53.087 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:54:53.314 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:54:53.365 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:54:53.366 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:54:53.366 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-19T00:54:53.366 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:54:54.992 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:54:55.068 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:54:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:54:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-19T00:54:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:54:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:54:55.069 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:54:55.326 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:54:55.403 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:54:55.403 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:54:55.403 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-19T00:54:55.403 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:54:55.403 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:54:55.403 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:54:55.885 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:54:55.952 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:54:55.952 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:54:55.952 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-19T00:54:55.952 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:54:55.953 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:54:55.953 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:54:56.070 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-19T00:54:56.656 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:54:56.688 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:54:56.738 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-19T00:54:56.837 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-19T00:54:56.866 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:54:57.193 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:54:57.226 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:54:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:54:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:54:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:54:58.697 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-19T00:54:58.728 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:54:58.798 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:54:58.799 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:54:58.799 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-19T00:54:58.799 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:54:58.799 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:54:58.799 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:54:58.853 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:54:58.892 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:54:58.982 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-19T00:54:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-19T00:54:59.964 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:55:00.033 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:55:00.033 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:55:00.033 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-19T00:55:00.034 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:55:00.034 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:55:00.034 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:55:00.081 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:55:00.160 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:55:00.160 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:55:00.160 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-19T00:55:00.160 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:55:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:55:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:55:00.307 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:55:00.307 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-19T00:55:00.307 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:55:00.309 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:55:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:55:00.675 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:55:00.749 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:55:00.749 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:55:00.750 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-19T00:55:00.750 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:55:00.750 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:55:00.750 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:55:05.540 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:55:05.585 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-19T00:55:05.621 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-19T00:55:05.658 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-19T00:55:05.918 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-19T00:55:06.034 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-19T00:55:06.088 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-19T00:55:06.143 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-19T00:55:06.204 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-19T00:55:06.250 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-19T00:55:06.286 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-19T00:55:06.321 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-19T00:55:06.373 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-19T00:55:06.463 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-19T00:55:06.571 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-19T00:55:06.613 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-19T00:55:07.474 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:55:07.519 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:55:07.573 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:55:08.804 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-19T00:55:08.899 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-19T00:55:08.956 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-19T00:55:09.088 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-19T00:55:09.128 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-19T00:55:09.162 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-19T00:55:09.195 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-19T00:55:09.263 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-19T00:55:09.263 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:55:09.408 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:55:09.450 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-19T00:55:09.450 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:55:09.571 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:55:09.616 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-19T00:55:09.678 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-19T00:55:09.712 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-19T00:55:09.744 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-19T00:55:09.813 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-19T00:55:09.908 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-19T00:55:09.957 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-19T00:55:09.990 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-19T00:55:10.024 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-19T00:55:10.058 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:55:10.091 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-19T00:55:10.126 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:55:10.126 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:55:10.435 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:55:10.436 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:55:10.597 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:55:10.627 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:55:10.669 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-19T00:55:10.727 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-19T00:55:10.762 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-19T00:55:10.879 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:55:11.132 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:55:11.132 INFO:teuthology.orchestra.run.smithi069.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-19T00:55:11.132 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:55:16.010 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-19T00:55:16.010 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-19T00:55:16.010 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-19T00:55:16.010 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-19T00:55:16.010 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-19T00:55:16.010 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-19T00:55:16.010 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-19T00:55:16.011 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-19T00:55:16.012 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-19T00:55:16.027 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-19T00:55:16.027 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-19T00:55:16.027 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-19T00:55:16.027 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-19T00:55:16.027 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-19T00:55:16.027 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-19T00:55:16.028 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-19T00:55:16.029 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-19T00:55:16.030 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-19T00:55:16.031 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-19T00:55:16.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-19T00:55:16.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-19T00:55:16.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-19T00:55:16.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-19T00:55:16.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-19T00:55:16.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-19T00:55:16.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-19T00:55:16.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-19T00:55:16.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-19T00:55:16.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-19T00:55:16.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-19T00:55:16.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:55:16.035 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-19T00:55:16.036 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-19T00:55:16.037 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-19T00:55:16.038 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:55:16.039 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-19T00:55:16.039 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-19T00:55:16.039 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-19T00:55:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-19T00:55:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:55:18.579 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: acl-2.3.1-4.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-19T00:55:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-19T00:55:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-19T00:55:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-19T00:55:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-19T00:55:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-19T00:55:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-19T00:55:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-19T00:55:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-19T00:55:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-19T00:55:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.2-8.el9.noarch 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.7.4-5.el9.x86_64 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.4-9.el9.x86_64 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: jq-1.6-16.el9.x86_64 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-19T00:55:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: lua-5.4.4-4.el9.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-19T00:55:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-13.11-1.el9.x86_64 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-19T00:55:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-19T00:55:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply-3.11-14.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-19T00:55:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-19T00:55:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: re2-1:20211101-3.el9.x86_64 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-19T00:55:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-19T00:55:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-19T00:55:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:55:18.593 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-19T00:55:19.285 DEBUG:teuthology.parallel:result is None 2024-03-19T00:55:23.479 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 61 B/s | 16 kB 04:21 2024-03-19T00:55:26.756 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 6.5 MB/s | 21 MB 00:03 2024-03-19T00:55:31.764 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-19T00:55:34.076 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:55:34.077 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T00:55:34.091 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-19T00:55:34.092 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-19T00:55:34.093 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-19T00:55:34.094 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-19T00:55:34.096 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-19T00:55:34.099 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-19T00:55:34.100 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-19T00:55:34.178 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-03-19T00:55:34.193 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================================ 2024-03-19T00:55:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-03-19T00:55:34.194 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================================ 2024-03-19T00:55:34.194 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-03-19T00:55:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-19T00:55:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-19T00:55:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-19T00:55:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-19T00:55:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-19T00:55:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-19T00:55:34.194 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-19T00:55:34.195 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-19T00:55:34.196 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-19T00:55:34.197 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-19T00:55:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-19T00:55:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-19T00:55:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-19T00:55:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-19T00:55:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-19T00:55:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-19T00:55:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-19T00:55:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T00:55:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-19T00:55:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-19T00:55:34.198 INFO:teuthology.orchestra.run.smithi086.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-19T00:55:34.199 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-19T00:55:34.200 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-19T00:55:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-19T00:55:34.202 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-19T00:55:34.203 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-19T00:55:34.204 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-19T00:55:34.205 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-19T00:55:34.206 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-19T00:55:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-19T00:55:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-19T00:55:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-19T00:55:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-19T00:55:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-19T00:55:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-19T00:55:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-19T00:55:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-19T00:55:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-19T00:55:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-19T00:55:34.207 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-19T00:55:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-19T00:55:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-19T00:55:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-19T00:55:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-19T00:55:34.208 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-03-19T00:55:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-19T00:55:34.208 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:55:34.209 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-03-19T00:55:34.209 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================================ 2024-03-19T00:55:34.209 INFO:teuthology.orchestra.run.smithi086.stdout:Install 150 Packages 2024-03-19T00:55:34.209 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 24 Packages 2024-03-19T00:55:34.209 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:55:34.210 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 224 M 2024-03-19T00:55:34.210 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-03-19T00:55:35.532 INFO:teuthology.orchestra.run.smithi086.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-03-19T00:55:35.857 INFO:teuthology.orchestra.run.smithi086.stdout:(2/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 2.7 MB/s | 891 kB 00:00 2024-03-19T00:55:35.932 INFO:teuthology.orchestra.run.smithi086.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 1.9 MB/s | 145 kB 00:00 2024-03-19T00:55:36.166 INFO:teuthology.orchestra.run.smithi086.stdout:(4/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 6.0 MB/s | 5.4 MB 00:00 2024-03-19T00:55:36.408 INFO:teuthology.orchestra.run.smithi086.stdout:(5/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 6.2 MB/s | 1.5 MB 00:00 2024-03-19T00:55:36.683 INFO:teuthology.orchestra.run.smithi086.stdout:(6/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 3.1 MB/s | 2.4 MB 00:00 2024-03-19T00:55:37.042 INFO:teuthology.orchestra.run.smithi086.stdout:(7/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 7.4 MB/s | 4.7 MB 00:00 2024-03-19T00:55:37.709 INFO:teuthology.orchestra.run.smithi086.stdout:(8/174): ceph-common-19.0.0-2206.gb6ae2491.el9. 8.5 MB/s | 21 MB 00:02 2024-03-19T00:55:37.776 INFO:teuthology.orchestra.run.smithi086.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 378 kB/s | 25 kB 00:00 2024-03-19T00:55:38.284 INFO:teuthology.orchestra.run.smithi086.stdout:(10/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el 7.9 MB/s | 9.8 MB 00:01 2024-03-19T00:55:38.368 INFO:teuthology.orchestra.run.smithi086.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 2.6 MB/s | 221 kB 00:00 2024-03-19T00:55:38.443 INFO:teuthology.orchestra.run.smithi086.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 362 kB/s | 27 kB 00:00 2024-03-19T00:55:38.510 INFO:teuthology.orchestra.run.smithi086.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 475 kB/s | 32 kB 00:00 2024-03-19T00:55:38.652 INFO:teuthology.orchestra.run.smithi086.stdout:(14/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 5.1 MB/s | 746 kB 00:00 2024-03-19T00:55:38.744 INFO:teuthology.orchestra.run.smithi086.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 1.7 MB/s | 163 kB 00:00 2024-03-19T00:55:38.827 INFO:teuthology.orchestra.run.smithi086.stdout:(16/174): librados-devel-19.0.0-2206.gb6ae2491. 1.5 MB/s | 127 kB 00:00 2024-03-19T00:55:38.927 INFO:teuthology.orchestra.run.smithi086.stdout:(17/174): libradosstriper1-19.0.0-2206.gb6ae249 4.9 MB/s | 501 kB 00:00 2024-03-19T00:55:39.678 INFO:teuthology.orchestra.run.smithi086.stdout:(18/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 6.9 MB/s | 5.1 MB 00:00 2024-03-19T00:55:39.745 INFO:teuthology.orchestra.run.smithi086.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 679 kB/s | 45 kB 00:00 2024-03-19T00:55:39.820 INFO:teuthology.orchestra.run.smithi086.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 1.7 MB/s | 131 kB 00:00 2024-03-19T00:55:39.895 INFO:teuthology.orchestra.run.smithi086.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 2.1 MB/s | 162 kB 00:00 2024-03-19T00:55:39.979 INFO:teuthology.orchestra.run.smithi086.stdout:(22/174): python3-rados-19.0.0-2206.gb6ae2491.e 3.8 MB/s | 321 kB 00:00 2024-03-19T00:55:40.062 INFO:teuthology.orchestra.run.smithi086.stdout:(23/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 3.5 MB/s | 298 kB 00:00 2024-03-19T00:55:40.138 INFO:teuthology.orchestra.run.smithi086.stdout:(24/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.3 MB/s | 99 kB 00:00 2024-03-19T00:55:40.213 INFO:teuthology.orchestra.run.smithi086.stdout:(25/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-19T00:55:40.605 INFO:teuthology.orchestra.run.smithi086.stdout:(26/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 7.9 MB/s | 3.1 MB 00:00 2024-03-19T00:55:40.683 INFO:teuthology.orchestra.run.smithi086.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 2.1 MB/s | 170 kB 00:00 2024-03-19T00:55:40.739 INFO:teuthology.orchestra.run.smithi086.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 613 kB/s | 34 kB 00:00 2024-03-19T00:55:40.814 INFO:teuthology.orchestra.run.smithi086.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 1.8 MB/s | 139 kB 00:00 2024-03-19T00:55:41.064 INFO:teuthology.orchestra.run.smithi086.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 7.2 MB/s | 1.8 MB 00:00 2024-03-19T00:55:41.256 INFO:teuthology.orchestra.run.smithi086.stdout:(31/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x8 3.7 MB/s | 17 MB 00:04 2024-03-19T00:55:41.432 INFO:teuthology.orchestra.run.smithi086.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2206.gb6 1.4 MB/s | 248 kB 00:00 2024-03-19T00:55:41.523 INFO:teuthology.orchestra.run.smithi086.stdout:(33/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 540 kB/s | 49 kB 00:00 2024-03-19T00:55:41.624 INFO:teuthology.orchestra.run.smithi086.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2206.gb 164 kB/s | 16 kB 00:00 2024-03-19T00:55:41.791 INFO:teuthology.orchestra.run.smithi086.stdout:(35/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 1.5 MB/s | 264 kB 00:00 2024-03-19T00:55:41.966 INFO:teuthology.orchestra.run.smithi086.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 8.2 MB/s | 7.4 MB 00:00 2024-03-19T00:55:42.041 INFO:teuthology.orchestra.run.smithi086.stdout:(37/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 244 kB/s | 18 kB 00:00 2024-03-19T00:55:42.067 INFO:teuthology.orchestra.run.smithi086.stdout:(38/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 2.7 MB/s | 749 kB 00:00 2024-03-19T00:55:42.576 INFO:teuthology.orchestra.run.smithi086.stdout:(39/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 9.9 MB/s | 48 MB 00:04 2024-03-19T00:55:42.609 INFO:teuthology.orchestra.run.smithi086.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 662 kB 00:00 2024-03-19T00:55:42.634 INFO:teuthology.orchestra.run.smithi086.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 1.7 MB/s | 920 kB 00:00 2024-03-19T00:55:42.659 INFO:teuthology.orchestra.run.smithi086.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 106 kB 00:00 2024-03-19T00:55:42.676 INFO:teuthology.orchestra.run.smithi086.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 765 kB/s | 32 kB 00:00 2024-03-19T00:55:42.701 INFO:teuthology.orchestra.run.smithi086.stdout:(44/174): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-03-19T00:55:42.777 INFO:teuthology.orchestra.run.smithi086.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-19T00:55:42.794 INFO:teuthology.orchestra.run.smithi086.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 17 kB 00:00 2024-03-19T00:55:42.810 INFO:teuthology.orchestra.run.smithi086.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 94 kB/s | 10 kB 00:00 2024-03-19T00:55:42.827 INFO:teuthology.orchestra.run.smithi086.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-19T00:55:42.844 INFO:teuthology.orchestra.run.smithi086.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-19T00:55:42.886 INFO:teuthology.orchestra.run.smithi086.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-19T00:55:42.911 INFO:teuthology.orchestra.run.smithi086.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-03-19T00:55:42.928 INFO:teuthology.orchestra.run.smithi086.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 569 kB/s | 46 kB 00:00 2024-03-19T00:55:42.962 INFO:teuthology.orchestra.run.smithi086.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-19T00:55:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-19T00:55:43.012 INFO:teuthology.orchestra.run.smithi086.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.5 MB/s | 217 kB 00:00 2024-03-19T00:55:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 516 kB/s | 34 kB 00:00 2024-03-19T00:55:43.105 INFO:teuthology.orchestra.run.smithi086.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 17 MB/s | 1.5 MB 00:00 2024-03-19T00:55:43.188 INFO:teuthology.orchestra.run.smithi086.stdout:(58/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-03-19T00:55:43.222 INFO:teuthology.orchestra.run.smithi086.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 4.1 MB/s | 811 kB 00:00 2024-03-19T00:55:43.247 INFO:teuthology.orchestra.run.smithi086.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 959 kB/s | 134 kB 00:00 2024-03-19T00:55:43.364 INFO:teuthology.orchestra.run.smithi086.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2024-03-19T00:55:43.389 INFO:teuthology.orchestra.run.smithi086.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 558 kB/s | 78 kB 00:00 2024-03-19T00:55:43.423 INFO:teuthology.orchestra.run.smithi086.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-03-19T00:55:43.515 INFO:teuthology.orchestra.run.smithi086.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 5.8 MB 00:00 2024-03-19T00:55:43.540 INFO:teuthology.orchestra.run.smithi086.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 167 kB/s | 24 kB 00:00 2024-03-19T00:55:43.565 INFO:teuthology.orchestra.run.smithi086.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 129 kB/s | 18 kB 00:00 2024-03-19T00:55:43.590 INFO:teuthology.orchestra.run.smithi086.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-19T00:55:43.624 INFO:teuthology.orchestra.run.smithi086.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 541 kB/s | 32 kB 00:00 2024-03-19T00:55:43.674 INFO:teuthology.orchestra.run.smithi086.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 217 kB 00:00 2024-03-19T00:55:43.750 INFO:teuthology.orchestra.run.smithi086.stdout:(70/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-03-19T00:55:43.775 INFO:teuthology.orchestra.run.smithi086.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-19T00:55:43.800 INFO:teuthology.orchestra.run.smithi086.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 552 kB/s | 69 kB 00:00 2024-03-19T00:55:43.825 INFO:teuthology.orchestra.run.smithi086.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 823 kB/s | 41 kB 00:00 2024-03-19T00:55:43.850 INFO:teuthology.orchestra.run.smithi086.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-19T00:55:43.876 INFO:teuthology.orchestra.run.smithi086.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-19T00:55:43.901 INFO:teuthology.orchestra.run.smithi086.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2024-03-19T00:55:43.926 INFO:teuthology.orchestra.run.smithi086.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-19T00:55:43.951 INFO:teuthology.orchestra.run.smithi086.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 568 kB/s | 43 kB 00:00 2024-03-19T00:55:44.318 INFO:teuthology.orchestra.run.smithi086.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-19T00:55:44.335 INFO:teuthology.orchestra.run.smithi086.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 101 kB/s | 39 kB 00:00 2024-03-19T00:55:44.352 INFO:teuthology.orchestra.run.smithi086.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 103 kB/s | 41 kB 00:00 2024-03-19T00:55:44.402 INFO:teuthology.orchestra.run.smithi086.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-03-19T00:55:44.427 INFO:teuthology.orchestra.run.smithi086.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 756 kB/s | 56 kB 00:00 2024-03-19T00:55:44.444 INFO:teuthology.orchestra.run.smithi086.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 685 kB/s | 63 kB 00:00 2024-03-19T00:55:44.461 INFO:teuthology.orchestra.run.smithi086.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 750 kB/s | 44 kB 00:00 2024-03-19T00:55:44.503 INFO:teuthology.orchestra.run.smithi086.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-19T00:55:44.528 INFO:teuthology.orchestra.run.smithi086.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2024-03-19T00:55:44.570 INFO:teuthology.orchestra.run.smithi086.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-19T00:55:44.595 INFO:teuthology.orchestra.run.smithi086.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-19T00:55:44.621 INFO:teuthology.orchestra.run.smithi086.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-19T00:55:44.646 INFO:teuthology.orchestra.run.smithi086.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-19T00:55:44.763 INFO:teuthology.orchestra.run.smithi086.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2024-03-19T00:55:44.769 INFO:teuthology.orchestra.run.smithi163.stdout:[MIRROR] elfutils-libelf-devel-0.190-2.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.190-2.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-19T00:55:44.769 INFO:teuthology.orchestra.run.smithi163.stdout:[MIRROR] libuuid-devel-2.37.4-18.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libuuid-devel-2.37.4-18.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-19T00:55:44.769 INFO:teuthology.orchestra.run.smithi163.stdout:[MIRROR] openssl-devel-3.0.7-27.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openssl-devel-3.0.7-27.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-19T00:55:44.788 INFO:teuthology.orchestra.run.smithi086.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-19T00:55:44.814 INFO:teuthology.orchestra.run.smithi086.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-19T00:55:44.839 INFO:teuthology.orchestra.run.smithi086.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-03-19T00:55:45.048 INFO:teuthology.orchestra.run.smithi163.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 198 B/s | 25 kB 02:11 2024-03-19T00:55:45.056 INFO:teuthology.orchestra.run.smithi086.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 80 kB/s | 21 kB 00:00 2024-03-19T00:55:45.073 INFO:teuthology.orchestra.run.smithi086.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 98 kB/s | 25 kB 00:00 2024-03-19T00:55:45.073 INFO:teuthology.orchestra.run.smithi163.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 167 B/s | 21 kB 02:11 2024-03-19T00:55:45.406 INFO:teuthology.orchestra.run.smithi086.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 700 kB/s | 245 kB 00:00 2024-03-19T00:55:45.407 INFO:teuthology.orchestra.run.smithi163.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 685 kB/s | 245 kB 00:00 2024-03-19T00:55:45.441 INFO:teuthology.orchestra.run.smithi163.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 675 kB/s | 248 kB 00:00 2024-03-19T00:55:45.506 INFO:teuthology.orchestra.run.smithi086.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 572 kB/s | 248 kB 00:00 2024-03-19T00:55:45.532 INFO:teuthology.orchestra.run.smithi086.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.7 MB/s | 216 kB 00:00 2024-03-19T00:55:45.566 INFO:teuthology.orchestra.run.smithi163.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 279 kB/s | 35 kB 00:00 2024-03-19T00:55:45.607 INFO:teuthology.orchestra.run.smithi086.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 348 kB/s | 35 kB 00:00 2024-03-19T00:55:45.624 INFO:teuthology.orchestra.run.smithi163.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 996 kB/s | 216 kB 00:00 2024-03-19T00:55:45.643 INFO:teuthology.orchestra.run.smithi086.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 402 kB/s | 44 kB 00:00 2024-03-19T00:55:45.685 INFO:teuthology.orchestra.run.smithi086.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 534 kB/s | 22 kB 00:00 2024-03-19T00:55:45.691 INFO:teuthology.orchestra.run.smithi163.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 355 kB/s | 44 kB 00:00 2024-03-19T00:55:45.710 INFO:teuthology.orchestra.run.smithi086.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 913 kB/s | 94 kB 00:00 2024-03-19T00:55:45.741 INFO:teuthology.orchestra.run.smithi163.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 801 kB/s | 94 kB 00:00 2024-03-19T00:55:45.827 INFO:teuthology.orchestra.run.smithi086.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-03-19T00:55:45.902 INFO:teuthology.orchestra.run.smithi086.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.9 MB/s | 4.1 MB 00:01 2024-03-19T00:55:45.959 INFO:teuthology.orchestra.run.smithi163.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-03-19T00:55:46.002 INFO:teuthology.orchestra.run.smithi086.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-03-19T00:55:46.069 INFO:teuthology.orchestra.run.smithi086.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 138 kB/s | 19 kB 00:00 2024-03-19T00:55:46.111 INFO:teuthology.orchestra.run.smithi086.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-03-19T00:55:46.136 INFO:teuthology.orchestra.run.smithi086.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-03-19T00:55:46.161 INFO:teuthology.orchestra.run.smithi086.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 971 kB/s | 49 kB 00:00 2024-03-19T00:55:46.228 INFO:teuthology.orchestra.run.smithi086.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-03-19T00:55:46.254 INFO:teuthology.orchestra.run.smithi086.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 575 kB/s | 67 kB 00:00 2024-03-19T00:55:46.279 INFO:teuthology.orchestra.run.smithi086.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-03-19T00:55:46.313 INFO:teuthology.orchestra.run.smithi086.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-03-19T00:55:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 744 kB/s | 43 kB 00:00 2024-03-19T00:55:46.355 INFO:teuthology.orchestra.run.smithi086.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 773 kB/s | 32 kB 00:00 2024-03-19T00:55:46.371 INFO:teuthology.orchestra.run.smithi086.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 450 kB/s | 15 kB 00:00 2024-03-19T00:55:46.397 INFO:teuthology.orchestra.run.smithi086.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-03-19T00:55:46.422 INFO:teuthology.orchestra.run.smithi086.stdout:(120/174): python3-asyncssh-2.13.2-4.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-03-19T00:55:46.425 INFO:teuthology.orchestra.run.smithi163.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-03-19T00:55:46.447 INFO:teuthology.orchestra.run.smithi086.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-03-19T00:55:46.473 INFO:teuthology.orchestra.run.smithi086.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-03-19T00:55:46.490 INFO:teuthology.orchestra.run.smithi086.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 526 kB/s | 35 kB 00:00 2024-03-19T00:55:46.515 INFO:teuthology.orchestra.run.smithi086.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-03-19T00:55:46.534 INFO:teuthology.orchestra.run.smithi163.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 953 kB/s | 548 kB 00:00 2024-03-19T00:55:46.540 INFO:teuthology.orchestra.run.smithi086.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-03-19T00:55:46.557 INFO:teuthology.orchestra.run.smithi086.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-03-19T00:55:46.574 INFO:teuthology.orchestra.run.smithi086.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 526 kB/s | 18 kB 00:00 2024-03-19T00:55:46.591 INFO:teuthology.orchestra.run.smithi086.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 690 kB/s | 23 kB 00:00 2024-03-19T00:55:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 577 kB/s | 19 kB 00:00 2024-03-19T00:55:46.625 INFO:teuthology.orchestra.run.smithi086.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 588 kB/s | 20 kB 00:00 2024-03-19T00:55:46.660 INFO:teuthology.orchestra.run.smithi163.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-03-19T00:55:46.692 INFO:teuthology.orchestra.run.smithi086.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-03-19T00:55:46.708 INFO:teuthology.orchestra.run.smithi086.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-03-19T00:55:46.733 INFO:teuthology.orchestra.run.smithi086.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-03-19T00:55:46.759 INFO:teuthology.orchestra.run.smithi086.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-19T00:55:46.810 INFO:teuthology.orchestra.run.smithi086.stdout:(135/174): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-03-19T00:55:46.826 INFO:teuthology.orchestra.run.smithi086.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-03-19T00:55:46.852 INFO:teuthology.orchestra.run.smithi086.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-03-19T00:55:46.868 INFO:teuthology.orchestra.run.smithi086.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 672 kB/s | 28 kB 00:00 2024-03-19T00:55:46.894 INFO:teuthology.orchestra.run.smithi086.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-03-19T00:55:46.919 INFO:teuthology.orchestra.run.smithi086.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-03-19T00:55:46.944 INFO:teuthology.orchestra.run.smithi086.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-19T00:55:46.969 INFO:teuthology.orchestra.run.smithi086.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-03-19T00:55:46.995 INFO:teuthology.orchestra.run.smithi086.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-03-19T00:55:47.020 INFO:teuthology.orchestra.run.smithi086.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-03-19T00:55:47.043 INFO:teuthology.orchestra.run.smithi163.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 500 kB/s | 309 kB 00:00 2024-03-19T00:55:47.054 INFO:teuthology.orchestra.run.smithi086.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-03-19T00:55:47.079 INFO:teuthology.orchestra.run.smithi086.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-03-19T00:55:47.104 INFO:teuthology.orchestra.run.smithi086.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-19T00:55:47.129 INFO:teuthology.orchestra.run.smithi086.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-03-19T00:55:47.154 INFO:teuthology.orchestra.run.smithi086.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-19T00:55:47.168 INFO:teuthology.orchestra.run.smithi163.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-03-19T00:55:47.205 INFO:teuthology.orchestra.run.smithi086.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-03-19T00:55:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:(110/174): libarrow-9.0.0-10.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-03-19T00:55:47.272 INFO:teuthology.orchestra.run.smithi086.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.9 MB/s | 321 kB 00:00 2024-03-19T00:55:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-03-19T00:55:47.427 INFO:teuthology.orchestra.run.smithi163.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-03-19T00:55:47.439 INFO:teuthology.orchestra.run.smithi086.stdout:(152/174): librados2-19.0.0-2206.gb6ae2491.el9. 11 MB/s | 3.4 MB 00:00 2024-03-19T00:55:47.453 INFO:teuthology.orchestra.run.smithi163.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 310 kB/s | 67 kB 00:00 2024-03-19T00:55:47.497 INFO:teuthology.orchestra.run.smithi086.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 10 MB/s | 2.3 MB 00:00 2024-03-19T00:55:47.539 INFO:teuthology.orchestra.run.smithi086.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-19T00:55:47.581 INFO:teuthology.orchestra.run.smithi086.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-19T00:55:47.623 INFO:teuthology.orchestra.run.smithi086.stdout:(156/174): NetworkManager-libnm-1.47.2-1.el9.x8 10 MB/s | 1.8 MB 00:00 2024-03-19T00:55:47.636 INFO:teuthology.orchestra.run.smithi163.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-03-19T00:55:47.648 INFO:teuthology.orchestra.run.smithi086.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-03-19T00:55:47.662 INFO:teuthology.orchestra.run.smithi163.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-03-19T00:55:47.673 INFO:teuthology.orchestra.run.smithi086.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 728 kB/s | 36 kB 00:00 2024-03-19T00:55:47.699 INFO:teuthology.orchestra.run.smithi086.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-03-19T00:55:47.724 INFO:teuthology.orchestra.run.smithi086.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 256 kB 00:00 2024-03-19T00:55:47.749 INFO:teuthology.orchestra.run.smithi086.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 459 kB/s | 23 kB 00:00 2024-03-19T00:55:47.753 INFO:teuthology.orchestra.run.smithi163.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-03-19T00:55:47.771 INFO:teuthology.orchestra.run.smithi163.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-03-19T00:55:47.774 INFO:teuthology.orchestra.run.smithi086.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-19T00:55:47.800 INFO:teuthology.orchestra.run.smithi086.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-19T00:55:47.825 INFO:teuthology.orchestra.run.smithi086.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-19T00:55:47.850 INFO:teuthology.orchestra.run.smithi086.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-19T00:55:47.871 INFO:teuthology.orchestra.run.smithi163.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-03-19T00:55:47.875 INFO:teuthology.orchestra.run.smithi086.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 573 kB/s | 29 kB 00:00 2024-03-19T00:55:47.926 INFO:teuthology.orchestra.run.smithi086.stdout:(167/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-03-19T00:55:47.938 INFO:teuthology.orchestra.run.smithi163.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-03-19T00:55:47.959 INFO:teuthology.orchestra.run.smithi086.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2024-03-19T00:55:48.001 INFO:teuthology.orchestra.run.smithi086.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 17 MB/s | 2.2 MB 00:00 2024-03-19T00:55:48.035 INFO:teuthology.orchestra.run.smithi086.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.0 MB/s | 465 kB 00:00 2024-03-19T00:55:48.038 INFO:teuthology.orchestra.run.smithi163.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-03-19T00:55:48.052 INFO:teuthology.orchestra.run.smithi086.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 619 kB/s | 31 kB 00:00 2024-03-19T00:55:48.097 INFO:teuthology.orchestra.run.smithi163.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-03-19T00:55:48.113 INFO:teuthology.orchestra.run.smithi086.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 14 MB/s | 1.1 MB 00:00 2024-03-19T00:55:48.131 INFO:teuthology.orchestra.run.smithi163.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-03-19T00:55:48.178 INFO:teuthology.orchestra.run.smithi086.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.1 MB/s | 2.3 MB 00:00 2024-03-19T00:55:48.231 INFO:teuthology.orchestra.run.smithi163.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-03-19T00:55:48.319 INFO:teuthology.orchestra.run.smithi086.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 278 kB/s | 74 kB 00:00 2024-03-19T00:55:48.326 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-03-19T00:55:48.326 INFO:teuthology.orchestra.run.smithi086.stdout:Total 16 MB/s | 224 MB 00:14 2024-03-19T00:55:48.373 INFO:teuthology.orchestra.run.smithi163.stdout:(124/174): openssl-devel-3.0.7-27.el9.x86_64.rp 31 kB/s | 4.1 MB 02:14 2024-03-19T00:55:48.398 INFO:teuthology.orchestra.run.smithi163.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 865 kB/s | 144 kB 00:00 2024-03-19T00:55:48.440 INFO:teuthology.orchestra.run.smithi163.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-03-19T00:55:48.507 INFO:teuthology.orchestra.run.smithi163.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-03-19T00:55:48.541 INFO:teuthology.orchestra.run.smithi163.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-03-19T00:55:48.566 INFO:teuthology.orchestra.run.smithi163.stdout:(129/174): python3-jaraco-8.2.1-3.el9.noarch.rp 55 kB/s | 11 kB 00:00 2024-03-19T00:55:48.608 INFO:teuthology.orchestra.run.smithi163.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-03-19T00:55:48.633 INFO:teuthology.orchestra.run.smithi163.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-03-19T00:55:48.734 INFO:teuthology.orchestra.run.smithi163.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-03-19T00:55:48.759 INFO:teuthology.orchestra.run.smithi163.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 629 kB/s | 79 kB 00:00 2024-03-19T00:55:48.783 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-03-19T00:55:48.861 INFO:teuthology.orchestra.run.smithi163.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 461 kB/s | 58 kB 00:00 2024-03-19T00:55:48.901 INFO:teuthology.orchestra.run.smithi163.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-03-19T00:55:48.969 INFO:teuthology.orchestra.run.smithi163.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-03-19T00:55:48.993 INFO:teuthology.orchestra.run.smithi163.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 978 kB/s | 90 kB 00:00 2024-03-19T00:55:49.053 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-03-19T00:55:49.053 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-03-19T00:55:49.077 INFO:teuthology.orchestra.run.smithi163.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-03-19T00:55:49.110 INFO:teuthology.orchestra.run.smithi163.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-03-19T00:55:49.203 INFO:teuthology.orchestra.run.smithi163.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-03-19T00:55:49.219 INFO:teuthology.orchestra.run.smithi163.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-03-19T00:55:49.287 INFO:teuthology.orchestra.run.smithi163.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-03-19T00:55:49.320 INFO:teuthology.orchestra.run.smithi163.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-03-19T00:55:49.370 INFO:teuthology.orchestra.run.smithi163.stdout:(144/174): python3-kubernetes-26.1.0-2.el9.noar 1.3 MB/s | 1.0 MB 00:00 2024-03-19T00:55:49.413 INFO:teuthology.orchestra.run.smithi163.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-03-19T00:55:49.446 INFO:teuthology.orchestra.run.smithi163.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 714 kB/s | 90 kB 00:00 2024-03-19T00:55:49.513 INFO:teuthology.orchestra.run.smithi163.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 198 kB/s | 20 kB 00:00 2024-03-19T00:55:49.580 INFO:teuthology.orchestra.run.smithi163.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-03-19T00:55:49.622 INFO:teuthology.orchestra.run.smithi163.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.7 MB/s | 427 kB 00:00 2024-03-19T00:55:49.689 INFO:teuthology.orchestra.run.smithi163.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-03-19T00:55:49.840 INFO:teuthology.orchestra.run.smithi163.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.5 MB/s | 321 kB 00:00 2024-03-19T00:55:50.023 INFO:teuthology.orchestra.run.smithi163.stdout:(152/174): librados2-19.0.0-2206.gb6ae2491.el9. 7.7 MB/s | 3.4 MB 00:00 2024-03-19T00:55:50.332 INFO:teuthology.orchestra.run.smithi163.stdout:(153/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 4.6 MB/s | 3.1 MB 00:00 2024-03-19T00:55:50.615 INFO:teuthology.orchestra.run.smithi163.stdout:(154/174): NetworkManager-1.47.2-1.el9.x86_64.r 3.0 MB/s | 2.3 MB 00:00 2024-03-19T00:55:50.682 INFO:teuthology.orchestra.run.smithi163.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 120 kB/s | 42 kB 00:00 2024-03-19T00:55:50.724 INFO:teuthology.orchestra.run.smithi163.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.2 MB/s | 248 kB 00:00 2024-03-19T00:55:50.741 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-03-19T00:55:50.742 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-03-19T00:55:50.766 INFO:teuthology.orchestra.run.smithi163.stdout:(157/174): NetworkManager-libnm-1.47.2-1.el9.x8 2.5 MB/s | 1.8 MB 00:00 2024-03-19T00:55:50.825 INFO:teuthology.orchestra.run.smithi163.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 363 kB/s | 36 kB 00:00 2024-03-19T00:55:50.867 INFO:teuthology.orchestra.run.smithi163.stdout:(159/174): acl-2.3.1-4.el9.x86_64.rpm 387 kB/s | 71 kB 00:00 2024-03-19T00:55:50.892 INFO:teuthology.orchestra.run.smithi163.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.5 MB/s | 192 kB 00:00 2024-03-19T00:55:50.934 INFO:teuthology.orchestra.run.smithi163.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-19T00:55:50.967 INFO:teuthology.orchestra.run.smithi163.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 229 kB/s | 23 kB 00:00 2024-03-19T00:55:51.001 INFO:teuthology.orchestra.run.smithi163.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 994 kB/s | 108 kB 00:00 2024-03-19T00:55:51.043 INFO:teuthology.orchestra.run.smithi163.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-03-19T00:55:51.110 INFO:teuthology.orchestra.run.smithi163.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 589 kB/s | 64 kB 00:00 2024-03-19T00:55:51.144 INFO:teuthology.orchestra.run.smithi163.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 286 kB/s | 29 kB 00:00 2024-03-19T00:55:51.169 INFO:teuthology.orchestra.run.smithi163.stdout:(167/174): libmount-2.37.4-18.el9.x86_64.rpm 674 kB/s | 136 kB 00:00 2024-03-19T00:55:51.227 INFO:teuthology.orchestra.run.smithi163.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 10 MB/s | 1.2 MB 00:00 2024-03-19T00:55:51.361 INFO:teuthology.orchestra.run.smithi163.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.4 MB/s | 465 kB 00:00 2024-03-19T00:55:51.478 INFO:teuthology.orchestra.run.smithi163.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 267 kB/s | 31 kB 00:00 2024-03-19T00:55:51.529 INFO:teuthology.orchestra.run.smithi163.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.6 MB/s | 2.2 MB 00:00 2024-03-19T00:55:51.604 INFO:teuthology.orchestra.run.smithi163.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.6 MB/s | 1.1 MB 00:00 2024-03-19T00:55:51.804 INFO:teuthology.orchestra.run.smithi163.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 270 kB/s | 74 kB 00:00 2024-03-19T00:55:51.904 INFO:teuthology.orchestra.run.smithi163.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.1 MB/s | 2.3 MB 00:00 2024-03-19T00:55:51.908 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-03-19T00:55:51.908 INFO:teuthology.orchestra.run.smithi163.stdout:Total 1.5 MB/s | 224 MB 02:26 2024-03-19T00:55:52.358 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-03-19T00:55:52.629 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-03-19T00:55:52.629 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-03-19T00:55:54.413 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-03-19T00:55:54.414 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-03-19T00:55:57.596 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-03-19T00:55:57.798 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-19T00:55:57.832 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-19T00:55:57.871 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:55:57.880 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:55:57.921 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-19T00:55:57.974 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-19T00:55:58.008 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-19T00:55:58.033 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-19T00:55:58.075 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-19T00:55:58.394 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-19T00:55:58.434 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:55:58.555 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:55:58.577 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:55:58.667 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:55:58.697 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:55:58.738 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-19T00:55:58.903 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-19T00:55:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-19T00:55:58.990 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:55:59.092 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:55:59.116 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:55:59.163 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:55:59.203 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-19T00:55:59.251 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:55:59.266 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:55:59.433 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:55:59.465 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:55:59.497 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-19T00:55:59.602 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-19T00:55:59.699 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:55:59.756 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:55:59.828 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:55:59.882 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:56:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:56:00.210 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-19T00:56:00.306 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-19T00:56:00.345 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-19T00:56:00.371 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-19T00:56:00.407 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-19T00:56:00.490 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-19T00:56:00.543 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-19T00:56:00.582 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-19T00:56:00.646 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-19T00:56:00.680 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-19T00:56:00.724 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-19T00:56:00.757 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-19T00:56:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-19T00:56:00.854 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-19T00:56:00.889 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-19T00:56:00.933 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-19T00:56:00.962 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-19T00:56:01.029 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-19T00:56:01.075 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-19T00:56:01.149 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-19T00:56:01.281 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-19T00:56:01.283 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-03-19T00:56:01.336 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-19T00:56:01.375 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-19T00:56:01.417 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-19T00:56:01.448 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:56:01.457 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T00:56:01.497 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-19T00:56:01.550 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-19T00:56:01.585 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-19T00:56:01.610 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-19T00:56:01.652 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-19T00:56:01.855 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-19T00:56:01.932 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-19T00:56:01.972 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-19T00:56:02.010 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-19T00:56:02.037 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:56:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:56:02.059 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:56:02.091 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-19T00:56:02.132 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-19T00:56:02.157 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:56:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-19T00:56:02.188 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:56:02.200 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-19T00:56:02.278 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:56:02.309 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:56:02.346 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-19T00:56:02.440 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-19T00:56:02.515 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-19T00:56:02.557 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-19T00:56:02.585 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:56:02.626 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-19T00:56:02.687 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:56:02.693 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-19T00:56:02.711 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:56:02.737 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-19T00:56:02.767 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T00:56:02.807 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-19T00:56:02.822 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:56:02.837 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T00:56:02.888 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-19T00:56:02.932 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-19T00:56:03.005 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:56:03.036 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T00:56:03.085 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-19T00:56:03.181 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-19T00:56:03.222 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-19T00:56:03.287 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:56:03.295 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-19T00:56:03.341 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-19T00:56:03.347 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T00:56:03.378 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-19T00:56:03.407 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:56:03.415 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-19T00:56:03.451 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-19T00:56:03.482 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:56:03.495 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-19T00:56:03.537 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-19T00:56:03.670 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-19T00:56:03.706 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T00:56:03.725 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-19T00:56:03.755 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-19T00:56:03.797 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-19T00:56:03.813 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-19T00:56:03.890 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-19T00:56:03.905 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-19T00:56:03.921 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-19T00:56:03.945 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-19T00:56:03.953 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-19T00:56:03.972 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-19T00:56:03.979 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-19T00:56:04.003 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-19T00:56:04.008 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-19T00:56:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-19T00:56:04.081 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-19T00:56:04.144 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-19T00:56:04.182 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-19T00:56:04.213 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-19T00:56:04.246 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-19T00:56:04.281 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-19T00:56:04.310 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-19T00:56:04.325 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-19T00:56:04.353 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-19T00:56:04.358 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-19T00:56:04.386 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-19T00:56:04.401 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-19T00:56:04.449 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-19T00:56:04.454 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-19T00:56:04.491 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-19T00:56:04.534 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-19T00:56:04.563 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-19T00:56:04.632 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-19T00:56:04.650 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-19T00:56:04.678 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-19T00:56:04.699 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-19T00:56:04.751 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-19T00:56:04.828 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-19T00:56:04.863 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-19T00:56:04.885 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-19T00:56:04.895 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-19T00:56:04.926 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-19T00:56:04.948 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-19T00:56:04.954 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-19T00:56:04.988 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-19T00:56:05.027 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-19T00:56:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-19T00:56:05.089 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-19T00:56:05.165 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-19T00:56:05.199 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-19T00:56:05.224 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-19T00:56:05.265 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-19T00:56:05.292 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-19T00:56:05.381 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-19T00:56:05.470 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-19T00:56:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-19T00:56:05.522 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-19T00:56:05.543 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-19T00:56:05.554 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-19T00:56:05.609 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-19T00:56:05.622 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-19T00:56:05.672 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:56:05.679 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T00:56:05.707 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-19T00:56:05.711 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-19T00:56:05.753 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-19T00:56:05.755 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-19T00:56:05.790 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-19T00:56:05.820 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-19T00:56:06.047 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-19T00:56:06.218 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-19T00:56:06.290 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-19T00:56:06.334 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-19T00:56:06.474 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-19T00:56:06.475 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-19T00:56:06.527 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-19T00:56:06.537 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-19T00:56:06.561 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-19T00:56:06.592 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-19T00:56:06.609 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-19T00:56:06.665 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-19T00:56:06.697 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-19T00:56:06.697 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-19T00:56:06.733 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-19T00:56:06.736 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-19T00:56:06.781 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-19T00:56:06.817 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-19T00:56:06.861 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-19T00:56:06.901 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-19T00:56:06.912 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-19T00:56:06.943 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-19T00:56:07.020 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-19T00:56:07.034 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-19T00:56:07.074 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-19T00:56:07.104 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-19T00:56:07.146 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-19T00:56:07.240 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-19T00:56:07.279 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-19T00:56:07.319 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-19T00:56:07.345 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-19T00:56:07.370 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-19T00:56:07.442 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-19T00:56:07.563 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-19T00:56:07.648 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-19T00:56:07.671 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-19T00:56:07.712 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-19T00:56:07.761 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-19T00:56:07.772 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-19T00:56:07.817 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-19T00:56:08.001 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-19T00:56:08.050 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-19T00:56:08.188 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-19T00:56:08.223 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-19T00:56:08.264 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-19T00:56:08.295 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-19T00:56:08.323 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-19T00:56:08.357 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-19T00:56:08.396 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-19T00:56:08.428 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-19T00:56:08.458 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-19T00:56:08.534 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-19T00:56:08.568 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-19T00:56:08.602 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-19T00:56:08.643 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-19T00:56:08.678 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-19T00:56:08.768 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-19T00:56:08.858 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-19T00:56:08.901 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-19T00:56:08.932 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-19T00:56:08.968 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-19T00:56:08.979 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-19T00:56:09.050 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-19T00:56:09.078 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-19T00:56:09.081 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-19T00:56:09.112 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-19T00:56:09.126 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-19T00:56:09.140 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-19T00:56:09.443 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-19T00:56:09.470 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:56:09.552 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:56:09.574 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-19T00:56:09.619 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-19T00:56:09.856 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-19T00:56:09.898 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-19T00:56:09.932 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-19T00:56:09.981 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-19T00:56:10.068 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-19T00:56:10.096 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-19T00:56:10.100 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-19T00:56:10.122 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:56:10.193 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:56:10.216 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-19T00:56:10.278 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-19T00:56:10.306 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-19T00:56:10.383 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-19T00:56:11.027 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-19T00:56:11.138 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-19T00:56:12.359 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-19T00:56:12.427 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:56:12.433 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-19T00:56:12.462 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:56:12.465 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-19T00:56:12.496 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-19T00:56:12.523 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-19T00:56:12.836 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-19T00:56:12.862 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:56:12.936 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T00:56:12.967 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-19T00:56:12.999 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:56:13.002 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-19T00:56:13.040 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:56:13.121 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:56:13.122 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-19T00:56:13.122 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-19T00:56:13.122 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:13.149 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:56:13.462 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-19T00:56:13.490 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:56:13.561 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T00:56:13.584 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-19T00:56:15.859 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:56:15.889 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:56:16.424 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T00:56:16.470 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:56:16.535 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T00:56:16.535 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-19T00:56:16.535 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-19T00:56:16.536 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:16.563 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:56:24.786 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:56:24.786 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-03-19T00:56:24.786 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-03-19T00:56:24.786 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-03-19T00:56:24.786 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-03-19T00:56:24.786 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-03-19T00:56:24.786 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-03-19T00:56:24.786 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-03-19T00:56:24.786 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:24.952 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:56:24.967 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:56:24.967 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:24.967 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-19T00:56:24.967 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:26.456 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:56:26.480 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:56:26.480 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:26.480 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-19T00:56:26.480 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:56:26.480 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:56:26.480 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:26.706 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:56:26.728 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:56:26.728 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:26.728 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-19T00:56:26.728 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:56:26.728 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:56:26.728 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:27.165 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:56:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:56:27.187 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:27.187 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-19T00:56:27.187 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:56:27.187 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:56:27.187 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:27.274 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-19T00:56:27.717 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:56:27.748 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:56:27.782 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-19T00:56:27.841 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-19T00:56:27.864 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:56:28.098 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:56:28.124 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:56:28.284 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T00:56:28.284 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2024-03-19T00:56:28.284 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2024-03-19T00:56:28.284 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2024-03-19T00:56:28.284 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2024-03-19T00:56:28.284 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2024-03-19T00:56:28.284 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /root 2024-03-19T00:56:28.285 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2024-03-19T00:56:28.285 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:28.454 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:56:28.469 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T00:56:28.469 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:28.469 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-19T00:56:28.469 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:29.089 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:56:29.115 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:56:29.215 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:56:29.379 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-19T00:56:29.409 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:56:29.432 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:56:29.432 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:29.432 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-19T00:56:29.432 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:56:29.432 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:56:29.432 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:29.477 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:56:29.487 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:56:29.533 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-19T00:56:29.976 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:56:30.002 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T00:56:30.002 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:30.002 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-19T00:56:30.002 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:56:30.002 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T00:56:30.002 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:30.226 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:56:30.248 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T00:56:30.248 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:30.249 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-19T00:56:30.249 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:56:30.249 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T00:56:30.249 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-19T00:56:30.445 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:56:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:56:30.468 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:30.468 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-19T00:56:30.468 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:56:30.468 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:56:30.468 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:30.512 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:56:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:56:30.538 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:30.538 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-19T00:56:30.538 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:30.579 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:56:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:56:30.602 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:30.602 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-19T00:56:30.602 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:56:30.602 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:56:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:30.694 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:56:30.716 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T00:56:30.716 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:30.716 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-19T00:56:30.716 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:56:30.716 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T00:56:30.716 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:30.804 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-19T00:56:30.914 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:56:30.936 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:56:30.937 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:30.937 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-19T00:56:30.937 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:56:30.937 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:56:30.937 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:31.197 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:56:31.228 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T00:56:31.260 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-19T00:56:31.320 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-19T00:56:31.343 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:56:31.576 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T00:56:31.602 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:56:32.570 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T00:56:32.594 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:56:32.694 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T00:56:32.848 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-19T00:56:32.871 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:56:32.894 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T00:56:32.894 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:32.894 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-19T00:56:32.894 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:56:32.894 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T00:56:32.894 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:32.930 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:56:32.941 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T00:56:32.986 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-19T00:56:33.861 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-19T00:56:33.899 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:56:33.922 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T00:56:33.922 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:33.922 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-19T00:56:33.922 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:56:33.922 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T00:56:33.922 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:33.967 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:56:33.988 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T00:56:33.989 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:33.989 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-19T00:56:33.989 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:34.034 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:56:34.056 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:56:34.056 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:34.056 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-19T00:56:34.056 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:56:34.056 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T00:56:34.056 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:34.360 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:56:34.382 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T00:56:34.382 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T00:56:34.382 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-19T00:56:34.382 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:56:34.382 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T00:56:34.382 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:35.367 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:56:35.408 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-19T00:56:35.439 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-19T00:56:35.466 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-19T00:56:35.644 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-19T00:56:35.757 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-19T00:56:35.798 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-19T00:56:35.837 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-19T00:56:35.897 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-19T00:56:35.940 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-19T00:56:35.973 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-19T00:56:36.005 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-19T00:56:36.035 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-19T00:56:36.089 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-19T00:56:36.150 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-19T00:56:36.183 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-19T00:56:36.819 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:56:36.856 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:56:36.875 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:56:37.702 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-19T00:56:37.784 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-19T00:56:37.825 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-19T00:56:37.906 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-19T00:56:37.978 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-19T00:56:38.003 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-19T00:56:38.029 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-19T00:56:38.055 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-19T00:56:38.055 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:56:38.143 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:56:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-19T00:56:38.181 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:56:38.249 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:56:38.275 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-19T00:56:38.308 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-19T00:56:38.334 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-19T00:56:38.359 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-19T00:56:38.391 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-19T00:56:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-19T00:56:38.471 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-19T00:56:38.496 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-19T00:56:38.522 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-19T00:56:38.547 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:56:38.572 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-19T00:56:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:56:38.578 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:56:38.758 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:56:38.758 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:56:38.822 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:56:38.850 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:56:38.864 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-19T00:56:38.874 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:56:38.895 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-19T00:56:38.899 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-19T00:56:38.922 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-19T00:56:38.932 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-19T00:56:38.959 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-19T00:56:39.022 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:56:39.102 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-19T00:56:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-19T00:56:39.271 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-19T00:56:39.310 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-19T00:56:39.371 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-19T00:56:39.406 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-19T00:56:39.430 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-19T00:56:39.471 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-19T00:56:39.501 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-19T00:56:39.564 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-19T00:56:39.625 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-19T00:56:39.658 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-19T00:56:40.213 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:56:40.248 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:56:40.268 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T00:56:41.097 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-19T00:56:41.170 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-19T00:56:41.210 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-19T00:56:41.291 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-19T00:56:41.332 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-19T00:56:41.357 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-19T00:56:41.382 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-19T00:56:41.409 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-19T00:56:41.409 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:56:41.497 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T00:56:41.535 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-19T00:56:41.535 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:56:41.620 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T00:56:41.646 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-19T00:56:41.679 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-19T00:56:41.705 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-19T00:56:41.729 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-19T00:56:41.763 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-19T00:56:41.802 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-19T00:56:41.835 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-19T00:56:41.860 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-19T00:56:41.886 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-19T00:56:41.911 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:56:41.936 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-19T00:56:41.942 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:56:41.942 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:56:42.123 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T00:56:42.123 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:56:42.197 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T00:56:42.222 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:56:42.247 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-19T00:56:42.280 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-19T00:56:42.307 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-19T00:56:42.372 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:56:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:56:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-19T00:56:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-19T00:56:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-19T00:56:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-19T00:56:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-19T00:56:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-19T00:56:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-19T00:56:43.847 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-19T00:56:43.848 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-19T00:56:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-19T00:56:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-19T00:56:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-19T00:56:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-19T00:56:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-19T00:56:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-19T00:56:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-19T00:56:43.849 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-19T00:56:43.864 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-19T00:56:43.865 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-19T00:56:43.866 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-19T00:56:43.867 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-19T00:56:43.868 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-19T00:56:43.869 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-19T00:56:43.870 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-19T00:56:43.871 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-19T00:56:43.872 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-19T00:56:43.873 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-19T00:56:43.874 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-19T00:56:43.875 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:56:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-19T00:56:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-19T00:56:43.876 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-19T00:56:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-19T00:56:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:46.839 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-03-19T00:56:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-19T00:56:46.839 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: acl-2.3.1-4.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.840 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.2-8.el9.noarch 2024-03-19T00:56:46.841 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.7.4-5.el9.x86_64 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:46.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:46.843 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.4-9.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: jq-1.6-16.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-19T00:56:46.845 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-13.11-1.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-19T00:56:46.846 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-19T00:56:46.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-19T00:56:46.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-19T00:56:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-19T00:56:46.850 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-19T00:56:46.851 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-3.el9.x86_64 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-19T00:56:46.852 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-19T00:56:46.853 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-19T00:56:46.853 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-19T00:56:46.853 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-03-19T00:56:47.249 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T00:56:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-19T00:56:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-19T00:56:47.252 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-19T00:56:47.252 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-19T00:56:47.252 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-19T00:56:47.266 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-19T00:56:47.266 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-19T00:56:47.266 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-19T00:56:47.266 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-19T00:56:47.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-19T00:56:47.268 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-19T00:56:47.269 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-19T00:56:47.269 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-19T00:56:47.269 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-19T00:56:47.269 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-19T00:56:47.269 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-19T00:56:47.269 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-19T00:56:47.269 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-19T00:56:47.269 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-19T00:56:47.269 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-19T00:56:47.269 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-19T00:56:47.269 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-19T00:56:47.270 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-19T00:56:47.271 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-19T00:56:47.272 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-19T00:56:47.273 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-19T00:56:47.274 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-19T00:56:47.274 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-19T00:56:47.274 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-19T00:56:47.274 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-19T00:56:47.274 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-19T00:56:47.274 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-19T00:56:47.274 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-19T00:56:47.274 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-19T00:56:47.274 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-19T00:56:47.274 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-19T00:56:47.274 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-19T00:56:47.275 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-19T00:56:47.276 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-19T00:56:47.277 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-19T00:56:47.278 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-19T00:56:47.279 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-19T00:56:47.279 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-19T00:56:47.279 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-19T00:56:47.279 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-19T00:56:47.279 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-19T00:56:47.279 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-19T00:56:47.333 DEBUG:teuthology.parallel:result is None 2024-03-19T00:56:50.293 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-19T00:56:50.294 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:50.294 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2024-03-19T00:56:50.294 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-19T00:56:50.294 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-19T00:56:50.294 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-19T00:56:50.294 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-19T00:56:50.294 INFO:teuthology.orchestra.run.smithi163.stdout: acl-2.3.1-4.el9.x86_64 2024-03-19T00:56:50.294 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-19T00:56:50.294 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-19T00:56:50.294 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-19T00:56:50.295 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: automake-1.16.2-8.el9.noarch 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: bison-3.7.4-5.el9.x86_64 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:50.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:50.297 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: flex-2.6.4-9.el9.x86_64 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: jq-1.6-16.el9.x86_64 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-19T00:56:50.298 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-19T00:56:50.299 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: lua-5.4.4-4.el9.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-13.11-1.el9.x86_64 2024-03-19T00:56:50.300 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-19T00:56:50.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-19T00:56:50.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply-3.11-14.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-19T00:56:50.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: re2-1:20211101-3.el9.x86_64 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-19T00:56:50.306 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-19T00:56:50.307 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-19T00:56:50.307 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T00:56:50.307 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-03-19T00:56:50.792 DEBUG:teuthology.parallel:result is None 2024-03-19T00:56:50.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:56:51.017 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-19T00:56:51.086 INFO:teuthology.orchestra.run.smithi069.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-19T00:56:51.086 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-19T00:56:51.087 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-19T00:56:51.088 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:56:51.229 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-19T00:56:51.255 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-19T00:56:51.255 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-19T00:56:51.255 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-19T00:56:51.257 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:56:51.392 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-19T00:56:51.415 INFO:teuthology.orchestra.run.smithi163.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-19T00:56:51.415 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-19T00:56:51.415 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-19T00:56:51.417 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-19T00:56:51.417 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T00:56:51.417 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-19T00:56:51.521 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-19T00:56:51.521 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-19T00:56:51.555 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-19T00:56:51.556 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-19T00:56:51.588 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-19T00:56:51.588 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T00:56:51.588 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2024-03-19T00:56:51.683 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-19T00:56:51.825 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-19T00:56:51.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-03-19T00:56:51.856 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-19T00:56:51.927 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-19T00:56:51.927 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2024-03-19T00:56:51.958 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-19T00:56:52.028 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-19T00:56:52.029 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T00:56:52.029 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-19T00:56:52.126 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-19T00:56:52.264 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-19T00:56:52.264 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-19T00:56:52.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-19T00:56:52.366 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-19T00:56:52.367 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-19T00:56:52.398 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-19T00:56:52.467 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-19T00:56:52.468 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T00:56:52.468 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2024-03-19T00:56:52.568 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-19T00:56:52.665 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-19T00:56:52.665 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-03-19T00:56:52.696 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-19T00:56:52.764 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-19T00:56:52.765 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2024-03-19T00:56:52.796 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-19T00:56:52.865 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-19T00:56:52.962 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-19T00:56:52.963 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:56:52.963 INFO:tasks.cephadm:Cluster fsid is 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:56:52.963 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-19T00:56:52.963 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.69', 'mon.b': '172.21.15.86', 'mon.c': '172.21.15.163'} 2024-03-19T00:56:52.963 INFO:tasks.cephadm:First mon is mon.a on smithi069 2024-03-19T00:56:52.963 INFO:tasks.cephadm:First mgr is x 2024-03-19T00:56:52.964 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-19T00:56:52.964 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname $(hostname -s) 2024-03-19T00:56:53.061 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-03-19T00:56:53.092 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname $(hostname -s) 2024-03-19T00:56:53.124 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-19T00:56:53.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:56:53.338 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77736/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2206-gb6ae2491', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2206.gb6ae2491'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-18 20:42:27.872142', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240318.181317', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-19T00:56:53.473 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-batrick-testing-20240318.181317, sha1 b6ae24918b03b1b8a19c9263a50655bba80bb810 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:56:53.475 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm 2024-03-19T00:56:53.475 DEBUG:teuthology.orchestra.run.smithi069:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-19T00:56:54.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 19 00:56 /home/ubuntu/cephtest/cephadm 2024-03-19T00:56:54.463 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-19T00:56:55.002 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 19 00:56 /home/ubuntu/cephtest/cephadm 2024-03-19T00:56:55.002 DEBUG:teuthology.orchestra.run.smithi163:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-19T00:56:55.540 INFO:teuthology.orchestra.run.smithi163.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 19 00:56 /home/ubuntu/cephtest/cephadm 2024-03-19T00:56:55.540 DEBUG:teuthology.orchestra.run.smithi069:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-19T00:56:55.599 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-19T00:56:55.617 DEBUG:teuthology.orchestra.run.smithi163:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-19T00:56:55.651 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 on all hosts... 2024-03-19T00:56:55.651 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-19T00:56:55.658 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-19T00:56:55.660 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-19T00:56:56.532 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T00:56:56.545 INFO:teuthology.orchestra.run.smithi163.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T00:56:56.732 INFO:teuthology.orchestra.run.smithi069.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T00:57:49.786 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T00:57:49.787 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T00:57:49.787 INFO:teuthology.orchestra.run.smithi069.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-19T00:57:49.787 INFO:teuthology.orchestra.run.smithi069.stdout: "repo_digests": [ 2024-03-19T00:57:49.787 INFO:teuthology.orchestra.run.smithi069.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-19T00:57:49.787 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T00:57:49.787 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T00:57:50.031 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-03-19T00:57:50.032 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T00:57:50.032 INFO:teuthology.orchestra.run.smithi163.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-19T00:57:50.032 INFO:teuthology.orchestra.run.smithi163.stdout: "repo_digests": [ 2024-03-19T00:57:50.032 INFO:teuthology.orchestra.run.smithi163.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-19T00:57:50.032 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2024-03-19T00:57:50.032 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-03-19T00:57:58.060 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-19T00:57:58.060 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T00:57:58.060 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-19T00:57:58.060 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-03-19T00:57:58.060 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-19T00:57:58.060 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-19T00:57:58.060 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-19T00:57:58.084 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph 2024-03-19T00:57:58.188 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-03-19T00:57:58.218 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph 2024-03-19T00:57:58.254 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /etc/ceph 2024-03-19T00:57:58.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-03-19T00:57:58.372 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /etc/ceph 2024-03-19T00:57:58.403 INFO:tasks.cephadm:Writing seed config... 2024-03-19T00:57:58.404 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-19T00:57:58.405 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-19T00:57:58.406 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-19T00:57:58.407 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-19T00:57:58.408 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T00:57:58.408 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-19T00:57:58.465 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 934de1c2-e58b-11ee-95c9-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-19T00:57:58.465 DEBUG:teuthology.orchestra.run.smithi069:mon.a> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.a.service 2024-03-19T00:57:58.513 DEBUG:teuthology.orchestra.run.smithi069:mgr.x> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.x.service 2024-03-19T00:57:58.557 INFO:tasks.cephadm:Bootstrapping... 2024-03-19T00:57:58.557 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 -v bootstrap --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.69 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:57:58.835 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-19T00:57:58.836 INFO:teuthology.orchestra.run.smithi069.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810', '-v', 'bootstrap', '--fsid', '934de1c2-e58b-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.69', '--skip-admin-label'] 2024-03-19T00:57:58.890 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-19T00:57:58.890 INFO:teuthology.orchestra.run.smithi069.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-19T00:57:58.891 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying podman|docker is present... 2024-03-19T00:57:58.942 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-19T00:57:58.942 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying lvm2 is present... 2024-03-19T00:57:58.942 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying time synchronization is in place... 2024-03-19T00:57:58.955 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-19T00:57:58.955 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-19T00:57:58.972 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-19T00:57:58.972 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-03-19T00:57:58.994 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2024-03-19T00:57:59.014 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2024-03-19T00:57:59.015 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2024-03-19T00:57:59.015 INFO:teuthology.orchestra.run.smithi069.stdout:Repeating the final host check... 2024-03-19T00:57:59.070 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-19T00:57:59.071 INFO:teuthology.orchestra.run.smithi069.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-19T00:57:59.071 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl is present 2024-03-19T00:57:59.071 INFO:teuthology.orchestra.run.smithi069.stdout:lvcreate is present 2024-03-19T00:57:59.085 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-19T00:57:59.085 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-19T00:57:59.100 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-19T00:57:59.101 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-03-19T00:57:59.120 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2024-03-19T00:57:59.140 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2024-03-19T00:57:59.140 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2024-03-19T00:57:59.140 INFO:teuthology.orchestra.run.smithi069.stdout:Host looks OK 2024-03-19T00:57:59.140 INFO:teuthology.orchestra.run.smithi069.stdout:Cluster fsid: 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:57:59.141 INFO:teuthology.orchestra.run.smithi069.stdout:Acquiring lock 140186812657088 on /run/cephadm/934de1c2-e58b-11ee-95c9-87774f69a715.lock 2024-03-19T00:57:59.141 INFO:teuthology.orchestra.run.smithi069.stdout:Lock 140186812657088 acquired on /run/cephadm/934de1c2-e58b-11ee-95c9-87774f69a715.lock 2024-03-19T00:57:59.142 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 3300 ... 2024-03-19T00:57:59.143 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 6789 ... 2024-03-19T00:57:59.144 INFO:teuthology.orchestra.run.smithi069.stdout:Base mon IP(s) is [172.21.15.69:3300, 172.21.15.69:6789], mon addrv is [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2024-03-19T00:57:59.158 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.69 metric 100 2024-03-19T00:57:59.158 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.69 metric 100 2024-03-19T00:57:59.171 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-19T00:57:59.171 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-19T00:57:59.171 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2024-03-19T00:57:59.182 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-19T00:57:59.182 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-19T00:57:59.182 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-19T00:57:59.182 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-19T00:57:59.182 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-03-19T00:57:59.183 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-19T00:57:59.185 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2024-03-19T00:57:59.185 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2024-03-19T00:57:59.185 INFO:teuthology.orchestra.run.smithi069.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-19T00:57:59.188 INFO:teuthology.orchestra.run.smithi069.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-19T00:57:59.190 INFO:teuthology.orchestra.run.smithi069.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T00:57:59.834 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c 2024-03-19T00:57:59.834 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T00:57:59.834 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Getting image source signatures 2024-03-19T00:57:59.834 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-19T00:57:59.834 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:93b5a8d362aeee7e8b1f0bbf18238e1bb256c50573ad5b72aff9e382b1bb3ccb 2024-03-19T00:57:59.834 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying config sha256:2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c 2024-03-19T00:57:59.834 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-19T00:58:01.035 INFO:teuthology.orchestra.run.smithi069.stdout:ceph: stdout ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev) 2024-03-19T00:58:01.035 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph version: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev) 2024-03-19T00:58:01.035 INFO:teuthology.orchestra.run.smithi069.stdout:Extracting ceph user uid/gid from container image... 2024-03-19T00:58:02.144 INFO:teuthology.orchestra.run.smithi069.stdout:stat: stdout 167 167 2024-03-19T00:58:02.144 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial keys... 2024-03-19T00:58:03.243 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQAa4/hlMwV9IRAAhggcef3x1ypVFOGT4lfkIA== 2024-03-19T00:58:04.373 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQAb4/hltK3sLRAAfajhxOKHX0au5ao9zM83/A== 2024-03-19T00:58:05.305 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQAc4/hlOCmFLhAAXLPkWLKv81x4G85w4Nl0uw== 2024-03-19T00:58:05.306 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial monmap... 2024-03-19T00:58:06.433 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-19T00:58:06.433 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-19T00:58:06.433 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:06.433 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-19T00:58:06.434 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool for a [v2:172.21.15.69:3300,v1:172.21.15.69:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-19T00:58:06.434 INFO:teuthology.orchestra.run.smithi069.stdout:setting min_mon_release = quincy 2024-03-19T00:58:06.434 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: set fsid to 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:06.434 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-19T00:58:06.434 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:58:06.434 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mon... 2024-03-19T00:58:07.509 INFO:teuthology.orchestra.run.smithi069.stdout:create mon.a on 2024-03-19T00:58:07.828 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-19T00:58:08.075 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-19T00:58:08.377 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-934de1c2-e58b-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-934de1c2-e58b-11ee-95c9-87774f69a715.target. 2024-03-19T00:58:08.377 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-934de1c2-e58b-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-934de1c2-e58b-11ee-95c9-87774f69a715.target. 2024-03-19T00:58:08.758 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.a 2024-03-19T00:58:08.758 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-19T00:58:09.057 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-934de1c2-e58b-11ee-95c9-87774f69a715.target.wants/ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-934de1c2-e58b-11ee-95c9-87774f69a715@.service. 2024-03-19T00:58:09.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:09 smithi069 podman[31330]: 2024-03-19 00:58:09.469000088 +0000 UTC m=+0.166500284 container create 18234ad9448d4efdabd5f655a274213f88d42dcc29378be8249e0d24d86c62c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:58:09.705 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:09 smithi069 podman[31330]: 2024-03-19 00:58:09.703351189 +0000 UTC m=+0.400851458 container init 18234ad9448d4efdabd5f655a274213f88d42dcc29378be8249e0d24d86c62c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-19T00:58:09.823 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T00:58:09.824 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-03-19T00:58:09.838 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T00:58:09.838 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-03-19T00:58:09.838 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-03-19T00:58:09.838 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2024-03-19T00:58:09.839 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon to start... 2024-03-19T00:58:09.839 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon... 2024-03-19T00:58:09.986 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:09 smithi069 podman[31330]: 2024-03-19 00:58:09.713200216 +0000 UTC m=+0.410700412 container start 18234ad9448d4efdabd5f655a274213f88d42dcc29378be8249e0d24d86c62c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-19T00:58:09.987 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:09 smithi069 bash[31330]: 18234ad9448d4efdabd5f655a274213f88d42dcc29378be8249e0d24d86c62c0 2024-03-19T00:58:09.987 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:09 smithi069 ceph-mon[31348]: mkfs 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:09.987 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:09 smithi069 ceph-mon[31348]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-19T00:58:09.987 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:09 smithi069 systemd[1]: Started Ceph mon.a for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T00:58:11.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-19T00:58:11.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: monmap epoch 1 2024-03-19T00:58:11.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:11.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: last_changed 2024-03-19T00:58:05.726813+0000 2024-03-19T00:58:11.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: created 2024-03-19T00:58:05.726813+0000 2024-03-19T00:58:11.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: min_mon_release 19 (squid) 2024-03-19T00:58:11.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: election_strategy: 1 2024-03-19T00:58:11.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-03-19T00:58:11.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: fsmap 2024-03-19T00:58:11.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: osdmap e1: 0 total, 0 up, 0 in 2024-03-19T00:58:11.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: mgrmap e1: no daemons active 2024-03-19T00:58:11.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:10 smithi069 ceph-mon[31348]: from='client.? 172.21.15.69:0/4168098831' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-19T00:58:11.213 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout cluster: 2024-03-19T00:58:11.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout id: 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:11.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-19T00:58:11.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:11.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout services: 2024-03-19T00:58:11.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.752794s) 2024-03-19T00:58:11.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-19T00:58:11.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-19T00:58:11.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:11.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout data: 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pgs: 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.452+0000 7fe5d3e00700 1 Processor -- start 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.452+0000 7fe5d3e00700 1 -- start start 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.452+0000 7fe5d3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d410a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.452+0000 7fe5d3e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d410ab10 con 0x7fe5d410a120 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.454+0000 7fe5d2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d410a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.454+0000 7fe5d2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d410a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42588/0 (socket says 172.21.15.69:42588) 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.454+0000 7fe5d2a00700 1 -- 172.21.15.69:0/1725463284 learned_addr learned my addr 172.21.15.69:0/1725463284 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.455+0000 7fe5d2a00700 1 -- 172.21.15.69:0/1725463284 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d410aee0 con 0x7fe5d410a120 2024-03-19T00:58:11.221 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.455+0000 7fe5d2a00700 1 --2- 172.21.15.69:0/1725463284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d410a540 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40098a0 tx=0x7fe5c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b00eea4c865d53c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.455+0000 7fe5d1600700 1 -- 172.21.15.69:0/1725463284 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5c4004160 con 0x7fe5d410a120 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.456+0000 7fe5d1600700 1 -- 172.21.15.69:0/1725463284 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c4004300 con 0x7fe5d410a120 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.456+0000 7fe5d1600700 1 -- 172.21.15.69:0/1725463284 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5c40036a0 con 0x7fe5d410a120 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.456+0000 7fe5d3e00700 1 -- 172.21.15.69:0/1725463284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 msgr2=0x7fe5d410a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.456+0000 7fe5d3e00700 1 --2- 172.21.15.69:0/1725463284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d410a540 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40098a0 tx=0x7fe5c4009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.456+0000 7fe5d3e00700 1 -- 172.21.15.69:0/1725463284 shutdown_connections 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.456+0000 7fe5d3e00700 1 --2- 172.21.15.69:0/1725463284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d410a540 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.456+0000 7fe5d3e00700 1 -- 172.21.15.69:0/1725463284 >> 172.21.15.69:0/1725463284 conn(0x7fe5d407d250 msgr2=0x7fe5d407d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.456+0000 7fe5d3e00700 1 -- 172.21.15.69:0/1725463284 shutdown_connections 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.456+0000 7fe5d3e00700 1 -- 172.21.15.69:0/1725463284 wait complete. 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.457+0000 7fe5d3e00700 1 Processor -- start 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.458+0000 7fe5d3e00700 1 -- start start 2024-03-19T00:58:11.222 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.458+0000 7fe5d3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.458+0000 7fe5d3e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d410b410 con 0x7fe5d410a120 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.458+0000 7fe5d2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.458+0000 7fe5d2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d419d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42596/0 (socket says 172.21.15.69:42596) 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.458+0000 7fe5d2a00700 1 -- 172.21.15.69:0/4168098831 learned_addr learned my addr 172.21.15.69:0/4168098831 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.459+0000 7fe5d2a00700 1 -- 172.21.15.69:0/4168098831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d419d890 con 0x7fe5d410a120 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.459+0000 7fe5d2a00700 1 --2- 172.21.15.69:0/4168098831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d419d350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000c00 tx=0x7fe5c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.459+0000 7fe5cb400700 1 -- 172.21.15.69:0/4168098831 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5c4003db0 con 0x7fe5d410a120 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.459+0000 7fe5cb400700 1 -- 172.21.15.69:0/4168098831 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c4011630 con 0x7fe5d410a120 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.459+0000 7fe5d3e00700 1 -- 172.21.15.69:0/4168098831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d419db20 con 0x7fe5d410a120 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.459+0000 7fe5cb400700 1 -- 172.21.15.69:0/4168098831 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5c4011950 con 0x7fe5d410a120 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.459+0000 7fe5d3e00700 1 -- 172.21.15.69:0/4168098831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d41a0810 con 0x7fe5d410a120 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.460+0000 7fe5cb400700 1 -- 172.21.15.69:0/4168098831 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe5c4011af0 con 0x7fe5d410a120 2024-03-19T00:58:11.223 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.460+0000 7fe5cb400700 1 -- 172.21.15.69:0/4168098831 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe5c401c450 con 0x7fe5d410a120 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.460+0000 7fe5d3e00700 1 -- 172.21.15.69:0/4168098831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d4063940 con 0x7fe5d410a120 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.464+0000 7fe5cb400700 1 -- 172.21.15.69:0/4168098831 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe5c4032020 con 0x7fe5d410a120 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.542+0000 7fe5d3e00700 1 -- 172.21.15.69:0/4168098831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe5d402dd30 con 0x7fe5d410a120 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.543+0000 7fe5cb400700 1 -- 172.21.15.69:0/4168098831 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fe5c4011de0 con 0x7fe5d410a120 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.544+0000 7fe5d3e00700 1 -- 172.21.15.69:0/4168098831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 msgr2=0x7fe5d419d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.544+0000 7fe5d3e00700 1 --2- 172.21.15.69:0/4168098831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d419d350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000c00 tx=0x7fe5c4004750 comp rx=0 tx=0).stop 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.544+0000 7fe5d3e00700 1 -- 172.21.15.69:0/4168098831 shutdown_connections 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.544+0000 7fe5d3e00700 1 --2- 172.21.15.69:0/4168098831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5d410a120 0x7fe5d419d350 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.544+0000 7fe5d3e00700 1 -- 172.21.15.69:0/4168098831 >> 172.21.15.69:0/4168098831 conn(0x7fe5d407d250 msgr2=0x7fe5d4193fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.544+0000 7fe5d3e00700 1 -- 172.21.15.69:0/4168098831 shutdown_connections 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:10.544+0000 7fe5d3e00700 1 -- 172.21.15.69:0/4168098831 wait complete. 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:mon is available 2024-03-19T00:58:11.224 INFO:teuthology.orchestra.run.smithi069.stdout:Assimilating anything we can from ceph.conf... 2024-03-19T00:58:12.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:11 smithi069 ceph-mon[31348]: from='client.? 172.21.15.69:0/2490139218' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-19T00:58:12.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:11 smithi069 ceph-mon[31348]: from='client.? 172.21.15.69:0/2490139218' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-19T00:58:12.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:12.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2024-03-19T00:58:12.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:12.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2024-03-19T00:58:12.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-19T00:58:12.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-19T00:58:12.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.780+0000 7f87d3e00700 1 Processor -- start 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.780+0000 7f87d3e00700 1 -- start start 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.781+0000 7f87d3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc108140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.781+0000 7f87d3e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc108710 con 0x7f87cc107d20 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.781+0000 7f87d1400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.781+0000 7f87d1400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc108140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42606/0 (socket says 172.21.15.69:42606) 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.781+0000 7f87d1400700 1 -- 172.21.15.69:0/588412102 learned_addr learned my addr 172.21.15.69:0/588412102 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.782+0000 7f87d1400700 1 -- 172.21.15.69:0/588412102 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc108e80 con 0x7f87cc107d20 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.782+0000 7f87d1400700 1 --2- 172.21.15.69:0/588412102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc108140 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f87c00098a0 tx=0x7f87c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a40107fbf12019 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.782+0000 7f87cbe00700 1 -- 172.21.15.69:0/588412102 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c0004160 con 0x7f87cc107d20 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.782+0000 7f87cbe00700 1 -- 172.21.15.69:0/588412102 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c0004300 con 0x7f87cc107d20 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.782+0000 7f87cbe00700 1 -- 172.21.15.69:0/588412102 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c00036a0 con 0x7f87cc107d20 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.783+0000 7f87d3e00700 1 -- 172.21.15.69:0/588412102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 msgr2=0x7f87cc108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.783+0000 7f87d3e00700 1 --2- 172.21.15.69:0/588412102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc108140 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f87c00098a0 tx=0x7f87c0009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:12.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.783+0000 7f87d3e00700 1 -- 172.21.15.69:0/588412102 shutdown_connections 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.783+0000 7f87d3e00700 1 --2- 172.21.15.69:0/588412102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc108140 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.783+0000 7f87d3e00700 1 -- 172.21.15.69:0/588412102 >> 172.21.15.69:0/588412102 conn(0x7f87cc1032e0 msgr2=0x7f87cc105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.783+0000 7f87d3e00700 1 -- 172.21.15.69:0/588412102 shutdown_connections 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.783+0000 7f87d3e00700 1 -- 172.21.15.69:0/588412102 wait complete. 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.784+0000 7f87d3e00700 1 Processor -- start 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.784+0000 7f87d3e00700 1 -- start start 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.785+0000 7f87d3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.785+0000 7f87d3e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1093b0 con 0x7f87cc107d20 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.785+0000 7f87d1400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.785+0000 7f87d1400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42618/0 (socket says 172.21.15.69:42618) 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.785+0000 7f87d1400700 1 -- 172.21.15.69:0/2490139218 learned_addr learned my addr 172.21.15.69:0/2490139218 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.785+0000 7f87d1400700 1 -- 172.21.15.69:0/2490139218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc19b920 con 0x7f87cc107d20 2024-03-19T00:58:12.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.786+0000 7f87d1400700 1 --2- 172.21.15.69:0/2490139218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc19b3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f87c0000c00 tx=0x7f87c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.786+0000 7f87ca000700 1 -- 172.21.15.69:0/2490139218 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c0003db0 con 0x7f87cc107d20 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.786+0000 7f87ca000700 1 -- 172.21.15.69:0/2490139218 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c0011630 con 0x7f87cc107d20 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.786+0000 7f87ca000700 1 -- 172.21.15.69:0/2490139218 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c0011950 con 0x7f87cc107d20 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.786+0000 7f87d3e00700 1 -- 172.21.15.69:0/2490139218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87cc19bbb0 con 0x7f87cc107d20 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.786+0000 7f87d3e00700 1 -- 172.21.15.69:0/2490139218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87cc19e850 con 0x7f87cc107d20 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.786+0000 7f87ca000700 1 -- 172.21.15.69:0/2490139218 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f87c0011af0 con 0x7f87cc107d20 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.786+0000 7f87ca000700 1 -- 172.21.15.69:0/2490139218 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f87c001c450 con 0x7f87cc107d20 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.787+0000 7f87d3e00700 1 -- 172.21.15.69:0/2490139218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87cc063940 con 0x7f87cc107d20 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.790+0000 7f87ca000700 1 -- 172.21.15.69:0/2490139218 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f87c0032020 con 0x7f87cc107d20 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.866+0000 7f87d3e00700 1 -- 172.21.15.69:0/2490139218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f87cc02dd30 con 0x7f87cc107d20 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.871+0000 7f87ca000700 1 -- 172.21.15.69:0/2490139218 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f87c0011de0 con 0x7f87cc107d20 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.872+0000 7f87d3e00700 1 -- 172.21.15.69:0/2490139218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 msgr2=0x7f87cc19b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.872+0000 7f87d3e00700 1 --2- 172.21.15.69:0/2490139218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc19b3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f87c0000c00 tx=0x7f87c0004750 comp rx=0 tx=0).stop 2024-03-19T00:58:12.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.873+0000 7f87d3e00700 1 -- 172.21.15.69:0/2490139218 shutdown_connections 2024-03-19T00:58:12.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.873+0000 7f87d3e00700 1 --2- 172.21.15.69:0/2490139218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc107d20 0x7f87cc19b3e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:12.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.873+0000 7f87d3e00700 1 -- 172.21.15.69:0/2490139218 >> 172.21.15.69:0/2490139218 conn(0x7f87cc1032e0 msgr2=0x7f87cc192020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:12.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.873+0000 7f87d3e00700 1 -- 172.21.15.69:0/2490139218 shutdown_connections 2024-03-19T00:58:12.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:11.873+0000 7f87d3e00700 1 -- 172.21.15.69:0/2490139218 wait complete. 2024-03-19T00:58:12.542 INFO:teuthology.orchestra.run.smithi069.stdout:Generating new minimal ceph.conf... 2024-03-19T00:58:14.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:13 smithi069 ceph-mon[31348]: from='client.? 172.21.15.69:0/2505808883' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:58:21.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.578+0000 7f3909400700 1 Processor -- start 2024-03-19T00:58:21.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.578+0000 7f3909400700 1 -- start start 2024-03-19T00:58:21.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.578+0000 7f3909400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390407c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:21.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.579+0000 7f3909400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f390407c820 con 0x7f390407dde0 2024-03-19T00:58:21.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.579+0000 7f3902800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390407c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:21.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.579+0000 7f3902800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390407c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42624/0 (socket says 172.21.15.69:42624) 2024-03-19T00:58:21.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.579+0000 7f3902800700 1 -- 172.21.15.69:0/1617412155 learned_addr learned my addr 172.21.15.69:0/1617412155 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:21.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.579+0000 7f3902800700 1 -- 172.21.15.69:0/1617412155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390407c9a0 con 0x7f390407dde0 2024-03-19T00:58:21.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.580+0000 7f3902800700 1 --2- 172.21.15.69:0/1617412155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390407c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f38f40098a0 tx=0x7f38f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3836f85860f9d00b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:21.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.580+0000 7f3901400700 1 -- 172.21.15.69:0/1617412155 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38f4004160 con 0x7f390407dde0 2024-03-19T00:58:21.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.580+0000 7f3901400700 1 -- 172.21.15.69:0/1617412155 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38f4004300 con 0x7f390407dde0 2024-03-19T00:58:21.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.580+0000 7f3901400700 1 -- 172.21.15.69:0/1617412155 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38f4011b50 con 0x7f390407dde0 2024-03-19T00:58:21.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.581+0000 7f3909400700 1 -- 172.21.15.69:0/1617412155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 msgr2=0x7f390407c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:21.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.581+0000 7f3909400700 1 --2- 172.21.15.69:0/1617412155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390407c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f38f40098a0 tx=0x7f38f4009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:21.390 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.581+0000 7f3909400700 1 -- 172.21.15.69:0/1617412155 shutdown_connections 2024-03-19T00:58:21.390 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.581+0000 7f3909400700 1 --2- 172.21.15.69:0/1617412155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390407c2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:21.390 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.581+0000 7f3909400700 1 -- 172.21.15.69:0/1617412155 >> 172.21.15.69:0/1617412155 conn(0x7f39041032d0 msgr2=0x7f39041056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:21.390 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.581+0000 7f3909400700 1 -- 172.21.15.69:0/1617412155 shutdown_connections 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.581+0000 7f3909400700 1 -- 172.21.15.69:0/1617412155 wait complete. 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.582+0000 7f3909400700 1 Processor -- start 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.583+0000 7f3909400700 1 -- start start 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.583+0000 7f3909400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.583+0000 7f3909400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f390410e2a0 con 0x7f390407dde0 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.583+0000 7f3902800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.583+0000 7f3902800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42630/0 (socket says 172.21.15.69:42630) 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.583+0000 7f3902800700 1 -- 172.21.15.69:0/2505808883 learned_addr learned my addr 172.21.15.69:0/2505808883 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.583+0000 7f3902800700 1 -- 172.21.15.69:0/2505808883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390419e350 con 0x7f390407dde0 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.584+0000 7f3902800700 1 --2- 172.21.15.69:0/2505808883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390419de10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f38f4011b60 tx=0x7f38f4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.584+0000 7f38fb400700 1 -- 172.21.15.69:0/2505808883 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38f4003b20 con 0x7f390407dde0 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.584+0000 7f38fb400700 1 -- 172.21.15.69:0/2505808883 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38f4003cc0 con 0x7f390407dde0 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.584+0000 7f38fb400700 1 -- 172.21.15.69:0/2505808883 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38f4024840 con 0x7f390407dde0 2024-03-19T00:58:21.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.584+0000 7f3909400700 1 -- 172.21.15.69:0/2505808883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390419e5e0 con 0x7f390407dde0 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.584+0000 7f3909400700 1 -- 172.21.15.69:0/2505808883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39041a12d0 con 0x7f390407dde0 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.585+0000 7f38fb400700 1 -- 172.21.15.69:0/2505808883 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f38f4022020 con 0x7f390407dde0 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.585+0000 7f38fb400700 1 -- 172.21.15.69:0/2505808883 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f38f4011e00 con 0x7f390407dde0 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.586+0000 7f3909400700 1 -- 172.21.15.69:0/2505808883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3904063940 con 0x7f390407dde0 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.588+0000 7f38fb400700 1 -- 172.21.15.69:0/2505808883 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f38f401a430 con 0x7f390407dde0 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.665+0000 7f3909400700 1 -- 172.21.15.69:0/2505808883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f39041a19b0 con 0x7f390407dde0 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.665+0000 7f38fb400700 1 -- 172.21.15.69:0/2505808883 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f38f4032030 con 0x7f390407dde0 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.667+0000 7f3909400700 1 -- 172.21.15.69:0/2505808883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 msgr2=0x7f390419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.667+0000 7f3909400700 1 --2- 172.21.15.69:0/2505808883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390419de10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f38f4011b60 tx=0x7f38f4011c40 comp rx=0 tx=0).stop 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.667+0000 7f3909400700 1 -- 172.21.15.69:0/2505808883 shutdown_connections 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.667+0000 7f3909400700 1 --2- 172.21.15.69:0/2505808883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f390407dde0 0x7f390419de10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.667+0000 7f3909400700 1 -- 172.21.15.69:0/2505808883 >> 172.21.15.69:0/2505808883 conn(0x7f39041032d0 msgr2=0x7f3904103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.667+0000 7f3909400700 1 -- 172.21.15.69:0/2505808883 shutdown_connections 2024-03-19T00:58:21.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:13.667+0000 7f3909400700 1 -- 172.21.15.69:0/2505808883 wait complete. 2024-03-19T00:58:21.393 INFO:teuthology.orchestra.run.smithi069.stdout:Restarting the monitor... 2024-03-19T00:58:21.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:21 smithi069 systemd[1]: Stopping Ceph mon.a for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T00:58:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:21 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a[31344]: 2024-03-19T00:58:21.807+0000 7fb14e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T00:58:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:21 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a[31344]: 2024-03-19T00:58:21.807+0000 7fb14e000700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-19T00:58:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:21 smithi069 podman[31592]: 2024-03-19 00:58:21.879527578 +0000 UTC m=+0.225439074 container died 18234ad9448d4efdabd5f655a274213f88d42dcc29378be8249e0d24d86c62c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-19T00:58:22.393 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:22 smithi069 podman[31592]: 2024-03-19 00:58:22.233956779 +0000 UTC m=+0.579868326 container cleanup 18234ad9448d4efdabd5f655a274213f88d42dcc29378be8249e0d24d86c62c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:58:22.394 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:22 smithi069 bash[31592]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a 2024-03-19T00:58:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:22 smithi069 podman[31608]: 2024-03-19 00:58:22.392837774 +0000 UTC m=+0.504454344 container remove 18234ad9448d4efdabd5f655a274213f88d42dcc29378be8249e0d24d86c62c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T00:58:23.174 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:22 smithi069 systemd[1]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-19T00:58:23.174 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:22 smithi069 systemd[1]: Stopped Ceph mon.a for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T00:58:23.174 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:22 smithi069 systemd[1]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.a.service: Consumed 1.683s CPU time. 2024-03-19T00:58:23.174 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:22 smithi069 systemd[1]: Starting Ceph mon.a for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T00:58:23.474 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 podman[31708]: 2024-03-19 00:58:23.080301075 +0000 UTC m=+0.032004938 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:58:23.474 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 podman[31708]: 2024-03-19 00:58:23.193625415 +0000 UTC m=+0.145329294 container create d1aee4c67e5b47beb54e6b513c8b123c7ff4ffa556a525e687e5856cfd7f573f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T00:58:23.474 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 podman[31708]: 2024-03-19 00:58:23.472544842 +0000 UTC m=+0.424248768 container init d1aee4c67e5b47beb54e6b513c8b123c7ff4ffa556a525e687e5856cfd7f573f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-19T00:58:23.563 INFO:teuthology.orchestra.run.smithi069.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-19T00:58:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 podman[31708]: 2024-03-19 00:58:23.483111501 +0000 UTC m=+0.434815348 container start d1aee4c67e5b47beb54e6b513c8b123c7ff4ffa556a525e687e5856cfd7f573f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-19T00:58:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: set uid:gid to 167:167 (ceph:ceph) 2024-03-19T00:58:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev), process ceph-mon, pid 2 2024-03-19T00:58:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: pidfile_write: ignore empty --pid-file 2024-03-19T00:58:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: load: jerasure load: lrc 2024-03-19T00:58:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: RocksDB version: 7.9.2 2024-03-19T00:58:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Git sha 0 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Compile date 2024-03-18 18:57:12 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: DB SUMMARY 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: DB Session ID: R710JQ6RYQZE5HQBYNGC 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: CURRENT file: CURRENT 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: IDENTITY file: IDENTITY 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104804 ; 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.error_if_exists: 0 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.create_if_missing: 0 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.paranoid_checks: 1 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-19T00:58:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.env: 0x55e8f2d5fc60 2024-03-19T00:58:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.fs: PosixFileSystem 2024-03-19T00:58:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.info_log: 0x55e8f46adb20 2024-03-19T00:58:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_file_opening_threads: 16 2024-03-19T00:58:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.statistics: (nil) 2024-03-19T00:58:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.use_fsync: 0 2024-03-19T00:58:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_log_file_size: 0 2024-03-19T00:58:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-19T00:58:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-19T00:58:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.keep_log_file_num: 1000 2024-03-19T00:58:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.recycle_log_file_num: 0 2024-03-19T00:58:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.allow_fallocate: 1 2024-03-19T00:58:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.allow_mmap_reads: 0 2024-03-19T00:58:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.allow_mmap_writes: 0 2024-03-19T00:58:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.use_direct_reads: 0 2024-03-19T00:58:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-19T00:58:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.create_missing_column_families: 0 2024-03-19T00:58:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.db_log_dir: 2024-03-19T00:58:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.wal_dir: 2024-03-19T00:58:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-19T00:58:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.advise_random_on_open: 1 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.db_write_buffer_size: 0 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.write_buffer_manager: 0x55e8f46801e0 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.rate_limiter: (nil) 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-19T00:58:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.wal_recovery_mode: 2 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.enable_thread_tracking: 0 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.enable_pipelined_write: 0 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.unordered_write: 0 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.row_cache: None 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.wal_filter: None 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.allow_ingest_behind: 0 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.two_write_queues: 0 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.manual_wal_flush: 0 2024-03-19T00:58:23.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.wal_compression: 0 2024-03-19T00:58:23.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.atomic_flush: 0 2024-03-19T00:58:23.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-19T00:58:23.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-19T00:58:23.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-19T00:58:23.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.log_readahead_size: 0 2024-03-19T00:58:23.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-19T00:58:23.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.best_efforts_recovery: 0 2024-03-19T00:58:23.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-19T00:58:23.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.allow_data_in_errors: 0 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.db_host_id: __hostname__ 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_background_jobs: 2 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_background_compactions: -1 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_subcompactions: 1 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_total_wal_size: 0 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-19T00:58:23.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_open_files: -1 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bytes_per_sync: 0 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_readahead_size: 0 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_background_flushes: -1 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Compression algorithms supported: 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: kZSTD supported: 0 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: kXpressCompression supported: 0 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: kLZ4HCCompression supported: 1 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: kZlibCompression supported: 1 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: kSnappyCompression supported: 1 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: kLZ4Compression supported: 1 2024-03-19T00:58:23.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: kBZip2Compression supported: 0 2024-03-19T00:58:23.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-19T00:58:23.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-19T00:58:23.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-19T00:58:23.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-19T00:58:23.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-19T00:58:23.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.merge_operator: 2024-03-19T00:58:23.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_filter: None 2024-03-19T00:58:23.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_filter_factory: None 2024-03-19T00:58:23.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.sst_partitioner_factory: None 2024-03-19T00:58:23.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-19T00:58:23.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-19T00:58:23.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e8f46adc40) 2024-03-19T00:58:23.794 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks: 1 2024-03-19T00:58:23.794 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_top_level_index_and_filter: 1 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_type: 0 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_index_type: 0 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_shortening: 1 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: checksum: 4 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: no_block_cache: 0 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache: 0x55e8f46b3090 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_name: BinnedLRUCache 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_options: 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: capacity : 536870912 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_shard_bits : 4 2024-03-19T00:58:23.795 INFO:journalctl@ceph.mon.a.smithi069.stdout: strict_capacity_limit : 0 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: high_pri_pool_ratio: 0.000 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_compressed: (nil) 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: persistent_cache: (nil) 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size: 4096 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size_deviation: 10 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_restart_interval: 16 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_block_restart_interval: 1 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: metadata_block_size: 4096 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: partition_filters: 0 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: use_delta_encoding: 1 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: filter_policy: bloomfilter 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: whole_key_filtering: 1 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: verify_compression: 0 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: read_amp_bytes_per_bit: 0 2024-03-19T00:58:23.796 INFO:journalctl@ceph.mon.a.smithi069.stdout: format_version: 5 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_index_compression: 1 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_align: 0 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout: max_auto_readahead_size: 262144 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout: prepopulate_block_cache: 0 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout: initial_auto_readahead_size: 8192 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_file_reads_for_auto_readahead: 2 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.write_buffer_size: 33554432 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_write_buffer_number: 2 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compression: NoCompression 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bottommost_compression: Disabled 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.prefix_extractor: nullptr 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.num_levels: 7 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-19T00:58:23.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compression_opts.level: 32767 2024-03-19T00:58:23.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compression_opts.strategy: 0 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compression_opts.enabled: false 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.target_file_size_base: 67108864 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-19T00:58:23.799 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.arena_block_size: 1048576 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.disable_auto_compactions: 0 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-19T00:58:23.800 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.table_properties_collectors: 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.inplace_update_support: 0 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.bloom_locality: 0 2024-03-19T00:58:23.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.max_successive_merges: 0 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.paranoid_file_checks: 0 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.force_consistency_checks: 1 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.report_bg_io_stats: 0 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.ttl: 2592000 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.enable_blob_files: false 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.min_blob_size: 0 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.blob_file_size: 268435456 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-19T00:58:23.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.blob_file_starting_level: 0 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ae39944b-de80-4a29-9539-761017123881 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710809903541726, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710809903544435, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99963, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97871, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14086, "raw_average_key_size": 49, "raw_value_size": 90213, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710809903, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ae39944b-de80-4a29-9539-761017123881", "db_session_id": "R710JQ6RYQZE5HQBYNGC", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710809903544649, "job": 1, "event": "recovery_finished"} 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-19T00:58:23.803 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e8f47a6000 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: DB pointer 0x55e8f4790000 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** DB Stats ** 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T00:58:23.804 INFO:journalctl@ceph.mon.a.smithi069.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: L0 2/0 99.46 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: Sum 2/0 99.46 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 58.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 58.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-19T00:58:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative compaction: 0.00 GB write, 10.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval compaction: 0.00 GB write, 10.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache BinnedLRUCache@0x55e8f46b3090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: starting mon.a rank 0 at public addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] at bind addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: mon.a@-1(???) e1 preinit fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: mon.a@-1(???).mds e1 new map 2024-03-19T00:58:23.806 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: mon.a@-1(???).mds e1 print_map 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout: e1 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout: legacy client fscid: -1 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout: No filesystems configured 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 bash[31708]: d1aee4c67e5b47beb54e6b513c8b123c7ff4ffa556a525e687e5856cfd7f573f 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 systemd[1]: Started Ceph mon.a for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-19T00:58:23.807 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: monmap epoch 1 2024-03-19T00:58:23.808 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:23.808 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: last_changed 2024-03-19T00:58:05.726813+0000 2024-03-19T00:58:23.808 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: created 2024-03-19T00:58:05.726813+0000 2024-03-19T00:58:23.808 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: min_mon_release 19 (squid) 2024-03-19T00:58:23.808 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: election_strategy: 1 2024-03-19T00:58:23.808 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-03-19T00:58:23.808 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: fsmap 2024-03-19T00:58:23.808 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: osdmap e1: 0 total, 0 up, 0 in 2024-03-19T00:58:23.808 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:23 smithi069 ceph-mon[31727]: mgrmap e1: no daemons active 2024-03-19T00:58:24.934 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.215+0000 7f8bd6600700 1 Processor -- start 2024-03-19T00:58:24.934 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.215+0000 7f8bd6600700 1 -- start start 2024-03-19T00:58:24.934 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.215+0000 7f8bd6600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd0106930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:24.934 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.215+0000 7f8bd6600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd007bed0 con 0x7f8bd0106510 2024-03-19T00:58:24.934 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.216+0000 7f8bcf400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd0106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:24.934 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.216+0000 7f8bcf400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd0106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52812/0 (socket says 172.21.15.69:52812) 2024-03-19T00:58:24.934 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.216+0000 7f8bcf400700 1 -- 172.21.15.69:0/1533665066 learned_addr learned my addr 172.21.15.69:0/1533665066 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:24.935 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.217+0000 7f8bcf400700 1 -- 172.21.15.69:0/1533665066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd0106e70 con 0x7f8bd0106510 2024-03-19T00:58:24.935 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.217+0000 7f8bcf400700 1 --2- 172.21.15.69:0/1533665066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd0106930 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00098a0 tx=0x7f8bc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affb6d8107b96bfe server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:24.935 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.217+0000 7f8bce000700 1 -- 172.21.15.69:0/1533665066 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc0004160 con 0x7f8bd0106510 2024-03-19T00:58:24.935 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.217+0000 7f8bce000700 1 -- 172.21.15.69:0/1533665066 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8bc0004300 con 0x7f8bd0106510 2024-03-19T00:58:24.935 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.217+0000 7f8bce000700 1 -- 172.21.15.69:0/1533665066 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc0011b50 con 0x7f8bd0106510 2024-03-19T00:58:24.935 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.218+0000 7f8bd6600700 1 -- 172.21.15.69:0/1533665066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 msgr2=0x7f8bd0106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:24.938 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.218+0000 7f8bd6600700 1 --2- 172.21.15.69:0/1533665066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd0106930 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00098a0 tx=0x7f8bc0009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:24.938 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.218+0000 7f8bd6600700 1 -- 172.21.15.69:0/1533665066 shutdown_connections 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.218+0000 7f8bd6600700 1 --2- 172.21.15.69:0/1533665066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd0106930 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.218+0000 7f8bd6600700 1 -- 172.21.15.69:0/1533665066 >> 172.21.15.69:0/1533665066 conn(0x7f8bd0102130 msgr2=0x7f8bd0104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.218+0000 7f8bd6600700 1 -- 172.21.15.69:0/1533665066 shutdown_connections 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.218+0000 7f8bd6600700 1 -- 172.21.15.69:0/1533665066 wait complete. 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.219+0000 7f8bd6600700 1 Processor -- start 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.219+0000 7f8bd6600700 1 -- start start 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.220+0000 7f8bd6600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd019de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.220+0000 7f8bd6600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd0107f00 con 0x7f8bd0106510 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.220+0000 7f8bcf400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd019de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.220+0000 7f8bcf400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd019de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52818/0 (socket says 172.21.15.69:52818) 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.220+0000 7f8bcf400700 1 -- 172.21.15.69:0/3770385122 learned_addr learned my addr 172.21.15.69:0/3770385122 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:24.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.220+0000 7f8bcf400700 1 -- 172.21.15.69:0/3770385122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd019e350 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.220+0000 7f8bcf400700 1 --2- 172.21.15.69:0/3770385122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd019de10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0011b60 tx=0x7f8bc0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.221+0000 7f8bc7e00700 1 -- 172.21.15.69:0/3770385122 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc0003b20 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.221+0000 7f8bc7e00700 1 -- 172.21.15.69:0/3770385122 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8bc0003cc0 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.221+0000 7f8bd6600700 1 -- 172.21.15.69:0/3770385122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd019e5e0 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.221+0000 7f8bc7e00700 1 -- 172.21.15.69:0/3770385122 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc0024840 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.221+0000 7f8bd6600700 1 -- 172.21.15.69:0/3770385122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd01a12d0 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.221+0000 7f8bc7e00700 1 -- 172.21.15.69:0/3770385122 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8bc0022020 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.221+0000 7f8bc7e00700 1 -- 172.21.15.69:0/3770385122 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8bc0011e00 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.222+0000 7f8bd6600700 1 -- 172.21.15.69:0/3770385122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd0063940 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.225+0000 7f8bc7e00700 1 -- 172.21.15.69:0/3770385122 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8bc0043dd0 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.299+0000 7f8bd6600700 1 -- 172.21.15.69:0/3770385122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f8bd002de30 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.302+0000 7f8bc7e00700 1 -- 172.21.15.69:0/3770385122 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f8bc0032020 con 0x7f8bd0106510 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.303+0000 7f8bd6600700 1 -- 172.21.15.69:0/3770385122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 msgr2=0x7f8bd019de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.303+0000 7f8bd6600700 1 --2- 172.21.15.69:0/3770385122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd019de10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0011b60 tx=0x7f8bc0011c40 comp rx=0 tx=0).stop 2024-03-19T00:58:24.940 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.303+0000 7f8bd6600700 1 -- 172.21.15.69:0/3770385122 shutdown_connections 2024-03-19T00:58:24.941 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.303+0000 7f8bd6600700 1 --2- 172.21.15.69:0/3770385122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bd0106510 0x7f8bd019de10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:24.941 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.303+0000 7f8bd6600700 1 -- 172.21.15.69:0/3770385122 >> 172.21.15.69:0/3770385122 conn(0x7f8bd0102130 msgr2=0x7f8bd0194a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:24.941 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.304+0000 7f8bd6600700 1 -- 172.21.15.69:0/3770385122 shutdown_connections 2024-03-19T00:58:24.941 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:24.304+0000 7f8bd6600700 1 -- 172.21.15.69:0/3770385122 wait complete. 2024-03-19T00:58:24.941 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-19T00:58:24.960 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:58:24.960 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mgr... 2024-03-19T00:58:24.962 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-19T00:58:24.963 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-19T00:58:25.401 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.x 2024-03-19T00:58:25.401 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.x.service: Unit ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.x.service not loaded. 2024-03-19T00:58:25.651 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-934de1c2-e58b-11ee-95c9-87774f69a715.target.wants/ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-934de1c2-e58b-11ee-95c9-87774f69a715@.service. 2024-03-19T00:58:25.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:25 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3770385122' entity='client.admin' 2024-03-19T00:58:26.029 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:25 smithi069 systemd[1]: Starting Ceph mgr.x for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T00:58:26.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:26 smithi069 podman[31937]: 2024-03-19 00:58:25.934369402 +0000 UTC m=+0.036716129 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:58:26.283 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:26 smithi069 podman[31937]: 2024-03-19 00:58:26.045424633 +0000 UTC m=+0.147771392 container create 1507fbc37617fcd688631aefa50718b9a927a1c425a7b365bb70e2a23aa0a8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T00:58:26.388 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T00:58:26.388 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-03-19T00:58:26.405 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T00:58:26.405 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-03-19T00:58:26.405 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-03-19T00:58:26.405 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2024-03-19T00:58:26.423 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T00:58:26.424 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-03-19T00:58:26.439 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T00:58:26.440 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-03-19T00:58:26.440 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-03-19T00:58:26.440 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-19T00:58:26.440 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr to start... 2024-03-19T00:58:26.440 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr... 2024-03-19T00:58:26.592 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:26 smithi069 podman[31937]: 2024-03-19 00:58:26.282728935 +0000 UTC m=+0.385075751 container init 1507fbc37617fcd688631aefa50718b9a927a1c425a7b365bb70e2a23aa0a8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T00:58:26.592 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:26 smithi069 podman[31937]: 2024-03-19 00:58:26.293276041 +0000 UTC m=+0.395622807 container start 1507fbc37617fcd688631aefa50718b9a927a1c425a7b365bb70e2a23aa0a8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-19T00:58:26.592 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:26 smithi069 bash[31937]: 1507fbc37617fcd688631aefa50718b9a927a1c425a7b365bb70e2a23aa0a8b2 2024-03-19T00:58:26.592 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:26 smithi069 systemd[1]: Started Ceph mgr.x for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T00:58:27.148 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:27 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:27.049+0000 7fc044249200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T00:58:27.482 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:27 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/269274218' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T00:58:27.483 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:27 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:27.174+0000 7fc044249200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T00:58:27.756 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:27.756 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:27.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-19T00:58:27.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T00:58:27.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:27.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T00:58:27.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T00:58:27.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:27.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T00:58:27.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:27.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T00:58:27.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T00:58:27.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T00:58:27.766 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T00:58:27.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T00:58:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T00:58:09.792860+0000", 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.041+0000 7f3c4a800700 1 Processor -- start 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.041+0000 7f3c4a800700 1 -- start start 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.042+0000 7f3c4a800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.042+0000 7f3c4a800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4407c840 con 0x7f3c4407de00 2024-03-19T00:58:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.042+0000 7f3c43400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.042+0000 7f3c43400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60012/0 (socket says 172.21.15.69:60012) 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.042+0000 7f3c43400700 1 -- 172.21.15.69:0/1376413645 learned_addr learned my addr 172.21.15.69:0/1376413645 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.043+0000 7f3c43400700 1 -- 172.21.15.69:0/1376413645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4407c9c0 con 0x7f3c4407de00 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.043+0000 7f3c43400700 1 --2- 172.21.15.69:0/1376413645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4407c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3c340098a0 tx=0x7f3c34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d8335fe8cbc071 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.043+0000 7f3c42000700 1 -- 172.21.15.69:0/1376413645 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c34004160 con 0x7f3c4407de00 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.043+0000 7f3c42000700 1 -- 172.21.15.69:0/1376413645 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3c34004300 con 0x7f3c4407de00 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.043+0000 7f3c42000700 1 -- 172.21.15.69:0/1376413645 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c34011b50 con 0x7f3c4407de00 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.044+0000 7f3c4a800700 1 -- 172.21.15.69:0/1376413645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 msgr2=0x7f3c4407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.044+0000 7f3c4a800700 1 --2- 172.21.15.69:0/1376413645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4407c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3c340098a0 tx=0x7f3c34009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.044+0000 7f3c4a800700 1 -- 172.21.15.69:0/1376413645 shutdown_connections 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.044+0000 7f3c4a800700 1 --2- 172.21.15.69:0/1376413645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4407c300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.044+0000 7f3c4a800700 1 -- 172.21.15.69:0/1376413645 >> 172.21.15.69:0/1376413645 conn(0x7f3c441032d0 msgr2=0x7f3c441056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:27.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.044+0000 7f3c4a800700 1 -- 172.21.15.69:0/1376413645 shutdown_connections 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.044+0000 7f3c4a800700 1 -- 172.21.15.69:0/1376413645 wait complete. 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.045+0000 7f3c4a800700 1 Processor -- start 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.045+0000 7f3c4a800700 1 -- start start 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.046+0000 7f3c4a800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.046+0000 7f3c4a800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4410e280 con 0x7f3c4407de00 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.046+0000 7f3c43400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.046+0000 7f3c43400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60020/0 (socket says 172.21.15.69:60020) 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.046+0000 7f3c43400700 1 -- 172.21.15.69:0/269274218 learned_addr learned my addr 172.21.15.69:0/269274218 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.046+0000 7f3c43400700 1 -- 172.21.15.69:0/269274218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4419e340 con 0x7f3c4407de00 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.047+0000 7f3c43400700 1 --2- 172.21.15.69:0/269274218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4419de00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3c34011b60 tx=0x7f3c34011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.047+0000 7f3c3be00700 1 -- 172.21.15.69:0/269274218 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c34003b20 con 0x7f3c4407de00 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.047+0000 7f3c3be00700 1 -- 172.21.15.69:0/269274218 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3c34003cc0 con 0x7f3c4407de00 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.047+0000 7f3c4a800700 1 -- 172.21.15.69:0/269274218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4419e5d0 con 0x7f3c4407de00 2024-03-19T00:58:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.047+0000 7f3c3be00700 1 -- 172.21.15.69:0/269274218 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c34024840 con 0x7f3c4407de00 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.047+0000 7f3c4a800700 1 -- 172.21.15.69:0/269274218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c441a12c0 con 0x7f3c4407de00 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.047+0000 7f3c3be00700 1 -- 172.21.15.69:0/269274218 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c34022020 con 0x7f3c4407de00 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.047+0000 7f3c3be00700 1 -- 172.21.15.69:0/269274218 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3c34011e00 con 0x7f3c4407de00 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.048+0000 7f3c4a800700 1 -- 172.21.15.69:0/269274218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c44063940 con 0x7f3c4407de00 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.052+0000 7f3c3be00700 1 -- 172.21.15.69:0/269274218 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3c34043dd0 con 0x7f3c4407de00 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.130+0000 7f3c4a800700 1 -- 172.21.15.69:0/269274218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3c441a19a0 con 0x7f3c4407de00 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.130+0000 7f3c3be00700 1 -- 172.21.15.69:0/269274218 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3c34032020 con 0x7f3c4407de00 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.131+0000 7f3c4a800700 1 -- 172.21.15.69:0/269274218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 msgr2=0x7f3c4419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.131+0000 7f3c4a800700 1 --2- 172.21.15.69:0/269274218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4419de00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3c34011b60 tx=0x7f3c34011c40 comp rx=0 tx=0).stop 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.131+0000 7f3c4a800700 1 -- 172.21.15.69:0/269274218 shutdown_connections 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.131+0000 7f3c4a800700 1 --2- 172.21.15.69:0/269274218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c4407de00 0x7f3c4419de00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.131+0000 7f3c4a800700 1 -- 172.21.15.69:0/269274218 >> 172.21.15.69:0/269274218 conn(0x7f3c441032d0 msgr2=0x7f3c44103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:27.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.132+0000 7f3c4a800700 1 -- 172.21.15.69:0/269274218 shutdown_connections 2024-03-19T00:58:27.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:27.132+0000 7f3c4a800700 1 -- 172.21.15.69:0/269274218 wait complete. 2024-03-19T00:58:27.773 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (1/15)... 2024-03-19T00:58:28.034 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:27 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:27.989+0000 7fc044249200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T00:58:28.404 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:28 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:28.109+0000 7fc044249200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T00:58:28.405 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:28 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:28.282+0000 7fc044249200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T00:58:28.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:28 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:28.403+0000 7fc044249200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T00:58:28.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:28 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:28.510+0000 7fc044249200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T00:58:29.208 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:28 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:28.837+0000 7fc044249200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T00:58:29.208 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:28 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:28.975+0000 7fc044249200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T00:58:29.208 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:29 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:29.086+0000 7fc044249200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T00:58:29.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:29 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:29.206+0000 7fc044249200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T00:58:29.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:29 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:29.419+0000 7fc044249200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T00:58:30.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:30 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3999883498' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T00:58:31.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:30.742+0000 7fc044249200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T00:58:31.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:30.915+0000 7fc044249200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T00:58:31.244 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:31.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:58:31.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T00:58:31.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T00:58:31.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T00:58:31.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T00:58:31.245 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T00:58:31.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:31.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T00:58:31.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T00:58:31.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-03-19T00:58:31.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:31.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T00:58:31.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T00:58:31.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:31.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-19T00:58:31.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T00:58:31.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:31.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T00:58:31.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T00:58:31.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:31.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T00:58:31.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:31.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T00:58:31.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T00:58:31.257 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T00:58:31.257 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T00:58:31.257 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T00:58:31.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:31.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T00:58:09.792860+0000", 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.269+0000 7f8295e00700 1 Processor -- start 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.270+0000 7f8295e00700 1 -- start start 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.270+0000 7f8295e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f8290107f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.270+0000 7f8295e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8290108500 con 0x7f8290107b10 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.270+0000 7f8294a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f8290107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.270+0000 7f8294a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f8290107f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60022/0 (socket says 172.21.15.69:60022) 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.270+0000 7f8294a00700 1 -- 172.21.15.69:0/3087458196 learned_addr learned my addr 172.21.15.69:0/3087458196 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.271+0000 7f8294a00700 1 -- 172.21.15.69:0/3087458196 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8290108cc0 con 0x7f8290107b10 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.271+0000 7f8294a00700 1 --2- 172.21.15.69:0/3087458196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f8290107f30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f82800098a0 tx=0x7f8280009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b69c685ad511be server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.271+0000 7f828fe00700 1 -- 172.21.15.69:0/3087458196 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8280004160 con 0x7f8290107b10 2024-03-19T00:58:31.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.271+0000 7f828fe00700 1 -- 172.21.15.69:0/3087458196 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8280004300 con 0x7f8290107b10 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.271+0000 7f828fe00700 1 -- 172.21.15.69:0/3087458196 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8280011b50 con 0x7f8290107b10 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.272+0000 7f8295e00700 1 -- 172.21.15.69:0/3087458196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 msgr2=0x7f8290107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.272+0000 7f8295e00700 1 --2- 172.21.15.69:0/3087458196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f8290107f30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f82800098a0 tx=0x7f8280009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.272+0000 7f8295e00700 1 -- 172.21.15.69:0/3087458196 shutdown_connections 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.272+0000 7f8295e00700 1 --2- 172.21.15.69:0/3087458196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f8290107f30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.272+0000 7f8295e00700 1 -- 172.21.15.69:0/3087458196 >> 172.21.15.69:0/3087458196 conn(0x7f82901030d0 msgr2=0x7f82901054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.272+0000 7f8295e00700 1 -- 172.21.15.69:0/3087458196 shutdown_connections 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.272+0000 7f8295e00700 1 -- 172.21.15.69:0/3087458196 wait complete. 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.274+0000 7f8295e00700 1 Processor -- start 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.274+0000 7f8295e00700 1 -- start start 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.274+0000 7f8295e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f829019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.274+0000 7f8295e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f829010c070 con 0x7f8290107b10 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.275+0000 7f8294a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f829019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:31.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.275+0000 7f8294a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f829019bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60036/0 (socket says 172.21.15.69:60036) 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.275+0000 7f8294a00700 1 -- 172.21.15.69:0/3999883498 learned_addr learned my addr 172.21.15.69:0/3999883498 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.275+0000 7f8294a00700 1 -- 172.21.15.69:0/3999883498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829019bff0 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.275+0000 7f8294a00700 1 --2- 172.21.15.69:0/3999883498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f829019bab0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8280000c00 tx=0x7f8280011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.275+0000 7f828e000700 1 -- 172.21.15.69:0/3999883498 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8280003b20 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.276+0000 7f828e000700 1 -- 172.21.15.69:0/3999883498 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8280003cc0 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.276+0000 7f8295e00700 1 -- 172.21.15.69:0/3999883498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829019c280 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.276+0000 7f828e000700 1 -- 172.21.15.69:0/3999883498 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8280024910 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.276+0000 7f8295e00700 1 -- 172.21.15.69:0/3999883498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829019ef70 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.276+0000 7f828e000700 1 -- 172.21.15.69:0/3999883498 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8280022020 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.276+0000 7f828e000700 1 -- 172.21.15.69:0/3999883498 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8280011e00 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.277+0000 7f8295e00700 1 -- 172.21.15.69:0/3999883498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8290063940 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.280+0000 7f828e000700 1 -- 172.21.15.69:0/3999883498 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f828001acd0 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.357+0000 7f8295e00700 1 -- 172.21.15.69:0/3999883498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f829019f4b0 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.358+0000 7f828e000700 1 -- 172.21.15.69:0/3999883498 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8280032030 con 0x7f8290107b10 2024-03-19T00:58:31.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.359+0000 7f8295e00700 1 -- 172.21.15.69:0/3999883498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 msgr2=0x7f829019bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:31.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.359+0000 7f8295e00700 1 --2- 172.21.15.69:0/3999883498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f829019bab0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8280000c00 tx=0x7f8280011c10 comp rx=0 tx=0).stop 2024-03-19T00:58:31.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.359+0000 7f8295e00700 1 -- 172.21.15.69:0/3999883498 shutdown_connections 2024-03-19T00:58:31.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.359+0000 7f8295e00700 1 --2- 172.21.15.69:0/3999883498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8290107b10 0x7f829019bab0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:31.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.359+0000 7f8295e00700 1 -- 172.21.15.69:0/3999883498 >> 172.21.15.69:0/3999883498 conn(0x7f82901030d0 msgr2=0x7f8290103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:31.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.359+0000 7f8295e00700 1 -- 172.21.15.69:0/3999883498 shutdown_connections 2024-03-19T00:58:31.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:30.359+0000 7f8295e00700 1 -- 172.21.15.69:0/3999883498 wait complete. 2024-03-19T00:58:31.263 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (2/15)... 2024-03-19T00:58:31.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:31.328+0000 7fc044249200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T00:58:31.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:31.443+0000 7fc044249200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T00:58:32.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:31.688+0000 7fc044249200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:58:32.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:31.922+0000 7fc044249200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T00:58:32.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:32.280+0000 7fc044249200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:58:32.913 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:32.614+0000 7fc044249200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T00:58:32.913 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:32.911+0000 7fc044249200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T00:58:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:33 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1051276101' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T00:58:34.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:33 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:33.951+0000 7fc044249200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T00:58:34.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:34.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:58:34.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T00:58:34.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T00:58:34.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T00:58:34.632 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T00:58:34.632 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T00:58:34.632 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:34.632 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T00:58:34.632 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T00:58:34.632 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-03-19T00:58:34.632 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T00:58:34.633 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T00:58:09.792860+0000", 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.814+0000 7f047ec00700 1 Processor -- start 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.814+0000 7f047ec00700 1 -- start start 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.815+0000 7f047ec00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.815+0000 7f047ec00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f047807c840 con 0x7f047807de50 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.815+0000 7f047d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.815+0000 7f047d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60046/0 (socket says 172.21.15.69:60046) 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.815+0000 7f047d800700 1 -- 172.21.15.69:0/4177506653 learned_addr learned my addr 172.21.15.69:0/4177506653 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.816+0000 7f047d800700 1 -- 172.21.15.69:0/4177506653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047807c9c0 con 0x7f047807de50 2024-03-19T00:58:34.636 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.816+0000 7f047d800700 1 --2- 172.21.15.69:0/4177506653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047807c300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f046c0098a0 tx=0x7f046c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eece4c0a98a4ac2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.816+0000 7f0477e00700 1 -- 172.21.15.69:0/4177506653 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c004160 con 0x7f047807de50 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.816+0000 7f0477e00700 1 -- 172.21.15.69:0/4177506653 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f046c004300 con 0x7f047807de50 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.816+0000 7f0477e00700 1 -- 172.21.15.69:0/4177506653 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c011b50 con 0x7f047807de50 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.817+0000 7f047ec00700 1 -- 172.21.15.69:0/4177506653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 msgr2=0x7f047807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.817+0000 7f047ec00700 1 --2- 172.21.15.69:0/4177506653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047807c300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f046c0098a0 tx=0x7f046c009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.817+0000 7f047ec00700 1 -- 172.21.15.69:0/4177506653 shutdown_connections 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.817+0000 7f047ec00700 1 --2- 172.21.15.69:0/4177506653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047807c300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.817+0000 7f047ec00700 1 -- 172.21.15.69:0/4177506653 >> 172.21.15.69:0/4177506653 conn(0x7f04781030d0 msgr2=0x7f04781054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.817+0000 7f047ec00700 1 -- 172.21.15.69:0/4177506653 shutdown_connections 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.817+0000 7f047ec00700 1 -- 172.21.15.69:0/4177506653 wait complete. 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.818+0000 7f047ec00700 1 Processor -- start 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.818+0000 7f047ec00700 1 -- start start 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.819+0000 7f047ec00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.819+0000 7f047ec00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f047810e0e0 con 0x7f047807de50 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.819+0000 7f047d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.819+0000 7f047d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047819dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60062/0 (socket says 172.21.15.69:60062) 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.819+0000 7f047d800700 1 -- 172.21.15.69:0/1051276101 learned_addr learned my addr 172.21.15.69:0/1051276101 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.819+0000 7f047d800700 1 -- 172.21.15.69:0/1051276101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047819e1b0 con 0x7f047807de50 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.820+0000 7f047d800700 1 --2- 172.21.15.69:0/1051276101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047819dc70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f046c011b60 tx=0x7f046c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.820+0000 7f0476000700 1 -- 172.21.15.69:0/1051276101 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c003b20 con 0x7f047807de50 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.820+0000 7f0476000700 1 -- 172.21.15.69:0/1051276101 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f046c003cc0 con 0x7f047807de50 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.820+0000 7f0476000700 1 -- 172.21.15.69:0/1051276101 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046c024840 con 0x7f047807de50 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.820+0000 7f047ec00700 1 -- 172.21.15.69:0/1051276101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047819e440 con 0x7f047807de50 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.820+0000 7f047ec00700 1 -- 172.21.15.69:0/1051276101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04781a1130 con 0x7f047807de50 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.820+0000 7f0476000700 1 -- 172.21.15.69:0/1051276101 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f046c022020 con 0x7f047807de50 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.820+0000 7f0476000700 1 -- 172.21.15.69:0/1051276101 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f046c011e00 con 0x7f047807de50 2024-03-19T00:58:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.821+0000 7f047ec00700 1 -- 172.21.15.69:0/1051276101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0478063940 con 0x7f047807de50 2024-03-19T00:58:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.824+0000 7f0476000700 1 -- 172.21.15.69:0/1051276101 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f046c043dd0 con 0x7f047807de50 2024-03-19T00:58:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.902+0000 7f047ec00700 1 -- 172.21.15.69:0/1051276101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f04781a1750 con 0x7f047807de50 2024-03-19T00:58:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.903+0000 7f0476000700 1 -- 172.21.15.69:0/1051276101 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f046c032020 con 0x7f047807de50 2024-03-19T00:58:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.905+0000 7f047ec00700 1 -- 172.21.15.69:0/1051276101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 msgr2=0x7f047819dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.905+0000 7f047ec00700 1 --2- 172.21.15.69:0/1051276101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047819dc70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f046c011b60 tx=0x7f046c011c40 comp rx=0 tx=0).stop 2024-03-19T00:58:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.905+0000 7f047ec00700 1 -- 172.21.15.69:0/1051276101 shutdown_connections 2024-03-19T00:58:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.905+0000 7f047ec00700 1 --2- 172.21.15.69:0/1051276101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f047807de50 0x7f047819dc70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.905+0000 7f047ec00700 1 -- 172.21.15.69:0/1051276101 >> 172.21.15.69:0/1051276101 conn(0x7f04781030d0 msgr2=0x7f0478103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.905+0000 7f047ec00700 1 -- 172.21.15.69:0/1051276101 shutdown_connections 2024-03-19T00:58:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:33.905+0000 7f047ec00700 1 -- 172.21.15.69:0/1051276101 wait complete. 2024-03-19T00:58:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (3/15)... 2024-03-19T00:58:35.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:34 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:34.649+0000 7fc044249200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T00:58:35.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:34 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:34.776+0000 7fc044249200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T00:58:35.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:35.048+0000 7fc044249200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T00:58:35.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: Activating manager daemon x 2024-03-19T00:58:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: mgrmap e2: x(active, starting, since 0.0106796s) 2024-03-19T00:58:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: from='mgr.14100 172.21.15.69:0/2604335182' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T00:58:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: from='mgr.14100 172.21.15.69:0/2604335182' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T00:58:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: from='mgr.14100 172.21.15.69:0/2604335182' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T00:58:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: from='mgr.14100 172.21.15.69:0/2604335182' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:58:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: from='mgr.14100 172.21.15.69:0/2604335182' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T00:58:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: Manager daemon x is now available 2024-03-19T00:58:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: from='mgr.14100 172.21.15.69:0/2604335182' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T00:58:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: from='mgr.14100 172.21.15.69:0/2604335182' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T00:58:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: from='mgr.14100 172.21.15.69:0/2604335182' entity='mgr.x' 2024-03-19T00:58:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:35 smithi069 ceph-mon[31727]: from='mgr.14100 172.21.15.69:0/2604335182' entity='mgr.x' 2024-03-19T00:58:36.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:36 smithi069 ceph-mon[31727]: from='mgr.14100 172.21.15.69:0/2604335182' entity='mgr.x' 2024-03-19T00:58:36.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:36 smithi069 ceph-mon[31727]: mgrmap e3: x(active, since 1.0203s) 2024-03-19T00:58:38.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:38 smithi069 ceph-mon[31727]: mgrmap e4: x(active, since 2s) 2024-03-19T00:58:38.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:38 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2970238897' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T00:58:38.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:38.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:58:38.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T00:58:38.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T00:58:38.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T00:58:38.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T00:58:38.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T00:58:38.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:38.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T00:58:38.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T00:58:38.239 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T00:58:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T00:58:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T00:58:09.792860+0000", 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.237+0000 7f07ce600700 1 Processor -- start 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.237+0000 7f07ce600700 1 -- start start 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.237+0000 7f07ce600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c8108140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.237+0000 7f07ce600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8108710 con 0x7f07c8107d20 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.238+0000 7f07c7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c8108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.238+0000 7f07c7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c8108140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55066/0 (socket says 172.21.15.69:55066) 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.238+0000 7f07c7400700 1 -- 172.21.15.69:0/3007683387 learned_addr learned my addr 172.21.15.69:0/3007683387 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.238+0000 7f07c7400700 1 -- 172.21.15.69:0/3007683387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c8108e80 con 0x7f07c8107d20 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.239+0000 7f07c7400700 1 --2- 172.21.15.69:0/3007683387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c8108140 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f07b80098a0 tx=0x7f07b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea78c6c914a613bf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.239+0000 7f07c6000700 1 -- 172.21.15.69:0/3007683387 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07b8004160 con 0x7f07c8107d20 2024-03-19T00:58:38.243 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.239+0000 7f07c6000700 1 -- 172.21.15.69:0/3007683387 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07b8004300 con 0x7f07c8107d20 2024-03-19T00:58:38.243 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.239+0000 7f07c6000700 1 -- 172.21.15.69:0/3007683387 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07b8011b50 con 0x7f07c8107d20 2024-03-19T00:58:38.243 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.239+0000 7f07ce600700 1 -- 172.21.15.69:0/3007683387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 msgr2=0x7f07c8108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:38.243 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.239+0000 7f07ce600700 1 --2- 172.21.15.69:0/3007683387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c8108140 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f07b80098a0 tx=0x7f07b8009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:38.243 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.240+0000 7f07ce600700 1 -- 172.21.15.69:0/3007683387 shutdown_connections 2024-03-19T00:58:38.243 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.240+0000 7f07ce600700 1 --2- 172.21.15.69:0/3007683387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c8108140 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:38.243 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.240+0000 7f07ce600700 1 -- 172.21.15.69:0/3007683387 >> 172.21.15.69:0/3007683387 conn(0x7f07c81032e0 msgr2=0x7f07c8105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:38.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.240+0000 7f07ce600700 1 -- 172.21.15.69:0/3007683387 shutdown_connections 2024-03-19T00:58:38.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.240+0000 7f07ce600700 1 -- 172.21.15.69:0/3007683387 wait complete. 2024-03-19T00:58:38.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.241+0000 7f07ce600700 1 Processor -- start 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.241+0000 7f07ce600700 1 -- start start 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.241+0000 7f07ce600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c807e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.241+0000 7f07ce600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07c810c230 con 0x7f07c8107d20 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.242+0000 7f07c7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c807e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.242+0000 7f07c7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c807e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55068/0 (socket says 172.21.15.69:55068) 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.242+0000 7f07c7400700 1 -- 172.21.15.69:0/2970238897 learned_addr learned my addr 172.21.15.69:0/2970238897 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.242+0000 7f07c7400700 1 -- 172.21.15.69:0/2970238897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c807e960 con 0x7f07c8107d20 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.242+0000 7f07c7400700 1 --2- 172.21.15.69:0/2970238897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c807e420 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f07b8011670 tx=0x7f07b8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.243+0000 7f07bfe00700 1 -- 172.21.15.69:0/2970238897 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07b80039c0 con 0x7f07c8107d20 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.243+0000 7f07bfe00700 1 -- 172.21.15.69:0/2970238897 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07b8011e50 con 0x7f07c8107d20 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.243+0000 7f07bfe00700 1 -- 172.21.15.69:0/2970238897 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07b801a5c0 con 0x7f07c8107d20 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.243+0000 7f07ce600700 1 -- 172.21.15.69:0/2970238897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c807cb50 con 0x7f07c8107d20 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.243+0000 7f07ce600700 1 -- 172.21.15.69:0/2970238897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c807d090 con 0x7f07c8107d20 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.244+0000 7f07bfe00700 1 -- 172.21.15.69:0/2970238897 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f07b8024430 con 0x7f07c8107d20 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.244+0000 7f07ce600700 1 -- 172.21.15.69:0/2970238897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c8063940 con 0x7f07c8107d20 2024-03-19T00:58:38.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.244+0000 7f07bfe00700 1 --2- 172.21.15.69:0/2970238897 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f07ac038ed0 0x7f07ac03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.245+0000 7f07bfe00700 1 -- 172.21.15.69:0/2970238897 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f07b804d1b0 con 0x7f07c8107d20 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.245+0000 7f07c6a00700 1 --2- 172.21.15.69:0/2970238897 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f07ac038ed0 0x7f07ac03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.245+0000 7f07c6a00700 1 --2- 172.21.15.69:0/2970238897 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f07ac038ed0 0x7f07ac03b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f07b4006fd0 tx=0x7f07b4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.251+0000 7f07bfe00700 1 -- 172.21.15.69:0/2970238897 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f07b8017ca0 con 0x7f07c8107d20 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.555+0000 7f07ce600700 1 -- 172.21.15.69:0/2970238897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f07c807d310 con 0x7f07c8107d20 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.556+0000 7f07bfe00700 1 -- 172.21.15.69:0/2970238897 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f07b804cbd0 con 0x7f07c8107d20 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.559+0000 7f07ce600700 1 -- 172.21.15.69:0/2970238897 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f07ac038ed0 msgr2=0x7f07ac03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.559+0000 7f07ce600700 1 --2- 172.21.15.69:0/2970238897 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f07ac038ed0 0x7f07ac03b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f07b4006fd0 tx=0x7f07b4006e40 comp rx=0 tx=0).stop 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.559+0000 7f07ce600700 1 -- 172.21.15.69:0/2970238897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 msgr2=0x7f07c807e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.559+0000 7f07ce600700 1 --2- 172.21.15.69:0/2970238897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c807e420 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f07b8011670 tx=0x7f07b8011b60 comp rx=0 tx=0).stop 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.559+0000 7f07ce600700 1 -- 172.21.15.69:0/2970238897 shutdown_connections 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.559+0000 7f07ce600700 1 --2- 172.21.15.69:0/2970238897 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f07ac038ed0 0x7f07ac03b390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.559+0000 7f07ce600700 1 --2- 172.21.15.69:0/2970238897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8107d20 0x7f07c807e420 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.559+0000 7f07ce600700 1 -- 172.21.15.69:0/2970238897 >> 172.21.15.69:0/2970238897 conn(0x7f07c81032e0 msgr2=0x7f07c8103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:38.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.559+0000 7f07ce600700 1 -- 172.21.15.69:0/2970238897 shutdown_connections 2024-03-19T00:58:38.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:37.559+0000 7f07ce600700 1 -- 172.21.15.69:0/2970238897 wait complete. 2024-03-19T00:58:38.255 INFO:teuthology.orchestra.run.smithi069.stdout:mgr is available 2024-03-19T00:58:39.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:39 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3788084958' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-19T00:58:39.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:39.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2024-03-19T00:58:39.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:39.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-19T00:58:39.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-19T00:58:39.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-19T00:58:39.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-19T00:58:39.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:39.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-19T00:58:39.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-19T00:58:39.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-03-19T00:58:39.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.822+0000 7faef8800700 1 Processor -- start 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.822+0000 7faef8800700 1 -- start start 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.823+0000 7faef8800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef007c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.823+0000 7faef8800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faef007c8a0 con 0x7faef007de60 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.823+0000 7faef5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef007c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.823+0000 7faef5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef007c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55076/0 (socket says 172.21.15.69:55076) 2024-03-19T00:58:39.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.823+0000 7faef5e00700 1 -- 172.21.15.69:0/3111834658 learned_addr learned my addr 172.21.15.69:0/3111834658 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.824+0000 7faef5e00700 1 -- 172.21.15.69:0/3111834658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef007ca20 con 0x7faef007de60 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.824+0000 7faef5e00700 1 --2- 172.21.15.69:0/3111834658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef007c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faee40098a0 tx=0x7faee4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdd00642f196010 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.824+0000 7faef4a00700 1 -- 172.21.15.69:0/3111834658 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee4004160 con 0x7faef007de60 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.824+0000 7faef4a00700 1 -- 172.21.15.69:0/3111834658 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faee4004300 con 0x7faef007de60 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.824+0000 7faef4a00700 1 -- 172.21.15.69:0/3111834658 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee4011b50 con 0x7faef007de60 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.825+0000 7faef8800700 1 -- 172.21.15.69:0/3111834658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 msgr2=0x7faef007c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.825+0000 7faef8800700 1 --2- 172.21.15.69:0/3111834658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef007c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faee40098a0 tx=0x7faee4009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.825+0000 7faef8800700 1 -- 172.21.15.69:0/3111834658 shutdown_connections 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.825+0000 7faef8800700 1 --2- 172.21.15.69:0/3111834658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef007c360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.825+0000 7faef8800700 1 -- 172.21.15.69:0/3111834658 >> 172.21.15.69:0/3111834658 conn(0x7faef01032d0 msgr2=0x7faef01056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.825+0000 7faef8800700 1 -- 172.21.15.69:0/3111834658 shutdown_connections 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.825+0000 7faef8800700 1 -- 172.21.15.69:0/3111834658 wait complete. 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.826+0000 7faef8800700 1 Processor -- start 2024-03-19T00:58:39.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.826+0000 7faef8800700 1 -- start start 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.826+0000 7faef8800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef01a2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.826+0000 7faef8800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faef0109e70 con 0x7faef007de60 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.827+0000 7faef5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef01a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.827+0000 7faef5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef01a2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55092/0 (socket says 172.21.15.69:55092) 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.827+0000 7faef5e00700 1 -- 172.21.15.69:0/3788084958 learned_addr learned my addr 172.21.15.69:0/3788084958 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.827+0000 7faef5e00700 1 -- 172.21.15.69:0/3788084958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef01a2740 con 0x7faef007de60 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.827+0000 7faef5e00700 1 --2- 172.21.15.69:0/3788084958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef01a2200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faee4011670 tx=0x7faee4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.827+0000 7faeeaa00700 1 -- 172.21.15.69:0/3788084958 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee40039c0 con 0x7faef007de60 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.828+0000 7faef8800700 1 -- 172.21.15.69:0/3788084958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef01a29d0 con 0x7faef007de60 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.828+0000 7faeeaa00700 1 -- 172.21.15.69:0/3788084958 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faee4011e50 con 0x7faef007de60 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.828+0000 7faeeaa00700 1 -- 172.21.15.69:0/3788084958 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee401a5c0 con 0x7faef007de60 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.828+0000 7faef8800700 1 -- 172.21.15.69:0/3788084958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef01a56c0 con 0x7faef007de60 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.829+0000 7faeeaa00700 1 -- 172.21.15.69:0/3788084958 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7faee401a8c0 con 0x7faef007de60 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.829+0000 7faef8800700 1 -- 172.21.15.69:0/3788084958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef0063940 con 0x7faef007de60 2024-03-19T00:58:39.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.829+0000 7faeeaa00700 1 --2- 172.21.15.69:0/3788084958 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7faed8038e80 0x7faed803b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.829+0000 7faeeaa00700 1 -- 172.21.15.69:0/3788084958 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faee404d3a0 con 0x7faef007de60 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.829+0000 7faef5400700 1 --2- 172.21.15.69:0/3788084958 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7faed8038e80 0x7faed803b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.830+0000 7faef5400700 1 --2- 172.21.15.69:0/3788084958 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7faed8038e80 0x7faed803b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faee0006fd0 tx=0x7faee0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:38.835+0000 7faeeaa00700 1 -- 172.21.15.69:0/3788084958 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faee404cdc0 con 0x7faef007de60 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.056+0000 7faef8800700 1 -- 172.21.15.69:0/3788084958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7faef00008d0 con 0x7faef007de60 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.059+0000 7faeeaa00700 1 -- 172.21.15.69:0/3788084958 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7faee402d910 con 0x7faef007de60 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.062+0000 7faef8800700 1 -- 172.21.15.69:0/3788084958 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7faed8038e80 msgr2=0x7faed803b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.062+0000 7faef8800700 1 --2- 172.21.15.69:0/3788084958 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7faed8038e80 0x7faed803b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faee0006fd0 tx=0x7faee0006e40 comp rx=0 tx=0).stop 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.062+0000 7faef8800700 1 -- 172.21.15.69:0/3788084958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 msgr2=0x7faef01a2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.062+0000 7faef8800700 1 --2- 172.21.15.69:0/3788084958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef01a2200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faee4011670 tx=0x7faee4011b60 comp rx=0 tx=0).stop 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.062+0000 7faef8800700 1 -- 172.21.15.69:0/3788084958 shutdown_connections 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.062+0000 7faef8800700 1 --2- 172.21.15.69:0/3788084958 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7faed8038e80 0x7faed803b340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.062+0000 7faef8800700 1 --2- 172.21.15.69:0/3788084958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef007de60 0x7faef01a2200 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:39.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.062+0000 7faef8800700 1 -- 172.21.15.69:0/3788084958 >> 172.21.15.69:0/3788084958 conn(0x7faef01032d0 msgr2=0x7faef01056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:39.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.062+0000 7faef8800700 1 -- 172.21.15.69:0/3788084958 shutdown_connections 2024-03-19T00:58:39.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:39.063+0000 7faef8800700 1 -- 172.21.15.69:0/3788084958 wait complete. 2024-03-19T00:58:39.738 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling cephadm module... 2024-03-19T00:58:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:40 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2683876114' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-19T00:58:41.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: ignoring --setuser ceph since I am not root 2024-03-19T00:58:41.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: ignoring --setgroup ceph since I am not root 2024-03-19T00:58:41.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.348+0000 7f2b25600700 1 Processor -- start 2024-03-19T00:58:41.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.348+0000 7f2b25600700 1 -- start start 2024-03-19T00:58:41.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.349+0000 7f2b25600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2010a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:41.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.349+0000 7f2b25600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2010aad0 con 0x7f2b2010a170 2024-03-19T00:58:41.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.349+0000 7f2b1ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2010a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:41.391 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.349+0000 7f2b1ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2010a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55108/0 (socket says 172.21.15.69:55108) 2024-03-19T00:58:41.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.349+0000 7f2b1ea00700 1 -- 172.21.15.69:0/3252482807 learned_addr learned my addr 172.21.15.69:0/3252482807 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:41.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.350+0000 7f2b1ea00700 1 -- 172.21.15.69:0/3252482807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2010afd0 con 0x7f2b2010a170 2024-03-19T00:58:41.392 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.350+0000 7f2b1ea00700 1 --2- 172.21.15.69:0/3252482807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2010a590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2b100098a0 tx=0x7f2b10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964c8e669f0202d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:41.396 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.350+0000 7f2b1d600700 1 -- 172.21.15.69:0/3252482807 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b10004160 con 0x7f2b2010a170 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.351+0000 7f2b1d600700 1 -- 172.21.15.69:0/3252482807 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2b10004300 con 0x7f2b2010a170 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.351+0000 7f2b1d600700 1 -- 172.21.15.69:0/3252482807 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b10011b50 con 0x7f2b2010a170 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.351+0000 7f2b25600700 1 -- 172.21.15.69:0/3252482807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 msgr2=0x7f2b2010a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.351+0000 7f2b25600700 1 --2- 172.21.15.69:0/3252482807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2010a590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2b100098a0 tx=0x7f2b10009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.352+0000 7f2b25600700 1 -- 172.21.15.69:0/3252482807 shutdown_connections 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.352+0000 7f2b25600700 1 --2- 172.21.15.69:0/3252482807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2010a590 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.352+0000 7f2b25600700 1 -- 172.21.15.69:0/3252482807 >> 172.21.15.69:0/3252482807 conn(0x7f2b2007d320 msgr2=0x7f2b2007d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.352+0000 7f2b25600700 1 -- 172.21.15.69:0/3252482807 shutdown_connections 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.352+0000 7f2b25600700 1 -- 172.21.15.69:0/3252482807 wait complete. 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.353+0000 7f2b25600700 1 Processor -- start 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.353+0000 7f2b25600700 1 -- start start 2024-03-19T00:58:41.397 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.353+0000 7f2b25600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.353+0000 7f2b25600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2010e290 con 0x7f2b2010a170 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.353+0000 7f2b1ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2019de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.353+0000 7f2b1ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2019de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55112/0 (socket says 172.21.15.69:55112) 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.353+0000 7f2b1ea00700 1 -- 172.21.15.69:0/2683876114 learned_addr learned my addr 172.21.15.69:0/2683876114 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.354+0000 7f2b1ea00700 1 -- 172.21.15.69:0/2683876114 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2019e370 con 0x7f2b2010a170 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.354+0000 7f2b1ea00700 1 --2- 172.21.15.69:0/2683876114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2019de30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2b10011b60 tx=0x7f2b10011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.354+0000 7f2b17400700 1 -- 172.21.15.69:0/2683876114 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b10003970 con 0x7f2b2010a170 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.354+0000 7f2b17400700 1 -- 172.21.15.69:0/2683876114 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2b10003b10 con 0x7f2b2010a170 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.354+0000 7f2b25600700 1 -- 172.21.15.69:0/2683876114 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2019e600 con 0x7f2b2010a170 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.354+0000 7f2b17400700 1 -- 172.21.15.69:0/2683876114 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b10024610 con 0x7f2b2010a170 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.354+0000 7f2b25600700 1 -- 172.21.15.69:0/2683876114 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b201a12f0 con 0x7f2b2010a170 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.355+0000 7f2b17400700 1 -- 172.21.15.69:0/2683876114 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f2b1001a9f0 con 0x7f2b2010a170 2024-03-19T00:58:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.356+0000 7f2b25600700 1 -- 172.21.15.69:0/2683876114 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b20063940 con 0x7f2b2010a170 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.356+0000 7f2b17400700 1 --2- 172.21.15.69:0/2683876114 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2b04038f20 0x7f2b0403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.356+0000 7f2b17400700 1 -- 172.21.15.69:0/2683876114 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2b10011e00 con 0x7f2b2010a170 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.356+0000 7f2b1e000700 1 --2- 172.21.15.69:0/2683876114 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2b04038f20 0x7f2b0403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.357+0000 7f2b1e000700 1 --2- 172.21.15.69:0/2683876114 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2b04038f20 0x7f2b0403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c006fd0 tx=0x7f2b0c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.361+0000 7f2b17400700 1 -- 172.21.15.69:0/2683876114 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2b10019e10 con 0x7f2b2010a170 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.619+0000 7f2b25600700 1 -- 172.21.15.69:0/2683876114 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2b201a14c0 con 0x7f2b2010a170 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.673+0000 7f2b17400700 1 -- 172.21.15.69:0/2683876114 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2b201a14c0 con 0x7f2b2010a170 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.675+0000 7f2b17400700 1 -- 172.21.15.69:0/2683876114 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f2b10024c40 con 0x7f2b2010a170 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.677+0000 7f2b25600700 1 -- 172.21.15.69:0/2683876114 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2b04038f20 msgr2=0x7f2b0403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.677+0000 7f2b25600700 1 --2- 172.21.15.69:0/2683876114 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2b04038f20 0x7f2b0403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c006fd0 tx=0x7f2b0c006e40 comp rx=0 tx=0).stop 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.677+0000 7f2b25600700 1 -- 172.21.15.69:0/2683876114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 msgr2=0x7f2b2019de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.677+0000 7f2b25600700 1 --2- 172.21.15.69:0/2683876114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2019de30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2b10011b60 tx=0x7f2b10011c40 comp rx=0 tx=0).stop 2024-03-19T00:58:41.399 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.678+0000 7f2b25600700 1 -- 172.21.15.69:0/2683876114 shutdown_connections 2024-03-19T00:58:41.400 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.678+0000 7f2b25600700 1 --2- 172.21.15.69:0/2683876114 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2b04038f20 0x7f2b0403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:41.400 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.678+0000 7f2b25600700 1 --2- 172.21.15.69:0/2683876114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b2010a170 0x7f2b2019de30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:41.400 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.678+0000 7f2b25600700 1 -- 172.21.15.69:0/2683876114 >> 172.21.15.69:0/2683876114 conn(0x7f2b2007d320 msgr2=0x7f2b20107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:41.400 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.678+0000 7f2b25600700 1 -- 172.21.15.69:0/2683876114 shutdown_connections 2024-03-19T00:58:41.400 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:40.678+0000 7f2b25600700 1 -- 172.21.15.69:0/2683876114 wait complete. 2024-03-19T00:58:41.683 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:41 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:41.473+0000 7f67e8f77200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T00:58:41.683 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:41 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:41.596+0000 7f67e8f77200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T00:58:41.972 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:41 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2683876114' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-19T00:58:41.973 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:41 smithi069 ceph-mon[31727]: mgrmap e5: x(active, since 5s) 2024-03-19T00:58:42.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:42.410+0000 7f67e8f77200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T00:58:42.851 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:42 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1576772966' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-19T00:58:42.852 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:42.535+0000 7f67e8f77200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T00:58:42.852 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:42.725+0000 7f67e8f77200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T00:58:42.852 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:42.850+0000 7f67e8f77200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T00:58:42.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:58:42.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-19T00:58:42.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-19T00:58:42.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-19T00:58:42.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-19T00:58:42.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:58:42.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.971+0000 7f9070200700 1 Processor -- start 2024-03-19T00:58:42.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.971+0000 7f9070200700 1 -- start start 2024-03-19T00:58:42.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.972+0000 7f9070200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:42.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.972+0000 7f9070200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f906807ea20 con 0x7f906807e600 2024-03-19T00:58:42.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.972+0000 7f906d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:42.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.972+0000 7f906d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906807cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55144/0 (socket says 172.21.15.69:55144) 2024-03-19T00:58:42.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.972+0000 7f906d800700 1 -- 172.21.15.69:0/2866747622 learned_addr learned my addr 172.21.15.69:0/2866747622 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:42.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.973+0000 7f906d800700 1 -- 172.21.15.69:0/2866747622 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906807d0b0 con 0x7f906807e600 2024-03-19T00:58:42.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.973+0000 7f906d800700 1 --2- 172.21.15.69:0/2866747622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906807cb70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f905c0098a0 tx=0x7f905c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8baacb90c58c2413 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:42.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.973+0000 7f9067e00700 1 -- 172.21.15.69:0/2866747622 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f905c004160 con 0x7f906807e600 2024-03-19T00:58:42.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.973+0000 7f9067e00700 1 -- 172.21.15.69:0/2866747622 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f905c004300 con 0x7f906807e600 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.973+0000 7f9067e00700 1 -- 172.21.15.69:0/2866747622 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f905c011b50 con 0x7f906807e600 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.974+0000 7f9070200700 1 -- 172.21.15.69:0/2866747622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 msgr2=0x7f906807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.974+0000 7f9070200700 1 --2- 172.21.15.69:0/2866747622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906807cb70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f905c0098a0 tx=0x7f905c009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.974+0000 7f9070200700 1 -- 172.21.15.69:0/2866747622 shutdown_connections 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.974+0000 7f9070200700 1 --2- 172.21.15.69:0/2866747622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906807cb70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.974+0000 7f9070200700 1 -- 172.21.15.69:0/2866747622 >> 172.21.15.69:0/2866747622 conn(0x7f90681033a0 msgr2=0x7f90681057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.974+0000 7f9070200700 1 -- 172.21.15.69:0/2866747622 shutdown_connections 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.974+0000 7f9070200700 1 -- 172.21.15.69:0/2866747622 wait complete. 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.975+0000 7f9070200700 1 Processor -- start 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.976+0000 7f9070200700 1 -- start start 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.976+0000 7f9070200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.976+0000 7f9070200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f906810e190 con 0x7f906807e600 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.976+0000 7f906d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.976+0000 7f906d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906819de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55154/0 (socket says 172.21.15.69:55154) 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.976+0000 7f906d800700 1 -- 172.21.15.69:0/1576772966 learned_addr learned my addr 172.21.15.69:0/1576772966 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.977+0000 7f906d800700 1 -- 172.21.15.69:0/1576772966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906819e3d0 con 0x7f906807e600 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.977+0000 7f906d800700 1 --2- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906819de90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f905c011b60 tx=0x7f905c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.977+0000 7f9066000700 1 -- 172.21.15.69:0/1576772966 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f905c003970 con 0x7f906807e600 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.977+0000 7f9066000700 1 -- 172.21.15.69:0/1576772966 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f905c003b10 con 0x7f906807e600 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.977+0000 7f9070200700 1 -- 172.21.15.69:0/1576772966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906819e660 con 0x7f906807e600 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.977+0000 7f9066000700 1 -- 172.21.15.69:0/1576772966 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f905c024610 con 0x7f906807e600 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.977+0000 7f9070200700 1 -- 172.21.15.69:0/1576772966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90681a1350 con 0x7f906807e600 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.979+0000 7f9070200700 1 -- 172.21.15.69:0/1576772966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9068063940 con 0x7f906807e600 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.979+0000 7f9066000700 1 -- 172.21.15.69:0/1576772966 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f905c01a430 con 0x7f906807e600 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.979+0000 7f9066000700 1 --2- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f9050038fd0 0x7f905003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.979+0000 7f9066000700 1 -- 172.21.15.69:0/1576772966 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f905c011e00 con 0x7f906807e600 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.979+0000 7f906ce00700 1 -- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f9050038fd0 msgr2=0x7f905003b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/2422305615 2024-03-19T00:58:42.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.979+0000 7f906ce00700 1 --2- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f9050038fd0 0x7f905003b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:41.988+0000 7f9066000700 1 -- 172.21.15.69:0/1576772966 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f905c051070 con 0x7f906807e600 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.180+0000 7f906ce00700 1 -- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f9050038fd0 msgr2=0x7f905003b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/2422305615 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.180+0000 7f906ce00700 1 --2- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f9050038fd0 0x7f905003b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.265+0000 7f9070200700 1 -- 172.21.15.69:0/1576772966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f90681a1520 con 0x7f906807e600 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.266+0000 7f9066000700 1 -- 172.21.15.69:0/1576772966 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f90681a1520 con 0x7f906807e600 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.269+0000 7f9070200700 1 -- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f9050038fd0 msgr2=0x7f905003b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.269+0000 7f9070200700 1 --2- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f9050038fd0 0x7f905003b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.269+0000 7f9070200700 1 -- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 msgr2=0x7f906819de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.269+0000 7f9070200700 1 --2- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906819de90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f905c011b60 tx=0x7f905c011c40 comp rx=0 tx=0).stop 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.269+0000 7f9070200700 1 -- 172.21.15.69:0/1576772966 shutdown_connections 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.269+0000 7f9070200700 1 --2- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f9050038fd0 0x7f905003b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.269+0000 7f9070200700 1 --2- 172.21.15.69:0/1576772966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f906807e600 0x7f906819de90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.269+0000 7f9070200700 1 -- 172.21.15.69:0/1576772966 >> 172.21.15.69:0/1576772966 conn(0x7f90681033a0 msgr2=0x7f9068103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.269+0000 7f9070200700 1 -- 172.21.15.69:0/1576772966 shutdown_connections 2024-03-19T00:58:42.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:42.270+0000 7f9070200700 1 -- 172.21.15.69:0/1576772966 wait complete. 2024-03-19T00:58:42.985 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2024-03-19T00:58:42.985 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 5... 2024-03-19T00:58:43.131 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:42.958+0000 7f67e8f77200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T00:58:43.443 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:43 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:43.304+0000 7f67e8f77200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T00:58:43.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:43 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:43.447+0000 7f67e8f77200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T00:58:43.783 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:43 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:43.564+0000 7f67e8f77200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T00:58:43.783 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:43 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:43.690+0000 7f67e8f77200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T00:58:44.207 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:43 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:43.910+0000 7f67e8f77200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T00:58:45.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:45 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:45.162+0000 7f67e8f77200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T00:58:45.749 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:45 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:45.330+0000 7f67e8f77200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T00:58:45.749 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:45 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:45.747+0000 7f67e8f77200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T00:58:46.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:45 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:45.863+0000 7f67e8f77200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T00:58:46.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:46 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:46.116+0000 7f67e8f77200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:58:46.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:46 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:46.360+0000 7f67e8f77200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T00:58:47.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:46 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:46.731+0000 7f67e8f77200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:58:47.347 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:47 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:47.047+0000 7f67e8f77200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T00:58:47.347 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:47 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:47.345+0000 7f67e8f77200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T00:58:48.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:48 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:48.346+0000 7f67e8f77200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T00:58:49.438 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:49.035+0000 7f67e8f77200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T00:58:49.438 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:49.161+0000 7f67e8f77200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T00:58:49.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:58:49.436+0000 7f67e8f77200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T00:58:49.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-mon[31727]: Active manager daemon x restarted 2024-03-19T00:58:49.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-mon[31727]: Activating manager daemon x 2024-03-19T00:58:49.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-mon[31727]: osdmap e2: 0 total, 0 up, 0 in 2024-03-19T00:58:49.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-mon[31727]: mgrmap e6: x(active, starting, since 0.00938276s) 2024-03-19T00:58:49.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:58:49.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T00:58:49.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T00:58:49.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T00:58:49.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T00:58:49.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:49 smithi069 ceph-mon[31727]: Manager daemon x is now available 2024-03-19T00:58:50.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:50 smithi069 ceph-mon[31727]: Found migration_current of "None". Setting to last migration. 2024-03-19T00:58:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:50 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:58:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:50 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:58:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:50 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:58:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:50 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:58:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:50 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:58:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:50 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T00:58:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:50 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T00:58:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:50 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:58:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:50 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:58:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:50 smithi069 ceph-mon[31727]: mgrmap e7: x(active, since 1.0159s) 2024-03-19T00:58:51.158 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:58:51.158 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-19T00:58:51.158 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-19T00:58:51.158 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:58:51.158 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.596+0000 7f2df4c00700 1 Processor -- start 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.596+0000 7f2df4c00700 1 -- start start 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.596+0000 7f2df4c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.596+0000 7f2df4c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2df007c840 con 0x7f2df007de00 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.597+0000 7f2dede00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.597+0000 7f2dede00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55160/0 (socket says 172.21.15.69:55160) 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.597+0000 7f2dede00700 1 -- 172.21.15.69:0/4212594170 learned_addr learned my addr 172.21.15.69:0/4212594170 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.597+0000 7f2dede00700 1 -- 172.21.15.69:0/4212594170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df007c9c0 con 0x7f2df007de00 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.598+0000 7f2dede00700 1 --2- 172.21.15.69:0/4212594170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df007c300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2de00098a0 tx=0x7f2de0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea50134cb5097c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.598+0000 7f2deca00700 1 -- 172.21.15.69:0/4212594170 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2de0004160 con 0x7f2df007de00 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.598+0000 7f2deca00700 1 -- 172.21.15.69:0/4212594170 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2de0004300 con 0x7f2df007de00 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.598+0000 7f2deca00700 1 -- 172.21.15.69:0/4212594170 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2de0011b50 con 0x7f2df007de00 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.599+0000 7f2df4c00700 1 -- 172.21.15.69:0/4212594170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 msgr2=0x7f2df007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.599+0000 7f2df4c00700 1 --2- 172.21.15.69:0/4212594170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df007c300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2de00098a0 tx=0x7f2de0009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.599+0000 7f2df4c00700 1 -- 172.21.15.69:0/4212594170 shutdown_connections 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.599+0000 7f2df4c00700 1 --2- 172.21.15.69:0/4212594170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df007c300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.599+0000 7f2df4c00700 1 -- 172.21.15.69:0/4212594170 >> 172.21.15.69:0/4212594170 conn(0x7f2df01032d0 msgr2=0x7f2df01056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.599+0000 7f2df4c00700 1 -- 172.21.15.69:0/4212594170 shutdown_connections 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.599+0000 7f2df4c00700 1 -- 172.21.15.69:0/4212594170 wait complete. 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.600+0000 7f2df4c00700 1 Processor -- start 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.600+0000 7f2df4c00700 1 -- start start 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.601+0000 7f2df4c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.601+0000 7f2df4c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010e280 con 0x7f2df007de00 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.601+0000 7f2dede00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.601+0000 7f2dede00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df019de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55176/0 (socket says 172.21.15.69:55176) 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.601+0000 7f2dede00700 1 -- 172.21.15.69:0/1740602762 learned_addr learned my addr 172.21.15.69:0/1740602762 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.601+0000 7f2dede00700 1 -- 172.21.15.69:0/1740602762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df019e340 con 0x7f2df007de00 2024-03-19T00:58:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.602+0000 7f2dede00700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df019de00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2de0011b60 tx=0x7f2de0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.602+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2de0003b20 con 0x7f2df007de00 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.602+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2de0003cc0 con 0x7f2df007de00 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.602+0000 7f2df4c00700 1 -- 172.21.15.69:0/1740602762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df019e5d0 con 0x7f2df007de00 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.602+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2de0024840 con 0x7f2df007de00 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.602+0000 7f2df4c00700 1 -- 172.21.15.69:0/1740602762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01a12c0 con 0x7f2df007de00 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.603+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f2de0022020 con 0x7f2df007de00 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.604+0000 7f2de6a00700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 0x7f2dd403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.604+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 --> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2dd403bb40 con 0x7f2dd4038f70 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.604+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2de0011e00 con 0x7f2df007de00 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.604+0000 7f2ded400700 1 -- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 msgr2=0x7f2dd403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/2422305615 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.604+0000 7f2ded400700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 0x7f2dd403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.805+0000 7f2ded400700 1 -- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 msgr2=0x7f2dd403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/2422305615 2024-03-19T00:58:51.161 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:43.805+0000 7f2ded400700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 0x7f2dd403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:44.206+0000 7f2ded400700 1 -- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 msgr2=0x7f2dd403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/2422305615 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:44.206+0000 7f2ded400700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 0x7f2dd403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:45.007+0000 7f2ded400700 1 -- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 msgr2=0x7f2dd403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/2422305615 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:45.007+0000 7f2ded400700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 0x7f2dd403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:46.609+0000 7f2ded400700 1 -- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 msgr2=0x7f2dd403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/2422305615 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:46.609+0000 7f2ded400700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 0x7f2dd403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:49.445+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 6) ==== 45873+0+0 (secure 0 0 0) 0x7f2de001a960 con 0x7f2df007de00 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:49.445+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 msgr2=0x7f2dd403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:49.445+0000 7f2de6a00700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 0x7f2dd403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.449+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7f2de004eb90 con 0x7f2df007de00 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.450+0000 7f2de6a00700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f2dd403c860 0x7f2dd403ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:51.162 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.450+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 --> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2dd403bb40 con 0x7f2dd403c860 2024-03-19T00:58:51.165 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.451+0000 7f2ded400700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f2dd403c860 0x7f2dd403ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:51.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.454+0000 7f2ded400700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f2dd403c860 0x7f2dd403ec50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc003a10 tx=0x7f2ddc0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:51.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.457+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 <== mgr.14120 v2:172.21.15.69:6800/4073584559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f2dd403bb40 con 0x7f2dd403c860 2024-03-19T00:58:51.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.468+0000 7f2df4c00700 1 -- 172.21.15.69:0/1740602762 --> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2df0063940 con 0x7f2dd403c860 2024-03-19T00:58:51.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.469+0000 7f2de6a00700 1 -- 172.21.15.69:0/1740602762 <== mgr.14120 v2:172.21.15.69:6800/4073584559 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f2df0063940 con 0x7f2dd403c860 2024-03-19T00:58:51.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.470+0000 7f2df4c00700 1 -- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f2dd403c860 msgr2=0x7f2dd403ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:51.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.470+0000 7f2df4c00700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f2dd403c860 0x7f2dd403ec50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc003a10 tx=0x7f2ddc0092b0 comp rx=0 tx=0).stop 2024-03-19T00:58:51.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.470+0000 7f2df4c00700 1 -- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 msgr2=0x7f2df019de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:51.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.470+0000 7f2df4c00700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df019de00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2de0011b60 tx=0x7f2de0011c40 comp rx=0 tx=0).stop 2024-03-19T00:58:51.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.470+0000 7f2df4c00700 1 -- 172.21.15.69:0/1740602762 shutdown_connections 2024-03-19T00:58:51.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.470+0000 7f2df4c00700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f2dd403c860 0x7f2dd403ec50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:51.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.470+0000 7f2df4c00700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2df007de00 0x7f2df019de00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:51.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.470+0000 7f2df4c00700 1 --2- 172.21.15.69:0/1740602762 >> [v2:172.21.15.69:6800/2422305615,v1:172.21.15.69:6801/2422305615] conn(0x7f2dd4038f70 0x7f2dd403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:51.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.470+0000 7f2df4c00700 1 -- 172.21.15.69:0/1740602762 >> 172.21.15.69:0/1740602762 conn(0x7f2df01032d0 msgr2=0x7f2df0103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:51.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.470+0000 7f2df4c00700 1 -- 172.21.15.69:0/1740602762 shutdown_connections 2024-03-19T00:58:51.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:50.470+0000 7f2df4c00700 1 -- 172.21.15.69:0/1740602762 wait complete. 2024-03-19T00:58:51.167 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 5 is available 2024-03-19T00:58:51.167 INFO:teuthology.orchestra.run.smithi069.stdout:Setting orchestrator backend to cephadm... 2024-03-19T00:58:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:51 smithi069 ceph-mon[31727]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-19T00:58:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:51 smithi069 ceph-mon[31727]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-19T00:58:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:51 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:58:52.693 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:52 smithi069 ceph-mon[31727]: [19/Mar/2024:00:58:50] ENGINE Bus STARTING 2024-03-19T00:58:52.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:52 smithi069 ceph-mon[31727]: [19/Mar/2024:00:58:51] ENGINE Serving on http://172.21.15.69:8765 2024-03-19T00:58:52.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:52 smithi069 ceph-mon[31727]: [19/Mar/2024:00:58:51] ENGINE Serving on https://172.21.15.69:7150 2024-03-19T00:58:52.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:52 smithi069 ceph-mon[31727]: [19/Mar/2024:00:58:51] ENGINE Bus STARTED 2024-03-19T00:58:52.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:52 smithi069 ceph-mon[31727]: mgrmap e8: x(active, since 2s) 2024-03-19T00:58:52.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:52 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:58:52.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:52 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:58:52.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.779+0000 7f57eca00700 1 Processor -- start 2024-03-19T00:58:52.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.780+0000 7f57eca00700 1 -- start start 2024-03-19T00:58:52.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.780+0000 7f57eca00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e807cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:52.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.780+0000 7f57eca00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f57e807d0d0 con 0x7f57e807e690 2024-03-19T00:58:52.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.780+0000 7f57e5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e807cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:52.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.781+0000 7f57e5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e807cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47544/0 (socket says 172.21.15.69:47544) 2024-03-19T00:58:52.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.781+0000 7f57e5e00700 1 -- 172.21.15.69:0/3543796853 learned_addr learned my addr 172.21.15.69:0/3543796853 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.781+0000 7f57e5e00700 1 -- 172.21.15.69:0/3543796853 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e807d250 con 0x7f57e807e690 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.781+0000 7f57e5e00700 1 --2- 172.21.15.69:0/3543796853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e807cb90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009e80 tx=0x7f57d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a432a4db627331 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.781+0000 7f57e5400700 1 -- 172.21.15.69:0/3543796853 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d8017070 con 0x7f57e807e690 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.782+0000 7f57e5400700 1 -- 172.21.15.69:0/3543796853 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57d800b8c0 con 0x7f57e807e690 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.782+0000 7f57e5400700 1 -- 172.21.15.69:0/3543796853 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d80045c0 con 0x7f57e807e690 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.782+0000 7f57eca00700 1 -- 172.21.15.69:0/3543796853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 msgr2=0x7f57e807cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.782+0000 7f57eca00700 1 --2- 172.21.15.69:0/3543796853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e807cb90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009e80 tx=0x7f57d800b330 comp rx=0 tx=0).stop 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.782+0000 7f57eca00700 1 -- 172.21.15.69:0/3543796853 shutdown_connections 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.782+0000 7f57eca00700 1 --2- 172.21.15.69:0/3543796853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e807cb90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.782+0000 7f57eca00700 1 -- 172.21.15.69:0/3543796853 >> 172.21.15.69:0/3543796853 conn(0x7f57e81032e0 msgr2=0x7f57e8105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.783+0000 7f57eca00700 1 -- 172.21.15.69:0/3543796853 shutdown_connections 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.783+0000 7f57eca00700 1 -- 172.21.15.69:0/3543796853 wait complete. 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.784+0000 7f57eca00700 1 Processor -- start 2024-03-19T00:58:52.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.784+0000 7f57eca00700 1 -- start start 2024-03-19T00:58:52.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.784+0000 7f57eca00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:52.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.784+0000 7f57eca00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f57e810e290 con 0x7f57e807e690 2024-03-19T00:58:52.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.784+0000 7f57e5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:52.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.784+0000 7f57e5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47550/0 (socket says 172.21.15.69:47550) 2024-03-19T00:58:52.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.784+0000 7f57e5e00700 1 -- 172.21.15.69:0/702126901 learned_addr learned my addr 172.21.15.69:0/702126901 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:52.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.785+0000 7f57e5e00700 1 -- 172.21.15.69:0/702126901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e819e2e0 con 0x7f57e807e690 2024-03-19T00:58:52.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.785+0000 7f57e5e00700 1 --2- 172.21.15.69:0/702126901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e819dda0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f57d8004350 tx=0x7f57d800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:52.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.785+0000 7f57df400700 1 -- 172.21.15.69:0/702126901 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d8020020 con 0x7f57e807e690 2024-03-19T00:58:52.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.785+0000 7f57df400700 1 -- 172.21.15.69:0/702126901 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57d8003c10 con 0x7f57e807e690 2024-03-19T00:58:52.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.785+0000 7f57df400700 1 -- 172.21.15.69:0/702126901 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d8017040 con 0x7f57e807e690 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.785+0000 7f57eca00700 1 -- 172.21.15.69:0/702126901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e819e570 con 0x7f57e807e690 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.785+0000 7f57eca00700 1 -- 172.21.15.69:0/702126901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e81a1220 con 0x7f57e807e690 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.786+0000 7f57df400700 1 -- 172.21.15.69:0/702126901 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f57d8023020 con 0x7f57e807e690 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.786+0000 7f57eca00700 1 -- 172.21.15.69:0/702126901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e8063940 con 0x7f57e807e690 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.787+0000 7f57df400700 1 --2- 172.21.15.69:0/702126901 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f57cc038f70 0x7f57cc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.787+0000 7f57df400700 1 -- 172.21.15.69:0/702126901 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f57d800f050 con 0x7f57e807e690 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.787+0000 7f57dd400700 1 --2- 172.21.15.69:0/702126901 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f57cc038f70 0x7f57cc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.788+0000 7f57dd400700 1 --2- 172.21.15.69:0/702126901 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f57cc038f70 0x7f57cc03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f57d0006fd0 tx=0x7f57d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:51.793+0000 7f57df400700 1 -- 172.21.15.69:0/702126901 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f57d802a020 con 0x7f57e807e690 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.029+0000 7f57eca00700 1 -- 172.21.15.69:0/702126901 --> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f57e8104470 con 0x7f57cc038f70 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.044+0000 7f57df400700 1 -- 172.21.15.69:0/702126901 <== mgr.14120 v2:172.21.15.69:6800/4073584559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f57e8104470 con 0x7f57cc038f70 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.048+0000 7f57eca00700 1 -- 172.21.15.69:0/702126901 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f57cc038f70 msgr2=0x7f57cc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:52.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.048+0000 7f57eca00700 1 --2- 172.21.15.69:0/702126901 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f57cc038f70 0x7f57cc03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f57d0006fd0 tx=0x7f57d0006e40 comp rx=0 tx=0).stop 2024-03-19T00:58:52.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.048+0000 7f57eca00700 1 -- 172.21.15.69:0/702126901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 msgr2=0x7f57e819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:52.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.048+0000 7f57eca00700 1 --2- 172.21.15.69:0/702126901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e819dda0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f57d8004350 tx=0x7f57d800bd50 comp rx=0 tx=0).stop 2024-03-19T00:58:52.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.048+0000 7f57eca00700 1 -- 172.21.15.69:0/702126901 shutdown_connections 2024-03-19T00:58:52.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.048+0000 7f57eca00700 1 --2- 172.21.15.69:0/702126901 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f57cc038f70 0x7f57cc03b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:52.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.048+0000 7f57eca00700 1 --2- 172.21.15.69:0/702126901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57e807e690 0x7f57e819dda0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:52.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.048+0000 7f57eca00700 1 -- 172.21.15.69:0/702126901 >> 172.21.15.69:0/702126901 conn(0x7f57e81032e0 msgr2=0x7f57e8103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:52.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.048+0000 7f57eca00700 1 -- 172.21.15.69:0/702126901 shutdown_connections 2024-03-19T00:58:52.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:52.048+0000 7f57eca00700 1 -- 172.21.15.69:0/702126901 wait complete. 2024-03-19T00:58:53.554 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:53 smithi069 ceph-mon[31727]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:58:54.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-19T00:58:54.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.298+0000 7f8868e00700 1 Processor -- start 2024-03-19T00:58:54.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.298+0000 7f8868e00700 1 -- start start 2024-03-19T00:58:54.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.299+0000 7f8868e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:54.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.299+0000 7f8868e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f886407c840 con 0x7f886407de50 2024-03-19T00:58:54.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.299+0000 7f8863400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.299+0000 7f8863400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47554/0 (socket says 172.21.15.69:47554) 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.299+0000 7f8863400700 1 -- 172.21.15.69:0/1776673395 learned_addr learned my addr 172.21.15.69:0/1776673395 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.300+0000 7f8863400700 1 -- 172.21.15.69:0/1776673395 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886407c9c0 con 0x7f886407de50 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.300+0000 7f8863400700 1 --2- 172.21.15.69:0/1776673395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886407c300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f88540098a0 tx=0x7f8854009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0475961775f7cb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.300+0000 7f8862000700 1 -- 172.21.15.69:0/1776673395 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8854004160 con 0x7f886407de50 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.300+0000 7f8862000700 1 -- 172.21.15.69:0/1776673395 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8854004300 con 0x7f886407de50 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.300+0000 7f8862000700 1 -- 172.21.15.69:0/1776673395 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8854011b50 con 0x7f886407de50 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.301+0000 7f8868e00700 1 -- 172.21.15.69:0/1776673395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 msgr2=0x7f886407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.301+0000 7f8868e00700 1 --2- 172.21.15.69:0/1776673395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886407c300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f88540098a0 tx=0x7f8854009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.301+0000 7f8868e00700 1 -- 172.21.15.69:0/1776673395 shutdown_connections 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.301+0000 7f8868e00700 1 --2- 172.21.15.69:0/1776673395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886407c300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.301+0000 7f8868e00700 1 -- 172.21.15.69:0/1776673395 >> 172.21.15.69:0/1776673395 conn(0x7f88641030d0 msgr2=0x7f88641054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.301+0000 7f8868e00700 1 -- 172.21.15.69:0/1776673395 shutdown_connections 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.301+0000 7f8868e00700 1 -- 172.21.15.69:0/1776673395 wait complete. 2024-03-19T00:58:54.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.302+0000 7f8868e00700 1 Processor -- start 2024-03-19T00:58:54.260 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.302+0000 7f8868e00700 1 -- start start 2024-03-19T00:58:54.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.303+0000 7f8868e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:54.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.303+0000 7f8868e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f886410e0e0 con 0x7f886407de50 2024-03-19T00:58:54.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.303+0000 7f8863400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:54.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.303+0000 7f8863400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886419dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47566/0 (socket says 172.21.15.69:47566) 2024-03-19T00:58:54.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.303+0000 7f8863400700 1 -- 172.21.15.69:0/958510313 learned_addr learned my addr 172.21.15.69:0/958510313 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:54.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.303+0000 7f8863400700 1 -- 172.21.15.69:0/958510313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886419e1b0 con 0x7f886407de50 2024-03-19T00:58:54.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.304+0000 7f8863400700 1 --2- 172.21.15.69:0/958510313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886419dc70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8854011b60 tx=0x7f8854011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:54.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.304+0000 7f885be00700 1 -- 172.21.15.69:0/958510313 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8854003b20 con 0x7f886407de50 2024-03-19T00:58:54.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.304+0000 7f885be00700 1 -- 172.21.15.69:0/958510313 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8854003cc0 con 0x7f886407de50 2024-03-19T00:58:54.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.304+0000 7f885be00700 1 -- 172.21.15.69:0/958510313 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8854024840 con 0x7f886407de50 2024-03-19T00:58:54.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.304+0000 7f8868e00700 1 -- 172.21.15.69:0/958510313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886419e440 con 0x7f886407de50 2024-03-19T00:58:54.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.304+0000 7f8868e00700 1 -- 172.21.15.69:0/958510313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88641a1130 con 0x7f886407de50 2024-03-19T00:58:54.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.306+0000 7f8868e00700 1 -- 172.21.15.69:0/958510313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8864063940 con 0x7f886407de50 2024-03-19T00:58:54.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.306+0000 7f885be00700 1 -- 172.21.15.69:0/958510313 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f8854022020 con 0x7f886407de50 2024-03-19T00:58:54.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.306+0000 7f885be00700 1 --2- 172.21.15.69:0/958510313 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8848038f70 0x7f884803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:54.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.306+0000 7f885be00700 1 -- 172.21.15.69:0/958510313 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8854011e00 con 0x7f886407de50 2024-03-19T00:58:54.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.306+0000 7f8862a00700 1 --2- 172.21.15.69:0/958510313 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8848038f70 0x7f884803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:54.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.307+0000 7f8862a00700 1 --2- 172.21.15.69:0/958510313 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8848038f70 0x7f884803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8850006fd0 tx=0x7f8850006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:54.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.312+0000 7f885be00700 1 -- 172.21.15.69:0/958510313 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f885400b3e0 con 0x7f886407de50 2024-03-19T00:58:54.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.532+0000 7f8868e00700 1 -- 172.21.15.69:0/958510313 --> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f8864104340 con 0x7f8848038f70 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.534+0000 7f885be00700 1 -- 172.21.15.69:0/958510313 <== mgr.14120 v2:172.21.15.69:6800/4073584559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f8864104340 con 0x7f8848038f70 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.537+0000 7f8868e00700 1 -- 172.21.15.69:0/958510313 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8848038f70 msgr2=0x7f884803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.537+0000 7f8868e00700 1 --2- 172.21.15.69:0/958510313 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8848038f70 0x7f884803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8850006fd0 tx=0x7f8850006e40 comp rx=0 tx=0).stop 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.537+0000 7f8868e00700 1 -- 172.21.15.69:0/958510313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 msgr2=0x7f886419dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.537+0000 7f8868e00700 1 --2- 172.21.15.69:0/958510313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886419dc70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8854011b60 tx=0x7f8854011c40 comp rx=0 tx=0).stop 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.537+0000 7f8868e00700 1 -- 172.21.15.69:0/958510313 shutdown_connections 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.537+0000 7f8868e00700 1 --2- 172.21.15.69:0/958510313 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8848038f70 0x7f884803b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.537+0000 7f8868e00700 1 --2- 172.21.15.69:0/958510313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f886407de50 0x7f886419dc70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.537+0000 7f8868e00700 1 -- 172.21.15.69:0/958510313 >> 172.21.15.69:0/958510313 conn(0x7f88641030d0 msgr2=0x7f8864103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.537+0000 7f8868e00700 1 -- 172.21.15.69:0/958510313 shutdown_connections 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:53.537+0000 7f8868e00700 1 -- 172.21.15.69:0/958510313 wait complete. 2024-03-19T00:58:54.263 INFO:teuthology.orchestra.run.smithi069.stdout:Generating ssh key... 2024-03-19T00:58:54.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:54 smithi069 ceph-mon[31727]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:58:55.932 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: Generating public/private rsa key pair. 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: Your identification has been saved in /tmp/tmpkql6moch/key. 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: Your public key has been saved in /tmp/tmpkql6moch/key.pub. 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: The key fingerprint is: 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: SHA256:4rPvWOx7orQntoLJEjaX+zLWYrKFHmo58KU6AGkLZTw ceph-934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: The key's randomart image is: 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: +---[RSA 3072]----+ 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: | . | 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: | E | 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: | + . | 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: |+. | 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: |+ . . . S | 2024-03-19T00:58:55.933 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: |++.o.. o | 2024-03-19T00:58:55.934 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: |+*+*o + o | 2024-03-19T00:58:55.934 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: |==OB.ooBo . | 2024-03-19T00:58:55.934 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: |+** =+*B=+ | 2024-03-19T00:58:55.934 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:58:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: +----[SHA256]-----+ 2024-03-19T00:58:56.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.914+0000 7f5a4e800700 1 Processor -- start 2024-03-19T00:58:56.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.914+0000 7f5a4e800700 1 -- start start 2024-03-19T00:58:56.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.914+0000 7f5a4e800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4807c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:56.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.914+0000 7f5a4e800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4807c8a0 con 0x7f5a4807ddf0 2024-03-19T00:58:56.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.915+0000 7f5a47400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4807c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:56.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.915+0000 7f5a47400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4807c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47582/0 (socket says 172.21.15.69:47582) 2024-03-19T00:58:56.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.915+0000 7f5a47400700 1 -- 172.21.15.69:0/2691172163 learned_addr learned my addr 172.21.15.69:0/2691172163 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:56.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.915+0000 7f5a47400700 1 -- 172.21.15.69:0/2691172163 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4807ca20 con 0x7f5a4807ddf0 2024-03-19T00:58:56.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.916+0000 7f5a47400700 1 --2- 172.21.15.69:0/2691172163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4807c360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5a380098a0 tx=0x7f5a38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f027a347b69377 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:56.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.916+0000 7f5a46000700 1 -- 172.21.15.69:0/2691172163 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a38004160 con 0x7f5a4807ddf0 2024-03-19T00:58:56.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.916+0000 7f5a46000700 1 -- 172.21.15.69:0/2691172163 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a38004300 con 0x7f5a4807ddf0 2024-03-19T00:58:56.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.916+0000 7f5a46000700 1 -- 172.21.15.69:0/2691172163 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a38011b50 con 0x7f5a4807ddf0 2024-03-19T00:58:56.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.917+0000 7f5a4e800700 1 -- 172.21.15.69:0/2691172163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 msgr2=0x7f5a4807c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:56.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.917+0000 7f5a4e800700 1 --2- 172.21.15.69:0/2691172163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4807c360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5a380098a0 tx=0x7f5a38009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:56.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.917+0000 7f5a4e800700 1 -- 172.21.15.69:0/2691172163 shutdown_connections 2024-03-19T00:58:56.257 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.917+0000 7f5a4e800700 1 --2- 172.21.15.69:0/2691172163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4807c360 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:56.257 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.917+0000 7f5a4e800700 1 -- 172.21.15.69:0/2691172163 >> 172.21.15.69:0/2691172163 conn(0x7f5a48103360 msgr2=0x7f5a48105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:56.257 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.917+0000 7f5a4e800700 1 -- 172.21.15.69:0/2691172163 shutdown_connections 2024-03-19T00:58:56.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.917+0000 7f5a4e800700 1 -- 172.21.15.69:0/2691172163 wait complete. 2024-03-19T00:58:56.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.918+0000 7f5a4e800700 1 Processor -- start 2024-03-19T00:58:56.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.918+0000 7f5a4e800700 1 -- start start 2024-03-19T00:58:56.258 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.919+0000 7f5a4e800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:56.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.919+0000 7f5a4e800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4810e1f0 con 0x7f5a4807ddf0 2024-03-19T00:58:56.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.919+0000 7f5a47400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:56.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.919+0000 7f5a47400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4819de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47596/0 (socket says 172.21.15.69:47596) 2024-03-19T00:58:56.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.919+0000 7f5a47400700 1 -- 172.21.15.69:0/2087009747 learned_addr learned my addr 172.21.15.69:0/2087009747 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:56.259 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.919+0000 7f5a47400700 1 -- 172.21.15.69:0/2087009747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4819e370 con 0x7f5a4807ddf0 2024-03-19T00:58:56.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.920+0000 7f5a47400700 1 --2- 172.21.15.69:0/2087009747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4819de30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5a38011670 tx=0x7f5a38011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:56.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.920+0000 7f5a3fe00700 1 -- 172.21.15.69:0/2087009747 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a380039c0 con 0x7f5a4807ddf0 2024-03-19T00:58:56.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.920+0000 7f5a3fe00700 1 -- 172.21.15.69:0/2087009747 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a38011e50 con 0x7f5a4807ddf0 2024-03-19T00:58:56.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.920+0000 7f5a4e800700 1 -- 172.21.15.69:0/2087009747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4819e600 con 0x7f5a4807ddf0 2024-03-19T00:58:56.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.920+0000 7f5a3fe00700 1 -- 172.21.15.69:0/2087009747 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a3801a5c0 con 0x7f5a4807ddf0 2024-03-19T00:58:56.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.920+0000 7f5a4e800700 1 -- 172.21.15.69:0/2087009747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a481a12f0 con 0x7f5a4807ddf0 2024-03-19T00:58:56.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.921+0000 7f5a4e800700 1 -- 172.21.15.69:0/2087009747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a48063940 con 0x7f5a4807ddf0 2024-03-19T00:58:56.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.921+0000 7f5a3fe00700 1 -- 172.21.15.69:0/2087009747 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f5a38024430 con 0x7f5a4807ddf0 2024-03-19T00:58:56.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.922+0000 7f5a3fe00700 1 --2- 172.21.15.69:0/2087009747 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f5a2c038f20 0x7f5a2c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:56.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.922+0000 7f5a3fe00700 1 -- 172.21.15.69:0/2087009747 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5a3804d230 con 0x7f5a4807ddf0 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.922+0000 7f5a46a00700 1 --2- 172.21.15.69:0/2087009747 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f5a2c038f20 0x7f5a2c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.923+0000 7f5a46a00700 1 --2- 172.21.15.69:0/2087009747 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f5a2c038f20 0x7f5a2c03b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5a34006fd0 tx=0x7f5a34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:54.927+0000 7f5a3fe00700 1 -- 172.21.15.69:0/2087009747 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5a38017ca0 con 0x7f5a4807ddf0 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.151+0000 7f5a4e800700 1 -- 172.21.15.69:0/2087009747 --> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f5a48104560 con 0x7f5a2c038f20 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.617+0000 7f5a3fe00700 1 -- 172.21.15.69:0/2087009747 <== mgr.14120 v2:172.21.15.69:6800/4073584559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5a48104560 con 0x7f5a2c038f20 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.621+0000 7f5a4e800700 1 -- 172.21.15.69:0/2087009747 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f5a2c038f20 msgr2=0x7f5a2c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.621+0000 7f5a4e800700 1 --2- 172.21.15.69:0/2087009747 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f5a2c038f20 0x7f5a2c03b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5a34006fd0 tx=0x7f5a34006e40 comp rx=0 tx=0).stop 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.621+0000 7f5a4e800700 1 -- 172.21.15.69:0/2087009747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 msgr2=0x7f5a4819de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.621+0000 7f5a4e800700 1 --2- 172.21.15.69:0/2087009747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4819de30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5a38011670 tx=0x7f5a38011b60 comp rx=0 tx=0).stop 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.621+0000 7f5a4e800700 1 -- 172.21.15.69:0/2087009747 shutdown_connections 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.621+0000 7f5a4e800700 1 --2- 172.21.15.69:0/2087009747 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f5a2c038f20 0x7f5a2c03b3e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.621+0000 7f5a4e800700 1 --2- 172.21.15.69:0/2087009747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a4807ddf0 0x7f5a4819de30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.621+0000 7f5a4e800700 1 -- 172.21.15.69:0/2087009747 >> 172.21.15.69:0/2087009747 conn(0x7f5a48103360 msgr2=0x7f5a48103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.621+0000 7f5a4e800700 1 -- 172.21.15.69:0/2087009747 shutdown_connections 2024-03-19T00:58:56.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:55.622+0000 7f5a4e800700 1 -- 172.21.15.69:0/2087009747 wait complete. 2024-03-19T00:58:56.776 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:56 smithi069 ceph-mon[31727]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:58:56.776 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:56 smithi069 ceph-mon[31727]: Generating ssh key... 2024-03-19T00:58:56.776 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:56 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:58:56.776 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:56 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:58:57.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZElbmgFaeL/glKigdAXtw09oGaz87AEKLPO/i6+Leg+zgPwpYJPkQ1G7rtrDXm9mkM5fo+sp1uuQ/nH/RXzI82QlDiREeEQHo5mBYgl/mEdZp5XfHBx8+LMBZY//OwVuAF0rWgwS8ev9zzTh9d5RF78ppNCmiZmj/FwoWnzjhWyUUQ2aJdCiKvJdANGZDwPgreO8+Q3WpGCoJTjMMB8Js+g0O0pxHvs+OuIrDYsIUkEOmC4sbw4hVSibYDsS3JGjp58Mt82CkThESVzjqXgqmoWEhX9A7M9bO+vjjksCP5PGd64sT3HzYUavux1wFucUpU5X51oBVar9DLVFdN0nT33EBYUgOK1f5c3nCGqh8IcraUEGg+aF8fCfBt1E0eatvSsYfBHmUDFmVKGhhUegEeRVTbM8JXbvuCofLqbl2RrzuLfbzbX6vk5+hFAyGfyvYTFp9Xw8gVWClqMKDMwQHXyzVzA0+skPivmNBzUCuw4CZrn0Mqh5dcV5bV7w7gXE= ceph-934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:58:57.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.854+0000 7fe019c00700 1 Processor -- start 2024-03-19T00:58:57.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.854+0000 7fe019c00700 1 -- start start 2024-03-19T00:58:57.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.854+0000 7fe019c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe014105e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:57.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.854+0000 7fe019c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe014106340 con 0x7fe0141059e0 2024-03-19T00:58:57.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.855+0000 7fe012a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe014105e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.855+0000 7fe012a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe014105e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50282/0 (socket says 172.21.15.69:50282) 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.855+0000 7fe012a00700 1 -- 172.21.15.69:0/413636078 learned_addr learned my addr 172.21.15.69:0/413636078 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.855+0000 7fe012a00700 1 -- 172.21.15.69:0/413636078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0141064c0 con 0x7fe0141059e0 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.856+0000 7fe012a00700 1 --2- 172.21.15.69:0/413636078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe014105e00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0040098a0 tx=0x7fe004009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c0eca33adee5c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.856+0000 7fe012000700 1 -- 172.21.15.69:0/413636078 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe004004160 con 0x7fe0141059e0 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.856+0000 7fe012000700 1 -- 172.21.15.69:0/413636078 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe004004300 con 0x7fe0141059e0 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.856+0000 7fe012000700 1 -- 172.21.15.69:0/413636078 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe004011b50 con 0x7fe0141059e0 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.856+0000 7fe019c00700 1 -- 172.21.15.69:0/413636078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 msgr2=0x7fe014105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.856+0000 7fe019c00700 1 --2- 172.21.15.69:0/413636078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe014105e00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0040098a0 tx=0x7fe004009bb0 comp rx=0 tx=0).stop 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.857+0000 7fe019c00700 1 -- 172.21.15.69:0/413636078 shutdown_connections 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.857+0000 7fe019c00700 1 --2- 172.21.15.69:0/413636078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe014105e00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.857+0000 7fe019c00700 1 -- 172.21.15.69:0/413636078 >> 172.21.15.69:0/413636078 conn(0x7fe014101480 msgr2=0x7fe0141038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.857+0000 7fe019c00700 1 -- 172.21.15.69:0/413636078 shutdown_connections 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.857+0000 7fe019c00700 1 -- 172.21.15.69:0/413636078 wait complete. 2024-03-19T00:58:57.772 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.858+0000 7fe019c00700 1 Processor -- start 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.858+0000 7fe019c00700 1 -- start start 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.858+0000 7fe019c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe01419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.858+0000 7fe019c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe01410c070 con 0x7fe0141059e0 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.858+0000 7fe012a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe01419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.858+0000 7fe012a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe01419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50298/0 (socket says 172.21.15.69:50298) 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.858+0000 7fe012a00700 1 -- 172.21.15.69:0/880909369 learned_addr learned my addr 172.21.15.69:0/880909369 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.859+0000 7fe012a00700 1 -- 172.21.15.69:0/880909369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01419e340 con 0x7fe0141059e0 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.859+0000 7fe012a00700 1 --2- 172.21.15.69:0/880909369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe01419de00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe004011b60 tx=0x7fe004011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.859+0000 7fe00b400700 1 -- 172.21.15.69:0/880909369 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe004003b20 con 0x7fe0141059e0 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.859+0000 7fe00b400700 1 -- 172.21.15.69:0/880909369 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe004003cc0 con 0x7fe0141059e0 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.859+0000 7fe019c00700 1 -- 172.21.15.69:0/880909369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01419e5d0 con 0x7fe0141059e0 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.859+0000 7fe00b400700 1 -- 172.21.15.69:0/880909369 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe004024840 con 0x7fe0141059e0 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.860+0000 7fe019c00700 1 -- 172.21.15.69:0/880909369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0141a12c0 con 0x7fe0141059e0 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.861+0000 7fe00b400700 1 -- 172.21.15.69:0/880909369 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fe004022020 con 0x7fe0141059e0 2024-03-19T00:58:57.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.861+0000 7fe019c00700 1 -- 172.21.15.69:0/880909369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe014063940 con 0x7fe0141059e0 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.861+0000 7fe00b400700 1 --2- 172.21.15.69:0/880909369 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fdff8038f70 0x7fdff803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.861+0000 7fe00b400700 1 -- 172.21.15.69:0/880909369 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe004011e00 con 0x7fe0141059e0 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.861+0000 7fe00be00700 1 --2- 172.21.15.69:0/880909369 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fdff8038f70 0x7fdff803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.862+0000 7fe00be00700 1 --2- 172.21.15.69:0/880909369 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fdff8038f70 0x7fdff803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdffc006fd0 tx=0x7fdffc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:56.867+0000 7fe00b400700 1 -- 172.21.15.69:0/880909369 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe00401a7b0 con 0x7fe0141059e0 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.091+0000 7fe019c00700 1 -- 172.21.15.69:0/880909369 --> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fe0141026d0 con 0x7fdff8038f70 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.092+0000 7fe00b400700 1 -- 172.21.15.69:0/880909369 <== mgr.14120 v2:172.21.15.69:6800/4073584559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fe0141026d0 con 0x7fdff8038f70 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.095+0000 7fe019c00700 1 -- 172.21.15.69:0/880909369 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fdff8038f70 msgr2=0x7fdff803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.095+0000 7fe019c00700 1 --2- 172.21.15.69:0/880909369 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fdff8038f70 0x7fdff803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdffc006fd0 tx=0x7fdffc006e40 comp rx=0 tx=0).stop 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.095+0000 7fe019c00700 1 -- 172.21.15.69:0/880909369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 msgr2=0x7fe01419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.095+0000 7fe019c00700 1 --2- 172.21.15.69:0/880909369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe01419de00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe004011b60 tx=0x7fe004011c40 comp rx=0 tx=0).stop 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.095+0000 7fe019c00700 1 -- 172.21.15.69:0/880909369 shutdown_connections 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.095+0000 7fe019c00700 1 --2- 172.21.15.69:0/880909369 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fdff8038f70 0x7fdff803b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.095+0000 7fe019c00700 1 --2- 172.21.15.69:0/880909369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0141059e0 0x7fe01419de00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:58:57.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.095+0000 7fe019c00700 1 -- 172.21.15.69:0/880909369 >> 172.21.15.69:0/880909369 conn(0x7fe014101480 msgr2=0x7fe014101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:58:57.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.095+0000 7fe019c00700 1 -- 172.21.15.69:0/880909369 shutdown_connections 2024-03-19T00:58:57.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:57.095+0000 7fe019c00700 1 -- 172.21.15.69:0/880909369 wait complete. 2024-03-19T00:58:57.775 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-19T00:58:57.775 INFO:teuthology.orchestra.run.smithi069.stdout:Adding key to root@localhost authorized_keys... 2024-03-19T00:58:57.775 INFO:teuthology.orchestra.run.smithi069.stdout:Adding host smithi069... 2024-03-19T00:58:58.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:57 smithi069 ceph-mon[31727]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:58:58.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:58:58 smithi069 ceph-mon[31727]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:59:01.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:00 smithi069 ceph-mon[31727]: Deploying cephadm binary to smithi069 2024-03-19T00:59:05.777 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Added host 'smithi069' with addr '172.21.15.69' 2024-03-19T00:59:05.777 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.394+0000 7efff9c00700 1 Processor -- start 2024-03-19T00:59:05.777 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.394+0000 7efff9c00700 1 -- start start 2024-03-19T00:59:05.778 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.395+0000 7efff9c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff407c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:05.778 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.395+0000 7efff9c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efff407c880 con 0x7efff407de40 2024-03-19T00:59:05.778 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.395+0000 7efff2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff407c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:05.778 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.395+0000 7efff2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff407c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50306/0 (socket says 172.21.15.69:50306) 2024-03-19T00:59:05.778 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.395+0000 7efff2a00700 1 -- 172.21.15.69:0/24691343 learned_addr learned my addr 172.21.15.69:0/24691343 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:05.778 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.396+0000 7efff2a00700 1 -- 172.21.15.69:0/24691343 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff407ca00 con 0x7efff407de40 2024-03-19T00:59:05.778 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.396+0000 7efff2a00700 1 --2- 172.21.15.69:0/24691343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff407c340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7effe4009e80 tx=0x7effe400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c2ad3b06ee0ec3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:05.778 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.397+0000 7efff1600700 1 -- 172.21.15.69:0/24691343 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effe4017070 con 0x7efff407de40 2024-03-19T00:59:05.778 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.397+0000 7efff1600700 1 -- 172.21.15.69:0/24691343 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7effe400b8c0 con 0x7efff407de40 2024-03-19T00:59:05.778 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.397+0000 7efff1600700 1 -- 172.21.15.69:0/24691343 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effe40045c0 con 0x7efff407de40 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.398+0000 7efff9c00700 1 -- 172.21.15.69:0/24691343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 msgr2=0x7efff407c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.398+0000 7efff9c00700 1 --2- 172.21.15.69:0/24691343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff407c340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7effe4009e80 tx=0x7effe400b330 comp rx=0 tx=0).stop 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.398+0000 7efff9c00700 1 -- 172.21.15.69:0/24691343 shutdown_connections 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.398+0000 7efff9c00700 1 --2- 172.21.15.69:0/24691343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff407c340 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.398+0000 7efff9c00700 1 -- 172.21.15.69:0/24691343 >> 172.21.15.69:0/24691343 conn(0x7efff41032d0 msgr2=0x7efff41056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.398+0000 7efff9c00700 1 -- 172.21.15.69:0/24691343 shutdown_connections 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.398+0000 7efff9c00700 1 -- 172.21.15.69:0/24691343 wait complete. 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.399+0000 7efff9c00700 1 Processor -- start 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.399+0000 7efff9c00700 1 -- start start 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.399+0000 7efff9c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.400+0000 7efff9c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efff410e2a0 con 0x7efff407de40 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.400+0000 7efff2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:05.779 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.400+0000 7efff2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50308/0 (socket says 172.21.15.69:50308) 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.400+0000 7efff2a00700 1 -- 172.21.15.69:0/773252791 learned_addr learned my addr 172.21.15.69:0/773252791 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.400+0000 7efff2a00700 1 -- 172.21.15.69:0/773252791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff419e2e0 con 0x7efff407de40 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.400+0000 7efff2a00700 1 --2- 172.21.15.69:0/773252791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff419dda0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7effe4004350 tx=0x7effe400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.401+0000 7effeb400700 1 -- 172.21.15.69:0/773252791 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effe4020020 con 0x7efff407de40 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.401+0000 7effeb400700 1 -- 172.21.15.69:0/773252791 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7effe4003c10 con 0x7efff407de40 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.401+0000 7efff9c00700 1 -- 172.21.15.69:0/773252791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff419e570 con 0x7efff407de40 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.401+0000 7effeb400700 1 -- 172.21.15.69:0/773252791 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effe4017040 con 0x7efff407de40 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.401+0000 7efff9c00700 1 -- 172.21.15.69:0/773252791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff41a1260 con 0x7efff407de40 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.402+0000 7effeb400700 1 -- 172.21.15.69:0/773252791 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7effe4023020 con 0x7efff407de40 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.402+0000 7efff9c00700 1 -- 172.21.15.69:0/773252791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff4063940 con 0x7efff407de40 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.403+0000 7effeb400700 1 --2- 172.21.15.69:0/773252791 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7effd8038f20 0x7effd803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.403+0000 7effeb400700 1 -- 172.21.15.69:0/773252791 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7effe400f050 con 0x7efff407de40 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.403+0000 7efff2000700 1 --2- 172.21.15.69:0/773252791 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7effd8038f20 0x7effd803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:05.780 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.404+0000 7efff2000700 1 --2- 172.21.15.69:0/773252791 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7effd8038f20 0x7effd803b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effe0006fd0 tx=0x7effe0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.408+0000 7effeb400700 1 -- 172.21.15.69:0/773252791 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7effe402a020 con 0x7efff407de40 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:58:58.632+0000 7efff9c00700 1 -- 172.21.15.69:0/773252791 --> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}) -- 0x7efff4104440 con 0x7effd8038f20 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:05.027+0000 7effeb400700 1 -- 172.21.15.69:0/773252791 <== mgr.14120 v2:172.21.15.69:6800/4073584559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7efff4104440 con 0x7effd8038f20 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:05.031+0000 7efff9c00700 1 -- 172.21.15.69:0/773252791 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7effd8038f20 msgr2=0x7effd803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:05.031+0000 7efff9c00700 1 --2- 172.21.15.69:0/773252791 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7effd8038f20 0x7effd803b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effe0006fd0 tx=0x7effe0006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:05.031+0000 7efff9c00700 1 -- 172.21.15.69:0/773252791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 msgr2=0x7efff419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:05.031+0000 7efff9c00700 1 --2- 172.21.15.69:0/773252791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff419dda0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7effe4004350 tx=0x7effe400bd50 comp rx=0 tx=0).stop 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:05.031+0000 7efff9c00700 1 -- 172.21.15.69:0/773252791 shutdown_connections 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:05.031+0000 7efff9c00700 1 --2- 172.21.15.69:0/773252791 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7effd8038f20 0x7effd803b3e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:05.031+0000 7efff9c00700 1 --2- 172.21.15.69:0/773252791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efff407de40 0x7efff419dda0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:05.031+0000 7efff9c00700 1 -- 172.21.15.69:0/773252791 >> 172.21.15.69:0/773252791 conn(0x7efff41032d0 msgr2=0x7efff4103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:05.031+0000 7efff9c00700 1 -- 172.21.15.69:0/773252791 shutdown_connections 2024-03-19T00:59:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:05.031+0000 7efff9c00700 1 -- 172.21.15.69:0/773252791 wait complete. 2024-03-19T00:59:05.782 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mon service... 2024-03-19T00:59:06.280 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:06 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:59:06.280 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:06 smithi069 ceph-mon[31727]: Added host smithi069 2024-03-19T00:59:06.280 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:06 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:59:07.348 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-19T00:59:07.348 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.449+0000 7f81fae00700 1 Processor -- start 2024-03-19T00:59:07.348 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.449+0000 7f81fae00700 1 -- start start 2024-03-19T00:59:07.348 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.449+0000 7f81fae00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f410ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:07.348 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.450+0000 7f81fae00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81f407bed0 con 0x7f81f4108720 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.450+0000 7f81f3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f410ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.450+0000 7f81f3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f410ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60248/0 (socket says 172.21.15.69:60248) 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.450+0000 7f81f3e00700 1 -- 172.21.15.69:0/2544880358 learned_addr learned my addr 172.21.15.69:0/2544880358 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.450+0000 7f81f3e00700 1 -- 172.21.15.69:0/2544880358 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f410b050 con 0x7f81f4108720 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.451+0000 7f81f3e00700 1 --2- 172.21.15.69:0/2544880358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f410ab10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f81e40098a0 tx=0x7f81e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316e9a9b6316f974 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.451+0000 7f81f2a00700 1 -- 172.21.15.69:0/2544880358 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81e4004160 con 0x7f81f4108720 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.451+0000 7f81f2a00700 1 -- 172.21.15.69:0/2544880358 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f81e4004300 con 0x7f81f4108720 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.451+0000 7f81f2a00700 1 -- 172.21.15.69:0/2544880358 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81e4011b50 con 0x7f81f4108720 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.452+0000 7f81fae00700 1 -- 172.21.15.69:0/2544880358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 msgr2=0x7f81f410ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.452+0000 7f81fae00700 1 --2- 172.21.15.69:0/2544880358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f410ab10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f81e40098a0 tx=0x7f81e4009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.452+0000 7f81fae00700 1 -- 172.21.15.69:0/2544880358 shutdown_connections 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.452+0000 7f81fae00700 1 --2- 172.21.15.69:0/2544880358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f410ab10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.452+0000 7f81fae00700 1 -- 172.21.15.69:0/2544880358 >> 172.21.15.69:0/2544880358 conn(0x7f81f4102130 msgr2=0x7f81f4104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:07.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.452+0000 7f81fae00700 1 -- 172.21.15.69:0/2544880358 shutdown_connections 2024-03-19T00:59:07.350 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.452+0000 7f81fae00700 1 -- 172.21.15.69:0/2544880358 wait complete. 2024-03-19T00:59:07.350 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.453+0000 7f81fae00700 1 Processor -- start 2024-03-19T00:59:07.350 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.453+0000 7f81fae00700 1 -- start start 2024-03-19T00:59:07.350 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.453+0000 7f81fae00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:07.350 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.453+0000 7f81fae00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81f410c080 con 0x7f81f4108720 2024-03-19T00:59:07.350 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.454+0000 7f81f3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:07.350 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.454+0000 7f81f3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60260/0 (socket says 172.21.15.69:60260) 2024-03-19T00:59:07.350 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.454+0000 7f81f3e00700 1 -- 172.21.15.69:0/592548718 learned_addr learned my addr 172.21.15.69:0/592548718 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:07.350 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.454+0000 7f81f3e00700 1 -- 172.21.15.69:0/592548718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f419c1a0 con 0x7f81f4108720 2024-03-19T00:59:07.350 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.454+0000 7f81f3e00700 1 --2- 172.21.15.69:0/592548718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f419bc60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f81e4011b60 tx=0x7f81e4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.454+0000 7f81f0c00700 1 -- 172.21.15.69:0/592548718 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81e4003b20 con 0x7f81f4108720 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.454+0000 7f81f0c00700 1 -- 172.21.15.69:0/592548718 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f81e4003cc0 con 0x7f81f4108720 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.454+0000 7f81fae00700 1 -- 172.21.15.69:0/592548718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f419c430 con 0x7f81f4108720 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.454+0000 7f81f0c00700 1 -- 172.21.15.69:0/592548718 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81e4024840 con 0x7f81f4108720 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.455+0000 7f81fae00700 1 -- 172.21.15.69:0/592548718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f419f0d0 con 0x7f81f4108720 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.456+0000 7f81f0c00700 1 -- 172.21.15.69:0/592548718 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f81e4022020 con 0x7f81f4108720 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.456+0000 7f81fae00700 1 -- 172.21.15.69:0/592548718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f4063940 con 0x7f81f4108720 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.456+0000 7f81f0c00700 1 --2- 172.21.15.69:0/592548718 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f81d8038f70 0x7f81d803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.456+0000 7f81f0c00700 1 -- 172.21.15.69:0/592548718 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f81e4011e00 con 0x7f81f4108720 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.457+0000 7f81f3400700 1 --2- 172.21.15.69:0/592548718 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f81d8038f70 0x7f81d803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.457+0000 7f81f3400700 1 --2- 172.21.15.69:0/592548718 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f81d8038f70 0x7f81d803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f81e0006fd0 tx=0x7f81e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.462+0000 7f81f0c00700 1 -- 172.21.15.69:0/592548718 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f81e404c950 con 0x7f81f4108720 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.704+0000 7f81fae00700 1 -- 172.21.15.69:0/592548718 --> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f81f4103350 con 0x7f81d8038f70 2024-03-19T00:59:07.351 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.714+0000 7f81f0c00700 1 -- 172.21.15.69:0/592548718 <== mgr.14120 v2:172.21.15.69:6800/4073584559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f81f4103350 con 0x7f81d8038f70 2024-03-19T00:59:07.352 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.717+0000 7f81fae00700 1 -- 172.21.15.69:0/592548718 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f81d8038f70 msgr2=0x7f81d803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:07.352 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.717+0000 7f81fae00700 1 --2- 172.21.15.69:0/592548718 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f81d8038f70 0x7f81d803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f81e0006fd0 tx=0x7f81e0006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:07.352 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.717+0000 7f81fae00700 1 -- 172.21.15.69:0/592548718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 msgr2=0x7f81f419bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:07.352 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.717+0000 7f81fae00700 1 --2- 172.21.15.69:0/592548718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f419bc60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f81e4011b60 tx=0x7f81e4011c40 comp rx=0 tx=0).stop 2024-03-19T00:59:07.352 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.717+0000 7f81fae00700 1 -- 172.21.15.69:0/592548718 shutdown_connections 2024-03-19T00:59:07.352 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.717+0000 7f81fae00700 1 --2- 172.21.15.69:0/592548718 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f81d8038f70 0x7f81d803b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:07.352 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.717+0000 7f81fae00700 1 --2- 172.21.15.69:0/592548718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81f4108720 0x7f81f419bc60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:07.352 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.717+0000 7f81fae00700 1 -- 172.21.15.69:0/592548718 >> 172.21.15.69:0/592548718 conn(0x7f81f4102130 msgr2=0x7f81f4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:07.352 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.718+0000 7f81fae00700 1 -- 172.21.15.69:0/592548718 shutdown_connections 2024-03-19T00:59:07.352 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:06.718+0000 7f81fae00700 1 -- 172.21.15.69:0/592548718 wait complete. 2024-03-19T00:59:07.352 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mgr service... 2024-03-19T00:59:07.883 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:07 smithi069 ceph-mon[31727]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:59:07.883 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:07 smithi069 ceph-mon[31727]: Saving service mon spec with placement count:5 2024-03-19T00:59:07.883 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:07 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:59:09.330 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-19T00:59:09.330 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.373+0000 7f04a4600700 1 Processor -- start 2024-03-19T00:59:09.330 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.373+0000 7f04a4600700 1 -- start start 2024-03-19T00:59:09.330 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.374+0000 7f04a4600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c082040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:09.330 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.374+0000 7f04a4600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f049c082610 con 0x7f049c081c20 2024-03-19T00:59:09.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.374+0000 7f04a1c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c082040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:09.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.374+0000 7f04a1c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c082040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60270/0 (socket says 172.21.15.69:60270) 2024-03-19T00:59:09.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.374+0000 7f04a1c00700 1 -- 172.21.15.69:0/3589587923 learned_addr learned my addr 172.21.15.69:0/3589587923 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:09.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.375+0000 7f04a1c00700 1 -- 172.21.15.69:0/3589587923 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c10f580 con 0x7f049c081c20 2024-03-19T00:59:09.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.375+0000 7f04a1c00700 1 --2- 172.21.15.69:0/3589587923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c082040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f04900098a0 tx=0x7f0490009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df9635f07f21fec server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:09.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.375+0000 7f049be00700 1 -- 172.21.15.69:0/3589587923 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0490004160 con 0x7f049c081c20 2024-03-19T00:59:09.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.376+0000 7f049be00700 1 -- 172.21.15.69:0/3589587923 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0490004300 con 0x7f049c081c20 2024-03-19T00:59:09.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.376+0000 7f049be00700 1 -- 172.21.15.69:0/3589587923 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0490011b50 con 0x7f049c081c20 2024-03-19T00:59:09.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.376+0000 7f04a4600700 1 -- 172.21.15.69:0/3589587923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 msgr2=0x7f049c082040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:09.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.376+0000 7f04a4600700 1 --2- 172.21.15.69:0/3589587923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c082040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f04900098a0 tx=0x7f0490009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:09.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.376+0000 7f04a4600700 1 -- 172.21.15.69:0/3589587923 shutdown_connections 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.376+0000 7f04a4600700 1 --2- 172.21.15.69:0/3589587923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c082040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.376+0000 7f04a4600700 1 -- 172.21.15.69:0/3589587923 >> 172.21.15.69:0/3589587923 conn(0x7f049c07d3c0 msgr2=0x7f049c07f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.376+0000 7f04a4600700 1 -- 172.21.15.69:0/3589587923 shutdown_connections 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.376+0000 7f04a4600700 1 -- 172.21.15.69:0/3589587923 wait complete. 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.377+0000 7f04a4600700 1 Processor -- start 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.378+0000 7f04a4600700 1 -- start start 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.378+0000 7f04a4600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c1a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.378+0000 7f04a4600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f049c1127b0 con 0x7f049c081c20 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.378+0000 7f04a1c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c1a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.378+0000 7f04a1c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c1a2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60272/0 (socket says 172.21.15.69:60272) 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.378+0000 7f04a1c00700 1 -- 172.21.15.69:0/2278421197 learned_addr learned my addr 172.21.15.69:0/2278421197 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.378+0000 7f04a1c00700 1 -- 172.21.15.69:0/2278421197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c1a2750 con 0x7f049c081c20 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.379+0000 7f04a1c00700 1 --2- 172.21.15.69:0/2278421197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c1a2210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0490000c00 tx=0x7f0490011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.379+0000 7f049a000700 1 -- 172.21.15.69:0/2278421197 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0490003b20 con 0x7f049c081c20 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.379+0000 7f049a000700 1 -- 172.21.15.69:0/2278421197 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0490003cc0 con 0x7f049c081c20 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.379+0000 7f04a4600700 1 -- 172.21.15.69:0/2278421197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049c1a29e0 con 0x7f049c081c20 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.379+0000 7f049a000700 1 -- 172.21.15.69:0/2278421197 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0490024910 con 0x7f049c081c20 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.379+0000 7f04a4600700 1 -- 172.21.15.69:0/2278421197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f049c1a56d0 con 0x7f049c081c20 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.380+0000 7f049a000700 1 -- 172.21.15.69:0/2278421197 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f0490022020 con 0x7f049c081c20 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.380+0000 7f04a4600700 1 -- 172.21.15.69:0/2278421197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f049c112930 con 0x7f049c081c20 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.381+0000 7f049a000700 1 --2- 172.21.15.69:0/2278421197 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f0484038f70 0x7f048403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.381+0000 7f049a000700 1 -- 172.21.15.69:0/2278421197 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0490011e00 con 0x7f049c081c20 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.381+0000 7f04a1200700 1 --2- 172.21.15.69:0/2278421197 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f0484038f70 0x7f048403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.382+0000 7f04a1200700 1 --2- 172.21.15.69:0/2278421197 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f0484038f70 0x7f048403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0488006fd0 tx=0x7f0488006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.386+0000 7f049a000700 1 -- 172.21.15.69:0/2278421197 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f049001acd0 con 0x7f049c081c20 2024-03-19T00:59:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.627+0000 7f04a4600700 1 -- 172.21.15.69:0/2278421197 --> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f049c07e5c0 con 0x7f0484038f70 2024-03-19T00:59:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.635+0000 7f049a000700 1 -- 172.21.15.69:0/2278421197 <== mgr.14120 v2:172.21.15.69:6800/4073584559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f049c07e5c0 con 0x7f0484038f70 2024-03-19T00:59:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.637+0000 7f04a4600700 1 -- 172.21.15.69:0/2278421197 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f0484038f70 msgr2=0x7f048403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.637+0000 7f04a4600700 1 --2- 172.21.15.69:0/2278421197 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f0484038f70 0x7f048403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0488006fd0 tx=0x7f0488006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.637+0000 7f04a4600700 1 -- 172.21.15.69:0/2278421197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 msgr2=0x7f049c1a2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.637+0000 7f04a4600700 1 --2- 172.21.15.69:0/2278421197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c1a2210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0490000c00 tx=0x7f0490011c10 comp rx=0 tx=0).stop 2024-03-19T00:59:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.638+0000 7f04a4600700 1 -- 172.21.15.69:0/2278421197 shutdown_connections 2024-03-19T00:59:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.638+0000 7f04a4600700 1 --2- 172.21.15.69:0/2278421197 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f0484038f70 0x7f048403b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.638+0000 7f04a4600700 1 --2- 172.21.15.69:0/2278421197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f049c081c20 0x7f049c1a2210 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.638+0000 7f04a4600700 1 -- 172.21.15.69:0/2278421197 >> 172.21.15.69:0/2278421197 conn(0x7f049c07d3c0 msgr2=0x7f049c07deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:09.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.638+0000 7f04a4600700 1 -- 172.21.15.69:0/2278421197 shutdown_connections 2024-03-19T00:59:09.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:08.638+0000 7f04a4600700 1 -- 172.21.15.69:0/2278421197 wait complete. 2024-03-19T00:59:09.900 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:09 smithi069 ceph-mon[31727]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:59:09.900 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:09 smithi069 ceph-mon[31727]: Saving service mgr spec with placement count:2 2024-03-19T00:59:09.900 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:09 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:59:09.900 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:09 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:59:10.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:10 smithi069 ceph-mon[31727]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:59:10.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:10 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:59:10.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:10 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/730456519' entity='client.admin' 2024-03-19T00:59:10.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.899+0000 7f1f75a00700 1 Processor -- start 2024-03-19T00:59:10.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.899+0000 7f1f75a00700 1 -- start start 2024-03-19T00:59:10.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.899+0000 7f1f75a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f70108130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:10.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.899+0000 7f1f75a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70108700 con 0x7f1f70107d10 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.900+0000 7f1f6ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f70108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.900+0000 7f1f6ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f70108130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60288/0 (socket says 172.21.15.69:60288) 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.900+0000 7f1f6ea00700 1 -- 172.21.15.69:0/835074061 learned_addr learned my addr 172.21.15.69:0/835074061 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.900+0000 7f1f6ea00700 1 -- 172.21.15.69:0/835074061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f70108e70 con 0x7f1f70107d10 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.901+0000 7f1f6ea00700 1 --2- 172.21.15.69:0/835074061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f70108130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1f600098a0 tx=0x7f1f60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3026559408e02d78 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.901+0000 7f1f6d600700 1 -- 172.21.15.69:0/835074061 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f60004160 con 0x7f1f70107d10 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.901+0000 7f1f6d600700 1 -- 172.21.15.69:0/835074061 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f60004300 con 0x7f1f70107d10 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.901+0000 7f1f6d600700 1 -- 172.21.15.69:0/835074061 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f60011b20 con 0x7f1f70107d10 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.902+0000 7f1f75a00700 1 -- 172.21.15.69:0/835074061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 msgr2=0x7f1f70108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.902+0000 7f1f75a00700 1 --2- 172.21.15.69:0/835074061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f70108130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1f600098a0 tx=0x7f1f60009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.902+0000 7f1f75a00700 1 -- 172.21.15.69:0/835074061 shutdown_connections 2024-03-19T00:59:10.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.902+0000 7f1f75a00700 1 --2- 172.21.15.69:0/835074061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f70108130 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.902+0000 7f1f75a00700 1 -- 172.21.15.69:0/835074061 >> 172.21.15.69:0/835074061 conn(0x7f1f701032d0 msgr2=0x7f1f701056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.902+0000 7f1f75a00700 1 -- 172.21.15.69:0/835074061 shutdown_connections 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.902+0000 7f1f75a00700 1 -- 172.21.15.69:0/835074061 wait complete. 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.903+0000 7f1f75a00700 1 Processor -- start 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.903+0000 7f1f75a00700 1 -- start start 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.903+0000 7f1f75a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f7019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.903+0000 7f1f75a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7010c220 con 0x7f1f70107d10 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.904+0000 7f1f6ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f7019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.904+0000 7f1f6ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f7019de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60302/0 (socket says 172.21.15.69:60302) 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.904+0000 7f1f6ea00700 1 -- 172.21.15.69:0/730456519 learned_addr learned my addr 172.21.15.69:0/730456519 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.904+0000 7f1f6ea00700 1 -- 172.21.15.69:0/730456519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7019e340 con 0x7f1f70107d10 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.904+0000 7f1f6ea00700 1 --2- 172.21.15.69:0/730456519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f7019de00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1f600117e0 tx=0x7f1f60011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.905+0000 7f1f67400700 1 -- 172.21.15.69:0/730456519 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f60003bf0 con 0x7f1f70107d10 2024-03-19T00:59:10.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.905+0000 7f1f67400700 1 -- 172.21.15.69:0/730456519 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f6001a910 con 0x7f1f70107d10 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.905+0000 7f1f75a00700 1 -- 172.21.15.69:0/730456519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7019e5d0 con 0x7f1f70107d10 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.905+0000 7f1f67400700 1 -- 172.21.15.69:0/730456519 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f60024970 con 0x7f1f70107d10 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.905+0000 7f1f75a00700 1 -- 172.21.15.69:0/730456519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f7019ea30 con 0x7f1f70107d10 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.906+0000 7f1f67400700 1 -- 172.21.15.69:0/730456519 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f1f60022020 con 0x7f1f70107d10 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.906+0000 7f1f75a00700 1 -- 172.21.15.69:0/730456519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f70063940 con 0x7f1f70107d10 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.907+0000 7f1f67400700 1 --2- 172.21.15.69:0/730456519 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f1f54038f70 0x7f1f5403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.907+0000 7f1f67400700 1 -- 172.21.15.69:0/730456519 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1f6004d300 con 0x7f1f70107d10 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.907+0000 7f1f6e000700 1 --2- 172.21.15.69:0/730456519 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f1f54038f70 0x7f1f5403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.908+0000 7f1f6e000700 1 --2- 172.21.15.69:0/730456519 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f1f54038f70 0x7f1f5403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:09.912+0000 7f1f67400700 1 -- 172.21.15.69:0/730456519 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1f6004cc50 con 0x7f1f70107d10 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.132+0000 7f1f75a00700 1 -- 172.21.15.69:0/730456519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f1f701a12c0 con 0x7f1f70107d10 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.136+0000 7f1f67400700 1 -- 172.21.15.69:0/730456519 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f1f6002d910 con 0x7f1f70107d10 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.140+0000 7f1f75a00700 1 -- 172.21.15.69:0/730456519 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f1f54038f70 msgr2=0x7f1f5403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:10.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.140+0000 7f1f75a00700 1 --2- 172.21.15.69:0/730456519 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f1f54038f70 0x7f1f5403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:10.861 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.140+0000 7f1f75a00700 1 -- 172.21.15.69:0/730456519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 msgr2=0x7f1f7019de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:10.861 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.140+0000 7f1f75a00700 1 --2- 172.21.15.69:0/730456519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f7019de00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1f600117e0 tx=0x7f1f60011ba0 comp rx=0 tx=0).stop 2024-03-19T00:59:10.861 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.140+0000 7f1f75a00700 1 -- 172.21.15.69:0/730456519 shutdown_connections 2024-03-19T00:59:10.861 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.140+0000 7f1f75a00700 1 --2- 172.21.15.69:0/730456519 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f1f54038f70 0x7f1f5403b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:10.861 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.140+0000 7f1f75a00700 1 --2- 172.21.15.69:0/730456519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f70107d10 0x7f1f7019de00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:10.861 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.140+0000 7f1f75a00700 1 -- 172.21.15.69:0/730456519 >> 172.21.15.69:0/730456519 conn(0x7f1f701032d0 msgr2=0x7f1f70103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:10.861 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.140+0000 7f1f75a00700 1 -- 172.21.15.69:0/730456519 shutdown_connections 2024-03-19T00:59:10.861 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:10.141+0000 7f1f75a00700 1 -- 172.21.15.69:0/730456519 wait complete. 2024-03-19T00:59:12.756 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:12 smithi069 ceph-mon[31727]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:59:12.756 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:12 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/797673681' entity='client.admin' 2024-03-19T00:59:12.756 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:12 smithi069 ceph-mon[31727]: from='mgr.14120 172.21.15.69:0/3148857341' entity='mgr.x' 2024-03-19T00:59:12.767 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.369+0000 7f391ae00700 1 Processor -- start 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.370+0000 7f391ae00700 1 -- start start 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.370+0000 7f391ae00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f3914108100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.370+0000 7f391ae00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3914108640 con 0x7f3914107ce0 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.371+0000 7f3913e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f3914108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.371+0000 7f3913e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f3914108100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60308/0 (socket says 172.21.15.69:60308) 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.371+0000 7f3913e00700 1 -- 172.21.15.69:0/2146773997 learned_addr learned my addr 172.21.15.69:0/2146773997 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.371+0000 7f3913e00700 1 -- 172.21.15.69:0/2146773997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39141087c0 con 0x7f3914107ce0 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.372+0000 7f3913e00700 1 --2- 172.21.15.69:0/2146773997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f3914108100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39040098a0 tx=0x7f3904009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47c25e7d34bc573 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.372+0000 7f3912a00700 1 -- 172.21.15.69:0/2146773997 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3904004160 con 0x7f3914107ce0 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.372+0000 7f3912a00700 1 -- 172.21.15.69:0/2146773997 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3904004300 con 0x7f3914107ce0 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.372+0000 7f3912a00700 1 -- 172.21.15.69:0/2146773997 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3904011b20 con 0x7f3914107ce0 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.373+0000 7f391ae00700 1 -- 172.21.15.69:0/2146773997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 msgr2=0x7f3914108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.373+0000 7f391ae00700 1 --2- 172.21.15.69:0/2146773997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f3914108100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39040098a0 tx=0x7f3904009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:12.768 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.373+0000 7f391ae00700 1 -- 172.21.15.69:0/2146773997 shutdown_connections 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.373+0000 7f391ae00700 1 --2- 172.21.15.69:0/2146773997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f3914108100 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.373+0000 7f391ae00700 1 -- 172.21.15.69:0/2146773997 >> 172.21.15.69:0/2146773997 conn(0x7f39141033a0 msgr2=0x7f39141057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.373+0000 7f391ae00700 1 -- 172.21.15.69:0/2146773997 shutdown_connections 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.373+0000 7f391ae00700 1 -- 172.21.15.69:0/2146773997 wait complete. 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.374+0000 7f391ae00700 1 Processor -- start 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.375+0000 7f391ae00700 1 -- start start 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.375+0000 7f391ae00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f391407e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.375+0000 7f391ae00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f391410c150 con 0x7f3914107ce0 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.375+0000 7f3913e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f391407e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.375+0000 7f3913e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f391407e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60324/0 (socket says 172.21.15.69:60324) 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.375+0000 7f3913e00700 1 -- 172.21.15.69:0/797673681 learned_addr learned my addr 172.21.15.69:0/797673681 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.376+0000 7f3913e00700 1 -- 172.21.15.69:0/797673681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391407e910 con 0x7f3914107ce0 2024-03-19T00:59:12.769 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.376+0000 7f3913e00700 1 --2- 172.21.15.69:0/797673681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f391407e3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3904011b60 tx=0x7f3904011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.376+0000 7f3910c00700 1 -- 172.21.15.69:0/797673681 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3904003b60 con 0x7f3914107ce0 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.376+0000 7f3910c00700 1 -- 172.21.15.69:0/797673681 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390401a910 con 0x7f3914107ce0 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.376+0000 7f391ae00700 1 -- 172.21.15.69:0/797673681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391407cb50 con 0x7f3914107ce0 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.376+0000 7f3910c00700 1 -- 172.21.15.69:0/797673681 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3904024930 con 0x7f3914107ce0 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.376+0000 7f391ae00700 1 -- 172.21.15.69:0/797673681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391407cfb0 con 0x7f3914107ce0 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.378+0000 7f391ae00700 1 -- 172.21.15.69:0/797673681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391410c2d0 con 0x7f3914107ce0 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.378+0000 7f3910c00700 1 -- 172.21.15.69:0/797673681 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f3904022020 con 0x7f3914107ce0 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.379+0000 7f3910c00700 1 --2- 172.21.15.69:0/797673681 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f38f8038f20 0x7f38f803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.379+0000 7f3910c00700 1 -- 172.21.15.69:0/797673681 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f390404d140 con 0x7f3914107ce0 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.379+0000 7f3913400700 1 --2- 172.21.15.69:0/797673681 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f38f8038f20 0x7f38f803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.380+0000 7f3913400700 1 --2- 172.21.15.69:0/797673681 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f38f8038f20 0x7f38f803b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3900006fd0 tx=0x7f3900006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:12.770 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.384+0000 7f3910c00700 1 -- 172.21.15.69:0/797673681 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f390400b3e0 con 0x7f3914107ce0 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.610+0000 7f391ae00700 1 -- 172.21.15.69:0/797673681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f3914063940 con 0x7f3914107ce0 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.614+0000 7f3910c00700 1 -- 172.21.15.69:0/797673681 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f3904016260 con 0x7f3914107ce0 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.617+0000 7f391ae00700 1 -- 172.21.15.69:0/797673681 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f38f8038f20 msgr2=0x7f38f803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.617+0000 7f391ae00700 1 --2- 172.21.15.69:0/797673681 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f38f8038f20 0x7f38f803b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3900006fd0 tx=0x7f3900006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.617+0000 7f391ae00700 1 -- 172.21.15.69:0/797673681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 msgr2=0x7f391407e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.617+0000 7f391ae00700 1 --2- 172.21.15.69:0/797673681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f391407e3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3904011b60 tx=0x7f3904011c40 comp rx=0 tx=0).stop 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.618+0000 7f391ae00700 1 -- 172.21.15.69:0/797673681 shutdown_connections 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.618+0000 7f391ae00700 1 --2- 172.21.15.69:0/797673681 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f38f8038f20 0x7f38f803b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.618+0000 7f391ae00700 1 --2- 172.21.15.69:0/797673681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3914107ce0 0x7f391407e3d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.618+0000 7f391ae00700 1 -- 172.21.15.69:0/797673681 >> 172.21.15.69:0/797673681 conn(0x7f39141033a0 msgr2=0x7f3914103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.618+0000 7f391ae00700 1 -- 172.21.15.69:0/797673681 shutdown_connections 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:11.618+0000 7f391ae00700 1 -- 172.21.15.69:0/797673681 wait complete. 2024-03-19T00:59:12.771 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling the dashboard module... 2024-03-19T00:59:15.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:14 smithi069 ceph-mon[31727]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:59:15.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:14 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3765658998' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-19T00:59:15.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:14 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: ignoring --setuser ceph since I am not root 2024-03-19T00:59:15.033 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:14 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: ignoring --setgroup ceph since I am not root 2024-03-19T00:59:15.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.329+0000 7fc7ec400700 1 Processor -- start 2024-03-19T00:59:15.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.329+0000 7fc7ec400700 1 -- start start 2024-03-19T00:59:15.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.330+0000 7fc7ec400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e410ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:15.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.330+0000 7fc7ec400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e407be70 con 0x7fc7e4108780 2024-03-19T00:59:15.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.330+0000 7fc7e9a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e410ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:15.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.330+0000 7fc7e9a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e410ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60334/0 (socket says 172.21.15.69:60334) 2024-03-19T00:59:15.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.330+0000 7fc7e9a00700 1 -- 172.21.15.69:0/3394238931 learned_addr learned my addr 172.21.15.69:0/3394238931 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:15.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.331+0000 7fc7e9a00700 1 -- 172.21.15.69:0/3394238931 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e410b0b0 con 0x7fc7e4108780 2024-03-19T00:59:15.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.331+0000 7fc7e9a00700 1 --2- 172.21.15.69:0/3394238931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e410ab70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80098a0 tx=0x7fc7d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5383bd910e175b2d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:15.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.331+0000 7fc7e3e00700 1 -- 172.21.15.69:0/3394238931 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7d8004160 con 0x7fc7e4108780 2024-03-19T00:59:15.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.332+0000 7fc7e3e00700 1 -- 172.21.15.69:0/3394238931 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d8004300 con 0x7fc7e4108780 2024-03-19T00:59:15.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.332+0000 7fc7e3e00700 1 -- 172.21.15.69:0/3394238931 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7d8011b20 con 0x7fc7e4108780 2024-03-19T00:59:15.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.332+0000 7fc7ec400700 1 -- 172.21.15.69:0/3394238931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 msgr2=0x7fc7e410ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:15.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.332+0000 7fc7ec400700 1 --2- 172.21.15.69:0/3394238931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e410ab70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80098a0 tx=0x7fc7d8009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:15.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.332+0000 7fc7ec400700 1 -- 172.21.15.69:0/3394238931 shutdown_connections 2024-03-19T00:59:15.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.332+0000 7fc7ec400700 1 --2- 172.21.15.69:0/3394238931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e410ab70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:15.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.332+0000 7fc7ec400700 1 -- 172.21.15.69:0/3394238931 >> 172.21.15.69:0/3394238931 conn(0x7fc7e4102130 msgr2=0x7fc7e4104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:15.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.333+0000 7fc7ec400700 1 -- 172.21.15.69:0/3394238931 shutdown_connections 2024-03-19T00:59:15.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.333+0000 7fc7ec400700 1 -- 172.21.15.69:0/3394238931 wait complete. 2024-03-19T00:59:15.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.334+0000 7fc7ec400700 1 Processor -- start 2024-03-19T00:59:15.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.334+0000 7fc7ec400700 1 -- start start 2024-03-19T00:59:15.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.334+0000 7fc7ec400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:15.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.334+0000 7fc7ec400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e410c0c0 con 0x7fc7e4108780 2024-03-19T00:59:15.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.334+0000 7fc7e9a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:15.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.334+0000 7fc7e9a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e419bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60344/0 (socket says 172.21.15.69:60344) 2024-03-19T00:59:15.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.334+0000 7fc7e9a00700 1 -- 172.21.15.69:0/3765658998 learned_addr learned my addr 172.21.15.69:0/3765658998 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:15.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.335+0000 7fc7e9a00700 1 -- 172.21.15.69:0/3765658998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e419c100 con 0x7fc7e4108780 2024-03-19T00:59:15.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.335+0000 7fc7e9a00700 1 --2- 172.21.15.69:0/3765658998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e419bbc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80117e0 tx=0x7fc7d8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:15.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.335+0000 7fc7e2000700 1 -- 172.21.15.69:0/3765658998 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7d8003ba0 con 0x7fc7e4108780 2024-03-19T00:59:15.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.335+0000 7fc7e2000700 1 -- 172.21.15.69:0/3765658998 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d801a910 con 0x7fc7e4108780 2024-03-19T00:59:15.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.335+0000 7fc7e2000700 1 -- 172.21.15.69:0/3765658998 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7d80248f0 con 0x7fc7e4108780 2024-03-19T00:59:15.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.336+0000 7fc7ec400700 1 -- 172.21.15.69:0/3765658998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e419c390 con 0x7fc7e4108780 2024-03-19T00:59:15.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.336+0000 7fc7ec400700 1 -- 172.21.15.69:0/3765658998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e419c7f0 con 0x7fc7e4108780 2024-03-19T00:59:15.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.337+0000 7fc7ec400700 1 -- 172.21.15.69:0/3765658998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e4063940 con 0x7fc7e4108780 2024-03-19T00:59:15.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.337+0000 7fc7e2000700 1 -- 172.21.15.69:0/3765658998 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fc7d8022020 con 0x7fc7e4108780 2024-03-19T00:59:15.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.338+0000 7fc7e2000700 1 --2- 172.21.15.69:0/3765658998 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fc7cc038f70 0x7fc7cc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:15.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.338+0000 7fc7e2000700 1 -- 172.21.15.69:0/3765658998 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc7d804d360 con 0x7fc7e4108780 2024-03-19T00:59:15.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.338+0000 7fc7e9000700 1 --2- 172.21.15.69:0/3765658998 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fc7cc038f70 0x7fc7cc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:15.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.339+0000 7fc7e9000700 1 --2- 172.21.15.69:0/3765658998 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fc7cc038f70 0x7fc7cc03b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0006fd0 tx=0x7fc7d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:15.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.344+0000 7fc7e2000700 1 -- 172.21.15.69:0/3765658998 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc7d801ab70 con 0x7fc7e4108780 2024-03-19T00:59:15.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:13.629+0000 7fc7ec400700 1 -- 172.21.15.69:0/3765658998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fc7e419f280 con 0x7fc7e4108780 2024-03-19T00:59:15.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.626+0000 7fc7e2000700 1 -- 172.21.15.69:0/3765658998 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fc7d804ccb0 con 0x7fc7e4108780 2024-03-19T00:59:15.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.630+0000 7fc7e2000700 1 -- 172.21.15.69:0/3765658998 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7fc7d802d3f0 con 0x7fc7e4108780 2024-03-19T00:59:15.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.630+0000 7fc7ec400700 1 -- 172.21.15.69:0/3765658998 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fc7cc038f70 msgr2=0x7fc7cc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:15.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.630+0000 7fc7ec400700 1 --2- 172.21.15.69:0/3765658998 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fc7cc038f70 0x7fc7cc03b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0006fd0 tx=0x7fc7d0006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:15.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.630+0000 7fc7ec400700 1 -- 172.21.15.69:0/3765658998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 msgr2=0x7fc7e419bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:15.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.630+0000 7fc7ec400700 1 --2- 172.21.15.69:0/3765658998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e419bbc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80117e0 tx=0x7fc7d8011ba0 comp rx=0 tx=0).stop 2024-03-19T00:59:15.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.632+0000 7fc7ec400700 1 -- 172.21.15.69:0/3765658998 shutdown_connections 2024-03-19T00:59:15.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.632+0000 7fc7ec400700 1 --2- 172.21.15.69:0/3765658998 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fc7cc038f70 0x7fc7cc03b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:15.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.632+0000 7fc7ec400700 1 --2- 172.21.15.69:0/3765658998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7e4108780 0x7fc7e419bbc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:15.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.632+0000 7fc7ec400700 1 -- 172.21.15.69:0/3765658998 >> 172.21.15.69:0/3765658998 conn(0x7fc7e4102130 msgr2=0x7fc7e4102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:15.428 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.632+0000 7fc7ec400700 1 -- 172.21.15.69:0/3765658998 shutdown_connections 2024-03-19T00:59:15.428 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:14.632+0000 7fc7ec400700 1 -- 172.21.15.69:0/3765658998 wait complete. 2024-03-19T00:59:15.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:15 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3765658998' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-19T00:59:15.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:15 smithi069 ceph-mon[31727]: mgrmap e9: x(active, since 25s) 2024-03-19T00:59:15.688 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:15 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:15.439+0000 7f2535621200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T00:59:15.688 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:15 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:15.560+0000 7f2535621200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T00:59:16.630 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:16 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:16.378+0000 7f2535621200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T00:59:16.630 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:16 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:16.497+0000 7f2535621200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T00:59:16.915 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:16 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3204514655' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-19T00:59:16.915 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:16 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:16.681+0000 7f2535621200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T00:59:16.916 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:16 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:16.803+0000 7f2535621200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T00:59:16.916 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:16 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:16.914+0000 7f2535621200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T00:59:16.979 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:59:16.979 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-19T00:59:16.979 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-19T00:59:16.979 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.995+0000 7fcdad200700 1 Processor -- start 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.995+0000 7fcdad200700 1 -- start start 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.995+0000 7fcdad200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.995+0000 7fcdad200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcda807c840 con 0x7fcda807de00 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.996+0000 7fcda5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.996+0000 7fcda5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39574/0 (socket says 172.21.15.69:39574) 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.996+0000 7fcda5e00700 1 -- 172.21.15.69:0/1249627458 learned_addr learned my addr 172.21.15.69:0/1249627458 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.996+0000 7fcda5e00700 1 -- 172.21.15.69:0/1249627458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda807c9c0 con 0x7fcda807de00 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.997+0000 7fcda5e00700 1 --2- 172.21.15.69:0/1249627458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda807c300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd980098a0 tx=0x7fcd98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7e0ce0db09ada7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:16.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.997+0000 7fcda4a00700 1 -- 172.21.15.69:0/1249627458 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd98004160 con 0x7fcda807de00 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.997+0000 7fcda4a00700 1 -- 172.21.15.69:0/1249627458 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd98004300 con 0x7fcda807de00 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.997+0000 7fcda4a00700 1 -- 172.21.15.69:0/1249627458 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd98011b20 con 0x7fcda807de00 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.998+0000 7fcdad200700 1 -- 172.21.15.69:0/1249627458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 msgr2=0x7fcda807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.998+0000 7fcdad200700 1 --2- 172.21.15.69:0/1249627458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda807c300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd980098a0 tx=0x7fcd98009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.998+0000 7fcdad200700 1 -- 172.21.15.69:0/1249627458 shutdown_connections 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.998+0000 7fcdad200700 1 --2- 172.21.15.69:0/1249627458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda807c300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.998+0000 7fcdad200700 1 -- 172.21.15.69:0/1249627458 >> 172.21.15.69:0/1249627458 conn(0x7fcda81032d0 msgr2=0x7fcda81056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.998+0000 7fcdad200700 1 -- 172.21.15.69:0/1249627458 shutdown_connections 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.998+0000 7fcdad200700 1 -- 172.21.15.69:0/1249627458 wait complete. 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:15.999+0000 7fcdad200700 1 Processor -- start 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.000+0000 7fcdad200700 1 -- start start 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.000+0000 7fcdad200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.000+0000 7fcdad200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcda810e270 con 0x7fcda807de00 2024-03-19T00:59:16.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.000+0000 7fcda5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.000+0000 7fcda5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda819dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39590/0 (socket says 172.21.15.69:39590) 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.000+0000 7fcda5e00700 1 -- 172.21.15.69:0/3204514655 learned_addr learned my addr 172.21.15.69:0/3204514655 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.001+0000 7fcda5e00700 1 -- 172.21.15.69:0/3204514655 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda819e2d0 con 0x7fcda807de00 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.001+0000 7fcda5e00700 1 --2- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda819dd90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd980117e0 tx=0x7fcd98011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.001+0000 7fcd9ea00700 1 -- 172.21.15.69:0/3204514655 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd98003bf0 con 0x7fcda807de00 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.001+0000 7fcd9ea00700 1 -- 172.21.15.69:0/3204514655 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd9801a910 con 0x7fcda807de00 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.001+0000 7fcd9ea00700 1 -- 172.21.15.69:0/3204514655 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd98024970 con 0x7fcda807de00 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.001+0000 7fcdad200700 1 -- 172.21.15.69:0/3204514655 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda819e560 con 0x7fcda807de00 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.002+0000 7fcdad200700 1 -- 172.21.15.69:0/3204514655 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda81a1210 con 0x7fcda807de00 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.002+0000 7fcdad200700 1 -- 172.21.15.69:0/3204514655 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda8063940 con 0x7fcda807de00 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.003+0000 7fcd9ea00700 1 -- 172.21.15.69:0/3204514655 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7fcd98022020 con 0x7fcda807de00 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.003+0000 7fcd9ea00700 1 --2- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fcd8c038f80 0x7fcd8c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:16.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.003+0000 7fcd9ea00700 1 -- 172.21.15.69:0/3204514655 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd9804d2a0 con 0x7fcda807de00 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.003+0000 7fcda5400700 1 -- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fcd8c038f80 msgr2=0x7fcd8c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4073584559 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.004+0000 7fcda5400700 1 --2- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fcd8c038f80 0x7fcd8c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.009+0000 7fcd9ea00700 1 -- 172.21.15.69:0/3204514655 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcd98011e20 con 0x7fcda807de00 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.205+0000 7fcda5400700 1 -- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fcd8c038f80 msgr2=0x7fcd8c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4073584559 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.205+0000 7fcda5400700 1 --2- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fcd8c038f80 0x7fcd8c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.288+0000 7fcdad200700 1 -- 172.21.15.69:0/3204514655 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcda81a17e0 con 0x7fcda807de00 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.289+0000 7fcd9ea00700 1 -- 172.21.15.69:0/3204514655 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fcd9800b3e0 con 0x7fcda807de00 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.292+0000 7fcdad200700 1 -- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fcd8c038f80 msgr2=0x7fcd8c03b440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.292+0000 7fcdad200700 1 --2- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fcd8c038f80 0x7fcd8c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.292+0000 7fcdad200700 1 -- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 msgr2=0x7fcda819dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.292+0000 7fcdad200700 1 --2- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda819dd90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd980117e0 tx=0x7fcd98011c10 comp rx=0 tx=0).stop 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.292+0000 7fcdad200700 1 -- 172.21.15.69:0/3204514655 shutdown_connections 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.292+0000 7fcdad200700 1 --2- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7fcd8c038f80 0x7fcd8c03b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:16.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.292+0000 7fcdad200700 1 --2- 172.21.15.69:0/3204514655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcda807de00 0x7fcda819dd90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:16.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.292+0000 7fcdad200700 1 -- 172.21.15.69:0/3204514655 >> 172.21.15.69:0/3204514655 conn(0x7fcda81032d0 msgr2=0x7fcda8103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:16.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.292+0000 7fcdad200700 1 -- 172.21.15.69:0/3204514655 shutdown_connections 2024-03-19T00:59:16.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:16.292+0000 7fcdad200700 1 -- 172.21.15.69:0/3204514655 wait complete. 2024-03-19T00:59:16.984 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2024-03-19T00:59:16.984 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 9... 2024-03-19T00:59:17.526 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:17 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:17.264+0000 7f2535621200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T00:59:17.527 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:17 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:17.407+0000 7f2535621200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T00:59:17.527 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:17 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:17.525+0000 7f2535621200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T00:59:17.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:17 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:17.652+0000 7f2535621200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T00:59:18.159 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:17 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:17.868+0000 7f2535621200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T00:59:19.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:19 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:19.120+0000 7f2535621200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T00:59:19.704 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:19 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:19.290+0000 7f2535621200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T00:59:20.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:19 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:19.702+0000 7f2535621200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T00:59:20.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:19 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:19.820+0000 7f2535621200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T00:59:20.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:20 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:20.077+0000 7f2535621200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:59:20.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:20 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:20.322+0000 7f2535621200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T00:59:21.009 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:20 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:20.694+0000 7f2535621200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T00:59:21.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:21 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:21.008+0000 7f2535621200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T00:59:21.680 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:21 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:21.327+0000 7f2535621200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T00:59:22.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:22 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:22.393+0000 7f2535621200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T00:59:23.501 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:23.090+0000 7f2535621200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T00:59:23.501 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:23.219+0000 7f2535621200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T00:59:23.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T00:59:23.499+0000 7f2535621200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T00:59:23.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: Active manager daemon x restarted 2024-03-19T00:59:23.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: Activating manager daemon x 2024-03-19T00:59:23.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: osdmap e3: 0 total, 0 up, 0 in 2024-03-19T00:59:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: mgrmap e10: x(active, starting, since 0.00493691s) 2024-03-19T00:59:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T00:59:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T00:59:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T00:59:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T00:59:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T00:59:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: Manager daemon x is now available 2024-03-19T00:59:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:59:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:59:24.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T00:59:24.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T00:59:24.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:24 smithi069 ceph-mon[31727]: mgrmap e11: x(active, since 1.01207s) 2024-03-19T00:59:25.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-03-19T00:59:25.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-19T00:59:25.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-19T00:59:25.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-03-19T00:59:25.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.549+0000 7f8f5cc00700 1 Processor -- start 2024-03-19T00:59:25.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.549+0000 7f8f5cc00700 1 -- start start 2024-03-19T00:59:25.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.549+0000 7f8f5cc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:25.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.549+0000 7f8f5cc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5807c820 con 0x7f8f5807dde0 2024-03-19T00:59:25.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.550+0000 7f8f55e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:25.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.550+0000 7f8f55e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39592/0 (socket says 172.21.15.69:39592) 2024-03-19T00:59:25.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.550+0000 7f8f55e00700 1 -- 172.21.15.69:0/1435100009 learned_addr learned my addr 172.21.15.69:0/1435100009 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:25.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.550+0000 7f8f55e00700 1 -- 172.21.15.69:0/1435100009 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5807c9a0 con 0x7f8f5807dde0 2024-03-19T00:59:25.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.551+0000 7f8f55e00700 1 --2- 172.21.15.69:0/1435100009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5807c2e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f480098a0 tx=0x7f8f48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed3ba284178f7ab server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:25.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.551+0000 7f8f54a00700 1 -- 172.21.15.69:0/1435100009 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f48004160 con 0x7f8f5807dde0 2024-03-19T00:59:25.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.551+0000 7f8f54a00700 1 -- 172.21.15.69:0/1435100009 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f48004300 con 0x7f8f5807dde0 2024-03-19T00:59:25.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.551+0000 7f8f54a00700 1 -- 172.21.15.69:0/1435100009 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f48011b20 con 0x7f8f5807dde0 2024-03-19T00:59:25.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.551+0000 7f8f5cc00700 1 -- 172.21.15.69:0/1435100009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 msgr2=0x7f8f5807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:25.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.551+0000 7f8f5cc00700 1 --2- 172.21.15.69:0/1435100009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5807c2e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f480098a0 tx=0x7f8f48009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:25.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.552+0000 7f8f5cc00700 1 -- 172.21.15.69:0/1435100009 shutdown_connections 2024-03-19T00:59:25.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.552+0000 7f8f5cc00700 1 --2- 172.21.15.69:0/1435100009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5807c2e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:25.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.552+0000 7f8f5cc00700 1 -- 172.21.15.69:0/1435100009 >> 172.21.15.69:0/1435100009 conn(0x7f8f581032d0 msgr2=0x7f8f581056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:25.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.552+0000 7f8f5cc00700 1 -- 172.21.15.69:0/1435100009 shutdown_connections 2024-03-19T00:59:25.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.552+0000 7f8f5cc00700 1 -- 172.21.15.69:0/1435100009 wait complete. 2024-03-19T00:59:25.468 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.553+0000 7f8f5cc00700 1 Processor -- start 2024-03-19T00:59:25.468 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.553+0000 7f8f5cc00700 1 -- start start 2024-03-19T00:59:25.468 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.553+0000 7f8f5cc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:25.468 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.553+0000 7f8f5cc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5810e290 con 0x7f8f5807dde0 2024-03-19T00:59:25.468 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.553+0000 7f8f55e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.553+0000 7f8f55e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5819ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39604/0 (socket says 172.21.15.69:39604) 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.553+0000 7f8f55e00700 1 -- 172.21.15.69:0/844611399 learned_addr learned my addr 172.21.15.69:0/844611399 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.554+0000 7f8f55e00700 1 -- 172.21.15.69:0/844611399 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5819e2f0 con 0x7f8f5807dde0 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.554+0000 7f8f55e00700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5819ddb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f480117e0 tx=0x7f8f48011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.554+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f48003bf0 con 0x7f8f5807dde0 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.554+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f4801a910 con 0x7f8f5807dde0 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.554+0000 7f8f5cc00700 1 -- 172.21.15.69:0/844611399 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5819e580 con 0x7f8f5807dde0 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.554+0000 7f8f5cc00700 1 -- 172.21.15.69:0/844611399 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f581a1230 con 0x7f8f5807dde0 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.554+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f48024970 con 0x7f8f5807dde0 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.556+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f8f48022020 con 0x7f8f5807dde0 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.556+0000 7f8f4ea00700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 0x7f8f3c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.556+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 --> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8f3c03bb50 con 0x7f8f3c038f80 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.556+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8f4804d220 con 0x7f8f5807dde0 2024-03-19T00:59:25.469 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.556+0000 7f8f55400700 1 -- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 msgr2=0x7f8f3c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4073584559 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.556+0000 7f8f55400700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 0x7f8f3c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.757+0000 7f8f55400700 1 -- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 msgr2=0x7f8f3c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4073584559 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:17.757+0000 7f8f55400700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 0x7f8f3c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:18.158+0000 7f8f55400700 1 -- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 msgr2=0x7f8f3c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4073584559 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:18.158+0000 7f8f55400700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 0x7f8f3c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:18.959+0000 7f8f55400700 1 -- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 msgr2=0x7f8f3c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4073584559 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:18.959+0000 7f8f55400700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 0x7f8f3c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:20.560+0000 7f8f55400700 1 -- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 msgr2=0x7f8f3c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/4073584559 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:20.560+0000 7f8f55400700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 0x7f8f3c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:23.506+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 10) ==== 45886+0+0 (secure 0 0 0) 0x7f8f4802d770 con 0x7f8f5807dde0 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:23.506+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 msgr2=0x7f8f3c03b440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:23.506+0000 7f8f4ea00700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 0x7f8f3c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:25.470 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.510+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7f8f4802bcf0 con 0x7f8f5807dde0 2024-03-19T00:59:25.471 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.510+0000 7f8f4ea00700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8f3c03ca30 0x7f8f3c03ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:25.471 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.510+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8f48010ec0 con 0x7f8f3c03ca30 2024-03-19T00:59:25.471 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.511+0000 7f8f55400700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8f3c03ca30 0x7f8f3c03ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:25.471 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.512+0000 7f8f55400700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8f3c03ca30 0x7f8f3c03ee20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8f44003a10 tx=0x7f8f44009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:25.472 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.514+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f8f48010ec0 con 0x7f8f3c03ca30 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.522+0000 7f8f5cc00700 1 -- 172.21.15.69:0/844611399 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8f58063940 con 0x7f8f3c03ca30 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.522+0000 7f8f4ea00700 1 -- 172.21.15.69:0/844611399 <== mgr.14152 v2:172.21.15.69:6800/3562156769 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f8f58063940 con 0x7f8f3c03ca30 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.523+0000 7f8f5cc00700 1 -- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8f3c03ca30 msgr2=0x7f8f3c03ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.523+0000 7f8f5cc00700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8f3c03ca30 0x7f8f3c03ee20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8f44003a10 tx=0x7f8f44009270 comp rx=0 tx=0).stop 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.523+0000 7f8f5cc00700 1 -- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 msgr2=0x7f8f5819ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.523+0000 7f8f5cc00700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5819ddb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f480117e0 tx=0x7f8f48011c10 comp rx=0 tx=0).stop 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.523+0000 7f8f5cc00700 1 -- 172.21.15.69:0/844611399 shutdown_connections 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.523+0000 7f8f5cc00700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8f3c03ca30 0x7f8f3c03ee20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.523+0000 7f8f5cc00700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f5807dde0 0x7f8f5819ddb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.523+0000 7f8f5cc00700 1 --2- 172.21.15.69:0/844611399 >> [v2:172.21.15.69:6800/4073584559,v1:172.21.15.69:6801/4073584559] conn(0x7f8f3c038f80 0x7f8f3c03b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.523+0000 7f8f5cc00700 1 -- 172.21.15.69:0/844611399 >> 172.21.15.69:0/844611399 conn(0x7f8f581032d0 msgr2=0x7f8f58103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.523+0000 7f8f5cc00700 1 -- 172.21.15.69:0/844611399 shutdown_connections 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:24.523+0000 7f8f5cc00700 1 -- 172.21.15.69:0/844611399 wait complete. 2024-03-19T00:59:25.473 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 9 is available 2024-03-19T00:59:25.474 INFO:teuthology.orchestra.run.smithi069.stdout:Generating a dashboard self-signed certificate... 2024-03-19T00:59:26.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:25 smithi069 ceph-mon[31727]: [19/Mar/2024:00:59:24] ENGINE Bus STARTING 2024-03-19T00:59:26.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:25 smithi069 ceph-mon[31727]: [19/Mar/2024:00:59:24] ENGINE Serving on http://172.21.15.69:8765 2024-03-19T00:59:26.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:25 smithi069 ceph-mon[31727]: [19/Mar/2024:00:59:24] ENGINE Serving on https://172.21.15.69:7150 2024-03-19T00:59:26.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:25 smithi069 ceph-mon[31727]: [19/Mar/2024:00:59:24] ENGINE Bus STARTED 2024-03-19T00:59:26.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:25 smithi069 ceph-mon[31727]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-19T00:59:26.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:25 smithi069 ceph-mon[31727]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-19T00:59:26.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:26 smithi069 ceph-mon[31727]: mgrmap e12: x(active, since 2s) 2024-03-19T00:59:26.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:26 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:27.806 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.602+0000 7fe24c800700 1 Processor -- start 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.602+0000 7fe24c800700 1 -- start start 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.603+0000 7fe24c800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24410abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.603+0000 7fe24c800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe24407c670 con 0x7fe2441087c0 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.603+0000 7fe249e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24410abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.604+0000 7fe249e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24410abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37376/0 (socket says 172.21.15.69:37376) 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.604+0000 7fe249e00700 1 -- 172.21.15.69:0/1359830773 learned_addr learned my addr 172.21.15.69:0/1359830773 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.604+0000 7fe249e00700 1 -- 172.21.15.69:0/1359830773 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24410b0f0 con 0x7fe2441087c0 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.604+0000 7fe249e00700 1 --2- 172.21.15.69:0/1359830773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24410abb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2380098a0 tx=0x7fe238009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f24a506610abb09 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.604+0000 7fe248a00700 1 -- 172.21.15.69:0/1359830773 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe238004160 con 0x7fe2441087c0 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.605+0000 7fe248a00700 1 -- 172.21.15.69:0/1359830773 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe238004300 con 0x7fe2441087c0 2024-03-19T00:59:27.807 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.605+0000 7fe248a00700 1 -- 172.21.15.69:0/1359830773 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe238011b20 con 0x7fe2441087c0 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.605+0000 7fe24c800700 1 -- 172.21.15.69:0/1359830773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 msgr2=0x7fe24410abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.605+0000 7fe24c800700 1 --2- 172.21.15.69:0/1359830773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24410abb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2380098a0 tx=0x7fe238009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.605+0000 7fe24c800700 1 -- 172.21.15.69:0/1359830773 shutdown_connections 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.605+0000 7fe24c800700 1 --2- 172.21.15.69:0/1359830773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24410abb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.605+0000 7fe24c800700 1 -- 172.21.15.69:0/1359830773 >> 172.21.15.69:0/1359830773 conn(0x7fe2441021d0 msgr2=0x7fe2441045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.606+0000 7fe24c800700 1 -- 172.21.15.69:0/1359830773 shutdown_connections 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.606+0000 7fe24c800700 1 -- 172.21.15.69:0/1359830773 wait complete. 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.607+0000 7fe24c800700 1 Processor -- start 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.607+0000 7fe24c800700 1 -- start start 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.607+0000 7fe24c800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.607+0000 7fe24c800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe24410c090 con 0x7fe2441087c0 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.607+0000 7fe249e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:27.808 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.608+0000 7fe249e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24419dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37382/0 (socket says 172.21.15.69:37382) 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.608+0000 7fe249e00700 1 -- 172.21.15.69:0/2258567721 learned_addr learned my addr 172.21.15.69:0/2258567721 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.608+0000 7fe249e00700 1 -- 172.21.15.69:0/2258567721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24419e280 con 0x7fe2441087c0 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.608+0000 7fe249e00700 1 --2- 172.21.15.69:0/2258567721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24419dd40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe2380117e0 tx=0x7fe238011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.609+0000 7fe23ea00700 1 -- 172.21.15.69:0/2258567721 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe238003bf0 con 0x7fe2441087c0 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.609+0000 7fe23ea00700 1 -- 172.21.15.69:0/2258567721 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23801a910 con 0x7fe2441087c0 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.609+0000 7fe23ea00700 1 -- 172.21.15.69:0/2258567721 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe238024970 con 0x7fe2441087c0 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.609+0000 7fe24c800700 1 -- 172.21.15.69:0/2258567721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24419e510 con 0x7fe2441087c0 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.609+0000 7fe24c800700 1 -- 172.21.15.69:0/2258567721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24419e970 con 0x7fe2441087c0 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.610+0000 7fe24c800700 1 -- 172.21.15.69:0/2258567721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe244063940 con 0x7fe2441087c0 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.610+0000 7fe23ea00700 1 -- 172.21.15.69:0/2258567721 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7fe238022020 con 0x7fe2441087c0 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.611+0000 7fe23ea00700 1 --2- 172.21.15.69:0/2258567721 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe22c038f80 0x7fe22c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.611+0000 7fe23ea00700 1 -- 172.21.15.69:0/2258567721 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe23804d2d0 con 0x7fe2441087c0 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.611+0000 7fe249400700 1 --2- 172.21.15.69:0/2258567721 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe22c038f80 0x7fe22c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:27.809 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.612+0000 7fe249400700 1 --2- 172.21.15.69:0/2258567721 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe22c038f80 0x7fe22c03b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe234006fd0 tx=0x7fe234006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:27.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.617+0000 7fe23ea00700 1 -- 172.21.15.69:0/2258567721 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe238011e20 con 0x7fe2441087c0 2024-03-19T00:59:27.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:26.842+0000 7fe24c800700 1 -- 172.21.15.69:0/2258567721 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fe2441032c0 con 0x7fe22c038f80 2024-03-19T00:59:27.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:27.026+0000 7fe23ea00700 1 -- 172.21.15.69:0/2258567721 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fe2441032c0 con 0x7fe22c038f80 2024-03-19T00:59:27.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:27.028+0000 7fe24c800700 1 -- 172.21.15.69:0/2258567721 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe22c038f80 msgr2=0x7fe22c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:27.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:27.028+0000 7fe24c800700 1 --2- 172.21.15.69:0/2258567721 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe22c038f80 0x7fe22c03b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe234006fd0 tx=0x7fe234006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:27.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:27.028+0000 7fe24c800700 1 -- 172.21.15.69:0/2258567721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 msgr2=0x7fe24419dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:27.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:27.028+0000 7fe24c800700 1 --2- 172.21.15.69:0/2258567721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24419dd40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe2380117e0 tx=0x7fe238011c10 comp rx=0 tx=0).stop 2024-03-19T00:59:27.810 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:27.029+0000 7fe24c800700 1 -- 172.21.15.69:0/2258567721 shutdown_connections 2024-03-19T00:59:27.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:27.029+0000 7fe24c800700 1 --2- 172.21.15.69:0/2258567721 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe22c038f80 0x7fe22c03b440 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:27.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:27.029+0000 7fe24c800700 1 --2- 172.21.15.69:0/2258567721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2441087c0 0x7fe24419dd40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:27.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:27.029+0000 7fe24c800700 1 -- 172.21.15.69:0/2258567721 >> 172.21.15.69:0/2258567721 conn(0x7fe2441021d0 msgr2=0x7fe244102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:27.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:27.029+0000 7fe24c800700 1 -- 172.21.15.69:0/2258567721 shutdown_connections 2024-03-19T00:59:27.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:27.029+0000 7fe24c800700 1 -- 172.21.15.69:0/2258567721 wait complete. 2024-03-19T00:59:27.814 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial admin user... 2024-03-19T00:59:28.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:28 smithi069 ceph-mon[31727]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:59:28.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:28.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:29.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:29.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:29 smithi069 ceph-mon[31727]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:59:29.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:29.575 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$xFcJjyeDP6pDjyT6f84NCepoMbqKl09ls.5.ct3MYkFK6cZNSOiLS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710809968, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-19T00:59:29.575 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.412+0000 7f9776e00700 1 Processor -- start 2024-03-19T00:59:29.575 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.413+0000 7f9776e00700 1 -- start start 2024-03-19T00:59:29.575 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.413+0000 7f9776e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f9770108130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:29.575 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.413+0000 7f9776e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9770108700 con 0x7f9770107d10 2024-03-19T00:59:29.575 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.413+0000 7f976fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f9770108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:29.575 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.413+0000 7f976fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f9770108130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37386/0 (socket says 172.21.15.69:37386) 2024-03-19T00:59:29.575 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.413+0000 7f976fe00700 1 -- 172.21.15.69:0/3533221569 learned_addr learned my addr 172.21.15.69:0/3533221569 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:29.575 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.414+0000 7f976fe00700 1 -- 172.21.15.69:0/3533221569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9770108e70 con 0x7f9770107d10 2024-03-19T00:59:29.575 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.414+0000 7f976fe00700 1 --2- 172.21.15.69:0/3533221569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f9770108130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f97600098a0 tx=0x7f9760009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baddf2719752cf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:29.575 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.414+0000 7f976ea00700 1 -- 172.21.15.69:0/3533221569 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9760004160 con 0x7f9770107d10 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.415+0000 7f976ea00700 1 -- 172.21.15.69:0/3533221569 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9760004300 con 0x7f9770107d10 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.415+0000 7f976ea00700 1 -- 172.21.15.69:0/3533221569 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9760011b20 con 0x7f9770107d10 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.415+0000 7f9776e00700 1 -- 172.21.15.69:0/3533221569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 msgr2=0x7f9770108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.415+0000 7f9776e00700 1 --2- 172.21.15.69:0/3533221569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f9770108130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f97600098a0 tx=0x7f9760009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.415+0000 7f9776e00700 1 -- 172.21.15.69:0/3533221569 shutdown_connections 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.415+0000 7f9776e00700 1 --2- 172.21.15.69:0/3533221569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f9770108130 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.415+0000 7f9776e00700 1 -- 172.21.15.69:0/3533221569 >> 172.21.15.69:0/3533221569 conn(0x7f97701032d0 msgr2=0x7f97701056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.415+0000 7f9776e00700 1 -- 172.21.15.69:0/3533221569 shutdown_connections 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.415+0000 7f9776e00700 1 -- 172.21.15.69:0/3533221569 wait complete. 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.416+0000 7f9776e00700 1 Processor -- start 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.417+0000 7f9776e00700 1 -- start start 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.417+0000 7f9776e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f977019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:29.576 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.417+0000 7f9776e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f977010c220 con 0x7f9770107d10 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.418+0000 7f976fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f977019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.418+0000 7f976fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f977019bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37402/0 (socket says 172.21.15.69:37402) 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.418+0000 7f976fe00700 1 -- 172.21.15.69:0/1495949974 learned_addr learned my addr 172.21.15.69:0/1495949974 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.418+0000 7f976fe00700 1 -- 172.21.15.69:0/1495949974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977019c140 con 0x7f9770107d10 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.418+0000 7f976fe00700 1 --2- 172.21.15.69:0/1495949974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f977019bc00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f97600117e0 tx=0x7f9760011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.418+0000 7f976cc00700 1 -- 172.21.15.69:0/1495949974 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9760003bf0 con 0x7f9770107d10 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.419+0000 7f976cc00700 1 -- 172.21.15.69:0/1495949974 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976001a910 con 0x7f9770107d10 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.419+0000 7f9776e00700 1 -- 172.21.15.69:0/1495949974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977019c3d0 con 0x7f9770107d10 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.419+0000 7f976cc00700 1 -- 172.21.15.69:0/1495949974 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97600249a0 con 0x7f9770107d10 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.419+0000 7f9776e00700 1 -- 172.21.15.69:0/1495949974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977019c830 con 0x7f9770107d10 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.420+0000 7f9776e00700 1 -- 172.21.15.69:0/1495949974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9770063940 con 0x7f9770107d10 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.420+0000 7f976cc00700 1 -- 172.21.15.69:0/1495949974 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f9760022020 con 0x7f9770107d10 2024-03-19T00:59:29.577 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.421+0000 7f976cc00700 1 --2- 172.21.15.69:0/1495949974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9754038f20 0x7f975403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.421+0000 7f976cc00700 1 -- 172.21.15.69:0/1495949974 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f976004d3b0 con 0x7f9770107d10 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.421+0000 7f976f400700 1 --2- 172.21.15.69:0/1495949974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9754038f20 0x7f975403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.422+0000 7f976f400700 1 --2- 172.21.15.69:0/1495949974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9754038f20 0x7f975403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f975c006fd0 tx=0x7f975c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.426+0000 7f976cc00700 1 -- 172.21.15.69:0/1495949974 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9760011e20 con 0x7f9770107d10 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.666+0000 7f9776e00700 1 -- 172.21.15.69:0/1495949974 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f977019f260 con 0x7f9754038f20 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.906+0000 7f976cc00700 1 -- 172.21.15.69:0/1495949974 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f977019f260 con 0x7f9754038f20 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.909+0000 7f9776e00700 1 -- 172.21.15.69:0/1495949974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9754038f20 msgr2=0x7f975403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.909+0000 7f9776e00700 1 --2- 172.21.15.69:0/1495949974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9754038f20 0x7f975403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f975c006fd0 tx=0x7f975c006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.909+0000 7f9776e00700 1 -- 172.21.15.69:0/1495949974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 msgr2=0x7f977019bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.909+0000 7f9776e00700 1 --2- 172.21.15.69:0/1495949974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f977019bc00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f97600117e0 tx=0x7f9760011c10 comp rx=0 tx=0).stop 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.909+0000 7f9776e00700 1 -- 172.21.15.69:0/1495949974 shutdown_connections 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.909+0000 7f9776e00700 1 --2- 172.21.15.69:0/1495949974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9754038f20 0x7f975403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:29.578 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.909+0000 7f9776e00700 1 --2- 172.21.15.69:0/1495949974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9770107d10 0x7f977019bc00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:29.579 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.909+0000 7f9776e00700 1 -- 172.21.15.69:0/1495949974 >> 172.21.15.69:0/1495949974 conn(0x7f97701032d0 msgr2=0x7f9770103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:29.579 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.909+0000 7f9776e00700 1 -- 172.21.15.69:0/1495949974 shutdown_connections 2024-03-19T00:59:29.579 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:28.910+0000 7f9776e00700 1 -- 172.21.15.69:0/1495949974 wait complete. 2024-03-19T00:59:29.579 INFO:teuthology.orchestra.run.smithi069.stdout:Fetching dashboard port number... 2024-03-19T00:59:31.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:31 smithi069 ceph-mon[31727]: mgrmap e13: x(active, since 6s) 2024-03-19T00:59:31.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:31 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3931474013' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-19T00:59:31.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 8443 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.374+0000 7fac24200700 1 Processor -- start 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.374+0000 7fac24200700 1 -- start start 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.375+0000 7fac24200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c07cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.375+0000 7fac24200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c07d0f0 con 0x7fac1c07e6b0 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.375+0000 7fac21800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c07cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.375+0000 7fac21800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c07cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37418/0 (socket says 172.21.15.69:37418) 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.375+0000 7fac21800700 1 -- 172.21.15.69:0/1685771140 learned_addr learned my addr 172.21.15.69:0/1685771140 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.376+0000 7fac21800700 1 -- 172.21.15.69:0/1685771140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c07d270 con 0x7fac1c07e6b0 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.376+0000 7fac21800700 1 --2- 172.21.15.69:0/1685771140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c07cbb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fac100098a0 tx=0x7fac10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22969db6123e5f63 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.376+0000 7fac1be00700 1 -- 172.21.15.69:0/1685771140 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac10004160 con 0x7fac1c07e6b0 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.376+0000 7fac1be00700 1 -- 172.21.15.69:0/1685771140 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac10004300 con 0x7fac1c07e6b0 2024-03-19T00:59:31.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.376+0000 7fac1be00700 1 -- 172.21.15.69:0/1685771140 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac10011b20 con 0x7fac1c07e6b0 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.377+0000 7fac24200700 1 -- 172.21.15.69:0/1685771140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 msgr2=0x7fac1c07cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.377+0000 7fac24200700 1 --2- 172.21.15.69:0/1685771140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c07cbb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fac100098a0 tx=0x7fac10009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.377+0000 7fac24200700 1 -- 172.21.15.69:0/1685771140 shutdown_connections 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.377+0000 7fac24200700 1 --2- 172.21.15.69:0/1685771140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c07cbb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.377+0000 7fac24200700 1 -- 172.21.15.69:0/1685771140 >> 172.21.15.69:0/1685771140 conn(0x7fac1c1032e0 msgr2=0x7fac1c105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.377+0000 7fac24200700 1 -- 172.21.15.69:0/1685771140 shutdown_connections 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.377+0000 7fac24200700 1 -- 172.21.15.69:0/1685771140 wait complete. 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.378+0000 7fac24200700 1 Processor -- start 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.378+0000 7fac24200700 1 -- start start 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.379+0000 7fac24200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.379+0000 7fac24200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c10e2c0 con 0x7fac1c07e6b0 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.379+0000 7fac21800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:31.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.379+0000 7fac21800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37420/0 (socket says 172.21.15.69:37420) 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.379+0000 7fac21800700 1 -- 172.21.15.69:0/3931474013 learned_addr learned my addr 172.21.15.69:0/3931474013 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.379+0000 7fac21800700 1 -- 172.21.15.69:0/3931474013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c19e2e0 con 0x7fac1c07e6b0 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.379+0000 7fac21800700 1 --2- 172.21.15.69:0/3931474013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c19dda0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fac100117e0 tx=0x7fac10011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.380+0000 7fac1a000700 1 -- 172.21.15.69:0/3931474013 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac10003bf0 con 0x7fac1c07e6b0 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.380+0000 7fac1a000700 1 -- 172.21.15.69:0/3931474013 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac1001a910 con 0x7fac1c07e6b0 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.380+0000 7fac24200700 1 -- 172.21.15.69:0/3931474013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c19e570 con 0x7fac1c07e6b0 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.380+0000 7fac1a000700 1 -- 172.21.15.69:0/3931474013 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac10024970 con 0x7fac1c07e6b0 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.380+0000 7fac24200700 1 -- 172.21.15.69:0/3931474013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c19e9d0 con 0x7fac1c07e6b0 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.381+0000 7fac1a000700 1 -- 172.21.15.69:0/3931474013 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fac10022020 con 0x7fac1c07e6b0 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.381+0000 7fac24200700 1 -- 172.21.15.69:0/3931474013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1c063940 con 0x7fac1c07e6b0 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.382+0000 7fac1a000700 1 --2- 172.21.15.69:0/3931474013 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fac04039020 0x7fac0403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.382+0000 7fac1a000700 1 -- 172.21.15.69:0/3931474013 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fac1004d360 con 0x7fac1c07e6b0 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.382+0000 7fac20e00700 1 --2- 172.21.15.69:0/3931474013 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fac04039020 0x7fac0403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:31.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.383+0000 7fac20e00700 1 --2- 172.21.15.69:0/3931474013 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fac04039020 0x7fac0403b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fac08006fd0 tx=0x7fac08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.387+0000 7fac1a000700 1 -- 172.21.15.69:0/3931474013 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fac10011e20 con 0x7fac1c07e6b0 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.607+0000 7fac24200700 1 -- 172.21.15.69:0/3931474013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fac1c1a1460 con 0x7fac1c07e6b0 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.608+0000 7fac1a000700 1 -- 172.21.15.69:0/3931474013 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fac1000b3e0 con 0x7fac1c07e6b0 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.611+0000 7fac24200700 1 -- 172.21.15.69:0/3931474013 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fac04039020 msgr2=0x7fac0403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.611+0000 7fac24200700 1 --2- 172.21.15.69:0/3931474013 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fac04039020 0x7fac0403b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fac08006fd0 tx=0x7fac08006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.611+0000 7fac24200700 1 -- 172.21.15.69:0/3931474013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 msgr2=0x7fac1c19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.611+0000 7fac24200700 1 --2- 172.21.15.69:0/3931474013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c19dda0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fac100117e0 tx=0x7fac10011c10 comp rx=0 tx=0).stop 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.611+0000 7fac24200700 1 -- 172.21.15.69:0/3931474013 shutdown_connections 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.611+0000 7fac24200700 1 --2- 172.21.15.69:0/3931474013 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fac04039020 0x7fac0403b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.611+0000 7fac24200700 1 --2- 172.21.15.69:0/3931474013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac1c07e6b0 0x7fac1c19dda0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.611+0000 7fac24200700 1 -- 172.21.15.69:0/3931474013 >> 172.21.15.69:0/3931474013 conn(0x7fac1c1032e0 msgr2=0x7fac1c103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.612+0000 7fac24200700 1 -- 172.21.15.69:0/3931474013 shutdown_connections 2024-03-19T00:59:31.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:30.612+0000 7fac24200700 1 -- 172.21.15.69:0/3931474013 wait complete. 2024-03-19T00:59:31.820 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T00:59:31.820 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-03-19T00:59:31.841 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T00:59:31.842 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-03-19T00:59:31.842 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-03-19T00:59:31.842 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-19T00:59:31.849 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph Dashboard is now available at: 2024-03-19T00:59:31.850 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:59:31.850 INFO:teuthology.orchestra.run.smithi069.stdout: URL: https://smithi069.front.sepia.ceph.com:8443/ 2024-03-19T00:59:31.850 INFO:teuthology.orchestra.run.smithi069.stdout: User: admin 2024-03-19T00:59:31.850 INFO:teuthology.orchestra.run.smithi069.stdout: Password: 22khqe99zi 2024-03-19T00:59:31.850 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:59:31.851 INFO:teuthology.orchestra.run.smithi069.stdout:Saving cluster configuration to /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config directory 2024-03-19T00:59:31.853 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling autotune for osd_memory_target 2024-03-19T00:59:33.382 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.429+0000 7f6dc2c00700 1 Processor -- start 2024-03-19T00:59:33.382 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.429+0000 7f6dc2c00700 1 -- start start 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.429+0000 7f6dc2c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.430+0000 7f6dc2c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc07c860 con 0x7f6dbc07de20 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.430+0000 7f6dbbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.430+0000 7f6dbbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37436/0 (socket says 172.21.15.69:37436) 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.430+0000 7f6dbbe00700 1 -- 172.21.15.69:0/1817760007 learned_addr learned my addr 172.21.15.69:0/1817760007 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.431+0000 7f6dbbe00700 1 -- 172.21.15.69:0/1817760007 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc07c9e0 con 0x7f6dbc07de20 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.431+0000 7f6dbbe00700 1 --2- 172.21.15.69:0/1817760007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc07c320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0098a0 tx=0x7f6dac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457fa72de973e989 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.431+0000 7f6dbaa00700 1 -- 172.21.15.69:0/1817760007 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dac004160 con 0x7f6dbc07de20 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.432+0000 7f6dbaa00700 1 -- 172.21.15.69:0/1817760007 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dac004300 con 0x7f6dbc07de20 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.432+0000 7f6dbaa00700 1 -- 172.21.15.69:0/1817760007 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dac011b20 con 0x7f6dbc07de20 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.432+0000 7f6dc2c00700 1 -- 172.21.15.69:0/1817760007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 msgr2=0x7f6dbc07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.432+0000 7f6dc2c00700 1 --2- 172.21.15.69:0/1817760007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc07c320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0098a0 tx=0x7f6dac009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.432+0000 7f6dc2c00700 1 -- 172.21.15.69:0/1817760007 shutdown_connections 2024-03-19T00:59:33.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.432+0000 7f6dc2c00700 1 --2- 172.21.15.69:0/1817760007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc07c320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.432+0000 7f6dc2c00700 1 -- 172.21.15.69:0/1817760007 >> 172.21.15.69:0/1817760007 conn(0x7f6dbc1032d0 msgr2=0x7f6dbc1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.433+0000 7f6dc2c00700 1 -- 172.21.15.69:0/1817760007 shutdown_connections 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.433+0000 7f6dc2c00700 1 -- 172.21.15.69:0/1817760007 wait complete. 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.434+0000 7f6dc2c00700 1 Processor -- start 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.434+0000 7f6dc2c00700 1 -- start start 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.434+0000 7f6dc2c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.434+0000 7f6dc2c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc10e2b0 con 0x7f6dbc07de20 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.434+0000 7f6dbbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.434+0000 7f6dbbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc19dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37438/0 (socket says 172.21.15.69:37438) 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.434+0000 7f6dbbe00700 1 -- 172.21.15.69:0/3067467839 learned_addr learned my addr 172.21.15.69:0/3067467839 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.435+0000 7f6dbbe00700 1 -- 172.21.15.69:0/3067467839 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc19e2d0 con 0x7f6dbc07de20 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.435+0000 7f6dbbe00700 1 --2- 172.21.15.69:0/3067467839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc19dd90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0117e0 tx=0x7f6dac011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.435+0000 7f6db8c00700 1 -- 172.21.15.69:0/3067467839 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dac003ba0 con 0x7f6dbc07de20 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.435+0000 7f6db8c00700 1 -- 172.21.15.69:0/3067467839 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dac01a910 con 0x7f6dbc07de20 2024-03-19T00:59:33.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.435+0000 7f6db8c00700 1 -- 172.21.15.69:0/3067467839 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dac0248f0 con 0x7f6dbc07de20 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.435+0000 7f6dc2c00700 1 -- 172.21.15.69:0/3067467839 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dbc19e560 con 0x7f6dbc07de20 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.436+0000 7f6dc2c00700 1 -- 172.21.15.69:0/3067467839 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dbc1a1210 con 0x7f6dbc07de20 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.437+0000 7f6dc2c00700 1 -- 172.21.15.69:0/3067467839 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dbc063940 con 0x7f6dbc07de20 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.437+0000 7f6db8c00700 1 -- 172.21.15.69:0/3067467839 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f6dac022020 con 0x7f6dbc07de20 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.437+0000 7f6db8c00700 1 --2- 172.21.15.69:0/3067467839 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6da0039070 0x7f6da003b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.437+0000 7f6db8c00700 1 -- 172.21.15.69:0/3067467839 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6dac04d550 con 0x7f6dbc07de20 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.437+0000 7f6dbb400700 1 --2- 172.21.15.69:0/3067467839 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6da0039070 0x7f6da003b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.438+0000 7f6dbb400700 1 --2- 172.21.15.69:0/3067467839 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6da0039070 0x7f6da003b530 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6da8006fd0 tx=0x7f6da8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.443+0000 7f6db8c00700 1 -- 172.21.15.69:0/3067467839 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6dac01ab70 con 0x7f6dbc07de20 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.663+0000 7f6dc2c00700 1 -- 172.21.15.69:0/3067467839 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f6dbc1a13e0 con 0x7f6dbc07de20 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.664+0000 7f6db8c00700 1 -- 172.21.15.69:0/3067467839 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f6dac010380 con 0x7f6dbc07de20 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.667+0000 7f6dc2c00700 1 -- 172.21.15.69:0/3067467839 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6da0039070 msgr2=0x7f6da003b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.667+0000 7f6dc2c00700 1 --2- 172.21.15.69:0/3067467839 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6da0039070 0x7f6da003b530 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6da8006fd0 tx=0x7f6da8006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:33.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.667+0000 7f6dc2c00700 1 -- 172.21.15.69:0/3067467839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 msgr2=0x7f6dbc19dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:33.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.667+0000 7f6dc2c00700 1 --2- 172.21.15.69:0/3067467839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc19dd90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0117e0 tx=0x7f6dac011ba0 comp rx=0 tx=0).stop 2024-03-19T00:59:33.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.667+0000 7f6dc2c00700 1 -- 172.21.15.69:0/3067467839 shutdown_connections 2024-03-19T00:59:33.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.667+0000 7f6dc2c00700 1 --2- 172.21.15.69:0/3067467839 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6da0039070 0x7f6da003b530 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:33.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.667+0000 7f6dc2c00700 1 --2- 172.21.15.69:0/3067467839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dbc07de20 0x7f6dbc19dd90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:33.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.667+0000 7f6dc2c00700 1 -- 172.21.15.69:0/3067467839 >> 172.21.15.69:0/3067467839 conn(0x7f6dbc1032d0 msgr2=0x7f6dbc103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:33.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.667+0000 7f6dc2c00700 1 -- 172.21.15.69:0/3067467839 shutdown_connections 2024-03-19T00:59:33.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:32.667+0000 7f6dc2c00700 1 -- 172.21.15.69:0/3067467839 wait complete. 2024-03-19T00:59:35.069 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.966+0000 7f605fa00700 1 Processor -- start 2024-03-19T00:59:35.069 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.967+0000 7f605fa00700 1 -- start start 2024-03-19T00:59:35.069 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.967+0000 7f605fa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605807cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:35.069 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.967+0000 7f605fa00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f605807d1c0 con 0x7f605807e740 2024-03-19T00:59:35.069 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.967+0000 7f605d000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605807cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:35.069 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.968+0000 7f605d000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605807cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37440/0 (socket says 172.21.15.69:37440) 2024-03-19T00:59:35.069 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.968+0000 7f605d000700 1 -- 172.21.15.69:0/1539177447 learned_addr learned my addr 172.21.15.69:0/1539177447 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:35.069 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.968+0000 7f605d000700 1 -- 172.21.15.69:0/1539177447 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605807d340 con 0x7f605807e740 2024-03-19T00:59:35.069 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.968+0000 7f605d000700 1 --2- 172.21.15.69:0/1539177447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605807cbf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f604c0098a0 tx=0x7f604c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e671be9cd1792f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.969+0000 7f6057400700 1 -- 172.21.15.69:0/1539177447 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f604c004160 con 0x7f605807e740 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.969+0000 7f6057400700 1 -- 172.21.15.69:0/1539177447 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f604c004300 con 0x7f605807e740 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.969+0000 7f6057400700 1 -- 172.21.15.69:0/1539177447 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f604c011b20 con 0x7f605807e740 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.969+0000 7f605fa00700 1 -- 172.21.15.69:0/1539177447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 msgr2=0x7f605807cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.969+0000 7f605fa00700 1 --2- 172.21.15.69:0/1539177447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605807cbf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f604c0098a0 tx=0x7f604c009bb0 comp rx=0 tx=0).stop 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.969+0000 7f605fa00700 1 -- 172.21.15.69:0/1539177447 shutdown_connections 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.969+0000 7f605fa00700 1 --2- 172.21.15.69:0/1539177447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605807cbf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.969+0000 7f605fa00700 1 -- 172.21.15.69:0/1539177447 >> 172.21.15.69:0/1539177447 conn(0x7f60581032a0 msgr2=0x7f60581056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.969+0000 7f605fa00700 1 -- 172.21.15.69:0/1539177447 shutdown_connections 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.970+0000 7f605fa00700 1 -- 172.21.15.69:0/1539177447 wait complete. 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.970+0000 7f605fa00700 1 Processor -- start 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.971+0000 7f605fa00700 1 -- start start 2024-03-19T00:59:35.070 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.971+0000 7f605fa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.971+0000 7f605fa00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f605810e2c0 con 0x7f605807e740 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.971+0000 7f605d000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.971+0000 7f605d000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37452/0 (socket says 172.21.15.69:37452) 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.971+0000 7f605d000700 1 -- 172.21.15.69:0/2624507945 learned_addr learned my addr 172.21.15.69:0/2624507945 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.972+0000 7f605d000700 1 -- 172.21.15.69:0/2624507945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605819e320 con 0x7f605807e740 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.972+0000 7f605d000700 1 --2- 172.21.15.69:0/2624507945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605819dde0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f604c011b60 tx=0x7f604c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.972+0000 7f6055600700 1 -- 172.21.15.69:0/2624507945 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f604c003b60 con 0x7f605807e740 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.972+0000 7f6055600700 1 -- 172.21.15.69:0/2624507945 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f604c01a910 con 0x7f605807e740 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.972+0000 7f605fa00700 1 -- 172.21.15.69:0/2624507945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605819e5b0 con 0x7f605807e740 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.972+0000 7f6055600700 1 -- 172.21.15.69:0/2624507945 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f604c024930 con 0x7f605807e740 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.972+0000 7f605fa00700 1 -- 172.21.15.69:0/2624507945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605819ea10 con 0x7f605807e740 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.973+0000 7f6055600700 1 -- 172.21.15.69:0/2624507945 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f604c022020 con 0x7f605807e740 2024-03-19T00:59:35.071 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.973+0000 7f605fa00700 1 -- 172.21.15.69:0/2624507945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6058063940 con 0x7f605807e740 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.974+0000 7f6055600700 1 --2- 172.21.15.69:0/2624507945 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6040039010 0x7f604003b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.974+0000 7f6055600700 1 -- 172.21.15.69:0/2624507945 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f604c04d340 con 0x7f605807e740 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.974+0000 7f6057e00700 1 --2- 172.21.15.69:0/2624507945 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6040039010 0x7f604003b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.975+0000 7f6057e00700 1 --2- 172.21.15.69:0/2624507945 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6040039010 0x7f604003b4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6044006fd0 tx=0x7f6044006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:33.979+0000 7f6055600700 1 -- 172.21.15.69:0/2624507945 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f604c024c50 con 0x7f605807e740 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.316+0000 7f605fa00700 1 -- 172.21.15.69:0/2624507945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f60581a13e0 con 0x7f605807e740 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.320+0000 7f6055600700 1 -- 172.21.15.69:0/2624507945 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f604c02d3f0 con 0x7f605807e740 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.323+0000 7f605fa00700 1 -- 172.21.15.69:0/2624507945 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6040039010 msgr2=0x7f604003b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.323+0000 7f605fa00700 1 --2- 172.21.15.69:0/2624507945 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6040039010 0x7f604003b4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6044006fd0 tx=0x7f6044006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.323+0000 7f605fa00700 1 -- 172.21.15.69:0/2624507945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 msgr2=0x7f605819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.323+0000 7f605fa00700 1 --2- 172.21.15.69:0/2624507945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605819dde0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f604c011b60 tx=0x7f604c011c40 comp rx=0 tx=0).stop 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.323+0000 7f605fa00700 1 -- 172.21.15.69:0/2624507945 shutdown_connections 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.323+0000 7f605fa00700 1 --2- 172.21.15.69:0/2624507945 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6040039010 0x7f604003b4d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.323+0000 7f605fa00700 1 --2- 172.21.15.69:0/2624507945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f605807e740 0x7f605819dde0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.323+0000 7f605fa00700 1 -- 172.21.15.69:0/2624507945 >> 172.21.15.69:0/2624507945 conn(0x7f60581032a0 msgr2=0x7f6058103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.323+0000 7f605fa00700 1 -- 172.21.15.69:0/2624507945 shutdown_connections 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-03-19T00:59:34.323+0000 7f605fa00700 1 -- 172.21.15.69:0/2624507945 wait complete. 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:Or, if you are only running a single cluster on this host: 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-19T00:59:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:59:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: ceph telemetry on 2024-03-19T00:59:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:59:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:For more information see: 2024-03-19T00:59:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:59:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-19T00:59:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:59:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:Bootstrap complete. 2024-03-19T00:59:35.162 INFO:tasks.cephadm:Fetching config... 2024-03-19T00:59:35.162 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T00:59:35.163 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-19T00:59:35.216 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-19T00:59:35.216 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T00:59:35.216 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-19T00:59:35.296 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-19T00:59:35.297 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T00:59:35.297 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-19T00:59:35.430 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-19T00:59:35.431 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T00:59:35.431 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-19T00:59:35.527 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-19T00:59:35.527 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZElbmgFaeL/glKigdAXtw09oGaz87AEKLPO/i6+Leg+zgPwpYJPkQ1G7rtrDXm9mkM5fo+sp1uuQ/nH/RXzI82QlDiREeEQHo5mBYgl/mEdZp5XfHBx8+LMBZY//OwVuAF0rWgwS8ev9zzTh9d5RF78ppNCmiZmj/FwoWnzjhWyUUQ2aJdCiKvJdANGZDwPgreO8+Q3WpGCoJTjMMB8Js+g0O0pxHvs+OuIrDYsIUkEOmC4sbw4hVSibYDsS3JGjp58Mt82CkThESVzjqXgqmoWEhX9A7M9bO+vjjksCP5PGd64sT3HzYUavux1wFucUpU5X51oBVar9DLVFdN0nT33EBYUgOK1f5c3nCGqh8IcraUEGg+aF8fCfBt1E0eatvSsYfBHmUDFmVKGhhUegEeRVTbM8JXbvuCofLqbl2RrzuLfbzbX6vk5+hFAyGfyvYTFp9Xw8gVWClqMKDMwQHXyzVzA0+skPivmNBzUCuw4CZrn0Mqh5dcV5bV7w7gXE= ceph-934de1c2-e58b-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-19T00:59:35.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:35 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2624507945' entity='client.admin' 2024-03-19T00:59:35.642 INFO:teuthology.orchestra.run.smithi069.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZElbmgFaeL/glKigdAXtw09oGaz87AEKLPO/i6+Leg+zgPwpYJPkQ1G7rtrDXm9mkM5fo+sp1uuQ/nH/RXzI82QlDiREeEQHo5mBYgl/mEdZp5XfHBx8+LMBZY//OwVuAF0rWgwS8ev9zzTh9d5RF78ppNCmiZmj/FwoWnzjhWyUUQ2aJdCiKvJdANGZDwPgreO8+Q3WpGCoJTjMMB8Js+g0O0pxHvs+OuIrDYsIUkEOmC4sbw4hVSibYDsS3JGjp58Mt82CkThESVzjqXgqmoWEhX9A7M9bO+vjjksCP5PGd64sT3HzYUavux1wFucUpU5X51oBVar9DLVFdN0nT33EBYUgOK1f5c3nCGqh8IcraUEGg+aF8fCfBt1E0eatvSsYfBHmUDFmVKGhhUegEeRVTbM8JXbvuCofLqbl2RrzuLfbzbX6vk5+hFAyGfyvYTFp9Xw8gVWClqMKDMwQHXyzVzA0+skPivmNBzUCuw4CZrn0Mqh5dcV5bV7w7gXE= ceph-934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:59:35.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZElbmgFaeL/glKigdAXtw09oGaz87AEKLPO/i6+Leg+zgPwpYJPkQ1G7rtrDXm9mkM5fo+sp1uuQ/nH/RXzI82QlDiREeEQHo5mBYgl/mEdZp5XfHBx8+LMBZY//OwVuAF0rWgwS8ev9zzTh9d5RF78ppNCmiZmj/FwoWnzjhWyUUQ2aJdCiKvJdANGZDwPgreO8+Q3WpGCoJTjMMB8Js+g0O0pxHvs+OuIrDYsIUkEOmC4sbw4hVSibYDsS3JGjp58Mt82CkThESVzjqXgqmoWEhX9A7M9bO+vjjksCP5PGd64sT3HzYUavux1wFucUpU5X51oBVar9DLVFdN0nT33EBYUgOK1f5c3nCGqh8IcraUEGg+aF8fCfBt1E0eatvSsYfBHmUDFmVKGhhUegEeRVTbM8JXbvuCofLqbl2RrzuLfbzbX6vk5+hFAyGfyvYTFp9Xw8gVWClqMKDMwQHXyzVzA0+skPivmNBzUCuw4CZrn0Mqh5dcV5bV7w7gXE= ceph-934de1c2-e58b-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-19T00:59:35.732 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZElbmgFaeL/glKigdAXtw09oGaz87AEKLPO/i6+Leg+zgPwpYJPkQ1G7rtrDXm9mkM5fo+sp1uuQ/nH/RXzI82QlDiREeEQHo5mBYgl/mEdZp5XfHBx8+LMBZY//OwVuAF0rWgwS8ev9zzTh9d5RF78ppNCmiZmj/FwoWnzjhWyUUQ2aJdCiKvJdANGZDwPgreO8+Q3WpGCoJTjMMB8Js+g0O0pxHvs+OuIrDYsIUkEOmC4sbw4hVSibYDsS3JGjp58Mt82CkThESVzjqXgqmoWEhX9A7M9bO+vjjksCP5PGd64sT3HzYUavux1wFucUpU5X51oBVar9DLVFdN0nT33EBYUgOK1f5c3nCGqh8IcraUEGg+aF8fCfBt1E0eatvSsYfBHmUDFmVKGhhUegEeRVTbM8JXbvuCofLqbl2RrzuLfbzbX6vk5+hFAyGfyvYTFp9Xw8gVWClqMKDMwQHXyzVzA0+skPivmNBzUCuw4CZrn0Mqh5dcV5bV7w7gXE= ceph-934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:59:35.747 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZElbmgFaeL/glKigdAXtw09oGaz87AEKLPO/i6+Leg+zgPwpYJPkQ1G7rtrDXm9mkM5fo+sp1uuQ/nH/RXzI82QlDiREeEQHo5mBYgl/mEdZp5XfHBx8+LMBZY//OwVuAF0rWgwS8ev9zzTh9d5RF78ppNCmiZmj/FwoWnzjhWyUUQ2aJdCiKvJdANGZDwPgreO8+Q3WpGCoJTjMMB8Js+g0O0pxHvs+OuIrDYsIUkEOmC4sbw4hVSibYDsS3JGjp58Mt82CkThESVzjqXgqmoWEhX9A7M9bO+vjjksCP5PGd64sT3HzYUavux1wFucUpU5X51oBVar9DLVFdN0nT33EBYUgOK1f5c3nCGqh8IcraUEGg+aF8fCfBt1E0eatvSsYfBHmUDFmVKGhhUegEeRVTbM8JXbvuCofLqbl2RrzuLfbzbX6vk5+hFAyGfyvYTFp9Xw8gVWClqMKDMwQHXyzVzA0+skPivmNBzUCuw4CZrn0Mqh5dcV5bV7w7gXE= ceph-934de1c2-e58b-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-19T00:59:35.793 INFO:teuthology.orchestra.run.smithi163.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZElbmgFaeL/glKigdAXtw09oGaz87AEKLPO/i6+Leg+zgPwpYJPkQ1G7rtrDXm9mkM5fo+sp1uuQ/nH/RXzI82QlDiREeEQHo5mBYgl/mEdZp5XfHBx8+LMBZY//OwVuAF0rWgwS8ev9zzTh9d5RF78ppNCmiZmj/FwoWnzjhWyUUQ2aJdCiKvJdANGZDwPgreO8+Q3WpGCoJTjMMB8Js+g0O0pxHvs+OuIrDYsIUkEOmC4sbw4hVSibYDsS3JGjp58Mt82CkThESVzjqXgqmoWEhX9A7M9bO+vjjksCP5PGd64sT3HzYUavux1wFucUpU5X51oBVar9DLVFdN0nT33EBYUgOK1f5c3nCGqh8IcraUEGg+aF8fCfBt1E0eatvSsYfBHmUDFmVKGhhUegEeRVTbM8JXbvuCofLqbl2RrzuLfbzbX6vk5+hFAyGfyvYTFp9Xw8gVWClqMKDMwQHXyzVzA0+skPivmNBzUCuw4CZrn0Mqh5dcV5bV7w7gXE= ceph-934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T00:59:35.808 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-19T00:59:36.236 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:59:38.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.802+0000 7f98c2200700 1 -- 172.21.15.69:0/1117267919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98bc103f80 msgr2=0x7f98bc1043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:38.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.802+0000 7f98c2200700 1 --2- 172.21.15.69:0/1117267919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98bc103f80 0x7f98bc1043a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009910 tx=0x7f98ac009c20 comp rx=0 tx=0).stop 2024-03-19T00:59:38.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.802+0000 7f98c2200700 1 -- 172.21.15.69:0/1117267919 shutdown_connections 2024-03-19T00:59:38.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.802+0000 7f98c2200700 1 --2- 172.21.15.69:0/1117267919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98bc103f80 0x7f98bc1043a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:38.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.802+0000 7f98c2200700 1 -- 172.21.15.69:0/1117267919 >> 172.21.15.69:0/1117267919 conn(0x7f98bc0ff4e0 msgr2=0x7f98bc101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:38.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.802+0000 7f98c2200700 1 -- 172.21.15.69:0/1117267919 shutdown_connections 2024-03-19T00:59:38.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.802+0000 7f98c2200700 1 -- 172.21.15.69:0/1117267919 wait complete. 2024-03-19T00:59:38.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.803+0000 7f98c2200700 1 Processor -- start 2024-03-19T00:59:38.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.804+0000 7f98c2200700 1 -- start start 2024-03-19T00:59:38.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.804+0000 7f98c2200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98bc103f80 0x7f98bc199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:38.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.804+0000 7f98c2200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc10a010 con 0x7f98bc103f80 2024-03-19T00:59:38.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.804+0000 7f98bb400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98bc103f80 0x7f98bc199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:38.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.804+0000 7f98bb400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98bc103f80 0x7f98bc199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40614/0 (socket says 172.21.15.69:40614) 2024-03-19T00:59:38.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.804+0000 7f98bb400700 1 -- 172.21.15.69:0/3012830743 learned_addr learned my addr 172.21.15.69:0/3012830743 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:38.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.805+0000 7f98bb400700 1 -- 172.21.15.69:0/3012830743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc199bc0 con 0x7f98bc103f80 2024-03-19T00:59:38.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.805+0000 7f98bb400700 1 --2- 172.21.15.69:0/3012830743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98bc103f80 0x7f98bc199680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98ac005850 tx=0x7f98ac005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:38.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.805+0000 7f98b3e00700 1 -- 172.21.15.69:0/3012830743 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98ac026070 con 0x7f98bc103f80 2024-03-19T00:59:38.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.805+0000 7f98b3e00700 1 -- 172.21.15.69:0/3012830743 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ac0044a0 con 0x7f98bc103f80 2024-03-19T00:59:38.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.805+0000 7f98c2200700 1 -- 172.21.15.69:0/3012830743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc199e50 con 0x7f98bc103f80 2024-03-19T00:59:38.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.805+0000 7f98b3e00700 1 -- 172.21.15.69:0/3012830743 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98ac003ab0 con 0x7f98bc103f80 2024-03-19T00:59:38.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.805+0000 7f98c2200700 1 -- 172.21.15.69:0/3012830743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc19cb40 con 0x7f98bc103f80 2024-03-19T00:59:38.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.806+0000 7f98b3e00700 1 -- 172.21.15.69:0/3012830743 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f98ac029070 con 0x7f98bc103f80 2024-03-19T00:59:38.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.806+0000 7f98c2200700 1 -- 172.21.15.69:0/3012830743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc10a790 con 0x7f98bc103f80 2024-03-19T00:59:38.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.807+0000 7f98b3e00700 1 --2- 172.21.15.69:0/3012830743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f98a0038fc0 0x7f98a003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:38.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.807+0000 7f98b3e00700 1 -- 172.21.15.69:0/3012830743 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f98ac01e070 con 0x7f98bc103f80 2024-03-19T00:59:38.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.807+0000 7f98baa00700 1 --2- 172.21.15.69:0/3012830743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f98a0038fc0 0x7f98a003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:38.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.808+0000 7f98baa00700 1 --2- 172.21.15.69:0/3012830743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f98a0038fc0 0x7f98a003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f98a8006fd0 tx=0x7f98a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:38.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:38.813+0000 7f98b3e00700 1 -- 172.21.15.69:0/3012830743 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f98ac013070 con 0x7f98bc103f80 2024-03-19T00:59:39.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.030+0000 7f98c2200700 1 -- 172.21.15.69:0/3012830743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f98bc02dcc0 con 0x7f98bc103f80 2024-03-19T00:59:39.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.035+0000 7f98b3e00700 1 -- 172.21.15.69:0/3012830743 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f98ac030020 con 0x7f98bc103f80 2024-03-19T00:59:39.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.039+0000 7f98c2200700 1 -- 172.21.15.69:0/3012830743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f98a0038fc0 msgr2=0x7f98a003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:39.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.039+0000 7f98c2200700 1 --2- 172.21.15.69:0/3012830743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f98a0038fc0 0x7f98a003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f98a8006fd0 tx=0x7f98a8006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:39.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.039+0000 7f98c2200700 1 -- 172.21.15.69:0/3012830743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98bc103f80 msgr2=0x7f98bc199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:39.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.039+0000 7f98c2200700 1 --2- 172.21.15.69:0/3012830743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98bc103f80 0x7f98bc199680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98ac005850 tx=0x7f98ac005030 comp rx=0 tx=0).stop 2024-03-19T00:59:39.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.039+0000 7f98c2200700 1 -- 172.21.15.69:0/3012830743 shutdown_connections 2024-03-19T00:59:39.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.039+0000 7f98c2200700 1 --2- 172.21.15.69:0/3012830743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f98a0038fc0 0x7f98a003b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:39.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.039+0000 7f98c2200700 1 --2- 172.21.15.69:0/3012830743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98bc103f80 0x7f98bc199680 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:39.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.039+0000 7f98c2200700 1 -- 172.21.15.69:0/3012830743 >> 172.21.15.69:0/3012830743 conn(0x7f98bc0ff4e0 msgr2=0x7f98bc0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:39.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.039+0000 7f98c2200700 1 -- 172.21.15.69:0/3012830743 shutdown_connections 2024-03-19T00:59:39.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:39.039+0000 7f98c2200700 1 -- 172.21.15.69:0/3012830743 wait complete. 2024-03-19T00:59:39.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:39.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:39.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:59:39.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:39.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:59:39.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:39.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:39 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3012830743' entity='client.admin' 2024-03-19T00:59:39.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:59:39.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:39.744 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-19T00:59:39.745 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-19T00:59:40.187 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:59:42.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.067+0000 7f5064a00700 1 -- 172.21.15.69:0/1466172685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5060102890 msgr2=0x7f5060104c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:42.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.067+0000 7f5064a00700 1 --2- 172.21.15.69:0/1466172685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5060102890 0x7f5060104c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5050009910 tx=0x7f5050009c20 comp rx=0 tx=0).stop 2024-03-19T00:59:42.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.067+0000 7f5064a00700 1 -- 172.21.15.69:0/1466172685 shutdown_connections 2024-03-19T00:59:42.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.067+0000 7f5064a00700 1 --2- 172.21.15.69:0/1466172685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5060102890 0x7f5060104c80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:42.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.067+0000 7f5064a00700 1 -- 172.21.15.69:0/1466172685 >> 172.21.15.69:0/1466172685 conn(0x7f50600fc1e0 msgr2=0x7f50600fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:42.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.067+0000 7f5064a00700 1 -- 172.21.15.69:0/1466172685 shutdown_connections 2024-03-19T00:59:42.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.067+0000 7f5064a00700 1 -- 172.21.15.69:0/1466172685 wait complete. 2024-03-19T00:59:42.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.068+0000 7f5064a00700 1 Processor -- start 2024-03-19T00:59:42.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.068+0000 7f5064a00700 1 -- start start 2024-03-19T00:59:42.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.069+0000 7f5064a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5060102890 0x7f5060199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:42.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.069+0000 7f5064a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5060107dd0 con 0x7f5060102890 2024-03-19T00:59:42.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.069+0000 7f505de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5060102890 0x7f5060199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:42.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.069+0000 7f505de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5060102890 0x7f5060199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40632/0 (socket says 172.21.15.69:40632) 2024-03-19T00:59:42.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.069+0000 7f505de00700 1 -- 172.21.15.69:0/3145005802 learned_addr learned my addr 172.21.15.69:0/3145005802 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:42.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.069+0000 7f505de00700 1 -- 172.21.15.69:0/3145005802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5060199b80 con 0x7f5060102890 2024-03-19T00:59:42.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.070+0000 7f505de00700 1 --2- 172.21.15.69:0/3145005802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5060102890 0x7f5060199640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5050005950 tx=0x7f5050005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:42.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.070+0000 7f5056a00700 1 -- 172.21.15.69:0/3145005802 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5050026070 con 0x7f5060102890 2024-03-19T00:59:42.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.070+0000 7f5056a00700 1 -- 172.21.15.69:0/3145005802 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f505000bc30 con 0x7f5060102890 2024-03-19T00:59:42.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.070+0000 7f5064a00700 1 -- 172.21.15.69:0/3145005802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5060199e10 con 0x7f5060102890 2024-03-19T00:59:42.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.070+0000 7f5056a00700 1 -- 172.21.15.69:0/3145005802 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5050003b30 con 0x7f5060102890 2024-03-19T00:59:42.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.070+0000 7f5064a00700 1 -- 172.21.15.69:0/3145005802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f506019cb00 con 0x7f5060102890 2024-03-19T00:59:42.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.071+0000 7f5064a00700 1 -- 172.21.15.69:0/3145005802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5060108550 con 0x7f5060102890 2024-03-19T00:59:42.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.071+0000 7f5056a00700 1 -- 172.21.15.69:0/3145005802 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f5050029070 con 0x7f5060102890 2024-03-19T00:59:42.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.072+0000 7f5056a00700 1 --2- 172.21.15.69:0/3145005802 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5044039020 0x7f504403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:42.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.072+0000 7f5056a00700 1 -- 172.21.15.69:0/3145005802 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f505001e070 con 0x7f5060102890 2024-03-19T00:59:42.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.072+0000 7f505d400700 1 --2- 172.21.15.69:0/3145005802 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5044039020 0x7f504403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:42.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.073+0000 7f505d400700 1 --2- 172.21.15.69:0/3145005802 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5044039020 0x7f504403b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f504c006fd0 tx=0x7f504c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.077+0000 7f5056a00700 1 -- 172.21.15.69:0/3145005802 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5050013070 con 0x7f5060102890 2024-03-19T00:59:42.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.307+0000 7f5064a00700 1 -- 172.21.15.69:0/3145005802 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f506002dcc0 con 0x7f5044039020 2024-03-19T00:59:42.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.315+0000 7f5056a00700 1 -- 172.21.15.69:0/3145005802 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f506002dcc0 con 0x7f5044039020 2024-03-19T00:59:42.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.318+0000 7f5064a00700 1 -- 172.21.15.69:0/3145005802 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5044039020 msgr2=0x7f504403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:42.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.318+0000 7f5064a00700 1 --2- 172.21.15.69:0/3145005802 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5044039020 0x7f504403b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f504c006fd0 tx=0x7f504c006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:42.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.318+0000 7f5064a00700 1 -- 172.21.15.69:0/3145005802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5060102890 msgr2=0x7f5060199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:42.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.318+0000 7f5064a00700 1 --2- 172.21.15.69:0/3145005802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5060102890 0x7f5060199640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5050005950 tx=0x7f5050005030 comp rx=0 tx=0).stop 2024-03-19T00:59:42.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.319+0000 7f5064a00700 1 -- 172.21.15.69:0/3145005802 shutdown_connections 2024-03-19T00:59:42.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.319+0000 7f5064a00700 1 --2- 172.21.15.69:0/3145005802 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5044039020 0x7f504403b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:42.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.319+0000 7f5064a00700 1 --2- 172.21.15.69:0/3145005802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5060102890 0x7f5060199640 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:42.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.319+0000 7f5064a00700 1 -- 172.21.15.69:0/3145005802 >> 172.21.15.69:0/3145005802 conn(0x7f50600fc1e0 msgr2=0x7f50600fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:42.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.319+0000 7f5064a00700 1 -- 172.21.15.69:0/3145005802 shutdown_connections 2024-03-19T00:59:42.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:42.319+0000 7f5064a00700 1 -- 172.21.15.69:0/3145005802 wait complete. 2024-03-19T00:59:43.081 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi086 2024-03-19T00:59:43.081 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-19T00:59:43.081 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.conf 2024-03-19T00:59:43.100 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-19T00:59:43.101 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:59:43.158 INFO:tasks.cephadm:Adding host smithi086 to orchestrator... 2024-03-19T00:59:43.158 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch host add smithi086 2024-03-19T00:59:43.561 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:59:43.706 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:43 smithi069 ceph-mon[31727]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:59:43.706 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:43 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:43.706 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:43 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:59:43.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:43 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:59:43.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:43 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:59:43.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:43 smithi069 ceph-mon[31727]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-19T00:59:44.733 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:44 smithi069 ceph-mon[31727]: Updating smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:59:44.734 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:44 smithi069 ceph-mon[31727]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:59:44.734 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:44 smithi069 ceph-mon[31727]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:59:45.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:45 smithi069 ceph-mon[31727]: Updating smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T00:59:45.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:45 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:45.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:45 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:45.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:45 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:45.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.622+0000 7f0224000700 1 -- 172.21.15.69:0/3071679197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f021c101f40 msgr2=0x7f021c102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:45.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.622+0000 7f0224000700 1 --2- 172.21.15.69:0/3071679197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f021c101f40 0x7f021c102320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f021000b0a0 tx=0x7f021000b3b0 comp rx=0 tx=0).stop 2024-03-19T00:59:45.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.622+0000 7f0224000700 1 -- 172.21.15.69:0/3071679197 shutdown_connections 2024-03-19T00:59:45.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.622+0000 7f0224000700 1 --2- 172.21.15.69:0/3071679197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f021c101f40 0x7f021c102320 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:45.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.622+0000 7f0224000700 1 -- 172.21.15.69:0/3071679197 >> 172.21.15.69:0/3071679197 conn(0x7f021c0fd7f0 msgr2=0x7f021c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:45.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.622+0000 7f0224000700 1 -- 172.21.15.69:0/3071679197 shutdown_connections 2024-03-19T00:59:45.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.622+0000 7f0224000700 1 -- 172.21.15.69:0/3071679197 wait complete. 2024-03-19T00:59:45.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.623+0000 7f0224000700 1 Processor -- start 2024-03-19T00:59:45.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.623+0000 7f0224000700 1 -- start start 2024-03-19T00:59:45.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.624+0000 7f0224000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f021c101f40 0x7f021c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:45.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.624+0000 7f0224000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f021c10c310 con 0x7f021c101f40 2024-03-19T00:59:45.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.624+0000 7f0221600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f021c101f40 0x7f021c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:45.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.624+0000 7f0221600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f021c101f40 0x7f021c19d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40658/0 (socket says 172.21.15.69:40658) 2024-03-19T00:59:45.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.624+0000 7f0221600700 1 -- 172.21.15.69:0/3014677497 learned_addr learned my addr 172.21.15.69:0/3014677497 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:45.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.624+0000 7f0221600700 1 -- 172.21.15.69:0/3014677497 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c19d920 con 0x7f021c101f40 2024-03-19T00:59:45.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.624+0000 7f0221600700 1 --2- 172.21.15.69:0/3014677497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f021c101f40 0x7f021c19d3e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f02100093e0 tx=0x7f0210003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:45.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.625+0000 7f021a000700 1 -- 172.21.15.69:0/3014677497 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0210017040 con 0x7f021c101f40 2024-03-19T00:59:45.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.625+0000 7f021a000700 1 -- 172.21.15.69:0/3014677497 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0210022070 con 0x7f021c101f40 2024-03-19T00:59:45.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.625+0000 7f021a000700 1 -- 172.21.15.69:0/3014677497 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f021001d440 con 0x7f021c101f40 2024-03-19T00:59:45.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.625+0000 7f0224000700 1 -- 172.21.15.69:0/3014677497 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c19dbb0 con 0x7f021c101f40 2024-03-19T00:59:45.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.625+0000 7f0224000700 1 -- 172.21.15.69:0/3014677497 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c1975a0 con 0x7f021c101f40 2024-03-19T00:59:45.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.626+0000 7f021a000700 1 -- 172.21.15.69:0/3014677497 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f0210004000 con 0x7f021c101f40 2024-03-19T00:59:45.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.626+0000 7f0224000700 1 -- 172.21.15.69:0/3014677497 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021c103420 con 0x7f021c101f40 2024-03-19T00:59:45.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.627+0000 7f021a000700 1 --2- 172.21.15.69:0/3014677497 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0204039070 0x7f020403b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:45.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.627+0000 7f021a000700 1 -- 172.21.15.69:0/3014677497 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f021000f050 con 0x7f021c101f40 2024-03-19T00:59:45.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.627+0000 7f0220c00700 1 --2- 172.21.15.69:0/3014677497 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0204039070 0x7f020403b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:45.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.628+0000 7f0220c00700 1 --2- 172.21.15.69:0/3014677497 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0204039070 0x7f020403b530 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0208006fd0 tx=0x7f0208006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:45.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.632+0000 7f021a000700 1 -- 172.21.15.69:0/3014677497 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0210007e20 con 0x7f021c101f40 2024-03-19T00:59:45.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:45.856+0000 7f0224000700 1 -- 172.21.15.69:0/3014677497 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}) -- 0x7f021c063940 con 0x7f0204039070 2024-03-19T00:59:46.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:46 smithi069 ceph-mon[31727]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:59:47.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:47 smithi069 ceph-mon[31727]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:59:47.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:47 smithi069 ceph-mon[31727]: Deploying cephadm binary to smithi086 2024-03-19T00:59:48.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:48 smithi069 ceph-mon[31727]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:59:50.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:50.542+0000 7f021a000700 1 -- 172.21.15.69:0/3014677497 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f021c063940 con 0x7f0204039070 2024-03-19T00:59:50.544 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi086' with addr '172.21.15.86' 2024-03-19T00:59:50.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:50.545+0000 7f0224000700 1 -- 172.21.15.69:0/3014677497 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0204039070 msgr2=0x7f020403b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:50.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:50.546+0000 7f0224000700 1 --2- 172.21.15.69:0/3014677497 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0204039070 0x7f020403b530 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0208006fd0 tx=0x7f0208006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:50.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:50.546+0000 7f0224000700 1 -- 172.21.15.69:0/3014677497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f021c101f40 msgr2=0x7f021c19d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:50.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:50.546+0000 7f0224000700 1 --2- 172.21.15.69:0/3014677497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f021c101f40 0x7f021c19d3e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f02100093e0 tx=0x7f0210003ce0 comp rx=0 tx=0).stop 2024-03-19T00:59:50.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:50.546+0000 7f0224000700 1 -- 172.21.15.69:0/3014677497 shutdown_connections 2024-03-19T00:59:50.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:50.546+0000 7f0224000700 1 --2- 172.21.15.69:0/3014677497 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0204039070 0x7f020403b530 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:50.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:50.546+0000 7f0224000700 1 --2- 172.21.15.69:0/3014677497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f021c101f40 0x7f021c19d3e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:50.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:50.546+0000 7f0224000700 1 -- 172.21.15.69:0/3014677497 >> 172.21.15.69:0/3014677497 conn(0x7f021c0fd7f0 msgr2=0x7f021c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:50.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:50.546+0000 7f0224000700 1 -- 172.21.15.69:0/3014677497 shutdown_connections 2024-03-19T00:59:50.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:50.546+0000 7f0224000700 1 -- 172.21.15.69:0/3014677497 wait complete. 2024-03-19T00:59:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:50 smithi069 ceph-mon[31727]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:59:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T00:59:51.440 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-19T00:59:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:51 smithi069 ceph-mon[31727]: Added host smithi086 2024-03-19T00:59:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:51 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:51 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:51.876 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:59:52.726 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:52 smithi069 ceph-mon[31727]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:59:53.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.817+0000 7f7e43800700 1 -- 172.21.15.69:0/3863376659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c101f40 msgr2=0x7f7e3c102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:53.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.817+0000 7f7e43800700 1 --2- 172.21.15.69:0/3863376659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c101f40 0x7f7e3c102320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009910 tx=0x7f7e30009c20 comp rx=0 tx=0).stop 2024-03-19T00:59:53.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.817+0000 7f7e43800700 1 -- 172.21.15.69:0/3863376659 shutdown_connections 2024-03-19T00:59:53.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.817+0000 7f7e43800700 1 --2- 172.21.15.69:0/3863376659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c101f40 0x7f7e3c102320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:53.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.817+0000 7f7e43800700 1 -- 172.21.15.69:0/3863376659 >> 172.21.15.69:0/3863376659 conn(0x7f7e3c0fd7f0 msgr2=0x7f7e3c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:53.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.817+0000 7f7e43800700 1 -- 172.21.15.69:0/3863376659 shutdown_connections 2024-03-19T00:59:53.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.817+0000 7f7e43800700 1 -- 172.21.15.69:0/3863376659 wait complete. 2024-03-19T00:59:53.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.818+0000 7f7e43800700 1 Processor -- start 2024-03-19T00:59:53.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.818+0000 7f7e43800700 1 -- start start 2024-03-19T00:59:53.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.818+0000 7f7e43800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c101f40 0x7f7e3c19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:53.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.818+0000 7f7e43800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c10c310 con 0x7f7e3c101f40 2024-03-19T00:59:53.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.819+0000 7f7e40e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c101f40 0x7f7e3c19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:53.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.819+0000 7f7e40e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c101f40 0x7f7e3c19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46214/0 (socket says 172.21.15.69:46214) 2024-03-19T00:59:53.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.819+0000 7f7e40e00700 1 -- 172.21.15.69:0/3973674075 learned_addr learned my addr 172.21.15.69:0/3973674075 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:53.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.819+0000 7f7e40e00700 1 -- 172.21.15.69:0/3973674075 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c19db10 con 0x7f7e3c101f40 2024-03-19T00:59:53.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.819+0000 7f7e40e00700 1 --2- 172.21.15.69:0/3973674075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c101f40 0x7f7e3c19d5d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e30006010 tx=0x7f7e30005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:53.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.820+0000 7f7e39600700 1 -- 172.21.15.69:0/3973674075 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e30026070 con 0x7f7e3c101f40 2024-03-19T00:59:53.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.820+0000 7f7e39600700 1 -- 172.21.15.69:0/3973674075 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3000bbd0 con 0x7f7e3c101f40 2024-03-19T00:59:53.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.820+0000 7f7e43800700 1 -- 172.21.15.69:0/3973674075 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3c19de00 con 0x7f7e3c101f40 2024-03-19T00:59:53.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.820+0000 7f7e39600700 1 -- 172.21.15.69:0/3973674075 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e30003910 con 0x7f7e3c101f40 2024-03-19T00:59:53.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.820+0000 7f7e43800700 1 -- 172.21.15.69:0/3973674075 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3c197870 con 0x7f7e3c101f40 2024-03-19T00:59:53.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.821+0000 7f7e43800700 1 -- 172.21.15.69:0/3973674075 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3c103420 con 0x7f7e3c101f40 2024-03-19T00:59:53.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.821+0000 7f7e39600700 1 -- 172.21.15.69:0/3973674075 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f7e30029070 con 0x7f7e3c101f40 2024-03-19T00:59:53.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.822+0000 7f7e39600700 1 --2- 172.21.15.69:0/3973674075 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7e24039020 0x7f7e2403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:53.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.822+0000 7f7e39600700 1 -- 172.21.15.69:0/3973674075 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7e3001e070 con 0x7f7e3c101f40 2024-03-19T00:59:53.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.822+0000 7f7e3be00700 1 --2- 172.21.15.69:0/3973674075 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7e24039020 0x7f7e2403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:53.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.823+0000 7f7e3be00700 1 --2- 172.21.15.69:0/3973674075 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7e24039020 0x7f7e2403b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7e28006fd0 tx=0x7f7e28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:53.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:53.828+0000 7f7e39600700 1 -- 172.21.15.69:0/3973674075 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7e30013070 con 0x7f7e3c101f40 2024-03-19T00:59:54.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.056+0000 7f7e43800700 1 -- 172.21.15.69:0/3973674075 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e3c198c00 con 0x7f7e24039020 2024-03-19T00:59:54.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.058+0000 7f7e39600700 1 -- 172.21.15.69:0/3973674075 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f7e3c198c00 con 0x7f7e24039020 2024-03-19T00:59:54.059 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T00:59:54.059 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}] 2024-03-19T00:59:54.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.061+0000 7f7e43800700 1 -- 172.21.15.69:0/3973674075 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7e24039020 msgr2=0x7f7e2403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:54.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.061+0000 7f7e43800700 1 --2- 172.21.15.69:0/3973674075 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7e24039020 0x7f7e2403b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7e28006fd0 tx=0x7f7e28006e40 comp rx=0 tx=0).stop 2024-03-19T00:59:54.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.061+0000 7f7e43800700 1 -- 172.21.15.69:0/3973674075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c101f40 msgr2=0x7f7e3c19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:54.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.061+0000 7f7e43800700 1 --2- 172.21.15.69:0/3973674075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c101f40 0x7f7e3c19d5d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e30006010 tx=0x7f7e30005030 comp rx=0 tx=0).stop 2024-03-19T00:59:54.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.061+0000 7f7e43800700 1 -- 172.21.15.69:0/3973674075 shutdown_connections 2024-03-19T00:59:54.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.061+0000 7f7e43800700 1 --2- 172.21.15.69:0/3973674075 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7e24039020 0x7f7e2403b4e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:54.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.061+0000 7f7e43800700 1 --2- 172.21.15.69:0/3973674075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e3c101f40 0x7f7e3c19d5d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:54.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.061+0000 7f7e43800700 1 -- 172.21.15.69:0/3973674075 >> 172.21.15.69:0/3973674075 conn(0x7f7e3c0fd7f0 msgr2=0x7f7e3c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:54.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.061+0000 7f7e43800700 1 -- 172.21.15.69:0/3973674075 shutdown_connections 2024-03-19T00:59:54.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:54.061+0000 7f7e43800700 1 -- 172.21.15.69:0/3973674075 wait complete. 2024-03-19T00:59:54.413 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:54.414 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:54 smithi069 ceph-mon[31727]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:59:54.756 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi163 2024-03-19T00:59:54.756 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-19T00:59:54.756 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.conf 2024-03-19T00:59:54.775 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-19T00:59:54.775 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:59:54.833 INFO:tasks.cephadm:Adding host smithi163 to orchestrator... 2024-03-19T00:59:54.833 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch host add smithi163 2024-03-19T00:59:55.276 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T00:59:55.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:55 smithi069 ceph-mon[31727]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T00:59:56.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:56 smithi069 ceph-mon[31727]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:59:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.348+0000 7f3fbe400700 1 -- 172.21.15.69:0/2665217877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fb8101f40 msgr2=0x7f3fb8102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T00:59:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.348+0000 7f3fbe400700 1 --2- 172.21.15.69:0/2665217877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fb8101f40 0x7f3fb8102320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009910 tx=0x7f3fa8009c20 comp rx=0 tx=0).stop 2024-03-19T00:59:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.348+0000 7f3fbe400700 1 -- 172.21.15.69:0/2665217877 shutdown_connections 2024-03-19T00:59:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.348+0000 7f3fbe400700 1 --2- 172.21.15.69:0/2665217877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fb8101f40 0x7f3fb8102320 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T00:59:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.349+0000 7f3fbe400700 1 -- 172.21.15.69:0/2665217877 >> 172.21.15.69:0/2665217877 conn(0x7f3fb80fd7f0 msgr2=0x7f3fb80ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T00:59:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.349+0000 7f3fbe400700 1 -- 172.21.15.69:0/2665217877 shutdown_connections 2024-03-19T00:59:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.349+0000 7f3fbe400700 1 -- 172.21.15.69:0/2665217877 wait complete. 2024-03-19T00:59:57.351 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:57 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:57.351 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:57 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:57.351 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:57 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:57.351 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:57 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:57.351 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:57 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-03-19T00:59:57.351 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:57 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T00:59:57.351 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:57 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T00:59:57.351 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:57 smithi069 ceph-mon[31727]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-19T00:59:57.351 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:57 smithi069 ceph-mon[31727]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T00:59:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.350+0000 7f3fbe400700 1 Processor -- start 2024-03-19T00:59:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.350+0000 7f3fbe400700 1 -- start start 2024-03-19T00:59:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.351+0000 7f3fbe400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fb8101f40 0x7f3fb819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.351+0000 7f3fbe400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb810c310 con 0x7f3fb8101f40 2024-03-19T00:59:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.351+0000 7f3fb7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fb8101f40 0x7f3fb819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.351+0000 7f3fb7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fb8101f40 0x7f3fb819d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50508/0 (socket says 172.21.15.69:50508) 2024-03-19T00:59:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.351+0000 7f3fb7400700 1 -- 172.21.15.69:0/4234999466 learned_addr learned my addr 172.21.15.69:0/4234999466 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T00:59:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.352+0000 7f3fb7400700 1 -- 172.21.15.69:0/4234999466 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb819dac0 con 0x7f3fb8101f40 2024-03-19T00:59:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.352+0000 7f3fb7400700 1 --2- 172.21.15.69:0/4234999466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fb8101f40 0x7f3fb819d580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8005850 tx=0x7f3fa8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.352+0000 7f3fafe00700 1 -- 172.21.15.69:0/4234999466 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fa8026070 con 0x7f3fb8101f40 2024-03-19T00:59:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.352+0000 7f3fafe00700 1 -- 172.21.15.69:0/4234999466 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa800bbd0 con 0x7f3fb8101f40 2024-03-19T00:59:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.352+0000 7f3fafe00700 1 -- 172.21.15.69:0/4234999466 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fa8003970 con 0x7f3fb8101f40 2024-03-19T00:59:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.352+0000 7f3fbe400700 1 -- 172.21.15.69:0/4234999466 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb819ddb0 con 0x7f3fb8101f40 2024-03-19T00:59:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.352+0000 7f3fbe400700 1 -- 172.21.15.69:0/4234999466 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb8197820 con 0x7f3fb8101f40 2024-03-19T00:59:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.353+0000 7f3fafe00700 1 -- 172.21.15.69:0/4234999466 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f3fa8029070 con 0x7f3fb8101f40 2024-03-19T00:59:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.354+0000 7f3fbe400700 1 -- 172.21.15.69:0/4234999466 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb8103420 con 0x7f3fb8101f40 2024-03-19T00:59:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.354+0000 7f3fafe00700 1 --2- 172.21.15.69:0/4234999466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3f9c039020 0x7f3f9c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T00:59:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.354+0000 7f3fafe00700 1 -- 172.21.15.69:0/4234999466 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3fa801e070 con 0x7f3fb8101f40 2024-03-19T00:59:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.354+0000 7f3fb6a00700 1 --2- 172.21.15.69:0/4234999466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3f9c039020 0x7f3f9c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T00:59:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.355+0000 7f3fb6a00700 1 --2- 172.21.15.69:0/4234999466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3f9c039020 0x7f3f9c03b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4006fd0 tx=0x7f3fa4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T00:59:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.359+0000 7f3fafe00700 1 -- 172.21.15.69:0/4234999466 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3fa8013070 con 0x7f3fb8101f40 2024-03-19T00:59:57.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T00:59:57.592+0000 7f3fbe400700 1 -- 172.21.15.69:0/4234999466 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}) -- 0x7f3fb8198bb0 con 0x7f3f9c039020 2024-03-19T00:59:58.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:58 smithi069 ceph-mon[31727]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-03-19T00:59:58.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:58 smithi069 ceph-mon[31727]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T00:59:58.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:58.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:58.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T00:59:58.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:58 smithi069 ceph-mon[31727]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T00:59:58.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:58 smithi069 ceph-mon[31727]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T00:59:59.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 00:59:59 smithi069 ceph-mon[31727]: Deploying cephadm binary to smithi163 2024-03-19T01:00:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:00 smithi069 ceph-mon[31727]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:02.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:02.255+0000 7f3fafe00700 1 -- 172.21.15.69:0/4234999466 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f3fb8198bb0 con 0x7f3f9c039020 2024-03-19T01:00:02.257 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi163' with addr '172.21.15.163' 2024-03-19T01:00:02.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:02.259+0000 7f3fbe400700 1 -- 172.21.15.69:0/4234999466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3f9c039020 msgr2=0x7f3f9c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:02.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:02.259+0000 7f3fbe400700 1 --2- 172.21.15.69:0/4234999466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3f9c039020 0x7f3f9c03b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4006fd0 tx=0x7f3fa4006e40 comp rx=0 tx=0).stop 2024-03-19T01:00:02.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:02.259+0000 7f3fbe400700 1 -- 172.21.15.69:0/4234999466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fb8101f40 msgr2=0x7f3fb819d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:02.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:02.259+0000 7f3fbe400700 1 --2- 172.21.15.69:0/4234999466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fb8101f40 0x7f3fb819d580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8005850 tx=0x7f3fa8005030 comp rx=0 tx=0).stop 2024-03-19T01:00:02.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:02.259+0000 7f3fbe400700 1 -- 172.21.15.69:0/4234999466 shutdown_connections 2024-03-19T01:00:02.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:02.259+0000 7f3fbe400700 1 --2- 172.21.15.69:0/4234999466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3f9c039020 0x7f3f9c03b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:02.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:02.259+0000 7f3fbe400700 1 --2- 172.21.15.69:0/4234999466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fb8101f40 0x7f3fb819d580 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:02.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:02.259+0000 7f3fbe400700 1 -- 172.21.15.69:0/4234999466 >> 172.21.15.69:0/4234999466 conn(0x7f3fb80fd7f0 msgr2=0x7f3fb8103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:02.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:02.259+0000 7f3fbe400700 1 -- 172.21.15.69:0/4234999466 shutdown_connections 2024-03-19T01:00:02.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:02.259+0000 7f3fbe400700 1 -- 172.21.15.69:0/4234999466 wait complete. 2024-03-19T01:00:02.924 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:02 smithi069 ceph-mon[31727]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:02.924 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:02.924 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:02.971 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-19T01:00:03.408 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:00:03.832 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:03 smithi069 ceph-mon[31727]: Added host smithi163 2024-03-19T01:00:03.832 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:03.832 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:05.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:04 smithi069 ceph-mon[31727]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:05.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.540+0000 7fb442a00700 1 -- 172.21.15.69:0/4226621573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb43c108570 msgr2=0x7fb43c108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:05.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.540+0000 7fb442a00700 1 --2- 172.21.15.69:0/4226621573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb43c108570 0x7fb43c108950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009910 tx=0x7fb42c009c20 comp rx=0 tx=0).stop 2024-03-19T01:00:05.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.540+0000 7fb442a00700 1 -- 172.21.15.69:0/4226621573 shutdown_connections 2024-03-19T01:00:05.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.540+0000 7fb442a00700 1 --2- 172.21.15.69:0/4226621573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb43c108570 0x7fb43c108950 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:05.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.540+0000 7fb442a00700 1 -- 172.21.15.69:0/4226621573 >> 172.21.15.69:0/4226621573 conn(0x7fb43c077ab0 msgr2=0x7fb43c079ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:05.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.540+0000 7fb442a00700 1 -- 172.21.15.69:0/4226621573 shutdown_connections 2024-03-19T01:00:05.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.540+0000 7fb442a00700 1 -- 172.21.15.69:0/4226621573 wait complete. 2024-03-19T01:00:05.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.541+0000 7fb442a00700 1 Processor -- start 2024-03-19T01:00:05.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.541+0000 7fb442a00700 1 -- start start 2024-03-19T01:00:05.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.542+0000 7fb442a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb43c108570 0x7fb43c19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:05.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.542+0000 7fb442a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c112940 con 0x7fb43c108570 2024-03-19T01:00:05.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.542+0000 7fb43be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb43c108570 0x7fb43c19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:05.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.542+0000 7fb43be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb43c108570 0x7fb43c19de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50520/0 (socket says 172.21.15.69:50520) 2024-03-19T01:00:05.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.542+0000 7fb43be00700 1 -- 172.21.15.69:0/946443000 learned_addr learned my addr 172.21.15.69:0/946443000 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:00:05.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.542+0000 7fb43be00700 1 -- 172.21.15.69:0/946443000 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43c19e340 con 0x7fb43c108570 2024-03-19T01:00:05.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.543+0000 7fb43be00700 1 --2- 172.21.15.69:0/946443000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb43c108570 0x7fb43c19de00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006010 tx=0x7fb42c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:05.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.543+0000 7fb438c00700 1 -- 172.21.15.69:0/946443000 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb42c026070 con 0x7fb43c108570 2024-03-19T01:00:05.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.543+0000 7fb438c00700 1 -- 172.21.15.69:0/946443000 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c00bbd0 con 0x7fb43c108570 2024-03-19T01:00:05.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.543+0000 7fb438c00700 1 -- 172.21.15.69:0/946443000 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb42c003910 con 0x7fb43c108570 2024-03-19T01:00:05.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.543+0000 7fb442a00700 1 -- 172.21.15.69:0/946443000 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43c19e630 con 0x7fb43c108570 2024-03-19T01:00:05.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.543+0000 7fb442a00700 1 -- 172.21.15.69:0/946443000 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43c1a26d0 con 0x7fb43c108570 2024-03-19T01:00:05.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.544+0000 7fb438c00700 1 -- 172.21.15.69:0/946443000 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fb42c029070 con 0x7fb43c108570 2024-03-19T01:00:05.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.544+0000 7fb442a00700 1 -- 172.21.15.69:0/946443000 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43c109a50 con 0x7fb43c108570 2024-03-19T01:00:05.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.545+0000 7fb438c00700 1 --2- 172.21.15.69:0/946443000 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb420039020 0x7fb42003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:05.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.545+0000 7fb438c00700 1 -- 172.21.15.69:0/946443000 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb42c01e070 con 0x7fb43c108570 2024-03-19T01:00:05.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.545+0000 7fb43b400700 1 --2- 172.21.15.69:0/946443000 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb420039020 0x7fb42003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:05.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.546+0000 7fb43b400700 1 --2- 172.21.15.69:0/946443000 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb420039020 0x7fb42003b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb428006fd0 tx=0x7fb428006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:05.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.550+0000 7fb438c00700 1 -- 172.21.15.69:0/946443000 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb42c013070 con 0x7fb43c108570 2024-03-19T01:00:05.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.777+0000 7fb442a00700 1 -- 172.21.15.69:0/946443000 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb43c19f160 con 0x7fb420039020 2024-03-19T01:00:05.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.779+0000 7fb438c00700 1 -- 172.21.15.69:0/946443000 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fb43c19f160 con 0x7fb420039020 2024-03-19T01:00:05.781 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:00:05.781 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.163", "hostname": "smithi163", "labels": [], "status": ""}] 2024-03-19T01:00:05.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.783+0000 7fb442a00700 1 -- 172.21.15.69:0/946443000 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb420039020 msgr2=0x7fb42003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:05.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.783+0000 7fb442a00700 1 --2- 172.21.15.69:0/946443000 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb420039020 0x7fb42003b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb428006fd0 tx=0x7fb428006e40 comp rx=0 tx=0).stop 2024-03-19T01:00:05.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.783+0000 7fb442a00700 1 -- 172.21.15.69:0/946443000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb43c108570 msgr2=0x7fb43c19de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:05.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.783+0000 7fb442a00700 1 --2- 172.21.15.69:0/946443000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb43c108570 0x7fb43c19de00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006010 tx=0x7fb42c005030 comp rx=0 tx=0).stop 2024-03-19T01:00:05.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.783+0000 7fb442a00700 1 -- 172.21.15.69:0/946443000 shutdown_connections 2024-03-19T01:00:05.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.783+0000 7fb442a00700 1 --2- 172.21.15.69:0/946443000 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb420039020 0x7fb42003b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:05.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.783+0000 7fb442a00700 1 --2- 172.21.15.69:0/946443000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb43c108570 0x7fb43c19de00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:05.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.783+0000 7fb442a00700 1 -- 172.21.15.69:0/946443000 >> 172.21.15.69:0/946443000 conn(0x7fb43c077ab0 msgr2=0x7fb43c109f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:05.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.783+0000 7fb442a00700 1 -- 172.21.15.69:0/946443000 shutdown_connections 2024-03-19T01:00:05.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:05.783+0000 7fb442a00700 1 -- 172.21.15.69:0/946443000 wait complete. 2024-03-19T01:00:06.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:06.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:06 smithi069 ceph-mon[31727]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:06.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:06 smithi069 ceph-mon[31727]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:00:06.550 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-19T01:00:06.551 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-19T01:00:06.984 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:00:08.754 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:08 smithi069 ceph-mon[31727]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:08.754 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:08.755 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:08.755 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:08.755 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:08.755 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:00:08.755 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:08.755 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.922+0000 7f908e000700 1 -- 172.21.15.69:0/1430575623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9088102ea0 msgr2=0x7f9088103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.922+0000 7f908e000700 1 --2- 172.21.15.69:0/1430575623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9088102ea0 0x7f9088103280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9078009910 tx=0x7f9078009c20 comp rx=0 tx=0).stop 2024-03-19T01:00:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.923+0000 7f908e000700 1 -- 172.21.15.69:0/1430575623 shutdown_connections 2024-03-19T01:00:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.923+0000 7f908e000700 1 --2- 172.21.15.69:0/1430575623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9088102ea0 0x7f9088103280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.923+0000 7f908e000700 1 -- 172.21.15.69:0/1430575623 >> 172.21.15.69:0/1430575623 conn(0x7f9088077850 msgr2=0x7f90880feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.923+0000 7f908e000700 1 -- 172.21.15.69:0/1430575623 shutdown_connections 2024-03-19T01:00:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.923+0000 7f908e000700 1 -- 172.21.15.69:0/1430575623 wait complete. 2024-03-19T01:00:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.924+0000 7f908e000700 1 Processor -- start 2024-03-19T01:00:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.924+0000 7f908e000700 1 -- start start 2024-03-19T01:00:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.925+0000 7f908e000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9088102ea0 0x7f908819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.925+0000 7f908e000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f908810c390 con 0x7f9088102ea0 2024-03-19T01:00:08.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.925+0000 7f9087400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9088102ea0 0x7f908819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:08.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.925+0000 7f9087400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9088102ea0 0x7f908819f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37982/0 (socket says 172.21.15.69:37982) 2024-03-19T01:00:08.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.925+0000 7f9087400700 1 -- 172.21.15.69:0/3738398575 learned_addr learned my addr 172.21.15.69:0/3738398575 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:00:08.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.925+0000 7f9087400700 1 -- 172.21.15.69:0/3738398575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908819fc20 con 0x7f9088102ea0 2024-03-19T01:00:08.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.926+0000 7f9087400700 1 --2- 172.21.15.69:0/3738398575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9088102ea0 0x7f908819f6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9078006010 tx=0x7f9078005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:08.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.926+0000 7f907fe00700 1 -- 172.21.15.69:0/3738398575 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9078026070 con 0x7f9088102ea0 2024-03-19T01:00:08.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.926+0000 7f907fe00700 1 -- 172.21.15.69:0/3738398575 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f907800bc30 con 0x7f9088102ea0 2024-03-19T01:00:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.926+0000 7f908e000700 1 -- 172.21.15.69:0/3738398575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908819feb0 con 0x7f9088102ea0 2024-03-19T01:00:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.926+0000 7f907fe00700 1 -- 172.21.15.69:0/3738398575 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90780039a0 con 0x7f9088102ea0 2024-03-19T01:00:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.926+0000 7f908e000700 1 -- 172.21.15.69:0/3738398575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90881998a0 con 0x7f9088102ea0 2024-03-19T01:00:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.927+0000 7f908e000700 1 -- 172.21.15.69:0/3738398575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9088069fe0 con 0x7f9088102ea0 2024-03-19T01:00:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.927+0000 7f907fe00700 1 -- 172.21.15.69:0/3738398575 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f9078029070 con 0x7f9088102ea0 2024-03-19T01:00:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.928+0000 7f907fe00700 1 --2- 172.21.15.69:0/3738398575 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f906c039020 0x7f906c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.928+0000 7f907fe00700 1 -- 172.21.15.69:0/3738398575 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f907801e070 con 0x7f9088102ea0 2024-03-19T01:00:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.928+0000 7f9086a00700 1 --2- 172.21.15.69:0/3738398575 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f906c039020 0x7f906c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.929+0000 7f9086a00700 1 --2- 172.21.15.69:0/3738398575 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f906c039020 0x7f906c03b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9074006fd0 tx=0x7f9074006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:08.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:08.935+0000 7f907fe00700 1 -- 172.21.15.69:0/3738398575 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9078013070 con 0x7f9088102ea0 2024-03-19T01:00:09.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.163+0000 7f908e000700 1 -- 172.21.15.69:0/3738398575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f908819add0 con 0x7f9088102ea0 2024-03-19T01:00:09.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.584+0000 7f907fe00700 1 -- 172.21.15.69:0/3738398575 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f9078030090 con 0x7f9088102ea0 2024-03-19T01:00:09.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.588+0000 7f908e000700 1 -- 172.21.15.69:0/3738398575 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f906c039020 msgr2=0x7f906c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:09.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.588+0000 7f908e000700 1 --2- 172.21.15.69:0/3738398575 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f906c039020 0x7f906c03b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9074006fd0 tx=0x7f9074006e40 comp rx=0 tx=0).stop 2024-03-19T01:00:09.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.588+0000 7f908e000700 1 -- 172.21.15.69:0/3738398575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9088102ea0 msgr2=0x7f908819f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:09.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.588+0000 7f908e000700 1 --2- 172.21.15.69:0/3738398575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9088102ea0 0x7f908819f6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9078006010 tx=0x7f9078005030 comp rx=0 tx=0).stop 2024-03-19T01:00:09.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.588+0000 7f908e000700 1 -- 172.21.15.69:0/3738398575 shutdown_connections 2024-03-19T01:00:09.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.588+0000 7f908e000700 1 --2- 172.21.15.69:0/3738398575 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f906c039020 0x7f906c03b4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:09.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.588+0000 7f908e000700 1 --2- 172.21.15.69:0/3738398575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9088102ea0 0x7f908819f6e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:09.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.588+0000 7f908e000700 1 -- 172.21.15.69:0/3738398575 >> 172.21.15.69:0/3738398575 conn(0x7f9088077850 msgr2=0x7f9088108060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:09.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.589+0000 7f908e000700 1 -- 172.21.15.69:0/3738398575 shutdown_connections 2024-03-19T01:00:09.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:09.589+0000 7f908e000700 1 -- 172.21.15.69:0/3738398575 wait complete. 2024-03-19T01:00:09.591 INFO:teuthology.orchestra.run.smithi069.stderr:adjusted tunables profile to default 2024-03-19T01:00:09.937 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:09 smithi069 ceph-mon[31727]: Updating smithi163:/etc/ceph/ceph.conf 2024-03-19T01:00:09.937 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:09 smithi069 ceph-mon[31727]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:09.937 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:09 smithi069 ceph-mon[31727]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:00:09.937 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:09 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3738398575' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-19T01:00:09.937 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:09.937 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:09.937 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:10.232 INFO:tasks.cephadm:Adding mon.a on smithi069 2024-03-19T01:00:10.232 INFO:tasks.cephadm:Adding mon.b on smithi086 2024-03-19T01:00:10.233 INFO:tasks.cephadm:Adding mon.c on smithi163 2024-03-19T01:00:10.233 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi069:172.21.15.69=a;smithi086:172.21.15.86=b;smithi163:172.21.15.163=c' 2024-03-19T01:00:10.464 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:11.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:10 smithi069 ceph-mon[31727]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:00:11.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:10 smithi069 ceph-mon[31727]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:11.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:10 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3738398575' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-19T01:00:11.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:10 smithi069 ceph-mon[31727]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.865+0000 7fcd66ffd700 1 -- 172.21.15.163:0/1116917900 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd58005590 con 0x7fcd68100e40 2024-03-19T01:00:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.865+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/1116917900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd68100e40 msgr2=0x7fcd68101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.865+0000 7fcd6e3ad700 1 --2- 172.21.15.163:0/1116917900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd68100e40 0x7fcd68101220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009910 tx=0x7fcd58009c20 comp rx=0 tx=0).stop 2024-03-19T01:00:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.865+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/1116917900 shutdown_connections 2024-03-19T01:00:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.865+0000 7fcd6e3ad700 1 --2- 172.21.15.163:0/1116917900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd68100e40 0x7fcd68101220 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.865+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/1116917900 >> 172.21.15.163:0/1116917900 conn(0x7fcd680fc6b0 msgr2=0x7fcd680fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.865+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/1116917900 shutdown_connections 2024-03-19T01:00:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.865+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/1116917900 wait complete. 2024-03-19T01:00:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.866+0000 7fcd6e3ad700 1 Processor -- start 2024-03-19T01:00:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.866+0000 7fcd6e3ad700 1 -- start start 2024-03-19T01:00:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.866+0000 7fcd6e3ad700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd68100e40 0x7fcd6819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.866+0000 7fcd6e3ad700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6810b210 con 0x7fcd68100e40 2024-03-19T01:00:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.867+0000 7fcd67fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd68100e40 0x7fcd6819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.867+0000 7fcd67fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd68100e40 0x7fcd6819d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.163:41596/0 (socket says 172.21.15.163:41596) 2024-03-19T01:00:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.867+0000 7fcd67fff700 1 -- 172.21.15.163:0/860925697 learned_addr learned my addr 172.21.15.163:0/860925697 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-19T01:00:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.867+0000 7fcd67fff700 1 -- 172.21.15.163:0/860925697 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6819dad0 con 0x7fcd68100e40 2024-03-19T01:00:11.869 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.868+0000 7fcd67fff700 1 --2- 172.21.15.163:0/860925697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd68100e40 0x7fcd6819d590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005850 tx=0x7fcd58005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:11.869 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.868+0000 7fcd657fa700 1 -- 172.21.15.163:0/860925697 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd58026070 con 0x7fcd68100e40 2024-03-19T01:00:11.870 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.868+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/860925697 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6819ddc0 con 0x7fcd68100e40 2024-03-19T01:00:11.870 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.868+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/860925697 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd68197830 con 0x7fcd68100e40 2024-03-19T01:00:11.870 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.868+0000 7fcd657fa700 1 -- 172.21.15.163:0/860925697 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd5800b6f0 con 0x7fcd68100e40 2024-03-19T01:00:11.870 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.868+0000 7fcd657fa700 1 -- 172.21.15.163:0/860925697 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd58030060 con 0x7fcd68100e40 2024-03-19T01:00:11.870 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.869+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/860925697 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd54005330 con 0x7fcd68100e40 2024-03-19T01:00:11.870 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.870+0000 7fcd657fa700 1 -- 172.21.15.163:0/860925697 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fcd58029070 con 0x7fcd68100e40 2024-03-19T01:00:11.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.870+0000 7fcd657fa700 1 --2- 172.21.15.163:0/860925697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcd50038fc0 0x7fcd5003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:11.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.870+0000 7fcd657fa700 1 -- 172.21.15.163:0/860925697 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcd5801e070 con 0x7fcd68100e40 2024-03-19T01:00:11.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.870+0000 7fcd677fe700 1 --2- 172.21.15.163:0/860925697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcd50038fc0 0x7fcd5003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:11.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.871+0000 7fcd677fe700 1 --2- 172.21.15.163:0/860925697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcd50038fc0 0x7fcd5003b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c006fd0 tx=0x7fcd5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:11.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:11.885+0000 7fcd657fa700 1 -- 172.21.15.163:0/860925697 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcd58013070 con 0x7fcd68100e40 2024-03-19T01:00:12.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.095+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/860925697 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi086:172.21.15.86=b;smithi163:172.21.15.163=c", "target": ["mon-mgr", ""]}) -- 0x7fcd54000c80 con 0x7fcd50038fc0 2024-03-19T01:00:12.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.110+0000 7fcd657fa700 1 -- 172.21.15.163:0/860925697 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcd54000c80 con 0x7fcd50038fc0 2024-03-19T01:00:12.110 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled mon update... 2024-03-19T01:00:12.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.111+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/860925697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcd50038fc0 msgr2=0x7fcd5003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:12.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.111+0000 7fcd6e3ad700 1 --2- 172.21.15.163:0/860925697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcd50038fc0 0x7fcd5003b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c006fd0 tx=0x7fcd5c006e40 comp rx=0 tx=0).stop 2024-03-19T01:00:12.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.111+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/860925697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd68100e40 msgr2=0x7fcd6819d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:12.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.111+0000 7fcd6e3ad700 1 --2- 172.21.15.163:0/860925697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd68100e40 0x7fcd6819d590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005850 tx=0x7fcd58005030 comp rx=0 tx=0).stop 2024-03-19T01:00:12.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.111+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/860925697 shutdown_connections 2024-03-19T01:00:12.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.111+0000 7fcd6e3ad700 1 --2- 172.21.15.163:0/860925697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcd50038fc0 0x7fcd5003b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:12.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.111+0000 7fcd6e3ad700 1 --2- 172.21.15.163:0/860925697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd68100e40 0x7fcd6819d590 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:12.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.111+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/860925697 >> 172.21.15.163:0/860925697 conn(0x7fcd680fc6b0 msgr2=0x7fcd6806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:12.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.111+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/860925697 shutdown_connections 2024-03-19T01:00:12.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:12.112+0000 7fcd6e3ad700 1 -- 172.21.15.163:0/860925697 wait complete. 2024-03-19T01:00:12.730 DEBUG:teuthology.orchestra.run.smithi086:mon.b> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.b.service 2024-03-19T01:00:12.733 DEBUG:teuthology.orchestra.run.smithi163:mon.c> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.c.service 2024-03-19T01:00:12.735 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-19T01:00:12.735 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-19T01:00:13.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:12 smithi069 ceph-mon[31727]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:13.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:13.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:13.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:13.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:13.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:13.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:13.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:13.433 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:13 smithi069 ceph-mon[31727]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi086:172.21.15.86=b;smithi163:172.21.15.163=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:00:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:13 smithi069 ceph-mon[31727]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi086:172.21.15.86=b;smithi163:172.21.15.163=c;count:3 2024-03-19T01:00:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:13 smithi069 ceph-mon[31727]: Deploying daemon mon.c on smithi163 2024-03-19T01:00:15.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:14 smithi069 ceph-mon[31727]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:15.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.678+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/2450422128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8d0101f40 msgr2=0x7fd8d0102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:15.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.678+0000 7fd8d4e8c700 1 --2- 172.21.15.163:0/2450422128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8d0101f40 0x7fd8d0102320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009910 tx=0x7fd8c0009c20 comp rx=0 tx=0).stop 2024-03-19T01:00:15.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.678+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/2450422128 shutdown_connections 2024-03-19T01:00:15.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.678+0000 7fd8d4e8c700 1 --2- 172.21.15.163:0/2450422128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8d0101f40 0x7fd8d0102320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:15.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.678+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/2450422128 >> 172.21.15.163:0/2450422128 conn(0x7fd8d00fd850 msgr2=0x7fd8d00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:15.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.679+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/2450422128 shutdown_connections 2024-03-19T01:00:15.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.679+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/2450422128 wait complete. 2024-03-19T01:00:15.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.679+0000 7fd8d4e8c700 1 Processor -- start 2024-03-19T01:00:15.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.679+0000 7fd8d4e8c700 1 -- start start 2024-03-19T01:00:15.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.680+0000 7fd8d4e8c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8d0101f40 0x7fd8d019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:15.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.680+0000 7fd8d4e8c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d010c310 con 0x7fd8d0101f40 2024-03-19T01:00:15.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.680+0000 7fd8ce59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8d0101f40 0x7fd8d019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:15.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.680+0000 7fd8ce59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8d0101f40 0x7fd8d019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.163:54258/0 (socket says 172.21.15.163:54258) 2024-03-19T01:00:15.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.680+0000 7fd8ce59c700 1 -- 172.21.15.163:0/16421857 learned_addr learned my addr 172.21.15.163:0/16421857 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-19T01:00:15.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.681+0000 7fd8ce59c700 1 -- 172.21.15.163:0/16421857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d019db50 con 0x7fd8d0101f40 2024-03-19T01:00:15.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.681+0000 7fd8ce59c700 1 --2- 172.21.15.163:0/16421857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8d0101f40 0x7fd8d019d610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0005850 tx=0x7fd8c0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:15.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.681+0000 7fd8c7fff700 1 -- 172.21.15.163:0/16421857 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8c0026070 con 0x7fd8d0101f40 2024-03-19T01:00:15.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.681+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/16421857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d019de40 con 0x7fd8d0101f40 2024-03-19T01:00:15.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.682+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/16421857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d01978b0 con 0x7fd8d0101f40 2024-03-19T01:00:15.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.682+0000 7fd8c7fff700 1 -- 172.21.15.163:0/16421857 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8c000bbd0 con 0x7fd8d0101f40 2024-03-19T01:00:15.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.682+0000 7fd8c7fff700 1 -- 172.21.15.163:0/16421857 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8c0030060 con 0x7fd8d0101f40 2024-03-19T01:00:15.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.682+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/16421857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d0103420 con 0x7fd8d0101f40 2024-03-19T01:00:15.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.683+0000 7fd8c7fff700 1 -- 172.21.15.163:0/16421857 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fd8c0029070 con 0x7fd8d0101f40 2024-03-19T01:00:15.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.683+0000 7fd8c7fff700 1 --2- 172.21.15.163:0/16421857 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd8bc038fc0 0x7fd8bc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:15.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.683+0000 7fd8c7fff700 1 -- 172.21.15.163:0/16421857 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd8c001e070 con 0x7fd8d0101f40 2024-03-19T01:00:15.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.684+0000 7fd8c5d9b700 1 --2- 172.21.15.163:0/16421857 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd8bc038fc0 0x7fd8bc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:15.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.685+0000 7fd8c5d9b700 1 --2- 172.21.15.163:0/16421857 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd8bc038fc0 0x7fd8bc03b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8006fd0 tx=0x7fd8b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:15.699 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.698+0000 7fd8c7fff700 1 -- 172.21.15.163:0/16421857 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd8c0013070 con 0x7fd8d0101f40 2024-03-19T01:00:15.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.977+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/16421857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd8d00008d0 con 0x7fd8d0101f40 2024-03-19T01:00:15.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.978+0000 7fd8c7fff700 1 -- 172.21.15.163:0/16421857 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fd8c0031250 con 0x7fd8d0101f40 2024-03-19T01:00:15.979 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T01:00:15.979 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","modified":"2024-03-19T00:58:05.726813Z","created":"2024-03-19T00:58:05.726813Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-19T01:00:15.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.980+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/16421857 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd8bc038fc0 msgr2=0x7fd8bc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:15.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.980+0000 7fd8d4e8c700 1 --2- 172.21.15.163:0/16421857 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd8bc038fc0 0x7fd8bc03b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8006fd0 tx=0x7fd8b8006e40 comp rx=0 tx=0).stop 2024-03-19T01:00:15.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.980+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/16421857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8d0101f40 msgr2=0x7fd8d019d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:15.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.980+0000 7fd8d4e8c700 1 --2- 172.21.15.163:0/16421857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8d0101f40 0x7fd8d019d610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0005850 tx=0x7fd8c0005030 comp rx=0 tx=0).stop 2024-03-19T01:00:15.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.980+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/16421857 shutdown_connections 2024-03-19T01:00:15.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.980+0000 7fd8d4e8c700 1 --2- 172.21.15.163:0/16421857 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd8bc038fc0 0x7fd8bc03b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:15.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.980+0000 7fd8d4e8c700 1 --2- 172.21.15.163:0/16421857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8d0101f40 0x7fd8d019d610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:15.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.980+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/16421857 >> 172.21.15.163:0/16421857 conn(0x7fd8d00fd850 msgr2=0x7fd8d0104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:15.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.980+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/16421857 shutdown_connections 2024-03-19T01:00:15.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:15.980+0000 7fd8d4e8c700 1 -- 172.21.15.163:0/16421857 wait complete. 2024-03-19T01:00:15.982 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-03-19T01:00:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:16 smithi069 ceph-mon[31727]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:16 smithi069 ceph-mon[31727]: from='client.? 172.21.15.163:0/16421857' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:00:17.588 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).mds e1 new map 2024-03-19T01:00:17.588 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-19T01:00:17.588 INFO:journalctl@ceph.mon.c.smithi163.stdout: e1 2024-03-19T01:00:17.588 INFO:journalctl@ceph.mon.c.smithi163.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-19T01:00:17.588 INFO:journalctl@ceph.mon.c.smithi163.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T01:00:17.588 INFO:journalctl@ceph.mon.c.smithi163.stdout: legacy client fscid: -1 2024-03-19T01:00:17.588 INFO:journalctl@ceph.mon.c.smithi163.stdout: 2024-03-19T01:00:17.588 INFO:journalctl@ceph.mon.c.smithi163.stdout: No filesystems configured 2024-03-19T01:00:17.588 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-19T01:00:17.588 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-19T01:00:17.731 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-19T01:00:17.732 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-19T01:00:17.966 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:00:18.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-19T01:00:18.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-19T01:00:18.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-19T01:00:18.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-19T01:00:18.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Added host smithi086 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Deploying cephadm binary to smithi163 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Added host smithi163 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Updating smithi163:/etc/ceph/ceph.conf 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3738398575' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3738398575' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi086:172.21.15.86=b;smithi163:172.21.15.163=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi086:172.21.15.86=b;smithi163:172.21.15.163=c;count:3 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: Deploying daemon mon.c on smithi163 2024-03-19T01:00:18.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.068 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:18.068 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: from='client.? 172.21.15.163:0/16421857' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:00:18.068 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 ceph-mon[67428]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-19T01:00:18.068 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 bash[67409]: 5e9f2948f746f4df094eb32109fdf9457cdba128ea7127d2293ff61de7bdfac8 2024-03-19T01:00:18.068 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:17 smithi163 systemd[1]: Started Ceph mon.c for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:00:22.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.891+0000 7fe8aa224700 1 -- 172.21.15.163:0/2337618704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe88c0035c0 msgr2=0x7fe88c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:22.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.891+0000 7fe8aa224700 1 --2- 172.21.15.163:0/2337618704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe88c0035c0 0x7fe88c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4100c10 tx=0x7fe894014ec0 comp rx=0 tx=0).stop 2024-03-19T01:00:22.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.891+0000 7fe8aa224700 1 -- 172.21.15.163:0/2337618704 shutdown_connections 2024-03-19T01:00:22.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.891+0000 7fe8aa224700 1 --2- 172.21.15.163:0/2337618704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe88c0035c0 0x7fe88c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:22.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.891+0000 7fe8aa224700 1 -- 172.21.15.163:0/2337618704 >> 172.21.15.163:0/2337618704 conn(0x7fe8a40fc7b0 msgr2=0x7fe8a40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:22.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.891+0000 7fe8aa224700 1 -- 172.21.15.163:0/2337618704 shutdown_connections 2024-03-19T01:00:22.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.891+0000 7fe8aa224700 1 -- 172.21.15.163:0/2337618704 wait complete. 2024-03-19T01:00:22.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.892+0000 7fe8aa224700 1 Processor -- start 2024-03-19T01:00:22.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8aa224700 1 -- start start 2024-03-19T01:00:22.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8aa224700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe88c0035c0 0x7fe8a419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:22.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8aa224700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a419fcf0 0x7fe8a4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:22.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8aa224700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4108d70 con 0x7fe8a419fcf0 2024-03-19T01:00:22.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8aa224700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4108bf0 con 0x7fe88c0035c0 2024-03-19T01:00:22.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8a37fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe88c0035c0 0x7fe8a419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:22.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8a37fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe88c0035c0 0x7fe8a419f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40316/0 (socket says 172.21.15.163:40316) 2024-03-19T01:00:22.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8a37fe700 1 -- 172.21.15.163:0/964226145 learned_addr learned my addr 172.21.15.163:0/964226145 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-19T01:00:22.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8a37fe700 1 -- 172.21.15.163:0/964226145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe88c0035c0 msgr2=0x7fe8a419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-19T01:00:22.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8a37fe700 1 -- 172.21.15.163:0/964226145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe88c0035c0 msgr2=0x7fe8a419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-19T01:00:22.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8a37fe700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe88c0035c0 0x7fe8a419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-19T01:00:22.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8a37fe700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe88c0035c0 0x7fe8a419f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:00:22.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.893+0000 7fe8a2ffd700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a419fcf0 0x7fe8a4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:22.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.894+0000 7fe8a2ffd700 1 -- 172.21.15.163:0/964226145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe88c0035c0 msgr2=0x7fe8a419f7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:00:22.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.894+0000 7fe8a2ffd700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe88c0035c0 0x7fe8a419f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:22.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.894+0000 7fe8a2ffd700 1 -- 172.21.15.163:0/964226145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a41a0330 con 0x7fe8a419fcf0 2024-03-19T01:00:22.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.894+0000 7fe8a2ffd700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a419fcf0 0x7fe8a4199970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe89800b4d0 tx=0x7fe89800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:22.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.895+0000 7fe8a0ff9700 1 -- 172.21.15.163:0/964226145 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe898011630 con 0x7fe8a419fcf0 2024-03-19T01:00:22.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.895+0000 7fe8a0ff9700 1 -- 172.21.15.163:0/964226145 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe898011cb0 con 0x7fe8a419fcf0 2024-03-19T01:00:22.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.895+0000 7fe8a0ff9700 1 -- 172.21.15.163:0/964226145 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe898010e60 con 0x7fe8a419fcf0 2024-03-19T01:00:22.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.895+0000 7fe8aa224700 1 -- 172.21.15.163:0/964226145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a4199eb0 con 0x7fe8a419fcf0 2024-03-19T01:00:22.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.895+0000 7fe8aa224700 1 -- 172.21.15.163:0/964226145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a419a440 con 0x7fe8a419fcf0 2024-03-19T01:00:22.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.896+0000 7fe8aa224700 1 -- 172.21.15.163:0/964226145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a4102420 con 0x7fe8a419fcf0 2024-03-19T01:00:22.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.897+0000 7fe8a0ff9700 1 -- 172.21.15.163:0/964226145 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fe898018020 con 0x7fe8a419fcf0 2024-03-19T01:00:22.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.897+0000 7fe8a0ff9700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe890038d80 0x7fe89003b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:22.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.897+0000 7fe8a0ff9700 1 -- 172.21.15.163:0/964226145 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe89804b9b0 con 0x7fe8a419fcf0 2024-03-19T01:00:22.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.897+0000 7fe8a37fe700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe890038d80 0x7fe89003b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:22.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.898+0000 7fe8a37fe700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe890038d80 0x7fe89003b240 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe89400f470 tx=0x7fe894021040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:22.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:22.912+0000 7fe8a0ff9700 1 -- 172.21.15.163:0/964226145 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe89801f020 con 0x7fe8a419fcf0 2024-03-19T01:00:23.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:23.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: mon.a calling monitor election 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: mon.c calling monitor election 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: monmap epoch 2 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:23.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: last_changed 2024-03-19T01:00:17.593381+0000 2024-03-19T01:00:23.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: created 2024-03-19T00:58:05.726813+0000 2024-03-19T01:00:23.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: min_mon_release 19 (squid) 2024-03-19T01:00:23.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: election_strategy: 1 2024-03-19T01:00:23.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-03-19T01:00:23.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-03-19T01:00:23.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: fsmap 2024-03-19T01:00:23.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:23.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: mgrmap e13: x(active, since 59s) 2024-03-19T01:00:23.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: overall HEALTH_OK 2024-03-19T01:00:23.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:23.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: mon.a calling monitor election 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: mon.c calling monitor election 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: monmap epoch 2 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: last_changed 2024-03-19T01:00:17.593381+0000 2024-03-19T01:00:23.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: created 2024-03-19T00:58:05.726813+0000 2024-03-19T01:00:23.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: min_mon_release 19 (squid) 2024-03-19T01:00:23.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: election_strategy: 1 2024-03-19T01:00:23.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-03-19T01:00:23.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-03-19T01:00:23.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: fsmap 2024-03-19T01:00:23.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:23.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: mgrmap e13: x(active, since 59s) 2024-03-19T01:00:23.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: overall HEALTH_OK 2024-03-19T01:00:23.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:23.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:23.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.192+0000 7fe8aa224700 1 -- 172.21.15.163:0/964226145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe8a419aea0 con 0x7fe8a419fcf0 2024-03-19T01:00:23.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.193+0000 7fe8a0ff9700 1 -- 172.21.15.163:0/964226145 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fe898014170 con 0x7fe8a419fcf0 2024-03-19T01:00:23.194 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T01:00:23.194 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":2,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","modified":"2024-03-19T01:00:17.593381Z","created":"2024-03-19T00:58:05.726813Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:3300","nonce":0},{"type":"v1","addr":"172.21.15.163:6789","nonce":0}]},"addr":"172.21.15.163:6789/0","public_addr":"172.21.15.163:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-19T01:00:23.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.195+0000 7fe8aa224700 1 -- 172.21.15.163:0/964226145 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe890038d80 msgr2=0x7fe89003b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:23.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.195+0000 7fe8aa224700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe890038d80 0x7fe89003b240 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe89400f470 tx=0x7fe894021040 comp rx=0 tx=0).stop 2024-03-19T01:00:23.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.195+0000 7fe8aa224700 1 -- 172.21.15.163:0/964226145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a419fcf0 msgr2=0x7fe8a4199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:23.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.195+0000 7fe8aa224700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a419fcf0 0x7fe8a4199970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe89800b4d0 tx=0x7fe89800b890 comp rx=0 tx=0).stop 2024-03-19T01:00:23.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.195+0000 7fe8aa224700 1 -- 172.21.15.163:0/964226145 shutdown_connections 2024-03-19T01:00:23.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.195+0000 7fe8aa224700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe890038d80 0x7fe89003b240 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:23.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.195+0000 7fe8aa224700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe88c0035c0 0x7fe8a419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:23.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.195+0000 7fe8aa224700 1 --2- 172.21.15.163:0/964226145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a419fcf0 0x7fe8a4199970 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:23.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.195+0000 7fe8aa224700 1 -- 172.21.15.163:0/964226145 >> 172.21.15.163:0/964226145 conn(0x7fe8a40fc7b0 msgr2=0x7fe8a40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:23.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.195+0000 7fe8aa224700 1 -- 172.21.15.163:0/964226145 shutdown_connections 2024-03-19T01:00:23.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:23.195+0000 7fe8aa224700 1 -- 172.21.15.163:0/964226145 wait complete. 2024-03-19T01:00:23.197 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 2 2024-03-19T01:00:23.799 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:23.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:23.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:23.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:23 smithi163 ceph-mon[67428]: Deploying daemon mon.b on smithi086 2024-03-19T01:00:23.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:23 smithi163 ceph-mon[67428]: from='client.? 172.21.15.163:0/964226145' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:00:23.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:23 smithi069 ceph-mon[31727]: Deploying daemon mon.b on smithi086 2024-03-19T01:00:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:23 smithi069 ceph-mon[31727]: from='client.? 172.21.15.163:0/964226145' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:00:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:24.827 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-19T01:00:24.827 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-19T01:00:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:24 smithi069 ceph-mon[31727]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:24 smithi163 ceph-mon[67428]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:25.071 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:00:26.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 podman[66810]: 2024-03-19 01:00:26.327210868 +0000 UTC m=+0.017397293 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:00:26.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.543+0000 7f36bd8bf700 1 -- 172.21.15.163:0/3042172513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b8102040 msgr2=0x7f36b8102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:26.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.543+0000 7f36bd8bf700 1 --2- 172.21.15.163:0/3042172513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b8102040 0x7f36b8102420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009910 tx=0x7f36a8009c20 comp rx=0 tx=0).stop 2024-03-19T01:00:26.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.543+0000 7f36bd8bf700 1 -- 172.21.15.163:0/3042172513 shutdown_connections 2024-03-19T01:00:26.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.543+0000 7f36bd8bf700 1 --2- 172.21.15.163:0/3042172513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b8102040 0x7f36b8102420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:26.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.543+0000 7f36bd8bf700 1 -- 172.21.15.163:0/3042172513 >> 172.21.15.163:0/3042172513 conn(0x7f36b80fd8b0 msgr2=0x7f36b80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:26.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.543+0000 7f36bd8bf700 1 -- 172.21.15.163:0/3042172513 shutdown_connections 2024-03-19T01:00:26.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.543+0000 7f36bd8bf700 1 -- 172.21.15.163:0/3042172513 wait complete. 2024-03-19T01:00:26.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.544+0000 7f36bd8bf700 1 Processor -- start 2024-03-19T01:00:26.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.544+0000 7f36bd8bf700 1 -- start start 2024-03-19T01:00:26.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.544+0000 7f36bd8bf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36b8102040 0x7f36b819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.544+0000 7f36bd8bf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b819dc20 0x7f36b81978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.544+0000 7f36bd8bf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f36b810c410 con 0x7f36b819dc20 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.544+0000 7f36bd8bf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f36b810c290 con 0x7f36b8102040 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.544+0000 7f36b6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36b8102040 0x7f36b819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.544+0000 7f36b6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36b8102040 0x7f36b819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33040/0 (socket says 172.21.15.163:33040) 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.544+0000 7f36b6ffd700 1 -- 172.21.15.163:0/1125623047 learned_addr learned my addr 172.21.15.163:0/1125623047 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.545+0000 7f36b6ffd700 1 -- 172.21.15.163:0/1125623047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36b8102040 msgr2=0x7f36b819d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.545+0000 7f36b6ffd700 1 -- 172.21.15.163:0/1125623047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36b8102040 msgr2=0x7f36b819d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.545+0000 7f36b6ffd700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36b8102040 0x7f36b819d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.545+0000 7f36b6ffd700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36b8102040 0x7f36b819d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.545+0000 7f36b67fc700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b819dc20 0x7f36b81978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.545+0000 7f36b67fc700 1 -- 172.21.15.163:0/1125623047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36b8102040 msgr2=0x7f36b819d6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:00:26.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.545+0000 7f36b67fc700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36b8102040 0x7f36b819d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:26.547 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.545+0000 7f36b67fc700 1 -- 172.21.15.163:0/1125623047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b819e260 con 0x7f36b819dc20 2024-03-19T01:00:26.547 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.546+0000 7f36b67fc700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b819dc20 0x7f36b81978a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00b4d0 tx=0x7f36ac00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:26.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.546+0000 7f36bc8bd700 1 -- 172.21.15.163:0/1125623047 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f36ac011630 con 0x7f36b819dc20 2024-03-19T01:00:26.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.546+0000 7f36bd8bf700 1 -- 172.21.15.163:0/1125623047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b8197de0 con 0x7f36b819dc20 2024-03-19T01:00:26.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.546+0000 7f36bd8bf700 1 -- 172.21.15.163:0/1125623047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b8198370 con 0x7f36b819dc20 2024-03-19T01:00:26.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.546+0000 7f36bc8bd700 1 -- 172.21.15.163:0/1125623047 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ac011cb0 con 0x7f36b819dc20 2024-03-19T01:00:26.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.546+0000 7f36bc8bd700 1 -- 172.21.15.163:0/1125623047 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f36ac0193b0 con 0x7f36b819dc20 2024-03-19T01:00:26.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.547+0000 7f36bd8bf700 1 -- 172.21.15.163:0/1125623047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b8103520 con 0x7f36b819dc20 2024-03-19T01:00:26.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.547+0000 7f36bc8bd700 1 -- 172.21.15.163:0/1125623047 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f36ac018020 con 0x7f36b819dc20 2024-03-19T01:00:26.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.548+0000 7f36bc8bd700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f36a0038d60 0x7f36a003b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:26.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.548+0000 7f36bc8bd700 1 -- 172.21.15.163:0/1125623047 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f36ac04bd30 con 0x7f36b819dc20 2024-03-19T01:00:26.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.548+0000 7f36b6ffd700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f36a0038d60 0x7f36a003b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:26.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.549+0000 7f36b6ffd700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f36a0038d60 0x7f36a003b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f36a800b5c0 tx=0x7f36a8009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:26.564 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.564+0000 7f36bc8bd700 1 -- 172.21.15.163:0/1125623047 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f36ac01f020 con 0x7f36b819dc20 2024-03-19T01:00:26.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.672+0000 7f36bc8bd700 1 -- 172.21.15.163:0/1125623047 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac01a590 con 0x7f36b819dc20 2024-03-19T01:00:26.839 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:26.838+0000 7f36bd8bf700 1 -- 172.21.15.163:0/1125623047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f36b8198dd0 con 0x7f36b819dc20 2024-03-19T01:00:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 podman[66810]: 2024-03-19 01:00:26.613846754 +0000 UTC m=+0.304033172 container init e818e2c8a5e84a182706c2dd741ed7f33f10fbc176223285e134ef1d75514c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-03-19T01:00:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 podman[66810]: 2024-03-19 01:00:26.616693424 +0000 UTC m=+0.306879840 container start e818e2c8a5e84a182706c2dd741ed7f33f10fbc176223285e134ef1d75514c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-b, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux ) 2024-03-19T01:00:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: set uid:gid to 167:167 (ceph:ceph) 2024-03-19T01:00:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev), process ceph-mon, pid 2 2024-03-19T01:00:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pidfile_write: ignore empty --pid-file 2024-03-19T01:00:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: load: jerasure load: lrc 2024-03-19T01:00:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: RocksDB version: 7.9.2 2024-03-19T01:00:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Git sha 0 2024-03-19T01:00:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Compile date 2024-03-18 18:57:12 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: DB SUMMARY 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: DB Session ID: UY5344EFWS7MOYE7DUY2 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: CURRENT file: CURRENT 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: IDENTITY file: IDENTITY 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.error_if_exists: 0 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.create_if_missing: 0 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.paranoid_checks: 1 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-19T01:00:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.env: 0x55f02a8bbc60 2024-03-19T01:00:27.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.fs: PosixFileSystem 2024-03-19T01:00:27.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.info_log: 0x55f02b5f9ae0 2024-03-19T01:00:27.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_file_opening_threads: 16 2024-03-19T01:00:27.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.statistics: (nil) 2024-03-19T01:00:27.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.use_fsync: 0 2024-03-19T01:00:27.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_log_file_size: 0 2024-03-19T01:00:27.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-19T01:00:27.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-19T01:00:27.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.keep_log_file_num: 1000 2024-03-19T01:00:27.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.recycle_log_file_num: 0 2024-03-19T01:00:27.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.allow_fallocate: 1 2024-03-19T01:00:27.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.allow_mmap_reads: 0 2024-03-19T01:00:27.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.allow_mmap_writes: 0 2024-03-19T01:00:27.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.use_direct_reads: 0 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.create_missing_column_families: 0 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.db_log_dir: 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.wal_dir: 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.advise_random_on_open: 1 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.db_write_buffer_size: 0 2024-03-19T01:00:27.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.write_buffer_manager: 0x55f02b5cc1e0 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.rate_limiter: (nil) 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.wal_recovery_mode: 2 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.enable_thread_tracking: 0 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.enable_pipelined_write: 0 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.unordered_write: 0 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-19T01:00:27.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.row_cache: None 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.wal_filter: None 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.allow_ingest_behind: 0 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.two_write_queues: 0 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.manual_wal_flush: 0 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.wal_compression: 0 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.atomic_flush: 0 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.log_readahead_size: 0 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.best_efforts_recovery: 0 2024-03-19T01:00:27.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.allow_data_in_errors: 0 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.db_host_id: __hostname__ 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_background_jobs: 2 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_background_compactions: -1 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_subcompactions: 1 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_total_wal_size: 0 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-19T01:00:27.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_open_files: -1 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bytes_per_sync: 0 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_readahead_size: 0 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_background_flushes: -1 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Compression algorithms supported: 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: kZSTD supported: 0 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: kXpressCompression supported: 0 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: kLZ4HCCompression supported: 1 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: kZlibCompression supported: 1 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: kSnappyCompression supported: 1 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: kLZ4Compression supported: 1 2024-03-19T01:00:27.015 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: kBZip2Compression supported: 0 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.merge_operator: 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_filter: None 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_filter_factory: None 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.sst_partitioner_factory: None 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f02b5f9be0) 2024-03-19T01:00:27.016 INFO:journalctl@ceph.mon.b.smithi086.stdout: cache_index_and_filter_blocks: 1 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: pin_top_level_index_and_filter: 1 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_type: 0 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: data_block_index_type: 0 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_shortening: 1 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: checksum: 4 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: no_block_cache: 0 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache: 0x55f02b5ff090 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_name: BinnedLRUCache 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_options: 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: capacity : 536870912 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: num_shard_bits : 4 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: strict_capacity_limit : 0 2024-03-19T01:00:27.017 INFO:journalctl@ceph.mon.b.smithi086.stdout: high_pri_pool_ratio: 0.000 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_compressed: (nil) 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: persistent_cache: (nil) 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_size: 4096 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_size_deviation: 10 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_restart_interval: 16 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_block_restart_interval: 1 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: metadata_block_size: 4096 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: partition_filters: 0 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: use_delta_encoding: 1 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: filter_policy: bloomfilter 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: whole_key_filtering: 1 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: verify_compression: 0 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: read_amp_bytes_per_bit: 0 2024-03-19T01:00:27.018 INFO:journalctl@ceph.mon.b.smithi086.stdout: format_version: 5 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout: enable_index_compression: 1 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_align: 0 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout: max_auto_readahead_size: 262144 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout: prepopulate_block_cache: 0 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout: initial_auto_readahead_size: 8192 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout: num_file_reads_for_auto_readahead: 2 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.write_buffer_size: 33554432 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_write_buffer_number: 2 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compression: NoCompression 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bottommost_compression: Disabled 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.prefix_extractor: nullptr 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.num_levels: 7 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-19T01:00:27.019 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compression_opts.level: 32767 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compression_opts.strategy: 0 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-19T01:00:27.020 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compression_opts.enabled: false 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.target_file_size_base: 67108864 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-19T01:00:27.021 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.arena_block_size: 1048576 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.disable_auto_compactions: 0 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-19T01:00:27.022 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.table_properties_collectors: 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.inplace_update_support: 0 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-19T01:00:27.023 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.bloom_locality: 0 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.max_successive_merges: 0 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.paranoid_file_checks: 0 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.force_consistency_checks: 1 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.report_bg_io_stats: 0 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.ttl: 2592000 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.enable_blob_files: false 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.min_blob_size: 0 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.blob_file_size: 268435456 2024-03-19T01:00:27.024 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.blob_file_starting_level: 0 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f8192563-0cc7-44f8-9719-367758280b43 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710810026645793, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710810026646251, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710810026, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f8192563-0cc7-44f8-9719-367758280b43", "db_session_id": "UY5344EFWS7MOYE7DUY2", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710810026646330, "job": 1, "event": "recovery_finished"} 2024-03-19T01:00:27.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f02b6f2000 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: DB pointer 0x55f02b6dc000 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** DB Stats ** 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T01:00:27.026 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** Compaction Stats [default] ** 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** Compaction Stats [default] ** 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-19T01:00:27.027 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: Block cache BinnedLRUCache@0x55f02b5ff090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-19T01:00:27.028 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: using public_addr v2:172.21.15.86:0/0 -> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: starting mon.b rank -1 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(???) e0 preinit fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).mds e1 new map 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout: e1 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout: legacy client fscid: -1 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout: No filesystems configured 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-19T01:00:27.029 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Added host smithi086 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.031 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Deploying cephadm binary to smithi163 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Added host smithi163 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:00:27.032 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Updating smithi163:/etc/ceph/ceph.conf 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3738398575' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.033 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3738398575' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi086:172.21.15.86=b;smithi163:172.21.15.163=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi086:172.21.15.86=b;smithi163:172.21.15.163=c;count:3 2024-03-19T01:00:27.034 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Deploying daemon mon.c on smithi163 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='client.? 172.21.15.163:0/16421857' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.a calling monitor election 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.c calling monitor election 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:27.035 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: monmap epoch 2 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: last_changed 2024-03-19T01:00:17.593381+0000 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: created 2024-03-19T00:58:05.726813+0000 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: min_mon_release 19 (squid) 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: election_strategy: 1 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: fsmap 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mgrmap e13: x(active, since 59s) 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: overall HEALTH_OK 2024-03-19T01:00:27.036 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.037 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.037 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:27.037 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:27.037 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:27.037 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: Deploying daemon mon.b on smithi086 2024-03-19T01:00:27.037 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='client.? 172.21.15.163:0/964226145' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:00:27.037 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:27.037 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:27.037 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 ceph-mon[66829]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-19T01:00:27.037 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 bash[66810]: e818e2c8a5e84a182706c2dd741ed7f33f10fbc176223285e134ef1d75514c31 2024-03-19T01:00:27.037 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:26 smithi086 systemd[1]: Started Ceph mon.b for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:00:31.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.690+0000 7f36bc8bd700 1 -- 172.21.15.163:0/1125623047 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f36ac010380 con 0x7f36b819dc20 2024-03-19T01:00:31.691 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-19T01:00:31.692 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":3,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","modified":"2024-03-19T01:00:26.670277Z","created":"2024-03-19T00:58:05.726813Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:3300","nonce":0},{"type":"v1","addr":"172.21.15.163:6789","nonce":0}]},"addr":"172.21.15.163:6789/0","public_addr":"172.21.15.163:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-19T01:00:31.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.693+0000 7f36bd8bf700 1 -- 172.21.15.163:0/1125623047 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f36a0038d60 msgr2=0x7f36a003b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:31.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.693+0000 7f36bd8bf700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f36a0038d60 0x7f36a003b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f36a800b5c0 tx=0x7f36a8009650 comp rx=0 tx=0).stop 2024-03-19T01:00:31.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.693+0000 7f36bd8bf700 1 -- 172.21.15.163:0/1125623047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b819dc20 msgr2=0x7f36b81978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:31.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.693+0000 7f36bd8bf700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b819dc20 0x7f36b81978a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00b4d0 tx=0x7f36ac00b890 comp rx=0 tx=0).stop 2024-03-19T01:00:31.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.693+0000 7f36bd8bf700 1 -- 172.21.15.163:0/1125623047 shutdown_connections 2024-03-19T01:00:31.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.693+0000 7f36bd8bf700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f36a0038d60 0x7f36a003b220 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:31.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.693+0000 7f36bd8bf700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36b8102040 0x7f36b819d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:31.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.693+0000 7f36bd8bf700 1 --2- 172.21.15.163:0/1125623047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36b819dc20 0x7f36b81978a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:31.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.693+0000 7f36bd8bf700 1 -- 172.21.15.163:0/1125623047 >> 172.21.15.163:0/1125623047 conn(0x7f36b80fd8b0 msgr2=0x7f36b8103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:31.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.693+0000 7f36bd8bf700 1 -- 172.21.15.163:0/1125623047 shutdown_connections 2024-03-19T01:00:31.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:31.693+0000 7f36bd8bf700 1 -- 172.21.15.163:0/1125623047 wait complete. 2024-03-19T01:00:31.695 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 3 2024-03-19T01:00:32.003 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: mon.c calling monitor election 2024-03-19T01:00:32.003 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:00:32.003 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: mon.a calling monitor election 2024-03-19T01:00:32.003 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: from='client.? 172.21.15.163:0/1125623047' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: monmap epoch 3 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: last_changed 2024-03-19T01:00:26.670277+0000 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: created 2024-03-19T00:58:05.726813+0000 2024-03-19T01:00:32.004 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: min_mon_release 19 (squid) 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: election_strategy: 1 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: fsmap 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: mgrmap e13: x(active, since 68s) 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: mon.b (rank 2) addr [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] is down (out of quorum) 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:32.005 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:31 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:32.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: mon.c calling monitor election 2024-03-19T01:00:32.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:00:32.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: mon.a calling monitor election 2024-03-19T01:00:32.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: from='client.? 172.21.15.163:0/1125623047' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: monmap epoch 3 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:32.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: last_changed 2024-03-19T01:00:26.670277+0000 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: created 2024-03-19T00:58:05.726813+0000 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: min_mon_release 19 (squid) 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: election_strategy: 1 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: fsmap 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: mgrmap e13: x(active, since 68s) 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: mon.b (rank 2) addr [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] is down (out of quorum) 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:32.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:31 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:32.306 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-19T01:00:32.306 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-19T01:00:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:00:33.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:32 smithi069 ceph-mon[31727]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:33.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:33.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:33.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:33.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:32 smithi163 ceph-mon[67428]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: mon.c calling monitor election 2024-03-19T01:00:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:00:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: mon.a calling monitor election 2024-03-19T01:00:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:00:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='client.? 172.21.15.163:0/1125623047' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:00:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: monmap epoch 3 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: last_changed 2024-03-19T01:00:26.670277+0000 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: created 2024-03-19T00:58:05.726813+0000 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: min_mon_release 19 (squid) 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: election_strategy: 1 2024-03-19T01:00:34.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: fsmap 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: mgrmap e13: x(active, since 68s) 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: mon.b (rank 2) addr [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] is down (out of quorum) 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:34.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:34.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.716+0000 7ff3bd800700 1 -- 172.21.15.69:0/3110801829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b810e400 msgr2=0x7ff3b810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.716+0000 7ff3bd800700 1 --2- 172.21.15.69:0/3110801829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b810e400 0x7ff3b810e7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009910 tx=0x7ff3a8009c20 comp rx=0 tx=0).stop 2024-03-19T01:00:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.716+0000 7ff3bd800700 1 -- 172.21.15.69:0/3110801829 shutdown_connections 2024-03-19T01:00:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.716+0000 7ff3bd800700 1 --2- 172.21.15.69:0/3110801829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b810e400 0x7ff3b810e7e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.716+0000 7ff3bd800700 1 -- 172.21.15.69:0/3110801829 >> 172.21.15.69:0/3110801829 conn(0x7ff3b8077b70 msgr2=0x7ff3b8079f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.716+0000 7ff3bd800700 1 -- 172.21.15.69:0/3110801829 shutdown_connections 2024-03-19T01:00:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.716+0000 7ff3bd800700 1 -- 172.21.15.69:0/3110801829 wait complete. 2024-03-19T01:00:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.717+0000 7ff3bd800700 1 Processor -- start 2024-03-19T01:00:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.718+0000 7ff3bd800700 1 -- start start 2024-03-19T01:00:34.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.718+0000 7ff3bd800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b810e400 0x7ff3b819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:34.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.718+0000 7ff3bd800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b819e510 0x7ff3b81a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.718+0000 7ff3bd800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3b81a2de0 0x7ff3b81a3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.718+0000 7ff3bd800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b8112900 con 0x7ff3b819e510 2024-03-19T01:00:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.718+0000 7ff3bd800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b8112780 con 0x7ff3b81a2de0 2024-03-19T01:00:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.718+0000 7ff3bd800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b8112a80 con 0x7ff3b810e400 2024-03-19T01:00:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.719+0000 7ff3b6a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b810e400 0x7ff3b819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.719+0000 7ff3b6000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b819e510 0x7ff3b81a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.719+0000 7ff3b6a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b810e400 0x7ff3b819dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55102/0 (socket says 172.21.15.69:55102) 2024-03-19T01:00:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.719+0000 7ff3b6a00700 1 -- 172.21.15.69:0/3764134530 learned_addr learned my addr 172.21.15.69:0/3764134530 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:00:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.719+0000 7ff3b7400700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3b81a2de0 0x7ff3b81a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.719+0000 7ff3b6a00700 1 -- 172.21.15.69:0/3764134530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3b81a2de0 msgr2=0x7ff3b81a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.719+0000 7ff3b6a00700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3b81a2de0 0x7ff3b81a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.719+0000 7ff3b6a00700 1 -- 172.21.15.69:0/3764134530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b819e510 msgr2=0x7ff3b81a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.719+0000 7ff3b6a00700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b819e510 0x7ff3b81a28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.719+0000 7ff3b6a00700 1 -- 172.21.15.69:0/3764134530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b81aab50 con 0x7ff3b810e400 2024-03-19T01:00:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.719+0000 7ff3b6a00700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b810e400 0x7ff3b819dfd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009fd0 tx=0x7ff3a800bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:34.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.720+0000 7ff3af400700 1 -- 172.21.15.69:0/3764134530 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8027070 con 0x7ff3b810e400 2024-03-19T01:00:34.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.720+0000 7ff3af400700 1 -- 172.21.15.69:0/3764134530 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a80050d0 con 0x7ff3b810e400 2024-03-19T01:00:34.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.720+0000 7ff3bd800700 1 -- 172.21.15.69:0/3764134530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b81aad80 con 0x7ff3b810e400 2024-03-19T01:00:34.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.720+0000 7ff3af400700 1 -- 172.21.15.69:0/3764134530 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8022e20 con 0x7ff3b810e400 2024-03-19T01:00:34.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.720+0000 7ff3bd800700 1 -- 172.21.15.69:0/3764134530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b81ab2b0 con 0x7ff3b810e400 2024-03-19T01:00:34.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.721+0000 7ff3bd800700 1 -- 172.21.15.69:0/3764134530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b8063940 con 0x7ff3b810e400 2024-03-19T01:00:34.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.721+0000 7ff3af400700 1 -- 172.21.15.69:0/3764134530 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7ff3a802a070 con 0x7ff3b810e400 2024-03-19T01:00:34.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.721+0000 7ff3af400700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff39c039450 0x7ff39c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:34.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.721+0000 7ff3af400700 1 -- 172.21.15.69:0/3764134530 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff3a804cef0 con 0x7ff3b810e400 2024-03-19T01:00:34.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.722+0000 7ff3b6000700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff39c039450 0x7ff39c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:34.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.722+0000 7ff3b6000700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff39c039450 0x7ff39c03b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009850 tx=0x7ff3a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:34.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.727+0000 7ff3af400700 1 -- 172.21.15.69:0/3764134530 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff3a8013070 con 0x7ff3b810e400 2024-03-19T01:00:34.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.954+0000 7ff3bd800700 1 -- 172.21.15.69:0/3764134530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff3b81ab600 con 0x7ff3b810e400 2024-03-19T01:00:34.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.954+0000 7ff3af400700 1 -- 172.21.15.69:0/3764134530 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7ff3a800e3e0 con 0x7ff3b810e400 2024-03-19T01:00:34.956 INFO:teuthology.orchestra.run.smithi069.stdout:# minimal ceph.conf for 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:34.956 INFO:teuthology.orchestra.run.smithi069.stdout:[global] 2024-03-19T01:00:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: fsid = 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: mon_host = [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: mon.b calling monitor election 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: Updating smithi163:/etc/ceph/ceph.conf 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: mon.b calling monitor election 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: mon.c calling monitor election 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: mon.a calling monitor election 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: monmap epoch 3 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: last_changed 2024-03-19T01:00:26.670277+0000 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: created 2024-03-19T00:58:05.726813+0000 2024-03-19T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: min_mon_release 19 (squid) 2024-03-19T01:00:34.958 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: election_strategy: 1 2024-03-19T01:00:34.958 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-03-19T01:00:34.958 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-03-19T01:00:34.958 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-03-19T01:00:34.958 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: fsmap 2024-03-19T01:00:34.958 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:34.958 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: mgrmap e13: x(active, since 70s) 2024-03-19T01:00:34.958 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-19T01:00:34.958 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: Cluster is now healthy 2024-03-19T01:00:34.958 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: overall HEALTH_OK 2024-03-19T01:00:34.959 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:34.959 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:34.959 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:34.959 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:34.959 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:34.959 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:34 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:34.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 -- 172.21.15.69:0/3764134530 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff39c039450 msgr2=0x7ff39c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:34.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff39c039450 0x7ff39c03b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009850 tx=0x7ff3a4008040 comp rx=0 tx=0).stop 2024-03-19T01:00:34.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 -- 172.21.15.69:0/3764134530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b810e400 msgr2=0x7ff3b819dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:34.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b810e400 0x7ff3b819dfd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009fd0 tx=0x7ff3a800bde0 comp rx=0 tx=0).stop 2024-03-19T01:00:34.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 -- 172.21.15.69:0/3764134530 shutdown_connections 2024-03-19T01:00:34.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff39c039450 0x7ff39c03b910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:34.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3b81a2de0 0x7ff3b81a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:34.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3b810e400 0x7ff3b819dfd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:34.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 --2- 172.21.15.69:0/3764134530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3b819e510 0x7ff3b81a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:34.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 -- 172.21.15.69:0/3764134530 >> 172.21.15.69:0/3764134530 conn(0x7ff3b8077b70 msgr2=0x7ff3b8077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:34.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 -- 172.21.15.69:0/3764134530 shutdown_connections 2024-03-19T01:00:34.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:34.958+0000 7ff3bd800700 1 -- 172.21.15.69:0/3764134530 wait complete. 2024-03-19T01:00:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: mon.b calling monitor election 2024-03-19T01:00:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-19T01:00:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-19T01:00:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: Updating smithi163:/etc/ceph/ceph.conf 2024-03-19T01:00:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: mon.b calling monitor election 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: mon.c calling monitor election 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: mon.a calling monitor election 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: monmap epoch 3 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: last_changed 2024-03-19T01:00:26.670277+0000 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: created 2024-03-19T00:58:05.726813+0000 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: min_mon_release 19 (squid) 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: election_strategy: 1 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: fsmap 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: mgrmap e13: x(active, since 70s) 2024-03-19T01:00:35.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-19T01:00:35.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: Cluster is now healthy 2024-03-19T01:00:35.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: overall HEALTH_OK 2024-03-19T01:00:35.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:35.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:35.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:34 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: mon.b calling monitor election 2024-03-19T01:00:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-19T01:00:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-19T01:00:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: Updating smithi163:/etc/ceph/ceph.conf 2024-03-19T01:00:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: mon.b calling monitor election 2024-03-19T01:00:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: mon.c calling monitor election 2024-03-19T01:00:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: mon.a calling monitor election 2024-03-19T01:00:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-19T01:00:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: monmap epoch 3 2024-03-19T01:00:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:00:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: last_changed 2024-03-19T01:00:26.670277+0000 2024-03-19T01:00:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: created 2024-03-19T00:58:05.726813+0000 2024-03-19T01:00:35.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: min_mon_release 19 (squid) 2024-03-19T01:00:35.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: election_strategy: 1 2024-03-19T01:00:35.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-03-19T01:00:35.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-03-19T01:00:35.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-03-19T01:00:35.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: fsmap 2024-03-19T01:00:35.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:00:35.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: mgrmap e13: x(active, since 70s) 2024-03-19T01:00:35.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-19T01:00:35.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: Cluster is now healthy 2024-03-19T01:00:35.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: overall HEALTH_OK 2024-03-19T01:00:35.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:35.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:35.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:34 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: Updating smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:35.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:35.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:35.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:00:35.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:35.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:35 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3764134530' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:35.822 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-19T01:00:35.822 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T01:00:35.822 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.conf 2024-03-19T01:00:35.898 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T01:00:35.898 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: Updating smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:35.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:35 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3764134530' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:36.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: Updating smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:35 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3764134530' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:36.016 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-19T01:00:36.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-03-19T01:00:36.050 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-19T01:00:36.051 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:00:36.121 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-19T01:00:36.121 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.conf 2024-03-19T01:00:36.155 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-19T01:00:36.155 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:00:36.224 INFO:tasks.cephadm:Adding mgr.x on smithi069 2024-03-19T01:00:36.225 INFO:tasks.cephadm:Adding mgr.y on smithi086 2024-03-19T01:00:36.225 INFO:tasks.cephadm:Adding mgr.z on smithi163 2024-03-19T01:00:36.225 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch apply mgr '3;smithi069=x;smithi086=y;smithi163=z' 2024-03-19T01:00:36.509 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:00:36.885 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:36 smithi163 ceph-mon[67428]: Reconfiguring mon.a (unknown last config time)... 2024-03-19T01:00:36.885 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:36 smithi163 ceph-mon[67428]: Reconfiguring daemon mon.a on smithi069 2024-03-19T01:00:36.885 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:36 smithi163 ceph-mon[67428]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:36 smithi086 ceph-mon[66829]: Reconfiguring mon.a (unknown last config time)... 2024-03-19T01:00:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:36 smithi086 ceph-mon[66829]: Reconfiguring daemon mon.a on smithi069 2024-03-19T01:00:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:36 smithi086 ceph-mon[66829]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:37.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:36 smithi069 ceph-mon[31727]: Reconfiguring mon.a (unknown last config time)... 2024-03-19T01:00:37.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:36 smithi069 ceph-mon[31727]: Reconfiguring daemon mon.a on smithi069 2024-03-19T01:00:37.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:36 smithi069 ceph-mon[31727]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:38.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.026+0000 7fde023b4700 1 -- 172.21.15.163:0/3736074079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddfc1034c0 msgr2=0x7fddfc1038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:38.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.026+0000 7fde023b4700 1 --2- 172.21.15.163:0/3736074079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddfc1034c0 0x7fddfc1038e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fddec009910 tx=0x7fddec009c20 comp rx=0 tx=0).stop 2024-03-19T01:00:38.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.026+0000 7fde023b4700 1 -- 172.21.15.163:0/3736074079 shutdown_connections 2024-03-19T01:00:38.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.026+0000 7fde023b4700 1 --2- 172.21.15.163:0/3736074079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddfc1034c0 0x7fddfc1038e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:38.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.026+0000 7fde023b4700 1 -- 172.21.15.163:0/3736074079 >> 172.21.15.163:0/3736074079 conn(0x7fddfc0fd400 msgr2=0x7fddfc0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.026+0000 7fde023b4700 1 -- 172.21.15.163:0/3736074079 shutdown_connections 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.026+0000 7fde023b4700 1 -- 172.21.15.163:0/3736074079 wait complete. 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.027+0000 7fde023b4700 1 Processor -- start 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.027+0000 7fde023b4700 1 -- start start 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.027+0000 7fde023b4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddfc1034c0 0x7fddfc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.027+0000 7fde023b4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddfc197d40 0x7fddfc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.027+0000 7fde023b4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc1981c0 0x7fddfc19f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.027+0000 7fde023b4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc107f30 con 0x7fddfc1034c0 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.027+0000 7fde023b4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc107db0 con 0x7fddfc1981c0 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.027+0000 7fde023b4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1080b0 con 0x7fddfc197d40 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddfb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddfc197d40 0x7fddfc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:38.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddfb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddfc197d40 0x7fddfc19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39328/0 (socket says 172.21.15.163:39328) 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddfb7fe700 1 -- 172.21.15.163:0/4167172821 learned_addr learned my addr 172.21.15.163:0/4167172821 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fde00951700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc1981c0 0x7fddfc19f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddfbfff700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddfc1034c0 0x7fddfc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddfb7fe700 1 -- 172.21.15.163:0/4167172821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc1981c0 msgr2=0x7fddfc19f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddfb7fe700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc1981c0 0x7fddfc19f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddfb7fe700 1 -- 172.21.15.163:0/4167172821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddfc1034c0 msgr2=0x7fddfc197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddfb7fe700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddfc1034c0 0x7fddfc197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddfb7fe700 1 -- 172.21.15.163:0/4167172821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc19fc10 con 0x7fddfc197d40 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddfbfff700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddfc1034c0 0x7fddfc197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddfb7fe700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddfc197d40 0x7fddfc19cdb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fddf000b780 tx=0x7fddf000bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddf97fa700 1 -- 172.21.15.163:0/4167172821 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0004230 con 0x7fddfc197d40 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fde023b4700 1 -- 172.21.15.163:0/4167172821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc19ff00 con 0x7fddfc197d40 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddf97fa700 1 -- 172.21.15.163:0/4167172821 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddf00043d0 con 0x7fddfc197d40 2024-03-19T01:00:38.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fddf97fa700 1 -- 172.21.15.163:0/4167172821 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0011c60 con 0x7fddfc197d40 2024-03-19T01:00:38.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.028+0000 7fde023b4700 1 -- 172.21.15.163:0/4167172821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc1a0400 con 0x7fddfc197d40 2024-03-19T01:00:38.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.029+0000 7fddf97fa700 1 -- 172.21.15.163:0/4167172821 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fddf001a400 con 0x7fddfc197d40 2024-03-19T01:00:38.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.029+0000 7fde023b4700 1 -- 172.21.15.163:0/4167172821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddfc063940 con 0x7fddfc197d40 2024-03-19T01:00:38.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.029+0000 7fddf97fa700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdde4039450 0x7fdde403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:38.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.029+0000 7fddf97fa700 1 -- 172.21.15.163:0/4167172821 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fddf004bc70 con 0x7fddfc197d40 2024-03-19T01:00:38.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.030+0000 7fddfbfff700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdde4039450 0x7fdde403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:38.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.031+0000 7fddfbfff700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdde4039450 0x7fdde403b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fddec005330 tx=0x7fddec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:38.035 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.034+0000 7fddf97fa700 1 -- 172.21.15.163:0/4167172821 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fddf001a840 con 0x7fddfc197d40 2024-03-19T01:00:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.239+0000 7fde023b4700 1 -- 172.21.15.163:0/4167172821 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi086=y;smithi163=z", "target": ["mon-mgr", ""]}) -- 0x7fddfc0ff340 con 0x7fdde4039450 2024-03-19T01:00:38.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.252+0000 7fddf97fa700 1 -- 172.21.15.163:0/4167172821 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fddfc0ff340 con 0x7fdde4039450 2024-03-19T01:00:38.254 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled mgr update... 2024-03-19T01:00:38.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 -- 172.21.15.163:0/4167172821 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdde4039450 msgr2=0x7fdde403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:38.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdde4039450 0x7fdde403b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fddec005330 tx=0x7fddec005fb0 comp rx=0 tx=0).stop 2024-03-19T01:00:38.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 -- 172.21.15.163:0/4167172821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddfc197d40 msgr2=0x7fddfc19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:38.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddfc197d40 0x7fddfc19cdb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fddf000b780 tx=0x7fddf000bb40 comp rx=0 tx=0).stop 2024-03-19T01:00:38.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 -- 172.21.15.163:0/4167172821 shutdown_connections 2024-03-19T01:00:38.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdde4039450 0x7fdde403b910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:38.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddfc1981c0 0x7fddfc19f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:38.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fddfc1034c0 0x7fddfc197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:38.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 --2- 172.21.15.163:0/4167172821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddfc197d40 0x7fddfc19cdb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:38.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 -- 172.21.15.163:0/4167172821 >> 172.21.15.163:0/4167172821 conn(0x7fddfc0fd400 msgr2=0x7fddfc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:38.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 -- 172.21.15.163:0/4167172821 shutdown_connections 2024-03-19T01:00:38.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:00:38.255+0000 7fde023b4700 1 -- 172.21.15.163:0/4167172821 wait complete. 2024-03-19T01:00:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:38 smithi086 ceph-mon[66829]: Reconfiguring mon.b (monmap changed)... 2024-03-19T01:00:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:00:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:38 smithi086 ceph-mon[66829]: Reconfiguring daemon mon.b on smithi086 2024-03-19T01:00:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:38 smithi086 ceph-mon[66829]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:38 smithi069 ceph-mon[31727]: Reconfiguring mon.b (monmap changed)... 2024-03-19T01:00:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:00:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:38 smithi069 ceph-mon[31727]: Reconfiguring daemon mon.b on smithi086 2024-03-19T01:00:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:38 smithi069 ceph-mon[31727]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:38 smithi163 ceph-mon[67428]: Reconfiguring mon.b (monmap changed)... 2024-03-19T01:00:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:00:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:38 smithi163 ceph-mon[67428]: Reconfiguring daemon mon.b on smithi086 2024-03-19T01:00:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:38 smithi163 ceph-mon[67428]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:38.836 DEBUG:teuthology.orchestra.run.smithi086:mgr.y> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.y.service 2024-03-19T01:00:38.839 DEBUG:teuthology.orchestra.run.smithi163:mgr.z> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.z.service 2024-03-19T01:00:38.841 INFO:tasks.cephadm:Deploying OSDs... 2024-03-19T01:00:38.841 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T01:00:38.841 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2024-03-19T01:00:38.900 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-19T01:00:38.900 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2024-03-19T01:00:38.953 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-19T01:00:38.953 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:38.953 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-19T01:00:38.953 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:38.953 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:38.953 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-03-19 00:59:36.684857085 +0000 2024-03-19T01:00:38.954 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-03-19 00:29:51.071281626 +0000 2024-03-19T01:00:38.954 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-03-19 00:29:51.071281626 +0000 2024-03-19T01:00:38.954 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-03-19 00:29:51.071281626 +0000 2024-03-19T01:00:38.954 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-19T01:00:39.042 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-03-19T01:00:39.042 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-03-19T01:00:39.042 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000556923 s, 919 kB/s 2024-03-19T01:00:39.045 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-19T01:00:39.097 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2024-03-19T01:00:39.143 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-19T01:00:39.143 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:39.143 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-19T01:00:39.143 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:39.143 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:39.143 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-03-19 00:59:36.781854837 +0000 2024-03-19T01:00:39.143 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-03-19 00:29:51.642268861 +0000 2024-03-19T01:00:39.143 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-03-19 00:29:51.642268861 +0000 2024-03-19T01:00:39.144 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-03-19 00:29:51.642268861 +0000 2024-03-19T01:00:39.144 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-19T01:00:39.279 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-03-19T01:00:39.279 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-03-19T01:00:39.279 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000354142 s, 1.4 MB/s 2024-03-19T01:00:39.283 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-19T01:00:39.343 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2024-03-19T01:00:39.393 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-19T01:00:39.394 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:39.394 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-19T01:00:39.394 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:39.394 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:39.394 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-03-19 00:59:36.837853539 +0000 2024-03-19T01:00:39.395 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-03-19 00:29:52.217256007 +0000 2024-03-19T01:00:39.395 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-03-19 00:29:52.217256007 +0000 2024-03-19T01:00:39.395 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-03-19 00:29:52.217256007 +0000 2024-03-19T01:00:39.395 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-19T01:00:39.481 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-03-19T01:00:39.482 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-03-19T01:00:39.482 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000408406 s, 1.3 MB/s 2024-03-19T01:00:39.485 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-19T01:00:39.537 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2024-03-19T01:00:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:39 smithi163 ceph-mon[67428]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi086=y;smithi163=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:00:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:39 smithi163 ceph-mon[67428]: Saving service mgr spec with placement smithi069=x;smithi086=y;smithi163=z;count:3 2024-03-19T01:00:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:39 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:39 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:39 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:39 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:00:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:39 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:39.585 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-19T01:00:39.585 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:39.586 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-19T01:00:39.586 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:39.586 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:39.586 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-03-19 00:59:36.891852288 +0000 2024-03-19T01:00:39.586 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-03-19 00:29:52.778243466 +0000 2024-03-19T01:00:39.586 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-03-19 00:29:52.778243466 +0000 2024-03-19T01:00:39.586 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-03-19 00:29:52.778243466 +0000 2024-03-19T01:00:39.586 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-19T01:00:39.657 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:39 smithi069 ceph-mon[31727]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi086=y;smithi163=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:00:39.657 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:39 smithi069 ceph-mon[31727]: Saving service mgr spec with placement smithi069=x;smithi086=y;smithi163=z;count:3 2024-03-19T01:00:39.657 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:39.657 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:39.658 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:39.658 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:00:39.658 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:39 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-03-19T01:00:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-03-19T01:00:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000351274 s, 1.5 MB/s 2024-03-19T01:00:39.669 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-19T01:00:39.725 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-19T01:00:39.725 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-03-19T01:00:39.744 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-19T01:00:39.744 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-03-19T01:00:39.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:39 smithi086 ceph-mon[66829]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi086=y;smithi163=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:00:39.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:39 smithi086 ceph-mon[66829]: Saving service mgr spec with placement smithi069=x;smithi086=y;smithi163=z;count:3 2024-03-19T01:00:39.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:39 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:39.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:39 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:39.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:39 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:00:39.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:39 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:00:39.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:39 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-19T01:00:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:39.773 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-19T01:00:39.773 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:39.773 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:39.773 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-19 01:00:26.139627235 +0000 2024-03-19T01:00:39.773 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-19 00:37:11.440626176 +0000 2024-03-19T01:00:39.773 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-19 00:37:11.440626176 +0000 2024-03-19T01:00:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-19 00:37:11.440626176 +0000 2024-03-19T01:00:39.773 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-19T01:00:39.842 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-19T01:00:39.842 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-19T01:00:39.842 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000226667 s, 2.3 MB/s 2024-03-19T01:00:39.843 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-19T01:00:39.901 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-03-19T01:00:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-19T01:00:39.957 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:39.957 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-03-19T01:00:39.957 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:39.957 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:39.957 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-19 01:00:26.140627213 +0000 2024-03-19T01:00:39.957 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-19 00:37:11.794618403 +0000 2024-03-19T01:00:39.957 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-19 00:37:11.794618403 +0000 2024-03-19T01:00:39.958 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-19 00:37:11.794618403 +0000 2024-03-19T01:00:39.958 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-19T01:00:40.024 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-19T01:00:40.024 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-19T01:00:40.024 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000228585 s, 2.2 MB/s 2024-03-19T01:00:40.025 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-19T01:00:40.082 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-03-19T01:00:40.138 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-19T01:00:40.138 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:40.138 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-19T01:00:40.138 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:40.138 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:40.138 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-19 01:00:26.140627213 +0000 2024-03-19T01:00:40.138 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-19 00:37:12.125611136 +0000 2024-03-19T01:00:40.138 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-19 00:37:12.125611136 +0000 2024-03-19T01:00:40.138 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-19 00:37:12.125611136 +0000 2024-03-19T01:00:40.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-19T01:00:40.204 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-19T01:00:40.204 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-19T01:00:40.204 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000269556 s, 1.9 MB/s 2024-03-19T01:00:40.205 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-19T01:00:40.262 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-03-19T01:00:40.318 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-19T01:00:40.318 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:40.318 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-19T01:00:40.318 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:40.318 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:40.318 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-19 01:00:26.140627213 +0000 2024-03-19T01:00:40.318 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-19 00:37:12.459603802 +0000 2024-03-19T01:00:40.318 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-19 00:37:12.459603802 +0000 2024-03-19T01:00:40.318 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-19 00:37:12.459603802 +0000 2024-03-19T01:00:40.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-19T01:00:40.383 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-19T01:00:40.383 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-19T01:00:40.383 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000184651 s, 2.8 MB/s 2024-03-19T01:00:40.384 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-19T01:00:40.440 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-19T01:00:40.440 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2024-03-19T01:00:40.454 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-19T01:00:40.455 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2024-03-19T01:00:40.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:40 smithi086 ceph-mon[66829]: Reconfiguring mon.c (monmap changed)... 2024-03-19T01:00:40.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:40 smithi086 ceph-mon[66829]: Reconfiguring daemon mon.c on smithi163 2024-03-19T01:00:40.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:40 smithi086 ceph-mon[66829]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:40.509 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-19T01:00:40.509 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:40.509 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-03-19T01:00:40.509 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:40.509 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:40.509 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-19 01:00:17.035234489 +0000 2024-03-19T01:00:40.509 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-19 00:39:26.324012312 +0000 2024-03-19T01:00:40.509 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-19 00:39:26.324012312 +0000 2024-03-19T01:00:40.509 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-03-19 00:39:26.324012312 +0000 2024-03-19T01:00:40.510 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-19T01:00:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:40 smithi163 ceph-mon[67428]: Reconfiguring mon.c (monmap changed)... 2024-03-19T01:00:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:40 smithi163 ceph-mon[67428]: Reconfiguring daemon mon.c on smithi163 2024-03-19T01:00:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:40 smithi163 ceph-mon[67428]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:40.576 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-19T01:00:40.576 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-19T01:00:40.576 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000250637 s, 2.0 MB/s 2024-03-19T01:00:40.577 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-19T01:00:40.633 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2024-03-19T01:00:40.687 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-19T01:00:40.687 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:40.687 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-19T01:00:40.687 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:40.688 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:40.688 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-19 01:00:17.036234465 +0000 2024-03-19T01:00:40.688 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-19 00:39:26.675003832 +0000 2024-03-19T01:00:40.688 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-19 00:39:26.675003832 +0000 2024-03-19T01:00:40.688 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-03-19 00:39:26.675003832 +0000 2024-03-19T01:00:40.688 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-19T01:00:40.753 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-19T01:00:40.753 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-19T01:00:40.753 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000187092 s, 2.7 MB/s 2024-03-19T01:00:40.754 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-19T01:00:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:40 smithi069 ceph-mon[31727]: Reconfiguring mon.c (monmap changed)... 2024-03-19T01:00:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:40 smithi069 ceph-mon[31727]: Reconfiguring daemon mon.c on smithi163 2024-03-19T01:00:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:40 smithi069 ceph-mon[31727]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:40.809 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2024-03-19T01:00:40.864 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-19T01:00:40.864 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:40.864 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-19T01:00:40.864 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:40.865 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:40.865 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-19 01:00:17.036234465 +0000 2024-03-19T01:00:40.865 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-19 00:39:27.011995692 +0000 2024-03-19T01:00:40.865 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-19 00:39:27.011995692 +0000 2024-03-19T01:00:40.865 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-03-19 00:39:27.011995692 +0000 2024-03-19T01:00:40.865 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-19T01:00:40.935 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-19T01:00:40.935 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-19T01:00:40.935 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000198832 s, 2.6 MB/s 2024-03-19T01:00:40.936 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-19T01:00:40.992 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2024-03-19T01:00:41.047 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-19T01:00:41.047 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:00:41.047 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-03-19T01:00:41.048 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:00:41.048 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:00:41.048 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-19 01:00:17.036234465 +0000 2024-03-19T01:00:41.048 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-19 00:39:27.353987430 +0000 2024-03-19T01:00:41.048 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-19 00:39:27.353987430 +0000 2024-03-19T01:00:41.048 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-03-19 00:39:27.353987430 +0000 2024-03-19T01:00:41.048 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-19T01:00:41.113 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-19T01:00:41.113 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-19T01:00:41.113 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000247619 s, 2.1 MB/s 2024-03-19T01:00:41.114 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-19T01:00:41.170 INFO:tasks.cephadm:Deploying osd.0 on smithi069 with /dev/vg_nvme/lv_4... 2024-03-19T01:00:41.170 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-19T01:00:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:00:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:00:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:00:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:41 smithi069 ceph-mon[31727]: Deploying daemon mgr.z on smithi163 2024-03-19T01:00:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:00:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:00:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:00:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:41 smithi163 ceph-mon[67428]: Deploying daemon mgr.z on smithi163 2024-03-19T01:00:41.614 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:00:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:00:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:00:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:00:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:41 smithi086 ceph-mon[66829]: Deploying daemon mgr.z on smithi163 2024-03-19T01:00:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:42 smithi163 ceph-mon[67428]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:42.562 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:42 smithi163 systemd[1]: Starting Ceph mgr.z for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:00:42.570 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:42 smithi069 ceph-mon[31727]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:42.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:42 smithi086 ceph-mon[66829]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:42.920 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:42 smithi163 podman[69277]: 2024-03-19 01:00:42.540874565 +0000 UTC m=+0.017783612 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:00:42.920 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:42 smithi163 podman[69277]: 2024-03-19 01:00:42.658799582 +0000 UTC m=+0.135708610 container create 52de883900050730f6a3359f4937d846d2d5bef1b706f6271f68f43b395a63fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-19T01:00:42.920 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:42 smithi163 podman[69277]: 2024-03-19 01:00:42.844327449 +0000 UTC m=+0.321236493 container init 52de883900050730f6a3359f4937d846d2d5bef1b706f6271f68f43b395a63fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-19T01:00:42.920 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:42 smithi163 podman[69277]: 2024-03-19 01:00:42.846907578 +0000 UTC m=+0.323816607 container start 52de883900050730f6a3359f4937d846d2d5bef1b706f6271f68f43b395a63fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T01:00:42.920 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:42 smithi163 ceph-mgr[69297]: -- 172.21.15.163:0/2440702432 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557be83dc5a0 con 0x557be83b0c00 2024-03-19T01:00:42.920 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:42 smithi163 bash[69277]: 52de883900050730f6a3359f4937d846d2d5bef1b706f6271f68f43b395a63fb 2024-03-19T01:00:43.311 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:42 smithi163 systemd[1]: Started Ceph mgr.z for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:00:43.811 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:43 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:43.397+0000 7fbe66899200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T01:00:43.811 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:43 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:43.491+0000 7fbe66899200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T01:00:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:00:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:00:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:00:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:44 smithi086 ceph-mon[66829]: Deploying daemon mgr.y on smithi086 2024-03-19T01:00:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:44 smithi086 ceph-mon[66829]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:00:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:00:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:00:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:44 smithi069 ceph-mon[31727]: Deploying daemon mgr.y on smithi086 2024-03-19T01:00:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:44 smithi069 ceph-mon[31727]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:44.311 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:44.133+0000 7fbe66899200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T01:00:44.311 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:44.224+0000 7fbe66899200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T01:00:44.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:44.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:00:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:00:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:00:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-mon[67428]: Deploying daemon mgr.y on smithi086 2024-03-19T01:00:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-mon[67428]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:44.316 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-19T01:00:44.317 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-19T01:00:44.317 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-03-19T01:00:44.317 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-03-19T01:00:44.317 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0353175 s, 297 MB/s 2024-03-19T01:00:44.317 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-03-19T01:00:44.807 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:44.366+0000 7fbe66899200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T01:00:44.807 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:44.461+0000 7fbe66899200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T01:00:44.807 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:44.543+0000 7fbe66899200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T01:00:44.807 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:44.805+0000 7fbe66899200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T01:00:45.047 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi069:vg_nvme/lv_4 2024-03-19T01:00:45.061 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:44.914+0000 7fbe66899200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T01:00:45.061 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:45 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:45.002+0000 7fbe66899200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T01:00:45.206 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:45 smithi086 systemd[1]: Starting Ceph mgr.y for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:00:45.459 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:45 smithi086 podman[68212]: 2024-03-19 01:00:45.205910314 +0000 UTC m=+0.106728191 container create 1d9b5a97ff38a92dbda8d130af67dddac4fb25e7dc638809eebb5b44fe6f6138 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:00:45.459 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:45 smithi086 podman[68212]: 2024-03-19 01:00:45.116929519 +0000 UTC m=+0.017747407 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:00:45.459 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:45 smithi086 podman[68212]: 2024-03-19 01:00:45.458514464 +0000 UTC m=+0.359332348 container init 1d9b5a97ff38a92dbda8d130af67dddac4fb25e7dc638809eebb5b44fe6f6138 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-19T01:00:45.483 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:00:45.561 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:45 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:45.097+0000 7fbe66899200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T01:00:45.561 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:45 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:45.272+0000 7fbe66899200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T01:00:45.755 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:45 smithi086 podman[68212]: 2024-03-19 01:00:45.461134415 +0000 UTC m=+0.361952285 container start 1d9b5a97ff38a92dbda8d130af67dddac4fb25e7dc638809eebb5b44fe6f6138 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-19T01:00:45.756 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:45 smithi086 ceph-mgr[68231]: -- 172.21.15.86:0/3173303875 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636cd6da5a0 con 0x5636cd6ae800 2024-03-19T01:00:45.756 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:45 smithi086 bash[68212]: 1d9b5a97ff38a92dbda8d130af67dddac4fb25e7dc638809eebb5b44fe6f6138 2024-03-19T01:00:45.756 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:45 smithi086 systemd[1]: Started Ceph mgr.y for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:00:46.505 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:46.015+0000 7fbc6250a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T01:00:46.506 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:46.112+0000 7fbc6250a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T01:00:46.561 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:46.253+0000 7fbe66899200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T01:00:46.561 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:46.389+0000 7fbe66899200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T01:00:46.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:46 smithi069 ceph-mon[31727]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:46.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:46.894 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:46.794+0000 7fbc6250a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T01:00:46.894 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:46 smithi086 ceph-mon[66829]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:46.894 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:46.895 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:46.895 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:46.895 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:46.895 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:47.036 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:46.742+0000 7fbe66899200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T01:00:47.037 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:46.834+0000 7fbe66899200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T01:00:47.037 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:47.034+0000 7fbe66899200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:00:47.037 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:46 smithi163 ceph-mon[67428]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:47.037 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:47.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:47.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:47.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:47.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:47.212 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:46.892+0000 7fbc6250a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T01:00:47.212 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:47.031+0000 7fbc6250a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T01:00:47.212 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:47.124+0000 7fbc6250a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T01:00:47.212 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:47.211+0000 7fbc6250a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T01:00:47.311 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:47.242+0000 7fbe66899200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T01:00:47.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.579+0000 7f7201c00700 1 -- 172.21.15.69:0/4064847887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71fc102440 msgr2=0x7f71fc1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:47.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.579+0000 7f7201c00700 1 --2- 172.21.15.69:0/4064847887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71fc102440 0x7f71fc1028a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009920 tx=0x7f71e8009c30 comp rx=0 tx=0).stop 2024-03-19T01:00:47.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.580+0000 7f7201c00700 1 -- 172.21.15.69:0/4064847887 shutdown_connections 2024-03-19T01:00:47.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.580+0000 7f7201c00700 1 --2- 172.21.15.69:0/4064847887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71fc102de0 0x7f71fc109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:47.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.580+0000 7f7201c00700 1 --2- 172.21.15.69:0/4064847887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71fc101240 0x7f71fc101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:47.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.580+0000 7f7201c00700 1 --2- 172.21.15.69:0/4064847887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71fc102440 0x7f71fc1028a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:47.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.580+0000 7f7201c00700 1 -- 172.21.15.69:0/4064847887 >> 172.21.15.69:0/4064847887 conn(0x7f71fc0fc7e0 msgr2=0x7f71fc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:00:47.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.580+0000 7f7201c00700 1 -- 172.21.15.69:0/4064847887 shutdown_connections 2024-03-19T01:00:47.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.580+0000 7f7201c00700 1 -- 172.21.15.69:0/4064847887 wait complete. 2024-03-19T01:00:47.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.581+0000 7f7201c00700 1 Processor -- start 2024-03-19T01:00:47.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f7201c00700 1 -- start start 2024-03-19T01:00:47.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f7201c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71fc101240 0x7f71fc105180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:47.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f7201c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71fc102440 0x7f71fc103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:47.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f7201c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71fc102de0 0x7f71fc103d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:47.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f7201c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc10ce70 con 0x7f71fc102de0 2024-03-19T01:00:47.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f7201c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc10ccf0 con 0x7f71fc102440 2024-03-19T01:00:47.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f7201c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc10cff0 con 0x7f71fc101240 2024-03-19T01:00:47.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f71fa000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71fc102440 0x7f71fc103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:47.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f71fb400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71fc102de0 0x7f71fc103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:47.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f71fa000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71fc102440 0x7f71fc103820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:49688/0 (socket says 172.21.15.69:49688) 2024-03-19T01:00:47.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f71fa000700 1 -- 172.21.15.69:0/940217606 learned_addr learned my addr 172.21.15.69:0/940217606 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:00:47.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.582+0000 7f71faa00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71fc101240 0x7f71fc105180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:47.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.583+0000 7f71fa000700 1 -- 172.21.15.69:0/940217606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71fc101240 msgr2=0x7f71fc105180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:47.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.583+0000 7f71fa000700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71fc101240 0x7f71fc105180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:47.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.583+0000 7f71fa000700 1 -- 172.21.15.69:0/940217606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71fc102de0 msgr2=0x7f71fc103d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:00:47.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.583+0000 7f71fa000700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71fc102de0 0x7f71fc103d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:00:47.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.583+0000 7f71fa000700 1 -- 172.21.15.69:0/940217606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc104630 con 0x7f71fc102440 2024-03-19T01:00:47.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.583+0000 7f71fb400700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71fc102de0 0x7f71fc103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:00:47.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.583+0000 7f71faa00700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71fc101240 0x7f71fc105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:00:47.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.583+0000 7f71fa000700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71fc102440 0x7f71fc103820 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009230 tx=0x7f71e8010f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:47.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.583+0000 7f71f3400700 1 -- 172.21.15.69:0/940217606 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80051f0 con 0x7f71fc102440 2024-03-19T01:00:47.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.583+0000 7f71f3400700 1 -- 172.21.15.69:0/940217606 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e800b560 con 0x7f71fc102440 2024-03-19T01:00:47.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.584+0000 7f7201c00700 1 -- 172.21.15.69:0/940217606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71fc1a2810 con 0x7f71fc102440 2024-03-19T01:00:47.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.584+0000 7f71f3400700 1 -- 172.21.15.69:0/940217606 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8003b70 con 0x7f71fc102440 2024-03-19T01:00:47.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.584+0000 7f7201c00700 1 -- 172.21.15.69:0/940217606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71fc1a2cc0 con 0x7f71fc102440 2024-03-19T01:00:47.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.585+0000 7f71f3400700 1 -- 172.21.15.69:0/940217606 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f71e8003d10 con 0x7f71fc102440 2024-03-19T01:00:47.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.585+0000 7f7201c00700 1 -- 172.21.15.69:0/940217606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71fc068fd0 con 0x7f71fc102440 2024-03-19T01:00:47.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.586+0000 7f71f3400700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f71e0039200 0x7f71e003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:00:47.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.586+0000 7f71f3400700 1 -- 172.21.15.69:0/940217606 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f71e804d340 con 0x7f71fc102440 2024-03-19T01:00:47.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.586+0000 7f71faa00700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f71e0039200 0x7f71e003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:00:47.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.587+0000 7f71faa00700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f71e0039200 0x7f71e003b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f71fc1022a0 tx=0x7f71ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:00:47.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.591+0000 7f71f3400700 1 -- 172.21.15.69:0/940217606 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f71e8013030 con 0x7f71fc102440 2024-03-19T01:00:47.755 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:47.477+0000 7fbc6250a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T01:00:47.756 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:47.583+0000 7fbc6250a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T01:00:47.756 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:47.667+0000 7fbc6250a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T01:00:47.790 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:47.528+0000 7fbe66899200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:00:47.790 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:47.788+0000 7fbe66899200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T01:00:47.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:47.824+0000 7f7201c00700 1 -- 172.21.15.69:0/940217606 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f71fc06a780 con 0x7f71e0039200 2024-03-19T01:00:48.061 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:48.021+0000 7fbe66899200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T01:00:48.255 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:47.760+0000 7fbc6250a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T01:00:48.255 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:47.930+0000 7fbc6250a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: Reconfiguring mgr.x (unknown last config time)... 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: Reconfiguring daemon mgr.x on smithi069 2024-03-19T01:00:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:00:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:00:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:48 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: Reconfiguring mgr.x (unknown last config time)... 2024-03-19T01:00:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:00:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:00:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: Reconfiguring daemon mgr.x on smithi069 2024-03-19T01:00:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:00:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:00:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:48.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: Reconfiguring mgr.x (unknown last config time)... 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: Reconfiguring daemon mgr.x on smithi069 2024-03-19T01:00:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:00:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:00:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:49.255 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:48 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:48.880+0000 7fbc6250a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T01:00:49.255 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:49.006+0000 7fbc6250a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T01:00:49.311 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:48 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:48.815+0000 7fbe66899200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T01:00:49.585 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:49.332+0000 7fbc6250a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T01:00:49.585 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:49.421+0000 7fbc6250a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T01:00:49.644 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:49 smithi069 ceph-mon[31727]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:00:49.644 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:49 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:49.644 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:49 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:49.644 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:49 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:49.695 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:49 smithi163 ceph-mon[67428]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:00:49.695 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:49 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:49.695 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:49 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:49.695 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:49 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:49.695 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:49.384+0000 7fbe66899200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T01:00:49.695 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:49.480+0000 7fbe66899200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T01:00:49.695 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:00:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:00:49.693+0000 7fbe66899200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T01:00:50.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:49 smithi086 ceph-mon[66829]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:00:50.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:49 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:50.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:49 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:50.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:49 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:50.006 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:49.616+0000 7fbc6250a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:00:50.006 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:49.801+0000 7fbc6250a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T01:00:50.337 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:50.081+0000 7fbc6250a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:00:50.338 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:50.336+0000 7fbc6250a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T01:00:50.588 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:50.567+0000 7fbc6250a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T01:00:50.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:50.619+0000 7f71f3400700 1 -- 172.21.15.69:0/940217606 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 14) ==== 91698+0+0 (secure 0 0 0) 0x7f71e8022cd0 con 0x7f71fc102440 2024-03-19T01:00:50.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:50 smithi069 ceph-mon[31727]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:50.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:50.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:50.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:50.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:50.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:50 smithi069 ceph-mon[31727]: Standby manager daemon z started 2024-03-19T01:00:50.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:50 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T01:00:50.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:50 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:00:50.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:50 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T01:00:50.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:50 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:00:51.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-mon[66829]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:51.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:51.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:51.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:51.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:51.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-mon[66829]: Standby manager daemon z started 2024-03-19T01:00:51.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T01:00:51.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:00:51.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T01:00:51.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:50 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:00:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:50 smithi163 ceph-mon[67428]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:00:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:00:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:00:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:00:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:50 smithi163 ceph-mon[67428]: Standby manager daemon z started 2024-03-19T01:00:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:50 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T01:00:51.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:50 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:00:51.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:50 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T01:00:51.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:50 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.163:0/2008083328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:00:51.622 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:51.363+0000 7fbc6250a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T01:00:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:51 smithi069 ceph-mon[31727]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-19T01:00:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:51 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T01:00:51.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:51 smithi086 ceph-mon[66829]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-19T01:00:51.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:51 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T01:00:52.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:51 smithi163 ceph-mon[67428]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-19T01:00:52.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:51 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T01:00:52.255 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:51.955+0000 7fbc6250a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T01:00:52.255 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:52.052+0000 7fbc6250a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T01:00:52.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:00:52.634+0000 7f71f3400700 1 -- 172.21.15.69:0/940217606 <== mon.2 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f71e800b710 con 0x7f71fc102440 2024-03-19T01:00:52.648 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:00:52.266+0000 7fbc6250a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T01:00:53.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-mon[66829]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:53.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-mon[66829]: Standby manager daemon y started 2024-03-19T01:00:53.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T01:00:53.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:00:53.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T01:00:53.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:00:53.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2712839898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d5a7ae-973f-48a0-a4f5-904f7be2b0bf"}]: dispatch 2024-03-19T01:00:53.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d5a7ae-973f-48a0-a4f5-904f7be2b0bf"}]: dispatch 2024-03-19T01:00:53.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53d5a7ae-973f-48a0-a4f5-904f7be2b0bf"}]': finished 2024-03-19T01:00:53.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-mon[66829]: osdmap e5: 1 total, 0 up, 1 in 2024-03-19T01:00:53.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:52 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:00:53.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:52 smithi069 ceph-mon[31727]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:53.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:52 smithi069 ceph-mon[31727]: Standby manager daemon y started 2024-03-19T01:00:53.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:52 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T01:00:53.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:52 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:00:53.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:52 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T01:00:53.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:52 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:00:53.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:52 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2712839898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d5a7ae-973f-48a0-a4f5-904f7be2b0bf"}]: dispatch 2024-03-19T01:00:53.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:52 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d5a7ae-973f-48a0-a4f5-904f7be2b0bf"}]: dispatch 2024-03-19T01:00:53.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:52 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53d5a7ae-973f-48a0-a4f5-904f7be2b0bf"}]': finished 2024-03-19T01:00:53.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:52 smithi069 ceph-mon[31727]: osdmap e5: 1 total, 0 up, 1 in 2024-03-19T01:00:53.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:52 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:00:53.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:52 smithi163 ceph-mon[67428]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:53.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:52 smithi163 ceph-mon[67428]: Standby manager daemon y started 2024-03-19T01:00:53.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:52 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T01:00:53.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:52 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:00:53.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:52 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T01:00:53.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:52 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.86:0/4015152662' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:00:53.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:52 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2712839898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d5a7ae-973f-48a0-a4f5-904f7be2b0bf"}]: dispatch 2024-03-19T01:00:53.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:52 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53d5a7ae-973f-48a0-a4f5-904f7be2b0bf"}]: dispatch 2024-03-19T01:00:53.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:52 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53d5a7ae-973f-48a0-a4f5-904f7be2b0bf"}]': finished 2024-03-19T01:00:53.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:52 smithi163 ceph-mon[67428]: osdmap e5: 1 total, 0 up, 1 in 2024-03-19T01:00:53.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:52 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:00:53.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:53 smithi069 ceph-mon[31727]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-03-19T01:00:53.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:53 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T01:00:53.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:53 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3981357314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:00:54.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:53 smithi086 ceph-mon[66829]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-03-19T01:00:54.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:53 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T01:00:54.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:53 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3981357314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:00:54.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:53 smithi163 ceph-mon[67428]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-03-19T01:00:54.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:53 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T01:00:54.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:53 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3981357314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:00:54.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:54 smithi086 ceph-mon[66829]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:55.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:54 smithi069 ceph-mon[31727]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:55.049 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:54 smithi163 ceph-mon[67428]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:57.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:56 smithi086 ceph-mon[66829]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:57.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:56 smithi069 ceph-mon[31727]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:57.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:56 smithi163 ceph-mon[67428]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:58.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:00:58 smithi069 ceph-mon[31727]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:59.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:00:58 smithi086 ceph-mon[66829]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:00:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:00:58 smithi163 ceph-mon[67428]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:01.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:00 smithi086 ceph-mon[66829]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:01.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:00 smithi069 ceph-mon[31727]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:01.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:00 smithi163 ceph-mon[67428]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:02.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:01 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-19T01:01:02.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:01 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:02.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:01 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-19T01:01:02.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:01 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:02.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:01 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-19T01:01:02.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:01 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:02.866 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:02 smithi069 ceph-mon[31727]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:02.866 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:02 smithi069 ceph-mon[31727]: Deploying daemon osd.0 on smithi069 2024-03-19T01:01:03.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:02 smithi086 ceph-mon[66829]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:03.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:02 smithi086 ceph-mon[66829]: Deploying daemon osd.0 on smithi069 2024-03-19T01:01:03.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:02 smithi163 ceph-mon[67428]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:03.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:02 smithi163 ceph-mon[67428]: Deploying daemon osd.0 on smithi069 2024-03-19T01:01:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:04 smithi086 ceph-mon[66829]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:04 smithi069 ceph-mon[31727]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:05.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:04 smithi163 ceph-mon[67428]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:06.965 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:06 smithi069 ceph-mon[31727]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:07.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:06 smithi086 ceph-mon[66829]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:07.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:06 smithi163 ceph-mon[67428]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:09.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:08 smithi086 ceph-mon[66829]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:08 smithi069 ceph-mon[31727]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:08 smithi163 ceph-mon[67428]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:10.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:10 smithi069 ceph-mon[31727]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:10.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:10 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:10.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:10 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:10.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:10 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:11.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:10 smithi086 ceph-mon[66829]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:11.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:10 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:11.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:10 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:11.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:10 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:11.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:10 smithi163 ceph-mon[67428]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:11.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:10 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:11.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:10 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:11.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:10 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:13.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:12 smithi086 ceph-mon[66829]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:13.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:12 smithi069 ceph-mon[31727]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:13.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:12 smithi163 ceph-mon[67428]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:14.185 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:13 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:14.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:13 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:14.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:13 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:14.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:13 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:14.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:13 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:14.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:13 smithi069 ceph-mon[31727]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:14.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:13 smithi069 ceph-mon[31727]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-19T01:01:14.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:13 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:14.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:13 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:14.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:13 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:14.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:13 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:14.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:13 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:14.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:13 smithi086 ceph-mon[66829]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:14.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:13 smithi086 ceph-mon[66829]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-19T01:01:14.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:13 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:14.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:13 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:14.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:13 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:14.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:13 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:14.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:13 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:14.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:13 smithi163 ceph-mon[67428]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:14.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:13 smithi163 ceph-mon[67428]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-19T01:01:14.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.315+0000 7f71f3400700 1 -- 172.21.15.69:0/940217606 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f71fc06a780 con 0x7f71e0039200 2024-03-19T01:01:14.317 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 0 on host 'smithi069' 2024-03-19T01:01:14.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.319+0000 7f7201c00700 1 -- 172.21.15.69:0/940217606 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f71e0039200 msgr2=0x7f71e003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:01:14.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.319+0000 7f7201c00700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f71e0039200 0x7f71e003b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f71fc1022a0 tx=0x7f71ec009260 comp rx=0 tx=0).stop 2024-03-19T01:01:14.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.319+0000 7f7201c00700 1 -- 172.21.15.69:0/940217606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71fc102440 msgr2=0x7f71fc103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:01:14.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.319+0000 7f7201c00700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71fc102440 0x7f71fc103820 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009230 tx=0x7f71e8010f80 comp rx=0 tx=0).stop 2024-03-19T01:01:14.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.319+0000 7f7201c00700 1 -- 172.21.15.69:0/940217606 shutdown_connections 2024-03-19T01:01:14.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.319+0000 7f7201c00700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f71e0039200 0x7f71e003b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:14.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.319+0000 7f7201c00700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71fc102de0 0x7f71fc103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:14.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.319+0000 7f7201c00700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71fc101240 0x7f71fc105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:14.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.319+0000 7f7201c00700 1 --2- 172.21.15.69:0/940217606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71fc102440 0x7f71fc103820 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:14.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.319+0000 7f7201c00700 1 -- 172.21.15.69:0/940217606 >> 172.21.15.69:0/940217606 conn(0x7f71fc0fc7e0 msgr2=0x7f71fc06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:01:14.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.320+0000 7f7201c00700 1 -- 172.21.15.69:0/940217606 shutdown_connections 2024-03-19T01:01:14.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:01:14.320+0000 7f7201c00700 1 -- 172.21.15.69:0/940217606 wait complete. 2024-03-19T01:01:15.070 DEBUG:teuthology.orchestra.run.smithi069:osd.0> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.0.service 2024-03-19T01:01:15.074 INFO:tasks.cephadm:Deploying osd.1 on smithi086 with /dev/vg_nvme/lv_4... 2024-03-19T01:01:15.074 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-19T01:01:15.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:14 smithi086 ceph-mon[66829]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-19T01:01:15.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:14 smithi086 ceph-mon[66829]: osdmap e6: 1 total, 0 up, 1 in 2024-03-19T01:01:15.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:14 smithi086 ceph-mon[66829]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:01:15.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:15.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:15.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:15.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:15.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:14 smithi069 ceph-mon[31727]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-19T01:01:15.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:14 smithi069 ceph-mon[31727]: osdmap e6: 1 total, 0 up, 1 in 2024-03-19T01:01:15.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:14 smithi069 ceph-mon[31727]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:01:15.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:15.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:15.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:15.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:15.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:14 smithi163 ceph-mon[67428]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-19T01:01:15.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:14 smithi163 ceph-mon[67428]: osdmap e6: 1 total, 0 up, 1 in 2024-03-19T01:01:15.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:14 smithi163 ceph-mon[67428]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:01:15.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:15.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:15.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:15.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:15.317 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:01:16.266 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:16 smithi086 ceph-mon[66829]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:01:16.267 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:16 smithi086 ceph-mon[66829]: osdmap e7: 1 total, 0 up, 1 in 2024-03-19T01:01:16.267 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:16 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:16.267 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:16 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:16.267 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:16 smithi086 ceph-mon[66829]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:16.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:16 smithi069 ceph-mon[31727]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:01:16.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:16 smithi069 ceph-mon[31727]: osdmap e7: 1 total, 0 up, 1 in 2024-03-19T01:01:16.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:16 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:16.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:16 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:16.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:16 smithi069 ceph-mon[31727]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:16.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:16 smithi163 ceph-mon[67428]: from='osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:01:16.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:16 smithi163 ceph-mon[67428]: osdmap e7: 1 total, 0 up, 1 in 2024-03-19T01:01:16.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:16 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:16.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:16 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:16.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:16 smithi163 ceph-mon[67428]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:01:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-19T01:01:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-19T01:01:17.112 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-19T01:01:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-19T01:01:17.112 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124967 s, 839 MB/s 2024-03-19T01:01:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-19T01:01:17.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:17 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:01:17.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:17 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:01:17.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:17 smithi069 ceph-mon[31727]: osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729] boot 2024-03-19T01:01:17.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:17 smithi069 ceph-mon[31727]: osdmap e8: 1 total, 1 up, 1 in 2024-03-19T01:01:17.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:17 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:17.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:17 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:01:17.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:17 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:01:17.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:17 smithi163 ceph-mon[67428]: osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729] boot 2024-03-19T01:01:17.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:17 smithi163 ceph-mon[67428]: osdmap e8: 1 total, 1 up, 1 in 2024-03-19T01:01:17.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:17.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:17 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:01:17.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:17 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:01:17.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:17 smithi086 ceph-mon[66829]: osd.0 [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729] boot 2024-03-19T01:01:17.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:17 smithi086 ceph-mon[66829]: osdmap e8: 1 total, 1 up, 1 in 2024-03-19T01:01:17.465 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:17 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:01:17.724 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-03-19T01:01:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:01:18.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:18 smithi069 ceph-mon[31727]: osdmap e9: 1 total, 1 up, 1 in 2024-03-19T01:01:18.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:18 smithi069 ceph-mon[31727]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:18.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:18 smithi163 ceph-mon[67428]: osdmap e9: 1 total, 1 up, 1 in 2024-03-19T01:01:18.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:18 smithi163 ceph-mon[67428]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:18.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:18 smithi086 ceph-mon[66829]: osdmap e9: 1 total, 1 up, 1 in 2024-03-19T01:01:18.350 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:18 smithi086 ceph-mon[66829]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:19.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.470+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2972241989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7bc101250 msgr2=0x7ff7bc101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:01:19.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.470+0000 7ff7c1e79700 1 --2- 172.21.15.86:0/2972241989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7bc101250 0x7ff7bc101670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0099c0 tx=0x7ff7ac009cd0 comp rx=0 tx=0).stop 2024-03-19T01:01:19.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.470+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2972241989 shutdown_connections 2024-03-19T01:01:19.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.470+0000 7ff7c1e79700 1 --2- 172.21.15.86:0/2972241989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7bc102df0 0x7ff7bc1096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.470+0000 7ff7c1e79700 1 --2- 172.21.15.86:0/2972241989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7bc101250 0x7ff7bc101670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.470+0000 7ff7c1e79700 1 --2- 172.21.15.86:0/2972241989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102450 0x7ff7bc1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.470+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2972241989 >> 172.21.15.86:0/2972241989 conn(0x7ff7bc0fc810 msgr2=0x7ff7bc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.470+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2972241989 shutdown_connections 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.470+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2972241989 wait complete. 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7c1e79700 1 Processor -- start 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7c1e79700 1 -- start start 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7c1e79700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7bc101250 0x7ff7bc06b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7c1e79700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102450 0x7ff7bc19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7c1e79700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7bc102df0 0x7ff7bc19fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7c1e79700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc10cf10 con 0x7ff7bc102450 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7c1e79700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc10cd90 con 0x7ff7bc101250 2024-03-19T01:01:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7c1e79700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc10d090 con 0x7ff7bc102df0 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7bb7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7bc101250 0x7ff7bc06b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7bb7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7bc101250 0x7ff7bc06b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45532/0 (socket says 172.21.15.86:45532) 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7bb7fe700 1 -- 172.21.15.86:0/2590551349 learned_addr learned my addr 172.21.15.86:0/2590551349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7bbfff700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7bc102df0 0x7ff7bc19fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7bb7fe700 1 -- 172.21.15.86:0/2590551349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7bc102df0 msgr2=0x7ff7bc19fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7bb7fe700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7bc102df0 0x7ff7bc19fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7bb7fe700 1 -- 172.21.15.86:0/2590551349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102450 msgr2=0x7ff7bc19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7bb7fe700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102450 0x7ff7bc19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.471+0000 7ff7bb7fe700 1 -- 172.21.15.86:0/2590551349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc1a0350 con 0x7ff7bc101250 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.472+0000 7ff7bb7fe700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7bc101250 0x7ff7bc06b5d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00c010 tx=0x7ff7ac005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.472+0000 7ff7b8ff9700 1 -- 172.21.15.86:0/2590551349 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac027070 con 0x7ff7bc101250 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.472+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2590551349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7bc1a05e0 con 0x7ff7bc101250 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.472+0000 7ff7b8ff9700 1 -- 172.21.15.86:0/2590551349 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ac0041d0 con 0x7ff7bc101250 2024-03-19T01:01:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.472+0000 7ff7b8ff9700 1 -- 172.21.15.86:0/2590551349 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac022e60 con 0x7ff7bc101250 2024-03-19T01:01:19.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.472+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2590551349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7bc1a0b90 con 0x7ff7bc101250 2024-03-19T01:01:19.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.473+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2590551349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7bc068fd0 con 0x7ff7bc101250 2024-03-19T01:01:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.474+0000 7ff7b8ff9700 1 -- 172.21.15.86:0/2590551349 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7ff7ac02a070 con 0x7ff7bc101250 2024-03-19T01:01:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.474+0000 7ff7b8ff9700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff7a40a2800 0x7ff7a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:01:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.475+0000 7ff7b8ff9700 1 -- 172.21.15.86:0/2590551349 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff7ac0cdf40 con 0x7ff7bc101250 2024-03-19T01:01:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.475+0000 7ff7baffd700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff7a40a2800 0x7ff7a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:01:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.476+0000 7ff7baffd700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff7a40a2800 0x7ff7a40a4cc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc06bb50 tx=0x7ff7b0006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:01:19.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.479+0000 7ff7b8ff9700 1 -- 172.21.15.86:0/2590551349 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff7ac013070 con 0x7ff7bc101250 2024-03-19T01:01:19.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:19.682+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2590551349 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff7bc062750 con 0x7ff7a40a2800 2024-03-19T01:01:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:01:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:01:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:01:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:21.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi069 2024-03-19T01:01:21.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:21.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:21.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:01:21.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi069 to 17090M 2024-03-19T01:01:21.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:21.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:21.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:21.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi069 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi069 to 17090M 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:21.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:01:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:01:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:01:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi069 2024-03-19T01:01:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:01:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi069 to 17090M 2024-03-19T01:01:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:22.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:22 smithi086 ceph-mon[66829]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:22 smithi086 ceph-mon[66829]: from='client.? 172.21.15.86:0/3379003789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7a78610-164b-454c-b229-91059a95c647"}]: dispatch 2024-03-19T01:01:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:22 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7a78610-164b-454c-b229-91059a95c647"}]: dispatch 2024-03-19T01:01:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:22 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7a78610-164b-454c-b229-91059a95c647"}]': finished 2024-03-19T01:01:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:22 smithi086 ceph-mon[66829]: osdmap e10: 2 total, 1 up, 2 in 2024-03-19T01:01:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:22 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:22 smithi069 ceph-mon[31727]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:22 smithi069 ceph-mon[31727]: from='client.? 172.21.15.86:0/3379003789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7a78610-164b-454c-b229-91059a95c647"}]: dispatch 2024-03-19T01:01:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:22 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7a78610-164b-454c-b229-91059a95c647"}]: dispatch 2024-03-19T01:01:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:22 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7a78610-164b-454c-b229-91059a95c647"}]': finished 2024-03-19T01:01:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:22 smithi069 ceph-mon[31727]: osdmap e10: 2 total, 1 up, 2 in 2024-03-19T01:01:23.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:22 smithi163 ceph-mon[67428]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:22 smithi163 ceph-mon[67428]: from='client.? 172.21.15.86:0/3379003789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7a78610-164b-454c-b229-91059a95c647"}]: dispatch 2024-03-19T01:01:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:22 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7a78610-164b-454c-b229-91059a95c647"}]: dispatch 2024-03-19T01:01:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:22 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7a78610-164b-454c-b229-91059a95c647"}]': finished 2024-03-19T01:01:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:22 smithi163 ceph-mon[67428]: osdmap e10: 2 total, 1 up, 2 in 2024-03-19T01:01:23.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:23.934 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:23 smithi069 ceph-mon[31727]: from='client.? 172.21.15.86:0/2679014131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:01:24.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:23 smithi086 ceph-mon[66829]: from='client.? 172.21.15.86:0/2679014131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:01:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:23 smithi163 ceph-mon[67428]: from='client.? 172.21.15.86:0/2679014131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:01:24.919 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:24 smithi069 ceph-mon[31727]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:25.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:24 smithi086 ceph-mon[66829]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:24 smithi163 ceph-mon[67428]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:26.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:26 smithi086 ceph-mon[66829]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:26.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:26 smithi069 ceph-mon[31727]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:27.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:26 smithi163 ceph-mon[67428]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:28 smithi086 ceph-mon[66829]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:29.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:28 smithi069 ceph-mon[31727]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:29.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:28 smithi163 ceph-mon[67428]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:29.975 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-19T01:01:29.975 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:30.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-19T01:01:30.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:30.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-19T01:01:30.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:30.681 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:30 smithi086 ceph-mon[66829]: Deploying daemon osd.1 on smithi086 2024-03-19T01:01:30.681 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:30 smithi086 ceph-mon[66829]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:31.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:30 smithi069 ceph-mon[31727]: Deploying daemon osd.1 on smithi086 2024-03-19T01:01:31.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:30 smithi069 ceph-mon[31727]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:31.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:30 smithi163 ceph-mon[67428]: Deploying daemon osd.1 on smithi086 2024-03-19T01:01:31.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:30 smithi163 ceph-mon[67428]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:33.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:32 smithi086 ceph-mon[66829]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:33.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:32 smithi069 ceph-mon[31727]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:32 smithi163 ceph-mon[67428]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:34 smithi086 ceph-mon[66829]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:34 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:34 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:34 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:34 smithi069 ceph-mon[31727]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:34 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:34 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:34 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:34 smithi163 ceph-mon[67428]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:34 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:34 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:34 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:36.946 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:36 smithi086 ceph-mon[66829]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:37.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:36 smithi069 ceph-mon[31727]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:37.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:36 smithi163 ceph-mon[67428]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.615+0000 7ff7b8ff9700 1 -- 172.21.15.86:0/2590551349 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff7bc062750 con 0x7ff7a40a2800 2024-03-19T01:01:37.616 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 1 on host 'smithi086' 2024-03-19T01:01:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2590551349 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff7a40a2800 msgr2=0x7ff7a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:01:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff7a40a2800 0x7ff7a40a4cc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc06bb50 tx=0x7ff7b0006d20 comp rx=0 tx=0).stop 2024-03-19T01:01:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2590551349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7bc101250 msgr2=0x7ff7bc06b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:01:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7bc101250 0x7ff7bc06b5d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00c010 tx=0x7ff7ac005e80 comp rx=0 tx=0).stop 2024-03-19T01:01:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2590551349 shutdown_connections 2024-03-19T01:01:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff7a40a2800 0x7ff7a40a4cc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7bc102df0 0x7ff7bc19fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7bc101250 0x7ff7bc06b5d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 --2- 172.21.15.86:0/2590551349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102450 0x7ff7bc19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2590551349 >> 172.21.15.86:0/2590551349 conn(0x7ff7bc0fc810 msgr2=0x7ff7bc105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:01:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2590551349 shutdown_connections 2024-03-19T01:01:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:01:37.617+0000 7ff7c1e79700 1 -- 172.21.15.86:0/2590551349 wait complete. 2024-03-19T01:01:38.193 DEBUG:teuthology.orchestra.run.smithi086:osd.1> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.1.service 2024-03-19T01:01:38.196 INFO:tasks.cephadm:Deploying osd.2 on smithi163 with /dev/vg_nvme/lv_4... 2024-03-19T01:01:38.196 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-19T01:01:38.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:38.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:38.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:38 smithi086 ceph-mon[66829]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:38.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.295 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:38 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:38 smithi163 ceph-mon[67428]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:38 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:38.441 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:01:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:38 smithi069 ceph-mon[31727]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:38 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:38.611 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:01:38 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1[71463]: 2024-03-19T01:01:38.293+0000 7fb317a797c0 -1 osd.1 0 log_to_monitors true 2024-03-19T01:01:39.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:39 smithi163 ceph-mon[67428]: from='osd.1 [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:01:39.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:39 smithi163 ceph-mon[67428]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:01:39.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:39 smithi086 ceph-mon[66829]: from='osd.1 [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:01:39.377 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:39 smithi086 ceph-mon[66829]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:01:39.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:39 smithi069 ceph-mon[31727]: from='osd.1 [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:01:39.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:39 smithi069 ceph-mon[31727]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:01:40.323 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-19T01:01:40.323 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-19T01:01:40.324 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2024-03-19T01:01:40.324 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2024-03-19T01:01:40.324 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114463 s, 916 MB/s 2024-03-19T01:01:40.324 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2024-03-19T01:01:40.324 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:40 smithi163 ceph-mon[67428]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-19T01:01:40.324 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:40 smithi163 ceph-mon[67428]: from='osd.1 [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:01:40.324 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:40 smithi163 ceph-mon[67428]: osdmap e11: 2 total, 1 up, 2 in 2024-03-19T01:01:40.324 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:40.324 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:40 smithi163 ceph-mon[67428]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:01:40.324 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:40 smithi163 ceph-mon[67428]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:40.324 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:40 smithi163 ceph-mon[67428]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:01:40.325 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:40 smithi163 ceph-mon[67428]: osdmap e12: 2 total, 1 up, 2 in 2024-03-19T01:01:40.505 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:01:40 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1[71463]: 2024-03-19T01:01:40.078+0000 7fb30d0de700 -1 osd.1 0 waiting for initial osdmap 2024-03-19T01:01:40.506 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:01:40 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1[71463]: 2024-03-19T01:01:40.084+0000 7fb308ed1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:01:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:40 smithi086 ceph-mon[66829]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-19T01:01:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:40 smithi086 ceph-mon[66829]: from='osd.1 [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:01:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:40 smithi086 ceph-mon[66829]: osdmap e11: 2 total, 1 up, 2 in 2024-03-19T01:01:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:40 smithi086 ceph-mon[66829]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:01:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:40 smithi086 ceph-mon[66829]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:40 smithi086 ceph-mon[66829]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:01:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:40 smithi086 ceph-mon[66829]: osdmap e12: 2 total, 1 up, 2 in 2024-03-19T01:01:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:40 smithi069 ceph-mon[31727]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-19T01:01:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:40 smithi069 ceph-mon[31727]: from='osd.1 [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:01:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:40 smithi069 ceph-mon[31727]: osdmap e11: 2 total, 1 up, 2 in 2024-03-19T01:01:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:40 smithi069 ceph-mon[31727]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:01:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:40 smithi069 ceph-mon[31727]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:40 smithi069 ceph-mon[31727]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:01:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:40 smithi069 ceph-mon[31727]: osdmap e12: 2 total, 1 up, 2 in 2024-03-19T01:01:40.935 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi163:vg_nvme/lv_4 2024-03-19T01:01:41.180 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:01:41.302 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:41.302 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:41.302 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.302 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.303 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:01:41.303 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.303 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:41.303 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:41.303 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.303 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:41 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:01:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:41 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:01:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:01:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:41 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:42.211 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:42 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:01:42.211 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:42 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:01:42.211 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:42 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi086 2024-03-19T01:01:42.211 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:42 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi086 to 17080M 2024-03-19T01:01:42.211 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:42 smithi163 ceph-mon[67428]: osd.1 [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946] boot 2024-03-19T01:01:42.211 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:42 smithi163 ceph-mon[67428]: osdmap e13: 2 total, 2 up, 2 in 2024-03-19T01:01:42.211 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:42 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:42.211 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:42 smithi163 ceph-mon[67428]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:42.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:42 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:01:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:42 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:01:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:42 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi086 2024-03-19T01:01:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:42 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi086 to 17080M 2024-03-19T01:01:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:42 smithi086 ceph-mon[66829]: osd.1 [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946] boot 2024-03-19T01:01:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:42 smithi086 ceph-mon[66829]: osdmap e13: 2 total, 2 up, 2 in 2024-03-19T01:01:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:42 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:42 smithi086 ceph-mon[66829]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:42 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:01:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:42 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:01:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:42 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi086 2024-03-19T01:01:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:42 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi086 to 17080M 2024-03-19T01:01:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:42 smithi069 ceph-mon[31727]: osd.1 [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946] boot 2024-03-19T01:01:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:42 smithi069 ceph-mon[31727]: osdmap e13: 2 total, 2 up, 2 in 2024-03-19T01:01:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:42 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:01:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:42 smithi069 ceph-mon[31727]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:01:42.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.709+0000 7f715793c700 1 -- 172.21.15.163:0/3451352528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7150102de0 msgr2=0x7f7150109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:01:42.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.709+0000 7f715793c700 1 --2- 172.21.15.163:0/3451352528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7150102de0 0x7f7150109690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f714800b0a0 tx=0x7f714800b3b0 comp rx=0 tx=0).stop 2024-03-19T01:01:42.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.709+0000 7f715793c700 1 -- 172.21.15.163:0/3451352528 shutdown_connections 2024-03-19T01:01:42.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.709+0000 7f715793c700 1 --2- 172.21.15.163:0/3451352528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7150102de0 0x7f7150109690 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:42.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.709+0000 7f715793c700 1 --2- 172.21.15.163:0/3451352528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7150101240 0x7f7150101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:42.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.709+0000 7f715793c700 1 --2- 172.21.15.163:0/3451352528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7150102440 0x7f71501028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:42.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.709+0000 7f715793c700 1 -- 172.21.15.163:0/3451352528 >> 172.21.15.163:0/3451352528 conn(0x7f71500fc800 msgr2=0x7f71500fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:01:42.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.709+0000 7f715793c700 1 -- 172.21.15.163:0/3451352528 shutdown_connections 2024-03-19T01:01:42.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.709+0000 7f715793c700 1 -- 172.21.15.163:0/3451352528 wait complete. 2024-03-19T01:01:42.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f715793c700 1 Processor -- start 2024-03-19T01:01:42.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f715793c700 1 -- start start 2024-03-19T01:01:42.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f715793c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7150101240 0x7f715006b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f715793c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7150102440 0x7f715019d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f715793c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7150102de0 0x7f715019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f715793c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f715010cef0 con 0x7f7150102de0 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f715793c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f715010cd70 con 0x7f7150102440 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f715793c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f715010d070 con 0x7f7150101240 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f71556d8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7150101240 0x7f715006b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f71556d8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7150101240 0x7f715006b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40754/0 (socket says 172.21.15.163:40754) 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f71556d8700 1 -- 172.21.15.163:0/1371909595 learned_addr learned my addr 172.21.15.163:0/1371909595 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.710+0000 7f7154ed7700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7150102440 0x7f715019d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.711+0000 7f71556d8700 1 -- 172.21.15.163:0/1371909595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7150102440 msgr2=0x7f715019d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.711+0000 7f71556d8700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7150102440 0x7f715019d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.711+0000 7f71556d8700 1 -- 172.21.15.163:0/1371909595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7150102de0 msgr2=0x7f715019f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:01:42.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.711+0000 7f71556d8700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7150102de0 0x7f715019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:01:42.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.711+0000 7f71556d8700 1 -- 172.21.15.163:0/1371909595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71501a01e0 con 0x7f7150101240 2024-03-19T01:01:42.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.711+0000 7f71556d8700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7150101240 0x7f715006b5f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f714c00b500 tx=0x7f714c00b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:01:42.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.711+0000 7f71467fc700 1 -- 172.21.15.163:0/1371909595 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c015020 con 0x7f7150101240 2024-03-19T01:01:42.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.711+0000 7f71467fc700 1 -- 172.21.15.163:0/1371909595 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f714c011d10 con 0x7f7150101240 2024-03-19T01:01:42.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.711+0000 7f715793c700 1 -- 172.21.15.163:0/1371909595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71501a04d0 con 0x7f7150101240 2024-03-19T01:01:42.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.711+0000 7f71467fc700 1 -- 172.21.15.163:0/1371909595 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c010e50 con 0x7f7150101240 2024-03-19T01:01:42.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.711+0000 7f715793c700 1 -- 172.21.15.163:0/1371909595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71501a0a90 con 0x7f7150101240 2024-03-19T01:01:42.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.712+0000 7f715793c700 1 -- 172.21.15.163:0/1371909595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7150068fd0 con 0x7f7150101240 2024-03-19T01:01:42.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.713+0000 7f71467fc700 1 -- 172.21.15.163:0/1371909595 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f714c018020 con 0x7f7150101240 2024-03-19T01:01:42.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.714+0000 7f71467fc700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f713c0a29a0 0x7f713c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:01:42.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.714+0000 7f71467fc700 1 -- 172.21.15.163:0/1371909595 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f714c0cce30 con 0x7f7150101240 2024-03-19T01:01:42.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.715+0000 7f7154ed7700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f713c0a29a0 0x7f713c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:01:42.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.716+0000 7f7154ed7700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f713c0a29a0 0x7f713c0a4e60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7140005950 tx=0x7f714000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:01:42.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.718+0000 7f71467fc700 1 -- 172.21.15.163:0/1371909595 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f714c01f020 con 0x7f7150101240 2024-03-19T01:01:42.926 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:01:42.925+0000 7f715793c700 1 -- 172.21.15.163:0/1371909595 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7150062750 con 0x7f713c0a29a0 2024-03-19T01:01:43.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:43 smithi163 ceph-mon[67428]: osdmap e14: 2 total, 2 up, 2 in 2024-03-19T01:01:43.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:43 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:01:43.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:43 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:01:43.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:43 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:43 smithi086 ceph-mon[66829]: osdmap e14: 2 total, 2 up, 2 in 2024-03-19T01:01:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:43 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:01:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:43 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:01:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:43 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:43.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:43 smithi069 ceph-mon[31727]: osdmap e14: 2 total, 2 up, 2 in 2024-03-19T01:01:43.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:43 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:01:43.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:43 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:01:43.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:43 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:44.377 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:44 smithi163 ceph-mon[67428]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:01:44.377 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:44 smithi163 ceph-mon[67428]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:44.407 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:44 smithi086 ceph-mon[66829]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:01:44.407 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:44 smithi086 ceph-mon[66829]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:44 smithi069 ceph-mon[31727]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:01:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:44 smithi069 ceph-mon[31727]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:45.683 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:45 smithi086 ceph-mon[66829]: from='client.? 172.21.15.163:0/3623629306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef693645-672d-4f7c-bfe9-1ba357b7fd50"}]: dispatch 2024-03-19T01:01:45.683 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:45 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef693645-672d-4f7c-bfe9-1ba357b7fd50"}]: dispatch 2024-03-19T01:01:45.684 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:45 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef693645-672d-4f7c-bfe9-1ba357b7fd50"}]': finished 2024-03-19T01:01:45.684 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:45 smithi086 ceph-mon[66829]: osdmap e15: 3 total, 2 up, 3 in 2024-03-19T01:01:45.684 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:45 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:01:45.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:45 smithi163 ceph-mon[67428]: from='client.? 172.21.15.163:0/3623629306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef693645-672d-4f7c-bfe9-1ba357b7fd50"}]: dispatch 2024-03-19T01:01:45.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:45 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef693645-672d-4f7c-bfe9-1ba357b7fd50"}]: dispatch 2024-03-19T01:01:45.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:45 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef693645-672d-4f7c-bfe9-1ba357b7fd50"}]': finished 2024-03-19T01:01:45.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:45 smithi163 ceph-mon[67428]: osdmap e15: 3 total, 2 up, 3 in 2024-03-19T01:01:45.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:45 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:01:46.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:45 smithi069 ceph-mon[31727]: from='client.? 172.21.15.163:0/3623629306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef693645-672d-4f7c-bfe9-1ba357b7fd50"}]: dispatch 2024-03-19T01:01:46.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:45 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef693645-672d-4f7c-bfe9-1ba357b7fd50"}]: dispatch 2024-03-19T01:01:46.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:45 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef693645-672d-4f7c-bfe9-1ba357b7fd50"}]': finished 2024-03-19T01:01:46.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:45 smithi069 ceph-mon[31727]: osdmap e15: 3 total, 2 up, 3 in 2024-03-19T01:01:46.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:45 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:01:46.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:46 smithi163 ceph-mon[67428]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:46.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:46 smithi163 ceph-mon[67428]: from='client.? 172.21.15.163:0/2825088578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:01:47.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:46 smithi086 ceph-mon[66829]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:47.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:46 smithi086 ceph-mon[66829]: from='client.? 172.21.15.163:0/2825088578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:01:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:46 smithi069 ceph-mon[31727]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:46 smithi069 ceph-mon[31727]: from='client.? 172.21.15.163:0/2825088578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:01:48.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:48 smithi163 ceph-mon[67428]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:49.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:48 smithi086 ceph-mon[66829]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:49.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:48 smithi069 ceph-mon[31727]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:50.697 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:50 smithi163 ceph-mon[67428]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:51.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:50 smithi086 ceph-mon[66829]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:50 smithi069 ceph-mon[31727]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:52.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:52 smithi163 ceph-mon[67428]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:53.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:52 smithi086 ceph-mon[66829]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:53.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:52 smithi069 ceph-mon[31727]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:53 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-19T01:01:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:53 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:53 smithi163 ceph-mon[67428]: Deploying daemon osd.2 on smithi163 2024-03-19T01:01:53.934 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:53 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-19T01:01:53.934 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:53 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:53.934 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:53 smithi069 ceph-mon[31727]: Deploying daemon osd.2 on smithi163 2024-03-19T01:01:54.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:53 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-19T01:01:54.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:53 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:01:54.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:53 smithi086 ceph-mon[66829]: Deploying daemon osd.2 on smithi163 2024-03-19T01:01:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:54 smithi163 ceph-mon[67428]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:55.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:54 smithi086 ceph-mon[66829]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:55.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:54 smithi069 ceph-mon[31727]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:56.739 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:56 smithi163 ceph-mon[67428]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:57.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:56 smithi086 ceph-mon[66829]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:57.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:56 smithi069 ceph-mon[31727]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:58.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:58 smithi163 ceph-mon[67428]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:58.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:58.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:58.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:01:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:59.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:58 smithi086 ceph-mon[66829]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:01:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:01:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:58 smithi069 ceph-mon[31727]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:01:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:01:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:01:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:00.769 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:00 smithi163 ceph-mon[67428]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:02:00.770 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:00.770 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:00.770 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:00.770 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:00.770 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:01.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:00 smithi086 ceph-mon[66829]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:02:01.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:01.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:01.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:01.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:01.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:01.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:00 smithi069 ceph-mon[31727]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:02:01.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:01.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:01.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:01.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:01.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:01.508 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.507+0000 7f71467fc700 1 -- 172.21.15.163:0/1371909595 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7150062750 con 0x7f713c0a29a0 2024-03-19T01:02:01.509 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 2 on host 'smithi163' 2024-03-19T01:02:01.510 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.509+0000 7f715793c700 1 -- 172.21.15.163:0/1371909595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f713c0a29a0 msgr2=0x7f713c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:01.511 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.509+0000 7f715793c700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f713c0a29a0 0x7f713c0a4e60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7140005950 tx=0x7f714000b410 comp rx=0 tx=0).stop 2024-03-19T01:02:01.511 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.509+0000 7f715793c700 1 -- 172.21.15.163:0/1371909595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7150101240 msgr2=0x7f715006b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:01.511 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.509+0000 7f715793c700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7150101240 0x7f715006b5f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f714c00b500 tx=0x7f714c00b810 comp rx=0 tx=0).stop 2024-03-19T01:02:01.511 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.510+0000 7f715793c700 1 -- 172.21.15.163:0/1371909595 shutdown_connections 2024-03-19T01:02:01.511 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.510+0000 7f715793c700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f713c0a29a0 0x7f713c0a4e60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:01.511 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.510+0000 7f715793c700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7150102de0 0x7f715019f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:01.511 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.510+0000 7f715793c700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7150101240 0x7f715006b5f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:01.511 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.510+0000 7f715793c700 1 --2- 172.21.15.163:0/1371909595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7150102440 0x7f715019d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:01.511 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.510+0000 7f715793c700 1 -- 172.21.15.163:0/1371909595 >> 172.21.15.163:0/1371909595 conn(0x7f71500fc800 msgr2=0x7f7150105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:02:01.511 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.510+0000 7f715793c700 1 -- 172.21.15.163:0/1371909595 shutdown_connections 2024-03-19T01:02:01.511 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:02:01.510+0000 7f715793c700 1 -- 172.21.15.163:0/1371909595 wait complete. 2024-03-19T01:02:02.220 DEBUG:teuthology.orchestra.run.smithi163:osd.2> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.2.service 2024-03-19T01:02:02.223 INFO:tasks.cephadm:Deploying osd.3 on smithi069 with /dev/vg_nvme/lv_3... 2024-03-19T01:02:02.223 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-19T01:02:02.654 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:02:02.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:02 smithi086 ceph-mon[66829]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:02:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:02 smithi086 ceph-mon[66829]: from='osd.2 [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:02:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:02 smithi086 ceph-mon[66829]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:02:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:02 smithi069 ceph-mon[31727]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:02:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:02 smithi069 ceph-mon[31727]: from='osd.2 [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:02:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:02 smithi069 ceph-mon[31727]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:02:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:02 smithi163 ceph-mon[67428]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:02:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:02 smithi163 ceph-mon[67428]: from='osd.2 [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:02:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:02 smithi163 ceph-mon[67428]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:02:03.711 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:03 smithi163 ceph-mon[67428]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-19T01:02:03.711 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:03 smithi163 ceph-mon[67428]: osdmap e16: 3 total, 2 up, 3 in 2024-03-19T01:02:03.711 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:03 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:03.711 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:03 smithi163 ceph-mon[67428]: from='osd.2 [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:02:03.711 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:03 smithi163 ceph-mon[67428]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:02:03.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:03 smithi069 ceph-mon[31727]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-19T01:02:03.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:03 smithi069 ceph-mon[31727]: osdmap e16: 3 total, 2 up, 3 in 2024-03-19T01:02:03.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:03.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:03 smithi069 ceph-mon[31727]: from='osd.2 [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:02:03.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:03 smithi069 ceph-mon[31727]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:02:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:03 smithi086 ceph-mon[66829]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-19T01:02:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:03 smithi086 ceph-mon[66829]: osdmap e16: 3 total, 2 up, 3 in 2024-03-19T01:02:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:03 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:03 smithi086 ceph-mon[66829]: from='osd.2 [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:02:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:03 smithi086 ceph-mon[66829]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:02:04.061 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:02:03 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2[72574]: 2024-03-19T01:02:03.943+0000 7f645ef11700 -1 osd.2 0 waiting for initial osdmap 2024-03-19T01:02:04.061 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:02:03 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2[72574]: 2024-03-19T01:02:03.947+0000 7f6456cfc700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:02:04.669 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:04 smithi069 ceph-mon[31727]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:02:04.669 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:04 smithi069 ceph-mon[31727]: osdmap e17: 3 total, 2 up, 3 in 2024-03-19T01:02:04.669 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:04 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:04.669 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:04 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:04.669 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:04 smithi069 ceph-mon[31727]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:02:04.669 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:04 smithi069 ceph-mon[31727]: from='osd.2 ' entity='osd.2' 2024-03-19T01:02:04.773 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:04 smithi163 ceph-mon[67428]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:02:04.773 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:04 smithi163 ceph-mon[67428]: osdmap e17: 3 total, 2 up, 3 in 2024-03-19T01:02:04.774 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:04 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:04.774 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:04 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:04.774 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:04 smithi163 ceph-mon[67428]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:02:04.774 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:04 smithi163 ceph-mon[67428]: from='osd.2 ' entity='osd.2' 2024-03-19T01:02:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:04 smithi086 ceph-mon[66829]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:02:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:04 smithi086 ceph-mon[66829]: osdmap e17: 3 total, 2 up, 3 in 2024-03-19T01:02:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:04 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:04 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:04 smithi086 ceph-mon[66829]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:02:05.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:04 smithi086 ceph-mon[66829]: from='osd.2 ' entity='osd.2' 2024-03-19T01:02:05.542 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-19T01:02:05.542 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-19T01:02:05.542 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-03-19T01:02:05.542 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-03-19T01:02:05.543 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0395533 s, 265 MB/s 2024-03-19T01:02:05.543 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-03-19T01:02:05.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:02:05.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:02:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: osd.2 [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503] boot 2024-03-19T01:02:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: osdmap e18: 3 total, 3 up, 3 in 2024-03-19T01:02:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:02:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:02:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:02:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: osd.2 [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503] boot 2024-03-19T01:02:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: osdmap e18: 3 total, 3 up, 3 in 2024-03-19T01:02:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:02:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:06.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:02:06.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:02:06.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:06.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: osd.2 [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503] boot 2024-03-19T01:02:06.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: osdmap e18: 3 total, 3 up, 3 in 2024-03-19T01:02:06.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:02:06.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:06.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:02:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:06.405 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi069:vg_nvme/lv_3 2024-03-19T01:02:06.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:06 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi163 2024-03-19T01:02:06.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:06 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi163 to 17080M 2024-03-19T01:02:06.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:06 smithi069 ceph-mon[31727]: pgmap v89: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:06.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:02:06.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:06 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi163 2024-03-19T01:02:06.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:06 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi163 to 17080M 2024-03-19T01:02:06.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:06 smithi163 ceph-mon[67428]: pgmap v89: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:06.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:02:06.829 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:02:07.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:06 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi163 2024-03-19T01:02:07.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:06 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi163 to 17080M 2024-03-19T01:02:07.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:06 smithi086 ceph-mon[66829]: pgmap v89: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:07.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:02:07.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:07 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-19T01:02:07.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:07 smithi163 ceph-mon[67428]: osdmap e19: 3 total, 3 up, 3 in 2024-03-19T01:02:07.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:07 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:02:07.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:07 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-19T01:02:07.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:07 smithi069 ceph-mon[31727]: osdmap e19: 3 total, 3 up, 3 in 2024-03-19T01:02:07.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:07 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:02:08.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:07 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-19T01:02:08.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:07 smithi086 ceph-mon[66829]: osdmap e19: 3 total, 3 up, 3 in 2024-03-19T01:02:08.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:07 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:02:08.436 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:02:08 smithi069 sudo[48292]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:02:08.437 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:02:08 smithi069 sudo[48292]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:08.437 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:02:08 smithi069 sudo[48292]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:08.437 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:02:08 smithi069 sudo[48292]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:08.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:08 smithi069 ceph-mon[31727]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:08.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-19T01:02:08.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:08 smithi069 ceph-mon[31727]: osdmap e20: 3 total, 3 up, 3 in 2024-03-19T01:02:08.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:08 smithi069 ceph-mon[31727]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:02:08.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:08 smithi069 ceph-mon[31727]: osdmap e21: 3 total, 3 up, 3 in 2024-03-19T01:02:08.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.918+0000 7f7234600700 1 -- 172.21.15.69:0/3316321349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722c105f00 msgr2=0x7f722c106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:08.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.918+0000 7f7234600700 1 --2- 172.21.15.69:0/3316321349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722c105f00 0x7f722c106380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7224009f90 tx=0x7f722400b330 comp rx=0 tx=0).stop 2024-03-19T01:02:08.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.919+0000 7f7234600700 1 -- 172.21.15.69:0/3316321349 shutdown_connections 2024-03-19T01:02:08.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.919+0000 7f7234600700 1 --2- 172.21.15.69:0/3316321349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722c105f00 0x7f722c106380 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:08.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.919+0000 7f7234600700 1 --2- 172.21.15.69:0/3316321349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f722c100bc0 0x7f722c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:08.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.919+0000 7f7234600700 1 --2- 172.21.15.69:0/3316321349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c103530 0x7f722c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:08.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.919+0000 7f7234600700 1 -- 172.21.15.69:0/3316321349 >> 172.21.15.69:0/3316321349 conn(0x7f722c0fc7e0 msgr2=0x7f722c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:02:08.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.919+0000 7f7234600700 1 -- 172.21.15.69:0/3316321349 shutdown_connections 2024-03-19T01:02:08.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.919+0000 7f7234600700 1 -- 172.21.15.69:0/3316321349 wait complete. 2024-03-19T01:02:08.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.920+0000 7f7234600700 1 Processor -- start 2024-03-19T01:02:08.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.920+0000 7f7234600700 1 -- start start 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7234600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f722c100bc0 0x7f722c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7234600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c103530 0x7f722c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7234600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722c105f00 0x7f722c1a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7234600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f722c114500 con 0x7f722c100bc0 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7234600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f722c114380 con 0x7f722c103530 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7234600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f722c114680 con 0x7f722c105f00 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7231200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c103530 0x7f722c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7231200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c103530 0x7f722c19f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58366/0 (socket says 172.21.15.69:58366) 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7231c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f722c100bc0 0x7f722c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7231200700 1 -- 172.21.15.69:0/462976862 learned_addr learned my addr 172.21.15.69:0/462976862 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7232600700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722c105f00 0x7f722c1a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:02:08.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7231200700 1 -- 172.21.15.69:0/462976862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722c105f00 msgr2=0x7f722c1a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.921+0000 7f7231200700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722c105f00 0x7f722c1a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.922+0000 7f7231200700 1 -- 172.21.15.69:0/462976862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f722c100bc0 msgr2=0x7f722c19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.922+0000 7f7231200700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f722c100bc0 0x7f722c19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.922+0000 7f7231200700 1 -- 172.21.15.69:0/462976862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722c1a6770 con 0x7f722c103530 2024-03-19T01:02:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.922+0000 7f7231c00700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f722c100bc0 0x7f722c19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:02:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.922+0000 7f7232600700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722c105f00 0x7f722c1a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:02:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.922+0000 7f7231200700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c103530 0x7f722c19f050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f721800e9c0 tx=0x7f721800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:02:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.922+0000 7f722a000700 1 -- 172.21.15.69:0/462976862 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721800cbd0 con 0x7f722c103530 2024-03-19T01:02:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.922+0000 7f7234600700 1 -- 172.21.15.69:0/462976862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f722c1a6a60 con 0x7f722c103530 2024-03-19T01:02:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.922+0000 7f722a000700 1 -- 172.21.15.69:0/462976862 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f721800ee60 con 0x7f722c103530 2024-03-19T01:02:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.922+0000 7f722a000700 1 -- 172.21.15.69:0/462976862 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218005680 con 0x7f722c103530 2024-03-19T01:02:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.922+0000 7f7234600700 1 -- 172.21.15.69:0/462976862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f722c1a6fa0 con 0x7f722c103530 2024-03-19T01:02:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.924+0000 7f7234600700 1 -- 172.21.15.69:0/462976862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722c068fd0 con 0x7f722c103530 2024-03-19T01:02:08.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.926+0000 7f722a000700 1 -- 172.21.15.69:0/462976862 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f7218004750 con 0x7f722c103530 2024-03-19T01:02:08.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.928+0000 7f722a000700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f72140a2880 0x7f72140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:02:08.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.928+0000 7f722a000700 1 -- 172.21.15.69:0/462976862 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f72180cd900 con 0x7f722c103530 2024-03-19T01:02:08.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.928+0000 7f7231c00700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f72140a2880 0x7f72140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:02:08.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.928+0000 7f7231c00700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f72140a2880 0x7f72140a4d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f72200114f0 tx=0x7f72200093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:02:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:08.932+0000 7f722a000700 1 -- 172.21.15.69:0/462976862 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7218053740 con 0x7f722c103530 2024-03-19T01:02:09.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:08 smithi086 ceph-mon[66829]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:09.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:08 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-19T01:02:09.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:08 smithi086 ceph-mon[66829]: osdmap e20: 3 total, 3 up, 3 in 2024-03-19T01:02:09.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:08 smithi086 ceph-mon[66829]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:02:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:08 smithi086 ceph-mon[66829]: osdmap e21: 3 total, 3 up, 3 in 2024-03-19T01:02:09.032 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:02:08 smithi069 sudo[48314]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:02:09.032 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:02:08 smithi069 sudo[48314]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:09.032 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:02:08 smithi069 sudo[48314]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:09.032 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:02:08 smithi069 sudo[48314]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:08 smithi163 ceph-mon[67428]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:08 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-19T01:02:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:08 smithi163 ceph-mon[67428]: osdmap e20: 3 total, 3 up, 3 in 2024-03-19T01:02:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:08 smithi163 ceph-mon[67428]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:02:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:08 smithi163 ceph-mon[67428]: osdmap e21: 3 total, 3 up, 3 in 2024-03-19T01:02:09.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:09.148+0000 7f7234600700 1 -- 172.21.15.69:0/462976862 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f722c062750 con 0x7f72140a2880 2024-03-19T01:02:09.431 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:02:09 smithi086 sudo[75180]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:02:09.432 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:02:09 smithi086 sudo[75180]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:09.432 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:02:09 smithi086 sudo[75180]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:09.432 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:02:09 smithi086 sudo[75180]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:09.755 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:02:09 smithi086 sudo[75183]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:02:09.755 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:02:09 smithi086 sudo[75183]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:09.755 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:02:09 smithi086 sudo[75183]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:09.756 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:02:09 smithi086 sudo[75183]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:09.969 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:02:09 smithi163 sudo[76258]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:02:09.969 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:02:09 smithi163 sudo[76258]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:09.969 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:02:09 smithi163 sudo[76258]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:09.969 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:02:09 smithi163 sudo[76258]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:09.970 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:09.970 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:09 smithi163 ceph-mon[67428]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:02:09.970 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:02:09.970 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:02:09.970 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:09.970 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:09.970 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:09 smithi163 ceph-mon[67428]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:10.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:10.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:09 smithi069 ceph-mon[31727]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:02:10.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:02:10.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:02:10.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:10.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:10.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:09 smithi069 ceph-mon[31727]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:10.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:10.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:09 smithi086 ceph-mon[66829]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:02:10.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:02:10.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:02:10.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:10.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:10.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:09 smithi086 ceph-mon[66829]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:10.311 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:02:09 smithi163 sudo[76261]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:02:10.311 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:02:09 smithi163 sudo[76261]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:10.311 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:02:09 smithi163 sudo[76261]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:10.311 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:02:09 smithi163 sudo[76261]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:11.024 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:10 smithi069 sudo[48474]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:02:11.024 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:10 smithi069 sudo[48474]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:11.024 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:10 smithi069 sudo[48474]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:11.024 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:10 smithi069 sudo[48474]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:11.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:11.135+0000 7f722a000700 1 -- 172.21.15.69:0/462976862 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7218097d20 con 0x7f722c103530 2024-03-19T01:02:11.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:11 smithi069 sudo[48482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:02:11.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:11 smithi069 sudo[48482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:11.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:11 smithi069 sudo[48482]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:11.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:11 smithi069 sudo[48482]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:11.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:11.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:11 smithi069 ceph-mon[31727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:02:11.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:11.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:11 smithi086 ceph-mon[66829]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:02:11.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:11 smithi163 sudo[76264]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:02:11.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:11 smithi163 sudo[76264]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:11.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:11 smithi163 sudo[76264]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:11.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:11 smithi163 sudo[76264]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:11.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:11 smithi163 sudo[76267]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:02:11.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:11 smithi163 sudo[76267]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:11.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:11 smithi163 sudo[76267]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:11.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:11 smithi163 sudo[76267]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:11.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:11.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:11 smithi163 ceph-mon[67428]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:02:12.105 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:11 smithi086 sudo[75186]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:02:12.105 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:11 smithi086 sudo[75186]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:12.105 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:11 smithi086 sudo[75186]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:12.105 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:11 smithi086 sudo[75186]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 sudo[75189]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 sudo[75189]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 sudo[75189]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 sudo[75189]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: Cluster is now healthy 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:02:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:02:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:02:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-19T01:02:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:02:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:02:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:02:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:02:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:02:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:02:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:12 smithi086 ceph-mon[66829]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: Cluster is now healthy 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:02:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:02:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:02:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:02:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:02:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:02:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:12 smithi069 ceph-mon[31727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:02:12.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:02:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:02:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: Cluster is now healthy 2024-03-19T01:02:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:02:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:02:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:02:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:02:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:02:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:02:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-19T01:02:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:02:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:02:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:02:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:02:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:02:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:02:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:12 smithi163 ceph-mon[67428]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:02:13.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:13 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2626997095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c06aaad-6a1f-4e27-b991-248826c99079"}]: dispatch 2024-03-19T01:02:13.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:13 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2626997095' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c06aaad-6a1f-4e27-b991-248826c99079"}]': finished 2024-03-19T01:02:13.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:13 smithi086 ceph-mon[66829]: osdmap e22: 4 total, 3 up, 4 in 2024-03-19T01:02:13.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:13 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:13.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:13 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2626997095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c06aaad-6a1f-4e27-b991-248826c99079"}]: dispatch 2024-03-19T01:02:13.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:13 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2626997095' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c06aaad-6a1f-4e27-b991-248826c99079"}]': finished 2024-03-19T01:02:13.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:13 smithi069 ceph-mon[31727]: osdmap e22: 4 total, 3 up, 4 in 2024-03-19T01:02:13.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:13 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:13.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:13 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2626997095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c06aaad-6a1f-4e27-b991-248826c99079"}]: dispatch 2024-03-19T01:02:13.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:13 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2626997095' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c06aaad-6a1f-4e27-b991-248826c99079"}]': finished 2024-03-19T01:02:13.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:13 smithi163 ceph-mon[67428]: osdmap e22: 4 total, 3 up, 4 in 2024-03-19T01:02:13.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:13 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:14.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:14 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/464346108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:02:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:14 smithi086 ceph-mon[66829]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:14.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:14 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/464346108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:02:14.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:14 smithi069 ceph-mon[31727]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:14 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/464346108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:02:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:14 smithi163 ceph-mon[67428]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:16.864 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:16 smithi069 ceph-mon[31727]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:17.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:16 smithi086 ceph-mon[66829]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:17.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:16 smithi163 ceph-mon[67428]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:18.903 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:18 smithi069 ceph-mon[31727]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 54 KiB/s wr, 5 op/s 2024-03-19T01:02:19.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:18 smithi086 ceph-mon[66829]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 54 KiB/s wr, 5 op/s 2024-03-19T01:02:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:18 smithi163 ceph-mon[67428]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 54 KiB/s wr, 5 op/s 2024-03-19T01:02:21.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:20 smithi086 ceph-mon[66829]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 38 KiB/s wr, 3 op/s 2024-03-19T01:02:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:20 smithi069 ceph-mon[31727]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 38 KiB/s wr, 3 op/s 2024-03-19T01:02:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:20 smithi163 ceph-mon[67428]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 38 KiB/s wr, 3 op/s 2024-03-19T01:02:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:21 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-19T01:02:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:21 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:22.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:21 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-19T01:02:22.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:21 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:22.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:21 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-19T01:02:22.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:21 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:22 smithi069 ceph-mon[31727]: Deploying daemon osd.3 on smithi069 2024-03-19T01:02:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:22 smithi069 ceph-mon[31727]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 30 KiB/s wr, 2 op/s 2024-03-19T01:02:23.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:22 smithi086 ceph-mon[66829]: Deploying daemon osd.3 on smithi069 2024-03-19T01:02:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:22 smithi086 ceph-mon[66829]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 30 KiB/s wr, 2 op/s 2024-03-19T01:02:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:22 smithi163 ceph-mon[67428]: Deploying daemon osd.3 on smithi069 2024-03-19T01:02:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:22 smithi163 ceph-mon[67428]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 30 KiB/s wr, 2 op/s 2024-03-19T01:02:24.730 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:24 smithi069 ceph-mon[31727]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 941 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-19T01:02:25.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:24 smithi086 ceph-mon[66829]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 941 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-19T01:02:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:24 smithi163 ceph-mon[67428]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 941 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-19T01:02:26.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:26 smithi069 ceph-mon[31727]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-19T01:02:27.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:26 smithi086 ceph-mon[66829]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-19T01:02:27.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:26 smithi163 ceph-mon[67428]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-19T01:02:28.967 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:28 smithi069 ceph-mon[31727]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-19T01:02:29.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:28 smithi086 ceph-mon[66829]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-19T01:02:29.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:28 smithi163 ceph-mon[67428]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-19T01:02:30.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:30.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:30.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:30.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:30 smithi069 ceph-mon[31727]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:30 smithi163 ceph-mon[67428]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:31.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:31.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:31.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:31.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:30 smithi086 ceph-mon[66829]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:32.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:32 smithi069 ceph-mon[31727]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:32 smithi086 ceph-mon[66829]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:32 smithi163 ceph-mon[67428]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:33 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:33 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:34.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:33 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:34.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:33 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:34.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:33 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:34.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:33 smithi163 ceph-mon[67428]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-19T01:02:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:34.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:34.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:33 smithi086 ceph-mon[66829]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-19T01:02:34.278 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:33 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:34.278 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:33 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:34.278 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:33 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:34.278 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:33 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:34.278 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:33 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:34.278 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:33 smithi069 ceph-mon[31727]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-19T01:02:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.639+0000 7f722a000700 1 -- 172.21.15.69:0/462976862 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f722c062750 con 0x7f72140a2880 2024-03-19T01:02:34.641 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 3 on host 'smithi069' 2024-03-19T01:02:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.644+0000 7f7234600700 1 -- 172.21.15.69:0/462976862 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f72140a2880 msgr2=0x7f72140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.644+0000 7f7234600700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f72140a2880 0x7f72140a4d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f72200114f0 tx=0x7f72200093e0 comp rx=0 tx=0).stop 2024-03-19T01:02:34.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.644+0000 7f7234600700 1 -- 172.21.15.69:0/462976862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c103530 msgr2=0x7f722c19f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:34.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.644+0000 7f7234600700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c103530 0x7f722c19f050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f721800e9c0 tx=0x7f721800ecd0 comp rx=0 tx=0).stop 2024-03-19T01:02:34.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.644+0000 7f7234600700 1 -- 172.21.15.69:0/462976862 shutdown_connections 2024-03-19T01:02:34.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.644+0000 7f7234600700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f72140a2880 0x7f72140a4d40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:34.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.644+0000 7f7234600700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722c105f00 0x7f722c1a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:34.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.644+0000 7f7234600700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f722c100bc0 0x7f722c19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:34.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.645+0000 7f7234600700 1 --2- 172.21.15.69:0/462976862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c103530 0x7f722c19f050 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:34.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.645+0000 7f7234600700 1 -- 172.21.15.69:0/462976862 >> 172.21.15.69:0/462976862 conn(0x7f722c0fc7e0 msgr2=0x7f722c101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:02:34.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.645+0000 7f7234600700 1 -- 172.21.15.69:0/462976862 shutdown_connections 2024-03-19T01:02:34.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:02:34.645+0000 7f7234600700 1 -- 172.21.15.69:0/462976862 wait complete. 2024-03-19T01:02:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:34 smithi163 ceph-mon[67428]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:34 smithi163 ceph-mon[67428]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-19T01:02:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:34 smithi163 ceph-mon[67428]: osdmap e23: 4 total, 3 up, 4 in 2024-03-19T01:02:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:34 smithi163 ceph-mon[67428]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:02:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:34 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:34 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:34 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:34 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:35.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:34 smithi069 ceph-mon[31727]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:35.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:34 smithi069 ceph-mon[31727]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-19T01:02:35.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:34 smithi069 ceph-mon[31727]: osdmap e23: 4 total, 3 up, 4 in 2024-03-19T01:02:35.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:34 smithi069 ceph-mon[31727]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:02:35.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:34 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:35.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:34 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:35.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:34 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:35.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:34 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:35.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:34 smithi086 ceph-mon[66829]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:35.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:34 smithi086 ceph-mon[66829]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-19T01:02:35.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:34 smithi086 ceph-mon[66829]: osdmap e23: 4 total, 3 up, 4 in 2024-03-19T01:02:35.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:34 smithi086 ceph-mon[66829]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:02:35.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:34 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:35.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:34 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:35.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:34 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:35.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:34 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:35.459 DEBUG:teuthology.orchestra.run.smithi069:osd.3> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.3.service 2024-03-19T01:02:35.463 INFO:tasks.cephadm:Deploying osd.4 on smithi086 with /dev/vg_nvme/lv_3... 2024-03-19T01:02:35.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-19T01:02:35.707 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:02:36.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:35 smithi069 ceph-mon[31727]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:02:36.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:35 smithi069 ceph-mon[31727]: osdmap e24: 4 total, 3 up, 4 in 2024-03-19T01:02:36.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:36.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:35 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:36.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:35 smithi069 ceph-mon[31727]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' 2024-03-19T01:02:36.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:35 smithi069 ceph-mon[31727]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:36.113 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:35 smithi086 ceph-mon[66829]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:02:36.113 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:35 smithi086 ceph-mon[66829]: osdmap e24: 4 total, 3 up, 4 in 2024-03-19T01:02:36.113 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:36.113 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:35 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:36.114 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:35 smithi086 ceph-mon[66829]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' 2024-03-19T01:02:36.114 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:35 smithi086 ceph-mon[66829]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:36.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:35 smithi163 ceph-mon[67428]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:02:36.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:35 smithi163 ceph-mon[67428]: osdmap e24: 4 total, 3 up, 4 in 2024-03-19T01:02:36.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:36.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:35 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:36.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:35 smithi163 ceph-mon[67428]: from='osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914]' entity='osd.3' 2024-03-19T01:02:36.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:35 smithi163 ceph-mon[67428]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:02:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:36 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:02:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:36 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:02:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:36 smithi086 ceph-mon[66829]: osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914] boot 2024-03-19T01:02:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:36 smithi086 ceph-mon[66829]: osdmap e25: 4 total, 4 up, 4 in 2024-03-19T01:02:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:36 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:37.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:36 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:02:37.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:36 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:02:37.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:36 smithi069 ceph-mon[31727]: osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914] boot 2024-03-19T01:02:37.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:36 smithi069 ceph-mon[31727]: osdmap e25: 4 total, 4 up, 4 in 2024-03-19T01:02:37.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:36 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:37.212 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:36 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:02:37.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:36 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:02:37.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:36 smithi163 ceph-mon[67428]: osd.3 [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914] boot 2024-03-19T01:02:37.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:36 smithi163 ceph-mon[67428]: osdmap e25: 4 total, 4 up, 4 in 2024-03-19T01:02:37.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:36 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:02:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-19T01:02:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-19T01:02:37.616 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-19T01:02:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-19T01:02:37.616 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113753 s, 922 MB/s 2024-03-19T01:02:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-19T01:02:38.149 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:37 smithi086 ceph-mon[66829]: osdmap e26: 4 total, 4 up, 4 in 2024-03-19T01:02:38.149 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:37 smithi086 ceph-mon[66829]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:38.175 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-03-19T01:02:38.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:37 smithi069 ceph-mon[31727]: osdmap e26: 4 total, 4 up, 4 in 2024-03-19T01:02:38.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:37 smithi069 ceph-mon[31727]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:37 smithi163 ceph-mon[67428]: osdmap e26: 4 total, 4 up, 4 in 2024-03-19T01:02:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:37 smithi163 ceph-mon[67428]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:02:39.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:38 smithi086 ceph-mon[66829]: osdmap e27: 4 total, 4 up, 4 in 2024-03-19T01:02:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:38 smithi086 ceph-mon[66829]: osdmap e28: 4 total, 4 up, 4 in 2024-03-19T01:02:39.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:38 smithi069 ceph-mon[31727]: osdmap e27: 4 total, 4 up, 4 in 2024-03-19T01:02:39.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:38 smithi069 ceph-mon[31727]: osdmap e28: 4 total, 4 up, 4 in 2024-03-19T01:02:39.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:38 smithi163 ceph-mon[67428]: osdmap e27: 4 total, 4 up, 4 in 2024-03-19T01:02:39.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:38 smithi163 ceph-mon[67428]: osdmap e28: 4 total, 4 up, 4 in 2024-03-19T01:02:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.877+0000 7fe8da5a1700 1 -- 172.21.15.86:0/1714484862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102340 msgr2=0x7fe8d4102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.877+0000 7fe8da5a1700 1 --2- 172.21.15.86:0/1714484862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102340 0x7fe8d4102760 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40099c0 tx=0x7fe8c4009cd0 comp rx=0 tx=0).stop 2024-03-19T01:02:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.877+0000 7fe8da5a1700 1 -- 172.21.15.86:0/1714484862 shutdown_connections 2024-03-19T01:02:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.877+0000 7fe8da5a1700 1 --2- 172.21.15.86:0/1714484862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d4103ee0 0x7fe8d410a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.877+0000 7fe8da5a1700 1 --2- 172.21.15.86:0/1714484862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102340 0x7fe8d4102760 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.877+0000 7fe8da5a1700 1 --2- 172.21.15.86:0/1714484862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4103540 0x7fe8d41039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.877+0000 7fe8da5a1700 1 -- 172.21.15.86:0/1714484862 >> 172.21.15.86:0/1714484862 conn(0x7fe8d40fd920 msgr2=0x7fe8d40ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:02:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.877+0000 7fe8da5a1700 1 -- 172.21.15.86:0/1714484862 shutdown_connections 2024-03-19T01:02:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.877+0000 7fe8da5a1700 1 -- 172.21.15.86:0/1714484862 wait complete. 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.877+0000 7fe8da5a1700 1 Processor -- start 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.877+0000 7fe8da5a1700 1 -- start start 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8da5a1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102340 0x7fe8d40760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8da5a1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d4103540 0x7fe8d4074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8da5a1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4103ee0 0x7fe8d4074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8da5a1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d410dfd0 con 0x7fe8d4103ee0 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8da5a1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d410de50 con 0x7fe8d4102340 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8da5a1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d410e150 con 0x7fe8d4103540 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d3fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102340 0x7fe8d40760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d3fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102340 0x7fe8d40760b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39550/0 (socket says 172.21.15.86:39550) 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d3fff700 1 -- 172.21.15.86:0/2086371793 learned_addr learned my addr 172.21.15.86:0/2086371793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d37fe700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d4103540 0x7fe8d4074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d3fff700 1 -- 172.21.15.86:0/2086371793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d4103540 msgr2=0x7fe8d4074700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d3fff700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d4103540 0x7fe8d4074700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d3fff700 1 -- 172.21.15.86:0/2086371793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4103ee0 msgr2=0x7fe8d4074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d3fff700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4103ee0 0x7fe8d4074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d3fff700 1 -- 172.21.15.86:0/2086371793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d40754f0 con 0x7fe8d4102340 2024-03-19T01:02:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d3fff700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102340 0x7fe8d40760b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098e0 tx=0x7fe8c40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:02:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d17fa700 1 -- 172.21.15.86:0/2086371793 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4026070 con 0x7fe8d4102340 2024-03-19T01:02:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d17fa700 1 -- 172.21.15.86:0/2086371793 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c4004210 con 0x7fe8d4102340 2024-03-19T01:02:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8d17fa700 1 -- 172.21.15.86:0/2086371793 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4021410 con 0x7fe8d4102340 2024-03-19T01:02:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8da5a1700 1 -- 172.21.15.86:0/2086371793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d41a2760 con 0x7fe8d4102340 2024-03-19T01:02:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.878+0000 7fe8da5a1700 1 -- 172.21.15.86:0/2086371793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d41a2be0 con 0x7fe8d4102340 2024-03-19T01:02:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.879+0000 7fe8da5a1700 1 -- 172.21.15.86:0/2086371793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d4068fd0 con 0x7fe8d4102340 2024-03-19T01:02:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.880+0000 7fe8d17fa700 1 -- 172.21.15.86:0/2086371793 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe8c4029070 con 0x7fe8d4102340 2024-03-19T01:02:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.881+0000 7fe8d17fa700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe8bc0a2910 0x7fe8bc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:02:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.881+0000 7fe8d17fa700 1 -- 172.21.15.86:0/2086371793 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7fe8c40ce500 con 0x7fe8d4102340 2024-03-19T01:02:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.881+0000 7fe8d37fe700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe8bc0a2910 0x7fe8bc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:02:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.882+0000 7fe8d37fe700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe8bc0a2910 0x7fe8bc0a4dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009a10 tx=0x7fe8c8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:02:39.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:39.885+0000 7fe8d17fa700 1 -- 172.21.15.86:0/2086371793 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe8c405d740 con 0x7fe8d4102340 2024-03-19T01:02:40.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:40.090+0000 7fe8da5a1700 1 -- 172.21.15.86:0/2086371793 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe8d4062750 con 0x7fe8bc0a2910 2024-03-19T01:02:40.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:40.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: osdmap e29: 4 total, 4 up, 4 in 2024-03-19T01:02:40.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:02:40.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:02:40.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:40.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:40.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:40.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:02:40.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:02:40.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:40.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:40.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:40.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:41.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:41.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: osdmap e29: 4 total, 4 up, 4 in 2024-03-19T01:02:41.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:02:41.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:02:41.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:41.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:41.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:41.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:02:41.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:02:41.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:41.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:41.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:41.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: osdmap e29: 4 total, 4 up, 4 in 2024-03-19T01:02:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:02:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:02:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:02:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:02:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:41.916 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:41 smithi086 ceph-mon[66829]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:02:41.916 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:41 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi069 2024-03-19T01:02:41.916 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:41 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi069 to 8545M 2024-03-19T01:02:42.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:41 smithi069 ceph-mon[31727]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:02:42.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:41 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi069 2024-03-19T01:02:42.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:41 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi069 to 8545M 2024-03-19T01:02:42.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:41 smithi163 ceph-mon[67428]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:02:42.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:41 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi069 2024-03-19T01:02:42.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:41 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi069 to 8545M 2024-03-19T01:02:43.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:42 smithi086 ceph-mon[66829]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:43.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:42 smithi069 ceph-mon[31727]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:43.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:42 smithi163 ceph-mon[67428]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:44.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:43 smithi086 ceph-mon[66829]: from='client.? 172.21.15.86:0/999962855' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cc3d6e9-ff8f-4cf1-b245-83311dc7564c"}]: dispatch 2024-03-19T01:02:44.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:43 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cc3d6e9-ff8f-4cf1-b245-83311dc7564c"}]: dispatch 2024-03-19T01:02:44.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:43 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cc3d6e9-ff8f-4cf1-b245-83311dc7564c"}]': finished 2024-03-19T01:02:44.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:43 smithi086 ceph-mon[66829]: osdmap e30: 5 total, 4 up, 5 in 2024-03-19T01:02:44.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:43 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:02:44.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:43 smithi086 ceph-mon[66829]: from='client.? 172.21.15.86:0/984600226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:02:44.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:43 smithi069 ceph-mon[31727]: from='client.? 172.21.15.86:0/999962855' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cc3d6e9-ff8f-4cf1-b245-83311dc7564c"}]: dispatch 2024-03-19T01:02:44.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:43 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cc3d6e9-ff8f-4cf1-b245-83311dc7564c"}]: dispatch 2024-03-19T01:02:44.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:43 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cc3d6e9-ff8f-4cf1-b245-83311dc7564c"}]': finished 2024-03-19T01:02:44.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:43 smithi069 ceph-mon[31727]: osdmap e30: 5 total, 4 up, 5 in 2024-03-19T01:02:44.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:43 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:02:44.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:43 smithi069 ceph-mon[31727]: from='client.? 172.21.15.86:0/984600226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:02:44.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:43 smithi163 ceph-mon[67428]: from='client.? 172.21.15.86:0/999962855' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cc3d6e9-ff8f-4cf1-b245-83311dc7564c"}]: dispatch 2024-03-19T01:02:44.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:43 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cc3d6e9-ff8f-4cf1-b245-83311dc7564c"}]: dispatch 2024-03-19T01:02:44.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:43 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cc3d6e9-ff8f-4cf1-b245-83311dc7564c"}]': finished 2024-03-19T01:02:44.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:43 smithi163 ceph-mon[67428]: osdmap e30: 5 total, 4 up, 5 in 2024-03-19T01:02:44.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:43 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:02:44.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:43 smithi163 ceph-mon[67428]: from='client.? 172.21.15.86:0/984600226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:02:45.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:44 smithi086 ceph-mon[66829]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-19T01:02:45.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:44 smithi069 ceph-mon[31727]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-19T01:02:45.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:44 smithi163 ceph-mon[67428]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-19T01:02:46.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:46 smithi086 ceph-mon[66829]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-19T01:02:46.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:46 smithi069 ceph-mon[31727]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-19T01:02:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:46 smithi163 ceph-mon[67428]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-19T01:02:48.986 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:48 smithi086 ceph-mon[66829]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:02:49.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:48 smithi069 ceph-mon[31727]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:02:49.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:48 smithi163 ceph-mon[67428]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:02:50.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:50 smithi086 ceph-mon[66829]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:02:50.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-19T01:02:50.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:50 smithi069 ceph-mon[31727]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:02:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-19T01:02:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:50 smithi163 ceph-mon[67428]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:02:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-19T01:02:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:51.973 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:51 smithi086 ceph-mon[66829]: Deploying daemon osd.4 on smithi086 2024-03-19T01:02:52.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:51 smithi069 ceph-mon[31727]: Deploying daemon osd.4 on smithi086 2024-03-19T01:02:52.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:51 smithi163 ceph-mon[67428]: Deploying daemon osd.4 on smithi086 2024-03-19T01:02:52.975 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:52 smithi086 ceph-mon[66829]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:02:53.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:52 smithi069 ceph-mon[31727]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:02:53.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:52 smithi163 ceph-mon[67428]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:02:55.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:54 smithi086 ceph-mon[66829]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:55.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:54 smithi069 ceph-mon[31727]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:55.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:54 smithi163 ceph-mon[67428]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:56.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:56 smithi086 ceph-mon[66829]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:56 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:56 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:56 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:56 smithi069 ceph-mon[31727]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:56 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:56 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:56 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:57.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:56 smithi163 ceph-mon[67428]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:57.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:56 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:57.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:56 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:57.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:56 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:02:59.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:58 smithi086 ceph-mon[66829]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:02:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:58 smithi069 ceph-mon[31727]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:59.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:59.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:59.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:02:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:58 smithi163 ceph-mon[67428]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:02:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:02:59.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:02:59.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:02:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:02:59.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.578+0000 7fe8d17fa700 1 -- 172.21.15.86:0/2086371793 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe8d4062750 con 0x7fe8bc0a2910 2024-03-19T01:02:59.580 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 4 on host 'smithi086' 2024-03-19T01:02:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 -- 172.21.15.86:0/2086371793 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe8bc0a2910 msgr2=0x7fe8bc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe8bc0a2910 0x7fe8bc0a4dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009a10 tx=0x7fe8c8009190 comp rx=0 tx=0).stop 2024-03-19T01:02:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 -- 172.21.15.86:0/2086371793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102340 msgr2=0x7fe8d40760b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:02:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102340 0x7fe8d40760b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098e0 tx=0x7fe8c40047b0 comp rx=0 tx=0).stop 2024-03-19T01:02:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 -- 172.21.15.86:0/2086371793 shutdown_connections 2024-03-19T01:02:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe8bc0a2910 0x7fe8bc0a4dd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4103ee0 0x7fe8d4074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102340 0x7fe8d40760b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 --2- 172.21.15.86:0/2086371793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d4103540 0x7fe8d4074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:02:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 -- 172.21.15.86:0/2086371793 >> 172.21.15.86:0/2086371793 conn(0x7fe8d40fd920 msgr2=0x7fe8d4106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:02:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 -- 172.21.15.86:0/2086371793 shutdown_connections 2024-03-19T01:02:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:02:59.581+0000 7fe8da5a1700 1 -- 172.21.15.86:0/2086371793 wait complete. 2024-03-19T01:03:00.256 DEBUG:teuthology.orchestra.run.smithi086:osd.4> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.4.service 2024-03-19T01:03:00.259 INFO:tasks.cephadm:Deploying osd.5 on smithi163 with /dev/vg_nvme/lv_3... 2024-03-19T01:03:00.259 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-19T01:03:00.503 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:03:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:00 smithi163 ceph-mon[67428]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:03:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:00 smithi163 ceph-mon[67428]: from='osd.4 [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:03:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:00 smithi163 ceph-mon[67428]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:03:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:00 smithi086 ceph-mon[66829]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:03:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:00 smithi086 ceph-mon[66829]: from='osd.4 [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:03:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:00 smithi086 ceph-mon[66829]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:03:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:00 smithi069 ceph-mon[31727]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:03:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:00 smithi069 ceph-mon[31727]: from='osd.4 [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:03:00.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:00 smithi069 ceph-mon[31727]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:03:01.664 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:01 smithi086 ceph-mon[66829]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-19T01:03:01.664 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:01 smithi086 ceph-mon[66829]: from='osd.4 [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:03:01.664 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:01 smithi086 ceph-mon[66829]: osdmap e31: 5 total, 4 up, 5 in 2024-03-19T01:03:01.665 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:01 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:01.665 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:01 smithi086 ceph-mon[66829]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:03:01.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:01 smithi163 ceph-mon[67428]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-19T01:03:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:01 smithi163 ceph-mon[67428]: from='osd.4 [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:03:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:01 smithi163 ceph-mon[67428]: osdmap e31: 5 total, 4 up, 5 in 2024-03-19T01:03:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:01 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:01 smithi163 ceph-mon[67428]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:03:02.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:01 smithi069 ceph-mon[31727]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-19T01:03:02.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:01 smithi069 ceph-mon[31727]: from='osd.4 [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:03:02.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:01 smithi069 ceph-mon[31727]: osdmap e31: 5 total, 4 up, 5 in 2024-03-19T01:03:02.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:01 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:02.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:01 smithi069 ceph-mon[31727]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:03:02.256 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:03:01 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4[78002]: 2024-03-19T01:03:01.945+0000 7f2f96c81700 -1 osd.4 0 waiting for initial osdmap 2024-03-19T01:03:02.256 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:03:01 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4[78002]: 2024-03-19T01:03:01.953+0000 7f2f8ea6c700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:03:02.513 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-19T01:03:02.513 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-19T01:03:02.513 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2024-03-19T01:03:02.513 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2024-03-19T01:03:02.514 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113821 s, 921 MB/s 2024-03-19T01:03:02.514 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2024-03-19T01:03:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:02 smithi086 ceph-mon[66829]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:03:02.800 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:02 smithi086 ceph-mon[66829]: osdmap e32: 5 total, 4 up, 5 in 2024-03-19T01:03:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:02 smithi086 ceph-mon[66829]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:03:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:02 smithi086 ceph-mon[66829]: from='osd.4 ' entity='osd.4' 2024-03-19T01:03:02.801 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:02 smithi163 ceph-mon[67428]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:03:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:02 smithi163 ceph-mon[67428]: osdmap e32: 5 total, 4 up, 5 in 2024-03-19T01:03:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:02.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:02 smithi163 ceph-mon[67428]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:03:02.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:02 smithi163 ceph-mon[67428]: from='osd.4 ' entity='osd.4' 2024-03-19T01:03:02.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:03.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:02 smithi069 ceph-mon[31727]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:03:03.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:02 smithi069 ceph-mon[31727]: osdmap e32: 5 total, 4 up, 5 in 2024-03-19T01:03:03.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:03.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:02 smithi069 ceph-mon[31727]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:03:03.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:02 smithi069 ceph-mon[31727]: from='osd.4 ' entity='osd.4' 2024-03-19T01:03:03.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:03.188 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi163:vg_nvme/lv_3 2024-03-19T01:03:03.427 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:03:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: osd.4 [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684] boot 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: osdmap e33: 5 total, 5 up, 5 in 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:03:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:03 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:03:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:03:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: osd.4 [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684] boot 2024-03-19T01:03:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: osdmap e33: 5 total, 5 up, 5 in 2024-03-19T01:03:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:04.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:03:04.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:03 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:04.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: osd.4 [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684] boot 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: osdmap e33: 5 total, 5 up, 5 in 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:03:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:04.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:04 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi086 2024-03-19T01:03:04.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:04 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi086 to 8540M 2024-03-19T01:03:04.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:04 smithi163 ceph-mon[67428]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:04.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:04 smithi163 ceph-mon[67428]: osdmap e34: 5 total, 5 up, 5 in 2024-03-19T01:03:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:04 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi086 2024-03-19T01:03:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:04 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi086 to 8540M 2024-03-19T01:03:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:04 smithi086 ceph-mon[66829]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:04 smithi086 ceph-mon[66829]: osdmap e34: 5 total, 5 up, 5 in 2024-03-19T01:03:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:04 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi086 2024-03-19T01:03:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:04 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi086 to 8540M 2024-03-19T01:03:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:04 smithi069 ceph-mon[31727]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:04 smithi069 ceph-mon[31727]: osdmap e34: 5 total, 5 up, 5 in 2024-03-19T01:03:05.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 -- 172.21.15.163:0/3935047739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c4102440 msgr2=0x7fd4c41028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:03:05.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 --2- 172.21.15.163:0/3935047739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c4102440 0x7fd4c41028a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4009990 tx=0x7fd4b4009ca0 comp rx=0 tx=0).stop 2024-03-19T01:03:05.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 -- 172.21.15.163:0/3935047739 shutdown_connections 2024-03-19T01:03:05.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 --2- 172.21.15.163:0/3935047739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4102de0 0x7fd4c4109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 --2- 172.21.15.163:0/3935047739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c4101240 0x7fd4c4101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 --2- 172.21.15.163:0/3935047739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c4102440 0x7fd4c41028a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 -- 172.21.15.163:0/3935047739 >> 172.21.15.163:0/3935047739 conn(0x7fd4c40fc800 msgr2=0x7fd4c40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 -- 172.21.15.163:0/3935047739 shutdown_connections 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 -- 172.21.15.163:0/3935047739 wait complete. 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 Processor -- start 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 -- start start 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4101240 0x7fd4c406b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c4102440 0x7fd4c419d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c4102de0 0x7fd4c419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c410ce90 con 0x7fd4c4101240 2024-03-19T01:03:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c410cd10 con 0x7fd4c4102de0 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.089+0000 7fd4cb334700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c410d010 con 0x7fd4c4102440 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4c88cf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c4102440 0x7fd4c419d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4c88cf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c4102440 0x7fd4c419d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:45464/0 (socket says 172.21.15.163:45464) 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4c88cf700 1 -- 172.21.15.163:0/3559477196 learned_addr learned my addr 172.21.15.163:0/3559477196 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4c98d1700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c4102de0 0x7fd4c419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4c88cf700 1 -- 172.21.15.163:0/3559477196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c4102de0 msgr2=0x7fd4c419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4c88cf700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c4102de0 0x7fd4c419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4c88cf700 1 -- 172.21.15.163:0/3559477196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4101240 msgr2=0x7fd4c406b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4c88cf700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4101240 0x7fd4c406b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4c88cf700 1 -- 172.21.15.163:0/3559477196 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c41a0150 con 0x7fd4c4102440 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4c88cf700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c4102440 0x7fd4c419d240 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40098e0 tx=0x7fd4b4009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4ba7fc700 1 -- 172.21.15.163:0/3559477196 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4027070 con 0x7fd4c4102440 2024-03-19T01:03:05.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4ba7fc700 1 -- 172.21.15.163:0/3559477196 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b4004150 con 0x7fd4c4102440 2024-03-19T01:03:05.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4cb334700 1 -- 172.21.15.163:0/3559477196 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c41a03e0 con 0x7fd4c4102440 2024-03-19T01:03:05.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4ba7fc700 1 -- 172.21.15.163:0/3559477196 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b400f6c0 con 0x7fd4c4102440 2024-03-19T01:03:05.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.090+0000 7fd4cb334700 1 -- 172.21.15.163:0/3559477196 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c41a0910 con 0x7fd4c4102440 2024-03-19T01:03:05.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.091+0000 7fd4cb334700 1 -- 172.21.15.163:0/3559477196 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c4068fd0 con 0x7fd4c4102440 2024-03-19T01:03:05.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.092+0000 7fd4ba7fc700 1 -- 172.21.15.163:0/3559477196 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd4b402a070 con 0x7fd4c4102440 2024-03-19T01:03:05.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.093+0000 7fd4ba7fc700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd4b00a2910 0x7fd4b00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:03:05.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.093+0000 7fd4ba7fc700 1 -- 172.21.15.163:0/3559477196 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fd4b40cf240 con 0x7fd4c4102440 2024-03-19T01:03:05.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.093+0000 7fd4c90d0700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd4b00a2910 0x7fd4b00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:03:05.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.094+0000 7fd4c90d0700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd4b00a2910 0x7fd4b00a4dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4c41022a0 tx=0x7fd4c0005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:03:05.097 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.097+0000 7fd4ba7fc700 1 -- 172.21.15.163:0/3559477196 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd4b4031020 con 0x7fd4c4102440 2024-03-19T01:03:05.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:05.299+0000 7fd4cb334700 1 -- 172.21.15.163:0/3559477196 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd4c4062810 con 0x7fd4b00a2910 2024-03-19T01:03:05.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:03:05.860 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:03:05.860 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:05.928 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:03:05.928 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:03:05.928 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:06.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:03:06.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:03:06.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:06.909 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:06 smithi163 ceph-mon[67428]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:03:06.909 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:06 smithi163 ceph-mon[67428]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:07.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:06 smithi086 ceph-mon[66829]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:03:07.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:06 smithi086 ceph-mon[66829]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:06 smithi069 ceph-mon[31727]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:03:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:06 smithi069 ceph-mon[31727]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:08.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:08 smithi163 ceph-mon[67428]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:08.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:08 smithi163 ceph-mon[67428]: from='client.? 172.21.15.163:0/3052739333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7808c335-63a3-46f1-a05d-8bf41c048ef4"}]: dispatch 2024-03-19T01:03:08.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:08 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7808c335-63a3-46f1-a05d-8bf41c048ef4"}]: dispatch 2024-03-19T01:03:08.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:08 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7808c335-63a3-46f1-a05d-8bf41c048ef4"}]': finished 2024-03-19T01:03:08.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:08 smithi163 ceph-mon[67428]: osdmap e35: 6 total, 5 up, 6 in 2024-03-19T01:03:08.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:08 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:08 smithi086 ceph-mon[66829]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:08 smithi086 ceph-mon[66829]: from='client.? 172.21.15.163:0/3052739333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7808c335-63a3-46f1-a05d-8bf41c048ef4"}]: dispatch 2024-03-19T01:03:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:08 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7808c335-63a3-46f1-a05d-8bf41c048ef4"}]: dispatch 2024-03-19T01:03:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:08 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7808c335-63a3-46f1-a05d-8bf41c048ef4"}]': finished 2024-03-19T01:03:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:08 smithi086 ceph-mon[66829]: osdmap e35: 6 total, 5 up, 6 in 2024-03-19T01:03:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:08 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:08 smithi069 ceph-mon[31727]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:08 smithi069 ceph-mon[31727]: from='client.? 172.21.15.163:0/3052739333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7808c335-63a3-46f1-a05d-8bf41c048ef4"}]: dispatch 2024-03-19T01:03:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:08 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7808c335-63a3-46f1-a05d-8bf41c048ef4"}]: dispatch 2024-03-19T01:03:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:08 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7808c335-63a3-46f1-a05d-8bf41c048ef4"}]': finished 2024-03-19T01:03:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:08 smithi069 ceph-mon[31727]: osdmap e35: 6 total, 5 up, 6 in 2024-03-19T01:03:09.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:09 smithi163 ceph-mon[67428]: from='client.? 172.21.15.163:0/3583776756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:03:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:09 smithi086 ceph-mon[66829]: from='client.? 172.21.15.163:0/3583776756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:03:10.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:09 smithi069 ceph-mon[31727]: from='client.? 172.21.15.163:0/3583776756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:03:11.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:10 smithi086 ceph-mon[66829]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:11.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:10 smithi069 ceph-mon[31727]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:11.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:10 smithi163 ceph-mon[67428]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:12.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:12 smithi163 ceph-mon[67428]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:13.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:12 smithi086 ceph-mon[66829]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:13.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:12 smithi069 ceph-mon[31727]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:15.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:14 smithi086 ceph-mon[66829]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:15.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:14 smithi069 ceph-mon[31727]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:15.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:14 smithi163 ceph-mon[67428]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:15.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:15 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-19T01:03:15.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:15 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:16.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:15 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-19T01:03:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:15 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:16.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:15 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-19T01:03:16.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:15 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:17.001 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:16 smithi086 ceph-mon[66829]: Deploying daemon osd.5 on smithi163 2024-03-19T01:03:17.001 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:16 smithi086 ceph-mon[66829]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:16 smithi069 ceph-mon[31727]: Deploying daemon osd.5 on smithi163 2024-03-19T01:03:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:16 smithi069 ceph-mon[31727]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:17.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:16 smithi163 ceph-mon[67428]: Deploying daemon osd.5 on smithi163 2024-03-19T01:03:17.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:16 smithi163 ceph-mon[67428]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:18.875 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:18 smithi163 ceph-mon[67428]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:18 smithi086 ceph-mon[66829]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:18 smithi069 ceph-mon[31727]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:21.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:20 smithi086 ceph-mon[66829]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:20 smithi069 ceph-mon[31727]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:20 smithi163 ceph-mon[67428]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:22.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:21 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:22.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:21 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:22.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:21 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:22.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:21 smithi086 ceph-mon[66829]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:22.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:21 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:22.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:21 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:22.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:21 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:22.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:21 smithi069 ceph-mon[31727]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:22.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:21 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:22.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:21 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:22.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:21 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:22.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:21 smithi163 ceph-mon[67428]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:24.484 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 5 on host 'smithi163' 2024-03-19T01:03:24.484 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.482+0000 7fd4ba7fc700 1 -- 172.21.15.163:0/3559477196 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd4c4062810 con 0x7fd4b00a2910 2024-03-19T01:03:24.485 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.484+0000 7fd4cb334700 1 -- 172.21.15.163:0/3559477196 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd4b00a2910 msgr2=0x7fd4b00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:03:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.484+0000 7fd4cb334700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd4b00a2910 0x7fd4b00a4dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4c41022a0 tx=0x7fd4c0005d20 comp rx=0 tx=0).stop 2024-03-19T01:03:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.484+0000 7fd4cb334700 1 -- 172.21.15.163:0/3559477196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c4102440 msgr2=0x7fd4c419d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:03:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.484+0000 7fd4cb334700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c4102440 0x7fd4c419d240 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40098e0 tx=0x7fd4b4009ee0 comp rx=0 tx=0).stop 2024-03-19T01:03:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.484+0000 7fd4cb334700 1 -- 172.21.15.163:0/3559477196 shutdown_connections 2024-03-19T01:03:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.484+0000 7fd4cb334700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd4b00a2910 0x7fd4b00a4dd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.484+0000 7fd4cb334700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c4102de0 0x7fd4c419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.484+0000 7fd4cb334700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4101240 0x7fd4c406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.484+0000 7fd4cb334700 1 --2- 172.21.15.163:0/3559477196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4c4102440 0x7fd4c419d240 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.484+0000 7fd4cb334700 1 -- 172.21.15.163:0/3559477196 >> 172.21.15.163:0/3559477196 conn(0x7fd4c40fc800 msgr2=0x7fd4c41056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:03:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.485+0000 7fd4cb334700 1 -- 172.21.15.163:0/3559477196 shutdown_connections 2024-03-19T01:03:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:03:24.485+0000 7fd4cb334700 1 -- 172.21.15.163:0/3559477196 wait complete. 2024-03-19T01:03:24.767 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:24 smithi163 ceph-mon[67428]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:24.768 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:24.768 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:24.986 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:24 smithi069 ceph-mon[31727]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:24.986 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:24.986 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:25.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:24 smithi086 ceph-mon[66829]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:25.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:25.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:25.163 DEBUG:teuthology.orchestra.run.smithi163:osd.5> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.5.service 2024-03-19T01:03:25.166 INFO:tasks.cephadm:Deploying osd.6 on smithi069 with /dev/vg_nvme/lv_2... 2024-03-19T01:03:25.166 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-19T01:03:25.597 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:03:25.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:25 smithi069 ceph-mon[31727]: from='osd.5 [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:03:25.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:25 smithi069 ceph-mon[31727]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:03:25.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:25 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:25.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:25 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:26.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:25 smithi086 ceph-mon[66829]: from='osd.5 [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:03:26.066 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:25 smithi086 ceph-mon[66829]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:03:26.066 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:25 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:26.066 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:25 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:26.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:25 smithi163 ceph-mon[67428]: from='osd.5 [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:03:26.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:25 smithi163 ceph-mon[67428]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:03:26.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:25 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:26.067 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:25 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:26 smithi086 ceph-mon[66829]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:26 smithi086 ceph-mon[66829]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-19T01:03:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:26 smithi086 ceph-mon[66829]: from='osd.5 [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:03:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:26 smithi086 ceph-mon[66829]: osdmap e36: 6 total, 5 up, 6 in 2024-03-19T01:03:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:26 smithi086 ceph-mon[66829]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:03:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:26 smithi086 ceph-mon[66829]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:03:26.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:26 smithi069 ceph-mon[31727]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:26.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:26 smithi069 ceph-mon[31727]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-19T01:03:26.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:26 smithi069 ceph-mon[31727]: from='osd.5 [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:03:26.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:26 smithi069 ceph-mon[31727]: osdmap e36: 6 total, 5 up, 6 in 2024-03-19T01:03:26.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:26 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:26.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:26 smithi069 ceph-mon[31727]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:03:26.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:26 smithi069 ceph-mon[31727]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:03:27.019 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:26 smithi163 ceph-mon[67428]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:27.019 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:26 smithi163 ceph-mon[67428]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-19T01:03:27.019 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:26 smithi163 ceph-mon[67428]: from='osd.5 [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:03:27.019 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:26 smithi163 ceph-mon[67428]: osdmap e36: 6 total, 5 up, 6 in 2024-03-19T01:03:27.019 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:26 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:27.019 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:26 smithi163 ceph-mon[67428]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:03:27.019 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:26 smithi163 ceph-mon[67428]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:03:27.311 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:03:27 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:03:27.066+0000 7f6cd02c3700 -1 osd.5 0 waiting for initial osdmap 2024-03-19T01:03:27.312 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:03:27 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:03:27.075+0000 7f6cc80ae700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:03:27.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:27 smithi069 ceph-mon[31727]: osdmap e37: 6 total, 5 up, 6 in 2024-03-19T01:03:27.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:27 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:27.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:27 smithi069 ceph-mon[31727]: from='osd.5 ' entity='osd.5' 2024-03-19T01:03:28.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:27 smithi086 ceph-mon[66829]: osdmap e37: 6 total, 5 up, 6 in 2024-03-19T01:03:28.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:27 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:28.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:27 smithi086 ceph-mon[66829]: from='osd.5 ' entity='osd.5' 2024-03-19T01:03:28.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:27 smithi163 ceph-mon[67428]: osdmap e37: 6 total, 5 up, 6 in 2024-03-19T01:03:28.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:27 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:28.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:27 smithi163 ceph-mon[67428]: from='osd.5 ' entity='osd.5' 2024-03-19T01:03:28.427 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-19T01:03:28.427 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-19T01:03:28.427 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-03-19T01:03:28.428 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-03-19T01:03:28.428 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0343619 s, 305 MB/s 2024-03-19T01:03:28.428 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-03-19T01:03:28.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:28 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:03:28.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:28 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:03:28.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:28 smithi069 ceph-mon[31727]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:28 smithi069 ceph-mon[31727]: osd.5 [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076] boot 2024-03-19T01:03:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:28 smithi069 ceph-mon[31727]: osdmap e38: 6 total, 6 up, 6 in 2024-03-19T01:03:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:28 smithi069 ceph-mon[31727]: osdmap e39: 6 total, 6 up, 6 in 2024-03-19T01:03:28.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:28 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:03:28.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:28 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:03:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:28 smithi163 ceph-mon[67428]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:28 smithi163 ceph-mon[67428]: osd.5 [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076] boot 2024-03-19T01:03:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:28 smithi163 ceph-mon[67428]: osdmap e38: 6 total, 6 up, 6 in 2024-03-19T01:03:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:28 smithi163 ceph-mon[67428]: osdmap e39: 6 total, 6 up, 6 in 2024-03-19T01:03:29.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:28 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:03:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:28 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:03:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:28 smithi086 ceph-mon[66829]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:03:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:28 smithi086 ceph-mon[66829]: osd.5 [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076] boot 2024-03-19T01:03:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:28 smithi086 ceph-mon[66829]: osdmap e38: 6 total, 6 up, 6 in 2024-03-19T01:03:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:03:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:28 smithi086 ceph-mon[66829]: osdmap e39: 6 total, 6 up, 6 in 2024-03-19T01:03:29.441 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi069:vg_nvme/lv_2 2024-03-19T01:03:29.875 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:03:30.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi163 2024-03-19T01:03:30.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:30.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:30.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi163 to 8540M 2024-03-19T01:03:30.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:30.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:03:30.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:30.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:30.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:03:30.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:30.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:29 smithi069 ceph-mon[31727]: osdmap e40: 6 total, 6 up, 6 in 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi163 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi163 to 8540M 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:03:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:29 smithi086 ceph-mon[66829]: osdmap e40: 6 total, 6 up, 6 in 2024-03-19T01:03:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi163 2024-03-19T01:03:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:03:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi163 to 8540M 2024-03-19T01:03:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:03:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:03:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:29 smithi163 ceph-mon[67428]: osdmap e40: 6 total, 6 up, 6 in 2024-03-19T01:03:31.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:31 smithi069 ceph-mon[31727]: osdmap e41: 6 total, 6 up, 6 in 2024-03-19T01:03:32.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:31 smithi086 ceph-mon[66829]: osdmap e41: 6 total, 6 up, 6 in 2024-03-19T01:03:32.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:31 smithi163 ceph-mon[67428]: osdmap e41: 6 total, 6 up, 6 in 2024-03-19T01:03:32.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.121+0000 7f8074c00700 1 -- 172.21.15.69:0/3249445001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8070076cc0 msgr2=0x7f80700770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:03:32.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.121+0000 7f8074c00700 1 --2- 172.21.15.69:0/3249445001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8070076cc0 0x7f80700770e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8060009f90 tx=0x7f806000b330 comp rx=0 tx=0).stop 2024-03-19T01:03:32.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.121+0000 7f8074c00700 1 -- 172.21.15.69:0/3249445001 shutdown_connections 2024-03-19T01:03:32.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.121+0000 7f8074c00700 1 --2- 172.21.15.69:0/3249445001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80700788d0 0x7f8070079150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:32.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.121+0000 7f8074c00700 1 --2- 172.21.15.69:0/3249445001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8070076cc0 0x7f80700770e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:32.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.121+0000 7f8074c00700 1 --2- 172.21.15.69:0/3249445001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8070077f10 0x7f8070078390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:32.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.121+0000 7f8074c00700 1 -- 172.21.15.69:0/3249445001 >> 172.21.15.69:0/3249445001 conn(0x7f80700ffa00 msgr2=0x7f8070101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:03:32.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.121+0000 7f8074c00700 1 -- 172.21.15.69:0/3249445001 shutdown_connections 2024-03-19T01:03:32.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.121+0000 7f8074c00700 1 -- 172.21.15.69:0/3249445001 wait complete. 2024-03-19T01:03:32.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.122+0000 7f8074c00700 1 Processor -- start 2024-03-19T01:03:32.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.122+0000 7f8074c00700 1 -- start start 2024-03-19T01:03:32.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f8074c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8070076cc0 0x7f807019ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:03:32.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f8074c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8070077f10 0x7f807019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:03:32.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f8074c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80700788d0 0x7f80701a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:03:32.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f8074c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8070114480 con 0x7f80700788d0 2024-03-19T01:03:32.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f8074c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8070114300 con 0x7f8070077f10 2024-03-19T01:03:32.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f8074c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8070114600 con 0x7f8070076cc0 2024-03-19T01:03:32.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f806f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8070076cc0 0x7f807019ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:03:32.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f806fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80700788d0 0x7f80701a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:03:32.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f806f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8070076cc0 0x7f807019ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60322/0 (socket says 172.21.15.69:60322) 2024-03-19T01:03:32.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f806f400700 1 -- 172.21.15.69:0/3439335162 learned_addr learned my addr 172.21.15.69:0/3439335162 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:03:32.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f8066a00700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8070077f10 0x7f807019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:03:32.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f806f400700 1 -- 172.21.15.69:0/3439335162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8070077f10 msgr2=0x7f807019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:03:32.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f806f400700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8070077f10 0x7f807019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:32.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f806f400700 1 -- 172.21.15.69:0/3439335162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80700788d0 msgr2=0x7f80701a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:03:32.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f806f400700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80700788d0 0x7f80701a5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:32.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f806f400700 1 -- 172.21.15.69:0/3439335162 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80701a6680 con 0x7f8070076cc0 2024-03-19T01:03:32.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.123+0000 7f806fe00700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80700788d0 0x7f80701a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:03:32.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.124+0000 7f806f400700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8070076cc0 0x7f807019ea20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8060007d30 tx=0x7f8060007d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:03:32.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.124+0000 7f806cc00700 1 -- 172.21.15.69:0/3439335162 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806001f070 con 0x7f8070076cc0 2024-03-19T01:03:32.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.124+0000 7f806cc00700 1 -- 172.21.15.69:0/3439335162 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8060004e40 con 0x7f8070076cc0 2024-03-19T01:03:32.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.124+0000 7f806cc00700 1 -- 172.21.15.69:0/3439335162 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806001b750 con 0x7f8070076cc0 2024-03-19T01:03:32.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.124+0000 7f8074c00700 1 -- 172.21.15.69:0/3439335162 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80701a6910 con 0x7f8070076cc0 2024-03-19T01:03:32.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.124+0000 7f8074c00700 1 -- 172.21.15.69:0/3439335162 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80701a6d70 con 0x7f8070076cc0 2024-03-19T01:03:32.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.125+0000 7f8074c00700 1 -- 172.21.15.69:0/3439335162 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8070068fd0 con 0x7f8070076cc0 2024-03-19T01:03:32.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.127+0000 7f806cc00700 1 -- 172.21.15.69:0/3439335162 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8060022070 con 0x7f8070076cc0 2024-03-19T01:03:32.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.129+0000 7f806cc00700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80540a2890 0x7f80540a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:03:32.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.129+0000 7f806cc00700 1 -- 172.21.15.69:0/3439335162 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f806000f020 con 0x7f8070076cc0 2024-03-19T01:03:32.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.129+0000 7f8066a00700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80540a2890 0x7f80540a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:03:32.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.129+0000 7f8066a00700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80540a2890 0x7f80540a4d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8058009750 tx=0x7f8058009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:03:32.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.131+0000 7f806cc00700 1 -- 172.21.15.69:0/3439335162 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8060029020 con 0x7f8070076cc0 2024-03-19T01:03:32.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:32.367+0000 7f8074c00700 1 -- 172.21.15.69:0/3439335162 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8070062750 con 0x7f80540a2890 2024-03-19T01:03:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:32 smithi086 ceph-mon[66829]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:33.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:32 smithi086 ceph-mon[66829]: osdmap e42: 6 total, 6 up, 6 in 2024-03-19T01:03:33.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:32 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:03:33.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:32 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:03:33.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:32 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:33.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:32 smithi069 ceph-mon[31727]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:33.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:32 smithi069 ceph-mon[31727]: osdmap e42: 6 total, 6 up, 6 in 2024-03-19T01:03:33.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:03:33.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:03:33.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:32 smithi163 ceph-mon[67428]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:32 smithi163 ceph-mon[67428]: osdmap e42: 6 total, 6 up, 6 in 2024-03-19T01:03:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:03:33.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:03:33.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:34.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:33 smithi086 ceph-mon[66829]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:03:34.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:33 smithi069 ceph-mon[31727]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:03:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:33 smithi163 ceph-mon[67428]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:03:34.907 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:34 smithi069 ceph-mon[31727]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-19T01:03:35.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:34 smithi086 ceph-mon[66829]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-19T01:03:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:34 smithi163 ceph-mon[67428]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-19T01:03:36.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:36 smithi069 ceph-mon[31727]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T01:03:36.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:36 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/648312767' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92498308-b815-42a5-acbd-433c52723dd5"}]: dispatch 2024-03-19T01:03:36.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:36 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/648312767' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92498308-b815-42a5-acbd-433c52723dd5"}]': finished 2024-03-19T01:03:36.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:36 smithi069 ceph-mon[31727]: osdmap e43: 7 total, 6 up, 7 in 2024-03-19T01:03:36.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:36 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:03:37.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:36 smithi086 ceph-mon[66829]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T01:03:37.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:36 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/648312767' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92498308-b815-42a5-acbd-433c52723dd5"}]: dispatch 2024-03-19T01:03:37.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:36 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/648312767' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92498308-b815-42a5-acbd-433c52723dd5"}]': finished 2024-03-19T01:03:37.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:36 smithi086 ceph-mon[66829]: osdmap e43: 7 total, 6 up, 7 in 2024-03-19T01:03:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:36 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:03:37.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:36 smithi163 ceph-mon[67428]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T01:03:37.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:36 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/648312767' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92498308-b815-42a5-acbd-433c52723dd5"}]: dispatch 2024-03-19T01:03:37.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:36 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/648312767' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92498308-b815-42a5-acbd-433c52723dd5"}]': finished 2024-03-19T01:03:37.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:36 smithi163 ceph-mon[67428]: osdmap e43: 7 total, 6 up, 7 in 2024-03-19T01:03:37.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:36 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:03:37.962 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:37 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/4025705917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:03:38.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:37 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/4025705917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:03:38.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:37 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/4025705917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:03:39.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:38 smithi086 ceph-mon[66829]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-19T01:03:39.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:38 smithi069 ceph-mon[31727]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-19T01:03:39.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:38 smithi163 ceph-mon[67428]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-19T01:03:40.957 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:40 smithi069 ceph-mon[31727]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:03:41.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:40 smithi086 ceph-mon[66829]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:03:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:40 smithi163 ceph-mon[67428]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:03:42.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:42 smithi069 ceph-mon[31727]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:03:43.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:42 smithi086 ceph-mon[66829]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:03:43.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:42 smithi163 ceph-mon[67428]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:03:44.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:44 smithi069 ceph-mon[31727]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:45.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:44 smithi086 ceph-mon[66829]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:45.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:44 smithi163 ceph-mon[67428]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:47.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:46 smithi086 ceph-mon[66829]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:47.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-19T01:03:47.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:46 smithi069 ceph-mon[31727]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-19T01:03:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:46 smithi163 ceph-mon[67428]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-19T01:03:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:03:48.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:47 smithi086 ceph-mon[66829]: Deploying daemon osd.6 on smithi069 2024-03-19T01:03:48.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:47 smithi069 ceph-mon[31727]: Deploying daemon osd.6 on smithi069 2024-03-19T01:03:48.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:47 smithi163 ceph-mon[67428]: Deploying daemon osd.6 on smithi069 2024-03-19T01:03:48.981 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:48 smithi069 ceph-mon[31727]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:49.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:48 smithi086 ceph-mon[66829]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:49.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:48 smithi163 ceph-mon[67428]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:51.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:50 smithi086 ceph-mon[66829]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:50 smithi069 ceph-mon[31727]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:50 smithi163 ceph-mon[67428]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:53.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:52 smithi086 ceph-mon[66829]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:53.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:52 smithi069 ceph-mon[31727]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:53.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:52 smithi163 ceph-mon[67428]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:54.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:54 smithi069 ceph-mon[31727]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:54.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:54.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:54.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:55.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:54 smithi086 ceph-mon[66829]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:55.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:55.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:55.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:55.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:54 smithi163 ceph-mon[67428]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:55.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:55.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:55.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:03:57.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:56 smithi086 ceph-mon[66829]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:57.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:56 smithi069 ceph-mon[31727]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:57.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:56 smithi163 ceph-mon[67428]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:58 smithi086 ceph-mon[66829]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:03:58 smithi086 ceph-mon[66829]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-19T01:03:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:58 smithi069 ceph-mon[31727]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:59.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:59.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:03:58 smithi069 ceph-mon[31727]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-19T01:03:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:58 smithi163 ceph-mon[67428]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:03:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:03:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:03:58 smithi163 ceph-mon[67428]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-19T01:03:59.729 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 6 on host 'smithi069' 2024-03-19T01:03:59.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.727+0000 7f806cc00700 1 -- 172.21.15.69:0/3439335162 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8070062750 con 0x7f80540a2890 2024-03-19T01:03:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.731+0000 7f8074c00700 1 -- 172.21.15.69:0/3439335162 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80540a2890 msgr2=0x7f80540a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:03:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.731+0000 7f8074c00700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80540a2890 0x7f80540a4d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8058009750 tx=0x7f8058009310 comp rx=0 tx=0).stop 2024-03-19T01:03:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.731+0000 7f8074c00700 1 -- 172.21.15.69:0/3439335162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8070076cc0 msgr2=0x7f807019ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:03:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.731+0000 7f8074c00700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8070076cc0 0x7f807019ea20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8060007d30 tx=0x7f8060007d60 comp rx=0 tx=0).stop 2024-03-19T01:03:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.731+0000 7f8074c00700 1 -- 172.21.15.69:0/3439335162 shutdown_connections 2024-03-19T01:03:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.731+0000 7f8074c00700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80540a2890 0x7f80540a4d50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.731+0000 7f8074c00700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80700788d0 0x7f80701a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.731+0000 7f8074c00700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8070076cc0 0x7f807019ea20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.731+0000 7f8074c00700 1 --2- 172.21.15.69:0/3439335162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8070077f10 0x7f807019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:03:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.731+0000 7f8074c00700 1 -- 172.21.15.69:0/3439335162 >> 172.21.15.69:0/3439335162 conn(0x7f80700ffa00 msgr2=0x7f8070101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:03:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.731+0000 7f8074c00700 1 -- 172.21.15.69:0/3439335162 shutdown_connections 2024-03-19T01:03:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:03:59.732+0000 7f8074c00700 1 -- 172.21.15.69:0/3439335162 wait complete. 2024-03-19T01:04:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:00 smithi086 ceph-mon[66829]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-19T01:04:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:00 smithi086 ceph-mon[66829]: osdmap e44: 7 total, 6 up, 7 in 2024-03-19T01:04:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:00 smithi086 ceph-mon[66829]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:04:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:00 smithi086 ceph-mon[66829]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:04:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:00 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:00.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-mon[31727]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-19T01:04:00.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-mon[31727]: osdmap e44: 7 total, 6 up, 7 in 2024-03-19T01:04:00.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-mon[31727]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:04:00.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:00.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:00.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:00.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:00.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-mon[31727]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:04:00.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:00.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:00.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:00.556 DEBUG:teuthology.orchestra.run.smithi069:osd.6> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.6.service 2024-03-19T01:04:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:00 smithi163 ceph-mon[67428]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-19T01:04:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:00 smithi163 ceph-mon[67428]: osdmap e44: 7 total, 6 up, 7 in 2024-03-19T01:04:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:00 smithi163 ceph-mon[67428]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:04:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:00 smithi163 ceph-mon[67428]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:04:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:00 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:00.563 INFO:tasks.cephadm:Deploying osd.7 on smithi086 with /dev/vg_nvme/lv_2... 2024-03-19T01:04:00.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-19T01:04:00.808 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:04:01.135 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:04:00.817+0000 7fd44f000700 -1 osd.6 0 waiting for initial osdmap 2024-03-19T01:04:01.135 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:04:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:04:00.853+0000 7fd444e00700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:04:01.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:01 smithi086 ceph-mon[66829]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:04:01.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:01 smithi086 ceph-mon[66829]: osdmap e45: 7 total, 6 up, 7 in 2024-03-19T01:04:01.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:01 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:01.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:01 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:01.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:01 smithi086 ceph-mon[66829]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' 2024-03-19T01:04:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:01 smithi069 ceph-mon[31727]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:04:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:01 smithi069 ceph-mon[31727]: osdmap e45: 7 total, 6 up, 7 in 2024-03-19T01:04:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:01 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:01 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:01 smithi069 ceph-mon[31727]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' 2024-03-19T01:04:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:01 smithi163 ceph-mon[67428]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:04:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:01 smithi163 ceph-mon[67428]: osdmap e45: 7 total, 6 up, 7 in 2024-03-19T01:04:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:01 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:01 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:01 smithi163 ceph-mon[67428]: from='osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341]' entity='osd.6' 2024-03-19T01:04:02.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:02 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:04:02.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:02 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:04:02.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:02.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:02 smithi069 ceph-mon[31727]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:04:02.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:02 smithi069 ceph-mon[31727]: osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341] boot 2024-03-19T01:04:02.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:02 smithi069 ceph-mon[31727]: osdmap e46: 7 total, 7 up, 7 in 2024-03-19T01:04:02.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:02.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:02 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:04:02.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:02 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:04:02.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:02.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:02 smithi086 ceph-mon[66829]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:04:02.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:02 smithi086 ceph-mon[66829]: osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341] boot 2024-03-19T01:04:02.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:02 smithi086 ceph-mon[66829]: osdmap e46: 7 total, 7 up, 7 in 2024-03-19T01:04:02.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:02 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:04:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:02 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:04:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:02 smithi163 ceph-mon[67428]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:04:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:02 smithi163 ceph-mon[67428]: osd.6 [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341] boot 2024-03-19T01:04:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:02 smithi163 ceph-mon[67428]: osdmap e46: 7 total, 7 up, 7 in 2024-03-19T01:04:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:04:02.744 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-19T01:04:02.744 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-19T01:04:02.744 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-19T01:04:02.744 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-19T01:04:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112405 s, 933 MB/s 2024-03-19T01:04:02.745 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 2024-03-19T01:04:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-19T01:04:03.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-03-19T01:04:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:04:04.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:04 smithi086 ceph-mon[66829]: osdmap e47: 7 total, 7 up, 7 in 2024-03-19T01:04:04.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:04 smithi086 ceph-mon[66829]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:04 smithi069 ceph-mon[31727]: osdmap e47: 7 total, 7 up, 7 in 2024-03-19T01:04:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:04 smithi069 ceph-mon[31727]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:04 smithi163 ceph-mon[67428]: osdmap e47: 7 total, 7 up, 7 in 2024-03-19T01:04:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:04 smithi163 ceph-mon[67428]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:05.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.270+0000 7f418f7d4700 1 -- 172.21.15.86:0/3784902257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4188102440 msgr2=0x7f41881028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.270+0000 7f418f7d4700 1 --2- 172.21.15.86:0/3784902257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4188102440 0x7f41881028a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f41840099c0 tx=0x7f4184009cd0 comp rx=0 tx=0).stop 2024-03-19T01:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.270+0000 7f418f7d4700 1 -- 172.21.15.86:0/3784902257 shutdown_connections 2024-03-19T01:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.270+0000 7f418f7d4700 1 --2- 172.21.15.86:0/3784902257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4188102de0 0x7f4188109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.270+0000 7f418f7d4700 1 --2- 172.21.15.86:0/3784902257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4188101240 0x7f4188101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.270+0000 7f418f7d4700 1 --2- 172.21.15.86:0/3784902257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4188102440 0x7f41881028a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.270+0000 7f418f7d4700 1 -- 172.21.15.86:0/3784902257 >> 172.21.15.86:0/3784902257 conn(0x7f41880fc800 msgr2=0x7f41880fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.271+0000 7f418f7d4700 1 -- 172.21.15.86:0/3784902257 shutdown_connections 2024-03-19T01:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.271+0000 7f418f7d4700 1 -- 172.21.15.86:0/3784902257 wait complete. 2024-03-19T01:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.271+0000 7f418f7d4700 1 Processor -- start 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.271+0000 7f418f7d4700 1 -- start start 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.271+0000 7f418f7d4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4188101240 0x7f4188198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.271+0000 7f418f7d4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4188102440 0x7f41881989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.271+0000 7f418f7d4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4188102de0 0x7f418819fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.271+0000 7f418f7d4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f418810ce10 con 0x7f4188101240 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.271+0000 7f418f7d4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f418810cc90 con 0x7f4188102440 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.271+0000 7f418f7d4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f418810cf90 con 0x7f4188102de0 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418cd6f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4188102440 0x7f41881989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418cd6f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4188102440 0x7f41881989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41784/0 (socket says 172.21.15.86:41784) 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418cd6f700 1 -- 172.21.15.86:0/2336368164 learned_addr learned my addr 172.21.15.86:0/2336368164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418dd71700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4188102de0 0x7f418819fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418d570700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4188101240 0x7f4188198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418cd6f700 1 -- 172.21.15.86:0/2336368164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4188102de0 msgr2=0x7f418819fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418cd6f700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4188102de0 0x7f418819fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418cd6f700 1 -- 172.21.15.86:0/2336368164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4188101240 msgr2=0x7f4188198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418cd6f700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4188101240 0x7f4188198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418cd6f700 1 -- 172.21.15.86:0/2336368164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41881a00d0 con 0x7f4188102440 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418dd71700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4188102de0 0x7f418819fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418d570700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4188101240 0x7f4188198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418cd6f700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4188102440 0x7f41881989b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4184000c00 tx=0x7f4184005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f417e7fc700 1 -- 172.21.15.86:0/2336368164 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184027070 con 0x7f4188102440 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f417e7fc700 1 -- 172.21.15.86:0/2336368164 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41840041d0 con 0x7f4188102440 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418f7d4700 1 -- 172.21.15.86:0/2336368164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41881a0360 con 0x7f4188102440 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f417e7fc700 1 -- 172.21.15.86:0/2336368164 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418400f460 con 0x7f4188102440 2024-03-19T01:04:05.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.272+0000 7f418f7d4700 1 -- 172.21.15.86:0/2336368164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41881a07b0 con 0x7f4188102440 2024-03-19T01:04:05.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.273+0000 7f418f7d4700 1 -- 172.21.15.86:0/2336368164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4188068fd0 con 0x7f4188102440 2024-03-19T01:04:05.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.274+0000 7f417e7fc700 1 -- 172.21.15.86:0/2336368164 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f418402a070 con 0x7f4188102440 2024-03-19T01:04:05.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.275+0000 7f417e7fc700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f41740a2890 0x7f41740a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:05.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.275+0000 7f417e7fc700 1 -- 172.21.15.86:0/2336368164 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f41840cf200 con 0x7f4188102440 2024-03-19T01:04:05.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.275+0000 7f418d570700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f41740a2890 0x7f41740a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:04:05.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.276+0000 7f418d570700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f41740a2890 0x7f41740a4d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4178005fd0 tx=0x7f4178005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:04:05.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.279+0000 7f417e7fc700 1 -- 172.21.15.86:0/2336368164 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4184013070 con 0x7f4188102440 2024-03-19T01:04:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:05.487+0000 7f418f7d4700 1 -- 172.21.15.86:0/2336368164 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f41880fe8a0 con 0x7f41740a2890 2024-03-19T01:04:06.431 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi069 2024-03-19T01:04:06.431 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.431 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.431 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:06.431 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:06.431 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:06.431 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi069 to 5696M 2024-03-19T01:04:06.431 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.431 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:06.431 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:06.432 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.432 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:04:06.432 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:04:06.432 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:04:06.432 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:06.432 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:06 smithi086 ceph-mon[66829]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi069 2024-03-19T01:04:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi069 to 5696M 2024-03-19T01:04:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:04:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:04:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:04:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:06 smithi069 ceph-mon[31727]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi069 2024-03-19T01:04:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi069 to 5696M 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:04:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:06.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:06 smithi163 ceph-mon[67428]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:08 smithi086 ceph-mon[66829]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:08 smithi086 ceph-mon[66829]: from='client.? 172.21.15.86:0/2110297049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4bc368-4502-4d9d-898d-641c779f7511"}]: dispatch 2024-03-19T01:04:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:08 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4bc368-4502-4d9d-898d-641c779f7511"}]: dispatch 2024-03-19T01:04:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:08 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f4bc368-4502-4d9d-898d-641c779f7511"}]': finished 2024-03-19T01:04:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:08 smithi086 ceph-mon[66829]: osdmap e48: 8 total, 7 up, 8 in 2024-03-19T01:04:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:08 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:08 smithi069 ceph-mon[31727]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:08 smithi069 ceph-mon[31727]: from='client.? 172.21.15.86:0/2110297049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4bc368-4502-4d9d-898d-641c779f7511"}]: dispatch 2024-03-19T01:04:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:08 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4bc368-4502-4d9d-898d-641c779f7511"}]: dispatch 2024-03-19T01:04:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:08 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f4bc368-4502-4d9d-898d-641c779f7511"}]': finished 2024-03-19T01:04:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:08 smithi069 ceph-mon[31727]: osdmap e48: 8 total, 7 up, 8 in 2024-03-19T01:04:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:08 smithi163 ceph-mon[67428]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:08 smithi163 ceph-mon[67428]: from='client.? 172.21.15.86:0/2110297049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4bc368-4502-4d9d-898d-641c779f7511"}]: dispatch 2024-03-19T01:04:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:08 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4bc368-4502-4d9d-898d-641c779f7511"}]: dispatch 2024-03-19T01:04:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:08 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f4bc368-4502-4d9d-898d-641c779f7511"}]': finished 2024-03-19T01:04:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:08 smithi163 ceph-mon[67428]: osdmap e48: 8 total, 7 up, 8 in 2024-03-19T01:04:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:08 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:09 smithi086 ceph-mon[66829]: from='client.? 172.21.15.86:0/1673943287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:04:10.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:09 smithi069 ceph-mon[31727]: from='client.? 172.21.15.86:0/1673943287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:04:10.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:09 smithi163 ceph-mon[67428]: from='client.? 172.21.15.86:0/1673943287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:04:11.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:10 smithi086 ceph-mon[66829]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:11.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:10 smithi069 ceph-mon[31727]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:11.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:10 smithi163 ceph-mon[67428]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:12.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:12 smithi086 ceph-mon[66829]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:13.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:12 smithi069 ceph-mon[31727]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:13.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:12 smithi163 ceph-mon[67428]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:14.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:14 smithi086 ceph-mon[66829]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:15.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:14 smithi069 ceph-mon[31727]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:15.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:14 smithi163 ceph-mon[67428]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:15 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-19T01:04:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:15 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:16.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:15 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-19T01:04:16.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:15 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:16.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:15 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-19T01:04:16.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:15 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:17.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:16 smithi086 ceph-mon[66829]: Deploying daemon osd.7 on smithi086 2024-03-19T01:04:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:16 smithi086 ceph-mon[66829]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:16 smithi069 ceph-mon[31727]: Deploying daemon osd.7 on smithi086 2024-03-19T01:04:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:16 smithi069 ceph-mon[31727]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:17.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:16 smithi163 ceph-mon[67428]: Deploying daemon osd.7 on smithi086 2024-03-19T01:04:17.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:16 smithi163 ceph-mon[67428]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:18.941 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:18 smithi086 ceph-mon[66829]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:18 smithi069 ceph-mon[31727]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:18 smithi163 ceph-mon[67428]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:20.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:20 smithi086 ceph-mon[66829]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:20 smithi069 ceph-mon[31727]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:20 smithi163 ceph-mon[67428]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:22.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:22 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:22.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:22 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:22.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:22 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:22.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:22 smithi086 ceph-mon[66829]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:22 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:22 smithi069 ceph-mon[31727]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:22 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:22 smithi163 ceph-mon[67428]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:24.621 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 7 on host 'smithi086' 2024-03-19T01:04:24.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.620+0000 7f417e7fc700 1 -- 172.21.15.86:0/2336368164 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f41880fe8a0 con 0x7f41740a2890 2024-03-19T01:04:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 -- 172.21.15.86:0/2336368164 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f41740a2890 msgr2=0x7f41740a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f41740a2890 0x7f41740a4d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4178005fd0 tx=0x7f4178005e40 comp rx=0 tx=0).stop 2024-03-19T01:04:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 -- 172.21.15.86:0/2336368164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4188102440 msgr2=0x7f41881989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4188102440 0x7f41881989b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4184000c00 tx=0x7f4184005e80 comp rx=0 tx=0).stop 2024-03-19T01:04:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 -- 172.21.15.86:0/2336368164 shutdown_connections 2024-03-19T01:04:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f41740a2890 0x7f41740a4d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4188102de0 0x7f418819fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4188101240 0x7f4188198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 --2- 172.21.15.86:0/2336368164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4188102440 0x7f41881989b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 -- 172.21.15.86:0/2336368164 >> 172.21.15.86:0/2336368164 conn(0x7f41880fc800 msgr2=0x7f41880fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:04:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 -- 172.21.15.86:0/2336368164 shutdown_connections 2024-03-19T01:04:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:04:24.622+0000 7f418f7d4700 1 -- 172.21.15.86:0/2336368164 wait complete. 2024-03-19T01:04:24.937 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:24 smithi086 ceph-mon[66829]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:24.937 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:24.937 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:24.937 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:24 smithi069 ceph-mon[31727]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:24 smithi163 ceph-mon[67428]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:25.252 DEBUG:teuthology.orchestra.run.smithi086:osd.7> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.7.service 2024-03-19T01:04:25.254 INFO:tasks.cephadm:Deploying osd.8 on smithi163 with /dev/vg_nvme/lv_2... 2024-03-19T01:04:25.254 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-19T01:04:25.501 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:04:25.950 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:25 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:25.951 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:25 smithi163 ceph-mon[67428]: from='osd.7 [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:04:25.951 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:25 smithi163 ceph-mon[67428]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:04:26.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:25 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:26.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:25 smithi086 ceph-mon[66829]: from='osd.7 [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:04:26.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:25 smithi086 ceph-mon[66829]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:04:26.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:25 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:26.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:25 smithi069 ceph-mon[31727]: from='osd.7 [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:04:26.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:25 smithi069 ceph-mon[31727]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:04:26.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:26 smithi163 ceph-mon[67428]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:26.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:26 smithi163 ceph-mon[67428]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-19T01:04:26.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:26 smithi163 ceph-mon[67428]: from='osd.7 [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:04:26.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:26 smithi163 ceph-mon[67428]: osdmap e49: 8 total, 7 up, 8 in 2024-03-19T01:04:26.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:26 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:26.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:26 smithi163 ceph-mon[67428]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:04:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:26 smithi086 ceph-mon[66829]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:26 smithi086 ceph-mon[66829]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-19T01:04:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:26 smithi086 ceph-mon[66829]: from='osd.7 [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:04:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:26 smithi086 ceph-mon[66829]: osdmap e49: 8 total, 7 up, 8 in 2024-03-19T01:04:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:27.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:26 smithi086 ceph-mon[66829]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:04:27.007 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:04:26 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:04:26.666+0000 7f4d0c2ef700 -1 osd.7 0 waiting for initial osdmap 2024-03-19T01:04:27.007 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:04:26 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:04:26.678+0000 7f4d080e2700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:04:27.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:26 smithi069 ceph-mon[31727]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:27.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:26 smithi069 ceph-mon[31727]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-19T01:04:27.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:26 smithi069 ceph-mon[31727]: from='osd.7 [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:04:27.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:26 smithi069 ceph-mon[31727]: osdmap e49: 8 total, 7 up, 8 in 2024-03-19T01:04:27.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:26 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:27.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:26 smithi069 ceph-mon[31727]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:04:27.592 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-19T01:04:27.592 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-19T01:04:27.592 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2024-03-19T01:04:27.592 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2024-03-19T01:04:27.592 INFO:teuthology.orchestra.run.smithi163.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116215 s, 902 MB/s 2024-03-19T01:04:27.592 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2024-03-19T01:04:27.929 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:27 smithi086 ceph-mon[66829]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:04:27.929 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:27 smithi086 ceph-mon[66829]: osdmap e50: 8 total, 7 up, 8 in 2024-03-19T01:04:27.929 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:27 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:27.929 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:27 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:27.936 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:27 smithi163 ceph-mon[67428]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:04:27.936 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:27 smithi163 ceph-mon[67428]: osdmap e50: 8 total, 7 up, 8 in 2024-03-19T01:04:27.936 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:27 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:27.936 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:27 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:27 smithi069 ceph-mon[31727]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:04:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:27 smithi069 ceph-mon[31727]: osdmap e50: 8 total, 7 up, 8 in 2024-03-19T01:04:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:27 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:27 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:28.198 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi163:vg_nvme/lv_2 2024-03-19T01:04:28.443 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:04:28.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: osd.7 [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392] boot 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: osdmap e51: 8 total, 8 up, 8 in 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: osdmap e52: 8 total, 8 up, 8 in 2024-03-19T01:04:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:04:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:04:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: osd.7 [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392] boot 2024-03-19T01:04:29.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: osdmap e51: 8 total, 8 up, 8 in 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: osdmap e52: 8 total, 8 up, 8 in 2024-03-19T01:04:29.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: osd.7 [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392] boot 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: osdmap e51: 8 total, 8 up, 8 in 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:29.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:29.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:29.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:29.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: osdmap e52: 8 total, 8 up, 8 in 2024-03-19T01:04:29.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:29.945 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:29 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi086 2024-03-19T01:04:29.945 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:29 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi086 to 5693M 2024-03-19T01:04:29.945 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:29.945 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:29.945 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:29.945 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:29 smithi163 ceph-mon[67428]: osdmap e53: 8 total, 8 up, 8 in 2024-03-19T01:04:30.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:29 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi086 2024-03-19T01:04:30.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:29 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi086 to 5693M 2024-03-19T01:04:30.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:30.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:30.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:30.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:29 smithi086 ceph-mon[66829]: osdmap e53: 8 total, 8 up, 8 in 2024-03-19T01:04:30.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:29 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi086 2024-03-19T01:04:30.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:29 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi086 to 5693M 2024-03-19T01:04:30.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:30.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:30.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:30.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:29 smithi069 ceph-mon[31727]: osdmap e53: 8 total, 8 up, 8 in 2024-03-19T01:04:30.178 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.175+0000 7f03b8abb700 1 -- 172.21.15.163:0/3248223446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03b4102440 msgr2=0x7f03b41028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:30.179 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.175+0000 7f03b8abb700 1 --2- 172.21.15.163:0/3248223446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03b4102440 0x7f03b41028a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009990 tx=0x7f03a8009ca0 comp rx=0 tx=0).stop 2024-03-19T01:04:30.179 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.176+0000 7f03b8abb700 1 -- 172.21.15.163:0/3248223446 shutdown_connections 2024-03-19T01:04:30.179 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.176+0000 7f03b8abb700 1 --2- 172.21.15.163:0/3248223446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b4102de0 0x7f03b4109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:30.179 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.176+0000 7f03b8abb700 1 --2- 172.21.15.163:0/3248223446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03b4101240 0x7f03b4101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:30.179 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.176+0000 7f03b8abb700 1 --2- 172.21.15.163:0/3248223446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03b4102440 0x7f03b41028a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:30.179 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.176+0000 7f03b8abb700 1 -- 172.21.15.163:0/3248223446 >> 172.21.15.163:0/3248223446 conn(0x7f03b40fc800 msgr2=0x7f03b40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:04:30.179 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.176+0000 7f03b8abb700 1 -- 172.21.15.163:0/3248223446 shutdown_connections 2024-03-19T01:04:30.179 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.176+0000 7f03b8abb700 1 -- 172.21.15.163:0/3248223446 wait complete. 2024-03-19T01:04:30.179 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.176+0000 7f03b8abb700 1 Processor -- start 2024-03-19T01:04:30.179 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.176+0000 7f03b8abb700 1 -- start start 2024-03-19T01:04:30.179 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.176+0000 7f03b8abb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03b4101240 0x7f03b406b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b8abb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b4102440 0x7f03b419d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b8abb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03b4102de0 0x7f03b419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b8abb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03b410ceb0 con 0x7f03b4102440 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b8abb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03b410cd30 con 0x7f03b4102de0 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b8abb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f03b410d030 con 0x7f03b4101240 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03b4101240 0x7f03b406b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03b4101240 0x7f03b406b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51658/0 (socket says 172.21.15.163:51658) 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b259c700 1 -- 172.21.15.163:0/2549178754 learned_addr learned my addr 172.21.15.163:0/2549178754 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b2d9d700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03b4102de0 0x7f03b419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b259c700 1 -- 172.21.15.163:0/2549178754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03b4102de0 msgr2=0x7f03b419f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b259c700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03b4102de0 0x7f03b419f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b259c700 1 -- 172.21.15.163:0/2549178754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b4102440 msgr2=0x7f03b419d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:30.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b259c700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b4102440 0x7f03b419d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:30.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b259c700 1 -- 172.21.15.163:0/2549178754 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b41a0200 con 0x7f03b4101240 2024-03-19T01:04:30.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b259c700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03b4101240 0x7f03b406b590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f03a400d980 tx=0x7f03a400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:04:30.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03a37fe700 1 -- 172.21.15.163:0/2549178754 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4017070 con 0x7f03b4101240 2024-03-19T01:04:30.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03a37fe700 1 -- 172.21.15.163:0/2549178754 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03a4009e10 con 0x7f03b4101240 2024-03-19T01:04:30.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03a37fe700 1 -- 172.21.15.163:0/2549178754 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a400f620 con 0x7f03b4101240 2024-03-19T01:04:30.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b8abb700 1 -- 172.21.15.163:0/2549178754 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b41a04f0 con 0x7f03b4101240 2024-03-19T01:04:30.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.177+0000 7f03b8abb700 1 -- 172.21.15.163:0/2549178754 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b41a0ab0 con 0x7f03b4101240 2024-03-19T01:04:30.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.178+0000 7f03b8abb700 1 -- 172.21.15.163:0/2549178754 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b4068fd0 con 0x7f03b4101240 2024-03-19T01:04:30.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.180+0000 7f03a37fe700 1 -- 172.21.15.163:0/2549178754 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f03a4009930 con 0x7f03b4101240 2024-03-19T01:04:30.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.180+0000 7f03a37fe700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f039c0a2910 0x7f039c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:30.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.180+0000 7f03a37fe700 1 -- 172.21.15.163:0/2549178754 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f03a40cda30 con 0x7f03b4101240 2024-03-19T01:04:30.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.181+0000 7f03b1d9b700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f039c0a2910 0x7f039c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:04:30.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.181+0000 7f03b1d9b700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f039c0a2910 0x7f039c0a4dd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005f50 tx=0x7f03a8005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:04:30.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.184+0000 7f03a37fe700 1 -- 172.21.15.163:0/2549178754 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f03a409b690 con 0x7f03b4101240 2024-03-19T01:04:30.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:30.390+0000 7f03b8abb700 1 -- 172.21.15.163:0/2549178754 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f03b4062750 con 0x7f039c0a2910 2024-03-19T01:04:31.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:30 smithi086 ceph-mon[66829]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:31.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:30 smithi086 ceph-mon[66829]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-19T01:04:31.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:04:31.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:04:31.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:31.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:30 smithi163 ceph-mon[67428]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:31.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:30 smithi163 ceph-mon[67428]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-19T01:04:31.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:04:31.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:04:31.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:31.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:30 smithi069 ceph-mon[31727]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:31.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:30 smithi069 ceph-mon[31727]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-19T01:04:31.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:04:31.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:04:31.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:32.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:31 smithi086 ceph-mon[66829]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:04:32.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:31 smithi086 ceph-mon[66829]: osdmap e54: 8 total, 8 up, 8 in 2024-03-19T01:04:32.013 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:31 smithi163 ceph-mon[67428]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:04:32.013 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:31 smithi163 ceph-mon[67428]: osdmap e54: 8 total, 8 up, 8 in 2024-03-19T01:04:32.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:31 smithi069 ceph-mon[31727]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:04:32.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:31 smithi069 ceph-mon[31727]: osdmap e54: 8 total, 8 up, 8 in 2024-03-19T01:04:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:32 smithi086 ceph-mon[66829]: pgmap v198: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:33.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:32 smithi086 ceph-mon[66829]: osdmap e55: 8 total, 8 up, 8 in 2024-03-19T01:04:33.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:32 smithi069 ceph-mon[31727]: pgmap v198: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:33.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:32 smithi069 ceph-mon[31727]: osdmap e55: 8 total, 8 up, 8 in 2024-03-19T01:04:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:32 smithi163 ceph-mon[67428]: pgmap v198: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:32 smithi163 ceph-mon[67428]: osdmap e55: 8 total, 8 up, 8 in 2024-03-19T01:04:34.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:33 smithi086 ceph-mon[66829]: from='client.? 172.21.15.163:0/2462944440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81553561-de57-4fb8-894b-d4c63532dc8d"}]: dispatch 2024-03-19T01:04:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:33 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81553561-de57-4fb8-894b-d4c63532dc8d"}]: dispatch 2024-03-19T01:04:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:33 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81553561-de57-4fb8-894b-d4c63532dc8d"}]': finished 2024-03-19T01:04:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:33 smithi086 ceph-mon[66829]: osdmap e56: 9 total, 8 up, 9 in 2024-03-19T01:04:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:34.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:33 smithi069 ceph-mon[31727]: from='client.? 172.21.15.163:0/2462944440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81553561-de57-4fb8-894b-d4c63532dc8d"}]: dispatch 2024-03-19T01:04:34.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:33 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81553561-de57-4fb8-894b-d4c63532dc8d"}]: dispatch 2024-03-19T01:04:34.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:33 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81553561-de57-4fb8-894b-d4c63532dc8d"}]': finished 2024-03-19T01:04:34.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:33 smithi069 ceph-mon[31727]: osdmap e56: 9 total, 8 up, 9 in 2024-03-19T01:04:34.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:33 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:33 smithi163 ceph-mon[67428]: from='client.? 172.21.15.163:0/2462944440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81553561-de57-4fb8-894b-d4c63532dc8d"}]: dispatch 2024-03-19T01:04:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:33 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81553561-de57-4fb8-894b-d4c63532dc8d"}]: dispatch 2024-03-19T01:04:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:33 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81553561-de57-4fb8-894b-d4c63532dc8d"}]': finished 2024-03-19T01:04:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:33 smithi163 ceph-mon[67428]: osdmap e56: 9 total, 8 up, 9 in 2024-03-19T01:04:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:33 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:34 smithi086 ceph-mon[66829]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:34 smithi086 ceph-mon[66829]: from='client.? 172.21.15.163:0/3387872875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:04:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:34 smithi086 ceph-mon[66829]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-19T01:04:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:34 smithi086 ceph-mon[66829]: Cluster is now healthy 2024-03-19T01:04:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:34 smithi069 ceph-mon[31727]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:34 smithi069 ceph-mon[31727]: from='client.? 172.21.15.163:0/3387872875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:04:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:34 smithi069 ceph-mon[31727]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-19T01:04:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:34 smithi069 ceph-mon[31727]: Cluster is now healthy 2024-03-19T01:04:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:34 smithi163 ceph-mon[67428]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:34 smithi163 ceph-mon[67428]: from='client.? 172.21.15.163:0/3387872875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:04:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:34 smithi163 ceph-mon[67428]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-19T01:04:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:34 smithi163 ceph-mon[67428]: Cluster is now healthy 2024-03-19T01:04:37.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:36 smithi086 ceph-mon[66829]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:37.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:36 smithi069 ceph-mon[31727]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:37.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:36 smithi163 ceph-mon[67428]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:39.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:38 smithi086 ceph-mon[66829]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:39.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:38 smithi069 ceph-mon[31727]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:39.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:38 smithi163 ceph-mon[67428]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:41.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:40 smithi086 ceph-mon[66829]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-19T01:04:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:40 smithi069 ceph-mon[31727]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-19T01:04:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:40 smithi163 ceph-mon[67428]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-19T01:04:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:42.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:41 smithi086 ceph-mon[66829]: Deploying daemon osd.8 on smithi163 2024-03-19T01:04:42.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:41 smithi069 ceph-mon[31727]: Deploying daemon osd.8 on smithi163 2024-03-19T01:04:42.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:41 smithi163 ceph-mon[67428]: Deploying daemon osd.8 on smithi163 2024-03-19T01:04:43.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:42 smithi086 ceph-mon[66829]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:43.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:42 smithi069 ceph-mon[31727]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:43.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:42 smithi163 ceph-mon[67428]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:45.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:44 smithi069 ceph-mon[31727]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:45.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:44 smithi163 ceph-mon[67428]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:45.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:44 smithi086 ceph-mon[66829]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:46.788 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:46 smithi163 ceph-mon[67428]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:46.789 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:46.789 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:46.789 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:47.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:46 smithi086 ceph-mon[66829]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:47.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:47.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:47.013 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:46 smithi069 ceph-mon[31727]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:49.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:48 smithi069 ceph-mon[31727]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:49.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:48 smithi163 ceph-mon[67428]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:49.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:48 smithi086 ceph-mon[66829]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:50.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.027+0000 7f03a37fe700 1 -- 172.21.15.163:0/2549178754 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f03b4062750 con 0x7f039c0a2910 2024-03-19T01:04:50.029 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 8 on host 'smithi163' 2024-03-19T01:04:50.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 -- 172.21.15.163:0/2549178754 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f039c0a2910 msgr2=0x7f039c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:50.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f039c0a2910 0x7f039c0a4dd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f03a8005f50 tx=0x7f03a8005e80 comp rx=0 tx=0).stop 2024-03-19T01:04:50.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 -- 172.21.15.163:0/2549178754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03b4101240 msgr2=0x7f03b406b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:50.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03b4101240 0x7f03b406b590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f03a400d980 tx=0x7f03a400dc90 comp rx=0 tx=0).stop 2024-03-19T01:04:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 -- 172.21.15.163:0/2549178754 shutdown_connections 2024-03-19T01:04:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f039c0a2910 0x7f039c0a4dd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03b4102de0 0x7f03b419f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03b4101240 0x7f03b406b590 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 --2- 172.21.15.163:0/2549178754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03b4102440 0x7f03b419d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 -- 172.21.15.163:0/2549178754 >> 172.21.15.163:0/2549178754 conn(0x7f03b40fc800 msgr2=0x7f03b4105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:04:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 -- 172.21.15.163:0/2549178754 shutdown_connections 2024-03-19T01:04:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:04:50.029+0000 7f03b8abb700 1 -- 172.21.15.163:0/2549178754 wait complete. 2024-03-19T01:04:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:50 smithi086 ceph-mon[66829]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:50 smithi086 ceph-mon[66829]: from='osd.8 [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:04:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:50 smithi086 ceph-mon[66829]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:04:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:50 smithi069 ceph-mon[31727]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:50 smithi069 ceph-mon[31727]: from='osd.8 [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:04:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:50 smithi069 ceph-mon[31727]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:04:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:50 smithi163 ceph-mon[67428]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:04:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:50 smithi163 ceph-mon[67428]: from='osd.8 [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:04:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:50 smithi163 ceph-mon[67428]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:04:50.644 DEBUG:teuthology.orchestra.run.smithi163:osd.8> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.8.service 2024-03-19T01:04:50.646 INFO:tasks.cephadm:Deploying osd.9 on smithi069 with /dev/vg_nvme/lv_1... 2024-03-19T01:04:50.646 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-19T01:04:51.097 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:04:52.142 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:51 smithi163 ceph-mon[67428]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-19T01:04:52.143 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:51 smithi163 ceph-mon[67428]: from='osd.8 [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:04:52.143 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:51 smithi163 ceph-mon[67428]: osdmap e57: 9 total, 8 up, 9 in 2024-03-19T01:04:52.143 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:51 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:52.143 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:51 smithi163 ceph-mon[67428]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:04:52.143 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:51 smithi163 ceph-mon[67428]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:51 smithi086 ceph-mon[66829]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-19T01:04:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:51 smithi086 ceph-mon[66829]: from='osd.8 [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:04:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:51 smithi086 ceph-mon[66829]: osdmap e57: 9 total, 8 up, 9 in 2024-03-19T01:04:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:51 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:51 smithi086 ceph-mon[66829]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:04:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:51 smithi086 ceph-mon[66829]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:51 smithi069 ceph-mon[31727]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-19T01:04:52.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:51 smithi069 ceph-mon[31727]: from='osd.8 [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:04:52.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:51 smithi069 ceph-mon[31727]: osdmap e57: 9 total, 8 up, 9 in 2024-03-19T01:04:52.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:51 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:52.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:51 smithi069 ceph-mon[31727]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:04:52.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:51 smithi069 ceph-mon[31727]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:04:52.811 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:04:52 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:04:52.473+0000 7f08fe5f9700 -1 osd.8 0 waiting for initial osdmap 2024-03-19T01:04:52.811 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:04:52 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:04:52.482+0000 7f08f63e4700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:04:53.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:53 smithi069 ceph-mon[31727]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:04:53.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:53 smithi069 ceph-mon[31727]: osdmap e58: 9 total, 8 up, 9 in 2024-03-19T01:04:53.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:53 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:53.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:53 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:53.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:53 smithi069 ceph-mon[31727]: from='osd.8 ' entity='osd.8' 2024-03-19T01:04:53.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:53 smithi086 ceph-mon[66829]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:04:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:53 smithi086 ceph-mon[66829]: osdmap e58: 9 total, 8 up, 9 in 2024-03-19T01:04:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:53 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:53 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:53 smithi086 ceph-mon[66829]: from='osd.8 ' entity='osd.8' 2024-03-19T01:04:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:53 smithi163 ceph-mon[67428]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:04:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:53 smithi163 ceph-mon[67428]: osdmap e58: 9 total, 8 up, 9 in 2024-03-19T01:04:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:53 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:53 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:53 smithi163 ceph-mon[67428]: from='osd.8 ' entity='osd.8' 2024-03-19T01:04:54.012 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-19T01:04:54.012 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-19T01:04:54.013 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-03-19T01:04:54.013 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-03-19T01:04:54.013 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0381872 s, 275 MB/s 2024-03-19T01:04:54.013 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-03-19T01:04:54.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:04:54.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:04:54.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: osd.8 [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778] boot 2024-03-19T01:04:54.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: osdmap e59: 9 total, 9 up, 9 in 2024-03-19T01:04:54.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:54.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:04:54.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: osdmap e60: 9 total, 9 up, 9 in 2024-03-19T01:04:54.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi163 2024-03-19T01:04:54.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:54.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:54.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:54.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi163 to 5693M 2024-03-19T01:04:54.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:54.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:54.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:04:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:04:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: osd.8 [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778] boot 2024-03-19T01:04:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: osdmap e59: 9 total, 9 up, 9 in 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: osdmap e60: 9 total, 9 up, 9 in 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi163 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi163 to 5693M 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:04:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:04:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: osd.8 [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778] boot 2024-03-19T01:04:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: osdmap e59: 9 total, 9 up, 9 in 2024-03-19T01:04:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:04:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:04:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: osdmap e60: 9 total, 9 up, 9 in 2024-03-19T01:04:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi163 2024-03-19T01:04:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:04:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi163 to 5693M 2024-03-19T01:04:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:04:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:04:54.964 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi069:vg_nvme/lv_1 2024-03-19T01:04:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:04:55.891 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:55 smithi069 ceph-mon[31727]: osdmap e61: 9 total, 9 up, 9 in 2024-03-19T01:04:56.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:55 smithi086 ceph-mon[66829]: osdmap e61: 9 total, 9 up, 9 in 2024-03-19T01:04:56.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:55 smithi163 ceph-mon[67428]: osdmap e61: 9 total, 9 up, 9 in 2024-03-19T01:04:56.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:56 smithi069 ceph-mon[31727]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:04:56.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:56 smithi069 ceph-mon[31727]: osdmap e62: 9 total, 9 up, 9 in 2024-03-19T01:04:57.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:56 smithi086 ceph-mon[66829]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:04:57.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:56 smithi086 ceph-mon[66829]: osdmap e62: 9 total, 9 up, 9 in 2024-03-19T01:04:57.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:56 smithi163 ceph-mon[67428]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:04:57.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:56 smithi163 ceph-mon[67428]: osdmap e62: 9 total, 9 up, 9 in 2024-03-19T01:04:57.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.616+0000 7fdd41e00700 1 -- 172.21.15.69:0/3292145698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd3c109b80 msgr2=0x7fdd3c109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:57.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.616+0000 7fdd41e00700 1 --2- 172.21.15.69:0/3292145698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd3c109b80 0x7fdd3c109fe0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009990 tx=0x7fdd28009ca0 comp rx=0 tx=0).stop 2024-03-19T01:04:57.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.617+0000 7fdd41e00700 1 -- 172.21.15.69:0/3292145698 shutdown_connections 2024-03-19T01:04:57.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.617+0000 7fdd41e00700 1 --2- 172.21.15.69:0/3292145698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd3c10a520 0x7fdd3c110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:57.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.617+0000 7fdd41e00700 1 --2- 172.21.15.69:0/3292145698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd3c108980 0x7fdd3c108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:57.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.617+0000 7fdd41e00700 1 --2- 172.21.15.69:0/3292145698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd3c109b80 0x7fdd3c109fe0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:57.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.617+0000 7fdd41e00700 1 -- 172.21.15.69:0/3292145698 >> 172.21.15.69:0/3292145698 conn(0x7fdd3c077950 msgr2=0x7fdd3c079d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:04:57.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.617+0000 7fdd41e00700 1 -- 172.21.15.69:0/3292145698 shutdown_connections 2024-03-19T01:04:57.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.617+0000 7fdd41e00700 1 -- 172.21.15.69:0/3292145698 wait complete. 2024-03-19T01:04:57.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.619+0000 7fdd41e00700 1 Processor -- start 2024-03-19T01:04:57.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.619+0000 7fdd41e00700 1 -- start start 2024-03-19T01:04:57.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.619+0000 7fdd41e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd3c108980 0x7fdd3c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:57.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.619+0000 7fdd41e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd3c109b80 0x7fdd3c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:57.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.619+0000 7fdd41e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd3c10a520 0x7fdd3c1a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:57.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.619+0000 7fdd41e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c114570 con 0x7fdd3c109b80 2024-03-19T01:04:57.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.619+0000 7fdd41e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c1143f0 con 0x7fdd3c108980 2024-03-19T01:04:57.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.619+0000 7fdd41e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c1146f0 con 0x7fdd3c10a520 2024-03-19T01:04:57.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.620+0000 7fdd3aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd3c108980 0x7fdd3c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:04:57.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.620+0000 7fdd3aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd3c108980 0x7fdd3c19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42912/0 (socket says 172.21.15.69:42912) 2024-03-19T01:04:57.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.620+0000 7fdd3b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd3c10a520 0x7fdd3c1a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:04:57.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.620+0000 7fdd3aa00700 1 -- 172.21.15.69:0/3877152837 learned_addr learned my addr 172.21.15.69:0/3877152837 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:04:57.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.620+0000 7fdd3a000700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd3c109b80 0x7fdd3c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:04:57.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.620+0000 7fdd3aa00700 1 -- 172.21.15.69:0/3877152837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd3c10a520 msgr2=0x7fdd3c1a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:57.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.620+0000 7fdd3aa00700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd3c10a520 0x7fdd3c1a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:57.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.620+0000 7fdd3aa00700 1 -- 172.21.15.69:0/3877152837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd3c109b80 msgr2=0x7fdd3c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:04:57.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.620+0000 7fdd3aa00700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd3c109b80 0x7fdd3c19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:04:57.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.620+0000 7fdd3aa00700 1 -- 172.21.15.69:0/3877152837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c1a67e0 con 0x7fdd3c108980 2024-03-19T01:04:57.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.620+0000 7fdd3aa00700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd3c108980 0x7fdd3c19eb80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00e970 tx=0x7fdd2c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:04:57.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.621+0000 7fdd40a00700 1 -- 172.21.15.69:0/3877152837 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c004220 con 0x7fdd3c108980 2024-03-19T01:04:57.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.621+0000 7fdd41e00700 1 -- 172.21.15.69:0/3877152837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3c1a6ad0 con 0x7fdd3c108980 2024-03-19T01:04:57.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.621+0000 7fdd40a00700 1 -- 172.21.15.69:0/3877152837 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd2c0043c0 con 0x7fdd3c108980 2024-03-19T01:04:57.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.621+0000 7fdd40a00700 1 -- 172.21.15.69:0/3877152837 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c003df0 con 0x7fdd3c108980 2024-03-19T01:04:57.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.621+0000 7fdd41e00700 1 -- 172.21.15.69:0/3877152837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3c1a7010 con 0x7fdd3c108980 2024-03-19T01:04:57.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.622+0000 7fdd41e00700 1 -- 172.21.15.69:0/3877152837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3c068fd0 con 0x7fdd3c108980 2024-03-19T01:04:57.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.624+0000 7fdd40a00700 1 -- 172.21.15.69:0/3877152837 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fdd2c00c7e0 con 0x7fdd3c108980 2024-03-19T01:04:57.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.625+0000 7fdd40a00700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdd200a2890 0x7fdd200a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:04:57.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.625+0000 7fdd40a00700 1 -- 172.21.15.69:0/3877152837 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fdd2c0ce360 con 0x7fdd3c108980 2024-03-19T01:04:57.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.626+0000 7fdd3a000700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdd200a2890 0x7fdd200a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:04:57.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.626+0000 7fdd3a000700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdd200a2890 0x7fdd200a4d50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800b2e0 tx=0x7fdd28005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:04:57.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.628+0000 7fdd40a00700 1 -- 172.21.15.69:0/3877152837 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdd2c09bda0 con 0x7fdd3c108980 2024-03-19T01:04:57.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:04:57.860+0000 7fdd41e00700 1 -- 172.21.15.69:0/3877152837 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdd3c062750 con 0x7fdd200a2890 2024-03-19T01:04:57.862 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:57 smithi069 ceph-mon[31727]: osdmap e63: 9 total, 9 up, 9 in 2024-03-19T01:04:58.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:57 smithi086 ceph-mon[66829]: osdmap e63: 9 total, 9 up, 9 in 2024-03-19T01:04:58.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:57 smithi163 ceph-mon[67428]: osdmap e63: 9 total, 9 up, 9 in 2024-03-19T01:04:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:58 smithi086 ceph-mon[66829]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:04:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:04:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:04:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:58 smithi069 ceph-mon[31727]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:04:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:04:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:04:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:58 smithi163 ceph-mon[67428]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:04:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:04:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:04:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:04:59.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:04:59 smithi069 ceph-mon[31727]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:05:00.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:04:59 smithi086 ceph-mon[66829]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:05:00.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:04:59 smithi163 ceph-mon[67428]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:05:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:00 smithi069 ceph-mon[31727]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-19T01:05:01.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:00 smithi086 ceph-mon[66829]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-19T01:05:01.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:00 smithi163 ceph-mon[67428]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-19T01:05:02.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:01 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3827522940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83e5279d-25d1-4acb-8fe9-47f4b32ad1a3"}]: dispatch 2024-03-19T01:05:02.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:01 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83e5279d-25d1-4acb-8fe9-47f4b32ad1a3"}]: dispatch 2024-03-19T01:05:02.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:01 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83e5279d-25d1-4acb-8fe9-47f4b32ad1a3"}]': finished 2024-03-19T01:05:02.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:01 smithi086 ceph-mon[66829]: osdmap e64: 10 total, 9 up, 10 in 2024-03-19T01:05:02.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:01 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:02.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:01 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3827522940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83e5279d-25d1-4acb-8fe9-47f4b32ad1a3"}]: dispatch 2024-03-19T01:05:02.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:01 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83e5279d-25d1-4acb-8fe9-47f4b32ad1a3"}]: dispatch 2024-03-19T01:05:02.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:01 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83e5279d-25d1-4acb-8fe9-47f4b32ad1a3"}]': finished 2024-03-19T01:05:02.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:01 smithi069 ceph-mon[31727]: osdmap e64: 10 total, 9 up, 10 in 2024-03-19T01:05:02.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:01 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:02.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:01 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3827522940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83e5279d-25d1-4acb-8fe9-47f4b32ad1a3"}]: dispatch 2024-03-19T01:05:02.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:01 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83e5279d-25d1-4acb-8fe9-47f4b32ad1a3"}]: dispatch 2024-03-19T01:05:02.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:01 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83e5279d-25d1-4acb-8fe9-47f4b32ad1a3"}]': finished 2024-03-19T01:05:02.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:01 smithi163 ceph-mon[67428]: osdmap e64: 10 total, 9 up, 10 in 2024-03-19T01:05:02.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:01 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:03.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:02 smithi086 ceph-mon[66829]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-19T01:05:03.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:02 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/4220298062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:05:03.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:02 smithi069 ceph-mon[31727]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-19T01:05:03.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:02 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/4220298062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:05:03.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:02 smithi163 ceph-mon[67428]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-19T01:05:03.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:02 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/4220298062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:05:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:04 smithi069 ceph-mon[31727]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-19T01:05:05.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:04 smithi163 ceph-mon[67428]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-19T01:05:05.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:04 smithi086 ceph-mon[66829]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-19T01:05:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:06 smithi069 ceph-mon[31727]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-19T01:05:07.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:06 smithi163 ceph-mon[67428]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-19T01:05:07.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:06 smithi086 ceph-mon[66829]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-19T01:05:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:08 smithi069 ceph-mon[31727]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:05:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:08 smithi163 ceph-mon[67428]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:05:09.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:08 smithi086 ceph-mon[66829]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:05:11.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:10 smithi069 ceph-mon[31727]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:11.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:10 smithi163 ceph-mon[67428]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:11.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:10 smithi086 ceph-mon[66829]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:12.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-19T01:05:12.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:12.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-19T01:05:12.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-19T01:05:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:13.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:12 smithi069 ceph-mon[31727]: Deploying daemon osd.9 on smithi069 2024-03-19T01:05:13.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:12 smithi069 ceph-mon[31727]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:13.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:12 smithi163 ceph-mon[67428]: Deploying daemon osd.9 on smithi069 2024-03-19T01:05:13.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:12 smithi163 ceph-mon[67428]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:13.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:12 smithi086 ceph-mon[66829]: Deploying daemon osd.9 on smithi069 2024-03-19T01:05:13.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:12 smithi086 ceph-mon[66829]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:15.058 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:14 smithi086 ceph-mon[66829]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:15.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:14 smithi163 ceph-mon[67428]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:15.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:14 smithi069 ceph-mon[31727]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:16.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:16 smithi069 ceph-mon[31727]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:17.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:16 smithi163 ceph-mon[67428]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:17.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:16 smithi086 ceph-mon[66829]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:18.969 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:18 smithi069 ceph-mon[31727]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:18 smithi163 ceph-mon[67428]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:19.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:18 smithi086 ceph-mon[66829]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:20.752 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:20.752 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:20.752 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:20 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:20.752 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:20 smithi069 ceph-mon[31727]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:20.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:20.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:20.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:20 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:20 smithi163 ceph-mon[67428]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:20 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:20 smithi086 ceph-mon[66829]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:22.743 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:22 smithi069 ceph-mon[31727]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:23.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:22 smithi086 ceph-mon[66829]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:22 smithi163 ceph-mon[67428]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:24.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.310+0000 7fdd40a00700 1 -- 172.21.15.69:0/3877152837 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdd3c062750 con 0x7fdd200a2890 2024-03-19T01:05:24.313 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 9 on host 'smithi069' 2024-03-19T01:05:24.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 -- 172.21.15.69:0/3877152837 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdd200a2890 msgr2=0x7fdd200a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:05:24.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdd200a2890 0x7fdd200a4d50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800b2e0 tx=0x7fdd28005fd0 comp rx=0 tx=0).stop 2024-03-19T01:05:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 -- 172.21.15.69:0/3877152837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd3c108980 msgr2=0x7fdd3c19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:05:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd3c108980 0x7fdd3c19eb80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00e970 tx=0x7fdd2c00ec80 comp rx=0 tx=0).stop 2024-03-19T01:05:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 -- 172.21.15.69:0/3877152837 shutdown_connections 2024-03-19T01:05:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdd200a2890 0x7fdd200a4d50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd3c10a520 0x7fdd3c1a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd3c108980 0x7fdd3c19eb80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 --2- 172.21.15.69:0/3877152837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd3c109b80 0x7fdd3c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 -- 172.21.15.69:0/3877152837 >> 172.21.15.69:0/3877152837 conn(0x7fdd3c077950 msgr2=0x7fdd3c10cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:05:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 -- 172.21.15.69:0/3877152837 shutdown_connections 2024-03-19T01:05:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:05:24.314+0000 7fdd41e00700 1 -- 172.21.15.69:0/3877152837 wait complete. 2024-03-19T01:05:24.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:24.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:24.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:24 smithi069 ceph-mon[31727]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-19T01:05:24.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:24 smithi069 ceph-mon[31727]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:24.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:24.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:24 smithi086 ceph-mon[66829]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-19T01:05:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:24 smithi086 ceph-mon[66829]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:24 smithi163 ceph-mon[67428]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-19T01:05:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:24 smithi163 ceph-mon[67428]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:25.126 DEBUG:teuthology.orchestra.run.smithi069:osd.9> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.9.service 2024-03-19T01:05:25.132 INFO:tasks.cephadm:Deploying osd.10 on smithi086 with /dev/vg_nvme/lv_1... 2024-03-19T01:05:25.132 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-19T01:05:25.379 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:05:25.500 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:25 smithi086 ceph-mon[66829]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-19T01:05:25.500 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:25 smithi086 ceph-mon[66829]: osdmap e65: 10 total, 9 up, 10 in 2024-03-19T01:05:25.500 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:25 smithi086 ceph-mon[66829]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:05:25.500 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:25 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:25.715 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:25 smithi069 ceph-mon[31727]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-19T01:05:25.715 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:25 smithi069 ceph-mon[31727]: osdmap e65: 10 total, 9 up, 10 in 2024-03-19T01:05:25.715 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:25 smithi069 ceph-mon[31727]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:05:25.715 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:25 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:25.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:25 smithi163 ceph-mon[67428]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-19T01:05:25.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:25 smithi163 ceph-mon[67428]: osdmap e65: 10 total, 9 up, 10 in 2024-03-19T01:05:25.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:25 smithi163 ceph-mon[67428]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-19T01:05:25.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:25 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:25.986 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:05:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:05:25.948+0000 7f8352600700 -1 osd.9 0 waiting for initial osdmap 2024-03-19T01:05:25.987 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:05:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:05:25.984+0000 7f8348400700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:05:26.627 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:26 smithi086 ceph-mon[66829]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:05:26.627 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:26 smithi086 ceph-mon[66829]: osdmap e66: 10 total, 9 up, 10 in 2024-03-19T01:05:26.627 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:26.627 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:26 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:26.627 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:26 smithi086 ceph-mon[66829]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:26.627 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:26 smithi086 ceph-mon[66829]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' 2024-03-19T01:05:26.711 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:26 smithi069 ceph-mon[31727]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:05:26.711 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:26 smithi069 ceph-mon[31727]: osdmap e66: 10 total, 9 up, 10 in 2024-03-19T01:05:26.712 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:26 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:26.712 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:26 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:26.712 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:26 smithi069 ceph-mon[31727]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:26.712 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:26 smithi069 ceph-mon[31727]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' 2024-03-19T01:05:26.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:26 smithi163 ceph-mon[67428]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-19T01:05:26.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:26 smithi163 ceph-mon[67428]: osdmap e66: 10 total, 9 up, 10 in 2024-03-19T01:05:26.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:26 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:26.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:26 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:26.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:26 smithi163 ceph-mon[67428]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:05:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:26 smithi163 ceph-mon[67428]: from='osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735]' entity='osd.9' 2024-03-19T01:05:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-19T01:05:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-19T01:05:27.338 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-19T01:05:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-19T01:05:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131007 s, 800 MB/s 2024-03-19T01:05:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-19T01:05:27.633 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:27 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:05:27.634 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:27 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:05:27.634 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:27 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:27.634 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:27 smithi086 ceph-mon[66829]: osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735] boot 2024-03-19T01:05:27.634 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:27 smithi086 ceph-mon[66829]: osdmap e67: 10 total, 10 up, 10 in 2024-03-19T01:05:27.634 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:27 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:27.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:27 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:05:27.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:27 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:05:27.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:27 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:27.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:27 smithi069 ceph-mon[31727]: osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735] boot 2024-03-19T01:05:27.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:27 smithi069 ceph-mon[31727]: osdmap e67: 10 total, 10 up, 10 in 2024-03-19T01:05:27.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:27 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:27 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:05:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:27 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:05:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:27 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:27 smithi163 ceph-mon[67428]: osd.9 [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735] boot 2024-03-19T01:05:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:27 smithi163 ceph-mon[67428]: osdmap e67: 10 total, 10 up, 10 in 2024-03-19T01:05:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:27 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:05:28.017 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-03-19T01:05:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:05:28.667 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:28 smithi086 ceph-mon[66829]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:28.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:28 smithi069 ceph-mon[31727]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:28.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:28 smithi163 ceph-mon[67428]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:29.642 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:29 smithi086 ceph-mon[66829]: osdmap e68: 10 total, 10 up, 10 in 2024-03-19T01:05:29.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:29 smithi069 ceph-mon[31727]: osdmap e68: 10 total, 10 up, 10 in 2024-03-19T01:05:29.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:29 smithi163 ceph-mon[67428]: osdmap e68: 10 total, 10 up, 10 in 2024-03-19T01:05:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.868+0000 7fb2c58a1700 1 -- 172.21.15.86:0/1908543073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c0102480 msgr2=0x7fb2c01028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:05:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.868+0000 7fb2c58a1700 1 --2- 172.21.15.86:0/1908543073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c0102480 0x7fb2c01028e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80099c0 tx=0x7fb2a8009cd0 comp rx=0 tx=0).stop 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.868+0000 7fb2c58a1700 1 -- 172.21.15.86:0/1908543073 shutdown_connections 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.868+0000 7fb2c58a1700 1 --2- 172.21.15.86:0/1908543073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2c0102e20 0x7fb2c01096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.868+0000 7fb2c58a1700 1 --2- 172.21.15.86:0/1908543073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2c0101280 0x7fb2c01016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.868+0000 7fb2c58a1700 1 --2- 172.21.15.86:0/1908543073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c0102480 0x7fb2c01028e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.868+0000 7fb2c58a1700 1 -- 172.21.15.86:0/1908543073 >> 172.21.15.86:0/1908543073 conn(0x7fb2c00fc800 msgr2=0x7fb2c00fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.868+0000 7fb2c58a1700 1 -- 172.21.15.86:0/1908543073 shutdown_connections 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.868+0000 7fb2c58a1700 1 -- 172.21.15.86:0/1908543073 wait complete. 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.868+0000 7fb2c58a1700 1 Processor -- start 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.868+0000 7fb2c58a1700 1 -- start start 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2c58a1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2c0101280 0x7fb2c01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2c58a1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2c0102480 0x7fb2c0198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2c58a1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c0102e20 0x7fb2c019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:05:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2c58a1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c010cf60 con 0x7fb2c0102480 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2c58a1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c010cde0 con 0x7fb2c0102e20 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2c58a1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c010d0e0 con 0x7fb2c0101280 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2bf7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c0102e20 0x7fb2c019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2bf7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c0102e20 0x7fb2c019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47768/0 (socket says 172.21.15.86:47768) 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2bf7fe700 1 -- 172.21.15.86:0/54977069 learned_addr learned my addr 172.21.15.86:0/54977069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2beffd700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2c0101280 0x7fb2c01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2bf7fe700 1 -- 172.21.15.86:0/54977069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2c0101280 msgr2=0x7fb2c01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2bf7fe700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2c0101280 0x7fb2c01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2bf7fe700 1 -- 172.21.15.86:0/54977069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2c0102480 msgr2=0x7fb2c0198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2bf7fe700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2c0102480 0x7fb2c0198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2bf7fe700 1 -- 172.21.15.86:0/54977069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c01a0320 con 0x7fb2c0102e20 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2bf7fe700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c0102e20 0x7fb2c019fc80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800b970 tx=0x7fb2b800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2c489f700 1 -- 172.21.15.86:0/54977069 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8019070 con 0x7fb2c0102e20 2024-03-19T01:05:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2c58a1700 1 -- 172.21.15.86:0/54977069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c01a0610 con 0x7fb2c0102e20 2024-03-19T01:05:29.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2c489f700 1 -- 172.21.15.86:0/54977069 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b800ce10 con 0x7fb2c0102e20 2024-03-19T01:05:29.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2c489f700 1 -- 172.21.15.86:0/54977069 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80073d0 con 0x7fb2c0102e20 2024-03-19T01:05:29.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.869+0000 7fb2c58a1700 1 -- 172.21.15.86:0/54977069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c01a0b80 con 0x7fb2c0102e20 2024-03-19T01:05:29.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.870+0000 7fb2c58a1700 1 -- 172.21.15.86:0/54977069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c0068fd0 con 0x7fb2c0102e20 2024-03-19T01:05:29.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.871+0000 7fb2c489f700 1 -- 172.21.15.86:0/54977069 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb2b801c070 con 0x7fb2c0102e20 2024-03-19T01:05:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.872+0000 7fb2c489f700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb2ac0a2890 0x7fb2ac0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:05:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.872+0000 7fb2c489f700 1 -- 172.21.15.86:0/54977069 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7fb2b80cdb50 con 0x7fb2c0102e20 2024-03-19T01:05:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.873+0000 7fb2beffd700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb2ac0a2890 0x7fb2ac0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:05:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.873+0000 7fb2beffd700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb2ac0a2890 0x7fb2ac0a4d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb2c01022e0 tx=0x7fb2b0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:05:29.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:29.876+0000 7fb2c489f700 1 -- 172.21.15.86:0/54977069 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb2b809b350 con 0x7fb2c0102e20 2024-03-19T01:05:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:30.084+0000 7fb2c58a1700 1 -- 172.21.15.86:0/54977069 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb2c006b140 con 0x7fb2ac0a2890 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:05:30.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:05:30.718 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:30.718 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:30.718 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:30.718 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:30.718 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:30 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:05:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:05:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:30 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:30 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:31.740 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:31 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi069 2024-03-19T01:05:31.740 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:31 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi069 to 4272M 2024-03-19T01:05:31.740 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:31 smithi086 ceph-mon[66829]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:05:31.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:31 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi069 2024-03-19T01:05:31.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:31 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi069 to 4272M 2024-03-19T01:05:31.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:31 smithi069 ceph-mon[31727]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:05:31.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:31 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi069 2024-03-19T01:05:31.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:31 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi069 to 4272M 2024-03-19T01:05:31.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:31 smithi163 ceph-mon[67428]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:05:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:32 smithi086 ceph-mon[66829]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:32.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:32 smithi069 ceph-mon[31727]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:32.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:32 smithi163 ceph-mon[67428]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:33 smithi086 ceph-mon[66829]: from='client.? 172.21.15.86:0/354637122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41af0156-b4aa-43f3-9e40-6396608fbfa8"}]: dispatch 2024-03-19T01:05:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:33 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41af0156-b4aa-43f3-9e40-6396608fbfa8"}]: dispatch 2024-03-19T01:05:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:33 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41af0156-b4aa-43f3-9e40-6396608fbfa8"}]': finished 2024-03-19T01:05:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:33 smithi086 ceph-mon[66829]: osdmap e69: 11 total, 10 up, 11 in 2024-03-19T01:05:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:33 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:33.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:33 smithi069 ceph-mon[31727]: from='client.? 172.21.15.86:0/354637122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41af0156-b4aa-43f3-9e40-6396608fbfa8"}]: dispatch 2024-03-19T01:05:33.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:33 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41af0156-b4aa-43f3-9e40-6396608fbfa8"}]: dispatch 2024-03-19T01:05:33.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:33 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41af0156-b4aa-43f3-9e40-6396608fbfa8"}]': finished 2024-03-19T01:05:33.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:33 smithi069 ceph-mon[31727]: osdmap e69: 11 total, 10 up, 11 in 2024-03-19T01:05:33.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:33 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:33 smithi163 ceph-mon[67428]: from='client.? 172.21.15.86:0/354637122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41af0156-b4aa-43f3-9e40-6396608fbfa8"}]: dispatch 2024-03-19T01:05:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:33 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41af0156-b4aa-43f3-9e40-6396608fbfa8"}]: dispatch 2024-03-19T01:05:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:33 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41af0156-b4aa-43f3-9e40-6396608fbfa8"}]': finished 2024-03-19T01:05:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:33 smithi163 ceph-mon[67428]: osdmap e69: 11 total, 10 up, 11 in 2024-03-19T01:05:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:33 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:34.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:34 smithi086 ceph-mon[66829]: from='client.? 172.21.15.86:0/3763106866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:05:34.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:34 smithi086 ceph-mon[66829]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:34.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:34 smithi069 ceph-mon[31727]: from='client.? 172.21.15.86:0/3763106866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:05:34.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:34 smithi069 ceph-mon[31727]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:34 smithi163 ceph-mon[67428]: from='client.? 172.21.15.86:0/3763106866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:05:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:34 smithi163 ceph-mon[67428]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:36.841 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:36 smithi086 ceph-mon[66829]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:36.919 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:36 smithi069 ceph-mon[31727]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:37.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:36 smithi163 ceph-mon[67428]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:38.981 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:38 smithi086 ceph-mon[66829]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:39.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:38 smithi069 ceph-mon[31727]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:39.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:38 smithi163 ceph-mon[67428]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:40.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:40 smithi086 ceph-mon[66829]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:40.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-19T01:05:40.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:40 smithi069 ceph-mon[31727]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-19T01:05:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:40 smithi163 ceph-mon[67428]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-19T01:05:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:41.979 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:41 smithi086 ceph-mon[66829]: Deploying daemon osd.10 on smithi086 2024-03-19T01:05:42.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:41 smithi069 ceph-mon[31727]: Deploying daemon osd.10 on smithi086 2024-03-19T01:05:42.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:41 smithi163 ceph-mon[67428]: Deploying daemon osd.10 on smithi086 2024-03-19T01:05:43.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:42 smithi086 ceph-mon[66829]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:43.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:42 smithi069 ceph-mon[31727]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:43.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:42 smithi163 ceph-mon[67428]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:45.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:44 smithi086 ceph-mon[66829]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:45.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:44 smithi069 ceph-mon[31727]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:45.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:44 smithi163 ceph-mon[67428]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:46.868 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:46 smithi086 ceph-mon[66829]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:46.868 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:46.868 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:46.868 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:46 smithi069 ceph-mon[31727]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:46 smithi163 ceph-mon[67428]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:48.928 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:48 smithi086 ceph-mon[66829]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.009+0000 7fb2c489f700 1 -- 172.21.15.86:0/54977069 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb2c006b140 con 0x7fb2ac0a2890 2024-03-19T01:05:49.010 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 10 on host 'smithi086' 2024-03-19T01:05:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 -- 172.21.15.86:0/54977069 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb2ac0a2890 msgr2=0x7fb2ac0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:05:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb2ac0a2890 0x7fb2ac0a4d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb2c01022e0 tx=0x7fb2b0009190 comp rx=0 tx=0).stop 2024-03-19T01:05:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 -- 172.21.15.86:0/54977069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c0102e20 msgr2=0x7fb2c019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:05:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c0102e20 0x7fb2c019fc80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800b970 tx=0x7fb2b800bc80 comp rx=0 tx=0).stop 2024-03-19T01:05:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 -- 172.21.15.86:0/54977069 shutdown_connections 2024-03-19T01:05:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb2ac0a2890 0x7fb2ac0a4d50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c0102e20 0x7fb2c019fc80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2c0101280 0x7fb2c01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 --2- 172.21.15.86:0/54977069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2c0102480 0x7fb2c0198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 -- 172.21.15.86:0/54977069 >> 172.21.15.86:0/54977069 conn(0x7fb2c00fc800 msgr2=0x7fb2c006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:05:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 -- 172.21.15.86:0/54977069 shutdown_connections 2024-03-19T01:05:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-19T01:05:49.011+0000 7fb2c58a1700 1 -- 172.21.15.86:0/54977069 wait complete. 2024-03-19T01:05:49.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:48 smithi069 ceph-mon[31727]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:49.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:48 smithi163 ceph-mon[67428]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:49.629 DEBUG:teuthology.orchestra.run.smithi086:osd.10> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.10.service 2024-03-19T01:05:49.633 INFO:tasks.cephadm:Deploying osd.11 on smithi163 with /dev/vg_nvme/lv_1... 2024-03-19T01:05:49.633 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-19T01:05:49.880 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:05:50.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:49 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:50.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:49 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:50.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:49 smithi086 ceph-mon[66829]: from='osd.10 [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:05:50.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:49 smithi086 ceph-mon[66829]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:05:50.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:49 smithi086 ceph-mon[66829]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:50.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:49 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:50.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:49 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:50.264 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:49 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:50.264 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:49 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:50.265 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:49 smithi163 ceph-mon[67428]: from='osd.10 [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:05:50.265 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:49 smithi163 ceph-mon[67428]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:05:50.265 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:49 smithi163 ceph-mon[67428]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:50.265 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:49 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:50.265 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:49 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:50.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:49 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:50.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:49 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:50.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:49 smithi069 ceph-mon[31727]: from='osd.10 [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:05:50.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:49 smithi069 ceph-mon[31727]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:05:50.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:49 smithi069 ceph-mon[31727]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:50.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:49 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:50.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:49 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:51.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:50 smithi086 ceph-mon[66829]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-19T01:05:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:50 smithi086 ceph-mon[66829]: osdmap e70: 11 total, 10 up, 11 in 2024-03-19T01:05:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:50 smithi086 ceph-mon[66829]: from='osd.10 [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:05:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:50 smithi086 ceph-mon[66829]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:05:51.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:50 smithi069 ceph-mon[31727]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-19T01:05:51.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:50 smithi069 ceph-mon[31727]: osdmap e70: 11 total, 10 up, 11 in 2024-03-19T01:05:51.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:51.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:50 smithi069 ceph-mon[31727]: from='osd.10 [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:05:51.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:50 smithi069 ceph-mon[31727]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:05:51.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:50 smithi163 ceph-mon[67428]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-19T01:05:51.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:50 smithi163 ceph-mon[67428]: osdmap e70: 11 total, 10 up, 11 in 2024-03-19T01:05:51.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:51.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:50 smithi163 ceph-mon[67428]: from='osd.10 [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:05:51.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:50 smithi163 ceph-mon[67428]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-19T01:05:51.666 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:05:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:05:51.393+0000 7f7cd9504700 -1 osd.10 0 waiting for initial osdmap 2024-03-19T01:05:51.666 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:05:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:05:51.406+0000 7f7cd0aee700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:05:52.038 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-19T01:05:52.038 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-19T01:05:52.039 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2024-03-19T01:05:52.039 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2024-03-19T01:05:52.039 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113868 s, 921 MB/s 2024-03-19T01:05:52.039 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2024-03-19T01:05:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:51 smithi086 ceph-mon[66829]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:05:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:51 smithi086 ceph-mon[66829]: osdmap e71: 11 total, 10 up, 11 in 2024-03-19T01:05:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:51 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:51 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:51 smithi086 ceph-mon[66829]: from='osd.10 ' entity='osd.10' 2024-03-19T01:05:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:51 smithi086 ceph-mon[66829]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:51 smithi069 ceph-mon[31727]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:05:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:51 smithi069 ceph-mon[31727]: osdmap e71: 11 total, 10 up, 11 in 2024-03-19T01:05:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:51 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:51 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:51 smithi069 ceph-mon[31727]: from='osd.10 ' entity='osd.10' 2024-03-19T01:05:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:51 smithi069 ceph-mon[31727]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:52.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:51 smithi163 ceph-mon[67428]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-19T01:05:52.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:51 smithi163 ceph-mon[67428]: osdmap e71: 11 total, 10 up, 11 in 2024-03-19T01:05:52.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:51 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:52.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:51 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:52.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:51 smithi163 ceph-mon[67428]: from='osd.10 ' entity='osd.10' 2024-03-19T01:05:52.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:51 smithi163 ceph-mon[67428]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:05:52.672 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi163:vg_nvme/lv_1 2024-03-19T01:05:52.921 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:05:53.129 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:53 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:05:53.129 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:53 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:05:53.129 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:53 smithi086 ceph-mon[66829]: osd.10 [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109] boot 2024-03-19T01:05:53.129 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:53 smithi086 ceph-mon[66829]: osdmap e72: 11 total, 11 up, 11 in 2024-03-19T01:05:53.129 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:53 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:53.129 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:53 smithi086 ceph-mon[66829]: osdmap e73: 11 total, 11 up, 11 in 2024-03-19T01:05:53.273 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:53 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:05:53.274 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:53 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:05:53.274 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:53 smithi163 ceph-mon[67428]: osd.10 [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109] boot 2024-03-19T01:05:53.274 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:53 smithi163 ceph-mon[67428]: osdmap e72: 11 total, 11 up, 11 in 2024-03-19T01:05:53.274 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:53 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:53.274 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:53 smithi163 ceph-mon[67428]: osdmap e73: 11 total, 11 up, 11 in 2024-03-19T01:05:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:53 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:05:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:53 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:05:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:53 smithi069 ceph-mon[31727]: osd.10 [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109] boot 2024-03-19T01:05:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:53 smithi069 ceph-mon[31727]: osdmap e72: 11 total, 11 up, 11 in 2024-03-19T01:05:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:53 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:05:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:53 smithi069 ceph-mon[31727]: osdmap e73: 11 total, 11 up, 11 in 2024-03-19T01:05:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi086 2024-03-19T01:05:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi086 to 4270M 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:05:54.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:54 smithi163 ceph-mon[67428]: osdmap e74: 11 total, 11 up, 11 in 2024-03-19T01:05:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi086 2024-03-19T01:05:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi086 to 4270M 2024-03-19T01:05:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:05:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:54 smithi086 ceph-mon[66829]: osdmap e74: 11 total, 11 up, 11 in 2024-03-19T01:05:54.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi086 2024-03-19T01:05:54.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:05:54.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi086 to 4270M 2024-03-19T01:05:54.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:05:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:05:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:05:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:05:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:54 smithi069 ceph-mon[31727]: osdmap e74: 11 total, 11 up, 11 in 2024-03-19T01:05:54.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.583+0000 7f89835e6700 1 -- 172.21.15.163:0/3704152633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f897c103f00 msgr2=0x7f897c10a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:05:54.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.583+0000 7f89835e6700 1 --2- 172.21.15.163:0/3704152633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f897c103f00 0x7f897c10a7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8978009990 tx=0x7f8978009ca0 comp rx=0 tx=0).stop 2024-03-19T01:05:54.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.583+0000 7f89835e6700 1 -- 172.21.15.163:0/3704152633 shutdown_connections 2024-03-19T01:05:54.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.583+0000 7f89835e6700 1 --2- 172.21.15.163:0/3704152633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f897c103f00 0x7f897c10a7b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:54.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.583+0000 7f89835e6700 1 --2- 172.21.15.163:0/3704152633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c102360 0x7f897c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:54.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.583+0000 7f89835e6700 1 --2- 172.21.15.163:0/3704152633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f897c103560 0x7f897c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:54.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.583+0000 7f89835e6700 1 -- 172.21.15.163:0/3704152633 >> 172.21.15.163:0/3704152633 conn(0x7f897c0fd920 msgr2=0x7f897c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:05:54.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.583+0000 7f89835e6700 1 -- 172.21.15.163:0/3704152633 shutdown_connections 2024-03-19T01:05:54.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.583+0000 7f89835e6700 1 -- 172.21.15.163:0/3704152633 wait complete. 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.583+0000 7f89835e6700 1 Processor -- start 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.583+0000 7f89835e6700 1 -- start start 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f89835e6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c102360 0x7f897c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f89835e6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f897c103560 0x7f897c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f89835e6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f897c103f00 0x7f897c19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f89835e6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10dfc0 con 0x7f897c103560 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f89835e6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10de40 con 0x7f897c102360 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f89835e6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10e140 con 0x7f897c103f00 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f8981b83700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f897c103f00 0x7f897c19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f8981b83700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f897c103f00 0x7f897c19fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58866/0 (socket says 172.21.15.163:58866) 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f8981b83700 1 -- 172.21.15.163:0/3640251891 learned_addr learned my addr 172.21.15.163:0/3640251891 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f8981382700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c102360 0x7f897c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:05:54.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f8981b83700 1 -- 172.21.15.163:0/3640251891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c102360 msgr2=0x7f897c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:05:54.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f8981b83700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c102360 0x7f897c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:54.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f8981b83700 1 -- 172.21.15.163:0/3640251891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f897c103560 msgr2=0x7f897c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:05:54.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f8981b83700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f897c103560 0x7f897c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:05:54.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f8981b83700 1 -- 172.21.15.163:0/3640251891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c1a01a0 con 0x7f897c103f00 2024-03-19T01:05:54.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f8981b83700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f897c103f00 0x7f897c19fb00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8978003890 tx=0x7f8978004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:05:54.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.584+0000 7f89727fc700 1 -- 172.21.15.163:0/3640251891 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978026070 con 0x7f897c103f00 2024-03-19T01:05:54.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.585+0000 7f89727fc700 1 -- 172.21.15.163:0/3640251891 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8978004800 con 0x7f897c103f00 2024-03-19T01:05:54.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.585+0000 7f89835e6700 1 -- 172.21.15.163:0/3640251891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c1a0430 con 0x7f897c103f00 2024-03-19T01:05:54.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.585+0000 7f89727fc700 1 -- 172.21.15.163:0/3640251891 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89780054f0 con 0x7f897c103f00 2024-03-19T01:05:54.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.585+0000 7f89835e6700 1 -- 172.21.15.163:0/3640251891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c1a0900 con 0x7f897c103f00 2024-03-19T01:05:54.589 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.585+0000 7f89835e6700 1 -- 172.21.15.163:0/3640251891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c068fd0 con 0x7f897c103f00 2024-03-19T01:05:54.589 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.587+0000 7f89727fc700 1 -- 172.21.15.163:0/3640251891 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8978029070 con 0x7f897c103f00 2024-03-19T01:05:54.589 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.587+0000 7f89727fc700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f89680a2890 0x7f89680a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:05:54.589 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.587+0000 7f89727fc700 1 -- 172.21.15.163:0/3640251891 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7f89780cf030 con 0x7f897c103f00 2024-03-19T01:05:54.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.588+0000 7f8981382700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f89680a2890 0x7f89680a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:05:54.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.588+0000 7f8981382700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f89680a2890 0x7f89680a4d50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f89740060b0 tx=0x7f8974007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:05:54.592 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.592+0000 7f89727fc700 1 -- 172.21.15.163:0/3640251891 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f897805d740 con 0x7f897c103f00 2024-03-19T01:05:54.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:05:54.795+0000 7f89835e6700 1 -- 172.21.15.163:0/3640251891 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f897c062750 con 0x7f89680a2890 2024-03-19T01:05:55.751 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:55 smithi163 ceph-mon[67428]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:05:55.751 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:55 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:05:55.751 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:55 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:05:55.751 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:55 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:55.752 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:55 smithi163 ceph-mon[67428]: osdmap e75: 11 total, 11 up, 11 in 2024-03-19T01:05:55.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:55 smithi086 ceph-mon[66829]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:05:55.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:55 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:05:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:55 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:05:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:55 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:55 smithi086 ceph-mon[66829]: osdmap e75: 11 total, 11 up, 11 in 2024-03-19T01:05:55.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:55 smithi069 ceph-mon[31727]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:05:55.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:55 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:05:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:55 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:05:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:55 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:05:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:55 smithi069 ceph-mon[31727]: osdmap e75: 11 total, 11 up, 11 in 2024-03-19T01:05:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:56 smithi069 ceph-mon[31727]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:05:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:56 smithi069 ceph-mon[31727]: osdmap e76: 11 total, 11 up, 11 in 2024-03-19T01:05:56.783 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:56 smithi163 ceph-mon[67428]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:05:56.783 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:56 smithi163 ceph-mon[67428]: osdmap e76: 11 total, 11 up, 11 in 2024-03-19T01:05:57.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:56 smithi086 ceph-mon[66829]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:05:57.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:56 smithi086 ceph-mon[66829]: osdmap e76: 11 total, 11 up, 11 in 2024-03-19T01:05:57.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:57 smithi069 ceph-mon[31727]: from='client.? 172.21.15.163:0/185609808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf43d278-aaee-4d11-9fda-3b72869c045c"}]: dispatch 2024-03-19T01:05:57.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:57 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf43d278-aaee-4d11-9fda-3b72869c045c"}]: dispatch 2024-03-19T01:05:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:57 smithi069 ceph-mon[31727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf43d278-aaee-4d11-9fda-3b72869c045c"}]': finished 2024-03-19T01:05:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:57 smithi069 ceph-mon[31727]: osdmap e77: 12 total, 11 up, 12 in 2024-03-19T01:05:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:57 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:05:57.956 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:57 smithi163 ceph-mon[67428]: from='client.? 172.21.15.163:0/185609808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf43d278-aaee-4d11-9fda-3b72869c045c"}]: dispatch 2024-03-19T01:05:57.956 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:57 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf43d278-aaee-4d11-9fda-3b72869c045c"}]: dispatch 2024-03-19T01:05:57.956 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:57 smithi163 ceph-mon[67428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf43d278-aaee-4d11-9fda-3b72869c045c"}]': finished 2024-03-19T01:05:57.956 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:57 smithi163 ceph-mon[67428]: osdmap e77: 12 total, 11 up, 12 in 2024-03-19T01:05:57.956 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:57 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:05:58.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:57 smithi086 ceph-mon[66829]: from='client.? 172.21.15.163:0/185609808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf43d278-aaee-4d11-9fda-3b72869c045c"}]: dispatch 2024-03-19T01:05:58.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:57 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf43d278-aaee-4d11-9fda-3b72869c045c"}]: dispatch 2024-03-19T01:05:58.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:57 smithi086 ceph-mon[66829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf43d278-aaee-4d11-9fda-3b72869c045c"}]': finished 2024-03-19T01:05:58.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:57 smithi086 ceph-mon[66829]: osdmap e77: 12 total, 11 up, 12 in 2024-03-19T01:05:58.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:57 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:05:59.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:58 smithi086 ceph-mon[66829]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:05:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:05:58 smithi086 ceph-mon[66829]: from='client.? 172.21.15.163:0/126475600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:05:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:58 smithi069 ceph-mon[31727]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:05:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:05:58 smithi069 ceph-mon[31727]: from='client.? 172.21.15.163:0/126475600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:05:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:58 smithi163 ceph-mon[67428]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:05:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:05:58 smithi163 ceph-mon[67428]: from='client.? 172.21.15.163:0/126475600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:06:01.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:00 smithi086 ceph-mon[66829]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-19T01:06:01.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:00 smithi069 ceph-mon[31727]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-19T01:06:01.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:00 smithi163 ceph-mon[67428]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-19T01:06:02.801 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:02 smithi163 ceph-mon[67428]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-19T01:06:03.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:02 smithi086 ceph-mon[66829]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-19T01:06:03.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:02 smithi069 ceph-mon[31727]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-19T01:06:04.981 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:04 smithi163 ceph-mon[67428]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:06:05.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:04 smithi086 ceph-mon[66829]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:06:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:04 smithi069 ceph-mon[31727]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:06:05.959 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-19T01:06:05.960 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:06.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-19T01:06:06.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:06.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-19T01:06:06.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:06.840 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:06 smithi163 ceph-mon[67428]: Deploying daemon osd.11 on smithi163 2024-03-19T01:06:06.840 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:06 smithi163 ceph-mon[67428]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-19T01:06:07.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:06 smithi086 ceph-mon[66829]: Deploying daemon osd.11 on smithi163 2024-03-19T01:06:07.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:06 smithi086 ceph-mon[66829]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-19T01:06:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:06 smithi069 ceph-mon[31727]: Deploying daemon osd.11 on smithi163 2024-03-19T01:06:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:06 smithi069 ceph-mon[31727]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-19T01:06:08.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:08 smithi069 ceph-mon[31727]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:06:08.962 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:08 smithi163 ceph-mon[67428]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:06:09.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:08 smithi086 ceph-mon[66829]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:06:10.877 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:10 smithi163 ceph-mon[67428]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-19T01:06:11.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:10 smithi086 ceph-mon[66829]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-19T01:06:11.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:10 smithi069 ceph-mon[31727]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-19T01:06:12.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:06:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:11 smithi086 ceph-mon[66829]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:12.272 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:12.272 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:12.272 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:06:12.272 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:11 smithi163 ceph-mon[67428]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:12.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:12.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:12.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:06:12.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:11 smithi069 ceph-mon[31727]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:14.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.529+0000 7f89727fc700 1 -- 172.21.15.163:0/3640251891 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f897c062750 con 0x7f89680a2890 2024-03-19T01:06:14.530 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 11 on host 'smithi163' 2024-03-19T01:06:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 -- 172.21.15.163:0/3640251891 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f89680a2890 msgr2=0x7f89680a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f89680a2890 0x7f89680a4d50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f89740060b0 tx=0x7f8974007b80 comp rx=0 tx=0).stop 2024-03-19T01:06:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 -- 172.21.15.163:0/3640251891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f897c103f00 msgr2=0x7f897c19fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f897c103f00 0x7f897c19fb00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8978003890 tx=0x7f8978004790 comp rx=0 tx=0).stop 2024-03-19T01:06:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 -- 172.21.15.163:0/3640251891 shutdown_connections 2024-03-19T01:06:14.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f89680a2890 0x7f89680a4d50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:14.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f897c103f00 0x7f897c19fb00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:14.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c102360 0x7f897c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:14.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 --2- 172.21.15.163:0/3640251891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f897c103560 0x7f897c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:14.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 -- 172.21.15.163:0/3640251891 >> 172.21.15.163:0/3640251891 conn(0x7f897c0fd920 msgr2=0x7f897c106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:14.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 -- 172.21.15.163:0/3640251891 shutdown_connections 2024-03-19T01:06:14.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:14.531+0000 7f89835e6700 1 -- 172.21.15.163:0/3640251891 wait complete. 2024-03-19T01:06:15.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:14 smithi086 ceph-mon[66829]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:15.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:15.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:15.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:14 smithi069 ceph-mon[31727]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:15.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:15.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:15.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:14 smithi163 ceph-mon[67428]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:15.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:15.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:15.246 DEBUG:teuthology.orchestra.run.smithi163:osd.11> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.11.service 2024-03-19T01:06:15.248 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-19T01:06:15.248 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-19T01:06:15.692 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:06:16.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:15 smithi086 ceph-mon[66829]: from='osd.11 [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:06:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:15 smithi086 ceph-mon[66829]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:06:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:15 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:15 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:16.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:15 smithi069 ceph-mon[31727]: from='osd.11 [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:06:16.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:15 smithi069 ceph-mon[31727]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:06:16.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:15 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:16.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:15 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:16.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:15 smithi163 ceph-mon[67428]: from='osd.11 [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:06:16.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:15 smithi163 ceph-mon[67428]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:06:16.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:15 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:16.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:15 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:16.986 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:16 smithi163 ceph-mon[67428]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:16.986 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:16 smithi163 ceph-mon[67428]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-19T01:06:16.986 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:16 smithi163 ceph-mon[67428]: osdmap e78: 12 total, 11 up, 12 in 2024-03-19T01:06:16.986 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:16 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:16.986 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:16 smithi163 ceph-mon[67428]: from='osd.11 [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:06:16.986 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:16 smithi163 ceph-mon[67428]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:06:16.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:06:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:06:16.732+0000 7f612abdc700 -1 osd.11 0 waiting for initial osdmap 2024-03-19T01:06:16.987 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:06:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:06:16.747+0000 7f61269cf700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:06:17.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:16 smithi086 ceph-mon[66829]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:17.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:16 smithi086 ceph-mon[66829]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-19T01:06:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:16 smithi086 ceph-mon[66829]: osdmap e78: 12 total, 11 up, 12 in 2024-03-19T01:06:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:16 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:16 smithi086 ceph-mon[66829]: from='osd.11 [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:06:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:16 smithi086 ceph-mon[66829]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:06:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:16 smithi069 ceph-mon[31727]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:16 smithi069 ceph-mon[31727]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-19T01:06:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:16 smithi069 ceph-mon[31727]: osdmap e78: 12 total, 11 up, 12 in 2024-03-19T01:06:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:16 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:16 smithi069 ceph-mon[31727]: from='osd.11 [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:06:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:16 smithi069 ceph-mon[31727]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-19T01:06:17.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.905+0000 7f0029a00700 1 -- 172.21.15.69:0/2720252849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0024106130 msgr2=0x7f002410c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.905+0000 7f0029a00700 1 --2- 172.21.15.69:0/2720252849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0024106130 0x7f002410c9e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f001c00b0a0 tx=0x7f001c00b3b0 comp rx=0 tx=0).stop 2024-03-19T01:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.906+0000 7f0029a00700 1 -- 172.21.15.69:0/2720252849 shutdown_connections 2024-03-19T01:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.906+0000 7f0029a00700 1 --2- 172.21.15.69:0/2720252849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0024106130 0x7f002410c9e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.906+0000 7f0029a00700 1 --2- 172.21.15.69:0/2720252849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0024104590 0x7f00241049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.906+0000 7f0029a00700 1 --2- 172.21.15.69:0/2720252849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024105790 0x7f0024105bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.906+0000 7f0029a00700 1 -- 172.21.15.69:0/2720252849 >> 172.21.15.69:0/2720252849 conn(0x7f00240ffb30 msgr2=0x7f0024101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.906+0000 7f0029a00700 1 -- 172.21.15.69:0/2720252849 shutdown_connections 2024-03-19T01:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.906+0000 7f0029a00700 1 -- 172.21.15.69:0/2720252849 wait complete. 2024-03-19T01:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.907+0000 7f0029a00700 1 Processor -- start 2024-03-19T01:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.907+0000 7f0029a00700 1 -- start start 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f0029a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0024104590 0x7f002419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f0029a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0024105790 0x7f002419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f0029a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024106130 0x7f00241a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f0029a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f00241101a0 con 0x7f0024104590 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f0022a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0024104590 0x7f002419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f0029a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0024110020 con 0x7f0024106130 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f0029a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0024110320 con 0x7f0024105790 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f001be00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0024105790 0x7f002419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f001be00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0024105790 0x7f002419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48144/0 (socket says 172.21.15.69:48144) 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f001be00700 1 -- 172.21.15.69:0/1856604524 learned_addr learned my addr 172.21.15.69:0/1856604524 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f0023400700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024106130 0x7f00241a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f001be00700 1 -- 172.21.15.69:0/1856604524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024106130 msgr2=0x7f00241a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f001be00700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024106130 0x7f00241a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f001be00700 1 -- 172.21.15.69:0/1856604524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0024104590 msgr2=0x7f002419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f001be00700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0024104590 0x7f002419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f001be00700 1 -- 172.21.15.69:0/1856604524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00241a22d0 con 0x7f0024105790 2024-03-19T01:06:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.908+0000 7f0023400700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024106130 0x7f00241a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:06:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.909+0000 7f001be00700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0024105790 0x7f002419abb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f000c00d9b0 tx=0x7f000c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.909+0000 7f001b400700 1 -- 172.21.15.69:0/1856604524 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c016070 con 0x7f0024105790 2024-03-19T01:06:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.909+0000 7f001b400700 1 -- 172.21.15.69:0/1856604524 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000c004d10 con 0x7f0024105790 2024-03-19T01:06:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.909+0000 7f0029a00700 1 -- 172.21.15.69:0/1856604524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00241a25c0 con 0x7f0024105790 2024-03-19T01:06:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.909+0000 7f001b400700 1 -- 172.21.15.69:0/1856604524 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c005630 con 0x7f0024105790 2024-03-19T01:06:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.909+0000 7f0029a00700 1 -- 172.21.15.69:0/1856604524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00241a2b00 con 0x7f0024105790 2024-03-19T01:06:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.910+0000 7f0029a00700 1 -- 172.21.15.69:0/1856604524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0024068fd0 con 0x7f0024105790 2024-03-19T01:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.912+0000 7f001b400700 1 -- 172.21.15.69:0/1856604524 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f000c004750 con 0x7f0024105790 2024-03-19T01:06:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.913+0000 7f001b400700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f00080a2890 0x7f00080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.914+0000 7f001b400700 1 -- 172.21.15.69:0/1856604524 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f000c0ce4c0 con 0x7f0024105790 2024-03-19T01:06:17.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.914+0000 7f0022a00700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f00080a2890 0x7f00080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:17.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.914+0000 7f0022a00700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f00080a2890 0x7f00080a4d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f00241055f0 tx=0x7f001400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:17.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:17.917+0000 7f001b400700 1 -- 172.21.15.69:0/1856604524 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f000c01f020 con 0x7f0024105790 2024-03-19T01:06:18.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:17 smithi086 ceph-mon[66829]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:06:18.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:17 smithi086 ceph-mon[66829]: osdmap e79: 12 total, 11 up, 12 in 2024-03-19T01:06:18.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:17 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:18.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:17 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:17 smithi069 ceph-mon[31727]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:06:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:17 smithi069 ceph-mon[31727]: osdmap e79: 12 total, 11 up, 12 in 2024-03-19T01:06:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:17 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:17 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:18.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:17 smithi163 ceph-mon[67428]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-19T01:06:18.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:17 smithi163 ceph-mon[67428]: osdmap e79: 12 total, 11 up, 12 in 2024-03-19T01:06:18.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:18.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:17 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:18.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.138+0000 7f0029a00700 1 -- 172.21.15.69:0/1856604524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0024062f20 con 0x7f0024105790 2024-03-19T01:06:18.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.139+0000 7f001b400700 1 -- 172.21.15.69:0/1856604524 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f000c09b960 con 0x7f0024105790 2024-03-19T01:06:18.140 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:18.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 -- 172.21.15.69:0/1856604524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f00080a2890 msgr2=0x7f00080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:18.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f00080a2890 0x7f00080a4d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f00241055f0 tx=0x7f001400a3b0 comp rx=0 tx=0).stop 2024-03-19T01:06:18.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 -- 172.21.15.69:0/1856604524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0024105790 msgr2=0x7f002419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:18.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0024105790 0x7f002419abb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f000c00d9b0 tx=0x7f000c00dcc0 comp rx=0 tx=0).stop 2024-03-19T01:06:18.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 -- 172.21.15.69:0/1856604524 shutdown_connections 2024-03-19T01:06:18.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f00080a2890 0x7f00080a4d50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:18.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024106130 0x7f00241a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:18.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0024104590 0x7f002419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:18.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 --2- 172.21.15.69:0/1856604524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0024105790 0x7f002419abb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:18.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 -- 172.21.15.69:0/1856604524 >> 172.21.15.69:0/1856604524 conn(0x7f00240ffb30 msgr2=0x7f00241089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:18.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 -- 172.21.15.69:0/1856604524 shutdown_connections 2024-03-19T01:06:18.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:18.142+0000 7f0029a00700 1 -- 172.21.15.69:0/1856604524 wait complete. 2024-03-19T01:06:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:18 smithi069 ceph-mon[31727]: purged_snaps scrub starts 2024-03-19T01:06:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:18 smithi069 ceph-mon[31727]: purged_snaps scrub ok 2024-03-19T01:06:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:18 smithi069 ceph-mon[31727]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:18 smithi069 ceph-mon[31727]: osd.11 [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559] boot 2024-03-19T01:06:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:18 smithi069 ceph-mon[31727]: osdmap e80: 12 total, 12 up, 12 in 2024-03-19T01:06:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:18 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:18 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1856604524' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T01:06:18.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:18 smithi163 ceph-mon[67428]: purged_snaps scrub starts 2024-03-19T01:06:18.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:18 smithi163 ceph-mon[67428]: purged_snaps scrub ok 2024-03-19T01:06:18.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:18 smithi163 ceph-mon[67428]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:18.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:18 smithi163 ceph-mon[67428]: osd.11 [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559] boot 2024-03-19T01:06:18.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:18 smithi163 ceph-mon[67428]: osdmap e80: 12 total, 12 up, 12 in 2024-03-19T01:06:18.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:18 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:18.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:18 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1856604524' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T01:06:19.003 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1710810377,"num_in_osds":12,"osd_in_since":1710810357,"num_remapped_pgs":0} 2024-03-19T01:06:19.003 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-19T01:06:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:18 smithi086 ceph-mon[66829]: purged_snaps scrub starts 2024-03-19T01:06:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:18 smithi086 ceph-mon[66829]: purged_snaps scrub ok 2024-03-19T01:06:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:18 smithi086 ceph-mon[66829]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:06:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:18 smithi086 ceph-mon[66829]: osd.11 [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559] boot 2024-03-19T01:06:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:18 smithi086 ceph-mon[66829]: osdmap e80: 12 total, 12 up, 12 in 2024-03-19T01:06:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:18 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:06:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:18 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1856604524' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T01:06:19.402 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:06:19.867 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: osdmap e81: 12 total, 12 up, 12 in 2024-03-19T01:06:19.867 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:06:19.868 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:19 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:20.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: osdmap e81: 12 total, 12 up, 12 in 2024-03-19T01:06:20.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:20.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:06:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:19 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:20.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: osdmap e81: 12 total, 12 up, 12 in 2024-03-19T01:06:20.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:20.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:20.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:20.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:20.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:20.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:06:20.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:20.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:20.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:06:20.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:20.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:06:20.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:20.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:06:20.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:19 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:20.960 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:20 smithi069 ceph-mon[31727]: Detected new or changed devices on smithi163 2024-03-19T01:06:20.960 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:20 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi163 to 4270M 2024-03-19T01:06:20.961 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:20 smithi069 ceph-mon[31727]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:20 smithi163 ceph-mon[67428]: Detected new or changed devices on smithi163 2024-03-19T01:06:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:20 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi163 to 4270M 2024-03-19T01:06:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:20 smithi163 ceph-mon[67428]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:21.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:20 smithi086 ceph-mon[66829]: Detected new or changed devices on smithi163 2024-03-19T01:06:21.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:20 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi163 to 4270M 2024-03-19T01:06:21.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:20 smithi086 ceph-mon[66829]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.469+0000 7fc4b6e00700 1 -- 172.21.15.69:0/1687672178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4b0104580 msgr2=0x7fc4b0104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.469+0000 7fc4b6e00700 1 --2- 172.21.15.69:0/1687672178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4b0104580 0x7fc4b0104a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0099c0 tx=0x7fc49c009cd0 comp rx=0 tx=0).stop 2024-03-19T01:06:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.469+0000 7fc4b6e00700 1 -- 172.21.15.69:0/1687672178 shutdown_connections 2024-03-19T01:06:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.469+0000 7fc4b6e00700 1 --2- 172.21.15.69:0/1687672178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4b0104f40 0x7fc4b0111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.469+0000 7fc4b6e00700 1 --2- 172.21.15.69:0/1687672178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4b010a630 0x7fc4b010aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.469+0000 7fc4b6e00700 1 --2- 172.21.15.69:0/1687672178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4b0104580 0x7fc4b0104a00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.469+0000 7fc4b6e00700 1 -- 172.21.15.69:0/1687672178 >> 172.21.15.69:0/1687672178 conn(0x7fc4b01000b0 msgr2=0x7fc4b01024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.469+0000 7fc4b6e00700 1 -- 172.21.15.69:0/1687672178 shutdown_connections 2024-03-19T01:06:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.470+0000 7fc4b6e00700 1 -- 172.21.15.69:0/1687672178 wait complete. 2024-03-19T01:06:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.471+0000 7fc4b6e00700 1 Processor -- start 2024-03-19T01:06:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.472+0000 7fc4b6e00700 1 -- start start 2024-03-19T01:06:21.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.472+0000 7fc4b6e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4b0104580 0x7fc4b01a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:21.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.472+0000 7fc4b6e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4b0104f40 0x7fc4b01a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:21.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.472+0000 7fc4b6e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4b010a630 0x7fc4b019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:21.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.472+0000 7fc4b6e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01148f0 con 0x7fc4b0104580 2024-03-19T01:06:21.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.472+0000 7fc4b6e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0114770 con 0x7fc4b0104f40 2024-03-19T01:06:21.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.472+0000 7fc4b6e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0114a70 con 0x7fc4b010a630 2024-03-19T01:06:21.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.473+0000 7fc4afe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4b0104580 0x7fc4b01a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:21.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.473+0000 7fc4af400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4b0104f40 0x7fc4b01a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:21.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.473+0000 7fc4afe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4b0104580 0x7fc4b01a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39598/0 (socket says 172.21.15.69:39598) 2024-03-19T01:06:21.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.473+0000 7fc4afe00700 1 -- 172.21.15.69:0/657286383 learned_addr learned my addr 172.21.15.69:0/657286383 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:21.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.473+0000 7fc4b4e00700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4b010a630 0x7fc4b019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:21.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.473+0000 7fc4afe00700 1 -- 172.21.15.69:0/657286383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4b010a630 msgr2=0x7fc4b019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:21.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.473+0000 7fc4afe00700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4b010a630 0x7fc4b019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:21.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.473+0000 7fc4afe00700 1 -- 172.21.15.69:0/657286383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4b0104f40 msgr2=0x7fc4b01a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:21.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.473+0000 7fc4afe00700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4b0104f40 0x7fc4b01a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:21.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.473+0000 7fc4afe00700 1 -- 172.21.15.69:0/657286383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b019b270 con 0x7fc4b0104580 2024-03-19T01:06:21.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.474+0000 7fc4afe00700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4b0104580 0x7fc4b01a0900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000d980 tx=0x7fc4a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:21.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.474+0000 7fc4acc00700 1 -- 172.21.15.69:0/657286383 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0017070 con 0x7fc4b0104580 2024-03-19T01:06:21.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.474+0000 7fc4acc00700 1 -- 172.21.15.69:0/657286383 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4a0005020 con 0x7fc4b0104580 2024-03-19T01:06:21.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.474+0000 7fc4b6e00700 1 -- 172.21.15.69:0/657286383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b019b560 con 0x7fc4b0104580 2024-03-19T01:06:21.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.474+0000 7fc4acc00700 1 -- 172.21.15.69:0/657286383 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a000fbf0 con 0x7fc4b0104580 2024-03-19T01:06:21.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.474+0000 7fc4b6e00700 1 -- 172.21.15.69:0/657286383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b01a77d0 con 0x7fc4b0104580 2024-03-19T01:06:21.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.476+0000 7fc4b6e00700 1 -- 172.21.15.69:0/657286383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b0068fd0 con 0x7fc4b0104580 2024-03-19T01:06:21.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.478+0000 7fc4acc00700 1 -- 172.21.15.69:0/657286383 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc4a00051d0 con 0x7fc4b0104580 2024-03-19T01:06:21.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.479+0000 7fc4acc00700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4940a2890 0x7fc4940a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:21.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.479+0000 7fc4acc00700 1 -- 172.21.15.69:0/657286383 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc4a00ce460 con 0x7fc4b0104580 2024-03-19T01:06:21.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.479+0000 7fc4af400700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4940a2890 0x7fc4940a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:21.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.480+0000 7fc4af400700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4940a2890 0x7fc4940a4d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0109cd0 tx=0x7fc49c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:21.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.484+0000 7fc4acc00700 1 -- 172.21.15.69:0/657286383 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc4a0020020 con 0x7fc4b0104580 2024-03-19T01:06:21.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.705+0000 7fc4b6e00700 1 -- 172.21.15.69:0/657286383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4b002dbb0 con 0x7fc4b0104580 2024-03-19T01:06:21.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.707+0000 7fc4acc00700 1 -- 172.21.15.69:0/657286383 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18397 (secure 0 0 0) 0x7fc4a009b900 con 0x7fc4b0104580 2024-03-19T01:06:21.708 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:21.708 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":81,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:06:18.746880+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:06:21.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.709+0000 7fc4b6e00700 1 -- 172.21.15.69:0/657286383 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4940a2890 msgr2=0x7fc4940a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:21.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.709+0000 7fc4b6e00700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4940a2890 0x7fc4940a4d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0109cd0 tx=0x7fc49c005fb0 comp rx=0 tx=0).stop 2024-03-19T01:06:21.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.710+0000 7fc4b6e00700 1 -- 172.21.15.69:0/657286383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4b0104580 msgr2=0x7fc4b01a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:21.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.710+0000 7fc4b6e00700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4b0104580 0x7fc4b01a0900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000d980 tx=0x7fc4a000dc90 comp rx=0 tx=0).stop 2024-03-19T01:06:21.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.710+0000 7fc4b6e00700 1 -- 172.21.15.69:0/657286383 shutdown_connections 2024-03-19T01:06:21.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.710+0000 7fc4b6e00700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4940a2890 0x7fc4940a4d50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:21.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.710+0000 7fc4b6e00700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4b010a630 0x7fc4b019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:21.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.710+0000 7fc4b6e00700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4b0104580 0x7fc4b01a0900 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:21.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.710+0000 7fc4b6e00700 1 --2- 172.21.15.69:0/657286383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4b0104f40 0x7fc4b01a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:21.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.710+0000 7fc4b6e00700 1 -- 172.21.15.69:0/657286383 >> 172.21.15.69:0/657286383 conn(0x7fc4b01000b0 msgr2=0x7fc4b0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:21.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.710+0000 7fc4b6e00700 1 -- 172.21.15.69:0/657286383 shutdown_connections 2024-03-19T01:06:21.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:21.710+0000 7fc4b6e00700 1 -- 172.21.15.69:0/657286383 wait complete. 2024-03-19T01:06:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:21 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/657286383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:06:22.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:21 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/657286383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:06:22.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:21 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/657286383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:06:22.517 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T01:06:22.517 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-19T01:06:22.963 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:06:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:22 smithi069 ceph-mon[31727]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:22 smithi163 ceph-mon[67428]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:23.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:22 smithi086 ceph-mon[66829]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:24.999 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:24 smithi069 ceph-mon[31727]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:24 smithi163 ceph-mon[67428]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:25.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.169+0000 7f6797800700 1 -- 172.21.15.69:0/3200708789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6790107630 msgr2=0x7f679010f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:25.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.169+0000 7f6797800700 1 --2- 172.21.15.69:0/3200708789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6790107630 0x7f679010f200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f678800b0d0 tx=0x7f678800b3e0 comp rx=0 tx=0).stop 2024-03-19T01:06:25.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.169+0000 7f6797800700 1 -- 172.21.15.69:0/3200708789 shutdown_connections 2024-03-19T01:06:25.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.169+0000 7f6797800700 1 --2- 172.21.15.69:0/3200708789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6790107630 0x7f679010f200 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:25.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.169+0000 7f6797800700 1 --2- 172.21.15.69:0/3200708789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6790102550 0x7f6790102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:25.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.169+0000 7f6797800700 1 --2- 172.21.15.69:0/3200708789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6790102f00 0x7f6790106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:25.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.170+0000 7f6797800700 1 -- 172.21.15.69:0/3200708789 >> 172.21.15.69:0/3200708789 conn(0x7f67900fddc0 msgr2=0x7f67901001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:25.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.170+0000 7f6797800700 1 -- 172.21.15.69:0/3200708789 shutdown_connections 2024-03-19T01:06:25.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.170+0000 7f6797800700 1 -- 172.21.15.69:0/3200708789 wait complete. 2024-03-19T01:06:25.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.171+0000 7f6797800700 1 Processor -- start 2024-03-19T01:06:25.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.171+0000 7f6797800700 1 -- start start 2024-03-19T01:06:25.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6797800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6790102550 0x7f6790074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:25.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6797800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6790102f00 0x7f6790074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:25.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6797800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6790107630 0x7f6790075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:25.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6797800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6790112690 con 0x7f6790102f00 2024-03-19T01:06:25.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6797800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6790112510 con 0x7f6790107630 2024-03-19T01:06:25.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6797800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6790112810 con 0x7f6790102550 2024-03-19T01:06:25.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6794e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6790102550 0x7f6790074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:25.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6787e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6790102f00 0x7f6790074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:25.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6794e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6790102550 0x7f6790074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48168/0 (socket says 172.21.15.69:48168) 2024-03-19T01:06:25.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6787e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6790102f00 0x7f6790074b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39604/0 (socket says 172.21.15.69:39604) 2024-03-19T01:06:25.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6794e00700 1 -- 172.21.15.69:0/1021305366 learned_addr learned my addr 172.21.15.69:0/1021305366 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:25.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.172+0000 7f6795800700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6790107630 0x7f6790075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.173+0000 7f6794e00700 1 -- 172.21.15.69:0/1021305366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6790107630 msgr2=0x7f6790075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.173+0000 7f6794e00700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6790107630 0x7f6790075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.173+0000 7f6794e00700 1 -- 172.21.15.69:0/1021305366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6790102f00 msgr2=0x7f6790074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.173+0000 7f6794e00700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6790102f00 0x7f6790074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.173+0000 7f6794e00700 1 -- 172.21.15.69:0/1021305366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67901a6ff0 con 0x7f6790102550 2024-03-19T01:06:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.173+0000 7f6794e00700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6790102550 0x7f6790074610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f678000e9a0 tx=0x7f678000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.173+0000 7f678e000700 1 -- 172.21.15.69:0/1021305366 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678000ca10 con 0x7f6790102550 2024-03-19T01:06:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.173+0000 7f678e000700 1 -- 172.21.15.69:0/1021305366 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f678000cbb0 con 0x7f6790102550 2024-03-19T01:06:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.173+0000 7f6797800700 1 -- 172.21.15.69:0/1021305366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67901a72e0 con 0x7f6790102550 2024-03-19T01:06:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.173+0000 7f678e000700 1 -- 172.21.15.69:0/1021305366 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780018680 con 0x7f6790102550 2024-03-19T01:06:25.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.173+0000 7f6797800700 1 -- 172.21.15.69:0/1021305366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67901a7870 con 0x7f6790102550 2024-03-19T01:06:25.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.174+0000 7f6797800700 1 -- 172.21.15.69:0/1021305366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6790068fd0 con 0x7f6790102550 2024-03-19T01:06:25.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.176+0000 7f678e000700 1 -- 172.21.15.69:0/1021305366 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f6780047a60 con 0x7f6790102550 2024-03-19T01:06:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.178+0000 7f678e000700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f67740a2910 0x7f67740a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.178+0000 7f678e000700 1 -- 172.21.15.69:0/1021305366 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f67800ce820 con 0x7f6790102550 2024-03-19T01:06:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.178+0000 7f6787e00700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f67740a2910 0x7f67740a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.178+0000 7f6787e00700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f67740a2910 0x7f67740a4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6790108b60 tx=0x7f6778009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:25.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.180+0000 7f678e000700 1 -- 172.21.15.69:0/1021305366 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f678009bcc0 con 0x7f6790102550 2024-03-19T01:06:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:24 smithi086 ceph-mon[66829]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:25.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.397+0000 7f6797800700 1 -- 172.21.15.69:0/1021305366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6790075b40 con 0x7f6790102550 2024-03-19T01:06:25.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.398+0000 7f678e000700 1 -- 172.21.15.69:0/1021305366 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f678009b5c0 con 0x7f6790102550 2024-03-19T01:06:25.400 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.401+0000 7f6797800700 1 -- 172.21.15.69:0/1021305366 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f67740a2910 msgr2=0x7f67740a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.401+0000 7f6797800700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f67740a2910 0x7f67740a4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6790108b60 tx=0x7f6778009260 comp rx=0 tx=0).stop 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.401+0000 7f6797800700 1 -- 172.21.15.69:0/1021305366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6790102550 msgr2=0x7f6790074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.401+0000 7f6797800700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6790102550 0x7f6790074610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f678000e9a0 tx=0x7f678000ecb0 comp rx=0 tx=0).stop 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.402+0000 7f6797800700 1 -- 172.21.15.69:0/1021305366 shutdown_connections 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.402+0000 7f6797800700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f67740a2910 0x7f67740a4dd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.402+0000 7f6797800700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6790107630 0x7f6790075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.402+0000 7f6797800700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6790102550 0x7f6790074610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.402+0000 7f6797800700 1 --2- 172.21.15.69:0/1021305366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6790102f00 0x7f6790074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.402+0000 7f6797800700 1 -- 172.21.15.69:0/1021305366 >> 172.21.15.69:0/1021305366 conn(0x7f67900fddc0 msgr2=0x7f67900fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.402+0000 7f6797800700 1 -- 172.21.15.69:0/1021305366 shutdown_connections 2024-03-19T01:06:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:25.402+0000 7f6797800700 1 -- 172.21.15.69:0/1021305366 wait complete. 2024-03-19T01:06:26.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:25 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1021305366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:06:26.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:25 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1021305366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:06:26.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:25 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1021305366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:06:26.280 INFO:tasks.cephadm:Adding mds.a on smithi069 2024-03-19T01:06:26.280 INFO:tasks.cephadm:Adding mds.d on smithi069 2024-03-19T01:06:26.280 INFO:tasks.cephadm:Adding mds.g on smithi069 2024-03-19T01:06:26.280 INFO:tasks.cephadm:Adding mds.j on smithi069 2024-03-19T01:06:26.281 INFO:tasks.cephadm:Adding mds.b on smithi086 2024-03-19T01:06:26.281 INFO:tasks.cephadm:Adding mds.e on smithi086 2024-03-19T01:06:26.281 INFO:tasks.cephadm:Adding mds.h on smithi086 2024-03-19T01:06:26.281 INFO:tasks.cephadm:Adding mds.k on smithi086 2024-03-19T01:06:26.281 INFO:tasks.cephadm:Adding mds.c on smithi163 2024-03-19T01:06:26.281 INFO:tasks.cephadm:Adding mds.f on smithi163 2024-03-19T01:06:26.281 INFO:tasks.cephadm:Adding mds.i on smithi163 2024-03-19T01:06:26.281 INFO:tasks.cephadm:Adding mds.l on smithi163 2024-03-19T01:06:26.281 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph orch apply mds all '12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l' 2024-03-19T01:06:26.532 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:06:26.939 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:26 smithi163 ceph-mon[67428]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:27.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:26 smithi086 ceph-mon[66829]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:27.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:26 smithi069 ceph-mon[31727]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:28.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.232+0000 7fafe14f0700 1 -- 172.21.15.163:0/1721202127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafdc103530 msgr2=0x7fafdc103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:28.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.232+0000 7fafe14f0700 1 --2- 172.21.15.163:0/1721202127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafdc103530 0x7fafdc103990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009990 tx=0x7fafd0009ca0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.232+0000 7fafe14f0700 1 -- 172.21.15.163:0/1721202127 shutdown_connections 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.232+0000 7fafe14f0700 1 --2- 172.21.15.163:0/1721202127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafdc103ed0 0x7fafdc10a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.232+0000 7fafe14f0700 1 --2- 172.21.15.163:0/1721202127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc102330 0x7fafdc102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.232+0000 7fafe14f0700 1 --2- 172.21.15.163:0/1721202127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafdc103530 0x7fafdc103990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.232+0000 7fafe14f0700 1 -- 172.21.15.163:0/1721202127 >> 172.21.15.163:0/1721202127 conn(0x7fafdc0fd8f0 msgr2=0x7fafdc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.233+0000 7fafe14f0700 1 -- 172.21.15.163:0/1721202127 shutdown_connections 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.233+0000 7fafe14f0700 1 -- 172.21.15.163:0/1721202127 wait complete. 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.233+0000 7fafe14f0700 1 Processor -- start 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.233+0000 7fafe14f0700 1 -- start start 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.233+0000 7fafe14f0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafdc102330 0x7fafdc0760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.233+0000 7fafe14f0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc103530 0x7fafdc074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.233+0000 7fafe14f0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafdc103ed0 0x7fafdc074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.233+0000 7fafe14f0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10e010 con 0x7fafdc103ed0 2024-03-19T01:06:28.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.233+0000 7fafe14f0700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10de90 con 0x7fafdc103530 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.233+0000 7fafe14f0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10e190 con 0x7fafdc102330 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafdaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafdc102330 0x7fafdc0760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafdaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafdc102330 0x7fafdc0760b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34384/0 (socket says 172.21.15.163:34384) 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafdaffd700 1 -- 172.21.15.163:0/2579083989 learned_addr learned my addr 172.21.15.163:0/2579083989 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafda7fc700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc103530 0x7fafdc074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafdaffd700 1 -- 172.21.15.163:0/2579083989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc103530 msgr2=0x7fafdc074700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafdaffd700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc103530 0x7fafdc074700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafdaffd700 1 -- 172.21.15.163:0/2579083989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafdc103ed0 msgr2=0x7fafdc074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafdaffd700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafdc103ed0 0x7fafdc074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafdaffd700 1 -- 172.21.15.163:0/2579083989 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc0754f0 con 0x7fafdc102330 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafdaffd700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafdc102330 0x7fafdc0760b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00b500 tx=0x7fafcc00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafc3fff700 1 -- 172.21.15.163:0/2579083989 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc015020 con 0x7fafdc102330 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafe14f0700 1 -- 172.21.15.163:0/2579083989 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc1a2820 con 0x7fafdc102330 2024-03-19T01:06:28.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafc3fff700 1 -- 172.21.15.163:0/2579083989 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafcc011d10 con 0x7fafdc102330 2024-03-19T01:06:28.238 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafc3fff700 1 -- 172.21.15.163:0/2579083989 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc00f2e0 con 0x7fafdc102330 2024-03-19T01:06:28.238 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.234+0000 7fafe14f0700 1 -- 172.21.15.163:0/2579083989 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc1a2de0 con 0x7fafdc102330 2024-03-19T01:06:28.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.235+0000 7fafe14f0700 1 -- 172.21.15.163:0/2579083989 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc068fd0 con 0x7fafdc102330 2024-03-19T01:06:28.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.236+0000 7fafc3fff700 1 -- 172.21.15.163:0/2579083989 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fafcc018020 con 0x7fafdc102330 2024-03-19T01:06:28.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.237+0000 7fafc3fff700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fafc40a2890 0x7fafc40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:28.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.237+0000 7fafc3fff700 1 -- 172.21.15.163:0/2579083989 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fafcc0cddd0 con 0x7fafdc102330 2024-03-19T01:06:28.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.237+0000 7fafda7fc700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fafc40a2890 0x7fafc40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:28.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.238+0000 7fafda7fc700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fafc40a2890 0x7fafc40a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fafd000b5c0 tx=0x7fafd0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:28.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.241+0000 7fafc3fff700 1 -- 172.21.15.163:0/2579083989 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fafcc01f020 con 0x7fafdc102330 2024-03-19T01:06:28.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.448+0000 7fafe14f0700 1 -- 172.21.15.163:0/2579083989 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}) -- 0x7fafdc062950 con 0x7fafc40a2890 2024-03-19T01:06:28.465 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.464+0000 7fafc3fff700 1 -- 172.21.15.163:0/2579083989 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fafdc062950 con 0x7fafc40a2890 2024-03-19T01:06:28.465 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled mds.all update... 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 -- 172.21.15.163:0/2579083989 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fafc40a2890 msgr2=0x7fafc40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fafc40a2890 0x7fafc40a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fafd000b5c0 tx=0x7fafd0005fb0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 -- 172.21.15.163:0/2579083989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafdc102330 msgr2=0x7fafdc0760b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafdc102330 0x7fafdc0760b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00b500 tx=0x7fafcc00b8c0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 -- 172.21.15.163:0/2579083989 shutdown_connections 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fafc40a2890 0x7fafc40a4d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafdc103ed0 0x7fafdc074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafdc102330 0x7fafdc0760b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 --2- 172.21.15.163:0/2579083989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafdc103530 0x7fafdc074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 -- 172.21.15.163:0/2579083989 >> 172.21.15.163:0/2579083989 conn(0x7fafdc0fd8f0 msgr2=0x7fafdc1068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 -- 172.21.15.163:0/2579083989 shutdown_connections 2024-03-19T01:06:28.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-19T01:06:28.467+0000 7fafe14f0700 1 -- 172.21.15.163:0/2579083989 wait complete. 2024-03-19T01:06:29.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:28 smithi163 ceph-mon[67428]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:29.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:29.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:28 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:06:29.254 DEBUG:teuthology.orchestra.run.smithi069:mds.a> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.a.service 2024-03-19T01:06:29.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:28 smithi086 ceph-mon[66829]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:29.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:29.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:28 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:06:29.261 DEBUG:teuthology.orchestra.run.smithi069:mds.d> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.d.service 2024-03-19T01:06:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:28 smithi069 ceph-mon[31727]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:28 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:06:29.286 DEBUG:teuthology.orchestra.run.smithi069:mds.g> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.g.service 2024-03-19T01:06:29.331 DEBUG:teuthology.orchestra.run.smithi069:mds.j> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.j.service 2024-03-19T01:06:29.377 DEBUG:teuthology.orchestra.run.smithi086:mds.b> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.b.service 2024-03-19T01:06:29.380 DEBUG:teuthology.orchestra.run.smithi086:mds.e> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.e.service 2024-03-19T01:06:29.423 DEBUG:teuthology.orchestra.run.smithi086:mds.h> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.h.service 2024-03-19T01:06:29.465 DEBUG:teuthology.orchestra.run.smithi086:mds.k> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.k.service 2024-03-19T01:06:29.508 DEBUG:teuthology.orchestra.run.smithi163:mds.c> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.c.service 2024-03-19T01:06:29.510 DEBUG:teuthology.orchestra.run.smithi163:mds.f> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.f.service 2024-03-19T01:06:29.553 DEBUG:teuthology.orchestra.run.smithi163:mds.i> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.i.service 2024-03-19T01:06:29.595 DEBUG:teuthology.orchestra.run.smithi163:mds.l> sudo journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.l.service 2024-03-19T01:06:29.638 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-19T01:06:29.639 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:06:29.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.862+0000 7f3030e00640 1 -- 172.21.15.69:0/2977461700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302c104a20 msgr2=0x7f302c106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:29.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.862+0000 7f3030e00640 1 --2- 172.21.15.69:0/2977461700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302c104a20 0x7f302c106e10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3018009860 tx=0x7f301802bb70 comp rx=0 tx=0).stop 2024-03-19T01:06:29.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.863+0000 7f3030e00640 1 -- 172.21.15.69:0/2977461700 shutdown_connections 2024-03-19T01:06:29.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.863+0000 7f3030e00640 1 --2- 172.21.15.69:0/2977461700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f302c107350 0x7f302c1097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:29.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.863+0000 7f3030e00640 1 --2- 172.21.15.69:0/2977461700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302c104a20 0x7f302c106e10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:29.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.863+0000 7f3030e00640 1 --2- 172.21.15.69:0/2977461700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f302c1020f0 0x7f302c1044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:29.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.863+0000 7f3030e00640 1 -- 172.21.15.69:0/2977461700 >> 172.21.15.69:0/2977461700 conn(0x7f302c0fbb00 msgr2=0x7f302c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:29.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.863+0000 7f3030e00640 1 -- 172.21.15.69:0/2977461700 shutdown_connections 2024-03-19T01:06:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.863+0000 7f3030e00640 1 -- 172.21.15.69:0/2977461700 wait complete. 2024-03-19T01:06:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.864+0000 7f3030e00640 1 Processor -- start 2024-03-19T01:06:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.864+0000 7f3030e00640 1 -- start start 2024-03-19T01:06:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.864+0000 7f3030e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f302c1020f0 0x7f302c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.864+0000 7f3030e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302c104a20 0x7f302c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.864+0000 7f3030e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f302c107350 0x7f302c192e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.864+0000 7f3030e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f302c10d2d0 con 0x7f302c1020f0 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.864+0000 7f3030e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f302c10d150 con 0x7f302c104a20 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.864+0000 7f3030e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f302c10d450 con 0x7f302c107350 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.864+0000 7f302be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f302c107350 0x7f302c192e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.864+0000 7f302be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f302c107350 0x7f302c192e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48442/0 (socket says 172.21.15.69:48442) 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.864+0000 7f302be00640 1 -- 172.21.15.69:0/1327659071 learned_addr learned my addr 172.21.15.69:0/1327659071 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f302b400640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f302c1020f0 0x7f302c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f302aa00640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302c104a20 0x7f302c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f302be00640 1 -- 172.21.15.69:0/1327659071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302c104a20 msgr2=0x7f302c196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f302be00640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302c104a20 0x7f302c196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f302be00640 1 -- 172.21.15.69:0/1327659071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f302c1020f0 msgr2=0x7f302c196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f302be00640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f302c1020f0 0x7f302c196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f302be00640 1 -- 172.21.15.69:0/1327659071 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c193750 con 0x7f302c107350 2024-03-19T01:06:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f302be00640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f302c107350 0x7f302c192e90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f302400eaa0 tx=0x7f302400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:29.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f3023e00640 1 -- 172.21.15.69:0/1327659071 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302400cdb0 con 0x7f302c107350 2024-03-19T01:06:29.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f3023e00640 1 -- 172.21.15.69:0/1327659071 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3024005020 con 0x7f302c107350 2024-03-19T01:06:29.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f3030e00640 1 -- 172.21.15.69:0/1327659071 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c1a5ff0 con 0x7f302c107350 2024-03-19T01:06:29.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.865+0000 7f3023e00640 1 -- 172.21.15.69:0/1327659071 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30240116b0 con 0x7f302c107350 2024-03-19T01:06:29.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.866+0000 7f3030e00640 1 -- 172.21.15.69:0/1327659071 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c1a63a0 con 0x7f302c107350 2024-03-19T01:06:29.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.867+0000 7f3030e00640 1 -- 172.21.15.69:0/1327659071 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302c068530 con 0x7f302c107350 2024-03-19T01:06:29.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.868+0000 7f3023e00640 1 -- 172.21.15.69:0/1327659071 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f3024005560 con 0x7f302c107350 2024-03-19T01:06:29.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.870+0000 7f3023e00640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f30100a2880 0x7f30100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:29.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.870+0000 7f3023e00640 1 -- 172.21.15.69:0/1327659071 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f30240ce900 con 0x7f302c107350 2024-03-19T01:06:29.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.870+0000 7f302b400640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f30100a2880 0x7f30100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:29.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.871+0000 7f302b400640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f30100a2880 0x7f30100a4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f301c0096c0 tx=0x7f301c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:29.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:29.874+0000 7f3023e00640 1 -- 172.21.15.69:0/1327659071 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3024016030 con 0x7f302c107350 2024-03-19T01:06:30.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:29 smithi069 ceph-mon[31727]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:06:30.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:29 smithi069 ceph-mon[31727]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2024-03-19T01:06:30.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:30.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:06:30.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:30.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:30.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:30.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:29 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:30.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:29 smithi163 ceph-mon[67428]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:06:30.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:29 smithi163 ceph-mon[67428]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2024-03-19T01:06:30.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:30.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:06:30.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:30.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:30.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:30.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:29 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:30.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.076+0000 7f3030e00640 1 -- 172.21.15.69:0/1327659071 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f302c062420 con 0x7f302c107350 2024-03-19T01:06:30.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.077+0000 7f3023e00640 1 -- 172.21.15.69:0/1327659071 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18397 (secure 0 0 0) 0x7f302409bda0 con 0x7f302c107350 2024-03-19T01:06:30.078 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:30.079 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":81,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:06:18.746880+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:06:30.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 -- 172.21.15.69:0/1327659071 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f30100a2880 msgr2=0x7f30100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:30.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f30100a2880 0x7f30100a4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f301c0096c0 tx=0x7f301c009260 comp rx=0 tx=0).stop 2024-03-19T01:06:30.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 -- 172.21.15.69:0/1327659071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f302c107350 msgr2=0x7f302c192e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:30.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f302c107350 0x7f302c192e90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f302400eaa0 tx=0x7f302400ef60 comp rx=0 tx=0).stop 2024-03-19T01:06:30.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 -- 172.21.15.69:0/1327659071 shutdown_connections 2024-03-19T01:06:30.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f30100a2880 0x7f30100a4d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f302c107350 0x7f302c192e90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f302c104a20 0x7f302c196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 --2- 172.21.15.69:0/1327659071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f302c1020f0 0x7f302c196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 -- 172.21.15.69:0/1327659071 >> 172.21.15.69:0/1327659071 conn(0x7f302c0fbb00 msgr2=0x7f302c1018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:30.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 -- 172.21.15.69:0/1327659071 shutdown_connections 2024-03-19T01:06:30.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.080+0000 7f3030e00640 1 -- 172.21.15.69:0/1327659071 wait complete. 2024-03-19T01:06:30.102 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T01:06:30.102 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:06:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:29 smithi086 ceph-mon[66829]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:06:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:29 smithi086 ceph-mon[66829]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2024-03-19T01:06:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:06:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:29 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:30.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.316+0000 7f0544e00640 1 -- 172.21.15.69:0/3862750490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0540108f90 msgr2=0x7f0540109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:30.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.316+0000 7f0544e00640 1 --2- 172.21.15.69:0/3862750490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0540108f90 0x7f0540109410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0528009900 tx=0x7f052802bc30 comp rx=0 tx=0).stop 2024-03-19T01:06:30.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.317+0000 7f0544e00640 1 -- 172.21.15.69:0/3862750490 shutdown_connections 2024-03-19T01:06:30.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.317+0000 7f0544e00640 1 --2- 172.21.15.69:0/3862750490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0540109950 0x7f0540110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.317+0000 7f0544e00640 1 --2- 172.21.15.69:0/3862750490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0540108f90 0x7f0540109410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.317+0000 7f0544e00640 1 --2- 172.21.15.69:0/3862750490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0540107e40 0x7f0540108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.317+0000 7f0544e00640 1 -- 172.21.15.69:0/3862750490 >> 172.21.15.69:0/3862750490 conn(0x7f0540076de0 msgr2=0x7f0540079220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:30.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.317+0000 7f0544e00640 1 -- 172.21.15.69:0/3862750490 shutdown_connections 2024-03-19T01:06:30.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.317+0000 7f0544e00640 1 -- 172.21.15.69:0/3862750490 wait complete. 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f0544e00640 1 Processor -- start 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f0544e00640 1 -- start start 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f0544e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0540107e40 0x7f054019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f0544e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0540108f90 0x7f054019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f0544e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0540109950 0x7f05401a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f0544e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0540113ac0 con 0x7f0540108f90 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f0544e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0540113940 con 0x7f0540107e40 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f0544e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0540113c40 con 0x7f0540109950 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f0535400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0540108f90 0x7f054019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f0535400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0540108f90 0x7f054019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51054/0 (socket says 172.21.15.69:51054) 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f0535400640 1 -- 172.21.15.69:0/2719219431 learned_addr learned my addr 172.21.15.69:0/2719219431 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f053de00640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0540107e40 0x7f054019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.318+0000 7f053e800640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0540109950 0x7f05401a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f0535400640 1 -- 172.21.15.69:0/2719219431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0540109950 msgr2=0x7f05401a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f0535400640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0540109950 0x7f05401a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f0535400640 1 -- 172.21.15.69:0/2719219431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0540107e40 msgr2=0x7f054019e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f0535400640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0540107e40 0x7f054019e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f0535400640 1 -- 172.21.15.69:0/2719219431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05401a5d80 con 0x7f0540108f90 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f053e800640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0540109950 0x7f05401a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f0535400640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0540108f90 0x7f054019e6a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f052802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f0537400640 1 -- 172.21.15.69:0/2719219431 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528041070 con 0x7f0540108f90 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f0544e00640 1 -- 172.21.15.69:0/2719219431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05401a6010 con 0x7f0540108f90 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f0537400640 1 -- 172.21.15.69:0/2719219431 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0528004450 con 0x7f0540108f90 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f0537400640 1 -- 172.21.15.69:0/2719219431 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528034440 con 0x7f0540108f90 2024-03-19T01:06:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.319+0000 7f0544e00640 1 -- 172.21.15.69:0/2719219431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05401a6550 con 0x7f0540108f90 2024-03-19T01:06:30.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.321+0000 7f0544e00640 1 -- 172.21.15.69:0/2719219431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054010f150 con 0x7f0540108f90 2024-03-19T01:06:30.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.323+0000 7f0537400640 1 -- 172.21.15.69:0/2719219431 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0528044070 con 0x7f0540108f90 2024-03-19T01:06:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.324+0000 7f0537400640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f05240a2800 0x7f05240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.324+0000 7f0537400640 1 -- 172.21.15.69:0/2719219431 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f05280f0330 con 0x7f0540108f90 2024-03-19T01:06:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.324+0000 7f053de00640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f05240a2800 0x7f05240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.325+0000 7f053de00640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f05240a2800 0x7f05240a4cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0530009bd0 tx=0x7f0530005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.327+0000 7f0537400640 1 -- 172.21.15.69:0/2719219431 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f05280bd210 con 0x7f0540108f90 2024-03-19T01:06:30.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.527+0000 7f0544e00640 1 -- 172.21.15.69:0/2719219431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f054002d1f0 con 0x7f0540108f90 2024-03-19T01:06:30.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.528+0000 7f0537400640 1 -- 172.21.15.69:0/2719219431 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f05280c12b0 con 0x7f0540108f90 2024-03-19T01:06:30.529 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:06:30.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 -- 172.21.15.69:0/2719219431 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f05240a2800 msgr2=0x7f05240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:30.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f05240a2800 0x7f05240a4cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0530009bd0 tx=0x7f0530005f90 comp rx=0 tx=0).stop 2024-03-19T01:06:30.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 -- 172.21.15.69:0/2719219431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0540108f90 msgr2=0x7f054019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:30.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0540108f90 0x7f054019e6a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f052802c640 comp rx=0 tx=0).stop 2024-03-19T01:06:30.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 -- 172.21.15.69:0/2719219431 shutdown_connections 2024-03-19T01:06:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f05240a2800 0x7f05240a4cc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0540109950 0x7f05401a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0540108f90 0x7f054019e6a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 --2- 172.21.15.69:0/2719219431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0540107e40 0x7f054019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 -- 172.21.15.69:0/2719219431 >> 172.21.15.69:0/2719219431 conn(0x7f0540076de0 msgr2=0x7f054010c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 -- 172.21.15.69:0/2719219431 shutdown_connections 2024-03-19T01:06:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.531+0000 7f0544e00640 1 -- 172.21.15.69:0/2719219431 wait complete. 2024-03-19T01:06:30.553 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:30.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.764+0000 7fc434600640 1 -- 172.21.15.69:0/1361572403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c103a60 msgr2=0x7fc42c103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:30.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.764+0000 7fc434600640 1 --2- 172.21.15.69:0/1361572403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c103a60 0x7fc42c103e80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc420009860 tx=0x7fc42002bb70 comp rx=0 tx=0).stop 2024-03-19T01:06:30.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.764+0000 7fc434600640 1 -- 172.21.15.69:0/1361572403 shutdown_connections 2024-03-19T01:06:30.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.764+0000 7fc434600640 1 --2- 172.21.15.69:0/1361572403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c105610 0x7fc42c10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.764+0000 7fc434600640 1 --2- 172.21.15.69:0/1361572403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42c104c50 0x7fc42c1050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.764+0000 7fc434600640 1 --2- 172.21.15.69:0/1361572403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c103a60 0x7fc42c103e80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.764+0000 7fc434600640 1 -- 172.21.15.69:0/1361572403 >> 172.21.15.69:0/1361572403 conn(0x7fc42c0ff010 msgr2=0x7fc42c101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:30.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.764+0000 7fc434600640 1 -- 172.21.15.69:0/1361572403 shutdown_connections 2024-03-19T01:06:30.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.764+0000 7fc434600640 1 -- 172.21.15.69:0/1361572403 wait complete. 2024-03-19T01:06:30.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.765+0000 7fc434600640 1 Processor -- start 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.765+0000 7fc434600640 1 -- start start 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc434600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42c103a60 0x7fc42c199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc434600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c104c50 0x7fc42c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc434600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c105610 0x7fc42c1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc434600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c10f6b0 con 0x7fc42c103a60 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc434600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c10f530 con 0x7fc42c104c50 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc434600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c10f830 con 0x7fc42c105610 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc431c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42c103a60 0x7fc42c199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc431c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42c103a60 0x7fc42c199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51068/0 (socket says 172.21.15.69:51068) 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc431c00640 1 -- 172.21.15.69:0/4185493051 learned_addr learned my addr 172.21.15.69:0/4185493051 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc431200640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c104c50 0x7fc42c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc432600640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c105610 0x7fc42c1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:30.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc431c00640 1 -- 172.21.15.69:0/4185493051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c105610 msgr2=0x7fc42c1a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:30.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc431c00640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c105610 0x7fc42c1a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc431c00640 1 -- 172.21.15.69:0/4185493051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c104c50 msgr2=0x7fc42c19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:30.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc431c00640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c104c50 0x7fc42c19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:30.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc431c00640 1 -- 172.21.15.69:0/4185493051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c1a1980 con 0x7fc42c103a60 2024-03-19T01:06:30.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.766+0000 7fc431200640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c104c50 0x7fc42c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:06:30.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.767+0000 7fc431c00640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42c103a60 0x7fc42c199d60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc4200349e0 tx=0x7fc420032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:30.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.767+0000 7fc42a000640 1 -- 172.21.15.69:0/4185493051 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420041020 con 0x7fc42c103a60 2024-03-19T01:06:30.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.767+0000 7fc42a000640 1 -- 172.21.15.69:0/4185493051 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc42002c9f0 con 0x7fc42c103a60 2024-03-19T01:06:30.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.767+0000 7fc434600640 1 -- 172.21.15.69:0/4185493051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c1a1c10 con 0x7fc42c103a60 2024-03-19T01:06:30.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.767+0000 7fc42a000640 1 -- 172.21.15.69:0/4185493051 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420039040 con 0x7fc42c103a60 2024-03-19T01:06:30.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.767+0000 7fc434600640 1 -- 172.21.15.69:0/4185493051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1a2100 con 0x7fc42c103a60 2024-03-19T01:06:30.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.768+0000 7fc434600640 1 -- 172.21.15.69:0/4185493051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c10ae00 con 0x7fc42c103a60 2024-03-19T01:06:30.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.770+0000 7fc42a000640 1 -- 172.21.15.69:0/4185493051 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc420044020 con 0x7fc42c103a60 2024-03-19T01:06:30.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.771+0000 7fc42a000640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4140a2800 0x7fc4140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:30.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.772+0000 7fc42a000640 1 -- 172.21.15.69:0/4185493051 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc4200efe90 con 0x7fc42c103a60 2024-03-19T01:06:30.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.772+0000 7fc431200640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4140a2800 0x7fc4140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:30.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.772+0000 7fc431200640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4140a2800 0x7fc4140a4cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc42c19b270 tx=0x7fc418009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:30.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:30.776+0000 7fc42a000640 1 -- 172.21.15.69:0/4185493051 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc42004b020 con 0x7fc42c103a60 2024-03-19T01:06:31.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.015+0000 7fc434600640 1 -- 172.21.15.69:0/4185493051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc42c02d1f0 con 0x7fc42c103a60 2024-03-19T01:06:31.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.016+0000 7fc42a000640 1 -- 172.21.15.69:0/4185493051 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fc4200ef090 con 0x7fc42c103a60 2024-03-19T01:06:31.017 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:31.017 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-19T01:06:31.017 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 1 2024-03-19T01:06:31.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.018+0000 7fc434600640 1 -- 172.21.15.69:0/4185493051 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4140a2800 msgr2=0x7fc4140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:31.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.018+0000 7fc434600640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4140a2800 0x7fc4140a4cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc42c19b270 tx=0x7fc418009260 comp rx=0 tx=0).stop 2024-03-19T01:06:31.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.018+0000 7fc434600640 1 -- 172.21.15.69:0/4185493051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42c103a60 msgr2=0x7fc42c199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:31.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.018+0000 7fc434600640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42c103a60 0x7fc42c199d60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc4200349e0 tx=0x7fc420032430 comp rx=0 tx=0).stop 2024-03-19T01:06:31.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.018+0000 7fc434600640 1 -- 172.21.15.69:0/4185493051 shutdown_connections 2024-03-19T01:06:31.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.018+0000 7fc434600640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4140a2800 0x7fc4140a4cc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:31.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.018+0000 7fc434600640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c105610 0x7fc42c1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:31.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.018+0000 7fc434600640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c104c50 0x7fc42c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:31.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.018+0000 7fc434600640 1 --2- 172.21.15.69:0/4185493051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42c103a60 0x7fc42c199d60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:31.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.018+0000 7fc434600640 1 -- 172.21.15.69:0/4185493051 >> 172.21.15.69:0/4185493051 conn(0x7fc42c0ff010 msgr2=0x7fc42c107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:31.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.018+0000 7fc434600640 1 -- 172.21.15.69:0/4185493051 shutdown_connections 2024-03-19T01:06:31.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:31.019+0000 7fc434600640 1 -- 172.21.15.69:0/4185493051 wait complete. 2024-03-19T01:06:31.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:30 smithi163 ceph-mon[67428]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:31.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:30 smithi163 ceph-mon[67428]: Deploying daemon mds.l on smithi163 2024-03-19T01:06:31.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:30 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1327659071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:06:31.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:30 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2719219431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:06:31.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:30 smithi086 ceph-mon[66829]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:31.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:30 smithi086 ceph-mon[66829]: Deploying daemon mds.l on smithi163 2024-03-19T01:06:31.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:30 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1327659071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:06:31.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:30 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2719219431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:06:31.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:30 smithi069 ceph-mon[31727]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:31.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:30 smithi069 ceph-mon[31727]: Deploying daemon mds.l on smithi163 2024-03-19T01:06:31.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:30 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1327659071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:06:31.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:30 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2719219431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:06:32.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:31 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/4185493051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:32.061 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:06:31 smithi163 systemd[1]: Starting Ceph mds.l for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:06:32.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:31 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/4185493051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:32.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:31 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/4185493051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:32.434 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:06:32 smithi163 podman[97933]: 2024-03-19 01:06:32.069268539 +0000 UTC m=+0.017679054 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:06:32.434 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:06:32 smithi163 podman[97933]: 2024-03-19 01:06:32.18992159 +0000 UTC m=+0.138332087 container create d46aed8c7f24f75d1d0b06217a6bfa6aeb08910df7aa2ebafbd764ad3122fd11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-19T01:06:32.434 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:06:32 smithi163 podman[97933]: 2024-03-19 01:06:32.434130631 +0000 UTC m=+0.382541153 container init d46aed8c7f24f75d1d0b06217a6bfa6aeb08910df7aa2ebafbd764ad3122fd11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:06:32.814 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:06:32 smithi163 podman[97933]: 2024-03-19 01:06:32.436710172 +0000 UTC m=+0.385120679 container start d46aed8c7f24f75d1d0b06217a6bfa6aeb08910df7aa2ebafbd764ad3122fd11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD) 2024-03-19T01:06:32.814 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-mds[97952]: -- 172.21.15.163:0/1161589296 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563998e83a40 con 0x563998e57000 2024-03-19T01:06:32.814 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l[97948]: starting mds.l at 2024-03-19T01:06:32.814 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:06:32 smithi163 bash[97933]: d46aed8c7f24f75d1d0b06217a6bfa6aeb08910df7aa2ebafbd764ad3122fd11 2024-03-19T01:06:32.814 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:06:32 smithi163 systemd[1]: Started Ceph mds.l for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:06:33.040 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:33.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:32 smithi069 ceph-mon[31727]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:33.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:32 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.163:6832/1468679776,v1:172.21.15.163:6833/1468679776] up:boot 2024-03-19T01:06:33.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:32 smithi069 ceph-mon[31727]: fsmap 1 up:standby 2024-03-19T01:06:33.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:06:33.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:33.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:33.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:33.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:33.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:33.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:32 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:32 smithi086 ceph-mon[66829]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:32 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.163:6832/1468679776,v1:172.21.15.163:6833/1468679776] up:boot 2024-03-19T01:06:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:32 smithi086 ceph-mon[66829]: fsmap 1 up:standby 2024-03-19T01:06:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:32 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:06:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:32 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:32 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:32 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:32 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:32 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:32 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.264+0000 7f0329800640 1 -- 172.21.15.69:0/1479041745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03241022e0 msgr2=0x7f03241046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.264+0000 7f0329800640 1 --2- 172.21.15.69:0/1479041745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03241022e0 0x7f03241046d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f031c009be0 tx=0x7f031c02bef0 comp rx=0 tx=0).stop 2024-03-19T01:06:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.264+0000 7f0329800640 1 -- 172.21.15.69:0/1479041745 shutdown_connections 2024-03-19T01:06:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.265+0000 7f0329800640 1 --2- 172.21.15.69:0/1479041745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0324107540 0x7f0324109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.265+0000 7f0329800640 1 --2- 172.21.15.69:0/1479041745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0324104c10 0x7f0324107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:33.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.265+0000 7f0329800640 1 --2- 172.21.15.69:0/1479041745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03241022e0 0x7f03241046d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:33.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.265+0000 7f0329800640 1 -- 172.21.15.69:0/1479041745 >> 172.21.15.69:0/1479041745 conn(0x7f03240fbcd0 msgr2=0x7f03240fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:33.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.265+0000 7f0329800640 1 -- 172.21.15.69:0/1479041745 shutdown_connections 2024-03-19T01:06:33.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.265+0000 7f0329800640 1 -- 172.21.15.69:0/1479041745 wait complete. 2024-03-19T01:06:33.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.266+0000 7f0329800640 1 Processor -- start 2024-03-19T01:06:33.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.266+0000 7f0329800640 1 -- start start 2024-03-19T01:06:33.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.266+0000 7f0329800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03241022e0 0x7f0324101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:33.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.266+0000 7f0329800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0324104c10 0x7f0324100290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:33.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.266+0000 7f0329800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0324107540 0x7f03241007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:33.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.266+0000 7f0329800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f032410d4e0 con 0x7f0324104c10 2024-03-19T01:06:33.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.266+0000 7f0329800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f032410d360 con 0x7f0324107540 2024-03-19T01:06:33.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.266+0000 7f0329800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f032410d660 con 0x7f03241022e0 2024-03-19T01:06:33.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.266+0000 7f0322a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03241022e0 0x7f0324101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:33.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0322a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03241022e0 0x7f0324101c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48490/0 (socket says 172.21.15.69:48490) 2024-03-19T01:06:33.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0322a00640 1 -- 172.21.15.69:0/4085850361 learned_addr learned my addr 172.21.15.69:0/4085850361 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0323400640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0324107540 0x7f03241007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0322000640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0324104c10 0x7f0324100290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0323400640 1 -- 172.21.15.69:0/4085850361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03241022e0 msgr2=0x7f0324101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0323400640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03241022e0 0x7f0324101c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0323400640 1 -- 172.21.15.69:0/4085850361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0324104c10 msgr2=0x7f0324100290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0323400640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0324104c10 0x7f0324100290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0323400640 1 -- 172.21.15.69:0/4085850361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0324101060 con 0x7f0324107540 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0322000640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0324104c10 0x7f0324100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0322a00640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03241022e0 0x7f0324101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.267+0000 7f0323400640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0324107540 0x7f03241007d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f031400e970 tx=0x7f031400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.268+0000 7f031b400640 1 -- 172.21.15.69:0/4085850361 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031400ccc0 con 0x7f0324107540 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.268+0000 7f031b400640 1 -- 172.21.15.69:0/4085850361 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0314005020 con 0x7f0324107540 2024-03-19T01:06:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.268+0000 7f0329800640 1 -- 172.21.15.69:0/4085850361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03241a1cd0 con 0x7f0324107540 2024-03-19T01:06:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.268+0000 7f031b400640 1 -- 172.21.15.69:0/4085850361 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314004a00 con 0x7f0324107540 2024-03-19T01:06:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.268+0000 7f0329800640 1 -- 172.21.15.69:0/4085850361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03241a21d0 con 0x7f0324107540 2024-03-19T01:06:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.269+0000 7f0329800640 1 -- 172.21.15.69:0/4085850361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03240fd7c0 con 0x7f0324107540 2024-03-19T01:06:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.271+0000 7f031b400640 1 -- 172.21.15.69:0/4085850361 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f03140051d0 con 0x7f0324107540 2024-03-19T01:06:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.272+0000 7f031b400640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03080a2880 0x7f03080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.273+0000 7f031b400640 1 -- 172.21.15.69:0/4085850361 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f03140ce600 con 0x7f0324107540 2024-03-19T01:06:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.273+0000 7f0322a00640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03080a2880 0x7f03080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.273+0000 7f0322a00640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03080a2880 0x7f03080a4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f031c0099a0 tx=0x7f031c004a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.276+0000 7f031b400640 1 -- 172.21.15.69:0/4085850361 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f031409baa0 con 0x7f0324107540 2024-03-19T01:06:33.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-mon[67428]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:33.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.163:6832/1468679776,v1:172.21.15.163:6833/1468679776] up:boot 2024-03-19T01:06:33.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-mon[67428]: fsmap 1 up:standby 2024-03-19T01:06:33.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:06:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:32 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:33.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.514+0000 7f0329800640 1 -- 172.21.15.69:0/4085850361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f032402d1f0 con 0x7f0324107540 2024-03-19T01:06:33.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.515+0000 7f031b400640 1 -- 172.21.15.69:0/4085850361 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f031409b3a0 con 0x7f0324107540 2024-03-19T01:06:33.516 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:33.516 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-19T01:06:33.516 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 2 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.517+0000 7f0329800640 1 -- 172.21.15.69:0/4085850361 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03080a2880 msgr2=0x7f03080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.517+0000 7f0329800640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03080a2880 0x7f03080a4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f031c0099a0 tx=0x7f031c004a80 comp rx=0 tx=0).stop 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.517+0000 7f0329800640 1 -- 172.21.15.69:0/4085850361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0324107540 msgr2=0x7f03241007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.517+0000 7f0329800640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0324107540 0x7f03241007d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f031400e970 tx=0x7f031400ee30 comp rx=0 tx=0).stop 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.518+0000 7f0329800640 1 -- 172.21.15.69:0/4085850361 shutdown_connections 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.518+0000 7f0329800640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03080a2880 0x7f03080a4d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.518+0000 7f0329800640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0324107540 0x7f03241007d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.518+0000 7f0329800640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0324104c10 0x7f0324100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.518+0000 7f0329800640 1 --2- 172.21.15.69:0/4085850361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03241022e0 0x7f0324101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.518+0000 7f0329800640 1 -- 172.21.15.69:0/4085850361 >> 172.21.15.69:0/4085850361 conn(0x7f03240fbcd0 msgr2=0x7f0324102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.518+0000 7f0329800640 1 -- 172.21.15.69:0/4085850361 shutdown_connections 2024-03-19T01:06:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:33.518+0000 7f0329800640 1 -- 172.21.15.69:0/4085850361 wait complete. 2024-03-19T01:06:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:33 smithi086 ceph-mon[66829]: Deploying daemon mds.k on smithi086 2024-03-19T01:06:34.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:33 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/4085850361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:33 smithi069 ceph-mon[31727]: Deploying daemon mds.k on smithi086 2024-03-19T01:06:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:33 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/4085850361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:34.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:33 smithi163 ceph-mon[67428]: Deploying daemon mds.k on smithi086 2024-03-19T01:06:34.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:33 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/4085850361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:34.983 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:34 smithi086 ceph-mon[66829]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:35.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:34 smithi069 ceph-mon[31727]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:35.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:34 smithi163 ceph-mon[67428]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:35.536 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:35.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.730+0000 7fa449200640 1 -- 172.21.15.69:0/1537857790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa444101690 msgr2=0x7fa444101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:35.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.730+0000 7fa449200640 1 --2- 172.21.15.69:0/1537857790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa444101690 0x7fa444101ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa434009900 tx=0x7fa43402bc30 comp rx=0 tx=0).stop 2024-03-19T01:06:35.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.730+0000 7fa449200640 1 -- 172.21.15.69:0/1537857790 shutdown_connections 2024-03-19T01:06:35.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.730+0000 7fa449200640 1 --2- 172.21.15.69:0/1537857790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444103240 0x7fa444109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:35.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.730+0000 7fa449200640 1 --2- 172.21.15.69:0/1537857790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444102880 0x7fa444102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:35.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.730+0000 7fa449200640 1 --2- 172.21.15.69:0/1537857790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa444101690 0x7fa444101ab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:35.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.730+0000 7fa449200640 1 -- 172.21.15.69:0/1537857790 >> 172.21.15.69:0/1537857790 conn(0x7fa4440fcc00 msgr2=0x7fa4440ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:35.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.730+0000 7fa449200640 1 -- 172.21.15.69:0/1537857790 shutdown_connections 2024-03-19T01:06:35.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.730+0000 7fa449200640 1 -- 172.21.15.69:0/1537857790 wait complete. 2024-03-19T01:06:35.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.731+0000 7fa449200640 1 Processor -- start 2024-03-19T01:06:35.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.731+0000 7fa449200640 1 -- start start 2024-03-19T01:06:35.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.731+0000 7fa449200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444101690 0x7fa4440755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:35.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa449200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444102880 0x7fa444073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:35.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa449200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa444103240 0x7fa444074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa449200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa44410d360 con 0x7fa444103240 2024-03-19T01:06:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa449200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa44410d1e0 con 0x7fa444101690 2024-03-19T01:06:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa449200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa44410d4e0 con 0x7fa444102880 2024-03-19T01:06:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa442a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444102880 0x7fa444073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa443e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa444103240 0x7fa444074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa442a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444102880 0x7fa444073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48520/0 (socket says 172.21.15.69:48520) 2024-03-19T01:06:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa442a00640 1 -- 172.21.15.69:0/2989041923 learned_addr learned my addr 172.21.15.69:0/2989041923 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa443400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444101690 0x7fa4440755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa443400640 1 -- 172.21.15.69:0/2989041923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444102880 msgr2=0x7fa444073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa443400640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444102880 0x7fa444073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa443400640 1 -- 172.21.15.69:0/2989041923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa444103240 msgr2=0x7fa444074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa443400640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa444103240 0x7fa444074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa443400640 1 -- 172.21.15.69:0/2989041923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4440749e0 con 0x7fa444101690 2024-03-19T01:06:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.732+0000 7fa442a00640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444102880 0x7fa444073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:06:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.733+0000 7fa443400640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444101690 0x7fa4440755d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa434009230 tx=0x7fa43402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.733+0000 7fa43be00640 1 -- 172.21.15.69:0/2989041923 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434043020 con 0x7fa444101690 2024-03-19T01:06:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.733+0000 7fa43be00640 1 -- 172.21.15.69:0/2989041923 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4340054e0 con 0x7fa444101690 2024-03-19T01:06:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.733+0000 7fa43be00640 1 -- 172.21.15.69:0/2989041923 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434003b60 con 0x7fa444101690 2024-03-19T01:06:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.733+0000 7fa449200640 1 -- 172.21.15.69:0/2989041923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4441a1ac0 con 0x7fa444101690 2024-03-19T01:06:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.733+0000 7fa449200640 1 -- 172.21.15.69:0/2989041923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4441a1f90 con 0x7fa444101690 2024-03-19T01:06:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.734+0000 7fa449200640 1 -- 172.21.15.69:0/2989041923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa444108950 con 0x7fa444101690 2024-03-19T01:06:35.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.736+0000 7fa43be00640 1 -- 172.21.15.69:0/2989041923 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa434046020 con 0x7fa444101690 2024-03-19T01:06:35.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.737+0000 7fa43be00640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa4280a2880 0x7fa4280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:35.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.737+0000 7fa43be00640 1 -- 172.21.15.69:0/2989041923 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa4340f05b0 con 0x7fa444101690 2024-03-19T01:06:35.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.737+0000 7fa442a00640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa4280a2880 0x7fa4280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:35.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.738+0000 7fa442a00640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa4280a2880 0x7fa4280a4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa444075340 tx=0x7fa430005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:35.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.741+0000 7fa43be00640 1 -- 172.21.15.69:0/2989041923 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa43404d020 con 0x7fa444101690 2024-03-19T01:06:35.756 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:06:35 smithi086 systemd[1]: Started Ceph mds.k for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:06:35.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.984+0000 7fa449200640 1 -- 172.21.15.69:0/2989041923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa444062420 con 0x7fa444101690 2024-03-19T01:06:35.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.985+0000 7fa43be00640 1 -- 172.21.15.69:0/2989041923 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fa4340bda50 con 0x7fa444101690 2024-03-19T01:06:35.986 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:35.986 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-19T01:06:35.986 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 3 2024-03-19T01:06:35.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 -- 172.21.15.69:0/2989041923 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa4280a2880 msgr2=0x7fa4280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:35.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa4280a2880 0x7fa4280a4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa444075340 tx=0x7fa430005f90 comp rx=0 tx=0).stop 2024-03-19T01:06:35.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 -- 172.21.15.69:0/2989041923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444101690 msgr2=0x7fa4440755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:35.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444101690 0x7fa4440755d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa434009230 tx=0x7fa43402c5d0 comp rx=0 tx=0).stop 2024-03-19T01:06:35.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 -- 172.21.15.69:0/2989041923 shutdown_connections 2024-03-19T01:06:35.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa4280a2880 0x7fa4280a4d40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:35.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa444103240 0x7fa444074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:35.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444102880 0x7fa444073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:35.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 --2- 172.21.15.69:0/2989041923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444101690 0x7fa4440755d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:35.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 -- 172.21.15.69:0/2989041923 >> 172.21.15.69:0/2989041923 conn(0x7fa4440fcc00 msgr2=0x7fa444105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:35.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 -- 172.21.15.69:0/2989041923 shutdown_connections 2024-03-19T01:06:35.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:35.988+0000 7fa449200640 1 -- 172.21.15.69:0/2989041923 wait complete. 2024-03-19T01:06:36.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:36.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:36.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:36.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:36.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:36.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:36.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: Deploying daemon mds.j on smithi069 2024-03-19T01:06:36.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:36.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.86:6832/3047552839,v1:172.21.15.86:6833/3047552839] up:boot 2024-03-19T01:06:36.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: fsmap 2 up:standby 2024-03-19T01:06:36.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:06:36.678 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:36 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2989041923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:36.692 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:36.692 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:36.692 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:36.693 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:36.693 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:36.693 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:36.693 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: Deploying daemon mds.j on smithi069 2024-03-19T01:06:36.693 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:36.693 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.86:6832/3047552839,v1:172.21.15.86:6833/3047552839] up:boot 2024-03-19T01:06:36.693 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: fsmap 2 up:standby 2024-03-19T01:06:36.693 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:06:36.693 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:36 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2989041923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:36.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:36.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: Deploying daemon mds.j on smithi069 2024-03-19T01:06:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.86:6832/3047552839,v1:172.21.15.86:6833/3047552839] up:boot 2024-03-19T01:06:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: fsmap 2 up:standby 2024-03-19T01:06:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:06:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:36 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2989041923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:38.009 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:38.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.268+0000 7fc9d9e00640 1 -- 172.21.15.69:0/1791589196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d4109180 msgr2=0x7fc9d410fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:38.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.268+0000 7fc9d9e00640 1 --2- 172.21.15.69:0/1791589196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d4109180 0x7fc9d410fb60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009900 tx=0x7fc9c002bc80 comp rx=0 tx=0).stop 2024-03-19T01:06:38.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.271+0000 7fc9d9e00640 1 -- 172.21.15.69:0/1791589196 shutdown_connections 2024-03-19T01:06:38.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.271+0000 7fc9d9e00640 1 --2- 172.21.15.69:0/1791589196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9d41100a0 0x7fc9d4112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:38.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.271+0000 7fc9d9e00640 1 --2- 172.21.15.69:0/1791589196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d4109180 0x7fc9d410fb60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:38.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.271+0000 7fc9d9e00640 1 --2- 172.21.15.69:0/1791589196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d4108820 0x7fc9d4108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:38.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.271+0000 7fc9d9e00640 1 -- 172.21.15.69:0/1791589196 >> 172.21.15.69:0/1791589196 conn(0x7fc9d406e690 msgr2=0x7fc9d4070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:38.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.271+0000 7fc9d9e00640 1 -- 172.21.15.69:0/1791589196 shutdown_connections 2024-03-19T01:06:38.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.271+0000 7fc9d9e00640 1 -- 172.21.15.69:0/1791589196 wait complete. 2024-03-19T01:06:38.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.273+0000 7fc9d9e00640 1 Processor -- start 2024-03-19T01:06:38.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.273+0000 7fc9d9e00640 1 -- start start 2024-03-19T01:06:38.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.274+0000 7fc9d9e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d4108820 0x7fc9d4119470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:38.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.274+0000 7fc9d9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d4109180 0x7fc9d41199b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.275+0000 7fc9d9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9d41100a0 0x7fc9d411ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.275+0000 7fc9d9e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4113ae0 con 0x7fc9d4108820 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.275+0000 7fc9d9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4113960 con 0x7fc9d4109180 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.275+0000 7fc9d9e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4113c60 con 0x7fc9d41100a0 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.275+0000 7fc9d2a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d4108820 0x7fc9d4119470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.275+0000 7fc9d2a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d4108820 0x7fc9d4119470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58340/0 (socket says 172.21.15.69:58340) 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.275+0000 7fc9d2a00640 1 -- 172.21.15.69:0/2996777524 learned_addr learned my addr 172.21.15.69:0/2996777524 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.275+0000 7fc9d3400640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9d41100a0 0x7fc9d411ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.275+0000 7fc9d2000640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d4109180 0x7fc9d41199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.276+0000 7fc9d3400640 1 -- 172.21.15.69:0/2996777524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d4109180 msgr2=0x7fc9d41199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.276+0000 7fc9d3400640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d4109180 0x7fc9d41199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.276+0000 7fc9d3400640 1 -- 172.21.15.69:0/2996777524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d4108820 msgr2=0x7fc9d4119470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.276+0000 7fc9d3400640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d4108820 0x7fc9d4119470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.276+0000 7fc9d3400640 1 -- 172.21.15.69:0/2996777524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d411f080 con 0x7fc9d41100a0 2024-03-19T01:06:38.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.278+0000 7fc9d3400640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9d41100a0 0x7fc9d411ea10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc005fd0 tx=0x7fc9cc00edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:38.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.279+0000 7fc9d8a00640 1 -- 172.21.15.69:0/2996777524 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc004d20 con 0x7fc9d41100a0 2024-03-19T01:06:38.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.279+0000 7fc9d8a00640 1 -- 172.21.15.69:0/2996777524 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9cc005020 con 0x7fc9d41100a0 2024-03-19T01:06:38.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.279+0000 7fc9d8a00640 1 -- 172.21.15.69:0/2996777524 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0116d0 con 0x7fc9d41100a0 2024-03-19T01:06:38.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.279+0000 7fc9d9e00640 1 -- 172.21.15.69:0/2996777524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d411f370 con 0x7fc9d41100a0 2024-03-19T01:06:38.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.279+0000 7fc9d9e00640 1 -- 172.21.15.69:0/2996777524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d411f840 con 0x7fc9d41100a0 2024-03-19T01:06:38.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.281+0000 7fc9d9e00640 1 -- 172.21.15.69:0/2996777524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d4068610 con 0x7fc9d41100a0 2024-03-19T01:06:38.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.285+0000 7fc9d8a00640 1 -- 172.21.15.69:0/2996777524 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc9cc005540 con 0x7fc9d41100a0 2024-03-19T01:06:38.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.286+0000 7fc9d8a00640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9b80a2950 0x7fc9b80a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:38.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.287+0000 7fc9d8a00640 1 -- 172.21.15.69:0/2996777524 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc9cc0ce730 con 0x7fc9d41100a0 2024-03-19T01:06:38.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.287+0000 7fc9d2a00640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9b80a2950 0x7fc9b80a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:38.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.288+0000 7fc9d2a00640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9b80a2950 0x7fc9b80a4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40058c0 tx=0x7fc9c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:38.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.290+0000 7fc9d8a00640 1 -- 172.21.15.69:0/2996777524 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc9cc016030 con 0x7fc9d41100a0 2024-03-19T01:06:38.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.535+0000 7fc9d9e00640 1 -- 172.21.15.69:0/2996777524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9d41b7a30 con 0x7fc9d41100a0 2024-03-19T01:06:38.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.536+0000 7fc9d8a00640 1 -- 172.21.15.69:0/2996777524 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fc9cc09bbd0 con 0x7fc9d41100a0 2024-03-19T01:06:38.537 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:38.537 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-19T01:06:38.537 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 3 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.538+0000 7fc9d9e00640 1 -- 172.21.15.69:0/2996777524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9b80a2950 msgr2=0x7fc9b80a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.538+0000 7fc9d9e00640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9b80a2950 0x7fc9b80a4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40058c0 tx=0x7fc9c4005f90 comp rx=0 tx=0).stop 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.538+0000 7fc9d9e00640 1 -- 172.21.15.69:0/2996777524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9d41100a0 msgr2=0x7fc9d411ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.538+0000 7fc9d9e00640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9d41100a0 0x7fc9d411ea10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc005fd0 tx=0x7fc9cc00edd0 comp rx=0 tx=0).stop 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.538+0000 7fc9d9e00640 1 -- 172.21.15.69:0/2996777524 shutdown_connections 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.538+0000 7fc9d9e00640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9b80a2950 0x7fc9b80a4e10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.538+0000 7fc9d9e00640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9d41100a0 0x7fc9d411ea10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.538+0000 7fc9d9e00640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9d4109180 0x7fc9d41199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.538+0000 7fc9d9e00640 1 --2- 172.21.15.69:0/2996777524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9d4108820 0x7fc9d4119470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.538+0000 7fc9d9e00640 1 -- 172.21.15.69:0/2996777524 >> 172.21.15.69:0/2996777524 conn(0x7fc9d406e690 msgr2=0x7fc9d40702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.539+0000 7fc9d9e00640 1 -- 172.21.15.69:0/2996777524 shutdown_connections 2024-03-19T01:06:38.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:38.539+0000 7fc9d9e00640 1 -- 172.21.15.69:0/2996777524 wait complete. 2024-03-19T01:06:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:38 smithi069 ceph-mon[31727]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:38 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2996777524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:38 smithi086 ceph-mon[66829]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:38 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2996777524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:39.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:38 smithi163 ceph-mon[67428]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:39.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:38 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2996777524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:39.211 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:06:38 smithi069 podman[69459]: 2024-03-19 01:06:38.881118957 +0000 UTC m=+0.149545117 container create 65388de9d2b5c7888854986dbdf5b13a7273b17d046d4d6a95407a3f8e7b6878 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-j, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-19T01:06:39.211 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:06:39 smithi069 podman[69459]: 2024-03-19 01:06:39.210045787 +0000 UTC m=+0.478471949 container init 65388de9d2b5c7888854986dbdf5b13a7273b17d046d4d6a95407a3f8e7b6878 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-19T01:06:39.533 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:06:39 smithi069 podman[69459]: 2024-03-19 01:06:39.221690688 +0000 UTC m=+0.490116828 container start 65388de9d2b5c7888854986dbdf5b13a7273b17d046d4d6a95407a3f8e7b6878 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-03-19T01:06:39.533 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:06:39 smithi069 ceph-mds[69478]: -- 172.21.15.69:0/2823177593 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d02b845a40 con 0x55d02b818800 2024-03-19T01:06:39.533 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:06:39 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-j[69473]: starting mds.j at 2024-03-19T01:06:39.533 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:06:39 smithi069 bash[69459]: 65388de9d2b5c7888854986dbdf5b13a7273b17d046d4d6a95407a3f8e7b6878 2024-03-19T01:06:39.533 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:06:39 smithi069 systemd[1]: Started Ceph mds.j for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:06:40.560 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:40 smithi086 ceph-mon[66829]: Deploying daemon mds.d on smithi069 2024-03-19T01:06:40.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:40 smithi086 ceph-mon[66829]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:40.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:40 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.69:6834/184915179,v1:172.21.15.69:6835/184915179] up:boot 2024-03-19T01:06:40.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:40 smithi086 ceph-mon[66829]: fsmap 3 up:standby 2024-03-19T01:06:40.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:40 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T01:06:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:40 smithi069 ceph-mon[31727]: Deploying daemon mds.d on smithi069 2024-03-19T01:06:40.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:40 smithi069 ceph-mon[31727]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:40.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:40 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.69:6834/184915179,v1:172.21.15.69:6835/184915179] up:boot 2024-03-19T01:06:40.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:40 smithi069 ceph-mon[31727]: fsmap 3 up:standby 2024-03-19T01:06:40.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:40 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T01:06:40.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.781+0000 7fe4d3000640 1 -- 172.21.15.69:0/2968738484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4cc105610 msgr2=0x7fe4cc10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:40.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.781+0000 7fe4d3000640 1 --2- 172.21.15.69:0/2968738484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4cc105610 0x7fe4cc10bea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009ca0 tx=0x7fe4c402c100 comp rx=0 tx=0).stop 2024-03-19T01:06:40.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.781+0000 7fe4d3000640 1 -- 172.21.15.69:0/2968738484 shutdown_connections 2024-03-19T01:06:40.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.781+0000 7fe4d3000640 1 --2- 172.21.15.69:0/2968738484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4cc105610 0x7fe4cc10bea0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:40.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.781+0000 7fe4d3000640 1 --2- 172.21.15.69:0/2968738484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4cc104c50 0x7fe4cc1050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:40.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.781+0000 7fe4d3000640 1 --2- 172.21.15.69:0/2968738484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4cc103a60 0x7fe4cc103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:40.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.781+0000 7fe4d3000640 1 -- 172.21.15.69:0/2968738484 >> 172.21.15.69:0/2968738484 conn(0x7fe4cc0fefd0 msgr2=0x7fe4cc101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:40.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.781+0000 7fe4d3000640 1 -- 172.21.15.69:0/2968738484 shutdown_connections 2024-03-19T01:06:40.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.781+0000 7fe4d3000640 1 -- 172.21.15.69:0/2968738484 wait complete. 2024-03-19T01:06:40.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.782+0000 7fe4d3000640 1 Processor -- start 2024-03-19T01:06:40.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4d3000640 1 -- start start 2024-03-19T01:06:40.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4d3000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4cc103a60 0x7fe4cc199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:40.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4d3000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4cc104c50 0x7fe4cc19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:40.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4d3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4cc105610 0x7fe4cc1a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4d3000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10f6e0 con 0x7fe4cc104c50 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4d3000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10f560 con 0x7fe4cc105610 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4d3000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10f860 con 0x7fe4cc103a60 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4cbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4cc103a60 0x7fe4cc199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4cbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4cc103a60 0x7fe4cc199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:49998/0 (socket says 172.21.15.69:49998) 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4cbe00640 1 -- 172.21.15.69:0/3063121799 learned_addr learned my addr 172.21.15.69:0/3063121799 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4c3400640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4cc104c50 0x7fe4cc19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.783+0000 7fe4d1000640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4cc105610 0x7fe4cc1a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.784+0000 7fe4cbe00640 1 -- 172.21.15.69:0/3063121799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4cc105610 msgr2=0x7fe4cc1a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.784+0000 7fe4cbe00640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4cc105610 0x7fe4cc1a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.784+0000 7fe4cbe00640 1 -- 172.21.15.69:0/3063121799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4cc104c50 msgr2=0x7fe4cc19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:40.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.784+0000 7fe4cbe00640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4cc104c50 0x7fe4cc19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:40.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.784+0000 7fe4cbe00640 1 -- 172.21.15.69:0/3063121799 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc1a1950 con 0x7fe4cc103a60 2024-03-19T01:06:40.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.784+0000 7fe4cbe00640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4cc103a60 0x7fe4cc199d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00e9c0 tx=0x7fe4bc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:40.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.784+0000 7fe4c9600640 1 -- 172.21.15.69:0/3063121799 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00cd80 con 0x7fe4cc103a60 2024-03-19T01:06:40.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.784+0000 7fe4c9600640 1 -- 172.21.15.69:0/3063121799 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4bc0035f0 con 0x7fe4cc103a60 2024-03-19T01:06:40.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.784+0000 7fe4c9600640 1 -- 172.21.15.69:0/3063121799 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc010690 con 0x7fe4cc103a60 2024-03-19T01:06:40.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.784+0000 7fe4d3000640 1 -- 172.21.15.69:0/3063121799 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc1a1c40 con 0x7fe4cc103a60 2024-03-19T01:06:40.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.784+0000 7fe4d3000640 1 -- 172.21.15.69:0/3063121799 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc1a2140 con 0x7fe4cc103a60 2024-03-19T01:06:40.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.785+0000 7fe4d3000640 1 -- 172.21.15.69:0/3063121799 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc10ad20 con 0x7fe4cc103a60 2024-03-19T01:06:40.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.787+0000 7fe4c9600640 1 -- 172.21.15.69:0/3063121799 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe4bc0037a0 con 0x7fe4cc103a60 2024-03-19T01:06:40.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.788+0000 7fe4c9600640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe4ac0a2800 0x7fe4ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:40.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.789+0000 7fe4c9600640 1 -- 172.21.15.69:0/3063121799 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe4bc0cea50 con 0x7fe4cc103a60 2024-03-19T01:06:40.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.789+0000 7fe4c3400640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe4ac0a2800 0x7fe4ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:40.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.789+0000 7fe4c3400640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe4ac0a2800 0x7fe4ac0a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40095f0 tx=0x7fe4b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:40.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:40.792+0000 7fe4c9600640 1 -- 172.21.15.69:0/3063121799 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe4bc09bef0 con 0x7fe4cc103a60 2024-03-19T01:06:40.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:40.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:40.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:40.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:40.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:40.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:40.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:40 smithi163 ceph-mon[67428]: Deploying daemon mds.d on smithi069 2024-03-19T01:06:40.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:40 smithi163 ceph-mon[67428]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:40.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:40 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.69:6834/184915179,v1:172.21.15.69:6835/184915179] up:boot 2024-03-19T01:06:40.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:40 smithi163 ceph-mon[67428]: fsmap 3 up:standby 2024-03-19T01:06:40.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:40 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T01:06:41.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.024+0000 7fe4d3000640 1 -- 172.21.15.69:0/3063121799 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4cc02d1f0 con 0x7fe4cc103a60 2024-03-19T01:06:41.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.026+0000 7fe4c9600640 1 -- 172.21.15.69:0/3063121799 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7fe4bc09b7f0 con 0x7fe4cc103a60 2024-03-19T01:06:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:41.026 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-19T01:06:41.027 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 4 2024-03-19T01:06:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 -- 172.21.15.69:0/3063121799 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe4ac0a2800 msgr2=0x7fe4ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe4ac0a2800 0x7fe4ac0a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40095f0 tx=0x7fe4b4005f90 comp rx=0 tx=0).stop 2024-03-19T01:06:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 -- 172.21.15.69:0/3063121799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4cc103a60 msgr2=0x7fe4cc199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4cc103a60 0x7fe4cc199d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00e9c0 tx=0x7fe4bc00ee80 comp rx=0 tx=0).stop 2024-03-19T01:06:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 -- 172.21.15.69:0/3063121799 shutdown_connections 2024-03-19T01:06:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe4ac0a2800 0x7fe4ac0a4cc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4cc105610 0x7fe4cc1a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4cc104c50 0x7fe4cc19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 --2- 172.21.15.69:0/3063121799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4cc103a60 0x7fe4cc199d40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 -- 172.21.15.69:0/3063121799 >> 172.21.15.69:0/3063121799 conn(0x7fe4cc0fefd0 msgr2=0x7fe4cc107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 -- 172.21.15.69:0/3063121799 shutdown_connections 2024-03-19T01:06:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:41.028+0000 7fe4d3000640 1 -- 172.21.15.69:0/3063121799 wait complete. 2024-03-19T01:06:41.719 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:41 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3063121799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:41.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:41 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3063121799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:42.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:41 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3063121799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:42.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:42 smithi069 ceph-mon[31727]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:42.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:42 smithi163 ceph-mon[67428]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:43.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:42 smithi086 ceph-mon[66829]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:43.048 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:43.120 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:06:42 smithi069 podman[69805]: 2024-03-19 01:06:42.895917674 +0000 UTC m=+0.157137245 container create c66cfdc41c180d37c8f7c4cad0d0c5459fa002ba6bc246a49d16aae689c57974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:06:43.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.247+0000 7f90f7e00640 1 -- 172.21.15.69:0/1720040715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90f0105610 msgr2=0x7f90f010bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:43.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.247+0000 7f90f7e00640 1 --2- 172.21.15.69:0/1720040715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90f0105610 0x7f90f010bea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009ca0 tx=0x7f90e802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:06:43.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.247+0000 7f90f7e00640 1 -- 172.21.15.69:0/1720040715 shutdown_connections 2024-03-19T01:06:43.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.247+0000 7f90f7e00640 1 --2- 172.21.15.69:0/1720040715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90f0105610 0x7f90f010bea0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.247+0000 7f90f7e00640 1 --2- 172.21.15.69:0/1720040715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f0104c50 0x7f90f01050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.247+0000 7f90f7e00640 1 --2- 172.21.15.69:0/1720040715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0103a60 0x7f90f0103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.247+0000 7f90f7e00640 1 -- 172.21.15.69:0/1720040715 >> 172.21.15.69:0/1720040715 conn(0x7f90f00ff010 msgr2=0x7f90f0101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.247+0000 7f90f7e00640 1 -- 172.21.15.69:0/1720040715 shutdown_connections 2024-03-19T01:06:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.247+0000 7f90f7e00640 1 -- 172.21.15.69:0/1720040715 wait complete. 2024-03-19T01:06:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.248+0000 7f90f7e00640 1 Processor -- start 2024-03-19T01:06:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.248+0000 7f90f7e00640 1 -- start start 2024-03-19T01:06:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.248+0000 7f90f7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90f0103a60 0x7f90f0199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f0104c50 0x7f90f019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0105610 0x7f90f01a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f7e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f90f010f6c0 con 0x7f90f0103a60 2024-03-19T01:06:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90f010f540 con 0x7f90f0105610 2024-03-19T01:06:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f7e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f90f010f840 con 0x7f90f0104c50 2024-03-19T01:06:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f5400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90f0103a60 0x7f90f0199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f5400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90f0103a60 0x7f90f0199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58410/0 (socket says 172.21.15.69:58410) 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f5400640 1 -- 172.21.15.69:0/510310061 learned_addr learned my addr 172.21.15.69:0/510310061 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f4a00640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f0104c50 0x7f90f019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f5e00640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0105610 0x7f90f01a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f5400640 1 -- 172.21.15.69:0/510310061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f0104c50 msgr2=0x7f90f019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f5400640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f0104c50 0x7f90f019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f5400640 1 -- 172.21.15.69:0/510310061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0105610 msgr2=0x7f90f01a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f5400640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0105610 0x7f90f01a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f5400640 1 -- 172.21.15.69:0/510310061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f01a1910 con 0x7f90f0103a60 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.249+0000 7f90f5400640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90f0103a60 0x7f90f0199d00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f90e400e9c0 tx=0x7f90e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.250+0000 7f90ee000640 1 -- 172.21.15.69:0/510310061 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e400cd80 con 0x7f90f0103a60 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.250+0000 7f90ee000640 1 -- 172.21.15.69:0/510310061 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e40035f0 con 0x7f90f0103a60 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.250+0000 7f90f7e00640 1 -- 172.21.15.69:0/510310061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f01a1c00 con 0x7f90f0103a60 2024-03-19T01:06:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.250+0000 7f90ee000640 1 -- 172.21.15.69:0/510310061 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4010690 con 0x7f90f0103a60 2024-03-19T01:06:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.250+0000 7f90f7e00640 1 -- 172.21.15.69:0/510310061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f01a2150 con 0x7f90f0103a60 2024-03-19T01:06:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.251+0000 7f90f7e00640 1 -- 172.21.15.69:0/510310061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f010ad80 con 0x7f90f0103a60 2024-03-19T01:06:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.253+0000 7f90ee000640 1 -- 172.21.15.69:0/510310061 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f90e40037a0 con 0x7f90f0103a60 2024-03-19T01:06:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.254+0000 7f90ee000640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90d80a2800 0x7f90d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.254+0000 7f90ee000640 1 -- 172.21.15.69:0/510310061 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f90e40ceaa0 con 0x7f90f0103a60 2024-03-19T01:06:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.254+0000 7f90f4a00640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90d80a2800 0x7f90d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.255+0000 7f90f4a00640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90d80a2800 0x7f90d80a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00bbe0 tx=0x7f90dc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:43.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.258+0000 7f90ee000640 1 -- 172.21.15.69:0/510310061 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f90e409bf40 con 0x7f90f0103a60 2024-03-19T01:06:43.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.505+0000 7f90f7e00640 1 -- 172.21.15.69:0/510310061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90f002d1f0 con 0x7f90f0103a60 2024-03-19T01:06:43.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.506+0000 7f90ee000640 1 -- 172.21.15.69:0/510310061 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f90e409b840 con 0x7f90f0103a60 2024-03-19T01:06:43.507 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:43.507 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-19T01:06:43.507 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 4 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.508+0000 7f90f7e00640 1 -- 172.21.15.69:0/510310061 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90d80a2800 msgr2=0x7f90d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.508+0000 7f90f7e00640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90d80a2800 0x7f90d80a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00bbe0 tx=0x7f90dc00b390 comp rx=0 tx=0).stop 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.508+0000 7f90f7e00640 1 -- 172.21.15.69:0/510310061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90f0103a60 msgr2=0x7f90f0199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.508+0000 7f90f7e00640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90f0103a60 0x7f90f0199d00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f90e400e9c0 tx=0x7f90e400ee80 comp rx=0 tx=0).stop 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.508+0000 7f90f7e00640 1 -- 172.21.15.69:0/510310061 shutdown_connections 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.508+0000 7f90f7e00640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90d80a2800 0x7f90d80a4cc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.508+0000 7f90f7e00640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0105610 0x7f90f01a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.508+0000 7f90f7e00640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f0104c50 0x7f90f019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.508+0000 7f90f7e00640 1 --2- 172.21.15.69:0/510310061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90f0103a60 0x7f90f0199d00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.508+0000 7f90f7e00640 1 -- 172.21.15.69:0/510310061 >> 172.21.15.69:0/510310061 conn(0x7f90f00ff010 msgr2=0x7f90f0107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.509+0000 7f90f7e00640 1 -- 172.21.15.69:0/510310061 shutdown_connections 2024-03-19T01:06:43.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:43.509+0000 7f90f7e00640 1 -- 172.21.15.69:0/510310061 wait complete. 2024-03-19T01:06:43.525 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:06:43 smithi069 podman[69805]: 2024-03-19 01:06:43.184017135 +0000 UTC m=+0.445236716 container init c66cfdc41c180d37c8f7c4cad0d0c5459fa002ba6bc246a49d16aae689c57974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS) 2024-03-19T01:06:43.525 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:06:43 smithi069 podman[69805]: 2024-03-19 01:06:43.193596975 +0000 UTC m=+0.454816516 container start c66cfdc41c180d37c8f7c4cad0d0c5459fa002ba6bc246a49d16aae689c57974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:06:43.525 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:06:43 smithi069 ceph-mds[69844]: -- 172.21.15.69:0/1452662631 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558914b47a40 con 0x558914b1b000 2024-03-19T01:06:43.525 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:06:43 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d[69820]: starting mds.d at 2024-03-19T01:06:43.526 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:06:43 smithi069 bash[69805]: c66cfdc41c180d37c8f7c4cad0d0c5459fa002ba6bc246a49d16aae689c57974 2024-03-19T01:06:43.526 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:06:43 smithi069 systemd[1]: Started Ceph mds.d for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:06:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: Deploying daemon mds.e on smithi086 2024-03-19T01:06:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/510310061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] up:boot 2024-03-19T01:06:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: fsmap 4 up:standby 2024-03-19T01:06:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:06:44.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:44 smithi086 ceph-mon[66829]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:44.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:44.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:44.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:44.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:44.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:44.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:44.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: Deploying daemon mds.e on smithi086 2024-03-19T01:06:44.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/510310061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:44.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] up:boot 2024-03-19T01:06:44.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: fsmap 4 up:standby 2024-03-19T01:06:44.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:06:44.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:44 smithi069 ceph-mon[31727]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:44.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:44.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:44.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:44.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:44.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:44.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:44.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: Deploying daemon mds.e on smithi086 2024-03-19T01:06:44.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/510310061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:44.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] up:boot 2024-03-19T01:06:44.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: fsmap 4 up:standby 2024-03-19T01:06:44.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:06:44.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:44 smithi163 ceph-mon[67428]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:45.528 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.744+0000 7fec37e00640 1 -- 172.21.15.69:0/1978845196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec301096f0 msgr2=0x7fec3010bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.744+0000 7fec37e00640 1 --2- 172.21.15.69:0/1978845196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec301096f0 0x7fec3010bb80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec28009ca0 tx=0x7fec2802c190 comp rx=0 tx=0).stop 2024-03-19T01:06:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.744+0000 7fec37e00640 1 -- 172.21.15.69:0/1978845196 shutdown_connections 2024-03-19T01:06:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.744+0000 7fec37e00640 1 --2- 172.21.15.69:0/1978845196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec301096f0 0x7fec3010bb80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.744+0000 7fec37e00640 1 --2- 172.21.15.69:0/1978845196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec30074200 0x7fec30074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.744+0000 7fec37e00640 1 --2- 172.21.15.69:0/1978845196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec300757e0 0x7fec30073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.744+0000 7fec37e00640 1 -- 172.21.15.69:0/1978845196 >> 172.21.15.69:0/1978845196 conn(0x7fec300fcdd0 msgr2=0x7fec300ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.744+0000 7fec37e00640 1 -- 172.21.15.69:0/1978845196 shutdown_connections 2024-03-19T01:06:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.744+0000 7fec37e00640 1 -- 172.21.15.69:0/1978845196 wait complete. 2024-03-19T01:06:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.745+0000 7fec37e00640 1 Processor -- start 2024-03-19T01:06:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.745+0000 7fec37e00640 1 -- start start 2024-03-19T01:06:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.745+0000 7fec37e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec30074200 0x7fec30199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.745+0000 7fec37e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec300757e0 0x7fec3019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.745+0000 7fec37e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec301096f0 0x7fec301a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.745+0000 7fec37e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec3010f6d0 con 0x7fec300757e0 2024-03-19T01:06:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.745+0000 7fec37e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec3010f550 con 0x7fec301096f0 2024-03-19T01:06:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.745+0000 7fec37e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fec3010f850 con 0x7fec30074200 2024-03-19T01:06:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec34a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec300757e0 0x7fec3019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec35e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec301096f0 0x7fec301a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec35e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec301096f0 0x7fec301a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38178/0 (socket says 172.21.15.69:38178) 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec35e00640 1 -- 172.21.15.69:0/2832958644 learned_addr learned my addr 172.21.15.69:0/2832958644 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec35400640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec30074200 0x7fec30199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec35e00640 1 -- 172.21.15.69:0/2832958644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec30074200 msgr2=0x7fec30199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec35e00640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec30074200 0x7fec30199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec35e00640 1 -- 172.21.15.69:0/2832958644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec300757e0 msgr2=0x7fec3019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec35e00640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec300757e0 0x7fec3019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec35e00640 1 -- 172.21.15.69:0/2832958644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec301a1900 con 0x7fec301096f0 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec34a00640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec300757e0 0x7fec3019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec35e00640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec301096f0 0x7fec301a1260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec28037040 tx=0x7fec28009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.746+0000 7fec2e000640 1 -- 172.21.15.69:0/2832958644 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28039040 con 0x7fec301096f0 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.747+0000 7fec37e00640 1 -- 172.21.15.69:0/2832958644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec301a1b90 con 0x7fec301096f0 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.747+0000 7fec2e000640 1 -- 172.21.15.69:0/2832958644 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec28044070 con 0x7fec301096f0 2024-03-19T01:06:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.747+0000 7fec2e000640 1 -- 172.21.15.69:0/2832958644 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec280043b0 con 0x7fec301096f0 2024-03-19T01:06:45.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.747+0000 7fec37e00640 1 -- 172.21.15.69:0/2832958644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec301a2080 con 0x7fec301096f0 2024-03-19T01:06:45.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.748+0000 7fec37e00640 1 -- 172.21.15.69:0/2832958644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec301084b0 con 0x7fec301096f0 2024-03-19T01:06:45.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.749+0000 7fec2e000640 1 -- 172.21.15.69:0/2832958644 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fec28004550 con 0x7fec301096f0 2024-03-19T01:06:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.751+0000 7fec2e000640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fec180a2800 0x7fec180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.751+0000 7fec2e000640 1 -- 172.21.15.69:0/2832958644 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fec280f0bf0 con 0x7fec301096f0 2024-03-19T01:06:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.751+0000 7fec35400640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fec180a2800 0x7fec180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.752+0000 7fec35400640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fec180a2800 0x7fec180a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec3019b1c0 tx=0x7fec24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:45.755 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:06:45 smithi086 systemd[1]: Starting Ceph mds.e for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:06:45.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.756+0000 7fec2e000640 1 -- 172.21.15.69:0/2832958644 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fec280be130 con 0x7fec301096f0 2024-03-19T01:06:45.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.993+0000 7fec37e00640 1 -- 172.21.15.69:0/2832958644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec3002d1f0 con 0x7fec301096f0 2024-03-19T01:06:45.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.995+0000 7fec2e000640 1 -- 172.21.15.69:0/2832958644 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fec280bda30 con 0x7fec301096f0 2024-03-19T01:06:45.997 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:45.997 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-03-19T01:06:45.997 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 5 2024-03-19T01:06:45.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.998+0000 7fec37e00640 1 -- 172.21.15.69:0/2832958644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fec180a2800 msgr2=0x7fec180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:46.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.998+0000 7fec37e00640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fec180a2800 0x7fec180a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec3019b1c0 tx=0x7fec24005f90 comp rx=0 tx=0).stop 2024-03-19T01:06:46.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.998+0000 7fec37e00640 1 -- 172.21.15.69:0/2832958644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec301096f0 msgr2=0x7fec301a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:46.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.998+0000 7fec37e00640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec301096f0 0x7fec301a1260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec28037040 tx=0x7fec28009360 comp rx=0 tx=0).stop 2024-03-19T01:06:46.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.999+0000 7fec37e00640 1 -- 172.21.15.69:0/2832958644 shutdown_connections 2024-03-19T01:06:46.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.999+0000 7fec37e00640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fec180a2800 0x7fec180a4cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:46.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.999+0000 7fec37e00640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec301096f0 0x7fec301a1260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:46.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.999+0000 7fec37e00640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec300757e0 0x7fec3019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:46.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.999+0000 7fec37e00640 1 --2- 172.21.15.69:0/2832958644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec30074200 0x7fec30199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:46.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.999+0000 7fec37e00640 1 -- 172.21.15.69:0/2832958644 >> 172.21.15.69:0/2832958644 conn(0x7fec300fcdd0 msgr2=0x7fec30105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:46.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.999+0000 7fec37e00640 1 -- 172.21.15.69:0/2832958644 shutdown_connections 2024-03-19T01:06:46.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:45.999+0000 7fec37e00640 1 -- 172.21.15.69:0/2832958644 wait complete. 2024-03-19T01:06:46.043 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:06:45 smithi086 podman[96986]: 2024-03-19 01:06:45.687173342 +0000 UTC m=+0.018285310 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:06:46.043 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:06:45 smithi086 podman[96986]: 2024-03-19 01:06:45.798871796 +0000 UTC m=+0.129983748 container create 26cfff715229157fcfa9d50cb293bc10d9877882871cd3163bda0dce6ac16c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True) 2024-03-19T01:06:46.044 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:06:46 smithi086 podman[96986]: 2024-03-19 01:06:46.043252825 +0000 UTC m=+0.374364779 container init 26cfff715229157fcfa9d50cb293bc10d9877882871cd3163bda0dce6ac16c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-19T01:06:46.506 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:06:46 smithi086 podman[96986]: 2024-03-19 01:06:46.045964935 +0000 UTC m=+0.377076888 container start 26cfff715229157fcfa9d50cb293bc10d9877882871cd3163bda0dce6ac16c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-19T01:06:46.506 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:06:46 smithi086 ceph-mds[97005]: -- 172.21.15.86:0/4037892132 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df096c7a40 con 0x55df0969b000 2024-03-19T01:06:46.506 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:06:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e[97001]: starting mds.e at 2024-03-19T01:06:46.506 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:06:46 smithi086 bash[96986]: 26cfff715229157fcfa9d50cb293bc10d9877882871cd3163bda0dce6ac16c55 2024-03-19T01:06:46.506 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:06:46 smithi086 systemd[1]: Started Ceph mds.e for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:06:46.938 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:46 smithi163 ceph-mon[67428]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:46.938 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:46 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2832958644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:46.938 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:46.938 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:46.938 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:46.938 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:46.938 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:46.938 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:46 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:47.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:46 smithi086 ceph-mon[66829]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:47.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:46 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2832958644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:47.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:47.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:47.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:47.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:47.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:47.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:46 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:46 smithi069 ceph-mon[31727]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:47.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:46 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2832958644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:47.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:47.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:47.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:47.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:47.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:47.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:46 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:47 smithi086 ceph-mon[66829]: Deploying daemon mds.f on smithi163 2024-03-19T01:06:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:47 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] up:boot 2024-03-19T01:06:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:47 smithi086 ceph-mon[66829]: fsmap 5 up:standby 2024-03-19T01:06:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:47 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:06:48.020 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:47 smithi069 ceph-mon[31727]: Deploying daemon mds.f on smithi163 2024-03-19T01:06:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:47 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] up:boot 2024-03-19T01:06:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:47 smithi069 ceph-mon[31727]: fsmap 5 up:standby 2024-03-19T01:06:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:47 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:06:48.041 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:47 smithi163 ceph-mon[67428]: Deploying daemon mds.f on smithi163 2024-03-19T01:06:48.041 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:47 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] up:boot 2024-03-19T01:06:48.042 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:47 smithi163 ceph-mon[67428]: fsmap 5 up:standby 2024-03-19T01:06:48.042 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:47 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:06:48.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.239+0000 7faaa0000640 1 -- 172.21.15.69:0/310739096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98109970 msgr2=0x7faa98110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:48.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.239+0000 7faaa0000640 1 --2- 172.21.15.69:0/310739096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98109970 0x7faa98110290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa90009ca0 tx=0x7faa9002c100 comp rx=0 tx=0).stop 2024-03-19T01:06:48.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.239+0000 7faaa0000640 1 -- 172.21.15.69:0/310739096 shutdown_connections 2024-03-19T01:06:48.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.239+0000 7faaa0000640 1 --2- 172.21.15.69:0/310739096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98109970 0x7faa98110290 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:48.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.239+0000 7faaa0000640 1 --2- 172.21.15.69:0/310739096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98108fb0 0x7faa98109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:48.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.239+0000 7faaa0000640 1 --2- 172.21.15.69:0/310739096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa98107e60 0x7faa98108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:48.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.239+0000 7faaa0000640 1 -- 172.21.15.69:0/310739096 >> 172.21.15.69:0/310739096 conn(0x7faa98076e30 msgr2=0x7faa98079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:48.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.239+0000 7faaa0000640 1 -- 172.21.15.69:0/310739096 shutdown_connections 2024-03-19T01:06:48.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.239+0000 7faaa0000640 1 -- 172.21.15.69:0/310739096 wait complete. 2024-03-19T01:06:48.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.240+0000 7faaa0000640 1 Processor -- start 2024-03-19T01:06:48.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.240+0000 7faaa0000640 1 -- start start 2024-03-19T01:06:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faaa0000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98107e60 0x7faa9819e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faaa0000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa98108fb0 0x7faa9819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faaa0000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98109970 0x7faa981a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faaa0000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faa98113b00 con 0x7faa98107e60 2024-03-19T01:06:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faaa0000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa98113980 con 0x7faa98108fb0 2024-03-19T01:06:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faaa0000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7faa98113c80 con 0x7faa98109970 2024-03-19T01:06:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faa9d600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98107e60 0x7faa9819e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faa9e000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98109970 0x7faa981a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faa9d600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98107e60 0x7faa9819e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55564/0 (socket says 172.21.15.69:55564) 2024-03-19T01:06:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faa9d600640 1 -- 172.21.15.69:0/3140557624 learned_addr learned my addr 172.21.15.69:0/3140557624 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faa9cc00640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa98108fb0 0x7faa9819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faa9e000640 1 -- 172.21.15.69:0/3140557624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa98108fb0 msgr2=0x7faa9819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faa9e000640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa98108fb0 0x7faa9819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faa9e000640 1 -- 172.21.15.69:0/3140557624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98107e60 msgr2=0x7faa9819e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faa9e000640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98107e60 0x7faa9819e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.241+0000 7faa9e000640 1 -- 172.21.15.69:0/3140557624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa981a5d00 con 0x7faa98109970 2024-03-19T01:06:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.242+0000 7faa9e000640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98109970 0x7faa981a5690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa9002c640 tx=0x7faa90035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.242+0000 7faa96000640 1 -- 172.21.15.69:0/3140557624 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90039040 con 0x7faa98109970 2024-03-19T01:06:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.242+0000 7faa96000640 1 -- 172.21.15.69:0/3140557624 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa90044070 con 0x7faa98109970 2024-03-19T01:06:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.242+0000 7faa96000640 1 -- 172.21.15.69:0/3140557624 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900043e0 con 0x7faa98109970 2024-03-19T01:06:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.242+0000 7faaa0000640 1 -- 172.21.15.69:0/3140557624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa981a5f90 con 0x7faa98109970 2024-03-19T01:06:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.242+0000 7faaa0000640 1 -- 172.21.15.69:0/3140557624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa981a6340 con 0x7faa98109970 2024-03-19T01:06:48.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.243+0000 7faaa0000640 1 -- 172.21.15.69:0/3140557624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9810f110 con 0x7faa98109970 2024-03-19T01:06:48.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.245+0000 7faa96000640 1 -- 172.21.15.69:0/3140557624 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7faa90004580 con 0x7faa98109970 2024-03-19T01:06:48.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.247+0000 7faa96000640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faa800a2800 0x7faa800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:48.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.247+0000 7faa96000640 1 -- 172.21.15.69:0/3140557624 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7faa900f0cc0 con 0x7faa98109970 2024-03-19T01:06:48.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.247+0000 7faa9d600640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faa800a2800 0x7faa800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:48.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.247+0000 7faa9d600640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faa800a2800 0x7faa800a4cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009cd0 tx=0x7faa8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:48.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.255+0000 7faa96000640 1 -- 172.21.15.69:0/3140557624 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faa900f7080 con 0x7faa98109970 2024-03-19T01:06:48.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.494+0000 7faaa0000640 1 -- 172.21.15.69:0/3140557624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa9802d1f0 con 0x7faa98109970 2024-03-19T01:06:48.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.495+0000 7faa96000640 1 -- 172.21.15.69:0/3140557624 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7faa900be160 con 0x7faa98109970 2024-03-19T01:06:48.496 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:48.497 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-19T01:06:48.497 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-03-19T01:06:48.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 -- 172.21.15.69:0/3140557624 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faa800a2800 msgr2=0x7faa800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:48.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faa800a2800 0x7faa800a4cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009cd0 tx=0x7faa8c005f90 comp rx=0 tx=0).stop 2024-03-19T01:06:48.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 -- 172.21.15.69:0/3140557624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98109970 msgr2=0x7faa981a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:48.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98109970 0x7faa981a5690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa9002c640 tx=0x7faa90035d90 comp rx=0 tx=0).stop 2024-03-19T01:06:48.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 -- 172.21.15.69:0/3140557624 shutdown_connections 2024-03-19T01:06:48.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faa800a2800 0x7faa800a4cc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:48.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98109970 0x7faa981a5690 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:48.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa98108fb0 0x7faa9819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:48.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 --2- 172.21.15.69:0/3140557624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98107e60 0x7faa9819e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:48.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 -- 172.21.15.69:0/3140557624 >> 172.21.15.69:0/3140557624 conn(0x7faa98076e30 msgr2=0x7faa9810c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:48.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 -- 172.21.15.69:0/3140557624 shutdown_connections 2024-03-19T01:06:48.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:48.499+0000 7faaa0000640 1 -- 172.21.15.69:0/3140557624 wait complete. 2024-03-19T01:06:48.708 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:06:48 smithi163 systemd[1]: Starting Ceph mds.f for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:06:48.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:48 smithi069 ceph-mon[31727]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:48.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:48 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3140557624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:49.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:48 smithi086 ceph-mon[66829]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:49.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:48 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3140557624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:49.038 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:48 smithi163 ceph-mon[67428]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:49.039 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:48 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3140557624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:49.039 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:06:48 smithi163 podman[98258]: 2024-03-19 01:06:48.663170242 +0000 UTC m=+0.018020422 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:06:49.039 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:06:48 smithi163 podman[98258]: 2024-03-19 01:06:48.794292246 +0000 UTC m=+0.149142411 container create 274807e936ebf9b8b294db696d65626cb987b890e9747ec1b28156d4d161a5df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-19T01:06:49.039 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:06:49 smithi163 podman[98258]: 2024-03-19 01:06:49.038585758 +0000 UTC m=+0.393435940 container init 274807e936ebf9b8b294db696d65626cb987b890e9747ec1b28156d4d161a5df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-19T01:06:49.311 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:06:49 smithi163 podman[98258]: 2024-03-19 01:06:49.041312262 +0000 UTC m=+0.396162432 container start 274807e936ebf9b8b294db696d65626cb987b890e9747ec1b28156d4d161a5df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:06:49.311 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:06:49 smithi163 ceph-mds[98277]: -- 172.21.15.163:0/2953849618 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e2c39e3a40 con 0x55e2c39b7000 2024-03-19T01:06:49.311 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:06:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f[98273]: starting mds.f at 2024-03-19T01:06:49.311 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:06:49 smithi163 bash[98258]: 274807e936ebf9b8b294db696d65626cb987b890e9747ec1b28156d4d161a5df 2024-03-19T01:06:49.311 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:06:49 smithi163 systemd[1]: Started Ceph mds.f for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:06:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:50 smithi086 ceph-mon[66829]: Deploying daemon mds.c on smithi163 2024-03-19T01:06:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:50 smithi086 ceph-mon[66829]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:50 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.163:6834/2175201817,v1:172.21.15.163:6835/2175201817] up:boot 2024-03-19T01:06:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:50 smithi086 ceph-mon[66829]: fsmap 6 up:standby 2024-03-19T01:06:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:50 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:06:50.520 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:50 smithi069 ceph-mon[31727]: Deploying daemon mds.c on smithi163 2024-03-19T01:06:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:50 smithi069 ceph-mon[31727]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:50 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.163:6834/2175201817,v1:172.21.15.163:6835/2175201817] up:boot 2024-03-19T01:06:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:50 smithi069 ceph-mon[31727]: fsmap 6 up:standby 2024-03-19T01:06:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:50 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:06:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:50 smithi163 ceph-mon[67428]: Deploying daemon mds.c on smithi163 2024-03-19T01:06:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:50 smithi163 ceph-mon[67428]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:50 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.163:6834/2175201817,v1:172.21.15.163:6835/2175201817] up:boot 2024-03-19T01:06:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:50 smithi163 ceph-mon[67428]: fsmap 6 up:standby 2024-03-19T01:06:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:50 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:06:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.739+0000 7f3e07e00640 1 -- 172.21.15.69:0/1262747951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e08102850 msgr2=0x7f3e08102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.739+0000 7f3e07e00640 1 --2- 172.21.15.69:0/1262747951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e08102850 0x7f3e08102cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009900 tx=0x7f3df002bc30 comp rx=0 tx=0).stop 2024-03-19T01:06:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.740+0000 7f3e07e00640 1 -- 172.21.15.69:0/1262747951 shutdown_connections 2024-03-19T01:06:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.740+0000 7f3e07e00640 1 --2- 172.21.15.69:0/1262747951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e08103210 0x7f3e08109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.740+0000 7f3e07e00640 1 --2- 172.21.15.69:0/1262747951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e08102850 0x7f3e08102cd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.740+0000 7f3e07e00640 1 --2- 172.21.15.69:0/1262747951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e08101660 0x7f3e08101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.740+0000 7f3e07e00640 1 -- 172.21.15.69:0/1262747951 >> 172.21.15.69:0/1262747951 conn(0x7f3e080fcbb0 msgr2=0x7f3e080ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.740+0000 7f3e07e00640 1 -- 172.21.15.69:0/1262747951 shutdown_connections 2024-03-19T01:06:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.740+0000 7f3e07e00640 1 -- 172.21.15.69:0/1262747951 wait complete. 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3e07e00640 1 Processor -- start 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3e07e00640 1 -- start start 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3e07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e08101660 0x7f3e08199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3e07e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e08102850 0x7f3e0819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3e07e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e08103210 0x7f3e081a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3e07e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0810d2f0 con 0x7f3e08102850 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3e07e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0810d170 con 0x7f3e08101660 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3e07e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0810d470 con 0x7f3e08103210 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3dffe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e08102850 0x7f3e0819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3dffe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e08102850 0x7f3e0819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55576/0 (socket says 172.21.15.69:55576) 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3dffe00640 1 -- 172.21.15.69:0/2025948753 learned_addr learned my addr 172.21.15.69:0/2025948753 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.741+0000 7f3e07400640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e08103210 0x7f3e081a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.742+0000 7f3e06a00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e08101660 0x7f3e08199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.742+0000 7f3dffe00640 1 -- 172.21.15.69:0/2025948753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e08103210 msgr2=0x7f3e081a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.742+0000 7f3dffe00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e08103210 0x7f3e081a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.742+0000 7f3dffe00640 1 -- 172.21.15.69:0/2025948753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e08101660 msgr2=0x7f3e08199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.742+0000 7f3dffe00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e08101660 0x7f3e08199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.742+0000 7f3dffe00640 1 -- 172.21.15.69:0/2025948753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e081a1830 con 0x7f3e08102850 2024-03-19T01:06:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.742+0000 7f3dffe00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e08102850 0x7f3e0819a1b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009230 tx=0x7f3df002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.742+0000 7f3dff400640 1 -- 172.21.15.69:0/2025948753 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0041070 con 0x7f3e08102850 2024-03-19T01:06:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.742+0000 7f3e07e00640 1 -- 172.21.15.69:0/2025948753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e081a1ac0 con 0x7f3e08102850 2024-03-19T01:06:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.742+0000 7f3dff400640 1 -- 172.21.15.69:0/2025948753 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df0004450 con 0x7f3e08102850 2024-03-19T01:06:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.742+0000 7f3dff400640 1 -- 172.21.15.69:0/2025948753 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0003e20 con 0x7f3e08102850 2024-03-19T01:06:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.743+0000 7f3e07e00640 1 -- 172.21.15.69:0/2025948753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e081a1fb0 con 0x7f3e08102850 2024-03-19T01:06:50.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.744+0000 7f3e07e00640 1 -- 172.21.15.69:0/2025948753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e081a21e0 con 0x7f3e08102850 2024-03-19T01:06:50.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.746+0000 7f3dff400640 1 -- 172.21.15.69:0/2025948753 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f3df0044070 con 0x7f3e08102850 2024-03-19T01:06:50.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.747+0000 7f3dff400640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3dec0a2800 0x7f3dec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:50.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.747+0000 7f3dff400640 1 -- 172.21.15.69:0/2025948753 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3df00f0620 con 0x7f3e08102850 2024-03-19T01:06:50.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.747+0000 7f3e06a00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3dec0a2800 0x7f3dec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:50.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.748+0000 7f3e06a00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3dec0a2800 0x7f3dec0a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3e0819b0f0 tx=0x7f3df8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:50.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.750+0000 7f3dff400640 1 -- 172.21.15.69:0/2025948753 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3df00bd470 con 0x7f3e08102850 2024-03-19T01:06:50.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.991+0000 7f3e07e00640 1 -- 172.21.15.69:0/2025948753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e081a23b0 con 0x7f3e08102850 2024-03-19T01:06:50.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.992+0000 7f3dff400640 1 -- 172.21.15.69:0/2025948753 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7f3df00c1510 con 0x7f3e08102850 2024-03-19T01:06:50.993 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:50.994 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-19T01:06:50.994 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 7 2024-03-19T01:06:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 -- 172.21.15.69:0/2025948753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3dec0a2800 msgr2=0x7f3dec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3dec0a2800 0x7f3dec0a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3e0819b0f0 tx=0x7f3df8005f90 comp rx=0 tx=0).stop 2024-03-19T01:06:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 -- 172.21.15.69:0/2025948753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e08102850 msgr2=0x7f3e0819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e08102850 0x7f3e0819a1b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009230 tx=0x7f3df002c650 comp rx=0 tx=0).stop 2024-03-19T01:06:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 -- 172.21.15.69:0/2025948753 shutdown_connections 2024-03-19T01:06:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3dec0a2800 0x7f3dec0a4cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:50.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e08103210 0x7f3e081a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:50.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e08102850 0x7f3e0819a1b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:50.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 --2- 172.21.15.69:0/2025948753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e08101660 0x7f3e08199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:50.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 -- 172.21.15.69:0/2025948753 >> 172.21.15.69:0/2025948753 conn(0x7f3e080fcbb0 msgr2=0x7f3e08073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:50.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 -- 172.21.15.69:0/2025948753 shutdown_connections 2024-03-19T01:06:50.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:50.995+0000 7f3e07e00640 1 -- 172.21.15.69:0/2025948753 wait complete. 2024-03-19T01:06:51.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:51 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2025948753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:51.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:51 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2025948753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:51 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2025948753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:51.910 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:06:51 smithi163 systemd[1]: Starting Ceph mds.c for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:06:51.969 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:06:51 smithi163 podman[98566]: 2024-03-19 01:06:51.540861252 +0000 UTC m=+0.017574399 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:06:51.969 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:06:51 smithi163 podman[98566]: 2024-03-19 01:06:51.708401912 +0000 UTC m=+0.185115049 container create 0c95bfbc9d13fc50e62edb6fab9b49409482dd8b09855b18617cd298e1a5f213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:06:51.970 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:06:51 smithi163 podman[98566]: 2024-03-19 01:06:51.969645851 +0000 UTC m=+0.446359007 container init 0c95bfbc9d13fc50e62edb6fab9b49409482dd8b09855b18617cd298e1a5f213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-03-19T01:06:52.223 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:06:51 smithi163 podman[98566]: 2024-03-19 01:06:51.972612338 +0000 UTC m=+0.449325492 container start 0c95bfbc9d13fc50e62edb6fab9b49409482dd8b09855b18617cd298e1a5f213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-19T01:06:52.223 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:06:51 smithi163 ceph-mds[98585]: -- 172.21.15.163:0/2617599641 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560cf3c15a40 con 0x560cf3be8800 2024-03-19T01:06:52.223 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:06:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c[98581]: starting mds.c at 2024-03-19T01:06:52.223 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:06:52 smithi163 bash[98566]: 0c95bfbc9d13fc50e62edb6fab9b49409482dd8b09855b18617cd298e1a5f213 2024-03-19T01:06:52.224 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:06:52 smithi163 systemd[1]: Started Ceph mds.c for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:06:52.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:52 smithi086 ceph-mon[66829]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:52 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:52 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:52 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:52 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:52 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:52 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:52 smithi069 ceph-mon[31727]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:52 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:52 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:52 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:52 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:52 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:52 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:52 smithi163 ceph-mon[67428]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:52 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:52 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:52 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:52 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:52 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:52 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:53.016 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:53.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.234+0000 7f2467200640 1 -- 172.21.15.69:0/3729727297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2468104a20 msgr2=0x7f2468106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:53.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.234+0000 7f2467200640 1 --2- 172.21.15.69:0/3729727297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2468104a20 0x7f2468106e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2450009ca0 tx=0x7f245002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:06:53.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.234+0000 7f2467200640 1 -- 172.21.15.69:0/3729727297 shutdown_connections 2024-03-19T01:06:53.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.235+0000 7f2467200640 1 --2- 172.21.15.69:0/3729727297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2468107350 0x7f24681097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:53.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.235+0000 7f2467200640 1 --2- 172.21.15.69:0/3729727297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2468104a20 0x7f2468106e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:53.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.235+0000 7f2467200640 1 --2- 172.21.15.69:0/3729727297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24681020f0 0x7f24681044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:53.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.235+0000 7f2467200640 1 -- 172.21.15.69:0/3729727297 >> 172.21.15.69:0/3729727297 conn(0x7f24680fbb00 msgr2=0x7f24680fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:53.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.235+0000 7f2467200640 1 -- 172.21.15.69:0/3729727297 shutdown_connections 2024-03-19T01:06:53.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.235+0000 7f2467200640 1 -- 172.21.15.69:0/3729727297 wait complete. 2024-03-19T01:06:53.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2467200640 1 Processor -- start 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2467200640 1 -- start start 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2467200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24681020f0 0x7f2468199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2467200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2468104a20 0x7f246819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2467200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2468107350 0x7f24681a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2467200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f246810d2f0 con 0x7f24681020f0 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2467200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f246810d170 con 0x7f2468104a20 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2467200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f246810d470 con 0x7f2468107350 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2465e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24681020f0 0x7f2468199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2465e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24681020f0 0x7f2468199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55594/0 (socket says 172.21.15.69:55594) 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2465e00640 1 -- 172.21.15.69:0/4137930628 learned_addr learned my addr 172.21.15.69:0/4137930628 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2465400640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2468104a20 0x7f246819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.236+0000 7f2466800640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2468107350 0x7f24681a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f2465e00640 1 -- 172.21.15.69:0/4137930628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2468107350 msgr2=0x7f24681a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f2465e00640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2468107350 0x7f24681a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f2465e00640 1 -- 172.21.15.69:0/4137930628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2468104a20 msgr2=0x7f246819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f2465e00640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2468104a20 0x7f246819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f2465e00640 1 -- 172.21.15.69:0/4137930628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24681a17e0 con 0x7f24681020f0 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f2465400640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2468104a20 0x7f246819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f2465e00640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24681020f0 0x7f2468199bc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f246000e9f0 tx=0x7f246000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f245ea00640 1 -- 172.21.15.69:0/4137930628 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246000cd80 con 0x7f24681020f0 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f245ea00640 1 -- 172.21.15.69:0/4137930628 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2460005500 con 0x7f24681020f0 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f2467200640 1 -- 172.21.15.69:0/4137930628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24681a1ad0 con 0x7f24681020f0 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f245ea00640 1 -- 172.21.15.69:0/4137930628 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460010690 con 0x7f24681020f0 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.237+0000 7f2467200640 1 -- 172.21.15.69:0/4137930628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24681a1fa0 con 0x7f24681020f0 2024-03-19T01:06:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.239+0000 7f2467200640 1 -- 172.21.15.69:0/4137930628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2468108ea0 con 0x7f24681020f0 2024-03-19T01:06:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.240+0000 7f245ea00640 1 -- 172.21.15.69:0/4137930628 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f24600046c0 con 0x7f24681020f0 2024-03-19T01:06:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.242+0000 7f245ea00640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f244c0a2800 0x7f244c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.242+0000 7f245ea00640 1 -- 172.21.15.69:0/4137930628 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f24600ceb00 con 0x7f24681020f0 2024-03-19T01:06:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.242+0000 7f2465400640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f244c0a2800 0x7f244c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.243+0000 7f2465400640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f244c0a2800 0x7f244c0a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f245002c700 tx=0x7f2450032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.246+0000 7f245ea00640 1 -- 172.21.15.69:0/4137930628 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f246009bfa0 con 0x7f24681020f0 2024-03-19T01:06:53.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.484+0000 7f2467200640 1 -- 172.21.15.69:0/4137930628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2468062420 con 0x7f24681020f0 2024-03-19T01:06:53.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.485+0000 7f245ea00640 1 -- 172.21.15.69:0/4137930628 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f246009b8a0 con 0x7f24681020f0 2024-03-19T01:06:53.486 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:53.486 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-19T01:06:53.486 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 8 2024-03-19T01:06:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.487+0000 7f2467200640 1 -- 172.21.15.69:0/4137930628 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f244c0a2800 msgr2=0x7f244c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.487+0000 7f2467200640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f244c0a2800 0x7f244c0a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f245002c700 tx=0x7f2450032590 comp rx=0 tx=0).stop 2024-03-19T01:06:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.487+0000 7f2467200640 1 -- 172.21.15.69:0/4137930628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24681020f0 msgr2=0x7f2468199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.487+0000 7f2467200640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24681020f0 0x7f2468199bc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f246000e9f0 tx=0x7f246000edf0 comp rx=0 tx=0).stop 2024-03-19T01:06:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.487+0000 7f2467200640 1 -- 172.21.15.69:0/4137930628 shutdown_connections 2024-03-19T01:06:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.487+0000 7f2467200640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f244c0a2800 0x7f244c0a4cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.487+0000 7f2467200640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2468107350 0x7f24681a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.487+0000 7f2467200640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2468104a20 0x7f246819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.487+0000 7f2467200640 1 --2- 172.21.15.69:0/4137930628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24681020f0 0x7f2468199bc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.487+0000 7f2467200640 1 -- 172.21.15.69:0/4137930628 >> 172.21.15.69:0/4137930628 conn(0x7f24680fbb00 msgr2=0x7f2468101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:53.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.488+0000 7f2467200640 1 -- 172.21.15.69:0/4137930628 shutdown_connections 2024-03-19T01:06:53.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:53.488+0000 7f2467200640 1 -- 172.21.15.69:0/4137930628 wait complete. 2024-03-19T01:06:53.504 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:53 smithi069 ceph-mon[31727]: Deploying daemon mds.h on smithi086 2024-03-19T01:06:53.504 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:53 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.163:6836/1677890136,v1:172.21.15.163:6837/1677890136] up:boot 2024-03-19T01:06:53.504 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:53 smithi069 ceph-mon[31727]: fsmap 7 up:standby 2024-03-19T01:06:53.504 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:53 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:06:53.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:53 smithi086 ceph-mon[66829]: Deploying daemon mds.h on smithi086 2024-03-19T01:06:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:53 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.163:6836/1677890136,v1:172.21.15.163:6837/1677890136] up:boot 2024-03-19T01:06:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:53 smithi086 ceph-mon[66829]: fsmap 7 up:standby 2024-03-19T01:06:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:53 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:06:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:53 smithi163 ceph-mon[67428]: Deploying daemon mds.h on smithi086 2024-03-19T01:06:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:53 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.163:6836/1677890136,v1:172.21.15.163:6837/1677890136] up:boot 2024-03-19T01:06:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:53 smithi163 ceph-mon[67428]: fsmap 7 up:standby 2024-03-19T01:06:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:53 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:06:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:54 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/4137930628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:54 smithi086 ceph-mon[66829]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:54.506 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:06:54 smithi086 systemd[1]: Starting Ceph mds.h for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:06:54.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:54 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/4137930628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:54.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:54 smithi069 ceph-mon[31727]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:54.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:54 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/4137930628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:54.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:54 smithi163 ceph-mon[67428]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:54.879 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:06:54 smithi086 podman[97303]: 2024-03-19 01:06:54.48067636 +0000 UTC m=+0.017926187 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:06:54.879 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:06:54 smithi086 podman[97303]: 2024-03-19 01:06:54.60206278 +0000 UTC m=+0.139312598 container create ed84330be6840d4dd6c6076573ead809970d59520c5c3ca32dfd1027d87e41c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-03-19T01:06:55.256 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:06:54 smithi086 podman[97303]: 2024-03-19 01:06:54.879689008 +0000 UTC m=+0.416938845 container init ed84330be6840d4dd6c6076573ead809970d59520c5c3ca32dfd1027d87e41c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:06:55.256 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:06:54 smithi086 podman[97303]: 2024-03-19 01:06:54.882300039 +0000 UTC m=+0.419549867 container start ed84330be6840d4dd6c6076573ead809970d59520c5c3ca32dfd1027d87e41c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-19T01:06:55.256 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:06:54 smithi086 ceph-mds[97322]: -- 172.21.15.86:0/2672117930 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5d0c15a40 con 0x55c5d0be8c00 2024-03-19T01:06:55.256 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:06:54 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h[97318]: starting mds.h at 2024-03-19T01:06:55.256 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:06:54 smithi086 bash[97303]: ed84330be6840d4dd6c6076573ead809970d59520c5c3ca32dfd1027d87e41c1 2024-03-19T01:06:55.256 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:06:54 smithi086 systemd[1]: Started Ceph mds.h for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:06:55.508 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:55.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.729+0000 7f6e45200640 1 -- 172.21.15.69:0/3859811442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e40103a60 msgr2=0x7f6e40103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:55.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.729+0000 7f6e45200640 1 --2- 172.21.15.69:0/3859811442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e40103a60 0x7f6e40103e80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009900 tx=0x7f6e3002bc30 comp rx=0 tx=0).stop 2024-03-19T01:06:55.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.729+0000 7f6e45200640 1 -- 172.21.15.69:0/3859811442 shutdown_connections 2024-03-19T01:06:55.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.729+0000 7f6e45200640 1 --2- 172.21.15.69:0/3859811442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e40105610 0x7f6e4010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:55.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.729+0000 7f6e45200640 1 --2- 172.21.15.69:0/3859811442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e40104c50 0x7f6e401050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:55.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.729+0000 7f6e45200640 1 --2- 172.21.15.69:0/3859811442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e40103a60 0x7f6e40103e80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:55.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.729+0000 7f6e45200640 1 -- 172.21.15.69:0/3859811442 >> 172.21.15.69:0/3859811442 conn(0x7f6e400ff010 msgr2=0x7f6e40101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:55.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.729+0000 7f6e45200640 1 -- 172.21.15.69:0/3859811442 shutdown_connections 2024-03-19T01:06:55.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.729+0000 7f6e45200640 1 -- 172.21.15.69:0/3859811442 wait complete. 2024-03-19T01:06:55.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.730+0000 7f6e45200640 1 Processor -- start 2024-03-19T01:06:55.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.730+0000 7f6e45200640 1 -- start start 2024-03-19T01:06:55.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e45200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e40103a60 0x7f6e40199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:55.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e45200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e40104c50 0x7f6e4019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e45200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e40105610 0x7f6e401a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e45200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4010f700 con 0x7f6e40103a60 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e45200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4010f580 con 0x7f6e40105610 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e45200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4010f880 con 0x7f6e40104c50 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e3d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e40104c50 0x7f6e4019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e3d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e40104c50 0x7f6e4019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59274/0 (socket says 172.21.15.69:59274) 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e3d400640 1 -- 172.21.15.69:0/2576214226 learned_addr learned my addr 172.21.15.69:0/2576214226 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e3e800640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e40105610 0x7f6e401a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e3de00640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e40103a60 0x7f6e40199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e3d400640 1 -- 172.21.15.69:0/2576214226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e40105610 msgr2=0x7f6e401a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e3d400640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e40105610 0x7f6e401a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e3d400640 1 -- 172.21.15.69:0/2576214226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e40103a60 msgr2=0x7f6e40199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:55.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e3d400640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e40103a60 0x7f6e40199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:55.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.731+0000 7f6e3d400640 1 -- 172.21.15.69:0/2576214226 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e401a1960 con 0x7f6e40104c50 2024-03-19T01:06:55.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.732+0000 7f6e3d400640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e40104c50 0x7f6e4019a280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00e9f0 tx=0x7f6e2c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:55.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.732+0000 7f6e36a00640 1 -- 172.21.15.69:0/2576214226 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c00cd80 con 0x7f6e40104c50 2024-03-19T01:06:55.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.732+0000 7f6e45200640 1 -- 172.21.15.69:0/2576214226 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e401a1c50 con 0x7f6e40104c50 2024-03-19T01:06:55.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.732+0000 7f6e36a00640 1 -- 172.21.15.69:0/2576214226 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2c010910 con 0x7f6e40104c50 2024-03-19T01:06:55.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.732+0000 7f6e36a00640 1 -- 172.21.15.69:0/2576214226 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0188f0 con 0x7f6e40104c50 2024-03-19T01:06:55.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.732+0000 7f6e45200640 1 -- 172.21.15.69:0/2576214226 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e401a21a0 con 0x7f6e40104c50 2024-03-19T01:06:55.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.733+0000 7f6e45200640 1 -- 172.21.15.69:0/2576214226 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e4010ad20 con 0x7f6e40104c50 2024-03-19T01:06:55.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.735+0000 7f6e36a00640 1 -- 172.21.15.69:0/2576214226 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f6e2c010ac0 con 0x7f6e40104c50 2024-03-19T01:06:55.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.737+0000 7f6e36a00640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e240a2800 0x7f6e240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:55.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.737+0000 7f6e36a00640 1 -- 172.21.15.69:0/2576214226 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6e2c0cea00 con 0x7f6e40104c50 2024-03-19T01:06:55.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.737+0000 7f6e3de00640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e240a2800 0x7f6e240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:55.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.738+0000 7f6e3de00640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e240a2800 0x7f6e240a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6e3002c170 tx=0x7f6e3002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:55.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.740+0000 7f6e36a00640 1 -- 172.21.15.69:0/2576214226 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6e2c09b850 con 0x7f6e40104c50 2024-03-19T01:06:55.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.989+0000 7f6e45200640 1 -- 172.21.15.69:0/2576214226 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e40062450 con 0x7f6e40104c50 2024-03-19T01:06:55.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.990+0000 7f6e36a00640 1 -- 172.21.15.69:0/2576214226 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f6e2c09f8f0 con 0x7f6e40104c50 2024-03-19T01:06:55.991 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:55.991 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-19T01:06:55.991 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 9 2024-03-19T01:06:55.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 -- 172.21.15.69:0/2576214226 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e240a2800 msgr2=0x7f6e240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:55.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e240a2800 0x7f6e240a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6e3002c170 tx=0x7f6e3002df90 comp rx=0 tx=0).stop 2024-03-19T01:06:55.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 -- 172.21.15.69:0/2576214226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e40104c50 msgr2=0x7f6e4019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:55.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e40104c50 0x7f6e4019a280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00e9f0 tx=0x7f6e2c00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:06:55.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 -- 172.21.15.69:0/2576214226 shutdown_connections 2024-03-19T01:06:55.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e240a2800 0x7f6e240a4cc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:55.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e40105610 0x7f6e401a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:55.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e40104c50 0x7f6e4019a280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:55.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 --2- 172.21.15.69:0/2576214226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e40103a60 0x7f6e40199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:55.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 -- 172.21.15.69:0/2576214226 >> 172.21.15.69:0/2576214226 conn(0x7f6e400ff010 msgr2=0x7f6e40107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:55.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 -- 172.21.15.69:0/2576214226 shutdown_connections 2024-03-19T01:06:55.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:55.993+0000 7f6e45200640 1 -- 172.21.15.69:0/2576214226 wait complete. 2024-03-19T01:06:56.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:56.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:56.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:56.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:56.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:56.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:56.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: Deploying daemon mds.i on smithi163 2024-03-19T01:06:56.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.86:6836/1996224781,v1:172.21.15.86:6837/1996224781] up:boot 2024-03-19T01:06:56.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: fsmap 8 up:standby 2024-03-19T01:06:56.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:06:56.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:56.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:56 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2576214226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: Deploying daemon mds.i on smithi163 2024-03-19T01:06:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.86:6836/1996224781,v1:172.21.15.86:6837/1996224781] up:boot 2024-03-19T01:06:56.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: fsmap 8 up:standby 2024-03-19T01:06:56.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:06:56.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:56.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:56 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2576214226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: Deploying daemon mds.i on smithi163 2024-03-19T01:06:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.86:6836/1996224781,v1:172.21.15.86:6837/1996224781] up:boot 2024-03-19T01:06:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: fsmap 8 up:standby 2024-03-19T01:06:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:06:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:56 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2576214226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:57.555 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:06:57 smithi163 systemd[1]: Starting Ceph mds.i for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:06:57.812 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:06:57 smithi163 podman[98869]: 2024-03-19 01:06:57.457545705 +0000 UTC m=+0.017739239 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:06:57.812 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:06:57 smithi163 podman[98869]: 2024-03-19 01:06:57.589682237 +0000 UTC m=+0.149875777 container create f72b47305772aaa4edb9aeda4a72a921b8f336ba138aa79a028b12e02e12a6eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-03-19T01:06:58.015 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:06:58.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.227+0000 7f9058800640 1 -- 172.21.15.69:0/3500423802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050105610 msgr2=0x7f905010bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:58.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.227+0000 7f9058800640 1 --2- 172.21.15.69:0/3500423802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050105610 0x7f905010bea0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f904c009ca0 tx=0x7f904c02c0f0 comp rx=0 tx=0).stop 2024-03-19T01:06:58.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.227+0000 7f9058800640 1 -- 172.21.15.69:0/3500423802 shutdown_connections 2024-03-19T01:06:58.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.227+0000 7f9058800640 1 --2- 172.21.15.69:0/3500423802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050105610 0x7f905010bea0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:58.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.227+0000 7f9058800640 1 --2- 172.21.15.69:0/3500423802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9050104c50 0x7f90501050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:58.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.227+0000 7f9058800640 1 --2- 172.21.15.69:0/3500423802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9050103a60 0x7f9050103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:58.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.227+0000 7f9058800640 1 -- 172.21.15.69:0/3500423802 >> 172.21.15.69:0/3500423802 conn(0x7f90500fefd0 msgr2=0x7f9050101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:58.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.227+0000 7f9058800640 1 -- 172.21.15.69:0/3500423802 shutdown_connections 2024-03-19T01:06:58.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.227+0000 7f9058800640 1 -- 172.21.15.69:0/3500423802 wait complete. 2024-03-19T01:06:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.228+0000 7f9058800640 1 Processor -- start 2024-03-19T01:06:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.228+0000 7f9058800640 1 -- start start 2024-03-19T01:06:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9058800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9050103a60 0x7f9050199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9058800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9050104c50 0x7f905019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9058800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050105610 0x7f90501a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9058800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f905010f6e0 con 0x7f9050105610 2024-03-19T01:06:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9058800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f905010f560 con 0x7f9050104c50 2024-03-19T01:06:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9058800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f905010f860 con 0x7f9050103a60 2024-03-19T01:06:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9056800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050105610 0x7f90501a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9056800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050105610 0x7f90501a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48640/0 (socket says 172.21.15.69:48640) 2024-03-19T01:06:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9056800640 1 -- 172.21.15.69:0/420351168 learned_addr learned my addr 172.21.15.69:0/420351168 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:06:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9055e00640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9050103a60 0x7f9050199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9055400640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9050104c50 0x7f905019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9056800640 1 -- 172.21.15.69:0/420351168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9050103a60 msgr2=0x7f9050199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9056800640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9050103a60 0x7f9050199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9056800640 1 -- 172.21.15.69:0/420351168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9050104c50 msgr2=0x7f905019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9056800640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9050104c50 0x7f905019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.229+0000 7f9056800640 1 -- 172.21.15.69:0/420351168 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90501a1950 con 0x7f9050105610 2024-03-19T01:06:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.230+0000 7f9055e00640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9050103a60 0x7f9050199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:06:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.230+0000 7f9056800640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050105610 0x7f90501a12b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f904c037040 tx=0x7f904c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:58.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.230+0000 7f904aa00640 1 -- 172.21.15.69:0/420351168 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c041020 con 0x7f9050105610 2024-03-19T01:06:58.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.230+0000 7f904aa00640 1 -- 172.21.15.69:0/420351168 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f904c034cd0 con 0x7f9050105610 2024-03-19T01:06:58.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.230+0000 7f9058800640 1 -- 172.21.15.69:0/420351168 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90501a1be0 con 0x7f9050105610 2024-03-19T01:06:58.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.230+0000 7f904aa00640 1 -- 172.21.15.69:0/420351168 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c039040 con 0x7f9050105610 2024-03-19T01:06:58.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.230+0000 7f9058800640 1 -- 172.21.15.69:0/420351168 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90501a20d0 con 0x7f9050105610 2024-03-19T01:06:58.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.231+0000 7f9058800640 1 -- 172.21.15.69:0/420351168 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905010ad80 con 0x7f9050105610 2024-03-19T01:06:58.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.233+0000 7f904aa00640 1 -- 172.21.15.69:0/420351168 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f904c044020 con 0x7f9050105610 2024-03-19T01:06:58.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.235+0000 7f904aa00640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90380a2800 0x7f90380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:06:58.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.235+0000 7f904aa00640 1 -- 172.21.15.69:0/420351168 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f904c0f0950 con 0x7f9050105610 2024-03-19T01:06:58.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.235+0000 7f9055e00640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90380a2800 0x7f90380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:06:58.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.236+0000 7f9055e00640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90380a2800 0x7f90380a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f905019b210 tx=0x7f9044005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:06:58.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.238+0000 7f904aa00640 1 -- 172.21.15.69:0/420351168 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f904c04b020 con 0x7f9050105610 2024-03-19T01:06:58.312 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:06:57 smithi163 podman[98869]: 2024-03-19 01:06:57.853900443 +0000 UTC m=+0.414093967 container init f72b47305772aaa4edb9aeda4a72a921b8f336ba138aa79a028b12e02e12a6eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-19T01:06:58.312 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:06:57 smithi163 podman[98869]: 2024-03-19 01:06:57.856604722 +0000 UTC m=+0.416798248 container start f72b47305772aaa4edb9aeda4a72a921b8f336ba138aa79a028b12e02e12a6eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-19T01:06:58.312 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:06:57 smithi163 ceph-mds[98888]: -- 172.21.15.163:0/240247474 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55562c5cba40 con 0x55562c59ec00 2024-03-19T01:06:58.312 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:06:57 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i[98884]: starting mds.i at 2024-03-19T01:06:58.312 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:06:57 smithi163 bash[98869]: f72b47305772aaa4edb9aeda4a72a921b8f336ba138aa79a028b12e02e12a6eb 2024-03-19T01:06:58.312 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:06:57 smithi163 systemd[1]: Started Ceph mds.i for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:06:58.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.475+0000 7f9058800640 1 -- 172.21.15.69:0/420351168 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f905002d1f0 con 0x7f9050105610 2024-03-19T01:06:58.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.477+0000 7f904aa00640 1 -- 172.21.15.69:0/420351168 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f904c0bddf0 con 0x7f9050105610 2024-03-19T01:06:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:06:58.478 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-19T01:06:58.478 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 9 2024-03-19T01:06:58.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.479+0000 7f9058800640 1 -- 172.21.15.69:0/420351168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90380a2800 msgr2=0x7f90380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:58.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.479+0000 7f9058800640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90380a2800 0x7f90380a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f905019b210 tx=0x7f9044005f90 comp rx=0 tx=0).stop 2024-03-19T01:06:58.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.479+0000 7f9058800640 1 -- 172.21.15.69:0/420351168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050105610 msgr2=0x7f90501a12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:06:58.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.479+0000 7f9058800640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050105610 0x7f90501a12b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f904c037040 tx=0x7f904c032f40 comp rx=0 tx=0).stop 2024-03-19T01:06:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.480+0000 7f9058800640 1 -- 172.21.15.69:0/420351168 shutdown_connections 2024-03-19T01:06:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.480+0000 7f9058800640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90380a2800 0x7f90380a4cc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.480+0000 7f9058800640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050105610 0x7f90501a12b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.480+0000 7f9058800640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9050104c50 0x7f905019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.480+0000 7f9058800640 1 --2- 172.21.15.69:0/420351168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9050103a60 0x7f9050199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:06:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.480+0000 7f9058800640 1 -- 172.21.15.69:0/420351168 >> 172.21.15.69:0/420351168 conn(0x7f90500fefd0 msgr2=0x7f9050107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:06:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.480+0000 7f9058800640 1 -- 172.21.15.69:0/420351168 shutdown_connections 2024-03-19T01:06:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:06:58.480+0000 7f9058800640 1 -- 172.21.15.69:0/420351168 wait complete. 2024-03-19T01:06:58.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:58 smithi069 ceph-mon[31727]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:58.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:58.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:58.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:58.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:58.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:58.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:58 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:58.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:58 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/420351168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:58 smithi086 ceph-mon[66829]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:58 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:59.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:58 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/420351168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:06:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:58 smithi163 ceph-mon[67428]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:06:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:59.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:06:59.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:06:59.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:06:59.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:58 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:06:59.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:58 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/420351168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:00.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:59 smithi086 ceph-mon[66829]: Deploying daemon mds.g on smithi069 2024-03-19T01:07:00.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:59 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] up:boot 2024-03-19T01:07:00.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:59 smithi086 ceph-mon[66829]: fsmap 9 up:standby 2024-03-19T01:07:00.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:06:59 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:07:00.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:59 smithi069 ceph-mon[31727]: Deploying daemon mds.g on smithi069 2024-03-19T01:07:00.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:59 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] up:boot 2024-03-19T01:07:00.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:59 smithi069 ceph-mon[31727]: fsmap 9 up:standby 2024-03-19T01:07:00.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:06:59 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:07:00.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:59 smithi163 ceph-mon[67428]: Deploying daemon mds.g on smithi069 2024-03-19T01:07:00.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:59 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] up:boot 2024-03-19T01:07:00.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:59 smithi163 ceph-mon[67428]: fsmap 9 up:standby 2024-03-19T01:07:00.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:06:59 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:07:00.500 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:07:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.789+0000 7fa1b3a00640 1 -- 172.21.15.69:0/3741843526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ac100a30 msgr2=0x7fa1ac100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.789+0000 7fa1b3a00640 1 --2- 172.21.15.69:0/3741843526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ac100a30 0x7fa1ac100e50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa198009900 tx=0x7fa19802bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.790+0000 7fa1b3a00640 1 -- 172.21.15.69:0/3741843526 shutdown_connections 2024-03-19T01:07:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.790+0000 7fa1b3a00640 1 --2- 172.21.15.69:0/3741843526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0ffec0 0x7fa1ac101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.790+0000 7fa1b3a00640 1 --2- 172.21.15.69:0/3741843526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1ac0ff500 0x7fa1ac0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.790+0000 7fa1b3a00640 1 --2- 172.21.15.69:0/3741843526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ac100a30 0x7fa1ac100e50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.790+0000 7fa1b3a00640 1 -- 172.21.15.69:0/3741843526 >> 172.21.15.69:0/3741843526 conn(0x7fa1ac0fb020 msgr2=0x7fa1ac0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.790+0000 7fa1b3a00640 1 -- 172.21.15.69:0/3741843526 shutdown_connections 2024-03-19T01:07:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.790+0000 7fa1b3a00640 1 -- 172.21.15.69:0/3741843526 wait complete. 2024-03-19T01:07:00.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.791+0000 7fa1b3a00640 1 Processor -- start 2024-03-19T01:07:00.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.792+0000 7fa1b3a00640 1 -- start start 2024-03-19T01:07:00.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.792+0000 7fa1b3a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ac0ff500 0x7fa1ac070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:00.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.792+0000 7fa1b3a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0ffec0 0x7fa1ac070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:00.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.792+0000 7fa1b3a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1ac100a30 0x7fa1ac198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.792+0000 7fa1b3a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac10f6f0 con 0x7fa1ac100a30 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.792+0000 7fa1b3a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac10f570 con 0x7fa1ac0ff500 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.792+0000 7fa1b3a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac10f870 con 0x7fa1ac0ffec0 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.793+0000 7fa1b1a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1ac100a30 0x7fa1ac198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.793+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0ffec0 0x7fa1ac070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.793+0000 7fa1b1a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1ac100a30 0x7fa1ac198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48656/0 (socket says 172.21.15.69:48656) 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.793+0000 7fa1b1a00640 1 -- 172.21.15.69:0/46786408 learned_addr learned my addr 172.21.15.69:0/46786408 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.792+0000 7fa1b1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ac0ff500 0x7fa1ac070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.793+0000 7fa1abe00640 1 -- 172.21.15.69:0/46786408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ac0ff500 msgr2=0x7fa1ac070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.793+0000 7fa1abe00640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ac0ff500 0x7fa1ac070140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.793+0000 7fa1abe00640 1 -- 172.21.15.69:0/46786408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1ac100a30 msgr2=0x7fa1ac198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.793+0000 7fa1abe00640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1ac100a30 0x7fa1ac198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.793+0000 7fa1abe00640 1 -- 172.21.15.69:0/46786408 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac199330 con 0x7fa1ac0ffec0 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.794+0000 7fa1abe00640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0ffec0 0x7fa1ac070680 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00c350 tx=0x7fa19c00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:00.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.794+0000 7fa1a9600640 1 -- 172.21.15.69:0/46786408 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c004020 con 0x7fa1ac0ffec0 2024-03-19T01:07:00.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.794+0000 7fa1a9600640 1 -- 172.21.15.69:0/46786408 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19c0041c0 con 0x7fa1ac0ffec0 2024-03-19T01:07:00.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.794+0000 7fa1b3a00640 1 -- 172.21.15.69:0/46786408 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ac102fd0 con 0x7fa1ac0ffec0 2024-03-19T01:07:00.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.794+0000 7fa1a9600640 1 -- 172.21.15.69:0/46786408 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00b9b0 con 0x7fa1ac0ffec0 2024-03-19T01:07:00.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.794+0000 7fa1b3a00640 1 -- 172.21.15.69:0/46786408 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ac103540 con 0x7fa1ac0ffec0 2024-03-19T01:07:00.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.796+0000 7fa1b3a00640 1 -- 172.21.15.69:0/46786408 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18c0052c0 con 0x7fa1ac0ffec0 2024-03-19T01:07:00.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.799+0000 7fa1a9600640 1 -- 172.21.15.69:0/46786408 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa19c016020 con 0x7fa1ac0ffec0 2024-03-19T01:07:00.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.800+0000 7fa1a9600640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa1900a2880 0x7fa1900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:00.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.800+0000 7fa1a9600640 1 -- 172.21.15.69:0/46786408 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa19c0decc0 con 0x7fa1ac0ffec0 2024-03-19T01:07:00.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.801+0000 7fa1b1000640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa1900a2880 0x7fa1900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:00.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.802+0000 7fa1b1000640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa1900a2880 0x7fa1900a4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa19802c240 tx=0x7fa19802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:00.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:00.805+0000 7fa1a9600640 1 -- 172.21.15.69:0/46786408 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa19c0e5080 con 0x7fa1ac0ffec0 2024-03-19T01:07:01.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:00 smithi086 ceph-mon[66829]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:01.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:00 smithi069 ceph-mon[31727]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:01.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.049+0000 7fa1b3a00640 1 -- 172.21.15.69:0/46786408 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa18c0055b0 con 0x7fa1ac0ffec0 2024-03-19T01:07:01.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.050+0000 7fa1a9600640 1 -- 172.21.15.69:0/46786408 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7fa19c0abb10 con 0x7fa1ac0ffec0 2024-03-19T01:07:01.051 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:01.051 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-19T01:07:01.052 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 10 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.052+0000 7fa1b3a00640 1 -- 172.21.15.69:0/46786408 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa1900a2880 msgr2=0x7fa1900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.052+0000 7fa1b3a00640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa1900a2880 0x7fa1900a4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa19802c240 tx=0x7fa19802df90 comp rx=0 tx=0).stop 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.052+0000 7fa1b3a00640 1 -- 172.21.15.69:0/46786408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0ffec0 msgr2=0x7fa1ac070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.052+0000 7fa1b3a00640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0ffec0 0x7fa1ac070680 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00c350 tx=0x7fa19c00c810 comp rx=0 tx=0).stop 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.053+0000 7fa1b3a00640 1 -- 172.21.15.69:0/46786408 shutdown_connections 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.053+0000 7fa1b3a00640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa1900a2880 0x7fa1900a4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.053+0000 7fa1b3a00640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1ac100a30 0x7fa1ac198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.053+0000 7fa1b3a00640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0ffec0 0x7fa1ac070680 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.053+0000 7fa1b3a00640 1 --2- 172.21.15.69:0/46786408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ac0ff500 0x7fa1ac070140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.053+0000 7fa1b3a00640 1 -- 172.21.15.69:0/46786408 >> 172.21.15.69:0/46786408 conn(0x7fa1ac0fb020 msgr2=0x7fa1ac0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.053+0000 7fa1b3a00640 1 -- 172.21.15.69:0/46786408 shutdown_connections 2024-03-19T01:07:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:01.053+0000 7fa1b3a00640 1 -- 172.21.15.69:0/46786408 wait complete. 2024-03-19T01:07:01.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:00 smithi163 ceph-mon[67428]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:01.709 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:07:01 smithi069 podman[70449]: 2024-03-19 01:07:01.435516234 +0000 UTC m=+0.203975036 container create 7adea5bd5206be3bc2b21c80569326c52472ae36b9561609b34aee3fb36066da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-g, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:07:01.709 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:07:01 smithi069 podman[70449]: 2024-03-19 01:07:01.70600913 +0000 UTC m=+0.474467969 container init 7adea5bd5206be3bc2b21c80569326c52472ae36b9561609b34aee3fb36066da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-19T01:07:02.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:01 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/46786408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:02.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:01 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/46786408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:02.032 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:07:01 smithi069 podman[70449]: 2024-03-19 01:07:01.719086004 +0000 UTC m=+0.487544823 container start 7adea5bd5206be3bc2b21c80569326c52472ae36b9561609b34aee3fb36066da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux ) 2024-03-19T01:07:02.032 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:07:01 smithi069 ceph-mds[70468]: -- 172.21.15.69:0/2929363459 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff3bd17a40 con 0x55ff3bcea800 2024-03-19T01:07:02.032 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:07:01 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-g[70464]: starting mds.g at 2024-03-19T01:07:02.032 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:07:01 smithi069 bash[70449]: 7adea5bd5206be3bc2b21c80569326c52472ae36b9561609b34aee3fb36066da 2024-03-19T01:07:02.032 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:07:01 smithi069 systemd[1]: Started Ceph mds.g for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:07:02.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:01 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/46786408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:02.983 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:02 smithi069 ceph-mon[31727]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:02.983 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:02.983 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:02.983 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:02.983 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:07:02.983 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:07:02.983 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:02 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:03.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:02 smithi086 ceph-mon[66829]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:03.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:03.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:03.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:03.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:07:03.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:07:03.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:02 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:03.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:02 smithi163 ceph-mon[67428]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:03.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:03.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:03.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:03.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:07:03.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:07:03.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:02 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:03.073 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:07:03.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.285+0000 7f8d3c600640 1 -- 172.21.15.69:0/3233484511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d34101930 msgr2=0x7f8d34101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:03.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.285+0000 7f8d3c600640 1 --2- 172.21.15.69:0/3233484511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d34101930 0x7f8d34101db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009900 tx=0x7f8d2002bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:03.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.285+0000 7f8d3c600640 1 -- 172.21.15.69:0/3233484511 shutdown_connections 2024-03-19T01:07:03.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.285+0000 7f8d3c600640 1 --2- 172.21.15.69:0/3233484511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d341022f0 0x7f8d34108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:03.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.285+0000 7f8d3c600640 1 --2- 172.21.15.69:0/3233484511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d34101930 0x7f8d34101db0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:03.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.285+0000 7f8d3c600640 1 --2- 172.21.15.69:0/3233484511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34100740 0x7f8d34100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:03.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.285+0000 7f8d3c600640 1 -- 172.21.15.69:0/3233484511 >> 172.21.15.69:0/3233484511 conn(0x7f8d340fbcb0 msgr2=0x7f8d340fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:03.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.285+0000 7f8d3c600640 1 -- 172.21.15.69:0/3233484511 shutdown_connections 2024-03-19T01:07:03.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.285+0000 7f8d3c600640 1 -- 172.21.15.69:0/3233484511 wait complete. 2024-03-19T01:07:03.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.286+0000 7f8d3c600640 1 Processor -- start 2024-03-19T01:07:03.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.286+0000 7f8d3c600640 1 -- start start 2024-03-19T01:07:03.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.286+0000 7f8d3c600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34100740 0x7f8d34199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:03.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.286+0000 7f8d3c600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d34101930 0x7f8d3419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:03.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.286+0000 7f8d3c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d341022f0 0x7f8d341a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:03.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.286+0000 7f8d3c600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3410c3c0 con 0x7f8d34100740 2024-03-19T01:07:03.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d3c600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3410c240 con 0x7f8d341022f0 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d3c600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3410c540 con 0x7f8d34101930 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d39c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34100740 0x7f8d34199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d39c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34100740 0x7f8d34199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48704/0 (socket says 172.21.15.69:48704) 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d39c00640 1 -- 172.21.15.69:0/1030767449 learned_addr learned my addr 172.21.15.69:0/1030767449 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d3a600640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d341022f0 0x7f8d341a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d39200640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d34101930 0x7f8d3419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d39c00640 1 -- 172.21.15.69:0/1030767449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d34101930 msgr2=0x7f8d3419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d39c00640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d34101930 0x7f8d3419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d39c00640 1 -- 172.21.15.69:0/1030767449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d341022f0 msgr2=0x7f8d341a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d39c00640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d341022f0 0x7f8d341a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.287+0000 7f8d39c00640 1 -- 172.21.15.69:0/1030767449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d341a1960 con 0x7f8d34100740 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.288+0000 7f8d39c00640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34100740 0x7f8d34199d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800e9c0 tx=0x7f8d2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.288+0000 7f8d32000640 1 -- 172.21.15.69:0/1030767449 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2800cd80 con 0x7f8d34100740 2024-03-19T01:07:03.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.288+0000 7f8d3c600640 1 -- 172.21.15.69:0/1030767449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d341a1c50 con 0x7f8d34100740 2024-03-19T01:07:03.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.288+0000 7f8d32000640 1 -- 172.21.15.69:0/1030767449 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d280035f0 con 0x7f8d34100740 2024-03-19T01:07:03.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.288+0000 7f8d32000640 1 -- 172.21.15.69:0/1030767449 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28010690 con 0x7f8d34100740 2024-03-19T01:07:03.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.288+0000 7f8d3c600640 1 -- 172.21.15.69:0/1030767449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d341a21a0 con 0x7f8d34100740 2024-03-19T01:07:03.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.290+0000 7f8d3c600640 1 -- 172.21.15.69:0/1030767449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d341a23d0 con 0x7f8d34100740 2024-03-19T01:07:03.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.291+0000 7f8d32000640 1 -- 172.21.15.69:0/1030767449 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8d280037a0 con 0x7f8d34100740 2024-03-19T01:07:03.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.292+0000 7f8d32000640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8d1c0a2800 0x7f8d1c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:03.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.293+0000 7f8d32000640 1 -- 172.21.15.69:0/1030767449 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8d280cead0 con 0x7f8d34100740 2024-03-19T01:07:03.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.293+0000 7f8d39200640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8d1c0a2800 0x7f8d1c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:03.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.293+0000 7f8d39200640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8d1c0a2800 0x7f8d1c0a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8d2002c7b0 tx=0x7f8d20005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:03.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.301+0000 7f8d32000640 1 -- 172.21.15.69:0/1030767449 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8d2809bf70 con 0x7f8d34100740 2024-03-19T01:07:03.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.546+0000 7f8d3c600640 1 -- 172.21.15.69:0/1030767449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d341a25a0 con 0x7f8d34100740 2024-03-19T01:07:03.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.547+0000 7f8d32000640 1 -- 172.21.15.69:0/1030767449 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f8d2809b870 con 0x7f8d34100740 2024-03-19T01:07:03.548 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:03.548 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-19T01:07:03.548 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 11 2024-03-19T01:07:03.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.549+0000 7f8d3c600640 1 -- 172.21.15.69:0/1030767449 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8d1c0a2800 msgr2=0x7f8d1c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:03.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.549+0000 7f8d3c600640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8d1c0a2800 0x7f8d1c0a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8d2002c7b0 tx=0x7f8d20005fd0 comp rx=0 tx=0).stop 2024-03-19T01:07:03.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.549+0000 7f8d3c600640 1 -- 172.21.15.69:0/1030767449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34100740 msgr2=0x7f8d34199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:03.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.549+0000 7f8d3c600640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34100740 0x7f8d34199d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800e9c0 tx=0x7f8d2800ee80 comp rx=0 tx=0).stop 2024-03-19T01:07:03.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.550+0000 7f8d3c600640 1 -- 172.21.15.69:0/1030767449 shutdown_connections 2024-03-19T01:07:03.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.550+0000 7f8d3c600640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f8d1c0a2800 0x7f8d1c0a4cc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:03.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.550+0000 7f8d3c600640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d341022f0 0x7f8d341a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:03.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.550+0000 7f8d3c600640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d34101930 0x7f8d3419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:03.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.550+0000 7f8d3c600640 1 --2- 172.21.15.69:0/1030767449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d34100740 0x7f8d34199d50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:03.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.550+0000 7f8d3c600640 1 -- 172.21.15.69:0/1030767449 >> 172.21.15.69:0/1030767449 conn(0x7f8d340fbcb0 msgr2=0x7f8d340fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:03.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.550+0000 7f8d3c600640 1 -- 172.21.15.69:0/1030767449 shutdown_connections 2024-03-19T01:07:03.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:03.550+0000 7f8d3c600640 1 -- 172.21.15.69:0/1030767449 wait complete. 2024-03-19T01:07:03.995 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:03 smithi069 ceph-mon[31727]: Deploying daemon mds.a on smithi069 2024-03-19T01:07:03.995 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:03 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.69:6838/1997100038,v1:172.21.15.69:6839/1997100038] up:boot 2024-03-19T01:07:03.995 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:03 smithi069 ceph-mon[31727]: fsmap 10 up:standby 2024-03-19T01:07:03.995 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:03 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T01:07:03.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:03 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1030767449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:04.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:03 smithi086 ceph-mon[66829]: Deploying daemon mds.a on smithi069 2024-03-19T01:07:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:03 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.69:6838/1997100038,v1:172.21.15.69:6839/1997100038] up:boot 2024-03-19T01:07:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:03 smithi086 ceph-mon[66829]: fsmap 10 up:standby 2024-03-19T01:07:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:03 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T01:07:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:03 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1030767449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:04.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:03 smithi163 ceph-mon[67428]: Deploying daemon mds.a on smithi069 2024-03-19T01:07:04.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:03 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.69:6838/1997100038,v1:172.21.15.69:6839/1997100038] up:boot 2024-03-19T01:07:04.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:03 smithi163 ceph-mon[67428]: fsmap 10 up:standby 2024-03-19T01:07:04.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:03 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T01:07:04.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:03 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1030767449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:04.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:04 smithi069 ceph-mon[31727]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:05.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:04 smithi086 ceph-mon[66829]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:05.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:04 smithi163 ceph-mon[67428]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:05.283 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:07:04 smithi069 systemd[1]: Starting Ceph mds.a for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:07:05.571 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:07:05.615 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:07:05 smithi069 podman[70794]: 2024-03-19 01:07:05.205066585 +0000 UTC m=+0.035277067 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:07:05.615 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:07:05 smithi069 podman[70794]: 2024-03-19 01:07:05.352943472 +0000 UTC m=+0.183153987 container create e20f4f9f69ca57d47970edaf30bcec3b65cb2254f4eeb4532f7b33839d296052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-19T01:07:05.615 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:07:05 smithi069 podman[70794]: 2024-03-19 01:07:05.614561292 +0000 UTC m=+0.444771777 container init e20f4f9f69ca57d47970edaf30bcec3b65cb2254f4eeb4532f7b33839d296052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-19T01:07:05.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.787+0000 7febcd800640 1 -- 172.21.15.69:0/1689235319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8105610 msgr2=0x7febc810bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:05.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.787+0000 7febcd800640 1 --2- 172.21.15.69:0/1689235319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8105610 0x7febc810bea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7febc0009be0 tx=0x7febc002bef0 comp rx=0 tx=0).stop 2024-03-19T01:07:05.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.787+0000 7febcd800640 1 -- 172.21.15.69:0/1689235319 shutdown_connections 2024-03-19T01:07:05.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.787+0000 7febcd800640 1 --2- 172.21.15.69:0/1689235319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8105610 0x7febc810bea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:05.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.787+0000 7febcd800640 1 --2- 172.21.15.69:0/1689235319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febc8104c50 0x7febc81050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:05.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.787+0000 7febcd800640 1 --2- 172.21.15.69:0/1689235319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febc8103a60 0x7febc8103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:05.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.787+0000 7febcd800640 1 -- 172.21.15.69:0/1689235319 >> 172.21.15.69:0/1689235319 conn(0x7febc80ff010 msgr2=0x7febc8101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:05.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.787+0000 7febcd800640 1 -- 172.21.15.69:0/1689235319 shutdown_connections 2024-03-19T01:07:05.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.787+0000 7febcd800640 1 -- 172.21.15.69:0/1689235319 wait complete. 2024-03-19T01:07:05.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.789+0000 7febcd800640 1 Processor -- start 2024-03-19T01:07:05.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.789+0000 7febcd800640 1 -- start start 2024-03-19T01:07:05.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.789+0000 7febcd800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febc8103a60 0x7febc8199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:05.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.789+0000 7febcd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8104c50 0x7febc819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:05.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.789+0000 7febcd800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febc8105610 0x7febc81a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:05.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.789+0000 7febcd800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7febc810f710 con 0x7febc8103a60 2024-03-19T01:07:05.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.789+0000 7febcd800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febc810f590 con 0x7febc8104c50 2024-03-19T01:07:05.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.789+0000 7febcd800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7febc810f890 con 0x7febc8105610 2024-03-19T01:07:05.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.790+0000 7febc6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febc8103a60 0x7febc8199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:05.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.790+0000 7febc6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8104c50 0x7febc819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:05.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.790+0000 7febc6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febc8103a60 0x7febc8199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48750/0 (socket says 172.21.15.69:48750) 2024-03-19T01:07:05.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.790+0000 7febc6a00640 1 -- 172.21.15.69:0/2597185557 learned_addr learned my addr 172.21.15.69:0/2597185557 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:05.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.790+0000 7febc7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febc8105610 0x7febc81a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:05.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.790+0000 7febc6000640 1 -- 172.21.15.69:0/2597185557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febc8105610 msgr2=0x7febc81a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:05.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.790+0000 7febc6000640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febc8105610 0x7febc81a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.790+0000 7febc6000640 1 -- 172.21.15.69:0/2597185557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febc8103a60 msgr2=0x7febc8199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.790+0000 7febc6000640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febc8103a60 0x7febc8199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.790+0000 7febc6000640 1 -- 172.21.15.69:0/2597185557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc81a1900 con 0x7febc8104c50 2024-03-19T01:07:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.790+0000 7febc6000640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8104c50 0x7febc819a200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7febb4005fd0 tx=0x7febb400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.791+0000 7febbf400640 1 -- 172.21.15.69:0/2597185557 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb400cd00 con 0x7febc8104c50 2024-03-19T01:07:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.791+0000 7febbf400640 1 -- 172.21.15.69:0/2597185557 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb40035f0 con 0x7febc8104c50 2024-03-19T01:07:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.791+0000 7febbf400640 1 -- 172.21.15.69:0/2597185557 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4010690 con 0x7febc8104c50 2024-03-19T01:07:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.791+0000 7febcd800640 1 -- 172.21.15.69:0/2597185557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc81a1bf0 con 0x7febc8104c50 2024-03-19T01:07:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.791+0000 7febcd800640 1 -- 172.21.15.69:0/2597185557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc81a2190 con 0x7febc8104c50 2024-03-19T01:07:05.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.792+0000 7febcd800640 1 -- 172.21.15.69:0/2597185557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc810ad20 con 0x7febc8104c50 2024-03-19T01:07:05.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.794+0000 7febbf400640 1 -- 172.21.15.69:0/2597185557 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7febb40037a0 con 0x7febc8104c50 2024-03-19T01:07:05.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.796+0000 7febbf400640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7febac0a2880 0x7febac0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:05.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.796+0000 7febbf400640 1 -- 172.21.15.69:0/2597185557 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7febb40ce830 con 0x7febc8104c50 2024-03-19T01:07:05.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.796+0000 7febc6a00640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7febac0a2880 0x7febac0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:05.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.797+0000 7febc6a00640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7febac0a2880 0x7febac0a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7febc819ab90 tx=0x7febb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:05.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:05.800+0000 7febbf400640 1 -- 172.21.15.69:0/2597185557 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7febb4009510 con 0x7febc8104c50 2024-03-19T01:07:06.032 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:07:05 smithi069 podman[70794]: 2024-03-19 01:07:05.623348252 +0000 UTC m=+0.453558730 container start e20f4f9f69ca57d47970edaf30bcec3b65cb2254f4eeb4532f7b33839d296052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-19T01:07:06.032 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:07:05 smithi069 ceph-mds[70829]: -- 172.21.15.69:0/2141249389 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c2a61ba40 con 0x564c2a5ee800 2024-03-19T01:07:06.032 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:07:05 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a[70809]: starting mds.a at 2024-03-19T01:07:06.032 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:07:05 smithi069 bash[70794]: e20f4f9f69ca57d47970edaf30bcec3b65cb2254f4eeb4532f7b33839d296052 2024-03-19T01:07:06.033 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:07:05 smithi069 systemd[1]: Started Ceph mds.a for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:07:06.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.046+0000 7febcd800640 1 -- 172.21.15.69:0/2597185557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febc802d5f0 con 0x7febc8104c50 2024-03-19T01:07:06.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.048+0000 7febbf400640 1 -- 172.21.15.69:0/2597185557 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7febb409bcd0 con 0x7febc8104c50 2024-03-19T01:07:06.049 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:06.049 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24580,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-03-19T01:07:06.049 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 12 2024-03-19T01:07:06.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.050+0000 7febcd800640 1 -- 172.21.15.69:0/2597185557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7febac0a2880 msgr2=0x7febac0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:06.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.051+0000 7febcd800640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7febac0a2880 0x7febac0a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7febc819ab90 tx=0x7febb8005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:06.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.051+0000 7febcd800640 1 -- 172.21.15.69:0/2597185557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8104c50 msgr2=0x7febc819a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:06.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.051+0000 7febcd800640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8104c50 0x7febc819a200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7febb4005fd0 tx=0x7febb400ee50 comp rx=0 tx=0).stop 2024-03-19T01:07:06.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.051+0000 7febcd800640 1 -- 172.21.15.69:0/2597185557 shutdown_connections 2024-03-19T01:07:06.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.051+0000 7febcd800640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7febac0a2880 0x7febac0a4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:06.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.051+0000 7febcd800640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febc8105610 0x7febc81a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:06.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.051+0000 7febcd800640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8104c50 0x7febc819a200 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:06.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.051+0000 7febcd800640 1 --2- 172.21.15.69:0/2597185557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febc8103a60 0x7febc8199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:06.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.051+0000 7febcd800640 1 -- 172.21.15.69:0/2597185557 >> 172.21.15.69:0/2597185557 conn(0x7febc80ff010 msgr2=0x7febc8107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:06.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.051+0000 7febcd800640 1 -- 172.21.15.69:0/2597185557 shutdown_connections 2024-03-19T01:07:06.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:06.051+0000 7febcd800640 1 -- 172.21.15.69:0/2597185557 wait complete. 2024-03-19T01:07:06.783 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:06 smithi086 ceph-mon[66829]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:06.783 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:06 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] up:boot 2024-03-19T01:07:06.783 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:06 smithi086 ceph-mon[66829]: fsmap 11 up:standby 2024-03-19T01:07:06.783 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:07:06.783 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:06.783 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:06.784 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:06.784 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:07:06.784 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:07:06.784 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:06.784 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:06 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2597185557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:06 smithi069 ceph-mon[31727]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:06 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] up:boot 2024-03-19T01:07:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:06 smithi069 ceph-mon[31727]: fsmap 11 up:standby 2024-03-19T01:07:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:07:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:07.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:07:07.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:07:07.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:07.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:06 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2597185557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:07.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:06 smithi163 ceph-mon[67428]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:07.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:06 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] up:boot 2024-03-19T01:07:07.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:06 smithi163 ceph-mon[67428]: fsmap 11 up:standby 2024-03-19T01:07:07.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:07:07.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:07.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:07.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:07.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:07:07.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:07:07.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:07.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:06 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2597185557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:07.844 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:07 smithi086 ceph-mon[66829]: Deploying daemon mds.b on smithi086 2024-03-19T01:07:08.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:07 smithi069 ceph-mon[31727]: Deploying daemon mds.b on smithi086 2024-03-19T01:07:08.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:07 smithi163 ceph-mon[67428]: Deploying daemon mds.b on smithi086 2024-03-19T01:07:08.070 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:07:08.247 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:08 smithi086 systemd[1]: Starting Ceph mds.b for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:07:08.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.291+0000 7f2713200640 1 -- 172.21.15.69:0/2045271537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2714109830 msgr2=0x7f27141100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:08.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.291+0000 7f2713200640 1 --2- 172.21.15.69:0/2045271537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2714109830 0x7f27141100c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f270c009ca0 tx=0x7f270c02c100 comp rx=0 tx=0).stop 2024-03-19T01:07:08.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.291+0000 7f2713200640 1 -- 172.21.15.69:0/2045271537 shutdown_connections 2024-03-19T01:07:08.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.291+0000 7f2713200640 1 --2- 172.21.15.69:0/2045271537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2714109830 0x7f27141100c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:08.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.291+0000 7f2713200640 1 --2- 172.21.15.69:0/2045271537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2714108e70 0x7f27141092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:08.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.291+0000 7f2713200640 1 --2- 172.21.15.69:0/2045271537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2714107c80 0x7f27141080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:08.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.291+0000 7f2713200640 1 -- 172.21.15.69:0/2045271537 >> 172.21.15.69:0/2045271537 conn(0x7f2714076c30 msgr2=0x7f2714079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:08.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.291+0000 7f2713200640 1 -- 172.21.15.69:0/2045271537 shutdown_connections 2024-03-19T01:07:08.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.291+0000 7f2713200640 1 -- 172.21.15.69:0/2045271537 wait complete. 2024-03-19T01:07:08.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.292+0000 7f2713200640 1 Processor -- start 2024-03-19T01:07:08.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.292+0000 7f2713200640 1 -- start start 2024-03-19T01:07:08.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2713200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2714107c80 0x7f271419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:08.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2713200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2714108e70 0x7f271419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2713200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2714109830 0x7f27141a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2713200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2714113900 con 0x7f2714109830 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2713200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2714113780 con 0x7f2714108e70 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2713200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2714113a80 con 0x7f2714107c80 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f270be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2714108e70 0x7f271419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2712800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2714109830 0x7f27141a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f270be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2714108e70 0x7f271419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44510/0 (socket says 172.21.15.69:44510) 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2711e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2714107c80 0x7f271419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f270be00640 1 -- 172.21.15.69:0/290259487 learned_addr learned my addr 172.21.15.69:0/290259487 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2712800640 1 -- 172.21.15.69:0/290259487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2714107c80 msgr2=0x7f271419def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2712800640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2714107c80 0x7f271419def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2712800640 1 -- 172.21.15.69:0/290259487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2714108e70 msgr2=0x7f271419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:08.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2712800640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2714108e70 0x7f271419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:08.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f2712800640 1 -- 172.21.15.69:0/290259487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27141a5b10 con 0x7f2714109830 2024-03-19T01:07:08.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.293+0000 7f270be00640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2714108e70 0x7f271419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:08.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.294+0000 7f2712800640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2714109830 0x7f27141a54a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f270c02c640 tx=0x7f270c035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:08.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.294+0000 7f270aa00640 1 -- 172.21.15.69:0/290259487 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c039040 con 0x7f2714109830 2024-03-19T01:07:08.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.294+0000 7f270aa00640 1 -- 172.21.15.69:0/290259487 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f270c044070 con 0x7f2714109830 2024-03-19T01:07:08.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.294+0000 7f270aa00640 1 -- 172.21.15.69:0/290259487 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c0043e0 con 0x7f2714109830 2024-03-19T01:07:08.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.294+0000 7f2713200640 1 -- 172.21.15.69:0/290259487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27141a5da0 con 0x7f2714109830 2024-03-19T01:07:08.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.294+0000 7f2713200640 1 -- 172.21.15.69:0/290259487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27141a61b0 con 0x7f2714109830 2024-03-19T01:07:08.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.295+0000 7f2713200640 1 -- 172.21.15.69:0/290259487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271410ef40 con 0x7f2714109830 2024-03-19T01:07:08.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.297+0000 7f270aa00640 1 -- 172.21.15.69:0/290259487 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f270c004580 con 0x7f2714109830 2024-03-19T01:07:08.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.298+0000 7f270aa00640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f26f80a2880 0x7f26f80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:08.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.299+0000 7f270aa00640 1 -- 172.21.15.69:0/290259487 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f270c0f0c40 con 0x7f2714109830 2024-03-19T01:07:08.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.299+0000 7f2711e00640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f26f80a2880 0x7f26f80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:08.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.299+0000 7f2711e00640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f26f80a2880 0x7f26f80a4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2714108cd0 tx=0x7f2704005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:08.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.303+0000 7f270aa00640 1 -- 172.21.15.69:0/290259487 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f270c0be0e0 con 0x7f2714109830 2024-03-19T01:07:08.506 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:08 smithi086 podman[97612]: 2024-03-19 01:07:08.149282942 +0000 UTC m=+0.018193450 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:07:08.506 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:08 smithi086 podman[97612]: 2024-03-19 01:07:08.272312365 +0000 UTC m=+0.141222871 container create f8a2b62ff9d5110359c45a6d4780a417ebf192dd64616544baa01c1a23e5468e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-19T01:07:08.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.542+0000 7f2713200640 1 -- 172.21.15.69:0/290259487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f271402d1f0 con 0x7f2714109830 2024-03-19T01:07:08.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.543+0000 7f270aa00640 1 -- 172.21.15.69:0/290259487 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f270c0bd9e0 con 0x7f2714109830 2024-03-19T01:07:08.545 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:08.545 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24580,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-03-19T01:07:08.545 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 12 2024-03-19T01:07:08.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 -- 172.21.15.69:0/290259487 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f26f80a2880 msgr2=0x7f26f80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:08.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f26f80a2880 0x7f26f80a4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2714108cd0 tx=0x7f2704005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:08.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 -- 172.21.15.69:0/290259487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2714109830 msgr2=0x7f27141a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:08.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2714109830 0x7f27141a54a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f270c02c640 tx=0x7f270c035d90 comp rx=0 tx=0).stop 2024-03-19T01:07:08.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 -- 172.21.15.69:0/290259487 shutdown_connections 2024-03-19T01:07:08.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f26f80a2880 0x7f26f80a4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:08.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2714109830 0x7f27141a54a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:08.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2714108e70 0x7f271419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:08.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 --2- 172.21.15.69:0/290259487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2714107c80 0x7f271419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:08.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 -- 172.21.15.69:0/290259487 >> 172.21.15.69:0/290259487 conn(0x7f2714076c30 msgr2=0x7f271410c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:08.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 -- 172.21.15.69:0/290259487 shutdown_connections 2024-03-19T01:07:08.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:08.546+0000 7f2713200640 1 -- 172.21.15.69:0/290259487 wait complete. 2024-03-19T01:07:08.771 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:08 smithi086 podman[97612]: 2024-03-19 01:07:08.507951246 +0000 UTC m=+0.376861755 container init f8a2b62ff9d5110359c45a6d4780a417ebf192dd64616544baa01c1a23e5468e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-19T01:07:08.771 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:08 smithi086 podman[97612]: 2024-03-19 01:07:08.510746929 +0000 UTC m=+0.379657437 container start f8a2b62ff9d5110359c45a6d4780a417ebf192dd64616544baa01c1a23e5468e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-19T01:07:08.771 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:08 smithi086 ceph-mds[97631]: -- 172.21.15.86:0/288540474 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5621fb8cda40 con 0x5621fb8a1000 2024-03-19T01:07:08.771 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:08 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b[97627]: starting mds.b at 2024-03-19T01:07:08.771 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:08 smithi086 bash[97612]: f8a2b62ff9d5110359c45a6d4780a417ebf192dd64616544baa01c1a23e5468e 2024-03-19T01:07:08.771 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:08 smithi086 systemd[1]: Started Ceph mds.b for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:07:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:08 smithi069 ceph-mon[31727]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:08 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/290259487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:09.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:08 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:08 smithi163 ceph-mon[67428]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:08 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/290259487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:08 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:08 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:09.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:08 smithi086 ceph-mon[66829]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:09.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:08 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/290259487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:09.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:08 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:09.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:08 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:10.027 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:09 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] up:boot 2024-03-19T01:07:10.027 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:09 smithi163 ceph-mon[67428]: fsmap 12 up:standby 2024-03-19T01:07:10.027 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:07:10.027 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:10.027 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:10.027 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:10.027 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:07:10.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:09 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] up:boot 2024-03-19T01:07:10.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:09 smithi069 ceph-mon[31727]: fsmap 12 up:standby 2024-03-19T01:07:10.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:07:10.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:10.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:10.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:10.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:07:10.043 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:09 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] up:boot 2024-03-19T01:07:10.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:09 smithi086 ceph-mon[66829]: fsmap 12 up:standby 2024-03-19T01:07:10.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:07:10.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:10.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:10.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:10.044 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:07:10.567 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:07:10.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.790+0000 7f4a18a00640 1 -- 172.21.15.69:0/2062162210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a14108fb0 msgr2=0x7f4a14109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:10.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.790+0000 7f4a18a00640 1 --2- 172.21.15.69:0/2062162210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a14108fb0 0x7f4a14109430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009900 tx=0x7f4a0002bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:10.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.790+0000 7f4a18a00640 1 -- 172.21.15.69:0/2062162210 shutdown_connections 2024-03-19T01:07:10.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.790+0000 7f4a18a00640 1 --2- 172.21.15.69:0/2062162210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a14109970 0x7f4a14110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:10.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.790+0000 7f4a18a00640 1 --2- 172.21.15.69:0/2062162210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a14108fb0 0x7f4a14109430 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:10.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.790+0000 7f4a18a00640 1 --2- 172.21.15.69:0/2062162210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a14107e60 0x7f4a14108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:10.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.790+0000 7f4a18a00640 1 -- 172.21.15.69:0/2062162210 >> 172.21.15.69:0/2062162210 conn(0x7f4a14076e30 msgr2=0x7f4a14079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:10.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.790+0000 7f4a18a00640 1 -- 172.21.15.69:0/2062162210 shutdown_connections 2024-03-19T01:07:10.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.790+0000 7f4a18a00640 1 -- 172.21.15.69:0/2062162210 wait complete. 2024-03-19T01:07:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.791+0000 7f4a18a00640 1 Processor -- start 2024-03-19T01:07:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.791+0000 7f4a18a00640 1 -- start start 2024-03-19T01:07:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.791+0000 7f4a18a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a14107e60 0x7f4a1419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.791+0000 7f4a18a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a14108fb0 0x7f4a1419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.791+0000 7f4a18a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a14109970 0x7f4a141a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.791+0000 7f4a18a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14113b50 con 0x7f4a14107e60 2024-03-19T01:07:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.791+0000 7f4a18a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141139d0 con 0x7f4a14109970 2024-03-19T01:07:10.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.791+0000 7f4a18a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14113cd0 con 0x7f4a14108fb0 2024-03-19T01:07:10.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.791+0000 7f4a11e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a14107e60 0x7f4a1419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:10.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a12800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a14109970 0x7f4a141a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:10.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a12800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a14109970 0x7f4a141a5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44520/0 (socket says 172.21.15.69:44520) 2024-03-19T01:07:10.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a12800640 1 -- 172.21.15.69:0/1228016714 learned_addr learned my addr 172.21.15.69:0/1228016714 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:10.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a11e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a14107e60 0x7f4a1419e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60074/0 (socket says 172.21.15.69:60074) 2024-03-19T01:07:10.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a11400640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a14108fb0 0x7f4a1419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:10.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a12800640 1 -- 172.21.15.69:0/1228016714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a14108fb0 msgr2=0x7f4a1419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a12800640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a14108fb0 0x7f4a1419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a12800640 1 -- 172.21.15.69:0/1228016714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a14107e60 msgr2=0x7f4a1419e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a12800640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a14107e60 0x7f4a1419e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a12800640 1 -- 172.21.15.69:0/1228016714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a141a5d80 con 0x7f4a14109970 2024-03-19T01:07:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a12800640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a14109970 0x7f4a141a5710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00efd0 tx=0x7f4a0c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a0aa00640 1 -- 172.21.15.69:0/1228016714 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c003b70 con 0x7f4a14109970 2024-03-19T01:07:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a18a00640 1 -- 172.21.15.69:0/1228016714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a141a6070 con 0x7f4a14109970 2024-03-19T01:07:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.792+0000 7f4a0aa00640 1 -- 172.21.15.69:0/1228016714 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0c003d10 con 0x7f4a14109970 2024-03-19T01:07:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.793+0000 7f4a0aa00640 1 -- 172.21.15.69:0/1228016714 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c0109f0 con 0x7f4a14109970 2024-03-19T01:07:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.793+0000 7f4a18a00640 1 -- 172.21.15.69:0/1228016714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a141a6540 con 0x7f4a14109970 2024-03-19T01:07:10.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.794+0000 7f4a18a00640 1 -- 172.21.15.69:0/1228016714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1410f170 con 0x7f4a14109970 2024-03-19T01:07:10.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.796+0000 7f4a0aa00640 1 -- 172.21.15.69:0/1228016714 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f4a0c004020 con 0x7f4a14109970 2024-03-19T01:07:10.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.798+0000 7f4a0aa00640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f49f80a2800 0x7f49f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:10.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.798+0000 7f4a0aa00640 1 -- 172.21.15.69:0/1228016714 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4a0c0cead0 con 0x7f4a14109970 2024-03-19T01:07:10.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.798+0000 7f4a11e00640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f49f80a2800 0x7f49f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:10.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.799+0000 7f4a11e00640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f49f80a2800 0x7f49f80a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009bd0 tx=0x7f4a04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:10.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:10.801+0000 7f4a0aa00640 1 -- 172.21.15.69:0/1228016714 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4a0c09bf70 con 0x7f4a14109970 2024-03-19T01:07:11.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:10 smithi069 ceph-mon[31727]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:11.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:10 smithi163 ceph-mon[67428]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:11.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.080+0000 7f4a18a00640 1 -- 172.21.15.69:0/1228016714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a1402d1f0 con 0x7f4a14109970 2024-03-19T01:07:11.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.081+0000 7f4a0aa00640 1 -- 172.21.15.69:0/1228016714 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7f4a0c09b870 con 0x7f4a14109970 2024-03-19T01:07:11.082 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:11.082 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24580,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-19T01:07:11.083 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 13 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.084+0000 7f4a18a00640 1 -- 172.21.15.69:0/1228016714 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f49f80a2800 msgr2=0x7f49f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.084+0000 7f4a18a00640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f49f80a2800 0x7f49f80a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009bd0 tx=0x7f4a04005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.084+0000 7f4a18a00640 1 -- 172.21.15.69:0/1228016714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a14109970 msgr2=0x7f4a141a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.084+0000 7f4a18a00640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a14109970 0x7f4a141a5710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00efd0 tx=0x7f4a0c00c6a0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.084+0000 7f4a18a00640 1 -- 172.21.15.69:0/1228016714 shutdown_connections 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.084+0000 7f4a18a00640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f49f80a2800 0x7f49f80a4cc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.084+0000 7f4a18a00640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a14109970 0x7f4a141a5710 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.084+0000 7f4a18a00640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a14108fb0 0x7f4a1419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.084+0000 7f4a18a00640 1 --2- 172.21.15.69:0/1228016714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a14107e60 0x7f4a1419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.084+0000 7f4a18a00640 1 -- 172.21.15.69:0/1228016714 >> 172.21.15.69:0/1228016714 conn(0x7f4a14076e30 msgr2=0x7f4a1410c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.085+0000 7f4a18a00640 1 -- 172.21.15.69:0/1228016714 shutdown_connections 2024-03-19T01:07:11.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.085+0000 7f4a18a00640 1 -- 172.21.15.69:0/1228016714 wait complete. 2024-03-19T01:07:11.104 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:07:11.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:10 smithi086 ceph-mon[66829]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:11.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.292+0000 7fb614400640 1 -- 172.21.15.69:0/2810144770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb60c104c50 msgr2=0x7fb60c107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.292+0000 7fb614400640 1 --2- 172.21.15.69:0/2810144770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb60c104c50 0x7fb60c107040 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009900 tx=0x7fb5f802bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.292+0000 7fb614400640 1 -- 172.21.15.69:0/2810144770 shutdown_connections 2024-03-19T01:07:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.292+0000 7fb614400640 1 --2- 172.21.15.69:0/2810144770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c107580 0x7fb60c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.292+0000 7fb614400640 1 --2- 172.21.15.69:0/2810144770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb60c104c50 0x7fb60c107040 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.292+0000 7fb614400640 1 --2- 172.21.15.69:0/2810144770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb60c102320 0x7fb60c104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.292+0000 7fb614400640 1 -- 172.21.15.69:0/2810144770 >> 172.21.15.69:0/2810144770 conn(0x7fb60c0fbcd0 msgr2=0x7fb60c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.292+0000 7fb614400640 1 -- 172.21.15.69:0/2810144770 shutdown_connections 2024-03-19T01:07:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.292+0000 7fb614400640 1 -- 172.21.15.69:0/2810144770 wait complete. 2024-03-19T01:07:11.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.294+0000 7fb614400640 1 Processor -- start 2024-03-19T01:07:11.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.294+0000 7fb614400640 1 -- start start 2024-03-19T01:07:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.294+0000 7fb614400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb60c102320 0x7fb60c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.294+0000 7fb614400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c104c50 0x7fb60c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb614400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb60c107580 0x7fb60c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb614400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c10d510 con 0x7fb60c102320 2024-03-19T01:07:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb614400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c10d390 con 0x7fb60c107580 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb614400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c10d690 con 0x7fb60c104c50 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb611000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c104c50 0x7fb60c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb611a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb60c102320 0x7fb60c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb611000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c104c50 0x7fb60c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:41924/0 (socket says 172.21.15.69:41924) 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb611000640 1 -- 172.21.15.69:0/3901775090 learned_addr learned my addr 172.21.15.69:0/3901775090 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb612400640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb60c107580 0x7fb60c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb611000640 1 -- 172.21.15.69:0/3901775090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb60c107580 msgr2=0x7fb60c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb611000640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb60c107580 0x7fb60c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb611000640 1 -- 172.21.15.69:0/3901775090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb60c102320 msgr2=0x7fb60c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb611000640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb60c102320 0x7fb60c197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb611000640 1 -- 172.21.15.69:0/3901775090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c19f7b0 con 0x7fb60c104c50 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.295+0000 7fb612400640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb60c107580 0x7fb60c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.296+0000 7fb611000640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c104c50 0x7fb60c198120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000c00 tx=0x7fb5f8005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:11.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.296+0000 7fb60a000640 1 -- 172.21.15.69:0/3901775090 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8043020 con 0x7fb60c104c50 2024-03-19T01:07:11.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.296+0000 7fb614400640 1 -- 172.21.15.69:0/3901775090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60c19fa40 con 0x7fb60c104c50 2024-03-19T01:07:11.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.296+0000 7fb60a000640 1 -- 172.21.15.69:0/3901775090 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f80054e0 con 0x7fb60c104c50 2024-03-19T01:07:11.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.296+0000 7fb60a000640 1 -- 172.21.15.69:0/3901775090 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8003e60 con 0x7fb60c104c50 2024-03-19T01:07:11.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.296+0000 7fb614400640 1 -- 172.21.15.69:0/3901775090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60c19ff90 con 0x7fb60c104c50 2024-03-19T01:07:11.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.297+0000 7fb614400640 1 -- 172.21.15.69:0/3901775090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb60c0fd7c0 con 0x7fb60c104c50 2024-03-19T01:07:11.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.299+0000 7fb60a000640 1 -- 172.21.15.69:0/3901775090 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb5f8046020 con 0x7fb60c104c50 2024-03-19T01:07:11.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.300+0000 7fb60a000640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5f40a2800 0x7fb5f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:11.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.300+0000 7fb60a000640 1 -- 172.21.15.69:0/3901775090 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb5f80f0770 con 0x7fb60c104c50 2024-03-19T01:07:11.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.301+0000 7fb611a00640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5f40a2800 0x7fb5f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:11.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.301+0000 7fb611a00640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5f40a2800 0x7fb5f40a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb600005d00 tx=0x7fb60000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:11.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.309+0000 7fb60a000640 1 -- 172.21.15.69:0/3901775090 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb5f804d020 con 0x7fb60c104c50 2024-03-19T01:07:11.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.515+0000 7fb614400640 1 -- 172.21.15.69:0/3901775090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb60c062420 con 0x7fb60c104c50 2024-03-19T01:07:11.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.517+0000 7fb60a000640 1 -- 172.21.15.69:0/3901775090 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18397 (secure 0 0 0) 0x7fb5f80bdc10 con 0x7fb60c104c50 2024-03-19T01:07:11.519 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:11.519 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":81,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:06:18.746880+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:07:11.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 -- 172.21.15.69:0/3901775090 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5f40a2800 msgr2=0x7fb5f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5f40a2800 0x7fb5f40a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb600005d00 tx=0x7fb60000b410 comp rx=0 tx=0).stop 2024-03-19T01:07:11.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 -- 172.21.15.69:0/3901775090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c104c50 msgr2=0x7fb60c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c104c50 0x7fb60c198120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000c00 tx=0x7fb5f8005dd0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 -- 172.21.15.69:0/3901775090 shutdown_connections 2024-03-19T01:07:11.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5f40a2800 0x7fb5f40a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb60c107580 0x7fb60c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c104c50 0x7fb60c198120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 --2- 172.21.15.69:0/3901775090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb60c102320 0x7fb60c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 -- 172.21.15.69:0/3901775090 >> 172.21.15.69:0/3901775090 conn(0x7fb60c0fbcd0 msgr2=0x7fb60c102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:11.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 -- 172.21.15.69:0/3901775090 shutdown_connections 2024-03-19T01:07:11.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.520+0000 7fb614400640 1 -- 172.21.15.69:0/3901775090 wait complete. 2024-03-19T01:07:11.540 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T01:07:11.540 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:07:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.737+0000 7f4c0b800640 1 -- 172.21.15.69:0/297439199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c04101830 msgr2=0x7f4c04101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.737+0000 7f4c0b800640 1 --2- 172.21.15.69:0/297439199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c04101830 0x7f4c04101c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009900 tx=0x7f4bf802bc40 comp rx=0 tx=0).stop 2024-03-19T01:07:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.737+0000 7f4c0b800640 1 -- 172.21.15.69:0/297439199 shutdown_connections 2024-03-19T01:07:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.737+0000 7f4c0b800640 1 --2- 172.21.15.69:0/297439199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c041033e0 0x7f4c04109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.737+0000 7f4c0b800640 1 --2- 172.21.15.69:0/297439199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c04102a20 0x7f4c04102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.737+0000 7f4c0b800640 1 --2- 172.21.15.69:0/297439199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c04101830 0x7f4c04101c50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.737+0000 7f4c0b800640 1 -- 172.21.15.69:0/297439199 >> 172.21.15.69:0/297439199 conn(0x7f4c040fcda0 msgr2=0x7f4c040ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.737+0000 7f4c0b800640 1 -- 172.21.15.69:0/297439199 shutdown_connections 2024-03-19T01:07:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.737+0000 7f4c0b800640 1 -- 172.21.15.69:0/297439199 wait complete. 2024-03-19T01:07:11.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.738+0000 7f4c0b800640 1 Processor -- start 2024-03-19T01:07:11.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c0b800640 1 -- start start 2024-03-19T01:07:11.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c0b800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c04101830 0x7f4c04199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:11.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c0b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c04102a20 0x7f4c0419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:11.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c0b800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c041033e0 0x7f4c041a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:11.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c0b800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0410d4d0 con 0x7f4c041033e0 2024-03-19T01:07:11.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c0b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0410d350 con 0x7f4c04102a20 2024-03-19T01:07:11.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c0b800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0410d650 con 0x7f4c04101830 2024-03-19T01:07:11.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c08e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c04101830 0x7f4c04199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:11.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c08e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c04101830 0x7f4c04199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:41940/0 (socket says 172.21.15.69:41940) 2024-03-19T01:07:11.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c08e00640 1 -- 172.21.15.69:0/462467262 learned_addr learned my addr 172.21.15.69:0/462467262 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:11.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c03e00640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c04102a20 0x7f4c0419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:11.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.739+0000 7f4c09800640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c041033e0 0x7f4c041a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:11.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.740+0000 7f4c08e00640 1 -- 172.21.15.69:0/462467262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c04102a20 msgr2=0x7f4c0419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.740+0000 7f4c08e00640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c04102a20 0x7f4c0419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.740+0000 7f4c08e00640 1 -- 172.21.15.69:0/462467262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c041033e0 msgr2=0x7f4c041a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.740+0000 7f4c08e00640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c041033e0 0x7f4c041a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.740+0000 7f4c08e00640 1 -- 172.21.15.69:0/462467262 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c041a19c0 con 0x7f4c04101830 2024-03-19T01:07:11.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.740+0000 7f4c09800640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c041033e0 0x7f4c041a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:11.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.740+0000 7f4c03e00640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c04102a20 0x7f4c0419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:11.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.740+0000 7f4c08e00640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c04101830 0x7f4c04199db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4bf802efd0 tx=0x7f4bf8005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:11.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.742+0000 7f4c01600640 1 -- 172.21.15.69:0/462467262 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8041070 con 0x7f4c04101830 2024-03-19T01:07:11.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.742+0000 7f4c01600640 1 -- 172.21.15.69:0/462467262 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bf80054e0 con 0x7f4c04101830 2024-03-19T01:07:11.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.742+0000 7f4c01600640 1 -- 172.21.15.69:0/462467262 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8034440 con 0x7f4c04101830 2024-03-19T01:07:11.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.742+0000 7f4c0b800640 1 -- 172.21.15.69:0/462467262 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c041a1c50 con 0x7f4c04101830 2024-03-19T01:07:11.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.742+0000 7f4c0b800640 1 -- 172.21.15.69:0/462467262 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c041a20c0 con 0x7f4c04101830 2024-03-19T01:07:11.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.744+0000 7f4c0b800640 1 -- 172.21.15.69:0/462467262 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c041a2290 con 0x7f4c04101830 2024-03-19T01:07:11.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.745+0000 7f4c01600640 1 -- 172.21.15.69:0/462467262 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f4bf8044070 con 0x7f4c04101830 2024-03-19T01:07:11.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.747+0000 7f4c01600640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4bec0a2880 0x7f4bec0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:11.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.747+0000 7f4c01600640 1 -- 172.21.15.69:0/462467262 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4bf80f02a0 con 0x7f4c04101830 2024-03-19T01:07:11.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.747+0000 7f4c03e00640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4bec0a2880 0x7f4bec0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:11.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.748+0000 7f4c03e00640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4bec0a2880 0x7f4bec0a4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0005d00 tx=0x7f4bf000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:11.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.755+0000 7f4c01600640 1 -- 172.21.15.69:0/462467262 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4bf80bd780 con 0x7f4c04101830 2024-03-19T01:07:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.956+0000 7f4c0b800640 1 -- 172.21.15.69:0/462467262 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c041a2460 con 0x7f4c04101830 2024-03-19T01:07:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.957+0000 7f4c01600640 1 -- 172.21.15.69:0/462467262 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f4bf80bd080 con 0x7f4c04101830 2024-03-19T01:07:11.958 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:07:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.959+0000 7f4c0b800640 1 -- 172.21.15.69:0/462467262 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4bec0a2880 msgr2=0x7f4bec0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.960+0000 7f4c0b800640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4bec0a2880 0x7f4bec0a4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0005d00 tx=0x7f4bf000b410 comp rx=0 tx=0).stop 2024-03-19T01:07:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.960+0000 7f4c0b800640 1 -- 172.21.15.69:0/462467262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c04101830 msgr2=0x7f4c04199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.960+0000 7f4c0b800640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c04101830 0x7f4c04199db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4bf802efd0 tx=0x7f4bf8005e40 comp rx=0 tx=0).stop 2024-03-19T01:07:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.960+0000 7f4c0b800640 1 -- 172.21.15.69:0/462467262 shutdown_connections 2024-03-19T01:07:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.960+0000 7f4c0b800640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4bec0a2880 0x7f4bec0a4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.960+0000 7f4c0b800640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c041033e0 0x7f4c041a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.960+0000 7f4c0b800640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c04102a20 0x7f4c0419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.960+0000 7f4c0b800640 1 --2- 172.21.15.69:0/462467262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c04101830 0x7f4c04199db0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.960+0000 7f4c0b800640 1 -- 172.21.15.69:0/462467262 >> 172.21.15.69:0/462467262 conn(0x7f4c040fcda0 msgr2=0x7f4c04073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.960+0000 7f4c0b800640 1 -- 172.21.15.69:0/462467262 shutdown_connections 2024-03-19T01:07:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:11.960+0000 7f4c0b800640 1 -- 172.21.15.69:0/462467262 wait complete. 2024-03-19T01:07:11.981 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-19T01:07:12.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.162+0000 7f80e5400640 1 -- 172.21.15.69:0/2259956006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e00761e0 msgr2=0x7f80e0076600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:12.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.162+0000 7f80e5400640 1 --2- 172.21.15.69:0/2259956006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e00761e0 0x7f80e0076600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009900 tx=0x7f80d002bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:12.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.162+0000 7f80e5400640 1 -- 172.21.15.69:0/2259956006 shutdown_connections 2024-03-19T01:07:12.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.162+0000 7f80e5400640 1 --2- 172.21.15.69:0/2259956006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e0077dc0 0x7f80e0078660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.162+0000 7f80e5400640 1 --2- 172.21.15.69:0/2259956006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e0077420 0x7f80e0077880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.162+0000 7f80e5400640 1 --2- 172.21.15.69:0/2259956006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e00761e0 0x7f80e0076600 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.162+0000 7f80e5400640 1 -- 172.21.15.69:0/2259956006 >> 172.21.15.69:0/2259956006 conn(0x7f80e00ff010 msgr2=0x7f80e0101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:12.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.162+0000 7f80e5400640 1 -- 172.21.15.69:0/2259956006 shutdown_connections 2024-03-19T01:07:12.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.162+0000 7f80e5400640 1 -- 172.21.15.69:0/2259956006 wait complete. 2024-03-19T01:07:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.163+0000 7f80e5400640 1 Processor -- start 2024-03-19T01:07:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.163+0000 7f80e5400640 1 -- start start 2024-03-19T01:07:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.163+0000 7f80e5400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e00761e0 0x7f80e019e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.163+0000 7f80e5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e0077420 0x7f80e019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.163+0000 7f80e5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e0077dc0 0x7f80e01a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.163+0000 7f80e5400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80e0113af0 con 0x7f80e00761e0 2024-03-19T01:07:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.163+0000 7f80e5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80e0113970 con 0x7f80e0077420 2024-03-19T01:07:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.163+0000 7f80e5400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f80e0113c70 con 0x7f80e0077dc0 2024-03-19T01:07:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.164+0000 7f80de800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e00761e0 0x7f80e019e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.164+0000 7f80de800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e00761e0 0x7f80e019e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60136/0 (socket says 172.21.15.69:60136) 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.164+0000 7f80de800640 1 -- 172.21.15.69:0/1140138596 learned_addr learned my addr 172.21.15.69:0/1140138596 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.164+0000 7f80dde00640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e0077420 0x7f80e019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.164+0000 7f80df200640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e0077dc0 0x7f80e01a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.164+0000 7f80de800640 1 -- 172.21.15.69:0/1140138596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e0077dc0 msgr2=0x7f80e01a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.164+0000 7f80de800640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e0077dc0 0x7f80e01a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.164+0000 7f80de800640 1 -- 172.21.15.69:0/1140138596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e0077420 msgr2=0x7f80e019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.164+0000 7f80de800640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e0077420 0x7f80e019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.164+0000 7f80de800640 1 -- 172.21.15.69:0/1140138596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e01a5da0 con 0x7f80e00761e0 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.164+0000 7f80de800640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e00761e0 0x7f80e019e180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009230 tx=0x7f80d0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.165+0000 7f80d7400640 1 -- 172.21.15.69:0/1140138596 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0041070 con 0x7f80e00761e0 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.165+0000 7f80d7400640 1 -- 172.21.15.69:0/1140138596 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d00044d0 con 0x7f80e00761e0 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.165+0000 7f80e5400640 1 -- 172.21.15.69:0/1140138596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e01a6030 con 0x7f80e00761e0 2024-03-19T01:07:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.165+0000 7f80d7400640 1 -- 172.21.15.69:0/1140138596 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0034440 con 0x7f80e00761e0 2024-03-19T01:07:12.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.165+0000 7f80e5400640 1 -- 172.21.15.69:0/1140138596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e01a6520 con 0x7f80e00761e0 2024-03-19T01:07:12.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.166+0000 7f80e5400640 1 -- 172.21.15.69:0/1140138596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e01105b0 con 0x7f80e00761e0 2024-03-19T01:07:12.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.168+0000 7f80d7400640 1 -- 172.21.15.69:0/1140138596 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f80d0044070 con 0x7f80e00761e0 2024-03-19T01:07:12.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.169+0000 7f80d7400640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80c40a2800 0x7f80c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:12.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.169+0000 7f80d7400640 1 -- 172.21.15.69:0/1140138596 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f80d00f02f0 con 0x7f80e00761e0 2024-03-19T01:07:12.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.169+0000 7f80dde00640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80c40a2800 0x7f80c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:12.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.170+0000 7f80dde00640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80c40a2800 0x7f80c40a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f80e019f690 tx=0x7f80cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:12.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.174+0000 7f80d7400640 1 -- 172.21.15.69:0/1140138596 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f80d00bd790 con 0x7f80e00761e0 2024-03-19T01:07:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:11 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1228016714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:11 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3901775090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:07:12.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:11 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:11 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1228016714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:12.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:11 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3901775090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:12.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:12.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:07:12.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:11 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:11 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1228016714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:12.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:11 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3901775090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:12.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:12.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:07:12.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:11 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:12.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.376+0000 7f80e5400640 1 -- 172.21.15.69:0/1140138596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f80e0062420 con 0x7f80e00761e0 2024-03-19T01:07:12.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.376+0000 7f80d7400640 1 -- 172.21.15.69:0/1140138596 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f80d00bd090 con 0x7f80e00761e0 2024-03-19T01:07:12.377 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:12.378 INFO:teuthology.orchestra.run.smithi069.stdout:[ 2024-03-19T01:07:12.378 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:07:12.378 INFO:teuthology.orchestra.run.smithi069.stdout: "poolnum": 1, 2024-03-19T01:07:12.378 INFO:teuthology.orchestra.run.smithi069.stdout: "poolname": ".mgr" 2024-03-19T01:07:12.378 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:07:12.378 INFO:teuthology.orchestra.run.smithi069.stdout:] 2024-03-19T01:07:12.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 -- 172.21.15.69:0/1140138596 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80c40a2800 msgr2=0x7f80c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:12.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80c40a2800 0x7f80c40a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f80e019f690 tx=0x7f80cc005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:12.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 -- 172.21.15.69:0/1140138596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e00761e0 msgr2=0x7f80e019e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:12.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e00761e0 0x7f80e019e180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009230 tx=0x7f80d0005d50 comp rx=0 tx=0).stop 2024-03-19T01:07:12.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 -- 172.21.15.69:0/1140138596 shutdown_connections 2024-03-19T01:07:12.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80c40a2800 0x7f80c40a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e0077dc0 0x7f80e01a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e0077420 0x7f80e019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 --2- 172.21.15.69:0/1140138596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e00761e0 0x7f80e019e180 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 -- 172.21.15.69:0/1140138596 >> 172.21.15.69:0/1140138596 conn(0x7f80e00ff010 msgr2=0x7f80e0100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:12.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 -- 172.21.15.69:0/1140138596 shutdown_connections 2024-03-19T01:07:12.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.379+0000 7f80e5400640 1 -- 172.21.15.69:0/1140138596 wait complete. 2024-03-19T01:07:12.398 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-19T01:07:12.398 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-19T01:07:12.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.607+0000 7f1447e00640 1 -- 172.21.15.69:0/1700091653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440075800 msgr2=0x7f1440073c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:12.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.607+0000 7f1447e00640 1 --2- 172.21.15.69:0/1700091653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440075800 0x7f1440073c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1434009900 tx=0x7f143402bcd0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.607+0000 7f1447e00640 1 -- 172.21.15.69:0/1700091653 shutdown_connections 2024-03-19T01:07:12.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.607+0000 7f1447e00640 1 --2- 172.21.15.69:0/1700091653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14401096c0 0x7f144010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.607+0000 7f1447e00640 1 --2- 172.21.15.69:0/1700091653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1440074220 0x7f1440074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.607+0000 7f1447e00640 1 --2- 172.21.15.69:0/1700091653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440075800 0x7f1440073c50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.607+0000 7f1447e00640 1 -- 172.21.15.69:0/1700091653 >> 172.21.15.69:0/1700091653 conn(0x7f14400fcdc0 msgr2=0x7f14400ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:12.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.607+0000 7f1447e00640 1 -- 172.21.15.69:0/1700091653 shutdown_connections 2024-03-19T01:07:12.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.607+0000 7f1447e00640 1 -- 172.21.15.69:0/1700091653 wait complete. 2024-03-19T01:07:12.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1447e00640 1 Processor -- start 2024-03-19T01:07:12.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1447e00640 1 -- start start 2024-03-19T01:07:12.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1447e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1440074220 0x7f1440199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:12.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1447e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440075800 0x7f144019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:12.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1447e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14401096c0 0x7f14401a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:12.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1447e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f144010f6f0 con 0x7f14401096c0 2024-03-19T01:07:12.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1447e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f144010f570 con 0x7f1440075800 2024-03-19T01:07:12.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1447e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f144010f870 con 0x7f1440074220 2024-03-19T01:07:12.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1445e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14401096c0 0x7f14401a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:12.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1444a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440075800 0x7f144019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1445e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14401096c0 0x7f14401a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60148/0 (socket says 172.21.15.69:60148) 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.608+0000 7f1445e00640 1 -- 172.21.15.69:0/3586509031 learned_addr learned my addr 172.21.15.69:0/3586509031 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f1445400640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1440074220 0x7f1440199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f1444a00640 1 -- 172.21.15.69:0/3586509031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1440074220 msgr2=0x7f1440199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f1444a00640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1440074220 0x7f1440199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f1444a00640 1 -- 172.21.15.69:0/3586509031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14401096c0 msgr2=0x7f14401a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f1444a00640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14401096c0 0x7f14401a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f1444a00640 1 -- 172.21.15.69:0/3586509031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14401a19c0 con 0x7f1440075800 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f1445400640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1440074220 0x7f1440199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f1444a00640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440075800 0x7f144019a2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f142c00e9f0 tx=0x7f142c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f143e000640 1 -- 172.21.15.69:0/3586509031 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c00ccc0 con 0x7f1440075800 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f143e000640 1 -- 172.21.15.69:0/3586509031 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142c005020 con 0x7f1440075800 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f1447e00640 1 -- 172.21.15.69:0/3586509031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14401a1cb0 con 0x7f1440075800 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.609+0000 7f143e000640 1 -- 172.21.15.69:0/3586509031 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c0049e0 con 0x7f1440075800 2024-03-19T01:07:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.610+0000 7f1447e00640 1 -- 172.21.15.69:0/3586509031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14401a21d0 con 0x7f1440075800 2024-03-19T01:07:12.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.611+0000 7f1447e00640 1 -- 172.21.15.69:0/3586509031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1440108480 con 0x7f1440075800 2024-03-19T01:07:12.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.612+0000 7f143e000640 1 -- 172.21.15.69:0/3586509031 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f142c005560 con 0x7f1440075800 2024-03-19T01:07:12.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.614+0000 7f143e000640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2880 0x7f14280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:12.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.614+0000 7f143e000640 1 -- 172.21.15.69:0/3586509031 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f142c0ce650 con 0x7f1440075800 2024-03-19T01:07:12.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.614+0000 7f1445400640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2880 0x7f14280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:12.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.615+0000 7f1445400640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2880 0x7f14280a4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f143402c2e0 tx=0x7f143402c920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:12.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.618+0000 7f143e000640 1 -- 172.21.15.69:0/3586509031 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f142c0051d0 con 0x7f1440075800 2024-03-19T01:07:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:12.825+0000 7f1447e00640 1 -- 172.21.15.69:0/3586509031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1440062420 con 0x7f1440075800 2024-03-19T01:07:13.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.429+0000 7f143e000640 1 -- 172.21.15.69:0/3586509031 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f142c09baf0 con 0x7f1440075800 2024-03-19T01:07:13.431 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' created 2024-03-19T01:07:13.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.435+0000 7f1447e00640 1 -- 172.21.15.69:0/3586509031 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2880 msgr2=0x7f14280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:13.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.435+0000 7f1447e00640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2880 0x7f14280a4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f143402c2e0 tx=0x7f143402c920 comp rx=0 tx=0).stop 2024-03-19T01:07:13.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.435+0000 7f1447e00640 1 -- 172.21.15.69:0/3586509031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440075800 msgr2=0x7f144019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:13.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.435+0000 7f1447e00640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440075800 0x7f144019a2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f142c00e9f0 tx=0x7f142c00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:07:13.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.436+0000 7f1447e00640 1 -- 172.21.15.69:0/3586509031 shutdown_connections 2024-03-19T01:07:13.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.436+0000 7f1447e00640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2880 0x7f14280a4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:13.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.436+0000 7f1447e00640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14401096c0 0x7f14401a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:13.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.436+0000 7f1447e00640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440075800 0x7f144019a2f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:13.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.436+0000 7f1447e00640 1 --2- 172.21.15.69:0/3586509031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1440074220 0x7f1440199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:13.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.436+0000 7f1447e00640 1 -- 172.21.15.69:0/3586509031 >> 172.21.15.69:0/3586509031 conn(0x7f14400fcdc0 msgr2=0x7f1440105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:13.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.437+0000 7f1447e00640 1 -- 172.21.15.69:0/3586509031 shutdown_connections 2024-03-19T01:07:13.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.437+0000 7f1447e00640 1 -- 172.21.15.69:0/3586509031 wait complete. 2024-03-19T01:07:13.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:13 smithi069 ceph-mon[31727]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:13.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:13 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/462467262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:07:13.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:13 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:07:13.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:13 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1140138596' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-19T01:07:13.460 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-19T01:07:13.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.650+0000 7fd22de00640 1 -- 172.21.15.69:0/2436598692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228107e40 msgr2=0x7fd228108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:13.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.650+0000 7fd22de00640 1 --2- 172.21.15.69:0/2436598692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228107e40 0x7fd228108260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd218009900 tx=0x7fd21802bc50 comp rx=0 tx=0).stop 2024-03-19T01:07:13.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.650+0000 7fd22de00640 1 -- 172.21.15.69:0/2436598692 shutdown_connections 2024-03-19T01:07:13.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.650+0000 7fd22de00640 1 --2- 172.21.15.69:0/2436598692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228109950 0x7fd228110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:13.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.650+0000 7fd22de00640 1 --2- 172.21.15.69:0/2436598692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd228108f90 0x7fd228109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:13.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.650+0000 7fd22de00640 1 --2- 172.21.15.69:0/2436598692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228107e40 0x7fd228108260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:13.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.650+0000 7fd22de00640 1 -- 172.21.15.69:0/2436598692 >> 172.21.15.69:0/2436598692 conn(0x7fd228076de0 msgr2=0x7fd228079220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:13.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.650+0000 7fd22de00640 1 -- 172.21.15.69:0/2436598692 shutdown_connections 2024-03-19T01:07:13.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.650+0000 7fd22de00640 1 -- 172.21.15.69:0/2436598692 wait complete. 2024-03-19T01:07:13.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.651+0000 7fd22de00640 1 Processor -- start 2024-03-19T01:07:13.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.652+0000 7fd22de00640 1 -- start start 2024-03-19T01:07:13.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.652+0000 7fd22de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd228107e40 0x7fd22819e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:13.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.652+0000 7fd22de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228108f90 0x7fd22819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:13.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.652+0000 7fd22de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228109950 0x7fd2281a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.652+0000 7fd22de00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd228113ac0 con 0x7fd228108f90 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.652+0000 7fd22de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd228113940 con 0x7fd228107e40 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.652+0000 7fd22de00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd228113c40 con 0x7fd228109950 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd226a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd228107e40 0x7fd22819e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd226a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd228107e40 0x7fd22819e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44622/0 (socket says 172.21.15.69:44622) 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd226a00640 1 -- 172.21.15.69:0/895496362 learned_addr learned my addr 172.21.15.69:0/895496362 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd226000640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228108f90 0x7fd22819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd227400640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228109950 0x7fd2281a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd226a00640 1 -- 172.21.15.69:0/895496362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228109950 msgr2=0x7fd2281a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd226a00640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228109950 0x7fd2281a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd226a00640 1 -- 172.21.15.69:0/895496362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228108f90 msgr2=0x7fd22819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd226a00640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228108f90 0x7fd22819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd226a00640 1 -- 172.21.15.69:0/895496362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2281a5df0 con 0x7fd228107e40 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd227400640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228109950 0x7fd2281a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:13.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd226a00640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd228107e40 0x7fd22819e1d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd218009230 tx=0x7fd218005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:13.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.653+0000 7fd22ca00640 1 -- 172.21.15.69:0/895496362 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd218042020 con 0x7fd228107e40 2024-03-19T01:07:13.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.654+0000 7fd22de00640 1 -- 172.21.15.69:0/895496362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2281a6080 con 0x7fd228107e40 2024-03-19T01:07:13.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.654+0000 7fd22ca00640 1 -- 172.21.15.69:0/895496362 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2180044d0 con 0x7fd228107e40 2024-03-19T01:07:13.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.654+0000 7fd22ca00640 1 -- 172.21.15.69:0/895496362 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21803e460 con 0x7fd228107e40 2024-03-19T01:07:13.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.654+0000 7fd22de00640 1 -- 172.21.15.69:0/895496362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2281a6620 con 0x7fd228107e40 2024-03-19T01:07:13.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.655+0000 7fd22de00640 1 -- 172.21.15.69:0/895496362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd22810f150 con 0x7fd228107e40 2024-03-19T01:07:13.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.656+0000 7fd22ca00640 1 -- 172.21.15.69:0/895496362 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd218045020 con 0x7fd228107e40 2024-03-19T01:07:13.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.658+0000 7fd22ca00640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd20c0a2800 0x7fd20c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:13.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.658+0000 7fd22ca00640 1 -- 172.21.15.69:0/895496362 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7fd2180f08a0 con 0x7fd228107e40 2024-03-19T01:07:13.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.658+0000 7fd226000640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd20c0a2800 0x7fd20c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:13.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.659+0000 7fd226000640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd20c0a2800 0x7fd20c0a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd2140095f0 tx=0x7fd214005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:13.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.662+0000 7fd22ca00640 1 -- 172.21.15.69:0/895496362 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd21804c020 con 0x7fd228107e40 2024-03-19T01:07:13.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:13 smithi086 ceph-mon[66829]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:13.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:13 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/462467262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:07:13.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:13 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:07:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:13 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1140138596' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-19T01:07:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:13 smithi163 ceph-mon[67428]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:13 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/462467262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:07:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:13 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:07:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:13 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1140138596' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-19T01:07:13.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:13.896+0000 7fd22de00640 1 -- 172.21.15.69:0/895496362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fd22802d1f0 con 0x7fd228107e40 2024-03-19T01:07:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.459+0000 7fd22ca00640 1 -- 172.21.15.69:0/895496362 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7fd2180bdbd0 con 0x7fd228107e40 2024-03-19T01:07:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' created 2024-03-19T01:07:14.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.469+0000 7fd22de00640 1 -- 172.21.15.69:0/895496362 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd20c0a2800 msgr2=0x7fd20c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:14.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.469+0000 7fd22de00640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd20c0a2800 0x7fd20c0a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd2140095f0 tx=0x7fd214005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:14.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.469+0000 7fd22de00640 1 -- 172.21.15.69:0/895496362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd228107e40 msgr2=0x7fd22819e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:14.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.469+0000 7fd22de00640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd228107e40 0x7fd22819e1d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd218009230 tx=0x7fd218005d50 comp rx=0 tx=0).stop 2024-03-19T01:07:14.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.470+0000 7fd22de00640 1 -- 172.21.15.69:0/895496362 shutdown_connections 2024-03-19T01:07:14.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.470+0000 7fd22de00640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd20c0a2800 0x7fd20c0a4cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:14.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.470+0000 7fd22de00640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd228109950 0x7fd2281a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:14.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.470+0000 7fd22de00640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228108f90 0x7fd22819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:14.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.470+0000 7fd22de00640 1 --2- 172.21.15.69:0/895496362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd228107e40 0x7fd22819e1d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:14.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.470+0000 7fd22de00640 1 -- 172.21.15.69:0/895496362 >> 172.21.15.69:0/895496362 conn(0x7fd228076de0 msgr2=0x7fd22810c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:14.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.471+0000 7fd22de00640 1 -- 172.21.15.69:0/895496362 shutdown_connections 2024-03-19T01:07:14.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.475+0000 7fd22de00640 1 -- 172.21.15.69:0/895496362 wait complete. 2024-03-19T01:07:14.502 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-19T01:07:14.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.683+0000 7fa691800640 1 -- 172.21.15.69:0/1594145311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c1033e0 msgr2=0x7fa68c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:14.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.683+0000 7fa691800640 1 --2- 172.21.15.69:0/1594145311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c1033e0 0x7fa68c109c70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa684009ca0 tx=0x7fa68402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:07:14.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.684+0000 7fa691800640 1 -- 172.21.15.69:0/1594145311 shutdown_connections 2024-03-19T01:07:14.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.684+0000 7fa691800640 1 --2- 172.21.15.69:0/1594145311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c1033e0 0x7fa68c109c70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:14.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.684+0000 7fa691800640 1 --2- 172.21.15.69:0/1594145311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c102a20 0x7fa68c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:14.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.684+0000 7fa691800640 1 --2- 172.21.15.69:0/1594145311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c101830 0x7fa68c101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:14.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.684+0000 7fa691800640 1 -- 172.21.15.69:0/1594145311 >> 172.21.15.69:0/1594145311 conn(0x7fa68c0fcdc0 msgr2=0x7fa68c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:14.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.684+0000 7fa691800640 1 -- 172.21.15.69:0/1594145311 shutdown_connections 2024-03-19T01:07:14.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.684+0000 7fa691800640 1 -- 172.21.15.69:0/1594145311 wait complete. 2024-03-19T01:07:14.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa691800640 1 Processor -- start 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa691800640 1 -- start start 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa691800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c101830 0x7fa68c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa691800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c102a20 0x7fa68c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa691800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c1033e0 0x7fa68c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa691800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c10d490 con 0x7fa68c101830 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa691800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c10d310 con 0x7fa68c102a20 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa691800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c10d610 con 0x7fa68c1033e0 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa68aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c101830 0x7fa68c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa68aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c101830 0x7fa68c199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60194/0 (socket says 172.21.15.69:60194) 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa68aa00640 1 -- 172.21.15.69:0/1967722129 learned_addr learned my addr 172.21.15.69:0/1967722129 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa68a000640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c102a20 0x7fa68c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.685+0000 7fa68b400640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c1033e0 0x7fa68c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.686+0000 7fa68b400640 1 -- 172.21.15.69:0/1967722129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c102a20 msgr2=0x7fa68c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.686+0000 7fa68b400640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c102a20 0x7fa68c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:14.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.686+0000 7fa68b400640 1 -- 172.21.15.69:0/1967722129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c101830 msgr2=0x7fa68c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:14.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.686+0000 7fa68b400640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c101830 0x7fa68c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:14.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.686+0000 7fa68b400640 1 -- 172.21.15.69:0/1967722129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68c1a1900 con 0x7fa68c1033e0 2024-03-19T01:07:14.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.686+0000 7fa68aa00640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c101830 0x7fa68c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:14.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.686+0000 7fa68b400640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c1033e0 0x7fa68c1a1260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa684037040 tx=0x7fa684032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:14.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.686+0000 7fa683400640 1 -- 172.21.15.69:0/1967722129 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa684041020 con 0x7fa68c1033e0 2024-03-19T01:07:14.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.686+0000 7fa683400640 1 -- 172.21.15.69:0/1967722129 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa684034cd0 con 0x7fa68c1033e0 2024-03-19T01:07:14.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.686+0000 7fa691800640 1 -- 172.21.15.69:0/1967722129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68c1a1b90 con 0x7fa68c1033e0 2024-03-19T01:07:14.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.686+0000 7fa683400640 1 -- 172.21.15.69:0/1967722129 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa684039040 con 0x7fa68c1033e0 2024-03-19T01:07:14.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.687+0000 7fa691800640 1 -- 172.21.15.69:0/1967722129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa68c1a20a0 con 0x7fa68c1033e0 2024-03-19T01:07:14.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.688+0000 7fa691800640 1 -- 172.21.15.69:0/1967722129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa68c1a2270 con 0x7fa68c1033e0 2024-03-19T01:07:14.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.689+0000 7fa683400640 1 -- 172.21.15.69:0/1967722129 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa684044020 con 0x7fa68c1033e0 2024-03-19T01:07:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.690+0000 7fa683400640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa6700a2880 0x7fa6700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.690+0000 7fa683400640 1 -- 172.21.15.69:0/1967722129 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fa6840f07b0 con 0x7fa68c1033e0 2024-03-19T01:07:14.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.691+0000 7fa68aa00640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa6700a2880 0x7fa6700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:14.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.691+0000 7fa68aa00640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa6700a2880 0x7fa6700a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa68c19b1c0 tx=0x7fa67c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:14.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.694+0000 7fa683400640 1 -- 172.21.15.69:0/1967722129 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa68404b020 con 0x7fa68c1033e0 2024-03-19T01:07:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:14 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3586509031' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T01:07:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:14 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T01:07:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:14 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-19T01:07:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:14 smithi086 ceph-mon[66829]: osdmap e82: 12 total, 12 up, 12 in 2024-03-19T01:07:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:14 smithi086 ceph-mon[66829]: pgmap v307: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:14 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3586509031' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T01:07:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:14 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T01:07:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:14 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-19T01:07:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:14 smithi069 ceph-mon[31727]: osdmap e82: 12 total, 12 up, 12 in 2024-03-19T01:07:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:14 smithi069 ceph-mon[31727]: pgmap v307: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:14 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3586509031' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T01:07:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:14 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T01:07:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:14 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-19T01:07:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:14 smithi163 ceph-mon[67428]: osdmap e82: 12 total, 12 up, 12 in 2024-03-19T01:07:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:14 smithi163 ceph-mon[67428]: pgmap v307: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:14.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:14.925+0000 7fa691800640 1 -- 172.21.15.69:0/1967722129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa68c1a29a0 con 0x7fa68c1033e0 2024-03-19T01:07:15.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.619+0000 7fa683400640 1 -- 172.21.15.69:0/1967722129 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa6840bd960 con 0x7fa68c1033e0 2024-03-19T01:07:15.621 INFO:teuthology.orchestra.run.smithi069.stderr:new fs with metadata pool 2 and data pool 3 2024-03-19T01:07:15.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.626+0000 7fa691800640 1 -- 172.21.15.69:0/1967722129 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa6700a2880 msgr2=0x7fa6700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:15.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.626+0000 7fa691800640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa6700a2880 0x7fa6700a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa68c19b1c0 tx=0x7fa67c005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:15.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.626+0000 7fa691800640 1 -- 172.21.15.69:0/1967722129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c1033e0 msgr2=0x7fa68c1a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:15.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.626+0000 7fa691800640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c1033e0 0x7fa68c1a1260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa684037040 tx=0x7fa684032ea0 comp rx=0 tx=0).stop 2024-03-19T01:07:15.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.627+0000 7fa691800640 1 -- 172.21.15.69:0/1967722129 shutdown_connections 2024-03-19T01:07:15.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.627+0000 7fa691800640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa6700a2880 0x7fa6700a4d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:15.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.627+0000 7fa691800640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c1033e0 0x7fa68c1a1260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:15.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.627+0000 7fa691800640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c102a20 0x7fa68c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:15.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.627+0000 7fa691800640 1 --2- 172.21.15.69:0/1967722129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c101830 0x7fa68c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:15.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.627+0000 7fa691800640 1 -- 172.21.15.69:0/1967722129 >> 172.21.15.69:0/1967722129 conn(0x7fa68c0fcdc0 msgr2=0x7fa68c073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:15.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.628+0000 7fa691800640 1 -- 172.21.15.69:0/1967722129 shutdown_connections 2024-03-19T01:07:15.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.628+0000 7fa691800640 1 -- 172.21.15.69:0/1967722129 wait complete. 2024-03-19T01:07:15.649 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-19T01:07:15.649 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-19T01:07:15.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/895496362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: osdmap e83: 12 total, 12 up, 12 in 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1967722129' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T01:07:15.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:15 smithi069 ceph-mon[31727]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-19T01:07:15.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/895496362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: osdmap e83: 12 total, 12 up, 12 in 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1967722129' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T01:07:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:15 smithi086 ceph-mon[66829]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-19T01:07:15.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/895496362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-19T01:07:15.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-19T01:07:15.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-19T01:07:15.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: osdmap e83: 12 total, 12 up, 12 in 2024-03-19T01:07:15.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:15.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:15.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:07:15.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:07:15.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:07:15.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:07:15.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1967722129' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T01:07:15.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T01:07:15.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:15 smithi163 ceph-mon[67428]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-19T01:07:15.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.841+0000 7fe5b5600640 1 -- 172.21.15.69:0/846703809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0103a50 msgr2=0x7fe5b0103e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:15.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.841+0000 7fe5b5600640 1 --2- 172.21.15.69:0/846703809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0103a50 0x7fe5b0103e70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009900 tx=0x7fe5a002bc50 comp rx=0 tx=0).stop 2024-03-19T01:07:15.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.841+0000 7fe5b5600640 1 -- 172.21.15.69:0/846703809 shutdown_connections 2024-03-19T01:07:15.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.841+0000 7fe5b5600640 1 --2- 172.21.15.69:0/846703809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5b0105600 0x7fe5b010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:15.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.841+0000 7fe5b5600640 1 --2- 172.21.15.69:0/846703809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b0104c40 0x7fe5b01050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:15.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.841+0000 7fe5b5600640 1 --2- 172.21.15.69:0/846703809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0103a50 0x7fe5b0103e70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:15.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.841+0000 7fe5b5600640 1 -- 172.21.15.69:0/846703809 >> 172.21.15.69:0/846703809 conn(0x7fe5b00fefe0 msgr2=0x7fe5b0101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:15.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.841+0000 7fe5b5600640 1 -- 172.21.15.69:0/846703809 shutdown_connections 2024-03-19T01:07:15.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.841+0000 7fe5b5600640 1 -- 172.21.15.69:0/846703809 wait complete. 2024-03-19T01:07:15.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.842+0000 7fe5b5600640 1 Processor -- start 2024-03-19T01:07:15.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.842+0000 7fe5b5600640 1 -- start start 2024-03-19T01:07:15.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5b5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b0103a50 0x7fe5b0199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:15.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5b5600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0104c40 0x7fe5b019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:15.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5b5600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5b0105600 0x7fe5b01a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:15.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5b5600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b010f6d0 con 0x7fe5b0104c40 2024-03-19T01:07:15.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5b5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b010f550 con 0x7fe5b0103a50 2024-03-19T01:07:15.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5b5600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b010f850 con 0x7fe5b0105600 2024-03-19T01:07:15.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5ae000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0104c40 0x7fe5b019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:15.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5aea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b0103a50 0x7fe5b0199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:15.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5ae000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0104c40 0x7fe5b019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60212/0 (socket says 172.21.15.69:60212) 2024-03-19T01:07:15.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5ae000640 1 -- 172.21.15.69:0/1246265695 learned_addr learned my addr 172.21.15.69:0/1246265695 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:15.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5af400640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5b0105600 0x7fe5b01a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:15.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5ae000640 1 -- 172.21.15.69:0/1246265695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5b0105600 msgr2=0x7fe5b01a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:15.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5ae000640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5b0105600 0x7fe5b01a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:15.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5ae000640 1 -- 172.21.15.69:0/1246265695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b0103a50 msgr2=0x7fe5b0199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:15.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5ae000640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b0103a50 0x7fe5b0199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:15.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5ae000640 1 -- 172.21.15.69:0/1246265695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b01a1a30 con 0x7fe5b0104c40 2024-03-19T01:07:15.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.843+0000 7fe5aea00640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b0103a50 0x7fe5b0199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:15.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.844+0000 7fe5ae000640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0104c40 0x7fe5b019a350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00e9f0 tx=0x7fe59c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:15.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.844+0000 7fe5a7400640 1 -- 172.21.15.69:0/1246265695 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00cd80 con 0x7fe5b0104c40 2024-03-19T01:07:15.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.844+0000 7fe5a7400640 1 -- 172.21.15.69:0/1246265695 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe59c010910 con 0x7fe5b0104c40 2024-03-19T01:07:15.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.844+0000 7fe5b5600640 1 -- 172.21.15.69:0/1246265695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b01a1d20 con 0x7fe5b0104c40 2024-03-19T01:07:15.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.844+0000 7fe5a7400640 1 -- 172.21.15.69:0/1246265695 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c018990 con 0x7fe5b0104c40 2024-03-19T01:07:15.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.844+0000 7fe5b5600640 1 -- 172.21.15.69:0/1246265695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b01a2260 con 0x7fe5b0104c40 2024-03-19T01:07:15.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.845+0000 7fe5b5600640 1 -- 172.21.15.69:0/1246265695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b010adf0 con 0x7fe5b0104c40 2024-03-19T01:07:15.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.848+0000 7fe5a7400640 1 -- 172.21.15.69:0/1246265695 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe59c010ac0 con 0x7fe5b0104c40 2024-03-19T01:07:15.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.849+0000 7fe5a7400640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5940a2880 0x7fe5940a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:15.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.850+0000 7fe5a7400640 1 -- 172.21.15.69:0/1246265695 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fe59c0cec40 con 0x7fe5b0104c40 2024-03-19T01:07:15.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.850+0000 7fe5aea00640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5940a2880 0x7fe5940a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:15.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.850+0000 7fe5aea00640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5940a2880 0x7fe5940a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0000c00 tx=0x7fe5a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:15.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:15.853+0000 7fe5a7400640 1 -- 172.21.15.69:0/1246265695 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe59c09b740 con 0x7fe5b0104c40 2024-03-19T01:07:16.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.060+0000 7fe5b5600640 1 -- 172.21.15.69:0/1246265695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fe5b00624c0 con 0x7fe5b0104c40 2024-03-19T01:07:16.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.454+0000 7fe5a7400640 1 -- 172.21.15.69:0/1246265695 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7fe59c09f7e0 con 0x7fe5b0104c40 2024-03-19T01:07:16.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.458+0000 7fe5b5600640 1 -- 172.21.15.69:0/1246265695 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5940a2880 msgr2=0x7fe5940a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:16.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.458+0000 7fe5b5600640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5940a2880 0x7fe5940a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0000c00 tx=0x7fe5a0005fd0 comp rx=0 tx=0).stop 2024-03-19T01:07:16.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.458+0000 7fe5b5600640 1 -- 172.21.15.69:0/1246265695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0104c40 msgr2=0x7fe5b019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:16.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.458+0000 7fe5b5600640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0104c40 0x7fe5b019a350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00e9f0 tx=0x7fe59c00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:07:16.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.458+0000 7fe5b5600640 1 -- 172.21.15.69:0/1246265695 shutdown_connections 2024-03-19T01:07:16.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.458+0000 7fe5b5600640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5940a2880 0x7fe5940a4d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:16.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.458+0000 7fe5b5600640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5b0105600 0x7fe5b01a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:16.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.458+0000 7fe5b5600640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5b0104c40 0x7fe5b019a350 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:16.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.458+0000 7fe5b5600640 1 --2- 172.21.15.69:0/1246265695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b0103a50 0x7fe5b0199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:16.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.458+0000 7fe5b5600640 1 -- 172.21.15.69:0/1246265695 >> 172.21.15.69:0/1246265695 conn(0x7fe5b00fefe0 msgr2=0x7fe5b0107fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:16.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.459+0000 7fe5b5600640 1 -- 172.21.15.69:0/1246265695 shutdown_connections 2024-03-19T01:07:16.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.459+0000 7fe5b5600640 1 -- 172.21.15.69:0/1246265695 wait complete. 2024-03-19T01:07:16.481 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-19T01:07:16.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.708+0000 7f836e400640 1 -- 172.21.15.69:0/160489718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8368108fb0 msgr2=0x7f8368109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:16.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.708+0000 7f836e400640 1 --2- 172.21.15.69:0/160489718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8368108fb0 0x7f8368109430 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8350009900 tx=0x7f835002bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:16.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.708+0000 7f836e400640 1 -- 172.21.15.69:0/160489718 shutdown_connections 2024-03-19T01:07:16.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.708+0000 7f836e400640 1 --2- 172.21.15.69:0/160489718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368109970 0x7f8368110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:16.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.708+0000 7f836e400640 1 --2- 172.21.15.69:0/160489718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8368108fb0 0x7f8368109430 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:16.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.708+0000 7f836e400640 1 --2- 172.21.15.69:0/160489718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8368107e60 0x7f8368108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:16.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.708+0000 7f836e400640 1 -- 172.21.15.69:0/160489718 >> 172.21.15.69:0/160489718 conn(0x7f8368076e30 msgr2=0x7f8368079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:16.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.708+0000 7f836e400640 1 -- 172.21.15.69:0/160489718 shutdown_connections 2024-03-19T01:07:16.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.708+0000 7f836e400640 1 -- 172.21.15.69:0/160489718 wait complete. 2024-03-19T01:07:16.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.709+0000 7f836e400640 1 Processor -- start 2024-03-19T01:07:16.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.709+0000 7f836e400640 1 -- start start 2024-03-19T01:07:16.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.709+0000 7f836e400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8368107e60 0x7f836819e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:16.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.709+0000 7f836e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8368108fb0 0x7f836819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:16.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.709+0000 7f836e400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368109970 0x7f83681a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:16.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.709+0000 7f836e400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8368113b50 con 0x7f8368109970 2024-03-19T01:07:16.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.709+0000 7f836e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83681139d0 con 0x7f8368108fb0 2024-03-19T01:07:16.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.709+0000 7f836e400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8368113cd0 con 0x7f8368107e60 2024-03-19T01:07:16.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.710+0000 7f835ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8368108fb0 0x7f836819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:16.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.710+0000 7f835ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8368108fb0 0x7f836819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42642/0 (socket says 172.21.15.69:42642) 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.710+0000 7f835ea00640 1 -- 172.21.15.69:0/1169837629 learned_addr learned my addr 172.21.15.69:0/1169837629 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.710+0000 7f8367e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368109970 0x7f83681a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.710+0000 7f8367400640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8368107e60 0x7f836819e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.710+0000 7f835ea00640 1 -- 172.21.15.69:0/1169837629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8368107e60 msgr2=0x7f836819e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.710+0000 7f835ea00640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8368107e60 0x7f836819e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.710+0000 7f835ea00640 1 -- 172.21.15.69:0/1169837629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368109970 msgr2=0x7f83681a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.710+0000 7f835ea00640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368109970 0x7f83681a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.710+0000 7f835ea00640 1 -- 172.21.15.69:0/1169837629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83681a5d80 con 0x7f8368108fb0 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.710+0000 7f835ea00640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8368108fb0 0x7f836819e6a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8350000c00 tx=0x7f835002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.711+0000 7f8364c00640 1 -- 172.21.15.69:0/1169837629 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350043020 con 0x7f8368108fb0 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.711+0000 7f8364c00640 1 -- 172.21.15.69:0/1169837629 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83500054e0 con 0x7f8368108fb0 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.711+0000 7f836e400640 1 -- 172.21.15.69:0/1169837629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83681a6010 con 0x7f8368108fb0 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.711+0000 7f8364c00640 1 -- 172.21.15.69:0/1169837629 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350003c40 con 0x7f8368108fb0 2024-03-19T01:07:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.711+0000 7f836e400640 1 -- 172.21.15.69:0/1169837629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83681a65b0 con 0x7f8368108fb0 2024-03-19T01:07:16.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.712+0000 7f836e400640 1 -- 172.21.15.69:0/1169837629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f836810f110 con 0x7f8368108fb0 2024-03-19T01:07:16.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.714+0000 7f8364c00640 1 -- 172.21.15.69:0/1169837629 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8350046020 con 0x7f8368108fb0 2024-03-19T01:07:16.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.715+0000 7f8364c00640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f834c0a2880 0x7f834c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:16.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.715+0000 7f8364c00640 1 -- 172.21.15.69:0/1169837629 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f83500f0f70 con 0x7f8368108fb0 2024-03-19T01:07:16.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.715+0000 7f8367400640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f834c0a2880 0x7f834c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:16.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.716+0000 7f8367400640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f834c0a2880 0x7f834c0a4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8358005fd0 tx=0x7f8358009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:16.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.717+0000 7f8364c00640 1 -- 172.21.15.69:0/1169837629 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f835004d020 con 0x7f8368108fb0 2024-03-19T01:07:16.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:16 smithi086 ceph-mon[66829]: osdmap e84: 12 total, 12 up, 12 in 2024-03-19T01:07:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:16 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-19T01:07:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:16 smithi086 ceph-mon[66829]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-19T01:07:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:16 smithi086 ceph-mon[66829]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-19T01:07:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:16 smithi086 ceph-mon[66829]: pgmap v310: 97 pgs: 3 creating+peering, 8 active+clean, 86 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:16 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1246265695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-19T01:07:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:16 smithi069 ceph-mon[31727]: osdmap e84: 12 total, 12 up, 12 in 2024-03-19T01:07:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:16 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-19T01:07:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:16 smithi069 ceph-mon[31727]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-19T01:07:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:16 smithi069 ceph-mon[31727]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-19T01:07:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:16 smithi069 ceph-mon[31727]: pgmap v310: 97 pgs: 3 creating+peering, 8 active+clean, 86 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:16 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1246265695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-19T01:07:16.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:16 smithi163 ceph-mon[67428]: osdmap e84: 12 total, 12 up, 12 in 2024-03-19T01:07:16.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:16 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-19T01:07:16.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:16 smithi163 ceph-mon[67428]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-19T01:07:16.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:16 smithi163 ceph-mon[67428]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-19T01:07:16.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:16 smithi163 ceph-mon[67428]: pgmap v310: 97 pgs: 3 creating+peering, 8 active+clean, 86 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:07:16.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:16 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1246265695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-19T01:07:16.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:16.917+0000 7f836e400640 1 -- 172.21.15.69:0/1169837629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f836802d1f0 con 0x7f8368108fb0 2024-03-19T01:07:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:17 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1246265695' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-19T01:07:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:17 smithi086 ceph-mon[66829]: osdmap e85: 12 total, 12 up, 12 in 2024-03-19T01:07:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:17 smithi086 ceph-mon[66829]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:07:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:17 smithi086 ceph-mon[66829]: Cluster is now healthy 2024-03-19T01:07:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:17 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] up:active 2024-03-19T01:07:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:17 smithi086 ceph-mon[66829]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:17 smithi086 ceph-mon[66829]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:17 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1169837629' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T01:07:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:17 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T01:07:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:17 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1246265695' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-19T01:07:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:17 smithi069 ceph-mon[31727]: osdmap e85: 12 total, 12 up, 12 in 2024-03-19T01:07:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:17 smithi069 ceph-mon[31727]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:07:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:17 smithi069 ceph-mon[31727]: Cluster is now healthy 2024-03-19T01:07:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:17 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] up:active 2024-03-19T01:07:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:17 smithi069 ceph-mon[31727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:17.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:17 smithi069 ceph-mon[31727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:17.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:17 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1169837629' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T01:07:17.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:17 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T01:07:17.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:17 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1246265695' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-19T01:07:17.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:17 smithi163 ceph-mon[67428]: osdmap e85: 12 total, 12 up, 12 in 2024-03-19T01:07:17.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:17 smithi163 ceph-mon[67428]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:07:17.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:17 smithi163 ceph-mon[67428]: Cluster is now healthy 2024-03-19T01:07:17.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:17 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] up:active 2024-03-19T01:07:17.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:17 smithi163 ceph-mon[67428]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:17.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:17 smithi163 ceph-mon[67428]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:17 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1169837629' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T01:07:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:17 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T01:07:18.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.650+0000 7f8364c00640 1 -- 172.21.15.69:0/1169837629 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f83500bdf80 con 0x7f8368108fb0 2024-03-19T01:07:18.652 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data_ec' created 2024-03-19T01:07:18.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.656+0000 7f836e400640 1 -- 172.21.15.69:0/1169837629 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f834c0a2880 msgr2=0x7f834c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:18.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.656+0000 7f836e400640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f834c0a2880 0x7f834c0a4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8358005fd0 tx=0x7f8358009500 comp rx=0 tx=0).stop 2024-03-19T01:07:18.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.656+0000 7f836e400640 1 -- 172.21.15.69:0/1169837629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8368108fb0 msgr2=0x7f836819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:18.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.656+0000 7f836e400640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8368108fb0 0x7f836819e6a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8350000c00 tx=0x7f835002c650 comp rx=0 tx=0).stop 2024-03-19T01:07:18.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.657+0000 7f836e400640 1 -- 172.21.15.69:0/1169837629 shutdown_connections 2024-03-19T01:07:18.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.657+0000 7f836e400640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f834c0a2880 0x7f834c0a4d40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:18.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.657+0000 7f836e400640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368109970 0x7f83681a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:18.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.657+0000 7f836e400640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8368108fb0 0x7f836819e6a0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8350000c00 tx=0x7f835002c650 comp rx=0 tx=0).stop 2024-03-19T01:07:18.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.657+0000 7f836e400640 1 --2- 172.21.15.69:0/1169837629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8368107e60 0x7f836819e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:18.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.657+0000 7f836e400640 1 -- 172.21.15.69:0/1169837629 >> 172.21.15.69:0/1169837629 conn(0x7f8368076e30 msgr2=0x7f836810c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.658+0000 7f836e400640 1 -- 172.21.15.69:0/1169837629 shutdown_connections 2024-03-19T01:07:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.658+0000 7f836e400640 1 -- 172.21.15.69:0/1169837629 wait complete. 2024-03-19T01:07:18.687 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-19T01:07:18.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.884+0000 7f592a400640 1 -- 172.21.15.69:0/1265708454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5924104c20 msgr2=0x7f59241050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:18.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.884+0000 7f592a400640 1 --2- 172.21.15.69:0/1265708454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5924104c20 0x7f59241050a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f590c009900 tx=0x7f590c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:18.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.885+0000 7f592a400640 1 -- 172.21.15.69:0/1265708454 shutdown_connections 2024-03-19T01:07:18.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.885+0000 7f592a400640 1 --2- 172.21.15.69:0/1265708454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59241055e0 0x7f592410be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:18.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.885+0000 7f592a400640 1 --2- 172.21.15.69:0/1265708454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5924104c20 0x7f59241050a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:18.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.885+0000 7f592a400640 1 --2- 172.21.15.69:0/1265708454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5924103a30 0x7f5924103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:18.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.885+0000 7f592a400640 1 -- 172.21.15.69:0/1265708454 >> 172.21.15.69:0/1265708454 conn(0x7f59240fefe0 msgr2=0x7f5924101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:18.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.885+0000 7f592a400640 1 -- 172.21.15.69:0/1265708454 shutdown_connections 2024-03-19T01:07:18.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.885+0000 7f592a400640 1 -- 172.21.15.69:0/1265708454 wait complete. 2024-03-19T01:07:18.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f592a400640 1 Processor -- start 2024-03-19T01:07:18.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f592a400640 1 -- start start 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f592a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5924103a30 0x7f5924199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f592a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5924104c20 0x7f592419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f592a400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59241055e0 0x7f59241a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f592a400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f592410f6e0 con 0x7f5924104c20 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f592a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f592410f560 con 0x7f5924103a30 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f592a400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f592410f860 con 0x7f59241055e0 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f5923400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5924103a30 0x7f5924199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f5923400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5924103a30 0x7f5924199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42654/0 (socket says 172.21.15.69:42654) 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f5923400640 1 -- 172.21.15.69:0/3604415477 learned_addr learned my addr 172.21.15.69:0/3604415477 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.886+0000 7f5922a00640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5924104c20 0x7f592419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f5923e00640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59241055e0 0x7f59241a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:18.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f5923400640 1 -- 172.21.15.69:0/3604415477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59241055e0 msgr2=0x7f59241a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f5923400640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59241055e0 0x7f59241a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f5923400640 1 -- 172.21.15.69:0/3604415477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5924104c20 msgr2=0x7f592419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f5923400640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5924104c20 0x7f592419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f5923400640 1 -- 172.21.15.69:0/3604415477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59241a1950 con 0x7f5924103a30 2024-03-19T01:07:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f5923e00640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59241055e0 0x7f59241a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:07:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f5923400640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5924103a30 0x7f5924199d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f591400e9c0 tx=0x7f591400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f591be00640 1 -- 172.21.15.69:0/3604415477 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591400cd80 con 0x7f5924103a30 2024-03-19T01:07:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f592a400640 1 -- 172.21.15.69:0/3604415477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59241a1c40 con 0x7f5924103a30 2024-03-19T01:07:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f591be00640 1 -- 172.21.15.69:0/3604415477 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59140035f0 con 0x7f5924103a30 2024-03-19T01:07:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f591be00640 1 -- 172.21.15.69:0/3604415477 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914010690 con 0x7f5924103a30 2024-03-19T01:07:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.887+0000 7f592a400640 1 -- 172.21.15.69:0/3604415477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59241a2190 con 0x7f5924103a30 2024-03-19T01:07:18.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.888+0000 7f592a400640 1 -- 172.21.15.69:0/3604415477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592410add0 con 0x7f5924103a30 2024-03-19T01:07:18.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.890+0000 7f591be00640 1 -- 172.21.15.69:0/3604415477 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f59140037a0 con 0x7f5924103a30 2024-03-19T01:07:18.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.891+0000 7f591be00640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f59080a2800 0x7f59080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:18.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.892+0000 7f591be00640 1 -- 172.21.15.69:0/3604415477 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f59140cf200 con 0x7f5924103a30 2024-03-19T01:07:18.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.892+0000 7f5922a00640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f59080a2800 0x7f59080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:18.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.892+0000 7f5922a00640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f59080a2800 0x7f59080a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f590c02c240 tx=0x7f590c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:18.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:18.900+0000 7f591be00640 1 -- 172.21.15.69:0/3604415477 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f591409c020 con 0x7f5924103a30 2024-03-19T01:07:19.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:18 smithi086 ceph-mon[66829]: pgmap v313: 97 pgs: 3 creating+peering, 54 active+clean, 40 unknown; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-03-19T01:07:19.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:18 smithi086 ceph-mon[66829]: osdmap e86: 12 total, 12 up, 12 in 2024-03-19T01:07:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:18 smithi069 ceph-mon[31727]: pgmap v313: 97 pgs: 3 creating+peering, 54 active+clean, 40 unknown; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-03-19T01:07:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:18 smithi069 ceph-mon[31727]: osdmap e86: 12 total, 12 up, 12 in 2024-03-19T01:07:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:18 smithi163 ceph-mon[67428]: pgmap v313: 97 pgs: 3 creating+peering, 54 active+clean, 40 unknown; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-03-19T01:07:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:18 smithi163 ceph-mon[67428]: osdmap e86: 12 total, 12 up, 12 in 2024-03-19T01:07:19.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.097+0000 7f592a400640 1 -- 172.21.15.69:0/3604415477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f592402d1f0 con 0x7f5924103a30 2024-03-19T01:07:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.653+0000 7f591be00640 1 -- 172.21.15.69:0/3604415477 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7f5914084e20 con 0x7f5924103a30 2024-03-19T01:07:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:set pool 4 allow_ec_overwrites to true 2024-03-19T01:07:19.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.658+0000 7f592a400640 1 -- 172.21.15.69:0/3604415477 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f59080a2800 msgr2=0x7f59080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:19.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.658+0000 7f592a400640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f59080a2800 0x7f59080a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f590c02c240 tx=0x7f590c02df90 comp rx=0 tx=0).stop 2024-03-19T01:07:19.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.658+0000 7f592a400640 1 -- 172.21.15.69:0/3604415477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5924103a30 msgr2=0x7f5924199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:19.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.658+0000 7f592a400640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5924103a30 0x7f5924199d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f591400e9c0 tx=0x7f591400ee80 comp rx=0 tx=0).stop 2024-03-19T01:07:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.660+0000 7f592a400640 1 -- 172.21.15.69:0/3604415477 shutdown_connections 2024-03-19T01:07:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.660+0000 7f592a400640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f59080a2800 0x7f59080a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.660+0000 7f592a400640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59241055e0 0x7f59241a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.660+0000 7f592a400640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5924104c20 0x7f592419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.660+0000 7f592a400640 1 --2- 172.21.15.69:0/3604415477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5924103a30 0x7f5924199d40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.660+0000 7f592a400640 1 -- 172.21.15.69:0/3604415477 >> 172.21.15.69:0/3604415477 conn(0x7f59240fefe0 msgr2=0x7f5924107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:19.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.662+0000 7f592a400640 1 -- 172.21.15.69:0/3604415477 shutdown_connections 2024-03-19T01:07:19.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.662+0000 7f592a400640 1 -- 172.21.15.69:0/3604415477 wait complete. 2024-03-19T01:07:19.698 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-19T01:07:19.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.879+0000 7f62dfe00640 1 -- 172.21.15.69:0/932270300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8107e40 msgr2=0x7f62d8108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:19.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.879+0000 7f62dfe00640 1 --2- 172.21.15.69:0/932270300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8107e40 0x7f62d8108260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009900 tx=0x7f62cc02bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:19.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.879+0000 7f62dfe00640 1 -- 172.21.15.69:0/932270300 shutdown_connections 2024-03-19T01:07:19.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.879+0000 7f62dfe00640 1 --2- 172.21.15.69:0/932270300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d8109950 0x7f62d8110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:19.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.879+0000 7f62dfe00640 1 --2- 172.21.15.69:0/932270300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d8108f90 0x7f62d8109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:19.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.879+0000 7f62dfe00640 1 --2- 172.21.15.69:0/932270300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8107e40 0x7f62d8108260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:19.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.879+0000 7f62dfe00640 1 -- 172.21.15.69:0/932270300 >> 172.21.15.69:0/932270300 conn(0x7f62d8076de0 msgr2=0x7f62d8079220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:19.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.879+0000 7f62dfe00640 1 -- 172.21.15.69:0/932270300 shutdown_connections 2024-03-19T01:07:19.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.879+0000 7f62dfe00640 1 -- 172.21.15.69:0/932270300 wait complete. 2024-03-19T01:07:19.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.880+0000 7f62dfe00640 1 Processor -- start 2024-03-19T01:07:19.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.880+0000 7f62dfe00640 1 -- start start 2024-03-19T01:07:19.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d8107e40 0x7f62d819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:19.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d8108f90 0x7f62d819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:19.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8109950 0x7f62d81a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:19.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dfe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8113b10 con 0x7f62d8108f90 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8113990 con 0x7f62d8109950 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dfe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8113c90 con 0x7f62d8107e40 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8109950 0x7f62d81a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dd400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d8107e40 0x7f62d819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8109950 0x7f62d81a56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42674/0 (socket says 172.21.15.69:42674) 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d8108f90 0x7f62d819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dde00640 1 -- 172.21.15.69:0/1918024259 learned_addr learned my addr 172.21.15.69:0/1918024259 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dde00640 1 -- 172.21.15.69:0/1918024259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d8107e40 msgr2=0x7f62d819e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dde00640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d8107e40 0x7f62d819e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dde00640 1 -- 172.21.15.69:0/1918024259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d8108f90 msgr2=0x7f62d819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dde00640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d8108f90 0x7f62d819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.881+0000 7f62dde00640 1 -- 172.21.15.69:0/1918024259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d81a5d50 con 0x7f62d8109950 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.882+0000 7f62dde00640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8109950 0x7f62d81a56e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62d000eaa0 tx=0x7f62d000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:19.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.882+0000 7f62d6000640 1 -- 172.21.15.69:0/1918024259 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d000cdb0 con 0x7f62d8109950 2024-03-19T01:07:19.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.882+0000 7f62d6000640 1 -- 172.21.15.69:0/1918024259 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d0005020 con 0x7f62d8109950 2024-03-19T01:07:19.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.882+0000 7f62d6000640 1 -- 172.21.15.69:0/1918024259 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d00116b0 con 0x7f62d8109950 2024-03-19T01:07:19.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.882+0000 7f62dfe00640 1 -- 172.21.15.69:0/1918024259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d81a5fe0 con 0x7f62d8109950 2024-03-19T01:07:19.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.882+0000 7f62dfe00640 1 -- 172.21.15.69:0/1918024259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d81a64b0 con 0x7f62d8109950 2024-03-19T01:07:19.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.883+0000 7f62dfe00640 1 -- 172.21.15.69:0/1918024259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d810f150 con 0x7f62d8109950 2024-03-19T01:07:19.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.885+0000 7f62d6000640 1 -- 172.21.15.69:0/1918024259 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f62d0005560 con 0x7f62d8109950 2024-03-19T01:07:19.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.886+0000 7f62d6000640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f62c00a2800 0x7f62c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:19.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.886+0000 7f62d6000640 1 -- 172.21.15.69:0/1918024259 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f62d00ceea0 con 0x7f62d8109950 2024-03-19T01:07:19.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.886+0000 7f62dd400640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f62c00a2800 0x7f62c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:19.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.887+0000 7f62dd400640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f62c00a2800 0x7f62c00a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f62cc02c240 tx=0x7f62cc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:19.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:19.890+0000 7f62d6000640 1 -- 172.21.15.69:0/1918024259 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f62d0016030 con 0x7f62d8109950 2024-03-19T01:07:20.005 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:19 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b[97627]: 2024-03-19T01:07:19.765+0000 7ff2d2e65700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-19T01:07:20.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:19 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-19T01:07:20.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:19 smithi086 ceph-mon[66829]: osdmap e87: 12 total, 12 up, 12 in 2024-03-19T01:07:20.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:19 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3604415477' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-19T01:07:20.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:19 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-19T01:07:20.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:19 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-19T01:07:20.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:19 smithi069 ceph-mon[31727]: osdmap e87: 12 total, 12 up, 12 in 2024-03-19T01:07:20.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:19 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3604415477' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-19T01:07:20.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:19 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-19T01:07:20.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:19 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-19T01:07:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:19 smithi163 ceph-mon[67428]: osdmap e87: 12 total, 12 up, 12 in 2024-03-19T01:07:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:19 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3604415477' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-19T01:07:20.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:19 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-19T01:07:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.121+0000 7f62dfe00640 1 -- 172.21.15.69:0/1918024259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f62d8062420 con 0x7f62d8109950 2024-03-19T01:07:20.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.740+0000 7f62d6000640 1 -- 172.21.15.69:0/1918024259 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f62d009b670 con 0x7f62d8109950 2024-03-19T01:07:20.742 INFO:teuthology.orchestra.run.smithi069.stderr:added data pool 4 to fsmap 2024-03-19T01:07:20.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 -- 172.21.15.69:0/1918024259 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f62c00a2800 msgr2=0x7f62c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:20.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f62c00a2800 0x7f62c00a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f62cc02c240 tx=0x7f62cc02df90 comp rx=0 tx=0).stop 2024-03-19T01:07:20.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 -- 172.21.15.69:0/1918024259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8109950 msgr2=0x7f62d81a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:20.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8109950 0x7f62d81a56e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62d000eaa0 tx=0x7f62d000ef60 comp rx=0 tx=0).stop 2024-03-19T01:07:20.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 -- 172.21.15.69:0/1918024259 shutdown_connections 2024-03-19T01:07:20.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f62c00a2800 0x7f62c00a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:20.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8109950 0x7f62d81a56e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:20.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62d8108f90 0x7f62d819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:20.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 --2- 172.21.15.69:0/1918024259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d8107e40 0x7f62d819e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:20.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 -- 172.21.15.69:0/1918024259 >> 172.21.15.69:0/1918024259 conn(0x7f62d8076de0 msgr2=0x7f62d810c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:20.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 -- 172.21.15.69:0/1918024259 shutdown_connections 2024-03-19T01:07:20.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.745+0000 7f62dfe00640 1 -- 172.21.15.69:0/1918024259 wait complete. 2024-03-19T01:07:20.768 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:07:20.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.978+0000 7fb873e00640 1 -- 172.21.15.69:0/2879681754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86c107e60 msgr2=0x7fb86c108280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:20.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.978+0000 7fb873e00640 1 --2- 172.21.15.69:0/2879681754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86c107e60 0x7fb86c108280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb860009900 tx=0x7fb86002bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:20.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.978+0000 7fb873e00640 1 -- 172.21.15.69:0/2879681754 shutdown_connections 2024-03-19T01:07:20.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.978+0000 7fb873e00640 1 --2- 172.21.15.69:0/2879681754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb86c109970 0x7fb86c110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:20.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.978+0000 7fb873e00640 1 --2- 172.21.15.69:0/2879681754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c108fb0 0x7fb86c109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:20.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.978+0000 7fb873e00640 1 --2- 172.21.15.69:0/2879681754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86c107e60 0x7fb86c108280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:20.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.978+0000 7fb873e00640 1 -- 172.21.15.69:0/2879681754 >> 172.21.15.69:0/2879681754 conn(0x7fb86c076e30 msgr2=0x7fb86c079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:20.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.978+0000 7fb873e00640 1 -- 172.21.15.69:0/2879681754 shutdown_connections 2024-03-19T01:07:20.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.978+0000 7fb873e00640 1 -- 172.21.15.69:0/2879681754 wait complete. 2024-03-19T01:07:20.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.979+0000 7fb873e00640 1 Processor -- start 2024-03-19T01:07:20.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.979+0000 7fb873e00640 1 -- start start 2024-03-19T01:07:20.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.979+0000 7fb873e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb86c107e60 0x7fb86c19e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:20.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb873e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86c108fb0 0x7fb86c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:20.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb873e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c109970 0x7fb86c1a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:20.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb873e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c113b50 con 0x7fb86c107e60 2024-03-19T01:07:20.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb873e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1139d0 con 0x7fb86c108fb0 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb873e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c113cd0 con 0x7fb86c109970 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb871400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb86c107e60 0x7fb86c19e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb870a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86c108fb0 0x7fb86c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb870a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86c108fb0 0x7fb86c19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42680/0 (socket says 172.21.15.69:42680) 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb870a00640 1 -- 172.21.15.69:0/464390301 learned_addr learned my addr 172.21.15.69:0/464390301 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb871e00640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c109970 0x7fb86c1a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb870a00640 1 -- 172.21.15.69:0/464390301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c109970 msgr2=0x7fb86c1a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb870a00640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c109970 0x7fb86c1a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb870a00640 1 -- 172.21.15.69:0/464390301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb86c107e60 msgr2=0x7fb86c19e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb870a00640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb86c107e60 0x7fb86c19e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb870a00640 1 -- 172.21.15.69:0/464390301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c1a5d60 con 0x7fb86c108fb0 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb871e00640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c109970 0x7fb86c1a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.980+0000 7fb870a00640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86c108fb0 0x7fb86c19e680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9c0 tx=0x7fb85c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.981+0000 7fb86a000640 1 -- 172.21.15.69:0/464390301 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00cd80 con 0x7fb86c108fb0 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.981+0000 7fb86a000640 1 -- 172.21.15.69:0/464390301 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85c0035f0 con 0x7fb86c108fb0 2024-03-19T01:07:20.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.981+0000 7fb86a000640 1 -- 172.21.15.69:0/464390301 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c010690 con 0x7fb86c108fb0 2024-03-19T01:07:20.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.981+0000 7fb873e00640 1 -- 172.21.15.69:0/464390301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c1a5ff0 con 0x7fb86c108fb0 2024-03-19T01:07:20.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.981+0000 7fb873e00640 1 -- 172.21.15.69:0/464390301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c1a64c0 con 0x7fb86c108fb0 2024-03-19T01:07:20.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.982+0000 7fb873e00640 1 -- 172.21.15.69:0/464390301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86c10f170 con 0x7fb86c108fb0 2024-03-19T01:07:20.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.983+0000 7fb86a000640 1 -- 172.21.15.69:0/464390301 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb85c0037a0 con 0x7fb86c108fb0 2024-03-19T01:07:20.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.985+0000 7fb86a000640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8540a2880 0x7fb8540a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:20.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.985+0000 7fb86a000640 1 -- 172.21.15.69:0/464390301 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb85c0cf070 con 0x7fb86c108fb0 2024-03-19T01:07:20.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.985+0000 7fb871400640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8540a2880 0x7fb8540a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:20.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.986+0000 7fb871400640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8540a2880 0x7fb8540a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb86002c240 tx=0x7fb86002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:20.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:20.989+0000 7fb86a000640 1 -- 172.21.15.69:0/464390301 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb85c09be70 con 0x7fb86c108fb0 2024-03-19T01:07:21.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:20 smithi086 ceph-mon[66829]: pgmap v315: 129 pgs: 3 creating+peering, 97 active+clean, 29 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-03-19T01:07:21.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:20 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-19T01:07:21.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:20 smithi086 ceph-mon[66829]: osdmap e88: 12 total, 12 up, 12 in 2024-03-19T01:07:21.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:20 smithi086 ceph-mon[66829]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:07:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:20 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1918024259' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-19T01:07:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:20 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-19T01:07:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:20 smithi069 ceph-mon[31727]: pgmap v315: 129 pgs: 3 creating+peering, 97 active+clean, 29 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-03-19T01:07:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:20 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-19T01:07:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:20 smithi069 ceph-mon[31727]: osdmap e88: 12 total, 12 up, 12 in 2024-03-19T01:07:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:20 smithi069 ceph-mon[31727]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:07:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:20 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1918024259' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-19T01:07:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:20 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-19T01:07:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:20 smithi163 ceph-mon[67428]: pgmap v315: 129 pgs: 3 creating+peering, 97 active+clean, 29 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-03-19T01:07:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:20 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-19T01:07:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:20 smithi163 ceph-mon[67428]: osdmap e88: 12 total, 12 up, 12 in 2024-03-19T01:07:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:20 smithi163 ceph-mon[67428]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:07:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:20 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1918024259' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-19T01:07:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:20 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-19T01:07:21.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.227+0000 7fb873e00640 1 -- 172.21.15.69:0/464390301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb86c02d1f0 con 0x7fb86c108fb0 2024-03-19T01:07:21.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.228+0000 7fb86a000640 1 -- 172.21.15.69:0/464390301 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11794 (secure 0 0 0) 0x7fb85c09b770 con 0x7fb86c108fb0 2024-03-19T01:07:21.229 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:21.230 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24580,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:07:20.123583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-03-19T01:07:21.230 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 17 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.231+0000 7fb873e00640 1 -- 172.21.15.69:0/464390301 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8540a2880 msgr2=0x7fb8540a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.231+0000 7fb873e00640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8540a2880 0x7fb8540a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb86002c240 tx=0x7fb86002df90 comp rx=0 tx=0).stop 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.231+0000 7fb873e00640 1 -- 172.21.15.69:0/464390301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86c108fb0 msgr2=0x7fb86c19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.231+0000 7fb873e00640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86c108fb0 0x7fb86c19e680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9c0 tx=0x7fb85c00ee80 comp rx=0 tx=0).stop 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.231+0000 7fb873e00640 1 -- 172.21.15.69:0/464390301 shutdown_connections 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.231+0000 7fb873e00640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8540a2880 0x7fb8540a4d40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.231+0000 7fb873e00640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c109970 0x7fb86c1a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.231+0000 7fb873e00640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86c108fb0 0x7fb86c19e680 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.231+0000 7fb873e00640 1 --2- 172.21.15.69:0/464390301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb86c107e60 0x7fb86c19e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.231+0000 7fb873e00640 1 -- 172.21.15.69:0/464390301 >> 172.21.15.69:0/464390301 conn(0x7fb86c076e30 msgr2=0x7fb86c10c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.232+0000 7fb873e00640 1 -- 172.21.15.69:0/464390301 shutdown_connections 2024-03-19T01:07:21.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.232+0000 7fb873e00640 1 -- 172.21.15.69:0/464390301 wait complete. 2024-03-19T01:07:21.252 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:07:21.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.465+0000 7f0b23c00640 1 -- 172.21.15.69:0/3878874816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b1c074220 msgr2=0x7f0b1c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:21.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.465+0000 7f0b23c00640 1 --2- 172.21.15.69:0/3878874816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b1c074220 0x7f0b1c074680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009900 tx=0x7f0b0402bcd0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.466+0000 7f0b23c00640 1 -- 172.21.15.69:0/3878874816 shutdown_connections 2024-03-19T01:07:21.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.466+0000 7f0b23c00640 1 --2- 172.21.15.69:0/3878874816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b1c1096d0 0x7f0b1c10bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.466+0000 7f0b23c00640 1 --2- 172.21.15.69:0/3878874816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b1c074220 0x7f0b1c074680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.466+0000 7f0b23c00640 1 --2- 172.21.15.69:0/3878874816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b1c075800 0x7f0b1c073c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.466+0000 7f0b23c00640 1 -- 172.21.15.69:0/3878874816 >> 172.21.15.69:0/3878874816 conn(0x7f0b1c0fcdd0 msgr2=0x7f0b1c0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:21.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.466+0000 7f0b23c00640 1 -- 172.21.15.69:0/3878874816 shutdown_connections 2024-03-19T01:07:21.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.466+0000 7f0b23c00640 1 -- 172.21.15.69:0/3878874816 wait complete. 2024-03-19T01:07:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b23c00640 1 Processor -- start 2024-03-19T01:07:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b23c00640 1 -- start start 2024-03-19T01:07:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b23c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b1c074220 0x7f0b1c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b23c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b1c075800 0x7f0b1c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b23c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b1c1096d0 0x7f0b1c1a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b23c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c10f6c0 con 0x7f0b1c074220 2024-03-19T01:07:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b23c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c10f540 con 0x7f0b1c075800 2024-03-19T01:07:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b23c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c10f840 con 0x7f0b1c1096d0 2024-03-19T01:07:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b1c075800 0x7f0b1c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b21200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b1c074220 0x7f0b1c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b1c075800 0x7f0b1c19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42686/0 (socket says 172.21.15.69:42686) 2024-03-19T01:07:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b13e00640 1 -- 172.21.15.69:0/665809337 learned_addr learned my addr 172.21.15.69:0/665809337 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.467+0000 7f0b21c00640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b1c1096d0 0x7f0b1c1a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.468+0000 7f0b13e00640 1 -- 172.21.15.69:0/665809337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b1c1096d0 msgr2=0x7f0b1c1a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.468+0000 7f0b13e00640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b1c1096d0 0x7f0b1c1a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.468+0000 7f0b13e00640 1 -- 172.21.15.69:0/665809337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b1c074220 msgr2=0x7f0b1c199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.468+0000 7f0b13e00640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b1c074220 0x7f0b1c199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.468+0000 7f0b13e00640 1 -- 172.21.15.69:0/665809337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1c1a1950 con 0x7f0b1c075800 2024-03-19T01:07:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.468+0000 7f0b13e00640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b1c075800 0x7f0b1c19a280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005ab0 tx=0x7f0b04005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.468+0000 7f0b1a000640 1 -- 172.21.15.69:0/665809337 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04041070 con 0x7f0b1c075800 2024-03-19T01:07:21.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.468+0000 7f0b1a000640 1 -- 172.21.15.69:0/665809337 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b040054e0 con 0x7f0b1c075800 2024-03-19T01:07:21.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.468+0000 7f0b23c00640 1 -- 172.21.15.69:0/665809337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1c1a1be0 con 0x7f0b1c075800 2024-03-19T01:07:21.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.468+0000 7f0b1a000640 1 -- 172.21.15.69:0/665809337 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04034440 con 0x7f0b1c075800 2024-03-19T01:07:21.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.468+0000 7f0b23c00640 1 -- 172.21.15.69:0/665809337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1c1a2120 con 0x7f0b1c075800 2024-03-19T01:07:21.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.469+0000 7f0b23c00640 1 -- 172.21.15.69:0/665809337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b1c108490 con 0x7f0b1c075800 2024-03-19T01:07:21.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.471+0000 7f0b1a000640 1 -- 172.21.15.69:0/665809337 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0b04044070 con 0x7f0b1c075800 2024-03-19T01:07:21.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.473+0000 7f0b1a000640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0b000a2880 0x7f0b000a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:21.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.473+0000 7f0b1a000640 1 -- 172.21.15.69:0/665809337 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f0b040f0b70 con 0x7f0b1c075800 2024-03-19T01:07:21.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.473+0000 7f0b21200640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0b000a2880 0x7f0b000a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:21.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.474+0000 7f0b21200640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0b000a2880 0x7f0b000a4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c19b210 tx=0x7f0b0c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:21.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.481+0000 7f0b1a000640 1 -- 172.21.15.69:0/665809337 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0b040bd380 con 0x7f0b1c075800 2024-03-19T01:07:21.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.682+0000 7f0b23c00640 1 -- 172.21.15.69:0/665809337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b1c0624c0 con 0x7f0b1c075800 2024-03-19T01:07:21.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.684+0000 7f0b1a000640 1 -- 172.21.15.69:0/665809337 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24069 (secure 0 0 0) 0x7f0b040c1420 con 0x7f0b1c075800 2024-03-19T01:07:21.686 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:21.686 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":89,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:07:20.123357+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:07:21.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.687+0000 7f0b23c00640 1 -- 172.21.15.69:0/665809337 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0b000a2880 msgr2=0x7f0b000a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:21.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.687+0000 7f0b23c00640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0b000a2880 0x7f0b000a4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c19b210 tx=0x7f0b0c00b390 comp rx=0 tx=0).stop 2024-03-19T01:07:21.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.687+0000 7f0b23c00640 1 -- 172.21.15.69:0/665809337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b1c075800 msgr2=0x7f0b1c19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:21.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.687+0000 7f0b23c00640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b1c075800 0x7f0b1c19a280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005ab0 tx=0x7f0b04005de0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.688+0000 7f0b23c00640 1 -- 172.21.15.69:0/665809337 shutdown_connections 2024-03-19T01:07:21.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.688+0000 7f0b23c00640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0b000a2880 0x7f0b000a4d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.688+0000 7f0b23c00640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b1c1096d0 0x7f0b1c1a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.688+0000 7f0b23c00640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b1c075800 0x7f0b1c19a280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.688+0000 7f0b23c00640 1 --2- 172.21.15.69:0/665809337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b1c074220 0x7f0b1c199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.688+0000 7f0b23c00640 1 -- 172.21.15.69:0/665809337 >> 172.21.15.69:0/665809337 conn(0x7f0b1c0fcdd0 msgr2=0x7f0b1c105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:21.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.688+0000 7f0b23c00640 1 -- 172.21.15.69:0/665809337 shutdown_connections 2024-03-19T01:07:21.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.688+0000 7f0b23c00640 1 -- 172.21.15.69:0/665809337 wait complete. 2024-03-19T01:07:21.709 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:07:21.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:21 smithi163 ceph-mon[67428]: osdmap e89: 12 total, 12 up, 12 in 2024-03-19T01:07:21.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:21 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-19T01:07:21.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:21 smithi163 ceph-mon[67428]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:21.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:21 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/464390301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:21.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.908+0000 7f8509e00640 1 -- 172.21.15.69:0/3313438447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8504104cb0 msgr2=0x7f85041070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:21.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.908+0000 7f8509e00640 1 --2- 172.21.15.69:0/3313438447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8504104cb0 0x7f85041070a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009900 tx=0x7f84f002bc40 comp rx=0 tx=0).stop 2024-03-19T01:07:21.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.908+0000 7f8509e00640 1 -- 172.21.15.69:0/3313438447 shutdown_connections 2024-03-19T01:07:21.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.908+0000 7f8509e00640 1 --2- 172.21.15.69:0/3313438447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85041075e0 0x7f85041099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.908+0000 7f8509e00640 1 --2- 172.21.15.69:0/3313438447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8504104cb0 0x7f85041070a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.908+0000 7f8509e00640 1 --2- 172.21.15.69:0/3313438447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8504102380 0x7f8504104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.908+0000 7f8509e00640 1 -- 172.21.15.69:0/3313438447 >> 172.21.15.69:0/3313438447 conn(0x7f85040fbcb0 msgr2=0x7f85040fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:21.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.908+0000 7f8509e00640 1 -- 172.21.15.69:0/3313438447 shutdown_connections 2024-03-19T01:07:21.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.908+0000 7f8509e00640 1 -- 172.21.15.69:0/3313438447 wait complete. 2024-03-19T01:07:21.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.909+0000 7f8509e00640 1 Processor -- start 2024-03-19T01:07:21.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.909+0000 7f8509e00640 1 -- start start 2024-03-19T01:07:21.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.909+0000 7f8509e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8504102380 0x7f8504197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:21.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.909+0000 7f8509e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8504104cb0 0x7f85041980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:21.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.909+0000 7f8509e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85041075e0 0x7f850419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:21.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.909+0000 7f8509e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f850410d520 con 0x7f8504104cb0 2024-03-19T01:07:21.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.909+0000 7f8509e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f850410d3a0 con 0x7f8504102380 2024-03-19T01:07:21.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.909+0000 7f8509e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f850410d6a0 con 0x7f85041075e0 2024-03-19T01:07:21.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.909+0000 7f8502a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8504102380 0x7f8504197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:21.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.909+0000 7f8502a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8504102380 0x7f8504197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42698/0 (socket says 172.21.15.69:42698) 2024-03-19T01:07:21.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.909+0000 7f8502a00640 1 -- 172.21.15.69:0/2259216293 learned_addr learned my addr 172.21.15.69:0/2259216293 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:21.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8502000640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8504104cb0 0x7f85041980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:21.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8503400640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85041075e0 0x7f850419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:21.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8502a00640 1 -- 172.21.15.69:0/2259216293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85041075e0 msgr2=0x7f850419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:21.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8502a00640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85041075e0 0x7f850419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8502a00640 1 -- 172.21.15.69:0/2259216293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8504104cb0 msgr2=0x7f85041980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:21.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8502a00640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8504104cb0 0x7f85041980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:21.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8502a00640 1 -- 172.21.15.69:0/2259216293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850419f7b0 con 0x7f8504102380 2024-03-19T01:07:21.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8502a00640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8504102380 0x7f8504197b90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f84f400ebc0 tx=0x7f84f400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:21.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8508a00640 1 -- 172.21.15.69:0/2259216293 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4010630 con 0x7f8504102380 2024-03-19T01:07:21.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8508a00640 1 -- 172.21.15.69:0/2259216293 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f4010cb0 con 0x7f8504102380 2024-03-19T01:07:21.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8508a00640 1 -- 172.21.15.69:0/2259216293 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4018bd0 con 0x7f8504102380 2024-03-19T01:07:21.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8509e00640 1 -- 172.21.15.69:0/2259216293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850419faa0 con 0x7f8504102380 2024-03-19T01:07:21.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.910+0000 7f8509e00640 1 -- 172.21.15.69:0/2259216293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850419ff70 con 0x7f8504102380 2024-03-19T01:07:21.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.911+0000 7f8509e00640 1 -- 172.21.15.69:0/2259216293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85040fd7a0 con 0x7f8504102380 2024-03-19T01:07:21.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.913+0000 7f8508a00640 1 -- 172.21.15.69:0/2259216293 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f84f40107d0 con 0x7f8504102380 2024-03-19T01:07:21.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.914+0000 7f8508a00640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84e80a2880 0x7f84e80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:21.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.914+0000 7f8508a00640 1 -- 172.21.15.69:0/2259216293 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f84f40cf3c0 con 0x7f8504102380 2024-03-19T01:07:21.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.915+0000 7f8502000640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84e80a2880 0x7f84e80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:21.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.915+0000 7f8502000640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84e80a2880 0x7f84e80a4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f84f002c180 tx=0x7f84f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:21.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:21.918+0000 7f8508a00640 1 -- 172.21.15.69:0/2259216293 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f84f409c1c0 con 0x7f8504102380 2024-03-19T01:07:22.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:21 smithi086 ceph-mon[66829]: osdmap e89: 12 total, 12 up, 12 in 2024-03-19T01:07:22.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:21 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-19T01:07:22.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:21 smithi086 ceph-mon[66829]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:22.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:21 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/464390301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:21 smithi069 ceph-mon[31727]: osdmap e89: 12 total, 12 up, 12 in 2024-03-19T01:07:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:21 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-19T01:07:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:21 smithi069 ceph-mon[31727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:21 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/464390301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:22.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.119+0000 7f8509e00640 1 -- 172.21.15.69:0/2259216293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8504062420 con 0x7f8504102380 2024-03-19T01:07:22.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.121+0000 7f8508a00640 1 -- 172.21.15.69:0/2259216293 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24069 (secure 0 0 0) 0x7f84f409bac0 con 0x7f8504102380 2024-03-19T01:07:22.122 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:22.123 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":89,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:07:20.123357+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:07:22.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 -- 172.21.15.69:0/2259216293 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84e80a2880 msgr2=0x7f84e80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:22.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84e80a2880 0x7f84e80a4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f84f002c180 tx=0x7f84f0000bc0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 -- 172.21.15.69:0/2259216293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8504102380 msgr2=0x7f8504197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:22.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8504102380 0x7f8504197b90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f84f400ebc0 tx=0x7f84f400c6a0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 -- 172.21.15.69:0/2259216293 shutdown_connections 2024-03-19T01:07:22.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84e80a2880 0x7f84e80a4d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85041075e0 0x7f850419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8504104cb0 0x7f85041980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 --2- 172.21.15.69:0/2259216293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8504102380 0x7f8504197b90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 -- 172.21.15.69:0/2259216293 >> 172.21.15.69:0/2259216293 conn(0x7f85040fbcb0 msgr2=0x7f8504102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:22.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 -- 172.21.15.69:0/2259216293 shutdown_connections 2024-03-19T01:07:22.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.123+0000 7f8509e00640 1 -- 172.21.15.69:0/2259216293 wait complete. 2024-03-19T01:07:22.162 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:07:22.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.376+0000 7fc1e3e00640 1 -- 172.21.15.69:0/1129779150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1dc104c50 msgr2=0x7fc1dc107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:22.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.376+0000 7fc1e3e00640 1 --2- 172.21.15.69:0/1129779150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1dc104c50 0x7fc1dc107040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009900 tx=0x7fc1c802bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:22.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.376+0000 7fc1e3e00640 1 -- 172.21.15.69:0/1129779150 shutdown_connections 2024-03-19T01:07:22.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.376+0000 7fc1e3e00640 1 --2- 172.21.15.69:0/1129779150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc107580 0x7fc1dc109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.376+0000 7fc1e3e00640 1 --2- 172.21.15.69:0/1129779150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1dc104c50 0x7fc1dc107040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.376+0000 7fc1e3e00640 1 --2- 172.21.15.69:0/1129779150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1dc102320 0x7fc1dc104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.376+0000 7fc1e3e00640 1 -- 172.21.15.69:0/1129779150 >> 172.21.15.69:0/1129779150 conn(0x7fc1dc0fbcb0 msgr2=0x7fc1dc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:22.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.376+0000 7fc1e3e00640 1 -- 172.21.15.69:0/1129779150 shutdown_connections 2024-03-19T01:07:22.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.376+0000 7fc1e3e00640 1 -- 172.21.15.69:0/1129779150 wait complete. 2024-03-19T01:07:22.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.377+0000 7fc1e3e00640 1 Processor -- start 2024-03-19T01:07:22.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.377+0000 7fc1e3e00640 1 -- start start 2024-03-19T01:07:22.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc102320 0x7fc1dc197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:22.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1dc104c50 0x7fc1dc1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:22.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1dc107580 0x7fc1dc19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:22.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e3e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc10d4b0 con 0x7fc1dc102320 2024-03-19T01:07:22.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc10d330 con 0x7fc1dc107580 2024-03-19T01:07:22.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e3e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc10d630 con 0x7fc1dc104c50 2024-03-19T01:07:22.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc102320 0x7fc1dc197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e0a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1dc104c50 0x7fc1dc1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1dc107580 0x7fc1dc19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e0a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1dc104c50 0x7fc1dc1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:39352/0 (socket says 172.21.15.69:39352) 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e0a00640 1 -- 172.21.15.69:0/2434752827 learned_addr learned my addr 172.21.15.69:0/2434752827 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e1e00640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1dc107580 0x7fc1dc19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42720/0 (socket says 172.21.15.69:42720) 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e1e00640 1 -- 172.21.15.69:0/2434752827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1dc104c50 msgr2=0x7fc1dc1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e1e00640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1dc104c50 0x7fc1dc1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e1e00640 1 -- 172.21.15.69:0/2434752827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc102320 msgr2=0x7fc1dc197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.378+0000 7fc1e1e00640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc102320 0x7fc1dc197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.379+0000 7fc1e1e00640 1 -- 172.21.15.69:0/2434752827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc19f7a0 con 0x7fc1dc107580 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.379+0000 7fc1e1e00640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1dc107580 0x7fc1dc19f130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009fd0 tx=0x7fc1d400b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.379+0000 7fc1da000640 1 -- 172.21.15.69:0/2434752827 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4004b20 con 0x7fc1dc107580 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.379+0000 7fc1da000640 1 -- 172.21.15.69:0/2434752827 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1d4004cc0 con 0x7fc1dc107580 2024-03-19T01:07:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.379+0000 7fc1da000640 1 -- 172.21.15.69:0/2434752827 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4005520 con 0x7fc1dc107580 2024-03-19T01:07:22.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.379+0000 7fc1e3e00640 1 -- 172.21.15.69:0/2434752827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1dc100170 con 0x7fc1dc107580 2024-03-19T01:07:22.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.379+0000 7fc1e3e00640 1 -- 172.21.15.69:0/2434752827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1dc100640 con 0x7fc1dc107580 2024-03-19T01:07:22.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.380+0000 7fc1e3e00640 1 -- 172.21.15.69:0/2434752827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1dc0fd7a0 con 0x7fc1dc107580 2024-03-19T01:07:22.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.382+0000 7fc1da000640 1 -- 172.21.15.69:0/2434752827 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc1d4005910 con 0x7fc1dc107580 2024-03-19T01:07:22.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.383+0000 7fc1da000640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1c40a2800 0x7fc1c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:22.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.384+0000 7fc1da000640 1 -- 172.21.15.69:0/2434752827 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc1d40d2d80 con 0x7fc1dc107580 2024-03-19T01:07:22.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.384+0000 7fc1e1400640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1c40a2800 0x7fc1c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:22.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.384+0000 7fc1e1400640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1c40a2800 0x7fc1c40a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009bd0 tx=0x7fc1d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:22.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.392+0000 7fc1da000640 1 -- 172.21.15.69:0/2434752827 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc1d409f530 con 0x7fc1dc107580 2024-03-19T01:07:22.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.626+0000 7fc1e3e00640 1 -- 172.21.15.69:0/2434752827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1dc101a20 con 0x7fc1dc107580 2024-03-19T01:07:22.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.627+0000 7fc1da000640 1 -- 172.21.15.69:0/2434752827 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11794 (secure 0 0 0) 0x7fc1d40a35d0 con 0x7fc1dc107580 2024-03-19T01:07:22.628 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:22.628 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24580,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:07:20.123583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-03-19T01:07:22.629 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 17 2024-03-19T01:07:22.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 -- 172.21.15.69:0/2434752827 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1c40a2800 msgr2=0x7fc1c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:22.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1c40a2800 0x7fc1c40a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009bd0 tx=0x7fc1d0005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:22.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 -- 172.21.15.69:0/2434752827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1dc107580 msgr2=0x7fc1dc19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:22.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1dc107580 0x7fc1dc19f130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009fd0 tx=0x7fc1d400b690 comp rx=0 tx=0).stop 2024-03-19T01:07:22.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 -- 172.21.15.69:0/2434752827 shutdown_connections 2024-03-19T01:07:22.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1c40a2800 0x7fc1c40a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1dc107580 0x7fc1dc19f130 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1dc104c50 0x7fc1dc1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 --2- 172.21.15.69:0/2434752827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc102320 0x7fc1dc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 -- 172.21.15.69:0/2434752827 >> 172.21.15.69:0/2434752827 conn(0x7fc1dc0fbcb0 msgr2=0x7fc1dc102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:22.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 -- 172.21.15.69:0/2434752827 shutdown_connections 2024-03-19T01:07:22.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.630+0000 7fc1e3e00640 1 -- 172.21.15.69:0/2434752827 wait complete. 2024-03-19T01:07:22.648 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:07:22.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.853+0000 7f97eec00640 1 -- 172.21.15.69:0/3720957833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8104a60 msgr2=0x7f97e8104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:22.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.853+0000 7f97eec00640 1 --2- 172.21.15.69:0/3720957833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8104a60 0x7f97e8104ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009900 tx=0x7f97d402bc40 comp rx=0 tx=0).stop 2024-03-19T01:07:22.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.853+0000 7f97eec00640 1 -- 172.21.15.69:0/3720957833 shutdown_connections 2024-03-19T01:07:22.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.853+0000 7f97eec00640 1 --2- 172.21.15.69:0/3720957833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97e8105420 0x7f97e810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.853+0000 7f97eec00640 1 --2- 172.21.15.69:0/3720957833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8104a60 0x7f97e8104ee0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.853+0000 7f97eec00640 1 --2- 172.21.15.69:0/3720957833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e8103870 0x7f97e8103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.853+0000 7f97eec00640 1 -- 172.21.15.69:0/3720957833 >> 172.21.15.69:0/3720957833 conn(0x7f97e80fee00 msgr2=0x7f97e8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:22.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.853+0000 7f97eec00640 1 -- 172.21.15.69:0/3720957833 shutdown_connections 2024-03-19T01:07:22.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.853+0000 7f97eec00640 1 -- 172.21.15.69:0/3720957833 wait complete. 2024-03-19T01:07:22.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.854+0000 7f97eec00640 1 Processor -- start 2024-03-19T01:07:22.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.854+0000 7f97eec00640 1 -- start start 2024-03-19T01:07:22.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.854+0000 7f97eec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e8103870 0x7f97e8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.854+0000 7f97eec00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8104a60 0x7f97e819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.854+0000 7f97eec00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97e8105420 0x7f97e81a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.854+0000 7f97eec00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97e810f540 con 0x7f97e8104a60 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.854+0000 7f97eec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97e810f3c0 con 0x7f97e8103870 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.854+0000 7f97eec00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f97e810f6c0 con 0x7f97e8105420 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97ed800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e8103870 0x7f97e8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97ed800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e8103870 0x7f97e8199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42734/0 (socket says 172.21.15.69:42734) 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97ed800640 1 -- 172.21.15.69:0/2600245743 learned_addr learned my addr 172.21.15.69:0/2600245743 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97ece00640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8104a60 0x7f97e819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97ee200640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97e8105420 0x7f97e81a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97ed800640 1 -- 172.21.15.69:0/2600245743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97e8105420 msgr2=0x7f97e81a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97ed800640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97e8105420 0x7f97e81a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97ed800640 1 -- 172.21.15.69:0/2600245743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8104a60 msgr2=0x7f97e819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:22.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97ed800640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8104a60 0x7f97e819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:22.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97ed800640 1 -- 172.21.15.69:0/2600245743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e81a1750 con 0x7f97e8103870 2024-03-19T01:07:22.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97ed800640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e8103870 0x7f97e8199b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00e9c0 tx=0x7f97dc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:22.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97e6000640 1 -- 172.21.15.69:0/2600245743 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00cde0 con 0x7f97e8103870 2024-03-19T01:07:22.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.855+0000 7f97e6000640 1 -- 172.21.15.69:0/2600245743 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97dc010910 con 0x7f97e8103870 2024-03-19T01:07:22.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.856+0000 7f97eec00640 1 -- 172.21.15.69:0/2600245743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e81a1a40 con 0x7f97e8103870 2024-03-19T01:07:22.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.856+0000 7f97e6000640 1 -- 172.21.15.69:0/2600245743 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc018a00 con 0x7f97e8103870 2024-03-19T01:07:22.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.856+0000 7f97eec00640 1 -- 172.21.15.69:0/2600245743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e81a1f60 con 0x7f97e8103870 2024-03-19T01:07:22.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.857+0000 7f97eec00640 1 -- 172.21.15.69:0/2600245743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e810ab30 con 0x7f97e8103870 2024-03-19T01:07:22.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.858+0000 7f97e6000640 1 -- 172.21.15.69:0/2600245743 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f97dc01f020 con 0x7f97e8103870 2024-03-19T01:07:22.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.859+0000 7f97e6000640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f97d00a2800 0x7f97d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.860+0000 7f97e6000640 1 -- 172.21.15.69:0/2600245743 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f97dc0cef90 con 0x7f97e8103870 2024-03-19T01:07:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.860+0000 7f97ece00640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f97d00a2800 0x7f97d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.860+0000 7f97ece00640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f97d00a2800 0x7f97d00a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f97d402c250 tx=0x7f97d402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:22.863+0000 7f97e6000640 1 -- 172.21.15.69:0/2600245743 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f97dc09bd90 con 0x7f97e8103870 2024-03-19T01:07:23.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:22 smithi086 ceph-mon[66829]: pgmap v318: 129 pgs: 3 creating+peering, 105 active+clean, 21 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-03-19T01:07:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:22 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/665809337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:22 smithi086 ceph-mon[66829]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:07:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:22 smithi086 ceph-mon[66829]: Cluster is now healthy 2024-03-19T01:07:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:22 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2259216293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:22 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2434752827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:22 smithi069 ceph-mon[31727]: pgmap v318: 129 pgs: 3 creating+peering, 105 active+clean, 21 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-03-19T01:07:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:22 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/665809337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:22 smithi069 ceph-mon[31727]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:07:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:22 smithi069 ceph-mon[31727]: Cluster is now healthy 2024-03-19T01:07:23.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:22 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2259216293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:23.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:22 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2434752827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:22 smithi163 ceph-mon[67428]: pgmap v318: 129 pgs: 3 creating+peering, 105 active+clean, 21 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-03-19T01:07:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:22 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/665809337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:22 smithi163 ceph-mon[67428]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:07:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:22 smithi163 ceph-mon[67428]: Cluster is now healthy 2024-03-19T01:07:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:22 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2259216293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:22 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2434752827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:23.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.062+0000 7f97eec00640 1 -- 172.21.15.69:0/2600245743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97e802d1f0 con 0x7f97e8103870 2024-03-19T01:07:23.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.064+0000 7f97e6000640 1 -- 172.21.15.69:0/2600245743 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24069 (secure 0 0 0) 0x7f97dc09b690 con 0x7f97e8103870 2024-03-19T01:07:23.066 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:23.066 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":89,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:07:20.123357+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.067+0000 7f97eec00640 1 -- 172.21.15.69:0/2600245743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f97d00a2800 msgr2=0x7f97d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.067+0000 7f97eec00640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f97d00a2800 0x7f97d00a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f97d402c250 tx=0x7f97d402cf90 comp rx=0 tx=0).stop 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.067+0000 7f97eec00640 1 -- 172.21.15.69:0/2600245743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e8103870 msgr2=0x7f97e8199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.067+0000 7f97eec00640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e8103870 0x7f97e8199b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00e9c0 tx=0x7f97dc00ee80 comp rx=0 tx=0).stop 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.067+0000 7f97eec00640 1 -- 172.21.15.69:0/2600245743 shutdown_connections 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.067+0000 7f97eec00640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f97d00a2800 0x7f97d00a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.067+0000 7f97eec00640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97e8105420 0x7f97e81a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.067+0000 7f97eec00640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8104a60 0x7f97e819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.067+0000 7f97eec00640 1 --2- 172.21.15.69:0/2600245743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e8103870 0x7f97e8199b30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.067+0000 7f97eec00640 1 -- 172.21.15.69:0/2600245743 >> 172.21.15.69:0/2600245743 conn(0x7f97e80fee00 msgr2=0x7f97e8107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.068+0000 7f97eec00640 1 -- 172.21.15.69:0/2600245743 shutdown_connections 2024-03-19T01:07:23.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.068+0000 7f97eec00640 1 -- 172.21.15.69:0/2600245743 wait complete. 2024-03-19T01:07:23.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-19T01:07:14.928258+0000', 'modified': '2024-03-19T01:07:20.123583+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6839/621579864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 621579864}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 621579864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24581, 'qdb_cluster': [24581]} 2024-03-19T01:07:23.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-19T01:07:23.088 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-19T01:07:23.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.295+0000 7fef73200640 1 -- 172.21.15.69:0/2050856206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c104c30 msgr2=0x7fef6c1050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:23.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.295+0000 7fef73200640 1 --2- 172.21.15.69:0/2050856206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c104c30 0x7fef6c1050b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fef54009900 tx=0x7fef5402bc40 comp rx=0 tx=0).stop 2024-03-19T01:07:23.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.296+0000 7fef73200640 1 -- 172.21.15.69:0/2050856206 shutdown_connections 2024-03-19T01:07:23.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.296+0000 7fef73200640 1 --2- 172.21.15.69:0/2050856206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6c1055f0 0x7fef6c10be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.296+0000 7fef73200640 1 --2- 172.21.15.69:0/2050856206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c104c30 0x7fef6c1050b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.296+0000 7fef73200640 1 --2- 172.21.15.69:0/2050856206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef6c103a40 0x7fef6c103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.296+0000 7fef73200640 1 -- 172.21.15.69:0/2050856206 >> 172.21.15.69:0/2050856206 conn(0x7fef6c0fefd0 msgr2=0x7fef6c1013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:23.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.296+0000 7fef73200640 1 -- 172.21.15.69:0/2050856206 shutdown_connections 2024-03-19T01:07:23.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.296+0000 7fef73200640 1 -- 172.21.15.69:0/2050856206 wait complete. 2024-03-19T01:07:23.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.297+0000 7fef73200640 1 Processor -- start 2024-03-19T01:07:23.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.297+0000 7fef73200640 1 -- start start 2024-03-19T01:07:23.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef73200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef6c103a40 0x7fef6c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef73200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6c104c30 0x7fef6c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef73200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c1055f0 0x7fef6c1a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef73200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c10f710 con 0x7fef6c103a40 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef73200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c10f590 con 0x7fef6c1055f0 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef73200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c10f890 con 0x7fef6c104c30 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef6be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef6c103a40 0x7fef6c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef6be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef6c103a40 0x7fef6c199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56106/0 (socket says 172.21.15.69:56106) 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef6be00640 1 -- 172.21.15.69:0/1571612135 learned_addr learned my addr 172.21.15.69:0/1571612135 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef6b400640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6c104c30 0x7fef6c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef71200640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c1055f0 0x7fef6c1a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef6be00640 1 -- 172.21.15.69:0/1571612135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6c104c30 msgr2=0x7fef6c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef6be00640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6c104c30 0x7fef6c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef6be00640 1 -- 172.21.15.69:0/1571612135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c1055f0 msgr2=0x7fef6c1a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef6be00640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c1055f0 0x7fef6c1a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef6be00640 1 -- 172.21.15.69:0/1571612135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c1a1960 con 0x7fef6c103a40 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.298+0000 7fef71200640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c1055f0 0x7fef6c1a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:23.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.299+0000 7fef6be00640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef6c103a40 0x7fef6c199d50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00e9c0 tx=0x7fef5c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:23.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.299+0000 7fef68c00640 1 -- 172.21.15.69:0/1571612135 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00cd80 con 0x7fef6c103a40 2024-03-19T01:07:23.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.299+0000 7fef68c00640 1 -- 172.21.15.69:0/1571612135 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef5c0035f0 con 0x7fef6c103a40 2024-03-19T01:07:23.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.299+0000 7fef73200640 1 -- 172.21.15.69:0/1571612135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c1a1c50 con 0x7fef6c103a40 2024-03-19T01:07:23.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.299+0000 7fef68c00640 1 -- 172.21.15.69:0/1571612135 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c010690 con 0x7fef6c103a40 2024-03-19T01:07:23.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.299+0000 7fef73200640 1 -- 172.21.15.69:0/1571612135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c1a21a0 con 0x7fef6c103a40 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.302+0000 7fef75f63740 1 Processor -- start 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.302+0000 7fef75f63740 1 -- start start 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.302+0000 7fef75f63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 0x560e67030810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.302+0000 7fef75f63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e670358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.302+0000 7fef75f63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e67035f50 0x560e670383e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.302+0000 7fef68c00640 1 -- 172.21.15.69:0/1571612135 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fef5c0037a0 con 0x7fef6c103a40 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.302+0000 7fef75f63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560e66f94d00 con 0x560e670303f0 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.302+0000 7fef75f63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560e66f3db00 con 0x560e67030de0 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.302+0000 7fef75f63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560e66f90da0 con 0x560e67035f50 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.303+0000 7fef6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e670358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.303+0000 7fef71200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 0x560e67030810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.303+0000 7fef6b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e67035f50 0x560e670383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.303+0000 7fef6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e670358e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42780/0 (socket says 172.21.15.69:42780) 2024-03-19T01:07:23.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.303+0000 7fef6be00640 1 -- 172.21.15.69:0/2831375976 learned_addr learned my addr 172.21.15.69:0/2831375976 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:23.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.303+0000 7fef6be00640 1 -- 172.21.15.69:0/2831375976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e67035f50 msgr2=0x560e670383e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:07:23.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.303+0000 7fef6be00640 1 --2- 172.21.15.69:0/2831375976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e67035f50 0x560e670383e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.303+0000 7fef6be00640 1 -- 172.21.15.69:0/2831375976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 msgr2=0x560e67030810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:07:23.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.303+0000 7fef6be00640 1 --2- 172.21.15.69:0/2831375976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 0x560e67030810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.303+0000 7fef6be00640 1 -- 172.21.15.69:0/2831375976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e66ca92d0 con 0x560e67030de0 2024-03-19T01:07:23.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.303+0000 7fef6be00640 1 --2- 172.21.15.69:0/2831375976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e670358e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0d7110 tx=0x7fef5c0d75d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80e8b8ffd9f28820 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:23.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.304+0000 7fef61600640 1 -- 172.21.15.69:0/2831375976 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0e2070 con 0x560e67030de0 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.304+0000 7fef61600640 1 -- 172.21.15.69:0/2831375976 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef5c0d12e0 con 0x560e67030de0 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.304+0000 7fef61600640 1 -- 172.21.15.69:0/2831375976 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0dd8d0 con 0x560e67030de0 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.304+0000 7fef75f63740 1 -- 172.21.15.69:0/2831375976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 msgr2=0x560e670358e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.304+0000 7fef75f63740 1 --2- 172.21.15.69:0/2831375976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e670358e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0d7110 tx=0x7fef5c0d75d0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.304+0000 7fef68c00640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fef500a2800 0x7fef500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.304+0000 7fef75f63740 1 -- 172.21.15.69:0/2831375976 shutdown_connections 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.304+0000 7fef75f63740 1 --2- 172.21.15.69:0/2831375976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e67035f50 0x560e670383e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.304+0000 7fef75f63740 1 --2- 172.21.15.69:0/2831375976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e670358e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.304+0000 7fef75f63740 1 --2- 172.21.15.69:0/2831375976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 0x560e67030810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.304+0000 7fef75f63740 1 -- 172.21.15.69:0/2831375976 >> 172.21.15.69:0/2831375976 conn(0x560e6702b960 msgr2=0x560e6702bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.305+0000 7fef71200640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fef500a2800 0x7fef500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.305+0000 7fef68c00640 1 -- 172.21.15.69:0/1571612135 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fef5c0cf120 con 0x7fef6c103a40 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.305+0000 7fef75f63740 1 -- 172.21.15.69:0/2831375976 shutdown_connections 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.305+0000 7fef75f63740 1 -- 172.21.15.69:0/2831375976 wait complete. 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.305+0000 7fef75f63740 1 -- start start 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.305+0000 7fef71200640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fef500a2800 0x7fef500a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x560e67031640 tx=0x7fef64009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.305+0000 7fef75f63740 1 Processor -- start 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef75f63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 0x560e6703a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef75f63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e6703ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef75f63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67035f50 0x560e67041e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef75f63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560e67038f00 con 0x560e67035f50 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef75f63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560e66f50ed0 con 0x560e67030de0 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef75f63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560e67039080 con 0x560e670303f0 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef75f63740 10 client.0 _ll_register_callbacks cb 0x560e66f4ec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef71200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67035f50 0x560e67041e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e6703ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef71200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67035f50 0x560e67041e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56118/0 (socket says 172.21.15.69:56118) 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef71200640 1 -- 172.21.15.69:0/144149268 learned_addr learned my addr 172.21.15.69:0/144149268 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef6be00640 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 0x560e6703a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef6b400640 1 -- 172.21.15.69:0/144149268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 msgr2=0x560e6703a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef6b400640 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 0x560e6703a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef6b400640 1 -- 172.21.15.69:0/144149268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67035f50 msgr2=0x560e67041e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.306+0000 7fef6b400640 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67035f50 0x560e67041e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.307+0000 7fef6b400640 1 -- 172.21.15.69:0/144149268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e66d736b0 con 0x560e67030de0 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.307+0000 7fef71200640 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67035f50 0x560e67041e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.307+0000 7fef6be00640 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 0x560e6703a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.307+0000 7fef6b400640 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e6703ae00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef54036040 tx=0x7fef54004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30e76d978ffae9c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.307+0000 7fef5a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:07:23.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.307+0000 7fef5a000640 1 -- 172.21.15.69:0/144149268 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54005d50 con 0x560e67030de0 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.307+0000 7fef5a000640 1 -- 172.21.15.69:0/144149268 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fef540048c0 con 0x560e67030de0 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.307+0000 7fef75f63740 1 -- 172.21.15.69:0/144149268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef5c0d12e0 con 0x560e67030de0 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.307+0000 7fef5a000640 1 -- 172.21.15.69:0/144149268 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fef54004ae0 con 0x560e67030de0 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.307+0000 7fef75f63740 10 client.24689 fetch_fsmap learned FSMap version 17 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.307+0000 7fef75f63740 1 -- 172.21.15.69:0/144149268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x560e67045220 con 0x560e67030de0 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.308+0000 7fef5a000640 1 -- 172.21.15.69:0/144149268 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef5c0d12e0 con 0x560e67030de0 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.308+0000 7fef5a000640 1 -- 172.21.15.69:0/144149268 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fef54046070 con 0x560e67030de0 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.308+0000 7fef75f63740 10 client.24689 fetch_fsmap finished waiting for FSMap version 17 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.308+0000 7fef75f63740 10 client.24689 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.308+0000 7fef75f63740 20 client.24689 populate_metadata read hostname 'smithi069' 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.308+0000 7fef75f63740 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560e670478b0 0x560e67048d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.308+0000 7fef75f63740 4 client.24689 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.308+0000 7fef75f63740 1 -- 172.21.15.69:0/144149268 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e66fa2240 con 0x560e670478b0 2024-03-19T01:07:23.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.308+0000 7fef6be00640 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560e670478b0 0x560e67048d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.309+0000 7fef6be00640 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560e670478b0 0x560e67048d20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9ab2b0a229b237 server_cookie=90bc2be520dd3f33 in_seq=0 out_seq=0 2024-03-19T01:07:23.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.309+0000 7fef5a000640 10 client.24689 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:07:23.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.309+0000 7fef5a000640 1 -- 172.21.15.69:0/144149268 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560e66fa2240 con 0x560e670478b0 2024-03-19T01:07:23.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.309+0000 7fef5a000640 10 client.24689 handle_command_reply: tid=0 2024-03-19T01:07:23.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.309+0000 7fef75f63740 1 client.24689 shutdown 2024-03-19T01:07:23.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef75f63740 1 -- 172.21.15.69:0/144149268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 msgr2=0x560e6703ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:23.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef75f63740 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e6703ae00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef54036040 tx=0x7fef54004270 comp rx=0 tx=0).stop 2024-03-19T01:07:23.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef75f63740 1 -- 172.21.15.69:0/144149268 shutdown_connections 2024-03-19T01:07:23.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef75f63740 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560e670478b0 0x560e67048d20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef75f63740 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67035f50 0x560e67041e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef5a000640 0 client.24689 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:07:23.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef75f63740 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e6703ae00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef75f63740 1 --2- 172.21.15.69:0/144149268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 0x560e6703a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef75f63740 1 -- 172.21.15.69:0/144149268 >> 172.21.15.69:0/144149268 conn(0x560e6702b960 msgr2=0x560e6702aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:23.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef75f63740 1 -- 172.21.15.69:0/144149268 shutdown_connections 2024-03-19T01:07:23.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef75f63740 1 -- 172.21.15.69:0/144149268 wait complete. 2024-03-19T01:07:23.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.310+0000 7fef75f63740 20 client.24689 trim_cache size 0 max 16384 2024-03-19T01:07:23.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.329+0000 7fef75f63740 1 Processor -- start 2024-03-19T01:07:23.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.329+0000 7fef75f63740 1 -- start start 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.329+0000 7fef75f63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 0x560e67038de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.329+0000 7fef75f63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e67041e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef75f63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e67035f50 0x560e6702bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef75f63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560e67047060 con 0x560e670303f0 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef75f63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560e67047730 con 0x560e67030de0 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef75f63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560e67047e00 con 0x560e67035f50 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef6be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 0x560e67038de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef6be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 0x560e67038de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56132/0 (socket says 172.21.15.69:56132) 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef6be00640 1 -- 172.21.15.69:0/3393255124 learned_addr learned my addr 172.21.15.69:0/3393255124 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef6b400640 1 --2- 172.21.15.69:0/3393255124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e67041e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef71200640 1 --2- 172.21.15.69:0/3393255124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e67035f50 0x560e6702bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef6be00640 1 -- 172.21.15.69:0/3393255124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e67035f50 msgr2=0x560e6702bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:07:23.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef6be00640 1 --2- 172.21.15.69:0/3393255124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e67035f50 0x560e6702bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef6be00640 1 -- 172.21.15.69:0/3393255124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 msgr2=0x560e67041e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:07:23.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef6be00640 1 --2- 172.21.15.69:0/3393255124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e67041e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef6be00640 1 -- 172.21.15.69:0/3393255124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e6703ba40 con 0x560e670303f0 2024-03-19T01:07:23.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef6b400640 1 --2- 172.21.15.69:0/3393255124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e67041e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:23.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.330+0000 7fef6be00640 1 --2- 172.21.15.69:0/3393255124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 0x560e67038de0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fef5c09b520 tx=0x7fef5c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fa720d78c3a0a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:23.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef60c00640 1 -- 172.21.15.69:0/3393255124 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c09b5f0 con 0x560e670303f0 2024-03-19T01:07:23.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef60c00640 1 -- 172.21.15.69:0/3393255124 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef5c0d12e0 con 0x560e670303f0 2024-03-19T01:07:23.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef60c00640 1 -- 172.21.15.69:0/3393255124 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c099960 con 0x560e670303f0 2024-03-19T01:07:23.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef75f63740 1 -- 172.21.15.69:0/3393255124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 msgr2=0x560e67038de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:23.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef75f63740 1 --2- 172.21.15.69:0/3393255124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 0x560e67038de0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fef5c09b520 tx=0x7fef5c000bc0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef75f63740 1 -- 172.21.15.69:0/3393255124 shutdown_connections 2024-03-19T01:07:23.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef75f63740 1 --2- 172.21.15.69:0/3393255124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e67035f50 0x560e6702bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef75f63740 1 --2- 172.21.15.69:0/3393255124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67030de0 0x560e67041e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef75f63740 1 --2- 172.21.15.69:0/3393255124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e670303f0 0x560e67038de0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef75f63740 1 -- 172.21.15.69:0/3393255124 >> 172.21.15.69:0/3393255124 conn(0x560e6702b960 msgr2=0x560e67026710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef75f63740 1 -- 172.21.15.69:0/3393255124 shutdown_connections 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.331+0000 7fef75f63740 1 -- 172.21.15.69:0/3393255124 wait complete. 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.332+0000 7fef75f63740 1 -- start start 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.332+0000 7fef75f63740 1 Processor -- start 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.332+0000 7fef75f63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 0x560e6703fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.332+0000 7fef75f63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67030de0 0x560e670b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.332+0000 7fef75f63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67035f50 0x560e670b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.332+0000 7fef75f63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560e67048ba0 con 0x560e67030de0 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.332+0000 7fef75f63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560e670484d0 con 0x560e67035f50 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.332+0000 7fef75f63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560e67049270 con 0x560e670303f0 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.332+0000 7fef75f63740 10 client.0 _ll_register_callbacks cb 0x7fef54009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:07:23.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef6b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67030de0 0x560e670b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef71200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67035f50 0x560e670b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef71200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67035f50 0x560e670b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42828/0 (socket says 172.21.15.69:42828) 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef71200640 1 -- 172.21.15.69:0/2132776532 learned_addr learned my addr 172.21.15.69:0/2132776532 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef6b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67030de0 0x560e670b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56138/0 (socket says 172.21.15.69:56138) 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef6be00640 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 0x560e6703fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef71200640 1 -- 172.21.15.69:0/2132776532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 msgr2=0x560e6703fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef71200640 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 0x560e6703fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef71200640 1 -- 172.21.15.69:0/2132776532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67030de0 msgr2=0x560e670b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef71200640 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67030de0 0x560e670b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef71200640 1 -- 172.21.15.69:0/2132776532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e670b48a0 con 0x560e67035f50 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef6be00640 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 0x560e6703fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef6b400640 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67030de0 0x560e670b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:23.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef71200640 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67035f50 0x560e670b4360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fef6401dbe0 tx=0x7fef64015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44212b35aab42a72 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.333+0000 7fef5a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef5a000640 1 -- 172.21.15.69:0/2132776532 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64010d60 con 0x560e67035f50 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef5a000640 1 -- 172.21.15.69:0/2132776532 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fef6401c460 con 0x560e67035f50 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef75f63740 1 -- 172.21.15.69:0/2132776532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef5c0d12e0 con 0x560e67035f50 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef5a000640 1 -- 172.21.15.69:0/2132776532 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fef64035050 con 0x560e67035f50 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef75f63740 10 client.24695 fetch_fsmap learned FSMap version 17 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef75f63740 1 -- 172.21.15.69:0/2132776532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x560e670b5b80 con 0x560e67035f50 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef5a000640 1 -- 172.21.15.69:0/2132776532 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef5c0d12e0 con 0x560e67035f50 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef5a000640 1 -- 172.21.15.69:0/2132776532 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fef6401a040 con 0x560e67035f50 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef75f63740 10 client.24695 fetch_fsmap finished waiting for FSMap version 17 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef75f63740 10 client.24695 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef75f63740 20 client.24695 populate_metadata read hostname 'smithi069' 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef75f63740 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560e670ba830 0x560e670b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:23.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef75f63740 4 client.24695 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.334+0000 7fef75f63740 1 -- 172.21.15.69:0/2132776532 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "status"}) -- 0x560e66f9c590 con 0x560e670ba830 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef6be00640 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560e670ba830 0x560e670b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef6be00640 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560e670ba830 0x560e670b9540 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec1212217c84a9d server_cookie=dd91bf75a11e2de0 in_seq=0 out_seq=0 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef5a000640 10 client.24695 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef5a000640 1 -- 172.21.15.69:0/2132776532 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x560e66f9c590 con 0x560e670ba830 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef5a000640 10 client.24695 handle_command_reply: tid=0 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef75f63740 1 client.24695 shutdown 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef75f63740 1 -- 172.21.15.69:0/2132776532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67035f50 msgr2=0x560e670b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef75f63740 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67035f50 0x560e670b4360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fef6401dbe0 tx=0x7fef64015040 comp rx=0 tx=0).stop 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef75f63740 1 -- 172.21.15.69:0/2132776532 shutdown_connections 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef75f63740 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560e670ba830 0x560e670b9540 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef75f63740 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e67035f50 0x560e670b4360 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef5a000640 0 client.24695 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef75f63740 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e67030de0 0x560e670b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef75f63740 1 --2- 172.21.15.69:0/2132776532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560e670303f0 0x560e6703fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.335+0000 7fef75f63740 1 -- 172.21.15.69:0/2132776532 >> 172.21.15.69:0/2132776532 conn(0x560e6702b960 msgr2=0x560e6702bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:23.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef75f63740 1 -- 172.21.15.69:0/2132776532 shutdown_connections 2024-03-19T01:07:23.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef75f63740 1 -- 172.21.15.69:0/2132776532 wait complete. 2024-03-19T01:07:23.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef75f63740 20 client.24695 trim_cache size 0 max 16384 2024-03-19T01:07:23.342 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:07:23.342 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T01:07:23.342 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-03-19T01:07:23.342 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24581, 2024-03-19T01:07:23.342 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-03-19T01:07:23.343 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-03-19T01:07:23.343 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-03-19T01:07:23.343 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 7.71823441, 2024-03-19T01:07:23.343 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 17, 2024-03-19T01:07:23.343 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 89, 2024-03-19T01:07:23.343 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T01:07:23.343 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 14.800312641 2024-03-19T01:07:23.343 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:07:23.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef73200640 1 -- 172.21.15.69:0/1571612135 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fef500a2800 msgr2=0x7fef500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:23.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef73200640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fef500a2800 0x7fef500a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x560e67031640 tx=0x7fef64009600 comp rx=0 tx=0).stop 2024-03-19T01:07:23.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef73200640 1 -- 172.21.15.69:0/1571612135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef6c103a40 msgr2=0x7fef6c199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:23.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef73200640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef6c103a40 0x7fef6c199d50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00e9c0 tx=0x7fef5c00ee80 comp rx=0 tx=0).stop 2024-03-19T01:07:23.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef73200640 1 -- 172.21.15.69:0/1571612135 shutdown_connections 2024-03-19T01:07:23.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef73200640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fef500a2800 0x7fef500a4cc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef73200640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef6c1055f0 0x7fef6c1a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef73200640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6c104c30 0x7fef6c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef73200640 1 --2- 172.21.15.69:0/1571612135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef6c103a40 0x7fef6c199d50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:23.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef73200640 1 -- 172.21.15.69:0/1571612135 >> 172.21.15.69:0/1571612135 conn(0x7fef6c0fefd0 msgr2=0x7fef6c107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:23.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.336+0000 7fef73200640 1 -- 172.21.15.69:0/1571612135 shutdown_connections 2024-03-19T01:07:23.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:23.337+0000 7fef73200640 1 -- 172.21.15.69:0/1571612135 wait complete. 2024-03-19T01:07:23.354 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-19T01:07:23.354 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-19T01:07:23.354 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-19T01:07:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:07:23.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:23 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2600245743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:24.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:23 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2600245743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:23 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2600245743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:24.988 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:24 smithi069 ceph-mon[31727]: pgmap v319: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-03-19T01:07:25.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:24 smithi086 ceph-mon[66829]: pgmap v319: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-03-19T01:07:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:24 smithi163 ceph-mon[67428]: pgmap v319: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-03-19T01:07:26.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.120+0000 7fe3ff200700 1 -- 172.21.15.69:0/959504719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3f8102e00 msgr2=0x7fe3f8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:26.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.120+0000 7fe3ff200700 1 --2- 172.21.15.69:0/959504719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3f8102e00 0x7fe3f8106e50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009990 tx=0x7fe3e8009ca0 comp rx=0 tx=0).stop 2024-03-19T01:07:26.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.120+0000 7fe3ff200700 1 -- 172.21.15.69:0/959504719 shutdown_connections 2024-03-19T01:07:26.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.120+0000 7fe3ff200700 1 --2- 172.21.15.69:0/959504719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f8107530 0x7fe3f810f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:26.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.120+0000 7fe3ff200700 1 --2- 172.21.15.69:0/959504719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3f8102450 0x7fe3f8102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:26.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.120+0000 7fe3ff200700 1 --2- 172.21.15.69:0/959504719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3f8102e00 0x7fe3f8106e50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:26.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.120+0000 7fe3ff200700 1 -- 172.21.15.69:0/959504719 >> 172.21.15.69:0/959504719 conn(0x7fe3f80fdce0 msgr2=0x7fe3f8100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:26.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.120+0000 7fe3ff200700 1 -- 172.21.15.69:0/959504719 shutdown_connections 2024-03-19T01:07:26.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.120+0000 7fe3ff200700 1 -- 172.21.15.69:0/959504719 wait complete. 2024-03-19T01:07:26.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.122+0000 7fe3ff200700 1 Processor -- start 2024-03-19T01:07:26.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.122+0000 7fe3ff200700 1 -- start start 2024-03-19T01:07:26.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.123+0000 7fe3ff200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3f8102450 0x7fe3f8074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:26.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.123+0000 7fe3ff200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f8102e00 0x7fe3f8074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:26.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.123+0000 7fe3ff200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3f8107530 0x7fe3f8075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:26.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.123+0000 7fe3ff200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f8112560 con 0x7fe3f8102450 2024-03-19T01:07:26.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.123+0000 7fe3ff200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f81123e0 con 0x7fe3f8107530 2024-03-19T01:07:26.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.123+0000 7fe3ff200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f81126e0 con 0x7fe3f8102e00 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.123+0000 7fe3fe800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3f8107530 0x7fe3f8075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.123+0000 7fe3fe800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3f8107530 0x7fe3f8075090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:41670/0 (socket says 172.21.15.69:41670) 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.123+0000 7fe3fde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3f8102450 0x7fe3f8074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.123+0000 7fe3fd400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f8102e00 0x7fe3f8074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.123+0000 7fe3fe800700 1 -- 172.21.15.69:0/3895573751 learned_addr learned my addr 172.21.15.69:0/3895573751 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.124+0000 7fe3fe800700 1 -- 172.21.15.69:0/3895573751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f8102e00 msgr2=0x7fe3f8074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.124+0000 7fe3fe800700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f8102e00 0x7fe3f8074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.124+0000 7fe3fe800700 1 -- 172.21.15.69:0/3895573751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3f8102450 msgr2=0x7fe3f8074610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.124+0000 7fe3fe800700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3f8102450 0x7fe3f8074610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.124+0000 7fe3fe800700 1 -- 172.21.15.69:0/3895573751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f81a6f10 con 0x7fe3f8107530 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.124+0000 7fe3fe800700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3f8107530 0x7fe3f8075090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400ef30 tx=0x7fe3f400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:26.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.124+0000 7fe3f2a00700 1 -- 172.21.15.69:0/3895573751 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40092e0 con 0x7fe3f8107530 2024-03-19T01:07:26.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.124+0000 7fe3f2a00700 1 -- 172.21.15.69:0/3895573751 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f4004bb0 con 0x7fe3f8107530 2024-03-19T01:07:26.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.124+0000 7fe3ff200700 1 -- 172.21.15.69:0/3895573751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f81a71a0 con 0x7fe3f8107530 2024-03-19T01:07:26.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.124+0000 7fe3f2a00700 1 -- 172.21.15.69:0/3895573751 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4004280 con 0x7fe3f8107530 2024-03-19T01:07:26.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.125+0000 7fe3ff200700 1 -- 172.21.15.69:0/3895573751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f81a7700 con 0x7fe3f8107530 2024-03-19T01:07:26.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.126+0000 7fe3ff200700 1 -- 172.21.15.69:0/3895573751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f8103930 con 0x7fe3f8107530 2024-03-19T01:07:26.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.127+0000 7fe3f2a00700 1 -- 172.21.15.69:0/3895573751 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe3f401b030 con 0x7fe3f8107530 2024-03-19T01:07:26.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.129+0000 7fe3f2a00700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe3e00a2910 0x7fe3e00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:26.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.129+0000 7fe3f2a00700 1 -- 172.21.15.69:0/3895573751 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe3f40ce690 con 0x7fe3f8107530 2024-03-19T01:07:26.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.129+0000 7fe3fde00700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe3e00a2910 0x7fe3e00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:26.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.130+0000 7fe3fde00700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe3e00a2910 0x7fe3e00a4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009c60 tx=0x7fe3ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:26.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.133+0000 7fe3f2a00700 1 -- 172.21.15.69:0/3895573751 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe3f4022020 con 0x7fe3f8107530 2024-03-19T01:07:26.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.404+0000 7fe3ff200700 1 -- 172.21.15.69:0/3895573751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe3f8075af0 con 0x7fe3f8107530 2024-03-19T01:07:26.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.405+0000 7fe3f2a00700 1 -- 172.21.15.69:0/3895573751 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fe3f4010040 con 0x7fe3f8107530 2024-03-19T01:07:26.406 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-19T01:07:26.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.409+0000 7fe3ff200700 1 -- 172.21.15.69:0/3895573751 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe3e00a2910 msgr2=0x7fe3e00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:26.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.409+0000 7fe3ff200700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe3e00a2910 0x7fe3e00a4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009c60 tx=0x7fe3ec009260 comp rx=0 tx=0).stop 2024-03-19T01:07:26.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.409+0000 7fe3ff200700 1 -- 172.21.15.69:0/3895573751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3f8107530 msgr2=0x7fe3f8075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:26.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.409+0000 7fe3ff200700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3f8107530 0x7fe3f8075090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400ef30 tx=0x7fe3f400c3a0 comp rx=0 tx=0).stop 2024-03-19T01:07:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.409+0000 7fe3ff200700 1 -- 172.21.15.69:0/3895573751 shutdown_connections 2024-03-19T01:07:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.409+0000 7fe3ff200700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe3e00a2910 0x7fe3e00a4dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.409+0000 7fe3ff200700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3f8107530 0x7fe3f8075090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.409+0000 7fe3ff200700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3f8102450 0x7fe3f8074610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.409+0000 7fe3ff200700 1 --2- 172.21.15.69:0/3895573751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f8102e00 0x7fe3f8074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.410+0000 7fe3ff200700 1 -- 172.21.15.69:0/3895573751 >> 172.21.15.69:0/3895573751 conn(0x7fe3f80fdce0 msgr2=0x7fe3f80fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.410+0000 7fe3ff200700 1 -- 172.21.15.69:0/3895573751 shutdown_connections 2024-03-19T01:07:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:26.410+0000 7fe3ff200700 1 -- 172.21.15.69:0/3895573751 wait complete. 2024-03-19T01:07:27.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:26 smithi086 ceph-mon[66829]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 0 op/s 2024-03-19T01:07:27.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:26 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3895573751' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T01:07:27.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:26 smithi069 ceph-mon[31727]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 0 op/s 2024-03-19T01:07:27.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:26 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3895573751' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T01:07:27.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:26 smithi163 ceph-mon[67428]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 0 op/s 2024-03-19T01:07:27.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:26 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3895573751' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T01:07:27.272 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-19T01:07:27.272 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-19T01:07:27.272 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-03-19T01:07:27.272 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-19T01:07:27.272 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-19T01:07:27.272 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-19T01:07:27.272 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-19T01:07:27.272 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-19T01:07:27.272 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-19T01:07:27.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:27.273 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-03-19T01:07:27.318 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-19T01:07:27.318 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-19T01:07:27.318 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-19T01:07:27.319 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:07:27.320 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-19T01:07:27.320 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 40478sec preferred_lft 40478sec 2024-03-19T01:07:27.320 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-03-19T01:07:27.320 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-03-19T01:07:27.320 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-19T01:07:27.320 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:07:27.321 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-19T01:07:27.321 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2024-03-19T01:07:27.321 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2024-03-19T01:07:27.321 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2024-03-19T01:07:27.321 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-19T01:07:27.321 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-19T01:07:27.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:07:27.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:07:27.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:27.863 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-19T01:07:27.933 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-03-19T01:07:27.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:27.936 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-03-19T01:07:28.001 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-03-19T01:07:28.001 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-19T01:07:28.001 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-19T01:07:28.001 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-19T01:07:28.001 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-19T01:07:28.002 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:07:28.003 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-19T01:07:28.003 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-19T01:07:28.003 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-19T01:07:28.003 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-19T01:07:28.003 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-19T01:07:28.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:07:28.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:07:28.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:28.802 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-03-19T01:07:28.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:28.855 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-03-19T01:07:28.904 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:07:28.905 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-19T01:07:28.905 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T01:07:28.905 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-19T01:07:28.905 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-19T01:07:29.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:28 smithi086 ceph-mon[66829]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-03-19T01:07:29.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:07:29.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:28 smithi069 ceph-mon[31727]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-03-19T01:07:29.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:28 smithi163 ceph-mon[67428]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-03-19T01:07:29.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:07:29.111 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-19T01:07:29.111 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:07:29.111 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-19T01:07:29.111 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-19T01:07:29.111 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-19T01:07:29.111 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-19T01:07:29.111 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-19T01:07:29.111 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-19T01:07:29.111 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-19T01:07:29.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:07:29.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:07:29.453 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:07:29.453 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-19T01:07:29.453 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-03-19T01:07:29.453 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2024-03-19T01:07:29.453 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-19T01:07:29.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:07:29.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:07:29.677 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-19T01:07:29.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-19T01:07:29.677 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-19T01:07:29.722 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-19T01:07:29.722 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-19T01:07:29.722 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-19T01:07:29.813 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-03-19T01:07:29.907 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-03-19T01:07:29.959 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-03-19T01:07:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/eb5af67b9727a91b8a25c850feed19d823ca72e572238fb860d0c8b8d6ecfac7/merged 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/71747480a49aaa7384f6e0408375bcccd15ee2be54d4d6b7aa0b1d7d711d313b/merged 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-03-19T01:07:29.961 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T01:07:29.962 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/719c0700ea78a487d04f6583ecdc30bf38c8a3fd01c36478e71700b9e761ef2c/merged 2024-03-19T01:07:29.962 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/acf04f52a0210180f2865af7322da255854fbda18660c2a5570c15cfe1fd8233/merged 2024-03-19T01:07:29.962 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/de568c9f15df23841a95bd53d6e94c664c3017a1fa9a67dc6227dd68119f5195/merged 2024-03-19T01:07:29.962 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/06b761e802e8e37361d64d853cf4e67eddbb2001d3d366a1d7f1d94b4a0642da/merged 2024-03-19T01:07:29.962 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/63fbe951a33868c92a3777084f511a670d2192e3d6a0a554ef5823a3dbc00ec0/merged 2024-03-19T01:07:29.962 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/c6973d1419ebba802d684cdd099209668eda969a4a27b107593406304311da62/merged 2024-03-19T01:07:29.962 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/21524eafeb0830d2d0f110f2882e4c151efcac6d35987dbce419f7b7e79bba9a/merged 2024-03-19T01:07:29.962 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/63d1e4ea1aafe4fd692c7c158d3fd311eae84cf3c44f18f72d98aa979dbf6288/merged 2024-03-19T01:07:29.962 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-03-19T01:07:29.962 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T01:07:29.962 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T01:07:29.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:29.963 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-03-19T01:07:30.012 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-19T01:07:30.012 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-19T01:07:30.017 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-03-19T01:07:30.101 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-03-19T01:07:30.146 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-03-19T01:07:30.146 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-03-19T01:07:30.146 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-03-19T01:07:30.146 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-03-19T01:07:30.146 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-03-19T01:07:30.146 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-03-19T01:07:30.146 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-19T01:07:30.147 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/eb5af67b9727a91b8a25c850feed19d823ca72e572238fb860d0c8b8d6ecfac7/merged 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/71747480a49aaa7384f6e0408375bcccd15ee2be54d4d6b7aa0b1d7d711d313b/merged 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/719c0700ea78a487d04f6583ecdc30bf38c8a3fd01c36478e71700b9e761ef2c/merged 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/acf04f52a0210180f2865af7322da255854fbda18660c2a5570c15cfe1fd8233/merged 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/de568c9f15df23841a95bd53d6e94c664c3017a1fa9a67dc6227dd68119f5195/merged 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/06b761e802e8e37361d64d853cf4e67eddbb2001d3d366a1d7f1d94b4a0642da/merged 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/63fbe951a33868c92a3777084f511a670d2192e3d6a0a554ef5823a3dbc00ec0/merged 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/c6973d1419ebba802d684cdd099209668eda969a4a27b107593406304311da62/merged 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/21524eafeb0830d2d0f110f2882e4c151efcac6d35987dbce419f7b7e79bba9a/merged 2024-03-19T01:07:30.148 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/63d1e4ea1aafe4fd692c7c158d3fd311eae84cf3c44f18f72d98aa979dbf6288/merged 2024-03-19T01:07:30.149 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-03-19T01:07:30.149 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T01:07:30.149 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T01:07:30.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:30.150 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-03-19T01:07:30.234 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:2024-03-19T01:07:30.232+0000 7f43b480d480 -1 init, newargv = 0x557d8f56d9d0 newargc=15 2024-03-19T01:07:30.235 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[72975]: starting ceph client 2024-03-19T01:07:30.246 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[72975]: starting fuse 2024-03-19T01:07:31.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:30 smithi086 ceph-mon[66829]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 717 B/s wr, 0 op/s 2024-03-19T01:07:31.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:30 smithi069 ceph-mon[31727]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 717 B/s wr, 0 op/s 2024-03-19T01:07:31.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:30 smithi163 ceph-mon[67428]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 717 B/s wr, 0 op/s 2024-03-19T01:07:31.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-03-19T01:07:31.442 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-03-19T01:07:31.500 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-03-19T01:07:31.500 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-03-19T01:07:31.500 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-03-19T01:07:31.500 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-03-19T01:07:31.500 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-03-19T01:07:31.500 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-03-19T01:07:31.500 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-03-19T01:07:31.501 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/eb5af67b9727a91b8a25c850feed19d823ca72e572238fb860d0c8b8d6ecfac7/merged 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/71747480a49aaa7384f6e0408375bcccd15ee2be54d4d6b7aa0b1d7d711d313b/merged 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/719c0700ea78a487d04f6583ecdc30bf38c8a3fd01c36478e71700b9e761ef2c/merged 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/acf04f52a0210180f2865af7322da255854fbda18660c2a5570c15cfe1fd8233/merged 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/de568c9f15df23841a95bd53d6e94c664c3017a1fa9a67dc6227dd68119f5195/merged 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/06b761e802e8e37361d64d853cf4e67eddbb2001d3d366a1d7f1d94b4a0642da/merged 2024-03-19T01:07:31.502 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/63fbe951a33868c92a3777084f511a670d2192e3d6a0a554ef5823a3dbc00ec0/merged 2024-03-19T01:07:31.503 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/c6973d1419ebba802d684cdd099209668eda969a4a27b107593406304311da62/merged 2024-03-19T01:07:31.503 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/21524eafeb0830d2d0f110f2882e4c151efcac6d35987dbce419f7b7e79bba9a/merged 2024-03-19T01:07:31.503 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/63d1e4ea1aafe4fd692c7c158d3fd311eae84cf3c44f18f72d98aa979dbf6288/merged 2024-03-19T01:07:31.503 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-03-19T01:07:31.503 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T01:07:31.503 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T01:07:31.503 INFO:teuthology.orchestra.run.smithi069.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-19T01:07:31.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:31.504 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-03-19T01:07:31.551 INFO:teuthology.orchestra.run.smithi069.stdout:118 2024-03-19T01:07:31.551 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-19T01:07:31.551 DEBUG:teuthology.orchestra.run.smithi069:> sudo stdin-killer -- python3 -c ' 2024-03-19T01:07:31.551 DEBUG:teuthology.orchestra.run.smithi069:> import glob 2024-03-19T01:07:31.551 DEBUG:teuthology.orchestra.run.smithi069:> import re 2024-03-19T01:07:31.551 DEBUG:teuthology.orchestra.run.smithi069:> import os 2024-03-19T01:07:31.551 DEBUG:teuthology.orchestra.run.smithi069:> import subprocess 2024-03-19T01:07:31.551 DEBUG:teuthology.orchestra.run.smithi069:> 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> def _find_admin_socket(client_name): 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> files = glob.glob(asok_path) 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> # Given a non-glob path, it better be there 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> if "*" not in asok_path: 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> assert(len(files) == 1) 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> return files[0] 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> for f in files: 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> contents = proc_f.read() 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> if mountpoint in contents: 2024-03-19T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi069:> return f 2024-03-19T01:07:31.553 DEBUG:teuthology.orchestra.run.smithi069:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-19T01:07:31.553 DEBUG:teuthology.orchestra.run.smithi069:> 2024-03-19T01:07:31.553 DEBUG:teuthology.orchestra.run.smithi069:> print(_find_admin_socket("client.admin")) 2024-03-19T01:07:31.553 DEBUG:teuthology.orchestra.run.smithi069:> ' 2024-03-19T01:07:31.720 INFO:teuthology.orchestra.run.smithi069.stdout:/var/run/ceph/ceph-client.admin.72975.asok 2024-03-19T01:07:31.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:07:31.736 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72975.asok 2024-03-19T01:07:31.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:31.736 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72975.asok status 2024-03-19T01:07:31.973 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "metadata": { 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_sha1": "b6ae24918b03b1b8a19c9263a50655bba80bb810", 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "entity_id": "admin", 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": "smithi069", 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "pid": "72975", 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "root": "/" 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_count": 0, 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_pinned_count": 0, 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 14598, 2024-03-19T01:07:31.974 INFO:teuthology.orchestra.run.smithi069.stdout: "inst": { 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "name": { 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "client", 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "num": 14598 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2502466562 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2502466562 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:07:31.975 INFO:teuthology.orchestra.run.smithi069.stdout: "inst_str": "client.14598 192.168.0.1:0/2502466562", 2024-03-19T01:07:31.976 INFO:teuthology.orchestra.run.smithi069.stdout: "addr_str": "192.168.0.1:0/2502466562", 2024-03-19T01:07:31.976 INFO:teuthology.orchestra.run.smithi069.stdout: "inode_count": 1, 2024-03-19T01:07:31.976 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_epoch": 17, 2024-03-19T01:07:31.976 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch": 89, 2024-03-19T01:07:31.976 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch_barrier": 0, 2024-03-19T01:07:31.976 INFO:teuthology.orchestra.run.smithi069.stdout: "blocklisted": false, 2024-03-19T01:07:31.976 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs" 2024-03-19T01:07:31.976 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:07:31.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:31.989 DEBUG:teuthology.orchestra.run.smithi069:> stat --file-system '--printf=%T 2024-03-19T01:07:31.989 DEBUG:teuthology.orchestra.run.smithi069:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-19T01:07:32.043 INFO:teuthology.orchestra.run.smithi069.stdout:fuseblk 2024-03-19T01:07:32.044 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-19T01:07:32.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:32.044 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-19T01:07:32.126 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-19T01:07:32.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-19T01:07:32.265 INFO:teuthology.orchestra.run.smithi069.stdout:# file: . 2024-03-19T01:07:32.265 INFO:teuthology.orchestra.run.smithi069.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-19T01:07:32.265 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:32.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:07:32.275 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T01:07:32.276 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T01:07:32.324 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:07:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.583+0000 7f52a2e00640 1 -- 172.21.15.69:0/834340483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c108fb0 msgr2=0x7f529c109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.583+0000 7f52a2e00640 1 --2- 172.21.15.69:0/834340483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c108fb0 0x7f529c109430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5284009900 tx=0x7f528402bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.583+0000 7f52a2e00640 1 -- 172.21.15.69:0/834340483 shutdown_connections 2024-03-19T01:07:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.583+0000 7f52a2e00640 1 --2- 172.21.15.69:0/834340483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f529c109970 0x7f529c110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.584+0000 7f52a2e00640 1 --2- 172.21.15.69:0/834340483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c108fb0 0x7f529c109430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.584+0000 7f52a2e00640 1 --2- 172.21.15.69:0/834340483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f529c107e60 0x7f529c108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.584+0000 7f52a2e00640 1 -- 172.21.15.69:0/834340483 >> 172.21.15.69:0/834340483 conn(0x7f529c076e30 msgr2=0x7f529c079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.584+0000 7f52a2e00640 1 -- 172.21.15.69:0/834340483 shutdown_connections 2024-03-19T01:07:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.584+0000 7f52a2e00640 1 -- 172.21.15.69:0/834340483 wait complete. 2024-03-19T01:07:32.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f52a2e00640 1 Processor -- start 2024-03-19T01:07:32.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f52a2e00640 1 -- start start 2024-03-19T01:07:32.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f52a2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c107e60 0x7f529c19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f52a2e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f529c108fb0 0x7f529c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f52a2e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f529c109970 0x7f529c1a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f52a2e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f529c113ae0 con 0x7f529c109970 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f52a2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f529c113960 con 0x7f529c107e60 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f52a2e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f529c113c60 con 0x7f529c108fb0 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f529be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c107e60 0x7f529c19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f52a0e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f529c109970 0x7f529c1a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f529be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c107e60 0x7f529c19e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:41684/0 (socket says 172.21.15.69:41684) 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f529be00640 1 -- 172.21.15.69:0/3697900963 learned_addr learned my addr 172.21.15.69:0/3697900963 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.585+0000 7f5293400640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f529c108fb0 0x7f529c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.586+0000 7f529be00640 1 -- 172.21.15.69:0/3697900963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f529c108fb0 msgr2=0x7f529c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.586+0000 7f529be00640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f529c108fb0 0x7f529c19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.586+0000 7f529be00640 1 -- 172.21.15.69:0/3697900963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f529c109970 msgr2=0x7f529c1a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.586+0000 7f529be00640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f529c109970 0x7f529c1a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.586+0000 7f529be00640 1 -- 172.21.15.69:0/3697900963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c1a5d70 con 0x7f529c107e60 2024-03-19T01:07:32.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.586+0000 7f529be00640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c107e60 0x7f529c19e150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f528c00eab0 tx=0x7f528c00ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:32.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.586+0000 7f5299600640 1 -- 172.21.15.69:0/3697900963 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c00ce40 con 0x7f529c107e60 2024-03-19T01:07:32.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.586+0000 7f5299600640 1 -- 172.21.15.69:0/3697900963 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528c011950 con 0x7f529c107e60 2024-03-19T01:07:32.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.586+0000 7f5299600640 1 -- 172.21.15.69:0/3697900963 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c010ae0 con 0x7f529c107e60 2024-03-19T01:07:32.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.586+0000 7f52a2e00640 1 -- 172.21.15.69:0/3697900963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529c1a6060 con 0x7f529c107e60 2024-03-19T01:07:32.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.587+0000 7f52a2e00640 1 -- 172.21.15.69:0/3697900963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f529c1a6530 con 0x7f529c107e60 2024-03-19T01:07:32.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.587+0000 7f52a2e00640 1 -- 172.21.15.69:0/3697900963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529c10f170 con 0x7f529c107e60 2024-03-19T01:07:32.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.589+0000 7f5299600640 1 -- 172.21.15.69:0/3697900963 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f528c011b00 con 0x7f529c107e60 2024-03-19T01:07:32.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.591+0000 7f5299600640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f52800a2800 0x7f52800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:32.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.591+0000 7f5299600640 1 -- 172.21.15.69:0/3697900963 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f528c0cf670 con 0x7f529c107e60 2024-03-19T01:07:32.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.591+0000 7f5293400640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f52800a2800 0x7f52800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:32.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.592+0000 7f5293400640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f52800a2800 0x7f52800a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5284000c00 tx=0x7f5284005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:32.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.593+0000 7f5299600640 1 -- 172.21.15.69:0/3697900963 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f528c016030 con 0x7f529c107e60 2024-03-19T01:07:32.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.796+0000 7f52a2e00640 1 -- 172.21.15.69:0/3697900963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f529c02d4a0 con 0x7f529c107e60 2024-03-19T01:07:32.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.798+0000 7f5299600640 1 -- 172.21.15.69:0/3697900963 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24069 (secure 0 0 0) 0x7f528c09c470 con 0x7f529c107e60 2024-03-19T01:07:32.799 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:32.799 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":89,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:07:20.123357+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.800+0000 7f52a2e00640 1 -- 172.21.15.69:0/3697900963 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f52800a2800 msgr2=0x7f52800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.800+0000 7f52a2e00640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f52800a2800 0x7f52800a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5284000c00 tx=0x7f5284005fd0 comp rx=0 tx=0).stop 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.800+0000 7f52a2e00640 1 -- 172.21.15.69:0/3697900963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c107e60 msgr2=0x7f529c19e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.800+0000 7f52a2e00640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c107e60 0x7f529c19e150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f528c00eab0 tx=0x7f528c00ef70 comp rx=0 tx=0).stop 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.800+0000 7f52a2e00640 1 -- 172.21.15.69:0/3697900963 shutdown_connections 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.800+0000 7f52a2e00640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f52800a2800 0x7f52800a4cc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.800+0000 7f52a2e00640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f529c109970 0x7f529c1a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.800+0000 7f52a2e00640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f529c108fb0 0x7f529c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.800+0000 7f52a2e00640 1 --2- 172.21.15.69:0/3697900963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c107e60 0x7f529c19e150 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.800+0000 7f52a2e00640 1 -- 172.21.15.69:0/3697900963 >> 172.21.15.69:0/3697900963 conn(0x7f529c076e30 msgr2=0x7f529c10c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.800+0000 7f52a2e00640 1 -- 172.21.15.69:0/3697900963 shutdown_connections 2024-03-19T01:07:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:32.801+0000 7f52a2e00640 1 -- 172.21.15.69:0/3697900963 wait complete. 2024-03-19T01:07:32.821 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:07:13.368130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:07:14.444206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T01:07:18.189146+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T01:07:32.821 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:07:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:32 smithi086 ceph-mon[66829]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-19T01:07:33.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:32 smithi069 ceph-mon[31727]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-19T01:07:33.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.039+0000 7f1457e00640 1 -- 172.21.15.69:0/1823259756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450107e40 msgr2=0x7f1450108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.039+0000 7f1457e00640 1 --2- 172.21.15.69:0/1823259756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450107e40 0x7f1450108260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1444009860 tx=0x7f144402bb70 comp rx=0 tx=0).stop 2024-03-19T01:07:33.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.039+0000 7f1457e00640 1 -- 172.21.15.69:0/1823259756 shutdown_connections 2024-03-19T01:07:33.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.039+0000 7f1457e00640 1 --2- 172.21.15.69:0/1823259756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1450109950 0x7f1450110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.039+0000 7f1457e00640 1 --2- 172.21.15.69:0/1823259756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1450108f90 0x7f1450109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.039+0000 7f1457e00640 1 --2- 172.21.15.69:0/1823259756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450107e40 0x7f1450108260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.039+0000 7f1457e00640 1 -- 172.21.15.69:0/1823259756 >> 172.21.15.69:0/1823259756 conn(0x7f1450076de0 msgr2=0x7f1450079220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:33.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.039+0000 7f1457e00640 1 -- 172.21.15.69:0/1823259756 shutdown_connections 2024-03-19T01:07:33.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.039+0000 7f1457e00640 1 -- 172.21.15.69:0/1823259756 wait complete. 2024-03-19T01:07:33.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.040+0000 7f1457e00640 1 Processor -- start 2024-03-19T01:07:33.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.040+0000 7f1457e00640 1 -- start start 2024-03-19T01:07:33.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1457e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1450107e40 0x7f145019e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1457e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1450108f90 0x7f145019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1457e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450109950 0x7f14501a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1457e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1450113ae0 con 0x7f1450107e40 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1457e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1450113960 con 0x7f1450109950 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1457e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1450113c60 con 0x7f1450108f90 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1455400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1450107e40 0x7f145019e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1455400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1450107e40 0x7f145019e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50252/0 (socket says 172.21.15.69:50252) 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1455400640 1 -- 172.21.15.69:0/1335761342 learned_addr learned my addr 172.21.15.69:0/1335761342 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1455e00640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450109950 0x7f14501a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1454a00640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1450108f90 0x7f145019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1455400640 1 -- 172.21.15.69:0/1335761342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1450108f90 msgr2=0x7f145019e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1455400640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1450108f90 0x7f145019e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1455400640 1 -- 172.21.15.69:0/1335761342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450109950 msgr2=0x7f14501a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1455400640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450109950 0x7f14501a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1455400640 1 -- 172.21.15.69:0/1335761342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14501a5d50 con 0x7f1450107e40 2024-03-19T01:07:33.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.041+0000 7f1455e00640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450109950 0x7f14501a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:33.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.042+0000 7f1455400640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1450107e40 0x7f145019e130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14440349e0 tx=0x7f1444032450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:33.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.042+0000 7f144e000640 1 -- 172.21.15.69:0/1335761342 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444041030 con 0x7f1450107e40 2024-03-19T01:07:33.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.042+0000 7f144e000640 1 -- 172.21.15.69:0/1335761342 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f144402c510 con 0x7f1450107e40 2024-03-19T01:07:33.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.042+0000 7f144e000640 1 -- 172.21.15.69:0/1335761342 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444039040 con 0x7f1450107e40 2024-03-19T01:07:33.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.042+0000 7f1457e00640 1 -- 172.21.15.69:0/1335761342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14501a5fe0 con 0x7f1450107e40 2024-03-19T01:07:33.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.042+0000 7f1457e00640 1 -- 172.21.15.69:0/1335761342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14501a63f0 con 0x7f1450107e40 2024-03-19T01:07:33.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.043+0000 7f1457e00640 1 -- 172.21.15.69:0/1335761342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145010f150 con 0x7f1450107e40 2024-03-19T01:07:33.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.046+0000 7f144e000640 1 -- 172.21.15.69:0/1335761342 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f1444044070 con 0x7f1450107e40 2024-03-19T01:07:33.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.047+0000 7f144e000640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14380a2800 0x7f14380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.047+0000 7f144e000640 1 -- 172.21.15.69:0/1335761342 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f14440f0150 con 0x7f1450107e40 2024-03-19T01:07:33.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.047+0000 7f1454a00640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14380a2800 0x7f14380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.048+0000 7f1454a00640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14380a2800 0x7f14380a4cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f145019f640 tx=0x7f1440009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:33.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.052+0000 7f144e000640 1 -- 172.21.15.69:0/1335761342 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f14440bcfb0 con 0x7f1450107e40 2024-03-19T01:07:33.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:32 smithi163 ceph-mon[67428]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-19T01:07:33.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.251+0000 7f1457e00640 1 -- 172.21.15.69:0/1335761342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1450062560 con 0x7f1450107e40 2024-03-19T01:07:33.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.251+0000 7f144e000640 1 -- 172.21.15.69:0/1335761342 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f14440bc8b0 con 0x7f1450107e40 2024-03-19T01:07:33.253 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:07:33.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 -- 172.21.15.69:0/1335761342 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14380a2800 msgr2=0x7f14380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14380a2800 0x7f14380a4cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f145019f640 tx=0x7f1440009260 comp rx=0 tx=0).stop 2024-03-19T01:07:33.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 -- 172.21.15.69:0/1335761342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1450107e40 msgr2=0x7f145019e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1450107e40 0x7f145019e130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14440349e0 tx=0x7f1444032450 comp rx=0 tx=0).stop 2024-03-19T01:07:33.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 -- 172.21.15.69:0/1335761342 shutdown_connections 2024-03-19T01:07:33.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14380a2800 0x7f14380a4cc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450109950 0x7f14501a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1450108f90 0x7f145019e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 --2- 172.21.15.69:0/1335761342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1450107e40 0x7f145019e130 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 -- 172.21.15.69:0/1335761342 >> 172.21.15.69:0/1335761342 conn(0x7f1450076de0 msgr2=0x7f145010c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:33.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 -- 172.21.15.69:0/1335761342 shutdown_connections 2024-03-19T01:07:33.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.254+0000 7f1457e00640 1 -- 172.21.15.69:0/1335761342 wait complete. 2024-03-19T01:07:33.275 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:07:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.493+0000 7fbd85a00640 1 -- 172.21.15.69:0/3262658516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80104c50 msgr2=0x7fbd801050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.493+0000 7fbd85a00640 1 --2- 172.21.15.69:0/3262658516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80104c50 0x7fbd801050d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009ca0 tx=0x7fbd6802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.493+0000 7fbd85a00640 1 -- 172.21.15.69:0/3262658516 shutdown_connections 2024-03-19T01:07:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.493+0000 7fbd85a00640 1 --2- 172.21.15.69:0/3262658516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd80105610 0x7fbd8010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.493+0000 7fbd85a00640 1 --2- 172.21.15.69:0/3262658516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80104c50 0x7fbd801050d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.493+0000 7fbd85a00640 1 --2- 172.21.15.69:0/3262658516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80103a60 0x7fbd80103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.493+0000 7fbd85a00640 1 -- 172.21.15.69:0/3262658516 >> 172.21.15.69:0/3262658516 conn(0x7fbd800fefd0 msgr2=0x7fbd80101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.493+0000 7fbd85a00640 1 -- 172.21.15.69:0/3262658516 shutdown_connections 2024-03-19T01:07:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.493+0000 7fbd85a00640 1 -- 172.21.15.69:0/3262658516 wait complete. 2024-03-19T01:07:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.494+0000 7fbd85a00640 1 Processor -- start 2024-03-19T01:07:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.494+0000 7fbd85a00640 1 -- start start 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd85a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80103a60 0x7fbd80199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd85a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80104c50 0x7fbd8019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd85a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd80105610 0x7fbd801a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd85a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8010f6e0 con 0x7fbd80105610 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd85a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8010f560 con 0x7fbd80104c50 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd85a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8010f860 con 0x7fbd80103a60 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd7e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80104c50 0x7fbd8019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd7e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80104c50 0x7fbd8019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:41724/0 (socket says 172.21.15.69:41724) 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd7e000640 1 -- 172.21.15.69:0/1567376130 learned_addr learned my addr 172.21.15.69:0/1567376130 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd7f400640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd80105610 0x7fbd801a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd7ea00640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80103a60 0x7fbd80199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd7e000640 1 -- 172.21.15.69:0/1567376130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80103a60 msgr2=0x7fbd80199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd7e000640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80103a60 0x7fbd80199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd7e000640 1 -- 172.21.15.69:0/1567376130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd80105610 msgr2=0x7fbd801a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd7e000640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd80105610 0x7fbd801a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd7e000640 1 -- 172.21.15.69:0/1567376130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd801a1990 con 0x7fbd80104c50 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.495+0000 7fbd7f400640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd80105610 0x7fbd801a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.496+0000 7fbd7e000640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80104c50 0x7fbd8019a2b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbd680325f0 tx=0x7fbd68032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.496+0000 7fbd77400640 1 -- 172.21.15.69:0/1567376130 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68041020 con 0x7fbd80104c50 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.496+0000 7fbd77400640 1 -- 172.21.15.69:0/1567376130 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd68034cd0 con 0x7fbd80104c50 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.496+0000 7fbd77400640 1 -- 172.21.15.69:0/1567376130 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68039040 con 0x7fbd80104c50 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.496+0000 7fbd85a00640 1 -- 172.21.15.69:0/1567376130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd801a1c20 con 0x7fbd80104c50 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.496+0000 7fbd85a00640 1 -- 172.21.15.69:0/1567376130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd801a2130 con 0x7fbd80104c50 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.497+0000 7fbd85a00640 1 -- 172.21.15.69:0/1567376130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8010ae20 con 0x7fbd80104c50 2024-03-19T01:07:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.499+0000 7fbd77400640 1 -- 172.21.15.69:0/1567376130 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fbd68044020 con 0x7fbd80104c50 2024-03-19T01:07:33.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.500+0000 7fbd77400640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd640a2880 0x7fbd640a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.500+0000 7fbd77400640 1 -- 172.21.15.69:0/1567376130 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbd680f0c50 con 0x7fbd80104c50 2024-03-19T01:07:33.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.500+0000 7fbd7ea00640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd640a2880 0x7fbd640a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.501+0000 7fbd7ea00640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd640a2880 0x7fbd640a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd80104ab0 tx=0x7fbd70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:33.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.507+0000 7fbd77400640 1 -- 172.21.15.69:0/1567376130 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbd6804b020 con 0x7fbd80104c50 2024-03-19T01:07:33.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.709+0000 7fbd85a00640 1 -- 172.21.15.69:0/1567376130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd80062420 con 0x7fbd80104c50 2024-03-19T01:07:33.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.719+0000 7fbd77400640 1 -- 172.21.15.69:0/1567376130 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fbd680bda50 con 0x7fbd80104c50 2024-03-19T01:07:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:07:33.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.722+0000 7fbd85a00640 1 -- 172.21.15.69:0/1567376130 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd640a2880 msgr2=0x7fbd640a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.722+0000 7fbd85a00640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd640a2880 0x7fbd640a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd80104ab0 tx=0x7fbd70005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.722+0000 7fbd85a00640 1 -- 172.21.15.69:0/1567376130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80104c50 msgr2=0x7fbd8019a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.722+0000 7fbd85a00640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80104c50 0x7fbd8019a2b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbd680325f0 tx=0x7fbd68032ec0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.722+0000 7fbd85a00640 1 -- 172.21.15.69:0/1567376130 shutdown_connections 2024-03-19T01:07:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.722+0000 7fbd85a00640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd640a2880 0x7fbd640a4d40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.722+0000 7fbd85a00640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd80105610 0x7fbd801a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.722+0000 7fbd85a00640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd80104c50 0x7fbd8019a2b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.722+0000 7fbd85a00640 1 --2- 172.21.15.69:0/1567376130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80103a60 0x7fbd80199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.722+0000 7fbd85a00640 1 -- 172.21.15.69:0/1567376130 >> 172.21.15.69:0/1567376130 conn(0x7fbd800fefd0 msgr2=0x7fbd80107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:33.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.723+0000 7fbd85a00640 1 -- 172.21.15.69:0/1567376130 shutdown_connections 2024-03-19T01:07:33.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.723+0000 7fbd85a00640 1 -- 172.21.15.69:0/1567376130 wait complete. 2024-03-19T01:07:33.743 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:07:33.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.962+0000 7f1637400640 1 -- 172.21.15.69:0/3293507896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630107e40 msgr2=0x7f1630108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.962+0000 7f1637400640 1 --2- 172.21.15.69:0/3293507896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630107e40 0x7f1630108260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1620009900 tx=0x7f162002bc50 comp rx=0 tx=0).stop 2024-03-19T01:07:33.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.962+0000 7f1637400640 1 -- 172.21.15.69:0/3293507896 shutdown_connections 2024-03-19T01:07:33.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.962+0000 7f1637400640 1 --2- 172.21.15.69:0/3293507896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1630109950 0x7f1630110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.962+0000 7f1637400640 1 --2- 172.21.15.69:0/3293507896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630108f90 0x7f1630109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.962+0000 7f1637400640 1 --2- 172.21.15.69:0/3293507896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630107e40 0x7f1630108260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.962+0000 7f1637400640 1 -- 172.21.15.69:0/3293507896 >> 172.21.15.69:0/3293507896 conn(0x7f1630076de0 msgr2=0x7f1630079220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:33.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.962+0000 7f1637400640 1 -- 172.21.15.69:0/3293507896 shutdown_connections 2024-03-19T01:07:33.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.962+0000 7f1637400640 1 -- 172.21.15.69:0/3293507896 wait complete. 2024-03-19T01:07:33.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.963+0000 7f1637400640 1 Processor -- start 2024-03-19T01:07:33.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.963+0000 7f1637400640 1 -- start start 2024-03-19T01:07:33.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1637400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1630107e40 0x7f163019e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1637400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630108f90 0x7f163019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1637400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109950 0x7f16301a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1637400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1630113b10 con 0x7f1630107e40 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1637400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1630113990 con 0x7f1630108f90 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1637400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1630113c90 con 0x7f1630109950 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1634a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1630107e40 0x7f163019e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1627e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630108f90 0x7f163019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1634a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1630107e40 0x7f163019e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50298/0 (socket says 172.21.15.69:50298) 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1634a00640 1 -- 172.21.15.69:0/1282707973 learned_addr learned my addr 172.21.15.69:0/1282707973 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1635400640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109950 0x7f16301a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1627e00640 1 -- 172.21.15.69:0/1282707973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109950 msgr2=0x7f16301a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1627e00640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109950 0x7f16301a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1627e00640 1 -- 172.21.15.69:0/1282707973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1630107e40 msgr2=0x7f163019e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:33.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1627e00640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1630107e40 0x7f163019e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:33.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1627e00640 1 -- 172.21.15.69:0/1282707973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16301a5d50 con 0x7f1630108f90 2024-03-19T01:07:33.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.964+0000 7f1634a00640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1630107e40 0x7f163019e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:33.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.965+0000 7f1627e00640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630108f90 0x7f163019e670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f161800e9c0 tx=0x7f161800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:33.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.965+0000 7f162e000640 1 -- 172.21.15.69:0/1282707973 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161800ccc0 con 0x7f1630108f90 2024-03-19T01:07:33.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.965+0000 7f162e000640 1 -- 172.21.15.69:0/1282707973 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16180035f0 con 0x7f1630108f90 2024-03-19T01:07:33.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.965+0000 7f162e000640 1 -- 172.21.15.69:0/1282707973 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161800ce60 con 0x7f1630108f90 2024-03-19T01:07:33.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.965+0000 7f1637400640 1 -- 172.21.15.69:0/1282707973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16301a5fe0 con 0x7f1630108f90 2024-03-19T01:07:33.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.965+0000 7f1637400640 1 -- 172.21.15.69:0/1282707973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16301a64b0 con 0x7f1630108f90 2024-03-19T01:07:33.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.966+0000 7f1637400640 1 -- 172.21.15.69:0/1282707973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f163010f150 con 0x7f1630108f90 2024-03-19T01:07:33.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.968+0000 7f162e000640 1 -- 172.21.15.69:0/1282707973 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f16180037a0 con 0x7f1630108f90 2024-03-19T01:07:33.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.969+0000 7f162e000640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f16140a2880 0x7f16140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:33.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.969+0000 7f162e000640 1 -- 172.21.15.69:0/1282707973 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f16180cefc0 con 0x7f1630108f90 2024-03-19T01:07:33.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.970+0000 7f1634a00640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f16140a2880 0x7f16140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:33.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.970+0000 7f1634a00640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f16140a2880 0x7f16140a4d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f16200096c0 tx=0x7f1620000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:33.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:33.973+0000 7f162e000640 1 -- 172.21.15.69:0/1282707973 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f161809bdc0 con 0x7f1630108f90 2024-03-19T01:07:34.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:33 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3697900963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:34.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:33 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1335761342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:07:34.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:33 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3697900963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:34.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:33 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1335761342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:07:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:33 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3697900963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:33 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1335761342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:07:34.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.176+0000 7f1637400640 1 -- 172.21.15.69:0/1282707973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1630062420 con 0x7f1630108f90 2024-03-19T01:07:34.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.177+0000 7f162e000640 1 -- 172.21.15.69:0/1282707973 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f161809b6c0 con 0x7f1630108f90 2024-03-19T01:07:34.178 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-03-19T01:07:34.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.180+0000 7f1637400640 1 -- 172.21.15.69:0/1282707973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f16140a2880 msgr2=0x7f16140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:34.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.180+0000 7f1637400640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f16140a2880 0x7f16140a4d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f16200096c0 tx=0x7f1620000bc0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.180+0000 7f1637400640 1 -- 172.21.15.69:0/1282707973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630108f90 msgr2=0x7f163019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:34.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.180+0000 7f1637400640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630108f90 0x7f163019e670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f161800e9c0 tx=0x7f161800ee80 comp rx=0 tx=0).stop 2024-03-19T01:07:34.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.180+0000 7f1637400640 1 -- 172.21.15.69:0/1282707973 shutdown_connections 2024-03-19T01:07:34.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.180+0000 7f1637400640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f16140a2880 0x7f16140a4d40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.180+0000 7f1637400640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109950 0x7f16301a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.180+0000 7f1637400640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630108f90 0x7f163019e670 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.180+0000 7f1637400640 1 --2- 172.21.15.69:0/1282707973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1630107e40 0x7f163019e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.180+0000 7f1637400640 1 -- 172.21.15.69:0/1282707973 >> 172.21.15.69:0/1282707973 conn(0x7f1630076de0 msgr2=0x7f163010c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:34.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.180+0000 7f1637400640 1 -- 172.21.15.69:0/1282707973 shutdown_connections 2024-03-19T01:07:34.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.181+0000 7f1637400640 1 -- 172.21.15.69:0/1282707973 wait complete. 2024-03-19T01:07:34.199 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T01:07:34.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.412+0000 7fc2c2800640 1 -- 172.21.15.69:0/3612367035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc074280 msgr2=0x7fc2bc074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:34.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.412+0000 7fc2c2800640 1 --2- 172.21.15.69:0/3612367035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc074280 0x7fc2bc074700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009930 tx=0x7fc2a402bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:34.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.412+0000 7fc2c2800640 1 -- 172.21.15.69:0/3612367035 shutdown_connections 2024-03-19T01:07:34.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.412+0000 7fc2c2800640 1 --2- 172.21.15.69:0/3612367035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc10de00 0x7fc2bc1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.412+0000 7fc2c2800640 1 --2- 172.21.15.69:0/3612367035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc074280 0x7fc2bc074700 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.412+0000 7fc2c2800640 1 --2- 172.21.15.69:0/3612367035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2bc075860 0x7fc2bc073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.412+0000 7fc2c2800640 1 -- 172.21.15.69:0/3612367035 >> 172.21.15.69:0/3612367035 conn(0x7fc2bc0fcda0 msgr2=0x7fc2bc0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:34.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.412+0000 7fc2c2800640 1 -- 172.21.15.69:0/3612367035 shutdown_connections 2024-03-19T01:07:34.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.412+0000 7fc2c2800640 1 -- 172.21.15.69:0/3612367035 wait complete. 2024-03-19T01:07:34.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.413+0000 7fc2c2800640 1 Processor -- start 2024-03-19T01:07:34.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.413+0000 7fc2c2800640 1 -- start start 2024-03-19T01:07:34.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2c2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc074280 0x7fc2bc19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:34.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2c2800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2bc075860 0x7fc2bc19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:34.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2c2800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc10de00 0x7fc2bc1a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:34.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2c2800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc113af0 con 0x7fc2bc075860 2024-03-19T01:07:34.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2c2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc113970 con 0x7fc2bc074280 2024-03-19T01:07:34.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2c2800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc113c70 con 0x7fc2bc10de00 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc074280 0x7fc2bc19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc074280 0x7fc2bc19e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:41782/0 (socket says 172.21.15.69:41782) 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2bb400640 1 -- 172.21.15.69:0/2485577221 learned_addr learned my addr 172.21.15.69:0/2485577221 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2baa00640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2bc075860 0x7fc2bc19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2bbe00640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc10de00 0x7fc2bc1a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2bb400640 1 -- 172.21.15.69:0/2485577221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc10de00 msgr2=0x7fc2bc1a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2bb400640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc10de00 0x7fc2bc1a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2bb400640 1 -- 172.21.15.69:0/2485577221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2bc075860 msgr2=0x7fc2bc19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2bb400640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2bc075860 0x7fc2bc19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.414+0000 7fc2bb400640 1 -- 172.21.15.69:0/2485577221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc1a5db0 con 0x7fc2bc074280 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.415+0000 7fc2bb400640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc074280 0x7fc2bc19e190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00ebc0 tx=0x7fc2ac00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.415+0000 7fc2b3e00640 1 -- 172.21.15.69:0/2485577221 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac010630 con 0x7fc2bc074280 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.415+0000 7fc2b3e00640 1 -- 172.21.15.69:0/2485577221 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac010cb0 con 0x7fc2bc074280 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.415+0000 7fc2b3e00640 1 -- 172.21.15.69:0/2485577221 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac018bd0 con 0x7fc2bc074280 2024-03-19T01:07:34.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.415+0000 7fc2c2800640 1 -- 172.21.15.69:0/2485577221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2bc1a60a0 con 0x7fc2bc074280 2024-03-19T01:07:34.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.415+0000 7fc2c2800640 1 -- 172.21.15.69:0/2485577221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2bc1a6570 con 0x7fc2bc074280 2024-03-19T01:07:34.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.416+0000 7fc2c2800640 1 -- 172.21.15.69:0/2485577221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2bc103fd0 con 0x7fc2bc074280 2024-03-19T01:07:34.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.418+0000 7fc2b3e00640 1 -- 172.21.15.69:0/2485577221 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc2ac0107d0 con 0x7fc2bc074280 2024-03-19T01:07:34.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.419+0000 7fc2b3e00640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc2a00a2880 0x7fc2a00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:34.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.419+0000 7fc2b3e00640 1 -- 172.21.15.69:0/2485577221 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc2ac0cf260 con 0x7fc2bc074280 2024-03-19T01:07:34.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.420+0000 7fc2baa00640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc2a00a2880 0x7fc2a00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:34.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.420+0000 7fc2baa00640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc2a00a2880 0x7fc2a00a4d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4036040 tx=0x7fc2a402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:34.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.423+0000 7fc2b3e00640 1 -- 172.21.15.69:0/2485577221 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc2ac09ba10 con 0x7fc2bc074280 2024-03-19T01:07:34.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.626+0000 7fc2c2800640 1 -- 172.21.15.69:0/2485577221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc2bc062450 con 0x7fc2bc074280 2024-03-19T01:07:34.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.627+0000 7fc2b3e00640 1 -- 172.21.15.69:0/2485577221 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7fc2ac09fab0 con 0x7fc2bc074280 2024-03-19T01:07:34.628 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:07:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 -- 172.21.15.69:0/2485577221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc2a00a2880 msgr2=0x7fc2a00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc2a00a2880 0x7fc2a00a4d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4036040 tx=0x7fc2a402df90 comp rx=0 tx=0).stop 2024-03-19T01:07:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 -- 172.21.15.69:0/2485577221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc074280 msgr2=0x7fc2bc19e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc074280 0x7fc2bc19e190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00ebc0 tx=0x7fc2ac00c6a0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 -- 172.21.15.69:0/2485577221 shutdown_connections 2024-03-19T01:07:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc2a00a2880 0x7fc2a00a4d40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc10de00 0x7fc2bc1a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2bc075860 0x7fc2bc19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 --2- 172.21.15.69:0/2485577221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc074280 0x7fc2bc19e190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 -- 172.21.15.69:0/2485577221 >> 172.21.15.69:0/2485577221 conn(0x7fc2bc0fcda0 msgr2=0x7fc2bc1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:34.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 -- 172.21.15.69:0/2485577221 shutdown_connections 2024-03-19T01:07:34.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.630+0000 7fc2c2800640 1 -- 172.21.15.69:0/2485577221 wait complete. 2024-03-19T01:07:34.651 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-19T01:07:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.854+0000 7f04a5a00640 1 -- 172.21.15.69:0/269939844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04a01073a0 msgr2=0x7f04a0109790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.854+0000 7f04a5a00640 1 --2- 172.21.15.69:0/269939844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04a01073a0 0x7f04a0109790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0498009ca0 tx=0x7f049802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.854+0000 7f04a5a00640 1 -- 172.21.15.69:0/269939844 shutdown_connections 2024-03-19T01:07:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.854+0000 7f04a5a00640 1 --2- 172.21.15.69:0/269939844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04a01073a0 0x7f04a0109790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.854+0000 7f04a5a00640 1 --2- 172.21.15.69:0/269939844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04a0104a70 0x7f04a0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.854+0000 7f04a5a00640 1 --2- 172.21.15.69:0/269939844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a0102140 0x7f04a0104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.854+0000 7f04a5a00640 1 -- 172.21.15.69:0/269939844 >> 172.21.15.69:0/269939844 conn(0x7f04a00fba50 msgr2=0x7f04a00fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.855+0000 7f04a5a00640 1 -- 172.21.15.69:0/269939844 shutdown_connections 2024-03-19T01:07:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.855+0000 7f04a5a00640 1 -- 172.21.15.69:0/269939844 wait complete. 2024-03-19T01:07:34.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.855+0000 7f04a5a00640 1 Processor -- start 2024-03-19T01:07:34.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f04a5a00640 1 -- start start 2024-03-19T01:07:34.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f04a5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04a0102140 0x7f04a0199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f04a5a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a0104a70 0x7f04a019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f04a5a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04a01073a0 0x7f04a01a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f04a5a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010d2a0 con 0x7f04a01073a0 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f04a5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010d120 con 0x7f04a0102140 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f04a5a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f04a010d420 con 0x7f04a0104a70 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f049fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04a0102140 0x7f04a0199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f049fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04a0102140 0x7f04a0199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:41804/0 (socket says 172.21.15.69:41804) 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f049fe00640 1 -- 172.21.15.69:0/2668986718 learned_addr learned my addr 172.21.15.69:0/2668986718 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f04a5000640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04a01073a0 0x7f04a01a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.856+0000 7f049f400640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a0104a70 0x7f04a019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f049fe00640 1 -- 172.21.15.69:0/2668986718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a0104a70 msgr2=0x7f04a019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f049fe00640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a0104a70 0x7f04a019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f049fe00640 1 -- 172.21.15.69:0/2668986718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04a01073a0 msgr2=0x7f04a01a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f049fe00640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04a01073a0 0x7f04a01a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f049fe00640 1 -- 172.21.15.69:0/2668986718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a01a1710 con 0x7f04a0102140 2024-03-19T01:07:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f049f400640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a0104a70 0x7f04a019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f049fe00640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04a0102140 0x7f04a0199af0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f049000e9f0 tx=0x7f049000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f049cc00640 1 -- 172.21.15.69:0/2668986718 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049000cd80 con 0x7f04a0102140 2024-03-19T01:07:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f049cc00640 1 -- 172.21.15.69:0/2668986718 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0490010910 con 0x7f04a0102140 2024-03-19T01:07:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f04a5a00640 1 -- 172.21.15.69:0/2668986718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a01a1a00 con 0x7f04a0102140 2024-03-19T01:07:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f049cc00640 1 -- 172.21.15.69:0/2668986718 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490018990 con 0x7f04a0102140 2024-03-19T01:07:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.857+0000 7f04a5a00640 1 -- 172.21.15.69:0/2668986718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a01a1f50 con 0x7f04a0102140 2024-03-19T01:07:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.858+0000 7f04a5a00640 1 -- 172.21.15.69:0/2668986718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a0068530 con 0x7f04a0102140 2024-03-19T01:07:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.860+0000 7f049cc00640 1 -- 172.21.15.69:0/2668986718 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0490010ac0 con 0x7f04a0102140 2024-03-19T01:07:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.861+0000 7f049cc00640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f04840a2800 0x7f04840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.862+0000 7f049cc00640 1 -- 172.21.15.69:0/2668986718 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f04900cf0a0 con 0x7f04a0102140 2024-03-19T01:07:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.862+0000 7f049f400640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f04840a2800 0x7f04840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.862+0000 7f049f400640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f04840a2800 0x7f04840a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f04a019b000 tx=0x7f048c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:34.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:34.865+0000 7f049cc00640 1 -- 172.21.15.69:0/2668986718 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f049009bf00 con 0x7f04a0102140 2024-03-19T01:07:35.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:34 smithi086 ceph-mon[66829]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-19T01:07:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:34 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1567376130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:07:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:34 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1282707973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:07:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:34 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2485577221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:07:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:34 smithi069 ceph-mon[31727]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-19T01:07:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:34 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1567376130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:07:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:34 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1282707973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:07:35.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:34 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2485577221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:07:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:34 smithi163 ceph-mon[67428]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-19T01:07:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:34 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1567376130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:07:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:34 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1282707973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:07:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:34 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2485577221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:07:35.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.076+0000 7f04a5a00640 1 -- 172.21.15.69:0/2668986718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f04a0062420 con 0x7f04a0102140 2024-03-19T01:07:35.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.076+0000 7f049cc00640 1 -- 172.21.15.69:0/2668986718 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+591 (secure 0 0 0) 0x7f049009b800 con 0x7f04a0102140 2024-03-19T01:07:35.077 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/4073584559 2024-03-20T00:59:23.503145+0000 2024-03-19T01:07:35.078 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/4073584559 2024-03-20T00:59:23.503145+0000 2024-03-19T01:07:35.078 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2580938130 2024-03-20T00:59:23.503145+0000 2024-03-19T01:07:35.078 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/2422305615 2024-03-20T00:58:49.440150+0000 2024-03-19T01:07:35.078 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4228526145 2024-03-20T00:58:49.440150+0000 2024-03-19T01:07:35.078 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/697036895 2024-03-20T00:58:49.440150+0000 2024-03-19T01:07:35.078 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/2422305615 2024-03-20T00:58:49.440150+0000 2024-03-19T01:07:35.078 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1824937012 2024-03-20T00:59:23.503145+0000 2024-03-19T01:07:35.078 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3077206365 2024-03-20T00:58:49.440150+0000 2024-03-19T01:07:35.078 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2948479024 2024-03-20T00:59:23.503145+0000 2024-03-19T01:07:35.078 INFO:teuthology.orchestra.run.smithi069.stderr:listed 10 entries 2024-03-19T01:07:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.079+0000 7f04a5a00640 1 -- 172.21.15.69:0/2668986718 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f04840a2800 msgr2=0x7f04840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.079+0000 7f04a5a00640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f04840a2800 0x7f04840a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f04a019b000 tx=0x7f048c005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.079+0000 7f04a5a00640 1 -- 172.21.15.69:0/2668986718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04a0102140 msgr2=0x7f04a0199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.079+0000 7f04a5a00640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04a0102140 0x7f04a0199af0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f049000e9f0 tx=0x7f049000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.079+0000 7f04a5a00640 1 -- 172.21.15.69:0/2668986718 shutdown_connections 2024-03-19T01:07:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.079+0000 7f04a5a00640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f04840a2800 0x7f04840a4cc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.079+0000 7f04a5a00640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04a01073a0 0x7f04a01a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.079+0000 7f04a5a00640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a0104a70 0x7f04a019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.079+0000 7f04a5a00640 1 --2- 172.21.15.69:0/2668986718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04a0102140 0x7f04a0199af0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.079+0000 7f04a5a00640 1 -- 172.21.15.69:0/2668986718 >> 172.21.15.69:0/2668986718 conn(0x7f04a00fba50 msgr2=0x7f04a0101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.080+0000 7f04a5a00640 1 -- 172.21.15.69:0/2668986718 shutdown_connections 2024-03-19T01:07:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.080+0000 7f04a5a00640 1 -- 172.21.15.69:0/2668986718 wait complete. 2024-03-19T01:07:35.098 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T01:07:35.098 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T01:07:35.146 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:07:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.401+0000 7f27fcc00640 1 -- 172.21.15.69:0/3685569073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f8108fb0 msgr2=0x7f27f8109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.401+0000 7f27fcc00640 1 --2- 172.21.15.69:0/3685569073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f8108fb0 0x7f27f8109430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009900 tx=0x7f27e002bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.401+0000 7f27fcc00640 1 -- 172.21.15.69:0/3685569073 shutdown_connections 2024-03-19T01:07:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.401+0000 7f27fcc00640 1 --2- 172.21.15.69:0/3685569073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27f8109970 0x7f27f8110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.401+0000 7f27fcc00640 1 --2- 172.21.15.69:0/3685569073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f8108fb0 0x7f27f8109430 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.401+0000 7f27fcc00640 1 --2- 172.21.15.69:0/3685569073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27f8107e60 0x7f27f8108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.401+0000 7f27fcc00640 1 -- 172.21.15.69:0/3685569073 >> 172.21.15.69:0/3685569073 conn(0x7f27f8076e30 msgr2=0x7f27f8079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.401+0000 7f27fcc00640 1 -- 172.21.15.69:0/3685569073 shutdown_connections 2024-03-19T01:07:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.401+0000 7f27fcc00640 1 -- 172.21.15.69:0/3685569073 wait complete. 2024-03-19T01:07:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.402+0000 7f27fcc00640 1 Processor -- start 2024-03-19T01:07:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27fcc00640 1 -- start start 2024-03-19T01:07:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27fcc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27f8107e60 0x7f27f819e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27fcc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27f8108fb0 0x7f27f819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27fcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f8109970 0x7f27f81a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27fcc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8113b30 con 0x7f27f8107e60 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27fcc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27f81139b0 con 0x7f27f8109970 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27fcc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8113cb0 con 0x7f27f8108fb0 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27f5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27f8107e60 0x7f27f819e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27f6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f8109970 0x7f27f81a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27f5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27f8107e60 0x7f27f819e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50342/0 (socket says 172.21.15.69:50342) 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27f5e00640 1 -- 172.21.15.69:0/1260524985 learned_addr learned my addr 172.21.15.69:0/1260524985 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.403+0000 7f27ed400640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27f8108fb0 0x7f27f819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.404+0000 7f27f6800640 1 -- 172.21.15.69:0/1260524985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27f8108fb0 msgr2=0x7f27f819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.404+0000 7f27f6800640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27f8108fb0 0x7f27f819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.404+0000 7f27f6800640 1 -- 172.21.15.69:0/1260524985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27f8107e60 msgr2=0x7f27f819e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.404+0000 7f27f6800640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27f8107e60 0x7f27f819e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.404+0000 7f27f6800640 1 -- 172.21.15.69:0/1260524985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f81a5d30 con 0x7f27f8109970 2024-03-19T01:07:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.404+0000 7f27f6800640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f8109970 0x7f27f81a56c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f27f000e940 tx=0x7f27f000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.404+0000 7f27ef400640 1 -- 172.21.15.69:0/1260524985 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0004d20 con 0x7f27f8109970 2024-03-19T01:07:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.404+0000 7f27ef400640 1 -- 172.21.15.69:0/1260524985 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f0005020 con 0x7f27f8109970 2024-03-19T01:07:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.404+0000 7f27ef400640 1 -- 172.21.15.69:0/1260524985 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00116b0 con 0x7f27f8109970 2024-03-19T01:07:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.404+0000 7f27fcc00640 1 -- 172.21.15.69:0/1260524985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f81a5fc0 con 0x7f27f8109970 2024-03-19T01:07:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.404+0000 7f27fcc00640 1 -- 172.21.15.69:0/1260524985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f81a6490 con 0x7f27f8109970 2024-03-19T01:07:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.405+0000 7f27fcc00640 1 -- 172.21.15.69:0/1260524985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f810f110 con 0x7f27f8109970 2024-03-19T01:07:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.407+0000 7f27ef400640 1 -- 172.21.15.69:0/1260524985 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f27f00054d0 con 0x7f27f8109970 2024-03-19T01:07:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.408+0000 7f27ef400640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27dc0a2800 0x7f27dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.409+0000 7f27ef400640 1 -- 172.21.15.69:0/1260524985 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f27f00cecc0 con 0x7f27f8109970 2024-03-19T01:07:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.409+0000 7f27f5e00640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27dc0a2800 0x7f27dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:35.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.409+0000 7f27f5e00640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27dc0a2800 0x7f27dc0a4cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27e8005fd0 tx=0x7f27e800b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:35.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.413+0000 7f27ef400640 1 -- 172.21.15.69:0/1260524985 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f27f00d5080 con 0x7f27f8109970 2024-03-19T01:07:35.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.612+0000 7f27fcc00640 1 -- 172.21.15.69:0/1260524985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27f8062420 con 0x7f27f8109970 2024-03-19T01:07:35.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.614+0000 7f27ef400640 1 -- 172.21.15.69:0/1260524985 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24069 (secure 0 0 0) 0x7f27f0016030 con 0x7f27f8109970 2024-03-19T01:07:35.616 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:35.616 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":89,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:07:20.123357+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.616+0000 7f27fcc00640 1 -- 172.21.15.69:0/1260524985 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27dc0a2800 msgr2=0x7f27dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.616+0000 7f27fcc00640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27dc0a2800 0x7f27dc0a4cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27e8005fd0 tx=0x7f27e800b4a0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.617+0000 7f27fcc00640 1 -- 172.21.15.69:0/1260524985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f8109970 msgr2=0x7f27f81a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.617+0000 7f27fcc00640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f8109970 0x7f27f81a56c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f27f000e940 tx=0x7f27f000ee00 comp rx=0 tx=0).stop 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.617+0000 7f27fcc00640 1 -- 172.21.15.69:0/1260524985 shutdown_connections 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.617+0000 7f27fcc00640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27dc0a2800 0x7f27dc0a4cc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.617+0000 7f27fcc00640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27f8109970 0x7f27f81a56c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.617+0000 7f27fcc00640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27f8108fb0 0x7f27f819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.617+0000 7f27fcc00640 1 --2- 172.21.15.69:0/1260524985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27f8107e60 0x7f27f819e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.617+0000 7f27fcc00640 1 -- 172.21.15.69:0/1260524985 >> 172.21.15.69:0/1260524985 conn(0x7f27f8076e30 msgr2=0x7f27f810c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.617+0000 7f27fcc00640 1 -- 172.21.15.69:0/1260524985 shutdown_connections 2024-03-19T01:07:35.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.617+0000 7f27fcc00640 1 -- 172.21.15.69:0/1260524985 wait complete. 2024-03-19T01:07:35.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:07:13.368130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:07:14.444206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T01:07:18.189146+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T01:07:35.638 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:07:35.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.852+0000 7fa23b200640 1 -- 172.21.15.69:0/4287273458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23c108f30 msgr2=0x7fa23c1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:35.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.852+0000 7fa23b200640 1 --2- 172.21.15.69:0/4287273458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23c108f30 0x7fa23c1093b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa224009900 tx=0x7fa22402bc40 comp rx=0 tx=0).stop 2024-03-19T01:07:35.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.853+0000 7fa23b200640 1 -- 172.21.15.69:0/4287273458 shutdown_connections 2024-03-19T01:07:35.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.853+0000 7fa23b200640 1 --2- 172.21.15.69:0/4287273458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa23c1098f0 0x7fa23c110180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.853+0000 7fa23b200640 1 --2- 172.21.15.69:0/4287273458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23c108f30 0x7fa23c1093b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.853+0000 7fa23b200640 1 --2- 172.21.15.69:0/4287273458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c107d40 0x7fa23c108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.853+0000 7fa23b200640 1 -- 172.21.15.69:0/4287273458 >> 172.21.15.69:0/4287273458 conn(0x7fa23c076cf0 msgr2=0x7fa23c079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:35.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.853+0000 7fa23b200640 1 -- 172.21.15.69:0/4287273458 shutdown_connections 2024-03-19T01:07:35.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.853+0000 7fa23b200640 1 -- 172.21.15.69:0/4287273458 wait complete. 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.854+0000 7fa23b200640 1 Processor -- start 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.854+0000 7fa23b200640 1 -- start start 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.854+0000 7fa23b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c107d40 0x7fa23c19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.854+0000 7fa23b200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23c108f30 0x7fa23c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.854+0000 7fa23b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa23c1098f0 0x7fa23c1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.854+0000 7fa23b200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c113a10 con 0x7fa23c108f30 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.854+0000 7fa23b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c113890 con 0x7fa23c107d40 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.854+0000 7fa23b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c113b90 con 0x7fa23c1098f0 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa239e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c107d40 0x7fa23c19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa239400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23c108f30 0x7fa23c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa239e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c107d40 0x7fa23c19e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44168/0 (socket says 172.21.15.69:44168) 2024-03-19T01:07:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa239e00640 1 -- 172.21.15.69:0/1128420372 learned_addr learned my addr 172.21.15.69:0/1128420372 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa23a800640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa23c1098f0 0x7fa23c1a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa239e00640 1 -- 172.21.15.69:0/1128420372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa23c1098f0 msgr2=0x7fa23c1a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa239e00640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa23c1098f0 0x7fa23c1a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa239e00640 1 -- 172.21.15.69:0/1128420372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23c108f30 msgr2=0x7fa23c19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa239e00640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23c108f30 0x7fa23c19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa239e00640 1 -- 172.21.15.69:0/1128420372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23c1a5c90 con 0x7fa23c107d40 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa239400640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23c108f30 0x7fa23c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.855+0000 7fa239e00640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c107d40 0x7fa23c19e070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00ebc0 tx=0x7fa22c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.856+0000 7fa232a00640 1 -- 172.21.15.69:0/1128420372 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c003b70 con 0x7fa23c107d40 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.856+0000 7fa232a00640 1 -- 172.21.15.69:0/1128420372 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa22c003d10 con 0x7fa23c107d40 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.856+0000 7fa23b200640 1 -- 172.21.15.69:0/1128420372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23c1a5f80 con 0x7fa23c107d40 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.856+0000 7fa232a00640 1 -- 172.21.15.69:0/1128420372 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c019980 con 0x7fa23c107d40 2024-03-19T01:07:35.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.856+0000 7fa23b200640 1 -- 172.21.15.69:0/1128420372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa23c1a63e0 con 0x7fa23c107d40 2024-03-19T01:07:35.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.857+0000 7fa23b200640 1 -- 172.21.15.69:0/1128420372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa23c10f000 con 0x7fa23c107d40 2024-03-19T01:07:35.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.859+0000 7fa232a00640 1 -- 172.21.15.69:0/1128420372 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa22c019b20 con 0x7fa23c107d40 2024-03-19T01:07:35.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.860+0000 7fa232a00640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa21c0a2800 0x7fa21c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:35.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.861+0000 7fa232a00640 1 -- 172.21.15.69:0/1128420372 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa22c0cf450 con 0x7fa23c107d40 2024-03-19T01:07:35.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.861+0000 7fa239400640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa21c0a2800 0x7fa21c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:35.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.861+0000 7fa239400640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa21c0a2800 0x7fa21c0a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa22402efd0 tx=0x7fa22402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:35.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:35.869+0000 7fa232a00640 1 -- 172.21.15.69:0/1128420372 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa22c09c2b0 con 0x7fa23c107d40 2024-03-19T01:07:36.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:35 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2668986718' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T01:07:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:35 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1260524985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:36.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:35 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2668986718' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T01:07:36.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:35 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1260524985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:36.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:35 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2668986718' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T01:07:36.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:35 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1260524985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:36.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.066+0000 7fa23b200640 1 -- 172.21.15.69:0/1128420372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa23c0624c0 con 0x7fa23c107d40 2024-03-19T01:07:36.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.067+0000 7fa232a00640 1 -- 172.21.15.69:0/1128420372 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fa22c09bbb0 con 0x7fa23c107d40 2024-03-19T01:07:36.068 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:07:36.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 -- 172.21.15.69:0/1128420372 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa21c0a2800 msgr2=0x7fa21c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa21c0a2800 0x7fa21c0a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa22402efd0 tx=0x7fa22402cf90 comp rx=0 tx=0).stop 2024-03-19T01:07:36.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 -- 172.21.15.69:0/1128420372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c107d40 msgr2=0x7fa23c19e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c107d40 0x7fa23c19e070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00ebc0 tx=0x7fa22c00c6a0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 -- 172.21.15.69:0/1128420372 shutdown_connections 2024-03-19T01:07:36.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa21c0a2800 0x7fa21c0a4cc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa23c1098f0 0x7fa23c1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23c108f30 0x7fa23c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 --2- 172.21.15.69:0/1128420372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c107d40 0x7fa23c19e070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 -- 172.21.15.69:0/1128420372 >> 172.21.15.69:0/1128420372 conn(0x7fa23c076cf0 msgr2=0x7fa23c10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:36.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 -- 172.21.15.69:0/1128420372 shutdown_connections 2024-03-19T01:07:36.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.070+0000 7fa23b200640 1 -- 172.21.15.69:0/1128420372 wait complete. 2024-03-19T01:07:36.089 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:07:36.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.308+0000 7f61e2000640 1 -- 172.21.15.69:0/505376845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61dc102a20 msgr2=0x7f61dc102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.308+0000 7f61e2000640 1 --2- 172.21.15.69:0/505376845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61dc102a20 0x7f61dc102ea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009ca0 tx=0x7f61c402c100 comp rx=0 tx=0).stop 2024-03-19T01:07:36.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.308+0000 7f61e2000640 1 -- 172.21.15.69:0/505376845 shutdown_connections 2024-03-19T01:07:36.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.308+0000 7f61e2000640 1 --2- 172.21.15.69:0/505376845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61dc1033e0 0x7f61dc109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.308+0000 7f61e2000640 1 --2- 172.21.15.69:0/505376845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61dc102a20 0x7f61dc102ea0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.308+0000 7f61e2000640 1 --2- 172.21.15.69:0/505376845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc101830 0x7f61dc101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.308+0000 7f61e2000640 1 -- 172.21.15.69:0/505376845 >> 172.21.15.69:0/505376845 conn(0x7f61dc0fcdc0 msgr2=0x7f61dc0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.308+0000 7f61e2000640 1 -- 172.21.15.69:0/505376845 shutdown_connections 2024-03-19T01:07:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.308+0000 7f61e2000640 1 -- 172.21.15.69:0/505376845 wait complete. 2024-03-19T01:07:36.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.309+0000 7f61e2000640 1 Processor -- start 2024-03-19T01:07:36.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.309+0000 7f61e2000640 1 -- start start 2024-03-19T01:07:36.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.309+0000 7f61e2000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61dc101830 0x7f61dc197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:36.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.309+0000 7f61e2000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61dc102a20 0x7f61dc1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:36.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.309+0000 7f61e2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc1033e0 0x7f61dc19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.309+0000 7f61e2000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10d490 con 0x7f61dc102a20 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.309+0000 7f61e2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10d310 con 0x7f61dc1033e0 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.309+0000 7f61e2000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10d610 con 0x7f61dc101830 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc1033e0 0x7f61dc19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc1033e0 0x7f61dc19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44180/0 (socket says 172.21.15.69:44180) 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61dbe00640 1 -- 172.21.15.69:0/2234519110 learned_addr learned my addr 172.21.15.69:0/2234519110 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61db400640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61dc101830 0x7f61dc197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61daa00640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61dc102a20 0x7f61dc1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61dbe00640 1 -- 172.21.15.69:0/2234519110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61dc101830 msgr2=0x7f61dc197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61dbe00640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61dc101830 0x7f61dc197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61dbe00640 1 -- 172.21.15.69:0/2234519110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61dc102a20 msgr2=0x7f61dc1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61dbe00640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61dc102a20 0x7f61dc1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61dbe00640 1 -- 172.21.15.69:0/2234519110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc19f780 con 0x7f61dc1033e0 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61dbe00640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc1033e0 0x7f61dc19f110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00d750 tx=0x7f61cc00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:36.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61d3e00640 1 -- 172.21.15.69:0/2234519110 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc017070 con 0x7f61dc1033e0 2024-03-19T01:07:36.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.310+0000 7f61d3e00640 1 -- 172.21.15.69:0/2234519110 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61cc005500 con 0x7f61dc1033e0 2024-03-19T01:07:36.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.311+0000 7f61e2000640 1 -- 172.21.15.69:0/2234519110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc19fa70 con 0x7f61dc1033e0 2024-03-19T01:07:36.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.311+0000 7f61d3e00640 1 -- 172.21.15.69:0/2234519110 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc004d90 con 0x7f61dc1033e0 2024-03-19T01:07:36.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.311+0000 7f61e2000640 1 -- 172.21.15.69:0/2234519110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc19ff90 con 0x7f61dc1033e0 2024-03-19T01:07:36.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.312+0000 7f61e2000640 1 -- 172.21.15.69:0/2234519110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61dc108bd0 con 0x7f61dc1033e0 2024-03-19T01:07:36.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.314+0000 7f61d3e00640 1 -- 172.21.15.69:0/2234519110 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f61cc004f30 con 0x7f61dc1033e0 2024-03-19T01:07:36.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.315+0000 7f61d3e00640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61c00a2880 0x7f61c00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:36.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.315+0000 7f61d3e00640 1 -- 172.21.15.69:0/2234519110 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f61cc0ce5f0 con 0x7f61dc1033e0 2024-03-19T01:07:36.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.315+0000 7f61db400640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61c00a2880 0x7f61c00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:36.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.316+0000 7f61db400640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61c00a2880 0x7f61c00a4d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f61d40095f0 tx=0x7f61d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:36.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.319+0000 7f61d3e00640 1 -- 172.21.15.69:0/2234519110 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f61cc09b450 con 0x7f61dc1033e0 2024-03-19T01:07:36.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.518+0000 7f61e2000640 1 -- 172.21.15.69:0/2234519110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f61dc062420 con 0x7f61dc1033e0 2024-03-19T01:07:36.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.518+0000 7f61d3e00640 1 -- 172.21.15.69:0/2234519110 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f61cc09ad50 con 0x7f61dc1033e0 2024-03-19T01:07:36.519 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:07:36.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 -- 172.21.15.69:0/2234519110 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61c00a2880 msgr2=0x7f61c00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61c00a2880 0x7f61c00a4d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f61d40095f0 tx=0x7f61d4005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:36.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 -- 172.21.15.69:0/2234519110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc1033e0 msgr2=0x7f61dc19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc1033e0 0x7f61dc19f110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00d750 tx=0x7f61cc00dc10 comp rx=0 tx=0).stop 2024-03-19T01:07:36.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 -- 172.21.15.69:0/2234519110 shutdown_connections 2024-03-19T01:07:36.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61c00a2880 0x7f61c00a4d40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61dc1033e0 0x7f61dc19f110 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61dc102a20 0x7f61dc1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 --2- 172.21.15.69:0/2234519110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61dc101830 0x7f61dc197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 -- 172.21.15.69:0/2234519110 >> 172.21.15.69:0/2234519110 conn(0x7f61dc0fcdc0 msgr2=0x7f61dc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:36.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 -- 172.21.15.69:0/2234519110 shutdown_connections 2024-03-19T01:07:36.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.521+0000 7f61e2000640 1 -- 172.21.15.69:0/2234519110 wait complete. 2024-03-19T01:07:36.542 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:07:36.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.761+0000 7f1449800640 1 -- 172.21.15.69:0/908010405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1444075860 msgr2=0x7f1444073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.761+0000 7f1449800640 1 --2- 172.21.15.69:0/908010405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1444075860 0x7f1444073cb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1434009930 tx=0x7f143402bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:36.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.761+0000 7f1449800640 1 -- 172.21.15.69:0/908010405 shutdown_connections 2024-03-19T01:07:36.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.761+0000 7f1449800640 1 --2- 172.21.15.69:0/908010405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144410de00 0x7f14441101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.761+0000 7f1449800640 1 --2- 172.21.15.69:0/908010405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1444074280 0x7f1444074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.761+0000 7f1449800640 1 --2- 172.21.15.69:0/908010405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1444075860 0x7f1444073cb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.761+0000 7f1449800640 1 -- 172.21.15.69:0/908010405 >> 172.21.15.69:0/908010405 conn(0x7f14440fcdc0 msgr2=0x7f14440ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:36.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.761+0000 7f1449800640 1 -- 172.21.15.69:0/908010405 shutdown_connections 2024-03-19T01:07:36.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.761+0000 7f1449800640 1 -- 172.21.15.69:0/908010405 wait complete. 2024-03-19T01:07:36.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.762+0000 7f1449800640 1 Processor -- start 2024-03-19T01:07:36.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f1449800640 1 -- start start 2024-03-19T01:07:36.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f1449800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1444074280 0x7f144419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:36.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f1449800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1444075860 0x7f144419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:36.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f1449800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144410de00 0x7f14441a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f1449800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1444113af0 con 0x7f1444075860 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f1449800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1444113970 con 0x7f144410de00 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f1449800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1444113c70 con 0x7f1444074280 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f1443400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144410de00 0x7f14441a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f143a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1444075860 0x7f144419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f1443400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144410de00 0x7f14441a5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44198/0 (socket says 172.21.15.69:44198) 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f1443400640 1 -- 172.21.15.69:0/1753960500 learned_addr learned my addr 172.21.15.69:0/1753960500 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.763+0000 7f1442a00640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1444074280 0x7f144419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f1443400640 1 -- 172.21.15.69:0/1753960500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1444074280 msgr2=0x7f144419e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f1443400640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1444074280 0x7f144419e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f1443400640 1 -- 172.21.15.69:0/1753960500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1444075860 msgr2=0x7f144419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f1443400640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1444075860 0x7f144419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f1443400640 1 -- 172.21.15.69:0/1753960500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14441a5db0 con 0x7f144410de00 2024-03-19T01:07:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f143a000640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1444075860 0x7f144419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f1443400640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144410de00 0x7f14441a5740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f143c00eb10 tx=0x7f143c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f143be00640 1 -- 172.21.15.69:0/1753960500 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c004b20 con 0x7f144410de00 2024-03-19T01:07:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f143be00640 1 -- 172.21.15.69:0/1753960500 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c004cc0 con 0x7f144410de00 2024-03-19T01:07:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f1449800640 1 -- 172.21.15.69:0/1753960500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14441a60a0 con 0x7f144410de00 2024-03-19T01:07:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f143be00640 1 -- 172.21.15.69:0/1753960500 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c010760 con 0x7f144410de00 2024-03-19T01:07:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.764+0000 7f1449800640 1 -- 172.21.15.69:0/1753960500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14441a65f0 con 0x7f144410de00 2024-03-19T01:07:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.765+0000 7f1449800640 1 -- 172.21.15.69:0/1753960500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1444103fd0 con 0x7f144410de00 2024-03-19T01:07:36.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.767+0000 7f143be00640 1 -- 172.21.15.69:0/1753960500 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f143c005020 con 0x7f144410de00 2024-03-19T01:07:36.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.768+0000 7f143be00640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2800 0x7f14280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:36.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.769+0000 7f143be00640 1 -- 172.21.15.69:0/1753960500 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f143c0cf3e0 con 0x7f144410de00 2024-03-19T01:07:36.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.769+0000 7f1442a00640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2800 0x7f14280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:36.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.769+0000 7f1442a00640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2800 0x7f14280a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1434036040 tx=0x7f143402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:36.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.772+0000 7f143be00640 1 -- 172.21.15.69:0/1753960500 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f143c09c1e0 con 0x7f144410de00 2024-03-19T01:07:36.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.971+0000 7f1449800640 1 -- 172.21.15.69:0/1753960500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1444062420 con 0x7f144410de00 2024-03-19T01:07:36.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.972+0000 7f143be00640 1 -- 172.21.15.69:0/1753960500 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f143c09bae0 con 0x7f144410de00 2024-03-19T01:07:36.973 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-03-19T01:07:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 -- 172.21.15.69:0/1753960500 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2800 msgr2=0x7f14280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2800 0x7f14280a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1434036040 tx=0x7f143402df90 comp rx=0 tx=0).stop 2024-03-19T01:07:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 -- 172.21.15.69:0/1753960500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144410de00 msgr2=0x7f14441a5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144410de00 0x7f14441a5740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f143c00eb10 tx=0x7f143c00c2d0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 -- 172.21.15.69:0/1753960500 shutdown_connections 2024-03-19T01:07:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f14280a2800 0x7f14280a4cc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f144410de00 0x7f14441a5740 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1444075860 0x7f144419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 --2- 172.21.15.69:0/1753960500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1444074280 0x7f144419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:36.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 -- 172.21.15.69:0/1753960500 >> 172.21.15.69:0/1753960500 conn(0x7f14440fcdc0 msgr2=0x7f14441011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:36.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 -- 172.21.15.69:0/1753960500 shutdown_connections 2024-03-19T01:07:36.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:36.975+0000 7f1449800640 1 -- 172.21.15.69:0/1753960500 wait complete. 2024-03-19T01:07:36.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:36 smithi069 ceph-mon[31727]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-19T01:07:36.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:36 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1128420372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:07:36.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:36 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2234519110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:07:36.996 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T01:07:37.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:36 smithi086 ceph-mon[66829]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-19T01:07:37.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:36 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1128420372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:07:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:36 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2234519110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:07:37.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:36 smithi163 ceph-mon[67428]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-19T01:07:37.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:36 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1128420372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:07:37.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:36 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2234519110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:07:37.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.205+0000 7f832a800640 1 -- 172.21.15.69:0/4260492022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324075860 msgr2=0x7f8324073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:37.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.205+0000 7f832a800640 1 --2- 172.21.15.69:0/4260492022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324075860 0x7f8324073cb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8314009930 tx=0x7f831402bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:37.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.206+0000 7f832a800640 1 -- 172.21.15.69:0/4260492022 shutdown_connections 2024-03-19T01:07:37.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.206+0000 7f832a800640 1 --2- 172.21.15.69:0/4260492022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f832410de00 0x7f83241101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.206+0000 7f832a800640 1 --2- 172.21.15.69:0/4260492022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8324074280 0x7f8324074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.206+0000 7f832a800640 1 --2- 172.21.15.69:0/4260492022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324075860 0x7f8324073cb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.206+0000 7f832a800640 1 -- 172.21.15.69:0/4260492022 >> 172.21.15.69:0/4260492022 conn(0x7f83240fcda0 msgr2=0x7f83240ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:37.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.206+0000 7f832a800640 1 -- 172.21.15.69:0/4260492022 shutdown_connections 2024-03-19T01:07:37.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.206+0000 7f832a800640 1 -- 172.21.15.69:0/4260492022 wait complete. 2024-03-19T01:07:37.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f832a800640 1 Processor -- start 2024-03-19T01:07:37.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f832a800640 1 -- start start 2024-03-19T01:07:37.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f832a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8324074280 0x7f832419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:37.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f832a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324075860 0x7f832419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f832a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f832410de00 0x7f83241a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f832a800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8324113af0 con 0x7f8324074280 2024-03-19T01:07:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f832a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8324113970 con 0x7f8324075860 2024-03-19T01:07:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f832a800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8324113c70 con 0x7f832410de00 2024-03-19T01:07:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f8322a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324075860 0x7f832419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f8323400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8324074280 0x7f832419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f8322a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324075860 0x7f832419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44216/0 (socket says 172.21.15.69:44216) 2024-03-19T01:07:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f8322a00640 1 -- 172.21.15.69:0/1939163651 learned_addr learned my addr 172.21.15.69:0/1939163651 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.207+0000 7f8323e00640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f832410de00 0x7f83241a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f8322a00640 1 -- 172.21.15.69:0/1939163651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f832410de00 msgr2=0x7f83241a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f8322a00640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f832410de00 0x7f83241a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f8322a00640 1 -- 172.21.15.69:0/1939163651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8324074280 msgr2=0x7f832419e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f8322a00640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8324074280 0x7f832419e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f8322a00640 1 -- 172.21.15.69:0/1939163651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83241a5db0 con 0x7f8324075860 2024-03-19T01:07:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f8323e00640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f832410de00 0x7f83241a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:07:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f8322a00640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324075860 0x7f832419e6d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f831000e9f0 tx=0x7f831000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f831be00640 1 -- 172.21.15.69:0/1939163651 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831000cd80 con 0x7f8324075860 2024-03-19T01:07:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f831be00640 1 -- 172.21.15.69:0/1939163651 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8310010910 con 0x7f8324075860 2024-03-19T01:07:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f832a800640 1 -- 172.21.15.69:0/1939163651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83241a60a0 con 0x7f8324075860 2024-03-19T01:07:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f831be00640 1 -- 172.21.15.69:0/1939163651 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310018990 con 0x7f8324075860 2024-03-19T01:07:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.208+0000 7f832a800640 1 -- 172.21.15.69:0/1939163651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83241a6640 con 0x7f8324075860 2024-03-19T01:07:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.209+0000 7f832a800640 1 -- 172.21.15.69:0/1939163651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8324103fd0 con 0x7f8324075860 2024-03-19T01:07:37.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.212+0000 7f831be00640 1 -- 172.21.15.69:0/1939163651 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8310010ac0 con 0x7f8324075860 2024-03-19T01:07:37.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.213+0000 7f831be00640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f83080a2880 0x7f83080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:37.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.213+0000 7f831be00640 1 -- 172.21.15.69:0/1939163651 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f83100cf060 con 0x7f8324075860 2024-03-19T01:07:37.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.213+0000 7f8323400640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f83080a2880 0x7f83080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:37.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.214+0000 7f8323400640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f83080a2880 0x7f83080a4d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8314036040 tx=0x7f831402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:37.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.217+0000 7f831be00640 1 -- 172.21.15.69:0/1939163651 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f831009be60 con 0x7f8324075860 2024-03-19T01:07:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.414+0000 7f832a800640 1 -- 172.21.15.69:0/1939163651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8324062420 con 0x7f8324075860 2024-03-19T01:07:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.414+0000 7f831be00640 1 -- 172.21.15.69:0/1939163651 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f831009b760 con 0x7f8324075860 2024-03-19T01:07:37.416 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:07:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.417+0000 7f832a800640 1 -- 172.21.15.69:0/1939163651 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f83080a2880 msgr2=0x7f83080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.417+0000 7f832a800640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f83080a2880 0x7f83080a4d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8314036040 tx=0x7f831402df90 comp rx=0 tx=0).stop 2024-03-19T01:07:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.417+0000 7f832a800640 1 -- 172.21.15.69:0/1939163651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324075860 msgr2=0x7f832419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.417+0000 7f832a800640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324075860 0x7f832419e6d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f831000e9f0 tx=0x7f831000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.417+0000 7f832a800640 1 -- 172.21.15.69:0/1939163651 shutdown_connections 2024-03-19T01:07:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.417+0000 7f832a800640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f83080a2880 0x7f83080a4d40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.417+0000 7f832a800640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f832410de00 0x7f83241a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.417+0000 7f832a800640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324075860 0x7f832419e6d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.417+0000 7f832a800640 1 --2- 172.21.15.69:0/1939163651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8324074280 0x7f832419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.417+0000 7f832a800640 1 -- 172.21.15.69:0/1939163651 >> 172.21.15.69:0/1939163651 conn(0x7f83240fcda0 msgr2=0x7f83241011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.418+0000 7f832a800640 1 -- 172.21.15.69:0/1939163651 shutdown_connections 2024-03-19T01:07:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.418+0000 7f832a800640 1 -- 172.21.15.69:0/1939163651 wait complete. 2024-03-19T01:07:37.436 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-19T01:07:37.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.645+0000 7f7a93000640 1 -- 172.21.15.69:0/2654113446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c1031e0 msgr2=0x7f7a8c109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:37.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.645+0000 7f7a93000640 1 --2- 172.21.15.69:0/2654113446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c1031e0 0x7f7a8c109a70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009900 tx=0x7f7a8402bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:37.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.645+0000 7f7a93000640 1 -- 172.21.15.69:0/2654113446 shutdown_connections 2024-03-19T01:07:37.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.645+0000 7f7a93000640 1 --2- 172.21.15.69:0/2654113446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c1031e0 0x7f7a8c109a70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.645+0000 7f7a93000640 1 --2- 172.21.15.69:0/2654113446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a8c102820 0x7f7a8c102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.645+0000 7f7a93000640 1 --2- 172.21.15.69:0/2654113446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a8c101630 0x7f7a8c101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.645+0000 7f7a93000640 1 -- 172.21.15.69:0/2654113446 >> 172.21.15.69:0/2654113446 conn(0x7f7a8c0fcb80 msgr2=0x7f7a8c0fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:37.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.645+0000 7f7a93000640 1 -- 172.21.15.69:0/2654113446 shutdown_connections 2024-03-19T01:07:37.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.645+0000 7f7a93000640 1 -- 172.21.15.69:0/2654113446 wait complete. 2024-03-19T01:07:37.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a93000640 1 Processor -- start 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a93000640 1 -- start start 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a93000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c101630 0x7f7a8c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a93000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a8c102820 0x7f7a8c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a93000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a8c1031e0 0x7f7a8c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a93000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c10d300 con 0x7f7a8c102820 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a93000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c10d180 con 0x7f7a8c101630 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a93000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c10d480 con 0x7f7a8c1031e0 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a91c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c101630 0x7f7a8c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a91200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a8c102820 0x7f7a8c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a91c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c101630 0x7f7a8c197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44228/0 (socket says 172.21.15.69:44228) 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.646+0000 7f7a91c00640 1 -- 172.21.15.69:0/163724589 learned_addr learned my addr 172.21.15.69:0/163724589 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:37.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a92600640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a8c1031e0 0x7f7a8c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a91c00640 1 -- 172.21.15.69:0/163724589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a8c1031e0 msgr2=0x7f7a8c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a91c00640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a8c1031e0 0x7f7a8c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a91c00640 1 -- 172.21.15.69:0/163724589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a8c102820 msgr2=0x7f7a8c197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a91c00640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a8c102820 0x7f7a8c197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a91c00640 1 -- 172.21.15.69:0/163724589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c19f570 con 0x7f7a8c101630 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a92600640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a8c1031e0 0x7f7a8c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a91c00640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c101630 0x7f7a8c197950 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00eb70 tx=0x7f7a7c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a8a000640 1 -- 172.21.15.69:0/163724589 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c003b70 con 0x7f7a8c101630 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a8a000640 1 -- 172.21.15.69:0/163724589 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a7c00ee70 con 0x7f7a8c101630 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a8a000640 1 -- 172.21.15.69:0/163724589 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c010830 con 0x7f7a8c101630 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a93000640 1 -- 172.21.15.69:0/163724589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c19f860 con 0x7f7a8c101630 2024-03-19T01:07:37.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.647+0000 7f7a93000640 1 -- 172.21.15.69:0/163724589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c19fd60 con 0x7f7a8c101630 2024-03-19T01:07:37.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.649+0000 7f7a93000640 1 -- 172.21.15.69:0/163724589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c1088f0 con 0x7f7a8c101630 2024-03-19T01:07:37.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.650+0000 7f7a8a000640 1 -- 172.21.15.69:0/163724589 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7a7c004020 con 0x7f7a8c101630 2024-03-19T01:07:37.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.652+0000 7f7a8a000640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a700a2880 0x7f7a700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:37.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.652+0000 7f7a8a000640 1 -- 172.21.15.69:0/163724589 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7a7c0cf3a0 con 0x7f7a8c101630 2024-03-19T01:07:37.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.652+0000 7f7a91200640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a700a2880 0x7f7a700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:37.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.653+0000 7f7a91200640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a700a2880 0x7f7a700a4d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c198e60 tx=0x7f7a78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:37.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.655+0000 7f7a8a000640 1 -- 172.21.15.69:0/163724589 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7a7c09c1a0 con 0x7f7a8c101630 2024-03-19T01:07:37.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.868+0000 7f7a93000640 1 -- 172.21.15.69:0/163724589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7a8c02d1f0 con 0x7f7a8c101630 2024-03-19T01:07:37.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.869+0000 7f7a8a000640 1 -- 172.21.15.69:0/163724589 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+591 (secure 0 0 0) 0x7f7a7c09baa0 con 0x7f7a8c101630 2024-03-19T01:07:37.871 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/4073584559 2024-03-20T00:59:23.503145+0000 2024-03-19T01:07:37.871 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/4073584559 2024-03-20T00:59:23.503145+0000 2024-03-19T01:07:37.871 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2580938130 2024-03-20T00:59:23.503145+0000 2024-03-19T01:07:37.871 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/2422305615 2024-03-20T00:58:49.440150+0000 2024-03-19T01:07:37.871 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4228526145 2024-03-20T00:58:49.440150+0000 2024-03-19T01:07:37.871 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/697036895 2024-03-20T00:58:49.440150+0000 2024-03-19T01:07:37.871 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/2422305615 2024-03-20T00:58:49.440150+0000 2024-03-19T01:07:37.871 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1824937012 2024-03-20T00:59:23.503145+0000 2024-03-19T01:07:37.871 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3077206365 2024-03-20T00:58:49.440150+0000 2024-03-19T01:07:37.871 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2948479024 2024-03-20T00:59:23.503145+0000 2024-03-19T01:07:37.872 INFO:teuthology.orchestra.run.smithi069.stderr:listed 10 entries 2024-03-19T01:07:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 -- 172.21.15.69:0/163724589 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a700a2880 msgr2=0x7f7a700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a700a2880 0x7f7a700a4d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c198e60 tx=0x7f7a78005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 -- 172.21.15.69:0/163724589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c101630 msgr2=0x7f7a8c197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c101630 0x7f7a8c197950 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00eb70 tx=0x7f7a7c00c6a0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 -- 172.21.15.69:0/163724589 shutdown_connections 2024-03-19T01:07:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a700a2880 0x7f7a700a4d40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a8c1031e0 0x7f7a8c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a8c102820 0x7f7a8c197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 --2- 172.21.15.69:0/163724589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c101630 0x7f7a8c197950 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:37.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 -- 172.21.15.69:0/163724589 >> 172.21.15.69:0/163724589 conn(0x7f7a8c0fcb80 msgr2=0x7f7a8c105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:37.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 -- 172.21.15.69:0/163724589 shutdown_connections 2024-03-19T01:07:37.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:37.872+0000 7f7a93000640 1 -- 172.21.15.69:0/163724589 wait complete. 2024-03-19T01:07:37.893 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi069.front.sepia.ceph.com... 2024-03-19T01:07:37.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:37.893 DEBUG:teuthology.orchestra.run.smithi069:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-19T01:07:37.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[72975]: fuse finished with error 0 and tester_r 0 2024-03-19T01:07:37.988 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:37 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1753960500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:07:37.988 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:37 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1939163651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:07:37.989 INFO:teuthology.orchestra.run:waiting for 300 2024-03-19T01:07:38.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:37 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1753960500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:07:38.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:37 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1939163651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:07:38.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:37 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1753960500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:07:38.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:37 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1939163651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:07:39.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:38 smithi086 ceph-mon[66829]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:38 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/163724589' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T01:07:39.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:38 smithi069 ceph-mon[31727]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:39.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:38 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/163724589' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T01:07:39.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:38 smithi163 ceph-mon[67428]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:39.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:38 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/163724589' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T01:07:41.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:40 smithi086 ceph-mon[66829]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:40 smithi069 ceph-mon[31727]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:40 smithi163 ceph-mon[67428]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:43.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:42 smithi086 ceph-mon[66829]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:43.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:42 smithi069 ceph-mon[31727]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:43.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:42 smithi163 ceph-mon[67428]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:44.090 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-03-19T01:07:44.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:07:44.091 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-19T01:07:44.154 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:07:44.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.368+0000 7f31ff200640 1 -- 172.21.15.69:0/45378913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3200108e70 msgr2=0x7f32001092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:44.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.368+0000 7f31ff200640 1 --2- 172.21.15.69:0/45378913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3200108e70 0x7f32001092f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009900 tx=0x7f31e802bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:44.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.368+0000 7f31ff200640 1 -- 172.21.15.69:0/45378913 shutdown_connections 2024-03-19T01:07:44.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.368+0000 7f31ff200640 1 --2- 172.21.15.69:0/45378913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3200109830 0x7f32001100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.368+0000 7f31ff200640 1 --2- 172.21.15.69:0/45378913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3200108e70 0x7f32001092f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.368+0000 7f31ff200640 1 --2- 172.21.15.69:0/45378913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200107c80 0x7f32001080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.368+0000 7f31ff200640 1 -- 172.21.15.69:0/45378913 >> 172.21.15.69:0/45378913 conn(0x7f3200076c30 msgr2=0x7f3200079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:44.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.368+0000 7f31ff200640 1 -- 172.21.15.69:0/45378913 shutdown_connections 2024-03-19T01:07:44.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.368+0000 7f31ff200640 1 -- 172.21.15.69:0/45378913 wait complete. 2024-03-19T01:07:44.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.369+0000 7f31ff200640 1 Processor -- start 2024-03-19T01:07:44.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.369+0000 7f31ff200640 1 -- start start 2024-03-19T01:07:44.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31ff200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3200107c80 0x7f320019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:44.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31ff200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200108e70 0x7f320019e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:44.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31ff200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3200109830 0x7f32001a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:44.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31ff200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3200113950 con 0x7f3200109830 2024-03-19T01:07:44.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31ff200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32001137d0 con 0x7f3200107c80 2024-03-19T01:07:44.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31ff200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3200113ad0 con 0x7f3200108e70 2024-03-19T01:07:44.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31fde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3200107c80 0x7f320019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:44.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31fe800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3200109830 0x7f32001a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:44.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31fd400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200108e70 0x7f320019e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:44.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31fde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3200107c80 0x7f320019df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44258/0 (socket says 172.21.15.69:44258) 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31fde00640 1 -- 172.21.15.69:0/1336104552 learned_addr learned my addr 172.21.15.69:0/1336104552 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31fde00640 1 -- 172.21.15.69:0/1336104552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200108e70 msgr2=0x7f320019e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31fde00640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200108e70 0x7f320019e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31fde00640 1 -- 172.21.15.69:0/1336104552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3200109830 msgr2=0x7f32001a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31fde00640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3200109830 0x7f32001a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.370+0000 7f31fde00640 1 -- 172.21.15.69:0/1336104552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32001a5b80 con 0x7f3200107c80 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.371+0000 7f31fde00640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3200107c80 0x7f320019df60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31f000e9c0 tx=0x7f31f000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.371+0000 7f31f6a00640 1 -- 172.21.15.69:0/1336104552 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f000cd80 con 0x7f3200107c80 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.371+0000 7f31f6a00640 1 -- 172.21.15.69:0/1336104552 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31f00035f0 con 0x7f3200107c80 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.371+0000 7f31ff200640 1 -- 172.21.15.69:0/1336104552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32001a5e70 con 0x7f3200107c80 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.371+0000 7f31f6a00640 1 -- 172.21.15.69:0/1336104552 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0010690 con 0x7f3200107c80 2024-03-19T01:07:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.371+0000 7f31ff200640 1 -- 172.21.15.69:0/1336104552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32001a62d0 con 0x7f3200107c80 2024-03-19T01:07:44.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.372+0000 7f31ff200640 1 -- 172.21.15.69:0/1336104552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f320010efa0 con 0x7f3200107c80 2024-03-19T01:07:44.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.374+0000 7f31f6a00640 1 -- 172.21.15.69:0/1336104552 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f31f00037a0 con 0x7f3200107c80 2024-03-19T01:07:44.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.375+0000 7f31f6a00640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f31e40a2880 0x7f31e40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:44.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.375+0000 7f31f6a00640 1 -- 172.21.15.69:0/1336104552 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f31f00cef70 con 0x7f3200107c80 2024-03-19T01:07:44.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.375+0000 7f31fd400640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f31e40a2880 0x7f31e40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:44.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.376+0000 7f31fd400640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f31e40a2880 0x7f31e40a4d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31e802c240 tx=0x7f31e802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:44.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.379+0000 7f31f6a00640 1 -- 172.21.15.69:0/1336104552 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f31f009bdd0 con 0x7f3200107c80 2024-03-19T01:07:44.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.576+0000 7f31ff200640 1 -- 172.21.15.69:0/1336104552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3200062450 con 0x7f3200107c80 2024-03-19T01:07:44.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.578+0000 7f31f6a00640 1 -- 172.21.15.69:0/1336104552 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24069 (secure 0 0 0) 0x7f31f00860e0 con 0x7f3200107c80 2024-03-19T01:07:44.579 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:44.580 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":89,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:07:20.123357+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:07:44.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.580+0000 7f31ff200640 1 -- 172.21.15.69:0/1336104552 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f31e40a2880 msgr2=0x7f31e40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:44.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.580+0000 7f31ff200640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f31e40a2880 0x7f31e40a4d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31e802c240 tx=0x7f31e802df90 comp rx=0 tx=0).stop 2024-03-19T01:07:44.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.580+0000 7f31ff200640 1 -- 172.21.15.69:0/1336104552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3200107c80 msgr2=0x7f320019df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:44.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.580+0000 7f31ff200640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3200107c80 0x7f320019df60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31f000e9c0 tx=0x7f31f000ee80 comp rx=0 tx=0).stop 2024-03-19T01:07:44.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.581+0000 7f31ff200640 1 -- 172.21.15.69:0/1336104552 shutdown_connections 2024-03-19T01:07:44.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.581+0000 7f31ff200640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f31e40a2880 0x7f31e40a4d40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.581+0000 7f31ff200640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3200109830 0x7f32001a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.581+0000 7f31ff200640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200108e70 0x7f320019e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.581+0000 7f31ff200640 1 --2- 172.21.15.69:0/1336104552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3200107c80 0x7f320019df60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.581+0000 7f31ff200640 1 -- 172.21.15.69:0/1336104552 >> 172.21.15.69:0/1336104552 conn(0x7f3200076c30 msgr2=0x7f320010c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:44.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.581+0000 7f31ff200640 1 -- 172.21.15.69:0/1336104552 shutdown_connections 2024-03-19T01:07:44.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.581+0000 7f31ff200640 1 -- 172.21.15.69:0/1336104552 wait complete. 2024-03-19T01:07:44.600 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:07:44.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.806+0000 7fadfb800640 1 -- 172.21.15.69:0/1357464560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadf4109970 msgr2=0x7fadf4110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:44.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.806+0000 7fadfb800640 1 --2- 172.21.15.69:0/1357464560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadf4109970 0x7fadf4110290 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fadec009ca0 tx=0x7fadec02c0f0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.806+0000 7fadfb800640 1 -- 172.21.15.69:0/1357464560 shutdown_connections 2024-03-19T01:07:44.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.806+0000 7fadfb800640 1 --2- 172.21.15.69:0/1357464560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadf4109970 0x7fadf4110290 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.806+0000 7fadfb800640 1 --2- 172.21.15.69:0/1357464560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf4108fb0 0x7fadf4109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.806+0000 7fadfb800640 1 --2- 172.21.15.69:0/1357464560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf4107e60 0x7fadf4108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.806+0000 7fadfb800640 1 -- 172.21.15.69:0/1357464560 >> 172.21.15.69:0/1357464560 conn(0x7fadf4076e30 msgr2=0x7fadf4079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:44.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.806+0000 7fadfb800640 1 -- 172.21.15.69:0/1357464560 shutdown_connections 2024-03-19T01:07:44.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.806+0000 7fadfb800640 1 -- 172.21.15.69:0/1357464560 wait complete. 2024-03-19T01:07:44.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.807+0000 7fadfb800640 1 Processor -- start 2024-03-19T01:07:44.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.807+0000 7fadfb800640 1 -- start start 2024-03-19T01:07:44.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.807+0000 7fadfb800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf4107e60 0x7fadf419e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:44.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.807+0000 7fadfb800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf4108fb0 0x7fadf419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:44.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.807+0000 7fadfb800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadf4109970 0x7fadf41a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:44.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.807+0000 7fadfb800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4113ae0 con 0x7fadf4108fb0 2024-03-19T01:07:44.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.807+0000 7fadfb800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4113960 con 0x7fadf4109970 2024-03-19T01:07:44.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.807+0000 7fadfb800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4113c60 con 0x7fadf4107e60 2024-03-19T01:07:44.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf4108fb0 0x7fadf419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadf8e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf4107e60 0x7fadf419e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadf9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadf4109970 0x7fadf41a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadf8e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf4107e60 0x7fadf419e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53430/0 (socket says 172.21.15.69:53430) 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf4108fb0 0x7fadf419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52438/0 (socket says 172.21.15.69:52438) 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadf8e00640 1 -- 172.21.15.69:0/1489295116 learned_addr learned my addr 172.21.15.69:0/1489295116 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadf9800640 1 -- 172.21.15.69:0/1489295116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf4107e60 msgr2=0x7fadf419e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadf9800640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf4107e60 0x7fadf419e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadf9800640 1 -- 172.21.15.69:0/1489295116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf4108fb0 msgr2=0x7fadf419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadf9800640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf4108fb0 0x7fadf419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadf9800640 1 -- 172.21.15.69:0/1489295116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf41a5cc0 con 0x7fadf4109970 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.808+0000 7fadf9800640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadf4109970 0x7fadf41a5650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fadec037040 tx=0x7fadec032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.809+0000 7fadf2000640 1 -- 172.21.15.69:0/1489295116 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec041020 con 0x7fadf4109970 2024-03-19T01:07:44.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.809+0000 7fadf2000640 1 -- 172.21.15.69:0/1489295116 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadec034cd0 con 0x7fadf4109970 2024-03-19T01:07:44.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.809+0000 7fadf2000640 1 -- 172.21.15.69:0/1489295116 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec039040 con 0x7fadf4109970 2024-03-19T01:07:44.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.809+0000 7fadfb800640 1 -- 172.21.15.69:0/1489295116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf41a5f50 con 0x7fadf4109970 2024-03-19T01:07:44.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.809+0000 7fadfb800640 1 -- 172.21.15.69:0/1489295116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf41a6410 con 0x7fadf4109970 2024-03-19T01:07:44.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.810+0000 7fadfb800640 1 -- 172.21.15.69:0/1489295116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf410f110 con 0x7fadf4109970 2024-03-19T01:07:44.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.811+0000 7fadf2000640 1 -- 172.21.15.69:0/1489295116 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fadec044020 con 0x7fadf4109970 2024-03-19T01:07:44.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.813+0000 7fadf2000640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fadd80a2880 0x7fadd80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:44.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.813+0000 7fadf2000640 1 -- 172.21.15.69:0/1489295116 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fadec0f0a50 con 0x7fadf4109970 2024-03-19T01:07:44.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.813+0000 7fadf8e00640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fadd80a2880 0x7fadd80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:44.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.813+0000 7fadf8e00640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fadd80a2880 0x7fadd80a4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fade4005fd0 tx=0x7fade400b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:44.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:44.817+0000 7fadf2000640 1 -- 172.21.15.69:0/1489295116 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fadec04b020 con 0x7fadf4109970 2024-03-19T01:07:45.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:44 smithi086 ceph-mon[66829]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T01:07:45.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:44 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1336104552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:45.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.016+0000 7fadfb800640 1 -- 172.21.15.69:0/1489295116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadf40624c0 con 0x7fadf4109970 2024-03-19T01:07:45.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.018+0000 7fadf2000640 1 -- 172.21.15.69:0/1489295116 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24069 (secure 0 0 0) 0x7fadec0bd8b0 con 0x7fadf4109970 2024-03-19T01:07:45.019 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:45.019 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":89,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:07:20.123357+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:07:45.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.020+0000 7fadfb800640 1 -- 172.21.15.69:0/1489295116 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fadd80a2880 msgr2=0x7fadd80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:45.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.020+0000 7fadfb800640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fadd80a2880 0x7fadd80a4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fade4005fd0 tx=0x7fade400b4a0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.020+0000 7fadfb800640 1 -- 172.21.15.69:0/1489295116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadf4109970 msgr2=0x7fadf41a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:45.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.020+0000 7fadfb800640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadf4109970 0x7fadf41a5650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fadec037040 tx=0x7fadec032ea0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.021+0000 7fadfb800640 1 -- 172.21.15.69:0/1489295116 shutdown_connections 2024-03-19T01:07:45.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.021+0000 7fadfb800640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fadd80a2880 0x7fadd80a4d40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.021+0000 7fadfb800640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadf4109970 0x7fadf41a5650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.021+0000 7fadfb800640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf4108fb0 0x7fadf419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.021+0000 7fadfb800640 1 --2- 172.21.15.69:0/1489295116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf4107e60 0x7fadf419e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.021+0000 7fadfb800640 1 -- 172.21.15.69:0/1489295116 >> 172.21.15.69:0/1489295116 conn(0x7fadf4076e30 msgr2=0x7fadf410c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:45.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.021+0000 7fadfb800640 1 -- 172.21.15.69:0/1489295116 shutdown_connections 2024-03-19T01:07:45.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.021+0000 7fadfb800640 1 -- 172.21.15.69:0/1489295116 wait complete. 2024-03-19T01:07:45.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:44 smithi069 ceph-mon[31727]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T01:07:45.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:44 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1336104552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:45.040 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-19T01:07:45.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:44 smithi163 ceph-mon[67428]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-19T01:07:45.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:44 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1336104552' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:45.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.249+0000 7fbfdbc00640 1 -- 172.21.15.69:0/2237719622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfd4107e40 msgr2=0x7fbfd4108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:45.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.249+0000 7fbfdbc00640 1 --2- 172.21.15.69:0/2237719622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfd4107e40 0x7fbfd4108260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009900 tx=0x7fbfc402bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:45.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.249+0000 7fbfdbc00640 1 -- 172.21.15.69:0/2237719622 shutdown_connections 2024-03-19T01:07:45.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.249+0000 7fbfdbc00640 1 --2- 172.21.15.69:0/2237719622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd4109950 0x7fbfd4110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.249+0000 7fbfdbc00640 1 --2- 172.21.15.69:0/2237719622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd4108f90 0x7fbfd4109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.249+0000 7fbfdbc00640 1 --2- 172.21.15.69:0/2237719622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfd4107e40 0x7fbfd4108260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.249+0000 7fbfdbc00640 1 -- 172.21.15.69:0/2237719622 >> 172.21.15.69:0/2237719622 conn(0x7fbfd4076de0 msgr2=0x7fbfd4079220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:45.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.249+0000 7fbfdbc00640 1 -- 172.21.15.69:0/2237719622 shutdown_connections 2024-03-19T01:07:45.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.249+0000 7fbfdbc00640 1 -- 172.21.15.69:0/2237719622 wait complete. 2024-03-19T01:07:45.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.250+0000 7fbfdbc00640 1 Processor -- start 2024-03-19T01:07:45.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.250+0000 7fbfdbc00640 1 -- start start 2024-03-19T01:07:45.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfdbc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfd4107e40 0x7fbfd419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:45.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfdbc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd4108f90 0x7fbfd419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:45.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfdbc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd4109950 0x7fbfd41a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:45.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfdbc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4113b10 con 0x7fbfd4109950 2024-03-19T01:07:45.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfdbc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4113990 con 0x7fbfd4107e40 2024-03-19T01:07:45.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfdbc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4113c90 con 0x7fbfd4108f90 2024-03-19T01:07:45.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfd9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfd4107e40 0x7fbfd419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:45.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfd9c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd4109950 0x7fbfd41a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:45.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfd9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfd4107e40 0x7fbfd419e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44302/0 (socket says 172.21.15.69:44302) 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfd9200640 1 -- 172.21.15.69:0/3070528025 learned_addr learned my addr 172.21.15.69:0/3070528025 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfd3e00640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd4108f90 0x7fbfd419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfd9200640 1 -- 172.21.15.69:0/3070528025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd4108f90 msgr2=0x7fbfd419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfd9200640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd4108f90 0x7fbfd419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfd9200640 1 -- 172.21.15.69:0/3070528025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd4109950 msgr2=0x7fbfd41a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfd9200640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd4109950 0x7fbfd41a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.251+0000 7fbfd9200640 1 -- 172.21.15.69:0/3070528025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd41a5d50 con 0x7fbfd4107e40 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.252+0000 7fbfd9200640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfd4107e40 0x7fbfd419e130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4005ab0 tx=0x7fbfc402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.252+0000 7fbfd1600640 1 -- 172.21.15.69:0/3070528025 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4043020 con 0x7fbfd4107e40 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.252+0000 7fbfd1600640 1 -- 172.21.15.69:0/3070528025 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc40054e0 con 0x7fbfd4107e40 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.252+0000 7fbfdbc00640 1 -- 172.21.15.69:0/3070528025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd41a5fe0 con 0x7fbfd4107e40 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.252+0000 7fbfd1600640 1 -- 172.21.15.69:0/3070528025 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4003ce0 con 0x7fbfd4107e40 2024-03-19T01:07:45.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.252+0000 7fbfdbc00640 1 -- 172.21.15.69:0/3070528025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd41a6440 con 0x7fbfd4107e40 2024-03-19T01:07:45.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.253+0000 7fbfdbc00640 1 -- 172.21.15.69:0/3070528025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd410f150 con 0x7fbfd4107e40 2024-03-19T01:07:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.255+0000 7fbfd1600640 1 -- 172.21.15.69:0/3070528025 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fbfc4046020 con 0x7fbfd4107e40 2024-03-19T01:07:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.256+0000 7fbfd1600640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbfb80a2800 0x7fbfb80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.256+0000 7fbfd1600640 1 -- 172.21.15.69:0/3070528025 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbfc40f0e80 con 0x7fbfd4107e40 2024-03-19T01:07:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.256+0000 7fbfd3e00640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbfb80a2800 0x7fbfb80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.257+0000 7fbfd3e00640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbfb80a2800 0x7fbfb80a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbfd419f640 tx=0x7fbfc0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.260+0000 7fbfd1600640 1 -- 172.21.15.69:0/3070528025 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbfc404d020 con 0x7fbfd4107e40 2024-03-19T01:07:45.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.481+0000 7fbfdbc00640 1 -- 172.21.15.69:0/3070528025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbfd402d1f0 con 0x7fbfd4107e40 2024-03-19T01:07:45.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.766+0000 7fbfd1600640 1 -- 172.21.15.69:0/3070528025 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fbfc40bdc80 con 0x7fbfd4107e40 2024-03-19T01:07:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 -- 172.21.15.69:0/3070528025 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbfb80a2800 msgr2=0x7fbfb80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbfb80a2800 0x7fbfb80a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbfd419f640 tx=0x7fbfc0005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 -- 172.21.15.69:0/3070528025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfd4107e40 msgr2=0x7fbfd419e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfd4107e40 0x7fbfd419e130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4005ab0 tx=0x7fbfc402c6c0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 -- 172.21.15.69:0/3070528025 shutdown_connections 2024-03-19T01:07:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbfb80a2800 0x7fbfb80a4cc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfd4109950 0x7fbfd41a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd4108f90 0x7fbfd419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 --2- 172.21.15.69:0/3070528025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfd4107e40 0x7fbfd419e130 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 -- 172.21.15.69:0/3070528025 >> 172.21.15.69:0/3070528025 conn(0x7fbfd4076de0 msgr2=0x7fbfd410c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 -- 172.21.15.69:0/3070528025 shutdown_connections 2024-03-19T01:07:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:45.771+0000 7fbfdbc00640 1 -- 172.21.15.69:0/3070528025 wait complete. 2024-03-19T01:07:45.794 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-19T01:07:45.794 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-19T01:07:46.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:45 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1489295116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:46.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:45 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3070528025' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T01:07:46.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:45 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T01:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.009+0000 7fa8b6a00640 1 -- 172.21.15.69:0/1963786261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01052a0 msgr2=0x7fa8b0105720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.009+0000 7fa8b6a00640 1 --2- 172.21.15.69:0/1963786261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01052a0 0x7fa8b0105720 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009c90 tx=0x7fa8a802c0b0 comp rx=0 tx=0).stop 2024-03-19T01:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.009+0000 7fa8b6a00640 1 -- 172.21.15.69:0/1963786261 shutdown_connections 2024-03-19T01:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.009+0000 7fa8b6a00640 1 --2- 172.21.15.69:0/1963786261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01052a0 0x7fa8b0105720 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.009+0000 7fa8b6a00640 1 --2- 172.21.15.69:0/1963786261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8b01028f0 0x7fa8b0104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.009+0000 7fa8b6a00640 1 --2- 172.21.15.69:0/1963786261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8b00fff80 0x7fa8b01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.009+0000 7fa8b6a00640 1 -- 172.21.15.69:0/1963786261 >> 172.21.15.69:0/1963786261 conn(0x7fa8b00fbba0 msgr2=0x7fa8b00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.010+0000 7fa8b6a00640 1 -- 172.21.15.69:0/1963786261 shutdown_connections 2024-03-19T01:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.010+0000 7fa8b6a00640 1 -- 172.21.15.69:0/1963786261 wait complete. 2024-03-19T01:07:46.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.011+0000 7fa8b6a00640 1 Processor -- start 2024-03-19T01:07:46.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.011+0000 7fa8b6a00640 1 -- start start 2024-03-19T01:07:46.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.011+0000 7fa8b6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8b00fff80 0x7fa8b019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.011+0000 7fa8b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028f0 0x7fa8b019e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.011+0000 7fa8b6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8b01052a0 0x7fa8b01a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.011+0000 7fa8b6a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b0113960 con 0x7fa8b01052a0 2024-03-19T01:07:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.011+0000 7fa8b6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b01137e0 con 0x7fa8b01028f0 2024-03-19T01:07:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.011+0000 7fa8b6a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b0113ae0 con 0x7fa8b00fff80 2024-03-19T01:07:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.011+0000 7fa8b6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8b01052a0 0x7fa8b01a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.011+0000 7fa8b4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028f0 0x7fa8b019e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8b6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8b01052a0 0x7fa8b01a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53618/0 (socket says 172.21.15.69:53618) 2024-03-19T01:07:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8b6000640 1 -- 172.21.15.69:0/329479986 learned_addr learned my addr 172.21.15.69:0/329479986 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8b4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028f0 0x7fa8b019e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44852/0 (socket says 172.21.15.69:44852) 2024-03-19T01:07:46.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8b5600640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8b00fff80 0x7fa8b019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:46.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8b6000640 1 -- 172.21.15.69:0/329479986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8b00fff80 msgr2=0x7fa8b019dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8b6000640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8b00fff80 0x7fa8b019dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8b6000640 1 -- 172.21.15.69:0/329479986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028f0 msgr2=0x7fa8b019e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8b6000640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028f0 0x7fa8b019e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8b6000640 1 -- 172.21.15.69:0/329479986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b01a5be0 con 0x7fa8b01052a0 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8b4c00640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028f0 0x7fa8b019e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8b6000640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8b01052a0 0x7fa8b01a5570 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8005cb0 tx=0x7fa8a80346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.012+0000 7fa8ae000640 1 -- 172.21.15.69:0/329479986 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8039040 con 0x7fa8b01052a0 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.013+0000 7fa8ae000640 1 -- 172.21.15.69:0/329479986 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a8043070 con 0x7fa8b01052a0 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.013+0000 7fa8b6a00640 1 -- 172.21.15.69:0/329479986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b01a5e70 con 0x7fa8b01052a0 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.013+0000 7fa8ae000640 1 -- 172.21.15.69:0/329479986 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8004040 con 0x7fa8b01052a0 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.013+0000 7fa8b6a00640 1 -- 172.21.15.69:0/329479986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b01a62d0 con 0x7fa8b01052a0 2024-03-19T01:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.014+0000 7fa8b6a00640 1 -- 172.21.15.69:0/329479986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b00fd690 con 0x7fa8b01052a0 2024-03-19T01:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.016+0000 7fa8ae000640 1 -- 172.21.15.69:0/329479986 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa8a8032880 con 0x7fa8b01052a0 2024-03-19T01:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.018+0000 7fa8ae000640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8980a2880 0x7fa8980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:46.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.018+0000 7fa8b5600640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8980a2880 0x7fa8980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:46.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.018+0000 7fa8ae000640 1 -- 172.21.15.69:0/329479986 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa8a80f1310 con 0x7fa8b01052a0 2024-03-19T01:07:46.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.019+0000 7fa8b5600640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8980a2880 0x7fa8980a4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4005d00 tx=0x7fa8a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.020+0000 7fa8ae000640 1 -- 172.21.15.69:0/329479986 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa8a80be2b0 con 0x7fa8b01052a0 2024-03-19T01:07:46.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:45 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1489295116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:46.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:45 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3070528025' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T01:07:46.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:45 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T01:07:46.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:45 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1489295116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:46.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:45 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3070528025' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T01:07:46.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:45 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T01:07:46.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.246+0000 7fa8b6a00640 1 -- 172.21.15.69:0/329479986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fa8b002d290 con 0x7fa8b01052a0 2024-03-19T01:07:46.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.777+0000 7fa8ae000640 1 -- 172.21.15.69:0/329479986 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fa8a80bdbb0 con 0x7fa8b01052a0 2024-03-19T01:07:46.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.780+0000 7fa8b6a00640 1 -- 172.21.15.69:0/329479986 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8980a2880 msgr2=0x7fa8980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:46.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.780+0000 7fa8b6a00640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8980a2880 0x7fa8980a4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4005d00 tx=0x7fa8a400b410 comp rx=0 tx=0).stop 2024-03-19T01:07:46.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.780+0000 7fa8b6a00640 1 -- 172.21.15.69:0/329479986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8b01052a0 msgr2=0x7fa8b01a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:46.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.780+0000 7fa8b6a00640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8b01052a0 0x7fa8b01a5570 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8005cb0 tx=0x7fa8a80346f0 comp rx=0 tx=0).stop 2024-03-19T01:07:46.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.781+0000 7fa8b6a00640 1 -- 172.21.15.69:0/329479986 shutdown_connections 2024-03-19T01:07:46.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.781+0000 7fa8b6a00640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8980a2880 0x7fa8980a4d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:46.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.781+0000 7fa8b6a00640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8b01052a0 0x7fa8b01a5570 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:46.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.781+0000 7fa8b6a00640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b01028f0 0x7fa8b019e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:46.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.781+0000 7fa8b6a00640 1 --2- 172.21.15.69:0/329479986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8b00fff80 0x7fa8b019dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:46.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.781+0000 7fa8b6a00640 1 -- 172.21.15.69:0/329479986 >> 172.21.15.69:0/329479986 conn(0x7fa8b00fbba0 msgr2=0x7fa8b0100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:46.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.781+0000 7fa8b6a00640 1 -- 172.21.15.69:0/329479986 shutdown_connections 2024-03-19T01:07:46.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.781+0000 7fa8b6a00640 1 -- 172.21.15.69:0/329479986 wait complete. 2024-03-19T01:07:46.808 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-19T01:07:46.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.998+0000 7fb1aaa00640 1 -- 172.21.15.69:0/2545832500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a4074240 msgr2=0x7fb1a40746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:46.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.998+0000 7fb1aaa00640 1 --2- 172.21.15.69:0/2545832500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a4074240 0x7fb1a40746c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb190009900 tx=0x7fb19002bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:46.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.998+0000 7fb1aaa00640 1 -- 172.21.15.69:0/2545832500 shutdown_connections 2024-03-19T01:07:46.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.998+0000 7fb1aaa00640 1 --2- 172.21.15.69:0/2545832500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a41095d0 0x7fb1a410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.998+0000 7fb1aaa00640 1 --2- 172.21.15.69:0/2545832500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a4074240 0x7fb1a40746c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.998+0000 7fb1aaa00640 1 --2- 172.21.15.69:0/2545832500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a4075820 0x7fb1a4073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.998+0000 7fb1aaa00640 1 -- 172.21.15.69:0/2545832500 >> 172.21.15.69:0/2545832500 conn(0x7fb1a40fcc90 msgr2=0x7fb1a40ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.998+0000 7fb1aaa00640 1 -- 172.21.15.69:0/2545832500 shutdown_connections 2024-03-19T01:07:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.998+0000 7fb1aaa00640 1 -- 172.21.15.69:0/2545832500 wait complete. 2024-03-19T01:07:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.999+0000 7fb1aaa00640 1 Processor -- start 2024-03-19T01:07:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:46.999+0000 7fb1aaa00640 1 -- start start 2024-03-19T01:07:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1aaa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4074240 0x7fb1a4199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1aaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a4075820 0x7fb1a419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1aaa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a41095d0 0x7fb1a41a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1aaa00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410f5d0 con 0x7fb1a41095d0 2024-03-19T01:07:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1aaa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410f450 con 0x7fb1a4075820 2024-03-19T01:07:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1aaa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410f750 con 0x7fb1a4074240 2024-03-19T01:07:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1a8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a4075820 0x7fb1a419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1a8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a4075820 0x7fb1a419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44868/0 (socket says 172.21.15.69:44868) 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1a8c00640 1 -- 172.21.15.69:0/274555088 learned_addr learned my addr 172.21.15.69:0/274555088 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1aa000640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a41095d0 0x7fb1a41a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1a9600640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4074240 0x7fb1a4199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1a8c00640 1 -- 172.21.15.69:0/274555088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4074240 msgr2=0x7fb1a4199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1a8c00640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4074240 0x7fb1a4199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1a8c00640 1 -- 172.21.15.69:0/274555088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a41095d0 msgr2=0x7fb1a41a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1a8c00640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a41095d0 0x7fb1a41a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1a8c00640 1 -- 172.21.15.69:0/274555088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a41a1830 con 0x7fb1a4075820 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1a9600640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4074240 0x7fb1a4199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.000+0000 7fb1aa000640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a41095d0 0x7fb1a41a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:47.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.001+0000 7fb1a8c00640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a4075820 0x7fb1a419a160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb190000c00 tx=0x7fb19002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:47.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.001+0000 7fb1a2000640 1 -- 172.21.15.69:0/274555088 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190043020 con 0x7fb1a4075820 2024-03-19T01:07:47.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.001+0000 7fb1a2000640 1 -- 172.21.15.69:0/274555088 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1900054e0 con 0x7fb1a4075820 2024-03-19T01:07:47.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.001+0000 7fb1aaa00640 1 -- 172.21.15.69:0/274555088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a41a1ac0 con 0x7fb1a4075820 2024-03-19T01:07:47.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.001+0000 7fb1a2000640 1 -- 172.21.15.69:0/274555088 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190003c40 con 0x7fb1a4075820 2024-03-19T01:07:47.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.001+0000 7fb1aaa00640 1 -- 172.21.15.69:0/274555088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a41a2060 con 0x7fb1a4075820 2024-03-19T01:07:47.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.002+0000 7fb1aaa00640 1 -- 172.21.15.69:0/274555088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a4108350 con 0x7fb1a4075820 2024-03-19T01:07:47.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.004+0000 7fb1a2000640 1 -- 172.21.15.69:0/274555088 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb190046020 con 0x7fb1a4075820 2024-03-19T01:07:47.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.005+0000 7fb1a2000640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb18c0a2880 0x7fb18c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:47.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.005+0000 7fb1a2000640 1 -- 172.21.15.69:0/274555088 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb1900f13f0 con 0x7fb1a4075820 2024-03-19T01:07:47.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.006+0000 7fb1a9600640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb18c0a2880 0x7fb18c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:47.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.006+0000 7fb1a9600640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb18c0a2880 0x7fb18c0a4d40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb1a419b0f0 tx=0x7fb198005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:47.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.007+0000 7fb1a2000640 1 -- 172.21.15.69:0/274555088 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb19004d020 con 0x7fb1a4075820 2024-03-19T01:07:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:46 smithi069 ceph-mon[31727]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:46 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-19T01:07:47.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:46 smithi069 ceph-mon[31727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:47.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:46 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/329479986' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-19T01:07:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:46 smithi163 ceph-mon[67428]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:46 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-19T01:07:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:46 smithi163 ceph-mon[67428]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:47.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:46 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/329479986' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-19T01:07:47.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.247+0000 7fb1aaa00640 1 -- 172.21.15.69:0/274555088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fb1a402d1f0 con 0x7fb1a4075820 2024-03-19T01:07:47.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:46 smithi086 ceph-mon[66829]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-19T01:07:47.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:46 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-19T01:07:47.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:46 smithi086 ceph-mon[66829]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:47.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:46 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/329479986' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-19T01:07:47.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.791+0000 7fb1a2000640 1 -- 172.21.15.69:0/274555088 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fb190050350 con 0x7fb1a4075820 2024-03-19T01:07:47.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.795+0000 7fb1aaa00640 1 -- 172.21.15.69:0/274555088 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb18c0a2880 msgr2=0x7fb18c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:47.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.795+0000 7fb1aaa00640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb18c0a2880 0x7fb18c0a4d40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb1a419b0f0 tx=0x7fb198005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:47.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.795+0000 7fb1aaa00640 1 -- 172.21.15.69:0/274555088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a4075820 msgr2=0x7fb1a419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:47.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.795+0000 7fb1aaa00640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a4075820 0x7fb1a419a160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb190000c00 tx=0x7fb19002c650 comp rx=0 tx=0).stop 2024-03-19T01:07:47.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.795+0000 7fb1aaa00640 1 -- 172.21.15.69:0/274555088 shutdown_connections 2024-03-19T01:07:47.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.795+0000 7fb1aaa00640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb18c0a2880 0x7fb18c0a4d40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:47.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.795+0000 7fb1aaa00640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1a41095d0 0x7fb1a41a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:47.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.795+0000 7fb1aaa00640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a4075820 0x7fb1a419a160 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:47.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.795+0000 7fb1aaa00640 1 --2- 172.21.15.69:0/274555088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1a4074240 0x7fb1a4199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:47.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.795+0000 7fb1aaa00640 1 -- 172.21.15.69:0/274555088 >> 172.21.15.69:0/274555088 conn(0x7fb1a40fcc90 msgr2=0x7fb1a4105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:47.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.796+0000 7fb1aaa00640 1 -- 172.21.15.69:0/274555088 shutdown_connections 2024-03-19T01:07:47.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:47.796+0000 7fb1aaa00640 1 -- 172.21.15.69:0/274555088 wait complete. 2024-03-19T01:07:47.823 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-19T01:07:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.015+0000 7fa57b600640 1 -- 172.21.15.69:0/4120686822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5741022e0 msgr2=0x7fa574108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.015+0000 7fa57b600640 1 --2- 172.21.15.69:0/4120686822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5741022e0 0x7fa574108b70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009900 tx=0x7fa56c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.015+0000 7fa57b600640 1 -- 172.21.15.69:0/4120686822 shutdown_connections 2024-03-19T01:07:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.015+0000 7fa57b600640 1 --2- 172.21.15.69:0/4120686822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5741022e0 0x7fa574108b70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:48.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.015+0000 7fa57b600640 1 --2- 172.21.15.69:0/4120686822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa574101920 0x7fa574101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:48.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.015+0000 7fa57b600640 1 --2- 172.21.15.69:0/4120686822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa574100730 0x7fa574100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:48.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.015+0000 7fa57b600640 1 -- 172.21.15.69:0/4120686822 >> 172.21.15.69:0/4120686822 conn(0x7fa5740fbcc0 msgr2=0x7fa5740fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:48.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.015+0000 7fa57b600640 1 -- 172.21.15.69:0/4120686822 shutdown_connections 2024-03-19T01:07:48.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.015+0000 7fa57b600640 1 -- 172.21.15.69:0/4120686822 wait complete. 2024-03-19T01:07:48.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.016+0000 7fa57b600640 1 Processor -- start 2024-03-19T01:07:48.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.017+0000 7fa57b600640 1 -- start start 2024-03-19T01:07:48.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.017+0000 7fa57b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa574100730 0x7fa574199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:48.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.017+0000 7fa57b600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa574101920 0x7fa57419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:48.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.017+0000 7fa57b600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5741022e0 0x7fa5741a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.017+0000 7fa57b600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa57410d4d0 con 0x7fa5741022e0 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.017+0000 7fa57b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa57410d350 con 0x7fa574100730 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.017+0000 7fa57b600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa57410d650 con 0x7fa574101920 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.017+0000 7fa578c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa574100730 0x7fa574199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.017+0000 7fa578c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa574100730 0x7fa574199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44896/0 (socket says 172.21.15.69:44896) 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.017+0000 7fa578c00640 1 -- 172.21.15.69:0/3200779261 learned_addr learned my addr 172.21.15.69:0/3200779261 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.017+0000 7fa579600640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5741022e0 0x7fa5741a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa573e00640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa574101920 0x7fa57419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa578c00640 1 -- 172.21.15.69:0/3200779261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa574101920 msgr2=0x7fa57419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa578c00640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa574101920 0x7fa57419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa578c00640 1 -- 172.21.15.69:0/3200779261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5741022e0 msgr2=0x7fa5741a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa578c00640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5741022e0 0x7fa5741a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa578c00640 1 -- 172.21.15.69:0/3200779261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5741a1920 con 0x7fa574100730 2024-03-19T01:07:48.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa579600640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5741022e0 0x7fa5741a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:48.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa578c00640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa574100730 0x7fa574199d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa56400eb70 tx=0x7fa56400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:48.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa571600640 1 -- 172.21.15.69:0/3200779261 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564003b70 con 0x7fa574100730 2024-03-19T01:07:48.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa571600640 1 -- 172.21.15.69:0/3200779261 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa56400ee70 con 0x7fa574100730 2024-03-19T01:07:48.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa57b600640 1 -- 172.21.15.69:0/3200779261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5741a1c10 con 0x7fa574100730 2024-03-19T01:07:48.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa571600640 1 -- 172.21.15.69:0/3200779261 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564010830 con 0x7fa574100730 2024-03-19T01:07:48.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.018+0000 7fa57b600640 1 -- 172.21.15.69:0/3200779261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5741a2160 con 0x7fa574100730 2024-03-19T01:07:48.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.020+0000 7fa57b600640 1 -- 172.21.15.69:0/3200779261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5741a2330 con 0x7fa574100730 2024-03-19T01:07:48.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.021+0000 7fa571600640 1 -- 172.21.15.69:0/3200779261 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa564004020 con 0x7fa574100730 2024-03-19T01:07:48.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.023+0000 7fa571600640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5580a2880 0x7fa5580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:48.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.024+0000 7fa571600640 1 -- 172.21.15.69:0/3200779261 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa5640cf500 con 0x7fa574100730 2024-03-19T01:07:48.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.024+0000 7fa573e00640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5580a2880 0x7fa5580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:48.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.024+0000 7fa573e00640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5580a2880 0x7fa5580a4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa560005fd0 tx=0x7fa560005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:48.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.026+0000 7fa571600640 1 -- 172.21.15.69:0/3200779261 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa56409c300 con 0x7fa574100730 2024-03-19T01:07:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:47 smithi069 ceph-mon[31727]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-19T01:07:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:47 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/329479986' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-19T01:07:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:47 smithi069 ceph-mon[31727]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-19T01:07:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:47 smithi069 ceph-mon[31727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:47 smithi069 ceph-mon[31727]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-19T01:07:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:47 smithi069 ceph-mon[31727]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-19T01:07:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:47 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/274555088' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T01:07:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:47 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T01:07:48.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:47 smithi163 ceph-mon[67428]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-19T01:07:48.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:47 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/329479986' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-19T01:07:48.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:47 smithi163 ceph-mon[67428]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-19T01:07:48.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:47 smithi163 ceph-mon[67428]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:48.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:47 smithi163 ceph-mon[67428]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-19T01:07:48.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:47 smithi163 ceph-mon[67428]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-19T01:07:48.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:47 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/274555088' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T01:07:48.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:47 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T01:07:48.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:47 smithi086 ceph-mon[66829]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-19T01:07:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:47 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/329479986' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-19T01:07:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:47 smithi086 ceph-mon[66829]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-19T01:07:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:47 smithi086 ceph-mon[66829]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-19T01:07:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:47 smithi086 ceph-mon[66829]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-19T01:07:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:47 smithi086 ceph-mon[66829]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-19T01:07:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:47 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/274555088' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T01:07:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:47 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T01:07:48.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.268+0000 7fa57b600640 1 -- 172.21.15.69:0/3200779261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa5741a2720 con 0x7fa574100730 2024-03-19T01:07:48.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.754+0000 7fa571600640 1 -- 172.21.15.69:0/3200779261 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7fa56409bc00 con 0x7fa574100730 2024-03-19T01:07:48.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.758+0000 7fa57b600640 1 -- 172.21.15.69:0/3200779261 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5580a2880 msgr2=0x7fa5580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:48.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.758+0000 7fa57b600640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5580a2880 0x7fa5580a4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa560005fd0 tx=0x7fa560005850 comp rx=0 tx=0).stop 2024-03-19T01:07:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.758+0000 7fa57b600640 1 -- 172.21.15.69:0/3200779261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa574100730 msgr2=0x7fa574199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.758+0000 7fa57b600640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa574100730 0x7fa574199d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa56400eb70 tx=0x7fa56400c6a0 comp rx=0 tx=0).stop 2024-03-19T01:07:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.758+0000 7fa57b600640 1 -- 172.21.15.69:0/3200779261 shutdown_connections 2024-03-19T01:07:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.758+0000 7fa57b600640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5580a2880 0x7fa5580a4d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.758+0000 7fa57b600640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5741022e0 0x7fa5741a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.758+0000 7fa57b600640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa574101920 0x7fa57419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.758+0000 7fa57b600640 1 --2- 172.21.15.69:0/3200779261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa574100730 0x7fa574199d10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.758+0000 7fa57b600640 1 -- 172.21.15.69:0/3200779261 >> 172.21.15.69:0/3200779261 conn(0x7fa5740fbcc0 msgr2=0x7fa574109530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.758+0000 7fa57b600640 1 -- 172.21.15.69:0/3200779261 shutdown_connections 2024-03-19T01:07:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:48.759+0000 7fa57b600640 1 -- 172.21.15.69:0/3200779261 wait complete. 2024-03-19T01:07:48.788 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-19T01:07:49.916 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-19T01:07:49.917 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-03-19T01:07:49.917 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-19T01:07:49.917 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] up:active 2024-03-19T01:07:49.917 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-19T01:07:49.917 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-19T01:07:49.917 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-19T01:07:49.917 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3200779261' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] up:active 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:48 smithi086 ceph-mon[66829]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] up:active 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-19T01:07:49.918 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3200779261' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] up:active 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:48 smithi163 ceph-mon[67428]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-03-19T01:07:49.919 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] up:active 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3200779261' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] up:active 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:creating} 3 up:standby-replay 5 up:standby 2024-03-19T01:07:49.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:48 smithi069 ceph-mon[31727]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-19T01:07:50.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.137+0000 7fe8acc00640 1 -- 172.21.15.69:0/1844286428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a8075800 msgr2=0x7fe8a8073c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:50.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.137+0000 7fe8acc00640 1 --2- 172.21.15.69:0/1844286428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a8075800 0x7fe8a8073c50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe898009900 tx=0x7fe89802bcc0 comp rx=0 tx=0).stop 2024-03-19T01:07:50.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.137+0000 7fe8acc00640 1 -- 172.21.15.69:0/1844286428 shutdown_connections 2024-03-19T01:07:50.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.137+0000 7fe8acc00640 1 --2- 172.21.15.69:0/1844286428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a81096c0 0x7fe8a810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:50.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.137+0000 7fe8acc00640 1 --2- 172.21.15.69:0/1844286428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a8074220 0x7fe8a8074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:50.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.137+0000 7fe8acc00640 1 --2- 172.21.15.69:0/1844286428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a8075800 0x7fe8a8073c50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:50.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.137+0000 7fe8acc00640 1 -- 172.21.15.69:0/1844286428 >> 172.21.15.69:0/1844286428 conn(0x7fe8a80fcdc0 msgr2=0x7fe8a80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:50.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.137+0000 7fe8acc00640 1 -- 172.21.15.69:0/1844286428 shutdown_connections 2024-03-19T01:07:50.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.137+0000 7fe8acc00640 1 -- 172.21.15.69:0/1844286428 wait complete. 2024-03-19T01:07:50.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8acc00640 1 Processor -- start 2024-03-19T01:07:50.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8acc00640 1 -- start start 2024-03-19T01:07:50.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8acc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a8074220 0x7fe8a8199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:50.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8075800 0x7fe8a819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:50.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8acc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a81096c0 0x7fe8a81a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:50.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8acc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a810f6d0 con 0x7fe8a8074220 2024-03-19T01:07:50.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8acc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a810f550 con 0x7fe8a8075800 2024-03-19T01:07:50.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8acc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a810f850 con 0x7fe8a81096c0 2024-03-19T01:07:50.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8a5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8075800 0x7fe8a819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8a5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a8074220 0x7fe8a8199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8a5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8075800 0x7fe8a819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44926/0 (socket says 172.21.15.69:44926) 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8a5400640 1 -- 172.21.15.69:0/1523747669 learned_addr learned my addr 172.21.15.69:0/1523747669 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.138+0000 7fe8a6800640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a81096c0 0x7fe8a81a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.139+0000 7fe8a5400640 1 -- 172.21.15.69:0/1523747669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a81096c0 msgr2=0x7fe8a81a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.139+0000 7fe8a5400640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a81096c0 0x7fe8a81a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.139+0000 7fe8a5400640 1 -- 172.21.15.69:0/1523747669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a8074220 msgr2=0x7fe8a8199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.139+0000 7fe8a5400640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a8074220 0x7fe8a8199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.139+0000 7fe8a5400640 1 -- 172.21.15.69:0/1523747669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a81a1990 con 0x7fe8a8075800 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.139+0000 7fe8a5400640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8075800 0x7fe8a819a2b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e9f0 tx=0x7fe89400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.139+0000 7fe89ea00640 1 -- 172.21.15.69:0/1523747669 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89400ccc0 con 0x7fe8a8075800 2024-03-19T01:07:50.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.139+0000 7fe89ea00640 1 -- 172.21.15.69:0/1523747669 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe894010910 con 0x7fe8a8075800 2024-03-19T01:07:50.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.139+0000 7fe8acc00640 1 -- 172.21.15.69:0/1523747669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a81a1c80 con 0x7fe8a8075800 2024-03-19T01:07:50.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.139+0000 7fe89ea00640 1 -- 172.21.15.69:0/1523747669 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8940188a0 con 0x7fe8a8075800 2024-03-19T01:07:50.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.139+0000 7fe8acc00640 1 -- 172.21.15.69:0/1523747669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a81a21d0 con 0x7fe8a8075800 2024-03-19T01:07:50.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.141+0000 7fe8acc00640 1 -- 172.21.15.69:0/1523747669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a8108480 con 0x7fe8a8075800 2024-03-19T01:07:50.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.143+0000 7fe89ea00640 1 -- 172.21.15.69:0/1523747669 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe894010ac0 con 0x7fe8a8075800 2024-03-19T01:07:50.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.145+0000 7fe89ea00640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe88c0a2880 0x7fe88c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:50.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.145+0000 7fe89ea00640 1 -- 172.21.15.69:0/1523747669 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe8940cf070 con 0x7fe8a8075800 2024-03-19T01:07:50.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.145+0000 7fe8a5e00640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe88c0a2880 0x7fe88c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:50.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.146+0000 7fe8a5e00640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe88c0a2880 0x7fe88c0a4d40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe89802c2d0 tx=0x7fe89802c910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:50.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.148+0000 7fe89ea00640 1 -- 172.21.15.69:0/1523747669 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe89409be70 con 0x7fe8a8075800 2024-03-19T01:07:50.255 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b[97627]: 2024-03-19T01:07:49.766+0000 7ff2d2e65700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-19T01:07:50.256 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b[97627]: 2024-03-19T01:07:49.766+0000 7ff2d2e65700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-19T01:07:50.256 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:07:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b[97627]: 2024-03-19T01:07:49.766+0000 7ff2d2e65700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-19T01:07:50.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.415+0000 7fe8acc00640 1 -- 172.21.15.69:0/1523747669 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe8a8061c60 con 0x7fe88c0a2880 2024-03-19T01:07:50.768 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:07:50.424+0000 7f248b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:07:50.769 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:07:50.424+0000 7f248b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:07:50.769 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:07:50.424+0000 7f248b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:07:50.769 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:07:50.424+0000 7f248b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:07:50.769 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:07:50.424+0000 7f248b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:07:50.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.821+0000 7fe89ea00640 1 -- 172.21.15.69:0/1523747669 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe8a8061c60 con 0x7fe88c0a2880 2024-03-19T01:07:50.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 -- 172.21.15.69:0/1523747669 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe88c0a2880 msgr2=0x7fe88c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:50.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe88c0a2880 0x7fe88c0a4d40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe89802c2d0 tx=0x7fe89802c910 comp rx=0 tx=0).stop 2024-03-19T01:07:50.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 -- 172.21.15.69:0/1523747669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8075800 msgr2=0x7fe8a819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:50.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8075800 0x7fe8a819a2b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e9f0 tx=0x7fe89400edf0 comp rx=0 tx=0).stop 2024-03-19T01:07:50.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 -- 172.21.15.69:0/1523747669 shutdown_connections 2024-03-19T01:07:50.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe88c0a2880 0x7fe88c0a4d40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a81096c0 0x7fe8a81a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8075800 0x7fe8a819a2b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 --2- 172.21.15.69:0/1523747669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8a8074220 0x7fe8a8199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 -- 172.21.15.69:0/1523747669 >> 172.21.15.69:0/1523747669 conn(0x7fe8a80fcdc0 msgr2=0x7fe8a8105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 -- 172.21.15.69:0/1523747669 shutdown_connections 2024-03-19T01:07:50.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:50.824+0000 7fe8acc00640 1 -- 172.21.15.69:0/1523747669 wait complete. 2024-03-19T01:07:50.845 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-19T01:07:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-mon[31727]: pgmap v332: 129 pgs: 129 active+clean; 583 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 3 op/s 2024-03-19T01:07:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] up:active 2024-03-19T01:07:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-mon[31727]: daemon mds.d assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-19T01:07:51.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-mon[31727]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-19T01:07:51.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-mon[31727]: Cluster is now healthy 2024-03-19T01:07:51.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-mon[31727]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-03-19T01:07:51.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-mon[31727]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-19T01:07:51.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:50 smithi069 ceph-mon[31727]: daemon mds.d is now active in filesystem cephfs as rank 4 2024-03-19T01:07:51.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.048+0000 7f4e68a00640 1 -- 172.21.15.69:0/1558342110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e641021e0 msgr2=0x7f4e641045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:51.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.048+0000 7f4e68a00640 1 --2- 172.21.15.69:0/1558342110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e641021e0 0x7f4e641045d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009900 tx=0x7f4e5402bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:51.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.048+0000 7f4e68a00640 1 -- 172.21.15.69:0/1558342110 shutdown_connections 2024-03-19T01:07:51.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.048+0000 7f4e68a00640 1 --2- 172.21.15.69:0/1558342110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e64107440 0x7f4e64109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.048+0000 7f4e68a00640 1 --2- 172.21.15.69:0/1558342110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e64104b10 0x7f4e64106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.048+0000 7f4e68a00640 1 --2- 172.21.15.69:0/1558342110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e641021e0 0x7f4e641045d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.048+0000 7f4e68a00640 1 -- 172.21.15.69:0/1558342110 >> 172.21.15.69:0/1558342110 conn(0x7f4e640fbb90 msgr2=0x7f4e640fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:51.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.048+0000 7f4e68a00640 1 -- 172.21.15.69:0/1558342110 shutdown_connections 2024-03-19T01:07:51.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.048+0000 7f4e68a00640 1 -- 172.21.15.69:0/1558342110 wait complete. 2024-03-19T01:07:51.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.049+0000 7f4e68a00640 1 Processor -- start 2024-03-19T01:07:51.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.049+0000 7f4e68a00640 1 -- start start 2024-03-19T01:07:51.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e68a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e641021e0 0x7f4e64101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:51.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e68a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e64104b10 0x7f4e64100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:51.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e68a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e64107440 0x7f4e641006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:51.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e68a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6410d3a0 con 0x7f4e64107440 2024-03-19T01:07:51.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e68a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6410d220 con 0x7f4e641021e0 2024-03-19T01:07:51.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e68a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6410d520 con 0x7f4e64104b10 2024-03-19T01:07:51.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e641021e0 0x7f4e64101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e62a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e64104b10 0x7f4e64100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e641021e0 0x7f4e64101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44954/0 (socket says 172.21.15.69:44954) 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e62a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e64104b10 0x7f4e64100190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53764/0 (socket says 172.21.15.69:53764) 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e63400640 1 -- 172.21.15.69:0/1300145160 learned_addr learned my addr 172.21.15.69:0/1300145160 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.050+0000 7f4e63e00640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e64107440 0x7f4e641006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e63400640 1 -- 172.21.15.69:0/1300145160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e64104b10 msgr2=0x7f4e64100190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e63400640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e64104b10 0x7f4e64100190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e63400640 1 -- 172.21.15.69:0/1300145160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e64107440 msgr2=0x7f4e641006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e63400640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e64107440 0x7f4e641006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e63400640 1 -- 172.21.15.69:0/1300145160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e64100f60 con 0x7f4e641021e0 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e62a00640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e64104b10 0x7f4e64100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e63400640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e641021e0 0x7f4e64101b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009230 tx=0x7f4e5402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e5be00640 1 -- 172.21.15.69:0/1300145160 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54041070 con 0x7f4e641021e0 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e5be00640 1 -- 172.21.15.69:0/1300145160 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e54004450 con 0x7f4e641021e0 2024-03-19T01:07:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e68a00640 1 -- 172.21.15.69:0/1300145160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e641a1b30 con 0x7f4e641021e0 2024-03-19T01:07:51.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e5be00640 1 -- 172.21.15.69:0/1300145160 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54003e20 con 0x7f4e641021e0 2024-03-19T01:07:51.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.051+0000 7f4e68a00640 1 -- 172.21.15.69:0/1300145160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e641a20d0 con 0x7f4e641021e0 2024-03-19T01:07:51.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.052+0000 7f4e68a00640 1 -- 172.21.15.69:0/1300145160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e640fd680 con 0x7f4e641021e0 2024-03-19T01:07:51.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.055+0000 7f4e5be00640 1 -- 172.21.15.69:0/1300145160 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f4e54044070 con 0x7f4e641021e0 2024-03-19T01:07:51.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.056+0000 7f4e5be00640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4e480a2880 0x7f4e480a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:51.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.056+0000 7f4e5be00640 1 -- 172.21.15.69:0/1300145160 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4e540f0b70 con 0x7f4e641021e0 2024-03-19T01:07:51.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.056+0000 7f4e62a00640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4e480a2880 0x7f4e480a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:51.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.057+0000 7f4e62a00640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4e480a2880 0x7f4e480a4d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4e500095f0 tx=0x7f4e50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:51.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.058+0000 7f4e5be00640 1 -- 172.21.15.69:0/1300145160 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4e540bd320 con 0x7f4e641021e0 2024-03-19T01:07:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:50 smithi163 ceph-mon[67428]: pgmap v332: 129 pgs: 129 active+clean; 583 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 3 op/s 2024-03-19T01:07:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:50 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] up:active 2024-03-19T01:07:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:50 smithi163 ceph-mon[67428]: daemon mds.d assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-19T01:07:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:50 smithi163 ceph-mon[67428]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-19T01:07:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:50 smithi163 ceph-mon[67428]: Cluster is now healthy 2024-03-19T01:07:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:50 smithi163 ceph-mon[67428]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-03-19T01:07:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:50 smithi163 ceph-mon[67428]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-19T01:07:51.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:50 smithi163 ceph-mon[67428]: daemon mds.d is now active in filesystem cephfs as rank 4 2024-03-19T01:07:51.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:50 smithi086 ceph-mon[66829]: pgmap v332: 129 pgs: 129 active+clean; 583 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 3 op/s 2024-03-19T01:07:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:50 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] up:active 2024-03-19T01:07:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:50 smithi086 ceph-mon[66829]: daemon mds.d assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-19T01:07:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:50 smithi086 ceph-mon[66829]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-19T01:07:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:50 smithi086 ceph-mon[66829]: Cluster is now healthy 2024-03-19T01:07:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:50 smithi086 ceph-mon[66829]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=e=up:active} 3 up:standby-replay 5 up:standby 2024-03-19T01:07:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:50 smithi086 ceph-mon[66829]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-19T01:07:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:50 smithi086 ceph-mon[66829]: daemon mds.d is now active in filesystem cephfs as rank 4 2024-03-19T01:07:51.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.315+0000 7f4e68a00640 1 -- 172.21.15.69:0/1300145160 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4e64061c60 con 0x7f4e480a2880 2024-03-19T01:07:51.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.337+0000 7f4e5be00640 1 -- 172.21.15.69:0/1300145160 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e64061c60 con 0x7f4e480a2880 2024-03-19T01:07:51.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 -- 172.21.15.69:0/1300145160 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4e480a2880 msgr2=0x7f4e480a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:51.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4e480a2880 0x7f4e480a4d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4e500095f0 tx=0x7f4e50005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:51.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 -- 172.21.15.69:0/1300145160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e641021e0 msgr2=0x7f4e64101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:51.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e641021e0 0x7f4e64101b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009230 tx=0x7f4e5402c650 comp rx=0 tx=0).stop 2024-03-19T01:07:51.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 -- 172.21.15.69:0/1300145160 shutdown_connections 2024-03-19T01:07:51.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4e480a2880 0x7f4e480a4d40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e64107440 0x7f4e641006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e64104b10 0x7f4e64100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 --2- 172.21.15.69:0/1300145160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e641021e0 0x7f4e64101b50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 -- 172.21.15.69:0/1300145160 >> 172.21.15.69:0/1300145160 conn(0x7f4e640fbb90 msgr2=0x7f4e64102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:51.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 -- 172.21.15.69:0/1300145160 shutdown_connections 2024-03-19T01:07:51.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.341+0000 7f4e68a00640 1 -- 172.21.15.69:0/1300145160 wait complete. 2024-03-19T01:07:51.368 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-19T01:07:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.590+0000 7fb32ae00640 1 -- 172.21.15.69:0/1016970666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324102910 msgr2=0x7fb324102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.590+0000 7fb32ae00640 1 --2- 172.21.15.69:0/1016970666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324102910 0x7fb324102d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb310009900 tx=0x7fb31002bc40 comp rx=0 tx=0).stop 2024-03-19T01:07:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.590+0000 7fb32ae00640 1 -- 172.21.15.69:0/1016970666 shutdown_connections 2024-03-19T01:07:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.590+0000 7fb32ae00640 1 --2- 172.21.15.69:0/1016970666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3241032d0 0x7fb324109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.590+0000 7fb32ae00640 1 --2- 172.21.15.69:0/1016970666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324102910 0x7fb324102d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.590+0000 7fb32ae00640 1 --2- 172.21.15.69:0/1016970666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324101720 0x7fb324101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.590+0000 7fb32ae00640 1 -- 172.21.15.69:0/1016970666 >> 172.21.15.69:0/1016970666 conn(0x7fb3240fcc90 msgr2=0x7fb3240ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.590+0000 7fb32ae00640 1 -- 172.21.15.69:0/1016970666 shutdown_connections 2024-03-19T01:07:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.590+0000 7fb32ae00640 1 -- 172.21.15.69:0/1016970666 wait complete. 2024-03-19T01:07:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.591+0000 7fb32ae00640 1 Processor -- start 2024-03-19T01:07:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.591+0000 7fb32ae00640 1 -- start start 2024-03-19T01:07:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.592+0000 7fb32ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324101720 0x7fb3240755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.592+0000 7fb32ae00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324102910 0x7fb324073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.592+0000 7fb32ae00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3241032d0 0x7fb324074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.592+0000 7fb32ae00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb32410d3d0 con 0x7fb324102910 2024-03-19T01:07:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.592+0000 7fb32ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb32410d250 con 0x7fb324101720 2024-03-19T01:07:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.592+0000 7fb32ae00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb32410d550 con 0x7fb3241032d0 2024-03-19T01:07:51.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.592+0000 7fb329a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324101720 0x7fb3240755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:51.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.592+0000 7fb329a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324101720 0x7fb3240755d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44962/0 (socket says 172.21.15.69:44962) 2024-03-19T01:07:51.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.592+0000 7fb329a00640 1 -- 172.21.15.69:0/503835920 learned_addr learned my addr 172.21.15.69:0/503835920 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:51.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.592+0000 7fb32a400640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3241032d0 0x7fb324074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:51.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.592+0000 7fb329000640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324102910 0x7fb324073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:51.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb329a00640 1 -- 172.21.15.69:0/503835920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3241032d0 msgr2=0x7fb324074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:51.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb329a00640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3241032d0 0x7fb324074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb329a00640 1 -- 172.21.15.69:0/503835920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324102910 msgr2=0x7fb324073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:51.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb329a00640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324102910 0x7fb324073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb329a00640 1 -- 172.21.15.69:0/503835920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3240749e0 con 0x7fb324101720 2024-03-19T01:07:51.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb329000640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324102910 0x7fb324073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:51.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb329a00640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324101720 0x7fb3240755d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb31800e9c0 tx=0x7fb31800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:51.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb322000640 1 -- 172.21.15.69:0/503835920 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31800cd80 con 0x7fb324101720 2024-03-19T01:07:51.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb322000640 1 -- 172.21.15.69:0/503835920 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3180035f0 con 0x7fb324101720 2024-03-19T01:07:51.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb32ae00640 1 -- 172.21.15.69:0/503835920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3241a1b60 con 0x7fb324101720 2024-03-19T01:07:51.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb322000640 1 -- 172.21.15.69:0/503835920 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318010690 con 0x7fb324101720 2024-03-19T01:07:51.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.593+0000 7fb32ae00640 1 -- 172.21.15.69:0/503835920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3241a20d0 con 0x7fb324101720 2024-03-19T01:07:51.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.594+0000 7fb32ae00640 1 -- 172.21.15.69:0/503835920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb324108a40 con 0x7fb324101720 2024-03-19T01:07:51.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.597+0000 7fb322000640 1 -- 172.21.15.69:0/503835920 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb3180037a0 con 0x7fb324101720 2024-03-19T01:07:51.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.599+0000 7fb322000640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb30c0a2880 0x7fb30c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:51.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.599+0000 7fb322000640 1 -- 172.21.15.69:0/503835920 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb3180cf140 con 0x7fb324101720 2024-03-19T01:07:51.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.599+0000 7fb329000640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb30c0a2880 0x7fb30c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:51.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.600+0000 7fb329000640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb30c0a2880 0x7fb30c0a4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb31002c250 tx=0x7fb310005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:51.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.606+0000 7fb322000640 1 -- 172.21.15.69:0/503835920 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb31809bf40 con 0x7fb324101720 2024-03-19T01:07:51.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.834+0000 7fb322000640 1 -- 172.21.15.69:0/503835920 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb318097eb0 con 0x7fb324101720 2024-03-19T01:07:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.862+0000 7fb32ae00640 1 -- 172.21.15.69:0/503835920 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fb324061c60 con 0x7fb30c0a2880 2024-03-19T01:07:51.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.868+0000 7fb322000640 1 -- 172.21.15.69:0/503835920 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb324061c60 con 0x7fb30c0a2880 2024-03-19T01:07:51.869 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/qa/sv_0/45a159b3-d795-4647-9a75-8f07f6fe6e9c 2024-03-19T01:07:51.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.870+0000 7fb32ae00640 1 -- 172.21.15.69:0/503835920 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb30c0a2880 msgr2=0x7fb30c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:51.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.870+0000 7fb32ae00640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb30c0a2880 0x7fb30c0a4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb31002c250 tx=0x7fb310005fd0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.870+0000 7fb32ae00640 1 -- 172.21.15.69:0/503835920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324101720 msgr2=0x7fb3240755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:51.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.870+0000 7fb32ae00640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324101720 0x7fb3240755d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb31800e9c0 tx=0x7fb31800ee80 comp rx=0 tx=0).stop 2024-03-19T01:07:51.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.870+0000 7fb32ae00640 1 -- 172.21.15.69:0/503835920 shutdown_connections 2024-03-19T01:07:51.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.870+0000 7fb32ae00640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb30c0a2880 0x7fb30c0a4d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.870+0000 7fb32ae00640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3241032d0 0x7fb324074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.870+0000 7fb32ae00640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb324102910 0x7fb324073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.870+0000 7fb32ae00640 1 --2- 172.21.15.69:0/503835920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb324101720 0x7fb3240755d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:51.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.871+0000 7fb32ae00640 1 -- 172.21.15.69:0/503835920 >> 172.21.15.69:0/503835920 conn(0x7fb3240fcc90 msgr2=0x7fb324105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:51.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.871+0000 7fb32ae00640 1 -- 172.21.15.69:0/503835920 shutdown_connections 2024-03-19T01:07:51.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:51.871+0000 7fb32ae00640 1 -- 172.21.15.69:0/503835920 wait complete. 2024-03-19T01:07:51.889 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-19T01:07:52.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:51 smithi163 ceph-mon[67428]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:52.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:51 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] up:active 2024-03-19T01:07:52.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:51 smithi163 ceph-mon[67428]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-19T01:07:52.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:51 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:07:52.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.079+0000 7fc731e00640 1 -- 172.21.15.69:0/67458172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c074220 msgr2=0x7fc72c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:52.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.079+0000 7fc731e00640 1 --2- 172.21.15.69:0/67458172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c074220 0x7fc72c074680 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc718009900 tx=0x7fc71802bcc0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.079+0000 7fc731e00640 1 -- 172.21.15.69:0/67458172 shutdown_connections 2024-03-19T01:07:52.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.079+0000 7fc731e00640 1 --2- 172.21.15.69:0/67458172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc72c1095b0 0x7fc72c10ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.079+0000 7fc731e00640 1 --2- 172.21.15.69:0/67458172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c074220 0x7fc72c074680 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.079+0000 7fc731e00640 1 --2- 172.21.15.69:0/67458172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc72c075800 0x7fc72c073c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.079+0000 7fc731e00640 1 -- 172.21.15.69:0/67458172 >> 172.21.15.69:0/67458172 conn(0x7fc72c0fcc90 msgr2=0x7fc72c0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:52.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.079+0000 7fc731e00640 1 -- 172.21.15.69:0/67458172 shutdown_connections 2024-03-19T01:07:52.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.079+0000 7fc731e00640 1 -- 172.21.15.69:0/67458172 wait complete. 2024-03-19T01:07:52.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.080+0000 7fc731e00640 1 Processor -- start 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc731e00640 1 -- start start 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc731e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c074220 0x7fc72c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc731e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc72c075800 0x7fc72c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc731e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc72c1095b0 0x7fc72c1a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc731e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c10f580 con 0x7fc72c075800 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc731e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c10f400 con 0x7fc72c074220 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc731e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c10f700 con 0x7fc72c1095b0 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc730a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c074220 0x7fc72c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc730a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c074220 0x7fc72c199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:44986/0 (socket says 172.21.15.69:44986) 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc730a00640 1 -- 172.21.15.69:0/71173302 learned_addr learned my addr 172.21.15.69:0/71173302 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc72be00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc72c075800 0x7fc72c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.081+0000 7fc731400640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc72c1095b0 0x7fc72c1a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:52.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.082+0000 7fc730a00640 1 -- 172.21.15.69:0/71173302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc72c1095b0 msgr2=0x7fc72c1a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:52.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.082+0000 7fc730a00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc72c1095b0 0x7fc72c1a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.082+0000 7fc730a00640 1 -- 172.21.15.69:0/71173302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc72c075800 msgr2=0x7fc72c19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:52.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.082+0000 7fc730a00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc72c075800 0x7fc72c19a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.082+0000 7fc730a00640 1 -- 172.21.15.69:0/71173302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c1a1880 con 0x7fc72c074220 2024-03-19T01:07:52.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.082+0000 7fc730a00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c074220 0x7fc72c199cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc72000e9f0 tx=0x7fc72000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:52.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.082+0000 7fc729600640 1 -- 172.21.15.69:0/71173302 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72000cd80 con 0x7fc72c074220 2024-03-19T01:07:52.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.082+0000 7fc729600640 1 -- 172.21.15.69:0/71173302 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc720010910 con 0x7fc72c074220 2024-03-19T01:07:52.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.082+0000 7fc729600640 1 -- 172.21.15.69:0/71173302 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720018990 con 0x7fc72c074220 2024-03-19T01:07:52.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.082+0000 7fc731e00640 1 -- 172.21.15.69:0/71173302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72c1a1b70 con 0x7fc72c074220 2024-03-19T01:07:52.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.082+0000 7fc731e00640 1 -- 172.21.15.69:0/71173302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72c1a2070 con 0x7fc72c074220 2024-03-19T01:07:52.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.084+0000 7fc731e00640 1 -- 172.21.15.69:0/71173302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc72c108370 con 0x7fc72c074220 2024-03-19T01:07:52.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.085+0000 7fc729600640 1 -- 172.21.15.69:0/71173302 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fc720010ac0 con 0x7fc72c074220 2024-03-19T01:07:52.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.087+0000 7fc729600640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc7140a2890 0x7fc7140a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:52.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.087+0000 7fc729600640 1 -- 172.21.15.69:0/71173302 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc7200cf400 con 0x7fc72c074220 2024-03-19T01:07:52.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.087+0000 7fc72be00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc7140a2890 0x7fc7140a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:52.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.088+0000 7fc72be00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc7140a2890 0x7fc7140a4d50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc718005ab0 tx=0x7fc71802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:52.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.089+0000 7fc729600640 1 -- 172.21.15.69:0/71173302 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc72009ba50 con 0x7fc72c074220 2024-03-19T01:07:52.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:51 smithi086 ceph-mon[66829]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:51 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] up:active 2024-03-19T01:07:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:51 smithi086 ceph-mon[66829]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-19T01:07:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:51 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:07:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:51 smithi069 ceph-mon[31727]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:51 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] up:active 2024-03-19T01:07:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:51 smithi069 ceph-mon[31727]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-19T01:07:52.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:51 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:07:52.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.418+0000 7fc731e00640 1 -- 172.21.15.69:0/71173302 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc72c061cf0 con 0x7fc7140a2890 2024-03-19T01:07:52.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.447+0000 7fc729600640 1 -- 172.21.15.69:0/71173302 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc72c061cf0 con 0x7fc7140a2890 2024-03-19T01:07:52.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.449+0000 7fc731e00640 1 -- 172.21.15.69:0/71173302 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc7140a2890 msgr2=0x7fc7140a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:52.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.449+0000 7fc731e00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc7140a2890 0x7fc7140a4d50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc718005ab0 tx=0x7fc71802df90 comp rx=0 tx=0).stop 2024-03-19T01:07:52.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.450+0000 7fc731e00640 1 -- 172.21.15.69:0/71173302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c074220 msgr2=0x7fc72c199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:52.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.450+0000 7fc731e00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c074220 0x7fc72c199cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc72000e9f0 tx=0x7fc72000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.450+0000 7fc731e00640 1 -- 172.21.15.69:0/71173302 shutdown_connections 2024-03-19T01:07:52.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.450+0000 7fc731e00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc7140a2890 0x7fc7140a4d50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.450+0000 7fc731e00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc72c1095b0 0x7fc72c1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.450+0000 7fc731e00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc72c075800 0x7fc72c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.450+0000 7fc731e00640 1 --2- 172.21.15.69:0/71173302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c074220 0x7fc72c199cc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.450+0000 7fc731e00640 1 -- 172.21.15.69:0/71173302 >> 172.21.15.69:0/71173302 conn(0x7fc72c0fcc90 msgr2=0x7fc72c105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:52.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.450+0000 7fc731e00640 1 -- 172.21.15.69:0/71173302 shutdown_connections 2024-03-19T01:07:52.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.450+0000 7fc731e00640 1 -- 172.21.15.69:0/71173302 wait complete. 2024-03-19T01:07:52.467 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-19T01:07:52.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.663+0000 7fc6ed600640 1 -- 172.21.15.69:0/3828815664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8102a20 msgr2=0x7fc6e8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:52.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.663+0000 7fc6ed600640 1 --2- 172.21.15.69:0/3828815664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8102a20 0x7fc6e8102ea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009900 tx=0x7fc6d402bc30 comp rx=0 tx=0).stop 2024-03-19T01:07:52.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.663+0000 7fc6ed600640 1 -- 172.21.15.69:0/3828815664 shutdown_connections 2024-03-19T01:07:52.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.663+0000 7fc6ed600640 1 --2- 172.21.15.69:0/3828815664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6e81033e0 0x7fc6e8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.663+0000 7fc6ed600640 1 --2- 172.21.15.69:0/3828815664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8102a20 0x7fc6e8102ea0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.663+0000 7fc6ed600640 1 --2- 172.21.15.69:0/3828815664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e8101830 0x7fc6e8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.663+0000 7fc6ed600640 1 -- 172.21.15.69:0/3828815664 >> 172.21.15.69:0/3828815664 conn(0x7fc6e80fcda0 msgr2=0x7fc6e80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:52.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.663+0000 7fc6ed600640 1 -- 172.21.15.69:0/3828815664 shutdown_connections 2024-03-19T01:07:52.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.663+0000 7fc6ed600640 1 -- 172.21.15.69:0/3828815664 wait complete. 2024-03-19T01:07:52.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.664+0000 7fc6ed600640 1 Processor -- start 2024-03-19T01:07:52.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.664+0000 7fc6ed600640 1 -- start start 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6ed600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6e8101830 0x7fc6e8199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6ed600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8102a20 0x7fc6e819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6ed600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e81033e0 0x7fc6e81a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6ed600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e810d500 con 0x7fc6e81033e0 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6ed600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e810d380 con 0x7fc6e8102a20 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6ed600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e810d680 con 0x7fc6e8101830 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6e6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6e8101830 0x7fc6e8199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6e6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8102a20 0x7fc6e819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6e6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6e8101830 0x7fc6e8199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53816/0 (socket says 172.21.15.69:53816) 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6e6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8102a20 0x7fc6e819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45006/0 (socket says 172.21.15.69:45006) 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6e6a00640 1 -- 172.21.15.69:0/4033429079 learned_addr learned my addr 172.21.15.69:0/4033429079 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6e7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e81033e0 0x7fc6e81a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:52.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6e6a00640 1 -- 172.21.15.69:0/4033429079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8102a20 msgr2=0x7fc6e819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:52.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6e6a00640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8102a20 0x7fc6e819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6e6a00640 1 -- 172.21.15.69:0/4033429079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e81033e0 msgr2=0x7fc6e81a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:52.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6e6a00640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e81033e0 0x7fc6e81a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.665+0000 7fc6e6a00640 1 -- 172.21.15.69:0/4033429079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e81a19c0 con 0x7fc6e8101830 2024-03-19T01:07:52.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.666+0000 7fc6e6a00640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6e8101830 0x7fc6e8199db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800e9c0 tx=0x7fc6d800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:52.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.666+0000 7fc6df400640 1 -- 172.21.15.69:0/4033429079 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d800cde0 con 0x7fc6e8101830 2024-03-19T01:07:52.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.666+0000 7fc6ed600640 1 -- 172.21.15.69:0/4033429079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e81a1cb0 con 0x7fc6e8101830 2024-03-19T01:07:52.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.666+0000 7fc6df400640 1 -- 172.21.15.69:0/4033429079 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d8010910 con 0x7fc6e8101830 2024-03-19T01:07:52.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.666+0000 7fc6df400640 1 -- 172.21.15.69:0/4033429079 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8018a10 con 0x7fc6e8101830 2024-03-19T01:07:52.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.666+0000 7fc6ed600640 1 -- 172.21.15.69:0/4033429079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e81a21b0 con 0x7fc6e8101830 2024-03-19T01:07:52.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.668+0000 7fc6dcc00640 1 -- 172.21.15.69:0/4033429079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e8062420 con 0x7fc6e8101830 2024-03-19T01:07:52.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.669+0000 7fc6df400640 1 -- 172.21.15.69:0/4033429079 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fc6d801f020 con 0x7fc6e8101830 2024-03-19T01:07:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.670+0000 7fc6df400640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6cc0a2890 0x7fc6cc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.670+0000 7fc6df400640 1 -- 172.21.15.69:0/4033429079 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc6d80cf0e0 con 0x7fc6e8101830 2024-03-19T01:07:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.670+0000 7fc6e6000640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6cc0a2890 0x7fc6cc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.671+0000 7fc6e6000640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6cc0a2890 0x7fc6cc0a4d50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc6d402c240 tx=0x7fc6d402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.674+0000 7fc6df400640 1 -- 172.21.15.69:0/4033429079 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc6d809bd80 con 0x7fc6e8101830 2024-03-19T01:07:52.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.915+0000 7fc6dcc00640 1 -- 172.21.15.69:0/4033429079 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fc6e80626d0 con 0x7fc6cc0a2890 2024-03-19T01:07:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.920+0000 7fc6df400640 1 -- 172.21.15.69:0/4033429079 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc6e80626d0 con 0x7fc6cc0a2890 2024-03-19T01:07:52.921 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/qa/sv_1/553c1530-b025-47e9-a3bd-4c853425f8e3 2024-03-19T01:07:52.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 -- 172.21.15.69:0/4033429079 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6cc0a2890 msgr2=0x7fc6cc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:52.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6cc0a2890 0x7fc6cc0a4d50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc6d402c240 tx=0x7fc6d402df90 comp rx=0 tx=0).stop 2024-03-19T01:07:52.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 -- 172.21.15.69:0/4033429079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6e8101830 msgr2=0x7fc6e8199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:52.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6e8101830 0x7fc6e8199db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800e9c0 tx=0x7fc6d800ee80 comp rx=0 tx=0).stop 2024-03-19T01:07:52.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 -- 172.21.15.69:0/4033429079 shutdown_connections 2024-03-19T01:07:52.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6cc0a2890 0x7fc6cc0a4d50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e81033e0 0x7fc6e81a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6e8102a20 0x7fc6e819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 --2- 172.21.15.69:0/4033429079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6e8101830 0x7fc6e8199db0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 -- 172.21.15.69:0/4033429079 >> 172.21.15.69:0/4033429079 conn(0x7fc6e80fcda0 msgr2=0x7fc6e8073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 -- 172.21.15.69:0/4033429079 shutdown_connections 2024-03-19T01:07:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:52.922+0000 7fc6dcc00640 1 -- 172.21.15.69:0/4033429079 wait complete. 2024-03-19T01:07:52.940 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:07:53.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.137+0000 7f85a4200640 1 -- 172.21.15.69:0/1698528391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f859c105610 msgr2=0x7f859c10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:53.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.137+0000 7f85a4200640 1 --2- 172.21.15.69:0/1698528391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f859c105610 0x7f859c10bea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8594009ca0 tx=0x7f859402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.138+0000 7f85a4200640 1 -- 172.21.15.69:0/1698528391 shutdown_connections 2024-03-19T01:07:53.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.138+0000 7f85a4200640 1 --2- 172.21.15.69:0/1698528391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f859c105610 0x7f859c10bea0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.138+0000 7f85a4200640 1 --2- 172.21.15.69:0/1698528391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f859c104c50 0x7f859c1050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.138+0000 7f85a4200640 1 --2- 172.21.15.69:0/1698528391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f859c103a60 0x7f859c103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.138+0000 7f85a4200640 1 -- 172.21.15.69:0/1698528391 >> 172.21.15.69:0/1698528391 conn(0x7f859c0fefd0 msgr2=0x7f859c101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:53.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.138+0000 7f85a4200640 1 -- 172.21.15.69:0/1698528391 shutdown_connections 2024-03-19T01:07:53.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.138+0000 7f85a4200640 1 -- 172.21.15.69:0/1698528391 wait complete. 2024-03-19T01:07:53.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.139+0000 7f85a4200640 1 Processor -- start 2024-03-19T01:07:53.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.139+0000 7f85a4200640 1 -- start start 2024-03-19T01:07:53.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a4200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f859c103a60 0x7f859c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:53.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a4200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f859c104c50 0x7f859c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:53.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a4200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f859c105610 0x7f859c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:53.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a4200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10f6c0 con 0x7f859c103a60 2024-03-19T01:07:53.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a4200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10f540 con 0x7f859c104c50 2024-03-19T01:07:53.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a4200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10f840 con 0x7f859c105610 2024-03-19T01:07:53.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f859c104c50 0x7f859c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a1800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f859c103a60 0x7f859c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f859c104c50 0x7f859c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45024/0 (socket says 172.21.15.69:45024) 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a1800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f859c103a60 0x7f859c199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53776/0 (socket says 172.21.15.69:53776) 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a0e00640 1 -- 172.21.15.69:0/3875964347 learned_addr learned my addr 172.21.15.69:0/3875964347 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a2200640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f859c105610 0x7f859c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a0e00640 1 -- 172.21.15.69:0/3875964347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f859c105610 msgr2=0x7f859c1a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a0e00640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f859c105610 0x7f859c1a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a0e00640 1 -- 172.21.15.69:0/3875964347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f859c103a60 msgr2=0x7f859c199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a0e00640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f859c103a60 0x7f859c199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a0e00640 1 -- 172.21.15.69:0/3875964347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c1a1930 con 0x7f859c104c50 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a2200640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f859c105610 0x7f859c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.140+0000 7f85a1800640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f859c103a60 0x7f859c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.141+0000 7f85a0e00640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f859c104c50 0x7f859c19a260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f858800e9f0 tx=0x7f858800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:53.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.141+0000 7f859a000640 1 -- 172.21.15.69:0/3875964347 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858800cd80 con 0x7f859c104c50 2024-03-19T01:07:53.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.141+0000 7f859a000640 1 -- 172.21.15.69:0/3875964347 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8588010910 con 0x7f859c104c50 2024-03-19T01:07:53.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.141+0000 7f85a4200640 1 -- 172.21.15.69:0/3875964347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859c1a1c20 con 0x7f859c104c50 2024-03-19T01:07:53.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.141+0000 7f859a000640 1 -- 172.21.15.69:0/3875964347 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588018990 con 0x7f859c104c50 2024-03-19T01:07:53.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.141+0000 7f85a4200640 1 -- 172.21.15.69:0/3875964347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f859c1a21c0 con 0x7f859c104c50 2024-03-19T01:07:53.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.142+0000 7f85a4200640 1 -- 172.21.15.69:0/3875964347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859c10ad80 con 0x7f859c104c50 2024-03-19T01:07:53.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.145+0000 7f859a000640 1 -- 172.21.15.69:0/3875964347 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f8588010ac0 con 0x7f859c104c50 2024-03-19T01:07:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.146+0000 7f859a000640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85840a2910 0x7f85840a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.146+0000 7f859a000640 1 -- 172.21.15.69:0/3875964347 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f85880cf1d0 con 0x7f859c104c50 2024-03-19T01:07:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.146+0000 7f85a1800640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85840a2910 0x7f85840a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.147+0000 7f85a1800640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85840a2910 0x7f85840a4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f859c19b1f0 tx=0x7f8590005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:53.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.150+0000 7f859a000640 1 -- 172.21.15.69:0/3875964347 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f858809b820 con 0x7f859c104c50 2024-03-19T01:07:53.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:52 smithi086 ceph-mon[66829]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:52 smithi086 ceph-mon[66829]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-19T01:07:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:52 smithi086 ceph-mon[66829]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-19T01:07:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:52 smithi086 ceph-mon[66829]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:52 smithi069 ceph-mon[31727]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:52 smithi069 ceph-mon[31727]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-19T01:07:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:52 smithi069 ceph-mon[31727]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-19T01:07:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:52 smithi069 ceph-mon[31727]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:52 smithi163 ceph-mon[67428]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:52 smithi163 ceph-mon[67428]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-19T01:07:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:52 smithi163 ceph-mon[67428]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-19T01:07:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:52 smithi163 ceph-mon[67428]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:53.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.393+0000 7f85a4200640 1 -- 172.21.15.69:0/3875964347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f859c02d1f0 con 0x7f859c104c50 2024-03-19T01:07:53.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.394+0000 7f859a000640 1 -- 172.21.15.69:0/3875964347 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7f858809f8c0 con 0x7f859c104c50 2024-03-19T01:07:53.395 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:53.395 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:07:50.808640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581,24476,24580,24494,24550]},"id":1}]} 2024-03-19T01:07:53.396 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-03-19T01:07:53.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.397+0000 7f85a4200640 1 -- 172.21.15.69:0/3875964347 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85840a2910 msgr2=0x7f85840a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:53.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.397+0000 7f85a4200640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85840a2910 0x7f85840a4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f859c19b1f0 tx=0x7f8590005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:53.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.397+0000 7f85a4200640 1 -- 172.21.15.69:0/3875964347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f859c104c50 msgr2=0x7f859c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:53.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.397+0000 7f85a4200640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f859c104c50 0x7f859c19a260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f858800e9f0 tx=0x7f858800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.397+0000 7f85a4200640 1 -- 172.21.15.69:0/3875964347 shutdown_connections 2024-03-19T01:07:53.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.397+0000 7f85a4200640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85840a2910 0x7f85840a4dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.397+0000 7f85a4200640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f859c105610 0x7f859c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.397+0000 7f85a4200640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f859c104c50 0x7f859c19a260 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.397+0000 7f85a4200640 1 --2- 172.21.15.69:0/3875964347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f859c103a60 0x7f859c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.397+0000 7f85a4200640 1 -- 172.21.15.69:0/3875964347 >> 172.21.15.69:0/3875964347 conn(0x7f859c0fefd0 msgr2=0x7f859c107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:53.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.398+0000 7f85a4200640 1 -- 172.21.15.69:0/3875964347 shutdown_connections 2024-03-19T01:07:53.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.398+0000 7f85a4200640 1 -- 172.21.15.69:0/3875964347 wait complete. 2024-03-19T01:07:53.426 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:07:53.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.634+0000 7fc54b200640 1 -- 172.21.15.69:0/273150788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c103230 msgr2=0x7fc54c109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:53.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.634+0000 7fc54b200640 1 --2- 172.21.15.69:0/273150788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c103230 0x7fc54c109ac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc544009be0 tx=0x7fc54402bef0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.634+0000 7fc54b200640 1 -- 172.21.15.69:0/273150788 shutdown_connections 2024-03-19T01:07:53.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.634+0000 7fc54b200640 1 --2- 172.21.15.69:0/273150788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c103230 0x7fc54c109ac0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.634+0000 7fc54b200640 1 --2- 172.21.15.69:0/273150788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c102870 0x7fc54c102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.634+0000 7fc54b200640 1 --2- 172.21.15.69:0/273150788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc54c101680 0x7fc54c101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.634+0000 7fc54b200640 1 -- 172.21.15.69:0/273150788 >> 172.21.15.69:0/273150788 conn(0x7fc54c0fcbf0 msgr2=0x7fc54c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:53.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.634+0000 7fc54b200640 1 -- 172.21.15.69:0/273150788 shutdown_connections 2024-03-19T01:07:53.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.634+0000 7fc54b200640 1 -- 172.21.15.69:0/273150788 wait complete. 2024-03-19T01:07:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.635+0000 7fc54b200640 1 Processor -- start 2024-03-19T01:07:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc54b200640 1 -- start start 2024-03-19T01:07:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc54b200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc54c101680 0x7fc54c197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc54b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c102870 0x7fc54c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc54b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c103230 0x7fc54c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc54b200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c10d350 con 0x7fc54c101680 2024-03-19T01:07:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc54b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c10d1d0 con 0x7fc54c103230 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc54b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c10d4d0 con 0x7fc54c102870 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc549e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc54c101680 0x7fc54c197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc549e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc54c101680 0x7fc54c197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53788/0 (socket says 172.21.15.69:53788) 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc549e00640 1 -- 172.21.15.69:0/841909650 learned_addr learned my addr 172.21.15.69:0/841909650 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc541400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c102870 0x7fc54c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.636+0000 7fc54a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c103230 0x7fc54c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.637+0000 7fc54a800640 1 -- 172.21.15.69:0/841909650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c102870 msgr2=0x7fc54c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.637+0000 7fc54a800640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c102870 0x7fc54c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.637+0000 7fc54a800640 1 -- 172.21.15.69:0/841909650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc54c101680 msgr2=0x7fc54c197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.637+0000 7fc54a800640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc54c101680 0x7fc54c197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.637+0000 7fc54a800640 1 -- 172.21.15.69:0/841909650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c19f550 con 0x7fc54c103230 2024-03-19T01:07:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.637+0000 7fc549e00640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc54c101680 0x7fc54c197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:53.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.637+0000 7fc54a800640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c103230 0x7fc54c19eee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc54402c430 tx=0x7fc5440337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:53.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.637+0000 7fc543400640 1 -- 172.21.15.69:0/841909650 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544043020 con 0x7fc54c103230 2024-03-19T01:07:53.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.638+0000 7fc543400640 1 -- 172.21.15.69:0/841909650 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc54402ca60 con 0x7fc54c103230 2024-03-19T01:07:53.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.638+0000 7fc54b200640 1 -- 172.21.15.69:0/841909650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c19f7e0 con 0x7fc54c103230 2024-03-19T01:07:53.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.638+0000 7fc543400640 1 -- 172.21.15.69:0/841909650 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544034690 con 0x7fc54c103230 2024-03-19T01:07:53.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.638+0000 7fc54b200640 1 -- 172.21.15.69:0/841909650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c19fd80 con 0x7fc54c103230 2024-03-19T01:07:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.639+0000 7fc54b200640 1 -- 172.21.15.69:0/841909650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c1089a0 con 0x7fc54c103230 2024-03-19T01:07:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.640+0000 7fc543400640 1 -- 172.21.15.69:0/841909650 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fc544046020 con 0x7fc54c103230 2024-03-19T01:07:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.642+0000 7fc543400640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc52c0a2910 0x7fc52c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.642+0000 7fc543400640 1 -- 172.21.15.69:0/841909650 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc5440f12b0 con 0x7fc54c103230 2024-03-19T01:07:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.642+0000 7fc549e00640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc52c0a2910 0x7fc52c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.643+0000 7fc549e00640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc52c0a2910 0x7fc52c0a4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0058c0 tx=0x7fc53c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.646+0000 7fc543400640 1 -- 172.21.15.69:0/841909650 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc54404d020 con 0x7fc54c103230 2024-03-19T01:07:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.882+0000 7fc54b200640 1 -- 172.21.15.69:0/841909650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc54c062420 con 0x7fc54c103230 2024-03-19T01:07:53.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.884+0000 7fc543400640 1 -- 172.21.15.69:0/841909650 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24069 (secure 0 0 0) 0x7fc5440be0c0 con 0x7fc54c103230 2024-03-19T01:07:53.886 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:53.886 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":89,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:07:20.123357+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:07:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 -- 172.21.15.69:0/841909650 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc52c0a2910 msgr2=0x7fc52c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc52c0a2910 0x7fc52c0a4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0058c0 tx=0x7fc53c005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 -- 172.21.15.69:0/841909650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c103230 msgr2=0x7fc54c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c103230 0x7fc54c19eee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc54402c430 tx=0x7fc5440337b0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 -- 172.21.15.69:0/841909650 shutdown_connections 2024-03-19T01:07:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc52c0a2910 0x7fc52c0a4dd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc54c103230 0x7fc54c19eee0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c102870 0x7fc54c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 --2- 172.21.15.69:0/841909650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc54c101680 0x7fc54c197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 -- 172.21.15.69:0/841909650 >> 172.21.15.69:0/841909650 conn(0x7fc54c0fcbf0 msgr2=0x7fc54c105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 -- 172.21.15.69:0/841909650 shutdown_connections 2024-03-19T01:07:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:53.888+0000 7fc54b200640 1 -- 172.21.15.69:0/841909650 wait complete. 2024-03-19T01:07:53.909 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-19T01:07:53.910 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T01:07:54.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.129+0000 7f25bb200640 1 -- 172.21.15.69:0/2131137803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b4077420 msgr2=0x7f25b4077880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:54.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.129+0000 7f25bb200640 1 --2- 172.21.15.69:0/2131137803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b4077420 0x7f25b4077880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009900 tx=0x7f25a402bc40 comp rx=0 tx=0).stop 2024-03-19T01:07:54.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.129+0000 7f25bb200640 1 -- 172.21.15.69:0/2131137803 shutdown_connections 2024-03-19T01:07:54.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.129+0000 7f25bb200640 1 --2- 172.21.15.69:0/2131137803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b4077dc0 0x7f25b4078660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.129+0000 7f25bb200640 1 --2- 172.21.15.69:0/2131137803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b4077420 0x7f25b4077880 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.129+0000 7f25bb200640 1 --2- 172.21.15.69:0/2131137803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b40761e0 0x7f25b4076600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.129+0000 7f25bb200640 1 -- 172.21.15.69:0/2131137803 >> 172.21.15.69:0/2131137803 conn(0x7f25b40fee00 msgr2=0x7f25b4101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:54.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.129+0000 7f25bb200640 1 -- 172.21.15.69:0/2131137803 shutdown_connections 2024-03-19T01:07:54.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.129+0000 7f25bb200640 1 -- 172.21.15.69:0/2131137803 wait complete. 2024-03-19T01:07:54.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.130+0000 7f25bb200640 1 Processor -- start 2024-03-19T01:07:54.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.130+0000 7f25bb200640 1 -- start start 2024-03-19T01:07:54.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.130+0000 7f25bb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b40761e0 0x7f25b419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:54.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.130+0000 7f25bb200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b4077420 0x7f25b419e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:54.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.130+0000 7f25bb200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b4077dc0 0x7f25b41a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:54.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.130+0000 7f25bb200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f25b4113900 con 0x7f25b4077dc0 2024-03-19T01:07:54.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.130+0000 7f25bb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25b4113780 con 0x7f25b40761e0 2024-03-19T01:07:54.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.130+0000 7f25bb200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f25b4113a80 con 0x7f25b4077420 2024-03-19T01:07:54.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.130+0000 7f25b9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b40761e0 0x7f25b419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:54.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.130+0000 7f25b9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b40761e0 0x7f25b419df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45066/0 (socket says 172.21.15.69:45066) 2024-03-19T01:07:54.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25b9e00640 1 -- 172.21.15.69:0/1550004374 learned_addr learned my addr 172.21.15.69:0/1550004374 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25b9400640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b4077420 0x7f25b419e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25ba800640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b4077dc0 0x7f25b41a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25b9e00640 1 -- 172.21.15.69:0/1550004374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b4077420 msgr2=0x7f25b419e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25b9e00640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b4077420 0x7f25b419e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25b9e00640 1 -- 172.21.15.69:0/1550004374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b4077dc0 msgr2=0x7f25b41a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25b9e00640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b4077dc0 0x7f25b41a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25b9e00640 1 -- 172.21.15.69:0/1550004374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b41a5b70 con 0x7f25b40761e0 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25ba800640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b4077dc0 0x7f25b41a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25b9e00640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b40761e0 0x7f25b419df50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f25a800e9f0 tx=0x7f25a800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25aea00640 1 -- 172.21.15.69:0/1550004374 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a800cd80 con 0x7f25b40761e0 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25aea00640 1 -- 172.21.15.69:0/1550004374 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25a8005500 con 0x7f25b40761e0 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.131+0000 7f25aea00640 1 -- 172.21.15.69:0/1550004374 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8010690 con 0x7f25b40761e0 2024-03-19T01:07:54.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.132+0000 7f25bb200640 1 -- 172.21.15.69:0/1550004374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b41a5e60 con 0x7f25b40761e0 2024-03-19T01:07:54.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.132+0000 7f25bb200640 1 -- 172.21.15.69:0/1550004374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b41a6270 con 0x7f25b40761e0 2024-03-19T01:07:54.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.133+0000 7f25bb200640 1 -- 172.21.15.69:0/1550004374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b4110370 con 0x7f25b40761e0 2024-03-19T01:07:54.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.136+0000 7f25aea00640 1 -- 172.21.15.69:0/1550004374 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f25a80046c0 con 0x7f25b40761e0 2024-03-19T01:07:54.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.137+0000 7f25aea00640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f259c0a2890 0x7f259c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:54.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.137+0000 7f25aea00640 1 -- 172.21.15.69:0/1550004374 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f25a80cf300 con 0x7f25b40761e0 2024-03-19T01:07:54.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.138+0000 7f25b9400640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f259c0a2890 0x7f259c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:54.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.138+0000 7f25aea00640 1 -- 172.21.15.69:0/1550004374 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f25a809bfa0 con 0x7f25b40761e0 2024-03-19T01:07:54.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.139+0000 7f25b9400640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f259c0a2890 0x7f259c0a4d50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f25a402c7c0 tx=0x7f25a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:54.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:53 smithi086 ceph-mon[66829]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:54.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:53 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3875964347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:53 smithi069 ceph-mon[31727]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:53 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3875964347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:53 smithi163 ceph-mon[67428]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:53 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3875964347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:07:54.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.335+0000 7f25bb200640 1 -- 172.21.15.69:0/1550004374 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25b4061c60 con 0x7f259c0a2890 2024-03-19T01:07:54.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.360+0000 7f25aea00640 1 -- 172.21.15.69:0/1550004374 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340672 (secure 0 0 0) 0x7f25b4061c60 con 0x7f259c0a2890 2024-03-19T01:07:54.367 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:54.370 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-03-19T01:07:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 -- 172.21.15.69:0/1550004374 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f259c0a2890 msgr2=0x7f259c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f259c0a2890 0x7f259c0a4d50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f25a402c7c0 tx=0x7f25a4005fd0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 -- 172.21.15.69:0/1550004374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b40761e0 msgr2=0x7f25b419df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b40761e0 0x7f25b419df50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f25a800e9f0 tx=0x7f25a800edf0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 -- 172.21.15.69:0/1550004374 shutdown_connections 2024-03-19T01:07:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f259c0a2890 0x7f259c0a4d50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b4077dc0 0x7f25b41a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b4077420 0x7f25b419e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 --2- 172.21.15.69:0/1550004374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b40761e0 0x7f25b419df50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 -- 172.21.15.69:0/1550004374 >> 172.21.15.69:0/1550004374 conn(0x7f25b40fee00 msgr2=0x7f25b41004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:54.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 -- 172.21.15.69:0/1550004374 shutdown_connections 2024-03-19T01:07:54.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.365+0000 7f25bb200640 1 -- 172.21.15.69:0/1550004374 wait complete. 2024-03-19T01:07:54.383 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-03-19T01:07:53.653572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620556,"num_objects":101,"num_object_clones":0,"num_object_copies":305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":200,"num_read_kb":189,"num_write":402,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369328,"kb_used_data":11856,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572496,"statfs":{"total":1151940427776,"available":1151562235904,"internally_reserved":0,"allocated":12140544,"data_stored":7587926,"data_compressed":282810,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25689,"num_objects":77,"num_object_clones":0,"num_object_copies":233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":64,"num_read_kb":70,"num_write":173,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005965"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249707+0000","last_change":"2024-03-19T01:07:15.500098+0000","last_active":"2024-03-19T01:07:21.249707+0000","last_peered":"2024-03-19T01:07:21.249707+0000","last_clean":"2024-03-19T01:07:21.249707+0000","last_became_active":"2024-03-19T01:07:15.499738+0000","last_became_peered":"2024-03-19T01:07:15.499738+0000","last_unstale":"2024-03-19T01:07:21.249707+0000","last_undegraded":"2024-03-19T01:07:21.249707+0000","last_fullsized":"2024-03-19T01:07:21.249707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:14.703153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839414+0000","last_change":"2024-03-19T01:07:15.492863+0000","last_active":"2024-03-19T01:07:20.839414+0000","last_peered":"2024-03-19T01:07:20.839414+0000","last_clean":"2024-03-19T01:07:20.839414+0000","last_became_active":"2024-03-19T01:07:15.492533+0000","last_became_peered":"2024-03-19T01:07:15.492533+0000","last_unstale":"2024-03-19T01:07:20.839414+0000","last_undegraded":"2024-03-19T01:07:20.839414+0000","last_fullsized":"2024-03-19T01:07:20.839414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:49.028352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.838935+0000","last_change":"2024-03-19T01:07:15.494174+0000","last_active":"2024-03-19T01:07:20.838935+0000","last_peered":"2024-03-19T01:07:20.838935+0000","last_clean":"2024-03-19T01:07:20.838935+0000","last_became_active":"2024-03-19T01:07:15.490562+0000","last_became_peered":"2024-03-19T01:07:15.490562+0000","last_unstale":"2024-03-19T01:07:20.838935+0000","last_undegraded":"2024-03-19T01:07:20.838935+0000","last_fullsized":"2024-03-19T01:07:20.838935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:18.664396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247974+0000","last_change":"2024-03-19T01:07:15.478397+0000","last_active":"2024-03-19T01:07:21.247974+0000","last_peered":"2024-03-19T01:07:21.247974+0000","last_clean":"2024-03-19T01:07:21.247974+0000","last_became_active":"2024-03-19T01:07:15.477000+0000","last_became_peered":"2024-03-19T01:07:15.477000+0000","last_unstale":"2024-03-19T01:07:21.247974+0000","last_undegraded":"2024-03-19T01:07:21.247974+0000","last_fullsized":"2024-03-19T01:07:21.247974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:58.534329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247754+0000","last_change":"2024-03-19T01:07:15.500138+0000","last_active":"2024-03-19T01:07:21.247754+0000","last_peered":"2024-03-19T01:07:21.247754+0000","last_clean":"2024-03-19T01:07:21.247754+0000","last_became_active":"2024-03-19T01:07:15.499759+0000","last_became_peered":"2024-03-19T01:07:15.499759+0000","last_unstale":"2024-03-19T01:07:21.247754+0000","last_undegraded":"2024-03-19T01:07:21.247754+0000","last_fullsized":"2024-03-19T01:07:21.247754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:23.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247576+0000","last_change":"2024-03-19T01:07:15.485146+0000","last_active":"2024-03-19T01:07:21.247576+0000","last_peered":"2024-03-19T01:07:21.247576+0000","last_clean":"2024-03-19T01:07:21.247576+0000","last_became_active":"2024-03-19T01:07:15.484680+0000","last_became_peered":"2024-03-19T01:07:15.484680+0000","last_unstale":"2024-03-19T01:07:21.247576+0000","last_undegraded":"2024-03-19T01:07:21.247576+0000","last_fullsized":"2024-03-19T01:07:21.247576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:12.659347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679127+0000","last_change":"2024-03-19T01:07:15.486327+0000","last_active":"2024-03-19T01:07:20.679127+0000","last_peered":"2024-03-19T01:07:20.679127+0000","last_clean":"2024-03-19T01:07:20.679127+0000","last_became_active":"2024-03-19T01:07:15.485954+0000","last_became_peered":"2024-03-19T01:07:15.485954+0000","last_unstale":"2024-03-19T01:07:20.679127+0000","last_undegraded":"2024-03-19T01:07:20.679127+0000","last_fullsized":"2024-03-19T01:07:20.679127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:53.316260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247929+0000","last_change":"2024-03-19T01:07:15.504993+0000","last_active":"2024-03-19T01:07:21.247929+0000","last_peered":"2024-03-19T01:07:21.247929+0000","last_clean":"2024-03-19T01:07:21.247929+0000","last_became_active":"2024-03-19T01:07:15.502771+0000","last_became_peered":"2024-03-19T01:07:15.502771+0000","last_unstale":"2024-03-19T01:07:21.247929+0000","last_undegraded":"2024-03-19T01:07:21.247929+0000","last_fullsized":"2024-03-19T01:07:21.247929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:22:33.712608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249048+0000","last_change":"2024-03-19T01:07:15.483876+0000","last_active":"2024-03-19T01:07:21.249048+0000","last_peered":"2024-03-19T01:07:21.249048+0000","last_clean":"2024-03-19T01:07:21.249048+0000","last_became_active":"2024-03-19T01:07:15.482744+0000","last_became_peered":"2024-03-19T01:07:15.482744+0000","last_unstale":"2024-03-19T01:07:21.249048+0000","last_undegraded":"2024-03-19T01:07:21.249048+0000","last_fullsized":"2024-03-19T01:07:21.249048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:21:19.870877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248538+0000","last_change":"2024-03-19T01:07:15.475655+0000","last_active":"2024-03-19T01:07:21.248538+0000","last_peered":"2024-03-19T01:07:21.248538+0000","last_clean":"2024-03-19T01:07:21.248538+0000","last_became_active":"2024-03-19T01:07:15.475265+0000","last_became_peered":"2024-03-19T01:07:15.475265+0000","last_unstale":"2024-03-19T01:07:21.248538+0000","last_undegraded":"2024-03-19T01:07:21.248538+0000","last_fullsized":"2024-03-19T01:07:21.248538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:24:38.898191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679401+0000","last_change":"2024-03-19T01:07:15.482988+0000","last_active":"2024-03-19T01:07:20.679401+0000","last_peered":"2024-03-19T01:07:20.679401+0000","last_clean":"2024-03-19T01:07:20.679401+0000","last_became_active":"2024-03-19T01:07:15.482673+0000","last_became_peered":"2024-03-19T01:07:15.482673+0000","last_unstale":"2024-03-19T01:07:20.679401+0000","last_undegraded":"2024-03-19T01:07:20.679401+0000","last_fullsized":"2024-03-19T01:07:20.679401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:15:48.807200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247876+0000","last_change":"2024-03-19T01:07:15.479593+0000","last_active":"2024-03-19T01:07:21.247876+0000","last_peered":"2024-03-19T01:07:21.247876+0000","last_clean":"2024-03-19T01:07:21.247876+0000","last_became_active":"2024-03-19T01:07:15.479220+0000","last_became_peered":"2024-03-19T01:07:15.479220+0000","last_unstale":"2024-03-19T01:07:21.247876+0000","last_undegraded":"2024-03-19T01:07:21.247876+0000","last_fullsized":"2024-03-19T01:07:21.247876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:40:24.390845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.748918+0000","last_change":"2024-03-19T01:07:15.464737+0000","last_active":"2024-03-19T01:07:20.748918+0000","last_peered":"2024-03-19T01:07:20.748918+0000","last_clean":"2024-03-19T01:07:20.748918+0000","last_became_active":"2024-03-19T01:07:15.463641+0000","last_became_peered":"2024-03-19T01:07:15.463641+0000","last_unstale":"2024-03-19T01:07:20.748918+0000","last_undegraded":"2024-03-19T01:07:20.748918+0000","last_fullsized":"2024-03-19T01:07:20.748918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:13.964397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.838612+0000","last_change":"2024-03-19T01:07:15.492803+0000","last_active":"2024-03-19T01:07:20.838612+0000","last_peered":"2024-03-19T01:07:20.838612+0000","last_clean":"2024-03-19T01:07:20.838612+0000","last_became_active":"2024-03-19T01:07:15.492422+0000","last_became_peered":"2024-03-19T01:07:15.492422+0000","last_unstale":"2024-03-19T01:07:20.838612+0000","last_undegraded":"2024-03-19T01:07:20.838612+0000","last_fullsized":"2024-03-19T01:07:20.838612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:59:08.292399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839201+0000","last_change":"2024-03-19T01:07:15.498073+0000","last_active":"2024-03-19T01:07:20.839201+0000","last_peered":"2024-03-19T01:07:20.839201+0000","last_clean":"2024-03-19T01:07:20.839201+0000","last_became_active":"2024-03-19T01:07:15.493650+0000","last_became_peered":"2024-03-19T01:07:15.493650+0000","last_unstale":"2024-03-19T01:07:20.839201+0000","last_undegraded":"2024-03-19T01:07:20.839201+0000","last_fullsized":"2024-03-19T01:07:20.839201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:25:28.908122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248189+0000","last_change":"2024-03-19T01:07:15.496434+0000","last_active":"2024-03-19T01:07:21.248189+0000","last_peered":"2024-03-19T01:07:21.248189+0000","last_clean":"2024-03-19T01:07:21.248189+0000","last_became_active":"2024-03-19T01:07:15.495956+0000","last_became_peered":"2024-03-19T01:07:15.495956+0000","last_unstale":"2024-03-19T01:07:21.248189+0000","last_undegraded":"2024-03-19T01:07:21.248189+0000","last_fullsized":"2024-03-19T01:07:21.248189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:49.505947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249606+0000","last_change":"2024-03-19T01:07:15.504771+0000","last_active":"2024-03-19T01:07:21.249606+0000","last_peered":"2024-03-19T01:07:21.249606+0000","last_clean":"2024-03-19T01:07:21.249606+0000","last_became_active":"2024-03-19T01:07:15.504485+0000","last_became_peered":"2024-03-19T01:07:15.504485+0000","last_unstale":"2024-03-19T01:07:21.249606+0000","last_undegraded":"2024-03-19T01:07:21.249606+0000","last_fullsized":"2024-03-19T01:07:21.249606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:17:36.760943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679381+0000","last_change":"2024-03-19T01:07:15.491045+0000","last_active":"2024-03-19T01:07:20.679381+0000","last_peered":"2024-03-19T01:07:20.679381+0000","last_clean":"2024-03-19T01:07:20.679381+0000","last_became_active":"2024-03-19T01:07:15.488914+0000","last_became_peered":"2024-03-19T01:07:15.488914+0000","last_unstale":"2024-03-19T01:07:20.679381+0000","last_undegraded":"2024-03-19T01:07:20.679381+0000","last_fullsized":"2024-03-19T01:07:20.679381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:14.853457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.676146+0000","last_change":"2024-03-19T01:07:15.495306+0000","last_active":"2024-03-19T01:07:20.676146+0000","last_peered":"2024-03-19T01:07:20.676146+0000","last_clean":"2024-03-19T01:07:20.676146+0000","last_became_active":"2024-03-19T01:07:15.494906+0000","last_became_peered":"2024-03-19T01:07:15.494906+0000","last_unstale":"2024-03-19T01:07:20.676146+0000","last_undegraded":"2024-03-19T01:07:20.676146+0000","last_fullsized":"2024-03-19T01:07:20.676146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:48.176410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.675787+0000","last_change":"2024-03-19T01:07:19.689402+0000","last_active":"2024-03-19T01:07:20.675787+0000","last_peered":"2024-03-19T01:07:20.675787+0000","last_clean":"2024-03-19T01:07:20.675787+0000","last_became_active":"2024-03-19T01:07:19.688799+0000","last_became_peered":"2024-03-19T01:07:19.688799+0000","last_unstale":"2024-03-19T01:07:20.675787+0000","last_undegraded":"2024-03-19T01:07:20.675787+0000","last_fullsized":"2024-03-19T01:07:20.675787+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:51.890047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:50.820739+0000","last_change":"2024-03-19T01:07:14.477348+0000","last_active":"2024-03-19T01:07:50.820739+0000","last_peered":"2024-03-19T01:07:50.820739+0000","last_clean":"2024-03-19T01:07:50.820739+0000","last_became_active":"2024-03-19T01:07:14.476946+0000","last_became_peered":"2024-03-19T01:07:14.476946+0000","last_unstale":"2024-03-19T01:07:50.820739+0000","last_undegraded":"2024-03-19T01:07:50.820739+0000","last_fullsized":"2024-03-19T01:07:50.820739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:19.269837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249233+0000","last_change":"2024-03-19T01:07:15.495646+0000","last_active":"2024-03-19T01:07:21.249233+0000","last_peered":"2024-03-19T01:07:21.249233+0000","last_clean":"2024-03-19T01:07:21.249233+0000","last_became_active":"2024-03-19T01:07:15.495203+0000","last_became_peered":"2024-03-19T01:07:15.495203+0000","last_unstale":"2024-03-19T01:07:21.249233+0000","last_undegraded":"2024-03-19T01:07:21.249233+0000","last_fullsized":"2024-03-19T01:07:21.249233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:01:31.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247484+0000","last_change":"2024-03-19T01:07:19.690465+0000","last_active":"2024-03-19T01:07:21.247484+0000","last_peered":"2024-03-19T01:07:21.247484+0000","last_clean":"2024-03-19T01:07:21.247484+0000","last_became_active":"2024-03-19T01:07:19.689852+0000","last_became_peered":"2024-03-19T01:07:19.689852+0000","last_unstale":"2024-03-19T01:07:21.247484+0000","last_undegraded":"2024-03-19T01:07:21.247484+0000","last_fullsized":"2024-03-19T01:07:21.247484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:18.891411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249459+0000","last_change":"2024-03-19T01:07:14.487165+0000","last_active":"2024-03-19T01:07:21.249459+0000","last_peered":"2024-03-19T01:07:21.249459+0000","last_clean":"2024-03-19T01:07:21.249459+0000","last_became_active":"2024-03-19T01:07:14.484194+0000","last_became_peered":"2024-03-19T01:07:14.484194+0000","last_unstale":"2024-03-19T01:07:21.249459+0000","last_undegraded":"2024-03-19T01:07:21.249459+0000","last_fullsized":"2024-03-19T01:07:21.249459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:54:47.914565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839380+0000","last_change":"2024-03-19T01:07:15.498687+0000","last_active":"2024-03-19T01:07:20.839380+0000","last_peered":"2024-03-19T01:07:20.839380+0000","last_clean":"2024-03-19T01:07:20.839380+0000","last_became_active":"2024-03-19T01:07:15.498146+0000","last_became_peered":"2024-03-19T01:07:15.498146+0000","last_unstale":"2024-03-19T01:07:20.839380+0000","last_undegraded":"2024-03-19T01:07:20.839380+0000","last_fullsized":"2024-03-19T01:07:20.839380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:47:21.846333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247854+0000","last_change":"2024-03-19T01:07:19.690691+0000","last_active":"2024-03-19T01:07:21.247854+0000","last_peered":"2024-03-19T01:07:21.247854+0000","last_clean":"2024-03-19T01:07:21.247854+0000","last_became_active":"2024-03-19T01:07:19.690083+0000","last_became_peered":"2024-03-19T01:07:19.690083+0000","last_unstale":"2024-03-19T01:07:21.247854+0000","last_undegraded":"2024-03-19T01:07:21.247854+0000","last_fullsized":"2024-03-19T01:07:21.247854+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:51:30.205694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:50.817823+0000","last_change":"2024-03-19T01:07:14.478366+0000","last_active":"2024-03-19T01:07:50.817823+0000","last_peered":"2024-03-19T01:07:50.817823+0000","last_clean":"2024-03-19T01:07:50.817823+0000","last_became_active":"2024-03-19T01:07:14.477998+0000","last_became_peered":"2024-03-19T01:07:14.477998+0000","last_unstale":"2024-03-19T01:07:50.817823+0000","last_undegraded":"2024-03-19T01:07:50.817823+0000","last_fullsized":"2024-03-19T01:07:50.817823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:33:31.284602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248995+0000","last_change":"2024-03-19T01:07:15.489371+0000","last_active":"2024-03-19T01:07:21.248995+0000","last_peered":"2024-03-19T01:07:21.248995+0000","last_clean":"2024-03-19T01:07:21.248995+0000","last_became_active":"2024-03-19T01:07:15.488975+0000","last_became_peered":"2024-03-19T01:07:15.488975+0000","last_unstale":"2024-03-19T01:07:21.248995+0000","last_undegraded":"2024-03-19T01:07:21.248995+0000","last_fullsized":"2024-03-19T01:07:21.248995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:04:47.640512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247980+0000","last_change":"2024-03-19T01:07:19.691590+0000","last_active":"2024-03-19T01:07:21.247980+0000","last_peered":"2024-03-19T01:07:21.247980+0000","last_clean":"2024-03-19T01:07:21.247980+0000","last_became_active":"2024-03-19T01:07:19.691037+0000","last_became_peered":"2024-03-19T01:07:19.691037+0000","last_unstale":"2024-03-19T01:07:21.247980+0000","last_undegraded":"2024-03-19T01:07:21.247980+0000","last_fullsized":"2024-03-19T01:07:21.247980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:20:02.007483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.794233+0000","last_change":"2024-03-19T01:07:14.475653+0000","last_active":"2024-03-19T01:07:49.794233+0000","last_peered":"2024-03-19T01:07:49.794233+0000","last_clean":"2024-03-19T01:07:49.794233+0000","last_became_active":"2024-03-19T01:07:14.475315+0000","last_became_peered":"2024-03-19T01:07:14.475315+0000","last_unstale":"2024-03-19T01:07:49.794233+0000","last_undegraded":"2024-03-19T01:07:49.794233+0000","last_fullsized":"2024-03-19T01:07:49.794233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:44:04.656950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248973+0000","last_change":"2024-03-19T01:07:15.480882+0000","last_active":"2024-03-19T01:07:21.248973+0000","last_peered":"2024-03-19T01:07:21.248973+0000","last_clean":"2024-03-19T01:07:21.248973+0000","last_became_active":"2024-03-19T01:07:15.480541+0000","last_became_peered":"2024-03-19T01:07:15.480541+0000","last_unstale":"2024-03-19T01:07:21.248973+0000","last_undegraded":"2024-03-19T01:07:21.248973+0000","last_fullsized":"2024-03-19T01:07:21.248973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:37:24.724531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.676034+0000","last_change":"2024-03-19T01:07:19.672979+0000","last_active":"2024-03-19T01:07:20.676034+0000","last_peered":"2024-03-19T01:07:20.676034+0000","last_clean":"2024-03-19T01:07:20.676034+0000","last_became_active":"2024-03-19T01:07:19.672530+0000","last_became_peered":"2024-03-19T01:07:19.672530+0000","last_unstale":"2024-03-19T01:07:20.676034+0000","last_undegraded":"2024-03-19T01:07:20.676034+0000","last_fullsized":"2024-03-19T01:07:20.676034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:12.948079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:50.761371+0000","last_change":"2024-03-19T01:07:14.475620+0000","last_active":"2024-03-19T01:07:50.761371+0000","last_peered":"2024-03-19T01:07:50.761371+0000","last_clean":"2024-03-19T01:07:50.761371+0000","last_became_active":"2024-03-19T01:07:14.475029+0000","last_became_peered":"2024-03-19T01:07:14.475029+0000","last_unstale":"2024-03-19T01:07:50.761371+0000","last_undegraded":"2024-03-19T01:07:50.761371+0000","last_fullsized":"2024-03-19T01:07:50.761371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:48:53.983457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248142+0000","last_change":"2024-03-19T01:07:15.484680+0000","last_active":"2024-03-19T01:07:21.248142+0000","last_peered":"2024-03-19T01:07:21.248142+0000","last_clean":"2024-03-19T01:07:21.248142+0000","last_became_active":"2024-03-19T01:07:15.484333+0000","last_became_peered":"2024-03-19T01:07:15.484333+0000","last_unstale":"2024-03-19T01:07:21.248142+0000","last_undegraded":"2024-03-19T01:07:21.248142+0000","last_fullsized":"2024-03-19T01:07:21.248142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:38:23.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.675882+0000","last_change":"2024-03-19T01:07:19.692211+0000","last_active":"2024-03-19T01:07:20.675882+0000","last_peered":"2024-03-19T01:07:20.675882+0000","last_clean":"2024-03-19T01:07:20.675882+0000","last_became_active":"2024-03-19T01:07:19.691756+0000","last_became_peered":"2024-03-19T01:07:19.691756+0000","last_unstale":"2024-03-19T01:07:20.675882+0000","last_undegraded":"2024-03-19T01:07:20.675882+0000","last_fullsized":"2024-03-19T01:07:20.675882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:14:07.357956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.771315+0000","last_change":"2024-03-19T01:07:14.481499+0000","last_active":"2024-03-19T01:07:48.771315+0000","last_peered":"2024-03-19T01:07:48.771315+0000","last_clean":"2024-03-19T01:07:48.771315+0000","last_became_active":"2024-03-19T01:07:14.481123+0000","last_became_peered":"2024-03-19T01:07:14.481123+0000","last_unstale":"2024-03-19T01:07:48.771315+0000","last_undegraded":"2024-03-19T01:07:48.771315+0000","last_fullsized":"2024-03-19T01:07:48.771315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:36:15.349748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249214+0000","last_change":"2024-03-19T01:07:15.499744+0000","last_active":"2024-03-19T01:07:21.249214+0000","last_peered":"2024-03-19T01:07:21.249214+0000","last_clean":"2024-03-19T01:07:21.249214+0000","last_became_active":"2024-03-19T01:07:15.499037+0000","last_became_peered":"2024-03-19T01:07:15.499037+0000","last_unstale":"2024-03-19T01:07:21.249214+0000","last_undegraded":"2024-03-19T01:07:21.249214+0000","last_fullsized":"2024-03-19T01:07:21.249214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:20:35.524664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249488+0000","last_change":"2024-03-19T01:07:19.684187+0000","last_active":"2024-03-19T01:07:21.249488+0000","last_peered":"2024-03-19T01:07:21.249488+0000","last_clean":"2024-03-19T01:07:21.249488+0000","last_became_active":"2024-03-19T01:07:19.683782+0000","last_became_peered":"2024-03-19T01:07:19.683782+0000","last_unstale":"2024-03-19T01:07:21.249488+0000","last_undegraded":"2024-03-19T01:07:21.249488+0000","last_fullsized":"2024-03-19T01:07:21.249488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:17:50.818003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:47.813999+0000","last_change":"2024-03-19T01:07:14.485538+0000","last_active":"2024-03-19T01:07:47.813999+0000","last_peered":"2024-03-19T01:07:47.813999+0000","last_clean":"2024-03-19T01:07:47.813999+0000","last_became_active":"2024-03-19T01:07:14.485004+0000","last_became_peered":"2024-03-19T01:07:14.485004+0000","last_unstale":"2024-03-19T01:07:47.813999+0000","last_undegraded":"2024-03-19T01:07:47.813999+0000","last_fullsized":"2024-03-19T01:07:47.813999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:55:50.853725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248200+0000","last_change":"2024-03-19T01:07:15.503370+0000","last_active":"2024-03-19T01:07:21.248200+0000","last_peered":"2024-03-19T01:07:21.248200+0000","last_clean":"2024-03-19T01:07:21.248200+0000","last_became_active":"2024-03-19T01:07:15.501375+0000","last_became_peered":"2024-03-19T01:07:15.501375+0000","last_unstale":"2024-03-19T01:07:21.248200+0000","last_undegraded":"2024-03-19T01:07:21.248200+0000","last_fullsized":"2024-03-19T01:07:21.248200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:36:01.814936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839459+0000","last_change":"2024-03-19T01:07:19.697067+0000","last_active":"2024-03-19T01:07:20.839459+0000","last_peered":"2024-03-19T01:07:20.839459+0000","last_clean":"2024-03-19T01:07:20.839459+0000","last_became_active":"2024-03-19T01:07:19.696364+0000","last_became_peered":"2024-03-19T01:07:19.696364+0000","last_unstale":"2024-03-19T01:07:20.839459+0000","last_undegraded":"2024-03-19T01:07:20.839459+0000","last_fullsized":"2024-03-19T01:07:20.839459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:34.388407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.795031+0000","last_change":"2024-03-19T01:07:14.475361+0000","last_active":"2024-03-19T01:07:49.795031+0000","last_peered":"2024-03-19T01:07:49.795031+0000","last_clean":"2024-03-19T01:07:49.795031+0000","last_became_active":"2024-03-19T01:07:14.474577+0000","last_became_peered":"2024-03-19T01:07:14.474577+0000","last_unstale":"2024-03-19T01:07:49.795031+0000","last_undegraded":"2024-03-19T01:07:49.795031+0000","last_fullsized":"2024-03-19T01:07:49.795031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:01:04.713405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.677482+0000","last_change":"2024-03-19T01:07:15.506310+0000","last_active":"2024-03-19T01:07:20.677482+0000","last_peered":"2024-03-19T01:07:20.677482+0000","last_clean":"2024-03-19T01:07:20.677482+0000","last_became_active":"2024-03-19T01:07:15.505792+0000","last_became_peered":"2024-03-19T01:07:15.505792+0000","last_unstale":"2024-03-19T01:07:20.677482+0000","last_undegraded":"2024-03-19T01:07:20.677482+0000","last_fullsized":"2024-03-19T01:07:20.677482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:40.630816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248268+0000","last_change":"2024-03-19T01:07:19.683632+0000","last_active":"2024-03-19T01:07:21.248268+0000","last_peered":"2024-03-19T01:07:21.248268+0000","last_clean":"2024-03-19T01:07:21.248268+0000","last_became_active":"2024-03-19T01:07:19.683204+0000","last_became_peered":"2024-03-19T01:07:19.683204+0000","last_unstale":"2024-03-19T01:07:21.248268+0000","last_undegraded":"2024-03-19T01:07:21.248268+0000","last_fullsized":"2024-03-19T01:07:21.248268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:58:53.512824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:51.761848+0000","last_change":"2024-03-19T01:07:14.471347+0000","last_active":"2024-03-19T01:07:51.761848+0000","last_peered":"2024-03-19T01:07:51.761848+0000","last_clean":"2024-03-19T01:07:51.761848+0000","last_became_active":"2024-03-19T01:07:14.471001+0000","last_became_peered":"2024-03-19T01:07:14.471001+0000","last_unstale":"2024-03-19T01:07:51.761848+0000","last_undegraded":"2024-03-19T01:07:51.761848+0000","last_fullsized":"2024-03-19T01:07:51.761848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:46.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249062+0000","last_change":"2024-03-19T01:07:15.486779+0000","last_active":"2024-03-19T01:07:21.249062+0000","last_peered":"2024-03-19T01:07:21.249062+0000","last_clean":"2024-03-19T01:07:21.249062+0000","last_became_active":"2024-03-19T01:07:15.486405+0000","last_became_peered":"2024-03-19T01:07:15.486405+0000","last_unstale":"2024-03-19T01:07:21.249062+0000","last_undegraded":"2024-03-19T01:07:21.249062+0000","last_fullsized":"2024-03-19T01:07:21.249062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:14:13.350169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.677700+0000","last_change":"2024-03-19T01:07:19.686892+0000","last_active":"2024-03-19T01:07:20.677700+0000","last_peered":"2024-03-19T01:07:20.677700+0000","last_clean":"2024-03-19T01:07:20.677700+0000","last_became_active":"2024-03-19T01:07:19.686414+0000","last_became_peered":"2024-03-19T01:07:19.686414+0000","last_unstale":"2024-03-19T01:07:20.677700+0000","last_undegraded":"2024-03-19T01:07:20.677700+0000","last_fullsized":"2024-03-19T01:07:20.677700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:18:18.105905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.794360+0000","last_change":"2024-03-19T01:07:14.474384+0000","last_active":"2024-03-19T01:07:49.794360+0000","last_peered":"2024-03-19T01:07:49.794360+0000","last_clean":"2024-03-19T01:07:49.794360+0000","last_became_active":"2024-03-19T01:07:14.473996+0000","last_became_peered":"2024-03-19T01:07:14.473996+0000","last_unstale":"2024-03-19T01:07:49.794360+0000","last_undegraded":"2024-03-19T01:07:49.794360+0000","last_fullsized":"2024-03-19T01:07:49.794360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:12:33.079612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839265+0000","last_change":"2024-03-19T01:07:15.499507+0000","last_active":"2024-03-19T01:07:20.839265+0000","last_peered":"2024-03-19T01:07:20.839265+0000","last_clean":"2024-03-19T01:07:20.839265+0000","last_became_active":"2024-03-19T01:07:15.497261+0000","last_became_peered":"2024-03-19T01:07:15.497261+0000","last_unstale":"2024-03-19T01:07:20.839265+0000","last_undegraded":"2024-03-19T01:07:20.839265+0000","last_fullsized":"2024-03-19T01:07:20.839265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:40:06.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248149+0000","last_change":"2024-03-19T01:07:19.690847+0000","last_active":"2024-03-19T01:07:21.248149+0000","last_peered":"2024-03-19T01:07:21.248149+0000","last_clean":"2024-03-19T01:07:21.248149+0000","last_became_active":"2024-03-19T01:07:19.689762+0000","last_became_peered":"2024-03-19T01:07:19.689762+0000","last_unstale":"2024-03-19T01:07:21.248149+0000","last_undegraded":"2024-03-19T01:07:21.248149+0000","last_fullsized":"2024-03-19T01:07:21.248149+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:24:40.213171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.758468+0000","last_change":"2024-03-19T01:07:14.473744+0000","last_active":"2024-03-19T01:07:48.758468+0000","last_peered":"2024-03-19T01:07:48.758468+0000","last_clean":"2024-03-19T01:07:48.758468+0000","last_became_active":"2024-03-19T01:07:14.473404+0000","last_became_peered":"2024-03-19T01:07:14.473404+0000","last_unstale":"2024-03-19T01:07:48.758468+0000","last_undegraded":"2024-03-19T01:07:48.758468+0000","last_fullsized":"2024-03-19T01:07:48.758468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:29:39.892678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249119+0000","last_change":"2024-03-19T01:07:15.481444+0000","last_active":"2024-03-19T01:07:21.249119+0000","last_peered":"2024-03-19T01:07:21.249119+0000","last_clean":"2024-03-19T01:07:21.249119+0000","last_became_active":"2024-03-19T01:07:15.481106+0000","last_became_peered":"2024-03-19T01:07:15.481106+0000","last_unstale":"2024-03-19T01:07:21.249119+0000","last_undegraded":"2024-03-19T01:07:21.249119+0000","last_fullsized":"2024-03-19T01:07:21.249119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:00.263063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.678244+0000","last_change":"2024-03-19T01:07:19.686919+0000","last_active":"2024-03-19T01:07:20.678244+0000","last_peered":"2024-03-19T01:07:20.678244+0000","last_clean":"2024-03-19T01:07:20.678244+0000","last_became_active":"2024-03-19T01:07:19.686416+0000","last_became_peered":"2024-03-19T01:07:19.686416+0000","last_unstale":"2024-03-19T01:07:20.678244+0000","last_undegraded":"2024-03-19T01:07:20.678244+0000","last_fullsized":"2024-03-19T01:07:20.678244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:29:11.068957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'22","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:51.767492+0000","last_change":"2024-03-19T01:07:14.468630+0000","last_active":"2024-03-19T01:07:51.767492+0000","last_peered":"2024-03-19T01:07:51.767492+0000","last_clean":"2024-03-19T01:07:51.767492+0000","last_became_active":"2024-03-19T01:07:14.468293+0000","last_became_peered":"2024-03-19T01:07:14.468293+0000","last_unstale":"2024-03-19T01:07:51.767492+0000","last_undegraded":"2024-03-19T01:07:51.767492+0000","last_fullsized":"2024-03-19T01:07:51.767492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:53.575507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":29,"num_write":24,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248459+0000","last_change":"2024-03-19T01:07:15.477226+0000","last_active":"2024-03-19T01:07:21.248459+0000","last_peered":"2024-03-19T01:07:21.248459+0000","last_clean":"2024-03-19T01:07:21.248459+0000","last_became_active":"2024-03-19T01:07:15.476882+0000","last_became_peered":"2024-03-19T01:07:15.476882+0000","last_unstale":"2024-03-19T01:07:21.248459+0000","last_undegraded":"2024-03-19T01:07:21.248459+0000","last_fullsized":"2024-03-19T01:07:21.248459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:11:10.051837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249620+0000","last_change":"2024-03-19T01:07:19.690004+0000","last_active":"2024-03-19T01:07:21.249620+0000","last_peered":"2024-03-19T01:07:21.249620+0000","last_clean":"2024-03-19T01:07:21.249620+0000","last_became_active":"2024-03-19T01:07:19.689573+0000","last_became_peered":"2024-03-19T01:07:19.689573+0000","last_unstale":"2024-03-19T01:07:21.249620+0000","last_undegraded":"2024-03-19T01:07:21.249620+0000","last_fullsized":"2024-03-19T01:07:21.249620+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:31:21.788219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.797100+0000","last_change":"2024-03-19T01:07:14.487373+0000","last_active":"2024-03-19T01:07:49.797100+0000","last_peered":"2024-03-19T01:07:49.797100+0000","last_clean":"2024-03-19T01:07:49.797100+0000","last_became_active":"2024-03-19T01:07:14.487077+0000","last_became_peered":"2024-03-19T01:07:14.487077+0000","last_unstale":"2024-03-19T01:07:49.797100+0000","last_undegraded":"2024-03-19T01:07:49.797100+0000","last_fullsized":"2024-03-19T01:07:49.797100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:33:01.789431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.748660+0000","last_change":"2024-03-19T01:07:15.498009+0000","last_active":"2024-03-19T01:07:20.748660+0000","last_peered":"2024-03-19T01:07:20.748660+0000","last_clean":"2024-03-19T01:07:20.748660+0000","last_became_active":"2024-03-19T01:07:15.496707+0000","last_became_peered":"2024-03-19T01:07:15.496707+0000","last_unstale":"2024-03-19T01:07:20.748660+0000","last_undegraded":"2024-03-19T01:07:20.748660+0000","last_fullsized":"2024-03-19T01:07:20.748660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:50.058299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248414+0000","last_change":"2024-03-19T01:07:19.677635+0000","last_active":"2024-03-19T01:07:21.248414+0000","last_peered":"2024-03-19T01:07:21.248414+0000","last_clean":"2024-03-19T01:07:21.248414+0000","last_became_active":"2024-03-19T01:07:19.677207+0000","last_became_peered":"2024-03-19T01:07:19.677207+0000","last_unstale":"2024-03-19T01:07:21.248414+0000","last_undegraded":"2024-03-19T01:07:21.248414+0000","last_fullsized":"2024-03-19T01:07:21.248414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:23:28.650962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.797335+0000","last_change":"2024-03-19T01:07:14.482054+0000","last_active":"2024-03-19T01:07:49.797335+0000","last_peered":"2024-03-19T01:07:49.797335+0000","last_clean":"2024-03-19T01:07:49.797335+0000","last_became_active":"2024-03-19T01:07:14.481703+0000","last_became_peered":"2024-03-19T01:07:14.481703+0000","last_unstale":"2024-03-19T01:07:49.797335+0000","last_undegraded":"2024-03-19T01:07:49.797335+0000","last_fullsized":"2024-03-19T01:07:49.797335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:02:43.565442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679344+0000","last_change":"2024-03-19T01:07:15.487236+0000","last_active":"2024-03-19T01:07:20.679344+0000","last_peered":"2024-03-19T01:07:20.679344+0000","last_clean":"2024-03-19T01:07:20.679344+0000","last_became_active":"2024-03-19T01:07:15.486931+0000","last_became_peered":"2024-03-19T01:07:15.486931+0000","last_unstale":"2024-03-19T01:07:20.679344+0000","last_undegraded":"2024-03-19T01:07:20.679344+0000","last_fullsized":"2024-03-19T01:07:20.679344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:22.201904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679377+0000","last_change":"2024-03-19T01:07:19.693813+0000","last_active":"2024-03-19T01:07:20.679377+0000","last_peered":"2024-03-19T01:07:20.679377+0000","last_clean":"2024-03-19T01:07:20.679377+0000","last_became_active":"2024-03-19T01:07:19.693384+0000","last_became_peered":"2024-03-19T01:07:19.693384+0000","last_unstale":"2024-03-19T01:07:20.679377+0000","last_undegraded":"2024-03-19T01:07:20.679377+0000","last_fullsized":"2024-03-19T01:07:20.679377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:09.372461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:51.769768+0000","last_change":"2024-03-19T01:07:14.468467+0000","last_active":"2024-03-19T01:07:51.769768+0000","last_peered":"2024-03-19T01:07:51.769768+0000","last_clean":"2024-03-19T01:07:51.769768+0000","last_became_active":"2024-03-19T01:07:14.468074+0000","last_became_peered":"2024-03-19T01:07:14.468074+0000","last_unstale":"2024-03-19T01:07:51.769768+0000","last_undegraded":"2024-03-19T01:07:51.769768+0000","last_fullsized":"2024-03-19T01:07:51.769768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:07:49.216998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679452+0000","last_change":"2024-03-19T01:07:15.475310+0000","last_active":"2024-03-19T01:07:20.679452+0000","last_peered":"2024-03-19T01:07:20.679452+0000","last_clean":"2024-03-19T01:07:20.679452+0000","last_became_active":"2024-03-19T01:07:15.474914+0000","last_became_peered":"2024-03-19T01:07:15.474914+0000","last_unstale":"2024-03-19T01:07:20.679452+0000","last_undegraded":"2024-03-19T01:07:20.679452+0000","last_fullsized":"2024-03-19T01:07:20.679452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:34.036892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.749576+0000","last_change":"2024-03-19T01:07:19.697937+0000","last_active":"2024-03-19T01:07:20.749576+0000","last_peered":"2024-03-19T01:07:20.749576+0000","last_clean":"2024-03-19T01:07:20.749576+0000","last_became_active":"2024-03-19T01:07:19.697447+0000","last_became_peered":"2024-03-19T01:07:19.697447+0000","last_unstale":"2024-03-19T01:07:20.749576+0000","last_undegraded":"2024-03-19T01:07:20.749576+0000","last_fullsized":"2024-03-19T01:07:20.749576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:49.259118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.774194+0000","last_change":"2024-03-19T01:07:14.476382+0000","last_active":"2024-03-19T01:07:48.774194+0000","last_peered":"2024-03-19T01:07:48.774194+0000","last_clean":"2024-03-19T01:07:48.774194+0000","last_became_active":"2024-03-19T01:07:14.476089+0000","last_became_peered":"2024-03-19T01:07:14.476089+0000","last_unstale":"2024-03-19T01:07:48.774194+0000","last_undegraded":"2024-03-19T01:07:48.774194+0000","last_fullsized":"2024-03-19T01:07:48.774194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:57:42.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249271+0000","last_change":"2024-03-19T01:07:15.480912+0000","last_active":"2024-03-19T01:07:21.249271+0000","last_peered":"2024-03-19T01:07:21.249271+0000","last_clean":"2024-03-19T01:07:21.249271+0000","last_became_active":"2024-03-19T01:07:15.480602+0000","last_became_peered":"2024-03-19T01:07:15.480602+0000","last_unstale":"2024-03-19T01:07:21.249271+0000","last_undegraded":"2024-03-19T01:07:21.249271+0000","last_fullsized":"2024-03-19T01:07:21.249271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:18:31.947781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249565+0000","last_change":"2024-03-19T01:07:19.688235+0000","last_active":"2024-03-19T01:07:21.249565+0000","last_peered":"2024-03-19T01:07:21.249565+0000","last_clean":"2024-03-19T01:07:21.249565+0000","last_became_active":"2024-03-19T01:07:19.687879+0000","last_became_peered":"2024-03-19T01:07:19.687879+0000","last_unstale":"2024-03-19T01:07:21.249565+0000","last_undegraded":"2024-03-19T01:07:21.249565+0000","last_fullsized":"2024-03-19T01:07:21.249565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:29.465971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.758216+0000","last_change":"2024-03-19T01:07:14.474346+0000","last_active":"2024-03-19T01:07:48.758216+0000","last_peered":"2024-03-19T01:07:48.758216+0000","last_clean":"2024-03-19T01:07:48.758216+0000","last_became_active":"2024-03-19T01:07:14.473989+0000","last_became_peered":"2024-03-19T01:07:14.473989+0000","last_unstale":"2024-03-19T01:07:48.758216+0000","last_undegraded":"2024-03-19T01:07:48.758216+0000","last_fullsized":"2024-03-19T01:07:48.758216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:41.004155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249223+0000","last_change":"2024-03-19T01:07:15.482326+0000","last_active":"2024-03-19T01:07:21.249223+0000","last_peered":"2024-03-19T01:07:21.249223+0000","last_clean":"2024-03-19T01:07:21.249223+0000","last_became_active":"2024-03-19T01:07:15.481492+0000","last_became_peered":"2024-03-19T01:07:15.481492+0000","last_unstale":"2024-03-19T01:07:21.249223+0000","last_undegraded":"2024-03-19T01:07:21.249223+0000","last_fullsized":"2024-03-19T01:07:21.249223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:17:31.572104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248476+0000","last_change":"2024-03-19T01:07:19.668853+0000","last_active":"2024-03-19T01:07:21.248476+0000","last_peered":"2024-03-19T01:07:21.248476+0000","last_clean":"2024-03-19T01:07:21.248476+0000","last_became_active":"2024-03-19T01:07:19.668428+0000","last_became_peered":"2024-03-19T01:07:19.668428+0000","last_unstale":"2024-03-19T01:07:21.248476+0000","last_undegraded":"2024-03-19T01:07:21.248476+0000","last_fullsized":"2024-03-19T01:07:21.248476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:50:43.409586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.678911+0000","last_change":"2024-03-19T01:07:14.469548+0000","last_active":"2024-03-19T01:07:20.678911+0000","last_peered":"2024-03-19T01:07:20.678911+0000","last_clean":"2024-03-19T01:07:20.678911+0000","last_became_active":"2024-03-19T01:07:14.469200+0000","last_became_peered":"2024-03-19T01:07:14.469200+0000","last_unstale":"2024-03-19T01:07:20.678911+0000","last_undegraded":"2024-03-19T01:07:20.678911+0000","last_fullsized":"2024-03-19T01:07:20.678911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:41:53.179343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248608+0000","last_change":"2024-03-19T01:07:15.495300+0000","last_active":"2024-03-19T01:07:21.248608+0000","last_peered":"2024-03-19T01:07:21.248608+0000","last_clean":"2024-03-19T01:07:21.248608+0000","last_became_active":"2024-03-19T01:07:15.494938+0000","last_became_peered":"2024-03-19T01:07:15.494938+0000","last_unstale":"2024-03-19T01:07:21.248608+0000","last_undegraded":"2024-03-19T01:07:21.248608+0000","last_fullsized":"2024-03-19T01:07:21.248608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:50:38.855857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.749242+0000","last_change":"2024-03-19T01:07:19.693261+0000","last_active":"2024-03-19T01:07:20.749242+0000","last_peered":"2024-03-19T01:07:20.749242+0000","last_clean":"2024-03-19T01:07:20.749242+0000","last_became_active":"2024-03-19T01:07:19.692598+0000","last_became_peered":"2024-03-19T01:07:19.692598+0000","last_unstale":"2024-03-19T01:07:20.749242+0000","last_undegraded":"2024-03-19T01:07:20.749242+0000","last_fullsized":"2024-03-19T01:07:20.749242+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:43:29.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:50.823201+0000","last_change":"2024-03-19T01:07:14.481569+0000","last_active":"2024-03-19T01:07:50.823201+0000","last_peered":"2024-03-19T01:07:50.823201+0000","last_clean":"2024-03-19T01:07:50.823201+0000","last_became_active":"2024-03-19T01:07:14.481209+0000","last_became_peered":"2024-03-19T01:07:14.481209+0000","last_unstale":"2024-03-19T01:07:50.823201+0000","last_undegraded":"2024-03-19T01:07:50.823201+0000","last_fullsized":"2024-03-19T01:07:50.823201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:39.411207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248157+0000","last_change":"2024-03-19T01:07:15.496852+0000","last_active":"2024-03-19T01:07:21.248157+0000","last_peered":"2024-03-19T01:07:21.248157+0000","last_clean":"2024-03-19T01:07:21.248157+0000","last_became_active":"2024-03-19T01:07:15.496455+0000","last_became_peered":"2024-03-19T01:07:15.496455+0000","last_unstale":"2024-03-19T01:07:21.248157+0000","last_undegraded":"2024-03-19T01:07:21.248157+0000","last_fullsized":"2024-03-19T01:07:21.248157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:43.419496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.749557+0000","last_change":"2024-03-19T01:07:19.692856+0000","last_active":"2024-03-19T01:07:20.749557+0000","last_peered":"2024-03-19T01:07:20.749557+0000","last_clean":"2024-03-19T01:07:20.749557+0000","last_became_active":"2024-03-19T01:07:19.690712+0000","last_became_peered":"2024-03-19T01:07:19.690712+0000","last_unstale":"2024-03-19T01:07:20.749557+0000","last_undegraded":"2024-03-19T01:07:20.749557+0000","last_fullsized":"2024-03-19T01:07:20.749557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:29:28.473062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248698+0000","last_change":"2024-03-19T01:07:15.474375+0000","last_active":"2024-03-19T01:07:21.248698+0000","last_peered":"2024-03-19T01:07:21.248698+0000","last_clean":"2024-03-19T01:07:21.248698+0000","last_became_active":"2024-03-19T01:07:15.473427+0000","last_became_peered":"2024-03-19T01:07:15.473427+0000","last_unstale":"2024-03-19T01:07:21.248698+0000","last_undegraded":"2024-03-19T01:07:21.248698+0000","last_fullsized":"2024-03-19T01:07:21.248698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:42:22.235054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:50.818534+0000","last_change":"2024-03-19T01:07:14.490924+0000","last_active":"2024-03-19T01:07:50.818534+0000","last_peered":"2024-03-19T01:07:50.818534+0000","last_clean":"2024-03-19T01:07:50.818534+0000","last_became_active":"2024-03-19T01:07:14.490621+0000","last_became_peered":"2024-03-19T01:07:14.490621+0000","last_unstale":"2024-03-19T01:07:50.818534+0000","last_undegraded":"2024-03-19T01:07:50.818534+0000","last_fullsized":"2024-03-19T01:07:50.818534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:02:52.254138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:22.692597+0000","last_change":"2024-03-19T01:07:19.672258+0000","last_active":"2024-03-19T01:07:22.692597+0000","last_peered":"2024-03-19T01:07:22.692597+0000","last_clean":"2024-03-19T01:07:22.692597+0000","last_became_active":"2024-03-19T01:07:19.671862+0000","last_became_peered":"2024-03-19T01:07:19.671862+0000","last_unstale":"2024-03-19T01:07:22.692597+0000","last_undegraded":"2024-03-19T01:07:22.692597+0000","last_fullsized":"2024-03-19T01:07:22.692597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:15.321703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248509+0000","last_change":"2024-03-19T01:07:15.493218+0000","last_active":"2024-03-19T01:07:21.248509+0000","last_peered":"2024-03-19T01:07:21.248509+0000","last_clean":"2024-03-19T01:07:21.248509+0000","last_became_active":"2024-03-19T01:07:15.492751+0000","last_became_peered":"2024-03-19T01:07:15.492751+0000","last_unstale":"2024-03-19T01:07:21.248509+0000","last_undegraded":"2024-03-19T01:07:21.248509+0000","last_fullsized":"2024-03-19T01:07:21.248509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:48:32.105214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.798687+0000","last_change":"2024-03-19T01:07:14.473349+0000","last_active":"2024-03-19T01:07:49.798687+0000","last_peered":"2024-03-19T01:07:49.798687+0000","last_clean":"2024-03-19T01:07:49.798687+0000","last_became_active":"2024-03-19T01:07:14.473014+0000","last_became_peered":"2024-03-19T01:07:14.473014+0000","last_unstale":"2024-03-19T01:07:49.798687+0000","last_undegraded":"2024-03-19T01:07:49.798687+0000","last_fullsized":"2024-03-19T01:07:49.798687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:27:34.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679117+0000","last_change":"2024-03-19T01:05:56.013594+0000","last_active":"2024-03-19T01:07:20.679117+0000","last_peered":"2024-03-19T01:07:20.679117+0000","last_clean":"2024-03-19T01:07:20.679117+0000","last_became_active":"2024-03-19T01:05:56.013084+0000","last_became_peered":"2024-03-19T01:05:56.013084+0000","last_unstale":"2024-03-19T01:07:20.679117+0000","last_undegraded":"2024-03-19T01:07:20.679117+0000","last_fullsized":"2024-03-19T01:07:20.679117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_clean_scrub_stamp":"2024-03-19T01:02:06.532343+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:23.440161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249530+0000","last_change":"2024-03-19T01:07:19.691111+0000","last_active":"2024-03-19T01:07:21.249530+0000","last_peered":"2024-03-19T01:07:21.249530+0000","last_clean":"2024-03-19T01:07:21.249530+0000","last_became_active":"2024-03-19T01:07:19.690618+0000","last_became_peered":"2024-03-19T01:07:19.690618+0000","last_unstale":"2024-03-19T01:07:21.249530+0000","last_undegraded":"2024-03-19T01:07:21.249530+0000","last_fullsized":"2024-03-19T01:07:21.249530+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:17.857394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.748435+0000","last_change":"2024-03-19T01:07:15.480452+0000","last_active":"2024-03-19T01:07:20.748435+0000","last_peered":"2024-03-19T01:07:20.748435+0000","last_clean":"2024-03-19T01:07:20.748435+0000","last_became_active":"2024-03-19T01:07:15.479897+0000","last_became_peered":"2024-03-19T01:07:15.479897+0000","last_unstale":"2024-03-19T01:07:20.748435+0000","last_undegraded":"2024-03-19T01:07:20.748435+0000","last_fullsized":"2024-03-19T01:07:20.748435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:37.841942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:47.810788+0000","last_change":"2024-03-19T01:07:14.474287+0000","last_active":"2024-03-19T01:07:47.810788+0000","last_peered":"2024-03-19T01:07:47.810788+0000","last_clean":"2024-03-19T01:07:47.810788+0000","last_became_active":"2024-03-19T01:07:14.473921+0000","last_became_peered":"2024-03-19T01:07:14.473921+0000","last_unstale":"2024-03-19T01:07:47.810788+0000","last_undegraded":"2024-03-19T01:07:47.810788+0000","last_fullsized":"2024-03-19T01:07:47.810788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:49.322029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248579+0000","last_change":"2024-03-19T01:07:19.685331+0000","last_active":"2024-03-19T01:07:21.248579+0000","last_peered":"2024-03-19T01:07:21.248579+0000","last_clean":"2024-03-19T01:07:21.248579+0000","last_became_active":"2024-03-19T01:07:19.684876+0000","last_became_peered":"2024-03-19T01:07:19.684876+0000","last_unstale":"2024-03-19T01:07:21.248579+0000","last_undegraded":"2024-03-19T01:07:21.248579+0000","last_fullsized":"2024-03-19T01:07:21.248579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:39.575348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.773339+0000","last_change":"2024-03-19T01:07:14.481019+0000","last_active":"2024-03-19T01:07:48.773339+0000","last_peered":"2024-03-19T01:07:48.773339+0000","last_clean":"2024-03-19T01:07:48.773339+0000","last_became_active":"2024-03-19T01:07:14.480666+0000","last_became_peered":"2024-03-19T01:07:14.480666+0000","last_unstale":"2024-03-19T01:07:48.773339+0000","last_undegraded":"2024-03-19T01:07:48.773339+0000","last_fullsized":"2024-03-19T01:07:48.773339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:13:33.669690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.749219+0000","last_change":"2024-03-19T01:07:15.490747+0000","last_active":"2024-03-19T01:07:20.749219+0000","last_peered":"2024-03-19T01:07:20.749219+0000","last_clean":"2024-03-19T01:07:20.749219+0000","last_became_active":"2024-03-19T01:07:15.486244+0000","last_became_peered":"2024-03-19T01:07:15.486244+0000","last_unstale":"2024-03-19T01:07:20.749219+0000","last_undegraded":"2024-03-19T01:07:20.749219+0000","last_fullsized":"2024-03-19T01:07:20.749219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:43.212972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248231+0000","last_change":"2024-03-19T01:07:19.684804+0000","last_active":"2024-03-19T01:07:21.248231+0000","last_peered":"2024-03-19T01:07:21.248231+0000","last_clean":"2024-03-19T01:07:21.248231+0000","last_became_active":"2024-03-19T01:07:19.683191+0000","last_became_peered":"2024-03-19T01:07:19.683191+0000","last_unstale":"2024-03-19T01:07:21.248231+0000","last_undegraded":"2024-03-19T01:07:21.248231+0000","last_fullsized":"2024-03-19T01:07:21.248231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:02.015499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.776065+0000","last_change":"2024-03-19T01:07:14.479282+0000","last_active":"2024-03-19T01:07:48.776065+0000","last_peered":"2024-03-19T01:07:48.776065+0000","last_clean":"2024-03-19T01:07:48.776065+0000","last_became_active":"2024-03-19T01:07:14.478876+0000","last_became_peered":"2024-03-19T01:07:14.478876+0000","last_unstale":"2024-03-19T01:07:48.776065+0000","last_undegraded":"2024-03-19T01:07:48.776065+0000","last_fullsized":"2024-03-19T01:07:48.776065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:54.179680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249258+0000","last_change":"2024-03-19T01:07:15.493498+0000","last_active":"2024-03-19T01:07:21.249258+0000","last_peered":"2024-03-19T01:07:21.249258+0000","last_clean":"2024-03-19T01:07:21.249258+0000","last_became_active":"2024-03-19T01:07:15.493110+0000","last_became_peered":"2024-03-19T01:07:15.493110+0000","last_unstale":"2024-03-19T01:07:21.249258+0000","last_undegraded":"2024-03-19T01:07:21.249258+0000","last_fullsized":"2024-03-19T01:07:21.249258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:55:23.608596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.675483+0000","last_change":"2024-03-19T01:07:19.671232+0000","last_active":"2024-03-19T01:07:20.675483+0000","last_peered":"2024-03-19T01:07:20.675483+0000","last_clean":"2024-03-19T01:07:20.675483+0000","last_became_active":"2024-03-19T01:07:19.670818+0000","last_became_peered":"2024-03-19T01:07:19.670818+0000","last_unstale":"2024-03-19T01:07:20.675483+0000","last_undegraded":"2024-03-19T01:07:20.675483+0000","last_fullsized":"2024-03-19T01:07:20.675483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:57:20.134372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.799339+0000","last_change":"2024-03-19T01:07:14.473568+0000","last_active":"2024-03-19T01:07:49.799339+0000","last_peered":"2024-03-19T01:07:49.799339+0000","last_clean":"2024-03-19T01:07:49.799339+0000","last_became_active":"2024-03-19T01:07:14.472555+0000","last_became_peered":"2024-03-19T01:07:14.472555+0000","last_unstale":"2024-03-19T01:07:49.799339+0000","last_undegraded":"2024-03-19T01:07:49.799339+0000","last_fullsized":"2024-03-19T01:07:49.799339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:28:01.541228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839538+0000","last_change":"2024-03-19T01:07:15.499905+0000","last_active":"2024-03-19T01:07:20.839538+0000","last_peered":"2024-03-19T01:07:20.839538+0000","last_clean":"2024-03-19T01:07:20.839538+0000","last_became_active":"2024-03-19T01:07:15.499560+0000","last_became_peered":"2024-03-19T01:07:15.499560+0000","last_unstale":"2024-03-19T01:07:20.839538+0000","last_undegraded":"2024-03-19T01:07:20.839538+0000","last_fullsized":"2024-03-19T01:07:20.839538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:06:26.531100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247604+0000","last_change":"2024-03-19T01:07:19.690310+0000","last_active":"2024-03-19T01:07:21.247604+0000","last_peered":"2024-03-19T01:07:21.247604+0000","last_clean":"2024-03-19T01:07:21.247604+0000","last_became_active":"2024-03-19T01:07:19.689877+0000","last_became_peered":"2024-03-19T01:07:19.689877+0000","last_unstale":"2024-03-19T01:07:21.247604+0000","last_undegraded":"2024-03-19T01:07:21.247604+0000","last_fullsized":"2024-03-19T01:07:21.247604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:06:12.730258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:46.799163+0000","last_change":"2024-03-19T01:07:14.466267+0000","last_active":"2024-03-19T01:07:46.799163+0000","last_peered":"2024-03-19T01:07:46.799163+0000","last_clean":"2024-03-19T01:07:46.799163+0000","last_became_active":"2024-03-19T01:07:14.465944+0000","last_became_peered":"2024-03-19T01:07:14.465944+0000","last_unstale":"2024-03-19T01:07:46.799163+0000","last_undegraded":"2024-03-19T01:07:46.799163+0000","last_fullsized":"2024-03-19T01:07:46.799163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:40.957200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247694+0000","last_change":"2024-03-19T01:07:15.475690+0000","last_active":"2024-03-19T01:07:21.247694+0000","last_peered":"2024-03-19T01:07:21.247694+0000","last_clean":"2024-03-19T01:07:21.247694+0000","last_became_active":"2024-03-19T01:07:15.475342+0000","last_became_peered":"2024-03-19T01:07:15.475342+0000","last_unstale":"2024-03-19T01:07:21.247694+0000","last_undegraded":"2024-03-19T01:07:21.247694+0000","last_fullsized":"2024-03-19T01:07:21.247694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:30:30.842462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679551+0000","last_change":"2024-03-19T01:07:19.695170+0000","last_active":"2024-03-19T01:07:20.679551+0000","last_peered":"2024-03-19T01:07:20.679551+0000","last_clean":"2024-03-19T01:07:20.679551+0000","last_became_active":"2024-03-19T01:07:19.694095+0000","last_became_peered":"2024-03-19T01:07:19.694095+0000","last_unstale":"2024-03-19T01:07:20.679551+0000","last_undegraded":"2024-03-19T01:07:20.679551+0000","last_fullsized":"2024-03-19T01:07:20.679551+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:33:12.420640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.795497+0000","last_change":"2024-03-19T01:07:14.487907+0000","last_active":"2024-03-19T01:07:49.795497+0000","last_peered":"2024-03-19T01:07:49.795497+0000","last_clean":"2024-03-19T01:07:49.795497+0000","last_became_active":"2024-03-19T01:07:14.487618+0000","last_became_peered":"2024-03-19T01:07:14.487618+0000","last_unstale":"2024-03-19T01:07:49.795497+0000","last_undegraded":"2024-03-19T01:07:49.795497+0000","last_fullsized":"2024-03-19T01:07:49.795497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:22:49.218797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248220+0000","last_change":"2024-03-19T01:07:15.492103+0000","last_active":"2024-03-19T01:07:21.248220+0000","last_peered":"2024-03-19T01:07:21.248220+0000","last_clean":"2024-03-19T01:07:21.248220+0000","last_became_active":"2024-03-19T01:07:15.491712+0000","last_became_peered":"2024-03-19T01:07:15.491712+0000","last_unstale":"2024-03-19T01:07:21.248220+0000","last_undegraded":"2024-03-19T01:07:21.248220+0000","last_fullsized":"2024-03-19T01:07:21.248220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:40:13.959986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:23.572718+0000","last_change":"2024-03-19T01:07:19.677392+0000","last_active":"2024-03-19T01:07:23.572718+0000","last_peered":"2024-03-19T01:07:23.572718+0000","last_clean":"2024-03-19T01:07:23.572718+0000","last_became_active":"2024-03-19T01:07:19.677024+0000","last_became_peered":"2024-03-19T01:07:19.677024+0000","last_unstale":"2024-03-19T01:07:23.572718+0000","last_undegraded":"2024-03-19T01:07:23.572718+0000","last_fullsized":"2024-03-19T01:07:23.572718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:01:43.404902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:47.798526+0000","last_change":"2024-03-19T01:07:14.481170+0000","last_active":"2024-03-19T01:07:47.798526+0000","last_peered":"2024-03-19T01:07:47.798526+0000","last_clean":"2024-03-19T01:07:47.798526+0000","last_became_active":"2024-03-19T01:07:14.480814+0000","last_became_peered":"2024-03-19T01:07:14.480814+0000","last_unstale":"2024-03-19T01:07:47.798526+0000","last_undegraded":"2024-03-19T01:07:47.798526+0000","last_fullsized":"2024-03-19T01:07:47.798526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:24:23.380331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248917+0000","last_change":"2024-03-19T01:07:15.505842+0000","last_active":"2024-03-19T01:07:21.248917+0000","last_peered":"2024-03-19T01:07:21.248917+0000","last_clean":"2024-03-19T01:07:21.248917+0000","last_became_active":"2024-03-19T01:07:15.505425+0000","last_became_peered":"2024-03-19T01:07:15.505425+0000","last_unstale":"2024-03-19T01:07:21.248917+0000","last_undegraded":"2024-03-19T01:07:21.248917+0000","last_fullsized":"2024-03-19T01:07:21.248917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:28.119864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247571+0000","last_change":"2024-03-19T01:07:19.683113+0000","last_active":"2024-03-19T01:07:21.247571+0000","last_peered":"2024-03-19T01:07:21.247571+0000","last_clean":"2024-03-19T01:07:21.247571+0000","last_became_active":"2024-03-19T01:07:19.682702+0000","last_became_peered":"2024-03-19T01:07:19.682702+0000","last_unstale":"2024-03-19T01:07:21.247571+0000","last_undegraded":"2024-03-19T01:07:21.247571+0000","last_fullsized":"2024-03-19T01:07:21.247571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:42.866408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:46.801664+0000","last_change":"2024-03-19T01:07:14.476562+0000","last_active":"2024-03-19T01:07:46.801664+0000","last_peered":"2024-03-19T01:07:46.801664+0000","last_clean":"2024-03-19T01:07:46.801664+0000","last_became_active":"2024-03-19T01:07:14.476185+0000","last_became_peered":"2024-03-19T01:07:14.476185+0000","last_unstale":"2024-03-19T01:07:46.801664+0000","last_undegraded":"2024-03-19T01:07:46.801664+0000","last_fullsized":"2024-03-19T01:07:46.801664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:35:19.860443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679657+0000","last_change":"2024-03-19T01:07:15.487163+0000","last_active":"2024-03-19T01:07:20.679657+0000","last_peered":"2024-03-19T01:07:20.679657+0000","last_clean":"2024-03-19T01:07:20.679657+0000","last_became_active":"2024-03-19T01:07:15.486845+0000","last_became_peered":"2024-03-19T01:07:15.486845+0000","last_unstale":"2024-03-19T01:07:20.679657+0000","last_undegraded":"2024-03-19T01:07:20.679657+0000","last_fullsized":"2024-03-19T01:07:20.679657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:39:09.389411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248781+0000","last_change":"2024-03-19T01:07:19.692768+0000","last_active":"2024-03-19T01:07:21.248781+0000","last_peered":"2024-03-19T01:07:21.248781+0000","last_clean":"2024-03-19T01:07:21.248781+0000","last_became_active":"2024-03-19T01:07:19.692382+0000","last_became_peered":"2024-03-19T01:07:19.692382+0000","last_unstale":"2024-03-19T01:07:21.248781+0000","last_undegraded":"2024-03-19T01:07:21.248781+0000","last_fullsized":"2024-03-19T01:07:21.248781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:26.804938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.769919+0000","last_change":"2024-03-19T01:07:14.472844+0000","last_active":"2024-03-19T01:07:48.769919+0000","last_peered":"2024-03-19T01:07:48.769919+0000","last_clean":"2024-03-19T01:07:48.769919+0000","last_became_active":"2024-03-19T01:07:14.472389+0000","last_became_peered":"2024-03-19T01:07:14.472389+0000","last_unstale":"2024-03-19T01:07:48.769919+0000","last_undegraded":"2024-03-19T01:07:48.769919+0000","last_fullsized":"2024-03-19T01:07:48.769919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:31.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679491+0000","last_change":"2024-03-19T01:07:15.482033+0000","last_active":"2024-03-19T01:07:20.679491+0000","last_peered":"2024-03-19T01:07:20.679491+0000","last_clean":"2024-03-19T01:07:20.679491+0000","last_became_active":"2024-03-19T01:07:15.481671+0000","last_became_peered":"2024-03-19T01:07:15.481671+0000","last_unstale":"2024-03-19T01:07:20.679491+0000","last_undegraded":"2024-03-19T01:07:20.679491+0000","last_fullsized":"2024-03-19T01:07:20.679491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:09.969374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839041+0000","last_change":"2024-03-19T01:07:19.696413+0000","last_active":"2024-03-19T01:07:20.839041+0000","last_peered":"2024-03-19T01:07:20.839041+0000","last_clean":"2024-03-19T01:07:20.839041+0000","last_became_active":"2024-03-19T01:07:19.695957+0000","last_became_peered":"2024-03-19T01:07:19.695957+0000","last_unstale":"2024-03-19T01:07:20.839041+0000","last_undegraded":"2024-03-19T01:07:20.839041+0000","last_fullsized":"2024-03-19T01:07:20.839041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:54:48.547854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.770718+0000","last_change":"2024-03-19T01:07:14.481677+0000","last_active":"2024-03-19T01:07:48.770718+0000","last_peered":"2024-03-19T01:07:48.770718+0000","last_clean":"2024-03-19T01:07:48.770718+0000","last_became_active":"2024-03-19T01:07:14.481138+0000","last_became_peered":"2024-03-19T01:07:14.481138+0000","last_unstale":"2024-03-19T01:07:48.770718+0000","last_undegraded":"2024-03-19T01:07:48.770718+0000","last_fullsized":"2024-03-19T01:07:48.770718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:52:33.105816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679467+0000","last_change":"2024-03-19T01:07:15.498352+0000","last_active":"2024-03-19T01:07:20.679467+0000","last_peered":"2024-03-19T01:07:20.679467+0000","last_clean":"2024-03-19T01:07:20.679467+0000","last_became_active":"2024-03-19T01:07:15.498040+0000","last_became_peered":"2024-03-19T01:07:15.498040+0000","last_unstale":"2024-03-19T01:07:20.679467+0000","last_undegraded":"2024-03-19T01:07:20.679467+0000","last_fullsized":"2024-03-19T01:07:20.679467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:57:37.571753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247889+0000","last_change":"2024-03-19T01:07:19.673291+0000","last_active":"2024-03-19T01:07:21.247889+0000","last_peered":"2024-03-19T01:07:21.247889+0000","last_clean":"2024-03-19T01:07:21.247889+0000","last_became_active":"2024-03-19T01:07:19.672704+0000","last_became_peered":"2024-03-19T01:07:19.672704+0000","last_unstale":"2024-03-19T01:07:21.247889+0000","last_undegraded":"2024-03-19T01:07:21.247889+0000","last_fullsized":"2024-03-19T01:07:21.247889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:01:42.019812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.797497+0000","last_change":"2024-03-19T01:07:14.489556+0000","last_active":"2024-03-19T01:07:49.797497+0000","last_peered":"2024-03-19T01:07:49.797497+0000","last_clean":"2024-03-19T01:07:49.797497+0000","last_became_active":"2024-03-19T01:07:14.489201+0000","last_became_peered":"2024-03-19T01:07:14.489201+0000","last_unstale":"2024-03-19T01:07:49.797497+0000","last_undegraded":"2024-03-19T01:07:49.797497+0000","last_fullsized":"2024-03-19T01:07:49.797497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:28:36.516185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679685+0000","last_change":"2024-03-19T01:07:15.477727+0000","last_active":"2024-03-19T01:07:20.679685+0000","last_peered":"2024-03-19T01:07:20.679685+0000","last_clean":"2024-03-19T01:07:20.679685+0000","last_became_active":"2024-03-19T01:07:15.477388+0000","last_became_peered":"2024-03-19T01:07:15.477388+0000","last_unstale":"2024-03-19T01:07:20.679685+0000","last_undegraded":"2024-03-19T01:07:20.679685+0000","last_fullsized":"2024-03-19T01:07:20.679685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:30.567326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248140+0000","last_change":"2024-03-19T01:07:15.479008+0000","last_active":"2024-03-19T01:07:21.248140+0000","last_peered":"2024-03-19T01:07:21.248140+0000","last_clean":"2024-03-19T01:07:21.248140+0000","last_became_active":"2024-03-19T01:07:15.478657+0000","last_became_peered":"2024-03-19T01:07:15.478657+0000","last_unstale":"2024-03-19T01:07:21.248140+0000","last_undegraded":"2024-03-19T01:07:21.248140+0000","last_fullsized":"2024-03-19T01:07:21.248140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:42:46.970012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.675962+0000","last_change":"2024-03-19T01:07:15.476033+0000","last_active":"2024-03-19T01:07:20.675962+0000","last_peered":"2024-03-19T01:07:20.675962+0000","last_clean":"2024-03-19T01:07:20.675962+0000","last_became_active":"2024-03-19T01:07:15.475675+0000","last_became_peered":"2024-03-19T01:07:15.475675+0000","last_unstale":"2024-03-19T01:07:20.675962+0000","last_undegraded":"2024-03-19T01:07:20.675962+0000","last_fullsized":"2024-03-19T01:07:20.675962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:01:44.290463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.675520+0000","last_change":"2024-03-19T01:07:15.495292+0000","last_active":"2024-03-19T01:07:20.675520+0000","last_peered":"2024-03-19T01:07:20.675520+0000","last_clean":"2024-03-19T01:07:20.675520+0000","last_became_active":"2024-03-19T01:07:15.494946+0000","last_became_peered":"2024-03-19T01:07:15.494946+0000","last_unstale":"2024-03-19T01:07:20.675520+0000","last_undegraded":"2024-03-19T01:07:20.675520+0000","last_fullsized":"2024-03-19T01:07:20.675520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:39.012221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.838891+0000","last_change":"2024-03-19T01:07:15.499816+0000","last_active":"2024-03-19T01:07:20.838891+0000","last_peered":"2024-03-19T01:07:20.838891+0000","last_clean":"2024-03-19T01:07:20.838891+0000","last_became_active":"2024-03-19T01:07:15.499031+0000","last_became_peered":"2024-03-19T01:07:15.499031+0000","last_unstale":"2024-03-19T01:07:20.838891+0000","last_undegraded":"2024-03-19T01:07:20.838891+0000","last_fullsized":"2024-03-19T01:07:20.838891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:34:25.344092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.749104+0000","last_change":"2024-03-19T01:07:15.501526+0000","last_active":"2024-03-19T01:07:20.749104+0000","last_peered":"2024-03-19T01:07:20.749104+0000","last_clean":"2024-03-19T01:07:20.749104+0000","last_became_active":"2024-03-19T01:07:15.500452+0000","last_became_peered":"2024-03-19T01:07:15.500452+0000","last_unstale":"2024-03-19T01:07:20.749104+0000","last_undegraded":"2024-03-19T01:07:20.749104+0000","last_fullsized":"2024-03-19T01:07:20.749104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:35:33.075780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248995+0000","last_change":"2024-03-19T01:07:15.477649+0000","last_active":"2024-03-19T01:07:21.248995+0000","last_peered":"2024-03-19T01:07:21.248995+0000","last_clean":"2024-03-19T01:07:21.248995+0000","last_became_active":"2024-03-19T01:07:15.477193+0000","last_became_peered":"2024-03-19T01:07:15.477193+0000","last_unstale":"2024-03-19T01:07:21.248995+0000","last_undegraded":"2024-03-19T01:07:21.248995+0000","last_fullsized":"2024-03-19T01:07:21.248995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:36.126642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.676211+0000","last_change":"2024-03-19T01:07:15.481920+0000","last_active":"2024-03-19T01:07:20.676211+0000","last_peered":"2024-03-19T01:07:20.676211+0000","last_clean":"2024-03-19T01:07:20.676211+0000","last_became_active":"2024-03-19T01:07:15.481290+0000","last_became_peered":"2024-03-19T01:07:15.481290+0000","last_unstale":"2024-03-19T01:07:20.676211+0000","last_undegraded":"2024-03-19T01:07:20.676211+0000","last_fullsized":"2024-03-19T01:07:20.676211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:37:58.028647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249429+0000","last_change":"2024-03-19T01:07:15.502305+0000","last_active":"2024-03-19T01:07:21.249429+0000","last_peered":"2024-03-19T01:07:21.249429+0000","last_clean":"2024-03-19T01:07:21.249429+0000","last_became_active":"2024-03-19T01:07:15.501491+0000","last_became_peered":"2024-03-19T01:07:15.501491+0000","last_unstale":"2024-03-19T01:07:21.249429+0000","last_undegraded":"2024-03-19T01:07:21.249429+0000","last_fullsized":"2024-03-19T01:07:21.249429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:01.999538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248584+0000","last_change":"2024-03-19T01:07:15.499590+0000","last_active":"2024-03-19T01:07:21.248584+0000","last_peered":"2024-03-19T01:07:21.248584+0000","last_clean":"2024-03-19T01:07:21.248584+0000","last_became_active":"2024-03-19T01:07:15.498991+0000","last_became_peered":"2024-03-19T01:07:15.498991+0000","last_unstale":"2024-03-19T01:07:21.248584+0000","last_undegraded":"2024-03-19T01:07:21.248584+0000","last_fullsized":"2024-03-19T01:07:21.248584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:09:43.510480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248456+0000","last_change":"2024-03-19T01:07:15.505022+0000","last_active":"2024-03-19T01:07:21.248456+0000","last_peered":"2024-03-19T01:07:21.248456+0000","last_clean":"2024-03-19T01:07:21.248456+0000","last_became_active":"2024-03-19T01:07:15.503684+0000","last_became_peered":"2024-03-19T01:07:15.503684+0000","last_unstale":"2024-03-19T01:07:21.248456+0000","last_undegraded":"2024-03-19T01:07:21.248456+0000","last_fullsized":"2024-03-19T01:07:21.248456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:56:47.580267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248843+0000","last_change":"2024-03-19T01:07:15.495634+0000","last_active":"2024-03-19T01:07:21.248843+0000","last_peered":"2024-03-19T01:07:21.248843+0000","last_clean":"2024-03-19T01:07:21.248843+0000","last_became_active":"2024-03-19T01:07:15.495206+0000","last_became_peered":"2024-03-19T01:07:15.495206+0000","last_unstale":"2024-03-19T01:07:21.248843+0000","last_undegraded":"2024-03-19T01:07:21.248843+0000","last_fullsized":"2024-03-19T01:07:21.248843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:30:29.965731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.748932+0000","last_change":"2024-03-19T01:07:15.501480+0000","last_active":"2024-03-19T01:07:20.748932+0000","last_peered":"2024-03-19T01:07:20.748932+0000","last_clean":"2024-03-19T01:07:20.748932+0000","last_became_active":"2024-03-19T01:07:15.501078+0000","last_became_peered":"2024-03-19T01:07:15.501078+0000","last_unstale":"2024-03-19T01:07:20.748932+0000","last_undegraded":"2024-03-19T01:07:20.748932+0000","last_fullsized":"2024-03-19T01:07:20.748932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:40:18.767874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248556+0000","last_change":"2024-03-19T01:07:15.480358+0000","last_active":"2024-03-19T01:07:21.248556+0000","last_peered":"2024-03-19T01:07:21.248556+0000","last_clean":"2024-03-19T01:07:21.248556+0000","last_became_active":"2024-03-19T01:07:15.479979+0000","last_became_peered":"2024-03-19T01:07:15.479979+0000","last_unstale":"2024-03-19T01:07:21.248556+0000","last_undegraded":"2024-03-19T01:07:21.248556+0000","last_fullsized":"2024-03-19T01:07:21.248556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:12:54.598435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247132,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":29954,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":64,"num_read_kb":70,"num_write":219,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":70344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192,"ondisk_log_size":192,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383700,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645338,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":1092851,"data_compressed":50398,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808863,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":482433,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":478715,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332138,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1079308,"data_compressed":50398,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":477937,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":502108,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":479393,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":483420,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":477645,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":482402,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1074499,"data_compressed":50398,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":25268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T01:07:54.390 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T01:07:54.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.628+0000 7f3454000640 1 -- 172.21.15.69:0/3155782569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f344c107660 msgr2=0x7f344c109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.628+0000 7f3454000640 1 --2- 172.21.15.69:0/3155782569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f344c107660 0x7f344c109a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3444009ca0 tx=0x7f344402c100 comp rx=0 tx=0).stop 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.628+0000 7f3454000640 1 -- 172.21.15.69:0/3155782569 shutdown_connections 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.628+0000 7f3454000640 1 --2- 172.21.15.69:0/3155782569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f344c107660 0x7f344c109a50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.628+0000 7f3454000640 1 --2- 172.21.15.69:0/3155782569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c104d30 0x7f344c107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.628+0000 7f3454000640 1 --2- 172.21.15.69:0/3155782569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c102400 0x7f344c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.628+0000 7f3454000640 1 -- 172.21.15.69:0/3155782569 >> 172.21.15.69:0/3155782569 conn(0x7f344c0fbd50 msgr2=0x7f344c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.628+0000 7f3454000640 1 -- 172.21.15.69:0/3155782569 shutdown_connections 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.629+0000 7f3454000640 1 -- 172.21.15.69:0/3155782569 wait complete. 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.629+0000 7f3454000640 1 Processor -- start 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3454000640 1 -- start start 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3454000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c102400 0x7f344c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:54.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3454000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f344c104d30 0x7f344c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3454000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c107660 0x7f344c1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3454000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f344c10d500 con 0x7f344c107660 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3454000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f344c10d380 con 0x7f344c102400 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3454000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f344c10d680 con 0x7f344c104d30 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3451600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c102400 0x7f344c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3450c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f344c104d30 0x7f344c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3450c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f344c104d30 0x7f344c19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53892/0 (socket says 172.21.15.69:53892) 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3451600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c102400 0x7f344c199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45084/0 (socket says 172.21.15.69:45084) 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3450c00640 1 -- 172.21.15.69:0/1202373339 learned_addr learned my addr 172.21.15.69:0/1202373339 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.630+0000 7f3452000640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c107660 0x7f344c1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f3450c00640 1 -- 172.21.15.69:0/1202373339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c102400 msgr2=0x7f344c199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f3450c00640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c102400 0x7f344c199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f3450c00640 1 -- 172.21.15.69:0/1202373339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c107660 msgr2=0x7f344c1a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f3450c00640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c107660 0x7f344c1a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f3450c00640 1 -- 172.21.15.69:0/1202373339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344c1a1910 con 0x7f344c104d30 2024-03-19T01:07:54.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f3452000640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c107660 0x7f344c1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:07:54.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f3451600640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c102400 0x7f344c199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:07:54.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f3450c00640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f344c104d30 0x7f344c19a240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f343800e9f0 tx=0x7f343800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:54.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f344a000640 1 -- 172.21.15.69:0/1202373339 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343800ccc0 con 0x7f344c104d30 2024-03-19T01:07:54.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f344a000640 1 -- 172.21.15.69:0/1202373339 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3438005500 con 0x7f344c104d30 2024-03-19T01:07:54.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f344a000640 1 -- 172.21.15.69:0/1202373339 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438010690 con 0x7f344c104d30 2024-03-19T01:07:54.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f3454000640 1 -- 172.21.15.69:0/1202373339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f344c1a1c00 con 0x7f344c104d30 2024-03-19T01:07:54.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.631+0000 7f3454000640 1 -- 172.21.15.69:0/1202373339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f344c1a2100 con 0x7f344c104d30 2024-03-19T01:07:54.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.633+0000 7f3454000640 1 -- 172.21.15.69:0/1202373339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f344c109150 con 0x7f344c104d30 2024-03-19T01:07:54.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.634+0000 7f344a000640 1 -- 172.21.15.69:0/1202373339 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f34380046c0 con 0x7f344c104d30 2024-03-19T01:07:54.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.636+0000 7f344a000640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f34340a2890 0x7f34340a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:54.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.636+0000 7f344a000640 1 -- 172.21.15.69:0/1202373339 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f34380cf2d0 con 0x7f344c104d30 2024-03-19T01:07:54.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.636+0000 7f3451600640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f34340a2890 0x7f34340a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:54.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.637+0000 7f3451600640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f34340a2890 0x7f34340a4d50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f344c19b1d0 tx=0x7f3440005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:54.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.640+0000 7f344a000640 1 -- 172.21.15.69:0/1202373339 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f343809bfd0 con 0x7f344c104d30 2024-03-19T01:07:54.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.851+0000 7f3454000640 1 -- 172.21.15.69:0/1202373339 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f344c061c60 con 0x7f34340a2890 2024-03-19T01:07:54.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.865+0000 7f344a000640 1 -- 172.21.15.69:0/1202373339 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340672 (secure 0 0 0) 0x7f344c061c60 con 0x7f34340a2890 2024-03-19T01:07:54.871 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:07:54.873 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-03-19T01:07:54.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.872+0000 7f3454000640 1 -- 172.21.15.69:0/1202373339 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f34340a2890 msgr2=0x7f34340a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:54.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.872+0000 7f3454000640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f34340a2890 0x7f34340a4d50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f344c19b1d0 tx=0x7f3440005f90 comp rx=0 tx=0).stop 2024-03-19T01:07:54.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.872+0000 7f3454000640 1 -- 172.21.15.69:0/1202373339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f344c104d30 msgr2=0x7f344c19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:54.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.872+0000 7f3454000640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f344c104d30 0x7f344c19a240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f343800e9f0 tx=0x7f343800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.872+0000 7f3454000640 1 -- 172.21.15.69:0/1202373339 shutdown_connections 2024-03-19T01:07:54.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.872+0000 7f3454000640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f34340a2890 0x7f34340a4d50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.872+0000 7f3454000640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c107660 0x7f344c1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.872+0000 7f3454000640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f344c104d30 0x7f344c19a240 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.872+0000 7f3454000640 1 --2- 172.21.15.69:0/1202373339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344c102400 0x7f344c199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:54.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.872+0000 7f3454000640 1 -- 172.21.15.69:0/1202373339 >> 172.21.15.69:0/1202373339 conn(0x7f344c0fbd50 msgr2=0x7f344c101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:54.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.873+0000 7f3454000640 1 -- 172.21.15.69:0/1202373339 shutdown_connections 2024-03-19T01:07:54.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:54.874+0000 7f3454000640 1 -- 172.21.15.69:0/1202373339 wait complete. 2024-03-19T01:07:54.896 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-03-19T01:07:53.653572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620556,"num_objects":101,"num_object_clones":0,"num_object_copies":305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":200,"num_read_kb":189,"num_write":402,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369328,"kb_used_data":11856,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572496,"statfs":{"total":1151940427776,"available":1151562235904,"internally_reserved":0,"allocated":12140544,"data_stored":7587926,"data_compressed":282810,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25689,"num_objects":77,"num_object_clones":0,"num_object_copies":233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":64,"num_read_kb":70,"num_write":173,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005965"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249707+0000","last_change":"2024-03-19T01:07:15.500098+0000","last_active":"2024-03-19T01:07:21.249707+0000","last_peered":"2024-03-19T01:07:21.249707+0000","last_clean":"2024-03-19T01:07:21.249707+0000","last_became_active":"2024-03-19T01:07:15.499738+0000","last_became_peered":"2024-03-19T01:07:15.499738+0000","last_unstale":"2024-03-19T01:07:21.249707+0000","last_undegraded":"2024-03-19T01:07:21.249707+0000","last_fullsized":"2024-03-19T01:07:21.249707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:14.703153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839414+0000","last_change":"2024-03-19T01:07:15.492863+0000","last_active":"2024-03-19T01:07:20.839414+0000","last_peered":"2024-03-19T01:07:20.839414+0000","last_clean":"2024-03-19T01:07:20.839414+0000","last_became_active":"2024-03-19T01:07:15.492533+0000","last_became_peered":"2024-03-19T01:07:15.492533+0000","last_unstale":"2024-03-19T01:07:20.839414+0000","last_undegraded":"2024-03-19T01:07:20.839414+0000","last_fullsized":"2024-03-19T01:07:20.839414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:49.028352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.838935+0000","last_change":"2024-03-19T01:07:15.494174+0000","last_active":"2024-03-19T01:07:20.838935+0000","last_peered":"2024-03-19T01:07:20.838935+0000","last_clean":"2024-03-19T01:07:20.838935+0000","last_became_active":"2024-03-19T01:07:15.490562+0000","last_became_peered":"2024-03-19T01:07:15.490562+0000","last_unstale":"2024-03-19T01:07:20.838935+0000","last_undegraded":"2024-03-19T01:07:20.838935+0000","last_fullsized":"2024-03-19T01:07:20.838935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:18.664396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247974+0000","last_change":"2024-03-19T01:07:15.478397+0000","last_active":"2024-03-19T01:07:21.247974+0000","last_peered":"2024-03-19T01:07:21.247974+0000","last_clean":"2024-03-19T01:07:21.247974+0000","last_became_active":"2024-03-19T01:07:15.477000+0000","last_became_peered":"2024-03-19T01:07:15.477000+0000","last_unstale":"2024-03-19T01:07:21.247974+0000","last_undegraded":"2024-03-19T01:07:21.247974+0000","last_fullsized":"2024-03-19T01:07:21.247974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:58.534329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247754+0000","last_change":"2024-03-19T01:07:15.500138+0000","last_active":"2024-03-19T01:07:21.247754+0000","last_peered":"2024-03-19T01:07:21.247754+0000","last_clean":"2024-03-19T01:07:21.247754+0000","last_became_active":"2024-03-19T01:07:15.499759+0000","last_became_peered":"2024-03-19T01:07:15.499759+0000","last_unstale":"2024-03-19T01:07:21.247754+0000","last_undegraded":"2024-03-19T01:07:21.247754+0000","last_fullsized":"2024-03-19T01:07:21.247754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:23.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247576+0000","last_change":"2024-03-19T01:07:15.485146+0000","last_active":"2024-03-19T01:07:21.247576+0000","last_peered":"2024-03-19T01:07:21.247576+0000","last_clean":"2024-03-19T01:07:21.247576+0000","last_became_active":"2024-03-19T01:07:15.484680+0000","last_became_peered":"2024-03-19T01:07:15.484680+0000","last_unstale":"2024-03-19T01:07:21.247576+0000","last_undegraded":"2024-03-19T01:07:21.247576+0000","last_fullsized":"2024-03-19T01:07:21.247576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:12.659347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679127+0000","last_change":"2024-03-19T01:07:15.486327+0000","last_active":"2024-03-19T01:07:20.679127+0000","last_peered":"2024-03-19T01:07:20.679127+0000","last_clean":"2024-03-19T01:07:20.679127+0000","last_became_active":"2024-03-19T01:07:15.485954+0000","last_became_peered":"2024-03-19T01:07:15.485954+0000","last_unstale":"2024-03-19T01:07:20.679127+0000","last_undegraded":"2024-03-19T01:07:20.679127+0000","last_fullsized":"2024-03-19T01:07:20.679127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:53.316260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247929+0000","last_change":"2024-03-19T01:07:15.504993+0000","last_active":"2024-03-19T01:07:21.247929+0000","last_peered":"2024-03-19T01:07:21.247929+0000","last_clean":"2024-03-19T01:07:21.247929+0000","last_became_active":"2024-03-19T01:07:15.502771+0000","last_became_peered":"2024-03-19T01:07:15.502771+0000","last_unstale":"2024-03-19T01:07:21.247929+0000","last_undegraded":"2024-03-19T01:07:21.247929+0000","last_fullsized":"2024-03-19T01:07:21.247929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:22:33.712608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249048+0000","last_change":"2024-03-19T01:07:15.483876+0000","last_active":"2024-03-19T01:07:21.249048+0000","last_peered":"2024-03-19T01:07:21.249048+0000","last_clean":"2024-03-19T01:07:21.249048+0000","last_became_active":"2024-03-19T01:07:15.482744+0000","last_became_peered":"2024-03-19T01:07:15.482744+0000","last_unstale":"2024-03-19T01:07:21.249048+0000","last_undegraded":"2024-03-19T01:07:21.249048+0000","last_fullsized":"2024-03-19T01:07:21.249048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:21:19.870877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248538+0000","last_change":"2024-03-19T01:07:15.475655+0000","last_active":"2024-03-19T01:07:21.248538+0000","last_peered":"2024-03-19T01:07:21.248538+0000","last_clean":"2024-03-19T01:07:21.248538+0000","last_became_active":"2024-03-19T01:07:15.475265+0000","last_became_peered":"2024-03-19T01:07:15.475265+0000","last_unstale":"2024-03-19T01:07:21.248538+0000","last_undegraded":"2024-03-19T01:07:21.248538+0000","last_fullsized":"2024-03-19T01:07:21.248538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:24:38.898191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679401+0000","last_change":"2024-03-19T01:07:15.482988+0000","last_active":"2024-03-19T01:07:20.679401+0000","last_peered":"2024-03-19T01:07:20.679401+0000","last_clean":"2024-03-19T01:07:20.679401+0000","last_became_active":"2024-03-19T01:07:15.482673+0000","last_became_peered":"2024-03-19T01:07:15.482673+0000","last_unstale":"2024-03-19T01:07:20.679401+0000","last_undegraded":"2024-03-19T01:07:20.679401+0000","last_fullsized":"2024-03-19T01:07:20.679401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:15:48.807200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247876+0000","last_change":"2024-03-19T01:07:15.479593+0000","last_active":"2024-03-19T01:07:21.247876+0000","last_peered":"2024-03-19T01:07:21.247876+0000","last_clean":"2024-03-19T01:07:21.247876+0000","last_became_active":"2024-03-19T01:07:15.479220+0000","last_became_peered":"2024-03-19T01:07:15.479220+0000","last_unstale":"2024-03-19T01:07:21.247876+0000","last_undegraded":"2024-03-19T01:07:21.247876+0000","last_fullsized":"2024-03-19T01:07:21.247876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:40:24.390845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.748918+0000","last_change":"2024-03-19T01:07:15.464737+0000","last_active":"2024-03-19T01:07:20.748918+0000","last_peered":"2024-03-19T01:07:20.748918+0000","last_clean":"2024-03-19T01:07:20.748918+0000","last_became_active":"2024-03-19T01:07:15.463641+0000","last_became_peered":"2024-03-19T01:07:15.463641+0000","last_unstale":"2024-03-19T01:07:20.748918+0000","last_undegraded":"2024-03-19T01:07:20.748918+0000","last_fullsized":"2024-03-19T01:07:20.748918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:13.964397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.838612+0000","last_change":"2024-03-19T01:07:15.492803+0000","last_active":"2024-03-19T01:07:20.838612+0000","last_peered":"2024-03-19T01:07:20.838612+0000","last_clean":"2024-03-19T01:07:20.838612+0000","last_became_active":"2024-03-19T01:07:15.492422+0000","last_became_peered":"2024-03-19T01:07:15.492422+0000","last_unstale":"2024-03-19T01:07:20.838612+0000","last_undegraded":"2024-03-19T01:07:20.838612+0000","last_fullsized":"2024-03-19T01:07:20.838612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:59:08.292399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839201+0000","last_change":"2024-03-19T01:07:15.498073+0000","last_active":"2024-03-19T01:07:20.839201+0000","last_peered":"2024-03-19T01:07:20.839201+0000","last_clean":"2024-03-19T01:07:20.839201+0000","last_became_active":"2024-03-19T01:07:15.493650+0000","last_became_peered":"2024-03-19T01:07:15.493650+0000","last_unstale":"2024-03-19T01:07:20.839201+0000","last_undegraded":"2024-03-19T01:07:20.839201+0000","last_fullsized":"2024-03-19T01:07:20.839201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:25:28.908122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248189+0000","last_change":"2024-03-19T01:07:15.496434+0000","last_active":"2024-03-19T01:07:21.248189+0000","last_peered":"2024-03-19T01:07:21.248189+0000","last_clean":"2024-03-19T01:07:21.248189+0000","last_became_active":"2024-03-19T01:07:15.495956+0000","last_became_peered":"2024-03-19T01:07:15.495956+0000","last_unstale":"2024-03-19T01:07:21.248189+0000","last_undegraded":"2024-03-19T01:07:21.248189+0000","last_fullsized":"2024-03-19T01:07:21.248189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:49.505947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249606+0000","last_change":"2024-03-19T01:07:15.504771+0000","last_active":"2024-03-19T01:07:21.249606+0000","last_peered":"2024-03-19T01:07:21.249606+0000","last_clean":"2024-03-19T01:07:21.249606+0000","last_became_active":"2024-03-19T01:07:15.504485+0000","last_became_peered":"2024-03-19T01:07:15.504485+0000","last_unstale":"2024-03-19T01:07:21.249606+0000","last_undegraded":"2024-03-19T01:07:21.249606+0000","last_fullsized":"2024-03-19T01:07:21.249606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:17:36.760943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679381+0000","last_change":"2024-03-19T01:07:15.491045+0000","last_active":"2024-03-19T01:07:20.679381+0000","last_peered":"2024-03-19T01:07:20.679381+0000","last_clean":"2024-03-19T01:07:20.679381+0000","last_became_active":"2024-03-19T01:07:15.488914+0000","last_became_peered":"2024-03-19T01:07:15.488914+0000","last_unstale":"2024-03-19T01:07:20.679381+0000","last_undegraded":"2024-03-19T01:07:20.679381+0000","last_fullsized":"2024-03-19T01:07:20.679381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:14.853457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.676146+0000","last_change":"2024-03-19T01:07:15.495306+0000","last_active":"2024-03-19T01:07:20.676146+0000","last_peered":"2024-03-19T01:07:20.676146+0000","last_clean":"2024-03-19T01:07:20.676146+0000","last_became_active":"2024-03-19T01:07:15.494906+0000","last_became_peered":"2024-03-19T01:07:15.494906+0000","last_unstale":"2024-03-19T01:07:20.676146+0000","last_undegraded":"2024-03-19T01:07:20.676146+0000","last_fullsized":"2024-03-19T01:07:20.676146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:48.176410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.675787+0000","last_change":"2024-03-19T01:07:19.689402+0000","last_active":"2024-03-19T01:07:20.675787+0000","last_peered":"2024-03-19T01:07:20.675787+0000","last_clean":"2024-03-19T01:07:20.675787+0000","last_became_active":"2024-03-19T01:07:19.688799+0000","last_became_peered":"2024-03-19T01:07:19.688799+0000","last_unstale":"2024-03-19T01:07:20.675787+0000","last_undegraded":"2024-03-19T01:07:20.675787+0000","last_fullsized":"2024-03-19T01:07:20.675787+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:51.890047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:50.820739+0000","last_change":"2024-03-19T01:07:14.477348+0000","last_active":"2024-03-19T01:07:50.820739+0000","last_peered":"2024-03-19T01:07:50.820739+0000","last_clean":"2024-03-19T01:07:50.820739+0000","last_became_active":"2024-03-19T01:07:14.476946+0000","last_became_peered":"2024-03-19T01:07:14.476946+0000","last_unstale":"2024-03-19T01:07:50.820739+0000","last_undegraded":"2024-03-19T01:07:50.820739+0000","last_fullsized":"2024-03-19T01:07:50.820739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:19.269837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249233+0000","last_change":"2024-03-19T01:07:15.495646+0000","last_active":"2024-03-19T01:07:21.249233+0000","last_peered":"2024-03-19T01:07:21.249233+0000","last_clean":"2024-03-19T01:07:21.249233+0000","last_became_active":"2024-03-19T01:07:15.495203+0000","last_became_peered":"2024-03-19T01:07:15.495203+0000","last_unstale":"2024-03-19T01:07:21.249233+0000","last_undegraded":"2024-03-19T01:07:21.249233+0000","last_fullsized":"2024-03-19T01:07:21.249233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:01:31.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247484+0000","last_change":"2024-03-19T01:07:19.690465+0000","last_active":"2024-03-19T01:07:21.247484+0000","last_peered":"2024-03-19T01:07:21.247484+0000","last_clean":"2024-03-19T01:07:21.247484+0000","last_became_active":"2024-03-19T01:07:19.689852+0000","last_became_peered":"2024-03-19T01:07:19.689852+0000","last_unstale":"2024-03-19T01:07:21.247484+0000","last_undegraded":"2024-03-19T01:07:21.247484+0000","last_fullsized":"2024-03-19T01:07:21.247484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:18.891411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249459+0000","last_change":"2024-03-19T01:07:14.487165+0000","last_active":"2024-03-19T01:07:21.249459+0000","last_peered":"2024-03-19T01:07:21.249459+0000","last_clean":"2024-03-19T01:07:21.249459+0000","last_became_active":"2024-03-19T01:07:14.484194+0000","last_became_peered":"2024-03-19T01:07:14.484194+0000","last_unstale":"2024-03-19T01:07:21.249459+0000","last_undegraded":"2024-03-19T01:07:21.249459+0000","last_fullsized":"2024-03-19T01:07:21.249459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:54:47.914565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839380+0000","last_change":"2024-03-19T01:07:15.498687+0000","last_active":"2024-03-19T01:07:20.839380+0000","last_peered":"2024-03-19T01:07:20.839380+0000","last_clean":"2024-03-19T01:07:20.839380+0000","last_became_active":"2024-03-19T01:07:15.498146+0000","last_became_peered":"2024-03-19T01:07:15.498146+0000","last_unstale":"2024-03-19T01:07:20.839380+0000","last_undegraded":"2024-03-19T01:07:20.839380+0000","last_fullsized":"2024-03-19T01:07:20.839380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:47:21.846333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247854+0000","last_change":"2024-03-19T01:07:19.690691+0000","last_active":"2024-03-19T01:07:21.247854+0000","last_peered":"2024-03-19T01:07:21.247854+0000","last_clean":"2024-03-19T01:07:21.247854+0000","last_became_active":"2024-03-19T01:07:19.690083+0000","last_became_peered":"2024-03-19T01:07:19.690083+0000","last_unstale":"2024-03-19T01:07:21.247854+0000","last_undegraded":"2024-03-19T01:07:21.247854+0000","last_fullsized":"2024-03-19T01:07:21.247854+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:51:30.205694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:50.817823+0000","last_change":"2024-03-19T01:07:14.478366+0000","last_active":"2024-03-19T01:07:50.817823+0000","last_peered":"2024-03-19T01:07:50.817823+0000","last_clean":"2024-03-19T01:07:50.817823+0000","last_became_active":"2024-03-19T01:07:14.477998+0000","last_became_peered":"2024-03-19T01:07:14.477998+0000","last_unstale":"2024-03-19T01:07:50.817823+0000","last_undegraded":"2024-03-19T01:07:50.817823+0000","last_fullsized":"2024-03-19T01:07:50.817823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:33:31.284602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248995+0000","last_change":"2024-03-19T01:07:15.489371+0000","last_active":"2024-03-19T01:07:21.248995+0000","last_peered":"2024-03-19T01:07:21.248995+0000","last_clean":"2024-03-19T01:07:21.248995+0000","last_became_active":"2024-03-19T01:07:15.488975+0000","last_became_peered":"2024-03-19T01:07:15.488975+0000","last_unstale":"2024-03-19T01:07:21.248995+0000","last_undegraded":"2024-03-19T01:07:21.248995+0000","last_fullsized":"2024-03-19T01:07:21.248995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:04:47.640512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247980+0000","last_change":"2024-03-19T01:07:19.691590+0000","last_active":"2024-03-19T01:07:21.247980+0000","last_peered":"2024-03-19T01:07:21.247980+0000","last_clean":"2024-03-19T01:07:21.247980+0000","last_became_active":"2024-03-19T01:07:19.691037+0000","last_became_peered":"2024-03-19T01:07:19.691037+0000","last_unstale":"2024-03-19T01:07:21.247980+0000","last_undegraded":"2024-03-19T01:07:21.247980+0000","last_fullsized":"2024-03-19T01:07:21.247980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:20:02.007483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.794233+0000","last_change":"2024-03-19T01:07:14.475653+0000","last_active":"2024-03-19T01:07:49.794233+0000","last_peered":"2024-03-19T01:07:49.794233+0000","last_clean":"2024-03-19T01:07:49.794233+0000","last_became_active":"2024-03-19T01:07:14.475315+0000","last_became_peered":"2024-03-19T01:07:14.475315+0000","last_unstale":"2024-03-19T01:07:49.794233+0000","last_undegraded":"2024-03-19T01:07:49.794233+0000","last_fullsized":"2024-03-19T01:07:49.794233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:44:04.656950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248973+0000","last_change":"2024-03-19T01:07:15.480882+0000","last_active":"2024-03-19T01:07:21.248973+0000","last_peered":"2024-03-19T01:07:21.248973+0000","last_clean":"2024-03-19T01:07:21.248973+0000","last_became_active":"2024-03-19T01:07:15.480541+0000","last_became_peered":"2024-03-19T01:07:15.480541+0000","last_unstale":"2024-03-19T01:07:21.248973+0000","last_undegraded":"2024-03-19T01:07:21.248973+0000","last_fullsized":"2024-03-19T01:07:21.248973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:37:24.724531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.676034+0000","last_change":"2024-03-19T01:07:19.672979+0000","last_active":"2024-03-19T01:07:20.676034+0000","last_peered":"2024-03-19T01:07:20.676034+0000","last_clean":"2024-03-19T01:07:20.676034+0000","last_became_active":"2024-03-19T01:07:19.672530+0000","last_became_peered":"2024-03-19T01:07:19.672530+0000","last_unstale":"2024-03-19T01:07:20.676034+0000","last_undegraded":"2024-03-19T01:07:20.676034+0000","last_fullsized":"2024-03-19T01:07:20.676034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:12.948079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:50.761371+0000","last_change":"2024-03-19T01:07:14.475620+0000","last_active":"2024-03-19T01:07:50.761371+0000","last_peered":"2024-03-19T01:07:50.761371+0000","last_clean":"2024-03-19T01:07:50.761371+0000","last_became_active":"2024-03-19T01:07:14.475029+0000","last_became_peered":"2024-03-19T01:07:14.475029+0000","last_unstale":"2024-03-19T01:07:50.761371+0000","last_undegraded":"2024-03-19T01:07:50.761371+0000","last_fullsized":"2024-03-19T01:07:50.761371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:48:53.983457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248142+0000","last_change":"2024-03-19T01:07:15.484680+0000","last_active":"2024-03-19T01:07:21.248142+0000","last_peered":"2024-03-19T01:07:21.248142+0000","last_clean":"2024-03-19T01:07:21.248142+0000","last_became_active":"2024-03-19T01:07:15.484333+0000","last_became_peered":"2024-03-19T01:07:15.484333+0000","last_unstale":"2024-03-19T01:07:21.248142+0000","last_undegraded":"2024-03-19T01:07:21.248142+0000","last_fullsized":"2024-03-19T01:07:21.248142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:38:23.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.675882+0000","last_change":"2024-03-19T01:07:19.692211+0000","last_active":"2024-03-19T01:07:20.675882+0000","last_peered":"2024-03-19T01:07:20.675882+0000","last_clean":"2024-03-19T01:07:20.675882+0000","last_became_active":"2024-03-19T01:07:19.691756+0000","last_became_peered":"2024-03-19T01:07:19.691756+0000","last_unstale":"2024-03-19T01:07:20.675882+0000","last_undegraded":"2024-03-19T01:07:20.675882+0000","last_fullsized":"2024-03-19T01:07:20.675882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:14:07.357956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.771315+0000","last_change":"2024-03-19T01:07:14.481499+0000","last_active":"2024-03-19T01:07:48.771315+0000","last_peered":"2024-03-19T01:07:48.771315+0000","last_clean":"2024-03-19T01:07:48.771315+0000","last_became_active":"2024-03-19T01:07:14.481123+0000","last_became_peered":"2024-03-19T01:07:14.481123+0000","last_unstale":"2024-03-19T01:07:48.771315+0000","last_undegraded":"2024-03-19T01:07:48.771315+0000","last_fullsized":"2024-03-19T01:07:48.771315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:36:15.349748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249214+0000","last_change":"2024-03-19T01:07:15.499744+0000","last_active":"2024-03-19T01:07:21.249214+0000","last_peered":"2024-03-19T01:07:21.249214+0000","last_clean":"2024-03-19T01:07:21.249214+0000","last_became_active":"2024-03-19T01:07:15.499037+0000","last_became_peered":"2024-03-19T01:07:15.499037+0000","last_unstale":"2024-03-19T01:07:21.249214+0000","last_undegraded":"2024-03-19T01:07:21.249214+0000","last_fullsized":"2024-03-19T01:07:21.249214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:20:35.524664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249488+0000","last_change":"2024-03-19T01:07:19.684187+0000","last_active":"2024-03-19T01:07:21.249488+0000","last_peered":"2024-03-19T01:07:21.249488+0000","last_clean":"2024-03-19T01:07:21.249488+0000","last_became_active":"2024-03-19T01:07:19.683782+0000","last_became_peered":"2024-03-19T01:07:19.683782+0000","last_unstale":"2024-03-19T01:07:21.249488+0000","last_undegraded":"2024-03-19T01:07:21.249488+0000","last_fullsized":"2024-03-19T01:07:21.249488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:17:50.818003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:47.813999+0000","last_change":"2024-03-19T01:07:14.485538+0000","last_active":"2024-03-19T01:07:47.813999+0000","last_peered":"2024-03-19T01:07:47.813999+0000","last_clean":"2024-03-19T01:07:47.813999+0000","last_became_active":"2024-03-19T01:07:14.485004+0000","last_became_peered":"2024-03-19T01:07:14.485004+0000","last_unstale":"2024-03-19T01:07:47.813999+0000","last_undegraded":"2024-03-19T01:07:47.813999+0000","last_fullsized":"2024-03-19T01:07:47.813999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:55:50.853725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248200+0000","last_change":"2024-03-19T01:07:15.503370+0000","last_active":"2024-03-19T01:07:21.248200+0000","last_peered":"2024-03-19T01:07:21.248200+0000","last_clean":"2024-03-19T01:07:21.248200+0000","last_became_active":"2024-03-19T01:07:15.501375+0000","last_became_peered":"2024-03-19T01:07:15.501375+0000","last_unstale":"2024-03-19T01:07:21.248200+0000","last_undegraded":"2024-03-19T01:07:21.248200+0000","last_fullsized":"2024-03-19T01:07:21.248200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:36:01.814936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839459+0000","last_change":"2024-03-19T01:07:19.697067+0000","last_active":"2024-03-19T01:07:20.839459+0000","last_peered":"2024-03-19T01:07:20.839459+0000","last_clean":"2024-03-19T01:07:20.839459+0000","last_became_active":"2024-03-19T01:07:19.696364+0000","last_became_peered":"2024-03-19T01:07:19.696364+0000","last_unstale":"2024-03-19T01:07:20.839459+0000","last_undegraded":"2024-03-19T01:07:20.839459+0000","last_fullsized":"2024-03-19T01:07:20.839459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:34.388407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.795031+0000","last_change":"2024-03-19T01:07:14.475361+0000","last_active":"2024-03-19T01:07:49.795031+0000","last_peered":"2024-03-19T01:07:49.795031+0000","last_clean":"2024-03-19T01:07:49.795031+0000","last_became_active":"2024-03-19T01:07:14.474577+0000","last_became_peered":"2024-03-19T01:07:14.474577+0000","last_unstale":"2024-03-19T01:07:49.795031+0000","last_undegraded":"2024-03-19T01:07:49.795031+0000","last_fullsized":"2024-03-19T01:07:49.795031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:01:04.713405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.677482+0000","last_change":"2024-03-19T01:07:15.506310+0000","last_active":"2024-03-19T01:07:20.677482+0000","last_peered":"2024-03-19T01:07:20.677482+0000","last_clean":"2024-03-19T01:07:20.677482+0000","last_became_active":"2024-03-19T01:07:15.505792+0000","last_became_peered":"2024-03-19T01:07:15.505792+0000","last_unstale":"2024-03-19T01:07:20.677482+0000","last_undegraded":"2024-03-19T01:07:20.677482+0000","last_fullsized":"2024-03-19T01:07:20.677482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:40.630816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248268+0000","last_change":"2024-03-19T01:07:19.683632+0000","last_active":"2024-03-19T01:07:21.248268+0000","last_peered":"2024-03-19T01:07:21.248268+0000","last_clean":"2024-03-19T01:07:21.248268+0000","last_became_active":"2024-03-19T01:07:19.683204+0000","last_became_peered":"2024-03-19T01:07:19.683204+0000","last_unstale":"2024-03-19T01:07:21.248268+0000","last_undegraded":"2024-03-19T01:07:21.248268+0000","last_fullsized":"2024-03-19T01:07:21.248268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:58:53.512824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:51.761848+0000","last_change":"2024-03-19T01:07:14.471347+0000","last_active":"2024-03-19T01:07:51.761848+0000","last_peered":"2024-03-19T01:07:51.761848+0000","last_clean":"2024-03-19T01:07:51.761848+0000","last_became_active":"2024-03-19T01:07:14.471001+0000","last_became_peered":"2024-03-19T01:07:14.471001+0000","last_unstale":"2024-03-19T01:07:51.761848+0000","last_undegraded":"2024-03-19T01:07:51.761848+0000","last_fullsized":"2024-03-19T01:07:51.761848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:46.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249062+0000","last_change":"2024-03-19T01:07:15.486779+0000","last_active":"2024-03-19T01:07:21.249062+0000","last_peered":"2024-03-19T01:07:21.249062+0000","last_clean":"2024-03-19T01:07:21.249062+0000","last_became_active":"2024-03-19T01:07:15.486405+0000","last_became_peered":"2024-03-19T01:07:15.486405+0000","last_unstale":"2024-03-19T01:07:21.249062+0000","last_undegraded":"2024-03-19T01:07:21.249062+0000","last_fullsized":"2024-03-19T01:07:21.249062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:14:13.350169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.677700+0000","last_change":"2024-03-19T01:07:19.686892+0000","last_active":"2024-03-19T01:07:20.677700+0000","last_peered":"2024-03-19T01:07:20.677700+0000","last_clean":"2024-03-19T01:07:20.677700+0000","last_became_active":"2024-03-19T01:07:19.686414+0000","last_became_peered":"2024-03-19T01:07:19.686414+0000","last_unstale":"2024-03-19T01:07:20.677700+0000","last_undegraded":"2024-03-19T01:07:20.677700+0000","last_fullsized":"2024-03-19T01:07:20.677700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:18:18.105905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.794360+0000","last_change":"2024-03-19T01:07:14.474384+0000","last_active":"2024-03-19T01:07:49.794360+0000","last_peered":"2024-03-19T01:07:49.794360+0000","last_clean":"2024-03-19T01:07:49.794360+0000","last_became_active":"2024-03-19T01:07:14.473996+0000","last_became_peered":"2024-03-19T01:07:14.473996+0000","last_unstale":"2024-03-19T01:07:49.794360+0000","last_undegraded":"2024-03-19T01:07:49.794360+0000","last_fullsized":"2024-03-19T01:07:49.794360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:12:33.079612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839265+0000","last_change":"2024-03-19T01:07:15.499507+0000","last_active":"2024-03-19T01:07:20.839265+0000","last_peered":"2024-03-19T01:07:20.839265+0000","last_clean":"2024-03-19T01:07:20.839265+0000","last_became_active":"2024-03-19T01:07:15.497261+0000","last_became_peered":"2024-03-19T01:07:15.497261+0000","last_unstale":"2024-03-19T01:07:20.839265+0000","last_undegraded":"2024-03-19T01:07:20.839265+0000","last_fullsized":"2024-03-19T01:07:20.839265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:40:06.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248149+0000","last_change":"2024-03-19T01:07:19.690847+0000","last_active":"2024-03-19T01:07:21.248149+0000","last_peered":"2024-03-19T01:07:21.248149+0000","last_clean":"2024-03-19T01:07:21.248149+0000","last_became_active":"2024-03-19T01:07:19.689762+0000","last_became_peered":"2024-03-19T01:07:19.689762+0000","last_unstale":"2024-03-19T01:07:21.248149+0000","last_undegraded":"2024-03-19T01:07:21.248149+0000","last_fullsized":"2024-03-19T01:07:21.248149+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:24:40.213171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.758468+0000","last_change":"2024-03-19T01:07:14.473744+0000","last_active":"2024-03-19T01:07:48.758468+0000","last_peered":"2024-03-19T01:07:48.758468+0000","last_clean":"2024-03-19T01:07:48.758468+0000","last_became_active":"2024-03-19T01:07:14.473404+0000","last_became_peered":"2024-03-19T01:07:14.473404+0000","last_unstale":"2024-03-19T01:07:48.758468+0000","last_undegraded":"2024-03-19T01:07:48.758468+0000","last_fullsized":"2024-03-19T01:07:48.758468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:29:39.892678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249119+0000","last_change":"2024-03-19T01:07:15.481444+0000","last_active":"2024-03-19T01:07:21.249119+0000","last_peered":"2024-03-19T01:07:21.249119+0000","last_clean":"2024-03-19T01:07:21.249119+0000","last_became_active":"2024-03-19T01:07:15.481106+0000","last_became_peered":"2024-03-19T01:07:15.481106+0000","last_unstale":"2024-03-19T01:07:21.249119+0000","last_undegraded":"2024-03-19T01:07:21.249119+0000","last_fullsized":"2024-03-19T01:07:21.249119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:00.263063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.678244+0000","last_change":"2024-03-19T01:07:19.686919+0000","last_active":"2024-03-19T01:07:20.678244+0000","last_peered":"2024-03-19T01:07:20.678244+0000","last_clean":"2024-03-19T01:07:20.678244+0000","last_became_active":"2024-03-19T01:07:19.686416+0000","last_became_peered":"2024-03-19T01:07:19.686416+0000","last_unstale":"2024-03-19T01:07:20.678244+0000","last_undegraded":"2024-03-19T01:07:20.678244+0000","last_fullsized":"2024-03-19T01:07:20.678244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:29:11.068957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'22","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:51.767492+0000","last_change":"2024-03-19T01:07:14.468630+0000","last_active":"2024-03-19T01:07:51.767492+0000","last_peered":"2024-03-19T01:07:51.767492+0000","last_clean":"2024-03-19T01:07:51.767492+0000","last_became_active":"2024-03-19T01:07:14.468293+0000","last_became_peered":"2024-03-19T01:07:14.468293+0000","last_unstale":"2024-03-19T01:07:51.767492+0000","last_undegraded":"2024-03-19T01:07:51.767492+0000","last_fullsized":"2024-03-19T01:07:51.767492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:53.575507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":29,"num_write":24,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248459+0000","last_change":"2024-03-19T01:07:15.477226+0000","last_active":"2024-03-19T01:07:21.248459+0000","last_peered":"2024-03-19T01:07:21.248459+0000","last_clean":"2024-03-19T01:07:21.248459+0000","last_became_active":"2024-03-19T01:07:15.476882+0000","last_became_peered":"2024-03-19T01:07:15.476882+0000","last_unstale":"2024-03-19T01:07:21.248459+0000","last_undegraded":"2024-03-19T01:07:21.248459+0000","last_fullsized":"2024-03-19T01:07:21.248459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:11:10.051837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249620+0000","last_change":"2024-03-19T01:07:19.690004+0000","last_active":"2024-03-19T01:07:21.249620+0000","last_peered":"2024-03-19T01:07:21.249620+0000","last_clean":"2024-03-19T01:07:21.249620+0000","last_became_active":"2024-03-19T01:07:19.689573+0000","last_became_peered":"2024-03-19T01:07:19.689573+0000","last_unstale":"2024-03-19T01:07:21.249620+0000","last_undegraded":"2024-03-19T01:07:21.249620+0000","last_fullsized":"2024-03-19T01:07:21.249620+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:31:21.788219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.797100+0000","last_change":"2024-03-19T01:07:14.487373+0000","last_active":"2024-03-19T01:07:49.797100+0000","last_peered":"2024-03-19T01:07:49.797100+0000","last_clean":"2024-03-19T01:07:49.797100+0000","last_became_active":"2024-03-19T01:07:14.487077+0000","last_became_peered":"2024-03-19T01:07:14.487077+0000","last_unstale":"2024-03-19T01:07:49.797100+0000","last_undegraded":"2024-03-19T01:07:49.797100+0000","last_fullsized":"2024-03-19T01:07:49.797100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:33:01.789431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.748660+0000","last_change":"2024-03-19T01:07:15.498009+0000","last_active":"2024-03-19T01:07:20.748660+0000","last_peered":"2024-03-19T01:07:20.748660+0000","last_clean":"2024-03-19T01:07:20.748660+0000","last_became_active":"2024-03-19T01:07:15.496707+0000","last_became_peered":"2024-03-19T01:07:15.496707+0000","last_unstale":"2024-03-19T01:07:20.748660+0000","last_undegraded":"2024-03-19T01:07:20.748660+0000","last_fullsized":"2024-03-19T01:07:20.748660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:50.058299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248414+0000","last_change":"2024-03-19T01:07:19.677635+0000","last_active":"2024-03-19T01:07:21.248414+0000","last_peered":"2024-03-19T01:07:21.248414+0000","last_clean":"2024-03-19T01:07:21.248414+0000","last_became_active":"2024-03-19T01:07:19.677207+0000","last_became_peered":"2024-03-19T01:07:19.677207+0000","last_unstale":"2024-03-19T01:07:21.248414+0000","last_undegraded":"2024-03-19T01:07:21.248414+0000","last_fullsized":"2024-03-19T01:07:21.248414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:23:28.650962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.797335+0000","last_change":"2024-03-19T01:07:14.482054+0000","last_active":"2024-03-19T01:07:49.797335+0000","last_peered":"2024-03-19T01:07:49.797335+0000","last_clean":"2024-03-19T01:07:49.797335+0000","last_became_active":"2024-03-19T01:07:14.481703+0000","last_became_peered":"2024-03-19T01:07:14.481703+0000","last_unstale":"2024-03-19T01:07:49.797335+0000","last_undegraded":"2024-03-19T01:07:49.797335+0000","last_fullsized":"2024-03-19T01:07:49.797335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:02:43.565442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679344+0000","last_change":"2024-03-19T01:07:15.487236+0000","last_active":"2024-03-19T01:07:20.679344+0000","last_peered":"2024-03-19T01:07:20.679344+0000","last_clean":"2024-03-19T01:07:20.679344+0000","last_became_active":"2024-03-19T01:07:15.486931+0000","last_became_peered":"2024-03-19T01:07:15.486931+0000","last_unstale":"2024-03-19T01:07:20.679344+0000","last_undegraded":"2024-03-19T01:07:20.679344+0000","last_fullsized":"2024-03-19T01:07:20.679344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:22.201904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679377+0000","last_change":"2024-03-19T01:07:19.693813+0000","last_active":"2024-03-19T01:07:20.679377+0000","last_peered":"2024-03-19T01:07:20.679377+0000","last_clean":"2024-03-19T01:07:20.679377+0000","last_became_active":"2024-03-19T01:07:19.693384+0000","last_became_peered":"2024-03-19T01:07:19.693384+0000","last_unstale":"2024-03-19T01:07:20.679377+0000","last_undegraded":"2024-03-19T01:07:20.679377+0000","last_fullsized":"2024-03-19T01:07:20.679377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:09.372461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:51.769768+0000","last_change":"2024-03-19T01:07:14.468467+0000","last_active":"2024-03-19T01:07:51.769768+0000","last_peered":"2024-03-19T01:07:51.769768+0000","last_clean":"2024-03-19T01:07:51.769768+0000","last_became_active":"2024-03-19T01:07:14.468074+0000","last_became_peered":"2024-03-19T01:07:14.468074+0000","last_unstale":"2024-03-19T01:07:51.769768+0000","last_undegraded":"2024-03-19T01:07:51.769768+0000","last_fullsized":"2024-03-19T01:07:51.769768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:07:49.216998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679452+0000","last_change":"2024-03-19T01:07:15.475310+0000","last_active":"2024-03-19T01:07:20.679452+0000","last_peered":"2024-03-19T01:07:20.679452+0000","last_clean":"2024-03-19T01:07:20.679452+0000","last_became_active":"2024-03-19T01:07:15.474914+0000","last_became_peered":"2024-03-19T01:07:15.474914+0000","last_unstale":"2024-03-19T01:07:20.679452+0000","last_undegraded":"2024-03-19T01:07:20.679452+0000","last_fullsized":"2024-03-19T01:07:20.679452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:34.036892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.749576+0000","last_change":"2024-03-19T01:07:19.697937+0000","last_active":"2024-03-19T01:07:20.749576+0000","last_peered":"2024-03-19T01:07:20.749576+0000","last_clean":"2024-03-19T01:07:20.749576+0000","last_became_active":"2024-03-19T01:07:19.697447+0000","last_became_peered":"2024-03-19T01:07:19.697447+0000","last_unstale":"2024-03-19T01:07:20.749576+0000","last_undegraded":"2024-03-19T01:07:20.749576+0000","last_fullsized":"2024-03-19T01:07:20.749576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:49.259118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.774194+0000","last_change":"2024-03-19T01:07:14.476382+0000","last_active":"2024-03-19T01:07:48.774194+0000","last_peered":"2024-03-19T01:07:48.774194+0000","last_clean":"2024-03-19T01:07:48.774194+0000","last_became_active":"2024-03-19T01:07:14.476089+0000","last_became_peered":"2024-03-19T01:07:14.476089+0000","last_unstale":"2024-03-19T01:07:48.774194+0000","last_undegraded":"2024-03-19T01:07:48.774194+0000","last_fullsized":"2024-03-19T01:07:48.774194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:57:42.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249271+0000","last_change":"2024-03-19T01:07:15.480912+0000","last_active":"2024-03-19T01:07:21.249271+0000","last_peered":"2024-03-19T01:07:21.249271+0000","last_clean":"2024-03-19T01:07:21.249271+0000","last_became_active":"2024-03-19T01:07:15.480602+0000","last_became_peered":"2024-03-19T01:07:15.480602+0000","last_unstale":"2024-03-19T01:07:21.249271+0000","last_undegraded":"2024-03-19T01:07:21.249271+0000","last_fullsized":"2024-03-19T01:07:21.249271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:18:31.947781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249565+0000","last_change":"2024-03-19T01:07:19.688235+0000","last_active":"2024-03-19T01:07:21.249565+0000","last_peered":"2024-03-19T01:07:21.249565+0000","last_clean":"2024-03-19T01:07:21.249565+0000","last_became_active":"2024-03-19T01:07:19.687879+0000","last_became_peered":"2024-03-19T01:07:19.687879+0000","last_unstale":"2024-03-19T01:07:21.249565+0000","last_undegraded":"2024-03-19T01:07:21.249565+0000","last_fullsized":"2024-03-19T01:07:21.249565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:29.465971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.758216+0000","last_change":"2024-03-19T01:07:14.474346+0000","last_active":"2024-03-19T01:07:48.758216+0000","last_peered":"2024-03-19T01:07:48.758216+0000","last_clean":"2024-03-19T01:07:48.758216+0000","last_became_active":"2024-03-19T01:07:14.473989+0000","last_became_peered":"2024-03-19T01:07:14.473989+0000","last_unstale":"2024-03-19T01:07:48.758216+0000","last_undegraded":"2024-03-19T01:07:48.758216+0000","last_fullsized":"2024-03-19T01:07:48.758216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:41.004155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249223+0000","last_change":"2024-03-19T01:07:15.482326+0000","last_active":"2024-03-19T01:07:21.249223+0000","last_peered":"2024-03-19T01:07:21.249223+0000","last_clean":"2024-03-19T01:07:21.249223+0000","last_became_active":"2024-03-19T01:07:15.481492+0000","last_became_peered":"2024-03-19T01:07:15.481492+0000","last_unstale":"2024-03-19T01:07:21.249223+0000","last_undegraded":"2024-03-19T01:07:21.249223+0000","last_fullsized":"2024-03-19T01:07:21.249223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:17:31.572104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248476+0000","last_change":"2024-03-19T01:07:19.668853+0000","last_active":"2024-03-19T01:07:21.248476+0000","last_peered":"2024-03-19T01:07:21.248476+0000","last_clean":"2024-03-19T01:07:21.248476+0000","last_became_active":"2024-03-19T01:07:19.668428+0000","last_became_peered":"2024-03-19T01:07:19.668428+0000","last_unstale":"2024-03-19T01:07:21.248476+0000","last_undegraded":"2024-03-19T01:07:21.248476+0000","last_fullsized":"2024-03-19T01:07:21.248476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:50:43.409586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.678911+0000","last_change":"2024-03-19T01:07:14.469548+0000","last_active":"2024-03-19T01:07:20.678911+0000","last_peered":"2024-03-19T01:07:20.678911+0000","last_clean":"2024-03-19T01:07:20.678911+0000","last_became_active":"2024-03-19T01:07:14.469200+0000","last_became_peered":"2024-03-19T01:07:14.469200+0000","last_unstale":"2024-03-19T01:07:20.678911+0000","last_undegraded":"2024-03-19T01:07:20.678911+0000","last_fullsized":"2024-03-19T01:07:20.678911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:41:53.179343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248608+0000","last_change":"2024-03-19T01:07:15.495300+0000","last_active":"2024-03-19T01:07:21.248608+0000","last_peered":"2024-03-19T01:07:21.248608+0000","last_clean":"2024-03-19T01:07:21.248608+0000","last_became_active":"2024-03-19T01:07:15.494938+0000","last_became_peered":"2024-03-19T01:07:15.494938+0000","last_unstale":"2024-03-19T01:07:21.248608+0000","last_undegraded":"2024-03-19T01:07:21.248608+0000","last_fullsized":"2024-03-19T01:07:21.248608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:50:38.855857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.749242+0000","last_change":"2024-03-19T01:07:19.693261+0000","last_active":"2024-03-19T01:07:20.749242+0000","last_peered":"2024-03-19T01:07:20.749242+0000","last_clean":"2024-03-19T01:07:20.749242+0000","last_became_active":"2024-03-19T01:07:19.692598+0000","last_became_peered":"2024-03-19T01:07:19.692598+0000","last_unstale":"2024-03-19T01:07:20.749242+0000","last_undegraded":"2024-03-19T01:07:20.749242+0000","last_fullsized":"2024-03-19T01:07:20.749242+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:43:29.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:50.823201+0000","last_change":"2024-03-19T01:07:14.481569+0000","last_active":"2024-03-19T01:07:50.823201+0000","last_peered":"2024-03-19T01:07:50.823201+0000","last_clean":"2024-03-19T01:07:50.823201+0000","last_became_active":"2024-03-19T01:07:14.481209+0000","last_became_peered":"2024-03-19T01:07:14.481209+0000","last_unstale":"2024-03-19T01:07:50.823201+0000","last_undegraded":"2024-03-19T01:07:50.823201+0000","last_fullsized":"2024-03-19T01:07:50.823201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:39.411207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248157+0000","last_change":"2024-03-19T01:07:15.496852+0000","last_active":"2024-03-19T01:07:21.248157+0000","last_peered":"2024-03-19T01:07:21.248157+0000","last_clean":"2024-03-19T01:07:21.248157+0000","last_became_active":"2024-03-19T01:07:15.496455+0000","last_became_peered":"2024-03-19T01:07:15.496455+0000","last_unstale":"2024-03-19T01:07:21.248157+0000","last_undegraded":"2024-03-19T01:07:21.248157+0000","last_fullsized":"2024-03-19T01:07:21.248157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:43.419496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.749557+0000","last_change":"2024-03-19T01:07:19.692856+0000","last_active":"2024-03-19T01:07:20.749557+0000","last_peered":"2024-03-19T01:07:20.749557+0000","last_clean":"2024-03-19T01:07:20.749557+0000","last_became_active":"2024-03-19T01:07:19.690712+0000","last_became_peered":"2024-03-19T01:07:19.690712+0000","last_unstale":"2024-03-19T01:07:20.749557+0000","last_undegraded":"2024-03-19T01:07:20.749557+0000","last_fullsized":"2024-03-19T01:07:20.749557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:29:28.473062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248698+0000","last_change":"2024-03-19T01:07:15.474375+0000","last_active":"2024-03-19T01:07:21.248698+0000","last_peered":"2024-03-19T01:07:21.248698+0000","last_clean":"2024-03-19T01:07:21.248698+0000","last_became_active":"2024-03-19T01:07:15.473427+0000","last_became_peered":"2024-03-19T01:07:15.473427+0000","last_unstale":"2024-03-19T01:07:21.248698+0000","last_undegraded":"2024-03-19T01:07:21.248698+0000","last_fullsized":"2024-03-19T01:07:21.248698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:42:22.235054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:50.818534+0000","last_change":"2024-03-19T01:07:14.490924+0000","last_active":"2024-03-19T01:07:50.818534+0000","last_peered":"2024-03-19T01:07:50.818534+0000","last_clean":"2024-03-19T01:07:50.818534+0000","last_became_active":"2024-03-19T01:07:14.490621+0000","last_became_peered":"2024-03-19T01:07:14.490621+0000","last_unstale":"2024-03-19T01:07:50.818534+0000","last_undegraded":"2024-03-19T01:07:50.818534+0000","last_fullsized":"2024-03-19T01:07:50.818534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:02:52.254138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:22.692597+0000","last_change":"2024-03-19T01:07:19.672258+0000","last_active":"2024-03-19T01:07:22.692597+0000","last_peered":"2024-03-19T01:07:22.692597+0000","last_clean":"2024-03-19T01:07:22.692597+0000","last_became_active":"2024-03-19T01:07:19.671862+0000","last_became_peered":"2024-03-19T01:07:19.671862+0000","last_unstale":"2024-03-19T01:07:22.692597+0000","last_undegraded":"2024-03-19T01:07:22.692597+0000","last_fullsized":"2024-03-19T01:07:22.692597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:15.321703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248509+0000","last_change":"2024-03-19T01:07:15.493218+0000","last_active":"2024-03-19T01:07:21.248509+0000","last_peered":"2024-03-19T01:07:21.248509+0000","last_clean":"2024-03-19T01:07:21.248509+0000","last_became_active":"2024-03-19T01:07:15.492751+0000","last_became_peered":"2024-03-19T01:07:15.492751+0000","last_unstale":"2024-03-19T01:07:21.248509+0000","last_undegraded":"2024-03-19T01:07:21.248509+0000","last_fullsized":"2024-03-19T01:07:21.248509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:48:32.105214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.798687+0000","last_change":"2024-03-19T01:07:14.473349+0000","last_active":"2024-03-19T01:07:49.798687+0000","last_peered":"2024-03-19T01:07:49.798687+0000","last_clean":"2024-03-19T01:07:49.798687+0000","last_became_active":"2024-03-19T01:07:14.473014+0000","last_became_peered":"2024-03-19T01:07:14.473014+0000","last_unstale":"2024-03-19T01:07:49.798687+0000","last_undegraded":"2024-03-19T01:07:49.798687+0000","last_fullsized":"2024-03-19T01:07:49.798687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:27:34.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679117+0000","last_change":"2024-03-19T01:05:56.013594+0000","last_active":"2024-03-19T01:07:20.679117+0000","last_peered":"2024-03-19T01:07:20.679117+0000","last_clean":"2024-03-19T01:07:20.679117+0000","last_became_active":"2024-03-19T01:05:56.013084+0000","last_became_peered":"2024-03-19T01:05:56.013084+0000","last_unstale":"2024-03-19T01:07:20.679117+0000","last_undegraded":"2024-03-19T01:07:20.679117+0000","last_fullsized":"2024-03-19T01:07:20.679117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_clean_scrub_stamp":"2024-03-19T01:02:06.532343+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:23.440161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249530+0000","last_change":"2024-03-19T01:07:19.691111+0000","last_active":"2024-03-19T01:07:21.249530+0000","last_peered":"2024-03-19T01:07:21.249530+0000","last_clean":"2024-03-19T01:07:21.249530+0000","last_became_active":"2024-03-19T01:07:19.690618+0000","last_became_peered":"2024-03-19T01:07:19.690618+0000","last_unstale":"2024-03-19T01:07:21.249530+0000","last_undegraded":"2024-03-19T01:07:21.249530+0000","last_fullsized":"2024-03-19T01:07:21.249530+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:17.857394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.748435+0000","last_change":"2024-03-19T01:07:15.480452+0000","last_active":"2024-03-19T01:07:20.748435+0000","last_peered":"2024-03-19T01:07:20.748435+0000","last_clean":"2024-03-19T01:07:20.748435+0000","last_became_active":"2024-03-19T01:07:15.479897+0000","last_became_peered":"2024-03-19T01:07:15.479897+0000","last_unstale":"2024-03-19T01:07:20.748435+0000","last_undegraded":"2024-03-19T01:07:20.748435+0000","last_fullsized":"2024-03-19T01:07:20.748435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:37.841942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:47.810788+0000","last_change":"2024-03-19T01:07:14.474287+0000","last_active":"2024-03-19T01:07:47.810788+0000","last_peered":"2024-03-19T01:07:47.810788+0000","last_clean":"2024-03-19T01:07:47.810788+0000","last_became_active":"2024-03-19T01:07:14.473921+0000","last_became_peered":"2024-03-19T01:07:14.473921+0000","last_unstale":"2024-03-19T01:07:47.810788+0000","last_undegraded":"2024-03-19T01:07:47.810788+0000","last_fullsized":"2024-03-19T01:07:47.810788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:49.322029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248579+0000","last_change":"2024-03-19T01:07:19.685331+0000","last_active":"2024-03-19T01:07:21.248579+0000","last_peered":"2024-03-19T01:07:21.248579+0000","last_clean":"2024-03-19T01:07:21.248579+0000","last_became_active":"2024-03-19T01:07:19.684876+0000","last_became_peered":"2024-03-19T01:07:19.684876+0000","last_unstale":"2024-03-19T01:07:21.248579+0000","last_undegraded":"2024-03-19T01:07:21.248579+0000","last_fullsized":"2024-03-19T01:07:21.248579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:39.575348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.773339+0000","last_change":"2024-03-19T01:07:14.481019+0000","last_active":"2024-03-19T01:07:48.773339+0000","last_peered":"2024-03-19T01:07:48.773339+0000","last_clean":"2024-03-19T01:07:48.773339+0000","last_became_active":"2024-03-19T01:07:14.480666+0000","last_became_peered":"2024-03-19T01:07:14.480666+0000","last_unstale":"2024-03-19T01:07:48.773339+0000","last_undegraded":"2024-03-19T01:07:48.773339+0000","last_fullsized":"2024-03-19T01:07:48.773339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:13:33.669690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.749219+0000","last_change":"2024-03-19T01:07:15.490747+0000","last_active":"2024-03-19T01:07:20.749219+0000","last_peered":"2024-03-19T01:07:20.749219+0000","last_clean":"2024-03-19T01:07:20.749219+0000","last_became_active":"2024-03-19T01:07:15.486244+0000","last_became_peered":"2024-03-19T01:07:15.486244+0000","last_unstale":"2024-03-19T01:07:20.749219+0000","last_undegraded":"2024-03-19T01:07:20.749219+0000","last_fullsized":"2024-03-19T01:07:20.749219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:43.212972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248231+0000","last_change":"2024-03-19T01:07:19.684804+0000","last_active":"2024-03-19T01:07:21.248231+0000","last_peered":"2024-03-19T01:07:21.248231+0000","last_clean":"2024-03-19T01:07:21.248231+0000","last_became_active":"2024-03-19T01:07:19.683191+0000","last_became_peered":"2024-03-19T01:07:19.683191+0000","last_unstale":"2024-03-19T01:07:21.248231+0000","last_undegraded":"2024-03-19T01:07:21.248231+0000","last_fullsized":"2024-03-19T01:07:21.248231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:02.015499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.776065+0000","last_change":"2024-03-19T01:07:14.479282+0000","last_active":"2024-03-19T01:07:48.776065+0000","last_peered":"2024-03-19T01:07:48.776065+0000","last_clean":"2024-03-19T01:07:48.776065+0000","last_became_active":"2024-03-19T01:07:14.478876+0000","last_became_peered":"2024-03-19T01:07:14.478876+0000","last_unstale":"2024-03-19T01:07:48.776065+0000","last_undegraded":"2024-03-19T01:07:48.776065+0000","last_fullsized":"2024-03-19T01:07:48.776065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:54.179680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249258+0000","last_change":"2024-03-19T01:07:15.493498+0000","last_active":"2024-03-19T01:07:21.249258+0000","last_peered":"2024-03-19T01:07:21.249258+0000","last_clean":"2024-03-19T01:07:21.249258+0000","last_became_active":"2024-03-19T01:07:15.493110+0000","last_became_peered":"2024-03-19T01:07:15.493110+0000","last_unstale":"2024-03-19T01:07:21.249258+0000","last_undegraded":"2024-03-19T01:07:21.249258+0000","last_fullsized":"2024-03-19T01:07:21.249258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:55:23.608596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.675483+0000","last_change":"2024-03-19T01:07:19.671232+0000","last_active":"2024-03-19T01:07:20.675483+0000","last_peered":"2024-03-19T01:07:20.675483+0000","last_clean":"2024-03-19T01:07:20.675483+0000","last_became_active":"2024-03-19T01:07:19.670818+0000","last_became_peered":"2024-03-19T01:07:19.670818+0000","last_unstale":"2024-03-19T01:07:20.675483+0000","last_undegraded":"2024-03-19T01:07:20.675483+0000","last_fullsized":"2024-03-19T01:07:20.675483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:57:20.134372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.799339+0000","last_change":"2024-03-19T01:07:14.473568+0000","last_active":"2024-03-19T01:07:49.799339+0000","last_peered":"2024-03-19T01:07:49.799339+0000","last_clean":"2024-03-19T01:07:49.799339+0000","last_became_active":"2024-03-19T01:07:14.472555+0000","last_became_peered":"2024-03-19T01:07:14.472555+0000","last_unstale":"2024-03-19T01:07:49.799339+0000","last_undegraded":"2024-03-19T01:07:49.799339+0000","last_fullsized":"2024-03-19T01:07:49.799339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:28:01.541228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839538+0000","last_change":"2024-03-19T01:07:15.499905+0000","last_active":"2024-03-19T01:07:20.839538+0000","last_peered":"2024-03-19T01:07:20.839538+0000","last_clean":"2024-03-19T01:07:20.839538+0000","last_became_active":"2024-03-19T01:07:15.499560+0000","last_became_peered":"2024-03-19T01:07:15.499560+0000","last_unstale":"2024-03-19T01:07:20.839538+0000","last_undegraded":"2024-03-19T01:07:20.839538+0000","last_fullsized":"2024-03-19T01:07:20.839538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:06:26.531100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247604+0000","last_change":"2024-03-19T01:07:19.690310+0000","last_active":"2024-03-19T01:07:21.247604+0000","last_peered":"2024-03-19T01:07:21.247604+0000","last_clean":"2024-03-19T01:07:21.247604+0000","last_became_active":"2024-03-19T01:07:19.689877+0000","last_became_peered":"2024-03-19T01:07:19.689877+0000","last_unstale":"2024-03-19T01:07:21.247604+0000","last_undegraded":"2024-03-19T01:07:21.247604+0000","last_fullsized":"2024-03-19T01:07:21.247604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:06:12.730258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:46.799163+0000","last_change":"2024-03-19T01:07:14.466267+0000","last_active":"2024-03-19T01:07:46.799163+0000","last_peered":"2024-03-19T01:07:46.799163+0000","last_clean":"2024-03-19T01:07:46.799163+0000","last_became_active":"2024-03-19T01:07:14.465944+0000","last_became_peered":"2024-03-19T01:07:14.465944+0000","last_unstale":"2024-03-19T01:07:46.799163+0000","last_undegraded":"2024-03-19T01:07:46.799163+0000","last_fullsized":"2024-03-19T01:07:46.799163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:40.957200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247694+0000","last_change":"2024-03-19T01:07:15.475690+0000","last_active":"2024-03-19T01:07:21.247694+0000","last_peered":"2024-03-19T01:07:21.247694+0000","last_clean":"2024-03-19T01:07:21.247694+0000","last_became_active":"2024-03-19T01:07:15.475342+0000","last_became_peered":"2024-03-19T01:07:15.475342+0000","last_unstale":"2024-03-19T01:07:21.247694+0000","last_undegraded":"2024-03-19T01:07:21.247694+0000","last_fullsized":"2024-03-19T01:07:21.247694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:30:30.842462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679551+0000","last_change":"2024-03-19T01:07:19.695170+0000","last_active":"2024-03-19T01:07:20.679551+0000","last_peered":"2024-03-19T01:07:20.679551+0000","last_clean":"2024-03-19T01:07:20.679551+0000","last_became_active":"2024-03-19T01:07:19.694095+0000","last_became_peered":"2024-03-19T01:07:19.694095+0000","last_unstale":"2024-03-19T01:07:20.679551+0000","last_undegraded":"2024-03-19T01:07:20.679551+0000","last_fullsized":"2024-03-19T01:07:20.679551+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:33:12.420640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.795497+0000","last_change":"2024-03-19T01:07:14.487907+0000","last_active":"2024-03-19T01:07:49.795497+0000","last_peered":"2024-03-19T01:07:49.795497+0000","last_clean":"2024-03-19T01:07:49.795497+0000","last_became_active":"2024-03-19T01:07:14.487618+0000","last_became_peered":"2024-03-19T01:07:14.487618+0000","last_unstale":"2024-03-19T01:07:49.795497+0000","last_undegraded":"2024-03-19T01:07:49.795497+0000","last_fullsized":"2024-03-19T01:07:49.795497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:22:49.218797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248220+0000","last_change":"2024-03-19T01:07:15.492103+0000","last_active":"2024-03-19T01:07:21.248220+0000","last_peered":"2024-03-19T01:07:21.248220+0000","last_clean":"2024-03-19T01:07:21.248220+0000","last_became_active":"2024-03-19T01:07:15.491712+0000","last_became_peered":"2024-03-19T01:07:15.491712+0000","last_unstale":"2024-03-19T01:07:21.248220+0000","last_undegraded":"2024-03-19T01:07:21.248220+0000","last_fullsized":"2024-03-19T01:07:21.248220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:40:13.959986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:23.572718+0000","last_change":"2024-03-19T01:07:19.677392+0000","last_active":"2024-03-19T01:07:23.572718+0000","last_peered":"2024-03-19T01:07:23.572718+0000","last_clean":"2024-03-19T01:07:23.572718+0000","last_became_active":"2024-03-19T01:07:19.677024+0000","last_became_peered":"2024-03-19T01:07:19.677024+0000","last_unstale":"2024-03-19T01:07:23.572718+0000","last_undegraded":"2024-03-19T01:07:23.572718+0000","last_fullsized":"2024-03-19T01:07:23.572718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:01:43.404902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:47.798526+0000","last_change":"2024-03-19T01:07:14.481170+0000","last_active":"2024-03-19T01:07:47.798526+0000","last_peered":"2024-03-19T01:07:47.798526+0000","last_clean":"2024-03-19T01:07:47.798526+0000","last_became_active":"2024-03-19T01:07:14.480814+0000","last_became_peered":"2024-03-19T01:07:14.480814+0000","last_unstale":"2024-03-19T01:07:47.798526+0000","last_undegraded":"2024-03-19T01:07:47.798526+0000","last_fullsized":"2024-03-19T01:07:47.798526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:24:23.380331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248917+0000","last_change":"2024-03-19T01:07:15.505842+0000","last_active":"2024-03-19T01:07:21.248917+0000","last_peered":"2024-03-19T01:07:21.248917+0000","last_clean":"2024-03-19T01:07:21.248917+0000","last_became_active":"2024-03-19T01:07:15.505425+0000","last_became_peered":"2024-03-19T01:07:15.505425+0000","last_unstale":"2024-03-19T01:07:21.248917+0000","last_undegraded":"2024-03-19T01:07:21.248917+0000","last_fullsized":"2024-03-19T01:07:21.248917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:28.119864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247571+0000","last_change":"2024-03-19T01:07:19.683113+0000","last_active":"2024-03-19T01:07:21.247571+0000","last_peered":"2024-03-19T01:07:21.247571+0000","last_clean":"2024-03-19T01:07:21.247571+0000","last_became_active":"2024-03-19T01:07:19.682702+0000","last_became_peered":"2024-03-19T01:07:19.682702+0000","last_unstale":"2024-03-19T01:07:21.247571+0000","last_undegraded":"2024-03-19T01:07:21.247571+0000","last_fullsized":"2024-03-19T01:07:21.247571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:42.866408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:46.801664+0000","last_change":"2024-03-19T01:07:14.476562+0000","last_active":"2024-03-19T01:07:46.801664+0000","last_peered":"2024-03-19T01:07:46.801664+0000","last_clean":"2024-03-19T01:07:46.801664+0000","last_became_active":"2024-03-19T01:07:14.476185+0000","last_became_peered":"2024-03-19T01:07:14.476185+0000","last_unstale":"2024-03-19T01:07:46.801664+0000","last_undegraded":"2024-03-19T01:07:46.801664+0000","last_fullsized":"2024-03-19T01:07:46.801664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:35:19.860443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679657+0000","last_change":"2024-03-19T01:07:15.487163+0000","last_active":"2024-03-19T01:07:20.679657+0000","last_peered":"2024-03-19T01:07:20.679657+0000","last_clean":"2024-03-19T01:07:20.679657+0000","last_became_active":"2024-03-19T01:07:15.486845+0000","last_became_peered":"2024-03-19T01:07:15.486845+0000","last_unstale":"2024-03-19T01:07:20.679657+0000","last_undegraded":"2024-03-19T01:07:20.679657+0000","last_fullsized":"2024-03-19T01:07:20.679657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:39:09.389411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248781+0000","last_change":"2024-03-19T01:07:19.692768+0000","last_active":"2024-03-19T01:07:21.248781+0000","last_peered":"2024-03-19T01:07:21.248781+0000","last_clean":"2024-03-19T01:07:21.248781+0000","last_became_active":"2024-03-19T01:07:19.692382+0000","last_became_peered":"2024-03-19T01:07:19.692382+0000","last_unstale":"2024-03-19T01:07:21.248781+0000","last_undegraded":"2024-03-19T01:07:21.248781+0000","last_fullsized":"2024-03-19T01:07:21.248781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:26.804938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.769919+0000","last_change":"2024-03-19T01:07:14.472844+0000","last_active":"2024-03-19T01:07:48.769919+0000","last_peered":"2024-03-19T01:07:48.769919+0000","last_clean":"2024-03-19T01:07:48.769919+0000","last_became_active":"2024-03-19T01:07:14.472389+0000","last_became_peered":"2024-03-19T01:07:14.472389+0000","last_unstale":"2024-03-19T01:07:48.769919+0000","last_undegraded":"2024-03-19T01:07:48.769919+0000","last_fullsized":"2024-03-19T01:07:48.769919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:31.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679491+0000","last_change":"2024-03-19T01:07:15.482033+0000","last_active":"2024-03-19T01:07:20.679491+0000","last_peered":"2024-03-19T01:07:20.679491+0000","last_clean":"2024-03-19T01:07:20.679491+0000","last_became_active":"2024-03-19T01:07:15.481671+0000","last_became_peered":"2024-03-19T01:07:15.481671+0000","last_unstale":"2024-03-19T01:07:20.679491+0000","last_undegraded":"2024-03-19T01:07:20.679491+0000","last_fullsized":"2024-03-19T01:07:20.679491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:09.969374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.839041+0000","last_change":"2024-03-19T01:07:19.696413+0000","last_active":"2024-03-19T01:07:20.839041+0000","last_peered":"2024-03-19T01:07:20.839041+0000","last_clean":"2024-03-19T01:07:20.839041+0000","last_became_active":"2024-03-19T01:07:19.695957+0000","last_became_peered":"2024-03-19T01:07:19.695957+0000","last_unstale":"2024-03-19T01:07:20.839041+0000","last_undegraded":"2024-03-19T01:07:20.839041+0000","last_fullsized":"2024-03-19T01:07:20.839041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:54:48.547854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:48.770718+0000","last_change":"2024-03-19T01:07:14.481677+0000","last_active":"2024-03-19T01:07:48.770718+0000","last_peered":"2024-03-19T01:07:48.770718+0000","last_clean":"2024-03-19T01:07:48.770718+0000","last_became_active":"2024-03-19T01:07:14.481138+0000","last_became_peered":"2024-03-19T01:07:14.481138+0000","last_unstale":"2024-03-19T01:07:48.770718+0000","last_undegraded":"2024-03-19T01:07:48.770718+0000","last_fullsized":"2024-03-19T01:07:48.770718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:52:33.105816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679467+0000","last_change":"2024-03-19T01:07:15.498352+0000","last_active":"2024-03-19T01:07:20.679467+0000","last_peered":"2024-03-19T01:07:20.679467+0000","last_clean":"2024-03-19T01:07:20.679467+0000","last_became_active":"2024-03-19T01:07:15.498040+0000","last_became_peered":"2024-03-19T01:07:15.498040+0000","last_unstale":"2024-03-19T01:07:20.679467+0000","last_undegraded":"2024-03-19T01:07:20.679467+0000","last_fullsized":"2024-03-19T01:07:20.679467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:57:37.571753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.247889+0000","last_change":"2024-03-19T01:07:19.673291+0000","last_active":"2024-03-19T01:07:21.247889+0000","last_peered":"2024-03-19T01:07:21.247889+0000","last_clean":"2024-03-19T01:07:21.247889+0000","last_became_active":"2024-03-19T01:07:19.672704+0000","last_became_peered":"2024-03-19T01:07:19.672704+0000","last_unstale":"2024-03-19T01:07:21.247889+0000","last_undegraded":"2024-03-19T01:07:21.247889+0000","last_fullsized":"2024-03-19T01:07:21.247889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:01:42.019812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:49.797497+0000","last_change":"2024-03-19T01:07:14.489556+0000","last_active":"2024-03-19T01:07:49.797497+0000","last_peered":"2024-03-19T01:07:49.797497+0000","last_clean":"2024-03-19T01:07:49.797497+0000","last_became_active":"2024-03-19T01:07:14.489201+0000","last_became_peered":"2024-03-19T01:07:14.489201+0000","last_unstale":"2024-03-19T01:07:49.797497+0000","last_undegraded":"2024-03-19T01:07:49.797497+0000","last_fullsized":"2024-03-19T01:07:49.797497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:28:36.516185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.679685+0000","last_change":"2024-03-19T01:07:15.477727+0000","last_active":"2024-03-19T01:07:20.679685+0000","last_peered":"2024-03-19T01:07:20.679685+0000","last_clean":"2024-03-19T01:07:20.679685+0000","last_became_active":"2024-03-19T01:07:15.477388+0000","last_became_peered":"2024-03-19T01:07:15.477388+0000","last_unstale":"2024-03-19T01:07:20.679685+0000","last_undegraded":"2024-03-19T01:07:20.679685+0000","last_fullsized":"2024-03-19T01:07:20.679685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:30.567326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248140+0000","last_change":"2024-03-19T01:07:15.479008+0000","last_active":"2024-03-19T01:07:21.248140+0000","last_peered":"2024-03-19T01:07:21.248140+0000","last_clean":"2024-03-19T01:07:21.248140+0000","last_became_active":"2024-03-19T01:07:15.478657+0000","last_became_peered":"2024-03-19T01:07:15.478657+0000","last_unstale":"2024-03-19T01:07:21.248140+0000","last_undegraded":"2024-03-19T01:07:21.248140+0000","last_fullsized":"2024-03-19T01:07:21.248140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:42:46.970012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.675962+0000","last_change":"2024-03-19T01:07:15.476033+0000","last_active":"2024-03-19T01:07:20.675962+0000","last_peered":"2024-03-19T01:07:20.675962+0000","last_clean":"2024-03-19T01:07:20.675962+0000","last_became_active":"2024-03-19T01:07:15.475675+0000","last_became_peered":"2024-03-19T01:07:15.475675+0000","last_unstale":"2024-03-19T01:07:20.675962+0000","last_undegraded":"2024-03-19T01:07:20.675962+0000","last_fullsized":"2024-03-19T01:07:20.675962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:01:44.290463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.675520+0000","last_change":"2024-03-19T01:07:15.495292+0000","last_active":"2024-03-19T01:07:20.675520+0000","last_peered":"2024-03-19T01:07:20.675520+0000","last_clean":"2024-03-19T01:07:20.675520+0000","last_became_active":"2024-03-19T01:07:15.494946+0000","last_became_peered":"2024-03-19T01:07:15.494946+0000","last_unstale":"2024-03-19T01:07:20.675520+0000","last_undegraded":"2024-03-19T01:07:20.675520+0000","last_fullsized":"2024-03-19T01:07:20.675520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:39.012221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.838891+0000","last_change":"2024-03-19T01:07:15.499816+0000","last_active":"2024-03-19T01:07:20.838891+0000","last_peered":"2024-03-19T01:07:20.838891+0000","last_clean":"2024-03-19T01:07:20.838891+0000","last_became_active":"2024-03-19T01:07:15.499031+0000","last_became_peered":"2024-03-19T01:07:15.499031+0000","last_unstale":"2024-03-19T01:07:20.838891+0000","last_undegraded":"2024-03-19T01:07:20.838891+0000","last_fullsized":"2024-03-19T01:07:20.838891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:34:25.344092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.749104+0000","last_change":"2024-03-19T01:07:15.501526+0000","last_active":"2024-03-19T01:07:20.749104+0000","last_peered":"2024-03-19T01:07:20.749104+0000","last_clean":"2024-03-19T01:07:20.749104+0000","last_became_active":"2024-03-19T01:07:15.500452+0000","last_became_peered":"2024-03-19T01:07:15.500452+0000","last_unstale":"2024-03-19T01:07:20.749104+0000","last_undegraded":"2024-03-19T01:07:20.749104+0000","last_fullsized":"2024-03-19T01:07:20.749104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:35:33.075780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248995+0000","last_change":"2024-03-19T01:07:15.477649+0000","last_active":"2024-03-19T01:07:21.248995+0000","last_peered":"2024-03-19T01:07:21.248995+0000","last_clean":"2024-03-19T01:07:21.248995+0000","last_became_active":"2024-03-19T01:07:15.477193+0000","last_became_peered":"2024-03-19T01:07:15.477193+0000","last_unstale":"2024-03-19T01:07:21.248995+0000","last_undegraded":"2024-03-19T01:07:21.248995+0000","last_fullsized":"2024-03-19T01:07:21.248995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:36.126642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.676211+0000","last_change":"2024-03-19T01:07:15.481920+0000","last_active":"2024-03-19T01:07:20.676211+0000","last_peered":"2024-03-19T01:07:20.676211+0000","last_clean":"2024-03-19T01:07:20.676211+0000","last_became_active":"2024-03-19T01:07:15.481290+0000","last_became_peered":"2024-03-19T01:07:15.481290+0000","last_unstale":"2024-03-19T01:07:20.676211+0000","last_undegraded":"2024-03-19T01:07:20.676211+0000","last_fullsized":"2024-03-19T01:07:20.676211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:37:58.028647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.249429+0000","last_change":"2024-03-19T01:07:15.502305+0000","last_active":"2024-03-19T01:07:21.249429+0000","last_peered":"2024-03-19T01:07:21.249429+0000","last_clean":"2024-03-19T01:07:21.249429+0000","last_became_active":"2024-03-19T01:07:15.501491+0000","last_became_peered":"2024-03-19T01:07:15.501491+0000","last_unstale":"2024-03-19T01:07:21.249429+0000","last_undegraded":"2024-03-19T01:07:21.249429+0000","last_fullsized":"2024-03-19T01:07:21.249429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:01.999538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248584+0000","last_change":"2024-03-19T01:07:15.499590+0000","last_active":"2024-03-19T01:07:21.248584+0000","last_peered":"2024-03-19T01:07:21.248584+0000","last_clean":"2024-03-19T01:07:21.248584+0000","last_became_active":"2024-03-19T01:07:15.498991+0000","last_became_peered":"2024-03-19T01:07:15.498991+0000","last_unstale":"2024-03-19T01:07:21.248584+0000","last_undegraded":"2024-03-19T01:07:21.248584+0000","last_fullsized":"2024-03-19T01:07:21.248584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:09:43.510480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248456+0000","last_change":"2024-03-19T01:07:15.505022+0000","last_active":"2024-03-19T01:07:21.248456+0000","last_peered":"2024-03-19T01:07:21.248456+0000","last_clean":"2024-03-19T01:07:21.248456+0000","last_became_active":"2024-03-19T01:07:15.503684+0000","last_became_peered":"2024-03-19T01:07:15.503684+0000","last_unstale":"2024-03-19T01:07:21.248456+0000","last_undegraded":"2024-03-19T01:07:21.248456+0000","last_fullsized":"2024-03-19T01:07:21.248456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:56:47.580267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248843+0000","last_change":"2024-03-19T01:07:15.495634+0000","last_active":"2024-03-19T01:07:21.248843+0000","last_peered":"2024-03-19T01:07:21.248843+0000","last_clean":"2024-03-19T01:07:21.248843+0000","last_became_active":"2024-03-19T01:07:15.495206+0000","last_became_peered":"2024-03-19T01:07:15.495206+0000","last_unstale":"2024-03-19T01:07:21.248843+0000","last_undegraded":"2024-03-19T01:07:21.248843+0000","last_fullsized":"2024-03-19T01:07:21.248843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:30:29.965731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:20.748932+0000","last_change":"2024-03-19T01:07:15.501480+0000","last_active":"2024-03-19T01:07:20.748932+0000","last_peered":"2024-03-19T01:07:20.748932+0000","last_clean":"2024-03-19T01:07:20.748932+0000","last_became_active":"2024-03-19T01:07:15.501078+0000","last_became_peered":"2024-03-19T01:07:15.501078+0000","last_unstale":"2024-03-19T01:07:20.748932+0000","last_undegraded":"2024-03-19T01:07:20.748932+0000","last_fullsized":"2024-03-19T01:07:20.748932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:40:18.767874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-19T01:07:21.248556+0000","last_change":"2024-03-19T01:07:15.480358+0000","last_active":"2024-03-19T01:07:21.248556+0000","last_peered":"2024-03-19T01:07:21.248556+0000","last_clean":"2024-03-19T01:07:21.248556+0000","last_became_active":"2024-03-19T01:07:15.479979+0000","last_became_peered":"2024-03-19T01:07:15.479979+0000","last_unstale":"2024-03-19T01:07:21.248556+0000","last_undegraded":"2024-03-19T01:07:21.248556+0000","last_fullsized":"2024-03-19T01:07:21.248556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:12:54.598435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247132,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":29954,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":64,"num_read_kb":70,"num_write":219,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":70344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192,"ondisk_log_size":192,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383700,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645338,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":1092851,"data_compressed":50398,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808863,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":482433,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":478715,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332138,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1079308,"data_compressed":50398,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":477937,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":502108,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":479393,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":483420,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":477645,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":482402,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1074499,"data_compressed":50398,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":25268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T01:07:54.901 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-19T01:07:54.901 INFO:tasks.cephadm:Setting up client nodes... 2024-03-19T01:07:54.901 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-19T01:07:55.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:54 smithi086 ceph-mon[66829]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:55.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:54 smithi086 ceph-mon[66829]: pgmap v334: 129 pgs: 129 active+clean; 606 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.3 KiB/s wr, 19 op/s 2024-03-19T01:07:55.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:54 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/841909650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:55.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:54 smithi069 ceph-mon[31727]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:55.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:54 smithi069 ceph-mon[31727]: pgmap v334: 129 pgs: 129 active+clean; 606 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.3 KiB/s wr, 19 op/s 2024-03-19T01:07:55.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:54 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/841909650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:55.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:54 smithi163 ceph-mon[67428]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:07:55.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:54 smithi163 ceph-mon[67428]: pgmap v334: 129 pgs: 129 active+clean; 606 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.3 KiB/s wr, 19 op/s 2024-03-19T01:07:55.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:54 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/841909650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:07:55.418 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:07:56.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:55 smithi086 ceph-mon[66829]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:07:56.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:55 smithi086 ceph-mon[66829]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:07:56.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:55 smithi069 ceph-mon[31727]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:07:56.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:55 smithi069 ceph-mon[31727]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:07:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:55 smithi163 ceph-mon[67428]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:07:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:55 smithi163 ceph-mon[67428]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:07:57.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:56 smithi086 ceph-mon[66829]: pgmap v335: 129 pgs: 129 active+clean; 606 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-03-19T01:07:57.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:56 smithi069 ceph-mon[31727]: pgmap v335: 129 pgs: 129 active+clean; 606 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-03-19T01:07:57.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:56 smithi163 ceph-mon[67428]: pgmap v335: 129 pgs: 129 active+clean; 606 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-03-19T01:07:57.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.790+0000 7f6e84000700 1 -- 172.21.15.69:0/112939605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e7c0762c0 msgr2=0x7f6e7c074720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:57.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.790+0000 7f6e84000700 1 --2- 172.21.15.69:0/112939605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e7c0762c0 0x7f6e7c074720 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009990 tx=0x7f6e70009ca0 comp rx=0 tx=0).stop 2024-03-19T01:07:57.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.790+0000 7f6e84000700 1 -- 172.21.15.69:0/112939605 shutdown_connections 2024-03-19T01:07:57.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.790+0000 7f6e84000700 1 --2- 172.21.15.69:0/112939605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e7c10a270 0x7f6e7c10c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:57.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.790+0000 7f6e84000700 1 --2- 172.21.15.69:0/112939605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e7c0762c0 0x7f6e7c074720 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:57.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.790+0000 7f6e84000700 1 --2- 172.21.15.69:0/112939605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e7c074cf0 0x7f6e7c075170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:57.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.790+0000 7f6e84000700 1 -- 172.21.15.69:0/112939605 >> 172.21.15.69:0/112939605 conn(0x7f6e7c0fd910 msgr2=0x7f6e7c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:57.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.790+0000 7f6e84000700 1 -- 172.21.15.69:0/112939605 shutdown_connections 2024-03-19T01:07:57.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.790+0000 7f6e84000700 1 -- 172.21.15.69:0/112939605 wait complete. 2024-03-19T01:07:57.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.791+0000 7f6e84000700 1 Processor -- start 2024-03-19T01:07:57.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.791+0000 7f6e84000700 1 -- start start 2024-03-19T01:07:57.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.791+0000 7f6e84000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e7c074cf0 0x7f6e7c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:57.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.791+0000 7f6e84000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e7c0762c0 0x7f6e7c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:57.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e84000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e7c10a270 0x7f6e7c1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:57.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e84000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c110160 con 0x7f6e7c074cf0 2024-03-19T01:07:57.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e84000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c10ffe0 con 0x7f6e7c0762c0 2024-03-19T01:07:57.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e84000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c1102e0 con 0x7f6e7c10a270 2024-03-19T01:07:57.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e80c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e7c0762c0 0x7f6e7c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:57.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e80c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e7c0762c0 0x7f6e7c19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40830/0 (socket says 172.21.15.69:40830) 2024-03-19T01:07:57.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e80c00700 1 -- 172.21.15.69:0/1769863446 learned_addr learned my addr 172.21.15.69:0/1769863446 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:07:57.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e81600700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e7c074cf0 0x7f6e7c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:57.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e82000700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e7c10a270 0x7f6e7c1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:57.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e80c00700 1 -- 172.21.15.69:0/1769863446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e7c10a270 msgr2=0x7f6e7c1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:57.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e80c00700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e7c10a270 0x7f6e7c1a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:57.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e80c00700 1 -- 172.21.15.69:0/1769863446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e7c074cf0 msgr2=0x7f6e7c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:57.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e80c00700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e7c074cf0 0x7f6e7c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:57.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.792+0000 7f6e80c00700 1 -- 172.21.15.69:0/1769863446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7c1a22f0 con 0x7f6e7c0762c0 2024-03-19T01:07:57.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.793+0000 7f6e80c00700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e7c0762c0 0x7f6e7c19abd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800e9a0 tx=0x7f6e6800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:57.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.793+0000 7f6e7a000700 1 -- 172.21.15.69:0/1769863446 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800cbd0 con 0x7f6e7c0762c0 2024-03-19T01:07:57.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.793+0000 7f6e7a000700 1 -- 172.21.15.69:0/1769863446 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e6800cd70 con 0x7f6e7c0762c0 2024-03-19T01:07:57.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.793+0000 7f6e7a000700 1 -- 172.21.15.69:0/1769863446 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e680187a0 con 0x7f6e7c0762c0 2024-03-19T01:07:57.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.793+0000 7f6e84000700 1 -- 172.21.15.69:0/1769863446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7c1a25e0 con 0x7f6e7c0762c0 2024-03-19T01:07:57.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.793+0000 7f6e84000700 1 -- 172.21.15.69:0/1769863446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e7c1a2b70 con 0x7f6e7c0762c0 2024-03-19T01:07:57.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.795+0000 7f6e84000700 1 -- 172.21.15.69:0/1769863446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e7c050820 con 0x7f6e7c0762c0 2024-03-19T01:07:57.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.797+0000 7f6e7a000700 1 -- 172.21.15.69:0/1769863446 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f6e680109a0 con 0x7f6e7c0762c0 2024-03-19T01:07:57.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.799+0000 7f6e7a000700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e640a2a50 0x7f6e640a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:07:57.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.799+0000 7f6e7a000700 1 -- 172.21.15.69:0/1769863446 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f6e680cefe0 con 0x7f6e7c0762c0 2024-03-19T01:07:57.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.800+0000 7f6e81600700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e640a2a50 0x7f6e640a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:07:57.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.800+0000 7f6e81600700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e640a2a50 0x7f6e640a4f10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6e70006010 tx=0x7f6e7001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:07:57.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:57.801+0000 7f6e7a000700 1 -- 172.21.15.69:0/1769863446 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6e6809bce0 con 0x7f6e7c0762c0 2024-03-19T01:07:58.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.153+0000 7f6e84000700 1 -- 172.21.15.69:0/1769863446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f6e7c02dbb0 con 0x7f6e7c0762c0 2024-03-19T01:07:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.158+0000 7f6e7a000700 1 -- 172.21.15.69:0/1769863446 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f6e6809b5e0 con 0x7f6e7c0762c0 2024-03-19T01:07:58.159 INFO:teuthology.orchestra.run.smithi069.stdout:[client.0] 2024-03-19T01:07:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBu5fhlNThDCRAAHEhrzxkjPkU+o46tprtQaA== 2024-03-19T01:07:58.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.162+0000 7f6e84000700 1 -- 172.21.15.69:0/1769863446 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e640a2a50 msgr2=0x7f6e640a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:58.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.162+0000 7f6e84000700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e640a2a50 0x7f6e640a4f10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6e70006010 tx=0x7f6e7001a040 comp rx=0 tx=0).stop 2024-03-19T01:07:58.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.162+0000 7f6e84000700 1 -- 172.21.15.69:0/1769863446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e7c0762c0 msgr2=0x7f6e7c19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:07:58.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.162+0000 7f6e84000700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e7c0762c0 0x7f6e7c19abd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800e9a0 tx=0x7f6e6800ecb0 comp rx=0 tx=0).stop 2024-03-19T01:07:58.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.162+0000 7f6e84000700 1 -- 172.21.15.69:0/1769863446 shutdown_connections 2024-03-19T01:07:58.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.162+0000 7f6e84000700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6e640a2a50 0x7f6e640a4f10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:58.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.162+0000 7f6e84000700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e7c10a270 0x7f6e7c1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:58.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.162+0000 7f6e84000700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e7c074cf0 0x7f6e7c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:58.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.162+0000 7f6e84000700 1 --2- 172.21.15.69:0/1769863446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e7c0762c0 0x7f6e7c19abd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:07:58.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.162+0000 7f6e84000700 1 -- 172.21.15.69:0/1769863446 >> 172.21.15.69:0/1769863446 conn(0x7f6e7c0fd910 msgr2=0x7f6e7c1061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:07:58.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.163+0000 7f6e84000700 1 -- 172.21.15.69:0/1769863446 shutdown_connections 2024-03-19T01:07:58.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:07:58.163+0000 7f6e84000700 1 -- 172.21.15.69:0/1769863446 wait complete. 2024-03-19T01:07:58.959 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:58 smithi069 ceph-mon[31727]: pgmap v336: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-03-19T01:07:58.959 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:58 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1769863446' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T01:07:58.960 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:58 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T01:07:58.960 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:07:58 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-19T01:07:59.003 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T01:07:59.003 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-19T01:07:59.003 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-19T01:07:59.127 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-19T01:07:59.128 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-19T01:07:59.128 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-19T01:07:59.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:58 smithi086 ceph-mon[66829]: pgmap v336: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-03-19T01:07:59.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:58 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1769863446' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T01:07:59.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:58 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T01:07:59.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:07:58 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-19T01:07:59.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:58 smithi163 ceph-mon[67428]: pgmap v336: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-03-19T01:07:59.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:58 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1769863446' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T01:07:59.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:58 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T01:07:59.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:07:58 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-19T01:07:59.582 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:01.030 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:00 smithi069 ceph-mon[31727]: pgmap v337: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2024-03-19T01:08:01.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:00 smithi086 ceph-mon[66829]: pgmap v337: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2024-03-19T01:08:01.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:00 smithi163 ceph-mon[67428]: pgmap v337: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2024-03-19T01:08:01.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.865+0000 7f80e7e00700 1 -- 172.21.15.69:0/2808397953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e810eb80 msgr2=0x7f80e8110f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:01.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.865+0000 7f80e7e00700 1 --2- 172.21.15.69:0/2808397953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e810eb80 0x7f80e8110f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b0a0 tx=0x7f80e000b3b0 comp rx=0 tx=0).stop 2024-03-19T01:08:01.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.865+0000 7f80e7e00700 1 -- 172.21.15.69:0/2808397953 shutdown_connections 2024-03-19T01:08:01.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.865+0000 7f80e7e00700 1 --2- 172.21.15.69:0/2808397953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e810eb80 0x7f80e8110f70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:01.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.865+0000 7f80e7e00700 1 --2- 172.21.15.69:0/2808397953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e8100fd0 0x7f80e81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:01.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.865+0000 7f80e7e00700 1 --2- 172.21.15.69:0/2808397953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e8101980 0x7f80e810e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:01.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.865+0000 7f80e7e00700 1 -- 172.21.15.69:0/2808397953 >> 172.21.15.69:0/2808397953 conn(0x7f80e80fcbf0 msgr2=0x7f80e80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:01.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.865+0000 7f80e7e00700 1 -- 172.21.15.69:0/2808397953 shutdown_connections 2024-03-19T01:08:01.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.866+0000 7f80e7e00700 1 -- 172.21.15.69:0/2808397953 wait complete. 2024-03-19T01:08:01.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.867+0000 7f80e7e00700 1 Processor -- start 2024-03-19T01:08:01.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.867+0000 7f80e7e00700 1 -- start start 2024-03-19T01:08:01.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.867+0000 7f80e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8100fd0 0x7f80e81a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:01.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.867+0000 7f80e7e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e8101980 0x7f80e81a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:01.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.867+0000 7f80e7e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e810eb80 0x7f80e819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:01.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.867+0000 7f80e7e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80e8114790 con 0x7f80e8101980 2024-03-19T01:08:01.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.867+0000 7f80e7e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80e8114610 con 0x7f80e8100fd0 2024-03-19T01:08:01.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.867+0000 7f80e7e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f80e8114910 con 0x7f80e810eb80 2024-03-19T01:08:01.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.867+0000 7f80e6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8100fd0 0x7f80e81a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:01.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.868+0000 7f80e7400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e810eb80 0x7f80e819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:01.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.868+0000 7f80e6000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e8101980 0x7f80e81a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:01.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.868+0000 7f80e6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8100fd0 0x7f80e81a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40852/0 (socket says 172.21.15.69:40852) 2024-03-19T01:08:01.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.868+0000 7f80e6a00700 1 -- 172.21.15.69:0/1754017915 learned_addr learned my addr 172.21.15.69:0/1754017915 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:01.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.868+0000 7f80e6a00700 1 -- 172.21.15.69:0/1754017915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e810eb80 msgr2=0x7f80e819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:01.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.868+0000 7f80e6a00700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e810eb80 0x7f80e819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:01.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.868+0000 7f80e6a00700 1 -- 172.21.15.69:0/1754017915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e8101980 msgr2=0x7f80e81a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:01.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.868+0000 7f80e6a00700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e8101980 0x7f80e81a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:01.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.868+0000 7f80e6a00700 1 -- 172.21.15.69:0/1754017915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e819b150 con 0x7f80e8100fd0 2024-03-19T01:08:01.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.868+0000 7f80e6a00700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8100fd0 0x7f80e81a0830 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80d800e9c0 tx=0x7f80d800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:01.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.869+0000 7f80eca00700 1 -- 172.21.15.69:0/1754017915 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d800cbd0 con 0x7f80e8100fd0 2024-03-19T01:08:01.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.869+0000 7f80eca00700 1 -- 172.21.15.69:0/1754017915 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d800ee60 con 0x7f80e8100fd0 2024-03-19T01:08:01.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.869+0000 7f80eca00700 1 -- 172.21.15.69:0/1754017915 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8005630 con 0x7f80e8100fd0 2024-03-19T01:08:01.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.869+0000 7f80e7e00700 1 -- 172.21.15.69:0/1754017915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e819b440 con 0x7f80e8100fd0 2024-03-19T01:08:01.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.869+0000 7f80e7e00700 1 -- 172.21.15.69:0/1754017915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e81a7700 con 0x7f80e8100fd0 2024-03-19T01:08:01.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.871+0000 7f80e7e00700 1 -- 172.21.15.69:0/1754017915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e8068fd0 con 0x7f80e8100fd0 2024-03-19T01:08:01.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.872+0000 7f80eca00700 1 -- 172.21.15.69:0/1754017915 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f80d8004750 con 0x7f80e8100fd0 2024-03-19T01:08:01.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.874+0000 7f80eca00700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80cc0a29a0 0x7f80cc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:01.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.874+0000 7f80eca00700 1 -- 172.21.15.69:0/1754017915 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f80d80cf080 con 0x7f80e8100fd0 2024-03-19T01:08:01.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.874+0000 7f80e6000700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80cc0a29a0 0x7f80cc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:01.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.875+0000 7f80e6000700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80cc0a29a0 0x7f80cc0a4e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80e81028c0 tx=0x7f80d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:01.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:01.877+0000 7f80eca00700 1 -- 172.21.15.69:0/1754017915 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f80d809bd80 con 0x7f80e8100fd0 2024-03-19T01:08:02.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.284+0000 7f80e7e00700 1 -- 172.21.15.69:0/1754017915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f80e802dbb0 con 0x7f80e8100fd0 2024-03-19T01:08:02.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.289+0000 7f80eca00700 1 -- 172.21.15.69:0/1754017915 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+262661 (secure 0 0 0) 0x7f80d809b680 con 0x7f80e8100fd0 2024-03-19T01:08:02.291 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:08:02.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.296+0000 7f80e7e00700 1 -- 172.21.15.69:0/1754017915 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80cc0a29a0 msgr2=0x7f80cc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:02.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.296+0000 7f80e7e00700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80cc0a29a0 0x7f80cc0a4e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80e81028c0 tx=0x7f80d4009260 comp rx=0 tx=0).stop 2024-03-19T01:08:02.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.296+0000 7f80e7e00700 1 -- 172.21.15.69:0/1754017915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8100fd0 msgr2=0x7f80e81a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:02.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.296+0000 7f80e7e00700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8100fd0 0x7f80e81a0830 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80d800e9c0 tx=0x7f80d800ecd0 comp rx=0 tx=0).stop 2024-03-19T01:08:02.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.296+0000 7f80e7e00700 1 -- 172.21.15.69:0/1754017915 shutdown_connections 2024-03-19T01:08:02.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.296+0000 7f80e7e00700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80cc0a29a0 0x7f80cc0a4e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:02.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.296+0000 7f80e7e00700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e810eb80 0x7f80e819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:02.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.296+0000 7f80e7e00700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8100fd0 0x7f80e81a0830 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:02.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.296+0000 7f80e7e00700 1 --2- 172.21.15.69:0/1754017915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80e8101980 0x7f80e81a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:02.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.296+0000 7f80e7e00700 1 -- 172.21.15.69:0/1754017915 >> 172.21.15.69:0/1754017915 conn(0x7f80e80fcbf0 msgr2=0x7f80e80fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:02.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.297+0000 7f80e7e00700 1 -- 172.21.15.69:0/1754017915 shutdown_connections 2024-03-19T01:08:02.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:02.297+0000 7f80e7e00700 1 -- 172.21.15.69:0/1754017915 wait complete. 2024-03-19T01:08:03.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:02 smithi069 ceph-mon[31727]: pgmap v338: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 22 op/s 2024-03-19T01:08:03.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:02 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1754017915' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-19T01:08:03.080 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3562156769},{"type":"v1","addr":"172.21.15.69:6801","nonce":3562156769}]},"active_addr":"172.21.15.69:6801/3562156769","active_change":"2024-03-19T00:59:23.503460+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.69:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3960674803}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2535372856}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2266394714}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2291738694}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":4077457107}]}]} 2024-03-19T01:08:03.086 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-19T01:08:03.086 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-19T01:08:03.086 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-19T01:08:03.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:02 smithi086 ceph-mon[66829]: pgmap v338: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 22 op/s 2024-03-19T01:08:03.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:02 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1754017915' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-19T01:08:03.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:02 smithi163 ceph-mon[67428]: pgmap v338: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 22 op/s 2024-03-19T01:08:03.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:02 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1754017915' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-19T01:08:03.586 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:05.255 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:04 smithi069 ceph-mon[31727]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 24 op/s 2024-03-19T01:08:05.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:04 smithi086 ceph-mon[66829]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 24 op/s 2024-03-19T01:08:05.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:04 smithi163 ceph-mon[67428]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 24 op/s 2024-03-19T01:08:05.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.843+0000 7f94b5400700 1 -- 172.21.15.69:0/2949526172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94b010edf0 msgr2=0x7f94b01111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:05.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.843+0000 7f94b5400700 1 --2- 172.21.15.69:0/2949526172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94b010edf0 0x7f94b01111e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f94a800b0a0 tx=0x7f94a800b3b0 comp rx=0 tx=0).stop 2024-03-19T01:08:05.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.843+0000 7f94b5400700 1 -- 172.21.15.69:0/2949526172 shutdown_connections 2024-03-19T01:08:05.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.843+0000 7f94b5400700 1 --2- 172.21.15.69:0/2949526172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94b010edf0 0x7f94b01111e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:05.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.843+0000 7f94b5400700 1 --2- 172.21.15.69:0/2949526172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b006a9f0 0x7f94b006add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:05.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.843+0000 7f94b5400700 1 --2- 172.21.15.69:0/2949526172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94b006b310 0x7f94b010e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:05.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.843+0000 7f94b5400700 1 -- 172.21.15.69:0/2949526172 >> 172.21.15.69:0/2949526172 conn(0x7f94b0077260 msgr2=0x7f94b0077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:05.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.843+0000 7f94b5400700 1 -- 172.21.15.69:0/2949526172 shutdown_connections 2024-03-19T01:08:05.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.843+0000 7f94b5400700 1 -- 172.21.15.69:0/2949526172 wait complete. 2024-03-19T01:08:05.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.845+0000 7f94b5400700 1 Processor -- start 2024-03-19T01:08:05.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.845+0000 7f94b5400700 1 -- start start 2024-03-19T01:08:05.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94b5400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b006a9f0 0x7f94b01a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:05.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94b5400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94b006b310 0x7f94b01a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:05.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94b5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94b010edf0 0x7f94b019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:05.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94b5400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0114860 con 0x7f94b006a9f0 2024-03-19T01:08:05.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94b5400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01146e0 con 0x7f94b010edf0 2024-03-19T01:08:05.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94b5400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01149e0 con 0x7f94b006b310 2024-03-19T01:08:05.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94ae800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b006a9f0 0x7f94b01a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:05.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94af200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94b010edf0 0x7f94b019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:05.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94a7e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94b006b310 0x7f94b01a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94af200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94b010edf0 0x7f94b019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36370/0 (socket says 172.21.15.69:36370) 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94a7e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94b006b310 0x7f94b01a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:40044/0 (socket says 172.21.15.69:40044) 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.846+0000 7f94af200700 1 -- 172.21.15.69:0/392266153 learned_addr learned my addr 172.21.15.69:0/392266153 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.847+0000 7f94af200700 1 -- 172.21.15.69:0/392266153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94b006b310 msgr2=0x7f94b01a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.847+0000 7f94af200700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94b006b310 0x7f94b01a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.847+0000 7f94af200700 1 -- 172.21.15.69:0/392266153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b006a9f0 msgr2=0x7f94b01a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.847+0000 7f94af200700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b006a9f0 0x7f94b01a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.847+0000 7f94af200700 1 -- 172.21.15.69:0/392266153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b019b440 con 0x7f94b010edf0 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.847+0000 7f94af200700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94b010edf0 0x7f94b019ab60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94a8006070 tx=0x7f94a8004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.847+0000 7f94a7400700 1 -- 172.21.15.69:0/392266153 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8021070 con 0x7f94b010edf0 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.847+0000 7f94a7400700 1 -- 172.21.15.69:0/392266153 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a8003c00 con 0x7f94b010edf0 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.847+0000 7f94a7400700 1 -- 172.21.15.69:0/392266153 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a80046c0 con 0x7f94b010edf0 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.847+0000 7f94b5400700 1 -- 172.21.15.69:0/392266153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b019b6d0 con 0x7f94b010edf0 2024-03-19T01:08:05.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.848+0000 7f94b5400700 1 -- 172.21.15.69:0/392266153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b01a77f0 con 0x7f94b010edf0 2024-03-19T01:08:05.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.848+0000 7f94b5400700 1 -- 172.21.15.69:0/392266153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b0101820 con 0x7f94b010edf0 2024-03-19T01:08:05.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.851+0000 7f94a7400700 1 -- 172.21.15.69:0/392266153 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f94a8003db0 con 0x7f94b010edf0 2024-03-19T01:08:05.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.853+0000 7f94a7400700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f94940a29a0 0x7f94940a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:05.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.853+0000 7f94a7400700 1 -- 172.21.15.69:0/392266153 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f94a80cffe0 con 0x7f94b010edf0 2024-03-19T01:08:05.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.853+0000 7f94ae800700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f94940a29a0 0x7f94940a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:05.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.854+0000 7f94ae800700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f94940a29a0 0x7f94940a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009c60 tx=0x7f94a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:05.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:05.859+0000 7f94a7400700 1 -- 172.21.15.69:0/392266153 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f94a809cd10 con 0x7f94b010edf0 2024-03-19T01:08:06.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.175+0000 7f94b5400700 1 -- 172.21.15.69:0/392266153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94b006a220 con 0x7f94b010edf0 2024-03-19T01:08:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.178+0000 7f94a7400700 1 -- 172.21.15.69:0/392266153 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24069 (secure 0 0 0) 0x7f94a809c610 con 0x7f94b010edf0 2024-03-19T01:08:06.180 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:08:06.180 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":89,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:07:20.123357+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:08:06.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.181+0000 7f94b5400700 1 -- 172.21.15.69:0/392266153 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f94940a29a0 msgr2=0x7f94940a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:06.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.181+0000 7f94b5400700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f94940a29a0 0x7f94940a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009c60 tx=0x7f94a0009260 comp rx=0 tx=0).stop 2024-03-19T01:08:06.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.181+0000 7f94b5400700 1 -- 172.21.15.69:0/392266153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94b010edf0 msgr2=0x7f94b019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:06.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.181+0000 7f94b5400700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94b010edf0 0x7f94b019ab60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94a8006070 tx=0x7f94a8004c60 comp rx=0 tx=0).stop 2024-03-19T01:08:06.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.182+0000 7f94b5400700 1 -- 172.21.15.69:0/392266153 shutdown_connections 2024-03-19T01:08:06.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.182+0000 7f94b5400700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f94940a29a0 0x7f94940a4e60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:06.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.182+0000 7f94b5400700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94b010edf0 0x7f94b019ab60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:06.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.182+0000 7f94b5400700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b006a9f0 0x7f94b01a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:06.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.182+0000 7f94b5400700 1 --2- 172.21.15.69:0/392266153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94b006b310 0x7f94b01a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:06.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.182+0000 7f94b5400700 1 -- 172.21.15.69:0/392266153 >> 172.21.15.69:0/392266153 conn(0x7f94b0077260 msgr2=0x7f94b0110240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:06.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.182+0000 7f94b5400700 1 -- 172.21.15.69:0/392266153 shutdown_connections 2024-03-19T01:08:06.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:06.182+0000 7f94b5400700 1 -- 172.21.15.69:0/392266153 wait complete. 2024-03-19T01:08:07.050 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-19T01:08:07.050 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-19T01:08:07.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:06 smithi069 ceph-mon[31727]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-03-19T01:08:07.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:06 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/392266153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:08:07.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:06 smithi086 ceph-mon[66829]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-03-19T01:08:07.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:06 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/392266153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:08:07.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:06 smithi163 ceph-mon[67428]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-03-19T01:08:07.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:06 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/392266153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:08:07.511 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:09.249 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:08 smithi069 ceph-mon[31727]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-03-19T01:08:09.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:08 smithi086 ceph-mon[66829]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-03-19T01:08:09.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:08 smithi163 ceph-mon[67428]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-03-19T01:08:09.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.793+0000 7fd81cc00700 1 -- 172.21.15.69:0/3767514636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd818074630 msgr2=0x7fd818074a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:09.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.793+0000 7fd81cc00700 1 --2- 172.21.15.69:0/3767514636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd818074630 0x7fd818074a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd808009990 tx=0x7fd808009ca0 comp rx=0 tx=0).stop 2024-03-19T01:08:09.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.793+0000 7fd81cc00700 1 -- 172.21.15.69:0/3767514636 shutdown_connections 2024-03-19T01:08:09.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.793+0000 7fd81cc00700 1 --2- 172.21.15.69:0/3767514636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81810ebf0 0x7fd818110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:09.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.793+0000 7fd81cc00700 1 --2- 172.21.15.69:0/3767514636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd818074630 0x7fd818074a10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:09.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.793+0000 7fd81cc00700 1 --2- 172.21.15.69:0/3767514636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd818074f50 0x7fd81810e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:09.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.793+0000 7fd81cc00700 1 -- 172.21.15.69:0/3767514636 >> 172.21.15.69:0/3767514636 conn(0x7fd8180fdde0 msgr2=0x7fd818100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:09.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.793+0000 7fd81cc00700 1 -- 172.21.15.69:0/3767514636 shutdown_connections 2024-03-19T01:08:09.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.793+0000 7fd81cc00700 1 -- 172.21.15.69:0/3767514636 wait complete. 2024-03-19T01:08:09.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.794+0000 7fd81cc00700 1 Processor -- start 2024-03-19T01:08:09.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.794+0000 7fd81cc00700 1 -- start start 2024-03-19T01:08:09.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.795+0000 7fd81cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd818074630 0x7fd81819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:09.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.795+0000 7fd81cc00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd818074f50 0x7fd81819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:09.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.795+0000 7fd81cc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd81810ebf0 0x7fd81819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:09.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.795+0000 7fd81cc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd818114870 con 0x7fd81810ebf0 2024-03-19T01:08:09.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.795+0000 7fd81cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181146f0 con 0x7fd818074630 2024-03-19T01:08:09.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.795+0000 7fd81cc00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181149f0 con 0x7fd818074f50 2024-03-19T01:08:09.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.795+0000 7fd815400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd818074f50 0x7fd81819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:09.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.795+0000 7fd815e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd818074630 0x7fd81819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:09.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.795+0000 7fd815400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd818074f50 0x7fd81819b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:40064/0 (socket says 172.21.15.69:40064) 2024-03-19T01:08:09.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.795+0000 7fd815400700 1 -- 172.21.15.69:0/686331867 learned_addr learned my addr 172.21.15.69:0/686331867 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:09.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.795+0000 7fd816800700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd81810ebf0 0x7fd81819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:09.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.796+0000 7fd815400700 1 -- 172.21.15.69:0/686331867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd818074630 msgr2=0x7fd81819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:09.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.796+0000 7fd815400700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd818074630 0x7fd81819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:09.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.796+0000 7fd815400700 1 -- 172.21.15.69:0/686331867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd81810ebf0 msgr2=0x7fd81819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:09.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.796+0000 7fd815400700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd81810ebf0 0x7fd81819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:09.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.796+0000 7fd815400700 1 -- 172.21.15.69:0/686331867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81819fd20 con 0x7fd818074f50 2024-03-19T01:08:09.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.796+0000 7fd815400700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd818074f50 0x7fd81819b2a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd80400e9a0 tx=0x7fd80400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:09.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.796+0000 7fd80ea00700 1 -- 172.21.15.69:0/686331867 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80400cbd0 con 0x7fd818074f50 2024-03-19T01:08:09.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.796+0000 7fd80ea00700 1 -- 172.21.15.69:0/686331867 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd80400cd70 con 0x7fd818074f50 2024-03-19T01:08:09.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.796+0000 7fd81cc00700 1 -- 172.21.15.69:0/686331867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8181a0010 con 0x7fd818074f50 2024-03-19T01:08:09.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.796+0000 7fd80ea00700 1 -- 172.21.15.69:0/686331867 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804018750 con 0x7fd818074f50 2024-03-19T01:08:09.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.796+0000 7fd81cc00700 1 -- 172.21.15.69:0/686331867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd818102240 con 0x7fd818074f50 2024-03-19T01:08:09.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.798+0000 7fd81cc00700 1 -- 172.21.15.69:0/686331867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd818068fd0 con 0x7fd818074f50 2024-03-19T01:08:09.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.799+0000 7fd80ea00700 1 -- 172.21.15.69:0/686331867 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fd804010430 con 0x7fd818074f50 2024-03-19T01:08:09.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.801+0000 7fd80ea00700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7fc0a29a0 0x7fd7fc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:09.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.801+0000 7fd80ea00700 1 -- 172.21.15.69:0/686331867 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd8040cf0d0 con 0x7fd818074f50 2024-03-19T01:08:09.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.801+0000 7fd815e00700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7fc0a29a0 0x7fd7fc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:09.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.802+0000 7fd815e00700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7fc0a29a0 0x7fd7fc0a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd80800b2e0 tx=0x7fd808005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:09.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:09.805+0000 7fd80ea00700 1 -- 172.21.15.69:0/686331867 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd80409bd70 con 0x7fd818074f50 2024-03-19T01:08:10.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.047+0000 7fd81cc00700 1 -- 172.21.15.69:0/686331867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd81802dbb0 con 0x7fd818074f50 2024-03-19T01:08:10.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.049+0000 7fd80ea00700 1 -- 172.21.15.69:0/686331867 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24069 (secure 0 0 0) 0x7fd80409b670 con 0x7fd818074f50 2024-03-19T01:08:10.053 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:08:10.053 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":89,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:07:20.123357+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:08:10.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.052+0000 7fd81cc00700 1 -- 172.21.15.69:0/686331867 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7fc0a29a0 msgr2=0x7fd7fc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:10.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.053+0000 7fd81cc00700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7fc0a29a0 0x7fd7fc0a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd80800b2e0 tx=0x7fd808005fd0 comp rx=0 tx=0).stop 2024-03-19T01:08:10.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.053+0000 7fd81cc00700 1 -- 172.21.15.69:0/686331867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd818074f50 msgr2=0x7fd81819b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:10.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.053+0000 7fd81cc00700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd818074f50 0x7fd81819b2a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd80400e9a0 tx=0x7fd80400ecb0 comp rx=0 tx=0).stop 2024-03-19T01:08:10.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.053+0000 7fd81cc00700 1 -- 172.21.15.69:0/686331867 shutdown_connections 2024-03-19T01:08:10.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.053+0000 7fd81cc00700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7fc0a29a0 0x7fd7fc0a4e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:10.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.053+0000 7fd81cc00700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd81810ebf0 0x7fd81819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:10.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.053+0000 7fd81cc00700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd818074630 0x7fd81819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:10.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.053+0000 7fd81cc00700 1 --2- 172.21.15.69:0/686331867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd818074f50 0x7fd81819b2a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:10.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.053+0000 7fd81cc00700 1 -- 172.21.15.69:0/686331867 >> 172.21.15.69:0/686331867 conn(0x7fd8180fdde0 msgr2=0x7fd8180feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:10.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.053+0000 7fd81cc00700 1 -- 172.21.15.69:0/686331867 shutdown_connections 2024-03-19T01:08:10.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:10.053+0000 7fd81cc00700 1 -- 172.21.15.69:0/686331867 wait complete. 2024-03-19T01:08:10.871 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-19T01:08:10.872 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-19T01:08:10.872 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-19T01:08:10.872 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-19T01:08:10.873 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-19T01:08:10.873 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-19T01:08:10.874 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-19T01:08:10.874 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-19T01:08:10.874 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-19T01:08:10.875 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-19T01:08:10.875 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-19T01:08:10.876 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-19T01:08:11.069 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:10 smithi069 ceph-mon[31727]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-19T01:08:11.069 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:10 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/686331867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:08:11.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:10 smithi086 ceph-mon[66829]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-19T01:08:11.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:10 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/686331867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:08:11.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:10 smithi163 ceph-mon[67428]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-19T01:08:11.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:10 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/686331867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:08:12.151 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:12.171 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:12.171 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:12.183 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:12.189 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:13.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:12 smithi069 ceph-mon[31727]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-19T01:08:13.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:12 smithi086 ceph-mon[66829]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-19T01:08:13.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:12 smithi163 ceph-mon[67428]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-19T01:08:15.175 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:14 smithi069 ceph-mon[31727]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-19T01:08:15.175 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:14 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:08:15.188 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:15.190 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:15.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:14 smithi086 ceph-mon[66829]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-19T01:08:15.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:14 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:08:15.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:14 smithi163 ceph-mon[67428]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-19T01:08:15.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:14 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:08:15.528 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:15.529 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:15.529 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:15.531 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:17.004 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:16 smithi069 ceph-mon[31727]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-19T01:08:17.004 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:16 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:17.004 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:16 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:17.004 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:16 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:17.004 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:16 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:17.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:16 smithi086 ceph-mon[66829]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-19T01:08:17.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:16 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:17.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:16 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:17.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:16 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:17.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:16 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:17.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:16 smithi163 ceph-mon[67428]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-19T01:08:17.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:16 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:17.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:16 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:17.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:16 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:17.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:16 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:08:18.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:17 smithi086 ceph-mon[66829]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-19T01:08:18.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:17 smithi069 ceph-mon[31727]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-19T01:08:18.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:17 smithi163 ceph-mon[67428]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-19T01:08:20.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.725+0000 7f2877200700 1 -- 172.21.15.69:0/4144079838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287006a960 msgr2=0x7f287010f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:20.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.725+0000 7f2877200700 1 --2- 172.21.15.69:0/4144079838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287006a960 0x7f287010f1a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f286800b0a0 tx=0x7f286800b3b0 comp rx=0 tx=0).stop 2024-03-19T01:08:20.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.725+0000 7f2877200700 1 -- 172.21.15.69:0/4144079838 shutdown_connections 2024-03-19T01:08:20.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.725+0000 7f2877200700 1 --2- 172.21.15.69:0/4144079838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f287006a960 0x7f287010f1a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:20.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.725+0000 7f2877200700 1 --2- 172.21.15.69:0/4144079838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2870108360 0x7f2870108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:20.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.725+0000 7f2877200700 1 --2- 172.21.15.69:0/4144079838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2870069fc0 0x7f287006a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:20.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.725+0000 7f2877200700 1 -- 172.21.15.69:0/4144079838 >> 172.21.15.69:0/4144079838 conn(0x7f2870078140 msgr2=0x7f2870078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:20.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.725+0000 7f2877200700 1 -- 172.21.15.69:0/4144079838 shutdown_connections 2024-03-19T01:08:20.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.726+0000 7f2877200700 1 -- 172.21.15.69:0/4144079838 wait complete. 2024-03-19T01:08:20.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f2877200700 1 Processor -- start 2024-03-19T01:08:20.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f2877200700 1 -- start start 2024-03-19T01:08:20.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f2877200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2870069fc0 0x7f287019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:20.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f2877200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f287006a960 0x7f287019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:20.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f2877200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2870108360 0x7f2870198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:20.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f2877200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2870112600 con 0x7f287006a960 2024-03-19T01:08:20.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f2877200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2870112480 con 0x7f2870108360 2024-03-19T01:08:20.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f2877200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2870112780 con 0x7f2870069fc0 2024-03-19T01:08:20.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f286fe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2870069fc0 0x7f287019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:20.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f286f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f287006a960 0x7f287019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f2875200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2870108360 0x7f2870198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f286fe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2870069fc0 0x7f287019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53606/0 (socket says 172.21.15.69:53606) 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.727+0000 7f286fe00700 1 -- 172.21.15.69:0/1894665897 learned_addr learned my addr 172.21.15.69:0/1894665897 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.728+0000 7f286fe00700 1 -- 172.21.15.69:0/1894665897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2870108360 msgr2=0x7f2870198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.728+0000 7f286fe00700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2870108360 0x7f2870198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.728+0000 7f286fe00700 1 -- 172.21.15.69:0/1894665897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f287006a960 msgr2=0x7f287019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.728+0000 7f286fe00700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f287006a960 0x7f287019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.728+0000 7f286fe00700 1 -- 172.21.15.69:0/1894665897 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28701990c0 con 0x7f2870069fc0 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.728+0000 7f2875200700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2870108360 0x7f2870198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.728+0000 7f286fe00700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2870069fc0 0x7f287019e750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2860012db0 tx=0x7f286000c5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.729+0000 7f286cc00700 1 -- 172.21.15.69:0/1894665897 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286001f070 con 0x7f2870069fc0 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.729+0000 7f286cc00700 1 -- 172.21.15.69:0/1894665897 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286001a910 con 0x7f2870069fc0 2024-03-19T01:08:20.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.729+0000 7f2877200700 1 -- 172.21.15.69:0/1894665897 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28701993b0 con 0x7f2870069fc0 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.729+0000 7f286cc00700 1 -- 172.21.15.69:0/1894665897 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28600188e0 con 0x7f2870069fc0 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.729+0000 7f2877200700 1 -- 172.21.15.69:0/1894665897 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2870101210 con 0x7f2870069fc0 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.730+0000 7f2877200700 1 -- 172.21.15.69:0/1894665897 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28700629e0 con 0x7f2870069fc0 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.733+0000 7f286cc00700 1 -- 172.21.15.69:0/1894665897 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f2860022070 con 0x7f2870069fc0 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.735+0000 7f286cc00700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f28540a29a0 0x7f28540a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.735+0000 7f286cc00700 1 -- 172.21.15.69:0/1894665897 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f28600d2c00 con 0x7f2870069fc0 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.735+0000 7f286f400700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f28540a29a0 0x7f28540a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.736+0000 7f286cc00700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109] conn(0x7f28540ad660 0x7f28540afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.736+0000 7f286cc00700 1 -- 172.21.15.69:0/1894665897 --> [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2860004500 con 0x7f28540ad660 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.736+0000 7f286cc00700 1 -- 172.21.15.69:0/1894665897 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f28600d2fb0 con 0x7f2870069fc0 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.736+0000 7f2875200700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109] conn(0x7f28540ad660 0x7f28540afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.736+0000 7f286f400700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f28540a29a0 0x7f28540a4e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2870078720 tx=0x7f285c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:20.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.736+0000 7f2875200700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109] conn(0x7f28540ad660 0x7f28540afaa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:20.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.737+0000 7f286cc00700 1 -- 172.21.15.69:0/1894665897 <== osd.10 v2:172.21.15.86:6824/4217204109 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2860004500 con 0x7f28540ad660 2024-03-19T01:08:20.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.757+0000 7f2877200700 1 -- 172.21.15.69:0/1894665897 --> [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2870062de0 con 0x7f28540ad660 2024-03-19T01:08:20.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.757+0000 7f286cc00700 1 -- 172.21.15.69:0/1894665897 <== osd.10 v2:172.21.15.86:6824/4217204109 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2870062de0 con 0x7f28540ad660 2024-03-19T01:08:20.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.758+0000 7f2877200700 1 -- 172.21.15.69:0/1894665897 >> [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109] conn(0x7f28540ad660 msgr2=0x7f28540afaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:20.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.758+0000 7f2877200700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109] conn(0x7f28540ad660 0x7f28540afaa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:20.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.758+0000 7f2877200700 1 -- 172.21.15.69:0/1894665897 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f28540a29a0 msgr2=0x7f28540a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:20.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.758+0000 7f2877200700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f28540a29a0 0x7f28540a4e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2870078720 tx=0x7f285c009260 comp rx=0 tx=0).stop 2024-03-19T01:08:20.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.758+0000 7f2877200700 1 -- 172.21.15.69:0/1894665897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2870069fc0 msgr2=0x7f287019e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:20.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.758+0000 7f2877200700 1 --2- 172.21.15.69:0/1894665897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2870069fc0 0x7f287019e750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2860012db0 tx=0x7f286000c5b0 comp rx=0 tx=0).stop 2024-03-19T01:08:20.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.758+0000 7f2875200700 1 -- 172.21.15.69:0/1894665897 reap_dead start 2024-03-19T01:08:20.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.758+0000 7f2877200700 1 -- 172.21.15.69:0/1894665897 shutdown_connections 2024-03-19T01:08:20.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.758+0000 7f2877200700 1 -- 172.21.15.69:0/1894665897 >> 172.21.15.69:0/1894665897 conn(0x7f2870078140 msgr2=0x7f28700ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:20.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.758+0000 7f2877200700 1 -- 172.21.15.69:0/1894665897 shutdown_connections 2024-03-19T01:08:20.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:19.758+0000 7f2877200700 1 -- 172.21.15.69:0/1894665897 wait complete. 2024-03-19T01:08:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:20 smithi069 ceph-mon[31727]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:08:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:20 smithi163 ceph-mon[67428]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:08:21.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:20 smithi086 ceph-mon[66829]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:08:23.022 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:22 smithi069 ceph-mon[31727]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:08:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:22 smithi163 ceph-mon[67428]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:08:23.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:22 smithi086 ceph-mon[66829]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:08:23.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.379+0000 7fab06200700 1 -- 172.21.15.69:0/1200808931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab00102450 msgr2=0x7fab00102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:23.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.379+0000 7fab06200700 1 --2- 172.21.15.69:0/1200808931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab00102450 0x7fab00102830 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009990 tx=0x7faaf4009ca0 comp rx=0 tx=0).stop 2024-03-19T01:08:23.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.379+0000 7fab06200700 1 -- 172.21.15.69:0/1200808931 shutdown_connections 2024-03-19T01:08:23.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.379+0000 7fab06200700 1 --2- 172.21.15.69:0/1200808931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab00107530 0x7fab0010f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:23.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.379+0000 7fab06200700 1 --2- 172.21.15.69:0/1200808931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab00102450 0x7fab00102830 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:23.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.379+0000 7fab06200700 1 --2- 172.21.15.69:0/1200808931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab00102e00 0x7fab00106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:23.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.379+0000 7fab06200700 1 -- 172.21.15.69:0/1200808931 >> 172.21.15.69:0/1200808931 conn(0x7fab000fdce0 msgr2=0x7fab00100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:23.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.379+0000 7fab06200700 1 -- 172.21.15.69:0/1200808931 shutdown_connections 2024-03-19T01:08:23.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.379+0000 7fab06200700 1 -- 172.21.15.69:0/1200808931 wait complete. 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.380+0000 7fab06200700 1 Processor -- start 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7fab06200700 1 -- start start 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7fab06200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab00102450 0x7fab0019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7fab06200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab00102e00 0x7fab0019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7fab06200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab00107530 0x7fab00198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7fab06200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab00112560 con 0x7fab00102450 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7fab06200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab001123e0 con 0x7fab00102e00 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7fab06200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fab001126e0 con 0x7fab00107530 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7faaffe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab00102e00 0x7fab0019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7faaffe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab00102e00 0x7fab0019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60822/0 (socket says 172.21.15.69:60822) 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7faaffe00700 1 -- 172.21.15.69:0/1962136847 learned_addr learned my addr 172.21.15.69:0/1962136847 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7fab04e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab00102450 0x7fab0019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:23.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.381+0000 7fab05800700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab00107530 0x7fab00198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.382+0000 7faaffe00700 1 -- 172.21.15.69:0/1962136847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab00107530 msgr2=0x7fab00198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.382+0000 7faaffe00700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab00107530 0x7fab00198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.382+0000 7faaffe00700 1 -- 172.21.15.69:0/1962136847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab00102450 msgr2=0x7fab0019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.382+0000 7faaffe00700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab00102450 0x7fab0019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.382+0000 7faaffe00700 1 -- 172.21.15.69:0/1962136847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab00199040 con 0x7fab00102e00 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.382+0000 7faaffe00700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab00102e00 0x7fab0019ebd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faaec00e970 tx=0x7faaec00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.382+0000 7faafd600700 1 -- 172.21.15.69:0/1962136847 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec004a30 con 0x7fab00102e00 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.382+0000 7faafd600700 1 -- 172.21.15.69:0/1962136847 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaec00ccc0 con 0x7fab00102e00 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.382+0000 7fab06200700 1 -- 172.21.15.69:0/1962136847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab00199330 con 0x7fab00102e00 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.382+0000 7faafd600700 1 -- 172.21.15.69:0/1962136847 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec005650 con 0x7fab00102e00 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.383+0000 7fab06200700 1 -- 172.21.15.69:0/1962136847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab001a54f0 con 0x7fab00102e00 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.384+0000 7fab06200700 1 -- 172.21.15.69:0/1962136847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab00062aa0 con 0x7fab00102e00 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.386+0000 7faafd600700 1 -- 172.21.15.69:0/1962136847 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7faaec004bd0 con 0x7fab00102e00 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.387+0000 7faafd600700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faae80a29a0 0x7faae80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:23.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.387+0000 7faafd600700 1 -- 172.21.15.69:0/1962136847 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7faaec0cefe0 con 0x7fab00102e00 2024-03-19T01:08:23.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.387+0000 7fab04e00700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faae80a29a0 0x7faae80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:23.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.387+0000 7faafd600700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684] conn(0x7faae80ad660 0x7faae80afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:23.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.387+0000 7faafd600700 1 -- 172.21.15.69:0/1962136847 --> [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faaec021e50 con 0x7faae80ad660 2024-03-19T01:08:23.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.387+0000 7faafd600700 1 -- 172.21.15.69:0/1962136847 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7faaec0d1600 con 0x7fab00102e00 2024-03-19T01:08:23.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.388+0000 7fab05800700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684] conn(0x7faae80ad660 0x7faae80afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:23.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.388+0000 7fab04e00700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faae80a29a0 0x7faae80a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faaf4006010 tx=0x7faaf400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:23.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.388+0000 7fab05800700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684] conn(0x7faae80ad660 0x7faae80afaa0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:23.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.389+0000 7faafd600700 1 -- 172.21.15.69:0/1962136847 <== osd.4 v2:172.21.15.86:6808/1709421684 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faaec021e50 con 0x7faae80ad660 2024-03-19T01:08:23.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.424+0000 7fab06200700 1 -- 172.21.15.69:0/1962136847 --> [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fab00062ea0 con 0x7faae80ad660 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.424+0000 7faafd600700 1 -- 172.21.15.69:0/1962136847 <== osd.4 v2:172.21.15.86:6808/1709421684 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fab00062ea0 con 0x7faae80ad660 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.425+0000 7fab06200700 1 -- 172.21.15.69:0/1962136847 >> [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684] conn(0x7faae80ad660 msgr2=0x7faae80afaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.425+0000 7fab06200700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684] conn(0x7faae80ad660 0x7faae80afaa0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.425+0000 7fab06200700 1 -- 172.21.15.69:0/1962136847 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faae80a29a0 msgr2=0x7faae80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.425+0000 7fab06200700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faae80a29a0 0x7faae80a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faaf4006010 tx=0x7faaf400b260 comp rx=0 tx=0).stop 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.425+0000 7fab06200700 1 -- 172.21.15.69:0/1962136847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab00102e00 msgr2=0x7fab0019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.425+0000 7fab06200700 1 --2- 172.21.15.69:0/1962136847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab00102e00 0x7fab0019ebd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faaec00e970 tx=0x7faaec00ec80 comp rx=0 tx=0).stop 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.425+0000 7fab05800700 1 -- 172.21.15.69:0/1962136847 reap_dead start 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.425+0000 7fab06200700 1 -- 172.21.15.69:0/1962136847 shutdown_connections 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.425+0000 7fab06200700 1 -- 172.21.15.69:0/1962136847 >> 172.21.15.69:0/1962136847 conn(0x7fab000fdce0 msgr2=0x7fab001066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.425+0000 7fab06200700 1 -- 172.21.15.69:0/1962136847 shutdown_connections 2024-03-19T01:08:23.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:23.425+0000 7fab06200700 1 -- 172.21.15.69:0/1962136847 wait complete. 2024-03-19T01:08:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-19T01:08:24.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-19T01:08:24.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-19T01:08:24.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-19T01:08:24.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-03-19T01:08:24.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-19T01:08:24.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-03-19T01:08:24.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-19T01:08:24.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-19T01:08:24.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:23 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-19T01:08:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-19T01:08:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-19T01:08:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-19T01:08:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-19T01:08:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-03-19T01:08:24.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-19T01:08:24.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-03-19T01:08:24.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-19T01:08:24.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-19T01:08:24.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:23 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-19T01:08:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:23 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-19T01:08:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:23 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-19T01:08:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:23 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-19T01:08:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:23 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-19T01:08:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:23 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-03-19T01:08:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:23 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-19T01:08:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:23 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-03-19T01:08:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:23 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-19T01:08:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:23 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-19T01:08:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:23 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-19T01:08:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-19T01:08:25.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-19T01:08:25.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-19T01:08:25.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-19T01:08:25.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-19T01:08:25.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-03-19T01:08:25.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-19T01:08:25.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-03-19T01:08:25.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-19T01:08:25.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-19T01:08:25.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-19T01:08:25.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:24 smithi163 ceph-mon[67428]: osdmap e90: 12 total, 12 up, 12 in 2024-03-19T01:08:25.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-19T01:08:25.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-19T01:08:25.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-19T01:08:25.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-19T01:08:25.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-19T01:08:25.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-03-19T01:08:25.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-19T01:08:25.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-03-19T01:08:25.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-19T01:08:25.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-19T01:08:25.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-19T01:08:25.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:24 smithi069 ceph-mon[31727]: osdmap e90: 12 total, 12 up, 12 in 2024-03-19T01:08:25.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-19T01:08:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-19T01:08:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-19T01:08:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-19T01:08:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-19T01:08:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-03-19T01:08:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-19T01:08:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-03-19T01:08:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-19T01:08:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-19T01:08:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-19T01:08:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:24 smithi086 ceph-mon[66829]: osdmap e90: 12 total, 12 up, 12 in 2024-03-19T01:08:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:25.972 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:25 smithi069 ceph-mon[31727]: osdmap e91: 12 total, 12 up, 12 in 2024-03-19T01:08:26.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:25 smithi163 ceph-mon[67428]: osdmap e91: 12 total, 12 up, 12 in 2024-03-19T01:08:26.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:25 smithi086 ceph-mon[66829]: osdmap e91: 12 total, 12 up, 12 in 2024-03-19T01:08:26.853 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:26 smithi069 ceph-mon[31727]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-03-19T01:08:27.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:26 smithi163 ceph-mon[67428]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-03-19T01:08:27.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:26 smithi086 ceph-mon[66829]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-03-19T01:08:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:141733920835 2024-03-19T01:08:27.769 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-19T01:08:28.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.917+0000 7f80fb200700 1 -- 172.21.15.69:0/3954250145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4074740 msgr2=0x7f80f4074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.917+0000 7f80fb200700 1 --2- 172.21.15.69:0/3954250145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4074740 0x7f80f4074b20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009990 tx=0x7f80e8009ca0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.917+0000 7f80fb200700 1 -- 172.21.15.69:0/3954250145 shutdown_connections 2024-03-19T01:08:28.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.917+0000 7f80fb200700 1 --2- 172.21.15.69:0/3954250145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f410ea30 0x7f80f4110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.917+0000 7f80fb200700 1 --2- 172.21.15.69:0/3954250145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4074740 0x7f80f4074b20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.917+0000 7f80fb200700 1 --2- 172.21.15.69:0/3954250145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80f4075060 0x7f80f410e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.917+0000 7f80fb200700 1 -- 172.21.15.69:0/3954250145 >> 172.21.15.69:0/3954250145 conn(0x7f80f40fdce0 msgr2=0x7f80f4100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:28.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.918+0000 7f80fb200700 1 -- 172.21.15.69:0/3954250145 shutdown_connections 2024-03-19T01:08:28.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.919+0000 7f80fb200700 1 -- 172.21.15.69:0/3954250145 wait complete. 2024-03-19T01:08:28.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.920+0000 7f80fb200700 1 Processor -- start 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.920+0000 7f80fb200700 1 -- start start 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.920+0000 7f80fb200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f4074740 0x7f80f4108d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.920+0000 7f80fb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4075060 0x7f80f4109240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.921+0000 7f80fb200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80f410ea30 0x7f80f4103e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.921+0000 7f80fb200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4102ad0 con 0x7f80f4074740 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.921+0000 7f80fb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4102950 con 0x7f80f4075060 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.921+0000 7f80fb200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4102c50 con 0x7f80f410ea30 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.921+0000 7f80f9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f4074740 0x7f80f4108d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.921+0000 7f80f9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4075060 0x7f80f4109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.921+0000 7f80f9e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f4074740 0x7f80f4108d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38084/0 (socket says 172.21.15.69:38084) 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.921+0000 7f80f9e00700 1 -- 172.21.15.69:0/1711138603 learned_addr learned my addr 172.21.15.69:0/1711138603 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.921+0000 7f80fa800700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80f410ea30 0x7f80f4103e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.922+0000 7f80f9e00700 1 -- 172.21.15.69:0/1711138603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80f410ea30 msgr2=0x7f80f4103e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.922+0000 7f80f9e00700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80f410ea30 0x7f80f4103e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.922+0000 7f80f9e00700 1 -- 172.21.15.69:0/1711138603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4075060 msgr2=0x7f80f4109240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.922+0000 7f80f9e00700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4075060 0x7f80f4109240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.922+0000 7f80f9e00700 1 -- 172.21.15.69:0/1711138603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f4104720 con 0x7f80f4074740 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.922+0000 7f80f9400700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4075060 0x7f80f4109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.922+0000 7f80f9e00700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f4074740 0x7f80f4108d00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80e8000c00 tx=0x7f80e800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.922+0000 7f80eea00700 1 -- 172.21.15.69:0/1711138603 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8027070 con 0x7f80f4074740 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.923+0000 7f80eea00700 1 -- 172.21.15.69:0/1711138603 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e8003d70 con 0x7f80f4074740 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.923+0000 7f80eea00700 1 -- 172.21.15.69:0/1711138603 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e800f460 con 0x7f80f4074740 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.923+0000 7f80fb200700 1 -- 172.21.15.69:0/1711138603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f41049b0 con 0x7f80f4074740 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.923+0000 7f80fb200700 1 -- 172.21.15.69:0/1711138603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f41b4410 con 0x7f80f4074740 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.924+0000 7f80fb200700 1 -- 172.21.15.69:0/1711138603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80f4062a20 con 0x7f80f4074740 2024-03-19T01:08:28.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.928+0000 7f80eea00700 1 -- 172.21.15.69:0/1711138603 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f80e802a070 con 0x7f80f4074740 2024-03-19T01:08:28.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.930+0000 7f80eea00700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80dc0a2920 0x7f80dc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:28.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.930+0000 7f80eea00700 1 -- 172.21.15.69:0/1711138603 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f80e80d0d50 con 0x7f80f4074740 2024-03-19T01:08:28.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.930+0000 7f80f9400700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80dc0a2920 0x7f80dc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:28.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.930+0000 7f80eea00700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392] conn(0x7f80dc0ada80 0x7f80dc0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:28.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.930+0000 7f80eea00700 1 -- 172.21.15.69:0/1711138603 --> [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80e8029e50 con 0x7f80dc0ada80 2024-03-19T01:08:28.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.930+0000 7f80eea00700 1 -- 172.21.15.69:0/1711138603 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f80e80d1100 con 0x7f80f4074740 2024-03-19T01:08:28.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.931+0000 7f80fa800700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392] conn(0x7f80dc0ada80 0x7f80dc0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:28.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.931+0000 7f80fa800700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392] conn(0x7f80dc0ada80 0x7f80dc0aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:28.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.931+0000 7f80f9400700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80dc0a2920 0x7f80dc0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f80f41055e0 tx=0x7f80e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:28.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.932+0000 7f80eea00700 1 -- 172.21.15.69:0/1711138603 <== osd.7 v2:172.21.15.86:6816/866804392 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80e8029e50 con 0x7f80dc0ada80 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.951+0000 7f80fb200700 1 -- 172.21.15.69:0/1711138603 --> [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f80f4062e20 con 0x7f80dc0ada80 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.951+0000 7f80eea00700 1 -- 172.21.15.69:0/1711138603 <== osd.7 v2:172.21.15.86:6816/866804392 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f80f4062e20 con 0x7f80dc0ada80 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.951+0000 7f80fb200700 1 -- 172.21.15.69:0/1711138603 >> [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392] conn(0x7f80dc0ada80 msgr2=0x7f80dc0aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.951+0000 7f80fb200700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392] conn(0x7f80dc0ada80 0x7f80dc0aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.951+0000 7f80fb200700 1 -- 172.21.15.69:0/1711138603 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80dc0a2920 msgr2=0x7f80dc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.952+0000 7f80fb200700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f80dc0a2920 0x7f80dc0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f80f41055e0 tx=0x7f80e4009260 comp rx=0 tx=0).stop 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.952+0000 7f80fb200700 1 -- 172.21.15.69:0/1711138603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f4074740 msgr2=0x7f80f4108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.952+0000 7f80fb200700 1 --2- 172.21.15.69:0/1711138603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f4074740 0x7f80f4108d00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80e8000c00 tx=0x7f80e800ba90 comp rx=0 tx=0).stop 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.952+0000 7f80fa800700 1 -- 172.21.15.69:0/1711138603 reap_dead start 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.952+0000 7f80fb200700 1 -- 172.21.15.69:0/1711138603 shutdown_connections 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.952+0000 7f80fb200700 1 -- 172.21.15.69:0/1711138603 >> 172.21.15.69:0/1711138603 conn(0x7f80f40fdce0 msgr2=0x7f80f40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.952+0000 7f80fb200700 1 -- 172.21.15.69:0/1711138603 shutdown_connections 2024-03-19T01:08:28.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:25.952+0000 7f80fb200700 1 -- 172.21.15.69:0/1711138603 wait complete. 2024-03-19T01:08:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.716+0000 7f9aebe00700 1 -- 172.21.15.69:0/839053742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ae4101210 msgr2=0x7f9ae4101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.716+0000 7f9aebe00700 1 --2- 172.21.15.69:0/839053742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ae4101210 0x7f9ae4101670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009990 tx=0x7f9ad0009ca0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.716+0000 7f9aebe00700 1 -- 172.21.15.69:0/839053742 shutdown_connections 2024-03-19T01:08:28.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.716+0000 7f9aebe00700 1 --2- 172.21.15.69:0/839053742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae4101bb0 0x7f9ae410e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.716+0000 7f9aebe00700 1 --2- 172.21.15.69:0/839053742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae4107250 0x7f9ae4107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.716+0000 7f9aebe00700 1 --2- 172.21.15.69:0/839053742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ae4101210 0x7f9ae4101670 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.716+0000 7f9aebe00700 1 -- 172.21.15.69:0/839053742 >> 172.21.15.69:0/839053742 conn(0x7f9ae4078140 msgr2=0x7f9ae4078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:28.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.716+0000 7f9aebe00700 1 -- 172.21.15.69:0/839053742 shutdown_connections 2024-03-19T01:08:28.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.716+0000 7f9aebe00700 1 -- 172.21.15.69:0/839053742 wait complete. 2024-03-19T01:08:28.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.718+0000 7f9aebe00700 1 Processor -- start 2024-03-19T01:08:28.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.718+0000 7f9aebe00700 1 -- start start 2024-03-19T01:08:28.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.718+0000 7f9aebe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae4101210 0x7f9ae41a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.718+0000 7f9aebe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ae4101bb0 0x7f9ae41a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.718+0000 7f9aebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae4107250 0x7f9ae419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.718+0000 7f9aebe00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae4111500 con 0x7f9ae4101210 2024-03-19T01:08:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.718+0000 7f9aebe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae4111380 con 0x7f9ae4107250 2024-03-19T01:08:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.718+0000 7f9aebe00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae4111680 con 0x7f9ae4101bb0 2024-03-19T01:08:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae8a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ae4101bb0 0x7f9ae41a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae9400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae4101210 0x7f9ae41a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:28.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae8a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ae4101bb0 0x7f9ae41a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:47398/0 (socket says 172.21.15.69:47398) 2024-03-19T01:08:28.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae8a00700 1 -- 172.21.15.69:0/563522021 learned_addr learned my addr 172.21.15.69:0/563522021 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:28.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae9e00700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae4107250 0x7f9ae419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae8a00700 1 -- 172.21.15.69:0/563522021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae4107250 msgr2=0x7f9ae419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae8a00700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae4107250 0x7f9ae419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae8a00700 1 -- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae4101210 msgr2=0x7f9ae41a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae8a00700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae4101210 0x7f9ae41a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae8a00700 1 -- 172.21.15.69:0/563522021 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae419b400 con 0x7f9ae4101bb0 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae9e00700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae4107250 0x7f9ae419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.719+0000 7f9ae9400700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ae4101210 0x7f9ae41a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.720+0000 7f9ae8a00700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ae4101bb0 0x7f9ae41a0f90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0000c00 tx=0x7f9ad000bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.720+0000 7f9ae2000700 1 -- 172.21.15.69:0/563522021 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0027070 con 0x7f9ae4101bb0 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.720+0000 7f9ae2000700 1 -- 172.21.15.69:0/563522021 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ad0003d70 con 0x7f9ae4101bb0 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.720+0000 7f9ae2000700 1 -- 172.21.15.69:0/563522021 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000f460 con 0x7f9ae4101bb0 2024-03-19T01:08:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.720+0000 7f9aebe00700 1 -- 172.21.15.69:0/563522021 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae419b690 con 0x7f9ae4101bb0 2024-03-19T01:08:28.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.720+0000 7f9aebe00700 1 -- 172.21.15.69:0/563522021 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae41a7800 con 0x7f9ae4101bb0 2024-03-19T01:08:28.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.721+0000 7f9aebe00700 1 -- 172.21.15.69:0/563522021 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ae406aa40 con 0x7f9ae4101bb0 2024-03-19T01:08:28.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.725+0000 7f9ae2000700 1 -- 172.21.15.69:0/563522021 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f9ad002a070 con 0x7f9ae4101bb0 2024-03-19T01:08:28.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.727+0000 7f9ae2000700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9acc0a29a0 0x7f9acc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:28.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.727+0000 7f9ae2000700 1 -- 172.21.15.69:0/563522021 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9ad00cfd30 con 0x7f9ae4101bb0 2024-03-19T01:08:28.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.727+0000 7f9ae9400700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9acc0a29a0 0x7f9acc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:28.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.727+0000 7f9ae2000700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914] conn(0x7f9acc0adb00 0x7f9acc0affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:28.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.727+0000 7f9ae2000700 1 -- 172.21.15.69:0/563522021 --> [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ad0018e50 con 0x7f9acc0adb00 2024-03-19T01:08:28.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.728+0000 7f9ae2000700 1 -- 172.21.15.69:0/563522021 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f9ad00d2350 con 0x7f9ae4101bb0 2024-03-19T01:08:28.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.728+0000 7f9ae9e00700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914] conn(0x7f9acc0adb00 0x7f9acc0affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:28.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.728+0000 7f9ae9400700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9acc0a29a0 0x7f9acc0a4e60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009c90 tx=0x7f9ad8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:28.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.728+0000 7f9ae9e00700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914] conn(0x7f9acc0adb00 0x7f9acc0affa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:28.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.730+0000 7f9ae2000700 1 -- 172.21.15.69:0/563522021 <== osd.3 v2:172.21.15.69:6810/2038881914 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9ad0018e50 con 0x7f9acc0adb00 2024-03-19T01:08:28.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.764+0000 7f9aebe00700 1 -- 172.21.15.69:0/563522021 --> [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9ae4102900 con 0x7f9acc0adb00 2024-03-19T01:08:28.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.766+0000 7f9ae2000700 1 -- 172.21.15.69:0/563522021 <== osd.3 v2:172.21.15.69:6810/2038881914 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9ae4102900 con 0x7f9acc0adb00 2024-03-19T01:08:28.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.766+0000 7f9aebe00700 1 -- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914] conn(0x7f9acc0adb00 msgr2=0x7f9acc0affa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.766+0000 7f9aebe00700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914] conn(0x7f9acc0adb00 0x7f9acc0affa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:28.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.767+0000 7f9aebe00700 1 -- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9acc0a29a0 msgr2=0x7f9acc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.767+0000 7f9aebe00700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9acc0a29a0 0x7f9acc0a4e60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009c90 tx=0x7f9ad8009260 comp rx=0 tx=0).stop 2024-03-19T01:08:28.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.767+0000 7f9aebe00700 1 -- 172.21.15.69:0/563522021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ae4101bb0 msgr2=0x7f9ae41a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:28.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.767+0000 7f9aebe00700 1 --2- 172.21.15.69:0/563522021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ae4101bb0 0x7f9ae41a0f90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0000c00 tx=0x7f9ad000bb60 comp rx=0 tx=0).stop 2024-03-19T01:08:28.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.767+0000 7f9ae9e00700 1 -- 172.21.15.69:0/563522021 reap_dead start 2024-03-19T01:08:28.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.767+0000 7f9aebe00700 1 -- 172.21.15.69:0/563522021 shutdown_connections 2024-03-19T01:08:28.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.767+0000 7f9aebe00700 1 -- 172.21.15.69:0/563522021 >> 172.21.15.69:0/563522021 conn(0x7f9ae4078140 msgr2=0x7f9ae4069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:28.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.767+0000 7f9aebe00700 1 -- 172.21.15.69:0/563522021 shutdown_connections 2024-03-19T01:08:28.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:28.767+0000 7f9aebe00700 1 -- 172.21.15.69:0/563522021 wait complete. 2024-03-19T01:08:29.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:28 smithi163 ceph-mon[67428]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 14 B/s, 0 objects/s recovering 2024-03-19T01:08:29.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:28 smithi086 ceph-mon[66829]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 14 B/s, 0 objects/s recovering 2024-03-19T01:08:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:28 smithi069 ceph-mon[31727]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 14 B/s, 0 objects/s recovering 2024-03-19T01:08:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.217+0000 7f4361400700 1 -- 172.21.15.69:0/2586282444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102e00 msgr2=0x7f435c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.217+0000 7f4361400700 1 --2- 172.21.15.69:0/2586282444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102e00 0x7f435c106e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4348009990 tx=0x7f4348009ca0 comp rx=0 tx=0).stop 2024-03-19T01:08:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.217+0000 7f4361400700 1 -- 172.21.15.69:0/2586282444 shutdown_connections 2024-03-19T01:08:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.217+0000 7f4361400700 1 --2- 172.21.15.69:0/2586282444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f435c107530 0x7f435c10f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.217+0000 7f4361400700 1 --2- 172.21.15.69:0/2586282444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f435c102450 0x7f435c102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.217+0000 7f4361400700 1 --2- 172.21.15.69:0/2586282444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102e00 0x7f435c106e50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.217+0000 7f4361400700 1 -- 172.21.15.69:0/2586282444 >> 172.21.15.69:0/2586282444 conn(0x7f435c0fdce0 msgr2=0x7f435c100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.217+0000 7f4361400700 1 -- 172.21.15.69:0/2586282444 shutdown_connections 2024-03-19T01:08:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.217+0000 7f4361400700 1 -- 172.21.15.69:0/2586282444 wait complete. 2024-03-19T01:08:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.218+0000 7f4361400700 1 Processor -- start 2024-03-19T01:08:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.218+0000 7f4361400700 1 -- start start 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f4361400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f435c102450 0x7f435c074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f4361400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102e00 0x7f435c074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f4361400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f435c107530 0x7f435c075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f4361400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f435c112540 con 0x7f435c107530 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f4361400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1123c0 con 0x7f435c102e00 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f4361400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1126c0 con 0x7f435c102450 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f435be00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f435c102450 0x7f435c074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f435be00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f435c102450 0x7f435c074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:47366/0 (socket says 172.21.15.69:47366) 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f435b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102e00 0x7f435c074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f435be00700 1 -- 172.21.15.69:0/2258934075 learned_addr learned my addr 172.21.15.69:0/2258934075 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f4360a00700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f435c107530 0x7f435c075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f435be00700 1 -- 172.21.15.69:0/2258934075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102e00 msgr2=0x7f435c074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f435be00700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102e00 0x7f435c074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.219+0000 7f435be00700 1 -- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f435c107530 msgr2=0x7f435c075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.220+0000 7f435be00700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f435c107530 0x7f435c075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.220+0000 7f435be00700 1 -- 172.21.15.69:0/2258934075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c1a6f40 con 0x7f435c102450 2024-03-19T01:08:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.220+0000 7f435be00700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f435c102450 0x7f435c074610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e9a0 tx=0x7f434c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.220+0000 7f4358c00700 1 -- 172.21.15.69:0/2258934075 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c00cbd0 con 0x7f435c102450 2024-03-19T01:08:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.220+0000 7f4358c00700 1 -- 172.21.15.69:0/2258934075 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434c00cd70 con 0x7f435c102450 2024-03-19T01:08:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.220+0000 7f4358c00700 1 -- 172.21.15.69:0/2258934075 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c018750 con 0x7f435c102450 2024-03-19T01:08:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.220+0000 7f4361400700 1 -- 172.21.15.69:0/2258934075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c1a7230 con 0x7f435c102450 2024-03-19T01:08:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.220+0000 7f4361400700 1 -- 172.21.15.69:0/2258934075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c1a76f0 con 0x7f435c102450 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.221+0000 7f4361400700 1 -- 172.21.15.69:0/2258934075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f435c062a20 con 0x7f435c102450 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.225+0000 7f4358c00700 1 -- 172.21.15.69:0/2258934075 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f434c010430 con 0x7f435c102450 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.226+0000 7f4358c00700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f43400a2920 0x7f43400a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.227+0000 7f4358c00700 1 -- 172.21.15.69:0/2258934075 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f434c0cfa50 con 0x7f435c102450 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.227+0000 7f435b400700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f43400a2920 0x7f43400a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.227+0000 7f4358c00700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729] conn(0x7f43400ada80 0x7f43400aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.227+0000 7f4358c00700 1 -- 172.21.15.69:0/2258934075 --> [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f434c00ae50 con 0x7f43400ada80 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.227+0000 7f4358c00700 1 -- 172.21.15.69:0/2258934075 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f434c0cfe90 con 0x7f435c102450 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.227+0000 7f4360a00700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729] conn(0x7f43400ada80 0x7f43400aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.227+0000 7f435b400700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f43400a2920 0x7f43400a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f435c0760a0 tx=0x7f4348005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.228+0000 7f4360a00700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729] conn(0x7f43400ada80 0x7f43400aff20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.229+0000 7f4358c00700 1 -- 172.21.15.69:0/2258934075 <== osd.0 v2:172.21.15.69:6802/3647912729 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f434c00ae50 con 0x7f43400ada80 2024-03-19T01:08:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.247+0000 7f4361400700 1 -- 172.21.15.69:0/2258934075 --> [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f435c062e20 con 0x7f43400ada80 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.248+0000 7f4358c00700 1 -- 172.21.15.69:0/2258934075 <== osd.0 v2:172.21.15.69:6802/3647912729 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f435c062e20 con 0x7f43400ada80 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.248+0000 7f4361400700 1 -- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729] conn(0x7f43400ada80 msgr2=0x7f43400aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.248+0000 7f4361400700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729] conn(0x7f43400ada80 0x7f43400aff20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.248+0000 7f4361400700 1 -- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f43400a2920 msgr2=0x7f43400a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.248+0000 7f4361400700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f43400a2920 0x7f43400a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f435c0760a0 tx=0x7f4348005fd0 comp rx=0 tx=0).stop 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.248+0000 7f4361400700 1 -- 172.21.15.69:0/2258934075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f435c102450 msgr2=0x7f435c074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.248+0000 7f4361400700 1 --2- 172.21.15.69:0/2258934075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f435c102450 0x7f435c074610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e9a0 tx=0x7f434c00ecb0 comp rx=0 tx=0).stop 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.248+0000 7f4360a00700 1 -- 172.21.15.69:0/2258934075 reap_dead start 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.249+0000 7f4361400700 1 -- 172.21.15.69:0/2258934075 shutdown_connections 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.249+0000 7f4361400700 1 -- 172.21.15.69:0/2258934075 >> 172.21.15.69:0/2258934075 conn(0x7f435c0fdce0 msgr2=0x7f435c0fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.249+0000 7f4361400700 1 -- 172.21.15.69:0/2258934075 shutdown_connections 2024-03-19T01:08:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:27.249+0000 7f4361400700 1 -- 172.21.15.69:0/2258934075 wait complete. 2024-03-19T01:08:29.635 INFO:teuthology.orchestra.run.smithi069.stdout:309237645344 2024-03-19T01:08:29.636 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-19T01:08:31.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:30 smithi069 ceph-mon[31727]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:08:31.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:30 smithi086 ceph-mon[66829]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:08:31.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:30 smithi163 ceph-mon[67428]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:08:32.675 INFO:teuthology.orchestra.run.smithi069.stdout:34359738457 2024-03-19T01:08:32.675 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-19T01:08:32.897 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:32 smithi069 ceph-mon[31727]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:08:33.122 INFO:teuthology.orchestra.run.smithi069.stdout:219043332145 2024-03-19T01:08:33.122 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-19T01:08:33.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:32 smithi086 ceph-mon[66829]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:08:33.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:32 smithi163 ceph-mon[67428]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:08:33.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.895+0000 7fb7ede00700 1 -- 172.21.15.69:0/4204608382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e8101350 msgr2=0x7fb7e8101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:33.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.895+0000 7fb7ede00700 1 --2- 172.21.15.69:0/4204608382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e8101350 0x7fb7e8101730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0099c0 tx=0x7fb7dc009cd0 comp rx=0 tx=0).stop 2024-03-19T01:08:33.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.895+0000 7fb7ede00700 1 -- 172.21.15.69:0/4204608382 shutdown_connections 2024-03-19T01:08:33.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.895+0000 7fb7ede00700 1 --2- 172.21.15.69:0/4204608382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e8106430 0x7fb7e810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:33.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.895+0000 7fb7ede00700 1 --2- 172.21.15.69:0/4204608382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e8101350 0x7fb7e8101730 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:33.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.895+0000 7fb7ede00700 1 --2- 172.21.15.69:0/4204608382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7e8101d00 0x7fb7e8105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:33.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.895+0000 7fb7ede00700 1 -- 172.21.15.69:0/4204608382 >> 172.21.15.69:0/4204608382 conn(0x7fb7e80fcbe0 msgr2=0x7fb7e80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:33.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.895+0000 7fb7ede00700 1 -- 172.21.15.69:0/4204608382 shutdown_connections 2024-03-19T01:08:33.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.895+0000 7fb7ede00700 1 -- 172.21.15.69:0/4204608382 wait complete. 2024-03-19T01:08:33.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.896+0000 7fb7ede00700 1 Processor -- start 2024-03-19T01:08:33.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.896+0000 7fb7ede00700 1 -- start start 2024-03-19T01:08:33.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.897+0000 7fb7ede00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7e8101350 0x7fb7e8102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:33.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.897+0000 7fb7ede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e8101d00 0x7fb7e8103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:33.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.897+0000 7fb7ede00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e8106430 0x7fb7e8103870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:33.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.897+0000 7fb7ede00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e8111410 con 0x7fb7e8101350 2024-03-19T01:08:33.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.897+0000 7fb7ede00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e8111290 con 0x7fb7e8101d00 2024-03-19T01:08:33.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.897+0000 7fb7ede00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e8111590 con 0x7fb7e8106430 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.897+0000 7fb7eca00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7e8101350 0x7fb7e8102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.897+0000 7fb7eca00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7e8101350 0x7fb7e8102df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38140/0 (socket says 172.21.15.69:38140) 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.897+0000 7fb7eca00700 1 -- 172.21.15.69:0/4023071831 learned_addr learned my addr 172.21.15.69:0/4023071831 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.897+0000 7fb7e7e00700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e8101d00 0x7fb7e8103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.897+0000 7fb7ed400700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e8106430 0x7fb7e8103870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.898+0000 7fb7e7e00700 1 -- 172.21.15.69:0/4023071831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e8106430 msgr2=0x7fb7e8103870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.898+0000 7fb7e7e00700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e8106430 0x7fb7e8103870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.898+0000 7fb7e7e00700 1 -- 172.21.15.69:0/4023071831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7e8101350 msgr2=0x7fb7e8102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.898+0000 7fb7e7e00700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb7e8101350 0x7fb7e8102df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.898+0000 7fb7e7e00700 1 -- 172.21.15.69:0/4023071831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e81a6eb0 con 0x7fb7e8101d00 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.898+0000 7fb7ed400700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e8106430 0x7fb7e8103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:08:33.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.898+0000 7fb7e7e00700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e8101d00 0x7fb7e8103330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400e9c0 tx=0x7fb7d400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:33.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.898+0000 7fb7e5600700 1 -- 172.21.15.69:0/4023071831 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d400eea0 con 0x7fb7e8101d00 2024-03-19T01:08:33.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.898+0000 7fb7e5600700 1 -- 172.21.15.69:0/4023071831 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d400cbd0 con 0x7fb7e8101d00 2024-03-19T01:08:33.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.899+0000 7fb7e5600700 1 -- 172.21.15.69:0/4023071831 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4018710 con 0x7fb7e8101d00 2024-03-19T01:08:33.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.899+0000 7fb7ede00700 1 -- 172.21.15.69:0/4023071831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e81a71a0 con 0x7fb7e8101d00 2024-03-19T01:08:33.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.899+0000 7fb7ede00700 1 -- 172.21.15.69:0/4023071831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e81a76e0 con 0x7fb7e8101d00 2024-03-19T01:08:33.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.900+0000 7fb7ede00700 1 -- 172.21.15.69:0/4023071831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7e8062a20 con 0x7fb7e8101d00 2024-03-19T01:08:33.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.903+0000 7fb7e5600700 1 -- 172.21.15.69:0/4023071831 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb7d4010430 con 0x7fb7e8101d00 2024-03-19T01:08:33.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.904+0000 7fb7e5600700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb7d00a2920 0x7fb7d00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:33.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.904+0000 7fb7e5600700 1 -- 172.21.15.69:0/4023071831 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb7d40cf1b0 con 0x7fb7e8101d00 2024-03-19T01:08:33.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.905+0000 7fb7eca00700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb7d00a2920 0x7fb7d00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:33.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.905+0000 7fb7e5600700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503] conn(0x7fb7d00ada80 0x7fb7d00aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:33.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.905+0000 7fb7e5600700 1 -- 172.21.15.69:0/4023071831 --> [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7d00b05f0 con 0x7fb7d00ada80 2024-03-19T01:08:33.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.905+0000 7fb7e5600700 1 -- 172.21.15.69:0/4023071831 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fb7d40d6050 con 0x7fb7e8101d00 2024-03-19T01:08:33.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.905+0000 7fb7ed400700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503] conn(0x7fb7d00ada80 0x7fb7d00aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:33.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.906+0000 7fb7eca00700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb7d00a2920 0x7fb7d00a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc003800 tx=0x7fb7dc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:33.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.906+0000 7fb7ed400700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503] conn(0x7fb7d00ada80 0x7fb7d00aff20 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:33.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.907+0000 7fb7e5600700 1 -- 172.21.15.69:0/4023071831 <== osd.2 v2:172.21.15.163:6800/3542508503 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7d00b05f0 con 0x7fb7d00ada80 2024-03-19T01:08:33.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.945+0000 7fb7ede00700 1 -- 172.21.15.69:0/4023071831 --> [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7e8062e20 con 0x7fb7d00ada80 2024-03-19T01:08:33.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.945+0000 7fb7e5600700 1 -- 172.21.15.69:0/4023071831 <== osd.2 v2:172.21.15.163:6800/3542508503 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb7e8062e20 con 0x7fb7d00ada80 2024-03-19T01:08:33.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.946+0000 7fb7ede00700 1 -- 172.21.15.69:0/4023071831 >> [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503] conn(0x7fb7d00ada80 msgr2=0x7fb7d00aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:33.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.946+0000 7fb7ede00700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503] conn(0x7fb7d00ada80 0x7fb7d00aff20 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:33.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.946+0000 7fb7ede00700 1 -- 172.21.15.69:0/4023071831 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb7d00a2920 msgr2=0x7fb7d00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:33.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.946+0000 7fb7ede00700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb7d00a2920 0x7fb7d00a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc003800 tx=0x7fb7dc01a040 comp rx=0 tx=0).stop 2024-03-19T01:08:33.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.946+0000 7fb7ede00700 1 -- 172.21.15.69:0/4023071831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e8101d00 msgr2=0x7fb7e8103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:33.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.946+0000 7fb7ede00700 1 --2- 172.21.15.69:0/4023071831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7e8101d00 0x7fb7e8103330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400e9c0 tx=0x7fb7d400ed80 comp rx=0 tx=0).stop 2024-03-19T01:08:33.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.946+0000 7fb7ed400700 1 -- 172.21.15.69:0/4023071831 reap_dead start 2024-03-19T01:08:33.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.946+0000 7fb7ede00700 1 -- 172.21.15.69:0/4023071831 shutdown_connections 2024-03-19T01:08:33.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.946+0000 7fb7ede00700 1 -- 172.21.15.69:0/4023071831 >> 172.21.15.69:0/4023071831 conn(0x7fb7e80fcbe0 msgr2=0x7fb7e81052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:33.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.947+0000 7fb7ede00700 1 -- 172.21.15.69:0/4023071831 shutdown_connections 2024-03-19T01:08:33.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:32.947+0000 7fb7ede00700 1 -- 172.21.15.69:0/4023071831 wait complete. 2024-03-19T01:08:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.889+0000 7f6b52000700 1 -- 172.21.15.69:0/1744950560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b4c10ed10 msgr2=0x7f6b4c111100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.889+0000 7f6b52000700 1 --2- 172.21.15.69:0/1744950560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b4c10ed10 0x7f6b4c111100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009f90 tx=0x7f6b4400b330 comp rx=0 tx=0).stop 2024-03-19T01:08:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.890+0000 7f6b52000700 1 -- 172.21.15.69:0/1744950560 shutdown_connections 2024-03-19T01:08:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.890+0000 7f6b52000700 1 --2- 172.21.15.69:0/1744950560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b4c10ed10 0x7f6b4c111100 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.890+0000 7f6b52000700 1 --2- 172.21.15.69:0/1744950560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b4c074540 0x7f6b4c074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.890+0000 7f6b52000700 1 --2- 172.21.15.69:0/1744950560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b4c074e60 0x7f6b4c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.890+0000 7f6b52000700 1 -- 172.21.15.69:0/1744950560 >> 172.21.15.69:0/1744950560 conn(0x7f6b4c079470 msgr2=0x7f6b4c079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.890+0000 7f6b52000700 1 -- 172.21.15.69:0/1744950560 shutdown_connections 2024-03-19T01:08:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.890+0000 7f6b52000700 1 -- 172.21.15.69:0/1744950560 wait complete. 2024-03-19T01:08:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.891+0000 7f6b52000700 1 Processor -- start 2024-03-19T01:08:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.891+0000 7f6b52000700 1 -- start start 2024-03-19T01:08:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.891+0000 7f6b52000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b4c074540 0x7f6b4c1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.891+0000 7f6b52000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b4c074e60 0x7f6b4c1a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.891+0000 7f6b52000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b4c10ed10 0x7f6b4c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.891+0000 7f6b52000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c114900 con 0x7f6b4c10ed10 2024-03-19T01:08:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.891+0000 7f6b52000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c114780 con 0x7f6b4c074e60 2024-03-19T01:08:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.891+0000 7f6b52000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c114a80 con 0x7f6b4c074540 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b4aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b4c074e60 0x7f6b4c1a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b4b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b4c074540 0x7f6b4c1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b4aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b4c074e60 0x7f6b4c1a0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42184/0 (socket says 172.21.15.69:42184) 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b4aa00700 1 -- 172.21.15.69:0/3312809145 learned_addr learned my addr 172.21.15.69:0/3312809145 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b4be00700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b4c10ed10 0x7f6b4c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b4aa00700 1 -- 172.21.15.69:0/3312809145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b4c074540 msgr2=0x7f6b4c1a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b4aa00700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b4c074540 0x7f6b4c1a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b4aa00700 1 -- 172.21.15.69:0/3312809145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b4c10ed10 msgr2=0x7f6b4c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b4aa00700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b4c10ed10 0x7f6b4c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b4aa00700 1 -- 172.21.15.69:0/3312809145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c19b2a0 con 0x7f6b4c074e60 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b4aa00700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b4c074e60 0x7f6b4c1a0e70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800e9a0 tx=0x7f6b3800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b43e00700 1 -- 172.21.15.69:0/3312809145 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3800cbd0 con 0x7f6b4c074e60 2024-03-19T01:08:34.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b43e00700 1 -- 172.21.15.69:0/3312809145 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b3800cd70 con 0x7f6b4c074e60 2024-03-19T01:08:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.892+0000 7f6b52000700 1 -- 172.21.15.69:0/3312809145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4c19b590 con 0x7f6b4c074e60 2024-03-19T01:08:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.893+0000 7f6b43e00700 1 -- 172.21.15.69:0/3312809145 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38018710 con 0x7f6b4c074e60 2024-03-19T01:08:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.893+0000 7f6b52000700 1 -- 172.21.15.69:0/3312809145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b4c1a7800 con 0x7f6b4c074e60 2024-03-19T01:08:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.893+0000 7f6b52000700 1 -- 172.21.15.69:0/3312809145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b4c0ffdc0 con 0x7f6b4c074e60 2024-03-19T01:08:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.895+0000 7f6b43e00700 1 -- 172.21.15.69:0/3312809145 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f6b38010430 con 0x7f6b4c074e60 2024-03-19T01:08:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.898+0000 7f6b43e00700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6b300a29a0 0x7f6b300a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.898+0000 7f6b43e00700 1 -- 172.21.15.69:0/3312809145 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6b380cf3a0 con 0x7f6b4c074e60 2024-03-19T01:08:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.898+0000 7f6b4b400700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6b300a29a0 0x7f6b300a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.898+0000 7f6b43e00700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946] conn(0x7f6b300adb00 0x7f6b300affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.898+0000 7f6b43e00700 1 -- 172.21.15.69:0/3312809145 --> [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b38012ec0 con 0x7f6b300adb00 2024-03-19T01:08:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.898+0000 7f6b43e00700 1 -- 172.21.15.69:0/3312809145 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f6b380cf750 con 0x7f6b4c074e60 2024-03-19T01:08:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.899+0000 7f6b4be00700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946] conn(0x7f6b300adb00 0x7f6b300affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.899+0000 7f6b4b400700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6b300a29a0 0x7f6b300a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c005fd0 tx=0x7f6b3c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.899+0000 7f6b4be00700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946] conn(0x7f6b300adb00 0x7f6b300affa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.900+0000 7f6b43e00700 1 -- 172.21.15.69:0/3312809145 <== osd.1 v2:172.21.15.86:6800/2662600946 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b38012ec0 con 0x7f6b300adb00 2024-03-19T01:08:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.918+0000 7f6b52000700 1 -- 172.21.15.69:0/3312809145 --> [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6b4c075c70 con 0x7f6b300adb00 2024-03-19T01:08:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.918+0000 7f6b43e00700 1 -- 172.21.15.69:0/3312809145 <== osd.1 v2:172.21.15.86:6800/2662600946 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6b4c075c70 con 0x7f6b300adb00 2024-03-19T01:08:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.918+0000 7f6b52000700 1 -- 172.21.15.69:0/3312809145 >> [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946] conn(0x7f6b300adb00 msgr2=0x7f6b300affa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.918+0000 7f6b52000700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946] conn(0x7f6b300adb00 0x7f6b300affa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.919+0000 7f6b52000700 1 -- 172.21.15.69:0/3312809145 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6b300a29a0 msgr2=0x7f6b300a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.919+0000 7f6b52000700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6b300a29a0 0x7f6b300a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c005fd0 tx=0x7f6b3c009310 comp rx=0 tx=0).stop 2024-03-19T01:08:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.919+0000 7f6b52000700 1 -- 172.21.15.69:0/3312809145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b4c074e60 msgr2=0x7f6b4c1a0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.919+0000 7f6b52000700 1 --2- 172.21.15.69:0/3312809145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b4c074e60 0x7f6b4c1a0e70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800e9a0 tx=0x7f6b3800ed60 comp rx=0 tx=0).stop 2024-03-19T01:08:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.919+0000 7f6b4be00700 1 -- 172.21.15.69:0/3312809145 reap_dead start 2024-03-19T01:08:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.919+0000 7f6b52000700 1 -- 172.21.15.69:0/3312809145 shutdown_connections 2024-03-19T01:08:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.919+0000 7f6b52000700 1 -- 172.21.15.69:0/3312809145 >> 172.21.15.69:0/3312809145 conn(0x7f6b4c079470 msgr2=0x7f6b4c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:34.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.919+0000 7f6b52000700 1 -- 172.21.15.69:0/3312809145 shutdown_connections 2024-03-19T01:08:34.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:31.919+0000 7f6b52000700 1 -- 172.21.15.69:0/3312809145 wait complete. 2024-03-19T01:08:35.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:34 smithi069 ceph-mon[31727]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:08:35.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:34 smithi086 ceph-mon[66829]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:08:35.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.937+0000 7f7ef4a00700 1 -- 172.21.15.69:0/2134078818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0107340 msgr2=0x7f7ef006aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:35.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.937+0000 7f7ef4a00700 1 --2- 172.21.15.69:0/2134078818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0107340 0x7f7ef006aa10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009990 tx=0x7f7ee0009ca0 comp rx=0 tx=0).stop 2024-03-19T01:08:35.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.938+0000 7f7ef4a00700 1 -- 172.21.15.69:0/2134078818 shutdown_connections 2024-03-19T01:08:35.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.938+0000 7f7ef4a00700 1 --2- 172.21.15.69:0/2134078818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef0104e40 0x7f7ef01052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:35.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.938+0000 7f7ef4a00700 1 --2- 172.21.15.69:0/2134078818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0107340 0x7f7ef006aa10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:35.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.938+0000 7f7ef4a00700 1 --2- 172.21.15.69:0/2134078818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef006af50 0x7f7ef006b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.938+0000 7f7ef4a00700 1 -- 172.21.15.69:0/2134078818 >> 172.21.15.69:0/2134078818 conn(0x7f7ef0077230 msgr2=0x7f7ef0077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.938+0000 7f7ef4a00700 1 -- 172.21.15.69:0/2134078818 shutdown_connections 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.938+0000 7f7ef4a00700 1 -- 172.21.15.69:0/2134078818 wait complete. 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.939+0000 7f7ef4a00700 1 Processor -- start 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.940+0000 7f7ef4a00700 1 -- start start 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.940+0000 7f7ef4a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef006af50 0x7f7ef019f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.940+0000 7f7ef4a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef0104e40 0x7f7ef019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.940+0000 7f7ef4a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0107340 0x7f7ef01a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.940+0000 7f7ef4a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0118d30 con 0x7f7ef0104e40 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.940+0000 7f7ef4a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0118bb0 con 0x7f7ef006af50 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.940+0000 7f7ef4a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0118eb0 con 0x7f7ef0107340 2024-03-19T01:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.940+0000 7f7eede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef006af50 0x7f7ef019f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.940+0000 7f7eede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef006af50 0x7f7ef019f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42224/0 (socket says 172.21.15.69:42224) 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.940+0000 7f7eede00700 1 -- 172.21.15.69:0/3430969584 learned_addr learned my addr 172.21.15.69:0/3430969584 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.941+0000 7f7ee7e00700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef0104e40 0x7f7ef019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.941+0000 7f7eee800700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0107340 0x7f7ef01a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.941+0000 7f7eede00700 1 -- 172.21.15.69:0/3430969584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0107340 msgr2=0x7f7ef01a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.941+0000 7f7eede00700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0107340 0x7f7ef01a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.941+0000 7f7eede00700 1 -- 172.21.15.69:0/3430969584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef0104e40 msgr2=0x7f7ef019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.941+0000 7f7eede00700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef0104e40 0x7f7ef019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.941+0000 7f7eede00700 1 -- 172.21.15.69:0/3430969584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef01a4200 con 0x7f7ef006af50 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.941+0000 7f7eede00700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef006af50 0x7f7ef019f240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.941+0000 7f7ee6a00700 1 -- 172.21.15.69:0/3430969584 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0027070 con 0x7f7ef006af50 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.942+0000 7f7ee6a00700 1 -- 172.21.15.69:0/3430969584 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee0003d70 con 0x7f7ef006af50 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.942+0000 7f7ef4a00700 1 -- 172.21.15.69:0/3430969584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef01a4490 con 0x7f7ef006af50 2024-03-19T01:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.942+0000 7f7ee6a00700 1 -- 172.21.15.69:0/3430969584 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee000f460 con 0x7f7ef006af50 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.942+0000 7f7ef4a00700 1 -- 172.21.15.69:0/3430969584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef01abd70 con 0x7f7ef006af50 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.943+0000 7f7ef4a00700 1 -- 172.21.15.69:0/3430969584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ef0100c90 con 0x7f7ef006af50 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.945+0000 7f7ee6a00700 1 -- 172.21.15.69:0/3430969584 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f7ee002a070 con 0x7f7ef006af50 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.946+0000 7f7ee6a00700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7ed40a29a0 0x7f7ed40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.946+0000 7f7ee6a00700 1 -- 172.21.15.69:0/3430969584 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7ee00d00c0 con 0x7f7ef006af50 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.947+0000 7f7ee6a00700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559] conn(0x7f7ed40adb00 0x7f7ed40affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.947+0000 7f7ee6a00700 1 -- 172.21.15.69:0/3430969584 --> [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ee0029e50 con 0x7f7ed40adb00 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.947+0000 7f7ee6a00700 1 -- 172.21.15.69:0/3430969584 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f7ee00d7050 con 0x7f7ef006af50 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.947+0000 7f7eee800700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559] conn(0x7f7ed40adb00 0x7f7ed40affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.947+0000 7f7ee7e00700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7ed40a29a0 0x7f7ed40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.948+0000 7f7ee7e00700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7ed40a29a0 0x7f7ed40a4e60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7ef006aca0 tx=0x7f7ed8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.948+0000 7f7eee800700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559] conn(0x7f7ed40adb00 0x7f7ed40affa0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.949+0000 7f7ee6a00700 1 -- 172.21.15.69:0/3430969584 <== osd.11 v2:172.21.15.163:6824/2918196559 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ee0029e50 con 0x7f7ed40adb00 2024-03-19T01:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.977+0000 7f7ef4a00700 1 -- 172.21.15.69:0/3430969584 --> [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7ef0101820 con 0x7f7ed40adb00 2024-03-19T01:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.977+0000 7f7ee6a00700 1 -- 172.21.15.69:0/3430969584 <== osd.11 v2:172.21.15.163:6824/2918196559 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7ef0101820 con 0x7f7ed40adb00 2024-03-19T01:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.978+0000 7f7ef4a00700 1 -- 172.21.15.69:0/3430969584 >> [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559] conn(0x7f7ed40adb00 msgr2=0x7f7ed40affa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.978+0000 7f7ef4a00700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559] conn(0x7f7ed40adb00 0x7f7ed40affa0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.978+0000 7f7ef4a00700 1 -- 172.21.15.69:0/3430969584 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7ed40a29a0 msgr2=0x7f7ed40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.978+0000 7f7ef4a00700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7ed40a29a0 0x7f7ed40a4e60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7ef006aca0 tx=0x7f7ed8009260 comp rx=0 tx=0).stop 2024-03-19T01:08:35.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.978+0000 7f7ef4a00700 1 -- 172.21.15.69:0/3430969584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef006af50 msgr2=0x7f7ef019f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:35.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.978+0000 7f7ef4a00700 1 --2- 172.21.15.69:0/3430969584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ef006af50 0x7f7ef019f240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee000bab0 comp rx=0 tx=0).stop 2024-03-19T01:08:35.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.978+0000 7f7eee800700 1 -- 172.21.15.69:0/3430969584 reap_dead start 2024-03-19T01:08:35.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.978+0000 7f7ef4a00700 1 -- 172.21.15.69:0/3430969584 shutdown_connections 2024-03-19T01:08:35.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.978+0000 7f7ef4a00700 1 -- 172.21.15.69:0/3430969584 >> 172.21.15.69:0/3430969584 conn(0x7f7ef0077230 msgr2=0x7f7ef0100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:35.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.978+0000 7f7ef4a00700 1 -- 172.21.15.69:0/3430969584 shutdown_connections 2024-03-19T01:08:35.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:33.978+0000 7f7ef4a00700 1 -- 172.21.15.69:0/3430969584 wait complete. 2024-03-19T01:08:35.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:34 smithi163 ceph-mon[67428]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:08:35.695 INFO:teuthology.orchestra.run.smithi069.stdout:107374182472 2024-03-19T01:08:35.695 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-19T01:08:36.131 INFO:teuthology.orchestra.run.smithi069.stdout:55834574932 2024-03-19T01:08:36.131 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-19T01:08:36.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.419+0000 7f3379e00700 1 -- 172.21.15.69:0/153803681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3374107690 msgr2=0x7f337410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:36.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.419+0000 7f3379e00700 1 --2- 172.21.15.69:0/153803681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3374107690 0x7f337410f260 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f336c00b0a0 tx=0x7f336c00b3b0 comp rx=0 tx=0).stop 2024-03-19T01:08:36.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.419+0000 7f3379e00700 1 -- 172.21.15.69:0/153803681 shutdown_connections 2024-03-19T01:08:36.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.420+0000 7f3379e00700 1 --2- 172.21.15.69:0/153803681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3374107690 0x7f337410f260 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:36.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.420+0000 7f3379e00700 1 --2- 172.21.15.69:0/153803681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33741025c0 0x7f33741029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:36.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.420+0000 7f3379e00700 1 --2- 172.21.15.69:0/153803681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3374102f70 0x7f3374106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:36.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.420+0000 7f3379e00700 1 -- 172.21.15.69:0/153803681 >> 172.21.15.69:0/153803681 conn(0x7f3374079470 msgr2=0x7f3374079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:36.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.420+0000 7f3379e00700 1 -- 172.21.15.69:0/153803681 shutdown_connections 2024-03-19T01:08:36.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.420+0000 7f3379e00700 1 -- 172.21.15.69:0/153803681 wait complete. 2024-03-19T01:08:36.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.421+0000 7f3379e00700 1 Processor -- start 2024-03-19T01:08:36.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.421+0000 7f3379e00700 1 -- start start 2024-03-19T01:08:36.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.422+0000 7f3379e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33741025c0 0x7f337419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:36.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.422+0000 7f3379e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3374102f70 0x7f337419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:36.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.422+0000 7f3379e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3374107690 0x7f337419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:36.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.422+0000 7f3379e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f33741126e0 con 0x7f3374102f70 2024-03-19T01:08:36.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.422+0000 7f3379e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3374112560 con 0x7f3374107690 2024-03-19T01:08:36.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.422+0000 7f3379e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3374112860 con 0x7f33741025c0 2024-03-19T01:08:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.423+0000 7f3372a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33741025c0 0x7f337419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.423+0000 7f3372000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3374102f70 0x7f337419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.423+0000 7f3372a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33741025c0 0x7f337419ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37778/0 (socket says 172.21.15.69:37778) 2024-03-19T01:08:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.423+0000 7f3372a00700 1 -- 172.21.15.69:0/1075506801 learned_addr learned my addr 172.21.15.69:0/1075506801 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.423+0000 7f3373400700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3374107690 0x7f337419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.423+0000 7f3372a00700 1 -- 172.21.15.69:0/1075506801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3374107690 msgr2=0x7f337419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.423+0000 7f3372a00700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3374107690 0x7f337419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.423+0000 7f3372a00700 1 -- 172.21.15.69:0/1075506801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3374102f70 msgr2=0x7f337419b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.423+0000 7f3372a00700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3374102f70 0x7f337419b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.423+0000 7f3372a00700 1 -- 172.21.15.69:0/1075506801 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337419fd90 con 0x7f33741025c0 2024-03-19T01:08:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.423+0000 7f3372a00700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33741025c0 0x7f337419ad80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f336400e9a0 tx=0x7f336400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.424+0000 7f3378a00700 1 -- 172.21.15.69:0/1075506801 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336400cbd0 con 0x7f33741025c0 2024-03-19T01:08:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.424+0000 7f3378a00700 1 -- 172.21.15.69:0/1075506801 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336400cd70 con 0x7f33741025c0 2024-03-19T01:08:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.424+0000 7f3379e00700 1 -- 172.21.15.69:0/1075506801 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33741a0080 con 0x7f33741025c0 2024-03-19T01:08:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.424+0000 7f3379e00700 1 -- 172.21.15.69:0/1075506801 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33741a7900 con 0x7f33741025c0 2024-03-19T01:08:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.424+0000 7f3378a00700 1 -- 172.21.15.69:0/1075506801 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364018750 con 0x7f33741025c0 2024-03-19T01:08:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.425+0000 7f3379e00700 1 -- 172.21.15.69:0/1075506801 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3354001030 con 0x7f33741025c0 2024-03-19T01:08:36.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.427+0000 7f3378a00700 1 -- 172.21.15.69:0/1075506801 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f3364010430 con 0x7f33741025c0 2024-03-19T01:08:36.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.428+0000 7f3378a00700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f33580a29a0 0x7f33580a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:36.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.429+0000 7f3378a00700 1 -- 172.21.15.69:0/1075506801 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f33640cfc70 con 0x7f33741025c0 2024-03-19T01:08:36.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.429+0000 7f3378a00700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076] conn(0x7f33580adb00 0x7f33580affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:36.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.429+0000 7f3378a00700 1 -- 172.21.15.69:0/1075506801 --> [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f33580b0670 con 0x7f33580adb00 2024-03-19T01:08:36.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.429+0000 7f3378a00700 1 -- 172.21.15.69:0/1075506801 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f33640d0020 con 0x7f33741025c0 2024-03-19T01:08:36.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.429+0000 7f3373400700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076] conn(0x7f33580adb00 0x7f33580affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:36.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.429+0000 7f3372000700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f33580a29a0 0x7f33580a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:36.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.430+0000 7f3373400700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076] conn(0x7f33580adb00 0x7f33580affa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:36.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.430+0000 7f3372000700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f33580a29a0 0x7f33580a4e60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f337410f710 tx=0x7f3360009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:36.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.431+0000 7f3378a00700 1 -- 172.21.15.69:0/1075506801 <== osd.5 v2:172.21.15.163:6808/95931076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f33580b0670 con 0x7f33580adb00 2024-03-19T01:08:36.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.450+0000 7f3379e00700 1 -- 172.21.15.69:0/1075506801 --> [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3354002e60 con 0x7f33580adb00 2024-03-19T01:08:36.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.450+0000 7f3378a00700 1 -- 172.21.15.69:0/1075506801 <== osd.5 v2:172.21.15.163:6808/95931076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3354002e60 con 0x7f33580adb00 2024-03-19T01:08:36.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.451+0000 7f3379e00700 1 -- 172.21.15.69:0/1075506801 >> [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076] conn(0x7f33580adb00 msgr2=0x7f33580affa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:36.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.451+0000 7f3379e00700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076] conn(0x7f33580adb00 0x7f33580affa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:36.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.451+0000 7f3379e00700 1 -- 172.21.15.69:0/1075506801 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f33580a29a0 msgr2=0x7f33580a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:36.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.451+0000 7f3379e00700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f33580a29a0 0x7f33580a4e60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f337410f710 tx=0x7f3360009260 comp rx=0 tx=0).stop 2024-03-19T01:08:36.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.451+0000 7f3379e00700 1 -- 172.21.15.69:0/1075506801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33741025c0 msgr2=0x7f337419ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:36.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.451+0000 7f3379e00700 1 --2- 172.21.15.69:0/1075506801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33741025c0 0x7f337419ad80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f336400e9a0 tx=0x7f336400ecb0 comp rx=0 tx=0).stop 2024-03-19T01:08:36.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.451+0000 7f3373400700 1 -- 172.21.15.69:0/1075506801 reap_dead start 2024-03-19T01:08:36.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.451+0000 7f3379e00700 1 -- 172.21.15.69:0/1075506801 shutdown_connections 2024-03-19T01:08:36.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.451+0000 7f3379e00700 1 -- 172.21.15.69:0/1075506801 >> 172.21.15.69:0/1075506801 conn(0x7f3374079470 msgr2=0x7f33741001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:36.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.452+0000 7f3379e00700 1 -- 172.21.15.69:0/1075506801 shutdown_connections 2024-03-19T01:08:36.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:36.452+0000 7f3379e00700 1 -- 172.21.15.69:0/1075506801 wait complete. 2024-03-19T01:08:36.963 INFO:teuthology.orchestra.run.smithi069.stdout:343597383710 2024-03-19T01:08:36.964 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-19T01:08:37.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:36 smithi086 ceph-mon[66829]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-19T01:08:37.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:36 smithi069 ceph-mon[31727]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-19T01:08:37.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:36 smithi163 ceph-mon[67428]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-19T01:08:38.269 INFO:teuthology.orchestra.run.smithi069.stdout:77309411409 2024-03-19T01:08:38.269 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-19T01:08:38.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.925+0000 7f650be00700 1 -- 172.21.15.69:0/3942411482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f650c1043e0 msgr2=0x7f650c104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:38.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.925+0000 7f650be00700 1 --2- 172.21.15.69:0/3942411482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f650c1043e0 0x7f650c104860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f64f80098f0 tx=0x7f64f8009c00 comp rx=0 tx=0).stop 2024-03-19T01:08:38.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.925+0000 7f650be00700 1 -- 172.21.15.69:0/3942411482 shutdown_connections 2024-03-19T01:08:38.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.925+0000 7f650be00700 1 --2- 172.21.15.69:0/3942411482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f650c104da0 0x7f650c1112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:38.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.925+0000 7f650be00700 1 --2- 172.21.15.69:0/3942411482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650c10a470 0x7f650c10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:38.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.925+0000 7f650be00700 1 --2- 172.21.15.69:0/3942411482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f650c1043e0 0x7f650c104860 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:38.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.925+0000 7f650be00700 1 -- 172.21.15.69:0/3942411482 >> 172.21.15.69:0/3942411482 conn(0x7f650c0ffef0 msgr2=0x7f650c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:38.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.925+0000 7f650be00700 1 -- 172.21.15.69:0/3942411482 shutdown_connections 2024-03-19T01:08:38.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.925+0000 7f650be00700 1 -- 172.21.15.69:0/3942411482 wait complete. 2024-03-19T01:08:38.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.927+0000 7f650be00700 1 Processor -- start 2024-03-19T01:08:38.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.927+0000 7f650be00700 1 -- start start 2024-03-19T01:08:38.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.927+0000 7f650be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650c1043e0 0x7f650c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:38.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.927+0000 7f650be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f650c104da0 0x7f650c1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:38.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650be00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f650c10a470 0x7f650c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:38.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650be00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f650c114720 con 0x7f650c104da0 2024-03-19T01:08:38.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f650c1145a0 con 0x7f650c1043e0 2024-03-19T01:08:38.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650c1043e0 0x7f650c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:38.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650c1043e0 0x7f650c1a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57396/0 (socket says 172.21.15.69:57396) 2024-03-19T01:08:38.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650aa00700 1 -- 172.21.15.69:0/2130998025 learned_addr learned my addr 172.21.15.69:0/2130998025 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:38.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f650c1148a0 con 0x7f650c10a470 2024-03-19T01:08:38.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650a000700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f650c104da0 0x7f650c1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:38.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650aa00700 1 -- 172.21.15.69:0/2130998025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f650c10a470 msgr2=0x7f650c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:38.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650aa00700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f650c10a470 0x7f650c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650aa00700 1 -- 172.21.15.69:0/2130998025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f650c104da0 msgr2=0x7f650c1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650aa00700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f650c104da0 0x7f650c1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.928+0000 7f650aa00700 1 -- 172.21.15.69:0/2130998025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c19b1c0 con 0x7f650c1043e0 2024-03-19T01:08:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.929+0000 7f650aa00700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650c1043e0 0x7f650c1a08a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00e9c0 tx=0x7f64fc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.929+0000 7f6503400700 1 -- 172.21.15.69:0/2130998025 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00cb50 con 0x7f650c1043e0 2024-03-19T01:08:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.929+0000 7f6503400700 1 -- 172.21.15.69:0/2130998025 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64fc004d10 con 0x7f650c1043e0 2024-03-19T01:08:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.929+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650c19b4b0 con 0x7f650c1043e0 2024-03-19T01:08:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.929+0000 7f6503400700 1 -- 172.21.15.69:0/2130998025 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc005680 con 0x7f650c1043e0 2024-03-19T01:08:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.929+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650c1a7770 con 0x7f650c1043e0 2024-03-19T01:08:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.931+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f64ec001030 con 0x7f650c1043e0 2024-03-19T01:08:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.934+0000 7f6503400700 1 -- 172.21.15.69:0/2130998025 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f64fc004830 con 0x7f650c1043e0 2024-03-19T01:08:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.935+0000 7f6503400700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f64f00a29a0 0x7f64f00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.936+0000 7f6503400700 1 -- 172.21.15.69:0/2130998025 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f64fc0cef00 con 0x7f650c1043e0 2024-03-19T01:08:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.936+0000 7f6503400700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778] conn(0x7f64f00adb00 0x7f64f00affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.936+0000 7f6503400700 1 -- 172.21.15.69:0/2130998025 --> [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64fc019e50 con 0x7f64f00adb00 2024-03-19T01:08:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.936+0000 7f6503400700 1 -- 172.21.15.69:0/2130998025 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f64fc0d1520 con 0x7f650c1043e0 2024-03-19T01:08:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.936+0000 7f650a000700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f64f00a29a0 0x7f64f00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.936+0000 7f650b400700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778] conn(0x7f64f00adb00 0x7f64f00affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.937+0000 7f650b400700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778] conn(0x7f64f00adb00 0x7f64f00affa0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.937+0000 7f650a000700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f64f00a29a0 0x7f64f00a4e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009810 tx=0x7f64f8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:38.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.938+0000 7f6503400700 1 -- 172.21.15.69:0/2130998025 <== osd.8 v2:172.21.15.163:6816/171535778 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f64fc019e50 con 0x7f64f00adb00 2024-03-19T01:08:38.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.956+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 --> [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f64ec002e60 con 0x7f64f00adb00 2024-03-19T01:08:38.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.956+0000 7f6503400700 1 -- 172.21.15.69:0/2130998025 <== osd.8 v2:172.21.15.163:6816/171535778 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f64ec002e60 con 0x7f64f00adb00 2024-03-19T01:08:38.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.957+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 >> [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778] conn(0x7f64f00adb00 msgr2=0x7f64f00affa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:38.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.957+0000 7f650be00700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778] conn(0x7f64f00adb00 0x7f64f00affa0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:38.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.957+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f64f00a29a0 msgr2=0x7f64f00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:38.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.957+0000 7f650be00700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f64f00a29a0 0x7f64f00a4e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009810 tx=0x7f64f8019040 comp rx=0 tx=0).stop 2024-03-19T01:08:38.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.957+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650c1043e0 msgr2=0x7f650c1a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:38.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.957+0000 7f650be00700 1 --2- 172.21.15.69:0/2130998025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650c1043e0 0x7f650c1a08a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00e9c0 tx=0x7f64fc00ecd0 comp rx=0 tx=0).stop 2024-03-19T01:08:38.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.957+0000 7f650b400700 1 -- 172.21.15.69:0/2130998025 reap_dead start 2024-03-19T01:08:38.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.957+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 shutdown_connections 2024-03-19T01:08:38.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.957+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 >> 172.21.15.69:0/2130998025 conn(0x7f650c0ffef0 msgr2=0x7f650c100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:38.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.958+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 shutdown_connections 2024-03-19T01:08:38.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:38.958+0000 7f650be00700 1 -- 172.21.15.69:0/2130998025 wait complete. 2024-03-19T01:08:39.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:38 smithi069 ceph-mon[31727]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:08:39.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.039+0000 7f48bfe00700 1 -- 172.21.15.69:0/747715512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c0102450 msgr2=0x7f48c0102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:39.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.039+0000 7f48bfe00700 1 --2- 172.21.15.69:0/747715512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c0102450 0x7f48c0102830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009990 tx=0x7f48b0009ca0 comp rx=0 tx=0).stop 2024-03-19T01:08:39.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.039+0000 7f48bfe00700 1 -- 172.21.15.69:0/747715512 shutdown_connections 2024-03-19T01:08:39.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.039+0000 7f48bfe00700 1 --2- 172.21.15.69:0/747715512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48c0107530 0x7f48c010f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:39.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.039+0000 7f48bfe00700 1 --2- 172.21.15.69:0/747715512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c0102450 0x7f48c0102830 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:39.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.039+0000 7f48bfe00700 1 --2- 172.21.15.69:0/747715512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c0102e00 0x7f48c0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:39.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.039+0000 7f48bfe00700 1 -- 172.21.15.69:0/747715512 >> 172.21.15.69:0/747715512 conn(0x7f48c00fdce0 msgr2=0x7f48c0100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:39.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.040+0000 7f48bfe00700 1 -- 172.21.15.69:0/747715512 shutdown_connections 2024-03-19T01:08:39.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.040+0000 7f48bfe00700 1 -- 172.21.15.69:0/747715512 wait complete. 2024-03-19T01:08:39.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.041+0000 7f48bfe00700 1 Processor -- start 2024-03-19T01:08:39.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.041+0000 7f48bfe00700 1 -- start start 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.042+0000 7f48bfe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c0102450 0x7f48c019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.042+0000 7f48bfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48c0102e00 0x7f48c019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.042+0000 7f48bfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c0107530 0x7f48c0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.042+0000 7f48bfe00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f48c0112510 con 0x7f48c0102e00 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.042+0000 7f48bfe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48c0112390 con 0x7f48c0107530 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.042+0000 7f48bfe00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f48c0112690 con 0x7f48c0102450 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.042+0000 7f48bea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c0102450 0x7f48c019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.042+0000 7f48bea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c0102450 0x7f48c019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37826/0 (socket says 172.21.15.69:37826) 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.042+0000 7f48bea00700 1 -- 172.21.15.69:0/2336823561 learned_addr learned my addr 172.21.15.69:0/2336823561 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.042+0000 7f48be000700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48c0102e00 0x7f48c019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48bea00700 1 -- 172.21.15.69:0/2336823561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c0107530 msgr2=0x7f48c0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:39.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48bf400700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c0107530 0x7f48c0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48bea00700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c0107530 0x7f48c0198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48bea00700 1 -- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48c0102e00 msgr2=0x7f48c019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48bea00700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48c0102e00 0x7f48c019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48bea00700 1 -- 172.21.15.69:0/2336823561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c0198ff0 con 0x7f48c0102450 2024-03-19T01:08:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48bf400700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c0107530 0x7f48c0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:08:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48bea00700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c0102450 0x7f48c019e640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f48b00095b0 tx=0x7f48b0005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48b7400700 1 -- 172.21.15.69:0/2336823561 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0026070 con 0x7f48c0102450 2024-03-19T01:08:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48b7400700 1 -- 172.21.15.69:0/2336823561 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48b000fb40 con 0x7f48c0102450 2024-03-19T01:08:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48bfe00700 1 -- 172.21.15.69:0/2336823561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c0199280 con 0x7f48c0102450 2024-03-19T01:08:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48bfe00700 1 -- 172.21.15.69:0/2336823561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c01a54a0 con 0x7f48c0102450 2024-03-19T01:08:39.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.043+0000 7f48b7400700 1 -- 172.21.15.69:0/2336823561 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0017650 con 0x7f48c0102450 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.045+0000 7f48bfe00700 1 -- 172.21.15.69:0/2336823561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48c0062a60 con 0x7f48c0102450 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.046+0000 7f48b7400700 1 -- 172.21.15.69:0/2336823561 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f48b0029070 con 0x7f48c0102450 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.049+0000 7f48b7400700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f48a40a29a0 0x7f48a40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.049+0000 7f48b7400700 1 -- 172.21.15.69:0/2336823561 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f48b00cfd00 con 0x7f48c0102450 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.049+0000 7f48be000700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f48a40a29a0 0x7f48a40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.049+0000 7f48b7400700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735] conn(0x7f48a40adb00 0x7f48a40affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.049+0000 7f48b7400700 1 -- 172.21.15.69:0/2336823561 --> [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48a40b0670 con 0x7f48a40adb00 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.049+0000 7f48b7400700 1 -- 172.21.15.69:0/2336823561 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f48b00d00b0 con 0x7f48c0102450 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.050+0000 7f48bf400700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735] conn(0x7f48a40adb00 0x7f48a40affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.050+0000 7f48be000700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f48a40a29a0 0x7f48a40a4e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48c0100380 tx=0x7f48ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.050+0000 7f48bf400700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735] conn(0x7f48a40adb00 0x7f48a40affa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:39.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.051+0000 7f48b7400700 1 -- 172.21.15.69:0/2336823561 <== osd.9 v2:172.21.15.69:6826/3178049735 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48a40b0670 con 0x7f48a40adb00 2024-03-19T01:08:39.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.070+0000 7f48bfe00700 1 -- 172.21.15.69:0/2336823561 --> [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f48c0062e90 con 0x7f48a40adb00 2024-03-19T01:08:39.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.071+0000 7f48b7400700 1 -- 172.21.15.69:0/2336823561 <== osd.9 v2:172.21.15.69:6826/3178049735 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f48c0062e90 con 0x7f48a40adb00 2024-03-19T01:08:39.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.071+0000 7f48bfe00700 1 -- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735] conn(0x7f48a40adb00 msgr2=0x7f48a40affa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:39.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.071+0000 7f48bfe00700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735] conn(0x7f48a40adb00 0x7f48a40affa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:39.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.072+0000 7f48bfe00700 1 -- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f48a40a29a0 msgr2=0x7f48a40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:39.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.072+0000 7f48bfe00700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f48a40a29a0 0x7f48a40a4e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48c0100380 tx=0x7f48ac009260 comp rx=0 tx=0).stop 2024-03-19T01:08:39.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.072+0000 7f48bfe00700 1 -- 172.21.15.69:0/2336823561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c0102450 msgr2=0x7f48c019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:39.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.072+0000 7f48bfe00700 1 --2- 172.21.15.69:0/2336823561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c0102450 0x7f48c019e640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f48b00095b0 tx=0x7f48b0005640 comp rx=0 tx=0).stop 2024-03-19T01:08:39.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.072+0000 7f48bf400700 1 -- 172.21.15.69:0/2336823561 reap_dead start 2024-03-19T01:08:39.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.072+0000 7f48bfe00700 1 -- 172.21.15.69:0/2336823561 shutdown_connections 2024-03-19T01:08:39.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.072+0000 7f48bfe00700 1 -- 172.21.15.69:0/2336823561 >> 172.21.15.69:0/2336823561 conn(0x7f48c00fdce0 msgr2=0x7f48c01066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:39.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.072+0000 7f48bfe00700 1 -- 172.21.15.69:0/2336823561 shutdown_connections 2024-03-19T01:08:39.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:39.072+0000 7f48bfe00700 1 -- 172.21.15.69:0/2336823561 wait complete. 2024-03-19T01:08:39.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:38 smithi086 ceph-mon[66829]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:08:39.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:38 smithi163 ceph-mon[67428]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:08:40.598 INFO:teuthology.orchestra.run.smithi069.stdout:163208757312 2024-03-19T01:08:40.599 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-19T01:08:40.943 INFO:teuthology.orchestra.run.smithi069.stdout:253403070511 2024-03-19T01:08:40.944 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-19T01:08:41.088 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:40 smithi069 ceph-mon[31727]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:08:41.115 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:40 smithi163 ceph-mon[67428]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:08:41.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:40 smithi086 ceph-mon[66829]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:08:41.501 INFO:teuthology.orchestra.run.smithi069.stdout:287762808873 2024-03-19T01:08:41.502 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-19T01:08:42.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.152+0000 7f9359c00700 1 -- 172.21.15.69:0/3450255175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354102450 msgr2=0x7f9354102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:42.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.152+0000 7f9359c00700 1 --2- 172.21.15.69:0/3450255175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354102450 0x7f9354102830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9344009990 tx=0x7f9344009ca0 comp rx=0 tx=0).stop 2024-03-19T01:08:42.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.152+0000 7f9359c00700 1 -- 172.21.15.69:0/3450255175 shutdown_connections 2024-03-19T01:08:42.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.153+0000 7f9359c00700 1 --2- 172.21.15.69:0/3450255175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9354107530 0x7f935410f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:42.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.153+0000 7f9359c00700 1 --2- 172.21.15.69:0/3450255175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354102450 0x7f9354102830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:42.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.153+0000 7f9359c00700 1 --2- 172.21.15.69:0/3450255175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9354102e00 0x7f9354106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:42.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.153+0000 7f9359c00700 1 -- 172.21.15.69:0/3450255175 >> 172.21.15.69:0/3450255175 conn(0x7f93540fdce0 msgr2=0x7f9354100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:42.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.153+0000 7f9359c00700 1 -- 172.21.15.69:0/3450255175 shutdown_connections 2024-03-19T01:08:42.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.153+0000 7f9359c00700 1 -- 172.21.15.69:0/3450255175 wait complete. 2024-03-19T01:08:42.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.154+0000 7f9359c00700 1 Processor -- start 2024-03-19T01:08:42.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.155+0000 7f9359c00700 1 -- start start 2024-03-19T01:08:42.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.155+0000 7f9359c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9354102450 0x7f9354074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:42.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.155+0000 7f9359c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354102e00 0x7f9354074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:42.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.155+0000 7f9359c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9354107530 0x7f9354075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:42.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.155+0000 7f9359c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9354112560 con 0x7f9354107530 2024-03-19T01:08:42.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.155+0000 7f9359c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93541123e0 con 0x7f9354102e00 2024-03-19T01:08:42.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.155+0000 7f9359c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f93541126e0 con 0x7f9354102450 2024-03-19T01:08:42.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.155+0000 7f9353e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9354102450 0x7f9354074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:42.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.155+0000 7f9353400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354102e00 0x7f9354074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:42.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.156+0000 7f9353e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9354102450 0x7f9354074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37846/0 (socket says 172.21.15.69:37846) 2024-03-19T01:08:42.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.156+0000 7f9353400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354102e00 0x7f9354074b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57434/0 (socket says 172.21.15.69:57434) 2024-03-19T01:08:42.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.156+0000 7f9353e00700 1 -- 172.21.15.69:0/1775702515 learned_addr learned my addr 172.21.15.69:0/1775702515 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:42.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.156+0000 7f9359200700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9354107530 0x7f9354075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.156+0000 7f9353e00700 1 -- 172.21.15.69:0/1775702515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354102e00 msgr2=0x7f9354074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.156+0000 7f9353e00700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354102e00 0x7f9354074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.156+0000 7f9353e00700 1 -- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9354107530 msgr2=0x7f9354075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.156+0000 7f9353e00700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9354107530 0x7f9354075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.156+0000 7f9353e00700 1 -- 172.21.15.69:0/1775702515 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93541a6f60 con 0x7f9354102450 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.156+0000 7f9353400700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9354102e00 0x7f9354074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.156+0000 7f9353e00700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9354102450 0x7f9354074610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9344005910 tx=0x7f93440047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.157+0000 7f9350c00700 1 -- 172.21.15.69:0/1775702515 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344027070 con 0x7f9354102450 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.157+0000 7f9350c00700 1 -- 172.21.15.69:0/1775702515 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f934400be70 con 0x7f9354102450 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.157+0000 7f9359c00700 1 -- 172.21.15.69:0/1775702515 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93541a71f0 con 0x7f9354102450 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.157+0000 7f9350c00700 1 -- 172.21.15.69:0/1775702515 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93440173f0 con 0x7f9354102450 2024-03-19T01:08:42.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.157+0000 7f9359c00700 1 -- 172.21.15.69:0/1775702515 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93541a76a0 con 0x7f9354102450 2024-03-19T01:08:42.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.158+0000 7f9359c00700 1 -- 172.21.15.69:0/1775702515 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9354062a20 con 0x7f9354102450 2024-03-19T01:08:42.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.160+0000 7f9350c00700 1 -- 172.21.15.69:0/1775702515 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f934402a070 con 0x7f9354102450 2024-03-19T01:08:42.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.161+0000 7f9350c00700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f93380a29a0 0x7f93380a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:42.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.161+0000 7f9350c00700 1 -- 172.21.15.69:0/1775702515 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f93440cfbd0 con 0x7f9354102450 2024-03-19T01:08:42.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.161+0000 7f9353400700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f93380a29a0 0x7f93380a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:42.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.161+0000 7f9350c00700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341] conn(0x7f93380adb00 0x7f93380affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:42.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.161+0000 7f9350c00700 1 -- 172.21.15.69:0/1775702515 --> [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9344001d30 con 0x7f93380adb00 2024-03-19T01:08:42.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.161+0000 7f9350c00700 1 -- 172.21.15.69:0/1775702515 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f93440cff80 con 0x7f9354102450 2024-03-19T01:08:42.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.162+0000 7f9359200700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341] conn(0x7f93380adb00 0x7f93380affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:42.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.162+0000 7f9353400700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f93380a29a0 0x7f93380a4e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9354108ae0 tx=0x7f934000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:42.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.162+0000 7f9359200700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341] conn(0x7f93380adb00 0x7f93380affa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:42.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.164+0000 7f9350c00700 1 -- 172.21.15.69:0/1775702515 <== osd.6 v2:172.21.15.69:6818/2819834341 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9344001d30 con 0x7f93380adb00 2024-03-19T01:08:42.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.182+0000 7f9359c00700 1 -- 172.21.15.69:0/1775702515 --> [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9354062ea0 con 0x7f93380adb00 2024-03-19T01:08:42.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.182+0000 7f9350c00700 1 -- 172.21.15.69:0/1775702515 <== osd.6 v2:172.21.15.69:6818/2819834341 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9354062ea0 con 0x7f93380adb00 2024-03-19T01:08:42.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.183+0000 7f9359c00700 1 -- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341] conn(0x7f93380adb00 msgr2=0x7f93380affa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:42.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.183+0000 7f9359c00700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341] conn(0x7f93380adb00 0x7f93380affa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:42.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.183+0000 7f9359c00700 1 -- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f93380a29a0 msgr2=0x7f93380a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:42.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.183+0000 7f9359c00700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f93380a29a0 0x7f93380a4e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9354108ae0 tx=0x7f934000b410 comp rx=0 tx=0).stop 2024-03-19T01:08:42.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.183+0000 7f9359c00700 1 -- 172.21.15.69:0/1775702515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9354102450 msgr2=0x7f9354074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.183+0000 7f9359c00700 1 --2- 172.21.15.69:0/1775702515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9354102450 0x7f9354074610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9344005910 tx=0x7f93440047b0 comp rx=0 tx=0).stop 2024-03-19T01:08:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.183+0000 7f9359200700 1 -- 172.21.15.69:0/1775702515 reap_dead start 2024-03-19T01:08:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.183+0000 7f9359c00700 1 -- 172.21.15.69:0/1775702515 shutdown_connections 2024-03-19T01:08:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.183+0000 7f9359c00700 1 -- 172.21.15.69:0/1775702515 >> 172.21.15.69:0/1775702515 conn(0x7f93540fdce0 msgr2=0x7f9354100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.183+0000 7f9359c00700 1 -- 172.21.15.69:0/1775702515 shutdown_connections 2024-03-19T01:08:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:42.183+0000 7f9359c00700 1 -- 172.21.15.69:0/1775702515 wait complete. 2024-03-19T01:08:42.526 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:43.147 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:42 smithi069 ceph-mon[31727]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-19T01:08:43.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:42 smithi086 ceph-mon[66829]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-19T01:08:43.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:42 smithi163 ceph-mon[67428]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-19T01:08:43.651 INFO:teuthology.orchestra.run.smithi069.stdout:197568495674 2024-03-19T01:08:43.652 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-19T01:08:44.599 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:45.184 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:44 smithi069 ceph-mon[31727]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:08:45.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:44 smithi086 ceph-mon[66829]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:08:45.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:44 smithi163 ceph-mon[67428]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:08:45.335 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:45.346 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:45.350 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:45.350 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:45.355 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:45.357 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:45.359 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:47.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:46 smithi069 ceph-mon[31727]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:08:47.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:46 smithi086 ceph-mon[66829]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:08:47.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:46 smithi163 ceph-mon[67428]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:08:47.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.633+0000 7f4f47a00700 1 -- 172.21.15.69:0/1351332588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f40107690 msgr2=0x7f4f4010f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:47.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.633+0000 7f4f47a00700 1 --2- 172.21.15.69:0/1351332588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f40107690 0x7f4f4010f260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009fc0 tx=0x7f4f3800b330 comp rx=0 tx=0).stop 2024-03-19T01:08:47.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.633+0000 7f4f47a00700 1 -- 172.21.15.69:0/1351332588 shutdown_connections 2024-03-19T01:08:47.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.633+0000 7f4f47a00700 1 --2- 172.21.15.69:0/1351332588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f40107690 0x7f4f4010f260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:47.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.633+0000 7f4f47a00700 1 --2- 172.21.15.69:0/1351332588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f401025c0 0x7f4f401029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:47.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.633+0000 7f4f47a00700 1 --2- 172.21.15.69:0/1351332588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f40102f70 0x7f4f40106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:47.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.633+0000 7f4f47a00700 1 -- 172.21.15.69:0/1351332588 >> 172.21.15.69:0/1351332588 conn(0x7f4f40079470 msgr2=0x7f4f40079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:47.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.633+0000 7f4f47a00700 1 -- 172.21.15.69:0/1351332588 shutdown_connections 2024-03-19T01:08:47.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.633+0000 7f4f47a00700 1 -- 172.21.15.69:0/1351332588 wait complete. 2024-03-19T01:08:47.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.634+0000 7f4f47a00700 1 Processor -- start 2024-03-19T01:08:47.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.635+0000 7f4f47a00700 1 -- start start 2024-03-19T01:08:47.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.635+0000 7f4f47a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f401025c0 0x7f4f4019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:47.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.635+0000 7f4f47a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f40102f70 0x7f4f4019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:47.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.635+0000 7f4f47a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40107690 0x7f4f4019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:47.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.635+0000 7f4f47a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4f401126e0 con 0x7f4f40102f70 2024-03-19T01:08:47.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.635+0000 7f4f47a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40112560 con 0x7f4f40107690 2024-03-19T01:08:47.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.635+0000 7f4f47a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40112860 con 0x7f4f401025c0 2024-03-19T01:08:47.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.635+0000 7f4f45000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f401025c0 0x7f4f4019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:47.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.635+0000 7f4f45000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f401025c0 0x7f4f4019ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48754/0 (socket says 172.21.15.69:48754) 2024-03-19T01:08:47.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.636+0000 7f4f45000700 1 -- 172.21.15.69:0/2082873557 learned_addr learned my addr 172.21.15.69:0/2082873557 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:47.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.636+0000 7f4f45a00700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40107690 0x7f4f4019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:47.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.636+0000 7f4f3fe00700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f40102f70 0x7f4f4019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:47.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.636+0000 7f4f45000700 1 -- 172.21.15.69:0/2082873557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40107690 msgr2=0x7f4f4019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:47.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.636+0000 7f4f45000700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40107690 0x7f4f4019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:47.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.636+0000 7f4f45000700 1 -- 172.21.15.69:0/2082873557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f40102f70 msgr2=0x7f4f4019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:47.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.636+0000 7f4f45000700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f40102f70 0x7f4f4019b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:47.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.636+0000 7f4f45000700 1 -- 172.21.15.69:0/2082873557 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4019fd40 con 0x7f4f401025c0 2024-03-19T01:08:47.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.636+0000 7f4f3fe00700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f40102f70 0x7f4f4019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:08:47.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.636+0000 7f4f45000700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f401025c0 0x7f4f4019ad30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400ec10 tx=0x7f4f3400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:47.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.636+0000 7f4f3d600700 1 -- 172.21.15.69:0/2082873557 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3400cd40 con 0x7f4f401025c0 2024-03-19T01:08:47.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.637+0000 7f4f3d600700 1 -- 172.21.15.69:0/2082873557 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f34004500 con 0x7f4f401025c0 2024-03-19T01:08:47.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.637+0000 7f4f3d600700 1 -- 172.21.15.69:0/2082873557 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34010690 con 0x7f4f401025c0 2024-03-19T01:08:47.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.637+0000 7f4f47a00700 1 -- 172.21.15.69:0/2082873557 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f401a0030 con 0x7f4f401025c0 2024-03-19T01:08:47.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.637+0000 7f4f47a00700 1 -- 172.21.15.69:0/2082873557 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f401a78b0 con 0x7f4f401025c0 2024-03-19T01:08:47.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.639+0000 7f4f47a00700 1 -- 172.21.15.69:0/2082873557 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f40068fd0 con 0x7f4f401025c0 2024-03-19T01:08:47.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.641+0000 7f4f3d600700 1 -- 172.21.15.69:0/2082873557 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f4f34003f10 con 0x7f4f401025c0 2024-03-19T01:08:47.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.642+0000 7f4f3d600700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4f280a29a0 0x7f4f280a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:47.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.642+0000 7f4f3d600700 1 -- 172.21.15.69:0/2082873557 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4f340cf2a0 con 0x7f4f401025c0 2024-03-19T01:08:47.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.643+0000 7f4f3fe00700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4f280a29a0 0x7f4f280a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:47.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.643+0000 7f4f3fe00700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4f280a29a0 0x7f4f280a4e60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f400765a0 tx=0x7f4f2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:47.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.646+0000 7f4f3d600700 1 -- 172.21.15.69:0/2082873557 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4f3409be20 con 0x7f4f401025c0 2024-03-19T01:08:47.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.923+0000 7f4f47a00700 1 -- 172.21.15.69:0/2082873557 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4f401a7a80 con 0x7f4f401025c0 2024-03-19T01:08:47.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.923+0000 7f4f3d600700 1 -- 172.21.15.69:0/2082873557 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4f3409b720 con 0x7f4f401025c0 2024-03-19T01:08:47.924 INFO:teuthology.orchestra.run.smithi069.stdout:163208757314 2024-03-19T01:08:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 -- 172.21.15.69:0/2082873557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4f280a29a0 msgr2=0x7f4f280a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4f280a29a0 0x7f4f280a4e60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f400765a0 tx=0x7f4f2c009260 comp rx=0 tx=0).stop 2024-03-19T01:08:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 -- 172.21.15.69:0/2082873557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f401025c0 msgr2=0x7f4f4019ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f401025c0 0x7f4f4019ad30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400ec10 tx=0x7f4f3400ef20 comp rx=0 tx=0).stop 2024-03-19T01:08:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 -- 172.21.15.69:0/2082873557 shutdown_connections 2024-03-19T01:08:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f4f280a29a0 0x7f4f280a4e60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f40107690 0x7f4f4019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f401025c0 0x7f4f4019ad30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 --2- 172.21.15.69:0/2082873557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f40102f70 0x7f4f4019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 -- 172.21.15.69:0/2082873557 >> 172.21.15.69:0/2082873557 conn(0x7f4f40079470 msgr2=0x7f4f4010d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 -- 172.21.15.69:0/2082873557 shutdown_connections 2024-03-19T01:08:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:47.927+0000 7f4f47a00700 1 -- 172.21.15.69:0/2082873557 wait complete. 2024-03-19T01:08:49.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:48 smithi069 ceph-mon[31727]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-19T01:08:49.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:48 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2082873557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-19T01:08:49.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:48 smithi086 ceph-mon[66829]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-19T01:08:49.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:48 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2082873557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-19T01:08:49.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.011+0000 7f0f34000700 1 -- 172.21.15.69:0/876689590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c102f00 msgr2=0x7f0f2c106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:49.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.011+0000 7f0f34000700 1 --2- 172.21.15.69:0/876689590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c102f00 0x7f0f2c106f50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009990 tx=0x7f0f18009ca0 comp rx=0 tx=0).stop 2024-03-19T01:08:49.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.011+0000 7f0f34000700 1 -- 172.21.15.69:0/876689590 shutdown_connections 2024-03-19T01:08:49.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.011+0000 7f0f34000700 1 --2- 172.21.15.69:0/876689590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f2c107630 0x7f0f2c10f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:49.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.011+0000 7f0f34000700 1 --2- 172.21.15.69:0/876689590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f2c102550 0x7f0f2c102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:49.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.011+0000 7f0f34000700 1 --2- 172.21.15.69:0/876689590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c102f00 0x7f0f2c106f50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:49.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.011+0000 7f0f34000700 1 -- 172.21.15.69:0/876689590 >> 172.21.15.69:0/876689590 conn(0x7f0f2c0fddc0 msgr2=0x7f0f2c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:49.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.012+0000 7f0f34000700 1 -- 172.21.15.69:0/876689590 shutdown_connections 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.012+0000 7f0f34000700 1 -- 172.21.15.69:0/876689590 wait complete. 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.013+0000 7f0f34000700 1 Processor -- start 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.013+0000 7f0f34000700 1 -- start start 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.014+0000 7f0f34000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c102550 0x7f0f2c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.014+0000 7f0f34000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f2c102f00 0x7f0f2c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.014+0000 7f0f34000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f2c107630 0x7f0f2c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.014+0000 7f0f34000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c112660 con 0x7f0f2c102f00 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.014+0000 7f0f34000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c1124e0 con 0x7f0f2c102550 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.014+0000 7f0f34000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c1127e0 con 0x7f0f2c107630 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.014+0000 7f0f31600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c102550 0x7f0f2c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.014+0000 7f0f30c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f2c102f00 0x7f0f2c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.014+0000 7f0f31600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c102550 0x7f0f2c19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:32976/0 (socket says 172.21.15.69:32976) 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.014+0000 7f0f31600700 1 -- 172.21.15.69:0/1252733828 learned_addr learned my addr 172.21.15.69:0/1252733828 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:49.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.014+0000 7f0f32000700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f2c107630 0x7f0f2c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.015+0000 7f0f31600700 1 -- 172.21.15.69:0/1252733828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f2c107630 msgr2=0x7f0f2c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.015+0000 7f0f31600700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f2c107630 0x7f0f2c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.015+0000 7f0f31600700 1 -- 172.21.15.69:0/1252733828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f2c102f00 msgr2=0x7f0f2c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.015+0000 7f0f31600700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f2c102f00 0x7f0f2c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.015+0000 7f0f31600700 1 -- 172.21.15.69:0/1252733828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c1990c0 con 0x7f0f2c102550 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.015+0000 7f0f31600700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c102550 0x7f0f2c19e7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000e9c0 tx=0x7f0f2000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.015+0000 7f0f2a000700 1 -- 172.21.15.69:0/1252733828 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2000cbd0 con 0x7f0f2c102550 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.015+0000 7f0f2a000700 1 -- 172.21.15.69:0/1252733828 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f2000ee60 con 0x7f0f2c102550 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.015+0000 7f0f34000700 1 -- 172.21.15.69:0/1252733828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c1993b0 con 0x7f0f2c102550 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.015+0000 7f0f2a000700 1 -- 172.21.15.69:0/1252733828 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f200056c0 con 0x7f0f2c102550 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.016+0000 7f0f34000700 1 -- 172.21.15.69:0/1252733828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c1a5670 con 0x7f0f2c102550 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.017+0000 7f0f34000700 1 -- 172.21.15.69:0/1252733828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2c068fd0 con 0x7f0f2c102550 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.018+0000 7f0f2a000700 1 -- 172.21.15.69:0/1252733828 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f0f20004830 con 0x7f0f2c102550 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.020+0000 7f0f2a000700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0f140a29a0 0x7f0f140a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:49.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.021+0000 7f0f2a000700 1 -- 172.21.15.69:0/1252733828 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0f200cf150 con 0x7f0f2c102550 2024-03-19T01:08:49.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.021+0000 7f0f30c00700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0f140a29a0 0x7f0f140a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:49.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.021+0000 7f0f30c00700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0f140a29a0 0x7f0f140a4e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c108c20 tx=0x7f0f1800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:49.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.023+0000 7f0f2a000700 1 -- 172.21.15.69:0/1252733828 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0f2000b2c0 con 0x7f0f2c102550 2024-03-19T01:08:49.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.306+0000 7f0f34000700 1 -- 172.21.15.69:0/1252733828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0f2c199e90 con 0x7f0f2c102550 2024-03-19T01:08:49.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.306+0000 7f0f2a000700 1 -- 172.21.15.69:0/1252733828 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0f2009bcd0 con 0x7f0f2c102550 2024-03-19T01:08:49.307 INFO:teuthology.orchestra.run.smithi069.stdout:107374182476 2024-03-19T01:08:49.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.309+0000 7f0f34000700 1 -- 172.21.15.69:0/1252733828 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0f140a29a0 msgr2=0x7f0f140a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:49.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.309+0000 7f0f34000700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0f140a29a0 0x7f0f140a4e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c108c20 tx=0x7f0f1800b260 comp rx=0 tx=0).stop 2024-03-19T01:08:49.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.309+0000 7f0f34000700 1 -- 172.21.15.69:0/1252733828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c102550 msgr2=0x7f0f2c19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:49.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.309+0000 7f0f34000700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c102550 0x7f0f2c19e7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000e9c0 tx=0x7f0f2000ecd0 comp rx=0 tx=0).stop 2024-03-19T01:08:49.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.309+0000 7f0f34000700 1 -- 172.21.15.69:0/1252733828 shutdown_connections 2024-03-19T01:08:49.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.309+0000 7f0f34000700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0f140a29a0 0x7f0f140a4e60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:49.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.309+0000 7f0f34000700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f2c107630 0x7f0f2c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:49.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.309+0000 7f0f34000700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c102550 0x7f0f2c19e7a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:49.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.309+0000 7f0f34000700 1 --2- 172.21.15.69:0/1252733828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f2c102f00 0x7f0f2c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:49.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.309+0000 7f0f34000700 1 -- 172.21.15.69:0/1252733828 >> 172.21.15.69:0/1252733828 conn(0x7f0f2c0fddc0 msgr2=0x7f0f2c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:49.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:48 smithi163 ceph-mon[67428]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-19T01:08:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:48 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2082873557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-19T01:08:49.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.310+0000 7f0f34000700 1 -- 172.21.15.69:0/1252733828 shutdown_connections 2024-03-19T01:08:49.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:49.310+0000 7f0f34000700 1 -- 172.21.15.69:0/1252733828 wait complete. 2024-03-19T01:08:50.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:49 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1252733828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-19T01:08:50.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:49 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1252733828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-19T01:08:50.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:49 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1252733828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-19T01:08:51.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:50 smithi069 ceph-mon[31727]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:08:51.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:50 smithi086 ceph-mon[66829]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:08:51.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:50 smithi163 ceph-mon[67428]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:08:51.365 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757312 got 163208757314 for osd.5 2024-03-19T01:08:51.366 DEBUG:teuthology.parallel:result is None 2024-03-19T01:08:51.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.606+0000 7fc556800700 1 -- 172.21.15.69:0/4229518586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc550102590 msgr2=0x7fc550102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:51.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.606+0000 7fc556800700 1 --2- 172.21.15.69:0/4229518586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc550102590 0x7fc550102970 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc540009920 tx=0x7fc540009c30 comp rx=0 tx=0).stop 2024-03-19T01:08:51.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.606+0000 7fc556800700 1 -- 172.21.15.69:0/4229518586 shutdown_connections 2024-03-19T01:08:51.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.606+0000 7fc556800700 1 --2- 172.21.15.69:0/4229518586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550107670 0x7fc55010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:51.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.606+0000 7fc556800700 1 --2- 172.21.15.69:0/4229518586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc550102590 0x7fc550102970 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:51.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.606+0000 7fc556800700 1 --2- 172.21.15.69:0/4229518586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc550102f40 0x7fc550106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:51.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.606+0000 7fc556800700 1 -- 172.21.15.69:0/4229518586 >> 172.21.15.69:0/4229518586 conn(0x7fc5500fdde0 msgr2=0x7fc550100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:51.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.606+0000 7fc556800700 1 -- 172.21.15.69:0/4229518586 shutdown_connections 2024-03-19T01:08:51.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.606+0000 7fc556800700 1 -- 172.21.15.69:0/4229518586 wait complete. 2024-03-19T01:08:51.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.607+0000 7fc556800700 1 Processor -- start 2024-03-19T01:08:51.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.607+0000 7fc556800700 1 -- start start 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc556800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc550102590 0x7fc550074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc556800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc550102f40 0x7fc550074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc556800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550107670 0x7fc550075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc556800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5501126a0 con 0x7fc550107670 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc556800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc550112520 con 0x7fc550102f40 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc556800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc550112820 con 0x7fc550102590 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc54f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc550102590 0x7fc550074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc54f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc550102590 0x7fc550074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48782/0 (socket says 172.21.15.69:48782) 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc54f400700 1 -- 172.21.15.69:0/992626149 learned_addr learned my addr 172.21.15.69:0/992626149 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc54ea00700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc550102f40 0x7fc550074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc54f400700 1 -- 172.21.15.69:0/992626149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc550102f40 msgr2=0x7fc550074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc54f400700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc550102f40 0x7fc550074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc54f400700 1 -- 172.21.15.69:0/992626149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550107670 msgr2=0x7fc550075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:51.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc54fe00700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550107670 0x7fc550075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc54f400700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550107670 0x7fc550075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.608+0000 7fc54f400700 1 -- 172.21.15.69:0/992626149 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5501a70d0 con 0x7fc550102590 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.609+0000 7fc54fe00700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550107670 0x7fc550075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.609+0000 7fc54f400700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc550102590 0x7fc550074610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc540000c00 tx=0x7fc54000fb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.609+0000 7fc547e00700 1 -- 172.21.15.69:0/992626149 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540027070 con 0x7fc550102590 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.609+0000 7fc547e00700 1 -- 172.21.15.69:0/992626149 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5400056d0 con 0x7fc550102590 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.609+0000 7fc556800700 1 -- 172.21.15.69:0/992626149 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5501a7360 con 0x7fc550102590 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.609+0000 7fc547e00700 1 -- 172.21.15.69:0/992626149 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5400049c0 con 0x7fc550102590 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.609+0000 7fc556800700 1 -- 172.21.15.69:0/992626149 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5501a77b0 con 0x7fc550102590 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.611+0000 7fc556800700 1 -- 172.21.15.69:0/992626149 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc550068fd0 con 0x7fc550102590 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.614+0000 7fc547e00700 1 -- 172.21.15.69:0/992626149 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fc54002a070 con 0x7fc550102590 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.615+0000 7fc547e00700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc5340a29a0 0x7fc5340a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:51.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.615+0000 7fc547e00700 1 -- 172.21.15.69:0/992626149 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc5400cf940 con 0x7fc550102590 2024-03-19T01:08:51.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.615+0000 7fc54ea00700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc5340a29a0 0x7fc5340a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:51.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.616+0000 7fc54ea00700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc5340a29a0 0x7fc5340a4e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc5501014b0 tx=0x7fc53c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:51.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.619+0000 7fc547e00700 1 -- 172.21.15.69:0/992626149 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc54005e740 con 0x7fc550102590 2024-03-19T01:08:51.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.871+0000 7fc556800700 1 -- 172.21.15.69:0/992626149 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc55010a8e0 con 0x7fc550102590 2024-03-19T01:08:51.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.872+0000 7fc547e00700 1 -- 172.21.15.69:0/992626149 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc540013030 con 0x7fc550102590 2024-03-19T01:08:51.873 INFO:teuthology.orchestra.run.smithi069.stdout:309237645350 2024-03-19T01:08:51.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.875+0000 7fc556800700 1 -- 172.21.15.69:0/992626149 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc5340a29a0 msgr2=0x7fc5340a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:51.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.875+0000 7fc556800700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc5340a29a0 0x7fc5340a4e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc5501014b0 tx=0x7fc53c009310 comp rx=0 tx=0).stop 2024-03-19T01:08:51.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.875+0000 7fc556800700 1 -- 172.21.15.69:0/992626149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc550102590 msgr2=0x7fc550074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:51.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.875+0000 7fc556800700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc550102590 0x7fc550074610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc540000c00 tx=0x7fc54000fb20 comp rx=0 tx=0).stop 2024-03-19T01:08:51.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.875+0000 7fc556800700 1 -- 172.21.15.69:0/992626149 shutdown_connections 2024-03-19T01:08:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.875+0000 7fc556800700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc5340a29a0 0x7fc5340a4e60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.875+0000 7fc556800700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550107670 0x7fc550075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.875+0000 7fc556800700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc550102590 0x7fc550074610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.875+0000 7fc556800700 1 --2- 172.21.15.69:0/992626149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc550102f40 0x7fc550074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.875+0000 7fc556800700 1 -- 172.21.15.69:0/992626149 >> 172.21.15.69:0/992626149 conn(0x7fc5500fdde0 msgr2=0x7fc5500ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.876+0000 7fc556800700 1 -- 172.21.15.69:0/992626149 shutdown_connections 2024-03-19T01:08:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:51.876+0000 7fc556800700 1 -- 172.21.15.69:0/992626149 wait complete. 2024-03-19T01:08:52.893 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182472 got 107374182476 for osd.3 2024-03-19T01:08:52.893 DEBUG:teuthology.parallel:result is None 2024-03-19T01:08:53.152 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:52 smithi069 ceph-mon[31727]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-19T01:08:53.152 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:52 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/992626149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-19T01:08:53.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:52 smithi086 ceph-mon[66829]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-19T01:08:53.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:52 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/992626149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-19T01:08:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:52 smithi163 ceph-mon[67428]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-19T01:08:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:52 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/992626149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-19T01:08:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:54.806 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:08:55.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:54 smithi069 ceph-mon[31727]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-19T01:08:55.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:54 smithi069 ceph-mon[31727]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-19T01:08:55.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:54 smithi086 ceph-mon[66829]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-19T01:08:55.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:54 smithi086 ceph-mon[66829]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-19T01:08:55.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:54 smithi163 ceph-mon[67428]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-19T01:08:55.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:54 smithi163 ceph-mon[67428]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-19T01:08:57.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:56 smithi069 ceph-mon[31727]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:08:57.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:56 smithi086 ceph-mon[66829]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:08:57.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:56 smithi163 ceph-mon[67428]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:08:58.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.031+0000 7fa8f1a00700 1 -- 172.21.15.69:0/1971446421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8ec069fc0 msgr2=0x7fa8ec06a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:58.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.031+0000 7fa8f1a00700 1 --2- 172.21.15.69:0/1971446421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8ec069fc0 0x7fa8ec06a420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009990 tx=0x7fa8d8009ca0 comp rx=0 tx=0).stop 2024-03-19T01:08:58.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.032+0000 7fa8f1a00700 1 -- 172.21.15.69:0/1971446421 shutdown_connections 2024-03-19T01:08:58.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.032+0000 7fa8f1a00700 1 --2- 172.21.15.69:0/1971446421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8ec06a960 0x7fa8ec10f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:58.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.032+0000 7fa8f1a00700 1 --2- 172.21.15.69:0/1971446421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8ec108360 0x7fa8ec108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:58.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.032+0000 7fa8f1a00700 1 --2- 172.21.15.69:0/1971446421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8ec069fc0 0x7fa8ec06a420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:58.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.032+0000 7fa8f1a00700 1 -- 172.21.15.69:0/1971446421 >> 172.21.15.69:0/1971446421 conn(0x7fa8ec078140 msgr2=0x7fa8ec078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:58.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.032+0000 7fa8f1a00700 1 -- 172.21.15.69:0/1971446421 shutdown_connections 2024-03-19T01:08:58.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.034+0000 7fa8f1a00700 1 -- 172.21.15.69:0/1971446421 wait complete. 2024-03-19T01:08:58.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.035+0000 7fa8f1a00700 1 Processor -- start 2024-03-19T01:08:58.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.036+0000 7fa8f1a00700 1 -- start start 2024-03-19T01:08:58.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.036+0000 7fa8f1a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8ec069fc0 0x7fa8ec19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:58.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.036+0000 7fa8f1a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8ec06a960 0x7fa8ec19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:58.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.036+0000 7fa8f1a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8ec108360 0x7fa8ec198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:58.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.036+0000 7fa8f1a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec112610 con 0x7fa8ec06a960 2024-03-19T01:08:58.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.036+0000 7fa8f1a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec112490 con 0x7fa8ec108360 2024-03-19T01:08:58.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.036+0000 7fa8f1a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec112790 con 0x7fa8ec069fc0 2024-03-19T01:08:58.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.037+0000 7fa8eb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8ec108360 0x7fa8ec198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:58.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.037+0000 7fa8ea000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8ec06a960 0x7fa8ec19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:58.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.037+0000 7fa8eaa00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8ec069fc0 0x7fa8ec19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:58.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.037+0000 7fa8eb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8ec108360 0x7fa8ec198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37780/0 (socket says 172.21.15.69:37780) 2024-03-19T01:08:58.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.037+0000 7fa8eb400700 1 -- 172.21.15.69:0/3904751435 learned_addr learned my addr 172.21.15.69:0/3904751435 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:08:58.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.037+0000 7fa8eaa00700 1 -- 172.21.15.69:0/3904751435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8ec108360 msgr2=0x7fa8ec198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:58.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.037+0000 7fa8eaa00700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8ec108360 0x7fa8ec198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:58.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.037+0000 7fa8eaa00700 1 -- 172.21.15.69:0/3904751435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8ec06a960 msgr2=0x7fa8ec19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:58.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.037+0000 7fa8eaa00700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8ec06a960 0x7fa8ec19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:58.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.037+0000 7fa8eaa00700 1 -- 172.21.15.69:0/3904751435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec199070 con 0x7fa8ec069fc0 2024-03-19T01:08:58.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.037+0000 7fa8eaa00700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8ec069fc0 0x7fa8ec19e750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00e9f0 tx=0x7fa8dc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:58.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.038+0000 7fa8e3400700 1 -- 172.21.15.69:0/3904751435 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc004220 con 0x7fa8ec069fc0 2024-03-19T01:08:58.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.038+0000 7fa8e3400700 1 -- 172.21.15.69:0/3904751435 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8dc004d10 con 0x7fa8ec069fc0 2024-03-19T01:08:58.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.038+0000 7fa8e3400700 1 -- 172.21.15.69:0/3904751435 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc005020 con 0x7fa8ec069fc0 2024-03-19T01:08:58.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.038+0000 7fa8f1a00700 1 -- 172.21.15.69:0/3904751435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ec199360 con 0x7fa8ec069fc0 2024-03-19T01:08:58.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.039+0000 7fa8f1a00700 1 -- 172.21.15.69:0/3904751435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ec101210 con 0x7fa8ec069fc0 2024-03-19T01:08:58.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.040+0000 7fa8f1a00700 1 -- 172.21.15.69:0/3904751435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8ec068fd0 con 0x7fa8ec069fc0 2024-03-19T01:08:58.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.043+0000 7fa8e3400700 1 -- 172.21.15.69:0/3904751435 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fa8dc004750 con 0x7fa8ec069fc0 2024-03-19T01:08:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.044+0000 7fa8e3400700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8d00a2910 0x7fa8d00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:08:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.044+0000 7fa8e3400700 1 -- 172.21.15.69:0/3904751435 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa8dc0cf1e0 con 0x7fa8ec069fc0 2024-03-19T01:08:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.045+0000 7fa8ea000700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8d00a2910 0x7fa8d00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:08:58.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.045+0000 7fa8ea000700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8d00a2910 0x7fa8d00a4dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0787b0 tx=0x7fa8d800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:08:58.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.048+0000 7fa8e3400700 1 -- 172.21.15.69:0/3904751435 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa8dc01c020 con 0x7fa8ec069fc0 2024-03-19T01:08:58.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.330+0000 7fa8f1a00700 1 -- 172.21.15.69:0/3904751435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa8ec199e40 con 0x7fa8ec069fc0 2024-03-19T01:08:58.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.331+0000 7fa8e3400700 1 -- 172.21.15.69:0/3904751435 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa8dc09bec0 con 0x7fa8ec069fc0 2024-03-19T01:08:58.333 INFO:teuthology.orchestra.run.smithi069.stdout:34359738463 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.333+0000 7fa8f1a00700 1 -- 172.21.15.69:0/3904751435 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8d00a2910 msgr2=0x7fa8d00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.333+0000 7fa8f1a00700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8d00a2910 0x7fa8d00a4dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0787b0 tx=0x7fa8d800b260 comp rx=0 tx=0).stop 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.333+0000 7fa8f1a00700 1 -- 172.21.15.69:0/3904751435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8ec069fc0 msgr2=0x7fa8ec19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.333+0000 7fa8f1a00700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8ec069fc0 0x7fa8ec19e750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00e9f0 tx=0x7fa8dc00ed00 comp rx=0 tx=0).stop 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.334+0000 7fa8f1a00700 1 -- 172.21.15.69:0/3904751435 shutdown_connections 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.334+0000 7fa8f1a00700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa8d00a2910 0x7fa8d00a4dd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.334+0000 7fa8f1a00700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8ec108360 0x7fa8ec198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.334+0000 7fa8f1a00700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8ec069fc0 0x7fa8ec19e750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.334+0000 7fa8f1a00700 1 --2- 172.21.15.69:0/3904751435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8ec06a960 0x7fa8ec19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.334+0000 7fa8f1a00700 1 -- 172.21.15.69:0/3904751435 >> 172.21.15.69:0/3904751435 conn(0x7fa8ec078140 msgr2=0x7fa8ec0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.334+0000 7fa8f1a00700 1 -- 172.21.15.69:0/3904751435 shutdown_connections 2024-03-19T01:08:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:08:58.334+0000 7fa8f1a00700 1 -- 172.21.15.69:0/3904751435 wait complete. 2024-03-19T01:08:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:58 smithi069 ceph-mon[31727]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-19T01:08:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:08:58 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3904751435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-19T01:08:59.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:58 smithi086 ceph-mon[66829]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-19T01:08:59.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:08:58 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3904751435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-19T01:08:59.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:58 smithi163 ceph-mon[67428]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-19T01:08:59.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:08:58 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3904751435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-19T01:08:59.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645344 got 309237645350 for osd.10 2024-03-19T01:08:59.463 DEBUG:teuthology.parallel:result is None 2024-03-19T01:09:00.272 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:09:01.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:00 smithi069 ceph-mon[31727]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-19T01:09:01.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:00 smithi086 ceph-mon[66829]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-19T01:09:01.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.027+0000 7fc623400700 1 -- 172.21.15.69:0/3832222363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c1025c0 msgr2=0x7fc61c1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:01.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.027+0000 7fc623400700 1 --2- 172.21.15.69:0/3832222363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c1025c0 0x7fc61c1029a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6100099c0 tx=0x7fc610009cd0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.027+0000 7fc623400700 1 -- 172.21.15.69:0/3832222363 shutdown_connections 2024-03-19T01:09:01.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.027+0000 7fc623400700 1 --2- 172.21.15.69:0/3832222363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61c107690 0x7fc61c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.027+0000 7fc623400700 1 --2- 172.21.15.69:0/3832222363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c1025c0 0x7fc61c1029a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.027+0000 7fc623400700 1 --2- 172.21.15.69:0/3832222363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc61c102f70 0x7fc61c106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.027+0000 7fc623400700 1 -- 172.21.15.69:0/3832222363 >> 172.21.15.69:0/3832222363 conn(0x7fc61c079470 msgr2=0x7fc61c079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:01.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.027+0000 7fc623400700 1 -- 172.21.15.69:0/3832222363 shutdown_connections 2024-03-19T01:09:01.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.027+0000 7fc623400700 1 -- 172.21.15.69:0/3832222363 wait complete. 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.029+0000 7fc623400700 1 Processor -- start 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.029+0000 7fc623400700 1 -- start start 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc623400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c1025c0 0x7fc61c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc623400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc61c102f70 0x7fc61c19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc623400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61c107690 0x7fc61c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc623400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c1126f0 con 0x7fc61c102f70 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc623400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c112570 con 0x7fc61c107690 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc623400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c112870 con 0x7fc61c1025c0 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc620a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c1025c0 0x7fc61c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc620a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c1025c0 0x7fc61c19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60932/0 (socket says 172.21.15.69:60932) 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc620a00700 1 -- 172.21.15.69:0/2256598621 learned_addr learned my addr 172.21.15.69:0/2256598621 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc621400700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61c107690 0x7fc61c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:01.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc620a00700 1 -- 172.21.15.69:0/2256598621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61c107690 msgr2=0x7fc61c19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc620a00700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61c107690 0x7fc61c19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc620a00700 1 -- 172.21.15.69:0/2256598621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc61c102f70 msgr2=0x7fc61c19b290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc620a00700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc61c102f70 0x7fc61c19b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.030+0000 7fc620a00700 1 -- 172.21.15.69:0/2256598621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c19fd60 con 0x7fc61c1025c0 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.031+0000 7fc620a00700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c1025c0 0x7fc61c19ad50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc610006010 tx=0x7fc610003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.031+0000 7fc619600700 1 -- 172.21.15.69:0/2256598621 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100050c0 con 0x7fc61c1025c0 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.031+0000 7fc619600700 1 -- 172.21.15.69:0/2256598621 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61000f460 con 0x7fc61c1025c0 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.031+0000 7fc619600700 1 -- 172.21.15.69:0/2256598621 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610017650 con 0x7fc61c1025c0 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.031+0000 7fc623400700 1 -- 172.21.15.69:0/2256598621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61c19fff0 con 0x7fc61c1025c0 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.031+0000 7fc623400700 1 -- 172.21.15.69:0/2256598621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61c1a78d0 con 0x7fc61c1025c0 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.032+0000 7fc623400700 1 -- 172.21.15.69:0/2256598621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61c103a90 con 0x7fc61c1025c0 2024-03-19T01:09:01.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.034+0000 7fc619600700 1 -- 172.21.15.69:0/2256598621 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fc61000f610 con 0x7fc61c1025c0 2024-03-19T01:09:01.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.035+0000 7fc619600700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6040a2890 0x7fc6040a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:01.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.036+0000 7fc619600700 1 -- 172.21.15.69:0/2256598621 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc6100cfbb0 con 0x7fc61c1025c0 2024-03-19T01:09:01.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.036+0000 7fc61be00700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6040a2890 0x7fc6040a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:01.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.036+0000 7fc61be00700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6040a2890 0x7fc6040a4d50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0763a0 tx=0x7fc608009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:01.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.039+0000 7fc619600700 1 -- 172.21.15.69:0/2256598621 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc610013070 con 0x7fc61c1025c0 2024-03-19T01:09:01.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:00 smithi163 ceph-mon[67428]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-19T01:09:01.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.349+0000 7fc623400700 1 -- 172.21.15.69:0/2256598621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc61c19c0b0 con 0x7fc61c1025c0 2024-03-19T01:09:01.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.350+0000 7fc619600700 1 -- 172.21.15.69:0/2256598621 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc610020370 con 0x7fc61c1025c0 2024-03-19T01:09:01.351 INFO:teuthology.orchestra.run.smithi069.stdout:141733920843 2024-03-19T01:09:01.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.353+0000 7fc623400700 1 -- 172.21.15.69:0/2256598621 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6040a2890 msgr2=0x7fc6040a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:01.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.353+0000 7fc623400700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6040a2890 0x7fc6040a4d50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0763a0 tx=0x7fc608009260 comp rx=0 tx=0).stop 2024-03-19T01:09:01.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.353+0000 7fc623400700 1 -- 172.21.15.69:0/2256598621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c1025c0 msgr2=0x7fc61c19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:01.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.353+0000 7fc623400700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c1025c0 0x7fc61c19ad50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc610006010 tx=0x7fc610003a90 comp rx=0 tx=0).stop 2024-03-19T01:09:01.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.353+0000 7fc623400700 1 -- 172.21.15.69:0/2256598621 shutdown_connections 2024-03-19T01:09:01.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.354+0000 7fc623400700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc6040a2890 0x7fc6040a4d50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.354+0000 7fc623400700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61c107690 0x7fc61c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.354+0000 7fc623400700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c1025c0 0x7fc61c19ad50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.354+0000 7fc623400700 1 --2- 172.21.15.69:0/2256598621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc61c102f70 0x7fc61c19b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.354+0000 7fc623400700 1 -- 172.21.15.69:0/2256598621 >> 172.21.15.69:0/2256598621 conn(0x7fc61c079470 msgr2=0x7fc61c075690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:01.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.354+0000 7fc623400700 1 -- 172.21.15.69:0/2256598621 shutdown_connections 2024-03-19T01:09:01.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.354+0000 7fc623400700 1 -- 172.21.15.69:0/2256598621 wait complete. 2024-03-19T01:09:01.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.597+0000 7f7646800700 1 -- 172.21.15.69:0/333862922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7640107690 msgr2=0x7f764010f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:01.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.597+0000 7f7646800700 1 --2- 172.21.15.69:0/333862922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7640107690 0x7f764010f260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7638009f90 tx=0x7f763800b330 comp rx=0 tx=0).stop 2024-03-19T01:09:01.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.597+0000 7f7646800700 1 -- 172.21.15.69:0/333862922 shutdown_connections 2024-03-19T01:09:01.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.597+0000 7f7646800700 1 --2- 172.21.15.69:0/333862922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7640107690 0x7f764010f260 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.597+0000 7f7646800700 1 --2- 172.21.15.69:0/333862922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76401025c0 0x7f76401029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.597+0000 7f7646800700 1 --2- 172.21.15.69:0/333862922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7640102f70 0x7f7640106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.597+0000 7f7646800700 1 -- 172.21.15.69:0/333862922 >> 172.21.15.69:0/333862922 conn(0x7f7640079470 msgr2=0x7f7640079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:01.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.597+0000 7f7646800700 1 -- 172.21.15.69:0/333862922 shutdown_connections 2024-03-19T01:09:01.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.597+0000 7f7646800700 1 -- 172.21.15.69:0/333862922 wait complete. 2024-03-19T01:09:01.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.599+0000 7f7646800700 1 Processor -- start 2024-03-19T01:09:01.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.599+0000 7f7646800700 1 -- start start 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.599+0000 7f7646800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76401025c0 0x7f7640074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.599+0000 7f7646800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7640102f70 0x7f7640074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.599+0000 7f7646800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7640107690 0x7f7640075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.599+0000 7f7646800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f76401126e0 con 0x7f7640102f70 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.599+0000 7f7646800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7640112560 con 0x7f7640107690 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.599+0000 7f7646800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7640112860 con 0x7f76401025c0 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.600+0000 7f763f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76401025c0 0x7f7640074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.600+0000 7f763f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76401025c0 0x7f7640074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60948/0 (socket says 172.21.15.69:60948) 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.600+0000 7f763f400700 1 -- 172.21.15.69:0/2017878046 learned_addr learned my addr 172.21.15.69:0/2017878046 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.600+0000 7f763ea00700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7640102f70 0x7f7640074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.600+0000 7f763fe00700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7640107690 0x7f7640075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.600+0000 7f763f400700 1 -- 172.21.15.69:0/2017878046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7640107690 msgr2=0x7f7640075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:01.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.600+0000 7f763f400700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7640107690 0x7f7640075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.600+0000 7f763f400700 1 -- 172.21.15.69:0/2017878046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7640102f70 msgr2=0x7f7640074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:01.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.600+0000 7f763f400700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7640102f70 0x7f7640074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.600+0000 7f763f400700 1 -- 172.21.15.69:0/2017878046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76401a7090 con 0x7f76401025c0 2024-03-19T01:09:01.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.600+0000 7f763f400700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76401025c0 0x7f7640074610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f763000e9a0 tx=0x7f763000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:01.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.601+0000 7f7637e00700 1 -- 172.21.15.69:0/2017878046 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763000ca10 con 0x7f76401025c0 2024-03-19T01:09:01.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.601+0000 7f7637e00700 1 -- 172.21.15.69:0/2017878046 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763000cbb0 con 0x7f76401025c0 2024-03-19T01:09:01.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.601+0000 7f7646800700 1 -- 172.21.15.69:0/2017878046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76401a7380 con 0x7f76401025c0 2024-03-19T01:09:01.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.601+0000 7f7637e00700 1 -- 172.21.15.69:0/2017878046 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630018680 con 0x7f76401025c0 2024-03-19T01:09:01.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.601+0000 7f7646800700 1 -- 172.21.15.69:0/2017878046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76401a78c0 con 0x7f76401025c0 2024-03-19T01:09:01.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.602+0000 7f7646800700 1 -- 172.21.15.69:0/2017878046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7640068fd0 con 0x7f76401025c0 2024-03-19T01:09:01.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.605+0000 7f7637e00700 1 -- 172.21.15.69:0/2017878046 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f7630010430 con 0x7f76401025c0 2024-03-19T01:09:01.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.607+0000 7f7637e00700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f76240a2890 0x7f76240a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:01.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.608+0000 7f7637e00700 1 -- 172.21.15.69:0/2017878046 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f76300cf060 con 0x7f76401025c0 2024-03-19T01:09:01.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.608+0000 7f763ea00700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f76240a2890 0x7f76240a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:01.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.608+0000 7f763ea00700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f76240a2890 0x7f76240a4d50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f762c009750 tx=0x7f762c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:01.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.610+0000 7f7637e00700 1 -- 172.21.15.69:0/2017878046 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f763009bd40 con 0x7f76401025c0 2024-03-19T01:09:01.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.858+0000 7f7646800700 1 -- 172.21.15.69:0/2017878046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f764002dbb0 con 0x7f76401025c0 2024-03-19T01:09:01.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.858+0000 7f7637e00700 1 -- 172.21.15.69:0/2017878046 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7630097030 con 0x7f76401025c0 2024-03-19T01:09:01.904 INFO:teuthology.orchestra.run.smithi069.stdout:343597383715 2024-03-19T01:09:01.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.861+0000 7f7646800700 1 -- 172.21.15.69:0/2017878046 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f76240a2890 msgr2=0x7f76240a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:01.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.861+0000 7f7646800700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f76240a2890 0x7f76240a4d50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f762c009750 tx=0x7f762c009310 comp rx=0 tx=0).stop 2024-03-19T01:09:01.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.861+0000 7f7646800700 1 -- 172.21.15.69:0/2017878046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76401025c0 msgr2=0x7f7640074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:01.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.861+0000 7f7646800700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76401025c0 0x7f7640074610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f763000e9a0 tx=0x7f763000ecb0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.861+0000 7f7646800700 1 -- 172.21.15.69:0/2017878046 shutdown_connections 2024-03-19T01:09:01.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.861+0000 7f7646800700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f76240a2890 0x7f76240a4d50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.861+0000 7f7646800700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7640107690 0x7f7640075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.861+0000 7f7646800700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76401025c0 0x7f7640074610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.861+0000 7f7646800700 1 --2- 172.21.15.69:0/2017878046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7640102f70 0x7f7640074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:01.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.861+0000 7f7646800700 1 -- 172.21.15.69:0/2017878046 >> 172.21.15.69:0/2017878046 conn(0x7f7640079470 msgr2=0x7f76401003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:01.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.862+0000 7f7646800700 1 -- 172.21.15.69:0/2017878046 shutdown_connections 2024-03-19T01:09:01.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:01.862+0000 7f7646800700 1 -- 172.21.15.69:0/2017878046 wait complete. 2024-03-19T01:09:02.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:01 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2256598621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-19T01:09:02.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:01 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2017878046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-19T01:09:02.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:01 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2256598621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-19T01:09:02.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:01 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2017878046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-19T01:09:02.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:01 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2256598621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-19T01:09:02.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:01 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2017878046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-19T01:09:03.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:02 smithi069 ceph-mon[31727]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:09:03.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:02 smithi086 ceph-mon[66829]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:09:03.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:02 smithi163 ceph-mon[67428]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:09:04.810 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383710 got 343597383715 for osd.11 2024-03-19T01:09:04.810 DEBUG:teuthology.parallel:result is None 2024-03-19T01:09:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:04 smithi069 ceph-mon[31727]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-19T01:09:05.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:04 smithi086 ceph-mon[66829]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-19T01:09:05.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:04 smithi163 ceph-mon[67428]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-19T01:09:05.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.385+0000 7fb9a6600700 1 -- 172.21.15.69:0/3607038712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a010edc0 msgr2=0x7fb9a01111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.385+0000 7fb9a6600700 1 --2- 172.21.15.69:0/3607038712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a010edc0 0x7fb9a01111b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb99800b0a0 tx=0x7fb99800b3b0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.385+0000 7fb9a6600700 1 -- 172.21.15.69:0/3607038712 shutdown_connections 2024-03-19T01:09:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.385+0000 7fb9a6600700 1 --2- 172.21.15.69:0/3607038712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a010edc0 0x7fb9a01111b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.385+0000 7fb9a6600700 1 --2- 172.21.15.69:0/3607038712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9a0069fc0 0x7fb9a006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.385+0000 7fb9a6600700 1 --2- 172.21.15.69:0/3607038712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a006a8e0 0x7fb9a010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.385+0000 7fb9a6600700 1 -- 172.21.15.69:0/3607038712 >> 172.21.15.69:0/3607038712 conn(0x7fb9a0078140 msgr2=0x7fb9a0078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.385+0000 7fb9a6600700 1 -- 172.21.15.69:0/3607038712 shutdown_connections 2024-03-19T01:09:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.386+0000 7fb9a6600700 1 -- 172.21.15.69:0/3607038712 wait complete. 2024-03-19T01:09:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.387+0000 7fb9a6600700 1 Processor -- start 2024-03-19T01:09:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.387+0000 7fb9a6600700 1 -- start start 2024-03-19T01:09:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.387+0000 7fb9a6600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a0069fc0 0x7fb9a01a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.388+0000 7fb9a6600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a006a8e0 0x7fb9a01a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.388+0000 7fb9a6600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9a010edc0 0x7fb9a019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.388+0000 7fb9a6600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0114810 con 0x7fb9a010edc0 2024-03-19T01:09:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.388+0000 7fb9a6600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0114690 con 0x7fb9a0069fc0 2024-03-19T01:09:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.388+0000 7fb9a6600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0114990 con 0x7fb9a006a8e0 2024-03-19T01:09:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.388+0000 7fb99fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9a010edc0 0x7fb9a019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.388+0000 7fb99ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a006a8e0 0x7fb9a01a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.388+0000 7fb99fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9a010edc0 0x7fb9a019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33212/0 (socket says 172.21.15.69:33212) 2024-03-19T01:09:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.388+0000 7fb99fe00700 1 -- 172.21.15.69:0/1753716403 learned_addr learned my addr 172.21.15.69:0/1753716403 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.388+0000 7fb99ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a006a8e0 0x7fb9a01a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60988/0 (socket says 172.21.15.69:60988) 2024-03-19T01:09:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.389+0000 7fb99f400700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a0069fc0 0x7fb9a01a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.389+0000 7fb99ea00700 1 -- 172.21.15.69:0/1753716403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a0069fc0 msgr2=0x7fb9a01a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.389+0000 7fb99ea00700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a0069fc0 0x7fb9a01a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.389+0000 7fb99ea00700 1 -- 172.21.15.69:0/1753716403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9a010edc0 msgr2=0x7fb9a019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.389+0000 7fb99ea00700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9a010edc0 0x7fb9a019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.389+0000 7fb99ea00700 1 -- 172.21.15.69:0/1753716403 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a019b230 con 0x7fb9a006a8e0 2024-03-19T01:09:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.389+0000 7fb99fe00700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9a010edc0 0x7fb9a019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.390+0000 7fb99ea00700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a006a8e0 0x7fb9a01a0e00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00e9f0 tx=0x7fb98c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.390+0000 7fb997e00700 1 -- 172.21.15.69:0/1753716403 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c00c9c0 con 0x7fb9a006a8e0 2024-03-19T01:09:05.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.390+0000 7fb997e00700 1 -- 172.21.15.69:0/1753716403 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb98c00cb60 con 0x7fb9a006a8e0 2024-03-19T01:09:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.390+0000 7fb997e00700 1 -- 172.21.15.69:0/1753716403 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c018680 con 0x7fb9a006a8e0 2024-03-19T01:09:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.390+0000 7fb9a6600700 1 -- 172.21.15.69:0/1753716403 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a019b520 con 0x7fb9a006a8e0 2024-03-19T01:09:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.390+0000 7fb9a6600700 1 -- 172.21.15.69:0/1753716403 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01a7790 con 0x7fb9a006a8e0 2024-03-19T01:09:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.392+0000 7fb9a6600700 1 -- 172.21.15.69:0/1753716403 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a0068fd0 con 0x7fb9a006a8e0 2024-03-19T01:09:05.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.395+0000 7fb997e00700 1 -- 172.21.15.69:0/1753716403 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb98c010430 con 0x7fb9a006a8e0 2024-03-19T01:09:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.396+0000 7fb997e00700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb9840a2910 0x7fb9840a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.397+0000 7fb997e00700 1 -- 172.21.15.69:0/1753716403 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb98c0cee80 con 0x7fb9a006a8e0 2024-03-19T01:09:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.397+0000 7fb99f400700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb9840a2910 0x7fb9840a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:05.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.398+0000 7fb99f400700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb9840a2910 0x7fb9840a4dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0078bb0 tx=0x7fb990009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:05.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.400+0000 7fb997e00700 1 -- 172.21.15.69:0/1753716403 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb98c09bb60 con 0x7fb9a006a8e0 2024-03-19T01:09:05.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.234+0000 7fd10c400700 1 -- 172.21.15.69:0/1141617768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104101bb0 msgr2=0x7fd10410e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:05.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.234+0000 7fd10c400700 1 --2- 172.21.15.69:0/1141617768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104101bb0 0x7fd10410e090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b0a0 tx=0x7fd0fc00b3b0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.234+0000 7fd10c400700 1 -- 172.21.15.69:0/1141617768 shutdown_connections 2024-03-19T01:09:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.234+0000 7fd10c400700 1 --2- 172.21.15.69:0/1141617768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104101bb0 0x7fd10410e090 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.234+0000 7fd10c400700 1 --2- 172.21.15.69:0/1141617768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd104107250 0x7fd104107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.234+0000 7fd10c400700 1 --2- 172.21.15.69:0/1141617768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd104101210 0x7fd104101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.234+0000 7fd10c400700 1 -- 172.21.15.69:0/1141617768 >> 172.21.15.69:0/1141617768 conn(0x7fd104078140 msgr2=0x7fd104078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.234+0000 7fd10c400700 1 -- 172.21.15.69:0/1141617768 shutdown_connections 2024-03-19T01:09:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.234+0000 7fd10c400700 1 -- 172.21.15.69:0/1141617768 wait complete. 2024-03-19T01:09:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.235+0000 7fd10c400700 1 Processor -- start 2024-03-19T01:09:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.236+0000 7fd10c400700 1 -- start start 2024-03-19T01:09:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.236+0000 7fd10c400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd104101210 0x7fd104102ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.236+0000 7fd10c400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd104101bb0 0x7fd104103400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.236+0000 7fd10c400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104107250 0x7fd104103940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.236+0000 7fd10c400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041114f0 con 0x7fd104101210 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.236+0000 7fd10c400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd104111370 con 0x7fd104107250 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.236+0000 7fd10c400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd104111670 con 0x7fd104101bb0 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.236+0000 7fd109a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd104101210 0x7fd104102ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.236+0000 7fd109000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd104101bb0 0x7fd104103400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.237+0000 7fd109a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd104101210 0x7fd104102ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33184/0 (socket says 172.21.15.69:33184) 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.237+0000 7fd109a00700 1 -- 172.21.15.69:0/3988223897 learned_addr learned my addr 172.21.15.69:0/3988223897 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.237+0000 7fd10a400700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104107250 0x7fd104103940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.237+0000 7fd10a400700 1 -- 172.21.15.69:0/3988223897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd104101bb0 msgr2=0x7fd104103400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.237+0000 7fd10a400700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd104101bb0 0x7fd104103400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.237+0000 7fd10a400700 1 -- 172.21.15.69:0/3988223897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd104101210 msgr2=0x7fd104102ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.237+0000 7fd10a400700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd104101210 0x7fd104102ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.237+0000 7fd10a400700 1 -- 172.21.15.69:0/3988223897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1041a6f60 con 0x7fd104107250 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.237+0000 7fd109000700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd104101bb0 0x7fd104103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.237+0000 7fd10a400700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104107250 0x7fd104103940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b070 tx=0x7fd0fc004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.238+0000 7fd102000700 1 -- 172.21.15.69:0/3988223897 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc021070 con 0x7fd104107250 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.238+0000 7fd102000700 1 -- 172.21.15.69:0/3988223897 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0fc007b80 con 0x7fd104107250 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.238+0000 7fd10c400700 1 -- 172.21.15.69:0/3988223897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd104198770 con 0x7fd104107250 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.238+0000 7fd10c400700 1 -- 172.21.15.69:0/3988223897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd104198d00 con 0x7fd104107250 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.238+0000 7fd102000700 1 -- 172.21.15.69:0/3988223897 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0052c0 con 0x7fd104107250 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.240+0000 7fd10c400700 1 -- 172.21.15.69:0/3988223897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e4005360 con 0x7fd104107250 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.242+0000 7fd102000700 1 -- 172.21.15.69:0/3988223897 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd0fc024070 con 0x7fd104107250 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.245+0000 7fd102000700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0ec0a2910 0x7fd0ec0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.245+0000 7fd109a00700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0ec0a2910 0x7fd0ec0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.246+0000 7fd102000700 1 -- 172.21.15.69:0/3988223897 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd0fc00f020 con 0x7fd104107250 2024-03-19T01:09:05.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.246+0000 7fd109a00700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0ec0a2910 0x7fd0ec0a4dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009ca0 tx=0x7fd0f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:05.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.254+0000 7fd102000700 1 -- 172.21.15.69:0/3988223897 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd0fc02b020 con 0x7fd104107250 2024-03-19T01:09:05.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.545+0000 7fd10c400700 1 -- 172.21.15.69:0/3988223897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd0e4005650 con 0x7fd104107250 2024-03-19T01:09:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.546+0000 7fd102000700 1 -- 172.21.15.69:0/3988223897 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd0fc09ce10 con 0x7fd104107250 2024-03-19T01:09:05.548 INFO:teuthology.orchestra.run.smithi069.stdout:55834574939 2024-03-19T01:09:05.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.549+0000 7fd10c400700 1 -- 172.21.15.69:0/3988223897 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0ec0a2910 msgr2=0x7fd0ec0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:05.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.550+0000 7fd10c400700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0ec0a2910 0x7fd0ec0a4dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009ca0 tx=0x7fd0f8009260 comp rx=0 tx=0).stop 2024-03-19T01:09:05.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.550+0000 7fd10c400700 1 -- 172.21.15.69:0/3988223897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104107250 msgr2=0x7fd104103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:05.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.550+0000 7fd10c400700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104107250 0x7fd104103940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b070 tx=0x7fd0fc004c80 comp rx=0 tx=0).stop 2024-03-19T01:09:05.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.550+0000 7fd10c400700 1 -- 172.21.15.69:0/3988223897 shutdown_connections 2024-03-19T01:09:05.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.550+0000 7fd10c400700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0ec0a2910 0x7fd0ec0a4dd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.550+0000 7fd10c400700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104107250 0x7fd104103940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.550+0000 7fd10c400700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd104101210 0x7fd104102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.550+0000 7fd10c400700 1 --2- 172.21.15.69:0/3988223897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd104101bb0 0x7fd104103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.550+0000 7fd10c400700 1 -- 172.21.15.69:0/3988223897 >> 172.21.15.69:0/3988223897 conn(0x7fd104078140 msgr2=0x7fd104100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:05.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.550+0000 7fd10c400700 1 -- 172.21.15.69:0/3988223897 shutdown_connections 2024-03-19T01:09:05.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.551+0000 7fd10c400700 1 -- 172.21.15.69:0/3988223897 wait complete. 2024-03-19T01:09:05.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.737+0000 7fb9a6600700 1 -- 172.21.15.69:0/1753716403 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb9a002dd70 con 0x7fb9a006a8e0 2024-03-19T01:09:05.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.737+0000 7fb997e00700 1 -- 172.21.15.69:0/1753716403 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb98c09b460 con 0x7fb9a006a8e0 2024-03-19T01:09:05.738 INFO:teuthology.orchestra.run.smithi069.stdout:253403070516 2024-03-19T01:09:05.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.739+0000 7fb9a6600700 1 -- 172.21.15.69:0/1753716403 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb9840a2910 msgr2=0x7fb9840a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:05.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.739+0000 7fb9a6600700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb9840a2910 0x7fb9840a4dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0078bb0 tx=0x7fb990009260 comp rx=0 tx=0).stop 2024-03-19T01:09:05.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.739+0000 7fb9a6600700 1 -- 172.21.15.69:0/1753716403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a006a8e0 msgr2=0x7fb9a01a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:05.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.739+0000 7fb9a6600700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a006a8e0 0x7fb9a01a0e00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00e9f0 tx=0x7fb98c00ed00 comp rx=0 tx=0).stop 2024-03-19T01:09:05.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.740+0000 7fb9a6600700 1 -- 172.21.15.69:0/1753716403 shutdown_connections 2024-03-19T01:09:05.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.740+0000 7fb9a6600700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb9840a2910 0x7fb9840a4dd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.740+0000 7fb9a6600700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9a010edc0 0x7fb9a019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.740+0000 7fb9a6600700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9a0069fc0 0x7fb9a01a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.740+0000 7fb9a6600700 1 --2- 172.21.15.69:0/1753716403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9a006a8e0 0x7fb9a01a0e00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:05.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.740+0000 7fb9a6600700 1 -- 172.21.15.69:0/1753716403 >> 172.21.15.69:0/1753716403 conn(0x7fb9a0078140 msgr2=0x7fb9a0110210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:05.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.740+0000 7fb9a6600700 1 -- 172.21.15.69:0/1753716403 shutdown_connections 2024-03-19T01:09:05.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:05.740+0000 7fb9a6600700 1 -- 172.21.15.69:0/1753716403 wait complete. 2024-03-19T01:09:06.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:09:06.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:05 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:09:06.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:05 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3988223897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-19T01:09:06.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:05 smithi086 ceph-mon[66829]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:09:06.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:05 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1753716403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-19T01:09:06.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:09:06.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:05 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:09:06.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:05 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3988223897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-19T01:09:06.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:05 smithi069 ceph-mon[31727]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:09:06.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:05 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1753716403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-19T01:09:06.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:09:06.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:05 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:09:06.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:05 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3988223897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-19T01:09:06.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:05 smithi163 ceph-mon[67428]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:09:06.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:05 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1753716403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-19T01:09:06.979 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574932 got 55834574939 for osd.1 2024-03-19T01:09:06.979 DEBUG:teuthology.parallel:result is None 2024-03-19T01:09:07.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:09:07.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:09:07.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:09:07.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:09:07.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:09:07.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:09:07.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:09:07.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:09:07.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:09:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.737+0000 7fdcde800700 1 -- 172.21.15.69:0/873559884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd81043e0 msgr2=0x7fdcd8104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.737+0000 7fdcde800700 1 --2- 172.21.15.69:0/873559884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd81043e0 0x7fdcd8104860 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009990 tx=0x7fdcc4009ca0 comp rx=0 tx=0).stop 2024-03-19T01:09:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.737+0000 7fdcde800700 1 -- 172.21.15.69:0/873559884 shutdown_connections 2024-03-19T01:09:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.737+0000 7fdcde800700 1 --2- 172.21.15.69:0/873559884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcd8104da0 0x7fdcd81112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.737+0000 7fdcde800700 1 --2- 172.21.15.69:0/873559884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd810a470 0x7fdcd810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.737+0000 7fdcde800700 1 --2- 172.21.15.69:0/873559884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd81043e0 0x7fdcd8104860 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.737+0000 7fdcde800700 1 -- 172.21.15.69:0/873559884 >> 172.21.15.69:0/873559884 conn(0x7fdcd80ffef0 msgr2=0x7fdcd8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.738+0000 7fdcde800700 1 -- 172.21.15.69:0/873559884 shutdown_connections 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.738+0000 7fdcde800700 1 -- 172.21.15.69:0/873559884 wait complete. 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.739+0000 7fdcde800700 1 Processor -- start 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.739+0000 7fdcde800700 1 -- start start 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.740+0000 7fdcde800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcd81043e0 0x7fdcd81a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.740+0000 7fdcde800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd8104da0 0x7fdcd81a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.740+0000 7fdcde800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd810a470 0x7fdcd819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.740+0000 7fdcde800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd8114740 con 0x7fdcd81043e0 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.740+0000 7fdcde800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd81145c0 con 0x7fdcd810a470 2024-03-19T01:09:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.740+0000 7fdcde800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd81148c0 con 0x7fdcd8104da0 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.740+0000 7fdcdde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd810a470 0x7fdcd819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.740+0000 7fdcdde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd810a470 0x7fdcd819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33036/0 (socket says 172.21.15.69:33036) 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.740+0000 7fdcdca00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd8104da0 0x7fdcd81a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.740+0000 7fdcdde00700 1 -- 172.21.15.69:0/1827230553 learned_addr learned my addr 172.21.15.69:0/1827230553 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.740+0000 7fdcdd400700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcd81043e0 0x7fdcd81a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcdde00700 1 -- 172.21.15.69:0/1827230553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd8104da0 msgr2=0x7fdcd81a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcdde00700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd8104da0 0x7fdcd81a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcdde00700 1 -- 172.21.15.69:0/1827230553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcd81043e0 msgr2=0x7fdcd81a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcdde00700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcd81043e0 0x7fdcd81a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcdde00700 1 -- 172.21.15.69:0/1827230553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd819b180 con 0x7fdcd810a470 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcdca00700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd8104da0 0x7fdcd81a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcdd400700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcd81043e0 0x7fdcd81a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:07.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcdde00700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd810a470 0x7fdcd819aa20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0007ee0 tx=0x7fdcd000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcd6000700 1 -- 172.21.15.69:0/1827230553 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd000dbd0 con 0x7fdcd810a470 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcd6000700 1 -- 172.21.15.69:0/1827230553 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd000dd70 con 0x7fdcd810a470 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcde800700 1 -- 172.21.15.69:0/1827230553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd819b470 con 0x7fdcd810a470 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.741+0000 7fdcd6000700 1 -- 172.21.15.69:0/1827230553 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0005a60 con 0x7fdcd810a470 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.742+0000 7fdcde800700 1 -- 172.21.15.69:0/1827230553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd81a7730 con 0x7fdcd810a470 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.743+0000 7fdcde800700 1 -- 172.21.15.69:0/1827230553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd8068fd0 con 0x7fdcd810a470 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.745+0000 7fdcd6000700 1 -- 172.21.15.69:0/1827230553 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fdcd001a020 con 0x7fdcd810a470 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.746+0000 7fdcd6000700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdcc00a2890 0x7fdcc00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.746+0000 7fdcd6000700 1 -- 172.21.15.69:0/1827230553 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdcd00ce580 con 0x7fdcd810a470 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.746+0000 7fdcdd400700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdcc00a2890 0x7fdcc00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.747+0000 7fdcdd400700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdcc00a2890 0x7fdcc00a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdcd819bb60 tx=0x7fdccc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:06.754+0000 7fdcd6000700 1 -- 172.21.15.69:0/1827230553 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdcd0010070 con 0x7fdcd810a470 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.059+0000 7fdcde800700 1 -- 172.21.15.69:0/1827230553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fdcd802dbb0 con 0x7fdcd810a470 2024-03-19T01:09:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.059+0000 7fdcd6000700 1 -- 172.21.15.69:0/1827230553 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdcd009b260 con 0x7fdcd810a470 2024-03-19T01:09:07.713 INFO:teuthology.orchestra.run.smithi069.stdout:287762808878 2024-03-19T01:09:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 -- 172.21.15.69:0/1827230553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdcc00a2890 msgr2=0x7fdcc00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdcc00a2890 0x7fdcc00a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdcd819bb60 tx=0x7fdccc009260 comp rx=0 tx=0).stop 2024-03-19T01:09:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 -- 172.21.15.69:0/1827230553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd810a470 msgr2=0x7fdcd819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd810a470 0x7fdcd819aa20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0007ee0 tx=0x7fdcd000d3b0 comp rx=0 tx=0).stop 2024-03-19T01:09:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 -- 172.21.15.69:0/1827230553 shutdown_connections 2024-03-19T01:09:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdcc00a2890 0x7fdcc00a4d50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd810a470 0x7fdcd819aa20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcd81043e0 0x7fdcd81a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 --2- 172.21.15.69:0/1827230553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd8104da0 0x7fdcd81a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 -- 172.21.15.69:0/1827230553 >> 172.21.15.69:0/1827230553 conn(0x7fdcd80ffef0 msgr2=0x7fdcd81004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 -- 172.21.15.69:0/1827230553 shutdown_connections 2024-03-19T01:09:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:07.063+0000 7fdcde800700 1 -- 172.21.15.69:0/1827230553 wait complete. 2024-03-19T01:09:08.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:08 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1827230553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-19T01:09:08.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:08 smithi086 ceph-mon[66829]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-19T01:09:08.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:08 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1827230553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-19T01:09:08.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:08 smithi069 ceph-mon[31727]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-19T01:09:08.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:08 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1827230553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-19T01:09:08.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:08 smithi163 ceph-mon[67428]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-19T01:09:08.475 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738463 for osd.0 2024-03-19T01:09:08.475 DEBUG:teuthology.parallel:result is None 2024-03-19T01:09:09.659 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920835 got 141733920843 for osd.4 2024-03-19T01:09:09.660 DEBUG:teuthology.parallel:result is None 2024-03-19T01:09:09.776 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070511 got 253403070516 for osd.8 2024-03-19T01:09:09.776 DEBUG:teuthology.parallel:result is None 2024-03-19T01:09:10.923 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:10 smithi069 ceph-mon[31727]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-19T01:09:10.923 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:10 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/422490229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-19T01:09:10.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.921+0000 7f797ee00700 1 -- 172.21.15.69:0/3137634375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978102450 msgr2=0x7f7978102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:10.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.921+0000 7f797ee00700 1 --2- 172.21.15.69:0/3137634375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978102450 0x7f7978102830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f796c009920 tx=0x7f796c009c30 comp rx=0 tx=0).stop 2024-03-19T01:09:10.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.921+0000 7f797ee00700 1 -- 172.21.15.69:0/3137634375 shutdown_connections 2024-03-19T01:09:10.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.921+0000 7f797ee00700 1 --2- 172.21.15.69:0/3137634375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978107530 0x7f797810f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.921+0000 7f797ee00700 1 --2- 172.21.15.69:0/3137634375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978102450 0x7f7978102830 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.921+0000 7f797ee00700 1 --2- 172.21.15.69:0/3137634375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7978102e00 0x7f7978106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.921+0000 7f797ee00700 1 -- 172.21.15.69:0/3137634375 >> 172.21.15.69:0/3137634375 conn(0x7f79780fdce0 msgr2=0x7f7978100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.921+0000 7f797ee00700 1 -- 172.21.15.69:0/3137634375 shutdown_connections 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.921+0000 7f797ee00700 1 -- 172.21.15.69:0/3137634375 wait complete. 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.923+0000 7f797ee00700 1 Processor -- start 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.923+0000 7f797ee00700 1 -- start start 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.923+0000 7f797ee00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978102450 0x7f79781a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.923+0000 7f797ee00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7978102e00 0x7f79781a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.923+0000 7f797ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978107530 0x7f797819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.923+0000 7f797ee00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7978112560 con 0x7f7978102e00 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.923+0000 7f797ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79781123e0 con 0x7f7978107530 2024-03-19T01:09:10.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.923+0000 7f797ee00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f79781126e0 con 0x7f7978102450 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.924+0000 7f797e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978107530 0x7f797819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.924+0000 7f797e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978107530 0x7f797819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33088/0 (socket says 172.21.15.69:33088) 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.924+0000 7f797e400700 1 -- 172.21.15.69:0/3308044306 learned_addr learned my addr 172.21.15.69:0/3308044306 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.924+0000 7f797d000700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7978102e00 0x7f79781a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.924+0000 7f797da00700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978102450 0x7f79781a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.924+0000 7f797e400700 1 -- 172.21.15.69:0/3308044306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978102450 msgr2=0x7f79781a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.924+0000 7f797e400700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978102450 0x7f79781a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.924+0000 7f797e400700 1 -- 172.21.15.69:0/3308044306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7978102e00 msgr2=0x7f79781a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.924+0000 7f797e400700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7978102e00 0x7f79781a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.924+0000 7f797e400700 1 -- 172.21.15.69:0/3308044306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797819b220 con 0x7f7978107530 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.924+0000 7f797da00700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978102450 0x7f79781a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.925+0000 7f797e400700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978107530 0x7f797819aac0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f797000c110 tx=0x7f797000c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.925+0000 7f7976000700 1 -- 172.21.15.69:0/3308044306 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797000cd90 con 0x7f7978107530 2024-03-19T01:09:10.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.925+0000 7f7976000700 1 -- 172.21.15.69:0/3308044306 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7970004bb0 con 0x7f7978107530 2024-03-19T01:09:10.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.925+0000 7f7976000700 1 -- 172.21.15.69:0/3308044306 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970007e00 con 0x7f7978107530 2024-03-19T01:09:10.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.926+0000 7f797ee00700 1 -- 172.21.15.69:0/3308044306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797819b4b0 con 0x7f7978107530 2024-03-19T01:09:10.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.926+0000 7f797ee00700 1 -- 172.21.15.69:0/3308044306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79781a77d0 con 0x7f7978107530 2024-03-19T01:09:10.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.927+0000 7f797ee00700 1 -- 172.21.15.69:0/3308044306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7978068fd0 con 0x7f7978107530 2024-03-19T01:09:10.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.929+0000 7f7976000700 1 -- 172.21.15.69:0/3308044306 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f797001a070 con 0x7f7978107530 2024-03-19T01:09:10.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.931+0000 7f7976000700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79600a2910 0x7f79600a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:10.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.931+0000 7f7976000700 1 -- 172.21.15.69:0/3308044306 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f79700ce3b0 con 0x7f7978107530 2024-03-19T01:09:10.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.931+0000 7f797da00700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79600a2910 0x7f79600a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:10.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.932+0000 7f797da00700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79600a2910 0x7f79600a4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f796c009680 tx=0x7f796c009f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:10.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:10.934+0000 7f7976000700 1 -- 172.21.15.69:0/3308044306 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f797004e740 con 0x7f7978107530 2024-03-19T01:09:11.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:10 smithi086 ceph-mon[66829]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-19T01:09:11.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:10 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/422490229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-19T01:09:11.014 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:10 smithi163 ceph-mon[67428]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-19T01:09:11.014 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:10 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/422490229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-19T01:09:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.004+0000 7f210d200700 1 -- 172.21.15.69:0/680473971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2108102f70 msgr2=0x7f2108106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.004+0000 7f210d200700 1 --2- 172.21.15.69:0/680473971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2108102f70 0x7f2108106fb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009990 tx=0x7f20f4009ca0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.005+0000 7f210d200700 1 -- 172.21.15.69:0/680473971 shutdown_connections 2024-03-19T01:09:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.005+0000 7f210d200700 1 --2- 172.21.15.69:0/680473971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2108107690 0x7f210810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.005+0000 7f210d200700 1 --2- 172.21.15.69:0/680473971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21081025c0 0x7f21081029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.005+0000 7f210d200700 1 --2- 172.21.15.69:0/680473971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2108102f70 0x7f2108106fb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.005+0000 7f210d200700 1 -- 172.21.15.69:0/680473971 >> 172.21.15.69:0/680473971 conn(0x7f2108079470 msgr2=0x7f2108079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.005+0000 7f210d200700 1 -- 172.21.15.69:0/680473971 shutdown_connections 2024-03-19T01:09:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.005+0000 7f210d200700 1 -- 172.21.15.69:0/680473971 wait complete. 2024-03-19T01:09:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.006+0000 7f210d200700 1 Processor -- start 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.006+0000 7f210d200700 1 -- start start 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.007+0000 7f210d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081025c0 0x7f210819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.007+0000 7f210d200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2108102f70 0x7f210819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.007+0000 7f210d200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2108107690 0x7f210819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.007+0000 7f210d200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f21081126d0 con 0x7f2108102f70 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.007+0000 7f210d200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2108112550 con 0x7f21081025c0 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.007+0000 7f210d200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2108112850 con 0x7f2108107690 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.007+0000 7f2105400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2108102f70 0x7f210819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.007+0000 7f2105e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081025c0 0x7f210819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.007+0000 7f2106800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2108107690 0x7f210819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.007+0000 7f2105400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2108102f70 0x7f210819b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46896/0 (socket says 172.21.15.69:46896) 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.008+0000 7f2105400700 1 -- 172.21.15.69:0/745570630 learned_addr learned my addr 172.21.15.69:0/745570630 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.008+0000 7f2106800700 1 -- 172.21.15.69:0/745570630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081025c0 msgr2=0x7f210819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.008+0000 7f2106800700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081025c0 0x7f210819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.008+0000 7f2106800700 1 -- 172.21.15.69:0/745570630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2108102f70 msgr2=0x7f210819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.008+0000 7f2106800700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2108102f70 0x7f210819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.008+0000 7f2106800700 1 -- 172.21.15.69:0/745570630 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210819fdb0 con 0x7f2108107690 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.008+0000 7f2105400700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2108102f70 0x7f210819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.008+0000 7f2105e00700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081025c0 0x7f210819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.008+0000 7f2106800700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2108107690 0x7f210819f670 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f210000efc0 tx=0x7f210000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.009+0000 7f20fea00700 1 -- 172.21.15.69:0/745570630 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21000092e0 con 0x7f2108107690 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.009+0000 7f20fea00700 1 -- 172.21.15.69:0/745570630 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100004bb0 con 0x7f2108107690 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.009+0000 7f210d200700 1 -- 172.21.15.69:0/745570630 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21081a0040 con 0x7f2108107690 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.009+0000 7f210d200700 1 -- 172.21.15.69:0/745570630 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081a7920 con 0x7f2108107690 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.009+0000 7f20fea00700 1 -- 172.21.15.69:0/745570630 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100004020 con 0x7f2108107690 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.010+0000 7f210d200700 1 -- 172.21.15.69:0/745570630 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e8005330 con 0x7f2108107690 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.012+0000 7f20fea00700 1 -- 172.21.15.69:0/745570630 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f210001b030 con 0x7f2108107690 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.013+0000 7f20fea00700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f20ec0a2910 0x7f20ec0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.013+0000 7f20fea00700 1 -- 172.21.15.69:0/745570630 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f21000ce5a0 con 0x7f2108107690 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.013+0000 7f2105e00700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f20ec0a2910 0x7f20ec0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.014+0000 7f2105e00700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f20ec0a2910 0x7f20ec0a4dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f20f8011490 tx=0x7f20f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.017+0000 7f20fea00700 1 -- 172.21.15.69:0/745570630 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2100022020 con 0x7f2108107690 2024-03-19T01:09:11.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.439+0000 7fd3e2200700 1 -- 172.21.15.69:0/3427786142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc102450 msgr2=0x7fd3dc102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.439+0000 7fd3e2200700 1 --2- 172.21.15.69:0/3427786142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc102450 0x7fd3dc102830 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009990 tx=0x7fd3d0009ca0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.440+0000 7fd3e2200700 1 -- 172.21.15.69:0/3427786142 shutdown_connections 2024-03-19T01:09:11.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.440+0000 7fd3e2200700 1 --2- 172.21.15.69:0/3427786142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc107530 0x7fd3dc10f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.440+0000 7fd3e2200700 1 --2- 172.21.15.69:0/3427786142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc102450 0x7fd3dc102830 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.440+0000 7fd3e2200700 1 --2- 172.21.15.69:0/3427786142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3dc102e00 0x7fd3dc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.440+0000 7fd3e2200700 1 -- 172.21.15.69:0/3427786142 >> 172.21.15.69:0/3427786142 conn(0x7fd3dc0fdce0 msgr2=0x7fd3dc100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:11.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.440+0000 7fd3e2200700 1 -- 172.21.15.69:0/3427786142 shutdown_connections 2024-03-19T01:09:11.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.440+0000 7fd3e2200700 1 -- 172.21.15.69:0/3427786142 wait complete. 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.441+0000 7fd3e2200700 1 Processor -- start 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.441+0000 7fd3e2200700 1 -- start start 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e2200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3dc102450 0x7fd3dc1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e2200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc102e00 0x7fd3dc1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e2200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc107530 0x7fd3dc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e2200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc112530 con 0x7fd3dc102e00 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e2200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1123b0 con 0x7fd3dc107530 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e2200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1126b0 con 0x7fd3dc102450 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e0e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3dc102450 0x7fd3dc1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc107530 0x7fd3dc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e0e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3dc102450 0x7fd3dc1a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:35610/0 (socket says 172.21.15.69:35610) 2024-03-19T01:09:11.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e0e00700 1 -- 172.21.15.69:0/422490229 learned_addr learned my addr 172.21.15.69:0/422490229 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:11.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3dbe00700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc102e00 0x7fd3dc1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:11.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e1800700 1 -- 172.21.15.69:0/422490229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3dc102450 msgr2=0x7fd3dc1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e1800700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3dc102450 0x7fd3dc1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e1800700 1 -- 172.21.15.69:0/422490229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc102e00 msgr2=0x7fd3dc1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e1800700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc102e00 0x7fd3dc1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e1800700 1 -- 172.21.15.69:0/422490229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc19b1c0 con 0x7fd3dc107530 2024-03-19T01:09:11.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.442+0000 7fd3e0e00700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3dc102450 0x7fd3dc1a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:11.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.443+0000 7fd3e1800700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc107530 0x7fd3dc19aa60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4007f10 tx=0x7fd3d400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:11.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.443+0000 7fd3d9600700 1 -- 172.21.15.69:0/422490229 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4004db0 con 0x7fd3dc107530 2024-03-19T01:09:11.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.443+0000 7fd3d9600700 1 -- 172.21.15.69:0/422490229 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d4009e20 con 0x7fd3dc107530 2024-03-19T01:09:11.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.443+0000 7fd3d9600700 1 -- 172.21.15.69:0/422490229 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4015400 con 0x7fd3dc107530 2024-03-19T01:09:11.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.443+0000 7fd3e2200700 1 -- 172.21.15.69:0/422490229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc19b4b0 con 0x7fd3dc107530 2024-03-19T01:09:11.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.443+0000 7fd3e2200700 1 -- 172.21.15.69:0/422490229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc1a7770 con 0x7fd3dc107530 2024-03-19T01:09:11.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.444+0000 7fd3e2200700 1 -- 172.21.15.69:0/422490229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3dc068fd0 con 0x7fd3dc107530 2024-03-19T01:09:11.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.446+0000 7fd3d9600700 1 -- 172.21.15.69:0/422490229 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd3d400db20 con 0x7fd3dc107530 2024-03-19T01:09:11.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.449+0000 7fd3d9600700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd3c40a2910 0x7fd3c40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:11.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.449+0000 7fd3d9600700 1 -- 172.21.15.69:0/422490229 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd3d40ce6a0 con 0x7fd3dc107530 2024-03-19T01:09:11.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.449+0000 7fd3e0e00700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd3c40a2910 0x7fd3c40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:11.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.450+0000 7fd3e0e00700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd3c40a2910 0x7fd3c40a4dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009750 tx=0x7fd3d001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:11.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.452+0000 7fd3d9600700 1 -- 172.21.15.69:0/422490229 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd3d4010070 con 0x7fd3dc107530 2024-03-19T01:09:11.164 INFO:teuthology.orchestra.run.smithi069.stdout:219043332154 2024-03-19T01:09:11.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.740+0000 7fd3e2200700 1 -- 172.21.15.69:0/422490229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd3dc1a79a0 con 0x7fd3dc107530 2024-03-19T01:09:11.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.741+0000 7fd3d9600700 1 -- 172.21.15.69:0/422490229 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd3d409b380 con 0x7fd3dc107530 2024-03-19T01:09:11.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.743+0000 7fd3e2200700 1 -- 172.21.15.69:0/422490229 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd3c40a2910 msgr2=0x7fd3c40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.743+0000 7fd3e2200700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd3c40a2910 0x7fd3c40a4dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009750 tx=0x7fd3d001a040 comp rx=0 tx=0).stop 2024-03-19T01:09:11.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.743+0000 7fd3e2200700 1 -- 172.21.15.69:0/422490229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc107530 msgr2=0x7fd3dc19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.743+0000 7fd3e2200700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc107530 0x7fd3dc19aa60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4007f10 tx=0x7fd3d400d3b0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.744+0000 7fd3e2200700 1 -- 172.21.15.69:0/422490229 shutdown_connections 2024-03-19T01:09:11.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.744+0000 7fd3e2200700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd3c40a2910 0x7fd3c40a4dd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.744+0000 7fd3e2200700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3dc107530 0x7fd3dc19aa60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.744+0000 7fd3e2200700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3dc102450 0x7fd3dc1a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.744+0000 7fd3e2200700 1 --2- 172.21.15.69:0/422490229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc102e00 0x7fd3dc1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.744+0000 7fd3e2200700 1 -- 172.21.15.69:0/422490229 >> 172.21.15.69:0/422490229 conn(0x7fd3dc0fdce0 msgr2=0x7fd3dc0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:11.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.744+0000 7fd3e2200700 1 -- 172.21.15.69:0/422490229 shutdown_connections 2024-03-19T01:09:11.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:09.744+0000 7fd3e2200700 1 -- 172.21.15.69:0/422490229 wait complete. 2024-03-19T01:09:11.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.208+0000 7f797ee00700 1 -- 172.21.15.69:0/3308044306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f79781a79a0 con 0x7f7978107530 2024-03-19T01:09:11.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.208+0000 7f7976000700 1 -- 172.21.15.69:0/3308044306 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7970010040 con 0x7f7978107530 2024-03-19T01:09:11.210 INFO:teuthology.orchestra.run.smithi069.stdout:77309411416 2024-03-19T01:09:11.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.212+0000 7f797ee00700 1 -- 172.21.15.69:0/3308044306 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79600a2910 msgr2=0x7f79600a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.212+0000 7f797ee00700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79600a2910 0x7f79600a4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f796c009680 tx=0x7f796c009f80 comp rx=0 tx=0).stop 2024-03-19T01:09:11.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.212+0000 7f797ee00700 1 -- 172.21.15.69:0/3308044306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978107530 msgr2=0x7f797819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.212+0000 7f797ee00700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978107530 0x7f797819aac0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f797000c110 tx=0x7f797000c4d0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.213+0000 7f797ee00700 1 -- 172.21.15.69:0/3308044306 shutdown_connections 2024-03-19T01:09:11.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.213+0000 7f797ee00700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79600a2910 0x7f79600a4dd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.213+0000 7f797ee00700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978107530 0x7f797819aac0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.213+0000 7f797ee00700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978102450 0x7f79781a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.213+0000 7f797ee00700 1 --2- 172.21.15.69:0/3308044306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7978102e00 0x7f79781a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.213+0000 7f797ee00700 1 -- 172.21.15.69:0/3308044306 >> 172.21.15.69:0/3308044306 conn(0x7f79780fdce0 msgr2=0x7f7978100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:11.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.213+0000 7f797ee00700 1 -- 172.21.15.69:0/3308044306 shutdown_connections 2024-03-19T01:09:11.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.213+0000 7f797ee00700 1 -- 172.21.15.69:0/3308044306 wait complete. 2024-03-19T01:09:11.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.282+0000 7f210d200700 1 -- 172.21.15.69:0/745570630 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f20e8005620 con 0x7f2108107690 2024-03-19T01:09:11.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.283+0000 7f20fea00700 1 -- 172.21.15.69:0/745570630 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2100010040 con 0x7f2108107690 2024-03-19T01:09:11.284 INFO:teuthology.orchestra.run.smithi069.stdout:197568495680 2024-03-19T01:09:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 -- 172.21.15.69:0/745570630 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f20ec0a2910 msgr2=0x7f20ec0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f20ec0a2910 0x7f20ec0a4dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f20f8011490 tx=0x7f20f8009260 comp rx=0 tx=0).stop 2024-03-19T01:09:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 -- 172.21.15.69:0/745570630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2108107690 msgr2=0x7f210819f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2108107690 0x7f210819f670 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f210000efc0 tx=0x7f210000c3a0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 -- 172.21.15.69:0/745570630 shutdown_connections 2024-03-19T01:09:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f20ec0a2910 0x7f20ec0a4dd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2108107690 0x7f210819f670 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21081025c0 0x7f210819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 --2- 172.21.15.69:0/745570630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2108102f70 0x7f210819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 -- 172.21.15.69:0/745570630 >> 172.21.15.69:0/745570630 conn(0x7f2108079470 msgr2=0x7f210810da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:11.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 -- 172.21.15.69:0/745570630 shutdown_connections 2024-03-19T01:09:11.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:11.286+0000 7f210d200700 1 -- 172.21.15.69:0/745570630 wait complete. 2024-03-19T01:09:11.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:11 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3308044306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-19T01:09:11.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:11 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/745570630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-19T01:09:12.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:11 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3308044306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-19T01:09:12.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:11 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/745570630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-19T01:09:12.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:11 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3308044306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-19T01:09:12.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:11 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/745570630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-19T01:09:12.497 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332145 got 219043332154 for osd.7 2024-03-19T01:09:12.497 DEBUG:teuthology.parallel:result is None 2024-03-19T01:09:12.844 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808873 got 287762808878 for osd.9 2024-03-19T01:09:12.844 DEBUG:teuthology.parallel:result is None 2024-03-19T01:09:13.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:12 smithi086 ceph-mon[66829]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:13.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:12 smithi069 ceph-mon[31727]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:13.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:12 smithi163 ceph-mon[67428]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:13.774 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495674 got 197568495680 for osd.6 2024-03-19T01:09:13.774 DEBUG:teuthology.parallel:result is None 2024-03-19T01:09:13.907 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411409 got 77309411416 for osd.2 2024-03-19T01:09:13.907 DEBUG:teuthology.parallel:result is None 2024-03-19T01:09:13.907 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-19T01:09:13.908 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-19T01:09:14.347 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:09:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:14 smithi069 ceph-mon[31727]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:15.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:14 smithi086 ceph-mon[66829]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:15.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:14 smithi163 ceph-mon[67428]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:16.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.595+0000 7f5bd2a00700 1 -- 172.21.15.69:0/1132506513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc102e00 msgr2=0x7f5bcc106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:16.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.595+0000 7f5bd2a00700 1 --2- 172.21.15.69:0/1132506513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc102e00 0x7f5bcc106e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009990 tx=0x7f5bb8009ca0 comp rx=0 tx=0).stop 2024-03-19T01:09:16.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.595+0000 7f5bd2a00700 1 -- 172.21.15.69:0/1132506513 shutdown_connections 2024-03-19T01:09:16.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.595+0000 7f5bd2a00700 1 --2- 172.21.15.69:0/1132506513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bcc107530 0x7f5bcc10f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:16.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.595+0000 7f5bd2a00700 1 --2- 172.21.15.69:0/1132506513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bcc102450 0x7f5bcc102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:16.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.595+0000 7f5bd2a00700 1 --2- 172.21.15.69:0/1132506513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc102e00 0x7f5bcc106e50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:16.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.595+0000 7f5bd2a00700 1 -- 172.21.15.69:0/1132506513 >> 172.21.15.69:0/1132506513 conn(0x7f5bcc0fdce0 msgr2=0x7f5bcc100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:16.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.595+0000 7f5bd2a00700 1 -- 172.21.15.69:0/1132506513 shutdown_connections 2024-03-19T01:09:16.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.595+0000 7f5bd2a00700 1 -- 172.21.15.69:0/1132506513 wait complete. 2024-03-19T01:09:16.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.597+0000 7f5bd2a00700 1 Processor -- start 2024-03-19T01:09:16.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.597+0000 7f5bd2a00700 1 -- start start 2024-03-19T01:09:16.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.597+0000 7f5bd2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bcc102450 0x7f5bcc1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:16.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.597+0000 7f5bd2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc102e00 0x7f5bcc1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:16.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd2a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bcc107530 0x7f5bcc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:16.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd2a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc112560 con 0x7f5bcc102450 2024-03-19T01:09:16.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd2a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1123e0 con 0x7f5bcc102e00 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd2a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1126e0 con 0x7f5bcc107530 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc102e00 0x7f5bcc1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc102e00 0x7f5bcc1a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58648/0 (socket says 172.21.15.69:58648) 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd0c00700 1 -- 172.21.15.69:0/3957577616 learned_addr learned my addr 172.21.15.69:0/3957577616 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd1600700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bcc102450 0x7f5bcc1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd2000700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bcc107530 0x7f5bcc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd0c00700 1 -- 172.21.15.69:0/3957577616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bcc107530 msgr2=0x7f5bcc19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd0c00700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bcc107530 0x7f5bcc19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd0c00700 1 -- 172.21.15.69:0/3957577616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bcc102450 msgr2=0x7f5bcc1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd0c00700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bcc102450 0x7f5bcc1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd0c00700 1 -- 172.21.15.69:0/3957577616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc19b200 con 0x7f5bcc102e00 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd2000700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bcc107530 0x7f5bcc19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:16.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.598+0000 7f5bd1600700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bcc102450 0x7f5bcc1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:16.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.599+0000 7f5bd0c00700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc102e00 0x7f5bcc1a0dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8000c00 tx=0x7f5bb800be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:16.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.599+0000 7f5bca000700 1 -- 172.21.15.69:0/3957577616 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8027070 con 0x7f5bcc102e00 2024-03-19T01:09:16.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.599+0000 7f5bca000700 1 -- 172.21.15.69:0/3957577616 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb8022470 con 0x7f5bcc102e00 2024-03-19T01:09:16.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.599+0000 7f5bd2a00700 1 -- 172.21.15.69:0/3957577616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bcc19b490 con 0x7f5bcc102e00 2024-03-19T01:09:16.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.599+0000 7f5bca000700 1 -- 172.21.15.69:0/3957577616 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8017650 con 0x7f5bcc102e00 2024-03-19T01:09:16.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.599+0000 7f5bd2a00700 1 -- 172.21.15.69:0/3957577616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bcc1a7760 con 0x7f5bcc102e00 2024-03-19T01:09:16.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.600+0000 7f5bd2a00700 1 -- 172.21.15.69:0/3957577616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bcc068fd0 con 0x7f5bcc102e00 2024-03-19T01:09:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.603+0000 7f5bca000700 1 -- 172.21.15.69:0/3957577616 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f5bb802a020 con 0x7f5bcc102e00 2024-03-19T01:09:16.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.605+0000 7f5bca000700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5bb40a2910 0x7f5bb40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:16.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.605+0000 7f5bca000700 1 -- 172.21.15.69:0/3957577616 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5bb80cfa60 con 0x7f5bcc102e00 2024-03-19T01:09:16.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.605+0000 7f5bd1600700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5bb40a2910 0x7f5bb40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:16.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.606+0000 7f5bd1600700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5bb40a2910 0x7f5bb40a4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0fe560 tx=0x7f5bc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:16.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.606+0000 7f5bca000700 1 -- 172.21.15.69:0/3957577616 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5bb8013070 con 0x7f5bcc102e00 2024-03-19T01:09:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.862+0000 7f5bd2a00700 1 -- 172.21.15.69:0/3957577616 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5bcc0ff690 con 0x7f5bb40a2910 2024-03-19T01:09:16.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.891+0000 7f5bca000700 1 -- 172.21.15.69:0/3957577616 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340819 (secure 0 0 0) 0x7f5bcc0ff690 con 0x7f5bb40a2910 2024-03-19T01:09:16.894 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:16.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 -- 172.21.15.69:0/3957577616 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5bb40a2910 msgr2=0x7f5bb40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:16.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5bb40a2910 0x7f5bb40a4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0fe560 tx=0x7f5bc0009260 comp rx=0 tx=0).stop 2024-03-19T01:09:16.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 -- 172.21.15.69:0/3957577616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc102e00 msgr2=0x7f5bcc1a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:16.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc102e00 0x7f5bcc1a0dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8000c00 tx=0x7f5bb800be70 comp rx=0 tx=0).stop 2024-03-19T01:09:16.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 -- 172.21.15.69:0/3957577616 shutdown_connections 2024-03-19T01:09:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5bb40a2910 0x7f5bb40a4dd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bcc107530 0x7f5bcc19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bcc102450 0x7f5bcc1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 --2- 172.21.15.69:0/3957577616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc102e00 0x7f5bcc1a0dd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 -- 172.21.15.69:0/3957577616 >> 172.21.15.69:0/3957577616 conn(0x7f5bcc0fdce0 msgr2=0x7f5bcc10d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 -- 172.21.15.69:0/3957577616 shutdown_connections 2024-03-19T01:09:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:16.899+0000 7f5bd2a00700 1 -- 172.21.15.69:0/3957577616 wait complete. 2024-03-19T01:09:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-03-19T01:09:17.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:16 smithi086 ceph-mon[66829]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:16 smithi069 ceph-mon[31727]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:17.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:16 smithi163 ceph-mon[67428]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:17.727 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-03-19T01:09:15.683109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660430,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1011,"num_read_kb":632,"num_write":432,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378088,"kb_used_data":12296,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563736,"statfs":{"total":1151940427776,"available":1151553265664,"internally_reserved":0,"allocated":12591104,"data_stored":7967096,"data_compressed":359643,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004967"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.660362+0000","last_change":"2024-03-19T01:07:15.500098+0000","last_active":"2024-03-19T01:08:53.660362+0000","last_peered":"2024-03-19T01:08:53.660362+0000","last_clean":"2024-03-19T01:08:53.660362+0000","last_became_active":"2024-03-19T01:07:15.499738+0000","last_became_peered":"2024-03-19T01:07:15.499738+0000","last_unstale":"2024-03-19T01:08:53.660362+0000","last_undegraded":"2024-03-19T01:08:53.660362+0000","last_fullsized":"2024-03-19T01:08:53.660362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:14.703153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.658198+0000","last_change":"2024-03-19T01:07:15.492863+0000","last_active":"2024-03-19T01:08:53.658198+0000","last_peered":"2024-03-19T01:08:53.658198+0000","last_clean":"2024-03-19T01:08:53.658198+0000","last_became_active":"2024-03-19T01:07:15.492533+0000","last_became_peered":"2024-03-19T01:07:15.492533+0000","last_unstale":"2024-03-19T01:08:53.658198+0000","last_undegraded":"2024-03-19T01:08:53.658198+0000","last_fullsized":"2024-03-19T01:08:53.658198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:49.028352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.664970+0000","last_change":"2024-03-19T01:07:15.494174+0000","last_active":"2024-03-19T01:08:53.664970+0000","last_peered":"2024-03-19T01:08:53.664970+0000","last_clean":"2024-03-19T01:08:53.664970+0000","last_became_active":"2024-03-19T01:07:15.490562+0000","last_became_peered":"2024-03-19T01:07:15.490562+0000","last_unstale":"2024-03-19T01:08:53.664970+0000","last_undegraded":"2024-03-19T01:08:53.664970+0000","last_fullsized":"2024-03-19T01:08:53.664970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:18.664396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.622982+0000","last_change":"2024-03-19T01:07:15.478397+0000","last_active":"2024-03-19T01:08:53.622982+0000","last_peered":"2024-03-19T01:08:53.622982+0000","last_clean":"2024-03-19T01:08:53.622982+0000","last_became_active":"2024-03-19T01:07:15.477000+0000","last_became_peered":"2024-03-19T01:07:15.477000+0000","last_unstale":"2024-03-19T01:08:53.622982+0000","last_undegraded":"2024-03-19T01:08:53.622982+0000","last_fullsized":"2024-03-19T01:08:53.622982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:58.534329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.650862+0000","last_change":"2024-03-19T01:07:15.500138+0000","last_active":"2024-03-19T01:08:53.650862+0000","last_peered":"2024-03-19T01:08:53.650862+0000","last_clean":"2024-03-19T01:08:53.650862+0000","last_became_active":"2024-03-19T01:07:15.499759+0000","last_became_peered":"2024-03-19T01:07:15.499759+0000","last_unstale":"2024-03-19T01:08:53.650862+0000","last_undegraded":"2024-03-19T01:08:53.650862+0000","last_fullsized":"2024-03-19T01:08:53.650862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:23.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.639845+0000","last_change":"2024-03-19T01:07:15.485146+0000","last_active":"2024-03-19T01:08:53.639845+0000","last_peered":"2024-03-19T01:08:53.639845+0000","last_clean":"2024-03-19T01:08:53.639845+0000","last_became_active":"2024-03-19T01:07:15.484680+0000","last_became_peered":"2024-03-19T01:07:15.484680+0000","last_unstale":"2024-03-19T01:08:53.639845+0000","last_undegraded":"2024-03-19T01:08:53.639845+0000","last_fullsized":"2024-03-19T01:08:53.639845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:12.659347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.648331+0000","last_change":"2024-03-19T01:07:15.486327+0000","last_active":"2024-03-19T01:08:53.648331+0000","last_peered":"2024-03-19T01:08:53.648331+0000","last_clean":"2024-03-19T01:08:53.648331+0000","last_became_active":"2024-03-19T01:07:15.485954+0000","last_became_peered":"2024-03-19T01:07:15.485954+0000","last_unstale":"2024-03-19T01:08:53.648331+0000","last_undegraded":"2024-03-19T01:08:53.648331+0000","last_fullsized":"2024-03-19T01:08:53.648331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:53.316260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.651352+0000","last_change":"2024-03-19T01:07:15.504993+0000","last_active":"2024-03-19T01:08:53.651352+0000","last_peered":"2024-03-19T01:08:53.651352+0000","last_clean":"2024-03-19T01:08:53.651352+0000","last_became_active":"2024-03-19T01:07:15.502771+0000","last_became_peered":"2024-03-19T01:07:15.502771+0000","last_unstale":"2024-03-19T01:08:53.651352+0000","last_undegraded":"2024-03-19T01:08:53.651352+0000","last_fullsized":"2024-03-19T01:08:53.651352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:22:33.712608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.648570+0000","last_change":"2024-03-19T01:07:15.483876+0000","last_active":"2024-03-19T01:08:53.648570+0000","last_peered":"2024-03-19T01:08:53.648570+0000","last_clean":"2024-03-19T01:08:53.648570+0000","last_became_active":"2024-03-19T01:07:15.482744+0000","last_became_peered":"2024-03-19T01:07:15.482744+0000","last_unstale":"2024-03-19T01:08:53.648570+0000","last_undegraded":"2024-03-19T01:08:53.648570+0000","last_fullsized":"2024-03-19T01:08:53.648570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:21:19.870877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.638591+0000","last_change":"2024-03-19T01:07:15.475655+0000","last_active":"2024-03-19T01:08:53.638591+0000","last_peered":"2024-03-19T01:08:53.638591+0000","last_clean":"2024-03-19T01:08:53.638591+0000","last_became_active":"2024-03-19T01:07:15.475265+0000","last_became_peered":"2024-03-19T01:07:15.475265+0000","last_unstale":"2024-03-19T01:08:53.638591+0000","last_undegraded":"2024-03-19T01:08:53.638591+0000","last_fullsized":"2024-03-19T01:08:53.638591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:24:38.898191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.653740+0000","last_change":"2024-03-19T01:07:15.482988+0000","last_active":"2024-03-19T01:08:53.653740+0000","last_peered":"2024-03-19T01:08:53.653740+0000","last_clean":"2024-03-19T01:08:53.653740+0000","last_became_active":"2024-03-19T01:07:15.482673+0000","last_became_peered":"2024-03-19T01:07:15.482673+0000","last_unstale":"2024-03-19T01:08:53.653740+0000","last_undegraded":"2024-03-19T01:08:53.653740+0000","last_fullsized":"2024-03-19T01:08:53.653740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:15:48.807200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.622597+0000","last_change":"2024-03-19T01:07:15.479593+0000","last_active":"2024-03-19T01:08:53.622597+0000","last_peered":"2024-03-19T01:08:53.622597+0000","last_clean":"2024-03-19T01:08:53.622597+0000","last_became_active":"2024-03-19T01:07:15.479220+0000","last_became_peered":"2024-03-19T01:07:15.479220+0000","last_unstale":"2024-03-19T01:08:53.622597+0000","last_undegraded":"2024-03-19T01:08:53.622597+0000","last_fullsized":"2024-03-19T01:08:53.622597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:40:24.390845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.661191+0000","last_change":"2024-03-19T01:07:15.464737+0000","last_active":"2024-03-19T01:08:53.661191+0000","last_peered":"2024-03-19T01:08:53.661191+0000","last_clean":"2024-03-19T01:08:53.661191+0000","last_became_active":"2024-03-19T01:07:15.463641+0000","last_became_peered":"2024-03-19T01:07:15.463641+0000","last_unstale":"2024-03-19T01:08:53.661191+0000","last_undegraded":"2024-03-19T01:08:53.661191+0000","last_fullsized":"2024-03-19T01:08:53.661191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:13.964397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.661921+0000","last_change":"2024-03-19T01:07:15.492803+0000","last_active":"2024-03-19T01:08:53.661921+0000","last_peered":"2024-03-19T01:08:53.661921+0000","last_clean":"2024-03-19T01:08:53.661921+0000","last_became_active":"2024-03-19T01:07:15.492422+0000","last_became_peered":"2024-03-19T01:07:15.492422+0000","last_unstale":"2024-03-19T01:08:53.661921+0000","last_undegraded":"2024-03-19T01:08:53.661921+0000","last_fullsized":"2024-03-19T01:08:53.661921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:59:08.292399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.664378+0000","last_change":"2024-03-19T01:07:15.498073+0000","last_active":"2024-03-19T01:08:53.664378+0000","last_peered":"2024-03-19T01:08:53.664378+0000","last_clean":"2024-03-19T01:08:53.664378+0000","last_became_active":"2024-03-19T01:07:15.493650+0000","last_became_peered":"2024-03-19T01:07:15.493650+0000","last_unstale":"2024-03-19T01:08:53.664378+0000","last_undegraded":"2024-03-19T01:08:53.664378+0000","last_fullsized":"2024-03-19T01:08:53.664378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:25:28.908122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.662071+0000","last_change":"2024-03-19T01:07:15.496434+0000","last_active":"2024-03-19T01:08:53.662071+0000","last_peered":"2024-03-19T01:08:53.662071+0000","last_clean":"2024-03-19T01:08:53.662071+0000","last_became_active":"2024-03-19T01:07:15.495956+0000","last_became_peered":"2024-03-19T01:07:15.495956+0000","last_unstale":"2024-03-19T01:08:53.662071+0000","last_undegraded":"2024-03-19T01:08:53.662071+0000","last_fullsized":"2024-03-19T01:08:53.662071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:49.505947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.660439+0000","last_change":"2024-03-19T01:07:15.504771+0000","last_active":"2024-03-19T01:08:53.660439+0000","last_peered":"2024-03-19T01:08:53.660439+0000","last_clean":"2024-03-19T01:08:53.660439+0000","last_became_active":"2024-03-19T01:07:15.504485+0000","last_became_peered":"2024-03-19T01:07:15.504485+0000","last_unstale":"2024-03-19T01:08:53.660439+0000","last_undegraded":"2024-03-19T01:08:53.660439+0000","last_fullsized":"2024-03-19T01:08:53.660439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:17:36.760943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.640237+0000","last_change":"2024-03-19T01:07:15.491045+0000","last_active":"2024-03-19T01:08:53.640237+0000","last_peered":"2024-03-19T01:08:53.640237+0000","last_clean":"2024-03-19T01:08:53.640237+0000","last_became_active":"2024-03-19T01:07:15.488914+0000","last_became_peered":"2024-03-19T01:07:15.488914+0000","last_unstale":"2024-03-19T01:08:53.640237+0000","last_undegraded":"2024-03-19T01:08:53.640237+0000","last_fullsized":"2024-03-19T01:08:53.640237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:14.853457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.645777+0000","last_change":"2024-03-19T01:07:15.495306+0000","last_active":"2024-03-19T01:08:53.645777+0000","last_peered":"2024-03-19T01:08:53.645777+0000","last_clean":"2024-03-19T01:08:53.645777+0000","last_became_active":"2024-03-19T01:07:15.494906+0000","last_became_peered":"2024-03-19T01:07:15.494906+0000","last_unstale":"2024-03-19T01:08:53.645777+0000","last_undegraded":"2024-03-19T01:08:53.645777+0000","last_fullsized":"2024-03-19T01:08:53.645777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:48.176410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782715+0000","last_change":"2024-03-19T01:07:19.689402+0000","last_active":"2024-03-19T01:08:24.782715+0000","last_peered":"2024-03-19T01:08:24.782715+0000","last_clean":"2024-03-19T01:08:24.782715+0000","last_became_active":"2024-03-19T01:07:19.688799+0000","last_became_peered":"2024-03-19T01:07:19.688799+0000","last_unstale":"2024-03-19T01:08:24.782715+0000","last_undegraded":"2024-03-19T01:08:24.782715+0000","last_fullsized":"2024-03-19T01:08:24.782715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:51.890047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:10.937004+0000","last_change":"2024-03-19T01:07:14.477348+0000","last_active":"2024-03-19T01:09:10.937004+0000","last_peered":"2024-03-19T01:09:10.937004+0000","last_clean":"2024-03-19T01:09:10.937004+0000","last_became_active":"2024-03-19T01:07:14.476946+0000","last_became_peered":"2024-03-19T01:07:14.476946+0000","last_unstale":"2024-03-19T01:09:10.937004+0000","last_undegraded":"2024-03-19T01:09:10.937004+0000","last_fullsized":"2024-03-19T01:09:10.937004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:19.269837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":83,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.639507+0000","last_change":"2024-03-19T01:07:15.495646+0000","last_active":"2024-03-19T01:08:53.639507+0000","last_peered":"2024-03-19T01:08:53.639507+0000","last_clean":"2024-03-19T01:08:53.639507+0000","last_became_active":"2024-03-19T01:07:15.495203+0000","last_became_peered":"2024-03-19T01:07:15.495203+0000","last_unstale":"2024-03-19T01:08:53.639507+0000","last_undegraded":"2024-03-19T01:08:53.639507+0000","last_fullsized":"2024-03-19T01:08:53.639507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:01:31.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782686+0000","last_change":"2024-03-19T01:07:19.690465+0000","last_active":"2024-03-19T01:08:24.782686+0000","last_peered":"2024-03-19T01:08:24.782686+0000","last_clean":"2024-03-19T01:08:24.782686+0000","last_became_active":"2024-03-19T01:07:19.689852+0000","last_became_peered":"2024-03-19T01:07:19.689852+0000","last_unstale":"2024-03-19T01:08:24.782686+0000","last_undegraded":"2024-03-19T01:08:24.782686+0000","last_fullsized":"2024-03-19T01:08:24.782686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:18.891411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.796417+0000","last_change":"2024-03-19T01:07:14.487165+0000","last_active":"2024-03-19T01:08:24.796417+0000","last_peered":"2024-03-19T01:08:24.796417+0000","last_clean":"2024-03-19T01:08:24.796417+0000","last_became_active":"2024-03-19T01:07:14.484194+0000","last_became_peered":"2024-03-19T01:07:14.484194+0000","last_unstale":"2024-03-19T01:08:24.796417+0000","last_undegraded":"2024-03-19T01:08:24.796417+0000","last_fullsized":"2024-03-19T01:08:24.796417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:54:47.914565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.665671+0000","last_change":"2024-03-19T01:07:15.498687+0000","last_active":"2024-03-19T01:08:53.665671+0000","last_peered":"2024-03-19T01:08:53.665671+0000","last_clean":"2024-03-19T01:08:53.665671+0000","last_became_active":"2024-03-19T01:07:15.498146+0000","last_became_peered":"2024-03-19T01:07:15.498146+0000","last_unstale":"2024-03-19T01:08:53.665671+0000","last_undegraded":"2024-03-19T01:08:53.665671+0000","last_fullsized":"2024-03-19T01:08:53.665671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:47:21.846333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783201+0000","last_change":"2024-03-19T01:07:19.690691+0000","last_active":"2024-03-19T01:08:24.783201+0000","last_peered":"2024-03-19T01:08:24.783201+0000","last_clean":"2024-03-19T01:08:24.783201+0000","last_became_active":"2024-03-19T01:07:19.690083+0000","last_became_peered":"2024-03-19T01:07:19.690083+0000","last_unstale":"2024-03-19T01:08:24.783201+0000","last_undegraded":"2024-03-19T01:08:24.783201+0000","last_fullsized":"2024-03-19T01:08:24.783201+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:51:30.205694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.801994+0000","last_change":"2024-03-19T01:08:24.801510+0000","last_active":"2024-03-19T01:08:24.801994+0000","last_peered":"2024-03-19T01:08:24.801994+0000","last_clean":"2024-03-19T01:08:24.801994+0000","last_became_active":"2024-03-19T01:08:24.789315+0000","last_became_peered":"2024-03-19T01:08:24.789315+0000","last_unstale":"2024-03-19T01:08:24.801994+0000","last_undegraded":"2024-03-19T01:08:24.801994+0000","last_fullsized":"2024-03-19T01:08:24.801994+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:24:03.243394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.656857+0000","last_change":"2024-03-19T01:07:15.489371+0000","last_active":"2024-03-19T01:08:53.656857+0000","last_peered":"2024-03-19T01:08:53.656857+0000","last_clean":"2024-03-19T01:08:53.656857+0000","last_became_active":"2024-03-19T01:07:15.488975+0000","last_became_peered":"2024-03-19T01:07:15.488975+0000","last_unstale":"2024-03-19T01:08:53.656857+0000","last_undegraded":"2024-03-19T01:08:53.656857+0000","last_fullsized":"2024-03-19T01:08:53.656857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:04:47.640512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782771+0000","last_change":"2024-03-19T01:07:19.691590+0000","last_active":"2024-03-19T01:08:24.782771+0000","last_peered":"2024-03-19T01:08:24.782771+0000","last_clean":"2024-03-19T01:08:24.782771+0000","last_became_active":"2024-03-19T01:07:19.691037+0000","last_became_peered":"2024-03-19T01:07:19.691037+0000","last_unstale":"2024-03-19T01:08:24.782771+0000","last_undegraded":"2024-03-19T01:08:24.782771+0000","last_fullsized":"2024-03-19T01:08:24.782771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:20:02.007483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786021+0000","last_change":"2024-03-19T01:07:14.475653+0000","last_active":"2024-03-19T01:08:24.786021+0000","last_peered":"2024-03-19T01:08:24.786021+0000","last_clean":"2024-03-19T01:08:24.786021+0000","last_became_active":"2024-03-19T01:07:14.475315+0000","last_became_peered":"2024-03-19T01:07:14.475315+0000","last_unstale":"2024-03-19T01:08:24.786021+0000","last_undegraded":"2024-03-19T01:08:24.786021+0000","last_fullsized":"2024-03-19T01:08:24.786021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:44:04.656950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.646245+0000","last_change":"2024-03-19T01:07:15.480882+0000","last_active":"2024-03-19T01:08:53.646245+0000","last_peered":"2024-03-19T01:08:53.646245+0000","last_clean":"2024-03-19T01:08:53.646245+0000","last_became_active":"2024-03-19T01:07:15.480541+0000","last_became_peered":"2024-03-19T01:07:15.480541+0000","last_unstale":"2024-03-19T01:08:53.646245+0000","last_undegraded":"2024-03-19T01:08:53.646245+0000","last_fullsized":"2024-03-19T01:08:53.646245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:37:24.724531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783363+0000","last_change":"2024-03-19T01:07:19.672979+0000","last_active":"2024-03-19T01:08:24.783363+0000","last_peered":"2024-03-19T01:08:24.783363+0000","last_clean":"2024-03-19T01:08:24.783363+0000","last_became_active":"2024-03-19T01:07:19.672530+0000","last_became_peered":"2024-03-19T01:07:19.672530+0000","last_unstale":"2024-03-19T01:08:24.783363+0000","last_undegraded":"2024-03-19T01:08:24.783363+0000","last_fullsized":"2024-03-19T01:08:24.783363+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:12.948079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:10.845039+0000","last_change":"2024-03-19T01:07:14.475620+0000","last_active":"2024-03-19T01:09:10.845039+0000","last_peered":"2024-03-19T01:09:10.845039+0000","last_clean":"2024-03-19T01:09:10.845039+0000","last_became_active":"2024-03-19T01:07:14.475029+0000","last_became_peered":"2024-03-19T01:07:14.475029+0000","last_unstale":"2024-03-19T01:09:10.845039+0000","last_undegraded":"2024-03-19T01:09:10.845039+0000","last_fullsized":"2024-03-19T01:09:10.845039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:48:53.983457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.634950+0000","last_change":"2024-03-19T01:07:15.484680+0000","last_active":"2024-03-19T01:08:53.634950+0000","last_peered":"2024-03-19T01:08:53.634950+0000","last_clean":"2024-03-19T01:08:53.634950+0000","last_became_active":"2024-03-19T01:07:15.484333+0000","last_became_peered":"2024-03-19T01:07:15.484333+0000","last_unstale":"2024-03-19T01:08:53.634950+0000","last_undegraded":"2024-03-19T01:08:53.634950+0000","last_fullsized":"2024-03-19T01:08:53.634950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:38:23.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.798971+0000","last_change":"2024-03-19T01:08:24.798971+0000","last_active":"2024-03-19T01:08:24.798971+0000","last_peered":"2024-03-19T01:08:24.798971+0000","last_clean":"2024-03-19T01:08:24.798971+0000","last_became_active":"2024-03-19T01:08:24.798504+0000","last_became_peered":"2024-03-19T01:08:24.798504+0000","last_unstale":"2024-03-19T01:08:24.798971+0000","last_undegraded":"2024-03-19T01:08:24.798971+0000","last_fullsized":"2024-03-19T01:08:24.798971+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.460490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.799220+0000","last_change":"2024-03-19T01:07:14.481499+0000","last_active":"2024-03-19T01:08:24.799220+0000","last_peered":"2024-03-19T01:08:24.799220+0000","last_clean":"2024-03-19T01:08:24.799220+0000","last_became_active":"2024-03-19T01:07:14.481123+0000","last_became_peered":"2024-03-19T01:07:14.481123+0000","last_unstale":"2024-03-19T01:08:24.799220+0000","last_undegraded":"2024-03-19T01:08:24.799220+0000","last_fullsized":"2024-03-19T01:08:24.799220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:36:15.349748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.648991+0000","last_change":"2024-03-19T01:07:15.499744+0000","last_active":"2024-03-19T01:08:53.648991+0000","last_peered":"2024-03-19T01:08:53.648991+0000","last_clean":"2024-03-19T01:08:53.648991+0000","last_became_active":"2024-03-19T01:07:15.499037+0000","last_became_peered":"2024-03-19T01:07:15.499037+0000","last_unstale":"2024-03-19T01:08:53.648991+0000","last_undegraded":"2024-03-19T01:08:53.648991+0000","last_fullsized":"2024-03-19T01:08:53.648991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:20:35.524664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785238+0000","last_change":"2024-03-19T01:07:19.684187+0000","last_active":"2024-03-19T01:08:24.785238+0000","last_peered":"2024-03-19T01:08:24.785238+0000","last_clean":"2024-03-19T01:08:24.785238+0000","last_became_active":"2024-03-19T01:07:19.683782+0000","last_became_peered":"2024-03-19T01:07:19.683782+0000","last_unstale":"2024-03-19T01:08:24.785238+0000","last_undegraded":"2024-03-19T01:08:24.785238+0000","last_fullsized":"2024-03-19T01:08:24.785238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:17:50.818003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.795451+0000","last_change":"2024-03-19T01:07:14.485538+0000","last_active":"2024-03-19T01:08:24.795451+0000","last_peered":"2024-03-19T01:08:24.795451+0000","last_clean":"2024-03-19T01:08:24.795451+0000","last_became_active":"2024-03-19T01:07:14.485004+0000","last_became_peered":"2024-03-19T01:07:14.485004+0000","last_unstale":"2024-03-19T01:08:24.795451+0000","last_undegraded":"2024-03-19T01:08:24.795451+0000","last_fullsized":"2024-03-19T01:08:24.795451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:55:50.853725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.658270+0000","last_change":"2024-03-19T01:07:15.503370+0000","last_active":"2024-03-19T01:08:53.658270+0000","last_peered":"2024-03-19T01:08:53.658270+0000","last_clean":"2024-03-19T01:08:53.658270+0000","last_became_active":"2024-03-19T01:07:15.501375+0000","last_became_peered":"2024-03-19T01:07:15.501375+0000","last_unstale":"2024-03-19T01:08:53.658270+0000","last_undegraded":"2024-03-19T01:08:53.658270+0000","last_fullsized":"2024-03-19T01:08:53.658270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:36:01.814936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.794012+0000","last_change":"2024-03-19T01:07:19.697067+0000","last_active":"2024-03-19T01:08:24.794012+0000","last_peered":"2024-03-19T01:08:24.794012+0000","last_clean":"2024-03-19T01:08:24.794012+0000","last_became_active":"2024-03-19T01:07:19.696364+0000","last_became_peered":"2024-03-19T01:07:19.696364+0000","last_unstale":"2024-03-19T01:08:24.794012+0000","last_undegraded":"2024-03-19T01:08:24.794012+0000","last_fullsized":"2024-03-19T01:08:24.794012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:34.388407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788402+0000","last_change":"2024-03-19T01:07:14.475361+0000","last_active":"2024-03-19T01:08:24.788402+0000","last_peered":"2024-03-19T01:08:24.788402+0000","last_clean":"2024-03-19T01:08:24.788402+0000","last_became_active":"2024-03-19T01:07:14.474577+0000","last_became_peered":"2024-03-19T01:07:14.474577+0000","last_unstale":"2024-03-19T01:08:24.788402+0000","last_undegraded":"2024-03-19T01:08:24.788402+0000","last_fullsized":"2024-03-19T01:08:24.788402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:01:04.713405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.650231+0000","last_change":"2024-03-19T01:07:15.506310+0000","last_active":"2024-03-19T01:08:53.650231+0000","last_peered":"2024-03-19T01:08:53.650231+0000","last_clean":"2024-03-19T01:08:53.650231+0000","last_became_active":"2024-03-19T01:07:15.505792+0000","last_became_peered":"2024-03-19T01:07:15.505792+0000","last_unstale":"2024-03-19T01:08:53.650231+0000","last_undegraded":"2024-03-19T01:08:53.650231+0000","last_fullsized":"2024-03-19T01:08:53.650231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:40.630816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.792151+0000","last_change":"2024-03-19T01:07:19.683632+0000","last_active":"2024-03-19T01:08:24.792151+0000","last_peered":"2024-03-19T01:08:24.792151+0000","last_clean":"2024-03-19T01:08:24.792151+0000","last_became_active":"2024-03-19T01:07:19.683204+0000","last_became_peered":"2024-03-19T01:07:19.683204+0000","last_unstale":"2024-03-19T01:08:24.792151+0000","last_undegraded":"2024-03-19T01:08:24.792151+0000","last_fullsized":"2024-03-19T01:08:24.792151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:58:53.512824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":197,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:11.845532+0000","last_change":"2024-03-19T01:07:14.471347+0000","last_active":"2024-03-19T01:09:11.845532+0000","last_peered":"2024-03-19T01:09:11.845532+0000","last_clean":"2024-03-19T01:09:11.845532+0000","last_became_active":"2024-03-19T01:07:14.471001+0000","last_became_peered":"2024-03-19T01:07:14.471001+0000","last_unstale":"2024-03-19T01:09:11.845532+0000","last_undegraded":"2024-03-19T01:09:11.845532+0000","last_fullsized":"2024-03-19T01:09:11.845532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:46.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.653843+0000","last_change":"2024-03-19T01:07:15.486779+0000","last_active":"2024-03-19T01:08:53.653843+0000","last_peered":"2024-03-19T01:08:53.653843+0000","last_clean":"2024-03-19T01:08:53.653843+0000","last_became_active":"2024-03-19T01:07:15.486405+0000","last_became_peered":"2024-03-19T01:07:15.486405+0000","last_unstale":"2024-03-19T01:08:53.653843+0000","last_undegraded":"2024-03-19T01:08:53.653843+0000","last_fullsized":"2024-03-19T01:08:53.653843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:14:13.350169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786062+0000","last_change":"2024-03-19T01:07:19.686892+0000","last_active":"2024-03-19T01:08:24.786062+0000","last_peered":"2024-03-19T01:08:24.786062+0000","last_clean":"2024-03-19T01:08:24.786062+0000","last_became_active":"2024-03-19T01:07:19.686414+0000","last_became_peered":"2024-03-19T01:07:19.686414+0000","last_unstale":"2024-03-19T01:08:24.786062+0000","last_undegraded":"2024-03-19T01:08:24.786062+0000","last_fullsized":"2024-03-19T01:08:24.786062+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:18:18.105905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.792588+0000","last_change":"2024-03-19T01:08:24.792588+0000","last_active":"2024-03-19T01:08:24.792588+0000","last_peered":"2024-03-19T01:08:24.792588+0000","last_clean":"2024-03-19T01:08:24.792588+0000","last_became_active":"2024-03-19T01:08:24.788228+0000","last_became_peered":"2024-03-19T01:08:24.788228+0000","last_unstale":"2024-03-19T01:08:24.792588+0000","last_undegraded":"2024-03-19T01:08:24.792588+0000","last_fullsized":"2024-03-19T01:08:24.792588+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:52.150165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.664391+0000","last_change":"2024-03-19T01:07:15.499507+0000","last_active":"2024-03-19T01:08:53.664391+0000","last_peered":"2024-03-19T01:08:53.664391+0000","last_clean":"2024-03-19T01:08:53.664391+0000","last_became_active":"2024-03-19T01:07:15.497261+0000","last_became_peered":"2024-03-19T01:07:15.497261+0000","last_unstale":"2024-03-19T01:08:53.664391+0000","last_undegraded":"2024-03-19T01:08:53.664391+0000","last_fullsized":"2024-03-19T01:08:53.664391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:40:06.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783334+0000","last_change":"2024-03-19T01:07:19.690847+0000","last_active":"2024-03-19T01:08:24.783334+0000","last_peered":"2024-03-19T01:08:24.783334+0000","last_clean":"2024-03-19T01:08:24.783334+0000","last_became_active":"2024-03-19T01:07:19.689762+0000","last_became_peered":"2024-03-19T01:07:19.689762+0000","last_unstale":"2024-03-19T01:08:24.783334+0000","last_undegraded":"2024-03-19T01:08:24.783334+0000","last_fullsized":"2024-03-19T01:08:24.783334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:24:40.213171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.789338+0000","last_change":"2024-03-19T01:07:14.473744+0000","last_active":"2024-03-19T01:08:24.789338+0000","last_peered":"2024-03-19T01:08:24.789338+0000","last_clean":"2024-03-19T01:08:24.789338+0000","last_became_active":"2024-03-19T01:07:14.473404+0000","last_became_peered":"2024-03-19T01:07:14.473404+0000","last_unstale":"2024-03-19T01:08:24.789338+0000","last_undegraded":"2024-03-19T01:08:24.789338+0000","last_fullsized":"2024-03-19T01:08:24.789338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:29:39.892678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.654720+0000","last_change":"2024-03-19T01:07:15.481444+0000","last_active":"2024-03-19T01:08:53.654720+0000","last_peered":"2024-03-19T01:08:53.654720+0000","last_clean":"2024-03-19T01:08:53.654720+0000","last_became_active":"2024-03-19T01:07:15.481106+0000","last_became_peered":"2024-03-19T01:07:15.481106+0000","last_unstale":"2024-03-19T01:08:53.654720+0000","last_undegraded":"2024-03-19T01:08:53.654720+0000","last_fullsized":"2024-03-19T01:08:53.654720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:00.263063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786527+0000","last_change":"2024-03-19T01:07:19.686919+0000","last_active":"2024-03-19T01:08:24.786527+0000","last_peered":"2024-03-19T01:08:24.786527+0000","last_clean":"2024-03-19T01:08:24.786527+0000","last_became_active":"2024-03-19T01:07:19.686416+0000","last_became_peered":"2024-03-19T01:07:19.686416+0000","last_unstale":"2024-03-19T01:08:24.786527+0000","last_undegraded":"2024-03-19T01:08:24.786527+0000","last_fullsized":"2024-03-19T01:08:24.786527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:29:11.068957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":241,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:11.846709+0000","last_change":"2024-03-19T01:07:14.468630+0000","last_active":"2024-03-19T01:09:11.846709+0000","last_peered":"2024-03-19T01:09:11.846709+0000","last_clean":"2024-03-19T01:09:11.846709+0000","last_became_active":"2024-03-19T01:07:14.468293+0000","last_became_peered":"2024-03-19T01:07:14.468293+0000","last_unstale":"2024-03-19T01:09:11.846709+0000","last_undegraded":"2024-03-19T01:09:11.846709+0000","last_fullsized":"2024-03-19T01:09:11.846709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:53.575507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":177,"num_read_kb":151,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.619313+0000","last_change":"2024-03-19T01:07:15.477226+0000","last_active":"2024-03-19T01:08:53.619313+0000","last_peered":"2024-03-19T01:08:53.619313+0000","last_clean":"2024-03-19T01:08:53.619313+0000","last_became_active":"2024-03-19T01:07:15.476882+0000","last_became_peered":"2024-03-19T01:07:15.476882+0000","last_unstale":"2024-03-19T01:08:53.619313+0000","last_undegraded":"2024-03-19T01:08:53.619313+0000","last_fullsized":"2024-03-19T01:08:53.619313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:11:10.051837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.789722+0000","last_change":"2024-03-19T01:07:19.690004+0000","last_active":"2024-03-19T01:08:24.789722+0000","last_peered":"2024-03-19T01:08:24.789722+0000","last_clean":"2024-03-19T01:08:24.789722+0000","last_became_active":"2024-03-19T01:07:19.689573+0000","last_became_peered":"2024-03-19T01:07:19.689573+0000","last_unstale":"2024-03-19T01:08:24.789722+0000","last_undegraded":"2024-03-19T01:08:24.789722+0000","last_fullsized":"2024-03-19T01:08:24.789722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:31:21.788219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782982+0000","last_change":"2024-03-19T01:07:14.487373+0000","last_active":"2024-03-19T01:08:24.782982+0000","last_peered":"2024-03-19T01:08:24.782982+0000","last_clean":"2024-03-19T01:08:24.782982+0000","last_became_active":"2024-03-19T01:07:14.487077+0000","last_became_peered":"2024-03-19T01:07:14.487077+0000","last_unstale":"2024-03-19T01:08:24.782982+0000","last_undegraded":"2024-03-19T01:08:24.782982+0000","last_fullsized":"2024-03-19T01:08:24.782982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:33:01.789431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.656863+0000","last_change":"2024-03-19T01:07:15.498009+0000","last_active":"2024-03-19T01:08:53.656863+0000","last_peered":"2024-03-19T01:08:53.656863+0000","last_clean":"2024-03-19T01:08:53.656863+0000","last_became_active":"2024-03-19T01:07:15.496707+0000","last_became_peered":"2024-03-19T01:07:15.496707+0000","last_unstale":"2024-03-19T01:08:53.656863+0000","last_undegraded":"2024-03-19T01:08:53.656863+0000","last_fullsized":"2024-03-19T01:08:53.656863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:50.058299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785106+0000","last_change":"2024-03-19T01:07:19.677635+0000","last_active":"2024-03-19T01:08:24.785106+0000","last_peered":"2024-03-19T01:08:24.785106+0000","last_clean":"2024-03-19T01:08:24.785106+0000","last_became_active":"2024-03-19T01:07:19.677207+0000","last_became_peered":"2024-03-19T01:07:19.677207+0000","last_unstale":"2024-03-19T01:08:24.785106+0000","last_undegraded":"2024-03-19T01:08:24.785106+0000","last_fullsized":"2024-03-19T01:08:24.785106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:23:28.650962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788907+0000","last_change":"2024-03-19T01:07:14.482054+0000","last_active":"2024-03-19T01:08:24.788907+0000","last_peered":"2024-03-19T01:08:24.788907+0000","last_clean":"2024-03-19T01:08:24.788907+0000","last_became_active":"2024-03-19T01:07:14.481703+0000","last_became_peered":"2024-03-19T01:07:14.481703+0000","last_unstale":"2024-03-19T01:08:24.788907+0000","last_undegraded":"2024-03-19T01:08:24.788907+0000","last_fullsized":"2024-03-19T01:08:24.788907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:02:43.565442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.657971+0000","last_change":"2024-03-19T01:07:15.487236+0000","last_active":"2024-03-19T01:08:53.657971+0000","last_peered":"2024-03-19T01:08:53.657971+0000","last_clean":"2024-03-19T01:08:53.657971+0000","last_became_active":"2024-03-19T01:07:15.486931+0000","last_became_peered":"2024-03-19T01:07:15.486931+0000","last_unstale":"2024-03-19T01:08:53.657971+0000","last_undegraded":"2024-03-19T01:08:53.657971+0000","last_fullsized":"2024-03-19T01:08:53.657971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:22.201904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788791+0000","last_change":"2024-03-19T01:07:19.693813+0000","last_active":"2024-03-19T01:08:24.788791+0000","last_peered":"2024-03-19T01:08:24.788791+0000","last_clean":"2024-03-19T01:08:24.788791+0000","last_became_active":"2024-03-19T01:07:19.693384+0000","last_became_peered":"2024-03-19T01:07:19.693384+0000","last_unstale":"2024-03-19T01:08:24.788791+0000","last_undegraded":"2024-03-19T01:08:24.788791+0000","last_fullsized":"2024-03-19T01:08:24.788791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:09.372461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:11.888899+0000","last_change":"2024-03-19T01:07:14.468467+0000","last_active":"2024-03-19T01:09:11.888899+0000","last_peered":"2024-03-19T01:09:11.888899+0000","last_clean":"2024-03-19T01:09:11.888899+0000","last_became_active":"2024-03-19T01:07:14.468074+0000","last_became_peered":"2024-03-19T01:07:14.468074+0000","last_unstale":"2024-03-19T01:09:11.888899+0000","last_undegraded":"2024-03-19T01:09:11.888899+0000","last_fullsized":"2024-03-19T01:09:11.888899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:07:49.216998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.642547+0000","last_change":"2024-03-19T01:07:15.475310+0000","last_active":"2024-03-19T01:08:53.642547+0000","last_peered":"2024-03-19T01:08:53.642547+0000","last_clean":"2024-03-19T01:08:53.642547+0000","last_became_active":"2024-03-19T01:07:15.474914+0000","last_became_peered":"2024-03-19T01:07:15.474914+0000","last_unstale":"2024-03-19T01:08:53.642547+0000","last_undegraded":"2024-03-19T01:08:53.642547+0000","last_fullsized":"2024-03-19T01:08:53.642547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:34.036892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.797430+0000","last_change":"2024-03-19T01:07:19.697937+0000","last_active":"2024-03-19T01:08:24.797430+0000","last_peered":"2024-03-19T01:08:24.797430+0000","last_clean":"2024-03-19T01:08:24.797430+0000","last_became_active":"2024-03-19T01:07:19.697447+0000","last_became_peered":"2024-03-19T01:07:19.697447+0000","last_unstale":"2024-03-19T01:08:24.797430+0000","last_undegraded":"2024-03-19T01:08:24.797430+0000","last_fullsized":"2024-03-19T01:08:24.797430+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:49.259118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785019+0000","last_change":"2024-03-19T01:07:14.476382+0000","last_active":"2024-03-19T01:08:24.785019+0000","last_peered":"2024-03-19T01:08:24.785019+0000","last_clean":"2024-03-19T01:08:24.785019+0000","last_became_active":"2024-03-19T01:07:14.476089+0000","last_became_peered":"2024-03-19T01:07:14.476089+0000","last_unstale":"2024-03-19T01:08:24.785019+0000","last_undegraded":"2024-03-19T01:08:24.785019+0000","last_fullsized":"2024-03-19T01:08:24.785019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:57:42.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.631229+0000","last_change":"2024-03-19T01:08:24.794335+0000","last_active":"2024-03-19T01:08:53.631229+0000","last_peered":"2024-03-19T01:08:53.631229+0000","last_clean":"2024-03-19T01:08:53.631229+0000","last_became_active":"2024-03-19T01:08:24.793725+0000","last_became_peered":"2024-03-19T01:08:24.793725+0000","last_unstale":"2024-03-19T01:08:53.631229+0000","last_undegraded":"2024-03-19T01:08:53.631229+0000","last_fullsized":"2024-03-19T01:08:53.631229+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:18.797824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788938+0000","last_change":"2024-03-19T01:07:19.688235+0000","last_active":"2024-03-19T01:08:24.788938+0000","last_peered":"2024-03-19T01:08:24.788938+0000","last_clean":"2024-03-19T01:08:24.788938+0000","last_became_active":"2024-03-19T01:07:19.687879+0000","last_became_peered":"2024-03-19T01:07:19.687879+0000","last_unstale":"2024-03-19T01:08:24.788938+0000","last_undegraded":"2024-03-19T01:08:24.788938+0000","last_fullsized":"2024-03-19T01:08:24.788938+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:29.465971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786220+0000","last_change":"2024-03-19T01:07:14.474346+0000","last_active":"2024-03-19T01:08:24.786220+0000","last_peered":"2024-03-19T01:08:24.786220+0000","last_clean":"2024-03-19T01:08:24.786220+0000","last_became_active":"2024-03-19T01:07:14.473989+0000","last_became_peered":"2024-03-19T01:07:14.473989+0000","last_unstale":"2024-03-19T01:08:24.786220+0000","last_undegraded":"2024-03-19T01:08:24.786220+0000","last_fullsized":"2024-03-19T01:08:24.786220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:41.004155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.645623+0000","last_change":"2024-03-19T01:07:15.482326+0000","last_active":"2024-03-19T01:08:53.645623+0000","last_peered":"2024-03-19T01:08:53.645623+0000","last_clean":"2024-03-19T01:08:53.645623+0000","last_became_active":"2024-03-19T01:07:15.481492+0000","last_became_peered":"2024-03-19T01:07:15.481492+0000","last_unstale":"2024-03-19T01:08:53.645623+0000","last_undegraded":"2024-03-19T01:08:53.645623+0000","last_fullsized":"2024-03-19T01:08:53.645623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:17:31.572104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786454+0000","last_change":"2024-03-19T01:07:19.668853+0000","last_active":"2024-03-19T01:08:24.786454+0000","last_peered":"2024-03-19T01:08:24.786454+0000","last_clean":"2024-03-19T01:08:24.786454+0000","last_became_active":"2024-03-19T01:07:19.668428+0000","last_became_peered":"2024-03-19T01:07:19.668428+0000","last_unstale":"2024-03-19T01:08:24.786454+0000","last_undegraded":"2024-03-19T01:08:24.786454+0000","last_fullsized":"2024-03-19T01:08:24.786454+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:50:43.409586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788150+0000","last_change":"2024-03-19T01:07:14.469548+0000","last_active":"2024-03-19T01:08:24.788150+0000","last_peered":"2024-03-19T01:08:24.788150+0000","last_clean":"2024-03-19T01:08:24.788150+0000","last_became_active":"2024-03-19T01:07:14.469200+0000","last_became_peered":"2024-03-19T01:07:14.469200+0000","last_unstale":"2024-03-19T01:08:24.788150+0000","last_undegraded":"2024-03-19T01:08:24.788150+0000","last_fullsized":"2024-03-19T01:08:24.788150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:41:53.179343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.649243+0000","last_change":"2024-03-19T01:07:15.495300+0000","last_active":"2024-03-19T01:08:53.649243+0000","last_peered":"2024-03-19T01:08:53.649243+0000","last_clean":"2024-03-19T01:08:53.649243+0000","last_became_active":"2024-03-19T01:07:15.494938+0000","last_became_peered":"2024-03-19T01:07:15.494938+0000","last_unstale":"2024-03-19T01:08:53.649243+0000","last_undegraded":"2024-03-19T01:08:53.649243+0000","last_fullsized":"2024-03-19T01:08:53.649243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:50:38.855857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.795694+0000","last_change":"2024-03-19T01:07:19.693261+0000","last_active":"2024-03-19T01:08:24.795694+0000","last_peered":"2024-03-19T01:08:24.795694+0000","last_clean":"2024-03-19T01:08:24.795694+0000","last_became_active":"2024-03-19T01:07:19.692598+0000","last_became_peered":"2024-03-19T01:07:19.692598+0000","last_unstale":"2024-03-19T01:08:24.795694+0000","last_undegraded":"2024-03-19T01:08:24.795694+0000","last_fullsized":"2024-03-19T01:08:24.795694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:43:29.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":115,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:10.937552+0000","last_change":"2024-03-19T01:07:14.481569+0000","last_active":"2024-03-19T01:09:10.937552+0000","last_peered":"2024-03-19T01:09:10.937552+0000","last_clean":"2024-03-19T01:09:10.937552+0000","last_became_active":"2024-03-19T01:07:14.481209+0000","last_became_peered":"2024-03-19T01:07:14.481209+0000","last_unstale":"2024-03-19T01:09:10.937552+0000","last_undegraded":"2024-03-19T01:09:10.937552+0000","last_fullsized":"2024-03-19T01:09:10.937552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:39.411207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.659688+0000","last_change":"2024-03-19T01:07:15.496852+0000","last_active":"2024-03-19T01:08:53.659688+0000","last_peered":"2024-03-19T01:08:53.659688+0000","last_clean":"2024-03-19T01:08:53.659688+0000","last_became_active":"2024-03-19T01:07:15.496455+0000","last_became_peered":"2024-03-19T01:07:15.496455+0000","last_unstale":"2024-03-19T01:08:53.659688+0000","last_undegraded":"2024-03-19T01:08:53.659688+0000","last_fullsized":"2024-03-19T01:08:53.659688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:43.419496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.797297+0000","last_change":"2024-03-19T01:07:19.692856+0000","last_active":"2024-03-19T01:08:24.797297+0000","last_peered":"2024-03-19T01:08:24.797297+0000","last_clean":"2024-03-19T01:08:24.797297+0000","last_became_active":"2024-03-19T01:07:19.690712+0000","last_became_peered":"2024-03-19T01:07:19.690712+0000","last_unstale":"2024-03-19T01:08:24.797297+0000","last_undegraded":"2024-03-19T01:08:24.797297+0000","last_fullsized":"2024-03-19T01:08:24.797297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:29:28.473062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.645836+0000","last_change":"2024-03-19T01:07:15.474375+0000","last_active":"2024-03-19T01:08:53.645836+0000","last_peered":"2024-03-19T01:08:53.645836+0000","last_clean":"2024-03-19T01:08:53.645836+0000","last_became_active":"2024-03-19T01:07:15.473427+0000","last_became_peered":"2024-03-19T01:07:15.473427+0000","last_unstale":"2024-03-19T01:08:53.645836+0000","last_undegraded":"2024-03-19T01:08:53.645836+0000","last_fullsized":"2024-03-19T01:08:53.645836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:42:22.235054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:10.886695+0000","last_change":"2024-03-19T01:07:14.490924+0000","last_active":"2024-03-19T01:09:10.886695+0000","last_peered":"2024-03-19T01:09:10.886695+0000","last_clean":"2024-03-19T01:09:10.886695+0000","last_became_active":"2024-03-19T01:07:14.490621+0000","last_became_peered":"2024-03-19T01:07:14.490621+0000","last_unstale":"2024-03-19T01:09:10.886695+0000","last_undegraded":"2024-03-19T01:09:10.886695+0000","last_fullsized":"2024-03-19T01:09:10.886695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:02:52.254138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.805196+0000","last_change":"2024-03-19T01:08:24.805196+0000","last_active":"2024-03-19T01:08:24.805196+0000","last_peered":"2024-03-19T01:08:24.805196+0000","last_clean":"2024-03-19T01:08:24.805196+0000","last_became_active":"2024-03-19T01:08:24.800225+0000","last_became_peered":"2024-03-19T01:08:24.800225+0000","last_unstale":"2024-03-19T01:08:24.805196+0000","last_undegraded":"2024-03-19T01:08:24.805196+0000","last_fullsized":"2024-03-19T01:08:24.805196+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:02.322846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.649908+0000","last_change":"2024-03-19T01:07:15.493218+0000","last_active":"2024-03-19T01:08:53.649908+0000","last_peered":"2024-03-19T01:08:53.649908+0000","last_clean":"2024-03-19T01:08:53.649908+0000","last_became_active":"2024-03-19T01:07:15.492751+0000","last_became_peered":"2024-03-19T01:07:15.492751+0000","last_unstale":"2024-03-19T01:08:53.649908+0000","last_undegraded":"2024-03-19T01:08:53.649908+0000","last_fullsized":"2024-03-19T01:08:53.649908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:48:32.105214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783301+0000","last_change":"2024-03-19T01:07:14.473349+0000","last_active":"2024-03-19T01:08:24.783301+0000","last_peered":"2024-03-19T01:08:24.783301+0000","last_clean":"2024-03-19T01:08:24.783301+0000","last_became_active":"2024-03-19T01:07:14.473014+0000","last_became_peered":"2024-03-19T01:07:14.473014+0000","last_unstale":"2024-03-19T01:08:24.783301+0000","last_undegraded":"2024-03-19T01:08:24.783301+0000","last_fullsized":"2024-03-19T01:08:24.783301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:27:34.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785047+0000","last_change":"2024-03-19T01:05:56.013594+0000","last_active":"2024-03-19T01:08:24.785047+0000","last_peered":"2024-03-19T01:08:24.785047+0000","last_clean":"2024-03-19T01:08:24.785047+0000","last_became_active":"2024-03-19T01:05:56.013084+0000","last_became_peered":"2024-03-19T01:05:56.013084+0000","last_unstale":"2024-03-19T01:08:24.785047+0000","last_undegraded":"2024-03-19T01:08:24.785047+0000","last_fullsized":"2024-03-19T01:08:24.785047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_clean_scrub_stamp":"2024-03-19T01:02:06.532343+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:23.440161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.789240+0000","last_change":"2024-03-19T01:07:19.691111+0000","last_active":"2024-03-19T01:08:24.789240+0000","last_peered":"2024-03-19T01:08:24.789240+0000","last_clean":"2024-03-19T01:08:24.789240+0000","last_became_active":"2024-03-19T01:07:19.690618+0000","last_became_peered":"2024-03-19T01:07:19.690618+0000","last_unstale":"2024-03-19T01:08:24.789240+0000","last_undegraded":"2024-03-19T01:08:24.789240+0000","last_fullsized":"2024-03-19T01:08:24.789240+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:17.857394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.638757+0000","last_change":"2024-03-19T01:08:24.801826+0000","last_active":"2024-03-19T01:08:53.638757+0000","last_peered":"2024-03-19T01:08:53.638757+0000","last_clean":"2024-03-19T01:08:53.638757+0000","last_became_active":"2024-03-19T01:08:24.801411+0000","last_became_peered":"2024-03-19T01:08:24.801411+0000","last_unstale":"2024-03-19T01:08:53.638757+0000","last_undegraded":"2024-03-19T01:08:53.638757+0000","last_fullsized":"2024-03-19T01:08:53.638757+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:31.798045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785058+0000","last_change":"2024-03-19T01:07:14.474287+0000","last_active":"2024-03-19T01:08:24.785058+0000","last_peered":"2024-03-19T01:08:24.785058+0000","last_clean":"2024-03-19T01:08:24.785058+0000","last_became_active":"2024-03-19T01:07:14.473921+0000","last_became_peered":"2024-03-19T01:07:14.473921+0000","last_unstale":"2024-03-19T01:08:24.785058+0000","last_undegraded":"2024-03-19T01:08:24.785058+0000","last_fullsized":"2024-03-19T01:08:24.785058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:49.322029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785011+0000","last_change":"2024-03-19T01:07:19.685331+0000","last_active":"2024-03-19T01:08:24.785011+0000","last_peered":"2024-03-19T01:08:24.785011+0000","last_clean":"2024-03-19T01:08:24.785011+0000","last_became_active":"2024-03-19T01:07:19.684876+0000","last_became_peered":"2024-03-19T01:07:19.684876+0000","last_unstale":"2024-03-19T01:08:24.785011+0000","last_undegraded":"2024-03-19T01:08:24.785011+0000","last_fullsized":"2024-03-19T01:08:24.785011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:39.575348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785730+0000","last_change":"2024-03-19T01:07:14.481019+0000","last_active":"2024-03-19T01:08:24.785730+0000","last_peered":"2024-03-19T01:08:24.785730+0000","last_clean":"2024-03-19T01:08:24.785730+0000","last_became_active":"2024-03-19T01:07:14.480666+0000","last_became_peered":"2024-03-19T01:07:14.480666+0000","last_unstale":"2024-03-19T01:08:24.785730+0000","last_undegraded":"2024-03-19T01:08:24.785730+0000","last_fullsized":"2024-03-19T01:08:24.785730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:13:33.669690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.645918+0000","last_change":"2024-03-19T01:07:15.490747+0000","last_active":"2024-03-19T01:08:53.645918+0000","last_peered":"2024-03-19T01:08:53.645918+0000","last_clean":"2024-03-19T01:08:53.645918+0000","last_became_active":"2024-03-19T01:07:15.486244+0000","last_became_peered":"2024-03-19T01:07:15.486244+0000","last_unstale":"2024-03-19T01:08:53.645918+0000","last_undegraded":"2024-03-19T01:08:53.645918+0000","last_fullsized":"2024-03-19T01:08:53.645918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:43.212972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.792337+0000","last_change":"2024-03-19T01:07:19.684804+0000","last_active":"2024-03-19T01:08:24.792337+0000","last_peered":"2024-03-19T01:08:24.792337+0000","last_clean":"2024-03-19T01:08:24.792337+0000","last_became_active":"2024-03-19T01:07:19.683191+0000","last_became_peered":"2024-03-19T01:07:19.683191+0000","last_unstale":"2024-03-19T01:08:24.792337+0000","last_undegraded":"2024-03-19T01:08:24.792337+0000","last_fullsized":"2024-03-19T01:08:24.792337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:02.015499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'14","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788392+0000","last_change":"2024-03-19T01:07:14.479282+0000","last_active":"2024-03-19T01:08:24.788392+0000","last_peered":"2024-03-19T01:08:24.788392+0000","last_clean":"2024-03-19T01:08:24.788392+0000","last_became_active":"2024-03-19T01:07:14.478876+0000","last_became_peered":"2024-03-19T01:07:14.478876+0000","last_unstale":"2024-03-19T01:08:24.788392+0000","last_undegraded":"2024-03-19T01:08:24.788392+0000","last_fullsized":"2024-03-19T01:08:24.788392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:54.179680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.651438+0000","last_change":"2024-03-19T01:07:15.493498+0000","last_active":"2024-03-19T01:08:53.651438+0000","last_peered":"2024-03-19T01:08:53.651438+0000","last_clean":"2024-03-19T01:08:53.651438+0000","last_became_active":"2024-03-19T01:07:15.493110+0000","last_became_peered":"2024-03-19T01:07:15.493110+0000","last_unstale":"2024-03-19T01:08:53.651438+0000","last_undegraded":"2024-03-19T01:08:53.651438+0000","last_fullsized":"2024-03-19T01:08:53.651438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:55:23.608596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782607+0000","last_change":"2024-03-19T01:07:19.671232+0000","last_active":"2024-03-19T01:08:24.782607+0000","last_peered":"2024-03-19T01:08:24.782607+0000","last_clean":"2024-03-19T01:08:24.782607+0000","last_became_active":"2024-03-19T01:07:19.670818+0000","last_became_peered":"2024-03-19T01:07:19.670818+0000","last_unstale":"2024-03-19T01:08:24.782607+0000","last_undegraded":"2024-03-19T01:08:24.782607+0000","last_fullsized":"2024-03-19T01:08:24.782607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:57:20.134372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783210+0000","last_change":"2024-03-19T01:07:14.473568+0000","last_active":"2024-03-19T01:08:24.783210+0000","last_peered":"2024-03-19T01:08:24.783210+0000","last_clean":"2024-03-19T01:08:24.783210+0000","last_became_active":"2024-03-19T01:07:14.472555+0000","last_became_peered":"2024-03-19T01:07:14.472555+0000","last_unstale":"2024-03-19T01:08:24.783210+0000","last_undegraded":"2024-03-19T01:08:24.783210+0000","last_fullsized":"2024-03-19T01:08:24.783210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:28:01.541228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.665645+0000","last_change":"2024-03-19T01:07:15.499905+0000","last_active":"2024-03-19T01:08:53.665645+0000","last_peered":"2024-03-19T01:08:53.665645+0000","last_clean":"2024-03-19T01:08:53.665645+0000","last_became_active":"2024-03-19T01:07:15.499560+0000","last_became_peered":"2024-03-19T01:07:15.499560+0000","last_unstale":"2024-03-19T01:08:53.665645+0000","last_undegraded":"2024-03-19T01:08:53.665645+0000","last_fullsized":"2024-03-19T01:08:53.665645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:06:26.531100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783121+0000","last_change":"2024-03-19T01:07:19.690310+0000","last_active":"2024-03-19T01:08:24.783121+0000","last_peered":"2024-03-19T01:08:24.783121+0000","last_clean":"2024-03-19T01:08:24.783121+0000","last_became_active":"2024-03-19T01:07:19.689877+0000","last_became_peered":"2024-03-19T01:07:19.689877+0000","last_unstale":"2024-03-19T01:08:24.783121+0000","last_undegraded":"2024-03-19T01:08:24.783121+0000","last_fullsized":"2024-03-19T01:08:24.783121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:06:12.730258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786831+0000","last_change":"2024-03-19T01:07:14.466267+0000","last_active":"2024-03-19T01:08:24.786831+0000","last_peered":"2024-03-19T01:08:24.786831+0000","last_clean":"2024-03-19T01:08:24.786831+0000","last_became_active":"2024-03-19T01:07:14.465944+0000","last_became_peered":"2024-03-19T01:07:14.465944+0000","last_unstale":"2024-03-19T01:08:24.786831+0000","last_undegraded":"2024-03-19T01:08:24.786831+0000","last_fullsized":"2024-03-19T01:08:24.786831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:40.957200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.605173+0000","last_change":"2024-03-19T01:07:15.475690+0000","last_active":"2024-03-19T01:08:53.605173+0000","last_peered":"2024-03-19T01:08:53.605173+0000","last_clean":"2024-03-19T01:08:53.605173+0000","last_became_active":"2024-03-19T01:07:15.475342+0000","last_became_peered":"2024-03-19T01:07:15.475342+0000","last_unstale":"2024-03-19T01:08:53.605173+0000","last_undegraded":"2024-03-19T01:08:53.605173+0000","last_fullsized":"2024-03-19T01:08:53.605173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:30:30.842462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.798173+0000","last_change":"2024-03-19T01:08:24.797680+0000","last_active":"2024-03-19T01:08:24.798173+0000","last_peered":"2024-03-19T01:08:24.798173+0000","last_clean":"2024-03-19T01:08:24.798173+0000","last_became_active":"2024-03-19T01:08:24.797194+0000","last_became_peered":"2024-03-19T01:08:24.797194+0000","last_unstale":"2024-03-19T01:08:24.798173+0000","last_undegraded":"2024-03-19T01:08:24.798173+0000","last_fullsized":"2024-03-19T01:08:24.798173+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:49.800726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.792871+0000","last_change":"2024-03-19T01:07:14.487907+0000","last_active":"2024-03-19T01:08:24.792871+0000","last_peered":"2024-03-19T01:08:24.792871+0000","last_clean":"2024-03-19T01:08:24.792871+0000","last_became_active":"2024-03-19T01:07:14.487618+0000","last_became_peered":"2024-03-19T01:07:14.487618+0000","last_unstale":"2024-03-19T01:08:24.792871+0000","last_undegraded":"2024-03-19T01:08:24.792871+0000","last_fullsized":"2024-03-19T01:08:24.792871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:22:49.218797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.659806+0000","last_change":"2024-03-19T01:07:15.492103+0000","last_active":"2024-03-19T01:08:53.659806+0000","last_peered":"2024-03-19T01:08:53.659806+0000","last_clean":"2024-03-19T01:08:53.659806+0000","last_became_active":"2024-03-19T01:07:15.491712+0000","last_became_peered":"2024-03-19T01:07:15.491712+0000","last_unstale":"2024-03-19T01:08:53.659806+0000","last_undegraded":"2024-03-19T01:08:53.659806+0000","last_fullsized":"2024-03-19T01:08:53.659806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:40:13.959986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782915+0000","last_change":"2024-03-19T01:07:19.677392+0000","last_active":"2024-03-19T01:08:24.782915+0000","last_peered":"2024-03-19T01:08:24.782915+0000","last_clean":"2024-03-19T01:08:24.782915+0000","last_became_active":"2024-03-19T01:07:19.677024+0000","last_became_peered":"2024-03-19T01:07:19.677024+0000","last_unstale":"2024-03-19T01:08:24.782915+0000","last_undegraded":"2024-03-19T01:08:24.782915+0000","last_fullsized":"2024-03-19T01:08:24.782915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:01:43.404902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.805878+0000","last_change":"2024-03-19T01:08:24.805363+0000","last_active":"2024-03-19T01:08:24.805878+0000","last_peered":"2024-03-19T01:08:24.805878+0000","last_clean":"2024-03-19T01:08:24.805878+0000","last_became_active":"2024-03-19T01:08:24.800810+0000","last_became_peered":"2024-03-19T01:08:24.800810+0000","last_unstale":"2024-03-19T01:08:24.805878+0000","last_undegraded":"2024-03-19T01:08:24.805878+0000","last_fullsized":"2024-03-19T01:08:24.805878+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:32:43.608140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.659108+0000","last_change":"2024-03-19T01:07:15.505842+0000","last_active":"2024-03-19T01:08:53.659108+0000","last_peered":"2024-03-19T01:08:53.659108+0000","last_clean":"2024-03-19T01:08:53.659108+0000","last_became_active":"2024-03-19T01:07:15.505425+0000","last_became_peered":"2024-03-19T01:07:15.505425+0000","last_unstale":"2024-03-19T01:08:53.659108+0000","last_undegraded":"2024-03-19T01:08:53.659108+0000","last_fullsized":"2024-03-19T01:08:53.659108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:28.119864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782948+0000","last_change":"2024-03-19T01:07:19.683113+0000","last_active":"2024-03-19T01:08:24.782948+0000","last_peered":"2024-03-19T01:08:24.782948+0000","last_clean":"2024-03-19T01:08:24.782948+0000","last_became_active":"2024-03-19T01:07:19.682702+0000","last_became_peered":"2024-03-19T01:07:19.682702+0000","last_unstale":"2024-03-19T01:08:24.782948+0000","last_undegraded":"2024-03-19T01:08:24.782948+0000","last_fullsized":"2024-03-19T01:08:24.782948+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:42.866408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785093+0000","last_change":"2024-03-19T01:07:14.476562+0000","last_active":"2024-03-19T01:08:24.785093+0000","last_peered":"2024-03-19T01:08:24.785093+0000","last_clean":"2024-03-19T01:08:24.785093+0000","last_became_active":"2024-03-19T01:07:14.476185+0000","last_became_peered":"2024-03-19T01:07:14.476185+0000","last_unstale":"2024-03-19T01:08:24.785093+0000","last_undegraded":"2024-03-19T01:08:24.785093+0000","last_fullsized":"2024-03-19T01:08:24.785093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:35:19.860443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.654246+0000","last_change":"2024-03-19T01:07:15.487163+0000","last_active":"2024-03-19T01:08:53.654246+0000","last_peered":"2024-03-19T01:08:53.654246+0000","last_clean":"2024-03-19T01:08:53.654246+0000","last_became_active":"2024-03-19T01:07:15.486845+0000","last_became_peered":"2024-03-19T01:07:15.486845+0000","last_unstale":"2024-03-19T01:08:53.654246+0000","last_undegraded":"2024-03-19T01:08:53.654246+0000","last_fullsized":"2024-03-19T01:08:53.654246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:39:09.389411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.787057+0000","last_change":"2024-03-19T01:07:19.692768+0000","last_active":"2024-03-19T01:08:24.787057+0000","last_peered":"2024-03-19T01:08:24.787057+0000","last_clean":"2024-03-19T01:08:24.787057+0000","last_became_active":"2024-03-19T01:07:19.692382+0000","last_became_peered":"2024-03-19T01:07:19.692382+0000","last_unstale":"2024-03-19T01:08:24.787057+0000","last_undegraded":"2024-03-19T01:08:24.787057+0000","last_fullsized":"2024-03-19T01:08:24.787057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:26.804938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":119,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:13.900322+0000","last_change":"2024-03-19T01:07:14.472844+0000","last_active":"2024-03-19T01:09:13.900322+0000","last_peered":"2024-03-19T01:09:13.900322+0000","last_clean":"2024-03-19T01:09:13.900322+0000","last_became_active":"2024-03-19T01:07:14.472389+0000","last_became_peered":"2024-03-19T01:07:14.472389+0000","last_unstale":"2024-03-19T01:09:13.900322+0000","last_undegraded":"2024-03-19T01:09:13.900322+0000","last_fullsized":"2024-03-19T01:09:13.900322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:31.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.653619+0000","last_change":"2024-03-19T01:07:15.482033+0000","last_active":"2024-03-19T01:08:53.653619+0000","last_peered":"2024-03-19T01:08:53.653619+0000","last_clean":"2024-03-19T01:08:53.653619+0000","last_became_active":"2024-03-19T01:07:15.481671+0000","last_became_peered":"2024-03-19T01:07:15.481671+0000","last_unstale":"2024-03-19T01:08:53.653619+0000","last_undegraded":"2024-03-19T01:08:53.653619+0000","last_fullsized":"2024-03-19T01:08:53.653619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:09.969374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.793574+0000","last_change":"2024-03-19T01:08:24.793574+0000","last_active":"2024-03-19T01:08:24.793574+0000","last_peered":"2024-03-19T01:08:24.793574+0000","last_clean":"2024-03-19T01:08:24.793574+0000","last_became_active":"2024-03-19T01:08:24.793012+0000","last_became_peered":"2024-03-19T01:08:24.793012+0000","last_unstale":"2024-03-19T01:08:24.793574+0000","last_undegraded":"2024-03-19T01:08:24.793574+0000","last_fullsized":"2024-03-19T01:08:24.793574+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:04:53.611271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.789545+0000","last_change":"2024-03-19T01:07:14.481677+0000","last_active":"2024-03-19T01:08:24.789545+0000","last_peered":"2024-03-19T01:08:24.789545+0000","last_clean":"2024-03-19T01:08:24.789545+0000","last_became_active":"2024-03-19T01:07:14.481138+0000","last_became_peered":"2024-03-19T01:07:14.481138+0000","last_unstale":"2024-03-19T01:08:24.789545+0000","last_undegraded":"2024-03-19T01:08:24.789545+0000","last_fullsized":"2024-03-19T01:08:24.789545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:52:33.105816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.652140+0000","last_change":"2024-03-19T01:07:15.498352+0000","last_active":"2024-03-19T01:08:53.652140+0000","last_peered":"2024-03-19T01:08:53.652140+0000","last_clean":"2024-03-19T01:08:53.652140+0000","last_became_active":"2024-03-19T01:07:15.498040+0000","last_became_peered":"2024-03-19T01:07:15.498040+0000","last_unstale":"2024-03-19T01:08:53.652140+0000","last_undegraded":"2024-03-19T01:08:53.652140+0000","last_fullsized":"2024-03-19T01:08:53.652140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:57:37.571753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788102+0000","last_change":"2024-03-19T01:07:19.673291+0000","last_active":"2024-03-19T01:08:24.788102+0000","last_peered":"2024-03-19T01:08:24.788102+0000","last_clean":"2024-03-19T01:08:24.788102+0000","last_became_active":"2024-03-19T01:07:19.672704+0000","last_became_peered":"2024-03-19T01:07:19.672704+0000","last_unstale":"2024-03-19T01:08:24.788102+0000","last_undegraded":"2024-03-19T01:08:24.788102+0000","last_fullsized":"2024-03-19T01:08:24.788102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:01:42.019812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.797532+0000","last_change":"2024-03-19T01:07:14.489556+0000","last_active":"2024-03-19T01:08:24.797532+0000","last_peered":"2024-03-19T01:08:24.797532+0000","last_clean":"2024-03-19T01:08:24.797532+0000","last_became_active":"2024-03-19T01:07:14.489201+0000","last_became_peered":"2024-03-19T01:07:14.489201+0000","last_unstale":"2024-03-19T01:08:24.797532+0000","last_undegraded":"2024-03-19T01:08:24.797532+0000","last_fullsized":"2024-03-19T01:08:24.797532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:28:36.516185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.647535+0000","last_change":"2024-03-19T01:07:15.477727+0000","last_active":"2024-03-19T01:08:53.647535+0000","last_peered":"2024-03-19T01:08:53.647535+0000","last_clean":"2024-03-19T01:08:53.647535+0000","last_became_active":"2024-03-19T01:07:15.477388+0000","last_became_peered":"2024-03-19T01:07:15.477388+0000","last_unstale":"2024-03-19T01:08:53.647535+0000","last_undegraded":"2024-03-19T01:08:53.647535+0000","last_fullsized":"2024-03-19T01:08:53.647535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:30.567326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.636741+0000","last_change":"2024-03-19T01:07:15.479008+0000","last_active":"2024-03-19T01:08:53.636741+0000","last_peered":"2024-03-19T01:08:53.636741+0000","last_clean":"2024-03-19T01:08:53.636741+0000","last_became_active":"2024-03-19T01:07:15.478657+0000","last_became_peered":"2024-03-19T01:07:15.478657+0000","last_unstale":"2024-03-19T01:08:53.636741+0000","last_undegraded":"2024-03-19T01:08:53.636741+0000","last_fullsized":"2024-03-19T01:08:53.636741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:42:46.970012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.640084+0000","last_change":"2024-03-19T01:08:24.789796+0000","last_active":"2024-03-19T01:08:53.640084+0000","last_peered":"2024-03-19T01:08:53.640084+0000","last_clean":"2024-03-19T01:08:53.640084+0000","last_became_active":"2024-03-19T01:08:24.789444+0000","last_became_peered":"2024-03-19T01:08:24.789444+0000","last_unstale":"2024-03-19T01:08:53.640084+0000","last_undegraded":"2024-03-19T01:08:53.640084+0000","last_fullsized":"2024-03-19T01:08:53.640084+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:41.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.658984+0000","last_change":"2024-03-19T01:07:15.495292+0000","last_active":"2024-03-19T01:08:53.658984+0000","last_peered":"2024-03-19T01:08:53.658984+0000","last_clean":"2024-03-19T01:08:53.658984+0000","last_became_active":"2024-03-19T01:07:15.494946+0000","last_became_peered":"2024-03-19T01:07:15.494946+0000","last_unstale":"2024-03-19T01:08:53.658984+0000","last_undegraded":"2024-03-19T01:08:53.658984+0000","last_fullsized":"2024-03-19T01:08:53.658984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:39.012221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.664494+0000","last_change":"2024-03-19T01:07:15.499816+0000","last_active":"2024-03-19T01:08:53.664494+0000","last_peered":"2024-03-19T01:08:53.664494+0000","last_clean":"2024-03-19T01:08:53.664494+0000","last_became_active":"2024-03-19T01:07:15.499031+0000","last_became_peered":"2024-03-19T01:07:15.499031+0000","last_unstale":"2024-03-19T01:08:53.664494+0000","last_undegraded":"2024-03-19T01:08:53.664494+0000","last_fullsized":"2024-03-19T01:08:53.664494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:34:25.344092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.662893+0000","last_change":"2024-03-19T01:07:15.501526+0000","last_active":"2024-03-19T01:08:53.662893+0000","last_peered":"2024-03-19T01:08:53.662893+0000","last_clean":"2024-03-19T01:08:53.662893+0000","last_became_active":"2024-03-19T01:07:15.500452+0000","last_became_peered":"2024-03-19T01:07:15.500452+0000","last_unstale":"2024-03-19T01:08:53.662893+0000","last_undegraded":"2024-03-19T01:08:53.662893+0000","last_fullsized":"2024-03-19T01:08:53.662893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:35:33.075780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.633014+0000","last_change":"2024-03-19T01:07:15.477649+0000","last_active":"2024-03-19T01:08:53.633014+0000","last_peered":"2024-03-19T01:08:53.633014+0000","last_clean":"2024-03-19T01:08:53.633014+0000","last_became_active":"2024-03-19T01:07:15.477193+0000","last_became_peered":"2024-03-19T01:07:15.477193+0000","last_unstale":"2024-03-19T01:08:53.633014+0000","last_undegraded":"2024-03-19T01:08:53.633014+0000","last_fullsized":"2024-03-19T01:08:53.633014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:36.126642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.641408+0000","last_change":"2024-03-19T01:07:15.481920+0000","last_active":"2024-03-19T01:08:53.641408+0000","last_peered":"2024-03-19T01:08:53.641408+0000","last_clean":"2024-03-19T01:08:53.641408+0000","last_became_active":"2024-03-19T01:07:15.481290+0000","last_became_peered":"2024-03-19T01:07:15.481290+0000","last_unstale":"2024-03-19T01:08:53.641408+0000","last_undegraded":"2024-03-19T01:08:53.641408+0000","last_fullsized":"2024-03-19T01:08:53.641408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:37:58.028647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.650606+0000","last_change":"2024-03-19T01:07:15.502305+0000","last_active":"2024-03-19T01:08:53.650606+0000","last_peered":"2024-03-19T01:08:53.650606+0000","last_clean":"2024-03-19T01:08:53.650606+0000","last_became_active":"2024-03-19T01:07:15.501491+0000","last_became_peered":"2024-03-19T01:07:15.501491+0000","last_unstale":"2024-03-19T01:08:53.650606+0000","last_undegraded":"2024-03-19T01:08:53.650606+0000","last_fullsized":"2024-03-19T01:08:53.650606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:01.999538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.649089+0000","last_change":"2024-03-19T01:07:15.499590+0000","last_active":"2024-03-19T01:08:53.649089+0000","last_peered":"2024-03-19T01:08:53.649089+0000","last_clean":"2024-03-19T01:08:53.649089+0000","last_became_active":"2024-03-19T01:07:15.498991+0000","last_became_peered":"2024-03-19T01:07:15.498991+0000","last_unstale":"2024-03-19T01:08:53.649089+0000","last_undegraded":"2024-03-19T01:08:53.649089+0000","last_fullsized":"2024-03-19T01:08:53.649089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:09:43.510480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.651658+0000","last_change":"2024-03-19T01:07:15.505022+0000","last_active":"2024-03-19T01:08:53.651658+0000","last_peered":"2024-03-19T01:08:53.651658+0000","last_clean":"2024-03-19T01:08:53.651658+0000","last_became_active":"2024-03-19T01:07:15.503684+0000","last_became_peered":"2024-03-19T01:07:15.503684+0000","last_unstale":"2024-03-19T01:08:53.651658+0000","last_undegraded":"2024-03-19T01:08:53.651658+0000","last_fullsized":"2024-03-19T01:08:53.651658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:56:47.580267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.653435+0000","last_change":"2024-03-19T01:07:15.495634+0000","last_active":"2024-03-19T01:08:53.653435+0000","last_peered":"2024-03-19T01:08:53.653435+0000","last_clean":"2024-03-19T01:08:53.653435+0000","last_became_active":"2024-03-19T01:07:15.495206+0000","last_became_peered":"2024-03-19T01:07:15.495206+0000","last_unstale":"2024-03-19T01:08:53.653435+0000","last_undegraded":"2024-03-19T01:08:53.653435+0000","last_fullsized":"2024-03-19T01:08:53.653435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:30:29.965731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.665906+0000","last_change":"2024-03-19T01:07:15.501480+0000","last_active":"2024-03-19T01:08:53.665906+0000","last_peered":"2024-03-19T01:08:53.665906+0000","last_clean":"2024-03-19T01:08:53.665906+0000","last_became_active":"2024-03-19T01:07:15.501078+0000","last_became_peered":"2024-03-19T01:07:15.501078+0000","last_unstale":"2024-03-19T01:08:53.665906+0000","last_undegraded":"2024-03-19T01:08:53.665906+0000","last_fullsized":"2024-03-19T01:08:53.665906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:40:18.767874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.645793+0000","last_change":"2024-03-19T01:07:15.480358+0000","last_active":"2024-03-19T01:08:53.645793+0000","last_peered":"2024-03-19T01:08:53.645793+0000","last_clean":"2024-03-19T01:08:53.645793+0000","last_became_active":"2024-03-19T01:07:15.479979+0000","last_became_peered":"2024-03-19T01:07:15.479979+0000","last_unstale":"2024-03-19T01:08:53.645793+0000","last_undegraded":"2024-03-19T01:08:53.645793+0000","last_fullsized":"2024-03-19T01:08:53.645793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:12:54.598435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247132,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69828,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":875,"num_read_kb":513,"num_write":249,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":209484,"data_compressed":5469,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":501433,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":1155751,"data_compressed":58168,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":505459,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":501741,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1098379,"data_compressed":56345,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":496867,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":560771,"data_compressed":22394,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":498323,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":560991,"data_compressed":22394,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":496575,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":501473,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1089333,"data_compressed":56345,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64626,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64846,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65142,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T01:09:17.733 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-19T01:09:18.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:17 smithi086 ceph-mon[66829]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:09:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:17 smithi069 ceph-mon[31727]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:09:18.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:17 smithi163 ceph-mon[67428]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:09:18.212 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:09:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:18 smithi069 ceph-mon[31727]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:18 smithi163 ceph-mon[67428]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:19.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:18 smithi086 ceph-mon[66829]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:20.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.442+0000 7f03c6200700 1 -- 172.21.15.69:0/1438218755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c0076340 msgr2=0x7f03c0074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:20.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.442+0000 7f03c6200700 1 --2- 172.21.15.69:0/1438218755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c0076340 0x7f03c0074540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009990 tx=0x7f03b0009ca0 comp rx=0 tx=0).stop 2024-03-19T01:09:20.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.442+0000 7f03c6200700 1 -- 172.21.15.69:0/1438218755 shutdown_connections 2024-03-19T01:09:20.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.442+0000 7f03c6200700 1 --2- 172.21.15.69:0/1438218755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c0112fa0 0x7f03c0115390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:20.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.442+0000 7f03c6200700 1 --2- 172.21.15.69:0/1438218755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c0076340 0x7f03c0074540 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:20.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.442+0000 7f03c6200700 1 --2- 172.21.15.69:0/1438218755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c0074a80 0x7f03c0074f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:20.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.442+0000 7f03c6200700 1 -- 172.21.15.69:0/1438218755 >> 172.21.15.69:0/1438218755 conn(0x7f03c0079440 msgr2=0x7f03c0079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:20.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.442+0000 7f03c6200700 1 -- 172.21.15.69:0/1438218755 shutdown_connections 2024-03-19T01:09:20.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.442+0000 7f03c6200700 1 -- 172.21.15.69:0/1438218755 wait complete. 2024-03-19T01:09:20.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.443+0000 7f03c6200700 1 Processor -- start 2024-03-19T01:09:20.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.443+0000 7f03c6200700 1 -- start start 2024-03-19T01:09:20.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03c6200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c0074a80 0x7f03c019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:20.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03c6200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c0076340 0x7f03c019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:20.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03c6200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c0112fa0 0x7f03c01a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:20.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03c6200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0118c20 con 0x7f03c0112fa0 2024-03-19T01:09:20.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03c6200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0118aa0 con 0x7f03c0076340 2024-03-19T01:09:20.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03c6200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0118da0 con 0x7f03c0074a80 2024-03-19T01:09:20.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03bf400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c0074a80 0x7f03c019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:20.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03bf400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c0074a80 0x7f03c019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53454/0 (socket says 172.21.15.69:53454) 2024-03-19T01:09:20.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03bf400700 1 -- 172.21.15.69:0/4253494973 learned_addr learned my addr 172.21.15.69:0/4253494973 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:20.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03b7e00700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c0076340 0x7f03c019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:20.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03bfe00700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c0112fa0 0x7f03c01a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:20.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03bf400700 1 -- 172.21.15.69:0/4253494973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c0076340 msgr2=0x7f03c019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:20.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03bf400700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c0076340 0x7f03c019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:20.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03bf400700 1 -- 172.21.15.69:0/4253494973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c0112fa0 msgr2=0x7f03c01a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:20.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.444+0000 7f03bf400700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c0112fa0 0x7f03c01a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:20.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.445+0000 7f03bf400700 1 -- 172.21.15.69:0/4253494973 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c01a4140 con 0x7f03c0074a80 2024-03-19T01:09:20.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.445+0000 7f03bf400700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c0074a80 0x7f03c019f130 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f03b0005910 tx=0x7f03b000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:20.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.445+0000 7f03bcc00700 1 -- 172.21.15.69:0/4253494973 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0027070 con 0x7f03c0074a80 2024-03-19T01:09:20.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.445+0000 7f03bcc00700 1 -- 172.21.15.69:0/4253494973 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b0003d70 con 0x7f03c0074a80 2024-03-19T01:09:20.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.445+0000 7f03bcc00700 1 -- 172.21.15.69:0/4253494973 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b000f460 con 0x7f03c0074a80 2024-03-19T01:09:20.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.446+0000 7f03c6200700 1 -- 172.21.15.69:0/4253494973 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c01a43d0 con 0x7f03c0074a80 2024-03-19T01:09:20.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.446+0000 7f03c6200700 1 -- 172.21.15.69:0/4253494973 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c01abcb0 con 0x7f03c0074a80 2024-03-19T01:09:20.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.447+0000 7f03c6200700 1 -- 172.21.15.69:0/4253494973 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c0075bf0 con 0x7f03c0074a80 2024-03-19T01:09:20.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.449+0000 7f03bcc00700 1 -- 172.21.15.69:0/4253494973 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f03b002a070 con 0x7f03c0074a80 2024-03-19T01:09:20.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.450+0000 7f03bcc00700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03a40a2890 0x7f03a40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:20.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.450+0000 7f03bcc00700 1 -- 172.21.15.69:0/4253494973 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f03b00cfbc0 con 0x7f03c0074a80 2024-03-19T01:09:20.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.450+0000 7f03b7e00700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03a40a2890 0x7f03a40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:20.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.451+0000 7f03b7e00700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03a40a2890 0x7f03a40a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f03c0079d40 tx=0x7f03a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:20.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.454+0000 7f03bcc00700 1 -- 172.21.15.69:0/4253494973 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f03b0013070 con 0x7f03c0074a80 2024-03-19T01:09:20.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.699+0000 7f03c6200700 1 -- 172.21.15.69:0/4253494973 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03c0112470 con 0x7f03a40a2890 2024-03-19T01:09:20.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.715+0000 7f03bcc00700 1 -- 172.21.15.69:0/4253494973 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340819 (secure 0 0 0) 0x7f03c0112470 con 0x7f03a40a2890 2024-03-19T01:09:20.721 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 -- 172.21.15.69:0/4253494973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03a40a2890 msgr2=0x7f03a40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03a40a2890 0x7f03a40a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f03c0079d40 tx=0x7f03a8009260 comp rx=0 tx=0).stop 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 -- 172.21.15.69:0/4253494973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c0074a80 msgr2=0x7f03c019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c0074a80 0x7f03c019f130 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f03b0005910 tx=0x7f03b000bab0 comp rx=0 tx=0).stop 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 -- 172.21.15.69:0/4253494973 shutdown_connections 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03a40a2890 0x7f03a40a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c0112fa0 0x7f03c01a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c0074a80 0x7f03c019f130 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 --2- 172.21.15.69:0/4253494973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c0076340 0x7f03c019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 -- 172.21.15.69:0/4253494973 >> 172.21.15.69:0/4253494973 conn(0x7f03c0079440 msgr2=0x7f03c0111d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 -- 172.21.15.69:0/4253494973 shutdown_connections 2024-03-19T01:09:20.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:20.721+0000 7f03c6200700 1 -- 172.21.15.69:0/4253494973 wait complete. 2024-03-19T01:09:20.725 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-03-19T01:09:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:20 smithi069 ceph-mon[31727]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:09:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:20 smithi163 ceph-mon[67428]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:09:21.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:20 smithi086 ceph-mon[66829]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:09:21.539 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-03-19T01:09:19.685201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660430,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1056,"num_read_kb":657,"num_write":432,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378088,"kb_used_data":12296,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563736,"statfs":{"total":1151940427776,"available":1151553265664,"internally_reserved":0,"allocated":12591104,"data_stored":7967096,"data_compressed":359643,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005830"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.660362+0000","last_change":"2024-03-19T01:07:15.500098+0000","last_active":"2024-03-19T01:08:53.660362+0000","last_peered":"2024-03-19T01:08:53.660362+0000","last_clean":"2024-03-19T01:08:53.660362+0000","last_became_active":"2024-03-19T01:07:15.499738+0000","last_became_peered":"2024-03-19T01:07:15.499738+0000","last_unstale":"2024-03-19T01:08:53.660362+0000","last_undegraded":"2024-03-19T01:08:53.660362+0000","last_fullsized":"2024-03-19T01:08:53.660362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:14.703153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.658198+0000","last_change":"2024-03-19T01:07:15.492863+0000","last_active":"2024-03-19T01:08:53.658198+0000","last_peered":"2024-03-19T01:08:53.658198+0000","last_clean":"2024-03-19T01:08:53.658198+0000","last_became_active":"2024-03-19T01:07:15.492533+0000","last_became_peered":"2024-03-19T01:07:15.492533+0000","last_unstale":"2024-03-19T01:08:53.658198+0000","last_undegraded":"2024-03-19T01:08:53.658198+0000","last_fullsized":"2024-03-19T01:08:53.658198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:49.028352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.664970+0000","last_change":"2024-03-19T01:07:15.494174+0000","last_active":"2024-03-19T01:08:53.664970+0000","last_peered":"2024-03-19T01:08:53.664970+0000","last_clean":"2024-03-19T01:08:53.664970+0000","last_became_active":"2024-03-19T01:07:15.490562+0000","last_became_peered":"2024-03-19T01:07:15.490562+0000","last_unstale":"2024-03-19T01:08:53.664970+0000","last_undegraded":"2024-03-19T01:08:53.664970+0000","last_fullsized":"2024-03-19T01:08:53.664970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:18.664396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.624807+0000","last_change":"2024-03-19T01:07:15.478397+0000","last_active":"2024-03-19T01:08:53.624807+0000","last_peered":"2024-03-19T01:08:53.624807+0000","last_clean":"2024-03-19T01:08:53.624807+0000","last_became_active":"2024-03-19T01:07:15.477000+0000","last_became_peered":"2024-03-19T01:07:15.477000+0000","last_unstale":"2024-03-19T01:08:53.624807+0000","last_undegraded":"2024-03-19T01:08:53.624807+0000","last_fullsized":"2024-03-19T01:08:53.624807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:58.534329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.650862+0000","last_change":"2024-03-19T01:07:15.500138+0000","last_active":"2024-03-19T01:08:53.650862+0000","last_peered":"2024-03-19T01:08:53.650862+0000","last_clean":"2024-03-19T01:08:53.650862+0000","last_became_active":"2024-03-19T01:07:15.499759+0000","last_became_peered":"2024-03-19T01:07:15.499759+0000","last_unstale":"2024-03-19T01:08:53.650862+0000","last_undegraded":"2024-03-19T01:08:53.650862+0000","last_fullsized":"2024-03-19T01:08:53.650862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:23.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.639845+0000","last_change":"2024-03-19T01:07:15.485146+0000","last_active":"2024-03-19T01:08:53.639845+0000","last_peered":"2024-03-19T01:08:53.639845+0000","last_clean":"2024-03-19T01:08:53.639845+0000","last_became_active":"2024-03-19T01:07:15.484680+0000","last_became_peered":"2024-03-19T01:07:15.484680+0000","last_unstale":"2024-03-19T01:08:53.639845+0000","last_undegraded":"2024-03-19T01:08:53.639845+0000","last_fullsized":"2024-03-19T01:08:53.639845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:12.659347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.648331+0000","last_change":"2024-03-19T01:07:15.486327+0000","last_active":"2024-03-19T01:08:53.648331+0000","last_peered":"2024-03-19T01:08:53.648331+0000","last_clean":"2024-03-19T01:08:53.648331+0000","last_became_active":"2024-03-19T01:07:15.485954+0000","last_became_peered":"2024-03-19T01:07:15.485954+0000","last_unstale":"2024-03-19T01:08:53.648331+0000","last_undegraded":"2024-03-19T01:08:53.648331+0000","last_fullsized":"2024-03-19T01:08:53.648331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:53.316260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.651352+0000","last_change":"2024-03-19T01:07:15.504993+0000","last_active":"2024-03-19T01:08:53.651352+0000","last_peered":"2024-03-19T01:08:53.651352+0000","last_clean":"2024-03-19T01:08:53.651352+0000","last_became_active":"2024-03-19T01:07:15.502771+0000","last_became_peered":"2024-03-19T01:07:15.502771+0000","last_unstale":"2024-03-19T01:08:53.651352+0000","last_undegraded":"2024-03-19T01:08:53.651352+0000","last_fullsized":"2024-03-19T01:08:53.651352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:22:33.712608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.648570+0000","last_change":"2024-03-19T01:07:15.483876+0000","last_active":"2024-03-19T01:08:53.648570+0000","last_peered":"2024-03-19T01:08:53.648570+0000","last_clean":"2024-03-19T01:08:53.648570+0000","last_became_active":"2024-03-19T01:07:15.482744+0000","last_became_peered":"2024-03-19T01:07:15.482744+0000","last_unstale":"2024-03-19T01:08:53.648570+0000","last_undegraded":"2024-03-19T01:08:53.648570+0000","last_fullsized":"2024-03-19T01:08:53.648570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:21:19.870877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.638591+0000","last_change":"2024-03-19T01:07:15.475655+0000","last_active":"2024-03-19T01:08:53.638591+0000","last_peered":"2024-03-19T01:08:53.638591+0000","last_clean":"2024-03-19T01:08:53.638591+0000","last_became_active":"2024-03-19T01:07:15.475265+0000","last_became_peered":"2024-03-19T01:07:15.475265+0000","last_unstale":"2024-03-19T01:08:53.638591+0000","last_undegraded":"2024-03-19T01:08:53.638591+0000","last_fullsized":"2024-03-19T01:08:53.638591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:24:38.898191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.653740+0000","last_change":"2024-03-19T01:07:15.482988+0000","last_active":"2024-03-19T01:08:53.653740+0000","last_peered":"2024-03-19T01:08:53.653740+0000","last_clean":"2024-03-19T01:08:53.653740+0000","last_became_active":"2024-03-19T01:07:15.482673+0000","last_became_peered":"2024-03-19T01:07:15.482673+0000","last_unstale":"2024-03-19T01:08:53.653740+0000","last_undegraded":"2024-03-19T01:08:53.653740+0000","last_fullsized":"2024-03-19T01:08:53.653740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:15:48.807200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.622597+0000","last_change":"2024-03-19T01:07:15.479593+0000","last_active":"2024-03-19T01:08:53.622597+0000","last_peered":"2024-03-19T01:08:53.622597+0000","last_clean":"2024-03-19T01:08:53.622597+0000","last_became_active":"2024-03-19T01:07:15.479220+0000","last_became_peered":"2024-03-19T01:07:15.479220+0000","last_unstale":"2024-03-19T01:08:53.622597+0000","last_undegraded":"2024-03-19T01:08:53.622597+0000","last_fullsized":"2024-03-19T01:08:53.622597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:40:24.390845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.661191+0000","last_change":"2024-03-19T01:07:15.464737+0000","last_active":"2024-03-19T01:08:53.661191+0000","last_peered":"2024-03-19T01:08:53.661191+0000","last_clean":"2024-03-19T01:08:53.661191+0000","last_became_active":"2024-03-19T01:07:15.463641+0000","last_became_peered":"2024-03-19T01:07:15.463641+0000","last_unstale":"2024-03-19T01:08:53.661191+0000","last_undegraded":"2024-03-19T01:08:53.661191+0000","last_fullsized":"2024-03-19T01:08:53.661191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:13.964397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.661921+0000","last_change":"2024-03-19T01:07:15.492803+0000","last_active":"2024-03-19T01:08:53.661921+0000","last_peered":"2024-03-19T01:08:53.661921+0000","last_clean":"2024-03-19T01:08:53.661921+0000","last_became_active":"2024-03-19T01:07:15.492422+0000","last_became_peered":"2024-03-19T01:07:15.492422+0000","last_unstale":"2024-03-19T01:08:53.661921+0000","last_undegraded":"2024-03-19T01:08:53.661921+0000","last_fullsized":"2024-03-19T01:08:53.661921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:59:08.292399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.664378+0000","last_change":"2024-03-19T01:07:15.498073+0000","last_active":"2024-03-19T01:08:53.664378+0000","last_peered":"2024-03-19T01:08:53.664378+0000","last_clean":"2024-03-19T01:08:53.664378+0000","last_became_active":"2024-03-19T01:07:15.493650+0000","last_became_peered":"2024-03-19T01:07:15.493650+0000","last_unstale":"2024-03-19T01:08:53.664378+0000","last_undegraded":"2024-03-19T01:08:53.664378+0000","last_fullsized":"2024-03-19T01:08:53.664378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:25:28.908122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.662071+0000","last_change":"2024-03-19T01:07:15.496434+0000","last_active":"2024-03-19T01:08:53.662071+0000","last_peered":"2024-03-19T01:08:53.662071+0000","last_clean":"2024-03-19T01:08:53.662071+0000","last_became_active":"2024-03-19T01:07:15.495956+0000","last_became_peered":"2024-03-19T01:07:15.495956+0000","last_unstale":"2024-03-19T01:08:53.662071+0000","last_undegraded":"2024-03-19T01:08:53.662071+0000","last_fullsized":"2024-03-19T01:08:53.662071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:49.505947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.660439+0000","last_change":"2024-03-19T01:07:15.504771+0000","last_active":"2024-03-19T01:08:53.660439+0000","last_peered":"2024-03-19T01:08:53.660439+0000","last_clean":"2024-03-19T01:08:53.660439+0000","last_became_active":"2024-03-19T01:07:15.504485+0000","last_became_peered":"2024-03-19T01:07:15.504485+0000","last_unstale":"2024-03-19T01:08:53.660439+0000","last_undegraded":"2024-03-19T01:08:53.660439+0000","last_fullsized":"2024-03-19T01:08:53.660439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:17:36.760943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.640237+0000","last_change":"2024-03-19T01:07:15.491045+0000","last_active":"2024-03-19T01:08:53.640237+0000","last_peered":"2024-03-19T01:08:53.640237+0000","last_clean":"2024-03-19T01:08:53.640237+0000","last_became_active":"2024-03-19T01:07:15.488914+0000","last_became_peered":"2024-03-19T01:07:15.488914+0000","last_unstale":"2024-03-19T01:08:53.640237+0000","last_undegraded":"2024-03-19T01:08:53.640237+0000","last_fullsized":"2024-03-19T01:08:53.640237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:14.853457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.645777+0000","last_change":"2024-03-19T01:07:15.495306+0000","last_active":"2024-03-19T01:08:53.645777+0000","last_peered":"2024-03-19T01:08:53.645777+0000","last_clean":"2024-03-19T01:08:53.645777+0000","last_became_active":"2024-03-19T01:07:15.494906+0000","last_became_peered":"2024-03-19T01:07:15.494906+0000","last_unstale":"2024-03-19T01:08:53.645777+0000","last_undegraded":"2024-03-19T01:08:53.645777+0000","last_fullsized":"2024-03-19T01:08:53.645777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:48.176410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782715+0000","last_change":"2024-03-19T01:07:19.689402+0000","last_active":"2024-03-19T01:08:24.782715+0000","last_peered":"2024-03-19T01:08:24.782715+0000","last_clean":"2024-03-19T01:08:24.782715+0000","last_became_active":"2024-03-19T01:07:19.688799+0000","last_became_peered":"2024-03-19T01:07:19.688799+0000","last_unstale":"2024-03-19T01:08:24.782715+0000","last_undegraded":"2024-03-19T01:08:24.782715+0000","last_fullsized":"2024-03-19T01:08:24.782715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:51.890047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":128,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:15.944191+0000","last_change":"2024-03-19T01:07:14.477348+0000","last_active":"2024-03-19T01:09:15.944191+0000","last_peered":"2024-03-19T01:09:15.944191+0000","last_clean":"2024-03-19T01:09:15.944191+0000","last_became_active":"2024-03-19T01:07:14.476946+0000","last_became_peered":"2024-03-19T01:07:14.476946+0000","last_unstale":"2024-03-19T01:09:15.944191+0000","last_undegraded":"2024-03-19T01:09:15.944191+0000","last_fullsized":"2024-03-19T01:09:15.944191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:19.269837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.647467+0000","last_change":"2024-03-19T01:07:15.495646+0000","last_active":"2024-03-19T01:08:53.647467+0000","last_peered":"2024-03-19T01:08:53.647467+0000","last_clean":"2024-03-19T01:08:53.647467+0000","last_became_active":"2024-03-19T01:07:15.495203+0000","last_became_peered":"2024-03-19T01:07:15.495203+0000","last_unstale":"2024-03-19T01:08:53.647467+0000","last_undegraded":"2024-03-19T01:08:53.647467+0000","last_fullsized":"2024-03-19T01:08:53.647467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:01:31.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782686+0000","last_change":"2024-03-19T01:07:19.690465+0000","last_active":"2024-03-19T01:08:24.782686+0000","last_peered":"2024-03-19T01:08:24.782686+0000","last_clean":"2024-03-19T01:08:24.782686+0000","last_became_active":"2024-03-19T01:07:19.689852+0000","last_became_peered":"2024-03-19T01:07:19.689852+0000","last_unstale":"2024-03-19T01:08:24.782686+0000","last_undegraded":"2024-03-19T01:08:24.782686+0000","last_fullsized":"2024-03-19T01:08:24.782686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:18.891411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.796469+0000","last_change":"2024-03-19T01:07:14.487165+0000","last_active":"2024-03-19T01:08:24.796469+0000","last_peered":"2024-03-19T01:08:24.796469+0000","last_clean":"2024-03-19T01:08:24.796469+0000","last_became_active":"2024-03-19T01:07:14.484194+0000","last_became_peered":"2024-03-19T01:07:14.484194+0000","last_unstale":"2024-03-19T01:08:24.796469+0000","last_undegraded":"2024-03-19T01:08:24.796469+0000","last_fullsized":"2024-03-19T01:08:24.796469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:54:47.914565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.665671+0000","last_change":"2024-03-19T01:07:15.498687+0000","last_active":"2024-03-19T01:08:53.665671+0000","last_peered":"2024-03-19T01:08:53.665671+0000","last_clean":"2024-03-19T01:08:53.665671+0000","last_became_active":"2024-03-19T01:07:15.498146+0000","last_became_peered":"2024-03-19T01:07:15.498146+0000","last_unstale":"2024-03-19T01:08:53.665671+0000","last_undegraded":"2024-03-19T01:08:53.665671+0000","last_fullsized":"2024-03-19T01:08:53.665671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:47:21.846333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783201+0000","last_change":"2024-03-19T01:07:19.690691+0000","last_active":"2024-03-19T01:08:24.783201+0000","last_peered":"2024-03-19T01:08:24.783201+0000","last_clean":"2024-03-19T01:08:24.783201+0000","last_became_active":"2024-03-19T01:07:19.690083+0000","last_became_peered":"2024-03-19T01:07:19.690083+0000","last_unstale":"2024-03-19T01:08:24.783201+0000","last_undegraded":"2024-03-19T01:08:24.783201+0000","last_fullsized":"2024-03-19T01:08:24.783201+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:51:30.205694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.801994+0000","last_change":"2024-03-19T01:08:24.801510+0000","last_active":"2024-03-19T01:08:24.801994+0000","last_peered":"2024-03-19T01:08:24.801994+0000","last_clean":"2024-03-19T01:08:24.801994+0000","last_became_active":"2024-03-19T01:08:24.789315+0000","last_became_peered":"2024-03-19T01:08:24.789315+0000","last_unstale":"2024-03-19T01:08:24.801994+0000","last_undegraded":"2024-03-19T01:08:24.801994+0000","last_fullsized":"2024-03-19T01:08:24.801994+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:24:03.243394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.656857+0000","last_change":"2024-03-19T01:07:15.489371+0000","last_active":"2024-03-19T01:08:53.656857+0000","last_peered":"2024-03-19T01:08:53.656857+0000","last_clean":"2024-03-19T01:08:53.656857+0000","last_became_active":"2024-03-19T01:07:15.488975+0000","last_became_peered":"2024-03-19T01:07:15.488975+0000","last_unstale":"2024-03-19T01:08:53.656857+0000","last_undegraded":"2024-03-19T01:08:53.656857+0000","last_fullsized":"2024-03-19T01:08:53.656857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:04:47.640512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782771+0000","last_change":"2024-03-19T01:07:19.691590+0000","last_active":"2024-03-19T01:08:24.782771+0000","last_peered":"2024-03-19T01:08:24.782771+0000","last_clean":"2024-03-19T01:08:24.782771+0000","last_became_active":"2024-03-19T01:07:19.691037+0000","last_became_peered":"2024-03-19T01:07:19.691037+0000","last_unstale":"2024-03-19T01:08:24.782771+0000","last_undegraded":"2024-03-19T01:08:24.782771+0000","last_fullsized":"2024-03-19T01:08:24.782771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:20:02.007483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786021+0000","last_change":"2024-03-19T01:07:14.475653+0000","last_active":"2024-03-19T01:08:24.786021+0000","last_peered":"2024-03-19T01:08:24.786021+0000","last_clean":"2024-03-19T01:08:24.786021+0000","last_became_active":"2024-03-19T01:07:14.475315+0000","last_became_peered":"2024-03-19T01:07:14.475315+0000","last_unstale":"2024-03-19T01:08:24.786021+0000","last_undegraded":"2024-03-19T01:08:24.786021+0000","last_fullsized":"2024-03-19T01:08:24.786021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:44:04.656950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.646245+0000","last_change":"2024-03-19T01:07:15.480882+0000","last_active":"2024-03-19T01:08:53.646245+0000","last_peered":"2024-03-19T01:08:53.646245+0000","last_clean":"2024-03-19T01:08:53.646245+0000","last_became_active":"2024-03-19T01:07:15.480541+0000","last_became_peered":"2024-03-19T01:07:15.480541+0000","last_unstale":"2024-03-19T01:08:53.646245+0000","last_undegraded":"2024-03-19T01:08:53.646245+0000","last_fullsized":"2024-03-19T01:08:53.646245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:37:24.724531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783363+0000","last_change":"2024-03-19T01:07:19.672979+0000","last_active":"2024-03-19T01:08:24.783363+0000","last_peered":"2024-03-19T01:08:24.783363+0000","last_clean":"2024-03-19T01:08:24.783363+0000","last_became_active":"2024-03-19T01:07:19.672530+0000","last_became_peered":"2024-03-19T01:07:19.672530+0000","last_unstale":"2024-03-19T01:08:24.783363+0000","last_undegraded":"2024-03-19T01:08:24.783363+0000","last_fullsized":"2024-03-19T01:08:24.783363+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:12.948079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:15.849866+0000","last_change":"2024-03-19T01:07:14.475620+0000","last_active":"2024-03-19T01:09:15.849866+0000","last_peered":"2024-03-19T01:09:15.849866+0000","last_clean":"2024-03-19T01:09:15.849866+0000","last_became_active":"2024-03-19T01:07:14.475029+0000","last_became_peered":"2024-03-19T01:07:14.475029+0000","last_unstale":"2024-03-19T01:09:15.849866+0000","last_undegraded":"2024-03-19T01:09:15.849866+0000","last_fullsized":"2024-03-19T01:09:15.849866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:48:53.983457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.634950+0000","last_change":"2024-03-19T01:07:15.484680+0000","last_active":"2024-03-19T01:08:53.634950+0000","last_peered":"2024-03-19T01:08:53.634950+0000","last_clean":"2024-03-19T01:08:53.634950+0000","last_became_active":"2024-03-19T01:07:15.484333+0000","last_became_peered":"2024-03-19T01:07:15.484333+0000","last_unstale":"2024-03-19T01:08:53.634950+0000","last_undegraded":"2024-03-19T01:08:53.634950+0000","last_fullsized":"2024-03-19T01:08:53.634950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:38:23.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.798971+0000","last_change":"2024-03-19T01:08:24.798971+0000","last_active":"2024-03-19T01:08:24.798971+0000","last_peered":"2024-03-19T01:08:24.798971+0000","last_clean":"2024-03-19T01:08:24.798971+0000","last_became_active":"2024-03-19T01:08:24.798504+0000","last_became_peered":"2024-03-19T01:08:24.798504+0000","last_unstale":"2024-03-19T01:08:24.798971+0000","last_undegraded":"2024-03-19T01:08:24.798971+0000","last_fullsized":"2024-03-19T01:08:24.798971+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.460490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.799220+0000","last_change":"2024-03-19T01:07:14.481499+0000","last_active":"2024-03-19T01:08:24.799220+0000","last_peered":"2024-03-19T01:08:24.799220+0000","last_clean":"2024-03-19T01:08:24.799220+0000","last_became_active":"2024-03-19T01:07:14.481123+0000","last_became_peered":"2024-03-19T01:07:14.481123+0000","last_unstale":"2024-03-19T01:08:24.799220+0000","last_undegraded":"2024-03-19T01:08:24.799220+0000","last_fullsized":"2024-03-19T01:08:24.799220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:36:15.349748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.648991+0000","last_change":"2024-03-19T01:07:15.499744+0000","last_active":"2024-03-19T01:08:53.648991+0000","last_peered":"2024-03-19T01:08:53.648991+0000","last_clean":"2024-03-19T01:08:53.648991+0000","last_became_active":"2024-03-19T01:07:15.499037+0000","last_became_peered":"2024-03-19T01:07:15.499037+0000","last_unstale":"2024-03-19T01:08:53.648991+0000","last_undegraded":"2024-03-19T01:08:53.648991+0000","last_fullsized":"2024-03-19T01:08:53.648991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:20:35.524664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785238+0000","last_change":"2024-03-19T01:07:19.684187+0000","last_active":"2024-03-19T01:08:24.785238+0000","last_peered":"2024-03-19T01:08:24.785238+0000","last_clean":"2024-03-19T01:08:24.785238+0000","last_became_active":"2024-03-19T01:07:19.683782+0000","last_became_peered":"2024-03-19T01:07:19.683782+0000","last_unstale":"2024-03-19T01:08:24.785238+0000","last_undegraded":"2024-03-19T01:08:24.785238+0000","last_fullsized":"2024-03-19T01:08:24.785238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:17:50.818003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.795451+0000","last_change":"2024-03-19T01:07:14.485538+0000","last_active":"2024-03-19T01:08:24.795451+0000","last_peered":"2024-03-19T01:08:24.795451+0000","last_clean":"2024-03-19T01:08:24.795451+0000","last_became_active":"2024-03-19T01:07:14.485004+0000","last_became_peered":"2024-03-19T01:07:14.485004+0000","last_unstale":"2024-03-19T01:08:24.795451+0000","last_undegraded":"2024-03-19T01:08:24.795451+0000","last_fullsized":"2024-03-19T01:08:24.795451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:55:50.853725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.658270+0000","last_change":"2024-03-19T01:07:15.503370+0000","last_active":"2024-03-19T01:08:53.658270+0000","last_peered":"2024-03-19T01:08:53.658270+0000","last_clean":"2024-03-19T01:08:53.658270+0000","last_became_active":"2024-03-19T01:07:15.501375+0000","last_became_peered":"2024-03-19T01:07:15.501375+0000","last_unstale":"2024-03-19T01:08:53.658270+0000","last_undegraded":"2024-03-19T01:08:53.658270+0000","last_fullsized":"2024-03-19T01:08:53.658270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:36:01.814936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.794012+0000","last_change":"2024-03-19T01:07:19.697067+0000","last_active":"2024-03-19T01:08:24.794012+0000","last_peered":"2024-03-19T01:08:24.794012+0000","last_clean":"2024-03-19T01:08:24.794012+0000","last_became_active":"2024-03-19T01:07:19.696364+0000","last_became_peered":"2024-03-19T01:07:19.696364+0000","last_unstale":"2024-03-19T01:08:24.794012+0000","last_undegraded":"2024-03-19T01:08:24.794012+0000","last_fullsized":"2024-03-19T01:08:24.794012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:34.388407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788402+0000","last_change":"2024-03-19T01:07:14.475361+0000","last_active":"2024-03-19T01:08:24.788402+0000","last_peered":"2024-03-19T01:08:24.788402+0000","last_clean":"2024-03-19T01:08:24.788402+0000","last_became_active":"2024-03-19T01:07:14.474577+0000","last_became_peered":"2024-03-19T01:07:14.474577+0000","last_unstale":"2024-03-19T01:08:24.788402+0000","last_undegraded":"2024-03-19T01:08:24.788402+0000","last_fullsized":"2024-03-19T01:08:24.788402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:01:04.713405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.650231+0000","last_change":"2024-03-19T01:07:15.506310+0000","last_active":"2024-03-19T01:08:53.650231+0000","last_peered":"2024-03-19T01:08:53.650231+0000","last_clean":"2024-03-19T01:08:53.650231+0000","last_became_active":"2024-03-19T01:07:15.505792+0000","last_became_peered":"2024-03-19T01:07:15.505792+0000","last_unstale":"2024-03-19T01:08:53.650231+0000","last_undegraded":"2024-03-19T01:08:53.650231+0000","last_fullsized":"2024-03-19T01:08:53.650231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:40.630816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.792151+0000","last_change":"2024-03-19T01:07:19.683632+0000","last_active":"2024-03-19T01:08:24.792151+0000","last_peered":"2024-03-19T01:08:24.792151+0000","last_clean":"2024-03-19T01:08:24.792151+0000","last_became_active":"2024-03-19T01:07:19.683204+0000","last_became_peered":"2024-03-19T01:07:19.683204+0000","last_unstale":"2024-03-19T01:08:24.792151+0000","last_undegraded":"2024-03-19T01:08:24.792151+0000","last_fullsized":"2024-03-19T01:08:24.792151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:58:53.512824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":207,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:16.850323+0000","last_change":"2024-03-19T01:07:14.471347+0000","last_active":"2024-03-19T01:09:16.850323+0000","last_peered":"2024-03-19T01:09:16.850323+0000","last_clean":"2024-03-19T01:09:16.850323+0000","last_became_active":"2024-03-19T01:07:14.471001+0000","last_became_peered":"2024-03-19T01:07:14.471001+0000","last_unstale":"2024-03-19T01:09:16.850323+0000","last_undegraded":"2024-03-19T01:09:16.850323+0000","last_fullsized":"2024-03-19T01:09:16.850323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:46.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.653843+0000","last_change":"2024-03-19T01:07:15.486779+0000","last_active":"2024-03-19T01:08:53.653843+0000","last_peered":"2024-03-19T01:08:53.653843+0000","last_clean":"2024-03-19T01:08:53.653843+0000","last_became_active":"2024-03-19T01:07:15.486405+0000","last_became_peered":"2024-03-19T01:07:15.486405+0000","last_unstale":"2024-03-19T01:08:53.653843+0000","last_undegraded":"2024-03-19T01:08:53.653843+0000","last_fullsized":"2024-03-19T01:08:53.653843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:14:13.350169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786062+0000","last_change":"2024-03-19T01:07:19.686892+0000","last_active":"2024-03-19T01:08:24.786062+0000","last_peered":"2024-03-19T01:08:24.786062+0000","last_clean":"2024-03-19T01:08:24.786062+0000","last_became_active":"2024-03-19T01:07:19.686414+0000","last_became_peered":"2024-03-19T01:07:19.686414+0000","last_unstale":"2024-03-19T01:08:24.786062+0000","last_undegraded":"2024-03-19T01:08:24.786062+0000","last_fullsized":"2024-03-19T01:08:24.786062+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:18:18.105905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.792588+0000","last_change":"2024-03-19T01:08:24.792588+0000","last_active":"2024-03-19T01:08:24.792588+0000","last_peered":"2024-03-19T01:08:24.792588+0000","last_clean":"2024-03-19T01:08:24.792588+0000","last_became_active":"2024-03-19T01:08:24.788228+0000","last_became_peered":"2024-03-19T01:08:24.788228+0000","last_unstale":"2024-03-19T01:08:24.792588+0000","last_undegraded":"2024-03-19T01:08:24.792588+0000","last_fullsized":"2024-03-19T01:08:24.792588+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:52.150165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.664391+0000","last_change":"2024-03-19T01:07:15.499507+0000","last_active":"2024-03-19T01:08:53.664391+0000","last_peered":"2024-03-19T01:08:53.664391+0000","last_clean":"2024-03-19T01:08:53.664391+0000","last_became_active":"2024-03-19T01:07:15.497261+0000","last_became_peered":"2024-03-19T01:07:15.497261+0000","last_unstale":"2024-03-19T01:08:53.664391+0000","last_undegraded":"2024-03-19T01:08:53.664391+0000","last_fullsized":"2024-03-19T01:08:53.664391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:40:06.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783334+0000","last_change":"2024-03-19T01:07:19.690847+0000","last_active":"2024-03-19T01:08:24.783334+0000","last_peered":"2024-03-19T01:08:24.783334+0000","last_clean":"2024-03-19T01:08:24.783334+0000","last_became_active":"2024-03-19T01:07:19.689762+0000","last_became_peered":"2024-03-19T01:07:19.689762+0000","last_unstale":"2024-03-19T01:08:24.783334+0000","last_undegraded":"2024-03-19T01:08:24.783334+0000","last_fullsized":"2024-03-19T01:08:24.783334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:24:40.213171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.789338+0000","last_change":"2024-03-19T01:07:14.473744+0000","last_active":"2024-03-19T01:08:24.789338+0000","last_peered":"2024-03-19T01:08:24.789338+0000","last_clean":"2024-03-19T01:08:24.789338+0000","last_became_active":"2024-03-19T01:07:14.473404+0000","last_became_peered":"2024-03-19T01:07:14.473404+0000","last_unstale":"2024-03-19T01:08:24.789338+0000","last_undegraded":"2024-03-19T01:08:24.789338+0000","last_fullsized":"2024-03-19T01:08:24.789338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:29:39.892678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.654720+0000","last_change":"2024-03-19T01:07:15.481444+0000","last_active":"2024-03-19T01:08:53.654720+0000","last_peered":"2024-03-19T01:08:53.654720+0000","last_clean":"2024-03-19T01:08:53.654720+0000","last_became_active":"2024-03-19T01:07:15.481106+0000","last_became_peered":"2024-03-19T01:07:15.481106+0000","last_unstale":"2024-03-19T01:08:53.654720+0000","last_undegraded":"2024-03-19T01:08:53.654720+0000","last_fullsized":"2024-03-19T01:08:53.654720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:00.263063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786527+0000","last_change":"2024-03-19T01:07:19.686919+0000","last_active":"2024-03-19T01:08:24.786527+0000","last_peered":"2024-03-19T01:08:24.786527+0000","last_clean":"2024-03-19T01:08:24.786527+0000","last_became_active":"2024-03-19T01:07:19.686416+0000","last_became_peered":"2024-03-19T01:07:19.686416+0000","last_unstale":"2024-03-19T01:08:24.786527+0000","last_undegraded":"2024-03-19T01:08:24.786527+0000","last_fullsized":"2024-03-19T01:08:24.786527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:29:11.068957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":251,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:16.850825+0000","last_change":"2024-03-19T01:07:14.468630+0000","last_active":"2024-03-19T01:09:16.850825+0000","last_peered":"2024-03-19T01:09:16.850825+0000","last_clean":"2024-03-19T01:09:16.850825+0000","last_became_active":"2024-03-19T01:07:14.468293+0000","last_became_peered":"2024-03-19T01:07:14.468293+0000","last_unstale":"2024-03-19T01:09:16.850825+0000","last_undegraded":"2024-03-19T01:09:16.850825+0000","last_fullsized":"2024-03-19T01:09:16.850825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:53.575507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":187,"num_read_kb":156,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.619313+0000","last_change":"2024-03-19T01:07:15.477226+0000","last_active":"2024-03-19T01:08:53.619313+0000","last_peered":"2024-03-19T01:08:53.619313+0000","last_clean":"2024-03-19T01:08:53.619313+0000","last_became_active":"2024-03-19T01:07:15.476882+0000","last_became_peered":"2024-03-19T01:07:15.476882+0000","last_unstale":"2024-03-19T01:08:53.619313+0000","last_undegraded":"2024-03-19T01:08:53.619313+0000","last_fullsized":"2024-03-19T01:08:53.619313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:11:10.051837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.789722+0000","last_change":"2024-03-19T01:07:19.690004+0000","last_active":"2024-03-19T01:08:24.789722+0000","last_peered":"2024-03-19T01:08:24.789722+0000","last_clean":"2024-03-19T01:08:24.789722+0000","last_became_active":"2024-03-19T01:07:19.689573+0000","last_became_peered":"2024-03-19T01:07:19.689573+0000","last_unstale":"2024-03-19T01:08:24.789722+0000","last_undegraded":"2024-03-19T01:08:24.789722+0000","last_fullsized":"2024-03-19T01:08:24.789722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:31:21.788219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782982+0000","last_change":"2024-03-19T01:07:14.487373+0000","last_active":"2024-03-19T01:08:24.782982+0000","last_peered":"2024-03-19T01:08:24.782982+0000","last_clean":"2024-03-19T01:08:24.782982+0000","last_became_active":"2024-03-19T01:07:14.487077+0000","last_became_peered":"2024-03-19T01:07:14.487077+0000","last_unstale":"2024-03-19T01:08:24.782982+0000","last_undegraded":"2024-03-19T01:08:24.782982+0000","last_fullsized":"2024-03-19T01:08:24.782982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:33:01.789431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.656863+0000","last_change":"2024-03-19T01:07:15.498009+0000","last_active":"2024-03-19T01:08:53.656863+0000","last_peered":"2024-03-19T01:08:53.656863+0000","last_clean":"2024-03-19T01:08:53.656863+0000","last_became_active":"2024-03-19T01:07:15.496707+0000","last_became_peered":"2024-03-19T01:07:15.496707+0000","last_unstale":"2024-03-19T01:08:53.656863+0000","last_undegraded":"2024-03-19T01:08:53.656863+0000","last_fullsized":"2024-03-19T01:08:53.656863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:50.058299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785106+0000","last_change":"2024-03-19T01:07:19.677635+0000","last_active":"2024-03-19T01:08:24.785106+0000","last_peered":"2024-03-19T01:08:24.785106+0000","last_clean":"2024-03-19T01:08:24.785106+0000","last_became_active":"2024-03-19T01:07:19.677207+0000","last_became_peered":"2024-03-19T01:07:19.677207+0000","last_unstale":"2024-03-19T01:08:24.785106+0000","last_undegraded":"2024-03-19T01:08:24.785106+0000","last_fullsized":"2024-03-19T01:08:24.785106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:23:28.650962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788907+0000","last_change":"2024-03-19T01:07:14.482054+0000","last_active":"2024-03-19T01:08:24.788907+0000","last_peered":"2024-03-19T01:08:24.788907+0000","last_clean":"2024-03-19T01:08:24.788907+0000","last_became_active":"2024-03-19T01:07:14.481703+0000","last_became_peered":"2024-03-19T01:07:14.481703+0000","last_unstale":"2024-03-19T01:08:24.788907+0000","last_undegraded":"2024-03-19T01:08:24.788907+0000","last_fullsized":"2024-03-19T01:08:24.788907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:02:43.565442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.657971+0000","last_change":"2024-03-19T01:07:15.487236+0000","last_active":"2024-03-19T01:08:53.657971+0000","last_peered":"2024-03-19T01:08:53.657971+0000","last_clean":"2024-03-19T01:08:53.657971+0000","last_became_active":"2024-03-19T01:07:15.486931+0000","last_became_peered":"2024-03-19T01:07:15.486931+0000","last_unstale":"2024-03-19T01:08:53.657971+0000","last_undegraded":"2024-03-19T01:08:53.657971+0000","last_fullsized":"2024-03-19T01:08:53.657971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:22.201904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788791+0000","last_change":"2024-03-19T01:07:19.693813+0000","last_active":"2024-03-19T01:08:24.788791+0000","last_peered":"2024-03-19T01:08:24.788791+0000","last_clean":"2024-03-19T01:08:24.788791+0000","last_became_active":"2024-03-19T01:07:19.693384+0000","last_became_peered":"2024-03-19T01:07:19.693384+0000","last_unstale":"2024-03-19T01:08:24.788791+0000","last_undegraded":"2024-03-19T01:08:24.788791+0000","last_fullsized":"2024-03-19T01:08:24.788791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:09.372461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":128,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:16.897734+0000","last_change":"2024-03-19T01:07:14.468467+0000","last_active":"2024-03-19T01:09:16.897734+0000","last_peered":"2024-03-19T01:09:16.897734+0000","last_clean":"2024-03-19T01:09:16.897734+0000","last_became_active":"2024-03-19T01:07:14.468074+0000","last_became_peered":"2024-03-19T01:07:14.468074+0000","last_unstale":"2024-03-19T01:09:16.897734+0000","last_undegraded":"2024-03-19T01:09:16.897734+0000","last_fullsized":"2024-03-19T01:09:16.897734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:07:49.216998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.642547+0000","last_change":"2024-03-19T01:07:15.475310+0000","last_active":"2024-03-19T01:08:53.642547+0000","last_peered":"2024-03-19T01:08:53.642547+0000","last_clean":"2024-03-19T01:08:53.642547+0000","last_became_active":"2024-03-19T01:07:15.474914+0000","last_became_peered":"2024-03-19T01:07:15.474914+0000","last_unstale":"2024-03-19T01:08:53.642547+0000","last_undegraded":"2024-03-19T01:08:53.642547+0000","last_fullsized":"2024-03-19T01:08:53.642547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:34.036892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.797430+0000","last_change":"2024-03-19T01:07:19.697937+0000","last_active":"2024-03-19T01:08:24.797430+0000","last_peered":"2024-03-19T01:08:24.797430+0000","last_clean":"2024-03-19T01:08:24.797430+0000","last_became_active":"2024-03-19T01:07:19.697447+0000","last_became_peered":"2024-03-19T01:07:19.697447+0000","last_unstale":"2024-03-19T01:08:24.797430+0000","last_undegraded":"2024-03-19T01:08:24.797430+0000","last_fullsized":"2024-03-19T01:08:24.797430+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:49.259118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785019+0000","last_change":"2024-03-19T01:07:14.476382+0000","last_active":"2024-03-19T01:08:24.785019+0000","last_peered":"2024-03-19T01:08:24.785019+0000","last_clean":"2024-03-19T01:08:24.785019+0000","last_became_active":"2024-03-19T01:07:14.476089+0000","last_became_peered":"2024-03-19T01:07:14.476089+0000","last_unstale":"2024-03-19T01:08:24.785019+0000","last_undegraded":"2024-03-19T01:08:24.785019+0000","last_fullsized":"2024-03-19T01:08:24.785019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:57:42.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.631229+0000","last_change":"2024-03-19T01:08:24.794335+0000","last_active":"2024-03-19T01:08:53.631229+0000","last_peered":"2024-03-19T01:08:53.631229+0000","last_clean":"2024-03-19T01:08:53.631229+0000","last_became_active":"2024-03-19T01:08:24.793725+0000","last_became_peered":"2024-03-19T01:08:24.793725+0000","last_unstale":"2024-03-19T01:08:53.631229+0000","last_undegraded":"2024-03-19T01:08:53.631229+0000","last_fullsized":"2024-03-19T01:08:53.631229+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:18.797824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788938+0000","last_change":"2024-03-19T01:07:19.688235+0000","last_active":"2024-03-19T01:08:24.788938+0000","last_peered":"2024-03-19T01:08:24.788938+0000","last_clean":"2024-03-19T01:08:24.788938+0000","last_became_active":"2024-03-19T01:07:19.687879+0000","last_became_peered":"2024-03-19T01:07:19.687879+0000","last_unstale":"2024-03-19T01:08:24.788938+0000","last_undegraded":"2024-03-19T01:08:24.788938+0000","last_fullsized":"2024-03-19T01:08:24.788938+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:29.465971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786220+0000","last_change":"2024-03-19T01:07:14.474346+0000","last_active":"2024-03-19T01:08:24.786220+0000","last_peered":"2024-03-19T01:08:24.786220+0000","last_clean":"2024-03-19T01:08:24.786220+0000","last_became_active":"2024-03-19T01:07:14.473989+0000","last_became_peered":"2024-03-19T01:07:14.473989+0000","last_unstale":"2024-03-19T01:08:24.786220+0000","last_undegraded":"2024-03-19T01:08:24.786220+0000","last_fullsized":"2024-03-19T01:08:24.786220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:41.004155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.645623+0000","last_change":"2024-03-19T01:07:15.482326+0000","last_active":"2024-03-19T01:08:53.645623+0000","last_peered":"2024-03-19T01:08:53.645623+0000","last_clean":"2024-03-19T01:08:53.645623+0000","last_became_active":"2024-03-19T01:07:15.481492+0000","last_became_peered":"2024-03-19T01:07:15.481492+0000","last_unstale":"2024-03-19T01:08:53.645623+0000","last_undegraded":"2024-03-19T01:08:53.645623+0000","last_fullsized":"2024-03-19T01:08:53.645623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:17:31.572104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786454+0000","last_change":"2024-03-19T01:07:19.668853+0000","last_active":"2024-03-19T01:08:24.786454+0000","last_peered":"2024-03-19T01:08:24.786454+0000","last_clean":"2024-03-19T01:08:24.786454+0000","last_became_active":"2024-03-19T01:07:19.668428+0000","last_became_peered":"2024-03-19T01:07:19.668428+0000","last_unstale":"2024-03-19T01:08:24.786454+0000","last_undegraded":"2024-03-19T01:08:24.786454+0000","last_fullsized":"2024-03-19T01:08:24.786454+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:50:43.409586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788150+0000","last_change":"2024-03-19T01:07:14.469548+0000","last_active":"2024-03-19T01:08:24.788150+0000","last_peered":"2024-03-19T01:08:24.788150+0000","last_clean":"2024-03-19T01:08:24.788150+0000","last_became_active":"2024-03-19T01:07:14.469200+0000","last_became_peered":"2024-03-19T01:07:14.469200+0000","last_unstale":"2024-03-19T01:08:24.788150+0000","last_undegraded":"2024-03-19T01:08:24.788150+0000","last_fullsized":"2024-03-19T01:08:24.788150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:41:53.179343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.649243+0000","last_change":"2024-03-19T01:07:15.495300+0000","last_active":"2024-03-19T01:08:53.649243+0000","last_peered":"2024-03-19T01:08:53.649243+0000","last_clean":"2024-03-19T01:08:53.649243+0000","last_became_active":"2024-03-19T01:07:15.494938+0000","last_became_peered":"2024-03-19T01:07:15.494938+0000","last_unstale":"2024-03-19T01:08:53.649243+0000","last_undegraded":"2024-03-19T01:08:53.649243+0000","last_fullsized":"2024-03-19T01:08:53.649243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:50:38.855857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.795694+0000","last_change":"2024-03-19T01:07:19.693261+0000","last_active":"2024-03-19T01:08:24.795694+0000","last_peered":"2024-03-19T01:08:24.795694+0000","last_clean":"2024-03-19T01:08:24.795694+0000","last_became_active":"2024-03-19T01:07:19.692598+0000","last_became_peered":"2024-03-19T01:07:19.692598+0000","last_unstale":"2024-03-19T01:08:24.795694+0000","last_undegraded":"2024-03-19T01:08:24.795694+0000","last_fullsized":"2024-03-19T01:08:24.795694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:43:29.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":120,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:15.944753+0000","last_change":"2024-03-19T01:07:14.481569+0000","last_active":"2024-03-19T01:09:15.944753+0000","last_peered":"2024-03-19T01:09:15.944753+0000","last_clean":"2024-03-19T01:09:15.944753+0000","last_became_active":"2024-03-19T01:07:14.481209+0000","last_became_peered":"2024-03-19T01:07:14.481209+0000","last_unstale":"2024-03-19T01:09:15.944753+0000","last_undegraded":"2024-03-19T01:09:15.944753+0000","last_fullsized":"2024-03-19T01:09:15.944753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:39.411207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.659688+0000","last_change":"2024-03-19T01:07:15.496852+0000","last_active":"2024-03-19T01:08:53.659688+0000","last_peered":"2024-03-19T01:08:53.659688+0000","last_clean":"2024-03-19T01:08:53.659688+0000","last_became_active":"2024-03-19T01:07:15.496455+0000","last_became_peered":"2024-03-19T01:07:15.496455+0000","last_unstale":"2024-03-19T01:08:53.659688+0000","last_undegraded":"2024-03-19T01:08:53.659688+0000","last_fullsized":"2024-03-19T01:08:53.659688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:43.419496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.797297+0000","last_change":"2024-03-19T01:07:19.692856+0000","last_active":"2024-03-19T01:08:24.797297+0000","last_peered":"2024-03-19T01:08:24.797297+0000","last_clean":"2024-03-19T01:08:24.797297+0000","last_became_active":"2024-03-19T01:07:19.690712+0000","last_became_peered":"2024-03-19T01:07:19.690712+0000","last_unstale":"2024-03-19T01:08:24.797297+0000","last_undegraded":"2024-03-19T01:08:24.797297+0000","last_fullsized":"2024-03-19T01:08:24.797297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:29:28.473062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.645999+0000","last_change":"2024-03-19T01:07:15.474375+0000","last_active":"2024-03-19T01:08:53.645999+0000","last_peered":"2024-03-19T01:08:53.645999+0000","last_clean":"2024-03-19T01:08:53.645999+0000","last_became_active":"2024-03-19T01:07:15.473427+0000","last_became_peered":"2024-03-19T01:07:15.473427+0000","last_unstale":"2024-03-19T01:08:53.645999+0000","last_undegraded":"2024-03-19T01:08:53.645999+0000","last_fullsized":"2024-03-19T01:08:53.645999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:42:22.235054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:15.895338+0000","last_change":"2024-03-19T01:07:14.490924+0000","last_active":"2024-03-19T01:09:15.895338+0000","last_peered":"2024-03-19T01:09:15.895338+0000","last_clean":"2024-03-19T01:09:15.895338+0000","last_became_active":"2024-03-19T01:07:14.490621+0000","last_became_peered":"2024-03-19T01:07:14.490621+0000","last_unstale":"2024-03-19T01:09:15.895338+0000","last_undegraded":"2024-03-19T01:09:15.895338+0000","last_fullsized":"2024-03-19T01:09:15.895338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:02:52.254138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.805196+0000","last_change":"2024-03-19T01:08:24.805196+0000","last_active":"2024-03-19T01:08:24.805196+0000","last_peered":"2024-03-19T01:08:24.805196+0000","last_clean":"2024-03-19T01:08:24.805196+0000","last_became_active":"2024-03-19T01:08:24.800225+0000","last_became_peered":"2024-03-19T01:08:24.800225+0000","last_unstale":"2024-03-19T01:08:24.805196+0000","last_undegraded":"2024-03-19T01:08:24.805196+0000","last_fullsized":"2024-03-19T01:08:24.805196+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:02.322846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.649908+0000","last_change":"2024-03-19T01:07:15.493218+0000","last_active":"2024-03-19T01:08:53.649908+0000","last_peered":"2024-03-19T01:08:53.649908+0000","last_clean":"2024-03-19T01:08:53.649908+0000","last_became_active":"2024-03-19T01:07:15.492751+0000","last_became_peered":"2024-03-19T01:07:15.492751+0000","last_unstale":"2024-03-19T01:08:53.649908+0000","last_undegraded":"2024-03-19T01:08:53.649908+0000","last_fullsized":"2024-03-19T01:08:53.649908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:48:32.105214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783354+0000","last_change":"2024-03-19T01:07:14.473349+0000","last_active":"2024-03-19T01:08:24.783354+0000","last_peered":"2024-03-19T01:08:24.783354+0000","last_clean":"2024-03-19T01:08:24.783354+0000","last_became_active":"2024-03-19T01:07:14.473014+0000","last_became_peered":"2024-03-19T01:07:14.473014+0000","last_unstale":"2024-03-19T01:08:24.783354+0000","last_undegraded":"2024-03-19T01:08:24.783354+0000","last_fullsized":"2024-03-19T01:08:24.783354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:27:34.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785047+0000","last_change":"2024-03-19T01:05:56.013594+0000","last_active":"2024-03-19T01:08:24.785047+0000","last_peered":"2024-03-19T01:08:24.785047+0000","last_clean":"2024-03-19T01:08:24.785047+0000","last_became_active":"2024-03-19T01:05:56.013084+0000","last_became_peered":"2024-03-19T01:05:56.013084+0000","last_unstale":"2024-03-19T01:08:24.785047+0000","last_undegraded":"2024-03-19T01:08:24.785047+0000","last_fullsized":"2024-03-19T01:08:24.785047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_clean_scrub_stamp":"2024-03-19T01:02:06.532343+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:23.440161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.789240+0000","last_change":"2024-03-19T01:07:19.691111+0000","last_active":"2024-03-19T01:08:24.789240+0000","last_peered":"2024-03-19T01:08:24.789240+0000","last_clean":"2024-03-19T01:08:24.789240+0000","last_became_active":"2024-03-19T01:07:19.690618+0000","last_became_peered":"2024-03-19T01:07:19.690618+0000","last_unstale":"2024-03-19T01:08:24.789240+0000","last_undegraded":"2024-03-19T01:08:24.789240+0000","last_fullsized":"2024-03-19T01:08:24.789240+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:17.857394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.638757+0000","last_change":"2024-03-19T01:08:24.801826+0000","last_active":"2024-03-19T01:08:53.638757+0000","last_peered":"2024-03-19T01:08:53.638757+0000","last_clean":"2024-03-19T01:08:53.638757+0000","last_became_active":"2024-03-19T01:08:24.801411+0000","last_became_peered":"2024-03-19T01:08:24.801411+0000","last_unstale":"2024-03-19T01:08:53.638757+0000","last_undegraded":"2024-03-19T01:08:53.638757+0000","last_fullsized":"2024-03-19T01:08:53.638757+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:31.798045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785114+0000","last_change":"2024-03-19T01:07:14.474287+0000","last_active":"2024-03-19T01:08:24.785114+0000","last_peered":"2024-03-19T01:08:24.785114+0000","last_clean":"2024-03-19T01:08:24.785114+0000","last_became_active":"2024-03-19T01:07:14.473921+0000","last_became_peered":"2024-03-19T01:07:14.473921+0000","last_unstale":"2024-03-19T01:08:24.785114+0000","last_undegraded":"2024-03-19T01:08:24.785114+0000","last_fullsized":"2024-03-19T01:08:24.785114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:49.322029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785011+0000","last_change":"2024-03-19T01:07:19.685331+0000","last_active":"2024-03-19T01:08:24.785011+0000","last_peered":"2024-03-19T01:08:24.785011+0000","last_clean":"2024-03-19T01:08:24.785011+0000","last_became_active":"2024-03-19T01:07:19.684876+0000","last_became_peered":"2024-03-19T01:07:19.684876+0000","last_unstale":"2024-03-19T01:08:24.785011+0000","last_undegraded":"2024-03-19T01:08:24.785011+0000","last_fullsized":"2024-03-19T01:08:24.785011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:39.575348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785730+0000","last_change":"2024-03-19T01:07:14.481019+0000","last_active":"2024-03-19T01:08:24.785730+0000","last_peered":"2024-03-19T01:08:24.785730+0000","last_clean":"2024-03-19T01:08:24.785730+0000","last_became_active":"2024-03-19T01:07:14.480666+0000","last_became_peered":"2024-03-19T01:07:14.480666+0000","last_unstale":"2024-03-19T01:08:24.785730+0000","last_undegraded":"2024-03-19T01:08:24.785730+0000","last_fullsized":"2024-03-19T01:08:24.785730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:13:33.669690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.645918+0000","last_change":"2024-03-19T01:07:15.490747+0000","last_active":"2024-03-19T01:08:53.645918+0000","last_peered":"2024-03-19T01:08:53.645918+0000","last_clean":"2024-03-19T01:08:53.645918+0000","last_became_active":"2024-03-19T01:07:15.486244+0000","last_became_peered":"2024-03-19T01:07:15.486244+0000","last_unstale":"2024-03-19T01:08:53.645918+0000","last_undegraded":"2024-03-19T01:08:53.645918+0000","last_fullsized":"2024-03-19T01:08:53.645918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:43.212972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.792337+0000","last_change":"2024-03-19T01:07:19.684804+0000","last_active":"2024-03-19T01:08:24.792337+0000","last_peered":"2024-03-19T01:08:24.792337+0000","last_clean":"2024-03-19T01:08:24.792337+0000","last_became_active":"2024-03-19T01:07:19.683191+0000","last_became_peered":"2024-03-19T01:07:19.683191+0000","last_unstale":"2024-03-19T01:08:24.792337+0000","last_undegraded":"2024-03-19T01:08:24.792337+0000","last_fullsized":"2024-03-19T01:08:24.792337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:02.015499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'14","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788392+0000","last_change":"2024-03-19T01:07:14.479282+0000","last_active":"2024-03-19T01:08:24.788392+0000","last_peered":"2024-03-19T01:08:24.788392+0000","last_clean":"2024-03-19T01:08:24.788392+0000","last_became_active":"2024-03-19T01:07:14.478876+0000","last_became_peered":"2024-03-19T01:07:14.478876+0000","last_unstale":"2024-03-19T01:08:24.788392+0000","last_undegraded":"2024-03-19T01:08:24.788392+0000","last_fullsized":"2024-03-19T01:08:24.788392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:54.179680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.651438+0000","last_change":"2024-03-19T01:07:15.493498+0000","last_active":"2024-03-19T01:08:53.651438+0000","last_peered":"2024-03-19T01:08:53.651438+0000","last_clean":"2024-03-19T01:08:53.651438+0000","last_became_active":"2024-03-19T01:07:15.493110+0000","last_became_peered":"2024-03-19T01:07:15.493110+0000","last_unstale":"2024-03-19T01:08:53.651438+0000","last_undegraded":"2024-03-19T01:08:53.651438+0000","last_fullsized":"2024-03-19T01:08:53.651438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:55:23.608596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782607+0000","last_change":"2024-03-19T01:07:19.671232+0000","last_active":"2024-03-19T01:08:24.782607+0000","last_peered":"2024-03-19T01:08:24.782607+0000","last_clean":"2024-03-19T01:08:24.782607+0000","last_became_active":"2024-03-19T01:07:19.670818+0000","last_became_peered":"2024-03-19T01:07:19.670818+0000","last_unstale":"2024-03-19T01:08:24.782607+0000","last_undegraded":"2024-03-19T01:08:24.782607+0000","last_fullsized":"2024-03-19T01:08:24.782607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:57:20.134372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783210+0000","last_change":"2024-03-19T01:07:14.473568+0000","last_active":"2024-03-19T01:08:24.783210+0000","last_peered":"2024-03-19T01:08:24.783210+0000","last_clean":"2024-03-19T01:08:24.783210+0000","last_became_active":"2024-03-19T01:07:14.472555+0000","last_became_peered":"2024-03-19T01:07:14.472555+0000","last_unstale":"2024-03-19T01:08:24.783210+0000","last_undegraded":"2024-03-19T01:08:24.783210+0000","last_fullsized":"2024-03-19T01:08:24.783210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:28:01.541228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.665645+0000","last_change":"2024-03-19T01:07:15.499905+0000","last_active":"2024-03-19T01:08:53.665645+0000","last_peered":"2024-03-19T01:08:53.665645+0000","last_clean":"2024-03-19T01:08:53.665645+0000","last_became_active":"2024-03-19T01:07:15.499560+0000","last_became_peered":"2024-03-19T01:07:15.499560+0000","last_unstale":"2024-03-19T01:08:53.665645+0000","last_undegraded":"2024-03-19T01:08:53.665645+0000","last_fullsized":"2024-03-19T01:08:53.665645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:06:26.531100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.783121+0000","last_change":"2024-03-19T01:07:19.690310+0000","last_active":"2024-03-19T01:08:24.783121+0000","last_peered":"2024-03-19T01:08:24.783121+0000","last_clean":"2024-03-19T01:08:24.783121+0000","last_became_active":"2024-03-19T01:07:19.689877+0000","last_became_peered":"2024-03-19T01:07:19.689877+0000","last_unstale":"2024-03-19T01:08:24.783121+0000","last_undegraded":"2024-03-19T01:08:24.783121+0000","last_fullsized":"2024-03-19T01:08:24.783121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:06:12.730258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.786831+0000","last_change":"2024-03-19T01:07:14.466267+0000","last_active":"2024-03-19T01:08:24.786831+0000","last_peered":"2024-03-19T01:08:24.786831+0000","last_clean":"2024-03-19T01:08:24.786831+0000","last_became_active":"2024-03-19T01:07:14.465944+0000","last_became_peered":"2024-03-19T01:07:14.465944+0000","last_unstale":"2024-03-19T01:08:24.786831+0000","last_undegraded":"2024-03-19T01:08:24.786831+0000","last_fullsized":"2024-03-19T01:08:24.786831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:40.957200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.605173+0000","last_change":"2024-03-19T01:07:15.475690+0000","last_active":"2024-03-19T01:08:53.605173+0000","last_peered":"2024-03-19T01:08:53.605173+0000","last_clean":"2024-03-19T01:08:53.605173+0000","last_became_active":"2024-03-19T01:07:15.475342+0000","last_became_peered":"2024-03-19T01:07:15.475342+0000","last_unstale":"2024-03-19T01:08:53.605173+0000","last_undegraded":"2024-03-19T01:08:53.605173+0000","last_fullsized":"2024-03-19T01:08:53.605173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:30:30.842462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.798173+0000","last_change":"2024-03-19T01:08:24.797680+0000","last_active":"2024-03-19T01:08:24.798173+0000","last_peered":"2024-03-19T01:08:24.798173+0000","last_clean":"2024-03-19T01:08:24.798173+0000","last_became_active":"2024-03-19T01:08:24.797194+0000","last_became_peered":"2024-03-19T01:08:24.797194+0000","last_unstale":"2024-03-19T01:08:24.798173+0000","last_undegraded":"2024-03-19T01:08:24.798173+0000","last_fullsized":"2024-03-19T01:08:24.798173+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:49.800726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.792871+0000","last_change":"2024-03-19T01:07:14.487907+0000","last_active":"2024-03-19T01:08:24.792871+0000","last_peered":"2024-03-19T01:08:24.792871+0000","last_clean":"2024-03-19T01:08:24.792871+0000","last_became_active":"2024-03-19T01:07:14.487618+0000","last_became_peered":"2024-03-19T01:07:14.487618+0000","last_unstale":"2024-03-19T01:08:24.792871+0000","last_undegraded":"2024-03-19T01:08:24.792871+0000","last_fullsized":"2024-03-19T01:08:24.792871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:22:49.218797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.659806+0000","last_change":"2024-03-19T01:07:15.492103+0000","last_active":"2024-03-19T01:08:53.659806+0000","last_peered":"2024-03-19T01:08:53.659806+0000","last_clean":"2024-03-19T01:08:53.659806+0000","last_became_active":"2024-03-19T01:07:15.491712+0000","last_became_peered":"2024-03-19T01:07:15.491712+0000","last_unstale":"2024-03-19T01:08:53.659806+0000","last_undegraded":"2024-03-19T01:08:53.659806+0000","last_fullsized":"2024-03-19T01:08:53.659806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:40:13.959986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782915+0000","last_change":"2024-03-19T01:07:19.677392+0000","last_active":"2024-03-19T01:08:24.782915+0000","last_peered":"2024-03-19T01:08:24.782915+0000","last_clean":"2024-03-19T01:08:24.782915+0000","last_became_active":"2024-03-19T01:07:19.677024+0000","last_became_peered":"2024-03-19T01:07:19.677024+0000","last_unstale":"2024-03-19T01:08:24.782915+0000","last_undegraded":"2024-03-19T01:08:24.782915+0000","last_fullsized":"2024-03-19T01:08:24.782915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:01:43.404902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.805878+0000","last_change":"2024-03-19T01:08:24.805363+0000","last_active":"2024-03-19T01:08:24.805878+0000","last_peered":"2024-03-19T01:08:24.805878+0000","last_clean":"2024-03-19T01:08:24.805878+0000","last_became_active":"2024-03-19T01:08:24.800810+0000","last_became_peered":"2024-03-19T01:08:24.800810+0000","last_unstale":"2024-03-19T01:08:24.805878+0000","last_undegraded":"2024-03-19T01:08:24.805878+0000","last_fullsized":"2024-03-19T01:08:24.805878+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:32:43.608140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.659108+0000","last_change":"2024-03-19T01:07:15.505842+0000","last_active":"2024-03-19T01:08:53.659108+0000","last_peered":"2024-03-19T01:08:53.659108+0000","last_clean":"2024-03-19T01:08:53.659108+0000","last_became_active":"2024-03-19T01:07:15.505425+0000","last_became_peered":"2024-03-19T01:07:15.505425+0000","last_unstale":"2024-03-19T01:08:53.659108+0000","last_undegraded":"2024-03-19T01:08:53.659108+0000","last_fullsized":"2024-03-19T01:08:53.659108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:28.119864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.782948+0000","last_change":"2024-03-19T01:07:19.683113+0000","last_active":"2024-03-19T01:08:24.782948+0000","last_peered":"2024-03-19T01:08:24.782948+0000","last_clean":"2024-03-19T01:08:24.782948+0000","last_became_active":"2024-03-19T01:07:19.682702+0000","last_became_peered":"2024-03-19T01:07:19.682702+0000","last_unstale":"2024-03-19T01:08:24.782948+0000","last_undegraded":"2024-03-19T01:08:24.782948+0000","last_fullsized":"2024-03-19T01:08:24.782948+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:42.866408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.785093+0000","last_change":"2024-03-19T01:07:14.476562+0000","last_active":"2024-03-19T01:08:24.785093+0000","last_peered":"2024-03-19T01:08:24.785093+0000","last_clean":"2024-03-19T01:08:24.785093+0000","last_became_active":"2024-03-19T01:07:14.476185+0000","last_became_peered":"2024-03-19T01:07:14.476185+0000","last_unstale":"2024-03-19T01:08:24.785093+0000","last_undegraded":"2024-03-19T01:08:24.785093+0000","last_fullsized":"2024-03-19T01:08:24.785093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:35:19.860443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.654246+0000","last_change":"2024-03-19T01:07:15.487163+0000","last_active":"2024-03-19T01:08:53.654246+0000","last_peered":"2024-03-19T01:08:53.654246+0000","last_clean":"2024-03-19T01:08:53.654246+0000","last_became_active":"2024-03-19T01:07:15.486845+0000","last_became_peered":"2024-03-19T01:07:15.486845+0000","last_unstale":"2024-03-19T01:08:53.654246+0000","last_undegraded":"2024-03-19T01:08:53.654246+0000","last_fullsized":"2024-03-19T01:08:53.654246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:39:09.389411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.787057+0000","last_change":"2024-03-19T01:07:19.692768+0000","last_active":"2024-03-19T01:08:24.787057+0000","last_peered":"2024-03-19T01:08:24.787057+0000","last_clean":"2024-03-19T01:08:24.787057+0000","last_became_active":"2024-03-19T01:07:19.692382+0000","last_became_peered":"2024-03-19T01:07:19.692382+0000","last_unstale":"2024-03-19T01:08:24.787057+0000","last_undegraded":"2024-03-19T01:08:24.787057+0000","last_fullsized":"2024-03-19T01:08:24.787057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:26.804938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":119,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:09:13.900322+0000","last_change":"2024-03-19T01:07:14.472844+0000","last_active":"2024-03-19T01:09:13.900322+0000","last_peered":"2024-03-19T01:09:13.900322+0000","last_clean":"2024-03-19T01:09:13.900322+0000","last_became_active":"2024-03-19T01:07:14.472389+0000","last_became_peered":"2024-03-19T01:07:14.472389+0000","last_unstale":"2024-03-19T01:09:13.900322+0000","last_undegraded":"2024-03-19T01:09:13.900322+0000","last_fullsized":"2024-03-19T01:09:13.900322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:31.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.653619+0000","last_change":"2024-03-19T01:07:15.482033+0000","last_active":"2024-03-19T01:08:53.653619+0000","last_peered":"2024-03-19T01:08:53.653619+0000","last_clean":"2024-03-19T01:08:53.653619+0000","last_became_active":"2024-03-19T01:07:15.481671+0000","last_became_peered":"2024-03-19T01:07:15.481671+0000","last_unstale":"2024-03-19T01:08:53.653619+0000","last_undegraded":"2024-03-19T01:08:53.653619+0000","last_fullsized":"2024-03-19T01:08:53.653619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:09.969374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.793574+0000","last_change":"2024-03-19T01:08:24.793574+0000","last_active":"2024-03-19T01:08:24.793574+0000","last_peered":"2024-03-19T01:08:24.793574+0000","last_clean":"2024-03-19T01:08:24.793574+0000","last_became_active":"2024-03-19T01:08:24.793012+0000","last_became_peered":"2024-03-19T01:08:24.793012+0000","last_unstale":"2024-03-19T01:08:24.793574+0000","last_undegraded":"2024-03-19T01:08:24.793574+0000","last_fullsized":"2024-03-19T01:08:24.793574+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:04:53.611271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.789545+0000","last_change":"2024-03-19T01:07:14.481677+0000","last_active":"2024-03-19T01:08:24.789545+0000","last_peered":"2024-03-19T01:08:24.789545+0000","last_clean":"2024-03-19T01:08:24.789545+0000","last_became_active":"2024-03-19T01:07:14.481138+0000","last_became_peered":"2024-03-19T01:07:14.481138+0000","last_unstale":"2024-03-19T01:08:24.789545+0000","last_undegraded":"2024-03-19T01:08:24.789545+0000","last_fullsized":"2024-03-19T01:08:24.789545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:52:33.105816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.652140+0000","last_change":"2024-03-19T01:07:15.498352+0000","last_active":"2024-03-19T01:08:53.652140+0000","last_peered":"2024-03-19T01:08:53.652140+0000","last_clean":"2024-03-19T01:08:53.652140+0000","last_became_active":"2024-03-19T01:07:15.498040+0000","last_became_peered":"2024-03-19T01:07:15.498040+0000","last_unstale":"2024-03-19T01:08:53.652140+0000","last_undegraded":"2024-03-19T01:08:53.652140+0000","last_fullsized":"2024-03-19T01:08:53.652140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:57:37.571753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.788102+0000","last_change":"2024-03-19T01:07:19.673291+0000","last_active":"2024-03-19T01:08:24.788102+0000","last_peered":"2024-03-19T01:08:24.788102+0000","last_clean":"2024-03-19T01:08:24.788102+0000","last_became_active":"2024-03-19T01:07:19.672704+0000","last_became_peered":"2024-03-19T01:07:19.672704+0000","last_unstale":"2024-03-19T01:08:24.788102+0000","last_undegraded":"2024-03-19T01:08:24.788102+0000","last_fullsized":"2024-03-19T01:08:24.788102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:01:42.019812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:24.797532+0000","last_change":"2024-03-19T01:07:14.489556+0000","last_active":"2024-03-19T01:08:24.797532+0000","last_peered":"2024-03-19T01:08:24.797532+0000","last_clean":"2024-03-19T01:08:24.797532+0000","last_became_active":"2024-03-19T01:07:14.489201+0000","last_became_peered":"2024-03-19T01:07:14.489201+0000","last_unstale":"2024-03-19T01:08:24.797532+0000","last_undegraded":"2024-03-19T01:08:24.797532+0000","last_fullsized":"2024-03-19T01:08:24.797532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:28:36.516185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.647535+0000","last_change":"2024-03-19T01:07:15.477727+0000","last_active":"2024-03-19T01:08:53.647535+0000","last_peered":"2024-03-19T01:08:53.647535+0000","last_clean":"2024-03-19T01:08:53.647535+0000","last_became_active":"2024-03-19T01:07:15.477388+0000","last_became_peered":"2024-03-19T01:07:15.477388+0000","last_unstale":"2024-03-19T01:08:53.647535+0000","last_undegraded":"2024-03-19T01:08:53.647535+0000","last_fullsized":"2024-03-19T01:08:53.647535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:30.567326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.636741+0000","last_change":"2024-03-19T01:07:15.479008+0000","last_active":"2024-03-19T01:08:53.636741+0000","last_peered":"2024-03-19T01:08:53.636741+0000","last_clean":"2024-03-19T01:08:53.636741+0000","last_became_active":"2024-03-19T01:07:15.478657+0000","last_became_peered":"2024-03-19T01:07:15.478657+0000","last_unstale":"2024-03-19T01:08:53.636741+0000","last_undegraded":"2024-03-19T01:08:53.636741+0000","last_fullsized":"2024-03-19T01:08:53.636741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:42:46.970012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.640084+0000","last_change":"2024-03-19T01:08:24.789796+0000","last_active":"2024-03-19T01:08:53.640084+0000","last_peered":"2024-03-19T01:08:53.640084+0000","last_clean":"2024-03-19T01:08:53.640084+0000","last_became_active":"2024-03-19T01:08:24.789444+0000","last_became_peered":"2024-03-19T01:08:24.789444+0000","last_unstale":"2024-03-19T01:08:53.640084+0000","last_undegraded":"2024-03-19T01:08:53.640084+0000","last_fullsized":"2024-03-19T01:08:53.640084+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:41.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.658984+0000","last_change":"2024-03-19T01:07:15.495292+0000","last_active":"2024-03-19T01:08:53.658984+0000","last_peered":"2024-03-19T01:08:53.658984+0000","last_clean":"2024-03-19T01:08:53.658984+0000","last_became_active":"2024-03-19T01:07:15.494946+0000","last_became_peered":"2024-03-19T01:07:15.494946+0000","last_unstale":"2024-03-19T01:08:53.658984+0000","last_undegraded":"2024-03-19T01:08:53.658984+0000","last_fullsized":"2024-03-19T01:08:53.658984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:39.012221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.664494+0000","last_change":"2024-03-19T01:07:15.499816+0000","last_active":"2024-03-19T01:08:53.664494+0000","last_peered":"2024-03-19T01:08:53.664494+0000","last_clean":"2024-03-19T01:08:53.664494+0000","last_became_active":"2024-03-19T01:07:15.499031+0000","last_became_peered":"2024-03-19T01:07:15.499031+0000","last_unstale":"2024-03-19T01:08:53.664494+0000","last_undegraded":"2024-03-19T01:08:53.664494+0000","last_fullsized":"2024-03-19T01:08:53.664494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:34:25.344092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.662893+0000","last_change":"2024-03-19T01:07:15.501526+0000","last_active":"2024-03-19T01:08:53.662893+0000","last_peered":"2024-03-19T01:08:53.662893+0000","last_clean":"2024-03-19T01:08:53.662893+0000","last_became_active":"2024-03-19T01:07:15.500452+0000","last_became_peered":"2024-03-19T01:07:15.500452+0000","last_unstale":"2024-03-19T01:08:53.662893+0000","last_undegraded":"2024-03-19T01:08:53.662893+0000","last_fullsized":"2024-03-19T01:08:53.662893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:35:33.075780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.633014+0000","last_change":"2024-03-19T01:07:15.477649+0000","last_active":"2024-03-19T01:08:53.633014+0000","last_peered":"2024-03-19T01:08:53.633014+0000","last_clean":"2024-03-19T01:08:53.633014+0000","last_became_active":"2024-03-19T01:07:15.477193+0000","last_became_peered":"2024-03-19T01:07:15.477193+0000","last_unstale":"2024-03-19T01:08:53.633014+0000","last_undegraded":"2024-03-19T01:08:53.633014+0000","last_fullsized":"2024-03-19T01:08:53.633014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:36.126642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.641408+0000","last_change":"2024-03-19T01:07:15.481920+0000","last_active":"2024-03-19T01:08:53.641408+0000","last_peered":"2024-03-19T01:08:53.641408+0000","last_clean":"2024-03-19T01:08:53.641408+0000","last_became_active":"2024-03-19T01:07:15.481290+0000","last_became_peered":"2024-03-19T01:07:15.481290+0000","last_unstale":"2024-03-19T01:08:53.641408+0000","last_undegraded":"2024-03-19T01:08:53.641408+0000","last_fullsized":"2024-03-19T01:08:53.641408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:37:58.028647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.650606+0000","last_change":"2024-03-19T01:07:15.502305+0000","last_active":"2024-03-19T01:08:53.650606+0000","last_peered":"2024-03-19T01:08:53.650606+0000","last_clean":"2024-03-19T01:08:53.650606+0000","last_became_active":"2024-03-19T01:07:15.501491+0000","last_became_peered":"2024-03-19T01:07:15.501491+0000","last_unstale":"2024-03-19T01:08:53.650606+0000","last_undegraded":"2024-03-19T01:08:53.650606+0000","last_fullsized":"2024-03-19T01:08:53.650606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:01.999538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.649089+0000","last_change":"2024-03-19T01:07:15.499590+0000","last_active":"2024-03-19T01:08:53.649089+0000","last_peered":"2024-03-19T01:08:53.649089+0000","last_clean":"2024-03-19T01:08:53.649089+0000","last_became_active":"2024-03-19T01:07:15.498991+0000","last_became_peered":"2024-03-19T01:07:15.498991+0000","last_unstale":"2024-03-19T01:08:53.649089+0000","last_undegraded":"2024-03-19T01:08:53.649089+0000","last_fullsized":"2024-03-19T01:08:53.649089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:09:43.510480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.651658+0000","last_change":"2024-03-19T01:07:15.505022+0000","last_active":"2024-03-19T01:08:53.651658+0000","last_peered":"2024-03-19T01:08:53.651658+0000","last_clean":"2024-03-19T01:08:53.651658+0000","last_became_active":"2024-03-19T01:07:15.503684+0000","last_became_peered":"2024-03-19T01:07:15.503684+0000","last_unstale":"2024-03-19T01:08:53.651658+0000","last_undegraded":"2024-03-19T01:08:53.651658+0000","last_fullsized":"2024-03-19T01:08:53.651658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:56:47.580267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.653435+0000","last_change":"2024-03-19T01:07:15.495634+0000","last_active":"2024-03-19T01:08:53.653435+0000","last_peered":"2024-03-19T01:08:53.653435+0000","last_clean":"2024-03-19T01:08:53.653435+0000","last_became_active":"2024-03-19T01:07:15.495206+0000","last_became_peered":"2024-03-19T01:07:15.495206+0000","last_unstale":"2024-03-19T01:08:53.653435+0000","last_undegraded":"2024-03-19T01:08:53.653435+0000","last_fullsized":"2024-03-19T01:08:53.653435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:30:29.965731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.665906+0000","last_change":"2024-03-19T01:07:15.501480+0000","last_active":"2024-03-19T01:08:53.665906+0000","last_peered":"2024-03-19T01:08:53.665906+0000","last_clean":"2024-03-19T01:08:53.665906+0000","last_became_active":"2024-03-19T01:07:15.501078+0000","last_became_peered":"2024-03-19T01:07:15.501078+0000","last_unstale":"2024-03-19T01:08:53.665906+0000","last_undegraded":"2024-03-19T01:08:53.665906+0000","last_fullsized":"2024-03-19T01:08:53.665906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:40:18.767874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-19T01:08:53.645793+0000","last_change":"2024-03-19T01:07:15.480358+0000","last_active":"2024-03-19T01:08:53.645793+0000","last_peered":"2024-03-19T01:08:53.645793+0000","last_clean":"2024-03-19T01:08:53.645793+0000","last_became_active":"2024-03-19T01:07:15.479979+0000","last_became_peered":"2024-03-19T01:07:15.479979+0000","last_unstale":"2024-03-19T01:08:53.645793+0000","last_undegraded":"2024-03-19T01:08:53.645793+0000","last_fullsized":"2024-03-19T01:08:53.645793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:12:54.598435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247132,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69828,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":920,"num_read_kb":538,"num_write":249,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":209484,"data_compressed":5469,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383719,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":501433,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645356,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":1155751,"data_compressed":58168,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808881,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":505459,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070519,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":501741,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332156,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1098379,"data_compressed":56345,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":496867,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":560771,"data_compressed":22394,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":498323,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":560991,"data_compressed":22394,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":496575,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":501473,"data_compressed":20571,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1089333,"data_compressed":56345,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64626,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64846,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65142,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T01:09:21.544 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-19T01:09:21.544 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-19T01:09:21.544 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-19T01:09:21.544 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-19T01:09:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:21 smithi069 ceph-mon[31727]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:09:22.043 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:09:22.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:21 smithi163 ceph-mon[67428]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:09:22.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:21 smithi086 ceph-mon[66829]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:09:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:22 smithi069 ceph-mon[31727]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:22 smithi163 ceph-mon[67428]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:23.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:22 smithi086 ceph-mon[66829]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:24.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.416+0000 7f6fe2a00700 1 -- 172.21.15.69:0/3115292995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc101950 msgr2=0x7f6fdc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:24.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.416+0000 7f6fe2a00700 1 --2- 172.21.15.69:0/3115292995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc101950 0x7f6fdc10e610 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009990 tx=0x7f6fc8009ca0 comp rx=0 tx=0).stop 2024-03-19T01:09:24.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.416+0000 7f6fe2a00700 1 -- 172.21.15.69:0/3115292995 shutdown_connections 2024-03-19T01:09:24.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.416+0000 7f6fe2a00700 1 --2- 172.21.15.69:0/3115292995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc10eb50 0x7f6fdc110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:24.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.416+0000 7f6fe2a00700 1 --2- 172.21.15.69:0/3115292995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc100fa0 0x7f6fdc101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:24.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.416+0000 7f6fe2a00700 1 --2- 172.21.15.69:0/3115292995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc101950 0x7f6fdc10e610 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:24.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.416+0000 7f6fe2a00700 1 -- 172.21.15.69:0/3115292995 >> 172.21.15.69:0/3115292995 conn(0x7f6fdc0fcbe0 msgr2=0x7f6fdc0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:24.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.416+0000 7f6fe2a00700 1 -- 172.21.15.69:0/3115292995 shutdown_connections 2024-03-19T01:09:24.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.416+0000 7f6fe2a00700 1 -- 172.21.15.69:0/3115292995 wait complete. 2024-03-19T01:09:24.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.417+0000 7f6fe2a00700 1 Processor -- start 2024-03-19T01:09:24.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.417+0000 7f6fe2a00700 1 -- start start 2024-03-19T01:09:24.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.418+0000 7f6fe2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc100fa0 0x7f6fdc1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:24.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.418+0000 7f6fe2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc101950 0x7f6fdc1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:24.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.418+0000 7f6fe2a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc10eb50 0x7f6fdc19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:24.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.418+0000 7f6fe2a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc114790 con 0x7f6fdc100fa0 2024-03-19T01:09:24.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.418+0000 7f6fe2a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc114610 con 0x7f6fdc101950 2024-03-19T01:09:24.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.418+0000 7f6fe2a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc114910 con 0x7f6fdc10eb50 2024-03-19T01:09:24.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.418+0000 7f6fe1600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc100fa0 0x7f6fdc1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:24.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.418+0000 7f6fe0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc101950 0x7f6fdc1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:24.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.418+0000 7f6fe1600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc100fa0 0x7f6fdc1a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59576/0 (socket says 172.21.15.69:59576) 2024-03-19T01:09:24.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.418+0000 7f6fe1600700 1 -- 172.21.15.69:0/4047631314 learned_addr learned my addr 172.21.15.69:0/4047631314 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:24.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.418+0000 7f6fe2000700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc10eb50 0x7f6fdc19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:24.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.419+0000 7f6fe0c00700 1 -- 172.21.15.69:0/4047631314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc10eb50 msgr2=0x7f6fdc19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:24.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.419+0000 7f6fe0c00700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc10eb50 0x7f6fdc19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:24.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.419+0000 7f6fe0c00700 1 -- 172.21.15.69:0/4047631314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc100fa0 msgr2=0x7f6fdc1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:24.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.419+0000 7f6fe0c00700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc100fa0 0x7f6fdc1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:24.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.419+0000 7f6fe0c00700 1 -- 172.21.15.69:0/4047631314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc19b1f0 con 0x7f6fdc101950 2024-03-19T01:09:24.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.419+0000 7f6fe0c00700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc101950 0x7f6fdc1a0dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80098e0 tx=0x7f6fc800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:24.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.419+0000 7f6fda000700 1 -- 172.21.15.69:0/4047631314 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8027070 con 0x7f6fdc101950 2024-03-19T01:09:24.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.419+0000 7f6fda000700 1 -- 172.21.15.69:0/4047631314 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fc8003b60 con 0x7f6fdc101950 2024-03-19T01:09:24.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.419+0000 7f6fe2a00700 1 -- 172.21.15.69:0/4047631314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc19b480 con 0x7f6fdc101950 2024-03-19T01:09:24.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.419+0000 7f6fda000700 1 -- 172.21.15.69:0/4047631314 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8022d50 con 0x7f6fdc101950 2024-03-19T01:09:24.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.419+0000 7f6fe2a00700 1 -- 172.21.15.69:0/4047631314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc1a7750 con 0x7f6fdc101950 2024-03-19T01:09:24.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.420+0000 7f6fe2a00700 1 -- 172.21.15.69:0/4047631314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc068fd0 con 0x7f6fdc101950 2024-03-19T01:09:24.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.422+0000 7f6fda000700 1 -- 172.21.15.69:0/4047631314 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f6fc802a070 con 0x7f6fdc101950 2024-03-19T01:09:24.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.424+0000 7f6fda000700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc40a2910 0x7f6fc40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:24.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.424+0000 7f6fda000700 1 -- 172.21.15.69:0/4047631314 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6fc80d00a0 con 0x7f6fdc101950 2024-03-19T01:09:24.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.424+0000 7f6fe1600700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc40a2910 0x7f6fc40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:24.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.425+0000 7f6fe1600700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc40a2910 0x7f6fc40a4dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc110f80 tx=0x7f6fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:24.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.426+0000 7f6fda000700 1 -- 172.21.15.69:0/4047631314 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6fc8013070 con 0x7f6fdc101950 2024-03-19T01:09:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.732+0000 7f6fe2a00700 1 -- 172.21.15.69:0/4047631314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6fdc19bf70 con 0x7f6fdc101950 2024-03-19T01:09:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.732+0000 7f6fda000700 1 -- 172.21.15.69:0/4047631314 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6fc809ce10 con 0x7f6fdc101950 2024-03-19T01:09:24.734 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:24.735 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-19T01:09:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.735+0000 7f6fe2a00700 1 -- 172.21.15.69:0/4047631314 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc40a2910 msgr2=0x7f6fc40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.735+0000 7f6fe2a00700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc40a2910 0x7f6fc40a4dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc110f80 tx=0x7f6fd0009260 comp rx=0 tx=0).stop 2024-03-19T01:09:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.736+0000 7f6fe2a00700 1 -- 172.21.15.69:0/4047631314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc101950 msgr2=0x7f6fdc1a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.736+0000 7f6fe2a00700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc101950 0x7f6fdc1a0dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80098e0 tx=0x7f6fc800b8a0 comp rx=0 tx=0).stop 2024-03-19T01:09:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.736+0000 7f6fe2a00700 1 -- 172.21.15.69:0/4047631314 shutdown_connections 2024-03-19T01:09:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.736+0000 7f6fe2a00700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc40a2910 0x7f6fc40a4dd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.736+0000 7f6fe2a00700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc10eb50 0x7f6fdc19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.736+0000 7f6fe2a00700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc100fa0 0x7f6fdc1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.736+0000 7f6fe2a00700 1 --2- 172.21.15.69:0/4047631314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc101950 0x7f6fdc1a0dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.736+0000 7f6fe2a00700 1 -- 172.21.15.69:0/4047631314 >> 172.21.15.69:0/4047631314 conn(0x7f6fdc0fcbe0 msgr2=0x7f6fdc10fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.736+0000 7f6fe2a00700 1 -- 172.21.15.69:0/4047631314 shutdown_connections 2024-03-19T01:09:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:24.736+0000 7f6fe2a00700 1 -- 172.21.15.69:0/4047631314 wait complete. 2024-03-19T01:09:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:24 smithi069 ceph-mon[31727]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]: dispatch 2024-03-19T01:09:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-03-19T01:09:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:24 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-03-19T01:09:25.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:24 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/4047631314' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-19T01:09:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:24 smithi163 ceph-mon[67428]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]: dispatch 2024-03-19T01:09:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-03-19T01:09:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:24 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-03-19T01:09:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:24 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/4047631314' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-19T01:09:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:24 smithi086 ceph-mon[66829]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]: dispatch 2024-03-19T01:09:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-03-19T01:09:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:24 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-03-19T01:09:25.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:24 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/4047631314' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-19T01:09:25.585 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-19T01:09:25.585 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.782+0000 7fb5f8000640 1 -- 172.21.15.69:0/340603712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f0100a30 msgr2=0x7fb5f0100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.782+0000 7fb5f8000640 1 --2- 172.21.15.69:0/340603712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f0100a30 0x7fb5f0100e90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009900 tx=0x7fb5dc02bc30 comp rx=0 tx=0).stop 2024-03-19T01:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.783+0000 7fb5f8000640 1 -- 172.21.15.69:0/340603712 shutdown_connections 2024-03-19T01:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.783+0000 7fb5f8000640 1 --2- 172.21.15.69:0/340603712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5f01013d0 0x7fb5f0101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.783+0000 7fb5f8000640 1 --2- 172.21.15.69:0/340603712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f0100a30 0x7fb5f0100e90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.783+0000 7fb5f8000640 1 --2- 172.21.15.69:0/340603712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f01000d0 0x7fb5f01004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.783+0000 7fb5f8000640 1 -- 172.21.15.69:0/340603712 >> 172.21.15.69:0/340603712 conn(0x7fb5f00fbcd0 msgr2=0x7fb5f00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.783+0000 7fb5f8000640 1 -- 172.21.15.69:0/340603712 shutdown_connections 2024-03-19T01:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.783+0000 7fb5f8000640 1 -- 172.21.15.69:0/340603712 wait complete. 2024-03-19T01:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.784+0000 7fb5f8000640 1 Processor -- start 2024-03-19T01:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.784+0000 7fb5f8000640 1 -- start start 2024-03-19T01:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.784+0000 7fb5f8000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f01000d0 0x7fb5f0199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.784+0000 7fb5f8000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5f0100a30 0x7fb5f019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.784+0000 7fb5f8000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f01013d0 0x7fb5f01a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.784+0000 7fb5f8000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f010f6a0 con 0x7fb5f01000d0 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.784+0000 7fb5f8000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f010f520 con 0x7fb5f01013d0 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.784+0000 7fb5f8000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f010f820 con 0x7fb5f0100a30 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.784+0000 7fb5f6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f01013d0 0x7fb5f01a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5f6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f01013d0 0x7fb5f01a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58716/0 (socket says 172.21.15.69:58716) 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5f4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5f0100a30 0x7fb5f019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5f6000640 1 -- 172.21.15.69:0/4226483615 learned_addr learned my addr 172.21.15.69:0/4226483615 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5f5600640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f01000d0 0x7fb5f0199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5f6000640 1 -- 172.21.15.69:0/4226483615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5f0100a30 msgr2=0x7fb5f019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5f6000640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5f0100a30 0x7fb5f019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5f6000640 1 -- 172.21.15.69:0/4226483615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f01000d0 msgr2=0x7fb5f0199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:25.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5f6000640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f01000d0 0x7fb5f0199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:25.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5f6000640 1 -- 172.21.15.69:0/4226483615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f01a1970 con 0x7fb5f01013d0 2024-03-19T01:09:25.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5f6000640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f01013d0 0x7fb5f01a1300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800efd0 tx=0x7fb5e800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:25.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5ee000640 1 -- 172.21.15.69:0/4226483615 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8003b70 con 0x7fb5f01013d0 2024-03-19T01:09:25.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5ee000640 1 -- 172.21.15.69:0/4226483615 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e8003d10 con 0x7fb5f01013d0 2024-03-19T01:09:25.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5ee000640 1 -- 172.21.15.69:0/4226483615 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80109f0 con 0x7fb5f01013d0 2024-03-19T01:09:25.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.785+0000 7fb5f8000640 1 -- 172.21.15.69:0/4226483615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f01a1c60 con 0x7fb5f01013d0 2024-03-19T01:09:25.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.786+0000 7fb5f8000640 1 -- 172.21.15.69:0/4226483615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f01a2130 con 0x7fb5f01013d0 2024-03-19T01:09:25.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.787+0000 7fb5f8000640 1 -- 172.21.15.69:0/4226483615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f0068a90 con 0x7fb5f01013d0 2024-03-19T01:09:25.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.794+0000 7fb5ee000640 1 -- 172.21.15.69:0/4226483615 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb5e8010b90 con 0x7fb5f01013d0 2024-03-19T01:09:25.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.796+0000 7fb5ee000640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5d80a2880 0x7fb5d80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:25.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.796+0000 7fb5ee000640 1 -- 172.21.15.69:0/4226483615 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fb5e80cf640 con 0x7fb5f01013d0 2024-03-19T01:09:25.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.797+0000 7fb5f5600640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5d80a2880 0x7fb5d80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:25.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.797+0000 7fb5ee000640 1 -- 172.21.15.69:0/4226483615 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb5f0068a90 con 0x7fb5f01013d0 2024-03-19T01:09:25.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:25.797+0000 7fb5f5600640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5d80a2880 0x7fb5d80a4d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009b60 tx=0x7fb5e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:26.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:25 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-03-19T01:09:26.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:25 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-03-19T01:09:26.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:25 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-03-19T01:09:26.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:25 smithi163 ceph-mon[67428]: osdmap e92: 12 total, 12 up, 12 in 2024-03-19T01:09:26.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.063+0000 7fb5f8000640 1 -- 172.21.15.69:0/4226483615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5f010aff0 con 0x7fb5f01013d0 2024-03-19T01:09:26.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.066+0000 7fb5ee000640 1 -- 172.21.15.69:0/4226483615 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24736 (secure 0 0 0) 0x7fb5f010aff0 con 0x7fb5f01013d0 2024-03-19T01:09:26.067 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:26.067 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:09:25.783660+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:09:26.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 -- 172.21.15.69:0/4226483615 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5d80a2880 msgr2=0x7fb5d80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:26.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5d80a2880 0x7fb5d80a4d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009b60 tx=0x7fb5e4005f90 comp rx=0 tx=0).stop 2024-03-19T01:09:26.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 -- 172.21.15.69:0/4226483615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f01013d0 msgr2=0x7fb5f01a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:26.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f01013d0 0x7fb5f01a1300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800efd0 tx=0x7fb5e800c6a0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 -- 172.21.15.69:0/4226483615 shutdown_connections 2024-03-19T01:09:26.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb5d80a2880 0x7fb5d80a4d40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f01013d0 0x7fb5f01a1300 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5f0100a30 0x7fb5f019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 --2- 172.21.15.69:0/4226483615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f01000d0 0x7fb5f0199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 -- 172.21.15.69:0/4226483615 >> 172.21.15.69:0/4226483615 conn(0x7fb5f00fbcd0 msgr2=0x7fb5f00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:26.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 -- 172.21.15.69:0/4226483615 shutdown_connections 2024-03-19T01:09:26.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.069+0000 7fb5f8000640 1 -- 172.21.15.69:0/4226483615 wait complete. 2024-03-19T01:09:26.086 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:25 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-03-19T01:09:26.086 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:25 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-03-19T01:09:26.086 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:25 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-03-19T01:09:26.087 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:25 smithi069 ceph-mon[31727]: osdmap e92: 12 total, 12 up, 12 in 2024-03-19T01:09:26.091 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:07:13.368130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:07:14.444206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T01:07:18.189146+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T01:09:26.091 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:09:26.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:25 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-03-19T01:09:26.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:25 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-03-19T01:09:26.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:25 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-03-19T01:09:26.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:25 smithi086 ceph-mon[66829]: osdmap e92: 12 total, 12 up, 12 in 2024-03-19T01:09:26.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.295+0000 7f28d9a00640 1 -- 172.21.15.69:0/3929465966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106270 msgr2=0x7f28d4108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:26.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.295+0000 7f28d9a00640 1 --2- 172.21.15.69:0/3929465966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106270 0x7f28d4108730 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009ca0 tx=0x7f28cc02c100 comp rx=0 tx=0).stop 2024-03-19T01:09:26.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.295+0000 7f28d9a00640 1 -- 172.21.15.69:0/3929465966 shutdown_connections 2024-03-19T01:09:26.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.295+0000 7f28d9a00640 1 --2- 172.21.15.69:0/3929465966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106270 0x7f28d4108730 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.295+0000 7f28d9a00640 1 --2- 172.21.15.69:0/3929465966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d41037d0 0x7f28d4105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.295+0000 7f28d9a00640 1 --2- 172.21.15.69:0/3929465966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28d4100ea0 0x7f28d4103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.295+0000 7f28d9a00640 1 -- 172.21.15.69:0/3929465966 >> 172.21.15.69:0/3929465966 conn(0x7f28d40faea0 msgr2=0x7f28d40fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:26.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.295+0000 7f28d9a00640 1 -- 172.21.15.69:0/3929465966 shutdown_connections 2024-03-19T01:09:26.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.295+0000 7f28d9a00640 1 -- 172.21.15.69:0/3929465966 wait complete. 2024-03-19T01:09:26.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.296+0000 7f28d9a00640 1 Processor -- start 2024-03-19T01:09:26.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.296+0000 7f28d9a00640 1 -- start start 2024-03-19T01:09:26.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d9a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d4100ea0 0x7f28d4102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:26.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28d41037d0 0x7f28d4101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:26.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106270 0x7f28d4101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d9a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f28d410c2c0 con 0x7f28d4100ea0 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28d410c140 con 0x7f28d4106270 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d9a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f28d410c440 con 0x7f28d41037d0 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28d41037d0 0x7f28d4101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28d41037d0 0x7f28d4101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45866/0 (socket says 172.21.15.69:45866) 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d3400640 1 -- 172.21.15.69:0/2109234686 learned_addr learned my addr 172.21.15.69:0/2109234686 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d9000640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106270 0x7f28d4101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d3e00640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d4100ea0 0x7f28d4102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d3400640 1 -- 172.21.15.69:0/2109234686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106270 msgr2=0x7f28d4101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d3400640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106270 0x7f28d4101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d3400640 1 -- 172.21.15.69:0/2109234686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d4100ea0 msgr2=0x7f28d4102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d3400640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d4100ea0 0x7f28d4102e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d3400640 1 -- 172.21.15.69:0/2109234686 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d4102210 con 0x7f28d41037d0 2024-03-19T01:09:26.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.297+0000 7f28d9000640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106270 0x7f28d4101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:26.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.298+0000 7f28d3400640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28d41037d0 0x7f28d4101440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f28c000d740 tx=0x7f28c000dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:26.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.298+0000 7f28d0c00640 1 -- 172.21.15.69:0/2109234686 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0017070 con 0x7f28d41037d0 2024-03-19T01:09:26.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.298+0000 7f28d9a00640 1 -- 172.21.15.69:0/2109234686 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d41a1b20 con 0x7f28d41037d0 2024-03-19T01:09:26.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.298+0000 7f28d0c00640 1 -- 172.21.15.69:0/2109234686 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c000de50 con 0x7f28d41037d0 2024-03-19T01:09:26.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.298+0000 7f28d0c00640 1 -- 172.21.15.69:0/2109234686 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c000f6b0 con 0x7f28d41037d0 2024-03-19T01:09:26.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.298+0000 7f28d9a00640 1 -- 172.21.15.69:0/2109234686 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d41a2070 con 0x7f28d41037d0 2024-03-19T01:09:26.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.299+0000 7f28d9a00640 1 -- 172.21.15.69:0/2109234686 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d41a2240 con 0x7f28d41037d0 2024-03-19T01:09:26.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.301+0000 7f28d0c00640 1 -- 172.21.15.69:0/2109234686 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f28c001a070 con 0x7f28d41037d0 2024-03-19T01:09:26.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.303+0000 7f28d0c00640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f28b80a2800 0x7f28b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:26.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.303+0000 7f28d0c00640 1 -- 172.21.15.69:0/2109234686 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f28c00ce780 con 0x7f28d41037d0 2024-03-19T01:09:26.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.303+0000 7f28d3e00640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f28b80a2800 0x7f28b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:26.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.304+0000 7f28d3e00640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f28b80a2800 0x7f28b80a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009bd0 tx=0x7f28c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:26.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.307+0000 7f28d0c00640 1 -- 172.21.15.69:0/2109234686 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f28c009adc0 con 0x7f28d41037d0 2024-03-19T01:09:26.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.664+0000 7f28d9a00640 1 -- 172.21.15.69:0/2109234686 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28d41a2530 con 0x7f28d41037d0 2024-03-19T01:09:26.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.664+0000 7f28d0c00640 1 -- 172.21.15.69:0/2109234686 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f28c009ee60 con 0x7f28d41037d0 2024-03-19T01:09:26.666 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 -- 172.21.15.69:0/2109234686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f28b80a2800 msgr2=0x7f28b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f28b80a2800 0x7f28b80a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009bd0 tx=0x7f28c4005f90 comp rx=0 tx=0).stop 2024-03-19T01:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 -- 172.21.15.69:0/2109234686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28d41037d0 msgr2=0x7f28d4101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28d41037d0 0x7f28d4101440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f28c000d740 tx=0x7f28c000dc00 comp rx=0 tx=0).stop 2024-03-19T01:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 -- 172.21.15.69:0/2109234686 shutdown_connections 2024-03-19T01:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f28b80a2800 0x7f28b80a4cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106270 0x7f28d4101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28d41037d0 0x7f28d4101440 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 --2- 172.21.15.69:0/2109234686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d4100ea0 0x7f28d4102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 -- 172.21.15.69:0/2109234686 >> 172.21.15.69:0/2109234686 conn(0x7f28d40faea0 msgr2=0x7f28d41000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 -- 172.21.15.69:0/2109234686 shutdown_connections 2024-03-19T01:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.667+0000 7f28d9a00640 1 -- 172.21.15.69:0/2109234686 wait complete. 2024-03-19T01:09:26.685 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:09:26.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.891+0000 7fdcbfe00640 1 -- 172.21.15.69:0/3200927118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb80757e0 msgr2=0x7fdcb8073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:26.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.891+0000 7fdcbfe00640 1 --2- 172.21.15.69:0/3200927118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb80757e0 0x7fdcb8073c30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009860 tx=0x7fdcac02bc00 comp rx=0 tx=0).stop 2024-03-19T01:09:26.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.891+0000 7fdcbfe00640 1 -- 172.21.15.69:0/3200927118 shutdown_connections 2024-03-19T01:09:26.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.892+0000 7fdcbfe00640 1 --2- 172.21.15.69:0/3200927118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcb81096e0 0x7fdcb810bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.892+0000 7fdcbfe00640 1 --2- 172.21.15.69:0/3200927118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8074200 0x7fdcb8074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.892+0000 7fdcbfe00640 1 --2- 172.21.15.69:0/3200927118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb80757e0 0x7fdcb8073c30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.892+0000 7fdcbfe00640 1 -- 172.21.15.69:0/3200927118 >> 172.21.15.69:0/3200927118 conn(0x7fdcb80fcdc0 msgr2=0x7fdcb80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:26.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.892+0000 7fdcbfe00640 1 -- 172.21.15.69:0/3200927118 shutdown_connections 2024-03-19T01:09:26.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.892+0000 7fdcbfe00640 1 -- 172.21.15.69:0/3200927118 wait complete. 2024-03-19T01:09:26.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbfe00640 1 Processor -- start 2024-03-19T01:09:26.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbfe00640 1 -- start start 2024-03-19T01:09:26.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8074200 0x7fdcb8199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:26.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb80757e0 0x7fdcb819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:26.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcb81096e0 0x7fdcb81a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:26.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbfe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb810f6b0 con 0x7fdcb81096e0 2024-03-19T01:09:26.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb810f530 con 0x7fdcb80757e0 2024-03-19T01:09:26.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbfe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb810f830 con 0x7fdcb8074200 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb80757e0 0x7fdcb819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb80757e0 0x7fdcb819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57008/0 (socket says 172.21.15.69:57008) 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbca00640 1 -- 172.21.15.69:0/1590715514 learned_addr learned my addr 172.21.15.69:0/1590715514 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbd400640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8074200 0x7fdcb8199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.893+0000 7fdcbde00640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcb81096e0 0x7fdcb81a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.894+0000 7fdcbca00640 1 -- 172.21.15.69:0/1590715514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8074200 msgr2=0x7fdcb8199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.894+0000 7fdcbca00640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8074200 0x7fdcb8199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.894+0000 7fdcbca00640 1 -- 172.21.15.69:0/1590715514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcb81096e0 msgr2=0x7fdcb81a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.894+0000 7fdcbca00640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcb81096e0 0x7fdcb81a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.894+0000 7fdcbca00640 1 -- 172.21.15.69:0/1590715514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb81a19d0 con 0x7fdcb80757e0 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.894+0000 7fdcbca00640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb80757e0 0x7fdcb819a300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdca400e9f0 tx=0x7fdca400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:26.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.894+0000 7fdcb6000640 1 -- 172.21.15.69:0/1590715514 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca400ccc0 con 0x7fdcb80757e0 2024-03-19T01:09:26.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.894+0000 7fdcb6000640 1 -- 172.21.15.69:0/1590715514 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca4005500 con 0x7fdcb80757e0 2024-03-19T01:09:26.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.894+0000 7fdcb6000640 1 -- 172.21.15.69:0/1590715514 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4010690 con 0x7fdcb80757e0 2024-03-19T01:09:26.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.894+0000 7fdcbfe00640 1 -- 172.21.15.69:0/1590715514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb81a1cc0 con 0x7fdcb80757e0 2024-03-19T01:09:26.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.895+0000 7fdcbfe00640 1 -- 172.21.15.69:0/1590715514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb81a21c0 con 0x7fdcb80757e0 2024-03-19T01:09:26.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.896+0000 7fdcbfe00640 1 -- 172.21.15.69:0/1590715514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb81084a0 con 0x7fdcb80757e0 2024-03-19T01:09:26.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.900+0000 7fdcb6000640 1 -- 172.21.15.69:0/1590715514 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fdca40046c0 con 0x7fdcb80757e0 2024-03-19T01:09:26.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.901+0000 7fdcb6000640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdca00a2880 0x7fdca00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:26.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.902+0000 7fdcb6000640 1 -- 172.21.15.69:0/1590715514 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fdca40cf140 con 0x7fdcb80757e0 2024-03-19T01:09:26.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.902+0000 7fdcbd400640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdca00a2880 0x7fdca00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:26.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.903+0000 7fdcbd400640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdca00a2880 0x7fdca00a4d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdcb819b290 tx=0x7fdcac034b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:26.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:26.910+0000 7fdcb6000640 1 -- 172.21.15.69:0/1590715514 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdca409bdd0 con 0x7fdcb80757e0 2024-03-19T01:09:26.927 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:26 smithi069 ceph-mon[31727]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-19T01:09:26.927 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:26 smithi069 ceph-mon[31727]: osdmap e93: 12 total, 12 up, 12 in 2024-03-19T01:09:26.927 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:26 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/4226483615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:09:26.927 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:26 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2109234686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:09:27.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:26 smithi163 ceph-mon[67428]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-19T01:09:27.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:26 smithi163 ceph-mon[67428]: osdmap e93: 12 total, 12 up, 12 in 2024-03-19T01:09:27.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:26 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/4226483615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:09:27.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:26 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2109234686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:09:27.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.154+0000 7fdcbfe00640 1 -- 172.21.15.69:0/1590715514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdcb8062450 con 0x7fdcb80757e0 2024-03-19T01:09:27.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.155+0000 7fdcb6000640 1 -- 172.21.15.69:0/1590715514 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fdca4097030 con 0x7fdcb80757e0 2024-03-19T01:09:27.156 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:09:27.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 -- 172.21.15.69:0/1590715514 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdca00a2880 msgr2=0x7fdca00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:27.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdca00a2880 0x7fdca00a4d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdcb819b290 tx=0x7fdcac034b50 comp rx=0 tx=0).stop 2024-03-19T01:09:27.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 -- 172.21.15.69:0/1590715514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb80757e0 msgr2=0x7fdcb819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:27.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb80757e0 0x7fdcb819a300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdca400e9f0 tx=0x7fdca400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 -- 172.21.15.69:0/1590715514 shutdown_connections 2024-03-19T01:09:27.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdca00a2880 0x7fdca00a4d40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcb81096e0 0x7fdcb81a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb80757e0 0x7fdcb819a300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 --2- 172.21.15.69:0/1590715514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8074200 0x7fdcb8199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 -- 172.21.15.69:0/1590715514 >> 172.21.15.69:0/1590715514 conn(0x7fdcb80fcdc0 msgr2=0x7fdcb8105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:27.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 -- 172.21.15.69:0/1590715514 shutdown_connections 2024-03-19T01:09:27.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.158+0000 7fdcbfe00640 1 -- 172.21.15.69:0/1590715514 wait complete. 2024-03-19T01:09:27.185 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:09:27.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:26 smithi086 ceph-mon[66829]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-19T01:09:27.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:26 smithi086 ceph-mon[66829]: osdmap e93: 12 total, 12 up, 12 in 2024-03-19T01:09:27.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:26 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/4226483615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:09:27.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:26 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2109234686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:09:27.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.426+0000 7f0330e00640 1 -- 172.21.15.69:0/3198589085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032c103920 msgr2=0x7f032c103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:27.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.426+0000 7f0330e00640 1 --2- 172.21.15.69:0/3198589085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032c103920 0x7f032c103d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f031c009900 tx=0x7f031c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:09:27.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.427+0000 7f0330e00640 1 -- 172.21.15.69:0/3198589085 shutdown_connections 2024-03-19T01:09:27.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.427+0000 7f0330e00640 1 --2- 172.21.15.69:0/3198589085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f032c1054d0 0x7f032c10bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.427+0000 7f0330e00640 1 --2- 172.21.15.69:0/3198589085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f032c104b10 0x7f032c104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.427+0000 7f0330e00640 1 --2- 172.21.15.69:0/3198589085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032c103920 0x7f032c103d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.427+0000 7f0330e00640 1 -- 172.21.15.69:0/3198589085 >> 172.21.15.69:0/3198589085 conn(0x7f032c0fee90 msgr2=0x7f032c1012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:27.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.427+0000 7f0330e00640 1 -- 172.21.15.69:0/3198589085 shutdown_connections 2024-03-19T01:09:27.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.427+0000 7f0330e00640 1 -- 172.21.15.69:0/3198589085 wait complete. 2024-03-19T01:09:27.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f0330e00640 1 Processor -- start 2024-03-19T01:09:27.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f0330e00640 1 -- start start 2024-03-19T01:09:27.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f0330e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032c103920 0x7f032c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:27.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f0330e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f032c104b10 0x7f032c19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f0330e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f032c1054d0 0x7f032c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f0330e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f032c10f5a0 con 0x7f032c1054d0 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f0330e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f032c10f420 con 0x7f032c103920 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f0330e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f032c10f720 con 0x7f032c104b10 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f032aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f032c104b10 0x7f032c19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f032b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032c103920 0x7f032c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f032aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f032c104b10 0x7f032c19a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45890/0 (socket says 172.21.15.69:45890) 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.428+0000 7f032aa00640 1 -- 172.21.15.69:0/3976504638 learned_addr learned my addr 172.21.15.69:0/3976504638 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.429+0000 7f032b400640 1 -- 172.21.15.69:0/3976504638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f032c104b10 msgr2=0x7f032c19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.429+0000 7f032b400640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f032c104b10 0x7f032c19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.429+0000 7f032b400640 1 -- 172.21.15.69:0/3976504638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f032c1054d0 msgr2=0x7f032c1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.429+0000 7f032be00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f032c1054d0 0x7f032c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:27.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.429+0000 7f032b400640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f032c1054d0 0x7f032c1a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.429+0000 7f032b400640 1 -- 172.21.15.69:0/3976504638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c1a1800 con 0x7f032c103920 2024-03-19T01:09:27.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.429+0000 7f032aa00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f032c104b10 0x7f032c19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:27.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.429+0000 7f032be00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f032c1054d0 0x7f032c1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:09:27.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.429+0000 7f032b400640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032c103920 0x7f032c199be0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f031c02c170 tx=0x7f031c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:27.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.430+0000 7f0323e00640 1 -- 172.21.15.69:0/3976504638 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c041070 con 0x7f032c103920 2024-03-19T01:09:27.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.430+0000 7f0323e00640 1 -- 172.21.15.69:0/3976504638 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f031c004450 con 0x7f032c103920 2024-03-19T01:09:27.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.430+0000 7f0330e00640 1 -- 172.21.15.69:0/3976504638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c1a1a90 con 0x7f032c103920 2024-03-19T01:09:27.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.430+0000 7f0323e00640 1 -- 172.21.15.69:0/3976504638 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c003e50 con 0x7f032c103920 2024-03-19T01:09:27.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.430+0000 7f0330e00640 1 -- 172.21.15.69:0/3976504638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c1a1fa0 con 0x7f032c103920 2024-03-19T01:09:27.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.431+0000 7f0330e00640 1 -- 172.21.15.69:0/3976504638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032c10ac40 con 0x7f032c103920 2024-03-19T01:09:27.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.433+0000 7f0323e00640 1 -- 172.21.15.69:0/3976504638 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f031c044070 con 0x7f032c103920 2024-03-19T01:09:27.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.435+0000 7f0323e00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03100a2a20 0x7f03100a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:27.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.435+0000 7f0323e00640 1 -- 172.21.15.69:0/3976504638 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f031c0f0b20 con 0x7f032c103920 2024-03-19T01:09:27.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.435+0000 7f032aa00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03100a2a20 0x7f03100a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:27.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.436+0000 7f032aa00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03100a2a20 0x7f03100a4ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f032c19b0f0 tx=0x7f0318005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:27.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.436+0000 7f0323e00640 1 -- 172.21.15.69:0/3976504638 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f031c0bd7b0 con 0x7f032c103920 2024-03-19T01:09:27.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.653+0000 7f0330e00640 1 -- 172.21.15.69:0/3976504638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f032c02d460 con 0x7f032c103920 2024-03-19T01:09:27.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.654+0000 7f0323e00640 1 -- 172.21.15.69:0/3976504638 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f031c0bd0b0 con 0x7f032c103920 2024-03-19T01:09:27.655 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-03-19T01:09:27.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 -- 172.21.15.69:0/3976504638 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03100a2a20 msgr2=0x7f03100a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:27.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03100a2a20 0x7f03100a4ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f032c19b0f0 tx=0x7f0318005f90 comp rx=0 tx=0).stop 2024-03-19T01:09:27.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 -- 172.21.15.69:0/3976504638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032c103920 msgr2=0x7f032c199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:27.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032c103920 0x7f032c199be0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f031c02c170 tx=0x7f031c02c650 comp rx=0 tx=0).stop 2024-03-19T01:09:27.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 -- 172.21.15.69:0/3976504638 shutdown_connections 2024-03-19T01:09:27.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f03100a2a20 0x7f03100a4ee0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f032c1054d0 0x7f032c1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f032c104b10 0x7f032c19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 --2- 172.21.15.69:0/3976504638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f032c103920 0x7f032c199be0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 -- 172.21.15.69:0/3976504638 >> 172.21.15.69:0/3976504638 conn(0x7f032c0fee90 msgr2=0x7f032c107e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:27.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 -- 172.21.15.69:0/3976504638 shutdown_connections 2024-03-19T01:09:27.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.656+0000 7f0330e00640 1 -- 172.21.15.69:0/3976504638 wait complete. 2024-03-19T01:09:27.675 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T01:09:27.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.896+0000 7f9d89200640 1 -- 172.21.15.69:0/2911530404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d84104c50 msgr2=0x7f9d841050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:27.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.896+0000 7f9d89200640 1 --2- 172.21.15.69:0/2911530404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d84104c50 0x7f9d841050d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009900 tx=0x7f9d7002bc30 comp rx=0 tx=0).stop 2024-03-19T01:09:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.896+0000 7f9d89200640 1 -- 172.21.15.69:0/2911530404 shutdown_connections 2024-03-19T01:09:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.896+0000 7f9d89200640 1 --2- 172.21.15.69:0/2911530404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d84105610 0x7f9d8410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.896+0000 7f9d89200640 1 --2- 172.21.15.69:0/2911530404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d84104c50 0x7f9d841050d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.896+0000 7f9d89200640 1 --2- 172.21.15.69:0/2911530404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d84103a60 0x7f9d84103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.896+0000 7f9d89200640 1 -- 172.21.15.69:0/2911530404 >> 172.21.15.69:0/2911530404 conn(0x7f9d840fefd0 msgr2=0x7f9d84101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.896+0000 7f9d89200640 1 -- 172.21.15.69:0/2911530404 shutdown_connections 2024-03-19T01:09:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.896+0000 7f9d89200640 1 -- 172.21.15.69:0/2911530404 wait complete. 2024-03-19T01:09:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.898+0000 7f9d89200640 1 Processor -- start 2024-03-19T01:09:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.898+0000 7f9d89200640 1 -- start start 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.898+0000 7f9d89200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d84103a60 0x7f9d84199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.898+0000 7f9d89200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d84104c50 0x7f9d8419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.898+0000 7f9d89200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d84105610 0x7f9d841a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.898+0000 7f9d89200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8410f730 con 0x7f9d84104c50 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.898+0000 7f9d89200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8410f5b0 con 0x7f9d84105610 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.898+0000 7f9d89200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8410f8b0 con 0x7f9d84103a60 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d81e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d84103a60 0x7f9d84199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d81e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d84103a60 0x7f9d84199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45902/0 (socket says 172.21.15.69:45902) 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d81e00640 1 -- 172.21.15.69:0/3566917569 learned_addr learned my addr 172.21.15.69:0/3566917569 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d81400640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d84104c50 0x7f9d8419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d82800640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d84105610 0x7f9d841a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d81e00640 1 -- 172.21.15.69:0/3566917569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d84105610 msgr2=0x7f9d841a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d81e00640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d84105610 0x7f9d841a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d81e00640 1 -- 172.21.15.69:0/3566917569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d84104c50 msgr2=0x7f9d8419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:27.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d81e00640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d84104c50 0x7f9d8419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:27.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d81e00640 1 -- 172.21.15.69:0/3566917569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d841a1960 con 0x7f9d84103a60 2024-03-19T01:09:27.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d81e00640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d84103a60 0x7f9d84199d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400e9c0 tx=0x7f9d7400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:27.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.899+0000 7f9d7aa00640 1 -- 172.21.15.69:0/3566917569 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7400cd80 con 0x7f9d84103a60 2024-03-19T01:09:27.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.900+0000 7f9d7aa00640 1 -- 172.21.15.69:0/3566917569 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d740035f0 con 0x7f9d84103a60 2024-03-19T01:09:27.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.900+0000 7f9d7aa00640 1 -- 172.21.15.69:0/3566917569 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74010690 con 0x7f9d84103a60 2024-03-19T01:09:27.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.900+0000 7f9d89200640 1 -- 172.21.15.69:0/3566917569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d841a1c50 con 0x7f9d84103a60 2024-03-19T01:09:27.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.900+0000 7f9d89200640 1 -- 172.21.15.69:0/3566917569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d841a2120 con 0x7f9d84103a60 2024-03-19T01:09:27.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.901+0000 7f9d89200640 1 -- 172.21.15.69:0/3566917569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8410ad80 con 0x7f9d84103a60 2024-03-19T01:09:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.903+0000 7f9d7aa00640 1 -- 172.21.15.69:0/3566917569 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9d740037a0 con 0x7f9d84103a60 2024-03-19T01:09:27.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.905+0000 7f9d7aa00640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9d680a2800 0x7f9d680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:27.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.906+0000 7f9d7aa00640 1 -- 172.21.15.69:0/3566917569 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f9d740cf250 con 0x7f9d84103a60 2024-03-19T01:09:27.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.906+0000 7f9d81400640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9d680a2800 0x7f9d680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:27.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.906+0000 7f9d81400640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9d680a2800 0x7f9d680a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d7002c240 tx=0x7f9d7002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:27.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:27.908+0000 7f9d7aa00640 1 -- 172.21.15.69:0/3566917569 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9d7409bee0 con 0x7f9d84103a60 2024-03-19T01:09:27.930 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:27 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1590715514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:09:27.931 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:27 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3976504638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:09:28.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:27 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1590715514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:09:28.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:27 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3976504638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:09:28.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.128+0000 7f9d89200640 1 -- 172.21.15.69:0/3566917569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9d8402d1f0 con 0x7f9d84103a60 2024-03-19T01:09:28.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.129+0000 7f9d7aa00640 1 -- 172.21.15.69:0/3566917569 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f9d8402d1f0 con 0x7f9d84103a60 2024-03-19T01:09:28.130 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:09:28.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 -- 172.21.15.69:0/3566917569 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9d680a2800 msgr2=0x7f9d680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:28.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9d680a2800 0x7f9d680a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d7002c240 tx=0x7f9d7002df90 comp rx=0 tx=0).stop 2024-03-19T01:09:28.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 -- 172.21.15.69:0/3566917569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d84103a60 msgr2=0x7f9d84199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:28.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d84103a60 0x7f9d84199d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400e9c0 tx=0x7f9d7400ee80 comp rx=0 tx=0).stop 2024-03-19T01:09:28.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 -- 172.21.15.69:0/3566917569 shutdown_connections 2024-03-19T01:09:28.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9d680a2800 0x7f9d680a4cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d84105610 0x7f9d841a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d84104c50 0x7f9d8419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 --2- 172.21.15.69:0/3566917569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d84103a60 0x7f9d84199d40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 -- 172.21.15.69:0/3566917569 >> 172.21.15.69:0/3566917569 conn(0x7f9d840fefd0 msgr2=0x7f9d84107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:28.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 -- 172.21.15.69:0/3566917569 shutdown_connections 2024-03-19T01:09:28.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.132+0000 7f9d89200640 1 -- 172.21.15.69:0/3566917569 wait complete. 2024-03-19T01:09:28.151 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:09:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:27 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1590715514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:09:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:27 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3976504638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:09:28.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.337+0000 7f1faf800640 1 -- 172.21.15.69:0/197128067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8100130 msgr2=0x7f1fa8100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:28.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.337+0000 7f1faf800640 1 --2- 172.21.15.69:0/197128067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8100130 0x7f1fa8100550 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009900 tx=0x7f1f9c02bc40 comp rx=0 tx=0).stop 2024-03-19T01:09:28.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.337+0000 7f1faf800640 1 -- 172.21.15.69:0/197128067 shutdown_connections 2024-03-19T01:09:28.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.337+0000 7f1faf800640 1 --2- 172.21.15.69:0/197128067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa8101430 0x7f1fa81018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.337+0000 7f1faf800640 1 --2- 172.21.15.69:0/197128067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fa8100a90 0x7f1fa8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.337+0000 7f1faf800640 1 --2- 172.21.15.69:0/197128067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8100130 0x7f1fa8100550 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.337+0000 7f1faf800640 1 -- 172.21.15.69:0/197128067 >> 172.21.15.69:0/197128067 conn(0x7f1fa80fbcd0 msgr2=0x7f1fa80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:28.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.337+0000 7f1faf800640 1 -- 172.21.15.69:0/197128067 shutdown_connections 2024-03-19T01:09:28.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.337+0000 7f1faf800640 1 -- 172.21.15.69:0/197128067 wait complete. 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.338+0000 7f1faf800640 1 Processor -- start 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.338+0000 7f1faf800640 1 -- start start 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1faf800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8100130 0x7f1fa8199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1faf800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa8100a90 0x7f1fa819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1faf800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fa8101430 0x7f1fa81a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1faf800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa810f710 con 0x7f1fa8101430 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1faf800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa810f590 con 0x7f1fa8100130 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1faf800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa810f890 con 0x7f1fa8100a90 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1face00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8100130 0x7f1fa8199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1face00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8100130 0x7f1fa8199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57068/0 (socket says 172.21.15.69:57068) 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1face00640 1 -- 172.21.15.69:0/2149082644 learned_addr learned my addr 172.21.15.69:0/2149082644 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:28.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1fa7e00640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa8100a90 0x7f1fa819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1fad800640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fa8101430 0x7f1fa81a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1face00640 1 -- 172.21.15.69:0/2149082644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa8100a90 msgr2=0x7f1fa819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1face00640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa8100a90 0x7f1fa819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1face00640 1 -- 172.21.15.69:0/2149082644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fa8101430 msgr2=0x7f1fa81a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1face00640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fa8101430 0x7f1fa81a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.339+0000 7f1face00640 1 -- 172.21.15.69:0/2149082644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa81a1970 con 0x7f1fa8100130 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.340+0000 7f1face00640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8100130 0x7f1fa8199d50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c02c8f0 tx=0x7f1f9c02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.340+0000 7f1fa5600640 1 -- 172.21.15.69:0/2149082644 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c041070 con 0x7f1fa8100130 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.340+0000 7f1fa5600640 1 -- 172.21.15.69:0/2149082644 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f9c004450 con 0x7f1fa8100130 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.340+0000 7f1faf800640 1 -- 172.21.15.69:0/2149082644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa81a1c00 con 0x7f1fa8100130 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.340+0000 7f1fa5600640 1 -- 172.21.15.69:0/2149082644 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c034440 con 0x7f1fa8100130 2024-03-19T01:09:28.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.340+0000 7f1faf800640 1 -- 172.21.15.69:0/2149082644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa81a20f0 con 0x7f1fa8100130 2024-03-19T01:09:28.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.341+0000 7f1faf800640 1 -- 172.21.15.69:0/2149082644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa810b010 con 0x7f1fa8100130 2024-03-19T01:09:28.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.343+0000 7f1fa5600640 1 -- 172.21.15.69:0/2149082644 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f1f9c044070 con 0x7f1fa8100130 2024-03-19T01:09:28.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.344+0000 7f1fa5600640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1f900a2800 0x7f1f900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:28.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.344+0000 7f1fa5600640 1 -- 172.21.15.69:0/2149082644 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f1f9c0f0ce0 con 0x7f1fa8100130 2024-03-19T01:09:28.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.344+0000 7f1fa7e00640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1f900a2800 0x7f1f900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:28.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.345+0000 7f1fa7e00640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1f900a2800 0x7f1f900a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1fa819b260 tx=0x7f1f94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:28.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.351+0000 7f1fa5600640 1 -- 172.21.15.69:0/2149082644 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1f9c0bd320 con 0x7f1fa8100130 2024-03-19T01:09:28.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.590+0000 7f1faf800640 1 -- 172.21.15.69:0/2149082644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fa802d3c0 con 0x7f1fa8100130 2024-03-19T01:09:28.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.591+0000 7f1fa5600640 1 -- 172.21.15.69:0/2149082644 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7f1f9c0c13c0 con 0x7f1fa8100130 2024-03-19T01:09:28.592 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:28.592 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:07:50.808640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581,24476,24580,24494,24550]},"id":1}]} 2024-03-19T01:09:28.592 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-03-19T01:09:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 -- 172.21.15.69:0/2149082644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1f900a2800 msgr2=0x7f1f900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1f900a2800 0x7f1f900a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1fa819b260 tx=0x7f1f94005f90 comp rx=0 tx=0).stop 2024-03-19T01:09:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 -- 172.21.15.69:0/2149082644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8100130 msgr2=0x7f1fa8199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8100130 0x7f1fa8199d50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c02c8f0 tx=0x7f1f9c02c660 comp rx=0 tx=0).stop 2024-03-19T01:09:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 -- 172.21.15.69:0/2149082644 shutdown_connections 2024-03-19T01:09:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1f900a2800 0x7f1f900a4cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fa8101430 0x7f1fa81a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa8100a90 0x7f1fa819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 --2- 172.21.15.69:0/2149082644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8100130 0x7f1fa8199d50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 -- 172.21.15.69:0/2149082644 >> 172.21.15.69:0/2149082644 conn(0x7f1fa80fbcd0 msgr2=0x7f1fa80fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:28.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 -- 172.21.15.69:0/2149082644 shutdown_connections 2024-03-19T01:09:28.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.594+0000 7f1faf800640 1 -- 172.21.15.69:0/2149082644 wait complete. 2024-03-19T01:09:28.613 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:09:28.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.825+0000 7f9e41200640 1 -- 172.21.15.69:0/1529582016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3c109540 msgr2=0x7f9e3c10b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:28.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.825+0000 7f9e41200640 1 --2- 172.21.15.69:0/1529582016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3c109540 0x7f9e3c10b9d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009ca0 tx=0x7f9e3402c190 comp rx=0 tx=0).stop 2024-03-19T01:09:28.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.825+0000 7f9e41200640 1 -- 172.21.15.69:0/1529582016 shutdown_connections 2024-03-19T01:09:28.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.825+0000 7f9e41200640 1 --2- 172.21.15.69:0/1529582016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3c109540 0x7f9e3c10b9d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.825+0000 7f9e41200640 1 --2- 172.21.15.69:0/1529582016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e3c074200 0x7f9e3c074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.825+0000 7f9e41200640 1 --2- 172.21.15.69:0/1529582016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e3c0757e0 0x7f9e3c073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.825+0000 7f9e41200640 1 -- 172.21.15.69:0/1529582016 >> 172.21.15.69:0/1529582016 conn(0x7f9e3c0fcc00 msgr2=0x7f9e3c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:28.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.825+0000 7f9e41200640 1 -- 172.21.15.69:0/1529582016 shutdown_connections 2024-03-19T01:09:28.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.826+0000 7f9e41200640 1 -- 172.21.15.69:0/1529582016 wait complete. 2024-03-19T01:09:28.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.826+0000 7f9e41200640 1 Processor -- start 2024-03-19T01:09:28.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e41200640 1 -- start start 2024-03-19T01:09:28.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e41200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3c074200 0x7f9e3c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:28.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e41200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e3c0757e0 0x7f9e3c19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e41200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e3c109540 0x7f9e3c1a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e41200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c10f520 con 0x7f9e3c0757e0 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e41200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c10f3a0 con 0x7f9e3c074200 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e41200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c10f6a0 con 0x7f9e3c109540 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e3b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3c074200 0x7f9e3c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e3b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3c074200 0x7f9e3c199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57094/0 (socket says 172.21.15.69:57094) 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e3b400640 1 -- 172.21.15.69:0/1571923781 learned_addr learned my addr 172.21.15.69:0/1571923781 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e3aa00640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e3c0757e0 0x7f9e3c19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e3be00640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e3c109540 0x7f9e3c1a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.827+0000 7f9e3b400640 1 -- 172.21.15.69:0/1571923781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e3c109540 msgr2=0x7f9e3c1a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.828+0000 7f9e3b400640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e3c109540 0x7f9e3c1a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.828+0000 7f9e3b400640 1 -- 172.21.15.69:0/1571923781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e3c0757e0 msgr2=0x7f9e3c19a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.828+0000 7f9e3b400640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e3c0757e0 0x7f9e3c19a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.828+0000 7f9e3b400640 1 -- 172.21.15.69:0/1571923781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3c1a1720 con 0x7f9e3c074200 2024-03-19T01:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.828+0000 7f9e3be00640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e3c109540 0x7f9e3c1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.828+0000 7f9e3b400640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3c074200 0x7f9e3c199b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00e9f0 tx=0x7f9e2c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.828+0000 7f9e33e00640 1 -- 172.21.15.69:0/1571923781 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c00cd80 con 0x7f9e3c074200 2024-03-19T01:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.828+0000 7f9e33e00640 1 -- 172.21.15.69:0/1571923781 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e2c005500 con 0x7f9e3c074200 2024-03-19T01:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.828+0000 7f9e33e00640 1 -- 172.21.15.69:0/1571923781 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c010690 con 0x7f9e3c074200 2024-03-19T01:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.828+0000 7f9e41200640 1 -- 172.21.15.69:0/1571923781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3c1a1a10 con 0x7f9e3c074200 2024-03-19T01:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.828+0000 7f9e41200640 1 -- 172.21.15.69:0/1571923781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e3c1a1f10 con 0x7f9e3c074200 2024-03-19T01:09:28.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.830+0000 7f9e41200640 1 -- 172.21.15.69:0/1571923781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e3c108300 con 0x7f9e3c074200 2024-03-19T01:09:28.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.831+0000 7f9e33e00640 1 -- 172.21.15.69:0/1571923781 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9e2c0046c0 con 0x7f9e3c074200 2024-03-19T01:09:28.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.833+0000 7f9e33e00640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e200a2800 0x7f9e200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:28.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.833+0000 7f9e33e00640 1 -- 172.21.15.69:0/1571923781 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f9e2c0cf260 con 0x7f9e3c074200 2024-03-19T01:09:28.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.833+0000 7f9e3aa00640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e200a2800 0x7f9e200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:28.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.834+0000 7f9e3aa00640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e200a2800 0x7f9e200a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9e280095f0 tx=0x7f9e28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:28.836+0000 7f9e33e00640 1 -- 172.21.15.69:0/1571923781 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9e2c09bef0 con 0x7f9e3c074200 2024-03-19T01:09:29.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:28 smithi069 ceph-mon[31727]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:29.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:28 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3566917569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:09:29.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:28 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2149082644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:09:29.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.048+0000 7f9e41200640 1 -- 172.21.15.69:0/1571923781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e3c02d1f0 con 0x7f9e3c074200 2024-03-19T01:09:29.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.050+0000 7f9e33e00640 1 -- 172.21.15.69:0/1571923781 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24736 (secure 0 0 0) 0x7f9e2c09b7f0 con 0x7f9e3c074200 2024-03-19T01:09:29.051 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:29.052 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:09:25.783660+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:09:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 -- 172.21.15.69:0/1571923781 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e200a2800 msgr2=0x7f9e200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e200a2800 0x7f9e200a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9e280095f0 tx=0x7f9e28005f90 comp rx=0 tx=0).stop 2024-03-19T01:09:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 -- 172.21.15.69:0/1571923781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3c074200 msgr2=0x7f9e3c199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3c074200 0x7f9e3c199b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00e9f0 tx=0x7f9e2c00edf0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 -- 172.21.15.69:0/1571923781 shutdown_connections 2024-03-19T01:09:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e200a2800 0x7f9e200a4cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e3c109540 0x7f9e3c1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e3c0757e0 0x7f9e3c19a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 --2- 172.21.15.69:0/1571923781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3c074200 0x7f9e3c199b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 -- 172.21.15.69:0/1571923781 >> 172.21.15.69:0/1571923781 conn(0x7f9e3c0fcc00 msgr2=0x7f9e3c1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:29.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 -- 172.21.15.69:0/1571923781 shutdown_connections 2024-03-19T01:09:29.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.053+0000 7f9e41200640 1 -- 172.21.15.69:0/1571923781 wait complete. 2024-03-19T01:09:29.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:28 smithi163 ceph-mon[67428]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:29.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:28 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3566917569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:09:29.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:28 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2149082644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:09:29.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:07:13.368130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:07:14.444206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T01:07:18.189146+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T01:09:29.073 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:09:29.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:28 smithi086 ceph-mon[66829]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:29.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:28 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3566917569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:09:29.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:28 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2149082644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:09:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.278+0000 7f5e19200640 1 -- 172.21.15.69:0/1223125176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e14074280 msgr2=0x7f5e14074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.278+0000 7f5e19200640 1 --2- 172.21.15.69:0/1223125176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e14074280 0x7f5e14074700 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009900 tx=0x7f5e0002bc30 comp rx=0 tx=0).stop 2024-03-19T01:09:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.278+0000 7f5e19200640 1 -- 172.21.15.69:0/1223125176 shutdown_connections 2024-03-19T01:09:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.278+0000 7f5e19200640 1 --2- 172.21.15.69:0/1223125176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e1410de00 0x7f5e141101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.278+0000 7f5e19200640 1 --2- 172.21.15.69:0/1223125176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e14074280 0x7f5e14074700 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.278+0000 7f5e19200640 1 --2- 172.21.15.69:0/1223125176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e14075860 0x7f5e14073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.278+0000 7f5e19200640 1 -- 172.21.15.69:0/1223125176 >> 172.21.15.69:0/1223125176 conn(0x7f5e140fcda0 msgr2=0x7f5e140ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.278+0000 7f5e19200640 1 -- 172.21.15.69:0/1223125176 shutdown_connections 2024-03-19T01:09:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.278+0000 7f5e19200640 1 -- 172.21.15.69:0/1223125176 wait complete. 2024-03-19T01:09:29.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.280+0000 7f5e19200640 1 Processor -- start 2024-03-19T01:09:29.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.280+0000 7f5e19200640 1 -- start start 2024-03-19T01:09:29.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e19200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e14074280 0x7f5e1419e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:29.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e19200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e14075860 0x7f5e1419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:29.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e19200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1410de00 0x7f5e141a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e19200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e14113ad0 con 0x7f5e14075860 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e19200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e14113950 con 0x7f5e1410de00 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e19200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5e14113c50 con 0x7f5e14074280 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e11e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e14074280 0x7f5e1419e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e11400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e14075860 0x7f5e1419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e12800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1410de00 0x7f5e141a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e11e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e14074280 0x7f5e1419e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45966/0 (socket says 172.21.15.69:45966) 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e11e00640 1 -- 172.21.15.69:0/2020205312 learned_addr learned my addr 172.21.15.69:0/2020205312 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e11e00640 1 -- 172.21.15.69:0/2020205312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1410de00 msgr2=0x7f5e141a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e11e00640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1410de00 0x7f5e141a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e11e00640 1 -- 172.21.15.69:0/2020205312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e14075860 msgr2=0x7f5e1419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e11e00640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e14075860 0x7f5e1419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.281+0000 7f5e11e00640 1 -- 172.21.15.69:0/2020205312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e141a5da0 con 0x7f5e14074280 2024-03-19T01:09:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.282+0000 7f5e11e00640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e14074280 0x7f5e1419e180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400e9f0 tx=0x7f5e0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:29.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.282+0000 7f5e0aa00640 1 -- 172.21.15.69:0/2020205312 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0400cd80 con 0x7f5e14074280 2024-03-19T01:09:29.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.282+0000 7f5e0aa00640 1 -- 172.21.15.69:0/2020205312 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e04010910 con 0x7f5e14074280 2024-03-19T01:09:29.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.282+0000 7f5e19200640 1 -- 172.21.15.69:0/2020205312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e141a6090 con 0x7f5e14074280 2024-03-19T01:09:29.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.282+0000 7f5e0aa00640 1 -- 172.21.15.69:0/2020205312 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04018990 con 0x7f5e14074280 2024-03-19T01:09:29.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.282+0000 7f5e19200640 1 -- 172.21.15.69:0/2020205312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e141a65b0 con 0x7f5e14074280 2024-03-19T01:09:29.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.283+0000 7f5e19200640 1 -- 172.21.15.69:0/2020205312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e14103fd0 con 0x7f5e14074280 2024-03-19T01:09:29.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.285+0000 7f5e0aa00640 1 -- 172.21.15.69:0/2020205312 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f5e04010ac0 con 0x7f5e14074280 2024-03-19T01:09:29.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.286+0000 7f5e0aa00640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5df80a2800 0x7f5df80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:29.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.287+0000 7f5e0aa00640 1 -- 172.21.15.69:0/2020205312 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f5e040cf1c0 con 0x7f5e14074280 2024-03-19T01:09:29.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.287+0000 7f5e11400640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5df80a2800 0x7f5df80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:29.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.287+0000 7f5e11400640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5df80a2800 0x7f5df80a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e0002c240 tx=0x7f5e00000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:29.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.290+0000 7f5e0aa00640 1 -- 172.21.15.69:0/2020205312 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5e0409b800 con 0x7f5e14074280 2024-03-19T01:09:29.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.493+0000 7f5e19200640 1 -- 172.21.15.69:0/2020205312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5e14062420 con 0x7f5e14074280 2024-03-19T01:09:29.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.493+0000 7f5e0aa00640 1 -- 172.21.15.69:0/2020205312 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f5e0409f8a0 con 0x7f5e14074280 2024-03-19T01:09:29.494 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:09:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 -- 172.21.15.69:0/2020205312 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5df80a2800 msgr2=0x7f5df80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5df80a2800 0x7f5df80a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e0002c240 tx=0x7f5e00000bc0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 -- 172.21.15.69:0/2020205312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e14074280 msgr2=0x7f5e1419e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e14074280 0x7f5e1419e180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400e9f0 tx=0x7f5e0400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 -- 172.21.15.69:0/2020205312 shutdown_connections 2024-03-19T01:09:29.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5df80a2800 0x7f5df80a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e1410de00 0x7f5e141a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e14075860 0x7f5e1419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 --2- 172.21.15.69:0/2020205312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e14074280 0x7f5e1419e180 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 -- 172.21.15.69:0/2020205312 >> 172.21.15.69:0/2020205312 conn(0x7f5e140fcda0 msgr2=0x7f5e141011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:29.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 -- 172.21.15.69:0/2020205312 shutdown_connections 2024-03-19T01:09:29.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.497+0000 7f5e19200640 1 -- 172.21.15.69:0/2020205312 wait complete. 2024-03-19T01:09:29.521 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:09:29.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.738+0000 7fd002a00640 1 -- 172.21.15.69:0/1437154259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcffc075800 msgr2=0x7fcffc073c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.738+0000 7fd002a00640 1 --2- 172.21.15.69:0/1437154259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcffc075800 0x7fcffc073c50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009900 tx=0x7fcff002bcd0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.739+0000 7fd002a00640 1 -- 172.21.15.69:0/1437154259 shutdown_connections 2024-03-19T01:09:29.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.739+0000 7fd002a00640 1 --2- 172.21.15.69:0/1437154259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcffc1095b0 0x7fcffc10ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.739+0000 7fd002a00640 1 --2- 172.21.15.69:0/1437154259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcffc074220 0x7fcffc074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.739+0000 7fd002a00640 1 --2- 172.21.15.69:0/1437154259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcffc075800 0x7fcffc073c50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.739+0000 7fd002a00640 1 -- 172.21.15.69:0/1437154259 >> 172.21.15.69:0/1437154259 conn(0x7fcffc0fcc90 msgr2=0x7fcffc0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:29.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.739+0000 7fd002a00640 1 -- 172.21.15.69:0/1437154259 shutdown_connections 2024-03-19T01:09:29.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.739+0000 7fd002a00640 1 -- 172.21.15.69:0/1437154259 wait complete. 2024-03-19T01:09:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.739+0000 7fd002a00640 1 Processor -- start 2024-03-19T01:09:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd002a00640 1 -- start start 2024-03-19T01:09:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd002a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcffc074220 0x7fcffc199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd002a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcffc075800 0x7fcffc19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd002a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcffc1095b0 0x7fcffc1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd002a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10f5c0 con 0x7fcffc074220 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd002a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10f440 con 0x7fcffc075800 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd002a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10f740 con 0x7fcffc1095b0 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd000c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcffc075800 0x7fcffc19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd001600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcffc074220 0x7fcffc199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd000c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcffc075800 0x7fcffc19a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57138/0 (socket says 172.21.15.69:57138) 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd000c00640 1 -- 172.21.15.69:0/724098686 learned_addr learned my addr 172.21.15.69:0/724098686 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.740+0000 7fd002000640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcffc1095b0 0x7fcffc1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.741+0000 7fd000c00640 1 -- 172.21.15.69:0/724098686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcffc1095b0 msgr2=0x7fcffc1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.741+0000 7fd000c00640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcffc1095b0 0x7fcffc1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.741+0000 7fd000c00640 1 -- 172.21.15.69:0/724098686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcffc074220 msgr2=0x7fcffc199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.741+0000 7fd000c00640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcffc074220 0x7fcffc199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.741+0000 7fd000c00640 1 -- 172.21.15.69:0/724098686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc1a1840 con 0x7fcffc075800 2024-03-19T01:09:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.741+0000 7fd000c00640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcffc075800 0x7fcffc19a170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e9f0 tx=0x7fcfe800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.741+0000 7fcffa000640 1 -- 172.21.15.69:0/724098686 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800cd80 con 0x7fcffc075800 2024-03-19T01:09:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.741+0000 7fcffa000640 1 -- 172.21.15.69:0/724098686 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe8010910 con 0x7fcffc075800 2024-03-19T01:09:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.741+0000 7fcffa000640 1 -- 172.21.15.69:0/724098686 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8018990 con 0x7fcffc075800 2024-03-19T01:09:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.741+0000 7fd002a00640 1 -- 172.21.15.69:0/724098686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc1a1b30 con 0x7fcffc075800 2024-03-19T01:09:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.741+0000 7fd002a00640 1 -- 172.21.15.69:0/724098686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcffc1a2050 con 0x7fcffc075800 2024-03-19T01:09:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.742+0000 7fd002a00640 1 -- 172.21.15.69:0/724098686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcffc108370 con 0x7fcffc075800 2024-03-19T01:09:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.745+0000 7fcffa000640 1 -- 172.21.15.69:0/724098686 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fcfe8010ac0 con 0x7fcffc075800 2024-03-19T01:09:29.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.747+0000 7fcffa000640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcfe40a2800 0x7fcfe40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:29.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.748+0000 7fcffa000640 1 -- 172.21.15.69:0/724098686 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fcfe80cf3a0 con 0x7fcffc075800 2024-03-19T01:09:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.748+0000 7fd001600640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcfe40a2800 0x7fcfe40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.748+0000 7fcffa000640 1 -- 172.21.15.69:0/724098686 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcfe806ec40 con 0x7fcffc075800 2024-03-19T01:09:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.749+0000 7fd001600640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcfe40a2800 0x7fcfe40a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcff002c2e0 tx=0x7fcff002c920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:29.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.950+0000 7fd002a00640 1 -- 172.21.15.69:0/724098686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcffc02d440 con 0x7fcffc075800 2024-03-19T01:09:29.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.951+0000 7fcffa000640 1 -- 172.21.15.69:0/724098686 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fcfe809b9e0 con 0x7fcffc075800 2024-03-19T01:09:29.952 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:09:29.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.954+0000 7fd002a00640 1 -- 172.21.15.69:0/724098686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcfe40a2800 msgr2=0x7fcfe40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.954+0000 7fd002a00640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcfe40a2800 0x7fcfe40a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcff002c2e0 tx=0x7fcff002c920 comp rx=0 tx=0).stop 2024-03-19T01:09:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.954+0000 7fd002a00640 1 -- 172.21.15.69:0/724098686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcffc075800 msgr2=0x7fcffc19a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.954+0000 7fd002a00640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcffc075800 0x7fcffc19a170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e9f0 tx=0x7fcfe800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.954+0000 7fd002a00640 1 -- 172.21.15.69:0/724098686 shutdown_connections 2024-03-19T01:09:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.954+0000 7fd002a00640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcfe40a2800 0x7fcfe40a4cc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.954+0000 7fd002a00640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcffc1095b0 0x7fcffc1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.954+0000 7fd002a00640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcffc075800 0x7fcffc19a170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.955+0000 7fd002a00640 1 --2- 172.21.15.69:0/724098686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcffc074220 0x7fcffc199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.955+0000 7fd002a00640 1 -- 172.21.15.69:0/724098686 >> 172.21.15.69:0/724098686 conn(0x7fcffc0fcc90 msgr2=0x7fcffc105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.955+0000 7fd002a00640 1 -- 172.21.15.69:0/724098686 shutdown_connections 2024-03-19T01:09:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:29.955+0000 7fd002a00640 1 -- 172.21.15.69:0/724098686 wait complete. 2024-03-19T01:09:29.974 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:09:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.189+0000 7f9477e00640 1 -- 172.21.15.69:0/2547274522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9478107d40 msgr2=0x7f9478108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.189+0000 7f9477e00640 1 --2- 172.21.15.69:0/2547274522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9478107d40 0x7f9478108160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9468009ca0 tx=0x7f946802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.189+0000 7f9477e00640 1 -- 172.21.15.69:0/2547274522 shutdown_connections 2024-03-19T01:09:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.189+0000 7f9477e00640 1 --2- 172.21.15.69:0/2547274522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94781098f0 0x7f9478110180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.189+0000 7f9477e00640 1 --2- 172.21.15.69:0/2547274522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9478108f30 0x7f94781093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.189+0000 7f9477e00640 1 --2- 172.21.15.69:0/2547274522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9478107d40 0x7f9478108160 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.189+0000 7f9477e00640 1 -- 172.21.15.69:0/2547274522 >> 172.21.15.69:0/2547274522 conn(0x7f9478076cf0 msgr2=0x7f9478079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.189+0000 7f9477e00640 1 -- 172.21.15.69:0/2547274522 shutdown_connections 2024-03-19T01:09:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.189+0000 7f9477e00640 1 -- 172.21.15.69:0/2547274522 wait complete. 2024-03-19T01:09:30.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.191+0000 7f9477e00640 1 Processor -- start 2024-03-19T01:09:30.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.191+0000 7f9477e00640 1 -- start start 2024-03-19T01:09:30.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.191+0000 7f9477e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9478107d40 0x7f947819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:30.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.191+0000 7f9477e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9478108f30 0x7f947819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:30.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.191+0000 7f9477e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94781098f0 0x7f94781a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:30.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.191+0000 7f9477e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9478113990 con 0x7f9478108f30 2024-03-19T01:09:30.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9477e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9478113810 con 0x7f9478107d40 2024-03-19T01:09:30.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9477e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9478113b10 con 0x7f94781098f0 2024-03-19T01:09:30.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9476a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9478107d40 0x7f947819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:30.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9476a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9478107d40 0x7f947819e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57144/0 (socket says 172.21.15.69:57144) 2024-03-19T01:09:30.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9476a00640 1 -- 172.21.15.69:0/3395020086 learned_addr learned my addr 172.21.15.69:0/3395020086 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:30.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9476000640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9478108f30 0x7f947819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:30.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9477400640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94781098f0 0x7f94781a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9476a00640 1 -- 172.21.15.69:0/3395020086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94781098f0 msgr2=0x7f94781a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9476a00640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94781098f0 0x7f94781a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9476a00640 1 -- 172.21.15.69:0/3395020086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9478108f30 msgr2=0x7f947819e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9476a00640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9478108f30 0x7f947819e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9476a00640 1 -- 172.21.15.69:0/3395020086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94781a5c20 con 0x7f9478107d40 2024-03-19T01:09:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.192+0000 7f9476a00640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9478107d40 0x7f947819e000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9468000f80 tx=0x7f9468032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.193+0000 7f946f400640 1 -- 172.21.15.69:0/3395020086 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468041020 con 0x7f9478107d40 2024-03-19T01:09:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.193+0000 7f946f400640 1 -- 172.21.15.69:0/3395020086 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9468034cd0 con 0x7f9478107d40 2024-03-19T01:09:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.193+0000 7f9477e00640 1 -- 172.21.15.69:0/3395020086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94781a5eb0 con 0x7f9478107d40 2024-03-19T01:09:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.193+0000 7f946f400640 1 -- 172.21.15.69:0/3395020086 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468039040 con 0x7f9478107d40 2024-03-19T01:09:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.193+0000 7f9477e00640 1 -- 172.21.15.69:0/3395020086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94781a6310 con 0x7f9478107d40 2024-03-19T01:09:30.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.194+0000 7f9477e00640 1 -- 172.21.15.69:0/3395020086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f947810f060 con 0x7f9478107d40 2024-03-19T01:09:30.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.196+0000 7f946f400640 1 -- 172.21.15.69:0/3395020086 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9468044020 con 0x7f9478107d40 2024-03-19T01:09:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.197+0000 7f946f400640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f945c0a2880 0x7f945c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.197+0000 7f946f400640 1 -- 172.21.15.69:0/3395020086 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f94680f0b90 con 0x7f9478107d40 2024-03-19T01:09:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.197+0000 7f9476000640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f945c0a2880 0x7f945c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:30.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.198+0000 7f9476000640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f945c0a2880 0x7f945c0a4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f947819f510 tx=0x7f9464005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:30.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.201+0000 7f946f400640 1 -- 172.21.15.69:0/3395020086 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f946804b020 con 0x7f9478107d40 2024-03-19T01:09:30.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:29 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1571923781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:09:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:29 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2020205312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:09:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:29 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1571923781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:09:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:29 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2020205312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:09:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:29 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1571923781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:09:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:29 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2020205312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:09:30.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.399+0000 7f9477e00640 1 -- 172.21.15.69:0/3395020086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9478062420 con 0x7f9478107d40 2024-03-19T01:09:30.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.400+0000 7f946f400640 1 -- 172.21.15.69:0/3395020086 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f94680348f0 con 0x7f9478107d40 2024-03-19T01:09:30.401 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.402+0000 7f9477e00640 1 -- 172.21.15.69:0/3395020086 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f945c0a2880 msgr2=0x7f945c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.402+0000 7f9477e00640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f945c0a2880 0x7f945c0a4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f947819f510 tx=0x7f9464005f90 comp rx=0 tx=0).stop 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.402+0000 7f9477e00640 1 -- 172.21.15.69:0/3395020086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9478107d40 msgr2=0x7f947819e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.402+0000 7f9477e00640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9478107d40 0x7f947819e000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9468000f80 tx=0x7f9468032ec0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.402+0000 7f9477e00640 1 -- 172.21.15.69:0/3395020086 shutdown_connections 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.402+0000 7f9477e00640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f945c0a2880 0x7f945c0a4d40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.402+0000 7f9477e00640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94781098f0 0x7f94781a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.402+0000 7f9477e00640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9478108f30 0x7f947819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.402+0000 7f9477e00640 1 --2- 172.21.15.69:0/3395020086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9478107d40 0x7f947819e000 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.403+0000 7f9477e00640 1 -- 172.21.15.69:0/3395020086 >> 172.21.15.69:0/3395020086 conn(0x7f9478076cf0 msgr2=0x7f947810c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.403+0000 7f9477e00640 1 -- 172.21.15.69:0/3395020086 shutdown_connections 2024-03-19T01:09:30.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.403+0000 7f9477e00640 1 -- 172.21.15.69:0/3395020086 wait complete. 2024-03-19T01:09:30.421 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T01:09:30.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.634+0000 7f0cdca00640 1 -- 172.21.15.69:0/3782870052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd8102340 msgr2=0x7f0cd8104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:30.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.634+0000 7f0cdca00640 1 --2- 172.21.15.69:0/3782870052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd8102340 0x7f0cd8104730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009900 tx=0x7f0cc802bc50 comp rx=0 tx=0).stop 2024-03-19T01:09:30.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.634+0000 7f0cdca00640 1 -- 172.21.15.69:0/3782870052 shutdown_connections 2024-03-19T01:09:30.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.634+0000 7f0cdca00640 1 --2- 172.21.15.69:0/3782870052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cd81075a0 0x7f0cd8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.635+0000 7f0cdca00640 1 --2- 172.21.15.69:0/3782870052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cd8104c70 0x7f0cd8107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.635+0000 7f0cdca00640 1 --2- 172.21.15.69:0/3782870052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd8102340 0x7f0cd8104730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.635+0000 7f0cdca00640 1 -- 172.21.15.69:0/3782870052 >> 172.21.15.69:0/3782870052 conn(0x7f0cd80fbcd0 msgr2=0x7f0cd80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:30.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.635+0000 7f0cdca00640 1 -- 172.21.15.69:0/3782870052 shutdown_connections 2024-03-19T01:09:30.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.635+0000 7f0cdca00640 1 -- 172.21.15.69:0/3782870052 wait complete. 2024-03-19T01:09:30.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.635+0000 7f0cdca00640 1 Processor -- start 2024-03-19T01:09:30.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cdca00640 1 -- start start 2024-03-19T01:09:30.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cdca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd8102340 0x7f0cd8101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:30.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cdca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cd8104c70 0x7f0cd81002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cdca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cd81075a0 0x7f0cd8100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cdca00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810d4e0 con 0x7f0cd81075a0 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cdca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810d360 con 0x7f0cd8104c70 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cdca00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810d660 con 0x7f0cd8102340 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cd5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cd8104c70 0x7f0cd81002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cd5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd8102340 0x7f0cd8101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cd5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cd8104c70 0x7f0cd81002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57162/0 (socket says 172.21.15.69:57162) 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cd5400640 1 -- 172.21.15.69:0/2473350455 learned_addr learned my addr 172.21.15.69:0/2473350455 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.636+0000 7f0cd6800640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cd81075a0 0x7f0cd8100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.637+0000 7f0cd5400640 1 -- 172.21.15.69:0/2473350455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd8102340 msgr2=0x7f0cd8101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.637+0000 7f0cd5400640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd8102340 0x7f0cd8101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.637+0000 7f0cd5400640 1 -- 172.21.15.69:0/2473350455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cd81075a0 msgr2=0x7f0cd8100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.637+0000 7f0cd5400640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cd81075a0 0x7f0cd8100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.637+0000 7f0cd5400640 1 -- 172.21.15.69:0/2473350455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd81010c0 con 0x7f0cd8104c70 2024-03-19T01:09:30.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.637+0000 7f0cd5400640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cd8104c70 0x7f0cd81002f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400e9f0 tx=0x7f0cc400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:30.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.637+0000 7f0ccea00640 1 -- 172.21.15.69:0/2473350455 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc400ccc0 con 0x7f0cd8104c70 2024-03-19T01:09:30.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.637+0000 7f0ccea00640 1 -- 172.21.15.69:0/2473350455 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc4005500 con 0x7f0cd8104c70 2024-03-19T01:09:30.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.637+0000 7f0ccea00640 1 -- 172.21.15.69:0/2473350455 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4010690 con 0x7f0cd8104c70 2024-03-19T01:09:30.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.637+0000 7f0cdca00640 1 -- 172.21.15.69:0/2473350455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd81a1d50 con 0x7f0cd8104c70 2024-03-19T01:09:30.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.637+0000 7f0cdca00640 1 -- 172.21.15.69:0/2473350455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd81a2190 con 0x7f0cd8104c70 2024-03-19T01:09:30.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.639+0000 7f0cdca00640 1 -- 172.21.15.69:0/2473350455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd80fd7c0 con 0x7f0cd8104c70 2024-03-19T01:09:30.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.640+0000 7f0ccea00640 1 -- 172.21.15.69:0/2473350455 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f0cc40046c0 con 0x7f0cd8104c70 2024-03-19T01:09:30.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.642+0000 7f0ccea00640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0cbc0a2880 0x7f0cbc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:30.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.642+0000 7f0ccea00640 1 -- 172.21.15.69:0/2473350455 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f0cc40cf250 con 0x7f0cd8104c70 2024-03-19T01:09:30.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.642+0000 7f0cd5e00640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0cbc0a2880 0x7f0cbc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:30.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.643+0000 7f0cd5e00640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0cbc0a2880 0x7f0cbc0a4d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0cc802c7d0 tx=0x7f0cc8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:30.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.645+0000 7f0ccea00640 1 -- 172.21.15.69:0/2473350455 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0cc409bee0 con 0x7f0cd8104c70 2024-03-19T01:09:30.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.861+0000 7f0cdca00640 1 -- 172.21.15.69:0/2473350455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0cd8062420 con 0x7f0cd8104c70 2024-03-19T01:09:30.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.862+0000 7f0ccea00640 1 -- 172.21.15.69:0/2473350455 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f0cc409b7e0 con 0x7f0cd8104c70 2024-03-19T01:09:30.863 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:09:30.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.865+0000 7f0cdca00640 1 -- 172.21.15.69:0/2473350455 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0cbc0a2880 msgr2=0x7f0cbc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:30.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.865+0000 7f0cdca00640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0cbc0a2880 0x7f0cbc0a4d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0cc802c7d0 tx=0x7f0cc8005fd0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.865+0000 7f0cdca00640 1 -- 172.21.15.69:0/2473350455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cd8104c70 msgr2=0x7f0cd81002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:30.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.865+0000 7f0cdca00640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cd8104c70 0x7f0cd81002f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400e9f0 tx=0x7f0cc400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.865+0000 7f0cdca00640 1 -- 172.21.15.69:0/2473350455 shutdown_connections 2024-03-19T01:09:30.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.865+0000 7f0cdca00640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0cbc0a2880 0x7f0cbc0a4d40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.865+0000 7f0cdca00640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0cd81075a0 0x7f0cd8100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.865+0000 7f0cdca00640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cd8104c70 0x7f0cd81002f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.865+0000 7f0cdca00640 1 --2- 172.21.15.69:0/2473350455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cd8102340 0x7f0cd8101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:30.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.865+0000 7f0cdca00640 1 -- 172.21.15.69:0/2473350455 >> 172.21.15.69:0/2473350455 conn(0x7f0cd80fbcd0 msgr2=0x7f0cd8102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:30.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.865+0000 7f0cdca00640 1 -- 172.21.15.69:0/2473350455 shutdown_connections 2024-03-19T01:09:30.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:30.866+0000 7f0cdca00640 1 -- 172.21.15.69:0/2473350455 wait complete. 2024-03-19T01:09:30.884 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:09:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.113+0000 7f8077e00640 1 -- 172.21.15.69:0/692471115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80781008a0 msgr2=0x7f8078100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.113+0000 7f8077e00640 1 --2- 172.21.15.69:0/692471115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80781008a0 0x7f8078100d00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8064009900 tx=0x7f806402bc30 comp rx=0 tx=0).stop 2024-03-19T01:09:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.113+0000 7f8077e00640 1 -- 172.21.15.69:0/692471115 shutdown_connections 2024-03-19T01:09:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.113+0000 7f8077e00640 1 --2- 172.21.15.69:0/692471115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8078101240 0x7f80781016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.113+0000 7f8077e00640 1 --2- 172.21.15.69:0/692471115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80781008a0 0x7f8078100d00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.113+0000 7f8077e00640 1 --2- 172.21.15.69:0/692471115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80780fff40 0x7f8078100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.113+0000 7f8077e00640 1 -- 172.21.15.69:0/692471115 >> 172.21.15.69:0/692471115 conn(0x7f80780fbb00 msgr2=0x7f80780fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.113+0000 7f8077e00640 1 -- 172.21.15.69:0/692471115 shutdown_connections 2024-03-19T01:09:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.113+0000 7f8077e00640 1 -- 172.21.15.69:0/692471115 wait complete. 2024-03-19T01:09:31.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.114+0000 7f8077e00640 1 Processor -- start 2024-03-19T01:09:31.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.114+0000 7f8077e00640 1 -- start start 2024-03-19T01:09:31.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8077e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80780fff40 0x7f8078199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:31.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8077e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80781008a0 0x7f807819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:31.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8077e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8078101240 0x7f80781a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:31.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8077e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f807810f500 con 0x7f80780fff40 2024-03-19T01:09:31.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8077e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f807810f380 con 0x7f8078101240 2024-03-19T01:09:31.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8077e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f807810f680 con 0x7f80781008a0 2024-03-19T01:09:31.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8076a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80780fff40 0x7f8078199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8077400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8078101240 0x7f80781a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8076a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80780fff40 0x7f8078199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33360/0 (socket says 172.21.15.69:33360) 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8076a00640 1 -- 172.21.15.69:0/3111957786 learned_addr learned my addr 172.21.15.69:0/3111957786 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8076000640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80781008a0 0x7f807819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8076a00640 1 -- 172.21.15.69:0/3111957786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80781008a0 msgr2=0x7f807819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8076a00640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80781008a0 0x7f807819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8076a00640 1 -- 172.21.15.69:0/3111957786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8078101240 msgr2=0x7f80781a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8076a00640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8078101240 0x7f80781a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8076a00640 1 -- 172.21.15.69:0/3111957786 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80781a1760 con 0x7f80780fff40 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.115+0000 7f8077400640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8078101240 0x7f80781a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.116+0000 7f8076a00640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80780fff40 0x7f8078199b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f806800e9f0 tx=0x7f806800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.116+0000 7f807ca00640 1 -- 172.21.15.69:0/3111957786 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806800ccc0 con 0x7f80780fff40 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.116+0000 7f807ca00640 1 -- 172.21.15.69:0/3111957786 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8068010910 con 0x7f80780fff40 2024-03-19T01:09:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.116+0000 7f8077e00640 1 -- 172.21.15.69:0/3111957786 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80781a1a50 con 0x7f80780fff40 2024-03-19T01:09:31.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.116+0000 7f807ca00640 1 -- 172.21.15.69:0/3111957786 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80680188a0 con 0x7f80780fff40 2024-03-19T01:09:31.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.116+0000 7f8077e00640 1 -- 172.21.15.69:0/3111957786 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80781a1fc0 con 0x7f80780fff40 2024-03-19T01:09:31.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.118+0000 7f8077e00640 1 -- 172.21.15.69:0/3111957786 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f807810ac90 con 0x7f80780fff40 2024-03-19T01:09:31.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.119+0000 7f807ca00640 1 -- 172.21.15.69:0/3111957786 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f8068010ac0 con 0x7f80780fff40 2024-03-19T01:09:31.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.121+0000 7f807ca00640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f805c0a2880 0x7f805c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:31.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.122+0000 7f807ca00640 1 -- 172.21.15.69:0/3111957786 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f80680cf1e0 con 0x7f80780fff40 2024-03-19T01:09:31.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.122+0000 7f8076000640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f805c0a2880 0x7f805c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:31.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.122+0000 7f8076000640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f805c0a2880 0x7f805c0a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f806402c240 tx=0x7f806402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:31.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.127+0000 7f807ca00640 1 -- 172.21.15.69:0/3111957786 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f806809be70 con 0x7f80780fff40 2024-03-19T01:09:31.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:30 smithi086 ceph-mon[66829]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:31.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:30 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/724098686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:09:31.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:30 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3395020086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:09:31.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:30 smithi069 ceph-mon[31727]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:31.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:30 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/724098686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:09:31.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:30 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3395020086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:09:31.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:30 smithi163 ceph-mon[67428]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:31.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:30 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/724098686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:09:31.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:30 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3395020086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:09:31.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.457+0000 7f8077e00640 1 -- 172.21.15.69:0/3111957786 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8078062450 con 0x7f80780fff40 2024-03-19T01:09:31.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.459+0000 7f807ca00640 1 -- 172.21.15.69:0/3111957786 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7f806809b770 con 0x7f80780fff40 2024-03-19T01:09:31.460 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:31.460 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:07:50.808640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581,24476,24580,24494,24550]},"id":1}]} 2024-03-19T01:09:31.460 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-03-19T01:09:31.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.461+0000 7f8077e00640 1 -- 172.21.15.69:0/3111957786 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f805c0a2880 msgr2=0x7f805c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:31.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.461+0000 7f8077e00640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f805c0a2880 0x7f805c0a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f806402c240 tx=0x7f806402df90 comp rx=0 tx=0).stop 2024-03-19T01:09:31.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.461+0000 7f8077e00640 1 -- 172.21.15.69:0/3111957786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80780fff40 msgr2=0x7f8078199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:31.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.461+0000 7f8077e00640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80780fff40 0x7f8078199b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f806800e9f0 tx=0x7f806800edf0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.461+0000 7f8077e00640 1 -- 172.21.15.69:0/3111957786 shutdown_connections 2024-03-19T01:09:31.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.461+0000 7f8077e00640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f805c0a2880 0x7f805c0a4d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.461+0000 7f8077e00640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8078101240 0x7f80781a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.461+0000 7f8077e00640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80781008a0 0x7f807819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.461+0000 7f8077e00640 1 --2- 172.21.15.69:0/3111957786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80780fff40 0x7f8078199b40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.461+0000 7f8077e00640 1 -- 172.21.15.69:0/3111957786 >> 172.21.15.69:0/3111957786 conn(0x7f80780fbb00 msgr2=0x7f80780fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:31.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.461+0000 7f8077e00640 1 -- 172.21.15.69:0/3111957786 shutdown_connections 2024-03-19T01:09:31.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.462+0000 7f8077e00640 1 -- 172.21.15.69:0/3111957786 wait complete. 2024-03-19T01:09:31.482 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:09:31.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.671+0000 7f200f200640 1 -- 172.21.15.69:0/2398616610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008101680 msgr2=0x7f2008101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:31.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.671+0000 7f200f200640 1 --2- 172.21.15.69:0/2398616610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008101680 0x7f2008101aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009900 tx=0x7f1ffc02bc30 comp rx=0 tx=0).stop 2024-03-19T01:09:31.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.671+0000 7f200f200640 1 -- 172.21.15.69:0/2398616610 shutdown_connections 2024-03-19T01:09:31.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.671+0000 7f200f200640 1 --2- 172.21.15.69:0/2398616610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008103230 0x7f2008109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.671+0000 7f200f200640 1 --2- 172.21.15.69:0/2398616610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2008102870 0x7f2008102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.671+0000 7f200f200640 1 --2- 172.21.15.69:0/2398616610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008101680 0x7f2008101aa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.671+0000 7f200f200640 1 -- 172.21.15.69:0/2398616610 >> 172.21.15.69:0/2398616610 conn(0x7f20080fcbf0 msgr2=0x7f20080ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:31.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.672+0000 7f200f200640 1 -- 172.21.15.69:0/2398616610 shutdown_connections 2024-03-19T01:09:31.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.672+0000 7f200f200640 1 -- 172.21.15.69:0/2398616610 wait complete. 2024-03-19T01:09:31.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.673+0000 7f200f200640 1 Processor -- start 2024-03-19T01:09:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.673+0000 7f200f200640 1 -- start start 2024-03-19T01:09:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008101680 0x7f2008197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200f200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2008102870 0x7f2008197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008103230 0x7f200819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200f200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f200810d370 con 0x7f2008102870 2024-03-19T01:09:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f200810d1f0 con 0x7f2008103230 2024-03-19T01:09:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f200810d4f0 con 0x7f2008101680 2024-03-19T01:09:31.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008101680 0x7f2008197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:31.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2008102870 0x7f2008197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:31.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008101680 0x7f2008197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46064/0 (socket says 172.21.15.69:46064) 2024-03-19T01:09:31.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200de00640 1 -- 172.21.15.69:0/3983814181 learned_addr learned my addr 172.21.15.69:0/3983814181 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:31.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200e800640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008103230 0x7f200819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:31.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200de00640 1 -- 172.21.15.69:0/3983814181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008103230 msgr2=0x7f200819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:31.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200de00640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008103230 0x7f200819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200de00640 1 -- 172.21.15.69:0/3983814181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2008102870 msgr2=0x7f2008197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:31.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200de00640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2008102870 0x7f2008197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.674+0000 7f200de00640 1 -- 172.21.15.69:0/3983814181 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200819f580 con 0x7f2008101680 2024-03-19T01:09:31.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.675+0000 7f200de00640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008101680 0x7f2008197960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc000c00 tx=0x7f1ffc02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:31.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.675+0000 7f2002a00640 1 -- 172.21.15.69:0/3983814181 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc043020 con 0x7f2008101680 2024-03-19T01:09:31.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.675+0000 7f2002a00640 1 -- 172.21.15.69:0/3983814181 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ffc0054e0 con 0x7f2008101680 2024-03-19T01:09:31.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.675+0000 7f2002a00640 1 -- 172.21.15.69:0/3983814181 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc003c40 con 0x7f2008101680 2024-03-19T01:09:31.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.675+0000 7f200f200640 1 -- 172.21.15.69:0/3983814181 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200819f810 con 0x7f2008101680 2024-03-19T01:09:31.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.675+0000 7f200f200640 1 -- 172.21.15.69:0/3983814181 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200819fd60 con 0x7f2008101680 2024-03-19T01:09:31.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.677+0000 7f200f200640 1 -- 172.21.15.69:0/3983814181 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2008108940 con 0x7f2008101680 2024-03-19T01:09:31.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.679+0000 7f2002a00640 1 -- 172.21.15.69:0/3983814181 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f1ffc046020 con 0x7f2008101680 2024-03-19T01:09:31.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.681+0000 7f2002a00640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1ff00a2800 0x7f1ff00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:31.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.681+0000 7f2002a00640 1 -- 172.21.15.69:0/3983814181 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f1ffc0f0dd0 con 0x7f2008101680 2024-03-19T01:09:31.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.681+0000 7f200d400640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1ff00a2800 0x7f1ff00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:31.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.682+0000 7f200d400640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1ff00a2800 0x7f1ff00a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2008198e70 tx=0x7f1ff8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:31.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.685+0000 7f2002a00640 1 -- 172.21.15.69:0/3983814181 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1ffc04d020 con 0x7f2008101680 2024-03-19T01:09:31.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.946+0000 7f200f200640 1 -- 172.21.15.69:0/3983814181 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2008062420 con 0x7f2008101680 2024-03-19T01:09:31.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.947+0000 7f2002a00640 1 -- 172.21.15.69:0/3983814181 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24736 (secure 0 0 0) 0x7f1ffc0bdac0 con 0x7f2008101680 2024-03-19T01:09:31.949 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:31.949 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:09:25.783660+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:09:31.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 -- 172.21.15.69:0/3983814181 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1ff00a2800 msgr2=0x7f1ff00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:31.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1ff00a2800 0x7f1ff00a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2008198e70 tx=0x7f1ff8005f90 comp rx=0 tx=0).stop 2024-03-19T01:09:31.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 -- 172.21.15.69:0/3983814181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008101680 msgr2=0x7f2008197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:31.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008101680 0x7f2008197960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc000c00 tx=0x7f1ffc02c650 comp rx=0 tx=0).stop 2024-03-19T01:09:31.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 -- 172.21.15.69:0/3983814181 shutdown_connections 2024-03-19T01:09:31.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f1ff00a2800 0x7f1ff00a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2008103230 0x7f200819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2008102870 0x7f2008197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 --2- 172.21.15.69:0/3983814181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008101680 0x7f2008197960 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:31.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 -- 172.21.15.69:0/3983814181 >> 172.21.15.69:0/3983814181 conn(0x7f20080fcbf0 msgr2=0x7f2008105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:31.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 -- 172.21.15.69:0/3983814181 shutdown_connections 2024-03-19T01:09:31.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:31.950+0000 7f200f200640 1 -- 172.21.15.69:0/3983814181 wait complete. 2024-03-19T01:09:31.972 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:09:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.177+0000 7fedcf200640 1 -- 172.21.15.69:0/658539313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedd0101680 msgr2=0x7fedd0101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.177+0000 7fedcf200640 1 --2- 172.21.15.69:0/658539313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedd0101680 0x7fedd0101aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009900 tx=0x7fedc002bc30 comp rx=0 tx=0).stop 2024-03-19T01:09:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.177+0000 7fedcf200640 1 -- 172.21.15.69:0/658539313 shutdown_connections 2024-03-19T01:09:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.177+0000 7fedcf200640 1 --2- 172.21.15.69:0/658539313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedd0103230 0x7fedd0109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.177+0000 7fedcf200640 1 --2- 172.21.15.69:0/658539313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedd0102870 0x7fedd0102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.177+0000 7fedcf200640 1 --2- 172.21.15.69:0/658539313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedd0101680 0x7fedd0101aa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.177+0000 7fedcf200640 1 -- 172.21.15.69:0/658539313 >> 172.21.15.69:0/658539313 conn(0x7fedd00fcbf0 msgr2=0x7fedd00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.177+0000 7fedcf200640 1 -- 172.21.15.69:0/658539313 shutdown_connections 2024-03-19T01:09:32.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.177+0000 7fedcf200640 1 -- 172.21.15.69:0/658539313 wait complete. 2024-03-19T01:09:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.178+0000 7fedcf200640 1 Processor -- start 2024-03-19T01:09:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.178+0000 7fedcf200640 1 -- start start 2024-03-19T01:09:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.178+0000 7fedcf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedd0101680 0x7fedd0197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.178+0000 7fedcf200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedd0102870 0x7fedd0197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcf200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedd0103230 0x7fedd019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcf200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010d350 con 0x7fedd0102870 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010d1d0 con 0x7fedd0101680 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcf200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010d4d0 con 0x7fedd0103230 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedd0101680 0x7fedd0197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedce800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedd0103230 0x7fedd019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedd0101680 0x7fedd0197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57224/0 (socket says 172.21.15.69:57224) 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcde00640 1 -- 172.21.15.69:0/1263458553 learned_addr learned my addr 172.21.15.69:0/1263458553 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcd400640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedd0102870 0x7fedd0197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcde00640 1 -- 172.21.15.69:0/1263458553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedd0103230 msgr2=0x7fedd019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcde00640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedd0103230 0x7fedd019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcde00640 1 -- 172.21.15.69:0/1263458553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedd0102870 msgr2=0x7fedd0197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcde00640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedd0102870 0x7fedd0197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedcde00640 1 -- 172.21.15.69:0/1263458553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd019f620 con 0x7fedd0101680 2024-03-19T01:09:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.179+0000 7fedce800640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedd0103230 0x7fedd019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:32.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.180+0000 7fedcde00640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedd0101680 0x7fedd0197a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005ab0 tx=0x7fedc002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.180+0000 7fedc6a00640 1 -- 172.21.15.69:0/1263458553 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0043020 con 0x7fedd0101680 2024-03-19T01:09:32.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.180+0000 7fedcf200640 1 -- 172.21.15.69:0/1263458553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd019f8b0 con 0x7fedd0101680 2024-03-19T01:09:32.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.180+0000 7fedc6a00640 1 -- 172.21.15.69:0/1263458553 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc00054e0 con 0x7fedd0101680 2024-03-19T01:09:32.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.180+0000 7fedc6a00640 1 -- 172.21.15.69:0/1263458553 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0003ce0 con 0x7fedd0101680 2024-03-19T01:09:32.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.180+0000 7fedcf200640 1 -- 172.21.15.69:0/1263458553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd019fe00 con 0x7fedd0101680 2024-03-19T01:09:32.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.182+0000 7fedcf200640 1 -- 172.21.15.69:0/1263458553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd0108940 con 0x7fedd0101680 2024-03-19T01:09:32.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.183+0000 7fedc6a00640 1 -- 172.21.15.69:0/1263458553 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fedc0046020 con 0x7fedd0101680 2024-03-19T01:09:32.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.184+0000 7fedc6a00640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fedb40a2800 0x7fedb40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.184+0000 7fedc6a00640 1 -- 172.21.15.69:0/1263458553 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fedc00f0f40 con 0x7fedd0101680 2024-03-19T01:09:32.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.185+0000 7fedcd400640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fedb40a2800 0x7fedb40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.185+0000 7fedcd400640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fedb40a2800 0x7fedb40a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fedd0198f10 tx=0x7fedbc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.188+0000 7fedc6a00640 1 -- 172.21.15.69:0/1263458553 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fedc004d020 con 0x7fedd0101680 2024-03-19T01:09:32.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:31 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2473350455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:09:32.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:31 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3111957786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:09:32.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:31 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2473350455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:09:32.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:31 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3111957786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:09:32.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:31 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2473350455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:09:32.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:31 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3111957786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:09:32.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.456+0000 7fedcf200640 1 -- 172.21.15.69:0/1263458553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedd002d310 con 0x7fedd0101680 2024-03-19T01:09:32.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.457+0000 7fedc6a00640 1 -- 172.21.15.69:0/1263458553 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7fedc00bdbd0 con 0x7fedd0101680 2024-03-19T01:09:32.458 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:32.458 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:07:50.808640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581,24476,24580,24494,24550]},"id":1}]} 2024-03-19T01:09:32.458 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-03-19T01:09:32.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.459+0000 7fedcf200640 1 -- 172.21.15.69:0/1263458553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fedb40a2800 msgr2=0x7fedb40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.459+0000 7fedcf200640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fedb40a2800 0x7fedb40a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fedd0198f10 tx=0x7fedbc00b410 comp rx=0 tx=0).stop 2024-03-19T01:09:32.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.460+0000 7fedcf200640 1 -- 172.21.15.69:0/1263458553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedd0101680 msgr2=0x7fedd0197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.460+0000 7fedcf200640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedd0101680 0x7fedd0197a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005ab0 tx=0x7fedc002c6c0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.460+0000 7fedcf200640 1 -- 172.21.15.69:0/1263458553 shutdown_connections 2024-03-19T01:09:32.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.460+0000 7fedcf200640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fedb40a2800 0x7fedb40a4cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.460+0000 7fedcf200640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedd0103230 0x7fedd019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.460+0000 7fedcf200640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedd0102870 0x7fedd0197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.460+0000 7fedcf200640 1 --2- 172.21.15.69:0/1263458553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedd0101680 0x7fedd0197a00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.460+0000 7fedcf200640 1 -- 172.21.15.69:0/1263458553 >> 172.21.15.69:0/1263458553 conn(0x7fedd00fcbf0 msgr2=0x7fedd0105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.460+0000 7fedcf200640 1 -- 172.21.15.69:0/1263458553 shutdown_connections 2024-03-19T01:09:32.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.460+0000 7fedcf200640 1 -- 172.21.15.69:0/1263458553 wait complete. 2024-03-19T01:09:32.479 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-19T01:07:14.928258+0000', 'modified': '2024-03-19T01:07:50.808640+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24581, 'mds_1': 24580, 'mds_2': 24550, 'mds_3': 24494, 'mds_4': 24476}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6833/1468679776', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6832', 'nonce': 1468679776}, {'type': 'v1', 'addr': '172.21.15.163:6833', 'nonce': 1468679776}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6833/3047552839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6832', 'nonce': 3047552839}, {'type': 'v1', 'addr': '172.21.15.86:6833', 'nonce': 3047552839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'd', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.69:6837/2154698833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6836', 'nonce': 2154698833}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 2154698833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.86:6835/786469397', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 786469397}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 786469397}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6835/2175201817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 2175201817}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 2175201817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/1996224781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 1996224781}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 1996224781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6837/1677890136', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 1677890136}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 1677890136}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.163:6839/2045902254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6838', 'nonce': 2045902254}, {'type': 'v1', 'addr': '172.21.15.163:6839', 'nonce': 2045902254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24580': {'gid': 24580, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.69:6841/4121471703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6840', 'nonce': 4121471703}, {'type': 'v1', 'addr': '172.21.15.69:6841', 'nonce': 4121471703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6839/621579864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 621579864}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 621579864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24581, 24476, 24580, 24494, 24550]} 2024-03-19T01:09:32.479 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-19T01:09:32.479 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-19T01:09:32.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.687+0000 7f1232c00640 1 -- 172.21.15.69:0/2923829121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f122c102820 msgr2=0x7f122c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.687+0000 7f1232c00640 1 --2- 172.21.15.69:0/2923829121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f122c102820 0x7f122c102ca0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1218009900 tx=0x7f121802bc30 comp rx=0 tx=0).stop 2024-03-19T01:09:32.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.687+0000 7f1232c00640 1 -- 172.21.15.69:0/2923829121 shutdown_connections 2024-03-19T01:09:32.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.687+0000 7f1232c00640 1 --2- 172.21.15.69:0/2923829121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f122c1031e0 0x7f122c109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.687+0000 7f1232c00640 1 --2- 172.21.15.69:0/2923829121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f122c102820 0x7f122c102ca0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.687+0000 7f1232c00640 1 --2- 172.21.15.69:0/2923829121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c101630 0x7f122c101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.687+0000 7f1232c00640 1 -- 172.21.15.69:0/2923829121 >> 172.21.15.69:0/2923829121 conn(0x7f122c0fcba0 msgr2=0x7f122c0fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.687+0000 7f1232c00640 1 -- 172.21.15.69:0/2923829121 shutdown_connections 2024-03-19T01:09:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.687+0000 7f1232c00640 1 -- 172.21.15.69:0/2923829121 wait complete. 2024-03-19T01:09:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.688+0000 7f1232c00640 1 Processor -- start 2024-03-19T01:09:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.688+0000 7f1232c00640 1 -- start start 2024-03-19T01:09:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.688+0000 7f1232c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c101630 0x7f122c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.688+0000 7f1232c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f122c102820 0x7f122c19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.688+0000 7f1232c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f122c1031e0 0x7f122c1a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.688+0000 7f1232c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f122c10d300 con 0x7f122c102820 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.688+0000 7f1232c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f122c10d180 con 0x7f122c101630 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.688+0000 7f1232c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f122c10d480 con 0x7f122c1031e0 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1232200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f122c1031e0 0x7f122c1a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1231800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c101630 0x7f122c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1232200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f122c1031e0 0x7f122c1a1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46098/0 (socket says 172.21.15.69:46098) 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1232200640 1 -- 172.21.15.69:0/1036791925 learned_addr learned my addr 172.21.15.69:0/1036791925 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1230e00640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f122c102820 0x7f122c19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1232200640 1 -- 172.21.15.69:0/1036791925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c101630 msgr2=0x7f122c199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1232200640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c101630 0x7f122c199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1232200640 1 -- 172.21.15.69:0/1036791925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f122c102820 msgr2=0x7f122c19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1232200640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f122c102820 0x7f122c19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1232200640 1 -- 172.21.15.69:0/1036791925 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c1a1780 con 0x7f122c1031e0 2024-03-19T01:09:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1231800640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c101630 0x7f122c199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.689+0000 7f1232200640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f122c1031e0 0x7f122c1a1110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f122400eb10 tx=0x7f122400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.690+0000 7f122a000640 1 -- 172.21.15.69:0/1036791925 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224004b20 con 0x7f122c1031e0 2024-03-19T01:09:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.690+0000 7f122a000640 1 -- 172.21.15.69:0/1036791925 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1224004cc0 con 0x7f122c1031e0 2024-03-19T01:09:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.690+0000 7f122a000640 1 -- 172.21.15.69:0/1036791925 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224010760 con 0x7f122c1031e0 2024-03-19T01:09:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.690+0000 7f1232c00640 1 -- 172.21.15.69:0/1036791925 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c1a1a70 con 0x7f122c1031e0 2024-03-19T01:09:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.690+0000 7f1232c00640 1 -- 172.21.15.69:0/1036791925 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c1a1f40 con 0x7f122c1031e0 2024-03-19T01:09:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.693+0000 7f122a000640 1 -- 172.21.15.69:0/1036791925 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f1224005020 con 0x7f122c1031e0 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.693+0000 7f1236f83740 1 Processor -- start 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.693+0000 7f1236f83740 1 -- start start 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1236f83740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922e5c0 0x556fd922e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1236f83740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9233ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1236f83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd92365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1236f83740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556fd9192d00 con 0x556fd922e5c0 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1236f83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556fd913bb00 con 0x556fd9234120 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1236f83740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556fd918eda0 con 0x556fd922efb0 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1231800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9233ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1231800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9233ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46108/0 (socket says 172.21.15.69:46108) 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1231800640 1 -- 172.21.15.69:0/2359133018 learned_addr learned my addr 172.21.15.69:0/2359133018 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1230e00640 1 --2- 172.21.15.69:0/2359133018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922e5c0 0x556fd922e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1230e00640 1 --2- 172.21.15.69:0/2359133018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd92365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1231800640 1 -- 172.21.15.69:0/2359133018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 msgr2=0x556fd92365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1231800640 1 --2- 172.21.15.69:0/2359133018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd92365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1231800640 1 -- 172.21.15.69:0/2359133018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922e5c0 msgr2=0x556fd922e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1231800640 1 --2- 172.21.15.69:0/2359133018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922e5c0 0x556fd922e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1231800640 1 -- 172.21.15.69:0/2359133018 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fd8ea72d0 con 0x556fd922efb0 2024-03-19T01:09:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.694+0000 7f1230e00640 1 --2- 172.21.15.69:0/2359133018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922e5c0 0x556fd922e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.695+0000 7f1231800640 1 --2- 172.21.15.69:0/2359133018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9233ab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f12200114f0 tx=0x7f12200119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=769c02c39e8498eb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.695+0000 7f121ea00640 1 -- 172.21.15.69:0/2359133018 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122001e070 con 0x556fd922efb0 2024-03-19T01:09:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.695+0000 7f121ea00640 1 -- 172.21.15.69:0/2359133018 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12200035f0 con 0x556fd922efb0 2024-03-19T01:09:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.695+0000 7f121ea00640 1 -- 172.21.15.69:0/2359133018 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122000f800 con 0x556fd922efb0 2024-03-19T01:09:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.695+0000 7f1236f83740 1 -- 172.21.15.69:0/2359133018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 msgr2=0x556fd9233ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.695+0000 7f1236f83740 1 --2- 172.21.15.69:0/2359133018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9233ab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f12200114f0 tx=0x7f12200119b0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.695+0000 7f122a000640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f12140a2880 0x7f12140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.696+0000 7f1236f83740 1 -- 172.21.15.69:0/2359133018 shutdown_connections 2024-03-19T01:09:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.696+0000 7f1236f83740 1 --2- 172.21.15.69:0/2359133018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd92365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.696+0000 7f1236f83740 1 --2- 172.21.15.69:0/2359133018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9233ab0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.696+0000 7f1236f83740 1 --2- 172.21.15.69:0/2359133018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922e5c0 0x556fd922e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.696+0000 7f1236f83740 1 -- 172.21.15.69:0/2359133018 >> 172.21.15.69:0/2359133018 conn(0x556fd9229b30 msgr2=0x556fd9229f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.696+0000 7f122a000640 1 -- 172.21.15.69:0/1036791925 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f12240cf350 con 0x7f122c1031e0 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.696+0000 7f1236f83740 1 -- 172.21.15.69:0/2359133018 shutdown_connections 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.696+0000 7f1230e00640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f12140a2880 0x7f12140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.696+0000 7f1236f83740 1 -- 172.21.15.69:0/2359133018 wait complete. 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.697+0000 7f1236f83740 1 -- start start 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.697+0000 7f1230e00640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f12140a2880 0x7f12140a4d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f121802c240 tx=0x7f1218005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.697+0000 7f1236f83740 1 Processor -- start 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.697+0000 7f1236f83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd922e5c0 0x556fd9238a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.697+0000 7f1236f83740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9238fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.697+0000 7f1236f83740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd9234120 0x556fd9240010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.697+0000 7f1236f83740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556fd92370d0 con 0x556fd9234120 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.697+0000 7f1236f83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556fd914eed0 con 0x556fd922e5c0 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.697+0000 7f1236f83740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556fd9237250 con 0x556fd922efb0 2024-03-19T01:09:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.697+0000 7f1236f83740 10 client.0 _ll_register_callbacks cb 0x556fd914cc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1230e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9238fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1231800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd922e5c0 0x556fd9238a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1231800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd922e5c0 0x556fd9238a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57262/0 (socket says 172.21.15.69:57262) 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1230e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9238fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46122/0 (socket says 172.21.15.69:46122) 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1231800640 1 -- 172.21.15.69:0/4205465809 learned_addr learned my addr 172.21.15.69:0/4205465809 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1232200640 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd9234120 0x556fd9240010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1231800640 1 -- 172.21.15.69:0/4205465809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 msgr2=0x556fd9238fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1231800640 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9238fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1231800640 1 -- 172.21.15.69:0/4205465809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd9234120 msgr2=0x556fd9240010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1231800640 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd9234120 0x556fd9240010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1231800640 1 -- 172.21.15.69:0/4205465809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fd8f716b0 con 0x556fd922e5c0 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1230e00640 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9238fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1232200640 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd9234120 0x556fd9240010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1231800640 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd922e5c0 0x556fd9238a90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1220011eb0 tx=0x7f1220004230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96db4151cdd03dfd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.698+0000 7f1213400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.699+0000 7f1213400640 1 -- 172.21.15.69:0/4205465809 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122001e070 con 0x556fd922e5c0 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.699+0000 7f1213400640 1 -- 172.21.15.69:0/4205465809 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f1220005250 con 0x556fd922e5c0 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.699+0000 7f1236f83740 1 -- 172.21.15.69:0/4205465809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12200035f0 con 0x556fd922e5c0 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.699+0000 7f1213400640 1 -- 172.21.15.69:0/4205465809 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1220031050 con 0x556fd922e5c0 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.699+0000 7f1236f83740 10 client.25151 fetch_fsmap learned FSMap version 28 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.699+0000 7f1236f83740 1 -- 172.21.15.69:0/4205465809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556fd9243380 con 0x556fd922e5c0 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.699+0000 7f1213400640 1 -- 172.21.15.69:0/4205465809 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12200035f0 con 0x556fd922e5c0 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.699+0000 7f1213400640 1 -- 172.21.15.69:0/4205465809 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f122000f5c0 con 0x556fd922e5c0 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.700+0000 7f1236f83740 10 client.25151 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.700+0000 7f1236f83740 10 client.25151 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.700+0000 7f1236f83740 20 client.25151 populate_metadata read hostname 'smithi069' 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.700+0000 7f1236f83740 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556fd9245800 0x556fd9246c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.700+0000 7f1236f83740 4 client.25151 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:09:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.700+0000 7f1236f83740 1 -- 172.21.15.69:0/4205465809 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fd91a0240 con 0x556fd9245800 2024-03-19T01:09:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.700+0000 7f1230e00640 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556fd9245800 0x556fd9246c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.701+0000 7f1230e00640 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556fd9245800 0x556fd9246c70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3a1973f9d204b3 server_cookie=152dc82b8caa7673 in_seq=0 out_seq=0 2024-03-19T01:09:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.701+0000 7f1213400640 10 client.25151 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:09:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1213400640 1 -- 172.21.15.69:0/4205465809 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556fd91a0240 con 0x556fd9245800 2024-03-19T01:09:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1213400640 10 client.25151 handle_command_reply: tid=0 2024-03-19T01:09:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1236f83740 1 client.25151 shutdown 2024-03-19T01:09:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1236f83740 1 -- 172.21.15.69:0/4205465809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd922e5c0 msgr2=0x556fd9238a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1236f83740 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd922e5c0 0x556fd9238a90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1220011eb0 tx=0x7f1220004230 comp rx=0 tx=0).stop 2024-03-19T01:09:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1236f83740 1 -- 172.21.15.69:0/4205465809 shutdown_connections 2024-03-19T01:09:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1236f83740 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556fd9245800 0x556fd9246c70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1236f83740 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd9234120 0x556fd9240010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1236f83740 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd9238fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1213400640 0 client.25151 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:09:32.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1236f83740 1 --2- 172.21.15.69:0/4205465809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd922e5c0 0x556fd9238a90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1236f83740 1 -- 172.21.15.69:0/4205465809 >> 172.21.15.69:0/4205465809 conn(0x556fd9229b30 msgr2=0x556fd9228d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1236f83740 1 -- 172.21.15.69:0/4205465809 shutdown_connections 2024-03-19T01:09:32.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.702+0000 7f1236f83740 1 -- 172.21.15.69:0/4205465809 wait complete. 2024-03-19T01:09:32.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.703+0000 7f1236f83740 20 client.25151 trim_cache size 0 max 16384 2024-03-19T01:09:32.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.722+0000 7f1236f83740 1 Processor -- start 2024-03-19T01:09:32.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.722+0000 7f1236f83740 1 -- start start 2024-03-19T01:09:32.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.722+0000 7f1236f83740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922e5c0 0x556fd9237620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.722+0000 7f1236f83740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd923ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.722+0000 7f1236f83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd922fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.722+0000 7f1236f83740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556fd9244fb0 con 0x556fd922e5c0 2024-03-19T01:09:32.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.722+0000 7f1236f83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556fd9245680 con 0x556fd9234120 2024-03-19T01:09:32.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.722+0000 7f1236f83740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556fd9245d50 con 0x556fd922efb0 2024-03-19T01:09:32.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1230e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd923ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1232200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd922fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1230e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd923ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46130/0 (socket says 172.21.15.69:46130) 2024-03-19T01:09:32.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1232200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd922fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57264/0 (socket says 172.21.15.69:57264) 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1230e00640 1 -- 172.21.15.69:0/1218095211 learned_addr learned my addr 172.21.15.69:0/1218095211 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1231800640 1 --2- 172.21.15.69:0/1218095211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922e5c0 0x556fd9237620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1230e00640 1 -- 172.21.15.69:0/1218095211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 msgr2=0x556fd922fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1230e00640 1 --2- 172.21.15.69:0/1218095211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd922fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1230e00640 1 -- 172.21.15.69:0/1218095211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922e5c0 msgr2=0x556fd9237620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1230e00640 1 --2- 172.21.15.69:0/1218095211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922e5c0 0x556fd9237620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1230e00640 1 -- 172.21.15.69:0/1218095211 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fd922f390 con 0x556fd922efb0 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1232200640 1 --2- 172.21.15.69:0/1218095211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd922fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.723+0000 7f1230e00640 1 --2- 172.21.15.69:0/1218095211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd923ff30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f121802c7b0 tx=0x7f1218043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5841677c7ade009d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f121e000640 1 -- 172.21.15.69:0/1218095211 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121805b070 con 0x556fd922efb0 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f121e000640 1 -- 172.21.15.69:0/1218095211 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f121804a040 con 0x556fd922efb0 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f121e000640 1 -- 172.21.15.69:0/1218095211 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218005110 con 0x556fd922efb0 2024-03-19T01:09:32.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f1236f83740 1 -- 172.21.15.69:0/1218095211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 msgr2=0x556fd923ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f1236f83740 1 --2- 172.21.15.69:0/1218095211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd923ff30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f121802c7b0 tx=0x7f1218043000 comp rx=0 tx=0).stop 2024-03-19T01:09:32.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f1236f83740 1 -- 172.21.15.69:0/1218095211 shutdown_connections 2024-03-19T01:09:32.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f1236f83740 1 --2- 172.21.15.69:0/1218095211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd922fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f1236f83740 1 --2- 172.21.15.69:0/1218095211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922efb0 0x556fd923ff30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f1236f83740 1 --2- 172.21.15.69:0/1218095211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922e5c0 0x556fd9237620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f1236f83740 1 -- 172.21.15.69:0/1218095211 >> 172.21.15.69:0/1218095211 conn(0x556fd9229b30 msgr2=0x556fd9228d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f1236f83740 1 -- 172.21.15.69:0/1218095211 shutdown_connections 2024-03-19T01:09:32.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.724+0000 7f1236f83740 1 -- 172.21.15.69:0/1218095211 wait complete. 2024-03-19T01:09:32.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.725+0000 7f1236f83740 1 -- start start 2024-03-19T01:09:32.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.725+0000 7f1236f83740 1 Processor -- start 2024-03-19T01:09:32.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1236f83740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922e5c0 0x556fd923e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1236f83740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922efb0 0x556fd923e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1236f83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd923ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1236f83740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556fd9246af0 con 0x556fd922efb0 2024-03-19T01:09:32.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1236f83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556fd9246420 con 0x556fd9234120 2024-03-19T01:09:32.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1236f83740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556fd92471c0 con 0x556fd922e5c0 2024-03-19T01:09:32.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1236f83740 10 client.0 _ll_register_callbacks cb 0x7f122001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:09:32.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1232200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd923ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1231800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922e5c0 0x556fd923e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1232200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd923ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57270/0 (socket says 172.21.15.69:57270) 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1232200640 1 -- 172.21.15.69:0/2170509057 learned_addr learned my addr 172.21.15.69:0/2170509057 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1230e00640 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922efb0 0x556fd923e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1232200640 1 -- 172.21.15.69:0/2170509057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922e5c0 msgr2=0x556fd923e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1232200640 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922e5c0 0x556fd923e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1232200640 1 -- 172.21.15.69:0/2170509057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922efb0 msgr2=0x556fd923e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.726+0000 7f1232200640 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922efb0 0x556fd923e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.727+0000 7f1232200640 1 -- 172.21.15.69:0/2170509057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fd923f560 con 0x556fd9234120 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.727+0000 7f1231800640 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922e5c0 0x556fd923e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.727+0000 7f1232200640 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd923ed30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f122409b870 tx=0x7f122409b4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad24565c24b4c962 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.727+0000 7f1213400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.727+0000 7f1213400640 1 -- 172.21.15.69:0/2170509057 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240b9060 con 0x556fd9234120 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.727+0000 7f1213400640 1 -- 172.21.15.69:0/2170509057 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f122409bb40 con 0x556fd9234120 2024-03-19T01:09:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.727+0000 7f1236f83740 1 -- 172.21.15.69:0/2170509057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f121804a040 con 0x556fd9234120 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.727+0000 7f1213400640 1 -- 172.21.15.69:0/2170509057 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f122409fa30 con 0x556fd9234120 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.727+0000 7f1236f83740 10 client.25157 fetch_fsmap learned FSMap version 28 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.727+0000 7f1236f83740 1 -- 172.21.15.69:0/2170509057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556fd92ab8a0 con 0x556fd9234120 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.728+0000 7f1213400640 1 -- 172.21.15.69:0/2170509057 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f121804a040 con 0x556fd9234120 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.728+0000 7f1213400640 1 -- 172.21.15.69:0/2170509057 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1224005310 con 0x556fd9234120 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.728+0000 7f1236f83740 10 client.25157 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.728+0000 7f1236f83740 10 client.25157 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.728+0000 7f1236f83740 20 client.25157 populate_metadata read hostname 'smithi069' 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.728+0000 7f1236f83740 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556fd92b0410 0x556fd92af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.728+0000 7f1236f83740 4 client.25157 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.728+0000 7f1236f83740 1 -- 172.21.15.69:0/2170509057 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "status"}) -- 0x556fd919a590 con 0x556fd92b0410 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.728+0000 7f1231800640 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556fd92b0410 0x556fd92af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1213400640 10 client.25157 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1231800640 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556fd92b0410 0x556fd92af120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f548a2c7124eb042 server_cookie=ddc0f773d86b577c in_seq=0 out_seq=0 2024-03-19T01:09:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1213400640 1 -- 172.21.15.69:0/2170509057 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556fd919a590 con 0x556fd92b0410 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1213400640 10 client.25157 handle_command_reply: tid=0 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1236f83740 1 client.25157 shutdown 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1236f83740 1 -- 172.21.15.69:0/2170509057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 msgr2=0x556fd923ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1236f83740 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd923ed30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f122409b870 tx=0x7f122409b4e0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1236f83740 1 -- 172.21.15.69:0/2170509057 shutdown_connections 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1236f83740 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556fd92b0410 0x556fd92af120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1236f83740 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fd9234120 0x556fd923ed30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1236f83740 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556fd922efb0 0x556fd923e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1213400640 0 client.25157 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1236f83740 1 --2- 172.21.15.69:0/2170509057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556fd922e5c0 0x556fd923e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1236f83740 1 -- 172.21.15.69:0/2170509057 >> 172.21.15.69:0/2170509057 conn(0x556fd9229b30 msgr2=0x556fd922d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.729+0000 7f1236f83740 1 -- 172.21.15.69:0/2170509057 shutdown_connections 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.730+0000 7f1236f83740 1 -- 172.21.15.69:0/2170509057 wait complete. 2024-03-19T01:09:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.730+0000 7f1236f83740 20 client.25157 trim_cache size 0 max 16384 2024-03-19T01:09:32.735 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:09:32.735 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T01:09:32.735 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 4, 2024-03-19T01:09:32.735 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24476, 2024-03-19T01:09:32.735 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-03-19T01:09:32.735 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-03-19T01:09:32.735 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-03-19T01:09:32.735 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 102.95358140499999, 2024-03-19T01:09:32.735 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-03-19T01:09:32.735 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 93, 2024-03-19T01:09:32.736 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T01:09:32.736 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 169.489166399 2024-03-19T01:09:32.736 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:09:32.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.730+0000 7f1232c00640 1 -- 172.21.15.69:0/1036791925 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f12140a2880 msgr2=0x7f12140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.730+0000 7f1232c00640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f12140a2880 0x7f12140a4d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f121802c240 tx=0x7f1218005fd0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.730+0000 7f1232c00640 1 -- 172.21.15.69:0/1036791925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f122c1031e0 msgr2=0x7f122c1a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.730+0000 7f1232c00640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f122c1031e0 0x7f122c1a1110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f122400eb10 tx=0x7f122400c2d0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.731+0000 7f1232c00640 1 -- 172.21.15.69:0/1036791925 shutdown_connections 2024-03-19T01:09:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.731+0000 7f1232c00640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f12140a2880 0x7f12140a4d40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.731+0000 7f1232c00640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f122c1031e0 0x7f122c1a1110 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.731+0000 7f1232c00640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f122c102820 0x7f122c19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.731+0000 7f1232c00640 1 --2- 172.21.15.69:0/1036791925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f122c101630 0x7f122c199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.731+0000 7f1232c00640 1 -- 172.21.15.69:0/1036791925 >> 172.21.15.69:0/1036791925 conn(0x7f122c0fcba0 msgr2=0x7f122c073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.731+0000 7f1232c00640 1 -- 172.21.15.69:0/1036791925 shutdown_connections 2024-03-19T01:09:32.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.731+0000 7f1232c00640 1 -- 172.21.15.69:0/1036791925 wait complete. 2024-03-19T01:09:32.750 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-19T01:09:32.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.952+0000 7fd973800640 1 -- 172.21.15.69:0/4033341516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c1075b0 msgr2=0x7fd96c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.952+0000 7fd973800640 1 --2- 172.21.15.69:0/4033341516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c1075b0 0x7fd96c1099a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd964009ca0 tx=0x7fd96402c100 comp rx=0 tx=0).stop 2024-03-19T01:09:32.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.952+0000 7fd973800640 1 -- 172.21.15.69:0/4033341516 shutdown_connections 2024-03-19T01:09:32.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.952+0000 7fd973800640 1 --2- 172.21.15.69:0/4033341516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c1075b0 0x7fd96c1099a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.952+0000 7fd973800640 1 --2- 172.21.15.69:0/4033341516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd96c104c80 0x7fd96c107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.952+0000 7fd973800640 1 --2- 172.21.15.69:0/4033341516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd96c102350 0x7fd96c104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.952+0000 7fd973800640 1 -- 172.21.15.69:0/4033341516 >> 172.21.15.69:0/4033341516 conn(0x7fd96c0fbcc0 msgr2=0x7fd96c0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.952+0000 7fd973800640 1 -- 172.21.15.69:0/4033341516 shutdown_connections 2024-03-19T01:09:32.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.952+0000 7fd973800640 1 -- 172.21.15.69:0/4033341516 wait complete. 2024-03-19T01:09:32.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.954+0000 7fd973800640 1 Processor -- start 2024-03-19T01:09:32.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.954+0000 7fd973800640 1 -- start start 2024-03-19T01:09:32.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.954+0000 7fd973800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd96c102350 0x7fd96c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.954+0000 7fd973800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c104c80 0x7fd96c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.954+0000 7fd973800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd96c1075b0 0x7fd96c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.954+0000 7fd973800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c10d4d0 con 0x7fd96c102350 2024-03-19T01:09:32.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.954+0000 7fd973800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c10d350 con 0x7fd96c104c80 2024-03-19T01:09:32.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.954+0000 7fd973800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c10d650 con 0x7fd96c1075b0 2024-03-19T01:09:32.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd970e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd96c102350 0x7fd96c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd96be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c104c80 0x7fd96c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd96be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c104c80 0x7fd96c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57282/0 (socket says 172.21.15.69:57282) 2024-03-19T01:09:32.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd96be00640 1 -- 172.21.15.69:0/2547241667 learned_addr learned my addr 172.21.15.69:0/2547241667 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:32.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd971800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd96c1075b0 0x7fd96c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd96be00640 1 -- 172.21.15.69:0/2547241667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd96c1075b0 msgr2=0x7fd96c1a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd96be00640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd96c1075b0 0x7fd96c1a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd96be00640 1 -- 172.21.15.69:0/2547241667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd96c102350 msgr2=0x7fd96c199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd96be00640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd96c102350 0x7fd96c199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd96be00640 1 -- 172.21.15.69:0/2547241667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c1a1930 con 0x7fd96c104c80 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd971800640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd96c1075b0 0x7fd96c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd970e00640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd96c102350 0x7fd96c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.955+0000 7fd96be00640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c104c80 0x7fd96c19a260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd95800e9f0 tx=0x7fd95800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.956+0000 7fd969600640 1 -- 172.21.15.69:0/2547241667 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95800ccc0 con 0x7fd96c104c80 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.956+0000 7fd973800640 1 -- 172.21.15.69:0/2547241667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c1a1c20 con 0x7fd96c104c80 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.956+0000 7fd969600640 1 -- 172.21.15.69:0/2547241667 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd958005500 con 0x7fd96c104c80 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.956+0000 7fd969600640 1 -- 172.21.15.69:0/2547241667 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958010690 con 0x7fd96c104c80 2024-03-19T01:09:32.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.956+0000 7fd973800640 1 -- 172.21.15.69:0/2547241667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c1a2170 con 0x7fd96c104c80 2024-03-19T01:09:32.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.960+0000 7fd976406740 1 Processor -- start 2024-03-19T01:09:32.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.960+0000 7fd976406740 1 -- start start 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd976406740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6206810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd976406740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb620b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd976406740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb620bf50 0x561fb620e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd976406740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561fb616ad00 con 0x561fb62063f0 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd976406740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fb6113b00 con 0x561fb620bf50 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd976406740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561fb6166da0 con 0x561fb6206de0 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd970e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb620b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd970e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb620b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46172/0 (socket says 172.21.15.69:46172) 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd970e00640 1 -- 172.21.15.69:0/1432203313 learned_addr learned my addr 172.21.15.69:0/1432203313 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd971800640 1 --2- 172.21.15.69:0/1432203313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6206810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd969600640 1 -- 172.21.15.69:0/2547241667 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd9580046c0 con 0x7fd96c104c80 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.961+0000 7fd970e00640 1 -- 172.21.15.69:0/1432203313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb620bf50 msgr2=0x561fb620e3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.962+0000 7fd970e00640 1 --2- 172.21.15.69:0/1432203313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb620bf50 0x561fb620e3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.962+0000 7fd970e00640 1 -- 172.21.15.69:0/1432203313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 msgr2=0x561fb6206810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.962+0000 7fd970e00640 1 --2- 172.21.15.69:0/1432203313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6206810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.962+0000 7fd970e00640 1 -- 172.21.15.69:0/1432203313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fb5e7f2d0 con 0x561fb6206de0 2024-03-19T01:09:32.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.962+0000 7fd971800640 1 --2- 172.21.15.69:0/1432203313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6206810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:32.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.962+0000 7fd970e00640 1 --2- 172.21.15.69:0/1432203313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb620b8e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9600114f0 tx=0x7fd9600119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfe9a25d07a19993 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.962+0000 7fd95e000640 1 -- 172.21.15.69:0/1432203313 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96001e070 con 0x561fb6206de0 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.962+0000 7fd95e000640 1 -- 172.21.15.69:0/1432203313 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9600035f0 con 0x561fb6206de0 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.962+0000 7fd95e000640 1 -- 172.21.15.69:0/1432203313 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96000f820 con 0x561fb6206de0 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd976406740 1 -- 172.21.15.69:0/1432203313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 msgr2=0x561fb620b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd976406740 1 --2- 172.21.15.69:0/1432203313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb620b8e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9600114f0 tx=0x7fd9600119b0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd976406740 1 -- 172.21.15.69:0/1432203313 shutdown_connections 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd976406740 1 --2- 172.21.15.69:0/1432203313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb620bf50 0x561fb620e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd976406740 1 --2- 172.21.15.69:0/1432203313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb620b8e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd976406740 1 --2- 172.21.15.69:0/1432203313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6206810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd976406740 1 -- 172.21.15.69:0/1432203313 >> 172.21.15.69:0/1432203313 conn(0x561fb6201960 msgr2=0x561fb6201d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd969600640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd9540a2880 0x7fd9540a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd976406740 1 -- 172.21.15.69:0/1432203313 shutdown_connections 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd969600640 1 -- 172.21.15.69:0/2547241667 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fd9580cf1a0 con 0x7fd96c104c80 2024-03-19T01:09:32.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd976406740 1 -- 172.21.15.69:0/1432203313 wait complete. 2024-03-19T01:09:32.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.963+0000 7fd970e00640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd9540a2880 0x7fd9540a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.964+0000 7fd976406740 1 -- start start 2024-03-19T01:09:32.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.964+0000 7fd970e00640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd9540a2880 0x7fd9540a4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd960011eb0 tx=0x7fd960000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.964+0000 7fd976406740 1 Processor -- start 2024-03-19T01:09:32.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.965+0000 7fd976406740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6210930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.965+0000 7fd976406740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb6206de0 0x561fb6210e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.965+0000 7fd976406740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb620bf50 0x561fb6217eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.965+0000 7fd976406740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561fb620ef00 con 0x561fb62063f0 2024-03-19T01:09:32.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.965+0000 7fd976406740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fb6126ed0 con 0x561fb6206de0 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.965+0000 7fd976406740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561fb620f080 con 0x561fb620bf50 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.965+0000 7fd976406740 10 client.0 _ll_register_callbacks cb 0x561fb6124c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.965+0000 7fd970e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6210930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.965+0000 7fd970e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6210930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33486/0 (socket says 172.21.15.69:33486) 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.965+0000 7fd970e00640 1 -- 172.21.15.69:0/2651126733 learned_addr learned my addr 172.21.15.69:0/2651126733 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.965+0000 7fd971800640 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb620bf50 0x561fb6217eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.966+0000 7fd96be00640 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb6206de0 0x561fb6210e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.966+0000 7fd970e00640 1 -- 172.21.15.69:0/2651126733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb620bf50 msgr2=0x561fb6217eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.966+0000 7fd970e00640 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb620bf50 0x561fb6217eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.966+0000 7fd970e00640 1 -- 172.21.15.69:0/2651126733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb6206de0 msgr2=0x561fb6210e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.966+0000 7fd970e00640 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb6206de0 0x561fb6210e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.966+0000 7fd970e00640 1 -- 172.21.15.69:0/2651126733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fb5f496b0 con 0x561fb62063f0 2024-03-19T01:09:32.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.966+0000 7fd970e00640 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6210930 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd960003f30 tx=0x7fd9600119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451c45a871d2a9b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.966+0000 7fd952a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.966+0000 7fd952a00640 1 -- 172.21.15.69:0/2651126733 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960032020 con 0x561fb62063f0 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.966+0000 7fd952a00640 1 -- 172.21.15.69:0/2651126733 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fd9600048c0 con 0x561fb62063f0 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.966+0000 7fd976406740 1 -- 172.21.15.69:0/2651126733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9600035f0 con 0x561fb62063f0 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.967+0000 7fd952a00640 1 -- 172.21.15.69:0/2651126733 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd96003d050 con 0x561fb62063f0 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.967+0000 7fd976406740 10 client.15051 fetch_fsmap learned FSMap version 28 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.967+0000 7fd976406740 1 -- 172.21.15.69:0/2651126733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fb621b290 con 0x561fb62063f0 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.967+0000 7fd952a00640 1 -- 172.21.15.69:0/2651126733 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9600035f0 con 0x561fb62063f0 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.967+0000 7fd952a00640 1 -- 172.21.15.69:0/2651126733 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd96002d030 con 0x561fb62063f0 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.967+0000 7fd976406740 10 client.15051 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.967+0000 7fd976406740 10 client.15051 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.967+0000 7fd976406740 20 client.15051 populate_metadata read hostname 'smithi069' 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.967+0000 7fd976406740 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x561fb621d720 0x561fb621eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.967+0000 7fd976406740 4 client.15051 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:09:32.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.967+0000 7fd976406740 1 -- 172.21.15.69:0/2651126733 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fb6178240 con 0x561fb621d720 2024-03-19T01:09:32.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.968+0000 7fd96be00640 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x561fb621d720 0x561fb621eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.968+0000 7fd96be00640 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x561fb621d720 0x561fb621eb90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4d7a2dd1eaff82a server_cookie=90fd6d4b98321df in_seq=0 out_seq=0 2024-03-19T01:09:32.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.968+0000 7fd952a00640 10 client.15051 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd952a00640 1 -- 172.21.15.69:0/2651126733 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561fb6178240 con 0x561fb621d720 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd952a00640 10 client.15051 handle_command_reply: tid=0 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd976406740 1 client.15051 shutdown 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd976406740 1 -- 172.21.15.69:0/2651126733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 msgr2=0x561fb6210930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd976406740 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6210930 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd960003f30 tx=0x7fd9600119b0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd976406740 1 -- 172.21.15.69:0/2651126733 shutdown_connections 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd976406740 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb620bf50 0x561fb6217eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd976406740 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x561fb621d720 0x561fb621eb90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd976406740 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb6206de0 0x561fb6210e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd976406740 1 --2- 172.21.15.69:0/2651126733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6210930 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd976406740 1 -- 172.21.15.69:0/2651126733 >> 172.21.15.69:0/2651126733 conn(0x561fb6201960 msgr2=0x561fb6200aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.969+0000 7fd952a00640 0 client.15051 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:09:32.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.970+0000 7fd976406740 1 -- 172.21.15.69:0/2651126733 shutdown_connections 2024-03-19T01:09:32.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.970+0000 7fd976406740 1 -- 172.21.15.69:0/2651126733 wait complete. 2024-03-19T01:09:32.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.970+0000 7fd976406740 20 client.15051 trim_cache size 0 max 16384 2024-03-19T01:09:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.989+0000 7fd976406740 1 Processor -- start 2024-03-19T01:09:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.989+0000 7fd976406740 1 -- start start 2024-03-19T01:09:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd976406740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb62063f0 0x561fb62077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd976406740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb6206de0 0x561fb6217e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd976406740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb620bf50 0x561fb621c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd976406740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561fb621ced0 con 0x561fb6206de0 2024-03-19T01:09:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd976406740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fb621d5a0 con 0x561fb62063f0 2024-03-19T01:09:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd976406740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561fb621dc70 con 0x561fb620bf50 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd96be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb6206de0 0x561fb6217e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd970e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb62063f0 0x561fb62077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd970e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb62063f0 0x561fb62077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57306/0 (socket says 172.21.15.69:57306) 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd96be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb6206de0 0x561fb6217e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33502/0 (socket says 172.21.15.69:33502) 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd970e00640 1 -- 172.21.15.69:0/1625903228 learned_addr learned my addr 172.21.15.69:0/1625903228 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.990+0000 7fd971800640 1 --2- 172.21.15.69:0/1625903228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb620bf50 0x561fb621c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.991+0000 7fd970e00640 1 -- 172.21.15.69:0/1625903228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb620bf50 msgr2=0x561fb621c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.991+0000 7fd970e00640 1 --2- 172.21.15.69:0/1625903228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb620bf50 0x561fb621c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.991+0000 7fd970e00640 1 -- 172.21.15.69:0/1625903228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb6206de0 msgr2=0x561fb6217e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.991+0000 7fd970e00640 1 --2- 172.21.15.69:0/1625903228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb6206de0 0x561fb6217e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.991+0000 7fd970e00640 1 -- 172.21.15.69:0/1625903228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fb62071c0 con 0x561fb62063f0 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.991+0000 7fd971800640 1 --2- 172.21.15.69:0/1625903228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb620bf50 0x561fb621c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.991+0000 7fd970e00640 1 --2- 172.21.15.69:0/1625903228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb62063f0 0x561fb62077e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd960003fd0 tx=0x7fd960003d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=688a80e197fe5929 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.991+0000 7fd95d600640 1 -- 172.21.15.69:0/1625903228 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960046070 con 0x561fb62063f0 2024-03-19T01:09:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.991+0000 7fd95d600640 1 -- 172.21.15.69:0/1625903228 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd960005d30 con 0x561fb62063f0 2024-03-19T01:09:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.991+0000 7fd95d600640 1 -- 172.21.15.69:0/1625903228 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960032040 con 0x561fb62063f0 2024-03-19T01:09:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.992+0000 7fd976406740 1 -- 172.21.15.69:0/1625903228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb62063f0 msgr2=0x561fb62077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.992+0000 7fd976406740 1 --2- 172.21.15.69:0/1625903228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb62063f0 0x561fb62077e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd960003fd0 tx=0x7fd960003d00 comp rx=0 tx=0).stop 2024-03-19T01:09:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.992+0000 7fd976406740 1 -- 172.21.15.69:0/1625903228 shutdown_connections 2024-03-19T01:09:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.992+0000 7fd976406740 1 --2- 172.21.15.69:0/1625903228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb620bf50 0x561fb621c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.992+0000 7fd976406740 1 --2- 172.21.15.69:0/1625903228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb6206de0 0x561fb6217e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.992+0000 7fd976406740 1 --2- 172.21.15.69:0/1625903228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb62063f0 0x561fb62077e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.992+0000 7fd976406740 1 -- 172.21.15.69:0/1625903228 >> 172.21.15.69:0/1625903228 conn(0x561fb6201960 msgr2=0x561fb6201d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.992+0000 7fd976406740 1 -- 172.21.15.69:0/1625903228 shutdown_connections 2024-03-19T01:09:32.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.992+0000 7fd976406740 1 -- 172.21.15.69:0/1625903228 wait complete. 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.992+0000 7fd976406740 1 -- start start 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd976406740 1 Processor -- start 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd976406740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6215dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd976406740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb627f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd976406740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb620bf50 0x561fb6281a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd976406740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561fb621ea10 con 0x561fb62063f0 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd976406740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fb621e340 con 0x561fb620bf50 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd976406740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561fb621f0e0 con 0x561fb6206de0 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd976406740 10 client.0 _ll_register_callbacks cb 0x7fd96001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd970e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6215dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd96be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb627f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd971800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb620bf50 0x561fb6281a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:32.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd96be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb627f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46194/0 (socket says 172.21.15.69:46194) 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd970e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6215dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33518/0 (socket says 172.21.15.69:33518) 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.993+0000 7fd96be00640 1 -- 172.21.15.69:0/3483963612 learned_addr learned my addr 172.21.15.69:0/3483963612 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd96be00640 1 -- 172.21.15.69:0/3483963612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb620bf50 msgr2=0x561fb6281a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd96be00640 1 --2- 172.21.15.69:0/3483963612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb620bf50 0x561fb6281a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd96be00640 1 -- 172.21.15.69:0/3483963612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 msgr2=0x561fb6215dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd96be00640 1 --2- 172.21.15.69:0/3483963612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6215dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd96be00640 1 -- 172.21.15.69:0/3483963612 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fb6282230 con 0x561fb6206de0 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd96be00640 1 --2- 172.21.15.69:0/3483963612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb627f470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9580980d0 tx=0x7fd95809b3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=209bf7c93ce21fd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd952a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd952a00640 1 -- 172.21.15.69:0/3483963612 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9580b8020 con 0x561fb6206de0 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd952a00640 1 -- 172.21.15.69:0/3483963612 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fd958097d70 con 0x561fb6206de0 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd976406740 1 -- 172.21.15.69:0/3483963612 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd960005d30 con 0x561fb6206de0 2024-03-19T01:09:32.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd952a00640 1 -- 172.21.15.69:0/3483963612 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9580056b0 con 0x561fb6206de0 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.994+0000 7fd976406740 10 client.25177 fetch_fsmap learned FSMap version 28 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.995+0000 7fd976406740 1 -- 172.21.15.69:0/3483963612 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fb6283600 con 0x561fb6206de0 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.995+0000 7fd952a00640 1 -- 172.21.15.69:0/3483963612 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9580a5080 con 0x561fb6206de0 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.995+0000 7fd952a00640 1 -- 172.21.15.69:0/3483963612 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd9580a3aa0 con 0x561fb6206de0 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.995+0000 7fd976406740 10 client.25177 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.995+0000 7fd976406740 10 client.25177 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.995+0000 7fd976406740 20 client.25177 populate_metadata read hostname 'smithi069' 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.995+0000 7fd976406740 1 --2- 172.21.15.69:0/3483963612 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x561fb6288160 0x561fb6286e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.995+0000 7fd976406740 4 client.25177 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.995+0000 7fd976406740 1 -- 172.21.15.69:0/3483963612 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "status"}) -- 0x561fb6172590 con 0x561fb6288160 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.995+0000 7fd970e00640 1 --2- 172.21.15.69:0/3483963612 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x561fb6288160 0x561fb6286e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd970e00640 1 --2- 172.21.15.69:0/3483963612 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x561fb6288160 0x561fb6286e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a32bf7e2aa98301 server_cookie=be0188e1b5810e1 in_seq=0 out_seq=0 2024-03-19T01:09:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd952a00640 10 client.25177 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd952a00640 1 -- 172.21.15.69:0/3483963612 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561fb6172590 con 0x561fb6288160 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd952a00640 10 client.25177 handle_command_reply: tid=0 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd976406740 1 client.25177 shutdown 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd976406740 1 -- 172.21.15.69:0/3483963612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 msgr2=0x561fb627f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd976406740 1 --2- 172.21.15.69:0/3483963612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb627f470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9580980d0 tx=0x7fd95809b3c0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd976406740 1 -- 172.21.15.69:0/3483963612 shutdown_connections 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd976406740 1 --2- 172.21.15.69:0/3483963612 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x561fb6288160 0x561fb6286e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd976406740 1 --2- 172.21.15.69:0/3483963612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb620bf50 0x561fb6281a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd976406740 1 --2- 172.21.15.69:0/3483963612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fb6206de0 0x561fb627f470 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd952a00640 0 client.25177 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd976406740 1 --2- 172.21.15.69:0/3483963612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561fb62063f0 0x561fb6215dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd976406740 1 -- 172.21.15.69:0/3483963612 >> 172.21.15.69:0/3483963612 conn(0x561fb6201960 msgr2=0x561fb621c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.996+0000 7fd976406740 1 -- 172.21.15.69:0/3483963612 shutdown_connections 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.997+0000 7fd976406740 1 -- 172.21.15.69:0/3483963612 wait complete. 2024-03-19T01:09:33.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.997+0000 7fd976406740 20 client.25177 trim_cache size 0 max 16384 2024-03-19T01:09:33.002 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:09:33.002 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T01:09:33.002 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 3, 2024-03-19T01:09:33.002 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24494, 2024-03-19T01:09:33.002 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-03-19T01:09:33.002 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-03-19T01:09:33.002 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-03-19T01:09:33.002 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 104.235838414, 2024-03-19T01:09:33.002 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-03-19T01:09:33.003 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 93, 2024-03-19T01:09:33.003 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T01:09:33.003 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 166.92577846200001 2024-03-19T01:09:33.003 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:09:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.997+0000 7fd973800640 1 -- 172.21.15.69:0/2547241667 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd9540a2880 msgr2=0x7fd9540a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.997+0000 7fd973800640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd9540a2880 0x7fd9540a4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd960011eb0 tx=0x7fd960000bc0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.997+0000 7fd973800640 1 -- 172.21.15.69:0/2547241667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c104c80 msgr2=0x7fd96c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.997+0000 7fd973800640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c104c80 0x7fd96c19a260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd95800e9f0 tx=0x7fd95800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.997+0000 7fd973800640 1 -- 172.21.15.69:0/2547241667 shutdown_connections 2024-03-19T01:09:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.997+0000 7fd973800640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd9540a2880 0x7fd9540a4d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.997+0000 7fd973800640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd96c1075b0 0x7fd96c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.998+0000 7fd973800640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd96c104c80 0x7fd96c19a260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.998+0000 7fd973800640 1 --2- 172.21.15.69:0/2547241667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd96c102350 0x7fd96c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.998+0000 7fd973800640 1 -- 172.21.15.69:0/2547241667 >> 172.21.15.69:0/2547241667 conn(0x7fd96c0fbcc0 msgr2=0x7fd96c101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.998+0000 7fd973800640 1 -- 172.21.15.69:0/2547241667 shutdown_connections 2024-03-19T01:09:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:32.998+0000 7fd973800640 1 -- 172.21.15.69:0/2547241667 wait complete. 2024-03-19T01:09:33.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:32 smithi069 ceph-mon[31727]: pgmap v387: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-19T01:09:33.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:32 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3983814181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:09:33.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:32 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1263458553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:09:33.019 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-19T01:09:33.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.212+0000 7f61d5600640 1 -- 172.21.15.69:0/2067112118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d0102a20 msgr2=0x7f61d0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.212+0000 7f61d5600640 1 --2- 172.21.15.69:0/2067112118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d0102a20 0x7f61d0102ea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009900 tx=0x7f61bc02bc30 comp rx=0 tx=0).stop 2024-03-19T01:09:33.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.212+0000 7f61d5600640 1 -- 172.21.15.69:0/2067112118 shutdown_connections 2024-03-19T01:09:33.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.212+0000 7f61d5600640 1 --2- 172.21.15.69:0/2067112118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61d01033e0 0x7f61d0109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.212+0000 7f61d5600640 1 --2- 172.21.15.69:0/2067112118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d0102a20 0x7f61d0102ea0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.212+0000 7f61d5600640 1 --2- 172.21.15.69:0/2067112118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61d0101830 0x7f61d0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.212+0000 7f61d5600640 1 -- 172.21.15.69:0/2067112118 >> 172.21.15.69:0/2067112118 conn(0x7f61d00fcdc0 msgr2=0x7f61d00ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.212+0000 7f61d5600640 1 -- 172.21.15.69:0/2067112118 shutdown_connections 2024-03-19T01:09:33.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.212+0000 7f61d5600640 1 -- 172.21.15.69:0/2067112118 wait complete. 2024-03-19T01:09:33.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.213+0000 7f61d5600640 1 Processor -- start 2024-03-19T01:09:33.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.213+0000 7f61d5600640 1 -- start start 2024-03-19T01:09:33.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.213+0000 7f61d5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d0101830 0x7f61d0199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.213+0000 7f61d5600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61d0102a20 0x7f61d019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.213+0000 7f61d5600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61d01033e0 0x7f61d01a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.213+0000 7f61d5600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f61d010d490 con 0x7f61d01033e0 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.213+0000 7f61d5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61d010d310 con 0x7f61d0101830 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.213+0000 7f61d5600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f61d010d610 con 0x7f61d0102a20 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d0101830 0x7f61d0199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61ce000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61d0102a20 0x7f61d019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d0101830 0x7f61d0199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57328/0 (socket says 172.21.15.69:57328) 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61cea00640 1 -- 172.21.15.69:0/3008658119 learned_addr learned my addr 172.21.15.69:0/3008658119 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61cf400640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61d01033e0 0x7f61d01a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61cea00640 1 -- 172.21.15.69:0/3008658119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61d0102a20 msgr2=0x7f61d019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61cea00640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61d0102a20 0x7f61d019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61cea00640 1 -- 172.21.15.69:0/3008658119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61d01033e0 msgr2=0x7f61d01a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61cea00640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61d01033e0 0x7f61d01a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61cea00640 1 -- 172.21.15.69:0/3008658119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d01a1950 con 0x7f61d0101830 2024-03-19T01:09:33.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61cea00640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d0101830 0x7f61d0199d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f61c000e9c0 tx=0x7f61c000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61c7400640 1 -- 172.21.15.69:0/3008658119 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c000cd80 con 0x7f61d0101830 2024-03-19T01:09:33.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.214+0000 7f61c7400640 1 -- 172.21.15.69:0/3008658119 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61c00035f0 con 0x7f61d0101830 2024-03-19T01:09:33.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.215+0000 7f61d5600640 1 -- 172.21.15.69:0/3008658119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d01a1c40 con 0x7f61d0101830 2024-03-19T01:09:33.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.215+0000 7f61c7400640 1 -- 172.21.15.69:0/3008658119 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0010690 con 0x7f61d0101830 2024-03-19T01:09:33.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.215+0000 7f61d5600640 1 -- 172.21.15.69:0/3008658119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d01a2190 con 0x7f61d0101830 2024-03-19T01:09:33.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.218+0000 7f61c7400640 1 -- 172.21.15.69:0/3008658119 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f61c00037a0 con 0x7f61d0101830 2024-03-19T01:09:33.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.219+0000 7f61c7400640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61b40a2880 0x7f61b40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.219+0000 7f61c7400640 1 -- 172.21.15.69:0/3008658119 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f61c00cf2c0 con 0x7f61d0101830 2024-03-19T01:09:33.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.219+0000 7f61d8215740 1 Processor -- start 2024-03-19T01:09:33.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.219+0000 7f61ce000640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61b40a2880 0x7f61b40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.219+0000 7f61d8215740 1 -- start start 2024-03-19T01:09:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.220+0000 7f61d8215740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce099020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.220+0000 7f61d8215740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce0995f0 0x55bfce09e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.220+0000 7f61d8215740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 0x55bfce0a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.220+0000 7f61d8215740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfcdff4d00 con 0x55bfce0995f0 2024-03-19T01:09:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.220+0000 7f61d8215740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bfcdf9db00 con 0x55bfce09e760 2024-03-19T01:09:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.220+0000 7f61d8215740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bfcdff0da0 con 0x55bfce098c00 2024-03-19T01:09:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.220+0000 7f61cea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce099020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.220+0000 7f61cea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce099020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46228/0 (socket says 172.21.15.69:46228) 2024-03-19T01:09:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.220+0000 7f61cea00640 1 -- 172.21.15.69:0/829539891 learned_addr learned my addr 172.21.15.69:0/829539891 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.220+0000 7f61cf400640 1 --2- 172.21.15.69:0/829539891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 0x55bfce0a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.220+0000 7f61ce000640 1 --2- 172.21.15.69:0/829539891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce0995f0 0x55bfce09e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.221+0000 7f61cea00640 1 -- 172.21.15.69:0/829539891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 msgr2=0x55bfce0a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.221+0000 7f61ce000640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61b40a2880 0x7f61b40a4d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f61bc02c7b0 tx=0x7f61bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.221+0000 7f61cea00640 1 --2- 172.21.15.69:0/829539891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 0x55bfce0a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.221+0000 7f61cea00640 1 -- 172.21.15.69:0/829539891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce0995f0 msgr2=0x55bfce09e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.221+0000 7f61cea00640 1 --2- 172.21.15.69:0/829539891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce0995f0 0x55bfce09e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.221+0000 7f61cea00640 1 -- 172.21.15.69:0/829539891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfcdd092d0 con 0x55bfce098c00 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.221+0000 7f61cf400640 1 --2- 172.21.15.69:0/829539891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 0x55bfce0a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.221+0000 7f61cea00640 1 --2- 172.21.15.69:0/829539891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce099020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61c00a3ac0 tx=0x7f61c009b270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57f97f101a254b76 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.221+0000 7f61bb400640 1 -- 172.21.15.69:0/829539891 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c00b4070 con 0x55bfce098c00 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.221+0000 7f61bb400640 1 -- 172.21.15.69:0/829539891 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61c009b850 con 0x55bfce098c00 2024-03-19T01:09:33.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.221+0000 7f61bb400640 1 -- 172.21.15.69:0/829539891 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0097e30 con 0x55bfce098c00 2024-03-19T01:09:33.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.222+0000 7f61d8215740 1 -- 172.21.15.69:0/829539891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 msgr2=0x55bfce099020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.222+0000 7f61d8215740 1 --2- 172.21.15.69:0/829539891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce099020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61c00a3ac0 tx=0x7f61c009b270 comp rx=0 tx=0).stop 2024-03-19T01:09:33.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.222+0000 7f61d8215740 1 -- 172.21.15.69:0/829539891 shutdown_connections 2024-03-19T01:09:33.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.222+0000 7f61d8215740 1 --2- 172.21.15.69:0/829539891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 0x55bfce0a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.222+0000 7f61d8215740 1 --2- 172.21.15.69:0/829539891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce0995f0 0x55bfce09e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.222+0000 7f61d8215740 1 --2- 172.21.15.69:0/829539891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce099020 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.222+0000 7f61d8215740 1 -- 172.21.15.69:0/829539891 >> 172.21.15.69:0/829539891 conn(0x55bfce08b960 msgr2=0x55bfce08bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.222+0000 7f61d8215740 1 -- 172.21.15.69:0/829539891 shutdown_connections 2024-03-19T01:09:33.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.222+0000 7f61d8215740 1 -- 172.21.15.69:0/829539891 wait complete. 2024-03-19T01:09:33.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.222+0000 7f61d8215740 1 -- start start 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.223+0000 7f61d8215740 1 Processor -- start 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.223+0000 7f61d8215740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 0x55bfce0a30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.223+0000 7f61d8215740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce0995f0 0x55bfce0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.223+0000 7f61d8215740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce09e760 0x55bfce0aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.223+0000 7f61d8215740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfce0a16b0 con 0x55bfce098c00 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.223+0000 7f61d8215740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bfcdfb0ed0 con 0x55bfce0995f0 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.223+0000 7f61d8215740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bfce0a1830 con 0x55bfce09e760 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.223+0000 7f61d8215740 10 client.0 _ll_register_callbacks cb 0x55bfcdfaec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.223+0000 7f61cea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 0x55bfce0a30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce0995f0 0x55bfce0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61cea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 0x55bfce0a30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33538/0 (socket says 172.21.15.69:33538) 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61cea00640 1 -- 172.21.15.69:0/1696122443 learned_addr learned my addr 172.21.15.69:0/1696122443 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61cf400640 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce09e760 0x55bfce0aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61cea00640 1 -- 172.21.15.69:0/1696122443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce09e760 msgr2=0x55bfce0aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61cea00640 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce09e760 0x55bfce0aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61cea00640 1 -- 172.21.15.69:0/1696122443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce0995f0 msgr2=0x55bfce0a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61cea00640 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce0995f0 0x55bfce0a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61cea00640 1 -- 172.21.15.69:0/1696122443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfcddd36b0 con 0x55bfce098c00 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61cea00640 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 0x55bfce0a30c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f61c000c850 tx=0x7f61c0086050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0971a563e4e794 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61afe00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61afe00640 1 -- 172.21.15.69:0/1696122443 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c007f060 con 0x55bfce098c00 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61afe00640 1 -- 172.21.15.69:0/1696122443 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f61c009cd90 con 0x55bfce098c00 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.224+0000 7f61d8215740 1 -- 172.21.15.69:0/1696122443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c009b850 con 0x55bfce098c00 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.225+0000 7f61afe00640 1 -- 172.21.15.69:0/1696122443 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61c00ba070 con 0x55bfce098c00 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.225+0000 7f61d8215740 10 client.15069 fetch_fsmap learned FSMap version 28 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.225+0000 7f61d8215740 1 -- 172.21.15.69:0/1696122443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bfce0ad9b0 con 0x55bfce098c00 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.225+0000 7f61afe00640 1 -- 172.21.15.69:0/1696122443 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61c009b850 con 0x55bfce098c00 2024-03-19T01:09:33.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.225+0000 7f61afe00640 1 -- 172.21.15.69:0/1696122443 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f61c009a8b0 con 0x55bfce098c00 2024-03-19T01:09:33.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.225+0000 7f61d8215740 10 client.15069 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:09:33.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.225+0000 7f61d8215740 10 client.15069 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T01:09:33.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.225+0000 7f61d8215740 20 client.15069 populate_metadata read hostname 'smithi069' 2024-03-19T01:09:33.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.225+0000 7f61d8215740 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55bfce0aed00 0x55bfce0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.225+0000 7f61d8215740 4 client.15069 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:09:33.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.225+0000 7f61d8215740 1 -- 172.21.15.69:0/1696122443 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfce002240 con 0x55bfce0aed00 2024-03-19T01:09:33.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.226+0000 7f61ce000640 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55bfce0aed00 0x55bfce0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.226+0000 7f61ce000640 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55bfce0aed00 0x55bfce0b0f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81972d1d51cb31da server_cookie=c0535d7640efbbe0 in_seq=0 out_seq=0 2024-03-19T01:09:33.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.226+0000 7f61afe00640 10 client.15069 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:09:33.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61afe00640 1 -- 172.21.15.69:0/1696122443 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bfce002240 con 0x55bfce0aed00 2024-03-19T01:09:33.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61afe00640 10 client.15069 handle_command_reply: tid=0 2024-03-19T01:09:33.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61d8215740 1 client.15069 shutdown 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61d8215740 1 -- 172.21.15.69:0/1696122443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 msgr2=0x55bfce0a30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61d8215740 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 0x55bfce0a30c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f61c000c850 tx=0x7f61c0086050 comp rx=0 tx=0).stop 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61d8215740 1 -- 172.21.15.69:0/1696122443 shutdown_connections 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61d8215740 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55bfce0aed00 0x55bfce0b0f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61d8215740 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce09e760 0x55bfce0aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61d8215740 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce0995f0 0x55bfce0a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61afe00640 0 client.15069 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61d8215740 1 --2- 172.21.15.69:0/1696122443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 0x55bfce0a30c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.227+0000 7f61d8215740 1 -- 172.21.15.69:0/1696122443 >> 172.21.15.69:0/1696122443 conn(0x55bfce08b960 msgr2=0x55bfce08aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.228+0000 7f61d8215740 1 -- 172.21.15.69:0/1696122443 shutdown_connections 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.228+0000 7f61d8215740 1 -- 172.21.15.69:0/1696122443 wait complete. 2024-03-19T01:09:33.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.228+0000 7f61d8215740 20 client.15069 trim_cache size 0 max 16384 2024-03-19T01:09:33.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:32 smithi086 ceph-mon[66829]: pgmap v387: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-19T01:09:33.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:32 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3983814181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:09:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:32 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1263458553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:09:33.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.261+0000 7f61d8215740 1 Processor -- start 2024-03-19T01:09:33.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.261+0000 7f61d8215740 1 -- start start 2024-03-19T01:09:33.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.261+0000 7f61d8215740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 0x55bfce0a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.261+0000 7f61d8215740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce0995f0 0x55bfce0aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.261+0000 7f61d8215740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 0x55bfce09a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.261+0000 7f61d8215740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfce0af240 con 0x55bfce098c00 2024-03-19T01:09:33.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.261+0000 7f61d8215740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bfce0af910 con 0x55bfce09e760 2024-03-19T01:09:33.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.261+0000 7f61d8215740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bfce0affe0 con 0x55bfce0995f0 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61cea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 0x55bfce0a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 0x55bfce09a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61ce000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce0995f0 0x55bfce0aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61ce000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce0995f0 0x55bfce0aa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46258/0 (socket says 172.21.15.69:46258) 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 0x55bfce09a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57362/0 (socket says 172.21.15.69:57362) 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61ce000640 1 -- 172.21.15.69:0/3679892207 learned_addr learned my addr 172.21.15.69:0/3679892207 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61ce000640 1 -- 172.21.15.69:0/3679892207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 msgr2=0x55bfce09a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61ce000640 1 --2- 172.21.15.69:0/3679892207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 0x55bfce09a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61ce000640 1 -- 172.21.15.69:0/3679892207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 msgr2=0x55bfce0a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61ce000640 1 --2- 172.21.15.69:0/3679892207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 0x55bfce0a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61ce000640 1 -- 172.21.15.69:0/3679892207 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfce085c90 con 0x55bfce0995f0 2024-03-19T01:09:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61cf400640 1 --2- 172.21.15.69:0/3679892207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 0x55bfce09a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61cea00640 1 --2- 172.21.15.69:0/3679892207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 0x55bfce0a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.262+0000 7f61ce000640 1 --2- 172.21.15.69:0/3679892207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce0995f0 0x55bfce0aa600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61bc02c710 tx=0x7f61bc040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3abf0d1ebf2dec9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61baa00640 1 -- 172.21.15.69:0/3679892207 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc053ca0 con 0x55bfce0995f0 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61baa00640 1 -- 172.21.15.69:0/3679892207 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61bc058070 con 0x55bfce0995f0 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61baa00640 1 -- 172.21.15.69:0/3679892207 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0046c0 con 0x55bfce0995f0 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61d8215740 1 -- 172.21.15.69:0/3679892207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce0995f0 msgr2=0x55bfce0aa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61d8215740 1 --2- 172.21.15.69:0/3679892207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce0995f0 0x55bfce0aa600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61bc02c710 tx=0x7f61bc040000 comp rx=0 tx=0).stop 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61d8215740 1 -- 172.21.15.69:0/3679892207 shutdown_connections 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61d8215740 1 --2- 172.21.15.69:0/3679892207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce09e760 0x55bfce09a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61d8215740 1 --2- 172.21.15.69:0/3679892207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce0995f0 0x55bfce0aa600 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61d8215740 1 --2- 172.21.15.69:0/3679892207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce098c00 0x55bfce0a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61d8215740 1 -- 172.21.15.69:0/3679892207 >> 172.21.15.69:0/3679892207 conn(0x55bfce08b960 msgr2=0x55bfce08bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61d8215740 1 -- 172.21.15.69:0/3679892207 shutdown_connections 2024-03-19T01:09:33.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.263+0000 7f61d8215740 1 -- 172.21.15.69:0/3679892207 wait complete. 2024-03-19T01:09:33.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.264+0000 7f61d8215740 1 -- start start 2024-03-19T01:09:33.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.264+0000 7f61d8215740 1 Processor -- start 2024-03-19T01:09:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.264+0000 7f61d8215740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce0a85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.264+0000 7f61d8215740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce0995f0 0x55bfce1117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.264+0000 7f61d8215740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce09e760 0x55bfce113d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.264+0000 7f61d8215740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfce0b0d80 con 0x55bfce09e760 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.264+0000 7f61d8215740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bfce0b06b0 con 0x55bfce0995f0 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.264+0000 7f61d8215740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bfce0b1450 con 0x55bfce098c00 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.264+0000 7f61d8215740 10 client.0 _ll_register_callbacks cb 0x7f61c00b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61cea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce0a85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61cea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce0a85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46264/0 (socket says 172.21.15.69:46264) 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61cea00640 1 -- 172.21.15.69:0/3486529370 learned_addr learned my addr 172.21.15.69:0/3486529370 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61ce000640 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce0995f0 0x55bfce1117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61cf400640 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce09e760 0x55bfce113d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61cea00640 1 -- 172.21.15.69:0/3486529370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce0995f0 msgr2=0x55bfce1117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61cea00640 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce0995f0 0x55bfce1117e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61cea00640 1 -- 172.21.15.69:0/3486529370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce09e760 msgr2=0x55bfce113d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61cea00640 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce09e760 0x55bfce113d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61cea00640 1 -- 172.21.15.69:0/3486529370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfce1145c0 con 0x55bfce098c00 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61ce000640 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce0995f0 0x55bfce1117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61cea00640 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce0a85f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f61c00058c0 tx=0x7f61c009b270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=723df8f53c5154e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61afe00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61afe00640 1 -- 172.21.15.69:0/3486529370 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c007f070 con 0x55bfce098c00 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61afe00640 1 -- 172.21.15.69:0/3486529370 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f61c00bb4e0 con 0x55bfce098c00 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.265+0000 7f61d8215740 1 -- 172.21.15.69:0/3486529370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61bc058070 con 0x55bfce098c00 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.266+0000 7f61afe00640 1 -- 172.21.15.69:0/3486529370 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61c00b4040 con 0x55bfce098c00 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.266+0000 7f61d8215740 10 client.25195 fetch_fsmap learned FSMap version 28 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.266+0000 7f61d8215740 1 -- 172.21.15.69:0/3486529370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bfce1158c0 con 0x55bfce098c00 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.266+0000 7f61afe00640 1 -- 172.21.15.69:0/3486529370 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61bc058070 con 0x55bfce098c00 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.266+0000 7f61afe00640 1 -- 172.21.15.69:0/3486529370 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f61c009a710 con 0x55bfce098c00 2024-03-19T01:09:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.266+0000 7f61d8215740 10 client.25195 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.266+0000 7f61d8215740 10 client.25195 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.266+0000 7f61d8215740 20 client.25195 populate_metadata read hostname 'smithi069' 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.266+0000 7f61d8215740 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55bfce11a2e0 0x55bfce118ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.266+0000 7f61d8215740 4 client.25195 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.266+0000 7f61d8215740 1 -- 172.21.15.69:0/3486529370 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "status"}) -- 0x55bfcdffc590 con 0x55bfce11a2e0 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61ce000640 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55bfce11a2e0 0x55bfce118ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61ce000640 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55bfce11a2e0 0x55bfce118ff0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4193c54f92d324 server_cookie=8db8624151964859 in_seq=0 out_seq=0 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61afe00640 10 client.25195 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61afe00640 1 -- 172.21.15.69:0/3486529370 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bfcdffc590 con 0x55bfce11a2e0 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61afe00640 10 client.25195 handle_command_reply: tid=0 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61d8215740 1 client.25195 shutdown 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61d8215740 1 -- 172.21.15.69:0/3486529370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 msgr2=0x55bfce0a85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61d8215740 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce0a85f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f61c00058c0 tx=0x7f61c009b270 comp rx=0 tx=0).stop 2024-03-19T01:09:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61d8215740 1 -- 172.21.15.69:0/3486529370 shutdown_connections 2024-03-19T01:09:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61d8215740 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55bfce11a2e0 0x55bfce118ff0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61d8215740 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfce09e760 0x55bfce113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61d8215740 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfce0995f0 0x55bfce1117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61d8215740 1 --2- 172.21.15.69:0/3486529370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bfce098c00 0x55bfce0a85f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61afe00640 0 client.25195 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:09:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.267+0000 7f61d8215740 1 -- 172.21.15.69:0/3486529370 >> 172.21.15.69:0/3486529370 conn(0x55bfce08b960 msgr2=0x55bfce09a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.268+0000 7f61d8215740 1 -- 172.21.15.69:0/3486529370 shutdown_connections 2024-03-19T01:09:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.268+0000 7f61d8215740 1 -- 172.21.15.69:0/3486529370 wait complete. 2024-03-19T01:09:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.268+0000 7f61d8215740 20 client.25195 trim_cache size 0 max 16384 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24550, 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 105.467564959, 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 93, 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 155.38663275799999 2024-03-19T01:09:33.275 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:09:33.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.268+0000 7f61d5600640 1 -- 172.21.15.69:0/3008658119 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61b40a2880 msgr2=0x7f61b40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.268+0000 7f61d5600640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61b40a2880 0x7f61b40a4d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f61bc02c7b0 tx=0x7f61bc005fd0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.268+0000 7f61d5600640 1 -- 172.21.15.69:0/3008658119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d0101830 msgr2=0x7f61d0199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.268+0000 7f61d5600640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d0101830 0x7f61d0199d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f61c000e9c0 tx=0x7f61c000ee80 comp rx=0 tx=0).stop 2024-03-19T01:09:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.268+0000 7f61d5600640 1 -- 172.21.15.69:0/3008658119 shutdown_connections 2024-03-19T01:09:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.269+0000 7f61d5600640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f61b40a2880 0x7f61b40a4d40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.269+0000 7f61d5600640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61d01033e0 0x7f61d01a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.269+0000 7f61d5600640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61d0102a20 0x7f61d019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.269+0000 7f61d5600640 1 --2- 172.21.15.69:0/3008658119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d0101830 0x7f61d0199d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.269+0000 7f61d5600640 1 -- 172.21.15.69:0/3008658119 >> 172.21.15.69:0/3008658119 conn(0x7f61d00fcdc0 msgr2=0x7f61d0073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.269+0000 7f61d5600640 1 -- 172.21.15.69:0/3008658119 shutdown_connections 2024-03-19T01:09:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.269+0000 7f61d5600640 1 -- 172.21.15.69:0/3008658119 wait complete. 2024-03-19T01:09:33.287 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-19T01:09:33.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:32 smithi163 ceph-mon[67428]: pgmap v387: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-19T01:09:33.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:32 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3983814181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:09:33.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:32 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1263458553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:09:33.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.481+0000 7f4718600640 1 -- 172.21.15.69:0/1310828356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710102a30 msgr2=0x7f4710102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.481+0000 7f4718600640 1 --2- 172.21.15.69:0/1310828356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710102a30 0x7f4710102eb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009900 tx=0x7f46fc02bc30 comp rx=0 tx=0).stop 2024-03-19T01:09:33.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.481+0000 7f4718600640 1 -- 172.21.15.69:0/1310828356 shutdown_connections 2024-03-19T01:09:33.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.481+0000 7f4718600640 1 --2- 172.21.15.69:0/1310828356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47101033f0 0x7f4710109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.481+0000 7f4718600640 1 --2- 172.21.15.69:0/1310828356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710102a30 0x7f4710102eb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.481+0000 7f4718600640 1 --2- 172.21.15.69:0/1310828356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4710101840 0x7f4710101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.481+0000 7f4718600640 1 -- 172.21.15.69:0/1310828356 >> 172.21.15.69:0/1310828356 conn(0x7f47100fcdd0 msgr2=0x7f47100ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.481+0000 7f4718600640 1 -- 172.21.15.69:0/1310828356 shutdown_connections 2024-03-19T01:09:33.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.481+0000 7f4718600640 1 -- 172.21.15.69:0/1310828356 wait complete. 2024-03-19T01:09:33.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.482+0000 7f4718600640 1 Processor -- start 2024-03-19T01:09:33.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.482+0000 7f4718600640 1 -- start start 2024-03-19T01:09:33.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.482+0000 7f4718600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4710101840 0x7f4710199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.482+0000 7f4718600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710102a30 0x7f471019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.482+0000 7f4718600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47101033f0 0x7f47101a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.482+0000 7f4718600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f471010d4f0 con 0x7f47101033f0 2024-03-19T01:09:33.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.482+0000 7f4718600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f471010d370 con 0x7f4710102a30 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.482+0000 7f4718600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f471010d670 con 0x7f4710101840 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.483+0000 7f4715c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4710101840 0x7f4710199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.483+0000 7f4715c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4710101840 0x7f4710199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46294/0 (socket says 172.21.15.69:46294) 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.483+0000 7f4715c00640 1 -- 172.21.15.69:0/976882865 learned_addr learned my addr 172.21.15.69:0/976882865 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.483+0000 7f4715200640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710102a30 0x7f471019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.483+0000 7f4716600640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47101033f0 0x7f47101a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.483+0000 7f4715c00640 1 -- 172.21.15.69:0/976882865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710102a30 msgr2=0x7f471019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.483+0000 7f4715c00640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710102a30 0x7f471019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.483+0000 7f4715c00640 1 -- 172.21.15.69:0/976882865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47101033f0 msgr2=0x7f47101a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.483+0000 7f4715c00640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47101033f0 0x7f47101a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.483+0000 7f4715c00640 1 -- 172.21.15.69:0/976882865 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47101a19d0 con 0x7f4710101840 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.483+0000 7f4715200640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710102a30 0x7f471019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.484+0000 7f4715c00640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4710101840 0x7f4710199dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f470400e9c0 tx=0x7f470400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.484+0000 7f470e000640 1 -- 172.21.15.69:0/976882865 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470400ccc0 con 0x7f4710101840 2024-03-19T01:09:33.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.484+0000 7f470e000640 1 -- 172.21.15.69:0/976882865 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47040035f0 con 0x7f4710101840 2024-03-19T01:09:33.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.484+0000 7f470e000640 1 -- 172.21.15.69:0/976882865 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470400ce60 con 0x7f4710101840 2024-03-19T01:09:33.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.484+0000 7f4718600640 1 -- 172.21.15.69:0/976882865 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47101a1cc0 con 0x7f4710101840 2024-03-19T01:09:33.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.484+0000 7f4718600640 1 -- 172.21.15.69:0/976882865 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47101a21c0 con 0x7f4710101840 2024-03-19T01:09:33.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.487+0000 7f470e000640 1 -- 172.21.15.69:0/976882865 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f47040037a0 con 0x7f4710101840 2024-03-19T01:09:33.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.488+0000 7f471b3a6740 1 Processor -- start 2024-03-19T01:09:33.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.488+0000 7f471b3a6740 1 -- start start 2024-03-19T01:09:33.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968bd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bdde0 0x55b5968c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968c2f50 0x55b5968c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f471b3a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b596821d00 con 0x55b5968bdde0 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f471b3a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5967cab00 con 0x55b5968c2f50 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f471b3a6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b59681dda0 con 0x55b5968bd3f0 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f470e000640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f46f80a2880 0x7f46f80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f4715200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968c2f50 0x55b5968c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f4715200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968c2f50 0x55b5968c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57404/0 (socket says 172.21.15.69:57404) 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f4715200640 1 -- 172.21.15.69:0/3472108217 learned_addr learned my addr 172.21.15.69:0/3472108217 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f470e000640 1 -- 172.21.15.69:0/976882865 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f47040cf180 con 0x7f4710101840 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f4716600640 1 --2- 172.21.15.69:0/3472108217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f4715c00640 1 --2- 172.21.15.69:0/3472108217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bdde0 0x55b5968c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f4716600640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f46f80a2880 0x7f46f80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f4715200640 1 -- 172.21.15.69:0/3472108217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 msgr2=0x55b5968bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f4715200640 1 --2- 172.21.15.69:0/3472108217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968bd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f4715200640 1 -- 172.21.15.69:0/3472108217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bdde0 msgr2=0x55b5968c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.489+0000 7f4715200640 1 --2- 172.21.15.69:0/3472108217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bdde0 0x55b5968c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.490+0000 7f4715200640 1 -- 172.21.15.69:0/3472108217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5965362d0 con 0x55b5968c2f50 2024-03-19T01:09:33.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.490+0000 7f4715c00640 1 --2- 172.21.15.69:0/3472108217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bdde0 0x55b5968c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:33.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.490+0000 7f4715200640 1 --2- 172.21.15.69:0/3472108217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968c2f50 0x55b5968c53e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f46fc005ab0 tx=0x7f46fc009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bfce324ff56fca6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.490+0000 7f4702a00640 1 -- 172.21.15.69:0/3472108217 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc045070 con 0x55b5968c2f50 2024-03-19T01:09:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.490+0000 7f4702a00640 1 -- 172.21.15.69:0/3472108217 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46fc02cd30 con 0x55b5968c2f50 2024-03-19T01:09:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.490+0000 7f4702a00640 1 -- 172.21.15.69:0/3472108217 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc005130 con 0x55b5968c2f50 2024-03-19T01:09:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.490+0000 7f4716600640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f46f80a2880 0x7f46f80a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55b5968be640 tx=0x7f4708013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.491+0000 7f471b3a6740 1 -- 172.21.15.69:0/3472108217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968c2f50 msgr2=0x55b5968c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.491+0000 7f471b3a6740 1 --2- 172.21.15.69:0/3472108217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968c2f50 0x55b5968c53e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f46fc005ab0 tx=0x7f46fc009560 comp rx=0 tx=0).stop 2024-03-19T01:09:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.491+0000 7f471b3a6740 1 -- 172.21.15.69:0/3472108217 shutdown_connections 2024-03-19T01:09:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.491+0000 7f471b3a6740 1 --2- 172.21.15.69:0/3472108217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968c2f50 0x55b5968c53e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.491+0000 7f471b3a6740 1 --2- 172.21.15.69:0/3472108217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bdde0 0x55b5968c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.491+0000 7f471b3a6740 1 --2- 172.21.15.69:0/3472108217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968bd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.491+0000 7f471b3a6740 1 -- 172.21.15.69:0/3472108217 >> 172.21.15.69:0/3472108217 conn(0x55b5968b8960 msgr2=0x55b5968b8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.491+0000 7f471b3a6740 1 -- 172.21.15.69:0/3472108217 shutdown_connections 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.491+0000 7f471b3a6740 1 -- 172.21.15.69:0/3472108217 wait complete. 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f471b3a6740 1 -- start start 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f471b3a6740 1 Processor -- start 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bd3f0 0x55b5968c7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968c2f50 0x55b5968ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f471b3a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b5968c5ea0 con 0x55b5968bd3f0 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f471b3a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5967dded0 con 0x55b5968bdde0 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f471b3a6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b5968c6020 con 0x55b5968c2f50 2024-03-19T01:09:33.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f471b3a6740 10 client.0 _ll_register_callbacks cb 0x55b5967dbc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f4715200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f4715200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57418/0 (socket says 172.21.15.69:57418) 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.492+0000 7f4715200640 1 -- 172.21.15.69:0/1959841230 learned_addr learned my addr 172.21.15.69:0/1959841230 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f4715c00640 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bd3f0 0x55b5968c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f4716600640 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968c2f50 0x55b5968ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f4715200640 1 -- 172.21.15.69:0/1959841230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968c2f50 msgr2=0x55b5968ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f4715200640 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968c2f50 0x55b5968ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f4715200640 1 -- 172.21.15.69:0/1959841230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bd3f0 msgr2=0x55b5968c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f4715200640 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bd3f0 0x55b5968c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f4715200640 1 -- 172.21.15.69:0/1959841230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5966006b0 con 0x55b5968bdde0 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f4715c00640 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bd3f0 0x55b5968c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f4715200640 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968c7d60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009900 tx=0x7f46fc005100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=220eeb3c7ab09b39 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f46f7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:09:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f46f7400640 1 -- 172.21.15.69:0/1959841230 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc045040 con 0x55b5968bdde0 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f471b3a6740 1 -- 172.21.15.69:0/1959841230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46fc02cd30 con 0x55b5968bdde0 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.493+0000 7f46f7400640 1 -- 172.21.15.69:0/1959841230 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f46fc04a450 con 0x55b5968bdde0 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.494+0000 7f46f7400640 1 -- 172.21.15.69:0/1959841230 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46fc05a050 con 0x55b5968bdde0 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.494+0000 7f471b3a6740 10 client.25202 fetch_fsmap learned FSMap version 28 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.494+0000 7f471b3a6740 1 -- 172.21.15.69:0/1959841230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5968d2110 con 0x55b5968bdde0 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.494+0000 7f46f7400640 1 -- 172.21.15.69:0/1959841230 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46fc02cd30 con 0x55b5968bdde0 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.494+0000 7f46f7400640 1 -- 172.21.15.69:0/1959841230 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f46fc03f7a0 con 0x55b5968bdde0 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.494+0000 7f471b3a6740 10 client.25202 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.494+0000 7f471b3a6740 10 client.25202 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.494+0000 7f471b3a6740 20 client.25202 populate_metadata read hostname 'smithi069' 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.494+0000 7f471b3a6740 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55b5968d4370 0x55b5968d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.494+0000 7f471b3a6740 4 client.25202 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.494+0000 7f471b3a6740 1 -- 172.21.15.69:0/1959841230 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b59682f240 con 0x55b5968d4370 2024-03-19T01:09:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.495+0000 7f4715c00640 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55b5968d4370 0x55b5968d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.495+0000 7f4715c00640 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55b5968d4370 0x55b5968d57e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f34faf379bb5d841 server_cookie=8287286964c1bac8 in_seq=0 out_seq=0 2024-03-19T01:09:33.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.495+0000 7f46f7400640 10 client.25202 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:09:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f46f7400640 1 -- 172.21.15.69:0/1959841230 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b59682f240 con 0x55b5968d4370 2024-03-19T01:09:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f46f7400640 10 client.25202 handle_command_reply: tid=0 2024-03-19T01:09:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 1 client.25202 shutdown 2024-03-19T01:09:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 1 -- 172.21.15.69:0/1959841230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 msgr2=0x55b5968c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968c7d60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009900 tx=0x7f46fc005100 comp rx=0 tx=0).stop 2024-03-19T01:09:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 1 -- 172.21.15.69:0/1959841230 shutdown_connections 2024-03-19T01:09:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968c2f50 0x55b5968ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55b5968d4370 0x55b5968d57e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968c7d60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 1 --2- 172.21.15.69:0/1959841230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968bd3f0 0x55b5968c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f46f7400640 0 client.25202 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:09:33.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 1 -- 172.21.15.69:0/1959841230 >> 172.21.15.69:0/1959841230 conn(0x55b5968b8960 msgr2=0x55b5968b7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 1 -- 172.21.15.69:0/1959841230 shutdown_connections 2024-03-19T01:09:33.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 1 -- 172.21.15.69:0/1959841230 wait complete. 2024-03-19T01:09:33.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.497+0000 7f471b3a6740 20 client.25202 trim_cache size 0 max 16384 2024-03-19T01:09:33.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.517+0000 7f471b3a6740 1 Processor -- start 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.517+0000 7f471b3a6740 1 -- start start 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968c2f50 0x55b5968b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f471b3a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b5968d3b20 con 0x55b5968c2f50 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f471b3a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5968d41f0 con 0x55b5968bdde0 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f471b3a6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b5968d48c0 con 0x55b5968bd3f0 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f4715c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f4715c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46320/0 (socket says 172.21.15.69:46320) 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f4715c00640 1 -- 172.21.15.69:0/1146862739 learned_addr learned my addr 172.21.15.69:0/1146862739 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f4715200640 1 --2- 172.21.15.69:0/1146862739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f4716600640 1 --2- 172.21.15.69:0/1146862739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968c2f50 0x55b5968b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f4715c00640 1 -- 172.21.15.69:0/1146862739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 msgr2=0x55b5968ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f4715c00640 1 --2- 172.21.15.69:0/1146862739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f4715c00640 1 -- 172.21.15.69:0/1146862739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968c2f50 msgr2=0x55b5968b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f4715c00640 1 --2- 172.21.15.69:0/1146862739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968c2f50 0x55b5968b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f4715c00640 1 -- 172.21.15.69:0/1146862739 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5968be1c0 con 0x55b5968bd3f0 2024-03-19T01:09:33.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.518+0000 7f4715200640 1 --2- 172.21.15.69:0/1146862739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:33.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.519+0000 7f4715c00640 1 --2- 172.21.15.69:0/1146862739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968c5340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4704009830 tx=0x7f47040ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e2f4f988f68be41 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.519+0000 7f4702000640 1 -- 172.21.15.69:0/1146862739 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470409b770 con 0x55b5968bd3f0 2024-03-19T01:09:33.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.519+0000 7f4702000640 1 -- 172.21.15.69:0/1146862739 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f470409f8c0 con 0x55b5968bd3f0 2024-03-19T01:09:33.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.519+0000 7f4702000640 1 -- 172.21.15.69:0/1146862739 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040a3a80 con 0x55b5968bd3f0 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.519+0000 7f471b3a6740 1 -- 172.21.15.69:0/1146862739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 msgr2=0x55b5968c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.519+0000 7f471b3a6740 1 --2- 172.21.15.69:0/1146862739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968c5340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4704009830 tx=0x7f47040ce040 comp rx=0 tx=0).stop 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.519+0000 7f471b3a6740 1 -- 172.21.15.69:0/1146862739 shutdown_connections 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.519+0000 7f471b3a6740 1 --2- 172.21.15.69:0/1146862739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968c2f50 0x55b5968b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.519+0000 7f471b3a6740 1 --2- 172.21.15.69:0/1146862739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.519+0000 7f471b3a6740 1 --2- 172.21.15.69:0/1146862739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968c5340 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.519+0000 7f471b3a6740 1 -- 172.21.15.69:0/1146862739 >> 172.21.15.69:0/1146862739 conn(0x55b5968b8960 msgr2=0x55b5968b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.520+0000 7f471b3a6740 1 -- 172.21.15.69:0/1146862739 shutdown_connections 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.520+0000 7f471b3a6740 1 -- 172.21.15.69:0/1146862739 wait complete. 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.520+0000 7f471b3a6740 1 -- start start 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.520+0000 7f471b3a6740 1 Processor -- start 2024-03-19T01:09:33.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f471b3a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968c2f50 0x55b5968ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f471b3a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b5968d5660 con 0x55b5968c2f50 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f471b3a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5968d4f90 con 0x55b5968bdde0 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f471b3a6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b5968d5d30 con 0x55b5968bd3f0 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f471b3a6740 10 client.0 _ll_register_callbacks cb 0x7f4708009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f4715200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f4715c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f4715200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57434/0 (socket says 172.21.15.69:57434) 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f4715200640 1 -- 172.21.15.69:0/3468455390 learned_addr learned my addr 172.21.15.69:0/3468455390 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.521+0000 7f4716600640 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968c2f50 0x55b5968ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f4715200640 1 -- 172.21.15.69:0/3468455390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 msgr2=0x55b5968cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f4715200640 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f4715200640 1 -- 172.21.15.69:0/3468455390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968c2f50 msgr2=0x55b5968ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f4715200640 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968c2f50 0x55b5968ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f4715200640 1 -- 172.21.15.69:0/3468455390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5968cead0 con 0x55b5968bdde0 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f4715c00640 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f4715200640 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968cb510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46fc02c7b0 tx=0x7f46fc005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e60e9b75f9ee38b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f46f7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f46f7400640 1 -- 172.21.15.69:0/3468455390 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc045040 con 0x55b5968bdde0 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f471b3a6740 1 -- 172.21.15.69:0/3468455390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f470409f8c0 con 0x55b5968bdde0 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f46f7400640 1 -- 172.21.15.69:0/3468455390 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f46fc004040 con 0x55b5968bdde0 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f46f7400640 1 -- 172.21.15.69:0/3468455390 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46fc005d50 con 0x55b5968bdde0 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f471b3a6740 10 client.25208 fetch_fsmap learned FSMap version 28 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.522+0000 7f471b3a6740 1 -- 172.21.15.69:0/3468455390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b59693abc0 con 0x55b5968bdde0 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.523+0000 7f46f7400640 1 -- 172.21.15.69:0/3468455390 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f470409f8c0 con 0x55b5968bdde0 2024-03-19T01:09:33.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.523+0000 7f46f7400640 1 -- 172.21.15.69:0/3468455390 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f46fc004310 con 0x55b5968bdde0 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.523+0000 7f471b3a6740 10 client.25208 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.523+0000 7f471b3a6740 10 client.25208 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.523+0000 7f471b3a6740 20 client.25208 populate_metadata read hostname 'smithi069' 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.523+0000 7f471b3a6740 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55b59693f670 0x55b59693c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.523+0000 7f471b3a6740 4 client.25208 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.523+0000 7f471b3a6740 1 -- 172.21.15.69:0/3468455390 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "status"}) -- 0x55b596829590 con 0x55b59693f670 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.523+0000 7f4715c00640 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55b59693f670 0x55b59693c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f4715c00640 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55b59693f670 0x55b59693c180 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9074b9797ca92957 server_cookie=abea55c41ba2d430 in_seq=0 out_seq=0 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f46f7400640 10 client.25208 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f46f7400640 1 -- 172.21.15.69:0/3468455390 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b596829590 con 0x55b59693f670 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f46f7400640 10 client.25208 handle_command_reply: tid=0 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f471b3a6740 1 client.25208 shutdown 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f471b3a6740 1 -- 172.21.15.69:0/3468455390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 msgr2=0x55b5968cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f471b3a6740 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968cb510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46fc02c7b0 tx=0x7f46fc005a20 comp rx=0 tx=0).stop 2024-03-19T01:09:33.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f471b3a6740 1 -- 172.21.15.69:0/3468455390 shutdown_connections 2024-03-19T01:09:33.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f471b3a6740 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5968c2f50 0x55b5968ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f471b3a6740 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55b59693f670 0x55b59693c180 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f471b3a6740 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5968bdde0 0x55b5968cb510 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f471b3a6740 1 --2- 172.21.15.69:0/3468455390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5968bd3f0 0x55b5968cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f471b3a6740 1 -- 172.21.15.69:0/3468455390 >> 172.21.15.69:0/3468455390 conn(0x55b5968b8960 msgr2=0x55b5968b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.524+0000 7f46f7400640 0 client.25208 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:09:33.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.525+0000 7f471b3a6740 1 -- 172.21.15.69:0/3468455390 shutdown_connections 2024-03-19T01:09:33.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.525+0000 7f471b3a6740 1 -- 172.21.15.69:0/3468455390 wait complete. 2024-03-19T01:09:33.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.525+0000 7f471b3a6740 20 client.25208 trim_cache size 0 max 16384 2024-03-19T01:09:33.530 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24580, 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 106.73533520700001, 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 93, 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 147.85178600899999 2024-03-19T01:09:33.531 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:09:33.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 -- 172.21.15.69:0/976882865 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f46f80a2880 msgr2=0x7f46f80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f46f80a2880 0x7f46f80a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55b5968be640 tx=0x7f4708013040 comp rx=0 tx=0).stop 2024-03-19T01:09:33.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 -- 172.21.15.69:0/976882865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4710101840 msgr2=0x7f4710199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4710101840 0x7f4710199dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f470400e9c0 tx=0x7f470400edc0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 -- 172.21.15.69:0/976882865 shutdown_connections 2024-03-19T01:09:33.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f46f80a2880 0x7f46f80a4d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47101033f0 0x7f47101a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4710102a30 0x7f471019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 --2- 172.21.15.69:0/976882865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4710101840 0x7f4710199dc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 -- 172.21.15.69:0/976882865 >> 172.21.15.69:0/976882865 conn(0x7f47100fcdd0 msgr2=0x7f4710073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 -- 172.21.15.69:0/976882865 shutdown_connections 2024-03-19T01:09:33.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.526+0000 7f4718600640 1 -- 172.21.15.69:0/976882865 wait complete. 2024-03-19T01:09:33.545 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-19T01:09:33.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.757+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4226652143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f84101640 msgr2=0x7f7f84101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.757+0000 7f7f8ae00640 1 --2- 172.21.15.69:0/4226652143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f84101640 0x7f7f84101a60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009900 tx=0x7f7f7802bc30 comp rx=0 tx=0).stop 2024-03-19T01:09:33.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.757+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4226652143 shutdown_connections 2024-03-19T01:09:33.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.757+0000 7f7f8ae00640 1 --2- 172.21.15.69:0/4226652143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f841031f0 0x7f7f84109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.757+0000 7f7f8ae00640 1 --2- 172.21.15.69:0/4226652143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f84102830 0x7f7f84102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.757+0000 7f7f8ae00640 1 --2- 172.21.15.69:0/4226652143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f84101640 0x7f7f84101a60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.757+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4226652143 >> 172.21.15.69:0/4226652143 conn(0x7f7f840fcbd0 msgr2=0x7f7f840feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.757+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4226652143 shutdown_connections 2024-03-19T01:09:33.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.757+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4226652143 wait complete. 2024-03-19T01:09:33.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.758+0000 7f7f8ae00640 1 Processor -- start 2024-03-19T01:09:33.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f8ae00640 1 -- start start 2024-03-19T01:09:33.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f8ae00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f84101640 0x7f7f840755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f8ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f84102830 0x7f7f84073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f8ae00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f841031f0 0x7f7f84074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f8ae00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8410d2e0 con 0x7f7f841031f0 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f8ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8410d160 con 0x7f7f84102830 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f8ae00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8410d460 con 0x7f7f84101640 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f89000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f84102830 0x7f7f84073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f89a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f84101640 0x7f7f840755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f89000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f84102830 0x7f7f84073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57440/0 (socket says 172.21.15.69:57440) 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f89000640 1 -- 172.21.15.69:0/4260098228 learned_addr learned my addr 172.21.15.69:0/4260098228 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.759+0000 7f7f8a400640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f841031f0 0x7f7f84074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.760+0000 7f7f89a00640 1 -- 172.21.15.69:0/4260098228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f84102830 msgr2=0x7f7f84073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.760+0000 7f7f89a00640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f84102830 0x7f7f84073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.760+0000 7f7f89a00640 1 -- 172.21.15.69:0/4260098228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f841031f0 msgr2=0x7f7f84074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.760+0000 7f7f89a00640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f841031f0 0x7f7f84074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.760+0000 7f7f89a00640 1 -- 172.21.15.69:0/4260098228 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f840749e0 con 0x7f7f84101640 2024-03-19T01:09:33.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.760+0000 7f7f89a00640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f84101640 0x7f7f840755d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009230 tx=0x7f7f7802c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.760+0000 7f7f82000640 1 -- 172.21.15.69:0/4260098228 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78043020 con 0x7f7f84101640 2024-03-19T01:09:33.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.760+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4260098228 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f841a1a40 con 0x7f7f84101640 2024-03-19T01:09:33.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.760+0000 7f7f82000640 1 -- 172.21.15.69:0/4260098228 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f780054e0 con 0x7f7f84101640 2024-03-19T01:09:33.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.760+0000 7f7f82000640 1 -- 172.21.15.69:0/4260098228 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78003b60 con 0x7f7f84101640 2024-03-19T01:09:33.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.760+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4260098228 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f841a1f60 con 0x7f7f84101640 2024-03-19T01:09:33.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.763+0000 7f7f82000640 1 -- 172.21.15.69:0/4260098228 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f7f78046020 con 0x7f7f84101640 2024-03-19T01:09:33.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.763+0000 7f7f8ed22740 1 Processor -- start 2024-03-19T01:09:33.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f8ed22740 1 -- start start 2024-03-19T01:09:33.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9563765c0 0x55a9563769e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a95637bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a95637c120 0x55a95637e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a9562dad00 con 0x55a9563765c0 2024-03-19T01:09:33.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a956283b00 con 0x55a956376fb0 2024-03-19T01:09:33.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a9562d6da0 con 0x55a95637c120 2024-03-19T01:09:33.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f89a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a95637bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f8a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9563765c0 0x55a9563769e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f89a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a95637bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57442/0 (socket says 172.21.15.69:57442) 2024-03-19T01:09:33.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f89a00640 1 -- 172.21.15.69:0/3889870123 learned_addr learned my addr 172.21.15.69:0/3889870123 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f8a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9563765c0 0x55a9563769e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33668/0 (socket says 172.21.15.69:33668) 2024-03-19T01:09:33.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.764+0000 7f7f89000640 1 --2- 172.21.15.69:0/3889870123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a95637c120 0x55a95637e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.765+0000 7f7f89a00640 1 -- 172.21.15.69:0/3889870123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a95637c120 msgr2=0x55a95637e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.765+0000 7f7f89a00640 1 --2- 172.21.15.69:0/3889870123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a95637c120 0x55a95637e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.765+0000 7f7f89a00640 1 -- 172.21.15.69:0/3889870123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9563765c0 msgr2=0x55a9563769e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.765+0000 7f7f89a00640 1 --2- 172.21.15.69:0/3889870123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9563765c0 0x55a9563769e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.765+0000 7f7f89a00640 1 -- 172.21.15.69:0/3889870123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a955fef2d0 con 0x55a956376fb0 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.765+0000 7f7f89a00640 1 --2- 172.21.15.69:0/3889870123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a95637bab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7f780f4df0 tx=0x7f7f780f4ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca2fa514534253a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.765+0000 7f7f76a00640 1 -- 172.21.15.69:0/3889870123 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f780f9c50 con 0x55a956376fb0 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.765+0000 7f7f76a00640 1 -- 172.21.15.69:0/3889870123 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f780fe070 con 0x55a956376fb0 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.765+0000 7f7f76a00640 1 -- 172.21.15.69:0/3889870123 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f781016a0 con 0x55a956376fb0 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.765+0000 7f7f82000640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7f6c0a2800 0x7f7f6c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.766+0000 7f7f82000640 1 -- 172.21.15.69:0/4260098228 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f7f780f0ee0 con 0x7f7f84101640 2024-03-19T01:09:33.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.766+0000 7f7f8a400640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7f6c0a2800 0x7f7f6c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.766+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3889870123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 msgr2=0x55a95637bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.766+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/3889870123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a95637bab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7f780f4df0 tx=0x7f7f780f4ee0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.766+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3889870123 shutdown_connections 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.766+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/3889870123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a95637c120 0x55a95637e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.766+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/3889870123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a95637bab0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.766+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/3889870123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a9563765c0 0x55a9563769e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.766+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3889870123 >> 172.21.15.69:0/3889870123 conn(0x55a956371b30 msgr2=0x55a956371f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.766+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3889870123 shutdown_connections 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.766+0000 7f7f8a400640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7f6c0a2800 0x7f7f6c0a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55a956377810 tx=0x7f7f7c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.767+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3889870123 wait complete. 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.767+0000 7f7f8ed22740 1 -- start start 2024-03-19T01:09:33.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.767+0000 7f7f8ed22740 1 Processor -- start 2024-03-19T01:09:33.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.767+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9563765c0 0x55a956380850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.767+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a956376fb0 0x55a956380d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a956387dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a95637ee90 con 0x55a95637c120 2024-03-19T01:09:33.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a956296ed0 con 0x55a9563765c0 2024-03-19T01:09:33.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a95637f010 con 0x55a956376fb0 2024-03-19T01:09:33.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f8ed22740 10 client.0 _ll_register_callbacks cb 0x55a956294c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:09:33.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f89a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9563765c0 0x55a956380850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f89000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a956376fb0 0x55a956380d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f89a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9563765c0 0x55a956380850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57446/0 (socket says 172.21.15.69:57446) 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f89a00640 1 -- 172.21.15.69:0/3358293437 learned_addr learned my addr 172.21.15.69:0/3358293437 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f8a400640 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a956387dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f89a00640 1 -- 172.21.15.69:0/3358293437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a956376fb0 msgr2=0x55a956380d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f89a00640 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a956376fb0 0x55a956380d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f89a00640 1 -- 172.21.15.69:0/3358293437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 msgr2=0x55a956387dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f89a00640 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a956387dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f89a00640 1 -- 172.21.15.69:0/3358293437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9560b96b0 con 0x55a9563765c0 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f89a00640 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9563765c0 0x55a956380850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7f78006010 tx=0x7f7f78000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eab857202e046d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.768+0000 7f7f6b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.769+0000 7f7f6b400640 1 -- 172.21.15.69:0/3358293437 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f780b9710 con 0x55a9563765c0 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.769+0000 7f7f6b400640 1 -- 172.21.15.69:0/3358293437 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f7f780ee6e0 con 0x55a9563765c0 2024-03-19T01:09:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.769+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3358293437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f780fe070 con 0x55a9563765c0 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.769+0000 7f7f6b400640 1 -- 172.21.15.69:0/3358293437 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f780bcdd0 con 0x55a9563765c0 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.769+0000 7f7f8ed22740 10 client.25220 fetch_fsmap learned FSMap version 28 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.769+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3358293437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a95637ee90 con 0x55a9563765c0 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.769+0000 7f7f6b400640 1 -- 172.21.15.69:0/3358293437 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f780fe070 con 0x55a9563765c0 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.769+0000 7f7f6b400640 1 -- 172.21.15.69:0/3358293437 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f780b9ae0 con 0x55a9563765c0 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.769+0000 7f7f8ed22740 10 client.25220 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.769+0000 7f7f8ed22740 10 client.25220 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.770+0000 7f7f8ed22740 20 client.25220 populate_metadata read hostname 'smithi069' 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.770+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a95638d510 0x55a95638e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.770+0000 7f7f8ed22740 4 client.25220 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.770+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3358293437 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9562e8240 con 0x55a95638d510 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.770+0000 7f7f89000640 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a95638d510 0x55a95638e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.770+0000 7f7f6b400640 10 client.25220 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.770+0000 7f7f89000640 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a95638d510 0x55a95638e980 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=716a4c6c2dd717cc server_cookie=480b08737884dc62 in_seq=0 out_seq=0 2024-03-19T01:09:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f6b400640 1 -- 172.21.15.69:0/3358293437 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a9562e8240 con 0x55a95638d510 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f6b400640 10 client.25220 handle_command_reply: tid=0 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f8ed22740 1 client.25220 shutdown 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3358293437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9563765c0 msgr2=0x55a956380850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9563765c0 0x55a956380850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7f78006010 tx=0x7f7f78000bc0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3358293437 shutdown_connections 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a95638d510 0x55a95638e980 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a956387dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f6b400640 0 client.25220 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a956376fb0 0x55a956380d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/3358293437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a9563765c0 0x55a956380850 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.771+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3358293437 >> 172.21.15.69:0/3358293437 conn(0x55a956371b30 msgr2=0x55a956370d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.772+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3358293437 shutdown_connections 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.772+0000 7f7f8ed22740 1 -- 172.21.15.69:0/3358293437 wait complete. 2024-03-19T01:09:33.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.772+0000 7f7f8ed22740 20 client.25220 trim_cache size 0 max 16384 2024-03-19T01:09:33.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.801+0000 7f7f8ed22740 1 Processor -- start 2024-03-19T01:09:33.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.801+0000 7f7f8ed22740 1 -- start start 2024-03-19T01:09:33.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.801+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 0x55a956371f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956387d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a9563784f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a95638ccc0 con 0x55a95637c120 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a95638d390 con 0x55a956376fb0 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a95638da60 con 0x55a9563765c0 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f89000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956387d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f89a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 0x55a956371f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f89000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956387d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57458/0 (socket says 172.21.15.69:57458) 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f89000640 1 -- 172.21.15.69:0/382664357 learned_addr learned my addr 172.21.15.69:0/382664357 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f8a400640 1 --2- 172.21.15.69:0/382664357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a9563784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f89000640 1 -- 172.21.15.69:0/382664357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 msgr2=0x55a956371f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f89000640 1 --2- 172.21.15.69:0/382664357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 0x55a956371f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f89000640 1 -- 172.21.15.69:0/382664357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 msgr2=0x55a9563784f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f89000640 1 --2- 172.21.15.69:0/382664357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a9563784f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f89000640 1 -- 172.21.15.69:0/382664357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a956377390 con 0x55a956376fb0 2024-03-19T01:09:33.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f89a00640 1 --2- 172.21.15.69:0/382664357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 0x55a956371f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:33.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.802+0000 7f7f89000640 1 --2- 172.21.15.69:0/382664357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956387d60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55a956372aa0 tx=0x7f7f70005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe94849b970cd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.803+0000 7f7f76000640 1 -- 172.21.15.69:0/382664357 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70003be0 con 0x55a956376fb0 2024-03-19T01:09:33.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.803+0000 7f7f76000640 1 -- 172.21.15.69:0/382664357 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f70003d80 con 0x55a956376fb0 2024-03-19T01:09:33.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.803+0000 7f7f76000640 1 -- 172.21.15.69:0/382664357 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70012b80 con 0x55a956376fb0 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.803+0000 7f7f8ed22740 1 -- 172.21.15.69:0/382664357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 msgr2=0x55a956387d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.803+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/382664357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956387d60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55a956372aa0 tx=0x7f7f70005f90 comp rx=0 tx=0).stop 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.804+0000 7f7f8ed22740 1 -- 172.21.15.69:0/382664357 shutdown_connections 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.804+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/382664357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a9563784f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.804+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/382664357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956387d60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.804+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/382664357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 0x55a956371f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.804+0000 7f7f8ed22740 1 -- 172.21.15.69:0/382664357 >> 172.21.15.69:0/382664357 conn(0x55a956371b30 msgr2=0x55a95637f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.804+0000 7f7f8ed22740 1 -- 172.21.15.69:0/382664357 shutdown_connections 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.804+0000 7f7f8ed22740 1 -- 172.21.15.69:0/382664357 wait complete. 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.804+0000 7f7f8ed22740 1 -- start start 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.804+0000 7f7f8ed22740 1 Processor -- start 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 0x55a956382ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956383410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f8ed22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a956386430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a95638e800 con 0x55a95637c120 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a95638e130 con 0x55a956376fb0 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f8ed22740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a95638eed0 con 0x55a9563765c0 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f8ed22740 10 client.0 _ll_register_callbacks cb 0x7f7f7c0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f89a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 0x55a956382ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f89000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956383410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f89a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 0x55a956382ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46392/0 (socket says 172.21.15.69:46392) 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f89a00640 1 -- 172.21.15.69:0/2164523721 learned_addr learned my addr 172.21.15.69:0/2164523721 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f89000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956383410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57474/0 (socket says 172.21.15.69:57474) 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.805+0000 7f7f8a400640 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a956386430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f89a00640 1 -- 172.21.15.69:0/2164523721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 msgr2=0x55a956383410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f89a00640 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956383410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f89a00640 1 -- 172.21.15.69:0/2164523721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 msgr2=0x55a956386430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f89a00640 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a956386430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f89a00640 1 -- 172.21.15.69:0/2164523721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a956386a20 con 0x55a9563765c0 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f8a400640 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a956386430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f89000640 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956383410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f89a00640 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 0x55a956382ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7f780b98d0 tx=0x7f7f78005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd74c36823d5115 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f6b400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f6b400640 1 -- 172.21.15.69:0/2164523721 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f780bcfb0 con 0x55a9563765c0 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f6b400640 1 -- 172.21.15.69:0/2164523721 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f7f780ee650 con 0x55a9563765c0 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.806+0000 7f7f8ed22740 1 -- 172.21.15.69:0/2164523721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f70003d80 con 0x55a9563765c0 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.807+0000 7f7f6b400640 1 -- 172.21.15.69:0/2164523721 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f780bd240 con 0x55a9563765c0 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.807+0000 7f7f8ed22740 10 client.25231 fetch_fsmap learned FSMap version 28 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.807+0000 7f7f8ed22740 1 -- 172.21.15.69:0/2164523721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a956386a20 con 0x55a9563765c0 2024-03-19T01:09:33.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.807+0000 7f7f6b400640 1 -- 172.21.15.69:0/2164523721 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f70003d80 con 0x55a9563765c0 2024-03-19T01:09:33.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.807+0000 7f7f6b400640 1 -- 172.21.15.69:0/2164523721 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f780d61a0 con 0x55a9563765c0 2024-03-19T01:09:33.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.807+0000 7f7f8ed22740 10 client.25231 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:09:33.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.807+0000 7f7f8ed22740 10 client.25231 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T01:09:33.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.807+0000 7f7f8ed22740 20 client.25231 populate_metadata read hostname 'smithi069' 2024-03-19T01:09:33.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.807+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a9563f74a0 0x55a956387dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:33.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.807+0000 7f7f8ed22740 4 client.25231 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:09:33.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.807+0000 7f7f8ed22740 1 -- 172.21.15.69:0/2164523721 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "status"}) -- 0x55a9562e2590 con 0x55a9563f74a0 2024-03-19T01:09:33.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.808+0000 7f7f89000640 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a9563f74a0 0x55a956387dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:33.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.808+0000 7f7f89000640 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a9563f74a0 0x55a956387dd0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc3c7f095ea37ed server_cookie=d7745006cc9831c in_seq=0 out_seq=0 2024-03-19T01:09:33.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.808+0000 7f7f6b400640 10 client.25231 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:09:33.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.808+0000 7f7f6b400640 1 -- 172.21.15.69:0/2164523721 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a9562e2590 con 0x55a9563f74a0 2024-03-19T01:09:33.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.808+0000 7f7f6b400640 10 client.25231 handle_command_reply: tid=0 2024-03-19T01:09:33.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.808+0000 7f7f8ed22740 1 client.25231 shutdown 2024-03-19T01:09:33.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f8ed22740 1 -- 172.21.15.69:0/2164523721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 msgr2=0x55a956382ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 0x55a956382ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7f780b98d0 tx=0x7f7f78005bd0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f8ed22740 1 -- 172.21.15.69:0/2164523721 shutdown_connections 2024-03-19T01:09:33.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a9563f74a0 0x55a956387dd0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a95637c120 0x55a956386430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a956376fb0 0x55a956383410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f8ed22740 1 --2- 172.21.15.69:0/2164523721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a9563765c0 0x55a956382ed0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f6b400640 0 client.25231 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:09:33.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f8ed22740 1 -- 172.21.15.69:0/2164523721 >> 172.21.15.69:0/2164523721 conn(0x55a956371b30 msgr2=0x55a956371f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f8ed22740 1 -- 172.21.15.69:0/2164523721 shutdown_connections 2024-03-19T01:09:33.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f8ed22740 1 -- 172.21.15.69:0/2164523721 wait complete. 2024-03-19T01:09:33.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.809+0000 7f7f8ed22740 20 client.25231 trim_cache size 0 max 16384 2024-03-19T01:09:33.814 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24581, 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 138.191284376, 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 93, 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 145.27336146299999 2024-03-19T01:09:33.815 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:09:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4260098228 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7f6c0a2800 msgr2=0x7f7f6c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7f6c0a2800 0x7f7f6c0a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55a956377810 tx=0x7f7f7c009500 comp rx=0 tx=0).stop 2024-03-19T01:09:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4260098228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f84101640 msgr2=0x7f7f840755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f84101640 0x7f7f840755d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009230 tx=0x7f7f7802c5d0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4260098228 shutdown_connections 2024-03-19T01:09:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7f6c0a2800 0x7f7f6c0a4cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f841031f0 0x7f7f84074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f84102830 0x7f7f84073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 --2- 172.21.15.69:0/4260098228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f84101640 0x7f7f840755d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:33.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4260098228 >> 172.21.15.69:0/4260098228 conn(0x7f7f840fcbd0 msgr2=0x7f7f84105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:33.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4260098228 shutdown_connections 2024-03-19T01:09:33.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:33.810+0000 7f7f8ae00640 1 -- 172.21.15.69:0/4260098228 wait complete. 2024-03-19T01:09:33.830 INFO:tasks.cephadm:Setup complete, yielding 2024-03-19T01:09:33.830 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-19T01:09:33.841 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-03-19T01:09:33.841 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-19T01:09:34.333 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:09:35.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:34 smithi086 ceph-mon[66829]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-19T01:09:35.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:34 smithi069 ceph-mon[31727]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-19T01:09:35.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:34 smithi163 ceph-mon[67428]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-19T01:09:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.548+0000 7f7a3d400700 1 -- 172.21.15.69:0/650715580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a38103550 msgr2=0x7f7a381039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.548+0000 7f7a3d400700 1 --2- 172.21.15.69:0/650715580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a38103550 0x7f7a381039b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009990 tx=0x7f7a24009ca0 comp rx=0 tx=0).stop 2024-03-19T01:09:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.548+0000 7f7a3d400700 1 -- 172.21.15.69:0/650715580 shutdown_connections 2024-03-19T01:09:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.548+0000 7f7a3d400700 1 --2- 172.21.15.69:0/650715580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a38103ef0 0x7f7a3810a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.548+0000 7f7a3d400700 1 --2- 172.21.15.69:0/650715580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a38102350 0x7f7a38102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.548+0000 7f7a3d400700 1 --2- 172.21.15.69:0/650715580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a38103550 0x7f7a381039b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.548+0000 7f7a3d400700 1 -- 172.21.15.69:0/650715580 >> 172.21.15.69:0/650715580 conn(0x7f7a380fd8b0 msgr2=0x7f7a380ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.548+0000 7f7a3d400700 1 -- 172.21.15.69:0/650715580 shutdown_connections 2024-03-19T01:09:36.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.548+0000 7f7a3d400700 1 -- 172.21.15.69:0/650715580 wait complete. 2024-03-19T01:09:36.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.549+0000 7f7a3d400700 1 Processor -- start 2024-03-19T01:09:36.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.550+0000 7f7a3d400700 1 -- start start 2024-03-19T01:09:36.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.550+0000 7f7a3d400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a38102350 0x7f7a381985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:36.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.550+0000 7f7a3d400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a38103550 0x7f7a38198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:36.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.550+0000 7f7a3d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a38103ef0 0x7f7a3819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:36.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.550+0000 7f7a3d400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810df60 con 0x7f7a38103550 2024-03-19T01:09:36.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.550+0000 7f7a3d400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810dde0 con 0x7f7a38103ef0 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.550+0000 7f7a3d400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810e0e0 con 0x7f7a38102350 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.550+0000 7f7a36800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a38102350 0x7f7a381985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a36800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a38102350 0x7f7a381985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50988/0 (socket says 172.21.15.69:50988) 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a36800700 1 -- 172.21.15.69:0/1562557640 learned_addr learned my addr 172.21.15.69:0/1562557640 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a37200700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a38103ef0 0x7f7a3819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a35e00700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a38103550 0x7f7a38198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a36800700 1 -- 172.21.15.69:0/1562557640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a38103ef0 msgr2=0x7f7a3819fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a36800700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a38103ef0 0x7f7a3819fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a36800700 1 -- 172.21.15.69:0/1562557640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a38103550 msgr2=0x7f7a38198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a36800700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a38103550 0x7f7a38198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a36800700 1 -- 172.21.15.69:0/1562557640 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a381a0200 con 0x7f7a38102350 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a35e00700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a38103550 0x7f7a38198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a37200700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a38103ef0 0x7f7a3819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:36.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.551+0000 7f7a36800700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a38102350 0x7f7a381985a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c170 tx=0x7f7a2800c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:36.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.552+0000 7f7a2f400700 1 -- 172.21.15.69:0/1562557640 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2800cdb0 con 0x7f7a38102350 2024-03-19T01:09:36.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.552+0000 7f7a2f400700 1 -- 172.21.15.69:0/1562557640 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a28018070 con 0x7f7a38102350 2024-03-19T01:09:36.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.552+0000 7f7a3d400700 1 -- 172.21.15.69:0/1562557640 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a38074540 con 0x7f7a38102350 2024-03-19T01:09:36.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.552+0000 7f7a2f400700 1 -- 172.21.15.69:0/1562557640 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28013780 con 0x7f7a38102350 2024-03-19T01:09:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.552+0000 7f7a3d400700 1 -- 172.21.15.69:0/1562557640 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a38074ad0 con 0x7f7a38102350 2024-03-19T01:09:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.553+0000 7f7a3d400700 1 -- 172.21.15.69:0/1562557640 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a38068fd0 con 0x7f7a38102350 2024-03-19T01:09:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.555+0000 7f7a2f400700 1 -- 172.21.15.69:0/1562557640 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f7a28012070 con 0x7f7a38102350 2024-03-19T01:09:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.557+0000 7f7a2f400700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a1c0a2910 0x7f7a1c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.557+0000 7f7a2f400700 1 -- 172.21.15.69:0/1562557640 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f7a280ce9e0 con 0x7f7a38102350 2024-03-19T01:09:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.557+0000 7f7a35e00700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a1c0a2910 0x7f7a1c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:36.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.558+0000 7f7a35e00700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a1c0a2910 0x7f7a1c0a4dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400b2e0 tx=0x7f7a2401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:36.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.559+0000 7f7a2f400700 1 -- 172.21.15.69:0/1562557640 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7a28022020 con 0x7f7a38102350 2024-03-19T01:09:36.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.875+0000 7f7a3d400700 1 -- 172.21.15.69:0/1562557640 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7a38075f20 con 0x7f7a1c0a2910 2024-03-19T01:09:36.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.876+0000 7f7a2f400700 1 -- 172.21.15.69:0/1562557640 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7a38075f20 con 0x7f7a1c0a2910 2024-03-19T01:09:36.877 INFO:teuthology.orchestra.run.smithi069.stdout:Backend: cephadm 2024-03-19T01:09:36.877 INFO:teuthology.orchestra.run.smithi069.stdout:Available: Yes 2024-03-19T01:09:36.878 INFO:teuthology.orchestra.run.smithi069.stdout:Paused: No 2024-03-19T01:09:36.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.880+0000 7f7a3d400700 1 -- 172.21.15.69:0/1562557640 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a1c0a2910 msgr2=0x7f7a1c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:36.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.880+0000 7f7a3d400700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a1c0a2910 0x7f7a1c0a4dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400b2e0 tx=0x7f7a2401a040 comp rx=0 tx=0).stop 2024-03-19T01:09:36.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.880+0000 7f7a3d400700 1 -- 172.21.15.69:0/1562557640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a38102350 msgr2=0x7f7a381985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:36.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.880+0000 7f7a3d400700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a38102350 0x7f7a381985a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c170 tx=0x7f7a2800c480 comp rx=0 tx=0).stop 2024-03-19T01:09:36.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.880+0000 7f7a3d400700 1 -- 172.21.15.69:0/1562557640 shutdown_connections 2024-03-19T01:09:36.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.880+0000 7f7a3d400700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7a1c0a2910 0x7f7a1c0a4dd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:36.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.880+0000 7f7a3d400700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a38103ef0 0x7f7a3819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:36.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.880+0000 7f7a3d400700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a38102350 0x7f7a381985a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:36.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.880+0000 7f7a3d400700 1 --2- 172.21.15.69:0/1562557640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a38103550 0x7f7a38198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:36.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.880+0000 7f7a3d400700 1 -- 172.21.15.69:0/1562557640 >> 172.21.15.69:0/1562557640 conn(0x7f7a380fd8b0 msgr2=0x7f7a38106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:36.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.880+0000 7f7a3d400700 1 -- 172.21.15.69:0/1562557640 shutdown_connections 2024-03-19T01:09:36.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:36.881+0000 7f7a3d400700 1 -- 172.21.15.69:0/1562557640 wait complete. 2024-03-19T01:09:37.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:36 smithi086 ceph-mon[66829]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-19T01:09:37.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:36 smithi069 ceph-mon[31727]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-19T01:09:37.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:36 smithi163 ceph-mon[67428]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-19T01:09:37.862 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-19T01:09:38.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:37 smithi086 ceph-mon[66829]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:38.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:37 smithi069 ceph-mon[31727]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:38.307 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:09:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:37 smithi163 ceph-mon[67428]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:39.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:38 smithi069 ceph-mon[31727]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 op/s 2024-03-19T01:09:39.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:38 smithi086 ceph-mon[66829]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 op/s 2024-03-19T01:09:39.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:38 smithi163 ceph-mon[67428]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 op/s 2024-03-19T01:09:40.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.570+0000 7f4408800700 1 -- 172.21.15.69:0/1968445087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4400108070 msgr2=0x7f440010a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:40.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.570+0000 7f4408800700 1 --2- 172.21.15.69:0/1968445087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4400108070 0x7f440010a4a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00b0a0 tx=0x7f43fc00b3b0 comp rx=0 tx=0).stop 2024-03-19T01:09:40.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.570+0000 7f4408800700 1 -- 172.21.15.69:0/1968445087 shutdown_connections 2024-03-19T01:09:40.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.570+0000 7f4408800700 1 --2- 172.21.15.69:0/1968445087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4400108070 0x7f440010a4a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:40.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.570+0000 7f4408800700 1 --2- 172.21.15.69:0/1968445087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4400102e10 0x7f4400105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:40.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.570+0000 7f4408800700 1 --2- 172.21.15.69:0/1968445087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4400105740 0x7f4400107b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:40.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.570+0000 7f4408800700 1 -- 172.21.15.69:0/1968445087 >> 172.21.15.69:0/1968445087 conn(0x7f44000fc7e0 msgr2=0x7f44000fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:40.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.570+0000 7f4408800700 1 -- 172.21.15.69:0/1968445087 shutdown_connections 2024-03-19T01:09:40.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.570+0000 7f4408800700 1 -- 172.21.15.69:0/1968445087 wait complete. 2024-03-19T01:09:40.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.572+0000 7f4408800700 1 Processor -- start 2024-03-19T01:09:40.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.572+0000 7f4408800700 1 -- start start 2024-03-19T01:09:40.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.572+0000 7f4408800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4400102e10 0x7f44001985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:40.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.572+0000 7f4408800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4400105740 0x7f4400198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:40.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.572+0000 7f4408800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4400108070 0x7f440019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:40.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4408800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f440010df60 con 0x7f4400105740 2024-03-19T01:09:40.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4408800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f440010dde0 con 0x7f4400102e10 2024-03-19T01:09:40.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4408800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f440010e0e0 con 0x7f4400108070 2024-03-19T01:09:40.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4405e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4400102e10 0x7f44001985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:40.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4405e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4400102e10 0x7f44001985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40506/0 (socket says 172.21.15.69:40506) 2024-03-19T01:09:40.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4406800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4400108070 0x7f440019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:40.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4405e00700 1 -- 172.21.15.69:0/1973643240 learned_addr learned my addr 172.21.15.69:0/1973643240 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:40.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4405400700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4400105740 0x7f4400198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:40.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4405e00700 1 -- 172.21.15.69:0/1973643240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4400108070 msgr2=0x7f440019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:40.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4405e00700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4400108070 0x7f440019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:40.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4405e00700 1 -- 172.21.15.69:0/1973643240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4400105740 msgr2=0x7f4400198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:40.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4405e00700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4400105740 0x7f4400198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:40.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4405e00700 1 -- 172.21.15.69:0/1973643240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44001a0240 con 0x7f4400102e10 2024-03-19T01:09:40.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.573+0000 7f4406800700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4400108070 0x7f440019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:40.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.574+0000 7f4405e00700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4400102e10 0x7f44001985e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43f400e9c0 tx=0x7f43f400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:40.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.574+0000 7f43faa00700 1 -- 172.21.15.69:0/1973643240 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f400cbd0 con 0x7f4400102e10 2024-03-19T01:09:40.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.574+0000 7f43faa00700 1 -- 172.21.15.69:0/1973643240 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43f400cd70 con 0x7f4400102e10 2024-03-19T01:09:40.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.574+0000 7f4408800700 1 -- 172.21.15.69:0/1973643240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44001a0530 con 0x7f4400102e10 2024-03-19T01:09:40.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.574+0000 7f43faa00700 1 -- 172.21.15.69:0/1973643240 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4010430 con 0x7f4400102e10 2024-03-19T01:09:40.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.574+0000 7f4408800700 1 -- 172.21.15.69:0/1973643240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44001a0a70 con 0x7f4400102e10 2024-03-19T01:09:40.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.575+0000 7f4408800700 1 -- 172.21.15.69:0/1973643240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4400068fd0 con 0x7f4400102e10 2024-03-19T01:09:40.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.577+0000 7f43faa00700 1 -- 172.21.15.69:0/1973643240 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f43f4004750 con 0x7f4400102e10 2024-03-19T01:09:40.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.578+0000 7f43faa00700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f43e80a2910 0x7f43e80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:40.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.579+0000 7f43faa00700 1 -- 172.21.15.69:0/1973643240 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f43f40cefe0 con 0x7f4400102e10 2024-03-19T01:09:40.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.579+0000 7f4405400700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f43e80a2910 0x7f43e80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:40.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.579+0000 7f4405400700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f43e80a2910 0x7f43e80a4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009c60 tx=0x7f43f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:40.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.582+0000 7f43faa00700 1 -- 172.21.15.69:0/1973643240 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f43f409bcd0 con 0x7f4400102e10 2024-03-19T01:09:40.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.837+0000 7f4408800700 1 -- 172.21.15.69:0/1973643240 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4400062750 con 0x7f43e80a2910 2024-03-19T01:09:40.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.869+0000 7f43faa00700 1 -- 172.21.15.69:0/1973643240 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f4400062750 con 0x7f43e80a2910 2024-03-19T01:09:40.871 INFO:teuthology.orchestra.run.smithi069.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-19T01:09:40.871 INFO:teuthology.orchestra.run.smithi069.stdout:mds.a smithi069 running (2m) 35s ago 2m 15.9M - 19.0.0-2206-gb6ae2491 2312c8e42d00 e20f4f9f69ca 2024-03-19T01:09:40.871 INFO:teuthology.orchestra.run.smithi069.stdout:mds.b smithi086 running (2m) 84s ago 2m 25.5M - 19.0.0-2206-gb6ae2491 2312c8e42d00 f8a2b62ff9d5 2024-03-19T01:09:40.871 INFO:teuthology.orchestra.run.smithi069.stdout:mds.c smithi163 running (2m) 84s ago 2m 19.6M - 19.0.0-2206-gb6ae2491 2312c8e42d00 0c95bfbc9d13 2024-03-19T01:09:40.871 INFO:teuthology.orchestra.run.smithi069.stdout:mds.d smithi069 running (2m) 35s ago 2m 15.5M - 19.0.0-2206-gb6ae2491 2312c8e42d00 c66cfdc41c18 2024-03-19T01:09:40.871 INFO:teuthology.orchestra.run.smithi069.stdout:mds.e smithi086 running (2m) 84s ago 2m 18.2M - 19.0.0-2206-gb6ae2491 2312c8e42d00 26cfff715229 2024-03-19T01:09:40.871 INFO:teuthology.orchestra.run.smithi069.stdout:mds.f smithi163 running (2m) 84s ago 2m 18.0M - 19.0.0-2206-gb6ae2491 2312c8e42d00 274807e936eb 2024-03-19T01:09:40.871 INFO:teuthology.orchestra.run.smithi069.stdout:mds.g smithi069 running (2m) 35s ago 2m 12.4M - 19.0.0-2206-gb6ae2491 2312c8e42d00 7adea5bd5206 2024-03-19T01:09:40.871 INFO:teuthology.orchestra.run.smithi069.stdout:mds.h smithi086 running (2m) 84s ago 2m 18.2M - 19.0.0-2206-gb6ae2491 2312c8e42d00 ed84330be684 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:mds.i smithi163 running (2m) 84s ago 2m 19.2M - 19.0.0-2206-gb6ae2491 2312c8e42d00 f72b47305772 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:mds.j smithi069 running (3m) 35s ago 3m 12.5M - 19.0.0-2206-gb6ae2491 2312c8e42d00 65388de9d2b5 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:mds.k smithi086 running (3m) 84s ago 3m 17.2M - 19.0.0-2206-gb6ae2491 2312c8e42d00 d18bf0781d2c 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:mds.l smithi163 running (3m) 84s ago 3m 17.3M - 19.0.0-2206-gb6ae2491 2312c8e42d00 d46aed8c7f24 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.x smithi069 *:9283,8765 running (11m) 35s ago 11m 514M - 19.0.0-2206-gb6ae2491 2312c8e42d00 1507fbc37617 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.y smithi086 *:8443,8765 running (8m) 84s ago 8m 453M - 19.0.0-2206-gb6ae2491 2312c8e42d00 1d9b5a97ff38 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.z smithi163 *:8443,8765 running (8m) 84s ago 8m 454M - 19.0.0-2206-gb6ae2491 2312c8e42d00 52de88390005 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:mon.a smithi069 running (11m) 35s ago 11m 47.9M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 d1aee4c67e5b 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:mon.b smithi086 running (9m) 84s ago 9m 43.7M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 e818e2c8a5e8 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:mon.c smithi163 running (9m) 84s ago 9m 42.7M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 5e9f2948f746 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 smithi069 running (8m) 35s ago 8m 45.0M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 29c8d2dbdd4f 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 smithi086 running (8m) 84s ago 8m 49.2M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 514b793e949d 2024-03-19T01:09:40.872 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 smithi163 running (7m) 84s ago 7m 57.5M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 742f2eea0dc4 2024-03-19T01:09:40.873 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 smithi069 running (7m) 35s ago 7m 53.2M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 9a9438706daf 2024-03-19T01:09:40.873 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 smithi086 running (6m) 84s ago 6m 54.3M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 f90667657001 2024-03-19T01:09:40.873 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 smithi163 running (6m) 84s ago 6m 55.5M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 9f818c46f6d4 2024-03-19T01:09:40.873 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 smithi069 running (5m) 35s ago 5m 51.9M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 3ed75fe455dc 2024-03-19T01:09:40.873 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 smithi086 running (5m) 84s ago 5m 45.0M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 4133f42b8377 2024-03-19T01:09:40.873 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 smithi163 running (4m) 84s ago 4m 57.9M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 3cdcb8e9565b 2024-03-19T01:09:40.873 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 smithi069 running (4m) 35s ago 4m 53.0M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 d1cbed7bce70 2024-03-19T01:09:40.873 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 smithi086 running (3m) 84s ago 3m 57.1M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 6a1ba4d29e85 2024-03-19T01:09:40.873 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 smithi163 running (3m) 84s ago 3m 43.8M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 507b831e9fe1 2024-03-19T01:09:40.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 -- 172.21.15.69:0/1973643240 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f43e80a2910 msgr2=0x7f43e80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:40.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f43e80a2910 0x7f43e80a4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009c60 tx=0x7f43f0009260 comp rx=0 tx=0).stop 2024-03-19T01:09:40.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 -- 172.21.15.69:0/1973643240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4400102e10 msgr2=0x7f44001985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:40.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4400102e10 0x7f44001985e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43f400e9c0 tx=0x7f43f400ecd0 comp rx=0 tx=0).stop 2024-03-19T01:09:40.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 -- 172.21.15.69:0/1973643240 shutdown_connections 2024-03-19T01:09:40.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f43e80a2910 0x7f43e80a4dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:40.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4400108070 0x7f440019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:40.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4400102e10 0x7f44001985e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:40.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 --2- 172.21.15.69:0/1973643240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4400105740 0x7f4400198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:40.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 -- 172.21.15.69:0/1973643240 >> 172.21.15.69:0/1973643240 conn(0x7f44000fc7e0 msgr2=0x7f44001037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:40.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 -- 172.21.15.69:0/1973643240 shutdown_connections 2024-03-19T01:09:40.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:40.873+0000 7f4408800700 1 -- 172.21.15.69:0/1973643240 wait complete. 2024-03-19T01:09:41.112 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:40 smithi163 ceph-mon[67428]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:09:41.237 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:40 smithi069 ceph-mon[31727]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:09:41.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:40 smithi086 ceph-mon[66829]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:09:41.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-19T01:09:41.854 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:41 smithi069 ceph-mon[31727]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:42.119 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:09:42.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:41 smithi086 ceph-mon[66829]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:42.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:41 smithi163 ceph-mon[67428]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:43.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:42 smithi069 ceph-mon[31727]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:43.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:42 smithi086 ceph-mon[66829]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:43.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:42 smithi163 ceph-mon[67428]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:44.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.261+0000 7fc168e00700 1 -- 172.21.15.69:0/3348182885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164103e20 msgr2=0x7fc16410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:44.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.261+0000 7fc168e00700 1 --2- 172.21.15.69:0/3348182885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164103e20 0x7fc16410a6d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00b0d0 tx=0x7fc15c00b3e0 comp rx=0 tx=0).stop 2024-03-19T01:09:44.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.261+0000 7fc168e00700 1 -- 172.21.15.69:0/3348182885 shutdown_connections 2024-03-19T01:09:44.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.261+0000 7fc168e00700 1 --2- 172.21.15.69:0/3348182885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164103e20 0x7fc16410a6d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:44.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.261+0000 7fc168e00700 1 --2- 172.21.15.69:0/3348182885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc164102280 0x7fc1641026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:44.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.261+0000 7fc168e00700 1 --2- 172.21.15.69:0/3348182885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc164103480 0x7fc1641038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:44.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.261+0000 7fc168e00700 1 -- 172.21.15.69:0/3348182885 >> 172.21.15.69:0/3348182885 conn(0x7fc1640fd820 msgr2=0x7fc1640ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:44.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.261+0000 7fc168e00700 1 -- 172.21.15.69:0/3348182885 shutdown_connections 2024-03-19T01:09:44.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.261+0000 7fc168e00700 1 -- 172.21.15.69:0/3348182885 wait complete. 2024-03-19T01:09:44.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.263+0000 7fc168e00700 1 Processor -- start 2024-03-19T01:09:44.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.263+0000 7fc168e00700 1 -- start start 2024-03-19T01:09:44.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.263+0000 7fc168e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc164102280 0x7fc1640760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:44.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.263+0000 7fc168e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc164103480 0x7fc164074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:44.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc168e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164103e20 0x7fc164074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc168e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc16410de60 con 0x7fc164102280 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc168e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc16410dce0 con 0x7fc164103e20 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc168e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc16410dfe0 con 0x7fc164103480 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc162a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc164103480 0x7fc164074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc162a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc164103480 0x7fc164074700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:51042/0 (socket says 172.21.15.69:51042) 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc162a00700 1 -- 172.21.15.69:0/2693462152 learned_addr learned my addr 172.21.15.69:0/2693462152 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc163400700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc164102280 0x7fc1640760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc163e00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164103e20 0x7fc164074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc162a00700 1 -- 172.21.15.69:0/2693462152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164103e20 msgr2=0x7fc164074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc162a00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164103e20 0x7fc164074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc162a00700 1 -- 172.21.15.69:0/2693462152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc164102280 msgr2=0x7fc1640760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc162a00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc164102280 0x7fc1640760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc162a00700 1 -- 172.21.15.69:0/2693462152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc164075180 con 0x7fc164103480 2024-03-19T01:09:44.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.264+0000 7fc163e00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164103e20 0x7fc164074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:44.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.265+0000 7fc162a00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc164103480 0x7fc164074700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc15000e9c0 tx=0x7fc15000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:44.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.265+0000 7fc15be00700 1 -- 172.21.15.69:0/2693462152 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15000cbd0 con 0x7fc164103480 2024-03-19T01:09:44.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.265+0000 7fc15be00700 1 -- 172.21.15.69:0/2693462152 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc15000cd70 con 0x7fc164103480 2024-03-19T01:09:44.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.265+0000 7fc168e00700 1 -- 172.21.15.69:0/2693462152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc164075470 con 0x7fc164103480 2024-03-19T01:09:44.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.265+0000 7fc15be00700 1 -- 172.21.15.69:0/2693462152 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150010430 con 0x7fc164103480 2024-03-19T01:09:44.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.265+0000 7fc168e00700 1 -- 172.21.15.69:0/2693462152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1641a6e70 con 0x7fc164103480 2024-03-19T01:09:44.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.266+0000 7fc168e00700 1 -- 172.21.15.69:0/2693462152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc164068fd0 con 0x7fc164103480 2024-03-19T01:09:44.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.268+0000 7fc15be00700 1 -- 172.21.15.69:0/2693462152 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fc150004750 con 0x7fc164103480 2024-03-19T01:09:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.271+0000 7fc15be00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1480a2910 0x7fc1480a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:44.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.271+0000 7fc15be00700 1 -- 172.21.15.69:0/2693462152 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fc1500cf1d0 con 0x7fc164103480 2024-03-19T01:09:44.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.271+0000 7fc163400700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1480a2910 0x7fc1480a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:44.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.272+0000 7fc163400700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1480a2910 0x7fc1480a4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1641032e0 tx=0x7fc154009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:44.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.277+0000 7fc15be00700 1 -- 172.21.15.69:0/2693462152 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc15009be60 con 0x7fc164103480 2024-03-19T01:09:44.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.614+0000 7fc168e00700 1 -- 172.21.15.69:0/2693462152 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fc164062750 con 0x7fc1480a2910 2024-03-19T01:09:44.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.622+0000 7fc15be00700 1 -- 172.21.15.69:0/2693462152 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fc164062750 con 0x7fc1480a2910 2024-03-19T01:09:44.623 INFO:teuthology.orchestra.run.smithi069.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-19T01:09:44.623 INFO:teuthology.orchestra.run.smithi069.stdout:mds.all 12/12 88s ago 3m smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2024-03-19T01:09:44.624 INFO:teuthology.orchestra.run.smithi069.stdout:mgr 3/3 88s ago 9m smithi069=x;smithi086=y;smithi163=z;count:3 2024-03-19T01:09:44.624 INFO:teuthology.orchestra.run.smithi069.stdout:mon 3/3 88s ago 9m smithi069:172.21.15.69=a;smithi086:172.21.15.86=b;smithi163:172.21.15.163=c;count:3 2024-03-19T01:09:44.624 INFO:teuthology.orchestra.run.smithi069.stdout:osd 12 88s ago - 2024-03-19T01:09:44.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.625+0000 7fc168e00700 1 -- 172.21.15.69:0/2693462152 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1480a2910 msgr2=0x7fc1480a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:44.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.625+0000 7fc168e00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1480a2910 0x7fc1480a4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1641032e0 tx=0x7fc154009260 comp rx=0 tx=0).stop 2024-03-19T01:09:44.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.625+0000 7fc168e00700 1 -- 172.21.15.69:0/2693462152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc164103480 msgr2=0x7fc164074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:44.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.625+0000 7fc168e00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc164103480 0x7fc164074700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc15000e9c0 tx=0x7fc15000ecd0 comp rx=0 tx=0).stop 2024-03-19T01:09:44.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.625+0000 7fc168e00700 1 -- 172.21.15.69:0/2693462152 shutdown_connections 2024-03-19T01:09:44.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.625+0000 7fc168e00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc1480a2910 0x7fc1480a4dd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:44.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.625+0000 7fc168e00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164103e20 0x7fc164074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:44.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.625+0000 7fc168e00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc164102280 0x7fc1640760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:44.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.625+0000 7fc168e00700 1 --2- 172.21.15.69:0/2693462152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc164103480 0x7fc164074700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:44.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.625+0000 7fc168e00700 1 -- 172.21.15.69:0/2693462152 >> 172.21.15.69:0/2693462152 conn(0x7fc1640fd820 msgr2=0x7fc164106720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:44.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.626+0000 7fc168e00700 1 -- 172.21.15.69:0/2693462152 shutdown_connections 2024-03-19T01:09:44.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:44.626+0000 7fc168e00700 1 -- 172.21.15.69:0/2693462152 wait complete. 2024-03-19T01:09:44.988 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:44 smithi069 ceph-mon[31727]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:45.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:44 smithi086 ceph-mon[66829]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:45.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:44 smithi163 ceph-mon[67428]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:45.406 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-19T01:09:45.846 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:09:46.157 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:45 smithi086 ceph-mon[66829]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:46.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:45 smithi069 ceph-mon[31727]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:46.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:45 smithi163 ceph-mon[67428]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:47.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:46 smithi069 ceph-mon[31727]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:47.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:46 smithi086 ceph-mon[66829]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:47.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:46 smithi163 ceph-mon[67428]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.109+0000 7f9526400700 1 -- 172.21.15.69:0/2611225207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9520103f30 msgr2=0x7f952010a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.109+0000 7f9526400700 1 --2- 172.21.15.69:0/2611225207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9520103f30 0x7f952010a7e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9518009990 tx=0x7f9518009ca0 comp rx=0 tx=0).stop 2024-03-19T01:09:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.109+0000 7f9526400700 1 -- 172.21.15.69:0/2611225207 shutdown_connections 2024-03-19T01:09:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.109+0000 7f9526400700 1 --2- 172.21.15.69:0/2611225207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9520103f30 0x7f952010a7e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.109+0000 7f9526400700 1 --2- 172.21.15.69:0/2611225207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9520102390 0x7f95201027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:48.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.109+0000 7f9526400700 1 --2- 172.21.15.69:0/2611225207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9520103590 0x7f95201039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:48.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.109+0000 7f9526400700 1 -- 172.21.15.69:0/2611225207 >> 172.21.15.69:0/2611225207 conn(0x7f95200fd930 msgr2=0x7f95200ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:48.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.109+0000 7f9526400700 1 -- 172.21.15.69:0/2611225207 shutdown_connections 2024-03-19T01:09:48.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.109+0000 7f9526400700 1 -- 172.21.15.69:0/2611225207 wait complete. 2024-03-19T01:09:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f9526400700 1 Processor -- start 2024-03-19T01:09:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f9526400700 1 -- start start 2024-03-19T01:09:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f9526400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9520102390 0x7f952019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f9526400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9520103590 0x7f952019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f9526400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9520103f30 0x7f95201a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f9526400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f952010dfa0 con 0x7f9520102390 2024-03-19T01:09:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f9526400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f952010de20 con 0x7f9520103f30 2024-03-19T01:09:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f9526400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f952010e120 con 0x7f9520103590 2024-03-19T01:09:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f9517e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9520103590 0x7f952019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f951f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9520102390 0x7f952019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f951fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9520103f30 0x7f95201a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f9517e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9520103590 0x7f952019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58446/0 (socket says 172.21.15.69:58446) 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.112+0000 7f9517e00700 1 -- 172.21.15.69:0/4153169180 learned_addr learned my addr 172.21.15.69:0/4153169180 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.111+0000 7f951f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9520102390 0x7f952019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50650/0 (socket says 172.21.15.69:50650) 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.112+0000 7f9517e00700 1 -- 172.21.15.69:0/4153169180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9520103f30 msgr2=0x7f95201a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.112+0000 7f9517e00700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9520103f30 0x7f95201a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.112+0000 7f9517e00700 1 -- 172.21.15.69:0/4153169180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9520102390 msgr2=0x7f952019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.112+0000 7f9517e00700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9520102390 0x7f952019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.112+0000 7f9517e00700 1 -- 172.21.15.69:0/4153169180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95201a2370 con 0x7f9520103590 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.112+0000 7f951fe00700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9520103f30 0x7f95201a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.112+0000 7f9517e00700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9520103590 0x7f952019ac50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f950800e9d0 tx=0x7f950800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.113+0000 7f951cc00700 1 -- 172.21.15.69:0/4153169180 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950800cbd0 con 0x7f9520103590 2024-03-19T01:09:48.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.113+0000 7f951cc00700 1 -- 172.21.15.69:0/4153169180 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f950800ee70 con 0x7f9520103590 2024-03-19T01:09:48.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.113+0000 7f9526400700 1 -- 172.21.15.69:0/4153169180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95201a2660 con 0x7f9520103590 2024-03-19T01:09:48.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.113+0000 7f951cc00700 1 -- 172.21.15.69:0/4153169180 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508018660 con 0x7f9520103590 2024-03-19T01:09:48.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.113+0000 7f9526400700 1 -- 172.21.15.69:0/4153169180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95201a2bf0 con 0x7f9520103590 2024-03-19T01:09:48.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.114+0000 7f9526400700 1 -- 172.21.15.69:0/4153169180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9520068fd0 con 0x7f9520103590 2024-03-19T01:09:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.118+0000 7f951cc00700 1 -- 172.21.15.69:0/4153169180 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9508010430 con 0x7f9520103590 2024-03-19T01:09:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.119+0000 7f951cc00700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f95040a2910 0x7f95040a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.119+0000 7f951cc00700 1 -- 172.21.15.69:0/4153169180 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f95080ceff0 con 0x7f9520103590 2024-03-19T01:09:48.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.119+0000 7f951f400700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f95040a2910 0x7f95040a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:48.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.120+0000 7f951f400700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f95040a2910 0x7f95040a4dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f95201033f0 tx=0x7f951000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:48.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.123+0000 7f951cc00700 1 -- 172.21.15.69:0/4153169180 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f950809bca0 con 0x7f9520103590 2024-03-19T01:09:48.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.417+0000 7f9526400700 1 -- 172.21.15.69:0/4153169180 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f9520075c60 con 0x7f95040a2910 2024-03-19T01:09:48.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.420+0000 7f951cc00700 1 -- 172.21.15.69:0/4153169180 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f9520075c60 con 0x7f95040a2910 2024-03-19T01:09:48.421 INFO:teuthology.orchestra.run.smithi069.stdout:HOST ADDR LABELS STATUS 2024-03-19T01:09:48.421 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 172.21.15.69 2024-03-19T01:09:48.422 INFO:teuthology.orchestra.run.smithi069.stdout:smithi086 172.21.15.86 2024-03-19T01:09:48.422 INFO:teuthology.orchestra.run.smithi069.stdout:smithi163 172.21.15.163 2024-03-19T01:09:48.422 INFO:teuthology.orchestra.run.smithi069.stdout:3 hosts in cluster 2024-03-19T01:09:48.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 -- 172.21.15.69:0/4153169180 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f95040a2910 msgr2=0x7f95040a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:48.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f95040a2910 0x7f95040a4dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f95201033f0 tx=0x7f951000b040 comp rx=0 tx=0).stop 2024-03-19T01:09:48.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 -- 172.21.15.69:0/4153169180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9520103590 msgr2=0x7f952019ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:48.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9520103590 0x7f952019ac50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f950800e9d0 tx=0x7f950800ece0 comp rx=0 tx=0).stop 2024-03-19T01:09:48.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 -- 172.21.15.69:0/4153169180 shutdown_connections 2024-03-19T01:09:48.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f95040a2910 0x7f95040a4dd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:48.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9520103f30 0x7f95201a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:48.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9520102390 0x7f952019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:48.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 --2- 172.21.15.69:0/4153169180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9520103590 0x7f952019ac50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:48.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 -- 172.21.15.69:0/4153169180 >> 172.21.15.69:0/4153169180 conn(0x7f95200fd930 msgr2=0x7f9520074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:48.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 -- 172.21.15.69:0/4153169180 shutdown_connections 2024-03-19T01:09:48.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:48.423+0000 7f9526400700 1 -- 172.21.15.69:0/4153169180 wait complete. 2024-03-19T01:09:49.189 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:48 smithi069 ceph-mon[31727]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:49.239 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-19T01:09:49.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:48 smithi086 ceph-mon[66829]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:49.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:48 smithi163 ceph-mon[67428]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:09:50.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:49 smithi069 ceph-mon[31727]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:50.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:49 smithi086 ceph-mon[66829]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:50.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:49 smithi163 ceph-mon[67428]: from='client.25255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:50.972 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:50 smithi069 ceph-mon[31727]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:09:51.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:50 smithi086 ceph-mon[66829]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:09:51.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:50 smithi163 ceph-mon[67428]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:09:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.847+0000 7f2406a00700 1 -- 172.21.15.69:0/1205626721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2400103460 msgr2=0x7f24001038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.847+0000 7f2406a00700 1 --2- 172.21.15.69:0/1205626721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2400103460 0x7f24001038c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0099c0 tx=0x7f23ec009cd0 comp rx=0 tx=0).stop 2024-03-19T01:09:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.847+0000 7f2406a00700 1 -- 172.21.15.69:0/1205626721 shutdown_connections 2024-03-19T01:09:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.847+0000 7f2406a00700 1 --2- 172.21.15.69:0/1205626721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2400103e00 0x7f240010a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.847+0000 7f2406a00700 1 --2- 172.21.15.69:0/1205626721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2400102260 0x7f2400102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.847+0000 7f2406a00700 1 --2- 172.21.15.69:0/1205626721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2400103460 0x7f24001038c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.847+0000 7f2406a00700 1 -- 172.21.15.69:0/1205626721 >> 172.21.15.69:0/1205626721 conn(0x7f24000fd820 msgr2=0x7f24000ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.847+0000 7f2406a00700 1 -- 172.21.15.69:0/1205626721 shutdown_connections 2024-03-19T01:09:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.847+0000 7f2406a00700 1 -- 172.21.15.69:0/1205626721 wait complete. 2024-03-19T01:09:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.848+0000 7f2406a00700 1 Processor -- start 2024-03-19T01:09:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.849+0000 7f2406a00700 1 -- start start 2024-03-19T01:09:51.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.849+0000 7f2406a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2400102260 0x7f2400198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:51.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.849+0000 7f2406a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2400103460 0x7f24001989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:51.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.849+0000 7f2406a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2400103e00 0x7f240019fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:51.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.849+0000 7f2406a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f240010de80 con 0x7f2400102260 2024-03-19T01:09:51.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.849+0000 7f2406a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f240010dd00 con 0x7f2400103460 2024-03-19T01:09:51.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.849+0000 7f2406a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f240010e000 con 0x7f2400103e00 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.849+0000 7f2404c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2400103460 0x7f24001989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.849+0000 7f2405600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2400102260 0x7f2400198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.849+0000 7f2404c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2400103460 0x7f24001989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33828/0 (socket says 172.21.15.69:33828) 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.850+0000 7f2404c00700 1 -- 172.21.15.69:0/2984297732 learned_addr learned my addr 172.21.15.69:0/2984297732 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.850+0000 7f2406000700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2400103e00 0x7f240019fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.850+0000 7f2404c00700 1 -- 172.21.15.69:0/2984297732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2400103e00 msgr2=0x7f240019fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.850+0000 7f2404c00700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2400103e00 0x7f240019fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.850+0000 7f2404c00700 1 -- 172.21.15.69:0/2984297732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2400102260 msgr2=0x7f2400198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.850+0000 7f2404c00700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2400102260 0x7f2400198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.850+0000 7f2404c00700 1 -- 172.21.15.69:0/2984297732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24001a00c0 con 0x7f2400103460 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.850+0000 7f2405600700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2400102260 0x7f2400198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.850+0000 7f2404c00700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2400103460 0x7f24001989a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f23ec005b40 tx=0x7f23ec009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.850+0000 7f23fe000700 1 -- 172.21.15.69:0/2984297732 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec027070 con 0x7f2400103460 2024-03-19T01:09:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.850+0000 7f23fe000700 1 -- 172.21.15.69:0/2984297732 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ec00fb40 con 0x7f2400103460 2024-03-19T01:09:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.851+0000 7f23fe000700 1 -- 172.21.15.69:0/2984297732 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec017650 con 0x7f2400103460 2024-03-19T01:09:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.851+0000 7f2406a00700 1 -- 172.21.15.69:0/2984297732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24001a0350 con 0x7f2400103460 2024-03-19T01:09:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.851+0000 7f2406a00700 1 -- 172.21.15.69:0/2984297732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24001a0750 con 0x7f2400103460 2024-03-19T01:09:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.852+0000 7f2406a00700 1 -- 172.21.15.69:0/2984297732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2400068fd0 con 0x7f2400103460 2024-03-19T01:09:51.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.854+0000 7f23fe000700 1 -- 172.21.15.69:0/2984297732 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f23ec00fcf0 con 0x7f2400103460 2024-03-19T01:09:51.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.856+0000 7f23fe000700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f23e80a2910 0x7f23e80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:51.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.857+0000 7f23fe000700 1 -- 172.21.15.69:0/2984297732 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f23ec0d0720 con 0x7f2400103460 2024-03-19T01:09:51.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.857+0000 7f2405600700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f23e80a2910 0x7f23e80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:51.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.857+0000 7f23fe000700 1 -- 172.21.15.69:0/2984297732 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f23ec030030 con 0x7f2400103460 2024-03-19T01:09:51.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:51.858+0000 7f2405600700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f23e80a2910 0x7f23e80a4dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24001032c0 tx=0x7f23f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:52.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.165+0000 7f2406a00700 1 -- 172.21.15.69:0/2984297732 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f2400062750 con 0x7f23e80a2910 2024-03-19T01:09:52.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.168+0000 7f23fe000700 1 -- 172.21.15.69:0/2984297732 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f2400062750 con 0x7f23e80a2910 2024-03-19T01:09:52.169 INFO:teuthology.orchestra.run.smithi069.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-19T01:09:52.169 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-19T01:09:52.169 INFO:teuthology.orchestra.run.smithi069.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-19T01:09:52.169 INFO:teuthology.orchestra.run.smithi069.stdout:smithi163 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GU400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-19T01:09:52.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.170+0000 7f2406a00700 1 -- 172.21.15.69:0/2984297732 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f23e80a2910 msgr2=0x7f23e80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:52.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.170+0000 7f2406a00700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f23e80a2910 0x7f23e80a4dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24001032c0 tx=0x7f23f4009260 comp rx=0 tx=0).stop 2024-03-19T01:09:52.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.170+0000 7f2406a00700 1 -- 172.21.15.69:0/2984297732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2400103460 msgr2=0x7f24001989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:52.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.170+0000 7f2406a00700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2400103460 0x7f24001989a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f23ec005b40 tx=0x7f23ec009580 comp rx=0 tx=0).stop 2024-03-19T01:09:52.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.170+0000 7f2406a00700 1 -- 172.21.15.69:0/2984297732 shutdown_connections 2024-03-19T01:09:52.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.170+0000 7f2406a00700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f23e80a2910 0x7f23e80a4dd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:52.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.170+0000 7f2406a00700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2400103e00 0x7f240019fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:52.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.170+0000 7f2406a00700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2400102260 0x7f2400198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:52.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.170+0000 7f2406a00700 1 --2- 172.21.15.69:0/2984297732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2400103460 0x7f24001989a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:52.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.170+0000 7f2406a00700 1 -- 172.21.15.69:0/2984297732 >> 172.21.15.69:0/2984297732 conn(0x7f24000fd820 msgr2=0x7f2400106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:52.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.171+0000 7f2406a00700 1 -- 172.21.15.69:0/2984297732 shutdown_connections 2024-03-19T01:09:52.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:52.171+0000 7f2406a00700 1 -- 172.21.15.69:0/2984297732 wait complete. 2024-03-19T01:09:53.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:52 smithi069 ceph-mon[31727]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:53.243 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-19T01:09:53.258 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-03-19T01:09:53.259 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-19T01:09:53.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:52 smithi086 ceph-mon[66829]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:52 smithi163 ceph-mon[67428]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:53.759 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:09:54.253 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:53 smithi069 ceph-mon[31727]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:54.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:53 smithi086 ceph-mon[66829]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:53 smithi163 ceph-mon[67428]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:09:54.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:54 smithi069 ceph-mon[31727]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:55.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:54 smithi086 ceph-mon[66829]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:55.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:54 smithi163 ceph-mon[67428]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:09:56.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.001+0000 7fe6dc400700 1 -- 172.21.15.69:0/1273838206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d41057b0 msgr2=0x7fe6d4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:56.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.001+0000 7fe6dc400700 1 --2- 172.21.15.69:0/1273838206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d41057b0 0x7fe6d4107ba0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009990 tx=0x7fe6c0009ca0 comp rx=0 tx=0).stop 2024-03-19T01:09:56.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.001+0000 7fe6dc400700 1 -- 172.21.15.69:0/1273838206 shutdown_connections 2024-03-19T01:09:56.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.001+0000 7fe6dc400700 1 --2- 172.21.15.69:0/1273838206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6d41080e0 0x7fe6d410a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:56.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.001+0000 7fe6dc400700 1 --2- 172.21.15.69:0/1273838206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6d4102e80 0x7fe6d4105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:56.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.001+0000 7fe6dc400700 1 --2- 172.21.15.69:0/1273838206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d41057b0 0x7fe6d4107ba0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:56.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.001+0000 7fe6dc400700 1 -- 172.21.15.69:0/1273838206 >> 172.21.15.69:0/1273838206 conn(0x7fe6d40fc7d0 msgr2=0x7fe6d40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:56.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.001+0000 7fe6dc400700 1 -- 172.21.15.69:0/1273838206 shutdown_connections 2024-03-19T01:09:56.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.001+0000 7fe6dc400700 1 -- 172.21.15.69:0/1273838206 wait complete. 2024-03-19T01:09:56.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.002+0000 7fe6dc400700 1 Processor -- start 2024-03-19T01:09:56.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.003+0000 7fe6dc400700 1 -- start start 2024-03-19T01:09:56.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.003+0000 7fe6dc400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6d4102e80 0x7fe6d41027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:56.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.003+0000 7fe6dc400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6d41057b0 0x7fe6d4100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:56.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.003+0000 7fe6dc400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d41080e0 0x7fe6d4101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:56.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.003+0000 7fe6dc400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d410ded0 con 0x7fe6d41057b0 2024-03-19T01:09:56.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.003+0000 7fe6dc400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d410dd50 con 0x7fe6d41080e0 2024-03-19T01:09:56.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.003+0000 7fe6dc400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d410e050 con 0x7fe6d4102e80 2024-03-19T01:09:56.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.003+0000 7fe6d9a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6d4102e80 0x7fe6d41027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:56.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.003+0000 7fe6d9a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6d4102e80 0x7fe6d41027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42564/0 (socket says 172.21.15.69:42564) 2024-03-19T01:09:56.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.003+0000 7fe6d9a00700 1 -- 172.21.15.69:0/551597358 learned_addr learned my addr 172.21.15.69:0/551597358 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:56.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.003+0000 7fe6da400700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d41080e0 0x7fe6d4101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:56.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.004+0000 7fe6d9000700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6d41057b0 0x7fe6d4100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:56.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.004+0000 7fe6d9a00700 1 -- 172.21.15.69:0/551597358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d41080e0 msgr2=0x7fe6d4101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:56.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.004+0000 7fe6d9a00700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d41080e0 0x7fe6d4101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:56.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.004+0000 7fe6d9a00700 1 -- 172.21.15.69:0/551597358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6d41057b0 msgr2=0x7fe6d4100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.004+0000 7fe6d9a00700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6d41057b0 0x7fe6d4100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.004+0000 7fe6d9a00700 1 -- 172.21.15.69:0/551597358 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d4101c20 con 0x7fe6d4102e80 2024-03-19T01:09:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.004+0000 7fe6d9a00700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6d4102e80 0x7fe6d41027e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800d980 tx=0x7fe6c800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.005+0000 7fe6d2000700 1 -- 172.21.15.69:0/551597358 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8016070 con 0x7fe6d4102e80 2024-03-19T01:09:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.005+0000 7fe6d2000700 1 -- 172.21.15.69:0/551597358 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8004d10 con 0x7fe6d4102e80 2024-03-19T01:09:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.005+0000 7fe6d2000700 1 -- 172.21.15.69:0/551597358 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8005670 con 0x7fe6d4102e80 2024-03-19T01:09:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.005+0000 7fe6dc400700 1 -- 172.21.15.69:0/551597358 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d41a26a0 con 0x7fe6d4102e80 2024-03-19T01:09:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.005+0000 7fe6dc400700 1 -- 172.21.15.69:0/551597358 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d41a2b60 con 0x7fe6d4102e80 2024-03-19T01:09:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.006+0000 7fe6dc400700 1 -- 172.21.15.69:0/551597358 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d4068fd0 con 0x7fe6d4102e80 2024-03-19T01:09:56.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.009+0000 7fe6d2000700 1 -- 172.21.15.69:0/551597358 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fe6c8004750 con 0x7fe6d4102e80 2024-03-19T01:09:56.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.010+0000 7fe6d2000700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe6bc0a2910 0x7fe6bc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:56.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.011+0000 7fe6d2000700 1 -- 172.21.15.69:0/551597358 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fe6c80cec60 con 0x7fe6d4102e80 2024-03-19T01:09:56.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.011+0000 7fe6d9000700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe6bc0a2910 0x7fe6bc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:56.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.012+0000 7fe6d9000700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe6bc0a2910 0x7fe6bc0a4dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000b5c0 tx=0x7fe6c0003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:56.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.015+0000 7fe6d2000700 1 -- 172.21.15.69:0/551597358 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe6c801f020 con 0x7fe6d4102e80 2024-03-19T01:09:56.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.285+0000 7fe6dc400700 1 -- 172.21.15.69:0/551597358 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe6d4062f20 con 0x7fe6d4102e80 2024-03-19T01:09:56.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.286+0000 7fe6d2000700 1 -- 172.21.15.69:0/551597358 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3090 (secure 0 0 0) 0x7fe6c809b8f0 con 0x7fe6d4102e80 2024-03-19T01:09:56.287 INFO:teuthology.orchestra.run.smithi069.stdout:e28 2024-03-19T01:09:56.287 INFO:teuthology.orchestra.run.smithi069.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-19T01:09:56.287 INFO:teuthology.orchestra.run.smithi069.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T01:09:56.287 INFO:teuthology.orchestra.run.smithi069.stdout:legacy client fscid: 1 2024-03-19T01:09:56.287 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:56.287 INFO:teuthology.orchestra.run.smithi069.stdout:Filesystem 'cephfs' (1) 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:fs_name cephfs 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 28 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:created 2024-03-19T01:07:14.928258+0000 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2024-03-19T01:07:50.808640+0000 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:tableserver 0 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:root 0 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:session_timeout 300 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:session_autoclose 300 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:max_file_size 1099511627776 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:max_xattr_size 65536 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:required_client_features {} 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure 0 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure_osd_epoch 0 2024-03-19T01:09:56.288 INFO:teuthology.orchestra.run.smithi069.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:max_mds 5 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:in 0,1,2,3,4 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:up {0=24581,1=24580,2=24550,3=24494,4=24476} 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:failed 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:damaged 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:stopped 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:data_pools [3,4] 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:metadata_pool 2 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:inline_data disabled 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:balancer 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:bal_rank_mask -1 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:standby_count_wanted 0 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:qdb_cluster leader: 24581 members: 24581,24476,24580,24494,24550 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.b{0:24581} state up:active seq 3 addr [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.c{0:24529} state up:standby-replay seq 1 addr [v2:172.21.15.163:6836/1677890136,v1:172.21.15.163:6837/1677890136] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.289 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.a{1:24580} state up:active seq 12 addr [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.h{1:24524} state up:standby-replay seq 1 addr [v2:172.21.15.86:6836/1996224781,v1:172.21.15.86:6837/1996224781] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.i{2:24550} state up:active seq 14 addr [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.f{2:24517} state up:standby-replay seq 1 addr [v2:172.21.15.163:6834/2175201817,v1:172.21.15.163:6835/2175201817] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.e{3:24494} state up:active seq 17 addr [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.k{3:24449} state up:standby-replay seq 1 addr [v2:172.21.15.86:6832/3047552839,v1:172.21.15.86:6833/3047552839] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.d{4:24476} state up:active seq 18 addr [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.l{4:24448} state up:standby-replay seq 1 addr [v2:172.21.15.163:6832/1468679776,v1:172.21.15.163:6833/1468679776] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout:Standby daemons: 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.j{-1:14382} state up:standby seq 1 addr [v2:172.21.15.69:6834/184915179,v1:172.21.15.69:6835/184915179] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.290 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.g{-1:14448} state up:standby seq 1 addr [v2:172.21.15.69:6838/1997100038,v1:172.21.15.69:6839/1997100038] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:09:56.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.289+0000 7fe6dc400700 1 -- 172.21.15.69:0/551597358 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe6bc0a2910 msgr2=0x7fe6bc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:56.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.289+0000 7fe6dc400700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe6bc0a2910 0x7fe6bc0a4dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000b5c0 tx=0x7fe6c0003960 comp rx=0 tx=0).stop 2024-03-19T01:09:56.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.289+0000 7fe6dc400700 1 -- 172.21.15.69:0/551597358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6d4102e80 msgr2=0x7fe6d41027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:56.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.289+0000 7fe6dc400700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6d4102e80 0x7fe6d41027e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800d980 tx=0x7fe6c800dc90 comp rx=0 tx=0).stop 2024-03-19T01:09:56.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.289+0000 7fe6dc400700 1 -- 172.21.15.69:0/551597358 shutdown_connections 2024-03-19T01:09:56.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.289+0000 7fe6dc400700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe6bc0a2910 0x7fe6bc0a4dd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:56.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.289+0000 7fe6dc400700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d41080e0 0x7fe6d4101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:56.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.289+0000 7fe6dc400700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6d4102e80 0x7fe6d41027e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:56.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.289+0000 7fe6dc400700 1 --2- 172.21.15.69:0/551597358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6d41057b0 0x7fe6d4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:56.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.289+0000 7fe6dc400700 1 -- 172.21.15.69:0/551597358 >> 172.21.15.69:0/551597358 conn(0x7fe6d40fc7d0 msgr2=0x7fe6d41037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:56.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.290+0000 7fe6dc400700 1 -- 172.21.15.69:0/551597358 shutdown_connections 2024-03-19T01:09:56.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:56.290+0000 7fe6dc400700 1 -- 172.21.15.69:0/551597358 wait complete. 2024-03-19T01:09:56.292 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-03-19T01:09:57.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:56 smithi069 ceph-mon[31727]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:57.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:56 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/551597358' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-19T01:09:57.188 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-19T01:09:57.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:56 smithi086 ceph-mon[66829]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:57.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:56 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/551597358' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-19T01:09:57.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:56 smithi163 ceph-mon[67428]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:09:57.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:56 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/551597358' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-19T01:09:57.711 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:09:59.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:09:58 smithi069 ceph-mon[31727]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:59.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:09:58 smithi086 ceph-mon[66829]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:59.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:09:58 smithi163 ceph-mon[67428]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:09:59.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.863+0000 7f15d1800700 1 -- 172.21.15.69:0/3656678828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15cc107f50 msgr2=0x7f15cc10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:59.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.863+0000 7f15d1800700 1 --2- 172.21.15.69:0/3656678828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15cc107f50 0x7f15cc10a3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f15c400b0a0 tx=0x7f15c400b3b0 comp rx=0 tx=0).stop 2024-03-19T01:09:59.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.864+0000 7f15d1800700 1 -- 172.21.15.69:0/3656678828 shutdown_connections 2024-03-19T01:09:59.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.864+0000 7f15d1800700 1 --2- 172.21.15.69:0/3656678828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15cc107f50 0x7f15cc10a3c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:59.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.864+0000 7f15d1800700 1 --2- 172.21.15.69:0/3656678828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15cc102cf0 0x7f15cc1050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.864+0000 7f15d1800700 1 --2- 172.21.15.69:0/3656678828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15cc105620 0x7f15cc107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.864+0000 7f15d1800700 1 -- 172.21.15.69:0/3656678828 >> 172.21.15.69:0/3656678828 conn(0x7f15cc0fc700 msgr2=0x7f15cc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.864+0000 7f15d1800700 1 -- 172.21.15.69:0/3656678828 shutdown_connections 2024-03-19T01:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.864+0000 7f15d1800700 1 -- 172.21.15.69:0/3656678828 wait complete. 2024-03-19T01:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.865+0000 7f15d1800700 1 Processor -- start 2024-03-19T01:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.865+0000 7f15d1800700 1 -- start start 2024-03-19T01:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.865+0000 7f15d1800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15cc102cf0 0x7f15cc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.865+0000 7f15d1800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15cc105620 0x7f15cc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.865+0000 7f15d1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15cc107f50 0x7f15cc19f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.865+0000 7f15d1800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc10de70 con 0x7f15cc102cf0 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.865+0000 7f15d1800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc10dcf0 con 0x7f15cc107f50 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.865+0000 7f15d1800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc10dff0 con 0x7f15cc105620 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.866+0000 7f15cbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15cc102cf0 0x7f15cc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.866+0000 7f15cb400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15cc105620 0x7f15cc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.866+0000 7f15cbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15cc102cf0 0x7f15cc198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57004/0 (socket says 172.21.15.69:57004) 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.866+0000 7f15cbe00700 1 -- 172.21.15.69:0/1418663467 learned_addr learned my addr 172.21.15.69:0/1418663467 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.866+0000 7f15d0e00700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15cc107f50 0x7f15cc19f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.866+0000 7f15d0e00700 1 -- 172.21.15.69:0/1418663467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15cc105620 msgr2=0x7f15cc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.866+0000 7f15d0e00700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15cc105620 0x7f15cc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.866+0000 7f15d0e00700 1 -- 172.21.15.69:0/1418663467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15cc102cf0 msgr2=0x7f15cc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.866+0000 7f15d0e00700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15cc102cf0 0x7f15cc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.866+0000 7f15d0e00700 1 -- 172.21.15.69:0/1418663467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc1a0060 con 0x7f15cc107f50 2024-03-19T01:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.867+0000 7f15d0e00700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15cc107f50 0x7f15cc19f9c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f15c4007fa0 tx=0x7f15c4004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.867+0000 7f15c8c00700 1 -- 172.21.15.69:0/1418663467 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4021070 con 0x7f15cc107f50 2024-03-19T01:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.867+0000 7f15c8c00700 1 -- 172.21.15.69:0/1418663467 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c4007b80 con 0x7f15cc107f50 2024-03-19T01:09:59.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.867+0000 7f15d1800700 1 -- 172.21.15.69:0/1418663467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15cc1a02f0 con 0x7f15cc107f50 2024-03-19T01:09:59.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.867+0000 7f15c8c00700 1 -- 172.21.15.69:0/1418663467 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4004750 con 0x7f15cc107f50 2024-03-19T01:09:59.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.867+0000 7f15d1800700 1 -- 172.21.15.69:0/1418663467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15cc1a07f0 con 0x7f15cc107f50 2024-03-19T01:09:59.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.868+0000 7f15d1800700 1 -- 172.21.15.69:0/1418663467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15cc068fd0 con 0x7f15cc107f50 2024-03-19T01:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.870+0000 7f15c8c00700 1 -- 172.21.15.69:0/1418663467 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f15c4003a00 con 0x7f15cc107f50 2024-03-19T01:09:59.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.872+0000 7f15c8c00700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f15b00a2910 0x7f15b00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:09:59.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.872+0000 7f15c8c00700 1 -- 172.21.15.69:0/1418663467 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f15c400f020 con 0x7f15cc107f50 2024-03-19T01:09:59.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.873+0000 7f15cbe00700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f15b00a2910 0x7f15b00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:09:59.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.873+0000 7f15cbe00700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f15b00a2910 0x7f15b00a4dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009680 tx=0x7f15bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:09:59.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:09:59.875+0000 7f15c8c00700 1 -- 172.21.15.69:0/1418663467 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f15c402a020 con 0x7f15cc107f50 2024-03-19T01:10:00.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.123+0000 7f15d1800700 1 -- 172.21.15.69:0/1418663467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f15cc062f20 con 0x7f15cc107f50 2024-03-19T01:10:00.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.124+0000 7f15c8c00700 1 -- 172.21.15.69:0/1418663467 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5660 (secure 0 0 0) 0x7f15c409cf50 con 0x7f15cc107f50 2024-03-19T01:10:00.126 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 93 2024-03-19T01:10:00.126 INFO:teuthology.orchestra.run.smithi069.stdout:fsid 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:10:00.126 INFO:teuthology.orchestra.run.smithi069.stdout:created 2024-03-19T00:58:09.792298+0000 2024-03-19T01:10:00.126 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2024-03-19T01:09:25.783660+0000 2024-03-19T01:10:00.126 INFO:teuthology.orchestra.run.smithi069.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-19T01:10:00.126 INFO:teuthology.orchestra.run.smithi069.stdout:crush_version 27 2024-03-19T01:10:00.126 INFO:teuthology.orchestra.run.smithi069.stdout:full_ratio 0.95 2024-03-19T01:10:00.126 INFO:teuthology.orchestra.run.smithi069.stdout:backfillfull_ratio 0.9 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:nearfull_ratio 0.85 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:require_min_compat_client luminous 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:min_compat_client luminous 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:require_osd_release squid 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:stretch_mode_enabled false 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:max_osd 12 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6802/3647912729,v1:172.21.15.69:6803/3647912729] [v2:172.21.15.69:6804/3647912729,v1:172.21.15.69:6805/3647912729] exists,up 53d5a7ae-973f-48a0-a4f5-904f7be2b0bf 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6800/2662600946,v1:172.21.15.86:6801/2662600946] [v2:172.21.15.86:6802/2662600946,v1:172.21.15.86:6803/2662600946] exists,up a7a78610-164b-454c-b229-91059a95c647 2024-03-19T01:10:00.127 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6800/3542508503,v1:172.21.15.163:6801/3542508503] [v2:172.21.15.163:6802/3542508503,v1:172.21.15.163:6803/3542508503] exists,up ef693645-672d-4f7c-bfe9-1ba357b7fd50 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6810/2038881914,v1:172.21.15.69:6811/2038881914] [v2:172.21.15.69:6812/2038881914,v1:172.21.15.69:6813/2038881914] exists,up 5c06aaad-6a1f-4e27-b991-248826c99079 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6808/1709421684,v1:172.21.15.86:6809/1709421684] [v2:172.21.15.86:6810/1709421684,v1:172.21.15.86:6811/1709421684] exists,up 6cc3d6e9-ff8f-4cf1-b245-83311dc7564c 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 up in weight 1 up_from 38 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6808/95931076,v1:172.21.15.163:6809/95931076] [v2:172.21.15.163:6810/95931076,v1:172.21.15.163:6811/95931076] exists,up 7808c335-63a3-46f1-a05d-8bf41c048ef4 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 up in weight 1 up_from 46 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6818/2819834341,v1:172.21.15.69:6819/2819834341] [v2:172.21.15.69:6820/2819834341,v1:172.21.15.69:6821/2819834341] exists,up 92498308-b815-42a5-acbd-433c52723dd5 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6816/866804392,v1:172.21.15.86:6817/866804392] [v2:172.21.15.86:6818/866804392,v1:172.21.15.86:6819/866804392] exists,up 9f4bc368-4502-4d9d-898d-641c779f7511 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 up in weight 1 up_from 59 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6816/171535778,v1:172.21.15.163:6817/171535778] [v2:172.21.15.163:6818/171535778,v1:172.21.15.163:6819/171535778] exists,up 81553561-de57-4fb8-894b-d4c63532dc8d 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 up in weight 1 up_from 67 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6826/3178049735,v1:172.21.15.69:6827/3178049735] [v2:172.21.15.69:6828/3178049735,v1:172.21.15.69:6829/3178049735] exists,up 83e5279d-25d1-4acb-8fe9-47f4b32ad1a3 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 up in weight 1 up_from 72 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6824/4217204109,v1:172.21.15.86:6825/4217204109] [v2:172.21.15.86:6826/4217204109,v1:172.21.15.86:6827/4217204109] exists,up 41af0156-b4aa-43f3-9e40-6396608fbfa8 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 up in weight 1 up_from 80 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6824/2918196559,v1:172.21.15.163:6825/2918196559] [v2:172.21.15.163:6826/2918196559,v1:172.21.15.163:6827/2918196559] exists,up bf43d278-aaee-4d11-9fda-3b72869c045c 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.a [2,11] 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.16 [2,4] 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.1d [8,11] 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.1 [4,9] 2024-03-19T01:10:00.128 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.11 [5,9] 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.21 [4,8] 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.28 [3,8] 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.37 [1,9] 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.3a [4,6] 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.5 [5,10] 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.b [9,8] 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.f [9,8] 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.1c [9,8] 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/4073584559 expires 2024-03-20T00:59:23.503145+0000 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/4073584559 expires 2024-03-20T00:59:23.503145+0000 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/2580938130 expires 2024-03-20T00:59:23.503145+0000 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/2422305615 expires 2024-03-20T00:58:49.440150+0000 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/4228526145 expires 2024-03-20T00:58:49.440150+0000 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/697036895 expires 2024-03-20T00:58:49.440150+0000 2024-03-19T01:10:00.129 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/2422305615 expires 2024-03-20T00:58:49.440150+0000 2024-03-19T01:10:00.130 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/1824937012 expires 2024-03-20T00:59:23.503145+0000 2024-03-19T01:10:00.130 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/3077206365 expires 2024-03-20T00:58:49.440150+0000 2024-03-19T01:10:00.130 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/2948479024 expires 2024-03-20T00:59:23.503145+0000 2024-03-19T01:10:00.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.126+0000 7f15d1800700 1 -- 172.21.15.69:0/1418663467 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f15b00a2910 msgr2=0x7f15b00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:00.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.126+0000 7f15d1800700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f15b00a2910 0x7f15b00a4dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009680 tx=0x7f15bc009260 comp rx=0 tx=0).stop 2024-03-19T01:10:00.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.126+0000 7f15d1800700 1 -- 172.21.15.69:0/1418663467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15cc107f50 msgr2=0x7f15cc19f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:00.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.126+0000 7f15d1800700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15cc107f50 0x7f15cc19f9c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f15c4007fa0 tx=0x7f15c4004c80 comp rx=0 tx=0).stop 2024-03-19T01:10:00.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.126+0000 7f15d1800700 1 -- 172.21.15.69:0/1418663467 shutdown_connections 2024-03-19T01:10:00.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.126+0000 7f15d1800700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f15b00a2910 0x7f15b00a4dd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:00.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.126+0000 7f15d1800700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15cc107f50 0x7f15cc19f9c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:00.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.126+0000 7f15d1800700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15cc102cf0 0x7f15cc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:00.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.126+0000 7f15d1800700 1 --2- 172.21.15.69:0/1418663467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15cc105620 0x7f15cc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:00.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.126+0000 7f15d1800700 1 -- 172.21.15.69:0/1418663467 >> 172.21.15.69:0/1418663467 conn(0x7f15cc0fc700 msgr2=0x7f15cc1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:00.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.128+0000 7f15d1800700 1 -- 172.21.15.69:0/1418663467 shutdown_connections 2024-03-19T01:10:00.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:00.128+0000 7f15d1800700 1 -- 172.21.15.69:0/1418663467 wait complete. 2024-03-19T01:10:01.060 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-19T01:10:01.073 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:10:01.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:00 smithi086 ceph-mon[66829]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:10:01.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:00 smithi086 ceph-mon[66829]: overall HEALTH_OK 2024-03-19T01:10:01.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:00 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1418663467' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-19T01:10:01.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:00 smithi069 ceph-mon[31727]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:10:01.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:00 smithi069 ceph-mon[31727]: overall HEALTH_OK 2024-03-19T01:10:01.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:00 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1418663467' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-19T01:10:01.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.288+0000 7feadea00640 1 -- 172.21.15.69:0/1118756400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fead81075a0 msgr2=0x7fead8109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:01.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.288+0000 7feadea00640 1 --2- 172.21.15.69:0/1118756400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fead81075a0 0x7fead8109990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fead0009ca0 tx=0x7fead002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.288+0000 7feadea00640 1 -- 172.21.15.69:0/1118756400 shutdown_connections 2024-03-19T01:10:01.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.288+0000 7feadea00640 1 --2- 172.21.15.69:0/1118756400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fead81075a0 0x7fead8109990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.288+0000 7feadea00640 1 --2- 172.21.15.69:0/1118756400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead8104c70 0x7fead8107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.288+0000 7feadea00640 1 --2- 172.21.15.69:0/1118756400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead8102340 0x7fead8104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.288+0000 7feadea00640 1 -- 172.21.15.69:0/1118756400 >> 172.21.15.69:0/1118756400 conn(0x7fead80fbcd0 msgr2=0x7fead80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:01.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.288+0000 7feadea00640 1 -- 172.21.15.69:0/1118756400 shutdown_connections 2024-03-19T01:10:01.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.288+0000 7feadea00640 1 -- 172.21.15.69:0/1118756400 wait complete. 2024-03-19T01:10:01.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.289+0000 7feadea00640 1 Processor -- start 2024-03-19T01:10:01.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7feadea00640 1 -- start start 2024-03-19T01:10:01.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7feadea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fead8102340 0x7fead8101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7feadea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead8104c70 0x7fead81002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7feadea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead81075a0 0x7fead8100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7feadea00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fead810d500 con 0x7fead8104c70 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7feadea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fead810d380 con 0x7fead81075a0 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7feadea00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fead810d680 con 0x7fead8102340 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7fead7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fead8102340 0x7fead8101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7fead7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead8104c70 0x7fead81002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7fead7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead8104c70 0x7fead81002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57014/0 (socket says 172.21.15.69:57014) 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7fead7400640 1 -- 172.21.15.69:0/1172791380 learned_addr learned my addr 172.21.15.69:0/1172791380 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7fead7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fead8102340 0x7fead8101cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42616/0 (socket says 172.21.15.69:42616) 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.290+0000 7feadca00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead81075a0 0x7fead8100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.291+0000 7feadca00640 1 -- 172.21.15.69:0/1172791380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fead8102340 msgr2=0x7fead8101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.291+0000 7feadca00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fead8102340 0x7fead8101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.291+0000 7feadca00640 1 -- 172.21.15.69:0/1172791380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead8104c70 msgr2=0x7fead81002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:01.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.291+0000 7feadca00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead8104c70 0x7fead81002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.291+0000 7feadca00640 1 -- 172.21.15.69:0/1172791380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead81010c0 con 0x7fead81075a0 2024-03-19T01:10:01.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.291+0000 7fead7400640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead8104c70 0x7fead81002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:01.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.291+0000 7feadca00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead81075a0 0x7fead8100830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fead0037040 tx=0x7fead0032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:01.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.291+0000 7fead4c00640 1 -- 172.21.15.69:0/1172791380 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0041020 con 0x7fead81075a0 2024-03-19T01:10:01.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.291+0000 7fead4c00640 1 -- 172.21.15.69:0/1172791380 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fead0034cd0 con 0x7fead81075a0 2024-03-19T01:10:01.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.291+0000 7feadea00640 1 -- 172.21.15.69:0/1172791380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead81a1c00 con 0x7fead81075a0 2024-03-19T01:10:01.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.292+0000 7fead4c00640 1 -- 172.21.15.69:0/1172791380 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0039040 con 0x7fead81075a0 2024-03-19T01:10:01.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.292+0000 7feadea00640 1 -- 172.21.15.69:0/1172791380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead81a2120 con 0x7fead81075a0 2024-03-19T01:10:01.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.293+0000 7feadea00640 1 -- 172.21.15.69:0/1172791380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead80fd7c0 con 0x7fead81075a0 2024-03-19T01:10:01.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.294+0000 7fead4c00640 1 -- 172.21.15.69:0/1172791380 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fead0044020 con 0x7fead81075a0 2024-03-19T01:10:01.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.296+0000 7fead4c00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7feabc0a2800 0x7feabc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:01.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.296+0000 7fead4c00640 1 -- 172.21.15.69:0/1172791380 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fead00f0c20 con 0x7fead81075a0 2024-03-19T01:10:01.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.296+0000 7fead7e00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7feabc0a2800 0x7feabc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:01.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.297+0000 7fead7e00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7feabc0a2800 0x7feabc0a4cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feac8009bd0 tx=0x7feac8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:01.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.304+0000 7fead4c00640 1 -- 172.21.15.69:0/1172791380 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fead004b020 con 0x7fead81075a0 2024-03-19T01:10:01.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:00 smithi163 ceph-mon[67428]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:10:01.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:00 smithi163 ceph-mon[67428]: overall HEALTH_OK 2024-03-19T01:10:01.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:00 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1418663467' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-19T01:10:01.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.504+0000 7feadea00640 1 -- 172.21.15.69:0/1172791380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fead8062420 con 0x7fead81075a0 2024-03-19T01:10:01.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.506+0000 7fead4c00640 1 -- 172.21.15.69:0/1172791380 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24736 (secure 0 0 0) 0x7fead00bd940 con 0x7fead81075a0 2024-03-19T01:10:01.507 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:01.507 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:09:25.783660+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:10:01.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 -- 172.21.15.69:0/1172791380 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7feabc0a2800 msgr2=0x7feabc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:01.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7feabc0a2800 0x7feabc0a4cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feac8009bd0 tx=0x7feac8005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:01.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 -- 172.21.15.69:0/1172791380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead81075a0 msgr2=0x7fead8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:01.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead81075a0 0x7fead8100830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fead0037040 tx=0x7fead0032ea0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 -- 172.21.15.69:0/1172791380 shutdown_connections 2024-03-19T01:10:01.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7feabc0a2800 0x7feabc0a4cc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fead81075a0 0x7fead8100830 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fead8104c70 0x7fead81002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 --2- 172.21.15.69:0/1172791380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fead8102340 0x7fead8101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 -- 172.21.15.69:0/1172791380 >> 172.21.15.69:0/1172791380 conn(0x7fead80fbcd0 msgr2=0x7fead8102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:01.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 -- 172.21.15.69:0/1172791380 shutdown_connections 2024-03-19T01:10:01.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.509+0000 7feadea00640 1 -- 172.21.15.69:0/1172791380 wait complete. 2024-03-19T01:10:01.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:07:13.368130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:07:14.444206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T01:07:18.189146+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T01:10:01.529 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:10:01.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.787+0000 7ff399c00640 1 -- 172.21.15.69:0/2156770728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3941028e0 msgr2=0x7ff394104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:01.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.787+0000 7ff399c00640 1 --2- 172.21.15.69:0/2156770728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3941028e0 0x7ff394104d50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff380009860 tx=0x7ff38002bb70 comp rx=0 tx=0).stop 2024-03-19T01:10:01.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.787+0000 7ff399c00640 1 -- 172.21.15.69:0/2156770728 shutdown_connections 2024-03-19T01:10:01.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.787+0000 7ff399c00640 1 --2- 172.21.15.69:0/2156770728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff394105290 0x7ff394105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.787+0000 7ff399c00640 1 --2- 172.21.15.69:0/2156770728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3941028e0 0x7ff394104d50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.787+0000 7ff399c00640 1 --2- 172.21.15.69:0/2156770728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3940fff70 0x7ff3941023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.787+0000 7ff399c00640 1 -- 172.21.15.69:0/2156770728 >> 172.21.15.69:0/2156770728 conn(0x7ff3940fbb90 msgr2=0x7ff3940fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:01.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.787+0000 7ff399c00640 1 -- 172.21.15.69:0/2156770728 shutdown_connections 2024-03-19T01:10:01.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.787+0000 7ff399c00640 1 -- 172.21.15.69:0/2156770728 wait complete. 2024-03-19T01:10:01.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.789+0000 7ff399c00640 1 Processor -- start 2024-03-19T01:10:01.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.789+0000 7ff399c00640 1 -- start start 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.789+0000 7ff399c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3940fff70 0x7ff39419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.789+0000 7ff399c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3941028e0 0x7ff39419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.789+0000 7ff399c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff394105290 0x7ff3941a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.789+0000 7ff399c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff394113950 con 0x7ff394105290 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.789+0000 7ff399c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941137d0 con 0x7ff3940fff70 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.789+0000 7ff399c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff394113ad0 con 0x7ff3941028e0 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.790+0000 7ff393e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3940fff70 0x7ff39419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.790+0000 7ff393e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3940fff70 0x7ff39419dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45168/0 (socket says 172.21.15.69:45168) 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.790+0000 7ff393400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3941028e0 0x7ff39419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.790+0000 7ff393e00640 1 -- 172.21.15.69:0/590994276 learned_addr learned my addr 172.21.15.69:0/590994276 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.790+0000 7ff399200640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff394105290 0x7ff3941a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.790+0000 7ff393400640 1 -- 172.21.15.69:0/590994276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3940fff70 msgr2=0x7ff39419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:01.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.790+0000 7ff393400640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3940fff70 0x7ff39419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.790+0000 7ff393400640 1 -- 172.21.15.69:0/590994276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff394105290 msgr2=0x7ff3941a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:01.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.790+0000 7ff393400640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff394105290 0x7ff3941a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:01.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.790+0000 7ff393400640 1 -- 172.21.15.69:0/590994276 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3941a5c10 con 0x7ff3941028e0 2024-03-19T01:10:01.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.790+0000 7ff393400640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3941028e0 0x7ff39419e530 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff38002c0b0 tx=0x7ff380032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:01.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.791+0000 7ff390c00640 1 -- 172.21.15.69:0/590994276 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380041020 con 0x7ff3941028e0 2024-03-19T01:10:01.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.791+0000 7ff390c00640 1 -- 172.21.15.69:0/590994276 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38002c9f0 con 0x7ff3941028e0 2024-03-19T01:10:01.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.791+0000 7ff399c00640 1 -- 172.21.15.69:0/590994276 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3941a5ea0 con 0x7ff3941028e0 2024-03-19T01:10:01.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.791+0000 7ff390c00640 1 -- 172.21.15.69:0/590994276 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380039040 con 0x7ff3941028e0 2024-03-19T01:10:01.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.791+0000 7ff399c00640 1 -- 172.21.15.69:0/590994276 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3941a6300 con 0x7ff3941028e0 2024-03-19T01:10:01.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.792+0000 7ff399c00640 1 -- 172.21.15.69:0/590994276 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3940fd680 con 0x7ff3941028e0 2024-03-19T01:10:01.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.794+0000 7ff390c00640 1 -- 172.21.15.69:0/590994276 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff380044020 con 0x7ff3941028e0 2024-03-19T01:10:01.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.795+0000 7ff390c00640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff3780a2800 0x7ff3780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:01.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.795+0000 7ff390c00640 1 -- 172.21.15.69:0/590994276 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7ff3800f0730 con 0x7ff3941028e0 2024-03-19T01:10:01.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.795+0000 7ff393e00640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff3780a2800 0x7ff3780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:01.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.796+0000 7ff393e00640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff3780a2800 0x7ff3780a4cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff3840058c0 tx=0x7ff384009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:01.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:01.799+0000 7ff390c00640 1 -- 172.21.15.69:0/590994276 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff38004b020 con 0x7ff3941028e0 2024-03-19T01:10:02.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.007+0000 7ff399c00640 1 -- 172.21.15.69:0/590994276 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff39402d410 con 0x7ff3941028e0 2024-03-19T01:10:02.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.008+0000 7ff390c00640 1 -- 172.21.15.69:0/590994276 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7ff3800bd3c0 con 0x7ff3941028e0 2024-03-19T01:10:02.009 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:10:02.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.010+0000 7ff399c00640 1 -- 172.21.15.69:0/590994276 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff3780a2800 msgr2=0x7ff3780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.010+0000 7ff399c00640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff3780a2800 0x7ff3780a4cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff3840058c0 tx=0x7ff384009260 comp rx=0 tx=0).stop 2024-03-19T01:10:02.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.010+0000 7ff399c00640 1 -- 172.21.15.69:0/590994276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3941028e0 msgr2=0x7ff39419e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.010+0000 7ff399c00640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3941028e0 0x7ff39419e530 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff38002c0b0 tx=0x7ff380032430 comp rx=0 tx=0).stop 2024-03-19T01:10:02.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.011+0000 7ff399c00640 1 -- 172.21.15.69:0/590994276 shutdown_connections 2024-03-19T01:10:02.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.011+0000 7ff399c00640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff3780a2800 0x7ff3780a4cc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.011+0000 7ff399c00640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff394105290 0x7ff3941a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.011+0000 7ff399c00640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3941028e0 0x7ff39419e530 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.011+0000 7ff399c00640 1 --2- 172.21.15.69:0/590994276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3940fff70 0x7ff39419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.011+0000 7ff399c00640 1 -- 172.21.15.69:0/590994276 >> 172.21.15.69:0/590994276 conn(0x7ff3940fbb90 msgr2=0x7ff3941008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:02.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.011+0000 7ff399c00640 1 -- 172.21.15.69:0/590994276 shutdown_connections 2024-03-19T01:10:02.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.011+0000 7ff399c00640 1 -- 172.21.15.69:0/590994276 wait complete. 2024-03-19T01:10:02.040 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:10:02.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.253+0000 7f813aa00640 1 -- 172.21.15.69:0/468164732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8134103920 msgr2=0x7f8134103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.253+0000 7f813aa00640 1 --2- 172.21.15.69:0/468164732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8134103920 0x7f8134103d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8128009900 tx=0x7f812802bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:02.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.253+0000 7f813aa00640 1 -- 172.21.15.69:0/468164732 shutdown_connections 2024-03-19T01:10:02.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.253+0000 7f813aa00640 1 --2- 172.21.15.69:0/468164732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81341054d0 0x7f813410bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.253+0000 7f813aa00640 1 --2- 172.21.15.69:0/468164732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8134104b10 0x7f8134104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.253+0000 7f813aa00640 1 --2- 172.21.15.69:0/468164732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8134103920 0x7f8134103d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.253+0000 7f813aa00640 1 -- 172.21.15.69:0/468164732 >> 172.21.15.69:0/468164732 conn(0x7f81340fee90 msgr2=0x7f81341012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:02.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.253+0000 7f813aa00640 1 -- 172.21.15.69:0/468164732 shutdown_connections 2024-03-19T01:10:02.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.253+0000 7f813aa00640 1 -- 172.21.15.69:0/468164732 wait complete. 2024-03-19T01:10:02.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:01 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1172791380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:02.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:01 smithi086 ceph-mon[66829]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:02.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.254+0000 7f813aa00640 1 Processor -- start 2024-03-19T01:10:02.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f813aa00640 1 -- start start 2024-03-19T01:10:02.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f813aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103920 0x7f8134199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:02.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f813aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8134104b10 0x7f813419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f813aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81341054d0 0x7f81341a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f813aa00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f813410f5c0 con 0x7f81341054d0 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f813aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f813410f440 con 0x7f8134103920 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f813aa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f813410f740 con 0x7f8134104b10 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f8139600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103920 0x7f8134199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f8139600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103920 0x7f8134199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45192/0 (socket says 172.21.15.69:45192) 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f8139600640 1 -- 172.21.15.69:0/271749974 learned_addr learned my addr 172.21.15.69:0/271749974 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f8138c00640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8134104b10 0x7f813419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.255+0000 7f813a000640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81341054d0 0x7f81341a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.256+0000 7f8139600640 1 -- 172.21.15.69:0/271749974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8134104b10 msgr2=0x7f813419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.256+0000 7f8139600640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8134104b10 0x7f813419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.256+0000 7f8139600640 1 -- 172.21.15.69:0/271749974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81341054d0 msgr2=0x7f81341a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.256+0000 7f8139600640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81341054d0 0x7f81341a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.256+0000 7f8139600640 1 -- 172.21.15.69:0/271749974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81341a1840 con 0x7f8134103920 2024-03-19T01:10:02.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.256+0000 7f8139600640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103920 0x7f8134199c80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8128000c00 tx=0x7f812802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:02.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.256+0000 7f8132000640 1 -- 172.21.15.69:0/271749974 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128043020 con 0x7f8134103920 2024-03-19T01:10:02.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.256+0000 7f813aa00640 1 -- 172.21.15.69:0/271749974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81341a1ad0 con 0x7f8134103920 2024-03-19T01:10:02.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.256+0000 7f8132000640 1 -- 172.21.15.69:0/271749974 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81280054e0 con 0x7f8134103920 2024-03-19T01:10:02.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.256+0000 7f8132000640 1 -- 172.21.15.69:0/271749974 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128003c40 con 0x7f8134103920 2024-03-19T01:10:02.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.257+0000 7f813aa00640 1 -- 172.21.15.69:0/271749974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81341a2020 con 0x7f8134103920 2024-03-19T01:10:02.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.258+0000 7f813aa00640 1 -- 172.21.15.69:0/271749974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8134068980 con 0x7f8134103920 2024-03-19T01:10:02.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.259+0000 7f8132000640 1 -- 172.21.15.69:0/271749974 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f8128046020 con 0x7f8134103920 2024-03-19T01:10:02.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.261+0000 7f8132000640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f811c0a2800 0x7f811c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:02.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.261+0000 7f8132000640 1 -- 172.21.15.69:0/271749974 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f81280f0ef0 con 0x7f8134103920 2024-03-19T01:10:02.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.261+0000 7f8138c00640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f811c0a2800 0x7f811c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:02.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.262+0000 7f8138c00640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f811c0a2800 0x7f811c0a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8120009bd0 tx=0x7f8120005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:02.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.269+0000 7f8132000640 1 -- 172.21.15.69:0/271749974 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f812804d020 con 0x7f8134103920 2024-03-19T01:10:02.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:01 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1172791380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:02.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:01 smithi069 ceph-mon[31727]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:02.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:01 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1172791380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:02.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:01 smithi163 ceph-mon[67428]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:02.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.469+0000 7f813aa00640 1 -- 172.21.15.69:0/271749974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8134062420 con 0x7f8134103920 2024-03-19T01:10:02.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.470+0000 7f8132000640 1 -- 172.21.15.69:0/271749974 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f81280bdbe0 con 0x7f8134103920 2024-03-19T01:10:02.471 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:10:02.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 -- 172.21.15.69:0/271749974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f811c0a2800 msgr2=0x7f811c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f811c0a2800 0x7f811c0a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8120009bd0 tx=0x7f8120005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:02.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 -- 172.21.15.69:0/271749974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103920 msgr2=0x7f8134199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103920 0x7f8134199c80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8128000c00 tx=0x7f812802c650 comp rx=0 tx=0).stop 2024-03-19T01:10:02.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 -- 172.21.15.69:0/271749974 shutdown_connections 2024-03-19T01:10:02.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f811c0a2800 0x7f811c0a4cc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81341054d0 0x7f81341a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8134104b10 0x7f813419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 --2- 172.21.15.69:0/271749974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103920 0x7f8134199c80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 -- 172.21.15.69:0/271749974 >> 172.21.15.69:0/271749974 conn(0x7f81340fee90 msgr2=0x7f8134107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:02.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 -- 172.21.15.69:0/271749974 shutdown_connections 2024-03-19T01:10:02.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.473+0000 7f813aa00640 1 -- 172.21.15.69:0/271749974 wait complete. 2024-03-19T01:10:02.500 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:10:02.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.715+0000 7fdba6a00640 1 -- 172.21.15.69:0/2368084127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdba01054d0 msgr2=0x7fdba010bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.715+0000 7fdba6a00640 1 --2- 172.21.15.69:0/2368084127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdba01054d0 0x7fdba010bd60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009ca0 tx=0x7fdb9802c100 comp rx=0 tx=0).stop 2024-03-19T01:10:02.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.716+0000 7fdba6a00640 1 -- 172.21.15.69:0/2368084127 shutdown_connections 2024-03-19T01:10:02.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.716+0000 7fdba6a00640 1 --2- 172.21.15.69:0/2368084127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdba01054d0 0x7fdba010bd60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.716+0000 7fdba6a00640 1 --2- 172.21.15.69:0/2368084127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba0104b10 0x7fdba0104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.716+0000 7fdba6a00640 1 --2- 172.21.15.69:0/2368084127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba0103920 0x7fdba0103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.716+0000 7fdba6a00640 1 -- 172.21.15.69:0/2368084127 >> 172.21.15.69:0/2368084127 conn(0x7fdba00fee90 msgr2=0x7fdba01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:02.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.716+0000 7fdba6a00640 1 -- 172.21.15.69:0/2368084127 shutdown_connections 2024-03-19T01:10:02.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.716+0000 7fdba6a00640 1 -- 172.21.15.69:0/2368084127 wait complete. 2024-03-19T01:10:02.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.717+0000 7fdba6a00640 1 Processor -- start 2024-03-19T01:10:02.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.717+0000 7fdba6a00640 1 -- start start 2024-03-19T01:10:02.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.718+0000 7fdba6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba0103920 0x7fdba0199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:02.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.718+0000 7fdba6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdba0104b10 0x7fdba019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:02.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.718+0000 7fdba6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba01054d0 0x7fdba01a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:02.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.718+0000 7fdba6a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdba010f5a0 con 0x7fdba0103920 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.718+0000 7fdba6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdba010f420 con 0x7fdba01054d0 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.718+0000 7fdba6a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdba010f720 con 0x7fdba0104b10 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.718+0000 7fdba5600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba0103920 0x7fdba0199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.718+0000 7fdba4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdba0104b10 0x7fdba019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.718+0000 7fdba6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba01054d0 0x7fdba01a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.718+0000 7fdba4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdba0104b10 0x7fdba019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42670/0 (socket says 172.21.15.69:42670) 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.718+0000 7fdba4c00640 1 -- 172.21.15.69:0/1025513062 learned_addr learned my addr 172.21.15.69:0/1025513062 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.719+0000 7fdba4c00640 1 -- 172.21.15.69:0/1025513062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba01054d0 msgr2=0x7fdba01a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.719+0000 7fdba4c00640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba01054d0 0x7fdba01a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.719+0000 7fdba4c00640 1 -- 172.21.15.69:0/1025513062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba0103920 msgr2=0x7fdba0199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.719+0000 7fdba4c00640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba0103920 0x7fdba0199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.719+0000 7fdba4c00640 1 -- 172.21.15.69:0/1025513062 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba01a17d0 con 0x7fdba0104b10 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.719+0000 7fdba6000640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba01054d0 0x7fdba01a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:02.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.719+0000 7fdba4c00640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdba0104b10 0x7fdba019a0f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00e9f0 tx=0x7fdb8c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:02.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.719+0000 7fdb9e000640 1 -- 172.21.15.69:0/1025513062 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00cd80 con 0x7fdba0104b10 2024-03-19T01:10:02.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.719+0000 7fdb9e000640 1 -- 172.21.15.69:0/1025513062 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb8c010910 con 0x7fdba0104b10 2024-03-19T01:10:02.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.719+0000 7fdba6a00640 1 -- 172.21.15.69:0/1025513062 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba01a1ac0 con 0x7fdba0104b10 2024-03-19T01:10:02.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.719+0000 7fdb9e000640 1 -- 172.21.15.69:0/1025513062 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c018990 con 0x7fdba0104b10 2024-03-19T01:10:02.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.720+0000 7fdba6a00640 1 -- 172.21.15.69:0/1025513062 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba01a2060 con 0x7fdba0104b10 2024-03-19T01:10:02.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.721+0000 7fdba6a00640 1 -- 172.21.15.69:0/1025513062 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba010abe0 con 0x7fdba0104b10 2024-03-19T01:10:02.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.723+0000 7fdb9e000640 1 -- 172.21.15.69:0/1025513062 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fdb8c010ac0 con 0x7fdba0104b10 2024-03-19T01:10:02.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.724+0000 7fdb9e000640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdb880a2880 0x7fdb880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:02.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.725+0000 7fdb9e000640 1 -- 172.21.15.69:0/1025513062 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fdb8c0cf210 con 0x7fdba0104b10 2024-03-19T01:10:02.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.725+0000 7fdba5600640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdb880a2880 0x7fdb880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:02.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.725+0000 7fdba5600640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdb880a2880 0x7fdb880a4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdba0104970 tx=0x7fdb94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:02.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.727+0000 7fdb9e000640 1 -- 172.21.15.69:0/1025513062 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdb8c09b8b0 con 0x7fdba0104b10 2024-03-19T01:10:02.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.958+0000 7fdba6a00640 1 -- 172.21.15.69:0/1025513062 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdba002d380 con 0x7fdba0104b10 2024-03-19T01:10:02.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.958+0000 7fdb9e000640 1 -- 172.21.15.69:0/1025513062 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fdb8c0cce20 con 0x7fdba0104b10 2024-03-19T01:10:02.962 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-03-19T01:10:02.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 -- 172.21.15.69:0/1025513062 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdb880a2880 msgr2=0x7fdb880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdb880a2880 0x7fdb880a4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdba0104970 tx=0x7fdb94005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:02.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 -- 172.21.15.69:0/1025513062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdba0104b10 msgr2=0x7fdba019a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:02.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdba0104b10 0x7fdba019a0f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00e9f0 tx=0x7fdb8c00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 -- 172.21.15.69:0/1025513062 shutdown_connections 2024-03-19T01:10:02.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdb880a2880 0x7fdb880a4d40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba01054d0 0x7fdba01a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdba0104b10 0x7fdba019a0f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 --2- 172.21.15.69:0/1025513062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba0103920 0x7fdba0199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:02.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 -- 172.21.15.69:0/1025513062 >> 172.21.15.69:0/1025513062 conn(0x7fdba00fee90 msgr2=0x7fdba0107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:02.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 -- 172.21.15.69:0/1025513062 shutdown_connections 2024-03-19T01:10:02.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:02.962+0000 7fdba6a00640 1 -- 172.21.15.69:0/1025513062 wait complete. 2024-03-19T01:10:02.984 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T01:10:03.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.198+0000 7fc47b000640 1 -- 172.21.15.69:0/3481005842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474077420 msgr2=0x7fc474077880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:03.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.198+0000 7fc47b000640 1 --2- 172.21.15.69:0/3481005842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474077420 0x7fc474077880 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc460009860 tx=0x7fc46002bb70 comp rx=0 tx=0).stop 2024-03-19T01:10:03.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.199+0000 7fc47b000640 1 -- 172.21.15.69:0/3481005842 shutdown_connections 2024-03-19T01:10:03.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.199+0000 7fc47b000640 1 --2- 172.21.15.69:0/3481005842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc474077dc0 0x7fc474078660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.199+0000 7fc47b000640 1 --2- 172.21.15.69:0/3481005842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474077420 0x7fc474077880 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.199+0000 7fc47b000640 1 --2- 172.21.15.69:0/3481005842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4740761e0 0x7fc474076600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.199+0000 7fc47b000640 1 -- 172.21.15.69:0/3481005842 >> 172.21.15.69:0/3481005842 conn(0x7fc4740fefd0 msgr2=0x7fc474101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:03.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.199+0000 7fc47b000640 1 -- 172.21.15.69:0/3481005842 shutdown_connections 2024-03-19T01:10:03.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.199+0000 7fc47b000640 1 -- 172.21.15.69:0/3481005842 wait complete. 2024-03-19T01:10:03.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc47b000640 1 Processor -- start 2024-03-19T01:10:03.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc47b000640 1 -- start start 2024-03-19T01:10:03.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc47b000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4740761e0 0x7fc47419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:03.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc47b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474077420 0x7fc47419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:03.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc47b000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc474077dc0 0x7fc4741a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:03.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc47b000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc474113af0 con 0x7fc474077dc0 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc47b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc474113970 con 0x7fc474077420 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc47b000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc474113c70 con 0x7fc4740761e0 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc473e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4740761e0 0x7fc47419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc473400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474077420 0x7fc47419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc473e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4740761e0 0x7fc47419e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42680/0 (socket says 172.21.15.69:42680) 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc473400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474077420 0x7fc47419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45228/0 (socket says 172.21.15.69:45228) 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc473e00640 1 -- 172.21.15.69:0/1560990990 learned_addr learned my addr 172.21.15.69:0/1560990990 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.200+0000 7fc479000640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc474077dc0 0x7fc4741a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.201+0000 7fc473400640 1 -- 172.21.15.69:0/1560990990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4740761e0 msgr2=0x7fc47419e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.201+0000 7fc473400640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4740761e0 0x7fc47419e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.201+0000 7fc473400640 1 -- 172.21.15.69:0/1560990990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc474077dc0 msgr2=0x7fc4741a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.201+0000 7fc473400640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc474077dc0 0x7fc4741a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.201+0000 7fc473400640 1 -- 172.21.15.69:0/1560990990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4741a5d90 con 0x7fc474077420 2024-03-19T01:10:03.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.201+0000 7fc473e00640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4740761e0 0x7fc47419e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:03.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.201+0000 7fc473400640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474077420 0x7fc47419e6b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc46002c0b0 tx=0x7fc460032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:03.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.201+0000 7fc470c00640 1 -- 172.21.15.69:0/1560990990 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460041020 con 0x7fc474077420 2024-03-19T01:10:03.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.201+0000 7fc470c00640 1 -- 172.21.15.69:0/1560990990 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc46002c9f0 con 0x7fc474077420 2024-03-19T01:10:03.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.201+0000 7fc47b000640 1 -- 172.21.15.69:0/1560990990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741a6020 con 0x7fc474077420 2024-03-19T01:10:03.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.202+0000 7fc470c00640 1 -- 172.21.15.69:0/1560990990 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460039040 con 0x7fc474077420 2024-03-19T01:10:03.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.202+0000 7fc47b000640 1 -- 172.21.15.69:0/1560990990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741a64e0 con 0x7fc474077420 2024-03-19T01:10:03.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.203+0000 7fc47b000640 1 -- 172.21.15.69:0/1560990990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4741103d0 con 0x7fc474077420 2024-03-19T01:10:03.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.204+0000 7fc470c00640 1 -- 172.21.15.69:0/1560990990 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fc460044020 con 0x7fc474077420 2024-03-19T01:10:03.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.207+0000 7fc470c00640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4580a2800 0x7fc4580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:03.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.207+0000 7fc470c00640 1 -- 172.21.15.69:0/1560990990 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fc4600f0650 con 0x7fc474077420 2024-03-19T01:10:03.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.207+0000 7fc473e00640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4580a2800 0x7fc4580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:03.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.208+0000 7fc473e00640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4580a2800 0x7fc4580a4cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc464011550 tx=0x7fc464009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:03.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.211+0000 7fc470c00640 1 -- 172.21.15.69:0/1560990990 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc46004b020 con 0x7fc474077420 2024-03-19T01:10:03.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:02 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/590994276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:03.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:02 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/271749974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:03.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:02 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/590994276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:03.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:02 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/271749974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:03.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:02 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/590994276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:03.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:02 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/271749974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:03.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.489+0000 7fc47b000640 1 -- 172.21.15.69:0/1560990990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc47402d1f0 con 0x7fc474077420 2024-03-19T01:10:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.490+0000 7fc470c00640 1 -- 172.21.15.69:0/1560990990 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fc460032e50 con 0x7fc474077420 2024-03-19T01:10:03.491 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:10:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.492+0000 7fc47b000640 1 -- 172.21.15.69:0/1560990990 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4580a2800 msgr2=0x7fc4580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.492+0000 7fc47b000640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4580a2800 0x7fc4580a4cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc464011550 tx=0x7fc464009260 comp rx=0 tx=0).stop 2024-03-19T01:10:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.492+0000 7fc47b000640 1 -- 172.21.15.69:0/1560990990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474077420 msgr2=0x7fc47419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.492+0000 7fc47b000640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474077420 0x7fc47419e6b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc46002c0b0 tx=0x7fc460032430 comp rx=0 tx=0).stop 2024-03-19T01:10:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.492+0000 7fc47b000640 1 -- 172.21.15.69:0/1560990990 shutdown_connections 2024-03-19T01:10:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.493+0000 7fc47b000640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc4580a2800 0x7fc4580a4cc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.493+0000 7fc47b000640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc474077dc0 0x7fc4741a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.493+0000 7fc47b000640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474077420 0x7fc47419e6b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.493+0000 7fc47b000640 1 --2- 172.21.15.69:0/1560990990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4740761e0 0x7fc47419e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.493+0000 7fc47b000640 1 -- 172.21.15.69:0/1560990990 >> 172.21.15.69:0/1560990990 conn(0x7fc4740fefd0 msgr2=0x7fc474100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.493+0000 7fc47b000640 1 -- 172.21.15.69:0/1560990990 shutdown_connections 2024-03-19T01:10:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.493+0000 7fc47b000640 1 -- 172.21.15.69:0/1560990990 wait complete. 2024-03-19T01:10:03.517 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:10:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.718+0000 7f55c8600640 1 -- 172.21.15.69:0/3121490190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55c0101950 msgr2=0x7f55c0101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.718+0000 7f55c8600640 1 --2- 172.21.15.69:0/3121490190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55c0101950 0x7f55c0101dd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009860 tx=0x7f55ac02bb70 comp rx=0 tx=0).stop 2024-03-19T01:10:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.718+0000 7f55c8600640 1 -- 172.21.15.69:0/3121490190 shutdown_connections 2024-03-19T01:10:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.718+0000 7f55c8600640 1 --2- 172.21.15.69:0/3121490190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55c0102310 0x7f55c0108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.718+0000 7f55c8600640 1 --2- 172.21.15.69:0/3121490190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55c0101950 0x7f55c0101dd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.718+0000 7f55c8600640 1 --2- 172.21.15.69:0/3121490190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c0100760 0x7f55c0100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.718+0000 7f55c8600640 1 -- 172.21.15.69:0/3121490190 >> 172.21.15.69:0/3121490190 conn(0x7f55c00fbcd0 msgr2=0x7f55c00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:03.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.718+0000 7f55c8600640 1 -- 172.21.15.69:0/3121490190 shutdown_connections 2024-03-19T01:10:03.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.718+0000 7f55c8600640 1 -- 172.21.15.69:0/3121490190 wait complete. 2024-03-19T01:10:03.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.719+0000 7f55c8600640 1 Processor -- start 2024-03-19T01:10:03.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c8600640 1 -- start start 2024-03-19T01:10:03.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c8600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55c0100760 0x7f55c010b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:03.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c8600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c0101950 0x7f55c0109720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:03.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c8600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55c0102310 0x7f55c0109c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:03.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c8600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55c010d500 con 0x7f55c0100760 2024-03-19T01:10:03.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c8600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55c010d380 con 0x7f55c0101950 2024-03-19T01:10:03.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c8600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f55c010d680 con 0x7f55c0102310 2024-03-19T01:10:03.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55c0100760 0x7f55c010b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:03.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c0101950 0x7f55c0109720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:03.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c6600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55c0102310 0x7f55c0109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:03.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55c0100760 0x7f55c010b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57082/0 (socket says 172.21.15.69:57082) 2024-03-19T01:10:03.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.720+0000 7f55c5c00640 1 -- 172.21.15.69:0/2646504825 learned_addr learned my addr 172.21.15.69:0/2646504825 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55c6600640 1 -- 172.21.15.69:0/2646504825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c0101950 msgr2=0x7f55c0109720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55c6600640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c0101950 0x7f55c0109720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55c6600640 1 -- 172.21.15.69:0/2646504825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55c0100760 msgr2=0x7f55c010b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55c6600640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55c0100760 0x7f55c010b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:03.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55c6600640 1 -- 172.21.15.69:0/2646504825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c010a4f0 con 0x7f55c0102310 2024-03-19T01:10:03.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55c5c00640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55c0100760 0x7f55c010b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:03.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55c5200640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c0101950 0x7f55c0109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:03.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55c6600640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55c0102310 0x7f55c0109c60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009fd0 tx=0x7f55b800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:03.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55be000640 1 -- 172.21.15.69:0/2646504825 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b800ce20 con 0x7f55c0102310 2024-03-19T01:10:03.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55be000640 1 -- 172.21.15.69:0/2646504825 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b8005020 con 0x7f55c0102310 2024-03-19T01:10:03.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55c8600640 1 -- 172.21.15.69:0/2646504825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c01a1c50 con 0x7f55c0102310 2024-03-19T01:10:03.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.721+0000 7f55be000640 1 -- 172.21.15.69:0/2646504825 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8010720 con 0x7f55c0102310 2024-03-19T01:10:03.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.722+0000 7f55c8600640 1 -- 172.21.15.69:0/2646504825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c01a2120 con 0x7f55c0102310 2024-03-19T01:10:03.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.723+0000 7f55c8600640 1 -- 172.21.15.69:0/2646504825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c0068a60 con 0x7f55c0102310 2024-03-19T01:10:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.726+0000 7f55be000640 1 -- 172.21.15.69:0/2646504825 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f55b8005540 con 0x7f55c0102310 2024-03-19T01:10:03.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.727+0000 7f55be000640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55a80a2800 0x7f55a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:03.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.727+0000 7f55be000640 1 -- 172.21.15.69:0/2646504825 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f55b80cf570 con 0x7f55c0102310 2024-03-19T01:10:03.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.727+0000 7f55c5c00640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55a80a2800 0x7f55a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:03.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.728+0000 7f55c5c00640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55a80a2800 0x7f55a80a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f55b40096c0 tx=0x7f55b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:03.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:03.731+0000 7f55be000640 1 -- 172.21.15.69:0/2646504825 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f55b809c200 con 0x7f55c0102310 2024-03-19T01:10:04.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.024+0000 7f55c8600640 1 -- 172.21.15.69:0/2646504825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55c00624c0 con 0x7f55c0102310 2024-03-19T01:10:04.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.025+0000 7f55be000640 1 -- 172.21.15.69:0/2646504825 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7f55b809bb00 con 0x7f55c0102310 2024-03-19T01:10:04.027 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:04.027 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:07:50.808640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581,24476,24580,24494,24550]},"id":1}]} 2024-03-19T01:10:04.027 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-03-19T01:10:04.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 -- 172.21.15.69:0/2646504825 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55a80a2800 msgr2=0x7f55a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:04.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55a80a2800 0x7f55a80a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f55b40096c0 tx=0x7f55b4009260 comp rx=0 tx=0).stop 2024-03-19T01:10:04.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 -- 172.21.15.69:0/2646504825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55c0102310 msgr2=0x7f55c0109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:04.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55c0102310 0x7f55c0109c60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009fd0 tx=0x7f55b800ef10 comp rx=0 tx=0).stop 2024-03-19T01:10:04.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 -- 172.21.15.69:0/2646504825 shutdown_connections 2024-03-19T01:10:04.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55a80a2800 0x7f55a80a4cc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55c0102310 0x7f55c0109c60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c0101950 0x7f55c0109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 --2- 172.21.15.69:0/2646504825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55c0100760 0x7f55c010b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 -- 172.21.15.69:0/2646504825 >> 172.21.15.69:0/2646504825 conn(0x7f55c00fbcd0 msgr2=0x7f55c0104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:04.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 -- 172.21.15.69:0/2646504825 shutdown_connections 2024-03-19T01:10:04.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.028+0000 7f55c8600640 1 -- 172.21.15.69:0/2646504825 wait complete. 2024-03-19T01:10:04.052 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:10:04.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.254+0000 7f4035600640 1 -- 172.21.15.69:0/170067207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4030077dc0 msgr2=0x7f4030078660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:04.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.254+0000 7f4035600640 1 --2- 172.21.15.69:0/170067207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4030077dc0 0x7f4030078660 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4028009ca0 tx=0x7f402802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.254+0000 7f4035600640 1 -- 172.21.15.69:0/170067207 shutdown_connections 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.254+0000 7f4035600640 1 --2- 172.21.15.69:0/170067207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4030077dc0 0x7f4030078660 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.254+0000 7f4035600640 1 --2- 172.21.15.69:0/170067207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4030077420 0x7f4030077880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.254+0000 7f4035600640 1 --2- 172.21.15.69:0/170067207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40300761e0 0x7f4030076600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.254+0000 7f4035600640 1 -- 172.21.15.69:0/170067207 >> 172.21.15.69:0/170067207 conn(0x7f40300fefd0 msgr2=0x7f40301013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.254+0000 7f4035600640 1 -- 172.21.15.69:0/170067207 shutdown_connections 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.254+0000 7f4035600640 1 -- 172.21.15.69:0/170067207 wait complete. 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.255+0000 7f4035600640 1 Processor -- start 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.255+0000 7f4035600640 1 -- start start 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f4035600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300761e0 0x7f403019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f4035600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4030077420 0x7f403019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f4035600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4030077dc0 0x7f40301a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f4035600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4030113ac0 con 0x7f4030077dc0 2024-03-19T01:10:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f4035600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4030113940 con 0x7f40300761e0 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f4035600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4030113c40 con 0x7f4030077420 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f402ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300761e0 0x7f403019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f4026000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4030077420 0x7f403019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f402ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300761e0 0x7f403019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45260/0 (socket says 172.21.15.69:45260) 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f402ea00640 1 -- 172.21.15.69:0/543073685 learned_addr learned my addr 172.21.15.69:0/543073685 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f402f400640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4030077dc0 0x7f40301a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f402ea00640 1 -- 172.21.15.69:0/543073685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4030077420 msgr2=0x7f403019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f402ea00640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4030077420 0x7f403019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f402ea00640 1 -- 172.21.15.69:0/543073685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4030077dc0 msgr2=0x7f40301a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.256+0000 7f402ea00640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4030077dc0 0x7f40301a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.257+0000 7f402ea00640 1 -- 172.21.15.69:0/543073685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40301a5c80 con 0x7f40300761e0 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.257+0000 7f4026000640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4030077420 0x7f403019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.257+0000 7f402ea00640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300761e0 0x7f403019e060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f402000ea60 tx=0x7f402000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.257+0000 7f4027e00640 1 -- 172.21.15.69:0/543073685 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402000cde0 con 0x7f40300761e0 2024-03-19T01:10:04.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.257+0000 7f4027e00640 1 -- 172.21.15.69:0/543073685 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40200035f0 con 0x7f40300761e0 2024-03-19T01:10:04.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.257+0000 7f4035600640 1 -- 172.21.15.69:0/543073685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40301a5f70 con 0x7f40300761e0 2024-03-19T01:10:04.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.257+0000 7f4027e00640 1 -- 172.21.15.69:0/543073685 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020010670 con 0x7f40300761e0 2024-03-19T01:10:04.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.257+0000 7f4035600640 1 -- 172.21.15.69:0/543073685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40301a63d0 con 0x7f40300761e0 2024-03-19T01:10:04.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.259+0000 7f4035600640 1 -- 172.21.15.69:0/543073685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4030068a60 con 0x7f40300761e0 2024-03-19T01:10:04.263 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:03 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1025513062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:04.264 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:03 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1560990990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:04.264 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:03 smithi086 ceph-mon[66829]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:10:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.260+0000 7f4027e00640 1 -- 172.21.15.69:0/543073685 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f40200037a0 con 0x7f40300761e0 2024-03-19T01:10:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.262+0000 7f4027e00640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f40140a2950 0x7f40140a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.263+0000 7f4027e00640 1 -- 172.21.15.69:0/543073685 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f40200cf090 con 0x7f40300761e0 2024-03-19T01:10:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.263+0000 7f4026000640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f40140a2950 0x7f40140a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.264+0000 7f4026000640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f40140a2950 0x7f40140a4e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f403019f570 tx=0x7f4018005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:04.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.265+0000 7f4027e00640 1 -- 172.21.15.69:0/543073685 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f402009bd20 con 0x7f40300761e0 2024-03-19T01:10:04.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:03 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1025513062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:04.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:03 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1560990990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:04.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:03 smithi069 ceph-mon[31727]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:10:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:03 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1025513062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:03 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1560990990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:03 smithi163 ceph-mon[67428]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:10:04.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.594+0000 7f4035600640 1 -- 172.21.15.69:0/543073685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4030062420 con 0x7f40300761e0 2024-03-19T01:10:04.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.595+0000 7f4027e00640 1 -- 172.21.15.69:0/543073685 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24736 (secure 0 0 0) 0x7f402009b620 con 0x7f40300761e0 2024-03-19T01:10:04.597 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:04.597 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:09:25.783660+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:10:04.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.599+0000 7f4035600640 1 -- 172.21.15.69:0/543073685 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f40140a2950 msgr2=0x7f40140a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:04.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.599+0000 7f4035600640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f40140a2950 0x7f40140a4e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f403019f570 tx=0x7f4018005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:04.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.599+0000 7f4035600640 1 -- 172.21.15.69:0/543073685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300761e0 msgr2=0x7f403019e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:04.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.599+0000 7f4035600640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300761e0 0x7f403019e060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f402000ea60 tx=0x7f402000ef20 comp rx=0 tx=0).stop 2024-03-19T01:10:04.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.599+0000 7f4035600640 1 -- 172.21.15.69:0/543073685 shutdown_connections 2024-03-19T01:10:04.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.599+0000 7f4035600640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f40140a2950 0x7f40140a4e10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.599+0000 7f4035600640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4030077dc0 0x7f40301a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.599+0000 7f4035600640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4030077420 0x7f403019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.599+0000 7f4035600640 1 --2- 172.21.15.69:0/543073685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300761e0 0x7f403019e060 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.599+0000 7f4035600640 1 -- 172.21.15.69:0/543073685 >> 172.21.15.69:0/543073685 conn(0x7f40300fefd0 msgr2=0x7f4030100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:04.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.599+0000 7f4035600640 1 -- 172.21.15.69:0/543073685 shutdown_connections 2024-03-19T01:10:04.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.600+0000 7f4035600640 1 -- 172.21.15.69:0/543073685 wait complete. 2024-03-19T01:10:04.626 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:07:13.368130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:07:14.444206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T01:07:18.189146+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T01:10:04.627 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:10:04.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.838+0000 7fb011e00640 1 -- 172.21.15.69:0/4026102994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb00c1032d0 msgr2=0x7fb00c109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:04.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.838+0000 7fb011e00640 1 --2- 172.21.15.69:0/4026102994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb00c1032d0 0x7fb00c109b60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb004009ca0 tx=0x7fb00402c100 comp rx=0 tx=0).stop 2024-03-19T01:10:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.838+0000 7fb011e00640 1 -- 172.21.15.69:0/4026102994 shutdown_connections 2024-03-19T01:10:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.838+0000 7fb011e00640 1 --2- 172.21.15.69:0/4026102994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb00c1032d0 0x7fb00c109b60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.838+0000 7fb011e00640 1 --2- 172.21.15.69:0/4026102994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb00c102910 0x7fb00c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.838+0000 7fb011e00640 1 --2- 172.21.15.69:0/4026102994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb00c101720 0x7fb00c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.838+0000 7fb011e00640 1 -- 172.21.15.69:0/4026102994 >> 172.21.15.69:0/4026102994 conn(0x7fb00c0fcc90 msgr2=0x7fb00c0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.838+0000 7fb011e00640 1 -- 172.21.15.69:0/4026102994 shutdown_connections 2024-03-19T01:10:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.838+0000 7fb011e00640 1 -- 172.21.15.69:0/4026102994 wait complete. 2024-03-19T01:10:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.840+0000 7fb011e00640 1 Processor -- start 2024-03-19T01:10:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.840+0000 7fb011e00640 1 -- start start 2024-03-19T01:10:04.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.840+0000 7fb011e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb00c101720 0x7fb00c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:04.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.840+0000 7fb011e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb00c102910 0x7fb00c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:04.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.840+0000 7fb011e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb00c1032d0 0x7fb00c1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:04.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.840+0000 7fb011e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c10d3a0 con 0x7fb00c1032d0 2024-03-19T01:10:04.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.840+0000 7fb011e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c10d220 con 0x7fb00c102910 2024-03-19T01:10:04.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.840+0000 7fb011e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c10d520 con 0x7fb00c101720 2024-03-19T01:10:04.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb010a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb00c101720 0x7fb00c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:04.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb00be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb00c102910 0x7fb00c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:04.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb010a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb00c101720 0x7fb00c199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42738/0 (socket says 172.21.15.69:42738) 2024-03-19T01:10:04.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb010a00640 1 -- 172.21.15.69:0/3613185637 learned_addr learned my addr 172.21.15.69:0/3613185637 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:04.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb011400640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb00c1032d0 0x7fb00c1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:04.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb010a00640 1 -- 172.21.15.69:0/3613185637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb00c102910 msgr2=0x7fb00c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:04.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb010a00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb00c102910 0x7fb00c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb010a00640 1 -- 172.21.15.69:0/3613185637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb00c1032d0 msgr2=0x7fb00c1a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:04.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb010a00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb00c1032d0 0x7fb00c1a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:04.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb010a00640 1 -- 172.21.15.69:0/3613185637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c1a17c0 con 0x7fb00c101720 2024-03-19T01:10:04.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb011400640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb00c1032d0 0x7fb00c1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:04.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb00be00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb00c102910 0x7fb00c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:04.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb010a00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb00c101720 0x7fb00c199ba0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb00000e9c0 tx=0x7fb00000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:04.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb009600640 1 -- 172.21.15.69:0/3613185637 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00000cd80 con 0x7fb00c101720 2024-03-19T01:10:04.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.841+0000 7fb009600640 1 -- 172.21.15.69:0/3613185637 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0000035f0 con 0x7fb00c101720 2024-03-19T01:10:04.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.842+0000 7fb011e00640 1 -- 172.21.15.69:0/3613185637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c1a1ab0 con 0x7fb00c101720 2024-03-19T01:10:04.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.842+0000 7fb009600640 1 -- 172.21.15.69:0/3613185637 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000010690 con 0x7fb00c101720 2024-03-19T01:10:04.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.842+0000 7fb011e00640 1 -- 172.21.15.69:0/3613185637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c1a2000 con 0x7fb00c101720 2024-03-19T01:10:04.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.843+0000 7fb011e00640 1 -- 172.21.15.69:0/3613185637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c1a21d0 con 0x7fb00c101720 2024-03-19T01:10:04.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.844+0000 7fb009600640 1 -- 172.21.15.69:0/3613185637 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb0000037a0 con 0x7fb00c101720 2024-03-19T01:10:04.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.846+0000 7fb009600640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faff40a2880 0x7faff40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:04.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.846+0000 7fb009600640 1 -- 172.21.15.69:0/3613185637 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fb0000cf1f0 con 0x7fb00c101720 2024-03-19T01:10:04.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.846+0000 7fb00be00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faff40a2880 0x7faff40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:04.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.847+0000 7fb00be00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faff40a2880 0x7faff40a4d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb00c19b0b0 tx=0x7faff8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:04.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:04.850+0000 7fb009600640 1 -- 172.21.15.69:0/3613185637 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb00009be80 con 0x7fb00c101720 2024-03-19T01:10:05.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.050+0000 7fb011e00640 1 -- 172.21.15.69:0/3613185637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb00c1a23a0 con 0x7fb00c101720 2024-03-19T01:10:05.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.051+0000 7fb009600640 1 -- 172.21.15.69:0/3613185637 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fb00009b780 con 0x7fb00c101720 2024-03-19T01:10:05.052 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:10:05.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.053+0000 7fb011e00640 1 -- 172.21.15.69:0/3613185637 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faff40a2880 msgr2=0x7faff40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:05.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.053+0000 7fb011e00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faff40a2880 0x7faff40a4d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb00c19b0b0 tx=0x7faff8005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.053+0000 7fb011e00640 1 -- 172.21.15.69:0/3613185637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb00c101720 msgr2=0x7fb00c199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.053+0000 7fb011e00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb00c101720 0x7fb00c199ba0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb00000e9c0 tx=0x7fb00000ee80 comp rx=0 tx=0).stop 2024-03-19T01:10:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.053+0000 7fb011e00640 1 -- 172.21.15.69:0/3613185637 shutdown_connections 2024-03-19T01:10:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.053+0000 7fb011e00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7faff40a2880 0x7faff40a4d40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.053+0000 7fb011e00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb00c1032d0 0x7fb00c1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.053+0000 7fb011e00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb00c102910 0x7fb00c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.053+0000 7fb011e00640 1 --2- 172.21.15.69:0/3613185637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb00c101720 0x7fb00c199ba0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.053+0000 7fb011e00640 1 -- 172.21.15.69:0/3613185637 >> 172.21.15.69:0/3613185637 conn(0x7fb00c0fcc90 msgr2=0x7fb00c073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.054+0000 7fb011e00640 1 -- 172.21.15.69:0/3613185637 shutdown_connections 2024-03-19T01:10:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.054+0000 7fb011e00640 1 -- 172.21.15.69:0/3613185637 wait complete. 2024-03-19T01:10:05.074 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:10:05.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:04 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2646504825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:05.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:04 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/543073685' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:05.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:04 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2646504825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:05.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:04 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/543073685' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:05.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.286+0000 7fbb2ee00640 1 -- 172.21.15.69:0/1904870591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb28107440 msgr2=0x7fbb28109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:05.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.286+0000 7fbb2ee00640 1 --2- 172.21.15.69:0/1904870591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb28107440 0x7fbb28109830 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009ca0 tx=0x7fbb2002c100 comp rx=0 tx=0).stop 2024-03-19T01:10:05.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.287+0000 7fbb2ee00640 1 -- 172.21.15.69:0/1904870591 shutdown_connections 2024-03-19T01:10:05.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.287+0000 7fbb2ee00640 1 --2- 172.21.15.69:0/1904870591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb28107440 0x7fbb28109830 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.287+0000 7fbb2ee00640 1 --2- 172.21.15.69:0/1904870591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb28104b10 0x7fbb28106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.287+0000 7fbb2ee00640 1 --2- 172.21.15.69:0/1904870591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb281021e0 0x7fbb281045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.287+0000 7fbb2ee00640 1 -- 172.21.15.69:0/1904870591 >> 172.21.15.69:0/1904870591 conn(0x7fbb280fbb90 msgr2=0x7fbb280fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:05.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.287+0000 7fbb2ee00640 1 -- 172.21.15.69:0/1904870591 shutdown_connections 2024-03-19T01:10:05.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.287+0000 7fbb2ee00640 1 -- 172.21.15.69:0/1904870591 wait complete. 2024-03-19T01:10:05.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.288+0000 7fbb2ee00640 1 Processor -- start 2024-03-19T01:10:05.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.288+0000 7fbb2ee00640 1 -- start start 2024-03-19T01:10:05.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2ee00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb281021e0 0x7fbb28101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:05.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb28104b10 0x7fbb28100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:05.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2ee00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb28107440 0x7fbb281006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:05.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2ee00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2810d380 con 0x7fbb28107440 2024-03-19T01:10:05.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2810d200 con 0x7fbb28104b10 2024-03-19T01:10:05.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2ee00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2810d500 con 0x7fbb281021e0 2024-03-19T01:10:05.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb28104b10 0x7fbb28100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:05.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2da00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb281021e0 0x7fbb28101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:05.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2da00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb281021e0 0x7fbb28101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42754/0 (socket says 172.21.15.69:42754) 2024-03-19T01:10:05.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2da00640 1 -- 172.21.15.69:0/2879216835 learned_addr learned my addr 172.21.15.69:0/2879216835 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:05.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2e400640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb28107440 0x7fbb281006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2d000640 1 -- 172.21.15.69:0/2879216835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb281021e0 msgr2=0x7fbb28101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2d000640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb281021e0 0x7fbb28101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.289+0000 7fbb2d000640 1 -- 172.21.15.69:0/2879216835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb28107440 msgr2=0x7fbb281006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.290+0000 7fbb2d000640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb28107440 0x7fbb281006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.290+0000 7fbb2d000640 1 -- 172.21.15.69:0/2879216835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb28100f60 con 0x7fbb28104b10 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.290+0000 7fbb2da00640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb281021e0 0x7fbb28101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.290+0000 7fbb2d000640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb28104b10 0x7fbb28100190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400e9c0 tx=0x7fbb1400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.290+0000 7fbb26000640 1 -- 172.21.15.69:0/2879216835 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400ccc0 con 0x7fbb28104b10 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.290+0000 7fbb2ee00640 1 -- 172.21.15.69:0/2879216835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb281a1ac0 con 0x7fbb28104b10 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.290+0000 7fbb26000640 1 -- 172.21.15.69:0/2879216835 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb14010910 con 0x7fbb28104b10 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.290+0000 7fbb26000640 1 -- 172.21.15.69:0/2879216835 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14018880 con 0x7fbb28104b10 2024-03-19T01:10:05.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.290+0000 7fbb2ee00640 1 -- 172.21.15.69:0/2879216835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb281a2010 con 0x7fbb28104b10 2024-03-19T01:10:05.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.291+0000 7fbb2ee00640 1 -- 172.21.15.69:0/2879216835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb280fd680 con 0x7fbb28104b10 2024-03-19T01:10:05.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.293+0000 7fbb26000640 1 -- 172.21.15.69:0/2879216835 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fbb14010ac0 con 0x7fbb28104b10 2024-03-19T01:10:05.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.294+0000 7fbb26000640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbb100a2800 0x7fbb100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:05.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.294+0000 7fbb26000640 1 -- 172.21.15.69:0/2879216835 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fbb140cf1d0 con 0x7fbb28104b10 2024-03-19T01:10:05.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.295+0000 7fbb2da00640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbb100a2800 0x7fbb100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:05.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.295+0000 7fbb2da00640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbb100a2800 0x7fbb100a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0095f0 tx=0x7fbb1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:05.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.298+0000 7fbb26000640 1 -- 172.21.15.69:0/2879216835 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbb1409b810 con 0x7fbb28104b10 2024-03-19T01:10:05.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:04 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2646504825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:05.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:04 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/543073685' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:05.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.582+0000 7fbb2ee00640 1 -- 172.21.15.69:0/2879216835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbb28062420 con 0x7fbb28104b10 2024-03-19T01:10:05.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.582+0000 7fbb26000640 1 -- 172.21.15.69:0/2879216835 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fbb1409f8b0 con 0x7fbb28104b10 2024-03-19T01:10:05.583 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:10:05.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 -- 172.21.15.69:0/2879216835 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbb100a2800 msgr2=0x7fbb100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:05.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbb100a2800 0x7fbb100a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0095f0 tx=0x7fbb1c005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:05.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 -- 172.21.15.69:0/2879216835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb28104b10 msgr2=0x7fbb28100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:05.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb28104b10 0x7fbb28100190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400e9c0 tx=0x7fbb1400ee80 comp rx=0 tx=0).stop 2024-03-19T01:10:05.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 -- 172.21.15.69:0/2879216835 shutdown_connections 2024-03-19T01:10:05.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbb100a2800 0x7fbb100a4cc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb28107440 0x7fbb281006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb28104b10 0x7fbb28100190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 --2- 172.21.15.69:0/2879216835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb281021e0 0x7fbb28101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 -- 172.21.15.69:0/2879216835 >> 172.21.15.69:0/2879216835 conn(0x7fbb280fbb90 msgr2=0x7fbb28102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:05.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 -- 172.21.15.69:0/2879216835 shutdown_connections 2024-03-19T01:10:05.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.585+0000 7fbb2ee00640 1 -- 172.21.15.69:0/2879216835 wait complete. 2024-03-19T01:10:05.604 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:10:05.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.806+0000 7f6fdb200640 1 -- 172.21.15.69:0/4190982541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc104b10 msgr2=0x7f6fdc104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:05.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.806+0000 7f6fdb200640 1 --2- 172.21.15.69:0/4190982541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc104b10 0x7f6fdc104f90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009900 tx=0x7f6fc802bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:05.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.806+0000 7f6fdb200640 1 -- 172.21.15.69:0/4190982541 shutdown_connections 2024-03-19T01:10:05.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.806+0000 7f6fdb200640 1 --2- 172.21.15.69:0/4190982541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc1054d0 0x7f6fdc10bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.806+0000 7f6fdb200640 1 --2- 172.21.15.69:0/4190982541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc104b10 0x7f6fdc104f90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.806+0000 7f6fdb200640 1 --2- 172.21.15.69:0/4190982541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc103920 0x7f6fdc103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.806+0000 7f6fdb200640 1 -- 172.21.15.69:0/4190982541 >> 172.21.15.69:0/4190982541 conn(0x7f6fdc0fee90 msgr2=0x7f6fdc1012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:05.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.806+0000 7f6fdb200640 1 -- 172.21.15.69:0/4190982541 shutdown_connections 2024-03-19T01:10:05.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.806+0000 7f6fdb200640 1 -- 172.21.15.69:0/4190982541 wait complete. 2024-03-19T01:10:05.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.807+0000 7f6fdb200640 1 Processor -- start 2024-03-19T01:10:05.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.807+0000 7f6fdb200640 1 -- start start 2024-03-19T01:10:05.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fdb200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc103920 0x7f6fdc199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:05.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fdb200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc104b10 0x7f6fdc19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:05.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fdb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc1054d0 0x7f6fdc1a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:05.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fdb200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc10f5f0 con 0x7f6fdc104b10 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fdb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc10f470 con 0x7f6fdc1054d0 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fdb200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc10f770 con 0x7f6fdc103920 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fd9400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc104b10 0x7f6fdc19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fd9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc103920 0x7f6fdc199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fda800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc1054d0 0x7f6fdc1a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fd9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc103920 0x7f6fdc199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42768/0 (socket says 172.21.15.69:42768) 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fda800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc1054d0 0x7f6fdc1a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45320/0 (socket says 172.21.15.69:45320) 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fd9400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc104b10 0x7f6fdc19a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57154/0 (socket says 172.21.15.69:57154) 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fd9e00640 1 -- 172.21.15.69:0/2493704143 learned_addr learned my addr 172.21.15.69:0/2493704143 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fda800640 1 -- 172.21.15.69:0/2493704143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc103920 msgr2=0x7f6fdc199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fda800640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc103920 0x7f6fdc199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fda800640 1 -- 172.21.15.69:0/2493704143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc104b10 msgr2=0x7f6fdc19a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.808+0000 7f6fda800640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc104b10 0x7f6fdc19a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.809+0000 7f6fda800640 1 -- 172.21.15.69:0/2493704143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc1a1860 con 0x7f6fdc1054d0 2024-03-19T01:10:05.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.809+0000 7f6fda800640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc1054d0 0x7f6fdc1a11c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009fd0 tx=0x7f6fd400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:05.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.809+0000 7f6fd2a00640 1 -- 172.21.15.69:0/2493704143 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd400ce20 con 0x7f6fdc1054d0 2024-03-19T01:10:05.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.809+0000 7f6fd2a00640 1 -- 172.21.15.69:0/2493704143 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd4005020 con 0x7f6fdc1054d0 2024-03-19T01:10:05.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.809+0000 7f6fd2a00640 1 -- 172.21.15.69:0/2493704143 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4010720 con 0x7f6fdc1054d0 2024-03-19T01:10:05.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.809+0000 7f6fdb200640 1 -- 172.21.15.69:0/2493704143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc1a1b50 con 0x7f6fdc1054d0 2024-03-19T01:10:05.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.809+0000 7f6fdb200640 1 -- 172.21.15.69:0/2493704143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc1a2050 con 0x7f6fdc1054d0 2024-03-19T01:10:05.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.810+0000 7f6fdb200640 1 -- 172.21.15.69:0/2493704143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc10ac90 con 0x7f6fdc1054d0 2024-03-19T01:10:05.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.814+0000 7f6fd2a00640 1 -- 172.21.15.69:0/2493704143 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f6fd4005540 con 0x7f6fdc1054d0 2024-03-19T01:10:05.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.815+0000 7f6fd2a00640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc00a2800 0x7f6fc00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:05.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.815+0000 7f6fd2a00640 1 -- 172.21.15.69:0/2493704143 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f6fd40cf900 con 0x7f6fdc1054d0 2024-03-19T01:10:05.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.816+0000 7f6fd9e00640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc00a2800 0x7f6fc00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:05.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.816+0000 7f6fd2a00640 1 -- 172.21.15.69:0/2493704143 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6fd409c590 con 0x7f6fdc1054d0 2024-03-19T01:10:05.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:05.816+0000 7f6fd9e00640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc00a2800 0x7f6fc00a4cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc19b120 tx=0x7f6fcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:06.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.054+0000 7f6fdb200640 1 -- 172.21.15.69:0/2493704143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6fdc02d380 con 0x7f6fdc1054d0 2024-03-19T01:10:06.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.055+0000 7f6fd2a00640 1 -- 172.21.15.69:0/2493704143 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f6fd409be90 con 0x7f6fdc1054d0 2024-03-19T01:10:06.056 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-03-19T01:10:06.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.057+0000 7f6fdb200640 1 -- 172.21.15.69:0/2493704143 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc00a2800 msgr2=0x7f6fc00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:06.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.057+0000 7f6fdb200640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc00a2800 0x7f6fc00a4cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc19b120 tx=0x7f6fcc005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:06.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.057+0000 7f6fdb200640 1 -- 172.21.15.69:0/2493704143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc1054d0 msgr2=0x7f6fdc1a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:06.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.057+0000 7f6fdb200640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc1054d0 0x7f6fdc1a11c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009fd0 tx=0x7f6fd400ef10 comp rx=0 tx=0).stop 2024-03-19T01:10:06.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.058+0000 7f6fdb200640 1 -- 172.21.15.69:0/2493704143 shutdown_connections 2024-03-19T01:10:06.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.058+0000 7f6fdb200640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f6fc00a2800 0x7f6fc00a4cc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.058+0000 7f6fdb200640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fdc1054d0 0x7f6fdc1a11c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.058+0000 7f6fdb200640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fdc104b10 0x7f6fdc19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.058+0000 7f6fdb200640 1 --2- 172.21.15.69:0/2493704143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fdc103920 0x7f6fdc199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.058+0000 7f6fdb200640 1 -- 172.21.15.69:0/2493704143 >> 172.21.15.69:0/2493704143 conn(0x7f6fdc0fee90 msgr2=0x7f6fdc107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:06.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.058+0000 7f6fdb200640 1 -- 172.21.15.69:0/2493704143 shutdown_connections 2024-03-19T01:10:06.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.058+0000 7f6fdb200640 1 -- 172.21.15.69:0/2493704143 wait complete. 2024-03-19T01:10:06.077 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T01:10:06.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:05 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3613185637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:06.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:05 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2879216835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:06.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:05 smithi086 ceph-mon[66829]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:06.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:05 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3613185637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:06.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:05 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2879216835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:06.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:05 smithi069 ceph-mon[31727]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:06.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.302+0000 7f9e93000640 1 -- 172.21.15.69:0/51222843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e8c102180 msgr2=0x7f9e8c104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:06.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.302+0000 7f9e93000640 1 --2- 172.21.15.69:0/51222843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e8c102180 0x7f9e8c104570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009900 tx=0x7f9e8002bc40 comp rx=0 tx=0).stop 2024-03-19T01:10:06.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.302+0000 7f9e93000640 1 -- 172.21.15.69:0/51222843 shutdown_connections 2024-03-19T01:10:06.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.302+0000 7f9e93000640 1 --2- 172.21.15.69:0/51222843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e8c1073e0 0x7f9e8c109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.302+0000 7f9e93000640 1 --2- 172.21.15.69:0/51222843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c104ab0 0x7f9e8c106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.302+0000 7f9e93000640 1 --2- 172.21.15.69:0/51222843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e8c102180 0x7f9e8c104570 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.302+0000 7f9e93000640 1 -- 172.21.15.69:0/51222843 >> 172.21.15.69:0/51222843 conn(0x7f9e8c0fbb90 msgr2=0x7f9e8c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:06.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.302+0000 7f9e93000640 1 -- 172.21.15.69:0/51222843 shutdown_connections 2024-03-19T01:10:06.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.303+0000 7f9e93000640 1 -- 172.21.15.69:0/51222843 wait complete. 2024-03-19T01:10:06.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.304+0000 7f9e93000640 1 Processor -- start 2024-03-19T01:10:06.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.304+0000 7f9e93000640 1 -- start start 2024-03-19T01:10:06.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.304+0000 7f9e93000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c102180 0x7f9e8c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.304+0000 7f9e93000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e8c104ab0 0x7f9e8c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e93000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e8c1073e0 0x7f9e8c1a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e93000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c10d3d0 con 0x7f9e8c104ab0 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e93000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c10d250 con 0x7f9e8c102180 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e93000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c10d550 con 0x7f9e8c1073e0 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e91c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c102180 0x7f9e8c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e91c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c102180 0x7f9e8c199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:32916/0 (socket says 172.21.15.69:32916) 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e91c00640 1 -- 172.21.15.69:0/2859858211 learned_addr learned my addr 172.21.15.69:0/2859858211 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e92600640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e8c1073e0 0x7f9e8c1a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e91200640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e8c104ab0 0x7f9e8c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e91c00640 1 -- 172.21.15.69:0/2859858211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e8c1073e0 msgr2=0x7f9e8c1a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e91c00640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e8c1073e0 0x7f9e8c1a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e91c00640 1 -- 172.21.15.69:0/2859858211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e8c104ab0 msgr2=0x7f9e8c19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:06.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e91c00640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e8c104ab0 0x7f9e8c19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e91c00640 1 -- 172.21.15.69:0/2859858211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8c1a1850 con 0x7f9e8c102180 2024-03-19T01:10:06.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.305+0000 7f9e91c00640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c102180 0x7f9e8c199c40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e8002c8f0 tx=0x7f9e80005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:06.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.306+0000 7f9e8a000640 1 -- 172.21.15.69:0/2859858211 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80041070 con 0x7f9e8c102180 2024-03-19T01:10:06.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.306+0000 7f9e8a000640 1 -- 172.21.15.69:0/2859858211 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e800044d0 con 0x7f9e8c102180 2024-03-19T01:10:06.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.306+0000 7f9e93000640 1 -- 172.21.15.69:0/2859858211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8c1a1ae0 con 0x7f9e8c102180 2024-03-19T01:10:06.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.306+0000 7f9e8a000640 1 -- 172.21.15.69:0/2859858211 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80034440 con 0x7f9e8c102180 2024-03-19T01:10:06.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.306+0000 7f9e93000640 1 -- 172.21.15.69:0/2859858211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8c1a1fd0 con 0x7f9e8c102180 2024-03-19T01:10:06.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.307+0000 7f9e93000640 1 -- 172.21.15.69:0/2859858211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e8c108f30 con 0x7f9e8c102180 2024-03-19T01:10:06.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.309+0000 7f9e8a000640 1 -- 172.21.15.69:0/2859858211 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9e80044070 con 0x7f9e8c102180 2024-03-19T01:10:06.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.310+0000 7f9e8a000640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e740a2800 0x7f9e740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:06.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.310+0000 7f9e8a000640 1 -- 172.21.15.69:0/2859858211 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f9e800f0b40 con 0x7f9e8c102180 2024-03-19T01:10:06.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.310+0000 7f9e91200640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e740a2800 0x7f9e740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:06.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.311+0000 7f9e91200640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e740a2800 0x7f9e740a4cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0694e0 tx=0x7f9e78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:06.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.314+0000 7f9e8a000640 1 -- 172.21.15.69:0/2859858211 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9e800bd7d0 con 0x7f9e8c102180 2024-03-19T01:10:06.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:05 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3613185637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:06.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:05 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2879216835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:06.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:05 smithi163 ceph-mon[67428]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:06.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.508+0000 7f9e93000640 1 -- 172.21.15.69:0/2859858211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9e8c062420 con 0x7f9e8c102180 2024-03-19T01:10:06.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.509+0000 7f9e8a000640 1 -- 172.21.15.69:0/2859858211 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f9e800bd0d0 con 0x7f9e8c102180 2024-03-19T01:10:06.510 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.511+0000 7f9e93000640 1 -- 172.21.15.69:0/2859858211 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e740a2800 msgr2=0x7f9e740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.511+0000 7f9e93000640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e740a2800 0x7f9e740a4cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0694e0 tx=0x7f9e78005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.511+0000 7f9e93000640 1 -- 172.21.15.69:0/2859858211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c102180 msgr2=0x7f9e8c199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.511+0000 7f9e93000640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c102180 0x7f9e8c199c40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e8002c8f0 tx=0x7f9e80005d50 comp rx=0 tx=0).stop 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.511+0000 7f9e93000640 1 -- 172.21.15.69:0/2859858211 shutdown_connections 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.511+0000 7f9e93000640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e740a2800 0x7f9e740a4cc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.511+0000 7f9e93000640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e8c1073e0 0x7f9e8c1a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.511+0000 7f9e93000640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e8c104ab0 0x7f9e8c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.511+0000 7f9e93000640 1 --2- 172.21.15.69:0/2859858211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c102180 0x7f9e8c199c40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.512+0000 7f9e93000640 1 -- 172.21.15.69:0/2859858211 >> 172.21.15.69:0/2859858211 conn(0x7f9e8c0fbb90 msgr2=0x7f9e8c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.512+0000 7f9e93000640 1 -- 172.21.15.69:0/2859858211 shutdown_connections 2024-03-19T01:10:06.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.512+0000 7f9e93000640 1 -- 172.21.15.69:0/2859858211 wait complete. 2024-03-19T01:10:06.529 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:10:06.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.758+0000 7f7197200640 1 -- 172.21.15.69:0/1067436005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7198109830 msgr2=0x7f71981100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:06.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.758+0000 7f7197200640 1 --2- 172.21.15.69:0/1067436005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7198109830 0x7f71981100c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7190009ca0 tx=0x7f719002c100 comp rx=0 tx=0).stop 2024-03-19T01:10:06.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.758+0000 7f7197200640 1 -- 172.21.15.69:0/1067436005 shutdown_connections 2024-03-19T01:10:06.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.758+0000 7f7197200640 1 --2- 172.21.15.69:0/1067436005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7198109830 0x7f71981100c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.758+0000 7f7197200640 1 --2- 172.21.15.69:0/1067436005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7198108e70 0x7f71981092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.758+0000 7f7197200640 1 --2- 172.21.15.69:0/1067436005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7198107c80 0x7f71981080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.758+0000 7f7197200640 1 -- 172.21.15.69:0/1067436005 >> 172.21.15.69:0/1067436005 conn(0x7f7198076c30 msgr2=0x7f7198079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:06.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.758+0000 7f7197200640 1 -- 172.21.15.69:0/1067436005 shutdown_connections 2024-03-19T01:10:06.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.758+0000 7f7197200640 1 -- 172.21.15.69:0/1067436005 wait complete. 2024-03-19T01:10:06.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.759+0000 7f7197200640 1 Processor -- start 2024-03-19T01:10:06.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.759+0000 7f7197200640 1 -- start start 2024-03-19T01:10:06.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7197200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7198107c80 0x7f719819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7197200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7198108e70 0x7f719819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7197200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7198109830 0x7f71981a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7197200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7198113900 con 0x7f7198108e70 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7197200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7198113780 con 0x7f7198109830 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7197200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7198113a80 con 0x7f7198107c80 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7195e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7198107c80 0x7f719819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f718fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7198108e70 0x7f719819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7196800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7198109830 0x7f71981a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7195e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7198107c80 0x7f719819dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57330/0 (socket says 172.21.15.69:57330) 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7195e00640 1 -- 172.21.15.69:0/4181090890 learned_addr learned my addr 172.21.15.69:0/4181090890 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7196800640 1 -- 172.21.15.69:0/4181090890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7198107c80 msgr2=0x7f719819dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7196800640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7198107c80 0x7f719819dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7196800640 1 -- 172.21.15.69:0/4181090890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7198108e70 msgr2=0x7f719819e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:06.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7196800640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7198108e70 0x7f719819e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:06.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.760+0000 7f7196800640 1 -- 172.21.15.69:0/4181090890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71981a5b00 con 0x7f7198109830 2024-03-19T01:10:06.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.761+0000 7f7196800640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7198109830 0x7f71981a5490 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7190037040 tx=0x7f7190035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:06.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.761+0000 7f718ea00640 1 -- 172.21.15.69:0/4181090890 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190039040 con 0x7f7198109830 2024-03-19T01:10:06.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.761+0000 7f718ea00640 1 -- 172.21.15.69:0/4181090890 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7190044070 con 0x7f7198109830 2024-03-19T01:10:06.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.761+0000 7f718ea00640 1 -- 172.21.15.69:0/4181090890 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190004350 con 0x7f7198109830 2024-03-19T01:10:06.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.761+0000 7f7197200640 1 -- 172.21.15.69:0/4181090890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71981a5d90 con 0x7f7198109830 2024-03-19T01:10:06.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.761+0000 7f7197200640 1 -- 172.21.15.69:0/4181090890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71981a6240 con 0x7f7198109830 2024-03-19T01:10:06.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.762+0000 7f7197200640 1 -- 172.21.15.69:0/4181090890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719810ef40 con 0x7f7198109830 2024-03-19T01:10:06.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.764+0000 7f718ea00640 1 -- 172.21.15.69:0/4181090890 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f71900044f0 con 0x7f7198109830 2024-03-19T01:10:06.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.765+0000 7f718ea00640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f717c0a2880 0x7f717c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:06.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.765+0000 7f718ea00640 1 -- 172.21.15.69:0/4181090890 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f71900f1300 con 0x7f7198109830 2024-03-19T01:10:06.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.766+0000 7f7195e00640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f717c0a2880 0x7f717c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:06.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.766+0000 7f7195e00640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f717c0a2880 0x7f717c0a4d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f71880058c0 tx=0x7f7188005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:06.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:06.774+0000 7f718ea00640 1 -- 172.21.15.69:0/4181090890 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f71900bdf90 con 0x7f7198109830 2024-03-19T01:10:07.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.037+0000 7f7197200640 1 -- 172.21.15.69:0/4181090890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71980624c0 con 0x7f7198109830 2024-03-19T01:10:07.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.038+0000 7f718ea00640 1 -- 172.21.15.69:0/4181090890 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7f71900a6e20 con 0x7f7198109830 2024-03-19T01:10:07.040 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:07.040 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:07:50.808640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581,24476,24580,24494,24550]},"id":1}]} 2024-03-19T01:10:07.040 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-03-19T01:10:07.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 -- 172.21.15.69:0/4181090890 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f717c0a2880 msgr2=0x7f717c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f717c0a2880 0x7f717c0a4d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f71880058c0 tx=0x7f7188005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:07.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 -- 172.21.15.69:0/4181090890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7198109830 msgr2=0x7f71981a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7198109830 0x7f71981a5490 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7190037040 tx=0x7f7190035d10 comp rx=0 tx=0).stop 2024-03-19T01:10:07.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 -- 172.21.15.69:0/4181090890 shutdown_connections 2024-03-19T01:10:07.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f717c0a2880 0x7f717c0a4d40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7198109830 0x7f71981a5490 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7198108e70 0x7f719819e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 --2- 172.21.15.69:0/4181090890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7198107c80 0x7f719819dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 -- 172.21.15.69:0/4181090890 >> 172.21.15.69:0/4181090890 conn(0x7f7198076c30 msgr2=0x7f719810c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:07.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 -- 172.21.15.69:0/4181090890 shutdown_connections 2024-03-19T01:10:07.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.041+0000 7f7197200640 1 -- 172.21.15.69:0/4181090890 wait complete. 2024-03-19T01:10:07.062 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:10:07.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:06 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2493704143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:07.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:06 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2859858211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:07.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:06 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:07.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:06 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2493704143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:07.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:06 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2859858211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:07.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:06 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:07.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.293+0000 7fca1a600640 1 -- 172.21.15.69:0/1199767891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca14101690 msgr2=0x7fca14101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.293+0000 7fca1a600640 1 --2- 172.21.15.69:0/1199767891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca14101690 0x7fca14101ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fca08009900 tx=0x7fca0802bc40 comp rx=0 tx=0).stop 2024-03-19T01:10:07.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.294+0000 7fca1a600640 1 -- 172.21.15.69:0/1199767891 shutdown_connections 2024-03-19T01:10:07.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.294+0000 7fca1a600640 1 --2- 172.21.15.69:0/1199767891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca14103240 0x7fca14109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.294+0000 7fca1a600640 1 --2- 172.21.15.69:0/1199767891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca14102880 0x7fca14102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.294+0000 7fca1a600640 1 --2- 172.21.15.69:0/1199767891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca14101690 0x7fca14101ab0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.294+0000 7fca1a600640 1 -- 172.21.15.69:0/1199767891 >> 172.21.15.69:0/1199767891 conn(0x7fca140fcc00 msgr2=0x7fca140ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:07.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.294+0000 7fca1a600640 1 -- 172.21.15.69:0/1199767891 shutdown_connections 2024-03-19T01:10:07.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.294+0000 7fca1a600640 1 -- 172.21.15.69:0/1199767891 wait complete. 2024-03-19T01:10:07.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.295+0000 7fca1a600640 1 Processor -- start 2024-03-19T01:10:07.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.295+0000 7fca1a600640 1 -- start start 2024-03-19T01:10:07.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca1a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca14101690 0x7fca141979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca1a600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca14102880 0x7fca14197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca1a600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca14103240 0x7fca1419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca1a600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fca1410d2e0 con 0x7fca14102880 2024-03-19T01:10:07.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca1a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca1410d160 con 0x7fca14101690 2024-03-19T01:10:07.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca1a600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fca1410d460 con 0x7fca14103240 2024-03-19T01:10:07.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca19200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca14101690 0x7fca141979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca13e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca14102880 0x7fca14197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca19200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca14101690 0x7fca141979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:32952/0 (socket says 172.21.15.69:32952) 2024-03-19T01:10:07.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca19200640 1 -- 172.21.15.69:0/625191183 learned_addr learned my addr 172.21.15.69:0/625191183 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:07.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca19c00640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca14103240 0x7fca1419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca19200640 1 -- 172.21.15.69:0/625191183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca14103240 msgr2=0x7fca1419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca19200640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca14103240 0x7fca1419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca19200640 1 -- 172.21.15.69:0/625191183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca14102880 msgr2=0x7fca14197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.296+0000 7fca19200640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca14102880 0x7fca14197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.297+0000 7fca19200640 1 -- 172.21.15.69:0/625191183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1419f600 con 0x7fca14101690 2024-03-19T01:10:07.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.297+0000 7fca19200640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca14101690 0x7fca141979e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fca08000c00 tx=0x7fca0802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:07.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.297+0000 7fca11600640 1 -- 172.21.15.69:0/625191183 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08041070 con 0x7fca14101690 2024-03-19T01:10:07.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.297+0000 7fca11600640 1 -- 172.21.15.69:0/625191183 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca08004450 con 0x7fca14101690 2024-03-19T01:10:07.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.297+0000 7fca11600640 1 -- 172.21.15.69:0/625191183 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08003e20 con 0x7fca14101690 2024-03-19T01:10:07.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.297+0000 7fca1a600640 1 -- 172.21.15.69:0/625191183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1419f890 con 0x7fca14101690 2024-03-19T01:10:07.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.298+0000 7fca1a600640 1 -- 172.21.15.69:0/625191183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1419fd30 con 0x7fca14101690 2024-03-19T01:10:07.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.299+0000 7fca1a600640 1 -- 172.21.15.69:0/625191183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca14108a80 con 0x7fca14101690 2024-03-19T01:10:07.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.302+0000 7fca11600640 1 -- 172.21.15.69:0/625191183 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fca08044070 con 0x7fca14101690 2024-03-19T01:10:07.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.303+0000 7fca11600640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9fc0a2800 0x7fc9fc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.304+0000 7fca11600640 1 -- 172.21.15.69:0/625191183 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fca080f0950 con 0x7fca14101690 2024-03-19T01:10:07.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.304+0000 7fca13e00640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9fc0a2800 0x7fc9fc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.305+0000 7fca13e00640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9fc0a2800 0x7fc9fc0a4cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fca000095f0 tx=0x7fca00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:07.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:06 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2493704143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:07.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:06 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2859858211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:07.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:06 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:07.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.312+0000 7fca11600640 1 -- 172.21.15.69:0/625191183 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fca080bd670 con 0x7fca14101690 2024-03-19T01:10:07.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.600+0000 7fca1a600640 1 -- 172.21.15.69:0/625191183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca14062420 con 0x7fca14101690 2024-03-19T01:10:07.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.602+0000 7fca11600640 1 -- 172.21.15.69:0/625191183 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24736 (secure 0 0 0) 0x7fca080bcf70 con 0x7fca14101690 2024-03-19T01:10:07.603 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:07.604 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:09:25.783660+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:10:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.604+0000 7fca1a600640 1 -- 172.21.15.69:0/625191183 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9fc0a2800 msgr2=0x7fc9fc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.604+0000 7fca1a600640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9fc0a2800 0x7fc9fc0a4cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fca000095f0 tx=0x7fca00005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:07.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.604+0000 7fca1a600640 1 -- 172.21.15.69:0/625191183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca14101690 msgr2=0x7fca141979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.604+0000 7fca1a600640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca14101690 0x7fca141979e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fca08000c00 tx=0x7fca0802c650 comp rx=0 tx=0).stop 2024-03-19T01:10:07.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.605+0000 7fca1a600640 1 -- 172.21.15.69:0/625191183 shutdown_connections 2024-03-19T01:10:07.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.605+0000 7fca1a600640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc9fc0a2800 0x7fc9fc0a4cc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.605+0000 7fca1a600640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca14103240 0x7fca1419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.605+0000 7fca1a600640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca14102880 0x7fca14197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.605+0000 7fca1a600640 1 --2- 172.21.15.69:0/625191183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca14101690 0x7fca141979e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.605+0000 7fca1a600640 1 -- 172.21.15.69:0/625191183 >> 172.21.15.69:0/625191183 conn(0x7fca140fcc00 msgr2=0x7fca14105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:07.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.605+0000 7fca1a600640 1 -- 172.21.15.69:0/625191183 shutdown_connections 2024-03-19T01:10:07.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.605+0000 7fca1a600640 1 -- 172.21.15.69:0/625191183 wait complete. 2024-03-19T01:10:07.631 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-19T01:07:14.928258+0000', 'modified': '2024-03-19T01:07:50.808640+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24581, 'mds_1': 24580, 'mds_2': 24550, 'mds_3': 24494, 'mds_4': 24476}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6833/1468679776', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6832', 'nonce': 1468679776}, {'type': 'v1', 'addr': '172.21.15.163:6833', 'nonce': 1468679776}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6833/3047552839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6832', 'nonce': 3047552839}, {'type': 'v1', 'addr': '172.21.15.86:6833', 'nonce': 3047552839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'd', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.69:6837/2154698833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6836', 'nonce': 2154698833}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 2154698833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.86:6835/786469397', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 786469397}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 786469397}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6835/2175201817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 2175201817}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 2175201817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/1996224781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 1996224781}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 1996224781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6837/1677890136', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 1677890136}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 1677890136}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.163:6839/2045902254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6838', 'nonce': 2045902254}, {'type': 'v1', 'addr': '172.21.15.163:6839', 'nonce': 2045902254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24580': {'gid': 24580, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.69:6841/4121471703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6840', 'nonce': 4121471703}, {'type': 'v1', 'addr': '172.21.15.69:6841', 'nonce': 4121471703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6839/621579864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 621579864}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 621579864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24581, 24476, 24580, 24494, 24550]} 2024-03-19T01:10:07.631 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-19T01:10:07.631 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-19T01:10:07.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.825+0000 7fd058800640 1 -- 172.21.15.69:0/1081097813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd050102a30 msgr2=0x7fd050102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.825+0000 7fd058800640 1 --2- 172.21.15.69:0/1081097813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd050102a30 0x7fd050102eb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd040009900 tx=0x7fd04002bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:07.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.825+0000 7fd058800640 1 -- 172.21.15.69:0/1081097813 shutdown_connections 2024-03-19T01:10:07.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.825+0000 7fd058800640 1 --2- 172.21.15.69:0/1081097813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0501033f0 0x7fd050109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.825+0000 7fd058800640 1 --2- 172.21.15.69:0/1081097813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd050102a30 0x7fd050102eb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.825+0000 7fd058800640 1 --2- 172.21.15.69:0/1081097813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd050101840 0x7fd050101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.825+0000 7fd058800640 1 -- 172.21.15.69:0/1081097813 >> 172.21.15.69:0/1081097813 conn(0x7fd0500fcdd0 msgr2=0x7fd0500ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:07.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.825+0000 7fd058800640 1 -- 172.21.15.69:0/1081097813 shutdown_connections 2024-03-19T01:10:07.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.826+0000 7fd058800640 1 -- 172.21.15.69:0/1081097813 wait complete. 2024-03-19T01:10:07.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.826+0000 7fd058800640 1 Processor -- start 2024-03-19T01:10:07.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd058800640 1 -- start start 2024-03-19T01:10:07.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd058800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd050101840 0x7fd050197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd058800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd050102a30 0x7fd0501980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd058800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0501033f0 0x7fd05019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd058800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd05010d4a0 con 0x7fd050102a30 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd058800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd05010d320 con 0x7fd0501033f0 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd058800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd05010d620 con 0x7fd050101840 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd055e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd050101840 0x7fd050197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd055400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd050102a30 0x7fd0501980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd055e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd050101840 0x7fd050197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57352/0 (socket says 172.21.15.69:57352) 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd055e00640 1 -- 172.21.15.69:0/2109110161 learned_addr learned my addr 172.21.15.69:0/2109110161 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.827+0000 7fd056800640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0501033f0 0x7fd05019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.828+0000 7fd055e00640 1 -- 172.21.15.69:0/2109110161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0501033f0 msgr2=0x7fd05019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.828+0000 7fd055e00640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0501033f0 0x7fd05019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.828+0000 7fd055e00640 1 -- 172.21.15.69:0/2109110161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd050102a30 msgr2=0x7fd0501980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.828+0000 7fd055e00640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd050102a30 0x7fd0501980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.828+0000 7fd055e00640 1 -- 172.21.15.69:0/2109110161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05019f790 con 0x7fd050101840 2024-03-19T01:10:07.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.828+0000 7fd055e00640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd050101840 0x7fd050197b70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd04400e9c0 tx=0x7fd04400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:07.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.828+0000 7fd04aa00640 1 -- 172.21.15.69:0/2109110161 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04400ccc0 con 0x7fd050101840 2024-03-19T01:10:07.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.828+0000 7fd04aa00640 1 -- 172.21.15.69:0/2109110161 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd044010910 con 0x7fd050101840 2024-03-19T01:10:07.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.828+0000 7fd04aa00640 1 -- 172.21.15.69:0/2109110161 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440187f0 con 0x7fd050101840 2024-03-19T01:10:07.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.828+0000 7fd058800640 1 -- 172.21.15.69:0/2109110161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05019fa80 con 0x7fd050101840 2024-03-19T01:10:07.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.828+0000 7fd058800640 1 -- 172.21.15.69:0/2109110161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05019ffa0 con 0x7fd050101840 2024-03-19T01:10:07.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.833+0000 7fd05b60e740 1 Processor -- start 2024-03-19T01:10:07.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.833+0000 7fd05b60e740 1 -- start start 2024-03-19T01:10:07.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.833+0000 7fd04aa00640 1 -- 172.21.15.69:0/2109110161 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd04401f020 con 0x7fd050101840 2024-03-19T01:10:07.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.833+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c27810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.833+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c27de0 0x55e3a5c2c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.833+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c2f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.833+0000 7fd05b60e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5b8bd00 con 0x55e3a5c27de0 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.833+0000 7fd05b60e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5b34b00 con 0x55e3a5c273f0 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.833+0000 7fd05b60e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5b87da0 con 0x55e3a5c2cf50 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.834+0000 7fd056800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c27810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.834+0000 7fd056800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c27810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:32974/0 (socket says 172.21.15.69:32974) 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.834+0000 7fd055e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c27de0 0x55e3a5c2c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.834+0000 7fd056800640 1 -- 172.21.15.69:0/3425680735 learned_addr learned my addr 172.21.15.69:0/3425680735 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.834+0000 7fd055400640 1 --2- 172.21.15.69:0/3425680735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c2f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.834+0000 7fd056800640 1 -- 172.21.15.69:0/3425680735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 msgr2=0x55e3a5c2f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.834+0000 7fd056800640 1 --2- 172.21.15.69:0/3425680735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c2f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.834+0000 7fd056800640 1 -- 172.21.15.69:0/3425680735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c27de0 msgr2=0x55e3a5c2c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.834+0000 7fd056800640 1 --2- 172.21.15.69:0/3425680735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c27de0 0x55e3a5c2c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.834+0000 7fd056800640 1 -- 172.21.15.69:0/3425680735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3a58a02d0 con 0x55e3a5c273f0 2024-03-19T01:10:07.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.834+0000 7fd055400640 1 --2- 172.21.15.69:0/3425680735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c2f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:07.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.835+0000 7fd056800640 1 --2- 172.21.15.69:0/3425680735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c27810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd04c011440 tx=0x7fd04c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30ecb53dc52bd368 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:07.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.835+0000 7fd03f000640 1 -- 172.21.15.69:0/3425680735 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c003c30 con 0x55e3a5c273f0 2024-03-19T01:10:07.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.835+0000 7fd03f000640 1 -- 172.21.15.69:0/3425680735 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd04c003dd0 con 0x55e3a5c273f0 2024-03-19T01:10:07.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.835+0000 7fd03f000640 1 -- 172.21.15.69:0/3425680735 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c00fb50 con 0x55e3a5c273f0 2024-03-19T01:10:07.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.835+0000 7fd04aa00640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0380a2800 0x7fd0380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.835+0000 7fd04aa00640 1 -- 172.21.15.69:0/2109110161 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fd0440cf000 con 0x7fd050101840 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.836+0000 7fd055e00640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0380a2800 0x7fd0380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.836+0000 7fd05b60e740 1 -- 172.21.15.69:0/3425680735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 msgr2=0x55e3a5c27810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.836+0000 7fd05b60e740 1 --2- 172.21.15.69:0/3425680735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c27810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd04c011440 tx=0x7fd04c011900 comp rx=0 tx=0).stop 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.836+0000 7fd05b60e740 1 -- 172.21.15.69:0/3425680735 shutdown_connections 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.836+0000 7fd055e00640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0380a2800 0x7fd0380a4cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd0440a3800 tx=0x7fd0440d2a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.836+0000 7fd05b60e740 1 --2- 172.21.15.69:0/3425680735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c2f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.836+0000 7fd05b60e740 1 --2- 172.21.15.69:0/3425680735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c27de0 0x55e3a5c2c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.836+0000 7fd05b60e740 1 --2- 172.21.15.69:0/3425680735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c27810 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.836+0000 7fd05b60e740 1 -- 172.21.15.69:0/3425680735 >> 172.21.15.69:0/3425680735 conn(0x55e3a5c22960 msgr2=0x55e3a5c22d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.836+0000 7fd05b60e740 1 -- 172.21.15.69:0/3425680735 shutdown_connections 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.836+0000 7fd05b60e740 1 -- 172.21.15.69:0/3425680735 wait complete. 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.837+0000 7fd05b60e740 1 -- start start 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.837+0000 7fd05b60e740 1 Processor -- start 2024-03-19T01:10:07.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c273f0 0x55e3a5c35d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c27de0 0x55e3a5c36260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c3d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd05b60e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5c2ff00 con 0x55e3a5c273f0 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd05b60e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5b47ed0 con 0x55e3a5c27de0 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd05b60e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5c30080 con 0x55e3a5c2cf50 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd05b60e740 10 client.0 _ll_register_callbacks cb 0x55e3a5b45c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd055400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c27de0 0x55e3a5c36260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd056800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c273f0 0x55e3a5c35d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd055400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c27de0 0x55e3a5c36260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:32976/0 (socket says 172.21.15.69:32976) 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd055400640 1 -- 172.21.15.69:0/371328713 learned_addr learned my addr 172.21.15.69:0/371328713 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.838+0000 7fd056800640 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c3d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd055400640 1 -- 172.21.15.69:0/371328713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 msgr2=0x55e3a5c3d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd055400640 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c3d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd055400640 1 -- 172.21.15.69:0/371328713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c273f0 msgr2=0x55e3a5c35d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:07.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd055400640 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c273f0 0x55e3a5c35d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd055400640 1 -- 172.21.15.69:0/371328713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3a596a6b0 con 0x55e3a5c27de0 2024-03-19T01:10:07.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd056800640 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c273f0 0x55e3a5c35d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:07.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd055400640 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c27de0 0x55e3a5c36260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd040036040 tx=0x7fd040004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=719ffce7b6ab4e36 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:07.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd033400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd033400640 1 -- 172.21.15.69:0/371328713 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040005d50 con 0x55e3a5c27de0 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd05b60e740 1 -- 172.21.15.69:0/371328713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd04c003dd0 con 0x55e3a5c27de0 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd033400640 1 -- 172.21.15.69:0/371328713 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fd040005230 con 0x55e3a5c27de0 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.839+0000 7fd033400640 1 -- 172.21.15.69:0/371328713 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd04005f050 con 0x55e3a5c27de0 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.840+0000 7fd05b60e740 10 client.25358 fetch_fsmap learned FSMap version 28 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.840+0000 7fd05b60e740 1 -- 172.21.15.69:0/371328713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e3a5c406d0 con 0x55e3a5c27de0 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.840+0000 7fd033400640 1 -- 172.21.15.69:0/371328713 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd04c003dd0 con 0x55e3a5c27de0 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.840+0000 7fd033400640 1 -- 172.21.15.69:0/371328713 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd040003860 con 0x55e3a5c27de0 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.840+0000 7fd05b60e740 10 client.25358 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.840+0000 7fd05b60e740 10 client.25358 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.840+0000 7fd05b60e740 20 client.25358 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.840+0000 7fd05b60e740 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55e3a5c42c80 0x55e3a5c440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.840+0000 7fd05b60e740 4 client.25358 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.840+0000 7fd05b60e740 1 -- 172.21.15.69:0/371328713 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3a5b99240 con 0x55e3a5c42c80 2024-03-19T01:10:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.841+0000 7fd056800640 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55e3a5c42c80 0x55e3a5c440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.841+0000 7fd056800640 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55e3a5c42c80 0x55e3a5c440f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d76cb2cbe4ed806e server_cookie=de50081b374b9d6 in_seq=0 out_seq=0 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.841+0000 7fd033400640 10 client.25358 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd033400640 1 -- 172.21.15.69:0/371328713 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e3a5b99240 con 0x55e3a5c42c80 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd033400640 10 client.25358 handle_command_reply: tid=0 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd05b60e740 1 client.25358 shutdown 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd05b60e740 1 -- 172.21.15.69:0/371328713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c27de0 msgr2=0x55e3a5c36260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd05b60e740 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c27de0 0x55e3a5c36260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd040036040 tx=0x7fd040004270 comp rx=0 tx=0).stop 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd05b60e740 1 -- 172.21.15.69:0/371328713 shutdown_connections 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd05b60e740 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55e3a5c42c80 0x55e3a5c440f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd05b60e740 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c3d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd05b60e740 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c27de0 0x55e3a5c36260 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd05b60e740 1 --2- 172.21.15.69:0/371328713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c273f0 0x55e3a5c35d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd05b60e740 1 -- 172.21.15.69:0/371328713 >> 172.21.15.69:0/371328713 conn(0x55e3a5c22960 msgr2=0x55e3a5c21aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:07.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.842+0000 7fd033400640 0 client.25358 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:10:07.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.843+0000 7fd05b60e740 1 -- 172.21.15.69:0/371328713 shutdown_connections 2024-03-19T01:10:07.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.843+0000 7fd05b60e740 1 -- 172.21.15.69:0/371328713 wait complete. 2024-03-19T01:10:07.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.843+0000 7fd05b60e740 20 client.25358 trim_cache size 0 max 16384 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd05b60e740 1 Processor -- start 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd05b60e740 1 -- start start 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c2f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c27de0 0x55e3a5c3d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c22d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd05b60e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5c42430 con 0x55e3a5c2cf50 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd05b60e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5c42b00 con 0x55e3a5c273f0 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd05b60e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5c431d0 con 0x55e3a5c27de0 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd056800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c2f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd055400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c27de0 0x55e3a5c3d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd055400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c27de0 0x55e3a5c3d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57368/0 (socket says 172.21.15.69:57368) 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd055400640 1 -- 172.21.15.69:0/2300012329 learned_addr learned my addr 172.21.15.69:0/2300012329 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:07.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.881+0000 7fd056800640 1 --2- 172.21.15.69:0/2300012329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c22d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.882+0000 7fd055400640 1 -- 172.21.15.69:0/2300012329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 msgr2=0x55e3a5c2f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:07.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.882+0000 7fd055400640 1 --2- 172.21.15.69:0/2300012329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c2f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.882+0000 7fd055400640 1 -- 172.21.15.69:0/2300012329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c2cf50 msgr2=0x55e3a5c22d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:07.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.882+0000 7fd055400640 1 --2- 172.21.15.69:0/2300012329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c22d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.882+0000 7fd055400640 1 -- 172.21.15.69:0/2300012329 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3a5c28670 con 0x55e3a5c27de0 2024-03-19T01:10:07.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.882+0000 7fd056800640 1 --2- 172.21.15.69:0/2300012329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c22d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:07.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.882+0000 7fd055400640 1 --2- 172.21.15.69:0/2300012329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c27de0 0x55e3a5c3d260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd040009900 tx=0x7fd040009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0b9efd41b7b8156 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:07.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.882+0000 7fd03e600640 1 -- 172.21.15.69:0/2300012329 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040005d50 con 0x55e3a5c27de0 2024-03-19T01:10:07.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.882+0000 7fd03e600640 1 -- 172.21.15.69:0/2300012329 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd040052070 con 0x55e3a5c27de0 2024-03-19T01:10:07.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.882+0000 7fd03e600640 1 -- 172.21.15.69:0/2300012329 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040034900 con 0x55e3a5c27de0 2024-03-19T01:10:07.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.883+0000 7fd05b60e740 1 -- 172.21.15.69:0/2300012329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c27de0 msgr2=0x55e3a5c3d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.883+0000 7fd05b60e740 1 --2- 172.21.15.69:0/2300012329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c27de0 0x55e3a5c3d260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd040009900 tx=0x7fd040009620 comp rx=0 tx=0).stop 2024-03-19T01:10:07.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.883+0000 7fd05b60e740 1 -- 172.21.15.69:0/2300012329 shutdown_connections 2024-03-19T01:10:07.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.883+0000 7fd05b60e740 1 --2- 172.21.15.69:0/2300012329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c2cf50 0x55e3a5c22d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.883+0000 7fd05b60e740 1 --2- 172.21.15.69:0/2300012329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c27de0 0x55e3a5c3d260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.883+0000 7fd05b60e740 1 --2- 172.21.15.69:0/2300012329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c2f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.883+0000 7fd05b60e740 1 -- 172.21.15.69:0/2300012329 >> 172.21.15.69:0/2300012329 conn(0x55e3a5c22960 msgr2=0x55e3a5c1d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.884+0000 7fd05b60e740 1 -- 172.21.15.69:0/2300012329 shutdown_connections 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.884+0000 7fd05b60e740 1 -- 172.21.15.69:0/2300012329 wait complete. 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.884+0000 7fd05b60e740 1 -- start start 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.884+0000 7fd05b60e740 1 Processor -- start 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c3b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c27de0 0x55e3a5ca49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd05b60e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5ca6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd05b60e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5c43f70 con 0x55e3a5c27de0 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd05b60e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5c438a0 con 0x55e3a5c273f0 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd05b60e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e3a5c44640 con 0x55e3a5c2cf50 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd05b60e740 10 client.0 _ll_register_callbacks cb 0x7fd040009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd056800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c3b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd056800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c3b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33002/0 (socket says 172.21.15.69:33002) 2024-03-19T01:10:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd056800640 1 -- 172.21.15.69:0/3521396022 learned_addr learned my addr 172.21.15.69:0/3521396022 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:07.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd056800640 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5ca6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd055400640 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c27de0 0x55e3a5ca49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd056800640 1 -- 172.21.15.69:0/3521396022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 msgr2=0x55e3a5ca6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.885+0000 7fd056800640 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5ca6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.886+0000 7fd056800640 1 -- 172.21.15.69:0/3521396022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c27de0 msgr2=0x55e3a5ca49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.886+0000 7fd056800640 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c27de0 0x55e3a5ca49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.886+0000 7fd056800640 1 -- 172.21.15.69:0/3521396022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3a5ca7460 con 0x55e3a5c273f0 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.886+0000 7fd056800640 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c3b1b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009710 tx=0x7fd04c013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0217e6b8d624f6e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.886+0000 7fd033400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.886+0000 7fd033400640 1 -- 172.21.15.69:0/3521396022 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c003b70 con 0x55e3a5c273f0 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.886+0000 7fd033400640 1 -- 172.21.15.69:0/3521396022 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fd04c02ec40 con 0x55e3a5c273f0 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.886+0000 7fd05b60e740 1 -- 172.21.15.69:0/3521396022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd040052070 con 0x55e3a5c273f0 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.886+0000 7fd033400640 1 -- 172.21.15.69:0/3521396022 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd04c02e1c0 con 0x55e3a5c273f0 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.886+0000 7fd05b60e740 10 client.25364 fetch_fsmap learned FSMap version 28 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.886+0000 7fd05b60e740 1 -- 172.21.15.69:0/3521396022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e3a5ca8730 con 0x55e3a5c273f0 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.887+0000 7fd033400640 1 -- 172.21.15.69:0/3521396022 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd040052070 con 0x55e3a5c273f0 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.887+0000 7fd033400640 1 -- 172.21.15.69:0/3521396022 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd04c03a070 con 0x55e3a5c273f0 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.887+0000 7fd05b60e740 10 client.25364 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.887+0000 7fd05b60e740 10 client.25364 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.887+0000 7fd05b60e740 20 client.25364 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:07.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.887+0000 7fd05b60e740 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55e3a5cad1a0 0x55e3a5cabeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.887+0000 7fd05b60e740 4 client.25364 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.887+0000 7fd05b60e740 1 -- 172.21.15.69:0/3521396022 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "status"}) -- 0x55e3a5b93590 con 0x55e3a5cad1a0 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.887+0000 7fd055400640 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55e3a5cad1a0 0x55e3a5cabeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.888+0000 7fd055400640 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55e3a5cad1a0 0x55e3a5cabeb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c9c2989fe65d5db2 server_cookie=ae276bd08395673 in_seq=0 out_seq=0 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.888+0000 7fd033400640 10 client.25364 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.888+0000 7fd033400640 1 -- 172.21.15.69:0/3521396022 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e3a5b93590 con 0x55e3a5cad1a0 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.888+0000 7fd033400640 10 client.25364 handle_command_reply: tid=0 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.888+0000 7fd05b60e740 1 client.25364 shutdown 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.888+0000 7fd05b60e740 1 -- 172.21.15.69:0/3521396022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 msgr2=0x55e3a5c3b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.888+0000 7fd05b60e740 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c3b1b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009710 tx=0x7fd04c013040 comp rx=0 tx=0).stop 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.889+0000 7fd05b60e740 1 -- 172.21.15.69:0/3521396022 shutdown_connections 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.889+0000 7fd05b60e740 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55e3a5cad1a0 0x55e3a5cabeb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.889+0000 7fd05b60e740 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e3a5c2cf50 0x55e3a5ca6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.889+0000 7fd05b60e740 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3a5c27de0 0x55e3a5ca49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.889+0000 7fd033400640 0 client.25364 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:10:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.889+0000 7fd05b60e740 1 --2- 172.21.15.69:0/3521396022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3a5c273f0 0x55e3a5c3b1b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.889+0000 7fd05b60e740 1 -- 172.21.15.69:0/3521396022 >> 172.21.15.69:0/3521396022 conn(0x55e3a5c22960 msgr2=0x55e3a5c22d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.889+0000 7fd05b60e740 1 -- 172.21.15.69:0/3521396022 shutdown_connections 2024-03-19T01:10:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.889+0000 7fd05b60e740 1 -- 172.21.15.69:0/3521396022 wait complete. 2024-03-19T01:10:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.889+0000 7fd05b60e740 20 client.25364 trim_cache size 0 max 16384 2024-03-19T01:10:07.893 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:10:07.893 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T01:10:07.893 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 4, 2024-03-19T01:10:07.893 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24476, 2024-03-19T01:10:07.893 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-03-19T01:10:07.894 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-03-19T01:10:07.894 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-03-19T01:10:07.894 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 138.11278027899999, 2024-03-19T01:10:07.894 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-03-19T01:10:07.894 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 93, 2024-03-19T01:10:07.894 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T01:10:07.894 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 204.648358143 2024-03-19T01:10:07.894 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:10:07.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 -- 172.21.15.69:0/2109110161 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0380a2800 msgr2=0x7fd0380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0380a2800 0x7fd0380a4cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd0440a3800 tx=0x7fd0440d2a40 comp rx=0 tx=0).stop 2024-03-19T01:10:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 -- 172.21.15.69:0/2109110161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd050101840 msgr2=0x7fd050197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd050101840 0x7fd050197b70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd04400e9c0 tx=0x7fd04400edc0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 -- 172.21.15.69:0/2109110161 shutdown_connections 2024-03-19T01:10:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd0380a2800 0x7fd0380a4cc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0501033f0 0x7fd05019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd050102a30 0x7fd0501980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 --2- 172.21.15.69:0/2109110161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd050101840 0x7fd050197b70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 -- 172.21.15.69:0/2109110161 >> 172.21.15.69:0/2109110161 conn(0x7fd0500fcdd0 msgr2=0x7fd050105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 -- 172.21.15.69:0/2109110161 shutdown_connections 2024-03-19T01:10:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:07.890+0000 7fd058800640 1 -- 172.21.15.69:0/2109110161 wait complete. 2024-03-19T01:10:07.911 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-19T01:10:07.978 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:07 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/4181090890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:07.978 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:07 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:07.978 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:07 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:07.978 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:07 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:07.978 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:07 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:07.979 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:07 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:07.979 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:07 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:07.979 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:07 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/625191183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:07.979 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:07 smithi069 ceph-mon[31727]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:10:08.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.120+0000 7f519b000640 1 -- 172.21.15.69:0/865696606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5194102a20 msgr2=0x7f5194102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.120+0000 7f519b000640 1 --2- 172.21.15.69:0/865696606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5194102a20 0x7f5194102ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5180009860 tx=0x7f518002bb70 comp rx=0 tx=0).stop 2024-03-19T01:10:08.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.120+0000 7f519b000640 1 -- 172.21.15.69:0/865696606 shutdown_connections 2024-03-19T01:10:08.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.120+0000 7f519b000640 1 --2- 172.21.15.69:0/865696606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51941033e0 0x7f5194109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.120+0000 7f519b000640 1 --2- 172.21.15.69:0/865696606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5194102a20 0x7f5194102ea0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.120+0000 7f519b000640 1 --2- 172.21.15.69:0/865696606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5194101830 0x7f5194101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.120+0000 7f519b000640 1 -- 172.21.15.69:0/865696606 >> 172.21.15.69:0/865696606 conn(0x7f51940fcdc0 msgr2=0x7f51940ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.120+0000 7f519b000640 1 -- 172.21.15.69:0/865696606 shutdown_connections 2024-03-19T01:10:08.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.121+0000 7f519b000640 1 -- 172.21.15.69:0/865696606 wait complete. 2024-03-19T01:10:08.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.121+0000 7f519b000640 1 Processor -- start 2024-03-19T01:10:08.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f519b000640 1 -- start start 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f519b000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194101830 0x7f51940755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f519b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5194102a20 0x7f5194073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f519b000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51941033e0 0x7f5194074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f519b000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f519410d490 con 0x7f51941033e0 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f519b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f519410d310 con 0x7f5194102a20 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f519b000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f519410d610 con 0x7f5194101830 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f5193e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194101830 0x7f51940755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f5193400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5194102a20 0x7f5194073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f5193e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194101830 0x7f51940755d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57400/0 (socket says 172.21.15.69:57400) 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f5193e00640 1 -- 172.21.15.69:0/2457089404 learned_addr learned my addr 172.21.15.69:0/2457089404 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f5199000640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51941033e0 0x7f5194074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.122+0000 7f5193400640 1 -- 172.21.15.69:0/2457089404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194101830 msgr2=0x7f51940755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.123+0000 7f5193400640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194101830 0x7f51940755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.123+0000 7f5193400640 1 -- 172.21.15.69:0/2457089404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51941033e0 msgr2=0x7f5194074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.123+0000 7f5193400640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51941033e0 0x7f5194074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.123+0000 7f5193400640 1 -- 172.21.15.69:0/2457089404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51940749e0 con 0x7f5194102a20 2024-03-19T01:10:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.123+0000 7f5193e00640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194101830 0x7f51940755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.123+0000 7f5199000640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51941033e0 0x7f5194074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.123+0000 7f5193400640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5194102a20 0x7f5194073c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f518002c0b0 tx=0x7f5180032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.123+0000 7f5190c00640 1 -- 172.21.15.69:0/2457089404 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180041020 con 0x7f5194102a20 2024-03-19T01:10:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.123+0000 7f5190c00640 1 -- 172.21.15.69:0/2457089404 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f518002c9f0 con 0x7f5194102a20 2024-03-19T01:10:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.123+0000 7f5190c00640 1 -- 172.21.15.69:0/2457089404 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180039040 con 0x7f5194102a20 2024-03-19T01:10:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.123+0000 7f519b000640 1 -- 172.21.15.69:0/2457089404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51941a1bc0 con 0x7f5194102a20 2024-03-19T01:10:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.124+0000 7f519b000640 1 -- 172.21.15.69:0/2457089404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51941a2090 con 0x7f5194102a20 2024-03-19T01:10:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.127+0000 7f5190c00640 1 -- 172.21.15.69:0/2457089404 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f5180044020 con 0x7f5194102a20 2024-03-19T01:10:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.128+0000 7f519ddec740 1 Processor -- start 2024-03-19T01:10:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.128+0000 7f519ddec740 1 -- start start 2024-03-19T01:10:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f519ddec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d756810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f519ddec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 0x55650d75b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f519ddec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 0x55650d75e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f519ddec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55650d6bad00 con 0x55650d7563f0 2024-03-19T01:10:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f519ddec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55650d663b00 con 0x55650d756de0 2024-03-19T01:10:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f519ddec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55650d6b6da0 con 0x55650d75bf50 2024-03-19T01:10:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f5193400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 0x55650d75e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f5193e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 0x55650d75b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f5193e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 0x55650d75b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33038/0 (socket says 172.21.15.69:33038) 2024-03-19T01:10:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f5193e00640 1 -- 172.21.15.69:0/4262348569 learned_addr learned my addr 172.21.15.69:0/4262348569 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f5199000640 1 --2- 172.21.15.69:0/4262348569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d756810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f5193e00640 1 -- 172.21.15.69:0/4262348569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 msgr2=0x55650d75e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f5190c00640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f51780a2880 0x7f51780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f5193e00640 1 --2- 172.21.15.69:0/4262348569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 0x55650d75e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f5193e00640 1 -- 172.21.15.69:0/4262348569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 msgr2=0x55650d756810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f5193e00640 1 --2- 172.21.15.69:0/4262348569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d756810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.129+0000 7f5193e00640 1 -- 172.21.15.69:0/4262348569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650d3cf2d0 con 0x55650d756de0 2024-03-19T01:10:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.130+0000 7f5190c00640 1 -- 172.21.15.69:0/2457089404 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f51800f0640 con 0x7f5194102a20 2024-03-19T01:10:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.130+0000 7f5193e00640 1 --2- 172.21.15.69:0/4262348569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 0x55650d75b8e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f51840115c0 tx=0x7f5184011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d8e29ee8a8f351f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.130+0000 7f5199000640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f51780a2880 0x7f51780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.130+0000 7f5189600640 1 -- 172.21.15.69:0/4262348569 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184004020 con 0x55650d756de0 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.130+0000 7f5189600640 1 -- 172.21.15.69:0/4262348569 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51840041c0 con 0x55650d756de0 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.130+0000 7f5189600640 1 -- 172.21.15.69:0/4262348569 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184003b30 con 0x55650d756de0 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.131+0000 7f519ddec740 1 -- 172.21.15.69:0/4262348569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 msgr2=0x55650d75b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.131+0000 7f5199000640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f51780a2880 0x7f51780a4d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55650d757640 tx=0x7f518c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.131+0000 7f519ddec740 1 --2- 172.21.15.69:0/4262348569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 0x55650d75b8e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f51840115c0 tx=0x7f5184011a80 comp rx=0 tx=0).stop 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.131+0000 7f519ddec740 1 -- 172.21.15.69:0/4262348569 shutdown_connections 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.131+0000 7f519ddec740 1 --2- 172.21.15.69:0/4262348569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 0x55650d75e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.131+0000 7f519ddec740 1 --2- 172.21.15.69:0/4262348569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 0x55650d75b8e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.131+0000 7f519ddec740 1 --2- 172.21.15.69:0/4262348569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d756810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.131+0000 7f519ddec740 1 -- 172.21.15.69:0/4262348569 >> 172.21.15.69:0/4262348569 conn(0x55650d751960 msgr2=0x55650d751d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.131+0000 7f519ddec740 1 -- 172.21.15.69:0/4262348569 shutdown_connections 2024-03-19T01:10:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.131+0000 7f519ddec740 1 -- 172.21.15.69:0/4262348569 wait complete. 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.132+0000 7f519ddec740 1 -- start start 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.132+0000 7f519ddec740 1 Processor -- start 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.132+0000 7f519ddec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d760900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.132+0000 7f519ddec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 0x55650d760e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.132+0000 7f519ddec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d75bf50 0x55650d767e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.132+0000 7f519ddec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55650d75ef00 con 0x55650d7563f0 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.132+0000 7f519ddec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55650d676ed0 con 0x55650d75bf50 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.133+0000 7f519ddec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55650d75f080 con 0x55650d756de0 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.133+0000 7f519ddec740 10 client.0 _ll_register_callbacks cb 0x55650d674c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.133+0000 7f5193e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d760900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.133+0000 7f5199000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d75bf50 0x55650d767e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.133+0000 7f5193e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d760900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55104/0 (socket says 172.21.15.69:55104) 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.133+0000 7f5199000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d75bf50 0x55650d767e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33044/0 (socket says 172.21.15.69:33044) 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.133+0000 7f5193e00640 1 -- 172.21.15.69:0/972174405 learned_addr learned my addr 172.21.15.69:0/972174405 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.133+0000 7f5193400640 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 0x55650d760e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.133+0000 7f5199000640 1 -- 172.21.15.69:0/972174405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 msgr2=0x55650d760e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.133+0000 7f5199000640 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 0x55650d760e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.134+0000 7f5199000640 1 -- 172.21.15.69:0/972174405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 msgr2=0x55650d760900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.134+0000 7f5199000640 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d760900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.134+0000 7f5199000640 1 -- 172.21.15.69:0/972174405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650d4996b0 con 0x55650d75bf50 2024-03-19T01:10:08.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.134+0000 7f5193e00640 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d760900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:08.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.134+0000 7f5199000640 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d75bf50 0x55650d767e80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f518c005fd0 tx=0x7f518c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2d51b219e7fb6fe server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.134+0000 7f517e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:10:08.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.134+0000 7f517e000640 1 -- 172.21.15.69:0/972174405 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c011da0 con 0x55650d75bf50 2024-03-19T01:10:08.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.134+0000 7f517e000640 1 -- 172.21.15.69:0/972174405 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f518c01e420 con 0x55650d75bf50 2024-03-19T01:10:08.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.134+0000 7f519ddec740 1 -- 172.21.15.69:0/972174405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51840041c0 con 0x55650d75bf50 2024-03-19T01:10:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.135+0000 7f517e000640 1 -- 172.21.15.69:0/972174405 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f518c030050 con 0x55650d75bf50 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.135+0000 7f519ddec740 10 client.25376 fetch_fsmap learned FSMap version 28 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.135+0000 7f519ddec740 1 -- 172.21.15.69:0/972174405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55650d76b260 con 0x55650d75bf50 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.135+0000 7f517e000640 1 -- 172.21.15.69:0/972174405 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51840041c0 con 0x55650d75bf50 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.135+0000 7f517e000640 1 -- 172.21.15.69:0/972174405 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f518c01f7c0 con 0x55650d75bf50 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.135+0000 7f519ddec740 10 client.25376 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.135+0000 7f519ddec740 10 client.25376 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.135+0000 7f519ddec740 20 client.25376 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.135+0000 7f519ddec740 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55650d76d960 0x55650d76edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.135+0000 7f519ddec740 4 client.25376 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.135+0000 7f519ddec740 1 -- 172.21.15.69:0/972174405 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55650d6c8240 con 0x55650d76d960 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.136+0000 7f5193e00640 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55650d76d960 0x55650d76edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.136+0000 7f5193e00640 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55650d76d960 0x55650d76edd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=67ef3e19f7f0f45f server_cookie=a84b2b9c9d0e6a8e in_seq=0 out_seq=0 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.136+0000 7f517e000640 10 client.25376 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.137+0000 7f517e000640 1 -- 172.21.15.69:0/972174405 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55650d6c8240 con 0x55650d76d960 2024-03-19T01:10:08.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.137+0000 7f517e000640 10 client.25376 handle_command_reply: tid=0 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.137+0000 7f519ddec740 1 client.25376 shutdown 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.137+0000 7f519ddec740 1 -- 172.21.15.69:0/972174405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d75bf50 msgr2=0x55650d767e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.137+0000 7f519ddec740 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d75bf50 0x55650d767e80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f518c005fd0 tx=0x7f518c004ab0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.137+0000 7f519ddec740 1 -- 172.21.15.69:0/972174405 shutdown_connections 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.137+0000 7f519ddec740 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55650d76d960 0x55650d76edd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.138+0000 7f517e000640 0 client.25376 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.138+0000 7f519ddec740 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d75bf50 0x55650d767e80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.138+0000 7f519ddec740 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 0x55650d760e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.138+0000 7f519ddec740 1 --2- 172.21.15.69:0/972174405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d760900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.138+0000 7f519ddec740 1 -- 172.21.15.69:0/972174405 >> 172.21.15.69:0/972174405 conn(0x55650d751960 msgr2=0x55650d750aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.138+0000 7f519ddec740 1 -- 172.21.15.69:0/972174405 shutdown_connections 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.138+0000 7f519ddec740 1 -- 172.21.15.69:0/972174405 wait complete. 2024-03-19T01:10:08.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.138+0000 7f519ddec740 20 client.25376 trim_cache size 0 max 16384 2024-03-19T01:10:08.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.159+0000 7f519ddec740 1 Processor -- start 2024-03-19T01:10:08.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.159+0000 7f519ddec740 1 -- start start 2024-03-19T01:10:08.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.159+0000 7f519ddec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d7563f0 0x55650d75ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.159+0000 7f519ddec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 0x55650d767e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.159+0000 7f519ddec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d75bf50 0x55650d751d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.159+0000 7f519ddec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55650d76d110 con 0x55650d75bf50 2024-03-19T01:10:08.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.159+0000 7f519ddec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55650d76d7e0 con 0x55650d7563f0 2024-03-19T01:10:08.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.159+0000 7f519ddec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55650d76deb0 con 0x55650d756de0 2024-03-19T01:10:08.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5193400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 0x55650d767e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5193e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d7563f0 0x55650d75ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5193400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 0x55650d767e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57442/0 (socket says 172.21.15.69:57442) 2024-03-19T01:10:08.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5193e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d7563f0 0x55650d75ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33046/0 (socket says 172.21.15.69:33046) 2024-03-19T01:10:08.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5193400640 1 -- 172.21.15.69:0/655623957 learned_addr learned my addr 172.21.15.69:0/655623957 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5193400640 1 -- 172.21.15.69:0/655623957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d7563f0 msgr2=0x55650d75ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5193400640 1 --2- 172.21.15.69:0/655623957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d7563f0 0x55650d75ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5193400640 1 -- 172.21.15.69:0/655623957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d75bf50 msgr2=0x55650d751d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5199000640 1 --2- 172.21.15.69:0/655623957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d75bf50 0x55650d751d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5193400640 1 --2- 172.21.15.69:0/655623957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d75bf50 0x55650d751d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5193400640 1 -- 172.21.15.69:0/655623957 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650d76ea30 con 0x55650d756de0 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.160+0000 7f5199000640 1 --2- 172.21.15.69:0/655623957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d75bf50 0x55650d751d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.161+0000 7f5193400640 1 --2- 172.21.15.69:0/655623957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 0x55650d767e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f51800edca0 tx=0x7f51800f60e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdf0533e6213db96 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.161+0000 7f5188c00640 1 -- 172.21.15.69:0/655623957 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51800bc9a0 con 0x55650d756de0 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.161+0000 7f5188c00640 1 -- 172.21.15.69:0/655623957 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51800bd2d0 con 0x55650d756de0 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.161+0000 7f5188c00640 1 -- 172.21.15.69:0/655623957 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51800bc530 con 0x55650d756de0 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.162+0000 7f519ddec740 1 -- 172.21.15.69:0/655623957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 msgr2=0x55650d767e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.162+0000 7f519ddec740 1 --2- 172.21.15.69:0/655623957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 0x55650d767e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f51800edca0 tx=0x7f51800f60e0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.162+0000 7f519ddec740 1 -- 172.21.15.69:0/655623957 shutdown_connections 2024-03-19T01:10:08.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.162+0000 7f519ddec740 1 --2- 172.21.15.69:0/655623957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d75bf50 0x55650d751d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.162+0000 7f519ddec740 1 --2- 172.21.15.69:0/655623957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d756de0 0x55650d767e40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.162+0000 7f519ddec740 1 --2- 172.21.15.69:0/655623957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d7563f0 0x55650d75ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.162+0000 7f519ddec740 1 -- 172.21.15.69:0/655623957 >> 172.21.15.69:0/655623957 conn(0x55650d751960 msgr2=0x55650d757790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.162+0000 7f519ddec740 1 -- 172.21.15.69:0/655623957 shutdown_connections 2024-03-19T01:10:08.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.162+0000 7f519ddec740 1 -- 172.21.15.69:0/655623957 wait complete. 2024-03-19T01:10:08.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.162+0000 7f519ddec740 1 -- start start 2024-03-19T01:10:08.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.162+0000 7f519ddec740 1 Processor -- start 2024-03-19T01:10:08.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f519ddec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d763440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f519ddec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 0x55650d763980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f519ddec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 0x55650d7669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f519ddec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55650d76ec50 con 0x55650d7563f0 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f519ddec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55650d76e580 con 0x55650d756de0 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f519ddec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55650d76f320 con 0x55650d75bf50 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f519ddec740 10 client.0 _ll_register_callbacks cb 0x7f518c009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f5199000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 0x55650d7669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f5199000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 0x55650d7669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57454/0 (socket says 172.21.15.69:57454) 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f5199000640 1 -- 172.21.15.69:0/3102895735 learned_addr learned my addr 172.21.15.69:0/3102895735 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f5193400640 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 0x55650d763980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f5193e00640 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d763440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.163+0000 7f5199000640 1 -- 172.21.15.69:0/3102895735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 msgr2=0x55650d763980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.164+0000 7f5199000640 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 0x55650d763980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.164+0000 7f5199000640 1 -- 172.21.15.69:0/3102895735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 msgr2=0x55650d763440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.164+0000 7f5199000640 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d763440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.164+0000 7f5199000640 1 -- 172.21.15.69:0/3102895735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650d766f90 con 0x55650d75bf50 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.164+0000 7f5193e00640 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d763440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.164+0000 7f5199000640 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 0x55650d7669a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f518c000f80 tx=0x7f518c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4831301909d602e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.164+0000 7f517e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.164+0000 7f517e000640 1 -- 172.21.15.69:0/3102895735 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c011da0 con 0x55650d75bf50 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.164+0000 7f517e000640 1 -- 172.21.15.69:0/3102895735 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f518c01f420 con 0x55650d75bf50 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.164+0000 7f519ddec740 1 -- 172.21.15.69:0/3102895735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51800bd2d0 con 0x55650d75bf50 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.165+0000 7f517e000640 1 -- 172.21.15.69:0/3102895735 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f518c01f620 con 0x55650d75bf50 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.165+0000 7f519ddec740 10 client.25387 fetch_fsmap learned FSMap version 28 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.165+0000 7f519ddec740 1 -- 172.21.15.69:0/3102895735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55650d7e46e0 con 0x55650d75bf50 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.165+0000 7f517e000640 1 -- 172.21.15.69:0/3102895735 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51800bd2d0 con 0x55650d75bf50 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.165+0000 7f517e000640 1 -- 172.21.15.69:0/3102895735 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f518c034430 con 0x55650d75bf50 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.165+0000 7f519ddec740 10 client.25387 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.165+0000 7f519ddec740 10 client.25387 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-19T01:10:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.165+0000 7f519ddec740 20 client.25387 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.165+0000 7f519ddec740 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55650d7e96b0 0x55650d7e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.165+0000 7f519ddec740 4 client.25387 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.165+0000 7f519ddec740 1 -- 172.21.15.69:0/3102895735 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "status"}) -- 0x55650d6c2590 con 0x55650d7e96b0 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.166+0000 7f5193e00640 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55650d7e96b0 0x55650d7e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.166+0000 7f517e000640 10 client.25387 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.166+0000 7f5193e00640 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55650d7e96b0 0x55650d7e5af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=319bd8a396e8fff2 server_cookie=47ed307e2a61cecc in_seq=0 out_seq=0 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.166+0000 7f517e000640 1 -- 172.21.15.69:0/3102895735 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55650d6c2590 con 0x55650d7e96b0 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f517e000640 10 client.25387 handle_command_reply: tid=0 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 1 client.25387 shutdown 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 1 -- 172.21.15.69:0/3102895735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 msgr2=0x55650d7669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 0x55650d7669a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f518c000f80 tx=0x7f518c011810 comp rx=0 tx=0).stop 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 1 -- 172.21.15.69:0/3102895735 shutdown_connections 2024-03-19T01:10:08.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55650d75bf50 0x55650d7669a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55650d7e96b0 0x55650d7e5af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f517e000640 0 client.25387 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:10:08.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55650d756de0 0x55650d763980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 1 --2- 172.21.15.69:0/3102895735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55650d7563f0 0x55650d763440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 1 -- 172.21.15.69:0/3102895735 >> 172.21.15.69:0/3102895735 conn(0x55650d751960 msgr2=0x55650d751d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 1 -- 172.21.15.69:0/3102895735 shutdown_connections 2024-03-19T01:10:08.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 1 -- 172.21.15.69:0/3102895735 wait complete. 2024-03-19T01:10:08.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.167+0000 7f519ddec740 20 client.25387 trim_cache size 0 max 16384 2024-03-19T01:10:08.172 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:10:08.172 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T01:10:08.173 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 3, 2024-03-19T01:10:08.173 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24494, 2024-03-19T01:10:08.173 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-03-19T01:10:08.173 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-03-19T01:10:08.173 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-03-19T01:10:08.173 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 139.40631817799999, 2024-03-19T01:10:08.173 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-03-19T01:10:08.173 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 93, 2024-03-19T01:10:08.173 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T01:10:08.173 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 202.09626051000001 2024-03-19T01:10:08.173 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:10:08.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.168+0000 7f519b000640 1 -- 172.21.15.69:0/2457089404 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f51780a2880 msgr2=0x7f51780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.168+0000 7f519b000640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f51780a2880 0x7f51780a4d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55650d757640 tx=0x7f518c004a40 comp rx=0 tx=0).stop 2024-03-19T01:10:08.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.168+0000 7f519b000640 1 -- 172.21.15.69:0/2457089404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5194102a20 msgr2=0x7f5194073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.168+0000 7f519b000640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5194102a20 0x7f5194073c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f518002c0b0 tx=0x7f5180032430 comp rx=0 tx=0).stop 2024-03-19T01:10:08.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.169+0000 7f519b000640 1 -- 172.21.15.69:0/2457089404 shutdown_connections 2024-03-19T01:10:08.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.169+0000 7f519b000640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f51780a2880 0x7f51780a4d40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.169+0000 7f519b000640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51941033e0 0x7f5194074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.169+0000 7f519b000640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5194102a20 0x7f5194073c10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.169+0000 7f519b000640 1 --2- 172.21.15.69:0/2457089404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194101830 0x7f51940755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.169+0000 7f519b000640 1 -- 172.21.15.69:0/2457089404 >> 172.21.15.69:0/2457089404 conn(0x7f51940fcdc0 msgr2=0x7f5194105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.169+0000 7f519b000640 1 -- 172.21.15.69:0/2457089404 shutdown_connections 2024-03-19T01:10:08.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.169+0000 7f519b000640 1 -- 172.21.15.69:0/2457089404 wait complete. 2024-03-19T01:10:08.190 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-19T01:10:08.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:07 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/4181090890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:08.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:07 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:07 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:07 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:07 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:07 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:07 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:07 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/625191183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:08.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:07 smithi163 ceph-mon[67428]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:10:08.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:07 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/4181090890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:08.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:07 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:07 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:07 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:07 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:07 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:07 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:08.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:07 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/625191183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:08.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:07 smithi086 ceph-mon[66829]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-19T01:10:08.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.392+0000 7f17f1400640 1 -- 172.21.15.69:0/1374701043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17ec104c50 msgr2=0x7f17ec1050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.392+0000 7f17f1400640 1 --2- 172.21.15.69:0/1374701043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17ec104c50 0x7f17ec1050d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009900 tx=0x7f17d802bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:08.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.392+0000 7f17f1400640 1 -- 172.21.15.69:0/1374701043 shutdown_connections 2024-03-19T01:10:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.392+0000 7f17f1400640 1 --2- 172.21.15.69:0/1374701043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec105610 0x7f17ec10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.392+0000 7f17f1400640 1 --2- 172.21.15.69:0/1374701043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17ec104c50 0x7f17ec1050d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.392+0000 7f17f1400640 1 --2- 172.21.15.69:0/1374701043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17ec103a60 0x7f17ec103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.392+0000 7f17f1400640 1 -- 172.21.15.69:0/1374701043 >> 172.21.15.69:0/1374701043 conn(0x7f17ec0fefd0 msgr2=0x7f17ec101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.392+0000 7f17f1400640 1 -- 172.21.15.69:0/1374701043 shutdown_connections 2024-03-19T01:10:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.392+0000 7f17f1400640 1 -- 172.21.15.69:0/1374701043 wait complete. 2024-03-19T01:10:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.393+0000 7f17f1400640 1 Processor -- start 2024-03-19T01:10:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.393+0000 7f17f1400640 1 -- start start 2024-03-19T01:10:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.393+0000 7f17f1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17ec103a60 0x7f17ec199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17f1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17ec104c50 0x7f17ec19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17f1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec105610 0x7f17ec1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17f1400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec10f710 con 0x7f17ec103a60 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17f1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec10f590 con 0x7f17ec104c50 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17f1400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec10f890 con 0x7f17ec105610 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17ec104c50 0x7f17ec19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17ec104c50 0x7f17ec19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33066/0 (socket says 172.21.15.69:33066) 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17e9e00640 1 -- 172.21.15.69:0/4164805568 learned_addr learned my addr 172.21.15.69:0/4164805568 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17eb200640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec105610 0x7f17ec1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17ea800640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17ec103a60 0x7f17ec199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17e9e00640 1 -- 172.21.15.69:0/4164805568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec105610 msgr2=0x7f17ec1a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17e9e00640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec105610 0x7f17ec1a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17e9e00640 1 -- 172.21.15.69:0/4164805568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17ec103a60 msgr2=0x7f17ec199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17e9e00640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17ec103a60 0x7f17ec199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.394+0000 7f17e9e00640 1 -- 172.21.15.69:0/4164805568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ec1a19b0 con 0x7f17ec104c50 2024-03-19T01:10:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.395+0000 7f17e9e00640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17ec104c50 0x7f17ec19a2e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f17d8000c00 tx=0x7f17d802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.395+0000 7f17e3400640 1 -- 172.21.15.69:0/4164805568 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8043020 con 0x7f17ec104c50 2024-03-19T01:10:08.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.395+0000 7f17e3400640 1 -- 172.21.15.69:0/4164805568 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d80054e0 con 0x7f17ec104c50 2024-03-19T01:10:08.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.395+0000 7f17f1400640 1 -- 172.21.15.69:0/4164805568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ec1a1c40 con 0x7f17ec104c50 2024-03-19T01:10:08.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.395+0000 7f17e3400640 1 -- 172.21.15.69:0/4164805568 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8003c40 con 0x7f17ec104c50 2024-03-19T01:10:08.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.395+0000 7f17f1400640 1 -- 172.21.15.69:0/4164805568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ec1a2190 con 0x7f17ec104c50 2024-03-19T01:10:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.398+0000 7f17f41a8740 1 Processor -- start 2024-03-19T01:10:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.398+0000 7f17f41a8740 1 -- start start 2024-03-19T01:10:08.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353018810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 0x55a35301d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a35301df50 0x55a3530203e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17f41a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a352f7cd00 con 0x55a35301df50 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17f41a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a352f25b00 con 0x55a3530183f0 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17f41a8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a352f78da0 con 0x55a353018de0 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17eb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353018810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 0x55a35301d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17eb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353018810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33070/0 (socket says 172.21.15.69:33070) 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 0x55a35301d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57476/0 (socket says 172.21.15.69:57476) 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17eb200640 1 -- 172.21.15.69:0/767085978 learned_addr learned my addr 172.21.15.69:0/767085978 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17eb200640 1 -- 172.21.15.69:0/767085978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 msgr2=0x55a35301d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17eb200640 1 --2- 172.21.15.69:0/767085978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 0x55a35301d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17eb200640 1 -- 172.21.15.69:0/767085978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a35301df50 msgr2=0x55a3530203e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17eb200640 1 --2- 172.21.15.69:0/767085978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a35301df50 0x55a3530203e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17eb200640 1 -- 172.21.15.69:0/767085978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a352c912d0 con 0x55a3530183f0 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.399+0000 7f17ea800640 1 --2- 172.21.15.69:0/767085978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 0x55a35301d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.400+0000 7f17eb200640 1 --2- 172.21.15.69:0/767085978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353018810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009fd0 tx=0x7f17e40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0aa335d8981b90 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.400+0000 7f17e3400640 1 -- 172.21.15.69:0/4164805568 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f17d8046020 con 0x7f17ec104c50 2024-03-19T01:10:08.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.400+0000 7f17d7400640 1 -- 172.21.15.69:0/767085978 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4004d20 con 0x55a3530183f0 2024-03-19T01:10:08.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.400+0000 7f17d7400640 1 -- 172.21.15.69:0/767085978 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e4004050 con 0x55a3530183f0 2024-03-19T01:10:08.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.400+0000 7f17d7400640 1 -- 172.21.15.69:0/767085978 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4005280 con 0x55a3530183f0 2024-03-19T01:10:08.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.401+0000 7f17f41a8740 1 -- 172.21.15.69:0/767085978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 msgr2=0x55a353018810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.401+0000 7f17f41a8740 1 --2- 172.21.15.69:0/767085978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353018810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009fd0 tx=0x7f17e40118d0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.401+0000 7f17e3400640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f17d00a2880 0x7f17d00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.401+0000 7f17e3400640 1 -- 172.21.15.69:0/4164805568 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f17d80f0e60 con 0x7f17ec104c50 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.401+0000 7f17f41a8740 1 -- 172.21.15.69:0/767085978 shutdown_connections 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.402+0000 7f17f41a8740 1 --2- 172.21.15.69:0/767085978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a35301df50 0x55a3530203e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.402+0000 7f17f41a8740 1 --2- 172.21.15.69:0/767085978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 0x55a35301d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.402+0000 7f17f41a8740 1 --2- 172.21.15.69:0/767085978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353018810 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.402+0000 7f17f41a8740 1 -- 172.21.15.69:0/767085978 >> 172.21.15.69:0/767085978 conn(0x55a353013960 msgr2=0x55a353013d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.402+0000 7f17ea800640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f17d00a2880 0x7f17d00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.402+0000 7f17f41a8740 1 -- 172.21.15.69:0/767085978 shutdown_connections 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.402+0000 7f17f41a8740 1 -- 172.21.15.69:0/767085978 wait complete. 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.402+0000 7f17ea800640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f17d00a2880 0x7f17d00a4d40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55a3530148d0 tx=0x7f17dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.403+0000 7f17f41a8740 1 -- start start 2024-03-19T01:10:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.403+0000 7f17f41a8740 1 Processor -- start 2024-03-19T01:10:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.403+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353022960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.403+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 0x55a353022ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.403+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a35301df50 0x55a353029ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.403+0000 7f17f41a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a353020f00 con 0x55a35301df50 2024-03-19T01:10:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.403+0000 7f17f41a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a352f38ed0 con 0x55a3530183f0 2024-03-19T01:10:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.403+0000 7f17f41a8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a353021080 con 0x55a353018de0 2024-03-19T01:10:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.403+0000 7f17f41a8740 10 client.0 _ll_register_callbacks cb 0x55a352f36c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353022960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353022960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33080/0 (socket says 172.21.15.69:33080) 2024-03-19T01:10:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17e9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 0x55a353022ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17ea800640 1 -- 172.21.15.69:0/100441091 learned_addr learned my addr 172.21.15.69:0/100441091 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17eb200640 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a35301df50 0x55a353029ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17e9e00640 1 -- 172.21.15.69:0/100441091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 msgr2=0x55a353022960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17e9e00640 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353022960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17e9e00640 1 -- 172.21.15.69:0/100441091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a35301df50 msgr2=0x55a353029ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17e9e00640 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a35301df50 0x55a353029ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17e9e00640 1 -- 172.21.15.69:0/100441091 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a352d5b6b0 con 0x55a353018de0 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17ea800640 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353022960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17eb200640 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a35301df50 0x55a353029ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17e9e00640 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 0x55a353022ea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f17d80ee600 tx=0x7f17d80bcf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ea22a4f36c3bf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.404+0000 7f17cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.405+0000 7f17cbe00640 1 -- 172.21.15.69:0/100441091 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8004250 con 0x55a353018de0 2024-03-19T01:10:08.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.405+0000 7f17f41a8740 1 -- 172.21.15.69:0/100441091 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e4004050 con 0x55a353018de0 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.405+0000 7f17cbe00640 1 -- 172.21.15.69:0/100441091 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f17d80c1120 con 0x55a353018de0 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.405+0000 7f17cbe00640 1 -- 172.21.15.69:0/100441091 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17d80035f0 con 0x55a353018de0 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.405+0000 7f17f41a8740 10 client.25399 fetch_fsmap learned FSMap version 28 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.405+0000 7f17f41a8740 1 -- 172.21.15.69:0/100441091 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a353021020 con 0x55a353018de0 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.405+0000 7f17cbe00640 1 -- 172.21.15.69:0/100441091 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e4004050 con 0x55a353018de0 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.405+0000 7f17cbe00640 1 -- 172.21.15.69:0/100441091 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f17d80d60c0 con 0x55a353018de0 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.406+0000 7f17f41a8740 10 client.25399 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.406+0000 7f17f41a8740 10 client.25399 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.406+0000 7f17f41a8740 20 client.25399 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.406+0000 7f17f41a8740 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a353038170 0x55a3530395e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.406+0000 7f17f41a8740 4 client.25399 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.406+0000 7f17f41a8740 1 -- 172.21.15.69:0/100441091 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a352f8a240 con 0x55a353038170 2024-03-19T01:10:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.406+0000 7f17ea800640 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a353038170 0x55a3530395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.407+0000 7f17ea800640 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a353038170 0x55a3530395e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d22d3734e4557c66 server_cookie=34973a0116660088 in_seq=0 out_seq=0 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.407+0000 7f17cbe00640 10 client.25399 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.407+0000 7f17cbe00640 1 -- 172.21.15.69:0/100441091 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a352f8a240 con 0x55a353038170 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.407+0000 7f17cbe00640 10 client.25399 handle_command_reply: tid=0 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.407+0000 7f17f41a8740 1 client.25399 shutdown 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17f41a8740 1 -- 172.21.15.69:0/100441091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 msgr2=0x55a353022ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17f41a8740 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 0x55a353022ea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f17d80ee600 tx=0x7f17d80bcf90 comp rx=0 tx=0).stop 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17f41a8740 1 -- 172.21.15.69:0/100441091 shutdown_connections 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17f41a8740 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a353038170 0x55a3530395e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17f41a8740 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a35301df50 0x55a353029ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17f41a8740 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a353018de0 0x55a353022ea0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17f41a8740 1 --2- 172.21.15.69:0/100441091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3530183f0 0x55a353022960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17f41a8740 1 -- 172.21.15.69:0/100441091 >> 172.21.15.69:0/100441091 conn(0x55a353013960 msgr2=0x55a353012aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17cbe00640 0 client.25399 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:10:08.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17f41a8740 1 -- 172.21.15.69:0/100441091 shutdown_connections 2024-03-19T01:10:08.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17f41a8740 1 -- 172.21.15.69:0/100441091 wait complete. 2024-03-19T01:10:08.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.408+0000 7f17f41a8740 20 client.25399 trim_cache size 0 max 16384 2024-03-19T01:10:08.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.436+0000 7f17f41a8740 1 Processor -- start 2024-03-19T01:10:08.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.436+0000 7f17f41a8740 1 -- start start 2024-03-19T01:10:08.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a3530183f0 0x55a353013d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a353018de0 0x55a353029ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a35301df50 0x55a353036d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17f41a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a353037920 con 0x55a353018de0 2024-03-19T01:10:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17f41a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a353037ff0 con 0x55a35301df50 2024-03-19T01:10:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17f41a8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a3530386c0 con 0x55a3530183f0 2024-03-19T01:10:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a3530183f0 0x55a353013d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a3530183f0 0x55a353013d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57490/0 (socket says 172.21.15.69:57490) 2024-03-19T01:10:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17e9e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a353018de0 0x55a353029ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17ea800640 1 -- 172.21.15.69:0/319221387 learned_addr learned my addr 172.21.15.69:0/319221387 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17eb200640 1 --2- 172.21.15.69:0/319221387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a35301df50 0x55a353036d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17ea800640 1 -- 172.21.15.69:0/319221387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a35301df50 msgr2=0x55a353036d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.437+0000 7f17ea800640 1 --2- 172.21.15.69:0/319221387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a35301df50 0x55a353036d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.438+0000 7f17ea800640 1 -- 172.21.15.69:0/319221387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a353018de0 msgr2=0x55a353029ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.438+0000 7f17ea800640 1 --2- 172.21.15.69:0/319221387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a353018de0 0x55a353029ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.438+0000 7f17ea800640 1 -- 172.21.15.69:0/319221387 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3530191c0 con 0x55a3530183f0 2024-03-19T01:10:08.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.438+0000 7f17ea800640 1 --2- 172.21.15.69:0/319221387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a3530183f0 0x55a353013d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55a353039f70 tx=0x7f17dc01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=915e4bf0462065ad server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.438+0000 7f17d6a00640 1 -- 172.21.15.69:0/319221387 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc02e190 con 0x55a3530183f0 2024-03-19T01:10:08.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.438+0000 7f17d6a00640 1 -- 172.21.15.69:0/319221387 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17dc032030 con 0x55a3530183f0 2024-03-19T01:10:08.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.438+0000 7f17d6a00640 1 -- 172.21.15.69:0/319221387 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc02ce20 con 0x55a3530183f0 2024-03-19T01:10:08.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.439+0000 7f17f41a8740 1 -- 172.21.15.69:0/319221387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a3530183f0 msgr2=0x55a353013d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.439+0000 7f17f41a8740 1 --2- 172.21.15.69:0/319221387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a3530183f0 0x55a353013d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55a353039f70 tx=0x7f17dc01b000 comp rx=0 tx=0).stop 2024-03-19T01:10:08.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.439+0000 7f17f41a8740 1 -- 172.21.15.69:0/319221387 shutdown_connections 2024-03-19T01:10:08.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.439+0000 7f17f41a8740 1 --2- 172.21.15.69:0/319221387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a35301df50 0x55a353036d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.439+0000 7f17f41a8740 1 --2- 172.21.15.69:0/319221387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a353018de0 0x55a353029ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.439+0000 7f17f41a8740 1 --2- 172.21.15.69:0/319221387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a3530183f0 0x55a353013d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.439+0000 7f17f41a8740 1 -- 172.21.15.69:0/319221387 >> 172.21.15.69:0/319221387 conn(0x55a353013960 msgr2=0x55a3530197e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.439+0000 7f17f41a8740 1 -- 172.21.15.69:0/319221387 shutdown_connections 2024-03-19T01:10:08.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.439+0000 7f17f41a8740 1 -- 172.21.15.69:0/319221387 wait complete. 2024-03-19T01:10:08.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.440+0000 7f17f41a8740 1 -- start start 2024-03-19T01:10:08.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.440+0000 7f17f41a8740 1 Processor -- start 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.441+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3530183f0 0x55a353024f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.441+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a353018de0 0x55a353025480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.441+0000 7f17f41a8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a35301df50 0x55a353028430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.441+0000 7f17f41a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a353039460 con 0x55a3530183f0 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.441+0000 7f17f41a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a353038d90 con 0x55a353018de0 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.441+0000 7f17f41a8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a353039b30 con 0x55a35301df50 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.441+0000 7f17ea800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3530183f0 0x55a353024f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.441+0000 7f17ea800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3530183f0 0x55a353024f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55182/0 (socket says 172.21.15.69:55182) 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.441+0000 7f17ea800640 1 -- 172.21.15.69:0/997441188 learned_addr learned my addr 172.21.15.69:0/997441188 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.441+0000 7f17f41a8740 10 client.0 _ll_register_callbacks cb 0x7f17d80b96a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.442+0000 7f17ea800640 1 -- 172.21.15.69:0/997441188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a35301df50 msgr2=0x55a353028430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.442+0000 7f17ea800640 1 --2- 172.21.15.69:0/997441188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a35301df50 0x55a353028430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.442+0000 7f17ea800640 1 -- 172.21.15.69:0/997441188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a353018de0 msgr2=0x55a353025480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.442+0000 7f17ea800640 1 --2- 172.21.15.69:0/997441188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a353018de0 0x55a353025480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.442+0000 7f17ea800640 1 -- 172.21.15.69:0/997441188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a353028a20 con 0x55a3530183f0 2024-03-19T01:10:08.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.442+0000 7f17ea800640 1 --2- 172.21.15.69:0/997441188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3530183f0 0x55a353024f40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17dc01e7f0 tx=0x7f17dc009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e5881b89023238 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.442+0000 7f17cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:10:08.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.443+0000 7f17cbe00640 1 -- 172.21.15.69:0/997441188 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc03e070 con 0x55a3530183f0 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.443+0000 7f17cbe00640 1 -- 172.21.15.69:0/997441188 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f17dc02cba0 con 0x55a3530183f0 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.443+0000 7f17f41a8740 1 -- 172.21.15.69:0/997441188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17dc032030 con 0x55a3530183f0 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.443+0000 7f17cbe00640 1 -- 172.21.15.69:0/997441188 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17dc04a050 con 0x55a3530183f0 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.443+0000 7f17f41a8740 10 client.15276 fetch_fsmap learned FSMap version 28 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.443+0000 7f17f41a8740 1 -- 172.21.15.69:0/997441188 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a353029d90 con 0x55a3530183f0 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.443+0000 7f17cbe00640 1 -- 172.21.15.69:0/997441188 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17dc032030 con 0x55a3530183f0 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.444+0000 7f17cbe00640 1 -- 172.21.15.69:0/997441188 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f17dc02e600 con 0x55a3530183f0 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.444+0000 7f17f41a8740 10 client.15276 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.444+0000 7f17f41a8740 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.444+0000 7f17f41a8740 20 client.15276 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.444+0000 7f17f41a8740 1 --2- 172.21.15.69:0/997441188 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a3530a2290 0x55a3530a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.444+0000 7f17f41a8740 4 client.15276 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:10:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.444+0000 7f17f41a8740 1 -- 172.21.15.69:0/997441188 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "status"}) -- 0x55a352f84590 con 0x55a3530a2290 2024-03-19T01:10:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.444+0000 7f17e9e00640 1 --2- 172.21.15.69:0/997441188 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a3530a2290 0x55a3530a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17cbe00640 10 client.15276 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:10:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17e9e00640 1 --2- 172.21.15.69:0/997441188 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a3530a2290 0x55a3530a0fa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=832f64862c25f2d7 server_cookie=ce54666654cf7143 in_seq=0 out_seq=0 2024-03-19T01:10:08.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17cbe00640 1 -- 172.21.15.69:0/997441188 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a352f84590 con 0x55a3530a2290 2024-03-19T01:10:08.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17cbe00640 10 client.15276 handle_command_reply: tid=0 2024-03-19T01:10:08.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17f41a8740 1 client.15276 shutdown 2024-03-19T01:10:08.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17f41a8740 1 -- 172.21.15.69:0/997441188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3530183f0 msgr2=0x55a353024f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17f41a8740 1 --2- 172.21.15.69:0/997441188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3530183f0 0x55a353024f40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17dc01e7f0 tx=0x7f17dc009990 comp rx=0 tx=0).stop 2024-03-19T01:10:08.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17f41a8740 1 -- 172.21.15.69:0/997441188 shutdown_connections 2024-03-19T01:10:08.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17f41a8740 1 --2- 172.21.15.69:0/997441188 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a3530a2290 0x55a3530a0fa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17f41a8740 1 --2- 172.21.15.69:0/997441188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a35301df50 0x55a353028430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17f41a8740 1 --2- 172.21.15.69:0/997441188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a353018de0 0x55a353025480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17f41a8740 1 --2- 172.21.15.69:0/997441188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3530183f0 0x55a353024f40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17cbe00640 0 client.15276 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:10:08.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.445+0000 7f17f41a8740 1 -- 172.21.15.69:0/997441188 >> 172.21.15.69:0/997441188 conn(0x55a353013960 msgr2=0x55a353013d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.446+0000 7f17f41a8740 1 -- 172.21.15.69:0/997441188 shutdown_connections 2024-03-19T01:10:08.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.446+0000 7f17f41a8740 1 -- 172.21.15.69:0/997441188 wait complete. 2024-03-19T01:10:08.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.446+0000 7f17f41a8740 20 client.15276 trim_cache size 0 max 16384 2024-03-19T01:10:08.449 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:10:08.449 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T01:10:08.449 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-03-19T01:10:08.449 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24550, 2024-03-19T01:10:08.449 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-03-19T01:10:08.450 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-03-19T01:10:08.450 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-03-19T01:10:08.450 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 140.64535826299999, 2024-03-19T01:10:08.450 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-03-19T01:10:08.450 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 93, 2024-03-19T01:10:08.450 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T01:10:08.450 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 190.56442896999999 2024-03-19T01:10:08.450 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:10:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.446+0000 7f17f1400640 1 -- 172.21.15.69:0/4164805568 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f17d00a2880 msgr2=0x7f17d00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.446+0000 7f17f1400640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f17d00a2880 0x7f17d00a4d40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55a3530148d0 tx=0x7f17dc005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.446+0000 7f17f1400640 1 -- 172.21.15.69:0/4164805568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17ec104c50 msgr2=0x7f17ec19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.446+0000 7f17f1400640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17ec104c50 0x7f17ec19a2e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f17d8000c00 tx=0x7f17d802c650 comp rx=0 tx=0).stop 2024-03-19T01:10:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.447+0000 7f17f1400640 1 -- 172.21.15.69:0/4164805568 shutdown_connections 2024-03-19T01:10:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.447+0000 7f17f1400640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f17d00a2880 0x7f17d00a4d40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.447+0000 7f17f1400640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec105610 0x7f17ec1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.447+0000 7f17f1400640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17ec104c50 0x7f17ec19a2e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.447+0000 7f17f1400640 1 --2- 172.21.15.69:0/4164805568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17ec103a60 0x7f17ec199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.447+0000 7f17f1400640 1 -- 172.21.15.69:0/4164805568 >> 172.21.15.69:0/4164805568 conn(0x7f17ec0fefd0 msgr2=0x7f17ec107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.447+0000 7f17f1400640 1 -- 172.21.15.69:0/4164805568 shutdown_connections 2024-03-19T01:10:08.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.447+0000 7f17f1400640 1 -- 172.21.15.69:0/4164805568 wait complete. 2024-03-19T01:10:08.468 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-19T01:10:08.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.659+0000 7fe5eae00640 1 -- 172.21.15.69:0/4167541565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e41031e0 msgr2=0x7fe5e4109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.659+0000 7fe5eae00640 1 --2- 172.21.15.69:0/4167541565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e41031e0 0x7fe5e4109a70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009be0 tx=0x7fe5dc02bef0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.659+0000 7fe5eae00640 1 -- 172.21.15.69:0/4167541565 shutdown_connections 2024-03-19T01:10:08.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.659+0000 7fe5eae00640 1 --2- 172.21.15.69:0/4167541565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e41031e0 0x7fe5e4109a70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.659+0000 7fe5eae00640 1 --2- 172.21.15.69:0/4167541565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102820 0x7fe5e4102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.659+0000 7fe5eae00640 1 --2- 172.21.15.69:0/4167541565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e4101630 0x7fe5e4101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.659+0000 7fe5eae00640 1 -- 172.21.15.69:0/4167541565 >> 172.21.15.69:0/4167541565 conn(0x7fe5e40fcb80 msgr2=0x7fe5e40fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.659+0000 7fe5eae00640 1 -- 172.21.15.69:0/4167541565 shutdown_connections 2024-03-19T01:10:08.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.659+0000 7fe5eae00640 1 -- 172.21.15.69:0/4167541565 wait complete. 2024-03-19T01:10:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.660+0000 7fe5eae00640 1 Processor -- start 2024-03-19T01:10:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.660+0000 7fe5eae00640 1 -- start start 2024-03-19T01:10:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.660+0000 7fe5eae00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e4101630 0x7fe5e41978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.660+0000 7fe5eae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102820 0x7fe5e4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.660+0000 7fe5eae00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e41031e0 0x7fe5e419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.660+0000 7fe5eae00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e410d300 con 0x7fe5e41031e0 2024-03-19T01:10:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.660+0000 7fe5eae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e410d180 con 0x7fe5e4102820 2024-03-19T01:10:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.660+0000 7fe5eae00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e410d480 con 0x7fe5e4101630 2024-03-19T01:10:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e4101630 0x7fe5e41978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5e9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102820 0x7fe5e4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5e9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102820 0x7fe5e4197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33108/0 (socket says 172.21.15.69:33108) 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e4101630 0x7fe5e41978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57516/0 (socket says 172.21.15.69:57516) 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5e9000640 1 -- 172.21.15.69:0/1127426030 learned_addr learned my addr 172.21.15.69:0/1127426030 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5ea400640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e41031e0 0x7fe5e419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5e9a00640 1 -- 172.21.15.69:0/1127426030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102820 msgr2=0x7fe5e4197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5e9a00640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102820 0x7fe5e4197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5e9a00640 1 -- 172.21.15.69:0/1127426030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e41031e0 msgr2=0x7fe5e419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5e9a00640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e41031e0 0x7fe5e419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5e9a00640 1 -- 172.21.15.69:0/1127426030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e419f500 con 0x7fe5e4101630 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.661+0000 7fe5e9a00640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e4101630 0x7fe5e41978e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800e9c0 tx=0x7fe5d800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.662+0000 7fe5e2000640 1 -- 172.21.15.69:0/1127426030 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800ccc0 con 0x7fe5e4101630 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.662+0000 7fe5e2000640 1 -- 172.21.15.69:0/1127426030 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d80035f0 con 0x7fe5e4101630 2024-03-19T01:10:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.662+0000 7fe5e2000640 1 -- 172.21.15.69:0/1127426030 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800ce60 con 0x7fe5e4101630 2024-03-19T01:10:08.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.662+0000 7fe5eae00640 1 -- 172.21.15.69:0/1127426030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e419f7f0 con 0x7fe5e4101630 2024-03-19T01:10:08.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.662+0000 7fe5eae00640 1 -- 172.21.15.69:0/1127426030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e419fcf0 con 0x7fe5e4101630 2024-03-19T01:10:08.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5ef00a740 1 Processor -- start 2024-03-19T01:10:08.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5ef00a740 1 -- start start 2024-03-19T01:10:08.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5e2000640 1 -- 172.21.15.69:0/1127426030 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fe5d80037a0 con 0x7fe5e4101630 2024-03-19T01:10:08.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d098b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098bfb0 0x55d0d0990ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d0991120 0x55d0d09935b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d0d08efd00 con 0x55d0d0991120 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d0d0898b00 con 0x55d0d098b5c0 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d0d08ebda0 con 0x55d0d098bfb0 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5ea400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d098b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098bfb0 0x55d0d0990ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5ea400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d098b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33116/0 (socket says 172.21.15.69:33116) 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5ea400640 1 -- 172.21.15.69:0/1317975149 learned_addr learned my addr 172.21.15.69:0/1317975149 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.665+0000 7fe5e9000640 1 --2- 172.21.15.69:0/1317975149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d0991120 0x55d0d09935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.666+0000 7fe5ea400640 1 -- 172.21.15.69:0/1317975149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098bfb0 msgr2=0x55d0d0990ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.666+0000 7fe5ea400640 1 --2- 172.21.15.69:0/1317975149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098bfb0 0x55d0d0990ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.666+0000 7fe5ea400640 1 -- 172.21.15.69:0/1317975149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d0991120 msgr2=0x55d0d09935b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.666+0000 7fe5ea400640 1 --2- 172.21.15.69:0/1317975149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d0991120 0x55d0d09935b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.666+0000 7fe5ea400640 1 -- 172.21.15.69:0/1317975149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0d06042d0 con 0x55d0d098b5c0 2024-03-19T01:10:08.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.666+0000 7fe5e9a00640 1 --2- 172.21.15.69:0/1317975149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098bfb0 0x55d0d0990ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:08.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.666+0000 7fe5ea400640 1 --2- 172.21.15.69:0/1317975149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d098b9e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc02c500 tx=0x7fe5dc009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231a0fe07bac753 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.666+0000 7fe5d6a00640 1 -- 172.21.15.69:0/1317975149 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc043030 con 0x55d0d098b5c0 2024-03-19T01:10:08.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5d6a00640 1 -- 172.21.15.69:0/1317975149 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5dc033580 con 0x55d0d098b5c0 2024-03-19T01:10:08.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5d6a00640 1 -- 172.21.15.69:0/1317975149 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc02c890 con 0x55d0d098b5c0 2024-03-19T01:10:08.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5e2000640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5cc0a2800 0x7fe5cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5e2000640 1 -- 172.21.15.69:0/1127426030 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fe5d80cf380 con 0x7fe5e4101630 2024-03-19T01:10:08.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5e9a00640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5cc0a2800 0x7fe5cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5ef00a740 1 -- 172.21.15.69:0/1317975149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 msgr2=0x55d0d098b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/1317975149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d098b9e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc02c500 tx=0x7fe5dc009330 comp rx=0 tx=0).stop 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5ef00a740 1 -- 172.21.15.69:0/1317975149 shutdown_connections 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/1317975149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d0991120 0x55d0d09935b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/1317975149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098bfb0 0x55d0d0990ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/1317975149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d098b9e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.667+0000 7fe5ef00a740 1 -- 172.21.15.69:0/1317975149 >> 172.21.15.69:0/1317975149 conn(0x55d0d0986b30 msgr2=0x55d0d0986f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.668+0000 7fe5ef00a740 1 -- 172.21.15.69:0/1317975149 shutdown_connections 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.668+0000 7fe5ef00a740 1 -- 172.21.15.69:0/1317975149 wait complete. 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.668+0000 7fe5e9a00640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5cc0a2800 0x7fe5cc0a4cc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55d0d098c3b0 tx=0x7fe5d80d2dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.668+0000 7fe5ef00a740 1 -- start start 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.668+0000 7fe5ef00a740 1 Processor -- start 2024-03-19T01:10:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098b5c0 0x55d0d0999ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098bfb0 0x55d0d099a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d0991120 0x55d0d09a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d0d0994070 con 0x55d0d0991120 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d0d08abed0 con 0x55d0d098bfb0 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d0d09941f0 con 0x55d0d098b5c0 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5ef00a740 10 client.0 _ll_register_callbacks cb 0x55d0d08a9c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5ea400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098b5c0 0x55d0d0999ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5ea400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098b5c0 0x55d0d0999ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57532/0 (socket says 172.21.15.69:57532) 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5ea400640 1 -- 172.21.15.69:0/3851583609 learned_addr learned my addr 172.21.15.69:0/3851583609 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5e9000640 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098bfb0 0x55d0d099a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5ea400640 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d0991120 0x55d0d09a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5e9000640 1 -- 172.21.15.69:0/3851583609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098b5c0 msgr2=0x55d0d0999ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5e9000640 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098b5c0 0x55d0d0999ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5e9000640 1 -- 172.21.15.69:0/3851583609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d0991120 msgr2=0x55d0d09a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5e9000640 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d0991120 0x55d0d09a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.669+0000 7fe5e9000640 1 -- 172.21.15.69:0/3851583609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0d06ce6b0 con 0x55d0d098bfb0 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.670+0000 7fe5e9000640 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098bfb0 0x55d0d099a430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4198df0 tx=0x7fe5d0012ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=678203427341ab24 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.670+0000 7fe5cb400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.670+0000 7fe5cb400640 1 -- 172.21.15.69:0/3851583609 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0004030 con 0x55d0d098bfb0 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.670+0000 7fe5cb400640 1 -- 172.21.15.69:0/3851583609 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fe5d00041d0 con 0x55d0d098bfb0 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.670+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3851583609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5dc033580 con 0x55d0d098bfb0 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.670+0000 7fe5cb400640 1 -- 172.21.15.69:0/3851583609 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5d00043d0 con 0x55d0d098bfb0 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.670+0000 7fe5ef00a740 10 client.25409 fetch_fsmap learned FSMap version 28 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.670+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3851583609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0d09a47e0 con 0x55d0d098bfb0 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.670+0000 7fe5cb400640 1 -- 172.21.15.69:0/3851583609 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5dc033580 con 0x55d0d098bfb0 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.671+0000 7fe5cb400640 1 -- 172.21.15.69:0/3851583609 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe5d0003b60 con 0x55d0d098bfb0 2024-03-19T01:10:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.671+0000 7fe5ef00a740 10 client.25409 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:10:08.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.671+0000 7fe5ef00a740 10 client.25409 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-19T01:10:08.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.671+0000 7fe5ef00a740 20 client.25409 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:08.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.671+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d0d09a6e60 0x55d0d09a82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.671+0000 7fe5ef00a740 4 client.25409 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:10:08.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.671+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3851583609 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0d08fd240 con 0x55d0d09a6e60 2024-03-19T01:10:08.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.671+0000 7fe5ea400640 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d0d09a6e60 0x55d0d09a82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.672+0000 7fe5ea400640 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d0d09a6e60 0x55d0d09a82d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c19cdf473d9d369c server_cookie=ae4bc3c1a7155a68 in_seq=0 out_seq=0 2024-03-19T01:10:08.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.672+0000 7fe5cb400640 10 client.25409 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:10:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.672+0000 7fe5cb400640 1 -- 172.21.15.69:0/3851583609 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d0d08fd240 con 0x55d0d09a6e60 2024-03-19T01:10:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.672+0000 7fe5cb400640 10 client.25409 handle_command_reply: tid=0 2024-03-19T01:10:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.672+0000 7fe5ef00a740 1 client.25409 shutdown 2024-03-19T01:10:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3851583609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098bfb0 msgr2=0x55d0d099a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098bfb0 0x55d0d099a430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4198df0 tx=0x7fe5d0012ca0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3851583609 shutdown_connections 2024-03-19T01:10:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d0991120 0x55d0d09a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d0d09a6e60 0x55d0d09a82d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098bfb0 0x55d0d099a430 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/3851583609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d098b5c0 0x55d0d0999ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3851583609 >> 172.21.15.69:0/3851583609 conn(0x55d0d0986b30 msgr2=0x55d0d0985d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5cb400640 0 client.25409 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:10:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3851583609 shutdown_connections 2024-03-19T01:10:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3851583609 wait complete. 2024-03-19T01:10:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.673+0000 7fe5ef00a740 20 client.25409 trim_cache size 0 max 16384 2024-03-19T01:10:08.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.691+0000 7fe5ef00a740 1 Processor -- start 2024-03-19T01:10:08.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5ef00a740 1 -- start start 2024-03-19T01:10:08.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098b5c0 0x55d0d098d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098bfb0 0x55d0d09a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d0991120 0x55d0d098da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d0d09a6610 con 0x55d0d098b5c0 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d0d09a6ce0 con 0x55d0d098bfb0 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d0d09a73b0 con 0x55d0d0991120 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5ea400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098b5c0 0x55d0d098d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5e9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098bfb0 0x55d0d09a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5ea400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098b5c0 0x55d0d098d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55232/0 (socket says 172.21.15.69:55232) 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5ea400640 1 -- 172.21.15.69:0/3902262538 learned_addr learned my addr 172.21.15.69:0/3902262538 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.692+0000 7fe5ea400640 1 --2- 172.21.15.69:0/3902262538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d0991120 0x55d0d098da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.693+0000 7fe5ea400640 1 -- 172.21.15.69:0/3902262538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d0991120 msgr2=0x55d0d098da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.693+0000 7fe5ea400640 1 --2- 172.21.15.69:0/3902262538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d0991120 0x55d0d098da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.693+0000 7fe5ea400640 1 -- 172.21.15.69:0/3902262538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098bfb0 msgr2=0x55d0d09a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.693+0000 7fe5ea400640 1 --2- 172.21.15.69:0/3902262538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098bfb0 0x55d0d09a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.693+0000 7fe5ea400640 1 -- 172.21.15.69:0/3902262538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0d09a8600 con 0x55d0d098b5c0 2024-03-19T01:10:08.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.693+0000 7fe5ea400640 1 --2- 172.21.15.69:0/3902262538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098b5c0 0x55d0d098d510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc000f80 tx=0x7fe5dc02bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1da8060193b90 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.693+0000 7fe5d6000640 1 -- 172.21.15.69:0/3902262538 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc05b070 con 0x55d0d098b5c0 2024-03-19T01:10:08.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.693+0000 7fe5d6000640 1 -- 172.21.15.69:0/3902262538 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5dc033c20 con 0x55d0d098b5c0 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.693+0000 7fe5d6000640 1 -- 172.21.15.69:0/3902262538 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc043040 con 0x55d0d098b5c0 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.694+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3902262538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098b5c0 msgr2=0x55d0d098d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.694+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/3902262538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098b5c0 0x55d0d098d510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc000f80 tx=0x7fe5dc02bef0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.694+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3902262538 shutdown_connections 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.694+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/3902262538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d0991120 0x55d0d098da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.694+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/3902262538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098bfb0 0x55d0d09a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.694+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/3902262538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098b5c0 0x55d0d098d510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.694+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3902262538 >> 172.21.15.69:0/3902262538 conn(0x55d0d0986b30 msgr2=0x55d0d0986f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.694+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3902262538 shutdown_connections 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.694+0000 7fe5ef00a740 1 -- 172.21.15.69:0/3902262538 wait complete. 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.694+0000 7fe5ef00a740 1 -- start start 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.694+0000 7fe5ef00a740 1 Processor -- start 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d099f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098bfb0 0x55d0d0a08bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ef00a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d0991120 0x55d0d0a0b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d0d09a8150 con 0x55d0d098bfb0 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d0d09a7a80 con 0x55d0d098b5c0 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ef00a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d0d09a8820 con 0x55d0d0991120 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ef00a740 10 client.0 _ll_register_callbacks cb 0x7fe5d0012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ea400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d099f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ea400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d099f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33138/0 (socket says 172.21.15.69:33138) 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ea400640 1 -- 172.21.15.69:0/484070597 learned_addr learned my addr 172.21.15.69:0/484070597 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5e9000640 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098bfb0 0x55d0d0a08bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ea400640 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d0991120 0x55d0d0a0b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ea400640 1 -- 172.21.15.69:0/484070597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d0991120 msgr2=0x55d0d0a0b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ea400640 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d0991120 0x55d0d0a0b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ea400640 1 -- 172.21.15.69:0/484070597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098bfb0 msgr2=0x55d0d0a08bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ea400640 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098bfb0 0x55d0d0a08bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.695+0000 7fe5ea400640 1 -- 172.21.15.69:0/484070597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0d0a0b8e0 con 0x55d0d098b5c0 2024-03-19T01:10:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.696+0000 7fe5ea400640 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d099f3b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55d0d09a1920 tx=0x7fe5dc034f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ccc294f6c542aa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.696+0000 7fe5cb400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:10:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.696+0000 7fe5cb400640 1 -- 172.21.15.69:0/484070597 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc05b070 con 0x55d0d098b5c0 2024-03-19T01:10:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.696+0000 7fe5ef00a740 1 -- 172.21.15.69:0/484070597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5dc033c20 con 0x55d0d098b5c0 2024-03-19T01:10:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.696+0000 7fe5cb400640 1 -- 172.21.15.69:0/484070597 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fe5dc004b40 con 0x55d0d098b5c0 2024-03-19T01:10:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.696+0000 7fe5cb400640 1 -- 172.21.15.69:0/484070597 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5dc043040 con 0x55d0d098b5c0 2024-03-19T01:10:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.696+0000 7fe5ef00a740 10 client.25415 fetch_fsmap learned FSMap version 28 2024-03-19T01:10:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.696+0000 7fe5ef00a740 1 -- 172.21.15.69:0/484070597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0d0a0cbc0 con 0x55d0d098b5c0 2024-03-19T01:10:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.696+0000 7fe5cb400640 1 -- 172.21.15.69:0/484070597 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5dc033c20 con 0x55d0d098b5c0 2024-03-19T01:10:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.697+0000 7fe5cb400640 1 -- 172.21.15.69:0/484070597 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe5dc05f020 con 0x55d0d098b5c0 2024-03-19T01:10:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.697+0000 7fe5ef00a740 10 client.25415 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:10:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.697+0000 7fe5ef00a740 10 client.25415 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-19T01:10:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.697+0000 7fe5ef00a740 20 client.25415 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.697+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d0d0a111d0 0x55d0d0a0fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.697+0000 7fe5ef00a740 4 client.25415 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:10:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.697+0000 7fe5ef00a740 1 -- 172.21.15.69:0/484070597 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "status"}) -- 0x55d0d08f7590 con 0x55d0d0a111d0 2024-03-19T01:10:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.697+0000 7fe5e9000640 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d0d0a111d0 0x55d0d0a0fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5e9000640 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d0d0a111d0 0x55d0d0a0fee0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d76891e0c6e3bb7c server_cookie=6ba312ea8fd32df0 in_seq=0 out_seq=0 2024-03-19T01:10:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5cb400640 10 client.25415 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:10:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5cb400640 1 -- 172.21.15.69:0/484070597 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d0d08f7590 con 0x55d0d0a111d0 2024-03-19T01:10:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5cb400640 10 client.25415 handle_command_reply: tid=0 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5ef00a740 1 client.25415 shutdown 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5ef00a740 1 -- 172.21.15.69:0/484070597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 msgr2=0x55d0d099f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d099f3b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55d0d09a1920 tx=0x7fe5dc034f90 comp rx=0 tx=0).stop 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5ef00a740 1 -- 172.21.15.69:0/484070597 shutdown_connections 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0d0991120 0x55d0d0a0b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0d098bfb0 0x55d0d0a08bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d0d0a111d0 0x55d0d0a0fee0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5ef00a740 1 --2- 172.21.15.69:0/484070597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0d098b5c0 0x55d0d099f3b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.698+0000 7fe5ef00a740 1 -- 172.21.15.69:0/484070597 >> 172.21.15.69:0/484070597 conn(0x55d0d0986b30 msgr2=0x55d0d098d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.699+0000 7fe5cb400640 0 client.25415 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.699+0000 7fe5ef00a740 1 -- 172.21.15.69:0/484070597 shutdown_connections 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.699+0000 7fe5ef00a740 1 -- 172.21.15.69:0/484070597 wait complete. 2024-03-19T01:10:08.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.699+0000 7fe5ef00a740 20 client.25415 trim_cache size 0 max 16384 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24580, 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 141.90941817000001, 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 93, 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 183.025868014 2024-03-19T01:10:08.704 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:10:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.699+0000 7fe5eae00640 1 -- 172.21.15.69:0/1127426030 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5cc0a2800 msgr2=0x7fe5cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.699+0000 7fe5eae00640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5cc0a2800 0x7fe5cc0a4cc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55d0d098c3b0 tx=0x7fe5d80d2dc0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.699+0000 7fe5eae00640 1 -- 172.21.15.69:0/1127426030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e4101630 msgr2=0x7fe5e41978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.699+0000 7fe5eae00640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e4101630 0x7fe5e41978e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800e9c0 tx=0x7fe5d800ee80 comp rx=0 tx=0).stop 2024-03-19T01:10:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.700+0000 7fe5eae00640 1 -- 172.21.15.69:0/1127426030 shutdown_connections 2024-03-19T01:10:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.700+0000 7fe5eae00640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fe5cc0a2800 0x7fe5cc0a4cc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.700+0000 7fe5eae00640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e41031e0 0x7fe5e419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.700+0000 7fe5eae00640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4102820 0x7fe5e4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.700+0000 7fe5eae00640 1 --2- 172.21.15.69:0/1127426030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e4101630 0x7fe5e41978e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.700+0000 7fe5eae00640 1 -- 172.21.15.69:0/1127426030 >> 172.21.15.69:0/1127426030 conn(0x7fe5e40fcb80 msgr2=0x7fe5e4105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.700+0000 7fe5eae00640 1 -- 172.21.15.69:0/1127426030 shutdown_connections 2024-03-19T01:10:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.700+0000 7fe5eae00640 1 -- 172.21.15.69:0/1127426030 wait complete. 2024-03-19T01:10:08.718 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-19T01:10:08.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.928+0000 7f24cd800640 1 -- 172.21.15.69:0/2757660956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c81096e0 msgr2=0x7f24c810bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.928+0000 7f24cd800640 1 --2- 172.21.15.69:0/2757660956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c81096e0 0x7f24c810bb70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009ca0 tx=0x7f24c002c190 comp rx=0 tx=0).stop 2024-03-19T01:10:08.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.928+0000 7f24cd800640 1 -- 172.21.15.69:0/2757660956 shutdown_connections 2024-03-19T01:10:08.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.928+0000 7f24cd800640 1 --2- 172.21.15.69:0/2757660956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c81096e0 0x7f24c810bb70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.928+0000 7f24cd800640 1 --2- 172.21.15.69:0/2757660956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c8074200 0x7f24c8074660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.928+0000 7f24cd800640 1 --2- 172.21.15.69:0/2757660956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24c80757e0 0x7f24c8073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.928+0000 7f24cd800640 1 -- 172.21.15.69:0/2757660956 >> 172.21.15.69:0/2757660956 conn(0x7f24c80fcdc0 msgr2=0x7f24c80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.929+0000 7f24cd800640 1 -- 172.21.15.69:0/2757660956 shutdown_connections 2024-03-19T01:10:08.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.929+0000 7f24cd800640 1 -- 172.21.15.69:0/2757660956 wait complete. 2024-03-19T01:10:08.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.930+0000 7f24cd800640 1 Processor -- start 2024-03-19T01:10:08.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.931+0000 7f24cd800640 1 -- start start 2024-03-19T01:10:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.931+0000 7f24cd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c8074200 0x7f24c8199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.931+0000 7f24cd800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c80757e0 0x7f24c819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.931+0000 7f24cd800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24c81096e0 0x7f24c81a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.931+0000 7f24cd800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f24c810f6c0 con 0x7f24c81096e0 2024-03-19T01:10:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.931+0000 7f24cd800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24c810f540 con 0x7f24c8074200 2024-03-19T01:10:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.931+0000 7f24cd800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f24c810f840 con 0x7f24c80757e0 2024-03-19T01:10:08.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.931+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c8074200 0x7f24c8199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.932+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c8074200 0x7f24c8199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33160/0 (socket says 172.21.15.69:33160) 2024-03-19T01:10:08.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.932+0000 7f24c6a00640 1 -- 172.21.15.69:0/2449467221 learned_addr learned my addr 172.21.15.69:0/2449467221 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.932+0000 7f24c6000640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c80757e0 0x7f24c819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.932+0000 7f24c7400640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24c81096e0 0x7f24c81a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.932+0000 7f24c6a00640 1 -- 172.21.15.69:0/2449467221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c80757e0 msgr2=0x7f24c819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.932+0000 7f24c6a00640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c80757e0 0x7f24c819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.932+0000 7f24c6a00640 1 -- 172.21.15.69:0/2449467221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24c81096e0 msgr2=0x7f24c81a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.932+0000 7f24c6a00640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24c81096e0 0x7f24c81a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.932+0000 7f24c6a00640 1 -- 172.21.15.69:0/2449467221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c81a1920 con 0x7f24c8074200 2024-03-19T01:10:08.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.932+0000 7f24c6a00640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c8074200 0x7f24c8199d10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f24b800e9f0 tx=0x7f24b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.933+0000 7f24bf400640 1 -- 172.21.15.69:0/2449467221 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b800cd80 con 0x7f24c8074200 2024-03-19T01:10:08.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.933+0000 7f24cd800640 1 -- 172.21.15.69:0/2449467221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c81a1c10 con 0x7f24c8074200 2024-03-19T01:10:08.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.933+0000 7f24cd800640 1 -- 172.21.15.69:0/2449467221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c81a21b0 con 0x7f24c8074200 2024-03-19T01:10:08.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.933+0000 7f24bf400640 1 -- 172.21.15.69:0/2449467221 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b8010910 con 0x7f24c8074200 2024-03-19T01:10:08.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.933+0000 7f24bf400640 1 -- 172.21.15.69:0/2449467221 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8018980 con 0x7f24c8074200 2024-03-19T01:10:08.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.937+0000 7f24d0422740 1 Processor -- start 2024-03-19T01:10:08.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.937+0000 7f24bf400640 1 -- 172.21.15.69:0/2449467221 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f24b8010430 con 0x7f24c8074200 2024-03-19T01:10:08.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.937+0000 7f24d0422740 1 -- start start 2024-03-19T01:10:08.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.938+0000 7f24d0422740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4f3f0 0x55b332f4f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.938+0000 7f24d0422740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4fde0 0x55b332f548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.938+0000 7f24d0422740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f573e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.938+0000 7f24d0422740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b332eb3d00 con 0x55b332f4f3f0 2024-03-19T01:10:08.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.938+0000 7f24d0422740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b332e5cb00 con 0x55b332f54f50 2024-03-19T01:10:08.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.938+0000 7f24d0422740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b332eafda0 con 0x55b332f4fde0 2024-03-19T01:10:08.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.938+0000 7f24c6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.939+0000 7f24c6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33162/0 (socket says 172.21.15.69:33162) 2024-03-19T01:10:08.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.939+0000 7f24c6000640 1 -- 172.21.15.69:0/245483558 learned_addr learned my addr 172.21.15.69:0/245483558 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.939+0000 7f24c6a00640 1 --2- 172.21.15.69:0/245483558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4fde0 0x55b332f548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.939+0000 7f24c7400640 1 --2- 172.21.15.69:0/245483558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4f3f0 0x55b332f4f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.939+0000 7f24c6000640 1 -- 172.21.15.69:0/245483558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4fde0 msgr2=0x55b332f548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.939+0000 7f24c6000640 1 --2- 172.21.15.69:0/245483558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4fde0 0x55b332f548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.939+0000 7f24c6000640 1 -- 172.21.15.69:0/245483558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4f3f0 msgr2=0x55b332f4f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.939+0000 7f24c6000640 1 --2- 172.21.15.69:0/245483558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4f3f0 0x55b332f4f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.939+0000 7f24c6000640 1 -- 172.21.15.69:0/245483558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b332bc82d0 con 0x55b332f54f50 2024-03-19T01:10:08.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.940+0000 7f24bf400640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f24ac0a2900 0x7f24ac0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.940+0000 7f24bf400640 1 -- 172.21.15.69:0/2449467221 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f24b80cf200 con 0x7f24c8074200 2024-03-19T01:10:08.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.940+0000 7f24c7400640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f24ac0a2900 0x7f24ac0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.940+0000 7f24c6000640 1 --2- 172.21.15.69:0/245483558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f573e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009d20 tx=0x7f24b4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da0ecf099c3d670e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.941+0000 7f24c7400640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f24ac0a2900 0x7f24ac0a4dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f24c002c6d0 tx=0x7f24c0005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.944+0000 7f24b3e00640 1 -- 172.21.15.69:0/245483558 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b401e070 con 0x55b332f54f50 2024-03-19T01:10:08.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.945+0000 7f24b3e00640 1 -- 172.21.15.69:0/245483558 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b4019b10 con 0x55b332f54f50 2024-03-19T01:10:08.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.945+0000 7f24b3e00640 1 -- 172.21.15.69:0/245483558 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b400ebf0 con 0x55b332f54f50 2024-03-19T01:10:08.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.945+0000 7f24d0422740 1 -- 172.21.15.69:0/245483558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 msgr2=0x55b332f573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.945+0000 7f24d0422740 1 --2- 172.21.15.69:0/245483558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f573e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009d20 tx=0x7f24b4011810 comp rx=0 tx=0).stop 2024-03-19T01:10:08.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.946+0000 7f24d0422740 1 -- 172.21.15.69:0/245483558 shutdown_connections 2024-03-19T01:10:08.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.946+0000 7f24d0422740 1 --2- 172.21.15.69:0/245483558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f573e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.946+0000 7f24d0422740 1 --2- 172.21.15.69:0/245483558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4fde0 0x55b332f548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.946+0000 7f24d0422740 1 --2- 172.21.15.69:0/245483558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4f3f0 0x55b332f4f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.946+0000 7f24d0422740 1 -- 172.21.15.69:0/245483558 >> 172.21.15.69:0/245483558 conn(0x55b332f4a960 msgr2=0x55b332f4ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.946+0000 7f24d0422740 1 -- 172.21.15.69:0/245483558 shutdown_connections 2024-03-19T01:10:08.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.946+0000 7f24d0422740 1 -- 172.21.15.69:0/245483558 wait complete. 2024-03-19T01:10:08.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.946+0000 7f24d0422740 1 -- start start 2024-03-19T01:10:08.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.946+0000 7f24d0422740 1 Processor -- start 2024-03-19T01:10:08.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24d0422740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4f3f0 0x55b332f598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24d0422740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4fde0 0x55b332f59e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24d0422740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 0x55b332f60e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24d0422740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b332f57f00 con 0x55b332f4f3f0 2024-03-19T01:10:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24d0422740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b332e6fed0 con 0x55b332f4fde0 2024-03-19T01:10:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24d0422740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b332f58080 con 0x55b332f54f50 2024-03-19T01:10:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24d0422740 10 client.0 _ll_register_callbacks cb 0x55b332e6dc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24c7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 0x55b332f60e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24c7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 0x55b332f60e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57570/0 (socket says 172.21.15.69:57570) 2024-03-19T01:10:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4f3f0 0x55b332f598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24c7400640 1 -- 172.21.15.69:0/243318368 learned_addr learned my addr 172.21.15.69:0/243318368 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.947+0000 7f24c6000640 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4fde0 0x55b332f59e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.948+0000 7f24c6a00640 1 -- 172.21.15.69:0/243318368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 msgr2=0x55b332f60e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.948+0000 7f24c6a00640 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 0x55b332f60e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.948+0000 7f24c6a00640 1 -- 172.21.15.69:0/243318368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4fde0 msgr2=0x55b332f59e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.948+0000 7f24c6a00640 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4fde0 0x55b332f59e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.948+0000 7f24c6a00640 1 -- 172.21.15.69:0/243318368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b332c926b0 con 0x55b332f4f3f0 2024-03-19T01:10:08.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.948+0000 7f24c7400640 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 0x55b332f60e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:08.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.948+0000 7f24c6a00640 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4f3f0 0x55b332f598c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55b332f50850 tx=0x7f24b80972e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec7eef015123f60 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.948+0000 7f24b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:10:08.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.948+0000 7f24b0c00640 1 -- 172.21.15.69:0/243318368 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b809b2d0 con 0x55b332f4f3f0 2024-03-19T01:10:08.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.948+0000 7f24b0c00640 1 -- 172.21.15.69:0/243318368 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f24b80d2cd0 con 0x55b332f4f3f0 2024-03-19T01:10:08.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.948+0000 7f24d0422740 1 -- 172.21.15.69:0/243318368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b4019b10 con 0x55b332f4f3f0 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.949+0000 7f24b0c00640 1 -- 172.21.15.69:0/243318368 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24b80998d0 con 0x55b332f4f3f0 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.949+0000 7f24d0422740 10 client.15306 fetch_fsmap learned FSMap version 28 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.949+0000 7f24d0422740 1 -- 172.21.15.69:0/243318368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b332f6ca30 con 0x55b332f4f3f0 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.949+0000 7f24b0c00640 1 -- 172.21.15.69:0/243318368 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b4019b10 con 0x55b332f4f3f0 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.949+0000 7f24b0c00640 1 -- 172.21.15.69:0/243318368 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f24b8097d70 con 0x55b332f4f3f0 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.949+0000 7f24d0422740 10 client.15306 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.949+0000 7f24d0422740 10 client.15306 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.949+0000 7f24d0422740 20 client.15306 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.949+0000 7f24d0422740 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b332f6f080 0x55b332f704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.949+0000 7f24d0422740 4 client.15306 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.949+0000 7f24d0422740 1 -- 172.21.15.69:0/243318368 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b332ec1240 con 0x55b332f6f080 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.950+0000 7f24c6000640 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b332f6f080 0x55b332f704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.950+0000 7f24c6000640 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b332f6f080 0x55b332f704f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5fd74662dd68ef0 server_cookie=5942591e40c9c079 in_seq=0 out_seq=0 2024-03-19T01:10:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.950+0000 7f24b0c00640 10 client.15306 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:08.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24b0c00640 1 -- 172.21.15.69:0/243318368 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b332ec1240 con 0x55b332f6f080 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24b0c00640 10 client.15306 handle_command_reply: tid=0 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 1 client.15306 shutdown 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 1 -- 172.21.15.69:0/243318368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4f3f0 msgr2=0x55b332f598c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4f3f0 0x55b332f598c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55b332f50850 tx=0x7f24b80972e0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 1 -- 172.21.15.69:0/243318368 shutdown_connections 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b332f6f080 0x55b332f704f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 0x55b332f60e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4fde0 0x55b332f59e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24b0c00640 0 client.15306 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 1 --2- 172.21.15.69:0/243318368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4f3f0 0x55b332f598c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 1 -- 172.21.15.69:0/243318368 >> 172.21.15.69:0/243318368 conn(0x55b332f4a960 msgr2=0x55b332f49aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 1 -- 172.21.15.69:0/243318368 shutdown_connections 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 1 -- 172.21.15.69:0/243318368 wait complete. 2024-03-19T01:10:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.951+0000 7f24d0422740 20 client.15306 trim_cache size 0 max 16384 2024-03-19T01:10:08.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.970+0000 7f24d0422740 1 Processor -- start 2024-03-19T01:10:08.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.970+0000 7f24d0422740 1 -- start start 2024-03-19T01:10:08.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24d0422740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4f3f0 0x55b332f58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24d0422740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 0x55b332f60e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24d0422740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f45730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24d0422740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b332f6e830 con 0x55b332f4fde0 2024-03-19T01:10:08.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24d0422740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b332f6ef00 con 0x55b332f54f50 2024-03-19T01:10:08.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24d0422740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b332f6f5d0 con 0x55b332f4f3f0 2024-03-19T01:10:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4f3f0 0x55b332f58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 0x55b332f60e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f45730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 0x55b332f60e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55286/0 (socket says 172.21.15.69:55286) 2024-03-19T01:10:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4f3f0 0x55b332f58ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:57582/0 (socket says 172.21.15.69:57582) 2024-03-19T01:10:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f45730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33182/0 (socket says 172.21.15.69:33182) 2024-03-19T01:10:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c6000640 1 -- 172.21.15.69:0/584665812 learned_addr learned my addr 172.21.15.69:0/584665812 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c6a00640 1 -- 172.21.15.69:0/584665812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 msgr2=0x55b332f45730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c6a00640 1 --2- 172.21.15.69:0/584665812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f45730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c6a00640 1 -- 172.21.15.69:0/584665812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 msgr2=0x55b332f60e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c6a00640 1 --2- 172.21.15.69:0/584665812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 0x55b332f60e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c6a00640 1 -- 172.21.15.69:0/584665812 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b332f501c0 con 0x55b332f4f3f0 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c7400640 1 --2- 172.21.15.69:0/584665812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f45730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.971+0000 7f24c6000640 1 --2- 172.21.15.69:0/584665812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 0x55b332f60e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.972+0000 7f24c6a00640 1 --2- 172.21.15.69:0/584665812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4f3f0 0x55b332f58ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f24b8000c00 tx=0x7f24b80cc970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a66ae6e8d1d5f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.972+0000 7f24b3400640 1 -- 172.21.15.69:0/584665812 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b809b0f0 con 0x55b332f4f3f0 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.972+0000 7f24b3400640 1 -- 172.21.15.69:0/584665812 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b809b790 con 0x55b332f4f3f0 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.972+0000 7f24b3400640 1 -- 172.21.15.69:0/584665812 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b809ae40 con 0x55b332f4f3f0 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.973+0000 7f24d0422740 1 -- 172.21.15.69:0/584665812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4f3f0 msgr2=0x55b332f58ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.973+0000 7f24d0422740 1 --2- 172.21.15.69:0/584665812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4f3f0 0x55b332f58ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f24b8000c00 tx=0x7f24b80cc970 comp rx=0 tx=0).stop 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.973+0000 7f24d0422740 1 -- 172.21.15.69:0/584665812 shutdown_connections 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.973+0000 7f24d0422740 1 --2- 172.21.15.69:0/584665812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f54f50 0x55b332f45730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.973+0000 7f24d0422740 1 --2- 172.21.15.69:0/584665812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 0x55b332f60e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.973+0000 7f24d0422740 1 --2- 172.21.15.69:0/584665812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f4f3f0 0x55b332f58ee0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.973+0000 7f24d0422740 1 -- 172.21.15.69:0/584665812 >> 172.21.15.69:0/584665812 conn(0x55b332f4a960 msgr2=0x55b332f50880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.973+0000 7f24d0422740 1 -- 172.21.15.69:0/584665812 shutdown_connections 2024-03-19T01:10:08.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.973+0000 7f24d0422740 1 -- 172.21.15.69:0/584665812 wait complete. 2024-03-19T01:10:08.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.973+0000 7f24d0422740 1 -- start start 2024-03-19T01:10:08.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.973+0000 7f24d0422740 1 Processor -- start 2024-03-19T01:10:08.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24d0422740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4f3f0 0x55b332f5edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24d0422740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 0x55b332fd0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24d0422740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 0x55b332fd3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24d0422740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b332f70370 con 0x55b332f4fde0 2024-03-19T01:10:08.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24d0422740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b332f6fca0 con 0x55b332f4f3f0 2024-03-19T01:10:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24d0422740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b332f70a40 con 0x55b332f54f50 2024-03-19T01:10:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24d0422740 10 client.0 _ll_register_callbacks cb 0x7f24b8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4f3f0 0x55b332f5edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4f3f0 0x55b332f5edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33196/0 (socket says 172.21.15.69:33196) 2024-03-19T01:10:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24c6a00640 1 -- 172.21.15.69:0/3214944955 learned_addr learned my addr 172.21.15.69:0/3214944955 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24c7400640 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 0x55b332fd3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24c6000640 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 0x55b332fd0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24c6a00640 1 -- 172.21.15.69:0/3214944955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 msgr2=0x55b332fd3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24c6a00640 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 0x55b332fd3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24c6a00640 1 -- 172.21.15.69:0/3214944955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 msgr2=0x55b332fd0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24c6a00640 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 0x55b332fd0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24c6a00640 1 -- 172.21.15.69:0/3214944955 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b332fd3860 con 0x55b332f4f3f0 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.974+0000 7f24c6000640 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 0x55b332fd0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24c6a00640 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4f3f0 0x55b332f5edf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24b80108e0 tx=0x7f24b80972e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=537f15092a810bd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24b0c00640 1 -- 172.21.15.69:0/3214944955 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b809b0f0 con 0x55b332f4f3f0 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24b0c00640 1 -- 172.21.15.69:0/3214944955 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f24b8018d30 con 0x55b332f4f3f0 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24d0422740 1 -- 172.21.15.69:0/3214944955 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b809b790 con 0x55b332f4f3f0 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24b0c00640 1 -- 172.21.15.69:0/3214944955 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24b809b2d0 con 0x55b332f4f3f0 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24d0422740 10 client.25433 fetch_fsmap learned FSMap version 28 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24d0422740 1 -- 172.21.15.69:0/3214944955 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b332fd4b40 con 0x55b332f4f3f0 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24b0c00640 1 -- 172.21.15.69:0/3214944955 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b809b790 con 0x55b332f4f3f0 2024-03-19T01:10:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24b0c00640 1 -- 172.21.15.69:0/3214944955 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f24b809a650 con 0x55b332f4f3f0 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24d0422740 10 client.25433 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24d0422740 10 client.25433 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24d0422740 20 client.25433 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.975+0000 7f24d0422740 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b332fd9440 0x55b332fd8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.976+0000 7f24d0422740 4 client.25433 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.976+0000 7f24d0422740 1 -- 172.21.15.69:0/3214944955 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "status"}) -- 0x55b332ebb590 con 0x55b332fd9440 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.976+0000 7f24c6000640 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b332fd9440 0x55b332fd8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.976+0000 7f24b0c00640 10 client.25433 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.976+0000 7f24c6000640 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b332fd9440 0x55b332fd8150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52004afc5a763d33 server_cookie=a6ccbac3fac300f1 in_seq=0 out_seq=0 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24b0c00640 1 -- 172.21.15.69:0/3214944955 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b332ebb590 con 0x55b332fd9440 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24b0c00640 10 client.25433 handle_command_reply: tid=0 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 1 client.25433 shutdown 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 1 -- 172.21.15.69:0/3214944955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4f3f0 msgr2=0x55b332f5edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4f3f0 0x55b332f5edf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24b80108e0 tx=0x7f24b80972e0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 1 -- 172.21.15.69:0/3214944955 shutdown_connections 2024-03-19T01:10:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b332fd9440 0x55b332fd8150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b332f54f50 0x55b332fd3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b332f4fde0 0x55b332fd0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 1 --2- 172.21.15.69:0/3214944955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b332f4f3f0 0x55b332f5edf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24b0c00640 0 client.25433 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 1 -- 172.21.15.69:0/3214944955 >> 172.21.15.69:0/3214944955 conn(0x55b332f4a960 msgr2=0x55b332f45710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 1 -- 172.21.15.69:0/3214944955 shutdown_connections 2024-03-19T01:10:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 1 -- 172.21.15.69:0/3214944955 wait complete. 2024-03-19T01:10:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.977+0000 7f24d0422740 20 client.25433 trim_cache size 0 max 16384 2024-03-19T01:10:08.983 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "934de1c2-e58b-11ee-95c9-87774f69a715", 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24581, 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 173.35953953399999, 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 93, 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 180.44161821500001 2024-03-19T01:10:08.984 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:10:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 -- 172.21.15.69:0/2449467221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f24ac0a2900 msgr2=0x7f24ac0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f24ac0a2900 0x7f24ac0a4dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f24c002c6d0 tx=0x7f24c0005da0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 -- 172.21.15.69:0/2449467221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c8074200 msgr2=0x7f24c8199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c8074200 0x7f24c8199d10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f24b800e9f0 tx=0x7f24b800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 -- 172.21.15.69:0/2449467221 shutdown_connections 2024-03-19T01:10:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f24ac0a2900 0x7f24ac0a4dc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24c81096e0 0x7f24c81a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c80757e0 0x7f24c819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 --2- 172.21.15.69:0/2449467221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24c8074200 0x7f24c8199d10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:08.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 -- 172.21.15.69:0/2449467221 >> 172.21.15.69:0/2449467221 conn(0x7f24c80fcdc0 msgr2=0x7f24c8105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:08.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 -- 172.21.15.69:0/2449467221 shutdown_connections 2024-03-19T01:10:08.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:08.978+0000 7f24cd800640 1 -- 172.21.15.69:0/2449467221 wait complete. 2024-03-19T01:10:08.996 INFO:teuthology.run_tasks:Running task kclient... 2024-03-19T01:10:09.010 INFO:tasks.kclient:Mounting kernel clients... 2024-03-19T01:10:09.011 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-19T01:10:09.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:10:09.011 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-03-19T01:10:09.056 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-19T01:10:09.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:10:09.058 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-19T01:10:09.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:10:09.175 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-03-19T01:10:09.298 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-03-19T01:10:09.298 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-19T01:10:09.298 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-19T01:10:09.299 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-19T01:10:09.766 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:10:10.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:10.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:10.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:10.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:09 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:10.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:10.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:10.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:10.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:09 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:10.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:10.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:10.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:10.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:09 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:11.212 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:10 smithi163 ceph-mon[67428]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:10:11.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:10 smithi086 ceph-mon[66829]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:10:11.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:10 smithi069 ceph-mon[31727]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-19T01:10:12.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.246+0000 7f906fe00700 1 -- 172.21.15.69:0/1536151614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9070102450 msgr2=0x7f9070102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:12.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.246+0000 7f906fe00700 1 --2- 172.21.15.69:0/1536151614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9070102450 0x7f9070102830 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9060009990 tx=0x7f9060009ca0 comp rx=0 tx=0).stop 2024-03-19T01:10:12.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.246+0000 7f906fe00700 1 -- 172.21.15.69:0/1536151614 shutdown_connections 2024-03-19T01:10:12.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.246+0000 7f906fe00700 1 --2- 172.21.15.69:0/1536151614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9070107530 0x7f907010f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:12.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.246+0000 7f906fe00700 1 --2- 172.21.15.69:0/1536151614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9070102450 0x7f9070102830 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:12.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.246+0000 7f906fe00700 1 --2- 172.21.15.69:0/1536151614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9070102e00 0x7f9070106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:12.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.246+0000 7f906fe00700 1 -- 172.21.15.69:0/1536151614 >> 172.21.15.69:0/1536151614 conn(0x7f90700fdce0 msgr2=0x7f9070100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:12.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.247+0000 7f906fe00700 1 -- 172.21.15.69:0/1536151614 shutdown_connections 2024-03-19T01:10:12.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.247+0000 7f906fe00700 1 -- 172.21.15.69:0/1536151614 wait complete. 2024-03-19T01:10:12.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.248+0000 7f906fe00700 1 Processor -- start 2024-03-19T01:10:12.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.248+0000 7f906fe00700 1 -- start start 2024-03-19T01:10:12.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.248+0000 7f906fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9070102450 0x7f9070075e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:12.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.248+0000 7f906fe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9070102e00 0x7f9070070f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:12.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.248+0000 7f906fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9070107530 0x7f90700714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:12.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.248+0000 7f906fe00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9070112560 con 0x7f9070107530 2024-03-19T01:10:12.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.248+0000 7f906fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90701123e0 con 0x7f9070102450 2024-03-19T01:10:12.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.248+0000 7f906fe00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f90701126e0 con 0x7f9070102e00 2024-03-19T01:10:12.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9070102450 0x7f9070075e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:12.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9070102450 0x7f9070075e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33202/0 (socket says 172.21.15.69:33202) 2024-03-19T01:10:12.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906ea00700 1 -- 172.21.15.69:0/3084736602 learned_addr learned my addr 172.21.15.69:0/3084736602 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:12.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906e000700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9070102e00 0x7f9070070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:12.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906f400700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9070107530 0x7f90700714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906ea00700 1 -- 172.21.15.69:0/3084736602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9070102e00 msgr2=0x7f9070070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906ea00700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9070102e00 0x7f9070070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906ea00700 1 -- 172.21.15.69:0/3084736602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9070107530 msgr2=0x7f90700714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906ea00700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9070107530 0x7f90700714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906ea00700 1 -- 172.21.15.69:0/3084736602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9070071bb0 con 0x7f9070102450 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906e000700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9070102e00 0x7f9070070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.249+0000 7f906ea00700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9070102450 0x7f9070075e90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9060005910 tx=0x7f906000b6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.250+0000 7f9067400700 1 -- 172.21.15.69:0/3084736602 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060027070 con 0x7f9070102450 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.250+0000 7f9067400700 1 -- 172.21.15.69:0/3084736602 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9060022470 con 0x7f9070102450 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.250+0000 7f9067400700 1 -- 172.21.15.69:0/3084736602 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060017650 con 0x7f9070102450 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.250+0000 7f906fe00700 1 -- 172.21.15.69:0/3084736602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90701afa80 con 0x7f9070102450 2024-03-19T01:10:12.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.250+0000 7f906fe00700 1 -- 172.21.15.69:0/3084736602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701afe20 con 0x7f9070102450 2024-03-19T01:10:12.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.251+0000 7f906fe00700 1 -- 172.21.15.69:0/3084736602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9070103930 con 0x7f9070102450 2024-03-19T01:10:12.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.255+0000 7f9067400700 1 -- 172.21.15.69:0/3084736602 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f906002a070 con 0x7f9070102450 2024-03-19T01:10:12.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.257+0000 7f9067400700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90540a2910 0x7f90540a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:12.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.257+0000 7f9067400700 1 -- 172.21.15.69:0/3084736602 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f90600cfba0 con 0x7f9070102450 2024-03-19T01:10:12.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.257+0000 7f906e000700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90540a2910 0x7f90540a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:12.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.258+0000 7f906e000700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90540a2910 0x7f90540a4dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f90700726e0 tx=0x7f905c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:12.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.260+0000 7f9067400700 1 -- 172.21.15.69:0/3084736602 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9060013070 con 0x7f9070102450 2024-03-19T01:10:12.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:12 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:12.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:12 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:12 smithi086 ceph-mon[66829]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:12.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:12 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:12 smithi069 ceph-mon[31727]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:12.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:12 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:12.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:12 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:12.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:12 smithi163 ceph-mon[67428]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:12.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.607+0000 7f906fe00700 1 -- 172.21.15.69:0/3084736602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9070072450 con 0x7f9070102450 2024-03-19T01:10:12.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.608+0000 7f9067400700 1 -- 172.21.15.69:0/3084736602 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f906009c8c0 con 0x7f9070102450 2024-03-19T01:10:12.609 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-19T01:10:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.610+0000 7f906fe00700 1 -- 172.21.15.69:0/3084736602 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90540a2910 msgr2=0x7f90540a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.610+0000 7f906fe00700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90540a2910 0x7f90540a4dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f90700726e0 tx=0x7f905c009260 comp rx=0 tx=0).stop 2024-03-19T01:10:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.610+0000 7f906fe00700 1 -- 172.21.15.69:0/3084736602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9070102450 msgr2=0x7f9070075e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.610+0000 7f906fe00700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9070102450 0x7f9070075e90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9060005910 tx=0x7f906000b6c0 comp rx=0 tx=0).stop 2024-03-19T01:10:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.611+0000 7f906fe00700 1 -- 172.21.15.69:0/3084736602 shutdown_connections 2024-03-19T01:10:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.611+0000 7f906fe00700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f90540a2910 0x7f90540a4dd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.611+0000 7f906fe00700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9070107530 0x7f90700714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.611+0000 7f906fe00700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9070102450 0x7f9070075e90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.611+0000 7f906fe00700 1 --2- 172.21.15.69:0/3084736602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9070102e00 0x7f9070070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.611+0000 7f906fe00700 1 -- 172.21.15.69:0/3084736602 >> 172.21.15.69:0/3084736602 conn(0x7f90700fdce0 msgr2=0x7f9070100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:12.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.611+0000 7f906fe00700 1 -- 172.21.15.69:0/3084736602 shutdown_connections 2024-03-19T01:10:12.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:12.611+0000 7f906fe00700 1 -- 172.21.15.69:0/3084736602 wait complete. 2024-03-19T01:10:13.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:13 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:10:13.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:13 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:10:13.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:13 smithi069 ceph-mon[31727]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:13.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:13 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3084736602' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T01:10:13.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:13 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:10:13.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:13 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:10:13.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:13 smithi086 ceph-mon[66829]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:13.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:13 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3084736602' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T01:10:13.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:13 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:10:13.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:13 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:10:13.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:13 smithi163 ceph-mon[67428]: from='mgr.14152 172.21.15.69:0/1611608673' entity='mgr.x' 2024-03-19T01:10:13.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:13 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3084736602' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T01:10:13.605 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-19T01:10:13.605 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-19T01:10:13.605 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-03-19T01:10:13.605 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-19T01:10:13.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-19T01:10:13.606 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-19T01:10:13.606 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-19T01:10:13.606 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-19T01:10:13.606 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-19T01:10:13.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:10:13.606 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-03-19T01:10:13.657 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-19T01:10:13.657 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-19T01:10:13.657 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-03-19T01:10:13.657 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-03-19T01:10:13.657 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-03-19T01:10:13.657 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-03-19T01:10:13.657 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T01:10:13.657 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:10:13.658 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-03-19T01:10:13.658 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T01:10:13.658 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:10:13.658 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-03-19T01:10:13.658 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T01:10:13.658 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:10:13.658 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-19T01:10:13.658 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:10:13.659 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-19T01:10:13.659 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 40312sec preferred_lft 40312sec 2024-03-19T01:10:13.659 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-03-19T01:10:13.659 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-03-19T01:10:13.660 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-19T01:10:13.660 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:10:13.660 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-19T01:10:13.660 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2024-03-19T01:10:13.660 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2024-03-19T01:10:13.660 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2024-03-19T01:10:13.660 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-19T01:10:13.660 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-19T01:10:13.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:10:13.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:10:13.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:10:13.940 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-19T01:10:14.016 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-03-19T01:10:14.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:10:14.021 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-03-19T01:10:14.068 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-03-19T01:10:14.068 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-19T01:10:14.068 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-19T01:10:14.069 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-19T01:10:14.069 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-19T01:10:14.069 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:10:14.069 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-19T01:10:14.069 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-19T01:10:14.069 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-19T01:10:14.069 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-19T01:10:14.069 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-19T01:10:14.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:10:14.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:10:14.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:10:14.310 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-03-19T01:10:14.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:10:14.353 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-03-19T01:10:14.447 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:10:14.447 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-19T01:10:14.447 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-19T01:10:14.447 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-19T01:10:14.447 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-19T01:10:14.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:10:14.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:10:14.627 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-19T01:10:14.627 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:10:14.627 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-19T01:10:14.627 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-19T01:10:14.627 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-19T01:10:14.627 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-19T01:10:14.627 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-19T01:10:14.627 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-19T01:10:14.627 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-19T01:10:14.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:10:14.814 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:14 smithi069 ceph-mon[31727]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:10:14.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:10:14.953 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:10:14.953 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-19T01:10:14.953 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-03-19T01:10:14.953 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2024-03-19T01:10:14.953 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-19T01:10:15.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:10:15.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:14 smithi163 ceph-mon[67428]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:10:15.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:10:15.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-19T01:10:15.156 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-19T01:10:15.196 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-19T01:10:15.197 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-19T01:10:15.197 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-19T01:10:15.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:14 smithi086 ceph-mon[66829]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-19T01:10:15.285 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-19T01:10:15.286 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-19T01:10:15.286 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/553c1530-b025-47e9-a3bd-4c853425f8e3 2024-03-19T01:10:15.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:10:15.286 DEBUG:teuthology.orchestra.run.smithi069:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/553c1530-b025-47e9-a3bd-4c853425f8e3 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-03-19T01:10:15.438 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-03-19T01:10:15.438 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-03-19T01:10:16.092 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-03-19T01:10:16.093 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-03-19T01:10:16.093 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: resolved to: "172.21.15.69:6789,172.21.15.86:6789,172.21.15.163:6789" 2024-03-19T01:10:16.093 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: trying mount with new device syntax: 0@934de1c2-e58b-11ee-95c9-87774f69a715.cephfs=/volumes/qa/sv_1/553c1530-b025-47e9-a3bd-4c853425f8e3 2024-03-19T01:10:16.093 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.69:6789/172.21.15.86:6789/172.21.15.163:6789" will pass to kernel 2024-03-19T01:10:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-19T01:10:16.097 INFO:teuthology.orchestra.run.smithi069.stdout: You just mounted a file system that supports labels which does not 2024-03-19T01:10:16.097 INFO:teuthology.orchestra.run.smithi069.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-19T01:10:16.097 INFO:teuthology.orchestra.run.smithi069.stdout: applications will generate AVC messages and not be allowed access to 2024-03-19T01:10:16.097 INFO:teuthology.orchestra.run.smithi069.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-19T01:10:16.102 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-19T01:10:16.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:10:16.102 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-19T01:10:16.182 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-19T01:10:16.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-19T01:10:16.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:10:16.287 INFO:teuthology.orchestra.run.smithi069.stdout:client25442 2024-03-19T01:10:16.288 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T01:10:16.288 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T01:10:16.338 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-19T01:10:16.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-19T01:10:16.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:10:16.493 INFO:teuthology.orchestra.run.smithi069.stdout:client25442 2024-03-19T01:10:16.493 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-19T01:10:16.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-19T01:10:16.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:10:16.621 INFO:teuthology.orchestra.run.smithi069.stdout:client25442 2024-03-19T01:10:16.622 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/934de1c2-e58b-11ee-95c9-87774f69a715.client25442/status') 2024-03-19T01:10:16.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:10:16.759 INFO:teuthology.orchestra.run.smithi069.stdout:instance: client.25442 (3)172.21.15.69:0/2073436206 2024-03-19T01:10:16.759 INFO:teuthology.orchestra.run.smithi069.stdout:blocklisted: false 2024-03-19T01:10:16.760 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records in 2024-03-19T01:10:16.760 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records out 2024-03-19T01:10:16.760 INFO:teuthology.orchestra.run.smithi069.stderr:71 bytes copied, 6.0085e-05 s, 1.2 MB/s 2024-03-19T01:10:16.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:10:16.772 INFO:teuthology.run_tasks:Running task exec... 2024-03-19T01:10:16.782 INFO:teuthology.task.exec:Executing custom commands... 2024-03-19T01:10:16.782 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-03-19T01:10:16.782 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-19T01:10:16.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.974+0000 7f84ebe00640 1 -- 172.21.15.69:0/1811588245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84ec0fff40 msgr2=0x7f84ec100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:16.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.974+0000 7f84ebe00640 1 --2- 172.21.15.69:0/1811588245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84ec0fff40 0x7f84ec100360 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009900 tx=0x7f84dc02bc40 comp rx=0 tx=0).stop 2024-03-19T01:10:16.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.974+0000 7f84ebe00640 1 -- 172.21.15.69:0/1811588245 shutdown_connections 2024-03-19T01:10:16.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.974+0000 7f84ebe00640 1 --2- 172.21.15.69:0/1811588245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84ec101240 0x7f84ec1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:16.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.974+0000 7f84ebe00640 1 --2- 172.21.15.69:0/1811588245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84ec1008a0 0x7f84ec100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:16.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.974+0000 7f84ebe00640 1 --2- 172.21.15.69:0/1811588245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84ec0fff40 0x7f84ec100360 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:16.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.974+0000 7f84ebe00640 1 -- 172.21.15.69:0/1811588245 >> 172.21.15.69:0/1811588245 conn(0x7f84ec0fbb00 msgr2=0x7f84ec0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:16.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.974+0000 7f84ebe00640 1 -- 172.21.15.69:0/1811588245 shutdown_connections 2024-03-19T01:10:16.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.974+0000 7f84ebe00640 1 -- 172.21.15.69:0/1811588245 wait complete. 2024-03-19T01:10:16.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.976+0000 7f84ebe00640 1 Processor -- start 2024-03-19T01:10:16.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.976+0000 7f84ebe00640 1 -- start start 2024-03-19T01:10:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.976+0000 7f84ebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84ec0fff40 0x7f84ec199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.976+0000 7f84ebe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84ec1008a0 0x7f84ec19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.976+0000 7f84ebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84ec101240 0x7f84ec1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.976+0000 7f84ebe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec10f4b0 con 0x7f84ec101240 2024-03-19T01:10:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.976+0000 7f84ebe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec10f330 con 0x7f84ec0fff40 2024-03-19T01:10:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.976+0000 7f84ebe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec10f630 con 0x7f84ec1008a0 2024-03-19T01:10:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.977+0000 7f84ea000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84ec1008a0 0x7f84ec19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.977+0000 7f84eaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84ec0fff40 0x7f84ec199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.977+0000 7f84ea000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84ec1008a0 0x7f84ec19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36024/0 (socket says 172.21.15.69:36024) 2024-03-19T01:10:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.977+0000 7f84ea000640 1 -- 172.21.15.69:0/1605269567 learned_addr learned my addr 172.21.15.69:0/1605269567 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.977+0000 7f84eb400640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84ec101240 0x7f84ec1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.977+0000 7f84ea000640 1 -- 172.21.15.69:0/1605269567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84ec0fff40 msgr2=0x7f84ec199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.977+0000 7f84ea000640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84ec0fff40 0x7f84ec199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.977+0000 7f84ea000640 1 -- 172.21.15.69:0/1605269567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84ec101240 msgr2=0x7f84ec1a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.977+0000 7f84ea000640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84ec101240 0x7f84ec1a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.977+0000 7f84ea000640 1 -- 172.21.15.69:0/1605269567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ec1a1750 con 0x7f84ec1008a0 2024-03-19T01:10:16.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.977+0000 7f84ea000640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84ec1008a0 0x7f84ec19a070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84d800e9f0 tx=0x7f84d800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:16.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.978+0000 7f84f0a00640 1 -- 172.21.15.69:0/1605269567 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d800cd80 con 0x7f84ec1008a0 2024-03-19T01:10:16.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.978+0000 7f84f0a00640 1 -- 172.21.15.69:0/1605269567 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d8010910 con 0x7f84ec1008a0 2024-03-19T01:10:16.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.978+0000 7f84f0a00640 1 -- 172.21.15.69:0/1605269567 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8018990 con 0x7f84ec1008a0 2024-03-19T01:10:16.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.978+0000 7f84ebe00640 1 -- 172.21.15.69:0/1605269567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ec1a1a40 con 0x7f84ec1008a0 2024-03-19T01:10:16.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.978+0000 7f84ebe00640 1 -- 172.21.15.69:0/1605269567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ec1a1f40 con 0x7f84ec1008a0 2024-03-19T01:10:16.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.979+0000 7f84ebe00640 1 -- 172.21.15.69:0/1605269567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ec10ae40 con 0x7f84ec1008a0 2024-03-19T01:10:16.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.981+0000 7f84f0a00640 1 -- 172.21.15.69:0/1605269567 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f84d8010ac0 con 0x7f84ec1008a0 2024-03-19T01:10:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.984+0000 7f84f0a00640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84d00a27f0 0x7f84d00a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.985+0000 7f84f0a00640 1 -- 172.21.15.69:0/1605269567 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f84d80cf1a0 con 0x7f84ec1008a0 2024-03-19T01:10:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.985+0000 7f84eaa00640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84d00a27f0 0x7f84d00a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:16.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.985+0000 7f84eaa00640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84d00a27f0 0x7f84d00a4cb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f84dc02c7c0 tx=0x7f84dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:16.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:16.986+0000 7f84f0a00640 1 -- 172.21.15.69:0/1605269567 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f84d809b7e0 con 0x7f84ec1008a0 2024-03-19T01:10:17.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:16 smithi069 ceph-mon[31727]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:17.056 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:16 smithi086 ceph-mon[66829]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:17.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:16 smithi163 ceph-mon[67428]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:10:17.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.371+0000 7f84ebe00640 1 -- 172.21.15.69:0/1605269567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f84ec02d1f0 con 0x7f84ec1008a0 2024-03-19T01:10:17.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.818+0000 7f84f0a00640 1 -- 172.21.15.69:0/1605269567 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f84d809f880 con 0x7f84ec1008a0 2024-03-19T01:10:17.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.821+0000 7f84ebe00640 1 -- 172.21.15.69:0/1605269567 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84d00a27f0 msgr2=0x7f84d00a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:17.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.821+0000 7f84ebe00640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84d00a27f0 0x7f84d00a4cb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f84dc02c7c0 tx=0x7f84dc005fd0 comp rx=0 tx=0).stop 2024-03-19T01:10:17.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.821+0000 7f84ebe00640 1 -- 172.21.15.69:0/1605269567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84ec1008a0 msgr2=0x7f84ec19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:17.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.821+0000 7f84ebe00640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84ec1008a0 0x7f84ec19a070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84d800e9f0 tx=0x7f84d800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:17.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.822+0000 7f84ebe00640 1 -- 172.21.15.69:0/1605269567 shutdown_connections 2024-03-19T01:10:17.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.822+0000 7f84ebe00640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f84d00a27f0 0x7f84d00a4cb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:17.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.822+0000 7f84ebe00640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84ec101240 0x7f84ec1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:17.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.822+0000 7f84ebe00640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84ec1008a0 0x7f84ec19a070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:17.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.822+0000 7f84ebe00640 1 --2- 172.21.15.69:0/1605269567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84ec0fff40 0x7f84ec199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:17.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.822+0000 7f84ebe00640 1 -- 172.21.15.69:0/1605269567 >> 172.21.15.69:0/1605269567 conn(0x7f84ec0fbb00 msgr2=0x7f84ec0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:17.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.822+0000 7f84ebe00640 1 -- 172.21.15.69:0/1605269567 shutdown_connections 2024-03-19T01:10:17.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:17.822+0000 7f84ebe00640 1 -- 172.21.15.69:0/1605269567 wait complete. 2024-03-19T01:10:17.849 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-19T01:10:18.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.041+0000 7f5486a00640 1 -- 172.21.15.69:0/3570266954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5480103420 msgr2=0x7f5480109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:18.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.041+0000 7f5486a00640 1 --2- 172.21.15.69:0/3570266954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5480103420 0x7f5480109cb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5478009ca0 tx=0x7f547802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.041+0000 7f5486a00640 1 -- 172.21.15.69:0/3570266954 shutdown_connections 2024-03-19T01:10:18.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.041+0000 7f5486a00640 1 --2- 172.21.15.69:0/3570266954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5480103420 0x7f5480109cb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.041+0000 7f5486a00640 1 --2- 172.21.15.69:0/3570266954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5480102a60 0x7f5480102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.041+0000 7f5486a00640 1 --2- 172.21.15.69:0/3570266954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5480101870 0x7f5480101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.042+0000 7f5486a00640 1 -- 172.21.15.69:0/3570266954 >> 172.21.15.69:0/3570266954 conn(0x7f54800fcde0 msgr2=0x7f54800ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:18.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.042+0000 7f5486a00640 1 -- 172.21.15.69:0/3570266954 shutdown_connections 2024-03-19T01:10:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.042+0000 7f5486a00640 1 -- 172.21.15.69:0/3570266954 wait complete. 2024-03-19T01:10:18.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.043+0000 7f5486a00640 1 Processor -- start 2024-03-19T01:10:18.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.043+0000 7f5486a00640 1 -- start start 2024-03-19T01:10:18.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.043+0000 7f5486a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5480101870 0x7f54800755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:18.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f5486a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5480102a60 0x7f5480073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:18.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f5486a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5480103420 0x7f5480074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:18.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f5486a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f548010d4f0 con 0x7f5480101870 2024-03-19T01:10:18.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f5486a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f548010d370 con 0x7f5480102a60 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f5486a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f548010d670 con 0x7f5480103420 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f547f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5480102a60 0x7f5480073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f547fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5480101870 0x7f54800755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f547f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5480102a60 0x7f5480073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54496/0 (socket says 172.21.15.69:54496) 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f547f400640 1 -- 172.21.15.69:0/3935551350 learned_addr learned my addr 172.21.15.69:0/3935551350 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f5484a00640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5480103420 0x7f5480074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f547f400640 1 -- 172.21.15.69:0/3935551350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5480103420 msgr2=0x7f5480074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f547f400640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5480103420 0x7f5480074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f547f400640 1 -- 172.21.15.69:0/3935551350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5480101870 msgr2=0x7f54800755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f547f400640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5480101870 0x7f54800755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f547f400640 1 -- 172.21.15.69:0/3935551350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54800749e0 con 0x7f5480102a60 2024-03-19T01:10:18.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.044+0000 7f5484a00640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5480103420 0x7f5480074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:18.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.045+0000 7f547f400640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5480102a60 0x7f5480073c10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f546c00d950 tx=0x7f546c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:18.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.045+0000 7f547cc00640 1 -- 172.21.15.69:0/3935551350 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c017030 con 0x7f5480102a60 2024-03-19T01:10:18.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.045+0000 7f547cc00640 1 -- 172.21.15.69:0/3935551350 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f546c004020 con 0x7f5480102a60 2024-03-19T01:10:18.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.045+0000 7f547cc00640 1 -- 172.21.15.69:0/3935551350 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c0047a0 con 0x7f5480102a60 2024-03-19T01:10:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.045+0000 7f5486a00640 1 -- 172.21.15.69:0/3935551350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54801a1b90 con 0x7f5480102a60 2024-03-19T01:10:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.045+0000 7f5486a00640 1 -- 172.21.15.69:0/3935551350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54801a2130 con 0x7f5480102a60 2024-03-19T01:10:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.046+0000 7f5486a00640 1 -- 172.21.15.69:0/3935551350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5480108b30 con 0x7f5480102a60 2024-03-19T01:10:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.049+0000 7f547cc00640 1 -- 172.21.15.69:0/3935551350 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f546c0041d0 con 0x7f5480102a60 2024-03-19T01:10:18.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.050+0000 7f547cc00640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f54640a2920 0x7f54640a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:18.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.050+0000 7f547cc00640 1 -- 172.21.15.69:0/3935551350 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f546c0ce760 con 0x7f5480102a60 2024-03-19T01:10:18.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.050+0000 7f547fe00640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f54640a2920 0x7f54640a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:18.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.051+0000 7f547fe00640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f54640a2920 0x7f54640a4de0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f54801028c0 tx=0x7f5470005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:18.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.059+0000 7f547cc00640 1 -- 172.21.15.69:0/3935551350 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f546c09b3f0 con 0x7f5480102a60 2024-03-19T01:10:18.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:17 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1605269567' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T01:10:18.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:17 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T01:10:18.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:17 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1605269567' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T01:10:18.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:17 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T01:10:18.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:17 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1605269567' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T01:10:18.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:17 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T01:10:18.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.285+0000 7f5486a00640 1 -- 172.21.15.69:0/3935551350 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f5480061dc0 con 0x7f54640a2920 2024-03-19T01:10:18.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.310+0000 7f547cc00640 1 -- 172.21.15.69:0/3935551350 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f5480061dc0 con 0x7f54640a2920 2024-03-19T01:10:18.311 INFO:teuthology.orchestra.run.smithi069.stdout:{} 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.312+0000 7f5486a00640 1 -- 172.21.15.69:0/3935551350 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f54640a2920 msgr2=0x7f54640a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.312+0000 7f5486a00640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f54640a2920 0x7f54640a4de0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f54801028c0 tx=0x7f5470005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.312+0000 7f5486a00640 1 -- 172.21.15.69:0/3935551350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5480102a60 msgr2=0x7f5480073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.312+0000 7f5486a00640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5480102a60 0x7f5480073c10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f546c00d950 tx=0x7f546c00de10 comp rx=0 tx=0).stop 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.312+0000 7f5486a00640 1 -- 172.21.15.69:0/3935551350 shutdown_connections 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.312+0000 7f5486a00640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f54640a2920 0x7f54640a4de0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.312+0000 7f5486a00640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5480103420 0x7f5480074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.312+0000 7f5486a00640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5480102a60 0x7f5480073c10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.312+0000 7f5486a00640 1 --2- 172.21.15.69:0/3935551350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5480101870 0x7f54800755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.312+0000 7f5486a00640 1 -- 172.21.15.69:0/3935551350 >> 172.21.15.69:0/3935551350 conn(0x7f54800fcde0 msgr2=0x7f5480105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.313+0000 7f5486a00640 1 -- 172.21.15.69:0/3935551350 shutdown_connections 2024-03-19T01:10:18.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.313+0000 7f5486a00640 1 -- 172.21.15.69:0/3935551350 wait complete. 2024-03-19T01:10:18.333 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-19T01:10:18.345 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-19T01:10:18.356 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:10:18.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.543+0000 7fcf4be00640 1 -- 172.21.15.69:0/1198191722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf4c101bf0 msgr2=0x7fcf4c102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:18.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.543+0000 7fcf4be00640 1 --2- 172.21.15.69:0/1198191722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf4c101bf0 0x7fcf4c102010 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009900 tx=0x7fcf3c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:18.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.543+0000 7fcf4be00640 1 -- 172.21.15.69:0/1198191722 shutdown_connections 2024-03-19T01:10:18.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.543+0000 7fcf4be00640 1 --2- 172.21.15.69:0/1198191722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c1000a0 0x7fcf4c100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.543+0000 7fcf4be00640 1 --2- 172.21.15.69:0/1198191722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf4c0ff6e0 0x7fcf4c0ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.543+0000 7fcf4be00640 1 --2- 172.21.15.69:0/1198191722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf4c101bf0 0x7fcf4c102010 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.544+0000 7fcf4be00640 1 -- 172.21.15.69:0/1198191722 >> 172.21.15.69:0/1198191722 conn(0x7fcf4c0fb260 msgr2=0x7fcf4c0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:18.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.544+0000 7fcf4be00640 1 -- 172.21.15.69:0/1198191722 shutdown_connections 2024-03-19T01:10:18.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.544+0000 7fcf4be00640 1 -- 172.21.15.69:0/1198191722 wait complete. 2024-03-19T01:10:18.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.544+0000 7fcf4be00640 1 Processor -- start 2024-03-19T01:10:18.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4be00640 1 -- start start 2024-03-19T01:10:18.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c0ff6e0 0x7fcf4c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:18.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf4c1000a0 0x7fcf4c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:18.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf4c101bf0 0x7fcf4c1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:18.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4be00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c10f520 con 0x7fcf4c1000a0 2024-03-19T01:10:18.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c10f3a0 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4be00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c10f6a0 con 0x7fcf4c101bf0 2024-03-19T01:10:18.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c0ff6e0 0x7fcf4c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:18.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c0ff6e0 0x7fcf4c199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54514/0 (socket says 172.21.15.69:54514) 2024-03-19T01:10:18.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4aa00640 1 -- 172.21.15.69:0/1855413917 learned_addr learned my addr 172.21.15.69:0/1855413917 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4a000640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf4c1000a0 0x7fcf4c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.545+0000 7fcf4b400640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf4c101bf0 0x7fcf4c1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf4aa00640 1 -- 172.21.15.69:0/1855413917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf4c101bf0 msgr2=0x7fcf4c1a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf4aa00640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf4c101bf0 0x7fcf4c1a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf4aa00640 1 -- 172.21.15.69:0/1855413917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf4c1000a0 msgr2=0x7fcf4c19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf4aa00640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf4c1000a0 0x7fcf4c19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf4aa00640 1 -- 172.21.15.69:0/1855413917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c1a1760 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf4a000640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf4c1000a0 0x7fcf4c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf4aa00640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c0ff6e0 0x7fcf4c199b40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c000c00 tx=0x7fcf3c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf43400640 1 -- 172.21.15.69:0/1855413917 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c043020 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf43400640 1 -- 172.21.15.69:0/1855413917 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c0054e0 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf43400640 1 -- 172.21.15.69:0/1855413917 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c003c40 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf4be00640 1 -- 172.21.15.69:0/1855413917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c1a19f0 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.546+0000 7fcf4be00640 1 -- 172.21.15.69:0/1855413917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c1a1ef0 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.548+0000 7fcf4be00640 1 -- 172.21.15.69:0/1855413917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4c067ca0 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.551+0000 7fcf43400640 1 -- 172.21.15.69:0/1855413917 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fcf3c046020 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.552+0000 7fcf43400640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcf300a2800 0x7fcf300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:18.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.552+0000 7fcf43400640 1 -- 172.21.15.69:0/1855413917 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fcf3c0f0f60 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.552+0000 7fcf4a000640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcf300a2800 0x7fcf300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:18.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.553+0000 7fcf4a000640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcf300a2800 0x7fcf300a4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c19b050 tx=0x7fcf38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:18.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.556+0000 7fcf43400640 1 -- 172.21.15.69:0/1855413917 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcf3c04d020 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.773+0000 7fcf4be00640 1 -- 172.21.15.69:0/1855413917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf4c062420 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.774+0000 7fcf43400640 1 -- 172.21.15.69:0/1855413917 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24736 (secure 0 0 0) 0x7fcf3c0bdbf0 con 0x7fcf4c0ff6e0 2024-03-19T01:10:18.776 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:18.776 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:09:25.783660+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:10:18.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.777+0000 7fcf4be00640 1 -- 172.21.15.69:0/1855413917 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcf300a2800 msgr2=0x7fcf300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.777+0000 7fcf4be00640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcf300a2800 0x7fcf300a4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c19b050 tx=0x7fcf38005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.777+0000 7fcf4be00640 1 -- 172.21.15.69:0/1855413917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c0ff6e0 msgr2=0x7fcf4c199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.777+0000 7fcf4be00640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c0ff6e0 0x7fcf4c199b40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c000c00 tx=0x7fcf3c02c650 comp rx=0 tx=0).stop 2024-03-19T01:10:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.778+0000 7fcf4be00640 1 -- 172.21.15.69:0/1855413917 shutdown_connections 2024-03-19T01:10:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.778+0000 7fcf4be00640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fcf300a2800 0x7fcf300a4cc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.778+0000 7fcf4be00640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf4c101bf0 0x7fcf4c1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.778+0000 7fcf4be00640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf4c1000a0 0x7fcf4c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.778+0000 7fcf4be00640 1 --2- 172.21.15.69:0/1855413917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c0ff6e0 0x7fcf4c199b40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.778+0000 7fcf4be00640 1 -- 172.21.15.69:0/1855413917 >> 172.21.15.69:0/1855413917 conn(0x7fcf4c0fb260 msgr2=0x7fcf4c0fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.778+0000 7fcf4be00640 1 -- 172.21.15.69:0/1855413917 shutdown_connections 2024-03-19T01:10:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:18.778+0000 7fcf4be00640 1 -- 172.21.15.69:0/1855413917 wait complete. 2024-03-19T01:10:18.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:18 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:18.292+0000 7f248b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:18.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:18 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:18.292+0000 7f248b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:18.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:18 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:18.292+0000 7f248b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:18.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:18 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:18.292+0000 7f248b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:18.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:18 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:18.292+0000 7f248b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:18.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:07:13.368130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:07:14.444206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T01:07:18.189146+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T01:10:18.797 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:10:19.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.026+0000 7f79f3200640 1 -- 172.21.15.69:0/3413256964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79f4101720 msgr2=0x7f79f4101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.026+0000 7f79f3200640 1 --2- 172.21.15.69:0/3413256964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79f4101720 0x7f79f4101b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009ca0 tx=0x7f79e402c100 comp rx=0 tx=0).stop 2024-03-19T01:10:19.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.026+0000 7f79f3200640 1 -- 172.21.15.69:0/3413256964 shutdown_connections 2024-03-19T01:10:19.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.026+0000 7f79f3200640 1 --2- 172.21.15.69:0/3413256964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79f41032d0 0x7f79f4109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.026+0000 7f79f3200640 1 --2- 172.21.15.69:0/3413256964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79f4102910 0x7f79f4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.026+0000 7f79f3200640 1 --2- 172.21.15.69:0/3413256964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79f4101720 0x7f79f4101b40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.026+0000 7f79f3200640 1 -- 172.21.15.69:0/3413256964 >> 172.21.15.69:0/3413256964 conn(0x7f79f40fcc90 msgr2=0x7f79f40ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:19.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.026+0000 7f79f3200640 1 -- 172.21.15.69:0/3413256964 shutdown_connections 2024-03-19T01:10:19.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.026+0000 7f79f3200640 1 -- 172.21.15.69:0/3413256964 wait complete. 2024-03-19T01:10:19.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.027+0000 7f79f3200640 1 Processor -- start 2024-03-19T01:10:19.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.027+0000 7f79f3200640 1 -- start start 2024-03-19T01:10:19.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.027+0000 7f79f3200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79f4101720 0x7f79f4199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.027+0000 7f79f3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79f4102910 0x7f79f419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.027+0000 7f79f3200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79f41032d0 0x7f79f41a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.027+0000 7f79f3200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f79f410d3a0 con 0x7f79f4101720 2024-03-19T01:10:19.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.027+0000 7f79f3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79f410d220 con 0x7f79f4102910 2024-03-19T01:10:19.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.027+0000 7f79f3200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f79f410d520 con 0x7f79f41032d0 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.028+0000 7f79e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79f4102910 0x7f79f419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.028+0000 7f79f2800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79f41032d0 0x7f79f41a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.028+0000 7f79e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79f4102910 0x7f79f419a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54532/0 (socket says 172.21.15.69:54532) 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.028+0000 7f79e9400640 1 -- 172.21.15.69:0/3065937320 learned_addr learned my addr 172.21.15.69:0/3065937320 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.028+0000 7f79f1e00640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79f4101720 0x7f79f4199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.028+0000 7f79e9400640 1 -- 172.21.15.69:0/3065937320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79f41032d0 msgr2=0x7f79f41a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.028+0000 7f79e9400640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79f41032d0 0x7f79f41a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.028+0000 7f79e9400640 1 -- 172.21.15.69:0/3065937320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79f4101720 msgr2=0x7f79f4199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.028+0000 7f79e9400640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79f4101720 0x7f79f4199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.028+0000 7f79e9400640 1 -- 172.21.15.69:0/3065937320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f41a18c0 con 0x7f79f4102910 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.028+0000 7f79e9400640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79f4102910 0x7f79f419a1c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00e9f0 tx=0x7f79dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.029+0000 7f79eb400640 1 -- 172.21.15.69:0/3065937320 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc00cd80 con 0x7f79f4102910 2024-03-19T01:10:19.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.029+0000 7f79eb400640 1 -- 172.21.15.69:0/3065937320 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79dc010910 con 0x7f79f4102910 2024-03-19T01:10:19.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.029+0000 7f79f3200640 1 -- 172.21.15.69:0/3065937320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f41a1bb0 con 0x7f79f4102910 2024-03-19T01:10:19.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.029+0000 7f79eb400640 1 -- 172.21.15.69:0/3065937320 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc018990 con 0x7f79f4102910 2024-03-19T01:10:19.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.029+0000 7f79f3200640 1 -- 172.21.15.69:0/3065937320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f41a20d0 con 0x7f79f4102910 2024-03-19T01:10:19.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.030+0000 7f79f3200640 1 -- 172.21.15.69:0/3065937320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f41a22a0 con 0x7f79f4102910 2024-03-19T01:10:19.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.032+0000 7f79eb400640 1 -- 172.21.15.69:0/3065937320 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f79dc010ac0 con 0x7f79f4102910 2024-03-19T01:10:19.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.033+0000 7f79eb400640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79d80a2800 0x7f79d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.033+0000 7f79eb400640 1 -- 172.21.15.69:0/3065937320 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f79dc0cf1a0 con 0x7f79f4102910 2024-03-19T01:10:19.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.033+0000 7f79f1e00640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79d80a2800 0x7f79d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.034+0000 7f79f1e00640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79d80a2800 0x7f79d80a4cc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f79e402c710 tx=0x7f79e40323e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:19.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.037+0000 7f79eb400640 1 -- 172.21.15.69:0/3065937320 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f79dc09b7e0 con 0x7f79f4102910 2024-03-19T01:10:19.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.237+0000 7f79f3200640 1 -- 172.21.15.69:0/3065937320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f79f41a2630 con 0x7f79f4102910 2024-03-19T01:10:19.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.238+0000 7f79eb400640 1 -- 172.21.15.69:0/3065937320 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f79dc09f880 con 0x7f79f4102910 2024-03-19T01:10:19.239 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:10:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.240+0000 7f79f3200640 1 -- 172.21.15.69:0/3065937320 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79d80a2800 msgr2=0x7f79d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.240+0000 7f79f3200640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79d80a2800 0x7f79d80a4cc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f79e402c710 tx=0x7f79e40323e0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.241+0000 7f79f3200640 1 -- 172.21.15.69:0/3065937320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79f4102910 msgr2=0x7f79f419a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.241+0000 7f79f3200640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79f4102910 0x7f79f419a1c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00e9f0 tx=0x7f79dc00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.241+0000 7f79f3200640 1 -- 172.21.15.69:0/3065937320 shutdown_connections 2024-03-19T01:10:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.241+0000 7f79f3200640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f79d80a2800 0x7f79d80a4cc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.241+0000 7f79f3200640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79f41032d0 0x7f79f41a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.241+0000 7f79f3200640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79f4102910 0x7f79f419a1c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.241+0000 7f79f3200640 1 --2- 172.21.15.69:0/3065937320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79f4101720 0x7f79f4199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.241+0000 7f79f3200640 1 -- 172.21.15.69:0/3065937320 >> 172.21.15.69:0/3065937320 conn(0x7f79f40fcc90 msgr2=0x7f79f4073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:19.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.241+0000 7f79f3200640 1 -- 172.21.15.69:0/3065937320 shutdown_connections 2024-03-19T01:10:19.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.241+0000 7f79f3200640 1 -- 172.21.15.69:0/3065937320 wait complete. 2024-03-19T01:10:19.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:18 smithi086 ceph-mon[66829]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-19T01:10:19.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:18 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-19T01:10:19.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:18 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:10:19.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:18 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1855413917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:19.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:18 smithi069 ceph-mon[31727]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-19T01:10:19.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:18 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-19T01:10:19.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:18 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:10:19.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:18 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1855413917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:19.266 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:10:19.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:18 smithi163 ceph-mon[67428]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-19T01:10:19.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:18 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-19T01:10:19.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:18 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:10:19.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:18 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1855413917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.469+0000 7fbc40e00640 1 -- 172.21.15.69:0/2205165106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c103a60 msgr2=0x7fbc3c103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.469+0000 7fbc40e00640 1 --2- 172.21.15.69:0/2205165106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c103a60 0x7fbc3c103e80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009900 tx=0x7fbc2c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.470+0000 7fbc40e00640 1 -- 172.21.15.69:0/2205165106 shutdown_connections 2024-03-19T01:10:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.470+0000 7fbc40e00640 1 --2- 172.21.15.69:0/2205165106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3c105610 0x7fbc3c10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.470+0000 7fbc40e00640 1 --2- 172.21.15.69:0/2205165106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc3c104c50 0x7fbc3c1050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.470+0000 7fbc40e00640 1 --2- 172.21.15.69:0/2205165106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c103a60 0x7fbc3c103e80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.470+0000 7fbc40e00640 1 -- 172.21.15.69:0/2205165106 >> 172.21.15.69:0/2205165106 conn(0x7fbc3c0fefd0 msgr2=0x7fbc3c101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.470+0000 7fbc40e00640 1 -- 172.21.15.69:0/2205165106 shutdown_connections 2024-03-19T01:10:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.470+0000 7fbc40e00640 1 -- 172.21.15.69:0/2205165106 wait complete. 2024-03-19T01:10:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.471+0000 7fbc40e00640 1 Processor -- start 2024-03-19T01:10:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.471+0000 7fbc40e00640 1 -- start start 2024-03-19T01:10:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.472+0000 7fbc40e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc3c103a60 0x7fbc3c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.472+0000 7fbc40e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c104c50 0x7fbc3c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.472+0000 7fbc40e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3c105610 0x7fbc3c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.472+0000 7fbc40e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10f700 con 0x7fbc3c103a60 2024-03-19T01:10:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.472+0000 7fbc40e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10f580 con 0x7fbc3c104c50 2024-03-19T01:10:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.472+0000 7fbc40e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10f880 con 0x7fbc3c105610 2024-03-19T01:10:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.472+0000 7fbc39e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc3c103a60 0x7fbc3c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.472+0000 7fbc33e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c104c50 0x7fbc3c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.472+0000 7fbc39e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc3c103a60 0x7fbc3c199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38964/0 (socket says 172.21.15.69:38964) 2024-03-19T01:10:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.472+0000 7fbc39e00640 1 -- 172.21.15.69:0/1838936168 learned_addr learned my addr 172.21.15.69:0/1838936168 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.472+0000 7fbc3a800640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3c105610 0x7fbc3c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc33e00640 1 -- 172.21.15.69:0/1838936168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3c105610 msgr2=0x7fbc3c1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc33e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3c105610 0x7fbc3c1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc33e00640 1 -- 172.21.15.69:0/1838936168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc3c103a60 msgr2=0x7fbc3c199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc33e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc3c103a60 0x7fbc3c199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc33e00640 1 -- 172.21.15.69:0/1838936168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c1a19d0 con 0x7fbc3c104c50 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc3a800640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3c105610 0x7fbc3c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc39e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc3c103a60 0x7fbc3c199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc33e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c104c50 0x7fbc3c19a300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400e9f0 tx=0x7fbc2400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc32a00640 1 -- 172.21.15.69:0/1838936168 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400ccc0 con 0x7fbc3c104c50 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc32a00640 1 -- 172.21.15.69:0/1838936168 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc24010910 con 0x7fbc3c104c50 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc40e00640 1 -- 172.21.15.69:0/1838936168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c1a1cc0 con 0x7fbc3c104c50 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.473+0000 7fbc32a00640 1 -- 172.21.15.69:0/1838936168 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24018960 con 0x7fbc3c104c50 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.474+0000 7fbc40e00640 1 -- 172.21.15.69:0/1838936168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c1a21b0 con 0x7fbc3c104c50 2024-03-19T01:10:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.475+0000 7fbc40e00640 1 -- 172.21.15.69:0/1838936168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3c10add0 con 0x7fbc3c104c50 2024-03-19T01:10:19.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.476+0000 7fbc32a00640 1 -- 172.21.15.69:0/1838936168 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fbc24010ac0 con 0x7fbc3c104c50 2024-03-19T01:10:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.478+0000 7fbc32a00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbc200a2880 0x7fbc200a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.478+0000 7fbc32a00640 1 -- 172.21.15.69:0/1838936168 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fbc240cf210 con 0x7fbc3c104c50 2024-03-19T01:10:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.478+0000 7fbc39e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbc200a2880 0x7fbc200a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.479+0000 7fbc39e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbc200a2880 0x7fbc200a4d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c19b290 tx=0x7fbc2c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:19.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.482+0000 7fbc32a00640 1 -- 172.21.15.69:0/1838936168 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbc2409b850 con 0x7fbc3c104c50 2024-03-19T01:10:19.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.694+0000 7fbc40e00640 1 -- 172.21.15.69:0/1838936168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbc3c0624f0 con 0x7fbc3c104c50 2024-03-19T01:10:19.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.695+0000 7fbc32a00640 1 -- 172.21.15.69:0/1838936168 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fbc2409f8f0 con 0x7fbc3c104c50 2024-03-19T01:10:19.696 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:10:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 -- 172.21.15.69:0/1838936168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbc200a2880 msgr2=0x7fbc200a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbc200a2880 0x7fbc200a4d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c19b290 tx=0x7fbc2c02df90 comp rx=0 tx=0).stop 2024-03-19T01:10:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 -- 172.21.15.69:0/1838936168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c104c50 msgr2=0x7fbc3c19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c104c50 0x7fbc3c19a300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400e9f0 tx=0x7fbc2400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 -- 172.21.15.69:0/1838936168 shutdown_connections 2024-03-19T01:10:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbc200a2880 0x7fbc200a4d40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3c105610 0x7fbc3c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c104c50 0x7fbc3c19a300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 --2- 172.21.15.69:0/1838936168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc3c103a60 0x7fbc3c199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 -- 172.21.15.69:0/1838936168 >> 172.21.15.69:0/1838936168 conn(0x7fbc3c0fefd0 msgr2=0x7fbc3c107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 -- 172.21.15.69:0/1838936168 shutdown_connections 2024-03-19T01:10:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.698+0000 7fbc40e00640 1 -- 172.21.15.69:0/1838936168 wait complete. 2024-03-19T01:10:19.723 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:10:19.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.924+0000 7fd66e400640 1 -- 172.21.15.69:0/3336006934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6681075c0 msgr2=0x7fd6681099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.924+0000 7fd66e400640 1 --2- 172.21.15.69:0/3336006934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6681075c0 0x7fd6681099b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd660009be0 tx=0x7fd66002bef0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.924+0000 7fd66e400640 1 -- 172.21.15.69:0/3336006934 shutdown_connections 2024-03-19T01:10:19.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.924+0000 7fd66e400640 1 --2- 172.21.15.69:0/3336006934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6681075c0 0x7fd6681099b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.924+0000 7fd66e400640 1 --2- 172.21.15.69:0/3336006934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd668104c90 0x7fd668107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.924+0000 7fd66e400640 1 --2- 172.21.15.69:0/3336006934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd668102360 0x7fd668104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.924+0000 7fd66e400640 1 -- 172.21.15.69:0/3336006934 >> 172.21.15.69:0/3336006934 conn(0x7fd6680fbcb0 msgr2=0x7fd6680fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:19.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.924+0000 7fd66e400640 1 -- 172.21.15.69:0/3336006934 shutdown_connections 2024-03-19T01:10:19.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.924+0000 7fd66e400640 1 -- 172.21.15.69:0/3336006934 wait complete. 2024-03-19T01:10:19.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.926+0000 7fd66e400640 1 Processor -- start 2024-03-19T01:10:19.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.926+0000 7fd66e400640 1 -- start start 2024-03-19T01:10:19.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.926+0000 7fd66e400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd668102360 0x7fd668101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.926+0000 7fd66e400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd668104c90 0x7fd668100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.926+0000 7fd66e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6681075c0 0x7fd668100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.926+0000 7fd66e400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd66810d4c0 con 0x7fd668102360 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.926+0000 7fd66e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd66810d340 con 0x7fd6681075c0 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.926+0000 7fd66e400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd66810d640 con 0x7fd668104c90 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.927+0000 7fd666a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd668104c90 0x7fd668100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.927+0000 7fd667400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd668102360 0x7fd668101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.927+0000 7fd667e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6681075c0 0x7fd668100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.927+0000 7fd666a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd668104c90 0x7fd668100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36122/0 (socket says 172.21.15.69:36122) 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.927+0000 7fd666a00640 1 -- 172.21.15.69:0/4028368188 learned_addr learned my addr 172.21.15.69:0/4028368188 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.927+0000 7fd667e00640 1 -- 172.21.15.69:0/4028368188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd668104c90 msgr2=0x7fd668100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.927+0000 7fd667e00640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd668104c90 0x7fd668100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.927+0000 7fd667e00640 1 -- 172.21.15.69:0/4028368188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd668102360 msgr2=0x7fd668101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.927+0000 7fd667e00640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd668102360 0x7fd668101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.927+0000 7fd667e00640 1 -- 172.21.15.69:0/4028368188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6681010e0 con 0x7fd6681075c0 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.927+0000 7fd666a00640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd668104c90 0x7fd668100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:19.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.928+0000 7fd667e00640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6681075c0 0x7fd668100850 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd660036040 tx=0x7fd6600337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:19.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.928+0000 7fd65fe00640 1 -- 172.21.15.69:0/4028368188 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660043020 con 0x7fd6681075c0 2024-03-19T01:10:19.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.928+0000 7fd66e400640 1 -- 172.21.15.69:0/4028368188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6681a1c60 con 0x7fd6681075c0 2024-03-19T01:10:19.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.928+0000 7fd65fe00640 1 -- 172.21.15.69:0/4028368188 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd66002ca60 con 0x7fd6681075c0 2024-03-19T01:10:19.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.928+0000 7fd65fe00640 1 -- 172.21.15.69:0/4028368188 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66003e6b0 con 0x7fd6681075c0 2024-03-19T01:10:19.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.928+0000 7fd66e400640 1 -- 172.21.15.69:0/4028368188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6681a21b0 con 0x7fd6681075c0 2024-03-19T01:10:19.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.929+0000 7fd66e400640 1 -- 172.21.15.69:0/4028368188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6680fd7a0 con 0x7fd6681075c0 2024-03-19T01:10:19.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.931+0000 7fd65fe00640 1 -- 172.21.15.69:0/4028368188 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd660046020 con 0x7fd6681075c0 2024-03-19T01:10:19.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.932+0000 7fd65fe00640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd64c0a2800 0x7fd64c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:19.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.932+0000 7fd65fe00640 1 -- 172.21.15.69:0/4028368188 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fd6600f13a0 con 0x7fd6681075c0 2024-03-19T01:10:19.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.932+0000 7fd667400640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd64c0a2800 0x7fd64c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:19.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.933+0000 7fd667400640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd64c0a2800 0x7fd64c0a4cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd658009530 tx=0x7fd658009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:19.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:19.937+0000 7fd65fe00640 1 -- 172.21.15.69:0/4028368188 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd66004d020 con 0x7fd6681075c0 2024-03-19T01:10:20.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:19 smithi069 ceph-mon[31727]: from='client.25454 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:20.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:19 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3065937320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:20.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:19 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1838936168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:20.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.143+0000 7fd66e400640 1 -- 172.21.15.69:0/4028368188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd66802d1f0 con 0x7fd6681075c0 2024-03-19T01:10:20.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.144+0000 7fd65fe00640 1 -- 172.21.15.69:0/4028368188 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fd6600be030 con 0x7fd6681075c0 2024-03-19T01:10:20.146 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-03-19T01:10:20.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.146+0000 7fd66e400640 1 -- 172.21.15.69:0/4028368188 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd64c0a2800 msgr2=0x7fd64c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:20.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.146+0000 7fd66e400640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd64c0a2800 0x7fd64c0a4cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd658009530 tx=0x7fd658009260 comp rx=0 tx=0).stop 2024-03-19T01:10:20.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.146+0000 7fd66e400640 1 -- 172.21.15.69:0/4028368188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6681075c0 msgr2=0x7fd668100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:20.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.146+0000 7fd66e400640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6681075c0 0x7fd668100850 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd660036040 tx=0x7fd6600337b0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.147+0000 7fd66e400640 1 -- 172.21.15.69:0/4028368188 shutdown_connections 2024-03-19T01:10:20.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.147+0000 7fd66e400640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd64c0a2800 0x7fd64c0a4cc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.147+0000 7fd66e400640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6681075c0 0x7fd668100850 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.147+0000 7fd66e400640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd668104c90 0x7fd668100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.147+0000 7fd66e400640 1 --2- 172.21.15.69:0/4028368188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd668102360 0x7fd668101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.147+0000 7fd66e400640 1 -- 172.21.15.69:0/4028368188 >> 172.21.15.69:0/4028368188 conn(0x7fd6680fbcb0 msgr2=0x7fd668102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:20.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.147+0000 7fd66e400640 1 -- 172.21.15.69:0/4028368188 shutdown_connections 2024-03-19T01:10:20.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.147+0000 7fd66e400640 1 -- 172.21.15.69:0/4028368188 wait complete. 2024-03-19T01:10:20.166 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T01:10:20.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:19 smithi086 ceph-mon[66829]: from='client.25454 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:20.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:19 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3065937320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:20.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:19 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1838936168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:19 smithi163 ceph-mon[67428]: from='client.25454 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:19 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3065937320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:19 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1838936168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:20.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.366+0000 7fb8a9a00640 1 -- 172.21.15.69:0/282708324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8a4075800 msgr2=0x7fb8a4073c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:20.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.366+0000 7fb8a9a00640 1 --2- 172.21.15.69:0/282708324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8a4075800 0x7fb8a4073c50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb894009900 tx=0x7fb89402bcd0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.367+0000 7fb8a9a00640 1 -- 172.21.15.69:0/282708324 shutdown_connections 2024-03-19T01:10:20.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.367+0000 7fb8a9a00640 1 --2- 172.21.15.69:0/282708324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8a41095b0 0x7fb8a410ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.367+0000 7fb8a9a00640 1 --2- 172.21.15.69:0/282708324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4074220 0x7fb8a4074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.367+0000 7fb8a9a00640 1 --2- 172.21.15.69:0/282708324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8a4075800 0x7fb8a4073c50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.367+0000 7fb8a9a00640 1 -- 172.21.15.69:0/282708324 >> 172.21.15.69:0/282708324 conn(0x7fb8a40fcc90 msgr2=0x7fb8a40ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:20.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.367+0000 7fb8a9a00640 1 -- 172.21.15.69:0/282708324 shutdown_connections 2024-03-19T01:10:20.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.367+0000 7fb8a9a00640 1 -- 172.21.15.69:0/282708324 wait complete. 2024-03-19T01:10:20.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a9a00640 1 Processor -- start 2024-03-19T01:10:20.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a9a00640 1 -- start start 2024-03-19T01:10:20.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8a4074220 0x7fb8a4199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:20.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a9a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8a4075800 0x7fb8a419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:20.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a41095b0 0x7fb8a41a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a9a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a410f5e0 con 0x7fb8a4075800 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a410f460 con 0x7fb8a41095b0 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a9a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a410f760 con 0x7fb8a4074220 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8a4074220 0x7fb8a4199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8a4074220 0x7fb8a4199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36130/0 (socket says 172.21.15.69:36130) 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a3e00640 1 -- 172.21.15.69:0/3719500092 learned_addr learned my addr 172.21.15.69:0/3719500092 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.368+0000 7fb8a9000640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a41095b0 0x7fb8a41a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.369+0000 7fb8a3400640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8a4075800 0x7fb8a419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.369+0000 7fb8a3e00640 1 -- 172.21.15.69:0/3719500092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a41095b0 msgr2=0x7fb8a41a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.369+0000 7fb8a3e00640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a41095b0 0x7fb8a41a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.369+0000 7fb8a3e00640 1 -- 172.21.15.69:0/3719500092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8a4075800 msgr2=0x7fb8a419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.369+0000 7fb8a3e00640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8a4075800 0x7fb8a419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.369+0000 7fb8a3e00640 1 -- 172.21.15.69:0/3719500092 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a41a18c0 con 0x7fb8a4074220 2024-03-19T01:10:20.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.369+0000 7fb8a3400640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8a4075800 0x7fb8a419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:20.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.369+0000 7fb8a3e00640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8a4074220 0x7fb8a4199c80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb894005ab0 tx=0x7fb89402c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:20.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.370+0000 7fb8a0c00640 1 -- 172.21.15.69:0/3719500092 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894043020 con 0x7fb8a4074220 2024-03-19T01:10:20.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.370+0000 7fb8a0c00640 1 -- 172.21.15.69:0/3719500092 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8940054e0 con 0x7fb8a4074220 2024-03-19T01:10:20.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.370+0000 7fb8a9a00640 1 -- 172.21.15.69:0/3719500092 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a41a1b50 con 0x7fb8a4074220 2024-03-19T01:10:20.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.370+0000 7fb8a0c00640 1 -- 172.21.15.69:0/3719500092 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894003ca0 con 0x7fb8a4074220 2024-03-19T01:10:20.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.370+0000 7fb8a9a00640 1 -- 172.21.15.69:0/3719500092 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a41a20a0 con 0x7fb8a4074220 2024-03-19T01:10:20.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.371+0000 7fb8a9a00640 1 -- 172.21.15.69:0/3719500092 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a4108370 con 0x7fb8a4074220 2024-03-19T01:10:20.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.373+0000 7fb8a0c00640 1 -- 172.21.15.69:0/3719500092 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb894046020 con 0x7fb8a4074220 2024-03-19T01:10:20.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.374+0000 7fb8a0c00640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8880a2800 0x7fb8880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:20.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.374+0000 7fb8a0c00640 1 -- 172.21.15.69:0/3719500092 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fb8940f0ff0 con 0x7fb8a4074220 2024-03-19T01:10:20.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.374+0000 7fb8a3400640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8880a2800 0x7fb8880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:20.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.375+0000 7fb8a3400640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8880a2800 0x7fb8880a4cc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb8a419b180 tx=0x7fb890005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:20.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.378+0000 7fb8a0c00640 1 -- 172.21.15.69:0/3719500092 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb89404d020 con 0x7fb8a4074220 2024-03-19T01:10:20.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.671+0000 7fb8a9a00640 1 -- 172.21.15.69:0/3719500092 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb8a4062420 con 0x7fb8a4074220 2024-03-19T01:10:20.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.672+0000 7fb8a0c00640 1 -- 172.21.15.69:0/3719500092 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fb8940bdcc0 con 0x7fb8a4074220 2024-03-19T01:10:20.673 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:10:20.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.674+0000 7fb8a9a00640 1 -- 172.21.15.69:0/3719500092 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8880a2800 msgr2=0x7fb8880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:20.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.674+0000 7fb8a9a00640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8880a2800 0x7fb8880a4cc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb8a419b180 tx=0x7fb890005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:20.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.674+0000 7fb8a9a00640 1 -- 172.21.15.69:0/3719500092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8a4074220 msgr2=0x7fb8a4199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:20.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.674+0000 7fb8a9a00640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8a4074220 0x7fb8a4199c80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb894005ab0 tx=0x7fb89402c760 comp rx=0 tx=0).stop 2024-03-19T01:10:20.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.675+0000 7fb8a9a00640 1 -- 172.21.15.69:0/3719500092 shutdown_connections 2024-03-19T01:10:20.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.675+0000 7fb8a9a00640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fb8880a2800 0x7fb8880a4cc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.675+0000 7fb8a9a00640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a41095b0 0x7fb8a41a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.675+0000 7fb8a9a00640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8a4075800 0x7fb8a419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.675+0000 7fb8a9a00640 1 --2- 172.21.15.69:0/3719500092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8a4074220 0x7fb8a4199c80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.675+0000 7fb8a9a00640 1 -- 172.21.15.69:0/3719500092 >> 172.21.15.69:0/3719500092 conn(0x7fb8a40fcc90 msgr2=0x7fb8a4105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:20.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.675+0000 7fb8a9a00640 1 -- 172.21.15.69:0/3719500092 shutdown_connections 2024-03-19T01:10:20.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.675+0000 7fb8a9a00640 1 -- 172.21.15.69:0/3719500092 wait complete. 2024-03-19T01:10:20.696 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:10:20.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.890+0000 7f3b3aa00640 1 -- 172.21.15.69:0/2809048115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b34105430 msgr2=0x7f3b3410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:20.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.890+0000 7f3b3aa00640 1 --2- 172.21.15.69:0/2809048115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b34105430 0x7f3b3410bcc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009ca0 tx=0x7f3b2c02c100 comp rx=0 tx=0).stop 2024-03-19T01:10:20.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.890+0000 7f3b3aa00640 1 -- 172.21.15.69:0/2809048115 shutdown_connections 2024-03-19T01:10:20.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.890+0000 7f3b3aa00640 1 --2- 172.21.15.69:0/2809048115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b34105430 0x7f3b3410bcc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.890+0000 7f3b3aa00640 1 --2- 172.21.15.69:0/2809048115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34104a70 0x7f3b34104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.890+0000 7f3b3aa00640 1 --2- 172.21.15.69:0/2809048115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b34103880 0x7f3b34103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.890+0000 7f3b3aa00640 1 -- 172.21.15.69:0/2809048115 >> 172.21.15.69:0/2809048115 conn(0x7f3b340fedf0 msgr2=0x7f3b34101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:20.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.890+0000 7f3b3aa00640 1 -- 172.21.15.69:0/2809048115 shutdown_connections 2024-03-19T01:10:20.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.890+0000 7f3b3aa00640 1 -- 172.21.15.69:0/2809048115 wait complete. 2024-03-19T01:10:20.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.891+0000 7f3b3aa00640 1 Processor -- start 2024-03-19T01:10:20.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.891+0000 7f3b3aa00640 1 -- start start 2024-03-19T01:10:20.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b3aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b34103880 0x7f3b34199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:20.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34104a70 0x7f3b3419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:20.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b3aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b34105430 0x7f3b341a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:20.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b3aa00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3410f500 con 0x7f3b34103880 2024-03-19T01:10:20.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b3aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3410f380 con 0x7f3b34104a70 2024-03-19T01:10:20.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b3aa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3410f680 con 0x7f3b34105430 2024-03-19T01:10:20.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b38c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34104a70 0x7f3b3419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:20.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b3a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b34105430 0x7f3b341a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:20.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b38c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34104a70 0x7f3b3419a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54598/0 (socket says 172.21.15.69:54598) 2024-03-19T01:10:20.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b38c00640 1 -- 172.21.15.69:0/3188401419 learned_addr learned my addr 172.21.15.69:0/3188401419 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:20.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b39600640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b34103880 0x7f3b34199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b38c00640 1 -- 172.21.15.69:0/3188401419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b34105430 msgr2=0x7f3b341a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b38c00640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b34105430 0x7f3b341a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b38c00640 1 -- 172.21.15.69:0/3188401419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b34103880 msgr2=0x7f3b34199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.892+0000 7f3b38c00640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b34103880 0x7f3b34199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.893+0000 7f3b38c00640 1 -- 172.21.15.69:0/3188401419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b341a1730 con 0x7f3b34104a70 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.893+0000 7f3b3a000640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b34105430 0x7f3b341a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.893+0000 7f3b38c00640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34104a70 0x7f3b3419a050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000e9f0 tx=0x7f3b2000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.893+0000 7f3b32000640 1 -- 172.21.15.69:0/3188401419 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2000cd80 con 0x7f3b34104a70 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.893+0000 7f3b32000640 1 -- 172.21.15.69:0/3188401419 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b20005500 con 0x7f3b34104a70 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.893+0000 7f3b32000640 1 -- 172.21.15.69:0/3188401419 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20010690 con 0x7f3b34104a70 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.893+0000 7f3b3aa00640 1 -- 172.21.15.69:0/3188401419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b341a1a20 con 0x7f3b34104a70 2024-03-19T01:10:20.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.893+0000 7f3b3aa00640 1 -- 172.21.15.69:0/3188401419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341a1f70 con 0x7f3b34104a70 2024-03-19T01:10:20.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.894+0000 7f3b3aa00640 1 -- 172.21.15.69:0/3188401419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3410aba0 con 0x7f3b34104a70 2024-03-19T01:10:20.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.899+0000 7f3b32000640 1 -- 172.21.15.69:0/3188401419 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f3b200046c0 con 0x7f3b34104a70 2024-03-19T01:10:20.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.900+0000 7f3b32000640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3b1c0a2910 0x7f3b1c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:20.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.900+0000 7f3b32000640 1 -- 172.21.15.69:0/3188401419 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f3b200cf2d0 con 0x7f3b34104a70 2024-03-19T01:10:20.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.901+0000 7f3b39600640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3b1c0a2910 0x7f3b1c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:20.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.901+0000 7f3b39600640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3b1c0a2910 0x7f3b1c0a4dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b28005fd0 tx=0x7f3b2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:20.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:20.904+0000 7f3b32000640 1 -- 172.21.15.69:0/3188401419 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3b2009be00 con 0x7f3b34104a70 2024-03-19T01:10:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:20 smithi069 ceph-mon[31727]: pgmap v411: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-19T01:10:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:20 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/4028368188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:21.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:20 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3719500092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:21.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.112+0000 7f3b3aa00640 1 -- 172.21.15.69:0/3188401419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b3402d410 con 0x7f3b34104a70 2024-03-19T01:10:21.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.114+0000 7f3b32000640 1 -- 172.21.15.69:0/3188401419 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24736 (secure 0 0 0) 0x7f3b2009b700 con 0x7f3b34104a70 2024-03-19T01:10:21.116 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:21.116 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:09:25.783660+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:10:21.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 -- 172.21.15.69:0/3188401419 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3b1c0a2910 msgr2=0x7f3b1c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:21.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3b1c0a2910 0x7f3b1c0a4dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b28005fd0 tx=0x7f3b2800b410 comp rx=0 tx=0).stop 2024-03-19T01:10:21.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 -- 172.21.15.69:0/3188401419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34104a70 msgr2=0x7f3b3419a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34104a70 0x7f3b3419a050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000e9f0 tx=0x7f3b2000edf0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 -- 172.21.15.69:0/3188401419 shutdown_connections 2024-03-19T01:10:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3b1c0a2910 0x7f3b1c0a4dd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b34105430 0x7f3b341a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b34104a70 0x7f3b3419a050 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 --2- 172.21.15.69:0/3188401419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b34103880 0x7f3b34199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 -- 172.21.15.69:0/3188401419 >> 172.21.15.69:0/3188401419 conn(0x7f3b340fedf0 msgr2=0x7f3b34107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 -- 172.21.15.69:0/3188401419 shutdown_connections 2024-03-19T01:10:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.117+0000 7f3b3aa00640 1 -- 172.21.15.69:0/3188401419 wait complete. 2024-03-19T01:10:21.143 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:07:13.368130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:07:14.444206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T01:07:18.189146+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T01:10:21.144 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:10:21.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:20 smithi086 ceph-mon[66829]: pgmap v411: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-19T01:10:21.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:20 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/4028368188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:21.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:20 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3719500092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:21.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:20 smithi163 ceph-mon[67428]: pgmap v411: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-19T01:10:21.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:20 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/4028368188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:21.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:20 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3719500092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:21.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.347+0000 7f576ac00640 1 -- 172.21.15.69:0/2423930763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5764103a60 msgr2=0x7f5764103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:21.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.347+0000 7f576ac00640 1 --2- 172.21.15.69:0/2423930763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5764103a60 0x7f5764103e80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5754009860 tx=0x7f575402bb70 comp rx=0 tx=0).stop 2024-03-19T01:10:21.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.347+0000 7f576ac00640 1 -- 172.21.15.69:0/2423930763 shutdown_connections 2024-03-19T01:10:21.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.347+0000 7f576ac00640 1 --2- 172.21.15.69:0/2423930763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5764105610 0x7f576410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.347+0000 7f576ac00640 1 --2- 172.21.15.69:0/2423930763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5764104c50 0x7f57641050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.347+0000 7f576ac00640 1 --2- 172.21.15.69:0/2423930763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5764103a60 0x7f5764103e80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.347+0000 7f576ac00640 1 -- 172.21.15.69:0/2423930763 >> 172.21.15.69:0/2423930763 conn(0x7f57640fefd0 msgr2=0x7f5764101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:21.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.347+0000 7f576ac00640 1 -- 172.21.15.69:0/2423930763 shutdown_connections 2024-03-19T01:10:21.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.347+0000 7f576ac00640 1 -- 172.21.15.69:0/2423930763 wait complete. 2024-03-19T01:10:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.348+0000 7f576ac00640 1 Processor -- start 2024-03-19T01:10:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.348+0000 7f576ac00640 1 -- start start 2024-03-19T01:10:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.348+0000 7f576ac00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5764103a60 0x7f5764199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.348+0000 7f576ac00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5764104c50 0x7f576419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.348+0000 7f576ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5764105610 0x7f57641a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.348+0000 7f576ac00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f576410f6e0 con 0x7f5764103a60 2024-03-19T01:10:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.348+0000 7f576ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f576410f560 con 0x7f5764105610 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.348+0000 7f576ac00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f576410f860 con 0x7f5764104c50 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5763e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5764103a60 0x7f5764199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5763400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5764104c50 0x7f576419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5768c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5764105610 0x7f57641a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5763400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5764104c50 0x7f576419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36158/0 (socket says 172.21.15.69:36158) 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5763400640 1 -- 172.21.15.69:0/2049148466 learned_addr learned my addr 172.21.15.69:0/2049148466 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5763400640 1 -- 172.21.15.69:0/2049148466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5764105610 msgr2=0x7f57641a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5763400640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5764105610 0x7f57641a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5763400640 1 -- 172.21.15.69:0/2049148466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5764103a60 msgr2=0x7f5764199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5763400640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5764103a60 0x7f5764199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5763400640 1 -- 172.21.15.69:0/2049148466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57641a19d0 con 0x7f5764104c50 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5763e00640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5764103a60 0x7f5764199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5768c00640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5764105610 0x7f57641a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.349+0000 7f5763400640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5764104c50 0x7f576419a2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f575000d950 tx=0x7f575000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.350+0000 7f5760c00640 1 -- 172.21.15.69:0/2049148466 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750017030 con 0x7f5764104c50 2024-03-19T01:10:21.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.350+0000 7f5760c00640 1 -- 172.21.15.69:0/2049148466 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5750004020 con 0x7f5764104c50 2024-03-19T01:10:21.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.350+0000 7f5760c00640 1 -- 172.21.15.69:0/2049148466 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57500047a0 con 0x7f5764104c50 2024-03-19T01:10:21.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.350+0000 7f576ac00640 1 -- 172.21.15.69:0/2049148466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57641a1cc0 con 0x7f5764104c50 2024-03-19T01:10:21.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.350+0000 7f576ac00640 1 -- 172.21.15.69:0/2049148466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57641a21c0 con 0x7f5764104c50 2024-03-19T01:10:21.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.351+0000 7f576ac00640 1 -- 172.21.15.69:0/2049148466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f576410ada0 con 0x7f5764104c50 2024-03-19T01:10:21.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.353+0000 7f5760c00640 1 -- 172.21.15.69:0/2049148466 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f57500041d0 con 0x7f5764104c50 2024-03-19T01:10:21.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.355+0000 7f5760c00640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f57480a2910 0x7f57480a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:21.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.355+0000 7f5760c00640 1 -- 172.21.15.69:0/2049148466 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f57500ce940 con 0x7f5764104c50 2024-03-19T01:10:21.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.355+0000 7f5763e00640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f57480a2910 0x7f57480a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:21.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.356+0000 7f5763e00640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f57480a2910 0x7f57480a4dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f57540349e0 tx=0x7f575402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:21.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.358+0000 7f5760c00640 1 -- 172.21.15.69:0/2049148466 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f575009b470 con 0x7f5764104c50 2024-03-19T01:10:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.557+0000 7f576ac00640 1 -- 172.21.15.69:0/2049148466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f576402d3a0 con 0x7f5764104c50 2024-03-19T01:10:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.558+0000 7f5760c00640 1 -- 172.21.15.69:0/2049148466 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f575009ad70 con 0x7f5764104c50 2024-03-19T01:10:21.559 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.560+0000 7f576ac00640 1 -- 172.21.15.69:0/2049148466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f57480a2910 msgr2=0x7f57480a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.560+0000 7f576ac00640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f57480a2910 0x7f57480a4dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f57540349e0 tx=0x7f575402e040 comp rx=0 tx=0).stop 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.560+0000 7f576ac00640 1 -- 172.21.15.69:0/2049148466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5764104c50 msgr2=0x7f576419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.560+0000 7f576ac00640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5764104c50 0x7f576419a2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f575000d950 tx=0x7f575000de10 comp rx=0 tx=0).stop 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.561+0000 7f576ac00640 1 -- 172.21.15.69:0/2049148466 shutdown_connections 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.561+0000 7f576ac00640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f57480a2910 0x7f57480a4dd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.561+0000 7f576ac00640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5764105610 0x7f57641a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.561+0000 7f576ac00640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5764104c50 0x7f576419a2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.561+0000 7f576ac00640 1 --2- 172.21.15.69:0/2049148466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5764103a60 0x7f5764199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.561+0000 7f576ac00640 1 -- 172.21.15.69:0/2049148466 >> 172.21.15.69:0/2049148466 conn(0x7f57640fefd0 msgr2=0x7f5764107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.561+0000 7f576ac00640 1 -- 172.21.15.69:0/2049148466 shutdown_connections 2024-03-19T01:10:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.561+0000 7f576ac00640 1 -- 172.21.15.69:0/2049148466 wait complete. 2024-03-19T01:10:21.580 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:10:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.776+0000 7f9e24600640 1 -- 172.21.15.69:0/1731866259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c105610 msgr2=0x7f9e1c10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.776+0000 7f9e24600640 1 --2- 172.21.15.69:0/1731866259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c105610 0x7f9e1c10bea0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009ca0 tx=0x7f9e1402c100 comp rx=0 tx=0).stop 2024-03-19T01:10:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.776+0000 7f9e24600640 1 -- 172.21.15.69:0/1731866259 shutdown_connections 2024-03-19T01:10:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.776+0000 7f9e24600640 1 --2- 172.21.15.69:0/1731866259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c105610 0x7f9e1c10bea0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.776+0000 7f9e24600640 1 --2- 172.21.15.69:0/1731866259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c104c50 0x7f9e1c1050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.776+0000 7f9e24600640 1 --2- 172.21.15.69:0/1731866259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e1c103a60 0x7f9e1c103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.776+0000 7f9e24600640 1 -- 172.21.15.69:0/1731866259 >> 172.21.15.69:0/1731866259 conn(0x7f9e1c0fefd0 msgr2=0x7f9e1c101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.776+0000 7f9e24600640 1 -- 172.21.15.69:0/1731866259 shutdown_connections 2024-03-19T01:10:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.776+0000 7f9e24600640 1 -- 172.21.15.69:0/1731866259 wait complete. 2024-03-19T01:10:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.777+0000 7f9e24600640 1 Processor -- start 2024-03-19T01:10:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e24600640 1 -- start start 2024-03-19T01:10:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e24600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e1c103a60 0x7f9e1c199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e24600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c104c50 0x7f9e1c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e24600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c105610 0x7f9e1c1a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e24600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c10f6e0 con 0x7f9e1c105610 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e24600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c10f560 con 0x7f9e1c104c50 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e24600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c10f860 con 0x7f9e1c103a60 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e21c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e1c103a60 0x7f9e1c199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e22600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c105610 0x7f9e1c1a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e21200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c104c50 0x7f9e1c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e21c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e1c103a60 0x7f9e1c199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36172/0 (socket says 172.21.15.69:36172) 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e21c00640 1 -- 172.21.15.69:0/2258996803 learned_addr learned my addr 172.21.15.69:0/2258996803 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.778+0000 7f9e21200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c104c50 0x7f9e1c19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54632/0 (socket says 172.21.15.69:54632) 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e21200640 1 -- 172.21.15.69:0/2258996803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e1c103a60 msgr2=0x7f9e1c199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e21200640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e1c103a60 0x7f9e1c199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e21200640 1 -- 172.21.15.69:0/2258996803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c105610 msgr2=0x7f9e1c1a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e21200640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c105610 0x7f9e1c1a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:21.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e21200640 1 -- 172.21.15.69:0/2258996803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c1a1920 con 0x7f9e1c104c50 2024-03-19T01:10:21.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e21c00640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e1c103a60 0x7f9e1c199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:21.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e21200640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c104c50 0x7f9e1c19a250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800e9f0 tx=0x7f9e0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:21.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e1a000640 1 -- 172.21.15.69:0/2258996803 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800ccc0 con 0x7f9e1c104c50 2024-03-19T01:10:21.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e1a000640 1 -- 172.21.15.69:0/2258996803 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e08010910 con 0x7f9e1c104c50 2024-03-19T01:10:21.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e24600640 1 -- 172.21.15.69:0/2258996803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c1a1c10 con 0x7f9e1c104c50 2024-03-19T01:10:21.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e1a000640 1 -- 172.21.15.69:0/2258996803 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08018960 con 0x7f9e1c104c50 2024-03-19T01:10:21.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.779+0000 7f9e24600640 1 -- 172.21.15.69:0/2258996803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c1a21b0 con 0x7f9e1c104c50 2024-03-19T01:10:21.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.781+0000 7f9e24600640 1 -- 172.21.15.69:0/2258996803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c10ad20 con 0x7f9e1c104c50 2024-03-19T01:10:21.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.783+0000 7f9e1a000640 1 -- 172.21.15.69:0/2258996803 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f9e08010ac0 con 0x7f9e1c104c50 2024-03-19T01:10:21.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.784+0000 7f9e1a000640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e040a2910 0x7f9e040a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:21.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.784+0000 7f9e1a000640 1 -- 172.21.15.69:0/2258996803 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f9e080cf3a0 con 0x7f9e1c104c50 2024-03-19T01:10:21.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.784+0000 7f9e21c00640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e040a2910 0x7f9e040a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:21.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.785+0000 7f9e21c00640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e040a2910 0x7f9e040a4dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c19b1e0 tx=0x7f9e10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:21.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:21.788+0000 7f9e1a000640 1 -- 172.21.15.69:0/2258996803 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9e0809b880 con 0x7f9e1c104c50 2024-03-19T01:10:21.846 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:21 smithi069 ceph-mon[31727]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-19T01:10:21.846 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:21 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3188401419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:21.846 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:21 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2049148466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:22.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.011+0000 7f9e24600640 1 -- 172.21.15.69:0/2258996803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9e1c02d410 con 0x7f9e1c104c50 2024-03-19T01:10:22.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.011+0000 7f9e1a000640 1 -- 172.21.15.69:0/2258996803 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f9e0809f920 con 0x7f9e1c104c50 2024-03-19T01:10:22.012 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:10:22.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 -- 172.21.15.69:0/2258996803 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e040a2910 msgr2=0x7f9e040a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e040a2910 0x7f9e040a4dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c19b1e0 tx=0x7f9e10005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:22.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 -- 172.21.15.69:0/2258996803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c104c50 msgr2=0x7f9e1c19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c104c50 0x7f9e1c19a250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800e9f0 tx=0x7f9e0800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 -- 172.21.15.69:0/2258996803 shutdown_connections 2024-03-19T01:10:22.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f9e040a2910 0x7f9e040a4dd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e1c105610 0x7f9e1c1a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e1c104c50 0x7f9e1c19a250 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 --2- 172.21.15.69:0/2258996803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e1c103a60 0x7f9e1c199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 -- 172.21.15.69:0/2258996803 >> 172.21.15.69:0/2258996803 conn(0x7f9e1c0fefd0 msgr2=0x7f9e1c107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:22.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 -- 172.21.15.69:0/2258996803 shutdown_connections 2024-03-19T01:10:22.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.014+0000 7f9e24600640 1 -- 172.21.15.69:0/2258996803 wait complete. 2024-03-19T01:10:22.034 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:10:22.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.246+0000 7f0967a00640 1 -- 172.21.15.69:0/39025018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09601033e0 msgr2=0x7f0960109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.246+0000 7f0967a00640 1 --2- 172.21.15.69:0/39025018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09601033e0 0x7f0960109c70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0958009ca0 tx=0x7f095802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.246+0000 7f0967a00640 1 -- 172.21.15.69:0/39025018 shutdown_connections 2024-03-19T01:10:22.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.246+0000 7f0967a00640 1 --2- 172.21.15.69:0/39025018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09601033e0 0x7f0960109c70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.246+0000 7f0967a00640 1 --2- 172.21.15.69:0/39025018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0960102a20 0x7f0960102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.246+0000 7f0967a00640 1 --2- 172.21.15.69:0/39025018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0960101830 0x7f0960101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.246+0000 7f0967a00640 1 -- 172.21.15.69:0/39025018 >> 172.21.15.69:0/39025018 conn(0x7f09600fcdc0 msgr2=0x7f09600ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:22.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.246+0000 7f0967a00640 1 -- 172.21.15.69:0/39025018 shutdown_connections 2024-03-19T01:10:22.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.246+0000 7f0967a00640 1 -- 172.21.15.69:0/39025018 wait complete. 2024-03-19T01:10:22.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.247+0000 7f0967a00640 1 Processor -- start 2024-03-19T01:10:22.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.248+0000 7f0967a00640 1 -- start start 2024-03-19T01:10:22.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.248+0000 7f0967a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0960101830 0x7f0960197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:22.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.248+0000 7f0967a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960102a20 0x7f0960198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:22.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.248+0000 7f0967a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09601033e0 0x7f096019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:22.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.248+0000 7f0967a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f096010d4b0 con 0x7f0960101830 2024-03-19T01:10:22.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.248+0000 7f0967a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f096010d330 con 0x7f0960102a20 2024-03-19T01:10:22.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.248+0000 7f0967a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f096010d630 con 0x7f09601033e0 2024-03-19T01:10:22.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.248+0000 7f095fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960102a20 0x7f0960198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:22.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.248+0000 7f0965000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0960101830 0x7f0960197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:22.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.248+0000 7f095fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960102a20 0x7f0960198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54652/0 (socket says 172.21.15.69:54652) 2024-03-19T01:10:22.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.248+0000 7f095fe00640 1 -- 172.21.15.69:0/2651959871 learned_addr learned my addr 172.21.15.69:0/2651959871 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:22.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.249+0000 7f0965a00640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09601033e0 0x7f096019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.249+0000 7f095fe00640 1 -- 172.21.15.69:0/2651959871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09601033e0 msgr2=0x7f096019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.249+0000 7f095fe00640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09601033e0 0x7f096019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.249+0000 7f095fe00640 1 -- 172.21.15.69:0/2651959871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0960101830 msgr2=0x7f0960197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.249+0000 7f095fe00640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0960101830 0x7f0960197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.249+0000 7f095fe00640 1 -- 172.21.15.69:0/2651959871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0960073a50 con 0x7f0960102a20 2024-03-19T01:10:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.249+0000 7f0965a00640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09601033e0 0x7f096019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.249+0000 7f095fe00640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960102a20 0x7f0960198030 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f094c00e9f0 tx=0x7f094c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.250+0000 7f095d600640 1 -- 172.21.15.69:0/2651959871 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c00cd80 con 0x7f0960102a20 2024-03-19T01:10:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.250+0000 7f095d600640 1 -- 172.21.15.69:0/2651959871 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f094c010910 con 0x7f0960102a20 2024-03-19T01:10:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.250+0000 7f0967a00640 1 -- 172.21.15.69:0/2651959871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0960073d40 con 0x7f0960102a20 2024-03-19T01:10:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.250+0000 7f095d600640 1 -- 172.21.15.69:0/2651959871 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c018990 con 0x7f0960102a20 2024-03-19T01:10:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.250+0000 7f0967a00640 1 -- 172.21.15.69:0/2651959871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0960074290 con 0x7f0960102a20 2024-03-19T01:10:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.251+0000 7f0967a00640 1 -- 172.21.15.69:0/2651959871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096002d1f0 con 0x7f0960102a20 2024-03-19T01:10:22.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.253+0000 7f095d600640 1 -- 172.21.15.69:0/2651959871 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f094c010ac0 con 0x7f0960102a20 2024-03-19T01:10:22.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:21 smithi086 ceph-mon[66829]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-19T01:10:22.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:21 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3188401419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:22.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:21 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2049148466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:22.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.255+0000 7f095d600640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f09480a2890 0x7f09480a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:22.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.255+0000 7f095d600640 1 -- 172.21.15.69:0/2651959871 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f094c0cf350 con 0x7f0960102a20 2024-03-19T01:10:22.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.255+0000 7f0965000640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f09480a2890 0x7f09480a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:22.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.256+0000 7f0965000640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f09480a2890 0x7f09480a4d50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0960102880 tx=0x7f0954005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:22.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.258+0000 7f095d600640 1 -- 172.21.15.69:0/2651959871 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f094c09b830 con 0x7f0960102a20 2024-03-19T01:10:22.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:21 smithi163 ceph-mon[67428]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-19T01:10:22.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:21 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3188401419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:22.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:21 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2049148466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:22.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.497+0000 7f0967a00640 1 -- 172.21.15.69:0/2651959871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0960108790 con 0x7f0960102a20 2024-03-19T01:10:22.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.498+0000 7f095d600640 1 -- 172.21.15.69:0/2651959871 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f094c09f8d0 con 0x7f0960102a20 2024-03-19T01:10:22.499 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-03-19T01:10:22.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 -- 172.21.15.69:0/2651959871 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f09480a2890 msgr2=0x7f09480a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f09480a2890 0x7f09480a4d50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0960102880 tx=0x7f0954005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:22.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 -- 172.21.15.69:0/2651959871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960102a20 msgr2=0x7f0960198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960102a20 0x7f0960198030 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f094c00e9f0 tx=0x7f094c00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 -- 172.21.15.69:0/2651959871 shutdown_connections 2024-03-19T01:10:22.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f09480a2890 0x7f09480a4d50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09601033e0 0x7f096019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0960102a20 0x7f0960198030 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 --2- 172.21.15.69:0/2651959871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0960101830 0x7f0960197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 -- 172.21.15.69:0/2651959871 >> 172.21.15.69:0/2651959871 conn(0x7f09600fcdc0 msgr2=0x7f0960105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:22.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 -- 172.21.15.69:0/2651959871 shutdown_connections 2024-03-19T01:10:22.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.501+0000 7f0967a00640 1 -- 172.21.15.69:0/2651959871 wait complete. 2024-03-19T01:10:22.530 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T01:10:22.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.754+0000 7f7b6fe00640 1 -- 172.21.15.69:0/2976773880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b70102200 msgr2=0x7f7b701045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.754+0000 7f7b6fe00640 1 --2- 172.21.15.69:0/2976773880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b70102200 0x7f7b701045f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009900 tx=0x7f7b6002bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:22.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.754+0000 7f7b6fe00640 1 -- 172.21.15.69:0/2976773880 shutdown_connections 2024-03-19T01:10:22.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.754+0000 7f7b6fe00640 1 --2- 172.21.15.69:0/2976773880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b70107460 0x7f7b70109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.754+0000 7f7b6fe00640 1 --2- 172.21.15.69:0/2976773880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b70104b30 0x7f7b70106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.754+0000 7f7b6fe00640 1 --2- 172.21.15.69:0/2976773880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b70102200 0x7f7b701045f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.754+0000 7f7b6fe00640 1 -- 172.21.15.69:0/2976773880 >> 172.21.15.69:0/2976773880 conn(0x7f7b700fbb90 msgr2=0x7f7b700fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:22.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.754+0000 7f7b6fe00640 1 -- 172.21.15.69:0/2976773880 shutdown_connections 2024-03-19T01:10:22.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.754+0000 7f7b6fe00640 1 -- 172.21.15.69:0/2976773880 wait complete. 2024-03-19T01:10:22.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.755+0000 7f7b6fe00640 1 Processor -- start 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.755+0000 7f7b6fe00640 1 -- start start 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.755+0000 7f7b6fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b70102200 0x7f7b70197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.755+0000 7f7b6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b70104b30 0x7f7b70197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.755+0000 7f7b6fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b70107460 0x7f7b7019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.755+0000 7f7b6fe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7010d3c0 con 0x7f7b70102200 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.755+0000 7f7b6fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7010d240 con 0x7f7b70104b30 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.755+0000 7f7b6fe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7010d540 con 0x7f7b70107460 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b70102200 0x7f7b70197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b70102200 0x7f7b70197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39074/0 (socket says 172.21.15.69:39074) 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6ea00640 1 -- 172.21.15.69:0/403616412 learned_addr learned my addr 172.21.15.69:0/403616412 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6e000640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b70104b30 0x7f7b70197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:22.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6f400640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b70107460 0x7f7b7019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6ea00640 1 -- 172.21.15.69:0/403616412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b70107460 msgr2=0x7f7b7019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6ea00640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b70107460 0x7f7b7019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6ea00640 1 -- 172.21.15.69:0/403616412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b70104b30 msgr2=0x7f7b70197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6ea00640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b70104b30 0x7f7b70197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6ea00640 1 -- 172.21.15.69:0/403616412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7019f690 con 0x7f7b70102200 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6f400640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b70107460 0x7f7b7019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6e000640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b70104b30 0x7f7b70197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.756+0000 7f7b6ea00640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b70102200 0x7f7b70197a70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7b60005ab0 tx=0x7f7b6002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.757+0000 7f7b67400640 1 -- 172.21.15.69:0/403616412 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60043020 con 0x7f7b70102200 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.757+0000 7f7b6fe00640 1 -- 172.21.15.69:0/403616412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7019f920 con 0x7f7b70102200 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.757+0000 7f7b6fe00640 1 -- 172.21.15.69:0/403616412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7019fec0 con 0x7f7b70102200 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.757+0000 7f7b67400640 1 -- 172.21.15.69:0/403616412 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b60004c80 con 0x7f7b70102200 2024-03-19T01:10:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.757+0000 7f7b67400640 1 -- 172.21.15.69:0/403616412 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60003850 con 0x7f7b70102200 2024-03-19T01:10:22.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.758+0000 7f7b6fe00640 1 -- 172.21.15.69:0/403616412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b500052c0 con 0x7f7b70102200 2024-03-19T01:10:22.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.762+0000 7f7b67400640 1 -- 172.21.15.69:0/403616412 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f7b60046020 con 0x7f7b70102200 2024-03-19T01:10:22.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.764+0000 7f7b67400640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7b540a2890 0x7f7b540a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:22.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.764+0000 7f7b67400640 1 -- 172.21.15.69:0/403616412 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f7b600f12e0 con 0x7f7b70102200 2024-03-19T01:10:22.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.764+0000 7f7b6e000640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7b540a2890 0x7f7b540a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:22.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.764+0000 7f7b67400640 1 -- 172.21.15.69:0/403616412 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7b6004d020 con 0x7f7b70102200 2024-03-19T01:10:22.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.765+0000 7f7b6e000640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7b540a2890 0x7f7b540a4d50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7b70198f80 tx=0x7f7b5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.967+0000 7f7b6fe00640 1 -- 172.21.15.69:0/403616412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7b50005880 con 0x7f7b70102200 2024-03-19T01:10:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.968+0000 7f7b67400640 1 -- 172.21.15.69:0/403616412 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f7b600bdf20 con 0x7f7b70102200 2024-03-19T01:10:22.969 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:10:22.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.970+0000 7f7b6fe00640 1 -- 172.21.15.69:0/403616412 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7b540a2890 msgr2=0x7f7b540a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.970+0000 7f7b6fe00640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7b540a2890 0x7f7b540a4d50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7b70198f80 tx=0x7f7b5c005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:22.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.970+0000 7f7b6fe00640 1 -- 172.21.15.69:0/403616412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b70102200 msgr2=0x7f7b70197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:22.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.970+0000 7f7b6fe00640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b70102200 0x7f7b70197a70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7b60005ab0 tx=0x7f7b6002c6c0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.970+0000 7f7b6fe00640 1 -- 172.21.15.69:0/403616412 shutdown_connections 2024-03-19T01:10:22.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.970+0000 7f7b6fe00640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f7b540a2890 0x7f7b540a4d50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.970+0000 7f7b6fe00640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b70107460 0x7f7b7019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.970+0000 7f7b6fe00640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b70104b30 0x7f7b70197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.970+0000 7f7b6fe00640 1 --2- 172.21.15.69:0/403616412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b70102200 0x7f7b70197a70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:22.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.970+0000 7f7b6fe00640 1 -- 172.21.15.69:0/403616412 >> 172.21.15.69:0/403616412 conn(0x7f7b700fbb90 msgr2=0x7f7b70102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:22.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.971+0000 7f7b6fe00640 1 -- 172.21.15.69:0/403616412 shutdown_connections 2024-03-19T01:10:22.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:22.971+0000 7f7b6fe00640 1 -- 172.21.15.69:0/403616412 wait complete. 2024-03-19T01:10:22.995 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-19T01:10:22.995 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:10:23.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.196+0000 7f58cfe00640 1 -- 172.21.15.69:0/1029001741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58d0101240 msgr2=0x7f58d01016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:23.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.196+0000 7f58cfe00640 1 --2- 172.21.15.69:0/1029001741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58d0101240 0x7f58d01016c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009ca0 tx=0x7f58c802c100 comp rx=0 tx=0).stop 2024-03-19T01:10:23.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.197+0000 7f58cfe00640 1 -- 172.21.15.69:0/1029001741 shutdown_connections 2024-03-19T01:10:23.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.197+0000 7f58cfe00640 1 --2- 172.21.15.69:0/1029001741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58d0101240 0x7f58d01016c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.197+0000 7f58cfe00640 1 --2- 172.21.15.69:0/1029001741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58d01008a0 0x7f58d0100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.197+0000 7f58cfe00640 1 --2- 172.21.15.69:0/1029001741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d00fff40 0x7f58d0100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.197+0000 7f58cfe00640 1 -- 172.21.15.69:0/1029001741 >> 172.21.15.69:0/1029001741 conn(0x7f58d00fbb00 msgr2=0x7f58d00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:23.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.197+0000 7f58cfe00640 1 -- 172.21.15.69:0/1029001741 shutdown_connections 2024-03-19T01:10:23.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.197+0000 7f58cfe00640 1 -- 172.21.15.69:0/1029001741 wait complete. 2024-03-19T01:10:23.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.198+0000 7f58cfe00640 1 Processor -- start 2024-03-19T01:10:23.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.198+0000 7f58cfe00640 1 -- start start 2024-03-19T01:10:23.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58d00fff40 0x7f58d0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:23.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d01008a0 0x7f58d019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:23.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58d0101240 0x7f58d01a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:23.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cfe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010f4e0 con 0x7f58d01008a0 2024-03-19T01:10:23.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010f360 con 0x7f58d00fff40 2024-03-19T01:10:23.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cfe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010f660 con 0x7f58d0101240 2024-03-19T01:10:23.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58d00fff40 0x7f58d0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:23.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cf400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58d0101240 0x7f58d01a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:23.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58d00fff40 0x7f58d0199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54676/0 (socket says 172.21.15.69:54676) 2024-03-19T01:10:23.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cea00640 1 -- 172.21.15.69:0/1151791027 learned_addr learned my addr 172.21.15.69:0/1151791027 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58c7e00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d01008a0 0x7f58d019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cea00640 1 -- 172.21.15.69:0/1151791027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58d0101240 msgr2=0x7f58d01a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cea00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58d0101240 0x7f58d01a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cea00640 1 -- 172.21.15.69:0/1151791027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d01008a0 msgr2=0x7f58d019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cea00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d01008a0 0x7f58d019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.199+0000 7f58cea00640 1 -- 172.21.15.69:0/1151791027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d01a1750 con 0x7f58d00fff40 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.200+0000 7f58cf400640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58d0101240 0x7f58d01a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.200+0000 7f58cea00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58d00fff40 0x7f58d0199b30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f58c000d950 tx=0x7f58c000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.200+0000 7f58c7400640 1 -- 172.21.15.69:0/1151791027 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0017070 con 0x7f58d00fff40 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.200+0000 7f58c7400640 1 -- 172.21.15.69:0/1151791027 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c0009c30 con 0x7f58d00fff40 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.200+0000 7f58c7400640 1 -- 172.21.15.69:0/1151791027 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c000f6b0 con 0x7f58d00fff40 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.200+0000 7f58cfe00640 1 -- 172.21.15.69:0/1151791027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d01a1a40 con 0x7f58d00fff40 2024-03-19T01:10:23.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.200+0000 7f58cfe00640 1 -- 172.21.15.69:0/1151791027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d01a1f90 con 0x7f58d00fff40 2024-03-19T01:10:23.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.201+0000 7f58cfe00640 1 -- 172.21.15.69:0/1151791027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d010ac90 con 0x7f58d00fff40 2024-03-19T01:10:23.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.203+0000 7f58c7400640 1 -- 172.21.15.69:0/1151791027 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f58c00109d0 con 0x7f58d00fff40 2024-03-19T01:10:23.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.204+0000 7f58c7400640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f58b40a2890 0x7f58b40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:23.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.204+0000 7f58c7400640 1 -- 172.21.15.69:0/1151791027 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f58c00cec00 con 0x7f58d00fff40 2024-03-19T01:10:23.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.205+0000 7f58c7e00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f58b40a2890 0x7f58b40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:23.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.205+0000 7f58c7e00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f58b40a2890 0x7f58b40a4d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f58d019b040 tx=0x7f58b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:23.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.213+0000 7f58c7400640 1 -- 172.21.15.69:0/1151791027 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f58c0009de0 con 0x7f58d00fff40 2024-03-19T01:10:23.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:22 smithi086 ceph-mon[66829]: pgmap v412: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-19T01:10:23.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:22 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2258996803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:23.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:22 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2651959871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:23.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:22 smithi069 ceph-mon[31727]: pgmap v412: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-19T01:10:23.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:22 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2258996803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:23.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:22 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2651959871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:23.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:22 smithi163 ceph-mon[67428]: pgmap v412: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-19T01:10:23.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:22 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2258996803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:23.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:22 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2651959871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:23.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.458+0000 7f58cfe00640 1 -- 172.21.15.69:0/1151791027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58d002d1f0 con 0x7f58d00fff40 2024-03-19T01:10:23.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.459+0000 7f58c7400640 1 -- 172.21.15.69:0/1151791027 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11930 (secure 0 0 0) 0x7f58c009b0e0 con 0x7f58d00fff40 2024-03-19T01:10:23.460 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:23.460 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:10:17.373602+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24550,24494,24580,24581,24476]},"id":1}]} 2024-03-19T01:10:23.460 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 29 2024-03-19T01:10:23.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 -- 172.21.15.69:0/1151791027 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f58b40a2890 msgr2=0x7f58b40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:23.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f58b40a2890 0x7f58b40a4d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f58d019b040 tx=0x7f58b8005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:23.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 -- 172.21.15.69:0/1151791027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58d00fff40 msgr2=0x7f58d0199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:23.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58d00fff40 0x7f58d0199b30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f58c000d950 tx=0x7f58c000de10 comp rx=0 tx=0).stop 2024-03-19T01:10:23.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 -- 172.21.15.69:0/1151791027 shutdown_connections 2024-03-19T01:10:23.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f58b40a2890 0x7f58b40a4d50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58d0101240 0x7f58d01a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d01008a0 0x7f58d019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 --2- 172.21.15.69:0/1151791027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58d00fff40 0x7f58d0199b30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 -- 172.21.15.69:0/1151791027 >> 172.21.15.69:0/1151791027 conn(0x7f58d00fbb00 msgr2=0x7f58d00fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:23.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 -- 172.21.15.69:0/1151791027 shutdown_connections 2024-03-19T01:10:23.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.462+0000 7f58cfe00640 1 -- 172.21.15.69:0/1151791027 wait complete. 2024-03-19T01:10:23.481 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-19T01:10:23.481 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-19T01:10:23.481 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T01:10:23.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.699+0000 7f0a51e00640 1 -- 172.21.15.69:0/3344991803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4c074220 msgr2=0x7f0a4c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:23.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.699+0000 7f0a51e00640 1 --2- 172.21.15.69:0/3344991803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4c074220 0x7f0a4c074680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009900 tx=0x7f0a3802bcd0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.699+0000 7f0a51e00640 1 -- 172.21.15.69:0/3344991803 shutdown_connections 2024-03-19T01:10:23.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.699+0000 7f0a51e00640 1 --2- 172.21.15.69:0/3344991803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4c109520 0x7f0a4c10b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.699+0000 7f0a51e00640 1 --2- 172.21.15.69:0/3344991803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4c074220 0x7f0a4c074680 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.699+0000 7f0a51e00640 1 --2- 172.21.15.69:0/3344991803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c075800 0x7f0a4c073c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.699+0000 7f0a51e00640 1 -- 172.21.15.69:0/3344991803 >> 172.21.15.69:0/3344991803 conn(0x7f0a4c0fcc00 msgr2=0x7f0a4c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:23.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.699+0000 7f0a51e00640 1 -- 172.21.15.69:0/3344991803 shutdown_connections 2024-03-19T01:10:23.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.699+0000 7f0a51e00640 1 -- 172.21.15.69:0/3344991803 wait complete. 2024-03-19T01:10:23.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.700+0000 7f0a51e00640 1 Processor -- start 2024-03-19T01:10:23.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a51e00640 1 -- start start 2024-03-19T01:10:23.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a51e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c074220 0x7f0a4c199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:23.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a51e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4c075800 0x7f0a4c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:23.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4c109520 0x7f0a4c1a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:23.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a51e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c10f560 con 0x7f0a4c075800 2024-03-19T01:10:23.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a51e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c10f3e0 con 0x7f0a4c109520 2024-03-19T01:10:23.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a51e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c10f6e0 con 0x7f0a4c074220 2024-03-19T01:10:23.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a4be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4c075800 0x7f0a4c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:23.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a51400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4c109520 0x7f0a4c1a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:23.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a50a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c074220 0x7f0a4c199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:23.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a51400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4c109520 0x7f0a4c1a1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54688/0 (socket says 172.21.15.69:54688) 2024-03-19T01:10:23.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a51400640 1 -- 172.21.15.69:0/3723525278 learned_addr learned my addr 172.21.15.69:0/3723525278 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:23.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.701+0000 7f0a50a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c074220 0x7f0a4c199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36268/0 (socket says 172.21.15.69:36268) 2024-03-19T01:10:23.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.702+0000 7f0a51400640 1 -- 172.21.15.69:0/3723525278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c074220 msgr2=0x7f0a4c199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:23.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.702+0000 7f0a51400640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c074220 0x7f0a4c199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.702+0000 7f0a51400640 1 -- 172.21.15.69:0/3723525278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4c075800 msgr2=0x7f0a4c19a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:23.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.702+0000 7f0a51400640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4c075800 0x7f0a4c19a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.702+0000 7f0a51400640 1 -- 172.21.15.69:0/3723525278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c1a17b0 con 0x7f0a4c109520 2024-03-19T01:10:23.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.702+0000 7f0a51400640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4c109520 0x7f0a4c1a1140 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400eb10 tx=0x7f0a4400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:23.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.702+0000 7f0a49600640 1 -- 172.21.15.69:0/3723525278 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44004b20 con 0x7f0a4c109520 2024-03-19T01:10:23.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.703+0000 7f0a49600640 1 -- 172.21.15.69:0/3723525278 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a44004cc0 con 0x7f0a4c109520 2024-03-19T01:10:23.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.703+0000 7f0a51e00640 1 -- 172.21.15.69:0/3723525278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c1a1aa0 con 0x7f0a4c109520 2024-03-19T01:10:23.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.703+0000 7f0a49600640 1 -- 172.21.15.69:0/3723525278 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44010760 con 0x7f0a4c109520 2024-03-19T01:10:23.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.703+0000 7f0a51e00640 1 -- 172.21.15.69:0/3723525278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c1a1fc0 con 0x7f0a4c109520 2024-03-19T01:10:23.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.704+0000 7f0a51e00640 1 -- 172.21.15.69:0/3723525278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a4c1082e0 con 0x7f0a4c109520 2024-03-19T01:10:23.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.705+0000 7f0a49600640 1 -- 172.21.15.69:0/3723525278 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f0a44005020 con 0x7f0a4c109520 2024-03-19T01:10:23.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.707+0000 7f0a49600640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0a340a2910 0x7f0a340a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:23.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.707+0000 7f0a49600640 1 -- 172.21.15.69:0/3723525278 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f0a440cf4e0 con 0x7f0a4c109520 2024-03-19T01:10:23.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.707+0000 7f0a50a00640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0a340a2910 0x7f0a340a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:23.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.708+0000 7f0a50a00640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0a340a2910 0x7f0a340a4dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009bd0 tx=0x7f0a40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:23.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.715+0000 7f0a49600640 1 -- 172.21.15.69:0/3723525278 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0a4409c010 con 0x7f0a4c109520 2024-03-19T01:10:23.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.934+0000 7f0a51e00640 1 -- 172.21.15.69:0/3723525278 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a4c061c60 con 0x7f0a340a2910 2024-03-19T01:10:23.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.951+0000 7f0a49600640 1 -- 172.21.15.69:0/3723525278 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340835 (secure 0 0 0) 0x7f0a4c061c60 con 0x7f0a340a2910 2024-03-19T01:10:23.957 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:23.959 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-03-19T01:10:23.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.958+0000 7f0a51e00640 1 -- 172.21.15.69:0/3723525278 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0a340a2910 msgr2=0x7f0a340a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:23.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.958+0000 7f0a51e00640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0a340a2910 0x7f0a340a4dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009bd0 tx=0x7f0a40005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:23.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.958+0000 7f0a51e00640 1 -- 172.21.15.69:0/3723525278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4c109520 msgr2=0x7f0a4c1a1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:23.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.958+0000 7f0a51e00640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4c109520 0x7f0a4c1a1140 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400eb10 tx=0x7f0a4400c2d0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.958+0000 7f0a51e00640 1 -- 172.21.15.69:0/3723525278 shutdown_connections 2024-03-19T01:10:23.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.958+0000 7f0a51e00640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f0a340a2910 0x7f0a340a4dd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.958+0000 7f0a51e00640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4c109520 0x7f0a4c1a1140 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.958+0000 7f0a51e00640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4c075800 0x7f0a4c19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.958+0000 7f0a51e00640 1 --2- 172.21.15.69:0/3723525278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c074220 0x7f0a4c199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:23.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.958+0000 7f0a51e00640 1 -- 172.21.15.69:0/3723525278 >> 172.21.15.69:0/3723525278 conn(0x7f0a4c0fcc00 msgr2=0x7f0a4c1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:23.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.959+0000 7f0a51e00640 1 -- 172.21.15.69:0/3723525278 shutdown_connections 2024-03-19T01:10:23.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:23.959+0000 7f0a51e00640 1 -- 172.21.15.69:0/3723525278 wait complete. 2024-03-19T01:10:23.979 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-03-19T01:10:23.708079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669049,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1711,"num_read_kb":994,"num_write":447,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378420,"kb_used_data":12608,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563404,"statfs":{"total":1151940427776,"available":1151552925696,"internally_reserved":0,"allocated":12910592,"data_stored":8219381,"data_compressed":430971,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8619,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":150,"num_read_kb":87,"num_write":15,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004408"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804627+0000","last_change":"2024-03-19T01:07:15.500098+0000","last_active":"2024-03-19T01:09:25.804627+0000","last_peered":"2024-03-19T01:09:25.804627+0000","last_clean":"2024-03-19T01:09:25.804627+0000","last_became_active":"2024-03-19T01:07:15.499738+0000","last_became_peered":"2024-03-19T01:07:15.499738+0000","last_unstale":"2024-03-19T01:09:25.804627+0000","last_undegraded":"2024-03-19T01:09:25.804627+0000","last_fullsized":"2024-03-19T01:09:25.804627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:14.703153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156585+0000","last_change":"2024-03-19T01:07:15.492863+0000","last_active":"2024-03-19T01:09:26.156585+0000","last_peered":"2024-03-19T01:09:26.156585+0000","last_clean":"2024-03-19T01:09:26.156585+0000","last_became_active":"2024-03-19T01:07:15.492533+0000","last_became_peered":"2024-03-19T01:07:15.492533+0000","last_unstale":"2024-03-19T01:09:26.156585+0000","last_undegraded":"2024-03-19T01:09:26.156585+0000","last_fullsized":"2024-03-19T01:09:26.156585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:49.028352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156213+0000","last_change":"2024-03-19T01:07:15.494174+0000","last_active":"2024-03-19T01:09:26.156213+0000","last_peered":"2024-03-19T01:09:26.156213+0000","last_clean":"2024-03-19T01:09:26.156213+0000","last_became_active":"2024-03-19T01:07:15.490562+0000","last_became_peered":"2024-03-19T01:07:15.490562+0000","last_unstale":"2024-03-19T01:09:26.156213+0000","last_undegraded":"2024-03-19T01:09:26.156213+0000","last_fullsized":"2024-03-19T01:09:26.156213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:18.664396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800428+0000","last_change":"2024-03-19T01:07:15.478397+0000","last_active":"2024-03-19T01:09:25.800428+0000","last_peered":"2024-03-19T01:09:25.800428+0000","last_clean":"2024-03-19T01:09:25.800428+0000","last_became_active":"2024-03-19T01:07:15.477000+0000","last_became_peered":"2024-03-19T01:07:15.477000+0000","last_unstale":"2024-03-19T01:09:25.800428+0000","last_undegraded":"2024-03-19T01:09:25.800428+0000","last_fullsized":"2024-03-19T01:09:25.800428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:58.534329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137621+0000","last_change":"2024-03-19T01:07:15.500138+0000","last_active":"2024-03-19T01:09:26.137621+0000","last_peered":"2024-03-19T01:09:26.137621+0000","last_clean":"2024-03-19T01:09:26.137621+0000","last_became_active":"2024-03-19T01:07:15.499759+0000","last_became_peered":"2024-03-19T01:07:15.499759+0000","last_unstale":"2024-03-19T01:09:26.137621+0000","last_undegraded":"2024-03-19T01:09:26.137621+0000","last_fullsized":"2024-03-19T01:09:26.137621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:23.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804946+0000","last_change":"2024-03-19T01:09:25.804601+0000","last_active":"2024-03-19T01:09:25.804946+0000","last_peered":"2024-03-19T01:09:25.804946+0000","last_clean":"2024-03-19T01:09:25.804946+0000","last_became_active":"2024-03-19T01:09:25.804332+0000","last_became_peered":"2024-03-19T01:09:25.804332+0000","last_unstale":"2024-03-19T01:09:25.804946+0000","last_undegraded":"2024-03-19T01:09:25.804946+0000","last_fullsized":"2024-03-19T01:09:25.804946+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:44:10.372309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962138+0000","last_change":"2024-03-19T01:07:15.486327+0000","last_active":"2024-03-19T01:09:25.962138+0000","last_peered":"2024-03-19T01:09:25.962138+0000","last_clean":"2024-03-19T01:09:25.962138+0000","last_became_active":"2024-03-19T01:07:15.485954+0000","last_became_peered":"2024-03-19T01:07:15.485954+0000","last_unstale":"2024-03-19T01:09:25.962138+0000","last_undegraded":"2024-03-19T01:09:25.962138+0000","last_fullsized":"2024-03-19T01:09:25.962138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:53.316260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914327+0000","last_change":"2024-03-19T01:07:15.504993+0000","last_active":"2024-03-19T01:09:25.914327+0000","last_peered":"2024-03-19T01:09:25.914327+0000","last_clean":"2024-03-19T01:09:25.914327+0000","last_became_active":"2024-03-19T01:07:15.502771+0000","last_became_peered":"2024-03-19T01:07:15.502771+0000","last_unstale":"2024-03-19T01:09:25.914327+0000","last_undegraded":"2024-03-19T01:09:25.914327+0000","last_fullsized":"2024-03-19T01:09:25.914327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:22:33.712608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.806615+0000","last_change":"2024-03-19T01:09:25.805915+0000","last_active":"2024-03-19T01:09:25.806615+0000","last_peered":"2024-03-19T01:09:25.806615+0000","last_clean":"2024-03-19T01:09:25.806615+0000","last_became_active":"2024-03-19T01:09:25.805344+0000","last_became_peered":"2024-03-19T01:09:25.805344+0000","last_unstale":"2024-03-19T01:09:25.806615+0000","last_undegraded":"2024-03-19T01:09:25.806615+0000","last_fullsized":"2024-03-19T01:09:25.806615+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:11:55.795773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801964+0000","last_change":"2024-03-19T01:07:15.475655+0000","last_active":"2024-03-19T01:09:25.801964+0000","last_peered":"2024-03-19T01:09:25.801964+0000","last_clean":"2024-03-19T01:09:25.801964+0000","last_became_active":"2024-03-19T01:07:15.475265+0000","last_became_peered":"2024-03-19T01:07:15.475265+0000","last_unstale":"2024-03-19T01:09:25.801964+0000","last_undegraded":"2024-03-19T01:09:25.801964+0000","last_fullsized":"2024-03-19T01:09:25.801964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:24:38.898191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962612+0000","last_change":"2024-03-19T01:07:15.482988+0000","last_active":"2024-03-19T01:09:25.962612+0000","last_peered":"2024-03-19T01:09:25.962612+0000","last_clean":"2024-03-19T01:09:25.962612+0000","last_became_active":"2024-03-19T01:07:15.482673+0000","last_became_peered":"2024-03-19T01:07:15.482673+0000","last_unstale":"2024-03-19T01:09:25.962612+0000","last_undegraded":"2024-03-19T01:09:25.962612+0000","last_fullsized":"2024-03-19T01:09:25.962612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:15:48.807200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800368+0000","last_change":"2024-03-19T01:07:15.479593+0000","last_active":"2024-03-19T01:09:25.800368+0000","last_peered":"2024-03-19T01:09:25.800368+0000","last_clean":"2024-03-19T01:09:25.800368+0000","last_became_active":"2024-03-19T01:07:15.479220+0000","last_became_peered":"2024-03-19T01:07:15.479220+0000","last_unstale":"2024-03-19T01:09:25.800368+0000","last_undegraded":"2024-03-19T01:09:25.800368+0000","last_fullsized":"2024-03-19T01:09:25.800368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:40:24.390845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804248+0000","last_change":"2024-03-19T01:07:15.464737+0000","last_active":"2024-03-19T01:09:25.804248+0000","last_peered":"2024-03-19T01:09:25.804248+0000","last_clean":"2024-03-19T01:09:25.804248+0000","last_became_active":"2024-03-19T01:07:15.463641+0000","last_became_peered":"2024-03-19T01:07:15.463641+0000","last_unstale":"2024-03-19T01:09:25.804248+0000","last_undegraded":"2024-03-19T01:09:25.804248+0000","last_fullsized":"2024-03-19T01:09:25.804248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:13.964397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.155743+0000","last_change":"2024-03-19T01:07:15.492803+0000","last_active":"2024-03-19T01:09:26.155743+0000","last_peered":"2024-03-19T01:09:26.155743+0000","last_clean":"2024-03-19T01:09:26.155743+0000","last_became_active":"2024-03-19T01:07:15.492422+0000","last_became_peered":"2024-03-19T01:07:15.492422+0000","last_unstale":"2024-03-19T01:09:26.155743+0000","last_undegraded":"2024-03-19T01:09:26.155743+0000","last_fullsized":"2024-03-19T01:09:26.155743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:59:08.292399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.155747+0000","last_change":"2024-03-19T01:07:15.498073+0000","last_active":"2024-03-19T01:09:26.155747+0000","last_peered":"2024-03-19T01:09:26.155747+0000","last_clean":"2024-03-19T01:09:26.155747+0000","last_became_active":"2024-03-19T01:07:15.493650+0000","last_became_peered":"2024-03-19T01:07:15.493650+0000","last_unstale":"2024-03-19T01:09:26.155747+0000","last_undegraded":"2024-03-19T01:09:26.155747+0000","last_fullsized":"2024-03-19T01:09:26.155747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:25:28.908122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154534+0000","last_change":"2024-03-19T01:07:15.496434+0000","last_active":"2024-03-19T01:09:26.154534+0000","last_peered":"2024-03-19T01:09:26.154534+0000","last_clean":"2024-03-19T01:09:26.154534+0000","last_became_active":"2024-03-19T01:07:15.495956+0000","last_became_peered":"2024-03-19T01:07:15.495956+0000","last_unstale":"2024-03-19T01:09:26.154534+0000","last_undegraded":"2024-03-19T01:09:26.154534+0000","last_fullsized":"2024-03-19T01:09:26.154534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:49.505947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804589+0000","last_change":"2024-03-19T01:07:15.504771+0000","last_active":"2024-03-19T01:09:25.804589+0000","last_peered":"2024-03-19T01:09:25.804589+0000","last_clean":"2024-03-19T01:09:25.804589+0000","last_became_active":"2024-03-19T01:07:15.504485+0000","last_became_peered":"2024-03-19T01:07:15.504485+0000","last_unstale":"2024-03-19T01:09:25.804589+0000","last_undegraded":"2024-03-19T01:09:25.804589+0000","last_fullsized":"2024-03-19T01:09:25.804589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:17:36.760943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.802629+0000","last_change":"2024-03-19T01:07:15.491045+0000","last_active":"2024-03-19T01:09:25.802629+0000","last_peered":"2024-03-19T01:09:25.802629+0000","last_clean":"2024-03-19T01:09:25.802629+0000","last_became_active":"2024-03-19T01:07:15.488914+0000","last_became_peered":"2024-03-19T01:07:15.488914+0000","last_unstale":"2024-03-19T01:09:25.802629+0000","last_undegraded":"2024-03-19T01:09:25.802629+0000","last_fullsized":"2024-03-19T01:09:25.802629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:14.853457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914298+0000","last_change":"2024-03-19T01:07:15.495306+0000","last_active":"2024-03-19T01:09:25.914298+0000","last_peered":"2024-03-19T01:09:25.914298+0000","last_clean":"2024-03-19T01:09:25.914298+0000","last_became_active":"2024-03-19T01:07:15.494906+0000","last_became_peered":"2024-03-19T01:07:15.494906+0000","last_unstale":"2024-03-19T01:09:25.914298+0000","last_undegraded":"2024-03-19T01:09:25.914298+0000","last_fullsized":"2024-03-19T01:09:25.914298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:48.176410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.913990+0000","last_change":"2024-03-19T01:07:19.689402+0000","last_active":"2024-03-19T01:09:25.913990+0000","last_peered":"2024-03-19T01:09:25.913990+0000","last_clean":"2024-03-19T01:09:25.913990+0000","last_became_active":"2024-03-19T01:07:19.688799+0000","last_became_peered":"2024-03-19T01:07:19.688799+0000","last_unstale":"2024-03-19T01:09:25.913990+0000","last_undegraded":"2024-03-19T01:09:25.913990+0000","last_fullsized":"2024-03-19T01:09:25.913990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:51.890047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":199,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.039278+0000","last_change":"2024-03-19T01:07:14.477348+0000","last_active":"2024-03-19T01:10:21.039278+0000","last_peered":"2024-03-19T01:10:21.039278+0000","last_clean":"2024-03-19T01:10:21.039278+0000","last_became_active":"2024-03-19T01:07:14.476946+0000","last_became_peered":"2024-03-19T01:07:14.476946+0000","last_unstale":"2024-03-19T01:10:21.039278+0000","last_undegraded":"2024-03-19T01:10:21.039278+0000","last_fullsized":"2024-03-19T01:10:21.039278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:19.269837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915607+0000","last_change":"2024-03-19T01:07:15.495646+0000","last_active":"2024-03-19T01:09:25.915607+0000","last_peered":"2024-03-19T01:09:25.915607+0000","last_clean":"2024-03-19T01:09:25.915607+0000","last_became_active":"2024-03-19T01:07:15.495203+0000","last_became_peered":"2024-03-19T01:07:15.495203+0000","last_unstale":"2024-03-19T01:09:25.915607+0000","last_undegraded":"2024-03-19T01:09:25.915607+0000","last_fullsized":"2024-03-19T01:09:25.915607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:01:31.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.799843+0000","last_change":"2024-03-19T01:07:19.690465+0000","last_active":"2024-03-19T01:09:25.799843+0000","last_peered":"2024-03-19T01:09:25.799843+0000","last_clean":"2024-03-19T01:09:25.799843+0000","last_became_active":"2024-03-19T01:07:19.689852+0000","last_became_peered":"2024-03-19T01:07:19.689852+0000","last_unstale":"2024-03-19T01:09:25.799843+0000","last_undegraded":"2024-03-19T01:09:25.799843+0000","last_fullsized":"2024-03-19T01:09:25.799843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:18.891411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915740+0000","last_change":"2024-03-19T01:07:14.487165+0000","last_active":"2024-03-19T01:09:25.915740+0000","last_peered":"2024-03-19T01:09:25.915740+0000","last_clean":"2024-03-19T01:09:25.915740+0000","last_became_active":"2024-03-19T01:07:14.484194+0000","last_became_peered":"2024-03-19T01:07:14.484194+0000","last_unstale":"2024-03-19T01:09:25.915740+0000","last_undegraded":"2024-03-19T01:09:25.915740+0000","last_fullsized":"2024-03-19T01:09:25.915740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:54:47.914565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156861+0000","last_change":"2024-03-19T01:07:15.498687+0000","last_active":"2024-03-19T01:09:26.156861+0000","last_peered":"2024-03-19T01:09:26.156861+0000","last_clean":"2024-03-19T01:09:26.156861+0000","last_became_active":"2024-03-19T01:07:15.498146+0000","last_became_peered":"2024-03-19T01:07:15.498146+0000","last_unstale":"2024-03-19T01:09:26.156861+0000","last_undegraded":"2024-03-19T01:09:26.156861+0000","last_fullsized":"2024-03-19T01:09:26.156861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:47:21.846333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800041+0000","last_change":"2024-03-19T01:07:19.690691+0000","last_active":"2024-03-19T01:09:25.800041+0000","last_peered":"2024-03-19T01:09:25.800041+0000","last_clean":"2024-03-19T01:09:25.800041+0000","last_became_active":"2024-03-19T01:07:19.690083+0000","last_became_peered":"2024-03-19T01:07:19.690083+0000","last_unstale":"2024-03-19T01:09:25.800041+0000","last_undegraded":"2024-03-19T01:09:25.800041+0000","last_fullsized":"2024-03-19T01:09:25.800041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:51:30.205694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804545+0000","last_change":"2024-03-19T01:08:24.801510+0000","last_active":"2024-03-19T01:09:25.804545+0000","last_peered":"2024-03-19T01:09:25.804545+0000","last_clean":"2024-03-19T01:09:25.804545+0000","last_became_active":"2024-03-19T01:08:24.789315+0000","last_became_peered":"2024-03-19T01:08:24.789315+0000","last_unstale":"2024-03-19T01:09:25.804545+0000","last_undegraded":"2024-03-19T01:09:25.804545+0000","last_fullsized":"2024-03-19T01:09:25.804545+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:24:03.243394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154320+0000","last_change":"2024-03-19T01:07:15.489371+0000","last_active":"2024-03-19T01:09:26.154320+0000","last_peered":"2024-03-19T01:09:26.154320+0000","last_clean":"2024-03-19T01:09:26.154320+0000","last_became_active":"2024-03-19T01:07:15.488975+0000","last_became_peered":"2024-03-19T01:07:15.488975+0000","last_unstale":"2024-03-19T01:09:26.154320+0000","last_undegraded":"2024-03-19T01:09:26.154320+0000","last_fullsized":"2024-03-19T01:09:26.154320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:04:47.640512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.799877+0000","last_change":"2024-03-19T01:07:19.691590+0000","last_active":"2024-03-19T01:09:25.799877+0000","last_peered":"2024-03-19T01:09:25.799877+0000","last_clean":"2024-03-19T01:09:25.799877+0000","last_became_active":"2024-03-19T01:07:19.691037+0000","last_became_peered":"2024-03-19T01:07:19.691037+0000","last_unstale":"2024-03-19T01:09:25.799877+0000","last_undegraded":"2024-03-19T01:09:25.799877+0000","last_fullsized":"2024-03-19T01:09:25.799877+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:20:02.007483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137775+0000","last_change":"2024-03-19T01:07:14.475653+0000","last_active":"2024-03-19T01:09:26.137775+0000","last_peered":"2024-03-19T01:09:26.137775+0000","last_clean":"2024-03-19T01:09:26.137775+0000","last_became_active":"2024-03-19T01:07:14.475315+0000","last_became_peered":"2024-03-19T01:07:14.475315+0000","last_unstale":"2024-03-19T01:09:26.137775+0000","last_undegraded":"2024-03-19T01:09:26.137775+0000","last_fullsized":"2024-03-19T01:09:26.137775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:44:04.656950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798438+0000","last_change":"2024-03-19T01:07:15.480882+0000","last_active":"2024-03-19T01:09:25.798438+0000","last_peered":"2024-03-19T01:09:25.798438+0000","last_clean":"2024-03-19T01:09:25.798438+0000","last_became_active":"2024-03-19T01:07:15.480541+0000","last_became_peered":"2024-03-19T01:07:15.480541+0000","last_unstale":"2024-03-19T01:09:25.798438+0000","last_undegraded":"2024-03-19T01:09:25.798438+0000","last_fullsized":"2024-03-19T01:09:25.798438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:37:24.724531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914196+0000","last_change":"2024-03-19T01:07:19.672979+0000","last_active":"2024-03-19T01:09:25.914196+0000","last_peered":"2024-03-19T01:09:25.914196+0000","last_clean":"2024-03-19T01:09:25.914196+0000","last_became_active":"2024-03-19T01:07:19.672530+0000","last_became_peered":"2024-03-19T01:07:19.672530+0000","last_unstale":"2024-03-19T01:09:25.914196+0000","last_undegraded":"2024-03-19T01:09:25.914196+0000","last_fullsized":"2024-03-19T01:09:25.914196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:12.948079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":193,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:20.916080+0000","last_change":"2024-03-19T01:07:14.475620+0000","last_active":"2024-03-19T01:10:20.916080+0000","last_peered":"2024-03-19T01:10:20.916080+0000","last_clean":"2024-03-19T01:10:20.916080+0000","last_became_active":"2024-03-19T01:07:14.475029+0000","last_became_peered":"2024-03-19T01:07:14.475029+0000","last_unstale":"2024-03-19T01:10:20.916080+0000","last_undegraded":"2024-03-19T01:10:20.916080+0000","last_fullsized":"2024-03-19T01:10:20.916080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:48:53.983457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137648+0000","last_change":"2024-03-19T01:07:15.484680+0000","last_active":"2024-03-19T01:09:26.137648+0000","last_peered":"2024-03-19T01:09:26.137648+0000","last_clean":"2024-03-19T01:09:26.137648+0000","last_became_active":"2024-03-19T01:07:15.484333+0000","last_became_peered":"2024-03-19T01:07:15.484333+0000","last_unstale":"2024-03-19T01:09:26.137648+0000","last_undegraded":"2024-03-19T01:09:26.137648+0000","last_fullsized":"2024-03-19T01:09:26.137648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:38:23.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914115+0000","last_change":"2024-03-19T01:08:24.798971+0000","last_active":"2024-03-19T01:09:25.914115+0000","last_peered":"2024-03-19T01:09:25.914115+0000","last_clean":"2024-03-19T01:09:25.914115+0000","last_became_active":"2024-03-19T01:08:24.798504+0000","last_became_peered":"2024-03-19T01:08:24.798504+0000","last_unstale":"2024-03-19T01:09:25.914115+0000","last_undegraded":"2024-03-19T01:09:25.914115+0000","last_fullsized":"2024-03-19T01:09:25.914115+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.460490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801639+0000","last_change":"2024-03-19T01:07:14.481499+0000","last_active":"2024-03-19T01:09:25.801639+0000","last_peered":"2024-03-19T01:09:25.801639+0000","last_clean":"2024-03-19T01:09:25.801639+0000","last_became_active":"2024-03-19T01:07:14.481123+0000","last_became_peered":"2024-03-19T01:07:14.481123+0000","last_unstale":"2024-03-19T01:09:25.801639+0000","last_undegraded":"2024-03-19T01:09:25.801639+0000","last_fullsized":"2024-03-19T01:09:25.801639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:36:15.349748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798153+0000","last_change":"2024-03-19T01:07:15.499744+0000","last_active":"2024-03-19T01:09:25.798153+0000","last_peered":"2024-03-19T01:09:25.798153+0000","last_clean":"2024-03-19T01:09:25.798153+0000","last_became_active":"2024-03-19T01:07:15.499037+0000","last_became_peered":"2024-03-19T01:07:15.499037+0000","last_unstale":"2024-03-19T01:09:25.798153+0000","last_undegraded":"2024-03-19T01:09:25.798153+0000","last_fullsized":"2024-03-19T01:09:25.798153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:20:35.524664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804490+0000","last_change":"2024-03-19T01:07:19.684187+0000","last_active":"2024-03-19T01:09:25.804490+0000","last_peered":"2024-03-19T01:09:25.804490+0000","last_clean":"2024-03-19T01:09:25.804490+0000","last_became_active":"2024-03-19T01:07:19.683782+0000","last_became_peered":"2024-03-19T01:07:19.683782+0000","last_unstale":"2024-03-19T01:09:25.804490+0000","last_undegraded":"2024-03-19T01:09:25.804490+0000","last_fullsized":"2024-03-19T01:09:25.804490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:17:50.818003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.802689+0000","last_change":"2024-03-19T01:07:14.485538+0000","last_active":"2024-03-19T01:09:25.802689+0000","last_peered":"2024-03-19T01:09:25.802689+0000","last_clean":"2024-03-19T01:09:25.802689+0000","last_became_active":"2024-03-19T01:07:14.485004+0000","last_became_peered":"2024-03-19T01:07:14.485004+0000","last_unstale":"2024-03-19T01:09:25.802689+0000","last_undegraded":"2024-03-19T01:09:25.802689+0000","last_fullsized":"2024-03-19T01:09:25.802689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:55:50.853725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914576+0000","last_change":"2024-03-19T01:07:15.503370+0000","last_active":"2024-03-19T01:09:25.914576+0000","last_peered":"2024-03-19T01:09:25.914576+0000","last_clean":"2024-03-19T01:09:25.914576+0000","last_became_active":"2024-03-19T01:07:15.501375+0000","last_became_peered":"2024-03-19T01:07:15.501375+0000","last_unstale":"2024-03-19T01:09:25.914576+0000","last_undegraded":"2024-03-19T01:09:25.914576+0000","last_fullsized":"2024-03-19T01:09:25.914576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:36:01.814936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156480+0000","last_change":"2024-03-19T01:07:19.697067+0000","last_active":"2024-03-19T01:09:26.156480+0000","last_peered":"2024-03-19T01:09:26.156480+0000","last_clean":"2024-03-19T01:09:26.156480+0000","last_became_active":"2024-03-19T01:07:19.696364+0000","last_became_peered":"2024-03-19T01:07:19.696364+0000","last_unstale":"2024-03-19T01:09:26.156480+0000","last_undegraded":"2024-03-19T01:09:26.156480+0000","last_fullsized":"2024-03-19T01:09:26.156480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:34.388407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.797972+0000","last_change":"2024-03-19T01:07:14.475361+0000","last_active":"2024-03-19T01:09:25.797972+0000","last_peered":"2024-03-19T01:09:25.797972+0000","last_clean":"2024-03-19T01:09:25.797972+0000","last_became_active":"2024-03-19T01:07:14.474577+0000","last_became_peered":"2024-03-19T01:07:14.474577+0000","last_unstale":"2024-03-19T01:09:25.797972+0000","last_undegraded":"2024-03-19T01:09:25.797972+0000","last_fullsized":"2024-03-19T01:09:25.797972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:01:04.713405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800954+0000","last_change":"2024-03-19T01:07:15.506310+0000","last_active":"2024-03-19T01:09:25.800954+0000","last_peered":"2024-03-19T01:09:25.800954+0000","last_clean":"2024-03-19T01:09:25.800954+0000","last_became_active":"2024-03-19T01:07:15.505792+0000","last_became_peered":"2024-03-19T01:07:15.505792+0000","last_unstale":"2024-03-19T01:09:25.800954+0000","last_undegraded":"2024-03-19T01:09:25.800954+0000","last_fullsized":"2024-03-19T01:09:25.800954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:40.630816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914516+0000","last_change":"2024-03-19T01:07:19.683632+0000","last_active":"2024-03-19T01:09:25.914516+0000","last_peered":"2024-03-19T01:09:25.914516+0000","last_clean":"2024-03-19T01:09:25.914516+0000","last_became_active":"2024-03-19T01:07:19.683204+0000","last_became_peered":"2024-03-19T01:07:19.683204+0000","last_unstale":"2024-03-19T01:09:25.914516+0000","last_undegraded":"2024-03-19T01:09:25.914516+0000","last_fullsized":"2024-03-19T01:09:25.914516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:58:53.512824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":342,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.916543+0000","last_change":"2024-03-19T01:07:14.471347+0000","last_active":"2024-03-19T01:10:21.916543+0000","last_peered":"2024-03-19T01:10:21.916543+0000","last_clean":"2024-03-19T01:10:21.916543+0000","last_became_active":"2024-03-19T01:07:14.471001+0000","last_became_peered":"2024-03-19T01:07:14.471001+0000","last_unstale":"2024-03-19T01:10:21.916543+0000","last_undegraded":"2024-03-19T01:10:21.916543+0000","last_fullsized":"2024-03-19T01:10:21.916543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:46.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":306,"num_read_kb":306,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154549+0000","last_change":"2024-03-19T01:07:15.486779+0000","last_active":"2024-03-19T01:09:26.154549+0000","last_peered":"2024-03-19T01:09:26.154549+0000","last_clean":"2024-03-19T01:09:26.154549+0000","last_became_active":"2024-03-19T01:07:15.486405+0000","last_became_peered":"2024-03-19T01:07:15.486405+0000","last_unstale":"2024-03-19T01:09:26.154549+0000","last_undegraded":"2024-03-19T01:09:26.154549+0000","last_fullsized":"2024-03-19T01:09:26.154549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:14:13.350169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800785+0000","last_change":"2024-03-19T01:07:19.686892+0000","last_active":"2024-03-19T01:09:25.800785+0000","last_peered":"2024-03-19T01:09:25.800785+0000","last_clean":"2024-03-19T01:09:25.800785+0000","last_became_active":"2024-03-19T01:07:19.686414+0000","last_became_peered":"2024-03-19T01:07:19.686414+0000","last_unstale":"2024-03-19T01:09:25.800785+0000","last_undegraded":"2024-03-19T01:09:25.800785+0000","last_fullsized":"2024-03-19T01:09:25.800785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:18:18.105905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962657+0000","last_change":"2024-03-19T01:08:24.792588+0000","last_active":"2024-03-19T01:09:25.962657+0000","last_peered":"2024-03-19T01:09:25.962657+0000","last_clean":"2024-03-19T01:09:25.962657+0000","last_became_active":"2024-03-19T01:08:24.788228+0000","last_became_peered":"2024-03-19T01:08:24.788228+0000","last_unstale":"2024-03-19T01:09:25.962657+0000","last_undegraded":"2024-03-19T01:09:25.962657+0000","last_fullsized":"2024-03-19T01:09:25.962657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:52.150165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156831+0000","last_change":"2024-03-19T01:07:15.499507+0000","last_active":"2024-03-19T01:09:26.156831+0000","last_peered":"2024-03-19T01:09:26.156831+0000","last_clean":"2024-03-19T01:09:26.156831+0000","last_became_active":"2024-03-19T01:07:15.497261+0000","last_became_peered":"2024-03-19T01:07:15.497261+0000","last_unstale":"2024-03-19T01:09:26.156831+0000","last_undegraded":"2024-03-19T01:09:26.156831+0000","last_fullsized":"2024-03-19T01:09:26.156831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:40:06.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800171+0000","last_change":"2024-03-19T01:07:19.690847+0000","last_active":"2024-03-19T01:09:25.800171+0000","last_peered":"2024-03-19T01:09:25.800171+0000","last_clean":"2024-03-19T01:09:25.800171+0000","last_became_active":"2024-03-19T01:07:19.689762+0000","last_became_peered":"2024-03-19T01:07:19.689762+0000","last_unstale":"2024-03-19T01:09:25.800171+0000","last_undegraded":"2024-03-19T01:09:25.800171+0000","last_fullsized":"2024-03-19T01:09:25.800171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:24:40.213171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798343+0000","last_change":"2024-03-19T01:07:14.473744+0000","last_active":"2024-03-19T01:09:25.798343+0000","last_peered":"2024-03-19T01:09:25.798343+0000","last_clean":"2024-03-19T01:09:25.798343+0000","last_became_active":"2024-03-19T01:07:14.473404+0000","last_became_peered":"2024-03-19T01:07:14.473404+0000","last_unstale":"2024-03-19T01:09:25.798343+0000","last_undegraded":"2024-03-19T01:09:25.798343+0000","last_fullsized":"2024-03-19T01:09:25.798343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:29:39.892678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154443+0000","last_change":"2024-03-19T01:07:15.481444+0000","last_active":"2024-03-19T01:09:26.154443+0000","last_peered":"2024-03-19T01:09:26.154443+0000","last_clean":"2024-03-19T01:09:26.154443+0000","last_became_active":"2024-03-19T01:07:15.481106+0000","last_became_peered":"2024-03-19T01:07:15.481106+0000","last_unstale":"2024-03-19T01:09:26.154443+0000","last_undegraded":"2024-03-19T01:09:26.154443+0000","last_fullsized":"2024-03-19T01:09:26.154443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:00.263063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801166+0000","last_change":"2024-03-19T01:07:19.686919+0000","last_active":"2024-03-19T01:09:25.801166+0000","last_peered":"2024-03-19T01:09:25.801166+0000","last_clean":"2024-03-19T01:09:25.801166+0000","last_became_active":"2024-03-19T01:07:19.686416+0000","last_became_peered":"2024-03-19T01:07:19.686416+0000","last_unstale":"2024-03-19T01:09:25.801166+0000","last_undegraded":"2024-03-19T01:09:25.801166+0000","last_fullsized":"2024-03-19T01:09:25.801166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:29:11.068957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'41","reported_seq":392,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.917526+0000","last_change":"2024-03-19T01:07:14.468630+0000","last_active":"2024-03-19T01:10:21.917526+0000","last_peered":"2024-03-19T01:10:21.917526+0000","last_clean":"2024-03-19T01:10:21.917526+0000","last_became_active":"2024-03-19T01:07:14.468293+0000","last_became_peered":"2024-03-19T01:07:14.468293+0000","last_unstale":"2024-03-19T01:10:21.917526+0000","last_undegraded":"2024-03-19T01:10:21.917526+0000","last_fullsized":"2024-03-19T01:10:21.917526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:53.575507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70405,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":319,"num_read_kb":230,"num_write":43,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137935+0000","last_change":"2024-03-19T01:07:15.477226+0000","last_active":"2024-03-19T01:09:26.137935+0000","last_peered":"2024-03-19T01:09:26.137935+0000","last_clean":"2024-03-19T01:09:26.137935+0000","last_became_active":"2024-03-19T01:07:15.476882+0000","last_became_peered":"2024-03-19T01:07:15.476882+0000","last_unstale":"2024-03-19T01:09:26.137935+0000","last_undegraded":"2024-03-19T01:09:26.137935+0000","last_fullsized":"2024-03-19T01:09:26.137935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:11:10.051837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798608+0000","last_change":"2024-03-19T01:07:19.690004+0000","last_active":"2024-03-19T01:09:25.798608+0000","last_peered":"2024-03-19T01:09:25.798608+0000","last_clean":"2024-03-19T01:09:25.798608+0000","last_became_active":"2024-03-19T01:07:19.689573+0000","last_became_peered":"2024-03-19T01:07:19.689573+0000","last_unstale":"2024-03-19T01:09:25.798608+0000","last_undegraded":"2024-03-19T01:09:25.798608+0000","last_fullsized":"2024-03-19T01:09:25.798608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:31:21.788219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914144+0000","last_change":"2024-03-19T01:07:14.487373+0000","last_active":"2024-03-19T01:09:25.914144+0000","last_peered":"2024-03-19T01:09:25.914144+0000","last_clean":"2024-03-19T01:09:25.914144+0000","last_became_active":"2024-03-19T01:07:14.487077+0000","last_became_peered":"2024-03-19T01:07:14.487077+0000","last_unstale":"2024-03-19T01:09:25.914144+0000","last_undegraded":"2024-03-19T01:09:25.914144+0000","last_fullsized":"2024-03-19T01:09:25.914144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:33:01.789431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.803723+0000","last_change":"2024-03-19T01:07:15.498009+0000","last_active":"2024-03-19T01:09:25.803723+0000","last_peered":"2024-03-19T01:09:25.803723+0000","last_clean":"2024-03-19T01:09:25.803723+0000","last_became_active":"2024-03-19T01:07:15.496707+0000","last_became_peered":"2024-03-19T01:07:15.496707+0000","last_unstale":"2024-03-19T01:09:25.803723+0000","last_undegraded":"2024-03-19T01:09:25.803723+0000","last_fullsized":"2024-03-19T01:09:25.803723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:50.058299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804327+0000","last_change":"2024-03-19T01:07:19.677635+0000","last_active":"2024-03-19T01:09:25.804327+0000","last_peered":"2024-03-19T01:09:25.804327+0000","last_clean":"2024-03-19T01:09:25.804327+0000","last_became_active":"2024-03-19T01:07:19.677207+0000","last_became_peered":"2024-03-19T01:07:19.677207+0000","last_unstale":"2024-03-19T01:09:25.804327+0000","last_undegraded":"2024-03-19T01:09:25.804327+0000","last_fullsized":"2024-03-19T01:09:25.804327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:23:28.650962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798247+0000","last_change":"2024-03-19T01:07:14.482054+0000","last_active":"2024-03-19T01:09:25.798247+0000","last_peered":"2024-03-19T01:09:25.798247+0000","last_clean":"2024-03-19T01:09:25.798247+0000","last_became_active":"2024-03-19T01:07:14.481703+0000","last_became_peered":"2024-03-19T01:07:14.481703+0000","last_unstale":"2024-03-19T01:09:25.798247+0000","last_undegraded":"2024-03-19T01:09:25.798247+0000","last_fullsized":"2024-03-19T01:09:25.798247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:02:43.565442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962388+0000","last_change":"2024-03-19T01:07:15.487236+0000","last_active":"2024-03-19T01:09:25.962388+0000","last_peered":"2024-03-19T01:09:25.962388+0000","last_clean":"2024-03-19T01:09:25.962388+0000","last_became_active":"2024-03-19T01:07:15.486931+0000","last_became_peered":"2024-03-19T01:07:15.486931+0000","last_unstale":"2024-03-19T01:09:25.962388+0000","last_undegraded":"2024-03-19T01:09:25.962388+0000","last_fullsized":"2024-03-19T01:09:25.962388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:22.201904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800840+0000","last_change":"2024-03-19T01:07:19.693813+0000","last_active":"2024-03-19T01:09:25.800840+0000","last_peered":"2024-03-19T01:09:25.800840+0000","last_clean":"2024-03-19T01:09:25.800840+0000","last_became_active":"2024-03-19T01:07:19.693384+0000","last_became_peered":"2024-03-19T01:07:19.693384+0000","last_unstale":"2024-03-19T01:09:25.800840+0000","last_undegraded":"2024-03-19T01:09:25.800840+0000","last_fullsized":"2024-03-19T01:09:25.800840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:09.372461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'9","reported_seq":202,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:22.006239+0000","last_change":"2024-03-19T01:07:14.468467+0000","last_active":"2024-03-19T01:10:22.006239+0000","last_peered":"2024-03-19T01:10:22.006239+0000","last_clean":"2024-03-19T01:10:22.006239+0000","last_became_active":"2024-03-19T01:07:14.468074+0000","last_became_peered":"2024-03-19T01:07:14.468074+0000","last_unstale":"2024-03-19T01:10:22.006239+0000","last_undegraded":"2024-03-19T01:10:22.006239+0000","last_fullsized":"2024-03-19T01:10:22.006239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:07:49.216998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":162,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962143+0000","last_change":"2024-03-19T01:07:15.475310+0000","last_active":"2024-03-19T01:09:25.962143+0000","last_peered":"2024-03-19T01:09:25.962143+0000","last_clean":"2024-03-19T01:09:25.962143+0000","last_became_active":"2024-03-19T01:07:15.474914+0000","last_became_peered":"2024-03-19T01:07:15.474914+0000","last_unstale":"2024-03-19T01:09:25.962143+0000","last_undegraded":"2024-03-19T01:09:25.962143+0000","last_fullsized":"2024-03-19T01:09:25.962143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:34.036892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804790+0000","last_change":"2024-03-19T01:07:19.697937+0000","last_active":"2024-03-19T01:09:25.804790+0000","last_peered":"2024-03-19T01:09:25.804790+0000","last_clean":"2024-03-19T01:09:25.804790+0000","last_became_active":"2024-03-19T01:07:19.697447+0000","last_became_peered":"2024-03-19T01:07:19.697447+0000","last_unstale":"2024-03-19T01:09:25.804790+0000","last_undegraded":"2024-03-19T01:09:25.804790+0000","last_fullsized":"2024-03-19T01:09:25.804790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:49.259118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137706+0000","last_change":"2024-03-19T01:07:14.476382+0000","last_active":"2024-03-19T01:09:26.137706+0000","last_peered":"2024-03-19T01:09:26.137706+0000","last_clean":"2024-03-19T01:09:26.137706+0000","last_became_active":"2024-03-19T01:07:14.476089+0000","last_became_peered":"2024-03-19T01:07:14.476089+0000","last_unstale":"2024-03-19T01:09:26.137706+0000","last_undegraded":"2024-03-19T01:09:26.137706+0000","last_fullsized":"2024-03-19T01:09:26.137706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:57:42.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800778+0000","last_change":"2024-03-19T01:08:24.794335+0000","last_active":"2024-03-19T01:09:25.800778+0000","last_peered":"2024-03-19T01:09:25.800778+0000","last_clean":"2024-03-19T01:09:25.800778+0000","last_became_active":"2024-03-19T01:08:24.793725+0000","last_became_peered":"2024-03-19T01:08:24.793725+0000","last_unstale":"2024-03-19T01:09:25.800778+0000","last_undegraded":"2024-03-19T01:09:25.800778+0000","last_fullsized":"2024-03-19T01:09:25.800778+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:18.797824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798009+0000","last_change":"2024-03-19T01:07:19.688235+0000","last_active":"2024-03-19T01:09:25.798009+0000","last_peered":"2024-03-19T01:09:25.798009+0000","last_clean":"2024-03-19T01:09:25.798009+0000","last_became_active":"2024-03-19T01:07:19.687879+0000","last_became_peered":"2024-03-19T01:07:19.687879+0000","last_unstale":"2024-03-19T01:09:25.798009+0000","last_undegraded":"2024-03-19T01:09:25.798009+0000","last_fullsized":"2024-03-19T01:09:25.798009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:29.465971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:33.028596+0000","last_change":"2024-03-19T01:07:14.474346+0000","last_active":"2024-03-19T01:09:33.028596+0000","last_peered":"2024-03-19T01:09:33.028596+0000","last_clean":"2024-03-19T01:09:33.028596+0000","last_became_active":"2024-03-19T01:07:14.473989+0000","last_became_peered":"2024-03-19T01:07:14.473989+0000","last_unstale":"2024-03-19T01:09:33.028596+0000","last_undegraded":"2024-03-19T01:09:33.028596+0000","last_fullsized":"2024-03-19T01:09:33.028596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:41.004155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804461+0000","last_change":"2024-03-19T01:07:15.482326+0000","last_active":"2024-03-19T01:09:25.804461+0000","last_peered":"2024-03-19T01:09:25.804461+0000","last_clean":"2024-03-19T01:09:25.804461+0000","last_became_active":"2024-03-19T01:07:15.481492+0000","last_became_peered":"2024-03-19T01:07:15.481492+0000","last_unstale":"2024-03-19T01:09:25.804461+0000","last_undegraded":"2024-03-19T01:09:25.804461+0000","last_fullsized":"2024-03-19T01:09:25.804461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:17:31.572104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137621+0000","last_change":"2024-03-19T01:07:19.668853+0000","last_active":"2024-03-19T01:09:26.137621+0000","last_peered":"2024-03-19T01:09:26.137621+0000","last_clean":"2024-03-19T01:09:26.137621+0000","last_became_active":"2024-03-19T01:07:19.668428+0000","last_became_peered":"2024-03-19T01:07:19.668428+0000","last_unstale":"2024-03-19T01:09:26.137621+0000","last_undegraded":"2024-03-19T01:09:26.137621+0000","last_fullsized":"2024-03-19T01:09:26.137621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:50:43.409586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801707+0000","last_change":"2024-03-19T01:07:14.469548+0000","last_active":"2024-03-19T01:09:25.801707+0000","last_peered":"2024-03-19T01:09:25.801707+0000","last_clean":"2024-03-19T01:09:25.801707+0000","last_became_active":"2024-03-19T01:07:14.469200+0000","last_became_peered":"2024-03-19T01:07:14.469200+0000","last_unstale":"2024-03-19T01:09:25.801707+0000","last_undegraded":"2024-03-19T01:09:25.801707+0000","last_fullsized":"2024-03-19T01:09:25.801707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:41:53.179343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801883+0000","last_change":"2024-03-19T01:07:15.495300+0000","last_active":"2024-03-19T01:09:25.801883+0000","last_peered":"2024-03-19T01:09:25.801883+0000","last_clean":"2024-03-19T01:09:25.801883+0000","last_became_active":"2024-03-19T01:07:15.494938+0000","last_became_peered":"2024-03-19T01:07:15.494938+0000","last_unstale":"2024-03-19T01:09:25.801883+0000","last_undegraded":"2024-03-19T01:09:25.801883+0000","last_fullsized":"2024-03-19T01:09:25.801883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:50:38.855857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804281+0000","last_change":"2024-03-19T01:07:19.693261+0000","last_active":"2024-03-19T01:09:25.804281+0000","last_peered":"2024-03-19T01:09:25.804281+0000","last_clean":"2024-03-19T01:09:25.804281+0000","last_became_active":"2024-03-19T01:07:19.692598+0000","last_became_peered":"2024-03-19T01:07:19.692598+0000","last_unstale":"2024-03-19T01:09:25.804281+0000","last_undegraded":"2024-03-19T01:09:25.804281+0000","last_fullsized":"2024-03-19T01:09:25.804281+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:43:29.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":192,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.039962+0000","last_change":"2024-03-19T01:07:14.481569+0000","last_active":"2024-03-19T01:10:21.039962+0000","last_peered":"2024-03-19T01:10:21.039962+0000","last_clean":"2024-03-19T01:10:21.039962+0000","last_became_active":"2024-03-19T01:07:14.481209+0000","last_became_peered":"2024-03-19T01:07:14.481209+0000","last_unstale":"2024-03-19T01:10:21.039962+0000","last_undegraded":"2024-03-19T01:10:21.039962+0000","last_fullsized":"2024-03-19T01:10:21.039962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:39.411207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":154,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801654+0000","last_change":"2024-03-19T01:07:15.496852+0000","last_active":"2024-03-19T01:09:25.801654+0000","last_peered":"2024-03-19T01:09:25.801654+0000","last_clean":"2024-03-19T01:09:25.801654+0000","last_became_active":"2024-03-19T01:07:15.496455+0000","last_became_peered":"2024-03-19T01:07:15.496455+0000","last_unstale":"2024-03-19T01:09:25.801654+0000","last_undegraded":"2024-03-19T01:09:25.801654+0000","last_fullsized":"2024-03-19T01:09:25.801654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:43.419496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.805138+0000","last_change":"2024-03-19T01:07:19.692856+0000","last_active":"2024-03-19T01:09:25.805138+0000","last_peered":"2024-03-19T01:09:25.805138+0000","last_clean":"2024-03-19T01:09:25.805138+0000","last_became_active":"2024-03-19T01:07:19.690712+0000","last_became_peered":"2024-03-19T01:07:19.690712+0000","last_unstale":"2024-03-19T01:09:25.805138+0000","last_undegraded":"2024-03-19T01:09:25.805138+0000","last_fullsized":"2024-03-19T01:09:25.805138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:29:28.473062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798036+0000","last_change":"2024-03-19T01:07:15.474375+0000","last_active":"2024-03-19T01:09:25.798036+0000","last_peered":"2024-03-19T01:09:25.798036+0000","last_clean":"2024-03-19T01:09:25.798036+0000","last_became_active":"2024-03-19T01:07:15.473427+0000","last_became_peered":"2024-03-19T01:07:15.473427+0000","last_unstale":"2024-03-19T01:09:25.798036+0000","last_undegraded":"2024-03-19T01:09:25.798036+0000","last_fullsized":"2024-03-19T01:09:25.798036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:42:22.235054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'7","reported_seq":194,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.004294+0000","last_change":"2024-03-19T01:07:14.490924+0000","last_active":"2024-03-19T01:10:21.004294+0000","last_peered":"2024-03-19T01:10:21.004294+0000","last_clean":"2024-03-19T01:10:21.004294+0000","last_became_active":"2024-03-19T01:07:14.490621+0000","last_became_peered":"2024-03-19T01:07:14.490621+0000","last_unstale":"2024-03-19T01:10:21.004294+0000","last_undegraded":"2024-03-19T01:10:21.004294+0000","last_fullsized":"2024-03-19T01:10:21.004294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:02:52.254138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":155,"num_read_kb":155,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962466+0000","last_change":"2024-03-19T01:08:24.805196+0000","last_active":"2024-03-19T01:09:25.962466+0000","last_peered":"2024-03-19T01:09:25.962466+0000","last_clean":"2024-03-19T01:09:25.962466+0000","last_became_active":"2024-03-19T01:08:24.800225+0000","last_became_peered":"2024-03-19T01:08:24.800225+0000","last_unstale":"2024-03-19T01:09:25.962466+0000","last_undegraded":"2024-03-19T01:09:25.962466+0000","last_fullsized":"2024-03-19T01:09:25.962466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:02.322846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914787+0000","last_change":"2024-03-19T01:07:15.493218+0000","last_active":"2024-03-19T01:09:25.914787+0000","last_peered":"2024-03-19T01:09:25.914787+0000","last_clean":"2024-03-19T01:09:25.914787+0000","last_became_active":"2024-03-19T01:07:15.492751+0000","last_became_peered":"2024-03-19T01:07:15.492751+0000","last_unstale":"2024-03-19T01:09:25.914787+0000","last_undegraded":"2024-03-19T01:09:25.914787+0000","last_fullsized":"2024-03-19T01:09:25.914787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:48:32.105214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800099+0000","last_change":"2024-03-19T01:07:14.473349+0000","last_active":"2024-03-19T01:09:25.800099+0000","last_peered":"2024-03-19T01:09:25.800099+0000","last_clean":"2024-03-19T01:09:25.800099+0000","last_became_active":"2024-03-19T01:07:14.473014+0000","last_became_peered":"2024-03-19T01:07:14.473014+0000","last_unstale":"2024-03-19T01:09:25.800099+0000","last_undegraded":"2024-03-19T01:09:25.800099+0000","last_fullsized":"2024-03-19T01:09:25.800099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:27:34.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.961963+0000","last_change":"2024-03-19T01:05:56.013594+0000","last_active":"2024-03-19T01:09:25.961963+0000","last_peered":"2024-03-19T01:09:25.961963+0000","last_clean":"2024-03-19T01:09:25.961963+0000","last_became_active":"2024-03-19T01:05:56.013084+0000","last_became_peered":"2024-03-19T01:05:56.013084+0000","last_unstale":"2024-03-19T01:09:25.961963+0000","last_undegraded":"2024-03-19T01:09:25.961963+0000","last_fullsized":"2024-03-19T01:09:25.961963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_clean_scrub_stamp":"2024-03-19T01:02:06.532343+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:23.440161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798358+0000","last_change":"2024-03-19T01:07:19.691111+0000","last_active":"2024-03-19T01:09:25.798358+0000","last_peered":"2024-03-19T01:09:25.798358+0000","last_clean":"2024-03-19T01:09:25.798358+0000","last_became_active":"2024-03-19T01:07:19.690618+0000","last_became_peered":"2024-03-19T01:07:19.690618+0000","last_unstale":"2024-03-19T01:09:25.798358+0000","last_undegraded":"2024-03-19T01:09:25.798358+0000","last_fullsized":"2024-03-19T01:09:25.798358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:17.857394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.802402+0000","last_change":"2024-03-19T01:08:24.801826+0000","last_active":"2024-03-19T01:09:25.802402+0000","last_peered":"2024-03-19T01:09:25.802402+0000","last_clean":"2024-03-19T01:09:25.802402+0000","last_became_active":"2024-03-19T01:08:24.801411+0000","last_became_peered":"2024-03-19T01:08:24.801411+0000","last_unstale":"2024-03-19T01:09:25.802402+0000","last_undegraded":"2024-03-19T01:09:25.802402+0000","last_fullsized":"2024-03-19T01:09:25.802402+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:31.798045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962068+0000","last_change":"2024-03-19T01:07:14.474287+0000","last_active":"2024-03-19T01:09:25.962068+0000","last_peered":"2024-03-19T01:09:25.962068+0000","last_clean":"2024-03-19T01:09:25.962068+0000","last_became_active":"2024-03-19T01:07:14.473921+0000","last_became_peered":"2024-03-19T01:07:14.473921+0000","last_unstale":"2024-03-19T01:09:25.962068+0000","last_undegraded":"2024-03-19T01:09:25.962068+0000","last_fullsized":"2024-03-19T01:09:25.962068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:49.322029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804258+0000","last_change":"2024-03-19T01:07:19.685331+0000","last_active":"2024-03-19T01:09:25.804258+0000","last_peered":"2024-03-19T01:09:25.804258+0000","last_clean":"2024-03-19T01:09:25.804258+0000","last_became_active":"2024-03-19T01:07:19.684876+0000","last_became_peered":"2024-03-19T01:07:19.684876+0000","last_unstale":"2024-03-19T01:09:25.804258+0000","last_undegraded":"2024-03-19T01:09:25.804258+0000","last_fullsized":"2024-03-19T01:09:25.804258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:39.575348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137876+0000","last_change":"2024-03-19T01:07:14.481019+0000","last_active":"2024-03-19T01:09:26.137876+0000","last_peered":"2024-03-19T01:09:26.137876+0000","last_clean":"2024-03-19T01:09:26.137876+0000","last_became_active":"2024-03-19T01:07:14.480666+0000","last_became_peered":"2024-03-19T01:07:14.480666+0000","last_unstale":"2024-03-19T01:09:26.137876+0000","last_undegraded":"2024-03-19T01:09:26.137876+0000","last_fullsized":"2024-03-19T01:09:26.137876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:13:33.669690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804528+0000","last_change":"2024-03-19T01:07:15.490747+0000","last_active":"2024-03-19T01:09:25.804528+0000","last_peered":"2024-03-19T01:09:25.804528+0000","last_clean":"2024-03-19T01:09:25.804528+0000","last_became_active":"2024-03-19T01:07:15.486244+0000","last_became_peered":"2024-03-19T01:07:15.486244+0000","last_unstale":"2024-03-19T01:09:25.804528+0000","last_undegraded":"2024-03-19T01:09:25.804528+0000","last_fullsized":"2024-03-19T01:09:25.804528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:43.212972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914397+0000","last_change":"2024-03-19T01:07:19.684804+0000","last_active":"2024-03-19T01:09:25.914397+0000","last_peered":"2024-03-19T01:09:25.914397+0000","last_clean":"2024-03-19T01:09:25.914397+0000","last_became_active":"2024-03-19T01:07:19.683191+0000","last_became_peered":"2024-03-19T01:07:19.683191+0000","last_unstale":"2024-03-19T01:09:25.914397+0000","last_undegraded":"2024-03-19T01:09:25.914397+0000","last_fullsized":"2024-03-19T01:09:25.914397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:02.015499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'16","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:16.639078+0000","last_change":"2024-03-19T01:07:14.479282+0000","last_active":"2024-03-19T01:10:16.639078+0000","last_peered":"2024-03-19T01:10:16.639078+0000","last_clean":"2024-03-19T01:10:16.639078+0000","last_became_active":"2024-03-19T01:07:14.478876+0000","last_became_peered":"2024-03-19T01:07:14.478876+0000","last_unstale":"2024-03-19T01:10:16.639078+0000","last_undegraded":"2024-03-19T01:10:16.639078+0000","last_fullsized":"2024-03-19T01:10:16.639078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:54.179680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915652+0000","last_change":"2024-03-19T01:07:15.493498+0000","last_active":"2024-03-19T01:09:25.915652+0000","last_peered":"2024-03-19T01:09:25.915652+0000","last_clean":"2024-03-19T01:09:25.915652+0000","last_became_active":"2024-03-19T01:07:15.493110+0000","last_became_peered":"2024-03-19T01:07:15.493110+0000","last_unstale":"2024-03-19T01:09:25.915652+0000","last_undegraded":"2024-03-19T01:09:25.915652+0000","last_fullsized":"2024-03-19T01:09:25.915652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:55:23.608596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.913805+0000","last_change":"2024-03-19T01:07:19.671232+0000","last_active":"2024-03-19T01:09:25.913805+0000","last_peered":"2024-03-19T01:09:25.913805+0000","last_clean":"2024-03-19T01:09:25.913805+0000","last_became_active":"2024-03-19T01:07:19.670818+0000","last_became_peered":"2024-03-19T01:07:19.670818+0000","last_unstale":"2024-03-19T01:09:25.913805+0000","last_undegraded":"2024-03-19T01:09:25.913805+0000","last_fullsized":"2024-03-19T01:09:25.913805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:57:20.134372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800619+0000","last_change":"2024-03-19T01:07:14.473568+0000","last_active":"2024-03-19T01:09:25.800619+0000","last_peered":"2024-03-19T01:09:25.800619+0000","last_clean":"2024-03-19T01:09:25.800619+0000","last_became_active":"2024-03-19T01:07:14.472555+0000","last_became_peered":"2024-03-19T01:07:14.472555+0000","last_unstale":"2024-03-19T01:09:25.800619+0000","last_undegraded":"2024-03-19T01:09:25.800619+0000","last_fullsized":"2024-03-19T01:09:25.800619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:28:01.541228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156691+0000","last_change":"2024-03-19T01:07:15.499905+0000","last_active":"2024-03-19T01:09:26.156691+0000","last_peered":"2024-03-19T01:09:26.156691+0000","last_clean":"2024-03-19T01:09:26.156691+0000","last_became_active":"2024-03-19T01:07:15.499560+0000","last_became_peered":"2024-03-19T01:07:15.499560+0000","last_unstale":"2024-03-19T01:09:26.156691+0000","last_undegraded":"2024-03-19T01:09:26.156691+0000","last_fullsized":"2024-03-19T01:09:26.156691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:06:26.531100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800426+0000","last_change":"2024-03-19T01:07:19.690310+0000","last_active":"2024-03-19T01:09:25.800426+0000","last_peered":"2024-03-19T01:09:25.800426+0000","last_clean":"2024-03-19T01:09:25.800426+0000","last_became_active":"2024-03-19T01:07:19.689877+0000","last_became_peered":"2024-03-19T01:07:19.689877+0000","last_unstale":"2024-03-19T01:09:25.800426+0000","last_undegraded":"2024-03-19T01:09:25.800426+0000","last_fullsized":"2024-03-19T01:09:25.800426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:06:12.730258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154081+0000","last_change":"2024-03-19T01:07:14.466267+0000","last_active":"2024-03-19T01:09:26.154081+0000","last_peered":"2024-03-19T01:09:26.154081+0000","last_clean":"2024-03-19T01:09:26.154081+0000","last_became_active":"2024-03-19T01:07:14.465944+0000","last_became_peered":"2024-03-19T01:07:14.465944+0000","last_unstale":"2024-03-19T01:09:26.154081+0000","last_undegraded":"2024-03-19T01:09:26.154081+0000","last_fullsized":"2024-03-19T01:09:26.154081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:40.957200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137477+0000","last_change":"2024-03-19T01:07:15.475690+0000","last_active":"2024-03-19T01:09:26.137477+0000","last_peered":"2024-03-19T01:09:26.137477+0000","last_clean":"2024-03-19T01:09:26.137477+0000","last_became_active":"2024-03-19T01:07:15.475342+0000","last_became_peered":"2024-03-19T01:07:15.475342+0000","last_unstale":"2024-03-19T01:09:26.137477+0000","last_undegraded":"2024-03-19T01:09:26.137477+0000","last_fullsized":"2024-03-19T01:09:26.137477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:30:30.842462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137994+0000","last_change":"2024-03-19T01:08:24.797680+0000","last_active":"2024-03-19T01:09:26.137994+0000","last_peered":"2024-03-19T01:09:26.137994+0000","last_clean":"2024-03-19T01:09:26.137994+0000","last_became_active":"2024-03-19T01:08:24.797194+0000","last_became_peered":"2024-03-19T01:08:24.797194+0000","last_unstale":"2024-03-19T01:09:26.137994+0000","last_undegraded":"2024-03-19T01:09:26.137994+0000","last_fullsized":"2024-03-19T01:09:26.137994+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:49.800726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914650+0000","last_change":"2024-03-19T01:07:14.487907+0000","last_active":"2024-03-19T01:09:25.914650+0000","last_peered":"2024-03-19T01:09:25.914650+0000","last_clean":"2024-03-19T01:09:25.914650+0000","last_became_active":"2024-03-19T01:07:14.487618+0000","last_became_peered":"2024-03-19T01:07:14.487618+0000","last_unstale":"2024-03-19T01:09:25.914650+0000","last_undegraded":"2024-03-19T01:09:25.914650+0000","last_fullsized":"2024-03-19T01:09:25.914650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:22:49.218797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804225+0000","last_change":"2024-03-19T01:07:15.492103+0000","last_active":"2024-03-19T01:09:25.804225+0000","last_peered":"2024-03-19T01:09:25.804225+0000","last_clean":"2024-03-19T01:09:25.804225+0000","last_became_active":"2024-03-19T01:07:15.491712+0000","last_became_peered":"2024-03-19T01:07:15.491712+0000","last_unstale":"2024-03-19T01:09:25.804225+0000","last_undegraded":"2024-03-19T01:09:25.804225+0000","last_fullsized":"2024-03-19T01:09:25.804225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:40:13.959986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.799962+0000","last_change":"2024-03-19T01:07:19.677392+0000","last_active":"2024-03-19T01:09:25.799962+0000","last_peered":"2024-03-19T01:09:25.799962+0000","last_clean":"2024-03-19T01:09:25.799962+0000","last_became_active":"2024-03-19T01:07:19.677024+0000","last_became_peered":"2024-03-19T01:07:19.677024+0000","last_unstale":"2024-03-19T01:09:25.799962+0000","last_undegraded":"2024-03-19T01:09:25.799962+0000","last_fullsized":"2024-03-19T01:09:25.799962+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:01:43.404902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804247+0000","last_change":"2024-03-19T01:08:24.805363+0000","last_active":"2024-03-19T01:09:25.804247+0000","last_peered":"2024-03-19T01:09:25.804247+0000","last_clean":"2024-03-19T01:09:25.804247+0000","last_became_active":"2024-03-19T01:08:24.800810+0000","last_became_peered":"2024-03-19T01:08:24.800810+0000","last_unstale":"2024-03-19T01:09:25.804247+0000","last_undegraded":"2024-03-19T01:09:25.804247+0000","last_fullsized":"2024-03-19T01:09:25.804247+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:32:43.608140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915314+0000","last_change":"2024-03-19T01:07:15.505842+0000","last_active":"2024-03-19T01:09:25.915314+0000","last_peered":"2024-03-19T01:09:25.915314+0000","last_clean":"2024-03-19T01:09:25.915314+0000","last_became_active":"2024-03-19T01:07:15.505425+0000","last_became_peered":"2024-03-19T01:07:15.505425+0000","last_unstale":"2024-03-19T01:09:25.915314+0000","last_undegraded":"2024-03-19T01:09:25.915314+0000","last_fullsized":"2024-03-19T01:09:25.915314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:28.119864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800240+0000","last_change":"2024-03-19T01:07:19.683113+0000","last_active":"2024-03-19T01:09:25.800240+0000","last_peered":"2024-03-19T01:09:25.800240+0000","last_clean":"2024-03-19T01:09:25.800240+0000","last_became_active":"2024-03-19T01:07:19.682702+0000","last_became_peered":"2024-03-19T01:07:19.682702+0000","last_unstale":"2024-03-19T01:09:25.800240+0000","last_undegraded":"2024-03-19T01:09:25.800240+0000","last_fullsized":"2024-03-19T01:09:25.800240+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:42.866408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137616+0000","last_change":"2024-03-19T01:07:14.476562+0000","last_active":"2024-03-19T01:09:26.137616+0000","last_peered":"2024-03-19T01:09:26.137616+0000","last_clean":"2024-03-19T01:09:26.137616+0000","last_became_active":"2024-03-19T01:07:14.476185+0000","last_became_peered":"2024-03-19T01:07:14.476185+0000","last_unstale":"2024-03-19T01:09:26.137616+0000","last_undegraded":"2024-03-19T01:09:26.137616+0000","last_fullsized":"2024-03-19T01:09:26.137616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:35:19.860443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962373+0000","last_change":"2024-03-19T01:07:15.487163+0000","last_active":"2024-03-19T01:09:25.962373+0000","last_peered":"2024-03-19T01:09:25.962373+0000","last_clean":"2024-03-19T01:09:25.962373+0000","last_became_active":"2024-03-19T01:07:15.486845+0000","last_became_peered":"2024-03-19T01:07:15.486845+0000","last_unstale":"2024-03-19T01:09:25.962373+0000","last_undegraded":"2024-03-19T01:09:25.962373+0000","last_fullsized":"2024-03-19T01:09:25.962373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:39:09.389411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154143+0000","last_change":"2024-03-19T01:07:19.692768+0000","last_active":"2024-03-19T01:09:26.154143+0000","last_peered":"2024-03-19T01:09:26.154143+0000","last_clean":"2024-03-19T01:09:26.154143+0000","last_became_active":"2024-03-19T01:07:19.692382+0000","last_became_peered":"2024-03-19T01:07:19.692382+0000","last_unstale":"2024-03-19T01:09:26.154143+0000","last_undegraded":"2024-03-19T01:09:26.154143+0000","last_fullsized":"2024-03-19T01:09:26.154143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:26.804938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":189,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:18.964737+0000","last_change":"2024-03-19T01:07:14.472844+0000","last_active":"2024-03-19T01:10:18.964737+0000","last_peered":"2024-03-19T01:10:18.964737+0000","last_clean":"2024-03-19T01:10:18.964737+0000","last_became_active":"2024-03-19T01:07:14.472389+0000","last_became_peered":"2024-03-19T01:07:14.472389+0000","last_unstale":"2024-03-19T01:10:18.964737+0000","last_undegraded":"2024-03-19T01:10:18.964737+0000","last_fullsized":"2024-03-19T01:10:18.964737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:31.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962639+0000","last_change":"2024-03-19T01:07:15.482033+0000","last_active":"2024-03-19T01:09:25.962639+0000","last_peered":"2024-03-19T01:09:25.962639+0000","last_clean":"2024-03-19T01:09:25.962639+0000","last_became_active":"2024-03-19T01:07:15.481671+0000","last_became_peered":"2024-03-19T01:07:15.481671+0000","last_unstale":"2024-03-19T01:09:25.962639+0000","last_undegraded":"2024-03-19T01:09:25.962639+0000","last_fullsized":"2024-03-19T01:09:25.962639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:09.969374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.155872+0000","last_change":"2024-03-19T01:08:24.793574+0000","last_active":"2024-03-19T01:09:26.155872+0000","last_peered":"2024-03-19T01:09:26.155872+0000","last_clean":"2024-03-19T01:09:26.155872+0000","last_became_active":"2024-03-19T01:08:24.793012+0000","last_became_peered":"2024-03-19T01:08:24.793012+0000","last_unstale":"2024-03-19T01:09:26.155872+0000","last_undegraded":"2024-03-19T01:09:26.155872+0000","last_fullsized":"2024-03-19T01:09:26.155872+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:04:53.611271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798450+0000","last_change":"2024-03-19T01:07:14.481677+0000","last_active":"2024-03-19T01:09:25.798450+0000","last_peered":"2024-03-19T01:09:25.798450+0000","last_clean":"2024-03-19T01:09:25.798450+0000","last_became_active":"2024-03-19T01:07:14.481138+0000","last_became_peered":"2024-03-19T01:07:14.481138+0000","last_unstale":"2024-03-19T01:09:25.798450+0000","last_undegraded":"2024-03-19T01:09:25.798450+0000","last_fullsized":"2024-03-19T01:09:25.798450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:52:33.105816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962431+0000","last_change":"2024-03-19T01:07:15.498352+0000","last_active":"2024-03-19T01:09:25.962431+0000","last_peered":"2024-03-19T01:09:25.962431+0000","last_clean":"2024-03-19T01:09:25.962431+0000","last_became_active":"2024-03-19T01:07:15.498040+0000","last_became_peered":"2024-03-19T01:07:15.498040+0000","last_unstale":"2024-03-19T01:09:25.962431+0000","last_undegraded":"2024-03-19T01:09:25.962431+0000","last_fullsized":"2024-03-19T01:09:25.962431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:57:37.571753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.797775+0000","last_change":"2024-03-19T01:07:19.673291+0000","last_active":"2024-03-19T01:09:25.797775+0000","last_peered":"2024-03-19T01:09:25.797775+0000","last_clean":"2024-03-19T01:09:25.797775+0000","last_became_active":"2024-03-19T01:07:19.672704+0000","last_became_peered":"2024-03-19T01:07:19.672704+0000","last_unstale":"2024-03-19T01:09:25.797775+0000","last_undegraded":"2024-03-19T01:09:25.797775+0000","last_fullsized":"2024-03-19T01:09:25.797775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:01:42.019812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804917+0000","last_change":"2024-03-19T01:07:14.489556+0000","last_active":"2024-03-19T01:09:25.804917+0000","last_peered":"2024-03-19T01:09:25.804917+0000","last_clean":"2024-03-19T01:09:25.804917+0000","last_became_active":"2024-03-19T01:07:14.489201+0000","last_became_peered":"2024-03-19T01:07:14.489201+0000","last_unstale":"2024-03-19T01:09:25.804917+0000","last_undegraded":"2024-03-19T01:09:25.804917+0000","last_fullsized":"2024-03-19T01:09:25.804917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:28:36.516185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962688+0000","last_change":"2024-03-19T01:07:15.477727+0000","last_active":"2024-03-19T01:09:25.962688+0000","last_peered":"2024-03-19T01:09:25.962688+0000","last_clean":"2024-03-19T01:09:25.962688+0000","last_became_active":"2024-03-19T01:07:15.477388+0000","last_became_peered":"2024-03-19T01:07:15.477388+0000","last_unstale":"2024-03-19T01:09:25.962688+0000","last_undegraded":"2024-03-19T01:09:25.962688+0000","last_fullsized":"2024-03-19T01:09:25.962688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:30.567326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804232+0000","last_change":"2024-03-19T01:07:15.479008+0000","last_active":"2024-03-19T01:09:25.804232+0000","last_peered":"2024-03-19T01:09:25.804232+0000","last_clean":"2024-03-19T01:09:25.804232+0000","last_became_active":"2024-03-19T01:07:15.478657+0000","last_became_peered":"2024-03-19T01:07:15.478657+0000","last_unstale":"2024-03-19T01:09:25.804232+0000","last_undegraded":"2024-03-19T01:09:25.804232+0000","last_fullsized":"2024-03-19T01:09:25.804232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:42:46.970012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914058+0000","last_change":"2024-03-19T01:08:24.789796+0000","last_active":"2024-03-19T01:09:25.914058+0000","last_peered":"2024-03-19T01:09:25.914058+0000","last_clean":"2024-03-19T01:09:25.914058+0000","last_became_active":"2024-03-19T01:08:24.789444+0000","last_became_peered":"2024-03-19T01:08:24.789444+0000","last_unstale":"2024-03-19T01:09:25.914058+0000","last_undegraded":"2024-03-19T01:09:25.914058+0000","last_fullsized":"2024-03-19T01:09:25.914058+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:41.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.913879+0000","last_change":"2024-03-19T01:07:15.495292+0000","last_active":"2024-03-19T01:09:25.913879+0000","last_peered":"2024-03-19T01:09:25.913879+0000","last_clean":"2024-03-19T01:09:25.913879+0000","last_became_active":"2024-03-19T01:07:15.494946+0000","last_became_peered":"2024-03-19T01:07:15.494946+0000","last_unstale":"2024-03-19T01:09:25.913879+0000","last_undegraded":"2024-03-19T01:09:25.913879+0000","last_fullsized":"2024-03-19T01:09:25.913879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:39.012221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156089+0000","last_change":"2024-03-19T01:07:15.499816+0000","last_active":"2024-03-19T01:09:26.156089+0000","last_peered":"2024-03-19T01:09:26.156089+0000","last_clean":"2024-03-19T01:09:26.156089+0000","last_became_active":"2024-03-19T01:07:15.499031+0000","last_became_peered":"2024-03-19T01:07:15.499031+0000","last_unstale":"2024-03-19T01:09:26.156089+0000","last_undegraded":"2024-03-19T01:09:26.156089+0000","last_fullsized":"2024-03-19T01:09:26.156089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:34:25.344092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804473+0000","last_change":"2024-03-19T01:07:15.501526+0000","last_active":"2024-03-19T01:09:25.804473+0000","last_peered":"2024-03-19T01:09:25.804473+0000","last_clean":"2024-03-19T01:09:25.804473+0000","last_became_active":"2024-03-19T01:07:15.500452+0000","last_became_peered":"2024-03-19T01:07:15.500452+0000","last_unstale":"2024-03-19T01:09:25.804473+0000","last_undegraded":"2024-03-19T01:09:25.804473+0000","last_fullsized":"2024-03-19T01:09:25.804473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:35:33.075780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804632+0000","last_change":"2024-03-19T01:07:15.477649+0000","last_active":"2024-03-19T01:09:25.804632+0000","last_peered":"2024-03-19T01:09:25.804632+0000","last_clean":"2024-03-19T01:09:25.804632+0000","last_became_active":"2024-03-19T01:07:15.477193+0000","last_became_peered":"2024-03-19T01:07:15.477193+0000","last_unstale":"2024-03-19T01:09:25.804632+0000","last_undegraded":"2024-03-19T01:09:25.804632+0000","last_fullsized":"2024-03-19T01:09:25.804632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:36.126642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914305+0000","last_change":"2024-03-19T01:07:15.481920+0000","last_active":"2024-03-19T01:09:25.914305+0000","last_peered":"2024-03-19T01:09:25.914305+0000","last_clean":"2024-03-19T01:09:25.914305+0000","last_became_active":"2024-03-19T01:07:15.481290+0000","last_became_peered":"2024-03-19T01:07:15.481290+0000","last_unstale":"2024-03-19T01:09:25.914305+0000","last_undegraded":"2024-03-19T01:09:25.914305+0000","last_fullsized":"2024-03-19T01:09:25.914305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:37:58.028647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915760+0000","last_change":"2024-03-19T01:07:15.502305+0000","last_active":"2024-03-19T01:09:25.915760+0000","last_peered":"2024-03-19T01:09:25.915760+0000","last_clean":"2024-03-19T01:09:25.915760+0000","last_became_active":"2024-03-19T01:07:15.501491+0000","last_became_peered":"2024-03-19T01:07:15.501491+0000","last_unstale":"2024-03-19T01:09:25.915760+0000","last_undegraded":"2024-03-19T01:09:25.915760+0000","last_fullsized":"2024-03-19T01:09:25.915760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:01.999538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.139189+0000","last_change":"2024-03-19T01:09:26.138802+0000","last_active":"2024-03-19T01:09:26.139189+0000","last_peered":"2024-03-19T01:09:26.139189+0000","last_clean":"2024-03-19T01:09:26.139189+0000","last_became_active":"2024-03-19T01:09:26.138423+0000","last_became_peered":"2024-03-19T01:09:26.138423+0000","last_unstale":"2024-03-19T01:09:26.139189+0000","last_undegraded":"2024-03-19T01:09:26.139189+0000","last_fullsized":"2024-03-19T01:09:26.139189+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:51:23.293922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914829+0000","last_change":"2024-03-19T01:07:15.505022+0000","last_active":"2024-03-19T01:09:25.914829+0000","last_peered":"2024-03-19T01:09:25.914829+0000","last_clean":"2024-03-19T01:09:25.914829+0000","last_became_active":"2024-03-19T01:07:15.503684+0000","last_became_peered":"2024-03-19T01:07:15.503684+0000","last_unstale":"2024-03-19T01:09:25.914829+0000","last_undegraded":"2024-03-19T01:09:25.914829+0000","last_fullsized":"2024-03-19T01:09:25.914829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:56:47.580267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154341+0000","last_change":"2024-03-19T01:07:15.495634+0000","last_active":"2024-03-19T01:09:26.154341+0000","last_peered":"2024-03-19T01:09:26.154341+0000","last_clean":"2024-03-19T01:09:26.154341+0000","last_became_active":"2024-03-19T01:07:15.495206+0000","last_became_peered":"2024-03-19T01:07:15.495206+0000","last_unstale":"2024-03-19T01:09:26.154341+0000","last_undegraded":"2024-03-19T01:09:26.154341+0000","last_fullsized":"2024-03-19T01:09:26.154341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:30:29.965731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804363+0000","last_change":"2024-03-19T01:07:15.501480+0000","last_active":"2024-03-19T01:09:25.804363+0000","last_peered":"2024-03-19T01:09:25.804363+0000","last_clean":"2024-03-19T01:09:25.804363+0000","last_became_active":"2024-03-19T01:07:15.501078+0000","last_became_peered":"2024-03-19T01:07:15.501078+0000","last_unstale":"2024-03-19T01:09:25.804363+0000","last_undegraded":"2024-03-19T01:09:25.804363+0000","last_fullsized":"2024-03-19T01:09:25.804363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:40:18.767874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804424+0000","last_change":"2024-03-19T01:07:15.480358+0000","last_active":"2024-03-19T01:09:25.804424+0000","last_peered":"2024-03-19T01:09:25.804424+0000","last_clean":"2024-03-19T01:09:25.804424+0000","last_became_active":"2024-03-19T01:07:15.479979+0000","last_became_peered":"2024-03-19T01:07:15.479979+0000","last_unstale":"2024-03-19T01:09:25.804424+0000","last_undegraded":"2024-03-19T01:09:25.804424+0000","last_fullsized":"2024-03-19T01:09:25.804424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:12:54.598435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247132,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78447,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1575,"num_read_kb":875,"num_write":264,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":235341,"data_compressed":5469,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383731,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":520302,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645369,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32336,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712816,"statfs":{"total":95995035648,"available":95961923584,"internally_reserved":0,"allocated":1187840,"data_stored":1182848,"data_compressed":64112,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808894,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":524328,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070532,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":520610,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332169,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1118336,"data_compressed":62289,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495695,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":516127,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":587562,"data_compressed":28338,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":517192,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":587391,"data_compressed":28338,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":515444,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":521039,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757334,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":1108202,"data_compressed":62289,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72548,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72377,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":73370,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T01:10:23.985 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T01:10:24.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.220+0000 7f3544e00640 1 -- 172.21.15.69:0/1251263078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35401008a0 msgr2=0x7f3540100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.220+0000 7f3544e00640 1 --2- 172.21.15.69:0/1251263078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35401008a0 0x7f3540100d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3528009900 tx=0x7f352802bc40 comp rx=0 tx=0).stop 2024-03-19T01:10:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.220+0000 7f3544e00640 1 -- 172.21.15.69:0/1251263078 shutdown_connections 2024-03-19T01:10:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.220+0000 7f3544e00640 1 --2- 172.21.15.69:0/1251263078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3540101240 0x7f35401016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.220+0000 7f3544e00640 1 --2- 172.21.15.69:0/1251263078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35401008a0 0x7f3540100d00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.220+0000 7f3544e00640 1 --2- 172.21.15.69:0/1251263078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35400fff40 0x7f3540100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.220+0000 7f3544e00640 1 -- 172.21.15.69:0/1251263078 >> 172.21.15.69:0/1251263078 conn(0x7f35400fbb00 msgr2=0x7f35400fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.220+0000 7f3544e00640 1 -- 172.21.15.69:0/1251263078 shutdown_connections 2024-03-19T01:10:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.220+0000 7f3544e00640 1 -- 172.21.15.69:0/1251263078 wait complete. 2024-03-19T01:10:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.221+0000 7f3544e00640 1 Processor -- start 2024-03-19T01:10:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f3544e00640 1 -- start start 2024-03-19T01:10:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f3544e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35400fff40 0x7f3540199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f3544e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35401008a0 0x7f354019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f3544e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3540101240 0x7f35401a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f3544e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f354010f4e0 con 0x7f35400fff40 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f3544e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f354010f360 con 0x7f3540101240 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f3544e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f354010f660 con 0x7f35401008a0 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f353f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35400fff40 0x7f3540199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f353f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35400fff40 0x7f3540199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39118/0 (socket says 172.21.15.69:39118) 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f353f400640 1 -- 172.21.15.69:0/1437325978 learned_addr learned my addr 172.21.15.69:0/1437325978 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f353fe00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3540101240 0x7f35401a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.222+0000 7f3537e00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35401008a0 0x7f354019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f353fe00640 1 -- 172.21.15.69:0/1437325978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35401008a0 msgr2=0x7f354019a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f353fe00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35401008a0 0x7f354019a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f353fe00640 1 -- 172.21.15.69:0/1437325978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35400fff40 msgr2=0x7f3540199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f353fe00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35400fff40 0x7f3540199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f353fe00640 1 -- 172.21.15.69:0/1437325978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35401a1770 con 0x7f3540101240 2024-03-19T01:10:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f353f400640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35400fff40 0x7f3540199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:24.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f353fe00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3540101240 0x7f35401a1100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3538009fd0 tx=0x7f353800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:24.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f353cc00640 1 -- 172.21.15.69:0/1437325978 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353800ce20 con 0x7f3540101240 2024-03-19T01:10:24.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f353cc00640 1 -- 172.21.15.69:0/1437325978 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3538005020 con 0x7f3540101240 2024-03-19T01:10:24.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f3544e00640 1 -- 172.21.15.69:0/1437325978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35401a1a60 con 0x7f3540101240 2024-03-19T01:10:24.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f353cc00640 1 -- 172.21.15.69:0/1437325978 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538010720 con 0x7f3540101240 2024-03-19T01:10:24.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.223+0000 7f3544e00640 1 -- 172.21.15.69:0/1437325978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35401a1fd0 con 0x7f3540101240 2024-03-19T01:10:24.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.224+0000 7f3544e00640 1 -- 172.21.15.69:0/1437325978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354010ae20 con 0x7f3540101240 2024-03-19T01:10:24.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.227+0000 7f353cc00640 1 -- 172.21.15.69:0/1437325978 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f3538005540 con 0x7f3540101240 2024-03-19T01:10:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.228+0000 7f353cc00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f35240a2910 0x7f35240a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.228+0000 7f353cc00640 1 -- 172.21.15.69:0/1437325978 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f35380cf580 con 0x7f3540101240 2024-03-19T01:10:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.229+0000 7f353f400640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f35240a2910 0x7f35240a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:24.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.229+0000 7f353f400640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f35240a2910 0x7f35240a4dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3530011550 tx=0x7f3530005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:24.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.232+0000 7f353cc00640 1 -- 172.21.15.69:0/1437325978 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f353809c0b0 con 0x7f3540101240 2024-03-19T01:10:24.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:23 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/403616412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:23 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1151791027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:24.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:23 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/403616412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:24.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:23 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1151791027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:24.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:23 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/403616412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:24.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:23 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1151791027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:24.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.547+0000 7f3544e00640 1 -- 172.21.15.69:0/1437325978 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3540061c60 con 0x7f35240a2910 2024-03-19T01:10:24.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.566+0000 7f353cc00640 1 -- 172.21.15.69:0/1437325978 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340835 (secure 0 0 0) 0x7f3540061c60 con 0x7f35240a2910 2024-03-19T01:10:24.572 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-03-19T01:10:24.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 -- 172.21.15.69:0/1437325978 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f35240a2910 msgr2=0x7f35240a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:24.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f35240a2910 0x7f35240a4dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3530011550 tx=0x7f3530005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:24.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 -- 172.21.15.69:0/1437325978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3540101240 msgr2=0x7f35401a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:24.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3540101240 0x7f35401a1100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3538009fd0 tx=0x7f353800ef10 comp rx=0 tx=0).stop 2024-03-19T01:10:24.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 -- 172.21.15.69:0/1437325978 shutdown_connections 2024-03-19T01:10:24.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f35240a2910 0x7f35240a4dd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3540101240 0x7f35401a1100 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35401008a0 0x7f354019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 --2- 172.21.15.69:0/1437325978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35400fff40 0x7f3540199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 -- 172.21.15.69:0/1437325978 >> 172.21.15.69:0/1437325978 conn(0x7f35400fbb00 msgr2=0x7f35400fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:24.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 -- 172.21.15.69:0/1437325978 shutdown_connections 2024-03-19T01:10:24.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.574+0000 7f3544e00640 1 -- 172.21.15.69:0/1437325978 wait complete. 2024-03-19T01:10:24.594 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-03-19T01:10:23.708079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669049,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1711,"num_read_kb":994,"num_write":447,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378420,"kb_used_data":12608,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563404,"statfs":{"total":1151940427776,"available":1151552925696,"internally_reserved":0,"allocated":12910592,"data_stored":8219381,"data_compressed":430971,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8619,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":150,"num_read_kb":87,"num_write":15,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004408"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804627+0000","last_change":"2024-03-19T01:07:15.500098+0000","last_active":"2024-03-19T01:09:25.804627+0000","last_peered":"2024-03-19T01:09:25.804627+0000","last_clean":"2024-03-19T01:09:25.804627+0000","last_became_active":"2024-03-19T01:07:15.499738+0000","last_became_peered":"2024-03-19T01:07:15.499738+0000","last_unstale":"2024-03-19T01:09:25.804627+0000","last_undegraded":"2024-03-19T01:09:25.804627+0000","last_fullsized":"2024-03-19T01:09:25.804627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:14.703153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156585+0000","last_change":"2024-03-19T01:07:15.492863+0000","last_active":"2024-03-19T01:09:26.156585+0000","last_peered":"2024-03-19T01:09:26.156585+0000","last_clean":"2024-03-19T01:09:26.156585+0000","last_became_active":"2024-03-19T01:07:15.492533+0000","last_became_peered":"2024-03-19T01:07:15.492533+0000","last_unstale":"2024-03-19T01:09:26.156585+0000","last_undegraded":"2024-03-19T01:09:26.156585+0000","last_fullsized":"2024-03-19T01:09:26.156585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:49.028352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156213+0000","last_change":"2024-03-19T01:07:15.494174+0000","last_active":"2024-03-19T01:09:26.156213+0000","last_peered":"2024-03-19T01:09:26.156213+0000","last_clean":"2024-03-19T01:09:26.156213+0000","last_became_active":"2024-03-19T01:07:15.490562+0000","last_became_peered":"2024-03-19T01:07:15.490562+0000","last_unstale":"2024-03-19T01:09:26.156213+0000","last_undegraded":"2024-03-19T01:09:26.156213+0000","last_fullsized":"2024-03-19T01:09:26.156213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:18.664396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800428+0000","last_change":"2024-03-19T01:07:15.478397+0000","last_active":"2024-03-19T01:09:25.800428+0000","last_peered":"2024-03-19T01:09:25.800428+0000","last_clean":"2024-03-19T01:09:25.800428+0000","last_became_active":"2024-03-19T01:07:15.477000+0000","last_became_peered":"2024-03-19T01:07:15.477000+0000","last_unstale":"2024-03-19T01:09:25.800428+0000","last_undegraded":"2024-03-19T01:09:25.800428+0000","last_fullsized":"2024-03-19T01:09:25.800428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:58.534329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137621+0000","last_change":"2024-03-19T01:07:15.500138+0000","last_active":"2024-03-19T01:09:26.137621+0000","last_peered":"2024-03-19T01:09:26.137621+0000","last_clean":"2024-03-19T01:09:26.137621+0000","last_became_active":"2024-03-19T01:07:15.499759+0000","last_became_peered":"2024-03-19T01:07:15.499759+0000","last_unstale":"2024-03-19T01:09:26.137621+0000","last_undegraded":"2024-03-19T01:09:26.137621+0000","last_fullsized":"2024-03-19T01:09:26.137621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:23.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804946+0000","last_change":"2024-03-19T01:09:25.804601+0000","last_active":"2024-03-19T01:09:25.804946+0000","last_peered":"2024-03-19T01:09:25.804946+0000","last_clean":"2024-03-19T01:09:25.804946+0000","last_became_active":"2024-03-19T01:09:25.804332+0000","last_became_peered":"2024-03-19T01:09:25.804332+0000","last_unstale":"2024-03-19T01:09:25.804946+0000","last_undegraded":"2024-03-19T01:09:25.804946+0000","last_fullsized":"2024-03-19T01:09:25.804946+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:44:10.372309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962138+0000","last_change":"2024-03-19T01:07:15.486327+0000","last_active":"2024-03-19T01:09:25.962138+0000","last_peered":"2024-03-19T01:09:25.962138+0000","last_clean":"2024-03-19T01:09:25.962138+0000","last_became_active":"2024-03-19T01:07:15.485954+0000","last_became_peered":"2024-03-19T01:07:15.485954+0000","last_unstale":"2024-03-19T01:09:25.962138+0000","last_undegraded":"2024-03-19T01:09:25.962138+0000","last_fullsized":"2024-03-19T01:09:25.962138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:53.316260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914327+0000","last_change":"2024-03-19T01:07:15.504993+0000","last_active":"2024-03-19T01:09:25.914327+0000","last_peered":"2024-03-19T01:09:25.914327+0000","last_clean":"2024-03-19T01:09:25.914327+0000","last_became_active":"2024-03-19T01:07:15.502771+0000","last_became_peered":"2024-03-19T01:07:15.502771+0000","last_unstale":"2024-03-19T01:09:25.914327+0000","last_undegraded":"2024-03-19T01:09:25.914327+0000","last_fullsized":"2024-03-19T01:09:25.914327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:22:33.712608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.806615+0000","last_change":"2024-03-19T01:09:25.805915+0000","last_active":"2024-03-19T01:09:25.806615+0000","last_peered":"2024-03-19T01:09:25.806615+0000","last_clean":"2024-03-19T01:09:25.806615+0000","last_became_active":"2024-03-19T01:09:25.805344+0000","last_became_peered":"2024-03-19T01:09:25.805344+0000","last_unstale":"2024-03-19T01:09:25.806615+0000","last_undegraded":"2024-03-19T01:09:25.806615+0000","last_fullsized":"2024-03-19T01:09:25.806615+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:11:55.795773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801964+0000","last_change":"2024-03-19T01:07:15.475655+0000","last_active":"2024-03-19T01:09:25.801964+0000","last_peered":"2024-03-19T01:09:25.801964+0000","last_clean":"2024-03-19T01:09:25.801964+0000","last_became_active":"2024-03-19T01:07:15.475265+0000","last_became_peered":"2024-03-19T01:07:15.475265+0000","last_unstale":"2024-03-19T01:09:25.801964+0000","last_undegraded":"2024-03-19T01:09:25.801964+0000","last_fullsized":"2024-03-19T01:09:25.801964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:24:38.898191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962612+0000","last_change":"2024-03-19T01:07:15.482988+0000","last_active":"2024-03-19T01:09:25.962612+0000","last_peered":"2024-03-19T01:09:25.962612+0000","last_clean":"2024-03-19T01:09:25.962612+0000","last_became_active":"2024-03-19T01:07:15.482673+0000","last_became_peered":"2024-03-19T01:07:15.482673+0000","last_unstale":"2024-03-19T01:09:25.962612+0000","last_undegraded":"2024-03-19T01:09:25.962612+0000","last_fullsized":"2024-03-19T01:09:25.962612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:15:48.807200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800368+0000","last_change":"2024-03-19T01:07:15.479593+0000","last_active":"2024-03-19T01:09:25.800368+0000","last_peered":"2024-03-19T01:09:25.800368+0000","last_clean":"2024-03-19T01:09:25.800368+0000","last_became_active":"2024-03-19T01:07:15.479220+0000","last_became_peered":"2024-03-19T01:07:15.479220+0000","last_unstale":"2024-03-19T01:09:25.800368+0000","last_undegraded":"2024-03-19T01:09:25.800368+0000","last_fullsized":"2024-03-19T01:09:25.800368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:40:24.390845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804248+0000","last_change":"2024-03-19T01:07:15.464737+0000","last_active":"2024-03-19T01:09:25.804248+0000","last_peered":"2024-03-19T01:09:25.804248+0000","last_clean":"2024-03-19T01:09:25.804248+0000","last_became_active":"2024-03-19T01:07:15.463641+0000","last_became_peered":"2024-03-19T01:07:15.463641+0000","last_unstale":"2024-03-19T01:09:25.804248+0000","last_undegraded":"2024-03-19T01:09:25.804248+0000","last_fullsized":"2024-03-19T01:09:25.804248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:13.964397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.155743+0000","last_change":"2024-03-19T01:07:15.492803+0000","last_active":"2024-03-19T01:09:26.155743+0000","last_peered":"2024-03-19T01:09:26.155743+0000","last_clean":"2024-03-19T01:09:26.155743+0000","last_became_active":"2024-03-19T01:07:15.492422+0000","last_became_peered":"2024-03-19T01:07:15.492422+0000","last_unstale":"2024-03-19T01:09:26.155743+0000","last_undegraded":"2024-03-19T01:09:26.155743+0000","last_fullsized":"2024-03-19T01:09:26.155743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:59:08.292399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.155747+0000","last_change":"2024-03-19T01:07:15.498073+0000","last_active":"2024-03-19T01:09:26.155747+0000","last_peered":"2024-03-19T01:09:26.155747+0000","last_clean":"2024-03-19T01:09:26.155747+0000","last_became_active":"2024-03-19T01:07:15.493650+0000","last_became_peered":"2024-03-19T01:07:15.493650+0000","last_unstale":"2024-03-19T01:09:26.155747+0000","last_undegraded":"2024-03-19T01:09:26.155747+0000","last_fullsized":"2024-03-19T01:09:26.155747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:25:28.908122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154534+0000","last_change":"2024-03-19T01:07:15.496434+0000","last_active":"2024-03-19T01:09:26.154534+0000","last_peered":"2024-03-19T01:09:26.154534+0000","last_clean":"2024-03-19T01:09:26.154534+0000","last_became_active":"2024-03-19T01:07:15.495956+0000","last_became_peered":"2024-03-19T01:07:15.495956+0000","last_unstale":"2024-03-19T01:09:26.154534+0000","last_undegraded":"2024-03-19T01:09:26.154534+0000","last_fullsized":"2024-03-19T01:09:26.154534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:49.505947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804589+0000","last_change":"2024-03-19T01:07:15.504771+0000","last_active":"2024-03-19T01:09:25.804589+0000","last_peered":"2024-03-19T01:09:25.804589+0000","last_clean":"2024-03-19T01:09:25.804589+0000","last_became_active":"2024-03-19T01:07:15.504485+0000","last_became_peered":"2024-03-19T01:07:15.504485+0000","last_unstale":"2024-03-19T01:09:25.804589+0000","last_undegraded":"2024-03-19T01:09:25.804589+0000","last_fullsized":"2024-03-19T01:09:25.804589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:17:36.760943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.802629+0000","last_change":"2024-03-19T01:07:15.491045+0000","last_active":"2024-03-19T01:09:25.802629+0000","last_peered":"2024-03-19T01:09:25.802629+0000","last_clean":"2024-03-19T01:09:25.802629+0000","last_became_active":"2024-03-19T01:07:15.488914+0000","last_became_peered":"2024-03-19T01:07:15.488914+0000","last_unstale":"2024-03-19T01:09:25.802629+0000","last_undegraded":"2024-03-19T01:09:25.802629+0000","last_fullsized":"2024-03-19T01:09:25.802629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:14.853457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914298+0000","last_change":"2024-03-19T01:07:15.495306+0000","last_active":"2024-03-19T01:09:25.914298+0000","last_peered":"2024-03-19T01:09:25.914298+0000","last_clean":"2024-03-19T01:09:25.914298+0000","last_became_active":"2024-03-19T01:07:15.494906+0000","last_became_peered":"2024-03-19T01:07:15.494906+0000","last_unstale":"2024-03-19T01:09:25.914298+0000","last_undegraded":"2024-03-19T01:09:25.914298+0000","last_fullsized":"2024-03-19T01:09:25.914298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:48.176410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.913990+0000","last_change":"2024-03-19T01:07:19.689402+0000","last_active":"2024-03-19T01:09:25.913990+0000","last_peered":"2024-03-19T01:09:25.913990+0000","last_clean":"2024-03-19T01:09:25.913990+0000","last_became_active":"2024-03-19T01:07:19.688799+0000","last_became_peered":"2024-03-19T01:07:19.688799+0000","last_unstale":"2024-03-19T01:09:25.913990+0000","last_undegraded":"2024-03-19T01:09:25.913990+0000","last_fullsized":"2024-03-19T01:09:25.913990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:51.890047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":199,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.039278+0000","last_change":"2024-03-19T01:07:14.477348+0000","last_active":"2024-03-19T01:10:21.039278+0000","last_peered":"2024-03-19T01:10:21.039278+0000","last_clean":"2024-03-19T01:10:21.039278+0000","last_became_active":"2024-03-19T01:07:14.476946+0000","last_became_peered":"2024-03-19T01:07:14.476946+0000","last_unstale":"2024-03-19T01:10:21.039278+0000","last_undegraded":"2024-03-19T01:10:21.039278+0000","last_fullsized":"2024-03-19T01:10:21.039278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:19.269837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915607+0000","last_change":"2024-03-19T01:07:15.495646+0000","last_active":"2024-03-19T01:09:25.915607+0000","last_peered":"2024-03-19T01:09:25.915607+0000","last_clean":"2024-03-19T01:09:25.915607+0000","last_became_active":"2024-03-19T01:07:15.495203+0000","last_became_peered":"2024-03-19T01:07:15.495203+0000","last_unstale":"2024-03-19T01:09:25.915607+0000","last_undegraded":"2024-03-19T01:09:25.915607+0000","last_fullsized":"2024-03-19T01:09:25.915607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:01:31.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.799843+0000","last_change":"2024-03-19T01:07:19.690465+0000","last_active":"2024-03-19T01:09:25.799843+0000","last_peered":"2024-03-19T01:09:25.799843+0000","last_clean":"2024-03-19T01:09:25.799843+0000","last_became_active":"2024-03-19T01:07:19.689852+0000","last_became_peered":"2024-03-19T01:07:19.689852+0000","last_unstale":"2024-03-19T01:09:25.799843+0000","last_undegraded":"2024-03-19T01:09:25.799843+0000","last_fullsized":"2024-03-19T01:09:25.799843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:18.891411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915740+0000","last_change":"2024-03-19T01:07:14.487165+0000","last_active":"2024-03-19T01:09:25.915740+0000","last_peered":"2024-03-19T01:09:25.915740+0000","last_clean":"2024-03-19T01:09:25.915740+0000","last_became_active":"2024-03-19T01:07:14.484194+0000","last_became_peered":"2024-03-19T01:07:14.484194+0000","last_unstale":"2024-03-19T01:09:25.915740+0000","last_undegraded":"2024-03-19T01:09:25.915740+0000","last_fullsized":"2024-03-19T01:09:25.915740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:54:47.914565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156861+0000","last_change":"2024-03-19T01:07:15.498687+0000","last_active":"2024-03-19T01:09:26.156861+0000","last_peered":"2024-03-19T01:09:26.156861+0000","last_clean":"2024-03-19T01:09:26.156861+0000","last_became_active":"2024-03-19T01:07:15.498146+0000","last_became_peered":"2024-03-19T01:07:15.498146+0000","last_unstale":"2024-03-19T01:09:26.156861+0000","last_undegraded":"2024-03-19T01:09:26.156861+0000","last_fullsized":"2024-03-19T01:09:26.156861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:47:21.846333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800041+0000","last_change":"2024-03-19T01:07:19.690691+0000","last_active":"2024-03-19T01:09:25.800041+0000","last_peered":"2024-03-19T01:09:25.800041+0000","last_clean":"2024-03-19T01:09:25.800041+0000","last_became_active":"2024-03-19T01:07:19.690083+0000","last_became_peered":"2024-03-19T01:07:19.690083+0000","last_unstale":"2024-03-19T01:09:25.800041+0000","last_undegraded":"2024-03-19T01:09:25.800041+0000","last_fullsized":"2024-03-19T01:09:25.800041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:51:30.205694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804545+0000","last_change":"2024-03-19T01:08:24.801510+0000","last_active":"2024-03-19T01:09:25.804545+0000","last_peered":"2024-03-19T01:09:25.804545+0000","last_clean":"2024-03-19T01:09:25.804545+0000","last_became_active":"2024-03-19T01:08:24.789315+0000","last_became_peered":"2024-03-19T01:08:24.789315+0000","last_unstale":"2024-03-19T01:09:25.804545+0000","last_undegraded":"2024-03-19T01:09:25.804545+0000","last_fullsized":"2024-03-19T01:09:25.804545+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:24:03.243394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154320+0000","last_change":"2024-03-19T01:07:15.489371+0000","last_active":"2024-03-19T01:09:26.154320+0000","last_peered":"2024-03-19T01:09:26.154320+0000","last_clean":"2024-03-19T01:09:26.154320+0000","last_became_active":"2024-03-19T01:07:15.488975+0000","last_became_peered":"2024-03-19T01:07:15.488975+0000","last_unstale":"2024-03-19T01:09:26.154320+0000","last_undegraded":"2024-03-19T01:09:26.154320+0000","last_fullsized":"2024-03-19T01:09:26.154320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:04:47.640512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.799877+0000","last_change":"2024-03-19T01:07:19.691590+0000","last_active":"2024-03-19T01:09:25.799877+0000","last_peered":"2024-03-19T01:09:25.799877+0000","last_clean":"2024-03-19T01:09:25.799877+0000","last_became_active":"2024-03-19T01:07:19.691037+0000","last_became_peered":"2024-03-19T01:07:19.691037+0000","last_unstale":"2024-03-19T01:09:25.799877+0000","last_undegraded":"2024-03-19T01:09:25.799877+0000","last_fullsized":"2024-03-19T01:09:25.799877+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:20:02.007483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137775+0000","last_change":"2024-03-19T01:07:14.475653+0000","last_active":"2024-03-19T01:09:26.137775+0000","last_peered":"2024-03-19T01:09:26.137775+0000","last_clean":"2024-03-19T01:09:26.137775+0000","last_became_active":"2024-03-19T01:07:14.475315+0000","last_became_peered":"2024-03-19T01:07:14.475315+0000","last_unstale":"2024-03-19T01:09:26.137775+0000","last_undegraded":"2024-03-19T01:09:26.137775+0000","last_fullsized":"2024-03-19T01:09:26.137775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:44:04.656950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798438+0000","last_change":"2024-03-19T01:07:15.480882+0000","last_active":"2024-03-19T01:09:25.798438+0000","last_peered":"2024-03-19T01:09:25.798438+0000","last_clean":"2024-03-19T01:09:25.798438+0000","last_became_active":"2024-03-19T01:07:15.480541+0000","last_became_peered":"2024-03-19T01:07:15.480541+0000","last_unstale":"2024-03-19T01:09:25.798438+0000","last_undegraded":"2024-03-19T01:09:25.798438+0000","last_fullsized":"2024-03-19T01:09:25.798438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:37:24.724531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914196+0000","last_change":"2024-03-19T01:07:19.672979+0000","last_active":"2024-03-19T01:09:25.914196+0000","last_peered":"2024-03-19T01:09:25.914196+0000","last_clean":"2024-03-19T01:09:25.914196+0000","last_became_active":"2024-03-19T01:07:19.672530+0000","last_became_peered":"2024-03-19T01:07:19.672530+0000","last_unstale":"2024-03-19T01:09:25.914196+0000","last_undegraded":"2024-03-19T01:09:25.914196+0000","last_fullsized":"2024-03-19T01:09:25.914196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:12.948079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":193,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:20.916080+0000","last_change":"2024-03-19T01:07:14.475620+0000","last_active":"2024-03-19T01:10:20.916080+0000","last_peered":"2024-03-19T01:10:20.916080+0000","last_clean":"2024-03-19T01:10:20.916080+0000","last_became_active":"2024-03-19T01:07:14.475029+0000","last_became_peered":"2024-03-19T01:07:14.475029+0000","last_unstale":"2024-03-19T01:10:20.916080+0000","last_undegraded":"2024-03-19T01:10:20.916080+0000","last_fullsized":"2024-03-19T01:10:20.916080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:48:53.983457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137648+0000","last_change":"2024-03-19T01:07:15.484680+0000","last_active":"2024-03-19T01:09:26.137648+0000","last_peered":"2024-03-19T01:09:26.137648+0000","last_clean":"2024-03-19T01:09:26.137648+0000","last_became_active":"2024-03-19T01:07:15.484333+0000","last_became_peered":"2024-03-19T01:07:15.484333+0000","last_unstale":"2024-03-19T01:09:26.137648+0000","last_undegraded":"2024-03-19T01:09:26.137648+0000","last_fullsized":"2024-03-19T01:09:26.137648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:38:23.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914115+0000","last_change":"2024-03-19T01:08:24.798971+0000","last_active":"2024-03-19T01:09:25.914115+0000","last_peered":"2024-03-19T01:09:25.914115+0000","last_clean":"2024-03-19T01:09:25.914115+0000","last_became_active":"2024-03-19T01:08:24.798504+0000","last_became_peered":"2024-03-19T01:08:24.798504+0000","last_unstale":"2024-03-19T01:09:25.914115+0000","last_undegraded":"2024-03-19T01:09:25.914115+0000","last_fullsized":"2024-03-19T01:09:25.914115+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.460490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801639+0000","last_change":"2024-03-19T01:07:14.481499+0000","last_active":"2024-03-19T01:09:25.801639+0000","last_peered":"2024-03-19T01:09:25.801639+0000","last_clean":"2024-03-19T01:09:25.801639+0000","last_became_active":"2024-03-19T01:07:14.481123+0000","last_became_peered":"2024-03-19T01:07:14.481123+0000","last_unstale":"2024-03-19T01:09:25.801639+0000","last_undegraded":"2024-03-19T01:09:25.801639+0000","last_fullsized":"2024-03-19T01:09:25.801639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:36:15.349748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798153+0000","last_change":"2024-03-19T01:07:15.499744+0000","last_active":"2024-03-19T01:09:25.798153+0000","last_peered":"2024-03-19T01:09:25.798153+0000","last_clean":"2024-03-19T01:09:25.798153+0000","last_became_active":"2024-03-19T01:07:15.499037+0000","last_became_peered":"2024-03-19T01:07:15.499037+0000","last_unstale":"2024-03-19T01:09:25.798153+0000","last_undegraded":"2024-03-19T01:09:25.798153+0000","last_fullsized":"2024-03-19T01:09:25.798153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:20:35.524664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804490+0000","last_change":"2024-03-19T01:07:19.684187+0000","last_active":"2024-03-19T01:09:25.804490+0000","last_peered":"2024-03-19T01:09:25.804490+0000","last_clean":"2024-03-19T01:09:25.804490+0000","last_became_active":"2024-03-19T01:07:19.683782+0000","last_became_peered":"2024-03-19T01:07:19.683782+0000","last_unstale":"2024-03-19T01:09:25.804490+0000","last_undegraded":"2024-03-19T01:09:25.804490+0000","last_fullsized":"2024-03-19T01:09:25.804490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:17:50.818003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.802689+0000","last_change":"2024-03-19T01:07:14.485538+0000","last_active":"2024-03-19T01:09:25.802689+0000","last_peered":"2024-03-19T01:09:25.802689+0000","last_clean":"2024-03-19T01:09:25.802689+0000","last_became_active":"2024-03-19T01:07:14.485004+0000","last_became_peered":"2024-03-19T01:07:14.485004+0000","last_unstale":"2024-03-19T01:09:25.802689+0000","last_undegraded":"2024-03-19T01:09:25.802689+0000","last_fullsized":"2024-03-19T01:09:25.802689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:55:50.853725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914576+0000","last_change":"2024-03-19T01:07:15.503370+0000","last_active":"2024-03-19T01:09:25.914576+0000","last_peered":"2024-03-19T01:09:25.914576+0000","last_clean":"2024-03-19T01:09:25.914576+0000","last_became_active":"2024-03-19T01:07:15.501375+0000","last_became_peered":"2024-03-19T01:07:15.501375+0000","last_unstale":"2024-03-19T01:09:25.914576+0000","last_undegraded":"2024-03-19T01:09:25.914576+0000","last_fullsized":"2024-03-19T01:09:25.914576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:36:01.814936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156480+0000","last_change":"2024-03-19T01:07:19.697067+0000","last_active":"2024-03-19T01:09:26.156480+0000","last_peered":"2024-03-19T01:09:26.156480+0000","last_clean":"2024-03-19T01:09:26.156480+0000","last_became_active":"2024-03-19T01:07:19.696364+0000","last_became_peered":"2024-03-19T01:07:19.696364+0000","last_unstale":"2024-03-19T01:09:26.156480+0000","last_undegraded":"2024-03-19T01:09:26.156480+0000","last_fullsized":"2024-03-19T01:09:26.156480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:34.388407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.797972+0000","last_change":"2024-03-19T01:07:14.475361+0000","last_active":"2024-03-19T01:09:25.797972+0000","last_peered":"2024-03-19T01:09:25.797972+0000","last_clean":"2024-03-19T01:09:25.797972+0000","last_became_active":"2024-03-19T01:07:14.474577+0000","last_became_peered":"2024-03-19T01:07:14.474577+0000","last_unstale":"2024-03-19T01:09:25.797972+0000","last_undegraded":"2024-03-19T01:09:25.797972+0000","last_fullsized":"2024-03-19T01:09:25.797972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:01:04.713405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800954+0000","last_change":"2024-03-19T01:07:15.506310+0000","last_active":"2024-03-19T01:09:25.800954+0000","last_peered":"2024-03-19T01:09:25.800954+0000","last_clean":"2024-03-19T01:09:25.800954+0000","last_became_active":"2024-03-19T01:07:15.505792+0000","last_became_peered":"2024-03-19T01:07:15.505792+0000","last_unstale":"2024-03-19T01:09:25.800954+0000","last_undegraded":"2024-03-19T01:09:25.800954+0000","last_fullsized":"2024-03-19T01:09:25.800954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:40.630816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914516+0000","last_change":"2024-03-19T01:07:19.683632+0000","last_active":"2024-03-19T01:09:25.914516+0000","last_peered":"2024-03-19T01:09:25.914516+0000","last_clean":"2024-03-19T01:09:25.914516+0000","last_became_active":"2024-03-19T01:07:19.683204+0000","last_became_peered":"2024-03-19T01:07:19.683204+0000","last_unstale":"2024-03-19T01:09:25.914516+0000","last_undegraded":"2024-03-19T01:09:25.914516+0000","last_fullsized":"2024-03-19T01:09:25.914516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:58:53.512824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":342,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.916543+0000","last_change":"2024-03-19T01:07:14.471347+0000","last_active":"2024-03-19T01:10:21.916543+0000","last_peered":"2024-03-19T01:10:21.916543+0000","last_clean":"2024-03-19T01:10:21.916543+0000","last_became_active":"2024-03-19T01:07:14.471001+0000","last_became_peered":"2024-03-19T01:07:14.471001+0000","last_unstale":"2024-03-19T01:10:21.916543+0000","last_undegraded":"2024-03-19T01:10:21.916543+0000","last_fullsized":"2024-03-19T01:10:21.916543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:46.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":306,"num_read_kb":306,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154549+0000","last_change":"2024-03-19T01:07:15.486779+0000","last_active":"2024-03-19T01:09:26.154549+0000","last_peered":"2024-03-19T01:09:26.154549+0000","last_clean":"2024-03-19T01:09:26.154549+0000","last_became_active":"2024-03-19T01:07:15.486405+0000","last_became_peered":"2024-03-19T01:07:15.486405+0000","last_unstale":"2024-03-19T01:09:26.154549+0000","last_undegraded":"2024-03-19T01:09:26.154549+0000","last_fullsized":"2024-03-19T01:09:26.154549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:14:13.350169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800785+0000","last_change":"2024-03-19T01:07:19.686892+0000","last_active":"2024-03-19T01:09:25.800785+0000","last_peered":"2024-03-19T01:09:25.800785+0000","last_clean":"2024-03-19T01:09:25.800785+0000","last_became_active":"2024-03-19T01:07:19.686414+0000","last_became_peered":"2024-03-19T01:07:19.686414+0000","last_unstale":"2024-03-19T01:09:25.800785+0000","last_undegraded":"2024-03-19T01:09:25.800785+0000","last_fullsized":"2024-03-19T01:09:25.800785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:18:18.105905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962657+0000","last_change":"2024-03-19T01:08:24.792588+0000","last_active":"2024-03-19T01:09:25.962657+0000","last_peered":"2024-03-19T01:09:25.962657+0000","last_clean":"2024-03-19T01:09:25.962657+0000","last_became_active":"2024-03-19T01:08:24.788228+0000","last_became_peered":"2024-03-19T01:08:24.788228+0000","last_unstale":"2024-03-19T01:09:25.962657+0000","last_undegraded":"2024-03-19T01:09:25.962657+0000","last_fullsized":"2024-03-19T01:09:25.962657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:52.150165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156831+0000","last_change":"2024-03-19T01:07:15.499507+0000","last_active":"2024-03-19T01:09:26.156831+0000","last_peered":"2024-03-19T01:09:26.156831+0000","last_clean":"2024-03-19T01:09:26.156831+0000","last_became_active":"2024-03-19T01:07:15.497261+0000","last_became_peered":"2024-03-19T01:07:15.497261+0000","last_unstale":"2024-03-19T01:09:26.156831+0000","last_undegraded":"2024-03-19T01:09:26.156831+0000","last_fullsized":"2024-03-19T01:09:26.156831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:40:06.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800171+0000","last_change":"2024-03-19T01:07:19.690847+0000","last_active":"2024-03-19T01:09:25.800171+0000","last_peered":"2024-03-19T01:09:25.800171+0000","last_clean":"2024-03-19T01:09:25.800171+0000","last_became_active":"2024-03-19T01:07:19.689762+0000","last_became_peered":"2024-03-19T01:07:19.689762+0000","last_unstale":"2024-03-19T01:09:25.800171+0000","last_undegraded":"2024-03-19T01:09:25.800171+0000","last_fullsized":"2024-03-19T01:09:25.800171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:24:40.213171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798343+0000","last_change":"2024-03-19T01:07:14.473744+0000","last_active":"2024-03-19T01:09:25.798343+0000","last_peered":"2024-03-19T01:09:25.798343+0000","last_clean":"2024-03-19T01:09:25.798343+0000","last_became_active":"2024-03-19T01:07:14.473404+0000","last_became_peered":"2024-03-19T01:07:14.473404+0000","last_unstale":"2024-03-19T01:09:25.798343+0000","last_undegraded":"2024-03-19T01:09:25.798343+0000","last_fullsized":"2024-03-19T01:09:25.798343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:29:39.892678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154443+0000","last_change":"2024-03-19T01:07:15.481444+0000","last_active":"2024-03-19T01:09:26.154443+0000","last_peered":"2024-03-19T01:09:26.154443+0000","last_clean":"2024-03-19T01:09:26.154443+0000","last_became_active":"2024-03-19T01:07:15.481106+0000","last_became_peered":"2024-03-19T01:07:15.481106+0000","last_unstale":"2024-03-19T01:09:26.154443+0000","last_undegraded":"2024-03-19T01:09:26.154443+0000","last_fullsized":"2024-03-19T01:09:26.154443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:00.263063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801166+0000","last_change":"2024-03-19T01:07:19.686919+0000","last_active":"2024-03-19T01:09:25.801166+0000","last_peered":"2024-03-19T01:09:25.801166+0000","last_clean":"2024-03-19T01:09:25.801166+0000","last_became_active":"2024-03-19T01:07:19.686416+0000","last_became_peered":"2024-03-19T01:07:19.686416+0000","last_unstale":"2024-03-19T01:09:25.801166+0000","last_undegraded":"2024-03-19T01:09:25.801166+0000","last_fullsized":"2024-03-19T01:09:25.801166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:29:11.068957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'41","reported_seq":392,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.917526+0000","last_change":"2024-03-19T01:07:14.468630+0000","last_active":"2024-03-19T01:10:21.917526+0000","last_peered":"2024-03-19T01:10:21.917526+0000","last_clean":"2024-03-19T01:10:21.917526+0000","last_became_active":"2024-03-19T01:07:14.468293+0000","last_became_peered":"2024-03-19T01:07:14.468293+0000","last_unstale":"2024-03-19T01:10:21.917526+0000","last_undegraded":"2024-03-19T01:10:21.917526+0000","last_fullsized":"2024-03-19T01:10:21.917526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:53.575507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70405,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":319,"num_read_kb":230,"num_write":43,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137935+0000","last_change":"2024-03-19T01:07:15.477226+0000","last_active":"2024-03-19T01:09:26.137935+0000","last_peered":"2024-03-19T01:09:26.137935+0000","last_clean":"2024-03-19T01:09:26.137935+0000","last_became_active":"2024-03-19T01:07:15.476882+0000","last_became_peered":"2024-03-19T01:07:15.476882+0000","last_unstale":"2024-03-19T01:09:26.137935+0000","last_undegraded":"2024-03-19T01:09:26.137935+0000","last_fullsized":"2024-03-19T01:09:26.137935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:11:10.051837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798608+0000","last_change":"2024-03-19T01:07:19.690004+0000","last_active":"2024-03-19T01:09:25.798608+0000","last_peered":"2024-03-19T01:09:25.798608+0000","last_clean":"2024-03-19T01:09:25.798608+0000","last_became_active":"2024-03-19T01:07:19.689573+0000","last_became_peered":"2024-03-19T01:07:19.689573+0000","last_unstale":"2024-03-19T01:09:25.798608+0000","last_undegraded":"2024-03-19T01:09:25.798608+0000","last_fullsized":"2024-03-19T01:09:25.798608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:31:21.788219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914144+0000","last_change":"2024-03-19T01:07:14.487373+0000","last_active":"2024-03-19T01:09:25.914144+0000","last_peered":"2024-03-19T01:09:25.914144+0000","last_clean":"2024-03-19T01:09:25.914144+0000","last_became_active":"2024-03-19T01:07:14.487077+0000","last_became_peered":"2024-03-19T01:07:14.487077+0000","last_unstale":"2024-03-19T01:09:25.914144+0000","last_undegraded":"2024-03-19T01:09:25.914144+0000","last_fullsized":"2024-03-19T01:09:25.914144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:33:01.789431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.803723+0000","last_change":"2024-03-19T01:07:15.498009+0000","last_active":"2024-03-19T01:09:25.803723+0000","last_peered":"2024-03-19T01:09:25.803723+0000","last_clean":"2024-03-19T01:09:25.803723+0000","last_became_active":"2024-03-19T01:07:15.496707+0000","last_became_peered":"2024-03-19T01:07:15.496707+0000","last_unstale":"2024-03-19T01:09:25.803723+0000","last_undegraded":"2024-03-19T01:09:25.803723+0000","last_fullsized":"2024-03-19T01:09:25.803723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:50.058299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804327+0000","last_change":"2024-03-19T01:07:19.677635+0000","last_active":"2024-03-19T01:09:25.804327+0000","last_peered":"2024-03-19T01:09:25.804327+0000","last_clean":"2024-03-19T01:09:25.804327+0000","last_became_active":"2024-03-19T01:07:19.677207+0000","last_became_peered":"2024-03-19T01:07:19.677207+0000","last_unstale":"2024-03-19T01:09:25.804327+0000","last_undegraded":"2024-03-19T01:09:25.804327+0000","last_fullsized":"2024-03-19T01:09:25.804327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:23:28.650962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798247+0000","last_change":"2024-03-19T01:07:14.482054+0000","last_active":"2024-03-19T01:09:25.798247+0000","last_peered":"2024-03-19T01:09:25.798247+0000","last_clean":"2024-03-19T01:09:25.798247+0000","last_became_active":"2024-03-19T01:07:14.481703+0000","last_became_peered":"2024-03-19T01:07:14.481703+0000","last_unstale":"2024-03-19T01:09:25.798247+0000","last_undegraded":"2024-03-19T01:09:25.798247+0000","last_fullsized":"2024-03-19T01:09:25.798247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:02:43.565442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962388+0000","last_change":"2024-03-19T01:07:15.487236+0000","last_active":"2024-03-19T01:09:25.962388+0000","last_peered":"2024-03-19T01:09:25.962388+0000","last_clean":"2024-03-19T01:09:25.962388+0000","last_became_active":"2024-03-19T01:07:15.486931+0000","last_became_peered":"2024-03-19T01:07:15.486931+0000","last_unstale":"2024-03-19T01:09:25.962388+0000","last_undegraded":"2024-03-19T01:09:25.962388+0000","last_fullsized":"2024-03-19T01:09:25.962388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:22.201904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800840+0000","last_change":"2024-03-19T01:07:19.693813+0000","last_active":"2024-03-19T01:09:25.800840+0000","last_peered":"2024-03-19T01:09:25.800840+0000","last_clean":"2024-03-19T01:09:25.800840+0000","last_became_active":"2024-03-19T01:07:19.693384+0000","last_became_peered":"2024-03-19T01:07:19.693384+0000","last_unstale":"2024-03-19T01:09:25.800840+0000","last_undegraded":"2024-03-19T01:09:25.800840+0000","last_fullsized":"2024-03-19T01:09:25.800840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:09.372461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'9","reported_seq":202,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:22.006239+0000","last_change":"2024-03-19T01:07:14.468467+0000","last_active":"2024-03-19T01:10:22.006239+0000","last_peered":"2024-03-19T01:10:22.006239+0000","last_clean":"2024-03-19T01:10:22.006239+0000","last_became_active":"2024-03-19T01:07:14.468074+0000","last_became_peered":"2024-03-19T01:07:14.468074+0000","last_unstale":"2024-03-19T01:10:22.006239+0000","last_undegraded":"2024-03-19T01:10:22.006239+0000","last_fullsized":"2024-03-19T01:10:22.006239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:07:49.216998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":162,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962143+0000","last_change":"2024-03-19T01:07:15.475310+0000","last_active":"2024-03-19T01:09:25.962143+0000","last_peered":"2024-03-19T01:09:25.962143+0000","last_clean":"2024-03-19T01:09:25.962143+0000","last_became_active":"2024-03-19T01:07:15.474914+0000","last_became_peered":"2024-03-19T01:07:15.474914+0000","last_unstale":"2024-03-19T01:09:25.962143+0000","last_undegraded":"2024-03-19T01:09:25.962143+0000","last_fullsized":"2024-03-19T01:09:25.962143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:34.036892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804790+0000","last_change":"2024-03-19T01:07:19.697937+0000","last_active":"2024-03-19T01:09:25.804790+0000","last_peered":"2024-03-19T01:09:25.804790+0000","last_clean":"2024-03-19T01:09:25.804790+0000","last_became_active":"2024-03-19T01:07:19.697447+0000","last_became_peered":"2024-03-19T01:07:19.697447+0000","last_unstale":"2024-03-19T01:09:25.804790+0000","last_undegraded":"2024-03-19T01:09:25.804790+0000","last_fullsized":"2024-03-19T01:09:25.804790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:49.259118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137706+0000","last_change":"2024-03-19T01:07:14.476382+0000","last_active":"2024-03-19T01:09:26.137706+0000","last_peered":"2024-03-19T01:09:26.137706+0000","last_clean":"2024-03-19T01:09:26.137706+0000","last_became_active":"2024-03-19T01:07:14.476089+0000","last_became_peered":"2024-03-19T01:07:14.476089+0000","last_unstale":"2024-03-19T01:09:26.137706+0000","last_undegraded":"2024-03-19T01:09:26.137706+0000","last_fullsized":"2024-03-19T01:09:26.137706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:57:42.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800778+0000","last_change":"2024-03-19T01:08:24.794335+0000","last_active":"2024-03-19T01:09:25.800778+0000","last_peered":"2024-03-19T01:09:25.800778+0000","last_clean":"2024-03-19T01:09:25.800778+0000","last_became_active":"2024-03-19T01:08:24.793725+0000","last_became_peered":"2024-03-19T01:08:24.793725+0000","last_unstale":"2024-03-19T01:09:25.800778+0000","last_undegraded":"2024-03-19T01:09:25.800778+0000","last_fullsized":"2024-03-19T01:09:25.800778+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:18.797824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798009+0000","last_change":"2024-03-19T01:07:19.688235+0000","last_active":"2024-03-19T01:09:25.798009+0000","last_peered":"2024-03-19T01:09:25.798009+0000","last_clean":"2024-03-19T01:09:25.798009+0000","last_became_active":"2024-03-19T01:07:19.687879+0000","last_became_peered":"2024-03-19T01:07:19.687879+0000","last_unstale":"2024-03-19T01:09:25.798009+0000","last_undegraded":"2024-03-19T01:09:25.798009+0000","last_fullsized":"2024-03-19T01:09:25.798009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:29.465971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:33.028596+0000","last_change":"2024-03-19T01:07:14.474346+0000","last_active":"2024-03-19T01:09:33.028596+0000","last_peered":"2024-03-19T01:09:33.028596+0000","last_clean":"2024-03-19T01:09:33.028596+0000","last_became_active":"2024-03-19T01:07:14.473989+0000","last_became_peered":"2024-03-19T01:07:14.473989+0000","last_unstale":"2024-03-19T01:09:33.028596+0000","last_undegraded":"2024-03-19T01:09:33.028596+0000","last_fullsized":"2024-03-19T01:09:33.028596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:41.004155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804461+0000","last_change":"2024-03-19T01:07:15.482326+0000","last_active":"2024-03-19T01:09:25.804461+0000","last_peered":"2024-03-19T01:09:25.804461+0000","last_clean":"2024-03-19T01:09:25.804461+0000","last_became_active":"2024-03-19T01:07:15.481492+0000","last_became_peered":"2024-03-19T01:07:15.481492+0000","last_unstale":"2024-03-19T01:09:25.804461+0000","last_undegraded":"2024-03-19T01:09:25.804461+0000","last_fullsized":"2024-03-19T01:09:25.804461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:17:31.572104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137621+0000","last_change":"2024-03-19T01:07:19.668853+0000","last_active":"2024-03-19T01:09:26.137621+0000","last_peered":"2024-03-19T01:09:26.137621+0000","last_clean":"2024-03-19T01:09:26.137621+0000","last_became_active":"2024-03-19T01:07:19.668428+0000","last_became_peered":"2024-03-19T01:07:19.668428+0000","last_unstale":"2024-03-19T01:09:26.137621+0000","last_undegraded":"2024-03-19T01:09:26.137621+0000","last_fullsized":"2024-03-19T01:09:26.137621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:50:43.409586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801707+0000","last_change":"2024-03-19T01:07:14.469548+0000","last_active":"2024-03-19T01:09:25.801707+0000","last_peered":"2024-03-19T01:09:25.801707+0000","last_clean":"2024-03-19T01:09:25.801707+0000","last_became_active":"2024-03-19T01:07:14.469200+0000","last_became_peered":"2024-03-19T01:07:14.469200+0000","last_unstale":"2024-03-19T01:09:25.801707+0000","last_undegraded":"2024-03-19T01:09:25.801707+0000","last_fullsized":"2024-03-19T01:09:25.801707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:41:53.179343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801883+0000","last_change":"2024-03-19T01:07:15.495300+0000","last_active":"2024-03-19T01:09:25.801883+0000","last_peered":"2024-03-19T01:09:25.801883+0000","last_clean":"2024-03-19T01:09:25.801883+0000","last_became_active":"2024-03-19T01:07:15.494938+0000","last_became_peered":"2024-03-19T01:07:15.494938+0000","last_unstale":"2024-03-19T01:09:25.801883+0000","last_undegraded":"2024-03-19T01:09:25.801883+0000","last_fullsized":"2024-03-19T01:09:25.801883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:50:38.855857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804281+0000","last_change":"2024-03-19T01:07:19.693261+0000","last_active":"2024-03-19T01:09:25.804281+0000","last_peered":"2024-03-19T01:09:25.804281+0000","last_clean":"2024-03-19T01:09:25.804281+0000","last_became_active":"2024-03-19T01:07:19.692598+0000","last_became_peered":"2024-03-19T01:07:19.692598+0000","last_unstale":"2024-03-19T01:09:25.804281+0000","last_undegraded":"2024-03-19T01:09:25.804281+0000","last_fullsized":"2024-03-19T01:09:25.804281+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:43:29.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":192,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.039962+0000","last_change":"2024-03-19T01:07:14.481569+0000","last_active":"2024-03-19T01:10:21.039962+0000","last_peered":"2024-03-19T01:10:21.039962+0000","last_clean":"2024-03-19T01:10:21.039962+0000","last_became_active":"2024-03-19T01:07:14.481209+0000","last_became_peered":"2024-03-19T01:07:14.481209+0000","last_unstale":"2024-03-19T01:10:21.039962+0000","last_undegraded":"2024-03-19T01:10:21.039962+0000","last_fullsized":"2024-03-19T01:10:21.039962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:39.411207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":154,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801654+0000","last_change":"2024-03-19T01:07:15.496852+0000","last_active":"2024-03-19T01:09:25.801654+0000","last_peered":"2024-03-19T01:09:25.801654+0000","last_clean":"2024-03-19T01:09:25.801654+0000","last_became_active":"2024-03-19T01:07:15.496455+0000","last_became_peered":"2024-03-19T01:07:15.496455+0000","last_unstale":"2024-03-19T01:09:25.801654+0000","last_undegraded":"2024-03-19T01:09:25.801654+0000","last_fullsized":"2024-03-19T01:09:25.801654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:43.419496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.805138+0000","last_change":"2024-03-19T01:07:19.692856+0000","last_active":"2024-03-19T01:09:25.805138+0000","last_peered":"2024-03-19T01:09:25.805138+0000","last_clean":"2024-03-19T01:09:25.805138+0000","last_became_active":"2024-03-19T01:07:19.690712+0000","last_became_peered":"2024-03-19T01:07:19.690712+0000","last_unstale":"2024-03-19T01:09:25.805138+0000","last_undegraded":"2024-03-19T01:09:25.805138+0000","last_fullsized":"2024-03-19T01:09:25.805138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:29:28.473062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798036+0000","last_change":"2024-03-19T01:07:15.474375+0000","last_active":"2024-03-19T01:09:25.798036+0000","last_peered":"2024-03-19T01:09:25.798036+0000","last_clean":"2024-03-19T01:09:25.798036+0000","last_became_active":"2024-03-19T01:07:15.473427+0000","last_became_peered":"2024-03-19T01:07:15.473427+0000","last_unstale":"2024-03-19T01:09:25.798036+0000","last_undegraded":"2024-03-19T01:09:25.798036+0000","last_fullsized":"2024-03-19T01:09:25.798036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:42:22.235054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'7","reported_seq":194,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.004294+0000","last_change":"2024-03-19T01:07:14.490924+0000","last_active":"2024-03-19T01:10:21.004294+0000","last_peered":"2024-03-19T01:10:21.004294+0000","last_clean":"2024-03-19T01:10:21.004294+0000","last_became_active":"2024-03-19T01:07:14.490621+0000","last_became_peered":"2024-03-19T01:07:14.490621+0000","last_unstale":"2024-03-19T01:10:21.004294+0000","last_undegraded":"2024-03-19T01:10:21.004294+0000","last_fullsized":"2024-03-19T01:10:21.004294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:02:52.254138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":155,"num_read_kb":155,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962466+0000","last_change":"2024-03-19T01:08:24.805196+0000","last_active":"2024-03-19T01:09:25.962466+0000","last_peered":"2024-03-19T01:09:25.962466+0000","last_clean":"2024-03-19T01:09:25.962466+0000","last_became_active":"2024-03-19T01:08:24.800225+0000","last_became_peered":"2024-03-19T01:08:24.800225+0000","last_unstale":"2024-03-19T01:09:25.962466+0000","last_undegraded":"2024-03-19T01:09:25.962466+0000","last_fullsized":"2024-03-19T01:09:25.962466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:02.322846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914787+0000","last_change":"2024-03-19T01:07:15.493218+0000","last_active":"2024-03-19T01:09:25.914787+0000","last_peered":"2024-03-19T01:09:25.914787+0000","last_clean":"2024-03-19T01:09:25.914787+0000","last_became_active":"2024-03-19T01:07:15.492751+0000","last_became_peered":"2024-03-19T01:07:15.492751+0000","last_unstale":"2024-03-19T01:09:25.914787+0000","last_undegraded":"2024-03-19T01:09:25.914787+0000","last_fullsized":"2024-03-19T01:09:25.914787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:48:32.105214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800099+0000","last_change":"2024-03-19T01:07:14.473349+0000","last_active":"2024-03-19T01:09:25.800099+0000","last_peered":"2024-03-19T01:09:25.800099+0000","last_clean":"2024-03-19T01:09:25.800099+0000","last_became_active":"2024-03-19T01:07:14.473014+0000","last_became_peered":"2024-03-19T01:07:14.473014+0000","last_unstale":"2024-03-19T01:09:25.800099+0000","last_undegraded":"2024-03-19T01:09:25.800099+0000","last_fullsized":"2024-03-19T01:09:25.800099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:27:34.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.961963+0000","last_change":"2024-03-19T01:05:56.013594+0000","last_active":"2024-03-19T01:09:25.961963+0000","last_peered":"2024-03-19T01:09:25.961963+0000","last_clean":"2024-03-19T01:09:25.961963+0000","last_became_active":"2024-03-19T01:05:56.013084+0000","last_became_peered":"2024-03-19T01:05:56.013084+0000","last_unstale":"2024-03-19T01:09:25.961963+0000","last_undegraded":"2024-03-19T01:09:25.961963+0000","last_fullsized":"2024-03-19T01:09:25.961963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_clean_scrub_stamp":"2024-03-19T01:02:06.532343+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:23.440161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798358+0000","last_change":"2024-03-19T01:07:19.691111+0000","last_active":"2024-03-19T01:09:25.798358+0000","last_peered":"2024-03-19T01:09:25.798358+0000","last_clean":"2024-03-19T01:09:25.798358+0000","last_became_active":"2024-03-19T01:07:19.690618+0000","last_became_peered":"2024-03-19T01:07:19.690618+0000","last_unstale":"2024-03-19T01:09:25.798358+0000","last_undegraded":"2024-03-19T01:09:25.798358+0000","last_fullsized":"2024-03-19T01:09:25.798358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:17.857394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.802402+0000","last_change":"2024-03-19T01:08:24.801826+0000","last_active":"2024-03-19T01:09:25.802402+0000","last_peered":"2024-03-19T01:09:25.802402+0000","last_clean":"2024-03-19T01:09:25.802402+0000","last_became_active":"2024-03-19T01:08:24.801411+0000","last_became_peered":"2024-03-19T01:08:24.801411+0000","last_unstale":"2024-03-19T01:09:25.802402+0000","last_undegraded":"2024-03-19T01:09:25.802402+0000","last_fullsized":"2024-03-19T01:09:25.802402+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:31.798045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962068+0000","last_change":"2024-03-19T01:07:14.474287+0000","last_active":"2024-03-19T01:09:25.962068+0000","last_peered":"2024-03-19T01:09:25.962068+0000","last_clean":"2024-03-19T01:09:25.962068+0000","last_became_active":"2024-03-19T01:07:14.473921+0000","last_became_peered":"2024-03-19T01:07:14.473921+0000","last_unstale":"2024-03-19T01:09:25.962068+0000","last_undegraded":"2024-03-19T01:09:25.962068+0000","last_fullsized":"2024-03-19T01:09:25.962068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:49.322029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804258+0000","last_change":"2024-03-19T01:07:19.685331+0000","last_active":"2024-03-19T01:09:25.804258+0000","last_peered":"2024-03-19T01:09:25.804258+0000","last_clean":"2024-03-19T01:09:25.804258+0000","last_became_active":"2024-03-19T01:07:19.684876+0000","last_became_peered":"2024-03-19T01:07:19.684876+0000","last_unstale":"2024-03-19T01:09:25.804258+0000","last_undegraded":"2024-03-19T01:09:25.804258+0000","last_fullsized":"2024-03-19T01:09:25.804258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:39.575348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137876+0000","last_change":"2024-03-19T01:07:14.481019+0000","last_active":"2024-03-19T01:09:26.137876+0000","last_peered":"2024-03-19T01:09:26.137876+0000","last_clean":"2024-03-19T01:09:26.137876+0000","last_became_active":"2024-03-19T01:07:14.480666+0000","last_became_peered":"2024-03-19T01:07:14.480666+0000","last_unstale":"2024-03-19T01:09:26.137876+0000","last_undegraded":"2024-03-19T01:09:26.137876+0000","last_fullsized":"2024-03-19T01:09:26.137876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:13:33.669690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804528+0000","last_change":"2024-03-19T01:07:15.490747+0000","last_active":"2024-03-19T01:09:25.804528+0000","last_peered":"2024-03-19T01:09:25.804528+0000","last_clean":"2024-03-19T01:09:25.804528+0000","last_became_active":"2024-03-19T01:07:15.486244+0000","last_became_peered":"2024-03-19T01:07:15.486244+0000","last_unstale":"2024-03-19T01:09:25.804528+0000","last_undegraded":"2024-03-19T01:09:25.804528+0000","last_fullsized":"2024-03-19T01:09:25.804528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:43.212972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914397+0000","last_change":"2024-03-19T01:07:19.684804+0000","last_active":"2024-03-19T01:09:25.914397+0000","last_peered":"2024-03-19T01:09:25.914397+0000","last_clean":"2024-03-19T01:09:25.914397+0000","last_became_active":"2024-03-19T01:07:19.683191+0000","last_became_peered":"2024-03-19T01:07:19.683191+0000","last_unstale":"2024-03-19T01:09:25.914397+0000","last_undegraded":"2024-03-19T01:09:25.914397+0000","last_fullsized":"2024-03-19T01:09:25.914397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:02.015499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'16","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:16.639078+0000","last_change":"2024-03-19T01:07:14.479282+0000","last_active":"2024-03-19T01:10:16.639078+0000","last_peered":"2024-03-19T01:10:16.639078+0000","last_clean":"2024-03-19T01:10:16.639078+0000","last_became_active":"2024-03-19T01:07:14.478876+0000","last_became_peered":"2024-03-19T01:07:14.478876+0000","last_unstale":"2024-03-19T01:10:16.639078+0000","last_undegraded":"2024-03-19T01:10:16.639078+0000","last_fullsized":"2024-03-19T01:10:16.639078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:54.179680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915652+0000","last_change":"2024-03-19T01:07:15.493498+0000","last_active":"2024-03-19T01:09:25.915652+0000","last_peered":"2024-03-19T01:09:25.915652+0000","last_clean":"2024-03-19T01:09:25.915652+0000","last_became_active":"2024-03-19T01:07:15.493110+0000","last_became_peered":"2024-03-19T01:07:15.493110+0000","last_unstale":"2024-03-19T01:09:25.915652+0000","last_undegraded":"2024-03-19T01:09:25.915652+0000","last_fullsized":"2024-03-19T01:09:25.915652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:55:23.608596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.913805+0000","last_change":"2024-03-19T01:07:19.671232+0000","last_active":"2024-03-19T01:09:25.913805+0000","last_peered":"2024-03-19T01:09:25.913805+0000","last_clean":"2024-03-19T01:09:25.913805+0000","last_became_active":"2024-03-19T01:07:19.670818+0000","last_became_peered":"2024-03-19T01:07:19.670818+0000","last_unstale":"2024-03-19T01:09:25.913805+0000","last_undegraded":"2024-03-19T01:09:25.913805+0000","last_fullsized":"2024-03-19T01:09:25.913805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:57:20.134372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800619+0000","last_change":"2024-03-19T01:07:14.473568+0000","last_active":"2024-03-19T01:09:25.800619+0000","last_peered":"2024-03-19T01:09:25.800619+0000","last_clean":"2024-03-19T01:09:25.800619+0000","last_became_active":"2024-03-19T01:07:14.472555+0000","last_became_peered":"2024-03-19T01:07:14.472555+0000","last_unstale":"2024-03-19T01:09:25.800619+0000","last_undegraded":"2024-03-19T01:09:25.800619+0000","last_fullsized":"2024-03-19T01:09:25.800619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:28:01.541228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156691+0000","last_change":"2024-03-19T01:07:15.499905+0000","last_active":"2024-03-19T01:09:26.156691+0000","last_peered":"2024-03-19T01:09:26.156691+0000","last_clean":"2024-03-19T01:09:26.156691+0000","last_became_active":"2024-03-19T01:07:15.499560+0000","last_became_peered":"2024-03-19T01:07:15.499560+0000","last_unstale":"2024-03-19T01:09:26.156691+0000","last_undegraded":"2024-03-19T01:09:26.156691+0000","last_fullsized":"2024-03-19T01:09:26.156691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:06:26.531100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800426+0000","last_change":"2024-03-19T01:07:19.690310+0000","last_active":"2024-03-19T01:09:25.800426+0000","last_peered":"2024-03-19T01:09:25.800426+0000","last_clean":"2024-03-19T01:09:25.800426+0000","last_became_active":"2024-03-19T01:07:19.689877+0000","last_became_peered":"2024-03-19T01:07:19.689877+0000","last_unstale":"2024-03-19T01:09:25.800426+0000","last_undegraded":"2024-03-19T01:09:25.800426+0000","last_fullsized":"2024-03-19T01:09:25.800426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:06:12.730258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154081+0000","last_change":"2024-03-19T01:07:14.466267+0000","last_active":"2024-03-19T01:09:26.154081+0000","last_peered":"2024-03-19T01:09:26.154081+0000","last_clean":"2024-03-19T01:09:26.154081+0000","last_became_active":"2024-03-19T01:07:14.465944+0000","last_became_peered":"2024-03-19T01:07:14.465944+0000","last_unstale":"2024-03-19T01:09:26.154081+0000","last_undegraded":"2024-03-19T01:09:26.154081+0000","last_fullsized":"2024-03-19T01:09:26.154081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:40.957200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137477+0000","last_change":"2024-03-19T01:07:15.475690+0000","last_active":"2024-03-19T01:09:26.137477+0000","last_peered":"2024-03-19T01:09:26.137477+0000","last_clean":"2024-03-19T01:09:26.137477+0000","last_became_active":"2024-03-19T01:07:15.475342+0000","last_became_peered":"2024-03-19T01:07:15.475342+0000","last_unstale":"2024-03-19T01:09:26.137477+0000","last_undegraded":"2024-03-19T01:09:26.137477+0000","last_fullsized":"2024-03-19T01:09:26.137477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:30:30.842462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137994+0000","last_change":"2024-03-19T01:08:24.797680+0000","last_active":"2024-03-19T01:09:26.137994+0000","last_peered":"2024-03-19T01:09:26.137994+0000","last_clean":"2024-03-19T01:09:26.137994+0000","last_became_active":"2024-03-19T01:08:24.797194+0000","last_became_peered":"2024-03-19T01:08:24.797194+0000","last_unstale":"2024-03-19T01:09:26.137994+0000","last_undegraded":"2024-03-19T01:09:26.137994+0000","last_fullsized":"2024-03-19T01:09:26.137994+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:49.800726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914650+0000","last_change":"2024-03-19T01:07:14.487907+0000","last_active":"2024-03-19T01:09:25.914650+0000","last_peered":"2024-03-19T01:09:25.914650+0000","last_clean":"2024-03-19T01:09:25.914650+0000","last_became_active":"2024-03-19T01:07:14.487618+0000","last_became_peered":"2024-03-19T01:07:14.487618+0000","last_unstale":"2024-03-19T01:09:25.914650+0000","last_undegraded":"2024-03-19T01:09:25.914650+0000","last_fullsized":"2024-03-19T01:09:25.914650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:22:49.218797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804225+0000","last_change":"2024-03-19T01:07:15.492103+0000","last_active":"2024-03-19T01:09:25.804225+0000","last_peered":"2024-03-19T01:09:25.804225+0000","last_clean":"2024-03-19T01:09:25.804225+0000","last_became_active":"2024-03-19T01:07:15.491712+0000","last_became_peered":"2024-03-19T01:07:15.491712+0000","last_unstale":"2024-03-19T01:09:25.804225+0000","last_undegraded":"2024-03-19T01:09:25.804225+0000","last_fullsized":"2024-03-19T01:09:25.804225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:40:13.959986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.799962+0000","last_change":"2024-03-19T01:07:19.677392+0000","last_active":"2024-03-19T01:09:25.799962+0000","last_peered":"2024-03-19T01:09:25.799962+0000","last_clean":"2024-03-19T01:09:25.799962+0000","last_became_active":"2024-03-19T01:07:19.677024+0000","last_became_peered":"2024-03-19T01:07:19.677024+0000","last_unstale":"2024-03-19T01:09:25.799962+0000","last_undegraded":"2024-03-19T01:09:25.799962+0000","last_fullsized":"2024-03-19T01:09:25.799962+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:01:43.404902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804247+0000","last_change":"2024-03-19T01:08:24.805363+0000","last_active":"2024-03-19T01:09:25.804247+0000","last_peered":"2024-03-19T01:09:25.804247+0000","last_clean":"2024-03-19T01:09:25.804247+0000","last_became_active":"2024-03-19T01:08:24.800810+0000","last_became_peered":"2024-03-19T01:08:24.800810+0000","last_unstale":"2024-03-19T01:09:25.804247+0000","last_undegraded":"2024-03-19T01:09:25.804247+0000","last_fullsized":"2024-03-19T01:09:25.804247+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:32:43.608140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915314+0000","last_change":"2024-03-19T01:07:15.505842+0000","last_active":"2024-03-19T01:09:25.915314+0000","last_peered":"2024-03-19T01:09:25.915314+0000","last_clean":"2024-03-19T01:09:25.915314+0000","last_became_active":"2024-03-19T01:07:15.505425+0000","last_became_peered":"2024-03-19T01:07:15.505425+0000","last_unstale":"2024-03-19T01:09:25.915314+0000","last_undegraded":"2024-03-19T01:09:25.915314+0000","last_fullsized":"2024-03-19T01:09:25.915314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:28.119864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800240+0000","last_change":"2024-03-19T01:07:19.683113+0000","last_active":"2024-03-19T01:09:25.800240+0000","last_peered":"2024-03-19T01:09:25.800240+0000","last_clean":"2024-03-19T01:09:25.800240+0000","last_became_active":"2024-03-19T01:07:19.682702+0000","last_became_peered":"2024-03-19T01:07:19.682702+0000","last_unstale":"2024-03-19T01:09:25.800240+0000","last_undegraded":"2024-03-19T01:09:25.800240+0000","last_fullsized":"2024-03-19T01:09:25.800240+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:42.866408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137616+0000","last_change":"2024-03-19T01:07:14.476562+0000","last_active":"2024-03-19T01:09:26.137616+0000","last_peered":"2024-03-19T01:09:26.137616+0000","last_clean":"2024-03-19T01:09:26.137616+0000","last_became_active":"2024-03-19T01:07:14.476185+0000","last_became_peered":"2024-03-19T01:07:14.476185+0000","last_unstale":"2024-03-19T01:09:26.137616+0000","last_undegraded":"2024-03-19T01:09:26.137616+0000","last_fullsized":"2024-03-19T01:09:26.137616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:35:19.860443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962373+0000","last_change":"2024-03-19T01:07:15.487163+0000","last_active":"2024-03-19T01:09:25.962373+0000","last_peered":"2024-03-19T01:09:25.962373+0000","last_clean":"2024-03-19T01:09:25.962373+0000","last_became_active":"2024-03-19T01:07:15.486845+0000","last_became_peered":"2024-03-19T01:07:15.486845+0000","last_unstale":"2024-03-19T01:09:25.962373+0000","last_undegraded":"2024-03-19T01:09:25.962373+0000","last_fullsized":"2024-03-19T01:09:25.962373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:39:09.389411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154143+0000","last_change":"2024-03-19T01:07:19.692768+0000","last_active":"2024-03-19T01:09:26.154143+0000","last_peered":"2024-03-19T01:09:26.154143+0000","last_clean":"2024-03-19T01:09:26.154143+0000","last_became_active":"2024-03-19T01:07:19.692382+0000","last_became_peered":"2024-03-19T01:07:19.692382+0000","last_unstale":"2024-03-19T01:09:26.154143+0000","last_undegraded":"2024-03-19T01:09:26.154143+0000","last_fullsized":"2024-03-19T01:09:26.154143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:26.804938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":189,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:18.964737+0000","last_change":"2024-03-19T01:07:14.472844+0000","last_active":"2024-03-19T01:10:18.964737+0000","last_peered":"2024-03-19T01:10:18.964737+0000","last_clean":"2024-03-19T01:10:18.964737+0000","last_became_active":"2024-03-19T01:07:14.472389+0000","last_became_peered":"2024-03-19T01:07:14.472389+0000","last_unstale":"2024-03-19T01:10:18.964737+0000","last_undegraded":"2024-03-19T01:10:18.964737+0000","last_fullsized":"2024-03-19T01:10:18.964737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:31.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962639+0000","last_change":"2024-03-19T01:07:15.482033+0000","last_active":"2024-03-19T01:09:25.962639+0000","last_peered":"2024-03-19T01:09:25.962639+0000","last_clean":"2024-03-19T01:09:25.962639+0000","last_became_active":"2024-03-19T01:07:15.481671+0000","last_became_peered":"2024-03-19T01:07:15.481671+0000","last_unstale":"2024-03-19T01:09:25.962639+0000","last_undegraded":"2024-03-19T01:09:25.962639+0000","last_fullsized":"2024-03-19T01:09:25.962639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:09.969374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.155872+0000","last_change":"2024-03-19T01:08:24.793574+0000","last_active":"2024-03-19T01:09:26.155872+0000","last_peered":"2024-03-19T01:09:26.155872+0000","last_clean":"2024-03-19T01:09:26.155872+0000","last_became_active":"2024-03-19T01:08:24.793012+0000","last_became_peered":"2024-03-19T01:08:24.793012+0000","last_unstale":"2024-03-19T01:09:26.155872+0000","last_undegraded":"2024-03-19T01:09:26.155872+0000","last_fullsized":"2024-03-19T01:09:26.155872+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:04:53.611271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798450+0000","last_change":"2024-03-19T01:07:14.481677+0000","last_active":"2024-03-19T01:09:25.798450+0000","last_peered":"2024-03-19T01:09:25.798450+0000","last_clean":"2024-03-19T01:09:25.798450+0000","last_became_active":"2024-03-19T01:07:14.481138+0000","last_became_peered":"2024-03-19T01:07:14.481138+0000","last_unstale":"2024-03-19T01:09:25.798450+0000","last_undegraded":"2024-03-19T01:09:25.798450+0000","last_fullsized":"2024-03-19T01:09:25.798450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:52:33.105816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962431+0000","last_change":"2024-03-19T01:07:15.498352+0000","last_active":"2024-03-19T01:09:25.962431+0000","last_peered":"2024-03-19T01:09:25.962431+0000","last_clean":"2024-03-19T01:09:25.962431+0000","last_became_active":"2024-03-19T01:07:15.498040+0000","last_became_peered":"2024-03-19T01:07:15.498040+0000","last_unstale":"2024-03-19T01:09:25.962431+0000","last_undegraded":"2024-03-19T01:09:25.962431+0000","last_fullsized":"2024-03-19T01:09:25.962431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:57:37.571753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.797775+0000","last_change":"2024-03-19T01:07:19.673291+0000","last_active":"2024-03-19T01:09:25.797775+0000","last_peered":"2024-03-19T01:09:25.797775+0000","last_clean":"2024-03-19T01:09:25.797775+0000","last_became_active":"2024-03-19T01:07:19.672704+0000","last_became_peered":"2024-03-19T01:07:19.672704+0000","last_unstale":"2024-03-19T01:09:25.797775+0000","last_undegraded":"2024-03-19T01:09:25.797775+0000","last_fullsized":"2024-03-19T01:09:25.797775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:01:42.019812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804917+0000","last_change":"2024-03-19T01:07:14.489556+0000","last_active":"2024-03-19T01:09:25.804917+0000","last_peered":"2024-03-19T01:09:25.804917+0000","last_clean":"2024-03-19T01:09:25.804917+0000","last_became_active":"2024-03-19T01:07:14.489201+0000","last_became_peered":"2024-03-19T01:07:14.489201+0000","last_unstale":"2024-03-19T01:09:25.804917+0000","last_undegraded":"2024-03-19T01:09:25.804917+0000","last_fullsized":"2024-03-19T01:09:25.804917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:28:36.516185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962688+0000","last_change":"2024-03-19T01:07:15.477727+0000","last_active":"2024-03-19T01:09:25.962688+0000","last_peered":"2024-03-19T01:09:25.962688+0000","last_clean":"2024-03-19T01:09:25.962688+0000","last_became_active":"2024-03-19T01:07:15.477388+0000","last_became_peered":"2024-03-19T01:07:15.477388+0000","last_unstale":"2024-03-19T01:09:25.962688+0000","last_undegraded":"2024-03-19T01:09:25.962688+0000","last_fullsized":"2024-03-19T01:09:25.962688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:30.567326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804232+0000","last_change":"2024-03-19T01:07:15.479008+0000","last_active":"2024-03-19T01:09:25.804232+0000","last_peered":"2024-03-19T01:09:25.804232+0000","last_clean":"2024-03-19T01:09:25.804232+0000","last_became_active":"2024-03-19T01:07:15.478657+0000","last_became_peered":"2024-03-19T01:07:15.478657+0000","last_unstale":"2024-03-19T01:09:25.804232+0000","last_undegraded":"2024-03-19T01:09:25.804232+0000","last_fullsized":"2024-03-19T01:09:25.804232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:42:46.970012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914058+0000","last_change":"2024-03-19T01:08:24.789796+0000","last_active":"2024-03-19T01:09:25.914058+0000","last_peered":"2024-03-19T01:09:25.914058+0000","last_clean":"2024-03-19T01:09:25.914058+0000","last_became_active":"2024-03-19T01:08:24.789444+0000","last_became_peered":"2024-03-19T01:08:24.789444+0000","last_unstale":"2024-03-19T01:09:25.914058+0000","last_undegraded":"2024-03-19T01:09:25.914058+0000","last_fullsized":"2024-03-19T01:09:25.914058+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:41.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.913879+0000","last_change":"2024-03-19T01:07:15.495292+0000","last_active":"2024-03-19T01:09:25.913879+0000","last_peered":"2024-03-19T01:09:25.913879+0000","last_clean":"2024-03-19T01:09:25.913879+0000","last_became_active":"2024-03-19T01:07:15.494946+0000","last_became_peered":"2024-03-19T01:07:15.494946+0000","last_unstale":"2024-03-19T01:09:25.913879+0000","last_undegraded":"2024-03-19T01:09:25.913879+0000","last_fullsized":"2024-03-19T01:09:25.913879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:39.012221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156089+0000","last_change":"2024-03-19T01:07:15.499816+0000","last_active":"2024-03-19T01:09:26.156089+0000","last_peered":"2024-03-19T01:09:26.156089+0000","last_clean":"2024-03-19T01:09:26.156089+0000","last_became_active":"2024-03-19T01:07:15.499031+0000","last_became_peered":"2024-03-19T01:07:15.499031+0000","last_unstale":"2024-03-19T01:09:26.156089+0000","last_undegraded":"2024-03-19T01:09:26.156089+0000","last_fullsized":"2024-03-19T01:09:26.156089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:34:25.344092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804473+0000","last_change":"2024-03-19T01:07:15.501526+0000","last_active":"2024-03-19T01:09:25.804473+0000","last_peered":"2024-03-19T01:09:25.804473+0000","last_clean":"2024-03-19T01:09:25.804473+0000","last_became_active":"2024-03-19T01:07:15.500452+0000","last_became_peered":"2024-03-19T01:07:15.500452+0000","last_unstale":"2024-03-19T01:09:25.804473+0000","last_undegraded":"2024-03-19T01:09:25.804473+0000","last_fullsized":"2024-03-19T01:09:25.804473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:35:33.075780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804632+0000","last_change":"2024-03-19T01:07:15.477649+0000","last_active":"2024-03-19T01:09:25.804632+0000","last_peered":"2024-03-19T01:09:25.804632+0000","last_clean":"2024-03-19T01:09:25.804632+0000","last_became_active":"2024-03-19T01:07:15.477193+0000","last_became_peered":"2024-03-19T01:07:15.477193+0000","last_unstale":"2024-03-19T01:09:25.804632+0000","last_undegraded":"2024-03-19T01:09:25.804632+0000","last_fullsized":"2024-03-19T01:09:25.804632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:36.126642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914305+0000","last_change":"2024-03-19T01:07:15.481920+0000","last_active":"2024-03-19T01:09:25.914305+0000","last_peered":"2024-03-19T01:09:25.914305+0000","last_clean":"2024-03-19T01:09:25.914305+0000","last_became_active":"2024-03-19T01:07:15.481290+0000","last_became_peered":"2024-03-19T01:07:15.481290+0000","last_unstale":"2024-03-19T01:09:25.914305+0000","last_undegraded":"2024-03-19T01:09:25.914305+0000","last_fullsized":"2024-03-19T01:09:25.914305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:37:58.028647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915760+0000","last_change":"2024-03-19T01:07:15.502305+0000","last_active":"2024-03-19T01:09:25.915760+0000","last_peered":"2024-03-19T01:09:25.915760+0000","last_clean":"2024-03-19T01:09:25.915760+0000","last_became_active":"2024-03-19T01:07:15.501491+0000","last_became_peered":"2024-03-19T01:07:15.501491+0000","last_unstale":"2024-03-19T01:09:25.915760+0000","last_undegraded":"2024-03-19T01:09:25.915760+0000","last_fullsized":"2024-03-19T01:09:25.915760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:01.999538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.139189+0000","last_change":"2024-03-19T01:09:26.138802+0000","last_active":"2024-03-19T01:09:26.139189+0000","last_peered":"2024-03-19T01:09:26.139189+0000","last_clean":"2024-03-19T01:09:26.139189+0000","last_became_active":"2024-03-19T01:09:26.138423+0000","last_became_peered":"2024-03-19T01:09:26.138423+0000","last_unstale":"2024-03-19T01:09:26.139189+0000","last_undegraded":"2024-03-19T01:09:26.139189+0000","last_fullsized":"2024-03-19T01:09:26.139189+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:51:23.293922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914829+0000","last_change":"2024-03-19T01:07:15.505022+0000","last_active":"2024-03-19T01:09:25.914829+0000","last_peered":"2024-03-19T01:09:25.914829+0000","last_clean":"2024-03-19T01:09:25.914829+0000","last_became_active":"2024-03-19T01:07:15.503684+0000","last_became_peered":"2024-03-19T01:07:15.503684+0000","last_unstale":"2024-03-19T01:09:25.914829+0000","last_undegraded":"2024-03-19T01:09:25.914829+0000","last_fullsized":"2024-03-19T01:09:25.914829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:56:47.580267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154341+0000","last_change":"2024-03-19T01:07:15.495634+0000","last_active":"2024-03-19T01:09:26.154341+0000","last_peered":"2024-03-19T01:09:26.154341+0000","last_clean":"2024-03-19T01:09:26.154341+0000","last_became_active":"2024-03-19T01:07:15.495206+0000","last_became_peered":"2024-03-19T01:07:15.495206+0000","last_unstale":"2024-03-19T01:09:26.154341+0000","last_undegraded":"2024-03-19T01:09:26.154341+0000","last_fullsized":"2024-03-19T01:09:26.154341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:30:29.965731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804363+0000","last_change":"2024-03-19T01:07:15.501480+0000","last_active":"2024-03-19T01:09:25.804363+0000","last_peered":"2024-03-19T01:09:25.804363+0000","last_clean":"2024-03-19T01:09:25.804363+0000","last_became_active":"2024-03-19T01:07:15.501078+0000","last_became_peered":"2024-03-19T01:07:15.501078+0000","last_unstale":"2024-03-19T01:09:25.804363+0000","last_undegraded":"2024-03-19T01:09:25.804363+0000","last_fullsized":"2024-03-19T01:09:25.804363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:40:18.767874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804424+0000","last_change":"2024-03-19T01:07:15.480358+0000","last_active":"2024-03-19T01:09:25.804424+0000","last_peered":"2024-03-19T01:09:25.804424+0000","last_clean":"2024-03-19T01:09:25.804424+0000","last_became_active":"2024-03-19T01:07:15.479979+0000","last_became_peered":"2024-03-19T01:07:15.479979+0000","last_unstale":"2024-03-19T01:09:25.804424+0000","last_undegraded":"2024-03-19T01:09:25.804424+0000","last_fullsized":"2024-03-19T01:09:25.804424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:12:54.598435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247132,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78447,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1575,"num_read_kb":875,"num_write":264,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":235341,"data_compressed":5469,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383731,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":520302,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645369,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32336,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712816,"statfs":{"total":95995035648,"available":95961923584,"internally_reserved":0,"allocated":1187840,"data_stored":1182848,"data_compressed":64112,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808894,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":524328,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070532,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":520610,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332169,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1118336,"data_compressed":62289,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495695,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":516127,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":587562,"data_compressed":28338,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":517192,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":587391,"data_compressed":28338,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":515444,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":521039,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757334,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":1108202,"data_compressed":62289,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72548,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72377,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":73370,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T01:10:24.599 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-19T01:10:24.599 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T01:10:24.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.846+0000 7f6094a00640 1 -- 172.21.15.69:0/3286040152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090101490 msgr2=0x7f6090101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:24.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.846+0000 7f6094a00640 1 --2- 172.21.15.69:0/3286040152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090101490 0x7f6090101910 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6088009ca0 tx=0x7f608802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.846+0000 7f6094a00640 1 -- 172.21.15.69:0/3286040152 shutdown_connections 2024-03-19T01:10:24.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.846+0000 7f6094a00640 1 --2- 172.21.15.69:0/3286040152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090101490 0x7f6090101910 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.846+0000 7f6094a00640 1 --2- 172.21.15.69:0/3286040152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6090100af0 0x7f6090100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.846+0000 7f6094a00640 1 --2- 172.21.15.69:0/3286040152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6090100190 0x7f60901005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.846+0000 7f6094a00640 1 -- 172.21.15.69:0/3286040152 >> 172.21.15.69:0/3286040152 conn(0x7f60900fbd50 msgr2=0x7f60900fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:24.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.846+0000 7f6094a00640 1 -- 172.21.15.69:0/3286040152 shutdown_connections 2024-03-19T01:10:24.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.846+0000 7f6094a00640 1 -- 172.21.15.69:0/3286040152 wait complete. 2024-03-19T01:10:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.847+0000 7f6094a00640 1 Processor -- start 2024-03-19T01:10:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.847+0000 7f6094a00640 1 -- start start 2024-03-19T01:10:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.847+0000 7f6094a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6090100190 0x7f6090199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.847+0000 7f6094a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6090100af0 0x7f609019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.847+0000 7f6094a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090101490 0x7f60901a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.847+0000 7f6094a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f609010f6a0 con 0x7f6090100190 2024-03-19T01:10:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.847+0000 7f6094a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f609010f520 con 0x7f6090101490 2024-03-19T01:10:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.847+0000 7f6094a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f609010f820 con 0x7f6090100af0 2024-03-19T01:10:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f608de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6090100190 0x7f6090199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f6087e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6090100af0 0x7f609019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f608e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090101490 0x7f60901a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f608de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6090100190 0x7f6090199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39130/0 (socket says 172.21.15.69:39130) 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f608de00640 1 -- 172.21.15.69:0/1520613433 learned_addr learned my addr 172.21.15.69:0/1520613433 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f608e800640 1 -- 172.21.15.69:0/1520613433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6090100af0 msgr2=0x7f609019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f608e800640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6090100af0 0x7f609019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f608e800640 1 -- 172.21.15.69:0/1520613433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6090100190 msgr2=0x7f6090199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f608e800640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6090100190 0x7f6090199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f608e800640 1 -- 172.21.15.69:0/1520613433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60901a18d0 con 0x7f6090101490 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f6087e00640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6090100af0 0x7f609019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.848+0000 7f608de00640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6090100190 0x7f6090199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.849+0000 7f608e800640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090101490 0x7f60901a1230 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6088037040 tx=0x7f6088032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.849+0000 7f6086a00640 1 -- 172.21.15.69:0/1520613433 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088041020 con 0x7f6090101490 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.849+0000 7f6086a00640 1 -- 172.21.15.69:0/1520613433 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6088034cd0 con 0x7f6090101490 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.849+0000 7f6086a00640 1 -- 172.21.15.69:0/1520613433 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088039040 con 0x7f6090101490 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.849+0000 7f6094a00640 1 -- 172.21.15.69:0/1520613433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60901a1b60 con 0x7f6090101490 2024-03-19T01:10:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.849+0000 7f6094a00640 1 -- 172.21.15.69:0/1520613433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60901a2070 con 0x7f6090101490 2024-03-19T01:10:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.850+0000 7f6094a00640 1 -- 172.21.15.69:0/1520613433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f609010aee0 con 0x7f6090101490 2024-03-19T01:10:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.853+0000 7f6086a00640 1 -- 172.21.15.69:0/1520613433 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f6088044020 con 0x7f6090101490 2024-03-19T01:10:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.855+0000 7f6086a00640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f60740a2910 0x7f60740a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.855+0000 7f6086a00640 1 -- 172.21.15.69:0/1520613433 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f60880f0b90 con 0x7f6090101490 2024-03-19T01:10:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.855+0000 7f608de00640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f60740a2910 0x7f60740a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.856+0000 7f608de00640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f60740a2910 0x7f60740a4dd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f609019ab60 tx=0x7f6080005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:24.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:24.859+0000 7f6086a00640 1 -- 172.21.15.69:0/1520613433 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f608804b020 con 0x7f6090101490 2024-03-19T01:10:25.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.054+0000 7f6094a00640 1 -- 172.21.15.69:0/1520613433 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6090061c60 con 0x7f60740a2910 2024-03-19T01:10:25.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.074+0000 7f6086a00640 1 -- 172.21.15.69:0/1520613433 <== mgr.14152 v2:172.21.15.69:6800/3562156769 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340835 (secure 0 0 0) 0x7f6090061c60 con 0x7f60740a2910 2024-03-19T01:10:25.078 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:25.082 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-03-19T01:10:25.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.081+0000 7f6094a00640 1 -- 172.21.15.69:0/1520613433 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f60740a2910 msgr2=0x7f60740a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:25.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.081+0000 7f6094a00640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f60740a2910 0x7f60740a4dd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f609019ab60 tx=0x7f6080005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:25.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.081+0000 7f6094a00640 1 -- 172.21.15.69:0/1520613433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090101490 msgr2=0x7f60901a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:25.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.081+0000 7f6094a00640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090101490 0x7f60901a1230 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6088037040 tx=0x7f6088032ea0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.082+0000 7f6094a00640 1 -- 172.21.15.69:0/1520613433 shutdown_connections 2024-03-19T01:10:25.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.082+0000 7f6094a00640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f60740a2910 0x7f60740a4dd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.082+0000 7f6094a00640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6090101490 0x7f60901a1230 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.082+0000 7f6094a00640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6090100af0 0x7f609019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.082+0000 7f6094a00640 1 --2- 172.21.15.69:0/1520613433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6090100190 0x7f6090199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.082+0000 7f6094a00640 1 -- 172.21.15.69:0/1520613433 >> 172.21.15.69:0/1520613433 conn(0x7f60900fbd50 msgr2=0x7f60900fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:25.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.082+0000 7f6094a00640 1 -- 172.21.15.69:0/1520613433 shutdown_connections 2024-03-19T01:10:25.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.082+0000 7f6094a00640 1 -- 172.21.15.69:0/1520613433 wait complete. 2024-03-19T01:10:25.101 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-03-19T01:10:23.708079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669049,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1711,"num_read_kb":994,"num_write":447,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378420,"kb_used_data":12608,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563404,"statfs":{"total":1151940427776,"available":1151552925696,"internally_reserved":0,"allocated":12910592,"data_stored":8219381,"data_compressed":430971,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8619,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":150,"num_read_kb":87,"num_write":15,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004408"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804627+0000","last_change":"2024-03-19T01:07:15.500098+0000","last_active":"2024-03-19T01:09:25.804627+0000","last_peered":"2024-03-19T01:09:25.804627+0000","last_clean":"2024-03-19T01:09:25.804627+0000","last_became_active":"2024-03-19T01:07:15.499738+0000","last_became_peered":"2024-03-19T01:07:15.499738+0000","last_unstale":"2024-03-19T01:09:25.804627+0000","last_undegraded":"2024-03-19T01:09:25.804627+0000","last_fullsized":"2024-03-19T01:09:25.804627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:14.703153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156585+0000","last_change":"2024-03-19T01:07:15.492863+0000","last_active":"2024-03-19T01:09:26.156585+0000","last_peered":"2024-03-19T01:09:26.156585+0000","last_clean":"2024-03-19T01:09:26.156585+0000","last_became_active":"2024-03-19T01:07:15.492533+0000","last_became_peered":"2024-03-19T01:07:15.492533+0000","last_unstale":"2024-03-19T01:09:26.156585+0000","last_undegraded":"2024-03-19T01:09:26.156585+0000","last_fullsized":"2024-03-19T01:09:26.156585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:49.028352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156213+0000","last_change":"2024-03-19T01:07:15.494174+0000","last_active":"2024-03-19T01:09:26.156213+0000","last_peered":"2024-03-19T01:09:26.156213+0000","last_clean":"2024-03-19T01:09:26.156213+0000","last_became_active":"2024-03-19T01:07:15.490562+0000","last_became_peered":"2024-03-19T01:07:15.490562+0000","last_unstale":"2024-03-19T01:09:26.156213+0000","last_undegraded":"2024-03-19T01:09:26.156213+0000","last_fullsized":"2024-03-19T01:09:26.156213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:29:18.664396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800428+0000","last_change":"2024-03-19T01:07:15.478397+0000","last_active":"2024-03-19T01:09:25.800428+0000","last_peered":"2024-03-19T01:09:25.800428+0000","last_clean":"2024-03-19T01:09:25.800428+0000","last_became_active":"2024-03-19T01:07:15.477000+0000","last_became_peered":"2024-03-19T01:07:15.477000+0000","last_unstale":"2024-03-19T01:09:25.800428+0000","last_undegraded":"2024-03-19T01:09:25.800428+0000","last_fullsized":"2024-03-19T01:09:25.800428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:58.534329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137621+0000","last_change":"2024-03-19T01:07:15.500138+0000","last_active":"2024-03-19T01:09:26.137621+0000","last_peered":"2024-03-19T01:09:26.137621+0000","last_clean":"2024-03-19T01:09:26.137621+0000","last_became_active":"2024-03-19T01:07:15.499759+0000","last_became_peered":"2024-03-19T01:07:15.499759+0000","last_unstale":"2024-03-19T01:09:26.137621+0000","last_undegraded":"2024-03-19T01:09:26.137621+0000","last_fullsized":"2024-03-19T01:09:26.137621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:23.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804946+0000","last_change":"2024-03-19T01:09:25.804601+0000","last_active":"2024-03-19T01:09:25.804946+0000","last_peered":"2024-03-19T01:09:25.804946+0000","last_clean":"2024-03-19T01:09:25.804946+0000","last_became_active":"2024-03-19T01:09:25.804332+0000","last_became_peered":"2024-03-19T01:09:25.804332+0000","last_unstale":"2024-03-19T01:09:25.804946+0000","last_undegraded":"2024-03-19T01:09:25.804946+0000","last_fullsized":"2024-03-19T01:09:25.804946+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:44:10.372309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962138+0000","last_change":"2024-03-19T01:07:15.486327+0000","last_active":"2024-03-19T01:09:25.962138+0000","last_peered":"2024-03-19T01:09:25.962138+0000","last_clean":"2024-03-19T01:09:25.962138+0000","last_became_active":"2024-03-19T01:07:15.485954+0000","last_became_peered":"2024-03-19T01:07:15.485954+0000","last_unstale":"2024-03-19T01:09:25.962138+0000","last_undegraded":"2024-03-19T01:09:25.962138+0000","last_fullsized":"2024-03-19T01:09:25.962138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:41:53.316260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914327+0000","last_change":"2024-03-19T01:07:15.504993+0000","last_active":"2024-03-19T01:09:25.914327+0000","last_peered":"2024-03-19T01:09:25.914327+0000","last_clean":"2024-03-19T01:09:25.914327+0000","last_became_active":"2024-03-19T01:07:15.502771+0000","last_became_peered":"2024-03-19T01:07:15.502771+0000","last_unstale":"2024-03-19T01:09:25.914327+0000","last_undegraded":"2024-03-19T01:09:25.914327+0000","last_fullsized":"2024-03-19T01:09:25.914327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:22:33.712608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.806615+0000","last_change":"2024-03-19T01:09:25.805915+0000","last_active":"2024-03-19T01:09:25.806615+0000","last_peered":"2024-03-19T01:09:25.806615+0000","last_clean":"2024-03-19T01:09:25.806615+0000","last_became_active":"2024-03-19T01:09:25.805344+0000","last_became_peered":"2024-03-19T01:09:25.805344+0000","last_unstale":"2024-03-19T01:09:25.806615+0000","last_undegraded":"2024-03-19T01:09:25.806615+0000","last_fullsized":"2024-03-19T01:09:25.806615+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:11:55.795773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801964+0000","last_change":"2024-03-19T01:07:15.475655+0000","last_active":"2024-03-19T01:09:25.801964+0000","last_peered":"2024-03-19T01:09:25.801964+0000","last_clean":"2024-03-19T01:09:25.801964+0000","last_became_active":"2024-03-19T01:07:15.475265+0000","last_became_peered":"2024-03-19T01:07:15.475265+0000","last_unstale":"2024-03-19T01:09:25.801964+0000","last_undegraded":"2024-03-19T01:09:25.801964+0000","last_fullsized":"2024-03-19T01:09:25.801964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:24:38.898191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962612+0000","last_change":"2024-03-19T01:07:15.482988+0000","last_active":"2024-03-19T01:09:25.962612+0000","last_peered":"2024-03-19T01:09:25.962612+0000","last_clean":"2024-03-19T01:09:25.962612+0000","last_became_active":"2024-03-19T01:07:15.482673+0000","last_became_peered":"2024-03-19T01:07:15.482673+0000","last_unstale":"2024-03-19T01:09:25.962612+0000","last_undegraded":"2024-03-19T01:09:25.962612+0000","last_fullsized":"2024-03-19T01:09:25.962612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:15:48.807200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800368+0000","last_change":"2024-03-19T01:07:15.479593+0000","last_active":"2024-03-19T01:09:25.800368+0000","last_peered":"2024-03-19T01:09:25.800368+0000","last_clean":"2024-03-19T01:09:25.800368+0000","last_became_active":"2024-03-19T01:07:15.479220+0000","last_became_peered":"2024-03-19T01:07:15.479220+0000","last_unstale":"2024-03-19T01:09:25.800368+0000","last_undegraded":"2024-03-19T01:09:25.800368+0000","last_fullsized":"2024-03-19T01:09:25.800368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:40:24.390845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804248+0000","last_change":"2024-03-19T01:07:15.464737+0000","last_active":"2024-03-19T01:09:25.804248+0000","last_peered":"2024-03-19T01:09:25.804248+0000","last_clean":"2024-03-19T01:09:25.804248+0000","last_became_active":"2024-03-19T01:07:15.463641+0000","last_became_peered":"2024-03-19T01:07:15.463641+0000","last_unstale":"2024-03-19T01:09:25.804248+0000","last_undegraded":"2024-03-19T01:09:25.804248+0000","last_fullsized":"2024-03-19T01:09:25.804248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:13.964397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.155743+0000","last_change":"2024-03-19T01:07:15.492803+0000","last_active":"2024-03-19T01:09:26.155743+0000","last_peered":"2024-03-19T01:09:26.155743+0000","last_clean":"2024-03-19T01:09:26.155743+0000","last_became_active":"2024-03-19T01:07:15.492422+0000","last_became_peered":"2024-03-19T01:07:15.492422+0000","last_unstale":"2024-03-19T01:09:26.155743+0000","last_undegraded":"2024-03-19T01:09:26.155743+0000","last_fullsized":"2024-03-19T01:09:26.155743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:59:08.292399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.155747+0000","last_change":"2024-03-19T01:07:15.498073+0000","last_active":"2024-03-19T01:09:26.155747+0000","last_peered":"2024-03-19T01:09:26.155747+0000","last_clean":"2024-03-19T01:09:26.155747+0000","last_became_active":"2024-03-19T01:07:15.493650+0000","last_became_peered":"2024-03-19T01:07:15.493650+0000","last_unstale":"2024-03-19T01:09:26.155747+0000","last_undegraded":"2024-03-19T01:09:26.155747+0000","last_fullsized":"2024-03-19T01:09:26.155747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:25:28.908122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154534+0000","last_change":"2024-03-19T01:07:15.496434+0000","last_active":"2024-03-19T01:09:26.154534+0000","last_peered":"2024-03-19T01:09:26.154534+0000","last_clean":"2024-03-19T01:09:26.154534+0000","last_became_active":"2024-03-19T01:07:15.495956+0000","last_became_peered":"2024-03-19T01:07:15.495956+0000","last_unstale":"2024-03-19T01:09:26.154534+0000","last_undegraded":"2024-03-19T01:09:26.154534+0000","last_fullsized":"2024-03-19T01:09:26.154534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:49.505947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804589+0000","last_change":"2024-03-19T01:07:15.504771+0000","last_active":"2024-03-19T01:09:25.804589+0000","last_peered":"2024-03-19T01:09:25.804589+0000","last_clean":"2024-03-19T01:09:25.804589+0000","last_became_active":"2024-03-19T01:07:15.504485+0000","last_became_peered":"2024-03-19T01:07:15.504485+0000","last_unstale":"2024-03-19T01:09:25.804589+0000","last_undegraded":"2024-03-19T01:09:25.804589+0000","last_fullsized":"2024-03-19T01:09:25.804589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:17:36.760943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.802629+0000","last_change":"2024-03-19T01:07:15.491045+0000","last_active":"2024-03-19T01:09:25.802629+0000","last_peered":"2024-03-19T01:09:25.802629+0000","last_clean":"2024-03-19T01:09:25.802629+0000","last_became_active":"2024-03-19T01:07:15.488914+0000","last_became_peered":"2024-03-19T01:07:15.488914+0000","last_unstale":"2024-03-19T01:09:25.802629+0000","last_undegraded":"2024-03-19T01:09:25.802629+0000","last_fullsized":"2024-03-19T01:09:25.802629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:14.853457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914298+0000","last_change":"2024-03-19T01:07:15.495306+0000","last_active":"2024-03-19T01:09:25.914298+0000","last_peered":"2024-03-19T01:09:25.914298+0000","last_clean":"2024-03-19T01:09:25.914298+0000","last_became_active":"2024-03-19T01:07:15.494906+0000","last_became_peered":"2024-03-19T01:07:15.494906+0000","last_unstale":"2024-03-19T01:09:25.914298+0000","last_undegraded":"2024-03-19T01:09:25.914298+0000","last_fullsized":"2024-03-19T01:09:25.914298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:48.176410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.913990+0000","last_change":"2024-03-19T01:07:19.689402+0000","last_active":"2024-03-19T01:09:25.913990+0000","last_peered":"2024-03-19T01:09:25.913990+0000","last_clean":"2024-03-19T01:09:25.913990+0000","last_became_active":"2024-03-19T01:07:19.688799+0000","last_became_peered":"2024-03-19T01:07:19.688799+0000","last_unstale":"2024-03-19T01:09:25.913990+0000","last_undegraded":"2024-03-19T01:09:25.913990+0000","last_fullsized":"2024-03-19T01:09:25.913990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:33:51.890047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":199,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.039278+0000","last_change":"2024-03-19T01:07:14.477348+0000","last_active":"2024-03-19T01:10:21.039278+0000","last_peered":"2024-03-19T01:10:21.039278+0000","last_clean":"2024-03-19T01:10:21.039278+0000","last_became_active":"2024-03-19T01:07:14.476946+0000","last_became_peered":"2024-03-19T01:07:14.476946+0000","last_unstale":"2024-03-19T01:10:21.039278+0000","last_undegraded":"2024-03-19T01:10:21.039278+0000","last_fullsized":"2024-03-19T01:10:21.039278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:19.269837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915607+0000","last_change":"2024-03-19T01:07:15.495646+0000","last_active":"2024-03-19T01:09:25.915607+0000","last_peered":"2024-03-19T01:09:25.915607+0000","last_clean":"2024-03-19T01:09:25.915607+0000","last_became_active":"2024-03-19T01:07:15.495203+0000","last_became_peered":"2024-03-19T01:07:15.495203+0000","last_unstale":"2024-03-19T01:09:25.915607+0000","last_undegraded":"2024-03-19T01:09:25.915607+0000","last_fullsized":"2024-03-19T01:09:25.915607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:01:31.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.799843+0000","last_change":"2024-03-19T01:07:19.690465+0000","last_active":"2024-03-19T01:09:25.799843+0000","last_peered":"2024-03-19T01:09:25.799843+0000","last_clean":"2024-03-19T01:09:25.799843+0000","last_became_active":"2024-03-19T01:07:19.689852+0000","last_became_peered":"2024-03-19T01:07:19.689852+0000","last_unstale":"2024-03-19T01:09:25.799843+0000","last_undegraded":"2024-03-19T01:09:25.799843+0000","last_fullsized":"2024-03-19T01:09:25.799843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:18.891411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915740+0000","last_change":"2024-03-19T01:07:14.487165+0000","last_active":"2024-03-19T01:09:25.915740+0000","last_peered":"2024-03-19T01:09:25.915740+0000","last_clean":"2024-03-19T01:09:25.915740+0000","last_became_active":"2024-03-19T01:07:14.484194+0000","last_became_peered":"2024-03-19T01:07:14.484194+0000","last_unstale":"2024-03-19T01:09:25.915740+0000","last_undegraded":"2024-03-19T01:09:25.915740+0000","last_fullsized":"2024-03-19T01:09:25.915740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:54:47.914565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156861+0000","last_change":"2024-03-19T01:07:15.498687+0000","last_active":"2024-03-19T01:09:26.156861+0000","last_peered":"2024-03-19T01:09:26.156861+0000","last_clean":"2024-03-19T01:09:26.156861+0000","last_became_active":"2024-03-19T01:07:15.498146+0000","last_became_peered":"2024-03-19T01:07:15.498146+0000","last_unstale":"2024-03-19T01:09:26.156861+0000","last_undegraded":"2024-03-19T01:09:26.156861+0000","last_fullsized":"2024-03-19T01:09:26.156861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:47:21.846333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800041+0000","last_change":"2024-03-19T01:07:19.690691+0000","last_active":"2024-03-19T01:09:25.800041+0000","last_peered":"2024-03-19T01:09:25.800041+0000","last_clean":"2024-03-19T01:09:25.800041+0000","last_became_active":"2024-03-19T01:07:19.690083+0000","last_became_peered":"2024-03-19T01:07:19.690083+0000","last_unstale":"2024-03-19T01:09:25.800041+0000","last_undegraded":"2024-03-19T01:09:25.800041+0000","last_fullsized":"2024-03-19T01:09:25.800041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:51:30.205694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804545+0000","last_change":"2024-03-19T01:08:24.801510+0000","last_active":"2024-03-19T01:09:25.804545+0000","last_peered":"2024-03-19T01:09:25.804545+0000","last_clean":"2024-03-19T01:09:25.804545+0000","last_became_active":"2024-03-19T01:08:24.789315+0000","last_became_peered":"2024-03-19T01:08:24.789315+0000","last_unstale":"2024-03-19T01:09:25.804545+0000","last_undegraded":"2024-03-19T01:09:25.804545+0000","last_fullsized":"2024-03-19T01:09:25.804545+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:24:03.243394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154320+0000","last_change":"2024-03-19T01:07:15.489371+0000","last_active":"2024-03-19T01:09:26.154320+0000","last_peered":"2024-03-19T01:09:26.154320+0000","last_clean":"2024-03-19T01:09:26.154320+0000","last_became_active":"2024-03-19T01:07:15.488975+0000","last_became_peered":"2024-03-19T01:07:15.488975+0000","last_unstale":"2024-03-19T01:09:26.154320+0000","last_undegraded":"2024-03-19T01:09:26.154320+0000","last_fullsized":"2024-03-19T01:09:26.154320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:04:47.640512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.799877+0000","last_change":"2024-03-19T01:07:19.691590+0000","last_active":"2024-03-19T01:09:25.799877+0000","last_peered":"2024-03-19T01:09:25.799877+0000","last_clean":"2024-03-19T01:09:25.799877+0000","last_became_active":"2024-03-19T01:07:19.691037+0000","last_became_peered":"2024-03-19T01:07:19.691037+0000","last_unstale":"2024-03-19T01:09:25.799877+0000","last_undegraded":"2024-03-19T01:09:25.799877+0000","last_fullsized":"2024-03-19T01:09:25.799877+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:20:02.007483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137775+0000","last_change":"2024-03-19T01:07:14.475653+0000","last_active":"2024-03-19T01:09:26.137775+0000","last_peered":"2024-03-19T01:09:26.137775+0000","last_clean":"2024-03-19T01:09:26.137775+0000","last_became_active":"2024-03-19T01:07:14.475315+0000","last_became_peered":"2024-03-19T01:07:14.475315+0000","last_unstale":"2024-03-19T01:09:26.137775+0000","last_undegraded":"2024-03-19T01:09:26.137775+0000","last_fullsized":"2024-03-19T01:09:26.137775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:44:04.656950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798438+0000","last_change":"2024-03-19T01:07:15.480882+0000","last_active":"2024-03-19T01:09:25.798438+0000","last_peered":"2024-03-19T01:09:25.798438+0000","last_clean":"2024-03-19T01:09:25.798438+0000","last_became_active":"2024-03-19T01:07:15.480541+0000","last_became_peered":"2024-03-19T01:07:15.480541+0000","last_unstale":"2024-03-19T01:09:25.798438+0000","last_undegraded":"2024-03-19T01:09:25.798438+0000","last_fullsized":"2024-03-19T01:09:25.798438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:37:24.724531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914196+0000","last_change":"2024-03-19T01:07:19.672979+0000","last_active":"2024-03-19T01:09:25.914196+0000","last_peered":"2024-03-19T01:09:25.914196+0000","last_clean":"2024-03-19T01:09:25.914196+0000","last_became_active":"2024-03-19T01:07:19.672530+0000","last_became_peered":"2024-03-19T01:07:19.672530+0000","last_unstale":"2024-03-19T01:09:25.914196+0000","last_undegraded":"2024-03-19T01:09:25.914196+0000","last_fullsized":"2024-03-19T01:09:25.914196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:12.948079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":193,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:20.916080+0000","last_change":"2024-03-19T01:07:14.475620+0000","last_active":"2024-03-19T01:10:20.916080+0000","last_peered":"2024-03-19T01:10:20.916080+0000","last_clean":"2024-03-19T01:10:20.916080+0000","last_became_active":"2024-03-19T01:07:14.475029+0000","last_became_peered":"2024-03-19T01:07:14.475029+0000","last_unstale":"2024-03-19T01:10:20.916080+0000","last_undegraded":"2024-03-19T01:10:20.916080+0000","last_fullsized":"2024-03-19T01:10:20.916080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:48:53.983457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137648+0000","last_change":"2024-03-19T01:07:15.484680+0000","last_active":"2024-03-19T01:09:26.137648+0000","last_peered":"2024-03-19T01:09:26.137648+0000","last_clean":"2024-03-19T01:09:26.137648+0000","last_became_active":"2024-03-19T01:07:15.484333+0000","last_became_peered":"2024-03-19T01:07:15.484333+0000","last_unstale":"2024-03-19T01:09:26.137648+0000","last_undegraded":"2024-03-19T01:09:26.137648+0000","last_fullsized":"2024-03-19T01:09:26.137648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:38:23.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914115+0000","last_change":"2024-03-19T01:08:24.798971+0000","last_active":"2024-03-19T01:09:25.914115+0000","last_peered":"2024-03-19T01:09:25.914115+0000","last_clean":"2024-03-19T01:09:25.914115+0000","last_became_active":"2024-03-19T01:08:24.798504+0000","last_became_peered":"2024-03-19T01:08:24.798504+0000","last_unstale":"2024-03-19T01:09:25.914115+0000","last_undegraded":"2024-03-19T01:09:25.914115+0000","last_fullsized":"2024-03-19T01:09:25.914115+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.460490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801639+0000","last_change":"2024-03-19T01:07:14.481499+0000","last_active":"2024-03-19T01:09:25.801639+0000","last_peered":"2024-03-19T01:09:25.801639+0000","last_clean":"2024-03-19T01:09:25.801639+0000","last_became_active":"2024-03-19T01:07:14.481123+0000","last_became_peered":"2024-03-19T01:07:14.481123+0000","last_unstale":"2024-03-19T01:09:25.801639+0000","last_undegraded":"2024-03-19T01:09:25.801639+0000","last_fullsized":"2024-03-19T01:09:25.801639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:36:15.349748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798153+0000","last_change":"2024-03-19T01:07:15.499744+0000","last_active":"2024-03-19T01:09:25.798153+0000","last_peered":"2024-03-19T01:09:25.798153+0000","last_clean":"2024-03-19T01:09:25.798153+0000","last_became_active":"2024-03-19T01:07:15.499037+0000","last_became_peered":"2024-03-19T01:07:15.499037+0000","last_unstale":"2024-03-19T01:09:25.798153+0000","last_undegraded":"2024-03-19T01:09:25.798153+0000","last_fullsized":"2024-03-19T01:09:25.798153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:20:35.524664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804490+0000","last_change":"2024-03-19T01:07:19.684187+0000","last_active":"2024-03-19T01:09:25.804490+0000","last_peered":"2024-03-19T01:09:25.804490+0000","last_clean":"2024-03-19T01:09:25.804490+0000","last_became_active":"2024-03-19T01:07:19.683782+0000","last_became_peered":"2024-03-19T01:07:19.683782+0000","last_unstale":"2024-03-19T01:09:25.804490+0000","last_undegraded":"2024-03-19T01:09:25.804490+0000","last_fullsized":"2024-03-19T01:09:25.804490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:17:50.818003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.802689+0000","last_change":"2024-03-19T01:07:14.485538+0000","last_active":"2024-03-19T01:09:25.802689+0000","last_peered":"2024-03-19T01:09:25.802689+0000","last_clean":"2024-03-19T01:09:25.802689+0000","last_became_active":"2024-03-19T01:07:14.485004+0000","last_became_peered":"2024-03-19T01:07:14.485004+0000","last_unstale":"2024-03-19T01:09:25.802689+0000","last_undegraded":"2024-03-19T01:09:25.802689+0000","last_fullsized":"2024-03-19T01:09:25.802689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:55:50.853725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914576+0000","last_change":"2024-03-19T01:07:15.503370+0000","last_active":"2024-03-19T01:09:25.914576+0000","last_peered":"2024-03-19T01:09:25.914576+0000","last_clean":"2024-03-19T01:09:25.914576+0000","last_became_active":"2024-03-19T01:07:15.501375+0000","last_became_peered":"2024-03-19T01:07:15.501375+0000","last_unstale":"2024-03-19T01:09:25.914576+0000","last_undegraded":"2024-03-19T01:09:25.914576+0000","last_fullsized":"2024-03-19T01:09:25.914576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:36:01.814936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156480+0000","last_change":"2024-03-19T01:07:19.697067+0000","last_active":"2024-03-19T01:09:26.156480+0000","last_peered":"2024-03-19T01:09:26.156480+0000","last_clean":"2024-03-19T01:09:26.156480+0000","last_became_active":"2024-03-19T01:07:19.696364+0000","last_became_peered":"2024-03-19T01:07:19.696364+0000","last_unstale":"2024-03-19T01:09:26.156480+0000","last_undegraded":"2024-03-19T01:09:26.156480+0000","last_fullsized":"2024-03-19T01:09:26.156480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:34.388407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.797972+0000","last_change":"2024-03-19T01:07:14.475361+0000","last_active":"2024-03-19T01:09:25.797972+0000","last_peered":"2024-03-19T01:09:25.797972+0000","last_clean":"2024-03-19T01:09:25.797972+0000","last_became_active":"2024-03-19T01:07:14.474577+0000","last_became_peered":"2024-03-19T01:07:14.474577+0000","last_unstale":"2024-03-19T01:09:25.797972+0000","last_undegraded":"2024-03-19T01:09:25.797972+0000","last_fullsized":"2024-03-19T01:09:25.797972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:01:04.713405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800954+0000","last_change":"2024-03-19T01:07:15.506310+0000","last_active":"2024-03-19T01:09:25.800954+0000","last_peered":"2024-03-19T01:09:25.800954+0000","last_clean":"2024-03-19T01:09:25.800954+0000","last_became_active":"2024-03-19T01:07:15.505792+0000","last_became_peered":"2024-03-19T01:07:15.505792+0000","last_unstale":"2024-03-19T01:09:25.800954+0000","last_undegraded":"2024-03-19T01:09:25.800954+0000","last_fullsized":"2024-03-19T01:09:25.800954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:40.630816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914516+0000","last_change":"2024-03-19T01:07:19.683632+0000","last_active":"2024-03-19T01:09:25.914516+0000","last_peered":"2024-03-19T01:09:25.914516+0000","last_clean":"2024-03-19T01:09:25.914516+0000","last_became_active":"2024-03-19T01:07:19.683204+0000","last_became_peered":"2024-03-19T01:07:19.683204+0000","last_unstale":"2024-03-19T01:09:25.914516+0000","last_undegraded":"2024-03-19T01:09:25.914516+0000","last_fullsized":"2024-03-19T01:09:25.914516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:58:53.512824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":342,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.916543+0000","last_change":"2024-03-19T01:07:14.471347+0000","last_active":"2024-03-19T01:10:21.916543+0000","last_peered":"2024-03-19T01:10:21.916543+0000","last_clean":"2024-03-19T01:10:21.916543+0000","last_became_active":"2024-03-19T01:07:14.471001+0000","last_became_peered":"2024-03-19T01:07:14.471001+0000","last_unstale":"2024-03-19T01:10:21.916543+0000","last_undegraded":"2024-03-19T01:10:21.916543+0000","last_fullsized":"2024-03-19T01:10:21.916543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:46.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":306,"num_read_kb":306,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154549+0000","last_change":"2024-03-19T01:07:15.486779+0000","last_active":"2024-03-19T01:09:26.154549+0000","last_peered":"2024-03-19T01:09:26.154549+0000","last_clean":"2024-03-19T01:09:26.154549+0000","last_became_active":"2024-03-19T01:07:15.486405+0000","last_became_peered":"2024-03-19T01:07:15.486405+0000","last_unstale":"2024-03-19T01:09:26.154549+0000","last_undegraded":"2024-03-19T01:09:26.154549+0000","last_fullsized":"2024-03-19T01:09:26.154549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:14:13.350169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800785+0000","last_change":"2024-03-19T01:07:19.686892+0000","last_active":"2024-03-19T01:09:25.800785+0000","last_peered":"2024-03-19T01:09:25.800785+0000","last_clean":"2024-03-19T01:09:25.800785+0000","last_became_active":"2024-03-19T01:07:19.686414+0000","last_became_peered":"2024-03-19T01:07:19.686414+0000","last_unstale":"2024-03-19T01:09:25.800785+0000","last_undegraded":"2024-03-19T01:09:25.800785+0000","last_fullsized":"2024-03-19T01:09:25.800785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:18:18.105905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962657+0000","last_change":"2024-03-19T01:08:24.792588+0000","last_active":"2024-03-19T01:09:25.962657+0000","last_peered":"2024-03-19T01:09:25.962657+0000","last_clean":"2024-03-19T01:09:25.962657+0000","last_became_active":"2024-03-19T01:08:24.788228+0000","last_became_peered":"2024-03-19T01:08:24.788228+0000","last_unstale":"2024-03-19T01:09:25.962657+0000","last_undegraded":"2024-03-19T01:09:25.962657+0000","last_fullsized":"2024-03-19T01:09:25.962657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:52.150165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156831+0000","last_change":"2024-03-19T01:07:15.499507+0000","last_active":"2024-03-19T01:09:26.156831+0000","last_peered":"2024-03-19T01:09:26.156831+0000","last_clean":"2024-03-19T01:09:26.156831+0000","last_became_active":"2024-03-19T01:07:15.497261+0000","last_became_peered":"2024-03-19T01:07:15.497261+0000","last_unstale":"2024-03-19T01:09:26.156831+0000","last_undegraded":"2024-03-19T01:09:26.156831+0000","last_fullsized":"2024-03-19T01:09:26.156831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:40:06.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800171+0000","last_change":"2024-03-19T01:07:19.690847+0000","last_active":"2024-03-19T01:09:25.800171+0000","last_peered":"2024-03-19T01:09:25.800171+0000","last_clean":"2024-03-19T01:09:25.800171+0000","last_became_active":"2024-03-19T01:07:19.689762+0000","last_became_peered":"2024-03-19T01:07:19.689762+0000","last_unstale":"2024-03-19T01:09:25.800171+0000","last_undegraded":"2024-03-19T01:09:25.800171+0000","last_fullsized":"2024-03-19T01:09:25.800171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:24:40.213171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798343+0000","last_change":"2024-03-19T01:07:14.473744+0000","last_active":"2024-03-19T01:09:25.798343+0000","last_peered":"2024-03-19T01:09:25.798343+0000","last_clean":"2024-03-19T01:09:25.798343+0000","last_became_active":"2024-03-19T01:07:14.473404+0000","last_became_peered":"2024-03-19T01:07:14.473404+0000","last_unstale":"2024-03-19T01:09:25.798343+0000","last_undegraded":"2024-03-19T01:09:25.798343+0000","last_fullsized":"2024-03-19T01:09:25.798343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:29:39.892678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154443+0000","last_change":"2024-03-19T01:07:15.481444+0000","last_active":"2024-03-19T01:09:26.154443+0000","last_peered":"2024-03-19T01:09:26.154443+0000","last_clean":"2024-03-19T01:09:26.154443+0000","last_became_active":"2024-03-19T01:07:15.481106+0000","last_became_peered":"2024-03-19T01:07:15.481106+0000","last_unstale":"2024-03-19T01:09:26.154443+0000","last_undegraded":"2024-03-19T01:09:26.154443+0000","last_fullsized":"2024-03-19T01:09:26.154443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:00.263063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801166+0000","last_change":"2024-03-19T01:07:19.686919+0000","last_active":"2024-03-19T01:09:25.801166+0000","last_peered":"2024-03-19T01:09:25.801166+0000","last_clean":"2024-03-19T01:09:25.801166+0000","last_became_active":"2024-03-19T01:07:19.686416+0000","last_became_peered":"2024-03-19T01:07:19.686416+0000","last_unstale":"2024-03-19T01:09:25.801166+0000","last_undegraded":"2024-03-19T01:09:25.801166+0000","last_fullsized":"2024-03-19T01:09:25.801166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:29:11.068957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'41","reported_seq":392,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.917526+0000","last_change":"2024-03-19T01:07:14.468630+0000","last_active":"2024-03-19T01:10:21.917526+0000","last_peered":"2024-03-19T01:10:21.917526+0000","last_clean":"2024-03-19T01:10:21.917526+0000","last_became_active":"2024-03-19T01:07:14.468293+0000","last_became_peered":"2024-03-19T01:07:14.468293+0000","last_unstale":"2024-03-19T01:10:21.917526+0000","last_undegraded":"2024-03-19T01:10:21.917526+0000","last_fullsized":"2024-03-19T01:10:21.917526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:53.575507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70405,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":319,"num_read_kb":230,"num_write":43,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137935+0000","last_change":"2024-03-19T01:07:15.477226+0000","last_active":"2024-03-19T01:09:26.137935+0000","last_peered":"2024-03-19T01:09:26.137935+0000","last_clean":"2024-03-19T01:09:26.137935+0000","last_became_active":"2024-03-19T01:07:15.476882+0000","last_became_peered":"2024-03-19T01:07:15.476882+0000","last_unstale":"2024-03-19T01:09:26.137935+0000","last_undegraded":"2024-03-19T01:09:26.137935+0000","last_fullsized":"2024-03-19T01:09:26.137935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:11:10.051837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798608+0000","last_change":"2024-03-19T01:07:19.690004+0000","last_active":"2024-03-19T01:09:25.798608+0000","last_peered":"2024-03-19T01:09:25.798608+0000","last_clean":"2024-03-19T01:09:25.798608+0000","last_became_active":"2024-03-19T01:07:19.689573+0000","last_became_peered":"2024-03-19T01:07:19.689573+0000","last_unstale":"2024-03-19T01:09:25.798608+0000","last_undegraded":"2024-03-19T01:09:25.798608+0000","last_fullsized":"2024-03-19T01:09:25.798608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:31:21.788219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914144+0000","last_change":"2024-03-19T01:07:14.487373+0000","last_active":"2024-03-19T01:09:25.914144+0000","last_peered":"2024-03-19T01:09:25.914144+0000","last_clean":"2024-03-19T01:09:25.914144+0000","last_became_active":"2024-03-19T01:07:14.487077+0000","last_became_peered":"2024-03-19T01:07:14.487077+0000","last_unstale":"2024-03-19T01:09:25.914144+0000","last_undegraded":"2024-03-19T01:09:25.914144+0000","last_fullsized":"2024-03-19T01:09:25.914144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:33:01.789431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.803723+0000","last_change":"2024-03-19T01:07:15.498009+0000","last_active":"2024-03-19T01:09:25.803723+0000","last_peered":"2024-03-19T01:09:25.803723+0000","last_clean":"2024-03-19T01:09:25.803723+0000","last_became_active":"2024-03-19T01:07:15.496707+0000","last_became_peered":"2024-03-19T01:07:15.496707+0000","last_unstale":"2024-03-19T01:09:25.803723+0000","last_undegraded":"2024-03-19T01:09:25.803723+0000","last_fullsized":"2024-03-19T01:09:25.803723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:50.058299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804327+0000","last_change":"2024-03-19T01:07:19.677635+0000","last_active":"2024-03-19T01:09:25.804327+0000","last_peered":"2024-03-19T01:09:25.804327+0000","last_clean":"2024-03-19T01:09:25.804327+0000","last_became_active":"2024-03-19T01:07:19.677207+0000","last_became_peered":"2024-03-19T01:07:19.677207+0000","last_unstale":"2024-03-19T01:09:25.804327+0000","last_undegraded":"2024-03-19T01:09:25.804327+0000","last_fullsized":"2024-03-19T01:09:25.804327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:23:28.650962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798247+0000","last_change":"2024-03-19T01:07:14.482054+0000","last_active":"2024-03-19T01:09:25.798247+0000","last_peered":"2024-03-19T01:09:25.798247+0000","last_clean":"2024-03-19T01:09:25.798247+0000","last_became_active":"2024-03-19T01:07:14.481703+0000","last_became_peered":"2024-03-19T01:07:14.481703+0000","last_unstale":"2024-03-19T01:09:25.798247+0000","last_undegraded":"2024-03-19T01:09:25.798247+0000","last_fullsized":"2024-03-19T01:09:25.798247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:02:43.565442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962388+0000","last_change":"2024-03-19T01:07:15.487236+0000","last_active":"2024-03-19T01:09:25.962388+0000","last_peered":"2024-03-19T01:09:25.962388+0000","last_clean":"2024-03-19T01:09:25.962388+0000","last_became_active":"2024-03-19T01:07:15.486931+0000","last_became_peered":"2024-03-19T01:07:15.486931+0000","last_unstale":"2024-03-19T01:09:25.962388+0000","last_undegraded":"2024-03-19T01:09:25.962388+0000","last_fullsized":"2024-03-19T01:09:25.962388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:51:22.201904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800840+0000","last_change":"2024-03-19T01:07:19.693813+0000","last_active":"2024-03-19T01:09:25.800840+0000","last_peered":"2024-03-19T01:09:25.800840+0000","last_clean":"2024-03-19T01:09:25.800840+0000","last_became_active":"2024-03-19T01:07:19.693384+0000","last_became_peered":"2024-03-19T01:07:19.693384+0000","last_unstale":"2024-03-19T01:09:25.800840+0000","last_undegraded":"2024-03-19T01:09:25.800840+0000","last_fullsized":"2024-03-19T01:09:25.800840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:22:09.372461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'9","reported_seq":202,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:22.006239+0000","last_change":"2024-03-19T01:07:14.468467+0000","last_active":"2024-03-19T01:10:22.006239+0000","last_peered":"2024-03-19T01:10:22.006239+0000","last_clean":"2024-03-19T01:10:22.006239+0000","last_became_active":"2024-03-19T01:07:14.468074+0000","last_became_peered":"2024-03-19T01:07:14.468074+0000","last_unstale":"2024-03-19T01:10:22.006239+0000","last_undegraded":"2024-03-19T01:10:22.006239+0000","last_fullsized":"2024-03-19T01:10:22.006239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:07:49.216998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":162,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962143+0000","last_change":"2024-03-19T01:07:15.475310+0000","last_active":"2024-03-19T01:09:25.962143+0000","last_peered":"2024-03-19T01:09:25.962143+0000","last_clean":"2024-03-19T01:09:25.962143+0000","last_became_active":"2024-03-19T01:07:15.474914+0000","last_became_peered":"2024-03-19T01:07:15.474914+0000","last_unstale":"2024-03-19T01:09:25.962143+0000","last_undegraded":"2024-03-19T01:09:25.962143+0000","last_fullsized":"2024-03-19T01:09:25.962143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:34.036892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804790+0000","last_change":"2024-03-19T01:07:19.697937+0000","last_active":"2024-03-19T01:09:25.804790+0000","last_peered":"2024-03-19T01:09:25.804790+0000","last_clean":"2024-03-19T01:09:25.804790+0000","last_became_active":"2024-03-19T01:07:19.697447+0000","last_became_peered":"2024-03-19T01:07:19.697447+0000","last_unstale":"2024-03-19T01:09:25.804790+0000","last_undegraded":"2024-03-19T01:09:25.804790+0000","last_fullsized":"2024-03-19T01:09:25.804790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:49.259118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137706+0000","last_change":"2024-03-19T01:07:14.476382+0000","last_active":"2024-03-19T01:09:26.137706+0000","last_peered":"2024-03-19T01:09:26.137706+0000","last_clean":"2024-03-19T01:09:26.137706+0000","last_became_active":"2024-03-19T01:07:14.476089+0000","last_became_peered":"2024-03-19T01:07:14.476089+0000","last_unstale":"2024-03-19T01:09:26.137706+0000","last_undegraded":"2024-03-19T01:09:26.137706+0000","last_fullsized":"2024-03-19T01:09:26.137706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:57:42.482493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800778+0000","last_change":"2024-03-19T01:08:24.794335+0000","last_active":"2024-03-19T01:09:25.800778+0000","last_peered":"2024-03-19T01:09:25.800778+0000","last_clean":"2024-03-19T01:09:25.800778+0000","last_became_active":"2024-03-19T01:08:24.793725+0000","last_became_peered":"2024-03-19T01:08:24.793725+0000","last_unstale":"2024-03-19T01:09:25.800778+0000","last_undegraded":"2024-03-19T01:09:25.800778+0000","last_fullsized":"2024-03-19T01:09:25.800778+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:42:18.797824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798009+0000","last_change":"2024-03-19T01:07:19.688235+0000","last_active":"2024-03-19T01:09:25.798009+0000","last_peered":"2024-03-19T01:09:25.798009+0000","last_clean":"2024-03-19T01:09:25.798009+0000","last_became_active":"2024-03-19T01:07:19.687879+0000","last_became_peered":"2024-03-19T01:07:19.687879+0000","last_unstale":"2024-03-19T01:09:25.798009+0000","last_undegraded":"2024-03-19T01:09:25.798009+0000","last_fullsized":"2024-03-19T01:09:25.798009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:16:29.465971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:33.028596+0000","last_change":"2024-03-19T01:07:14.474346+0000","last_active":"2024-03-19T01:09:33.028596+0000","last_peered":"2024-03-19T01:09:33.028596+0000","last_clean":"2024-03-19T01:09:33.028596+0000","last_became_active":"2024-03-19T01:07:14.473989+0000","last_became_peered":"2024-03-19T01:07:14.473989+0000","last_unstale":"2024-03-19T01:09:33.028596+0000","last_undegraded":"2024-03-19T01:09:33.028596+0000","last_fullsized":"2024-03-19T01:09:33.028596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:41.004155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804461+0000","last_change":"2024-03-19T01:07:15.482326+0000","last_active":"2024-03-19T01:09:25.804461+0000","last_peered":"2024-03-19T01:09:25.804461+0000","last_clean":"2024-03-19T01:09:25.804461+0000","last_became_active":"2024-03-19T01:07:15.481492+0000","last_became_peered":"2024-03-19T01:07:15.481492+0000","last_unstale":"2024-03-19T01:09:25.804461+0000","last_undegraded":"2024-03-19T01:09:25.804461+0000","last_fullsized":"2024-03-19T01:09:25.804461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:17:31.572104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137621+0000","last_change":"2024-03-19T01:07:19.668853+0000","last_active":"2024-03-19T01:09:26.137621+0000","last_peered":"2024-03-19T01:09:26.137621+0000","last_clean":"2024-03-19T01:09:26.137621+0000","last_became_active":"2024-03-19T01:07:19.668428+0000","last_became_peered":"2024-03-19T01:07:19.668428+0000","last_unstale":"2024-03-19T01:09:26.137621+0000","last_undegraded":"2024-03-19T01:09:26.137621+0000","last_fullsized":"2024-03-19T01:09:26.137621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:50:43.409586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801707+0000","last_change":"2024-03-19T01:07:14.469548+0000","last_active":"2024-03-19T01:09:25.801707+0000","last_peered":"2024-03-19T01:09:25.801707+0000","last_clean":"2024-03-19T01:09:25.801707+0000","last_became_active":"2024-03-19T01:07:14.469200+0000","last_became_peered":"2024-03-19T01:07:14.469200+0000","last_unstale":"2024-03-19T01:09:25.801707+0000","last_undegraded":"2024-03-19T01:09:25.801707+0000","last_fullsized":"2024-03-19T01:09:25.801707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:41:53.179343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801883+0000","last_change":"2024-03-19T01:07:15.495300+0000","last_active":"2024-03-19T01:09:25.801883+0000","last_peered":"2024-03-19T01:09:25.801883+0000","last_clean":"2024-03-19T01:09:25.801883+0000","last_became_active":"2024-03-19T01:07:15.494938+0000","last_became_peered":"2024-03-19T01:07:15.494938+0000","last_unstale":"2024-03-19T01:09:25.801883+0000","last_undegraded":"2024-03-19T01:09:25.801883+0000","last_fullsized":"2024-03-19T01:09:25.801883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:50:38.855857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804281+0000","last_change":"2024-03-19T01:07:19.693261+0000","last_active":"2024-03-19T01:09:25.804281+0000","last_peered":"2024-03-19T01:09:25.804281+0000","last_clean":"2024-03-19T01:09:25.804281+0000","last_became_active":"2024-03-19T01:07:19.692598+0000","last_became_peered":"2024-03-19T01:07:19.692598+0000","last_unstale":"2024-03-19T01:09:25.804281+0000","last_undegraded":"2024-03-19T01:09:25.804281+0000","last_fullsized":"2024-03-19T01:09:25.804281+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:43:29.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":192,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.039962+0000","last_change":"2024-03-19T01:07:14.481569+0000","last_active":"2024-03-19T01:10:21.039962+0000","last_peered":"2024-03-19T01:10:21.039962+0000","last_clean":"2024-03-19T01:10:21.039962+0000","last_became_active":"2024-03-19T01:07:14.481209+0000","last_became_peered":"2024-03-19T01:07:14.481209+0000","last_unstale":"2024-03-19T01:10:21.039962+0000","last_undegraded":"2024-03-19T01:10:21.039962+0000","last_fullsized":"2024-03-19T01:10:21.039962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:39.411207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":154,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.801654+0000","last_change":"2024-03-19T01:07:15.496852+0000","last_active":"2024-03-19T01:09:25.801654+0000","last_peered":"2024-03-19T01:09:25.801654+0000","last_clean":"2024-03-19T01:09:25.801654+0000","last_became_active":"2024-03-19T01:07:15.496455+0000","last_became_peered":"2024-03-19T01:07:15.496455+0000","last_unstale":"2024-03-19T01:09:25.801654+0000","last_undegraded":"2024-03-19T01:09:25.801654+0000","last_fullsized":"2024-03-19T01:09:25.801654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:12:43.419496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.805138+0000","last_change":"2024-03-19T01:07:19.692856+0000","last_active":"2024-03-19T01:09:25.805138+0000","last_peered":"2024-03-19T01:09:25.805138+0000","last_clean":"2024-03-19T01:09:25.805138+0000","last_became_active":"2024-03-19T01:07:19.690712+0000","last_became_peered":"2024-03-19T01:07:19.690712+0000","last_unstale":"2024-03-19T01:09:25.805138+0000","last_undegraded":"2024-03-19T01:09:25.805138+0000","last_fullsized":"2024-03-19T01:09:25.805138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:29:28.473062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798036+0000","last_change":"2024-03-19T01:07:15.474375+0000","last_active":"2024-03-19T01:09:25.798036+0000","last_peered":"2024-03-19T01:09:25.798036+0000","last_clean":"2024-03-19T01:09:25.798036+0000","last_became_active":"2024-03-19T01:07:15.473427+0000","last_became_peered":"2024-03-19T01:07:15.473427+0000","last_unstale":"2024-03-19T01:09:25.798036+0000","last_undegraded":"2024-03-19T01:09:25.798036+0000","last_fullsized":"2024-03-19T01:09:25.798036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:42:22.235054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'7","reported_seq":194,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:21.004294+0000","last_change":"2024-03-19T01:07:14.490924+0000","last_active":"2024-03-19T01:10:21.004294+0000","last_peered":"2024-03-19T01:10:21.004294+0000","last_clean":"2024-03-19T01:10:21.004294+0000","last_became_active":"2024-03-19T01:07:14.490621+0000","last_became_peered":"2024-03-19T01:07:14.490621+0000","last_unstale":"2024-03-19T01:10:21.004294+0000","last_undegraded":"2024-03-19T01:10:21.004294+0000","last_fullsized":"2024-03-19T01:10:21.004294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:02:52.254138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":155,"num_read_kb":155,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962466+0000","last_change":"2024-03-19T01:08:24.805196+0000","last_active":"2024-03-19T01:09:25.962466+0000","last_peered":"2024-03-19T01:09:25.962466+0000","last_clean":"2024-03-19T01:09:25.962466+0000","last_became_active":"2024-03-19T01:08:24.800225+0000","last_became_peered":"2024-03-19T01:08:24.800225+0000","last_unstale":"2024-03-19T01:09:25.962466+0000","last_undegraded":"2024-03-19T01:09:25.962466+0000","last_fullsized":"2024-03-19T01:09:25.962466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:50:02.322846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914787+0000","last_change":"2024-03-19T01:07:15.493218+0000","last_active":"2024-03-19T01:09:25.914787+0000","last_peered":"2024-03-19T01:09:25.914787+0000","last_clean":"2024-03-19T01:09:25.914787+0000","last_became_active":"2024-03-19T01:07:15.492751+0000","last_became_peered":"2024-03-19T01:07:15.492751+0000","last_unstale":"2024-03-19T01:09:25.914787+0000","last_undegraded":"2024-03-19T01:09:25.914787+0000","last_fullsized":"2024-03-19T01:09:25.914787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:48:32.105214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800099+0000","last_change":"2024-03-19T01:07:14.473349+0000","last_active":"2024-03-19T01:09:25.800099+0000","last_peered":"2024-03-19T01:09:25.800099+0000","last_clean":"2024-03-19T01:09:25.800099+0000","last_became_active":"2024-03-19T01:07:14.473014+0000","last_became_peered":"2024-03-19T01:07:14.473014+0000","last_unstale":"2024-03-19T01:09:25.800099+0000","last_undegraded":"2024-03-19T01:09:25.800099+0000","last_fullsized":"2024-03-19T01:09:25.800099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:27:34.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.961963+0000","last_change":"2024-03-19T01:05:56.013594+0000","last_active":"2024-03-19T01:09:25.961963+0000","last_peered":"2024-03-19T01:09:25.961963+0000","last_clean":"2024-03-19T01:09:25.961963+0000","last_became_active":"2024-03-19T01:05:56.013084+0000","last_became_peered":"2024-03-19T01:05:56.013084+0000","last_unstale":"2024-03-19T01:09:25.961963+0000","last_undegraded":"2024-03-19T01:09:25.961963+0000","last_fullsized":"2024-03-19T01:09:25.961963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:02:06.532343+0000","last_clean_scrub_stamp":"2024-03-19T01:02:06.532343+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:23.440161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798358+0000","last_change":"2024-03-19T01:07:19.691111+0000","last_active":"2024-03-19T01:09:25.798358+0000","last_peered":"2024-03-19T01:09:25.798358+0000","last_clean":"2024-03-19T01:09:25.798358+0000","last_became_active":"2024-03-19T01:07:19.690618+0000","last_became_peered":"2024-03-19T01:07:19.690618+0000","last_unstale":"2024-03-19T01:09:25.798358+0000","last_undegraded":"2024-03-19T01:09:25.798358+0000","last_fullsized":"2024-03-19T01:09:25.798358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:17.857394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.802402+0000","last_change":"2024-03-19T01:08:24.801826+0000","last_active":"2024-03-19T01:09:25.802402+0000","last_peered":"2024-03-19T01:09:25.802402+0000","last_clean":"2024-03-19T01:09:25.802402+0000","last_became_active":"2024-03-19T01:08:24.801411+0000","last_became_peered":"2024-03-19T01:08:24.801411+0000","last_unstale":"2024-03-19T01:09:25.802402+0000","last_undegraded":"2024-03-19T01:09:25.802402+0000","last_fullsized":"2024-03-19T01:09:25.802402+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:34:31.798045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962068+0000","last_change":"2024-03-19T01:07:14.474287+0000","last_active":"2024-03-19T01:09:25.962068+0000","last_peered":"2024-03-19T01:09:25.962068+0000","last_clean":"2024-03-19T01:09:25.962068+0000","last_became_active":"2024-03-19T01:07:14.473921+0000","last_became_peered":"2024-03-19T01:07:14.473921+0000","last_unstale":"2024-03-19T01:09:25.962068+0000","last_undegraded":"2024-03-19T01:09:25.962068+0000","last_fullsized":"2024-03-19T01:09:25.962068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:49:49.322029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804258+0000","last_change":"2024-03-19T01:07:19.685331+0000","last_active":"2024-03-19T01:09:25.804258+0000","last_peered":"2024-03-19T01:09:25.804258+0000","last_clean":"2024-03-19T01:09:25.804258+0000","last_became_active":"2024-03-19T01:07:19.684876+0000","last_became_peered":"2024-03-19T01:07:19.684876+0000","last_unstale":"2024-03-19T01:09:25.804258+0000","last_undegraded":"2024-03-19T01:09:25.804258+0000","last_fullsized":"2024-03-19T01:09:25.804258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:39.575348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137876+0000","last_change":"2024-03-19T01:07:14.481019+0000","last_active":"2024-03-19T01:09:26.137876+0000","last_peered":"2024-03-19T01:09:26.137876+0000","last_clean":"2024-03-19T01:09:26.137876+0000","last_became_active":"2024-03-19T01:07:14.480666+0000","last_became_peered":"2024-03-19T01:07:14.480666+0000","last_unstale":"2024-03-19T01:09:26.137876+0000","last_undegraded":"2024-03-19T01:09:26.137876+0000","last_fullsized":"2024-03-19T01:09:26.137876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:13:33.669690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804528+0000","last_change":"2024-03-19T01:07:15.490747+0000","last_active":"2024-03-19T01:09:25.804528+0000","last_peered":"2024-03-19T01:09:25.804528+0000","last_clean":"2024-03-19T01:09:25.804528+0000","last_became_active":"2024-03-19T01:07:15.486244+0000","last_became_peered":"2024-03-19T01:07:15.486244+0000","last_unstale":"2024-03-19T01:09:25.804528+0000","last_undegraded":"2024-03-19T01:09:25.804528+0000","last_fullsized":"2024-03-19T01:09:25.804528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:43.212972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914397+0000","last_change":"2024-03-19T01:07:19.684804+0000","last_active":"2024-03-19T01:09:25.914397+0000","last_peered":"2024-03-19T01:09:25.914397+0000","last_clean":"2024-03-19T01:09:25.914397+0000","last_became_active":"2024-03-19T01:07:19.683191+0000","last_became_peered":"2024-03-19T01:07:19.683191+0000","last_unstale":"2024-03-19T01:09:25.914397+0000","last_undegraded":"2024-03-19T01:09:25.914397+0000","last_fullsized":"2024-03-19T01:09:25.914397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:13:02.015499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'16","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:16.639078+0000","last_change":"2024-03-19T01:07:14.479282+0000","last_active":"2024-03-19T01:10:16.639078+0000","last_peered":"2024-03-19T01:10:16.639078+0000","last_clean":"2024-03-19T01:10:16.639078+0000","last_became_active":"2024-03-19T01:07:14.478876+0000","last_became_peered":"2024-03-19T01:07:14.478876+0000","last_unstale":"2024-03-19T01:10:16.639078+0000","last_undegraded":"2024-03-19T01:10:16.639078+0000","last_fullsized":"2024-03-19T01:10:16.639078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:54.179680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915652+0000","last_change":"2024-03-19T01:07:15.493498+0000","last_active":"2024-03-19T01:09:25.915652+0000","last_peered":"2024-03-19T01:09:25.915652+0000","last_clean":"2024-03-19T01:09:25.915652+0000","last_became_active":"2024-03-19T01:07:15.493110+0000","last_became_peered":"2024-03-19T01:07:15.493110+0000","last_unstale":"2024-03-19T01:09:25.915652+0000","last_undegraded":"2024-03-19T01:09:25.915652+0000","last_fullsized":"2024-03-19T01:09:25.915652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:55:23.608596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.913805+0000","last_change":"2024-03-19T01:07:19.671232+0000","last_active":"2024-03-19T01:09:25.913805+0000","last_peered":"2024-03-19T01:09:25.913805+0000","last_clean":"2024-03-19T01:09:25.913805+0000","last_became_active":"2024-03-19T01:07:19.670818+0000","last_became_peered":"2024-03-19T01:07:19.670818+0000","last_unstale":"2024-03-19T01:09:25.913805+0000","last_undegraded":"2024-03-19T01:09:25.913805+0000","last_fullsized":"2024-03-19T01:09:25.913805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:57:20.134372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800619+0000","last_change":"2024-03-19T01:07:14.473568+0000","last_active":"2024-03-19T01:09:25.800619+0000","last_peered":"2024-03-19T01:09:25.800619+0000","last_clean":"2024-03-19T01:09:25.800619+0000","last_became_active":"2024-03-19T01:07:14.472555+0000","last_became_peered":"2024-03-19T01:07:14.472555+0000","last_unstale":"2024-03-19T01:09:25.800619+0000","last_undegraded":"2024-03-19T01:09:25.800619+0000","last_fullsized":"2024-03-19T01:09:25.800619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:28:01.541228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156691+0000","last_change":"2024-03-19T01:07:15.499905+0000","last_active":"2024-03-19T01:09:26.156691+0000","last_peered":"2024-03-19T01:09:26.156691+0000","last_clean":"2024-03-19T01:09:26.156691+0000","last_became_active":"2024-03-19T01:07:15.499560+0000","last_became_peered":"2024-03-19T01:07:15.499560+0000","last_unstale":"2024-03-19T01:09:26.156691+0000","last_undegraded":"2024-03-19T01:09:26.156691+0000","last_fullsized":"2024-03-19T01:09:26.156691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:06:26.531100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800426+0000","last_change":"2024-03-19T01:07:19.690310+0000","last_active":"2024-03-19T01:09:25.800426+0000","last_peered":"2024-03-19T01:09:25.800426+0000","last_clean":"2024-03-19T01:09:25.800426+0000","last_became_active":"2024-03-19T01:07:19.689877+0000","last_became_peered":"2024-03-19T01:07:19.689877+0000","last_unstale":"2024-03-19T01:09:25.800426+0000","last_undegraded":"2024-03-19T01:09:25.800426+0000","last_fullsized":"2024-03-19T01:09:25.800426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:06:12.730258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154081+0000","last_change":"2024-03-19T01:07:14.466267+0000","last_active":"2024-03-19T01:09:26.154081+0000","last_peered":"2024-03-19T01:09:26.154081+0000","last_clean":"2024-03-19T01:09:26.154081+0000","last_became_active":"2024-03-19T01:07:14.465944+0000","last_became_peered":"2024-03-19T01:07:14.465944+0000","last_unstale":"2024-03-19T01:09:26.154081+0000","last_undegraded":"2024-03-19T01:09:26.154081+0000","last_fullsized":"2024-03-19T01:09:26.154081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:38:40.957200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137477+0000","last_change":"2024-03-19T01:07:15.475690+0000","last_active":"2024-03-19T01:09:26.137477+0000","last_peered":"2024-03-19T01:09:26.137477+0000","last_clean":"2024-03-19T01:09:26.137477+0000","last_became_active":"2024-03-19T01:07:15.475342+0000","last_became_peered":"2024-03-19T01:07:15.475342+0000","last_unstale":"2024-03-19T01:09:26.137477+0000","last_undegraded":"2024-03-19T01:09:26.137477+0000","last_fullsized":"2024-03-19T01:09:26.137477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:30:30.842462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137994+0000","last_change":"2024-03-19T01:08:24.797680+0000","last_active":"2024-03-19T01:09:26.137994+0000","last_peered":"2024-03-19T01:09:26.137994+0000","last_clean":"2024-03-19T01:09:26.137994+0000","last_became_active":"2024-03-19T01:08:24.797194+0000","last_became_peered":"2024-03-19T01:08:24.797194+0000","last_unstale":"2024-03-19T01:09:26.137994+0000","last_undegraded":"2024-03-19T01:09:26.137994+0000","last_fullsized":"2024-03-19T01:09:26.137994+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:52:49.800726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914650+0000","last_change":"2024-03-19T01:07:14.487907+0000","last_active":"2024-03-19T01:09:25.914650+0000","last_peered":"2024-03-19T01:09:25.914650+0000","last_clean":"2024-03-19T01:09:25.914650+0000","last_became_active":"2024-03-19T01:07:14.487618+0000","last_became_peered":"2024-03-19T01:07:14.487618+0000","last_unstale":"2024-03-19T01:09:25.914650+0000","last_undegraded":"2024-03-19T01:09:25.914650+0000","last_fullsized":"2024-03-19T01:09:25.914650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:22:49.218797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804225+0000","last_change":"2024-03-19T01:07:15.492103+0000","last_active":"2024-03-19T01:09:25.804225+0000","last_peered":"2024-03-19T01:09:25.804225+0000","last_clean":"2024-03-19T01:09:25.804225+0000","last_became_active":"2024-03-19T01:07:15.491712+0000","last_became_peered":"2024-03-19T01:07:15.491712+0000","last_unstale":"2024-03-19T01:09:25.804225+0000","last_undegraded":"2024-03-19T01:09:25.804225+0000","last_fullsized":"2024-03-19T01:09:25.804225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:40:13.959986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.799962+0000","last_change":"2024-03-19T01:07:19.677392+0000","last_active":"2024-03-19T01:09:25.799962+0000","last_peered":"2024-03-19T01:09:25.799962+0000","last_clean":"2024-03-19T01:09:25.799962+0000","last_became_active":"2024-03-19T01:07:19.677024+0000","last_became_peered":"2024-03-19T01:07:19.677024+0000","last_unstale":"2024-03-19T01:09:25.799962+0000","last_undegraded":"2024-03-19T01:09:25.799962+0000","last_fullsized":"2024-03-19T01:09:25.799962+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:01:43.404902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804247+0000","last_change":"2024-03-19T01:08:24.805363+0000","last_active":"2024-03-19T01:09:25.804247+0000","last_peered":"2024-03-19T01:09:25.804247+0000","last_clean":"2024-03-19T01:09:25.804247+0000","last_became_active":"2024-03-19T01:08:24.800810+0000","last_became_peered":"2024-03-19T01:08:24.800810+0000","last_unstale":"2024-03-19T01:09:25.804247+0000","last_undegraded":"2024-03-19T01:09:25.804247+0000","last_fullsized":"2024-03-19T01:09:25.804247+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:32:43.608140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915314+0000","last_change":"2024-03-19T01:07:15.505842+0000","last_active":"2024-03-19T01:09:25.915314+0000","last_peered":"2024-03-19T01:09:25.915314+0000","last_clean":"2024-03-19T01:09:25.915314+0000","last_became_active":"2024-03-19T01:07:15.505425+0000","last_became_peered":"2024-03-19T01:07:15.505425+0000","last_unstale":"2024-03-19T01:09:25.915314+0000","last_undegraded":"2024-03-19T01:09:25.915314+0000","last_fullsized":"2024-03-19T01:09:25.915314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:28.119864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.800240+0000","last_change":"2024-03-19T01:07:19.683113+0000","last_active":"2024-03-19T01:09:25.800240+0000","last_peered":"2024-03-19T01:09:25.800240+0000","last_clean":"2024-03-19T01:09:25.800240+0000","last_became_active":"2024-03-19T01:07:19.682702+0000","last_became_peered":"2024-03-19T01:07:19.682702+0000","last_unstale":"2024-03-19T01:09:25.800240+0000","last_undegraded":"2024-03-19T01:09:25.800240+0000","last_fullsized":"2024-03-19T01:09:25.800240+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:42.866408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.137616+0000","last_change":"2024-03-19T01:07:14.476562+0000","last_active":"2024-03-19T01:09:26.137616+0000","last_peered":"2024-03-19T01:09:26.137616+0000","last_clean":"2024-03-19T01:09:26.137616+0000","last_became_active":"2024-03-19T01:07:14.476185+0000","last_became_peered":"2024-03-19T01:07:14.476185+0000","last_unstale":"2024-03-19T01:09:26.137616+0000","last_undegraded":"2024-03-19T01:09:26.137616+0000","last_fullsized":"2024-03-19T01:09:26.137616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:35:19.860443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962373+0000","last_change":"2024-03-19T01:07:15.487163+0000","last_active":"2024-03-19T01:09:25.962373+0000","last_peered":"2024-03-19T01:09:25.962373+0000","last_clean":"2024-03-19T01:09:25.962373+0000","last_became_active":"2024-03-19T01:07:15.486845+0000","last_became_peered":"2024-03-19T01:07:15.486845+0000","last_unstale":"2024-03-19T01:09:25.962373+0000","last_undegraded":"2024-03-19T01:09:25.962373+0000","last_fullsized":"2024-03-19T01:09:25.962373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:39:09.389411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154143+0000","last_change":"2024-03-19T01:07:19.692768+0000","last_active":"2024-03-19T01:09:26.154143+0000","last_peered":"2024-03-19T01:09:26.154143+0000","last_clean":"2024-03-19T01:09:26.154143+0000","last_became_active":"2024-03-19T01:07:19.692382+0000","last_became_peered":"2024-03-19T01:07:19.692382+0000","last_unstale":"2024-03-19T01:09:26.154143+0000","last_undegraded":"2024-03-19T01:09:26.154143+0000","last_fullsized":"2024-03-19T01:09:26.154143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:26.804938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":189,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:10:18.964737+0000","last_change":"2024-03-19T01:07:14.472844+0000","last_active":"2024-03-19T01:10:18.964737+0000","last_peered":"2024-03-19T01:10:18.964737+0000","last_clean":"2024-03-19T01:10:18.964737+0000","last_became_active":"2024-03-19T01:07:14.472389+0000","last_became_peered":"2024-03-19T01:07:14.472389+0000","last_unstale":"2024-03-19T01:10:18.964737+0000","last_undegraded":"2024-03-19T01:10:18.964737+0000","last_fullsized":"2024-03-19T01:10:18.964737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:26:31.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962639+0000","last_change":"2024-03-19T01:07:15.482033+0000","last_active":"2024-03-19T01:09:25.962639+0000","last_peered":"2024-03-19T01:09:25.962639+0000","last_clean":"2024-03-19T01:09:25.962639+0000","last_became_active":"2024-03-19T01:07:15.481671+0000","last_became_peered":"2024-03-19T01:07:15.481671+0000","last_unstale":"2024-03-19T01:09:25.962639+0000","last_undegraded":"2024-03-19T01:09:25.962639+0000","last_fullsized":"2024-03-19T01:09:25.962639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:54:09.969374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.155872+0000","last_change":"2024-03-19T01:08:24.793574+0000","last_active":"2024-03-19T01:09:26.155872+0000","last_peered":"2024-03-19T01:09:26.155872+0000","last_clean":"2024-03-19T01:09:26.155872+0000","last_became_active":"2024-03-19T01:08:24.793012+0000","last_became_peered":"2024-03-19T01:08:24.793012+0000","last_unstale":"2024-03-19T01:09:26.155872+0000","last_undegraded":"2024-03-19T01:09:26.155872+0000","last_fullsized":"2024-03-19T01:09:26.155872+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:04:53.611271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.798450+0000","last_change":"2024-03-19T01:07:14.481677+0000","last_active":"2024-03-19T01:09:25.798450+0000","last_peered":"2024-03-19T01:09:25.798450+0000","last_clean":"2024-03-19T01:09:25.798450+0000","last_became_active":"2024-03-19T01:07:14.481138+0000","last_became_peered":"2024-03-19T01:07:14.481138+0000","last_unstale":"2024-03-19T01:09:25.798450+0000","last_undegraded":"2024-03-19T01:09:25.798450+0000","last_fullsized":"2024-03-19T01:09:25.798450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:52:33.105816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962431+0000","last_change":"2024-03-19T01:07:15.498352+0000","last_active":"2024-03-19T01:09:25.962431+0000","last_peered":"2024-03-19T01:09:25.962431+0000","last_clean":"2024-03-19T01:09:25.962431+0000","last_became_active":"2024-03-19T01:07:15.498040+0000","last_became_peered":"2024-03-19T01:07:15.498040+0000","last_unstale":"2024-03-19T01:09:25.962431+0000","last_undegraded":"2024-03-19T01:09:25.962431+0000","last_fullsized":"2024-03-19T01:09:25.962431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:57:37.571753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.797775+0000","last_change":"2024-03-19T01:07:19.673291+0000","last_active":"2024-03-19T01:09:25.797775+0000","last_peered":"2024-03-19T01:09:25.797775+0000","last_clean":"2024-03-19T01:09:25.797775+0000","last_became_active":"2024-03-19T01:07:19.672704+0000","last_became_peered":"2024-03-19T01:07:19.672704+0000","last_unstale":"2024-03-19T01:09:25.797775+0000","last_undegraded":"2024-03-19T01:09:25.797775+0000","last_fullsized":"2024-03-19T01:09:25.797775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:18.635391+0000","last_clean_scrub_stamp":"2024-03-19T01:07:18.635391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:01:42.019812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804917+0000","last_change":"2024-03-19T01:07:14.489556+0000","last_active":"2024-03-19T01:09:25.804917+0000","last_peered":"2024-03-19T01:09:25.804917+0000","last_clean":"2024-03-19T01:09:25.804917+0000","last_became_active":"2024-03-19T01:07:14.489201+0000","last_became_peered":"2024-03-19T01:07:14.489201+0000","last_unstale":"2024-03-19T01:09:25.804917+0000","last_undegraded":"2024-03-19T01:09:25.804917+0000","last_fullsized":"2024-03-19T01:09:25.804917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:13.418292+0000","last_clean_scrub_stamp":"2024-03-19T01:07:13.418292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:28:36.516185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.962688+0000","last_change":"2024-03-19T01:07:15.477727+0000","last_active":"2024-03-19T01:09:25.962688+0000","last_peered":"2024-03-19T01:09:25.962688+0000","last_clean":"2024-03-19T01:09:25.962688+0000","last_became_active":"2024-03-19T01:07:15.477388+0000","last_became_peered":"2024-03-19T01:07:15.477388+0000","last_unstale":"2024-03-19T01:09:25.962688+0000","last_undegraded":"2024-03-19T01:09:25.962688+0000","last_fullsized":"2024-03-19T01:09:25.962688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:31:30.567326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804232+0000","last_change":"2024-03-19T01:07:15.479008+0000","last_active":"2024-03-19T01:09:25.804232+0000","last_peered":"2024-03-19T01:09:25.804232+0000","last_clean":"2024-03-19T01:09:25.804232+0000","last_became_active":"2024-03-19T01:07:15.478657+0000","last_became_peered":"2024-03-19T01:07:15.478657+0000","last_unstale":"2024-03-19T01:09:25.804232+0000","last_undegraded":"2024-03-19T01:09:25.804232+0000","last_fullsized":"2024-03-19T01:09:25.804232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:42:46.970012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914058+0000","last_change":"2024-03-19T01:08:24.789796+0000","last_active":"2024-03-19T01:09:25.914058+0000","last_peered":"2024-03-19T01:09:25.914058+0000","last_clean":"2024-03-19T01:09:25.914058+0000","last_became_active":"2024-03-19T01:08:24.789444+0000","last_became_peered":"2024-03-19T01:08:24.789444+0000","last_unstale":"2024-03-19T01:09:25.914058+0000","last_undegraded":"2024-03-19T01:09:25.914058+0000","last_fullsized":"2024-03-19T01:09:25.914058+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:41.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.913879+0000","last_change":"2024-03-19T01:07:15.495292+0000","last_active":"2024-03-19T01:09:25.913879+0000","last_peered":"2024-03-19T01:09:25.913879+0000","last_clean":"2024-03-19T01:09:25.913879+0000","last_became_active":"2024-03-19T01:07:15.494946+0000","last_became_peered":"2024-03-19T01:07:15.494946+0000","last_unstale":"2024-03-19T01:09:25.913879+0000","last_undegraded":"2024-03-19T01:09:25.913879+0000","last_fullsized":"2024-03-19T01:09:25.913879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:13:39.012221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.156089+0000","last_change":"2024-03-19T01:07:15.499816+0000","last_active":"2024-03-19T01:09:26.156089+0000","last_peered":"2024-03-19T01:09:26.156089+0000","last_clean":"2024-03-19T01:09:26.156089+0000","last_became_active":"2024-03-19T01:07:15.499031+0000","last_became_peered":"2024-03-19T01:07:15.499031+0000","last_unstale":"2024-03-19T01:09:26.156089+0000","last_undegraded":"2024-03-19T01:09:26.156089+0000","last_fullsized":"2024-03-19T01:09:26.156089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:34:25.344092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804473+0000","last_change":"2024-03-19T01:07:15.501526+0000","last_active":"2024-03-19T01:09:25.804473+0000","last_peered":"2024-03-19T01:09:25.804473+0000","last_clean":"2024-03-19T01:09:25.804473+0000","last_became_active":"2024-03-19T01:07:15.500452+0000","last_became_peered":"2024-03-19T01:07:15.500452+0000","last_unstale":"2024-03-19T01:09:25.804473+0000","last_undegraded":"2024-03-19T01:09:25.804473+0000","last_fullsized":"2024-03-19T01:09:25.804473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:35:33.075780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804632+0000","last_change":"2024-03-19T01:07:15.477649+0000","last_active":"2024-03-19T01:09:25.804632+0000","last_peered":"2024-03-19T01:09:25.804632+0000","last_clean":"2024-03-19T01:09:25.804632+0000","last_became_active":"2024-03-19T01:07:15.477193+0000","last_became_peered":"2024-03-19T01:07:15.477193+0000","last_unstale":"2024-03-19T01:09:25.804632+0000","last_undegraded":"2024-03-19T01:09:25.804632+0000","last_fullsized":"2024-03-19T01:09:25.804632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:36.126642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914305+0000","last_change":"2024-03-19T01:07:15.481920+0000","last_active":"2024-03-19T01:09:25.914305+0000","last_peered":"2024-03-19T01:09:25.914305+0000","last_clean":"2024-03-19T01:09:25.914305+0000","last_became_active":"2024-03-19T01:07:15.481290+0000","last_became_peered":"2024-03-19T01:07:15.481290+0000","last_unstale":"2024-03-19T01:09:25.914305+0000","last_undegraded":"2024-03-19T01:09:25.914305+0000","last_fullsized":"2024-03-19T01:09:25.914305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:37:58.028647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.915760+0000","last_change":"2024-03-19T01:07:15.502305+0000","last_active":"2024-03-19T01:09:25.915760+0000","last_peered":"2024-03-19T01:09:25.915760+0000","last_clean":"2024-03-19T01:09:25.915760+0000","last_became_active":"2024-03-19T01:07:15.501491+0000","last_became_peered":"2024-03-19T01:07:15.501491+0000","last_unstale":"2024-03-19T01:09:25.915760+0000","last_undegraded":"2024-03-19T01:09:25.915760+0000","last_fullsized":"2024-03-19T01:09:25.915760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:01.999538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.139189+0000","last_change":"2024-03-19T01:09:26.138802+0000","last_active":"2024-03-19T01:09:26.139189+0000","last_peered":"2024-03-19T01:09:26.139189+0000","last_clean":"2024-03-19T01:09:26.139189+0000","last_became_active":"2024-03-19T01:09:26.138423+0000","last_became_peered":"2024-03-19T01:09:26.138423+0000","last_unstale":"2024-03-19T01:09:26.139189+0000","last_undegraded":"2024-03-19T01:09:26.139189+0000","last_fullsized":"2024-03-19T01:09:26.139189+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:51:23.293922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.914829+0000","last_change":"2024-03-19T01:07:15.505022+0000","last_active":"2024-03-19T01:09:25.914829+0000","last_peered":"2024-03-19T01:09:25.914829+0000","last_clean":"2024-03-19T01:09:25.914829+0000","last_became_active":"2024-03-19T01:07:15.503684+0000","last_became_peered":"2024-03-19T01:07:15.503684+0000","last_unstale":"2024-03-19T01:09:25.914829+0000","last_undegraded":"2024-03-19T01:09:25.914829+0000","last_fullsized":"2024-03-19T01:09:25.914829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:56:47.580267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:26.154341+0000","last_change":"2024-03-19T01:07:15.495634+0000","last_active":"2024-03-19T01:09:26.154341+0000","last_peered":"2024-03-19T01:09:26.154341+0000","last_clean":"2024-03-19T01:09:26.154341+0000","last_became_active":"2024-03-19T01:07:15.495206+0000","last_became_peered":"2024-03-19T01:07:15.495206+0000","last_unstale":"2024-03-19T01:09:26.154341+0000","last_undegraded":"2024-03-19T01:09:26.154341+0000","last_fullsized":"2024-03-19T01:09:26.154341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:30:29.965731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804363+0000","last_change":"2024-03-19T01:07:15.501480+0000","last_active":"2024-03-19T01:09:25.804363+0000","last_peered":"2024-03-19T01:09:25.804363+0000","last_clean":"2024-03-19T01:09:25.804363+0000","last_became_active":"2024-03-19T01:07:15.501078+0000","last_became_peered":"2024-03-19T01:07:15.501078+0000","last_unstale":"2024-03-19T01:09:25.804363+0000","last_undegraded":"2024-03-19T01:09:25.804363+0000","last_fullsized":"2024-03-19T01:09:25.804363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:40:18.767874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-19T01:09:25.804424+0000","last_change":"2024-03-19T01:07:15.480358+0000","last_active":"2024-03-19T01:09:25.804424+0000","last_peered":"2024-03-19T01:09:25.804424+0000","last_clean":"2024-03-19T01:09:25.804424+0000","last_became_active":"2024-03-19T01:07:15.479979+0000","last_became_peered":"2024-03-19T01:07:15.479979+0000","last_unstale":"2024-03-19T01:09:25.804424+0000","last_undegraded":"2024-03-19T01:09:25.804424+0000","last_fullsized":"2024-03-19T01:09:25.804424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:07:14.445184+0000","last_clean_scrub_stamp":"2024-03-19T01:07:14.445184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:12:54.598435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247132,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78447,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1575,"num_read_kb":875,"num_write":264,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":235341,"data_compressed":5469,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383731,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":520302,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645369,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32336,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712816,"statfs":{"total":95995035648,"available":95961923584,"internally_reserved":0,"allocated":1187840,"data_stored":1182848,"data_compressed":64112,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808894,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":524328,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070532,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":520610,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332169,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1118336,"data_compressed":62289,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495695,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":516127,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":587562,"data_compressed":28338,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":517192,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":587391,"data_compressed":28338,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":515444,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":521039,"data_compressed":26515,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757334,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":1108202,"data_compressed":62289,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34131,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35774,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72548,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72377,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":73370,"data_compressed":1823,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T01:10:25.107 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:10:25.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:24 smithi069 ceph-mon[31727]: pgmap v413: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-03-19T01:10:25.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:24 smithi086 ceph-mon[66829]: pgmap v413: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-03-19T01:10:25.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:24 smithi163 ceph-mon[67428]: pgmap v413: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-03-19T01:10:25.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.338+0000 7fba78000640 1 -- 172.21.15.69:0/2539469169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70101450 msgr2=0x7fba701018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:25.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.338+0000 7fba78000640 1 --2- 172.21.15.69:0/2539469169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70101450 0x7fba701018d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fba68009900 tx=0x7fba6802bcc0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.338+0000 7fba78000640 1 -- 172.21.15.69:0/2539469169 shutdown_connections 2024-03-19T01:10:25.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.338+0000 7fba78000640 1 --2- 172.21.15.69:0/2539469169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70101450 0x7fba701018d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.338+0000 7fba78000640 1 --2- 172.21.15.69:0/2539469169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba70100ab0 0x7fba70100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.338+0000 7fba78000640 1 --2- 172.21.15.69:0/2539469169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba70100150 0x7fba70100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.338+0000 7fba78000640 1 -- 172.21.15.69:0/2539469169 >> 172.21.15.69:0/2539469169 conn(0x7fba700fbcd0 msgr2=0x7fba700fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:25.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.338+0000 7fba78000640 1 -- 172.21.15.69:0/2539469169 shutdown_connections 2024-03-19T01:10:25.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.338+0000 7fba78000640 1 -- 172.21.15.69:0/2539469169 wait complete. 2024-03-19T01:10:25.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.340+0000 7fba78000640 1 Processor -- start 2024-03-19T01:10:25.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.340+0000 7fba78000640 1 -- start start 2024-03-19T01:10:25.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba78000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba70100150 0x7fba70199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:25.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba78000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70100ab0 0x7fba7019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:25.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba78000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba70101450 0x7fba701a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:25.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba78000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010f6e0 con 0x7fba70101450 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba78000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010f560 con 0x7fba70100150 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba78000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010f860 con 0x7fba70100ab0 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba75600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba70100150 0x7fba70199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba75600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba70100150 0x7fba70199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54734/0 (socket says 172.21.15.69:54734) 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba75600640 1 -- 172.21.15.69:0/146294697 learned_addr learned my addr 172.21.15.69:0/146294697 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba74c00640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70100ab0 0x7fba7019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba76000640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba70101450 0x7fba701a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba75600640 1 -- 172.21.15.69:0/146294697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70100ab0 msgr2=0x7fba7019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba75600640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70100ab0 0x7fba7019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba75600640 1 -- 172.21.15.69:0/146294697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba70101450 msgr2=0x7fba701a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba75600640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba70101450 0x7fba701a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba75600640 1 -- 172.21.15.69:0/146294697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba701a1920 con 0x7fba70100150 2024-03-19T01:10:25.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.341+0000 7fba74c00640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70100ab0 0x7fba7019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:25.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.342+0000 7fba75600640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba70100150 0x7fba70199d10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fba64009d50 tx=0x7fba6400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:25.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.342+0000 7fba6e000640 1 -- 172.21.15.69:0/146294697 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64003b70 con 0x7fba70100150 2024-03-19T01:10:25.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.342+0000 7fba6e000640 1 -- 172.21.15.69:0/146294697 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba64003d10 con 0x7fba70100150 2024-03-19T01:10:25.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.342+0000 7fba6e000640 1 -- 172.21.15.69:0/146294697 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64010a20 con 0x7fba70100150 2024-03-19T01:10:25.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.342+0000 7fba78000640 1 -- 172.21.15.69:0/146294697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba701a1c10 con 0x7fba70100150 2024-03-19T01:10:25.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.342+0000 7fba78000640 1 -- 172.21.15.69:0/146294697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba701a2110 con 0x7fba70100150 2024-03-19T01:10:25.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.344+0000 7fba78000640 1 -- 172.21.15.69:0/146294697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7010afe0 con 0x7fba70100150 2024-03-19T01:10:25.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.347+0000 7fba6e000640 1 -- 172.21.15.69:0/146294697 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fba64003ec0 con 0x7fba70100150 2024-03-19T01:10:25.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.348+0000 7fba6e000640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fba580a2910 0x7fba580a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:25.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.348+0000 7fba6e000640 1 -- 172.21.15.69:0/146294697 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fba640cf750 con 0x7fba70100150 2024-03-19T01:10:25.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.348+0000 7fba74c00640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fba580a2910 0x7fba580a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:25.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.349+0000 7fba74c00640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fba580a2910 0x7fba580a4dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009fc0 tx=0x7fba5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:25.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.352+0000 7fba6e000640 1 -- 172.21.15.69:0/146294697 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fba6409c280 con 0x7fba70100150 2024-03-19T01:10:25.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.645+0000 7fba78000640 1 -- 172.21.15.69:0/146294697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba7002d410 con 0x7fba70100150 2024-03-19T01:10:25.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.647+0000 7fba6e000640 1 -- 172.21.15.69:0/146294697 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24736 (secure 0 0 0) 0x7fba6409bb80 con 0x7fba70100150 2024-03-19T01:10:25.649 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:25.649 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:09:25.783660+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:10:25.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 -- 172.21.15.69:0/146294697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fba580a2910 msgr2=0x7fba580a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:25.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fba580a2910 0x7fba580a4dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009fc0 tx=0x7fba5c005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:25.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 -- 172.21.15.69:0/146294697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba70100150 msgr2=0x7fba70199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:25.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba70100150 0x7fba70199d10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fba64009d50 tx=0x7fba6400c6a0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 -- 172.21.15.69:0/146294697 shutdown_connections 2024-03-19T01:10:25.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fba580a2910 0x7fba580a4dd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba70101450 0x7fba701a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70100ab0 0x7fba7019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 --2- 172.21.15.69:0/146294697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba70100150 0x7fba70199d10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 -- 172.21.15.69:0/146294697 >> 172.21.15.69:0/146294697 conn(0x7fba700fbcd0 msgr2=0x7fba700fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:25.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 -- 172.21.15.69:0/146294697 shutdown_connections 2024-03-19T01:10:25.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.651+0000 7fba78000640 1 -- 172.21.15.69:0/146294697 wait complete. 2024-03-19T01:10:25.672 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:07:13.368130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:07:14.444206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T01:07:18.189146+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T01:10:25.672 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:10:25.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.877+0000 7f3a85000640 1 -- 172.21.15.69:0/2899689543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a80074280 msgr2=0x7f3a80074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:25.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.877+0000 7f3a85000640 1 --2- 172.21.15.69:0/2899689543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a80074280 0x7f3a80074700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009930 tx=0x7f3a6c02bc20 comp rx=0 tx=0).stop 2024-03-19T01:10:25.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.877+0000 7f3a85000640 1 -- 172.21.15.69:0/2899689543 shutdown_connections 2024-03-19T01:10:25.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.877+0000 7f3a85000640 1 --2- 172.21.15.69:0/2899689543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a8010de00 0x7f3a801101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.877+0000 7f3a85000640 1 --2- 172.21.15.69:0/2899689543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a80074280 0x7f3a80074700 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.877+0000 7f3a85000640 1 --2- 172.21.15.69:0/2899689543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a80075860 0x7f3a80073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.877+0000 7f3a85000640 1 -- 172.21.15.69:0/2899689543 >> 172.21.15.69:0/2899689543 conn(0x7f3a800fcda0 msgr2=0x7f3a800ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:25.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.877+0000 7f3a85000640 1 -- 172.21.15.69:0/2899689543 shutdown_connections 2024-03-19T01:10:25.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.877+0000 7f3a85000640 1 -- 172.21.15.69:0/2899689543 wait complete. 2024-03-19T01:10:25.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.878+0000 7f3a85000640 1 Processor -- start 2024-03-19T01:10:25.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.878+0000 7f3a85000640 1 -- start start 2024-03-19T01:10:25.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.878+0000 7f3a85000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a80074280 0x7f3a8019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:25.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a85000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a80075860 0x7f3a8019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:25.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a85000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a8010de00 0x7f3a801a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:25.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a85000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3a80113af0 con 0x7f3a80075860 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a85000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a80113970 con 0x7f3a8010de00 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a85000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3a80113c70 con 0x7f3a80074280 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a7de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a80074280 0x7f3a8019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a7de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a80074280 0x7f3a8019e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42614/0 (socket says 172.21.15.69:42614) 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a7de00640 1 -- 172.21.15.69:0/294780126 learned_addr learned my addr 172.21.15.69:0/294780126 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a7e800640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a8010de00 0x7f3a801a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a7d400640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a80075860 0x7f3a8019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a7de00640 1 -- 172.21.15.69:0/294780126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a8010de00 msgr2=0x7f3a801a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a7de00640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a8010de00 0x7f3a801a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a7de00640 1 -- 172.21.15.69:0/294780126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a80075860 msgr2=0x7f3a8019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a7de00640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a80075860 0x7f3a8019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a7de00640 1 -- 172.21.15.69:0/294780126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a801a5de0 con 0x7f3a80074280 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.879+0000 7f3a7de00640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a80074280 0x7f3a8019e1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000eb90 tx=0x7f3a7000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:25.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.880+0000 7f3a76a00640 1 -- 172.21.15.69:0/294780126 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70004d20 con 0x7f3a80074280 2024-03-19T01:10:25.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.880+0000 7f3a76a00640 1 -- 172.21.15.69:0/294780126 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a7000ee90 con 0x7f3a80074280 2024-03-19T01:10:25.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.880+0000 7f3a76a00640 1 -- 172.21.15.69:0/294780126 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70010830 con 0x7f3a80074280 2024-03-19T01:10:25.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.880+0000 7f3a85000640 1 -- 172.21.15.69:0/294780126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a801a60d0 con 0x7f3a80074280 2024-03-19T01:10:25.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.880+0000 7f3a85000640 1 -- 172.21.15.69:0/294780126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a801a65a0 con 0x7f3a80074280 2024-03-19T01:10:25.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.881+0000 7f3a85000640 1 -- 172.21.15.69:0/294780126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a801037b0 con 0x7f3a80074280 2024-03-19T01:10:25.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.883+0000 7f3a76a00640 1 -- 172.21.15.69:0/294780126 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f3a70005020 con 0x7f3a80074280 2024-03-19T01:10:25.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.884+0000 7f3a76a00640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3a640a2910 0x7f3a640a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:25.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.884+0000 7f3a76a00640 1 -- 172.21.15.69:0/294780126 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f3a700cf5b0 con 0x7f3a80074280 2024-03-19T01:10:25.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.884+0000 7f3a7d400640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3a640a2910 0x7f3a640a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:25.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.885+0000 7f3a7d400640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3a640a2910 0x7f3a640a4dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c036040 tx=0x7f3a6c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:25.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:25.888+0000 7f3a76a00640 1 -- 172.21.15.69:0/294780126 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3a7009c0e0 con 0x7f3a80074280 2024-03-19T01:10:26.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:25 smithi069 ceph-mon[31727]: from='client.25532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:10:26.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:25 smithi069 ceph-mon[31727]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:10:26.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:25 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/146294697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:26.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.164+0000 7f3a85000640 1 -- 172.21.15.69:0/294780126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3a80062420 con 0x7f3a80074280 2024-03-19T01:10:26.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.164+0000 7f3a76a00640 1 -- 172.21.15.69:0/294780126 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f3a7009b9e0 con 0x7f3a80074280 2024-03-19T01:10:26.165 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:10:26.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 -- 172.21.15.69:0/294780126 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3a640a2910 msgr2=0x7f3a640a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:26.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3a640a2910 0x7f3a640a4dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c036040 tx=0x7f3a6c000bc0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 -- 172.21.15.69:0/294780126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a80074280 msgr2=0x7f3a8019e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:26.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a80074280 0x7f3a8019e1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000eb90 tx=0x7f3a7000c6a0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 -- 172.21.15.69:0/294780126 shutdown_connections 2024-03-19T01:10:26.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f3a640a2910 0x7f3a640a4dd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a8010de00 0x7f3a801a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a80075860 0x7f3a8019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 --2- 172.21.15.69:0/294780126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a80074280 0x7f3a8019e1c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 -- 172.21.15.69:0/294780126 >> 172.21.15.69:0/294780126 conn(0x7f3a800fcda0 msgr2=0x7f3a800ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:26.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 -- 172.21.15.69:0/294780126 shutdown_connections 2024-03-19T01:10:26.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.168+0000 7f3a85000640 1 -- 172.21.15.69:0/294780126 wait complete. 2024-03-19T01:10:26.190 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:10:26.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:25 smithi086 ceph-mon[66829]: from='client.25532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:10:26.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:25 smithi086 ceph-mon[66829]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:10:26.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:25 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/146294697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:26.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:25 smithi163 ceph-mon[67428]: from='client.25532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:10:26.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:25 smithi163 ceph-mon[67428]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:10:26.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:25 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/146294697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.405+0000 7f2317800640 1 -- 172.21.15.69:0/2429330150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2310102a20 msgr2=0x7f2310102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.405+0000 7f2317800640 1 --2- 172.21.15.69:0/2429330150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2310102a20 0x7f2310102ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009860 tx=0x7f22fc02bb70 comp rx=0 tx=0).stop 2024-03-19T01:10:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.406+0000 7f2317800640 1 -- 172.21.15.69:0/2429330150 shutdown_connections 2024-03-19T01:10:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.406+0000 7f2317800640 1 --2- 172.21.15.69:0/2429330150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23101033e0 0x7f2310109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.406+0000 7f2317800640 1 --2- 172.21.15.69:0/2429330150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2310102a20 0x7f2310102ea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.406+0000 7f2317800640 1 --2- 172.21.15.69:0/2429330150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2310101830 0x7f2310101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.406+0000 7f2317800640 1 -- 172.21.15.69:0/2429330150 >> 172.21.15.69:0/2429330150 conn(0x7f23100fcda0 msgr2=0x7f23100ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.406+0000 7f2317800640 1 -- 172.21.15.69:0/2429330150 shutdown_connections 2024-03-19T01:10:26.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.406+0000 7f2317800640 1 -- 172.21.15.69:0/2429330150 wait complete. 2024-03-19T01:10:26.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.407+0000 7f2317800640 1 Processor -- start 2024-03-19T01:10:26.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.407+0000 7f2317800640 1 -- start start 2024-03-19T01:10:26.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2317800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2310101830 0x7f23100755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:26.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2317800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2310102a20 0x7f2310073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:26.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2317800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23101033e0 0x7f2310074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:26.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2317800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f231010d490 con 0x7f23101033e0 2024-03-19T01:10:26.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2317800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f231010d310 con 0x7f2310101830 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2317800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f231010d610 con 0x7f2310102a20 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2314e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2310101830 0x7f23100755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2314e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2310101830 0x7f23100755d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38142/0 (socket says 172.21.15.69:38142) 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2314e00640 1 -- 172.21.15.69:0/2532503748 learned_addr learned my addr 172.21.15.69:0/2532503748 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f230fe00640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2310102a20 0x7f2310073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2315800640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23101033e0 0x7f2310074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2314e00640 1 -- 172.21.15.69:0/2532503748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2310102a20 msgr2=0x7f2310073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2314e00640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2310102a20 0x7f2310073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2314e00640 1 -- 172.21.15.69:0/2532503748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23101033e0 msgr2=0x7f2310074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2314e00640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23101033e0 0x7f2310074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.408+0000 7f2314e00640 1 -- 172.21.15.69:0/2532503748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23100749e0 con 0x7f2310101830 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.409+0000 7f2314e00640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2310101830 0x7f23100755d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f230400e9c0 tx=0x7f230400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.409+0000 7f230d600640 1 -- 172.21.15.69:0/2532503748 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230400cd80 con 0x7f2310101830 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.409+0000 7f230d600640 1 -- 172.21.15.69:0/2532503748 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23040035f0 con 0x7f2310101830 2024-03-19T01:10:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.409+0000 7f230d600640 1 -- 172.21.15.69:0/2532503748 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304010690 con 0x7f2310101830 2024-03-19T01:10:26.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.409+0000 7f2317800640 1 -- 172.21.15.69:0/2532503748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23101a1c90 con 0x7f2310101830 2024-03-19T01:10:26.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.409+0000 7f2317800640 1 -- 172.21.15.69:0/2532503748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23101a21b0 con 0x7f2310101830 2024-03-19T01:10:26.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.411+0000 7f2317800640 1 -- 172.21.15.69:0/2532503748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2310108bd0 con 0x7f2310101830 2024-03-19T01:10:26.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.412+0000 7f230d600640 1 -- 172.21.15.69:0/2532503748 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f23040037a0 con 0x7f2310101830 2024-03-19T01:10:26.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.414+0000 7f230d600640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f22f80a2910 0x7f22f80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:26.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.414+0000 7f230d600640 1 -- 172.21.15.69:0/2532503748 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f23040cf420 con 0x7f2310101830 2024-03-19T01:10:26.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.414+0000 7f230fe00640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f22f80a2910 0x7f22f80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:26.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.415+0000 7f230fe00640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f22f80a2910 0x7f22f80a4dd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f22fc02c180 tx=0x7f22fc034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:26.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.417+0000 7f230d600640 1 -- 172.21.15.69:0/2532503748 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f230409bf50 con 0x7f2310101830 2024-03-19T01:10:26.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.639+0000 7f2317800640 1 -- 172.21.15.69:0/2532503748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f231002d1f0 con 0x7f2310101830 2024-03-19T01:10:26.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.640+0000 7f230d600640 1 -- 172.21.15.69:0/2532503748 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f231002d1f0 con 0x7f2310101830 2024-03-19T01:10:26.641 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:10:26.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.642+0000 7f2317800640 1 -- 172.21.15.69:0/2532503748 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f22f80a2910 msgr2=0x7f22f80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:26.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.642+0000 7f2317800640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f22f80a2910 0x7f22f80a4dd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f22fc02c180 tx=0x7f22fc034ad0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.642+0000 7f2317800640 1 -- 172.21.15.69:0/2532503748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2310101830 msgr2=0x7f23100755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:26.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.642+0000 7f2317800640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2310101830 0x7f23100755d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f230400e9c0 tx=0x7f230400ee80 comp rx=0 tx=0).stop 2024-03-19T01:10:26.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.642+0000 7f2317800640 1 -- 172.21.15.69:0/2532503748 shutdown_connections 2024-03-19T01:10:26.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.642+0000 7f2317800640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f22f80a2910 0x7f22f80a4dd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.642+0000 7f2317800640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23101033e0 0x7f2310074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.642+0000 7f2317800640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2310102a20 0x7f2310073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.642+0000 7f2317800640 1 --2- 172.21.15.69:0/2532503748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2310101830 0x7f23100755d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.642+0000 7f2317800640 1 -- 172.21.15.69:0/2532503748 >> 172.21.15.69:0/2532503748 conn(0x7f23100fcda0 msgr2=0x7f2310105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:26.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.643+0000 7f2317800640 1 -- 172.21.15.69:0/2532503748 shutdown_connections 2024-03-19T01:10:26.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.643+0000 7f2317800640 1 -- 172.21.15.69:0/2532503748 wait complete. 2024-03-19T01:10:26.664 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:10:26.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.879+0000 7f8564a00640 1 -- 172.21.15.69:0/1522593981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85601073e0 msgr2=0x7f8560109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:26.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.879+0000 7f8564a00640 1 --2- 172.21.15.69:0/1522593981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85601073e0 0x7f8560109830 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8558009ca0 tx=0x7f855802c100 comp rx=0 tx=0).stop 2024-03-19T01:10:26.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.879+0000 7f8564a00640 1 -- 172.21.15.69:0/1522593981 shutdown_connections 2024-03-19T01:10:26.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.879+0000 7f8564a00640 1 --2- 172.21.15.69:0/1522593981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85601073e0 0x7f8560109830 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.879+0000 7f8564a00640 1 --2- 172.21.15.69:0/1522593981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560104ab0 0x7f8560106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.879+0000 7f8564a00640 1 --2- 172.21.15.69:0/1522593981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8560102180 0x7f8560104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.879+0000 7f8564a00640 1 -- 172.21.15.69:0/1522593981 >> 172.21.15.69:0/1522593981 conn(0x7f85600fbb90 msgr2=0x7f85600fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:26.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.879+0000 7f8564a00640 1 -- 172.21.15.69:0/1522593981 shutdown_connections 2024-03-19T01:10:26.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.880+0000 7f8564a00640 1 -- 172.21.15.69:0/1522593981 wait complete. 2024-03-19T01:10:26.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.881+0000 7f8564a00640 1 Processor -- start 2024-03-19T01:10:26.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.881+0000 7f8564a00640 1 -- start start 2024-03-19T01:10:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f8564a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560102180 0x7f8560199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f8564a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8560104ab0 0x7f856019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f8564a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85601073e0 0x7f85601a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f8564a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f856010d380 con 0x7f8560104ab0 2024-03-19T01:10:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f8564a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f856010d200 con 0x7f8560102180 2024-03-19T01:10:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f8564a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f856010d500 con 0x7f85601073e0 2024-03-19T01:10:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f855de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8560104ab0 0x7f856019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f855de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8560104ab0 0x7f856019a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47012/0 (socket says 172.21.15.69:47012) 2024-03-19T01:10:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f855de00640 1 -- 172.21.15.69:0/588549595 learned_addr learned my addr 172.21.15.69:0/588549595 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:26.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f855f200640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85601073e0 0x7f85601a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:26.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f855e800640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560102180 0x7f8560199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:26.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f855f200640 1 -- 172.21.15.69:0/588549595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560102180 msgr2=0x7f8560199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:26.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.882+0000 7f855f200640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560102180 0x7f8560199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.883+0000 7f855f200640 1 -- 172.21.15.69:0/588549595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8560104ab0 msgr2=0x7f856019a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:26.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.883+0000 7f855f200640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8560104ab0 0x7f856019a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:26.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.883+0000 7f855f200640 1 -- 172.21.15.69:0/588549595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85601a17f0 con 0x7f85601073e0 2024-03-19T01:10:26.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.883+0000 7f855de00640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8560104ab0 0x7f856019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:26.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.883+0000 7f855f200640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85601073e0 0x7f85601a1180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8558037040 tx=0x7f8558035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:26.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.883+0000 7f8557400640 1 -- 172.21.15.69:0/588549595 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558039040 con 0x7f85601073e0 2024-03-19T01:10:26.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.883+0000 7f8557400640 1 -- 172.21.15.69:0/588549595 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8558044070 con 0x7f85601073e0 2024-03-19T01:10:26.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.883+0000 7f8557400640 1 -- 172.21.15.69:0/588549595 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558004350 con 0x7f85601073e0 2024-03-19T01:10:26.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.883+0000 7f8564a00640 1 -- 172.21.15.69:0/588549595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85601a1a80 con 0x7f85601073e0 2024-03-19T01:10:26.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.883+0000 7f8564a00640 1 -- 172.21.15.69:0/588549595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85601a1ef0 con 0x7f85601073e0 2024-03-19T01:10:26.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.885+0000 7f8554c00640 1 -- 172.21.15.69:0/588549595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8538005290 con 0x7f85601073e0 2024-03-19T01:10:26.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.888+0000 7f8557400640 1 -- 172.21.15.69:0/588549595 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f85580044f0 con 0x7f85601073e0 2024-03-19T01:10:26.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.889+0000 7f8557400640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85440a2910 0x7f85440a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:26.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.889+0000 7f8557400640 1 -- 172.21.15.69:0/588549595 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f85580f12e0 con 0x7f85601073e0 2024-03-19T01:10:26.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.889+0000 7f855e800640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85440a2910 0x7f85440a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:26.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.890+0000 7f855e800640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85440a2910 0x7f85440a4dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8550009bd0 tx=0x7f8550005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:26.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:26.893+0000 7f8557400640 1 -- 172.21.15.69:0/588549595 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f85580bdf80 con 0x7f85601073e0 2024-03-19T01:10:26.978 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:26 smithi069 ceph-mon[31727]: from='client.25544 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:10:26.978 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:26 smithi069 ceph-mon[31727]: pgmap v414: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-19T01:10:26.978 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:26 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/294780126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:26.978 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:26 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2532503748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:27.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.144+0000 7f8554c00640 1 -- 172.21.15.69:0/588549595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8538005850 con 0x7f85601073e0 2024-03-19T01:10:27.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.144+0000 7f8557400640 1 -- 172.21.15.69:0/588549595 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f85580098b0 con 0x7f85601073e0 2024-03-19T01:10:27.147 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-03-19T01:10:27.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 -- 172.21.15.69:0/588549595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85440a2910 msgr2=0x7f85440a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:27.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85440a2910 0x7f85440a4dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8550009bd0 tx=0x7f8550005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:27.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 -- 172.21.15.69:0/588549595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85601073e0 msgr2=0x7f85601a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:27.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85601073e0 0x7f85601a1180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8558037040 tx=0x7f8558035d10 comp rx=0 tx=0).stop 2024-03-19T01:10:27.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 -- 172.21.15.69:0/588549595 shutdown_connections 2024-03-19T01:10:27.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f85440a2910 0x7f85440a4dd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85601073e0 0x7f85601a1180 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8560104ab0 0x7f856019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 --2- 172.21.15.69:0/588549595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560102180 0x7f8560199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 -- 172.21.15.69:0/588549595 >> 172.21.15.69:0/588549595 conn(0x7f85600fbb90 msgr2=0x7f8560101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:27.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 -- 172.21.15.69:0/588549595 shutdown_connections 2024-03-19T01:10:27.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.147+0000 7f8554c00640 1 -- 172.21.15.69:0/588549595 wait complete. 2024-03-19T01:10:27.166 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T01:10:27.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:26 smithi086 ceph-mon[66829]: from='client.25544 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:10:27.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:26 smithi086 ceph-mon[66829]: pgmap v414: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-19T01:10:27.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:26 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/294780126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:27.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:26 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2532503748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:27.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:26 smithi163 ceph-mon[67428]: from='client.25544 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:10:27.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:26 smithi163 ceph-mon[67428]: pgmap v414: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-19T01:10:27.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:26 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/294780126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:10:27.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:26 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2532503748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:10:27.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.373+0000 7f560aa00640 1 -- 172.21.15.69:0/197672742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5604107520 msgr2=0x7f5604109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.373+0000 7f560aa00640 1 --2- 172.21.15.69:0/197672742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5604107520 0x7f5604109950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009ca0 tx=0x7f55fc02c0f0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.373+0000 7f560aa00640 1 -- 172.21.15.69:0/197672742 shutdown_connections 2024-03-19T01:10:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.373+0000 7f560aa00640 1 --2- 172.21.15.69:0/197672742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5604107520 0x7f5604109950 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.373+0000 7f560aa00640 1 --2- 172.21.15.69:0/197672742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5604104bf0 0x7f5604106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.373+0000 7f560aa00640 1 --2- 172.21.15.69:0/197672742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56041022c0 0x7f56041046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.373+0000 7f560aa00640 1 -- 172.21.15.69:0/197672742 >> 172.21.15.69:0/197672742 conn(0x7f56040fbcb0 msgr2=0x7f56040fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.374+0000 7f560aa00640 1 -- 172.21.15.69:0/197672742 shutdown_connections 2024-03-19T01:10:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.374+0000 7f560aa00640 1 -- 172.21.15.69:0/197672742 wait complete. 2024-03-19T01:10:27.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f560aa00640 1 Processor -- start 2024-03-19T01:10:27.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f560aa00640 1 -- start start 2024-03-19T01:10:27.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f560aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56041022c0 0x7f5604199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:27.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f560aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5604104bf0 0x7f560419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f560aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5604107520 0x7f56041a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f560aa00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f560410d4a0 con 0x7f5604107520 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f560aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f560410d320 con 0x7f5604104bf0 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f560aa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f560410d620 con 0x7f56041022c0 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f5603e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56041022c0 0x7f5604199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f5603400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5604104bf0 0x7f560419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f5603400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5604104bf0 0x7f560419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38176/0 (socket says 172.21.15.69:38176) 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f5603400640 1 -- 172.21.15.69:0/2426783370 learned_addr learned my addr 172.21.15.69:0/2426783370 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.375+0000 7f5603e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56041022c0 0x7f5604199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42650/0 (socket says 172.21.15.69:42650) 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5608a00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5604107520 0x7f56041a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5603e00640 1 -- 172.21.15.69:0/2426783370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5604104bf0 msgr2=0x7f560419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5603e00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5604104bf0 0x7f560419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5603e00640 1 -- 172.21.15.69:0/2426783370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5604107520 msgr2=0x7f56041a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5603e00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5604107520 0x7f56041a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5603e00640 1 -- 172.21.15.69:0/2426783370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56041a1960 con 0x7f56041022c0 2024-03-19T01:10:27.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5603400640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5604104bf0 0x7f560419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:27.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5608a00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5604107520 0x7f56041a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:27.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5603e00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56041022c0 0x7f5604199d50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e9f0 tx=0x7f55f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:27.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5600c00640 1 -- 172.21.15.69:0/2426783370 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400ccc0 con 0x7f56041022c0 2024-03-19T01:10:27.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5600c00640 1 -- 172.21.15.69:0/2426783370 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f4010910 con 0x7f56041022c0 2024-03-19T01:10:27.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.376+0000 7f5600c00640 1 -- 172.21.15.69:0/2426783370 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4018960 con 0x7f56041022c0 2024-03-19T01:10:27.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.377+0000 7f560aa00640 1 -- 172.21.15.69:0/2426783370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56041a1c50 con 0x7f56041022c0 2024-03-19T01:10:27.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.377+0000 7f560aa00640 1 -- 172.21.15.69:0/2426783370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56041a21a0 con 0x7f56041022c0 2024-03-19T01:10:27.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.378+0000 7f560aa00640 1 -- 172.21.15.69:0/2426783370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5604109010 con 0x7f56041022c0 2024-03-19T01:10:27.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.380+0000 7f5600c00640 1 -- 172.21.15.69:0/2426783370 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f55f4010ac0 con 0x7f56041022c0 2024-03-19T01:10:27.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.381+0000 7f5600c00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55e80a2890 0x7f55e80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:27.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.382+0000 7f5600c00640 1 -- 172.21.15.69:0/2426783370 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f55f40cf4f0 con 0x7f56041022c0 2024-03-19T01:10:27.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.382+0000 7f5603400640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55e80a2890 0x7f55e80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:27.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.382+0000 7f5603400640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55e80a2890 0x7f55e80a4d50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f55f00058c0 tx=0x7f55f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:27.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.385+0000 7f5600c00640 1 -- 172.21.15.69:0/2426783370 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f55f409c020 con 0x7f56041022c0 2024-03-19T01:10:27.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.699+0000 7f560aa00640 1 -- 172.21.15.69:0/2426783370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f560402d1f0 con 0x7f56041022c0 2024-03-19T01:10:27.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.700+0000 7f5600c00640 1 -- 172.21.15.69:0/2426783370 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f560402d1f0 con 0x7f56041022c0 2024-03-19T01:10:27.702 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:10:27.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 -- 172.21.15.69:0/2426783370 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55e80a2890 msgr2=0x7f55e80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:27.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55e80a2890 0x7f55e80a4d50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f55f00058c0 tx=0x7f55f0005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:27.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 -- 172.21.15.69:0/2426783370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56041022c0 msgr2=0x7f5604199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:27.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56041022c0 0x7f5604199d50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e9f0 tx=0x7f55f400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 -- 172.21.15.69:0/2426783370 shutdown_connections 2024-03-19T01:10:27.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f55e80a2890 0x7f55e80a4d50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5604107520 0x7f56041a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5604104bf0 0x7f560419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 --2- 172.21.15.69:0/2426783370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56041022c0 0x7f5604199d50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 -- 172.21.15.69:0/2426783370 >> 172.21.15.69:0/2426783370 conn(0x7f56040fbcb0 msgr2=0x7f56041019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:27.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 -- 172.21.15.69:0/2426783370 shutdown_connections 2024-03-19T01:10:27.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.703+0000 7f560aa00640 1 -- 172.21.15.69:0/2426783370 wait complete. 2024-03-19T01:10:27.723 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:10:27.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.914+0000 7ff42b200640 1 -- 172.21.15.69:0/1566921532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff42c101680 msgr2=0x7ff42c101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.914+0000 7ff42b200640 1 --2- 172.21.15.69:0/1566921532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff42c101680 0x7ff42c101aa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009ca0 tx=0x7ff41c02c100 comp rx=0 tx=0).stop 2024-03-19T01:10:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.914+0000 7ff42b200640 1 -- 172.21.15.69:0/1566921532 shutdown_connections 2024-03-19T01:10:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.914+0000 7ff42b200640 1 --2- 172.21.15.69:0/1566921532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c103230 0x7ff42c109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.914+0000 7ff42b200640 1 --2- 172.21.15.69:0/1566921532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42c102870 0x7ff42c102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.914+0000 7ff42b200640 1 --2- 172.21.15.69:0/1566921532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff42c101680 0x7ff42c101aa0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.914+0000 7ff42b200640 1 -- 172.21.15.69:0/1566921532 >> 172.21.15.69:0/1566921532 conn(0x7ff42c0fcbf0 msgr2=0x7ff42c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.914+0000 7ff42b200640 1 -- 172.21.15.69:0/1566921532 shutdown_connections 2024-03-19T01:10:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.914+0000 7ff42b200640 1 -- 172.21.15.69:0/1566921532 wait complete. 2024-03-19T01:10:27.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.915+0000 7ff42b200640 1 Processor -- start 2024-03-19T01:10:27.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.915+0000 7ff42b200640 1 -- start start 2024-03-19T01:10:27.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.916+0000 7ff42b200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff42c101680 0x7ff42c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.916+0000 7ff42b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42c102870 0x7ff42c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.916+0000 7ff42b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c103230 0x7ff42c1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.916+0000 7ff42b200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c10d300 con 0x7ff42c101680 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.916+0000 7ff42b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c10d180 con 0x7ff42c103230 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.916+0000 7ff42b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c10d480 con 0x7ff42c102870 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.916+0000 7ff429e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff42c101680 0x7ff42c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.916+0000 7ff429e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff42c101680 0x7ff42c199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47070/0 (socket says 172.21.15.69:47070) 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.916+0000 7ff429e00640 1 -- 172.21.15.69:0/2616334907 learned_addr learned my addr 172.21.15.69:0/2616334907 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.916+0000 7ff421400640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42c102870 0x7ff42c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.916+0000 7ff42a800640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c103230 0x7ff42c1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.917+0000 7ff421400640 1 -- 172.21.15.69:0/2616334907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c103230 msgr2=0x7ff42c1a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.917+0000 7ff421400640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c103230 0x7ff42c1a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.917+0000 7ff421400640 1 -- 172.21.15.69:0/2616334907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff42c101680 msgr2=0x7ff42c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.917+0000 7ff421400640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff42c101680 0x7ff42c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.917+0000 7ff421400640 1 -- 172.21.15.69:0/2616334907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c1a17e0 con 0x7ff42c102870 2024-03-19T01:10:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.917+0000 7ff429e00640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff42c101680 0x7ff42c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.917+0000 7ff421400640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42c102870 0x7ff42c19a100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff41400e9f0 tx=0x7ff41400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.917+0000 7ff423400640 1 -- 172.21.15.69:0/2616334907 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41400ccc0 con 0x7ff42c102870 2024-03-19T01:10:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.917+0000 7ff423400640 1 -- 172.21.15.69:0/2616334907 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff414005020 con 0x7ff42c102870 2024-03-19T01:10:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.917+0000 7ff423400640 1 -- 172.21.15.69:0/2616334907 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414004a00 con 0x7ff42c102870 2024-03-19T01:10:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.917+0000 7ff42b200640 1 -- 172.21.15.69:0/2616334907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c1a1ad0 con 0x7ff42c102870 2024-03-19T01:10:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.918+0000 7ff42b200640 1 -- 172.21.15.69:0/2616334907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c1a1fd0 con 0x7ff42c102870 2024-03-19T01:10:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.919+0000 7ff42b200640 1 -- 172.21.15.69:0/2616334907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42c1a2260 con 0x7ff42c102870 2024-03-19T01:10:27.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.922+0000 7ff423400640 1 -- 172.21.15.69:0/2616334907 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7ff4140051d0 con 0x7ff42c102870 2024-03-19T01:10:27.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.923+0000 7ff423400640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff4100a2890 0x7ff4100a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:27.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.923+0000 7ff423400640 1 -- 172.21.15.69:0/2616334907 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7ff4140ceeb0 con 0x7ff42c102870 2024-03-19T01:10:27.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.923+0000 7ff429e00640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff4100a2890 0x7ff4100a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:27.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.924+0000 7ff429e00640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff4100a2890 0x7ff4100a4d50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff41c005e30 tx=0x7ff41c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:27.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:27.931+0000 7ff423400640 1 -- 172.21.15.69:0/2616334907 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff41409ba40 con 0x7ff42c102870 2024-03-19T01:10:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:27 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/588549595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:27 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2426783370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.210+0000 7ff42b200640 1 -- 172.21.15.69:0/2616334907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff42c1a2430 con 0x7ff42c102870 2024-03-19T01:10:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.211+0000 7ff423400640 1 -- 172.21.15.69:0/2616334907 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11930 (secure 0 0 0) 0x7ff41409b340 con 0x7ff42c102870 2024-03-19T01:10:28.212 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:28.212 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:10:17.373602+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24550,24494,24580,24581,24476]},"id":1}]} 2024-03-19T01:10:28.213 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 29 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.214+0000 7ff42b200640 1 -- 172.21.15.69:0/2616334907 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff4100a2890 msgr2=0x7ff4100a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.214+0000 7ff42b200640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff4100a2890 0x7ff4100a4d50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff41c005e30 tx=0x7ff41c004a40 comp rx=0 tx=0).stop 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.214+0000 7ff42b200640 1 -- 172.21.15.69:0/2616334907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42c102870 msgr2=0x7ff42c19a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.214+0000 7ff42b200640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42c102870 0x7ff42c19a100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff41400e9f0 tx=0x7ff41400edf0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.215+0000 7ff42b200640 1 -- 172.21.15.69:0/2616334907 shutdown_connections 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.215+0000 7ff42b200640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7ff4100a2890 0x7ff4100a4d50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.215+0000 7ff42b200640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c103230 0x7ff42c1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.215+0000 7ff42b200640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42c102870 0x7ff42c19a100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.215+0000 7ff42b200640 1 --2- 172.21.15.69:0/2616334907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff42c101680 0x7ff42c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.215+0000 7ff42b200640 1 -- 172.21.15.69:0/2616334907 >> 172.21.15.69:0/2616334907 conn(0x7ff42c0fcbf0 msgr2=0x7ff42c073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.215+0000 7ff42b200640 1 -- 172.21.15.69:0/2616334907 shutdown_connections 2024-03-19T01:10:28.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.215+0000 7ff42b200640 1 -- 172.21.15.69:0/2616334907 wait complete. 2024-03-19T01:10:28.233 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:10:28.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:27 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/588549595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:28.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:27 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2426783370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:28.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:27 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/588549595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:10:28.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:27 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2426783370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:10:28.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.435+0000 7f5dcb200640 1 -- 172.21.15.69:0/1065555172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dcc103870 msgr2=0x7f5dcc103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.435+0000 7f5dcb200640 1 --2- 172.21.15.69:0/1065555172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dcc103870 0x7f5dcc103c90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009900 tx=0x7f5dbc02bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.435+0000 7f5dcb200640 1 -- 172.21.15.69:0/1065555172 shutdown_connections 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.435+0000 7f5dcb200640 1 --2- 172.21.15.69:0/1065555172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc105420 0x7f5dcc10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.435+0000 7f5dcb200640 1 --2- 172.21.15.69:0/1065555172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dcc104a60 0x7f5dcc104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.435+0000 7f5dcb200640 1 --2- 172.21.15.69:0/1065555172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dcc103870 0x7f5dcc103c90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.435+0000 7f5dcb200640 1 -- 172.21.15.69:0/1065555172 >> 172.21.15.69:0/1065555172 conn(0x7f5dcc0fee00 msgr2=0x7f5dcc101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.435+0000 7f5dcb200640 1 -- 172.21.15.69:0/1065555172 shutdown_connections 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.435+0000 7f5dcb200640 1 -- 172.21.15.69:0/1065555172 wait complete. 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.436+0000 7f5dcb200640 1 Processor -- start 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.436+0000 7f5dcb200640 1 -- start start 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.436+0000 7f5dcb200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc103870 0x7f5dcc199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.436+0000 7f5dcb200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dcc104a60 0x7f5dcc19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.436+0000 7f5dcb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dcc105420 0x7f5dcc1a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.436+0000 7f5dcb200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc10f540 con 0x7f5dcc104a60 2024-03-19T01:10:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.436+0000 7f5dcb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc10f3c0 con 0x7f5dcc105420 2024-03-19T01:10:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.436+0000 7f5dcb200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc10f6c0 con 0x7f5dcc103870 2024-03-19T01:10:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dc9400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dcc104a60 0x7f5dcc19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dcc105420 0x7f5dcc1a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dc9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc103870 0x7f5dcc199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dc9400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dcc104a60 0x7f5dcc19a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47088/0 (socket says 172.21.15.69:47088) 2024-03-19T01:10:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dc9400640 1 -- 172.21.15.69:0/3697031015 learned_addr learned my addr 172.21.15.69:0/3697031015 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dcc105420 0x7f5dcc1a1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38198/0 (socket says 172.21.15.69:38198) 2024-03-19T01:10:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dc9e00640 1 -- 172.21.15.69:0/3697031015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dcc105420 msgr2=0x7f5dcc1a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dc9e00640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dcc105420 0x7f5dcc1a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dc9e00640 1 -- 172.21.15.69:0/3697031015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dcc104a60 msgr2=0x7f5dcc19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dc9e00640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dcc104a60 0x7f5dcc19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dc9e00640 1 -- 172.21.15.69:0/3697031015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc1a1780 con 0x7f5dcc103870 2024-03-19T01:10:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dca800640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dcc105420 0x7f5dcc1a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.437+0000 7f5dc9e00640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc103870 0x7f5dcc199b60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009230 tx=0x7f5dbc02c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.438+0000 7f5dc2a00640 1 -- 172.21.15.69:0/3697031015 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc043020 con 0x7f5dcc103870 2024-03-19T01:10:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.438+0000 7f5dc2a00640 1 -- 172.21.15.69:0/3697031015 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dbc0054e0 con 0x7f5dcc103870 2024-03-19T01:10:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.438+0000 7f5dcb200640 1 -- 172.21.15.69:0/3697031015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dcc1a1a10 con 0x7f5dcc103870 2024-03-19T01:10:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.438+0000 7f5dc2a00640 1 -- 172.21.15.69:0/3697031015 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc003b60 con 0x7f5dcc103870 2024-03-19T01:10:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.438+0000 7f5dcb200640 1 -- 172.21.15.69:0/3697031015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dcc1a1f80 con 0x7f5dcc103870 2024-03-19T01:10:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.439+0000 7f5dcb200640 1 -- 172.21.15.69:0/3697031015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dcc10ab30 con 0x7f5dcc103870 2024-03-19T01:10:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.440+0000 7f5dc2a00640 1 -- 172.21.15.69:0/3697031015 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f5dbc046020 con 0x7f5dcc103870 2024-03-19T01:10:28.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.442+0000 7f5dc2a00640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5db00a2910 0x7f5db00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.442+0000 7f5dc2a00640 1 -- 172.21.15.69:0/3697031015 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f5dbc0f0e40 con 0x7f5dcc103870 2024-03-19T01:10:28.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.442+0000 7f5dc9400640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5db00a2910 0x7f5db00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.443+0000 7f5dc9400640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5db00a2910 0x7f5db00a4dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc19b070 tx=0x7f5db8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:28.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.445+0000 7f5dc2a00640 1 -- 172.21.15.69:0/3697031015 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5dbc04d020 con 0x7f5dcc103870 2024-03-19T01:10:28.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.658+0000 7f5dcb200640 1 -- 172.21.15.69:0/3697031015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5dcc02d440 con 0x7f5dcc103870 2024-03-19T01:10:28.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.659+0000 7f5dc2a00640 1 -- 172.21.15.69:0/3697031015 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24736 (secure 0 0 0) 0x7f5dbc0bda80 con 0x7f5dcc103870 2024-03-19T01:10:28.661 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:10:28.661 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:09:25.783660+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:10:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 -- 172.21.15.69:0/3697031015 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5db00a2910 msgr2=0x7f5db00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5db00a2910 0x7f5db00a4dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc19b070 tx=0x7f5db8005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 -- 172.21.15.69:0/3697031015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc103870 msgr2=0x7f5dcc199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc103870 0x7f5dcc199b60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009230 tx=0x7f5dbc02c5d0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 -- 172.21.15.69:0/3697031015 shutdown_connections 2024-03-19T01:10:28.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f5db00a2910 0x7f5db00a4dd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dcc105420 0x7f5dcc1a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dcc104a60 0x7f5dcc19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 --2- 172.21.15.69:0/3697031015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc103870 0x7f5dcc199b60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 -- 172.21.15.69:0/3697031015 >> 172.21.15.69:0/3697031015 conn(0x7f5dcc0fee00 msgr2=0x7f5dcc107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:28.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 -- 172.21.15.69:0/3697031015 shutdown_connections 2024-03-19T01:10:28.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.662+0000 7f5dcb200640 1 -- 172.21.15.69:0/3697031015 wait complete. 2024-03-19T01:10:28.685 DEBUG:tasks.fwd_scrub:Yielding 2024-03-19T01:10:28.685 INFO:teuthology.run_tasks:Running task exec... 2024-03-19T01:10:28.694 INFO:teuthology.task.exec:Executing custom commands... 2024-03-19T01:10:28.695 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-03-19T01:10:28.695 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-19T01:10:28.695 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-19T01:10:28.695 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:10:28.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:10:28.696 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:10:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.903+0000 7fc19e000640 1 -- 172.21.15.69:0/3317631105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc198104c70 msgr2=0x7fc1981050f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.903+0000 7fc19e000640 1 --2- 172.21.15.69:0/3317631105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc198104c70 0x7fc1981050f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc184009900 tx=0x7fc18402bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.904+0000 7fc19e000640 1 -- 172.21.15.69:0/3317631105 shutdown_connections 2024-03-19T01:10:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.904+0000 7fc19e000640 1 --2- 172.21.15.69:0/3317631105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc198105630 0x7fc19810bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.904+0000 7fc19e000640 1 --2- 172.21.15.69:0/3317631105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc198104c70 0x7fc1981050f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.904+0000 7fc19e000640 1 --2- 172.21.15.69:0/3317631105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc198103a80 0x7fc198103ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.904+0000 7fc19e000640 1 -- 172.21.15.69:0/3317631105 >> 172.21.15.69:0/3317631105 conn(0x7fc1980ff010 msgr2=0x7fc198101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:28.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.904+0000 7fc19e000640 1 -- 172.21.15.69:0/3317631105 shutdown_connections 2024-03-19T01:10:28.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.904+0000 7fc19e000640 1 -- 172.21.15.69:0/3317631105 wait complete. 2024-03-19T01:10:28.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.905+0000 7fc19e000640 1 Processor -- start 2024-03-19T01:10:28.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.905+0000 7fc19e000640 1 -- start start 2024-03-19T01:10:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.905+0000 7fa5b2400640 1 -- 172.21.15.69:0/964126524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5ac1095b0 msgr2=0x7fa5ac10ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.905+0000 7fa5b2400640 1 --2- 172.21.15.69:0/964126524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5ac1095b0 0x7fa5ac10ba50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009be0 tx=0x7fa5a402bf80 comp rx=0 tx=0).stop 2024-03-19T01:10:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fa5b2400640 1 -- 172.21.15.69:0/964126524 shutdown_connections 2024-03-19T01:10:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fa5b2400640 1 --2- 172.21.15.69:0/964126524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5ac1095b0 0x7fa5ac10ba50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fa5b2400640 1 --2- 172.21.15.69:0/964126524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5ac074220 0x7fa5ac074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fa5b2400640 1 --2- 172.21.15.69:0/964126524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5ac075800 0x7fa5ac073c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fa5b2400640 1 -- 172.21.15.69:0/964126524 >> 172.21.15.69:0/964126524 conn(0x7fa5ac0fcc90 msgr2=0x7fa5ac0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc19e000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc198103a80 0x7fc198110bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc19e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc198104c70 0x7fc198111100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc19e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc198105630 0x7fc198116110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc19e000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc198076fc0 con 0x7fc198105630 2024-03-19T01:10:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc19e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc198076e40 con 0x7fc198104c70 2024-03-19T01:10:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc19e000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc198077140 con 0x7fc198103a80 2024-03-19T01:10:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc197400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc198103a80 0x7fc198110bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc197400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc198103a80 0x7fc198110bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42700/0 (socket says 172.21.15.69:42700) 2024-03-19T01:10:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc197400640 1 -- 172.21.15.69:0/2089386773 learned_addr learned my addr 172.21.15.69:0/2089386773 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc196a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc198104c70 0x7fc198111100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc197e00640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc198105630 0x7fc198116110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc197400640 1 -- 172.21.15.69:0/2089386773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc198104c70 msgr2=0x7fc198111100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc197400640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc198104c70 0x7fc198111100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fa5b2400640 1 -- 172.21.15.69:0/964126524 shutdown_connections 2024-03-19T01:10:28.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fa5b2400640 1 -- 172.21.15.69:0/964126524 wait complete. 2024-03-19T01:10:28.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc197400640 1 -- 172.21.15.69:0/2089386773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc198105630 msgr2=0x7fc198116110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc197400640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc198105630 0x7fc198116110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.906+0000 7fc197400640 1 -- 172.21.15.69:0/2089386773 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc198116780 con 0x7fc198103a80 2024-03-19T01:10:28.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.907+0000 7fc197400640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc198103a80 0x7fc198110bc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc18800d950 tx=0x7fc18800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:28.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.907+0000 7fc18fe00640 1 -- 172.21.15.69:0/2089386773 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188017020 con 0x7fc198103a80 2024-03-19T01:10:28.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.907+0000 7fa5b2400640 1 Processor -- start 2024-03-19T01:10:28.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.907+0000 7fc18fe00640 1 -- 172.21.15.69:0/2089386773 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc188009c30 con 0x7fc198103a80 2024-03-19T01:10:28.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.907+0000 7fc19e000640 1 -- 172.21.15.69:0/2089386773 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1981ae930 con 0x7fc198103a80 2024-03-19T01:10:28.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.907+0000 7fc18fe00640 1 -- 172.21.15.69:0/2089386773 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18800f720 con 0x7fc198103a80 2024-03-19T01:10:28.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.907+0000 7fc19e000640 1 -- 172.21.15.69:0/2089386773 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1981aed30 con 0x7fc198103a80 2024-03-19T01:10:28.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.907+0000 7fa5b2400640 1 -- start start 2024-03-19T01:10:28.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.907+0000 7fa5b2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5ac074220 0x7fa5ac104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5b2400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5ac075800 0x7fa5ac104fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5b2400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5ac1095b0 0x7fa5ac101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5b2400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac10f5d0 con 0x7fa5ac1095b0 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5b2400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac10f450 con 0x7fa5ac074220 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5ac075800 0x7fa5ac104fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5b1a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5ac1095b0 0x7fa5ac101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5b2400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac10f750 con 0x7fa5ac075800 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5b1a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5ac1095b0 0x7fa5ac101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47122/0 (socket says 172.21.15.69:47122) 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5b1a00640 1 -- 172.21.15.69:0/532026648 learned_addr learned my addr 172.21.15.69:0/532026648 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5b1000640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5ac074220 0x7fa5ac104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5abe00640 1 -- 172.21.15.69:0/532026648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5ac074220 msgr2=0x7fa5ac104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5abe00640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5ac074220 0x7fa5ac104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5abe00640 1 -- 172.21.15.69:0/532026648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5ac1095b0 msgr2=0x7fa5ac101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5abe00640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5ac1095b0 0x7fa5ac101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5abe00640 1 -- 172.21.15.69:0/532026648 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac101ed0 con 0x7fa5ac075800 2024-03-19T01:10:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.908+0000 7fa5b1000640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5ac074220 0x7fa5ac104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:28.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.909+0000 7fc19e000640 1 -- 172.21.15.69:0/2089386773 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19804f580 con 0x7fc198103a80 2024-03-19T01:10:28.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.909+0000 7fa5abe00640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5ac075800 0x7fa5ac104fe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa59800e9f0 tx=0x7fa59800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:28.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.909+0000 7fa5a9600640 1 -- 172.21.15.69:0/532026648 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59800cd80 con 0x7fa5ac075800 2024-03-19T01:10:28.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.909+0000 7fa5a9600640 1 -- 172.21.15.69:0/532026648 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa598010910 con 0x7fa5ac075800 2024-03-19T01:10:28.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.909+0000 7fa5b2400640 1 -- 172.21.15.69:0/532026648 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac1997d0 con 0x7fa5ac075800 2024-03-19T01:10:28.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.909+0000 7fa5a9600640 1 -- 172.21.15.69:0/532026648 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598018970 con 0x7fa5ac075800 2024-03-19T01:10:28.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.909+0000 7fa5b2400640 1 -- 172.21.15.69:0/532026648 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac199d70 con 0x7fa5ac075800 2024-03-19T01:10:28.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.910+0000 7fc18fe00640 1 -- 172.21.15.69:0/2089386773 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fc18801a020 con 0x7fc198103a80 2024-03-19T01:10:28.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.913+0000 7fc18fe00640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc17c0a29c0 0x7fc17c0a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.913+0000 7fc18fe00640 1 -- 172.21.15.69:0/2089386773 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fc1880cef10 con 0x7fc198103a80 2024-03-19T01:10:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.913+0000 7fc196a00640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc17c0a29c0 0x7fc17c0a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.912+0000 7fa5b660e740 1 Processor -- start 2024-03-19T01:10:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.913+0000 7fa5b660e740 1 -- start start 2024-03-19T01:10:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.913+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 0x5567c596c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.913+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 0x5567c59711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.913+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 0x5567c5973cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.913+0000 7fa5b660e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5567c58cffb0 con 0x5567c596c6f0 2024-03-19T01:10:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.913+0000 7fa5b660e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5567c5878ab0 con 0x5567c5971860 2024-03-19T01:10:28.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.913+0000 7fa5b660e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5567c58cc050 con 0x5567c596bd00 2024-03-19T01:10:28.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.913+0000 7fa5b1a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 0x5567c596c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5b1a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 0x5567c596c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42714/0 (socket says 172.21.15.69:42714) 2024-03-19T01:10:28.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5b1000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 0x5567c59711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5b1a00640 1 -- 172.21.15.69:0/3848514746 learned_addr learned my addr 172.21.15.69:0/3848514746 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:28.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5a9600640 1 -- 172.21.15.69:0/532026648 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fa59801f020 con 0x7fa5ac075800 2024-03-19T01:10:28.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fc196a00640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc17c0a29c0 0x7fc17c0a4e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc18402c240 tx=0x7fc18402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:28.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5abe00640 1 --2- 172.21.15.69:0/3848514746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 0x5567c5973cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5b1000640 1 -- 172.21.15.69:0/3848514746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 msgr2=0x5567c596c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:28.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5b1000640 1 --2- 172.21.15.69:0/3848514746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 0x5567c596c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5b1000640 1 -- 172.21.15.69:0/3848514746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 msgr2=0x5567c5973cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5b1000640 1 --2- 172.21.15.69:0/3848514746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 0x5567c5973cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5b1000640 1 -- 172.21.15.69:0/3848514746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c5896e00 con 0x5567c596c6f0 2024-03-19T01:10:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5abe00640 1 --2- 172.21.15.69:0/3848514746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 0x5567c5973cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:10:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.914+0000 7fa5b1000640 1 --2- 172.21.15.69:0/3848514746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 0x5567c59711f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0005fd0 tx=0x7fa5a00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db649697e43edc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.915+0000 7fa59e000640 1 -- 172.21.15.69:0/3848514746 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0004020 con 0x5567c596c6f0 2024-03-19T01:10:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.915+0000 7fa59e000640 1 -- 172.21.15.69:0/3848514746 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a00041c0 con 0x5567c596c6f0 2024-03-19T01:10:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.915+0000 7fa59e000640 1 -- 172.21.15.69:0/3848514746 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a000f900 con 0x5567c596c6f0 2024-03-19T01:10:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.915+0000 7fa5b660e740 1 -- 172.21.15.69:0/3848514746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 msgr2=0x5567c59711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.915+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3848514746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 0x5567c59711f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0005fd0 tx=0x7fa5a00118d0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.916+0000 7fc18fe00640 1 -- 172.21.15.69:0/2089386773 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc188021020 con 0x7fc198103a80 2024-03-19T01:10:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.916+0000 7fa5b660e740 1 -- 172.21.15.69:0/3848514746 shutdown_connections 2024-03-19T01:10:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.916+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3848514746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 0x5567c5973cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.916+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3848514746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 0x5567c59711f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.916+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3848514746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 0x5567c596c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.916+0000 7fa5b660e740 1 -- 172.21.15.69:0/3848514746 >> 172.21.15.69:0/3848514746 conn(0x5567c5967270 msgr2=0x5567c5967670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.916+0000 7fa5a9600640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5940a2910 0x7fa5940a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.916+0000 7fa5b660e740 1 -- 172.21.15.69:0/3848514746 shutdown_connections 2024-03-19T01:10:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.916+0000 7fa5b660e740 1 -- 172.21.15.69:0/3848514746 wait complete. 2024-03-19T01:10:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.916+0000 7fa5a9600640 1 -- 172.21.15.69:0/532026648 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fa5980cef40 con 0x7fa5ac075800 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.916+0000 7fa5b1000640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5940a2910 0x7fa5940a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.917+0000 7fa5b660e740 1 -- start start 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.917+0000 7fa5b660e740 1 Processor -- start 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.917+0000 7fa5b1000640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5940a2910 0x7fa5940a4dd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa5a001b040 tx=0x7fa5a0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 0x5567c59761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c5976700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c5971860 0x5567c597d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5b660e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5567c5974810 con 0x5567c5971860 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5b660e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5567c588bf00 con 0x5567c596bd00 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5b660e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5567c5974990 con 0x5567c596c6f0 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5b660e740 10 client.0 _ll_register_callbacks cb 0x5567c588b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c5976700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c5976700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42722/0 (socket says 172.21.15.69:42722) 2024-03-19T01:10:28.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5abe00640 1 -- 172.21.15.69:0/2865664907 learned_addr learned my addr 172.21.15.69:0/2865664907 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5b1000640 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 0x5567c59761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5abe00640 1 -- 172.21.15.69:0/2865664907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 msgr2=0x5567c59761c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5abe00640 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 0x5567c59761c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5abe00640 1 -- 172.21.15.69:0/2865664907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c5971860 msgr2=0x5567c597d740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.918+0000 7fa5b1a00640 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c5971860 0x5567c597d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa5abe00640 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c5971860 0x5567c597d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa5abe00640 1 -- 172.21.15.69:0/2865664907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c56ac7e0 con 0x5567c596c6f0 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa5b1000640 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 0x5567c59761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa5abe00640 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c5976700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa59809b300 tx=0x7fa59809ad90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aa6591253ab9a6e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa592a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa592a00640 1 -- 172.21.15.69:0/2865664907 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59809f4c0 con 0x5567c596c6f0 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa592a00640 1 -- 172.21.15.69:0/2865664907 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fa598010ac0 con 0x5567c596c6f0 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa5b660e740 1 -- 172.21.15.69:0/2865664907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a00041c0 con 0x5567c596c6f0 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa592a00640 1 -- 172.21.15.69:0/2865664907 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5980a3b10 con 0x5567c596c6f0 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa5b660e740 10 client.25606 fetch_fsmap learned FSMap version 29 2024-03-19T01:10:28.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa5b660e740 1 -- 172.21.15.69:0/2865664907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5567c570cf70 con 0x5567c596c6f0 2024-03-19T01:10:28.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.919+0000 7fa592a00640 1 -- 172.21.15.69:0/2865664907 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a00041c0 con 0x5567c596c6f0 2024-03-19T01:10:28.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.920+0000 7fa592a00640 1 -- 172.21.15.69:0/2865664907 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa5980a3680 con 0x5567c596c6f0 2024-03-19T01:10:28.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.920+0000 7fa5b660e740 10 client.25606 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:10:28.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.920+0000 7fa5b660e740 10 client.25606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:10:28.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.920+0000 7fa5b660e740 20 client.25606 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:28.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.920+0000 7fa5b660e740 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5567c5981fa0 0x5567c5984390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.920+0000 7fa5b660e740 4 client.25606 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:10:28.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.920+0000 7fa5b660e740 1 -- 172.21.15.69:0/2865664907 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567c58dd4f0 con 0x5567c5981fa0 2024-03-19T01:10:28.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.920+0000 7fa5b1000640 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5567c5981fa0 0x5567c5984390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.921+0000 7fa592a00640 10 client.25606 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:28.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.921+0000 7fa5b1000640 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5567c5981fa0 0x5567c5984390 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f2f2611e41fcd8 server_cookie=4e7e1437b4cb95a7 in_seq=0 out_seq=0 2024-03-19T01:10:28.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.921+0000 7fa592a00640 1 -- 172.21.15.69:0/2865664907 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5567c58dd4f0 con 0x5567c5981fa0 2024-03-19T01:10:28.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.921+0000 7fa592a00640 10 client.25606 handle_command_reply: tid=0 2024-03-19T01:10:28.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 1 client.25606 shutdown 2024-03-19T01:10:28.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 1 -- 172.21.15.69:0/2865664907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 msgr2=0x5567c5976700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c5976700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa59809b300 tx=0x7fa59809ad90 comp rx=0 tx=0).stop 2024-03-19T01:10:28.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 1 -- 172.21.15.69:0/2865664907 shutdown_connections 2024-03-19T01:10:28.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5567c5981fa0 0x5567c5984390 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c5971860 0x5567c597d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c5976700 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 1 --2- 172.21.15.69:0/2865664907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 0x5567c59761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 1 -- 172.21.15.69:0/2865664907 >> 172.21.15.69:0/2865664907 conn(0x5567c5967270 msgr2=0x5567c5966470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:28.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa592a00640 0 client.25606 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:28.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 1 -- 172.21.15.69:0/2865664907 shutdown_connections 2024-03-19T01:10:28.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 1 -- 172.21.15.69:0/2865664907 wait complete. 2024-03-19T01:10:28.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.922+0000 7fa5b660e740 20 client.25606 trim_cache size 0 max 16384 2024-03-19T01:10:28.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.940+0000 7fa5b660e740 1 Processor -- start 2024-03-19T01:10:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5b660e740 1 -- start start 2024-03-19T01:10:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 0x5567c596abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c597d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c5971860 0x5567c5988c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5b660e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1b9180 con 0x5567c5971860 2024-03-19T01:10:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5b660e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1bacc0 con 0x5567c596bd00 2024-03-19T01:10:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5b660e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1b9850 con 0x5567c596c6f0 2024-03-19T01:10:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5b1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 0x5567c596abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c597d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c597d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42726/0 (socket says 172.21.15.69:42726) 2024-03-19T01:10:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5b1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 0x5567c596abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38244/0 (socket says 172.21.15.69:38244) 2024-03-19T01:10:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5abe00640 1 -- 172.21.15.69:0/3553628907 learned_addr learned my addr 172.21.15.69:0/3553628907 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.941+0000 7fa5b1a00640 1 --2- 172.21.15.69:0/3553628907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c5971860 0x5567c5988c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.942+0000 7fa5abe00640 1 -- 172.21.15.69:0/3553628907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 msgr2=0x5567c596abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.942+0000 7fa5abe00640 1 --2- 172.21.15.69:0/3553628907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 0x5567c596abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.942+0000 7fa5abe00640 1 -- 172.21.15.69:0/3553628907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c5971860 msgr2=0x5567c5988c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.942+0000 7fa5abe00640 1 --2- 172.21.15.69:0/3553628907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c5971860 0x5567c5988c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.942+0000 7fa5abe00640 1 -- 172.21.15.69:0/3553628907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c5984700 con 0x5567c596c6f0 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.942+0000 7fa5b1000640 1 --2- 172.21.15.69:0/3553628907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 0x5567c596abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.942+0000 7fa5b1a00640 1 --2- 172.21.15.69:0/3553628907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c5971860 0x5567c5988c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.942+0000 7fa5abe00640 1 --2- 172.21.15.69:0/3553628907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c597d700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa598097d40 tx=0x7fa59809b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2254b38f4485b57 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.942+0000 7fa59d600640 1 -- 172.21.15.69:0/3553628907 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59809f4c0 con 0x5567c596c6f0 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.942+0000 7fa59d600640 1 -- 172.21.15.69:0/3553628907 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa598010ac0 con 0x5567c596c6f0 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.942+0000 7fa59d600640 1 -- 172.21.15.69:0/3553628907 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59809a7c0 con 0x5567c596c6f0 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.943+0000 7fa5b660e740 1 -- 172.21.15.69:0/3553628907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 msgr2=0x5567c597d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.943+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3553628907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c597d700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa598097d40 tx=0x7fa59809b4f0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.943+0000 7fa5b660e740 1 -- 172.21.15.69:0/3553628907 shutdown_connections 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.943+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3553628907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c5971860 0x5567c5988c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.943+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3553628907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596c6f0 0x5567c597d700 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.943+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3553628907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c596bd00 0x5567c596abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.943+0000 7fa5b660e740 1 -- 172.21.15.69:0/3553628907 >> 172.21.15.69:0/3553628907 conn(0x5567c5967270 msgr2=0x5567c596a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.943+0000 7fa5b660e740 1 -- 172.21.15.69:0/3553628907 shutdown_connections 2024-03-19T01:10:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.943+0000 7fa5b660e740 1 -- 172.21.15.69:0/3553628907 wait complete. 2024-03-19T01:10:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.943+0000 7fa5b660e740 1 -- start start 2024-03-19T01:10:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5b660e740 1 Processor -- start 2024-03-19T01:10:28.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 0x5567c597b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 0x5567c597baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5b660e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 0x5567c597ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5b660e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1ba5f0 con 0x5567c596c6f0 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5b660e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1b9f20 con 0x5567c5971860 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5b660e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac1b7160 con 0x5567c596bd00 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5b660e740 10 client.0 _ll_register_callbacks cb 0x7fa59809b1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 0x5567c597baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 0x5567c597baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47146/0 (socket says 172.21.15.69:47146) 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5abe00640 1 -- 172.21.15.69:0/3558045964 learned_addr learned my addr 172.21.15.69:0/3558045964 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.944+0000 7fa5b1000640 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 0x5567c597b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.945+0000 7fa5b1a00640 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 0x5567c597ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.945+0000 7fa5b1a00640 1 -- 172.21.15.69:0/3558045964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 msgr2=0x5567c597b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.945+0000 7fa5b1a00640 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 0x5567c597b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.945+0000 7fa5b1a00640 1 -- 172.21.15.69:0/3558045964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 msgr2=0x5567c597baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:28.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.945+0000 7fa5b1a00640 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 0x5567c597baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.945+0000 7fa5b1a00640 1 -- 172.21.15.69:0/3558045964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c55a4170 con 0x5567c5971860 2024-03-19T01:10:28.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.945+0000 7fa5abe00640 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 0x5567c597baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:28.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.945+0000 7fa5b1000640 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 0x5567c597b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:28.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.945+0000 7fa5b1a00640 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 0x5567c597ca40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4036040 tx=0x7fa5a402cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e33bda79502dfbb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:28.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.945+0000 7fa592a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:10:28.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.945+0000 7fa592a00640 1 -- 172.21.15.69:0/3558045964 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4004040 con 0x5567c5971860 2024-03-19T01:10:28.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.946+0000 7fa5b660e740 1 -- 172.21.15.69:0/3558045964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa598010ac0 con 0x5567c5971860 2024-03-19T01:10:28.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa592a00640 1 -- 172.21.15.69:0/3558045964 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fa5a40041e0 con 0x5567c5971860 2024-03-19T01:10:28.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa592a00640 1 -- 172.21.15.69:0/3558045964 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5a40043e0 con 0x5567c5971860 2024-03-19T01:10:28.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa5b660e740 10 client.25604 fetch_fsmap learned FSMap version 29 2024-03-19T01:10:28.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa5b660e740 1 -- 172.21.15.69:0/3558045964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5567c59f9840 con 0x5567c5971860 2024-03-19T01:10:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa592a00640 1 -- 172.21.15.69:0/3558045964 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa598010ac0 con 0x5567c5971860 2024-03-19T01:10:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa592a00640 1 -- 172.21.15.69:0/3558045964 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa5a4041430 con 0x5567c5971860 2024-03-19T01:10:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa5b660e740 10 client.25604 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:10:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa5b660e740 10 client.25604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:10:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa5b660e740 20 client.25604 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5567c59fa190 0x5567c59fc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa5b660e740 4 client.25604 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:10:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.948+0000 7fa5b660e740 1 -- 172.21.15.69:0/3558045964 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5567c58d7840 con 0x5567c59fa190 2024-03-19T01:10:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.949+0000 7fa5b1000640 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5567c59fa190 0x5567c59fc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.949+0000 7fa592a00640 10 client.25604 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.949+0000 7fa5b1000640 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5567c59fa190 0x5567c59fc5b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ede0b0bebff6c8 server_cookie=b4e573cdb49bbe2 in_seq=0 out_seq=0 2024-03-19T01:10:28.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.950+0000 7fa592a00640 1 -- 172.21.15.69:0/3558045964 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5567c58d7840 con 0x5567c59fa190 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.950+0000 7fa592a00640 10 client.25604 handle_command_reply: tid=0 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.950+0000 7fa5b660e740 1 client.25604 shutdown 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.950+0000 7fa5b660e740 1 -- 172.21.15.69:0/3558045964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 msgr2=0x5567c597ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.950+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 0x5567c597ca40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4036040 tx=0x7fa5a402cb50 comp rx=0 tx=0).stop 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.951+0000 7fa5b660e740 1 -- 172.21.15.69:0/3558045964 shutdown_connections 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.951+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5567c59fa190 0x5567c59fc5b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.951+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567c5971860 0x5567c597ca40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.951+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567c596c6f0 0x5567c597baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.951+0000 7fa5b660e740 1 --2- 172.21.15.69:0/3558045964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5567c596bd00 0x5567c597b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.951+0000 7fa5b660e740 1 -- 172.21.15.69:0/3558045964 >> 172.21.15.69:0/3558045964 conn(0x5567c5967270 msgr2=0x5567c596ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.951+0000 7fa592a00640 0 client.25604 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.951+0000 7fa5b660e740 1 -- 172.21.15.69:0/3558045964 shutdown_connections 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.951+0000 7fa5b660e740 1 -- 172.21.15.69:0/3558045964 wait complete. 2024-03-19T01:10:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.951+0000 7fa5b660e740 20 client.25604 trim_cache size 0 max 16384 2024-03-19T01:10:28.954 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:10:28.954 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:10:28.954 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "45e9e45e-8751-4b27-878e-3f26331a9a30", 2024-03-19T01:10:28.954 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:10:28.954 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:10:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 -- 172.21.15.69:0/532026648 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5940a2910 msgr2=0x7fa5940a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5940a2910 0x7fa5940a4dd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa5a001b040 tx=0x7fa5a0005850 comp rx=0 tx=0).stop 2024-03-19T01:10:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 -- 172.21.15.69:0/532026648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5ac075800 msgr2=0x7fa5ac104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5ac075800 0x7fa5ac104fe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa59800e9f0 tx=0x7fa59800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 -- 172.21.15.69:0/532026648 shutdown_connections 2024-03-19T01:10:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fa5940a2910 0x7fa5940a4dd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5ac1095b0 0x7fa5ac101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5ac075800 0x7fa5ac104fe0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 --2- 172.21.15.69:0/532026648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5ac074220 0x7fa5ac104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 -- 172.21.15.69:0/532026648 >> 172.21.15.69:0/532026648 conn(0x7fa5ac0fcc90 msgr2=0x7fa5ac105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 -- 172.21.15.69:0/532026648 shutdown_connections 2024-03-19T01:10:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:28.952+0000 7fa5b2400640 1 -- 172.21.15.69:0/532026648 wait complete. 2024-03-19T01:10:28.968 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:28 smithi069 ceph-mon[31727]: pgmap v415: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-19T01:10:28.968 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:28 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2616334907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:28.968 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:28 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3697031015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:28.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:10:28.971 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:10:29.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.157+0000 7fbda7200640 1 -- 172.21.15.69:0/2566354645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbda81021b0 msgr2=0x7fbda8108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:29.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.157+0000 7fbda7200640 1 --2- 172.21.15.69:0/2566354645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbda81021b0 0x7fbda8108a40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009ca0 tx=0x7fbda002c100 comp rx=0 tx=0).stop 2024-03-19T01:10:29.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.157+0000 7fbda7200640 1 -- 172.21.15.69:0/2566354645 shutdown_connections 2024-03-19T01:10:29.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.158+0000 7fbda7200640 1 --2- 172.21.15.69:0/2566354645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbda81021b0 0x7fbda8108a40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.158+0000 7fbda7200640 1 --2- 172.21.15.69:0/2566354645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbda81017f0 0x7fbda8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.158+0000 7fbda7200640 1 --2- 172.21.15.69:0/2566354645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda8100600 0x7fbda8100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.158+0000 7fbda7200640 1 -- 172.21.15.69:0/2566354645 >> 172.21.15.69:0/2566354645 conn(0x7fbda80fbb90 msgr2=0x7fbda80fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:29.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.158+0000 7fbda7200640 1 -- 172.21.15.69:0/2566354645 shutdown_connections 2024-03-19T01:10:29.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.158+0000 7fbda7200640 1 -- 172.21.15.69:0/2566354645 wait complete. 2024-03-19T01:10:29.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbda7200640 1 Processor -- start 2024-03-19T01:10:29.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbda7200640 1 -- start start 2024-03-19T01:10:29.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbda7200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbda8100600 0x7fbda8104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbda7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda81017f0 0x7fbda8102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbda7200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbda81021b0 0x7fbda8103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbda7200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810c280 con 0x7fbda8100600 2024-03-19T01:10:29.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbda7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810c100 con 0x7fbda81017f0 2024-03-19T01:10:29.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbda7200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810c400 con 0x7fbda81021b0 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbda5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbda8100600 0x7fbda8104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbda5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbda8100600 0x7fbda8104590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47162/0 (socket says 172.21.15.69:47162) 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbda5e00640 1 -- 172.21.15.69:0/3915873318 learned_addr learned my addr 172.21.15.69:0/3915873318 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.159+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda81017f0 0x7fbda8102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbda6800640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbda81021b0 0x7fbda8103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbd9fe00640 1 -- 172.21.15.69:0/3915873318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbda81021b0 msgr2=0x7fbda8103110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbda81021b0 0x7fbda8103110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbd9fe00640 1 -- 172.21.15.69:0/3915873318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbda8100600 msgr2=0x7fbda8104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbda8100600 0x7fbda8104590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbd9fe00640 1 -- 172.21.15.69:0/3915873318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda81039a0 con 0x7fbda81017f0 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbda6800640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbda81021b0 0x7fbda8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda81017f0 0x7fbda8102bd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000e9f0 tx=0x7fbd9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbd9ea00640 1 -- 172.21.15.69:0/3915873318 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000cd80 con 0x7fbda81017f0 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbd9ea00640 1 -- 172.21.15.69:0/3915873318 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd900035f0 con 0x7fbda81017f0 2024-03-19T01:10:29.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbd9ea00640 1 -- 172.21.15.69:0/3915873318 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90010690 con 0x7fbda81017f0 2024-03-19T01:10:29.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.160+0000 7fbda7200640 1 -- 172.21.15.69:0/3915873318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda81a1ad0 con 0x7fbda81017f0 2024-03-19T01:10:29.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.161+0000 7fbda7200640 1 -- 172.21.15.69:0/3915873318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda81a1fd0 con 0x7fbda81017f0 2024-03-19T01:10:29.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.165+0000 7fbd9ea00640 1 -- 172.21.15.69:0/3915873318 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fbd90003b30 con 0x7fbda81017f0 2024-03-19T01:10:29.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.165+0000 7fbdafe7a740 1 Processor -- start 2024-03-19T01:10:29.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.165+0000 7fbdafe7a740 1 -- start start 2024-03-19T01:10:29.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 0x5650f14e5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e5940 0x5650f14ea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 0x5650f14ecf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5650f144a2c0 con 0x5650f14eaab0 2024-03-19T01:10:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5650f13f2b60 con 0x5650f14e5940 2024-03-19T01:10:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5650f1446360 con 0x5650f14e4f50 2024-03-19T01:10:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbda6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 0x5650f14e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbda5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e5940 0x5650f14ea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbda6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 0x5650f14e5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42766/0 (socket says 172.21.15.69:42766) 2024-03-19T01:10:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbda6800640 1 -- 172.21.15.69:0/755939492 learned_addr learned my addr 172.21.15.69:0/755939492 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/755939492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 0x5650f14ecf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbda6800640 1 -- 172.21.15.69:0/755939492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e5940 msgr2=0x5650f14ea440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbda6800640 1 --2- 172.21.15.69:0/755939492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e5940 0x5650f14ea440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbda6800640 1 -- 172.21.15.69:0/755939492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 msgr2=0x5650f14ecf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:29.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbda6800640 1 --2- 172.21.15.69:0/755939492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 0x5650f14ecf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbda6800640 1 -- 172.21.15.69:0/755939492 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f12287d0 con 0x5650f14e4f50 2024-03-19T01:10:29.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.166+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/755939492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 0x5650f14ecf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:10:29.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.167+0000 7fbd9ea00640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd8c0a2910 0x7fbd8c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.167+0000 7fbda6800640 1 --2- 172.21.15.69:0/755939492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 0x5650f14e5370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbda00324c0 tx=0x7fbda0005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a74d6765f84bce49 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:29.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.167+0000 7fbd9ea00640 1 -- 172.21.15.69:0/3915873318 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fbd900cf2e0 con 0x7fbda81017f0 2024-03-19T01:10:29.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.167+0000 7fbd97400640 1 -- 172.21.15.69:0/755939492 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda0043020 con 0x5650f14e4f50 2024-03-19T01:10:29.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.167+0000 7fbda5e00640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd8c0a2910 0x7fbd8c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.167+0000 7fbd97400640 1 -- 172.21.15.69:0/755939492 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda0003970 con 0x5650f14e4f50 2024-03-19T01:10:29.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.167+0000 7fbd97400640 1 -- 172.21.15.69:0/755939492 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda0039040 con 0x5650f14e4f50 2024-03-19T01:10:29.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.168+0000 7fbdafe7a740 1 -- 172.21.15.69:0/755939492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 msgr2=0x5650f14e5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:29.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.168+0000 7fbda5e00640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd8c0a2910 0x7fbd8c0a4dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5650f14e5d40 tx=0x7fbd98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:29.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.168+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/755939492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 0x5650f14e5370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbda00324c0 tx=0x7fbda0005f70 comp rx=0 tx=0).stop 2024-03-19T01:10:29.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.168+0000 7fbdafe7a740 1 -- 172.21.15.69:0/755939492 shutdown_connections 2024-03-19T01:10:29.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.168+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/755939492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 0x5650f14ecf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.168+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/755939492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e5940 0x5650f14ea440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.168+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/755939492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 0x5650f14e5370 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.168+0000 7fbdafe7a740 1 -- 172.21.15.69:0/755939492 >> 172.21.15.69:0/755939492 conn(0x5650f14e04c0 msgr2=0x5650f14e08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:29.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.168+0000 7fbdafe7a740 1 -- 172.21.15.69:0/755939492 shutdown_connections 2024-03-19T01:10:29.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.168+0000 7fbdafe7a740 1 -- 172.21.15.69:0/755939492 wait complete. 2024-03-19T01:10:29.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.169+0000 7fbdafe7a740 1 -- start start 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.169+0000 7fbdafe7a740 1 Processor -- start 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.170+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 0x5650f14ef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.170+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e5940 0x5650f14efa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.170+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14eaab0 0x5650f14f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.170+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5650f14eda00 con 0x5650f14e5940 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.170+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5650f1405fc0 con 0x5650f14eaab0 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.170+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5650f14edb80 con 0x5650f14e4f50 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.170+0000 7fbdafe7a740 10 client.0 _ll_register_callbacks cb 0x5650f1405990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.170+0000 7fbd9fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e5940 0x5650f14efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.170+0000 7fbd9fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e5940 0x5650f14efa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47174/0 (socket says 172.21.15.69:47174) 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.170+0000 7fbd9fe00640 1 -- 172.21.15.69:0/3986924784 learned_addr learned my addr 172.21.15.69:0/3986924784 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbda5e00640 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 0x5650f14ef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbda6800640 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14eaab0 0x5650f14f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbd9fe00640 1 -- 172.21.15.69:0/3986924784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 msgr2=0x5650f14ef4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 0x5650f14ef4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbd9fe00640 1 -- 172.21.15.69:0/3986924784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14eaab0 msgr2=0x5650f14f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14eaab0 0x5650f14f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbd9fe00640 1 -- 172.21.15.69:0/3986924784 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f115e3f0 con 0x5650f14e5940 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e5940 0x5650f14efa00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd90098140 tx=0x7fbd9009b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63d1b2332906f02 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbd8be00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbd8be00640 1 -- 172.21.15.69:0/3986924784 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd900a3f10 con 0x5650f14e5940 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbd8be00640 1 -- 172.21.15.69:0/3986924784 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fbd900b4540 con 0x5650f14e5940 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.171+0000 7fbdafe7a740 1 -- 172.21.15.69:0/3986924784 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbda0003970 con 0x5650f14e5940 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.172+0000 7fbd8be00640 1 -- 172.21.15.69:0/3986924784 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd90009430 con 0x5650f14e5940 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.172+0000 7fbdafe7a740 10 client.15489 fetch_fsmap learned FSMap version 29 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.172+0000 7fbdafe7a740 1 -- 172.21.15.69:0/3986924784 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650f111e0f0 con 0x5650f14e5940 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.172+0000 7fbd8be00640 1 -- 172.21.15.69:0/3986924784 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda0003970 con 0x5650f14e5940 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.172+0000 7fbd8be00640 1 -- 172.21.15.69:0/3986924784 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbd90097de0 con 0x5650f14e5940 2024-03-19T01:10:29.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.172+0000 7fbdafe7a740 10 client.15489 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.172+0000 7fbdafe7a740 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.173+0000 7fbdafe7a740 20 client.15489 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.173+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5650f14fb330 0x5650f14fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.173+0000 7fbdafe7a740 4 client.15489 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.173+0000 7fbdafe7a740 1 -- 172.21.15.69:0/3986924784 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650f13eb610 con 0x5650f14fb330 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.173+0000 7fbda5e00640 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5650f14fb330 0x5650f14fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.173+0000 7fbda5e00640 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5650f14fb330 0x5650f14fd760 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62816a9cb2f89406 server_cookie=ed86bfcd9d1d5fae in_seq=0 out_seq=0 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.173+0000 7fbd8be00640 10 client.15489 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbd8be00640 1 -- 172.21.15.69:0/3986924784 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5650f13eb610 con 0x5650f14fb330 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbd8be00640 10 client.15489 handle_command_reply: tid=0 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbdafe7a740 1 client.15489 shutdown 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbdafe7a740 1 -- 172.21.15.69:0/3986924784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e5940 msgr2=0x5650f14efa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e5940 0x5650f14efa00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd90098140 tx=0x7fbd9009b1c0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbdafe7a740 1 -- 172.21.15.69:0/3986924784 shutdown_connections 2024-03-19T01:10:29.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5650f14fb330 0x5650f14fd760 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14eaab0 0x5650f14f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e5940 0x5650f14efa00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/3986924784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e4f50 0x5650f14ef4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbdafe7a740 1 -- 172.21.15.69:0/3986924784 >> 172.21.15.69:0/3986924784 conn(0x5650f14e04c0 msgr2=0x5650f14df6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:29.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.174+0000 7fbd8be00640 0 client.15489 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:29.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.175+0000 7fbdafe7a740 1 -- 172.21.15.69:0/3986924784 shutdown_connections 2024-03-19T01:10:29.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.175+0000 7fbdafe7a740 1 -- 172.21.15.69:0/3986924784 wait complete. 2024-03-19T01:10:29.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.175+0000 7fbdafe7a740 20 client.15489 trim_cache size 0 max 16384 2024-03-19T01:10:29.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbdafe7a740 1 Processor -- start 2024-03-19T01:10:29.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbdafe7a740 1 -- start start 2024-03-19T01:10:29.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e4f50 0x5650f14e6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 0x5650f14e7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 0x5650f14f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81ac540 con 0x5650f14eaab0 2024-03-19T01:10:29.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81ae080 con 0x5650f14e4f50 2024-03-19T01:10:29.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81acc10 con 0x5650f14e5940 2024-03-19T01:10:29.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbda5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e4f50 0x5650f14e6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbd9fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 0x5650f14e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbda5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e4f50 0x5650f14e6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38292/0 (socket says 172.21.15.69:38292) 2024-03-19T01:10:29.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.194+0000 7fbda5e00640 1 -- 172.21.15.69:0/1915551336 learned_addr learned my addr 172.21.15.69:0/1915551336 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:29.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.195+0000 7fbda5e00640 1 -- 172.21.15.69:0/1915551336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 msgr2=0x5650f14e7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:29.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.195+0000 7fbda6800640 1 --2- 172.21.15.69:0/1915551336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 0x5650f14f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.195+0000 7fbda5e00640 1 --2- 172.21.15.69:0/1915551336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 0x5650f14e7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.195+0000 7fbda5e00640 1 -- 172.21.15.69:0/1915551336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 msgr2=0x5650f14f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:29.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.195+0000 7fbda5e00640 1 --2- 172.21.15.69:0/1915551336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 0x5650f14f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.195+0000 7fbda5e00640 1 -- 172.21.15.69:0/1915551336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f14e1230 con 0x5650f14e4f50 2024-03-19T01:10:29.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.195+0000 7fbda6800640 1 --2- 172.21.15.69:0/1915551336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 0x5650f14f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:10:29.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.195+0000 7fbda5e00640 1 --2- 172.21.15.69:0/1915551336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e4f50 0x5650f14e6ed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbd980058c0 tx=0x7fbd98017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33247f5675de068 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:29.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.195+0000 7fbd96a00640 1 -- 172.21.15.69:0/1915551336 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9802b020 con 0x5650f14e4f50 2024-03-19T01:10:29.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.195+0000 7fbd96a00640 1 -- 172.21.15.69:0/1915551336 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9802b6a0 con 0x5650f14e4f50 2024-03-19T01:10:29.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.195+0000 7fbd96a00640 1 -- 172.21.15.69:0/1915551336 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98003e50 con 0x5650f14e4f50 2024-03-19T01:10:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.196+0000 7fbdafe7a740 1 -- 172.21.15.69:0/1915551336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e4f50 msgr2=0x5650f14e6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.196+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/1915551336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e4f50 0x5650f14e6ed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbd980058c0 tx=0x7fbd98017000 comp rx=0 tx=0).stop 2024-03-19T01:10:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.196+0000 7fbdafe7a740 1 -- 172.21.15.69:0/1915551336 shutdown_connections 2024-03-19T01:10:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.196+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/1915551336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14eaab0 0x5650f14f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.196+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/1915551336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 0x5650f14e7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.196+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/1915551336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14e4f50 0x5650f14e6ed0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.196+0000 7fbdafe7a740 1 -- 172.21.15.69:0/1915551336 >> 172.21.15.69:0/1915551336 conn(0x5650f14e04c0 msgr2=0x5650f14ef4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.196+0000 7fbdafe7a740 1 -- 172.21.15.69:0/1915551336 shutdown_connections 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.196+0000 7fbdafe7a740 1 -- 172.21.15.69:0/1915551336 wait complete. 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbdafe7a740 1 -- start start 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbdafe7a740 1 Processor -- start 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e4f50 0x5650f14f3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 0x5650f14f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbdafe7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14eaab0 0x5650f155f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81ad9b0 con 0x5650f14e4f50 2024-03-19T01:10:29.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81ad2e0 con 0x5650f14eaab0 2024-03-19T01:10:29.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbdafe7a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81aa520 con 0x5650f14e5940 2024-03-19T01:10:29.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbdafe7a740 10 client.0 _ll_register_callbacks cb 0x7fbd9009b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:29.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbd9fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 0x5650f14f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbda5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e4f50 0x5650f14f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbd9fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 0x5650f14f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42794/0 (socket says 172.21.15.69:42794) 2024-03-19T01:10:29.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbd9fe00640 1 -- 172.21.15.69:0/327685360 learned_addr learned my addr 172.21.15.69:0/327685360 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:29.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.197+0000 7fbda6800640 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14eaab0 0x5650f155f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbd9fe00640 1 -- 172.21.15.69:0/327685360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14eaab0 msgr2=0x5650f155f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:29.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14eaab0 0x5650f155f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbd9fe00640 1 -- 172.21.15.69:0/327685360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e4f50 msgr2=0x5650f14f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:29.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e4f50 0x5650f14f3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbd9fe00640 1 -- 172.21.15.69:0/327685360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f14f6d30 con 0x5650f14e5940 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbda6800640 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14eaab0 0x5650f155f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbda5e00640 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e4f50 0x5650f14f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbd9fe00640 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 0x5650f14f3c60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbd900cce60 tx=0x7fbd900cf240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdf4206e552dc212 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbd8be00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbd8be00640 1 -- 172.21.15.69:0/327685360 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd900a3f10 con 0x5650f14e5940 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbdafe7a740 1 -- 172.21.15.69:0/327685360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd9802b6a0 con 0x5650f14e5940 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.198+0000 7fbd8be00640 1 -- 172.21.15.69:0/327685360 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fbd90097de0 con 0x5650f14e5940 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbd8be00640 1 -- 172.21.15.69:0/327685360 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd9009b770 con 0x5650f14e5940 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbdafe7a740 10 client.25630 fetch_fsmap learned FSMap version 29 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbdafe7a740 1 -- 172.21.15.69:0/327685360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650f1560b80 con 0x5650f14e5940 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbd8be00640 1 -- 172.21.15.69:0/327685360 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9802b6a0 con 0x5650f14e5940 2024-03-19T01:10:29.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbd8be00640 1 -- 172.21.15.69:0/327685360 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbd9009a630 con 0x5650f14e5940 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbdafe7a740 10 client.25630 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbdafe7a740 10 client.25630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbdafe7a740 20 client.25630 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5650f1561520 0x5650f1563940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbdafe7a740 4 client.25630 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbdafe7a740 1 -- 172.21.15.69:0/327685360 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650f1457800 con 0x5650f1561520 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.199+0000 7fbda5e00640 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5650f1561520 0x5650f1563940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbda5e00640 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5650f1561520 0x5650f1563940 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6024018fb7368cd7 server_cookie=5369451dfacdb0d4 in_seq=0 out_seq=0 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbd8be00640 10 client.25630 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbd8be00640 1 -- 172.21.15.69:0/327685360 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5650f1457800 con 0x5650f1561520 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbd8be00640 10 client.25630 handle_command_reply: tid=0 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbdafe7a740 1 client.25630 shutdown 2024-03-19T01:10:29.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbdafe7a740 1 -- 172.21.15.69:0/327685360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 msgr2=0x5650f14f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:29.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 0x5650f14f3c60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbd900cce60 tx=0x7fbd900cf240 comp rx=0 tx=0).stop 2024-03-19T01:10:29.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbdafe7a740 1 -- 172.21.15.69:0/327685360 shutdown_connections 2024-03-19T01:10:29.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5650f1561520 0x5650f1563940 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5650f14eaab0 0x5650f155f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5650f14e5940 0x5650f14f3c60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbd8be00640 0 client.25630 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:29.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbdafe7a740 1 --2- 172.21.15.69:0/327685360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650f14e4f50 0x5650f14f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.200+0000 7fbdafe7a740 1 -- 172.21.15.69:0/327685360 >> 172.21.15.69:0/327685360 conn(0x5650f14e04c0 msgr2=0x5650f14ef3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:29.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.201+0000 7fbdafe7a740 1 -- 172.21.15.69:0/327685360 shutdown_connections 2024-03-19T01:10:29.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.201+0000 7fbdafe7a740 1 -- 172.21.15.69:0/327685360 wait complete. 2024-03-19T01:10:29.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.201+0000 7fbdafe7a740 20 client.25630 trim_cache size 0 max 16384 2024-03-19T01:10:29.209 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:10:29.209 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T01:10:29.209 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:10:29.209 INFO:teuthology.orchestra.run.smithi069.stdout: "45e9e45e-8751-4b27-878e-3f26331a9a30": { 2024-03-19T01:10:29.209 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:10:29.209 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "45e9e45e-8751-4b27-878e-3f26331a9a30", 2024-03-19T01:10:29.209 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:10:29.209 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:10:29.209 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:10:29.209 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:10:29.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.201+0000 7fbda7200640 1 -- 172.21.15.69:0/3915873318 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd8c0a2910 msgr2=0x7fbd8c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:29.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.201+0000 7fbda7200640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd8c0a2910 0x7fbd8c0a4dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5650f14e5d40 tx=0x7fbd98005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:29.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.201+0000 7fbda7200640 1 -- 172.21.15.69:0/3915873318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda81017f0 msgr2=0x7fbda8102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:29.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.201+0000 7fbda7200640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda81017f0 0x7fbda8102bd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000e9f0 tx=0x7fbd9000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.202+0000 7fbda7200640 1 -- 172.21.15.69:0/3915873318 shutdown_connections 2024-03-19T01:10:29.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.202+0000 7fbda7200640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fbd8c0a2910 0x7fbd8c0a4dd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.202+0000 7fbda7200640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbda81021b0 0x7fbda8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.202+0000 7fbda7200640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbda81017f0 0x7fbda8102bd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.202+0000 7fbda7200640 1 --2- 172.21.15.69:0/3915873318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbda8100600 0x7fbda8104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.202+0000 7fbda7200640 1 -- 172.21.15.69:0/3915873318 >> 172.21.15.69:0/3915873318 conn(0x7fbda80fbb90 msgr2=0x7fbda80fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:29.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.202+0000 7fbda7200640 1 -- 172.21.15.69:0/3915873318 shutdown_connections 2024-03-19T01:10:29.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.202+0000 7fbda7200640 1 -- 172.21.15.69:0/3915873318 wait complete. 2024-03-19T01:10:29.221 INFO:tasks.cephfs.filesystem:scrub status for tag:45e9e45e-8751-4b27-878e-3f26331a9a30 - {'path': '/', 'tag': '45e9e45e-8751-4b27-878e-3f26331a9a30', 'options': 'recursive,force'} 2024-03-19T01:10:29.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:28 smithi086 ceph-mon[66829]: pgmap v415: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-19T01:10:29.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:28 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2616334907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:29.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:28 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3697031015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:29.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.304+0000 7fc19e000640 1 -- 172.21.15.69:0/2089386773 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fc198108cc0 con 0x7fc198103a80 2024-03-19T01:10:29.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:28 smithi163 ceph-mon[67428]: pgmap v415: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-19T01:10:29.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:28 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2616334907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:10:29.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:28 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3697031015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:10:29.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.893+0000 7fc18fe00640 1 -- 172.21.15.69:0/2089386773 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 20) ==== 137358+0+0 (secure 0 0 0) 0x7fc1880979b0 con 0x7fc198103a80 2024-03-19T01:10:29.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.893+0000 7fc18fe00640 1 -- 172.21.15.69:0/2089386773 <== mon.1 v2:172.21.15.163:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) ==== 92+0+0 (secure 0 0 0) 0x7fc188062ce0 con 0x7fc198103a80 2024-03-19T01:10:29.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.900+0000 7fc19e000640 1 -- 172.21.15.69:0/2089386773 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc17c0a29c0 msgr2=0x7fc17c0a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:29.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.900+0000 7fc19e000640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc17c0a29c0 0x7fc17c0a4e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc18402c240 tx=0x7fc18402df90 comp rx=0 tx=0).stop 2024-03-19T01:10:29.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.900+0000 7fc19e000640 1 -- 172.21.15.69:0/2089386773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc198103a80 msgr2=0x7fc198110bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:29.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.900+0000 7fc19e000640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc198103a80 0x7fc198110bc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc18800d950 tx=0x7fc18800de10 comp rx=0 tx=0).stop 2024-03-19T01:10:29.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.901+0000 7fc19e000640 1 -- 172.21.15.69:0/2089386773 shutdown_connections 2024-03-19T01:10:29.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.901+0000 7fc19e000640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fc17c0a29c0 0x7fc17c0a4e80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.901+0000 7fc19e000640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc198105630 0x7fc198116110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.901+0000 7fc19e000640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc198104c70 0x7fc198111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.901+0000 7fc19e000640 1 --2- 172.21.15.69:0/2089386773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc198103a80 0x7fc198110bc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:29.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.901+0000 7fc19e000640 1 -- 172.21.15.69:0/2089386773 >> 172.21.15.69:0/2089386773 conn(0x7fc1980ff010 msgr2=0x7fc198105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:29.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.901+0000 7fc19e000640 1 -- 172.21.15.69:0/2089386773 shutdown_connections 2024-03-19T01:10:29.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:29.901+0000 7fc19e000640 1 -- 172.21.15.69:0/2089386773 wait complete. 2024-03-19T01:10:29.929 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-19T01:10:30.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.139+0000 7f2755600640 1 -- 172.21.15.69:0/1735640184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750075860 msgr2=0x7f2750073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:30.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.139+0000 7f2755600640 1 --2- 172.21.15.69:0/1735640184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750075860 0x7f2750073cb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2740009930 tx=0x7f274002bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:30.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.139+0000 7f2755600640 1 -- 172.21.15.69:0/1735640184 shutdown_connections 2024-03-19T01:10:30.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.139+0000 7f2755600640 1 --2- 172.21.15.69:0/1735640184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f275010de20 0x7f2750110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.139+0000 7f2755600640 1 --2- 172.21.15.69:0/1735640184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2750074280 0x7f2750074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.139+0000 7f2755600640 1 --2- 172.21.15.69:0/1735640184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750075860 0x7f2750073cb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.139+0000 7f2755600640 1 -- 172.21.15.69:0/1735640184 >> 172.21.15.69:0/1735640184 conn(0x7f27500fcde0 msgr2=0x7f27500ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:30.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.139+0000 7f2755600640 1 -- 172.21.15.69:0/1735640184 shutdown_connections 2024-03-19T01:10:30.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.139+0000 7f2755600640 1 -- 172.21.15.69:0/1735640184 wait complete. 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.140+0000 7f2755600640 1 Processor -- start 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.140+0000 7f2755600640 1 -- start start 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.140+0000 7f2755600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2750074280 0x7f275019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.140+0000 7f2755600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750075860 0x7f275019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.140+0000 7f2755600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f275010de20 0x7f27501a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.140+0000 7f2755600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2750113af0 con 0x7f275010de20 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.140+0000 7f2755600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2750113970 con 0x7f2750075860 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.140+0000 7f2755600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2750113c70 con 0x7f2750074280 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.141+0000 7f274ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2750074280 0x7f275019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.141+0000 7f274e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750075860 0x7f275019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.141+0000 7f274ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2750074280 0x7f275019e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:42832/0 (socket says 172.21.15.69:42832) 2024-03-19T01:10:30.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.141+0000 7f274ea00640 1 -- 172.21.15.69:0/288424687 learned_addr learned my addr 172.21.15.69:0/288424687 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.141+0000 7f274f400640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f275010de20 0x7f27501a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.141+0000 7f274ea00640 1 -- 172.21.15.69:0/288424687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750075860 msgr2=0x7f275019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.141+0000 7f274ea00640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750075860 0x7f275019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.141+0000 7f274ea00640 1 -- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f275010de20 msgr2=0x7f27501a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.141+0000 7f274ea00640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f275010de20 0x7f27501a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.141+0000 7f274ea00640 1 -- 172.21.15.69:0/288424687 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27501a5cb0 con 0x7f2750074280 2024-03-19T01:10:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.141+0000 7f274ea00640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2750074280 0x7f275019e090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2740000c00 tx=0x7f274002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.142+0000 7f2747400640 1 -- 172.21.15.69:0/288424687 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740043020 con 0x7f2750074280 2024-03-19T01:10:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.142+0000 7f2747400640 1 -- 172.21.15.69:0/288424687 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27400054e0 con 0x7f2750074280 2024-03-19T01:10:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.142+0000 7f2755600640 1 -- 172.21.15.69:0/288424687 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27501a5f40 con 0x7f2750074280 2024-03-19T01:10:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.142+0000 7f2747400640 1 -- 172.21.15.69:0/288424687 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740003c40 con 0x7f2750074280 2024-03-19T01:10:30.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.143+0000 7f2755600640 1 -- 172.21.15.69:0/288424687 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27501a63a0 con 0x7f2750074280 2024-03-19T01:10:30.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.144+0000 7f2755600640 1 -- 172.21.15.69:0/288424687 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2750103ff0 con 0x7f2750074280 2024-03-19T01:10:30.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.145+0000 7f2747400640 1 -- 172.21.15.69:0/288424687 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 137358+0+0 (secure 0 0 0) 0x7f2740046020 con 0x7f2750074280 2024-03-19T01:10:30.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.147+0000 7f2747400640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27340a2990 0x7f27340a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:30.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.147+0000 7f274e000640 1 -- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27340a2990 msgr2=0x7f27340a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3562156769 2024-03-19T01:10:30.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.147+0000 7f2747400640 1 -- 172.21.15.69:0/288424687 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f27400f0cb0 con 0x7f2750074280 2024-03-19T01:10:30.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.147+0000 7f274e000640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27340a2990 0x7f27340a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:10:30.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.151+0000 7f2747400640 1 -- 172.21.15.69:0/288424687 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f274004d030 con 0x7f2750074280 2024-03-19T01:10:30.255 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:29 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: ignoring --setuser ceph since I am not root 2024-03-19T01:10:30.256 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:29 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: ignoring --setgroup ceph since I am not root 2024-03-19T01:10:30.256 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:29 smithi086 ceph-mgr[68231]: -- 172.21.15.86:0/3518069548 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a51551e5a0 con 0x55a5154f2c00 2024-03-19T01:10:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:29 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:10:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:10:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:10:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:10:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:29 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2089386773' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-19T01:10:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:29 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-19T01:10:30.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: ignoring --setuser ceph since I am not root 2024-03-19T01:10:30.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: ignoring --setgroup ceph since I am not root 2024-03-19T01:10:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:29 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:10:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:10:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:10:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:10:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:29 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2089386773' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-19T01:10:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:29 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-19T01:10:30.311 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:29 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: ignoring --setuser ceph since I am not root 2024-03-19T01:10:30.311 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:29 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: ignoring --setgroup ceph since I am not root 2024-03-19T01:10:30.311 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:29 smithi163 ceph-mgr[69297]: -- 172.21.15.163:0/2701987010 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574649585a0 con 0x55746492d000 2024-03-19T01:10:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:29 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:10:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:10:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:10:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:10:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:29 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2089386773' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-19T01:10:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:29 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-19T01:10:30.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.348+0000 7f274e000640 1 -- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27340a2990 msgr2=0x7f27340a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3562156769 2024-03-19T01:10:30.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.348+0000 7f274e000640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27340a2990 0x7f27340a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T01:10:30.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.425+0000 7f2755600640 1 -- 172.21.15.69:0/288424687 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f27500624c0 con 0x7f2750074280 2024-03-19T01:10:30.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.432+0000 7f2747400640 1 -- 172.21.15.69:0/288424687 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f27400bd950 con 0x7f2750074280 2024-03-19T01:10:30.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 -- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27340a2990 msgr2=0x7f27340a4e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:10:30.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27340a2990 0x7f27340a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 -- 172.21.15.69:0/288424687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2750074280 msgr2=0x7f275019e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:30.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2750074280 0x7f275019e090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2740000c00 tx=0x7f274002c650 comp rx=0 tx=0).stop 2024-03-19T01:10:30.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 -- 172.21.15.69:0/288424687 shutdown_connections 2024-03-19T01:10:30.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7f27340a2990 0x7f27340a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f275010de20 0x7f27501a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2750075860 0x7f275019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 --2- 172.21.15.69:0/288424687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2750074280 0x7f275019e090 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 -- 172.21.15.69:0/288424687 >> 172.21.15.69:0/288424687 conn(0x7f27500fcde0 msgr2=0x7f27501011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 -- 172.21.15.69:0/288424687 shutdown_connections 2024-03-19T01:10:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.435+0000 7f2755600640 1 -- 172.21.15.69:0/288424687 wait complete. 2024-03-19T01:10:30.455 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-19T01:10:30.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.644+0000 7fd7a0a00640 1 -- 172.21.15.69:0/2823061300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79c103240 msgr2=0x7fd79c109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:30.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.644+0000 7fd7a0a00640 1 --2- 172.21.15.69:0/2823061300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79c103240 0x7fd79c109ad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd794009be0 tx=0x7fd79402bef0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.644+0000 7fd7a0a00640 1 -- 172.21.15.69:0/2823061300 shutdown_connections 2024-03-19T01:10:30.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.644+0000 7fd7a0a00640 1 --2- 172.21.15.69:0/2823061300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79c103240 0x7fd79c109ad0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.644+0000 7fd7a0a00640 1 --2- 172.21.15.69:0/2823061300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd79c102880 0x7fd79c102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.644+0000 7fd7a0a00640 1 --2- 172.21.15.69:0/2823061300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c101690 0x7fd79c101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.644+0000 7fd7a0a00640 1 -- 172.21.15.69:0/2823061300 >> 172.21.15.69:0/2823061300 conn(0x7fd79c0fcc00 msgr2=0x7fd79c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:30.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.645+0000 7fd7a0a00640 1 -- 172.21.15.69:0/2823061300 shutdown_connections 2024-03-19T01:10:30.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.645+0000 7fd7a0a00640 1 -- 172.21.15.69:0/2823061300 wait complete. 2024-03-19T01:10:30.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.646+0000 7fd7a0a00640 1 Processor -- start 2024-03-19T01:10:30.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.646+0000 7fd7a0a00640 1 -- start start 2024-03-19T01:10:30.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.646+0000 7fd7a0a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd79c101690 0x7fd79c107410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:30.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.646+0000 7fd7a0a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c102880 0x7fd79c107950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:30.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.646+0000 7fd7a0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79c103240 0x7fd79c103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:30.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.646+0000 7fd7a0a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c10d360 con 0x7fd79c101690 2024-03-19T01:10:30.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.646+0000 7fd7a0a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c10d1e0 con 0x7fd79c103240 2024-03-19T01:10:30.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.646+0000 7fd7a0a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c10d4e0 con 0x7fd79c102880 2024-03-19T01:10:30.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.647+0000 7fd79b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd79c101690 0x7fd79c107410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.647+0000 7fd79b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd79c101690 0x7fd79c107410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47240/0 (socket says 172.21.15.69:47240) 2024-03-19T01:10:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.647+0000 7fd79b400640 1 -- 172.21.15.69:0/1876024753 learned_addr learned my addr 172.21.15.69:0/1876024753 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.647+0000 7fd79aa00640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c102880 0x7fd79c107950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.647+0000 7fd79be00640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79c103240 0x7fd79c103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.647+0000 7fd79b400640 1 -- 172.21.15.69:0/1876024753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c102880 msgr2=0x7fd79c107950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.647+0000 7fd79b400640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c102880 0x7fd79c107950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.647+0000 7fd79b400640 1 -- 172.21.15.69:0/1876024753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79c103240 msgr2=0x7fd79c103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.647+0000 7fd79b400640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79c103240 0x7fd79c103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.647+0000 7fd79b400640 1 -- 172.21.15.69:0/1876024753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c104840 con 0x7fd79c101690 2024-03-19T01:10:30.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.647+0000 7fd79b400640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd79c101690 0x7fd79c107410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd78800e9f0 tx=0x7fd78800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:30.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.648+0000 7fd793e00640 1 -- 172.21.15.69:0/1876024753 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78800ccc0 con 0x7fd79c101690 2024-03-19T01:10:30.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.648+0000 7fd793e00640 1 -- 172.21.15.69:0/1876024753 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7880035f0 con 0x7fd79c101690 2024-03-19T01:10:30.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.648+0000 7fd793e00640 1 -- 172.21.15.69:0/1876024753 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78800ce60 con 0x7fd79c101690 2024-03-19T01:10:30.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.648+0000 7fd7a0a00640 1 -- 172.21.15.69:0/1876024753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c1a5f80 con 0x7fd79c101690 2024-03-19T01:10:30.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.648+0000 7fd7a0a00640 1 -- 172.21.15.69:0/1876024753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c1a6330 con 0x7fd79c101690 2024-03-19T01:10:30.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.649+0000 7fd7a0a00640 1 -- 172.21.15.69:0/1876024753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79c04fe00 con 0x7fd79c101690 2024-03-19T01:10:30.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.654+0000 7fd793e00640 1 -- 172.21.15.69:0/1876024753 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 137358+0+0 (secure 0 0 0) 0x7fd788003b10 con 0x7fd79c101690 2024-03-19T01:10:30.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.656+0000 7fd793e00640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7800a2950 0x7fd7800a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:30.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.657+0000 7fd79aa00640 1 -- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7800a2950 msgr2=0x7fd7800a4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3562156769 2024-03-19T01:10:30.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.657+0000 7fd793e00640 1 -- 172.21.15.69:0/1876024753 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fd7880cf310 con 0x7fd79c101690 2024-03-19T01:10:30.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.657+0000 7fd79aa00640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7800a2950 0x7fd7800a4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:10:30.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.657+0000 7fd793e00640 1 -- 172.21.15.69:0/1876024753 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd79c04fe00 con 0x7fd79c101690 2024-03-19T01:10:30.811 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:30 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:30.477+0000 7fd0b9e88200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T01:10:30.811 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:30 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:30.571+0000 7fd0b9e88200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T01:10:30.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.858+0000 7fd79aa00640 1 -- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7800a2950 msgr2=0x7fd7800a4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3562156769 2024-03-19T01:10:30.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.858+0000 7fd79aa00640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7800a2950 0x7fd7800a4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T01:10:30.910 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:30 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:30.520+0000 7f591b557200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T01:10:30.910 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:30 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:30.614+0000 7f591b557200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T01:10:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.909+0000 7fd7a0a00640 1 -- 172.21.15.69:0/1876024753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fd79c1a65e0 con 0x7fd79c101690 2024-03-19T01:10:30.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.918+0000 7fd793e00640 1 -- 172.21.15.69:0/1876024753 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7fd78809be40 con 0x7fd79c101690 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.920+0000 7fd7a0a00640 1 -- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7800a2950 msgr2=0x7fd7800a4e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.920+0000 7fd7a0a00640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7800a2950 0x7fd7800a4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.921+0000 7fd7a0a00640 1 -- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd79c101690 msgr2=0x7fd79c107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.921+0000 7fd7a0a00640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd79c101690 0x7fd79c107410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd78800e9f0 tx=0x7fd78800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.921+0000 7fd7a0a00640 1 -- 172.21.15.69:0/1876024753 shutdown_connections 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.921+0000 7fd7a0a00640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fd7800a2950 0x7fd7800a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.921+0000 7fd7a0a00640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd79c103240 0x7fd79c103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.921+0000 7fd7a0a00640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c102880 0x7fd79c107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.921+0000 7fd7a0a00640 1 --2- 172.21.15.69:0/1876024753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd79c101690 0x7fd79c107410 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.921+0000 7fd7a0a00640 1 -- 172.21.15.69:0/1876024753 >> 172.21.15.69:0/1876024753 conn(0x7fd79c0fcc00 msgr2=0x7fd79c073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.921+0000 7fd7a0a00640 1 -- 172.21.15.69:0/1876024753 shutdown_connections 2024-03-19T01:10:30.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:30.921+0000 7fd7a0a00640 1 -- 172.21.15.69:0/1876024753 wait complete. 2024-03-19T01:10:30.938 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-19T01:10:31.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.109+0000 7fdf23200640 1 -- 172.21.15.69:0/1888649524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1c107ca0 msgr2=0x7fdf1c1080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:31.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.109+0000 7fdf23200640 1 --2- 172.21.15.69:0/1888649524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1c107ca0 0x7fdf1c1080c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009900 tx=0x7fdf1002bc50 comp rx=0 tx=0).stop 2024-03-19T01:10:31.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.110+0000 7fdf23200640 1 -- 172.21.15.69:0/1888649524 shutdown_connections 2024-03-19T01:10:31.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.110+0000 7fdf23200640 1 --2- 172.21.15.69:0/1888649524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1c109850 0x7fdf1c1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:31.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.110+0000 7fdf23200640 1 --2- 172.21.15.69:0/1888649524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf1c108e90 0x7fdf1c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:31.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.110+0000 7fdf23200640 1 --2- 172.21.15.69:0/1888649524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1c107ca0 0x7fdf1c1080c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:31.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.110+0000 7fdf23200640 1 -- 172.21.15.69:0/1888649524 >> 172.21.15.69:0/1888649524 conn(0x7fdf1c076c60 msgr2=0x7fdf1c079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:31.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.110+0000 7fdf23200640 1 -- 172.21.15.69:0/1888649524 shutdown_connections 2024-03-19T01:10:31.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.110+0000 7fdf23200640 1 -- 172.21.15.69:0/1888649524 wait complete. 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf23200640 1 Processor -- start 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf23200640 1 -- start start 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf23200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1c107ca0 0x7fdf1c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf23200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf1c108e90 0x7fdf1c19e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf23200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1c109850 0x7fdf1c1a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf23200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c113970 con 0x7fdf1c107ca0 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf23200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1137f0 con 0x7fdf1c108e90 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf23200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c113af0 con 0x7fdf1c109850 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf1c108e90 0x7fdf1c19e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf1c108e90 0x7fdf1c19e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38376/0 (socket says 172.21.15.69:38376) 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf22800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1c109850 0x7fdf1c1a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf21400640 1 -- 172.21.15.69:0/2739660682 learned_addr learned my addr 172.21.15.69:0/2739660682 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:31.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.111+0000 7fdf21e00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1c107ca0 0x7fdf1c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.112+0000 7fdf22800640 1 -- 172.21.15.69:0/2739660682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf1c108e90 msgr2=0x7fdf1c19e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.112+0000 7fdf22800640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf1c108e90 0x7fdf1c19e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.112+0000 7fdf22800640 1 -- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1c107ca0 msgr2=0x7fdf1c19df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.112+0000 7fdf22800640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1c107ca0 0x7fdf1c19df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.112+0000 7fdf22800640 1 -- 172.21.15.69:0/2739660682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c1a5ba0 con 0x7fdf1c109850 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.112+0000 7fdf21400640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf1c108e90 0x7fdf1c19e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.112+0000 7fdf22800640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1c109850 0x7fdf1c1a5530 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009fd0 tx=0x7fdf1800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.112+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800ce20 con 0x7fdf1c109850 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.112+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf18005020 con 0x7fdf1c109850 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.112+0000 7fdf23200640 1 -- 172.21.15.69:0/2739660682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c1a5e90 con 0x7fdf1c109850 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.112+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18010720 con 0x7fdf1c109850 2024-03-19T01:10:31.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.113+0000 7fdf23200640 1 -- 172.21.15.69:0/2739660682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c1a63a0 con 0x7fdf1c109850 2024-03-19T01:10:31.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.114+0000 7fdf23200640 1 -- 172.21.15.69:0/2739660682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c10efc0 con 0x7fdf1c109850 2024-03-19T01:10:31.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.117+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 137358+0+0 (secure 0 0 0) 0x7fdf18005540 con 0x7fdf1c109850 2024-03-19T01:10:31.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.120+0000 7fdf16a00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 0x7fdf040a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:31.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.120+0000 7fdf21e00640 1 -- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 msgr2=0x7fdf040a4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3562156769 2024-03-19T01:10:31.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.120+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7fdf180cf5f0 con 0x7fdf1c109850 2024-03-19T01:10:31.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.120+0000 7fdf21e00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 0x7fdf040a4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:10:31.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.120+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdf180cfb10 con 0x7fdf1c109850 2024-03-19T01:10:31.219 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:30 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-19T01:10:31.219 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:30 smithi163 ceph-mon[67428]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-19T01:10:31.220 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:30 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' 2024-03-19T01:10:31.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:30 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-19T01:10:31.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:30 smithi086 ceph-mon[66829]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-19T01:10:31.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:30 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' 2024-03-19T01:10:31.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:30.936+0000 7f61b95f3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T01:10:31.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:31.118+0000 7f61b95f3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T01:10:31.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:30 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-19T01:10:31.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:30 smithi069 ceph-mon[31727]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-19T01:10:31.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:30 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' 2024-03-19T01:10:31.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.321+0000 7fdf21e00640 1 -- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 msgr2=0x7fdf040a4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3562156769 2024-03-19T01:10:31.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.321+0000 7fdf21e00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 0x7fdf040a4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T01:10:31.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.362+0000 7fdf23200640 1 -- 172.21.15.69:0/2739660682 --> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fdf1c061c60 con 0x7fdf040a2950 2024-03-19T01:10:31.551 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:31 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:31.218+0000 7fd0b9e88200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T01:10:31.551 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:31 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:31.312+0000 7fd0b9e88200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T01:10:31.551 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:31 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:31.454+0000 7fd0b9e88200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T01:10:31.683 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:31 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:31.424+0000 7f591b557200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T01:10:31.683 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:31 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:31.530+0000 7f591b557200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T01:10:31.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.722+0000 7fdf21e00640 1 -- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 msgr2=0x7fdf040a4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3562156769 2024-03-19T01:10:31.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:31.722+0000 7fdf21e00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 0x7fdf040a4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-19T01:10:31.811 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:31 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:31.550+0000 7fd0b9e88200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T01:10:31.811 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:31 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:31.633+0000 7fd0b9e88200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T01:10:32.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:31 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1876024753' entity='client.admin' 2024-03-19T01:10:32.006 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:31 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:31.681+0000 7f591b557200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T01:10:32.006 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:31 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:31.780+0000 7f591b557200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T01:10:32.006 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:31 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:31.871+0000 7f591b557200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T01:10:32.227 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:31 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1876024753' entity='client.admin' 2024-03-19T01:10:32.243 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:31 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:31.934+0000 7fd0b9e88200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T01:10:32.243 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:32 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:32.044+0000 7fd0b9e88200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T01:10:32.243 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:32 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:32.130+0000 7fd0b9e88200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T01:10:32.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:31.998+0000 7f61b95f3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T01:10:32.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:32.122+0000 7f61b95f3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T01:10:32.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:31 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1876024753' entity='client.admin' 2024-03-19T01:10:32.481 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:32.164+0000 7f591b557200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T01:10:32.481 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:32.290+0000 7f591b557200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T01:10:32.481 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:32.376+0000 7f591b557200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T01:10:32.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:32.523+0000 7fdf21e00640 1 -- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 msgr2=0x7fdf040a4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3562156769 2024-03-19T01:10:32.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:32.523+0000 7fdf21e00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 0x7fdf040a4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-19T01:10:32.561 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:32 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:32.226+0000 7fd0b9e88200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T01:10:32.561 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:32 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:32.404+0000 7fd0b9e88200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T01:10:32.755 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:32.480+0000 7f591b557200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T01:10:32.756 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:32.685+0000 7f591b557200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T01:10:32.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:32.304+0000 7f61b95f3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T01:10:32.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:32.431+0000 7f61b95f3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T01:10:32.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:32.543+0000 7f61b95f3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T01:10:33.154 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:32.891+0000 7f61b95f3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T01:10:33.154 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:33 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:33.036+0000 7f61b95f3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T01:10:33.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:33 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:33.152+0000 7f61b95f3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T01:10:33.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:33 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:33.281+0000 7f61b95f3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T01:10:33.811 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:33 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:33.374+0000 7fd0b9e88200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T01:10:33.811 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:33 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:33.505+0000 7fd0b9e88200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T01:10:34.005 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:33 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:33.724+0000 7f591b557200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T01:10:34.006 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:33 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:33.856+0000 7f591b557200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T01:10:34.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:33 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:33.536+0000 7f61b95f3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T01:10:34.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:34.124+0000 7fdf21e00640 1 -- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 msgr2=0x7fdf040a4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3562156769 2024-03-19T01:10:34.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:34.124+0000 7fdf21e00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 0x7fdf040a4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-19T01:10:34.128 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:33 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:33.840+0000 7fd0b9e88200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T01:10:34.128 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:33 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:33.931+0000 7fd0b9e88200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T01:10:34.129 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:34 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:34.127+0000 7fd0b9e88200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:10:34.505 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:34 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:34.217+0000 7f591b557200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T01:10:34.506 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:34 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:34.317+0000 7f591b557200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T01:10:34.561 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:34 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:34.313+0000 7fd0b9e88200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T01:10:34.980 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:34 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:34.649+0000 7fd0b9e88200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:10:35.005 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:34 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:34.583+0000 7f591b557200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:10:35.005 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:34 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:34.810+0000 7f591b557200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T01:10:35.311 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:34 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:34.979+0000 7fd0b9e88200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T01:10:35.311 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:35 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:35.220+0000 7fd0b9e88200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T01:10:35.416 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:35 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:35.140+0000 7f591b557200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:10:35.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:35.041+0000 7f61b95f3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T01:10:35.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:35.233+0000 7f61b95f3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T01:10:35.755 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:35 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:35.415+0000 7f591b557200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T01:10:35.756 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:35 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:35.656+0000 7f591b557200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T01:10:36.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:35.679+0000 7f61b95f3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T01:10:36.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:35.800+0000 7f61b95f3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T01:10:36.340 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:36 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:36.063+0000 7f61b95f3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:10:36.561 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:36 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:36.071+0000 7fd0b9e88200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T01:10:36.734 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:36 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:36.339+0000 7f61b95f3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T01:10:37.005 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:36 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:36.602+0000 7f591b557200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T01:10:37.020 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:36 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:36.659+0000 7fd0b9e88200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T01:10:37.020 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:36 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:36.758+0000 7fd0b9e88200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T01:10:37.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:36 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:36.732+0000 7f61b95f3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:10:37.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:37.104+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 21) ==== 137358+0+0 (secure 0 0 0) 0x7fdf180980f0 con 0x7fdf1c109850 2024-03-19T01:10:37.311 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:10:37 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z[69293]: 2024-03-19T01:10:37.019+0000 7fd0b9e88200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T01:10:37.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:37 smithi163 ceph-mon[67428]: Standby manager daemon z restarted 2024-03-19T01:10:37.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:37 smithi163 ceph-mon[67428]: Standby manager daemon z started 2024-03-19T01:10:37.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:37 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T01:10:37.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:37 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:10:37.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:37 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T01:10:37.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:37 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:10:37.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:37.327+0000 7fdf21e00640 1 -- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 msgr2=0x7fdf040a4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3562156769 2024-03-19T01:10:37.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:37.327+0000 7fdf21e00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 0x7fdf040a4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-19T01:10:37.363 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:37 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:37.248+0000 7f591b557200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T01:10:37.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:37 smithi086 ceph-mon[66829]: Standby manager daemon z restarted 2024-03-19T01:10:37.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:37 smithi086 ceph-mon[66829]: Standby manager daemon z started 2024-03-19T01:10:37.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:37 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T01:10:37.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:37 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:10:37.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:37 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T01:10:37.364 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:37 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:10:37.433 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:37 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:37.106+0000 7f61b95f3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T01:10:37.433 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:37 smithi069 ceph-mon[31727]: Standby manager daemon z restarted 2024-03-19T01:10:37.433 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:37 smithi069 ceph-mon[31727]: Standby manager daemon z started 2024-03-19T01:10:37.434 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:37 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T01:10:37.434 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:37 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:10:37.434 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:37 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T01:10:37.434 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:37 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.163:0/3654568874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:10:37.755 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:37 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:37.362+0000 7f591b557200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T01:10:37.756 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:10:37 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y[68227]: 2024-03-19T01:10:37.579+0000 7f591b557200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T01:10:37.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:37 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:37.432+0000 7f61b95f3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T01:10:38.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:38.127+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 <== mon.1 v2:172.21.15.163:3300/0 8 ==== mgrmap(e 22) ==== 137358+0+0 (secure 0 0 0) 0x7fdf1809aed0 con 0x7fdf1c109850 2024-03-19T01:10:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:38 smithi086 ceph-mon[66829]: mgrmap e21: x(active, since 11m), standbys: y, z 2024-03-19T01:10:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:38 smithi086 ceph-mon[66829]: Standby manager daemon y restarted 2024-03-19T01:10:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:38 smithi086 ceph-mon[66829]: Standby manager daemon y started 2024-03-19T01:10:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:38 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T01:10:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:38 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:10:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:38 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T01:10:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:38 smithi086 ceph-mon[66829]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:10:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:38 smithi069 ceph-mon[31727]: mgrmap e21: x(active, since 11m), standbys: y, z 2024-03-19T01:10:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:38 smithi069 ceph-mon[31727]: Standby manager daemon y restarted 2024-03-19T01:10:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:38 smithi069 ceph-mon[31727]: Standby manager daemon y started 2024-03-19T01:10:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:38 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T01:10:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:38 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:10:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:38 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T01:10:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:38 smithi069 ceph-mon[31727]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:10:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:38 smithi163 ceph-mon[67428]: mgrmap e21: x(active, since 11m), standbys: y, z 2024-03-19T01:10:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:38 smithi163 ceph-mon[67428]: Standby manager daemon y restarted 2024-03-19T01:10:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:38 smithi163 ceph-mon[67428]: Standby manager daemon y started 2024-03-19T01:10:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:38 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T01:10:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:38 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:10:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:38 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T01:10:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:38 smithi163 ceph-mon[67428]: from='mgr.? 172.21.15.86:0/2119555801' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:10:39.032 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:38 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:38.669+0000 7f61b95f3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T01:10:39.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:39 smithi086 ceph-mon[66829]: mgrmap e22: x(active, since 11m), standbys: y, z 2024-03-19T01:10:39.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:39 smithi069 ceph-mon[31727]: mgrmap e22: x(active, since 11m), standbys: y, z 2024-03-19T01:10:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:39 smithi163 ceph-mon[67428]: mgrmap e22: x(active, since 11m), standbys: y, z 2024-03-19T01:10:40.003 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:39 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:39.553+0000 7f61b95f3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T01:10:40.003 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:39 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:39.695+0000 7f61b95f3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T01:10:40.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:40.022+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 <== mon.1 v2:172.21.15.163:3300/0 9 ==== mgrmap(e 23) ==== 136975+0+0 (secure 0 0 0) 0x7fdf180a3d90 con 0x7fdf1c109850 2024-03-19T01:10:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:40.022+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 msgr2=0x7fdf040a4e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:10:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:40.022+0000 7fdf16a00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/3562156769,v1:172.21.15.69:6801/3562156769] conn(0x7fdf040a2950 0x7fdf040a4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:40.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:40.001+0000 7f61b95f3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T01:10:40.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: Active manager daemon x restarted 2024-03-19T01:10:40.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: Activating manager daemon x 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: osdmap e94: 12 total, 12 up, 12 in 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: mgrmap e23: x(active, starting, since 0.0183189s), standbys: y, z 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:10:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:10:40.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: Manager daemon x is now available 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:40.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:40.359 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: Active manager daemon x restarted 2024-03-19T01:10:40.359 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: Activating manager daemon x 2024-03-19T01:10:40.359 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: osdmap e94: 12 total, 12 up, 12 in 2024-03-19T01:10:40.359 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: mgrmap e23: x(active, starting, since 0.0183189s), standbys: y, z 2024-03-19T01:10:40.359 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:10:40.359 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:10:40.359 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T01:10:40.360 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T01:10:40.361 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T01:10:40.362 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: Manager daemon x is now available 2024-03-19T01:10:40.362 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:40.362 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:40 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:40.398 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: Active manager daemon x restarted 2024-03-19T01:10:40.398 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: Activating manager daemon x 2024-03-19T01:10:40.398 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: osdmap e94: 12 total, 12 up, 12 in 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: mgrmap e23: x(active, starting, since 0.0183189s), standbys: y, z 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:10:40.399 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:10:40.400 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:10:40.401 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:10:40.401 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:10:40.401 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T01:10:40.401 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T01:10:40.401 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T01:10:40.401 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: Manager daemon x is now available 2024-03-19T01:10:40.401 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:40.401 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:40 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:40.680 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:40.370+0000 7f610be00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:40.680 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:40.370+0000 7f610be00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:40.680 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:40.370+0000 7f610be00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:40.680 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:40.370+0000 7f610be00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:40.680 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:40.370+0000 7f610be00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:40.680 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:40.371+0000 7f610dc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:40.680 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:40.371+0000 7f610dc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:40.681 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:40.371+0000 7f610dc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:40.681 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:40.371+0000 7f610dc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:40.681 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:10:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:10:40.371+0000 7f610dc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:10:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.032+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 <== mon.1 v2:172.21.15.163:3300/0 10 ==== mgrmap(e 24) ==== 137102+0+0 (secure 0 0 0) 0x7fdf180989c0 con 0x7fdf1c109850 2024-03-19T01:10:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.033+0000 7fdf16a00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdf04067f70 0x7fdf04068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.033+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 --> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fdf18020e70 con 0x7fdf04067f70 2024-03-19T01:10:41.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.041+0000 7fdf21e00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdf04067f70 0x7fdf04068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:41.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.047+0000 7fdf21e00640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdf04067f70 0x7fdf04068350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdf1002c190 tx=0x7fdf10000bc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:41.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.141+0000 7fdf16a00640 1 -- 172.21.15.69:0/2739660682 <== mgr.25645 v2:172.21.15.69:6800/2931658273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdf18020e70 con 0x7fdf04067f70 2024-03-19T01:10:41.143 INFO:teuthology.orchestra.run.smithi069.stdout:Schedule set for path / 2024-03-19T01:10:41.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.144+0000 7fdf23200640 1 -- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdf04067f70 msgr2=0x7fdf04068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:41.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.144+0000 7fdf23200640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdf04067f70 0x7fdf04068350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdf1002c190 tx=0x7fdf10000bc0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.144+0000 7fdf23200640 1 -- 172.21.15.69:0/2739660682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1c109850 msgr2=0x7fdf1c1a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:41.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.144+0000 7fdf23200640 1 --2- 172.21.15.69:0/2739660682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1c109850 0x7fdf1c1a5530 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009fd0 tx=0x7fdf1800ef10 comp rx=0 tx=0).stop 2024-03-19T01:10:41.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.144+0000 7fdf22800640 1 -- 172.21.15.69:0/2739660682 reap_dead start 2024-03-19T01:10:41.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.145+0000 7fdf23200640 1 -- 172.21.15.69:0/2739660682 shutdown_connections 2024-03-19T01:10:41.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.145+0000 7fdf23200640 1 -- 172.21.15.69:0/2739660682 >> 172.21.15.69:0/2739660682 conn(0x7fdf1c076c60 msgr2=0x7fdf1c10c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:41.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.145+0000 7fdf23200640 1 -- 172.21.15.69:0/2739660682 shutdown_connections 2024-03-19T01:10:41.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.145+0000 7fdf23200640 1 -- 172.21.15.69:0/2739660682 wait complete. 2024-03-19T01:10:41.168 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-19T01:10:41.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.344+0000 7f8437200640 1 -- 172.21.15.69:0/2214396846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8438104a90 msgr2=0x7f8438104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:41.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.344+0000 7f8437200640 1 --2- 172.21.15.69:0/2214396846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8438104a90 0x7f8438104f10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8424009900 tx=0x7f842402bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:41.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.344+0000 7f8437200640 1 -- 172.21.15.69:0/2214396846 shutdown_connections 2024-03-19T01:10:41.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.344+0000 7f8437200640 1 --2- 172.21.15.69:0/2214396846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8438105450 0x7f843810bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.344+0000 7f8437200640 1 --2- 172.21.15.69:0/2214396846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8438104a90 0x7f8438104f10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.344+0000 7f8437200640 1 --2- 172.21.15.69:0/2214396846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84381038a0 0x7f8438103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.344+0000 7f8437200640 1 -- 172.21.15.69:0/2214396846 >> 172.21.15.69:0/2214396846 conn(0x7f84380fee30 msgr2=0x7f8438101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:41.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.344+0000 7f8437200640 1 -- 172.21.15.69:0/2214396846 shutdown_connections 2024-03-19T01:10:41.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.344+0000 7f8437200640 1 -- 172.21.15.69:0/2214396846 wait complete. 2024-03-19T01:10:41.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.345+0000 7f8437200640 1 Processor -- start 2024-03-19T01:10:41.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.345+0000 7f8437200640 1 -- start start 2024-03-19T01:10:41.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.345+0000 7f8437200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84381038a0 0x7f8438199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:41.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.345+0000 7f8437200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438104a90 0x7f843819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:41.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.345+0000 7f8437200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8438105450 0x7f84381a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:41.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8437200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f843810f570 con 0x7f8438105450 2024-03-19T01:10:41.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8437200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f843810f3f0 con 0x7f8438104a90 2024-03-19T01:10:41.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8437200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f843810f6f0 con 0x7f84381038a0 2024-03-19T01:10:41.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8435e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84381038a0 0x7f8438199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8435400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438104a90 0x7f843819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8435e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84381038a0 0x7f8438199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:49584/0 (socket says 172.21.15.69:49584) 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8435e00640 1 -- 172.21.15.69:0/554322 learned_addr learned my addr 172.21.15.69:0/554322 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8436800640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8438105450 0x7f84381a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8435400640 1 -- 172.21.15.69:0/554322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84381038a0 msgr2=0x7f8438199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8435400640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84381038a0 0x7f8438199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8435400640 1 -- 172.21.15.69:0/554322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8438105450 msgr2=0x7f84381a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8435400640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8438105450 0x7f84381a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8435400640 1 -- 172.21.15.69:0/554322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84381a17a0 con 0x7f8438104a90 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8436800640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8438105450 0x7f84381a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.346+0000 7f8435400640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438104a90 0x7f843819a0c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8424036040 tx=0x7f842402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.347+0000 7f842ea00640 1 -- 172.21.15.69:0/554322 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424043020 con 0x7f8438104a90 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.347+0000 7f8437200640 1 -- 172.21.15.69:0/554322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84381a1a30 con 0x7f8438104a90 2024-03-19T01:10:41.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.347+0000 7f842ea00640 1 -- 172.21.15.69:0/554322 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84240054e0 con 0x7f8438104a90 2024-03-19T01:10:41.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.347+0000 7f842ea00640 1 -- 172.21.15.69:0/554322 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424003c10 con 0x7f8438104a90 2024-03-19T01:10:41.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.347+0000 7f8437200640 1 -- 172.21.15.69:0/554322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84381a1f50 con 0x7f8438104a90 2024-03-19T01:10:41.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.348+0000 7f8437200640 1 -- 172.21.15.69:0/554322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843810ab60 con 0x7f8438104a90 2024-03-19T01:10:41.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.350+0000 7f842ea00640 1 -- 172.21.15.69:0/554322 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 137102+0+0 (secure 0 0 0) 0x7f8424046020 con 0x7f8438104a90 2024-03-19T01:10:41.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.351+0000 7f842ea00640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f841c0a25e0 0x7f841c0a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:41.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.351+0000 7f842ea00640 1 -- 172.21.15.69:0/554322 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f84240f0bf0 con 0x7f8438104a90 2024-03-19T01:10:41.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.351+0000 7f8435e00640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f841c0a25e0 0x7f841c0a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:41.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.352+0000 7f8435e00640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f841c0a25e0 0x7f841c0a4aa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f84280058c0 tx=0x7f8428005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:41.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.355+0000 7f842ea00640 1 -- 172.21.15.69:0/554322 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f842404d020 con 0x7f8438104a90 2024-03-19T01:10:41.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:41 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T01:10:41.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:41 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T01:10:41.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:41 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T01:10:41.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:41 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T01:10:41.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:41 smithi086 ceph-mon[66829]: mgrmap e24: x(active, since 1.03485s), standbys: y, z 2024-03-19T01:10:41.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:41 smithi086 ceph-mon[66829]: from='client.25663 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:41.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:41 smithi086 ceph-mon[66829]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:41.506 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:41 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T01:10:41.506 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:41 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T01:10:41.506 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:41 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T01:10:41.507 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:41 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T01:10:41.507 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:41 smithi163 ceph-mon[67428]: mgrmap e24: x(active, since 1.03485s), standbys: y, z 2024-03-19T01:10:41.507 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:41 smithi163 ceph-mon[67428]: from='client.25663 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:41.507 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:41 smithi163 ceph-mon[67428]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:41 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T01:10:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:41 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T01:10:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:41 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T01:10:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:41 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T01:10:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:41 smithi069 ceph-mon[31727]: mgrmap e24: x(active, since 1.03485s), standbys: y, z 2024-03-19T01:10:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:41 smithi069 ceph-mon[31727]: from='client.25663 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:41 smithi069 ceph-mon[31727]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:41.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.649+0000 7f8437200640 1 -- 172.21.15.69:0/554322 --> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f8438061d00 con 0x7f841c0a25e0 2024-03-19T01:10:41.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.724+0000 7f842ea00640 1 -- 172.21.15.69:0/554322 <== mgr.25645 v2:172.21.15.69:6800/2931658273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8438061d00 con 0x7f841c0a25e0 2024-03-19T01:10:41.725 INFO:teuthology.orchestra.run.smithi069.stdout:Retention added to path / 2024-03-19T01:10:41.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 -- 172.21.15.69:0/554322 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f841c0a25e0 msgr2=0x7f841c0a4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:41.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f841c0a25e0 0x7f841c0a4aa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f84280058c0 tx=0x7f8428005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:41.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 -- 172.21.15.69:0/554322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438104a90 msgr2=0x7f843819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438104a90 0x7f843819a0c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8424036040 tx=0x7f842402c650 comp rx=0 tx=0).stop 2024-03-19T01:10:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 -- 172.21.15.69:0/554322 shutdown_connections 2024-03-19T01:10:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8438105450 0x7f84381a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f841c0a25e0 0x7f841c0a4aa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438104a90 0x7f843819a0c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 --2- 172.21.15.69:0/554322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84381038a0 0x7f8438199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 -- 172.21.15.69:0/554322 >> 172.21.15.69:0/554322 conn(0x7f84380fee30 msgr2=0x7f8438107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 -- 172.21.15.69:0/554322 shutdown_connections 2024-03-19T01:10:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.727+0000 7f8437200640 1 -- 172.21.15.69:0/554322 wait complete. 2024-03-19T01:10:41.745 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-19T01:10:41.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.961+0000 7faaa1c00640 1 -- 172.21.15.69:0/1561465872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa9c1020f0 msgr2=0x7faa9c1044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:41.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.961+0000 7faaa1c00640 1 --2- 172.21.15.69:0/1561465872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa9c1020f0 0x7faa9c1044e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009900 tx=0x7faa8c02bc50 comp rx=0 tx=0).stop 2024-03-19T01:10:41.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.962+0000 7faaa1c00640 1 -- 172.21.15.69:0/1561465872 shutdown_connections 2024-03-19T01:10:41.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.962+0000 7faaa1c00640 1 --2- 172.21.15.69:0/1561465872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa9c107350 0x7faa9c1097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.962+0000 7faaa1c00640 1 --2- 172.21.15.69:0/1561465872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9c104a20 0x7faa9c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.962+0000 7faaa1c00640 1 --2- 172.21.15.69:0/1561465872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa9c1020f0 0x7faa9c1044e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.962+0000 7faaa1c00640 1 -- 172.21.15.69:0/1561465872 >> 172.21.15.69:0/1561465872 conn(0x7faa9c0fbb00 msgr2=0x7faa9c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:41.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.962+0000 7faaa1c00640 1 -- 172.21.15.69:0/1561465872 shutdown_connections 2024-03-19T01:10:41.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.962+0000 7faaa1c00640 1 -- 172.21.15.69:0/1561465872 wait complete. 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.963+0000 7faaa1c00640 1 Processor -- start 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.963+0000 7faaa1c00640 1 -- start start 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faaa1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa9c1020f0 0x7faa9c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faaa1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9c104a20 0x7faa9c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faaa1c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa9c107350 0x7faa9c1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faaa1c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c10d350 con 0x7faa9c107350 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faaa1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c10d1d0 con 0x7faa9c104a20 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faaa1c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c10d4d0 con 0x7faa9c1020f0 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faa9be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa9c1020f0 0x7faa9c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faa9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9c104a20 0x7faa9c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faa9be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa9c1020f0 0x7faa9c199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:49602/0 (socket says 172.21.15.69:49602) 2024-03-19T01:10:41.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faa9be00640 1 -- 172.21.15.69:0/3336122698 learned_addr learned my addr 172.21.15.69:0/3336122698 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:41.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faa9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9c104a20 0x7faa9c19a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55866/0 (socket says 172.21.15.69:55866) 2024-03-19T01:10:41.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faaa1200640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa9c107350 0x7faa9c1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:41.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faa9be00640 1 -- 172.21.15.69:0/3336122698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9c104a20 msgr2=0x7faa9c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:41.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faa9be00640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9c104a20 0x7faa9c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faa9be00640 1 -- 172.21.15.69:0/3336122698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa9c107350 msgr2=0x7faa9c1a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:41.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.964+0000 7faa9be00640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa9c107350 0x7faa9c1a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:41.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.965+0000 7faa9be00640 1 -- 172.21.15.69:0/3336122698 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c1a17c0 con 0x7faa9c1020f0 2024-03-19T01:10:41.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.965+0000 7faa9be00640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa9c1020f0 0x7faa9c199ba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faa8c02c260 tx=0x7faa8c02c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:41.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.965+0000 7faa98c00640 1 -- 172.21.15.69:0/3336122698 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c041070 con 0x7faa9c1020f0 2024-03-19T01:10:41.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.965+0000 7faa98c00640 1 -- 172.21.15.69:0/3336122698 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa8c004450 con 0x7faa9c1020f0 2024-03-19T01:10:41.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.965+0000 7faaa1c00640 1 -- 172.21.15.69:0/3336122698 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9c1a1a50 con 0x7faa9c1020f0 2024-03-19T01:10:41.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.965+0000 7faa98c00640 1 -- 172.21.15.69:0/3336122698 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c034440 con 0x7faa9c1020f0 2024-03-19T01:10:41.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.965+0000 7faaa1c00640 1 -- 172.21.15.69:0/3336122698 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9c1a1ff0 con 0x7faa9c1020f0 2024-03-19T01:10:41.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.966+0000 7faaa1c00640 1 -- 172.21.15.69:0/3336122698 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9c108ea0 con 0x7faa9c1020f0 2024-03-19T01:10:41.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.970+0000 7faa98c00640 1 -- 172.21.15.69:0/3336122698 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 24) ==== 137102+0+0 (secure 0 0 0) 0x7faa8c044070 con 0x7faa9c1020f0 2024-03-19T01:10:41.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.971+0000 7faa98c00640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faa800a2660 0x7faa800a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:41.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.971+0000 7faa98c00640 1 -- 172.21.15.69:0/3336122698 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7faa8c0f0560 con 0x7faa9c1020f0 2024-03-19T01:10:41.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.971+0000 7faa9b400640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faa800a2660 0x7faa800a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:41.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.972+0000 7faa9b400640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faa800a2660 0x7faa800a4b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa880095f0 tx=0x7faa88005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:41.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:41.977+0000 7faa98c00640 1 -- 172.21.15.69:0/3336122698 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faa8c0bcdc0 con 0x7faa9c1020f0 2024-03-19T01:10:42.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.153+0000 7faa98c00640 1 -- 172.21.15.69:0/3336122698 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 25) ==== 137410+0+0 (secure 0 0 0) 0x7faa8c0b9430 con 0x7faa9c1020f0 2024-03-19T01:10:42.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.274+0000 7faaa1c00640 1 -- 172.21.15.69:0/3336122698 --> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7faa9c061c60 con 0x7faa800a2660 2024-03-19T01:10:42.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.283+0000 7faa98c00640 1 -- 172.21.15.69:0/3336122698 <== mgr.25645 v2:172.21.15.69:6800/2931658273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7faa9c061c60 con 0x7faa800a2660 2024-03-19T01:10:42.284 INFO:teuthology.orchestra.run.smithi069.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-19T00:00:00", "created": "2024-03-19T01:10:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.286+0000 7faaa1c00640 1 -- 172.21.15.69:0/3336122698 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faa800a2660 msgr2=0x7faa800a4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.286+0000 7faaa1c00640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faa800a2660 0x7faa800a4b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa880095f0 tx=0x7faa88005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.286+0000 7faaa1c00640 1 -- 172.21.15.69:0/3336122698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa9c1020f0 msgr2=0x7faa9c199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.286+0000 7faaa1c00640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa9c1020f0 0x7faa9c199ba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faa8c02c260 tx=0x7faa8c02c670 comp rx=0 tx=0).stop 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.287+0000 7faaa1c00640 1 -- 172.21.15.69:0/3336122698 shutdown_connections 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.287+0000 7faaa1c00640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa9c107350 0x7faa9c1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.287+0000 7faaa1c00640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faa800a2660 0x7faa800a4b20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.287+0000 7faaa1c00640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9c104a20 0x7faa9c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.287+0000 7faaa1c00640 1 --2- 172.21.15.69:0/3336122698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa9c1020f0 0x7faa9c199ba0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.287+0000 7faaa1c00640 1 -- 172.21.15.69:0/3336122698 >> 172.21.15.69:0/3336122698 conn(0x7faa9c0fbb00 msgr2=0x7faa9c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.287+0000 7faaa1c00640 1 -- 172.21.15.69:0/3336122698 shutdown_connections 2024-03-19T01:10:42.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.287+0000 7faaa1c00640 1 -- 172.21.15.69:0/3336122698 wait complete. 2024-03-19T01:10:42.304 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-19T01:10:42.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.500+0000 7f86ef200640 1 -- 172.21.15.69:0/4131562012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f0101240 msgr2=0x7f86f01016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:42.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.500+0000 7f86ef200640 1 --2- 172.21.15.69:0/4131562012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f0101240 0x7f86f01016c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009ca0 tx=0x7f86e802c100 comp rx=0 tx=0).stop 2024-03-19T01:10:42.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.500+0000 7f86ef200640 1 -- 172.21.15.69:0/4131562012 shutdown_connections 2024-03-19T01:10:42.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.500+0000 7f86ef200640 1 --2- 172.21.15.69:0/4131562012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f0101240 0x7f86f01016c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.500+0000 7f86ef200640 1 --2- 172.21.15.69:0/4131562012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f01008a0 0x7f86f0100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.500+0000 7f86ef200640 1 --2- 172.21.15.69:0/4131562012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86f00fff40 0x7f86f0100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.500+0000 7f86ef200640 1 -- 172.21.15.69:0/4131562012 >> 172.21.15.69:0/4131562012 conn(0x7f86f00fbb00 msgr2=0x7f86f00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:42.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.500+0000 7f86ef200640 1 -- 172.21.15.69:0/4131562012 shutdown_connections 2024-03-19T01:10:42.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.500+0000 7f86ef200640 1 -- 172.21.15.69:0/4131562012 wait complete. 2024-03-19T01:10:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.501+0000 7f86ef200640 1 Processor -- start 2024-03-19T01:10:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.501+0000 7f86ef200640 1 -- start start 2024-03-19T01:10:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ef200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f00fff40 0x7f86f0199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ef200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86f01008a0 0x7f86f019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ef200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f0101240 0x7f86f01a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ef200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86f010f4c0 con 0x7f86f01008a0 2024-03-19T01:10:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ef200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86f010f340 con 0x7f86f0101240 2024-03-19T01:10:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ef200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f86f010f640 con 0x7f86f00fff40 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ede00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f00fff40 0x7f86f0199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ed400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86f01008a0 0x7f86f019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ede00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f00fff40 0x7f86f0199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:49634/0 (socket says 172.21.15.69:49634) 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ede00640 1 -- 172.21.15.69:0/2531358962 learned_addr learned my addr 172.21.15.69:0/2531358962 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ee800640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f0101240 0x7f86f01a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ede00640 1 -- 172.21.15.69:0/2531358962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f0101240 msgr2=0x7f86f01a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ede00640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f0101240 0x7f86f01a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ede00640 1 -- 172.21.15.69:0/2531358962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86f01008a0 msgr2=0x7f86f019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ede00640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86f01008a0 0x7f86f019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.502+0000 7f86ede00640 1 -- 172.21.15.69:0/2531358962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01a16f0 con 0x7f86f00fff40 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.503+0000 7f86ede00640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f00fff40 0x7f86f0199ad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f86e000e9f0 tx=0x7f86e000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.503+0000 7f86e6a00640 1 -- 172.21.15.69:0/2531358962 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e000cd80 con 0x7f86f00fff40 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.503+0000 7f86ef200640 1 -- 172.21.15.69:0/2531358962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01a19e0 con 0x7f86f00fff40 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.503+0000 7f86e6a00640 1 -- 172.21.15.69:0/2531358962 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e0010910 con 0x7f86f00fff40 2024-03-19T01:10:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.503+0000 7f86e6a00640 1 -- 172.21.15.69:0/2531358962 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e00188f0 con 0x7f86f00fff40 2024-03-19T01:10:42.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.503+0000 7f86ef200640 1 -- 172.21.15.69:0/2531358962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a1f30 con 0x7f86f00fff40 2024-03-19T01:10:42.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.505+0000 7f86ef200640 1 -- 172.21.15.69:0/2531358962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f004fdd0 con 0x7f86f00fff40 2024-03-19T01:10:42.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.506+0000 7f86e6a00640 1 -- 172.21.15.69:0/2531358962 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 25) ==== 137410+0+0 (secure 0 0 0) 0x7f86e0010ac0 con 0x7f86f00fff40 2024-03-19T01:10:42.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.507+0000 7f86e6a00640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f86d40a2960 0x7f86d40a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:42.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.508+0000 7f86e6a00640 1 -- 172.21.15.69:0/2531358962 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f86e00cf640 con 0x7f86f00fff40 2024-03-19T01:10:42.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.508+0000 7f86ed400640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f86d40a2960 0x7f86d40a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:42.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.509+0000 7f86ed400640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f86d40a2960 0x7f86d40a4e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0095f0 tx=0x7f86dc005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:42.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.511+0000 7f86e6a00640 1 -- 172.21.15.69:0/2531358962 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f86e009b930 con 0x7f86f00fff40 2024-03-19T01:10:42.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.787+0000 7f86ef200640 1 -- 172.21.15.69:0/2531358962 --> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f86f0061d20 con 0x7f86d40a2960 2024-03-19T01:10:42.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.797+0000 7f86e6a00640 1 -- 172.21.15.69:0/2531358962 <== mgr.25645 v2:172.21.15.69:6800/2931658273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f86f0061d20 con 0x7f86d40a2960 2024-03-19T01:10:42.798 INFO:teuthology.orchestra.run.smithi069.stdout:/ 1m 6m3h 2024-03-19T01:10:42.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.800+0000 7f86ef200640 1 -- 172.21.15.69:0/2531358962 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f86d40a2960 msgr2=0x7f86d40a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.800+0000 7f86ef200640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f86d40a2960 0x7f86d40a4e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0095f0 tx=0x7f86dc005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.800+0000 7f86ef200640 1 -- 172.21.15.69:0/2531358962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f00fff40 msgr2=0x7f86f0199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.800+0000 7f86ef200640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f00fff40 0x7f86f0199ad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f86e000e9f0 tx=0x7f86e000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.800+0000 7f86ef200640 1 -- 172.21.15.69:0/2531358962 shutdown_connections 2024-03-19T01:10:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.800+0000 7f86ef200640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f86d40a2960 0x7f86d40a4e20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.800+0000 7f86ef200640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86f0101240 0x7f86f01a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.800+0000 7f86ef200640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86f01008a0 0x7f86f019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.800+0000 7f86ef200640 1 --2- 172.21.15.69:0/2531358962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f00fff40 0x7f86f0199ad0 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f86e000e9f0 tx=0x7f86e000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:10:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.800+0000 7f86ef200640 1 -- 172.21.15.69:0/2531358962 >> 172.21.15.69:0/2531358962 conn(0x7f86f00fbb00 msgr2=0x7f86f00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.801+0000 7f86ef200640 1 -- 172.21.15.69:0/2531358962 shutdown_connections 2024-03-19T01:10:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:42.801+0000 7f86ef200640 1 -- 172.21.15.69:0/2531358962 wait complete. 2024-03-19T01:10:42.819 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-19T01:10:42.896 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-19T01:10:42.906 INFO:teuthology.run_tasks:Running task workunit... 2024-03-19T01:10:42.919 INFO:tasks.workunit:Pulling workunits from ref e48b1f6c0608b2615e1890cc46286c3877d6e910 2024-03-19T01:10:42.919 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-19T01:10:42.920 INFO:tasks.workunit:timeout=3h 2024-03-19T01:10:42.920 INFO:tasks.workunit:cleanup=True 2024-03-19T01:10:42.920 DEBUG:teuthology.orchestra.run.smithi069:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-19T01:10:42.965 INFO:teuthology.orchestra.run.smithi069.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-19T01:10:42.965 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-19T01:10:42.965 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 76h/118d Inode: 1099511628283 Links: 2 2024-03-19T01:10:42.965 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:10:42.965 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-19T01:10:42.966 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-03-19 01:07:52.426172356 +0000 2024-03-19T01:10:42.966 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-03-19 01:07:52.426172356 +0000 2024-03-19T01:10:42.966 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-03-19 01:10:16.173238476 +0000 2024-03-19T01:10:42.966 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-03-19 01:07:52.426172356 +0000 2024-03-19T01:10:42.966 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-19T01:10:42.966 DEBUG:teuthology.orchestra.run.smithi069:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-19T01:10:43.074 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:43 smithi069 ceph-mon[31727]: [19/Mar/2024:01:10:41] ENGINE Bus STARTING 2024-03-19T01:10:43.074 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:43 smithi069 ceph-mon[31727]: [19/Mar/2024:01:10:41] ENGINE Serving on http://172.21.15.69:8765 2024-03-19T01:10:43.074 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:43 smithi069 ceph-mon[31727]: [19/Mar/2024:01:10:41] ENGINE Serving on https://172.21.15.69:7150 2024-03-19T01:10:43.074 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:43 smithi069 ceph-mon[31727]: [19/Mar/2024:01:10:41] ENGINE Bus STARTED 2024-03-19T01:10:43.074 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:43 smithi069 ceph-mon[31727]: from='client.25697 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:43.074 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:43 smithi069 ceph-mon[31727]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:43.074 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:43 smithi069 ceph-mon[31727]: mgrmap e25: x(active, since 2s), standbys: y, z 2024-03-19T01:10:43.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:43 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:43.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:43 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:43.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:43 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:43.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:43 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:43.098 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e48b1f6c0608b2615e1890cc46286c3877d6e910 2024-03-19T01:10:43.146 INFO:tasks.workunit.client.0.smithi069.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-19T01:10:43.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:43 smithi163 ceph-mon[67428]: [19/Mar/2024:01:10:41] ENGINE Bus STARTING 2024-03-19T01:10:43.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:43 smithi163 ceph-mon[67428]: [19/Mar/2024:01:10:41] ENGINE Serving on http://172.21.15.69:8765 2024-03-19T01:10:43.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:43 smithi163 ceph-mon[67428]: [19/Mar/2024:01:10:41] ENGINE Serving on https://172.21.15.69:7150 2024-03-19T01:10:43.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:43 smithi163 ceph-mon[67428]: [19/Mar/2024:01:10:41] ENGINE Bus STARTED 2024-03-19T01:10:43.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:43 smithi163 ceph-mon[67428]: from='client.25697 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:43.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:43 smithi163 ceph-mon[67428]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:43.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:43 smithi163 ceph-mon[67428]: mgrmap e25: x(active, since 2s), standbys: y, z 2024-03-19T01:10:43.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:43 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:43.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:43 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:43.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:43 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:43.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:43 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:43 smithi086 ceph-mon[66829]: [19/Mar/2024:01:10:41] ENGINE Bus STARTING 2024-03-19T01:10:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:43 smithi086 ceph-mon[66829]: [19/Mar/2024:01:10:41] ENGINE Serving on http://172.21.15.69:8765 2024-03-19T01:10:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:43 smithi086 ceph-mon[66829]: [19/Mar/2024:01:10:41] ENGINE Serving on https://172.21.15.69:7150 2024-03-19T01:10:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:43 smithi086 ceph-mon[66829]: [19/Mar/2024:01:10:41] ENGINE Bus STARTED 2024-03-19T01:10:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:43 smithi086 ceph-mon[66829]: from='client.25697 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:43 smithi086 ceph-mon[66829]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:43 smithi086 ceph-mon[66829]: mgrmap e25: x(active, since 2s), standbys: y, z 2024-03-19T01:10:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:43 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:43 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:43 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:43 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='client.25702 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='client.25714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:44 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='client.25702 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:44.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='client.25714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:44.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:44 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='client.25702 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='client.25714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:10:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:44 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:45.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:45 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi163 to 174.1M 2024-03-19T01:10:45.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:45 smithi163 ceph-mon[67428]: Unable to set osd_memory_target on smithi163 to 182627737: error parsing value: Value '182627737' is below minimum 939524096 2024-03-19T01:10:45.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:45 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi086 to 174.1M 2024-03-19T01:10:45.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:45 smithi163 ceph-mon[67428]: Unable to set osd_memory_target on smithi086 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-19T01:10:45.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:45 smithi163 ceph-mon[67428]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:45.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:45 smithi163 ceph-mon[67428]: mgrmap e26: x(active, since 4s), standbys: y, z 2024-03-19T01:10:45.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:45 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi163 to 174.1M 2024-03-19T01:10:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:45 smithi086 ceph-mon[66829]: Unable to set osd_memory_target on smithi163 to 182627737: error parsing value: Value '182627737' is below minimum 939524096 2024-03-19T01:10:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:45 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi086 to 174.1M 2024-03-19T01:10:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:45 smithi086 ceph-mon[66829]: Unable to set osd_memory_target on smithi086 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-19T01:10:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:45 smithi086 ceph-mon[66829]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:45 smithi086 ceph-mon[66829]: mgrmap e26: x(active, since 4s), standbys: y, z 2024-03-19T01:10:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:45 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi163 to 174.1M 2024-03-19T01:10:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:45 smithi069 ceph-mon[31727]: Unable to set osd_memory_target on smithi163 to 182627737: error parsing value: Value '182627737' is below minimum 939524096 2024-03-19T01:10:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:45 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi086 to 174.1M 2024-03-19T01:10:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:45 smithi069 ceph-mon[31727]: Unable to set osd_memory_target on smithi086 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-19T01:10:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:45 smithi069 ceph-mon[31727]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:45 smithi069 ceph-mon[31727]: mgrmap e26: x(active, since 4s), standbys: y, z 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi069 to 176.7M 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: Unable to set osd_memory_target on smithi069 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-19T01:10:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:10:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:10:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-19T01:10:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-19T01:10:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: Updating smithi163:/etc/ceph/ceph.conf 2024-03-19T01:10:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:10:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:10:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:46 smithi086 ceph-mon[66829]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:46.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:46.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:46.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi069 to 176.7M 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: Unable to set osd_memory_target on smithi069 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-19T01:10:46.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: Updating smithi163:/etc/ceph/ceph.conf 2024-03-19T01:10:46.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:10:46.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:10:46.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:46 smithi069 ceph-mon[31727]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:46.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:46.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:10:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi069 to 176.7M 2024-03-19T01:10:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: Unable to set osd_memory_target on smithi069 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-19T01:10:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:10:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:10:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-19T01:10:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-19T01:10:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: Updating smithi163:/etc/ceph/ceph.conf 2024-03-19T01:10:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:10:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:10:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:46 smithi163 ceph-mon[67428]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:10:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:47 smithi086 ceph-mon[66829]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:10:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:47 smithi086 ceph-mon[66829]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:10:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:47 smithi086 ceph-mon[66829]: Updating smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:10:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:47 smithi086 ceph-mon[66829]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:10:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:47 smithi086 ceph-mon[66829]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:10:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:47 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:47 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:47 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:47 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:47 smithi086 ceph-mon[66829]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:10:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:47 smithi069 ceph-mon[31727]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:10:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:47 smithi069 ceph-mon[31727]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:10:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:47 smithi069 ceph-mon[31727]: Updating smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:10:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:47 smithi069 ceph-mon[31727]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:10:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:47 smithi069 ceph-mon[31727]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:10:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:47 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:47 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:47 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:47 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:47 smithi069 ceph-mon[31727]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:10:48.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:47 smithi163 ceph-mon[67428]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:10:48.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:47 smithi163 ceph-mon[67428]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:10:48.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:47 smithi163 ceph-mon[67428]: Updating smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:10:48.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:47 smithi163 ceph-mon[67428]: Updating smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:10:48.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:47 smithi163 ceph-mon[67428]: Updating smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:10:48.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:47 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:47 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:47 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:47 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:48.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:47 smithi163 ceph-mon[67428]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:10:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:48 smithi086 ceph-mon[66829]: Updating smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:10:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:48 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:48 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:48 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:48 smithi086 ceph-mon[66829]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-19T01:10:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:48 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:48 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:10:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:48 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:10:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:48 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:49.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:48 smithi069 ceph-mon[31727]: Updating smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:10:49.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:48 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:49.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:48 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:49.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:48 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:49.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:48 smithi069 ceph-mon[31727]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-19T01:10:49.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:48 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:49.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:48 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:10:49.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:48 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:10:49.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:48 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:49.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:48 smithi163 ceph-mon[67428]: Updating smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:10:49.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:48 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:49.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:48 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:49.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:48 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:49.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:48 smithi163 ceph-mon[67428]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-19T01:10:49.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:48 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:10:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:48 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:10:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:48 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:10:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:48 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:10:50.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:50 smithi086 ceph-mon[66829]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-19T01:10:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:50 smithi069 ceph-mon[31727]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-19T01:10:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:50 smithi163 ceph-mon[67428]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-19T01:10:52.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:52 smithi086 ceph-mon[66829]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-19T01:10:52.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:52 smithi069 ceph-mon[31727]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-19T01:10:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:52 smithi163 ceph-mon[67428]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-19T01:10:54.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:54 smithi086 ceph-mon[66829]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-19T01:10:54.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:54 smithi069 ceph-mon[31727]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-19T01:10:54.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:54 smithi163 ceph-mon[67428]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-19T01:10:56.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:56 smithi086 ceph-mon[66829]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-19T01:10:56.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:56 smithi069 ceph-mon[31727]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-19T01:10:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:56 smithi163 ceph-mon[67428]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-19T01:10:58.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:10:58 smithi086 ceph-mon[66829]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-19T01:10:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:10:58 smithi069 ceph-mon[31727]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-19T01:10:58.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:10:58 smithi163 ceph-mon[67428]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-19T01:10:59.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:10:59.222 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:10:59.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.468+0000 7fc36b400640 1 -- 172.21.15.69:0/934113097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc364104380 msgr2=0x7fc364110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.468+0000 7fc36b400640 1 --2- 172.21.15.69:0/934113097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc364104380 0x7fc364110890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009cd0 tx=0x7fc35c02c0e0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.468+0000 7fc36b400640 1 -- 172.21.15.69:0/934113097 shutdown_connections 2024-03-19T01:10:59.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.468+0000 7fc36b400640 1 --2- 172.21.15.69:0/934113097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc364104380 0x7fc364110890 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.468+0000 7fc36b400640 1 --2- 172.21.15.69:0/934113097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3641039c0 0x7fc364103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.468+0000 7fc36b400640 1 --2- 172.21.15.69:0/934113097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3641099d0 0x7fc364109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.468+0000 7fc36b400640 1 -- 172.21.15.69:0/934113097 >> 172.21.15.69:0/934113097 conn(0x7fc3640ff490 msgr2=0x7fc3641018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:59.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.468+0000 7fc36b400640 1 -- 172.21.15.69:0/934113097 shutdown_connections 2024-03-19T01:10:59.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.468+0000 7fc36b400640 1 -- 172.21.15.69:0/934113097 wait complete. 2024-03-19T01:10:59.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.469+0000 7fc36b400640 1 Processor -- start 2024-03-19T01:10:59.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.470+0000 7fc36b400640 1 -- start start 2024-03-19T01:10:59.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.470+0000 7fc36b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3641039c0 0x7fc364075d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.470+0000 7fc36b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc364104380 0x7fc364076270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.470+0000 7fc36b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3641099d0 0x7fc3640767b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.470+0000 7fc36b400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc364113de0 con 0x7fc364104380 2024-03-19T01:10:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.470+0000 7fc36b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc364113c60 con 0x7fc3641099d0 2024-03-19T01:10:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.470+0000 7fc36b400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc364113f60 con 0x7fc3641039c0 2024-03-19T01:10:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.470+0000 7fc363e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc364104380 0x7fc364076270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.470+0000 7fc363e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc364104380 0x7fc364076270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45130/0 (socket says 172.21.15.69:45130) 2024-03-19T01:10:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.470+0000 7fc369400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3641099d0 0x7fc3640767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.470+0000 7fc363e00640 1 -- 172.21.15.69:0/274248900 learned_addr learned my addr 172.21.15.69:0/274248900 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.471+0000 7fc368a00640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3641039c0 0x7fc364075d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.471+0000 7fc369400640 1 -- 172.21.15.69:0/274248900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3641039c0 msgr2=0x7fc364075d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.471+0000 7fc369400640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3641039c0 0x7fc364075d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.471+0000 7fc369400640 1 -- 172.21.15.69:0/274248900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc364104380 msgr2=0x7fc364076270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.471+0000 7fc369400640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc364104380 0x7fc364076270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.471+0000 7fc369400640 1 -- 172.21.15.69:0/274248900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36419a070 con 0x7fc3641099d0 2024-03-19T01:10:59.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.471+0000 7fc368a00640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3641039c0 0x7fc364075d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:59.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.471+0000 7fc363e00640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc364104380 0x7fc364076270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:59.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.472+0000 7fc369400640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3641099d0 0x7fc3640767b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc35c037040 tx=0x7fc35c032460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:59.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.472+0000 7fc361600640 1 -- 172.21.15.69:0/274248900 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c039040 con 0x7fc3641099d0 2024-03-19T01:10:59.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.472+0000 7fc361600640 1 -- 172.21.15.69:0/274248900 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35c044070 con 0x7fc3641099d0 2024-03-19T01:10:59.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.472+0000 7fc361600640 1 -- 172.21.15.69:0/274248900 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0042a0 con 0x7fc3641099d0 2024-03-19T01:10:59.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.472+0000 7fc36b400640 1 -- 172.21.15.69:0/274248900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36419a300 con 0x7fc3641099d0 2024-03-19T01:10:59.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.472+0000 7fc36b400640 1 -- 172.21.15.69:0/274248900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36419a770 con 0x7fc3641099d0 2024-03-19T01:10:59.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.477+0000 7fc36e16e740 1 Processor -- start 2024-03-19T01:10:59.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.477+0000 7fc361600640 1 -- 172.21.15.69:0/274248900 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 137466+0+0 (secure 0 0 0) 0x7fc35c032590 con 0x7fc3641099d0 2024-03-19T01:10:59.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.478+0000 7fc36e16e740 1 -- start start 2024-03-19T01:10:59.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.478+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf09120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.478+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf09660 0x55b08bf11d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.478+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf12240 0x55b08bf14630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.478+0000 7fc36e16e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b08be6d2c0 con 0x55b08bf09660 2024-03-19T01:10:59.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.478+0000 7fc36e16e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b08be15b60 con 0x55b08bf08d40 2024-03-19T01:10:59.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.478+0000 7fc36e16e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b08be69360 con 0x55b08bf12240 2024-03-19T01:10:59.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.479+0000 7fc363e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf09120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.479+0000 7fc368a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf09660 0x55b08bf11d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.479+0000 7fc363e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf09120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57318/0 (socket says 172.21.15.69:57318) 2024-03-19T01:10:59.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.479+0000 7fc363e00640 1 -- 172.21.15.69:0/454404315 learned_addr learned my addr 172.21.15.69:0/454404315 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:59.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.479+0000 7fc363e00640 1 --2- 172.21.15.69:0/454404315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf12240 0x55b08bf14630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.479+0000 7fc363e00640 1 -- 172.21.15.69:0/454404315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf12240 msgr2=0x55b08bf14630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:59.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.479+0000 7fc363e00640 1 --2- 172.21.15.69:0/454404315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf12240 0x55b08bf14630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.479+0000 7fc363e00640 1 -- 172.21.15.69:0/454404315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf09660 msgr2=0x55b08bf11d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.479+0000 7fc363e00640 1 --2- 172.21.15.69:0/454404315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf09660 0x55b08bf11d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.479+0000 7fc363e00640 1 -- 172.21.15.69:0/454404315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b08bc4b7d0 con 0x55b08bf08d40 2024-03-19T01:10:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.479+0000 7fc363e00640 1 --2- 172.21.15.69:0/454404315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf09120 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc350011cc0 tx=0x7fc350009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5952d61ab5831707 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.480+0000 7fc361600640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc34c0a2a00 0x7fc34c0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.480+0000 7fc355c00640 1 -- 172.21.15.69:0/454404315 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35000fc50 con 0x55b08bf08d40 2024-03-19T01:10:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.480+0000 7fc355c00640 1 -- 172.21.15.69:0/454404315 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35000fdf0 con 0x55b08bf08d40 2024-03-19T01:10:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.480+0000 7fc355c00640 1 -- 172.21.15.69:0/454404315 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35000e700 con 0x55b08bf08d40 2024-03-19T01:10:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.480+0000 7fc361600640 1 -- 172.21.15.69:0/274248900 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc35c0f1780 con 0x7fc3641099d0 2024-03-19T01:10:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.480+0000 7fc368a00640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc34c0a2a00 0x7fc34c0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.481+0000 7fc36e16e740 1 -- 172.21.15.69:0/454404315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 msgr2=0x55b08bf09120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.481+0000 7fc36e16e740 1 --2- 172.21.15.69:0/454404315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf09120 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc350011cc0 tx=0x7fc350009c80 comp rx=0 tx=0).stop 2024-03-19T01:10:59.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.481+0000 7fc36e16e740 1 -- 172.21.15.69:0/454404315 shutdown_connections 2024-03-19T01:10:59.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.481+0000 7fc36e16e740 1 --2- 172.21.15.69:0/454404315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf12240 0x55b08bf14630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.481+0000 7fc36e16e740 1 --2- 172.21.15.69:0/454404315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf09660 0x55b08bf11d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.481+0000 7fc36e16e740 1 --2- 172.21.15.69:0/454404315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf09120 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.481+0000 7fc36e16e740 1 -- 172.21.15.69:0/454404315 >> 172.21.15.69:0/454404315 conn(0x55b08bf03470 msgr2=0x55b08bf03850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.481+0000 7fc368a00640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc34c0a2a00 0x7fc34c0a4ec0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x55b08bf08810 tx=0x7fc358005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.481+0000 7fc36e16e740 1 -- 172.21.15.69:0/454404315 shutdown_connections 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.481+0000 7fc36e16e740 1 -- 172.21.15.69:0/454404315 wait complete. 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.482+0000 7fc36e16e740 1 -- start start 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.482+0000 7fc36e16e740 1 Processor -- start 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.482+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf08d40 0x55b08bf0f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf09660 0x55b08bf0f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 0x55b08bf0fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc36e16e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b08bf15030 con 0x55b08bf09660 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc36e16e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b08be28fc0 con 0x55b08bf12240 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc36e16e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b08bf151b0 con 0x55b08bf08d40 2024-03-19T01:10:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc368a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf08d40 0x55b08bf0f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc363e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf09660 0x55b08bf0f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc368a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf08d40 0x55b08bf0f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52242/0 (socket says 172.21.15.69:52242) 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc368a00640 1 -- 172.21.15.69:0/1199019255 learned_addr learned my addr 172.21.15.69:0/1199019255 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc36e16e740 10 client.0 _ll_register_callbacks cb 0x55b08be28990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc369400640 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 0x55b08bf0fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc368a00640 1 -- 172.21.15.69:0/1199019255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 msgr2=0x55b08bf0fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc368a00640 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 0x55b08bf0fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc368a00640 1 -- 172.21.15.69:0/1199019255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf09660 msgr2=0x55b08bf0f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc368a00640 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf09660 0x55b08bf0f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.483+0000 7fc368a00640 1 -- 172.21.15.69:0/1199019255 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b08bb813f0 con 0x55b08bf08d40 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.484+0000 7fc363e00640 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf09660 0x55b08bf0f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.484+0000 7fc36e16e740 1 -- 172.21.15.69:0/1199019255 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc35000fdf0 con 0x55b08bf08d40 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.484+0000 7fc369400640 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 0x55b08bf0fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.484+0000 7fc368a00640 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf08d40 0x55b08bf0f310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc35800fb90 tx=0x7fc3580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=437dd41c65eaefeb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.484+0000 7fc34a000640 10 client.25726 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:10:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.484+0000 7fc34a000640 1 -- 172.21.15.69:0/1199019255 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35802e060 con 0x55b08bf08d40 2024-03-19T01:10:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.484+0000 7fc34a000640 1 -- 172.21.15.69:0/1199019255 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc358018030 con 0x55b08bf08d40 2024-03-19T01:10:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.484+0000 7fc34a000640 1 -- 172.21.15.69:0/1199019255 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc35800fe60 con 0x55b08bf08d40 2024-03-19T01:10:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.485+0000 7fc36e16e740 10 client.25726 fetch_fsmap learned FSMap version 29 2024-03-19T01:10:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.485+0000 7fc36e16e740 1 -- 172.21.15.69:0/1199019255 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55b08bb410f0 con 0x55b08bf08d40 2024-03-19T01:10:59.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.485+0000 7fc34a000640 1 -- 172.21.15.69:0/1199019255 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc35801e8a0 con 0x55b08bf08d40 2024-03-19T01:10:59.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.485+0000 7fc36e16e740 10 client.25726 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.485+0000 7fc36e16e740 10 client.25726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.485+0000 7fc36e16e740 20 client.25726 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.485+0000 7fc36e16e740 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b08bf22500 0x55b08bf24910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.485+0000 7fc36e16e740 4 client.25726 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.485+0000 7fc36e16e740 1 -- 172.21.15.69:0/1199019255 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b08be0e610 con 0x55b08bf22500 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.486+0000 7fc363e00640 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b08bf22500 0x55b08bf24910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.486+0000 7fc363e00640 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b08bf22500 0x55b08bf24910 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4bf2c76b33a637 server_cookie=836f4cd672804bde in_seq=0 out_seq=0 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.486+0000 7fc34a000640 10 client.25726 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.487+0000 7fc34a000640 1 -- 172.21.15.69:0/1199019255 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b08be0e610 con 0x55b08bf22500 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.487+0000 7fc34a000640 10 client.25726 handle_command_reply: tid=0 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.487+0000 7fc36e16e740 1 client.25726 shutdown 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.487+0000 7fc36e16e740 1 -- 172.21.15.69:0/1199019255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf08d40 msgr2=0x55b08bf0f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.487+0000 7fc36e16e740 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf08d40 0x55b08bf0f310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc35800fb90 tx=0x7fc3580046c0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.487+0000 7fc36e16e740 1 -- 172.21.15.69:0/1199019255 shutdown_connections 2024-03-19T01:10:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.487+0000 7fc36e16e740 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b08bf22500 0x55b08bf24910 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.487+0000 7fc36e16e740 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 0x55b08bf0fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.487+0000 7fc36e16e740 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf09660 0x55b08bf0f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.487+0000 7fc36e16e740 1 --2- 172.21.15.69:0/1199019255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf08d40 0x55b08bf0f310 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.487+0000 7fc36e16e740 1 -- 172.21.15.69:0/1199019255 >> 172.21.15.69:0/1199019255 conn(0x55b08bf03470 msgr2=0x55b08bf051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.488+0000 7fc34a000640 0 client.25726 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.488+0000 7fc36e16e740 1 -- 172.21.15.69:0/1199019255 shutdown_connections 2024-03-19T01:10:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.488+0000 7fc36e16e740 1 -- 172.21.15.69:0/1199019255 wait complete. 2024-03-19T01:10:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.488+0000 7fc36e16e740 20 client.25726 trim_cache size 0 max 16384 2024-03-19T01:10:59.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.521+0000 7fc36e16e740 1 Processor -- start 2024-03-19T01:10:59.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.521+0000 7fc36e16e740 1 -- start start 2024-03-19T01:10:59.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.521+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf08d40 0x55b08bf11a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.521+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 0x55b08bf1de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.521+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 0x55b08bf0e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.521+0000 7fc36e16e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641bda50 con 0x55b08bf08d40 2024-03-19T01:10:59.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.521+0000 7fc36e16e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641bf590 con 0x55b08bf12240 2024-03-19T01:10:59.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.521+0000 7fc36e16e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641be120 con 0x55b08bf09660 2024-03-19T01:10:59.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc369400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 0x55b08bf0e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc369400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 0x55b08bf0e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57348/0 (socket says 172.21.15.69:57348) 2024-03-19T01:10:59.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc369400640 1 -- 172.21.15.69:0/2535459128 learned_addr learned my addr 172.21.15.69:0/2535459128 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:59.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc363e00640 1 --2- 172.21.15.69:0/2535459128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 0x55b08bf1de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc368a00640 1 --2- 172.21.15.69:0/2535459128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf08d40 0x55b08bf11a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc369400640 1 -- 172.21.15.69:0/2535459128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 msgr2=0x55b08bf1de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:59.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc369400640 1 --2- 172.21.15.69:0/2535459128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 0x55b08bf1de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc369400640 1 -- 172.21.15.69:0/2535459128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf08d40 msgr2=0x55b08bf11a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:59.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc369400640 1 --2- 172.21.15.69:0/2535459128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf08d40 0x55b08bf11a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc369400640 1 -- 172.21.15.69:0/2535459128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b08bef4950 con 0x55b08bf12240 2024-03-19T01:10:59.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc368a00640 1 --2- 172.21.15.69:0/2535459128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf08d40 0x55b08bf11a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:59.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.522+0000 7fc369400640 1 --2- 172.21.15.69:0/2535459128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 0x55b08bf0e470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0bd6f0 tx=0x7fc35c0093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94e7cad8a7cfd05d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:59.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.523+0000 7fc355200640 1 -- 172.21.15.69:0/2535459128 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0be0f0 con 0x55b08bf12240 2024-03-19T01:10:59.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.523+0000 7fc355200640 1 -- 172.21.15.69:0/2535459128 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35c0bd9f0 con 0x55b08bf12240 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.523+0000 7fc355200640 1 -- 172.21.15.69:0/2535459128 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0eeca0 con 0x55b08bf12240 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.523+0000 7fc36e16e740 1 -- 172.21.15.69:0/2535459128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 msgr2=0x55b08bf0e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.524+0000 7fc36e16e740 1 --2- 172.21.15.69:0/2535459128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 0x55b08bf0e470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0bd6f0 tx=0x7fc35c0093f0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.524+0000 7fc36e16e740 1 -- 172.21.15.69:0/2535459128 shutdown_connections 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.524+0000 7fc36e16e740 1 --2- 172.21.15.69:0/2535459128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf12240 0x55b08bf0e470 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.524+0000 7fc36e16e740 1 --2- 172.21.15.69:0/2535459128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 0x55b08bf1de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.524+0000 7fc36e16e740 1 --2- 172.21.15.69:0/2535459128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf08d40 0x55b08bf11a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.524+0000 7fc36e16e740 1 -- 172.21.15.69:0/2535459128 >> 172.21.15.69:0/2535459128 conn(0x55b08bf03470 msgr2=0x55b08bf01e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.524+0000 7fc36e16e740 1 -- 172.21.15.69:0/2535459128 shutdown_connections 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.524+0000 7fc36e16e740 1 -- 172.21.15.69:0/2535459128 wait complete. 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.524+0000 7fc36e16e740 1 -- start start 2024-03-19T01:10:59.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.525+0000 7fc36e16e740 1 Processor -- start 2024-03-19T01:10:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.525+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf1a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.525+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 0x55b08bf159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.525+0000 7fc36e16e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf12240 0x55b08bf15f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.525+0000 7fc36e16e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641beec0 con 0x55b08bf12240 2024-03-19T01:10:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.525+0000 7fc36e16e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641be7f0 con 0x55b08bf08d40 2024-03-19T01:10:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.525+0000 7fc36e16e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641bba30 con 0x55b08bf09660 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc36e16e740 10 client.0 _ll_register_callbacks cb 0x7fc35800fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc368a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc363e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 0x55b08bf159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc368a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf1a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57354/0 (socket says 172.21.15.69:57354) 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc363e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 0x55b08bf159f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52266/0 (socket says 172.21.15.69:52266) 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc368a00640 1 -- 172.21.15.69:0/2271956151 learned_addr learned my addr 172.21.15.69:0/2271956151 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc369400640 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf12240 0x55b08bf15f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc368a00640 1 -- 172.21.15.69:0/2271956151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 msgr2=0x55b08bf159f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc368a00640 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 0x55b08bf159f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc368a00640 1 -- 172.21.15.69:0/2271956151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf12240 msgr2=0x55b08bf15f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc368a00640 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf12240 0x55b08bf15f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc368a00640 1 -- 172.21.15.69:0/2271956151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b08bf16760 con 0x55b08bf08d40 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc369400640 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf12240 0x55b08bf15f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.526+0000 7fc363e00640 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 0x55b08bf159f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:10:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.527+0000 7fc368a00640 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf1a680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc358012fb0 tx=0x7fc35801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f06a6710f18346a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.527+0000 7fc34a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:10:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.527+0000 7fc34a000640 1 -- 172.21.15.69:0/2271956151 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35802e040 con 0x55b08bf08d40 2024-03-19T01:10:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.527+0000 7fc34a000640 1 -- 172.21.15.69:0/2271956151 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc358004b70 con 0x55b08bf08d40 2024-03-19T01:10:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.527+0000 7fc36e16e740 1 -- 172.21.15.69:0/2271956151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc35c0bd9f0 con 0x55b08bf08d40 2024-03-19T01:10:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.527+0000 7fc34a000640 1 -- 172.21.15.69:0/2271956151 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc358004d70 con 0x55b08bf08d40 2024-03-19T01:10:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.527+0000 7fc36e16e740 10 client.25730 fetch_fsmap learned FSMap version 29 2024-03-19T01:10:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.527+0000 7fc36e16e740 1 -- 172.21.15.69:0/2271956151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b08bf88850 con 0x55b08bf08d40 2024-03-19T01:10:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.528+0000 7fc34a000640 1 -- 172.21.15.69:0/2271956151 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35c0bd9f0 con 0x55b08bf08d40 2024-03-19T01:10:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.528+0000 7fc34a000640 1 -- 172.21.15.69:0/2271956151 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc3580035f0 con 0x55b08bf08d40 2024-03-19T01:10:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.528+0000 7fc36e16e740 10 client.25730 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:10:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.528+0000 7fc36e16e740 10 client.25730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:10:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.528+0000 7fc36e16e740 20 client.25730 populate_metadata read hostname 'smithi069' 2024-03-19T01:10:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.528+0000 7fc36e16e740 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b08bf88ff0 0x55b08bf8b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.528+0000 7fc36e16e740 4 client.25730 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.528+0000 7fc36e16e740 1 -- 172.21.15.69:0/2271956151 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b08be7a800 con 0x55b08bf88ff0 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.528+0000 7fc363e00640 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b08bf88ff0 0x55b08bf8b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.529+0000 7fc363e00640 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b08bf88ff0 0x55b08bf8b410 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e62a09840e942 server_cookie=8420b1066c0b90ce in_seq=0 out_seq=0 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.529+0000 7fc34a000640 10 client.25730 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.529+0000 7fc34a000640 1 -- 172.21.15.69:0/2271956151 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b08be7a800 con 0x55b08bf88ff0 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.529+0000 7fc34a000640 10 client.25730 handle_command_reply: tid=0 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.529+0000 7fc36e16e740 1 client.25730 shutdown 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.529+0000 7fc36e16e740 1 -- 172.21.15.69:0/2271956151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 msgr2=0x55b08bf1a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.529+0000 7fc36e16e740 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf1a680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc358012fb0 tx=0x7fc35801c000 comp rx=0 tx=0).stop 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.530+0000 7fc36e16e740 1 -- 172.21.15.69:0/2271956151 shutdown_connections 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.530+0000 7fc36e16e740 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b08bf88ff0 0x55b08bf8b410 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.530+0000 7fc36e16e740 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b08bf12240 0x55b08bf15f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.530+0000 7fc36e16e740 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b08bf09660 0x55b08bf159f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.530+0000 7fc36e16e740 1 --2- 172.21.15.69:0/2271956151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08bf08d40 0x55b08bf1a680 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.530+0000 7fc36e16e740 1 -- 172.21.15.69:0/2271956151 >> 172.21.15.69:0/2271956151 conn(0x55b08bf03470 msgr2=0x55b08bf026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:59.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.530+0000 7fc34a000640 0 client.25730 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:10:59.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.530+0000 7fc36e16e740 1 -- 172.21.15.69:0/2271956151 shutdown_connections 2024-03-19T01:10:59.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.530+0000 7fc36e16e740 1 -- 172.21.15.69:0/2271956151 wait complete. 2024-03-19T01:10:59.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.530+0000 7fc36e16e740 20 client.25730 trim_cache size 0 max 16384 2024-03-19T01:10:59.536 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:10:59.536 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:10:59.536 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:10:59.536 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:10:59.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 -- 172.21.15.69:0/274248900 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc34c0a2a00 msgr2=0x7fc34c0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc34c0a2a00 0x7fc34c0a4ec0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x55b08bf08810 tx=0x7fc358005f90 comp rx=0 tx=0).stop 2024-03-19T01:10:59.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 -- 172.21.15.69:0/274248900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3641099d0 msgr2=0x7fc3640767b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3641099d0 0x7fc3640767b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc35c037040 tx=0x7fc35c032460 comp rx=0 tx=0).stop 2024-03-19T01:10:59.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 -- 172.21.15.69:0/274248900 shutdown_connections 2024-03-19T01:10:59.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc34c0a2a00 0x7fc34c0a4ec0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3641099d0 0x7fc3640767b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc364104380 0x7fc364076270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 --2- 172.21.15.69:0/274248900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3641039c0 0x7fc364075d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 -- 172.21.15.69:0/274248900 >> 172.21.15.69:0/274248900 conn(0x7fc3640ff490 msgr2=0x7fc3640ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:59.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 -- 172.21.15.69:0/274248900 shutdown_connections 2024-03-19T01:10:59.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.531+0000 7fc36b400640 1 -- 172.21.15.69:0/274248900 wait complete. 2024-03-19T01:10:59.552 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:10:59.552 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:10:59.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.767+0000 7f78f6400640 1 -- 172.21.15.69:0/569859348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78f0068ec0 msgr2=0x7f78f0069320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.767+0000 7f78f6400640 1 --2- 172.21.15.69:0/569859348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78f0068ec0 0x7f78f0069320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009900 tx=0x7f78dc02bc30 comp rx=0 tx=0).stop 2024-03-19T01:10:59.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.768+0000 7f78f6400640 1 -- 172.21.15.69:0/569859348 shutdown_connections 2024-03-19T01:10:59.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.768+0000 7f78f6400640 1 --2- 172.21.15.69:0/569859348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f010ded0 0x7f78f01102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.768+0000 7f78f6400640 1 --2- 172.21.15.69:0/569859348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78f0068ec0 0x7f78f0069320 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.768+0000 7f78f6400640 1 --2- 172.21.15.69:0/569859348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f0102530 0x7f78f0068980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.768+0000 7f78f6400640 1 -- 172.21.15.69:0/569859348 >> 172.21.15.69:0/569859348 conn(0x7f78f0076130 msgr2=0x7f78f0076540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:10:59.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.768+0000 7f78f6400640 1 -- 172.21.15.69:0/569859348 shutdown_connections 2024-03-19T01:10:59.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.768+0000 7f78f6400640 1 -- 172.21.15.69:0/569859348 wait complete. 2024-03-19T01:10:59.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.769+0000 7f78f6400640 1 Processor -- start 2024-03-19T01:10:59.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.769+0000 7f78f6400640 1 -- start start 2024-03-19T01:10:59.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78f0068ec0 0x7f78f019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f6400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f0102530 0x7f78f019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f6400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f010ded0 0x7f78f019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f6400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0113c70 con 0x7f78f0102530 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f6400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0113af0 con 0x7f78f0068ec0 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f6400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0113df0 con 0x7f78f010ded0 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78f0068ec0 0x7f78f019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78f0068ec0 0x7f78f019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57366/0 (socket says 172.21.15.69:57366) 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f5000640 1 -- 172.21.15.69:0/3067728456 learned_addr learned my addr 172.21.15.69:0/3067728456 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78efe00640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f0102530 0x7f78f019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f5a00640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f010ded0 0x7f78f019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f5000640 1 -- 172.21.15.69:0/3067728456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f010ded0 msgr2=0x7f78f019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f5000640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f010ded0 0x7f78f019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f5000640 1 -- 172.21.15.69:0/3067728456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f0102530 msgr2=0x7f78f019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f5000640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f0102530 0x7f78f019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:10:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78f5000640 1 -- 172.21.15.69:0/3067728456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f019f0e0 con 0x7f78f0068ec0 2024-03-19T01:10:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.770+0000 7f78efe00640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f0102530 0x7f78f019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:10:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.771+0000 7f78f5000640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78f0068ec0 0x7f78f019a0f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f78e400d950 tx=0x7f78e400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.771+0000 7f78ed600640 1 -- 172.21.15.69:0/3067728456 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4017030 con 0x7f78f0068ec0 2024-03-19T01:10:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.771+0000 7f78f6400640 1 -- 172.21.15.69:0/3067728456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f019f3d0 con 0x7f78f0068ec0 2024-03-19T01:10:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.771+0000 7f78ed600640 1 -- 172.21.15.69:0/3067728456 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e4004020 con 0x7f78f0068ec0 2024-03-19T01:10:59.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.771+0000 7f78ed600640 1 -- 172.21.15.69:0/3067728456 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e40047a0 con 0x7f78f0068ec0 2024-03-19T01:10:59.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.771+0000 7f78f6400640 1 -- 172.21.15.69:0/3067728456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f01a6c60 con 0x7f78f0068ec0 2024-03-19T01:10:59.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.773+0000 7f78f6400640 1 -- 172.21.15.69:0/3067728456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f0068530 con 0x7f78f0068ec0 2024-03-19T01:10:59.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.774+0000 7f78ed600640 1 -- 172.21.15.69:0/3067728456 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 137466+0+0 (secure 0 0 0) 0x7f78e40041d0 con 0x7f78f0068ec0 2024-03-19T01:10:59.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.776+0000 7f78ed600640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f78d80a2a80 0x7f78d80a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:10:59.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.776+0000 7f78ed600640 1 -- 172.21.15.69:0/3067728456 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f78e40cedf0 con 0x7f78f0068ec0 2024-03-19T01:10:59.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.777+0000 7f78efe00640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f78d80a2a80 0x7f78d80a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:10:59.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.777+0000 7f78efe00640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f78d80a2a80 0x7f78d80a4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f78dc006010 tx=0x7f78dc02df90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:10:59.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:10:59.779+0000 7f78ed600640 1 -- 172.21.15.69:0/3067728456 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f78e409b5c0 con 0x7f78f0068ec0 2024-03-19T01:11:00.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.177+0000 7f78f6400640 1 -- 172.21.15.69:0/3067728456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78f01a6ef0 con 0x7f78f0068ec0 2024-03-19T01:11:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.178+0000 7f78ed600640 1 -- 172.21.15.69:0/3067728456 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11930 (secure 0 0 0) 0x7f78e409aec0 con 0x7f78f0068ec0 2024-03-19T01:11:00.179 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:11:00.179 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:10:17.373602+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24550,24494,24580,24581,24476]},"id":1}]} 2024-03-19T01:11:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 29 2024-03-19T01:11:00.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.181+0000 7f78f6400640 1 -- 172.21.15.69:0/3067728456 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f78d80a2a80 msgr2=0x7f78d80a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.181+0000 7f78f6400640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f78d80a2a80 0x7f78d80a4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f78dc006010 tx=0x7f78dc02df90 comp rx=0 tx=0).stop 2024-03-19T01:11:00.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.181+0000 7f78f6400640 1 -- 172.21.15.69:0/3067728456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78f0068ec0 msgr2=0x7f78f019a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.181+0000 7f78f6400640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78f0068ec0 0x7f78f019a0f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f78e400d950 tx=0x7f78e400de10 comp rx=0 tx=0).stop 2024-03-19T01:11:00.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.182+0000 7f78f6400640 1 -- 172.21.15.69:0/3067728456 shutdown_connections 2024-03-19T01:11:00.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.182+0000 7f78f6400640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78f010ded0 0x7f78f019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.182+0000 7f78f6400640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78f0102530 0x7f78f019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.182+0000 7f78f6400640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f78d80a2a80 0x7f78d80a4f40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.182+0000 7f78f6400640 1 --2- 172.21.15.69:0/3067728456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78f0068ec0 0x7f78f019a0f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.182+0000 7f78f6400640 1 -- 172.21.15.69:0/3067728456 >> 172.21.15.69:0/3067728456 conn(0x7f78f0076130 msgr2=0x7f78f00ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.182+0000 7f78f6400640 1 -- 172.21.15.69:0/3067728456 shutdown_connections 2024-03-19T01:11:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.182+0000 7f78f6400640 1 -- 172.21.15.69:0/3067728456 wait complete. 2024-03-19T01:11:00.203 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:11:00.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.442+0000 7f72c9600640 1 -- 172.21.15.69:0/1026851593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c4103170 msgr2=0x7f72c410b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.442+0000 7f72c9600640 1 --2- 172.21.15.69:0/1026851593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c4103170 0x7f72c410b650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009900 tx=0x7f72b002bcc0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.442+0000 7f72c9600640 1 -- 172.21.15.69:0/1026851593 shutdown_connections 2024-03-19T01:11:00.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.442+0000 7f72c9600640 1 --2- 172.21.15.69:0/1026851593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c410bc70 0x7f72c410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.442+0000 7f72c9600640 1 --2- 172.21.15.69:0/1026851593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c4103170 0x7f72c410b650 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.442+0000 7f72c9600640 1 --2- 172.21.15.69:0/1026851593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72c4102850 0x7f72c4102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.442+0000 7f72c9600640 1 -- 172.21.15.69:0/1026851593 >> 172.21.15.69:0/1026851593 conn(0x7f72c40fc070 msgr2=0x7f72c40fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.443+0000 7f72c9600640 1 -- 172.21.15.69:0/1026851593 shutdown_connections 2024-03-19T01:11:00.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.443+0000 7f72c9600640 1 -- 172.21.15.69:0/1026851593 wait complete. 2024-03-19T01:11:00.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.444+0000 7f72c9600640 1 Processor -- start 2024-03-19T01:11:00.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.444+0000 7f72c9600640 1 -- start start 2024-03-19T01:11:00.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c9600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c4102850 0x7f72c4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c9600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72c4103170 0x7f72c4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c410bc70 0x7f72c419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c9600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4111ac0 con 0x7f72c4103170 2024-03-19T01:11:00.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4111940 con 0x7f72c410bc70 2024-03-19T01:11:00.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c9600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4111c40 con 0x7f72c4102850 2024-03-19T01:11:00.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c4102850 0x7f72c4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c410bc70 0x7f72c419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c4102850 0x7f72c4197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52278/0 (socket says 172.21.15.69:52278) 2024-03-19T01:11:00.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c410bc70 0x7f72c419c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57382/0 (socket says 172.21.15.69:57382) 2024-03-19T01:11:00.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c3e00640 1 -- 172.21.15.69:0/2189666258 learned_addr learned my addr 172.21.15.69:0/2189666258 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:00.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.445+0000 7f72c3400640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72c4103170 0x7f72c4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c8c00640 1 -- 172.21.15.69:0/2189666258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c4102850 msgr2=0x7f72c4197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c8c00640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c4102850 0x7f72c4197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c8c00640 1 -- 172.21.15.69:0/2189666258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72c4103170 msgr2=0x7f72c4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c8c00640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72c4103170 0x7f72c4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c8c00640 1 -- 172.21.15.69:0/2189666258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c419cf80 con 0x7f72c410bc70 2024-03-19T01:11:00.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c3400640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72c4103170 0x7f72c4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:00.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c8c00640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c410bc70 0x7f72c419c850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00b540 tx=0x7f72bc00ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c0c00640 1 -- 172.21.15.69:0/2189666258 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc004b20 con 0x7f72c410bc70 2024-03-19T01:11:00.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c0c00640 1 -- 172.21.15.69:0/2189666258 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72bc004cc0 con 0x7f72c410bc70 2024-03-19T01:11:00.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c9600640 1 -- 172.21.15.69:0/2189666258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c419d270 con 0x7f72c410bc70 2024-03-19T01:11:00.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c0c00640 1 -- 172.21.15.69:0/2189666258 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc01dad0 con 0x7f72c410bc70 2024-03-19T01:11:00.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.446+0000 7f72c9600640 1 -- 172.21.15.69:0/2189666258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c41003d0 con 0x7f72c410bc70 2024-03-19T01:11:00.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.449+0000 7f72c0c00640 1 -- 172.21.15.69:0/2189666258 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 137466+0+0 (secure 0 0 0) 0x7f72bc024020 con 0x7f72c410bc70 2024-03-19T01:11:00.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.450+0000 7f72cda14740 1 Processor -- start 2024-03-19T01:11:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.450+0000 7f72cda14740 1 -- start start 2024-03-19T01:11:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.450+0000 7f72cda14740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 0x55b6aa494fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72cda14740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa49dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72cda14740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa49e0f0 0x55b6aa4a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72cda14740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b6aa3f9380 con 0x55b6aa49e0f0 2024-03-19T01:11:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72cda14740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6aa3a1d20 con 0x55b6aa494bf0 2024-03-19T01:11:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72cda14740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b6aa3f5420 con 0x55b6aa495510 2024-03-19T01:11:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 0x55b6aa494fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 0x55b6aa494fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57388/0 (socket says 172.21.15.69:57388) 2024-03-19T01:11:00.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72c3400640 1 -- 172.21.15.69:0/48702787 learned_addr learned my addr 172.21.15.69:0/48702787 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:00.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72c3400640 1 --2- 172.21.15.69:0/48702787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa49e0f0 0x55b6aa4a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72c3e00640 1 --2- 172.21.15.69:0/48702787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa49dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72c0c00640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72a80a2a80 0x7f72a80a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72c3400640 1 -- 172.21.15.69:0/48702787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 msgr2=0x55b6aa49dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72c3400640 1 --2- 172.21.15.69:0/48702787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa49dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72c3400640 1 -- 172.21.15.69:0/48702787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa49e0f0 msgr2=0x55b6aa4a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72c3400640 1 --2- 172.21.15.69:0/48702787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa49e0f0 0x55b6aa4a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.451+0000 7f72c3400640 1 -- 172.21.15.69:0/48702787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6aa10d3f0 con 0x55b6aa494bf0 2024-03-19T01:11:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.452+0000 7f72c0c00640 1 -- 172.21.15.69:0/2189666258 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f72bc0d7390 con 0x7f72c410bc70 2024-03-19T01:11:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.452+0000 7f72c3400640 1 --2- 172.21.15.69:0/48702787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 0x55b6aa494fd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f72b0005c40 tx=0x7f72b0005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae7a6b50144fb549 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.452+0000 7f72c8c00640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72a80a2a80 0x7f72a80a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.452+0000 7f72b9200640 1 -- 172.21.15.69:0/48702787 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00040d0 con 0x55b6aa494bf0 2024-03-19T01:11:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.452+0000 7f72b9200640 1 -- 172.21.15.69:0/48702787 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b002cdf0 con 0x55b6aa494bf0 2024-03-19T01:11:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.452+0000 7f72b9200640 1 -- 172.21.15.69:0/48702787 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00054d0 con 0x55b6aa494bf0 2024-03-19T01:11:00.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.453+0000 7f72cda14740 1 -- 172.21.15.69:0/48702787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 msgr2=0x55b6aa494fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.453+0000 7f72c8c00640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72a80a2a80 0x7f72a80a4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0d4660 tx=0x7f72bc004a40 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.453+0000 7f72cda14740 1 --2- 172.21.15.69:0/48702787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 0x55b6aa494fd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f72b0005c40 tx=0x7f72b0005b70 comp rx=0 tx=0).stop 2024-03-19T01:11:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.454+0000 7f72cda14740 1 -- 172.21.15.69:0/48702787 shutdown_connections 2024-03-19T01:11:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.454+0000 7f72cda14740 1 --2- 172.21.15.69:0/48702787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa49e0f0 0x55b6aa4a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.454+0000 7f72cda14740 1 --2- 172.21.15.69:0/48702787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa49dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.454+0000 7f72cda14740 1 --2- 172.21.15.69:0/48702787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 0x55b6aa494fd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.454+0000 7f72cda14740 1 -- 172.21.15.69:0/48702787 >> 172.21.15.69:0/48702787 conn(0x55b6aa48f320 msgr2=0x55b6aa48f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.454+0000 7f72cda14740 1 -- 172.21.15.69:0/48702787 shutdown_connections 2024-03-19T01:11:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.454+0000 7f72cda14740 1 -- 172.21.15.69:0/48702787 wait complete. 2024-03-19T01:11:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.454+0000 7f72cda14740 1 -- start start 2024-03-19T01:11:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.455+0000 7f72cda14740 1 Processor -- start 2024-03-19T01:11:00.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.455+0000 7f72cda14740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa494bf0 0x55b6aa49b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.455+0000 7f72cda14740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa49b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.455+0000 7f72cda14740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa49e0f0 0x55b6aa49bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.455+0000 7f72cda14740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b6aa4a0e80 con 0x55b6aa494bf0 2024-03-19T01:11:00.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.455+0000 7f72cda14740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6aa3b5160 con 0x55b6aa49e0f0 2024-03-19T01:11:00.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.455+0000 7f72cda14740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b6aa4a1000 con 0x55b6aa495510 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.455+0000 7f72cda14740 10 client.0 _ll_register_callbacks cb 0x55b6aa3f1170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.456+0000 7f72c3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa494bf0 0x55b6aa49b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.456+0000 7f72c3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa494bf0 0x55b6aa49b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45212/0 (socket says 172.21.15.69:45212) 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.456+0000 7f72c3400640 1 -- 172.21.15.69:0/2759311337 learned_addr learned my addr 172.21.15.69:0/2759311337 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.456+0000 7f72c3e00640 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa49b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.456+0000 7f72c3400640 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa49e0f0 0x55b6aa49bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.456+0000 7f72c3e00640 1 -- 172.21.15.69:0/2759311337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa49e0f0 msgr2=0x55b6aa49bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.456+0000 7f72c3e00640 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa49e0f0 0x55b6aa49bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.456+0000 7f72c3e00640 1 -- 172.21.15.69:0/2759311337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa494bf0 msgr2=0x55b6aa49b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.456+0000 7f72c3e00640 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa494bf0 0x55b6aa49b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.456+0000 7f72c3e00640 1 -- 172.21.15.69:0/2759311337 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6aa1d77d0 con 0x55b6aa495510 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.457+0000 7f72c3e00640 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa49b6d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f72b4011e70 tx=0x7f72b4009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdab99abc3409187 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.457+0000 7f72ad600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.457+0000 7f72ad600640 1 -- 172.21.15.69:0/2759311337 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b401d020 con 0x55b6aa495510 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.457+0000 7f72ad600640 1 -- 172.21.15.69:0/2759311337 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f72b4004460 con 0x55b6aa495510 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.457+0000 7f72cda14740 1 -- 172.21.15.69:0/2759311337 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b002cdf0 con 0x55b6aa495510 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.457+0000 7f72ad600640 1 -- 172.21.15.69:0/2759311337 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72b4005020 con 0x55b6aa495510 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.458+0000 7f72cda14740 10 client.25750 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.458+0000 7f72cda14740 1 -- 172.21.15.69:0/2759311337 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6aa4ad210 con 0x55b6aa495510 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.458+0000 7f72ad600640 1 -- 172.21.15.69:0/2759311337 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b002cdf0 con 0x55b6aa495510 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.458+0000 7f72ad600640 1 -- 172.21.15.69:0/2759311337 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f72b4003660 con 0x55b6aa495510 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.458+0000 7f72cda14740 10 client.25750 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.458+0000 7f72cda14740 10 client.25750 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.458+0000 7f72cda14740 20 client.25750 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.458+0000 7f72cda14740 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b6aa4ae6d0 0x55b6aa4b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.458+0000 7f72cda14740 4 client.25750 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:00.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.458+0000 7f72cda14740 1 -- 172.21.15.69:0/2759311337 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6aa4068c0 con 0x55b6aa4ae6d0 2024-03-19T01:11:00.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.459+0000 7f72c3400640 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b6aa4ae6d0 0x55b6aa4b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.459+0000 7f72c3400640 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b6aa4ae6d0 0x55b6aa4b0b00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3003fe7dec8f5dd server_cookie=2aeca1dd7f3bd1db in_seq=0 out_seq=0 2024-03-19T01:11:00.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.459+0000 7f72ad600640 10 client.25750 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:11:00.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.460+0000 7f72ad600640 1 -- 172.21.15.69:0/2759311337 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b6aa4068c0 con 0x55b6aa4ae6d0 2024-03-19T01:11:00.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72ad600640 10 client.25750 handle_command_reply: tid=0 2024-03-19T01:11:00.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72cda14740 1 client.25750 shutdown 2024-03-19T01:11:00.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72cda14740 1 -- 172.21.15.69:0/2759311337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 msgr2=0x55b6aa49b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72cda14740 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa49b6d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f72b4011e70 tx=0x7f72b4009900 comp rx=0 tx=0).stop 2024-03-19T01:11:00.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72cda14740 1 -- 172.21.15.69:0/2759311337 shutdown_connections 2024-03-19T01:11:00.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72cda14740 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b6aa4ae6d0 0x55b6aa4b0b00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72cda14740 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa49e0f0 0x55b6aa49bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72ad600640 0 client.25750 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:11:00.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72cda14740 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa49b6d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72cda14740 1 --2- 172.21.15.69:0/2759311337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa494bf0 0x55b6aa49b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72cda14740 1 -- 172.21.15.69:0/2759311337 >> 172.21.15.69:0/2759311337 conn(0x55b6aa48f320 msgr2=0x55b6aa491070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72cda14740 1 -- 172.21.15.69:0/2759311337 shutdown_connections 2024-03-19T01:11:00.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.461+0000 7f72cda14740 1 -- 172.21.15.69:0/2759311337 wait complete. 2024-03-19T01:11:00.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.462+0000 7f72cda14740 20 client.25750 trim_cache size 0 max 16384 2024-03-19T01:11:00.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.483+0000 7f72cda14740 1 Processor -- start 2024-03-19T01:11:00.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.483+0000 7f72cda14740 1 -- start start 2024-03-19T01:11:00.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.483+0000 7f72cda14740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 0x55b6aa49d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.483+0000 7f72cda14740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa4a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.483+0000 7f72cda14740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa49e0f0 0x55b6aa4b6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.483+0000 7f72cda14740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f72c41ac990 con 0x55b6aa49e0f0 2024-03-19T01:11:00.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.483+0000 7f72cda14740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72c41ae4d0 con 0x55b6aa494bf0 2024-03-19T01:11:00.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.483+0000 7f72cda14740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f72c41ad060 con 0x55b6aa495510 2024-03-19T01:11:00.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.484+0000 7f72c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa4a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.484+0000 7f72c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa4a9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52310/0 (socket says 172.21.15.69:52310) 2024-03-19T01:11:00.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.484+0000 7f72c3e00640 1 -- 172.21.15.69:0/3261242475 learned_addr learned my addr 172.21.15.69:0/3261242475 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:00.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.484+0000 7f72c3400640 1 --2- 172.21.15.69:0/3261242475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 0x55b6aa49d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.484+0000 7f72c3400640 1 --2- 172.21.15.69:0/3261242475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa49e0f0 0x55b6aa4b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.484+0000 7f72c3400640 1 -- 172.21.15.69:0/3261242475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 msgr2=0x55b6aa4a9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.484+0000 7f72c3400640 1 --2- 172.21.15.69:0/3261242475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa4a9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.484+0000 7f72c3400640 1 -- 172.21.15.69:0/3261242475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa49e0f0 msgr2=0x55b6aa4b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.484+0000 7f72c3400640 1 --2- 172.21.15.69:0/3261242475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa49e0f0 0x55b6aa4b6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.484+0000 7f72c3400640 1 -- 172.21.15.69:0/3261242475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6aa4b1180 con 0x55b6aa494bf0 2024-03-19T01:11:00.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.485+0000 7f72c3e00640 1 --2- 172.21.15.69:0/3261242475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa4a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:00.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.485+0000 7f72c3400640 1 --2- 172.21.15.69:0/3261242475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 0x55b6aa49d8c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55b6aa4b1380 tx=0x7f72b002bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13694e47ff7fce60 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.485+0000 7f72afe00640 1 -- 172.21.15.69:0/3261242475 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b002cc70 con 0x55b6aa494bf0 2024-03-19T01:11:00.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.485+0000 7f72afe00640 1 -- 172.21.15.69:0/3261242475 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b002ce10 con 0x55b6aa494bf0 2024-03-19T01:11:00.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.485+0000 7f72afe00640 1 -- 172.21.15.69:0/3261242475 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0003830 con 0x55b6aa494bf0 2024-03-19T01:11:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.486+0000 7f72cda14740 1 -- 172.21.15.69:0/3261242475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 msgr2=0x55b6aa49d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.486+0000 7f72cda14740 1 --2- 172.21.15.69:0/3261242475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 0x55b6aa49d8c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55b6aa4b1380 tx=0x7f72b002bcc0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.486+0000 7f72cda14740 1 -- 172.21.15.69:0/3261242475 shutdown_connections 2024-03-19T01:11:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.486+0000 7f72cda14740 1 --2- 172.21.15.69:0/3261242475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa49e0f0 0x55b6aa4b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.486+0000 7f72cda14740 1 --2- 172.21.15.69:0/3261242475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa4a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.486+0000 7f72cda14740 1 --2- 172.21.15.69:0/3261242475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa494bf0 0x55b6aa49d8c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.486+0000 7f72cda14740 1 -- 172.21.15.69:0/3261242475 >> 172.21.15.69:0/3261242475 conn(0x55b6aa48f320 msgr2=0x55b6aa49b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.486+0000 7f72cda14740 1 -- 172.21.15.69:0/3261242475 shutdown_connections 2024-03-19T01:11:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.486+0000 7f72cda14740 1 -- 172.21.15.69:0/3261242475 wait complete. 2024-03-19T01:11:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.487+0000 7f72cda14740 1 -- start start 2024-03-19T01:11:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.487+0000 7f72cda14740 1 Processor -- start 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.487+0000 7f72cda14740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa494bf0 0x55b6aa4980c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.487+0000 7f72cda14740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa498600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72cda14740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa49e0f0 0x55b6aa513a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72cda14740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f72c41ade00 con 0x55b6aa494bf0 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72cda14740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72c41ad730 con 0x55b6aa49e0f0 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72cda14740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f72c41aa970 con 0x55b6aa495510 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72cda14740 10 client.0 _ll_register_callbacks cb 0x7f72b4010510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa498600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72c3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa494bf0 0x55b6aa4980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa498600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52312/0 (socket says 172.21.15.69:52312) 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72c3e00640 1 -- 172.21.15.69:0/3794268716 learned_addr learned my addr 172.21.15.69:0/3794268716 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:00.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72c3400640 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa49e0f0 0x55b6aa513a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72c3e00640 1 -- 172.21.15.69:0/3794268716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa49e0f0 msgr2=0x55b6aa513a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72c3e00640 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa49e0f0 0x55b6aa513a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72c3e00640 1 -- 172.21.15.69:0/3794268716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa494bf0 msgr2=0x55b6aa4980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72c3e00640 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa494bf0 0x55b6aa4980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72c3e00640 1 -- 172.21.15.69:0/3794268716 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6aa498ed0 con 0x55b6aa495510 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.488+0000 7f72c3400640 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa49e0f0 0x55b6aa513a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.489+0000 7f72c3e00640 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa498600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f72b4003b20 tx=0x7f72b4005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce602ba4de2570e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.489+0000 7f72ad600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.489+0000 7f72ad600640 1 -- 172.21.15.69:0/3794268716 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b401d040 con 0x55b6aa495510 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.489+0000 7f72cda14740 1 -- 172.21.15.69:0/3794268716 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b002ce10 con 0x55b6aa495510 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.489+0000 7f72ad600640 1 -- 172.21.15.69:0/3794268716 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f72b40053c0 con 0x55b6aa495510 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.489+0000 7f72ad600640 1 -- 172.21.15.69:0/3794268716 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72b403d050 con 0x55b6aa495510 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.489+0000 7f72cda14740 10 client.25756 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:00.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.489+0000 7f72cda14740 1 -- 172.21.15.69:0/3794268716 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6aa5150b0 con 0x55b6aa495510 2024-03-19T01:11:00.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.490+0000 7f72ad600640 1 -- 172.21.15.69:0/3794268716 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b002ce10 con 0x55b6aa495510 2024-03-19T01:11:00.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.490+0000 7f72ad600640 1 -- 172.21.15.69:0/3794268716 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f72b400f5a0 con 0x55b6aa495510 2024-03-19T01:11:00.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.490+0000 7f72cda14740 10 client.25756 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:00.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.490+0000 7f72cda14740 10 client.25756 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:11:00.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.490+0000 7f72cda14740 20 client.25756 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:00.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.490+0000 7f72cda14740 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b6aa5158b0 0x55b6aa517cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.490+0000 7f72cda14740 4 client.25756 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:11:00.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.490+0000 7f72cda14740 1 -- 172.21.15.69:0/3794268716 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6aa400c10 con 0x55b6aa5158b0 2024-03-19T01:11:00.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.490+0000 7f72c3400640 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b6aa5158b0 0x55b6aa517cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.491+0000 7f72c3400640 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b6aa5158b0 0x55b6aa517cd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dee1f8da0fad33c6 server_cookie=25d345468b85c3fa in_seq=0 out_seq=0 2024-03-19T01:11:00.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.491+0000 7f72ad600640 10 client.25756 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:11:00.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.491+0000 7f72ad600640 1 -- 172.21.15.69:0/3794268716 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6aa400c10 con 0x55b6aa5158b0 2024-03-19T01:11:00.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.492+0000 7f72ad600640 10 client.25756 handle_command_reply: tid=0 2024-03-19T01:11:00.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.492+0000 7f72cda14740 1 client.25756 shutdown 2024-03-19T01:11:00.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72cda14740 1 -- 172.21.15.69:0/3794268716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 msgr2=0x55b6aa498600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72cda14740 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa498600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f72b4003b20 tx=0x7f72b4005d30 comp rx=0 tx=0).stop 2024-03-19T01:11:00.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72cda14740 1 -- 172.21.15.69:0/3794268716 shutdown_connections 2024-03-19T01:11:00.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72cda14740 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b6aa5158b0 0x55b6aa517cd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72cda14740 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6aa49e0f0 0x55b6aa513a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72ad600640 0 client.25756 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:11:00.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72cda14740 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6aa495510 0x55b6aa498600 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72cda14740 1 --2- 172.21.15.69:0/3794268716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b6aa494bf0 0x55b6aa4980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72cda14740 1 -- 172.21.15.69:0/3794268716 >> 172.21.15.69:0/3794268716 conn(0x55b6aa48f320 msgr2=0x55b6aa499660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72cda14740 1 -- 172.21.15.69:0/3794268716 shutdown_connections 2024-03-19T01:11:00.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72cda14740 1 -- 172.21.15.69:0/3794268716 wait complete. 2024-03-19T01:11:00.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.493+0000 7f72cda14740 20 client.25756 trim_cache size 0 max 16384 2024-03-19T01:11:00.496 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:11:00.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.494+0000 7f72c9600640 1 -- 172.21.15.69:0/2189666258 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72a80a2a80 msgr2=0x7f72a80a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.494+0000 7f72c9600640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72a80a2a80 0x7f72a80a4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0d4660 tx=0x7f72bc004a40 comp rx=0 tx=0).stop 2024-03-19T01:11:00.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.494+0000 7f72c9600640 1 -- 172.21.15.69:0/2189666258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c410bc70 msgr2=0x7f72c419c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.494+0000 7f72c9600640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c410bc70 0x7f72c419c850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00b540 tx=0x7f72bc00ba00 comp rx=0 tx=0).stop 2024-03-19T01:11:00.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.494+0000 7f72c9600640 1 -- 172.21.15.69:0/2189666258 shutdown_connections 2024-03-19T01:11:00.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.494+0000 7f72c9600640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72a80a2a80 0x7f72a80a4f40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.494+0000 7f72c9600640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72c410bc70 0x7f72c419c850 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.494+0000 7f72c9600640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72c4103170 0x7f72c4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.494+0000 7f72c9600640 1 --2- 172.21.15.69:0/2189666258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c4102850 0x7f72c4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.494+0000 7f72c9600640 1 -- 172.21.15.69:0/2189666258 >> 172.21.15.69:0/2189666258 conn(0x7f72c40fc070 msgr2=0x7f72c40fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.495+0000 7f72c9600640 1 -- 172.21.15.69:0/2189666258 shutdown_connections 2024-03-19T01:11:00.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.495+0000 7f72c9600640 1 -- 172.21.15.69:0/2189666258 wait complete. 2024-03-19T01:11:00.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:00 smithi086 ceph-mon[66829]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-19T01:11:00.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:00 smithi069 ceph-mon[31727]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-19T01:11:00.514 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:11:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:00 smithi163 ceph-mon[67428]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-19T01:11:00.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.733+0000 7f7fa7e00640 1 -- 172.21.15.69:0/1241172032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa81098b0 msgr2=0x7f7fa8109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.733+0000 7f7fa7e00640 1 --2- 172.21.15.69:0/1241172032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa81098b0 0x7f7fa8109c90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009900 tx=0x7f7f9802bc30 comp rx=0 tx=0).stop 2024-03-19T01:11:00.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.733+0000 7f7fa7e00640 1 -- 172.21.15.69:0/1241172032 shutdown_connections 2024-03-19T01:11:00.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.733+0000 7f7fa7e00640 1 --2- 172.21.15.69:0/1241172032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fa8104260 0x7f7fa8110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.733+0000 7f7fa7e00640 1 --2- 172.21.15.69:0/1241172032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa81038a0 0x7f7fa8103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.733+0000 7f7fa7e00640 1 --2- 172.21.15.69:0/1241172032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa81098b0 0x7f7fa8109c90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.733+0000 7f7fa7e00640 1 -- 172.21.15.69:0/1241172032 >> 172.21.15.69:0/1241172032 conn(0x7f7fa80ff330 msgr2=0x7f7fa8101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.734+0000 7f7fa7e00640 1 -- 172.21.15.69:0/1241172032 shutdown_connections 2024-03-19T01:11:00.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.734+0000 7f7fa7e00640 1 -- 172.21.15.69:0/1241172032 wait complete. 2024-03-19T01:11:00.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.735+0000 7f7fa7e00640 1 Processor -- start 2024-03-19T01:11:00.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.735+0000 7f7fa7e00640 1 -- start start 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.735+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fa81038a0 0x7f7fa819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.735+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa8104260 0x7f7fa819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.735+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa81098b0 0x7f7fa819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.735+0000 7f7fa7e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa8113d10 con 0x7f7fa81038a0 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.735+0000 7f7fa7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa8113b90 con 0x7f7fa81098b0 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.735+0000 7f7fa7e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa8113e90 con 0x7f7fa8104260 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.736+0000 7f7fa6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fa81038a0 0x7f7fa819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.736+0000 7f7fa7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa81098b0 0x7f7fa819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.736+0000 7f7fa6000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa8104260 0x7f7fa819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.736+0000 7f7fa7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa81098b0 0x7f7fa819ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57440/0 (socket says 172.21.15.69:57440) 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.736+0000 7f7fa7400640 1 -- 172.21.15.69:0/3350825222 learned_addr learned my addr 172.21.15.69:0/3350825222 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.736+0000 7f7fa6000640 1 -- 172.21.15.69:0/3350825222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa81098b0 msgr2=0x7f7fa819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.736+0000 7f7fa6000640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa81098b0 0x7f7fa819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.736+0000 7f7fa6000640 1 -- 172.21.15.69:0/3350825222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fa81038a0 msgr2=0x7f7fa819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.736+0000 7f7fa6000640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fa81038a0 0x7f7fa819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.736+0000 7f7fa6000640 1 -- 172.21.15.69:0/3350825222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa819f180 con 0x7f7fa8104260 2024-03-19T01:11:00.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.737+0000 7f7fa6000640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa8104260 0x7f7fa819a6d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400e9f0 tx=0x7f7f9400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.737+0000 7f7f9f400640 1 -- 172.21.15.69:0/3350825222 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9400ccc0 con 0x7f7fa8104260 2024-03-19T01:11:00.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.737+0000 7f7f9f400640 1 -- 172.21.15.69:0/3350825222 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f94005020 con 0x7f7fa8104260 2024-03-19T01:11:00.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.737+0000 7f7f9f400640 1 -- 172.21.15.69:0/3350825222 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f940049e0 con 0x7f7fa8104260 2024-03-19T01:11:00.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.737+0000 7f7fa7e00640 1 -- 172.21.15.69:0/3350825222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa819f470 con 0x7f7fa8104260 2024-03-19T01:11:00.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.737+0000 7f7fa7e00640 1 -- 172.21.15.69:0/3350825222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa81a6d00 con 0x7f7fa8104260 2024-03-19T01:11:00.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.742+0000 7f7fb07e3740 1 Processor -- start 2024-03-19T01:11:00.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.742+0000 7f7fb07e3740 1 -- start start 2024-03-19T01:11:00.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d70fa00 0x56164d70fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d7103b0 0x56164d7213c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 0x56164d723cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56164d674380 con 0x56164d721900 2024-03-19T01:11:00.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56164d61cd20 con 0x56164d70fa00 2024-03-19T01:11:00.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56164d670420 con 0x56164d7103b0 2024-03-19T01:11:00.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fa6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d7103b0 0x56164d7213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fa6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d7103b0 0x56164d7213c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52378/0 (socket says 172.21.15.69:52378) 2024-03-19T01:11:00.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fa6a00640 1 -- 172.21.15.69:0/2516289429 learned_addr learned my addr 172.21.15.69:0/2516289429 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:00.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fa6a00640 1 -- 172.21.15.69:0/2516289429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d70fa00 msgr2=0x56164d70fde0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:11:00.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fa6a00640 1 --2- 172.21.15.69:0/2516289429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d70fa00 0x56164d70fde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fa6a00640 1 -- 172.21.15.69:0/2516289429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 msgr2=0x56164d723cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:11:00.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fa6a00640 1 --2- 172.21.15.69:0/2516289429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 0x56164d723cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7fa6a00640 1 -- 172.21.15.69:0/2516289429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164d3883f0 con 0x56164d7103b0 2024-03-19T01:11:00.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.743+0000 7f7f9f400640 1 -- 172.21.15.69:0/3350825222 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137466+0+0 (secure 0 0 0) 0x7f7f94005560 con 0x7f7fa8104260 2024-03-19T01:11:00.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.744+0000 7f7fa6a00640 1 --2- 172.21.15.69:0/2516289429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d7103b0 0x56164d7213c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009900 tx=0x7f7f980095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d94ed3502914c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.744+0000 7f7f93e00640 1 -- 172.21.15.69:0/2516289429 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98004240 con 0x56164d7103b0 2024-03-19T01:11:00.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.744+0000 7f7f93e00640 1 -- 172.21.15.69:0/2516289429 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f980043e0 con 0x56164d7103b0 2024-03-19T01:11:00.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.744+0000 7f7f93e00640 1 -- 172.21.15.69:0/2516289429 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f980050f0 con 0x56164d7103b0 2024-03-19T01:11:00.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.745+0000 7f7f9f400640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f8c0a2ac0 0x7f7f8c0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.746+0000 7f7f9f400640 1 -- 172.21.15.69:0/3350825222 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f7f940cf370 con 0x7f7fa8104260 2024-03-19T01:11:00.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.747+0000 7f7fa7400640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f8c0a2ac0 0x7f7f8c0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.747+0000 7f7fb07e3740 1 -- 172.21.15.69:0/2516289429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d7103b0 msgr2=0x56164d7213c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.747+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/2516289429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d7103b0 0x56164d7213c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009900 tx=0x7f7f980095f0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.748+0000 7f7fb07e3740 1 -- 172.21.15.69:0/2516289429 shutdown_connections 2024-03-19T01:11:00.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.748+0000 7f7fa7400640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f8c0a2ac0 0x7f7f8c0a4f80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00049d0 tx=0x7f7fa0009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.748+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/2516289429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 0x56164d723cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.748+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/2516289429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d7103b0 0x56164d7213c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.748+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/2516289429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d70fa00 0x56164d70fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.748+0000 7f7fb07e3740 1 -- 172.21.15.69:0/2516289429 >> 172.21.15.69:0/2516289429 conn(0x56164d70a320 msgr2=0x56164d70a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.748+0000 7f7fb07e3740 1 -- 172.21.15.69:0/2516289429 shutdown_connections 2024-03-19T01:11:00.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.749+0000 7f7fb07e3740 1 -- 172.21.15.69:0/2516289429 wait complete. 2024-03-19T01:11:00.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.749+0000 7f7fb07e3740 1 -- start start 2024-03-19T01:11:00.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.750+0000 7f7fb07e3740 1 Processor -- start 2024-03-19T01:11:00.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.750+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d71b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.750+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 0x56164d721310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.750+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 0x56164d71bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.750+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56164d710ef0 con 0x56164d721900 2024-03-19T01:11:00.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.750+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56164d630160 con 0x56164d7103b0 2024-03-19T01:11:00.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.750+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56164d711070 con 0x56164d70fa00 2024-03-19T01:11:00.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.750+0000 7f7fb07e3740 10 client.0 _ll_register_callbacks cb 0x56164d66c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:00.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.750+0000 7f7fa6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 0x56164d721310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.750+0000 7f7fa6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 0x56164d721310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57462/0 (socket says 172.21.15.69:57462) 2024-03-19T01:11:00.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.750+0000 7f7fa6000640 1 -- 172.21.15.69:0/3009529440 learned_addr learned my addr 172.21.15.69:0/3009529440 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:00.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.751+0000 7f7fa6000640 1 -- 172.21.15.69:0/3009529440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 msgr2=0x56164d71b5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:11:00.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.751+0000 7f7fa6000640 1 --2- 172.21.15.69:0/3009529440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d71b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.751+0000 7f7fa6000640 1 -- 172.21.15.69:0/3009529440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 msgr2=0x56164d71bc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:11:00.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.751+0000 7f7fa6000640 1 --2- 172.21.15.69:0/3009529440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 0x56164d71bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.751+0000 7f7fa6000640 1 -- 172.21.15.69:0/3009529440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164d4527d0 con 0x56164d7103b0 2024-03-19T01:11:00.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.752+0000 7f7fa6000640 1 --2- 172.21.15.69:0/3009529440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 0x56164d721310 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400c6a0 tx=0x7f7f9409ae60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83fb6905cd6c7f03 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.752+0000 7f7f90c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:00.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.752+0000 7f7f90c00640 1 -- 172.21.15.69:0/3009529440 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9401c040 con 0x56164d7103b0 2024-03-19T01:11:00.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.752+0000 7f7f90c00640 1 -- 172.21.15.69:0/3009529440 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f7f9409f120 con 0x56164d7103b0 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.752+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3009529440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f980043e0 con 0x56164d7103b0 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.753+0000 7f7f90c00640 1 -- 172.21.15.69:0/3009529440 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7f940cca90 con 0x56164d7103b0 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.753+0000 7f7fb07e3740 10 client.25769 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.753+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3009529440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56164d7392d0 con 0x56164d7103b0 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.753+0000 7f7f90c00640 1 -- 172.21.15.69:0/3009529440 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f980043e0 con 0x56164d7103b0 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.753+0000 7f7f90c00640 1 -- 172.21.15.69:0/3009529440 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7f9409b6a0 con 0x56164d7103b0 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.753+0000 7f7fb07e3740 10 client.25769 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.753+0000 7f7fb07e3740 10 client.25769 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.753+0000 7f7fb07e3740 20 client.25769 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.753+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3009529440 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x56164d73a6e0 0x56164d73cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.753+0000 7f7fb07e3740 4 client.25769 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:00.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.753+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3009529440 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56164d6818c0 con 0x56164d73a6e0 2024-03-19T01:11:00.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.754+0000 7f7fa6a00640 1 --2- 172.21.15.69:0/3009529440 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x56164d73a6e0 0x56164d73cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.754+0000 7f7fa6a00640 1 --2- 172.21.15.69:0/3009529440 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x56164d73a6e0 0x56164d73cad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a3028681699198af server_cookie=367d371ad36579ff in_seq=0 out_seq=0 2024-03-19T01:11:00.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.754+0000 7f7f90c00640 10 client.25769 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:11:00.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.755+0000 7f7f90c00640 1 -- 172.21.15.69:0/3009529440 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56164d6818c0 con 0x56164d73a6e0 2024-03-19T01:11:00.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.755+0000 7f7f90c00640 10 client.25769 handle_command_reply: tid=0 2024-03-19T01:11:00.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7fb07e3740 1 client.25769 shutdown 2024-03-19T01:11:00.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3009529440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 msgr2=0x56164d721310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3009529440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 0x56164d721310 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400c6a0 tx=0x7f7f9409ae60 comp rx=0 tx=0).stop 2024-03-19T01:11:00.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3009529440 shutdown_connections 2024-03-19T01:11:00.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3009529440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 0x56164d71bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3009529440 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x56164d73a6e0 0x56164d73cad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3009529440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 0x56164d721310 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3009529440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d71b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3009529440 >> 172.21.15.69:0/3009529440 conn(0x56164d70a320 msgr2=0x56164d70c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7f90c00640 0 client.25769 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:11:00.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3009529440 shutdown_connections 2024-03-19T01:11:00.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.756+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3009529440 wait complete. 2024-03-19T01:11:00.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.757+0000 7f7fb07e3740 20 client.25769 trim_cache size 0 max 16384 2024-03-19T01:11:00.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.776+0000 7f7fb07e3740 1 Processor -- start 2024-03-19T01:11:00.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.776+0000 7f7fb07e3740 1 -- start start 2024-03-19T01:11:00.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.776+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d721280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.776+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d7103b0 0x56164d7423a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.777+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d721900 0x56164d71b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.777+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81ac910 con 0x56164d7103b0 2024-03-19T01:11:00.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.777+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81ae450 con 0x56164d721900 2024-03-19T01:11:00.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.777+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81acfe0 con 0x56164d70fa00 2024-03-19T01:11:00.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.777+0000 7f7fa6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d721280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.777+0000 7f7fa6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d721280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52406/0 (socket says 172.21.15.69:52406) 2024-03-19T01:11:00.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.777+0000 7f7fa6a00640 1 -- 172.21.15.69:0/3555003016 learned_addr learned my addr 172.21.15.69:0/3555003016 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:00.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.777+0000 7f7fa7400640 1 --2- 172.21.15.69:0/3555003016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d721900 0x56164d71b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.778+0000 7f7fa6000640 1 --2- 172.21.15.69:0/3555003016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d7103b0 0x56164d7423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.778+0000 7f7fa7400640 1 -- 172.21.15.69:0/3555003016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 msgr2=0x56164d721280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.778+0000 7f7fa7400640 1 --2- 172.21.15.69:0/3555003016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d721280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.778+0000 7f7fa7400640 1 -- 172.21.15.69:0/3555003016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d7103b0 msgr2=0x56164d7423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.778+0000 7f7fa7400640 1 --2- 172.21.15.69:0/3555003016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d7103b0 0x56164d7423a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.778+0000 7f7fa7400640 1 -- 172.21.15.69:0/3555003016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164d70f830 con 0x56164d721900 2024-03-19T01:11:00.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.779+0000 7f7fa7400640 1 --2- 172.21.15.69:0/3555003016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d721900 0x56164d71b130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009270 tx=0x7f7fa0013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f131fb0e4e49ed server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.779+0000 7f7f93400640 1 -- 172.21.15.69:0/3555003016 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa001c050 con 0x56164d721900 2024-03-19T01:11:00.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.779+0000 7f7f93400640 1 -- 172.21.15.69:0/3555003016 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa001c1f0 con 0x56164d721900 2024-03-19T01:11:00.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.779+0000 7f7f93400640 1 -- 172.21.15.69:0/3555003016 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0005280 con 0x56164d721900 2024-03-19T01:11:00.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.780+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3555003016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d721900 msgr2=0x56164d71b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.780+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3555003016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d721900 0x56164d71b130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009270 tx=0x7f7fa0013040 comp rx=0 tx=0).stop 2024-03-19T01:11:00.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.780+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3555003016 shutdown_connections 2024-03-19T01:11:00.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.780+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3555003016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d721900 0x56164d71b130 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.780+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3555003016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d7103b0 0x56164d7423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.780+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3555003016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d721280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.780+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3555003016 >> 172.21.15.69:0/3555003016 conn(0x56164d70a320 msgr2=0x56164d711d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:11:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:11:00.742+0000 7f60e5200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:11:00.783 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:11:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:11:00.742+0000 7f60e5200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:11:00.783 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:11:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:11:00.742+0000 7f60e5200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:11:00.783 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:11:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:11:00.742+0000 7f60e5200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:11:00.783 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:11:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:11:00.742+0000 7f60e5200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:11:00.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.781+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3555003016 shutdown_connections 2024-03-19T01:11:00.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.781+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3555003016 wait complete. 2024-03-19T01:11:00.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.782+0000 7f7fb07e3740 1 -- start start 2024-03-19T01:11:00.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.782+0000 7f7fb07e3740 1 Processor -- start 2024-03-19T01:11:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.783+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d7996a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.783+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 0x56164d799be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.783+0000 7f7fb07e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 0x56164d749810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81add80 con 0x56164d721900 2024-03-19T01:11:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81ad6b0 con 0x56164d7103b0 2024-03-19T01:11:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fb07e3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa81aa8f0 con 0x56164d70fa00 2024-03-19T01:11:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fa6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 0x56164d799be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fa6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d7996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fa7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 0x56164d749810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fa6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 0x56164d799be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57494/0 (socket says 172.21.15.69:57494) 2024-03-19T01:11:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fa6000640 1 -- 172.21.15.69:0/3940076411 learned_addr learned my addr 172.21.15.69:0/3940076411 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:00.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fb07e3740 10 client.0 _ll_register_callbacks cb 0x7f7f940cc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fa6a00640 1 -- 172.21.15.69:0/3940076411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 msgr2=0x56164d799be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fa6a00640 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 0x56164d799be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fa6a00640 1 -- 172.21.15.69:0/3940076411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 msgr2=0x56164d749810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fa6a00640 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 0x56164d749810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fa6a00640 1 -- 172.21.15.69:0/3940076411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164d749f00 con 0x56164d70fa00 2024-03-19T01:11:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.784+0000 7f7fa7400640 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 0x56164d749810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.785+0000 7f7fa6a00640 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d7996a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009850 tx=0x7f7f9802c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2942e63f2965fde server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.785+0000 7f7f90c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.785+0000 7f7f90c00640 1 -- 172.21.15.69:0/3940076411 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98056060 con 0x56164d70fa00 2024-03-19T01:11:00.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.786+0000 7f7f90c00640 1 -- 172.21.15.69:0/3940076411 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f7f980503b0 con 0x56164d70fa00 2024-03-19T01:11:00.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.786+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3940076411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fa001c1f0 con 0x56164d70fa00 2024-03-19T01:11:00.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.786+0000 7f7f90c00640 1 -- 172.21.15.69:0/3940076411 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7f980505d0 con 0x56164d70fa00 2024-03-19T01:11:00.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.786+0000 7f7fb07e3740 10 client.25777 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:00.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.786+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3940076411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56164d7a0ea0 con 0x56164d70fa00 2024-03-19T01:11:00.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.787+0000 7f7f90c00640 1 -- 172.21.15.69:0/3940076411 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa001c1f0 con 0x56164d70fa00 2024-03-19T01:11:00.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.787+0000 7f7f90c00640 1 -- 172.21.15.69:0/3940076411 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7f98004f20 con 0x56164d70fa00 2024-03-19T01:11:00.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.787+0000 7f7fb07e3740 10 client.25777 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:00.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.787+0000 7f7fb07e3740 10 client.25777 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:11:00.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.788+0000 7f7fb07e3740 20 client.25777 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:00.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.788+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x56164d7a1430 0x56164d7a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:00.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.788+0000 7f7fb07e3740 4 client.25777 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:11:00.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.788+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3940076411 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56164d67bc10 con 0x56164d7a1430 2024-03-19T01:11:00.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.788+0000 7f7fa6000640 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x56164d7a1430 0x56164d7a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7fa6000640 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x56164d7a1430 0x56164d7a38d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2f7c96127ad1dbc server_cookie=1f5aa0a9f069d0f4 in_seq=0 out_seq=0 2024-03-19T01:11:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7f90c00640 10 client.25777 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:11:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7f90c00640 1 -- 172.21.15.69:0/3940076411 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56164d67bc10 con 0x56164d7a1430 2024-03-19T01:11:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7f90c00640 10 client.25777 handle_command_reply: tid=0 2024-03-19T01:11:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7fb07e3740 1 client.25777 shutdown 2024-03-19T01:11:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3940076411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 msgr2=0x56164d7996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d7996a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009850 tx=0x7f7f9802c790 comp rx=0 tx=0).stop 2024-03-19T01:11:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3940076411 shutdown_connections 2024-03-19T01:11:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56164d721900 0x56164d749810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x56164d7a1430 0x56164d7a38d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56164d7103b0 0x56164d799be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7fb07e3740 1 --2- 172.21.15.69:0/3940076411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56164d70fa00 0x56164d7996a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7f90c00640 0 client.25777 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:11:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.789+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3940076411 >> 172.21.15.69:0/3940076411 conn(0x56164d70a320 msgr2=0x56164d708aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.790+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3940076411 shutdown_connections 2024-03-19T01:11:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.790+0000 7f7fb07e3740 1 -- 172.21.15.69:0/3940076411 wait complete. 2024-03-19T01:11:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.790+0000 7f7fb07e3740 20 client.25777 trim_cache size 0 max 16384 2024-03-19T01:11:00.793 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:11:00.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.791+0000 7f7fa7e00640 1 -- 172.21.15.69:0/3350825222 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f8c0a2ac0 msgr2=0x7f7f8c0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.791+0000 7f7fa7e00640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f8c0a2ac0 0x7f7f8c0a4f80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00049d0 tx=0x7f7fa0009500 comp rx=0 tx=0).stop 2024-03-19T01:11:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.791+0000 7f7fa7e00640 1 -- 172.21.15.69:0/3350825222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa8104260 msgr2=0x7f7fa819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.791+0000 7f7fa7e00640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa8104260 0x7f7fa819a6d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400e9f0 tx=0x7f7f9400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.792+0000 7f7fa7e00640 1 -- 172.21.15.69:0/3350825222 shutdown_connections 2024-03-19T01:11:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.792+0000 7f7fa7e00640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f8c0a2ac0 0x7f7f8c0a4f80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.792+0000 7f7fa7e00640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa81098b0 0x7f7fa819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.792+0000 7f7fa7e00640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa8104260 0x7f7fa819a6d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.792+0000 7f7fa7e00640 1 --2- 172.21.15.69:0/3350825222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fa81038a0 0x7f7fa819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.792+0000 7f7fa7e00640 1 -- 172.21.15.69:0/3350825222 >> 172.21.15.69:0/3350825222 conn(0x7f7fa80ff330 msgr2=0x7f7fa80ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.792+0000 7f7fa7e00640 1 -- 172.21.15.69:0/3350825222 shutdown_connections 2024-03-19T01:11:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:00.793+0000 7f7fa7e00640 1 -- 172.21.15.69:0/3350825222 wait complete. 2024-03-19T01:11:00.817 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:11:01.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.023+0000 7f29fd400640 1 -- 172.21.15.69:0/1519981110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f81098b0 msgr2=0x7f29f8109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.023+0000 7f29fd400640 1 --2- 172.21.15.69:0/1519981110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f81098b0 0x7f29f8109c90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009900 tx=0x7f29e802bc30 comp rx=0 tx=0).stop 2024-03-19T01:11:01.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.023+0000 7f29fd400640 1 -- 172.21.15.69:0/1519981110 shutdown_connections 2024-03-19T01:11:01.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.023+0000 7f29fd400640 1 --2- 172.21.15.69:0/1519981110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29f8104260 0x7f29f8110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.023+0000 7f29fd400640 1 --2- 172.21.15.69:0/1519981110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f81038a0 0x7f29f8103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.023+0000 7f29fd400640 1 --2- 172.21.15.69:0/1519981110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f81098b0 0x7f29f8109c90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.023+0000 7f29fd400640 1 -- 172.21.15.69:0/1519981110 >> 172.21.15.69:0/1519981110 conn(0x7f29f80ff350 msgr2=0x7f29f8101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.023+0000 7f29fd400640 1 -- 172.21.15.69:0/1519981110 shutdown_connections 2024-03-19T01:11:01.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.023+0000 7f29fd400640 1 -- 172.21.15.69:0/1519981110 wait complete. 2024-03-19T01:11:01.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.024+0000 7f29fd400640 1 Processor -- start 2024-03-19T01:11:01.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.024+0000 7f29fd400640 1 -- start start 2024-03-19T01:11:01.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.025+0000 7f29fd400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f81038a0 0x7f29f819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.025+0000 7f29fd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f8104260 0x7f29f819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.025+0000 7f29fd400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29f81098b0 0x7f29f819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.025+0000 7f29fd400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29f8113d10 con 0x7f29f81038a0 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.025+0000 7f29fd400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29f8113b90 con 0x7f29f8104260 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.025+0000 7f29fd400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f29f8113e90 con 0x7f29f81098b0 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.025+0000 7f29f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f8104260 0x7f29f819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.025+0000 7f29f7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f81038a0 0x7f29f819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.025+0000 7f29f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f8104260 0x7f29f819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57508/0 (socket says 172.21.15.69:57508) 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.025+0000 7f29f7400640 1 -- 172.21.15.69:0/3904731530 learned_addr learned my addr 172.21.15.69:0/3904731530 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.025+0000 7f29fca00640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29f81098b0 0x7f29f819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.026+0000 7f29f7400640 1 -- 172.21.15.69:0/3904731530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29f81098b0 msgr2=0x7f29f819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.026+0000 7f29f7400640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29f81098b0 0x7f29f819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.026+0000 7f29f7400640 1 -- 172.21.15.69:0/3904731530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f81038a0 msgr2=0x7f29f819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.026+0000 7f29f7400640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f81038a0 0x7f29f819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.026+0000 7f29f7400640 1 -- 172.21.15.69:0/3904731530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f819f180 con 0x7f29f8104260 2024-03-19T01:11:01.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.026+0000 7f29f7e00640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f81038a0 0x7f29f819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:01.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.026+0000 7f29f7400640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f8104260 0x7f29f819a6d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f29e400e9f0 tx=0x7f29e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.026+0000 7f29f4c00640 1 -- 172.21.15.69:0/3904731530 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e400cd80 con 0x7f29f8104260 2024-03-19T01:11:01.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.026+0000 7f29f4c00640 1 -- 172.21.15.69:0/3904731530 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e4010910 con 0x7f29f8104260 2024-03-19T01:11:01.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.026+0000 7f29f4c00640 1 -- 172.21.15.69:0/3904731530 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4018990 con 0x7f29f8104260 2024-03-19T01:11:01.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.027+0000 7f29fd400640 1 -- 172.21.15.69:0/3904731530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f819f470 con 0x7f29f8104260 2024-03-19T01:11:01.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.027+0000 7f29fd400640 1 -- 172.21.15.69:0/3904731530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f81a6d00 con 0x7f29f8104260 2024-03-19T01:11:01.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.031+0000 7f2a01816740 1 Processor -- start 2024-03-19T01:11:01.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.031+0000 7f2a01816740 1 -- start start 2024-03-19T01:11:01.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f29f4c00640 1 -- 172.21.15.69:0/3904731530 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 137466+0+0 (secure 0 0 0) 0x7f29e4010ac0 con 0x7f29f8104260 2024-03-19T01:11:01.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f2a01816740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d4bf0 0x564bd79d4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f2a01816740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 0x564bd79ddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f2a01816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79de0f0 0x564bd79e04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f2a01816740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564bd7939380 con 0x564bd79d5510 2024-03-19T01:11:01.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f2a01816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564bd78e1d20 con 0x564bd79de0f0 2024-03-19T01:11:01.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f2a01816740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564bd7935420 con 0x564bd79d4bf0 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f29fca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d4bf0 0x564bd79d4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f29f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79de0f0 0x564bd79e04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f29f7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 0x564bd79ddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f29f7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 0x564bd79ddbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45330/0 (socket says 172.21.15.69:45330) 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f29f7e00640 1 -- 172.21.15.69:0/2137974893 learned_addr learned my addr 172.21.15.69:0/2137974893 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.032+0000 7f29f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79de0f0 0x564bd79e04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57518/0 (socket says 172.21.15.69:57518) 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.033+0000 7f29f7e00640 1 -- 172.21.15.69:0/2137974893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d4bf0 msgr2=0x564bd79d4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.033+0000 7f29f7e00640 1 --2- 172.21.15.69:0/2137974893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d4bf0 0x564bd79d4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.033+0000 7f29f7e00640 1 -- 172.21.15.69:0/2137974893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79de0f0 msgr2=0x564bd79e04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.033+0000 7f29f7e00640 1 --2- 172.21.15.69:0/2137974893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79de0f0 0x564bd79e04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.033+0000 7f29f7e00640 1 -- 172.21.15.69:0/2137974893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bd764d3f0 con 0x564bd79d5510 2024-03-19T01:11:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.033+0000 7f29f7400640 1 --2- 172.21.15.69:0/2137974893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79de0f0 0x564bd79e04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:01.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.033+0000 7f29f7e00640 1 --2- 172.21.15.69:0/2137974893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 0x564bd79ddbb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009230 tx=0x7f29e8009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd865e242b0705b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.034+0000 7f29ed200640 1 -- 172.21.15.69:0/2137974893 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8042070 con 0x564bd79d5510 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.034+0000 7f29f4c00640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f29dc0a2a80 0x7f29dc0a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.034+0000 7f29ed200640 1 -- 172.21.15.69:0/2137974893 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e8004a70 con 0x564bd79d5510 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.034+0000 7f29ed200640 1 -- 172.21.15.69:0/2137974893 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8005250 con 0x564bd79d5510 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.034+0000 7f29f4c00640 1 -- 172.21.15.69:0/3904731530 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f29e40cf6c0 con 0x7f29f8104260 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.034+0000 7f29fca00640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f29dc0a2a80 0x7f29dc0a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.035+0000 7f2a01816740 1 -- 172.21.15.69:0/2137974893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 msgr2=0x564bd79ddbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.035+0000 7f2a01816740 1 --2- 172.21.15.69:0/2137974893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 0x564bd79ddbb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009230 tx=0x7f29e8009670 comp rx=0 tx=0).stop 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.035+0000 7f2a01816740 1 -- 172.21.15.69:0/2137974893 shutdown_connections 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.035+0000 7f2a01816740 1 --2- 172.21.15.69:0/2137974893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79de0f0 0x564bd79e04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.035+0000 7f2a01816740 1 --2- 172.21.15.69:0/2137974893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 0x564bd79ddbb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.035+0000 7f2a01816740 1 --2- 172.21.15.69:0/2137974893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d4bf0 0x564bd79d4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.035+0000 7f2a01816740 1 -- 172.21.15.69:0/2137974893 >> 172.21.15.69:0/2137974893 conn(0x564bd79cf320 msgr2=0x564bd79cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.035+0000 7f2a01816740 1 -- 172.21.15.69:0/2137974893 shutdown_connections 2024-03-19T01:11:01.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.035+0000 7f2a01816740 1 -- 172.21.15.69:0/2137974893 wait complete. 2024-03-19T01:11:01.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.036+0000 7f2a01816740 1 -- start start 2024-03-19T01:11:01.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.036+0000 7f2a01816740 1 Processor -- start 2024-03-19T01:11:01.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.036+0000 7f29fca00640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f29dc0a2a80 0x7f29dc0a4f40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29f0005fd0 tx=0x7f29f0009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f2a01816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 0x564bd79db1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f2a01816740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 0x564bd79db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f2a01816740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79de0f0 0x564bd79dbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f2a01816740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564bd79e0e80 con 0x564bd79d5510 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f2a01816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564bd78f5160 con 0x564bd79d4bf0 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f2a01816740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564bd79e1000 con 0x564bd79de0f0 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f2a01816740 10 client.0 _ll_register_callbacks cb 0x564bd7931170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f29f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 0x564bd79db1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f29f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 0x564bd79db1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57530/0 (socket says 172.21.15.69:57530) 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f29f7e00640 1 -- 172.21.15.69:0/1295664337 learned_addr learned my addr 172.21.15.69:0/1295664337 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f29f7400640 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 0x564bd79db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f29f7e00640 1 -- 172.21.15.69:0/1295664337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79de0f0 msgr2=0x564bd79dbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f29f7e00640 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79de0f0 0x564bd79dbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f29f7e00640 1 -- 172.21.15.69:0/1295664337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 msgr2=0x564bd79db6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f29f7e00640 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 0x564bd79db6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f29f7e00640 1 -- 172.21.15.69:0/1295664337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bd77177d0 con 0x564bd79d4bf0 2024-03-19T01:11:01.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f29f7400640 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 0x564bd79db6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:11:01.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.037+0000 7f29f7e00640 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 0x564bd79db1b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f29e802c170 tx=0x7f29e8004460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b1ee036a3a9525f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.038+0000 7f29e1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:01.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.038+0000 7f29e1600640 1 -- 172.21.15.69:0/1295664337 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8042070 con 0x564bd79d4bf0 2024-03-19T01:11:01.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.038+0000 7f2a01816740 1 -- 172.21.15.69:0/1295664337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e8004a70 con 0x564bd79d4bf0 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.038+0000 7f29e1600640 1 -- 172.21.15.69:0/1295664337 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f29e8004ff0 con 0x564bd79d4bf0 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.038+0000 7f29e1600640 1 -- 172.21.15.69:0/1295664337 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29e8059050 con 0x564bd79d4bf0 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.038+0000 7f2a01816740 10 client.25787 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.038+0000 7f2a01816740 1 -- 172.21.15.69:0/1295664337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bd79ed1b0 con 0x564bd79d4bf0 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.038+0000 7f29e1600640 1 -- 172.21.15.69:0/1295664337 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e8004a70 con 0x564bd79d4bf0 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.039+0000 7f29e1600640 1 -- 172.21.15.69:0/1295664337 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f29e8003b80 con 0x564bd79d4bf0 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.039+0000 7f2a01816740 10 client.25787 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.039+0000 7f2a01816740 10 client.25787 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.039+0000 7f2a01816740 20 client.25787 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.039+0000 7f2a01816740 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564bd79ee500 0x564bd79f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.039+0000 7f2a01816740 4 client.25787 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.039+0000 7f2a01816740 1 -- 172.21.15.69:0/1295664337 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bd79468c0 con 0x564bd79ee500 2024-03-19T01:11:01.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.039+0000 7f29f7400640 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564bd79ee500 0x564bd79f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.040+0000 7f29f7400640 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564bd79ee500 0x564bd79f08f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53394ff3e27c547f server_cookie=7d1aeb3770272d1d in_seq=0 out_seq=0 2024-03-19T01:11:01.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.040+0000 7f29e1600640 10 client.25787 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f29e1600640 1 -- 172.21.15.69:0/1295664337 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f29e40aa9c0 con 0x564bd79ee500 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f29e1600640 10 client.25787 handle_command_reply: tid=0 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 1 client.25787 shutdown 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 1 -- 172.21.15.69:0/1295664337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 msgr2=0x564bd79db1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 0x564bd79db1b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f29e802c170 tx=0x7f29e8004460 comp rx=0 tx=0).stop 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 1 -- 172.21.15.69:0/1295664337 shutdown_connections 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564bd79ee500 0x564bd79f08f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79de0f0 0x564bd79dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d5510 0x564bd79db6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 1 --2- 172.21.15.69:0/1295664337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 0x564bd79db1b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 1 -- 172.21.15.69:0/1295664337 >> 172.21.15.69:0/1295664337 conn(0x564bd79cf320 msgr2=0x564bd79d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f29e1600640 0 client.25787 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:11:01.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 1 -- 172.21.15.69:0/1295664337 shutdown_connections 2024-03-19T01:11:01.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 1 -- 172.21.15.69:0/1295664337 wait complete. 2024-03-19T01:11:01.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.041+0000 7f2a01816740 20 client.25787 trim_cache size 0 max 16384 2024-03-19T01:11:01.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.062+0000 7f2a01816740 1 Processor -- start 2024-03-19T01:11:01.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.062+0000 7f2a01816740 1 -- start start 2024-03-19T01:11:01.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.062+0000 7f2a01816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 0x564bd79dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.062+0000 7f2a01816740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d5510 0x564bd79daa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.062+0000 7f2a01816740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79de0f0 0x564bd79e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.062+0000 7f2a01816740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29f81b0d20 con 0x564bd79de0f0 2024-03-19T01:11:01.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.062+0000 7f2a01816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29f81b2860 con 0x564bd79d4bf0 2024-03-19T01:11:01.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.062+0000 7f2a01816740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f29f81b13f0 con 0x564bd79d5510 2024-03-19T01:11:01.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29f7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d5510 0x564bd79daa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29f7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d5510 0x564bd79daa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52454/0 (socket says 172.21.15.69:52454) 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29f7400640 1 -- 172.21.15.69:0/653454516 learned_addr learned my addr 172.21.15.69:0/653454516 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29fca00640 1 --2- 172.21.15.69:0/653454516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79de0f0 0x564bd79e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29f7e00640 1 --2- 172.21.15.69:0/653454516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 0x564bd79dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29f7400640 1 -- 172.21.15.69:0/653454516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 msgr2=0x564bd79dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29f7400640 1 --2- 172.21.15.69:0/653454516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 0x564bd79dd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29f7400640 1 -- 172.21.15.69:0/653454516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79de0f0 msgr2=0x564bd79e9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29f7400640 1 --2- 172.21.15.69:0/653454516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79de0f0 0x564bd79e9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29f7400640 1 -- 172.21.15.69:0/653454516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bd79d5340 con 0x564bd79d5510 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29f7400640 1 --2- 172.21.15.69:0/653454516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d5510 0x564bd79daa90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f29e409b540 tx=0x7f29e40d1b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cdaf29a0170a66b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29e3e00640 1 -- 172.21.15.69:0/653454516 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e409b840 con 0x564bd79d5510 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.063+0000 7f29e3e00640 1 -- 172.21.15.69:0/653454516 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564bd79468c0 con 0x564bd79d5510 2024-03-19T01:11:01.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.064+0000 7f29e3e00640 1 -- 172.21.15.69:0/653454516 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e40cf280 con 0x564bd79d5510 2024-03-19T01:11:01.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.064+0000 7f2a01816740 1 -- 172.21.15.69:0/653454516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d5510 msgr2=0x564bd79daa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.064+0000 7f2a01816740 1 --2- 172.21.15.69:0/653454516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d5510 0x564bd79daa90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f29e409b540 tx=0x7f29e40d1b10 comp rx=0 tx=0).stop 2024-03-19T01:11:01.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.064+0000 7f2a01816740 1 -- 172.21.15.69:0/653454516 shutdown_connections 2024-03-19T01:11:01.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.064+0000 7f2a01816740 1 --2- 172.21.15.69:0/653454516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79de0f0 0x564bd79e9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.064+0000 7f2a01816740 1 --2- 172.21.15.69:0/653454516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79d5510 0x564bd79daa90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.064+0000 7f2a01816740 1 --2- 172.21.15.69:0/653454516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d4bf0 0x564bd79dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.064+0000 7f2a01816740 1 -- 172.21.15.69:0/653454516 >> 172.21.15.69:0/653454516 conn(0x564bd79cf320 msgr2=0x564bd79d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.065+0000 7f2a01816740 1 -- 172.21.15.69:0/653454516 shutdown_connections 2024-03-19T01:11:01.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.065+0000 7f2a01816740 1 -- 172.21.15.69:0/653454516 wait complete. 2024-03-19T01:11:01.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.065+0000 7f2a01816740 1 -- start start 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.065+0000 7f2a01816740 1 Processor -- start 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f2a01816740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d4bf0 0x564bd79f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f2a01816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d5510 0x564bd79f0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f2a01816740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79de0f0 0x564bd79f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f2a01816740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29f81b2190 con 0x564bd79d4bf0 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f2a01816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29f81b1ac0 con 0x564bd79d5510 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f2a01816740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f29f81aed00 con 0x564bd79de0f0 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f2a01816740 10 client.0 _ll_register_callbacks cb 0x7f29e40cc7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f29f7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d4bf0 0x564bd79f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f29f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d5510 0x564bd79f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f29f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d5510 0x564bd79f0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57540/0 (socket says 172.21.15.69:57540) 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f29f7400640 1 -- 172.21.15.69:0/1172367759 learned_addr learned my addr 172.21.15.69:0/1172367759 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f29fca00640 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79de0f0 0x564bd79f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f29f7400640 1 -- 172.21.15.69:0/1172367759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79de0f0 msgr2=0x564bd79f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.066+0000 7f29f7400640 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79de0f0 0x564bd79f0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f29f7400640 1 -- 172.21.15.69:0/1172367759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d4bf0 msgr2=0x564bd79f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f29f7400640 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d4bf0 0x564bd79f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f29f7400640 1 -- 172.21.15.69:0/1172367759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bd79f1300 con 0x564bd79d5510 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f29f7e00640 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d4bf0 0x564bd79f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f29f7400640 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d5510 0x564bd79f0880 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f29e40d4010 tx=0x7f29e40d5480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b2674cc46e42f2e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f29e1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f29e1600640 1 -- 172.21.15.69:0/1172367759 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e409f8e0 con 0x564bd79d5510 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f29e1600640 1 -- 172.21.15.69:0/1172367759 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f29e4099b40 con 0x564bd79d5510 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f2a01816740 1 -- 172.21.15.69:0/1172367759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564bd79468c0 con 0x564bd79d5510 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f29e1600640 1 -- 172.21.15.69:0/1172367759 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29e40a3f30 con 0x564bd79d5510 2024-03-19T01:11:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f2a01816740 10 client.25793 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:01.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.067+0000 7f2a01816740 1 -- 172.21.15.69:0/1172367759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bd7a541b0 con 0x564bd79d5510 2024-03-19T01:11:01.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.068+0000 7f29e1600640 1 -- 172.21.15.69:0/1172367759 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e40a5030 con 0x564bd79d5510 2024-03-19T01:11:01.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.068+0000 7f29e1600640 1 -- 172.21.15.69:0/1172367759 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f29e40ccbe0 con 0x564bd79d5510 2024-03-19T01:11:01.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.068+0000 7f2a01816740 10 client.25793 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:01.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.068+0000 7f2a01816740 10 client.25793 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:11:01.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.068+0000 7f2a01816740 20 client.25793 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:01.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.068+0000 7f2a01816740 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564bd7a54820 0x564bd7a56c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.068+0000 7f2a01816740 4 client.25793 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:11:01.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.068+0000 7f2a01816740 1 -- 172.21.15.69:0/1172367759 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bd7940c10 con 0x564bd7a54820 2024-03-19T01:11:01.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.069+0000 7f29f7e00640 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564bd7a54820 0x564bd7a56c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.069+0000 7f29e1600640 10 client.25793 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.069+0000 7f29f7e00640 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564bd7a54820 0x564bd7a56c40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1059a44c98fcf609 server_cookie=e6da11fbd240a421 in_seq=0 out_seq=0 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f29e1600640 1 -- 172.21.15.69:0/1172367759 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bd7940c10 con 0x564bd7a54820 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f29e1600640 10 client.25793 handle_command_reply: tid=0 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 1 client.25793 shutdown 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 1 -- 172.21.15.69:0/1172367759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d5510 msgr2=0x564bd79f0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d5510 0x564bd79f0880 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f29e40d4010 tx=0x7f29e40d5480 comp rx=0 tx=0).stop 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 1 -- 172.21.15.69:0/1172367759 shutdown_connections 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564bd7a54820 0x564bd7a56c40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564bd79de0f0 0x564bd79f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bd79d5510 0x564bd79f0880 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 1 --2- 172.21.15.69:0/1172367759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bd79d4bf0 0x564bd79f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 1 -- 172.21.15.69:0/1172367759 >> 172.21.15.69:0/1172367759 conn(0x564bd79cf320 msgr2=0x564bd79d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f29e1600640 0 client.25793 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:11:01.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 1 -- 172.21.15.69:0/1172367759 shutdown_connections 2024-03-19T01:11:01.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 1 -- 172.21.15.69:0/1172367759 wait complete. 2024-03-19T01:11:01.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.070+0000 7f2a01816740 20 client.25793 trim_cache size 0 max 16384 2024-03-19T01:11:01.075 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.071+0000 7f29fd400640 1 -- 172.21.15.69:0/3904731530 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f29dc0a2a80 msgr2=0x7f29dc0a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.071+0000 7f29fd400640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f29dc0a2a80 0x7f29dc0a4f40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29f0005fd0 tx=0x7f29f0009500 comp rx=0 tx=0).stop 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.071+0000 7f29fd400640 1 -- 172.21.15.69:0/3904731530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f8104260 msgr2=0x7f29f819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.071+0000 7f29fd400640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f8104260 0x7f29f819a6d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f29e400e9f0 tx=0x7f29e400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.071+0000 7f29fd400640 1 -- 172.21.15.69:0/3904731530 shutdown_connections 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.071+0000 7f29fd400640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29f81098b0 0x7f29f819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.071+0000 7f29fd400640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f29dc0a2a80 0x7f29dc0a4f40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.071+0000 7f29fd400640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f8104260 0x7f29f819a6d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.071+0000 7f29fd400640 1 --2- 172.21.15.69:0/3904731530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29f81038a0 0x7f29f819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.071+0000 7f29fd400640 1 -- 172.21.15.69:0/3904731530 >> 172.21.15.69:0/3904731530 conn(0x7f29f80ff350 msgr2=0x7f29f80ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.072+0000 7f29fd400640 1 -- 172.21.15.69:0/3904731530 shutdown_connections 2024-03-19T01:11:01.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.072+0000 7f29fd400640 1 -- 172.21.15.69:0/3904731530 wait complete. 2024-03-19T01:11:01.090 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:11:01.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.326+0000 7f43d4c00640 1 -- 172.21.15.69:0/4214679121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d01038a0 msgr2=0x7f43d0103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.326+0000 7f43d4c00640 1 --2- 172.21.15.69:0/4214679121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d01038a0 0x7f43d0103d20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009900 tx=0x7f43bc02bc40 comp rx=0 tx=0).stop 2024-03-19T01:11:01.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.327+0000 7f43d4c00640 1 -- 172.21.15.69:0/4214679121 shutdown_connections 2024-03-19T01:11:01.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.327+0000 7f43d4c00640 1 --2- 172.21.15.69:0/4214679121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43d0104260 0x7f43d0110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.327+0000 7f43d4c00640 1 --2- 172.21.15.69:0/4214679121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d01038a0 0x7f43d0103d20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.327+0000 7f43d4c00640 1 --2- 172.21.15.69:0/4214679121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d01098b0 0x7f43d0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.327+0000 7f43d4c00640 1 -- 172.21.15.69:0/4214679121 >> 172.21.15.69:0/4214679121 conn(0x7f43d00ff330 msgr2=0x7f43d0101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.327+0000 7f43d4c00640 1 -- 172.21.15.69:0/4214679121 shutdown_connections 2024-03-19T01:11:01.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.327+0000 7f43d4c00640 1 -- 172.21.15.69:0/4214679121 wait complete. 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.328+0000 7f43d4c00640 1 Processor -- start 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.328+0000 7f43d4c00640 1 -- start start 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.328+0000 7f43d4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d01038a0 0x7f43d019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.328+0000 7f43d4c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43d0104260 0x7f43d019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.328+0000 7f43d4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d01098b0 0x7f43d019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.328+0000 7f43d4c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0113d10 con 0x7f43d0104260 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.328+0000 7f43d4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0113b90 con 0x7f43d01038a0 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.328+0000 7f43d4c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0113e90 con 0x7f43d01098b0 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.329+0000 7f43cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d01038a0 0x7f43d019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.329+0000 7f43cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d01038a0 0x7f43d019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57554/0 (socket says 172.21.15.69:57554) 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.329+0000 7f43cf400640 1 -- 172.21.15.69:0/2420303960 learned_addr learned my addr 172.21.15.69:0/2420303960 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.329+0000 7f43cea00640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43d0104260 0x7f43d019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.329+0000 7f43cfe00640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d01098b0 0x7f43d019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.329+0000 7f43cf400640 1 -- 172.21.15.69:0/2420303960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d01098b0 msgr2=0x7f43d019ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.329+0000 7f43cf400640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d01098b0 0x7f43d019ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.329+0000 7f43cf400640 1 -- 172.21.15.69:0/2420303960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43d0104260 msgr2=0x7f43d019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.329+0000 7f43cf400640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43d0104260 0x7f43d019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.329+0000 7f43cf400640 1 -- 172.21.15.69:0/2420303960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d019f160 con 0x7f43d01038a0 2024-03-19T01:11:01.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.329+0000 7f43cf400640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d01038a0 0x7f43d019a170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e9f0 tx=0x7f43c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.330+0000 7f43c7e00640 1 -- 172.21.15.69:0/2420303960 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c000cd80 con 0x7f43d01038a0 2024-03-19T01:11:01.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.330+0000 7f43c7e00640 1 -- 172.21.15.69:0/2420303960 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c0010910 con 0x7f43d01038a0 2024-03-19T01:11:01.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.330+0000 7f43c7e00640 1 -- 172.21.15.69:0/2420303960 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0018980 con 0x7f43d01038a0 2024-03-19T01:11:01.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.330+0000 7f43d4c00640 1 -- 172.21.15.69:0/2420303960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d019f450 con 0x7f43d01038a0 2024-03-19T01:11:01.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.330+0000 7f43d4c00640 1 -- 172.21.15.69:0/2420303960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d01a6ce0 con 0x7f43d01038a0 2024-03-19T01:11:01.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.333+0000 7f43d8f87740 1 Processor -- start 2024-03-19T01:11:01.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.334+0000 7f43d8f87740 1 -- start start 2024-03-19T01:11:01.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.334+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591932718a0 0x559193271c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.334+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932721c0 0x55919328bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.334+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 0x55919328e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.334+0000 7f43d8f87740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5591931d6380 con 0x5591932718a0 2024-03-19T01:11:01.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.334+0000 7f43d8f87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55919317ed20 con 0x55919328c100 2024-03-19T01:11:01.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.334+0000 7f43d8f87740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5591931d2420 con 0x5591932721c0 2024-03-19T01:11:01.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.334+0000 7f43cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 0x55919328e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.334+0000 7f43cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 0x55919328e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57556/0 (socket says 172.21.15.69:57556) 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.334+0000 7f43cea00640 1 -- 172.21.15.69:0/272786989 learned_addr learned my addr 172.21.15.69:0/272786989 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.335+0000 7f43cfe00640 1 --2- 172.21.15.69:0/272786989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591932718a0 0x559193271c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.335+0000 7f43cea00640 1 -- 172.21.15.69:0/272786989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932721c0 msgr2=0x55919328bbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.335+0000 7f43cea00640 1 --2- 172.21.15.69:0/272786989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932721c0 0x55919328bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.335+0000 7f43cea00640 1 -- 172.21.15.69:0/272786989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591932718a0 msgr2=0x559193271c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.335+0000 7f43cea00640 1 --2- 172.21.15.69:0/272786989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591932718a0 0x559193271c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.335+0000 7f43cea00640 1 -- 172.21.15.69:0/272786989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559192eea3f0 con 0x55919328c100 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.335+0000 7f43c7e00640 1 -- 172.21.15.69:0/2420303960 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 137466+0+0 (secure 0 0 0) 0x7f43c0010ac0 con 0x7f43d01038a0 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.336+0000 7f43cea00640 1 --2- 172.21.15.69:0/272786989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 0x55919328e4f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009900 tx=0x7f43bc005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd093a7e9d9d58ee server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.336+0000 7f43bbe00640 1 -- 172.21.15.69:0/272786989 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc004240 con 0x55919328c100 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.336+0000 7f43bbe00640 1 -- 172.21.15.69:0/272786989 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43bc0043e0 con 0x55919328c100 2024-03-19T01:11:01.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.336+0000 7f43bbe00640 1 -- 172.21.15.69:0/272786989 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc0051e0 con 0x55919328c100 2024-03-19T01:11:01.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43c7e00640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f43b40a2a80 0x7f43b40a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43d8f87740 1 -- 172.21.15.69:0/272786989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 msgr2=0x55919328e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43d8f87740 1 --2- 172.21.15.69:0/272786989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 0x55919328e4f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009900 tx=0x7f43bc005d50 comp rx=0 tx=0).stop 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43c7e00640 1 -- 172.21.15.69:0/2420303960 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f43c00cf7b0 con 0x7f43d01038a0 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43d8f87740 1 -- 172.21.15.69:0/272786989 shutdown_connections 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43d8f87740 1 --2- 172.21.15.69:0/272786989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 0x55919328e4f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43d8f87740 1 --2- 172.21.15.69:0/272786989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932721c0 0x55919328bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43d8f87740 1 --2- 172.21.15.69:0/272786989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591932718a0 0x559193271c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43cfe00640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f43b40a2a80 0x7f43b40a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43d8f87740 1 -- 172.21.15.69:0/272786989 >> 172.21.15.69:0/272786989 conn(0x55919326c320 msgr2=0x55919326c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43d8f87740 1 -- 172.21.15.69:0/272786989 shutdown_connections 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.337+0000 7f43d8f87740 1 -- 172.21.15.69:0/272786989 wait complete. 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.338+0000 7f43cfe00640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f43b40a2a80 0x7f43b40a4f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43c8005fd0 tx=0x7f43c8009600 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.338+0000 7f43d8f87740 1 -- start start 2024-03-19T01:11:01.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.338+0000 7f43d8f87740 1 Processor -- start 2024-03-19T01:11:01.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.339+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x55919327d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.339+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932721c0 0x55919327d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.339+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55919328c100 0x559193278810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.339+0000 7f43d8f87740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559193272d30 con 0x55919328c100 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.339+0000 7f43d8f87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559193192160 con 0x5591932721c0 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.339+0000 7f43d8f87740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559193272eb0 con 0x5591932718a0 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.339+0000 7f43d8f87740 10 client.0 _ll_register_callbacks cb 0x5591931ce170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.339+0000 7f43cf400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x55919327d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.339+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55919328c100 0x559193278810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.339+0000 7f43cf400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x55919327d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52488/0 (socket says 172.21.15.69:52488) 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.339+0000 7f43cf400640 1 -- 172.21.15.69:0/2009897335 learned_addr learned my addr 172.21.15.69:0/2009897335 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.340+0000 7f43cea00640 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932721c0 0x55919327d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.340+0000 7f43cf400640 1 -- 172.21.15.69:0/2009897335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932721c0 msgr2=0x55919327d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.340+0000 7f43cf400640 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932721c0 0x55919327d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.340+0000 7f43cf400640 1 -- 172.21.15.69:0/2009897335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55919328c100 msgr2=0x559193278810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.340+0000 7f43cf400640 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55919328c100 0x559193278810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.340+0000 7f43cf400640 1 -- 172.21.15.69:0/2009897335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559192fb47d0 con 0x5591932718a0 2024-03-19T01:11:01.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.340+0000 7f43cf400640 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x55919327d410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f43c00ccb10 tx=0x7f43c009b300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d52ebf480b688df9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.340+0000 7f43b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:01.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.340+0000 7f43b8c00640 1 -- 172.21.15.69:0/2009897335 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c00b4060 con 0x5591932718a0 2024-03-19T01:11:01.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.340+0000 7f43b8c00640 1 -- 172.21.15.69:0/2009897335 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f43c00ccd30 con 0x5591932718a0 2024-03-19T01:11:01.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.341+0000 7f43d8f87740 1 -- 172.21.15.69:0/2009897335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43bc0043e0 con 0x5591932718a0 2024-03-19T01:11:01.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.341+0000 7f43b8c00640 1 -- 172.21.15.69:0/2009897335 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f43c00b9050 con 0x5591932718a0 2024-03-19T01:11:01.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.341+0000 7f43d8f87740 10 client.25801 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.342+0000 7f43d8f87740 1 -- 172.21.15.69:0/2009897335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591932876c0 con 0x5591932718a0 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.342+0000 7f43b8c00640 1 -- 172.21.15.69:0/2009897335 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43bc0043e0 con 0x5591932718a0 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.342+0000 7f43b8c00640 1 -- 172.21.15.69:0/2009897335 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f43c00cf3b0 con 0x5591932718a0 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.342+0000 7f43d8f87740 10 client.25801 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.342+0000 7f43d8f87740 10 client.25801 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.342+0000 7f43d8f87740 20 client.25801 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.342+0000 7f43d8f87740 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x559193288b80 0x55919328aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.342+0000 7f43d8f87740 4 client.25801 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.342+0000 7f43d8f87740 1 -- 172.21.15.69:0/2009897335 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591931e38c0 con 0x559193288b80 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.343+0000 7f43cea00640 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x559193288b80 0x55919328aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.343+0000 7f43cea00640 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x559193288b80 0x55919328aff0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=908ad48652e4a230 server_cookie=b249632496f51a1e in_seq=0 out_seq=0 2024-03-19T01:11:01.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.344+0000 7f43b8c00640 10 client.25801 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:11:01.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.344+0000 7f43b8c00640 1 -- 172.21.15.69:0/2009897335 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5591931e38c0 con 0x559193288b80 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.344+0000 7f43b8c00640 10 client.25801 handle_command_reply: tid=0 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.344+0000 7f43d8f87740 1 client.25801 shutdown 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43d8f87740 1 -- 172.21.15.69:0/2009897335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 msgr2=0x55919327d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43d8f87740 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x55919327d410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f43c00ccb10 tx=0x7f43c009b300 comp rx=0 tx=0).stop 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43d8f87740 1 -- 172.21.15.69:0/2009897335 shutdown_connections 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43d8f87740 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55919328c100 0x559193278810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43d8f87740 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x559193288b80 0x55919328aff0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43d8f87740 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932721c0 0x55919327d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43d8f87740 1 --2- 172.21.15.69:0/2009897335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x55919327d410 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43d8f87740 1 -- 172.21.15.69:0/2009897335 >> 172.21.15.69:0/2009897335 conn(0x55919326c320 msgr2=0x55919326e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43b8c00640 0 client.25801 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43d8f87740 1 -- 172.21.15.69:0/2009897335 shutdown_connections 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43d8f87740 1 -- 172.21.15.69:0/2009897335 wait complete. 2024-03-19T01:11:01.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.345+0000 7f43d8f87740 20 client.25801 trim_cache size 0 max 16384 2024-03-19T01:11:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.371+0000 7f43d8f87740 1 Processor -- start 2024-03-19T01:11:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.371+0000 7f43d8f87740 1 -- start start 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.371+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x559193278460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.371+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591932721c0 0x559193275a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.371+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 0x559193275fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.371+0000 7f43d8f87740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01a8490 con 0x5591932721c0 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.371+0000 7f43d8f87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01a9fd0 con 0x55919328c100 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.371+0000 7f43d8f87740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01a8b60 con 0x5591932718a0 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43cf400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x559193278460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43cf400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x559193278460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52496/0 (socket says 172.21.15.69:52496) 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43cf400640 1 -- 172.21.15.69:0/123552902 learned_addr learned my addr 172.21.15.69:0/123552902 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43cfe00640 1 --2- 172.21.15.69:0/123552902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 0x559193275fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43cea00640 1 --2- 172.21.15.69:0/123552902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591932721c0 0x559193275a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43cf400640 1 -- 172.21.15.69:0/123552902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 msgr2=0x559193275fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43cf400640 1 --2- 172.21.15.69:0/123552902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 0x559193275fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43cf400640 1 -- 172.21.15.69:0/123552902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591932721c0 msgr2=0x559193275a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43cf400640 1 --2- 172.21.15.69:0/123552902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591932721c0 0x559193275a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43cf400640 1 -- 172.21.15.69:0/123552902 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591932716d0 con 0x5591932718a0 2024-03-19T01:11:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43cf400640 1 --2- 172.21.15.69:0/123552902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x559193278460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f43c00cd140 tx=0x7f43c009b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1e8aa8b7c8a7919 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43bb400640 1 -- 172.21.15.69:0/123552902 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c009b420 con 0x5591932718a0 2024-03-19T01:11:01.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43bb400640 1 -- 172.21.15.69:0/123552902 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c009fa30 con 0x5591932718a0 2024-03-19T01:11:01.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.372+0000 7f43bb400640 1 -- 172.21.15.69:0/123552902 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0086290 con 0x5591932718a0 2024-03-19T01:11:01.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.373+0000 7f43d8f87740 1 -- 172.21.15.69:0/123552902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 msgr2=0x559193278460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.373+0000 7f43d8f87740 1 --2- 172.21.15.69:0/123552902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x559193278460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f43c00cd140 tx=0x7f43c009b370 comp rx=0 tx=0).stop 2024-03-19T01:11:01.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.373+0000 7f43d8f87740 1 -- 172.21.15.69:0/123552902 shutdown_connections 2024-03-19T01:11:01.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.373+0000 7f43d8f87740 1 --2- 172.21.15.69:0/123552902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55919328c100 0x559193275fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.373+0000 7f43d8f87740 1 --2- 172.21.15.69:0/123552902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591932721c0 0x559193275a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.373+0000 7f43d8f87740 1 --2- 172.21.15.69:0/123552902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932718a0 0x559193278460 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.373+0000 7f43d8f87740 1 -- 172.21.15.69:0/123552902 >> 172.21.15.69:0/123552902 conn(0x55919326c320 msgr2=0x559193277d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.373+0000 7f43d8f87740 1 -- 172.21.15.69:0/123552902 shutdown_connections 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.374+0000 7f43d8f87740 1 -- 172.21.15.69:0/123552902 wait complete. 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.374+0000 7f43d8f87740 1 -- start start 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.374+0000 7f43d8f87740 1 Processor -- start 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932718a0 0x5591932ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932721c0 0x5591932ead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43d8f87740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55919328c100 0x559193292fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43d8f87740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01a9900 con 0x55919328c100 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43d8f87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01a9230 con 0x5591932718a0 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43d8f87740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0073010 con 0x5591932721c0 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43d8f87740 10 client.0 _ll_register_callbacks cb 0x7f43c00cc910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932718a0 0x5591932ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932718a0 0x5591932ea7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57576/0 (socket says 172.21.15.69:57576) 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43cf400640 1 -- 172.21.15.69:0/1696961844 learned_addr learned my addr 172.21.15.69:0/1696961844 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43cfe00640 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55919328c100 0x559193292fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43cea00640 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932721c0 0x5591932ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43cf400640 1 -- 172.21.15.69:0/1696961844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932721c0 msgr2=0x5591932ead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43cf400640 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932721c0 0x5591932ead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43cf400640 1 -- 172.21.15.69:0/1696961844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55919328c100 msgr2=0x559193292fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43cf400640 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55919328c100 0x559193292fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43cf400640 1 -- 172.21.15.69:0/1696961844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559193293690 con 0x5591932718a0 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.375+0000 7f43cfe00640 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55919328c100 0x559193292fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43cf400640 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932718a0 0x5591932ea7d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f43c009b5e0 tx=0x7f43c009b300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33a5da6b4cb6ddda server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43b8c00640 1 -- 172.21.15.69:0/1696961844 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c00b4070 con 0x5591932718a0 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43b8c00640 1 -- 172.21.15.69:0/1696961844 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f43c009ad30 con 0x5591932718a0 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43d8f87740 1 -- 172.21.15.69:0/1696961844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43c009fa30 con 0x5591932718a0 2024-03-19T01:11:01.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43b8c00640 1 -- 172.21.15.69:0/1696961844 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f43c00ab050 con 0x5591932718a0 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43d8f87740 10 client.25811 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43d8f87740 1 -- 172.21.15.69:0/1696961844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591932f1fd0 con 0x5591932718a0 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43b8c00640 1 -- 172.21.15.69:0/1696961844 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c009fa30 con 0x5591932718a0 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43b8c00640 1 -- 172.21.15.69:0/1696961844 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f43c0097630 con 0x5591932718a0 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43d8f87740 10 client.25811 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43d8f87740 10 client.25811 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.376+0000 7f43d8f87740 20 client.25811 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.377+0000 7f43d8f87740 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5591932f24a0 0x5591932f4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.377+0000 7f43d8f87740 4 client.25811 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.377+0000 7f43d8f87740 1 -- 172.21.15.69:0/1696961844 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591931ddc10 con 0x5591932f24a0 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.377+0000 7f43cea00640 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5591932f24a0 0x5591932f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43b8c00640 10 client.25811 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43cea00640 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5591932f24a0 0x5591932f4890 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1855abdddfb6a2c9 server_cookie=1ae30a7073f8f340 in_seq=0 out_seq=0 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43b8c00640 1 -- 172.21.15.69:0/1696961844 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591931ddc10 con 0x5591932f24a0 2024-03-19T01:11:01.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43b8c00640 10 client.25811 handle_command_reply: tid=0 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43d8f87740 1 client.25811 shutdown 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43d8f87740 1 -- 172.21.15.69:0/1696961844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932718a0 msgr2=0x5591932ea7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43d8f87740 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932718a0 0x5591932ea7d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f43c009b5e0 tx=0x7f43c009b300 comp rx=0 tx=0).stop 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43d8f87740 1 -- 172.21.15.69:0/1696961844 shutdown_connections 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43d8f87740 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55919328c100 0x559193292fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43d8f87740 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5591932721c0 0x5591932ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43d8f87740 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5591932f24a0 0x5591932f4890 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43d8f87740 1 --2- 172.21.15.69:0/1696961844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591932718a0 0x5591932ea7d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43b8c00640 0 client.25811 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43d8f87740 1 -- 172.21.15.69:0/1696961844 >> 172.21.15.69:0/1696961844 conn(0x55919326c320 msgr2=0x5591932785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43d8f87740 1 -- 172.21.15.69:0/1696961844 shutdown_connections 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.378+0000 7f43d8f87740 1 -- 172.21.15.69:0/1696961844 wait complete. 2024-03-19T01:11:01.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.379+0000 7f43d8f87740 20 client.25811 trim_cache size 0 max 16384 2024-03-19T01:11:01.386 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:11:01.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.379+0000 7f43d4c00640 1 -- 172.21.15.69:0/2420303960 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f43b40a2a80 msgr2=0x7f43b40a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.379+0000 7f43d4c00640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f43b40a2a80 0x7f43b40a4f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43c8005fd0 tx=0x7f43c8009600 comp rx=0 tx=0).stop 2024-03-19T01:11:01.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.379+0000 7f43d4c00640 1 -- 172.21.15.69:0/2420303960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d01038a0 msgr2=0x7f43d019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.379+0000 7f43d4c00640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d01038a0 0x7f43d019a170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e9f0 tx=0x7f43c000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.379+0000 7f43d4c00640 1 -- 172.21.15.69:0/2420303960 shutdown_connections 2024-03-19T01:11:01.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.379+0000 7f43d4c00640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d01098b0 0x7f43d019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.379+0000 7f43d4c00640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43d0104260 0x7f43d019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.379+0000 7f43d4c00640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f43b40a2a80 0x7f43b40a4f40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.379+0000 7f43d4c00640 1 --2- 172.21.15.69:0/2420303960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d01038a0 0x7f43d019a170 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.379+0000 7f43d4c00640 1 -- 172.21.15.69:0/2420303960 >> 172.21.15.69:0/2420303960 conn(0x7f43d00ff330 msgr2=0x7f43d00ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.380+0000 7f43d4c00640 1 -- 172.21.15.69:0/2420303960 shutdown_connections 2024-03-19T01:11:01.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.380+0000 7f43d4c00640 1 -- 172.21.15.69:0/2420303960 wait complete. 2024-03-19T01:11:01.395 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:01 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3067728456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:11:01.399 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:11:01.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:01 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3067728456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:11:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:01 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3067728456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:11:01.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.595+0000 7f04c7e00640 1 -- 172.21.15.69:0/3739601917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c8102050 msgr2=0x7f04c810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.595+0000 7f04c7e00640 1 --2- 172.21.15.69:0/3739601917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c8102050 0x7f04c810e560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009cd0 tx=0x7f04c002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.595+0000 7f04c7e00640 1 -- 172.21.15.69:0/3739601917 shutdown_connections 2024-03-19T01:11:01.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.595+0000 7f04c7e00640 1 --2- 172.21.15.69:0/3739601917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c8102050 0x7f04c810e560 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.595+0000 7f04c7e00640 1 --2- 172.21.15.69:0/3739601917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04c8101690 0x7f04c8101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.595+0000 7f04c7e00640 1 --2- 172.21.15.69:0/3739601917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c81076a0 0x7f04c8107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.595+0000 7f04c7e00640 1 -- 172.21.15.69:0/3739601917 >> 172.21.15.69:0/3739601917 conn(0x7f04c80fd1c0 msgr2=0x7f04c80ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.595+0000 7f04c7e00640 1 -- 172.21.15.69:0/3739601917 shutdown_connections 2024-03-19T01:11:01.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.595+0000 7f04c7e00640 1 -- 172.21.15.69:0/3739601917 wait complete. 2024-03-19T01:11:01.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.596+0000 7f04c7e00640 1 Processor -- start 2024-03-19T01:11:01.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.596+0000 7f04c7e00640 1 -- start start 2024-03-19T01:11:01.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.597+0000 7f04c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8101690 0x7f04c8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.597+0000 7f04c7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c8102050 0x7f04c8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.597+0000 7f04c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04c81076a0 0x7f04c819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.597+0000 7f04c7e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8111ab0 con 0x7f04c81076a0 2024-03-19T01:11:01.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.597+0000 7f04c7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8111930 con 0x7f04c8101690 2024-03-19T01:11:01.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.597+0000 7f04c7e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8111c30 con 0x7f04c8102050 2024-03-19T01:11:01.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.597+0000 7f04c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8101690 0x7f04c8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.597+0000 7f04c6000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c8102050 0x7f04c8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.597+0000 7f04c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8101690 0x7f04c8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57602/0 (socket says 172.21.15.69:57602) 2024-03-19T01:11:01.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.597+0000 7f04c6a00640 1 -- 172.21.15.69:0/711045427 learned_addr learned my addr 172.21.15.69:0/711045427 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.597+0000 7f04c7400640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04c81076a0 0x7f04c819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04c6000640 1 -- 172.21.15.69:0/711045427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8101690 msgr2=0x7f04c8197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04c6000640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8101690 0x7f04c8197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04c6000640 1 -- 172.21.15.69:0/711045427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04c81076a0 msgr2=0x7f04c819c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04c6000640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04c81076a0 0x7f04c819c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04c6000640 1 -- 172.21.15.69:0/711045427 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c819cf70 con 0x7f04c8102050 2024-03-19T01:11:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04c6a00640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8101690 0x7f04c8197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04c6000640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c8102050 0x7f04c8198510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f04b4005fd0 tx=0x7f04b400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04bf400640 1 -- 172.21.15.69:0/711045427 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4009c30 con 0x7f04c8102050 2024-03-19T01:11:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04bf400640 1 -- 172.21.15.69:0/711045427 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b4009dd0 con 0x7f04c8102050 2024-03-19T01:11:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04c7e00640 1 -- 172.21.15.69:0/711045427 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c819d260 con 0x7f04c8102050 2024-03-19T01:11:01.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04bf400640 1 -- 172.21.15.69:0/711045427 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4010460 con 0x7f04c8102050 2024-03-19T01:11:01.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.598+0000 7f04c7e00640 1 -- 172.21.15.69:0/711045427 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c81a4af0 con 0x7f04c8102050 2024-03-19T01:11:01.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.603+0000 7f04d056f740 1 Processor -- start 2024-03-19T01:11:01.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.603+0000 7f04d056f740 1 -- start start 2024-03-19T01:11:01.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.603+0000 7f04d056f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba960fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.603+0000 7f04d056f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 0x555cba969bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04d056f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba96a0f0 0x555cba96c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04d056f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555cba8c5380 con 0x555cba961510 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04d056f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555cba86dd20 con 0x555cba96a0f0 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04d056f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555cba8c1420 con 0x555cba960bf0 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04bf400640 1 -- 172.21.15.69:0/711045427 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137466+0+0 (secure 0 0 0) 0x7f04b4003ef0 con 0x7f04c8102050 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04c7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba960fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04c7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba960fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52538/0 (socket says 172.21.15.69:52538) 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04c7400640 1 -- 172.21.15.69:0/634674569 learned_addr learned my addr 172.21.15.69:0/634674569 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04c6000640 1 --2- 172.21.15.69:0/634674569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba96a0f0 0x555cba96c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04c7400640 1 -- 172.21.15.69:0/634674569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba96a0f0 msgr2=0x555cba96c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04c6a00640 1 --2- 172.21.15.69:0/634674569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 0x555cba969bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04c7400640 1 --2- 172.21.15.69:0/634674569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba96a0f0 0x555cba96c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04c7400640 1 -- 172.21.15.69:0/634674569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 msgr2=0x555cba969bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04c7400640 1 --2- 172.21.15.69:0/634674569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 0x555cba969bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04c7400640 1 -- 172.21.15.69:0/634674569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cba5d93f0 con 0x555cba960bf0 2024-03-19T01:11:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.604+0000 7f04c6a00640 1 --2- 172.21.15.69:0/634674569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 0x555cba969bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:01.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.605+0000 7f04c7400640 1 --2- 172.21.15.69:0/634674569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba960fd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f04c0004af0 tx=0x7f04c0005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b24bf8b06a3af899 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.605+0000 7f04b3e00640 1 -- 172.21.15.69:0/634674569 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0043070 con 0x555cba960bf0 2024-03-19T01:11:01.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.605+0000 7f04b3e00640 1 -- 172.21.15.69:0/634674569 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c002cca0 con 0x555cba960bf0 2024-03-19T01:11:01.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.605+0000 7f04b3e00640 1 -- 172.21.15.69:0/634674569 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0039040 con 0x555cba960bf0 2024-03-19T01:11:01.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.605+0000 7f04bf400640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04ac0a2a00 0x7f04ac0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.606+0000 7f04bf400640 1 -- 172.21.15.69:0/711045427 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f04b40cee60 con 0x7f04c8102050 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.606+0000 7f04d056f740 1 -- 172.21.15.69:0/634674569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 msgr2=0x555cba960fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.606+0000 7f04d056f740 1 --2- 172.21.15.69:0/634674569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba960fd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f04c0004af0 tx=0x7f04c0005f80 comp rx=0 tx=0).stop 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.606+0000 7f04d056f740 1 -- 172.21.15.69:0/634674569 shutdown_connections 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.606+0000 7f04d056f740 1 --2- 172.21.15.69:0/634674569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba96a0f0 0x555cba96c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.606+0000 7f04d056f740 1 --2- 172.21.15.69:0/634674569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 0x555cba969bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.606+0000 7f04d056f740 1 --2- 172.21.15.69:0/634674569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba960fd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.606+0000 7f04d056f740 1 -- 172.21.15.69:0/634674569 >> 172.21.15.69:0/634674569 conn(0x555cba95b320 msgr2=0x555cba95b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.606+0000 7f04c6a00640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04ac0a2a00 0x7f04ac0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.606+0000 7f04d056f740 1 -- 172.21.15.69:0/634674569 shutdown_connections 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.607+0000 7f04d056f740 1 -- 172.21.15.69:0/634674569 wait complete. 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.607+0000 7f04d056f740 1 -- start start 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.607+0000 7f04c6a00640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04ac0a2a00 0x7f04ac0a4ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x555cba9606c0 tx=0x7f04b8005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.607+0000 7f04d056f740 1 Processor -- start 2024-03-19T01:11:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.608+0000 7f04d056f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba9742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.608+0000 7f04d056f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 0x555cba974830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.608+0000 7f04d056f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba96a0f0 0x555cba96e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.608+0000 7f04d056f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555cba96ce80 con 0x555cba961510 2024-03-19T01:11:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.608+0000 7f04d056f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555cba881160 con 0x555cba96a0f0 2024-03-19T01:11:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.608+0000 7f04d056f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555cba96d000 con 0x555cba960bf0 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.608+0000 7f04d056f740 10 client.0 _ll_register_callbacks cb 0x555cba8bd170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.608+0000 7f04c6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba9742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.608+0000 7f04c6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba9742f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52540/0 (socket says 172.21.15.69:52540) 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.608+0000 7f04c6a00640 1 -- 172.21.15.69:0/3526566780 learned_addr learned my addr 172.21.15.69:0/3526566780 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.609+0000 7f04c6000640 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 0x555cba974830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.609+0000 7f04c7400640 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba96a0f0 0x555cba96e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.609+0000 7f04c6000640 1 -- 172.21.15.69:0/3526566780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 msgr2=0x555cba9742f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.609+0000 7f04c6000640 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba9742f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.609+0000 7f04c6000640 1 -- 172.21.15.69:0/3526566780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba96a0f0 msgr2=0x555cba96e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.609+0000 7f04c6000640 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba96a0f0 0x555cba96e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.609+0000 7f04c6000640 1 -- 172.21.15.69:0/3526566780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cba6a37d0 con 0x555cba961510 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.609+0000 7f04c6000640 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 0x555cba974830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04b40cc130 tx=0x7f04b409ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f845f84392be99 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.609+0000 7f04b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.610+0000 7f04b0c00640 1 -- 172.21.15.69:0/3526566780 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4020050 con 0x555cba961510 2024-03-19T01:11:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.610+0000 7f04b0c00640 1 -- 172.21.15.69:0/3526566780 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f04b40cc3e0 con 0x555cba961510 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.610+0000 7f04d056f740 1 -- 172.21.15.69:0/3526566780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c002cca0 con 0x555cba961510 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.610+0000 7f04b0c00640 1 -- 172.21.15.69:0/3526566780 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f04b40cc5e0 con 0x555cba961510 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.610+0000 7f04d056f740 10 client.15669 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.610+0000 7f04d056f740 1 -- 172.21.15.69:0/3526566780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555cba97d540 con 0x555cba961510 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.611+0000 7f04b0c00640 1 -- 172.21.15.69:0/3526566780 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c002cca0 con 0x555cba961510 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.611+0000 7f04b0c00640 1 -- 172.21.15.69:0/3526566780 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f04b4095bb0 con 0x555cba961510 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.611+0000 7f04d056f740 10 client.15669 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.611+0000 7f04d056f740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.611+0000 7f04d056f740 20 client.15669 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.611+0000 7f04d056f740 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555cba97ea00 0x555cba980df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.611+0000 7f04d056f740 4 client.15669 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.611+0000 7f04d056f740 1 -- 172.21.15.69:0/3526566780 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cba8d28c0 con 0x555cba97ea00 2024-03-19T01:11:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.611+0000 7f04c6a00640 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555cba97ea00 0x555cba980df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.612+0000 7f04c6a00640 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555cba97ea00 0x555cba980df0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a062ffb9057c482f server_cookie=c35f3cda35af014 in_seq=0 out_seq=0 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.612+0000 7f04b0c00640 10 client.15669 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.613+0000 7f04b0c00640 1 -- 172.21.15.69:0/3526566780 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555cba8d28c0 con 0x555cba97ea00 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.613+0000 7f04b0c00640 10 client.15669 handle_command_reply: tid=0 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.613+0000 7f04d056f740 1 client.15669 shutdown 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.613+0000 7f04d056f740 1 -- 172.21.15.69:0/3526566780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 msgr2=0x555cba974830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.613+0000 7f04d056f740 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 0x555cba974830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04b40cc130 tx=0x7f04b409ae10 comp rx=0 tx=0).stop 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.613+0000 7f04d056f740 1 -- 172.21.15.69:0/3526566780 shutdown_connections 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.613+0000 7f04d056f740 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555cba97ea00 0x555cba980df0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.613+0000 7f04d056f740 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba96a0f0 0x555cba96e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.613+0000 7f04d056f740 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba961510 0x555cba974830 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.614+0000 7f04d056f740 1 --2- 172.21.15.69:0/3526566780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba9742f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.614+0000 7f04b0c00640 0 client.15669 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.614+0000 7f04d056f740 1 -- 172.21.15.69:0/3526566780 >> 172.21.15.69:0/3526566780 conn(0x555cba95b320 msgr2=0x555cba95d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.614+0000 7f04d056f740 1 -- 172.21.15.69:0/3526566780 shutdown_connections 2024-03-19T01:11:01.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.614+0000 7f04d056f740 1 -- 172.21.15.69:0/3526566780 wait complete. 2024-03-19T01:11:01.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.614+0000 7f04d056f740 20 client.15669 trim_cache size 0 max 16384 2024-03-19T01:11:01.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.641+0000 7f04d056f740 1 Processor -- start 2024-03-19T01:11:01.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04d056f740 1 -- start start 2024-03-19T01:11:01.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04d056f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba974550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04d056f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba961510 0x555cba986650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04d056f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba96a0f0 0x555cba9687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04d056f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81ac950 con 0x555cba96a0f0 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04d056f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81ae490 con 0x555cba961510 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04d056f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81ad020 con 0x555cba960bf0 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04c6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba974550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04c6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba974550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52556/0 (socket says 172.21.15.69:52556) 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04c6a00640 1 -- 172.21.15.69:0/2194254930 learned_addr learned my addr 172.21.15.69:0/2194254930 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04c6000640 1 --2- 172.21.15.69:0/2194254930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba961510 0x555cba986650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.642+0000 7f04c7400640 1 --2- 172.21.15.69:0/2194254930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba96a0f0 0x555cba9687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.643+0000 7f04c6a00640 1 -- 172.21.15.69:0/2194254930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba961510 msgr2=0x555cba986650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.643+0000 7f04c6a00640 1 --2- 172.21.15.69:0/2194254930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba961510 0x555cba986650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.643+0000 7f04c6a00640 1 -- 172.21.15.69:0/2194254930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba96a0f0 msgr2=0x555cba9687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.643+0000 7f04c6a00640 1 --2- 172.21.15.69:0/2194254930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba96a0f0 0x555cba9687e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.643+0000 7f04c6a00640 1 -- 172.21.15.69:0/2194254930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cba981160 con 0x555cba960bf0 2024-03-19T01:11:01.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.643+0000 7f04c6a00640 1 --2- 172.21.15.69:0/2194254930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba974550 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x555cba9815c0 tx=0x7f04b8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc29e3f5f3c62905 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.643+0000 7f04b3400640 1 -- 172.21.15.69:0/2194254930 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b802ad90 con 0x555cba960bf0 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.643+0000 7f04b3400640 1 -- 172.21.15.69:0/2194254930 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b802f070 con 0x555cba960bf0 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.643+0000 7f04b3400640 1 -- 172.21.15.69:0/2194254930 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b801ee40 con 0x555cba960bf0 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.644+0000 7f04d056f740 1 -- 172.21.15.69:0/2194254930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 msgr2=0x555cba974550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.644+0000 7f04d056f740 1 --2- 172.21.15.69:0/2194254930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba974550 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x555cba9815c0 tx=0x7f04b8016000 comp rx=0 tx=0).stop 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.644+0000 7f04d056f740 1 -- 172.21.15.69:0/2194254930 shutdown_connections 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.644+0000 7f04d056f740 1 --2- 172.21.15.69:0/2194254930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba96a0f0 0x555cba9687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.644+0000 7f04d056f740 1 --2- 172.21.15.69:0/2194254930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba961510 0x555cba986650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.644+0000 7f04d056f740 1 --2- 172.21.15.69:0/2194254930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba974550 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.644+0000 7f04d056f740 1 -- 172.21.15.69:0/2194254930 >> 172.21.15.69:0/2194254930 conn(0x555cba95b320 msgr2=0x555cba973e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.644+0000 7f04d056f740 1 -- 172.21.15.69:0/2194254930 shutdown_connections 2024-03-19T01:11:01.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.644+0000 7f04d056f740 1 -- 172.21.15.69:0/2194254930 wait complete. 2024-03-19T01:11:01.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.645+0000 7f04d056f740 1 -- start start 2024-03-19T01:11:01.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.645+0000 7f04d056f740 1 Processor -- start 2024-03-19T01:11:01.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04d056f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba97a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04d056f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba961510 0x555cba97a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04d056f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba96a0f0 0x555cba978840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04d056f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81addc0 con 0x555cba96a0f0 2024-03-19T01:11:01.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04d056f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81ad6f0 con 0x555cba961510 2024-03-19T01:11:01.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04d056f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81aa930 con 0x555cba960bf0 2024-03-19T01:11:01.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04c6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba97a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04d056f740 10 client.0 _ll_register_callbacks cb 0x7f04b40cbfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04c6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba97a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52564/0 (socket says 172.21.15.69:52564) 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04c6a00640 1 -- 172.21.15.69:0/1727944506 learned_addr learned my addr 172.21.15.69:0/1727944506 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04c6000640 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba961510 0x555cba97a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.646+0000 7f04c7400640 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba96a0f0 0x555cba978840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04c6a00640 1 -- 172.21.15.69:0/1727944506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba961510 msgr2=0x555cba97a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04c6a00640 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba961510 0x555cba97a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04c6a00640 1 -- 172.21.15.69:0/1727944506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba96a0f0 msgr2=0x555cba978840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04c6a00640 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba96a0f0 0x555cba978840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04c6a00640 1 -- 172.21.15.69:0/1727944506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cba978f30 con 0x555cba960bf0 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04c6000640 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba961510 0x555cba97a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04c6a00640 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba97a2a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009460 tx=0x7f04b801ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a573f2df28cf085d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04b0c00640 1 -- 172.21.15.69:0/1727944506 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8038070 con 0x555cba960bf0 2024-03-19T01:11:01.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04b0c00640 1 -- 172.21.15.69:0/1727944506 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f04b802ad90 con 0x555cba960bf0 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04d056f740 1 -- 172.21.15.69:0/1727944506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b802f070 con 0x555cba960bf0 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04b0c00640 1 -- 172.21.15.69:0/1727944506 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f04b8044050 con 0x555cba960bf0 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.647+0000 7f04d056f740 10 client.25825 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.648+0000 7f04d056f740 1 -- 172.21.15.69:0/1727944506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555cba9e57d0 con 0x555cba960bf0 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.648+0000 7f04b0c00640 1 -- 172.21.15.69:0/1727944506 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b802f070 con 0x555cba960bf0 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.648+0000 7f04b0c00640 1 -- 172.21.15.69:0/1727944506 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f04b80052a0 con 0x555cba960bf0 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.648+0000 7f04d056f740 10 client.25825 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.648+0000 7f04d056f740 10 client.25825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.648+0000 7f04d056f740 20 client.25825 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.648+0000 7f04d056f740 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555cba9e5e60 0x555cba9e8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.648+0000 7f04d056f740 4 client.25825 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.648+0000 7f04d056f740 1 -- 172.21.15.69:0/1727944506 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cba8ccc10 con 0x555cba9e5e60 2024-03-19T01:11:01.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.648+0000 7f04c6000640 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555cba9e5e60 0x555cba9e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04c6000640 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555cba9e5e60 0x555cba9e8280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e68819d64b6e51 server_cookie=eb060e1108f68720 in_seq=0 out_seq=0 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04b0c00640 10 client.25825 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04b0c00640 1 -- 172.21.15.69:0/1727944506 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cba8ccc10 con 0x555cba9e5e60 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04b0c00640 10 client.25825 handle_command_reply: tid=0 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04d056f740 1 client.25825 shutdown 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04d056f740 1 -- 172.21.15.69:0/1727944506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 msgr2=0x555cba97a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04d056f740 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba97a2a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009460 tx=0x7f04b801ec40 comp rx=0 tx=0).stop 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04d056f740 1 -- 172.21.15.69:0/1727944506 shutdown_connections 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04d056f740 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555cba9e5e60 0x555cba9e8280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04d056f740 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cba96a0f0 0x555cba978840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04d056f740 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cba961510 0x555cba97a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04d056f740 1 --2- 172.21.15.69:0/1727944506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555cba960bf0 0x555cba97a2a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04d056f740 1 -- 172.21.15.69:0/1727944506 >> 172.21.15.69:0/1727944506 conn(0x555cba95b320 msgr2=0x555cba96e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.649+0000 7f04b0c00640 0 client.25825 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:01.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.650+0000 7f04d056f740 1 -- 172.21.15.69:0/1727944506 shutdown_connections 2024-03-19T01:11:01.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.650+0000 7f04d056f740 1 -- 172.21.15.69:0/1727944506 wait complete. 2024-03-19T01:11:01.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.650+0000 7f04d056f740 20 client.25825 trim_cache size 0 max 16384 2024-03-19T01:11:01.654 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 -- 172.21.15.69:0/711045427 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04ac0a2a00 msgr2=0x7f04ac0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04ac0a2a00 0x7f04ac0a4ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x555cba9606c0 tx=0x7f04b8005f90 comp rx=0 tx=0).stop 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 -- 172.21.15.69:0/711045427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c8102050 msgr2=0x7f04c8198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c8102050 0x7f04c8198510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f04b4005fd0 tx=0x7f04b400dde0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 -- 172.21.15.69:0/711045427 shutdown_connections 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04c81076a0 0x7f04c819c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c8102050 0x7f04c8198510 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04ac0a2a00 0x7f04ac0a4ec0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 --2- 172.21.15.69:0/711045427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c8101690 0x7f04c8197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 -- 172.21.15.69:0/711045427 >> 172.21.15.69:0/711045427 conn(0x7f04c80fd1c0 msgr2=0x7f04c80fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 -- 172.21.15.69:0/711045427 shutdown_connections 2024-03-19T01:11:01.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:01.651+0000 7f04c7e00640 1 -- 172.21.15.69:0/711045427 wait complete. 2024-03-19T01:11:02.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:02 smithi086 ceph-mon[66829]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-19T01:11:02.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:02 smithi069 ceph-mon[31727]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-19T01:11:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:02 smithi163 ceph-mon[67428]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-19T01:11:02.671 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:11:02.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:11:02.672 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:11:02.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.923+0000 7f2e22c00640 1 -- 172.21.15.69:0/4041863521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c102050 msgr2=0x7f2e1c10e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:02.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.923+0000 7f2e22c00640 1 --2- 172.21.15.69:0/4041863521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c102050 0x7f2e1c10e560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009ca0 tx=0x7f2e1402c100 comp rx=0 tx=0).stop 2024-03-19T01:11:02.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.923+0000 7f2e22c00640 1 -- 172.21.15.69:0/4041863521 shutdown_connections 2024-03-19T01:11:02.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.923+0000 7f2e22c00640 1 --2- 172.21.15.69:0/4041863521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c102050 0x7f2e1c10e560 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.923+0000 7f2e22c00640 1 --2- 172.21.15.69:0/4041863521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e1c101690 0x7f2e1c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.923+0000 7f2e22c00640 1 --2- 172.21.15.69:0/4041863521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e1c1076a0 0x7f2e1c107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.923+0000 7f2e22c00640 1 -- 172.21.15.69:0/4041863521 >> 172.21.15.69:0/4041863521 conn(0x7f2e1c0fd1c0 msgr2=0x7f2e1c0ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:02.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.923+0000 7f2e22c00640 1 -- 172.21.15.69:0/4041863521 shutdown_connections 2024-03-19T01:11:02.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.924+0000 7f2e22c00640 1 -- 172.21.15.69:0/4041863521 wait complete. 2024-03-19T01:11:02.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.925+0000 7f2e22c00640 1 Processor -- start 2024-03-19T01:11:02.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.925+0000 7f2e22c00640 1 -- start start 2024-03-19T01:11:02.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e22c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c101690 0x7f2e1c19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e22c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e1c102050 0x7f2e1c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e22c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e1c1076a0 0x7f2e1c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e22c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c111ab0 con 0x7f2e1c102050 2024-03-19T01:11:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e22c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c111930 con 0x7f2e1c101690 2024-03-19T01:11:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e22c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c111c30 con 0x7f2e1c1076a0 2024-03-19T01:11:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e21800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c101690 0x7f2e1c19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e21800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c101690 0x7f2e1c19a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57640/0 (socket says 172.21.15.69:57640) 2024-03-19T01:11:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e21800640 1 -- 172.21.15.69:0/2166196552 learned_addr learned my addr 172.21.15.69:0/2166196552 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e22200640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e1c1076a0 0x7f2e1c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e21800640 1 -- 172.21.15.69:0/2166196552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e1c1076a0 msgr2=0x7f2e1c19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:02.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e21800640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e1c1076a0 0x7f2e1c19e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.926+0000 7f2e21800640 1 -- 172.21.15.69:0/2166196552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e1c102050 msgr2=0x7f2e1c19a650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:11:02.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.927+0000 7f2e20e00640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e1c102050 0x7f2e1c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.927+0000 7f2e21800640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e1c102050 0x7f2e1c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.927+0000 7f2e21800640 1 -- 172.21.15.69:0/2166196552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c19f100 con 0x7f2e1c101690 2024-03-19T01:11:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.927+0000 7f2e22200640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e1c1076a0 0x7f2e1c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.927+0000 7f2e20e00640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e1c102050 0x7f2e1c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.928+0000 7f2e21800640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c101690 0x7f2e1c19a110 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800e9f0 tx=0x7f2e0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.928+0000 7f2e1a000640 1 -- 172.21.15.69:0/2166196552 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800cd80 con 0x7f2e1c101690 2024-03-19T01:11:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.928+0000 7f2e1a000640 1 -- 172.21.15.69:0/2166196552 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e08010910 con 0x7f2e1c101690 2024-03-19T01:11:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.928+0000 7f2e1a000640 1 -- 172.21.15.69:0/2166196552 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08018980 con 0x7f2e1c101690 2024-03-19T01:11:02.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.928+0000 7f2e22c00640 1 -- 172.21.15.69:0/2166196552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1c19f3f0 con 0x7f2e1c101690 2024-03-19T01:11:02.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.929+0000 7f2e22c00640 1 -- 172.21.15.69:0/2166196552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1c1084f0 con 0x7f2e1c101690 2024-03-19T01:11:02.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.934+0000 7f2e26e61740 1 Processor -- start 2024-03-19T01:11:02.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.934+0000 7f2e26e61740 1 -- start start 2024-03-19T01:11:02.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.934+0000 7f2e1a000640 1 -- 172.21.15.69:0/2166196552 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7f2e08010ac0 con 0x7f2e1c101690 2024-03-19T01:11:02.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.936+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 0x5654966e30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.936+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 0x5654966ebca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.936+0000 7f2e22200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 0x5654966e30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.936+0000 7f2e22200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 0x5654966e30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45482/0 (socket says 172.21.15.69:45482) 2024-03-19T01:11:02.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.937+0000 7f2e21800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 0x5654966ebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.937+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966ec1e0 0x5654966ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.937+0000 7f2e21800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 0x5654966ebca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57652/0 (socket says 172.21.15.69:57652) 2024-03-19T01:11:02.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.937+0000 7f2e26e61740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x565496645fb0 con 0x5654966e2ce0 2024-03-19T01:11:02.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.937+0000 7f2e26e61740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654965eeab0 con 0x5654966e3600 2024-03-19T01:11:02.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.937+0000 7f2e26e61740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x565496642050 con 0x5654966ec1e0 2024-03-19T01:11:02.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.937+0000 7f2e22200640 1 -- 172.21.15.69:0/644304313 learned_addr learned my addr 172.21.15.69:0/644304313 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:02.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.938+0000 7f2e21800640 1 -- 172.21.15.69:0/644304313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966ec1e0 msgr2=0x5654966ee5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:02.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.937+0000 7f2e20e00640 1 --2- 172.21.15.69:0/644304313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966ec1e0 0x5654966ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.938+0000 7f2e21800640 1 --2- 172.21.15.69:0/644304313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966ec1e0 0x5654966ee5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.938+0000 7f2e21800640 1 -- 172.21.15.69:0/644304313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 msgr2=0x5654966e30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:02.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.938+0000 7f2e21800640 1 --2- 172.21.15.69:0/644304313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 0x5654966e30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.938+0000 7f2e21800640 1 -- 172.21.15.69:0/644304313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549660ce00 con 0x5654966e3600 2024-03-19T01:11:02.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.938+0000 7f2e20e00640 1 --2- 172.21.15.69:0/644304313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966ec1e0 0x5654966ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:02.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.938+0000 7f2e22200640 1 --2- 172.21.15.69:0/644304313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 0x5654966e30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:02.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.938+0000 7f2e1a000640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e040a2b10 0x7f2e040a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.939+0000 7f2e1a000640 1 -- 172.21.15.69:0/2166196552 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f2e080cf810 con 0x7f2e1c101690 2024-03-19T01:11:02.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.939+0000 7f2e21800640 1 --2- 172.21.15.69:0/644304313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 0x5654966ebca0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2e080cca20 tx=0x7f2e080d5ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cf2d16adf5c1257 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:02.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.939+0000 7f2e22200640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e040a2b10 0x7f2e040a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.941+0000 7f2e0ea00640 1 -- 172.21.15.69:0/644304313 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08097b00 con 0x5654966e3600 2024-03-19T01:11:02.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.941+0000 7f2e0ea00640 1 -- 172.21.15.69:0/644304313 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0809bf60 con 0x5654966e3600 2024-03-19T01:11:02.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.941+0000 7f2e0ea00640 1 -- 172.21.15.69:0/644304313 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0809b1c0 con 0x5654966e3600 2024-03-19T01:11:02.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.942+0000 7f2e26e61740 1 -- 172.21.15.69:0/644304313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 msgr2=0x5654966ebca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:02.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.942+0000 7f2e26e61740 1 --2- 172.21.15.69:0/644304313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 0x5654966ebca0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2e080cca20 tx=0x7f2e080d5ef0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.942+0000 7f2e22200640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e040a2b10 0x7f2e040a4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009a60 tx=0x7f2e14005d80 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:02.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.942+0000 7f2e26e61740 1 -- 172.21.15.69:0/644304313 shutdown_connections 2024-03-19T01:11:02.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.942+0000 7f2e26e61740 1 --2- 172.21.15.69:0/644304313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966ec1e0 0x5654966ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.942+0000 7f2e26e61740 1 --2- 172.21.15.69:0/644304313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 0x5654966ebca0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.942+0000 7f2e26e61740 1 --2- 172.21.15.69:0/644304313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 0x5654966e30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.942+0000 7f2e26e61740 1 -- 172.21.15.69:0/644304313 >> 172.21.15.69:0/644304313 conn(0x5654966dd410 msgr2=0x5654966dd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:02.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.942+0000 7f2e26e61740 1 -- 172.21.15.69:0/644304313 shutdown_connections 2024-03-19T01:11:02.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.942+0000 7f2e26e61740 1 -- 172.21.15.69:0/644304313 wait complete. 2024-03-19T01:11:02.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.943+0000 7f2e26e61740 1 -- start start 2024-03-19T01:11:02.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.943+0000 7f2e26e61740 1 Processor -- start 2024-03-19T01:11:02.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.944+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 0x5654966e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.944+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e3600 0x5654966e97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.944+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966ec1e0 0x5654966e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.944+0000 7f2e26e61740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5654966eefd0 con 0x5654966e2ce0 2024-03-19T01:11:02.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.944+0000 7f2e26e61740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565496601f00 con 0x5654966ec1e0 2024-03-19T01:11:02.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.944+0000 7f2e26e61740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5654966ef150 con 0x5654966e3600 2024-03-19T01:11:02.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.944+0000 7f2e26e61740 10 client.0 _ll_register_callbacks cb 0x5654966018d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:02.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.945+0000 7f2e22200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966ec1e0 0x5654966e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.945+0000 7f2e22200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966ec1e0 0x5654966e9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57662/0 (socket says 172.21.15.69:57662) 2024-03-19T01:11:02.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.945+0000 7f2e22200640 1 -- 172.21.15.69:0/3231541751 learned_addr learned my addr 172.21.15.69:0/3231541751 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:02.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.945+0000 7f2e20e00640 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e3600 0x5654966e97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.945+0000 7f2e21800640 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 0x5654966e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.946+0000 7f2e22200640 1 -- 172.21.15.69:0/3231541751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e3600 msgr2=0x5654966e97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:02.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.946+0000 7f2e22200640 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e3600 0x5654966e97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.946+0000 7f2e22200640 1 -- 172.21.15.69:0/3231541751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 msgr2=0x5654966e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:02.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.946+0000 7f2e22200640 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 0x5654966e9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.946+0000 7f2e22200640 1 -- 172.21.15.69:0/3231541751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654964227e0 con 0x5654966ec1e0 2024-03-19T01:11:02.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.946+0000 7f2e22200640 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966ec1e0 0x5654966e9d00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2e1402c640 tx=0x7f2e1402c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fab4c6a979edfccb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:02.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.946+0000 7f2e03400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:02.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.946+0000 7f2e03400640 1 -- 172.21.15.69:0/3231541751 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14052020 con 0x5654966ec1e0 2024-03-19T01:11:02.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.947+0000 7f2e03400640 1 -- 172.21.15.69:0/3231541751 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f2e14041670 con 0x5654966ec1e0 2024-03-19T01:11:02.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.947+0000 7f2e26e61740 1 -- 172.21.15.69:0/3231541751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0809bf60 con 0x5654966ec1e0 2024-03-19T01:11:02.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.947+0000 7f2e03400640 1 -- 172.21.15.69:0/3231541751 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e14039040 con 0x5654966ec1e0 2024-03-19T01:11:02.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.947+0000 7f2e26e61740 10 client.25841 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:02.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.947+0000 7f2e26e61740 1 -- 172.21.15.69:0/3231541751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565496482f70 con 0x5654966ec1e0 2024-03-19T01:11:02.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.948+0000 7f2e03400640 1 -- 172.21.15.69:0/3231541751 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0809bf60 con 0x5654966ec1e0 2024-03-19T01:11:02.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.948+0000 7f2e03400640 1 -- 172.21.15.69:0/3231541751 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e1403f040 con 0x5654966ec1e0 2024-03-19T01:11:02.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.948+0000 7f2e26e61740 10 client.25841 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:02.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.948+0000 7f2e26e61740 10 client.25841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:02.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.948+0000 7f2e26e61740 20 client.25841 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:02.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.948+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5654966fc530 0x5654966fe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.948+0000 7f2e26e61740 4 client.25841 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:02.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.948+0000 7f2e26e61740 1 -- 172.21.15.69:0/3231541751 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654966534f0 con 0x5654966fc530 2024-03-19T01:11:02.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.948+0000 7f2e21800640 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5654966fc530 0x5654966fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.949+0000 7f2e21800640 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5654966fc530 0x5654966fe950 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370edd0585666fdd server_cookie=fd4459a3afa4ab55 in_seq=0 out_seq=0 2024-03-19T01:11:02.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.949+0000 7f2e03400640 10 client.25841 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:02.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e03400640 1 -- 172.21.15.69:0/3231541751 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5654966534f0 con 0x5654966fc530 2024-03-19T01:11:02.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e03400640 10 client.25841 handle_command_reply: tid=0 2024-03-19T01:11:02.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e26e61740 1 client.25841 shutdown 2024-03-19T01:11:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e26e61740 1 -- 172.21.15.69:0/3231541751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966ec1e0 msgr2=0x5654966e9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966ec1e0 0x5654966e9d00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2e1402c640 tx=0x7f2e1402c100 comp rx=0 tx=0).stop 2024-03-19T01:11:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e26e61740 1 -- 172.21.15.69:0/3231541751 shutdown_connections 2024-03-19T01:11:02.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5654966fc530 0x5654966fe950 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966ec1e0 0x5654966e9d00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e3600 0x5654966e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3231541751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e2ce0 0x5654966e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e26e61740 1 -- 172.21.15.69:0/3231541751 >> 172.21.15.69:0/3231541751 conn(0x5654966dd410 msgr2=0x5654966df160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:02.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.950+0000 7f2e03400640 0 client.25841 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:02.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.951+0000 7f2e26e61740 1 -- 172.21.15.69:0/3231541751 shutdown_connections 2024-03-19T01:11:02.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.951+0000 7f2e26e61740 1 -- 172.21.15.69:0/3231541751 wait complete. 2024-03-19T01:11:02.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.951+0000 7f2e26e61740 20 client.25841 trim_cache size 0 max 16384 2024-03-19T01:11:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.987+0000 7f2e26e61740 1 Processor -- start 2024-03-19T01:11:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.987+0000 7f2e26e61740 1 -- start start 2024-03-19T01:11:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.987+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e2ce0 0x5654966eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.987+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e3600 0x5654966e9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.987+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966ec1e0 0x565496704220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.987+0000 7f2e26e61740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b52e0 con 0x5654966e3600 2024-03-19T01:11:02.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.987+0000 7f2e26e61740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b6e20 con 0x5654966e2ce0 2024-03-19T01:11:02.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.987+0000 7f2e26e61740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b59b0 con 0x5654966ec1e0 2024-03-19T01:11:02.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.988+0000 7f2e21800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e2ce0 0x5654966eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.988+0000 7f2e21800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e2ce0 0x5654966eb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57678/0 (socket says 172.21.15.69:57678) 2024-03-19T01:11:02.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.988+0000 7f2e21800640 1 -- 172.21.15.69:0/1469396089 learned_addr learned my addr 172.21.15.69:0/1469396089 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:02.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.988+0000 7f2e21800640 1 -- 172.21.15.69:0/1469396089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966ec1e0 msgr2=0x565496704220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:11:02.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.988+0000 7f2e21800640 1 --2- 172.21.15.69:0/1469396089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966ec1e0 0x565496704220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.988+0000 7f2e21800640 1 -- 172.21.15.69:0/1469396089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e3600 msgr2=0x5654966e9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:02.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.988+0000 7f2e21800640 1 --2- 172.21.15.69:0/1469396089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e3600 0x5654966e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.988+0000 7f2e21800640 1 -- 172.21.15.69:0/1469396089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549631a170 con 0x5654966e2ce0 2024-03-19T01:11:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.990+0000 7f2e21800640 1 --2- 172.21.15.69:0/1469396089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e2ce0 0x5654966eb9b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2e080cd1a0 tx=0x7f2e08023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1b5884fc33e2132 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.990+0000 7f2e0e000640 1 -- 172.21.15.69:0/1469396089 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0809b3a0 con 0x5654966e2ce0 2024-03-19T01:11:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.990+0000 7f2e0e000640 1 -- 172.21.15.69:0/1469396089 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e080cc960 con 0x5654966e2ce0 2024-03-19T01:11:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.990+0000 7f2e0e000640 1 -- 172.21.15.69:0/1469396089 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0809b630 con 0x5654966e2ce0 2024-03-19T01:11:02.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.991+0000 7f2e26e61740 1 -- 172.21.15.69:0/1469396089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e2ce0 msgr2=0x5654966eb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:02.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.991+0000 7f2e26e61740 1 --2- 172.21.15.69:0/1469396089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e2ce0 0x5654966eb9b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2e080cd1a0 tx=0x7f2e08023040 comp rx=0 tx=0).stop 2024-03-19T01:11:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.992+0000 7f2e26e61740 1 -- 172.21.15.69:0/1469396089 shutdown_connections 2024-03-19T01:11:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.992+0000 7f2e26e61740 1 --2- 172.21.15.69:0/1469396089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966ec1e0 0x565496704220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.992+0000 7f2e26e61740 1 --2- 172.21.15.69:0/1469396089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966e3600 0x5654966e9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.992+0000 7f2e26e61740 1 --2- 172.21.15.69:0/1469396089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e2ce0 0x5654966eb9b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.992+0000 7f2e26e61740 1 -- 172.21.15.69:0/1469396089 >> 172.21.15.69:0/1469396089 conn(0x5654966dd410 msgr2=0x5654966de5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.992+0000 7f2e26e61740 1 -- 172.21.15.69:0/1469396089 shutdown_connections 2024-03-19T01:11:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.992+0000 7f2e26e61740 1 -- 172.21.15.69:0/1469396089 wait complete. 2024-03-19T01:11:02.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.992+0000 7f2e26e61740 1 -- start start 2024-03-19T01:11:02.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.993+0000 7f2e26e61740 1 Processor -- start 2024-03-19T01:11:02.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.993+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e2ce0 0x5654966effc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.993+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 0x5654966f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.993+0000 7f2e26e61740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966ec1e0 0x565496761850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.993+0000 7f2e26e61740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b6750 con 0x5654966ec1e0 2024-03-19T01:11:02.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.993+0000 7f2e26e61740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b6080 con 0x5654966e3600 2024-03-19T01:11:02.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.993+0000 7f2e26e61740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b3320 con 0x5654966e2ce0 2024-03-19T01:11:02.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.993+0000 7f2e26e61740 10 client.0 _ll_register_callbacks cb 0x7f2e14032950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:02.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.994+0000 7f2e21800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e2ce0 0x5654966effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.994+0000 7f2e21800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e2ce0 0x5654966effc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52626/0 (socket says 172.21.15.69:52626) 2024-03-19T01:11:02.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.994+0000 7f2e21800640 1 -- 172.21.15.69:0/3709423222 learned_addr learned my addr 172.21.15.69:0/3709423222 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:02.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.994+0000 7f2e20e00640 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 0x5654966f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.994+0000 7f2e22200640 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966ec1e0 0x565496761850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.994+0000 7f2e21800640 1 -- 172.21.15.69:0/3709423222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 msgr2=0x5654966f0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:02.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.994+0000 7f2e21800640 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 0x5654966f0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.994+0000 7f2e21800640 1 -- 172.21.15.69:0/3709423222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966ec1e0 msgr2=0x565496761850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:02.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.994+0000 7f2e21800640 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966ec1e0 0x565496761850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:02.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.994+0000 7f2e21800640 1 -- 172.21.15.69:0/3709423222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654966f0dd0 con 0x5654966e2ce0 2024-03-19T01:11:02.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.995+0000 7f2e21800640 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e2ce0 0x5654966effc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e08099b10 tx=0x7f2e0800cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd8f940fe4607af server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:02.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.995+0000 7f2e03400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:02.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.995+0000 7f2e03400640 1 -- 172.21.15.69:0/3709423222 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0809b630 con 0x5654966e2ce0 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.995+0000 7f2e03400640 1 -- 172.21.15.69:0/3709423222 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f2e080860c0 con 0x5654966e2ce0 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.995+0000 7f2e26e61740 1 -- 172.21.15.69:0/3709423222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e080cc960 con 0x5654966e2ce0 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.995+0000 7f2e03400640 1 -- 172.21.15.69:0/3709423222 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e0809b3a0 con 0x5654966e2ce0 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.995+0000 7f2e26e61740 10 client.25837 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.995+0000 7f2e26e61740 1 -- 172.21.15.69:0/3709423222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565496762ea0 con 0x5654966e2ce0 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.996+0000 7f2e03400640 1 -- 172.21.15.69:0/3709423222 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e080cc960 con 0x5654966e2ce0 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.996+0000 7f2e03400640 1 -- 172.21.15.69:0/3709423222 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e080ce070 con 0x5654966e2ce0 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.996+0000 7f2e26e61740 10 client.25837 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.996+0000 7f2e26e61740 10 client.25837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.996+0000 7f2e26e61740 20 client.25837 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.996+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5654967630d0 0x5654967654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.996+0000 7f2e26e61740 4 client.25837 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:11:02.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.996+0000 7f2e26e61740 1 -- 172.21.15.69:0/3709423222 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56549664d840 con 0x5654967630d0 2024-03-19T01:11:02.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.998+0000 7f2e20e00640 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5654967630d0 0x5654967654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:02.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.998+0000 7f2e20e00640 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5654967630d0 0x5654967654c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=690034f9f52cf237 server_cookie=42b713a4f7fa67ba in_seq=0 out_seq=0 2024-03-19T01:11:02.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:02.998+0000 7f2e03400640 10 client.25837 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.000+0000 7f2e03400640 1 -- 172.21.15.69:0/3709423222 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56549664d840 con 0x5654967630d0 2024-03-19T01:11:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.000+0000 7f2e03400640 10 client.25837 handle_command_reply: tid=0 2024-03-19T01:11:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.000+0000 7f2e26e61740 1 client.25837 shutdown 2024-03-19T01:11:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.000+0000 7f2e26e61740 1 -- 172.21.15.69:0/3709423222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e2ce0 msgr2=0x5654966effc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.001+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e2ce0 0x5654966effc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e08099b10 tx=0x7f2e0800cf20 comp rx=0 tx=0).stop 2024-03-19T01:11:03.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.002+0000 7f2e26e61740 1 -- 172.21.15.69:0/3709423222 shutdown_connections 2024-03-19T01:11:03.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.002+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5654967630d0 0x5654967654c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.002+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654966ec1e0 0x565496761850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.002+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654966e3600 0x5654966f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.002+0000 7f2e03400640 0 client.25837 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:03.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.002+0000 7f2e26e61740 1 --2- 172.21.15.69:0/3709423222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5654966e2ce0 0x5654966effc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.002+0000 7f2e26e61740 1 -- 172.21.15.69:0/3709423222 >> 172.21.15.69:0/3709423222 conn(0x5654966dd410 msgr2=0x5654966dee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:03.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.003+0000 7f2e26e61740 1 -- 172.21.15.69:0/3709423222 shutdown_connections 2024-03-19T01:11:03.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.003+0000 7f2e26e61740 1 -- 172.21.15.69:0/3709423222 wait complete. 2024-03-19T01:11:03.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.003+0000 7f2e26e61740 20 client.25837 trim_cache size 0 max 16384 2024-03-19T01:11:03.005 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:11:03.005 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:11:03.005 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "91ff3fbf-e734-4c75-8079-1d475afd55d4", 2024-03-19T01:11:03.005 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:11:03.005 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:11:03.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.004+0000 7f2e22c00640 1 -- 172.21.15.69:0/2166196552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e040a2b10 msgr2=0x7f2e040a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.004+0000 7f2e22c00640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e040a2b10 0x7f2e040a4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009a60 tx=0x7f2e14005d80 comp rx=0 tx=0).stop 2024-03-19T01:11:03.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.004+0000 7f2e22c00640 1 -- 172.21.15.69:0/2166196552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c101690 msgr2=0x7f2e1c19a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.004+0000 7f2e22c00640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c101690 0x7f2e1c19a110 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800e9f0 tx=0x7f2e0800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.005+0000 7f2e22c00640 1 -- 172.21.15.69:0/2166196552 shutdown_connections 2024-03-19T01:11:03.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.005+0000 7f2e22c00640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e1c1076a0 0x7f2e1c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.005+0000 7f2e22c00640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e1c102050 0x7f2e1c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.005+0000 7f2e22c00640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e040a2b10 0x7f2e040a4fd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.005+0000 7f2e22c00640 1 --2- 172.21.15.69:0/2166196552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c101690 0x7f2e1c19a110 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.005+0000 7f2e22c00640 1 -- 172.21.15.69:0/2166196552 >> 172.21.15.69:0/2166196552 conn(0x7f2e1c0fd1c0 msgr2=0x7f2e1c10cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:03.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.005+0000 7f2e22c00640 1 -- 172.21.15.69:0/2166196552 shutdown_connections 2024-03-19T01:11:03.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.005+0000 7f2e22c00640 1 -- 172.21.15.69:0/2166196552 wait complete. 2024-03-19T01:11:03.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:11:03.029 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:11:03.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:03 smithi069 ceph-mon[31727]: mgrmap e27: x(active, since 22s), standbys: y, z 2024-03-19T01:11:03.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.309+0000 7fc56ac00640 1 -- 172.21.15.69:0/2292877426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc564104380 msgr2=0x7fc564110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.309+0000 7fc56ac00640 1 --2- 172.21.15.69:0/2292877426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc564104380 0x7fc564110890 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009c10 tx=0x7fc55c02bf00 comp rx=0 tx=0).stop 2024-03-19T01:11:03.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.312+0000 7fc56ac00640 1 -- 172.21.15.69:0/2292877426 shutdown_connections 2024-03-19T01:11:03.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.312+0000 7fc56ac00640 1 --2- 172.21.15.69:0/2292877426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc564104380 0x7fc564110890 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.312+0000 7fc56ac00640 1 --2- 172.21.15.69:0/2292877426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5641039c0 0x7fc564103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.312+0000 7fc56ac00640 1 --2- 172.21.15.69:0/2292877426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5641099d0 0x7fc564109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.312+0000 7fc56ac00640 1 -- 172.21.15.69:0/2292877426 >> 172.21.15.69:0/2292877426 conn(0x7fc5640ff470 msgr2=0x7fc564101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:03.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.313+0000 7fc56ac00640 1 -- 172.21.15.69:0/2292877426 shutdown_connections 2024-03-19T01:11:03.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.313+0000 7fc56ac00640 1 -- 172.21.15.69:0/2292877426 wait complete. 2024-03-19T01:11:03.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.315+0000 7fc56ac00640 1 Processor -- start 2024-03-19T01:11:03.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.315+0000 7fc56ac00640 1 -- start start 2024-03-19T01:11:03.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.315+0000 7fc56ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5641039c0 0x7fc564072fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.315+0000 7fc56ac00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc564104380 0x7fc564073510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.315+0000 7fc56ac00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5641099d0 0x7fc56406c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.315+0000 7fc56ac00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc564113e00 con 0x7fc5641099d0 2024-03-19T01:11:03.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.315+0000 7fc56ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc564113c80 con 0x7fc5641039c0 2024-03-19T01:11:03.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.315+0000 7fc56ac00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc564113f80 con 0x7fc564104380 2024-03-19T01:11:03.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.316+0000 7fc563e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5641039c0 0x7fc564072fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.316+0000 7fc563e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5641039c0 0x7fc564072fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57698/0 (socket says 172.21.15.69:57698) 2024-03-19T01:11:03.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.316+0000 7fc563e00640 1 -- 172.21.15.69:0/4111980027 learned_addr learned my addr 172.21.15.69:0/4111980027 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:03.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.316+0000 7fc55be00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc564104380 0x7fc564073510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.316+0000 7fc568c00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5641099d0 0x7fc56406c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.316+0000 7fc563e00640 1 -- 172.21.15.69:0/4111980027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc564104380 msgr2=0x7fc564073510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.316+0000 7fc563e00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc564104380 0x7fc564073510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.316+0000 7fc563e00640 1 -- 172.21.15.69:0/4111980027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5641099d0 msgr2=0x7fc56406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.316+0000 7fc563e00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5641099d0 0x7fc56406c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.316+0000 7fc563e00640 1 -- 172.21.15.69:0/4111980027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56406caf0 con 0x7fc5641039c0 2024-03-19T01:11:03.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.317+0000 7fc563e00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5641039c0 0x7fc564072fd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc55400c920 tx=0x7fc55400cd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:03.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.317+0000 7fc561600640 1 -- 172.21.15.69:0/4111980027 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc554007bb0 con 0x7fc5641039c0 2024-03-19T01:11:03.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.317+0000 7fc561600640 1 -- 172.21.15.69:0/4111980027 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc554007d50 con 0x7fc5641039c0 2024-03-19T01:11:03.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.317+0000 7fc56ac00640 1 -- 172.21.15.69:0/4111980027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56406cde0 con 0x7fc5641039c0 2024-03-19T01:11:03.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.317+0000 7fc561600640 1 -- 172.21.15.69:0/4111980027 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5540187d0 con 0x7fc5641039c0 2024-03-19T01:11:03.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.317+0000 7fc56ac00640 1 -- 172.21.15.69:0/4111980027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc564070e10 con 0x7fc5641039c0 2024-03-19T01:11:03.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.322+0000 7fc56d9b8740 1 Processor -- start 2024-03-19T01:11:03.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.322+0000 7fc561600640 1 -- 172.21.15.69:0/4111980027 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7fc55400f450 con 0x7fc5641039c0 2024-03-19T01:11:03.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.324+0000 7fc56d9b8740 1 -- start start 2024-03-19T01:11:03.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.324+0000 7fc56d9b8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.324+0000 7fc56d9b8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 0x5645e63f3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.324+0000 7fc56d9b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63f3a50 0x5645e63f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.324+0000 7fc56d9b8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5645e63462c0 con 0x5645e63eae70 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.324+0000 7fc56d9b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5645e62eeb60 con 0x5645e63f3a50 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.324+0000 7fc56d9b8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5645e6342360 con 0x5645e63ea550 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.325+0000 7fc568c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.325+0000 7fc568c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ea930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52640/0 (socket says 172.21.15.69:52640) 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.325+0000 7fc568c00640 1 -- 172.21.15.69:0/659782472 learned_addr learned my addr 172.21.15.69:0/659782472 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.325+0000 7fc55be00640 1 --2- 172.21.15.69:0/659782472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63f3a50 0x5645e63f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.325+0000 7fc568c00640 1 -- 172.21.15.69:0/659782472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63f3a50 msgr2=0x5645e63f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.325+0000 7fc563e00640 1 --2- 172.21.15.69:0/659782472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 0x5645e63f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.325+0000 7fc568c00640 1 --2- 172.21.15.69:0/659782472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63f3a50 0x5645e63f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.325+0000 7fc568c00640 1 -- 172.21.15.69:0/659782472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 msgr2=0x5645e63f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.325+0000 7fc568c00640 1 --2- 172.21.15.69:0/659782472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 0x5645e63f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.325+0000 7fc568c00640 1 -- 172.21.15.69:0/659782472 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e61247d0 con 0x5645e63ea550 2024-03-19T01:11:03.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.325+0000 7fc563e00640 1 --2- 172.21.15.69:0/659782472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 0x5645e63f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:03.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.326+0000 7fc561600640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5480a2b90 0x7fc5480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.326+0000 7fc568c00640 1 --2- 172.21.15.69:0/659782472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ea930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc55c033420 tx=0x7fc55c009940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64db1bdd4699d494 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:03.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.326+0000 7fc561600640 1 -- 172.21.15.69:0/4111980027 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc5540cf5b0 con 0x7fc5641039c0 2024-03-19T01:11:03.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.326+0000 7fc559200640 1 -- 172.21.15.69:0/659782472 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c065070 con 0x5645e63ea550 2024-03-19T01:11:03.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.326+0000 7fc559200640 1 -- 172.21.15.69:0/659782472 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc55c02c8a0 con 0x5645e63ea550 2024-03-19T01:11:03.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.326+0000 7fc559200640 1 -- 172.21.15.69:0/659782472 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c0044a0 con 0x5645e63ea550 2024-03-19T01:11:03.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.327+0000 7fc56d9b8740 1 -- 172.21.15.69:0/659782472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 msgr2=0x5645e63ea930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.327+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/659782472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ea930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc55c033420 tx=0x7fc55c009940 comp rx=0 tx=0).stop 2024-03-19T01:11:03.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.327+0000 7fc563e00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5480a2b90 0x7fc5480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.328+0000 7fc56d9b8740 1 -- 172.21.15.69:0/659782472 shutdown_connections 2024-03-19T01:11:03.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.328+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/659782472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63f3a50 0x5645e63f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.328+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/659782472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 0x5645e63f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.328+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/659782472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ea930 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.328+0000 7fc56d9b8740 1 -- 172.21.15.69:0/659782472 >> 172.21.15.69:0/659782472 conn(0x5645e63e4c80 msgr2=0x5645e63e5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:03.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.328+0000 7fc56d9b8740 1 -- 172.21.15.69:0/659782472 shutdown_connections 2024-03-19T01:11:03.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.328+0000 7fc56d9b8740 1 -- 172.21.15.69:0/659782472 wait complete. 2024-03-19T01:11:03.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.328+0000 7fc563e00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5480a2b90 0x7fc5480a5050 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc5540cc7c0 tx=0x7fc55400fe30 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:03.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.329+0000 7fc56d9b8740 1 -- start start 2024-03-19T01:11:03.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.329+0000 7fc56d9b8740 1 Processor -- start 2024-03-19T01:11:03.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.331+0000 7fc56d9b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 0x5645e63f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.331+0000 7fc56d9b8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63eae70 0x5645e63f1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.331+0000 7fc56d9b8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63f3a50 0x5645e63f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.331+0000 7fc568c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 0x5645e63f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.331+0000 7fc568c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 0x5645e63f0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:57714/0 (socket says 172.21.15.69:57714) 2024-03-19T01:11:03.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.331+0000 7fc568c00640 1 -- 172.21.15.69:0/3389996580 learned_addr learned my addr 172.21.15.69:0/3389996580 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.331+0000 7fc56d9b8740 1 -- 172.21.15.69:0/3389996580 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5645e63f67e0 con 0x5645e63f3a50 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.331+0000 7fc56d9b8740 1 -- 172.21.15.69:0/3389996580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5645e6301fc0 con 0x5645e63ea550 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.331+0000 7fc56d9b8740 1 -- 172.21.15.69:0/3389996580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5645e63f6960 con 0x5645e63eae70 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.331+0000 7fc55be00640 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63eae70 0x5645e63f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.332+0000 7fc55be00640 1 -- 172.21.15.69:0/3389996580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 msgr2=0x5645e63f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.332+0000 7fc55be00640 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 0x5645e63f0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.332+0000 7fc55be00640 1 -- 172.21.15.69:0/3389996580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63f3a50 msgr2=0x5645e63f15d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.332+0000 7fc56d9b8740 10 client.0 _ll_register_callbacks cb 0x5645e6301990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.332+0000 7fc568c00640 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63f3a50 0x5645e63f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.332+0000 7fc55be00640 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63f3a50 0x5645e63f15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.332+0000 7fc55be00640 1 -- 172.21.15.69:0/3389996580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e605a3f0 con 0x5645e63eae70 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.332+0000 7fc55be00640 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63eae70 0x5645e63f1090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc56406d810 tx=0x7fc54c011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0ac047b1807825 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.333+0000 7fc56d9b8740 1 -- 172.21.15.69:0/3389996580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc55c02c8a0 con 0x5645e63eae70 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.333+0000 7fc551600640 10 client.25849 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.333+0000 7fc551600640 1 -- 172.21.15.69:0/3389996580 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c01e020 con 0x5645e63eae70 2024-03-19T01:11:03.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.333+0000 7fc551600640 1 -- 172.21.15.69:0/3389996580 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc54c00fbc0 con 0x5645e63eae70 2024-03-19T01:11:03.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.333+0000 7fc551600640 1 -- 172.21.15.69:0/3389996580 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc54c00fe20 con 0x5645e63eae70 2024-03-19T01:11:03.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.333+0000 7fc56d9b8740 10 client.25849 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:03.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.333+0000 7fc56d9b8740 1 -- 172.21.15.69:0/3389996580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5645e601a0f0 con 0x5645e63eae70 2024-03-19T01:11:03.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.336+0000 7fc551600640 1 -- 172.21.15.69:0/3389996580 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc54c019070 con 0x5645e63eae70 2024-03-19T01:11:03.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.336+0000 7fc56d9b8740 10 client.25849 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:03.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.336+0000 7fc56d9b8740 10 client.25849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:03.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.336+0000 7fc56d9b8740 20 client.25849 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:03.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.336+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5645e63deb80 0x5645e63e0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.336+0000 7fc56d9b8740 4 client.25849 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:03.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.336+0000 7fc56d9b8740 1 -- 172.21.15.69:0/3389996580 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645e62e7610 con 0x5645e63deb80 2024-03-19T01:11:03.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.336+0000 7fc568c00640 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5645e63deb80 0x5645e63e0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.337+0000 7fc568c00640 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5645e63deb80 0x5645e63e0fb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd6e58c739579643 server_cookie=ed2de71a8c8b2295 in_seq=0 out_seq=0 2024-03-19T01:11:03.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.337+0000 7fc551600640 10 client.25849 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:03.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.338+0000 7fc551600640 1 -- 172.21.15.69:0/3389996580 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5645e62e7610 con 0x5645e63deb80 2024-03-19T01:11:03.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.338+0000 7fc551600640 10 client.25849 handle_command_reply: tid=0 2024-03-19T01:11:03.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.338+0000 7fc56d9b8740 1 client.25849 shutdown 2024-03-19T01:11:03.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.338+0000 7fc56d9b8740 1 -- 172.21.15.69:0/3389996580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63eae70 msgr2=0x5645e63f1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.338+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63eae70 0x5645e63f1090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc56406d810 tx=0x7fc54c011b50 comp rx=0 tx=0).stop 2024-03-19T01:11:03.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.338+0000 7fc56d9b8740 1 -- 172.21.15.69:0/3389996580 shutdown_connections 2024-03-19T01:11:03.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.339+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5645e63deb80 0x5645e63e0fb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.339+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63f3a50 0x5645e63f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.339+0000 7fc551600640 0 client.25849 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:03.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.339+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63eae70 0x5645e63f1090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.339+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/3389996580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 0x5645e63f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.339+0000 7fc56d9b8740 1 -- 172.21.15.69:0/3389996580 >> 172.21.15.69:0/3389996580 conn(0x5645e63e4c80 msgr2=0x5645e63e6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:03.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.339+0000 7fc56d9b8740 1 -- 172.21.15.69:0/3389996580 shutdown_connections 2024-03-19T01:11:03.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.339+0000 7fc56d9b8740 1 -- 172.21.15.69:0/3389996580 wait complete. 2024-03-19T01:11:03.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.340+0000 7fc56d9b8740 20 client.25849 trim_cache size 0 max 16384 2024-03-19T01:11:03.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.375+0000 7fc56d9b8740 1 Processor -- start 2024-03-19T01:11:03.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.376+0000 7fc56d9b8740 1 -- start start 2024-03-19T01:11:03.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.376+0000 7fc56d9b8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63deb80 0x5645e63f3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.376+0000 7fc56d9b8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.376+0000 7fc56d9b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63eae70 0x5645e63e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.376+0000 7fc56d9b8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc56419e440 con 0x5645e63deb80 2024-03-19T01:11:03.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.376+0000 7fc56d9b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc56419ff80 con 0x5645e63eae70 2024-03-19T01:11:03.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.376+0000 7fc56d9b8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc56419eb10 con 0x5645e63ea550 2024-03-19T01:11:03.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.378+0000 7fc55be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.378+0000 7fc55be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ff580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52656/0 (socket says 172.21.15.69:52656) 2024-03-19T01:11:03.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.378+0000 7fc55be00640 1 -- 172.21.15.69:0/7938970 learned_addr learned my addr 172.21.15.69:0/7938970 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:03.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.378+0000 7fc55be00640 1 -- 172.21.15.69:0/7938970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63eae70 msgr2=0x5645e63e3eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:11:03.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.379+0000 7fc55be00640 1 --2- 172.21.15.69:0/7938970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63eae70 0x5645e63e3eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.379+0000 7fc55be00640 1 -- 172.21.15.69:0/7938970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63deb80 msgr2=0x5645e63f3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:03.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.379+0000 7fc55be00640 1 --2- 172.21.15.69:0/7938970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63deb80 0x5645e63f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.379+0000 7fc55be00640 1 -- 172.21.15.69:0/7938970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e64005f0 con 0x5645e63ea550 2024-03-19T01:11:03.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.379+0000 7fc55be00640 1 --2- 172.21.15.69:0/7938970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ff580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc54c011b90 tx=0x7fc54c018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6daf1a8ccf358c83 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:03.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.379+0000 7fc553e00640 1 -- 172.21.15.69:0/7938970 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c000e60 con 0x5645e63ea550 2024-03-19T01:11:03.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.380+0000 7fc553e00640 1 -- 172.21.15.69:0/7938970 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc55c02c8a0 con 0x5645e63ea550 2024-03-19T01:11:03.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.380+0000 7fc553e00640 1 -- 172.21.15.69:0/7938970 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c000e60 con 0x5645e63ea550 2024-03-19T01:11:03.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.381+0000 7fc56d9b8740 1 -- 172.21.15.69:0/7938970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 msgr2=0x5645e63ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.381+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/7938970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ff580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc54c011b90 tx=0x7fc54c018000 comp rx=0 tx=0).stop 2024-03-19T01:11:03.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.381+0000 7fc56d9b8740 1 -- 172.21.15.69:0/7938970 shutdown_connections 2024-03-19T01:11:03.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.381+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/7938970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63eae70 0x5645e63e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.381+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/7938970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63ea550 0x5645e63ff580 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.381+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/7938970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63deb80 0x5645e63f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.382+0000 7fc56d9b8740 1 -- 172.21.15.69:0/7938970 >> 172.21.15.69:0/7938970 conn(0x5645e63e4c80 msgr2=0x5645e63e37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:03.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.384+0000 7fc56d9b8740 1 -- 172.21.15.69:0/7938970 shutdown_connections 2024-03-19T01:11:03.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.384+0000 7fc56d9b8740 1 -- 172.21.15.69:0/7938970 wait complete. 2024-03-19T01:11:03.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.385+0000 7fc56d9b8740 1 -- start start 2024-03-19T01:11:03.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.385+0000 7fc56d9b8740 1 Processor -- start 2024-03-19T01:11:03.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.386+0000 7fc56d9b8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63deb80 0x5645e63fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.386+0000 7fc56d9b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 0x5645e63ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.386+0000 7fc568c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63deb80 0x5645e63fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.386+0000 7fc568c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63deb80 0x5645e63fbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52658/0 (socket says 172.21.15.69:52658) 2024-03-19T01:11:03.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.387+0000 7fc568c00640 1 -- 172.21.15.69:0/395778671 learned_addr learned my addr 172.21.15.69:0/395778671 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:03.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.387+0000 7fc55be00640 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 0x5645e63ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.387+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 0x5645e63ed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.387+0000 7fc56d9b8740 1 -- 172.21.15.69:0/395778671 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc56419f8b0 con 0x5645e63eae70 2024-03-19T01:11:03.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.387+0000 7fc56d9b8740 1 -- 172.21.15.69:0/395778671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc56419f1e0 con 0x5645e63ea550 2024-03-19T01:11:03.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.388+0000 7fc56d9b8740 1 -- 172.21.15.69:0/395778671 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc56419c460 con 0x5645e63deb80 2024-03-19T01:11:03.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.388+0000 7fc56d9b8740 10 client.0 _ll_register_callbacks cb 0x7fc54c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:03.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.388+0000 7fc568c00640 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 0x5645e63ed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.389+0000 7fc55be00640 1 -- 172.21.15.69:0/395778671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63deb80 msgr2=0x5645e63fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:03.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.389+0000 7fc55be00640 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63deb80 0x5645e63fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.389+0000 7fc55be00640 1 -- 172.21.15.69:0/395778671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 msgr2=0x5645e63ed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:03.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.389+0000 7fc55be00640 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 0x5645e63ed730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.389+0000 7fc55be00640 1 -- 172.21.15.69:0/395778671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e63eaca0 con 0x5645e63ea550 2024-03-19T01:11:03.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.389+0000 7fc568c00640 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 0x5645e63ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:03.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.389+0000 7fc55be00640 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 0x5645e63ed1f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc54c000c00 tx=0x7fc54c02cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76cc2096cbb072c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:03.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.389+0000 7fc551600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:03.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.390+0000 7fc551600640 1 -- 172.21.15.69:0/395778671 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c01e040 con 0x5645e63ea550 2024-03-19T01:11:03.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.390+0000 7fc551600640 1 -- 172.21.15.69:0/395778671 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc54c011070 con 0x5645e63ea550 2024-03-19T01:11:03.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.390+0000 7fc56d9b8740 1 -- 172.21.15.69:0/395778671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc55c02c8a0 con 0x5645e63ea550 2024-03-19T01:11:03.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.390+0000 7fc551600640 1 -- 172.21.15.69:0/395778671 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc54c028060 con 0x5645e63ea550 2024-03-19T01:11:03.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.390+0000 7fc56d9b8740 10 client.25859 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:03.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.390+0000 7fc56d9b8740 1 -- 172.21.15.69:0/395778671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5645e6473140 con 0x5645e63ea550 2024-03-19T01:11:03.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.391+0000 7fc551600640 1 -- 172.21.15.69:0/395778671 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc55c02c8a0 con 0x5645e63ea550 2024-03-19T01:11:03.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.391+0000 7fc551600640 1 -- 172.21.15.69:0/395778671 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc54c019030 con 0x5645e63ea550 2024-03-19T01:11:03.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.391+0000 7fc56d9b8740 10 client.25859 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:03.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.391+0000 7fc56d9b8740 10 client.25859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:03.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.391+0000 7fc56d9b8740 20 client.25859 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:03.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.391+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5645e63f3a50 0x5645e6475800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:03.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.391+0000 7fc56d9b8740 4 client.25859 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:11:03.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.391+0000 7fc56d9b8740 1 -- 172.21.15.69:0/395778671 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5645e6353800 con 0x5645e63f3a50 2024-03-19T01:11:03.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.391+0000 7fc568c00640 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5645e63f3a50 0x5645e6475800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:03.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.392+0000 7fc568c00640 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5645e63f3a50 0x5645e6475800 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1fa7ea081500d8 server_cookie=e492f611466a558 in_seq=0 out_seq=0 2024-03-19T01:11:03.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.392+0000 7fc551600640 10 client.25859 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:03.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.392+0000 7fc551600640 1 -- 172.21.15.69:0/395778671 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5645e6353800 con 0x5645e63f3a50 2024-03-19T01:11:03.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.392+0000 7fc551600640 10 client.25859 handle_command_reply: tid=0 2024-03-19T01:11:03.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.392+0000 7fc56d9b8740 1 client.25859 shutdown 2024-03-19T01:11:03.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc56d9b8740 1 -- 172.21.15.69:0/395778671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 msgr2=0x5645e63ed1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 0x5645e63ed1f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc54c000c00 tx=0x7fc54c02cd50 comp rx=0 tx=0).stop 2024-03-19T01:11:03.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc56d9b8740 1 -- 172.21.15.69:0/395778671 shutdown_connections 2024-03-19T01:11:03.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5645e63f3a50 0x5645e6475800 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc551600640 0 client.25859 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:03.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645e63eae70 0x5645e63ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645e63ea550 0x5645e63ed1f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc56d9b8740 1 --2- 172.21.15.69:0/395778671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5645e63deb80 0x5645e63fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc56d9b8740 1 -- 172.21.15.69:0/395778671 >> 172.21.15.69:0/395778671 conn(0x5645e63e4c80 msgr2=0x5645e63dae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:03.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc56d9b8740 1 -- 172.21.15.69:0/395778671 shutdown_connections 2024-03-19T01:11:03.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc56d9b8740 1 -- 172.21.15.69:0/395778671 wait complete. 2024-03-19T01:11:03.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.394+0000 7fc56d9b8740 20 client.25859 trim_cache size 0 max 16384 2024-03-19T01:11:03.396 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:11:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T01:11:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:11:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: "91ff3fbf-e734-4c75-8079-1d475afd55d4": { 2024-03-19T01:11:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:11:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "91ff3fbf-e734-4c75-8079-1d475afd55d4", 2024-03-19T01:11:03.397 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:11:03.397 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:11:03.397 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:11:03.397 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:11:03.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.395+0000 7fc56ac00640 1 -- 172.21.15.69:0/4111980027 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5480a2b90 msgr2=0x7fc5480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.395+0000 7fc56ac00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5480a2b90 0x7fc5480a5050 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc5540cc7c0 tx=0x7fc55400fe30 comp rx=0 tx=0).stop 2024-03-19T01:11:03.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.395+0000 7fc56ac00640 1 -- 172.21.15.69:0/4111980027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5641039c0 msgr2=0x7fc564072fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:03.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.395+0000 7fc56ac00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5641039c0 0x7fc564072fd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc55400c920 tx=0x7fc55400cd20 comp rx=0 tx=0).stop 2024-03-19T01:11:03.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.395+0000 7fc56ac00640 1 -- 172.21.15.69:0/4111980027 shutdown_connections 2024-03-19T01:11:03.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.395+0000 7fc56ac00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5641099d0 0x7fc56406c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.395+0000 7fc56ac00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc564104380 0x7fc564073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.395+0000 7fc56ac00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5480a2b90 0x7fc5480a5050 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.395+0000 7fc56ac00640 1 --2- 172.21.15.69:0/4111980027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5641039c0 0x7fc564072fd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:03.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.395+0000 7fc56ac00640 1 -- 172.21.15.69:0/4111980027 >> 172.21.15.69:0/4111980027 conn(0x7fc5640ff470 msgr2=0x7fc5640ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:03.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.396+0000 7fc56ac00640 1 -- 172.21.15.69:0/4111980027 shutdown_connections 2024-03-19T01:11:03.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:03.396+0000 7fc56ac00640 1 -- 172.21.15.69:0/4111980027 wait complete. 2024-03-19T01:11:03.419 INFO:tasks.cephfs.filesystem:scrub status for tag:91ff3fbf-e734-4c75-8079-1d475afd55d4 - {'path': '/', 'tag': '91ff3fbf-e734-4c75-8079-1d475afd55d4', 'options': 'recursive,force'} 2024-03-19T01:11:03.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:03 smithi086 ceph-mon[66829]: mgrmap e27: x(active, since 22s), standbys: y, z 2024-03-19T01:11:03.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:03 smithi163 ceph-mon[67428]: mgrmap e27: x(active, since 22s), standbys: y, z 2024-03-19T01:11:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:04 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:11:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:11:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:04 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:11:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:04 smithi086 ceph-mon[66829]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-19T01:11:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:04 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:11:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:11:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:04 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:11:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:04 smithi069 ceph-mon[31727]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-19T01:11:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:04 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:11:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:11:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:04 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:11:04.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:04 smithi163 ceph-mon[67428]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-19T01:11:06.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:06 smithi086 ceph-mon[66829]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-19T01:11:06.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:06 smithi069 ceph-mon[31727]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-19T01:11:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:06 smithi163 ceph-mon[67428]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-19T01:11:08.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:08 smithi086 ceph-mon[66829]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-19T01:11:08.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:08 smithi069 ceph-mon[31727]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-19T01:11:08.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:08 smithi163 ceph-mon[67428]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-19T01:11:10.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:10 smithi086 ceph-mon[66829]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-19T01:11:10.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:10 smithi069 ceph-mon[31727]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-19T01:11:10.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:10 smithi163 ceph-mon[67428]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-19T01:11:12.378 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:12 smithi163 ceph-mon[67428]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-19T01:11:12.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:12 smithi086 ceph-mon[66829]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-19T01:11:12.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:12 smithi069 ceph-mon[31727]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-19T01:11:14.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:14 smithi086 ceph-mon[66829]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-03-19T01:11:14.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:14 smithi069 ceph-mon[31727]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-03-19T01:11:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:14 smithi163 ceph-mon[67428]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-03-19T01:11:16.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:16 smithi086 ceph-mon[66829]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-19T01:11:16.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:16 smithi069 ceph-mon[31727]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-19T01:11:16.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:16 smithi163 ceph-mon[67428]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-19T01:11:18.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:18 smithi086 ceph-mon[66829]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-19T01:11:18.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:18 smithi069 ceph-mon[31727]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-19T01:11:18.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:18 smithi163 ceph-mon[67428]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-19T01:11:20.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:20 smithi086 ceph-mon[66829]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:11:20.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:20 smithi069 ceph-mon[31727]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:11:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:20 smithi163 ceph-mon[67428]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:11:22.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:22 smithi086 ceph-mon[66829]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-19T01:11:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:22 smithi069 ceph-mon[31727]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-19T01:11:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:22 smithi163 ceph-mon[67428]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-19T01:11:24.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:24 smithi086 ceph-mon[66829]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-19T01:11:24.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:24 smithi069 ceph-mon[31727]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-19T01:11:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:24 smithi163 ceph-mon[67428]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-19T01:11:26.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:26 smithi086 ceph-mon[66829]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-19T01:11:26.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:26 smithi069 ceph-mon[31727]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-19T01:11:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:26 smithi163 ceph-mon[67428]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-19T01:11:28.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:28 smithi069 ceph-mon[31727]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-19T01:11:28.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:28 smithi086 ceph-mon[66829]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-19T01:11:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:28 smithi163 ceph-mon[67428]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-19T01:11:30.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:30 smithi086 ceph-mon[66829]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-19T01:11:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:30 smithi069 ceph-mon[31727]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-19T01:11:30.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:30 smithi163 ceph-mon[67428]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-19T01:11:32.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:32 smithi086 ceph-mon[66829]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-19T01:11:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:32 smithi069 ceph-mon[31727]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-19T01:11:32.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:32 smithi163 ceph-mon[67428]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-19T01:11:33.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:11:33.420 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:11:33.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.670+0000 7f3499400640 1 -- 172.21.15.69:0/3565447875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3494103170 msgr2=0x7f349410b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.670+0000 7f3499400640 1 --2- 172.21.15.69:0/3565447875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3494103170 0x7f349410b650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3480009860 tx=0x7f348002bc10 comp rx=0 tx=0).stop 2024-03-19T01:11:33.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.670+0000 7f3499400640 1 -- 172.21.15.69:0/3565447875 shutdown_connections 2024-03-19T01:11:33.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.670+0000 7f3499400640 1 --2- 172.21.15.69:0/3565447875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349410bc70 0x7f349410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.670+0000 7f3499400640 1 --2- 172.21.15.69:0/3565447875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3494103170 0x7f349410b650 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.670+0000 7f3499400640 1 --2- 172.21.15.69:0/3565447875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3494102850 0x7f3494102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.670+0000 7f3499400640 1 -- 172.21.15.69:0/3565447875 >> 172.21.15.69:0/3565447875 conn(0x7f34940fc070 msgr2=0x7f34940fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.670+0000 7f3499400640 1 -- 172.21.15.69:0/3565447875 shutdown_connections 2024-03-19T01:11:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.670+0000 7f3499400640 1 -- 172.21.15.69:0/3565447875 wait complete. 2024-03-19T01:11:33.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.672+0000 7f3499400640 1 Processor -- start 2024-03-19T01:11:33.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.672+0000 7f3499400640 1 -- start start 2024-03-19T01:11:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.672+0000 7f3499400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3494102850 0x7f349419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.672+0000 7f3499400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3494103170 0x7f349419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.672+0000 7f3499400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349410bc70 0x7f349419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.672+0000 7f3499400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3494113cb0 con 0x7f3494102850 2024-03-19T01:11:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.672+0000 7f3499400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3494113b30 con 0x7f349410bc70 2024-03-19T01:11:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.672+0000 7f3499400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3494113e30 con 0x7f3494103170 2024-03-19T01:11:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.673+0000 7f3493400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3494103170 0x7f349419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.673+0000 7f3493400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3494103170 0x7f349419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46592/0 (socket says 172.21.15.69:46592) 2024-03-19T01:11:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.673+0000 7f3493400640 1 -- 172.21.15.69:0/231138819 learned_addr learned my addr 172.21.15.69:0/231138819 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.673+0000 7f3493e00640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3494102850 0x7f349419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.673+0000 7f3498a00640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349410bc70 0x7f349419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.673+0000 7f3493400640 1 -- 172.21.15.69:0/231138819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349410bc70 msgr2=0x7f349419ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.673+0000 7f3493400640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349410bc70 0x7f349419ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.673+0000 7f3493400640 1 -- 172.21.15.69:0/231138819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3494102850 msgr2=0x7f349419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.673+0000 7f3493400640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3494102850 0x7f349419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.673+0000 7f3493400640 1 -- 172.21.15.69:0/231138819 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349419f160 con 0x7f3494103170 2024-03-19T01:11:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.673+0000 7f3493e00640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3494102850 0x7f349419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:33.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.674+0000 7f3493400640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3494103170 0x7f349419a6b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3480037040 tx=0x7f34800344c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:33.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.674+0000 7f3490c00640 1 -- 172.21.15.69:0/231138819 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480041020 con 0x7f3494103170 2024-03-19T01:11:33.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.674+0000 7f3490c00640 1 -- 172.21.15.69:0/231138819 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f348002ca90 con 0x7f3494103170 2024-03-19T01:11:33.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.674+0000 7f3499400640 1 -- 172.21.15.69:0/231138819 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349419f3f0 con 0x7f3494103170 2024-03-19T01:11:33.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.674+0000 7f3490c00640 1 -- 172.21.15.69:0/231138819 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480039040 con 0x7f3494103170 2024-03-19T01:11:33.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.675+0000 7f3499400640 1 -- 172.21.15.69:0/231138819 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f349419f870 con 0x7f3494103170 2024-03-19T01:11:33.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.679+0000 7f349d66d740 1 Processor -- start 2024-03-19T01:11:33.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.679+0000 7f349d66d740 1 -- start start 2024-03-19T01:11:33.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.680+0000 7f349d66d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab469310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.680+0000 7f349d66d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab469850 0x55afab471ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.680+0000 7f3490c00640 1 -- 172.21.15.69:0/231138819 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7f3480044020 con 0x7f3494103170 2024-03-19T01:11:33.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.680+0000 7f349d66d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab472430 0x55afab474820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.680+0000 7f349d66d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55afab3cd2c0 con 0x55afab469850 2024-03-19T01:11:33.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.680+0000 7f349d66d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afab375b60 con 0x55afab468f30 2024-03-19T01:11:33.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.680+0000 7f349d66d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55afab3c9360 con 0x55afab472430 2024-03-19T01:11:33.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.680+0000 7f3498a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab469310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.680+0000 7f3498a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab469310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52274/0 (socket says 172.21.15.69:52274) 2024-03-19T01:11:33.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.680+0000 7f3498a00640 1 -- 172.21.15.69:0/2404007300 learned_addr learned my addr 172.21.15.69:0/2404007300 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:33.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.680+0000 7f3493e00640 1 --2- 172.21.15.69:0/2404007300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab469850 0x55afab471ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.681+0000 7f3498a00640 1 -- 172.21.15.69:0/2404007300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab472430 msgr2=0x55afab474820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:11:33.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.681+0000 7f3498a00640 1 --2- 172.21.15.69:0/2404007300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab472430 0x55afab474820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.681+0000 7f3498a00640 1 -- 172.21.15.69:0/2404007300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab469850 msgr2=0x55afab471ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:33.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.681+0000 7f3498a00640 1 --2- 172.21.15.69:0/2404007300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab469850 0x55afab471ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.681+0000 7f3498a00640 1 -- 172.21.15.69:0/2404007300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afab1ab7d0 con 0x55afab468f30 2024-03-19T01:11:33.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.681+0000 7f3493e00640 1 --2- 172.21.15.69:0/2404007300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab469850 0x55afab471ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:33.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.681+0000 7f3498a00640 1 --2- 172.21.15.69:0/2404007300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab469310 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f348c009e30 tx=0x7f348c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da80f549a6cf3603 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:33.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.681+0000 7f3490c00640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f34780a2b90 0x7f34780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.682+0000 7f3489200640 1 -- 172.21.15.69:0/2404007300 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c011640 con 0x55afab468f30 2024-03-19T01:11:33.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.682+0000 7f3489200640 1 -- 172.21.15.69:0/2404007300 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f348c011cc0 con 0x55afab468f30 2024-03-19T01:11:33.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.682+0000 7f3489200640 1 -- 172.21.15.69:0/2404007300 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c01bd70 con 0x55afab468f30 2024-03-19T01:11:33.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.682+0000 7f3490c00640 1 -- 172.21.15.69:0/231138819 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f34800f0b40 con 0x7f3494103170 2024-03-19T01:11:33.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.682+0000 7f3493e00640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f34780a2b90 0x7f34780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.683+0000 7f349d66d740 1 -- 172.21.15.69:0/2404007300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 msgr2=0x55afab469310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.683+0000 7f349d66d740 1 --2- 172.21.15.69:0/2404007300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab469310 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f348c009e30 tx=0x7f348c00f730 comp rx=0 tx=0).stop 2024-03-19T01:11:33.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.683+0000 7f349d66d740 1 -- 172.21.15.69:0/2404007300 shutdown_connections 2024-03-19T01:11:33.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.683+0000 7f349d66d740 1 --2- 172.21.15.69:0/2404007300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab472430 0x55afab474820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.683+0000 7f349d66d740 1 --2- 172.21.15.69:0/2404007300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab469850 0x55afab471ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.683+0000 7f349d66d740 1 --2- 172.21.15.69:0/2404007300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab469310 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.683+0000 7f349d66d740 1 -- 172.21.15.69:0/2404007300 >> 172.21.15.69:0/2404007300 conn(0x55afab463660 msgr2=0x55afab463a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:33.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.683+0000 7f3493e00640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f34780a2b90 0x7f34780a5050 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3484005ab0 tx=0x7f3484009260 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:33.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.683+0000 7f349d66d740 1 -- 172.21.15.69:0/2404007300 shutdown_connections 2024-03-19T01:11:33.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.683+0000 7f349d66d740 1 -- 172.21.15.69:0/2404007300 wait complete. 2024-03-19T01:11:33.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.684+0000 7f349d66d740 1 -- start start 2024-03-19T01:11:33.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.684+0000 7f349d66d740 1 Processor -- start 2024-03-19T01:11:33.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f349d66d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab46f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f349d66d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f349d66d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab472430 0x55afab470020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f349d66d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55afab486230 con 0x55afab472430 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f349d66d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afab388fc0 con 0x55afab468f30 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f349d66d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55afab4863b0 con 0x55afab469850 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f3493e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab46f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f3493400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f3493400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46608/0 (socket says 172.21.15.69:46608) 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f3493400640 1 -- 172.21.15.69:0/2036199672 learned_addr learned my addr 172.21.15.69:0/2036199672 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f349d66d740 10 client.0 _ll_register_callbacks cb 0x55afab388990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f3493400640 1 -- 172.21.15.69:0/2036199672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 msgr2=0x55afab46f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f3493400640 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab46f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f3493400640 1 -- 172.21.15.69:0/2036199672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab472430 msgr2=0x55afab470020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.685+0000 7f3498a00640 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab472430 0x55afab470020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.686+0000 7f3493400640 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab472430 0x55afab470020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.686+0000 7f3493400640 1 -- 172.21.15.69:0/2036199672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afab0e13f0 con 0x55afab469850 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.686+0000 7f3498a00640 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab472430 0x55afab470020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.686+0000 7f3493400640 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46fae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f348002c220 tx=0x7f34800bc780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b07f8e354aa73e33 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.686+0000 7f347d600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.686+0000 7f347d600640 1 -- 172.21.15.69:0/2036199672 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800bcc40 con 0x55afab469850 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.686+0000 7f349d66d740 1 -- 172.21.15.69:0/2036199672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f348c011cc0 con 0x55afab469850 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.686+0000 7f347d600640 1 -- 172.21.15.69:0/2036199672 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f34800bcea0 con 0x55afab469850 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.686+0000 7f347d600640 1 -- 172.21.15.69:0/2036199672 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f34800c0d90 con 0x55afab469850 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.686+0000 7f349d66d740 10 client.25864 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.686+0000 7f349d66d740 1 -- 172.21.15.69:0/2036199672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55afab0a10f0 con 0x55afab469850 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.687+0000 7f347d600640 1 -- 172.21.15.69:0/2036199672 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f348c011cc0 con 0x55afab469850 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.687+0000 7f347d600640 1 -- 172.21.15.69:0/2036199672 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f348003c030 con 0x55afab469850 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.687+0000 7f349d66d740 10 client.25864 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.687+0000 7f349d66d740 10 client.25864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.687+0000 7f349d66d740 20 client.25864 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.687+0000 7f349d66d740 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afab4937d0 0x55afab495c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.687+0000 7f349d66d740 4 client.25864 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.687+0000 7f349d66d740 1 -- 172.21.15.69:0/2036199672 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afab36e610 con 0x55afab4937d0 2024-03-19T01:11:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.688+0000 7f3493e00640 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afab4937d0 0x55afab495c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.688+0000 7f3493e00640 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afab4937d0 0x55afab495c00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d90c1dbdf0c600b server_cookie=914eafa944a4e417 in_seq=0 out_seq=0 2024-03-19T01:11:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.688+0000 7f347d600640 10 client.25864 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f347d600640 1 -- 172.21.15.69:0/2036199672 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55afab36e610 con 0x55afab4937d0 2024-03-19T01:11:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f347d600640 10 client.25864 handle_command_reply: tid=0 2024-03-19T01:11:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 1 client.25864 shutdown 2024-03-19T01:11:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 1 -- 172.21.15.69:0/2036199672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 msgr2=0x55afab46fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46fae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f348002c220 tx=0x7f34800bc780 comp rx=0 tx=0).stop 2024-03-19T01:11:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 1 -- 172.21.15.69:0/2036199672 shutdown_connections 2024-03-19T01:11:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afab4937d0 0x55afab495c00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab472430 0x55afab470020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46fae0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 1 --2- 172.21.15.69:0/2036199672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab46f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 1 -- 172.21.15.69:0/2036199672 >> 172.21.15.69:0/2036199672 conn(0x55afab463660 msgr2=0x55afab4653b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:33.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f347d600640 0 client.25864 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:33.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 1 -- 172.21.15.69:0/2036199672 shutdown_connections 2024-03-19T01:11:33.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 1 -- 172.21.15.69:0/2036199672 wait complete. 2024-03-19T01:11:33.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.689+0000 7f349d66d740 20 client.25864 trim_cache size 0 max 16384 2024-03-19T01:11:33.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.708+0000 7f349d66d740 1 Processor -- start 2024-03-19T01:11:33.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.708+0000 7f349d66d740 1 -- start start 2024-03-19T01:11:33.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.708+0000 7f349d66d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab46e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.708+0000 7f349d66d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab48efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.708+0000 7f349d66d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab472430 0x55afab46fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.708+0000 7f349d66d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f34941a8450 con 0x55afab472430 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.708+0000 7f349d66d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34941a9f90 con 0x55afab468f30 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.708+0000 7f349d66d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f34941a8b20 con 0x55afab469850 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.708+0000 7f3493e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab46e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.708+0000 7f3493e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab46e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52288/0 (socket says 172.21.15.69:52288) 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.708+0000 7f3493e00640 1 -- 172.21.15.69:0/2318201162 learned_addr learned my addr 172.21.15.69:0/2318201162 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.709+0000 7f3493400640 1 --2- 172.21.15.69:0/2318201162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab48efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.709+0000 7f3498a00640 1 --2- 172.21.15.69:0/2318201162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab472430 0x55afab46fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.709+0000 7f3493400640 1 -- 172.21.15.69:0/2318201162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 msgr2=0x55afab46e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.709+0000 7f3493400640 1 --2- 172.21.15.69:0/2318201162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab46e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.709+0000 7f3493400640 1 -- 172.21.15.69:0/2318201162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab472430 msgr2=0x55afab46fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.709+0000 7f3493400640 1 --2- 172.21.15.69:0/2318201162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab472430 0x55afab46fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.709+0000 7f3493400640 1 -- 172.21.15.69:0/2318201162 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afab4900c0 con 0x55afab469850 2024-03-19T01:11:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.709+0000 7f3493400640 1 --2- 172.21.15.69:0/2318201162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab48efd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f34800b9610 tx=0x7f34800bb150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8c26eaa06d8598f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:33.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.709+0000 7f347fe00640 1 -- 172.21.15.69:0/2318201162 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800bcc40 con 0x55afab469850 2024-03-19T01:11:33.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.710+0000 7f347fe00640 1 -- 172.21.15.69:0/2318201162 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34800a7040 con 0x55afab469850 2024-03-19T01:11:33.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.710+0000 7f347fe00640 1 -- 172.21.15.69:0/2318201162 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800c4f50 con 0x55afab469850 2024-03-19T01:11:33.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.710+0000 7f349d66d740 1 -- 172.21.15.69:0/2318201162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 msgr2=0x55afab48efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.710+0000 7f349d66d740 1 --2- 172.21.15.69:0/2318201162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab48efd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f34800b9610 tx=0x7f34800bb150 comp rx=0 tx=0).stop 2024-03-19T01:11:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.710+0000 7f349d66d740 1 -- 172.21.15.69:0/2318201162 shutdown_connections 2024-03-19T01:11:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.710+0000 7f349d66d740 1 --2- 172.21.15.69:0/2318201162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab472430 0x55afab46fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.710+0000 7f349d66d740 1 --2- 172.21.15.69:0/2318201162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab48efd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.710+0000 7f349d66d740 1 --2- 172.21.15.69:0/2318201162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab468f30 0x55afab46e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.710+0000 7f349d66d740 1 -- 172.21.15.69:0/2318201162 >> 172.21.15.69:0/2318201162 conn(0x55afab463660 msgr2=0x55afab465340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.711+0000 7f349d66d740 1 -- 172.21.15.69:0/2318201162 shutdown_connections 2024-03-19T01:11:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.711+0000 7f349d66d740 1 -- 172.21.15.69:0/2318201162 wait complete. 2024-03-19T01:11:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.711+0000 7f349d66d740 1 -- start start 2024-03-19T01:11:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.712+0000 7f349d66d740 1 Processor -- start 2024-03-19T01:11:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.712+0000 7f349d66d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab468f30 0x55afab46c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.712+0000 7f349d66d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.712+0000 7f349d66d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab472430 0x55afab46cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.712+0000 7f349d66d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f34941a98c0 con 0x55afab468f30 2024-03-19T01:11:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.712+0000 7f349d66d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34941a91f0 con 0x55afab472430 2024-03-19T01:11:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.712+0000 7f349d66d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3494073070 con 0x55afab469850 2024-03-19T01:11:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.712+0000 7f349d66d740 10 client.0 _ll_register_callbacks cb 0x7f34800b8ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.712+0000 7f3493400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.712+0000 7f3493400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46614/0 (socket says 172.21.15.69:46614) 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.712+0000 7f3493400640 1 -- 172.21.15.69:0/3514728855 learned_addr learned my addr 172.21.15.69:0/3514728855 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.713+0000 7f3498a00640 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab472430 0x55afab46cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.713+0000 7f3493e00640 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab468f30 0x55afab46c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.713+0000 7f3493400640 1 -- 172.21.15.69:0/3514728855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab472430 msgr2=0x55afab46cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.713+0000 7f3493400640 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab472430 0x55afab46cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.713+0000 7f3493400640 1 -- 172.21.15.69:0/3514728855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab468f30 msgr2=0x55afab46c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.713+0000 7f3493400640 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab468f30 0x55afab46c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.713+0000 7f3493400640 1 -- 172.21.15.69:0/3514728855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afab501350 con 0x55afab469850 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.713+0000 7f3493400640 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46c650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f348003d040 tx=0x7f3480045000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1236c72fcbc37836 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.713+0000 7f347d600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.713+0000 7f347d600640 1 -- 172.21.15.69:0/3514728855 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34800da070 con 0x55afab469850 2024-03-19T01:11:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.713+0000 7f347d600640 1 -- 172.21.15.69:0/3514728855 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f34800c4f50 con 0x55afab469850 2024-03-19T01:11:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f349d66d740 1 -- 172.21.15.69:0/3514728855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34800a7040 con 0x55afab469850 2024-03-19T01:11:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f347d600640 1 -- 172.21.15.69:0/3514728855 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f34800bcc40 con 0x55afab469850 2024-03-19T01:11:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f349d66d740 10 client.25870 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f349d66d740 1 -- 172.21.15.69:0/3514728855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55afab501350 con 0x55afab469850 2024-03-19T01:11:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f347d600640 1 -- 172.21.15.69:0/3514728855 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34800a7040 con 0x55afab469850 2024-03-19T01:11:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f347d600640 1 -- 172.21.15.69:0/3514728855 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f34800db340 con 0x55afab469850 2024-03-19T01:11:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f349d66d740 10 client.25870 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f349d66d740 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f349d66d740 20 client.25870 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f349d66d740 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afab502830 0x55afab504c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f349d66d740 4 client.25870 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:11:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.714+0000 7f349d66d740 1 -- 172.21.15.69:0/3514728855 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55afab3da800 con 0x55afab502830 2024-03-19T01:11:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.715+0000 7f3493e00640 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afab502830 0x55afab504c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.715+0000 7f3493e00640 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afab502830 0x55afab504c50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d69d0d646ac792 server_cookie=ac31ac19b16f89b0 in_seq=0 out_seq=0 2024-03-19T01:11:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.715+0000 7f347d600640 10 client.25870 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.715+0000 7f347d600640 1 -- 172.21.15.69:0/3514728855 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55afab3da800 con 0x55afab502830 2024-03-19T01:11:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.715+0000 7f347d600640 10 client.25870 handle_command_reply: tid=0 2024-03-19T01:11:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.715+0000 7f349d66d740 1 client.25870 shutdown 2024-03-19T01:11:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.715+0000 7f349d66d740 1 -- 172.21.15.69:0/3514728855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 msgr2=0x55afab46c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.715+0000 7f349d66d740 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46c650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f348003d040 tx=0x7f3480045000 comp rx=0 tx=0).stop 2024-03-19T01:11:33.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.716+0000 7f349d66d740 1 -- 172.21.15.69:0/3514728855 shutdown_connections 2024-03-19T01:11:33.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.716+0000 7f349d66d740 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afab502830 0x55afab504c50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.716+0000 7f349d66d740 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afab472430 0x55afab46cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.716+0000 7f349d66d740 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afab469850 0x55afab46c650 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.716+0000 7f347d600640 0 client.25870 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:33.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.716+0000 7f349d66d740 1 --2- 172.21.15.69:0/3514728855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afab468f30 0x55afab46c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.716+0000 7f349d66d740 1 -- 172.21.15.69:0/3514728855 >> 172.21.15.69:0/3514728855 conn(0x55afab463660 msgr2=0x55afab461e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:33.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.716+0000 7f349d66d740 1 -- 172.21.15.69:0/3514728855 shutdown_connections 2024-03-19T01:11:33.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.717+0000 7f349d66d740 1 -- 172.21.15.69:0/3514728855 wait complete. 2024-03-19T01:11:33.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.717+0000 7f349d66d740 20 client.25870 trim_cache size 0 max 16384 2024-03-19T01:11:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:11:33.721 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:11:33.721 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:11:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:11:33.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.718+0000 7f3499400640 1 -- 172.21.15.69:0/231138819 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f34780a2b90 msgr2=0x7f34780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.718+0000 7f3499400640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f34780a2b90 0x7f34780a5050 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3484005ab0 tx=0x7f3484009260 comp rx=0 tx=0).stop 2024-03-19T01:11:33.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.718+0000 7f3499400640 1 -- 172.21.15.69:0/231138819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3494103170 msgr2=0x7f349419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.718+0000 7f3499400640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3494103170 0x7f349419a6b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3480037040 tx=0x7f34800344c0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.718+0000 7f3499400640 1 -- 172.21.15.69:0/231138819 shutdown_connections 2024-03-19T01:11:33.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.718+0000 7f3499400640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f34780a2b90 0x7f34780a5050 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.718+0000 7f3499400640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f349410bc70 0x7f349419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.718+0000 7f3499400640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3494103170 0x7f349419a6b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.718+0000 7f3499400640 1 --2- 172.21.15.69:0/231138819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3494102850 0x7f349419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.718+0000 7f3499400640 1 -- 172.21.15.69:0/231138819 >> 172.21.15.69:0/231138819 conn(0x7f34940fc070 msgr2=0x7f34940fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:33.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.719+0000 7f3499400640 1 -- 172.21.15.69:0/231138819 shutdown_connections 2024-03-19T01:11:33.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.719+0000 7f3499400640 1 -- 172.21.15.69:0/231138819 wait complete. 2024-03-19T01:11:33.744 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:11:33.745 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:11:33.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.982+0000 7ff4efa00640 1 -- 172.21.15.69:0/3239093275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e8100610 msgr2=0x7ff4e81009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.982+0000 7ff4efa00640 1 --2- 172.21.15.69:0/3239093275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e8100610 0x7ff4e81009f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009900 tx=0x7ff4dc02bc30 comp rx=0 tx=0).stop 2024-03-19T01:11:33.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.983+0000 7ff4efa00640 1 -- 172.21.15.69:0/3239093275 shutdown_connections 2024-03-19T01:11:33.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.983+0000 7ff4efa00640 1 --2- 172.21.15.69:0/3239093275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e810e1c0 0x7ff4e81105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.983+0000 7ff4efa00640 1 --2- 172.21.15.69:0/3239093275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e8100fc0 0x7ff4e810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.983+0000 7ff4efa00640 1 --2- 172.21.15.69:0/3239093275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e8100610 0x7ff4e81009f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.983+0000 7ff4efa00640 1 -- 172.21.15.69:0/3239093275 >> 172.21.15.69:0/3239093275 conn(0x7ff4e8077820 msgr2=0x7ff4e8077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:33.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.983+0000 7ff4efa00640 1 -- 172.21.15.69:0/3239093275 shutdown_connections 2024-03-19T01:11:33.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.983+0000 7ff4efa00640 1 -- 172.21.15.69:0/3239093275 wait complete. 2024-03-19T01:11:33.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4efa00640 1 Processor -- start 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4efa00640 1 -- start start 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4efa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e8100610 0x7ff4e819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4efa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e8100fc0 0x7ff4e819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4efa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e810e1c0 0x7ff4e819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4efa00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e8113e90 con 0x7ff4e810e1c0 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4efa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e8113d10 con 0x7ff4e8100610 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4efa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e8114010 con 0x7ff4e8100fc0 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4ed000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e8100610 0x7ff4e819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4ed000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e8100610 0x7ff4e819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52308/0 (socket says 172.21.15.69:52308) 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4ed000640 1 -- 172.21.15.69:0/209735826 learned_addr learned my addr 172.21.15.69:0/209735826 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.984+0000 7ff4e7e00640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e8100fc0 0x7ff4e819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.985+0000 7ff4eda00640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e810e1c0 0x7ff4e819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.985+0000 7ff4ed000640 1 -- 172.21.15.69:0/209735826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e8100fc0 msgr2=0x7ff4e819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.985+0000 7ff4ed000640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e8100fc0 0x7ff4e819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.985+0000 7ff4ed000640 1 -- 172.21.15.69:0/209735826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e810e1c0 msgr2=0x7ff4e819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:33.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.985+0000 7ff4ed000640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e810e1c0 0x7ff4e819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:33.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.985+0000 7ff4ed000640 1 -- 172.21.15.69:0/209735826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e819f300 con 0x7ff4e8100610 2024-03-19T01:11:33.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.985+0000 7ff4eda00640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e810e1c0 0x7ff4e819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:33.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.985+0000 7ff4ed000640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e8100610 0x7ff4e819a310 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc02c240 tx=0x7ff4dc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:33.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.986+0000 7ff4e5600640 1 -- 172.21.15.69:0/209735826 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc043020 con 0x7ff4e8100610 2024-03-19T01:11:33.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.986+0000 7ff4e5600640 1 -- 172.21.15.69:0/209735826 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc0054e0 con 0x7ff4e8100610 2024-03-19T01:11:33.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.986+0000 7ff4e5600640 1 -- 172.21.15.69:0/209735826 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc003b20 con 0x7ff4e8100610 2024-03-19T01:11:33.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.987+0000 7ff4efa00640 1 -- 172.21.15.69:0/209735826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e819f590 con 0x7ff4e8100610 2024-03-19T01:11:33.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.987+0000 7ff4efa00640 1 -- 172.21.15.69:0/209735826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e81a6e80 con 0x7ff4e8100610 2024-03-19T01:11:33.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.988+0000 7ff4efa00640 1 -- 172.21.15.69:0/209735826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e80691c0 con 0x7ff4e8100610 2024-03-19T01:11:33.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.991+0000 7ff4e5600640 1 -- 172.21.15.69:0/209735826 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7ff4dc046020 con 0x7ff4e8100610 2024-03-19T01:11:33.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.992+0000 7ff4e5600640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff4d00a2b10 0x7ff4d00a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:33.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.992+0000 7ff4e5600640 1 -- 172.21.15.69:0/209735826 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7ff4dc0f14d0 con 0x7ff4e8100610 2024-03-19T01:11:33.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.993+0000 7ff4e7e00640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff4d00a2b10 0x7ff4d00a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:33.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.993+0000 7ff4e5600640 1 -- 172.21.15.69:0/209735826 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff4dc04d070 con 0x7ff4e8100610 2024-03-19T01:11:33.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:33.993+0000 7ff4e7e00640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff4d00a2b10 0x7ff4d00a4fd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40095f0 tx=0x7ff4d4005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:34 smithi069 ceph-mon[31727]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:11:34.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.307+0000 7ff4efa00640 1 -- 172.21.15.69:0/209735826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4e8068610 con 0x7ff4e8100610 2024-03-19T01:11:34.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.308+0000 7ff4e5600640 1 -- 172.21.15.69:0/209735826 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11930 (secure 0 0 0) 0x7ff4dc0bde50 con 0x7ff4e8100610 2024-03-19T01:11:34.309 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:11:34.309 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:10:17.373602+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24550,24494,24580,24581,24476]},"id":1}]} 2024-03-19T01:11:34.309 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 29 2024-03-19T01:11:34.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 -- 172.21.15.69:0/209735826 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff4d00a2b10 msgr2=0x7ff4d00a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff4d00a2b10 0x7ff4d00a4fd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40095f0 tx=0x7ff4d4005f90 comp rx=0 tx=0).stop 2024-03-19T01:11:34.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 -- 172.21.15.69:0/209735826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e8100610 msgr2=0x7ff4e819a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e8100610 0x7ff4e819a310 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc02c240 tx=0x7ff4dc0042f0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 -- 172.21.15.69:0/209735826 shutdown_connections 2024-03-19T01:11:34.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4e810e1c0 0x7ff4e819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e8100fc0 0x7ff4e819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff4d00a2b10 0x7ff4d00a4fd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 --2- 172.21.15.69:0/209735826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e8100610 0x7ff4e819a310 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 -- 172.21.15.69:0/209735826 >> 172.21.15.69:0/209735826 conn(0x7ff4e8077820 msgr2=0x7ff4e810f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 -- 172.21.15.69:0/209735826 shutdown_connections 2024-03-19T01:11:34.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.312+0000 7ff4efa00640 1 -- 172.21.15.69:0/209735826 wait complete. 2024-03-19T01:11:34.336 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:11:34.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:34 smithi086 ceph-mon[66829]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:11:34.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:34 smithi163 ceph-mon[67428]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:11:34.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.572+0000 7fee56800640 1 -- 172.21.15.69:0/2363369280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee501099c0 msgr2=0x7fee50109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.572+0000 7fee56800640 1 --2- 172.21.15.69:0/2363369280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee501099c0 0x7fee50109da0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fee40009930 tx=0x7fee4002bc20 comp rx=0 tx=0).stop 2024-03-19T01:11:34.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.572+0000 7fee56800640 1 -- 172.21.15.69:0/2363369280 shutdown_connections 2024-03-19T01:11:34.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.572+0000 7fee56800640 1 --2- 172.21.15.69:0/2363369280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee50104370 0x7fee50110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.572+0000 7fee56800640 1 --2- 172.21.15.69:0/2363369280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee501039b0 0x7fee50103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.572+0000 7fee56800640 1 --2- 172.21.15.69:0/2363369280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee501099c0 0x7fee50109da0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.572+0000 7fee56800640 1 -- 172.21.15.69:0/2363369280 >> 172.21.15.69:0/2363369280 conn(0x7fee500ff4a0 msgr2=0x7fee501018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.572+0000 7fee56800640 1 -- 172.21.15.69:0/2363369280 shutdown_connections 2024-03-19T01:11:34.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.572+0000 7fee56800640 1 -- 172.21.15.69:0/2363369280 wait complete. 2024-03-19T01:11:34.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.573+0000 7fee56800640 1 Processor -- start 2024-03-19T01:11:34.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.573+0000 7fee56800640 1 -- start start 2024-03-19T01:11:34.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.573+0000 7fee56800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee501039b0 0x7fee5019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.573+0000 7fee56800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee50104370 0x7fee5019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.573+0000 7fee56800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee501099c0 0x7fee5019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.573+0000 7fee56800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee50113e20 con 0x7fee50104370 2024-03-19T01:11:34.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.573+0000 7fee56800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee50113ca0 con 0x7fee501099c0 2024-03-19T01:11:34.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.573+0000 7fee56800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fee50113fa0 con 0x7fee501039b0 2024-03-19T01:11:34.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.574+0000 7fee4f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee501039b0 0x7fee5019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.574+0000 7fee4f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee501039b0 0x7fee5019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46642/0 (socket says 172.21.15.69:46642) 2024-03-19T01:11:34.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.574+0000 7fee4f400640 1 -- 172.21.15.69:0/4179355078 learned_addr learned my addr 172.21.15.69:0/4179355078 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:34.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.574+0000 7fee4fe00640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee501099c0 0x7fee5019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.574+0000 7fee4ea00640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee50104370 0x7fee5019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.574+0000 7fee4f400640 1 -- 172.21.15.69:0/4179355078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee501099c0 msgr2=0x7fee5019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.574+0000 7fee4f400640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee501099c0 0x7fee5019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.574+0000 7fee4f400640 1 -- 172.21.15.69:0/4179355078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee50104370 msgr2=0x7fee5019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.574+0000 7fee4f400640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee50104370 0x7fee5019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.574+0000 7fee4f400640 1 -- 172.21.15.69:0/4179355078 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5019f2b0 con 0x7fee501039b0 2024-03-19T01:11:34.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.574+0000 7fee4f400640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee501039b0 0x7fee5019a2c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fee40005ab0 tx=0x7fee400042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.575+0000 7fee47e00640 1 -- 172.21.15.69:0/4179355078 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40043020 con 0x7fee501039b0 2024-03-19T01:11:34.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.575+0000 7fee47e00640 1 -- 172.21.15.69:0/4179355078 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee400044d0 con 0x7fee501039b0 2024-03-19T01:11:34.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.575+0000 7fee56800640 1 -- 172.21.15.69:0/4179355078 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5019f540 con 0x7fee501039b0 2024-03-19T01:11:34.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.575+0000 7fee47e00640 1 -- 172.21.15.69:0/4179355078 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40003bb0 con 0x7fee501039b0 2024-03-19T01:11:34.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.575+0000 7fee56800640 1 -- 172.21.15.69:0/4179355078 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee501a6e30 con 0x7fee501039b0 2024-03-19T01:11:34.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.578+0000 7fee59405740 1 Processor -- start 2024-03-19T01:11:34.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.578+0000 7fee59405740 1 -- start start 2024-03-19T01:11:34.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.578+0000 7fee59405740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x555700814610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee59405740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555700814b50 0x55570081d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee59405740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570081d730 0x55570081fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee59405740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555700770380 con 0x555700814b50 2024-03-19T01:11:34.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee59405740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555700718d20 con 0x55570081d730 2024-03-19T01:11:34.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee59405740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55570076c420 con 0x555700814230 2024-03-19T01:11:34.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee4fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x555700814610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee4fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x555700814610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46644/0 (socket says 172.21.15.69:46644) 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee4fe00640 1 -- 172.21.15.69:0/3092645980 learned_addr learned my addr 172.21.15.69:0/3092645980 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee4ea00640 1 --2- 172.21.15.69:0/3092645980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570081d730 0x55570081fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee4fe00640 1 -- 172.21.15.69:0/3092645980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570081d730 msgr2=0x55570081fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee4fe00640 1 --2- 172.21.15.69:0/3092645980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570081d730 0x55570081fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee4fe00640 1 -- 172.21.15.69:0/3092645980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555700814b50 msgr2=0x55570081d1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee4fe00640 1 --2- 172.21.15.69:0/3092645980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555700814b50 0x55570081d1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee4fe00640 1 -- 172.21.15.69:0/3092645980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557004843f0 con 0x555700814230 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.579+0000 7fee47e00640 1 -- 172.21.15.69:0/4179355078 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7fee40046020 con 0x7fee501039b0 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.580+0000 7fee4fe00640 1 --2- 172.21.15.69:0/3092645980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x555700814610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fee48009f50 tx=0x7fee480118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=258356530d2416c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.580+0000 7fee44c00640 1 -- 172.21.15.69:0/3092645980 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48004d20 con 0x555700814230 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.580+0000 7fee44c00640 1 -- 172.21.15.69:0/3092645980 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee48004050 con 0x555700814230 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.580+0000 7fee44c00640 1 -- 172.21.15.69:0/3092645980 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48005280 con 0x555700814230 2024-03-19T01:11:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee59405740 1 -- 172.21.15.69:0/3092645980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 msgr2=0x555700814610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee59405740 1 --2- 172.21.15.69:0/3092645980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x555700814610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fee48009f50 tx=0x7fee480118d0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee47e00640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fee340a2b10 0x7fee340a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee47e00640 1 -- 172.21.15.69:0/4179355078 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fee400f13e0 con 0x7fee501039b0 2024-03-19T01:11:34.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee59405740 1 -- 172.21.15.69:0/3092645980 shutdown_connections 2024-03-19T01:11:34.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee59405740 1 --2- 172.21.15.69:0/3092645980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570081d730 0x55570081fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee59405740 1 --2- 172.21.15.69:0/3092645980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555700814b50 0x55570081d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee59405740 1 --2- 172.21.15.69:0/3092645980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x555700814610 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee59405740 1 -- 172.21.15.69:0/3092645980 >> 172.21.15.69:0/3092645980 conn(0x55570080e960 msgr2=0x55570080ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee59405740 1 -- 172.21.15.69:0/3092645980 shutdown_connections 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee4fe00640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fee340a2b10 0x7fee340a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.581+0000 7fee59405740 1 -- 172.21.15.69:0/3092645980 wait complete. 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.582+0000 7fee59405740 1 -- start start 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.582+0000 7fee4fe00640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fee340a2b10 0x7fee340a4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fee48009d60 tx=0x7fee48004700 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.582+0000 7fee59405740 1 Processor -- start 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee59405740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814230 0x55570081a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee59405740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555700814b50 0x55570081ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee59405740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570081d730 0x55570081b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee59405740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555700820520 con 0x555700814b50 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee59405740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55570072c160 con 0x555700814230 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee59405740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5557008206a0 con 0x55570081d730 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee59405740 10 client.0 _ll_register_callbacks cb 0x555700768170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee4f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814230 0x55570081a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee4f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814230 0x55570081a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52338/0 (socket says 172.21.15.69:52338) 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee4f400640 1 -- 172.21.15.69:0/1758777209 learned_addr learned my addr 172.21.15.69:0/1758777209 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee4fe00640 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570081d730 0x55570081b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.583+0000 7fee4ea00640 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555700814b50 0x55570081ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee4f400640 1 -- 172.21.15.69:0/1758777209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570081d730 msgr2=0x55570081b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee4f400640 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570081d730 0x55570081b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee4f400640 1 -- 172.21.15.69:0/1758777209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555700814b50 msgr2=0x55570081ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee4f400640 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555700814b50 0x55570081ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee4f400640 1 -- 172.21.15.69:0/1758777209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570054e7d0 con 0x555700814230 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee4ea00640 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555700814b50 0x55570081ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee4f400640 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814230 0x55570081a820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fee4002c160 tx=0x7fee400bd370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c84bc650ac87fe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee39600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee39600640 1 -- 172.21.15.69:0/1758777209 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee400eee60 con 0x555700814230 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee59405740 1 -- 172.21.15.69:0/1758777209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee48004050 con 0x555700814230 2024-03-19T01:11:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee39600640 1 -- 172.21.15.69:0/1758777209 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fee4002cb80 con 0x555700814230 2024-03-19T01:11:34.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee39600640 1 -- 172.21.15.69:0/1758777209 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee400bd4e0 con 0x555700814230 2024-03-19T01:11:34.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee59405740 10 client.25892 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:34.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.584+0000 7fee59405740 1 -- 172.21.15.69:0/1758777209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55570082c880 con 0x555700814230 2024-03-19T01:11:34.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.585+0000 7fee39600640 1 -- 172.21.15.69:0/1758777209 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee48004050 con 0x555700814230 2024-03-19T01:11:34.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.585+0000 7fee39600640 1 -- 172.21.15.69:0/1758777209 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee400c57f0 con 0x555700814230 2024-03-19T01:11:34.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.585+0000 7fee59405740 10 client.25892 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:34.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.585+0000 7fee59405740 10 client.25892 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:11:34.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.585+0000 7fee59405740 20 client.25892 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:34.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.585+0000 7fee59405740 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55570082dc20 0x5557008300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.585+0000 7fee59405740 4 client.25892 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:34.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.585+0000 7fee59405740 1 -- 172.21.15.69:0/1758777209 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55570077d8c0 con 0x55570082dc20 2024-03-19T01:11:34.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.585+0000 7fee4ea00640 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55570082dc20 0x5557008300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.586+0000 7fee4ea00640 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55570082dc20 0x5557008300c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=188d4dff67c2a531 server_cookie=b552feace252ec31 in_seq=0 out_seq=0 2024-03-19T01:11:34.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.586+0000 7fee39600640 10 client.25892 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:11:34.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee39600640 1 -- 172.21.15.69:0/1758777209 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55570077d8c0 con 0x55570082dc20 2024-03-19T01:11:34.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee39600640 10 client.25892 handle_command_reply: tid=0 2024-03-19T01:11:34.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee59405740 1 client.25892 shutdown 2024-03-19T01:11:34.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee59405740 1 -- 172.21.15.69:0/1758777209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814230 msgr2=0x55570081a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee59405740 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814230 0x55570081a820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fee4002c160 tx=0x7fee400bd370 comp rx=0 tx=0).stop 2024-03-19T01:11:34.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee59405740 1 -- 172.21.15.69:0/1758777209 shutdown_connections 2024-03-19T01:11:34.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee59405740 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55570082dc20 0x5557008300c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee59405740 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570081d730 0x55570081b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee59405740 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555700814b50 0x55570081ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee59405740 1 --2- 172.21.15.69:0/1758777209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814230 0x55570081a820 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee59405740 1 -- 172.21.15.69:0/1758777209 >> 172.21.15.69:0/1758777209 conn(0x55570080e960 msgr2=0x5557008106b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.587+0000 7fee39600640 0 client.25892 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:11:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.588+0000 7fee59405740 1 -- 172.21.15.69:0/1758777209 shutdown_connections 2024-03-19T01:11:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.588+0000 7fee59405740 1 -- 172.21.15.69:0/1758777209 wait complete. 2024-03-19T01:11:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.588+0000 7fee59405740 20 client.25892 trim_cache size 0 max 16384 2024-03-19T01:11:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.605+0000 7fee59405740 1 Processor -- start 2024-03-19T01:11:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.605+0000 7fee59405740 1 -- start start 2024-03-19T01:11:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.606+0000 7fee59405740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x5557008051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.606+0000 7fee59405740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 0x5557008292c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.606+0000 7fee59405740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570081d730 0x555700835990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.606+0000 7fee59405740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee501acb10 con 0x55570081d730 2024-03-19T01:11:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.606+0000 7fee59405740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee501ae650 con 0x555700814b50 2024-03-19T01:11:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.606+0000 7fee59405740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fee501ad1e0 con 0x555700814230 2024-03-19T01:11:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.606+0000 7fee4f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x5557008051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.606+0000 7fee4ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 0x5557008292c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.606+0000 7fee4f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x5557008051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46654/0 (socket says 172.21.15.69:46654) 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.606+0000 7fee4f400640 1 -- 172.21.15.69:0/208846626 learned_addr learned my addr 172.21.15.69:0/208846626 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.606+0000 7fee4ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 0x5557008292c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52350/0 (socket says 172.21.15.69:52350) 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.607+0000 7fee4ea00640 1 -- 172.21.15.69:0/208846626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 msgr2=0x5557008051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.607+0000 7fee4fe00640 1 --2- 172.21.15.69:0/208846626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570081d730 0x555700835990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.607+0000 7fee4ea00640 1 --2- 172.21.15.69:0/208846626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x5557008051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.607+0000 7fee4ea00640 1 -- 172.21.15.69:0/208846626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570081d730 msgr2=0x555700835990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.607+0000 7fee4ea00640 1 --2- 172.21.15.69:0/208846626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570081d730 0x555700835990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.607+0000 7fee4ea00640 1 -- 172.21.15.69:0/208846626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570080e490 con 0x555700814b50 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.607+0000 7fee4fe00640 1 --2- 172.21.15.69:0/208846626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570081d730 0x555700835990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.607+0000 7fee4ea00640 1 --2- 172.21.15.69:0/208846626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 0x5557008292c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x555700830930 tx=0x7fee3c00d500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9c431b4784f2ff server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.607+0000 7fee3be00640 1 -- 172.21.15.69:0/208846626 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c01b530 con 0x555700814b50 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.607+0000 7fee3be00640 1 -- 172.21.15.69:0/208846626 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c01b6d0 con 0x555700814b50 2024-03-19T01:11:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.607+0000 7fee3be00640 1 -- 172.21.15.69:0/208846626 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0126d0 con 0x555700814b50 2024-03-19T01:11:34.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.608+0000 7fee59405740 1 -- 172.21.15.69:0/208846626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 msgr2=0x5557008292c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.608+0000 7fee59405740 1 --2- 172.21.15.69:0/208846626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 0x5557008292c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x555700830930 tx=0x7fee3c00d500 comp rx=0 tx=0).stop 2024-03-19T01:11:34.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.608+0000 7fee59405740 1 -- 172.21.15.69:0/208846626 shutdown_connections 2024-03-19T01:11:34.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.608+0000 7fee59405740 1 --2- 172.21.15.69:0/208846626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570081d730 0x555700835990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.608+0000 7fee59405740 1 --2- 172.21.15.69:0/208846626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 0x5557008292c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.608+0000 7fee59405740 1 --2- 172.21.15.69:0/208846626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x5557008051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.608+0000 7fee59405740 1 -- 172.21.15.69:0/208846626 >> 172.21.15.69:0/208846626 conn(0x55570080e960 msgr2=0x55570081a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.609+0000 7fee59405740 1 -- 172.21.15.69:0/208846626 shutdown_connections 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.609+0000 7fee59405740 1 -- 172.21.15.69:0/208846626 wait complete. 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.609+0000 7fee59405740 1 -- start start 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.609+0000 7fee59405740 1 Processor -- start 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee59405740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x5557008228c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee59405740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 0x555700822e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee59405740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570081d730 0x555700823340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee59405740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee501adf80 con 0x55570081d730 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee59405740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee501ad8b0 con 0x555700814b50 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee59405740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fee501aaaf0 con 0x555700814230 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee59405740 10 client.0 _ll_register_callbacks cb 0x7fee4002ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee4f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x5557008228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee4f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x5557008228c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46656/0 (socket says 172.21.15.69:46656) 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee4f400640 1 -- 172.21.15.69:0/3327743009 learned_addr learned my addr 172.21.15.69:0/3327743009 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee4ea00640 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 0x555700822e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.610+0000 7fee4fe00640 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570081d730 0x555700823340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee4f400640 1 -- 172.21.15.69:0/3327743009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 msgr2=0x555700822e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee4f400640 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 0x555700822e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee4f400640 1 -- 172.21.15.69:0/3327743009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570081d730 msgr2=0x555700823340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee4f400640 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570081d730 0x555700823340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee4f400640 1 -- 172.21.15.69:0/3327743009 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555700893fd0 con 0x555700814230 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee4f400640 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x5557008228c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fee400b9eb0 tx=0x7fee400035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a60c887be9799f5c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee39600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee39600640 1 -- 172.21.15.69:0/3327743009 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee400eee60 con 0x555700814230 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee39600640 1 -- 172.21.15.69:0/3327743009 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fee400c1120 con 0x555700814230 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee59405740 1 -- 172.21.15.69:0/3327743009 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3c01b6d0 con 0x555700814230 2024-03-19T01:11:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee39600640 1 -- 172.21.15.69:0/3327743009 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee400b9780 con 0x555700814230 2024-03-19T01:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee59405740 10 client.25894 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.611+0000 7fee59405740 1 -- 172.21.15.69:0/3327743009 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5557008950c0 con 0x555700814230 2024-03-19T01:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.612+0000 7fee39600640 1 -- 172.21.15.69:0/3327743009 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c01b6d0 con 0x555700814230 2024-03-19T01:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.612+0000 7fee39600640 1 -- 172.21.15.69:0/3327743009 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee400bc930 con 0x555700814230 2024-03-19T01:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.612+0000 7fee59405740 10 client.25894 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.612+0000 7fee59405740 10 client.25894 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.612+0000 7fee59405740 20 client.25894 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.612+0000 7fee59405740 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555700895650 0x555700897a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.612+0000 7fee59405740 4 client.25894 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.612+0000 7fee59405740 1 -- 172.21.15.69:0/3327743009 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555700777c10 con 0x555700895650 2024-03-19T01:11:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.612+0000 7fee4ea00640 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555700895650 0x555700897a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee39600640 10 client.25894 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee4ea00640 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555700895650 0x555700897a40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=375227d588228042 server_cookie=7002408e87b14fd4 in_seq=0 out_seq=0 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee39600640 1 -- 172.21.15.69:0/3327743009 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555700777c10 con 0x555700895650 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee39600640 10 client.25894 handle_command_reply: tid=0 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee59405740 1 client.25894 shutdown 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee59405740 1 -- 172.21.15.69:0/3327743009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 msgr2=0x5557008228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee59405740 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x5557008228c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fee400b9eb0 tx=0x7fee400035f0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee59405740 1 -- 172.21.15.69:0/3327743009 shutdown_connections 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee59405740 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555700895650 0x555700897a40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee59405740 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570081d730 0x555700823340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee39600640 0 client.25894 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee59405740 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555700814b50 0x555700822e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee59405740 1 --2- 172.21.15.69:0/3327743009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555700814230 0x5557008228c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.613+0000 7fee59405740 1 -- 172.21.15.69:0/3327743009 >> 172.21.15.69:0/3327743009 conn(0x55570080e960 msgr2=0x555700805050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.614+0000 7fee59405740 1 -- 172.21.15.69:0/3327743009 shutdown_connections 2024-03-19T01:11:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.614+0000 7fee59405740 1 -- 172.21.15.69:0/3327743009 wait complete. 2024-03-19T01:11:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.614+0000 7fee59405740 20 client.25894 trim_cache size 0 max 16384 2024-03-19T01:11:34.619 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:11:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 -- 172.21.15.69:0/4179355078 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fee340a2b10 msgr2=0x7fee340a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fee340a2b10 0x7fee340a4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fee48009d60 tx=0x7fee48004700 comp rx=0 tx=0).stop 2024-03-19T01:11:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 -- 172.21.15.69:0/4179355078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee501039b0 msgr2=0x7fee5019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee501039b0 0x7fee5019a2c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fee40005ab0 tx=0x7fee400042e0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 -- 172.21.15.69:0/4179355078 shutdown_connections 2024-03-19T01:11:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fee340a2b10 0x7fee340a4fd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee501099c0 0x7fee5019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee50104370 0x7fee5019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 --2- 172.21.15.69:0/4179355078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee501039b0 0x7fee5019a2c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 -- 172.21.15.69:0/4179355078 >> 172.21.15.69:0/4179355078 conn(0x7fee500ff4a0 msgr2=0x7fee500ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 -- 172.21.15.69:0/4179355078 shutdown_connections 2024-03-19T01:11:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.615+0000 7fee56800640 1 -- 172.21.15.69:0/4179355078 wait complete. 2024-03-19T01:11:34.636 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:11:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.844+0000 7f759d800640 1 -- 172.21.15.69:0/3542404631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7598102a60 msgr2=0x7f7598102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.844+0000 7f759d800640 1 --2- 172.21.15.69:0/3542404631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7598102a60 0x7f7598102e40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7588009ca0 tx=0x7f758802c180 comp rx=0 tx=0).stop 2024-03-19T01:11:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.844+0000 7f759d800640 1 -- 172.21.15.69:0/3542404631 shutdown_connections 2024-03-19T01:11:34.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.844+0000 7f759d800640 1 --2- 172.21.15.69:0/3542404631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f759810be30 0x7f759810e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.844+0000 7f759d800640 1 --2- 172.21.15.69:0/3542404631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7598103380 0x7f759810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.844+0000 7f759d800640 1 --2- 172.21.15.69:0/3542404631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7598102a60 0x7f7598102e40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.844+0000 7f759d800640 1 -- 172.21.15.69:0/3542404631 >> 172.21.15.69:0/3542404631 conn(0x7f7598076aa0 msgr2=0x7f7598076eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.844+0000 7f759d800640 1 -- 172.21.15.69:0/3542404631 shutdown_connections 2024-03-19T01:11:34.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.844+0000 7f759d800640 1 -- 172.21.15.69:0/3542404631 wait complete. 2024-03-19T01:11:34.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.845+0000 7f759d800640 1 Processor -- start 2024-03-19T01:11:34.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.845+0000 7f759d800640 1 -- start start 2024-03-19T01:11:34.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f759d800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7598102a60 0x7f75981980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f759d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7598103380 0x7f7598198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f759d800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f759810be30 0x7f759819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f759d800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7598111bc0 con 0x7f759810be30 2024-03-19T01:11:34.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f759d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7598111a40 con 0x7f7598103380 2024-03-19T01:11:34.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f759d800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7598111d40 con 0x7f7598102a60 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f7596a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7598102a60 0x7f75981980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f758e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7598103380 0x7f7598198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f758e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7598103380 0x7f7598198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52376/0 (socket says 172.21.15.69:52376) 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f758e000640 1 -- 172.21.15.69:0/3284600943 learned_addr learned my addr 172.21.15.69:0/3284600943 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f7597400640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f759810be30 0x7f759819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f7596a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7598102a60 0x7f75981980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46672/0 (socket says 172.21.15.69:46672) 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f758e000640 1 -- 172.21.15.69:0/3284600943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7598102a60 msgr2=0x7f75981980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f758e000640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7598102a60 0x7f75981980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f758e000640 1 -- 172.21.15.69:0/3284600943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f759810be30 msgr2=0x7f759819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f758e000640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f759810be30 0x7f759819c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.846+0000 7f758e000640 1 -- 172.21.15.69:0/3284600943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759819d0b0 con 0x7f7598103380 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.847+0000 7f7596a00640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7598102a60 0x7f75981980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.847+0000 7f758e000640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7598103380 0x7f7598198600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7580009de0 tx=0x7f758000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.847+0000 7f758fe00640 1 -- 172.21.15.69:0/3284600943 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758001f070 con 0x7f7598103380 2024-03-19T01:11:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.847+0000 7f759d800640 1 -- 172.21.15.69:0/3284600943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759819d3a0 con 0x7f7598103380 2024-03-19T01:11:34.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.847+0000 7f758fe00640 1 -- 172.21.15.69:0/3284600943 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758001a8f0 con 0x7f7598103380 2024-03-19T01:11:34.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.847+0000 7f758fe00640 1 -- 172.21.15.69:0/3284600943 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75800199a0 con 0x7f7598103380 2024-03-19T01:11:34.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.847+0000 7f759d800640 1 -- 172.21.15.69:0/3284600943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75981005e0 con 0x7f7598103380 2024-03-19T01:11:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.851+0000 7f75a05e8740 1 Processor -- start 2024-03-19T01:11:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.851+0000 7f75a05e8740 1 -- start start 2024-03-19T01:11:34.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.852+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 0x5587229d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.852+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229d6340 0x5587229de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.852+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229def20 0x5587229e1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.852+0000 7f75a05e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55872293a380 con 0x5587229d5a20 2024-03-19T01:11:34.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.852+0000 7f75a05e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5587228e2d20 con 0x5587229def20 2024-03-19T01:11:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.852+0000 7f75a05e8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558722936420 con 0x5587229d6340 2024-03-19T01:11:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.852+0000 7f758fe00640 1 -- 172.21.15.69:0/3284600943 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7f7580022070 con 0x7f7598103380 2024-03-19T01:11:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.852+0000 7f7597400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 0x5587229d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.852+0000 7f758e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229def20 0x5587229e1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.852+0000 7f7597400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 0x5587229d5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51634/0 (socket says 172.21.15.69:51634) 2024-03-19T01:11:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.852+0000 7f7597400640 1 -- 172.21.15.69:0/3433764094 learned_addr learned my addr 172.21.15.69:0/3433764094 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.853+0000 7f758e000640 1 -- 172.21.15.69:0/3433764094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229d6340 msgr2=0x5587229de9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.853+0000 7f758e000640 1 --2- 172.21.15.69:0/3433764094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229d6340 0x5587229de9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.853+0000 7f758e000640 1 -- 172.21.15.69:0/3433764094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 msgr2=0x5587229d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.853+0000 7f758e000640 1 --2- 172.21.15.69:0/3433764094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 0x5587229d5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.853+0000 7f758e000640 1 -- 172.21.15.69:0/3433764094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55872264e3f0 con 0x5587229def20 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.853+0000 7f758e000640 1 --2- 172.21.15.69:0/3433764094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229def20 0x5587229e1310 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f75800d02a0 tx=0x7f75800db670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41a12dfd73210c0c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.853+0000 7f7587e00640 1 -- 172.21.15.69:0/3433764094 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75800e6070 con 0x5587229def20 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.854+0000 7f7587e00640 1 -- 172.21.15.69:0/3433764094 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75800d39c0 con 0x5587229def20 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.854+0000 7f7587e00640 1 -- 172.21.15.69:0/3433764094 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75800e19e0 con 0x5587229def20 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.854+0000 7f75a05e8740 1 -- 172.21.15.69:0/3433764094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229def20 msgr2=0x5587229e1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.854+0000 7f75a05e8740 1 --2- 172.21.15.69:0/3433764094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229def20 0x5587229e1310 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f75800d02a0 tx=0x7f75800db670 comp rx=0 tx=0).stop 2024-03-19T01:11:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.854+0000 7f75a05e8740 1 -- 172.21.15.69:0/3433764094 shutdown_connections 2024-03-19T01:11:34.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.854+0000 7f75a05e8740 1 --2- 172.21.15.69:0/3433764094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229def20 0x5587229e1310 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.854+0000 7f75a05e8740 1 --2- 172.21.15.69:0/3433764094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229d6340 0x5587229de9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.854+0000 7f75a05e8740 1 --2- 172.21.15.69:0/3433764094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 0x5587229d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.854+0000 7f75a05e8740 1 -- 172.21.15.69:0/3433764094 >> 172.21.15.69:0/3433764094 conn(0x5587229d0150 msgr2=0x5587229d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.854+0000 7f758fe00640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f757c0a2a90 0x7f757c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.854+0000 7f75a05e8740 1 -- 172.21.15.69:0/3433764094 shutdown_connections 2024-03-19T01:11:34.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.855+0000 7f7597400640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f757c0a2a90 0x7f757c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.855+0000 7f758fe00640 1 -- 172.21.15.69:0/3284600943 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f75800d30f0 con 0x7f7598103380 2024-03-19T01:11:34.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.855+0000 7f75a05e8740 1 -- 172.21.15.69:0/3433764094 wait complete. 2024-03-19T01:11:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.855+0000 7f7597400640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f757c0a2a90 0x7f757c0a4f50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7590009da0 tx=0x7f7590005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.855+0000 7f75a05e8740 1 -- start start 2024-03-19T01:11:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f75a05e8740 1 Processor -- start 2024-03-19T01:11:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d5a20 0x5587229dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d6340 0x5587229dc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229def20 0x5587229dca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f75a05e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5587229e1b90 con 0x5587229d6340 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f75a05e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5587228f6160 con 0x5587229d5a20 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f75a05e8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5587229e1d10 con 0x5587229def20 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f75a05e8740 10 client.0 _ll_register_callbacks cb 0x558722932170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f758e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d6340 0x5587229dc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f7596a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d5a20 0x5587229dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f7596a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d5a20 0x5587229dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52402/0 (socket says 172.21.15.69:52402) 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f7596a00640 1 -- 172.21.15.69:0/2313689684 learned_addr learned my addr 172.21.15.69:0/2313689684 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.856+0000 7f7597400640 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229def20 0x5587229dca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.857+0000 7f7596a00640 1 -- 172.21.15.69:0/2313689684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229def20 msgr2=0x5587229dca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.857+0000 7f7596a00640 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229def20 0x5587229dca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.857+0000 7f7596a00640 1 -- 172.21.15.69:0/2313689684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d6340 msgr2=0x5587229dc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.857+0000 7f7596a00640 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d6340 0x5587229dc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.857+0000 7f7596a00640 1 -- 172.21.15.69:0/2313689684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587227187d0 con 0x5587229d5a20 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.857+0000 7f7596a00640 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d5a20 0x5587229dbf80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f758802c6c0 tx=0x7f7588003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a56dcda581fbd938 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.857+0000 7f7584c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.857+0000 7f7584c00640 1 -- 172.21.15.69:0/2313689684 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588043020 con 0x5587229d5a20 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.857+0000 7f75a05e8740 1 -- 172.21.15.69:0/2313689684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75800d39c0 con 0x5587229d5a20 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.857+0000 7f7584c00640 1 -- 172.21.15.69:0/2313689684 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f75880349f0 con 0x5587229d5a20 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.858+0000 7f7584c00640 1 -- 172.21.15.69:0/2313689684 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f758804e050 con 0x5587229d5a20 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.858+0000 7f75a05e8740 10 client.25910 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.858+0000 7f75a05e8740 1 -- 172.21.15.69:0/2313689684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587229ede60 con 0x5587229d5a20 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.858+0000 7f7584c00640 1 -- 172.21.15.69:0/2313689684 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75800d39c0 con 0x5587229d5a20 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.858+0000 7f7584c00640 1 -- 172.21.15.69:0/2313689684 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7588032670 con 0x5587229d5a20 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.858+0000 7f75a05e8740 10 client.25910 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.858+0000 7f75a05e8740 10 client.25910 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.858+0000 7f75a05e8740 20 client.25910 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.858+0000 7f75a05e8740 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5587229ef320 0x5587229f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.858+0000 7f75a05e8740 4 client.25910 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.858+0000 7f75a05e8740 1 -- 172.21.15.69:0/2313689684 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587229478c0 con 0x5587229ef320 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.859+0000 7f758e000640 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5587229ef320 0x5587229f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.859+0000 7f758e000640 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5587229ef320 0x5587229f17a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3fe6c32c0e94f536 server_cookie=195734ee734bb416 in_seq=0 out_seq=0 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.859+0000 7f7584c00640 10 client.25910 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f7584c00640 1 -- 172.21.15.69:0/2313689684 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5587229478c0 con 0x5587229ef320 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f7584c00640 10 client.25910 handle_command_reply: tid=0 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 1 client.25910 shutdown 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 1 -- 172.21.15.69:0/2313689684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d5a20 msgr2=0x5587229dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d5a20 0x5587229dbf80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f758802c6c0 tx=0x7f7588003e70 comp rx=0 tx=0).stop 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 1 -- 172.21.15.69:0/2313689684 shutdown_connections 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229def20 0x5587229dca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d6340 0x5587229dc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5587229ef320 0x5587229f17a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 1 --2- 172.21.15.69:0/2313689684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d5a20 0x5587229dbf80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 1 -- 172.21.15.69:0/2313689684 >> 172.21.15.69:0/2313689684 conn(0x5587229d0150 msgr2=0x5587229d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f7584c00640 0 client.25910 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:11:34.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 1 -- 172.21.15.69:0/2313689684 shutdown_connections 2024-03-19T01:11:34.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 1 -- 172.21.15.69:0/2313689684 wait complete. 2024-03-19T01:11:34.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.860+0000 7f75a05e8740 20 client.25910 trim_cache size 0 max 16384 2024-03-19T01:11:34.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.880+0000 7f75a05e8740 1 Processor -- start 2024-03-19T01:11:34.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.880+0000 7f75a05e8740 1 -- start start 2024-03-19T01:11:34.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.880+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229d5a20 0x5587229de6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.880+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229db100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.880+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229def20 0x5587229ea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.880+0000 7f75a05e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75981acb90 con 0x5587229def20 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.880+0000 7f75a05e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75981ae6d0 con 0x5587229d6340 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.880+0000 7f75a05e8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f75981ad260 con 0x5587229d5a20 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f758e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229db100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f758e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229db100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52406/0 (socket says 172.21.15.69:52406) 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f758e000640 1 -- 172.21.15.69:0/3493814097 learned_addr learned my addr 172.21.15.69:0/3493814097 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f7596a00640 1 --2- 172.21.15.69:0/3493814097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229d5a20 0x5587229de6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f7597400640 1 --2- 172.21.15.69:0/3493814097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229def20 0x5587229ea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f758e000640 1 -- 172.21.15.69:0/3493814097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229d5a20 msgr2=0x5587229de6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f758e000640 1 --2- 172.21.15.69:0/3493814097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229d5a20 0x5587229de6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f758e000640 1 -- 172.21.15.69:0/3493814097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229def20 msgr2=0x5587229ea930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f758e000640 1 --2- 172.21.15.69:0/3493814097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229def20 0x5587229ea930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f758e000640 1 -- 172.21.15.69:0/3493814097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587229d6d90 con 0x5587229d6340 2024-03-19T01:11:34.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f7597400640 1 --2- 172.21.15.69:0/3493814097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229def20 0x5587229ea930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:11:34.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f758e000640 1 --2- 172.21.15.69:0/3493814097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229db100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f75800e1510 tx=0x7f75800e1540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed5d5f0164bb35e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.881+0000 7f7587400640 1 -- 172.21.15.69:0/3493814097 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758009b430 con 0x5587229d6340 2024-03-19T01:11:34.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.882+0000 7f7587400640 1 -- 172.21.15.69:0/3493814097 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75800dbe70 con 0x5587229d6340 2024-03-19T01:11:34.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.882+0000 7f7587400640 1 -- 172.21.15.69:0/3493814097 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758001aaa0 con 0x5587229d6340 2024-03-19T01:11:34.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.882+0000 7f75a05e8740 1 -- 172.21.15.69:0/3493814097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 msgr2=0x5587229db100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.882+0000 7f75a05e8740 1 --2- 172.21.15.69:0/3493814097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229db100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f75800e1510 tx=0x7f75800e1540 comp rx=0 tx=0).stop 2024-03-19T01:11:34.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.882+0000 7f75a05e8740 1 -- 172.21.15.69:0/3493814097 shutdown_connections 2024-03-19T01:11:34.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.882+0000 7f75a05e8740 1 --2- 172.21.15.69:0/3493814097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229def20 0x5587229ea930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.882+0000 7f75a05e8740 1 --2- 172.21.15.69:0/3493814097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229db100 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.882+0000 7f75a05e8740 1 --2- 172.21.15.69:0/3493814097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229d5a20 0x5587229de6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.882+0000 7f75a05e8740 1 -- 172.21.15.69:0/3493814097 >> 172.21.15.69:0/3493814097 conn(0x5587229d0150 msgr2=0x5587229ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.882+0000 7f75a05e8740 1 -- 172.21.15.69:0/3493814097 shutdown_connections 2024-03-19T01:11:34.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.882+0000 7f75a05e8740 1 -- 172.21.15.69:0/3493814097 wait complete. 2024-03-19T01:11:34.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.883+0000 7f75a05e8740 1 -- start start 2024-03-19T01:11:34.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.883+0000 7f75a05e8740 1 Processor -- start 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.883+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 0x5587229d8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.883+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229d9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.883+0000 7f75a05e8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229def20 0x5587229d9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.883+0000 7f75a05e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75981ae000 con 0x5587229d5a20 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.883+0000 7f75a05e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75981ad930 con 0x5587229d6340 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.883+0000 7f75a05e8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f75981aab70 con 0x5587229def20 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.883+0000 7f75a05e8740 10 client.0 _ll_register_callbacks cb 0x7f7580005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f7596a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 0x5587229d8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f758e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229d9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f758e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229d9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52412/0 (socket says 172.21.15.69:52412) 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f758e000640 1 -- 172.21.15.69:0/400329840 learned_addr learned my addr 172.21.15.69:0/400329840 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f7597400640 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229def20 0x5587229d9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f758e000640 1 -- 172.21.15.69:0/400329840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229def20 msgr2=0x5587229d9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f758e000640 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229def20 0x5587229d9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f758e000640 1 -- 172.21.15.69:0/400329840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 msgr2=0x5587229d8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f758e000640 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 0x5587229d8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f758e000640 1 -- 172.21.15.69:0/400329840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558722a54990 con 0x5587229d6340 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f7596a00640 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 0x5587229d8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f758e000640 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229d9300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f758009ee90 tx=0x7f75800193b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3729724fa77ab702 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f7584c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f7584c00640 1 -- 172.21.15.69:0/400329840 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758009eaf0 con 0x5587229d6340 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f7584c00640 1 -- 172.21.15.69:0/400329840 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f758009b800 con 0x5587229d6340 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.884+0000 7f75a05e8740 1 -- 172.21.15.69:0/400329840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75800dbe70 con 0x5587229d6340 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f7584c00640 1 -- 172.21.15.69:0/400329840 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f758009ecd0 con 0x5587229d6340 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f75a05e8740 10 client.25916 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f75a05e8740 1 -- 172.21.15.69:0/400329840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558722a55c70 con 0x5587229d6340 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f7584c00640 1 -- 172.21.15.69:0/400329840 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75800dbe70 con 0x5587229d6340 2024-03-19T01:11:34.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f7584c00640 1 -- 172.21.15.69:0/400329840 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7580019b40 con 0x5587229d6340 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f75a05e8740 10 client.25916 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f75a05e8740 10 client.25916 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f75a05e8740 20 client.25916 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f75a05e8740 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x558722a55f00 0x558722a582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f75a05e8740 4 client.25916 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f75a05e8740 1 -- 172.21.15.69:0/400329840 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558722941c10 con 0x558722a55f00 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.885+0000 7f7596a00640 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x558722a55f00 0x558722a582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f7596a00640 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x558722a55f00 0x558722a582f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=191a4d36b71dd24d server_cookie=5c7b372d48dba11d in_seq=0 out_seq=0 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f7584c00640 10 client.25916 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f7584c00640 1 -- 172.21.15.69:0/400329840 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558722941c10 con 0x558722a55f00 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f7584c00640 10 client.25916 handle_command_reply: tid=0 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 1 client.25916 shutdown 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 1 -- 172.21.15.69:0/400329840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 msgr2=0x5587229d9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229d9300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f758009ee90 tx=0x7f75800193b0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 1 -- 172.21.15.69:0/400329840 shutdown_connections 2024-03-19T01:11:34.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5587229def20 0x5587229d9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x558722a55f00 0x558722a582f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587229d6340 0x5587229d9300 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 1 --2- 172.21.15.69:0/400329840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5587229d5a20 0x5587229d8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f7584c00640 0 client.25916 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:11:34.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 1 -- 172.21.15.69:0/400329840 >> 172.21.15.69:0/400329840 conn(0x5587229d0150 msgr2=0x5587229d1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 1 -- 172.21.15.69:0/400329840 shutdown_connections 2024-03-19T01:11:34.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 1 -- 172.21.15.69:0/400329840 wait complete. 2024-03-19T01:11:34.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.886+0000 7f75a05e8740 20 client.25916 trim_cache size 0 max 16384 2024-03-19T01:11:34.892 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:11:34.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 -- 172.21.15.69:0/3284600943 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f757c0a2a90 msgr2=0x7f757c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f757c0a2a90 0x7f757c0a4f50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7590009da0 tx=0x7f7590005f90 comp rx=0 tx=0).stop 2024-03-19T01:11:34.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 -- 172.21.15.69:0/3284600943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7598103380 msgr2=0x7f7598198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:34.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7598103380 0x7f7598198600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7580009de0 tx=0x7f758000c6a0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 -- 172.21.15.69:0/3284600943 shutdown_connections 2024-03-19T01:11:34.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f759810be30 0x7f759819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f757c0a2a90 0x7f757c0a4f50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7598103380 0x7f7598198600 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 --2- 172.21.15.69:0/3284600943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7598102a60 0x7f75981980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:34.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 -- 172.21.15.69:0/3284600943 >> 172.21.15.69:0/3284600943 conn(0x7f7598076aa0 msgr2=0x7f75980fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:34.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 -- 172.21.15.69:0/3284600943 shutdown_connections 2024-03-19T01:11:34.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:34.887+0000 7f759d800640 1 -- 172.21.15.69:0/3284600943 wait complete. 2024-03-19T01:11:34.911 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:11:35.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.140+0000 7fc987800640 1 -- 172.21.15.69:0/1110395521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc98010be30 msgr2=0x7fc98010e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.140+0000 7fc987800640 1 --2- 172.21.15.69:0/1110395521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc98010be30 0x7fc98010e260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc978009be0 tx=0x7fc97802bf80 comp rx=0 tx=0).stop 2024-03-19T01:11:35.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.140+0000 7fc987800640 1 -- 172.21.15.69:0/1110395521 shutdown_connections 2024-03-19T01:11:35.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.140+0000 7fc987800640 1 --2- 172.21.15.69:0/1110395521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc98010be30 0x7fc98010e260 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.140+0000 7fc987800640 1 --2- 172.21.15.69:0/1110395521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc980103380 0x7fc98010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.140+0000 7fc987800640 1 --2- 172.21.15.69:0/1110395521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc980102a60 0x7fc980102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.140+0000 7fc987800640 1 -- 172.21.15.69:0/1110395521 >> 172.21.15.69:0/1110395521 conn(0x7fc980076aa0 msgr2=0x7fc980076eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.140+0000 7fc987800640 1 -- 172.21.15.69:0/1110395521 shutdown_connections 2024-03-19T01:11:35.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.140+0000 7fc987800640 1 -- 172.21.15.69:0/1110395521 wait complete. 2024-03-19T01:11:35.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.141+0000 7fc987800640 1 Processor -- start 2024-03-19T01:11:35.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.142+0000 7fc987800640 1 -- start start 2024-03-19T01:11:35.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.142+0000 7fc987800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc980102a60 0x7fc9801006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.142+0000 7fc987800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc980103380 0x7fc980100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.142+0000 7fc987800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc98010be30 0x7fc980101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.142+0000 7fc987800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc980111c10 con 0x7fc980102a60 2024-03-19T01:11:35.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.142+0000 7fc987800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc980111a90 con 0x7fc98010be30 2024-03-19T01:11:35.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.142+0000 7fc987800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc980111d90 con 0x7fc980103380 2024-03-19T01:11:35.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.142+0000 7fc984e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc980102a60 0x7fc9801006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.142+0000 7fc97fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc980103380 0x7fc980100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.143+0000 7fc984e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc980102a60 0x7fc9801006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51670/0 (socket says 172.21.15.69:51670) 2024-03-19T01:11:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.143+0000 7fc984e00640 1 -- 172.21.15.69:0/3550910303 learned_addr learned my addr 172.21.15.69:0/3550910303 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.143+0000 7fc985800640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc98010be30 0x7fc980101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.143+0000 7fc984e00640 1 -- 172.21.15.69:0/3550910303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc980103380 msgr2=0x7fc980100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.143+0000 7fc984e00640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc980103380 0x7fc980100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.143+0000 7fc984e00640 1 -- 172.21.15.69:0/3550910303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc98010be30 msgr2=0x7fc980101130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.143+0000 7fc984e00640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc98010be30 0x7fc980101130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.143+0000 7fc984e00640 1 -- 172.21.15.69:0/3550910303 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9801a6570 con 0x7fc980102a60 2024-03-19T01:11:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.143+0000 7fc97fe00640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc980103380 0x7fc980100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:35.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.143+0000 7fc984e00640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc980102a60 0x7fc9801006b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc97400e9f0 tx=0x7fc97400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.144+0000 7fc97d600640 1 -- 172.21.15.69:0/3550910303 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97400ccc0 con 0x7fc980102a60 2024-03-19T01:11:35.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.144+0000 7fc987800640 1 -- 172.21.15.69:0/3550910303 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9801a6860 con 0x7fc980102a60 2024-03-19T01:11:35.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.144+0000 7fc97d600640 1 -- 172.21.15.69:0/3550910303 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc974005500 con 0x7fc980102a60 2024-03-19T01:11:35.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.144+0000 7fc97d600640 1 -- 172.21.15.69:0/3550910303 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974010690 con 0x7fc980102a60 2024-03-19T01:11:35.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.144+0000 7fc987800640 1 -- 172.21.15.69:0/3550910303 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9801a6db0 con 0x7fc980102a60 2024-03-19T01:11:35.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.148+0000 7fc97d600640 1 -- 172.21.15.69:0/3550910303 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7fc9740046c0 con 0x7fc980102a60 2024-03-19T01:11:35.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.148+0000 7fc98a5cf740 1 Processor -- start 2024-03-19T01:11:35.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.149+0000 7fc98a5cf740 1 -- start start 2024-03-19T01:11:35.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.149+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9fea20 0x56000b9fee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.149+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.149+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000ba07f20 0x56000ba0a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.149+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56000b963380 con 0x56000b9fea20 2024-03-19T01:11:35.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.149+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56000b90bd20 con 0x56000ba07f20 2024-03-19T01:11:35.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.149+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56000b95f420 con 0x56000b9ff340 2024-03-19T01:11:35.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.149+0000 7fc97d600640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc9680a2ac0 0x7fc9680a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.149+0000 7fc97d600640 1 -- 172.21.15.69:0/3550910303 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc9740cf6d0 con 0x7fc980102a60 2024-03-19T01:11:35.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.149+0000 7fc984e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.149+0000 7fc984e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba079e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46712/0 (socket says 172.21.15.69:46712) 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc97fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000ba07f20 0x56000ba0a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc984e00640 1 -- 172.21.15.69:0/2381890276 learned_addr learned my addr 172.21.15.69:0/2381890276 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc985800640 1 --2- 172.21.15.69:0/2381890276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9fea20 0x56000b9fee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc985800640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc9680a2ac0 0x7fc9680a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc984e00640 1 -- 172.21.15.69:0/2381890276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000ba07f20 msgr2=0x56000ba0a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc984e00640 1 --2- 172.21.15.69:0/2381890276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000ba07f20 0x56000ba0a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc984e00640 1 -- 172.21.15.69:0/2381890276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9fea20 msgr2=0x56000b9fee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc984e00640 1 --2- 172.21.15.69:0/2381890276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9fea20 0x56000b9fee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc984e00640 1 -- 172.21.15.69:0/2381890276 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56000b6773f0 con 0x56000b9ff340 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc97fe00640 1 --2- 172.21.15.69:0/2381890276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000ba07f20 0x56000ba0a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:11:35.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc984e00640 1 --2- 172.21.15.69:0/2381890276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba079e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc974097b20 tx=0x7fc974009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a095275dbe44eb6f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.150+0000 7fc972000640 1 -- 172.21.15.69:0/2381890276 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9740b4070 con 0x56000b9ff340 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.151+0000 7fc972000640 1 -- 172.21.15.69:0/2381890276 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97409bdc0 con 0x56000b9ff340 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.151+0000 7fc972000640 1 -- 172.21.15.69:0/2381890276 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9740cf250 con 0x56000b9ff340 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.151+0000 7fc98a5cf740 1 -- 172.21.15.69:0/2381890276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 msgr2=0x56000ba079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.151+0000 7fc985800640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc9680a2ac0 0x7fc9680a4f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc978004af0 tx=0x7fc978005fd0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.151+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/2381890276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba079e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc974097b20 tx=0x7fc974009f50 comp rx=0 tx=0).stop 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.151+0000 7fc98a5cf740 1 -- 172.21.15.69:0/2381890276 shutdown_connections 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.151+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/2381890276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000ba07f20 0x56000ba0a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.151+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/2381890276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba079e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.151+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/2381890276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9fea20 0x56000b9fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.151+0000 7fc98a5cf740 1 -- 172.21.15.69:0/2381890276 >> 172.21.15.69:0/2381890276 conn(0x56000b9f9150 msgr2=0x56000b9f9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.152+0000 7fc98a5cf740 1 -- 172.21.15.69:0/2381890276 shutdown_connections 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.152+0000 7fc98a5cf740 1 -- 172.21.15.69:0/2381890276 wait complete. 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.152+0000 7fc98a5cf740 1 -- start start 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.152+0000 7fc98a5cf740 1 Processor -- start 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.153+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba04fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.153+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.153+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000ba07f20 0x56000ba05a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.153+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56000ba0ad10 con 0x56000ba07f20 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.153+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56000b91f160 con 0x56000b9fea20 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.153+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56000ba0ae90 con 0x56000b9ff340 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.153+0000 7fc98a5cf740 10 client.0 _ll_register_callbacks cb 0x56000b95b170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.153+0000 7fc984e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba04fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.153+0000 7fc984e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba04fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52440/0 (socket says 172.21.15.69:52440) 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.153+0000 7fc984e00640 1 -- 172.21.15.69:0/3976542951 learned_addr learned my addr 172.21.15.69:0/3976542951 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.153+0000 7fc985800640 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000ba07f20 0x56000ba05a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.154+0000 7fc97fe00640 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.154+0000 7fc984e00640 1 -- 172.21.15.69:0/3976542951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 msgr2=0x56000ba054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.154+0000 7fc984e00640 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba054f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.154+0000 7fc984e00640 1 -- 172.21.15.69:0/3976542951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000ba07f20 msgr2=0x56000ba05a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.154+0000 7fc984e00640 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000ba07f20 0x56000ba05a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.154+0000 7fc984e00640 1 -- 172.21.15.69:0/3976542951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000b7417d0 con 0x56000b9fea20 2024-03-19T01:11:35.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.154+0000 7fc984e00640 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba04fb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc97400c6a0 tx=0x7fc97409b310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1820538d8457757 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.154+0000 7fc966a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:35.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.154+0000 7fc966a00640 1 -- 172.21.15.69:0/3976542951 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9740b4040 con 0x56000b9fea20 2024-03-19T01:11:35.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc966a00640 1 -- 172.21.15.69:0/3976542951 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc9740a39d0 con 0x56000b9fea20 2024-03-19T01:11:35.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc98a5cf740 1 -- 172.21.15.69:0/3976542951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc97409bdc0 con 0x56000b9fea20 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc966a00640 1 -- 172.21.15.69:0/3976542951 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc97409f810 con 0x56000b9fea20 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc98a5cf740 10 client.25928 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc98a5cf740 1 -- 172.21.15.69:0/3976542951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56000ba0ad10 con 0x56000b9fea20 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc966a00640 1 -- 172.21.15.69:0/3976542951 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97409bdc0 con 0x56000b9fea20 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc966a00640 1 -- 172.21.15.69:0/3976542951 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc97409ab10 con 0x56000b9fea20 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc98a5cf740 10 client.25928 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc98a5cf740 10 client.25928 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc98a5cf740 20 client.25928 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56000ba181c0 0x56000ba1a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc98a5cf740 4 client.25928 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.155+0000 7fc98a5cf740 1 -- 172.21.15.69:0/3976542951 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56000b9708c0 con 0x56000ba181c0 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.156+0000 7fc97fe00640 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56000ba181c0 0x56000ba1a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.156+0000 7fc966a00640 10 client.25928 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:11:35.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.156+0000 7fc97fe00640 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56000ba181c0 0x56000ba1a5b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b0ee6d892713a24 server_cookie=28dcfdd2e5c50535 in_seq=0 out_seq=0 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc966a00640 1 -- 172.21.15.69:0/3976542951 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56000b9708c0 con 0x56000ba181c0 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc966a00640 10 client.25928 handle_command_reply: tid=0 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc98a5cf740 1 client.25928 shutdown 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc98a5cf740 1 -- 172.21.15.69:0/3976542951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 msgr2=0x56000ba04fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba04fb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc97400c6a0 tx=0x7fc97409b310 comp rx=0 tx=0).stop 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc98a5cf740 1 -- 172.21.15.69:0/3976542951 shutdown_connections 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56000ba181c0 0x56000ba1a5b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000ba07f20 0x56000ba05a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc966a00640 0 client.25928 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/3976542951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba04fb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc98a5cf740 1 -- 172.21.15.69:0/3976542951 >> 172.21.15.69:0/3976542951 conn(0x56000b9f9150 msgr2=0x56000b9faea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc98a5cf740 1 -- 172.21.15.69:0/3976542951 shutdown_connections 2024-03-19T01:11:35.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.157+0000 7fc98a5cf740 1 -- 172.21.15.69:0/3976542951 wait complete. 2024-03-19T01:11:35.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.158+0000 7fc98a5cf740 20 client.25928 trim_cache size 0 max 16384 2024-03-19T01:11:35.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.171+0000 7fc98a5cf740 1 Processor -- start 2024-03-19T01:11:35.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc98a5cf740 1 -- start start 2024-03-19T01:11:35.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba07600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9ff340 0x56000ba13ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000ba07f20 0x56000b9f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc96000ff70 con 0x56000b9ff340 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc960011ab0 con 0x56000b9fea20 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc960010640 con 0x56000ba07f20 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc984e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba07600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc984e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba07600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52450/0 (socket says 172.21.15.69:52450) 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc984e00640 1 -- 172.21.15.69:0/1482455084 learned_addr learned my addr 172.21.15.69:0/1482455084 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc985800640 1 --2- 172.21.15.69:0/1482455084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000ba07f20 0x56000b9f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc97fe00640 1 --2- 172.21.15.69:0/1482455084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9ff340 0x56000ba13ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc984e00640 1 -- 172.21.15.69:0/1482455084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000ba07f20 msgr2=0x56000b9f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc984e00640 1 --2- 172.21.15.69:0/1482455084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000ba07f20 0x56000b9f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc984e00640 1 -- 172.21.15.69:0/1482455084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9ff340 msgr2=0x56000ba13ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc984e00640 1 --2- 172.21.15.69:0/1482455084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9ff340 0x56000ba13ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.172+0000 7fc984e00640 1 -- 172.21.15.69:0/1482455084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56000ba03f60 con 0x56000b9fea20 2024-03-19T01:11:35.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.173+0000 7fc97fe00640 1 --2- 172.21.15.69:0/1482455084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9ff340 0x56000ba13ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:11:35.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.173+0000 7fc984e00640 1 --2- 172.21.15.69:0/1482455084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba07600 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc97409b600 tx=0x7fc974097fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50c6781ce8d76c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.173+0000 7fc971600640 1 -- 172.21.15.69:0/1482455084 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974097960 con 0x56000b9fea20 2024-03-19T01:11:35.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.173+0000 7fc971600640 1 -- 172.21.15.69:0/1482455084 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97409bdc0 con 0x56000b9fea20 2024-03-19T01:11:35.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.173+0000 7fc971600640 1 -- 172.21.15.69:0/1482455084 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97400ce60 con 0x56000b9fea20 2024-03-19T01:11:35.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.174+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1482455084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 msgr2=0x56000ba07600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.174+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/1482455084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba07600 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc97409b600 tx=0x7fc974097fb0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.174+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1482455084 shutdown_connections 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.174+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/1482455084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000ba07f20 0x56000b9f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.174+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/1482455084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9ff340 0x56000ba13ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.174+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/1482455084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000b9fea20 0x56000ba07600 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.174+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1482455084 >> 172.21.15.69:0/1482455084 conn(0x56000b9f9150 msgr2=0x56000b9f9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.174+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1482455084 shutdown_connections 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.174+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1482455084 wait complete. 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.174+0000 7fc98a5cf740 1 -- start start 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc98a5cf740 1 Processor -- start 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9fea20 0x56000ba0c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba0c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc98a5cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000ba07f20 0x56000ba7d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc9600113e0 con 0x56000b9fea20 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc960010d10 con 0x56000ba07f20 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc98a5cf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc96000df50 con 0x56000b9ff340 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc98a5cf740 10 client.0 _ll_register_callbacks cb 0x7fc9740b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc97fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba0c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc97fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba0c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46736/0 (socket says 172.21.15.69:46736) 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc985800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000ba07f20 0x56000ba7d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc97fe00640 1 -- 172.21.15.69:0/1782122920 learned_addr learned my addr 172.21.15.69:0/1782122920 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.175+0000 7fc984e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9fea20 0x56000ba0c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc97fe00640 1 -- 172.21.15.69:0/1782122920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000ba07f20 msgr2=0x56000ba7d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc97fe00640 1 --2- 172.21.15.69:0/1782122920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000ba07f20 0x56000ba7d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc97fe00640 1 -- 172.21.15.69:0/1782122920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9fea20 msgr2=0x56000ba0c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc97fe00640 1 --2- 172.21.15.69:0/1782122920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9fea20 0x56000ba0c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc97fe00640 1 -- 172.21.15.69:0/1782122920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000ba7d890 con 0x56000b9ff340 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc97fe00640 1 --2- 172.21.15.69:0/1782122920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba0c650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x56000b9fa2e0 tx=0x7fc96c012ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48cdd744e0a9da39 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc966a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc966a00640 1 -- 172.21.15.69:0/1782122920 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c01fa40 con 0x56000b9ff340 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc966a00640 1 -- 172.21.15.69:0/1782122920 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc96c01fbe0 con 0x56000b9ff340 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1782122920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc97409bdc0 con 0x56000b9ff340 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc966a00640 1 -- 172.21.15.69:0/1782122920 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc96c034050 con 0x56000b9ff340 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc98a5cf740 10 client.25927 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.176+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1782122920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56000ba7ebf0 con 0x56000b9ff340 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.177+0000 7fc966a00640 1 -- 172.21.15.69:0/1782122920 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97409bdc0 con 0x56000b9ff340 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.177+0000 7fc966a00640 1 -- 172.21.15.69:0/1782122920 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc96c02b6b0 con 0x56000b9ff340 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.177+0000 7fc98a5cf740 10 client.25927 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:35.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.177+0000 7fc98a5cf740 10 client.25927 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:11:35.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.177+0000 7fc98a5cf740 20 client.25927 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:35.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.177+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/1782122920 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56000ba7f350 0x56000ba81770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.177+0000 7fc98a5cf740 4 client.25927 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:11:35.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.177+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1782122920 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56000b96ac10 con 0x56000ba7f350 2024-03-19T01:11:35.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.177+0000 7fc984e00640 1 --2- 172.21.15.69:0/1782122920 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56000ba7f350 0x56000ba81770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc984e00640 1 --2- 172.21.15.69:0/1782122920 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56000ba7f350 0x56000ba81770 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38592a636a77e85c server_cookie=b504b7a9bd5bdc03 in_seq=0 out_seq=0 2024-03-19T01:11:35.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc966a00640 10 client.25927 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:11:35.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc966a00640 1 -- 172.21.15.69:0/1782122920 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56000b96ac10 con 0x56000ba7f350 2024-03-19T01:11:35.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc966a00640 10 client.25927 handle_command_reply: tid=0 2024-03-19T01:11:35.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc98a5cf740 1 client.25927 shutdown 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1782122920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 msgr2=0x56000ba0c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/1782122920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba0c650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x56000b9fa2e0 tx=0x7fc96c012ce0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1782122920 shutdown_connections 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/1782122920 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56000ba7f350 0x56000ba81770 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/1782122920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56000ba07f20 0x56000ba7d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/1782122920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56000b9ff340 0x56000ba0c650 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc966a00640 0 client.25927 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc98a5cf740 1 --2- 172.21.15.69:0/1782122920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56000b9fea20 0x56000ba0c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1782122920 >> 172.21.15.69:0/1782122920 conn(0x56000b9f9150 msgr2=0x56000b9f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1782122920 shutdown_connections 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.178+0000 7fc98a5cf740 1 -- 172.21.15.69:0/1782122920 wait complete. 2024-03-19T01:11:35.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc98a5cf740 20 client.25927 trim_cache size 0 max 16384 2024-03-19T01:11:35.186 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:11:35.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc987800640 1 -- 172.21.15.69:0/3550910303 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc9680a2ac0 msgr2=0x7fc9680a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc987800640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc9680a2ac0 0x7fc9680a4f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc978004af0 tx=0x7fc978005fd0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc987800640 1 -- 172.21.15.69:0/3550910303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc980102a60 msgr2=0x7fc9801006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc987800640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc980102a60 0x7fc9801006b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc97400e9f0 tx=0x7fc97400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc987800640 1 -- 172.21.15.69:0/3550910303 shutdown_connections 2024-03-19T01:11:35.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc987800640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc9680a2ac0 0x7fc9680a4f80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc987800640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc98010be30 0x7fc980101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc987800640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc980103380 0x7fc980100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc987800640 1 --2- 172.21.15.69:0/3550910303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc980102a60 0x7fc9801006b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc987800640 1 -- 172.21.15.69:0/3550910303 >> 172.21.15.69:0/3550910303 conn(0x7fc980076aa0 msgr2=0x7fc9800fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.179+0000 7fc987800640 1 -- 172.21.15.69:0/3550910303 shutdown_connections 2024-03-19T01:11:35.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.180+0000 7fc987800640 1 -- 172.21.15.69:0/3550910303 wait complete. 2024-03-19T01:11:35.199 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:11:35.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:35 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/209735826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:11:35.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.428+0000 7ff363e00640 1 -- 172.21.15.69:0/4208585357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3641038a0 msgr2=0x7ff364103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.428+0000 7ff363e00640 1 --2- 172.21.15.69:0/4208585357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3641038a0 0x7ff364103d20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff350009900 tx=0x7ff35002bc50 comp rx=0 tx=0).stop 2024-03-19T01:11:35.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.429+0000 7ff363e00640 1 -- 172.21.15.69:0/4208585357 shutdown_connections 2024-03-19T01:11:35.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.429+0000 7ff363e00640 1 --2- 172.21.15.69:0/4208585357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff364104260 0x7ff364110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.429+0000 7ff363e00640 1 --2- 172.21.15.69:0/4208585357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3641038a0 0x7ff364103d20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.429+0000 7ff363e00640 1 --2- 172.21.15.69:0/4208585357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3641098b0 0x7ff364109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.429+0000 7ff363e00640 1 -- 172.21.15.69:0/4208585357 >> 172.21.15.69:0/4208585357 conn(0x7ff3640ff330 msgr2=0x7ff364101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.429+0000 7ff363e00640 1 -- 172.21.15.69:0/4208585357 shutdown_connections 2024-03-19T01:11:35.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.429+0000 7ff363e00640 1 -- 172.21.15.69:0/4208585357 wait complete. 2024-03-19T01:11:35.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.430+0000 7ff363e00640 1 Processor -- start 2024-03-19T01:11:35.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.430+0000 7ff363e00640 1 -- start start 2024-03-19T01:11:35.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.430+0000 7ff363e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3641038a0 0x7ff36419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.430+0000 7ff363e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff364104260 0x7ff36419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.430+0000 7ff363e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3641098b0 0x7ff36419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.430+0000 7ff363e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff364113d10 con 0x7ff3641098b0 2024-03-19T01:11:35.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.430+0000 7ff363e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff364113b90 con 0x7ff3641038a0 2024-03-19T01:11:35.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.430+0000 7ff363e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff364113e90 con 0x7ff364104260 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff362a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3641038a0 0x7ff36419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff362000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff364104260 0x7ff36419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff362a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3641038a0 0x7ff36419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52488/0 (socket says 172.21.15.69:52488) 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff362a00640 1 -- 172.21.15.69:0/2862224829 learned_addr learned my addr 172.21.15.69:0/2862224829 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff363400640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3641098b0 0x7ff36419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff362a00640 1 -- 172.21.15.69:0/2862224829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff364104260 msgr2=0x7ff36419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff362a00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff364104260 0x7ff36419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff362a00640 1 -- 172.21.15.69:0/2862224829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3641098b0 msgr2=0x7ff36419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff362a00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3641098b0 0x7ff36419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff362a00640 1 -- 172.21.15.69:0/2862224829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36419f1e0 con 0x7ff3641038a0 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff362000640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff364104260 0x7ff36419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:35.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.431+0000 7ff362a00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3641038a0 0x7ff36419a1f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff35400e9f0 tx=0x7ff35400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.432+0000 7ff368a00640 1 -- 172.21.15.69:0/2862224829 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35400cd80 con 0x7ff3641038a0 2024-03-19T01:11:35.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.432+0000 7ff368a00640 1 -- 172.21.15.69:0/2862224829 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff354010910 con 0x7ff3641038a0 2024-03-19T01:11:35.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.432+0000 7ff363e00640 1 -- 172.21.15.69:0/2862224829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff36419f4d0 con 0x7ff3641038a0 2024-03-19T01:11:35.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.432+0000 7ff368a00640 1 -- 172.21.15.69:0/2862224829 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354018990 con 0x7ff3641038a0 2024-03-19T01:11:35.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.432+0000 7ff363e00640 1 -- 172.21.15.69:0/2862224829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3641a6d60 con 0x7ff3641038a0 2024-03-19T01:11:35.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.435+0000 7ff36cb83740 1 Processor -- start 2024-03-19T01:11:35.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.435+0000 7ff36cb83740 1 -- start start 2024-03-19T01:11:35.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.435+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1d7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.435+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1e0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1e10f0 0x55696f1e34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff36cb83740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55696f13c380 con 0x55696f1e10f0 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff36cb83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55696f0e4d20 con 0x55696f1d7bf0 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff36cb83740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55696f138420 con 0x55696f1d8510 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff368a00640 1 -- 172.21.15.69:0/2862224829 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7ff354010ac0 con 0x7ff3641038a0 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff363400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1d7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff362000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1e10f0 0x55696f1e34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff363400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1d7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52490/0 (socket says 172.21.15.69:52490) 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff363400640 1 -- 172.21.15.69:0/1690691196 learned_addr learned my addr 172.21.15.69:0/1690691196 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff362a00640 1 --2- 172.21.15.69:0/1690691196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1e0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff363400640 1 -- 172.21.15.69:0/1690691196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 msgr2=0x55696f1e0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff363400640 1 --2- 172.21.15.69:0/1690691196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1e0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.436+0000 7ff363400640 1 -- 172.21.15.69:0/1690691196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1e10f0 msgr2=0x55696f1e34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.437+0000 7ff363400640 1 --2- 172.21.15.69:0/1690691196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1e10f0 0x55696f1e34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.437+0000 7ff363400640 1 -- 172.21.15.69:0/1690691196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696ee503f0 con 0x55696f1d7bf0 2024-03-19T01:11:35.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.437+0000 7ff362000640 1 --2- 172.21.15.69:0/1690691196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1e10f0 0x55696f1e34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:35.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.437+0000 7ff368a00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3480a2b10 0x7ff3480a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.437+0000 7ff363400640 1 --2- 172.21.15.69:0/1690691196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1d7fd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0039f0 tx=0x7ff35c003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e63fddd1bcfc115f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.438+0000 7ff368a00640 1 -- 172.21.15.69:0/2862224829 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7ff3540cf750 con 0x7ff3641038a0 2024-03-19T01:11:35.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.438+0000 7ff34fe00640 1 -- 172.21.15.69:0/1690691196 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c01a470 con 0x55696f1d7bf0 2024-03-19T01:11:35.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.438+0000 7ff362a00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3480a2b10 0x7ff3480a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.438+0000 7ff34fe00640 1 -- 172.21.15.69:0/1690691196 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff35c015070 con 0x55696f1d7bf0 2024-03-19T01:11:35.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.438+0000 7ff34fe00640 1 -- 172.21.15.69:0/1690691196 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c010900 con 0x55696f1d7bf0 2024-03-19T01:11:35.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.439+0000 7ff36cb83740 1 -- 172.21.15.69:0/1690691196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 msgr2=0x55696f1d7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.439+0000 7ff36cb83740 1 --2- 172.21.15.69:0/1690691196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1d7fd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0039f0 tx=0x7ff35c003eb0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.439+0000 7ff362a00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3480a2b10 0x7ff3480a4fd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff3540cc960 tx=0x7ff354000bc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.440+0000 7ff36cb83740 1 -- 172.21.15.69:0/1690691196 shutdown_connections 2024-03-19T01:11:35.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.440+0000 7ff36cb83740 1 --2- 172.21.15.69:0/1690691196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1e10f0 0x55696f1e34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.440+0000 7ff36cb83740 1 --2- 172.21.15.69:0/1690691196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1e0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.440+0000 7ff36cb83740 1 --2- 172.21.15.69:0/1690691196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1d7fd0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.440+0000 7ff36cb83740 1 -- 172.21.15.69:0/1690691196 >> 172.21.15.69:0/1690691196 conn(0x55696f1d2320 msgr2=0x55696f1d2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.440+0000 7ff36cb83740 1 -- 172.21.15.69:0/1690691196 shutdown_connections 2024-03-19T01:11:35.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.440+0000 7ff36cb83740 1 -- 172.21.15.69:0/1690691196 wait complete. 2024-03-19T01:11:35.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.440+0000 7ff36cb83740 1 -- start start 2024-03-19T01:11:35.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.440+0000 7ff36cb83740 1 Processor -- start 2024-03-19T01:11:35.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1de180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d8510 0x55696f1de6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1e10f0 0x55696f1dec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff36cb83740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55696f1e3ee0 con 0x55696f1d8510 2024-03-19T01:11:35.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff36cb83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55696f0f8160 con 0x55696f1d7bf0 2024-03-19T01:11:35.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff36cb83740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55696f1e4060 con 0x55696f1e10f0 2024-03-19T01:11:35.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff36cb83740 10 client.0 _ll_register_callbacks cb 0x55696f134170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:35.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff363400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1de180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff363400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1de180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52498/0 (socket says 172.21.15.69:52498) 2024-03-19T01:11:35.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff363400640 1 -- 172.21.15.69:0/660161306 learned_addr learned my addr 172.21.15.69:0/660161306 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff362000640 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d8510 0x55696f1de6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.441+0000 7ff363400640 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1e10f0 0x55696f1dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.442+0000 7ff363400640 1 -- 172.21.15.69:0/660161306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1e10f0 msgr2=0x55696f1dec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.442+0000 7ff363400640 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1e10f0 0x55696f1dec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.442+0000 7ff363400640 1 -- 172.21.15.69:0/660161306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d8510 msgr2=0x55696f1de6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.442+0000 7ff363400640 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d8510 0x55696f1de6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.442+0000 7ff363400640 1 -- 172.21.15.69:0/660161306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696ef1a7d0 con 0x55696f1d7bf0 2024-03-19T01:11:35.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.442+0000 7ff362000640 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d8510 0x55696f1de6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:35.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.442+0000 7ff363400640 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1de180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0105e0 tx=0x7ff35c0104e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f5bf0f2fce2e13 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.442+0000 7ff34cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:35.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.443+0000 7ff34cc00640 1 -- 172.21.15.69:0/660161306 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c0117b0 con 0x55696f1d7bf0 2024-03-19T01:11:35.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.443+0000 7ff34cc00640 1 -- 172.21.15.69:0/660161306 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7ff35c011950 con 0x55696f1d7bf0 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.443+0000 7ff36cb83740 1 -- 172.21.15.69:0/660161306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff35c015070 con 0x55696f1d7bf0 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.443+0000 7ff34cc00640 1 -- 172.21.15.69:0/660161306 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff35c011b50 con 0x55696f1d7bf0 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.443+0000 7ff36cb83740 10 client.25946 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.443+0000 7ff36cb83740 1 -- 172.21.15.69:0/660161306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55696f1f89e0 con 0x55696f1d7bf0 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.443+0000 7ff34cc00640 1 -- 172.21.15.69:0/660161306 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff35c015070 con 0x55696f1d7bf0 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.444+0000 7ff34cc00640 1 -- 172.21.15.69:0/660161306 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff35c01a630 con 0x55696f1d7bf0 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.444+0000 7ff36cb83740 10 client.25946 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.444+0000 7ff36cb83740 10 client.25946 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.444+0000 7ff36cb83740 20 client.25946 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.444+0000 7ff36cb83740 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55696f1f9d90 0x55696f1fc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.444+0000 7ff36cb83740 4 client.25946 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.444+0000 7ff36cb83740 1 -- 172.21.15.69:0/660161306 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55696f1498c0 con 0x55696f1f9d90 2024-03-19T01:11:35.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.444+0000 7ff362000640 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55696f1f9d90 0x55696f1fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.445+0000 7ff362000640 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55696f1f9d90 0x55696f1fc180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=329c8f0abe74b165 server_cookie=408fa28329af4bb0 in_seq=0 out_seq=0 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.445+0000 7ff34cc00640 10 client.25946 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.446+0000 7ff34cc00640 1 -- 172.21.15.69:0/660161306 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55696f1498c0 con 0x55696f1f9d90 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.446+0000 7ff34cc00640 10 client.25946 handle_command_reply: tid=0 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.446+0000 7ff36cb83740 1 client.25946 shutdown 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.446+0000 7ff36cb83740 1 -- 172.21.15.69:0/660161306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 msgr2=0x55696f1de180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.446+0000 7ff36cb83740 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1de180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0105e0 tx=0x7ff35c0104e0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.446+0000 7ff36cb83740 1 -- 172.21.15.69:0/660161306 shutdown_connections 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.446+0000 7ff36cb83740 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1e10f0 0x55696f1dec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.447+0000 7ff36cb83740 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d8510 0x55696f1de6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.447+0000 7ff36cb83740 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55696f1f9d90 0x55696f1fc180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.447+0000 7ff36cb83740 1 --2- 172.21.15.69:0/660161306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1de180 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.447+0000 7ff36cb83740 1 -- 172.21.15.69:0/660161306 >> 172.21.15.69:0/660161306 conn(0x55696f1d2320 msgr2=0x55696f1d4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.447+0000 7ff34cc00640 0 client.25946 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:11:35.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.447+0000 7ff36cb83740 1 -- 172.21.15.69:0/660161306 shutdown_connections 2024-03-19T01:11:35.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.447+0000 7ff36cb83740 1 -- 172.21.15.69:0/660161306 wait complete. 2024-03-19T01:11:35.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.447+0000 7ff36cb83740 20 client.25946 trim_cache size 0 max 16384 2024-03-19T01:11:35.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.461+0000 7ff36cb83740 1 Processor -- start 2024-03-19T01:11:35.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.461+0000 7ff36cb83740 1 -- start start 2024-03-19T01:11:35.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.461+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1f5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.461+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1dd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.461+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1e10f0 0x55696f201830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.461+0000 7ff36cb83740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641aca10 con 0x55696f1e10f0 2024-03-19T01:11:35.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.461+0000 7ff36cb83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641ae550 con 0x55696f1d7bf0 2024-03-19T01:11:35.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.461+0000 7ff36cb83740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641ad0e0 con 0x55696f1d8510 2024-03-19T01:11:35.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.462+0000 7ff362000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1dd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.462+0000 7ff362000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1dd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46786/0 (socket says 172.21.15.69:46786) 2024-03-19T01:11:35.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.462+0000 7ff362000640 1 -- 172.21.15.69:0/1165648354 learned_addr learned my addr 172.21.15.69:0/1165648354 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.462+0000 7ff363400640 1 --2- 172.21.15.69:0/1165648354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.462+0000 7ff363400640 1 --2- 172.21.15.69:0/1165648354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1e10f0 0x55696f201830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.462+0000 7ff362000640 1 -- 172.21.15.69:0/1165648354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 msgr2=0x55696f1f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.462+0000 7ff362000640 1 --2- 172.21.15.69:0/1165648354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.462+0000 7ff362000640 1 -- 172.21.15.69:0/1165648354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1e10f0 msgr2=0x55696f201830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.462+0000 7ff362000640 1 --2- 172.21.15.69:0/1165648354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1e10f0 0x55696f201830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.462+0000 7ff362000640 1 -- 172.21.15.69:0/1165648354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696f1d8340 con 0x55696f1d8510 2024-03-19T01:11:35.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.462+0000 7ff362000640 1 --2- 172.21.15.69:0/1165648354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1dd930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff350000c00 tx=0x7ff35002e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98751a0afe2f901b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.463+0000 7ff34f400640 1 -- 172.21.15.69:0/1165648354 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35004d030 con 0x55696f1d8510 2024-03-19T01:11:35.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.463+0000 7ff34f400640 1 -- 172.21.15.69:0/1165648354 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff350049bd0 con 0x55696f1d8510 2024-03-19T01:11:35.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.463+0000 7ff34f400640 1 -- 172.21.15.69:0/1165648354 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35002cb30 con 0x55696f1d8510 2024-03-19T01:11:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.463+0000 7ff36cb83740 1 -- 172.21.15.69:0/1165648354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 msgr2=0x55696f1dd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.464+0000 7ff36cb83740 1 --2- 172.21.15.69:0/1165648354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1dd930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff350000c00 tx=0x7ff35002e040 comp rx=0 tx=0).stop 2024-03-19T01:11:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.464+0000 7ff36cb83740 1 -- 172.21.15.69:0/1165648354 shutdown_connections 2024-03-19T01:11:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.464+0000 7ff36cb83740 1 --2- 172.21.15.69:0/1165648354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1e10f0 0x55696f201830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.464+0000 7ff36cb83740 1 --2- 172.21.15.69:0/1165648354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1dd930 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.464+0000 7ff36cb83740 1 --2- 172.21.15.69:0/1165648354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1d7bf0 0x55696f1f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.464+0000 7ff36cb83740 1 -- 172.21.15.69:0/1165648354 >> 172.21.15.69:0/1165648354 conn(0x55696f1d2320 msgr2=0x55696f1d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.464+0000 7ff36cb83740 1 -- 172.21.15.69:0/1165648354 shutdown_connections 2024-03-19T01:11:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.464+0000 7ff36cb83740 1 -- 172.21.15.69:0/1165648354 wait complete. 2024-03-19T01:11:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.464+0000 7ff36cb83740 1 -- start start 2024-03-19T01:11:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff36cb83740 1 Processor -- start 2024-03-19T01:11:35.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d7bf0 0x55696f1eccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1ed230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff36cb83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1e10f0 0x55696f25de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff36cb83740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641ade80 con 0x55696f1d7bf0 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff36cb83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641ad7b0 con 0x55696f1e10f0 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff36cb83740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641aa9f0 con 0x55696f1d8510 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff36cb83740 10 client.0 _ll_register_callbacks cb 0x7ff350009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff363400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d7bf0 0x55696f1eccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff362000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1ed230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff363400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d7bf0 0x55696f1eccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51772/0 (socket says 172.21.15.69:51772) 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.465+0000 7ff363400640 1 -- 172.21.15.69:0/670559367 learned_addr learned my addr 172.21.15.69:0/670559367 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.466+0000 7ff363400640 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1e10f0 0x55696f25de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.466+0000 7ff362000640 1 -- 172.21.15.69:0/670559367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1e10f0 msgr2=0x55696f25de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.466+0000 7ff362000640 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1e10f0 0x55696f25de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.466+0000 7ff362000640 1 -- 172.21.15.69:0/670559367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d7bf0 msgr2=0x55696f1eccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.466+0000 7ff362000640 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d7bf0 0x55696f1eccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.466+0000 7ff362000640 1 -- 172.21.15.69:0/670559367 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696f25e390 con 0x55696f1d8510 2024-03-19T01:11:35.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.466+0000 7ff362000640 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1ed230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55696f1d2950 tx=0x7ff350049580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27657c1951d755ef server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.466+0000 7ff34cc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.466+0000 7ff34cc00640 1 -- 172.21.15.69:0/670559367 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3500044d0 con 0x55696f1d8510 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.466+0000 7ff34cc00640 1 -- 172.21.15.69:0/670559367 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7ff35002c8d0 con 0x55696f1d8510 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.466+0000 7ff36cb83740 1 -- 172.21.15.69:0/670559367 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff350049bd0 con 0x55696f1d8510 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff34cc00640 1 -- 172.21.15.69:0/670559367 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff35002cb30 con 0x55696f1d8510 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff36cb83740 10 client.25945 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff36cb83740 1 -- 172.21.15.69:0/670559367 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55696f25f590 con 0x55696f1d8510 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff34cc00640 1 -- 172.21.15.69:0/670559367 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff350049bd0 con 0x55696f1d8510 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff34cc00640 1 -- 172.21.15.69:0/670559367 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff3500037f0 con 0x55696f1d8510 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff36cb83740 10 client.25945 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff36cb83740 10 client.25945 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff36cb83740 20 client.25945 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff36cb83740 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55696f25fc80 0x55696f2620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff36cb83740 4 client.25945 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:11:35.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff36cb83740 1 -- 172.21.15.69:0/670559367 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55696f143c10 con 0x55696f25fc80 2024-03-19T01:11:35.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.467+0000 7ff363400640 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55696f25fc80 0x55696f2620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.468+0000 7ff363400640 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55696f25fc80 0x55696f2620a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92237ce1e72c20c6 server_cookie=38cd7d289272a4c5 in_seq=0 out_seq=0 2024-03-19T01:11:35.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.468+0000 7ff34cc00640 10 client.25945 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:11:35.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.468+0000 7ff34cc00640 1 -- 172.21.15.69:0/670559367 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55696f143c10 con 0x55696f25fc80 2024-03-19T01:11:35.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.468+0000 7ff34cc00640 10 client.25945 handle_command_reply: tid=0 2024-03-19T01:11:35.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.468+0000 7ff36cb83740 1 client.25945 shutdown 2024-03-19T01:11:35.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff36cb83740 1 -- 172.21.15.69:0/670559367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 msgr2=0x55696f1ed230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff36cb83740 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1ed230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55696f1d2950 tx=0x7ff350049580 comp rx=0 tx=0).stop 2024-03-19T01:11:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff36cb83740 1 -- 172.21.15.69:0/670559367 shutdown_connections 2024-03-19T01:11:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff36cb83740 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55696f25fc80 0x55696f2620a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff36cb83740 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55696f1e10f0 0x55696f25de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff36cb83740 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55696f1d8510 0x55696f1ed230 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff36cb83740 1 --2- 172.21.15.69:0/670559367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55696f1d7bf0 0x55696f1eccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff36cb83740 1 -- 172.21.15.69:0/670559367 >> 172.21.15.69:0/670559367 conn(0x55696f1d2320 msgr2=0x55696f1de140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff34cc00640 0 client.25945 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:11:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff36cb83740 1 -- 172.21.15.69:0/670559367 shutdown_connections 2024-03-19T01:11:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff36cb83740 1 -- 172.21.15.69:0/670559367 wait complete. 2024-03-19T01:11:35.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.469+0000 7ff36cb83740 20 client.25945 trim_cache size 0 max 16384 2024-03-19T01:11:35.474 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:11:35.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 -- 172.21.15.69:0/2862224829 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3480a2b10 msgr2=0x7ff3480a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3480a2b10 0x7ff3480a4fd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff3540cc960 tx=0x7ff354000bc0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 -- 172.21.15.69:0/2862224829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3641038a0 msgr2=0x7ff36419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3641038a0 0x7ff36419a1f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff35400e9f0 tx=0x7ff35400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 -- 172.21.15.69:0/2862224829 shutdown_connections 2024-03-19T01:11:35.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3641098b0 0x7ff36419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff364104260 0x7ff36419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3480a2b10 0x7ff3480a4fd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 --2- 172.21.15.69:0/2862224829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3641038a0 0x7ff36419a1f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 -- 172.21.15.69:0/2862224829 >> 172.21.15.69:0/2862224829 conn(0x7ff3640ff330 msgr2=0x7ff3640ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 -- 172.21.15.69:0/2862224829 shutdown_connections 2024-03-19T01:11:35.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.470+0000 7ff363e00640 1 -- 172.21.15.69:0/2862224829 wait complete. 2024-03-19T01:11:35.493 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:11:35.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:35 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/209735826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:11:35.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:35 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/209735826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:11:35.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.669+0000 7fcac1e00640 1 -- 172.21.15.69:0/2126813237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc1099d0 msgr2=0x7fcabc109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.669+0000 7fcac1e00640 1 --2- 172.21.15.69:0/2126813237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc1099d0 0x7fcabc109db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009ca0 tx=0x7fcaac02c100 comp rx=0 tx=0).stop 2024-03-19T01:11:35.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.669+0000 7fcac1e00640 1 -- 172.21.15.69:0/2126813237 shutdown_connections 2024-03-19T01:11:35.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.669+0000 7fcac1e00640 1 --2- 172.21.15.69:0/2126813237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcabc104380 0x7fcabc110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.669+0000 7fcac1e00640 1 --2- 172.21.15.69:0/2126813237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcabc1039c0 0x7fcabc103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.669+0000 7fcac1e00640 1 --2- 172.21.15.69:0/2126813237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc1099d0 0x7fcabc109db0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.669+0000 7fcac1e00640 1 -- 172.21.15.69:0/2126813237 >> 172.21.15.69:0/2126813237 conn(0x7fcabc0ff4d0 msgr2=0x7fcabc1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.669+0000 7fcac1e00640 1 -- 172.21.15.69:0/2126813237 shutdown_connections 2024-03-19T01:11:35.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.669+0000 7fcac1e00640 1 -- 172.21.15.69:0/2126813237 wait complete. 2024-03-19T01:11:35.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.670+0000 7fcac1e00640 1 Processor -- start 2024-03-19T01:11:35.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.670+0000 7fcac1e00640 1 -- start start 2024-03-19T01:11:35.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.670+0000 7fcac1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc1039c0 0x7fcabc19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcac1e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcabc104380 0x7fcabc19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcac1e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcabc1099d0 0x7fcabc19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcac1e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc113db0 con 0x7fcabc104380 2024-03-19T01:11:35.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcac1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc113c30 con 0x7fcabc1039c0 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcac1e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc113f30 con 0x7fcabc1099d0 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcabaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc1039c0 0x7fcabc19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcabaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc1039c0 0x7fcabc19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52532/0 (socket says 172.21.15.69:52532) 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcabaa00640 1 -- 172.21.15.69:0/3465119604 learned_addr learned my addr 172.21.15.69:0/3465119604 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcabb400640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcabc1099d0 0x7fcabc19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcab2000640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcabc104380 0x7fcabc19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcabaa00640 1 -- 172.21.15.69:0/3465119604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcabc1099d0 msgr2=0x7fcabc19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcabaa00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcabc1099d0 0x7fcabc19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcabaa00640 1 -- 172.21.15.69:0/3465119604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcabc104380 msgr2=0x7fcabc19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcabaa00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcabc104380 0x7fcabc19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcabaa00640 1 -- 172.21.15.69:0/3465119604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc19f2f0 con 0x7fcabc1039c0 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.671+0000 7fcabb400640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcabc1099d0 0x7fcabc19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.672+0000 7fcabaa00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc1039c0 0x7fcabc19a350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcaac005e30 tx=0x7fcaac0346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.672+0000 7fcac0a00640 1 -- 172.21.15.69:0/3465119604 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac039050 con 0x7fcabc1039c0 2024-03-19T01:11:35.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.672+0000 7fcac0a00640 1 -- 172.21.15.69:0/3465119604 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaac045070 con 0x7fcabc1039c0 2024-03-19T01:11:35.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.672+0000 7fcac0a00640 1 -- 172.21.15.69:0/3465119604 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac0046c0 con 0x7fcabc1039c0 2024-03-19T01:11:35.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.672+0000 7fcac1e00640 1 -- 172.21.15.69:0/3465119604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc19f580 con 0x7fcabc1039c0 2024-03-19T01:11:35.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.672+0000 7fcac1e00640 1 -- 172.21.15.69:0/3465119604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1a6e70 con 0x7fcabc1039c0 2024-03-19T01:11:35.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.676+0000 7fcac4bed740 1 Processor -- start 2024-03-19T01:11:35.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.676+0000 7fcac4bed740 1 -- start start 2024-03-19T01:11:35.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.676+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef28610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28b50 0x555d4ef311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef31730 0x555d4ef33b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcac4bed740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555d4ee84380 con 0x555d4ef31730 2024-03-19T01:11:35.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcac4bed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555d4ee2cd20 con 0x555d4ef28230 2024-03-19T01:11:35.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcac4bed740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555d4ee80420 con 0x555d4ef28b50 2024-03-19T01:11:35.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcabb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef28610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcabb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef28610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52546/0 (socket says 172.21.15.69:52546) 2024-03-19T01:11:35.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcabb400640 1 -- 172.21.15.69:0/4019178187 learned_addr learned my addr 172.21.15.69:0/4019178187 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcac0a00640 1 -- 172.21.15.69:0/3465119604 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7fcaac003970 con 0x7fcabc1039c0 2024-03-19T01:11:35.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcab2000640 1 --2- 172.21.15.69:0/4019178187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef31730 0x555d4ef33b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcabb400640 1 -- 172.21.15.69:0/4019178187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28b50 msgr2=0x555d4ef311f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:11:35.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcabb400640 1 --2- 172.21.15.69:0/4019178187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28b50 0x555d4ef311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcabb400640 1 -- 172.21.15.69:0/4019178187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef31730 msgr2=0x555d4ef33b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcabb400640 1 --2- 172.21.15.69:0/4019178187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef31730 0x555d4ef33b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcabb400640 1 -- 172.21.15.69:0/4019178187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4eb983f0 con 0x555d4ef28230 2024-03-19T01:11:35.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.677+0000 7fcab2000640 1 --2- 172.21.15.69:0/4019178187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef31730 0x555d4ef33b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:11:35.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.678+0000 7fcabb400640 1 --2- 172.21.15.69:0/4019178187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef28610 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcab4010400 tx=0x7fcab40108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b3752518d1ed3cf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.678+0000 7fcaabe00640 1 -- 172.21.15.69:0/4019178187 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4011760 con 0x555d4ef28230 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.678+0000 7fcaabe00640 1 -- 172.21.15.69:0/4019178187 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab4011de0 con 0x555d4ef28230 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.678+0000 7fcaabe00640 1 -- 172.21.15.69:0/4019178187 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4018380 con 0x555d4ef28230 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.679+0000 7fcac0a00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaa00a2b10 0x7fcaa00a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.679+0000 7fcac0a00640 1 -- 172.21.15.69:0/3465119604 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fcaac0f1a00 con 0x7fcabc1039c0 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.679+0000 7fcac4bed740 1 -- 172.21.15.69:0/4019178187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 msgr2=0x555d4ef28610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.679+0000 7fcac4bed740 1 --2- 172.21.15.69:0/4019178187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef28610 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcab4010400 tx=0x7fcab40108c0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.679+0000 7fcabaa00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaa00a2b10 0x7fcaa00a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.679+0000 7fcac4bed740 1 -- 172.21.15.69:0/4019178187 shutdown_connections 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.679+0000 7fcac4bed740 1 --2- 172.21.15.69:0/4019178187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef31730 0x555d4ef33b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.679+0000 7fcac4bed740 1 --2- 172.21.15.69:0/4019178187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28b50 0x555d4ef311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.679+0000 7fcac4bed740 1 --2- 172.21.15.69:0/4019178187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef28610 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.679+0000 7fcac4bed740 1 -- 172.21.15.69:0/4019178187 >> 172.21.15.69:0/4019178187 conn(0x555d4ef22960 msgr2=0x555d4ef22d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.680+0000 7fcac4bed740 1 -- 172.21.15.69:0/4019178187 shutdown_connections 2024-03-19T01:11:35.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.680+0000 7fcac4bed740 1 -- 172.21.15.69:0/4019178187 wait complete. 2024-03-19T01:11:35.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.680+0000 7fcabaa00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaa00a2b10 0x7fcaa00a4fd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0eed80 tx=0x7fcaac0095c0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.680+0000 7fcac4bed740 1 -- start start 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.681+0000 7fcac4bed740 1 Processor -- start 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.681+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef2e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.681+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef2ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.681+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef31730 0x555d4ef2f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.681+0000 7fcac4bed740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555d4ef34520 con 0x555d4ef28b50 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.681+0000 7fcac4bed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555d4ee40160 con 0x555d4ef28230 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.681+0000 7fcac4bed740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555d4ef346a0 con 0x555d4ef31730 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.681+0000 7fcac4bed740 10 client.0 _ll_register_callbacks cb 0x555d4ee7c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcab2000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef2ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcabb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef2e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcab2000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef2ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51808/0 (socket says 172.21.15.69:51808) 2024-03-19T01:11:35.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcab2000640 1 -- 172.21.15.69:0/1365211118 learned_addr learned my addr 172.21.15.69:0/1365211118 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcabb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef2e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52562/0 (socket says 172.21.15.69:52562) 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcabb400640 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef31730 0x555d4ef2f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcab2000640 1 -- 172.21.15.69:0/1365211118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef31730 msgr2=0x555d4ef2f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcab2000640 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef31730 0x555d4ef2f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcab2000640 1 -- 172.21.15.69:0/1365211118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 msgr2=0x555d4ef2e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcab2000640 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef2e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcab2000640 1 -- 172.21.15.69:0/1365211118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4ec627d0 con 0x555d4ef28b50 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.682+0000 7fcabb400640 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef2e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.683+0000 7fcab2000640 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef2ecc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4012730 tx=0x7fcaa4012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caca4d5c2413d118 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.683+0000 7fcaa8c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.683+0000 7fcaa8c00640 1 -- 172.21.15.69:0/1365211118 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4004050 con 0x555d4ef28b50 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.683+0000 7fcaa8c00640 1 -- 172.21.15.69:0/1365211118 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fcaa40041f0 con 0x555d4ef28b50 2024-03-19T01:11:35.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.683+0000 7fcac4bed740 1 -- 172.21.15.69:0/1365211118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab4011de0 con 0x555d4ef28b50 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.683+0000 7fcaa8c00640 1 -- 172.21.15.69:0/1365211118 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcaa4024050 con 0x555d4ef28b50 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.683+0000 7fcac4bed740 10 client.15777 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.683+0000 7fcac4bed740 1 -- 172.21.15.69:0/1365211118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d4ef408a0 con 0x555d4ef28b50 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.684+0000 7fcaa8c00640 1 -- 172.21.15.69:0/1365211118 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab4011de0 con 0x555d4ef28b50 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.684+0000 7fcaa8c00640 1 -- 172.21.15.69:0/1365211118 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcaa4003ae0 con 0x555d4ef28b50 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.684+0000 7fcac4bed740 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.684+0000 7fcac4bed740 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.684+0000 7fcac4bed740 20 client.15777 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.684+0000 7fcac4bed740 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555d4ef41e00 0x555d4ef44230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.684+0000 7fcac4bed740 4 client.15777 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.684+0000 7fcac4bed740 1 -- 172.21.15.69:0/1365211118 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d4ee918c0 con 0x555d4ef41e00 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.684+0000 7fcabb400640 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555d4ef41e00 0x555d4ef44230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.685+0000 7fcaa8c00640 10 client.15777 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:35.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.686+0000 7fcabb400640 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555d4ef41e00 0x555d4ef44230 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3fea42495b3562 server_cookie=f0f18f9bc2542061 in_seq=0 out_seq=0 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.686+0000 7fcaa8c00640 1 -- 172.21.15.69:0/1365211118 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555d4ee918c0 con 0x555d4ef41e00 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.686+0000 7fcaa8c00640 10 client.15777 handle_command_reply: tid=0 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.686+0000 7fcac4bed740 1 client.15777 shutdown 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcac4bed740 1 -- 172.21.15.69:0/1365211118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 msgr2=0x555d4ef2ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcac4bed740 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef2ecc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4012730 tx=0x7fcaa4012bf0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcac4bed740 1 -- 172.21.15.69:0/1365211118 shutdown_connections 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcac4bed740 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555d4ef41e00 0x555d4ef44230 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcac4bed740 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef31730 0x555d4ef2f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcac4bed740 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef2ecc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcaa8c00640 0 client.15777 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcac4bed740 1 --2- 172.21.15.69:0/1365211118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef28230 0x555d4ef2e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcac4bed740 1 -- 172.21.15.69:0/1365211118 >> 172.21.15.69:0/1365211118 conn(0x555d4ef22960 msgr2=0x555d4ef246b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcac4bed740 1 -- 172.21.15.69:0/1365211118 shutdown_connections 2024-03-19T01:11:35.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcac4bed740 1 -- 172.21.15.69:0/1365211118 wait complete. 2024-03-19T01:11:35.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.687+0000 7fcac4bed740 20 client.15777 trim_cache size 0 max 16384 2024-03-19T01:11:35.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.716+0000 7fcac4bed740 1 Processor -- start 2024-03-19T01:11:35.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.716+0000 7fcac4bed740 1 -- start start 2024-03-19T01:11:35.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.716+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef30eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.716+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef3d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.716+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef31730 0x555d4ef244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.716+0000 7fcac4bed740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1acb20 con 0x555d4ef28b50 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.716+0000 7fcac4bed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1ae660 con 0x555d4ef31730 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.716+0000 7fcac4bed740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1ad1f0 con 0x555d4ef28230 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcabb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef30eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcabb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef30eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46844/0 (socket says 172.21.15.69:46844) 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcabb400640 1 -- 172.21.15.69:0/2415013524 learned_addr learned my addr 172.21.15.69:0/2415013524 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcabb400640 1 --2- 172.21.15.69:0/2415013524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef31730 0x555d4ef244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcab2000640 1 --2- 172.21.15.69:0/2415013524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef3d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcabb400640 1 -- 172.21.15.69:0/2415013524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef31730 msgr2=0x555d4ef244c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcabb400640 1 --2- 172.21.15.69:0/2415013524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef31730 0x555d4ef244c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcabb400640 1 -- 172.21.15.69:0/2415013524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 msgr2=0x555d4ef3d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcabb400640 1 --2- 172.21.15.69:0/2415013524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef3d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcabb400640 1 -- 172.21.15.69:0/2415013524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4ef3d860 con 0x555d4ef28230 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcab2000640 1 --2- 172.21.15.69:0/2415013524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef3d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:11:35.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.717+0000 7fcabb400640 1 --2- 172.21.15.69:0/2415013524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef30eb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcab4019b00 tx=0x7fcab4010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f954858cd2c993a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.718+0000 7fcaab400640 1 -- 172.21.15.69:0/2415013524 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab40193d0 con 0x555d4ef28230 2024-03-19T01:11:35.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.718+0000 7fcaab400640 1 -- 172.21.15.69:0/2415013524 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab40055b0 con 0x555d4ef28230 2024-03-19T01:11:35.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.718+0000 7fcaab400640 1 -- 172.21.15.69:0/2415013524 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab402e810 con 0x555d4ef28230 2024-03-19T01:11:35.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.719+0000 7fcac4bed740 1 -- 172.21.15.69:0/2415013524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 msgr2=0x555d4ef30eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.719+0000 7fcac4bed740 1 --2- 172.21.15.69:0/2415013524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef30eb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcab4019b00 tx=0x7fcab4010fd0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.719+0000 7fcac4bed740 1 -- 172.21.15.69:0/2415013524 shutdown_connections 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.719+0000 7fcac4bed740 1 --2- 172.21.15.69:0/2415013524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef31730 0x555d4ef244c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.719+0000 7fcac4bed740 1 --2- 172.21.15.69:0/2415013524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef3d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.719+0000 7fcac4bed740 1 --2- 172.21.15.69:0/2415013524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef30eb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.719+0000 7fcac4bed740 1 -- 172.21.15.69:0/2415013524 >> 172.21.15.69:0/2415013524 conn(0x555d4ef22960 msgr2=0x555d4ef23db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.719+0000 7fcac4bed740 1 -- 172.21.15.69:0/2415013524 shutdown_connections 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.719+0000 7fcac4bed740 1 -- 172.21.15.69:0/2415013524 wait complete. 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.719+0000 7fcac4bed740 1 -- start start 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.720+0000 7fcac4bed740 1 Processor -- start 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.720+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef39950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.720+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef36920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.720+0000 7fcac4bed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef31730 0x555d4ef36e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.720+0000 7fcac4bed740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1adf90 con 0x555d4ef28b50 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.720+0000 7fcac4bed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1ad8c0 con 0x555d4ef31730 2024-03-19T01:11:35.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.720+0000 7fcac4bed740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1aab00 con 0x555d4ef28230 2024-03-19T01:11:35.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.720+0000 7fcac4bed740 10 client.0 _ll_register_callbacks cb 0x7fcaa4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:35.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.721+0000 7fcabb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef39950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.721+0000 7fcabb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef39950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46860/0 (socket says 172.21.15.69:46860) 2024-03-19T01:11:35.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.721+0000 7fcabb400640 1 -- 172.21.15.69:0/639743985 learned_addr learned my addr 172.21.15.69:0/639743985 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:35.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.721+0000 7fcab2000640 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef36920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.721+0000 7fcabb400640 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef31730 0x555d4ef36e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.721+0000 7fcabb400640 1 -- 172.21.15.69:0/639743985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef31730 msgr2=0x555d4ef36e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.721+0000 7fcabb400640 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef31730 0x555d4ef36e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.721+0000 7fcabb400640 1 -- 172.21.15.69:0/639743985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 msgr2=0x555d4ef36920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:35.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.721+0000 7fcabb400640 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef36920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.721+0000 7fcabb400640 1 -- 172.21.15.69:0/639743985 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4ef39f90 con 0x555d4ef28230 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcabb400640 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef39950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcab402e2c0 tx=0x7fcab402e3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e33a9dd2adde059 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcaa8c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcaa8c00640 1 -- 172.21.15.69:0/639743985 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab40193d0 con 0x555d4ef28230 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcaa8c00640 1 -- 172.21.15.69:0/639743985 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fcab4037040 con 0x555d4ef28230 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcac4bed740 1 -- 172.21.15.69:0/639743985 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab40055b0 con 0x555d4ef28230 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcaa8c00640 1 -- 172.21.15.69:0/639743985 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcab4019570 con 0x555d4ef28230 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcac4bed740 10 client.25960 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcac4bed740 1 -- 172.21.15.69:0/639743985 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d4efa8b40 con 0x555d4ef28230 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcaa8c00640 1 -- 172.21.15.69:0/639743985 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab40055b0 con 0x555d4ef28230 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcaa8c00640 1 -- 172.21.15.69:0/639743985 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcab4003950 con 0x555d4ef28230 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcac4bed740 10 client.25960 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcac4bed740 10 client.25960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.722+0000 7fcac4bed740 20 client.25960 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.723+0000 7fcac4bed740 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555d4efa8f60 0x555d4efab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.723+0000 7fcac4bed740 4 client.25960 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.723+0000 7fcac4bed740 1 -- 172.21.15.69:0/639743985 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d4ee8bc10 con 0x555d4efa8f60 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.723+0000 7fcab2000640 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555d4efa8f60 0x555d4efab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.723+0000 7fcaa8c00640 10 client.25960 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.723+0000 7fcab2000640 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555d4efa8f60 0x555d4efab380 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c9e2acc15a01828 server_cookie=7bc4ac54eb0b16c5 in_seq=0 out_seq=0 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcaa8c00640 1 -- 172.21.15.69:0/639743985 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d4ee8bc10 con 0x555d4efa8f60 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcaa8c00640 10 client.25960 handle_command_reply: tid=0 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 1 client.25960 shutdown 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 1 -- 172.21.15.69:0/639743985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 msgr2=0x555d4ef39950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef39950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcab402e2c0 tx=0x7fcab402e3f0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 1 -- 172.21.15.69:0/639743985 shutdown_connections 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x555d4efa8f60 0x555d4efab380 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d4ef31730 0x555d4ef36e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d4ef28b50 0x555d4ef36920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 1 --2- 172.21.15.69:0/639743985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555d4ef28230 0x555d4ef39950 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcaa8c00640 0 client.25960 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:35.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 1 -- 172.21.15.69:0/639743985 >> 172.21.15.69:0/639743985 conn(0x555d4ef22960 msgr2=0x555d4ef24640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 1 -- 172.21.15.69:0/639743985 shutdown_connections 2024-03-19T01:11:35.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 1 -- 172.21.15.69:0/639743985 wait complete. 2024-03-19T01:11:35.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.724+0000 7fcac4bed740 20 client.25960 trim_cache size 0 max 16384 2024-03-19T01:11:35.728 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:11:35.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 -- 172.21.15.69:0/3465119604 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaa00a2b10 msgr2=0x7fcaa00a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaa00a2b10 0x7fcaa00a4fd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0eed80 tx=0x7fcaac0095c0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 -- 172.21.15.69:0/3465119604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc1039c0 msgr2=0x7fcabc19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:35.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc1039c0 0x7fcabc19a350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcaac005e30 tx=0x7fcaac0346f0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 -- 172.21.15.69:0/3465119604 shutdown_connections 2024-03-19T01:11:35.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcabc1099d0 0x7fcabc19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcabc104380 0x7fcabc19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaa00a2b10 0x7fcaa00a4fd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 --2- 172.21.15.69:0/3465119604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc1039c0 0x7fcabc19a350 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:35.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 -- 172.21.15.69:0/3465119604 >> 172.21.15.69:0/3465119604 conn(0x7fcabc0ff4d0 msgr2=0x7fcabc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:35.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 -- 172.21.15.69:0/3465119604 shutdown_connections 2024-03-19T01:11:35.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:35.725+0000 7fcac1e00640 1 -- 172.21.15.69:0/3465119604 wait complete. 2024-03-19T01:11:36.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:36 smithi086 ceph-mon[66829]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-19T01:11:36.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:36 smithi069 ceph-mon[31727]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-19T01:11:36.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:36 smithi163 ceph-mon[67428]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-19T01:11:36.747 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:11:36.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:11:36.747 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:11:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.975+0000 7f442b200640 1 -- 172.21.15.69:0/4150434513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f442c104260 msgr2=0x7f442c110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.975+0000 7f442b200640 1 --2- 172.21.15.69:0/4150434513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f442c104260 0x7f442c110770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4424009cd0 tx=0x7f442402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.975+0000 7f442b200640 1 -- 172.21.15.69:0/4150434513 shutdown_connections 2024-03-19T01:11:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.975+0000 7f442b200640 1 --2- 172.21.15.69:0/4150434513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f442c104260 0x7f442c110770 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.975+0000 7f442b200640 1 --2- 172.21.15.69:0/4150434513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c1038a0 0x7f442c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.975+0000 7f442b200640 1 --2- 172.21.15.69:0/4150434513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c1098b0 0x7f442c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.975+0000 7f442b200640 1 -- 172.21.15.69:0/4150434513 >> 172.21.15.69:0/4150434513 conn(0x7f442c0ff330 msgr2=0x7f442c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.975+0000 7f442b200640 1 -- 172.21.15.69:0/4150434513 shutdown_connections 2024-03-19T01:11:36.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.975+0000 7f442b200640 1 -- 172.21.15.69:0/4150434513 wait complete. 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.976+0000 7f442b200640 1 Processor -- start 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.976+0000 7f442b200640 1 -- start start 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.976+0000 7f442b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c1038a0 0x7f442c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.976+0000 7f442b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f442c104260 0x7f442c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.976+0000 7f442b200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c1098b0 0x7f442c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.976+0000 7f442b200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f442c113cc0 con 0x7f442c1098b0 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.976+0000 7f442b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f442c113b40 con 0x7f442c1038a0 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.976+0000 7f442b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f442c113e40 con 0x7f442c104260 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f4429e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c1038a0 0x7f442c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f4429e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c1038a0 0x7f442c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35786/0 (socket says 172.21.15.69:35786) 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f4429e00640 1 -- 172.21.15.69:0/1483250482 learned_addr learned my addr 172.21.15.69:0/1483250482 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f4429400640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f442c104260 0x7f442c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f442a800640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c1098b0 0x7f442c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f4429e00640 1 -- 172.21.15.69:0/1483250482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f442c104260 msgr2=0x7f442c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f4429e00640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f442c104260 0x7f442c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f4429e00640 1 -- 172.21.15.69:0/1483250482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c1098b0 msgr2=0x7f442c19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f4429e00640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c1098b0 0x7f442c19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f4429e00640 1 -- 172.21.15.69:0/1483250482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c19f170 con 0x7f442c1038a0 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f4429400640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f442c104260 0x7f442c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.977+0000 7f4429e00640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c1038a0 0x7f442c19a180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f441c00e9f0 tx=0x7f441c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.978+0000 7f4422a00640 1 -- 172.21.15.69:0/1483250482 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c00cd80 con 0x7f442c1038a0 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.978+0000 7f442b200640 1 -- 172.21.15.69:0/1483250482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c19f460 con 0x7f442c1038a0 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.978+0000 7f4422a00640 1 -- 172.21.15.69:0/1483250482 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f441c005500 con 0x7f442c1038a0 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.978+0000 7f4422a00640 1 -- 172.21.15.69:0/1483250482 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c010690 con 0x7f442c1038a0 2024-03-19T01:11:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.978+0000 7f442b200640 1 -- 172.21.15.69:0/1483250482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c1a6cf0 con 0x7f442c1038a0 2024-03-19T01:11:36.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.981+0000 7f4422a00640 1 -- 172.21.15.69:0/1483250482 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7f441c0046c0 con 0x7f442c1038a0 2024-03-19T01:11:36.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.982+0000 7f443377a740 1 Processor -- start 2024-03-19T01:11:36.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.982+0000 7f443377a740 1 -- start start 2024-03-19T01:11:36.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.982+0000 7f443377a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92998ce0 0x559b929990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:36.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.982+0000 7f443377a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92999600 0x559b929a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:36.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.982+0000 7f443377a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b929a21e0 0x559b929a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:36.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.982+0000 7f443377a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559b928fbfb0 con 0x559b929a21e0 2024-03-19T01:11:36.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f443377a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559b928a4ab0 con 0x559b92999600 2024-03-19T01:11:36.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f443377a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559b928f8050 con 0x559b92998ce0 2024-03-19T01:11:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f442a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92998ce0 0x559b929990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f442a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92998ce0 0x559b929990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36882/0 (socket says 172.21.15.69:36882) 2024-03-19T01:11:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f442a800640 1 -- 172.21.15.69:0/3799496591 learned_addr learned my addr 172.21.15.69:0/3799496591 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f4422a00640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f44100a2a90 0x7f44100a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f4429e00640 1 --2- 172.21.15.69:0/3799496591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92999600 0x559b929a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f4429400640 1 --2- 172.21.15.69:0/3799496591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b929a21e0 0x559b929a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f442a800640 1 -- 172.21.15.69:0/3799496591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92999600 msgr2=0x559b929a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f442a800640 1 --2- 172.21.15.69:0/3799496591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92999600 0x559b929a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f442a800640 1 -- 172.21.15.69:0/3799496591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b929a21e0 msgr2=0x559b929a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f442a800640 1 --2- 172.21.15.69:0/3799496591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b929a21e0 0x559b929a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f442a800640 1 -- 172.21.15.69:0/3799496591 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b928c2e00 con 0x559b92998ce0 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f4429400640 1 --2- 172.21.15.69:0/3799496591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b929a21e0 0x559b929a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f4422a00640 1 -- 172.21.15.69:0/1483250482 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f441c0cf980 con 0x7f442c1038a0 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.983+0000 7f442a800640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f44100a2a90 0x7f44100a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.984+0000 7f442a800640 1 --2- 172.21.15.69:0/3799496591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92998ce0 0x559b929990c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4424009ca0 tx=0x7f4424032460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72631349c3842ef8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.984+0000 7f4417000640 1 -- 172.21.15.69:0/3799496591 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424039040 con 0x559b92998ce0 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.984+0000 7f4417000640 1 -- 172.21.15.69:0/3799496591 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4424003e50 con 0x559b92998ce0 2024-03-19T01:11:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.984+0000 7f4417000640 1 -- 172.21.15.69:0/3799496591 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424034440 con 0x559b92998ce0 2024-03-19T01:11:36.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.985+0000 7f443377a740 1 -- 172.21.15.69:0/3799496591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92998ce0 msgr2=0x559b929990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:36.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.985+0000 7f443377a740 1 --2- 172.21.15.69:0/3799496591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92998ce0 0x559b929990c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4424009ca0 tx=0x7f4424032460 comp rx=0 tx=0).stop 2024-03-19T01:11:36.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.985+0000 7f442a800640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f44100a2a90 0x7f44100a4f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f442404b000 tx=0x7f4424044000 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:36.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.985+0000 7f443377a740 1 -- 172.21.15.69:0/3799496591 shutdown_connections 2024-03-19T01:11:36.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.985+0000 7f443377a740 1 --2- 172.21.15.69:0/3799496591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b929a21e0 0x559b929a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.985+0000 7f443377a740 1 --2- 172.21.15.69:0/3799496591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92999600 0x559b929a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.985+0000 7f443377a740 1 --2- 172.21.15.69:0/3799496591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92998ce0 0x559b929990c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.985+0000 7f443377a740 1 -- 172.21.15.69:0/3799496591 >> 172.21.15.69:0/3799496591 conn(0x559b92993410 msgr2=0x559b929937f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.985+0000 7f443377a740 1 -- 172.21.15.69:0/3799496591 shutdown_connections 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.985+0000 7f443377a740 1 -- 172.21.15.69:0/3799496591 wait complete. 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.986+0000 7f443377a740 1 -- start start 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.986+0000 7f443377a740 1 Processor -- start 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f443377a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b92998ce0 0x559b9299f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f443377a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92999600 0x559b9299f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f443377a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 0x559b9299fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f443377a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559b929a4fd0 con 0x559b92998ce0 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f443377a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559b928b7f00 con 0x559b92999600 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f443377a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559b929a5150 con 0x559b929a21e0 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f443377a740 10 client.0 _ll_register_callbacks cb 0x559b928b78d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f442a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 0x559b9299fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:36.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f442a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 0x559b9299fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36894/0 (socket says 172.21.15.69:36894) 2024-03-19T01:11:36.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f442a800640 1 -- 172.21.15.69:0/1667827736 learned_addr learned my addr 172.21.15.69:0/1667827736 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:36.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f4429e00640 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b92998ce0 0x559b9299f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:36.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f4429400640 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92999600 0x559b9299f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:36.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f442a800640 1 -- 172.21.15.69:0/1667827736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92999600 msgr2=0x559b9299f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:36.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f442a800640 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92999600 0x559b9299f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f442a800640 1 -- 172.21.15.69:0/1667827736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b92998ce0 msgr2=0x559b9299f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:36.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f442a800640 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b92998ce0 0x559b9299f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.987+0000 7f442a800640 1 -- 172.21.15.69:0/1667827736 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b926d87e0 con 0x559b929a21e0 2024-03-19T01:11:36.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.988+0000 7f442a800640 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 0x559b9299fc70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4424009cd0 tx=0x7f4424004640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6ad051e37a5936c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:36.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.988+0000 7f440b400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:11:36.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.988+0000 7f440b400640 1 -- 172.21.15.69:0/1667827736 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424032c90 con 0x559b929a21e0 2024-03-19T01:11:36.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.988+0000 7f443377a740 1 -- 172.21.15.69:0/1667827736 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4424003e50 con 0x559b929a21e0 2024-03-19T01:11:36.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.988+0000 7f440b400640 1 -- 172.21.15.69:0/1667827736 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4424003970 con 0x559b929a21e0 2024-03-19T01:11:36.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.988+0000 7f440b400640 1 -- 172.21.15.69:0/1667827736 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4424061050 con 0x559b929a21e0 2024-03-19T01:11:36.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.988+0000 7f443377a740 10 client.25972 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:36.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.988+0000 7f443377a740 1 -- 172.21.15.69:0/1667827736 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b92738f70 con 0x559b929a21e0 2024-03-19T01:11:36.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.989+0000 7f440b400640 1 -- 172.21.15.69:0/1667827736 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4424003e50 con 0x559b929a21e0 2024-03-19T01:11:36.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.989+0000 7f440b400640 1 -- 172.21.15.69:0/1667827736 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f442403f070 con 0x559b929a21e0 2024-03-19T01:11:36.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.989+0000 7f443377a740 10 client.25972 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.989+0000 7f443377a740 10 client.25972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.989+0000 7f443377a740 20 client.25972 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.989+0000 7f443377a740 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559b929b2340 0x559b929b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.989+0000 7f443377a740 4 client.25972 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.989+0000 7f443377a740 1 -- 172.21.15.69:0/1667827736 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b929094f0 con 0x559b929b2340 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.989+0000 7f4429e00640 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559b929b2340 0x559b929b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.990+0000 7f4429e00640 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559b929b2340 0x559b929b4730 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f837ea98dd39ebf9 server_cookie=ed927bb721aa2db9 in_seq=0 out_seq=0 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.990+0000 7f440b400640 10 client.25972 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f440b400640 1 -- 172.21.15.69:0/1667827736 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559b929094f0 con 0x559b929b2340 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f440b400640 10 client.25972 handle_command_reply: tid=0 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 1 client.25972 shutdown 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 1 -- 172.21.15.69:0/1667827736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 msgr2=0x559b9299fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:36.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 0x559b9299fc70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4424009cd0 tx=0x7f4424004640 comp rx=0 tx=0).stop 2024-03-19T01:11:36.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 1 -- 172.21.15.69:0/1667827736 shutdown_connections 2024-03-19T01:11:36.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559b929b2340 0x559b929b4730 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 0x559b9299fc70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92999600 0x559b9299f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f440b400640 0 client.25972 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:36.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 1 --2- 172.21.15.69:0/1667827736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b92998ce0 0x559b9299f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:36.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 1 -- 172.21.15.69:0/1667827736 >> 172.21.15.69:0/1667827736 conn(0x559b92993410 msgr2=0x559b92995160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:36.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 1 -- 172.21.15.69:0/1667827736 shutdown_connections 2024-03-19T01:11:36.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 1 -- 172.21.15.69:0/1667827736 wait complete. 2024-03-19T01:11:36.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:36.991+0000 7f443377a740 20 client.25972 trim_cache size 0 max 16384 2024-03-19T01:11:37.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.012+0000 7f443377a740 1 Processor -- start 2024-03-19T01:11:37.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.012+0000 7f443377a740 1 -- start start 2024-03-19T01:11:37.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f443377a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929a1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f443377a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92999600 0x559b929add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f443377a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b929a21e0 0x559b929b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f443377a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1b0de0 con 0x559b929a21e0 2024-03-19T01:11:37.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f443377a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1b2920 con 0x559b92998ce0 2024-03-19T01:11:37.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f443377a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1b14b0 con 0x559b92999600 2024-03-19T01:11:37.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f4429e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f4429e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35800/0 (socket says 172.21.15.69:35800) 2024-03-19T01:11:37.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f4429e00640 1 -- 172.21.15.69:0/579905629 learned_addr learned my addr 172.21.15.69:0/579905629 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f4429400640 1 --2- 172.21.15.69:0/579905629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92999600 0x559b929add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f442a800640 1 --2- 172.21.15.69:0/579905629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b929a21e0 0x559b929b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f4429e00640 1 -- 172.21.15.69:0/579905629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92999600 msgr2=0x559b929add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f4429e00640 1 --2- 172.21.15.69:0/579905629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92999600 0x559b929add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f4429e00640 1 -- 172.21.15.69:0/579905629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b929a21e0 msgr2=0x559b929b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f4429e00640 1 --2- 172.21.15.69:0/579905629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b929a21e0 0x559b929b9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f4429e00640 1 -- 172.21.15.69:0/579905629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b925d0170 con 0x559b92998ce0 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.013+0000 7f4429400640 1 --2- 172.21.15.69:0/579905629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92999600 0x559b929add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.014+0000 7f4429e00640 1 --2- 172.21.15.69:0/579905629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929a1920 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f441c097dd0 tx=0x7f441c086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=158acb08400461b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.014+0000 7f4416600640 1 -- 172.21.15.69:0/579905629 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c0b3070 con 0x559b92998ce0 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.014+0000 7f4416600640 1 -- 172.21.15.69:0/579905629 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f441c09c070 con 0x559b92998ce0 2024-03-19T01:11:37.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.014+0000 7f4416600640 1 -- 172.21.15.69:0/579905629 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c097fe0 con 0x559b92998ce0 2024-03-19T01:11:37.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.015+0000 7f443377a740 1 -- 172.21.15.69:0/579905629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 msgr2=0x559b929a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.015+0000 7f443377a740 1 --2- 172.21.15.69:0/579905629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929a1920 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f441c097dd0 tx=0x7f441c086040 comp rx=0 tx=0).stop 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.015+0000 7f443377a740 1 -- 172.21.15.69:0/579905629 shutdown_connections 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.015+0000 7f443377a740 1 --2- 172.21.15.69:0/579905629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b929a21e0 0x559b929b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.015+0000 7f443377a740 1 --2- 172.21.15.69:0/579905629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b92999600 0x559b929add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.015+0000 7f443377a740 1 --2- 172.21.15.69:0/579905629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929a1920 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.015+0000 7f443377a740 1 -- 172.21.15.69:0/579905629 >> 172.21.15.69:0/579905629 conn(0x559b92993410 msgr2=0x559b9299fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.015+0000 7f443377a740 1 -- 172.21.15.69:0/579905629 shutdown_connections 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.015+0000 7f443377a740 1 -- 172.21.15.69:0/579905629 wait complete. 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.015+0000 7f443377a740 1 -- start start 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.015+0000 7f443377a740 1 Processor -- start 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f443377a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f443377a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b92999600 0x559b929a7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f443377a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 0x559b929a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f443377a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1b2250 con 0x559b92999600 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f443377a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1b1b80 con 0x559b92998ce0 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f443377a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1aedc0 con 0x559b929a21e0 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f443377a740 10 client.0 _ll_register_callbacks cb 0x7f441c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f4429e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f4429e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929aa3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35804/0 (socket says 172.21.15.69:35804) 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f4429e00640 1 -- 172.21.15.69:0/1761772431 learned_addr learned my addr 172.21.15.69:0/1761772431 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f442a800640 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 0x559b929a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.016+0000 7f4429400640 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b92999600 0x559b929a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.017+0000 7f4429e00640 1 -- 172.21.15.69:0/1761772431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 msgr2=0x559b929a78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.017+0000 7f4429e00640 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 0x559b929a78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.017+0000 7f4429e00640 1 -- 172.21.15.69:0/1761772431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b92999600 msgr2=0x559b929a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.017+0000 7f4429e00640 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b92999600 0x559b929a7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.017+0000 7f4429e00640 1 -- 172.21.15.69:0/1761772431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b929a7df0 con 0x559b92998ce0 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.017+0000 7f4429e00640 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929aa3a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f441c00c6a0 tx=0x7f441c005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=748590528777af1d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.017+0000 7f440b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.017+0000 7f440b400640 1 -- 172.21.15.69:0/1761772431 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c0b3070 con 0x559b92998ce0 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.017+0000 7f443377a740 1 -- 172.21.15.69:0/1761772431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f441c09c070 con 0x559b92998ce0 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.017+0000 7f440b400640 1 -- 172.21.15.69:0/1761772431 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f441c0a3c80 con 0x559b92998ce0 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.018+0000 7f440b400640 1 -- 172.21.15.69:0/1761772431 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f441c0a4110 con 0x559b92998ce0 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.018+0000 7f443377a740 10 client.25988 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.018+0000 7f443377a740 1 -- 172.21.15.69:0/1761772431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b92a18e70 con 0x559b92998ce0 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.018+0000 7f440b400640 1 -- 172.21.15.69:0/1761772431 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f441c09c070 con 0x559b92998ce0 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.018+0000 7f440b400640 1 -- 172.21.15.69:0/1761772431 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f441c0861b0 con 0x559b92998ce0 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.018+0000 7f443377a740 10 client.25988 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:37.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.018+0000 7f443377a740 10 client.25988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:37.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.018+0000 7f443377a740 20 client.25988 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:37.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.018+0000 7f443377a740 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559b92a195a0 0x559b92a1b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.018+0000 7f443377a740 4 client.25988 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:11:37.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.018+0000 7f443377a740 1 -- 172.21.15.69:0/1761772431 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559b92903840 con 0x559b92a195a0 2024-03-19T01:11:37.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.019+0000 7f4429400640 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559b92a195a0 0x559b92a1b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.019+0000 7f4429400640 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559b92a195a0 0x559b92a1b9c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c54960a293ac7154 server_cookie=e38fcdccdde4de2b in_seq=0 out_seq=0 2024-03-19T01:11:37.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.019+0000 7f440b400640 10 client.25988 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:37.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f440b400640 1 -- 172.21.15.69:0/1761772431 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559b92903840 con 0x559b92a195a0 2024-03-19T01:11:37.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f440b400640 10 client.25988 handle_command_reply: tid=0 2024-03-19T01:11:37.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f443377a740 1 client.25988 shutdown 2024-03-19T01:11:37.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f443377a740 1 -- 172.21.15.69:0/1761772431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 msgr2=0x559b929aa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f443377a740 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929aa3a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f441c00c6a0 tx=0x7f441c005c70 comp rx=0 tx=0).stop 2024-03-19T01:11:37.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f443377a740 1 -- 172.21.15.69:0/1761772431 shutdown_connections 2024-03-19T01:11:37.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f443377a740 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559b92a195a0 0x559b92a1b9c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f443377a740 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b929a21e0 0x559b929a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f443377a740 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b92999600 0x559b929a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f443377a740 1 --2- 172.21.15.69:0/1761772431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b92998ce0 0x559b929aa3a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f443377a740 1 -- 172.21.15.69:0/1761772431 >> 172.21.15.69:0/1761772431 conn(0x559b92993410 msgr2=0x559b92991be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:37.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.020+0000 7f440b400640 0 client.25988 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:37.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.021+0000 7f443377a740 1 -- 172.21.15.69:0/1761772431 shutdown_connections 2024-03-19T01:11:37.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.021+0000 7f443377a740 1 -- 172.21.15.69:0/1761772431 wait complete. 2024-03-19T01:11:37.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.021+0000 7f443377a740 20 client.25988 trim_cache size 0 max 16384 2024-03-19T01:11:37.026 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:11:37.027 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:11:37.027 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "4a5444b0-d30e-4e46-9895-401df225fa61", 2024-03-19T01:11:37.027 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:11:37.027 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:11:37.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.021+0000 7f442b200640 1 -- 172.21.15.69:0/1483250482 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f44100a2a90 msgr2=0x7f44100a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.021+0000 7f442b200640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f44100a2a90 0x7f44100a4f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f442404b000 tx=0x7f4424044000 comp rx=0 tx=0).stop 2024-03-19T01:11:37.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.021+0000 7f442b200640 1 -- 172.21.15.69:0/1483250482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c1038a0 msgr2=0x7f442c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.021+0000 7f442b200640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c1038a0 0x7f442c19a180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f441c00e9f0 tx=0x7f441c00edf0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.022+0000 7f442b200640 1 -- 172.21.15.69:0/1483250482 shutdown_connections 2024-03-19T01:11:37.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.022+0000 7f442b200640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c1098b0 0x7f442c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.022+0000 7f442b200640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f442c104260 0x7f442c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.022+0000 7f442b200640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f44100a2a90 0x7f44100a4f50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.022+0000 7f442b200640 1 --2- 172.21.15.69:0/1483250482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c1038a0 0x7f442c19a180 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.022+0000 7f442b200640 1 -- 172.21.15.69:0/1483250482 >> 172.21.15.69:0/1483250482 conn(0x7f442c0ff330 msgr2=0x7f442c0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:37.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.022+0000 7f442b200640 1 -- 172.21.15.69:0/1483250482 shutdown_connections 2024-03-19T01:11:37.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.022+0000 7f442b200640 1 -- 172.21.15.69:0/1483250482 wait complete. 2024-03-19T01:11:37.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:11:37.043 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:11:37.211 INFO:tasks.workunit.client.0.smithi069.stderr:Note: switching to 'e48b1f6c0608b2615e1890cc46286c3877d6e910'. 2024-03-19T01:11:37.211 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-03-19T01:11:37.211 INFO:tasks.workunit.client.0.smithi069.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-19T01:11:37.211 INFO:tasks.workunit.client.0.smithi069.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-19T01:11:37.211 INFO:tasks.workunit.client.0.smithi069.stderr:state without impacting any branches by switching back to a branch. 2024-03-19T01:11:37.211 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-03-19T01:11:37.211 INFO:tasks.workunit.client.0.smithi069.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-19T01:11:37.211 INFO:tasks.workunit.client.0.smithi069.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-19T01:11:37.212 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-03-19T01:11:37.212 INFO:tasks.workunit.client.0.smithi069.stderr: git switch -c 2024-03-19T01:11:37.212 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-03-19T01:11:37.212 INFO:tasks.workunit.client.0.smithi069.stderr:Or undo this operation with: 2024-03-19T01:11:37.212 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-03-19T01:11:37.212 INFO:tasks.workunit.client.0.smithi069.stderr: git switch - 2024-03-19T01:11:37.212 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-03-19T01:11:37.212 INFO:tasks.workunit.client.0.smithi069.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-19T01:11:37.212 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-03-19T01:11:37.212 INFO:tasks.workunit.client.0.smithi069.stderr:HEAD is now at e48b1f6c060 testfix 2024-03-19T01:11:37.223 DEBUG:teuthology.orchestra.run.smithi069:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-19T01:11:37.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.268+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1775303071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa01089e0 msgr2=0x7fbfa010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.268+0000 7fbfa5c00640 1 --2- 172.21.15.69:0/1775303071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa01089e0 0x7fbfa010ca20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009900 tx=0x7fbf8c02bc40 comp rx=0 tx=0).stop 2024-03-19T01:11:37.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.268+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1775303071 shutdown_connections 2024-03-19T01:11:37.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.268+0000 7fbfa5c00640 1 --2- 172.21.15.69:0/1775303071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa010d100 0x7fbfa0114cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.268+0000 7fbfa5c00640 1 --2- 172.21.15.69:0/1775303071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa01089e0 0x7fbfa010ca20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.268+0000 7fbfa5c00640 1 --2- 172.21.15.69:0/1775303071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfa0108030 0x7fbfa0108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.268+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1775303071 >> 172.21.15.69:0/1775303071 conn(0x7fbfa0075d60 msgr2=0x7fbfa0078180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:37.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.268+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1775303071 shutdown_connections 2024-03-19T01:11:37.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.268+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1775303071 wait complete. 2024-03-19T01:11:37.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.269+0000 7fbfa5c00640 1 Processor -- start 2024-03-19T01:11:37.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.269+0000 7fbfa5c00640 1 -- start start 2024-03-19T01:11:37.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbfa5c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa0108030 0x7fbfa019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbfa5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa010d100 0x7fbfa019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbfa5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfa01a2f90 0x7fbfa01a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbfa5c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01181c0 con 0x7fbfa01a2f90 2024-03-19T01:11:37.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbfa5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa0118040 con 0x7fbfa010d100 2024-03-19T01:11:37.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbfa5c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa0118340 con 0x7fbfa0108030 2024-03-19T01:11:37.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa010d100 0x7fbfa019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa010d100 0x7fbfa019ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35816/0 (socket says 172.21.15.69:35816) 2024-03-19T01:11:37.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa0108030 0x7fbfa019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9e000640 1 -- 172.21.15.69:0/1385402706 learned_addr learned my addr 172.21.15.69:0/1385402706 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:37.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9f400640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfa01a2f90 0x7fbfa01a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9e000640 1 -- 172.21.15.69:0/1385402706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa0108030 msgr2=0x7fbfa019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9e000640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa0108030 0x7fbfa019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9e000640 1 -- 172.21.15.69:0/1385402706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfa01a2f90 msgr2=0x7fbfa01a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9e000640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfa01a2f90 0x7fbfa01a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9e000640 1 -- 172.21.15.69:0/1385402706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa01a3a80 con 0x7fbfa010d100 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9ea00640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa0108030 0x7fbfa019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.270+0000 7fbf9f400640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfa01a2f90 0x7fbfa01a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.271+0000 7fbf9e000640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa010d100 0x7fbfa019ec60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c02c250 tx=0x7fbf8c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.271+0000 7fbf97400640 1 -- 172.21.15.69:0/1385402706 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c041070 con 0x7fbfa010d100 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.271+0000 7fbf97400640 1 -- 172.21.15.69:0/1385402706 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8c02cca0 con 0x7fbfa010d100 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.271+0000 7fbf97400640 1 -- 172.21.15.69:0/1385402706 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c003da0 con 0x7fbfa010d100 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.271+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1385402706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa01a3d10 con 0x7fbfa010d100 2024-03-19T01:11:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.271+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1385402706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa01ab480 con 0x7fbfa010d100 2024-03-19T01:11:37.276 INFO:tasks.workunit.client.0.smithi069.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-19T01:11:37.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.275+0000 7fbfa8a11740 1 Processor -- start 2024-03-19T01:11:37.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbfa8a11740 1 -- start start 2024-03-19T01:11:37.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbf97400640 1 -- 172.21.15.69:0/1385402706 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7fbf8c044070 con 0x7fbfa010d100 2024-03-19T01:11:37.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 0x5595e7400120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e7408d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 0x5595e740b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5595e73642c0 con 0x5595e7409240 2024-03-19T01:11:37.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595e730cb60 con 0x5595e7400660 2024-03-19T01:11:37.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5595e7360360 con 0x5595e73ffd40 2024-03-19T01:11:37.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbf9f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 0x5595e7400120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbf9f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 0x5595e7400120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36946/0 (socket says 172.21.15.69:36946) 2024-03-19T01:11:37.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbf9f400640 1 -- 172.21.15.69:0/1386096258 learned_addr learned my addr 172.21.15.69:0/1386096258 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:37.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.276+0000 7fbf9e000640 1 --2- 172.21.15.69:0/1386096258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 0x5595e740b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.277+0000 7fbf9f400640 1 -- 172.21.15.69:0/1386096258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 msgr2=0x5595e7408d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:11:37.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.277+0000 7fbf9f400640 1 --2- 172.21.15.69:0/1386096258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e7408d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.277+0000 7fbf9f400640 1 -- 172.21.15.69:0/1386096258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 msgr2=0x5595e740b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.277+0000 7fbf9f400640 1 --2- 172.21.15.69:0/1386096258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 0x5595e740b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.277+0000 7fbf9f400640 1 -- 172.21.15.69:0/1386096258 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e71427d0 con 0x5595e73ffd40 2024-03-19T01:11:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.277+0000 7fbf9f400640 1 --2- 172.21.15.69:0/1386096258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 0x5595e7400120 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009fd0 tx=0x7fbf98010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efb9d793650c932c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.277+0000 7fbf8be00640 1 -- 172.21.15.69:0/1386096258 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98011700 con 0x5595e73ffd40 2024-03-19T01:11:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.277+0000 7fbf8be00640 1 -- 172.21.15.69:0/1386096258 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf98011d80 con 0x5595e73ffd40 2024-03-19T01:11:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.277+0000 7fbf8be00640 1 -- 172.21.15.69:0/1386096258 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800e3c0 con 0x5595e73ffd40 2024-03-19T01:11:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.278+0000 7fbf97400640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf840a2b10 0x7fbf840a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.278+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1386096258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 msgr2=0x5595e7400120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.278+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1386096258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 0x5595e7400120 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009fd0 tx=0x7fbf98010860 comp rx=0 tx=0).stop 2024-03-19T01:11:37.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.278+0000 7fbf97400640 1 -- 172.21.15.69:0/1385402706 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fbf8c0f1170 con 0x7fbfa010d100 2024-03-19T01:11:37.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.278+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1386096258 shutdown_connections 2024-03-19T01:11:37.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.278+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1386096258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 0x5595e740b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.278+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1386096258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e7408d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.278+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1386096258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 0x5595e7400120 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.278+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1386096258 >> 172.21.15.69:0/1386096258 conn(0x5595e73fa470 msgr2=0x5595e73fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:37.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.278+0000 7fbf9ea00640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf840a2b10 0x7fbf840a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.279+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1386096258 shutdown_connections 2024-03-19T01:11:37.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.279+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1386096258 wait complete. 2024-03-19T01:11:37.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.279+0000 7fbf9ea00640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf840a2b10 0x7fbf840a4fd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x5595e73ff810 tx=0x7fbf9000b390 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:37.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.280+0000 7fbfa8a11740 1 -- start start 2024-03-19T01:11:37.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.280+0000 7fbfa8a11740 1 Processor -- start 2024-03-19T01:11:37.282 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-19T01:11:37.282 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-19T01:11:37.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.280+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 0x5595e7406210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.280+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e7406750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.280+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 0x5595e7406c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.280+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5595e740c030 con 0x5595e7409240 2024-03-19T01:11:37.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.280+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595e731ffc0 con 0x5595e7400660 2024-03-19T01:11:37.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.280+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5595e740c1b0 con 0x5595e73ffd40 2024-03-19T01:11:37.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbfa8a11740 10 client.0 _ll_register_callbacks cb 0x5595e731f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:37.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbf9f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 0x5595e7406c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbf9ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 0x5595e7406210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbf9f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 0x5595e7406c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52704/0 (socket says 172.21.15.69:52704) 2024-03-19T01:11:37.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbf9f400640 1 -- 172.21.15.69:0/1855027699 learned_addr learned my addr 172.21.15.69:0/1855027699 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:37.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbf9f400640 1 -- 172.21.15.69:0/1855027699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 msgr2=0x5595e7406210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:37.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbf9f400640 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 0x5595e7406210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbf9f400640 1 -- 172.21.15.69:0/1855027699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 msgr2=0x5595e7406750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:37.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbf9e000640 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e7406750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbf9f400640 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e7406750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbf9f400640 1 -- 172.21.15.69:0/1855027699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e70783f0 con 0x5595e7409240 2024-03-19T01:11:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.281+0000 7fbf9e000640 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e7406750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:11:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.282+0000 7fbf9f400640 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 0x5595e7406c90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbf9801b950 tx=0x7fbf9801b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7931cd5f84b3510 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.282+0000 7fbf88c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:11:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.282+0000 7fbf88c00640 1 -- 172.21.15.69:0/1855027699 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98004e50 con 0x5595e7409240 2024-03-19T01:11:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.282+0000 7fbf88c00640 1 -- 172.21.15.69:0/1855027699 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fbf98011500 con 0x5595e7409240 2024-03-19T01:11:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.282+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1855027699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf98011d80 con 0x5595e7409240 2024-03-19T01:11:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.282+0000 7fbf88c00640 1 -- 172.21.15.69:0/1855027699 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbf98033050 con 0x5595e7409240 2024-03-19T01:11:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.282+0000 7fbfa8a11740 10 client.15810 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.282+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1855027699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5595e70380f0 con 0x5595e7409240 2024-03-19T01:11:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.283+0000 7fbf88c00640 1 -- 172.21.15.69:0/1855027699 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf98011d80 con 0x5595e7409240 2024-03-19T01:11:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.283+0000 7fbf88c00640 1 -- 172.21.15.69:0/1855027699 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbf98005740 con 0x5595e7409240 2024-03-19T01:11:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.283+0000 7fbfa8a11740 10 client.15810 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.283+0000 7fbfa8a11740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.283+0000 7fbfa8a11740 20 client.15810 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.283+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595e7419450 0x5595e741b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.283+0000 7fbfa8a11740 4 client.15810 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:11:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.283+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1855027699 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595e7305610 con 0x5595e7419450 2024-03-19T01:11:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.284+0000 7fbf9ea00640 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595e7419450 0x5595e741b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.285+0000 7fbf88c00640 10 client.15810 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:37.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.285+0000 7fbf9ea00640 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595e7419450 0x5595e741b840 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9daacd97fabfeca server_cookie=903ef43f6931785c in_seq=0 out_seq=0 2024-03-19T01:11:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.286+0000 7fbf88c00640 1 -- 172.21.15.69:0/1855027699 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5595e7305610 con 0x5595e7419450 2024-03-19T01:11:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.286+0000 7fbf88c00640 10 client.15810 handle_command_reply: tid=0 2024-03-19T01:11:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.286+0000 7fbfa8a11740 1 client.15810 shutdown 2024-03-19T01:11:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1855027699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 msgr2=0x5595e7406c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 0x5595e7406c90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbf9801b950 tx=0x7fbf9801b830 comp rx=0 tx=0).stop 2024-03-19T01:11:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1855027699 shutdown_connections 2024-03-19T01:11:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595e7419450 0x5595e741b840 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7409240 0x5595e7406c90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e7406750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1855027699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e73ffd40 0x5595e7406210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1855027699 >> 172.21.15.69:0/1855027699 conn(0x5595e73fa470 msgr2=0x5595e73fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbf88c00640 0 client.15810 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1855027699 shutdown_connections 2024-03-19T01:11:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1855027699 wait complete. 2024-03-19T01:11:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.287+0000 7fbfa8a11740 20 client.15810 trim_cache size 0 max 16384 2024-03-19T01:11:37.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbfa8a11740 1 Processor -- start 2024-03-19T01:11:37.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbfa8a11740 1 -- start start 2024-03-19T01:11:37.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e73ffd40 0x5595e7408940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7400660 0x5595e7405390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 0x5595e7414dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01b5360 con 0x5595e7400660 2024-03-19T01:11:37.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01b6ea0 con 0x5595e73ffd40 2024-03-19T01:11:37.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01b5a30 con 0x5595e7409240 2024-03-19T01:11:37.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbf9ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e73ffd40 0x5595e7408940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbf9f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 0x5595e7414dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbf9ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e73ffd40 0x5595e7408940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35826/0 (socket says 172.21.15.69:35826) 2024-03-19T01:11:37.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.306+0000 7fbf9ea00640 1 -- 172.21.15.69:0/1299510231 learned_addr learned my addr 172.21.15.69:0/1299510231 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:37.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.307+0000 7fbf9e000640 1 --2- 172.21.15.69:0/1299510231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7400660 0x5595e7405390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.307+0000 7fbf9ea00640 1 -- 172.21.15.69:0/1299510231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 msgr2=0x5595e7414dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:37.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.307+0000 7fbf9ea00640 1 --2- 172.21.15.69:0/1299510231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 0x5595e7414dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.307+0000 7fbf9ea00640 1 -- 172.21.15.69:0/1299510231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7400660 msgr2=0x5595e7405390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:37.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.307+0000 7fbf9ea00640 1 --2- 172.21.15.69:0/1299510231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7400660 0x5595e7405390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.307+0000 7fbf9ea00640 1 -- 172.21.15.69:0/1299510231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e741bbb0 con 0x5595e73ffd40 2024-03-19T01:11:37.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.307+0000 7fbf9e000640 1 --2- 172.21.15.69:0/1299510231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7400660 0x5595e7405390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:11:37.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.307+0000 7fbf9ea00640 1 --2- 172.21.15.69:0/1299510231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e73ffd40 0x5595e7408940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbf90000c00 tx=0x7fbf90013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=566d9a6c11b54678 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:37.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.307+0000 7fbf8b400640 1 -- 172.21.15.69:0/1299510231 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90027c30 con 0x5595e73ffd40 2024-03-19T01:11:37.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.307+0000 7fbf8b400640 1 -- 172.21.15.69:0/1299510231 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf900272d0 con 0x5595e73ffd40 2024-03-19T01:11:37.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.307+0000 7fbf8b400640 1 -- 172.21.15.69:0/1299510231 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9001ede0 con 0x5595e73ffd40 2024-03-19T01:11:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.308+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1299510231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e73ffd40 msgr2=0x5595e7408940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.308+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1299510231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e73ffd40 0x5595e7408940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbf90000c00 tx=0x7fbf90013000 comp rx=0 tx=0).stop 2024-03-19T01:11:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.308+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1299510231 shutdown_connections 2024-03-19T01:11:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.308+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1299510231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 0x5595e7414dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.308+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1299510231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e7400660 0x5595e7405390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.308+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1299510231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e73ffd40 0x5595e7408940 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.308+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1299510231 >> 172.21.15.69:0/1299510231 conn(0x5595e73fa470 msgr2=0x5595e73f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.308+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1299510231 shutdown_connections 2024-03-19T01:11:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.308+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1299510231 wait complete. 2024-03-19T01:11:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.309+0000 7fbfa8a11740 1 -- start start 2024-03-19T01:11:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.309+0000 7fbfa8a11740 1 Processor -- start 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e73ffd40 0x5595e7411400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e74028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbfa8a11740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 0x5595e7402df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01b67d0 con 0x5595e73ffd40 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01b6100 con 0x5595e7400660 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbfa8a11740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01b3340 con 0x5595e7409240 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbfa8a11740 10 client.0 _ll_register_callbacks cb 0x7fbf98011f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbf9e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e74028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbf9f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 0x5595e7402df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbf9f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 0x5595e7402df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36976/0 (socket says 172.21.15.69:36976) 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbf9f400640 1 -- 172.21.15.69:0/1124596590 learned_addr learned my addr 172.21.15.69:0/1124596590 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbf9ea00640 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e73ffd40 0x5595e7411400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbf9e000640 1 -- 172.21.15.69:0/1124596590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 msgr2=0x5595e7402df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbf9e000640 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 0x5595e7402df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbf9e000640 1 -- 172.21.15.69:0/1124596590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e73ffd40 msgr2=0x5595e7411400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbf9e000640 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e73ffd40 0x5595e7411400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbf9e000640 1 -- 172.21.15.69:0/1124596590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e7403330 con 0x5595e7400660 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.310+0000 7fbf9f400640 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 0x5595e7402df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.311+0000 7fbf9e000640 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e74028b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0ee430 tx=0x7fbf8c02ce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70f3a0eed1c88e31 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.311+0000 7fbf88c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.311+0000 7fbf88c00640 1 -- 172.21.15.69:0/1124596590 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0bcdb0 con 0x5595e7400660 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.311+0000 7fbf88c00640 1 -- 172.21.15.69:0/1124596590 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fbf8c0bd4d0 con 0x5595e7400660 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.311+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1124596590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf900272d0 con 0x5595e7400660 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.311+0000 7fbf88c00640 1 -- 172.21.15.69:0/1124596590 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbf8c0bcbd0 con 0x5595e7400660 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.311+0000 7fbfa8a11740 10 client.26000 fetch_fsmap learned FSMap version 29 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.311+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1124596590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5595e747f1f0 con 0x5595e7400660 2024-03-19T01:11:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.311+0000 7fbf88c00640 1 -- 172.21.15.69:0/1124596590 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf900272d0 con 0x5595e7400660 2024-03-19T01:11:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.312+0000 7fbf88c00640 1 -- 172.21.15.69:0/1124596590 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbf8c0c5580 con 0x5595e7400660 2024-03-19T01:11:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.312+0000 7fbfa8a11740 10 client.26000 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:11:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.312+0000 7fbfa8a11740 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:11:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.312+0000 7fbfa8a11740 20 client.26000 populate_metadata read hostname 'smithi069' 2024-03-19T01:11:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.312+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595e747f920 0x5595e7481d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:11:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.312+0000 7fbfa8a11740 4 client.26000 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:11:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.312+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1124596590 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5595e7371800 con 0x5595e747f920 2024-03-19T01:11:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.312+0000 7fbf9ea00640 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595e747f920 0x5595e7481d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbf9ea00640 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595e747f920 0x5595e7481d40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3f61b84d55622a server_cookie=3a5d44c6992875b1 in_seq=0 out_seq=0 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbf88c00640 10 client.26000 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbf88c00640 1 -- 172.21.15.69:0/1124596590 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5595e7371800 con 0x5595e747f920 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbf88c00640 10 client.26000 handle_command_reply: tid=0 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbfa8a11740 1 client.26000 shutdown 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1124596590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 msgr2=0x5595e74028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e74028b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0ee430 tx=0x7fbf8c02ce50 comp rx=0 tx=0).stop 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1124596590 shutdown_connections 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595e747f920 0x5595e7481d40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595e7409240 0x5595e7402df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595e7400660 0x5595e74028b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbfa8a11740 1 --2- 172.21.15.69:0/1124596590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595e73ffd40 0x5595e7411400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1124596590 >> 172.21.15.69:0/1124596590 conn(0x5595e73fa470 msgr2=0x5595e74061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:37.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbf88c00640 0 client.26000 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:11:37.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1124596590 shutdown_connections 2024-03-19T01:11:37.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.313+0000 7fbfa8a11740 1 -- 172.21.15.69:0/1124596590 wait complete. 2024-03-19T01:11:37.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa8a11740 20 client.26000 trim_cache size 0 max 16384 2024-03-19T01:11:37.320 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:11:37.321 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T01:11:37.321 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:11:37.321 INFO:teuthology.orchestra.run.smithi069.stdout: "4a5444b0-d30e-4e46-9895-401df225fa61": { 2024-03-19T01:11:37.321 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:11:37.321 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "4a5444b0-d30e-4e46-9895-401df225fa61", 2024-03-19T01:11:37.321 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:11:37.321 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:11:37.321 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:11:37.321 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:11:37.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1385402706 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf840a2b10 msgr2=0x7fbf840a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf840a2b10 0x7fbf840a4fd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x5595e73ff810 tx=0x7fbf9000b390 comp rx=0 tx=0).stop 2024-03-19T01:11:37.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1385402706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa010d100 msgr2=0x7fbfa019ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:11:37.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa010d100 0x7fbfa019ec60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c02c250 tx=0x7fbf8c004360 comp rx=0 tx=0).stop 2024-03-19T01:11:37.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1385402706 shutdown_connections 2024-03-19T01:11:37.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfa01a2f90 0x7fbfa01a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf840a2b10 0x7fbf840a4fd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa010d100 0x7fbfa019ec60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 --2- 172.21.15.69:0/1385402706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa0108030 0x7fbfa019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:11:37.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1385402706 >> 172.21.15.69:0/1385402706 conn(0x7fbfa0075d60 msgr2=0x7fbfa0077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:11:37.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1385402706 shutdown_connections 2024-03-19T01:11:37.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:11:37.314+0000 7fbfa5c00640 1 -- 172.21.15.69:0/1385402706 wait complete. 2024-03-19T01:11:37.336 INFO:tasks.cephfs.filesystem:scrub status for tag:4a5444b0-d30e-4e46-9895-401df225fa61 - {'path': '/', 'tag': '4a5444b0-d30e-4e46-9895-401df225fa61', 'options': 'recursive,force'} 2024-03-19T01:11:38.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:37 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:11:38.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:38.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:11:38.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:38.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:37 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:11:38.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:38.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:11:38.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:38.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:37 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:11:38.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:38.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:11:38.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:11:38.277 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-19T01:11:38.430 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-19T01:11:38.524 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-19T01:11:38.528 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-19T01:11:38.528 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-19T01:11:38.607 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-19T01:11:38.616 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T01:11:38.616 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-19T01:11:38.662 INFO:tasks.workunit:Running workunits matching kernel_untar_build.sh on client.0... 2024-03-19T01:11:38.663 INFO:tasks.workunit:Running workunit kernel_untar_build.sh... 2024-03-19T01:11:38.663 DEBUG:teuthology.orchestra.run.smithi069:workunit test kernel_untar_build.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e48b1f6c0608b2615e1890cc46286c3877d6e910 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh 2024-03-19T01:11:38.762 INFO:tasks.workunit.client.0.smithi069.stderr:+ wget -O linux.tar.xz http://download.ceph.com/qa/linux-6.5.11.tar.xz 2024-03-19T01:11:38.820 INFO:tasks.workunit.client.0.smithi069.stderr:--2024-03-19 01:11:38-- http://download.ceph.com/qa/linux-6.5.11.tar.xz 2024-03-19T01:11:38.821 INFO:tasks.workunit.client.0.smithi069.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-19T01:11:38.895 INFO:tasks.workunit.client.0.smithi069.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-19T01:11:38.966 INFO:tasks.workunit.client.0.smithi069.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-19T01:11:38.966 INFO:tasks.workunit.client.0.smithi069.stderr:Length: 139010916 (133M) [application/octet-stream] 2024-03-19T01:11:38.966 INFO:tasks.workunit.client.0.smithi069.stderr:Saving to: ‘linux.tar.xz’ 2024-03-19T01:11:38.970 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-03-19T01:11:39.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:38 smithi086 ceph-mon[66829]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-19T01:11:39.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:38 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:11:39.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:38 smithi069 ceph-mon[31727]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-19T01:11:39.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:38 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:11:39.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:38 smithi163 ceph-mon[67428]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-19T01:11:39.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:38 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:11:39.121 INFO:tasks.workunit.client.0.smithi069.stderr: 0K .......... .......... .......... .......... .......... 0% 324K 6m59s 2024-03-19T01:11:39.294 INFO:tasks.workunit.client.0.smithi069.stderr: 50K .......... .......... .......... .......... .......... 0% 288K 7m25s 2024-03-19T01:11:39.298 INFO:tasks.workunit.client.0.smithi069.stderr: 100K .......... .......... .......... .......... .......... 0% 38.6M 4m58s 2024-03-19T01:11:39.298 INFO:tasks.workunit.client.0.smithi069.stderr: 150K .......... .......... .......... .......... .......... 0% 47.5M 3m44s 2024-03-19T01:11:39.316 INFO:tasks.workunit.client.0.smithi069.stderr: 200K .......... .......... .......... .......... .......... 0% 2.53M 3m10s 2024-03-19T01:11:39.316 INFO:tasks.workunit.client.0.smithi069.stderr: 250K .......... .......... .......... .......... .......... 0% 73.2M 2m38s 2024-03-19T01:11:39.336 INFO:tasks.workunit.client.0.smithi069.stderr: 300K .......... .......... .......... .......... .......... 0% 2.44M 2m23s 2024-03-19T01:11:39.337 INFO:tasks.workunit.client.0.smithi069.stderr: 350K .......... .......... .......... .......... .......... 0% 42.1M 2m6s 2024-03-19T01:11:39.357 INFO:tasks.workunit.client.0.smithi069.stderr: 400K .......... .......... .......... .......... .......... 0% 2.52M 1m58s 2024-03-19T01:11:39.358 INFO:tasks.workunit.client.0.smithi069.stderr: 450K .......... .......... .......... .......... .......... 0% 42.8M 1m46s 2024-03-19T01:11:39.377 INFO:tasks.workunit.client.0.smithi069.stderr: 500K .......... .......... .......... .......... .......... 0% 2.50M 1m41s 2024-03-19T01:11:39.396 INFO:tasks.workunit.client.0.smithi069.stderr: 550K .......... .......... .......... .......... .......... 0% 2.73M 97s 2024-03-19T01:11:39.398 INFO:tasks.workunit.client.0.smithi069.stderr: 600K .......... .......... .......... .......... .......... 0% 21.9M 90s 2024-03-19T01:11:39.418 INFO:tasks.workunit.client.0.smithi069.stderr: 650K .......... .......... .......... .......... .......... 0% 2.53M 87s 2024-03-19T01:11:39.419 INFO:tasks.workunit.client.0.smithi069.stderr: 700K .......... .......... .......... .......... .......... 0% 41.1M 81s 2024-03-19T01:11:39.438 INFO:tasks.workunit.client.0.smithi069.stderr: 750K .......... .......... .......... .......... .......... 0% 2.52M 80s 2024-03-19T01:11:39.438 INFO:tasks.workunit.client.0.smithi069.stderr: 800K .......... .......... .......... .......... .......... 0% 65.3M 75s 2024-03-19T01:11:39.458 INFO:tasks.workunit.client.0.smithi069.stderr: 850K .......... .......... .......... .......... .......... 0% 2.47M 74s 2024-03-19T01:11:39.459 INFO:tasks.workunit.client.0.smithi069.stderr: 900K .......... .......... .......... .......... .......... 0% 39.0M 70s 2024-03-19T01:11:39.480 INFO:tasks.workunit.client.0.smithi069.stderr: 950K .......... .......... .......... .......... .......... 0% 2.52M 69s 2024-03-19T01:11:39.480 INFO:tasks.workunit.client.0.smithi069.stderr: 1000K .......... .......... .......... .......... .......... 0% 40.1M 66s 2024-03-19T01:11:39.499 INFO:tasks.workunit.client.0.smithi069.stderr: 1050K .......... .......... .......... .......... .......... 0% 2.63M 65s 2024-03-19T01:11:39.499 INFO:tasks.workunit.client.0.smithi069.stderr: 1100K .......... .......... .......... .......... .......... 0% 63.0M 62s 2024-03-19T01:11:39.519 INFO:tasks.workunit.client.0.smithi069.stderr: 1150K .......... .......... .......... .......... .......... 0% 2.48M 62s 2024-03-19T01:11:39.537 INFO:tasks.workunit.client.0.smithi069.stderr: 1200K .......... .......... .......... .......... .......... 0% 71.3M 60s 2024-03-19T01:11:39.541 INFO:tasks.workunit.client.0.smithi069.stderr: 1250K .......... .......... .......... .......... .......... 0% 2.42M 59s 2024-03-19T01:11:39.558 INFO:tasks.workunit.client.0.smithi069.stderr: 1300K .......... .......... .......... .......... .......... 0% 2.70M 59s 2024-03-19T01:11:39.561 INFO:tasks.workunit.client.0.smithi069.stderr: 1350K .......... .......... .......... .......... .......... 1% 27.3M 57s 2024-03-19T01:11:39.581 INFO:tasks.workunit.client.0.smithi069.stderr: 1400K .......... .......... .......... .......... .......... 1% 2.50M 57s 2024-03-19T01:11:39.581 INFO:tasks.workunit.client.0.smithi069.stderr: 1450K .......... .......... .......... .......... .......... 1% 47.0M 55s 2024-03-19T01:11:39.599 INFO:tasks.workunit.client.0.smithi069.stderr: 1500K .......... .......... .......... .......... .......... 1% 2.51M 55s 2024-03-19T01:11:39.601 INFO:tasks.workunit.client.0.smithi069.stderr: 1550K .......... .......... .......... .......... .......... 1% 43.2M 53s 2024-03-19T01:11:39.621 INFO:tasks.workunit.client.0.smithi069.stderr: 1600K .......... .......... .......... .......... .......... 1% 2.47M 53s 2024-03-19T01:11:39.621 INFO:tasks.workunit.client.0.smithi069.stderr: 1650K .......... .......... .......... .......... .......... 1% 101M 52s 2024-03-19T01:11:39.642 INFO:tasks.workunit.client.0.smithi069.stderr: 1700K .......... .......... .......... .......... .......... 1% 2.46M 52s 2024-03-19T01:11:39.642 INFO:tasks.workunit.client.0.smithi069.stderr: 1750K .......... .......... .......... .......... .......... 1% 62.4M 50s 2024-03-19T01:11:39.662 INFO:tasks.workunit.client.0.smithi069.stderr: 1800K .......... .......... .......... .......... .......... 1% 2.52M 50s 2024-03-19T01:11:39.662 INFO:tasks.workunit.client.0.smithi069.stderr: 1850K .......... .......... .......... .......... .......... 1% 67.6M 49s 2024-03-19T01:11:39.683 INFO:tasks.workunit.client.0.smithi069.stderr: 1900K .......... .......... .......... .......... .......... 1% 2.53M 49s 2024-03-19T01:11:39.683 INFO:tasks.workunit.client.0.smithi069.stderr: 1950K .......... .......... .......... .......... .......... 1% 49.9M 48s 2024-03-19T01:11:39.704 INFO:tasks.workunit.client.0.smithi069.stderr: 2000K .......... .......... .......... .......... .......... 1% 2.54M 48s 2024-03-19T01:11:39.704 INFO:tasks.workunit.client.0.smithi069.stderr: 2050K .......... .......... .......... .......... .......... 1% 24.3M 47s 2024-03-19T01:11:39.721 INFO:tasks.workunit.client.0.smithi069.stderr: 2100K .......... .......... .......... .......... .......... 1% 2.68M 47s 2024-03-19T01:11:39.724 INFO:tasks.workunit.client.0.smithi069.stderr: 2150K .......... .......... .......... .......... .......... 1% 22.4M 46s 2024-03-19T01:11:39.742 INFO:tasks.workunit.client.0.smithi069.stderr: 2200K .......... .......... .......... .......... .......... 1% 2.68M 46s 2024-03-19T01:11:39.744 INFO:tasks.workunit.client.0.smithi069.stderr: 2250K .......... .......... .......... .......... .......... 1% 22.7M 45s 2024-03-19T01:11:39.762 INFO:tasks.workunit.client.0.smithi069.stderr: 2300K .......... .......... .......... .......... .......... 1% 2.72M 45s 2024-03-19T01:11:39.764 INFO:tasks.workunit.client.0.smithi069.stderr: 2350K .......... .......... .......... .......... .......... 1% 17.8M 44s 2024-03-19T01:11:39.782 INFO:tasks.workunit.client.0.smithi069.stderr: 2400K .......... .......... .......... .......... .......... 1% 2.81M 44s 2024-03-19T01:11:39.785 INFO:tasks.workunit.client.0.smithi069.stderr: 2450K .......... .......... .......... .......... .......... 1% 16.7M 44s 2024-03-19T01:11:39.803 INFO:tasks.workunit.client.0.smithi069.stderr: 2500K .......... .......... .......... .......... .......... 1% 2.80M 44s 2024-03-19T01:11:39.806 INFO:tasks.workunit.client.0.smithi069.stderr: 2550K .......... .......... .......... .......... .......... 1% 17.0M 43s 2024-03-19T01:11:39.828 INFO:tasks.workunit.client.0.smithi069.stderr: 2600K .......... .......... .......... .......... .......... 1% 2.27M 43s 2024-03-19T01:11:39.828 INFO:tasks.workunit.client.0.smithi069.stderr: 2650K .......... .......... .......... .......... .......... 1% 77.2M 42s 2024-03-19T01:11:39.847 INFO:tasks.workunit.client.0.smithi069.stderr: 2700K .......... .......... .......... .......... .......... 2% 2.52M 43s 2024-03-19T01:11:39.849 INFO:tasks.workunit.client.0.smithi069.stderr: 2750K .......... .......... .......... .......... .......... 2% 35.5M 42s 2024-03-19T01:11:39.849 INFO:tasks.workunit.client.0.smithi069.stderr: 2800K .......... .......... .......... .......... .......... 2% 105M 41s 2024-03-19T01:11:39.869 INFO:tasks.workunit.client.0.smithi069.stderr: 2850K .......... .......... .......... .......... .......... 2% 2.47M 41s 2024-03-19T01:11:39.870 INFO:tasks.workunit.client.0.smithi069.stderr: 2900K .......... .......... .......... .......... .......... 2% 53.9M 41s 2024-03-19T01:11:39.890 INFO:tasks.workunit.client.0.smithi069.stderr: 2950K .......... .......... .......... .......... .......... 2% 2.55M 41s 2024-03-19T01:11:39.890 INFO:tasks.workunit.client.0.smithi069.stderr: 3000K .......... .......... .......... .......... .......... 2% 74.8M 40s 2024-03-19T01:11:39.910 INFO:tasks.workunit.client.0.smithi069.stderr: 3050K .......... .......... .......... .......... .......... 2% 2.47M 40s 2024-03-19T01:11:39.910 INFO:tasks.workunit.client.0.smithi069.stderr: 3100K .......... .......... .......... .......... .......... 2% 49.2M 40s 2024-03-19T01:11:39.931 INFO:tasks.workunit.client.0.smithi069.stderr: 3150K .......... .......... .......... .......... .......... 2% 2.54M 40s 2024-03-19T01:11:39.931 INFO:tasks.workunit.client.0.smithi069.stderr: 3200K .......... .......... .......... .......... .......... 2% 49.6M 39s 2024-03-19T01:11:39.951 INFO:tasks.workunit.client.0.smithi069.stderr: 3250K .......... .......... .......... .......... .......... 2% 2.56M 39s 2024-03-19T01:11:39.951 INFO:tasks.workunit.client.0.smithi069.stderr: 3300K .......... .......... .......... .......... .......... 2% 42.0M 39s 2024-03-19T01:11:39.972 INFO:tasks.workunit.client.0.smithi069.stderr: 3350K .......... .......... .......... .......... .......... 2% 2.56M 39s 2024-03-19T01:11:39.972 INFO:tasks.workunit.client.0.smithi069.stderr: 3400K .......... .......... .......... .......... .......... 2% 40.2M 39s 2024-03-19T01:11:39.972 INFO:tasks.workunit.client.0.smithi069.stderr: 3450K .......... .......... .......... .......... .......... 2% 48.2M 38s 2024-03-19T01:11:39.991 INFO:tasks.workunit.client.0.smithi069.stderr: 3500K .......... .......... .......... .......... .......... 2% 2.60M 38s 2024-03-19T01:11:39.991 INFO:tasks.workunit.client.0.smithi069.stderr: 3550K .......... .......... .......... .......... .......... 2% 74.6M 38s 2024-03-19T01:11:40.012 INFO:tasks.workunit.client.0.smithi069.stderr: 3600K .......... .......... .......... .......... .......... 2% 2.47M 38s 2024-03-19T01:11:40.012 INFO:tasks.workunit.client.0.smithi069.stderr: 3650K .......... .......... .......... .......... .......... 2% 71.5M 37s 2024-03-19T01:11:40.031 INFO:tasks.workunit.client.0.smithi069.stderr: 3700K .......... .......... .......... .......... .......... 2% 2.51M 37s 2024-03-19T01:11:40.032 INFO:tasks.workunit.client.0.smithi069.stderr: 3750K .......... .......... .......... .......... .......... 2% 50.5M 37s 2024-03-19T01:11:40.053 INFO:tasks.workunit.client.0.smithi069.stderr: 3800K .......... .......... .......... .......... .......... 2% 2.55M 37s 2024-03-19T01:11:40.053 INFO:tasks.workunit.client.0.smithi069.stderr: 3850K .......... .......... .......... .......... .......... 2% 41.7M 37s 2024-03-19T01:11:40.071 INFO:tasks.workunit.client.0.smithi069.stderr: 3900K .......... .......... .......... .......... .......... 2% 2.72M 37s 2024-03-19T01:11:40.073 INFO:tasks.workunit.client.0.smithi069.stderr: 3950K .......... .......... .......... .......... .......... 2% 23.0M 36s 2024-03-19T01:11:40.075 INFO:tasks.workunit.client.0.smithi069.stderr: 4000K .......... .......... .......... .......... .......... 2% 21.1M 36s 2024-03-19T01:11:40.093 INFO:tasks.workunit.client.0.smithi069.stderr: 4050K .......... .......... .......... .......... .......... 3% 2.73M 36s 2024-03-19T01:11:40.094 INFO:tasks.workunit.client.0.smithi069.stderr: 4100K .......... .......... .......... .......... .......... 3% 38.7M 36s 2024-03-19T01:11:40.113 INFO:tasks.workunit.client.0.smithi069.stderr: 4150K .......... .......... .......... .......... .......... 3% 2.62M 36s 2024-03-19T01:11:40.113 INFO:tasks.workunit.client.0.smithi069.stderr: 4200K .......... .......... .......... .......... .......... 3% 54.1M 36s 2024-03-19T01:11:40.133 INFO:tasks.workunit.client.0.smithi069.stderr: 4250K .......... .......... .......... .......... .......... 3% 2.64M 36s 2024-03-19T01:11:40.134 INFO:tasks.workunit.client.0.smithi069.stderr: 4300K .......... .......... .......... .......... .......... 3% 24.6M 35s 2024-03-19T01:11:40.151 INFO:tasks.workunit.client.0.smithi069.stderr: 4350K .......... .......... .......... .......... .......... 3% 2.75M 35s 2024-03-19T01:11:40.154 INFO:tasks.workunit.client.0.smithi069.stderr: 4400K .......... .......... .......... .......... .......... 3% 22.7M 35s 2024-03-19T01:11:40.157 INFO:tasks.workunit.client.0.smithi069.stderr: 4450K .......... .......... .......... .......... .......... 3% 15.1M 35s 2024-03-19T01:11:40.174 INFO:tasks.workunit.client.0.smithi069.stderr: 4500K .......... .......... .......... .......... .......... 3% 2.95M 35s 2024-03-19T01:11:40.178 INFO:tasks.workunit.client.0.smithi069.stderr: 4550K .......... .......... .......... .......... .......... 3% 11.6M 35s 2024-03-19T01:11:40.199 INFO:tasks.workunit.client.0.smithi069.stderr: 4600K .......... .......... .......... .......... .......... 3% 2.27M 35s 2024-03-19T01:11:40.203 INFO:tasks.workunit.client.0.smithi069.stderr: 4650K .......... .......... .......... .......... .......... 3% 16.7M 34s 2024-03-19T01:11:40.218 INFO:tasks.workunit.client.0.smithi069.stderr: 4700K .......... .......... .......... .......... .......... 3% 3.00M 35s 2024-03-19T01:11:40.221 INFO:tasks.workunit.client.0.smithi069.stderr: 4750K .......... .......... .......... .......... .......... 3% 15.8M 34s 2024-03-19T01:11:40.229 INFO:tasks.workunit.client.0.smithi069.stderr: 4800K .......... .......... .......... .......... .......... 3% 6.75M 34s 2024-03-19T01:11:40.247 INFO:tasks.workunit.client.0.smithi069.stderr: 4850K .......... .......... .......... .......... .......... 3% 2.58M 34s 2024-03-19T01:11:40.253 INFO:tasks.workunit.client.0.smithi069.stderr: 4900K .......... .......... .......... .......... .......... 3% 8.59M 34s 2024-03-19T01:11:40.271 INFO:tasks.workunit.client.0.smithi069.stderr: 4950K .......... .......... .......... .......... .......... 3% 2.72M 34s 2024-03-19T01:11:40.280 INFO:tasks.workunit.client.0.smithi069.stderr: 5000K .......... .......... .......... .......... .......... 3% 6.00M 34s 2024-03-19T01:11:40.281 INFO:tasks.workunit.client.0.smithi069.stderr: 5050K .......... .......... .......... .......... .......... 3% 23.5M 34s 2024-03-19T01:11:40.302 INFO:tasks.workunit.client.0.smithi069.stderr: 5100K .......... .......... .......... .......... .......... 3% 2.58M 34s 2024-03-19T01:11:40.303 INFO:tasks.workunit.client.0.smithi069.stderr: 5150K .......... .......... .......... .......... .......... 3% 18.2M 34s 2024-03-19T01:11:40.326 INFO:tasks.workunit.client.0.smithi069.stderr: 5200K .......... .......... .......... .......... .......... 3% 2.13M 34s 2024-03-19T01:11:40.330 INFO:tasks.workunit.client.0.smithi069.stderr: 5250K .......... .......... .......... .......... .......... 3% 11.5M 34s 2024-03-19T01:11:40.352 INFO:tasks.workunit.client.0.smithi069.stderr: 5300K .......... .......... .......... .......... .......... 3% 2.23M 34s 2024-03-19T01:11:40.357 INFO:tasks.workunit.client.0.smithi069.stderr: 5350K .......... .......... .......... .......... .......... 3% 11.1M 34s 2024-03-19T01:11:40.362 INFO:tasks.workunit.client.0.smithi069.stderr: 5400K .......... .......... .......... .......... .......... 4% 7.97M 33s 2024-03-19T01:11:40.383 INFO:tasks.workunit.client.0.smithi069.stderr: 5450K .......... .......... .......... .......... .......... 4% 2.34M 34s 2024-03-19T01:11:40.392 INFO:tasks.workunit.client.0.smithi069.stderr: 5500K .......... .......... .......... .......... .......... 4% 5.80M 33s 2024-03-19T01:11:40.409 INFO:tasks.workunit.client.0.smithi069.stderr: 5550K .......... .......... .......... .......... .......... 4% 2.87M 34s 2024-03-19T01:11:40.415 INFO:tasks.workunit.client.0.smithi069.stderr: 5600K .......... .......... .......... .......... .......... 4% 7.48M 33s 2024-03-19T01:11:40.428 INFO:tasks.workunit.client.0.smithi069.stderr: 5650K .......... .......... .......... .......... .......... 4% 3.93M 33s 2024-03-19T01:11:40.443 INFO:tasks.workunit.client.0.smithi069.stderr: 5700K .......... .......... .......... .......... .......... 4% 3.23M 33s 2024-03-19T01:11:40.455 INFO:tasks.workunit.client.0.smithi069.stderr: 5750K .......... .......... .......... .......... .......... 4% 4.22M 33s 2024-03-19T01:11:40.471 INFO:tasks.workunit.client.0.smithi069.stderr: 5800K .......... .......... .......... .......... .......... 4% 3.02M 33s 2024-03-19T01:11:40.479 INFO:tasks.workunit.client.0.smithi069.stderr: 5850K .......... .......... .......... .......... .......... 4% 5.86M 33s 2024-03-19T01:11:40.491 INFO:tasks.workunit.client.0.smithi069.stderr: 5900K .......... .......... .......... .......... .......... 4% 4.14M 33s 2024-03-19T01:11:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:40 smithi086 ceph-mon[66829]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:11:40.509 INFO:tasks.workunit.client.0.smithi069.stderr: 5950K .......... .......... .......... .......... .......... 4% 2.75M 33s 2024-03-19T01:11:40.522 INFO:tasks.workunit.client.0.smithi069.stderr: 6000K .......... .......... .......... .......... .......... 4% 3.74M 33s 2024-03-19T01:11:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:40 smithi069 ceph-mon[31727]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:11:40.538 INFO:tasks.workunit.client.0.smithi069.stderr: 6050K .......... .......... .......... .......... .......... 4% 2.96M 33s 2024-03-19T01:11:40.551 INFO:tasks.workunit.client.0.smithi069.stderr: 6100K .......... .......... .......... .......... .......... 4% 4.34M 33s 2024-03-19T01:11:40.560 INFO:tasks.workunit.client.0.smithi069.stderr: 6150K .......... .......... .......... .......... .......... 4% 4.38M 33s 2024-03-19T01:11:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:40 smithi163 ceph-mon[67428]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:11:40.579 INFO:tasks.workunit.client.0.smithi069.stderr: 6200K .......... .......... .......... .......... .......... 4% 2.58M 33s 2024-03-19T01:11:40.597 INFO:tasks.workunit.client.0.smithi069.stderr: 6250K .......... .......... .......... .......... .......... 4% 2.71M 34s 2024-03-19T01:11:40.612 INFO:tasks.workunit.client.0.smithi069.stderr: 6300K .......... .......... .......... .......... .......... 4% 3.25M 34s 2024-03-19T01:11:40.628 INFO:tasks.workunit.client.0.smithi069.stderr: 6350K .......... .......... .......... .......... .......... 4% 3.23M 34s 2024-03-19T01:11:40.640 INFO:tasks.workunit.client.0.smithi069.stderr: 6400K .......... .......... .......... .......... .......... 4% 3.86M 34s 2024-03-19T01:11:40.660 INFO:tasks.workunit.client.0.smithi069.stderr: 6450K .......... .......... .......... .......... .......... 4% 2.47M 34s 2024-03-19T01:11:40.682 INFO:tasks.workunit.client.0.smithi069.stderr: 6500K .......... .......... .......... .......... .......... 4% 2.37M 34s 2024-03-19T01:11:40.694 INFO:tasks.workunit.client.0.smithi069.stderr: 6550K .......... .......... .......... .......... .......... 4% 3.65M 34s 2024-03-19T01:11:40.713 INFO:tasks.workunit.client.0.smithi069.stderr: 6600K .......... .......... .......... .......... .......... 4% 2.61M 34s 2024-03-19T01:11:40.730 INFO:tasks.workunit.client.0.smithi069.stderr: 6650K .......... .......... .......... .......... .......... 4% 2.86M 34s 2024-03-19T01:11:40.751 INFO:tasks.workunit.client.0.smithi069.stderr: 6700K .......... .......... .......... .......... .......... 4% 2.34M 34s 2024-03-19T01:11:40.773 INFO:tasks.workunit.client.0.smithi069.stderr: 6750K .......... .......... .......... .......... .......... 5% 2.52M 34s 2024-03-19T01:11:40.790 INFO:tasks.workunit.client.0.smithi069.stderr: 6800K .......... .......... .......... .......... .......... 5% 2.40M 34s 2024-03-19T01:11:40.807 INFO:tasks.workunit.client.0.smithi069.stderr: 6850K .......... .......... .......... .......... .......... 5% 3.24M 34s 2024-03-19T01:11:40.826 INFO:tasks.workunit.client.0.smithi069.stderr: 6900K .......... .......... .......... .......... .......... 5% 2.43M 34s 2024-03-19T01:11:40.832 INFO:tasks.workunit.client.0.smithi069.stderr: 6950K .......... .......... .......... .......... .......... 5% 7.64M 34s 2024-03-19T01:11:40.838 INFO:tasks.workunit.client.0.smithi069.stderr: 7000K .......... .......... .......... .......... .......... 5% 8.01M 34s 2024-03-19T01:11:40.847 INFO:tasks.workunit.client.0.smithi069.stderr: 7050K .......... .......... .......... .......... .......... 5% 5.35M 34s 2024-03-19T01:11:40.856 INFO:tasks.workunit.client.0.smithi069.stderr: 7100K .......... .......... .......... .......... .......... 5% 5.59M 34s 2024-03-19T01:11:40.867 INFO:tasks.workunit.client.0.smithi069.stderr: 7150K .......... .......... .......... .......... .......... 5% 4.53M 34s 2024-03-19T01:11:40.872 INFO:tasks.workunit.client.0.smithi069.stderr: 7200K .......... .......... .......... .......... .......... 5% 13.9M 34s 2024-03-19T01:11:40.878 INFO:tasks.workunit.client.0.smithi069.stderr: 7250K .......... .......... .......... .......... .......... 5% 6.16M 34s 2024-03-19T01:11:40.888 INFO:tasks.workunit.client.0.smithi069.stderr: 7300K .......... .......... .......... .......... .......... 5% 5.25M 34s 2024-03-19T01:11:40.894 INFO:tasks.workunit.client.0.smithi069.stderr: 7350K .......... .......... .......... .......... .......... 5% 7.65M 33s 2024-03-19T01:11:40.901 INFO:tasks.workunit.client.0.smithi069.stderr: 7400K .......... .......... .......... .......... .......... 5% 6.93M 33s 2024-03-19T01:11:40.908 INFO:tasks.workunit.client.0.smithi069.stderr: 7450K .......... .......... .......... .......... .......... 5% 6.84M 33s 2024-03-19T01:11:40.918 INFO:tasks.workunit.client.0.smithi069.stderr: 7500K .......... .......... .......... .......... .......... 5% 5.07M 33s 2024-03-19T01:11:40.931 INFO:tasks.workunit.client.0.smithi069.stderr: 7550K .......... .......... .......... .......... .......... 5% 4.64M 33s 2024-03-19T01:11:40.933 INFO:tasks.workunit.client.0.smithi069.stderr: 7600K .......... .......... .......... .......... .......... 5% 9.80M 33s 2024-03-19T01:11:40.939 INFO:tasks.workunit.client.0.smithi069.stderr: 7650K .......... .......... .......... .......... .......... 5% 7.76M 33s 2024-03-19T01:11:40.949 INFO:tasks.workunit.client.0.smithi069.stderr: 7700K .......... .......... .......... .......... .......... 5% 5.26M 33s 2024-03-19T01:11:40.956 INFO:tasks.workunit.client.0.smithi069.stderr: 7750K .......... .......... .......... .......... .......... 5% 6.40M 33s 2024-03-19T01:11:40.969 INFO:tasks.workunit.client.0.smithi069.stderr: 7800K .......... .......... .......... .......... .......... 5% 9.00M 33s 2024-03-19T01:11:40.970 INFO:tasks.workunit.client.0.smithi069.stderr: 7850K .......... .......... .......... .......... .......... 5% 5.94M 32s 2024-03-19T01:11:40.980 INFO:tasks.workunit.client.0.smithi069.stderr: 7900K .......... .......... .......... .......... .......... 5% 4.98M 32s 2024-03-19T01:11:40.990 INFO:tasks.workunit.client.0.smithi069.stderr: 7950K .......... .......... .......... .......... .......... 5% 5.11M 32s 2024-03-19T01:11:40.995 INFO:tasks.workunit.client.0.smithi069.stderr: 8000K .......... .......... .......... .......... .......... 5% 9.75M 32s 2024-03-19T01:11:41.001 INFO:tasks.workunit.client.0.smithi069.stderr: 8050K .......... .......... .......... .......... .......... 5% 7.82M 32s 2024-03-19T01:11:41.011 INFO:tasks.workunit.client.0.smithi069.stderr: 8100K .......... .......... .......... .......... .......... 6% 5.07M 32s 2024-03-19T01:11:41.017 INFO:tasks.workunit.client.0.smithi069.stderr: 8150K .......... .......... .......... .......... .......... 6% 6.78M 32s 2024-03-19T01:11:41.023 INFO:tasks.workunit.client.0.smithi069.stderr: 8200K .......... .......... .......... .......... .......... 6% 9.43M 32s 2024-03-19T01:11:41.031 INFO:tasks.workunit.client.0.smithi069.stderr: 8250K .......... .......... .......... .......... .......... 6% 5.89M 32s 2024-03-19T01:11:41.041 INFO:tasks.workunit.client.0.smithi069.stderr: 8300K .......... .......... .......... .......... .......... 6% 4.86M 32s 2024-03-19T01:11:41.051 INFO:tasks.workunit.client.0.smithi069.stderr: 8350K .......... .......... .......... .......... .......... 6% 5.28M 32s 2024-03-19T01:11:41.053 INFO:tasks.workunit.client.0.smithi069.stderr: 8400K .......... .......... .......... .......... .......... 6% 16.4M 31s 2024-03-19T01:11:41.062 INFO:tasks.workunit.client.0.smithi069.stderr: 8450K .......... .......... .......... .......... .......... 6% 5.91M 31s 2024-03-19T01:11:41.071 INFO:tasks.workunit.client.0.smithi069.stderr: 8500K .......... .......... .......... .......... .......... 6% 5.16M 31s 2024-03-19T01:11:41.076 INFO:tasks.workunit.client.0.smithi069.stderr: 8550K .......... .......... .......... .......... .......... 6% 9.69M 31s 2024-03-19T01:11:41.082 INFO:tasks.workunit.client.0.smithi069.stderr: 8600K .......... .......... .......... .......... .......... 6% 7.87M 31s 2024-03-19T01:11:41.092 INFO:tasks.workunit.client.0.smithi069.stderr: 8650K .......... .......... .......... .......... .......... 6% 5.24M 31s 2024-03-19T01:11:41.101 INFO:tasks.workunit.client.0.smithi069.stderr: 8700K .......... .......... .......... .......... .......... 6% 5.14M 31s 2024-03-19T01:11:41.104 INFO:tasks.workunit.client.0.smithi069.stderr: 8750K .......... .......... .......... .......... .......... 6% 16.5M 31s 2024-03-19T01:11:41.112 INFO:tasks.workunit.client.0.smithi069.stderr: 8800K .......... .......... .......... .......... .......... 6% 5.87M 31s 2024-03-19T01:11:41.121 INFO:tasks.workunit.client.0.smithi069.stderr: 8850K .......... .......... .......... .......... .......... 6% 5.22M 31s 2024-03-19T01:11:41.134 INFO:tasks.workunit.client.0.smithi069.stderr: 8900K .......... .......... .......... .......... .......... 6% 5.01M 31s 2024-03-19T01:11:41.135 INFO:tasks.workunit.client.0.smithi069.stderr: 8950K .......... .......... .......... .......... .......... 6% 33.7M 31s 2024-03-19T01:11:41.143 INFO:tasks.workunit.client.0.smithi069.stderr: 9000K .......... .......... .......... .......... .......... 6% 4.81M 30s 2024-03-19T01:11:41.153 INFO:tasks.workunit.client.0.smithi069.stderr: 9050K .......... .......... .......... .......... .......... 6% 5.34M 30s 2024-03-19T01:11:41.157 INFO:tasks.workunit.client.0.smithi069.stderr: 9100K .......... .......... .......... .......... .......... 6% 9.44M 30s 2024-03-19T01:11:41.164 INFO:tasks.workunit.client.0.smithi069.stderr: 9150K .......... .......... .......... .......... .......... 6% 7.73M 30s 2024-03-19T01:11:41.173 INFO:tasks.workunit.client.0.smithi069.stderr: 9200K .......... .......... .......... .......... .......... 6% 5.26M 30s 2024-03-19T01:11:41.178 INFO:tasks.workunit.client.0.smithi069.stderr: 9250K .......... .......... .......... .......... .......... 6% 10.0M 30s 2024-03-19T01:11:41.184 INFO:tasks.workunit.client.0.smithi069.stderr: 9300K .......... .......... .......... .......... .......... 6% 7.39M 30s 2024-03-19T01:11:41.193 INFO:tasks.workunit.client.0.smithi069.stderr: 9350K .......... .......... .......... .......... .......... 6% 5.52M 30s 2024-03-19T01:11:41.201 INFO:tasks.workunit.client.0.smithi069.stderr: 9400K .......... .......... .......... .......... .......... 6% 6.37M 30s 2024-03-19T01:11:41.205 INFO:tasks.workunit.client.0.smithi069.stderr: 9450K .......... .......... .......... .......... .......... 6% 10.5M 30s 2024-03-19T01:11:41.214 INFO:tasks.workunit.client.0.smithi069.stderr: 9500K .......... .......... .......... .......... .......... 7% 5.82M 30s 2024-03-19T01:11:41.224 INFO:tasks.workunit.client.0.smithi069.stderr: 9550K .......... .......... .......... .......... .......... 7% 4.87M 30s 2024-03-19T01:11:41.234 INFO:tasks.workunit.client.0.smithi069.stderr: 9600K .......... .......... .......... .......... .......... 7% 5.24M 30s 2024-03-19T01:11:41.236 INFO:tasks.workunit.client.0.smithi069.stderr: 9650K .......... .......... .......... .......... .......... 7% 47.8M 29s 2024-03-19T01:11:41.245 INFO:tasks.workunit.client.0.smithi069.stderr: 9700K .......... .......... .......... .......... .......... 7% 4.76M 29s 2024-03-19T01:11:41.255 INFO:tasks.workunit.client.0.smithi069.stderr: 9750K .......... .......... .......... .......... .......... 7% 5.27M 29s 2024-03-19T01:11:41.256 INFO:tasks.workunit.client.0.smithi069.stderr: 9800K .......... .......... .......... .......... .......... 7% 17.8M 29s 2024-03-19T01:11:41.265 INFO:tasks.workunit.client.0.smithi069.stderr: 9850K .......... .......... .......... .......... .......... 7% 5.87M 29s 2024-03-19T01:11:41.275 INFO:tasks.workunit.client.0.smithi069.stderr: 9900K .......... .......... .......... .......... .......... 7% 5.16M 29s 2024-03-19T01:11:41.279 INFO:tasks.workunit.client.0.smithi069.stderr: 9950K .......... .......... .......... .......... .......... 7% 9.41M 29s 2024-03-19T01:11:41.285 INFO:tasks.workunit.client.0.smithi069.stderr: 10000K .......... .......... .......... .......... .......... 7% 8.18M 29s 2024-03-19T01:11:41.295 INFO:tasks.workunit.client.0.smithi069.stderr: 10050K .......... .......... .......... .......... .......... 7% 5.23M 29s 2024-03-19T01:11:41.300 INFO:tasks.workunit.client.0.smithi069.stderr: 10100K .......... .......... .......... .......... .......... 7% 9.29M 29s 2024-03-19T01:11:41.306 INFO:tasks.workunit.client.0.smithi069.stderr: 10150K .......... .......... .......... .......... .......... 7% 8.38M 29s 2024-03-19T01:11:41.316 INFO:tasks.workunit.client.0.smithi069.stderr: 10200K .......... .......... .......... .......... .......... 7% 4.98M 29s 2024-03-19T01:11:41.323 INFO:tasks.workunit.client.0.smithi069.stderr: 10250K .......... .......... .......... .......... .......... 7% 6.56M 29s 2024-03-19T01:11:41.327 INFO:tasks.workunit.client.0.smithi069.stderr: 10300K .......... .......... .......... .......... .......... 7% 13.4M 29s 2024-03-19T01:11:41.336 INFO:tasks.workunit.client.0.smithi069.stderr: 10350K .......... .......... .......... .......... .......... 7% 5.45M 29s 2024-03-19T01:11:41.343 INFO:tasks.workunit.client.0.smithi069.stderr: 10400K .......... .......... .......... .......... .......... 7% 6.24M 29s 2024-03-19T01:11:41.347 INFO:tasks.workunit.client.0.smithi069.stderr: 10450K .......... .......... .......... .......... .......... 7% 14.3M 28s 2024-03-19T01:11:41.356 INFO:tasks.workunit.client.0.smithi069.stderr: 10500K .......... .......... .......... .......... .......... 7% 5.34M 28s 2024-03-19T01:11:41.364 INFO:tasks.workunit.client.0.smithi069.stderr: 10550K .......... .......... .......... .......... .......... 7% 6.30M 28s 2024-03-19T01:11:41.368 INFO:tasks.workunit.client.0.smithi069.stderr: 10600K .......... .......... .......... .......... .......... 7% 11.7M 28s 2024-03-19T01:11:41.378 INFO:tasks.workunit.client.0.smithi069.stderr: 10650K .......... .......... .......... .......... .......... 7% 5.31M 28s 2024-03-19T01:11:41.384 INFO:tasks.workunit.client.0.smithi069.stderr: 10700K .......... .......... .......... .......... .......... 7% 7.06M 28s 2024-03-19T01:11:41.388 INFO:tasks.workunit.client.0.smithi069.stderr: 10750K .......... .......... .......... .......... .......... 7% 11.3M 28s 2024-03-19T01:11:41.398 INFO:tasks.workunit.client.0.smithi069.stderr: 10800K .......... .......... .......... .......... .......... 7% 5.27M 28s 2024-03-19T01:11:41.406 INFO:tasks.workunit.client.0.smithi069.stderr: 10850K .......... .......... .......... .......... .......... 8% 5.76M 28s 2024-03-19T01:11:41.409 INFO:tasks.workunit.client.0.smithi069.stderr: 10900K .......... .......... .......... .......... .......... 8% 18.9M 28s 2024-03-19T01:11:41.418 INFO:tasks.workunit.client.0.smithi069.stderr: 10950K .......... .......... .......... .......... .......... 8% 5.30M 28s 2024-03-19T01:11:41.427 INFO:tasks.workunit.client.0.smithi069.stderr: 11000K .......... .......... .......... .......... .......... 8% 5.43M 28s 2024-03-19T01:11:41.429 INFO:tasks.workunit.client.0.smithi069.stderr: 11050K .......... .......... .......... .......... .......... 8% 23.7M 28s 2024-03-19T01:11:41.439 INFO:tasks.workunit.client.0.smithi069.stderr: 11100K .......... .......... .......... .......... .......... 8% 5.12M 28s 2024-03-19T01:11:41.447 INFO:tasks.workunit.client.0.smithi069.stderr: 11150K .......... .......... .......... .......... .......... 8% 5.64M 28s 2024-03-19T01:11:41.449 INFO:tasks.workunit.client.0.smithi069.stderr: 11200K .......... .......... .......... .......... .......... 8% 23.4M 27s 2024-03-19T01:11:41.459 INFO:tasks.workunit.client.0.smithi069.stderr: 11250K .......... .......... .......... .......... .......... 8% 5.03M 27s 2024-03-19T01:11:41.468 INFO:tasks.workunit.client.0.smithi069.stderr: 11300K .......... .......... .......... .......... .......... 8% 5.63M 27s 2024-03-19T01:11:41.479 INFO:tasks.workunit.client.0.smithi069.stderr: 11350K .......... .......... .......... .......... .......... 8% 4.71M 27s 2024-03-19T01:11:41.480 INFO:tasks.workunit.client.0.smithi069.stderr: 11400K .......... .......... .......... .......... .......... 8% 39.4M 27s 2024-03-19T01:11:41.488 INFO:tasks.workunit.client.0.smithi069.stderr: 11450K .......... .......... .......... .......... .......... 8% 5.53M 27s 2024-03-19T01:11:41.499 INFO:tasks.workunit.client.0.smithi069.stderr: 11500K .......... .......... .......... .......... .......... 8% 4.65M 27s 2024-03-19T01:11:41.499 INFO:tasks.workunit.client.0.smithi069.stderr: 11550K .......... .......... .......... .......... .......... 8% 59.6M 27s 2024-03-19T01:11:41.509 INFO:tasks.workunit.client.0.smithi069.stderr: 11600K .......... .......... .......... .......... .......... 8% 5.30M 27s 2024-03-19T01:11:41.521 INFO:tasks.workunit.client.0.smithi069.stderr: 11650K .......... .......... .......... .......... .......... 8% 4.67M 27s 2024-03-19T01:11:41.521 INFO:tasks.workunit.client.0.smithi069.stderr: 11700K .......... .......... .......... .......... .......... 8% 38.3M 27s 2024-03-19T01:11:41.530 INFO:tasks.workunit.client.0.smithi069.stderr: 11750K .......... .......... .......... .......... .......... 8% 5.60M 27s 2024-03-19T01:11:41.540 INFO:tasks.workunit.client.0.smithi069.stderr: 11800K .......... .......... .......... .......... .......... 8% 31.8M 27s 2024-03-19T01:11:41.540 INFO:tasks.workunit.client.0.smithi069.stderr: 11850K .......... .......... .......... .......... .......... 8% 5.09M 27s 2024-03-19T01:11:41.550 INFO:tasks.workunit.client.0.smithi069.stderr: 11900K .......... .......... .......... .......... .......... 8% 5.23M 27s 2024-03-19T01:11:41.561 INFO:tasks.workunit.client.0.smithi069.stderr: 11950K .......... .......... .......... .......... .......... 8% 4.67M 27s 2024-03-19T01:11:41.561 INFO:tasks.workunit.client.0.smithi069.stderr: 12000K .......... .......... .......... .......... .......... 8% 68.9M 27s 2024-03-19T01:11:41.567 INFO:tasks.workunit.client.0.smithi069.stderr: 12050K .......... .......... .......... .......... .......... 8% 10.8M 27s 2024-03-19T01:11:41.571 INFO:tasks.workunit.client.0.smithi069.stderr: 12100K .......... .......... .......... .......... .......... 8% 8.44M 27s 2024-03-19T01:11:41.581 INFO:tasks.workunit.client.0.smithi069.stderr: 12150K .......... .......... .......... .......... .......... 8% 4.89M 26s 2024-03-19T01:11:41.585 INFO:tasks.workunit.client.0.smithi069.stderr: 12200K .......... .......... .......... .......... .......... 9% 10.9M 26s 2024-03-19T01:11:41.591 INFO:tasks.workunit.client.0.smithi069.stderr: 12250K .......... .......... .......... .......... .......... 9% 8.72M 26s 2024-03-19T01:11:41.601 INFO:tasks.workunit.client.0.smithi069.stderr: 12300K .......... .......... .......... .......... .......... 9% 4.82M 26s 2024-03-19T01:11:41.605 INFO:tasks.workunit.client.0.smithi069.stderr: 12350K .......... .......... .......... .......... .......... 9% 12.8M 26s 2024-03-19T01:11:41.612 INFO:tasks.workunit.client.0.smithi069.stderr: 12400K .......... .......... .......... .......... .......... 9% 7.38M 26s 2024-03-19T01:11:41.623 INFO:tasks.workunit.client.0.smithi069.stderr: 12450K .......... .......... .......... .......... .......... 9% 4.72M 26s 2024-03-19T01:11:41.623 INFO:tasks.workunit.client.0.smithi069.stderr: 12500K .......... .......... .......... .......... .......... 9% 39.8M 26s 2024-03-19T01:11:41.632 INFO:tasks.workunit.client.0.smithi069.stderr: 12550K .......... .......... .......... .......... .......... 9% 5.61M 26s 2024-03-19T01:11:41.643 INFO:tasks.workunit.client.0.smithi069.stderr: 12600K .......... .......... .......... .......... .......... 9% 4.88M 26s 2024-03-19T01:11:41.643 INFO:tasks.workunit.client.0.smithi069.stderr: 12650K .......... .......... .......... .......... .......... 9% 40.6M 26s 2024-03-19T01:11:41.652 INFO:tasks.workunit.client.0.smithi069.stderr: 12700K .......... .......... .......... .......... .......... 9% 5.57M 26s 2024-03-19T01:11:41.664 INFO:tasks.workunit.client.0.smithi069.stderr: 12750K .......... .......... .......... .......... .......... 9% 4.75M 26s 2024-03-19T01:11:41.664 INFO:tasks.workunit.client.0.smithi069.stderr: 12800K .......... .......... .......... .......... .......... 9% 32.8M 26s 2024-03-19T01:11:41.673 INFO:tasks.workunit.client.0.smithi069.stderr: 12850K .......... .......... .......... .......... .......... 9% 5.77M 26s 2024-03-19T01:11:41.684 INFO:tasks.workunit.client.0.smithi069.stderr: 12900K .......... .......... .......... .......... .......... 9% 4.82M 26s 2024-03-19T01:11:41.684 INFO:tasks.workunit.client.0.smithi069.stderr: 12950K .......... .......... .......... .......... .......... 9% 43.7M 26s 2024-03-19T01:11:41.693 INFO:tasks.workunit.client.0.smithi069.stderr: 13000K .......... .......... .......... .......... .......... 9% 5.50M 26s 2024-03-19T01:11:41.704 INFO:tasks.workunit.client.0.smithi069.stderr: 13050K .......... .......... .......... .......... .......... 9% 4.75M 26s 2024-03-19T01:11:41.704 INFO:tasks.workunit.client.0.smithi069.stderr: 13100K .......... .......... .......... .......... .......... 9% 44.2M 26s 2024-03-19T01:11:41.713 INFO:tasks.workunit.client.0.smithi069.stderr: 13150K .......... .......... .......... .......... .......... 9% 8.00M 25s 2024-03-19T01:11:41.725 INFO:tasks.workunit.client.0.smithi069.stderr: 13200K .......... .......... .......... .......... .......... 9% 3.71M 25s 2024-03-19T01:11:41.725 INFO:tasks.workunit.client.0.smithi069.stderr: 13250K .......... .......... .......... .......... .......... 9% 37.9M 25s 2024-03-19T01:11:41.730 INFO:tasks.workunit.client.0.smithi069.stderr: 13300K .......... .......... .......... .......... .......... 9% 8.36M 25s 2024-03-19T01:11:41.734 INFO:tasks.workunit.client.0.smithi069.stderr: 13350K .......... .......... .......... .......... .......... 9% 12.2M 25s 2024-03-19T01:11:41.745 INFO:tasks.workunit.client.0.smithi069.stderr: 13400K .......... .......... .......... .......... .......... 9% 4.70M 25s 2024-03-19T01:11:41.749 INFO:tasks.workunit.client.0.smithi069.stderr: 13450K .......... .......... .......... .......... .......... 9% 13.6M 25s 2024-03-19T01:11:41.754 INFO:tasks.workunit.client.0.smithi069.stderr: 13500K .......... .......... .......... .......... .......... 9% 7.92M 25s 2024-03-19T01:11:41.765 INFO:tasks.workunit.client.0.smithi069.stderr: 13550K .......... .......... .......... .......... .......... 10% 4.72M 25s 2024-03-19T01:11:41.768 INFO:tasks.workunit.client.0.smithi069.stderr: 13600K .......... .......... .......... .......... .......... 10% 40.4M 25s 2024-03-19T01:11:41.775 INFO:tasks.workunit.client.0.smithi069.stderr: 13650K .......... .......... .......... .......... .......... 10% 5.65M 25s 2024-03-19T01:11:41.785 INFO:tasks.workunit.client.0.smithi069.stderr: 13700K .......... .......... .......... .......... .......... 10% 4.89M 25s 2024-03-19T01:11:41.785 INFO:tasks.workunit.client.0.smithi069.stderr: 13750K .......... .......... .......... .......... .......... 10% 53.2M 25s 2024-03-19T01:11:41.795 INFO:tasks.workunit.client.0.smithi069.stderr: 13800K .......... .......... .......... .......... .......... 10% 5.42M 25s 2024-03-19T01:11:41.806 INFO:tasks.workunit.client.0.smithi069.stderr: 13850K .......... .......... .......... .......... .......... 10% 4.68M 25s 2024-03-19T01:11:41.806 INFO:tasks.workunit.client.0.smithi069.stderr: 13900K .......... .......... .......... .......... .......... 10% 38.7M 25s 2024-03-19T01:11:41.815 INFO:tasks.workunit.client.0.smithi069.stderr: 13950K .......... .......... .......... .......... .......... 10% 5.88M 25s 2024-03-19T01:11:41.826 INFO:tasks.workunit.client.0.smithi069.stderr: 14000K .......... .......... .......... .......... .......... 10% 4.55M 25s 2024-03-19T01:11:41.826 INFO:tasks.workunit.client.0.smithi069.stderr: 14050K .......... .......... .......... .......... .......... 10% 63.0M 25s 2024-03-19T01:11:41.832 INFO:tasks.workunit.client.0.smithi069.stderr: 14100K .......... .......... .......... .......... .......... 10% 13.4M 25s 2024-03-19T01:11:41.836 INFO:tasks.workunit.client.0.smithi069.stderr: 14150K .......... .......... .......... .......... .......... 10% 8.05M 25s 2024-03-19T01:11:41.848 INFO:tasks.workunit.client.0.smithi069.stderr: 14200K .......... .......... .......... .......... .......... 10% 4.45M 25s 2024-03-19T01:11:41.850 INFO:tasks.workunit.client.0.smithi069.stderr: 14250K .......... .......... .......... .......... .......... 10% 17.2M 24s 2024-03-19T01:11:41.856 INFO:tasks.workunit.client.0.smithi069.stderr: 14300K .......... .......... .......... .......... .......... 10% 7.57M 24s 2024-03-19T01:11:41.868 INFO:tasks.workunit.client.0.smithi069.stderr: 14350K .......... .......... .......... .......... .......... 10% 4.40M 24s 2024-03-19T01:11:41.869 INFO:tasks.workunit.client.0.smithi069.stderr: 14400K .......... .......... .......... .......... .......... 10% 42.0M 24s 2024-03-19T01:11:41.876 INFO:tasks.workunit.client.0.smithi069.stderr: 14450K .......... .......... .......... .......... .......... 10% 6.22M 24s 2024-03-19T01:11:41.888 INFO:tasks.workunit.client.0.smithi069.stderr: 14500K .......... .......... .......... .......... .......... 10% 4.54M 24s 2024-03-19T01:11:41.888 INFO:tasks.workunit.client.0.smithi069.stderr: 14550K .......... .......... .......... .......... .......... 10% 73.4M 24s 2024-03-19T01:11:41.897 INFO:tasks.workunit.client.0.smithi069.stderr: 14600K .......... .......... .......... .......... .......... 10% 5.55M 24s 2024-03-19T01:11:41.906 INFO:tasks.workunit.client.0.smithi069.stderr: 14650K .......... .......... .......... .......... .......... 10% 4.70M 24s 2024-03-19T01:11:41.908 INFO:tasks.workunit.client.0.smithi069.stderr: 14700K .......... .......... .......... .......... .......... 10% 48.8M 24s 2024-03-19T01:11:41.910 INFO:tasks.workunit.client.0.smithi069.stderr: 14750K .......... .......... .......... .......... .......... 10% 17.9M 24s 2024-03-19T01:11:41.917 INFO:tasks.workunit.client.0.smithi069.stderr: 14800K .......... .......... .......... .......... .......... 10% 7.21M 24s 2024-03-19T01:11:41.929 INFO:tasks.workunit.client.0.smithi069.stderr: 14850K .......... .......... .......... .......... .......... 10% 4.52M 24s 2024-03-19T01:11:41.929 INFO:tasks.workunit.client.0.smithi069.stderr: 14900K .......... .......... .......... .......... .......... 11% 81.1M 24s 2024-03-19T01:11:41.938 INFO:tasks.workunit.client.0.smithi069.stderr: 14950K .......... .......... .......... .......... .......... 11% 5.54M 24s 2024-03-19T01:11:41.949 INFO:tasks.workunit.client.0.smithi069.stderr: 15000K .......... .......... .......... .......... .......... 11% 4.53M 24s 2024-03-19T01:11:41.949 INFO:tasks.workunit.client.0.smithi069.stderr: 15050K .......... .......... .......... .......... .......... 11% 73.1M 24s 2024-03-19T01:11:41.958 INFO:tasks.workunit.client.0.smithi069.stderr: 15100K .......... .......... .......... .......... .......... 11% 5.58M 24s 2024-03-19T01:11:41.959 INFO:tasks.workunit.client.0.smithi069.stderr: 15150K .......... .......... .......... .......... .......... 11% 71.3M 24s 2024-03-19T01:11:41.970 INFO:tasks.workunit.client.0.smithi069.stderr: 15200K .......... .......... .......... .......... .......... 11% 4.62M 24s 2024-03-19T01:11:41.971 INFO:tasks.workunit.client.0.smithi069.stderr: 15250K .......... .......... .......... .......... .......... 11% 18.6M 24s 2024-03-19T01:11:41.979 INFO:tasks.workunit.client.0.smithi069.stderr: 15300K .......... .......... .......... .......... .......... 11% 6.84M 24s 2024-03-19T01:11:41.989 INFO:tasks.workunit.client.0.smithi069.stderr: 15350K .......... .......... .......... .......... .......... 11% 4.70M 24s 2024-03-19T01:11:41.990 INFO:tasks.workunit.client.0.smithi069.stderr: 15400K .......... .......... .......... .......... .......... 11% 73.1M 24s 2024-03-19T01:11:41.999 INFO:tasks.workunit.client.0.smithi069.stderr: 15450K .......... .......... .......... .......... .......... 11% 5.46M 24s 2024-03-19T01:11:42.010 INFO:tasks.workunit.client.0.smithi069.stderr: 15500K .......... .......... .......... .......... .......... 11% 4.63M 24s 2024-03-19T01:11:42.010 INFO:tasks.workunit.client.0.smithi069.stderr: 15550K .......... .......... .......... .......... .......... 11% 72.4M 23s 2024-03-19T01:11:42.019 INFO:tasks.workunit.client.0.smithi069.stderr: 15600K .......... .......... .......... .......... .......... 11% 5.52M 23s 2024-03-19T01:11:42.029 INFO:tasks.workunit.client.0.smithi069.stderr: 15650K .......... .......... .......... .......... .......... 11% 69.9M 23s 2024-03-19T01:11:42.031 INFO:tasks.workunit.client.0.smithi069.stderr: 15700K .......... .......... .......... .......... .......... 11% 4.43M 23s 2024-03-19T01:11:42.032 INFO:tasks.workunit.client.0.smithi069.stderr: 15750K .......... .......... .......... .......... .......... 11% 43.6M 23s 2024-03-19T01:11:42.040 INFO:tasks.workunit.client.0.smithi069.stderr: 15800K .......... .......... .......... .......... .......... 11% 5.85M 23s 2024-03-19T01:11:42.050 INFO:tasks.workunit.client.0.smithi069.stderr: 15850K .......... .......... .......... .......... .......... 11% 4.73M 23s 2024-03-19T01:11:42.051 INFO:tasks.workunit.client.0.smithi069.stderr: 15900K .......... .......... .......... .......... .......... 11% 39.9M 23s 2024-03-19T01:11:42.060 INFO:tasks.workunit.client.0.smithi069.stderr: 15950K .......... .......... .......... .......... .......... 11% 6.00M 23s 2024-03-19T01:11:42.071 INFO:tasks.workunit.client.0.smithi069.stderr: 16000K .......... .......... .......... .......... .......... 11% 4.66M 23s 2024-03-19T01:11:42.071 INFO:tasks.workunit.client.0.smithi069.stderr: 16050K .......... .......... .......... .......... .......... 11% 60.8M 23s 2024-03-19T01:11:42.076 INFO:tasks.workunit.client.0.smithi069.stderr: 16100K .......... .......... .......... .......... .......... 11% 8.77M 23s 2024-03-19T01:11:42.081 INFO:tasks.workunit.client.0.smithi069.stderr: 16150K .......... .......... .......... .......... .......... 11% 11.7M 23s 2024-03-19T01:11:42.091 INFO:tasks.workunit.client.0.smithi069.stderr: 16200K .......... .......... .......... .......... .......... 11% 4.70M 23s 2024-03-19T01:11:42.091 INFO:tasks.workunit.client.0.smithi069.stderr: 16250K .......... .......... .......... .......... .......... 12% 80.1M 23s 2024-03-19T01:11:42.101 INFO:tasks.workunit.client.0.smithi069.stderr: 16300K .......... .......... .......... .......... .......... 12% 5.40M 23s 2024-03-19T01:11:42.111 INFO:tasks.workunit.client.0.smithi069.stderr: 16350K .......... .......... .......... .......... .......... 12% 4.61M 23s 2024-03-19T01:11:42.113 INFO:tasks.workunit.client.0.smithi069.stderr: 16400K .......... .......... .......... .......... .......... 12% 48.7M 23s 2024-03-19T01:11:42.121 INFO:tasks.workunit.client.0.smithi069.stderr: 16450K .......... .......... .......... .......... .......... 12% 5.78M 23s 2024-03-19T01:11:42.130 INFO:tasks.workunit.client.0.smithi069.stderr: 16500K .......... .......... .......... .......... .......... 12% 4.98M 23s 2024-03-19T01:11:42.133 INFO:tasks.workunit.client.0.smithi069.stderr: 16550K .......... .......... .......... .......... .......... 12% 27.8M 23s 2024-03-19T01:11:42.134 INFO:tasks.workunit.client.0.smithi069.stderr: 16600K .......... .......... .......... .......... .......... 12% 23.2M 23s 2024-03-19T01:11:42.142 INFO:tasks.workunit.client.0.smithi069.stderr: 16650K .......... .......... .......... .......... .......... 12% 6.61M 23s 2024-03-19T01:11:42.153 INFO:tasks.workunit.client.0.smithi069.stderr: 16700K .......... .......... .......... .......... .......... 12% 4.75M 23s 2024-03-19T01:11:42.153 INFO:tasks.workunit.client.0.smithi069.stderr: 16750K .......... .......... .......... .......... .......... 12% 40.7M 23s 2024-03-19T01:11:42.162 INFO:tasks.workunit.client.0.smithi069.stderr: 16800K .......... .......... .......... .......... .......... 12% 5.68M 23s 2024-03-19T01:11:42.171 INFO:tasks.workunit.client.0.smithi069.stderr: 16850K .......... .......... .......... .......... .......... 12% 5.10M 23s 2024-03-19T01:11:42.173 INFO:tasks.workunit.client.0.smithi069.stderr: 16900K .......... .......... .......... .......... .......... 12% 31.4M 22s 2024-03-19T01:11:42.178 INFO:tasks.workunit.client.0.smithi069.stderr: 16950K .......... .......... .......... .......... .......... 12% 21.9M 22s 2024-03-19T01:11:42.182 INFO:tasks.workunit.client.0.smithi069.stderr: 17000K .......... .......... .......... .......... .......... 12% 6.91M 22s 2024-03-19T01:11:42.195 INFO:tasks.workunit.client.0.smithi069.stderr: 17050K .......... .......... .......... .......... .......... 12% 4.48M 22s 2024-03-19T01:11:42.195 INFO:tasks.workunit.client.0.smithi069.stderr: 17100K .......... .......... .......... .......... .......... 12% 44.1M 22s 2024-03-19T01:11:42.203 INFO:tasks.workunit.client.0.smithi069.stderr: 17150K .......... .......... .......... .......... .......... 12% 5.95M 22s 2024-03-19T01:11:42.214 INFO:tasks.workunit.client.0.smithi069.stderr: 17200K .......... .......... .......... .......... .......... 12% 4.89M 22s 2024-03-19T01:11:42.214 INFO:tasks.workunit.client.0.smithi069.stderr: 17250K .......... .......... .......... .......... .......... 12% 30.6M 22s 2024-03-19T01:11:42.216 INFO:tasks.workunit.client.0.smithi069.stderr: 17300K .......... .......... .......... .......... .......... 12% 27.3M 22s 2024-03-19T01:11:42.223 INFO:tasks.workunit.client.0.smithi069.stderr: 17350K .......... .......... .......... .......... .......... 12% 6.89M 22s 2024-03-19T01:11:42.233 INFO:tasks.workunit.client.0.smithi069.stderr: 17400K .......... .......... .......... .......... .......... 12% 4.72M 22s 2024-03-19T01:11:42.235 INFO:tasks.workunit.client.0.smithi069.stderr: 17450K .......... .......... .......... .......... .......... 12% 32.8M 22s 2024-03-19T01:11:42.243 INFO:tasks.workunit.client.0.smithi069.stderr: 17500K .......... .......... .......... .......... .......... 12% 5.92M 22s 2024-03-19T01:11:42.256 INFO:tasks.workunit.client.0.smithi069.stderr: 17550K .......... .......... .......... .......... .......... 12% 4.74M 22s 2024-03-19T01:11:42.256 INFO:tasks.workunit.client.0.smithi069.stderr: 17600K .......... .......... .......... .......... .......... 13% 39.3M 22s 2024-03-19T01:11:42.256 INFO:tasks.workunit.client.0.smithi069.stderr: 17650K .......... .......... .......... .......... .......... 13% 37.6M 22s 2024-03-19T01:11:42.263 INFO:tasks.workunit.client.0.smithi069.stderr: 17700K .......... .......... .......... .......... .......... 13% 6.33M 22s 2024-03-19T01:11:42.274 INFO:tasks.workunit.client.0.smithi069.stderr: 17750K .......... .......... .......... .......... .......... 13% 4.75M 22s 2024-03-19T01:11:42.276 INFO:tasks.workunit.client.0.smithi069.stderr: 17800K .......... .......... .......... .......... .......... 13% 31.8M 22s 2024-03-19T01:11:42.284 INFO:tasks.workunit.client.0.smithi069.stderr: 17850K .......... .......... .......... .......... .......... 13% 5.97M 22s 2024-03-19T01:11:42.296 INFO:tasks.workunit.client.0.smithi069.stderr: 17900K .......... .......... .......... .......... .......... 13% 4.61M 22s 2024-03-19T01:11:42.296 INFO:tasks.workunit.client.0.smithi069.stderr: 17950K .......... .......... .......... .......... .......... 13% 44.6M 22s 2024-03-19T01:11:42.296 INFO:tasks.workunit.client.0.smithi069.stderr: 18000K .......... .......... .......... .......... .......... 13% 76.0M 22s 2024-03-19T01:11:42.304 INFO:tasks.workunit.client.0.smithi069.stderr: 18050K .......... .......... .......... .......... .......... 13% 5.77M 22s 2024-03-19T01:11:42.315 INFO:tasks.workunit.client.0.smithi069.stderr: 18100K .......... .......... .......... .......... .......... 13% 4.78M 22s 2024-03-19T01:11:42.316 INFO:tasks.workunit.client.0.smithi069.stderr: 18150K .......... .......... .......... .......... .......... 13% 36.7M 22s 2024-03-19T01:11:42.317 INFO:tasks.workunit.client.0.smithi069.stderr: 18200K .......... .......... .......... .......... .......... 13% 29.7M 22s 2024-03-19T01:11:42.335 INFO:tasks.workunit.client.0.smithi069.stderr: 18250K .......... .......... .......... .......... .......... 13% 2.88M 22s 2024-03-19T01:11:42.335 INFO:tasks.workunit.client.0.smithi069.stderr: 18300K .......... .......... .......... .......... .......... 13% 60.4M 22s 2024-03-19T01:11:42.337 INFO:tasks.workunit.client.0.smithi069.stderr: 18350K .......... .......... .......... .......... .......... 13% 26.7M 22s 2024-03-19T01:11:42.345 INFO:tasks.workunit.client.0.smithi069.stderr: 18400K .......... .......... .......... .......... .......... 13% 6.26M 21s 2024-03-19T01:11:42.355 INFO:tasks.workunit.client.0.smithi069.stderr: 18450K .......... .......... .......... .......... .......... 13% 4.81M 21s 2024-03-19T01:11:42.357 INFO:tasks.workunit.client.0.smithi069.stderr: 18500K .......... .......... .......... .......... .......... 13% 29.6M 21s 2024-03-19T01:11:42.358 INFO:tasks.workunit.client.0.smithi069.stderr: 18550K .......... .......... .......... .......... .......... 13% 36.3M 21s 2024-03-19T01:11:42.377 INFO:tasks.workunit.client.0.smithi069.stderr: 18600K .......... .......... .......... .......... .......... 13% 2.88M 21s 2024-03-19T01:11:42.377 INFO:tasks.workunit.client.0.smithi069.stderr: 18650K .......... .......... .......... .......... .......... 13% 35.3M 21s 2024-03-19T01:11:42.378 INFO:tasks.workunit.client.0.smithi069.stderr: 18700K .......... .......... .......... .......... .......... 13% 39.0M 21s 2024-03-19T01:11:42.386 INFO:tasks.workunit.client.0.smithi069.stderr: 18750K .......... .......... .......... .......... .......... 13% 6.19M 21s 2024-03-19T01:11:42.396 INFO:tasks.workunit.client.0.smithi069.stderr: 18800K .......... .......... .......... .......... .......... 13% 4.83M 21s 2024-03-19T01:11:42.398 INFO:tasks.workunit.client.0.smithi069.stderr: 18850K .......... .......... .......... .......... .......... 13% 29.7M 21s 2024-03-19T01:11:42.398 INFO:tasks.workunit.client.0.smithi069.stderr: 18900K .......... .......... .......... .......... .......... 13% 79.2M 21s 2024-03-19T01:11:42.406 INFO:tasks.workunit.client.0.smithi069.stderr: 18950K .......... .......... .......... .......... .......... 13% 5.81M 21s 2024-03-19T01:11:42.416 INFO:tasks.workunit.client.0.smithi069.stderr: 19000K .......... .......... .......... .......... .......... 14% 4.84M 21s 2024-03-19T01:11:42.418 INFO:tasks.workunit.client.0.smithi069.stderr: 19050K .......... .......... .......... .......... .......... 14% 33.5M 21s 2024-03-19T01:11:42.426 INFO:tasks.workunit.client.0.smithi069.stderr: 19100K .......... .......... .......... .......... .......... 14% 5.90M 21s 2024-03-19T01:11:42.436 INFO:tasks.workunit.client.0.smithi069.stderr: 19150K .......... .......... .......... .......... .......... 14% 4.79M 21s 2024-03-19T01:11:42.438 INFO:tasks.workunit.client.0.smithi069.stderr: 19200K .......... .......... .......... .......... .......... 14% 32.4M 21s 2024-03-19T01:11:42.439 INFO:tasks.workunit.client.0.smithi069.stderr: 19250K .......... .......... .......... .......... .......... 14% 40.0M 21s 2024-03-19T01:11:42.447 INFO:tasks.workunit.client.0.smithi069.stderr: 19300K .......... .......... .......... .......... .......... 14% 6.19M 21s 2024-03-19T01:11:42.457 INFO:tasks.workunit.client.0.smithi069.stderr: 19350K .......... .......... .......... .......... .......... 14% 4.71M 21s 2024-03-19T01:11:42.458 INFO:tasks.workunit.client.0.smithi069.stderr: 19400K .......... .......... .......... .......... .......... 14% 39.1M 21s 2024-03-19T01:11:42.460 INFO:tasks.workunit.client.0.smithi069.stderr: 19450K .......... .......... .......... .......... .......... 14% 35.6M 21s 2024-03-19T01:11:42.467 INFO:tasks.workunit.client.0.smithi069.stderr: 19500K .......... .......... .......... .......... .......... 14% 6.28M 21s 2024-03-19T01:11:42.478 INFO:tasks.workunit.client.0.smithi069.stderr: 19550K .......... .......... .......... .......... .......... 14% 4.73M 21s 2024-03-19T01:11:42.480 INFO:tasks.workunit.client.0.smithi069.stderr: 19600K .......... .......... .......... .......... .......... 14% 38.2M 21s 2024-03-19T01:11:42.487 INFO:tasks.workunit.client.0.smithi069.stderr: 19650K .......... .......... .......... .......... .......... 14% 8.27M 21s 2024-03-19T01:11:42.497 INFO:tasks.workunit.client.0.smithi069.stderr: 19700K .......... .......... .......... .......... .......... 14% 4.00M 21s 2024-03-19T01:11:42.499 INFO:tasks.workunit.client.0.smithi069.stderr: 19750K .......... .......... .......... .......... .......... 14% 38.4M 21s 2024-03-19T01:11:42.500 INFO:tasks.workunit.client.0.smithi069.stderr: 19800K .......... .......... .......... .......... .......... 14% 38.9M 21s 2024-03-19T01:11:42.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:42 smithi086 ceph-mon[66829]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-19T01:11:42.508 INFO:tasks.workunit.client.0.smithi069.stderr: 19850K .......... .......... .......... .......... .......... 14% 6.14M 21s 2024-03-19T01:11:42.519 INFO:tasks.workunit.client.0.smithi069.stderr: 19900K .......... .......... .......... .......... .......... 14% 4.77M 21s 2024-03-19T01:11:42.520 INFO:tasks.workunit.client.0.smithi069.stderr: 19950K .......... .......... .......... .......... .......... 14% 32.7M 21s 2024-03-19T01:11:42.521 INFO:tasks.workunit.client.0.smithi069.stderr: 20000K .......... .......... .......... .......... .......... 14% 39.1M 21s 2024-03-19T01:11:42.528 INFO:tasks.workunit.client.0.smithi069.stderr: 20050K .......... .......... .......... .......... .......... 14% 6.17M 21s 2024-03-19T01:11:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:42 smithi069 ceph-mon[31727]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-19T01:11:42.539 INFO:tasks.workunit.client.0.smithi069.stderr: 20100K .......... .......... .......... .......... .......... 14% 4.84M 20s 2024-03-19T01:11:42.541 INFO:tasks.workunit.client.0.smithi069.stderr: 20150K .......... .......... .......... .......... .......... 14% 38.0M 20s 2024-03-19T01:11:42.541 INFO:tasks.workunit.client.0.smithi069.stderr: 20200K .......... .......... .......... .......... .......... 14% 34.6M 20s 2024-03-19T01:11:42.558 INFO:tasks.workunit.client.0.smithi069.stderr: 20250K .......... .......... .......... .......... .......... 14% 2.90M 20s 2024-03-19T01:11:42.561 INFO:tasks.workunit.client.0.smithi069.stderr: 20300K .......... .......... .......... .......... .......... 14% 39.3M 20s 2024-03-19T01:11:42.561 INFO:tasks.workunit.client.0.smithi069.stderr: 20350K .......... .......... .......... .......... .......... 15% 40.2M 20s 2024-03-19T01:11:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:42 smithi163 ceph-mon[67428]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-19T01:11:42.568 INFO:tasks.workunit.client.0.smithi069.stderr: 20400K .......... .......... .......... .......... .......... 15% 6.59M 20s 2024-03-19T01:11:42.582 INFO:tasks.workunit.client.0.smithi069.stderr: 20450K .......... .......... .......... .......... .......... 15% 4.55M 20s 2024-03-19T01:11:42.582 INFO:tasks.workunit.client.0.smithi069.stderr: 20500K .......... .......... .......... .......... .......... 15% 43.5M 20s 2024-03-19T01:11:42.582 INFO:tasks.workunit.client.0.smithi069.stderr: 20550K .......... .......... .......... .......... .......... 15% 34.4M 20s 2024-03-19T01:11:42.590 INFO:tasks.workunit.client.0.smithi069.stderr: 20600K .......... .......... .......... .......... .......... 15% 6.02M 20s 2024-03-19T01:11:42.602 INFO:tasks.workunit.client.0.smithi069.stderr: 20650K .......... .......... .......... .......... .......... 15% 4.85M 20s 2024-03-19T01:11:42.602 INFO:tasks.workunit.client.0.smithi069.stderr: 20700K .......... .......... .......... .......... .......... 15% 40.8M 20s 2024-03-19T01:11:42.602 INFO:tasks.workunit.client.0.smithi069.stderr: 20750K .......... .......... .......... .......... .......... 15% 41.3M 20s 2024-03-19T01:11:42.619 INFO:tasks.workunit.client.0.smithi069.stderr: 20800K .......... .......... .......... .......... .......... 15% 2.91M 20s 2024-03-19T01:11:42.623 INFO:tasks.workunit.client.0.smithi069.stderr: 20850K .......... .......... .......... .......... .......... 15% 36.2M 20s 2024-03-19T01:11:42.623 INFO:tasks.workunit.client.0.smithi069.stderr: 20900K .......... .......... .......... .......... .......... 15% 39.9M 20s 2024-03-19T01:11:42.627 INFO:tasks.workunit.client.0.smithi069.stderr: 20950K .......... .......... .......... .......... .......... 15% 35.5M 20s 2024-03-19T01:11:42.640 INFO:tasks.workunit.client.0.smithi069.stderr: 21000K .......... .......... .......... .......... .......... 15% 2.93M 20s 2024-03-19T01:11:42.641 INFO:tasks.workunit.client.0.smithi069.stderr: 21050K .......... .......... .......... .......... .......... 15% 35.6M 20s 2024-03-19T01:11:42.643 INFO:tasks.workunit.client.0.smithi069.stderr: 21100K .......... .......... .......... .......... .......... 15% 43.7M 20s 2024-03-19T01:11:42.650 INFO:tasks.workunit.client.0.smithi069.stderr: 21150K .......... .......... .......... .......... .......... 15% 6.41M 20s 2024-03-19T01:11:42.660 INFO:tasks.workunit.client.0.smithi069.stderr: 21200K .......... .......... .......... .......... .......... 15% 4.64M 20s 2024-03-19T01:11:42.663 INFO:tasks.workunit.client.0.smithi069.stderr: 21250K .......... .......... .......... .......... .......... 15% 33.7M 20s 2024-03-19T01:11:42.663 INFO:tasks.workunit.client.0.smithi069.stderr: 21300K .......... .......... .......... .......... .......... 15% 47.1M 20s 2024-03-19T01:11:42.671 INFO:tasks.workunit.client.0.smithi069.stderr: 21350K .......... .......... .......... .......... .......... 15% 5.89M 20s 2024-03-19T01:11:42.681 INFO:tasks.workunit.client.0.smithi069.stderr: 21400K .......... .......... .......... .......... .......... 15% 4.80M 20s 2024-03-19T01:11:42.683 INFO:tasks.workunit.client.0.smithi069.stderr: 21450K .......... .......... .......... .......... .......... 15% 41.0M 20s 2024-03-19T01:11:42.683 INFO:tasks.workunit.client.0.smithi069.stderr: 21500K .......... .......... .......... .......... .......... 15% 57.0M 20s 2024-03-19T01:11:42.701 INFO:tasks.workunit.client.0.smithi069.stderr: 21550K .......... .......... .......... .......... .......... 15% 2.86M 20s 2024-03-19T01:11:42.704 INFO:tasks.workunit.client.0.smithi069.stderr: 21600K .......... .......... .......... .......... .......... 15% 26.3M 20s 2024-03-19T01:11:42.704 INFO:tasks.workunit.client.0.smithi069.stderr: 21650K .......... .......... .......... .......... .......... 15% 43.3M 20s 2024-03-19T01:11:42.709 INFO:tasks.workunit.client.0.smithi069.stderr: 21700K .......... .......... .......... .......... .......... 16% 8.32M 20s 2024-03-19T01:11:42.722 INFO:tasks.workunit.client.0.smithi069.stderr: 21750K .......... .......... .......... .......... .......... 16% 4.09M 20s 2024-03-19T01:11:42.724 INFO:tasks.workunit.client.0.smithi069.stderr: 21800K .......... .......... .......... .......... .......... 16% 29.5M 20s 2024-03-19T01:11:42.725 INFO:tasks.workunit.client.0.smithi069.stderr: 21850K .......... .......... .......... .......... .......... 16% 74.1M 20s 2024-03-19T01:11:42.731 INFO:tasks.workunit.client.0.smithi069.stderr: 21900K .......... .......... .......... .......... .......... 16% 6.25M 20s 2024-03-19T01:11:42.742 INFO:tasks.workunit.client.0.smithi069.stderr: 21950K .......... .......... .......... .......... .......... 16% 4.71M 20s 2024-03-19T01:11:42.744 INFO:tasks.workunit.client.0.smithi069.stderr: 22000K .......... .......... .......... .......... .......... 16% 27.7M 19s 2024-03-19T01:11:42.744 INFO:tasks.workunit.client.0.smithi069.stderr: 22050K .......... .......... .......... .......... .......... 16% 60.5M 19s 2024-03-19T01:11:42.752 INFO:tasks.workunit.client.0.smithi069.stderr: 22100K .......... .......... .......... .......... .......... 16% 5.93M 19s 2024-03-19T01:11:42.763 INFO:tasks.workunit.client.0.smithi069.stderr: 22150K .......... .......... .......... .......... .......... 16% 4.90M 19s 2024-03-19T01:11:42.765 INFO:tasks.workunit.client.0.smithi069.stderr: 22200K .......... .......... .......... .......... .......... 16% 28.9M 19s 2024-03-19T01:11:42.765 INFO:tasks.workunit.client.0.smithi069.stderr: 22250K .......... .......... .......... .......... .......... 16% 43.3M 19s 2024-03-19T01:11:42.774 INFO:tasks.workunit.client.0.smithi069.stderr: 22300K .......... .......... .......... .......... .......... 16% 6.43M 19s 2024-03-19T01:11:42.783 INFO:tasks.workunit.client.0.smithi069.stderr: 22350K .......... .......... .......... .......... .......... 16% 5.04M 19s 2024-03-19T01:11:42.785 INFO:tasks.workunit.client.0.smithi069.stderr: 22400K .......... .......... .......... .......... .......... 16% 28.1M 19s 2024-03-19T01:11:42.785 INFO:tasks.workunit.client.0.smithi069.stderr: 22450K .......... .......... .......... .......... .......... 16% 71.3M 19s 2024-03-19T01:11:42.803 INFO:tasks.workunit.client.0.smithi069.stderr: 22500K .......... .......... .......... .......... .......... 16% 5.84M 19s 2024-03-19T01:11:42.803 INFO:tasks.workunit.client.0.smithi069.stderr: 22550K .......... .......... .......... .......... .......... 16% 4.86M 19s 2024-03-19T01:11:42.805 INFO:tasks.workunit.client.0.smithi069.stderr: 22600K .......... .......... .......... .......... .......... 16% 41.5M 19s 2024-03-19T01:11:42.805 INFO:tasks.workunit.client.0.smithi069.stderr: 22650K .......... .......... .......... .......... .......... 16% 69.7M 19s 2024-03-19T01:11:42.826 INFO:tasks.workunit.client.0.smithi069.stderr: 22700K .......... .......... .......... .......... .......... 16% 2.44M 19s 2024-03-19T01:11:42.827 INFO:tasks.workunit.client.0.smithi069.stderr: 22750K .......... .......... .......... .......... .......... 16% 36.7M 19s 2024-03-19T01:11:42.828 INFO:tasks.workunit.client.0.smithi069.stderr: 22800K .......... .......... .......... .......... .......... 16% 35.7M 19s 2024-03-19T01:11:42.829 INFO:tasks.workunit.client.0.smithi069.stderr: 22850K .......... .......... .......... .......... .......... 16% 42.7M 19s 2024-03-19T01:11:42.847 INFO:tasks.workunit.client.0.smithi069.stderr: 22900K .......... .......... .......... .......... .......... 16% 2.80M 19s 2024-03-19T01:11:42.848 INFO:tasks.workunit.client.0.smithi069.stderr: 22950K .......... .......... .......... .......... .......... 16% 32.2M 19s 2024-03-19T01:11:42.850 INFO:tasks.workunit.client.0.smithi069.stderr: 23000K .......... .......... .......... .......... .......... 16% 42.1M 19s 2024-03-19T01:11:42.850 INFO:tasks.workunit.client.0.smithi069.stderr: 23050K .......... .......... .......... .......... .......... 17% 42.6M 19s 2024-03-19T01:11:42.867 INFO:tasks.workunit.client.0.smithi069.stderr: 23100K .......... .......... .......... .......... .......... 17% 2.95M 19s 2024-03-19T01:11:42.868 INFO:tasks.workunit.client.0.smithi069.stderr: 23150K .......... .......... .......... .......... .......... 17% 37.2M 19s 2024-03-19T01:11:42.870 INFO:tasks.workunit.client.0.smithi069.stderr: 23200K .......... .......... .......... .......... .......... 17% 37.0M 19s 2024-03-19T01:11:42.870 INFO:tasks.workunit.client.0.smithi069.stderr: 23250K .......... .......... .......... .......... .......... 17% 41.6M 19s 2024-03-19T01:11:42.887 INFO:tasks.workunit.client.0.smithi069.stderr: 23300K .......... .......... .......... .......... .......... 17% 2.94M 19s 2024-03-19T01:11:42.889 INFO:tasks.workunit.client.0.smithi069.stderr: 23350K .......... .......... .......... .......... .......... 17% 35.7M 19s 2024-03-19T01:11:42.890 INFO:tasks.workunit.client.0.smithi069.stderr: 23400K .......... .......... .......... .......... .......... 17% 36.7M 19s 2024-03-19T01:11:42.891 INFO:tasks.workunit.client.0.smithi069.stderr: 23450K .......... .......... .......... .......... .......... 17% 41.8M 19s 2024-03-19T01:11:42.908 INFO:tasks.workunit.client.0.smithi069.stderr: 23500K .......... .......... .......... .......... .......... 17% 2.97M 19s 2024-03-19T01:11:42.909 INFO:tasks.workunit.client.0.smithi069.stderr: 23550K .......... .......... .......... .......... .......... 17% 30.6M 19s 2024-03-19T01:11:42.910 INFO:tasks.workunit.client.0.smithi069.stderr: 23600K .......... .......... .......... .......... .......... 17% 42.4M 19s 2024-03-19T01:11:42.911 INFO:tasks.workunit.client.0.smithi069.stderr: 23650K .......... .......... .......... .......... .......... 17% 34.6M 19s 2024-03-19T01:11:42.928 INFO:tasks.workunit.client.0.smithi069.stderr: 23700K .......... .......... .......... .......... .......... 17% 2.99M 19s 2024-03-19T01:11:42.929 INFO:tasks.workunit.client.0.smithi069.stderr: 23750K .......... .......... .......... .......... .......... 17% 31.8M 19s 2024-03-19T01:11:42.931 INFO:tasks.workunit.client.0.smithi069.stderr: 23800K .......... .......... .......... .......... .......... 17% 40.1M 19s 2024-03-19T01:11:42.933 INFO:tasks.workunit.client.0.smithi069.stderr: 23850K .......... .......... .......... .......... .......... 17% 42.5M 19s 2024-03-19T01:11:42.936 INFO:tasks.workunit.client.0.smithi069.stderr: 23900K .......... .......... .......... .......... .......... 17% 12.2M 19s 2024-03-19T01:11:42.950 INFO:tasks.workunit.client.0.smithi069.stderr: 23950K .......... .......... .......... .......... .......... 17% 3.56M 19s 2024-03-19T01:11:42.951 INFO:tasks.workunit.client.0.smithi069.stderr: 24000K .......... .......... .......... .......... .......... 17% 35.2M 19s 2024-03-19T01:11:42.953 INFO:tasks.workunit.client.0.smithi069.stderr: 24050K .......... .......... .......... .......... .......... 17% 18.8M 18s 2024-03-19T01:11:42.956 INFO:tasks.workunit.client.0.smithi069.stderr: 24100K .......... .......... .......... .......... .......... 17% 19.0M 18s 2024-03-19T01:11:42.971 INFO:tasks.workunit.client.0.smithi069.stderr: 24150K .......... .......... .......... .......... .......... 17% 3.55M 18s 2024-03-19T01:11:42.971 INFO:tasks.workunit.client.0.smithi069.stderr: 24200K .......... .......... .......... .......... .......... 17% 39.1M 18s 2024-03-19T01:11:42.974 INFO:tasks.workunit.client.0.smithi069.stderr: 24250K .......... .......... .......... .......... .......... 17% 18.1M 18s 2024-03-19T01:11:42.977 INFO:tasks.workunit.client.0.smithi069.stderr: 24300K .......... .......... .......... .......... .......... 17% 17.5M 18s 2024-03-19T01:11:42.991 INFO:tasks.workunit.client.0.smithi069.stderr: 24350K .......... .......... .......... .......... .......... 17% 3.57M 18s 2024-03-19T01:11:42.993 INFO:tasks.workunit.client.0.smithi069.stderr: 24400K .......... .......... .......... .......... .......... 18% 37.8M 18s 2024-03-19T01:11:42.993 INFO:tasks.workunit.client.0.smithi069.stderr: 24450K .......... .......... .......... .......... .......... 18% 42.2M 18s 2024-03-19T01:11:42.997 INFO:tasks.workunit.client.0.smithi069.stderr: 24500K .......... .......... .......... .......... .......... 18% 11.1M 18s 2024-03-19T01:11:43.010 INFO:tasks.workunit.client.0.smithi069.stderr: 24550K .......... .......... .......... .......... .......... 18% 3.67M 18s 2024-03-19T01:11:43.012 INFO:tasks.workunit.client.0.smithi069.stderr: 24600K .......... .......... .......... .......... .......... 18% 37.9M 18s 2024-03-19T01:11:43.015 INFO:tasks.workunit.client.0.smithi069.stderr: 24650K .......... .......... .......... .......... .......... 18% 37.0M 18s 2024-03-19T01:11:43.018 INFO:tasks.workunit.client.0.smithi069.stderr: 24700K .......... .......... .......... .......... .......... 18% 11.2M 18s 2024-03-19T01:11:43.030 INFO:tasks.workunit.client.0.smithi069.stderr: 24750K .......... .......... .......... .......... .......... 18% 3.70M 18s 2024-03-19T01:11:43.032 INFO:tasks.workunit.client.0.smithi069.stderr: 24800K .......... .......... .......... .......... .......... 18% 31.6M 18s 2024-03-19T01:11:43.036 INFO:tasks.workunit.client.0.smithi069.stderr: 24850K .......... .......... .......... .......... .......... 18% 15.4M 18s 2024-03-19T01:11:43.037 INFO:tasks.workunit.client.0.smithi069.stderr: 24900K .......... .......... .......... .......... .......... 18% 19.4M 18s 2024-03-19T01:11:43.051 INFO:tasks.workunit.client.0.smithi069.stderr: 24950K .......... .......... .......... .......... .......... 18% 3.75M 18s 2024-03-19T01:11:43.053 INFO:tasks.workunit.client.0.smithi069.stderr: 25000K .......... .......... .......... .......... .......... 18% 26.4M 18s 2024-03-19T01:11:43.056 INFO:tasks.workunit.client.0.smithi069.stderr: 25050K .......... .......... .......... .......... .......... 18% 16.7M 18s 2024-03-19T01:11:43.058 INFO:tasks.workunit.client.0.smithi069.stderr: 25100K .......... .......... .......... .......... .......... 18% 19.0M 18s 2024-03-19T01:11:43.072 INFO:tasks.workunit.client.0.smithi069.stderr: 25150K .......... .......... .......... .......... .......... 18% 3.79M 18s 2024-03-19T01:11:43.074 INFO:tasks.workunit.client.0.smithi069.stderr: 25200K .......... .......... .......... .......... .......... 18% 30.6M 18s 2024-03-19T01:11:43.074 INFO:tasks.workunit.client.0.smithi069.stderr: 25250K .......... .......... .......... .......... .......... 18% 36.6M 18s 2024-03-19T01:11:43.078 INFO:tasks.workunit.client.0.smithi069.stderr: 25300K .......... .......... .......... .......... .......... 18% 11.3M 18s 2024-03-19T01:11:43.091 INFO:tasks.workunit.client.0.smithi069.stderr: 25350K .......... .......... .......... .......... .......... 18% 3.87M 18s 2024-03-19T01:11:43.093 INFO:tasks.workunit.client.0.smithi069.stderr: 25400K .......... .......... .......... .......... .......... 18% 25.6M 18s 2024-03-19T01:11:43.094 INFO:tasks.workunit.client.0.smithi069.stderr: 25450K .......... .......... .......... .......... .......... 18% 36.0M 18s 2024-03-19T01:11:43.098 INFO:tasks.workunit.client.0.smithi069.stderr: 25500K .......... .......... .......... .......... .......... 18% 12.1M 18s 2024-03-19T01:11:43.099 INFO:tasks.workunit.client.0.smithi069.stderr: 25550K .......... .......... .......... .......... .......... 18% 30.3M 18s 2024-03-19T01:11:43.113 INFO:tasks.workunit.client.0.smithi069.stderr: 25600K .......... .......... .......... .......... .......... 18% 3.74M 18s 2024-03-19T01:11:43.114 INFO:tasks.workunit.client.0.smithi069.stderr: 25650K .......... .......... .......... .......... .......... 18% 36.8M 18s 2024-03-19T01:11:43.119 INFO:tasks.workunit.client.0.smithi069.stderr: 25700K .......... .......... .......... .......... .......... 18% 11.7M 18s 2024-03-19T01:11:43.120 INFO:tasks.workunit.client.0.smithi069.stderr: 25750K .......... .......... .......... .......... .......... 19% 37.5M 18s 2024-03-19T01:11:43.134 INFO:tasks.workunit.client.0.smithi069.stderr: 25800K .......... .......... .......... .......... .......... 19% 3.60M 18s 2024-03-19T01:11:43.134 INFO:tasks.workunit.client.0.smithi069.stderr: 25850K .......... .......... .......... .......... .......... 19% 49.7M 18s 2024-03-19T01:11:43.138 INFO:tasks.workunit.client.0.smithi069.stderr: 25900K .......... .......... .......... .......... .......... 19% 11.8M 18s 2024-03-19T01:11:43.140 INFO:tasks.workunit.client.0.smithi069.stderr: 25950K .......... .......... .......... .......... .......... 19% 31.6M 18s 2024-03-19T01:11:43.154 INFO:tasks.workunit.client.0.smithi069.stderr: 26000K .......... .......... .......... .......... .......... 19% 3.79M 18s 2024-03-19T01:11:43.155 INFO:tasks.workunit.client.0.smithi069.stderr: 26050K .......... .......... .......... .......... .......... 19% 32.4M 18s 2024-03-19T01:11:43.157 INFO:tasks.workunit.client.0.smithi069.stderr: 26100K .......... .......... .......... .......... .......... 19% 15.4M 18s 2024-03-19T01:11:43.160 INFO:tasks.workunit.client.0.smithi069.stderr: 26150K .......... .......... .......... .......... .......... 19% 19.2M 18s 2024-03-19T01:11:43.173 INFO:tasks.workunit.client.0.smithi069.stderr: 26200K .......... .......... .......... .......... .......... 19% 3.76M 18s 2024-03-19T01:11:43.176 INFO:tasks.workunit.client.0.smithi069.stderr: 26250K .......... .......... .......... .......... .......... 19% 24.6M 18s 2024-03-19T01:11:43.178 INFO:tasks.workunit.client.0.smithi069.stderr: 26300K .......... .......... .......... .......... .......... 19% 19.4M 17s 2024-03-19T01:11:43.182 INFO:tasks.workunit.client.0.smithi069.stderr: 26350K .......... .......... .......... .......... .......... 19% 17.7M 17s 2024-03-19T01:11:43.193 INFO:tasks.workunit.client.0.smithi069.stderr: 26400K .......... .......... .......... .......... .......... 19% 3.93M 17s 2024-03-19T01:11:43.196 INFO:tasks.workunit.client.0.smithi069.stderr: 26450K .......... .......... .......... .......... .......... 19% 20.3M 17s 2024-03-19T01:11:43.198 INFO:tasks.workunit.client.0.smithi069.stderr: 26500K .......... .......... .......... .......... .......... 19% 18.1M 17s 2024-03-19T01:11:43.201 INFO:tasks.workunit.client.0.smithi069.stderr: 26550K .......... .......... .......... .......... .......... 19% 17.3M 17s 2024-03-19T01:11:43.202 INFO:tasks.workunit.client.0.smithi069.stderr: 26600K .......... .......... .......... .......... .......... 19% 43.2M 17s 2024-03-19T01:11:43.217 INFO:tasks.workunit.client.0.smithi069.stderr: 26650K .......... .......... .......... .......... .......... 19% 3.60M 17s 2024-03-19T01:11:43.217 INFO:tasks.workunit.client.0.smithi069.stderr: 26700K .......... .......... .......... .......... .......... 19% 55.4M 17s 2024-03-19T01:11:43.220 INFO:tasks.workunit.client.0.smithi069.stderr: 26750K .......... .......... .......... .......... .......... 19% 12.1M 17s 2024-03-19T01:11:43.222 INFO:tasks.workunit.client.0.smithi069.stderr: 26800K .......... .......... .......... .......... .......... 19% 35.8M 17s 2024-03-19T01:11:43.236 INFO:tasks.workunit.client.0.smithi069.stderr: 26850K .......... .......... .......... .......... .......... 19% 3.55M 17s 2024-03-19T01:11:43.237 INFO:tasks.workunit.client.0.smithi069.stderr: 26900K .......... .......... .......... .......... .......... 19% 63.9M 17s 2024-03-19T01:11:43.240 INFO:tasks.workunit.client.0.smithi069.stderr: 26950K .......... .......... .......... .......... .......... 19% 12.1M 17s 2024-03-19T01:11:43.243 INFO:tasks.workunit.client.0.smithi069.stderr: 27000K .......... .......... .......... .......... .......... 19% 28.3M 17s 2024-03-19T01:11:43.256 INFO:tasks.workunit.client.0.smithi069.stderr: 27050K .......... .......... .......... .......... .......... 19% 3.62M 17s 2024-03-19T01:11:43.256 INFO:tasks.workunit.client.0.smithi069.stderr: 27100K .......... .......... .......... .......... .......... 19% 50.2M 17s 2024-03-19T01:11:43.259 INFO:tasks.workunit.client.0.smithi069.stderr: 27150K .......... .......... .......... .......... .......... 20% 18.4M 17s 2024-03-19T01:11:43.264 INFO:tasks.workunit.client.0.smithi069.stderr: 27200K .......... .......... .......... .......... .......... 20% 11.0M 17s 2024-03-19T01:11:43.265 INFO:tasks.workunit.client.0.smithi069.stderr: 27250K .......... .......... .......... .......... .......... 20% 31.0M 17s 2024-03-19T01:11:43.277 INFO:tasks.workunit.client.0.smithi069.stderr: 27300K .......... .......... .......... .......... .......... 20% 4.27M 17s 2024-03-19T01:11:43.279 INFO:tasks.workunit.client.0.smithi069.stderr: 27350K .......... .......... .......... .......... .......... 20% 18.6M 17s 2024-03-19T01:11:43.282 INFO:tasks.workunit.client.0.smithi069.stderr: 27400K .......... .......... .......... .......... .......... 20% 16.7M 17s 2024-03-19T01:11:43.286 INFO:tasks.workunit.client.0.smithi069.stderr: 27450K .......... .......... .......... .......... .......... 20% 14.9M 17s 2024-03-19T01:11:43.297 INFO:tasks.workunit.client.0.smithi069.stderr: 27500K .......... .......... .......... .......... .......... 20% 4.12M 17s 2024-03-19T01:11:43.298 INFO:tasks.workunit.client.0.smithi069.stderr: 27550K .......... .......... .......... .......... .......... 20% 47.2M 17s 2024-03-19T01:11:43.301 INFO:tasks.workunit.client.0.smithi069.stderr: 27600K .......... .......... .......... .......... .......... 20% 14.6M 17s 2024-03-19T01:11:43.306 INFO:tasks.workunit.client.0.smithi069.stderr: 27650K .......... .......... .......... .......... .......... 20% 11.5M 17s 2024-03-19T01:11:43.317 INFO:tasks.workunit.client.0.smithi069.stderr: 27700K .......... .......... .......... .......... .......... 20% 4.37M 17s 2024-03-19T01:11:43.318 INFO:tasks.workunit.client.0.smithi069.stderr: 27750K .......... .......... .......... .......... .......... 20% 37.8M 17s 2024-03-19T01:11:43.322 INFO:tasks.workunit.client.0.smithi069.stderr: 27800K .......... .......... .......... .......... .......... 20% 15.5M 17s 2024-03-19T01:11:43.325 INFO:tasks.workunit.client.0.smithi069.stderr: 27850K .......... .......... .......... .......... .......... 20% 13.7M 17s 2024-03-19T01:11:43.338 INFO:tasks.workunit.client.0.smithi069.stderr: 27900K .......... .......... .......... .......... .......... 20% 4.14M 17s 2024-03-19T01:11:43.339 INFO:tasks.workunit.client.0.smithi069.stderr: 27950K .......... .......... .......... .......... .......... 20% 39.6M 17s 2024-03-19T01:11:43.340 INFO:tasks.workunit.client.0.smithi069.stderr: 28000K .......... .......... .......... .......... .......... 20% 19.7M 17s 2024-03-19T01:11:43.344 INFO:tasks.workunit.client.0.smithi069.stderr: 28050K .......... .......... .......... .......... .......... 20% 11.9M 17s 2024-03-19T01:11:43.348 INFO:tasks.workunit.client.0.smithi069.stderr: 28100K .......... .......... .......... .......... .......... 20% 14.5M 17s 2024-03-19T01:11:43.359 INFO:tasks.workunit.client.0.smithi069.stderr: 28150K .......... .......... .......... .......... .......... 20% 4.74M 17s 2024-03-19T01:11:43.360 INFO:tasks.workunit.client.0.smithi069.stderr: 28200K .......... .......... .......... .......... .......... 20% 22.7M 17s 2024-03-19T01:11:43.363 INFO:tasks.workunit.client.0.smithi069.stderr: 28250K .......... .......... .......... .......... .......... 20% 16.5M 17s 2024-03-19T01:11:43.368 INFO:tasks.workunit.client.0.smithi069.stderr: 28300K .......... .......... .......... .......... .......... 20% 10.4M 17s 2024-03-19T01:11:43.379 INFO:tasks.workunit.client.0.smithi069.stderr: 28350K .......... .......... .......... .......... .......... 20% 4.81M 17s 2024-03-19T01:11:43.379 INFO:tasks.workunit.client.0.smithi069.stderr: 28400K .......... .......... .......... .......... .......... 20% 62.9M 17s 2024-03-19T01:11:43.382 INFO:tasks.workunit.client.0.smithi069.stderr: 28450K .......... .......... .......... .......... .......... 20% 13.6M 17s 2024-03-19T01:11:43.389 INFO:tasks.workunit.client.0.smithi069.stderr: 28500K .......... .......... .......... .......... .......... 21% 8.54M 17s 2024-03-19T01:11:43.398 INFO:tasks.workunit.client.0.smithi069.stderr: 28550K .......... .......... .......... .......... .......... 21% 5.03M 17s 2024-03-19T01:11:43.399 INFO:tasks.workunit.client.0.smithi069.stderr: 28600K .......... .......... .......... .......... .......... 21% 36.9M 17s 2024-03-19T01:11:43.403 INFO:tasks.workunit.client.0.smithi069.stderr: 28650K .......... .......... .......... .......... .......... 21% 14.2M 17s 2024-03-19T01:11:43.407 INFO:tasks.workunit.client.0.smithi069.stderr: 28700K .......... .......... .......... .......... .......... 21% 11.1M 17s 2024-03-19T01:11:43.420 INFO:tasks.workunit.client.0.smithi069.stderr: 28750K .......... .......... .......... .......... .......... 21% 4.45M 17s 2024-03-19T01:11:43.420 INFO:tasks.workunit.client.0.smithi069.stderr: 28800K .......... .......... .......... .......... .......... 21% 40.8M 17s 2024-03-19T01:11:43.421 INFO:tasks.workunit.client.0.smithi069.stderr: 28850K .......... .......... .......... .......... .......... 21% 22.2M 16s 2024-03-19T01:11:43.426 INFO:tasks.workunit.client.0.smithi069.stderr: 28900K .......... .......... .......... .......... .......... 21% 9.84M 16s 2024-03-19T01:11:43.430 INFO:tasks.workunit.client.0.smithi069.stderr: 28950K .......... .......... .......... .......... .......... 21% 11.5M 16s 2024-03-19T01:11:43.440 INFO:tasks.workunit.client.0.smithi069.stderr: 29000K .......... .......... .......... .......... .......... 21% 5.61M 16s 2024-03-19T01:11:43.441 INFO:tasks.workunit.client.0.smithi069.stderr: 29050K .......... .......... .......... .......... .......... 21% 43.6M 16s 2024-03-19T01:11:43.445 INFO:tasks.workunit.client.0.smithi069.stderr: 29100K .......... .......... .......... .......... .......... 21% 11.0M 16s 2024-03-19T01:11:43.451 INFO:tasks.workunit.client.0.smithi069.stderr: 29150K .......... .......... .......... .......... .......... 21% 8.62M 16s 2024-03-19T01:11:43.460 INFO:tasks.workunit.client.0.smithi069.stderr: 29200K .......... .......... .......... .......... .......... 21% 5.58M 16s 2024-03-19T01:11:43.461 INFO:tasks.workunit.client.0.smithi069.stderr: 29250K .......... .......... .......... .......... .......... 21% 56.5M 16s 2024-03-19T01:11:43.464 INFO:tasks.workunit.client.0.smithi069.stderr: 29300K .......... .......... .......... .......... .......... 21% 13.8M 16s 2024-03-19T01:11:43.472 INFO:tasks.workunit.client.0.smithi069.stderr: 29350K .......... .......... .......... .......... .......... 21% 5.82M 16s 2024-03-19T01:11:43.480 INFO:tasks.workunit.client.0.smithi069.stderr: 29400K .......... .......... .......... .......... .......... 21% 6.62M 16s 2024-03-19T01:11:43.482 INFO:tasks.workunit.client.0.smithi069.stderr: 29450K .......... .......... .......... .......... .......... 21% 21.5M 16s 2024-03-19T01:11:43.483 INFO:tasks.workunit.client.0.smithi069.stderr: 29500K .......... .......... .......... .......... .......... 21% 30.9M 16s 2024-03-19T01:11:43.489 INFO:tasks.workunit.client.0.smithi069.stderr: 29550K .......... .......... .......... .......... .......... 21% 7.86M 16s 2024-03-19T01:11:43.494 INFO:tasks.workunit.client.0.smithi069.stderr: 29600K .......... .......... .......... .......... .......... 21% 11.3M 16s 2024-03-19T01:11:43.502 INFO:tasks.workunit.client.0.smithi069.stderr: 29650K .......... .......... .......... .......... .......... 21% 7.22M 16s 2024-03-19T01:11:43.503 INFO:tasks.workunit.client.0.smithi069.stderr: 29700K .......... .......... .......... .......... .......... 21% 19.7M 16s 2024-03-19T01:11:43.508 INFO:tasks.workunit.client.0.smithi069.stderr: 29750K .......... .......... .......... .......... .......... 21% 8.91M 16s 2024-03-19T01:11:43.513 INFO:tasks.workunit.client.0.smithi069.stderr: 29800K .......... .......... .......... .......... .......... 21% 9.94M 16s 2024-03-19T01:11:43.520 INFO:tasks.workunit.client.0.smithi069.stderr: 29850K .......... .......... .......... .......... .......... 22% 7.04M 16s 2024-03-19T01:11:43.523 INFO:tasks.workunit.client.0.smithi069.stderr: 29900K .......... .......... .......... .......... .......... 22% 20.7M 16s 2024-03-19T01:11:43.525 INFO:tasks.workunit.client.0.smithi069.stderr: 29950K .......... .......... .......... .......... .......... 22% 20.4M 16s 2024-03-19T01:11:43.537 INFO:tasks.workunit.client.0.smithi069.stderr: 30000K .......... .......... .......... .......... .......... 22% 4.21M 16s 2024-03-19T01:11:43.544 INFO:tasks.workunit.client.0.smithi069.stderr: 30050K .......... .......... .......... .......... .......... 22% 6.72M 16s 2024-03-19T01:11:43.547 INFO:tasks.workunit.client.0.smithi069.stderr: 30100K .......... .......... .......... .......... .......... 22% 17.2M 16s 2024-03-19T01:11:43.549 INFO:tasks.workunit.client.0.smithi069.stderr: 30150K .......... .......... .......... .......... .......... 22% 23.4M 16s 2024-03-19T01:11:43.555 INFO:tasks.workunit.client.0.smithi069.stderr: 30200K .......... .......... .......... .......... .......... 22% 8.24M 16s 2024-03-19T01:11:43.558 INFO:tasks.workunit.client.0.smithi069.stderr: 30250K .......... .......... .......... .......... .......... 22% 13.5M 16s 2024-03-19T01:11:43.565 INFO:tasks.workunit.client.0.smithi069.stderr: 30300K .......... .......... .......... .......... .......... 22% 6.96M 16s 2024-03-19T01:11:43.569 INFO:tasks.workunit.client.0.smithi069.stderr: 30350K .......... .......... .......... .......... .......... 22% 15.3M 16s 2024-03-19T01:11:43.576 INFO:tasks.workunit.client.0.smithi069.stderr: 30400K .......... .......... .......... .......... .......... 22% 6.93M 16s 2024-03-19T01:11:43.580 INFO:tasks.workunit.client.0.smithi069.stderr: 30450K .......... .......... .......... .......... .......... 22% 11.0M 16s 2024-03-19T01:11:43.585 INFO:tasks.workunit.client.0.smithi069.stderr: 30500K .......... .......... .......... .......... .......... 22% 9.52M 16s 2024-03-19T01:11:43.590 INFO:tasks.workunit.client.0.smithi069.stderr: 30550K .......... .......... .......... .......... .......... 22% 14.8M 16s 2024-03-19T01:11:43.600 INFO:tasks.workunit.client.0.smithi069.stderr: 30600K .......... .......... .......... .......... .......... 22% 12.4M 16s 2024-03-19T01:11:43.605 INFO:tasks.workunit.client.0.smithi069.stderr: 30650K .......... .......... .......... .......... .......... 22% 3.94M 16s 2024-03-19T01:11:43.610 INFO:tasks.workunit.client.0.smithi069.stderr: 30700K .......... .......... .......... .......... .......... 22% 10.2M 16s 2024-03-19T01:11:43.617 INFO:tasks.workunit.client.0.smithi069.stderr: 30750K .......... .......... .......... .......... .......... 22% 16.8M 16s 2024-03-19T01:11:43.618 INFO:tasks.workunit.client.0.smithi069.stderr: 30800K .......... .......... .......... .......... .......... 22% 10.6M 16s 2024-03-19T01:11:43.627 INFO:tasks.workunit.client.0.smithi069.stderr: 30850K .......... .......... .......... .......... .......... 22% 5.23M 16s 2024-03-19T01:11:43.630 INFO:tasks.workunit.client.0.smithi069.stderr: 30900K .......... .......... .......... .......... .......... 22% 21.3M 16s 2024-03-19T01:11:43.633 INFO:tasks.workunit.client.0.smithi069.stderr: 30950K .......... .......... .......... .......... .......... 22% 11.1M 16s 2024-03-19T01:11:43.640 INFO:tasks.workunit.client.0.smithi069.stderr: 31000K .......... .......... .......... .......... .......... 22% 7.18M 16s 2024-03-19T01:11:43.649 INFO:tasks.workunit.client.0.smithi069.stderr: 31050K .......... .......... .......... .......... .......... 22% 8.69M 16s 2024-03-19T01:11:43.651 INFO:tasks.workunit.client.0.smithi069.stderr: 31100K .......... .......... .......... .......... .......... 22% 8.95M 16s 2024-03-19T01:11:43.655 INFO:tasks.workunit.client.0.smithi069.stderr: 31150K .......... .......... .......... .......... .......... 22% 13.7M 16s 2024-03-19T01:11:43.661 INFO:tasks.workunit.client.0.smithi069.stderr: 31200K .......... .......... .......... .......... .......... 23% 7.89M 16s 2024-03-19T01:11:43.671 INFO:tasks.workunit.client.0.smithi069.stderr: 31250K .......... .......... .......... .......... .......... 23% 4.96M 16s 2024-03-19T01:11:43.678 INFO:tasks.workunit.client.0.smithi069.stderr: 31300K .......... .......... .......... .......... .......... 23% 6.92M 16s 2024-03-19T01:11:43.681 INFO:tasks.workunit.client.0.smithi069.stderr: 31350K .......... .......... .......... .......... .......... 23% 19.0M 16s 2024-03-19T01:11:43.687 INFO:tasks.workunit.client.0.smithi069.stderr: 31400K .......... .......... .......... .......... .......... 23% 6.71M 16s 2024-03-19T01:11:43.694 INFO:tasks.workunit.client.0.smithi069.stderr: 31450K .......... .......... .......... .......... .......... 23% 8.84M 16s 2024-03-19T01:11:43.701 INFO:tasks.workunit.client.0.smithi069.stderr: 31500K .......... .......... .......... .......... .......... 23% 6.45M 16s 2024-03-19T01:11:43.705 INFO:tasks.workunit.client.0.smithi069.stderr: 31550K .......... .......... .......... .......... .......... 23% 11.2M 16s 2024-03-19T01:11:43.710 INFO:tasks.workunit.client.0.smithi069.stderr: 31600K .......... .......... .......... .......... .......... 23% 9.37M 16s 2024-03-19T01:11:43.717 INFO:tasks.workunit.client.0.smithi069.stderr: 31650K .......... .......... .......... .......... .......... 23% 8.54M 16s 2024-03-19T01:11:43.724 INFO:tasks.workunit.client.0.smithi069.stderr: 31700K .......... .......... .......... .......... .......... 23% 6.06M 16s 2024-03-19T01:11:43.728 INFO:tasks.workunit.client.0.smithi069.stderr: 31750K .......... .......... .......... .......... .......... 23% 11.1M 16s 2024-03-19T01:11:43.733 INFO:tasks.workunit.client.0.smithi069.stderr: 31800K .......... .......... .......... .......... .......... 23% 10.1M 16s 2024-03-19T01:11:43.742 INFO:tasks.workunit.client.0.smithi069.stderr: 31850K .......... .......... .......... .......... .......... 23% 6.11M 16s 2024-03-19T01:11:43.749 INFO:tasks.workunit.client.0.smithi069.stderr: 31900K .......... .......... .......... .......... .......... 23% 6.88M 16s 2024-03-19T01:11:43.754 INFO:tasks.workunit.client.0.smithi069.stderr: 31950K .......... .......... .......... .......... .......... 23% 8.07M 16s 2024-03-19T01:11:43.761 INFO:tasks.workunit.client.0.smithi069.stderr: 32000K .......... .......... .......... .......... .......... 23% 7.60M 16s 2024-03-19T01:11:43.764 INFO:tasks.workunit.client.0.smithi069.stderr: 32050K .......... .......... .......... .......... .......... 23% 14.3M 15s 2024-03-19T01:11:43.773 INFO:tasks.workunit.client.0.smithi069.stderr: 32100K .......... .......... .......... .......... .......... 23% 5.77M 15s 2024-03-19T01:11:43.781 INFO:tasks.workunit.client.0.smithi069.stderr: 32150K .......... .......... .......... .......... .......... 23% 6.19M 15s 2024-03-19T01:11:43.785 INFO:tasks.workunit.client.0.smithi069.stderr: 32200K .......... .......... .......... .......... .......... 23% 10.6M 15s 2024-03-19T01:11:43.794 INFO:tasks.workunit.client.0.smithi069.stderr: 32250K .......... .......... .......... .......... .......... 23% 8.65M 15s 2024-03-19T01:11:43.797 INFO:tasks.workunit.client.0.smithi069.stderr: 32300K .......... .......... .......... .......... .......... 23% 7.47M 15s 2024-03-19T01:11:43.804 INFO:tasks.workunit.client.0.smithi069.stderr: 32350K .......... .......... .......... .......... .......... 23% 7.45M 15s 2024-03-19T01:11:43.811 INFO:tasks.workunit.client.0.smithi069.stderr: 32400K .......... .......... .......... .......... .......... 23% 8.37M 15s 2024-03-19T01:11:43.817 INFO:tasks.workunit.client.0.smithi069.stderr: 32450K .......... .......... .......... .......... .......... 23% 6.68M 15s 2024-03-19T01:11:43.824 INFO:tasks.workunit.client.0.smithi069.stderr: 32500K .......... .......... .......... .......... .......... 23% 6.98M 15s 2024-03-19T01:11:43.829 INFO:tasks.workunit.client.0.smithi069.stderr: 32550K .......... .......... .......... .......... .......... 24% 10.2M 15s 2024-03-19T01:11:43.836 INFO:tasks.workunit.client.0.smithi069.stderr: 32600K .......... .......... .......... .......... .......... 24% 6.80M 15s 2024-03-19T01:11:43.843 INFO:tasks.workunit.client.0.smithi069.stderr: 32650K .......... .......... .......... .......... .......... 24% 6.87M 15s 2024-03-19T01:11:43.850 INFO:tasks.workunit.client.0.smithi069.stderr: 32700K .......... .......... .......... .......... .......... 24% 8.13M 15s 2024-03-19T01:11:43.858 INFO:tasks.workunit.client.0.smithi069.stderr: 32750K .......... .......... .......... .......... .......... 24% 6.92M 15s 2024-03-19T01:11:43.862 INFO:tasks.workunit.client.0.smithi069.stderr: 32800K .......... .......... .......... .......... .......... 24% 7.86M 15s 2024-03-19T01:11:43.869 INFO:tasks.workunit.client.0.smithi069.stderr: 32850K .......... .......... .......... .......... .......... 24% 7.42M 15s 2024-03-19T01:11:43.878 INFO:tasks.workunit.client.0.smithi069.stderr: 32900K .......... .......... .......... .......... .......... 24% 5.78M 15s 2024-03-19T01:11:43.885 INFO:tasks.workunit.client.0.smithi069.stderr: 32950K .......... .......... .......... .......... .......... 24% 6.79M 15s 2024-03-19T01:11:43.890 INFO:tasks.workunit.client.0.smithi069.stderr: 33000K .......... .......... .......... .......... .......... 24% 9.76M 15s 2024-03-19T01:11:43.896 INFO:tasks.workunit.client.0.smithi069.stderr: 33050K .......... .......... .......... .......... .......... 24% 7.92M 15s 2024-03-19T01:11:43.902 INFO:tasks.workunit.client.0.smithi069.stderr: 33100K .......... .......... .......... .......... .......... 24% 8.54M 15s 2024-03-19T01:11:43.911 INFO:tasks.workunit.client.0.smithi069.stderr: 33150K .......... .......... .......... .......... .......... 24% 5.23M 15s 2024-03-19T01:11:43.916 INFO:tasks.workunit.client.0.smithi069.stderr: 33200K .......... .......... .......... .......... .......... 24% 9.53M 15s 2024-03-19T01:11:43.924 INFO:tasks.workunit.client.0.smithi069.stderr: 33250K .......... .......... .......... .......... .......... 24% 6.17M 15s 2024-03-19T01:11:43.929 INFO:tasks.workunit.client.0.smithi069.stderr: 33300K .......... .......... .......... .......... .......... 24% 8.96M 15s 2024-03-19T01:11:43.937 INFO:tasks.workunit.client.0.smithi069.stderr: 33350K .......... .......... .......... .......... .......... 24% 6.29M 15s 2024-03-19T01:11:43.943 INFO:tasks.workunit.client.0.smithi069.stderr: 33400K .......... .......... .......... .......... .......... 24% 7.55M 15s 2024-03-19T01:11:43.950 INFO:tasks.workunit.client.0.smithi069.stderr: 33450K .......... .......... .......... .......... .......... 24% 7.72M 15s 2024-03-19T01:11:43.958 INFO:tasks.workunit.client.0.smithi069.stderr: 33500K .......... .......... .......... .......... .......... 24% 6.22M 15s 2024-03-19T01:11:43.964 INFO:tasks.workunit.client.0.smithi069.stderr: 33550K .......... .......... .......... .......... .......... 24% 7.73M 15s 2024-03-19T01:11:43.971 INFO:tasks.workunit.client.0.smithi069.stderr: 33600K .......... .......... .......... .......... .......... 24% 7.12M 15s 2024-03-19T01:11:43.981 INFO:tasks.workunit.client.0.smithi069.stderr: 33650K .......... .......... .......... .......... .......... 24% 5.09M 15s 2024-03-19T01:11:43.988 INFO:tasks.workunit.client.0.smithi069.stderr: 33700K .......... .......... .......... .......... .......... 24% 10.4M 15s 2024-03-19T01:11:43.994 INFO:tasks.workunit.client.0.smithi069.stderr: 33750K .......... .......... .......... .......... .......... 24% 5.24M 15s 2024-03-19T01:11:44.002 INFO:tasks.workunit.client.0.smithi069.stderr: 33800K .......... .......... .......... .......... .......... 24% 9.05M 15s 2024-03-19T01:11:44.011 INFO:tasks.workunit.client.0.smithi069.stderr: 33850K .......... .......... .......... .......... .......... 24% 4.33M 15s 2024-03-19T01:11:44.019 INFO:tasks.workunit.client.0.smithi069.stderr: 33900K .......... .......... .......... .......... .......... 25% 7.04M 15s 2024-03-19T01:11:44.028 INFO:tasks.workunit.client.0.smithi069.stderr: 33950K .......... .......... .......... .......... .......... 25% 5.40M 15s 2024-03-19T01:11:44.033 INFO:tasks.workunit.client.0.smithi069.stderr: 34000K .......... .......... .......... .......... .......... 25% 8.70M 15s 2024-03-19T01:11:44.042 INFO:tasks.workunit.client.0.smithi069.stderr: 34050K .......... .......... .......... .......... .......... 25% 5.78M 15s 2024-03-19T01:11:44.050 INFO:tasks.workunit.client.0.smithi069.stderr: 34100K .......... .......... .......... .......... .......... 25% 5.52M 15s 2024-03-19T01:11:44.056 INFO:tasks.workunit.client.0.smithi069.stderr: 34150K .......... .......... .......... .......... .......... 25% 8.69M 15s 2024-03-19T01:11:44.064 INFO:tasks.workunit.client.0.smithi069.stderr: 34200K .......... .......... .......... .......... .......... 25% 6.17M 15s 2024-03-19T01:11:44.070 INFO:tasks.workunit.client.0.smithi069.stderr: 34250K .......... .......... .......... .......... .......... 25% 7.78M 15s 2024-03-19T01:11:44.080 INFO:tasks.workunit.client.0.smithi069.stderr: 34300K .......... .......... .......... .......... .......... 25% 5.30M 15s 2024-03-19T01:11:44.097 INFO:tasks.workunit.client.0.smithi069.stderr: 34350K .......... .......... .......... .......... .......... 25% 2.83M 15s 2024-03-19T01:11:44.103 INFO:tasks.workunit.client.0.smithi069.stderr: 34400K .......... .......... .......... .......... .......... 25% 7.12M 15s 2024-03-19T01:11:44.111 INFO:tasks.workunit.client.0.smithi069.stderr: 34450K .......... .......... .......... .......... .......... 25% 6.60M 15s 2024-03-19T01:11:44.118 INFO:tasks.workunit.client.0.smithi069.stderr: 34500K .......... .......... .......... .......... .......... 25% 6.53M 15s 2024-03-19T01:11:44.134 INFO:tasks.workunit.client.0.smithi069.stderr: 34550K .......... .......... .......... .......... .......... 25% 3.04M 15s 2024-03-19T01:11:44.143 INFO:tasks.workunit.client.0.smithi069.stderr: 34600K .......... .......... .......... .......... .......... 25% 5.64M 15s 2024-03-19T01:11:44.153 INFO:tasks.workunit.client.0.smithi069.stderr: 34650K .......... .......... .......... .......... .......... 25% 4.96M 15s 2024-03-19T01:11:44.159 INFO:tasks.workunit.client.0.smithi069.stderr: 34700K .......... .......... .......... .......... .......... 25% 7.69M 15s 2024-03-19T01:11:44.168 INFO:tasks.workunit.client.0.smithi069.stderr: 34750K .......... .......... .......... .......... .......... 25% 5.34M 15s 2024-03-19T01:11:44.174 INFO:tasks.workunit.client.0.smithi069.stderr: 34800K .......... .......... .......... .......... .......... 25% 8.94M 15s 2024-03-19T01:11:44.178 INFO:tasks.workunit.client.0.smithi069.stderr: 34850K .......... .......... .......... .......... .......... 25% 10.9M 15s 2024-03-19T01:11:44.183 INFO:tasks.workunit.client.0.smithi069.stderr: 34900K .......... .......... .......... .......... .......... 25% 9.12M 15s 2024-03-19T01:11:44.188 INFO:tasks.workunit.client.0.smithi069.stderr: 34950K .......... .......... .......... .......... .......... 25% 11.3M 15s 2024-03-19T01:11:44.193 INFO:tasks.workunit.client.0.smithi069.stderr: 35000K .......... .......... .......... .......... .......... 25% 11.2M 15s 2024-03-19T01:11:44.198 INFO:tasks.workunit.client.0.smithi069.stderr: 35050K .......... .......... .......... .......... .......... 25% 12.2M 15s 2024-03-19T01:11:44.199 INFO:tasks.workunit.client.0.smithi069.stderr: 35100K .......... .......... .......... .......... .......... 25% 15.2M 15s 2024-03-19T01:11:44.206 INFO:tasks.workunit.client.0.smithi069.stderr: 35150K .......... .......... .......... .......... .......... 25% 9.91M 15s 2024-03-19T01:11:44.210 INFO:tasks.workunit.client.0.smithi069.stderr: 35200K .......... .......... .......... .......... .......... 25% 12.2M 15s 2024-03-19T01:11:44.214 INFO:tasks.workunit.client.0.smithi069.stderr: 35250K .......... .......... .......... .......... .......... 26% 9.63M 15s 2024-03-19T01:11:44.217 INFO:tasks.workunit.client.0.smithi069.stderr: 35300K .......... .......... .......... .......... .......... 26% 12.8M 15s 2024-03-19T01:11:44.220 INFO:tasks.workunit.client.0.smithi069.stderr: 35350K .......... .......... .......... .......... .......... 26% 15.5M 15s 2024-03-19T01:11:44.226 INFO:tasks.workunit.client.0.smithi069.stderr: 35400K .......... .......... .......... .......... .......... 26% 9.17M 15s 2024-03-19T01:11:44.230 INFO:tasks.workunit.client.0.smithi069.stderr: 35450K .......... .......... .......... .......... .......... 26% 10.9M 15s 2024-03-19T01:11:44.235 INFO:tasks.workunit.client.0.smithi069.stderr: 35500K .......... .......... .......... .......... .......... 26% 9.25M 15s 2024-03-19T01:11:44.238 INFO:tasks.workunit.client.0.smithi069.stderr: 35550K .......... .......... .......... .......... .......... 26% 19.6M 15s 2024-03-19T01:11:44.242 INFO:tasks.workunit.client.0.smithi069.stderr: 35600K .......... .......... .......... .......... .......... 26% 10.4M 15s 2024-03-19T01:11:44.246 INFO:tasks.workunit.client.0.smithi069.stderr: 35650K .......... .......... .......... .......... .......... 26% 12.9M 15s 2024-03-19T01:11:44.251 INFO:tasks.workunit.client.0.smithi069.stderr: 35700K .......... .......... .......... .......... .......... 26% 9.50M 15s 2024-03-19T01:11:44.256 INFO:tasks.workunit.client.0.smithi069.stderr: 35750K .......... .......... .......... .......... .......... 26% 9.77M 15s 2024-03-19T01:11:44.259 INFO:tasks.workunit.client.0.smithi069.stderr: 35800K .......... .......... .......... .......... .......... 26% 17.5M 15s 2024-03-19T01:11:44.264 INFO:tasks.workunit.client.0.smithi069.stderr: 35850K .......... .......... .......... .......... .......... 26% 9.89M 15s 2024-03-19T01:11:44.268 INFO:tasks.workunit.client.0.smithi069.stderr: 35900K .......... .......... .......... .......... .......... 26% 13.0M 15s 2024-03-19T01:11:44.273 INFO:tasks.workunit.client.0.smithi069.stderr: 35950K .......... .......... .......... .......... .......... 26% 9.16M 15s 2024-03-19T01:11:44.277 INFO:tasks.workunit.client.0.smithi069.stderr: 36000K .......... .......... .......... .......... .......... 26% 11.8M 15s 2024-03-19T01:11:44.280 INFO:tasks.workunit.client.0.smithi069.stderr: 36050K .......... .......... .......... .......... .......... 26% 16.0M 15s 2024-03-19T01:11:44.286 INFO:tasks.workunit.client.0.smithi069.stderr: 36100K .......... .......... .......... .......... .......... 26% 10.1M 15s 2024-03-19T01:11:44.291 INFO:tasks.workunit.client.0.smithi069.stderr: 36150K .......... .......... .......... .......... .......... 26% 12.6M 15s 2024-03-19T01:11:44.295 INFO:tasks.workunit.client.0.smithi069.stderr: 36200K .......... .......... .......... .......... .......... 26% 9.53M 15s 2024-03-19T01:11:44.299 INFO:tasks.workunit.client.0.smithi069.stderr: 36250K .......... .......... .......... .......... .......... 26% 11.7M 15s 2024-03-19T01:11:44.302 INFO:tasks.workunit.client.0.smithi069.stderr: 36300K .......... .......... .......... .......... .......... 26% 16.6M 15s 2024-03-19T01:11:44.307 INFO:tasks.workunit.client.0.smithi069.stderr: 36350K .......... .......... .......... .......... .......... 26% 9.17M 15s 2024-03-19T01:11:44.312 INFO:tasks.workunit.client.0.smithi069.stderr: 36400K .......... .......... .......... .......... .......... 26% 10.8M 15s 2024-03-19T01:11:44.317 INFO:tasks.workunit.client.0.smithi069.stderr: 36450K .......... .......... .......... .......... .......... 26% 9.69M 15s 2024-03-19T01:11:44.320 INFO:tasks.workunit.client.0.smithi069.stderr: 36500K .......... .......... .......... .......... .......... 26% 18.1M 15s 2024-03-19T01:11:44.324 INFO:tasks.workunit.client.0.smithi069.stderr: 36550K .......... .......... .......... .......... .......... 26% 10.5M 15s 2024-03-19T01:11:44.329 INFO:tasks.workunit.client.0.smithi069.stderr: 36600K .......... .......... .......... .......... .......... 26% 13.8M 14s 2024-03-19T01:11:44.333 INFO:tasks.workunit.client.0.smithi069.stderr: 36650K .......... .......... .......... .......... .......... 27% 8.96M 14s 2024-03-19T01:11:44.339 INFO:tasks.workunit.client.0.smithi069.stderr: 36700K .......... .......... .......... .......... .......... 27% 9.94M 14s 2024-03-19T01:11:44.342 INFO:tasks.workunit.client.0.smithi069.stderr: 36750K .......... .......... .......... .......... .......... 27% 14.9M 14s 2024-03-19T01:11:44.347 INFO:tasks.workunit.client.0.smithi069.stderr: 36800K .......... .......... .......... .......... .......... 27% 9.41M 14s 2024-03-19T01:11:44.350 INFO:tasks.workunit.client.0.smithi069.stderr: 36850K .......... .......... .......... .......... .......... 27% 12.5M 14s 2024-03-19T01:11:44.355 INFO:tasks.workunit.client.0.smithi069.stderr: 36900K .......... .......... .......... .......... .......... 27% 10.7M 14s 2024-03-19T01:11:44.358 INFO:tasks.workunit.client.0.smithi069.stderr: 36950K .......... .......... .......... .......... .......... 27% 12.9M 14s 2024-03-19T01:11:44.362 INFO:tasks.workunit.client.0.smithi069.stderr: 37000K .......... .......... .......... .......... .......... 27% 15.1M 14s 2024-03-19T01:11:44.368 INFO:tasks.workunit.client.0.smithi069.stderr: 37050K .......... .......... .......... .......... .......... 27% 9.89M 14s 2024-03-19T01:11:44.372 INFO:tasks.workunit.client.0.smithi069.stderr: 37100K .......... .......... .......... .......... .......... 27% 8.60M 14s 2024-03-19T01:11:44.375 INFO:tasks.workunit.client.0.smithi069.stderr: 37150K .......... .......... .......... .......... .......... 27% 14.9M 14s 2024-03-19T01:11:44.380 INFO:tasks.workunit.client.0.smithi069.stderr: 37200K .......... .......... .......... .......... .......... 27% 12.0M 14s 2024-03-19T01:11:44.384 INFO:tasks.workunit.client.0.smithi069.stderr: 37250K .......... .......... .......... .......... .......... 27% 14.9M 14s 2024-03-19T01:11:44.388 INFO:tasks.workunit.client.0.smithi069.stderr: 37300K .......... .......... .......... .......... .......... 27% 9.22M 14s 2024-03-19T01:11:44.394 INFO:tasks.workunit.client.0.smithi069.stderr: 37350K .......... .......... .......... .......... .......... 27% 8.75M 14s 2024-03-19T01:11:44.398 INFO:tasks.workunit.client.0.smithi069.stderr: 37400K .......... .......... .......... .......... .......... 27% 12.1M 14s 2024-03-19T01:11:44.400 INFO:tasks.workunit.client.0.smithi069.stderr: 37450K .......... .......... .......... .......... .......... 27% 18.1M 14s 2024-03-19T01:11:44.406 INFO:tasks.workunit.client.0.smithi069.stderr: 37500K .......... .......... .......... .......... .......... 27% 10.9M 14s 2024-03-19T01:11:44.410 INFO:tasks.workunit.client.0.smithi069.stderr: 37550K .......... .......... .......... .......... .......... 27% 9.78M 14s 2024-03-19T01:11:44.416 INFO:tasks.workunit.client.0.smithi069.stderr: 37600K .......... .......... .......... .......... .......... 27% 8.83M 14s 2024-03-19T01:11:44.419 INFO:tasks.workunit.client.0.smithi069.stderr: 37650K .......... .......... .......... .......... .......... 27% 16.0M 14s 2024-03-19T01:11:44.423 INFO:tasks.workunit.client.0.smithi069.stderr: 37700K .......... .......... .......... .......... .......... 27% 13.5M 14s 2024-03-19T01:11:44.427 INFO:tasks.workunit.client.0.smithi069.stderr: 37750K .......... .......... .......... .......... .......... 27% 9.62M 14s 2024-03-19T01:11:44.431 INFO:tasks.workunit.client.0.smithi069.stderr: 37800K .......... .......... .......... .......... .......... 27% 12.5M 14s 2024-03-19T01:11:44.436 INFO:tasks.workunit.client.0.smithi069.stderr: 37850K .......... .......... .......... .......... .......... 27% 9.53M 14s 2024-03-19T01:11:44.439 INFO:tasks.workunit.client.0.smithi069.stderr: 37900K .......... .......... .......... .......... .......... 27% 14.2M 14s 2024-03-19T01:11:44.443 INFO:tasks.workunit.client.0.smithi069.stderr: 37950K .......... .......... .......... .......... .......... 27% 14.5M 14s 2024-03-19T01:11:44.449 INFO:tasks.workunit.client.0.smithi069.stderr: 38000K .......... .......... .......... .......... .......... 28% 8.39M 14s 2024-03-19T01:11:44.454 INFO:tasks.workunit.client.0.smithi069.stderr: 38050K .......... .......... .......... .......... .......... 28% 10.8M 14s 2024-03-19T01:11:44.459 INFO:tasks.workunit.client.0.smithi069.stderr: 38100K .......... .......... .......... .......... .......... 28% 9.42M 14s 2024-03-19T01:11:44.461 INFO:tasks.workunit.client.0.smithi069.stderr: 38150K .......... .......... .......... .......... .......... 28% 23.9M 14s 2024-03-19T01:11:44.466 INFO:tasks.workunit.client.0.smithi069.stderr: 38200K .......... .......... .......... .......... .......... 28% 9.19M 14s 2024-03-19T01:11:44.469 INFO:tasks.workunit.client.0.smithi069.stderr: 38250K .......... .......... .......... .......... .......... 28% 14.7M 14s 2024-03-19T01:11:44.475 INFO:tasks.workunit.client.0.smithi069.stderr: 38300K .......... .......... .......... .......... .......... 28% 10.1M 14s 2024-03-19T01:11:44.480 INFO:tasks.workunit.client.0.smithi069.stderr: 38350K .......... .......... .......... .......... .......... 28% 8.58M 14s 2024-03-19T01:11:44.481 INFO:tasks.workunit.client.0.smithi069.stderr: 38400K .......... .......... .......... .......... .......... 28% 39.2M 14s 2024-03-19T01:11:44.486 INFO:tasks.workunit.client.0.smithi069.stderr: 38450K .......... .......... .......... .......... .......... 28% 9.63M 14s 2024-03-19T01:11:44.490 INFO:tasks.workunit.client.0.smithi069.stderr: 38500K .......... .......... .......... .......... .......... 28% 14.1M 14s 2024-03-19T01:11:44.494 INFO:tasks.workunit.client.0.smithi069.stderr: 38550K .......... .......... .......... .......... .......... 28% 10.2M 14s 2024-03-19T01:11:44.500 INFO:tasks.workunit.client.0.smithi069.stderr: 38600K .......... .......... .......... .......... .......... 28% 8.38M 14s 2024-03-19T01:11:44.502 INFO:tasks.workunit.client.0.smithi069.stderr: 38650K .......... .......... .......... .......... .......... 28% 32.1M 14s 2024-03-19T01:11:44.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:44 smithi086 ceph-mon[66829]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-03-19T01:11:44.506 INFO:tasks.workunit.client.0.smithi069.stderr: 38700K .......... .......... .......... .......... .......... 28% 10.5M 14s 2024-03-19T01:11:44.510 INFO:tasks.workunit.client.0.smithi069.stderr: 38750K .......... .......... .......... .......... .......... 28% 14.0M 14s 2024-03-19T01:11:44.515 INFO:tasks.workunit.client.0.smithi069.stderr: 38800K .......... .......... .......... .......... .......... 28% 10.0M 14s 2024-03-19T01:11:44.522 INFO:tasks.workunit.client.0.smithi069.stderr: 38850K .......... .......... .......... .......... .......... 28% 8.24M 14s 2024-03-19T01:11:44.522 INFO:tasks.workunit.client.0.smithi069.stderr: 38900K .......... .......... .......... .......... .......... 28% 36.7M 14s 2024-03-19T01:11:44.526 INFO:tasks.workunit.client.0.smithi069.stderr: 38950K .......... .......... .......... .......... .......... 28% 10.5M 14s 2024-03-19T01:11:44.530 INFO:tasks.workunit.client.0.smithi069.stderr: 39000K .......... .......... .......... .......... .......... 28% 13.3M 14s 2024-03-19T01:11:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:44 smithi069 ceph-mon[31727]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-03-19T01:11:44.535 INFO:tasks.workunit.client.0.smithi069.stderr: 39050K .......... .......... .......... .......... .......... 28% 10.5M 14s 2024-03-19T01:11:44.541 INFO:tasks.workunit.client.0.smithi069.stderr: 39100K .......... .......... .......... .......... .......... 28% 8.03M 14s 2024-03-19T01:11:44.543 INFO:tasks.workunit.client.0.smithi069.stderr: 39150K .......... .......... .......... .......... .......... 28% 27.9M 14s 2024-03-19T01:11:44.547 INFO:tasks.workunit.client.0.smithi069.stderr: 39200K .......... .......... .......... .......... .......... 28% 11.9M 14s 2024-03-19T01:11:44.550 INFO:tasks.workunit.client.0.smithi069.stderr: 39250K .......... .......... .......... .......... .......... 28% 13.0M 14s 2024-03-19T01:11:44.555 INFO:tasks.workunit.client.0.smithi069.stderr: 39300K .......... .......... .......... .......... .......... 28% 10.5M 14s 2024-03-19T01:11:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:44 smithi163 ceph-mon[67428]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-03-19T01:11:44.562 INFO:tasks.workunit.client.0.smithi069.stderr: 39350K .......... .......... .......... .......... .......... 29% 8.26M 14s 2024-03-19T01:11:44.563 INFO:tasks.workunit.client.0.smithi069.stderr: 39400K .......... .......... .......... .......... .......... 29% 30.8M 14s 2024-03-19T01:11:44.567 INFO:tasks.workunit.client.0.smithi069.stderr: 39450K .......... .......... .......... .......... .......... 29% 10.6M 14s 2024-03-19T01:11:44.571 INFO:tasks.workunit.client.0.smithi069.stderr: 39500K .......... .......... .......... .......... .......... 29% 13.4M 14s 2024-03-19T01:11:44.576 INFO:tasks.workunit.client.0.smithi069.stderr: 39550K .......... .......... .......... .......... .......... 29% 10.4M 14s 2024-03-19T01:11:44.582 INFO:tasks.workunit.client.0.smithi069.stderr: 39600K .......... .......... .......... .......... .......... 29% 8.46M 14s 2024-03-19T01:11:44.583 INFO:tasks.workunit.client.0.smithi069.stderr: 39650K .......... .......... .......... .......... .......... 29% 32.3M 14s 2024-03-19T01:11:44.587 INFO:tasks.workunit.client.0.smithi069.stderr: 39700K .......... .......... .......... .......... .......... 29% 10.4M 14s 2024-03-19T01:11:44.591 INFO:tasks.workunit.client.0.smithi069.stderr: 39750K .......... .......... .......... .......... .......... 29% 14.5M 14s 2024-03-19T01:11:44.596 INFO:tasks.workunit.client.0.smithi069.stderr: 39800K .......... .......... .......... .......... .......... 29% 9.97M 14s 2024-03-19T01:11:44.604 INFO:tasks.workunit.client.0.smithi069.stderr: 39850K .......... .......... .......... .......... .......... 29% 8.43M 14s 2024-03-19T01:11:44.604 INFO:tasks.workunit.client.0.smithi069.stderr: 39900K .......... .......... .......... .......... .......... 29% 29.9M 14s 2024-03-19T01:11:44.608 INFO:tasks.workunit.client.0.smithi069.stderr: 39950K .......... .......... .......... .......... .......... 29% 10.6M 14s 2024-03-19T01:11:44.612 INFO:tasks.workunit.client.0.smithi069.stderr: 40000K .......... .......... .......... .......... .......... 29% 13.5M 13s 2024-03-19T01:11:44.618 INFO:tasks.workunit.client.0.smithi069.stderr: 40050K .......... .......... .......... .......... .......... 29% 10.2M 13s 2024-03-19T01:11:44.622 INFO:tasks.workunit.client.0.smithi069.stderr: 40100K .......... .......... .......... .......... .......... 29% 8.49M 13s 2024-03-19T01:11:44.624 INFO:tasks.workunit.client.0.smithi069.stderr: 40150K .......... .......... .......... .......... .......... 29% 34.8M 13s 2024-03-19T01:11:44.631 INFO:tasks.workunit.client.0.smithi069.stderr: 40200K .......... .......... .......... .......... .......... 29% 10.1M 13s 2024-03-19T01:11:44.631 INFO:tasks.workunit.client.0.smithi069.stderr: 40250K .......... .......... .......... .......... .......... 29% 15.4M 13s 2024-03-19T01:11:44.636 INFO:tasks.workunit.client.0.smithi069.stderr: 40300K .......... .......... .......... .......... .......... 29% 9.65M 13s 2024-03-19T01:11:44.642 INFO:tasks.workunit.client.0.smithi069.stderr: 40350K .......... .......... .......... .......... .......... 29% 8.48M 13s 2024-03-19T01:11:44.644 INFO:tasks.workunit.client.0.smithi069.stderr: 40400K .......... .......... .......... .......... .......... 29% 32.6M 13s 2024-03-19T01:11:44.649 INFO:tasks.workunit.client.0.smithi069.stderr: 40450K .......... .......... .......... .......... .......... 29% 10.1M 13s 2024-03-19T01:11:44.652 INFO:tasks.workunit.client.0.smithi069.stderr: 40500K .......... .......... .......... .......... .......... 29% 15.7M 13s 2024-03-19T01:11:44.657 INFO:tasks.workunit.client.0.smithi069.stderr: 40550K .......... .......... .......... .......... .......... 29% 9.27M 13s 2024-03-19T01:11:44.662 INFO:tasks.workunit.client.0.smithi069.stderr: 40600K .......... .......... .......... .......... .......... 29% 8.83M 13s 2024-03-19T01:11:44.664 INFO:tasks.workunit.client.0.smithi069.stderr: 40650K .......... .......... .......... .......... .......... 29% 31.5M 13s 2024-03-19T01:11:44.669 INFO:tasks.workunit.client.0.smithi069.stderr: 40700K .......... .......... .......... .......... .......... 30% 9.98M 13s 2024-03-19T01:11:44.672 INFO:tasks.workunit.client.0.smithi069.stderr: 40750K .......... .......... .......... .......... .......... 30% 16.1M 13s 2024-03-19T01:11:44.677 INFO:tasks.workunit.client.0.smithi069.stderr: 40800K .......... .......... .......... .......... .......... 30% 9.41M 13s 2024-03-19T01:11:44.683 INFO:tasks.workunit.client.0.smithi069.stderr: 40850K .......... .......... .......... .......... .......... 30% 8.82M 13s 2024-03-19T01:11:44.684 INFO:tasks.workunit.client.0.smithi069.stderr: 40900K .......... .......... .......... .......... .......... 30% 43.7M 13s 2024-03-19T01:11:44.689 INFO:tasks.workunit.client.0.smithi069.stderr: 40950K .......... .......... .......... .......... .......... 30% 9.00M 13s 2024-03-19T01:11:44.692 INFO:tasks.workunit.client.0.smithi069.stderr: 41000K .......... .......... .......... .......... .......... 30% 16.1M 13s 2024-03-19T01:11:44.697 INFO:tasks.workunit.client.0.smithi069.stderr: 41050K .......... .......... .......... .......... .......... 30% 9.56M 13s 2024-03-19T01:11:44.703 INFO:tasks.workunit.client.0.smithi069.stderr: 41100K .......... .......... .......... .......... .......... 30% 8.69M 13s 2024-03-19T01:11:44.704 INFO:tasks.workunit.client.0.smithi069.stderr: 41150K .......... .......... .......... .......... .......... 30% 50.8M 13s 2024-03-19T01:11:44.710 INFO:tasks.workunit.client.0.smithi069.stderr: 41200K .......... .......... .......... .......... .......... 30% 8.82M 13s 2024-03-19T01:11:44.713 INFO:tasks.workunit.client.0.smithi069.stderr: 41250K .......... .......... .......... .......... .......... 30% 16.2M 13s 2024-03-19T01:11:44.718 INFO:tasks.workunit.client.0.smithi069.stderr: 41300K .......... .......... .......... .......... .......... 30% 9.33M 13s 2024-03-19T01:11:44.723 INFO:tasks.workunit.client.0.smithi069.stderr: 41350K .......... .......... .......... .......... .......... 30% 8.62M 13s 2024-03-19T01:11:44.724 INFO:tasks.workunit.client.0.smithi069.stderr: 41400K .......... .......... .......... .......... .......... 30% 60.6M 13s 2024-03-19T01:11:44.730 INFO:tasks.workunit.client.0.smithi069.stderr: 41450K .......... .......... .......... .......... .......... 30% 8.77M 13s 2024-03-19T01:11:44.733 INFO:tasks.workunit.client.0.smithi069.stderr: 41500K .......... .......... .......... .......... .......... 30% 15.7M 13s 2024-03-19T01:11:44.736 INFO:tasks.workunit.client.0.smithi069.stderr: 41550K .......... .......... .......... .......... .......... 30% 15.3M 13s 2024-03-19T01:11:44.745 INFO:tasks.workunit.client.0.smithi069.stderr: 41600K .......... .......... .......... .......... .......... 30% 6.39M 13s 2024-03-19T01:11:44.747 INFO:tasks.workunit.client.0.smithi069.stderr: 41650K .......... .......... .......... .......... .......... 30% 58.7M 13s 2024-03-19T01:11:44.750 INFO:tasks.workunit.client.0.smithi069.stderr: 41700K .......... .......... .......... .......... .......... 30% 8.94M 13s 2024-03-19T01:11:44.752 INFO:tasks.workunit.client.0.smithi069.stderr: 41750K .......... .......... .......... .......... .......... 30% 20.4M 13s 2024-03-19T01:11:44.756 INFO:tasks.workunit.client.0.smithi069.stderr: 41800K .......... .......... .......... .......... .......... 30% 12.4M 13s 2024-03-19T01:11:44.765 INFO:tasks.workunit.client.0.smithi069.stderr: 41850K .......... .......... .......... .......... .......... 30% 6.28M 13s 2024-03-19T01:11:44.768 INFO:tasks.workunit.client.0.smithi069.stderr: 41900K .......... .......... .......... .......... .......... 30% 59.5M 13s 2024-03-19T01:11:44.771 INFO:tasks.workunit.client.0.smithi069.stderr: 41950K .......... .......... .......... .......... .......... 30% 8.92M 13s 2024-03-19T01:11:44.773 INFO:tasks.workunit.client.0.smithi069.stderr: 42000K .......... .......... .......... .......... .......... 30% 20.3M 13s 2024-03-19T01:11:44.777 INFO:tasks.workunit.client.0.smithi069.stderr: 42050K .......... .......... .......... .......... .......... 31% 12.6M 13s 2024-03-19T01:11:44.784 INFO:tasks.workunit.client.0.smithi069.stderr: 42100K .......... .......... .......... .......... .......... 31% 6.28M 13s 2024-03-19T01:11:44.788 INFO:tasks.workunit.client.0.smithi069.stderr: 42150K .......... .......... .......... .......... .......... 31% 15.0M 13s 2024-03-19T01:11:44.791 INFO:tasks.workunit.client.0.smithi069.stderr: 42200K .......... .......... .......... .......... .......... 31% 16.2M 13s 2024-03-19T01:11:44.794 INFO:tasks.workunit.client.0.smithi069.stderr: 42250K .......... .......... .......... .......... .......... 31% 21.5M 13s 2024-03-19T01:11:44.797 INFO:tasks.workunit.client.0.smithi069.stderr: 42300K .......... .......... .......... .......... .......... 31% 12.4M 13s 2024-03-19T01:11:44.806 INFO:tasks.workunit.client.0.smithi069.stderr: 42350K .......... .......... .......... .......... .......... 31% 6.28M 13s 2024-03-19T01:11:44.809 INFO:tasks.workunit.client.0.smithi069.stderr: 42400K .......... .......... .......... .......... .......... 31% 14.5M 13s 2024-03-19T01:11:44.811 INFO:tasks.workunit.client.0.smithi069.stderr: 42450K .......... .......... .......... .......... .......... 31% 16.1M 13s 2024-03-19T01:11:44.813 INFO:tasks.workunit.client.0.smithi069.stderr: 42500K .......... .......... .......... .......... .......... 31% 22.5M 13s 2024-03-19T01:11:44.817 INFO:tasks.workunit.client.0.smithi069.stderr: 42550K .......... .......... .......... .......... .......... 31% 12.9M 13s 2024-03-19T01:11:44.827 INFO:tasks.workunit.client.0.smithi069.stderr: 42600K .......... .......... .......... .......... .......... 31% 6.08M 13s 2024-03-19T01:11:44.829 INFO:tasks.workunit.client.0.smithi069.stderr: 42650K .......... .......... .......... .......... .......... 31% 15.0M 13s 2024-03-19T01:11:44.832 INFO:tasks.workunit.client.0.smithi069.stderr: 42700K .......... .......... .......... .......... .......... 31% 15.9M 13s 2024-03-19T01:11:44.834 INFO:tasks.workunit.client.0.smithi069.stderr: 42750K .......... .......... .......... .......... .......... 31% 22.8M 13s 2024-03-19T01:11:44.838 INFO:tasks.workunit.client.0.smithi069.stderr: 42800K .......... .......... .......... .......... .......... 31% 12.9M 13s 2024-03-19T01:11:44.846 INFO:tasks.workunit.client.0.smithi069.stderr: 42850K .......... .......... .......... .......... .......... 31% 5.99M 13s 2024-03-19T01:11:44.849 INFO:tasks.workunit.client.0.smithi069.stderr: 42900K .......... .......... .......... .......... .......... 31% 15.5M 13s 2024-03-19T01:11:44.850 INFO:tasks.workunit.client.0.smithi069.stderr: 42950K .......... .......... .......... .......... .......... 31% 69.5M 13s 2024-03-19T01:11:44.854 INFO:tasks.workunit.client.0.smithi069.stderr: 43000K .......... .......... .......... .......... .......... 31% 10.8M 13s 2024-03-19T01:11:44.858 INFO:tasks.workunit.client.0.smithi069.stderr: 43050K .......... .......... .......... .......... .......... 31% 12.6M 13s 2024-03-19T01:11:44.867 INFO:tasks.workunit.client.0.smithi069.stderr: 43100K .......... .......... .......... .......... .......... 31% 6.07M 13s 2024-03-19T01:11:44.870 INFO:tasks.workunit.client.0.smithi069.stderr: 43150K .......... .......... .......... .......... .......... 31% 15.7M 13s 2024-03-19T01:11:44.870 INFO:tasks.workunit.client.0.smithi069.stderr: 43200K .......... .......... .......... .......... .......... 31% 65.0M 13s 2024-03-19T01:11:44.874 INFO:tasks.workunit.client.0.smithi069.stderr: 43250K .......... .......... .......... .......... .......... 31% 11.1M 13s 2024-03-19T01:11:44.878 INFO:tasks.workunit.client.0.smithi069.stderr: 43300K .......... .......... .......... .......... .......... 31% 12.1M 13s 2024-03-19T01:11:44.887 INFO:tasks.workunit.client.0.smithi069.stderr: 43350K .......... .......... .......... .......... .......... 31% 6.05M 13s 2024-03-19T01:11:44.890 INFO:tasks.workunit.client.0.smithi069.stderr: 43400K .......... .......... .......... .......... .......... 32% 16.0M 13s 2024-03-19T01:11:44.890 INFO:tasks.workunit.client.0.smithi069.stderr: 43450K .......... .......... .......... .......... .......... 32% 78.4M 13s 2024-03-19T01:11:44.895 INFO:tasks.workunit.client.0.smithi069.stderr: 43500K .......... .......... .......... .......... .......... 32% 10.9M 13s 2024-03-19T01:11:44.898 INFO:tasks.workunit.client.0.smithi069.stderr: 43550K .......... .......... .......... .......... .......... 32% 12.2M 13s 2024-03-19T01:11:44.908 INFO:tasks.workunit.client.0.smithi069.stderr: 43600K .......... .......... .......... .......... .......... 32% 6.12M 13s 2024-03-19T01:11:44.908 INFO:tasks.workunit.client.0.smithi069.stderr: 43650K .......... .......... .......... .......... .......... 32% 51.4M 13s 2024-03-19T01:11:44.911 INFO:tasks.workunit.client.0.smithi069.stderr: 43700K .......... .......... .......... .......... .......... 32% 16.0M 13s 2024-03-19T01:11:44.914 INFO:tasks.workunit.client.0.smithi069.stderr: 43750K .......... .......... .......... .......... .......... 32% 14.5M 12s 2024-03-19T01:11:44.918 INFO:tasks.workunit.client.0.smithi069.stderr: 43800K .......... .......... .......... .......... .......... 32% 12.0M 12s 2024-03-19T01:11:44.925 INFO:tasks.workunit.client.0.smithi069.stderr: 43850K .......... .......... .......... .......... .......... 32% 6.58M 12s 2024-03-19T01:11:44.928 INFO:tasks.workunit.client.0.smithi069.stderr: 43900K .......... .......... .......... .......... .......... 32% 21.9M 12s 2024-03-19T01:11:44.931 INFO:tasks.workunit.client.0.smithi069.stderr: 43950K .......... .......... .......... .......... .......... 32% 16.6M 12s 2024-03-19T01:11:44.933 INFO:tasks.workunit.client.0.smithi069.stderr: 44000K .......... .......... .......... .......... .......... 32% 16.7M 12s 2024-03-19T01:11:44.936 INFO:tasks.workunit.client.0.smithi069.stderr: 44050K .......... .......... .......... .......... .......... 32% 17.2M 12s 2024-03-19T01:11:44.946 INFO:tasks.workunit.client.0.smithi069.stderr: 44100K .......... .......... .......... .......... .......... 32% 5.32M 12s 2024-03-19T01:11:44.949 INFO:tasks.workunit.client.0.smithi069.stderr: 44150K .......... .......... .......... .......... .......... 32% 21.7M 12s 2024-03-19T01:11:44.951 INFO:tasks.workunit.client.0.smithi069.stderr: 44200K .......... .......... .......... .......... .......... 32% 17.3M 12s 2024-03-19T01:11:44.951 INFO:tasks.workunit.client.0.smithi069.stderr: 44250K .......... .......... .......... .......... .......... 32% 86.6M 12s 2024-03-19T01:11:44.956 INFO:tasks.workunit.client.0.smithi069.stderr: 44300K .......... .......... .......... .......... .......... 32% 10.7M 12s 2024-03-19T01:11:44.965 INFO:tasks.workunit.client.0.smithi069.stderr: 44350K .......... .......... .......... .......... .......... 32% 5.16M 12s 2024-03-19T01:11:44.967 INFO:tasks.workunit.client.0.smithi069.stderr: 44400K .......... .......... .......... .......... .......... 32% 20.2M 12s 2024-03-19T01:11:44.968 INFO:tasks.workunit.client.0.smithi069.stderr: 44450K .......... .......... .......... .......... .......... 32% 46.8M 12s 2024-03-19T01:11:44.972 INFO:tasks.workunit.client.0.smithi069.stderr: 44500K .......... .......... .......... .......... .......... 32% 15.8M 12s 2024-03-19T01:11:44.976 INFO:tasks.workunit.client.0.smithi069.stderr: 44550K .......... .......... .......... .......... .......... 32% 11.1M 12s 2024-03-19T01:11:44.980 INFO:tasks.workunit.client.0.smithi069.stderr: 44600K .......... .......... .......... .......... .......... 32% 11.7M 12s 2024-03-19T01:11:44.987 INFO:tasks.workunit.client.0.smithi069.stderr: 44650K .......... .......... .......... .......... .......... 32% 7.52M 12s 2024-03-19T01:11:44.989 INFO:tasks.workunit.client.0.smithi069.stderr: 44700K .......... .......... .......... .......... .......... 32% 23.7M 12s 2024-03-19T01:11:44.992 INFO:tasks.workunit.client.0.smithi069.stderr: 44750K .......... .......... .......... .......... .......... 33% 16.5M 12s 2024-03-19T01:11:44.997 INFO:tasks.workunit.client.0.smithi069.stderr: 44800K .......... .......... .......... .......... .......... 33% 10.9M 12s 2024-03-19T01:11:44.999 INFO:tasks.workunit.client.0.smithi069.stderr: 44850K .......... .......... .......... .......... .......... 33% 15.4M 12s 2024-03-19T01:11:45.007 INFO:tasks.workunit.client.0.smithi069.stderr: 44900K .......... .......... .......... .......... .......... 33% 6.66M 12s 2024-03-19T01:11:45.009 INFO:tasks.workunit.client.0.smithi069.stderr: 44950K .......... .......... .......... .......... .......... 33% 22.6M 12s 2024-03-19T01:11:45.012 INFO:tasks.workunit.client.0.smithi069.stderr: 45000K .......... .......... .......... .......... .......... 33% 18.2M 12s 2024-03-19T01:11:45.015 INFO:tasks.workunit.client.0.smithi069.stderr: 45050K .......... .......... .......... .......... .......... 33% 14.5M 12s 2024-03-19T01:11:45.017 INFO:tasks.workunit.client.0.smithi069.stderr: 45100K .......... .......... .......... .......... .......... 33% 24.0M 12s 2024-03-19T01:11:45.028 INFO:tasks.workunit.client.0.smithi069.stderr: 45150K .......... .......... .......... .......... .......... 33% 5.25M 12s 2024-03-19T01:11:45.029 INFO:tasks.workunit.client.0.smithi069.stderr: 45200K .......... .......... .......... .......... .......... 33% 19.9M 12s 2024-03-19T01:11:45.033 INFO:tasks.workunit.client.0.smithi069.stderr: 45250K .......... .......... .......... .......... .......... 33% 16.1M 12s 2024-03-19T01:11:45.033 INFO:tasks.workunit.client.0.smithi069.stderr: 45300K .......... .......... .......... .......... .......... 33% 50.3M 12s 2024-03-19T01:11:45.037 INFO:tasks.workunit.client.0.smithi069.stderr: 45350K .......... .......... .......... .......... .......... 33% 11.2M 12s 2024-03-19T01:11:45.041 INFO:tasks.workunit.client.0.smithi069.stderr: 45400K .......... .......... .......... .......... .......... 33% 11.4M 12s 2024-03-19T01:11:45.048 INFO:tasks.workunit.client.0.smithi069.stderr: 45450K .......... .......... .......... .......... .......... 33% 7.61M 12s 2024-03-19T01:11:45.050 INFO:tasks.workunit.client.0.smithi069.stderr: 45500K .......... .......... .......... .......... .......... 33% 19.3M 12s 2024-03-19T01:11:45.053 INFO:tasks.workunit.client.0.smithi069.stderr: 45550K .......... .......... .......... .......... .......... 33% 18.5M 12s 2024-03-19T01:11:45.058 INFO:tasks.workunit.client.0.smithi069.stderr: 45600K .......... .......... .......... .......... .......... 33% 11.7M 12s 2024-03-19T01:11:45.060 INFO:tasks.workunit.client.0.smithi069.stderr: 45650K .......... .......... .......... .......... .......... 33% 13.8M 12s 2024-03-19T01:11:45.068 INFO:tasks.workunit.client.0.smithi069.stderr: 45700K .......... .......... .......... .......... .......... 33% 6.90M 12s 2024-03-19T01:11:45.071 INFO:tasks.workunit.client.0.smithi069.stderr: 45750K .......... .......... .......... .......... .......... 33% 18.7M 12s 2024-03-19T01:11:45.074 INFO:tasks.workunit.client.0.smithi069.stderr: 45800K .......... .......... .......... .......... .......... 33% 18.3M 12s 2024-03-19T01:11:45.076 INFO:tasks.workunit.client.0.smithi069.stderr: 45850K .......... .......... .......... .......... .......... 33% 16.0M 12s 2024-03-19T01:11:45.078 INFO:tasks.workunit.client.0.smithi069.stderr: 45900K .......... .......... .......... .......... .......... 33% 24.8M 12s 2024-03-19T01:11:45.087 INFO:tasks.workunit.client.0.smithi069.stderr: 45950K .......... .......... .......... .......... .......... 33% 5.15M 12s 2024-03-19T01:11:45.091 INFO:tasks.workunit.client.0.smithi069.stderr: 46000K .......... .......... .......... .......... .......... 33% 17.8M 12s 2024-03-19T01:11:45.094 INFO:tasks.workunit.client.0.smithi069.stderr: 46050K .......... .......... .......... .......... .......... 33% 18.1M 12s 2024-03-19T01:11:45.094 INFO:tasks.workunit.client.0.smithi069.stderr: 46100K .......... .......... .......... .......... .......... 33% 55.5M 12s 2024-03-19T01:11:45.098 INFO:tasks.workunit.client.0.smithi069.stderr: 46150K .......... .......... .......... .......... .......... 34% 11.4M 12s 2024-03-19T01:11:45.102 INFO:tasks.workunit.client.0.smithi069.stderr: 46200K .......... .......... .......... .......... .......... 34% 11.7M 12s 2024-03-19T01:11:45.110 INFO:tasks.workunit.client.0.smithi069.stderr: 46250K .......... .......... .......... .......... .......... 34% 6.41M 12s 2024-03-19T01:11:45.111 INFO:tasks.workunit.client.0.smithi069.stderr: 46300K .......... .......... .......... .......... .......... 34% 42.5M 12s 2024-03-19T01:11:45.114 INFO:tasks.workunit.client.0.smithi069.stderr: 46350K .......... .......... .......... .......... .......... 34% 17.3M 12s 2024-03-19T01:11:45.118 INFO:tasks.workunit.client.0.smithi069.stderr: 46400K .......... .......... .......... .......... .......... 34% 11.0M 12s 2024-03-19T01:11:45.122 INFO:tasks.workunit.client.0.smithi069.stderr: 46450K .......... .......... .......... .......... .......... 34% 15.3M 12s 2024-03-19T01:11:45.129 INFO:tasks.workunit.client.0.smithi069.stderr: 46500K .......... .......... .......... .......... .......... 34% 6.52M 12s 2024-03-19T01:11:45.132 INFO:tasks.workunit.client.0.smithi069.stderr: 46550K .......... .......... .......... .......... .......... 34% 21.1M 12s 2024-03-19T01:11:45.135 INFO:tasks.workunit.client.0.smithi069.stderr: 46600K .......... .......... .......... .......... .......... 34% 16.9M 12s 2024-03-19T01:11:45.137 INFO:tasks.workunit.client.0.smithi069.stderr: 46650K .......... .......... .......... .......... .......... 34% 16.7M 12s 2024-03-19T01:11:45.139 INFO:tasks.workunit.client.0.smithi069.stderr: 46700K .......... .......... .......... .......... .......... 34% 23.0M 12s 2024-03-19T01:11:45.148 INFO:tasks.workunit.client.0.smithi069.stderr: 46750K .......... .......... .......... .......... .......... 34% 5.34M 12s 2024-03-19T01:11:45.153 INFO:tasks.workunit.client.0.smithi069.stderr: 46800K .......... .......... .......... .......... .......... 34% 19.5M 12s 2024-03-19T01:11:45.155 INFO:tasks.workunit.client.0.smithi069.stderr: 46850K .......... .......... .......... .......... .......... 34% 15.6M 12s 2024-03-19T01:11:45.156 INFO:tasks.workunit.client.0.smithi069.stderr: 46900K .......... .......... .......... .......... .......... 34% 43.0M 12s 2024-03-19T01:11:45.159 INFO:tasks.workunit.client.0.smithi069.stderr: 46950K .......... .......... .......... .......... .......... 34% 12.5M 12s 2024-03-19T01:11:45.163 INFO:tasks.workunit.client.0.smithi069.stderr: 47000K .......... .......... .......... .......... .......... 34% 11.5M 12s 2024-03-19T01:11:45.172 INFO:tasks.workunit.client.0.smithi069.stderr: 47050K .......... .......... .......... .......... .......... 34% 6.39M 12s 2024-03-19T01:11:45.172 INFO:tasks.workunit.client.0.smithi069.stderr: 47100K .......... .......... .......... .......... .......... 34% 47.1M 12s 2024-03-19T01:11:45.175 INFO:tasks.workunit.client.0.smithi069.stderr: 47150K .......... .......... .......... .......... .......... 34% 15.5M 12s 2024-03-19T01:11:45.180 INFO:tasks.workunit.client.0.smithi069.stderr: 47200K .......... .......... .......... .......... .......... 34% 16.3M 12s 2024-03-19T01:11:45.183 INFO:tasks.workunit.client.0.smithi069.stderr: 47250K .......... .......... .......... .......... .......... 34% 10.8M 12s 2024-03-19T01:11:45.190 INFO:tasks.workunit.client.0.smithi069.stderr: 47300K .......... .......... .......... .......... .......... 34% 6.80M 12s 2024-03-19T01:11:45.192 INFO:tasks.workunit.client.0.smithi069.stderr: 47350K .......... .......... .......... .......... .......... 34% 21.7M 12s 2024-03-19T01:11:45.196 INFO:tasks.workunit.client.0.smithi069.stderr: 47400K .......... .......... .......... .......... .......... 34% 16.2M 12s 2024-03-19T01:11:45.198 INFO:tasks.workunit.client.0.smithi069.stderr: 47450K .......... .......... .......... .......... .......... 34% 15.6M 12s 2024-03-19T01:11:45.203 INFO:tasks.workunit.client.0.smithi069.stderr: 47500K .......... .......... .......... .......... .......... 35% 11.2M 12s 2024-03-19T01:11:45.210 INFO:tasks.workunit.client.0.smithi069.stderr: 47550K .......... .......... .......... .......... .......... 35% 7.02M 12s 2024-03-19T01:11:45.213 INFO:tasks.workunit.client.0.smithi069.stderr: 47600K .......... .......... .......... .......... .......... 35% 20.1M 12s 2024-03-19T01:11:45.216 INFO:tasks.workunit.client.0.smithi069.stderr: 47650K .......... .......... .......... .......... .......... 35% 15.6M 12s 2024-03-19T01:11:45.218 INFO:tasks.workunit.client.0.smithi069.stderr: 47700K .......... .......... .......... .......... .......... 35% 15.8M 12s 2024-03-19T01:11:45.220 INFO:tasks.workunit.client.0.smithi069.stderr: 47750K .......... .......... .......... .......... .......... 35% 23.7M 12s 2024-03-19T01:11:45.224 INFO:tasks.workunit.client.0.smithi069.stderr: 47800K .......... .......... .......... .......... .......... 35% 11.7M 11s 2024-03-19T01:11:45.237 INFO:tasks.workunit.client.0.smithi069.stderr: 47850K .......... .......... .......... .......... .......... 35% 6.37M 11s 2024-03-19T01:11:45.237 INFO:tasks.workunit.client.0.smithi069.stderr: 47900K .......... .......... .......... .......... .......... 35% 43.1M 11s 2024-03-19T01:11:45.237 INFO:tasks.workunit.client.0.smithi069.stderr: 47950K .......... .......... .......... .......... .......... 35% 14.9M 11s 2024-03-19T01:11:45.241 INFO:tasks.workunit.client.0.smithi069.stderr: 48000K .......... .......... .......... .......... .......... 35% 12.7M 11s 2024-03-19T01:11:45.244 INFO:tasks.workunit.client.0.smithi069.stderr: 48050K .......... .......... .......... .......... .......... 35% 13.8M 11s 2024-03-19T01:11:45.251 INFO:tasks.workunit.client.0.smithi069.stderr: 48100K .......... .......... .......... .......... .......... 35% 6.94M 11s 2024-03-19T01:11:45.253 INFO:tasks.workunit.client.0.smithi069.stderr: 48150K .......... .......... .......... .......... .......... 35% 24.3M 11s 2024-03-19T01:11:45.257 INFO:tasks.workunit.client.0.smithi069.stderr: 48200K .......... .......... .......... .......... .......... 35% 12.5M 11s 2024-03-19T01:11:45.260 INFO:tasks.workunit.client.0.smithi069.stderr: 48250K .......... .......... .......... .......... .......... 35% 19.0M 11s 2024-03-19T01:11:45.264 INFO:tasks.workunit.client.0.smithi069.stderr: 48300K .......... .......... .......... .......... .......... 35% 11.4M 11s 2024-03-19T01:11:45.270 INFO:tasks.workunit.client.0.smithi069.stderr: 48350K .......... .......... .......... .......... .......... 35% 7.13M 11s 2024-03-19T01:11:45.273 INFO:tasks.workunit.client.0.smithi069.stderr: 48400K .......... .......... .......... .......... .......... 35% 19.3M 11s 2024-03-19T01:11:45.276 INFO:tasks.workunit.client.0.smithi069.stderr: 48450K .......... .......... .......... .......... .......... 35% 14.7M 11s 2024-03-19T01:11:45.280 INFO:tasks.workunit.client.0.smithi069.stderr: 48500K .......... .......... .......... .......... .......... 35% 15.1M 11s 2024-03-19T01:11:45.281 INFO:tasks.workunit.client.0.smithi069.stderr: 48550K .......... .......... .......... .......... .......... 35% 28.2M 11s 2024-03-19T01:11:45.285 INFO:tasks.workunit.client.0.smithi069.stderr: 48600K .......... .......... .......... .......... .......... 35% 12.0M 11s 2024-03-19T01:11:45.293 INFO:tasks.workunit.client.0.smithi069.stderr: 48650K .......... .......... .......... .......... .......... 35% 6.36M 11s 2024-03-19T01:11:45.297 INFO:tasks.workunit.client.0.smithi069.stderr: 48700K .......... .......... .......... .......... .......... 35% 14.8M 11s 2024-03-19T01:11:45.297 INFO:tasks.workunit.client.0.smithi069.stderr: 48750K .......... .......... .......... .......... .......... 35% 44.5M 11s 2024-03-19T01:11:45.302 INFO:tasks.workunit.client.0.smithi069.stderr: 48800K .......... .......... .......... .......... .......... 35% 12.0M 11s 2024-03-19T01:11:45.306 INFO:tasks.workunit.client.0.smithi069.stderr: 48850K .......... .......... .......... .......... .......... 36% 15.1M 11s 2024-03-19T01:11:45.312 INFO:tasks.workunit.client.0.smithi069.stderr: 48900K .......... .......... .......... .......... .......... 36% 6.71M 11s 2024-03-19T01:11:45.315 INFO:tasks.workunit.client.0.smithi069.stderr: 48950K .......... .......... .......... .......... .......... 36% 22.9M 11s 2024-03-19T01:11:45.317 INFO:tasks.workunit.client.0.smithi069.stderr: 49000K .......... .......... .......... .......... .......... 36% 15.7M 11s 2024-03-19T01:11:45.321 INFO:tasks.workunit.client.0.smithi069.stderr: 49050K .......... .......... .......... .......... .......... 36% 13.9M 11s 2024-03-19T01:11:45.325 INFO:tasks.workunit.client.0.smithi069.stderr: 49100K .......... .......... .......... .......... .......... 36% 34.5M 11s 2024-03-19T01:11:45.331 INFO:tasks.workunit.client.0.smithi069.stderr: 49150K .......... .......... .......... .......... .......... 36% 5.31M 11s 2024-03-19T01:11:45.335 INFO:tasks.workunit.client.0.smithi069.stderr: 49200K .......... .......... .......... .......... .......... 36% 19.8M 11s 2024-03-19T01:11:45.335 INFO:tasks.workunit.client.0.smithi069.stderr: 49250K .......... .......... .......... .......... .......... 36% 50.8M 11s 2024-03-19T01:11:45.338 INFO:tasks.workunit.client.0.smithi069.stderr: 49300K .......... .......... .......... .......... .......... 36% 14.5M 11s 2024-03-19T01:11:45.342 INFO:tasks.workunit.client.0.smithi069.stderr: 49350K .......... .......... .......... .......... .......... 36% 12.2M 11s 2024-03-19T01:11:45.347 INFO:tasks.workunit.client.0.smithi069.stderr: 49400K .......... .......... .......... .......... .......... 36% 11.9M 11s 2024-03-19T01:11:45.353 INFO:tasks.workunit.client.0.smithi069.stderr: 49450K .......... .......... .......... .......... .......... 36% 7.55M 11s 2024-03-19T01:11:45.356 INFO:tasks.workunit.client.0.smithi069.stderr: 49500K .......... .......... .......... .......... .......... 36% 25.1M 11s 2024-03-19T01:11:45.359 INFO:tasks.workunit.client.0.smithi069.stderr: 49550K .......... .......... .......... .......... .......... 36% 14.2M 11s 2024-03-19T01:11:45.363 INFO:tasks.workunit.client.0.smithi069.stderr: 49600K .......... .......... .......... .......... .......... 36% 14.0M 11s 2024-03-19T01:11:45.367 INFO:tasks.workunit.client.0.smithi069.stderr: 49650K .......... .......... .......... .......... .......... 36% 13.4M 11s 2024-03-19T01:11:45.372 INFO:tasks.workunit.client.0.smithi069.stderr: 49700K .......... .......... .......... .......... .......... 36% 7.18M 11s 2024-03-19T01:11:45.376 INFO:tasks.workunit.client.0.smithi069.stderr: 49750K .......... .......... .......... .......... .......... 36% 20.3M 11s 2024-03-19T01:11:45.376 INFO:tasks.workunit.client.0.smithi069.stderr: 49800K .......... .......... .......... .......... .......... 36% 42.5M 11s 2024-03-19T01:11:45.379 INFO:tasks.workunit.client.0.smithi069.stderr: 49850K .......... .......... .......... .......... .......... 36% 14.2M 11s 2024-03-19T01:11:45.383 INFO:tasks.workunit.client.0.smithi069.stderr: 49900K .......... .......... .......... .......... .......... 36% 12.6M 11s 2024-03-19T01:11:45.388 INFO:tasks.workunit.client.0.smithi069.stderr: 49950K .......... .......... .......... .......... .......... 36% 11.4M 11s 2024-03-19T01:11:45.394 INFO:tasks.workunit.client.0.smithi069.stderr: 50000K .......... .......... .......... .......... .......... 36% 7.56M 11s 2024-03-19T01:11:45.396 INFO:tasks.workunit.client.0.smithi069.stderr: 50050K .......... .......... .......... .......... .......... 36% 25.2M 11s 2024-03-19T01:11:45.399 INFO:tasks.workunit.client.0.smithi069.stderr: 50100K .......... .......... .......... .......... .......... 36% 15.0M 11s 2024-03-19T01:11:45.403 INFO:tasks.workunit.client.0.smithi069.stderr: 50150K .......... .......... .......... .......... .......... 36% 13.3M 11s 2024-03-19T01:11:45.406 INFO:tasks.workunit.client.0.smithi069.stderr: 50200K .......... .......... .......... .......... .......... 37% 14.3M 11s 2024-03-19T01:11:45.408 INFO:tasks.workunit.client.0.smithi069.stderr: 50250K .......... .......... .......... .......... .......... 37% 23.6M 11s 2024-03-19T01:11:45.417 INFO:tasks.workunit.client.0.smithi069.stderr: 50300K .......... .......... .......... .......... .......... 37% 6.45M 11s 2024-03-19T01:11:45.420 INFO:tasks.workunit.client.0.smithi069.stderr: 50350K .......... .......... .......... .......... .......... 37% 16.6M 11s 2024-03-19T01:11:45.420 INFO:tasks.workunit.client.0.smithi069.stderr: 50400K .......... .......... .......... .......... .......... 37% 49.3M 11s 2024-03-19T01:11:45.424 INFO:tasks.workunit.client.0.smithi069.stderr: 50450K .......... .......... .......... .......... .......... 37% 12.4M 11s 2024-03-19T01:11:45.427 INFO:tasks.workunit.client.0.smithi069.stderr: 50500K .......... .......... .......... .......... .......... 37% 12.4M 11s 2024-03-19T01:11:45.435 INFO:tasks.workunit.client.0.smithi069.stderr: 50550K .......... .......... .......... .......... .......... 37% 6.76M 11s 2024-03-19T01:11:45.437 INFO:tasks.workunit.client.0.smithi069.stderr: 50600K .......... .......... .......... .......... .......... 37% 32.1M 11s 2024-03-19T01:11:45.440 INFO:tasks.workunit.client.0.smithi069.stderr: 50650K .......... .......... .......... .......... .......... 37% 14.8M 11s 2024-03-19T01:11:45.443 INFO:tasks.workunit.client.0.smithi069.stderr: 50700K .......... .......... .......... .......... .......... 37% 12.9M 11s 2024-03-19T01:11:45.445 INFO:tasks.workunit.client.0.smithi069.stderr: 50750K .......... .......... .......... .......... .......... 37% 37.3M 11s 2024-03-19T01:11:45.449 INFO:tasks.workunit.client.0.smithi069.stderr: 50800K .......... .......... .......... .......... .......... 37% 12.0M 11s 2024-03-19T01:11:45.456 INFO:tasks.workunit.client.0.smithi069.stderr: 50850K .......... .......... .......... .......... .......... 37% 7.29M 11s 2024-03-19T01:11:45.457 INFO:tasks.workunit.client.0.smithi069.stderr: 50900K .......... .......... .......... .......... .......... 37% 30.8M 11s 2024-03-19T01:11:45.460 INFO:tasks.workunit.client.0.smithi069.stderr: 50950K .......... .......... .......... .......... .......... 37% 14.5M 11s 2024-03-19T01:11:45.464 INFO:tasks.workunit.client.0.smithi069.stderr: 51000K .......... .......... .......... .......... .......... 37% 13.8M 11s 2024-03-19T01:11:45.468 INFO:tasks.workunit.client.0.smithi069.stderr: 51050K .......... .......... .......... .......... .......... 37% 13.7M 11s 2024-03-19T01:11:45.474 INFO:tasks.workunit.client.0.smithi069.stderr: 51100K .......... .......... .......... .......... .......... 37% 6.98M 11s 2024-03-19T01:11:45.477 INFO:tasks.workunit.client.0.smithi069.stderr: 51150K .......... .......... .......... .......... .......... 37% 18.0M 11s 2024-03-19T01:11:45.480 INFO:tasks.workunit.client.0.smithi069.stderr: 51200K .......... .......... .......... .......... .......... 37% 18.3M 11s 2024-03-19T01:11:45.481 INFO:tasks.workunit.client.0.smithi069.stderr: 51250K .......... .......... .......... .......... .......... 37% 43.9M 11s 2024-03-19T01:11:45.485 INFO:tasks.workunit.client.0.smithi069.stderr: 51300K .......... .......... .......... .......... .......... 37% 11.8M 11s 2024-03-19T01:11:45.489 INFO:tasks.workunit.client.0.smithi069.stderr: 51350K .......... .......... .......... .......... .......... 37% 12.1M 11s 2024-03-19T01:11:45.496 INFO:tasks.workunit.client.0.smithi069.stderr: 51400K .......... .......... .......... .......... .......... 37% 7.00M 11s 2024-03-19T01:11:45.498 INFO:tasks.workunit.client.0.smithi069.stderr: 51450K .......... .......... .......... .......... .......... 37% 28.4M 11s 2024-03-19T01:11:45.501 INFO:tasks.workunit.client.0.smithi069.stderr: 51500K .......... .......... .......... .......... .......... 37% 16.5M 11s 2024-03-19T01:11:45.505 INFO:tasks.workunit.client.0.smithi069.stderr: 51550K .......... .......... .......... .......... .......... 38% 13.6M 11s 2024-03-19T01:11:45.508 INFO:tasks.workunit.client.0.smithi069.stderr: 51600K .......... .......... .......... .......... .......... 38% 13.5M 11s 2024-03-19T01:11:45.510 INFO:tasks.workunit.client.0.smithi069.stderr: 51650K .......... .......... .......... .......... .......... 38% 28.6M 11s 2024-03-19T01:11:45.517 INFO:tasks.workunit.client.0.smithi069.stderr: 51700K .......... .......... .......... .......... .......... 38% 6.23M 11s 2024-03-19T01:11:45.518 INFO:tasks.workunit.client.0.smithi069.stderr: 51750K .......... .......... .......... .......... .......... 38% 45.8M 11s 2024-03-19T01:11:45.522 INFO:tasks.workunit.client.0.smithi069.stderr: 51800K .......... .......... .......... .......... .......... 38% 15.7M 11s 2024-03-19T01:11:45.526 INFO:tasks.workunit.client.0.smithi069.stderr: 51850K .......... .......... .......... .......... .......... 38% 12.8M 11s 2024-03-19T01:11:45.529 INFO:tasks.workunit.client.0.smithi069.stderr: 51900K .......... .......... .......... .......... .......... 38% 13.1M 11s 2024-03-19T01:11:45.536 INFO:tasks.workunit.client.0.smithi069.stderr: 51950K .......... .......... .......... .......... .......... 38% 6.93M 11s 2024-03-19T01:11:45.539 INFO:tasks.workunit.client.0.smithi069.stderr: 52000K .......... .......... .......... .......... .......... 38% 22.2M 11s 2024-03-19T01:11:45.542 INFO:tasks.workunit.client.0.smithi069.stderr: 52050K .......... .......... .......... .......... .......... 38% 17.0M 11s 2024-03-19T01:11:45.545 INFO:tasks.workunit.client.0.smithi069.stderr: 52100K .......... .......... .......... .......... .......... 38% 13.4M 11s 2024-03-19T01:11:45.546 INFO:tasks.workunit.client.0.smithi069.stderr: 52150K .......... .......... .......... .......... .......... 38% 38.5M 11s 2024-03-19T01:11:45.551 INFO:tasks.workunit.client.0.smithi069.stderr: 52200K .......... .......... .......... .......... .......... 38% 11.8M 11s 2024-03-19T01:11:45.558 INFO:tasks.workunit.client.0.smithi069.stderr: 52250K .......... .......... .......... .......... .......... 38% 7.27M 11s 2024-03-19T01:11:45.559 INFO:tasks.workunit.client.0.smithi069.stderr: 52300K .......... .......... .......... .......... .......... 38% 23.9M 11s 2024-03-19T01:11:45.563 INFO:tasks.workunit.client.0.smithi069.stderr: 52350K .......... .......... .......... .......... .......... 38% 15.9M 10s 2024-03-19T01:11:45.566 INFO:tasks.workunit.client.0.smithi069.stderr: 52400K .......... .......... .......... .......... .......... 38% 13.9M 10s 2024-03-19T01:11:45.570 INFO:tasks.workunit.client.0.smithi069.stderr: 52450K .......... .......... .......... .......... .......... 38% 13.2M 10s 2024-03-19T01:11:45.577 INFO:tasks.workunit.client.0.smithi069.stderr: 52500K .......... .......... .......... .......... .......... 38% 6.49M 10s 2024-03-19T01:11:45.580 INFO:tasks.workunit.client.0.smithi069.stderr: 52550K .......... .......... .......... .......... .......... 38% 23.1M 10s 2024-03-19T01:11:45.583 INFO:tasks.workunit.client.0.smithi069.stderr: 52600K .......... .......... .......... .......... .......... 38% 18.3M 10s 2024-03-19T01:11:45.583 INFO:tasks.workunit.client.0.smithi069.stderr: 52650K .......... .......... .......... .......... .......... 38% 43.8M 10s 2024-03-19T01:11:45.587 INFO:tasks.workunit.client.0.smithi069.stderr: 52700K .......... .......... .......... .......... .......... 38% 12.6M 10s 2024-03-19T01:11:45.591 INFO:tasks.workunit.client.0.smithi069.stderr: 52750K .......... .......... .......... .......... .......... 38% 12.0M 10s 2024-03-19T01:11:45.598 INFO:tasks.workunit.client.0.smithi069.stderr: 52800K .......... .......... .......... .......... .......... 38% 7.01M 10s 2024-03-19T01:11:45.600 INFO:tasks.workunit.client.0.smithi069.stderr: 52850K .......... .......... .......... .......... .......... 38% 24.2M 10s 2024-03-19T01:11:45.603 INFO:tasks.workunit.client.0.smithi069.stderr: 52900K .......... .......... .......... .......... .......... 39% 16.9M 10s 2024-03-19T01:11:45.606 INFO:tasks.workunit.client.0.smithi069.stderr: 52950K .......... .......... .......... .......... .......... 39% 13.9M 10s 2024-03-19T01:11:45.610 INFO:tasks.workunit.client.0.smithi069.stderr: 53000K .......... .......... .......... .......... .......... 39% 13.1M 10s 2024-03-19T01:11:45.611 INFO:tasks.workunit.client.0.smithi069.stderr: 53050K .......... .......... .......... .......... .......... 39% 31.7M 10s 2024-03-19T01:11:45.619 INFO:tasks.workunit.client.0.smithi069.stderr: 53100K .......... .......... .......... .......... .......... 39% 6.48M 10s 2024-03-19T01:11:45.620 INFO:tasks.workunit.client.0.smithi069.stderr: 53150K .......... .......... .......... .......... .......... 39% 33.1M 10s 2024-03-19T01:11:45.624 INFO:tasks.workunit.client.0.smithi069.stderr: 53200K .......... .......... .......... .......... .......... 39% 16.4M 10s 2024-03-19T01:11:45.628 INFO:tasks.workunit.client.0.smithi069.stderr: 53250K .......... .......... .......... .......... .......... 39% 13.2M 10s 2024-03-19T01:11:45.631 INFO:tasks.workunit.client.0.smithi069.stderr: 53300K .......... .......... .......... .......... .......... 39% 13.9M 10s 2024-03-19T01:11:45.638 INFO:tasks.workunit.client.0.smithi069.stderr: 53350K .......... .......... .......... .......... .......... 39% 6.51M 10s 2024-03-19T01:11:45.641 INFO:tasks.workunit.client.0.smithi069.stderr: 53400K .......... .......... .......... .......... .......... 39% 23.5M 10s 2024-03-19T01:11:45.646 INFO:tasks.workunit.client.0.smithi069.stderr: 53450K .......... .......... .......... .......... .......... 39% 17.5M 10s 2024-03-19T01:11:45.648 INFO:tasks.workunit.client.0.smithi069.stderr: 53500K .......... .......... .......... .......... .......... 39% 12.8M 10s 2024-03-19T01:11:45.648 INFO:tasks.workunit.client.0.smithi069.stderr: 53550K .......... .......... .......... .......... .......... 39% 48.8M 10s 2024-03-19T01:11:45.652 INFO:tasks.workunit.client.0.smithi069.stderr: 53600K .......... .......... .......... .......... .......... 39% 12.1M 10s 2024-03-19T01:11:45.659 INFO:tasks.workunit.client.0.smithi069.stderr: 53650K .......... .......... .......... .......... .......... 39% 6.80M 10s 2024-03-19T01:11:45.661 INFO:tasks.workunit.client.0.smithi069.stderr: 53700K .......... .......... .......... .......... .......... 39% 24.8M 10s 2024-03-19T01:11:45.664 INFO:tasks.workunit.client.0.smithi069.stderr: 53750K .......... .......... .......... .......... .......... 39% 16.6M 10s 2024-03-19T01:11:45.668 INFO:tasks.workunit.client.0.smithi069.stderr: 53800K .......... .......... .......... .......... .......... 39% 14.2M 10s 2024-03-19T01:11:45.673 INFO:tasks.workunit.client.0.smithi069.stderr: 53850K .......... .......... .......... .......... .......... 39% 12.9M 10s 2024-03-19T01:11:45.673 INFO:tasks.workunit.client.0.smithi069.stderr: 53900K .......... .......... .......... .......... .......... 39% 34.4M 10s 2024-03-19T01:11:45.682 INFO:tasks.workunit.client.0.smithi069.stderr: 53950K .......... .......... .......... .......... .......... 39% 6.08M 10s 2024-03-19T01:11:45.683 INFO:tasks.workunit.client.0.smithi069.stderr: 54000K .......... .......... .......... .......... .......... 39% 17.7M 10s 2024-03-19T01:11:45.685 INFO:tasks.workunit.client.0.smithi069.stderr: 54050K .......... .......... .......... .......... .......... 39% 37.6M 10s 2024-03-19T01:11:45.689 INFO:tasks.workunit.client.0.smithi069.stderr: 54100K .......... .......... .......... .......... .......... 39% 13.8M 10s 2024-03-19T01:11:45.692 INFO:tasks.workunit.client.0.smithi069.stderr: 54150K .......... .......... .......... .......... .......... 39% 12.8M 10s 2024-03-19T01:11:45.700 INFO:tasks.workunit.client.0.smithi069.stderr: 54200K .......... .......... .......... .......... .......... 39% 6.56M 10s 2024-03-19T01:11:45.702 INFO:tasks.workunit.client.0.smithi069.stderr: 54250K .......... .......... .......... .......... .......... 39% 25.6M 10s 2024-03-19T01:11:45.705 INFO:tasks.workunit.client.0.smithi069.stderr: 54300K .......... .......... .......... .......... .......... 40% 17.0M 10s 2024-03-19T01:11:45.709 INFO:tasks.workunit.client.0.smithi069.stderr: 54350K .......... .......... .......... .......... .......... 40% 13.2M 10s 2024-03-19T01:11:45.709 INFO:tasks.workunit.client.0.smithi069.stderr: 54400K .......... .......... .......... .......... .......... 40% 47.6M 10s 2024-03-19T01:11:45.714 INFO:tasks.workunit.client.0.smithi069.stderr: 54450K .......... .......... .......... .......... .......... 40% 10.7M 10s 2024-03-19T01:11:45.721 INFO:tasks.workunit.client.0.smithi069.stderr: 54500K .......... .......... .......... .......... .......... 40% 6.67M 10s 2024-03-19T01:11:45.723 INFO:tasks.workunit.client.0.smithi069.stderr: 54550K .......... .......... .......... .......... .......... 40% 35.1M 10s 2024-03-19T01:11:45.725 INFO:tasks.workunit.client.0.smithi069.stderr: 54600K .......... .......... .......... .......... .......... 40% 15.9M 10s 2024-03-19T01:11:45.729 INFO:tasks.workunit.client.0.smithi069.stderr: 54650K .......... .......... .......... .......... .......... 40% 13.6M 10s 2024-03-19T01:11:45.733 INFO:tasks.workunit.client.0.smithi069.stderr: 54700K .......... .......... .......... .......... .......... 40% 12.5M 10s 2024-03-19T01:11:45.741 INFO:tasks.workunit.client.0.smithi069.stderr: 54750K .......... .......... .......... .......... .......... 40% 6.63M 10s 2024-03-19T01:11:45.742 INFO:tasks.workunit.client.0.smithi069.stderr: 54800K .......... .......... .......... .......... .......... 40% 29.2M 10s 2024-03-19T01:11:45.745 INFO:tasks.workunit.client.0.smithi069.stderr: 54850K .......... .......... .......... .......... .......... 40% 16.7M 10s 2024-03-19T01:11:45.746 INFO:tasks.workunit.client.0.smithi069.stderr: 54900K .......... .......... .......... .......... .......... 40% 34.3M 10s 2024-03-19T01:11:45.750 INFO:tasks.workunit.client.0.smithi069.stderr: 54950K .......... .......... .......... .......... .......... 40% 13.7M 10s 2024-03-19T01:11:45.754 INFO:tasks.workunit.client.0.smithi069.stderr: 55000K .......... .......... .......... .......... .......... 40% 11.5M 10s 2024-03-19T01:11:45.762 INFO:tasks.workunit.client.0.smithi069.stderr: 55050K .......... .......... .......... .......... .......... 40% 6.93M 10s 2024-03-19T01:11:45.763 INFO:tasks.workunit.client.0.smithi069.stderr: 55100K .......... .......... .......... .......... .......... 40% 29.7M 10s 2024-03-19T01:11:45.766 INFO:tasks.workunit.client.0.smithi069.stderr: 55150K .......... .......... .......... .......... .......... 40% 16.5M 10s 2024-03-19T01:11:45.769 INFO:tasks.workunit.client.0.smithi069.stderr: 55200K .......... .......... .......... .......... .......... 40% 13.8M 10s 2024-03-19T01:11:45.774 INFO:tasks.workunit.client.0.smithi069.stderr: 55250K .......... .......... .......... .......... .......... 40% 12.8M 10s 2024-03-19T01:11:45.775 INFO:tasks.workunit.client.0.smithi069.stderr: 55300K .......... .......... .......... .......... .......... 40% 24.7M 10s 2024-03-19T01:11:45.783 INFO:tasks.workunit.client.0.smithi069.stderr: 55350K .......... .......... .......... .......... .......... 40% 6.83M 10s 2024-03-19T01:11:45.786 INFO:tasks.workunit.client.0.smithi069.stderr: 55400K .......... .......... .......... .......... .......... 40% 36.3M 10s 2024-03-19T01:11:45.787 INFO:tasks.workunit.client.0.smithi069.stderr: 55450K .......... .......... .......... .......... .......... 40% 15.3M 10s 2024-03-19T01:11:45.791 INFO:tasks.workunit.client.0.smithi069.stderr: 55500K .......... .......... .......... .......... .......... 40% 14.5M 10s 2024-03-19T01:11:45.795 INFO:tasks.workunit.client.0.smithi069.stderr: 55550K .......... .......... .......... .......... .......... 40% 12.2M 10s 2024-03-19T01:11:45.802 INFO:tasks.workunit.client.0.smithi069.stderr: 55600K .......... .......... .......... .......... .......... 40% 6.35M 10s 2024-03-19T01:11:45.804 INFO:tasks.workunit.client.0.smithi069.stderr: 55650K .......... .......... .......... .......... .......... 41% 27.7M 10s 2024-03-19T01:11:45.808 INFO:tasks.workunit.client.0.smithi069.stderr: 55700K .......... .......... .......... .......... .......... 41% 18.6M 10s 2024-03-19T01:11:45.808 INFO:tasks.workunit.client.0.smithi069.stderr: 55750K .......... .......... .......... .......... .......... 41% 27.8M 10s 2024-03-19T01:11:45.811 INFO:tasks.workunit.client.0.smithi069.stderr: 55800K .......... .......... .......... .......... .......... 41% 17.3M 10s 2024-03-19T01:11:45.815 INFO:tasks.workunit.client.0.smithi069.stderr: 55850K .......... .......... .......... .......... .......... 41% 13.0M 10s 2024-03-19T01:11:45.825 INFO:tasks.workunit.client.0.smithi069.stderr: 55900K .......... .......... .......... .......... .......... 41% 4.78M 10s 2024-03-19T01:11:45.828 INFO:tasks.workunit.client.0.smithi069.stderr: 55950K .......... .......... .......... .......... .......... 41% 40.1M 10s 2024-03-19T01:11:45.828 INFO:tasks.workunit.client.0.smithi069.stderr: 56000K .......... .......... .......... .......... .......... 41% 41.2M 10s 2024-03-19T01:11:45.829 INFO:tasks.workunit.client.0.smithi069.stderr: 56050K .......... .......... .......... .......... .......... 41% 30.0M 10s 2024-03-19T01:11:45.832 INFO:tasks.workunit.client.0.smithi069.stderr: 56100K .......... .......... .......... .......... .......... 41% 18.4M 10s 2024-03-19T01:11:45.836 INFO:tasks.workunit.client.0.smithi069.stderr: 56150K .......... .......... .......... .......... .......... 41% 12.1M 10s 2024-03-19T01:11:45.846 INFO:tasks.workunit.client.0.smithi069.stderr: 56200K .......... .......... .......... .......... .......... 41% 4.70M 10s 2024-03-19T01:11:45.848 INFO:tasks.workunit.client.0.smithi069.stderr: 56250K .......... .......... .......... .......... .......... 41% 39.6M 10s 2024-03-19T01:11:45.849 INFO:tasks.workunit.client.0.smithi069.stderr: 56300K .......... .......... .......... .......... .......... 41% 35.4M 10s 2024-03-19T01:11:45.851 INFO:tasks.workunit.client.0.smithi069.stderr: 56350K .......... .......... .......... .......... .......... 41% 26.5M 10s 2024-03-19T01:11:45.855 INFO:tasks.workunit.client.0.smithi069.stderr: 56400K .......... .......... .......... .......... .......... 41% 11.3M 10s 2024-03-19T01:11:45.856 INFO:tasks.workunit.client.0.smithi069.stderr: 56450K .......... .......... .......... .......... .......... 41% 38.7M 10s 2024-03-19T01:11:45.867 INFO:tasks.workunit.client.0.smithi069.stderr: 56500K .......... .......... .......... .......... .......... 41% 4.54M 10s 2024-03-19T01:11:45.868 INFO:tasks.workunit.client.0.smithi069.stderr: 56550K .......... .......... .......... .......... .......... 41% 36.3M 10s 2024-03-19T01:11:45.870 INFO:tasks.workunit.client.0.smithi069.stderr: 56600K .......... .......... .......... .......... .......... 41% 26.2M 10s 2024-03-19T01:11:45.872 INFO:tasks.workunit.client.0.smithi069.stderr: 56650K .......... .......... .......... .......... .......... 41% 23.1M 10s 2024-03-19T01:11:45.875 INFO:tasks.workunit.client.0.smithi069.stderr: 56700K .......... .......... .......... .......... .......... 41% 16.4M 10s 2024-03-19T01:11:45.877 INFO:tasks.workunit.client.0.smithi069.stderr: 56750K .......... .......... .......... .......... .......... 41% 27.4M 10s 2024-03-19T01:11:45.888 INFO:tasks.workunit.client.0.smithi069.stderr: 56800K .......... .......... .......... .......... .......... 41% 4.71M 10s 2024-03-19T01:11:45.889 INFO:tasks.workunit.client.0.smithi069.stderr: 56850K .......... .......... .......... .......... .......... 41% 41.7M 10s 2024-03-19T01:11:45.890 INFO:tasks.workunit.client.0.smithi069.stderr: 56900K .......... .......... .......... .......... .......... 41% 22.1M 10s 2024-03-19T01:11:45.893 INFO:tasks.workunit.client.0.smithi069.stderr: 56950K .......... .......... .......... .......... .......... 41% 22.6M 10s 2024-03-19T01:11:45.895 INFO:tasks.workunit.client.0.smithi069.stderr: 57000K .......... .......... .......... .......... .......... 42% 17.0M 10s 2024-03-19T01:11:45.897 INFO:tasks.workunit.client.0.smithi069.stderr: 57050K .......... .......... .......... .......... .......... 42% 26.9M 10s 2024-03-19T01:11:45.909 INFO:tasks.workunit.client.0.smithi069.stderr: 57100K .......... .......... .......... .......... .......... 42% 4.70M 10s 2024-03-19T01:11:45.909 INFO:tasks.workunit.client.0.smithi069.stderr: 57150K .......... .......... .......... .......... .......... 42% 42.1M 10s 2024-03-19T01:11:45.911 INFO:tasks.workunit.client.0.smithi069.stderr: 57200K .......... .......... .......... .......... .......... 42% 25.6M 10s 2024-03-19T01:11:45.913 INFO:tasks.workunit.client.0.smithi069.stderr: 57250K .......... .......... .......... .......... .......... 42% 20.6M 10s 2024-03-19T01:11:45.916 INFO:tasks.workunit.client.0.smithi069.stderr: 57300K .......... .......... .......... .......... .......... 42% 19.6M 10s 2024-03-19T01:11:45.927 INFO:tasks.workunit.client.0.smithi069.stderr: 57350K .......... .......... .......... .......... .......... 42% 4.38M 10s 2024-03-19T01:11:45.929 INFO:tasks.workunit.client.0.smithi069.stderr: 57400K .......... .......... .......... .......... .......... 42% 36.7M 9s 2024-03-19T01:11:45.929 INFO:tasks.workunit.client.0.smithi069.stderr: 57450K .......... .......... .......... .......... .......... 42% 42.4M 9s 2024-03-19T01:11:45.931 INFO:tasks.workunit.client.0.smithi069.stderr: 57500K .......... .......... .......... .......... .......... 42% 24.2M 9s 2024-03-19T01:11:45.936 INFO:tasks.workunit.client.0.smithi069.stderr: 57550K .......... .......... .......... .......... .......... 42% 11.7M 9s 2024-03-19T01:11:45.938 INFO:tasks.workunit.client.0.smithi069.stderr: 57600K .......... .......... .......... .......... .......... 42% 16.0M 9s 2024-03-19T01:11:45.947 INFO:tasks.workunit.client.0.smithi069.stderr: 57650K .......... .......... .......... .......... .......... 42% 5.61M 9s 2024-03-19T01:11:45.949 INFO:tasks.workunit.client.0.smithi069.stderr: 57700K .......... .......... .......... .......... .......... 42% 36.9M 9s 2024-03-19T01:11:45.950 INFO:tasks.workunit.client.0.smithi069.stderr: 57750K .......... .......... .......... .......... .......... 42% 36.7M 9s 2024-03-19T01:11:45.953 INFO:tasks.workunit.client.0.smithi069.stderr: 57800K .......... .......... .......... .......... .......... 42% 18.9M 9s 2024-03-19T01:11:45.957 INFO:tasks.workunit.client.0.smithi069.stderr: 57850K .......... .......... .......... .......... .......... 42% 10.9M 9s 2024-03-19T01:11:45.959 INFO:tasks.workunit.client.0.smithi069.stderr: 57900K .......... .......... .......... .......... .......... 42% 24.7M 9s 2024-03-19T01:11:45.968 INFO:tasks.workunit.client.0.smithi069.stderr: 57950K .......... .......... .......... .......... .......... 42% 5.53M 9s 2024-03-19T01:11:45.969 INFO:tasks.workunit.client.0.smithi069.stderr: 58000K .......... .......... .......... .......... .......... 42% 38.2M 9s 2024-03-19T01:11:45.971 INFO:tasks.workunit.client.0.smithi069.stderr: 58050K .......... .......... .......... .......... .......... 42% 31.3M 9s 2024-03-19T01:11:45.973 INFO:tasks.workunit.client.0.smithi069.stderr: 58100K .......... .......... .......... .......... .......... 42% 21.7M 9s 2024-03-19T01:11:45.977 INFO:tasks.workunit.client.0.smithi069.stderr: 58150K .......... .......... .......... .......... .......... 42% 10.5M 9s 2024-03-19T01:11:45.980 INFO:tasks.workunit.client.0.smithi069.stderr: 58200K .......... .......... .......... .......... .......... 42% 24.0M 9s 2024-03-19T01:11:45.988 INFO:tasks.workunit.client.0.smithi069.stderr: 58250K .......... .......... .......... .......... .......... 42% 5.54M 9s 2024-03-19T01:11:45.990 INFO:tasks.workunit.client.0.smithi069.stderr: 58300K .......... .......... .......... .......... .......... 42% 37.3M 9s 2024-03-19T01:11:45.991 INFO:tasks.workunit.client.0.smithi069.stderr: 58350K .......... .......... .......... .......... .......... 43% 38.1M 9s 2024-03-19T01:11:45.993 INFO:tasks.workunit.client.0.smithi069.stderr: 58400K .......... .......... .......... .......... .......... 43% 22.7M 9s 2024-03-19T01:11:45.998 INFO:tasks.workunit.client.0.smithi069.stderr: 58450K .......... .......... .......... .......... .......... 43% 9.38M 9s 2024-03-19T01:11:46.000 INFO:tasks.workunit.client.0.smithi069.stderr: 58500K .......... .......... .......... .......... .......... 43% 31.4M 9s 2024-03-19T01:11:46.011 INFO:tasks.workunit.client.0.smithi069.stderr: 58550K .......... .......... .......... .......... .......... 43% 5.50M 9s 2024-03-19T01:11:46.011 INFO:tasks.workunit.client.0.smithi069.stderr: 58600K .......... .......... .......... .......... .......... 43% 38.5M 9s 2024-03-19T01:11:46.012 INFO:tasks.workunit.client.0.smithi069.stderr: 58650K .......... .......... .......... .......... .......... 43% 36.7M 9s 2024-03-19T01:11:46.013 INFO:tasks.workunit.client.0.smithi069.stderr: 58700K .......... .......... .......... .......... .......... 43% 22.2M 9s 2024-03-19T01:11:46.019 INFO:tasks.workunit.client.0.smithi069.stderr: 58750K .......... .......... .......... .......... .......... 43% 9.61M 9s 2024-03-19T01:11:46.021 INFO:tasks.workunit.client.0.smithi069.stderr: 58800K .......... .......... .......... .......... .......... 43% 24.8M 9s 2024-03-19T01:11:46.029 INFO:tasks.workunit.client.0.smithi069.stderr: 58850K .......... .......... .......... .......... .......... 43% 5.72M 9s 2024-03-19T01:11:46.030 INFO:tasks.workunit.client.0.smithi069.stderr: 58900K .......... .......... .......... .......... .......... 43% 38.2M 9s 2024-03-19T01:11:46.037 INFO:tasks.workunit.client.0.smithi069.stderr: 58950K .......... .......... .......... .......... .......... 43% 7.79M 9s 2024-03-19T01:11:46.039 INFO:tasks.workunit.client.0.smithi069.stderr: 59000K .......... .......... .......... .......... .......... 43% 25.1M 9s 2024-03-19T01:11:46.040 INFO:tasks.workunit.client.0.smithi069.stderr: 59050K .......... .......... .......... .......... .......... 43% 26.2M 9s 2024-03-19T01:11:46.043 INFO:tasks.workunit.client.0.smithi069.stderr: 59100K .......... .......... .......... .......... .......... 43% 31.6M 9s 2024-03-19T01:11:46.049 INFO:tasks.workunit.client.0.smithi069.stderr: 59150K .......... .......... .......... .......... .......... 43% 6.41M 9s 2024-03-19T01:11:46.050 INFO:tasks.workunit.client.0.smithi069.stderr: 59200K .......... .......... .......... .......... .......... 43% 51.8M 9s 2024-03-19T01:11:46.061 INFO:tasks.workunit.client.0.smithi069.stderr: 59250K .......... .......... .......... .......... .......... 43% 7.34M 9s 2024-03-19T01:11:46.061 INFO:tasks.workunit.client.0.smithi069.stderr: 59300K .......... .......... .......... .......... .......... 43% 23.6M 9s 2024-03-19T01:11:46.061 INFO:tasks.workunit.client.0.smithi069.stderr: 59350K .......... .......... .......... .......... .......... 43% 31.5M 9s 2024-03-19T01:11:46.062 INFO:tasks.workunit.client.0.smithi069.stderr: 59400K .......... .......... .......... .......... .......... 43% 29.8M 9s 2024-03-19T01:11:46.071 INFO:tasks.workunit.client.0.smithi069.stderr: 59450K .......... .......... .......... .......... .......... 43% 6.28M 9s 2024-03-19T01:11:46.074 INFO:tasks.workunit.client.0.smithi069.stderr: 59500K .......... .......... .......... .......... .......... 43% 14.2M 9s 2024-03-19T01:11:46.079 INFO:tasks.workunit.client.0.smithi069.stderr: 59550K .......... .......... .......... .......... .......... 43% 11.1M 9s 2024-03-19T01:11:46.080 INFO:tasks.workunit.client.0.smithi069.stderr: 59600K .......... .......... .......... .......... .......... 43% 31.9M 9s 2024-03-19T01:11:46.081 INFO:tasks.workunit.client.0.smithi069.stderr: 59650K .......... .......... .......... .......... .......... 43% 30.9M 9s 2024-03-19T01:11:46.084 INFO:tasks.workunit.client.0.smithi069.stderr: 59700K .......... .......... .......... .......... .......... 44% 32.6M 9s 2024-03-19T01:11:46.092 INFO:tasks.workunit.client.0.smithi069.stderr: 59750K .......... .......... .......... .......... .......... 44% 5.93M 9s 2024-03-19T01:11:46.094 INFO:tasks.workunit.client.0.smithi069.stderr: 59800K .......... .......... .......... .......... .......... 44% 16.0M 9s 2024-03-19T01:11:46.101 INFO:tasks.workunit.client.0.smithi069.stderr: 59850K .......... .......... .......... .......... .......... 44% 6.59M 9s 2024-03-19T01:11:46.103 INFO:tasks.workunit.client.0.smithi069.stderr: 59900K .......... .......... .......... .......... .......... 44% 23.3M 9s 2024-03-19T01:11:46.107 INFO:tasks.workunit.client.0.smithi069.stderr: 59950K .......... .......... .......... .......... .......... 44% 12.4M 9s 2024-03-19T01:11:46.111 INFO:tasks.workunit.client.0.smithi069.stderr: 60000K .......... .......... .......... .......... .......... 44% 14.4M 9s 2024-03-19T01:11:46.115 INFO:tasks.workunit.client.0.smithi069.stderr: 60050K .......... .......... .......... .......... .......... 44% 19.6M 9s 2024-03-19T01:11:46.116 INFO:tasks.workunit.client.0.smithi069.stderr: 60100K .......... .......... .......... .......... .......... 44% 30.6M 9s 2024-03-19T01:11:46.122 INFO:tasks.workunit.client.0.smithi069.stderr: 60150K .......... .......... .......... .......... .......... 44% 7.16M 9s 2024-03-19T01:11:46.126 INFO:tasks.workunit.client.0.smithi069.stderr: 60200K .......... .......... .......... .......... .......... 44% 11.6M 9s 2024-03-19T01:11:46.131 INFO:tasks.workunit.client.0.smithi069.stderr: 60250K .......... .......... .......... .......... .......... 44% 12.1M 9s 2024-03-19T01:11:46.134 INFO:tasks.workunit.client.0.smithi069.stderr: 60300K .......... .......... .......... .......... .......... 44% 12.6M 9s 2024-03-19T01:11:46.138 INFO:tasks.workunit.client.0.smithi069.stderr: 60350K .......... .......... .......... .......... .......... 44% 12.4M 9s 2024-03-19T01:11:46.141 INFO:tasks.workunit.client.0.smithi069.stderr: 60400K .......... .......... .......... .......... .......... 44% 14.4M 9s 2024-03-19T01:11:46.146 INFO:tasks.workunit.client.0.smithi069.stderr: 60450K .......... .......... .......... .......... .......... 44% 11.4M 9s 2024-03-19T01:11:46.148 INFO:tasks.workunit.client.0.smithi069.stderr: 60500K .......... .......... .......... .......... .......... 44% 21.5M 9s 2024-03-19T01:11:46.151 INFO:tasks.workunit.client.0.smithi069.stderr: 60550K .......... .......... .......... .......... .......... 44% 18.4M 9s 2024-03-19T01:11:46.155 INFO:tasks.workunit.client.0.smithi069.stderr: 60600K .......... .......... .......... .......... .......... 44% 12.7M 9s 2024-03-19T01:11:46.158 INFO:tasks.workunit.client.0.smithi069.stderr: 60650K .......... .......... .......... .......... .......... 44% 13.4M 9s 2024-03-19T01:11:46.164 INFO:tasks.workunit.client.0.smithi069.stderr: 60700K .......... .......... .......... .......... .......... 44% 8.36M 9s 2024-03-19T01:11:46.168 INFO:tasks.workunit.client.0.smithi069.stderr: 60750K .......... .......... .......... .......... .......... 44% 13.1M 9s 2024-03-19T01:11:46.172 INFO:tasks.workunit.client.0.smithi069.stderr: 60800K .......... .......... .......... .......... .......... 44% 10.1M 9s 2024-03-19T01:11:46.176 INFO:tasks.workunit.client.0.smithi069.stderr: 60850K .......... .......... .......... .......... .......... 44% 14.2M 9s 2024-03-19T01:11:46.180 INFO:tasks.workunit.client.0.smithi069.stderr: 60900K .......... .......... .......... .......... .......... 44% 12.4M 9s 2024-03-19T01:11:46.183 INFO:tasks.workunit.client.0.smithi069.stderr: 60950K .......... .......... .......... .......... .......... 44% 17.9M 9s 2024-03-19T01:11:46.187 INFO:tasks.workunit.client.0.smithi069.stderr: 61000K .......... .......... .......... .......... .......... 44% 14.5M 9s 2024-03-19T01:11:46.188 INFO:tasks.workunit.client.0.smithi069.stderr: 61050K .......... .......... .......... .......... .......... 45% 23.9M 9s 2024-03-19T01:11:46.196 INFO:tasks.workunit.client.0.smithi069.stderr: 61100K .......... .......... .......... .......... .......... 45% 6.03M 9s 2024-03-19T01:11:46.202 INFO:tasks.workunit.client.0.smithi069.stderr: 61150K .......... .......... .......... .......... .......... 45% 8.41M 9s 2024-03-19T01:11:46.205 INFO:tasks.workunit.client.0.smithi069.stderr: 61200K .......... .......... .......... .......... .......... 45% 12.6M 9s 2024-03-19T01:11:46.208 INFO:tasks.workunit.client.0.smithi069.stderr: 61250K .......... .......... .......... .......... .......... 45% 16.7M 9s 2024-03-19T01:11:46.212 INFO:tasks.workunit.client.0.smithi069.stderr: 61300K .......... .......... .......... .......... .......... 45% 14.0M 9s 2024-03-19T01:11:46.215 INFO:tasks.workunit.client.0.smithi069.stderr: 61350K .......... .......... .......... .......... .......... 45% 16.8M 9s 2024-03-19T01:11:46.220 INFO:tasks.workunit.client.0.smithi069.stderr: 61400K .......... .......... .......... .......... .......... 45% 10.2M 9s 2024-03-19T01:11:46.223 INFO:tasks.workunit.client.0.smithi069.stderr: 61450K .......... .......... .......... .......... .......... 45% 16.3M 9s 2024-03-19T01:11:46.227 INFO:tasks.workunit.client.0.smithi069.stderr: 61500K .......... .......... .......... .......... .......... 45% 16.1M 9s 2024-03-19T01:11:46.233 INFO:tasks.workunit.client.0.smithi069.stderr: 61550K .......... .......... .......... .......... .......... 45% 6.29M 9s 2024-03-19T01:11:46.237 INFO:tasks.workunit.client.0.smithi069.stderr: 61600K .......... .......... .......... .......... .......... 45% 13.9M 9s 2024-03-19T01:11:46.241 INFO:tasks.workunit.client.0.smithi069.stderr: 61650K .......... .......... .......... .......... .......... 45% 12.7M 9s 2024-03-19T01:11:46.245 INFO:tasks.workunit.client.0.smithi069.stderr: 61700K .......... .......... .......... .......... .......... 45% 12.4M 9s 2024-03-19T01:11:46.248 INFO:tasks.workunit.client.0.smithi069.stderr: 61750K .......... .......... .......... .......... .......... 45% 13.3M 9s 2024-03-19T01:11:46.252 INFO:tasks.workunit.client.0.smithi069.stderr: 61800K .......... .......... .......... .......... .......... 45% 12.3M 9s 2024-03-19T01:11:46.255 INFO:tasks.workunit.client.0.smithi069.stderr: 61850K .......... .......... .......... .......... .......... 45% 15.2M 9s 2024-03-19T01:11:46.264 INFO:tasks.workunit.client.0.smithi069.stderr: 61900K .......... .......... .......... .......... .......... 45% 6.38M 9s 2024-03-19T01:11:46.267 INFO:tasks.workunit.client.0.smithi069.stderr: 61950K .......... .......... .......... .......... .......... 45% 13.3M 9s 2024-03-19T01:11:46.272 INFO:tasks.workunit.client.0.smithi069.stderr: 62000K .......... .......... .......... .......... .......... 45% 10.7M 9s 2024-03-19T01:11:46.275 INFO:tasks.workunit.client.0.smithi069.stderr: 62050K .......... .......... .......... .......... .......... 45% 13.3M 9s 2024-03-19T01:11:46.279 INFO:tasks.workunit.client.0.smithi069.stderr: 62100K .......... .......... .......... .......... .......... 45% 11.4M 9s 2024-03-19T01:11:46.283 INFO:tasks.workunit.client.0.smithi069.stderr: 62150K .......... .......... .......... .......... .......... 45% 15.7M 9s 2024-03-19T01:11:46.289 INFO:tasks.workunit.client.0.smithi069.stderr: 62200K .......... .......... .......... .......... .......... 45% 7.68M 9s 2024-03-19T01:11:46.292 INFO:tasks.workunit.client.0.smithi069.stderr: 62250K .......... .......... .......... .......... .......... 45% 16.8M 9s 2024-03-19T01:11:46.295 INFO:tasks.workunit.client.0.smithi069.stderr: 62300K .......... .......... .......... .......... .......... 45% 17.9M 9s 2024-03-19T01:11:46.300 INFO:tasks.workunit.client.0.smithi069.stderr: 62350K .......... .......... .......... .......... .......... 45% 9.91M 9s 2024-03-19T01:11:46.304 INFO:tasks.workunit.client.0.smithi069.stderr: 62400K .......... .......... .......... .......... .......... 46% 10.7M 9s 2024-03-19T01:11:46.309 INFO:tasks.workunit.client.0.smithi069.stderr: 62450K .......... .......... .......... .......... .......... 46% 10.4M 9s 2024-03-19T01:11:46.314 INFO:tasks.workunit.client.0.smithi069.stderr: 62500K .......... .......... .......... .......... .......... 46% 20.1M 9s 2024-03-19T01:11:46.317 INFO:tasks.workunit.client.0.smithi069.stderr: 62550K .......... .......... .......... .......... .......... 46% 8.36M 9s 2024-03-19T01:11:46.320 INFO:tasks.workunit.client.0.smithi069.stderr: 62600K .......... .......... .......... .......... .......... 46% 14.4M 9s 2024-03-19T01:11:46.325 INFO:tasks.workunit.client.0.smithi069.stderr: 62650K .......... .......... .......... .......... .......... 46% 11.0M 9s 2024-03-19T01:11:46.330 INFO:tasks.workunit.client.0.smithi069.stderr: 62700K .......... .......... .......... .......... .......... 46% 9.84M 9s 2024-03-19T01:11:46.335 INFO:tasks.workunit.client.0.smithi069.stderr: 62750K .......... .......... .......... .......... .......... 46% 8.82M 9s 2024-03-19T01:11:46.339 INFO:tasks.workunit.client.0.smithi069.stderr: 62800K .......... .......... .......... .......... .......... 46% 11.4M 9s 2024-03-19T01:11:46.345 INFO:tasks.workunit.client.0.smithi069.stderr: 62850K .......... .......... .......... .......... .......... 46% 10.9M 9s 2024-03-19T01:11:46.350 INFO:tasks.workunit.client.0.smithi069.stderr: 62900K .......... .......... .......... .......... .......... 46% 9.95M 9s 2024-03-19T01:11:46.353 INFO:tasks.workunit.client.0.smithi069.stderr: 62950K .......... .......... .......... .......... .......... 46% 12.6M 9s 2024-03-19T01:11:46.358 INFO:tasks.workunit.client.0.smithi069.stderr: 63000K .......... .......... .......... .......... .......... 46% 9.99M 9s 2024-03-19T01:11:46.364 INFO:tasks.workunit.client.0.smithi069.stderr: 63050K .......... .......... .......... .......... .......... 46% 7.57M 9s 2024-03-19T01:11:46.368 INFO:tasks.workunit.client.0.smithi069.stderr: 63100K .......... .......... .......... .......... .......... 46% 13.8M 9s 2024-03-19T01:11:46.372 INFO:tasks.workunit.client.0.smithi069.stderr: 63150K .......... .......... .......... .......... .......... 46% 11.4M 9s 2024-03-19T01:11:46.377 INFO:tasks.workunit.client.0.smithi069.stderr: 63200K .......... .......... .......... .......... .......... 46% 9.63M 8s 2024-03-19T01:11:46.381 INFO:tasks.workunit.client.0.smithi069.stderr: 63250K .......... .......... .......... .......... .......... 46% 11.9M 8s 2024-03-19T01:11:46.386 INFO:tasks.workunit.client.0.smithi069.stderr: 63300K .......... .......... .......... .......... .......... 46% 10.9M 8s 2024-03-19T01:11:46.391 INFO:tasks.workunit.client.0.smithi069.stderr: 63350K .......... .......... .......... .......... .......... 46% 9.91M 8s 2024-03-19T01:11:46.396 INFO:tasks.workunit.client.0.smithi069.stderr: 63400K .......... .......... .......... .......... .......... 46% 9.38M 8s 2024-03-19T01:11:46.401 INFO:tasks.workunit.client.0.smithi069.stderr: 63450K .......... .......... .......... .......... .......... 46% 9.58M 8s 2024-03-19T01:11:46.405 INFO:tasks.workunit.client.0.smithi069.stderr: 63500K .......... .......... .......... .......... .......... 46% 11.1M 8s 2024-03-19T01:11:46.410 INFO:tasks.workunit.client.0.smithi069.stderr: 63550K .......... .......... .......... .......... .......... 46% 10.7M 8s 2024-03-19T01:11:46.414 INFO:tasks.workunit.client.0.smithi069.stderr: 63600K .......... .......... .......... .......... .......... 46% 12.3M 8s 2024-03-19T01:11:46.419 INFO:tasks.workunit.client.0.smithi069.stderr: 63650K .......... .......... .......... .......... .......... 46% 9.97M 8s 2024-03-19T01:11:46.424 INFO:tasks.workunit.client.0.smithi069.stderr: 63700K .......... .......... .......... .......... .......... 46% 10.4M 8s 2024-03-19T01:11:46.429 INFO:tasks.workunit.client.0.smithi069.stderr: 63750K .......... .......... .......... .......... .......... 46% 9.53M 8s 2024-03-19T01:11:46.435 INFO:tasks.workunit.client.0.smithi069.stderr: 63800K .......... .......... .......... .......... .......... 47% 8.29M 8s 2024-03-19T01:11:46.440 INFO:tasks.workunit.client.0.smithi069.stderr: 63850K .......... .......... .......... .......... .......... 47% 9.87M 8s 2024-03-19T01:11:46.443 INFO:tasks.workunit.client.0.smithi069.stderr: 63900K .......... .......... .......... .......... .......... 47% 14.0M 8s 2024-03-19T01:11:46.447 INFO:tasks.workunit.client.0.smithi069.stderr: 63950K .......... .......... .......... .......... .......... 47% 10.9M 8s 2024-03-19T01:11:46.452 INFO:tasks.workunit.client.0.smithi069.stderr: 64000K .......... .......... .......... .......... .......... 47% 10.3M 8s 2024-03-19T01:11:46.457 INFO:tasks.workunit.client.0.smithi069.stderr: 64050K .......... .......... .......... .......... .......... 47% 10.6M 8s 2024-03-19T01:11:46.462 INFO:tasks.workunit.client.0.smithi069.stderr: 64100K .......... .......... .......... .......... .......... 47% 9.32M 8s 2024-03-19T01:11:46.468 INFO:tasks.workunit.client.0.smithi069.stderr: 64150K .......... .......... .......... .......... .......... 47% 10.7M 8s 2024-03-19T01:11:46.473 INFO:tasks.workunit.client.0.smithi069.stderr: 64200K .......... .......... .......... .......... .......... 47% 8.35M 8s 2024-03-19T01:11:46.478 INFO:tasks.workunit.client.0.smithi069.stderr: 64250K .......... .......... .......... .......... .......... 47% 8.95M 8s 2024-03-19T01:11:46.483 INFO:tasks.workunit.client.0.smithi069.stderr: 64300K .......... .......... .......... .......... .......... 47% 10.0M 8s 2024-03-19T01:11:46.488 INFO:tasks.workunit.client.0.smithi069.stderr: 64350K .......... .......... .......... .......... .......... 47% 10.1M 8s 2024-03-19T01:11:46.492 INFO:tasks.workunit.client.0.smithi069.stderr: 64400K .......... .......... .......... .......... .......... 47% 11.3M 8s 2024-03-19T01:11:46.497 INFO:tasks.workunit.client.0.smithi069.stderr: 64450K .......... .......... .......... .......... .......... 47% 9.16M 8s 2024-03-19T01:11:46.503 INFO:tasks.workunit.client.0.smithi069.stderr: 64500K .......... .......... .......... .......... .......... 47% 9.22M 8s 2024-03-19T01:11:46.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:46 smithi086 ceph-mon[66829]: pgmap v36: 129 pgs: 129 active+clean; 27 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.1 MiB/s wr, 13 op/s 2024-03-19T01:11:46.507 INFO:tasks.workunit.client.0.smithi069.stderr: 64550K .......... .......... .......... .......... .......... 47% 10.5M 8s 2024-03-19T01:11:46.513 INFO:tasks.workunit.client.0.smithi069.stderr: 64600K .......... .......... .......... .......... .......... 47% 8.41M 8s 2024-03-19T01:11:46.518 INFO:tasks.workunit.client.0.smithi069.stderr: 64650K .......... .......... .......... .......... .......... 47% 9.71M 8s 2024-03-19T01:11:46.522 INFO:tasks.workunit.client.0.smithi069.stderr: 64700K .......... .......... .......... .......... .......... 47% 14.2M 8s 2024-03-19T01:11:46.527 INFO:tasks.workunit.client.0.smithi069.stderr: 64750K .......... .......... .......... .......... .......... 47% 8.35M 8s 2024-03-19T01:11:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:46 smithi069 ceph-mon[31727]: pgmap v36: 129 pgs: 129 active+clean; 27 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.1 MiB/s wr, 13 op/s 2024-03-19T01:11:46.533 INFO:tasks.workunit.client.0.smithi069.stderr: 64800K .......... .......... .......... .......... .......... 47% 9.49M 8s 2024-03-19T01:11:46.538 INFO:tasks.workunit.client.0.smithi069.stderr: 64850K .......... .......... .......... .......... .......... 47% 9.03M 8s 2024-03-19T01:11:46.543 INFO:tasks.workunit.client.0.smithi069.stderr: 64900K .......... .......... .......... .......... .......... 47% 9.44M 8s 2024-03-19T01:11:46.547 INFO:tasks.workunit.client.0.smithi069.stderr: 64950K .......... .......... .......... .......... .......... 47% 11.1M 8s 2024-03-19T01:11:46.552 INFO:tasks.workunit.client.0.smithi069.stderr: 65000K .......... .......... .......... .......... .......... 47% 9.86M 8s 2024-03-19T01:11:46.558 INFO:tasks.workunit.client.0.smithi069.stderr: 65050K .......... .......... .......... .......... .......... 47% 8.58M 8s 2024-03-19T01:11:46.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:46 smithi163 ceph-mon[67428]: pgmap v36: 129 pgs: 129 active+clean; 27 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.1 MiB/s wr, 13 op/s 2024-03-19T01:11:46.563 INFO:tasks.workunit.client.0.smithi069.stderr: 65100K .......... .......... .......... .......... .......... 47% 9.86M 8s 2024-03-19T01:11:46.568 INFO:tasks.workunit.client.0.smithi069.stderr: 65150K .......... .......... .......... .......... .......... 48% 10.4M 8s 2024-03-19T01:11:46.573 INFO:tasks.workunit.client.0.smithi069.stderr: 65200K .......... .......... .......... .......... .......... 48% 9.87M 8s 2024-03-19T01:11:46.578 INFO:tasks.workunit.client.0.smithi069.stderr: 65250K .......... .......... .......... .......... .......... 48% 9.68M 8s 2024-03-19T01:11:46.582 INFO:tasks.workunit.client.0.smithi069.stderr: 65300K .......... .......... .......... .......... .......... 48% 10.8M 8s 2024-03-19T01:11:46.589 INFO:tasks.workunit.client.0.smithi069.stderr: 65350K .......... .......... .......... .......... .......... 48% 7.99M 8s 2024-03-19T01:11:46.594 INFO:tasks.workunit.client.0.smithi069.stderr: 65400K .......... .......... .......... .......... .......... 48% 9.21M 8s 2024-03-19T01:11:46.598 INFO:tasks.workunit.client.0.smithi069.stderr: 65450K .......... .......... .......... .......... .......... 48% 13.2M 8s 2024-03-19T01:11:46.604 INFO:tasks.workunit.client.0.smithi069.stderr: 65500K .......... .......... .......... .......... .......... 48% 8.80M 8s 2024-03-19T01:11:46.608 INFO:tasks.workunit.client.0.smithi069.stderr: 65550K .......... .......... .......... .......... .......... 48% 10.4M 8s 2024-03-19T01:11:46.613 INFO:tasks.workunit.client.0.smithi069.stderr: 65600K .......... .......... .......... .......... .......... 48% 9.99M 8s 2024-03-19T01:11:46.618 INFO:tasks.workunit.client.0.smithi069.stderr: 65650K .......... .......... .......... .......... .......... 48% 8.63M 8s 2024-03-19T01:11:46.625 INFO:tasks.workunit.client.0.smithi069.stderr: 65700K .......... .......... .......... .......... .......... 48% 7.38M 8s 2024-03-19T01:11:46.629 INFO:tasks.workunit.client.0.smithi069.stderr: 65750K .......... .......... .......... .......... .......... 48% 12.6M 8s 2024-03-19T01:11:46.634 INFO:tasks.workunit.client.0.smithi069.stderr: 65800K .......... .......... .......... .......... .......... 48% 9.47M 8s 2024-03-19T01:11:46.640 INFO:tasks.workunit.client.0.smithi069.stderr: 65850K .......... .......... .......... .......... .......... 48% 7.30M 8s 2024-03-19T01:11:46.646 INFO:tasks.workunit.client.0.smithi069.stderr: 65900K .......... .......... .......... .......... .......... 48% 9.48M 8s 2024-03-19T01:11:46.652 INFO:tasks.workunit.client.0.smithi069.stderr: 65950K .......... .......... .......... .......... .......... 48% 8.48M 8s 2024-03-19T01:11:46.657 INFO:tasks.workunit.client.0.smithi069.stderr: 66000K .......... .......... .......... .......... .......... 48% 10.9M 8s 2024-03-19T01:11:46.662 INFO:tasks.workunit.client.0.smithi069.stderr: 66050K .......... .......... .......... .......... .......... 48% 8.60M 8s 2024-03-19T01:11:46.667 INFO:tasks.workunit.client.0.smithi069.stderr: 66100K .......... .......... .......... .......... .......... 48% 8.47M 8s 2024-03-19T01:11:46.674 INFO:tasks.workunit.client.0.smithi069.stderr: 66150K .......... .......... .......... .......... .......... 48% 7.65M 8s 2024-03-19T01:11:46.680 INFO:tasks.workunit.client.0.smithi069.stderr: 66200K .......... .......... .......... .......... .......... 48% 9.09M 8s 2024-03-19T01:11:46.685 INFO:tasks.workunit.client.0.smithi069.stderr: 66250K .......... .......... .......... .......... .......... 48% 9.20M 8s 2024-03-19T01:11:46.689 INFO:tasks.workunit.client.0.smithi069.stderr: 66300K .......... .......... .......... .......... .......... 48% 9.73M 8s 2024-03-19T01:11:46.695 INFO:tasks.workunit.client.0.smithi069.stderr: 66350K .......... .......... .......... .......... .......... 48% 9.48M 8s 2024-03-19T01:11:46.701 INFO:tasks.workunit.client.0.smithi069.stderr: 66400K .......... .......... .......... .......... .......... 48% 7.58M 8s 2024-03-19T01:11:46.708 INFO:tasks.workunit.client.0.smithi069.stderr: 66450K .......... .......... .......... .......... .......... 48% 7.66M 8s 2024-03-19T01:11:46.713 INFO:tasks.workunit.client.0.smithi069.stderr: 66500K .......... .......... .......... .......... .......... 49% 9.36M 8s 2024-03-19T01:11:46.719 INFO:tasks.workunit.client.0.smithi069.stderr: 66550K .......... .......... .......... .......... .......... 49% 8.55M 8s 2024-03-19T01:11:46.724 INFO:tasks.workunit.client.0.smithi069.stderr: 66600K .......... .......... .......... .......... .......... 49% 8.00M 8s 2024-03-19T01:11:46.729 INFO:tasks.workunit.client.0.smithi069.stderr: 66650K .......... .......... .......... .......... .......... 49% 10.6M 8s 2024-03-19T01:11:46.734 INFO:tasks.workunit.client.0.smithi069.stderr: 66700K .......... .......... .......... .......... .......... 49% 9.68M 8s 2024-03-19T01:11:46.741 INFO:tasks.workunit.client.0.smithi069.stderr: 66750K .......... .......... .......... .......... .......... 49% 6.91M 8s 2024-03-19T01:11:46.747 INFO:tasks.workunit.client.0.smithi069.stderr: 66800K .......... .......... .......... .......... .......... 49% 7.97M 8s 2024-03-19T01:11:46.753 INFO:tasks.workunit.client.0.smithi069.stderr: 66850K .......... .......... .......... .......... .......... 49% 10.3M 8s 2024-03-19T01:11:46.759 INFO:tasks.workunit.client.0.smithi069.stderr: 66900K .......... .......... .......... .......... .......... 49% 7.44M 8s 2024-03-19T01:11:46.764 INFO:tasks.workunit.client.0.smithi069.stderr: 66950K .......... .......... .......... .......... .......... 49% 8.88M 8s 2024-03-19T01:11:46.769 INFO:tasks.workunit.client.0.smithi069.stderr: 67000K .......... .......... .......... .......... .......... 49% 10.0M 8s 2024-03-19T01:11:46.776 INFO:tasks.workunit.client.0.smithi069.stderr: 67050K .......... .......... .......... .......... .......... 49% 6.90M 8s 2024-03-19T01:11:46.782 INFO:tasks.workunit.client.0.smithi069.stderr: 67100K .......... .......... .......... .......... .......... 49% 8.81M 8s 2024-03-19T01:11:46.788 INFO:tasks.workunit.client.0.smithi069.stderr: 67150K .......... .......... .......... .......... .......... 49% 7.43M 8s 2024-03-19T01:11:46.795 INFO:tasks.workunit.client.0.smithi069.stderr: 67200K .......... .......... .......... .......... .......... 49% 7.20M 8s 2024-03-19T01:11:46.801 INFO:tasks.workunit.client.0.smithi069.stderr: 67250K .......... .......... .......... .......... .......... 49% 7.80M 8s 2024-03-19T01:11:46.806 INFO:tasks.workunit.client.0.smithi069.stderr: 67300K .......... .......... .......... .......... .......... 49% 9.80M 8s 2024-03-19T01:11:46.812 INFO:tasks.workunit.client.0.smithi069.stderr: 67350K .......... .......... .......... .......... .......... 49% 8.84M 8s 2024-03-19T01:11:46.818 INFO:tasks.workunit.client.0.smithi069.stderr: 67400K .......... .......... .......... .......... .......... 49% 7.92M 8s 2024-03-19T01:11:46.826 INFO:tasks.workunit.client.0.smithi069.stderr: 67450K .......... .......... .......... .......... .......... 49% 6.34M 8s 2024-03-19T01:11:46.832 INFO:tasks.workunit.client.0.smithi069.stderr: 67500K .......... .......... .......... .......... .......... 49% 9.59M 8s 2024-03-19T01:11:46.837 INFO:tasks.workunit.client.0.smithi069.stderr: 67550K .......... .......... .......... .......... .......... 49% 7.29M 8s 2024-03-19T01:11:46.843 INFO:tasks.workunit.client.0.smithi069.stderr: 67600K .......... .......... .......... .......... .......... 49% 8.86M 8s 2024-03-19T01:11:46.847 INFO:tasks.workunit.client.0.smithi069.stderr: 67650K .......... .......... .......... .......... .......... 49% 11.0M 8s 2024-03-19T01:11:46.853 INFO:tasks.workunit.client.0.smithi069.stderr: 67700K .......... .......... .......... .......... .......... 49% 8.06M 8s 2024-03-19T01:11:46.864 INFO:tasks.workunit.client.0.smithi069.stderr: 67750K .......... .......... .......... .......... .......... 49% 4.71M 8s 2024-03-19T01:11:46.870 INFO:tasks.workunit.client.0.smithi069.stderr: 67800K .......... .......... .......... .......... .......... 49% 9.13M 8s 2024-03-19T01:11:46.878 INFO:tasks.workunit.client.0.smithi069.stderr: 67850K .......... .......... .......... .......... .......... 50% 7.31M 8s 2024-03-19T01:11:46.883 INFO:tasks.workunit.client.0.smithi069.stderr: 67900K .......... .......... .......... .......... .......... 50% 7.14M 8s 2024-03-19T01:11:46.888 INFO:tasks.workunit.client.0.smithi069.stderr: 67950K .......... .......... .......... .......... .......... 50% 8.71M 8s 2024-03-19T01:11:46.892 INFO:tasks.workunit.client.0.smithi069.stderr: 68000K .......... .......... .......... .......... .......... 50% 12.4M 8s 2024-03-19T01:11:46.902 INFO:tasks.workunit.client.0.smithi069.stderr: 68050K .......... .......... .......... .......... .......... 50% 4.74M 8s 2024-03-19T01:11:46.907 INFO:tasks.workunit.client.0.smithi069.stderr: 68100K .......... .......... .......... .......... .......... 50% 10.9M 8s 2024-03-19T01:11:46.915 INFO:tasks.workunit.client.0.smithi069.stderr: 68150K .......... .......... .......... .......... .......... 50% 6.78M 8s 2024-03-19T01:11:46.922 INFO:tasks.workunit.client.0.smithi069.stderr: 68200K .......... .......... .......... .......... .......... 50% 5.94M 8s 2024-03-19T01:11:46.928 INFO:tasks.workunit.client.0.smithi069.stderr: 68250K .......... .......... .......... .......... .......... 50% 9.75M 8s 2024-03-19T01:11:46.933 INFO:tasks.workunit.client.0.smithi069.stderr: 68300K .......... .......... .......... .......... .......... 50% 9.33M 8s 2024-03-19T01:11:46.938 INFO:tasks.workunit.client.0.smithi069.stderr: 68350K .......... .......... .......... .......... .......... 50% 8.11M 8s 2024-03-19T01:11:46.953 INFO:tasks.workunit.client.0.smithi069.stderr: 68400K .......... .......... .......... .......... .......... 50% 3.37M 8s 2024-03-19T01:11:46.958 INFO:tasks.workunit.client.0.smithi069.stderr: 68450K .......... .......... .......... .......... .......... 50% 9.45M 8s 2024-03-19T01:11:46.967 INFO:tasks.workunit.client.0.smithi069.stderr: 68500K .......... .......... .......... .......... .......... 50% 5.58M 8s 2024-03-19T01:11:46.973 INFO:tasks.workunit.client.0.smithi069.stderr: 68550K .......... .......... .......... .......... .......... 50% 7.86M 8s 2024-03-19T01:11:46.979 INFO:tasks.workunit.client.0.smithi069.stderr: 68600K .......... .......... .......... .......... .......... 50% 8.55M 8s 2024-03-19T01:11:46.985 INFO:tasks.workunit.client.0.smithi069.stderr: 68650K .......... .......... .......... .......... .......... 50% 7.72M 8s 2024-03-19T01:11:46.996 INFO:tasks.workunit.client.0.smithi069.stderr: 68700K .......... .......... .......... .......... .......... 50% 4.67M 8s 2024-03-19T01:11:47.003 INFO:tasks.workunit.client.0.smithi069.stderr: 68750K .......... .......... .......... .......... .......... 50% 6.48M 8s 2024-03-19T01:11:47.012 INFO:tasks.workunit.client.0.smithi069.stderr: 68800K .......... .......... .......... .......... .......... 50% 5.52M 8s 2024-03-19T01:11:47.018 INFO:tasks.workunit.client.0.smithi069.stderr: 68850K .......... .......... .......... .......... .......... 50% 8.09M 8s 2024-03-19T01:11:47.024 INFO:tasks.workunit.client.0.smithi069.stderr: 68900K .......... .......... .......... .......... .......... 50% 8.57M 8s 2024-03-19T01:11:47.029 INFO:tasks.workunit.client.0.smithi069.stderr: 68950K .......... .......... .......... .......... .......... 50% 10.2M 8s 2024-03-19T01:11:47.036 INFO:tasks.workunit.client.0.smithi069.stderr: 69000K .......... .......... .......... .......... .......... 50% 6.34M 8s 2024-03-19T01:11:47.047 INFO:tasks.workunit.client.0.smithi069.stderr: 69050K .......... .......... .......... .......... .......... 50% 4.74M 8s 2024-03-19T01:11:47.055 INFO:tasks.workunit.client.0.smithi069.stderr: 69100K .......... .......... .......... .......... .......... 50% 5.96M 8s 2024-03-19T01:11:47.063 INFO:tasks.workunit.client.0.smithi069.stderr: 69150K .......... .......... .......... .......... .......... 50% 6.20M 8s 2024-03-19T01:11:47.069 INFO:tasks.workunit.client.0.smithi069.stderr: 69200K .......... .......... .......... .......... .......... 51% 7.96M 8s 2024-03-19T01:11:47.078 INFO:tasks.workunit.client.0.smithi069.stderr: 69250K .......... .......... .......... .......... .......... 51% 7.22M 8s 2024-03-19T01:11:47.081 INFO:tasks.workunit.client.0.smithi069.stderr: 69300K .......... .......... .......... .......... .......... 51% 9.42M 8s 2024-03-19T01:11:47.090 INFO:tasks.workunit.client.0.smithi069.stderr: 69350K .......... .......... .......... .......... .......... 51% 5.31M 8s 2024-03-19T01:11:47.099 INFO:tasks.workunit.client.0.smithi069.stderr: 69400K .......... .......... .......... .......... .......... 51% 5.56M 8s 2024-03-19T01:11:47.108 INFO:tasks.workunit.client.0.smithi069.stderr: 69450K .......... .......... .......... .......... .......... 51% 5.17M 8s 2024-03-19T01:11:47.113 INFO:tasks.workunit.client.0.smithi069.stderr: 69500K .......... .......... .......... .......... .......... 51% 10.9M 8s 2024-03-19T01:11:47.120 INFO:tasks.workunit.client.0.smithi069.stderr: 69550K .......... .......... .......... .......... .......... 51% 6.92M 8s 2024-03-19T01:11:47.126 INFO:tasks.workunit.client.0.smithi069.stderr: 69600K .......... .......... .......... .......... .......... 51% 7.88M 8s 2024-03-19T01:11:47.134 INFO:tasks.workunit.client.0.smithi069.stderr: 69650K .......... .......... .......... .......... .......... 51% 6.03M 8s 2024-03-19T01:11:47.145 INFO:tasks.workunit.client.0.smithi069.stderr: 69700K .......... .......... .......... .......... .......... 51% 5.23M 8s 2024-03-19T01:11:47.153 INFO:tasks.workunit.client.0.smithi069.stderr: 69750K .......... .......... .......... .......... .......... 51% 5.22M 8s 2024-03-19T01:11:47.159 INFO:tasks.workunit.client.0.smithi069.stderr: 69800K .......... .......... .......... .......... .......... 51% 7.31M 8s 2024-03-19T01:11:47.166 INFO:tasks.workunit.client.0.smithi069.stderr: 69850K .......... .......... .......... .......... .......... 51% 7.63M 8s 2024-03-19T01:11:47.171 INFO:tasks.workunit.client.0.smithi069.stderr: 69900K .......... .......... .......... .......... .......... 51% 9.08M 8s 2024-03-19T01:11:47.179 INFO:tasks.workunit.client.0.smithi069.stderr: 69950K .......... .......... .......... .......... .......... 51% 7.23M 8s 2024-03-19T01:11:47.189 INFO:tasks.workunit.client.0.smithi069.stderr: 70000K .......... .......... .......... .......... .......... 51% 4.52M 8s 2024-03-19T01:11:47.197 INFO:tasks.workunit.client.0.smithi069.stderr: 70050K .......... .......... .......... .......... .......... 51% 6.10M 8s 2024-03-19T01:11:47.206 INFO:tasks.workunit.client.0.smithi069.stderr: 70100K .......... .......... .......... .......... .......... 51% 5.19M 8s 2024-03-19T01:11:47.210 INFO:tasks.workunit.client.0.smithi069.stderr: 70150K .......... .......... .......... .......... .......... 51% 11.7M 8s 2024-03-19T01:11:47.218 INFO:tasks.workunit.client.0.smithi069.stderr: 70200K .......... .......... .......... .......... .......... 51% 6.18M 8s 2024-03-19T01:11:47.225 INFO:tasks.workunit.client.0.smithi069.stderr: 70250K .......... .......... .......... .......... .......... 51% 7.29M 8s 2024-03-19T01:11:47.234 INFO:tasks.workunit.client.0.smithi069.stderr: 70300K .......... .......... .......... .......... .......... 51% 5.16M 8s 2024-03-19T01:11:47.245 INFO:tasks.workunit.client.0.smithi069.stderr: 70350K .......... .......... .......... .......... .......... 51% 4.80M 8s 2024-03-19T01:11:47.253 INFO:tasks.workunit.client.0.smithi069.stderr: 70400K .......... .......... .......... .......... .......... 51% 5.80M 8s 2024-03-19T01:11:47.261 INFO:tasks.workunit.client.0.smithi069.stderr: 70450K .......... .......... .......... .......... .......... 51% 5.74M 8s 2024-03-19T01:11:47.269 INFO:tasks.workunit.client.0.smithi069.stderr: 70500K .......... .......... .......... .......... .......... 51% 6.64M 8s 2024-03-19T01:11:47.275 INFO:tasks.workunit.client.0.smithi069.stderr: 70550K .......... .......... .......... .......... .......... 52% 7.62M 8s 2024-03-19T01:11:47.284 INFO:tasks.workunit.client.0.smithi069.stderr: 70600K .......... .......... .......... .......... .......... 52% 6.01M 8s 2024-03-19T01:11:47.296 INFO:tasks.workunit.client.0.smithi069.stderr: 70650K .......... .......... .......... .......... .......... 52% 6.08M 8s 2024-03-19T01:11:47.304 INFO:tasks.workunit.client.0.smithi069.stderr: 70700K .......... .......... .......... .......... .......... 52% 3.72M 8s 2024-03-19T01:11:47.311 INFO:tasks.workunit.client.0.smithi069.stderr: 70750K .......... .......... .......... .......... .......... 52% 7.12M 8s 2024-03-19T01:11:47.320 INFO:tasks.workunit.client.0.smithi069.stderr: 70800K .......... .......... .......... .......... .......... 52% 5.96M 8s 2024-03-19T01:11:47.327 INFO:tasks.workunit.client.0.smithi069.stderr: 70850K .......... .......... .......... .......... .......... 52% 6.62M 8s 2024-03-19T01:11:47.341 INFO:tasks.workunit.client.0.smithi069.stderr: 70900K .......... .......... .......... .......... .......... 52% 4.10M 8s 2024-03-19T01:11:47.350 INFO:tasks.workunit.client.0.smithi069.stderr: 70950K .......... .......... .......... .......... .......... 52% 4.50M 8s 2024-03-19T01:11:47.361 INFO:tasks.workunit.client.0.smithi069.stderr: 71000K .......... .......... .......... .......... .......... 52% 4.30M 8s 2024-03-19T01:11:47.370 INFO:tasks.workunit.client.0.smithi069.stderr: 71050K .......... .......... .......... .......... .......... 52% 5.46M 8s 2024-03-19T01:11:47.378 INFO:tasks.workunit.client.0.smithi069.stderr: 71100K .......... .......... .......... .......... .......... 52% 6.19M 8s 2024-03-19T01:11:47.386 INFO:tasks.workunit.client.0.smithi069.stderr: 71150K .......... .......... .......... .......... .......... 52% 6.06M 8s 2024-03-19T01:11:47.393 INFO:tasks.workunit.client.0.smithi069.stderr: 71200K .......... .......... .......... .......... .......... 52% 6.57M 8s 2024-03-19T01:11:47.407 INFO:tasks.workunit.client.0.smithi069.stderr: 71250K .......... .......... .......... .......... .......... 52% 3.80M 8s 2024-03-19T01:11:47.415 INFO:tasks.workunit.client.0.smithi069.stderr: 71300K .......... .......... .......... .......... .......... 52% 6.30M 8s 2024-03-19T01:11:47.426 INFO:tasks.workunit.client.0.smithi069.stderr: 71350K .......... .......... .......... .......... .......... 52% 3.91M 8s 2024-03-19T01:11:47.434 INFO:tasks.workunit.client.0.smithi069.stderr: 71400K .......... .......... .......... .......... .......... 52% 6.62M 8s 2024-03-19T01:11:47.442 INFO:tasks.workunit.client.0.smithi069.stderr: 71450K .......... .......... .......... .......... .......... 52% 5.88M 8s 2024-03-19T01:11:47.448 INFO:tasks.workunit.client.0.smithi069.stderr: 71500K .......... .......... .......... .......... .......... 52% 8.84M 8s 2024-03-19T01:11:47.462 INFO:tasks.workunit.client.0.smithi069.stderr: 71550K .......... .......... .......... .......... .......... 52% 3.32M 8s 2024-03-19T01:11:47.472 INFO:tasks.workunit.client.0.smithi069.stderr: 71600K .......... .......... .......... .......... .......... 52% 5.10M 8s 2024-03-19T01:11:47.486 INFO:tasks.workunit.client.0.smithi069.stderr: 71650K .......... .......... .......... .......... .......... 52% 4.47M 8s 2024-03-19T01:11:47.491 INFO:tasks.workunit.client.0.smithi069.stderr: 71700K .......... .......... .......... .......... .......... 52% 6.01M 8s 2024-03-19T01:11:47.499 INFO:tasks.workunit.client.0.smithi069.stderr: 71750K .......... .......... .......... .......... .......... 52% 6.21M 8s 2024-03-19T01:11:47.508 INFO:tasks.workunit.client.0.smithi069.stderr: 71800K .......... .......... .......... .......... .......... 52% 5.12M 8s 2024-03-19T01:11:47.515 INFO:tasks.workunit.client.0.smithi069.stderr: 71850K .......... .......... .......... .......... .......... 52% 7.29M 8s 2024-03-19T01:11:47.530 INFO:tasks.workunit.client.0.smithi069.stderr: 71900K .......... .......... .......... .......... .......... 53% 3.18M 8s 2024-03-19T01:11:47.540 INFO:tasks.workunit.client.0.smithi069.stderr: 71950K .......... .......... .......... .......... .......... 53% 4.92M 8s 2024-03-19T01:11:47.551 INFO:tasks.workunit.client.0.smithi069.stderr: 72000K .......... .......... .......... .......... .......... 53% 4.53M 8s 2024-03-19T01:11:47.558 INFO:tasks.workunit.client.0.smithi069.stderr: 72050K .......... .......... .......... .......... .......... 53% 6.83M 8s 2024-03-19T01:11:47.569 INFO:tasks.workunit.client.0.smithi069.stderr: 72100K .......... .......... .......... .......... .......... 53% 4.73M 8s 2024-03-19T01:11:47.578 INFO:tasks.workunit.client.0.smithi069.stderr: 72150K .......... .......... .......... .......... .......... 53% 5.44M 8s 2024-03-19T01:11:47.589 INFO:tasks.workunit.client.0.smithi069.stderr: 72200K .......... .......... .......... .......... .......... 53% 4.19M 8s 2024-03-19T01:11:47.601 INFO:tasks.workunit.client.0.smithi069.stderr: 72250K .......... .......... .......... .......... .......... 53% 4.15M 8s 2024-03-19T01:11:47.612 INFO:tasks.workunit.client.0.smithi069.stderr: 72300K .......... .......... .......... .......... .......... 53% 4.30M 8s 2024-03-19T01:11:47.623 INFO:tasks.workunit.client.0.smithi069.stderr: 72350K .......... .......... .......... .......... .......... 53% 4.75M 8s 2024-03-19T01:11:47.632 INFO:tasks.workunit.client.0.smithi069.stderr: 72400K .......... .......... .......... .......... .......... 53% 5.97M 8s 2024-03-19T01:11:47.641 INFO:tasks.workunit.client.0.smithi069.stderr: 72450K .......... .......... .......... .......... .......... 53% 4.87M 8s 2024-03-19T01:11:47.648 INFO:tasks.workunit.client.0.smithi069.stderr: 72500K .......... .......... .......... .......... .......... 53% 6.49M 8s 2024-03-19T01:11:47.665 INFO:tasks.workunit.client.0.smithi069.stderr: 72550K .......... .......... .......... .......... .......... 53% 2.92M 8s 2024-03-19T01:11:47.674 INFO:tasks.workunit.client.0.smithi069.stderr: 72600K .......... .......... .......... .......... .......... 53% 5.32M 8s 2024-03-19T01:11:47.687 INFO:tasks.workunit.client.0.smithi069.stderr: 72650K .......... .......... .......... .......... .......... 53% 5.00M 8s 2024-03-19T01:11:47.694 INFO:tasks.workunit.client.0.smithi069.stderr: 72700K .......... .......... .......... .......... .......... 53% 4.70M 8s 2024-03-19T01:11:47.707 INFO:tasks.workunit.client.0.smithi069.stderr: 72750K .......... .......... .......... .......... .......... 53% 4.91M 8s 2024-03-19T01:11:47.717 INFO:tasks.workunit.client.0.smithi069.stderr: 72800K .......... .......... .......... .......... .......... 53% 3.76M 8s 2024-03-19T01:11:47.726 INFO:tasks.workunit.client.0.smithi069.stderr: 72850K .......... .......... .......... .......... .......... 53% 5.78M 8s 2024-03-19T01:11:47.742 INFO:tasks.workunit.client.0.smithi069.stderr: 72900K .......... .......... .......... .......... .......... 53% 3.04M 8s 2024-03-19T01:11:47.750 INFO:tasks.workunit.client.0.smithi069.stderr: 72950K .......... .......... .......... .......... .......... 53% 5.63M 8s 2024-03-19T01:11:47.763 INFO:tasks.workunit.client.0.smithi069.stderr: 73000K .......... .......... .......... .......... .......... 53% 4.34M 8s 2024-03-19T01:11:47.771 INFO:tasks.workunit.client.0.smithi069.stderr: 73050K .......... .......... .......... .......... .......... 53% 5.19M 8s 2024-03-19T01:11:47.786 INFO:tasks.workunit.client.0.smithi069.stderr: 73100K .......... .......... .......... .......... .......... 53% 3.30M 8s 2024-03-19T01:11:47.795 INFO:tasks.workunit.client.0.smithi069.stderr: 73150K .......... .......... .......... .......... .......... 53% 5.37M 8s 2024-03-19T01:11:47.808 INFO:tasks.workunit.client.0.smithi069.stderr: 73200K .......... .......... .......... .......... .......... 53% 5.30M 8s 2024-03-19T01:11:47.824 INFO:tasks.workunit.client.0.smithi069.stderr: 73250K .......... .......... .......... .......... .......... 53% 2.65M 8s 2024-03-19T01:11:47.830 INFO:tasks.workunit.client.0.smithi069.stderr: 73300K .......... .......... .......... .......... .......... 54% 7.19M 8s 2024-03-19T01:11:47.842 INFO:tasks.workunit.client.0.smithi069.stderr: 73350K .......... .......... .......... .......... .......... 54% 3.80M 8s 2024-03-19T01:11:47.852 INFO:tasks.workunit.client.0.smithi069.stderr: 73400K .......... .......... .......... .......... .......... 54% 4.96M 8s 2024-03-19T01:11:47.868 INFO:tasks.workunit.client.0.smithi069.stderr: 73450K .......... .......... .......... .......... .......... 54% 3.44M 8s 2024-03-19T01:11:47.878 INFO:tasks.workunit.client.0.smithi069.stderr: 73500K .......... .......... .......... .......... .......... 54% 4.30M 8s 2024-03-19T01:11:47.971 INFO:tasks.workunit.client.0.smithi069.stderr: 73550K .......... .......... .......... .......... .......... 54% 551K 8s 2024-03-19T01:11:47.971 INFO:tasks.workunit.client.0.smithi069.stderr: 73600K .......... .......... .......... .......... .......... 54% 72.5M 8s 2024-03-19T01:11:47.972 INFO:tasks.workunit.client.0.smithi069.stderr: 73650K .......... .......... .......... .......... .......... 54% 78.4M 8s 2024-03-19T01:11:48.008 INFO:tasks.workunit.client.0.smithi069.stderr: 73700K .......... .......... .......... .......... .......... 54% 1.29M 8s 2024-03-19T01:11:48.012 INFO:tasks.workunit.client.0.smithi069.stderr: 73750K .......... .......... .......... .......... .......... 54% 17.3M 8s 2024-03-19T01:11:48.012 INFO:tasks.workunit.client.0.smithi069.stderr: 73800K .......... .......... .......... .......... .......... 54% 72.2M 8s 2024-03-19T01:11:48.039 INFO:tasks.workunit.client.0.smithi069.stderr: 73850K .......... .......... .......... .......... .......... 54% 1.78M 8s 2024-03-19T01:11:48.042 INFO:tasks.workunit.client.0.smithi069.stderr: 73900K .......... .......... .......... .......... .......... 54% 77.5M 8s 2024-03-19T01:11:48.042 INFO:tasks.workunit.client.0.smithi069.stderr: 73950K .......... .......... .......... .......... .......... 54% 64.1M 8s 2024-03-19T01:11:48.042 INFO:tasks.workunit.client.0.smithi069.stderr: 74000K .......... .......... .......... .......... .......... 54% 66.6M 8s 2024-03-19T01:11:48.043 INFO:tasks.workunit.client.0.smithi069.stderr: 74050K .......... .......... .......... .......... .......... 54% 64.3M 8s 2024-03-19T01:11:48.051 INFO:tasks.workunit.client.0.smithi069.stderr: 74100K .......... .......... .......... .......... .......... 54% 5.73M 8s 2024-03-19T01:11:48.067 INFO:tasks.workunit.client.0.smithi069.stderr: 74150K .......... .......... .......... .......... .......... 54% 2.97M 8s 2024-03-19T01:11:48.080 INFO:tasks.workunit.client.0.smithi069.stderr: 74200K .......... .......... .......... .......... .......... 54% 3.73M 8s 2024-03-19T01:11:48.102 INFO:tasks.workunit.client.0.smithi069.stderr: 74250K .......... .......... .......... .......... .......... 54% 2.39M 8s 2024-03-19T01:11:48.116 INFO:tasks.workunit.client.0.smithi069.stderr: 74300K .......... .......... .......... .......... .......... 54% 3.09M 8s 2024-03-19T01:11:48.133 INFO:tasks.workunit.client.0.smithi069.stderr: 74350K .......... .......... .......... .......... .......... 54% 2.92M 8s 2024-03-19T01:11:48.141 INFO:tasks.workunit.client.0.smithi069.stderr: 74400K .......... .......... .......... .......... .......... 54% 5.53M 8s 2024-03-19T01:11:48.155 INFO:tasks.workunit.client.0.smithi069.stderr: 74450K .......... .......... .......... .......... .......... 54% 4.08M 8s 2024-03-19T01:11:48.173 INFO:tasks.workunit.client.0.smithi069.stderr: 74500K .......... .......... .......... .......... .......... 54% 2.47M 8s 2024-03-19T01:11:48.182 INFO:tasks.workunit.client.0.smithi069.stderr: 74550K .......... .......... .......... .......... .......... 54% 5.75M 8s 2024-03-19T01:11:48.190 INFO:tasks.workunit.client.0.smithi069.stderr: 74600K .......... .......... .......... .......... .......... 54% 6.13M 8s 2024-03-19T01:11:48.194 INFO:tasks.workunit.client.0.smithi069.stderr: 74650K .......... .......... .......... .......... .......... 55% 10.2M 8s 2024-03-19T01:11:48.199 INFO:tasks.workunit.client.0.smithi069.stderr: 74700K .......... .......... .......... .......... .......... 55% 9.88M 8s 2024-03-19T01:11:48.206 INFO:tasks.workunit.client.0.smithi069.stderr: 74750K .......... .......... .......... .......... .......... 55% 6.70M 8s 2024-03-19T01:11:48.212 INFO:tasks.workunit.client.0.smithi069.stderr: 74800K .......... .......... .......... .......... .......... 55% 9.31M 8s 2024-03-19T01:11:48.217 INFO:tasks.workunit.client.0.smithi069.stderr: 74850K .......... .......... .......... .......... .......... 55% 11.0M 8s 2024-03-19T01:11:48.220 INFO:tasks.workunit.client.0.smithi069.stderr: 74900K .......... .......... .......... .......... .......... 55% 15.1M 8s 2024-03-19T01:11:48.222 INFO:tasks.workunit.client.0.smithi069.stderr: 74950K .......... .......... .......... .......... .......... 55% 17.1M 7s 2024-03-19T01:11:48.228 INFO:tasks.workunit.client.0.smithi069.stderr: 75000K .......... .......... .......... .......... .......... 55% 8.96M 7s 2024-03-19T01:11:48.234 INFO:tasks.workunit.client.0.smithi069.stderr: 75050K .......... .......... .......... .......... .......... 55% 8.38M 7s 2024-03-19T01:11:48.238 INFO:tasks.workunit.client.0.smithi069.stderr: 75100K .......... .......... .......... .......... .......... 55% 12.3M 7s 2024-03-19T01:11:48.241 INFO:tasks.workunit.client.0.smithi069.stderr: 75150K .......... .......... .......... .......... .......... 55% 17.6M 7s 2024-03-19T01:11:48.244 INFO:tasks.workunit.client.0.smithi069.stderr: 75200K .......... .......... .......... .......... .......... 55% 13.6M 7s 2024-03-19T01:11:48.250 INFO:tasks.workunit.client.0.smithi069.stderr: 75250K .......... .......... .......... .......... .......... 55% 8.25M 7s 2024-03-19T01:11:48.255 INFO:tasks.workunit.client.0.smithi069.stderr: 75300K .......... .......... .......... .......... .......... 55% 9.59M 7s 2024-03-19T01:11:48.258 INFO:tasks.workunit.client.0.smithi069.stderr: 75350K .......... .......... .......... .......... .......... 55% 14.4M 7s 2024-03-19T01:11:48.261 INFO:tasks.workunit.client.0.smithi069.stderr: 75400K .......... .......... .......... .......... .......... 55% 16.3M 7s 2024-03-19T01:11:48.266 INFO:tasks.workunit.client.0.smithi069.stderr: 75450K .......... .......... .......... .......... .......... 55% 10.3M 7s 2024-03-19T01:11:48.273 INFO:tasks.workunit.client.0.smithi069.stderr: 75500K .......... .......... .......... .......... .......... 55% 7.78M 7s 2024-03-19T01:11:48.277 INFO:tasks.workunit.client.0.smithi069.stderr: 75550K .......... .......... .......... .......... .......... 55% 12.9M 7s 2024-03-19T01:11:48.280 INFO:tasks.workunit.client.0.smithi069.stderr: 75600K .......... .......... .......... .......... .......... 55% 14.4M 7s 2024-03-19T01:11:48.282 INFO:tasks.workunit.client.0.smithi069.stderr: 75650K .......... .......... .......... .......... .......... 55% 18.2M 7s 2024-03-19T01:11:48.288 INFO:tasks.workunit.client.0.smithi069.stderr: 75700K .......... .......... .......... .......... .......... 55% 9.43M 7s 2024-03-19T01:11:48.293 INFO:tasks.workunit.client.0.smithi069.stderr: 75750K .......... .......... .......... .......... .......... 55% 8.81M 7s 2024-03-19T01:11:48.297 INFO:tasks.workunit.client.0.smithi069.stderr: 75800K .......... .......... .......... .......... .......... 55% 12.5M 7s 2024-03-19T01:11:48.300 INFO:tasks.workunit.client.0.smithi069.stderr: 75850K .......... .......... .......... .......... .......... 55% 16.2M 7s 2024-03-19T01:11:48.303 INFO:tasks.workunit.client.0.smithi069.stderr: 75900K .......... .......... .......... .......... .......... 55% 16.8M 7s 2024-03-19T01:11:48.310 INFO:tasks.workunit.client.0.smithi069.stderr: 75950K .......... .......... .......... .......... .......... 55% 9.76M 7s 2024-03-19T01:11:48.313 INFO:tasks.workunit.client.0.smithi069.stderr: 76000K .......... .......... .......... .......... .......... 56% 8.62M 7s 2024-03-19T01:11:48.317 INFO:tasks.workunit.client.0.smithi069.stderr: 76050K .......... .......... .......... .......... .......... 56% 13.1M 7s 2024-03-19T01:11:48.322 INFO:tasks.workunit.client.0.smithi069.stderr: 76100K .......... .......... .......... .......... .......... 56% 12.9M 7s 2024-03-19T01:11:48.324 INFO:tasks.workunit.client.0.smithi069.stderr: 76150K .......... .......... .......... .......... .......... 56% 21.7M 7s 2024-03-19T01:11:48.329 INFO:tasks.workunit.client.0.smithi069.stderr: 76200K .......... .......... .......... .......... .......... 56% 9.79M 7s 2024-03-19T01:11:48.334 INFO:tasks.workunit.client.0.smithi069.stderr: 76250K .......... .......... .......... .......... .......... 56% 9.23M 7s 2024-03-19T01:11:48.337 INFO:tasks.workunit.client.0.smithi069.stderr: 76300K .......... .......... .......... .......... .......... 56% 12.3M 7s 2024-03-19T01:11:48.342 INFO:tasks.workunit.client.0.smithi069.stderr: 76350K .......... .......... .......... .......... .......... 56% 12.5M 7s 2024-03-19T01:11:48.344 INFO:tasks.workunit.client.0.smithi069.stderr: 76400K .......... .......... .......... .......... .......... 56% 23.2M 7s 2024-03-19T01:11:48.349 INFO:tasks.workunit.client.0.smithi069.stderr: 76450K .......... .......... .......... .......... .......... 56% 9.67M 7s 2024-03-19T01:11:48.354 INFO:tasks.workunit.client.0.smithi069.stderr: 76500K .......... .......... .......... .......... .......... 56% 9.19M 7s 2024-03-19T01:11:48.358 INFO:tasks.workunit.client.0.smithi069.stderr: 76550K .......... .......... .......... .......... .......... 56% 12.1M 7s 2024-03-19T01:11:48.363 INFO:tasks.workunit.client.0.smithi069.stderr: 76600K .......... .......... .......... .......... .......... 56% 11.6M 7s 2024-03-19T01:11:48.364 INFO:tasks.workunit.client.0.smithi069.stderr: 76650K .......... .......... .......... .......... .......... 56% 27.6M 7s 2024-03-19T01:11:48.369 INFO:tasks.workunit.client.0.smithi069.stderr: 76700K .......... .......... .......... .......... .......... 56% 9.85M 7s 2024-03-19T01:11:48.375 INFO:tasks.workunit.client.0.smithi069.stderr: 76750K .......... .......... .......... .......... .......... 56% 9.12M 7s 2024-03-19T01:11:48.378 INFO:tasks.workunit.client.0.smithi069.stderr: 76800K .......... .......... .......... .......... .......... 56% 12.3M 7s 2024-03-19T01:11:48.383 INFO:tasks.workunit.client.0.smithi069.stderr: 76850K .......... .......... .......... .......... .......... 56% 12.5M 7s 2024-03-19T01:11:48.384 INFO:tasks.workunit.client.0.smithi069.stderr: 76900K .......... .......... .......... .......... .......... 56% 22.5M 7s 2024-03-19T01:11:48.389 INFO:tasks.workunit.client.0.smithi069.stderr: 76950K .......... .......... .......... .......... .......... 56% 9.51M 7s 2024-03-19T01:11:48.395 INFO:tasks.workunit.client.0.smithi069.stderr: 77000K .......... .......... .......... .......... .......... 56% 9.58M 7s 2024-03-19T01:11:48.399 INFO:tasks.workunit.client.0.smithi069.stderr: 77050K .......... .......... .......... .......... .......... 56% 12.4M 7s 2024-03-19T01:11:48.403 INFO:tasks.workunit.client.0.smithi069.stderr: 77100K .......... .......... .......... .......... .......... 56% 11.8M 7s 2024-03-19T01:11:48.405 INFO:tasks.workunit.client.0.smithi069.stderr: 77150K .......... .......... .......... .......... .......... 56% 24.6M 7s 2024-03-19T01:11:48.410 INFO:tasks.workunit.client.0.smithi069.stderr: 77200K .......... .......... .......... .......... .......... 56% 9.45M 7s 2024-03-19T01:11:48.415 INFO:tasks.workunit.client.0.smithi069.stderr: 77250K .......... .......... .......... .......... .......... 56% 9.62M 7s 2024-03-19T01:11:48.419 INFO:tasks.workunit.client.0.smithi069.stderr: 77300K .......... .......... .......... .......... .......... 56% 12.3M 7s 2024-03-19T01:11:48.423 INFO:tasks.workunit.client.0.smithi069.stderr: 77350K .......... .......... .......... .......... .......... 57% 12.2M 7s 2024-03-19T01:11:48.425 INFO:tasks.workunit.client.0.smithi069.stderr: 77400K .......... .......... .......... .......... .......... 57% 21.7M 7s 2024-03-19T01:11:48.430 INFO:tasks.workunit.client.0.smithi069.stderr: 77450K .......... .......... .......... .......... .......... 57% 9.25M 7s 2024-03-19T01:11:48.435 INFO:tasks.workunit.client.0.smithi069.stderr: 77500K .......... .......... .......... .......... .......... 57% 9.90M 7s 2024-03-19T01:11:48.439 INFO:tasks.workunit.client.0.smithi069.stderr: 77550K .......... .......... .......... .......... .......... 57% 12.8M 7s 2024-03-19T01:11:48.443 INFO:tasks.workunit.client.0.smithi069.stderr: 77600K .......... .......... .......... .......... .......... 57% 12.6M 7s 2024-03-19T01:11:48.445 INFO:tasks.workunit.client.0.smithi069.stderr: 77650K .......... .......... .......... .......... .......... 57% 20.1M 7s 2024-03-19T01:11:48.450 INFO:tasks.workunit.client.0.smithi069.stderr: 77700K .......... .......... .......... .......... .......... 57% 9.32M 7s 2024-03-19T01:11:48.456 INFO:tasks.workunit.client.0.smithi069.stderr: 77750K .......... .......... .......... .......... .......... 57% 9.85M 7s 2024-03-19T01:11:48.459 INFO:tasks.workunit.client.0.smithi069.stderr: 77800K .......... .......... .......... .......... .......... 57% 12.9M 7s 2024-03-19T01:11:48.464 INFO:tasks.workunit.client.0.smithi069.stderr: 77850K .......... .......... .......... .......... .......... 57% 12.1M 7s 2024-03-19T01:11:48.465 INFO:tasks.workunit.client.0.smithi069.stderr: 77900K .......... .......... .......... .......... .......... 57% 23.1M 7s 2024-03-19T01:11:48.471 INFO:tasks.workunit.client.0.smithi069.stderr: 77950K .......... .......... .......... .......... .......... 57% 9.03M 7s 2024-03-19T01:11:48.476 INFO:tasks.workunit.client.0.smithi069.stderr: 78000K .......... .......... .......... .......... .......... 57% 9.54M 7s 2024-03-19T01:11:48.482 INFO:tasks.workunit.client.0.smithi069.stderr: 78050K .......... .......... .......... .......... .......... 57% 13.3M 7s 2024-03-19T01:11:48.484 INFO:tasks.workunit.client.0.smithi069.stderr: 78100K .......... .......... .......... .......... .......... 57% 12.5M 7s 2024-03-19T01:11:48.486 INFO:tasks.workunit.client.0.smithi069.stderr: 78150K .......... .......... .......... .......... .......... 57% 22.6M 7s 2024-03-19T01:11:48.491 INFO:tasks.workunit.client.0.smithi069.stderr: 78200K .......... .......... .......... .......... .......... 57% 9.12M 7s 2024-03-19T01:11:48.497 INFO:tasks.workunit.client.0.smithi069.stderr: 78250K .......... .......... .......... .......... .......... 57% 9.21M 7s 2024-03-19T01:11:48.500 INFO:tasks.workunit.client.0.smithi069.stderr: 78300K .......... .......... .......... .......... .......... 57% 13.3M 7s 2024-03-19T01:11:48.504 INFO:tasks.workunit.client.0.smithi069.stderr: 78350K .......... .......... .......... .......... .......... 57% 13.1M 7s 2024-03-19T01:11:48.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:48 smithi086 ceph-mon[66829]: pgmap v37: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.1 MiB/s wr, 12 op/s 2024-03-19T01:11:48.506 INFO:tasks.workunit.client.0.smithi069.stderr: 78400K .......... .......... .......... .......... .......... 57% 22.3M 7s 2024-03-19T01:11:48.512 INFO:tasks.workunit.client.0.smithi069.stderr: 78450K .......... .......... .......... .......... .......... 57% 12.0M 7s 2024-03-19T01:11:48.518 INFO:tasks.workunit.client.0.smithi069.stderr: 78500K .......... .......... .......... .......... .......... 57% 7.57M 7s 2024-03-19T01:11:48.521 INFO:tasks.workunit.client.0.smithi069.stderr: 78550K .......... .......... .......... .......... .......... 57% 13.1M 7s 2024-03-19T01:11:48.527 INFO:tasks.workunit.client.0.smithi069.stderr: 78600K .......... .......... .......... .......... .......... 57% 11.6M 7s 2024-03-19T01:11:48.527 INFO:tasks.workunit.client.0.smithi069.stderr: 78650K .......... .......... .......... .......... .......... 57% 27.1M 7s 2024-03-19T01:11:48.530 INFO:tasks.workunit.client.0.smithi069.stderr: 78700K .......... .......... .......... .......... .......... 58% 12.2M 7s 2024-03-19T01:11:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:48 smithi069 ceph-mon[31727]: pgmap v37: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.1 MiB/s wr, 12 op/s 2024-03-19T01:11:48.538 INFO:tasks.workunit.client.0.smithi069.stderr: 78750K .......... .......... .......... .......... .......... 58% 6.24M 7s 2024-03-19T01:11:48.541 INFO:tasks.workunit.client.0.smithi069.stderr: 78800K .......... .......... .......... .......... .......... 58% 20.6M 7s 2024-03-19T01:11:48.544 INFO:tasks.workunit.client.0.smithi069.stderr: 78850K .......... .......... .......... .......... .......... 58% 12.3M 7s 2024-03-19T01:11:48.546 INFO:tasks.workunit.client.0.smithi069.stderr: 78900K .......... .......... .......... .......... .......... 58% 23.8M 7s 2024-03-19T01:11:48.552 INFO:tasks.workunit.client.0.smithi069.stderr: 78950K .......... .......... .......... .......... .......... 58% 12.5M 7s 2024-03-19T01:11:48.557 INFO:tasks.workunit.client.0.smithi069.stderr: 79000K .......... .......... .......... .......... .......... 58% 7.27M 7s 2024-03-19T01:11:48.561 INFO:tasks.workunit.client.0.smithi069.stderr: 79050K .......... .......... .......... .......... .......... 58% 13.6M 7s 2024-03-19T01:11:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:48 smithi163 ceph-mon[67428]: pgmap v37: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.1 MiB/s wr, 12 op/s 2024-03-19T01:11:48.565 INFO:tasks.workunit.client.0.smithi069.stderr: 79100K .......... .......... .......... .......... .......... 58% 12.7M 7s 2024-03-19T01:11:48.567 INFO:tasks.workunit.client.0.smithi069.stderr: 79150K .......... .......... .......... .......... .......... 58% 21.9M 7s 2024-03-19T01:11:48.571 INFO:tasks.workunit.client.0.smithi069.stderr: 79200K .......... .......... .......... .......... .......... 58% 12.6M 7s 2024-03-19T01:11:48.577 INFO:tasks.workunit.client.0.smithi069.stderr: 79250K .......... .......... .......... .......... .......... 58% 7.33M 7s 2024-03-19T01:11:48.581 INFO:tasks.workunit.client.0.smithi069.stderr: 79300K .......... .......... .......... .......... .......... 58% 13.3M 7s 2024-03-19T01:11:48.585 INFO:tasks.workunit.client.0.smithi069.stderr: 79350K .......... .......... .......... .......... .......... 58% 12.8M 7s 2024-03-19T01:11:48.587 INFO:tasks.workunit.client.0.smithi069.stderr: 79400K .......... .......... .......... .......... .......... 58% 21.6M 7s 2024-03-19T01:11:48.593 INFO:tasks.workunit.client.0.smithi069.stderr: 79450K .......... .......... .......... .......... .......... 58% 12.5M 7s 2024-03-19T01:11:48.597 INFO:tasks.workunit.client.0.smithi069.stderr: 79500K .......... .......... .......... .......... .......... 58% 7.38M 7s 2024-03-19T01:11:48.601 INFO:tasks.workunit.client.0.smithi069.stderr: 79550K .......... .......... .......... .......... .......... 58% 13.5M 7s 2024-03-19T01:11:48.606 INFO:tasks.workunit.client.0.smithi069.stderr: 79600K .......... .......... .......... .......... .......... 58% 12.6M 7s 2024-03-19T01:11:48.607 INFO:tasks.workunit.client.0.smithi069.stderr: 79650K .......... .......... .......... .......... .......... 58% 22.6M 7s 2024-03-19T01:11:48.613 INFO:tasks.workunit.client.0.smithi069.stderr: 79700K .......... .......... .......... .......... .......... 58% 12.1M 7s 2024-03-19T01:11:48.618 INFO:tasks.workunit.client.0.smithi069.stderr: 79750K .......... .......... .......... .......... .......... 58% 7.27M 7s 2024-03-19T01:11:48.622 INFO:tasks.workunit.client.0.smithi069.stderr: 79800K .......... .......... .......... .......... .......... 58% 14.0M 7s 2024-03-19T01:11:48.625 INFO:tasks.workunit.client.0.smithi069.stderr: 79850K .......... .......... .......... .......... .......... 58% 12.6M 7s 2024-03-19T01:11:48.627 INFO:tasks.workunit.client.0.smithi069.stderr: 79900K .......... .......... .......... .......... .......... 58% 23.0M 7s 2024-03-19T01:11:48.631 INFO:tasks.workunit.client.0.smithi069.stderr: 79950K .......... .......... .......... .......... .......... 58% 12.4M 7s 2024-03-19T01:11:48.636 INFO:tasks.workunit.client.0.smithi069.stderr: 80000K .......... .......... .......... .......... .......... 58% 10.4M 7s 2024-03-19T01:11:48.640 INFO:tasks.workunit.client.0.smithi069.stderr: 80050K .......... .......... .......... .......... .......... 59% 13.4M 7s 2024-03-19T01:11:48.646 INFO:tasks.workunit.client.0.smithi069.stderr: 80100K .......... .......... .......... .......... .......... 59% 8.44M 7s 2024-03-19T01:11:48.646 INFO:tasks.workunit.client.0.smithi069.stderr: 80150K .......... .......... .......... .......... .......... 59% 113M 7s 2024-03-19T01:11:48.651 INFO:tasks.workunit.client.0.smithi069.stderr: 80200K .......... .......... .......... .......... .......... 59% 10.4M 7s 2024-03-19T01:11:48.657 INFO:tasks.workunit.client.0.smithi069.stderr: 80250K .......... .......... .......... .......... .......... 59% 7.97M 7s 2024-03-19T01:11:48.660 INFO:tasks.workunit.client.0.smithi069.stderr: 80300K .......... .......... .......... .......... .......... 59% 24.9M 7s 2024-03-19T01:11:48.667 INFO:tasks.workunit.client.0.smithi069.stderr: 80350K .......... .......... .......... .......... .......... 59% 7.22M 7s 2024-03-19T01:11:48.667 INFO:tasks.workunit.client.0.smithi069.stderr: 80400K .......... .......... .......... .......... .......... 59% 83.6M 7s 2024-03-19T01:11:48.671 INFO:tasks.workunit.client.0.smithi069.stderr: 80450K .......... .......... .......... .......... .......... 59% 10.5M 7s 2024-03-19T01:11:48.675 INFO:tasks.workunit.client.0.smithi069.stderr: 80500K .......... .......... .......... .......... .......... 59% 11.7M 7s 2024-03-19T01:11:48.679 INFO:tasks.workunit.client.0.smithi069.stderr: 80550K .......... .......... .......... .......... .......... 59% 11.9M 7s 2024-03-19T01:11:48.683 INFO:tasks.workunit.client.0.smithi069.stderr: 80600K .......... .......... .......... .......... .......... 59% 14.2M 7s 2024-03-19T01:11:48.688 INFO:tasks.workunit.client.0.smithi069.stderr: 80650K .......... .......... .......... .......... .......... 59% 11.7M 7s 2024-03-19T01:11:48.690 INFO:tasks.workunit.client.0.smithi069.stderr: 80700K .......... .......... .......... .......... .......... 59% 14.1M 7s 2024-03-19T01:11:48.694 INFO:tasks.workunit.client.0.smithi069.stderr: 80750K .......... .......... .......... .......... .......... 59% 12.2M 7s 2024-03-19T01:11:48.700 INFO:tasks.workunit.client.0.smithi069.stderr: 80800K .......... .......... .......... .......... .......... 59% 9.24M 7s 2024-03-19T01:11:48.702 INFO:tasks.workunit.client.0.smithi069.stderr: 80850K .......... .......... .......... .......... .......... 59% 21.8M 7s 2024-03-19T01:11:48.708 INFO:tasks.workunit.client.0.smithi069.stderr: 80900K .......... .......... .......... .......... .......... 59% 8.87M 7s 2024-03-19T01:11:48.708 INFO:tasks.workunit.client.0.smithi069.stderr: 80950K .......... .......... .......... .......... .......... 59% 43.3M 7s 2024-03-19T01:11:48.713 INFO:tasks.workunit.client.0.smithi069.stderr: 81000K .......... .......... .......... .......... .......... 59% 11.2M 7s 2024-03-19T01:11:48.718 INFO:tasks.workunit.client.0.smithi069.stderr: 81050K .......... .......... .......... .......... .......... 59% 9.57M 7s 2024-03-19T01:11:48.723 INFO:tasks.workunit.client.0.smithi069.stderr: 81100K .......... .......... .......... .......... .......... 59% 11.8M 7s 2024-03-19T01:11:48.729 INFO:tasks.workunit.client.0.smithi069.stderr: 81150K .......... .......... .......... .......... .......... 59% 8.08M 7s 2024-03-19T01:11:48.730 INFO:tasks.workunit.client.0.smithi069.stderr: 81200K .......... .......... .......... .......... .......... 59% 85.9M 7s 2024-03-19T01:11:48.732 INFO:tasks.workunit.client.0.smithi069.stderr: 81250K .......... .......... .......... .......... .......... 59% 13.8M 7s 2024-03-19T01:11:48.738 INFO:tasks.workunit.client.0.smithi069.stderr: 81300K .......... .......... .......... .......... .......... 59% 12.2M 7s 2024-03-19T01:11:48.741 INFO:tasks.workunit.client.0.smithi069.stderr: 81350K .......... .......... .......... .......... .......... 59% 9.55M 7s 2024-03-19T01:11:48.749 INFO:tasks.workunit.client.0.smithi069.stderr: 81400K .......... .......... .......... .......... .......... 59% 7.26M 7s 2024-03-19T01:11:48.749 INFO:tasks.workunit.client.0.smithi069.stderr: 81450K .......... .......... .......... .......... .......... 60% 88.5M 7s 2024-03-19T01:11:48.752 INFO:tasks.workunit.client.0.smithi069.stderr: 81500K .......... .......... .......... .......... .......... 60% 13.0M 7s 2024-03-19T01:11:48.755 INFO:tasks.workunit.client.0.smithi069.stderr: 81550K .......... .......... .......... .......... .......... 60% 16.1M 6s 2024-03-19T01:11:48.761 INFO:tasks.workunit.client.0.smithi069.stderr: 81600K .......... .......... .......... .......... .......... 60% 9.30M 6s 2024-03-19T01:11:48.763 INFO:tasks.workunit.client.0.smithi069.stderr: 81650K .......... .......... .......... .......... .......... 60% 21.2M 6s 2024-03-19T01:11:48.769 INFO:tasks.workunit.client.0.smithi069.stderr: 81700K .......... .......... .......... .......... .......... 60% 8.64M 6s 2024-03-19T01:11:48.769 INFO:tasks.workunit.client.0.smithi069.stderr: 81750K .......... .......... .......... .......... .......... 60% 53.3M 6s 2024-03-19T01:11:48.774 INFO:tasks.workunit.client.0.smithi069.stderr: 81800K .......... .......... .......... .......... .......... 60% 11.2M 6s 2024-03-19T01:11:48.779 INFO:tasks.workunit.client.0.smithi069.stderr: 81850K .......... .......... .......... .......... .......... 60% 9.34M 6s 2024-03-19T01:11:48.783 INFO:tasks.workunit.client.0.smithi069.stderr: 81900K .......... .......... .......... .......... .......... 60% 12.2M 6s 2024-03-19T01:11:48.790 INFO:tasks.workunit.client.0.smithi069.stderr: 81950K .......... .......... .......... .......... .......... 60% 8.39M 6s 2024-03-19T01:11:48.790 INFO:tasks.workunit.client.0.smithi069.stderr: 82000K .......... .......... .......... .......... .......... 60% 106M 6s 2024-03-19T01:11:48.793 INFO:tasks.workunit.client.0.smithi069.stderr: 82050K .......... .......... .......... .......... .......... 60% 12.4M 6s 2024-03-19T01:11:48.799 INFO:tasks.workunit.client.0.smithi069.stderr: 82100K .......... .......... .......... .......... .......... 60% 8.48M 6s 2024-03-19T01:11:48.802 INFO:tasks.workunit.client.0.smithi069.stderr: 82150K .......... .......... .......... .......... .......... 60% 15.8M 6s 2024-03-19T01:11:48.811 INFO:tasks.workunit.client.0.smithi069.stderr: 82200K .......... .......... .......... .......... .......... 60% 7.10M 6s 2024-03-19T01:11:48.811 INFO:tasks.workunit.client.0.smithi069.stderr: 82250K .......... .......... .......... .......... .......... 60% 66.5M 6s 2024-03-19T01:11:48.813 INFO:tasks.workunit.client.0.smithi069.stderr: 82300K .......... .......... .......... .......... .......... 60% 16.0M 6s 2024-03-19T01:11:48.817 INFO:tasks.workunit.client.0.smithi069.stderr: 82350K .......... .......... .......... .......... .......... 60% 10.7M 6s 2024-03-19T01:11:48.824 INFO:tasks.workunit.client.0.smithi069.stderr: 82400K .......... .......... .......... .......... .......... 60% 7.47M 6s 2024-03-19T01:11:48.825 INFO:tasks.workunit.client.0.smithi069.stderr: 82450K .......... .......... .......... .......... .......... 60% 30.5M 6s 2024-03-19T01:11:48.831 INFO:tasks.workunit.client.0.smithi069.stderr: 82500K .......... .......... .......... .......... .......... 60% 10.6M 6s 2024-03-19T01:11:48.831 INFO:tasks.workunit.client.0.smithi069.stderr: 82550K .......... .......... .......... .......... .......... 60% 64.3M 6s 2024-03-19T01:11:48.836 INFO:tasks.workunit.client.0.smithi069.stderr: 82600K .......... .......... .......... .......... .......... 60% 8.69M 6s 2024-03-19T01:11:48.845 INFO:tasks.workunit.client.0.smithi069.stderr: 82650K .......... .......... .......... .......... .......... 60% 5.72M 6s 2024-03-19T01:11:48.845 INFO:tasks.workunit.client.0.smithi069.stderr: 82700K .......... .......... .......... .......... .......... 60% 59.3M 6s 2024-03-19T01:11:48.851 INFO:tasks.workunit.client.0.smithi069.stderr: 82750K .......... .......... .......... .......... .......... 60% 11.8M 6s 2024-03-19T01:11:48.851 INFO:tasks.workunit.client.0.smithi069.stderr: 82800K .......... .......... .......... .......... .......... 61% 72.0M 6s 2024-03-19T01:11:48.855 INFO:tasks.workunit.client.0.smithi069.stderr: 82850K .......... .......... .......... .......... .......... 61% 11.3M 6s 2024-03-19T01:11:48.867 INFO:tasks.workunit.client.0.smithi069.stderr: 82900K .......... .......... .......... .......... .......... 61% 4.89M 6s 2024-03-19T01:11:48.868 INFO:tasks.workunit.client.0.smithi069.stderr: 82950K .......... .......... .......... .......... .......... 61% 58.4M 6s 2024-03-19T01:11:48.868 INFO:tasks.workunit.client.0.smithi069.stderr: 83000K .......... .......... .......... .......... .......... 61% 47.8M 6s 2024-03-19T01:11:48.872 INFO:tasks.workunit.client.0.smithi069.stderr: 83050K .......... .......... .......... .......... .......... 61% 12.3M 6s 2024-03-19T01:11:48.875 INFO:tasks.workunit.client.0.smithi069.stderr: 83100K .......... .......... .......... .......... .......... 61% 13.0M 6s 2024-03-19T01:11:48.878 INFO:tasks.workunit.client.0.smithi069.stderr: 83150K .......... .......... .......... .......... .......... 61% 14.0M 6s 2024-03-19T01:11:48.888 INFO:tasks.workunit.client.0.smithi069.stderr: 83200K .......... .......... .......... .......... .......... 61% 6.06M 6s 2024-03-19T01:11:48.888 INFO:tasks.workunit.client.0.smithi069.stderr: 83250K .......... .......... .......... .......... .......... 61% 47.5M 6s 2024-03-19T01:11:48.891 INFO:tasks.workunit.client.0.smithi069.stderr: 83300K .......... .......... .......... .......... .......... 61% 13.4M 6s 2024-03-19T01:11:48.894 INFO:tasks.workunit.client.0.smithi069.stderr: 83350K .......... .......... .......... .......... .......... 61% 16.4M 6s 2024-03-19T01:11:48.897 INFO:tasks.workunit.client.0.smithi069.stderr: 83400K .......... .......... .......... .......... .......... 61% 12.9M 6s 2024-03-19T01:11:48.906 INFO:tasks.workunit.client.0.smithi069.stderr: 83450K .......... .......... .......... .......... .......... 61% 5.79M 6s 2024-03-19T01:11:48.908 INFO:tasks.workunit.client.0.smithi069.stderr: 83500K .......... .......... .......... .......... .......... 61% 37.8M 6s 2024-03-19T01:11:48.912 INFO:tasks.workunit.client.0.smithi069.stderr: 83550K .......... .......... .......... .......... .......... 61% 14.2M 6s 2024-03-19T01:11:48.912 INFO:tasks.workunit.client.0.smithi069.stderr: 83600K .......... .......... .......... .......... .......... 61% 61.3M 6s 2024-03-19T01:11:48.915 INFO:tasks.workunit.client.0.smithi069.stderr: 83650K .......... .......... .......... .......... .......... 61% 13.2M 6s 2024-03-19T01:11:48.928 INFO:tasks.workunit.client.0.smithi069.stderr: 83700K .......... .......... .......... .......... .......... 61% 4.49M 6s 2024-03-19T01:11:48.928 INFO:tasks.workunit.client.0.smithi069.stderr: 83750K .......... .......... .......... .......... .......... 61% 68.4M 6s 2024-03-19T01:11:48.932 INFO:tasks.workunit.client.0.smithi069.stderr: 83800K .......... .......... .......... .......... .......... 61% 47.6M 6s 2024-03-19T01:11:48.932 INFO:tasks.workunit.client.0.smithi069.stderr: 83850K .......... .......... .......... .......... .......... 61% 12.6M 6s 2024-03-19T01:11:48.935 INFO:tasks.workunit.client.0.smithi069.stderr: 83900K .......... .......... .......... .......... .......... 61% 16.5M 6s 2024-03-19T01:11:48.939 INFO:tasks.workunit.client.0.smithi069.stderr: 83950K .......... .......... .......... .......... .......... 61% 12.7M 6s 2024-03-19T01:11:48.947 INFO:tasks.workunit.client.0.smithi069.stderr: 84000K .......... .......... .......... .......... .......... 61% 5.92M 6s 2024-03-19T01:11:48.948 INFO:tasks.workunit.client.0.smithi069.stderr: 84050K .......... .......... .......... .......... .......... 61% 31.3M 6s 2024-03-19T01:11:48.953 INFO:tasks.workunit.client.0.smithi069.stderr: 84100K .......... .......... .......... .......... .......... 61% 14.2M 6s 2024-03-19T01:11:48.957 INFO:tasks.workunit.client.0.smithi069.stderr: 84150K .......... .......... .......... .......... .......... 62% 16.1M 6s 2024-03-19T01:11:48.957 INFO:tasks.workunit.client.0.smithi069.stderr: 84200K .......... .......... .......... .......... .......... 62% 25.3M 6s 2024-03-19T01:11:48.974 INFO:tasks.workunit.client.0.smithi069.stderr: 84250K .......... .......... .......... .......... .......... 62% 4.85M 6s 2024-03-19T01:11:48.974 INFO:tasks.workunit.client.0.smithi069.stderr: 84300K .......... .......... .......... .......... .......... 62% 33.8M 6s 2024-03-19T01:11:48.974 INFO:tasks.workunit.client.0.smithi069.stderr: 84350K .......... .......... .......... .......... .......... 62% 13.8M 6s 2024-03-19T01:11:48.974 INFO:tasks.workunit.client.0.smithi069.stderr: 84400K .......... .......... .......... .......... .......... 62% 69.8M 6s 2024-03-19T01:11:48.976 INFO:tasks.workunit.client.0.smithi069.stderr: 84450K .......... .......... .......... .......... .......... 62% 13.5M 6s 2024-03-19T01:11:48.979 INFO:tasks.workunit.client.0.smithi069.stderr: 84500K .......... .......... .......... .......... .......... 62% 13.8M 6s 2024-03-19T01:11:48.988 INFO:tasks.workunit.client.0.smithi069.stderr: 84550K .......... .......... .......... .......... .......... 62% 6.06M 6s 2024-03-19T01:11:48.993 INFO:tasks.workunit.client.0.smithi069.stderr: 84600K .......... .......... .......... .......... .......... 62% 11.8M 6s 2024-03-19T01:11:48.993 INFO:tasks.workunit.client.0.smithi069.stderr: 84650K .......... .......... .......... .......... .......... 62% 58.6M 6s 2024-03-19T01:11:48.996 INFO:tasks.workunit.client.0.smithi069.stderr: 84700K .......... .......... .......... .......... .......... 62% 15.2M 6s 2024-03-19T01:11:49.000 INFO:tasks.workunit.client.0.smithi069.stderr: 84750K .......... .......... .......... .......... .......... 62% 13.4M 6s 2024-03-19T01:11:49.008 INFO:tasks.workunit.client.0.smithi069.stderr: 84800K .......... .......... .......... .......... .......... 62% 6.01M 6s 2024-03-19T01:11:49.009 INFO:tasks.workunit.client.0.smithi069.stderr: 84850K .......... .......... .......... .......... .......... 62% 41.9M 6s 2024-03-19T01:11:49.014 INFO:tasks.workunit.client.0.smithi069.stderr: 84900K .......... .......... .......... .......... .......... 62% 13.0M 6s 2024-03-19T01:11:49.014 INFO:tasks.workunit.client.0.smithi069.stderr: 84950K .......... .......... .......... .......... .......... 62% 59.0M 6s 2024-03-19T01:11:49.018 INFO:tasks.workunit.client.0.smithi069.stderr: 85000K .......... .......... .......... .......... .......... 62% 11.9M 6s 2024-03-19T01:11:49.028 INFO:tasks.workunit.client.0.smithi069.stderr: 85050K .......... .......... .......... .......... .......... 62% 4.89M 6s 2024-03-19T01:11:49.028 INFO:tasks.workunit.client.0.smithi069.stderr: 85100K .......... .......... .......... .......... .......... 62% 61.5M 6s 2024-03-19T01:11:49.034 INFO:tasks.workunit.client.0.smithi069.stderr: 85150K .......... .......... .......... .......... .......... 62% 11.7M 6s 2024-03-19T01:11:49.034 INFO:tasks.workunit.client.0.smithi069.stderr: 85200K .......... .......... .......... .......... .......... 62% 61.1M 6s 2024-03-19T01:11:49.037 INFO:tasks.workunit.client.0.smithi069.stderr: 85250K .......... .......... .......... .......... .......... 62% 14.1M 6s 2024-03-19T01:11:49.040 INFO:tasks.workunit.client.0.smithi069.stderr: 85300K .......... .......... .......... .......... .......... 62% 13.6M 6s 2024-03-19T01:11:49.048 INFO:tasks.workunit.client.0.smithi069.stderr: 85350K .......... .......... .......... .......... .......... 62% 6.21M 6s 2024-03-19T01:11:49.049 INFO:tasks.workunit.client.0.smithi069.stderr: 85400K .......... .......... .......... .......... .......... 62% 39.2M 6s 2024-03-19T01:11:49.053 INFO:tasks.workunit.client.0.smithi069.stderr: 85450K .......... .......... .......... .......... .......... 62% 13.6M 6s 2024-03-19T01:11:49.057 INFO:tasks.workunit.client.0.smithi069.stderr: 85500K .......... .......... .......... .......... .......... 63% 14.7M 6s 2024-03-19T01:11:49.058 INFO:tasks.workunit.client.0.smithi069.stderr: 85550K .......... .......... .......... .......... .......... 63% 26.9M 6s 2024-03-19T01:11:49.061 INFO:tasks.workunit.client.0.smithi069.stderr: 85600K .......... .......... .......... .......... .......... 63% 16.9M 6s 2024-03-19T01:11:49.071 INFO:tasks.workunit.client.0.smithi069.stderr: 85650K .......... .......... .......... .......... .......... 63% 6.15M 6s 2024-03-19T01:11:49.073 INFO:tasks.workunit.client.0.smithi069.stderr: 85700K .......... .......... .......... .......... .......... 63% 44.5M 6s 2024-03-19T01:11:49.074 INFO:tasks.workunit.client.0.smithi069.stderr: 85750K .......... .......... .......... .......... .......... 63% 12.4M 6s 2024-03-19T01:11:49.077 INFO:tasks.workunit.client.0.smithi069.stderr: 85800K .......... .......... .......... .......... .......... 63% 16.1M 6s 2024-03-19T01:11:49.081 INFO:tasks.workunit.client.0.smithi069.stderr: 85850K .......... .......... .......... .......... .......... 63% 12.9M 6s 2024-03-19T01:11:49.089 INFO:tasks.workunit.client.0.smithi069.stderr: 85900K .......... .......... .......... .......... .......... 63% 6.13M 6s 2024-03-19T01:11:49.090 INFO:tasks.workunit.client.0.smithi069.stderr: 85950K .......... .......... .......... .......... .......... 63% 63.6M 6s 2024-03-19T01:11:49.093 INFO:tasks.workunit.client.0.smithi069.stderr: 86000K .......... .......... .......... .......... .......... 63% 11.9M 6s 2024-03-19T01:11:49.094 INFO:tasks.workunit.client.0.smithi069.stderr: 86050K .......... .......... .......... .......... .......... 63% 58.6M 6s 2024-03-19T01:11:49.097 INFO:tasks.workunit.client.0.smithi069.stderr: 86100K .......... .......... .......... .......... .......... 63% 15.3M 6s 2024-03-19T01:11:49.102 INFO:tasks.workunit.client.0.smithi069.stderr: 86150K .......... .......... .......... .......... .......... 63% 11.8M 6s 2024-03-19T01:11:49.110 INFO:tasks.workunit.client.0.smithi069.stderr: 86200K .......... .......... .......... .......... .......... 63% 6.31M 6s 2024-03-19T01:11:49.111 INFO:tasks.workunit.client.0.smithi069.stderr: 86250K .......... .......... .......... .......... .......... 63% 42.2M 6s 2024-03-19T01:11:49.115 INFO:tasks.workunit.client.0.smithi069.stderr: 86300K .......... .......... .......... .......... .......... 63% 12.7M 6s 2024-03-19T01:11:49.118 INFO:tasks.workunit.client.0.smithi069.stderr: 86350K .......... .......... .......... .......... .......... 63% 17.8M 6s 2024-03-19T01:11:49.123 INFO:tasks.workunit.client.0.smithi069.stderr: 86400K .......... .......... .......... .......... .......... 63% 11.2M 6s 2024-03-19T01:11:49.123 INFO:tasks.workunit.client.0.smithi069.stderr: 86450K .......... .......... .......... .......... .......... 63% 40.7M 6s 2024-03-19T01:11:49.132 INFO:tasks.workunit.client.0.smithi069.stderr: 86500K .......... .......... .......... .......... .......... 63% 6.55M 6s 2024-03-19T01:11:49.133 INFO:tasks.workunit.client.0.smithi069.stderr: 86550K .......... .......... .......... .......... .......... 63% 13.9M 6s 2024-03-19T01:11:49.135 INFO:tasks.workunit.client.0.smithi069.stderr: 86600K .......... .......... .......... .......... .......... 63% 43.0M 6s 2024-03-19T01:11:49.138 INFO:tasks.workunit.client.0.smithi069.stderr: 86650K .......... .......... .......... .......... .......... 63% 14.6M 6s 2024-03-19T01:11:49.143 INFO:tasks.workunit.client.0.smithi069.stderr: 86700K .......... .......... .......... .......... .......... 63% 12.0M 6s 2024-03-19T01:11:49.152 INFO:tasks.workunit.client.0.smithi069.stderr: 86750K .......... .......... .......... .......... .......... 63% 6.11M 6s 2024-03-19T01:11:49.152 INFO:tasks.workunit.client.0.smithi069.stderr: 86800K .......... .......... .......... .......... .......... 63% 52.8M 6s 2024-03-19T01:11:49.155 INFO:tasks.workunit.client.0.smithi069.stderr: 86850K .......... .......... .......... .......... .......... 64% 13.8M 6s 2024-03-19T01:11:49.155 INFO:tasks.workunit.client.0.smithi069.stderr: 86900K .......... .......... .......... .......... .......... 64% 59.3M 6s 2024-03-19T01:11:49.159 INFO:tasks.workunit.client.0.smithi069.stderr: 86950K .......... .......... .......... .......... .......... 64% 15.4M 6s 2024-03-19T01:11:49.163 INFO:tasks.workunit.client.0.smithi069.stderr: 87000K .......... .......... .......... .......... .......... 64% 12.1M 6s 2024-03-19T01:11:49.172 INFO:tasks.workunit.client.0.smithi069.stderr: 87050K .......... .......... .......... .......... .......... 64% 5.90M 6s 2024-03-19T01:11:49.172 INFO:tasks.workunit.client.0.smithi069.stderr: 87100K .......... .......... .......... .......... .......... 64% 77.9M 6s 2024-03-19T01:11:49.176 INFO:tasks.workunit.client.0.smithi069.stderr: 87150K .......... .......... .......... .......... .......... 64% 12.8M 6s 2024-03-19T01:11:49.179 INFO:tasks.workunit.client.0.smithi069.stderr: 87200K .......... .......... .......... .......... .......... 64% 14.6M 6s 2024-03-19T01:11:49.183 INFO:tasks.workunit.client.0.smithi069.stderr: 87250K .......... .......... .......... .......... .......... 64% 12.6M 6s 2024-03-19T01:11:49.190 INFO:tasks.workunit.client.0.smithi069.stderr: 87300K .......... .......... .......... .......... .......... 64% 6.45M 6s 2024-03-19T01:11:49.192 INFO:tasks.workunit.client.0.smithi069.stderr: 87350K .......... .......... .......... .......... .......... 64% 36.6M 6s 2024-03-19T01:11:49.194 INFO:tasks.workunit.client.0.smithi069.stderr: 87400K .......... .......... .......... .......... .......... 64% 15.3M 6s 2024-03-19T01:11:49.196 INFO:tasks.workunit.client.0.smithi069.stderr: 87450K .......... .......... .......... .......... .......... 64% 42.3M 6s 2024-03-19T01:11:49.199 INFO:tasks.workunit.client.0.smithi069.stderr: 87500K .......... .......... .......... .......... .......... 64% 15.1M 6s 2024-03-19T01:11:49.204 INFO:tasks.workunit.client.0.smithi069.stderr: 87550K .......... .......... .......... .......... .......... 64% 12.1M 6s 2024-03-19T01:11:49.213 INFO:tasks.workunit.client.0.smithi069.stderr: 87600K .......... .......... .......... .......... .......... 64% 6.30M 6s 2024-03-19T01:11:49.213 INFO:tasks.workunit.client.0.smithi069.stderr: 87650K .......... .......... .......... .......... .......... 64% 38.5M 6s 2024-03-19T01:11:49.216 INFO:tasks.workunit.client.0.smithi069.stderr: 87700K .......... .......... .......... .......... .......... 64% 13.0M 6s 2024-03-19T01:11:49.219 INFO:tasks.workunit.client.0.smithi069.stderr: 87750K .......... .......... .......... .......... .......... 64% 17.3M 6s 2024-03-19T01:11:49.221 INFO:tasks.workunit.client.0.smithi069.stderr: 87800K .......... .......... .......... .......... .......... 64% 24.1M 6s 2024-03-19T01:11:49.223 INFO:tasks.workunit.client.0.smithi069.stderr: 87850K .......... .......... .......... .......... .......... 64% 17.3M 6s 2024-03-19T01:11:49.232 INFO:tasks.workunit.client.0.smithi069.stderr: 87900K .......... .......... .......... .......... .......... 64% 5.81M 6s 2024-03-19T01:11:49.235 INFO:tasks.workunit.client.0.smithi069.stderr: 87950K .......... .......... .......... .......... .......... 64% 15.1M 6s 2024-03-19T01:11:49.238 INFO:tasks.workunit.client.0.smithi069.stderr: 88000K .......... .......... .......... .......... .......... 64% 37.5M 6s 2024-03-19T01:11:49.240 INFO:tasks.workunit.client.0.smithi069.stderr: 88050K .......... .......... .......... .......... .......... 64% 15.3M 6s 2024-03-19T01:11:49.244 INFO:tasks.workunit.client.0.smithi069.stderr: 88100K .......... .......... .......... .......... .......... 64% 12.6M 6s 2024-03-19T01:11:49.251 INFO:tasks.workunit.client.0.smithi069.stderr: 88150K .......... .......... .......... .......... .......... 64% 6.37M 6s 2024-03-19T01:11:49.253 INFO:tasks.workunit.client.0.smithi069.stderr: 88200K .......... .......... .......... .......... .......... 65% 37.2M 6s 2024-03-19T01:11:49.257 INFO:tasks.workunit.client.0.smithi069.stderr: 88250K .......... .......... .......... .......... .......... 65% 12.9M 6s 2024-03-19T01:11:49.257 INFO:tasks.workunit.client.0.smithi069.stderr: 88300K .......... .......... .......... .......... .......... 65% 116M 6s 2024-03-19T01:11:49.260 INFO:tasks.workunit.client.0.smithi069.stderr: 88350K .......... .......... .......... .......... .......... 65% 14.2M 6s 2024-03-19T01:11:49.264 INFO:tasks.workunit.client.0.smithi069.stderr: 88400K .......... .......... .......... .......... .......... 65% 12.5M 6s 2024-03-19T01:11:49.273 INFO:tasks.workunit.client.0.smithi069.stderr: 88450K .......... .......... .......... .......... .......... 65% 5.76M 6s 2024-03-19T01:11:49.274 INFO:tasks.workunit.client.0.smithi069.stderr: 88500K .......... .......... .......... .......... .......... 65% 97.4M 5s 2024-03-19T01:11:49.277 INFO:tasks.workunit.client.0.smithi069.stderr: 88550K .......... .......... .......... .......... .......... 65% 13.2M 5s 2024-03-19T01:11:49.281 INFO:tasks.workunit.client.0.smithi069.stderr: 88600K .......... .......... .......... .......... .......... 65% 15.8M 5s 2024-03-19T01:11:49.285 INFO:tasks.workunit.client.0.smithi069.stderr: 88650K .......... .......... .......... .......... .......... 65% 11.7M 5s 2024-03-19T01:11:49.292 INFO:tasks.workunit.client.0.smithi069.stderr: 88700K .......... .......... .......... .......... .......... 65% 6.31M 5s 2024-03-19T01:11:49.294 INFO:tasks.workunit.client.0.smithi069.stderr: 88750K .......... .......... .......... .......... .......... 65% 36.9M 5s 2024-03-19T01:11:49.296 INFO:tasks.workunit.client.0.smithi069.stderr: 88800K .......... .......... .......... .......... .......... 65% 15.5M 5s 2024-03-19T01:11:49.298 INFO:tasks.workunit.client.0.smithi069.stderr: 88850K .......... .......... .......... .......... .......... 65% 43.5M 5s 2024-03-19T01:11:49.301 INFO:tasks.workunit.client.0.smithi069.stderr: 88900K .......... .......... .......... .......... .......... 65% 14.7M 5s 2024-03-19T01:11:49.305 INFO:tasks.workunit.client.0.smithi069.stderr: 88950K .......... .......... .......... .......... .......... 65% 12.3M 5s 2024-03-19T01:11:49.314 INFO:tasks.workunit.client.0.smithi069.stderr: 89000K .......... .......... .......... .......... .......... 65% 6.25M 5s 2024-03-19T01:11:49.314 INFO:tasks.workunit.client.0.smithi069.stderr: 89050K .......... .......... .......... .......... .......... 65% 43.6M 5s 2024-03-19T01:11:49.318 INFO:tasks.workunit.client.0.smithi069.stderr: 89100K .......... .......... .......... .......... .......... 65% 13.4M 5s 2024-03-19T01:11:49.322 INFO:tasks.workunit.client.0.smithi069.stderr: 89150K .......... .......... .......... .......... .......... 65% 11.4M 5s 2024-03-19T01:11:49.322 INFO:tasks.workunit.client.0.smithi069.stderr: 89200K .......... .......... .......... .......... .......... 65% 72.2M 5s 2024-03-19T01:11:49.325 INFO:tasks.workunit.client.0.smithi069.stderr: 89250K .......... .......... .......... .......... .......... 65% 16.3M 5s 2024-03-19T01:11:49.334 INFO:tasks.workunit.client.0.smithi069.stderr: 89300K .......... .......... .......... .......... .......... 65% 5.75M 5s 2024-03-19T01:11:49.339 INFO:tasks.workunit.client.0.smithi069.stderr: 89350K .......... .......... .......... .......... .......... 65% 15.4M 5s 2024-03-19T01:11:49.339 INFO:tasks.workunit.client.0.smithi069.stderr: 89400K .......... .......... .......... .......... .......... 65% 37.6M 5s 2024-03-19T01:11:49.343 INFO:tasks.workunit.client.0.smithi069.stderr: 89450K .......... .......... .......... .......... .......... 65% 11.2M 5s 2024-03-19T01:11:49.346 INFO:tasks.workunit.client.0.smithi069.stderr: 89500K .......... .......... .......... .......... .......... 65% 18.2M 5s 2024-03-19T01:11:49.353 INFO:tasks.workunit.client.0.smithi069.stderr: 89550K .......... .......... .......... .......... .......... 66% 6.19M 5s 2024-03-19T01:11:49.355 INFO:tasks.workunit.client.0.smithi069.stderr: 89600K .......... .......... .......... .......... .......... 66% 38.9M 5s 2024-03-19T01:11:49.359 INFO:tasks.workunit.client.0.smithi069.stderr: 89650K .......... .......... .......... .......... .......... 66% 13.0M 5s 2024-03-19T01:11:49.359 INFO:tasks.workunit.client.0.smithi069.stderr: 89700K .......... .......... .......... .......... .......... 66% 70.7M 5s 2024-03-19T01:11:49.363 INFO:tasks.workunit.client.0.smithi069.stderr: 89750K .......... .......... .......... .......... .......... 66% 11.6M 5s 2024-03-19T01:11:49.366 INFO:tasks.workunit.client.0.smithi069.stderr: 89800K .......... .......... .......... .......... .......... 66% 18.3M 5s 2024-03-19T01:11:49.375 INFO:tasks.workunit.client.0.smithi069.stderr: 89850K .......... .......... .......... .......... .......... 66% 5.78M 5s 2024-03-19T01:11:49.375 INFO:tasks.workunit.client.0.smithi069.stderr: 89900K .......... .......... .......... .......... .......... 66% 152M 5s 2024-03-19T01:11:49.380 INFO:tasks.workunit.client.0.smithi069.stderr: 89950K .......... .......... .......... .......... .......... 66% 11.1M 5s 2024-03-19T01:11:49.383 INFO:tasks.workunit.client.0.smithi069.stderr: 90000K .......... .......... .......... .......... .......... 66% 12.0M 5s 2024-03-19T01:11:49.383 INFO:tasks.workunit.client.0.smithi069.stderr: 90050K .......... .......... .......... .......... .......... 66% 151M 5s 2024-03-19T01:11:49.395 INFO:tasks.workunit.client.0.smithi069.stderr: 90100K .......... .......... .......... .......... .......... 66% 4.73M 5s 2024-03-19T01:11:49.395 INFO:tasks.workunit.client.0.smithi069.stderr: 90150K .......... .......... .......... .......... .......... 66% 51.2M 5s 2024-03-19T01:11:49.398 INFO:tasks.workunit.client.0.smithi069.stderr: 90200K .......... .......... .......... .......... .......... 66% 14.4M 5s 2024-03-19T01:11:49.400 INFO:tasks.workunit.client.0.smithi069.stderr: 90250K .......... .......... .......... .......... .......... 66% 31.6M 5s 2024-03-19T01:11:49.405 INFO:tasks.workunit.client.0.smithi069.stderr: 90300K .......... .......... .......... .......... .......... 66% 11.5M 5s 2024-03-19T01:11:49.407 INFO:tasks.workunit.client.0.smithi069.stderr: 90350K .......... .......... .......... .......... .......... 66% 18.8M 5s 2024-03-19T01:11:49.417 INFO:tasks.workunit.client.0.smithi069.stderr: 90400K .......... .......... .......... .......... .......... 66% 5.86M 5s 2024-03-19T01:11:49.417 INFO:tasks.workunit.client.0.smithi069.stderr: 90450K .......... .......... .......... .......... .......... 66% 54.2M 5s 2024-03-19T01:11:49.421 INFO:tasks.workunit.client.0.smithi069.stderr: 90500K .......... .......... .......... .......... .......... 66% 13.1M 5s 2024-03-19T01:11:49.424 INFO:tasks.workunit.client.0.smithi069.stderr: 90550K .......... .......... .......... .......... .......... 66% 56.7M 5s 2024-03-19T01:11:49.425 INFO:tasks.workunit.client.0.smithi069.stderr: 90600K .......... .......... .......... .......... .......... 66% 13.1M 5s 2024-03-19T01:11:49.427 INFO:tasks.workunit.client.0.smithi069.stderr: 90650K .......... .......... .......... .......... .......... 66% 17.8M 5s 2024-03-19T01:11:49.436 INFO:tasks.workunit.client.0.smithi069.stderr: 90700K .......... .......... .......... .......... .......... 66% 5.70M 5s 2024-03-19T01:11:49.436 INFO:tasks.workunit.client.0.smithi069.stderr: 90750K .......... .......... .......... .......... .......... 66% 126M 5s 2024-03-19T01:11:49.440 INFO:tasks.workunit.client.0.smithi069.stderr: 90800K .......... .......... .......... .......... .......... 66% 10.9M 5s 2024-03-19T01:11:49.445 INFO:tasks.workunit.client.0.smithi069.stderr: 90850K .......... .......... .......... .......... .......... 66% 12.3M 5s 2024-03-19T01:11:49.447 INFO:tasks.workunit.client.0.smithi069.stderr: 90900K .......... .......... .......... .......... .......... 66% 19.2M 5s 2024-03-19T01:11:49.457 INFO:tasks.workunit.client.0.smithi069.stderr: 90950K .......... .......... .......... .......... .......... 67% 5.26M 5s 2024-03-19T01:11:49.457 INFO:tasks.workunit.client.0.smithi069.stderr: 91000K .......... .......... .......... .......... .......... 67% 88.1M 5s 2024-03-19T01:11:49.459 INFO:tasks.workunit.client.0.smithi069.stderr: 91050K .......... .......... .......... .......... .......... 67% 15.0M 5s 2024-03-19T01:11:49.461 INFO:tasks.workunit.client.0.smithi069.stderr: 91100K .......... .......... .......... .......... .......... 67% 56.8M 5s 2024-03-19T01:11:49.465 INFO:tasks.workunit.client.0.smithi069.stderr: 91150K .......... .......... .......... .......... .......... 67% 12.0M 5s 2024-03-19T01:11:49.467 INFO:tasks.workunit.client.0.smithi069.stderr: 91200K .......... .......... .......... .......... .......... 67% 17.9M 5s 2024-03-19T01:11:49.477 INFO:tasks.workunit.client.0.smithi069.stderr: 91250K .......... .......... .......... .......... .......... 67% 5.65M 5s 2024-03-19T01:11:49.477 INFO:tasks.workunit.client.0.smithi069.stderr: 91300K .......... .......... .......... .......... .......... 67% 48.3M 5s 2024-03-19T01:11:49.481 INFO:tasks.workunit.client.0.smithi069.stderr: 91350K .......... .......... .......... .......... .......... 67% 16.0M 5s 2024-03-19T01:11:49.481 INFO:tasks.workunit.client.0.smithi069.stderr: 91400K .......... .......... .......... .......... .......... 67% 52.5M 5s 2024-03-19T01:11:49.485 INFO:tasks.workunit.client.0.smithi069.stderr: 91450K .......... .......... .......... .......... .......... 67% 11.7M 5s 2024-03-19T01:11:49.488 INFO:tasks.workunit.client.0.smithi069.stderr: 91500K .......... .......... .......... .......... .......... 67% 18.6M 5s 2024-03-19T01:11:49.499 INFO:tasks.workunit.client.0.smithi069.stderr: 91550K .......... .......... .......... .......... .......... 67% 5.42M 5s 2024-03-19T01:11:49.499 INFO:tasks.workunit.client.0.smithi069.stderr: 91600K .......... .......... .......... .......... .......... 67% 68.4M 5s 2024-03-19T01:11:49.500 INFO:tasks.workunit.client.0.smithi069.stderr: 91650K .......... .......... .......... .......... .......... 67% 16.7M 5s 2024-03-19T01:11:49.505 INFO:tasks.workunit.client.0.smithi069.stderr: 91700K .......... .......... .......... .......... .......... 67% 10.9M 5s 2024-03-19T01:11:49.505 INFO:tasks.workunit.client.0.smithi069.stderr: 91750K .......... .......... .......... .......... .......... 67% 116M 5s 2024-03-19T01:11:49.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:49 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:11:49.518 INFO:tasks.workunit.client.0.smithi069.stderr: 91800K .......... .......... .......... .......... .......... 67% 4.35M 5s 2024-03-19T01:11:49.518 INFO:tasks.workunit.client.0.smithi069.stderr: 91850K .......... .......... .......... .......... .......... 67% 80.2M 5s 2024-03-19T01:11:49.518 INFO:tasks.workunit.client.0.smithi069.stderr: 91900K .......... .......... .......... .......... .......... 67% 113M 5s 2024-03-19T01:11:49.522 INFO:tasks.workunit.client.0.smithi069.stderr: 91950K .......... .......... .......... .......... .......... 67% 13.9M 5s 2024-03-19T01:11:49.527 INFO:tasks.workunit.client.0.smithi069.stderr: 92000K .......... .......... .......... .......... .......... 67% 11.1M 5s 2024-03-19T01:11:49.529 INFO:tasks.workunit.client.0.smithi069.stderr: 92050K .......... .......... .......... .......... .......... 67% 18.8M 5s 2024-03-19T01:11:49.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:49 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:11:49.539 INFO:tasks.workunit.client.0.smithi069.stderr: 92100K .......... .......... .......... .......... .......... 67% 5.46M 5s 2024-03-19T01:11:49.539 INFO:tasks.workunit.client.0.smithi069.stderr: 92150K .......... .......... .......... .......... .......... 67% 44.4M 5s 2024-03-19T01:11:49.544 INFO:tasks.workunit.client.0.smithi069.stderr: 92200K .......... .......... .......... .......... .......... 67% 20.3M 5s 2024-03-19T01:11:49.544 INFO:tasks.workunit.client.0.smithi069.stderr: 92250K .......... .......... .......... .......... .......... 67% 52.0M 5s 2024-03-19T01:11:49.547 INFO:tasks.workunit.client.0.smithi069.stderr: 92300K .......... .......... .......... .......... .......... 68% 10.9M 5s 2024-03-19T01:11:49.549 INFO:tasks.workunit.client.0.smithi069.stderr: 92350K .......... .......... .......... .......... .......... 68% 20.5M 5s 2024-03-19T01:11:49.559 INFO:tasks.workunit.client.0.smithi069.stderr: 92400K .......... .......... .......... .......... .......... 68% 5.35M 5s 2024-03-19T01:11:49.559 INFO:tasks.workunit.client.0.smithi069.stderr: 92450K .......... .......... .......... .......... .......... 68% 74.4M 5s 2024-03-19T01:11:49.562 INFO:tasks.workunit.client.0.smithi069.stderr: 92500K .......... .......... .......... .......... .......... 68% 18.6M 5s 2024-03-19T01:11:49.562 INFO:tasks.workunit.client.0.smithi069.stderr: 92550K .......... .......... .......... .......... .......... 68% 36.4M 5s 2024-03-19T01:11:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:49 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:11:49.567 INFO:tasks.workunit.client.0.smithi069.stderr: 92600K .......... .......... .......... .......... .......... 68% 12.1M 5s 2024-03-19T01:11:49.569 INFO:tasks.workunit.client.0.smithi069.stderr: 92650K .......... .......... .......... .......... .......... 68% 18.1M 5s 2024-03-19T01:11:49.580 INFO:tasks.workunit.client.0.smithi069.stderr: 92700K .......... .......... .......... .......... .......... 68% 5.35M 5s 2024-03-19T01:11:49.580 INFO:tasks.workunit.client.0.smithi069.stderr: 92750K .......... .......... .......... .......... .......... 68% 97.1M 5s 2024-03-19T01:11:49.581 INFO:tasks.workunit.client.0.smithi069.stderr: 92800K .......... .......... .......... .......... .......... 68% 18.8M 5s 2024-03-19T01:11:49.583 INFO:tasks.workunit.client.0.smithi069.stderr: 92850K .......... .......... .......... .......... .......... 68% 27.3M 5s 2024-03-19T01:11:49.587 INFO:tasks.workunit.client.0.smithi069.stderr: 92900K .......... .......... .......... .......... .......... 68% 13.3M 5s 2024-03-19T01:11:49.589 INFO:tasks.workunit.client.0.smithi069.stderr: 92950K .......... .......... .......... .......... .......... 68% 17.9M 5s 2024-03-19T01:11:49.600 INFO:tasks.workunit.client.0.smithi069.stderr: 93000K .......... .......... .......... .......... .......... 68% 5.39M 5s 2024-03-19T01:11:49.600 INFO:tasks.workunit.client.0.smithi069.stderr: 93050K .......... .......... .......... .......... .......... 68% 94.3M 5s 2024-03-19T01:11:49.602 INFO:tasks.workunit.client.0.smithi069.stderr: 93100K .......... .......... .......... .......... .......... 68% 15.6M 5s 2024-03-19T01:11:49.603 INFO:tasks.workunit.client.0.smithi069.stderr: 93150K .......... .......... .......... .......... .......... 68% 40.6M 5s 2024-03-19T01:11:49.607 INFO:tasks.workunit.client.0.smithi069.stderr: 93200K .......... .......... .......... .......... .......... 68% 13.4M 5s 2024-03-19T01:11:49.610 INFO:tasks.workunit.client.0.smithi069.stderr: 93250K .......... .......... .......... .......... .......... 68% 17.1M 5s 2024-03-19T01:11:49.620 INFO:tasks.workunit.client.0.smithi069.stderr: 93300K .......... .......... .......... .......... .......... 68% 5.41M 5s 2024-03-19T01:11:49.620 INFO:tasks.workunit.client.0.smithi069.stderr: 93350K .......... .......... .......... .......... .......... 68% 80.5M 5s 2024-03-19T01:11:49.623 INFO:tasks.workunit.client.0.smithi069.stderr: 93400K .......... .......... .......... .......... .......... 68% 15.0M 5s 2024-03-19T01:11:49.623 INFO:tasks.workunit.client.0.smithi069.stderr: 93450K .......... .......... .......... .......... .......... 68% 53.9M 5s 2024-03-19T01:11:49.627 INFO:tasks.workunit.client.0.smithi069.stderr: 93500K .......... .......... .......... .......... .......... 68% 13.4M 5s 2024-03-19T01:11:49.630 INFO:tasks.workunit.client.0.smithi069.stderr: 93550K .......... .......... .......... .......... .......... 68% 16.6M 5s 2024-03-19T01:11:49.640 INFO:tasks.workunit.client.0.smithi069.stderr: 93600K .......... .......... .......... .......... .......... 68% 5.36M 5s 2024-03-19T01:11:49.640 INFO:tasks.workunit.client.0.smithi069.stderr: 93650K .......... .......... .......... .......... .......... 69% 95.6M 5s 2024-03-19T01:11:49.643 INFO:tasks.workunit.client.0.smithi069.stderr: 93700K .......... .......... .......... .......... .......... 69% 15.5M 5s 2024-03-19T01:11:49.644 INFO:tasks.workunit.client.0.smithi069.stderr: 93750K .......... .......... .......... .......... .......... 69% 50.5M 5s 2024-03-19T01:11:49.647 INFO:tasks.workunit.client.0.smithi069.stderr: 93800K .......... .......... .......... .......... .......... 69% 13.7M 5s 2024-03-19T01:11:49.650 INFO:tasks.workunit.client.0.smithi069.stderr: 93850K .......... .......... .......... .......... .......... 69% 15.6M 5s 2024-03-19T01:11:49.661 INFO:tasks.workunit.client.0.smithi069.stderr: 93900K .......... .......... .......... .......... .......... 69% 5.40M 5s 2024-03-19T01:11:49.661 INFO:tasks.workunit.client.0.smithi069.stderr: 93950K .......... .......... .......... .......... .......... 69% 97.3M 5s 2024-03-19T01:11:49.663 INFO:tasks.workunit.client.0.smithi069.stderr: 94000K .......... .......... .......... .......... .......... 69% 16.0M 5s 2024-03-19T01:11:49.664 INFO:tasks.workunit.client.0.smithi069.stderr: 94050K .......... .......... .......... .......... .......... 69% 48.6M 5s 2024-03-19T01:11:49.668 INFO:tasks.workunit.client.0.smithi069.stderr: 94100K .......... .......... .......... .......... .......... 69% 13.3M 5s 2024-03-19T01:11:49.681 INFO:tasks.workunit.client.0.smithi069.stderr: 94150K .......... .......... .......... .......... .......... 69% 16.0M 5s 2024-03-19T01:11:49.681 INFO:tasks.workunit.client.0.smithi069.stderr: 94200K .......... .......... .......... .......... .......... 69% 4.98M 5s 2024-03-19T01:11:49.681 INFO:tasks.workunit.client.0.smithi069.stderr: 94250K .......... .......... .......... .......... .......... 69% 76.4M 5s 2024-03-19T01:11:49.684 INFO:tasks.workunit.client.0.smithi069.stderr: 94300K .......... .......... .......... .......... .......... 69% 20.5M 5s 2024-03-19T01:11:49.684 INFO:tasks.workunit.client.0.smithi069.stderr: 94350K .......... .......... .......... .......... .......... 69% 53.8M 5s 2024-03-19T01:11:49.688 INFO:tasks.workunit.client.0.smithi069.stderr: 94400K .......... .......... .......... .......... .......... 69% 13.7M 5s 2024-03-19T01:11:49.700 INFO:tasks.workunit.client.0.smithi069.stderr: 94450K .......... .......... .......... .......... .......... 69% 4.16M 5s 2024-03-19T01:11:49.700 INFO:tasks.workunit.client.0.smithi069.stderr: 94500K .......... .......... .......... .......... .......... 69% 121M 5s 2024-03-19T01:11:49.700 INFO:tasks.workunit.client.0.smithi069.stderr: 94550K .......... .......... .......... .......... .......... 69% 107M 5s 2024-03-19T01:11:49.703 INFO:tasks.workunit.client.0.smithi069.stderr: 94600K .......... .......... .......... .......... .......... 69% 16.5M 5s 2024-03-19T01:11:49.708 INFO:tasks.workunit.client.0.smithi069.stderr: 94650K .......... .......... .......... .......... .......... 69% 11.6M 5s 2024-03-19T01:11:49.709 INFO:tasks.workunit.client.0.smithi069.stderr: 94700K .......... .......... .......... .......... .......... 69% 67.8M 5s 2024-03-19T01:11:49.722 INFO:tasks.workunit.client.0.smithi069.stderr: 94750K .......... .......... .......... .......... .......... 69% 4.21M 5s 2024-03-19T01:11:49.722 INFO:tasks.workunit.client.0.smithi069.stderr: 94800K .......... .......... .......... .......... .......... 69% 108M 5s 2024-03-19T01:11:49.722 INFO:tasks.workunit.client.0.smithi069.stderr: 94850K .......... .......... .......... .......... .......... 69% 70.8M 5s 2024-03-19T01:11:49.724 INFO:tasks.workunit.client.0.smithi069.stderr: 94900K .......... .......... .......... .......... .......... 69% 17.7M 5s 2024-03-19T01:11:49.729 INFO:tasks.workunit.client.0.smithi069.stderr: 94950K .......... .......... .......... .......... .......... 69% 11.7M 5s 2024-03-19T01:11:49.729 INFO:tasks.workunit.client.0.smithi069.stderr: 95000K .......... .......... .......... .......... .......... 70% 66.7M 5s 2024-03-19T01:11:49.742 INFO:tasks.workunit.client.0.smithi069.stderr: 95050K .......... .......... .......... .......... .......... 70% 4.23M 5s 2024-03-19T01:11:49.742 INFO:tasks.workunit.client.0.smithi069.stderr: 95100K .......... .......... .......... .......... .......... 70% 83.6M 5s 2024-03-19T01:11:49.745 INFO:tasks.workunit.client.0.smithi069.stderr: 95150K .......... .......... .......... .......... .......... 70% 24.1M 5s 2024-03-19T01:11:49.747 INFO:tasks.workunit.client.0.smithi069.stderr: 95200K .......... .......... .......... .......... .......... 70% 26.1M 5s 2024-03-19T01:11:49.749 INFO:tasks.workunit.client.0.smithi069.stderr: 95250K .......... .......... .......... .......... .......... 70% 13.6M 5s 2024-03-19T01:11:49.750 INFO:tasks.workunit.client.0.smithi069.stderr: 95300K .......... .......... .......... .......... .......... 70% 27.3M 5s 2024-03-19T01:11:49.762 INFO:tasks.workunit.client.0.smithi069.stderr: 95350K .......... .......... .......... .......... .......... 70% 4.72M 5s 2024-03-19T01:11:49.762 INFO:tasks.workunit.client.0.smithi069.stderr: 95400K .......... .......... .......... .......... .......... 70% 73.3M 5s 2024-03-19T01:11:49.763 INFO:tasks.workunit.client.0.smithi069.stderr: 95450K .......... .......... .......... .......... .......... 70% 23.7M 5s 2024-03-19T01:11:49.765 INFO:tasks.workunit.client.0.smithi069.stderr: 95500K .......... .......... .......... .......... .......... 70% 27.6M 5s 2024-03-19T01:11:49.770 INFO:tasks.workunit.client.0.smithi069.stderr: 95550K .......... .......... .......... .......... .......... 70% 12.7M 5s 2024-03-19T01:11:49.771 INFO:tasks.workunit.client.0.smithi069.stderr: 95600K .......... .......... .......... .......... .......... 70% 30.7M 5s 2024-03-19T01:11:49.784 INFO:tasks.workunit.client.0.smithi069.stderr: 95650K .......... .......... .......... .......... .......... 70% 4.58M 5s 2024-03-19T01:11:49.784 INFO:tasks.workunit.client.0.smithi069.stderr: 95700K .......... .......... .......... .......... .......... 70% 69.2M 5s 2024-03-19T01:11:49.784 INFO:tasks.workunit.client.0.smithi069.stderr: 95750K .......... .......... .......... .......... .......... 70% 28.6M 5s 2024-03-19T01:11:49.785 INFO:tasks.workunit.client.0.smithi069.stderr: 95800K .......... .......... .......... .......... .......... 70% 26.2M 5s 2024-03-19T01:11:49.790 INFO:tasks.workunit.client.0.smithi069.stderr: 95850K .......... .......... .......... .......... .......... 70% 11.9M 4s 2024-03-19T01:11:49.793 INFO:tasks.workunit.client.0.smithi069.stderr: 95900K .......... .......... .......... .......... .......... 70% 16.6M 4s 2024-03-19T01:11:49.804 INFO:tasks.workunit.client.0.smithi069.stderr: 95950K .......... .......... .......... .......... .......... 70% 5.22M 4s 2024-03-19T01:11:49.804 INFO:tasks.workunit.client.0.smithi069.stderr: 96000K .......... .......... .......... .......... .......... 70% 61.0M 4s 2024-03-19T01:11:49.807 INFO:tasks.workunit.client.0.smithi069.stderr: 96050K .......... .......... .......... .......... .......... 70% 12.4M 4s 2024-03-19T01:11:49.813 INFO:tasks.workunit.client.0.smithi069.stderr: 96100K .......... .......... .......... .......... .......... 70% 8.87M 4s 2024-03-19T01:11:49.818 INFO:tasks.workunit.client.0.smithi069.stderr: 96150K .......... .......... .......... .......... .......... 70% 9.78M 4s 2024-03-19T01:11:49.821 INFO:tasks.workunit.client.0.smithi069.stderr: 96200K .......... .......... .......... .......... .......... 70% 12.1M 4s 2024-03-19T01:11:49.826 INFO:tasks.workunit.client.0.smithi069.stderr: 96250K .......... .......... .......... .......... .......... 70% 20.6M 4s 2024-03-19T01:11:49.826 INFO:tasks.workunit.client.0.smithi069.stderr: 96300K .......... .......... .......... .......... .......... 70% 37.4M 4s 2024-03-19T01:11:49.828 INFO:tasks.workunit.client.0.smithi069.stderr: 96350K .......... .......... .......... .......... .......... 71% 15.9M 4s 2024-03-19T01:11:49.832 INFO:tasks.workunit.client.0.smithi069.stderr: 96400K .......... .......... .......... .......... .......... 71% 10.7M 4s 2024-03-19T01:11:49.838 INFO:tasks.workunit.client.0.smithi069.stderr: 96450K .......... .......... .......... .......... .......... 71% 9.56M 4s 2024-03-19T01:11:49.843 INFO:tasks.workunit.client.0.smithi069.stderr: 96500K .......... .......... .......... .......... .......... 71% 9.02M 4s 2024-03-19T01:11:49.846 INFO:tasks.workunit.client.0.smithi069.stderr: 96550K .......... .......... .......... .......... .......... 71% 19.5M 4s 2024-03-19T01:11:49.847 INFO:tasks.workunit.client.0.smithi069.stderr: 96600K .......... .......... .......... .......... .......... 71% 29.3M 4s 2024-03-19T01:11:49.850 INFO:tasks.workunit.client.0.smithi069.stderr: 96650K .......... .......... .......... .......... .......... 71% 16.9M 4s 2024-03-19T01:11:49.853 INFO:tasks.workunit.client.0.smithi069.stderr: 96700K .......... .......... .......... .......... .......... 71% 17.4M 4s 2024-03-19T01:11:49.858 INFO:tasks.workunit.client.0.smithi069.stderr: 96750K .......... .......... .......... .......... .......... 71% 9.90M 4s 2024-03-19T01:11:49.863 INFO:tasks.workunit.client.0.smithi069.stderr: 96800K .......... .......... .......... .......... .......... 71% 9.56M 4s 2024-03-19T01:11:49.866 INFO:tasks.workunit.client.0.smithi069.stderr: 96850K .......... .......... .......... .......... .......... 71% 16.7M 4s 2024-03-19T01:11:49.867 INFO:tasks.workunit.client.0.smithi069.stderr: 96900K .......... .......... .......... .......... .......... 71% 29.5M 4s 2024-03-19T01:11:49.875 INFO:tasks.workunit.client.0.smithi069.stderr: 96950K .......... .......... .......... .......... .......... 71% 6.83M 4s 2024-03-19T01:11:49.881 INFO:tasks.workunit.client.0.smithi069.stderr: 97000K .......... .......... .......... .......... .......... 71% 8.89M 4s 2024-03-19T01:11:49.886 INFO:tasks.workunit.client.0.smithi069.stderr: 97050K .......... .......... .......... .......... .......... 71% 8.99M 4s 2024-03-19T01:11:49.892 INFO:tasks.workunit.client.0.smithi069.stderr: 97100K .......... .......... .......... .......... .......... 71% 8.05M 4s 2024-03-19T01:11:49.894 INFO:tasks.workunit.client.0.smithi069.stderr: 97150K .......... .......... .......... .......... .......... 71% 19.9M 4s 2024-03-19T01:11:49.897 INFO:tasks.workunit.client.0.smithi069.stderr: 97200K .......... .......... .......... .......... .......... 71% 18.1M 4s 2024-03-19T01:11:49.900 INFO:tasks.workunit.client.0.smithi069.stderr: 97250K .......... .......... .......... .......... .......... 71% 17.3M 4s 2024-03-19T01:11:49.904 INFO:tasks.workunit.client.0.smithi069.stderr: 97300K .......... .......... .......... .......... .......... 71% 9.77M 4s 2024-03-19T01:11:49.908 INFO:tasks.workunit.client.0.smithi069.stderr: 97350K .......... .......... .......... .......... .......... 71% 13.3M 4s 2024-03-19T01:11:49.917 INFO:tasks.workunit.client.0.smithi069.stderr: 97400K .......... .......... .......... .......... .......... 71% 6.38M 4s 2024-03-19T01:11:49.919 INFO:tasks.workunit.client.0.smithi069.stderr: 97450K .......... .......... .......... .......... .......... 71% 17.0M 4s 2024-03-19T01:11:49.921 INFO:tasks.workunit.client.0.smithi069.stderr: 97500K .......... .......... .......... .......... .......... 71% 22.4M 4s 2024-03-19T01:11:49.923 INFO:tasks.workunit.client.0.smithi069.stderr: 97550K .......... .......... .......... .......... .......... 71% 21.6M 4s 2024-03-19T01:11:49.926 INFO:tasks.workunit.client.0.smithi069.stderr: 97600K .......... .......... .......... .......... .......... 71% 20.7M 4s 2024-03-19T01:11:49.929 INFO:tasks.workunit.client.0.smithi069.stderr: 97650K .......... .......... .......... .......... .......... 71% 15.8M 4s 2024-03-19T01:11:49.937 INFO:tasks.workunit.client.0.smithi069.stderr: 97700K .......... .......... .......... .......... .......... 72% 5.56M 4s 2024-03-19T01:11:49.946 INFO:tasks.workunit.client.0.smithi069.stderr: 97750K .......... .......... .......... .......... .......... 72% 5.90M 4s 2024-03-19T01:11:49.958 INFO:tasks.workunit.client.0.smithi069.stderr: 97800K .......... .......... .......... .......... .......... 72% 12.7M 4s 2024-03-19T01:11:49.958 INFO:tasks.workunit.client.0.smithi069.stderr: 97850K .......... .......... .......... .......... .......... 72% 14.1M 4s 2024-03-19T01:11:49.958 INFO:tasks.workunit.client.0.smithi069.stderr: 97900K .......... .......... .......... .......... .......... 72% 16.4M 4s 2024-03-19T01:11:49.960 INFO:tasks.workunit.client.0.smithi069.stderr: 97950K .......... .......... .......... .......... .......... 72% 13.1M 4s 2024-03-19T01:11:49.968 INFO:tasks.workunit.client.0.smithi069.stderr: 98000K .......... .......... .......... .......... .......... 72% 6.15M 4s 2024-03-19T01:11:49.973 INFO:tasks.workunit.client.0.smithi069.stderr: 98050K .......... .......... .......... .......... .......... 72% 9.73M 4s 2024-03-19T01:11:49.977 INFO:tasks.workunit.client.0.smithi069.stderr: 98100K .......... .......... .......... .......... .......... 72% 14.3M 4s 2024-03-19T01:11:49.982 INFO:tasks.workunit.client.0.smithi069.stderr: 98150K .......... .......... .......... .......... .......... 72% 12.4M 4s 2024-03-19T01:11:49.987 INFO:tasks.workunit.client.0.smithi069.stderr: 98200K .......... .......... .......... .......... .......... 72% 11.7M 4s 2024-03-19T01:11:49.991 INFO:tasks.workunit.client.0.smithi069.stderr: 98250K .......... .......... .......... .......... .......... 72% 9.67M 4s 2024-03-19T01:11:50.004 INFO:tasks.workunit.client.0.smithi069.stderr: 98300K .......... .......... .......... .......... .......... 72% 12.1M 4s 2024-03-19T01:11:50.004 INFO:tasks.workunit.client.0.smithi069.stderr: 98350K .......... .......... .......... .......... .......... 72% 17.9M 4s 2024-03-19T01:11:50.004 INFO:tasks.workunit.client.0.smithi069.stderr: 98400K .......... .......... .......... .......... .......... 72% 25.9M 4s 2024-03-19T01:11:50.004 INFO:tasks.workunit.client.0.smithi069.stderr: 98450K .......... .......... .......... .......... .......... 72% 10.2M 4s 2024-03-19T01:11:50.017 INFO:tasks.workunit.client.0.smithi069.stderr: 98500K .......... .......... .......... .......... .......... 72% 3.78M 4s 2024-03-19T01:11:50.020 INFO:tasks.workunit.client.0.smithi069.stderr: 98550K .......... .......... .......... .......... .......... 72% 16.7M 4s 2024-03-19T01:11:50.022 INFO:tasks.workunit.client.0.smithi069.stderr: 98600K .......... .......... .......... .......... .......... 72% 16.0M 4s 2024-03-19T01:11:50.025 INFO:tasks.workunit.client.0.smithi069.stderr: 98650K .......... .......... .......... .......... .......... 72% 13.2M 4s 2024-03-19T01:11:50.028 INFO:tasks.workunit.client.0.smithi069.stderr: 98700K .......... .......... .......... .......... .......... 72% 17.4M 4s 2024-03-19T01:11:50.032 INFO:tasks.workunit.client.0.smithi069.stderr: 98750K .......... .......... .......... .......... .......... 72% 13.2M 4s 2024-03-19T01:11:50.042 INFO:tasks.workunit.client.0.smithi069.stderr: 98800K .......... .......... .......... .......... .......... 72% 4.64M 4s 2024-03-19T01:11:50.046 INFO:tasks.workunit.client.0.smithi069.stderr: 98850K .......... .......... .......... .......... .......... 72% 14.6M 4s 2024-03-19T01:11:50.049 INFO:tasks.workunit.client.0.smithi069.stderr: 98900K .......... .......... .......... .......... .......... 72% 13.8M 4s 2024-03-19T01:11:50.053 INFO:tasks.workunit.client.0.smithi069.stderr: 98950K .......... .......... .......... .......... .......... 72% 13.4M 4s 2024-03-19T01:11:50.057 INFO:tasks.workunit.client.0.smithi069.stderr: 99000K .......... .......... .......... .......... .......... 72% 15.3M 4s 2024-03-19T01:11:50.061 INFO:tasks.workunit.client.0.smithi069.stderr: 99050K .......... .......... .......... .......... .......... 73% 10.8M 4s 2024-03-19T01:11:50.069 INFO:tasks.workunit.client.0.smithi069.stderr: 99100K .......... .......... .......... .......... .......... 73% 6.87M 4s 2024-03-19T01:11:50.071 INFO:tasks.workunit.client.0.smithi069.stderr: 99150K .......... .......... .......... .......... .......... 73% 14.7M 4s 2024-03-19T01:11:50.078 INFO:tasks.workunit.client.0.smithi069.stderr: 99200K .......... .......... .......... .......... .......... 73% 7.26M 4s 2024-03-19T01:11:50.083 INFO:tasks.workunit.client.0.smithi069.stderr: 99250K .......... .......... .......... .......... .......... 73% 9.88M 4s 2024-03-19T01:11:50.089 INFO:tasks.workunit.client.0.smithi069.stderr: 99300K .......... .......... .......... .......... .......... 73% 9.35M 4s 2024-03-19T01:11:50.094 INFO:tasks.workunit.client.0.smithi069.stderr: 99350K .......... .......... .......... .......... .......... 73% 10.4M 4s 2024-03-19T01:11:50.098 INFO:tasks.workunit.client.0.smithi069.stderr: 99400K .......... .......... .......... .......... .......... 73% 8.58M 4s 2024-03-19T01:11:50.102 INFO:tasks.workunit.client.0.smithi069.stderr: 99450K .......... .......... .......... .......... .......... 73% 18.1M 4s 2024-03-19T01:11:50.108 INFO:tasks.workunit.client.0.smithi069.stderr: 99500K .......... .......... .......... .......... .......... 73% 7.56M 4s 2024-03-19T01:11:50.114 INFO:tasks.workunit.client.0.smithi069.stderr: 99550K .......... .......... .......... .......... .......... 73% 7.60M 4s 2024-03-19T01:11:50.120 INFO:tasks.workunit.client.0.smithi069.stderr: 99600K .......... .......... .......... .......... .......... 73% 7.78M 4s 2024-03-19T01:11:50.125 INFO:tasks.workunit.client.0.smithi069.stderr: 99650K .......... .......... .......... .......... .......... 73% 10.1M 4s 2024-03-19T01:11:50.130 INFO:tasks.workunit.client.0.smithi069.stderr: 99700K .......... .......... .......... .......... .......... 73% 10.6M 4s 2024-03-19T01:11:50.134 INFO:tasks.workunit.client.0.smithi069.stderr: 99750K .......... .......... .......... .......... .......... 73% 12.4M 4s 2024-03-19T01:11:50.137 INFO:tasks.workunit.client.0.smithi069.stderr: 99800K .......... .......... .......... .......... .......... 73% 14.1M 4s 2024-03-19T01:11:50.149 INFO:tasks.workunit.client.0.smithi069.stderr: 99850K .......... .......... .......... .......... .......... 73% 4.20M 4s 2024-03-19T01:11:50.153 INFO:tasks.workunit.client.0.smithi069.stderr: 99900K .......... .......... .......... .......... .......... 73% 11.8M 4s 2024-03-19T01:11:50.158 INFO:tasks.workunit.client.0.smithi069.stderr: 99950K .......... .......... .......... .......... .......... 73% 9.95M 4s 2024-03-19T01:11:50.163 INFO:tasks.workunit.client.0.smithi069.stderr:100000K .......... .......... .......... .......... .......... 73% 9.43M 4s 2024-03-19T01:11:50.168 INFO:tasks.workunit.client.0.smithi069.stderr:100050K .......... .......... .......... .......... .......... 73% 9.78M 4s 2024-03-19T01:11:50.173 INFO:tasks.workunit.client.0.smithi069.stderr:100100K .......... .......... .......... .......... .......... 73% 9.53M 4s 2024-03-19T01:11:50.181 INFO:tasks.workunit.client.0.smithi069.stderr:100150K .......... .......... .......... .......... .......... 73% 5.84M 4s 2024-03-19T01:11:50.186 INFO:tasks.workunit.client.0.smithi069.stderr:100200K .......... .......... .......... .......... .......... 73% 10.1M 4s 2024-03-19T01:11:50.192 INFO:tasks.workunit.client.0.smithi069.stderr:100250K .......... .......... .......... .......... .......... 73% 8.51M 4s 2024-03-19T01:11:50.198 INFO:tasks.workunit.client.0.smithi069.stderr:100300K .......... .......... .......... .......... .......... 73% 7.92M 4s 2024-03-19T01:11:50.203 INFO:tasks.workunit.client.0.smithi069.stderr:100350K .......... .......... .......... .......... .......... 73% 10.1M 4s 2024-03-19T01:11:50.208 INFO:tasks.workunit.client.0.smithi069.stderr:100400K .......... .......... .......... .......... .......... 73% 10.2M 4s 2024-03-19T01:11:50.214 INFO:tasks.workunit.client.0.smithi069.stderr:100450K .......... .......... .......... .......... .......... 74% 7.83M 4s 2024-03-19T01:11:50.219 INFO:tasks.workunit.client.0.smithi069.stderr:100500K .......... .......... .......... .......... .......... 74% 9.46M 4s 2024-03-19T01:11:50.226 INFO:tasks.workunit.client.0.smithi069.stderr:100550K .......... .......... .......... .......... .......... 74% 7.59M 4s 2024-03-19T01:11:50.231 INFO:tasks.workunit.client.0.smithi069.stderr:100600K .......... .......... .......... .......... .......... 74% 7.98M 4s 2024-03-19T01:11:50.237 INFO:tasks.workunit.client.0.smithi069.stderr:100650K .......... .......... .......... .......... .......... 74% 9.84M 4s 2024-03-19T01:11:50.242 INFO:tasks.workunit.client.0.smithi069.stderr:100700K .......... .......... .......... .......... .......... 74% 9.59M 4s 2024-03-19T01:11:50.246 INFO:tasks.workunit.client.0.smithi069.stderr:100750K .......... .......... .......... .......... .......... 74% 9.75M 4s 2024-03-19T01:11:50.253 INFO:tasks.workunit.client.0.smithi069.stderr:100800K .......... .......... .......... .......... .......... 74% 7.96M 4s 2024-03-19T01:11:50.260 INFO:tasks.workunit.client.0.smithi069.stderr:100850K .......... .......... .......... .......... .......... 74% 7.05M 4s 2024-03-19T01:11:50.266 INFO:tasks.workunit.client.0.smithi069.stderr:100900K .......... .......... .......... .......... .......... 74% 7.27M 4s 2024-03-19T01:11:50.272 INFO:tasks.workunit.client.0.smithi069.stderr:100950K .......... .......... .......... .......... .......... 74% 8.95M 4s 2024-03-19T01:11:50.276 INFO:tasks.workunit.client.0.smithi069.stderr:101000K .......... .......... .......... .......... .......... 74% 10.9M 4s 2024-03-19T01:11:50.283 INFO:tasks.workunit.client.0.smithi069.stderr:101050K .......... .......... .......... .......... .......... 74% 7.67M 4s 2024-03-19T01:11:50.289 INFO:tasks.workunit.client.0.smithi069.stderr:101100K .......... .......... .......... .......... .......... 74% 8.91M 4s 2024-03-19T01:11:50.293 INFO:tasks.workunit.client.0.smithi069.stderr:101150K .......... .......... .......... .......... .......... 74% 9.52M 4s 2024-03-19T01:11:50.301 INFO:tasks.workunit.client.0.smithi069.stderr:101200K .......... .......... .......... .......... .......... 74% 6.63M 4s 2024-03-19T01:11:50.308 INFO:tasks.workunit.client.0.smithi069.stderr:101250K .......... .......... .......... .......... .......... 74% 6.41M 4s 2024-03-19T01:11:50.314 INFO:tasks.workunit.client.0.smithi069.stderr:101300K .......... .......... .......... .......... .......... 74% 8.90M 4s 2024-03-19T01:11:50.321 INFO:tasks.workunit.client.0.smithi069.stderr:101350K .......... .......... .......... .......... .......... 74% 6.91M 4s 2024-03-19T01:11:50.326 INFO:tasks.workunit.client.0.smithi069.stderr:101400K .......... .......... .......... .......... .......... 74% 9.17M 4s 2024-03-19T01:11:50.331 INFO:tasks.workunit.client.0.smithi069.stderr:101450K .......... .......... .......... .......... .......... 74% 9.55M 4s 2024-03-19T01:11:50.337 INFO:tasks.workunit.client.0.smithi069.stderr:101500K .......... .......... .......... .......... .......... 74% 9.96M 4s 2024-03-19T01:11:50.345 INFO:tasks.workunit.client.0.smithi069.stderr:101550K .......... .......... .......... .......... .......... 74% 6.03M 4s 2024-03-19T01:11:50.351 INFO:tasks.workunit.client.0.smithi069.stderr:101600K .......... .......... .......... .......... .......... 74% 7.38M 4s 2024-03-19T01:11:50.357 INFO:tasks.workunit.client.0.smithi069.stderr:101650K .......... .......... .......... .......... .......... 74% 8.38M 4s 2024-03-19T01:11:50.363 INFO:tasks.workunit.client.0.smithi069.stderr:101700K .......... .......... .......... .......... .......... 74% 8.00M 4s 2024-03-19T01:11:50.369 INFO:tasks.workunit.client.0.smithi069.stderr:101750K .......... .......... .......... .......... .......... 74% 8.91M 4s 2024-03-19T01:11:50.374 INFO:tasks.workunit.client.0.smithi069.stderr:101800K .......... .......... .......... .......... .......... 75% 9.32M 4s 2024-03-19T01:11:50.382 INFO:tasks.workunit.client.0.smithi069.stderr:101850K .......... .......... .......... .......... .......... 75% 6.06M 4s 2024-03-19T01:11:50.388 INFO:tasks.workunit.client.0.smithi069.stderr:101900K .......... .......... .......... .......... .......... 75% 7.79M 4s 2024-03-19T01:11:50.394 INFO:tasks.workunit.client.0.smithi069.stderr:101950K .......... .......... .......... .......... .......... 75% 7.63M 4s 2024-03-19T01:11:50.400 INFO:tasks.workunit.client.0.smithi069.stderr:102000K .......... .......... .......... .......... .......... 75% 9.81M 4s 2024-03-19T01:11:50.405 INFO:tasks.workunit.client.0.smithi069.stderr:102050K .......... .......... .......... .......... .......... 75% 8.00M 4s 2024-03-19T01:11:50.411 INFO:tasks.workunit.client.0.smithi069.stderr:102100K .......... .......... .......... .......... .......... 75% 8.17M 4s 2024-03-19T01:11:50.420 INFO:tasks.workunit.client.0.smithi069.stderr:102150K .......... .......... .......... .......... .......... 75% 7.96M 4s 2024-03-19T01:11:50.427 INFO:tasks.workunit.client.0.smithi069.stderr:102200K .......... .......... .......... .......... .......... 75% 5.24M 4s 2024-03-19T01:11:50.434 INFO:tasks.workunit.client.0.smithi069.stderr:102250K .......... .......... .......... .......... .......... 75% 6.60M 4s 2024-03-19T01:11:50.442 INFO:tasks.workunit.client.0.smithi069.stderr:102300K .......... .......... .......... .......... .......... 75% 6.66M 4s 2024-03-19T01:11:50.448 INFO:tasks.workunit.client.0.smithi069.stderr:102350K .......... .......... .......... .......... .......... 75% 8.21M 4s 2024-03-19T01:11:50.454 INFO:tasks.workunit.client.0.smithi069.stderr:102400K .......... .......... .......... .......... .......... 75% 10.6M 4s 2024-03-19T01:11:50.460 INFO:tasks.workunit.client.0.smithi069.stderr:102450K .......... .......... .......... .......... .......... 75% 6.58M 4s 2024-03-19T01:11:50.468 INFO:tasks.workunit.client.0.smithi069.stderr:102500K .......... .......... .......... .......... .......... 75% 5.59M 4s 2024-03-19T01:11:50.477 INFO:tasks.workunit.client.0.smithi069.stderr:102550K .......... .......... .......... .......... .......... 75% 5.59M 4s 2024-03-19T01:11:50.484 INFO:tasks.workunit.client.0.smithi069.stderr:102600K .......... .......... .......... .......... .......... 75% 8.12M 4s 2024-03-19T01:11:50.490 INFO:tasks.workunit.client.0.smithi069.stderr:102650K .......... .......... .......... .......... .......... 75% 6.77M 4s 2024-03-19T01:11:50.496 INFO:tasks.workunit.client.0.smithi069.stderr:102700K .......... .......... .......... .......... .......... 75% 8.10M 4s 2024-03-19T01:11:50.502 INFO:tasks.workunit.client.0.smithi069.stderr:102750K .......... .......... .......... .......... .......... 75% 8.21M 4s 2024-03-19T01:11:50.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:50 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:11:50.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:50 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:11:50.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:50 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:11:50.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:50 smithi086 ceph-mon[66829]: pgmap v38: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-03-19T01:11:50.509 INFO:tasks.workunit.client.0.smithi069.stderr:102800K .......... .......... .......... .......... .......... 75% 7.10M 4s 2024-03-19T01:11:50.519 INFO:tasks.workunit.client.0.smithi069.stderr:102850K .......... .......... .......... .......... .......... 75% 5.34M 4s 2024-03-19T01:11:50.526 INFO:tasks.workunit.client.0.smithi069.stderr:102900K .......... .......... .......... .......... .......... 75% 6.40M 4s 2024-03-19T01:11:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:50 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:11:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:50 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:11:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:50 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:11:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:50 smithi069 ceph-mon[31727]: pgmap v38: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-03-19T01:11:50.532 INFO:tasks.workunit.client.0.smithi069.stderr:102950K .......... .......... .......... .......... .......... 75% 8.32M 4s 2024-03-19T01:11:50.538 INFO:tasks.workunit.client.0.smithi069.stderr:103000K .......... .......... .......... .......... .......... 75% 7.08M 4s 2024-03-19T01:11:50.546 INFO:tasks.workunit.client.0.smithi069.stderr:103050K .......... .......... .......... .......... .......... 75% 6.71M 4s 2024-03-19T01:11:50.553 INFO:tasks.workunit.client.0.smithi069.stderr:103100K .......... .......... .......... .......... .......... 75% 6.43M 4s 2024-03-19T01:11:50.560 INFO:tasks.workunit.client.0.smithi069.stderr:103150K .......... .......... .......... .......... .......... 76% 7.09M 4s 2024-03-19T01:11:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:50 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:11:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:50 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:11:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:50 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:11:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:50 smithi163 ceph-mon[67428]: pgmap v38: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-03-19T01:11:50.570 INFO:tasks.workunit.client.0.smithi069.stderr:103200K .......... .......... .......... .......... .......... 76% 5.43M 4s 2024-03-19T01:11:50.576 INFO:tasks.workunit.client.0.smithi069.stderr:103250K .......... .......... .......... .......... .......... 76% 8.07M 4s 2024-03-19T01:11:50.581 INFO:tasks.workunit.client.0.smithi069.stderr:103300K .......... .......... .......... .......... .......... 76% 8.53M 4s 2024-03-19T01:11:50.587 INFO:tasks.workunit.client.0.smithi069.stderr:103350K .......... .......... .......... .......... .......... 76% 8.97M 4s 2024-03-19T01:11:50.595 INFO:tasks.workunit.client.0.smithi069.stderr:103400K .......... .......... .......... .......... .......... 76% 6.98M 4s 2024-03-19T01:11:50.602 INFO:tasks.workunit.client.0.smithi069.stderr:103450K .......... .......... .......... .......... .......... 76% 5.57M 4s 2024-03-19T01:11:50.610 INFO:tasks.workunit.client.0.smithi069.stderr:103500K .......... .......... .......... .......... .......... 76% 6.00M 4s 2024-03-19T01:11:50.619 INFO:tasks.workunit.client.0.smithi069.stderr:103550K .......... .......... .......... .......... .......... 76% 5.90M 4s 2024-03-19T01:11:50.625 INFO:tasks.workunit.client.0.smithi069.stderr:103600K .......... .......... .......... .......... .......... 76% 7.24M 4s 2024-03-19T01:11:50.631 INFO:tasks.workunit.client.0.smithi069.stderr:103650K .......... .......... .......... .......... .......... 76% 8.69M 4s 2024-03-19T01:11:50.638 INFO:tasks.workunit.client.0.smithi069.stderr:103700K .......... .......... .......... .......... .......... 76% 6.96M 4s 2024-03-19T01:11:50.644 INFO:tasks.workunit.client.0.smithi069.stderr:103750K .......... .......... .......... .......... .......... 76% 7.90M 4s 2024-03-19T01:11:50.652 INFO:tasks.workunit.client.0.smithi069.stderr:103800K .......... .......... .......... .......... .......... 76% 5.96M 4s 2024-03-19T01:11:50.662 INFO:tasks.workunit.client.0.smithi069.stderr:103850K .......... .......... .......... .......... .......... 76% 4.91M 4s 2024-03-19T01:11:50.669 INFO:tasks.workunit.client.0.smithi069.stderr:103900K .......... .......... .......... .......... .......... 76% 7.65M 4s 2024-03-19T01:11:50.676 INFO:tasks.workunit.client.0.smithi069.stderr:103950K .......... .......... .......... .......... .......... 76% 7.05M 4s 2024-03-19T01:11:50.680 INFO:tasks.workunit.client.0.smithi069.stderr:104000K .......... .......... .......... .......... .......... 76% 11.1M 4s 2024-03-19T01:11:50.690 INFO:tasks.workunit.client.0.smithi069.stderr:104050K .......... .......... .......... .......... .......... 76% 5.18M 4s 2024-03-19T01:11:50.697 INFO:tasks.workunit.client.0.smithi069.stderr:104100K .......... .......... .......... .......... .......... 76% 6.35M 4s 2024-03-19T01:11:50.707 INFO:tasks.workunit.client.0.smithi069.stderr:104150K .......... .......... .......... .......... .......... 76% 5.82M 4s 2024-03-19T01:11:50.715 INFO:tasks.workunit.client.0.smithi069.stderr:104200K .......... .......... .......... .......... .......... 76% 5.14M 4s 2024-03-19T01:11:50.722 INFO:tasks.workunit.client.0.smithi069.stderr:104250K .......... .......... .......... .......... .......... 76% 7.54M 4s 2024-03-19T01:11:50.728 INFO:tasks.workunit.client.0.smithi069.stderr:104300K .......... .......... .......... .......... .......... 76% 7.54M 4s 2024-03-19T01:11:50.736 INFO:tasks.workunit.client.0.smithi069.stderr:104350K .......... .......... .......... .......... .......... 76% 6.47M 4s 2024-03-19T01:11:50.744 INFO:tasks.workunit.client.0.smithi069.stderr:104400K .......... .......... .......... .......... .......... 76% 6.58M 4s 2024-03-19T01:11:50.755 INFO:tasks.workunit.client.0.smithi069.stderr:104450K .......... .......... .......... .......... .......... 76% 4.06M 4s 2024-03-19T01:11:50.766 INFO:tasks.workunit.client.0.smithi069.stderr:104500K .......... .......... .......... .......... .......... 77% 6.61M 4s 2024-03-19T01:11:50.772 INFO:tasks.workunit.client.0.smithi069.stderr:104550K .......... .......... .......... .......... .......... 77% 5.29M 4s 2024-03-19T01:11:50.779 INFO:tasks.workunit.client.0.smithi069.stderr:104600K .......... .......... .......... .......... .......... 77% 7.25M 4s 2024-03-19T01:11:50.785 INFO:tasks.workunit.client.0.smithi069.stderr:104650K .......... .......... .......... .......... .......... 77% 6.84M 4s 2024-03-19T01:11:50.795 INFO:tasks.workunit.client.0.smithi069.stderr:104700K .......... .......... .......... .......... .......... 77% 5.59M 4s 2024-03-19T01:11:50.805 INFO:tasks.workunit.client.0.smithi069.stderr:104750K .......... .......... .......... .......... .......... 77% 5.13M 3s 2024-03-19T01:11:50.817 INFO:tasks.workunit.client.0.smithi069.stderr:104800K .......... .......... .......... .......... .......... 77% 3.81M 3s 2024-03-19T01:11:50.823 INFO:tasks.workunit.client.0.smithi069.stderr:104850K .......... .......... .......... .......... .......... 77% 8.08M 3s 2024-03-19T01:11:50.829 INFO:tasks.workunit.client.0.smithi069.stderr:104900K .......... .......... .......... .......... .......... 77% 7.45M 3s 2024-03-19T01:11:50.835 INFO:tasks.workunit.client.0.smithi069.stderr:104950K .......... .......... .......... .......... .......... 77% 8.62M 3s 2024-03-19T01:11:50.842 INFO:tasks.workunit.client.0.smithi069.stderr:105000K .......... .......... .......... .......... .......... 77% 6.39M 3s 2024-03-19T01:11:50.855 INFO:tasks.workunit.client.0.smithi069.stderr:105050K .......... .......... .......... .......... .......... 77% 3.99M 3s 2024-03-19T01:11:50.866 INFO:tasks.workunit.client.0.smithi069.stderr:105100K .......... .......... .......... .......... .......... 77% 4.37M 3s 2024-03-19T01:11:50.871 INFO:tasks.workunit.client.0.smithi069.stderr:105150K .......... .......... .......... .......... .......... 77% 11.3M 3s 2024-03-19T01:11:50.873 INFO:tasks.workunit.client.0.smithi069.stderr:105200K .......... .......... .......... .......... .......... 77% 21.9M 3s 2024-03-19T01:11:50.877 INFO:tasks.workunit.client.0.smithi069.stderr:105250K .......... .......... .......... .......... .......... 77% 11.5M 3s 2024-03-19T01:11:50.880 INFO:tasks.workunit.client.0.smithi069.stderr:105300K .......... .......... .......... .......... .......... 77% 17.4M 3s 2024-03-19T01:11:50.883 INFO:tasks.workunit.client.0.smithi069.stderr:105350K .......... .......... .......... .......... .......... 77% 13.1M 3s 2024-03-19T01:11:50.889 INFO:tasks.workunit.client.0.smithi069.stderr:105400K .......... .......... .......... .......... .......... 77% 8.05M 3s 2024-03-19T01:11:50.897 INFO:tasks.workunit.client.0.smithi069.stderr:105450K .......... .......... .......... .......... .......... 77% 6.93M 3s 2024-03-19T01:11:50.899 INFO:tasks.workunit.client.0.smithi069.stderr:105500K .......... .......... .......... .......... .......... 77% 17.8M 3s 2024-03-19T01:11:50.902 INFO:tasks.workunit.client.0.smithi069.stderr:105550K .......... .......... .......... .......... .......... 77% 14.5M 3s 2024-03-19T01:11:50.906 INFO:tasks.workunit.client.0.smithi069.stderr:105600K .......... .......... .......... .......... .......... 77% 15.1M 3s 2024-03-19T01:11:50.909 INFO:tasks.workunit.client.0.smithi069.stderr:105650K .......... .......... .......... .......... .......... 77% 15.9M 3s 2024-03-19T01:11:50.913 INFO:tasks.workunit.client.0.smithi069.stderr:105700K .......... .......... .......... .......... .......... 77% 12.2M 3s 2024-03-19T01:11:50.918 INFO:tasks.workunit.client.0.smithi069.stderr:105750K .......... .......... .......... .......... .......... 77% 9.94M 3s 2024-03-19T01:11:50.922 INFO:tasks.workunit.client.0.smithi069.stderr:105800K .......... .......... .......... .......... .......... 77% 13.5M 3s 2024-03-19T01:11:50.925 INFO:tasks.workunit.client.0.smithi069.stderr:105850K .......... .......... .......... .......... .......... 78% 15.4M 3s 2024-03-19T01:11:50.929 INFO:tasks.workunit.client.0.smithi069.stderr:105900K .......... .......... .......... .......... .......... 78% 14.2M 3s 2024-03-19T01:11:50.930 INFO:tasks.workunit.client.0.smithi069.stderr:105950K .......... .......... .......... .......... .......... 78% 22.4M 3s 2024-03-19T01:11:50.933 INFO:tasks.workunit.client.0.smithi069.stderr:106000K .......... .......... .......... .......... .......... 78% 15.3M 3s 2024-03-19T01:11:50.937 INFO:tasks.workunit.client.0.smithi069.stderr:106050K .......... .......... .......... .......... .......... 78% 14.6M 3s 2024-03-19T01:11:50.940 INFO:tasks.workunit.client.0.smithi069.stderr:106100K .......... .......... .......... .......... .......... 78% 13.5M 3s 2024-03-19T01:11:50.943 INFO:tasks.workunit.client.0.smithi069.stderr:106150K .......... .......... .......... .......... .......... 78% 15.7M 3s 2024-03-19T01:11:50.949 INFO:tasks.workunit.client.0.smithi069.stderr:106200K .......... .......... .......... .......... .......... 78% 10.7M 3s 2024-03-19T01:11:50.950 INFO:tasks.workunit.client.0.smithi069.stderr:106250K .......... .......... .......... .......... .......... 78% 26.9M 3s 2024-03-19T01:11:50.952 INFO:tasks.workunit.client.0.smithi069.stderr:106300K .......... .......... .......... .......... .......... 78% 21.2M 3s 2024-03-19T01:11:50.957 INFO:tasks.workunit.client.0.smithi069.stderr:106350K .......... .......... .......... .......... .......... 78% 11.1M 3s 2024-03-19T01:11:50.960 INFO:tasks.workunit.client.0.smithi069.stderr:106400K .......... .......... .......... .......... .......... 78% 14.9M 3s 2024-03-19T01:11:50.964 INFO:tasks.workunit.client.0.smithi069.stderr:106450K .......... .......... .......... .......... .......... 78% 14.7M 3s 2024-03-19T01:11:50.967 INFO:tasks.workunit.client.0.smithi069.stderr:106500K .......... .......... .......... .......... .......... 78% 15.4M 3s 2024-03-19T01:11:50.972 INFO:tasks.workunit.client.0.smithi069.stderr:106550K .......... .......... .......... .......... .......... 78% 18.0M 3s 2024-03-19T01:11:50.972 INFO:tasks.workunit.client.0.smithi069.stderr:106600K .......... .......... .......... .......... .......... 78% 17.6M 3s 2024-03-19T01:11:50.976 INFO:tasks.workunit.client.0.smithi069.stderr:106650K .......... .......... .......... .......... .......... 78% 12.1M 3s 2024-03-19T01:11:50.979 INFO:tasks.workunit.client.0.smithi069.stderr:106700K .......... .......... .......... .......... .......... 78% 14.8M 3s 2024-03-19T01:11:50.983 INFO:tasks.workunit.client.0.smithi069.stderr:106750K .......... .......... .......... .......... .......... 78% 14.9M 3s 2024-03-19T01:11:50.985 INFO:tasks.workunit.client.0.smithi069.stderr:106800K .......... .......... .......... .......... .......... 78% 23.4M 3s 2024-03-19T01:11:50.990 INFO:tasks.workunit.client.0.smithi069.stderr:106850K .......... .......... .......... .......... .......... 78% 10.5M 3s 2024-03-19T01:11:50.991 INFO:tasks.workunit.client.0.smithi069.stderr:106900K .......... .......... .......... .......... .......... 78% 25.2M 3s 2024-03-19T01:11:50.996 INFO:tasks.workunit.client.0.smithi069.stderr:106950K .......... .......... .......... .......... .......... 78% 14.0M 3s 2024-03-19T01:11:51.000 INFO:tasks.workunit.client.0.smithi069.stderr:107000K .......... .......... .......... .......... .......... 78% 10.5M 3s 2024-03-19T01:11:51.001 INFO:tasks.workunit.client.0.smithi069.stderr:107050K .......... .......... .......... .......... .......... 78% 22.3M 3s 2024-03-19T01:11:51.005 INFO:tasks.workunit.client.0.smithi069.stderr:107100K .......... .......... .......... .......... .......... 78% 16.3M 3s 2024-03-19T01:11:51.007 INFO:tasks.workunit.client.0.smithi069.stderr:107150K .......... .......... .......... .......... .......... 78% 17.9M 3s 2024-03-19T01:11:51.011 INFO:tasks.workunit.client.0.smithi069.stderr:107200K .......... .......... .......... .......... .......... 79% 11.8M 3s 2024-03-19T01:11:51.015 INFO:tasks.workunit.client.0.smithi069.stderr:107250K .......... .......... .......... .......... .......... 79% 14.8M 3s 2024-03-19T01:11:51.018 INFO:tasks.workunit.client.0.smithi069.stderr:107300K .......... .......... .......... .......... .......... 79% 16.0M 3s 2024-03-19T01:11:51.022 INFO:tasks.workunit.client.0.smithi069.stderr:107350K .......... .......... .......... .......... .......... 79% 12.9M 3s 2024-03-19T01:11:51.024 INFO:tasks.workunit.client.0.smithi069.stderr:107400K .......... .......... .......... .......... .......... 79% 17.5M 3s 2024-03-19T01:11:51.028 INFO:tasks.workunit.client.0.smithi069.stderr:107450K .......... .......... .......... .......... .......... 79% 15.5M 3s 2024-03-19T01:11:51.032 INFO:tasks.workunit.client.0.smithi069.stderr:107500K .......... .......... .......... .......... .......... 79% 13.2M 3s 2024-03-19T01:11:51.033 INFO:tasks.workunit.client.0.smithi069.stderr:107550K .......... .......... .......... .......... .......... 79% 25.4M 3s 2024-03-19T01:11:51.038 INFO:tasks.workunit.client.0.smithi069.stderr:107600K .......... .......... .......... .......... .......... 79% 10.1M 3s 2024-03-19T01:11:51.041 INFO:tasks.workunit.client.0.smithi069.stderr:107650K .......... .......... .......... .......... .......... 79% 15.4M 3s 2024-03-19T01:11:51.045 INFO:tasks.workunit.client.0.smithi069.stderr:107700K .......... .......... .......... .......... .......... 79% 14.6M 3s 2024-03-19T01:11:51.046 INFO:tasks.workunit.client.0.smithi069.stderr:107750K .......... .......... .......... .......... .......... 79% 24.5M 3s 2024-03-19T01:11:51.050 INFO:tasks.workunit.client.0.smithi069.stderr:107800K .......... .......... .......... .......... .......... 79% 13.1M 3s 2024-03-19T01:11:51.053 INFO:tasks.workunit.client.0.smithi069.stderr:107850K .......... .......... .......... .......... .......... 79% 15.4M 3s 2024-03-19T01:11:51.057 INFO:tasks.workunit.client.0.smithi069.stderr:107900K .......... .......... .......... .......... .......... 79% 13.0M 3s 2024-03-19T01:11:51.061 INFO:tasks.workunit.client.0.smithi069.stderr:107950K .......... .......... .......... .......... .......... 79% 11.7M 3s 2024-03-19T01:11:51.064 INFO:tasks.workunit.client.0.smithi069.stderr:108000K .......... .......... .......... .......... .......... 79% 20.8M 3s 2024-03-19T01:11:51.065 INFO:tasks.workunit.client.0.smithi069.stderr:108050K .......... .......... .......... .......... .......... 79% 26.3M 3s 2024-03-19T01:11:51.070 INFO:tasks.workunit.client.0.smithi069.stderr:108100K .......... .......... .......... .......... .......... 79% 10.1M 3s 2024-03-19T01:11:51.072 INFO:tasks.workunit.client.0.smithi069.stderr:108150K .......... .......... .......... .......... .......... 79% 23.1M 3s 2024-03-19T01:11:51.078 INFO:tasks.workunit.client.0.smithi069.stderr:108200K .......... .......... .......... .......... .......... 79% 9.84M 3s 2024-03-19T01:11:51.081 INFO:tasks.workunit.client.0.smithi069.stderr:108250K .......... .......... .......... .......... .......... 79% 16.0M 3s 2024-03-19T01:11:51.083 INFO:tasks.workunit.client.0.smithi069.stderr:108300K .......... .......... .......... .......... .......... 79% 22.3M 3s 2024-03-19T01:11:51.086 INFO:tasks.workunit.client.0.smithi069.stderr:108350K .......... .......... .......... .......... .......... 79% 16.0M 3s 2024-03-19T01:11:51.088 INFO:tasks.workunit.client.0.smithi069.stderr:108400K .......... .......... .......... .......... .......... 79% 17.7M 3s 2024-03-19T01:11:51.092 INFO:tasks.workunit.client.0.smithi069.stderr:108450K .......... .......... .......... .......... .......... 79% 11.9M 3s 2024-03-19T01:11:51.096 INFO:tasks.workunit.client.0.smithi069.stderr:108500K .......... .......... .......... .......... .......... 79% 13.9M 3s 2024-03-19T01:11:51.099 INFO:tasks.workunit.client.0.smithi069.stderr:108550K .......... .......... .......... .......... .......... 79% 18.5M 3s 2024-03-19T01:11:51.103 INFO:tasks.workunit.client.0.smithi069.stderr:108600K .......... .......... .......... .......... .......... 80% 12.1M 3s 2024-03-19T01:11:51.106 INFO:tasks.workunit.client.0.smithi069.stderr:108650K .......... .......... .......... .......... .......... 80% 16.9M 3s 2024-03-19T01:11:51.109 INFO:tasks.workunit.client.0.smithi069.stderr:108700K .......... .......... .......... .......... .......... 80% 17.2M 3s 2024-03-19T01:11:51.112 INFO:tasks.workunit.client.0.smithi069.stderr:108750K .......... .......... .......... .......... .......... 80% 12.5M 3s 2024-03-19T01:11:51.114 INFO:tasks.workunit.client.0.smithi069.stderr:108800K .......... .......... .......... .......... .......... 80% 26.0M 3s 2024-03-19T01:11:51.119 INFO:tasks.workunit.client.0.smithi069.stderr:108850K .......... .......... .......... .......... .......... 80% 10.6M 3s 2024-03-19T01:11:51.123 INFO:tasks.workunit.client.0.smithi069.stderr:108900K .......... .......... .......... .......... .......... 80% 14.3M 3s 2024-03-19T01:11:51.126 INFO:tasks.workunit.client.0.smithi069.stderr:108950K .......... .......... .......... .......... .......... 80% 14.3M 3s 2024-03-19T01:11:51.128 INFO:tasks.workunit.client.0.smithi069.stderr:109000K .......... .......... .......... .......... .......... 80% 25.1M 3s 2024-03-19T01:11:51.131 INFO:tasks.workunit.client.0.smithi069.stderr:109050K .......... .......... .......... .......... .......... 80% 13.3M 3s 2024-03-19T01:11:51.135 INFO:tasks.workunit.client.0.smithi069.stderr:109100K .......... .......... .......... .......... .......... 80% 15.0M 3s 2024-03-19T01:11:51.139 INFO:tasks.workunit.client.0.smithi069.stderr:109150K .......... .......... .......... .......... .......... 80% 12.0M 3s 2024-03-19T01:11:51.143 INFO:tasks.workunit.client.0.smithi069.stderr:109200K .......... .......... .......... .......... .......... 80% 12.0M 3s 2024-03-19T01:11:51.145 INFO:tasks.workunit.client.0.smithi069.stderr:109250K .......... .......... .......... .......... .......... 80% 17.4M 3s 2024-03-19T01:11:51.148 INFO:tasks.workunit.client.0.smithi069.stderr:109300K .......... .......... .......... .......... .......... 80% 21.0M 3s 2024-03-19T01:11:51.153 INFO:tasks.workunit.client.0.smithi069.stderr:109350K .......... .......... .......... .......... .......... 80% 9.57M 3s 2024-03-19T01:11:51.155 INFO:tasks.workunit.client.0.smithi069.stderr:109400K .......... .......... .......... .......... .......... 80% 27.5M 3s 2024-03-19T01:11:51.159 INFO:tasks.workunit.client.0.smithi069.stderr:109450K .......... .......... .......... .......... .......... 80% 11.6M 3s 2024-03-19T01:11:51.162 INFO:tasks.workunit.client.0.smithi069.stderr:109500K .......... .......... .......... .......... .......... 80% 16.6M 3s 2024-03-19T01:11:51.165 INFO:tasks.workunit.client.0.smithi069.stderr:109550K .......... .......... .......... .......... .......... 80% 16.5M 3s 2024-03-19T01:11:51.169 INFO:tasks.workunit.client.0.smithi069.stderr:109600K .......... .......... .......... .......... .......... 80% 11.1M 3s 2024-03-19T01:11:51.176 INFO:tasks.workunit.client.0.smithi069.stderr:109650K .......... .......... .......... .......... .......... 80% 16.9M 3s 2024-03-19T01:11:51.176 INFO:tasks.workunit.client.0.smithi069.stderr:109700K .......... .......... .......... .......... .......... 80% 19.6M 3s 2024-03-19T01:11:51.177 INFO:tasks.workunit.client.0.smithi069.stderr:109750K .......... .......... .......... .......... .......... 80% 18.3M 3s 2024-03-19T01:11:51.180 INFO:tasks.workunit.client.0.smithi069.stderr:109800K .......... .......... .......... .......... .......... 80% 18.2M 3s 2024-03-19T01:11:51.185 INFO:tasks.workunit.client.0.smithi069.stderr:109850K .......... .......... .......... .......... .......... 80% 10.2M 3s 2024-03-19T01:11:51.190 INFO:tasks.workunit.client.0.smithi069.stderr:109900K .......... .......... .......... .......... .......... 80% 10.7M 3s 2024-03-19T01:11:51.191 INFO:tasks.workunit.client.0.smithi069.stderr:109950K .......... .......... .......... .......... .......... 81% 31.6M 3s 2024-03-19T01:11:51.195 INFO:tasks.workunit.client.0.smithi069.stderr:110000K .......... .......... .......... .......... .......... 81% 12.8M 3s 2024-03-19T01:11:51.195 INFO:tasks.workunit.client.0.smithi069.stderr:110050K .......... .......... .......... .......... .......... 81% 88.0M 3s 2024-03-19T01:11:51.200 INFO:tasks.workunit.client.0.smithi069.stderr:110100K .......... .......... .......... .......... .......... 81% 10.5M 3s 2024-03-19T01:11:51.203 INFO:tasks.workunit.client.0.smithi069.stderr:110150K .......... .......... .......... .......... .......... 81% 17.3M 3s 2024-03-19T01:11:51.205 INFO:tasks.workunit.client.0.smithi069.stderr:110200K .......... .......... .......... .......... .......... 81% 17.2M 3s 2024-03-19T01:11:51.210 INFO:tasks.workunit.client.0.smithi069.stderr:110250K .......... .......... .......... .......... .......... 81% 10.8M 3s 2024-03-19T01:11:51.216 INFO:tasks.workunit.client.0.smithi069.stderr:110300K .......... .......... .......... .......... .......... 81% 10.3M 3s 2024-03-19T01:11:51.216 INFO:tasks.workunit.client.0.smithi069.stderr:110350K .......... .......... .......... .......... .......... 81% 81.1M 3s 2024-03-19T01:11:51.221 INFO:tasks.workunit.client.0.smithi069.stderr:110400K .......... .......... .......... .......... .......... 81% 18.5M 3s 2024-03-19T01:11:51.221 INFO:tasks.workunit.client.0.smithi069.stderr:110450K .......... .......... .......... .......... .......... 81% 18.6M 3s 2024-03-19T01:11:51.226 INFO:tasks.workunit.client.0.smithi069.stderr:110500K .......... .......... .......... .......... .......... 81% 13.1M 3s 2024-03-19T01:11:51.230 INFO:tasks.workunit.client.0.smithi069.stderr:110550K .......... .......... .......... .......... .......... 81% 8.52M 3s 2024-03-19T01:11:51.231 INFO:tasks.workunit.client.0.smithi069.stderr:110600K .......... .......... .......... .......... .......... 81% 33.8M 3s 2024-03-19T01:11:51.236 INFO:tasks.workunit.client.0.smithi069.stderr:110650K .......... .......... .......... .......... .......... 81% 12.9M 3s 2024-03-19T01:11:51.239 INFO:tasks.workunit.client.0.smithi069.stderr:110700K .......... .......... .......... .......... .......... 81% 16.8M 3s 2024-03-19T01:11:51.241 INFO:tasks.workunit.client.0.smithi069.stderr:110750K .......... .......... .......... .......... .......... 81% 18.7M 3s 2024-03-19T01:11:51.245 INFO:tasks.workunit.client.0.smithi069.stderr:110800K .......... .......... .......... .......... .......... 81% 13.1M 3s 2024-03-19T01:11:51.246 INFO:tasks.workunit.client.0.smithi069.stderr:110850K .......... .......... .......... .......... .......... 81% 29.0M 3s 2024-03-19T01:11:51.251 INFO:tasks.workunit.client.0.smithi069.stderr:110900K .......... .......... .......... .......... .......... 81% 11.1M 3s 2024-03-19T01:11:51.256 INFO:tasks.workunit.client.0.smithi069.stderr:110950K .......... .......... .......... .......... .......... 81% 9.99M 3s 2024-03-19T01:11:51.256 INFO:tasks.workunit.client.0.smithi069.stderr:111000K .......... .......... .......... .......... .......... 81% 110M 3s 2024-03-19T01:11:51.261 INFO:tasks.workunit.client.0.smithi069.stderr:111050K .......... .......... .......... .......... .......... 81% 9.76M 3s 2024-03-19T01:11:51.262 INFO:tasks.workunit.client.0.smithi069.stderr:111100K .......... .......... .......... .......... .......... 81% 111M 3s 2024-03-19T01:11:51.267 INFO:tasks.workunit.client.0.smithi069.stderr:111150K .......... .......... .......... .......... .......... 81% 10.4M 3s 2024-03-19T01:11:51.268 INFO:tasks.workunit.client.0.smithi069.stderr:111200K .......... .......... .......... .......... .......... 81% 23.9M 3s 2024-03-19T01:11:51.272 INFO:tasks.workunit.client.0.smithi069.stderr:111250K .......... .......... .......... .......... .......... 81% 12.0M 3s 2024-03-19T01:11:51.277 INFO:tasks.workunit.client.0.smithi069.stderr:111300K .......... .......... .......... .......... .......... 82% 12.5M 3s 2024-03-19T01:11:51.279 INFO:tasks.workunit.client.0.smithi069.stderr:111350K .......... .......... .......... .......... .......... 82% 61.5M 3s 2024-03-19T01:11:51.282 INFO:tasks.workunit.client.0.smithi069.stderr:111400K .......... .......... .......... .......... .......... 82% 10.1M 3s 2024-03-19T01:11:51.285 INFO:tasks.workunit.client.0.smithi069.stderr:111450K .......... .......... .......... .......... .......... 82% 13.4M 3s 2024-03-19T01:11:51.287 INFO:tasks.workunit.client.0.smithi069.stderr:111500K .......... .......... .......... .......... .......... 82% 34.4M 3s 2024-03-19T01:11:51.292 INFO:tasks.workunit.client.0.smithi069.stderr:111550K .......... .......... .......... .......... .......... 82% 8.68M 3s 2024-03-19T01:11:51.297 INFO:tasks.workunit.client.0.smithi069.stderr:111600K .......... .......... .......... .......... .......... 82% 12.9M 3s 2024-03-19T01:11:51.297 INFO:tasks.workunit.client.0.smithi069.stderr:111650K .......... .......... .......... .......... .......... 82% 60.4M 3s 2024-03-19T01:11:51.303 INFO:tasks.workunit.client.0.smithi069.stderr:111700K .......... .......... .......... .......... .......... 82% 9.97M 3s 2024-03-19T01:11:51.303 INFO:tasks.workunit.client.0.smithi069.stderr:111750K .......... .......... .......... .......... .......... 82% 87.8M 3s 2024-03-19T01:11:51.307 INFO:tasks.workunit.client.0.smithi069.stderr:111800K .......... .......... .......... .......... .......... 82% 11.3M 3s 2024-03-19T01:11:51.308 INFO:tasks.workunit.client.0.smithi069.stderr:111850K .......... .......... .......... .......... .......... 82% 27.8M 3s 2024-03-19T01:11:51.315 INFO:tasks.workunit.client.0.smithi069.stderr:111900K .......... .......... .......... .......... .......... 82% 8.01M 3s 2024-03-19T01:11:51.318 INFO:tasks.workunit.client.0.smithi069.stderr:111950K .......... .......... .......... .......... .......... 82% 21.9M 3s 2024-03-19T01:11:51.318 INFO:tasks.workunit.client.0.smithi069.stderr:112000K .......... .......... .......... .......... .......... 82% 73.7M 3s 2024-03-19T01:11:51.323 INFO:tasks.workunit.client.0.smithi069.stderr:112050K .......... .......... .......... .......... .......... 82% 10.0M 3s 2024-03-19T01:11:51.326 INFO:tasks.workunit.client.0.smithi069.stderr:112100K .......... .......... .......... .......... .......... 82% 13.7M 3s 2024-03-19T01:11:51.328 INFO:tasks.workunit.client.0.smithi069.stderr:112150K .......... .......... .......... .......... .......... 82% 30.4M 3s 2024-03-19T01:11:51.333 INFO:tasks.workunit.client.0.smithi069.stderr:112200K .......... .......... .......... .......... .......... 82% 8.95M 3s 2024-03-19T01:11:51.338 INFO:tasks.workunit.client.0.smithi069.stderr:112250K .......... .......... .......... .......... .......... 82% 12.8M 3s 2024-03-19T01:11:51.338 INFO:tasks.workunit.client.0.smithi069.stderr:112300K .......... .......... .......... .......... .......... 82% 76.3M 3s 2024-03-19T01:11:51.343 INFO:tasks.workunit.client.0.smithi069.stderr:112350K .......... .......... .......... .......... .......... 82% 9.77M 3s 2024-03-19T01:11:51.348 INFO:tasks.workunit.client.0.smithi069.stderr:112400K .......... .......... .......... .......... .......... 82% 20.8M 3s 2024-03-19T01:11:51.348 INFO:tasks.workunit.client.0.smithi069.stderr:112450K .......... .......... .......... .......... .......... 82% 24.9M 3s 2024-03-19T01:11:51.348 INFO:tasks.workunit.client.0.smithi069.stderr:112500K .......... .......... .......... .......... .......... 82% 35.1M 3s 2024-03-19T01:11:51.355 INFO:tasks.workunit.client.0.smithi069.stderr:112550K .......... .......... .......... .......... .......... 82% 6.82M 3s 2024-03-19T01:11:51.359 INFO:tasks.workunit.client.0.smithi069.stderr:112600K .......... .......... .......... .......... .......... 82% 20.7M 3s 2024-03-19T01:11:51.359 INFO:tasks.workunit.client.0.smithi069.stderr:112650K .......... .......... .......... .......... .......... 83% 76.3M 3s 2024-03-19T01:11:51.363 INFO:tasks.workunit.client.0.smithi069.stderr:112700K .......... .......... .......... .......... .......... 83% 10.3M 3s 2024-03-19T01:11:51.367 INFO:tasks.workunit.client.0.smithi069.stderr:112750K .......... .......... .......... .......... .......... 83% 13.8M 3s 2024-03-19T01:11:51.369 INFO:tasks.workunit.client.0.smithi069.stderr:112800K .......... .......... .......... .......... .......... 83% 33.8M 3s 2024-03-19T01:11:51.374 INFO:tasks.workunit.client.0.smithi069.stderr:112850K .......... .......... .......... .......... .......... 83% 8.96M 3s 2024-03-19T01:11:51.380 INFO:tasks.workunit.client.0.smithi069.stderr:112900K .......... .......... .......... .......... .......... 83% 12.3M 3s 2024-03-19T01:11:51.380 INFO:tasks.workunit.client.0.smithi069.stderr:112950K .......... .......... .......... .......... .......... 83% 70.2M 2s 2024-03-19T01:11:51.383 INFO:tasks.workunit.client.0.smithi069.stderr:113000K .......... .......... .......... .......... .......... 83% 10.3M 2s 2024-03-19T01:11:51.385 INFO:tasks.workunit.client.0.smithi069.stderr:113050K .......... .......... .......... .......... .......... 83% 18.8M 2s 2024-03-19T01:11:51.387 INFO:tasks.workunit.client.0.smithi069.stderr:113100K .......... .......... .......... .......... .......... 83% 30.3M 2s 2024-03-19T01:11:51.389 INFO:tasks.workunit.client.0.smithi069.stderr:113150K .......... .......... .......... .......... .......... 83% 31.5M 2s 2024-03-19T01:11:51.394 INFO:tasks.workunit.client.0.smithi069.stderr:113200K .......... .......... .......... .......... .......... 83% 8.87M 2s 2024-03-19T01:11:51.399 INFO:tasks.workunit.client.0.smithi069.stderr:113250K .......... .......... .......... .......... .......... 83% 12.2M 2s 2024-03-19T01:11:51.399 INFO:tasks.workunit.client.0.smithi069.stderr:113300K .......... .......... .......... .......... .......... 83% 67.0M 2s 2024-03-19T01:11:51.404 INFO:tasks.workunit.client.0.smithi069.stderr:113350K .......... .......... .......... .......... .......... 83% 10.4M 2s 2024-03-19T01:11:51.407 INFO:tasks.workunit.client.0.smithi069.stderr:113400K .......... .......... .......... .......... .......... 83% 13.5M 2s 2024-03-19T01:11:51.409 INFO:tasks.workunit.client.0.smithi069.stderr:113450K .......... .......... .......... .......... .......... 83% 35.9M 2s 2024-03-19T01:11:51.413 INFO:tasks.workunit.client.0.smithi069.stderr:113500K .......... .......... .......... .......... .......... 83% 11.2M 2s 2024-03-19T01:11:51.420 INFO:tasks.workunit.client.0.smithi069.stderr:113550K .......... .......... .......... .......... .......... 83% 9.66M 2s 2024-03-19T01:11:51.420 INFO:tasks.workunit.client.0.smithi069.stderr:113600K .......... .......... .......... .......... .......... 83% 55.0M 2s 2024-03-19T01:11:51.424 INFO:tasks.workunit.client.0.smithi069.stderr:113650K .......... .......... .......... .......... .......... 83% 24.5M 2s 2024-03-19T01:11:51.425 INFO:tasks.workunit.client.0.smithi069.stderr:113700K .......... .......... .......... .......... .......... 83% 14.5M 2s 2024-03-19T01:11:51.429 INFO:tasks.workunit.client.0.smithi069.stderr:113750K .......... .......... .......... .......... .......... 83% 11.4M 2s 2024-03-19T01:11:51.429 INFO:tasks.workunit.client.0.smithi069.stderr:113800K .......... .......... .......... .......... .......... 83% 66.3M 2s 2024-03-19T01:11:51.435 INFO:tasks.workunit.client.0.smithi069.stderr:113850K .......... .......... .......... .......... .......... 83% 8.91M 2s 2024-03-19T01:11:51.440 INFO:tasks.workunit.client.0.smithi069.stderr:113900K .......... .......... .......... .......... .......... 83% 12.4M 2s 2024-03-19T01:11:51.440 INFO:tasks.workunit.client.0.smithi069.stderr:113950K .......... .......... .......... .......... .......... 83% 62.2M 2s 2024-03-19T01:11:51.447 INFO:tasks.workunit.client.0.smithi069.stderr:114000K .......... .......... .......... .......... .......... 84% 10.7M 2s 2024-03-19T01:11:51.448 INFO:tasks.workunit.client.0.smithi069.stderr:114050K .......... .......... .......... .......... .......... 84% 13.0M 2s 2024-03-19T01:11:51.452 INFO:tasks.workunit.client.0.smithi069.stderr:114100K .......... .......... .......... .......... .......... 84% 36.4M 2s 2024-03-19T01:11:51.452 INFO:tasks.workunit.client.0.smithi069.stderr:114150K .......... .......... .......... .......... .......... 84% 35.1M 2s 2024-03-19T01:11:51.463 INFO:tasks.workunit.client.0.smithi069.stderr:114200K .......... .......... .......... .......... .......... 84% 6.13M 2s 2024-03-19T01:11:51.463 INFO:tasks.workunit.client.0.smithi069.stderr:114250K .......... .......... .......... .......... .......... 84% 53.7M 2s 2024-03-19T01:11:51.463 INFO:tasks.workunit.client.0.smithi069.stderr:114300K .......... .......... .......... .......... .......... 84% 21.4M 2s 2024-03-19T01:11:51.465 INFO:tasks.workunit.client.0.smithi069.stderr:114350K .......... .......... .......... .......... .......... 84% 14.3M 2s 2024-03-19T01:11:51.470 INFO:tasks.workunit.client.0.smithi069.stderr:114400K .......... .......... .......... .......... .......... 84% 12.1M 2s 2024-03-19T01:11:51.470 INFO:tasks.workunit.client.0.smithi069.stderr:114450K .......... .......... .......... .......... .......... 84% 96.6M 2s 2024-03-19T01:11:51.475 INFO:tasks.workunit.client.0.smithi069.stderr:114500K .......... .......... .......... .......... .......... 84% 9.30M 2s 2024-03-19T01:11:51.480 INFO:tasks.workunit.client.0.smithi069.stderr:114550K .......... .......... .......... .......... .......... 84% 11.8M 2s 2024-03-19T01:11:51.480 INFO:tasks.workunit.client.0.smithi069.stderr:114600K .......... .......... .......... .......... .......... 84% 64.4M 2s 2024-03-19T01:11:51.485 INFO:tasks.workunit.client.0.smithi069.stderr:114650K .......... .......... .......... .......... .......... 84% 9.33M 2s 2024-03-19T01:11:51.487 INFO:tasks.workunit.client.0.smithi069.stderr:114700K .......... .......... .......... .......... .......... 84% 22.6M 2s 2024-03-19T01:11:51.490 INFO:tasks.workunit.client.0.smithi069.stderr:114750K .......... .......... .......... .......... .......... 84% 19.2M 2s 2024-03-19T01:11:51.491 INFO:tasks.workunit.client.0.smithi069.stderr:114800K .......... .......... .......... .......... .......... 84% 32.7M 2s 2024-03-19T01:11:51.498 INFO:tasks.workunit.client.0.smithi069.stderr:114850K .......... .......... .......... .......... .......... 84% 7.59M 2s 2024-03-19T01:11:51.501 INFO:tasks.workunit.client.0.smithi069.stderr:114900K .......... .......... .......... .......... .......... 84% 22.9M 2s 2024-03-19T01:11:51.501 INFO:tasks.workunit.client.0.smithi069.stderr:114950K .......... .......... .......... .......... .......... 84% 73.4M 2s 2024-03-19T01:11:51.507 INFO:tasks.workunit.client.0.smithi069.stderr:115000K .......... .......... .......... .......... .......... 84% 9.40M 2s 2024-03-19T01:11:51.513 INFO:tasks.workunit.client.0.smithi069.stderr:115050K .......... .......... .......... .......... .......... 84% 13.9M 2s 2024-03-19T01:11:51.513 INFO:tasks.workunit.client.0.smithi069.stderr:115100K .......... .......... .......... .......... .......... 84% 39.1M 2s 2024-03-19T01:11:51.516 INFO:tasks.workunit.client.0.smithi069.stderr:115150K .......... .......... .......... .......... .......... 84% 9.42M 2s 2024-03-19T01:11:51.522 INFO:tasks.workunit.client.0.smithi069.stderr:115200K .......... .......... .......... .......... .......... 84% 11.9M 2s 2024-03-19T01:11:51.522 INFO:tasks.workunit.client.0.smithi069.stderr:115250K .......... .......... .......... .......... .......... 84% 68.1M 2s 2024-03-19T01:11:51.527 INFO:tasks.workunit.client.0.smithi069.stderr:115300K .......... .......... .......... .......... .......... 84% 9.35M 2s 2024-03-19T01:11:51.529 INFO:tasks.workunit.client.0.smithi069.stderr:115350K .......... .......... .......... .......... .......... 85% 24.9M 2s 2024-03-19T01:11:51.532 INFO:tasks.workunit.client.0.smithi069.stderr:115400K .......... .......... .......... .......... .......... 85% 18.4M 2s 2024-03-19T01:11:51.532 INFO:tasks.workunit.client.0.smithi069.stderr:115450K .......... .......... .......... .......... .......... 85% 69.0M 2s 2024-03-19T01:11:51.538 INFO:tasks.workunit.client.0.smithi069.stderr:115500K .......... .......... .......... .......... .......... 85% 6.90M 2s 2024-03-19T01:11:51.541 INFO:tasks.workunit.client.0.smithi069.stderr:115550K .......... .......... .......... .......... .......... 85% 20.5M 2s 2024-03-19T01:11:51.543 INFO:tasks.workunit.client.0.smithi069.stderr:115600K .......... .......... .......... .......... .......... 85% 22.1M 2s 2024-03-19T01:11:51.546 INFO:tasks.workunit.client.0.smithi069.stderr:115650K .......... .......... .......... .......... .......... 85% 13.7M 2s 2024-03-19T01:11:51.550 INFO:tasks.workunit.client.0.smithi069.stderr:115700K .......... .......... .......... .......... .......... 85% 14.2M 2s 2024-03-19T01:11:51.552 INFO:tasks.workunit.client.0.smithi069.stderr:115750K .......... .......... .......... .......... .......... 85% 40.4M 2s 2024-03-19T01:11:51.552 INFO:tasks.workunit.client.0.smithi069.stderr:115800K .......... .......... .......... .......... .......... 85% 38.6M 2s 2024-03-19T01:11:51.561 INFO:tasks.workunit.client.0.smithi069.stderr:115850K .......... .......... .......... .......... .......... 85% 6.01M 2s 2024-03-19T01:11:51.562 INFO:tasks.workunit.client.0.smithi069.stderr:115900K .......... .......... .......... .......... .......... 85% 60.6M 2s 2024-03-19T01:11:51.567 INFO:tasks.workunit.client.0.smithi069.stderr:115950K .......... .......... .......... .......... .......... 85% 9.62M 2s 2024-03-19T01:11:51.570 INFO:tasks.workunit.client.0.smithi069.stderr:116000K .......... .......... .......... .......... .......... 85% 24.8M 2s 2024-03-19T01:11:51.570 INFO:tasks.workunit.client.0.smithi069.stderr:116050K .......... .......... .......... .......... .......... 85% 22.8M 2s 2024-03-19T01:11:51.572 INFO:tasks.workunit.client.0.smithi069.stderr:116100K .......... .......... .......... .......... .......... 85% 34.2M 2s 2024-03-19T01:11:51.577 INFO:tasks.workunit.client.0.smithi069.stderr:116150K .......... .......... .......... .......... .......... 85% 10.1M 2s 2024-03-19T01:11:51.584 INFO:tasks.workunit.client.0.smithi069.stderr:116200K .......... .......... .......... .......... .......... 85% 10.8M 2s 2024-03-19T01:11:51.584 INFO:tasks.workunit.client.0.smithi069.stderr:116250K .......... .......... .......... .......... .......... 85% 63.2M 2s 2024-03-19T01:11:51.587 INFO:tasks.workunit.client.0.smithi069.stderr:116300K .......... .......... .......... .......... .......... 85% 10.3M 2s 2024-03-19T01:11:51.593 INFO:tasks.workunit.client.0.smithi069.stderr:116350K .......... .......... .......... .......... .......... 85% 14.1M 2s 2024-03-19T01:11:51.593 INFO:tasks.workunit.client.0.smithi069.stderr:116400K .......... .......... .......... .......... .......... 85% 37.2M 2s 2024-03-19T01:11:51.593 INFO:tasks.workunit.client.0.smithi069.stderr:116450K .......... .......... .......... .......... .......... 85% 35.4M 2s 2024-03-19T01:11:51.599 INFO:tasks.workunit.client.0.smithi069.stderr:116500K .......... .......... .......... .......... .......... 85% 8.18M 2s 2024-03-19T01:11:51.602 INFO:tasks.workunit.client.0.smithi069.stderr:116550K .......... .......... .......... .......... .......... 85% 16.1M 2s 2024-03-19T01:11:51.604 INFO:tasks.workunit.client.0.smithi069.stderr:116600K .......... .......... .......... .......... .......... 85% 25.2M 2s 2024-03-19T01:11:51.607 INFO:tasks.workunit.client.0.smithi069.stderr:116650K .......... .......... .......... .......... .......... 85% 13.2M 2s 2024-03-19T01:11:51.611 INFO:tasks.workunit.client.0.smithi069.stderr:116700K .......... .......... .......... .......... .......... 86% 15.0M 2s 2024-03-19T01:11:51.612 INFO:tasks.workunit.client.0.smithi069.stderr:116750K .......... .......... .......... .......... .......... 86% 38.7M 2s 2024-03-19T01:11:51.613 INFO:tasks.workunit.client.0.smithi069.stderr:116800K .......... .......... .......... .......... .......... 86% 39.6M 2s 2024-03-19T01:11:51.624 INFO:tasks.workunit.client.0.smithi069.stderr:116850K .......... .......... .......... .......... .......... 86% 5.85M 2s 2024-03-19T01:11:51.625 INFO:tasks.workunit.client.0.smithi069.stderr:116900K .......... .......... .......... .......... .......... 86% 93.7M 2s 2024-03-19T01:11:51.630 INFO:tasks.workunit.client.0.smithi069.stderr:116950K .......... .......... .......... .......... .......... 86% 9.77M 2s 2024-03-19T01:11:51.630 INFO:tasks.workunit.client.0.smithi069.stderr:117000K .......... .......... .......... .......... .......... 86% 23.2M 2s 2024-03-19T01:11:51.638 INFO:tasks.workunit.client.0.smithi069.stderr:117050K .......... .......... .......... .......... .......... 86% 23.0M 2s 2024-03-19T01:11:51.639 INFO:tasks.workunit.client.0.smithi069.stderr:117100K .......... .......... .......... .......... .......... 86% 35.2M 2s 2024-03-19T01:11:51.639 INFO:tasks.workunit.client.0.smithi069.stderr:117150K .......... .......... .......... .......... .......... 86% 12.6M 2s 2024-03-19T01:11:51.645 INFO:tasks.workunit.client.0.smithi069.stderr:117200K .......... .......... .......... .......... .......... 86% 8.80M 2s 2024-03-19T01:11:51.645 INFO:tasks.workunit.client.0.smithi069.stderr:117250K .......... .......... .......... .......... .......... 86% 48.7M 2s 2024-03-19T01:11:51.648 INFO:tasks.workunit.client.0.smithi069.stderr:117300K .......... .......... .......... .......... .......... 86% 10.3M 2s 2024-03-19T01:11:51.651 INFO:tasks.workunit.client.0.smithi069.stderr:117350K .......... .......... .......... .......... .......... 86% 16.8M 2s 2024-03-19T01:11:51.653 INFO:tasks.workunit.client.0.smithi069.stderr:117400K .......... .......... .......... .......... .......... 86% 29.7M 2s 2024-03-19T01:11:51.654 INFO:tasks.workunit.client.0.smithi069.stderr:117450K .......... .......... .......... .......... .......... 86% 36.1M 2s 2024-03-19T01:11:51.658 INFO:tasks.workunit.client.0.smithi069.stderr:117500K .......... .......... .......... .......... .......... 86% 11.7M 2s 2024-03-19T01:11:51.663 INFO:tasks.workunit.client.0.smithi069.stderr:117550K .......... .......... .......... .......... .......... 86% 10.3M 2s 2024-03-19T01:11:51.665 INFO:tasks.workunit.client.0.smithi069.stderr:117600K .......... .......... .......... .......... .......... 86% 22.3M 2s 2024-03-19T01:11:51.669 INFO:tasks.workunit.client.0.smithi069.stderr:117650K .......... .......... .......... .......... .......... 86% 13.7M 2s 2024-03-19T01:11:51.672 INFO:tasks.workunit.client.0.smithi069.stderr:117700K .......... .......... .......... .......... .......... 86% 13.1M 2s 2024-03-19T01:11:51.675 INFO:tasks.workunit.client.0.smithi069.stderr:117750K .......... .......... .......... .......... .......... 86% 49.6M 2s 2024-03-19T01:11:51.677 INFO:tasks.workunit.client.0.smithi069.stderr:117800K .......... .......... .......... .......... .......... 86% 32.0M 2s 2024-03-19T01:11:51.680 INFO:tasks.workunit.client.0.smithi069.stderr:117850K .......... .......... .......... .......... .......... 86% 8.77M 2s 2024-03-19T01:11:51.684 INFO:tasks.workunit.client.0.smithi069.stderr:117900K .......... .......... .......... .......... .......... 86% 15.7M 2s 2024-03-19T01:11:51.689 INFO:tasks.workunit.client.0.smithi069.stderr:117950K .......... .......... .......... .......... .......... 86% 10.3M 2s 2024-03-19T01:11:51.691 INFO:tasks.workunit.client.0.smithi069.stderr:118000K .......... .......... .......... .......... .......... 86% 18.3M 2s 2024-03-19T01:11:51.693 INFO:tasks.workunit.client.0.smithi069.stderr:118050K .......... .......... .......... .......... .......... 86% 17.2M 2s 2024-03-19T01:11:51.695 INFO:tasks.workunit.client.0.smithi069.stderr:118100K .......... .......... .......... .......... .......... 87% 36.1M 2s 2024-03-19T01:11:51.697 INFO:tasks.workunit.client.0.smithi069.stderr:118150K .......... .......... .......... .......... .......... 87% 20.2M 2s 2024-03-19T01:11:51.704 INFO:tasks.workunit.client.0.smithi069.stderr:118200K .......... .......... .......... .......... .......... 87% 9.20M 2s 2024-03-19T01:11:51.704 INFO:tasks.workunit.client.0.smithi069.stderr:118250K .......... .......... .......... .......... .......... 87% 47.4M 2s 2024-03-19T01:11:51.709 INFO:tasks.workunit.client.0.smithi069.stderr:118300K .......... .......... .......... .......... .......... 87% 9.37M 2s 2024-03-19T01:11:51.711 INFO:tasks.workunit.client.0.smithi069.stderr:118350K .......... .......... .......... .......... .......... 87% 23.7M 2s 2024-03-19T01:11:51.716 INFO:tasks.workunit.client.0.smithi069.stderr:118400K .......... .......... .......... .......... .......... 87% 20.1M 2s 2024-03-19T01:11:51.716 INFO:tasks.workunit.client.0.smithi069.stderr:118450K .......... .......... .......... .......... .......... 87% 28.4M 2s 2024-03-19T01:11:51.721 INFO:tasks.workunit.client.0.smithi069.stderr:118500K .......... .......... .......... .......... .......... 87% 8.66M 2s 2024-03-19T01:11:51.724 INFO:tasks.workunit.client.0.smithi069.stderr:118550K .......... .......... .......... .......... .......... 87% 17.5M 2s 2024-03-19T01:11:51.726 INFO:tasks.workunit.client.0.smithi069.stderr:118600K .......... .......... .......... .......... .......... 87% 21.2M 2s 2024-03-19T01:11:51.730 INFO:tasks.workunit.client.0.smithi069.stderr:118650K .......... .......... .......... .......... .......... 87% 12.3M 2s 2024-03-19T01:11:51.733 INFO:tasks.workunit.client.0.smithi069.stderr:118700K .......... .......... .......... .......... .......... 87% 15.6M 2s 2024-03-19T01:11:51.734 INFO:tasks.workunit.client.0.smithi069.stderr:118750K .......... .......... .......... .......... .......... 87% 31.9M 2s 2024-03-19T01:11:51.738 INFO:tasks.workunit.client.0.smithi069.stderr:118800K .......... .......... .......... .......... .......... 87% 15.0M 2s 2024-03-19T01:11:51.742 INFO:tasks.workunit.client.0.smithi069.stderr:118850K .......... .......... .......... .......... .......... 87% 12.7M 2s 2024-03-19T01:11:51.744 INFO:tasks.workunit.client.0.smithi069.stderr:118900K .......... .......... .......... .......... .......... 87% 18.1M 2s 2024-03-19T01:11:51.750 INFO:tasks.workunit.client.0.smithi069.stderr:118950K .......... .......... .......... .......... .......... 87% 10.4M 2s 2024-03-19T01:11:51.752 INFO:tasks.workunit.client.0.smithi069.stderr:119000K .......... .......... .......... .......... .......... 87% 18.7M 2s 2024-03-19T01:11:51.755 INFO:tasks.workunit.client.0.smithi069.stderr:119050K .......... .......... .......... .......... .......... 87% 21.2M 2s 2024-03-19T01:11:51.756 INFO:tasks.workunit.client.0.smithi069.stderr:119100K .......... .......... .......... .......... .......... 87% 36.5M 2s 2024-03-19T01:11:51.758 INFO:tasks.workunit.client.0.smithi069.stderr:119150K .......... .......... .......... .......... .......... 87% 15.3M 2s 2024-03-19T01:11:51.762 INFO:tasks.workunit.client.0.smithi069.stderr:119200K .......... .......... .......... .......... .......... 87% 13.5M 2s 2024-03-19T01:11:51.765 INFO:tasks.workunit.client.0.smithi069.stderr:119250K .......... .......... .......... .......... .......... 87% 16.3M 2s 2024-03-19T01:11:51.770 INFO:tasks.workunit.client.0.smithi069.stderr:119300K .......... .......... .......... .......... .......... 87% 9.93M 2s 2024-03-19T01:11:51.772 INFO:tasks.workunit.client.0.smithi069.stderr:119350K .......... .......... .......... .......... .......... 87% 19.8M 2s 2024-03-19T01:11:51.776 INFO:tasks.workunit.client.0.smithi069.stderr:119400K .......... .......... .......... .......... .......... 87% 21.5M 2s 2024-03-19T01:11:51.777 INFO:tasks.workunit.client.0.smithi069.stderr:119450K .......... .......... .......... .......... .......... 88% 28.0M 2s 2024-03-19T01:11:51.780 INFO:tasks.workunit.client.0.smithi069.stderr:119500K .......... .......... .......... .......... .......... 88% 13.4M 2s 2024-03-19T01:11:51.788 INFO:tasks.workunit.client.0.smithi069.stderr:119550K .......... .......... .......... .......... .......... 88% 10.7M 2s 2024-03-19T01:11:51.788 INFO:tasks.workunit.client.0.smithi069.stderr:119600K .......... .......... .......... .......... .......... 88% 22.7M 2s 2024-03-19T01:11:51.790 INFO:tasks.workunit.client.0.smithi069.stderr:119650K .......... .......... .......... .......... .......... 88% 13.3M 2s 2024-03-19T01:11:51.795 INFO:tasks.workunit.client.0.smithi069.stderr:119700K .......... .......... .......... .......... .......... 88% 13.3M 2s 2024-03-19T01:11:51.796 INFO:tasks.workunit.client.0.smithi069.stderr:119750K .......... .......... .......... .......... .......... 88% 31.9M 2s 2024-03-19T01:11:51.799 INFO:tasks.workunit.client.0.smithi069.stderr:119800K .......... .......... .......... .......... .......... 88% 16.0M 2s 2024-03-19T01:11:51.802 INFO:tasks.workunit.client.0.smithi069.stderr:119850K .......... .......... .......... .......... .......... 88% 13.0M 2s 2024-03-19T01:11:51.805 INFO:tasks.workunit.client.0.smithi069.stderr:119900K .......... .......... .......... .......... .......... 88% 16.2M 2s 2024-03-19T01:11:51.808 INFO:tasks.workunit.client.0.smithi069.stderr:119950K .......... .......... .......... .......... .......... 88% 22.2M 2s 2024-03-19T01:11:51.813 INFO:tasks.workunit.client.0.smithi069.stderr:120000K .......... .......... .......... .......... .......... 88% 10.0M 2s 2024-03-19T01:11:51.815 INFO:tasks.workunit.client.0.smithi069.stderr:120050K .......... .......... .......... .......... .......... 88% 20.1M 2s 2024-03-19T01:11:51.817 INFO:tasks.workunit.client.0.smithi069.stderr:120100K .......... .......... .......... .......... .......... 88% 37.0M 2s 2024-03-19T01:11:51.822 INFO:tasks.workunit.client.0.smithi069.stderr:120150K .......... .......... .......... .......... .......... 88% 8.13M 2s 2024-03-19T01:11:51.825 INFO:tasks.workunit.client.0.smithi069.stderr:120200K .......... .......... .......... .......... .......... 88% 19.5M 2s 2024-03-19T01:11:51.826 INFO:tasks.workunit.client.0.smithi069.stderr:120250K .......... .......... .......... .......... .......... 88% 24.5M 2s 2024-03-19T01:11:51.831 INFO:tasks.workunit.client.0.smithi069.stderr:120300K .......... .......... .......... .......... .......... 88% 13.9M 2s 2024-03-19T01:11:51.834 INFO:tasks.workunit.client.0.smithi069.stderr:120350K .......... .......... .......... .......... .......... 88% 16.7M 2s 2024-03-19T01:11:51.836 INFO:tasks.workunit.client.0.smithi069.stderr:120400K .......... .......... .......... .......... .......... 88% 20.7M 2s 2024-03-19T01:11:51.838 INFO:tasks.workunit.client.0.smithi069.stderr:120450K .......... .......... .......... .......... .......... 88% 28.0M 2s 2024-03-19T01:11:51.843 INFO:tasks.workunit.client.0.smithi069.stderr:120500K .......... .......... .......... .......... .......... 88% 8.18M 2s 2024-03-19T01:11:51.846 INFO:tasks.workunit.client.0.smithi069.stderr:120550K .......... .......... .......... .......... .......... 88% 20.4M 2s 2024-03-19T01:11:51.848 INFO:tasks.workunit.client.0.smithi069.stderr:120600K .......... .......... .......... .......... .......... 88% 23.3M 2s 2024-03-19T01:11:51.851 INFO:tasks.workunit.client.0.smithi069.stderr:120650K .......... .......... .......... .......... .......... 88% 14.2M 2s 2024-03-19T01:11:51.854 INFO:tasks.workunit.client.0.smithi069.stderr:120700K .......... .......... .......... .......... .......... 88% 20.1M 2s 2024-03-19T01:11:51.857 INFO:tasks.workunit.client.0.smithi069.stderr:120750K .......... .......... .......... .......... .......... 88% 16.4M 2s 2024-03-19T01:11:51.863 INFO:tasks.workunit.client.0.smithi069.stderr:120800K .......... .......... .......... .......... .......... 89% 7.72M 2s 2024-03-19T01:11:51.864 INFO:tasks.workunit.client.0.smithi069.stderr:120850K .......... .......... .......... .......... .......... 89% 34.0M 2s 2024-03-19T01:11:51.867 INFO:tasks.workunit.client.0.smithi069.stderr:120900K .......... .......... .......... .......... .......... 89% 18.1M 2s 2024-03-19T01:11:51.869 INFO:tasks.workunit.client.0.smithi069.stderr:120950K .......... .......... .......... .......... .......... 89% 28.9M 2s 2024-03-19T01:11:51.887 INFO:tasks.workunit.client.0.smithi069.stderr:121000K .......... .......... .......... .......... .......... 89% 2.83M 2s 2024-03-19T01:11:51.887 INFO:tasks.workunit.client.0.smithi069.stderr:121050K .......... .......... .......... .......... .......... 89% 44.4M 2s 2024-03-19T01:11:51.889 INFO:tasks.workunit.client.0.smithi069.stderr:121100K .......... .......... .......... .......... .......... 89% 23.9M 2s 2024-03-19T01:11:51.893 INFO:tasks.workunit.client.0.smithi069.stderr:121150K .......... .......... .......... .......... .......... 89% 14.3M 2s 2024-03-19T01:11:51.895 INFO:tasks.workunit.client.0.smithi069.stderr:121200K .......... .......... .......... .......... .......... 89% 21.9M 2s 2024-03-19T01:11:51.898 INFO:tasks.workunit.client.0.smithi069.stderr:121250K .......... .......... .......... .......... .......... 89% 20.8M 2s 2024-03-19T01:11:51.900 INFO:tasks.workunit.client.0.smithi069.stderr:121300K .......... .......... .......... .......... .......... 89% 25.8M 2s 2024-03-19T01:11:51.908 INFO:tasks.workunit.client.0.smithi069.stderr:121350K .......... .......... .......... .......... .......... 89% 6.52M 2s 2024-03-19T01:11:51.909 INFO:tasks.workunit.client.0.smithi069.stderr:121400K .......... .......... .......... .......... .......... 89% 29.2M 2s 2024-03-19T01:11:51.910 INFO:tasks.workunit.client.0.smithi069.stderr:121450K .......... .......... .......... .......... .......... 89% 22.5M 2s 2024-03-19T01:11:51.913 INFO:tasks.workunit.client.0.smithi069.stderr:121500K .......... .......... .......... .......... .......... 89% 17.6M 2s 2024-03-19T01:11:51.916 INFO:tasks.workunit.client.0.smithi069.stderr:121550K .......... .......... .......... .......... .......... 89% 21.4M 2s 2024-03-19T01:11:51.918 INFO:tasks.workunit.client.0.smithi069.stderr:121600K .......... .......... .......... .......... .......... 89% 21.3M 2s 2024-03-19T01:11:51.921 INFO:tasks.workunit.client.0.smithi069.stderr:121650K .......... .......... .......... .......... .......... 89% 17.9M 1s 2024-03-19T01:11:51.928 INFO:tasks.workunit.client.0.smithi069.stderr:121700K .......... .......... .......... .......... .......... 89% 7.61M 1s 2024-03-19T01:11:51.929 INFO:tasks.workunit.client.0.smithi069.stderr:121750K .......... .......... .......... .......... .......... 89% 26.7M 1s 2024-03-19T01:11:51.931 INFO:tasks.workunit.client.0.smithi069.stderr:121800K .......... .......... .......... .......... .......... 89% 23.1M 1s 2024-03-19T01:11:51.933 INFO:tasks.workunit.client.0.smithi069.stderr:121850K .......... .......... .......... .......... .......... 89% 18.0M 1s 2024-03-19T01:11:51.936 INFO:tasks.workunit.client.0.smithi069.stderr:121900K .......... .......... .......... .......... .......... 89% 17.6M 1s 2024-03-19T01:11:51.938 INFO:tasks.workunit.client.0.smithi069.stderr:121950K .......... .......... .......... .......... .......... 89% 22.2M 1s 2024-03-19T01:11:51.946 INFO:tasks.workunit.client.0.smithi069.stderr:122000K .......... .......... .......... .......... .......... 89% 6.83M 1s 2024-03-19T01:11:51.954 INFO:tasks.workunit.client.0.smithi069.stderr:122050K .......... .......... .......... .......... .......... 89% 5.91M 1s 2024-03-19T01:11:51.957 INFO:tasks.workunit.client.0.smithi069.stderr:122100K .......... .......... .......... .......... .......... 89% 13.5M 1s 2024-03-19T01:11:51.962 INFO:tasks.workunit.client.0.smithi069.stderr:122150K .......... .......... .......... .......... .......... 90% 10.5M 1s 2024-03-19T01:11:51.964 INFO:tasks.workunit.client.0.smithi069.stderr:122200K .......... .......... .......... .......... .......... 90% 28.3M 1s 2024-03-19T01:11:51.968 INFO:tasks.workunit.client.0.smithi069.stderr:122250K .......... .......... .......... .......... .......... 90% 13.8M 1s 2024-03-19T01:11:51.969 INFO:tasks.workunit.client.0.smithi069.stderr:122300K .......... .......... .......... .......... .......... 90% 27.6M 1s 2024-03-19T01:11:51.973 INFO:tasks.workunit.client.0.smithi069.stderr:122350K .......... .......... .......... .......... .......... 90% 13.2M 1s 2024-03-19T01:11:51.980 INFO:tasks.workunit.client.0.smithi069.stderr:122400K .......... .......... .......... .......... .......... 90% 6.41M 1s 2024-03-19T01:11:51.983 INFO:tasks.workunit.client.0.smithi069.stderr:122450K .......... .......... .......... .......... .......... 90% 31.5M 1s 2024-03-19T01:11:51.984 INFO:tasks.workunit.client.0.smithi069.stderr:122500K .......... .......... .......... .......... .......... 90% 28.1M 1s 2024-03-19T01:11:51.985 INFO:tasks.workunit.client.0.smithi069.stderr:122550K .......... .......... .......... .......... .......... 90% 28.2M 1s 2024-03-19T01:11:51.991 INFO:tasks.workunit.client.0.smithi069.stderr:122600K .......... .......... .......... .......... .......... 90% 22.4M 1s 2024-03-19T01:11:51.991 INFO:tasks.workunit.client.0.smithi069.stderr:122650K .......... .......... .......... .......... .......... 90% 26.4M 1s 2024-03-19T01:11:51.994 INFO:tasks.workunit.client.0.smithi069.stderr:122700K .......... .......... .......... .......... .......... 90% 11.0M 1s 2024-03-19T01:11:52.001 INFO:tasks.workunit.client.0.smithi069.stderr:122750K .......... .......... .......... .......... .......... 90% 6.97M 1s 2024-03-19T01:11:52.006 INFO:tasks.workunit.client.0.smithi069.stderr:122800K .......... .......... .......... .......... .......... 90% 9.96M 1s 2024-03-19T01:11:52.011 INFO:tasks.workunit.client.0.smithi069.stderr:122850K .......... .......... .......... .......... .......... 90% 8.90M 1s 2024-03-19T01:11:52.014 INFO:tasks.workunit.client.0.smithi069.stderr:122900K .......... .......... .......... .......... .......... 90% 16.8M 1s 2024-03-19T01:11:52.017 INFO:tasks.workunit.client.0.smithi069.stderr:122950K .......... .......... .......... .......... .......... 90% 16.2M 1s 2024-03-19T01:11:52.021 INFO:tasks.workunit.client.0.smithi069.stderr:123000K .......... .......... .......... .......... .......... 90% 12.0M 1s 2024-03-19T01:11:52.026 INFO:tasks.workunit.client.0.smithi069.stderr:123050K .......... .......... .......... .......... .......... 90% 10.3M 1s 2024-03-19T01:11:52.034 INFO:tasks.workunit.client.0.smithi069.stderr:123100K .......... .......... .......... .......... .......... 90% 6.52M 1s 2024-03-19T01:11:52.038 INFO:tasks.workunit.client.0.smithi069.stderr:123150K .......... .......... .......... .......... .......... 90% 13.6M 1s 2024-03-19T01:11:52.040 INFO:tasks.workunit.client.0.smithi069.stderr:123200K .......... .......... .......... .......... .......... 90% 14.3M 1s 2024-03-19T01:11:52.043 INFO:tasks.workunit.client.0.smithi069.stderr:123250K .......... .......... .......... .......... .......... 90% 18.9M 1s 2024-03-19T01:11:52.048 INFO:tasks.workunit.client.0.smithi069.stderr:123300K .......... .......... .......... .......... .......... 90% 10.1M 1s 2024-03-19T01:11:52.052 INFO:tasks.workunit.client.0.smithi069.stderr:123350K .......... .......... .......... .......... .......... 90% 14.2M 1s 2024-03-19T01:11:52.055 INFO:tasks.workunit.client.0.smithi069.stderr:123400K .......... .......... .......... .......... .......... 90% 16.2M 1s 2024-03-19T01:11:52.056 INFO:tasks.workunit.client.0.smithi069.stderr:123450K .......... .......... .......... .......... .......... 90% 25.1M 1s 2024-03-19T01:11:52.059 INFO:tasks.workunit.client.0.smithi069.stderr:123500K .......... .......... .......... .......... .......... 91% 23.9M 1s 2024-03-19T01:11:52.061 INFO:tasks.workunit.client.0.smithi069.stderr:123550K .......... .......... .......... .......... .......... 91% 22.1M 1s 2024-03-19T01:11:52.065 INFO:tasks.workunit.client.0.smithi069.stderr:123600K .......... .......... .......... .......... .......... 91% 12.6M 1s 2024-03-19T01:11:52.070 INFO:tasks.workunit.client.0.smithi069.stderr:123650K .......... .......... .......... .......... .......... 91% 8.70M 1s 2024-03-19T01:11:52.075 INFO:tasks.workunit.client.0.smithi069.stderr:123700K .......... .......... .......... .......... .......... 91% 10.3M 1s 2024-03-19T01:11:52.080 INFO:tasks.workunit.client.0.smithi069.stderr:123750K .......... .......... .......... .......... .......... 91% 8.65M 1s 2024-03-19T01:11:52.084 INFO:tasks.workunit.client.0.smithi069.stderr:123800K .......... .......... .......... .......... .......... 91% 16.0M 1s 2024-03-19T01:11:52.086 INFO:tasks.workunit.client.0.smithi069.stderr:123850K .......... .......... .......... .......... .......... 91% 17.4M 1s 2024-03-19T01:11:52.090 INFO:tasks.workunit.client.0.smithi069.stderr:123900K .......... .......... .......... .......... .......... 91% 14.8M 1s 2024-03-19T01:11:52.093 INFO:tasks.workunit.client.0.smithi069.stderr:123950K .......... .......... .......... .......... .......... 91% 13.0M 1s 2024-03-19T01:11:52.100 INFO:tasks.workunit.client.0.smithi069.stderr:124000K .......... .......... .......... .......... .......... 91% 8.50M 1s 2024-03-19T01:11:52.109 INFO:tasks.workunit.client.0.smithi069.stderr:124050K .......... .......... .......... .......... .......... 91% 5.33M 1s 2024-03-19T01:11:52.111 INFO:tasks.workunit.client.0.smithi069.stderr:124100K .......... .......... .......... .......... .......... 91% 15.9M 1s 2024-03-19T01:11:52.115 INFO:tasks.workunit.client.0.smithi069.stderr:124150K .......... .......... .......... .......... .......... 91% 14.1M 1s 2024-03-19T01:11:52.118 INFO:tasks.workunit.client.0.smithi069.stderr:124200K .......... .......... .......... .......... .......... 91% 17.7M 1s 2024-03-19T01:11:52.122 INFO:tasks.workunit.client.0.smithi069.stderr:124250K .......... .......... .......... .......... .......... 91% 13.6M 1s 2024-03-19T01:11:52.126 INFO:tasks.workunit.client.0.smithi069.stderr:124300K .......... .......... .......... .......... .......... 91% 11.6M 1s 2024-03-19T01:11:52.130 INFO:tasks.workunit.client.0.smithi069.stderr:124350K .......... .......... .......... .......... .......... 91% 9.93M 1s 2024-03-19T01:11:52.133 INFO:tasks.workunit.client.0.smithi069.stderr:124400K .......... .......... .......... .......... .......... 91% 20.2M 1s 2024-03-19T01:11:52.138 INFO:tasks.workunit.client.0.smithi069.stderr:124450K .......... .......... .......... .......... .......... 91% 8.48M 1s 2024-03-19T01:11:52.143 INFO:tasks.workunit.client.0.smithi069.stderr:124500K .......... .......... .......... .......... .......... 91% 10.7M 1s 2024-03-19T01:11:52.146 INFO:tasks.workunit.client.0.smithi069.stderr:124550K .......... .......... .......... .......... .......... 91% 15.2M 1s 2024-03-19T01:11:52.152 INFO:tasks.workunit.client.0.smithi069.stderr:124600K .......... .......... .......... .......... .......... 91% 8.86M 1s 2024-03-19T01:11:52.156 INFO:tasks.workunit.client.0.smithi069.stderr:124650K .......... .......... .......... .......... .......... 91% 11.4M 1s 2024-03-19T01:11:52.160 INFO:tasks.workunit.client.0.smithi069.stderr:124700K .......... .......... .......... .......... .......... 91% 11.4M 1s 2024-03-19T01:11:52.163 INFO:tasks.workunit.client.0.smithi069.stderr:124750K .......... .......... .......... .......... .......... 91% 18.7M 1s 2024-03-19T01:11:52.168 INFO:tasks.workunit.client.0.smithi069.stderr:124800K .......... .......... .......... .......... .......... 91% 10.1M 1s 2024-03-19T01:11:52.172 INFO:tasks.workunit.client.0.smithi069.stderr:124850K .......... .......... .......... .......... .......... 92% 10.4M 1s 2024-03-19T01:11:52.176 INFO:tasks.workunit.client.0.smithi069.stderr:124900K .......... .......... .......... .......... .......... 92% 14.3M 1s 2024-03-19T01:11:52.182 INFO:tasks.workunit.client.0.smithi069.stderr:124950K .......... .......... .......... .......... .......... 92% 8.44M 1s 2024-03-19T01:11:52.186 INFO:tasks.workunit.client.0.smithi069.stderr:125000K .......... .......... .......... .......... .......... 92% 11.3M 1s 2024-03-19T01:11:52.190 INFO:tasks.workunit.client.0.smithi069.stderr:125050K .......... .......... .......... .......... .......... 92% 11.3M 1s 2024-03-19T01:11:52.193 INFO:tasks.workunit.client.0.smithi069.stderr:125100K .......... .......... .......... .......... .......... 92% 17.4M 1s 2024-03-19T01:11:52.197 INFO:tasks.workunit.client.0.smithi069.stderr:125150K .......... .......... .......... .......... .......... 92% 11.5M 1s 2024-03-19T01:11:52.202 INFO:tasks.workunit.client.0.smithi069.stderr:125200K .......... .......... .......... .......... .......... 92% 10.8M 1s 2024-03-19T01:11:52.207 INFO:tasks.workunit.client.0.smithi069.stderr:125250K .......... .......... .......... .......... .......... 92% 10.5M 1s 2024-03-19T01:11:52.217 INFO:tasks.workunit.client.0.smithi069.stderr:125300K .......... .......... .......... .......... .......... 92% 4.94M 1s 2024-03-19T01:11:52.220 INFO:tasks.workunit.client.0.smithi069.stderr:125350K .......... .......... .......... .......... .......... 92% 12.2M 1s 2024-03-19T01:11:52.224 INFO:tasks.workunit.client.0.smithi069.stderr:125400K .......... .......... .......... .......... .......... 92% 12.0M 1s 2024-03-19T01:11:52.228 INFO:tasks.workunit.client.0.smithi069.stderr:125450K .......... .......... .......... .......... .......... 92% 15.1M 1s 2024-03-19T01:11:52.232 INFO:tasks.workunit.client.0.smithi069.stderr:125500K .......... .......... .......... .......... .......... 92% 12.8M 1s 2024-03-19T01:11:52.236 INFO:tasks.workunit.client.0.smithi069.stderr:125550K .......... .......... .......... .......... .......... 92% 10.9M 1s 2024-03-19T01:11:52.240 INFO:tasks.workunit.client.0.smithi069.stderr:125600K .......... .......... .......... .......... .......... 92% 13.2M 1s 2024-03-19T01:11:52.250 INFO:tasks.workunit.client.0.smithi069.stderr:125650K .......... .......... .......... .......... .......... 92% 4.83M 1s 2024-03-19T01:11:52.254 INFO:tasks.workunit.client.0.smithi069.stderr:125700K .......... .......... .......... .......... .......... 92% 18.0M 1s 2024-03-19T01:11:52.257 INFO:tasks.workunit.client.0.smithi069.stderr:125750K .......... .......... .......... .......... .......... 92% 9.80M 1s 2024-03-19T01:11:52.262 INFO:tasks.workunit.client.0.smithi069.stderr:125800K .......... .......... .......... .......... .......... 92% 11.3M 1s 2024-03-19T01:11:52.266 INFO:tasks.workunit.client.0.smithi069.stderr:125850K .......... .......... .......... .......... .......... 92% 11.3M 1s 2024-03-19T01:11:52.272 INFO:tasks.workunit.client.0.smithi069.stderr:125900K .......... .......... .......... .......... .......... 92% 8.65M 1s 2024-03-19T01:11:52.275 INFO:tasks.workunit.client.0.smithi069.stderr:125950K .......... .......... .......... .......... .......... 92% 21.2M 1s 2024-03-19T01:11:52.282 INFO:tasks.workunit.client.0.smithi069.stderr:126000K .......... .......... .......... .......... .......... 92% 6.22M 1s 2024-03-19T01:11:52.288 INFO:tasks.workunit.client.0.smithi069.stderr:126050K .......... .......... .......... .......... .......... 92% 7.74M 1s 2024-03-19T01:11:52.292 INFO:tasks.workunit.client.0.smithi069.stderr:126100K .......... .......... .......... .......... .......... 92% 14.4M 1s 2024-03-19T01:11:52.297 INFO:tasks.workunit.client.0.smithi069.stderr:126150K .......... .......... .......... .......... .......... 92% 9.97M 1s 2024-03-19T01:11:52.301 INFO:tasks.workunit.client.0.smithi069.stderr:126200K .......... .......... .......... .......... .......... 92% 10.5M 1s 2024-03-19T01:11:52.306 INFO:tasks.workunit.client.0.smithi069.stderr:126250K .......... .......... .......... .......... .......... 93% 11.6M 1s 2024-03-19T01:11:52.308 INFO:tasks.workunit.client.0.smithi069.stderr:126300K .......... .......... .......... .......... .......... 93% 18.9M 1s 2024-03-19T01:11:52.316 INFO:tasks.workunit.client.0.smithi069.stderr:126350K .......... .......... .......... .......... .......... 93% 6.52M 1s 2024-03-19T01:11:52.321 INFO:tasks.workunit.client.0.smithi069.stderr:126400K .......... .......... .......... .......... .......... 93% 9.28M 1s 2024-03-19T01:11:52.326 INFO:tasks.workunit.client.0.smithi069.stderr:126450K .......... .......... .......... .......... .......... 93% 10.4M 1s 2024-03-19T01:11:52.330 INFO:tasks.workunit.client.0.smithi069.stderr:126500K .......... .......... .......... .......... .......... 93% 12.1M 1s 2024-03-19T01:11:52.336 INFO:tasks.workunit.client.0.smithi069.stderr:126550K .......... .......... .......... .......... .......... 93% 8.43M 1s 2024-03-19T01:11:52.340 INFO:tasks.workunit.client.0.smithi069.stderr:126600K .......... .......... .......... .......... .......... 93% 11.8M 1s 2024-03-19T01:11:52.342 INFO:tasks.workunit.client.0.smithi069.stderr:126650K .......... .......... .......... .......... .......... 93% 19.0M 1s 2024-03-19T01:11:52.350 INFO:tasks.workunit.client.0.smithi069.stderr:126700K .......... .......... .......... .......... .......... 93% 6.06M 1s 2024-03-19T01:11:52.354 INFO:tasks.workunit.client.0.smithi069.stderr:126750K .......... .......... .......... .......... .......... 93% 13.8M 1s 2024-03-19T01:11:52.362 INFO:tasks.workunit.client.0.smithi069.stderr:126800K .......... .......... .......... .......... .......... 93% 6.21M 1s 2024-03-19T01:11:52.366 INFO:tasks.workunit.client.0.smithi069.stderr:126850K .......... .......... .......... .......... .......... 93% 10.3M 1s 2024-03-19T01:11:52.373 INFO:tasks.workunit.client.0.smithi069.stderr:126900K .......... .......... .......... .......... .......... 93% 7.68M 1s 2024-03-19T01:11:52.377 INFO:tasks.workunit.client.0.smithi069.stderr:126950K .......... .......... .......... .......... .......... 93% 12.1M 1s 2024-03-19T01:11:52.379 INFO:tasks.workunit.client.0.smithi069.stderr:127000K .......... .......... .......... .......... .......... 93% 17.7M 1s 2024-03-19T01:11:52.388 INFO:tasks.workunit.client.0.smithi069.stderr:127050K .......... .......... .......... .......... .......... 93% 5.81M 1s 2024-03-19T01:11:52.391 INFO:tasks.workunit.client.0.smithi069.stderr:127100K .......... .......... .......... .......... .......... 93% 15.6M 1s 2024-03-19T01:11:52.399 INFO:tasks.workunit.client.0.smithi069.stderr:127150K .......... .......... .......... .......... .......... 93% 6.40M 1s 2024-03-19T01:11:52.405 INFO:tasks.workunit.client.0.smithi069.stderr:127200K .......... .......... .......... .......... .......... 93% 7.88M 1s 2024-03-19T01:11:52.410 INFO:tasks.workunit.client.0.smithi069.stderr:127250K .......... .......... .......... .......... .......... 93% 9.32M 1s 2024-03-19T01:11:52.414 INFO:tasks.workunit.client.0.smithi069.stderr:127300K .......... .......... .......... .......... .......... 93% 10.4M 1s 2024-03-19T01:11:52.419 INFO:tasks.workunit.client.0.smithi069.stderr:127350K .......... .......... .......... .......... .......... 93% 11.3M 1s 2024-03-19T01:11:52.425 INFO:tasks.workunit.client.0.smithi069.stderr:127400K .......... .......... .......... .......... .......... 93% 8.76M 1s 2024-03-19T01:11:52.427 INFO:tasks.workunit.client.0.smithi069.stderr:127450K .......... .......... .......... .......... .......... 93% 17.9M 1s 2024-03-19T01:11:52.437 INFO:tasks.workunit.client.0.smithi069.stderr:127500K .......... .......... .......... .......... .......... 93% 5.23M 1s 2024-03-19T01:11:52.445 INFO:tasks.workunit.client.0.smithi069.stderr:127550K .......... .......... .......... .......... .......... 93% 6.10M 1s 2024-03-19T01:11:52.452 INFO:tasks.workunit.client.0.smithi069.stderr:127600K .......... .......... .......... .......... .......... 94% 7.31M 1s 2024-03-19T01:11:52.457 INFO:tasks.workunit.client.0.smithi069.stderr:127650K .......... .......... .......... .......... .......... 94% 9.10M 1s 2024-03-19T01:11:52.462 INFO:tasks.workunit.client.0.smithi069.stderr:127700K .......... .......... .......... .......... .......... 94% 10.0M 1s 2024-03-19T01:11:52.466 INFO:tasks.workunit.client.0.smithi069.stderr:127750K .......... .......... .......... .......... .......... 94% 11.9M 1s 2024-03-19T01:11:52.471 INFO:tasks.workunit.client.0.smithi069.stderr:127800K .......... .......... .......... .......... .......... 94% 9.86M 1s 2024-03-19T01:11:52.476 INFO:tasks.workunit.client.0.smithi069.stderr:127850K .......... .......... .......... .......... .......... 94% 9.45M 1s 2024-03-19T01:11:52.485 INFO:tasks.workunit.client.0.smithi069.stderr:127900K .......... .......... .......... .......... .......... 94% 5.42M 1s 2024-03-19T01:11:52.492 INFO:tasks.workunit.client.0.smithi069.stderr:127950K .......... .......... .......... .......... .......... 94% 7.16M 1s 2024-03-19T01:11:52.498 INFO:tasks.workunit.client.0.smithi069.stderr:128000K .......... .......... .......... .......... .......... 94% 7.80M 1s 2024-03-19T01:11:52.502 INFO:tasks.workunit.client.0.smithi069.stderr:128050K .......... .......... .......... .......... .......... 94% 13.0M 1s 2024-03-19T01:11:52.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:52 smithi086 ceph-mon[66829]: pgmap v39: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.1 MiB/s wr, 15 op/s 2024-03-19T01:11:52.507 INFO:tasks.workunit.client.0.smithi069.stderr:128100K .......... .......... .......... .......... .......... 94% 9.20M 1s 2024-03-19T01:11:52.516 INFO:tasks.workunit.client.0.smithi069.stderr:128150K .......... .......... .......... .......... .......... 94% 11.7M 1s 2024-03-19T01:11:52.517 INFO:tasks.workunit.client.0.smithi069.stderr:128200K .......... .......... .......... .......... .......... 94% 8.32M 1s 2024-03-19T01:11:52.526 INFO:tasks.workunit.client.0.smithi069.stderr:128250K .......... .......... .......... .......... .......... 94% 5.80M 1s 2024-03-19T01:11:52.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:52 smithi069 ceph-mon[31727]: pgmap v39: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.1 MiB/s wr, 15 op/s 2024-03-19T01:11:52.535 INFO:tasks.workunit.client.0.smithi069.stderr:128300K .......... .......... .......... .......... .......... 94% 5.07M 1s 2024-03-19T01:11:52.543 INFO:tasks.workunit.client.0.smithi069.stderr:128350K .......... .......... .......... .......... .......... 94% 6.35M 1s 2024-03-19T01:11:52.549 INFO:tasks.workunit.client.0.smithi069.stderr:128400K .......... .......... .......... .......... .......... 94% 9.54M 1s 2024-03-19T01:11:52.553 INFO:tasks.workunit.client.0.smithi069.stderr:128450K .......... .......... .......... .......... .......... 94% 8.85M 1s 2024-03-19T01:11:52.558 INFO:tasks.workunit.client.0.smithi069.stderr:128500K .......... .......... .......... .......... .......... 94% 10.2M 1s 2024-03-19T01:11:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:52 smithi163 ceph-mon[67428]: pgmap v39: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.1 MiB/s wr, 15 op/s 2024-03-19T01:11:52.564 INFO:tasks.workunit.client.0.smithi069.stderr:128550K .......... .......... .......... .......... .......... 94% 9.09M 1s 2024-03-19T01:11:52.573 INFO:tasks.workunit.client.0.smithi069.stderr:128600K .......... .......... .......... .......... .......... 94% 5.01M 1s 2024-03-19T01:11:52.583 INFO:tasks.workunit.client.0.smithi069.stderr:128650K .......... .......... .......... .......... .......... 94% 6.13M 1s 2024-03-19T01:11:52.589 INFO:tasks.workunit.client.0.smithi069.stderr:128700K .......... .......... .......... .......... .......... 94% 6.52M 1s 2024-03-19T01:11:52.595 INFO:tasks.workunit.client.0.smithi069.stderr:128750K .......... .......... .......... .......... .......... 94% 7.93M 1s 2024-03-19T01:11:52.600 INFO:tasks.workunit.client.0.smithi069.stderr:128800K .......... .......... .......... .......... .......... 94% 9.73M 1s 2024-03-19T01:11:52.605 INFO:tasks.workunit.client.0.smithi069.stderr:128850K .......... .......... .......... .......... .......... 94% 9.83M 1s 2024-03-19T01:11:52.609 INFO:tasks.workunit.client.0.smithi069.stderr:128900K .......... .......... .......... .......... .......... 94% 10.4M 1s 2024-03-19T01:11:52.616 INFO:tasks.workunit.client.0.smithi069.stderr:128950K .......... .......... .......... .......... .......... 95% 7.21M 1s 2024-03-19T01:11:52.632 INFO:tasks.workunit.client.0.smithi069.stderr:129000K .......... .......... .......... .......... .......... 95% 3.14M 1s 2024-03-19T01:11:52.643 INFO:tasks.workunit.client.0.smithi069.stderr:129050K .......... .......... .......... .......... .......... 95% 5.06M 1s 2024-03-19T01:11:52.648 INFO:tasks.workunit.client.0.smithi069.stderr:129100K .......... .......... .......... .......... .......... 95% 9.07M 1s 2024-03-19T01:11:52.652 INFO:tasks.workunit.client.0.smithi069.stderr:129150K .......... .......... .......... .......... .......... 95% 9.57M 1s 2024-03-19T01:11:52.661 INFO:tasks.workunit.client.0.smithi069.stderr:129200K .......... .......... .......... .......... .......... 95% 5.48M 1s 2024-03-19T01:11:52.667 INFO:tasks.workunit.client.0.smithi069.stderr:129250K .......... .......... .......... .......... .......... 95% 9.06M 1s 2024-03-19T01:11:52.673 INFO:tasks.workunit.client.0.smithi069.stderr:129300K .......... .......... .......... .......... .......... 95% 7.59M 1s 2024-03-19T01:11:52.687 INFO:tasks.workunit.client.0.smithi069.stderr:129350K .......... .......... .......... .......... .......... 95% 3.64M 1s 2024-03-19T01:11:52.697 INFO:tasks.workunit.client.0.smithi069.stderr:129400K .......... .......... .......... .......... .......... 95% 4.66M 1s 2024-03-19T01:11:52.704 INFO:tasks.workunit.client.0.smithi069.stderr:129450K .......... .......... .......... .......... .......... 95% 6.62M 1s 2024-03-19T01:11:52.709 INFO:tasks.workunit.client.0.smithi069.stderr:129500K .......... .......... .......... .......... .......... 95% 10.7M 1s 2024-03-19T01:11:52.717 INFO:tasks.workunit.client.0.smithi069.stderr:129550K .......... .......... .......... .......... .......... 95% 5.84M 1s 2024-03-19T01:11:52.725 INFO:tasks.workunit.client.0.smithi069.stderr:129600K .......... .......... .......... .......... .......... 95% 7.60M 1s 2024-03-19T01:11:52.731 INFO:tasks.workunit.client.0.smithi069.stderr:129650K .......... .......... .......... .......... .......... 95% 6.37M 1s 2024-03-19T01:11:52.741 INFO:tasks.workunit.client.0.smithi069.stderr:129700K .......... .......... .......... .......... .......... 95% 4.67M 1s 2024-03-19T01:11:52.759 INFO:tasks.workunit.client.0.smithi069.stderr:129750K .......... .......... .......... .......... .......... 95% 3.06M 1s 2024-03-19T01:11:52.763 INFO:tasks.workunit.client.0.smithi069.stderr:129800K .......... .......... .......... .......... .......... 95% 8.56M 1s 2024-03-19T01:11:52.769 INFO:tasks.workunit.client.0.smithi069.stderr:129850K .......... .......... .......... .......... .......... 95% 9.02M 1s 2024-03-19T01:11:52.776 INFO:tasks.workunit.client.0.smithi069.stderr:129900K .......... .......... .......... .......... .......... 95% 6.19M 1s 2024-03-19T01:11:52.782 INFO:tasks.workunit.client.0.smithi069.stderr:129950K .......... .......... .......... .......... .......... 95% 8.68M 1s 2024-03-19T01:11:52.790 INFO:tasks.workunit.client.0.smithi069.stderr:130000K .......... .......... .......... .......... .......... 95% 5.91M 1s 2024-03-19T01:11:52.802 INFO:tasks.workunit.client.0.smithi069.stderr:130050K .......... .......... .......... .......... .......... 95% 4.41M 1s 2024-03-19T01:11:52.815 INFO:tasks.workunit.client.0.smithi069.stderr:130100K .......... .......... .......... .......... .......... 95% 3.67M 1s 2024-03-19T01:11:52.823 INFO:tasks.workunit.client.0.smithi069.stderr:130150K .......... .......... .......... .......... .......... 95% 6.11M 1s 2024-03-19T01:11:52.827 INFO:tasks.workunit.client.0.smithi069.stderr:130200K .......... .......... .......... .......... .......... 95% 11.5M 1s 2024-03-19T01:11:52.833 INFO:tasks.workunit.client.0.smithi069.stderr:130250K .......... .......... .......... .......... .......... 95% 10.3M 1s 2024-03-19T01:11:52.842 INFO:tasks.workunit.client.0.smithi069.stderr:130300K .......... .......... .......... .......... .......... 96% 5.20M 1s 2024-03-19T01:11:52.849 INFO:tasks.workunit.client.0.smithi069.stderr:130350K .......... .......... .......... .......... .......... 96% 5.79M 1s 2024-03-19T01:11:52.861 INFO:tasks.workunit.client.0.smithi069.stderr:130400K .......... .......... .......... .......... .......... 96% 4.08M 1s 2024-03-19T01:11:52.873 INFO:tasks.workunit.client.0.smithi069.stderr:130450K .......... .......... .......... .......... .......... 96% 4.30M 1s 2024-03-19T01:11:52.882 INFO:tasks.workunit.client.0.smithi069.stderr:130500K .......... .......... .......... .......... .......... 96% 5.64M 1s 2024-03-19T01:11:52.889 INFO:tasks.workunit.client.0.smithi069.stderr:130550K .......... .......... .......... .......... .......... 96% 7.16M 1s 2024-03-19T01:11:52.894 INFO:tasks.workunit.client.0.smithi069.stderr:130600K .......... .......... .......... .......... .......... 96% 9.32M 1s 2024-03-19T01:11:52.903 INFO:tasks.workunit.client.0.smithi069.stderr:130650K .......... .......... .......... .......... .......... 96% 5.14M 1s 2024-03-19T01:11:52.913 INFO:tasks.workunit.client.0.smithi069.stderr:130700K .......... .......... .......... .......... .......... 96% 5.46M 1s 2024-03-19T01:11:52.924 INFO:tasks.workunit.client.0.smithi069.stderr:130750K .......... .......... .......... .......... .......... 96% 4.30M 1s 2024-03-19T01:11:52.936 INFO:tasks.workunit.client.0.smithi069.stderr:130800K .......... .......... .......... .......... .......... 96% 4.13M 1s 2024-03-19T01:11:52.945 INFO:tasks.workunit.client.0.smithi069.stderr:130850K .......... .......... .......... .......... .......... 96% 5.64M 1s 2024-03-19T01:11:52.948 INFO:tasks.workunit.client.0.smithi069.stderr:130900K .......... .......... .......... .......... .......... 96% 11.6M 1s 2024-03-19T01:11:52.951 INFO:tasks.workunit.client.0.smithi069.stderr:130950K .......... .......... .......... .......... .......... 96% 14.8M 1s 2024-03-19T01:11:52.966 INFO:tasks.workunit.client.0.smithi069.stderr:131000K .......... .......... .......... .......... .......... 96% 3.98M 1s 2024-03-19T01:11:52.973 INFO:tasks.workunit.client.0.smithi069.stderr:131050K .......... .......... .......... .......... .......... 96% 4.96M 0s 2024-03-19T01:11:52.988 INFO:tasks.workunit.client.0.smithi069.stderr:131100K .......... .......... .......... .......... .......... 96% 4.05M 0s 2024-03-19T01:11:52.998 INFO:tasks.workunit.client.0.smithi069.stderr:131150K .......... .......... .......... .......... .......... 96% 3.86M 0s 2024-03-19T01:11:53.007 INFO:tasks.workunit.client.0.smithi069.stderr:131200K .......... .......... .......... .......... .......... 96% 6.34M 0s 2024-03-19T01:11:53.014 INFO:tasks.workunit.client.0.smithi069.stderr:131250K .......... .......... .......... .......... .......... 96% 6.11M 0s 2024-03-19T01:11:53.019 INFO:tasks.workunit.client.0.smithi069.stderr:131300K .......... .......... .......... .......... .......... 96% 8.59M 0s 2024-03-19T01:11:53.032 INFO:tasks.workunit.client.0.smithi069.stderr:131350K .......... .......... .......... .......... .......... 96% 3.89M 0s 2024-03-19T01:11:53.040 INFO:tasks.workunit.client.0.smithi069.stderr:131400K .......... .......... .......... .......... .......... 96% 5.73M 0s 2024-03-19T01:11:53.055 INFO:tasks.workunit.client.0.smithi069.stderr:131450K .......... .......... .......... .......... .......... 96% 3.45M 0s 2024-03-19T01:11:53.067 INFO:tasks.workunit.client.0.smithi069.stderr:131500K .......... .......... .......... .......... .......... 96% 4.10M 0s 2024-03-19T01:11:53.073 INFO:tasks.workunit.client.0.smithi069.stderr:131550K .......... .......... .......... .......... .......... 96% 7.14M 0s 2024-03-19T01:11:53.086 INFO:tasks.workunit.client.0.smithi069.stderr:131600K .......... .......... .......... .......... .......... 96% 4.10M 0s 2024-03-19T01:11:53.092 INFO:tasks.workunit.client.0.smithi069.stderr:131650K .......... .......... .......... .......... .......... 97% 7.62M 0s 2024-03-19T01:11:53.098 INFO:tasks.workunit.client.0.smithi069.stderr:131700K .......... .......... .......... .......... .......... 97% 7.58M 0s 2024-03-19T01:11:53.110 INFO:tasks.workunit.client.0.smithi069.stderr:131750K .......... .......... .......... .......... .......... 97% 4.12M 0s 2024-03-19T01:11:53.121 INFO:tasks.workunit.client.0.smithi069.stderr:131800K .......... .......... .......... .......... .......... 97% 4.55M 0s 2024-03-19T01:11:53.133 INFO:tasks.workunit.client.0.smithi069.stderr:131850K .......... .......... .......... .......... .......... 97% 4.04M 0s 2024-03-19T01:11:53.141 INFO:tasks.workunit.client.0.smithi069.stderr:131900K .......... .......... .......... .......... .......... 97% 6.52M 0s 2024-03-19T01:11:53.148 INFO:tasks.workunit.client.0.smithi069.stderr:131950K .......... .......... .......... .......... .......... 97% 6.14M 0s 2024-03-19T01:11:53.158 INFO:tasks.workunit.client.0.smithi069.stderr:132000K .......... .......... .......... .......... .......... 97% 4.81M 0s 2024-03-19T01:11:53.165 INFO:tasks.workunit.client.0.smithi069.stderr:132050K .......... .......... .......... .......... .......... 97% 7.18M 0s 2024-03-19T01:11:53.175 INFO:tasks.workunit.client.0.smithi069.stderr:132100K .......... .......... .......... .......... .......... 97% 4.92M 0s 2024-03-19T01:11:53.187 INFO:tasks.workunit.client.0.smithi069.stderr:132150K .......... .......... .......... .......... .......... 97% 4.09M 0s 2024-03-19T01:11:53.198 INFO:tasks.workunit.client.0.smithi069.stderr:132200K .......... .......... .......... .......... .......... 97% 4.35M 0s 2024-03-19T01:11:53.205 INFO:tasks.workunit.client.0.smithi069.stderr:132250K .......... .......... .......... .......... .......... 97% 7.82M 0s 2024-03-19T01:11:53.208 INFO:tasks.workunit.client.0.smithi069.stderr:132300K .......... .......... .......... .......... .......... 97% 12.5M 0s 2024-03-19T01:11:53.213 INFO:tasks.workunit.client.0.smithi069.stderr:132350K .......... .......... .......... .......... .......... 97% 9.88M 0s 2024-03-19T01:11:53.219 INFO:tasks.workunit.client.0.smithi069.stderr:132400K .......... .......... .......... .......... .......... 97% 14.0M 0s 2024-03-19T01:11:53.221 INFO:tasks.workunit.client.0.smithi069.stderr:132450K .......... .......... .......... .......... .......... 97% 16.0M 0s 2024-03-19T01:11:53.224 INFO:tasks.workunit.client.0.smithi069.stderr:132500K .......... .......... .......... .......... .......... 97% 11.3M 0s 2024-03-19T01:11:53.229 INFO:tasks.workunit.client.0.smithi069.stderr:132550K .......... .......... .......... .......... .......... 97% 9.60M 0s 2024-03-19T01:11:53.233 INFO:tasks.workunit.client.0.smithi069.stderr:132600K .......... .......... .......... .......... .......... 97% 13.1M 0s 2024-03-19T01:11:53.237 INFO:tasks.workunit.client.0.smithi069.stderr:132650K .......... .......... .......... .......... .......... 97% 13.6M 0s 2024-03-19T01:11:53.241 INFO:tasks.workunit.client.0.smithi069.stderr:132700K .......... .......... .......... .......... .......... 97% 12.3M 0s 2024-03-19T01:11:53.244 INFO:tasks.workunit.client.0.smithi069.stderr:132750K .......... .......... .......... .......... .......... 97% 17.7M 0s 2024-03-19T01:11:53.246 INFO:tasks.workunit.client.0.smithi069.stderr:132800K .......... .......... .......... .......... .......... 97% 19.7M 0s 2024-03-19T01:11:53.249 INFO:tasks.workunit.client.0.smithi069.stderr:132850K .......... .......... .......... .......... .......... 97% 16.0M 0s 2024-03-19T01:11:53.252 INFO:tasks.workunit.client.0.smithi069.stderr:132900K .......... .......... .......... .......... .......... 97% 15.3M 0s 2024-03-19T01:11:53.256 INFO:tasks.workunit.client.0.smithi069.stderr:132950K .......... .......... .......... .......... .......... 97% 12.4M 0s 2024-03-19T01:11:53.260 INFO:tasks.workunit.client.0.smithi069.stderr:133000K .......... .......... .......... .......... .......... 98% 14.0M 0s 2024-03-19T01:11:53.262 INFO:tasks.workunit.client.0.smithi069.stderr:133050K .......... .......... .......... .......... .......... 98% 17.5M 0s 2024-03-19T01:11:53.267 INFO:tasks.workunit.client.0.smithi069.stderr:133100K .......... .......... .......... .......... .......... 98% 12.3M 0s 2024-03-19T01:11:53.270 INFO:tasks.workunit.client.0.smithi069.stderr:133150K .......... .......... .......... .......... .......... 98% 19.5M 0s 2024-03-19T01:11:53.272 INFO:tasks.workunit.client.0.smithi069.stderr:133200K .......... .......... .......... .......... .......... 98% 15.6M 0s 2024-03-19T01:11:53.275 INFO:tasks.workunit.client.0.smithi069.stderr:133250K .......... .......... .......... .......... .......... 98% 18.2M 0s 2024-03-19T01:11:53.277 INFO:tasks.workunit.client.0.smithi069.stderr:133300K .......... .......... .......... .......... .......... 98% 18.2M 0s 2024-03-19T01:11:53.281 INFO:tasks.workunit.client.0.smithi069.stderr:133350K .......... .......... .......... .......... .......... 98% 15.3M 0s 2024-03-19T01:11:53.283 INFO:tasks.workunit.client.0.smithi069.stderr:133400K .......... .......... .......... .......... .......... 98% 21.3M 0s 2024-03-19T01:11:53.286 INFO:tasks.workunit.client.0.smithi069.stderr:133450K .......... .......... .......... .......... .......... 98% 15.8M 0s 2024-03-19T01:11:53.289 INFO:tasks.workunit.client.0.smithi069.stderr:133500K .......... .......... .......... .......... .......... 98% 15.9M 0s 2024-03-19T01:11:53.291 INFO:tasks.workunit.client.0.smithi069.stderr:133550K .......... .......... .......... .......... .......... 98% 19.0M 0s 2024-03-19T01:11:53.294 INFO:tasks.workunit.client.0.smithi069.stderr:133600K .......... .......... .......... .......... .......... 98% 19.3M 0s 2024-03-19T01:11:53.297 INFO:tasks.workunit.client.0.smithi069.stderr:133650K .......... .......... .......... .......... .......... 98% 17.4M 0s 2024-03-19T01:11:53.301 INFO:tasks.workunit.client.0.smithi069.stderr:133700K .......... .......... .......... .......... .......... 98% 14.2M 0s 2024-03-19T01:11:53.303 INFO:tasks.workunit.client.0.smithi069.stderr:133750K .......... .......... .......... .......... .......... 98% 18.6M 0s 2024-03-19T01:11:53.305 INFO:tasks.workunit.client.0.smithi069.stderr:133800K .......... .......... .......... .......... .......... 98% 34.1M 0s 2024-03-19T01:11:53.309 INFO:tasks.workunit.client.0.smithi069.stderr:133850K .......... .......... .......... .......... .......... 98% 11.7M 0s 2024-03-19T01:11:53.311 INFO:tasks.workunit.client.0.smithi069.stderr:133900K .......... .......... .......... .......... .......... 98% 19.4M 0s 2024-03-19T01:11:53.314 INFO:tasks.workunit.client.0.smithi069.stderr:133950K .......... .......... .......... .......... .......... 98% 18.9M 0s 2024-03-19T01:11:53.317 INFO:tasks.workunit.client.0.smithi069.stderr:134000K .......... .......... .......... .......... .......... 98% 14.8M 0s 2024-03-19T01:11:53.319 INFO:tasks.workunit.client.0.smithi069.stderr:134050K .......... .......... .......... .......... .......... 98% 23.4M 0s 2024-03-19T01:11:53.323 INFO:tasks.workunit.client.0.smithi069.stderr:134100K .......... .......... .......... .......... .......... 98% 13.4M 0s 2024-03-19T01:11:53.325 INFO:tasks.workunit.client.0.smithi069.stderr:134150K .......... .......... .......... .......... .......... 98% 25.1M 0s 2024-03-19T01:11:53.329 INFO:tasks.workunit.client.0.smithi069.stderr:134200K .......... .......... .......... .......... .......... 98% 15.4M 0s 2024-03-19T01:11:53.331 INFO:tasks.workunit.client.0.smithi069.stderr:134250K .......... .......... .......... .......... .......... 98% 18.2M 0s 2024-03-19T01:11:53.332 INFO:tasks.workunit.client.0.smithi069.stderr:134300K .......... .......... .......... .......... .......... 98% 23.3M 0s 2024-03-19T01:11:53.336 INFO:tasks.workunit.client.0.smithi069.stderr:134350K .......... .......... .......... .......... .......... 99% 16.2M 0s 2024-03-19T01:11:53.339 INFO:tasks.workunit.client.0.smithi069.stderr:134400K .......... .......... .......... .......... .......... 99% 14.6M 0s 2024-03-19T01:11:53.342 INFO:tasks.workunit.client.0.smithi069.stderr:134450K .......... .......... .......... .......... .......... 99% 19.0M 0s 2024-03-19T01:11:53.345 INFO:tasks.workunit.client.0.smithi069.stderr:134500K .......... .......... .......... .......... .......... 99% 15.2M 0s 2024-03-19T01:11:53.346 INFO:tasks.workunit.client.0.smithi069.stderr:134550K .......... .......... .......... .......... .......... 99% 31.9M 0s 2024-03-19T01:11:53.350 INFO:tasks.workunit.client.0.smithi069.stderr:134600K .......... .......... .......... .......... .......... 99% 13.1M 0s 2024-03-19T01:11:53.352 INFO:tasks.workunit.client.0.smithi069.stderr:134650K .......... .......... .......... .......... .......... 99% 26.7M 0s 2024-03-19T01:11:53.355 INFO:tasks.workunit.client.0.smithi069.stderr:134700K .......... .......... .......... .......... .......... 99% 15.3M 0s 2024-03-19T01:11:53.359 INFO:tasks.workunit.client.0.smithi069.stderr:134750K .......... .......... .......... .......... .......... 99% 13.4M 0s 2024-03-19T01:11:53.360 INFO:tasks.workunit.client.0.smithi069.stderr:134800K .......... .......... .......... .......... .......... 99% 42.9M 0s 2024-03-19T01:11:53.364 INFO:tasks.workunit.client.0.smithi069.stderr:134850K .......... .......... .......... .......... .......... 99% 13.7M 0s 2024-03-19T01:11:53.366 INFO:tasks.workunit.client.0.smithi069.stderr:134900K .......... .......... .......... .......... .......... 99% 20.7M 0s 2024-03-19T01:11:53.369 INFO:tasks.workunit.client.0.smithi069.stderr:134950K .......... .......... .......... .......... .......... 99% 12.5M 0s 2024-03-19T01:11:53.372 INFO:tasks.workunit.client.0.smithi069.stderr:135000K .......... .......... .......... .......... .......... 99% 29.4M 0s 2024-03-19T01:11:53.373 INFO:tasks.workunit.client.0.smithi069.stderr:135050K .......... .......... .......... .......... .......... 99% 27.7M 0s 2024-03-19T01:11:53.377 INFO:tasks.workunit.client.0.smithi069.stderr:135100K .......... .......... .......... .......... .......... 99% 14.7M 0s 2024-03-19T01:11:53.380 INFO:tasks.workunit.client.0.smithi069.stderr:135150K .......... .......... .......... .......... .......... 99% 14.5M 0s 2024-03-19T01:11:53.383 INFO:tasks.workunit.client.0.smithi069.stderr:135200K .......... .......... .......... .......... .......... 99% 14.3M 0s 2024-03-19T01:11:53.386 INFO:tasks.workunit.client.0.smithi069.stderr:135250K .......... .......... .......... .......... .......... 99% 20.9M 0s 2024-03-19T01:11:53.387 INFO:tasks.workunit.client.0.smithi069.stderr:135300K .......... .......... .......... .......... .......... 99% 28.1M 0s 2024-03-19T01:11:53.391 INFO:tasks.workunit.client.0.smithi069.stderr:135350K .......... .......... .......... .......... .......... 99% 13.0M 0s 2024-03-19T01:11:53.392 INFO:tasks.workunit.client.0.smithi069.stderr:135400K .......... .......... .......... .......... .......... 99% 31.0M 0s 2024-03-19T01:11:53.396 INFO:tasks.workunit.client.0.smithi069.stderr:135450K .......... .......... .......... .......... .......... 99% 16.9M 0s 2024-03-19T01:11:53.399 INFO:tasks.workunit.client.0.smithi069.stderr:135500K .......... .......... .......... .......... .......... 99% 13.5M 0s 2024-03-19T01:11:53.401 INFO:tasks.workunit.client.0.smithi069.stderr:135550K .......... .......... .......... .......... .......... 99% 28.8M 0s 2024-03-19T01:11:53.405 INFO:tasks.workunit.client.0.smithi069.stderr:135600K .......... .......... .......... .......... .......... 99% 13.4M 0s 2024-03-19T01:11:53.407 INFO:tasks.workunit.client.0.smithi069.stderr:135650K .......... .......... .......... .......... .......... 99% 17.7M 0s 2024-03-19T01:11:53.412 INFO:tasks.workunit.client.0.smithi069.stderr:135700K .......... .......... .......... .......... .......... 99% 14.1M 0s 2024-03-19T01:11:53.412 INFO:tasks.workunit.client.0.smithi069.stderr:135750K .. 100% 5.30T=14s 2024-03-19T01:11:53.412 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-03-19T01:11:53.412 INFO:tasks.workunit.client.0.smithi069.stderr:2024-03-19 01:11:53 (9.18 MB/s) - ‘linux.tar.xz’ saved [139010916/139010916] 2024-03-19T01:11:53.413 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-03-19T01:11:53.414 INFO:tasks.workunit.client.0.smithi069.stderr:+ mkdir t 2024-03-19T01:11:53.421 INFO:tasks.workunit.client.0.smithi069.stderr:+ cd t 2024-03-19T01:11:53.421 INFO:tasks.workunit.client.0.smithi069.stderr:+ tar xJf ../linux.tar.xz 2024-03-19T01:11:54.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:54 smithi086 ceph-mon[66829]: pgmap v40: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-03-19T01:11:54.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:54 smithi069 ceph-mon[31727]: pgmap v40: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-03-19T01:11:54.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:54 smithi163 ceph-mon[67428]: pgmap v40: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-03-19T01:11:56.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:56 smithi086 ceph-mon[66829]: pgmap v41: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.1 MiB/s wr, 13 op/s 2024-03-19T01:11:56.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:56 smithi069 ceph-mon[31727]: pgmap v41: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.1 MiB/s wr, 13 op/s 2024-03-19T01:11:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:56 smithi163 ceph-mon[67428]: pgmap v41: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.1 MiB/s wr, 13 op/s 2024-03-19T01:11:58.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:11:58 smithi086 ceph-mon[66829]: pgmap v42: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.0 MiB/s wr, 10 op/s 2024-03-19T01:11:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:11:58 smithi069 ceph-mon[31727]: pgmap v42: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.0 MiB/s wr, 10 op/s 2024-03-19T01:11:58.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:11:58 smithi163 ceph-mon[67428]: pgmap v42: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.0 MiB/s wr, 10 op/s 2024-03-19T01:12:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:00 smithi086 ceph-mon[66829]: pgmap v43: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-03-19T01:12:00.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:00 smithi069 ceph-mon[31727]: pgmap v43: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-03-19T01:12:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:00 smithi163 ceph-mon[67428]: pgmap v43: 129 pgs: 129 active+clean; 39 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-03-19T01:12:02.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:02 smithi086 ceph-mon[66829]: pgmap v44: 129 pgs: 129 active+clean; 96 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.8 MiB/s wr, 32 op/s 2024-03-19T01:12:02.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:02 smithi069 ceph-mon[31727]: pgmap v44: 129 pgs: 129 active+clean; 96 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.8 MiB/s wr, 32 op/s 2024-03-19T01:12:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:02 smithi163 ceph-mon[67428]: pgmap v44: 129 pgs: 129 active+clean; 96 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.8 MiB/s wr, 32 op/s 2024-03-19T01:12:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:04 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:12:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:04 smithi086 ceph-mon[66829]: pgmap v45: 129 pgs: 129 active+clean; 96 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.8 MiB/s wr, 28 op/s 2024-03-19T01:12:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:04 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:12:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:04 smithi069 ceph-mon[31727]: pgmap v45: 129 pgs: 129 active+clean; 96 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.8 MiB/s wr, 28 op/s 2024-03-19T01:12:05.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:04 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:12:05.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:04 smithi163 ceph-mon[67428]: pgmap v45: 129 pgs: 129 active+clean; 96 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.8 MiB/s wr, 28 op/s 2024-03-19T01:12:06.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:06 smithi086 ceph-mon[66829]: pgmap v46: 129 pgs: 129 active+clean; 97 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 4.9 MiB/s wr, 35 op/s 2024-03-19T01:12:06.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:06 smithi069 ceph-mon[31727]: pgmap v46: 129 pgs: 129 active+clean; 97 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 4.9 MiB/s wr, 35 op/s 2024-03-19T01:12:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:06 smithi163 ceph-mon[67428]: pgmap v46: 129 pgs: 129 active+clean; 97 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 4.9 MiB/s wr, 35 op/s 2024-03-19T01:12:07.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:12:07.338 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:12:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.560+0000 7f54a1e00640 1 -- 172.21.15.69:0/2726358798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549c112540 msgr2=0x7f549c114930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.560+0000 7f54a1e00640 1 --2- 172.21.15.69:0/2726358798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549c112540 0x7f549c114930 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5494009cd0 tx=0x7f549402c0e0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.560+0000 7f54a1e00640 1 -- 172.21.15.69:0/2726358798 shutdown_connections 2024-03-19T01:12:07.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.560+0000 7f54a1e00640 1 --2- 172.21.15.69:0/2726358798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549c112540 0x7f549c114930 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.560+0000 7f54a1e00640 1 --2- 172.21.15.69:0/2726358798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c076670 0x7f549c112000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.560+0000 7f54a1e00640 1 --2- 172.21.15.69:0/2726358798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c075d50 0x7f549c076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.560+0000 7f54a1e00640 1 -- 172.21.15.69:0/2726358798 >> 172.21.15.69:0/2726358798 conn(0x7f549c0ff330 msgr2=0x7f549c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:07.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.560+0000 7f54a1e00640 1 -- 172.21.15.69:0/2726358798 shutdown_connections 2024-03-19T01:12:07.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.560+0000 7f54a1e00640 1 -- 172.21.15.69:0/2726358798 wait complete. 2024-03-19T01:12:07.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.562+0000 7f54a1e00640 1 Processor -- start 2024-03-19T01:12:07.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.562+0000 7f54a1e00640 1 -- start start 2024-03-19T01:12:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.562+0000 7f54a1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549c075d50 0x7f549c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a1e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c076670 0x7f549c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a1e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c112540 0x7f549c1a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a1e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1180a0 con 0x7f549c076670 2024-03-19T01:12:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f549c117f20 con 0x7f549c075d50 2024-03-19T01:12:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a1e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f549c118220 con 0x7f549c112540 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549c075d50 0x7f549c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549c075d50 0x7f549c19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37486/0 (socket says 172.21.15.69:37486) 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a0a00640 1 -- 172.21.15.69:0/3635295234 learned_addr learned my addr 172.21.15.69:0/3635295234 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f549be00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c076670 0x7f549c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a1400640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c112540 0x7f549c1a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a0a00640 1 -- 172.21.15.69:0/3635295234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c112540 msgr2=0x7f549c1a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a0a00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c112540 0x7f549c1a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a0a00640 1 -- 172.21.15.69:0/3635295234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c076670 msgr2=0x7f549c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a0a00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c076670 0x7f549c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.563+0000 7f54a0a00640 1 -- 172.21.15.69:0/3635295234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c1a35a0 con 0x7f549c075d50 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.564+0000 7f54a0a00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549c075d50 0x7f549c19e600 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f549000e9f0 tx=0x7f549000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.564+0000 7f54a1400640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c112540 0x7f549c1a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:12:07.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.564+0000 7f5499600640 1 -- 172.21.15.69:0/3635295234 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549000cd80 con 0x7f549c075d50 2024-03-19T01:12:07.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.564+0000 7f5499600640 1 -- 172.21.15.69:0/3635295234 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5490005500 con 0x7f549c075d50 2024-03-19T01:12:07.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.564+0000 7f5499600640 1 -- 172.21.15.69:0/3635295234 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490010690 con 0x7f549c075d50 2024-03-19T01:12:07.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.564+0000 7f54a1e00640 1 -- 172.21.15.69:0/3635295234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c1a3890 con 0x7f549c075d50 2024-03-19T01:12:07.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.564+0000 7f54a1e00640 1 -- 172.21.15.69:0/3635295234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c1ab120 con 0x7f549c075d50 2024-03-19T01:12:07.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.568+0000 7f54a61c1740 1 Processor -- start 2024-03-19T01:12:07.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f54a61c1740 1 -- start start 2024-03-19T01:12:07.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f5499600640 1 -- 172.21.15.69:0/3635295234 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7f54900046c0 con 0x7f549c075d50 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747470310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561747470850 0x561747478ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747479430 0x56174747b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f54a61c1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5617473d42c0 con 0x561747479430 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f54a61c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56174737cb60 con 0x561747470850 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f54a61c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5617473d0360 con 0x56174746ff30 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f54a1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747470310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f54a1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747470310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58686/0 (socket says 172.21.15.69:58686) 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f54a1400640 1 -- 172.21.15.69:0/1352878084 learned_addr learned my addr 172.21.15.69:0/1352878084 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.569+0000 7f54a0a00640 1 --2- 172.21.15.69:0/1352878084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561747470850 0x561747478ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.570+0000 7f549be00640 1 --2- 172.21.15.69:0/1352878084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747479430 0x56174747b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.570+0000 7f54a1400640 1 -- 172.21.15.69:0/1352878084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561747470850 msgr2=0x561747478ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.570+0000 7f54a1400640 1 --2- 172.21.15.69:0/1352878084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561747470850 0x561747478ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.570+0000 7f54a1400640 1 -- 172.21.15.69:0/1352878084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747479430 msgr2=0x56174747b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:07.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.570+0000 7f54a1400640 1 --2- 172.21.15.69:0/1352878084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747479430 0x56174747b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.570+0000 7f54a1400640 1 -- 172.21.15.69:0/1352878084 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617471b27d0 con 0x56174746ff30 2024-03-19T01:12:07.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.570+0000 7f54a0a00640 1 --2- 172.21.15.69:0/1352878084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561747470850 0x561747478ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:07.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.570+0000 7f549be00640 1 --2- 172.21.15.69:0/1352878084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747479430 0x56174747b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:12:07.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.570+0000 7f54a1400640 1 --2- 172.21.15.69:0/1352878084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747470310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5494004af0 tx=0x7f54940323d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=840ce05ac2897ca7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.570+0000 7f548dc00640 1 -- 172.21.15.69:0/1352878084 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494039050 con 0x56174746ff30 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.570+0000 7f548dc00640 1 -- 172.21.15.69:0/1352878084 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494045070 con 0x56174746ff30 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.571+0000 7f548dc00640 1 -- 172.21.15.69:0/1352878084 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494004040 con 0x56174746ff30 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.571+0000 7f5499600640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54840a2b10 0x7f54840a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.571+0000 7f5499600640 1 -- 172.21.15.69:0/3635295234 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f54900cf960 con 0x7f549c075d50 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.571+0000 7f54a0a00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54840a2b10 0x7f54840a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.571+0000 7f54a61c1740 1 -- 172.21.15.69:0/1352878084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 msgr2=0x561747470310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.571+0000 7f54a61c1740 1 --2- 172.21.15.69:0/1352878084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747470310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5494004af0 tx=0x7f54940323d0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.572+0000 7f54a61c1740 1 -- 172.21.15.69:0/1352878084 shutdown_connections 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.572+0000 7f54a61c1740 1 --2- 172.21.15.69:0/1352878084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747479430 0x56174747b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.572+0000 7f54a61c1740 1 --2- 172.21.15.69:0/1352878084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561747470850 0x561747478ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.572+0000 7f54a61c1740 1 --2- 172.21.15.69:0/1352878084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747470310 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.572+0000 7f54a61c1740 1 -- 172.21.15.69:0/1352878084 >> 172.21.15.69:0/1352878084 conn(0x56174746a660 msgr2=0x56174746aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.572+0000 7f54a61c1740 1 -- 172.21.15.69:0/1352878084 shutdown_connections 2024-03-19T01:12:07.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.572+0000 7f54a0a00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54840a2b10 0x7f54840a4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f54900ccb70 tx=0x7f5490004a40 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:07.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.572+0000 7f54a61c1740 1 -- 172.21.15.69:0/1352878084 wait complete. 2024-03-19T01:12:07.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.572+0000 7f54a61c1740 1 -- start start 2024-03-19T01:12:07.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.573+0000 7f54a61c1740 1 Processor -- start 2024-03-19T01:12:07.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.573+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747476440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.573+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747470850 0x561747476980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.573+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561747479430 0x561747476ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.573+0000 7f54a61c1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56174747c1c0 con 0x561747470850 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.573+0000 7f54a61c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56174738ffc0 con 0x561747479430 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.573+0000 7f54a61c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56174747c340 con 0x56174746ff30 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.573+0000 7f54a61c1740 10 client.0 _ll_register_callbacks cb 0x56174738f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.574+0000 7f54a1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747476440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.574+0000 7f54a1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747476440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58692/0 (socket says 172.21.15.69:58692) 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.574+0000 7f54a1400640 1 -- 172.21.15.69:0/3808331658 learned_addr learned my addr 172.21.15.69:0/3808331658 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.574+0000 7f54a1400640 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561747479430 0x561747476ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.574+0000 7f549be00640 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747470850 0x561747476980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.574+0000 7f54a1400640 1 -- 172.21.15.69:0/3808331658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561747479430 msgr2=0x561747476ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.574+0000 7f54a1400640 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561747479430 0x561747476ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.574+0000 7f54a1400640 1 -- 172.21.15.69:0/3808331658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747470850 msgr2=0x561747476980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:07.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.574+0000 7f54a1400640 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747470850 0x561747476980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.574+0000 7f54a1400640 1 -- 172.21.15.69:0/3808331658 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617470e83f0 con 0x56174746ff30 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.575+0000 7f54a1400640 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747476440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5494009600 tx=0x7f5494003bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=459545547fa195e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.575+0000 7f5482000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.575+0000 7f5482000640 1 -- 172.21.15.69:0/3808331658 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494004220 con 0x56174746ff30 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.575+0000 7f5482000640 1 -- 172.21.15.69:0/3808331658 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f54940043c0 con 0x56174746ff30 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.575+0000 7f54a61c1740 1 -- 172.21.15.69:0/3808331658 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5494045070 con 0x56174746ff30 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.575+0000 7f5482000640 1 -- 172.21.15.69:0/3808331658 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f549405c050 con 0x56174746ff30 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.575+0000 7f54a61c1740 10 client.26005 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.575+0000 7f54a61c1740 1 -- 172.21.15.69:0/3808331658 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617470a80f0 con 0x56174746ff30 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.575+0000 7f5482000640 1 -- 172.21.15.69:0/3808331658 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494045070 con 0x56174746ff30 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.576+0000 7f5482000640 1 -- 172.21.15.69:0/3808331658 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f5494057030 con 0x56174746ff30 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.576+0000 7f54a61c1740 10 client.26005 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.576+0000 7f54a61c1740 10 client.26005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.576+0000 7f54a61c1740 20 client.26005 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.576+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561747489530 0x56174748b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.576+0000 7f54a61c1740 4 client.26005 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.576+0000 7f54a61c1740 1 -- 172.21.15.69:0/3808331658 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561747375610 con 0x561747489530 2024-03-19T01:12:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.576+0000 7f549be00640 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561747489530 0x56174748b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.577+0000 7f5482000640 10 client.26005 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.577+0000 7f549be00640 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561747489530 0x56174748b920 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fec105c869cb448 server_cookie=ba57614194096dc3 in_seq=0 out_seq=0 2024-03-19T01:12:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.579+0000 7f5482000640 1 -- 172.21.15.69:0/3808331658 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561747375610 con 0x561747489530 2024-03-19T01:12:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.579+0000 7f5482000640 10 client.26005 handle_command_reply: tid=0 2024-03-19T01:12:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.579+0000 7f54a61c1740 1 client.26005 shutdown 2024-03-19T01:12:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.579+0000 7f54a61c1740 1 -- 172.21.15.69:0/3808331658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 msgr2=0x561747476440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.579+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747476440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5494009600 tx=0x7f5494003bb0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.580+0000 7f54a61c1740 1 -- 172.21.15.69:0/3808331658 shutdown_connections 2024-03-19T01:12:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.580+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561747489530 0x56174748b920 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.580+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561747479430 0x561747476ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.580+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747470850 0x561747476980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.580+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3808331658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56174746ff30 0x561747476440 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.580+0000 7f54a61c1740 1 -- 172.21.15.69:0/3808331658 >> 172.21.15.69:0/3808331658 conn(0x56174746a660 msgr2=0x56174746c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:07.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.580+0000 7f5482000640 0 client.26005 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:07.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.580+0000 7f54a61c1740 1 -- 172.21.15.69:0/3808331658 shutdown_connections 2024-03-19T01:12:07.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.580+0000 7f54a61c1740 1 -- 172.21.15.69:0/3808331658 wait complete. 2024-03-19T01:12:07.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.580+0000 7f54a61c1740 20 client.26005 trim_cache size 0 max 16384 2024-03-19T01:12:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.601+0000 7f54a61c1740 1 Processor -- start 2024-03-19T01:12:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.601+0000 7f54a61c1740 1 -- start start 2024-03-19T01:12:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x561747478b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561747470850 0x5617474755a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747479430 0x561747484f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a61c1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b50e0 con 0x561747479430 2024-03-19T01:12:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a61c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b6cb0 con 0x56174746ff30 2024-03-19T01:12:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a61c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b57b0 con 0x561747470850 2024-03-19T01:12:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x561747478b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f549be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561747470850 0x5617474755a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x561747478b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37510/0 (socket says 172.21.15.69:37510) 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a1400640 1 -- 172.21.15.69:0/2961272448 learned_addr learned my addr 172.21.15.69:0/2961272448 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a1400640 1 --2- 172.21.15.69:0/2961272448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747479430 0x561747484f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a1400640 1 -- 172.21.15.69:0/2961272448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561747470850 msgr2=0x5617474755a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a1400640 1 --2- 172.21.15.69:0/2961272448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561747470850 0x5617474755a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a1400640 1 -- 172.21.15.69:0/2961272448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747479430 msgr2=0x561747484f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a1400640 1 --2- 172.21.15.69:0/2961272448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747479430 0x561747484f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f54a1400640 1 -- 172.21.15.69:0/2961272448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561747472020 con 0x56174746ff30 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.602+0000 7f549be00640 1 --2- 172.21.15.69:0/2961272448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561747470850 0x5617474755a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.603+0000 7f54a1400640 1 --2- 172.21.15.69:0/2961272448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x561747478b70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5494002230 tx=0x7f5494000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce28235fa1b77c0f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.603+0000 7f548d200640 1 -- 172.21.15.69:0/2961272448 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494039040 con 0x56174746ff30 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.603+0000 7f548d200640 1 -- 172.21.15.69:0/2961272448 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494045070 con 0x56174746ff30 2024-03-19T01:12:07.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.603+0000 7f548d200640 1 -- 172.21.15.69:0/2961272448 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494004040 con 0x56174746ff30 2024-03-19T01:12:07.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.604+0000 7f54a61c1740 1 -- 172.21.15.69:0/2961272448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 msgr2=0x561747478b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.604+0000 7f54a61c1740 1 --2- 172.21.15.69:0/2961272448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x561747478b70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5494002230 tx=0x7f5494000f40 comp rx=0 tx=0).stop 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.604+0000 7f54a61c1740 1 -- 172.21.15.69:0/2961272448 shutdown_connections 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.604+0000 7f54a61c1740 1 --2- 172.21.15.69:0/2961272448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747479430 0x561747484f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.604+0000 7f54a61c1740 1 --2- 172.21.15.69:0/2961272448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561747470850 0x5617474755a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.604+0000 7f54a61c1740 1 --2- 172.21.15.69:0/2961272448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x561747478b70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.604+0000 7f54a61c1740 1 -- 172.21.15.69:0/2961272448 >> 172.21.15.69:0/2961272448 conn(0x56174746a660 msgr2=0x561747468e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.604+0000 7f54a61c1740 1 -- 172.21.15.69:0/2961272448 shutdown_connections 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.604+0000 7f54a61c1740 1 -- 172.21.15.69:0/2961272448 wait complete. 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.604+0000 7f54a61c1740 1 -- start start 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.605+0000 7f54a61c1740 1 Processor -- start 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.605+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x56174747d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.605+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747470850 0x56174747d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.605+0000 7f54a61c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561747479430 0x5617474ee550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.605+0000 7f54a61c1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b6550 con 0x561747470850 2024-03-19T01:12:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.605+0000 7f54a61c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b5e80 con 0x56174746ff30 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.605+0000 7f54a61c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b30c0 con 0x561747479430 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.605+0000 7f54a61c1740 10 client.0 _ll_register_callbacks cb 0x7f5494009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f54a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x56174747d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f54a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x56174747d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37512/0 (socket says 172.21.15.69:37512) 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f54a1400640 1 -- 172.21.15.69:0/3435630820 learned_addr learned my addr 172.21.15.69:0/3435630820 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f54a1400640 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561747479430 0x5617474ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f549be00640 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747470850 0x56174747d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f54a1400640 1 -- 172.21.15.69:0/3435630820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561747479430 msgr2=0x5617474ee550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f54a1400640 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561747479430 0x5617474ee550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f54a1400640 1 -- 172.21.15.69:0/3435630820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747470850 msgr2=0x56174747d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f54a1400640 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747470850 0x56174747d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f54a1400640 1 -- 172.21.15.69:0/3435630820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56174748bc90 con 0x56174746ff30 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f54a1400640 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x56174747d290 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5494006010 tx=0x7f5494003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa70dae024a3f96c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:07.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.606+0000 7f5482000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.607+0000 7f5482000640 1 -- 172.21.15.69:0/3435630820 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494004d40 con 0x56174746ff30 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.607+0000 7f5482000640 1 -- 172.21.15.69:0/3435630820 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f5494040450 con 0x56174746ff30 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.607+0000 7f54a61c1740 1 -- 172.21.15.69:0/3435630820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5494045070 con 0x56174746ff30 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.607+0000 7f5482000640 1 -- 172.21.15.69:0/3435630820 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5494073050 con 0x56174746ff30 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.607+0000 7f54a61c1740 10 client.26018 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.607+0000 7f54a61c1740 1 -- 172.21.15.69:0/3435630820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617474efa70 con 0x56174746ff30 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.607+0000 7f5482000640 1 -- 172.21.15.69:0/3435630820 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494045070 con 0x56174746ff30 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.607+0000 7f5482000640 1 -- 172.21.15.69:0/3435630820 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f5494057040 con 0x56174746ff30 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.607+0000 7f54a61c1740 10 client.26018 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.607+0000 7f54a61c1740 10 client.26018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.607+0000 7f54a61c1740 20 client.26018 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.608+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5617474f01a0 0x5617474f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.608+0000 7f54a61c1740 4 client.26018 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:12:07.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.608+0000 7f54a61c1740 1 -- 172.21.15.69:0/3435630820 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5617473e1800 con 0x5617474f01a0 2024-03-19T01:12:07.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.608+0000 7f549be00640 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5617474f01a0 0x5617474f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.608+0000 7f549be00640 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5617474f01a0 0x5617474f25c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a83c632d8b28d7 server_cookie=e7c6011551d0a12 in_seq=0 out_seq=0 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.608+0000 7f5482000640 10 client.26018 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f5482000640 1 -- 172.21.15.69:0/3435630820 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5617473e1800 con 0x5617474f01a0 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f5482000640 10 client.26018 handle_command_reply: tid=0 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 1 client.26018 shutdown 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 1 -- 172.21.15.69:0/3435630820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 msgr2=0x56174747d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x56174747d290 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5494006010 tx=0x7f5494003970 comp rx=0 tx=0).stop 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 1 -- 172.21.15.69:0/3435630820 shutdown_connections 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5617474f01a0 0x5617474f25c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561747479430 0x5617474ee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561747470850 0x56174747d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 1 --2- 172.21.15.69:0/3435630820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174746ff30 0x56174747d290 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 1 -- 172.21.15.69:0/3435630820 >> 172.21.15.69:0/3435630820 conn(0x56174746a660 msgr2=0x561747476c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:07.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f5482000640 0 client.26018 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:07.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 1 -- 172.21.15.69:0/3435630820 shutdown_connections 2024-03-19T01:12:07.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 1 -- 172.21.15.69:0/3435630820 wait complete. 2024-03-19T01:12:07.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.611+0000 7f54a61c1740 20 client.26018 trim_cache size 0 max 16384 2024-03-19T01:12:07.614 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:12:07.614 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:12:07.614 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:12:07.614 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:12:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 -- 172.21.15.69:0/3635295234 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54840a2b10 msgr2=0x7f54840a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54840a2b10 0x7f54840a4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f54900ccb70 tx=0x7f5490004a40 comp rx=0 tx=0).stop 2024-03-19T01:12:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 -- 172.21.15.69:0/3635295234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549c075d50 msgr2=0x7f549c19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549c075d50 0x7f549c19e600 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f549000e9f0 tx=0x7f549000edf0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 -- 172.21.15.69:0/3635295234 shutdown_connections 2024-03-19T01:12:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c112540 0x7f549c1a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c076670 0x7f549c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54840a2b10 0x7f54840a4fd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 --2- 172.21.15.69:0/3635295234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549c075d50 0x7f549c19e600 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 -- 172.21.15.69:0/3635295234 >> 172.21.15.69:0/3635295234 conn(0x7f549c0ff330 msgr2=0x7f549c113c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:07.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 -- 172.21.15.69:0/3635295234 shutdown_connections 2024-03-19T01:12:07.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.612+0000 7f54a1e00640 1 -- 172.21.15.69:0/3635295234 wait complete. 2024-03-19T01:12:07.632 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:12:07.632 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:12:07.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.888+0000 7f04f4600640 1 -- 172.21.15.69:0/654493203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec073bd0 msgr2=0x7f04ec073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.888+0000 7f04f4600640 1 --2- 172.21.15.69:0/654493203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec073bd0 0x7f04ec073fb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009ca0 tx=0x7f04e002c180 comp rx=0 tx=0).stop 2024-03-19T01:12:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.888+0000 7f04f4600640 1 -- 172.21.15.69:0/654493203 shutdown_connections 2024-03-19T01:12:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.888+0000 7f04f4600640 1 --2- 172.21.15.69:0/654493203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04ec10e070 0x7f04ec110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.888+0000 7f04f4600640 1 --2- 172.21.15.69:0/654493203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ec0744f0 0x7f04ec10db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.888+0000 7f04f4600640 1 --2- 172.21.15.69:0/654493203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec073bd0 0x7f04ec073fb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.888+0000 7f04f4600640 1 -- 172.21.15.69:0/654493203 >> 172.21.15.69:0/654493203 conn(0x7f04ec0fd2c0 msgr2=0x7f04ec0ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.888+0000 7f04f4600640 1 -- 172.21.15.69:0/654493203 shutdown_connections 2024-03-19T01:12:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.888+0000 7f04f4600640 1 -- 172.21.15.69:0/654493203 wait complete. 2024-03-19T01:12:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.889+0000 7f04f4600640 1 Processor -- start 2024-03-19T01:12:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.890+0000 7f04f4600640 1 -- start start 2024-03-19T01:12:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.890+0000 7f04f4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec073bd0 0x7f04ec19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.890+0000 7f04f4600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04ec0744f0 0x7f04ec19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.890+0000 7f04f4600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ec10e070 0x7f04ec19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.890+0000 7f04f4600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec113d90 con 0x7f04ec10e070 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.890+0000 7f04f4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec113c10 con 0x7f04ec073bd0 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.890+0000 7f04f4600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec113f10 con 0x7f04ec0744f0 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.890+0000 7f04f1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec073bd0 0x7f04ec19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.890+0000 7f04f1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec073bd0 0x7f04ec19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37528/0 (socket says 172.21.15.69:37528) 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.890+0000 7f04f1c00640 1 -- 172.21.15.69:0/2876940233 learned_addr learned my addr 172.21.15.69:0/2876940233 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.890+0000 7f04f1200640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04ec0744f0 0x7f04ec19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04f2600640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ec10e070 0x7f04ec19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04f1c00640 1 -- 172.21.15.69:0/2876940233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04ec0744f0 msgr2=0x7f04ec19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04f1c00640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04ec0744f0 0x7f04ec19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04f1c00640 1 -- 172.21.15.69:0/2876940233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ec10e070 msgr2=0x7f04ec19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:07.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04f1c00640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ec10e070 0x7f04ec19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:07.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04f1c00640 1 -- 172.21.15.69:0/2876940233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ec19f2e0 con 0x7f04ec073bd0 2024-03-19T01:12:07.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04f2600640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ec10e070 0x7f04ec19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:07.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04f1c00640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec073bd0 0x7f04ec19a2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f04e0005e30 tx=0x7f04e0035e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:07.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04ea000640 1 -- 172.21.15.69:0/2876940233 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0039040 con 0x7f04ec073bd0 2024-03-19T01:12:07.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04ea000640 1 -- 172.21.15.69:0/2876940233 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e0044070 con 0x7f04ec073bd0 2024-03-19T01:12:07.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04ea000640 1 -- 172.21.15.69:0/2876940233 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0032960 con 0x7f04ec073bd0 2024-03-19T01:12:07.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.891+0000 7f04f4600640 1 -- 172.21.15.69:0/2876940233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ec19f570 con 0x7f04ec073bd0 2024-03-19T01:12:07.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.892+0000 7f04f4600640 1 -- 172.21.15.69:0/2876940233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ec1a6e60 con 0x7f04ec073bd0 2024-03-19T01:12:07.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.893+0000 7f04f4600640 1 -- 172.21.15.69:0/2876940233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ec068a60 con 0x7f04ec073bd0 2024-03-19T01:12:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.895+0000 7f04ea000640 1 -- 172.21.15.69:0/2876940233 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7f04e0032b00 con 0x7f04ec073bd0 2024-03-19T01:12:07.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.896+0000 7f04ea000640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04d40a2b10 0x7f04d40a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:07.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.896+0000 7f04ea000640 1 -- 172.21.15.69:0/2876940233 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f04e00f1570 con 0x7f04ec073bd0 2024-03-19T01:12:07.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.896+0000 7f04f1200640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04d40a2b10 0x7f04d40a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:07.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.897+0000 7f04f1200640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04d40a2b10 0x7f04d40a4fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0ff9c0 tx=0x7f04d8005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:07.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:07.905+0000 7f04ea000640 1 -- 172.21.15.69:0/2876940233 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f04e004e020 con 0x7f04ec073bd0 2024-03-19T01:12:08.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.187+0000 7f04f4600640 1 -- 172.21.15.69:0/2876940233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04ec19b640 con 0x7f04ec073bd0 2024-03-19T01:12:08.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.188+0000 7f04ea000640 1 -- 172.21.15.69:0/2876940233 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f04e00bde60 con 0x7f04ec073bd0 2024-03-19T01:12:08.189 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:12:08.190 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:12:03.544058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24476,24581,24580,24550,24494]},"id":1}]} 2024-03-19T01:12:08.190 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 30 2024-03-19T01:12:08.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 -- 172.21.15.69:0/2876940233 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04d40a2b10 msgr2=0x7f04d40a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04d40a2b10 0x7f04d40a4fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0ff9c0 tx=0x7f04d8005f90 comp rx=0 tx=0).stop 2024-03-19T01:12:08.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 -- 172.21.15.69:0/2876940233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec073bd0 msgr2=0x7f04ec19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec073bd0 0x7f04ec19a2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f04e0005e30 tx=0x7f04e0035e00 comp rx=0 tx=0).stop 2024-03-19T01:12:08.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 -- 172.21.15.69:0/2876940233 shutdown_connections 2024-03-19T01:12:08.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04ec10e070 0x7f04ec19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04ec0744f0 0x7f04ec19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f04d40a2b10 0x7f04d40a4fd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 --2- 172.21.15.69:0/2876940233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec073bd0 0x7f04ec19a2f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 -- 172.21.15.69:0/2876940233 >> 172.21.15.69:0/2876940233 conn(0x7f04ec0fd2c0 msgr2=0x7f04ec0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 -- 172.21.15.69:0/2876940233 shutdown_connections 2024-03-19T01:12:08.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.192+0000 7f04f4600640 1 -- 172.21.15.69:0/2876940233 wait complete. 2024-03-19T01:12:08.217 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:12:08.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.434+0000 7f0a75a00640 1 -- 172.21.15.69:0/2524760867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a70100640 msgr2=0x7f0a70100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.434+0000 7f0a75a00640 1 --2- 172.21.15.69:0/2524760867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a70100640 0x7f0a70100a20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009900 tx=0x7f0a6002bcd0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.434+0000 7f0a75a00640 1 -- 172.21.15.69:0/2524760867 shutdown_connections 2024-03-19T01:12:08.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.434+0000 7f0a75a00640 1 --2- 172.21.15.69:0/2524760867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a70105a40 0x7f0a70105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.434+0000 7f0a75a00640 1 --2- 172.21.15.69:0/2524760867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a70100f60 0x7f0a70105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.434+0000 7f0a75a00640 1 --2- 172.21.15.69:0/2524760867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a70100640 0x7f0a70100a20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.434+0000 7f0a75a00640 1 -- 172.21.15.69:0/2524760867 >> 172.21.15.69:0/2524760867 conn(0x7f0a700fc070 msgr2=0x7f0a700fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.434+0000 7f0a75a00640 1 -- 172.21.15.69:0/2524760867 shutdown_connections 2024-03-19T01:12:08.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.435+0000 7f0a75a00640 1 -- 172.21.15.69:0/2524760867 wait complete. 2024-03-19T01:12:08.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.436+0000 7f0a75a00640 1 Processor -- start 2024-03-19T01:12:08.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.436+0000 7f0a75a00640 1 -- start start 2024-03-19T01:12:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.436+0000 7f0a75a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a70100640 0x7f0a7019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.436+0000 7f0a75a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a70100f60 0x7f0a7019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.436+0000 7f0a75a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a70105a40 0x7f0a701a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.436+0000 7f0a75a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a70118010 con 0x7f0a70100f60 2024-03-19T01:12:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.436+0000 7f0a75a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a70117e90 con 0x7f0a70100640 2024-03-19T01:12:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.436+0000 7f0a75a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0a70118190 con 0x7f0a70105a40 2024-03-19T01:12:08.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a70100640 0x7f0a7019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a70100640 0x7f0a7019e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37554/0 (socket says 172.21.15.69:37554) 2024-03-19T01:12:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a6fe00640 1 -- 172.21.15.69:0/135909713 learned_addr learned my addr 172.21.15.69:0/135909713 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a75000640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a70105a40 0x7f0a701a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a67400640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a70100f60 0x7f0a7019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a6fe00640 1 -- 172.21.15.69:0/135909713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a70105a40 msgr2=0x7f0a701a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a70105a40 0x7f0a701a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a6fe00640 1 -- 172.21.15.69:0/135909713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a70100f60 msgr2=0x7f0a7019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a70100f60 0x7f0a7019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a6fe00640 1 -- 172.21.15.69:0/135909713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a701a34c0 con 0x7f0a70100640 2024-03-19T01:12:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a67400640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a70100f60 0x7f0a7019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:08.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.437+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a70100640 0x7f0a7019e4d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0a60005ab0 tx=0x7f0a60004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.438+0000 7f0a6d600640 1 -- 172.21.15.69:0/135909713 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60043020 con 0x7f0a70100640 2024-03-19T01:12:08.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.438+0000 7f0a6d600640 1 -- 172.21.15.69:0/135909713 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a600054e0 con 0x7f0a70100640 2024-03-19T01:12:08.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.438+0000 7f0a6d600640 1 -- 172.21.15.69:0/135909713 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60003bd0 con 0x7f0a70100640 2024-03-19T01:12:08.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.438+0000 7f0a75a00640 1 -- 172.21.15.69:0/135909713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a701a3750 con 0x7f0a70100640 2024-03-19T01:12:08.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.438+0000 7f0a75a00640 1 -- 172.21.15.69:0/135909713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a701ab040 con 0x7f0a70100640 2024-03-19T01:12:08.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.442+0000 7f0a6d600640 1 -- 172.21.15.69:0/135909713 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7f0a60046020 con 0x7f0a70100640 2024-03-19T01:12:08.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.443+0000 7f0a799fd740 1 Processor -- start 2024-03-19T01:12:08.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.443+0000 7f0a799fd740 1 -- start start 2024-03-19T01:12:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.443+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 0x55682a088fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.443+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a089510 0x55682a091bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.443+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a0920f0 0x55682a0944e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.444+0000 7f0a799fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556829fed380 con 0x55682a0920f0 2024-03-19T01:12:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.444+0000 7f0a799fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556829f95d20 con 0x55682a089510 2024-03-19T01:12:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.444+0000 7f0a799fd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556829fe9420 con 0x55682a088bf0 2024-03-19T01:12:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.444+0000 7f0a6d600640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0a540a2b10 0x7f0a540a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.444+0000 7f0a6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a089510 0x55682a091bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.444+0000 7f0a67400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a0920f0 0x55682a0944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.444+0000 7f0a6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a089510 0x55682a091bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37564/0 (socket says 172.21.15.69:37564) 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.444+0000 7f0a75000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 0x55682a088fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.444+0000 7f0a6fe00640 1 -- 172.21.15.69:0/4077465043 learned_addr learned my addr 172.21.15.69:0/4077465043 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.444+0000 7f0a6d600640 1 -- 172.21.15.69:0/135909713 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f0a600f1390 con 0x7f0a70100640 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.444+0000 7f0a75000640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0a540a2b10 0x7f0a540a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.445+0000 7f0a6fe00640 1 -- 172.21.15.69:0/4077465043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 msgr2=0x55682a088fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.445+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/4077465043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 0x55682a088fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.445+0000 7f0a6fe00640 1 -- 172.21.15.69:0/4077465043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a0920f0 msgr2=0x55682a0944e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.445+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/4077465043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a0920f0 0x55682a0944e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.445+0000 7f0a6fe00640 1 -- 172.21.15.69:0/4077465043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556829d013f0 con 0x55682a089510 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.445+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/4077465043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a089510 0x55682a091bb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0a600c5760 tx=0x7f0a6002bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b254bcf36931fd00 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.445+0000 7f0a65600640 1 -- 172.21.15.69:0/4077465043 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600bd030 con 0x55682a089510 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.445+0000 7f0a65600640 1 -- 172.21.15.69:0/4077465043 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a60004c80 con 0x55682a089510 2024-03-19T01:12:08.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.445+0000 7f0a65600640 1 -- 172.21.15.69:0/4077465043 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600bce50 con 0x55682a089510 2024-03-19T01:12:08.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.446+0000 7f0a799fd740 1 -- 172.21.15.69:0/4077465043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a089510 msgr2=0x55682a091bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.446+0000 7f0a75000640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0a540a2b10 0x7f0a540a4fd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009fd0 tx=0x7f0a68013040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.446+0000 7f0a799fd740 1 --2- 172.21.15.69:0/4077465043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a089510 0x55682a091bb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0a600c5760 tx=0x7f0a6002bcd0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.446+0000 7f0a799fd740 1 -- 172.21.15.69:0/4077465043 shutdown_connections 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.446+0000 7f0a799fd740 1 --2- 172.21.15.69:0/4077465043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a0920f0 0x55682a0944e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.446+0000 7f0a799fd740 1 --2- 172.21.15.69:0/4077465043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a089510 0x55682a091bb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.446+0000 7f0a799fd740 1 --2- 172.21.15.69:0/4077465043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 0x55682a088fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.446+0000 7f0a799fd740 1 -- 172.21.15.69:0/4077465043 >> 172.21.15.69:0/4077465043 conn(0x55682a083320 msgr2=0x55682a083700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.446+0000 7f0a799fd740 1 -- 172.21.15.69:0/4077465043 shutdown_connections 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.446+0000 7f0a799fd740 1 -- 172.21.15.69:0/4077465043 wait complete. 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.447+0000 7f0a799fd740 1 -- start start 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.447+0000 7f0a799fd740 1 Processor -- start 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a088bf0 0x55682a08f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 0x55682a08f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a0920f0 0x55682a08fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a799fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55682a094ca0 con 0x55682a0920f0 2024-03-19T01:12:08.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a799fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556829fa9160 con 0x55682a088bf0 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a799fd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55682a094e20 con 0x55682a089510 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a088bf0 0x55682a08f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a67400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 0x55682a08f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a799fd740 10 client.0 _ll_register_callbacks cb 0x556829fe5170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a088bf0 0x55682a08f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37576/0 (socket says 172.21.15.69:37576) 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a6fe00640 1 -- 172.21.15.69:0/474422473 learned_addr learned my addr 172.21.15.69:0/474422473 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a75000640 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a0920f0 0x55682a08fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a67400640 1 -- 172.21.15.69:0/474422473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a088bf0 msgr2=0x55682a08f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a67400640 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a088bf0 0x55682a08f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a67400640 1 -- 172.21.15.69:0/474422473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a0920f0 msgr2=0x55682a08fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a67400640 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a0920f0 0x55682a08fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.448+0000 7f0a67400640 1 -- 172.21.15.69:0/474422473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556829dcb7d0 con 0x55682a089510 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.449+0000 7f0a67400640 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 0x55682a08f610 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a580126d0 tx=0x7f0a58012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=506913d183854823 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.449+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a088bf0 0x55682a08f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:08.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.449+0000 7f0a5e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:08.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.449+0000 7f0a5e000640 1 -- 172.21.15.69:0/474422473 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58004320 con 0x55682a089510 2024-03-19T01:12:08.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.449+0000 7f0a5e000640 1 -- 172.21.15.69:0/474422473 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f0a58011940 con 0x55682a089510 2024-03-19T01:12:08.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.449+0000 7f0a799fd740 1 -- 172.21.15.69:0/474422473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a60004c80 con 0x55682a089510 2024-03-19T01:12:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.449+0000 7f0a5e000640 1 -- 172.21.15.69:0/474422473 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a580044c0 con 0x55682a089510 2024-03-19T01:12:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.449+0000 7f0a799fd740 10 client.26029 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.449+0000 7f0a799fd740 1 -- 172.21.15.69:0/474422473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55682a0a0e80 con 0x55682a089510 2024-03-19T01:12:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.449+0000 7f0a5e000640 1 -- 172.21.15.69:0/474422473 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a60004c80 con 0x55682a089510 2024-03-19T01:12:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.450+0000 7f0a5e000640 1 -- 172.21.15.69:0/474422473 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f0a5800f440 con 0x55682a089510 2024-03-19T01:12:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.450+0000 7f0a799fd740 10 client.26029 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.450+0000 7f0a799fd740 10 client.26029 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:12:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.450+0000 7f0a799fd740 20 client.26029 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.450+0000 7f0a799fd740 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55682a0a2340 0x55682a0a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.450+0000 7f0a799fd740 4 client.26029 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:08.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.450+0000 7f0a799fd740 1 -- 172.21.15.69:0/474422473 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556829ffa8c0 con 0x55682a0a2340 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.450+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55682a0a2340 0x55682a0a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.451+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55682a0a2340 0x55682a0a47c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1f77a737535f829 server_cookie=a213aa4969d5478a in_seq=0 out_seq=0 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.451+0000 7f0a5e000640 10 client.26029 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a5e000640 1 -- 172.21.15.69:0/474422473 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556829ffa8c0 con 0x55682a0a2340 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a5e000640 10 client.26029 handle_command_reply: tid=0 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a799fd740 1 client.26029 shutdown 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a799fd740 1 -- 172.21.15.69:0/474422473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 msgr2=0x55682a08f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a799fd740 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 0x55682a08f610 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a580126d0 tx=0x7f0a58012b90 comp rx=0 tx=0).stop 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a799fd740 1 -- 172.21.15.69:0/474422473 shutdown_connections 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a799fd740 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55682a0a2340 0x55682a0a47c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a799fd740 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a0920f0 0x55682a08fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a799fd740 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 0x55682a08f610 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a5e000640 0 client.26029 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a799fd740 1 --2- 172.21.15.69:0/474422473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a088bf0 0x55682a08f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a799fd740 1 -- 172.21.15.69:0/474422473 >> 172.21.15.69:0/474422473 conn(0x55682a083320 msgr2=0x55682a085070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.452+0000 7f0a799fd740 1 -- 172.21.15.69:0/474422473 shutdown_connections 2024-03-19T01:12:08.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.453+0000 7f0a799fd740 1 -- 172.21.15.69:0/474422473 wait complete. 2024-03-19T01:12:08.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.453+0000 7f0a799fd740 20 client.26029 trim_cache size 0 max 16384 2024-03-19T01:12:08.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.481+0000 7f0a799fd740 1 Processor -- start 2024-03-19T01:12:08.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.481+0000 7f0a799fd740 1 -- start start 2024-03-19T01:12:08.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.481+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 0x55682a091800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a089510 0x55682a09d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a0920f0 0x55682a0aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a799fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701b50d0 con 0x55682a089510 2024-03-19T01:12:08.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a799fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701b6ca0 con 0x55682a0920f0 2024-03-19T01:12:08.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a799fd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701b57a0 con 0x55682a088bf0 2024-03-19T01:12:08.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a67400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a089510 0x55682a09d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a6fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 0x55682a091800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a75000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a0920f0 0x55682a0aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a6fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 0x55682a091800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58784/0 (socket says 172.21.15.69:58784) 2024-03-19T01:12:08.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a6fe00640 1 -- 172.21.15.69:0/1744023299 learned_addr learned my addr 172.21.15.69:0/1744023299 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a6fe00640 1 -- 172.21.15.69:0/1744023299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a0920f0 msgr2=0x55682a0aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/1744023299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a0920f0 0x55682a0aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a6fe00640 1 -- 172.21.15.69:0/1744023299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a089510 msgr2=0x55682a09d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/1744023299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a089510 0x55682a09d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a6fe00640 1 -- 172.21.15.69:0/1744023299 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682a09ded0 con 0x55682a088bf0 2024-03-19T01:12:08.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.482+0000 7f0a67400640 1 --2- 172.21.15.69:0/1744023299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a089510 0x55682a09d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:08.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.483+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/1744023299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 0x55682a091800 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009a20 tx=0x7f0a600b98c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45ef87c0c9b60d7a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.483+0000 7f0a64c00640 1 -- 172.21.15.69:0/1744023299 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600bd030 con 0x55682a088bf0 2024-03-19T01:12:08.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.483+0000 7f0a64c00640 1 -- 172.21.15.69:0/1744023299 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a60004c80 con 0x55682a088bf0 2024-03-19T01:12:08.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.483+0000 7f0a64c00640 1 -- 172.21.15.69:0/1744023299 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600c5c90 con 0x55682a088bf0 2024-03-19T01:12:08.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.484+0000 7f0a799fd740 1 -- 172.21.15.69:0/1744023299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 msgr2=0x55682a091800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.484+0000 7f0a799fd740 1 --2- 172.21.15.69:0/1744023299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 0x55682a091800 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009a20 tx=0x7f0a600b98c0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.484+0000 7f0a799fd740 1 -- 172.21.15.69:0/1744023299 shutdown_connections 2024-03-19T01:12:08.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.484+0000 7f0a799fd740 1 --2- 172.21.15.69:0/1744023299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a0920f0 0x55682a0aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.484+0000 7f0a799fd740 1 --2- 172.21.15.69:0/1744023299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a089510 0x55682a09d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.484+0000 7f0a799fd740 1 --2- 172.21.15.69:0/1744023299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a088bf0 0x55682a091800 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.484+0000 7f0a799fd740 1 -- 172.21.15.69:0/1744023299 >> 172.21.15.69:0/1744023299 conn(0x55682a083320 msgr2=0x55682a082330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.484+0000 7f0a799fd740 1 -- 172.21.15.69:0/1744023299 shutdown_connections 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.484+0000 7f0a799fd740 1 -- 172.21.15.69:0/1744023299 wait complete. 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.484+0000 7f0a799fd740 1 -- start start 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.485+0000 7f0a799fd740 1 Processor -- start 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.485+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a088bf0 0x55682a08bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.485+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 0x55682a08c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.485+0000 7f0a799fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a0920f0 0x55682a08c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.485+0000 7f0a799fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701b6540 con 0x55682a088bf0 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.485+0000 7f0a799fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701b5e70 con 0x55682a0920f0 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.485+0000 7f0a799fd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701b30b0 con 0x55682a089510 2024-03-19T01:12:08.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.485+0000 7f0a799fd740 10 client.0 _ll_register_callbacks cb 0x7f0a58009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a67400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 0x55682a08c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a67400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 0x55682a08c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58792/0 (socket says 172.21.15.69:58792) 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a67400640 1 -- 172.21.15.69:0/3102858821 learned_addr learned my addr 172.21.15.69:0/3102858821 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a6fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a088bf0 0x55682a08bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a75000640 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a0920f0 0x55682a08c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a67400640 1 -- 172.21.15.69:0/3102858821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a0920f0 msgr2=0x55682a08c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a67400640 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a0920f0 0x55682a08c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a67400640 1 -- 172.21.15.69:0/3102858821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a088bf0 msgr2=0x55682a08bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a67400640 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a088bf0 0x55682a08bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a67400640 1 -- 172.21.15.69:0/3102858821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682a107700 con 0x55682a089510 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a67400640 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 0x55682a08c120 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55682a08e930 tx=0x7f0a5800ff00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0acc5117acbadf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.486+0000 7f0a5e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:08.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.487+0000 7f0a5e000640 1 -- 172.21.15.69:0/3102858821 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5800f650 con 0x55682a089510 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.487+0000 7f0a799fd740 1 -- 172.21.15.69:0/3102858821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a60004c80 con 0x55682a089510 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.487+0000 7f0a5e000640 1 -- 172.21.15.69:0/3102858821 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f0a58018070 con 0x55682a089510 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.487+0000 7f0a5e000640 1 -- 172.21.15.69:0/3102858821 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a58005030 con 0x55682a089510 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.487+0000 7f0a799fd740 10 client.26035 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.487+0000 7f0a799fd740 1 -- 172.21.15.69:0/3102858821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55682a108890 con 0x55682a089510 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.487+0000 7f0a5e000640 1 -- 172.21.15.69:0/3102858821 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a60004c80 con 0x55682a089510 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.487+0000 7f0a5e000640 1 -- 172.21.15.69:0/3102858821 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f0a580046f0 con 0x55682a089510 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.487+0000 7f0a799fd740 10 client.26035 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.487+0000 7f0a799fd740 10 client.26035 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.487+0000 7f0a799fd740 20 client.26035 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.488+0000 7f0a799fd740 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55682a1090c0 0x55682a10b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.488+0000 7f0a799fd740 4 client.26035 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:12:08.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.488+0000 7f0a799fd740 1 -- 172.21.15.69:0/3102858821 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556829ff4c10 con 0x55682a1090c0 2024-03-19T01:12:08.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.488+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55682a1090c0 0x55682a10b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.488+0000 7f0a6fe00640 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55682a1090c0 0x55682a10b4e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=209cd5692e8b101a server_cookie=309969f90d93c220 in_seq=0 out_seq=0 2024-03-19T01:12:08.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.488+0000 7f0a5e000640 10 client.26035 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:12:08.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a5e000640 1 -- 172.21.15.69:0/3102858821 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556829ff4c10 con 0x55682a1090c0 2024-03-19T01:12:08.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a5e000640 10 client.26035 handle_command_reply: tid=0 2024-03-19T01:12:08.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 1 client.26035 shutdown 2024-03-19T01:12:08.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 1 -- 172.21.15.69:0/3102858821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 msgr2=0x55682a08c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 0x55682a08c120 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55682a08e930 tx=0x7f0a5800ff00 comp rx=0 tx=0).stop 2024-03-19T01:12:08.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 1 -- 172.21.15.69:0/3102858821 shutdown_connections 2024-03-19T01:12:08.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55682a1090c0 0x55682a10b4e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55682a0920f0 0x55682a08c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55682a089510 0x55682a08c120 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 1 --2- 172.21.15.69:0/3102858821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55682a088bf0 0x55682a08bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a5e000640 0 client.26035 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:12:08.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 1 -- 172.21.15.69:0/3102858821 >> 172.21.15.69:0/3102858821 conn(0x55682a083320 msgr2=0x55682a082330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 1 -- 172.21.15.69:0/3102858821 shutdown_connections 2024-03-19T01:12:08.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 1 -- 172.21.15.69:0/3102858821 wait complete. 2024-03-19T01:12:08.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.489+0000 7f0a799fd740 20 client.26035 trim_cache size 0 max 16384 2024-03-19T01:12:08.494 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:12:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 -- 172.21.15.69:0/135909713 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0a540a2b10 msgr2=0x7f0a540a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0a540a2b10 0x7f0a540a4fd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009fd0 tx=0x7f0a68013040 comp rx=0 tx=0).stop 2024-03-19T01:12:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 -- 172.21.15.69:0/135909713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a70100640 msgr2=0x7f0a7019e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a70100640 0x7f0a7019e4d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0a60005ab0 tx=0x7f0a60004970 comp rx=0 tx=0).stop 2024-03-19T01:12:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 -- 172.21.15.69:0/135909713 shutdown_connections 2024-03-19T01:12:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a70105a40 0x7f0a701a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a70100f60 0x7f0a7019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0a540a2b10 0x7f0a540a4fd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 --2- 172.21.15.69:0/135909713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a70100640 0x7f0a7019e4d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 -- 172.21.15.69:0/135909713 >> 172.21.15.69:0/135909713 conn(0x7f0a700fc070 msgr2=0x7f0a701033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 -- 172.21.15.69:0/135909713 shutdown_connections 2024-03-19T01:12:08.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.490+0000 7f0a75a00640 1 -- 172.21.15.69:0/135909713 wait complete. 2024-03-19T01:12:08.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:08 smithi086 ceph-mon[66829]: pgmap v47: 129 pgs: 129 active+clean; 102 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 5.4 MiB/s wr, 75 op/s 2024-03-19T01:12:08.506 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:08 smithi069 ceph-mon[31727]: pgmap v47: 129 pgs: 129 active+clean; 102 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 5.4 MiB/s wr, 75 op/s 2024-03-19T01:12:08.508 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:12:08.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:08 smithi163 ceph-mon[67428]: pgmap v47: 129 pgs: 129 active+clean; 102 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 5.4 MiB/s wr, 75 op/s 2024-03-19T01:12:08.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.715+0000 7f83f3e00640 1 -- 172.21.15.69:0/3700317043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83f4074510 msgr2=0x7f83f410d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.715+0000 7f83f3e00640 1 --2- 172.21.15.69:0/3700317043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83f4074510 0x7f83f410d9f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009900 tx=0x7f83dc02bcc0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.715+0000 7f83f3e00640 1 -- 172.21.15.69:0/3700317043 shutdown_connections 2024-03-19T01:12:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.715+0000 7f83f3e00640 1 --2- 172.21.15.69:0/3700317043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f410df30 0x7f83f4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.715+0000 7f83f3e00640 1 --2- 172.21.15.69:0/3700317043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83f4074510 0x7f83f410d9f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.715+0000 7f83f3e00640 1 --2- 172.21.15.69:0/3700317043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83f4073bf0 0x7f83f4073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.715+0000 7f83f3e00640 1 -- 172.21.15.69:0/3700317043 >> 172.21.15.69:0/3700317043 conn(0x7f83f40fd1c0 msgr2=0x7f83f40ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.716+0000 7f83f3e00640 1 -- 172.21.15.69:0/3700317043 shutdown_connections 2024-03-19T01:12:08.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.716+0000 7f83f3e00640 1 -- 172.21.15.69:0/3700317043 wait complete. 2024-03-19T01:12:08.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.717+0000 7f83f3e00640 1 Processor -- start 2024-03-19T01:12:08.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.717+0000 7f83f3e00640 1 -- start start 2024-03-19T01:12:08.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.717+0000 7f83f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83f4073bf0 0x7f83f419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.717+0000 7f83f3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83f4074510 0x7f83f419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.717+0000 7f83f3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f410df30 0x7f83f419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f3e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4113cb0 con 0x7f83f410df30 2024-03-19T01:12:08.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4113b30 con 0x7f83f4073bf0 2024-03-19T01:12:08.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f3e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4113e30 con 0x7f83f4074510 2024-03-19T01:12:08.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83f4073bf0 0x7f83f419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83f4073bf0 0x7f83f419a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37598/0 (socket says 172.21.15.69:37598) 2024-03-19T01:12:08.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f2a00640 1 -- 172.21.15.69:0/2187691657 learned_addr learned my addr 172.21.15.69:0/2187691657 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83ea000640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83f4074510 0x7f83f419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f3400640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f410df30 0x7f83f419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f2a00640 1 -- 172.21.15.69:0/2187691657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83f4074510 msgr2=0x7f83f419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f2a00640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83f4074510 0x7f83f419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f2a00640 1 -- 172.21.15.69:0/2187691657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f410df30 msgr2=0x7f83f419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f2a00640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f410df30 0x7f83f419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f2a00640 1 -- 172.21.15.69:0/2187691657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f419f220 con 0x7f83f4073bf0 2024-03-19T01:12:08.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.718+0000 7f83f2a00640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83f4073bf0 0x7f83f419a280 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f83e400eb90 tx=0x7f83e400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.719+0000 7f83f8a00640 1 -- 172.21.15.69:0/2187691657 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4004d20 con 0x7f83f4073bf0 2024-03-19T01:12:08.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.719+0000 7f83f8a00640 1 -- 172.21.15.69:0/2187691657 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e400ee90 con 0x7f83f4073bf0 2024-03-19T01:12:08.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.719+0000 7f83f8a00640 1 -- 172.21.15.69:0/2187691657 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4010830 con 0x7f83f4073bf0 2024-03-19T01:12:08.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.719+0000 7f83f3e00640 1 -- 172.21.15.69:0/2187691657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f419f510 con 0x7f83f4073bf0 2024-03-19T01:12:08.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.719+0000 7f83f3e00640 1 -- 172.21.15.69:0/2187691657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f41a6da0 con 0x7f83f4073bf0 2024-03-19T01:12:08.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.723+0000 7f83fcbc4740 1 Processor -- start 2024-03-19T01:12:08.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.723+0000 7f83fcbc4740 1 -- start start 2024-03-19T01:12:08.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e37fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 0x55c9d8e40bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e410f0 0x55c9d8e434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8d9c380 con 0x55c9d8e410f0 2024-03-19T01:12:08.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8d44d20 con 0x55c9d8e37bf0 2024-03-19T01:12:08.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8d98420 con 0x55c9d8e38510 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83f8a00640 1 -- 172.21.15.69:0/2187691657 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7f83e4005020 con 0x7f83f4073bf0 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e37fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e37fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37608/0 (socket says 172.21.15.69:37608) 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83f3400640 1 -- 172.21.15.69:0/4179338467 learned_addr learned my addr 172.21.15.69:0/4179338467 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83ea000640 1 --2- 172.21.15.69:0/4179338467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e410f0 0x55c9d8e434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83f2a00640 1 --2- 172.21.15.69:0/4179338467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 0x55c9d8e40bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83f3400640 1 -- 172.21.15.69:0/4179338467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 msgr2=0x55c9d8e40bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83f3400640 1 --2- 172.21.15.69:0/4179338467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 0x55c9d8e40bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83f3400640 1 -- 172.21.15.69:0/4179338467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e410f0 msgr2=0x55c9d8e434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83f3400640 1 --2- 172.21.15.69:0/4179338467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e410f0 0x55c9d8e434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83f3400640 1 -- 172.21.15.69:0/4179338467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d8ab03f0 con 0x55c9d8e37bf0 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.724+0000 7f83f2a00640 1 --2- 172.21.15.69:0/4179338467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 0x55c9d8e40bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.725+0000 7f83f3400640 1 --2- 172.21.15.69:0/4179338467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e37fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00db50 tx=0x7f83ec00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fa0f95a84241497 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.725+0000 7f83e3e00640 1 -- 172.21.15.69:0/4179338467 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00de40 con 0x55c9d8e37bf0 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.725+0000 7f83e3e00640 1 -- 172.21.15.69:0/4179338467 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ec0035f0 con 0x55c9d8e37bf0 2024-03-19T01:12:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.725+0000 7f83e3e00640 1 -- 172.21.15.69:0/4179338467 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0116a0 con 0x55c9d8e37bf0 2024-03-19T01:12:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83fcbc4740 1 -- 172.21.15.69:0/4179338467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 msgr2=0x55c9d8e37fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/4179338467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e37fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00db50 tx=0x7f83ec00f730 comp rx=0 tx=0).stop 2024-03-19T01:12:08.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83fcbc4740 1 -- 172.21.15.69:0/4179338467 shutdown_connections 2024-03-19T01:12:08.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/4179338467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e410f0 0x55c9d8e434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/4179338467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 0x55c9d8e40bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/4179338467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e37fd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83fcbc4740 1 -- 172.21.15.69:0/4179338467 >> 172.21.15.69:0/4179338467 conn(0x55c9d8e32320 msgr2=0x55c9d8e32700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83fcbc4740 1 -- 172.21.15.69:0/4179338467 shutdown_connections 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83fcbc4740 1 -- 172.21.15.69:0/4179338467 wait complete. 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83f8a00640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f83d80a29a0 0x7f83d80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83fcbc4740 1 -- start start 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.726+0000 7f83f8a00640 1 -- 172.21.15.69:0/2187691657 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f83e40cfc20 con 0x7f83f4073bf0 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.727+0000 7f83f3400640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f83d80a29a0 0x7f83d80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.727+0000 7f83fcbc4740 1 Processor -- start 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.727+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e3e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e410f0 0x55c9d8e3ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8e43ee0 con 0x55c9d8e37bf0 2024-03-19T01:12:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8d58160 con 0x55c9d8e38510 2024-03-19T01:12:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8e44060 con 0x55c9d8e410f0 2024-03-19T01:12:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83fcbc4740 10 client.0 _ll_register_callbacks cb 0x55c9d8d94170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e3e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e3e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37622/0 (socket says 172.21.15.69:37622) 2024-03-19T01:12:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83ea000640 1 -- 172.21.15.69:0/1401910140 learned_addr learned my addr 172.21.15.69:0/1401910140 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83f3400640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f83d80a29a0 0x7f83d80a4e60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00db50 tx=0x7f83ec005fd0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83f2a00640 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83ea000640 1 -- 172.21.15.69:0/1401910140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e410f0 msgr2=0x55c9d8e3ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83ea000640 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e410f0 0x55c9d8e3ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83ea000640 1 -- 172.21.15.69:0/1401910140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 msgr2=0x55c9d8e3e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83ea000640 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83ea000640 1 -- 172.21.15.69:0/1401910140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d8b7a7d0 con 0x55c9d8e38510 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.728+0000 7f83f2a00640 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.729+0000 7f83ea000640 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e3e6e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f83dc02c200 tx=0x7f83dc009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c41849be9cc52633 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.729+0000 7f83e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.729+0000 7f83e0c00640 1 -- 172.21.15.69:0/1401910140 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc004250 con 0x55c9d8e38510 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.729+0000 7f83e0c00640 1 -- 172.21.15.69:0/1401910140 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f83dc0043f0 con 0x55c9d8e38510 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.729+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1401910140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83ec0035f0 con 0x55c9d8e38510 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.729+0000 7f83e0c00640 1 -- 172.21.15.69:0/1401910140 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83dc0046c0 con 0x55c9d8e38510 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.729+0000 7f83fcbc4740 10 client.26054 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.729+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1401910140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c9d8e50140 con 0x55c9d8e38510 2024-03-19T01:12:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.730+0000 7f83e0c00640 1 -- 172.21.15.69:0/1401910140 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ec0035f0 con 0x55c9d8e38510 2024-03-19T01:12:08.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.730+0000 7f83e0c00640 1 -- 172.21.15.69:0/1401910140 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f83dc0035f0 con 0x55c9d8e38510 2024-03-19T01:12:08.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.730+0000 7f83fcbc4740 10 client.26054 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:08.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.730+0000 7f83fcbc4740 10 client.26054 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:12:08.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.730+0000 7f83fcbc4740 20 client.26054 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:08.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.730+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c9d8e51600 0x55c9d8e53aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.730+0000 7f83fcbc4740 4 client.26054 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:08.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.730+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1401910140 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9d8da98c0 con 0x55c9d8e51600 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.730+0000 7f83f2a00640 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c9d8e51600 0x55c9d8e53aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.731+0000 7f83f2a00640 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c9d8e51600 0x55c9d8e53aa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d2a36e7004c4a4ed server_cookie=f9962367f9884636 in_seq=0 out_seq=0 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.731+0000 7f83e0c00640 10 client.26054 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83e0c00640 1 -- 172.21.15.69:0/1401910140 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c9d8da98c0 con 0x55c9d8e51600 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83e0c00640 10 client.26054 handle_command_reply: tid=0 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 1 client.26054 shutdown 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1401910140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 msgr2=0x55c9d8e3e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e3e6e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f83dc02c200 tx=0x7f83dc009620 comp rx=0 tx=0).stop 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1401910140 shutdown_connections 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e410f0 0x55c9d8e3ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c9d8e51600 0x55c9d8e53aa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e3e6e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/1401910140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83e0c00640 0 client.26054 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:12:08.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1401910140 >> 172.21.15.69:0/1401910140 conn(0x55c9d8e32320 msgr2=0x55c9d8e34070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1401910140 shutdown_connections 2024-03-19T01:12:08.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1401910140 wait complete. 2024-03-19T01:12:08.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.732+0000 7f83fcbc4740 20 client.26054 trim_cache size 0 max 16384 2024-03-19T01:12:08.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.749+0000 7f83fcbc4740 1 Processor -- start 2024-03-19T01:12:08.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.749+0000 7f83fcbc4740 1 -- start start 2024-03-19T01:12:08.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.749+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.749+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e4cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.749+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e410f0 0x55c9d8e3e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.749+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83f41b0da0 con 0x55c9d8e37bf0 2024-03-19T01:12:08.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.749+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83f41b2970 con 0x55c9d8e38510 2024-03-19T01:12:08.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.749+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f83f41b1470 con 0x55c9d8e410f0 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e4cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83f2a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e4cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37636/0 (socket says 172.21.15.69:37636) 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83ea000640 1 -- 172.21.15.69:0/1003156008 learned_addr learned my addr 172.21.15.69:0/1003156008 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83f3400640 1 --2- 172.21.15.69:0/1003156008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e410f0 0x55c9d8e3e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83ea000640 1 -- 172.21.15.69:0/1003156008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e410f0 msgr2=0x55c9d8e3e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83ea000640 1 --2- 172.21.15.69:0/1003156008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e410f0 0x55c9d8e3e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83ea000640 1 -- 172.21.15.69:0/1003156008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 msgr2=0x55c9d8e3dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83ea000640 1 --2- 172.21.15.69:0/1003156008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83ea000640 1 -- 172.21.15.69:0/1003156008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d8e407c0 con 0x55c9d8e38510 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83f2a00640 1 --2- 172.21.15.69:0/1003156008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:08.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83ea000640 1 --2- 172.21.15.69:0/1003156008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e4cc80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f83dc003990 tx=0x7f83dc02bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a94eec2953abf9b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.750+0000 7f83e3400640 1 -- 172.21.15.69:0/1003156008 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc003c00 con 0x55c9d8e38510 2024-03-19T01:12:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.751+0000 7f83e3400640 1 -- 172.21.15.69:0/1003156008 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83dc003da0 con 0x55c9d8e38510 2024-03-19T01:12:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.751+0000 7f83e3400640 1 -- 172.21.15.69:0/1003156008 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0054a0 con 0x55c9d8e38510 2024-03-19T01:12:08.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.751+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1003156008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 msgr2=0x55c9d8e4cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.751+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/1003156008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e4cc80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f83dc003990 tx=0x7f83dc02bcc0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.751+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1003156008 shutdown_connections 2024-03-19T01:12:08.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.751+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/1003156008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e410f0 0x55c9d8e3e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.751+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/1003156008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e38510 0x55c9d8e4cc80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.751+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/1003156008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.751+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1003156008 >> 172.21.15.69:0/1003156008 conn(0x55c9d8e32320 msgr2=0x55c9d8e3e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.751+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1003156008 shutdown_connections 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.752+0000 7f83fcbc4740 1 -- 172.21.15.69:0/1003156008 wait complete. 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.752+0000 7f83fcbc4740 1 -- start start 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.752+0000 7f83fcbc4740 1 Processor -- start 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.752+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 0x55c9d8e3b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83fcbc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e410f0 0x55c9d8e3b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83f41b2210 con 0x55c9d8e410f0 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83f41b1b40 con 0x55c9d8e37bf0 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83fcbc4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f83f41aed80 con 0x55c9d8e38510 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83fcbc4740 10 client.0 _ll_register_callbacks cb 0x7f83dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83f2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83ea000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 0x55c9d8e3b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83f2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37648/0 (socket says 172.21.15.69:37648) 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83f2a00640 1 -- 172.21.15.69:0/3172430818 learned_addr learned my addr 172.21.15.69:0/3172430818 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83ea000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 0x55c9d8e3b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58834/0 (socket says 172.21.15.69:58834) 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83f3400640 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e410f0 0x55c9d8e3b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83f2a00640 1 -- 172.21.15.69:0/3172430818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 msgr2=0x55c9d8e3b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83f2a00640 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 0x55c9d8e3b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83f2a00640 1 -- 172.21.15.69:0/3172430818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e410f0 msgr2=0x55c9d8e3b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83f2a00640 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e410f0 0x55c9d8e3b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83f2a00640 1 -- 172.21.15.69:0/3172430818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d8eb69a0 con 0x55c9d8e37bf0 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.753+0000 7f83ea000640 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 0x55c9d8e3b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.754+0000 7f83f2a00640 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3ad60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f83e4098070 tx=0x7f83e400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68b16c1f8d9f133d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.754+0000 7f83e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.754+0000 7f83e0c00640 1 -- 172.21.15.69:0/3172430818 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e409bc10 con 0x55c9d8e37bf0 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.754+0000 7f83e0c00640 1 -- 172.21.15.69:0/3172430818 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f83e4099f10 con 0x55c9d8e37bf0 2024-03-19T01:12:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.754+0000 7f83fcbc4740 1 -- 172.21.15.69:0/3172430818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c9d8da3c10 con 0x55c9d8e37bf0 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.754+0000 7f83e0c00640 1 -- 172.21.15.69:0/3172430818 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83e409fd60 con 0x55c9d8e37bf0 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.754+0000 7f83fcbc4740 10 client.26060 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.754+0000 7f83fcbc4740 1 -- 172.21.15.69:0/3172430818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c9d8eb7cc0 con 0x55c9d8e37bf0 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.754+0000 7f83e0c00640 1 -- 172.21.15.69:0/3172430818 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c9d8da3c10 con 0x55c9d8e37bf0 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.755+0000 7f83e0c00640 1 -- 172.21.15.69:0/3172430818 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f83e40cd0b0 con 0x55c9d8e37bf0 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.755+0000 7f83fcbc4740 10 client.26060 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.755+0000 7f83fcbc4740 10 client.26060 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.755+0000 7f83fcbc4740 20 client.26060 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.755+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c9d8eb82f0 0x55c9d8eba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.755+0000 7f83fcbc4740 4 client.26060 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.755+0000 7f83fcbc4740 1 -- 172.21.15.69:0/3172430818 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9d8d21560 con 0x55c9d8eb82f0 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.755+0000 7f83ea000640 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c9d8eb82f0 0x55c9d8eba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83e0c00640 10 client.26060 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:12:08.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83ea000640 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c9d8eb82f0 0x55c9d8eba6e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4e729e15543ea8f server_cookie=4feeda11d4a397cb in_seq=0 out_seq=0 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83e0c00640 1 -- 172.21.15.69:0/3172430818 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9d8d21560 con 0x55c9d8eb82f0 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83e0c00640 10 client.26060 handle_command_reply: tid=0 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83fcbc4740 1 client.26060 shutdown 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83fcbc4740 1 -- 172.21.15.69:0/3172430818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 msgr2=0x55c9d8e3ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3ad60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f83e4098070 tx=0x7f83e400cf90 comp rx=0 tx=0).stop 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83fcbc4740 1 -- 172.21.15.69:0/3172430818 shutdown_connections 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c9d8e410f0 0x55c9d8e3b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9d8e38510 0x55c9d8e3b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c9d8eb82f0 0x55c9d8eba6e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83fcbc4740 1 --2- 172.21.15.69:0/3172430818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9d8e37bf0 0x55c9d8e3ad60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83e0c00640 0 client.26060 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:12:08.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83fcbc4740 1 -- 172.21.15.69:0/3172430818 >> 172.21.15.69:0/3172430818 conn(0x55c9d8e32320 msgr2=0x55c9d8e31330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83fcbc4740 1 -- 172.21.15.69:0/3172430818 shutdown_connections 2024-03-19T01:12:08.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.756+0000 7f83fcbc4740 1 -- 172.21.15.69:0/3172430818 wait complete. 2024-03-19T01:12:08.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.757+0000 7f83fcbc4740 20 client.26060 trim_cache size 0 max 16384 2024-03-19T01:12:08.762 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:12:08.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.757+0000 7f83f3e00640 1 -- 172.21.15.69:0/2187691657 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f83d80a29a0 msgr2=0x7f83d80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.757+0000 7f83f3e00640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f83d80a29a0 0x7f83d80a4e60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00db50 tx=0x7f83ec005fd0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.757+0000 7f83f3e00640 1 -- 172.21.15.69:0/2187691657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83f4073bf0 msgr2=0x7f83f419a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.757+0000 7f83f3e00640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83f4073bf0 0x7f83f419a280 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f83e400eb90 tx=0x7f83e400c6a0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.757+0000 7f83f3e00640 1 -- 172.21.15.69:0/2187691657 shutdown_connections 2024-03-19T01:12:08.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.757+0000 7f83f3e00640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f410df30 0x7f83f419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.757+0000 7f83f3e00640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83f4074510 0x7f83f419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.757+0000 7f83f3e00640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f83d80a29a0 0x7f83d80a4e60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.757+0000 7f83f3e00640 1 --2- 172.21.15.69:0/2187691657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83f4073bf0 0x7f83f419a280 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.757+0000 7f83f3e00640 1 -- 172.21.15.69:0/2187691657 >> 172.21.15.69:0/2187691657 conn(0x7f83f40fd1c0 msgr2=0x7f83f40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.758+0000 7f83f3e00640 1 -- 172.21.15.69:0/2187691657 shutdown_connections 2024-03-19T01:12:08.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.758+0000 7f83f3e00640 1 -- 172.21.15.69:0/2187691657 wait complete. 2024-03-19T01:12:08.778 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:12:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.975+0000 7f3291400640 1 -- 172.21.15.69:0/782538920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100de0 msgr2=0x7f328c10daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.975+0000 7f3291400640 1 --2- 172.21.15.69:0/782538920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100de0 0x7f328c10daa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3278009900 tx=0x7f327802bc40 comp rx=0 tx=0).stop 2024-03-19T01:12:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.975+0000 7f3291400640 1 -- 172.21.15.69:0/782538920 shutdown_connections 2024-03-19T01:12:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.975+0000 7f3291400640 1 --2- 172.21.15.69:0/782538920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328c10dfe0 0x7f328c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.975+0000 7f3291400640 1 --2- 172.21.15.69:0/782538920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100de0 0x7f328c10daa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.975+0000 7f3291400640 1 --2- 172.21.15.69:0/782538920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328c100430 0x7f328c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.975+0000 7f3291400640 1 -- 172.21.15.69:0/782538920 >> 172.21.15.69:0/782538920 conn(0x7f328c0fc070 msgr2=0x7f328c0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.975+0000 7f3291400640 1 -- 172.21.15.69:0/782538920 shutdown_connections 2024-03-19T01:12:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.975+0000 7f3291400640 1 -- 172.21.15.69:0/782538920 wait complete. 2024-03-19T01:12:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.976+0000 7f3291400640 1 Processor -- start 2024-03-19T01:12:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.976+0000 7f3291400640 1 -- start start 2024-03-19T01:12:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.976+0000 7f3291400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328c100430 0x7f328c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.976+0000 7f3291400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100de0 0x7f328c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.976+0000 7f3291400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328c10dfe0 0x7f328c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.976+0000 7f3291400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f328c113ce0 con 0x7f328c100430 2024-03-19T01:12:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f3291400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f328c113b60 con 0x7f328c10dfe0 2024-03-19T01:12:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f3291400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f328c113e60 con 0x7f328c100de0 2024-03-19T01:12:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f3290a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328c10dfe0 0x7f328c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f328be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328c100430 0x7f328c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f3290a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328c10dfe0 0x7f328c19ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37656/0 (socket says 172.21.15.69:37656) 2024-03-19T01:12:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f3290a00640 1 -- 172.21.15.69:0/2192898653 learned_addr learned my addr 172.21.15.69:0/2192898653 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f328b400640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100de0 0x7f328c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f3290a00640 1 -- 172.21.15.69:0/2192898653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100de0 msgr2=0x7f328c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f3290a00640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100de0 0x7f328c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f3290a00640 1 -- 172.21.15.69:0/2192898653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328c100430 msgr2=0x7f328c19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f3290a00640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328c100430 0x7f328c19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.977+0000 7f3290a00640 1 -- 172.21.15.69:0/2192898653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328c19f170 con 0x7f328c10dfe0 2024-03-19T01:12:08.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.978+0000 7f3290a00640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328c10dfe0 0x7f328c19ea40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f328400eb40 tx=0x7f328400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.978+0000 7f3288c00640 1 -- 172.21.15.69:0/2192898653 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284004b20 con 0x7f328c10dfe0 2024-03-19T01:12:08.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.978+0000 7f3288c00640 1 -- 172.21.15.69:0/2192898653 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3284004cc0 con 0x7f328c10dfe0 2024-03-19T01:12:08.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.978+0000 7f3291400640 1 -- 172.21.15.69:0/2192898653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328c19f460 con 0x7f328c10dfe0 2024-03-19T01:12:08.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.978+0000 7f3288c00640 1 -- 172.21.15.69:0/2192898653 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284010850 con 0x7f328c10dfe0 2024-03-19T01:12:08.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.978+0000 7f3291400640 1 -- 172.21.15.69:0/2192898653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328c1a6cf0 con 0x7f328c10dfe0 2024-03-19T01:12:08.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.981+0000 7f3288c00640 1 -- 172.21.15.69:0/2192898653 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7f3284005020 con 0x7f328c10dfe0 2024-03-19T01:12:08.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.982+0000 7f329561e740 1 Processor -- start 2024-03-19T01:12:08.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.982+0000 7f329561e740 1 -- start start 2024-03-19T01:12:08.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.982+0000 7f329561e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfb9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.982+0000 7f329561e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfba510 0x55f66cfc2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.982+0000 7f329561e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfc54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.982+0000 7f329561e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f66cf1e380 con 0x55f66cfc30f0 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.982+0000 7f329561e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f66cec6d20 con 0x55f66cfb9bf0 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.982+0000 7f329561e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f66cf1a420 con 0x55f66cfba510 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.982+0000 7f328b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfb9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f328b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfb9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37660/0 (socket says 172.21.15.69:37660) 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f328be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfba510 0x55f66cfc2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f328b400640 1 -- 172.21.15.69:0/926617953 learned_addr learned my addr 172.21.15.69:0/926617953 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f328b400640 1 --2- 172.21.15.69:0/926617953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfc54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f328be00640 1 -- 172.21.15.69:0/926617953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 msgr2=0x55f66cfb9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f328be00640 1 --2- 172.21.15.69:0/926617953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfb9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f328be00640 1 -- 172.21.15.69:0/926617953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 msgr2=0x55f66cfc54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f328be00640 1 --2- 172.21.15.69:0/926617953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfc54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f328be00640 1 -- 172.21.15.69:0/926617953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66cc323f0 con 0x55f66cfba510 2024-03-19T01:12:08.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f328b400640 1 --2- 172.21.15.69:0/926617953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfb9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:08.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f328be00640 1 --2- 172.21.15.69:0/926617953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfba510 0x55f66cfc2bb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f327c009c20 tx=0x7f327c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f39e8c0490d0ee server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.983+0000 7f3288c00640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f32700a2a90 0x7f32700a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.984+0000 7f3281200640 1 -- 172.21.15.69:0/926617953 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c004050 con 0x55f66cfba510 2024-03-19T01:12:08.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.984+0000 7f3281200640 1 -- 172.21.15.69:0/926617953 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f327c005500 con 0x55f66cfba510 2024-03-19T01:12:08.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.984+0000 7f3281200640 1 -- 172.21.15.69:0/926617953 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c004dd0 con 0x55f66cfba510 2024-03-19T01:12:08.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.984+0000 7f3288c00640 1 -- 172.21.15.69:0/2192898653 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f32840cfc50 con 0x7f328c10dfe0 2024-03-19T01:12:08.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.984+0000 7f328b400640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f32700a2a90 0x7f32700a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.984+0000 7f329561e740 1 -- 172.21.15.69:0/926617953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfba510 msgr2=0x55f66cfc2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.984+0000 7f329561e740 1 --2- 172.21.15.69:0/926617953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfba510 0x55f66cfc2bb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f327c009c20 tx=0x7f327c011810 comp rx=0 tx=0).stop 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.984+0000 7f329561e740 1 -- 172.21.15.69:0/926617953 shutdown_connections 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.984+0000 7f329561e740 1 --2- 172.21.15.69:0/926617953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfc54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.984+0000 7f329561e740 1 --2- 172.21.15.69:0/926617953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfba510 0x55f66cfc2bb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.984+0000 7f329561e740 1 --2- 172.21.15.69:0/926617953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfb9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.985+0000 7f329561e740 1 -- 172.21.15.69:0/926617953 >> 172.21.15.69:0/926617953 conn(0x55f66cfb4320 msgr2=0x55f66cfb4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.985+0000 7f328b400640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f32700a2a90 0x7f32700a4f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55f66cfb54b0 tx=0x7f327802bc40 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.985+0000 7f329561e740 1 -- 172.21.15.69:0/926617953 shutdown_connections 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.985+0000 7f329561e740 1 -- 172.21.15.69:0/926617953 wait complete. 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.985+0000 7f329561e740 1 -- start start 2024-03-19T01:12:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.985+0000 7f329561e740 1 Processor -- start 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f329561e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f329561e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfba510 0x55f66cfc0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f329561e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfc30f0 0x55f66cfc0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f329561e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f66cfc5ee0 con 0x55f66cfba510 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f329561e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f66ceda160 con 0x55f66cfb9bf0 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f329561e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f66cfc6060 con 0x55f66cfc30f0 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f329561e740 10 client.0 _ll_register_callbacks cb 0x55f66cf16170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f328be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f328be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37662/0 (socket says 172.21.15.69:37662) 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f328be00640 1 -- 172.21.15.69:0/751173999 learned_addr learned my addr 172.21.15.69:0/751173999 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f328b400640 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfba510 0x55f66cfc0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.986+0000 7f3290a00640 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfc30f0 0x55f66cfc0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f328be00640 1 -- 172.21.15.69:0/751173999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfc30f0 msgr2=0x55f66cfc0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f328be00640 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfc30f0 0x55f66cfc0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f328be00640 1 -- 172.21.15.69:0/751173999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfba510 msgr2=0x55f66cfc0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:08.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f328be00640 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfba510 0x55f66cfc0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f328be00640 1 -- 172.21.15.69:0/751173999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66ccfc7d0 con 0x55f66cfb9bf0 2024-03-19T01:12:08.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f328b400640 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfba510 0x55f66cfc0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:08.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f3290a00640 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfc30f0 0x55f66cfc0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:08.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f328be00640 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc0100 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f327c005720 tx=0x7f327c005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1711694c941db56 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:08.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f3275600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:08.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f3275600640 1 -- 172.21.15.69:0/751173999 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c029070 con 0x55f66cfb9bf0 2024-03-19T01:12:08.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f3275600640 1 -- 172.21.15.69:0/751173999 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f327c004d50 con 0x55f66cfb9bf0 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.987+0000 7f329561e740 1 -- 172.21.15.69:0/751173999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327c005500 con 0x55f66cfb9bf0 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.988+0000 7f3275600640 1 -- 172.21.15.69:0/751173999 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f327c035050 con 0x55f66cfb9bf0 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.988+0000 7f329561e740 10 client.26072 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.988+0000 7f329561e740 1 -- 172.21.15.69:0/751173999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f66cfd2160 con 0x55f66cfb9bf0 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.988+0000 7f3275600640 1 -- 172.21.15.69:0/751173999 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f327c005500 con 0x55f66cfb9bf0 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.988+0000 7f3275600640 1 -- 172.21.15.69:0/751173999 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f327c023430 con 0x55f66cfb9bf0 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.988+0000 7f329561e740 10 client.26072 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.988+0000 7f329561e740 10 client.26072 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.988+0000 7f329561e740 20 client.26072 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.988+0000 7f329561e740 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f66cfd34b0 0x55f66cfd58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.988+0000 7f329561e740 4 client.26072 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.988+0000 7f329561e740 1 -- 172.21.15.69:0/751173999 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f66cf2b8c0 con 0x55f66cfd34b0 2024-03-19T01:12:08.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.989+0000 7f328b400640 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f66cfd34b0 0x55f66cfd58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:08.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.989+0000 7f3275600640 10 client.26072 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:12:08.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.989+0000 7f328b400640 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f66cfd34b0 0x55f66cfd58a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9daaf0820cabdca server_cookie=c717943a9dae66d in_seq=0 out_seq=0 2024-03-19T01:12:08.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f3275600640 1 -- 172.21.15.69:0/751173999 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f66cf2b8c0 con 0x55f66cfd34b0 2024-03-19T01:12:08.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f3275600640 10 client.26072 handle_command_reply: tid=0 2024-03-19T01:12:08.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 1 client.26072 shutdown 2024-03-19T01:12:08.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 1 -- 172.21.15.69:0/751173999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 msgr2=0x55f66cfc0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:08.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc0100 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f327c005720 tx=0x7f327c005750 comp rx=0 tx=0).stop 2024-03-19T01:12:08.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 1 -- 172.21.15.69:0/751173999 shutdown_connections 2024-03-19T01:12:08.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f66cfd34b0 0x55f66cfd58a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfc30f0 0x55f66cfc0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfba510 0x55f66cfc0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:08.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f3275600640 0 client.26072 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:12:08.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 1 --2- 172.21.15.69:0/751173999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc0100 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 1 -- 172.21.15.69:0/751173999 >> 172.21.15.69:0/751173999 conn(0x55f66cfb4320 msgr2=0x55f66cfb6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 1 -- 172.21.15.69:0/751173999 shutdown_connections 2024-03-19T01:12:09.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 1 -- 172.21.15.69:0/751173999 wait complete. 2024-03-19T01:12:09.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:08.991+0000 7f329561e740 20 client.26072 trim_cache size 0 max 16384 2024-03-19T01:12:09.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.007+0000 7f329561e740 1 Processor -- start 2024-03-19T01:12:09.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.007+0000 7f329561e740 1 -- start start 2024-03-19T01:12:09.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.007+0000 7f329561e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.007+0000 7f329561e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfbf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.007+0000 7f329561e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfcec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.007+0000 7f329561e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1b0dc0 con 0x55f66cfc30f0 2024-03-19T01:12:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.007+0000 7f329561e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1b2990 con 0x55f66cfba510 2024-03-19T01:12:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.007+0000 7f329561e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1b1490 con 0x55f66cfb9bf0 2024-03-19T01:12:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f328b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfbf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f328be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f3290a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfcec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f328b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfbf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37674/0 (socket says 172.21.15.69:37674) 2024-03-19T01:12:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f328b400640 1 -- 172.21.15.69:0/2062555091 learned_addr learned my addr 172.21.15.69:0/2062555091 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f328be00640 1 -- 172.21.15.69:0/2062555091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 msgr2=0x55f66cfbf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f328be00640 1 --2- 172.21.15.69:0/2062555091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfbf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f328be00640 1 -- 172.21.15.69:0/2062555091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 msgr2=0x55f66cfcec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f328be00640 1 --2- 172.21.15.69:0/2062555091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfcec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f328be00640 1 -- 172.21.15.69:0/2062555091 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66cfc5810 con 0x55f66cfb9bf0 2024-03-19T01:12:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f328b400640 1 --2- 172.21.15.69:0/2062555091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfbf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f328be00640 1 --2- 172.21.15.69:0/2062555091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc2830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f327c009b40 tx=0x7f327c005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=698c77ca63e8f48c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.008+0000 7f3277e00640 1 -- 172.21.15.69:0/2062555091 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c029070 con 0x55f66cfb9bf0 2024-03-19T01:12:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.009+0000 7f3277e00640 1 -- 172.21.15.69:0/2062555091 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f327c004480 con 0x55f66cfb9bf0 2024-03-19T01:12:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.009+0000 7f3277e00640 1 -- 172.21.15.69:0/2062555091 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c02e440 con 0x55f66cfb9bf0 2024-03-19T01:12:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.009+0000 7f329561e740 1 -- 172.21.15.69:0/2062555091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfb9bf0 msgr2=0x55f66cfc2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.009+0000 7f329561e740 1 --2- 172.21.15.69:0/2062555091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc2830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f327c009b40 tx=0x7f327c005970 comp rx=0 tx=0).stop 2024-03-19T01:12:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.009+0000 7f329561e740 1 -- 172.21.15.69:0/2062555091 shutdown_connections 2024-03-19T01:12:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.009+0000 7f329561e740 1 --2- 172.21.15.69:0/2062555091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfcec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.009+0000 7f329561e740 1 --2- 172.21.15.69:0/2062555091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfbf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.009+0000 7f329561e740 1 --2- 172.21.15.69:0/2062555091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc2830 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.009+0000 7f329561e740 1 -- 172.21.15.69:0/2062555091 >> 172.21.15.69:0/2062555091 conn(0x55f66cfb4320 msgr2=0x55f66cfb3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.009+0000 7f329561e740 1 -- 172.21.15.69:0/2062555091 shutdown_connections 2024-03-19T01:12:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.009+0000 7f329561e740 1 -- 172.21.15.69:0/2062555091 wait complete. 2024-03-19T01:12:09.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.010+0000 7f329561e740 1 -- start start 2024-03-19T01:12:09.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.010+0000 7f329561e740 1 Processor -- start 2024-03-19T01:12:09.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f329561e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f329561e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfc67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f329561e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfc6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f329561e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1b2230 con 0x55f66cfc30f0 2024-03-19T01:12:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f329561e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1b1b60 con 0x55f66cfba510 2024-03-19T01:12:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f329561e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1aeda0 con 0x55f66cfb9bf0 2024-03-19T01:12:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f329561e740 10 client.0 _ll_register_callbacks cb 0x7f327c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f3290a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfc6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f328b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfc67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f328be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f328b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfc67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37676/0 (socket says 172.21.15.69:37676) 2024-03-19T01:12:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f328b400640 1 -- 172.21.15.69:0/3320931306 learned_addr learned my addr 172.21.15.69:0/3320931306 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f3290a00640 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfc6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44382/0 (socket says 172.21.15.69:44382) 2024-03-19T01:12:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f3290a00640 1 -- 172.21.15.69:0/3320931306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfb9bf0 msgr2=0x55f66cfc6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f3290a00640 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f3290a00640 1 -- 172.21.15.69:0/3320931306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 msgr2=0x55f66cfc67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f3290a00640 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfc67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f3290a00640 1 -- 172.21.15.69:0/3320931306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66d0378c0 con 0x55f66cfc30f0 2024-03-19T01:12:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.011+0000 7f328b400640 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfc67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.012+0000 7f3290a00640 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfc6cf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f32840cce00 tx=0x7f328409b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220d4ee427382022 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.012+0000 7f3275600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:12:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.012+0000 7f3275600640 1 -- 172.21.15.69:0/3320931306 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32840b8020 con 0x55f66cfc30f0 2024-03-19T01:12:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.012+0000 7f3275600640 1 -- 172.21.15.69:0/3320931306 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f32840ba4c0 con 0x55f66cfc30f0 2024-03-19T01:12:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.012+0000 7f329561e740 1 -- 172.21.15.69:0/3320931306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327c004480 con 0x55f66cfc30f0 2024-03-19T01:12:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.012+0000 7f3275600640 1 -- 172.21.15.69:0/3320931306 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f328409c340 con 0x55f66cfc30f0 2024-03-19T01:12:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.012+0000 7f329561e740 10 client.15891 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.012+0000 7f329561e740 1 -- 172.21.15.69:0/3320931306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f66d038b40 con 0x55f66cfc30f0 2024-03-19T01:12:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.013+0000 7f3275600640 1 -- 172.21.15.69:0/3320931306 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f327c004480 con 0x55f66cfc30f0 2024-03-19T01:12:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.013+0000 7f3275600640 1 -- 172.21.15.69:0/3320931306 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f328409b090 con 0x55f66cfc30f0 2024-03-19T01:12:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.013+0000 7f329561e740 10 client.15891 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.013+0000 7f329561e740 10 client.15891 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.013+0000 7f329561e740 20 client.15891 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.013+0000 7f329561e740 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f66d039270 0x55f66d03b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.013+0000 7f329561e740 4 client.15891 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.013+0000 7f329561e740 1 -- 172.21.15.69:0/3320931306 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f66cf25c10 con 0x55f66d039270 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.013+0000 7f328be00640 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f66d039270 0x55f66d03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f328be00640 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f66d039270 0x55f66d03b690 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e1b138e8ed634df server_cookie=2ac709f837a7100 in_seq=0 out_seq=0 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f3275600640 10 client.15891 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f3275600640 1 -- 172.21.15.69:0/3320931306 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f66cf25c10 con 0x55f66d039270 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f3275600640 10 client.15891 handle_command_reply: tid=0 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f329561e740 1 client.15891 shutdown 2024-03-19T01:12:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f329561e740 1 -- 172.21.15.69:0/3320931306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 msgr2=0x55f66cfc6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f329561e740 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfc6cf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f32840cce00 tx=0x7f328409b660 comp rx=0 tx=0).stop 2024-03-19T01:12:09.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f329561e740 1 -- 172.21.15.69:0/3320931306 shutdown_connections 2024-03-19T01:12:09.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f329561e740 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f66d039270 0x55f66d03b690 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f329561e740 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f66cfc30f0 0x55f66cfc6cf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f329561e740 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66cfba510 0x55f66cfc67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f329561e740 1 --2- 172.21.15.69:0/3320931306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f66cfb9bf0 0x55f66cfc6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f329561e740 1 -- 172.21.15.69:0/3320931306 >> 172.21.15.69:0/3320931306 conn(0x55f66cfb4320 msgr2=0x55f66cfb2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.014+0000 7f3275600640 0 client.15891 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:12:09.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.015+0000 7f329561e740 1 -- 172.21.15.69:0/3320931306 shutdown_connections 2024-03-19T01:12:09.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.015+0000 7f329561e740 1 -- 172.21.15.69:0/3320931306 wait complete. 2024-03-19T01:12:09.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.015+0000 7f329561e740 20 client.15891 trim_cache size 0 max 16384 2024-03-19T01:12:09.024 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:12:09.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 -- 172.21.15.69:0/2192898653 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f32700a2a90 msgr2=0x7f32700a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f32700a2a90 0x7f32700a4f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55f66cfb54b0 tx=0x7f327802bc40 comp rx=0 tx=0).stop 2024-03-19T01:12:09.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 -- 172.21.15.69:0/2192898653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328c10dfe0 msgr2=0x7f328c19ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328c10dfe0 0x7f328c19ea40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f328400eb40 tx=0x7f328400c2d0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 -- 172.21.15.69:0/2192898653 shutdown_connections 2024-03-19T01:12:09.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f32700a2a90 0x7f32700a4f50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328c10dfe0 0x7f328c19ea40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c100de0 0x7f328c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 --2- 172.21.15.69:0/2192898653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328c100430 0x7f328c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 -- 172.21.15.69:0/2192898653 >> 172.21.15.69:0/2192898653 conn(0x7f328c0fc070 msgr2=0x7f328c0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 -- 172.21.15.69:0/2192898653 shutdown_connections 2024-03-19T01:12:09.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.016+0000 7f3291400640 1 -- 172.21.15.69:0/2192898653 wait complete. 2024-03-19T01:12:09.036 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:12:09.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.241+0000 7fa65f200640 1 -- 172.21.15.69:0/2013843314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa660073bf0 msgr2=0x7fa660073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.241+0000 7fa65f200640 1 --2- 172.21.15.69:0/2013843314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa660073bf0 0x7fa660073fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa650009ca0 tx=0x7fa65002c180 comp rx=0 tx=0).stop 2024-03-19T01:12:09.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.241+0000 7fa65f200640 1 -- 172.21.15.69:0/2013843314 shutdown_connections 2024-03-19T01:12:09.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.241+0000 7fa65f200640 1 --2- 172.21.15.69:0/2013843314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa66010df30 0x7fa660110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.241+0000 7fa65f200640 1 --2- 172.21.15.69:0/2013843314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa660074510 0x7fa66010d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.241+0000 7fa65f200640 1 --2- 172.21.15.69:0/2013843314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa660073bf0 0x7fa660073fd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.241+0000 7fa65f200640 1 -- 172.21.15.69:0/2013843314 >> 172.21.15.69:0/2013843314 conn(0x7fa6600fd1c0 msgr2=0x7fa6600ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.242+0000 7fa65f200640 1 -- 172.21.15.69:0/2013843314 shutdown_connections 2024-03-19T01:12:09.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.242+0000 7fa65f200640 1 -- 172.21.15.69:0/2013843314 wait complete. 2024-03-19T01:12:09.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65f200640 1 Processor -- start 2024-03-19T01:12:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65f200640 1 -- start start 2024-03-19T01:12:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa660073bf0 0x7fa66019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa660074510 0x7fa66019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65f200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa66010df30 0x7fa66019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65f200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa660113c90 con 0x7fa66010df30 2024-03-19T01:12:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa660113b10 con 0x7fa660073bf0 2024-03-19T01:12:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa660113e10 con 0x7fa660074510 2024-03-19T01:12:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa660073bf0 0x7fa66019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa660073bf0 0x7fa66019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37706/0 (socket says 172.21.15.69:37706) 2024-03-19T01:12:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65de00640 1 -- 172.21.15.69:0/2124374774 learned_addr learned my addr 172.21.15.69:0/2124374774 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65d400640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa660074510 0x7fa66019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.243+0000 7fa65e800640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa66010df30 0x7fa66019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.244+0000 7fa65de00640 1 -- 172.21.15.69:0/2124374774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa660074510 msgr2=0x7fa66019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.244+0000 7fa65de00640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa660074510 0x7fa66019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.244+0000 7fa65de00640 1 -- 172.21.15.69:0/2124374774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa66010df30 msgr2=0x7fa66019ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.244+0000 7fa65de00640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa66010df30 0x7fa66019ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.244+0000 7fa65de00640 1 -- 172.21.15.69:0/2124374774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66019f160 con 0x7fa660073bf0 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.244+0000 7fa65de00640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa660073bf0 0x7fa66019a170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa650005e30 tx=0x7fa650035d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.244+0000 7fa656a00640 1 -- 172.21.15.69:0/2124374774 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650039040 con 0x7fa660073bf0 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.245+0000 7fa65f200640 1 -- 172.21.15.69:0/2124374774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66019f3f0 con 0x7fa660073bf0 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.245+0000 7fa656a00640 1 -- 172.21.15.69:0/2124374774 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa650044070 con 0x7fa660073bf0 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.245+0000 7fa656a00640 1 -- 172.21.15.69:0/2124374774 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6500328d0 con 0x7fa660073bf0 2024-03-19T01:12:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.245+0000 7fa65f200640 1 -- 172.21.15.69:0/2124374774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6601a6ce0 con 0x7fa660073bf0 2024-03-19T01:12:09.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.248+0000 7fa656a00640 1 -- 172.21.15.69:0/2124374774 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7fa650032a70 con 0x7fa660073bf0 2024-03-19T01:12:09.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.248+0000 7fa667d71740 1 Processor -- start 2024-03-19T01:12:09.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.248+0000 7fa667d71740 1 -- start start 2024-03-19T01:12:09.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.248+0000 7fa667d71740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625ebf0 0x55f65625efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.248+0000 7fa667d71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 0x55f656267bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa667d71740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6562680f0 0x55f65626a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa667d71740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f6561c3380 con 0x55f65625ebf0 2024-03-19T01:12:09.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa667d71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f65616bd20 con 0x55f65625f510 2024-03-19T01:12:09.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa667d71740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f6561bf420 con 0x55f6562680f0 2024-03-19T01:12:09.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa65e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625ebf0 0x55f65625efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa65de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 0x55f656267bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa65e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625ebf0 0x55f65625efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44424/0 (socket says 172.21.15.69:44424) 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa65e800640 1 -- 172.21.15.69:0/3107892759 learned_addr learned my addr 172.21.15.69:0/3107892759 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa65e800640 1 -- 172.21.15.69:0/3107892759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6562680f0 msgr2=0x55f65626a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa65e800640 1 --2- 172.21.15.69:0/3107892759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6562680f0 0x55f65626a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa65e800640 1 -- 172.21.15.69:0/3107892759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 msgr2=0x55f656267bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa65e800640 1 --2- 172.21.15.69:0/3107892759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 0x55f656267bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa65e800640 1 -- 172.21.15.69:0/3107892759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f655ed73f0 con 0x55f65625ebf0 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.249+0000 7fa65de00640 1 --2- 172.21.15.69:0/3107892759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 0x55f656267bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.250+0000 7fa65e800640 1 --2- 172.21.15.69:0/3107892759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625ebf0 0x55f65625efd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa658009b80 tx=0x7fa658011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d232e39ccd56715 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.250+0000 7fa64b400640 1 -- 172.21.15.69:0/3107892759 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658004030 con 0x55f65625ebf0 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.250+0000 7fa64b400640 1 -- 172.21.15.69:0/3107892759 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6580041d0 con 0x55f65625ebf0 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.250+0000 7fa64b400640 1 -- 172.21.15.69:0/3107892759 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6580196d0 con 0x55f65625ebf0 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.250+0000 7fa656a00640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6440a2a90 0x7fa6440a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.250+0000 7fa656a00640 1 -- 172.21.15.69:0/2124374774 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa6500f18c0 con 0x7fa660073bf0 2024-03-19T01:12:09.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.251+0000 7fa65de00640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6440a2a90 0x7fa6440a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.251+0000 7fa667d71740 1 -- 172.21.15.69:0/3107892759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625ebf0 msgr2=0x55f65625efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.251+0000 7fa667d71740 1 --2- 172.21.15.69:0/3107892759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625ebf0 0x55f65625efd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa658009b80 tx=0x7fa658011810 comp rx=0 tx=0).stop 2024-03-19T01:12:09.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.251+0000 7fa667d71740 1 -- 172.21.15.69:0/3107892759 shutdown_connections 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.251+0000 7fa667d71740 1 --2- 172.21.15.69:0/3107892759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6562680f0 0x55f65626a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.251+0000 7fa667d71740 1 --2- 172.21.15.69:0/3107892759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 0x55f656267bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.251+0000 7fa667d71740 1 --2- 172.21.15.69:0/3107892759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625ebf0 0x55f65625efd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.251+0000 7fa667d71740 1 -- 172.21.15.69:0/3107892759 >> 172.21.15.69:0/3107892759 conn(0x55f656259320 msgr2=0x55f656259700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.251+0000 7fa667d71740 1 -- 172.21.15.69:0/3107892759 shutdown_connections 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.251+0000 7fa65de00640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6440a2a90 0x7fa6440a4f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6500eebe0 tx=0x7fa6500f7530 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.251+0000 7fa667d71740 1 -- 172.21.15.69:0/3107892759 wait complete. 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.252+0000 7fa667d71740 1 -- start start 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.252+0000 7fa667d71740 1 Processor -- start 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.252+0000 7fa667d71740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 0x55f656265170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.252+0000 7fa667d71740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625f510 0x55f6562656b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.252+0000 7fa667d71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6562680f0 0x55f656265bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.252+0000 7fa667d71740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f65626aee0 con 0x55f65625f510 2024-03-19T01:12:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.252+0000 7fa667d71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f65617f160 con 0x55f6562680f0 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.252+0000 7fa667d71740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f65626b060 con 0x55f65625ebf0 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.252+0000 7fa667d71740 10 client.0 _ll_register_callbacks cb 0x55f6561bb170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.253+0000 7fa65e800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 0x55f656265170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.253+0000 7fa65e800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 0x55f656265170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58914/0 (socket says 172.21.15.69:58914) 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.253+0000 7fa65e800640 1 -- 172.21.15.69:0/1990660000 learned_addr learned my addr 172.21.15.69:0/1990660000 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.253+0000 7fa65d400640 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625f510 0x55f6562656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.253+0000 7fa65e800640 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6562680f0 0x55f656265bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.253+0000 7fa65e800640 1 -- 172.21.15.69:0/1990660000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6562680f0 msgr2=0x55f656265bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.253+0000 7fa65e800640 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6562680f0 0x55f656265bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.253+0000 7fa65e800640 1 -- 172.21.15.69:0/1990660000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625f510 msgr2=0x55f6562656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.253+0000 7fa65e800640 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625f510 0x55f6562656b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.253+0000 7fa65e800640 1 -- 172.21.15.69:0/1990660000 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f655fa17d0 con 0x55f65625ebf0 2024-03-19T01:12:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.254+0000 7fa65e800640 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 0x55f656265170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa658003f90 tx=0x7fa658003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2f2ac8aa8742867 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.254+0000 7fa63fe00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.254+0000 7fa63fe00640 1 -- 172.21.15.69:0/1990660000 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658005020 con 0x55f65625ebf0 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.254+0000 7fa667d71740 1 -- 172.21.15.69:0/1990660000 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6580041d0 con 0x55f65625ebf0 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.254+0000 7fa63fe00640 1 -- 172.21.15.69:0/1990660000 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa65801f070 con 0x55f65625ebf0 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.254+0000 7fa63fe00640 1 -- 172.21.15.69:0/1990660000 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6580051c0 con 0x55f65625ebf0 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.254+0000 7fa667d71740 10 client.26074 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.254+0000 7fa667d71740 1 -- 172.21.15.69:0/1990660000 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f6562771d0 con 0x55f65625ebf0 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.254+0000 7fa63fe00640 1 -- 172.21.15.69:0/1990660000 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6580041d0 con 0x55f65625ebf0 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.255+0000 7fa63fe00640 1 -- 172.21.15.69:0/1990660000 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fa658018070 con 0x55f65625ebf0 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.255+0000 7fa667d71740 10 client.26074 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.255+0000 7fa667d71740 10 client.26074 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.255+0000 7fa667d71740 20 client.26074 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.255+0000 7fa667d71740 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55f6562785e0 0x55f65627a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.255+0000 7fa667d71740 4 client.26074 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:09.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.255+0000 7fa667d71740 1 -- 172.21.15.69:0/1990660000 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6561d08c0 con 0x55f6562785e0 2024-03-19T01:12:09.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.255+0000 7fa65d400640 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55f6562785e0 0x55f65627a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.256+0000 7fa65d400640 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55f6562785e0 0x55f65627a9d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5025a0ce6c62a27 server_cookie=29dd1a893e7e74d7 in_seq=0 out_seq=0 2024-03-19T01:12:09.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.256+0000 7fa63fe00640 10 client.26074 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:12:09.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.256+0000 7fa63fe00640 1 -- 172.21.15.69:0/1990660000 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f6561d08c0 con 0x55f6562785e0 2024-03-19T01:12:09.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.256+0000 7fa63fe00640 10 client.26074 handle_command_reply: tid=0 2024-03-19T01:12:09.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 1 client.26074 shutdown 2024-03-19T01:12:09.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 1 -- 172.21.15.69:0/1990660000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 msgr2=0x55f656265170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 0x55f656265170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa658003f90 tx=0x7fa658003e90 comp rx=0 tx=0).stop 2024-03-19T01:12:09.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 1 -- 172.21.15.69:0/1990660000 shutdown_connections 2024-03-19T01:12:09.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55f6562785e0 0x55f65627a9d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6562680f0 0x55f656265bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625f510 0x55f6562656b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa63fe00640 0 client.26074 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:12:09.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 1 --2- 172.21.15.69:0/1990660000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 0x55f656265170 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 1 -- 172.21.15.69:0/1990660000 >> 172.21.15.69:0/1990660000 conn(0x55f656259320 msgr2=0x55f65625b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 1 -- 172.21.15.69:0/1990660000 shutdown_connections 2024-03-19T01:12:09.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 1 -- 172.21.15.69:0/1990660000 wait complete. 2024-03-19T01:12:09.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.257+0000 7fa667d71740 20 client.26074 trim_cache size 0 max 16384 2024-03-19T01:12:09.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.276+0000 7fa667d71740 1 Processor -- start 2024-03-19T01:12:09.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.276+0000 7fa667d71740 1 -- start start 2024-03-19T01:12:09.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.276+0000 7fa667d71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625ebf0 0x55f656273c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.276+0000 7fa667d71740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625f510 0x55f6562801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.276+0000 7fa667d71740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6562680f0 0x55f656265a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.276+0000 7fa667d71740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601b0d40 con 0x55f65625f510 2024-03-19T01:12:09.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.276+0000 7fa667d71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601b2910 con 0x55f65625ebf0 2024-03-19T01:12:09.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.276+0000 7fa667d71740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601b1410 con 0x55f6562680f0 2024-03-19T01:12:09.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.276+0000 7fa65e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625ebf0 0x55f656273c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.276+0000 7fa65d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625f510 0x55f6562801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa65e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625ebf0 0x55f656273c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37748/0 (socket says 172.21.15.69:37748) 2024-03-19T01:12:09.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa65e800640 1 -- 172.21.15.69:0/840799101 learned_addr learned my addr 172.21.15.69:0/840799101 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa65e800640 1 --2- 172.21.15.69:0/840799101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6562680f0 0x55f656265a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa65e800640 1 -- 172.21.15.69:0/840799101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6562680f0 msgr2=0x55f656265a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa65e800640 1 --2- 172.21.15.69:0/840799101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6562680f0 0x55f656265a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa65e800640 1 -- 172.21.15.69:0/840799101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625f510 msgr2=0x55f6562801b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa65e800640 1 --2- 172.21.15.69:0/840799101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625f510 0x55f6562801b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa65e800640 1 -- 172.21.15.69:0/840799101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f65627ad40 con 0x55f65625ebf0 2024-03-19T01:12:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa65d400640 1 --2- 172.21.15.69:0/840799101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625f510 0x55f6562801b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa65e800640 1 --2- 172.21.15.69:0/840799101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625ebf0 0x55f656273c80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa6580059f0 tx=0x7fa658011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3346cc728d16687e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa64aa00640 1 -- 172.21.15.69:0/840799101 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658005020 con 0x55f65625ebf0 2024-03-19T01:12:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa64aa00640 1 -- 172.21.15.69:0/840799101 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa658011d10 con 0x55f65625ebf0 2024-03-19T01:12:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.277+0000 7fa64aa00640 1 -- 172.21.15.69:0/840799101 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658005570 con 0x55f65625ebf0 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.278+0000 7fa667d71740 1 -- 172.21.15.69:0/840799101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625ebf0 msgr2=0x55f656273c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.278+0000 7fa667d71740 1 --2- 172.21.15.69:0/840799101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625ebf0 0x55f656273c80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa6580059f0 tx=0x7fa658011810 comp rx=0 tx=0).stop 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.278+0000 7fa667d71740 1 -- 172.21.15.69:0/840799101 shutdown_connections 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.278+0000 7fa667d71740 1 --2- 172.21.15.69:0/840799101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6562680f0 0x55f656265a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.278+0000 7fa667d71740 1 --2- 172.21.15.69:0/840799101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f65625f510 0x55f6562801b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.278+0000 7fa667d71740 1 --2- 172.21.15.69:0/840799101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625ebf0 0x55f656273c80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.278+0000 7fa667d71740 1 -- 172.21.15.69:0/840799101 >> 172.21.15.69:0/840799101 conn(0x55f656259320 msgr2=0x55f656258330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.278+0000 7fa667d71740 1 -- 172.21.15.69:0/840799101 shutdown_connections 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.278+0000 7fa667d71740 1 -- 172.21.15.69:0/840799101 wait complete. 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.279+0000 7fa667d71740 1 -- start start 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.279+0000 7fa667d71740 1 Processor -- start 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.279+0000 7fa667d71740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 0x55f6562704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.279+0000 7fa667d71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 0x55f6562619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.279+0000 7fa667d71740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6562680f0 0x55f656261f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.279+0000 7fa667d71740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601b21b0 con 0x55f6562680f0 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.279+0000 7fa667d71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601b1ae0 con 0x55f65625f510 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.279+0000 7fa667d71740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa6601aed20 con 0x55f65625ebf0 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.279+0000 7fa667d71740 10 client.0 _ll_register_callbacks cb 0x7fa6500f7400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa65e800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 0x55f6562704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa65d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 0x55f6562619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa65d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 0x55f6562619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37752/0 (socket says 172.21.15.69:37752) 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa65d400640 1 -- 172.21.15.69:0/693985173 learned_addr learned my addr 172.21.15.69:0/693985173 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa65e800640 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6562680f0 0x55f656261f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa65d400640 1 -- 172.21.15.69:0/693985173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 msgr2=0x55f6562704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa65d400640 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 0x55f6562704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa65d400640 1 -- 172.21.15.69:0/693985173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6562680f0 msgr2=0x55f656261f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa65d400640 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6562680f0 0x55f656261f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa65d400640 1 -- 172.21.15.69:0/693985173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6562624f0 con 0x55f65625f510 2024-03-19T01:12:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa65d400640 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 0x55f6562619c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55f6562649d0 tx=0x7fa64c01c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=663f7e928c213ef9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.280+0000 7fa63fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:09.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.281+0000 7fa63fe00640 1 -- 172.21.15.69:0/693985173 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c004030 con 0x55f65625f510 2024-03-19T01:12:09.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.281+0000 7fa63fe00640 1 -- 172.21.15.69:0/693985173 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa64c0041d0 con 0x55f65625f510 2024-03-19T01:12:09.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.281+0000 7fa667d71740 1 -- 172.21.15.69:0/693985173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa658011d10 con 0x55f65625f510 2024-03-19T01:12:09.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.281+0000 7fa63fe00640 1 -- 172.21.15.69:0/693985173 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa64c02c050 con 0x55f65625f510 2024-03-19T01:12:09.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.281+0000 7fa667d71740 10 client.26096 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:09.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.281+0000 7fa667d71740 1 -- 172.21.15.69:0/693985173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f6562de5d0 con 0x55f65625f510 2024-03-19T01:12:09.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.281+0000 7fa63fe00640 1 -- 172.21.15.69:0/693985173 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa64c020020 con 0x55f65625f510 2024-03-19T01:12:09.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.281+0000 7fa63fe00640 1 -- 172.21.15.69:0/693985173 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fa64c005510 con 0x55f65625f510 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.282+0000 7fa667d71740 10 client.26096 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.282+0000 7fa667d71740 10 client.26096 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.282+0000 7fa667d71740 20 client.26096 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.282+0000 7fa667d71740 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55f6562ded50 0x55f6562e1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.282+0000 7fa667d71740 4 client.26096 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.282+0000 7fa667d71740 1 -- 172.21.15.69:0/693985173 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6561cac10 con 0x55f6562ded50 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.282+0000 7fa65e800640 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55f6562ded50 0x55f6562e1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.282+0000 7fa65e800640 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55f6562ded50 0x55f6562e1170 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a958582928dbf9 server_cookie=62881d82a273fc89 in_seq=0 out_seq=0 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.282+0000 7fa63fe00640 10 client.26096 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa63fe00640 1 -- 172.21.15.69:0/693985173 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6561cac10 con 0x55f6562ded50 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa63fe00640 10 client.26096 handle_command_reply: tid=0 2024-03-19T01:12:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 1 client.26096 shutdown 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 1 -- 172.21.15.69:0/693985173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 msgr2=0x55f6562619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 0x55f6562619c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55f6562649d0 tx=0x7fa64c01c6e0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 1 -- 172.21.15.69:0/693985173 shutdown_connections 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6562680f0 0x55f656261f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55f6562ded50 0x55f6562e1170 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f65625f510 0x55f6562619c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 1 --2- 172.21.15.69:0/693985173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f65625ebf0 0x55f6562704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 1 -- 172.21.15.69:0/693985173 >> 172.21.15.69:0/693985173 conn(0x55f656259320 msgr2=0x55f65625a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa63fe00640 0 client.26096 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 1 -- 172.21.15.69:0/693985173 shutdown_connections 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 1 -- 172.21.15.69:0/693985173 wait complete. 2024-03-19T01:12:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.283+0000 7fa667d71740 20 client.26096 trim_cache size 0 max 16384 2024-03-19T01:12:09.289 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:12:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.284+0000 7fa65f200640 1 -- 172.21.15.69:0/2124374774 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6440a2a90 msgr2=0x7fa6440a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.284+0000 7fa65f200640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6440a2a90 0x7fa6440a4f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6500eebe0 tx=0x7fa6500f7530 comp rx=0 tx=0).stop 2024-03-19T01:12:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.284+0000 7fa65f200640 1 -- 172.21.15.69:0/2124374774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa660073bf0 msgr2=0x7fa66019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.284+0000 7fa65f200640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa660073bf0 0x7fa66019a170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa650005e30 tx=0x7fa650035d90 comp rx=0 tx=0).stop 2024-03-19T01:12:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.284+0000 7fa65f200640 1 -- 172.21.15.69:0/2124374774 shutdown_connections 2024-03-19T01:12:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.284+0000 7fa65f200640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa66010df30 0x7fa66019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.284+0000 7fa65f200640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa660074510 0x7fa66019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.284+0000 7fa65f200640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6440a2a90 0x7fa6440a4f50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.284+0000 7fa65f200640 1 --2- 172.21.15.69:0/2124374774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa660073bf0 0x7fa66019a170 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.284+0000 7fa65f200640 1 -- 172.21.15.69:0/2124374774 >> 172.21.15.69:0/2124374774 conn(0x7fa6600fd1c0 msgr2=0x7fa6600fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.285+0000 7fa65f200640 1 -- 172.21.15.69:0/2124374774 shutdown_connections 2024-03-19T01:12:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.285+0000 7fa65f200640 1 -- 172.21.15.69:0/2124374774 wait complete. 2024-03-19T01:12:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:09 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2876940233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:12:09.311 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:12:09.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:09 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2876940233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:12:09.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.523+0000 7f6129a00640 1 -- 172.21.15.69:0/615485277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61241038a0 msgr2=0x7f6124103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.523+0000 7f6129a00640 1 --2- 172.21.15.69:0/615485277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61241038a0 0x7f6124103d20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6110009930 tx=0x7f611002bc20 comp rx=0 tx=0).stop 2024-03-19T01:12:09.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.523+0000 7f6129a00640 1 -- 172.21.15.69:0/615485277 shutdown_connections 2024-03-19T01:12:09.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.523+0000 7f6129a00640 1 --2- 172.21.15.69:0/615485277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6124104260 0x7f6124110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.523+0000 7f6129a00640 1 --2- 172.21.15.69:0/615485277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61241038a0 0x7f6124103d20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.523+0000 7f6129a00640 1 --2- 172.21.15.69:0/615485277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61241098b0 0x7f6124109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.523+0000 7f6129a00640 1 -- 172.21.15.69:0/615485277 >> 172.21.15.69:0/615485277 conn(0x7f61240ff330 msgr2=0x7f6124101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.523+0000 7f6129a00640 1 -- 172.21.15.69:0/615485277 shutdown_connections 2024-03-19T01:12:09.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.523+0000 7f6129a00640 1 -- 172.21.15.69:0/615485277 wait complete. 2024-03-19T01:12:09.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.524+0000 7f6129a00640 1 Processor -- start 2024-03-19T01:12:09.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.524+0000 7f6129a00640 1 -- start start 2024-03-19T01:12:09.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6129a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61241038a0 0x7f612419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6129a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6124104260 0x7f612419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6129a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61241098b0 0x7f612419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6129a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6124113cd0 con 0x7f61241038a0 2024-03-19T01:12:09.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6129a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6124113b50 con 0x7f61241098b0 2024-03-19T01:12:09.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6129a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6124113e50 con 0x7f6124104260 2024-03-19T01:12:09.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6123e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61241038a0 0x7f612419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6123400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6124104260 0x7f612419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6129000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61241098b0 0x7f612419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6123400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6124104260 0x7f612419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58932/0 (socket says 172.21.15.69:58932) 2024-03-19T01:12:09.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6129000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61241098b0 0x7f612419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37760/0 (socket says 172.21.15.69:37760) 2024-03-19T01:12:09.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6123400640 1 -- 172.21.15.69:0/501300797 learned_addr learned my addr 172.21.15.69:0/501300797 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6123400640 1 -- 172.21.15.69:0/501300797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61241098b0 msgr2=0x7f612419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6123400640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61241098b0 0x7f612419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6123400640 1 -- 172.21.15.69:0/501300797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61241038a0 msgr2=0x7f612419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6123400640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61241038a0 0x7f612419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.525+0000 7f6123400640 1 -- 172.21.15.69:0/501300797 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612419f1a0 con 0x7f6124104260 2024-03-19T01:12:09.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.526+0000 7f6123400640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6124104260 0x7f612419a6f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6110005ab0 tx=0x7f6110005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.526+0000 7f6120c00640 1 -- 172.21.15.69:0/501300797 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110043020 con 0x7f6124104260 2024-03-19T01:12:09.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.526+0000 7f6120c00640 1 -- 172.21.15.69:0/501300797 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6110005000 con 0x7f6124104260 2024-03-19T01:12:09.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.526+0000 7f6120c00640 1 -- 172.21.15.69:0/501300797 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110003a80 con 0x7f6124104260 2024-03-19T01:12:09.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.526+0000 7f6129a00640 1 -- 172.21.15.69:0/501300797 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612419f430 con 0x7f6124104260 2024-03-19T01:12:09.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.526+0000 7f6129a00640 1 -- 172.21.15.69:0/501300797 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61241a6d20 con 0x7f6124104260 2024-03-19T01:12:09.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.529+0000 7f612dc76740 1 Processor -- start 2024-03-19T01:12:09.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f612dc76740 1 -- start start 2024-03-19T01:12:09.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f6120c00640 1 -- 172.21.15.69:0/501300797 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7f6110046020 con 0x7f6124104260 2024-03-19T01:12:09.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f612dc76740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 0x557a29c00fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f612dc76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c09bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f612dc76740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c0c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f612dc76740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557a29b65380 con 0x557a29c0a0f0 2024-03-19T01:12:09.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f612dc76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a29b0dd20 con 0x557a29c01510 2024-03-19T01:12:09.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f612dc76740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557a29b61420 con 0x557a29c00bf0 2024-03-19T01:12:09.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f6129000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 0x557a29c00fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f6129000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 0x557a29c00fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58944/0 (socket says 172.21.15.69:58944) 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f6129000640 1 -- 172.21.15.69:0/67206566 learned_addr learned my addr 172.21.15.69:0/67206566 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.530+0000 7f6123e00640 1 --2- 172.21.15.69:0/67206566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c09bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.531+0000 7f6123400640 1 --2- 172.21.15.69:0/67206566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c0c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.531+0000 7f6123e00640 1 -- 172.21.15.69:0/67206566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 msgr2=0x557a29c00fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.531+0000 7f6123e00640 1 --2- 172.21.15.69:0/67206566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 0x557a29c00fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.531+0000 7f6123e00640 1 -- 172.21.15.69:0/67206566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 msgr2=0x557a29c0c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.531+0000 7f6123e00640 1 --2- 172.21.15.69:0/67206566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c0c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.531+0000 7f6123e00640 1 -- 172.21.15.69:0/67206566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a298793f0 con 0x557a29c01510 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.531+0000 7f6123400640 1 --2- 172.21.15.69:0/67206566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c0c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.531+0000 7f6129000640 1 --2- 172.21.15.69:0/67206566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 0x557a29c00fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.531+0000 7f6123e00640 1 --2- 172.21.15.69:0/67206566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c09bb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6114009c20 tx=0x7f6114011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d46d2a2e44276754 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.531+0000 7f6119600640 1 -- 172.21.15.69:0/67206566 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114004030 con 0x557a29c01510 2024-03-19T01:12:09.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.532+0000 7f6119600640 1 -- 172.21.15.69:0/67206566 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61140041d0 con 0x557a29c01510 2024-03-19T01:12:09.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.532+0000 7f6119600640 1 -- 172.21.15.69:0/67206566 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114019760 con 0x557a29c01510 2024-03-19T01:12:09.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.532+0000 7f612dc76740 1 -- 172.21.15.69:0/67206566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 msgr2=0x557a29c09bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.532+0000 7f612dc76740 1 --2- 172.21.15.69:0/67206566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c09bb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6114009c20 tx=0x7f6114011810 comp rx=0 tx=0).stop 2024-03-19T01:12:09.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.532+0000 7f6120c00640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f61080a2b10 0x7f61080a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.532+0000 7f612dc76740 1 -- 172.21.15.69:0/67206566 shutdown_connections 2024-03-19T01:12:09.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.532+0000 7f612dc76740 1 --2- 172.21.15.69:0/67206566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c0c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.532+0000 7f612dc76740 1 --2- 172.21.15.69:0/67206566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c09bb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.532+0000 7f612dc76740 1 --2- 172.21.15.69:0/67206566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 0x557a29c00fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.532+0000 7f6120c00640 1 -- 172.21.15.69:0/501300797 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f61100f0d70 con 0x7f6124104260 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.532+0000 7f612dc76740 1 -- 172.21.15.69:0/67206566 >> 172.21.15.69:0/67206566 conn(0x557a29bfb320 msgr2=0x557a29bfb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.533+0000 7f6129000640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f61080a2b10 0x7f61080a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.533+0000 7f612dc76740 1 -- 172.21.15.69:0/67206566 shutdown_connections 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.533+0000 7f612dc76740 1 -- 172.21.15.69:0/67206566 wait complete. 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.533+0000 7f612dc76740 1 -- start start 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.533+0000 7f6129000640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f61080a2b10 0x7f61080a4fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f611c005fd0 tx=0x7f611c009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.533+0000 7f612dc76740 1 Processor -- start 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f612dc76740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c00bf0 0x557a29c07140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f612dc76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c07680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f612dc76740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c0a0f0 0x557a29c07bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f612dc76740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557a29c0cee0 con 0x557a29c00bf0 2024-03-19T01:12:09.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f612dc76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a29b21160 con 0x557a29c01510 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f612dc76740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557a29c0d060 con 0x557a29c0a0f0 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f612dc76740 10 client.0 _ll_register_callbacks cb 0x557a29b5d170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f6129000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c0a0f0 0x557a29c07bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f6123e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c00bf0 0x557a29c07140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f6129000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c0a0f0 0x557a29c07bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58954/0 (socket says 172.21.15.69:58954) 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f6129000640 1 -- 172.21.15.69:0/3951246298 learned_addr learned my addr 172.21.15.69:0/3951246298 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.534+0000 7f6123400640 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c07680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.535+0000 7f6129000640 1 -- 172.21.15.69:0/3951246298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 msgr2=0x557a29c07680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.535+0000 7f6129000640 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c07680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.535+0000 7f6129000640 1 -- 172.21.15.69:0/3951246298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c00bf0 msgr2=0x557a29c07140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.535+0000 7f6129000640 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c00bf0 0x557a29c07140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.535+0000 7f6129000640 1 -- 172.21.15.69:0/3951246298 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a299437d0 con 0x557a29c0a0f0 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.535+0000 7f6123400640 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c07680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.535+0000 7f6129000640 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c0a0f0 0x557a29c07bc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f611c00fad0 tx=0x7f611c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=743e25a9e73ea09f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.535+0000 7f610e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.535+0000 7f610e000640 1 -- 172.21.15.69:0/3951246298 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c004b20 con 0x557a29c0a0f0 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.535+0000 7f610e000640 1 -- 172.21.15.69:0/3951246298 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f611c017030 con 0x557a29c0a0f0 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.536+0000 7f612dc76740 1 -- 172.21.15.69:0/3951246298 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61140041d0 con 0x557a29c0a0f0 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.536+0000 7f610e000640 1 -- 172.21.15.69:0/3951246298 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f611c004cc0 con 0x557a29c0a0f0 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.536+0000 7f612dc76740 10 client.26092 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.536+0000 7f612dc76740 1 -- 172.21.15.69:0/3951246298 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a29c19200 con 0x557a29c0a0f0 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.536+0000 7f610e000640 1 -- 172.21.15.69:0/3951246298 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61140041d0 con 0x557a29c0a0f0 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.536+0000 7f610e000640 1 -- 172.21.15.69:0/3951246298 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f611c004030 con 0x557a29c0a0f0 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.536+0000 7f612dc76740 10 client.26092 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.536+0000 7f612dc76740 10 client.26092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.537+0000 7f612dc76740 20 client.26092 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.537+0000 7f612dc76740 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557a29c1a760 0x557a29c1cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.537+0000 7f612dc76740 4 client.26092 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.537+0000 7f612dc76740 1 -- 172.21.15.69:0/3951246298 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a29b728c0 con 0x557a29c1a760 2024-03-19T01:12:09.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.537+0000 7f6123e00640 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557a29c1a760 0x557a29c1cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.538+0000 7f6123e00640 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557a29c1a760 0x557a29c1cbc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8f114a5d343b4e1 server_cookie=fc998b0396884b8 in_seq=0 out_seq=0 2024-03-19T01:12:09.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.538+0000 7f610e000640 10 client.26092 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:09.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.538+0000 7f610e000640 1 -- 172.21.15.69:0/3951246298 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557a29b728c0 con 0x557a29c1a760 2024-03-19T01:12:09.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.538+0000 7f610e000640 10 client.26092 handle_command_reply: tid=0 2024-03-19T01:12:09.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.538+0000 7f612dc76740 1 client.26092 shutdown 2024-03-19T01:12:09.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f612dc76740 1 -- 172.21.15.69:0/3951246298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c0a0f0 msgr2=0x557a29c07bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f612dc76740 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c0a0f0 0x557a29c07bc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f611c00fad0 tx=0x7f611c004ab0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f612dc76740 1 -- 172.21.15.69:0/3951246298 shutdown_connections 2024-03-19T01:12:09.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f612dc76740 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557a29c1a760 0x557a29c1cbc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f612dc76740 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c0a0f0 0x557a29c07bc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f612dc76740 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c07680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f612dc76740 1 --2- 172.21.15.69:0/3951246298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c00bf0 0x557a29c07140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f610e000640 0 client.26092 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f612dc76740 1 -- 172.21.15.69:0/3951246298 >> 172.21.15.69:0/3951246298 conn(0x557a29bfb320 msgr2=0x557a29bfd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f612dc76740 1 -- 172.21.15.69:0/3951246298 shutdown_connections 2024-03-19T01:12:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f612dc76740 1 -- 172.21.15.69:0/3951246298 wait complete. 2024-03-19T01:12:09.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.539+0000 7f612dc76740 20 client.26092 trim_cache size 0 max 16384 2024-03-19T01:12:09.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f612dc76740 1 Processor -- start 2024-03-19T01:12:09.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f612dc76740 1 -- start start 2024-03-19T01:12:09.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f612dc76740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 0x557a29c09870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f612dc76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c15c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f612dc76740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c22490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f612dc76740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f61241b0dc0 con 0x557a29c0a0f0 2024-03-19T01:12:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f612dc76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61241b2990 con 0x557a29c01510 2024-03-19T01:12:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f612dc76740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f61241b1490 con 0x557a29c00bf0 2024-03-19T01:12:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f6123e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 0x557a29c09870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f6123400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c15c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f6123e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 0x557a29c09870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58960/0 (socket says 172.21.15.69:58960) 2024-03-19T01:12:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f6123e00640 1 -- 172.21.15.69:0/10648098 learned_addr learned my addr 172.21.15.69:0/10648098 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.556+0000 7f6129000640 1 --2- 172.21.15.69:0/10648098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c22490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.557+0000 7f6123400640 1 -- 172.21.15.69:0/10648098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 msgr2=0x557a29c09870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.557+0000 7f6123400640 1 --2- 172.21.15.69:0/10648098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 0x557a29c09870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.557+0000 7f6123400640 1 -- 172.21.15.69:0/10648098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 msgr2=0x557a29c22490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.557+0000 7f6123400640 1 --2- 172.21.15.69:0/10648098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c22490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.557+0000 7f6123400640 1 -- 172.21.15.69:0/10648098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a29c1d240 con 0x557a29c01510 2024-03-19T01:12:09.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.557+0000 7f6123400640 1 --2- 172.21.15.69:0/10648098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c15c80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f61100ee030 tx=0x7f61100b9110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cabe1f70650f0879 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.557+0000 7f6118c00640 1 -- 172.21.15.69:0/10648098 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100bc7d0 con 0x557a29c01510 2024-03-19T01:12:09.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.557+0000 7f6118c00640 1 -- 172.21.15.69:0/10648098 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61100bd570 con 0x557a29c01510 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.557+0000 7f6118c00640 1 -- 172.21.15.69:0/10648098 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100054e0 con 0x557a29c01510 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.558+0000 7f612dc76740 1 -- 172.21.15.69:0/10648098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 msgr2=0x557a29c15c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.558+0000 7f612dc76740 1 --2- 172.21.15.69:0/10648098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c15c80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f61100ee030 tx=0x7f61100b9110 comp rx=0 tx=0).stop 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.558+0000 7f612dc76740 1 -- 172.21.15.69:0/10648098 shutdown_connections 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.558+0000 7f612dc76740 1 --2- 172.21.15.69:0/10648098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c22490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.558+0000 7f612dc76740 1 --2- 172.21.15.69:0/10648098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c01510 0x557a29c15c80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.558+0000 7f612dc76740 1 --2- 172.21.15.69:0/10648098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c00bf0 0x557a29c09870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.558+0000 7f612dc76740 1 -- 172.21.15.69:0/10648098 >> 172.21.15.69:0/10648098 conn(0x557a29bfb320 msgr2=0x557a29bfa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.558+0000 7f612dc76740 1 -- 172.21.15.69:0/10648098 shutdown_connections 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.558+0000 7f612dc76740 1 -- 172.21.15.69:0/10648098 wait complete. 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.559+0000 7f612dc76740 1 -- start start 2024-03-19T01:12:09.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.559+0000 7f612dc76740 1 Processor -- start 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f612dc76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c00bf0 0x557a29c04300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f612dc76740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c01510 0x557a29c04840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f612dc76740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c7fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f612dc76740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f61241b2230 con 0x557a29c0a0f0 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f612dc76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61241b1b60 con 0x557a29c00bf0 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f612dc76740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f61241aeda0 con 0x557a29c01510 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f612dc76740 10 client.0 _ll_register_callbacks cb 0x7f611c009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f6123e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c00bf0 0x557a29c04300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f6123400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c01510 0x557a29c04840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f6123e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c00bf0 0x557a29c04300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37788/0 (socket says 172.21.15.69:37788) 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f6123e00640 1 -- 172.21.15.69:0/3789125317 learned_addr learned my addr 172.21.15.69:0/3789125317 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:09.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.560+0000 7f6129000640 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c7fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f6123400640 1 -- 172.21.15.69:0/3789125317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c00bf0 msgr2=0x557a29c04300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f6123400640 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c00bf0 0x557a29c04300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f6123400640 1 -- 172.21.15.69:0/3789125317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 msgr2=0x557a29c7fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f6123400640 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c7fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f6123400640 1 -- 172.21.15.69:0/3789125317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a29c80000 con 0x557a29c01510 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f6123e00640 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c00bf0 0x557a29c04300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f6129000640 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c7fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f6123400640 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c01510 0x557a29c04840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6110045fd0 tx=0x7f61100bc840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e78151fa1858a19 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f610e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f610e000640 1 -- 172.21.15.69:0/3789125317 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100d9070 con 0x557a29c01510 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f610e000640 1 -- 172.21.15.69:0/3789125317 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f61100bd0d0 con 0x557a29c01510 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f612dc76740 1 -- 172.21.15.69:0/3789125317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61100bd570 con 0x557a29c01510 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f610e000640 1 -- 172.21.15.69:0/3789125317 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61100c5610 con 0x557a29c01510 2024-03-19T01:12:09.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f612dc76740 10 client.26098 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.561+0000 7f612dc76740 1 -- 172.21.15.69:0/3789125317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a29c81200 con 0x557a29c01510 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.562+0000 7f610e000640 1 -- 172.21.15.69:0/3789125317 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61100bd570 con 0x557a29c01510 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.562+0000 7f610e000640 1 -- 172.21.15.69:0/3789125317 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f61100d3020 con 0x557a29c01510 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.562+0000 7f612dc76740 10 client.26098 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.562+0000 7f612dc76740 10 client.26098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.562+0000 7f612dc76740 20 client.26098 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.562+0000 7f612dc76740 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557a29c81790 0x557a29c83bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.562+0000 7f612dc76740 4 client.26098 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.562+0000 7f612dc76740 1 -- 172.21.15.69:0/3789125317 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a29b6cc10 con 0x557a29c81790 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.562+0000 7f6123e00640 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557a29c81790 0x557a29c83bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.563+0000 7f6123e00640 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557a29c81790 0x557a29c83bb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4af224c092ce749e server_cookie=4b7700d3b5d513f in_seq=0 out_seq=0 2024-03-19T01:12:09.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.563+0000 7f610e000640 10 client.26098 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:09.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:09 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2876940233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f610e000640 1 -- 172.21.15.69:0/3789125317 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a29b6cc10 con 0x557a29c81790 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f610e000640 10 client.26098 handle_command_reply: tid=0 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 1 client.26098 shutdown 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 1 -- 172.21.15.69:0/3789125317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c01510 msgr2=0x557a29c04840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c01510 0x557a29c04840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6110045fd0 tx=0x7f61100bc840 comp rx=0 tx=0).stop 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 1 -- 172.21.15.69:0/3789125317 shutdown_connections 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557a29c81790 0x557a29c83bb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a29c0a0f0 0x557a29c7fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557a29c01510 0x557a29c04840 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 1 --2- 172.21.15.69:0/3789125317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a29c00bf0 0x557a29c04300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f610e000640 0 client.26098 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:09.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 1 -- 172.21.15.69:0/3789125317 >> 172.21.15.69:0/3789125317 conn(0x557a29bfb320 msgr2=0x557a29bfc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 1 -- 172.21.15.69:0/3789125317 shutdown_connections 2024-03-19T01:12:09.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 1 -- 172.21.15.69:0/3789125317 wait complete. 2024-03-19T01:12:09.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.564+0000 7f612dc76740 20 client.26098 trim_cache size 0 max 16384 2024-03-19T01:12:09.570 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 -- 172.21.15.69:0/501300797 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f61080a2b10 msgr2=0x7f61080a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f61080a2b10 0x7f61080a4fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f611c005fd0 tx=0x7f611c009500 comp rx=0 tx=0).stop 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 -- 172.21.15.69:0/501300797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6124104260 msgr2=0x7f612419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6124104260 0x7f612419a6f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6110005ab0 tx=0x7f6110005e80 comp rx=0 tx=0).stop 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 -- 172.21.15.69:0/501300797 shutdown_connections 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f61080a2b10 0x7f61080a4fd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61241098b0 0x7f612419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6124104260 0x7f612419a6f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 --2- 172.21.15.69:0/501300797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61241038a0 0x7f612419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 -- 172.21.15.69:0/501300797 >> 172.21.15.69:0/501300797 conn(0x7f61240ff330 msgr2=0x7f61240ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 -- 172.21.15.69:0/501300797 shutdown_connections 2024-03-19T01:12:09.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:09.565+0000 7f6129a00640 1 -- 172.21.15.69:0/501300797 wait complete. 2024-03-19T01:12:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:10 smithi086 ceph-mon[66829]: pgmap v48: 129 pgs: 129 active+clean; 102 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 5.4 MiB/s wr, 74 op/s 2024-03-19T01:12:10.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:10 smithi069 ceph-mon[31727]: pgmap v48: 129 pgs: 129 active+clean; 102 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 5.4 MiB/s wr, 74 op/s 2024-03-19T01:12:10.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:10 smithi163 ceph-mon[67428]: pgmap v48: 129 pgs: 129 active+clean; 102 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 5.4 MiB/s wr, 74 op/s 2024-03-19T01:12:10.584 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:12:10.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:12:10.585 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:12:10.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.806+0000 7f0729e00640 1 -- 172.21.15.69:0/3677432210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0724102ab0 msgr2=0x7f0724102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:10.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.806+0000 7f0729e00640 1 --2- 172.21.15.69:0/3677432210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0724102ab0 0x7f0724102e90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0714009ca0 tx=0x7f071402c180 comp rx=0 tx=0).stop 2024-03-19T01:12:10.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.806+0000 7f0729e00640 1 -- 172.21.15.69:0/3677432210 shutdown_connections 2024-03-19T01:12:10.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.806+0000 7f0729e00640 1 --2- 172.21.15.69:0/3677432210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f072410bdf0 0x7f072410e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.806+0000 7f0729e00640 1 --2- 172.21.15.69:0/3677432210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07241033d0 0x7f072410b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.806+0000 7f0729e00640 1 --2- 172.21.15.69:0/3677432210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0724102ab0 0x7f0724102e90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.806+0000 7f0729e00640 1 -- 172.21.15.69:0/3677432210 >> 172.21.15.69:0/3677432210 conn(0x7f0724077880 msgr2=0x7f0724077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:10.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.806+0000 7f0729e00640 1 -- 172.21.15.69:0/3677432210 shutdown_connections 2024-03-19T01:12:10.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.806+0000 7f0729e00640 1 -- 172.21.15.69:0/3677432210 wait complete. 2024-03-19T01:12:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.807+0000 7f0729e00640 1 Processor -- start 2024-03-19T01:12:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.807+0000 7f0729e00640 1 -- start start 2024-03-19T01:12:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.807+0000 7f0729e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0724102ab0 0x7f07241980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.807+0000 7f0729e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07241033d0 0x7f07241985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.807+0000 7f0729e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f072410bdf0 0x7f072419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0729e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0724111bc0 con 0x7f072410bdf0 2024-03-19T01:12:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0729e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0724111a40 con 0x7f07241033d0 2024-03-19T01:12:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0729e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0724111d40 con 0x7f0724102ab0 2024-03-19T01:12:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0722000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07241033d0 0x7f07241985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0722a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0724102ab0 0x7f07241980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0723400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f072410bdf0 0x7f072419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0722000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07241033d0 0x7f07241985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37804/0 (socket says 172.21.15.69:37804) 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0722a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0724102ab0 0x7f07241980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:58986/0 (socket says 172.21.15.69:58986) 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0722000640 1 -- 172.21.15.69:0/2810988632 learned_addr learned my addr 172.21.15.69:0/2810988632 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0722000640 1 -- 172.21.15.69:0/2810988632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0724102ab0 msgr2=0x7f07241980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0722000640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0724102ab0 0x7f07241980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0722000640 1 -- 172.21.15.69:0/2810988632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f072410bdf0 msgr2=0x7f072419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0722000640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f072410bdf0 0x7f072419c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0722000640 1 -- 172.21.15.69:0/2810988632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072419d090 con 0x7f07241033d0 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.808+0000 7f0722000640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07241033d0 0x7f07241985e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f071000e9f0 tx=0x7f071000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.809+0000 7f0728a00640 1 -- 172.21.15.69:0/2810988632 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071000cd80 con 0x7f07241033d0 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.809+0000 7f0729e00640 1 -- 172.21.15.69:0/2810988632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072419d380 con 0x7f07241033d0 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.809+0000 7f0728a00640 1 -- 172.21.15.69:0/2810988632 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0710010910 con 0x7f07241033d0 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.809+0000 7f0728a00640 1 -- 172.21.15.69:0/2810988632 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100188f0 con 0x7f07241033d0 2024-03-19T01:12:10.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.809+0000 7f0729e00640 1 -- 172.21.15.69:0/2810988632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07241a4c10 con 0x7f07241033d0 2024-03-19T01:12:10.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.813+0000 7f072ca1e740 1 Processor -- start 2024-03-19T01:12:10.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f072ca1e740 1 -- start start 2024-03-19T01:12:10.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f0728a00640 1 -- 172.21.15.69:0/2810988632 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7f0710010ac0 con 0x7f07241033d0 2024-03-19T01:12:10.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 0x56224a6e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6e4e90 0x56224a6f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6f2410 0x56224a6f4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f072ca1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56224a647fb0 con 0x56224a6e4e90 2024-03-19T01:12:10.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f072ca1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56224a5f0ab0 con 0x56224a6e44e0 2024-03-19T01:12:10.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f072ca1e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56224a644050 con 0x56224a6f2410 2024-03-19T01:12:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f0723400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 0x56224a6e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f0723400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 0x56224a6e48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37810/0 (socket says 172.21.15.69:37810) 2024-03-19T01:12:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f0723400640 1 -- 172.21.15.69:0/1484892086 learned_addr learned my addr 172.21.15.69:0/1484892086 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f0722000640 1 --2- 172.21.15.69:0/1484892086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6f2410 0x56224a6f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.814+0000 7f0722a00640 1 --2- 172.21.15.69:0/1484892086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6e4e90 0x56224a6f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f0723400640 1 -- 172.21.15.69:0/1484892086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6f2410 msgr2=0x56224a6f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:10.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f0723400640 1 --2- 172.21.15.69:0/1484892086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6f2410 0x56224a6f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f0723400640 1 -- 172.21.15.69:0/1484892086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6e4e90 msgr2=0x56224a6f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:10.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f0723400640 1 --2- 172.21.15.69:0/1484892086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6e4e90 0x56224a6f1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f0723400640 1 -- 172.21.15.69:0/1484892086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56224a60ee00 con 0x56224a6e44e0 2024-03-19T01:12:10.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f0722a00640 1 --2- 172.21.15.69:0/1484892086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6e4e90 0x56224a6f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:10.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f0723400640 1 --2- 172.21.15.69:0/1484892086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 0x56224a6e48c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f071c009c10 tx=0x7f071c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cfa3fe00f06c82f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:10.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f0728a00640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07080a2a90 0x7f07080a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f070fe00640 1 -- 172.21.15.69:0/1484892086 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c01e070 con 0x56224a6e44e0 2024-03-19T01:12:10.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f0728a00640 1 -- 172.21.15.69:0/2810988632 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f07100cf890 con 0x7f07241033d0 2024-03-19T01:12:10.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f070fe00640 1 -- 172.21.15.69:0/1484892086 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f071c019b10 con 0x56224a6e44e0 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.815+0000 7f070fe00640 1 -- 172.21.15.69:0/1484892086 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c00e9d0 con 0x56224a6e44e0 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.816+0000 7f0722a00640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07080a2a90 0x7f07080a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.816+0000 7f072ca1e740 1 -- 172.21.15.69:0/1484892086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 msgr2=0x56224a6e48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.816+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1484892086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 0x56224a6e48c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f071c009c10 tx=0x7f071c011810 comp rx=0 tx=0).stop 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.816+0000 7f072ca1e740 1 -- 172.21.15.69:0/1484892086 shutdown_connections 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.816+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1484892086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6f2410 0x56224a6f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.816+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1484892086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6e4e90 0x56224a6f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.816+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1484892086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 0x56224a6e48c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.816+0000 7f072ca1e740 1 -- 172.21.15.69:0/1484892086 >> 172.21.15.69:0/1484892086 conn(0x56224a6df230 msgr2=0x56224a6df610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.817+0000 7f0722a00640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07080a2a90 0x7f07080a4f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0714004af0 tx=0x7f071402e040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.817+0000 7f072ca1e740 1 -- 172.21.15.69:0/1484892086 shutdown_connections 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.817+0000 7f072ca1e740 1 -- 172.21.15.69:0/1484892086 wait complete. 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.817+0000 7f072ca1e740 1 -- start start 2024-03-19T01:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.817+0000 7f072ca1e740 1 Processor -- start 2024-03-19T01:12:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 0x56224a6ef450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6ef990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6f2410 0x56224a6efed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f072ca1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56224a6f51d0 con 0x56224a6f2410 2024-03-19T01:12:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f072ca1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56224a603f00 con 0x56224a6e44e0 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f072ca1e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56224a6f5350 con 0x56224a6e4e90 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f072ca1e740 10 client.0 _ll_register_callbacks cb 0x56224a6038d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f0722000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6ef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f0722000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6ef990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59012/0 (socket says 172.21.15.69:59012) 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f0722000640 1 -- 172.21.15.69:0/3721170745 learned_addr learned my addr 172.21.15.69:0/3721170745 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f0723400640 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6f2410 0x56224a6efed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.818+0000 7f0722a00640 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 0x56224a6ef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f0722000640 1 -- 172.21.15.69:0/3721170745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 msgr2=0x56224a6ef450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f0722000640 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 0x56224a6ef450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f0722000640 1 -- 172.21.15.69:0/3721170745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6f2410 msgr2=0x56224a6efed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f0722000640 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6f2410 0x56224a6efed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f0722000640 1 -- 172.21.15.69:0/3721170745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56224a4247e0 con 0x56224a6e4e90 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f0722a00640 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 0x56224a6ef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f0722000640 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6ef990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f071000e9c0 tx=0x7f07100d5750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c65867129255017 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f070cc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f070cc00640 1 -- 172.21.15.69:0/3721170745 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071009b2f0 con 0x56224a6e4e90 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f070cc00640 1 -- 172.21.15.69:0/3721170745 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f07100cf470 con 0x56224a6e4e90 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f072ca1e740 1 -- 172.21.15.69:0/3721170745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f071c019b10 con 0x56224a6e4e90 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f070cc00640 1 -- 172.21.15.69:0/3721170745 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f071009b990 con 0x56224a6e4e90 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f072ca1e740 10 client.26110 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.819+0000 7f072ca1e740 1 -- 172.21.15.69:0/3721170745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56224a484f70 con 0x56224a6e4e90 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.820+0000 7f070cc00640 1 -- 172.21.15.69:0/3721170745 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f071c019b10 con 0x56224a6e4e90 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.820+0000 7f070cc00640 1 -- 172.21.15.69:0/3721170745 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f071009ade0 con 0x56224a6e4e90 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.820+0000 7f072ca1e740 10 client.26110 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.820+0000 7f072ca1e740 10 client.26110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.820+0000 7f072ca1e740 20 client.26110 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.820+0000 7f072ca1e740 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56224a702840 0x56224a704c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.820+0000 7f072ca1e740 4 client.26110 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.820+0000 7f072ca1e740 1 -- 172.21.15.69:0/3721170745 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56224a6554f0 con 0x56224a702840 2024-03-19T01:12:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.820+0000 7f0722a00640 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56224a702840 0x56224a704c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.821+0000 7f0722a00640 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56224a702840 0x56224a704c30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff8143c2010db44c server_cookie=2bfde61b2425053a in_seq=0 out_seq=0 2024-03-19T01:12:10.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.821+0000 7f070cc00640 10 client.26110 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.822+0000 7f070cc00640 1 -- 172.21.15.69:0/3721170745 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56224a6554f0 con 0x56224a702840 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.822+0000 7f070cc00640 10 client.26110 handle_command_reply: tid=0 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 1 client.26110 shutdown 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 1 -- 172.21.15.69:0/3721170745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 msgr2=0x56224a6ef990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6ef990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f071000e9c0 tx=0x7f07100d5750 comp rx=0 tx=0).stop 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 1 -- 172.21.15.69:0/3721170745 shutdown_connections 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56224a702840 0x56224a704c30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6f2410 0x56224a6efed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6ef990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 1 --2- 172.21.15.69:0/3721170745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e44e0 0x56224a6ef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 1 -- 172.21.15.69:0/3721170745 >> 172.21.15.69:0/3721170745 conn(0x56224a6df230 msgr2=0x56224a6e0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f070cc00640 0 client.26110 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:10.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 1 -- 172.21.15.69:0/3721170745 shutdown_connections 2024-03-19T01:12:10.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 1 -- 172.21.15.69:0/3721170745 wait complete. 2024-03-19T01:12:10.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.823+0000 7f072ca1e740 20 client.26110 trim_cache size 0 max 16384 2024-03-19T01:12:10.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.842+0000 7f072ca1e740 1 Processor -- start 2024-03-19T01:12:10.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.842+0000 7f072ca1e740 1 -- start start 2024-03-19T01:12:10.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6e44e0 0x56224a6ee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6f2410 0x56224a70a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f072ca1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07241acb20 con 0x56224a6e44e0 2024-03-19T01:12:10.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f072ca1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07241ae6f0 con 0x56224a6f2410 2024-03-19T01:12:10.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f072ca1e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f07241ad1f0 con 0x56224a6e4e90 2024-03-19T01:12:10.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f0722a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6e44e0 0x56224a6ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f0722000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f0723400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6f2410 0x56224a70a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f0722000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6fdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59016/0 (socket says 172.21.15.69:59016) 2024-03-19T01:12:10.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f0722000640 1 -- 172.21.15.69:0/1398426244 learned_addr learned my addr 172.21.15.69:0/1398426244 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:10.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f0723400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6f2410 0x56224a70a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37830/0 (socket says 172.21.15.69:37830) 2024-03-19T01:12:10.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f0722000640 1 -- 172.21.15.69:0/1398426244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6f2410 msgr2=0x56224a70a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.843+0000 7f0722000640 1 --2- 172.21.15.69:0/1398426244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6f2410 0x56224a70a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.844+0000 7f0722000640 1 -- 172.21.15.69:0/1398426244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6e44e0 msgr2=0x56224a6ee390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.844+0000 7f0722000640 1 --2- 172.21.15.69:0/1398426244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6e44e0 0x56224a6ee390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.844+0000 7f0722000640 1 -- 172.21.15.69:0/1398426244 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56224a31c170 con 0x56224a6e4e90 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.844+0000 7f0722000640 1 --2- 172.21.15.69:0/1398426244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6fdfb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0710097df0 tx=0x7f071009b240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b1d7c204d1f53e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.844+0000 7f070f400640 1 -- 172.21.15.69:0/1398426244 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071009b760 con 0x56224a6e4e90 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.844+0000 7f070f400640 1 -- 172.21.15.69:0/1398426244 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07100cc9e0 con 0x56224a6e4e90 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.844+0000 7f070f400640 1 -- 172.21.15.69:0/1398426244 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100ccd80 con 0x56224a6e4e90 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.845+0000 7f072ca1e740 1 -- 172.21.15.69:0/1398426244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 msgr2=0x56224a6fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.845+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1398426244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6fdfb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0710097df0 tx=0x7f071009b240 comp rx=0 tx=0).stop 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.845+0000 7f072ca1e740 1 -- 172.21.15.69:0/1398426244 shutdown_connections 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.845+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1398426244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6f2410 0x56224a70a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.845+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1398426244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e4e90 0x56224a6fdfb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.845+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1398426244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6e44e0 0x56224a6ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.845+0000 7f072ca1e740 1 -- 172.21.15.69:0/1398426244 >> 172.21.15.69:0/1398426244 conn(0x56224a6df230 msgr2=0x56224a6de2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:10.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.845+0000 7f072ca1e740 1 -- 172.21.15.69:0/1398426244 shutdown_connections 2024-03-19T01:12:10.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.845+0000 7f072ca1e740 1 -- 172.21.15.69:0/1398426244 wait complete. 2024-03-19T01:12:10.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.846+0000 7f072ca1e740 1 -- start start 2024-03-19T01:12:10.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.846+0000 7f072ca1e740 1 Processor -- start 2024-03-19T01:12:10.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.846+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e44e0 0x56224a6f6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.846+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e4e90 0x56224a6f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.846+0000 7f072ca1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6f2410 0x56224a767b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.846+0000 7f072ca1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07241adf90 con 0x56224a6f2410 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.846+0000 7f072ca1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07241ad8c0 con 0x56224a6e4e90 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.846+0000 7f072ca1e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f07241aab00 con 0x56224a6e44e0 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.847+0000 7f072ca1e740 10 client.0 _ll_register_callbacks cb 0x7f07100d5200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.847+0000 7f0722a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e44e0 0x56224a6f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.847+0000 7f0722000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e4e90 0x56224a6f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.847+0000 7f0722a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e44e0 0x56224a6f6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59020/0 (socket says 172.21.15.69:59020) 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.847+0000 7f0722a00640 1 -- 172.21.15.69:0/1901776174 learned_addr learned my addr 172.21.15.69:0/1901776174 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.847+0000 7f0723400640 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6f2410 0x56224a767b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.847+0000 7f0722000640 1 -- 172.21.15.69:0/1901776174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e44e0 msgr2=0x56224a6f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:10.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.847+0000 7f0722000640 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e44e0 0x56224a6f6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.847+0000 7f0722000640 1 -- 172.21.15.69:0/1901776174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6f2410 msgr2=0x56224a767b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.847+0000 7f0722000640 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6f2410 0x56224a767b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.847+0000 7f0722000640 1 -- 172.21.15.69:0/1901776174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56224a6f70b0 con 0x56224a6e4e90 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.848+0000 7f0722000640 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e4e90 0x56224a6f67b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x56224a6f1f40 tx=0x7f07100ba2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b3044e6ef37737f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.848+0000 7f070cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.848+0000 7f070cc00640 1 -- 172.21.15.69:0/1901776174 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100aa020 con 0x56224a6e4e90 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.848+0000 7f072ca1e740 1 -- 172.21.15.69:0/1901776174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07100cc9e0 con 0x56224a6e4e90 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.848+0000 7f070cc00640 1 -- 172.21.15.69:0/1901776174 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f07100ccf80 con 0x56224a6e4e90 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.848+0000 7f070cc00640 1 -- 172.21.15.69:0/1901776174 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07100a4080 con 0x56224a6e4e90 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.848+0000 7f072ca1e740 10 client.26132 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.848+0000 7f072ca1e740 1 -- 172.21.15.69:0/1901776174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56224a769180 con 0x56224a6e4e90 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.848+0000 7f070cc00640 1 -- 172.21.15.69:0/1901776174 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07100cc9e0 con 0x56224a6e4e90 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.848+0000 7f070cc00640 1 -- 172.21.15.69:0/1901776174 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f071009a970 con 0x56224a6e4e90 2024-03-19T01:12:10.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.849+0000 7f072ca1e740 10 client.26132 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:10.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.849+0000 7f072ca1e740 10 client.26132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:10.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.849+0000 7f072ca1e740 20 client.26132 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:10.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.849+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56224a769790 0x56224a76bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:10.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.849+0000 7f072ca1e740 4 client.26132 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:12:10.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.849+0000 7f072ca1e740 1 -- 172.21.15.69:0/1901776174 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56224a64f840 con 0x56224a769790 2024-03-19T01:12:10.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.849+0000 7f0722a00640 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56224a769790 0x56224a76bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:10.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.849+0000 7f0722a00640 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56224a769790 0x56224a76bbb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33524fadf4b9482 server_cookie=40146ab5f883014a in_seq=0 out_seq=0 2024-03-19T01:12:10.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.849+0000 7f070cc00640 10 client.26132 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:10.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.851+0000 7f070cc00640 1 -- 172.21.15.69:0/1901776174 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56224a64f840 con 0x56224a769790 2024-03-19T01:12:10.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.851+0000 7f070cc00640 10 client.26132 handle_command_reply: tid=0 2024-03-19T01:12:10.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.851+0000 7f072ca1e740 1 client.26132 shutdown 2024-03-19T01:12:10.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f072ca1e740 1 -- 172.21.15.69:0/1901776174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e4e90 msgr2=0x56224a6f67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:10.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e4e90 0x56224a6f67b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x56224a6f1f40 tx=0x7f07100ba2b0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f072ca1e740 1 -- 172.21.15.69:0/1901776174 shutdown_connections 2024-03-19T01:12:10.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56224a769790 0x56224a76bbb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56224a6f2410 0x56224a767b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56224a6e4e90 0x56224a6f67b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f072ca1e740 1 --2- 172.21.15.69:0/1901776174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56224a6e44e0 0x56224a6f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f070cc00640 0 client.26132 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:10.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f072ca1e740 1 -- 172.21.15.69:0/1901776174 >> 172.21.15.69:0/1901776174 conn(0x56224a6df230 msgr2=0x56224a6e0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:10.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f072ca1e740 1 -- 172.21.15.69:0/1901776174 shutdown_connections 2024-03-19T01:12:10.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f072ca1e740 1 -- 172.21.15.69:0/1901776174 wait complete. 2024-03-19T01:12:10.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.852+0000 7f072ca1e740 20 client.26132 trim_cache size 0 max 16384 2024-03-19T01:12:10.855 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:12:10.856 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:12:10.856 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "f00a8cef-ea6c-4ee6-81bb-639644686de8", 2024-03-19T01:12:10.856 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:12:10.856 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:12:10.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 -- 172.21.15.69:0/2810988632 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07080a2a90 msgr2=0x7f07080a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:10.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07080a2a90 0x7f07080a4f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0714004af0 tx=0x7f071402e040 comp rx=0 tx=0).stop 2024-03-19T01:12:10.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 -- 172.21.15.69:0/2810988632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07241033d0 msgr2=0x7f07241985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:10.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07241033d0 0x7f07241985e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f071000e9f0 tx=0x7f071000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 -- 172.21.15.69:0/2810988632 shutdown_connections 2024-03-19T01:12:10.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f072410bdf0 0x7f072419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07080a2a90 0x7f07080a4f50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07241033d0 0x7f07241985e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 --2- 172.21.15.69:0/2810988632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0724102ab0 0x7f07241980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:10.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 -- 172.21.15.69:0/2810988632 >> 172.21.15.69:0/2810988632 conn(0x7f0724077880 msgr2=0x7f07240fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:10.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 -- 172.21.15.69:0/2810988632 shutdown_connections 2024-03-19T01:12:10.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:10.853+0000 7f0729e00640 1 -- 172.21.15.69:0/2810988632 wait complete. 2024-03-19T01:12:10.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:12:10.875 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:12:11.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.092+0000 7fa693c00640 1 -- 172.21.15.69:0/3072307208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c1077e0 msgr2=0x7fa68c107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:11.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.092+0000 7fa693c00640 1 --2- 172.21.15.69:0/3072307208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c1077e0 0x7fa68c107bc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa680009930 tx=0x7fa68002bc30 comp rx=0 tx=0).stop 2024-03-19T01:12:11.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.092+0000 7fa693c00640 1 -- 172.21.15.69:0/3072307208 shutdown_connections 2024-03-19T01:12:11.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.092+0000 7fa693c00640 1 --2- 172.21.15.69:0/3072307208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c06a710 0x7fa68c10e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.092+0000 7fa693c00640 1 --2- 172.21.15.69:0/3072307208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c069d90 0x7fa68c06a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.092+0000 7fa693c00640 1 --2- 172.21.15.69:0/3072307208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c1077e0 0x7fa68c107bc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.092+0000 7fa693c00640 1 -- 172.21.15.69:0/3072307208 >> 172.21.15.69:0/3072307208 conn(0x7fa68c077700 msgr2=0x7fa68c077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:11.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.092+0000 7fa693c00640 1 -- 172.21.15.69:0/3072307208 shutdown_connections 2024-03-19T01:12:11.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.092+0000 7fa693c00640 1 -- 172.21.15.69:0/3072307208 wait complete. 2024-03-19T01:12:11.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.093+0000 7fa693c00640 1 Processor -- start 2024-03-19T01:12:11.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.093+0000 7fa693c00640 1 -- start start 2024-03-19T01:12:11.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa693c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c069d90 0x7fa68c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa693c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c06a710 0x7fa68c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa693c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c1077e0 0x7fa68c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa693c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c111c30 con 0x7fa68c069d90 2024-03-19T01:12:11.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa693c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c111ab0 con 0x7fa68c1077e0 2024-03-19T01:12:11.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa693c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c111db0 con 0x7fa68c06a710 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa691c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c1077e0 0x7fa68c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa68be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c06a710 0x7fa68c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa691200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c069d90 0x7fa68c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa68be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c06a710 0x7fa68c19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59042/0 (socket says 172.21.15.69:59042) 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa691c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c1077e0 0x7fa68c19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37850/0 (socket says 172.21.15.69:37850) 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.094+0000 7fa68be00640 1 -- 172.21.15.69:0/3255900890 learned_addr learned my addr 172.21.15.69:0/3255900890 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa68be00640 1 -- 172.21.15.69:0/3255900890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c1077e0 msgr2=0x7fa68c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa68be00640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c1077e0 0x7fa68c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa68be00640 1 -- 172.21.15.69:0/3255900890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c069d90 msgr2=0x7fa68c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa68be00640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c069d90 0x7fa68c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa68be00640 1 -- 172.21.15.69:0/3255900890 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68c19f2c0 con 0x7fa68c06a710 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa691200640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c069d90 0x7fa68c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa68be00640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c06a710 0x7fa68c19a810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa67800e9f0 tx=0x7fa67800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa689600640 1 -- 172.21.15.69:0/3255900890 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67800ccc0 con 0x7fa68c06a710 2024-03-19T01:12:11.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa689600640 1 -- 172.21.15.69:0/3255900890 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa678010910 con 0x7fa68c06a710 2024-03-19T01:12:11.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa693c00640 1 -- 172.21.15.69:0/3255900890 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68c19f5b0 con 0x7fa68c06a710 2024-03-19T01:12:11.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa689600640 1 -- 172.21.15.69:0/3255900890 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6780188a0 con 0x7fa68c06a710 2024-03-19T01:12:11.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.095+0000 7fa693c00640 1 -- 172.21.15.69:0/3255900890 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa68c1a6e40 con 0x7fa68c06a710 2024-03-19T01:12:11.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.098+0000 7fa69697e740 1 Processor -- start 2024-03-19T01:12:11.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.099+0000 7fa69697e740 1 -- start start 2024-03-19T01:12:11.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.099+0000 7fa69697e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc3d40 0x55bb34dc4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.099+0000 7fa69697e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dc4660 0x55bb34dccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.099+0000 7fa69697e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dcd240 0x55bb34dcf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.099+0000 7fa69697e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb34d282c0 con 0x55bb34dc4660 2024-03-19T01:12:11.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.099+0000 7fa69697e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb34cd0b60 con 0x55bb34dcd240 2024-03-19T01:12:11.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.099+0000 7fa69697e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bb34d24360 con 0x55bb34dc3d40 2024-03-19T01:12:11.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.099+0000 7fa691c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc3d40 0x55bb34dc4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.099+0000 7fa691c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc3d40 0x55bb34dc4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59048/0 (socket says 172.21.15.69:59048) 2024-03-19T01:12:11.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.099+0000 7fa691c00640 1 -- 172.21.15.69:0/802581237 learned_addr learned my addr 172.21.15.69:0/802581237 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:11.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.099+0000 7fa691200640 1 --2- 172.21.15.69:0/802581237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dc4660 0x55bb34dccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.100+0000 7fa691c00640 1 -- 172.21.15.69:0/802581237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dcd240 msgr2=0x55bb34dcf630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:12:11.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.100+0000 7fa691c00640 1 --2- 172.21.15.69:0/802581237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dcd240 0x55bb34dcf630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.100+0000 7fa691c00640 1 -- 172.21.15.69:0/802581237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dc4660 msgr2=0x55bb34dccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:11.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.100+0000 7fa691c00640 1 --2- 172.21.15.69:0/802581237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dc4660 0x55bb34dccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.100+0000 7fa691c00640 1 -- 172.21.15.69:0/802581237 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb34b067d0 con 0x55bb34dc3d40 2024-03-19T01:12:11.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.100+0000 7fa689600640 1 -- 172.21.15.69:0/3255900890 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 27) ==== 137523+0+0 (secure 0 0 0) 0x7fa678010ac0 con 0x7fa68c06a710 2024-03-19T01:12:11.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.100+0000 7fa691c00640 1 --2- 172.21.15.69:0/802581237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc3d40 0x55bb34dc4120 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa684009f60 tx=0x7fa684009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c4a4ebd73c67c6b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:11.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.100+0000 7fa67e000640 1 -- 172.21.15.69:0/802581237 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68401e070 con 0x55bb34dc3d40 2024-03-19T01:12:11.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.100+0000 7fa67e000640 1 -- 172.21.15.69:0/802581237 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa684004030 con 0x55bb34dc3d40 2024-03-19T01:12:11.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.100+0000 7fa67e000640 1 -- 172.21.15.69:0/802581237 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68400f6c0 con 0x55bb34dc3d40 2024-03-19T01:12:11.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.101+0000 7fa689600640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6740a2b10 0x7fa6740a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.101+0000 7fa69697e740 1 -- 172.21.15.69:0/802581237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc3d40 msgr2=0x55bb34dc4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.101+0000 7fa69697e740 1 --2- 172.21.15.69:0/802581237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc3d40 0x55bb34dc4120 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa684009f60 tx=0x7fa684009f90 comp rx=0 tx=0).stop 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.101+0000 7fa689600640 1 -- 172.21.15.69:0/3255900890 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa6780cf8c0 con 0x7fa68c06a710 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.101+0000 7fa69697e740 1 -- 172.21.15.69:0/802581237 shutdown_connections 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.101+0000 7fa69697e740 1 --2- 172.21.15.69:0/802581237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dcd240 0x55bb34dcf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.101+0000 7fa69697e740 1 --2- 172.21.15.69:0/802581237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dc4660 0x55bb34dccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.101+0000 7fa691200640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6740a2b10 0x7fa6740a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.101+0000 7fa69697e740 1 --2- 172.21.15.69:0/802581237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc3d40 0x55bb34dc4120 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.102+0000 7fa69697e740 1 -- 172.21.15.69:0/802581237 >> 172.21.15.69:0/802581237 conn(0x55bb34dbe470 msgr2=0x55bb34dbe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.102+0000 7fa69697e740 1 -- 172.21.15.69:0/802581237 shutdown_connections 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.102+0000 7fa69697e740 1 -- 172.21.15.69:0/802581237 wait complete. 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.102+0000 7fa691200640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6740a2b10 0x7fa6740a4fd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa680009900 tx=0x7fa68002df90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.103+0000 7fa69697e740 1 -- start start 2024-03-19T01:12:11.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.103+0000 7fa69697e740 1 Processor -- start 2024-03-19T01:12:11.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa69697e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34dca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa69697e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 0x55bb34dca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa69697e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dcd240 0x55bb34dcae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa69697e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb34dd0030 con 0x55bb34dcd240 2024-03-19T01:12:11.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa69697e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb34ce3fc0 con 0x55bb34dc3d40 2024-03-19T01:12:11.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa69697e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bb34dd01b0 con 0x55bb34dc4660 2024-03-19T01:12:11.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa69697e740 10 client.0 _ll_register_callbacks cb 0x55bb34ce3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:11.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa691200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34dca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa68be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 0x55bb34dca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa68be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 0x55bb34dca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59052/0 (socket says 172.21.15.69:59052) 2024-03-19T01:12:11.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa68be00640 1 -- 172.21.15.69:0/795912038 learned_addr learned my addr 172.21.15.69:0/795912038 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:11.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.104+0000 7fa691c00640 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dcd240 0x55bb34dcae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.105+0000 7fa68be00640 1 -- 172.21.15.69:0/795912038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 msgr2=0x55bb34dca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:11.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.105+0000 7fa68be00640 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34dca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.105+0000 7fa68be00640 1 -- 172.21.15.69:0/795912038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dcd240 msgr2=0x55bb34dcae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:11.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.105+0000 7fa68be00640 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dcd240 0x55bb34dcae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.105+0000 7fa68be00640 1 -- 172.21.15.69:0/795912038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb34a3c3f0 con 0x55bb34dc4660 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.105+0000 7fa68be00640 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 0x55bb34dca8e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa67800c6a0 tx=0x7fa67809c070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf9311ec4307ff server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.105+0000 7fa672a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.105+0000 7fa672a00640 1 -- 172.21.15.69:0/795912038 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6780862b0 con 0x55bb34dc4660 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.105+0000 7fa672a00640 1 -- 172.21.15.69:0/795912038 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa6780cf460 con 0x55bb34dc4660 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.105+0000 7fa69697e740 1 -- 172.21.15.69:0/795912038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa684004030 con 0x55bb34dc4660 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.105+0000 7fa672a00640 1 -- 172.21.15.69:0/795912038 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6780a4050 con 0x55bb34dc4660 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.106+0000 7fa69697e740 10 client.26128 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.106+0000 7fa69697e740 1 -- 172.21.15.69:0/795912038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb349fc0f0 con 0x55bb34dc4660 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.106+0000 7fa672a00640 1 -- 172.21.15.69:0/795912038 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa684004030 con 0x55bb34dc4660 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.106+0000 7fa672a00640 1 -- 172.21.15.69:0/795912038 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fa678097600 con 0x55bb34dc4660 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.106+0000 7fa69697e740 10 client.26128 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.106+0000 7fa69697e740 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:11.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.106+0000 7fa69697e740 20 client.26128 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:11.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.106+0000 7fa69697e740 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb34de6030 0x55bb34de8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.106+0000 7fa69697e740 4 client.26128 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:11.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.106+0000 7fa69697e740 1 -- 172.21.15.69:0/795912038 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb34cc9610 con 0x55bb34de6030 2024-03-19T01:12:11.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.106+0000 7fa691200640 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb34de6030 0x55bb34de8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.107+0000 7fa691200640 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb34de6030 0x55bb34de8420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802d6efc4cef999 server_cookie=d29983476e90f1ae in_seq=0 out_seq=0 2024-03-19T01:12:11.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.107+0000 7fa672a00640 10 client.26128 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:11.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa672a00640 1 -- 172.21.15.69:0/795912038 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bb34cc9610 con 0x55bb34de6030 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa672a00640 10 client.26128 handle_command_reply: tid=0 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 1 client.26128 shutdown 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 1 -- 172.21.15.69:0/795912038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 msgr2=0x55bb34dca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 0x55bb34dca8e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa67800c6a0 tx=0x7fa67809c070 comp rx=0 tx=0).stop 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 1 -- 172.21.15.69:0/795912038 shutdown_connections 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb34de6030 0x55bb34de8420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dcd240 0x55bb34dcae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 0x55bb34dca8e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa672a00640 0 client.26128 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 1 --2- 172.21.15.69:0/795912038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34dca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 1 -- 172.21.15.69:0/795912038 >> 172.21.15.69:0/795912038 conn(0x55bb34dbe470 msgr2=0x55bb34dc01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 1 -- 172.21.15.69:0/795912038 shutdown_connections 2024-03-19T01:12:11.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 1 -- 172.21.15.69:0/795912038 wait complete. 2024-03-19T01:12:11.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.109+0000 7fa69697e740 20 client.26128 trim_cache size 0 max 16384 2024-03-19T01:12:11.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.136+0000 7fa69697e740 1 Processor -- start 2024-03-19T01:12:11.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.136+0000 7fa69697e740 1 -- start start 2024-03-19T01:12:11.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa69697e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34dccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa69697e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dc4660 0x55bb34dc92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa69697e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dcd240 0x55bb34de1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa69697e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c1acaa0 con 0x55bb34dc4660 2024-03-19T01:12:11.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa69697e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c1ae670 con 0x55bb34dc3d40 2024-03-19T01:12:11.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa69697e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c1ad170 con 0x55bb34dcd240 2024-03-19T01:12:11.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa691200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34dccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa691c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dcd240 0x55bb34de1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa68be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dc4660 0x55bb34dc92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa691200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34dccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37866/0 (socket says 172.21.15.69:37866) 2024-03-19T01:12:11.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa691c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dcd240 0x55bb34de1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59066/0 (socket says 172.21.15.69:59066) 2024-03-19T01:12:11.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa691200640 1 -- 172.21.15.69:0/3532781465 learned_addr learned my addr 172.21.15.69:0/3532781465 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:11.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa691c00640 1 -- 172.21.15.69:0/3532781465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 msgr2=0x55bb34dccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:11.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa691c00640 1 --2- 172.21.15.69:0/3532781465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34dccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa691c00640 1 -- 172.21.15.69:0/3532781465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dc4660 msgr2=0x55bb34dc92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:11.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa691c00640 1 --2- 172.21.15.69:0/3532781465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dc4660 0x55bb34dc92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.137+0000 7fa691c00640 1 -- 172.21.15.69:0/3532781465 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb34dcf960 con 0x55bb34dcd240 2024-03-19T01:12:11.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.138+0000 7fa691c00640 1 --2- 172.21.15.69:0/3532781465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dcd240 0x55bb34de1660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa684011d60 tx=0x7fa684004990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12af8d973c33c91d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:11.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.138+0000 7fa67d600640 1 -- 172.21.15.69:0/3532781465 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68401e040 con 0x55bb34dcd240 2024-03-19T01:12:11.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.138+0000 7fa67d600640 1 -- 172.21.15.69:0/3532781465 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa684003970 con 0x55bb34dcd240 2024-03-19T01:12:11.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.138+0000 7fa67d600640 1 -- 172.21.15.69:0/3532781465 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6840046b0 con 0x55bb34dcd240 2024-03-19T01:12:11.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.139+0000 7fa69697e740 1 -- 172.21.15.69:0/3532781465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dcd240 msgr2=0x55bb34de1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:11.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.139+0000 7fa69697e740 1 --2- 172.21.15.69:0/3532781465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dcd240 0x55bb34de1660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa684011d60 tx=0x7fa684004990 comp rx=0 tx=0).stop 2024-03-19T01:12:11.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.139+0000 7fa69697e740 1 -- 172.21.15.69:0/3532781465 shutdown_connections 2024-03-19T01:12:11.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.139+0000 7fa69697e740 1 --2- 172.21.15.69:0/3532781465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dcd240 0x55bb34de1660 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.139+0000 7fa69697e740 1 --2- 172.21.15.69:0/3532781465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dc4660 0x55bb34dc92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.139+0000 7fa69697e740 1 --2- 172.21.15.69:0/3532781465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34dccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.139+0000 7fa69697e740 1 -- 172.21.15.69:0/3532781465 >> 172.21.15.69:0/3532781465 conn(0x55bb34dbe470 msgr2=0x55bb34dbd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:11.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.139+0000 7fa69697e740 1 -- 172.21.15.69:0/3532781465 shutdown_connections 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.139+0000 7fa69697e740 1 -- 172.21.15.69:0/3532781465 wait complete. 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.140+0000 7fa69697e740 1 -- start start 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.140+0000 7fa69697e740 1 Processor -- start 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa69697e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34e454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa69697e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 0x55bb34e45a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa69697e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dcd240 0x55bb34dd9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa69697e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c1adf10 con 0x55bb34dcd240 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa69697e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c1ad840 con 0x55bb34dc3d40 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa69697e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c1aaaa0 con 0x55bb34dc4660 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa69697e740 10 client.0 _ll_register_callbacks cb 0x7fa67809b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa691200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34e454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa68be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 0x55bb34e45a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa691200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34e454f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37872/0 (socket says 172.21.15.69:37872) 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa691200640 1 -- 172.21.15.69:0/3376562785 learned_addr learned my addr 172.21.15.69:0/3376562785 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.141+0000 7fa691c00640 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dcd240 0x55bb34dd9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa68be00640 1 -- 172.21.15.69:0/3376562785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 msgr2=0x55bb34e454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa68be00640 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34e454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa68be00640 1 -- 172.21.15.69:0/3376562785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dcd240 msgr2=0x55bb34dd9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa68be00640 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dcd240 0x55bb34dd9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa68be00640 1 -- 172.21.15.69:0/3376562785 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb34dda470 con 0x55bb34dc4660 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa691c00640 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dcd240 0x55bb34dd9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa691200640 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34e454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa68be00640 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 0x55bb34e45a30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa678099b60 tx=0x7fa6780097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb39718e03ba50e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa672a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa672a00640 1 -- 172.21.15.69:0/3376562785 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67809b3f0 con 0x55bb34dc4660 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa69697e740 1 -- 172.21.15.69:0/3376562785 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa684003970 con 0x55bb34dc4660 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa672a00640 1 -- 172.21.15.69:0/3376562785 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa6780cf660 con 0x55bb34dc4660 2024-03-19T01:12:11.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.142+0000 7fa672a00640 1 -- 172.21.15.69:0/3376562785 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa678097b50 con 0x55bb34dc4660 2024-03-19T01:12:11.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.143+0000 7fa69697e740 10 client.26134 fetch_fsmap learned FSMap version 30 2024-03-19T01:12:11.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.143+0000 7fa69697e740 1 -- 172.21.15.69:0/3376562785 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb34e4cc50 con 0x55bb34dc4660 2024-03-19T01:12:11.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.143+0000 7fa672a00640 1 -- 172.21.15.69:0/3376562785 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa684003970 con 0x55bb34dc4660 2024-03-19T01:12:11.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.143+0000 7fa672a00640 1 -- 172.21.15.69:0/3376562785 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fa67809a840 con 0x55bb34dc4660 2024-03-19T01:12:11.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.143+0000 7fa69697e740 10 client.26134 fetch_fsmap finished waiting for FSMap version 30 2024-03-19T01:12:11.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.143+0000 7fa69697e740 10 client.26134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:11.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.143+0000 7fa69697e740 20 client.26134 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:11.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.143+0000 7fa69697e740 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb34e4d290 0x55bb34e4f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:11.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.143+0000 7fa69697e740 4 client.26134 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:12:11.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.143+0000 7fa69697e740 1 -- 172.21.15.69:0/3376562785 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb34d35800 con 0x55bb34e4d290 2024-03-19T01:12:11.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.143+0000 7fa691200640 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb34e4d290 0x55bb34e4f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:11.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.144+0000 7fa672a00640 10 client.26134 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.144+0000 7fa691200640 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb34e4d290 0x55bb34e4f6b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5082987599c58ba5 server_cookie=d328882947680195 in_seq=0 out_seq=0 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.146+0000 7fa672a00640 1 -- 172.21.15.69:0/3376562785 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55bb34d35800 con 0x55bb34e4d290 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.146+0000 7fa672a00640 10 client.26134 handle_command_reply: tid=0 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.146+0000 7fa69697e740 1 client.26134 shutdown 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.146+0000 7fa69697e740 1 -- 172.21.15.69:0/3376562785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 msgr2=0x55bb34e45a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.146+0000 7fa69697e740 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 0x55bb34e45a30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa678099b60 tx=0x7fa6780097a0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.147+0000 7fa69697e740 1 -- 172.21.15.69:0/3376562785 shutdown_connections 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.147+0000 7fa69697e740 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb34e4d290 0x55bb34e4f6b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.147+0000 7fa69697e740 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb34dcd240 0x55bb34dd9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.147+0000 7fa69697e740 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb34dc4660 0x55bb34e45a30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.147+0000 7fa672a00640 0 client.26134 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.147+0000 7fa69697e740 1 --2- 172.21.15.69:0/3376562785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb34dc3d40 0x55bb34e454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.147+0000 7fa69697e740 1 -- 172.21.15.69:0/3376562785 >> 172.21.15.69:0/3376562785 conn(0x55bb34dbe470 msgr2=0x55bb34dbf770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:11.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.147+0000 7fa69697e740 1 -- 172.21.15.69:0/3376562785 shutdown_connections 2024-03-19T01:12:11.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.147+0000 7fa69697e740 1 -- 172.21.15.69:0/3376562785 wait complete. 2024-03-19T01:12:11.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.147+0000 7fa69697e740 20 client.26134 trim_cache size 0 max 16384 2024-03-19T01:12:11.152 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:12:11.152 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (580 inodes in the stack)", 2024-03-19T01:12:11.152 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:12:11.152 INFO:teuthology.orchestra.run.smithi069.stdout: "f00a8cef-ea6c-4ee6-81bb-639644686de8": { 2024-03-19T01:12:11.152 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:12:11.152 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "f00a8cef-ea6c-4ee6-81bb-639644686de8", 2024-03-19T01:12:11.152 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:12:11.152 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:12:11.152 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:12:11.153 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:12:11.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 -- 172.21.15.69:0/3255900890 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6740a2b10 msgr2=0x7fa6740a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:11.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6740a2b10 0x7fa6740a4fd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa680009900 tx=0x7fa68002df90 comp rx=0 tx=0).stop 2024-03-19T01:12:11.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 -- 172.21.15.69:0/3255900890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c06a710 msgr2=0x7fa68c19a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:11.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c06a710 0x7fa68c19a810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa67800e9f0 tx=0x7fa67800edf0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 -- 172.21.15.69:0/3255900890 shutdown_connections 2024-03-19T01:12:11.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6740a2b10 0x7fa6740a4fd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa68c1077e0 0x7fa68c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa68c06a710 0x7fa68c19a810 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 --2- 172.21.15.69:0/3255900890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa68c069d90 0x7fa68c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:11.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 -- 172.21.15.69:0/3255900890 >> 172.21.15.69:0/3255900890 conn(0x7fa68c077700 msgr2=0x7fa68c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:11.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 -- 172.21.15.69:0/3255900890 shutdown_connections 2024-03-19T01:12:11.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:11.148+0000 7fa693c00640 1 -- 172.21.15.69:0/3255900890 wait complete. 2024-03-19T01:12:11.166 INFO:tasks.cephfs.filesystem:scrub status for tag:f00a8cef-ea6c-4ee6-81bb-639644686de8 - {'path': '/', 'tag': 'f00a8cef-ea6c-4ee6-81bb-639644686de8', 'options': 'recursive,force'} 2024-03-19T01:12:11.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:11.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:11.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:11.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:11.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.712 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:12:12.712 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.712 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:12.712 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:12.712 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.712 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:12.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:12.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:12.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:12.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.713 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:12 smithi163 ceph-mon[67428]: pgmap v49: 129 pgs: 129 active+clean; 155 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.9 MiB/s wr, 135 op/s 2024-03-19T01:12:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:12:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:12 smithi086 ceph-mon[66829]: pgmap v49: 129 pgs: 129 active+clean; 155 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.9 MiB/s wr, 135 op/s 2024-03-19T01:12:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:12:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:12 smithi069 ceph-mon[31727]: pgmap v49: 129 pgs: 129 active+clean; 155 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.9 MiB/s wr, 135 op/s 2024-03-19T01:12:13.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:13 smithi086 ceph-mon[66829]: mgrmap e28: x(active, since 92s), standbys: y, z 2024-03-19T01:12:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:13 smithi069 ceph-mon[31727]: mgrmap e28: x(active, since 92s), standbys: y, z 2024-03-19T01:12:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:13 smithi163 ceph-mon[67428]: mgrmap e28: x(active, since 92s), standbys: y, z 2024-03-19T01:12:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:14 smithi086 ceph-mon[66829]: pgmap v50: 129 pgs: 129 active+clean; 155 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 111 op/s 2024-03-19T01:12:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:14 smithi069 ceph-mon[31727]: pgmap v50: 129 pgs: 129 active+clean; 155 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 111 op/s 2024-03-19T01:12:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:14 smithi163 ceph-mon[67428]: pgmap v50: 129 pgs: 129 active+clean; 155 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 111 op/s 2024-03-19T01:12:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:14.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:15.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:15.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:15.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:15.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:15.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:16.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:16 smithi086 ceph-mon[66829]: pgmap v51: 129 pgs: 129 active+clean; 159 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 127 op/s 2024-03-19T01:12:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:16 smithi069 ceph-mon[31727]: pgmap v51: 129 pgs: 129 active+clean; 159 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 127 op/s 2024-03-19T01:12:16.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:16 smithi163 ceph-mon[67428]: pgmap v51: 129 pgs: 129 active+clean; 159 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 127 op/s 2024-03-19T01:12:18.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:18 smithi086 ceph-mon[66829]: pgmap v52: 129 pgs: 129 active+clean; 160 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 138 op/s 2024-03-19T01:12:18.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:18 smithi069 ceph-mon[31727]: pgmap v52: 129 pgs: 129 active+clean; 160 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 138 op/s 2024-03-19T01:12:18.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:18 smithi163 ceph-mon[67428]: pgmap v52: 129 pgs: 129 active+clean; 160 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 138 op/s 2024-03-19T01:12:20.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:20 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:12:20.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:20 smithi086 ceph-mon[66829]: pgmap v53: 129 pgs: 129 active+clean; 160 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 95 op/s 2024-03-19T01:12:20.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:20 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:12:20.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:20 smithi069 ceph-mon[31727]: pgmap v53: 129 pgs: 129 active+clean; 160 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 95 op/s 2024-03-19T01:12:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:20 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:12:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:20 smithi163 ceph-mon[67428]: pgmap v53: 129 pgs: 129 active+clean; 160 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 95 op/s 2024-03-19T01:12:22.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:22 smithi086 ceph-mon[66829]: pgmap v54: 129 pgs: 129 active+clean; 166 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 117 op/s 2024-03-19T01:12:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:22 smithi069 ceph-mon[31727]: pgmap v54: 129 pgs: 129 active+clean; 166 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 117 op/s 2024-03-19T01:12:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:22 smithi163 ceph-mon[67428]: pgmap v54: 129 pgs: 129 active+clean; 166 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 117 op/s 2024-03-19T01:12:24.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:23 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:12:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:23 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:12:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:23 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:12:25.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:24 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:12:25.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:24 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:12:25.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:24 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:12:25.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:24 smithi086 ceph-mon[66829]: pgmap v55: 129 pgs: 129 active+clean; 166 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-03-19T01:12:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:24 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:12:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:24 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:12:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:24 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:12:25.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:24 smithi069 ceph-mon[31727]: pgmap v55: 129 pgs: 129 active+clean; 166 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-03-19T01:12:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:24 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:12:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:24 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:12:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:24 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:12:25.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:24 smithi163 ceph-mon[67428]: pgmap v55: 129 pgs: 129 active+clean; 166 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-03-19T01:12:26.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:26 smithi086 ceph-mon[66829]: pgmap v56: 129 pgs: 129 active+clean; 169 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-03-19T01:12:26.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:26 smithi069 ceph-mon[31727]: pgmap v56: 129 pgs: 129 active+clean; 169 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-03-19T01:12:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:26 smithi163 ceph-mon[67428]: pgmap v56: 129 pgs: 129 active+clean; 169 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-03-19T01:12:28.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:28 smithi086 ceph-mon[66829]: pgmap v57: 129 pgs: 129 active+clean; 169 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.0 MiB/s wr, 57 op/s 2024-03-19T01:12:28.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:28 smithi069 ceph-mon[31727]: pgmap v57: 129 pgs: 129 active+clean; 169 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.0 MiB/s wr, 57 op/s 2024-03-19T01:12:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:28 smithi163 ceph-mon[67428]: pgmap v57: 129 pgs: 129 active+clean; 169 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.0 MiB/s wr, 57 op/s 2024-03-19T01:12:30.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:30 smithi086 ceph-mon[66829]: pgmap v58: 129 pgs: 129 active+clean; 169 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 885 KiB/s wr, 39 op/s 2024-03-19T01:12:30.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:30 smithi069 ceph-mon[31727]: pgmap v58: 129 pgs: 129 active+clean; 169 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 885 KiB/s wr, 39 op/s 2024-03-19T01:12:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:30 smithi163 ceph-mon[67428]: pgmap v58: 129 pgs: 129 active+clean; 169 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 885 KiB/s wr, 39 op/s 2024-03-19T01:12:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:32 smithi086 ceph-mon[66829]: pgmap v59: 129 pgs: 129 active+clean; 171 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.1 MiB/s wr, 57 op/s 2024-03-19T01:12:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:32 smithi069 ceph-mon[31727]: pgmap v59: 129 pgs: 129 active+clean; 171 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.1 MiB/s wr, 57 op/s 2024-03-19T01:12:32.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:32 smithi163 ceph-mon[67428]: pgmap v59: 129 pgs: 129 active+clean; 171 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.1 MiB/s wr, 57 op/s 2024-03-19T01:12:34.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:34 smithi086 ceph-mon[66829]: pgmap v60: 129 pgs: 129 active+clean; 171 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 527 KiB/s wr, 35 op/s 2024-03-19T01:12:34.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:34 smithi069 ceph-mon[31727]: pgmap v60: 129 pgs: 129 active+clean; 171 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 527 KiB/s wr, 35 op/s 2024-03-19T01:12:34.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:34 smithi163 ceph-mon[67428]: pgmap v60: 129 pgs: 129 active+clean; 171 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 527 KiB/s wr, 35 op/s 2024-03-19T01:12:36.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:36 smithi086 ceph-mon[66829]: pgmap v61: 129 pgs: 129 active+clean; 178 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-19T01:12:36.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:36 smithi069 ceph-mon[31727]: pgmap v61: 129 pgs: 129 active+clean; 178 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-19T01:12:36.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:36 smithi163 ceph-mon[67428]: pgmap v61: 129 pgs: 129 active+clean; 178 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-19T01:12:38.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:38 smithi086 ceph-mon[66829]: pgmap v62: 129 pgs: 129 active+clean; 180 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.1 MiB/s wr, 87 op/s 2024-03-19T01:12:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:38 smithi069 ceph-mon[31727]: pgmap v62: 129 pgs: 129 active+clean; 180 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.1 MiB/s wr, 87 op/s 2024-03-19T01:12:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:38 smithi163 ceph-mon[67428]: pgmap v62: 129 pgs: 129 active+clean; 180 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.1 MiB/s wr, 87 op/s 2024-03-19T01:12:40.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:40 smithi086 ceph-mon[66829]: pgmap v63: 129 pgs: 129 active+clean; 180 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.0 MiB/s wr, 80 op/s 2024-03-19T01:12:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:40 smithi069 ceph-mon[31727]: pgmap v63: 129 pgs: 129 active+clean; 180 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.0 MiB/s wr, 80 op/s 2024-03-19T01:12:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:40 smithi163 ceph-mon[67428]: pgmap v63: 129 pgs: 129 active+clean; 180 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.0 MiB/s wr, 80 op/s 2024-03-19T01:12:41.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:12:41.167 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:12:41.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.388+0000 7f5337e00640 1 -- 172.21.15.69:0/3951360580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5338102040 msgr2=0x7f533810e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.388+0000 7f5337e00640 1 --2- 172.21.15.69:0/3951360580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5338102040 0x7f533810e550 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5330009ca0 tx=0x7f533002c100 comp rx=0 tx=0).stop 2024-03-19T01:12:41.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.388+0000 7f5337e00640 1 -- 172.21.15.69:0/3951360580 shutdown_connections 2024-03-19T01:12:41.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.388+0000 7f5337e00640 1 --2- 172.21.15.69:0/3951360580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5338102040 0x7f533810e550 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.388+0000 7f5337e00640 1 --2- 172.21.15.69:0/3951360580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5338101680 0x7f5338101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.388+0000 7f5337e00640 1 --2- 172.21.15.69:0/3951360580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5338107690 0x7f5338107a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.388+0000 7f5337e00640 1 -- 172.21.15.69:0/3951360580 >> 172.21.15.69:0/3951360580 conn(0x7f53380fd190 msgr2=0x7f53380ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:41.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.388+0000 7f5337e00640 1 -- 172.21.15.69:0/3951360580 shutdown_connections 2024-03-19T01:12:41.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.388+0000 7f5337e00640 1 -- 172.21.15.69:0/3951360580 wait complete. 2024-03-19T01:12:41.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.389+0000 7f5337e00640 1 Processor -- start 2024-03-19T01:12:41.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.389+0000 7f5337e00640 1 -- start start 2024-03-19T01:12:41.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5338101680 0x7f5338073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5338102040 0x7f5338074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5338107690 0x7f53380745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5338111a60 con 0x7f5338102040 2024-03-19T01:12:41.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53381118e0 con 0x7f5338107690 2024-03-19T01:12:41.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5338111be0 con 0x7f5338101680 2024-03-19T01:12:41.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f532fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5338102040 0x7f5338074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5336a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5338101680 0x7f5338073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5338107690 0x7f53380745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5336a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5338101680 0x7f5338073b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53760/0 (socket says 172.21.15.69:53760) 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5336a00640 1 -- 172.21.15.69:0/3920389323 learned_addr learned my addr 172.21.15.69:0/3920389323 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337400640 1 -- 172.21.15.69:0/3920389323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5338101680 msgr2=0x7f5338073b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337400640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5338101680 0x7f5338073b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337400640 1 -- 172.21.15.69:0/3920389323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5338102040 msgr2=0x7f5338074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337400640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5338102040 0x7f5338074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5337400640 1 -- 172.21.15.69:0/3920389323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53381a63c0 con 0x7f5338107690 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f5336a00640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5338101680 0x7f5338073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.390+0000 7f532fe00640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5338102040 0x7f5338074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.391+0000 7f5337400640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5338107690 0x7f53380745a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5330009bf0 tx=0x7f5330034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.391+0000 7f532f400640 1 -- 172.21.15.69:0/3920389323 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330039050 con 0x7f5338107690 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.391+0000 7f5337e00640 1 -- 172.21.15.69:0/3920389323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53381a6650 con 0x7f5338107690 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.391+0000 7f532f400640 1 -- 172.21.15.69:0/3920389323 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5330045070 con 0x7f5338107690 2024-03-19T01:12:41.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.391+0000 7f532f400640 1 -- 172.21.15.69:0/3920389323 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53300044c0 con 0x7f5338107690 2024-03-19T01:12:41.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.391+0000 7f5337e00640 1 -- 172.21.15.69:0/3920389323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53381a6b10 con 0x7f5338107690 2024-03-19T01:12:41.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.394+0000 7f532f400640 1 -- 172.21.15.69:0/3920389323 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f5330004660 con 0x7f5338107690 2024-03-19T01:12:41.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.394+0000 7f5340780740 1 Processor -- start 2024-03-19T01:12:41.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.395+0000 7f5340780740 1 -- start start 2024-03-19T01:12:41.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.395+0000 7f5340780740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a02235f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.395+0000 7f5340780740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235f850 0x55a022367ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.395+0000 7f5340780740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a02236a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.395+0000 7f5340780740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a0222c32c0 con 0x55a02235f850 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.395+0000 7f5340780740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a02226bb60 con 0x55a02235ef30 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.395+0000 7f5340780740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a0222bf360 con 0x55a022368430 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.395+0000 7f532fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a02235f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.395+0000 7f5336a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235f850 0x55a022367ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.395+0000 7f532fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a02235f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36352/0 (socket says 172.21.15.69:36352) 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.395+0000 7f532fe00640 1 -- 172.21.15.69:0/1021841811 learned_addr learned my addr 172.21.15.69:0/1021841811 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.396+0000 7f532fe00640 1 --2- 172.21.15.69:0/1021841811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a02236a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.396+0000 7f532f400640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f531c0a2970 0x7f531c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.396+0000 7f532fe00640 1 -- 172.21.15.69:0/1021841811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 msgr2=0x55a02236a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:41.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.396+0000 7f532fe00640 1 --2- 172.21.15.69:0/1021841811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a02236a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.396+0000 7f532fe00640 1 -- 172.21.15.69:0/1021841811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235f850 msgr2=0x55a022367ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:41.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.396+0000 7f532fe00640 1 --2- 172.21.15.69:0/1021841811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235f850 0x55a022367ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.396+0000 7f532f400640 1 -- 172.21.15.69:0/3920389323 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f53300f1690 con 0x7f5338107690 2024-03-19T01:12:41.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.396+0000 7f532fe00640 1 -- 172.21.15.69:0/1021841811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0220a17d0 con 0x55a02235ef30 2024-03-19T01:12:41.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.396+0000 7f5336a00640 1 --2- 172.21.15.69:0/1021841811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235f850 0x55a022367ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:41.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.396+0000 7f5337400640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f531c0a2970 0x7f531c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.396+0000 7f532fe00640 1 --2- 172.21.15.69:0/1021841811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a02235f310 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f53200119f0 tx=0x7f5320009d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7da78c0d0f42282 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:41.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5327e00640 1 -- 172.21.15.69:0/1021841811 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532001a070 con 0x55a02235ef30 2024-03-19T01:12:41.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5327e00640 1 -- 172.21.15.69:0/1021841811 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f532000fe20 con 0x55a02235ef30 2024-03-19T01:12:41.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5327e00640 1 -- 172.21.15.69:0/1021841811 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53200047a0 con 0x55a02235ef30 2024-03-19T01:12:41.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5340780740 1 -- 172.21.15.69:0/1021841811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 msgr2=0x55a02235f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5340780740 1 --2- 172.21.15.69:0/1021841811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a02235f310 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f53200119f0 tx=0x7f5320009d10 comp rx=0 tx=0).stop 2024-03-19T01:12:41.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5337400640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f531c0a2970 0x7f531c0a4e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f53300b9e40 tx=0x7f53300095c0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5340780740 1 -- 172.21.15.69:0/1021841811 shutdown_connections 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5340780740 1 --2- 172.21.15.69:0/1021841811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a02236a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5340780740 1 --2- 172.21.15.69:0/1021841811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235f850 0x55a022367ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5340780740 1 --2- 172.21.15.69:0/1021841811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a02235f310 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5340780740 1 -- 172.21.15.69:0/1021841811 >> 172.21.15.69:0/1021841811 conn(0x55a022359660 msgr2=0x55a022359a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.397+0000 7f5340780740 1 -- 172.21.15.69:0/1021841811 shutdown_connections 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.398+0000 7f5340780740 1 -- 172.21.15.69:0/1021841811 wait complete. 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.398+0000 7f5340780740 1 -- start start 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.398+0000 7f5340780740 1 Processor -- start 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5340780740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235ef30 0x55a022365500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5340780740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022365a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5340780740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a022365f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5340780740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a02236b220 con 0x55a02235ef30 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5340780740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a02227efc0 con 0x55a02235f850 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5340780740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a02236b3a0 con 0x55a022368430 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5340780740 10 client.0 _ll_register_callbacks cb 0x55a02227e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5336a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022365a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5336a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022365a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36358/0 (socket says 172.21.15.69:36358) 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5336a00640 1 -- 172.21.15.69:0/2694097142 learned_addr learned my addr 172.21.15.69:0/2694097142 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f532fe00640 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235ef30 0x55a022365500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f532fe00640 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a022365f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5336a00640 1 -- 172.21.15.69:0/2694097142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 msgr2=0x55a022365f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5336a00640 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a022365f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5336a00640 1 -- 172.21.15.69:0/2694097142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235ef30 msgr2=0x55a022365500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5336a00640 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235ef30 0x55a022365500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f5336a00640 1 -- 172.21.15.69:0/2694097142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a021fd73f0 con 0x55a02235f850 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.399+0000 7f532fe00640 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a022365f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:12:41.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.400+0000 7f5336a00640 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022365a40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5328012d20 tx=0x7f532800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9eddae1b5c65b5c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:41.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.400+0000 7f5324c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:41.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.400+0000 7f5324c00640 1 -- 172.21.15.69:0/2694097142 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328010430 con 0x55a02235f850 2024-03-19T01:12:41.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.400+0000 7f5340780740 1 -- 172.21.15.69:0/2694097142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f532000fe20 con 0x55a02235f850 2024-03-19T01:12:41.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.400+0000 7f5324c00640 1 -- 172.21.15.69:0/2694097142 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f53280105d0 con 0x55a02235f850 2024-03-19T01:12:41.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.400+0000 7f5324c00640 1 -- 172.21.15.69:0/2694097142 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f53280107d0 con 0x55a02235f850 2024-03-19T01:12:41.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.400+0000 7f5340780740 10 client.26159 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:41.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.400+0000 7f5340780740 1 -- 172.21.15.69:0/2694097142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a021f970f0 con 0x55a02235f850 2024-03-19T01:12:41.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.401+0000 7f5324c00640 1 -- 172.21.15.69:0/2694097142 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f532000fe20 con 0x55a02235f850 2024-03-19T01:12:41.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.401+0000 7f5324c00640 1 -- 172.21.15.69:0/2694097142 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f532801c860 con 0x55a02235f850 2024-03-19T01:12:41.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.401+0000 7f5340780740 10 client.26159 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:41.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.401+0000 7f5340780740 10 client.26159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:41.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.401+0000 7f5340780740 20 client.26159 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:41.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.401+0000 7f5340780740 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a022378890 0x55a02237ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.401+0000 7f5340780740 4 client.26159 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:41.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.401+0000 7f5340780740 1 -- 172.21.15.69:0/2694097142 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a022264610 con 0x55a022378890 2024-03-19T01:12:41.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.401+0000 7f532fe00640 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a022378890 0x55a02237ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.402+0000 7f532fe00640 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a022378890 0x55a02237ad10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a09c2526556f19 server_cookie=61a594a2f4edcae0 in_seq=0 out_seq=0 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.402+0000 7f5324c00640 10 client.26159 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.403+0000 7f5324c00640 1 -- 172.21.15.69:0/2694097142 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a022264610 con 0x55a022378890 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.403+0000 7f5324c00640 10 client.26159 handle_command_reply: tid=0 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.403+0000 7f5340780740 1 client.26159 shutdown 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5340780740 1 -- 172.21.15.69:0/2694097142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 msgr2=0x55a022365a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5340780740 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022365a40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5328012d20 tx=0x7f532800f790 comp rx=0 tx=0).stop 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5340780740 1 -- 172.21.15.69:0/2694097142 shutdown_connections 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5340780740 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a022378890 0x55a02237ad10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5340780740 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a022365f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5340780740 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022365a40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5324c00640 0 client.26159 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5340780740 1 --2- 172.21.15.69:0/2694097142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235ef30 0x55a022365500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5340780740 1 -- 172.21.15.69:0/2694097142 >> 172.21.15.69:0/2694097142 conn(0x55a022359660 msgr2=0x55a02235b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:41.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5340780740 1 -- 172.21.15.69:0/2694097142 shutdown_connections 2024-03-19T01:12:41.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5340780740 1 -- 172.21.15.69:0/2694097142 wait complete. 2024-03-19T01:12:41.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.404+0000 7f5340780740 20 client.26159 trim_cache size 0 max 16384 2024-03-19T01:12:41.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.425+0000 7f5340780740 1 Processor -- start 2024-03-19T01:12:41.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.425+0000 7f5340780740 1 -- start start 2024-03-19T01:12:41.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5340780740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235ef30 0x55a022367c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5340780740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022373fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5340780740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a022364660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5340780740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f53381b0c10 con 0x55a02235ef30 2024-03-19T01:12:41.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5340780740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53381b2810 con 0x55a02235f850 2024-03-19T01:12:41.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5340780740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f53381b12e0 con 0x55a022368430 2024-03-19T01:12:41.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5336a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022373fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f532fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235ef30 0x55a022367c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5336a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022373fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36372/0 (socket says 172.21.15.69:36372) 2024-03-19T01:12:41.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f532fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235ef30 0x55a022367c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48528/0 (socket says 172.21.15.69:48528) 2024-03-19T01:12:41.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5336a00640 1 -- 172.21.15.69:0/439416506 learned_addr learned my addr 172.21.15.69:0/439416506 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:41.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f532fe00640 1 --2- 172.21.15.69:0/439416506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a022364660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5336a00640 1 -- 172.21.15.69:0/439416506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 msgr2=0x55a022364660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:41.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5336a00640 1 --2- 172.21.15.69:0/439416506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a022364660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5336a00640 1 -- 172.21.15.69:0/439416506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235ef30 msgr2=0x55a022367c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:41.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5336a00640 1 --2- 172.21.15.69:0/439416506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235ef30 0x55a022367c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.426+0000 7f5336a00640 1 -- 172.21.15.69:0/439416506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a02235f680 con 0x55a02235f850 2024-03-19T01:12:41.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.427+0000 7f5336a00640 1 --2- 172.21.15.69:0/439416506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022373fc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5328009620 tx=0x7f5328009fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e35d6df7884b2b66 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.427+0000 7f5327400640 1 -- 172.21.15.69:0/439416506 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328010430 con 0x55a02235f850 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.427+0000 7f5327400640 1 -- 172.21.15.69:0/439416506 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5328010e50 con 0x55a02235f850 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.427+0000 7f5327400640 1 -- 172.21.15.69:0/439416506 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328005640 con 0x55a02235f850 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.428+0000 7f5340780740 1 -- 172.21.15.69:0/439416506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 msgr2=0x55a022373fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.428+0000 7f5340780740 1 --2- 172.21.15.69:0/439416506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022373fc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5328009620 tx=0x7f5328009fa0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.428+0000 7f5340780740 1 -- 172.21.15.69:0/439416506 shutdown_connections 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.428+0000 7f5340780740 1 --2- 172.21.15.69:0/439416506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a022368430 0x55a022364660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.428+0000 7f5340780740 1 --2- 172.21.15.69:0/439416506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235f850 0x55a022373fc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.428+0000 7f5340780740 1 --2- 172.21.15.69:0/439416506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a02235ef30 0x55a022367c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.428+0000 7f5340780740 1 -- 172.21.15.69:0/439416506 >> 172.21.15.69:0/439416506 conn(0x55a022359660 msgr2=0x55a022357e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.428+0000 7f5340780740 1 -- 172.21.15.69:0/439416506 shutdown_connections 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.428+0000 7f5340780740 1 -- 172.21.15.69:0/439416506 wait complete. 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.428+0000 7f5340780740 1 -- start start 2024-03-19T01:12:41.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.428+0000 7f5340780740 1 Processor -- start 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f5340780740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a022370900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f5340780740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a02235f850 0x55a02236bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f5340780740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a022368430 0x55a02236c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f5340780740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f53381b2080 con 0x55a022368430 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f5340780740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53381b19b0 con 0x55a02235ef30 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f5340780740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f53381aebf0 con 0x55a02235f850 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f5340780740 10 client.0 _ll_register_callbacks cb 0x7f5328012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f532fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a022370900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f532fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a022370900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36382/0 (socket says 172.21.15.69:36382) 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f5336a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a02235f850 0x55a02236bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f532fe00640 1 -- 172.21.15.69:0/2180707893 learned_addr learned my addr 172.21.15.69:0/2180707893 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.429+0000 7f532fe00640 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a022368430 0x55a02236c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.430+0000 7f532fe00640 1 -- 172.21.15.69:0/2180707893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a02235f850 msgr2=0x55a02236bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:41.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.430+0000 7f532fe00640 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a02235f850 0x55a02236bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.430+0000 7f532fe00640 1 -- 172.21.15.69:0/2180707893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a022368430 msgr2=0x55a02236c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.430+0000 7f532fe00640 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a022368430 0x55a02236c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.430+0000 7f532fe00640 1 -- 172.21.15.69:0/2180707893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a02236ca00 con 0x55a02235ef30 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.430+0000 7f532fe00640 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a022370900 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5320009ce0 tx=0x7f5320020040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84a5f91346710c19 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.430+0000 7f5324c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.430+0000 7f5324c00640 1 -- 172.21.15.69:0/2180707893 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320016040 con 0x55a02235ef30 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.430+0000 7f5324c00640 1 -- 172.21.15.69:0/2180707893 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f53200053d0 con 0x55a02235ef30 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.430+0000 7f5340780740 1 -- 172.21.15.69:0/2180707893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5328010e50 con 0x55a02235ef30 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.431+0000 7f5324c00640 1 -- 172.21.15.69:0/2180707893 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f532001a040 con 0x55a02235ef30 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.431+0000 7f5340780740 10 client.26165 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.431+0000 7f5340780740 1 -- 172.21.15.69:0/2180707893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a0223decb0 con 0x55a02235ef30 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.431+0000 7f5324c00640 1 -- 172.21.15.69:0/2180707893 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5328010e50 con 0x55a02235ef30 2024-03-19T01:12:41.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.431+0000 7f5324c00640 1 -- 172.21.15.69:0/2180707893 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f532001e400 con 0x55a02235ef30 2024-03-19T01:12:41.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.431+0000 7f5340780740 10 client.26165 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:41.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.431+0000 7f5340780740 10 client.26165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:41.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.431+0000 7f5340780740 20 client.26165 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:41.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.431+0000 7f5340780740 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a0223df1c0 0x55a0223e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.431+0000 7f5340780740 4 client.26165 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:12:41.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.431+0000 7f5340780740 1 -- 172.21.15.69:0/2180707893 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0222d0800 con 0x55a0223df1c0 2024-03-19T01:12:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.432+0000 7f5336a00640 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a0223df1c0 0x55a0223e15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.432+0000 7f5336a00640 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a0223df1c0 0x55a0223e15e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b3ba2d2e38f6794 server_cookie=70f529ec9b53dff9 in_seq=0 out_seq=0 2024-03-19T01:12:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.432+0000 7f5324c00640 10 client.26165 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5324c00640 1 -- 172.21.15.69:0/2180707893 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a0222d0800 con 0x55a0223df1c0 2024-03-19T01:12:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5324c00640 10 client.26165 handle_command_reply: tid=0 2024-03-19T01:12:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 1 client.26165 shutdown 2024-03-19T01:12:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 1 -- 172.21.15.69:0/2180707893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 msgr2=0x55a022370900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a022370900 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5320009ce0 tx=0x7f5320020040 comp rx=0 tx=0).stop 2024-03-19T01:12:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 1 -- 172.21.15.69:0/2180707893 shutdown_connections 2024-03-19T01:12:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a0223df1c0 0x55a0223e15e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a022368430 0x55a02236c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5324c00640 0 client.26165 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:41.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a02235f850 0x55a02236bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 1 --2- 172.21.15.69:0/2180707893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a02235ef30 0x55a022370900 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 1 -- 172.21.15.69:0/2180707893 >> 172.21.15.69:0/2180707893 conn(0x55a022359660 msgr2=0x55a0223654c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:41.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 1 -- 172.21.15.69:0/2180707893 shutdown_connections 2024-03-19T01:12:41.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 1 -- 172.21.15.69:0/2180707893 wait complete. 2024-03-19T01:12:41.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.434+0000 7f5340780740 20 client.26165 trim_cache size 0 max 16384 2024-03-19T01:12:41.438 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:12:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:12:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:12:41.439 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 -- 172.21.15.69:0/3920389323 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f531c0a2970 msgr2=0x7f531c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f531c0a2970 0x7f531c0a4e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f53300b9e40 tx=0x7f53300095c0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 -- 172.21.15.69:0/3920389323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5338107690 msgr2=0x7f53380745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5338107690 0x7f53380745a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5330009bf0 tx=0x7f5330034670 comp rx=0 tx=0).stop 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 -- 172.21.15.69:0/3920389323 shutdown_connections 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f531c0a2970 0x7f531c0a4e30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5338107690 0x7f53380745a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5338102040 0x7f5338074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 --2- 172.21.15.69:0/3920389323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5338101680 0x7f5338073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 -- 172.21.15.69:0/3920389323 >> 172.21.15.69:0/3920389323 conn(0x7f53380fd190 msgr2=0x7f53380fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 -- 172.21.15.69:0/3920389323 shutdown_connections 2024-03-19T01:12:41.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.435+0000 7f5337e00640 1 -- 172.21.15.69:0/3920389323 wait complete. 2024-03-19T01:12:41.454 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:12:41.455 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:12:41.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.662+0000 7f4c91a00640 1 -- 172.21.15.69:0/2924759790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c8c068980 msgr2=0x7f4c8c068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.662+0000 7f4c91a00640 1 --2- 172.21.15.69:0/2924759790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c8c068980 0x7f4c8c068de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009900 tx=0x7f4c7402bc40 comp rx=0 tx=0).stop 2024-03-19T01:12:41.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.662+0000 7f4c91a00640 1 -- 172.21.15.69:0/2924759790 shutdown_connections 2024-03-19T01:12:41.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.662+0000 7f4c91a00640 1 --2- 172.21.15.69:0/2924759790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c069320 0x7f4c8c10e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.662+0000 7f4c91a00640 1 --2- 172.21.15.69:0/2924759790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c8c068980 0x7f4c8c068de0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.662+0000 7f4c91a00640 1 --2- 172.21.15.69:0/2924759790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c1076e0 0x7f4c8c107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.662+0000 7f4c91a00640 1 -- 172.21.15.69:0/2924759790 >> 172.21.15.69:0/2924759790 conn(0x7f4c8c076130 msgr2=0x7f4c8c076540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:41.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.662+0000 7f4c91a00640 1 -- 172.21.15.69:0/2924759790 shutdown_connections 2024-03-19T01:12:41.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.662+0000 7f4c91a00640 1 -- 172.21.15.69:0/2924759790 wait complete. 2024-03-19T01:12:41.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.663+0000 7f4c91a00640 1 Processor -- start 2024-03-19T01:12:41.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.663+0000 7f4c91a00640 1 -- start start 2024-03-19T01:12:41.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.664+0000 7f4c91a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c068980 0x7f4c8c19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.664+0000 7f4c91a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c069320 0x7f4c8c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.664+0000 7f4c91a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c8c1076e0 0x7f4c8c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.664+0000 7f4c91a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c111a90 con 0x7f4c8c069320 2024-03-19T01:12:41.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.664+0000 7f4c91a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c111910 con 0x7f4c8c068980 2024-03-19T01:12:41.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.664+0000 7f4c91a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c111c10 con 0x7f4c8c1076e0 2024-03-19T01:12:41.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.664+0000 7f4c8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c068980 0x7f4c8c19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.664+0000 7f4c8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c068980 0x7f4c8c19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36402/0 (socket says 172.21.15.69:36402) 2024-03-19T01:12:41.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.664+0000 7f4c8be00640 1 -- 172.21.15.69:0/3051971865 learned_addr learned my addr 172.21.15.69:0/3051971865 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:41.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.664+0000 7f4c91000640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c8c1076e0 0x7f4c8c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.664+0000 7f4c83400640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c069320 0x7f4c8c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c8be00640 1 -- 172.21.15.69:0/3051971865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c8c1076e0 msgr2=0x7f4c8c19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c8be00640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c8c1076e0 0x7f4c8c19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c8be00640 1 -- 172.21.15.69:0/3051971865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c069320 msgr2=0x7f4c8c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:41.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c8be00640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c069320 0x7f4c8c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:41.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c8be00640 1 -- 172.21.15.69:0/3051971865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c19f140 con 0x7f4c8c068980 2024-03-19T01:12:41.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c83400640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c069320 0x7f4c8c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:41.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c8be00640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c068980 0x7f4c8c19a150 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00eb90 tx=0x7f4c7c00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:41.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c89600640 1 -- 172.21.15.69:0/3051971865 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c004b20 con 0x7f4c8c068980 2024-03-19T01:12:41.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c89600640 1 -- 172.21.15.69:0/3051971865 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7c004cc0 con 0x7f4c8c068980 2024-03-19T01:12:41.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c91a00640 1 -- 172.21.15.69:0/3051971865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c19f430 con 0x7f4c8c068980 2024-03-19T01:12:41.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c89600640 1 -- 172.21.15.69:0/3051971865 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c010720 con 0x7f4c8c068980 2024-03-19T01:12:41.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.665+0000 7f4c91a00640 1 -- 172.21.15.69:0/3051971865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1a6cc0 con 0x7f4c8c068980 2024-03-19T01:12:41.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.666+0000 7f4c91a00640 1 -- 172.21.15.69:0/3051971865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c068530 con 0x7f4c8c068980 2024-03-19T01:12:41.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.670+0000 7f4c89600640 1 -- 172.21.15.69:0/3051971865 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f4c7c0108c0 con 0x7f4c8c068980 2024-03-19T01:12:41.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.671+0000 7f4c89600640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c700a2970 0x7f4c700a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:41.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.671+0000 7f4c89600640 1 -- 172.21.15.69:0/3051971865 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4c7c0cfa50 con 0x7f4c8c068980 2024-03-19T01:12:41.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.671+0000 7f4c83400640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c700a2970 0x7f4c700a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:41.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.672+0000 7f4c83400640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c700a2970 0x7f4c700a4e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c74006010 tx=0x7f4c74005a20 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:41.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:41.676+0000 7f4c89600640 1 -- 172.21.15.69:0/3051971865 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4c7c053740 con 0x7f4c8c068980 2024-03-19T01:12:42.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.100+0000 7f4c91a00640 1 -- 172.21.15.69:0/3051971865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c8c19b4a0 con 0x7f4c8c068980 2024-03-19T01:12:42.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.102+0000 7f4c89600640 1 -- 172.21.15.69:0/3051971865 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11937 (secure 0 0 0) 0x7f4c7c09c3f0 con 0x7f4c8c068980 2024-03-19T01:12:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:12:42.106 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:12:18.556169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:12:42.106 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 31 2024-03-19T01:12:42.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.106+0000 7f4c91a00640 1 -- 172.21.15.69:0/3051971865 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c700a2970 msgr2=0x7f4c700a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.106+0000 7f4c91a00640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c700a2970 0x7f4c700a4e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c74006010 tx=0x7f4c74005a20 comp rx=0 tx=0).stop 2024-03-19T01:12:42.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.106+0000 7f4c91a00640 1 -- 172.21.15.69:0/3051971865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c068980 msgr2=0x7f4c8c19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.106+0000 7f4c91a00640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c068980 0x7f4c8c19a150 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00eb90 tx=0x7f4c7c00ef90 comp rx=0 tx=0).stop 2024-03-19T01:12:42.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.107+0000 7f4c91a00640 1 -- 172.21.15.69:0/3051971865 shutdown_connections 2024-03-19T01:12:42.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.107+0000 7f4c91a00640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c8c1076e0 0x7f4c8c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.107+0000 7f4c91a00640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c069320 0x7f4c8c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.107+0000 7f4c91a00640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c700a2970 0x7f4c700a4e30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.107+0000 7f4c91a00640 1 --2- 172.21.15.69:0/3051971865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c068980 0x7f4c8c19a150 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.107+0000 7f4c91a00640 1 -- 172.21.15.69:0/3051971865 >> 172.21.15.69:0/3051971865 conn(0x7f4c8c076130 msgr2=0x7f4c8c0ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.107+0000 7f4c91a00640 1 -- 172.21.15.69:0/3051971865 shutdown_connections 2024-03-19T01:12:42.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.107+0000 7f4c91a00640 1 -- 172.21.15.69:0/3051971865 wait complete. 2024-03-19T01:12:42.140 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:12:42.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.482+0000 7f748be00640 1 -- 172.21.15.69:0/743984599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c108820 msgr2=0x7f748c108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.482+0000 7f748be00640 1 --2- 172.21.15.69:0/743984599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c108820 0x7f748c108c00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f747c009930 tx=0x7f747c02bc50 comp rx=0 tx=0).stop 2024-03-19T01:12:42.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.483+0000 7f748be00640 1 -- 172.21.15.69:0/743984599 shutdown_connections 2024-03-19T01:12:42.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.483+0000 7f748be00640 1 --2- 172.21.15.69:0/743984599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c114970 0x7f748c116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.483+0000 7f748be00640 1 --2- 172.21.15.69:0/743984599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c109140 0x7f748c114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.483+0000 7f748be00640 1 --2- 172.21.15.69:0/743984599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c108820 0x7f748c108c00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.483+0000 7f748be00640 1 -- 172.21.15.69:0/743984599 >> 172.21.15.69:0/743984599 conn(0x7f748c06c770 msgr2=0x7f748c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.484+0000 7f748be00640 1 -- 172.21.15.69:0/743984599 shutdown_connections 2024-03-19T01:12:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.484+0000 7f748be00640 1 -- 172.21.15.69:0/743984599 wait complete. 2024-03-19T01:12:42.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.486+0000 7f748be00640 1 Processor -- start 2024-03-19T01:12:42.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.487+0000 7f748be00640 1 -- start start 2024-03-19T01:12:42.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.488+0000 7f748be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c108820 0x7f748c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.488+0000 7f748be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c109140 0x7f748c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.488+0000 7f748be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c114970 0x7f748c1a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.488+0000 7f748be00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f748c118130 con 0x7f748c108820 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.488+0000 7f748be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f748c117fb0 con 0x7f748c109140 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.488+0000 7f748be00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f748c1182b0 con 0x7f748c114970 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.488+0000 7f748a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c109140 0x7f748c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.488+0000 7f748a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c109140 0x7f748c19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36418/0 (socket says 172.21.15.69:36418) 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.488+0000 7f748a000640 1 -- 172.21.15.69:0/266990108 learned_addr learned my addr 172.21.15.69:0/266990108 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.488+0000 7f748b400640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c114970 0x7f748c1a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.489+0000 7f748aa00640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c108820 0x7f748c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.489+0000 7f748a000640 1 -- 172.21.15.69:0/266990108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c114970 msgr2=0x7f748c1a2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.489+0000 7f748a000640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c114970 0x7f748c1a2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.489+0000 7f748a000640 1 -- 172.21.15.69:0/266990108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c108820 msgr2=0x7f748c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.489+0000 7f748a000640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c108820 0x7f748c19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.489+0000 7f748a000640 1 -- 172.21.15.69:0/266990108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c1a3600 con 0x7f748c109140 2024-03-19T01:12:42.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.489+0000 7f748b400640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c114970 0x7f748c1a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:42.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.489+0000 7f748a000640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c109140 0x7f748c19eb50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f747800b450 tx=0x7f747800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.490+0000 7f7490a00640 1 -- 172.21.15.69:0/266990108 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478015020 con 0x7f748c109140 2024-03-19T01:12:42.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.490+0000 7f7490a00640 1 -- 172.21.15.69:0/266990108 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7478011d70 con 0x7f748c109140 2024-03-19T01:12:42.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.490+0000 7f7490a00640 1 -- 172.21.15.69:0/266990108 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747800f340 con 0x7f748c109140 2024-03-19T01:12:42.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.490+0000 7f748be00640 1 -- 172.21.15.69:0/266990108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748c1a38f0 con 0x7f748c109140 2024-03-19T01:12:42.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.490+0000 7f748be00640 1 -- 172.21.15.69:0/266990108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f748c1b3990 con 0x7f748c109140 2024-03-19T01:12:42.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.496+0000 7f7494a2e740 1 Processor -- start 2024-03-19T01:12:42.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.496+0000 7f7494a2e740 1 -- start start 2024-03-19T01:12:42.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.497+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600065fbbf0 0x5600065fbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.497+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fc510 0x560006604bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.497+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x5600066074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.497+0000 7f7494a2e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560006560380 con 0x5600065fbbf0 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.497+0000 7f7494a2e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560006508d20 con 0x5600065fc510 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.497+0000 7f7494a2e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56000655c420 con 0x5600066050f0 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.497+0000 7f748aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fc510 0x560006604bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.497+0000 7f748aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fc510 0x560006604bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36420/0 (socket says 172.21.15.69:36420) 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.497+0000 7f748aa00640 1 -- 172.21.15.69:0/3890310458 learned_addr learned my addr 172.21.15.69:0/3890310458 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.497+0000 7f7490a00640 1 -- 172.21.15.69:0/266990108 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f7478018020 con 0x7f748c109140 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.498+0000 7f748aa00640 1 -- 172.21.15.69:0/3890310458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 msgr2=0x5600066074e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.498+0000 7f748aa00640 1 --2- 172.21.15.69:0/3890310458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x5600066074e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.498+0000 7f748aa00640 1 -- 172.21.15.69:0/3890310458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600065fbbf0 msgr2=0x5600065fbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.498+0000 7f748aa00640 1 --2- 172.21.15.69:0/3890310458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600065fbbf0 0x5600065fbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.498+0000 7f748aa00640 1 -- 172.21.15.69:0/3890310458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600062743f0 con 0x5600065fc510 2024-03-19T01:12:42.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.500+0000 7f7490a00640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74700a2ab0 0x7f74700a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.501+0000 7f748b400640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74700a2ab0 0x7f74700a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.502+0000 7f748b400640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74700a2ab0 0x7f74700a4f70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f748c06ad40 tx=0x7f7484009f50 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.502+0000 7f748aa00640 1 --2- 172.21.15.69:0/3890310458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fc510 0x560006604bb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f747c02c260 tx=0x7f747c02c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7c40ed5f2479c24 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.502+0000 7f7490a00640 1 -- 172.21.15.69:0/266990108 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f74780ce9b0 con 0x7f748c109140 2024-03-19T01:12:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.503+0000 7f7477e00640 1 -- 172.21.15.69:0/3890310458 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c042070 con 0x5600065fc510 2024-03-19T01:12:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.503+0000 7f7477e00640 1 -- 172.21.15.69:0/3890310458 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f747c004240 con 0x5600065fc510 2024-03-19T01:12:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.503+0000 7f7477e00640 1 -- 172.21.15.69:0/3890310458 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c004920 con 0x5600065fc510 2024-03-19T01:12:42.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.504+0000 7f7494a2e740 1 -- 172.21.15.69:0/3890310458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fc510 msgr2=0x560006604bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.504+0000 7f7494a2e740 1 --2- 172.21.15.69:0/3890310458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fc510 0x560006604bb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f747c02c260 tx=0x7f747c02c5f0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.504+0000 7f7494a2e740 1 -- 172.21.15.69:0/3890310458 shutdown_connections 2024-03-19T01:12:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.504+0000 7f7494a2e740 1 --2- 172.21.15.69:0/3890310458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x5600066074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.504+0000 7f7494a2e740 1 --2- 172.21.15.69:0/3890310458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fc510 0x560006604bb0 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f747c02c260 tx=0x7f747c02c5f0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.504+0000 7f7494a2e740 1 --2- 172.21.15.69:0/3890310458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600065fbbf0 0x5600065fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.504+0000 7f7494a2e740 1 -- 172.21.15.69:0/3890310458 >> 172.21.15.69:0/3890310458 conn(0x5600065f6320 msgr2=0x5600065f6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:42 smithi086 ceph-mon[66829]: pgmap v64: 129 pgs: 129 active+clean; 188 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-03-19T01:12:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:42 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3051971865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:12:42.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.505+0000 7f7494a2e740 1 -- 172.21.15.69:0/3890310458 shutdown_connections 2024-03-19T01:12:42.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.505+0000 7f7494a2e740 1 -- 172.21.15.69:0/3890310458 wait complete. 2024-03-19T01:12:42.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.505+0000 7f7494a2e740 1 -- start start 2024-03-19T01:12:42.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.506+0000 7f7494a2e740 1 Processor -- start 2024-03-19T01:12:42.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.507+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 0x560006602310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.507+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600066050f0 0x560006602850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.507+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560006602de0 0x5600066112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.507+0000 7f7494a2e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560006607e80 con 0x5600066050f0 2024-03-19T01:12:42.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.507+0000 7f748b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560006602de0 0x5600066112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.507+0000 7f748b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560006602de0 0x5600066112f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53850/0 (socket says 172.21.15.69:53850) 2024-03-19T01:12:42.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.507+0000 7f748b400640 1 -- 172.21.15.69:0/1169490908 learned_addr learned my addr 172.21.15.69:0/1169490908 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:42.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.508+0000 7f748a000640 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600066050f0 0x560006602850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.508+0000 7f7494a2e740 1 -- 172.21.15.69:0/1169490908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56000651c160 con 0x5600065fbbf0 2024-03-19T01:12:42.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.508+0000 7f7494a2e740 1 -- 172.21.15.69:0/1169490908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560006608000 con 0x560006602de0 2024-03-19T01:12:42.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.508+0000 7f7494a2e740 10 client.0 _ll_register_callbacks cb 0x560006558170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:42.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.508+0000 7f748aa00640 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 0x560006602310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.508+0000 7f748b400640 1 -- 172.21.15.69:0/1169490908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 msgr2=0x560006602310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.509+0000 7f748b400640 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 0x560006602310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.509+0000 7f748b400640 1 -- 172.21.15.69:0/1169490908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600066050f0 msgr2=0x560006602850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.509+0000 7f748b400640 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600066050f0 0x560006602850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.509+0000 7f748b400640 1 -- 172.21.15.69:0/1169490908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000633e7d0 con 0x560006602de0 2024-03-19T01:12:42.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.509+0000 7f748a000640 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600066050f0 0x560006602850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:42.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.509+0000 7f748aa00640 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 0x560006602310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:12:42.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.509+0000 7f748b400640 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560006602de0 0x5600066112f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f74840107c0 tx=0x7f74840096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a75f9bf44e8f5b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.509+0000 7f7474c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:42.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.509+0000 7f7474c00640 1 -- 172.21.15.69:0/1169490908 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7484011dd0 con 0x560006602de0 2024-03-19T01:12:42.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.509+0000 7f7474c00640 1 -- 172.21.15.69:0/1169490908 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f748401e420 con 0x560006602de0 2024-03-19T01:12:42.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.509+0000 7f7494a2e740 1 -- 172.21.15.69:0/1169490908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f747c004240 con 0x560006602de0 2024-03-19T01:12:42.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.510+0000 7f7474c00640 1 -- 172.21.15.69:0/1169490908 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f748402f050 con 0x560006602de0 2024-03-19T01:12:42.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.510+0000 7f7494a2e740 10 client.26164 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:42.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.510+0000 7f7494a2e740 1 -- 172.21.15.69:0/1169490908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560006614810 con 0x560006602de0 2024-03-19T01:12:42.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.511+0000 7f7474c00640 1 -- 172.21.15.69:0/1169490908 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f747c004240 con 0x560006602de0 2024-03-19T01:12:42.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.511+0000 7f7474c00640 1 -- 172.21.15.69:0/1169490908 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f7484026810 con 0x560006602de0 2024-03-19T01:12:42.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.511+0000 7f7494a2e740 10 client.26164 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:42.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.511+0000 7f7494a2e740 10 client.26164 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:12:42.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.511+0000 7f7494a2e740 20 client.26164 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:42.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.511+0000 7f7494a2e740 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x560006615cd0 0x5600066180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.512+0000 7f7494a2e740 4 client.26164 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:42.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.512+0000 7f7494a2e740 1 -- 172.21.15.69:0/1169490908 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56000656d8c0 con 0x560006615cd0 2024-03-19T01:12:42.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.513+0000 7f748aa00640 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x560006615cd0 0x5600066180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.518+0000 7f748aa00640 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x560006615cd0 0x5600066180c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e94b260ee4366ec6 server_cookie=119391cf18456684 in_seq=0 out_seq=0 2024-03-19T01:12:42.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.518+0000 7f7474c00640 10 client.26164 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:12:42.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7474c00640 1 -- 172.21.15.69:0/1169490908 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56000656d8c0 con 0x560006615cd0 2024-03-19T01:12:42.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7474c00640 10 client.26164 handle_command_reply: tid=0 2024-03-19T01:12:42.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7494a2e740 1 client.26164 shutdown 2024-03-19T01:12:42.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7494a2e740 1 -- 172.21.15.69:0/1169490908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560006602de0 msgr2=0x5600066112f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7494a2e740 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560006602de0 0x5600066112f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f74840107c0 tx=0x7f74840096c0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7494a2e740 1 -- 172.21.15.69:0/1169490908 shutdown_connections 2024-03-19T01:12:42.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7494a2e740 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x560006615cd0 0x5600066180c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7494a2e740 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560006602de0 0x5600066112f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7494a2e740 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600066050f0 0x560006602850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7494a2e740 1 --2- 172.21.15.69:0/1169490908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 0x560006602310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7494a2e740 1 -- 172.21.15.69:0/1169490908 >> 172.21.15.69:0/1169490908 conn(0x5600065f6320 msgr2=0x5600065f8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.520+0000 7f7474c00640 0 client.26164 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:12:42.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.521+0000 7f7494a2e740 1 -- 172.21.15.69:0/1169490908 shutdown_connections 2024-03-19T01:12:42.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.521+0000 7f7494a2e740 1 -- 172.21.15.69:0/1169490908 wait complete. 2024-03-19T01:12:42.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.521+0000 7f7494a2e740 20 client.26164 trim_cache size 0 max 16384 2024-03-19T01:12:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:42 smithi069 ceph-mon[31727]: pgmap v64: 129 pgs: 129 active+clean; 188 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-03-19T01:12:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:42 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3051971865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:12:42.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.550+0000 7f7494a2e740 1 Processor -- start 2024-03-19T01:12:42.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.550+0000 7f7494a2e740 1 -- start start 2024-03-19T01:12:42.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.550+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600065fbbf0 0x560006617ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.550+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560006602de0 0x56000661d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.550+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x560006610c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.550+0000 7f7494a2e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f748c1bd950 con 0x5600065fbbf0 2024-03-19T01:12:42.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.550+0000 7f7494a2e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f748c1bf550 con 0x560006602de0 2024-03-19T01:12:42.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.550+0000 7f7494a2e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f748c1be020 con 0x5600066050f0 2024-03-19T01:12:42.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.550+0000 7f748aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600065fbbf0 0x560006617ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.551+0000 7f748a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560006602de0 0x56000661d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.551+0000 7f748aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600065fbbf0 0x560006617ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48596/0 (socket says 172.21.15.69:48596) 2024-03-19T01:12:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.551+0000 7f748aa00640 1 -- 172.21.15.69:0/2167717526 learned_addr learned my addr 172.21.15.69:0/2167717526 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.551+0000 7f748b400640 1 --2- 172.21.15.69:0/2167717526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x560006610c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.551+0000 7f748a000640 1 -- 172.21.15.69:0/2167717526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 msgr2=0x560006610c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.551+0000 7f748a000640 1 --2- 172.21.15.69:0/2167717526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x560006610c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.551+0000 7f748a000640 1 -- 172.21.15.69:0/2167717526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600065fbbf0 msgr2=0x560006617ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.551+0000 7f748a000640 1 --2- 172.21.15.69:0/2167717526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600065fbbf0 0x560006617ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.551+0000 7f748a000640 1 -- 172.21.15.69:0/2167717526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600066008a0 con 0x560006602de0 2024-03-19T01:12:42.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.551+0000 7f748aa00640 1 --2- 172.21.15.69:0/2167717526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600065fbbf0 0x560006617ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:42.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.552+0000 7f748a000640 1 --2- 172.21.15.69:0/2167717526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560006602de0 0x56000661d950 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f74780cbe50 tx=0x7f747809a780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67656b434935fcab server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.552+0000 7f7477400640 1 -- 172.21.15.69:0/2167717526 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74780b8070 con 0x560006602de0 2024-03-19T01:12:42.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.552+0000 7f7477400640 1 -- 172.21.15.69:0/2167717526 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f747809eeb0 con 0x560006602de0 2024-03-19T01:12:42.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.552+0000 7f7477400640 1 -- 172.21.15.69:0/2167717526 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747809a4a0 con 0x560006602de0 2024-03-19T01:12:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.553+0000 7f7494a2e740 1 -- 172.21.15.69:0/2167717526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560006602de0 msgr2=0x56000661d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.553+0000 7f7494a2e740 1 --2- 172.21.15.69:0/2167717526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560006602de0 0x56000661d950 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f74780cbe50 tx=0x7f747809a780 comp rx=0 tx=0).stop 2024-03-19T01:12:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.553+0000 7f7494a2e740 1 -- 172.21.15.69:0/2167717526 shutdown_connections 2024-03-19T01:12:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.553+0000 7f7494a2e740 1 --2- 172.21.15.69:0/2167717526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x560006610c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.553+0000 7f7494a2e740 1 --2- 172.21.15.69:0/2167717526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560006602de0 0x56000661d950 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.553+0000 7f7494a2e740 1 --2- 172.21.15.69:0/2167717526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600065fbbf0 0x560006617ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.553+0000 7f7494a2e740 1 -- 172.21.15.69:0/2167717526 >> 172.21.15.69:0/2167717526 conn(0x5600065f6320 msgr2=0x560006600c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.553+0000 7f7494a2e740 1 -- 172.21.15.69:0/2167717526 shutdown_connections 2024-03-19T01:12:42.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.554+0000 7f7494a2e740 1 -- 172.21.15.69:0/2167717526 wait complete. 2024-03-19T01:12:42.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.554+0000 7f7494a2e740 1 -- start start 2024-03-19T01:12:42.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.554+0000 7f7494a2e740 1 Processor -- start 2024-03-19T01:12:42.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.555+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 0x56000660d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.555+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560006602de0 0x5600065fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.555+0000 7f7494a2e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x5600065ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.555+0000 7f7494a2e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f748c1bedc0 con 0x560006602de0 2024-03-19T01:12:42.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.555+0000 7f7494a2e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f748c1be6f0 con 0x5600065fbbf0 2024-03-19T01:12:42.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.555+0000 7f7494a2e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f748c1bb930 con 0x5600066050f0 2024-03-19T01:12:42.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.555+0000 7f7494a2e740 10 client.0 _ll_register_callbacks cb 0x7f748400e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:42.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 0x56000660d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 0x56000660d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36458/0 (socket says 172.21.15.69:36458) 2024-03-19T01:12:42.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748aa00640 1 -- 172.21.15.69:0/750313123 learned_addr learned my addr 172.21.15.69:0/750313123 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:42.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748a000640 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560006602de0 0x5600065fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748b400640 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x5600065ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748aa00640 1 -- 172.21.15.69:0/750313123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 msgr2=0x5600065ff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748aa00640 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x5600065ff2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748aa00640 1 -- 172.21.15.69:0/750313123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560006602de0 msgr2=0x5600065fece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748aa00640 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560006602de0 0x5600065fece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748aa00640 1 -- 172.21.15.69:0/750313123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600065ffaf0 con 0x5600065fbbf0 2024-03-19T01:12:42.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748a000640 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560006602de0 0x5600065fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:42.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.556+0000 7f748b400640 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x5600065ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:12:42.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.557+0000 7f748aa00640 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 0x56000660d620 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f747c02c260 tx=0x7f747c0053a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c197f33c37675a69 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.557+0000 7f7474c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:42.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.557+0000 7f7474c00640 1 -- 172.21.15.69:0/750313123 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c055060 con 0x5600065fbbf0 2024-03-19T01:12:42.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.557+0000 7f7474c00640 1 -- 172.21.15.69:0/750313123 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f747c02c8a0 con 0x5600065fbbf0 2024-03-19T01:12:42.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.557+0000 7f7494a2e740 1 -- 172.21.15.69:0/750313123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f747809eeb0 con 0x5600065fbbf0 2024-03-19T01:12:42.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.558+0000 7f7474c00640 1 -- 172.21.15.69:0/750313123 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f747c042040 con 0x5600065fbbf0 2024-03-19T01:12:42.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.558+0000 7f7494a2e740 10 client.26186 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:42.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.558+0000 7f7494a2e740 1 -- 172.21.15.69:0/750313123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56000667c020 con 0x5600065fbbf0 2024-03-19T01:12:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:42 smithi163 ceph-mon[67428]: pgmap v64: 129 pgs: 129 active+clean; 188 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-03-19T01:12:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:42 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3051971865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:12:42.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.559+0000 7f7474c00640 1 -- 172.21.15.69:0/750313123 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f747809eeb0 con 0x5600065fbbf0 2024-03-19T01:12:42.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.559+0000 7f7474c00640 1 -- 172.21.15.69:0/750313123 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f747c047570 con 0x5600065fbbf0 2024-03-19T01:12:42.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.559+0000 7f7494a2e740 10 client.26186 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:42.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.559+0000 7f7494a2e740 10 client.26186 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:12:42.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.559+0000 7f7494a2e740 20 client.26186 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:42.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.559+0000 7f7494a2e740 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56000667c360 0x56000667e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.559+0000 7f7494a2e740 4 client.26186 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:12:42.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.559+0000 7f7494a2e740 1 -- 172.21.15.69:0/750313123 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560006567c10 con 0x56000667c360 2024-03-19T01:12:42.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.559+0000 7f748a000640 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56000667c360 0x56000667e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.560+0000 7f748a000640 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56000667c360 0x56000667e750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95d98ef8a50ee295 server_cookie=64aa68f2ccf81d19 in_seq=0 out_seq=0 2024-03-19T01:12:42.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.560+0000 7f7474c00640 10 client.26186 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:12:42.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.561+0000 7f7474c00640 1 -- 172.21.15.69:0/750313123 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560006567c10 con 0x56000667c360 2024-03-19T01:12:42.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.561+0000 7f7474c00640 10 client.26186 handle_command_reply: tid=0 2024-03-19T01:12:42.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.561+0000 7f7494a2e740 1 client.26186 shutdown 2024-03-19T01:12:42.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.562+0000 7f7494a2e740 1 -- 172.21.15.69:0/750313123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 msgr2=0x56000660d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.562+0000 7f7494a2e740 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 0x56000660d620 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f747c02c260 tx=0x7f747c0053a0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.563+0000 7f7494a2e740 1 -- 172.21.15.69:0/750313123 shutdown_connections 2024-03-19T01:12:42.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.563+0000 7f7494a2e740 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56000667c360 0x56000667e750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.563+0000 7f7494a2e740 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5600066050f0 0x5600065ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.563+0000 7f7494a2e740 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560006602de0 0x5600065fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.563+0000 7f7494a2e740 1 --2- 172.21.15.69:0/750313123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600065fbbf0 0x56000660d620 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.563+0000 7f7494a2e740 1 -- 172.21.15.69:0/750313123 >> 172.21.15.69:0/750313123 conn(0x5600065f6320 msgr2=0x5600065f7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.563+0000 7f7474c00640 0 client.26186 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:12:42.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.563+0000 7f7494a2e740 1 -- 172.21.15.69:0/750313123 shutdown_connections 2024-03-19T01:12:42.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.565+0000 7f7494a2e740 1 -- 172.21.15.69:0/750313123 wait complete. 2024-03-19T01:12:42.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.565+0000 7f7494a2e740 20 client.26186 trim_cache size 0 max 16384 2024-03-19T01:12:42.567 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:12:42.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.566+0000 7f748be00640 1 -- 172.21.15.69:0/266990108 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74700a2ab0 msgr2=0x7f74700a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.566+0000 7f748be00640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74700a2ab0 0x7f74700a4f70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f748c06ad40 tx=0x7f7484009f50 comp rx=0 tx=0).stop 2024-03-19T01:12:42.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.566+0000 7f748be00640 1 -- 172.21.15.69:0/266990108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c109140 msgr2=0x7f748c19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.566+0000 7f748be00640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c109140 0x7f748c19eb50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f747800b450 tx=0x7f747800b910 comp rx=0 tx=0).stop 2024-03-19T01:12:42.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.567+0000 7f748be00640 1 -- 172.21.15.69:0/266990108 shutdown_connections 2024-03-19T01:12:42.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.567+0000 7f748be00640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f748c114970 0x7f748c1a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.567+0000 7f748be00640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74700a2ab0 0x7f74700a4f70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.567+0000 7f748be00640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c109140 0x7f748c19eb50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.567+0000 7f748be00640 1 --2- 172.21.15.69:0/266990108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c108820 0x7f748c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.567+0000 7f748be00640 1 -- 172.21.15.69:0/266990108 >> 172.21.15.69:0/266990108 conn(0x7f748c06c770 msgr2=0x7f748c0703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.567+0000 7f748be00640 1 -- 172.21.15.69:0/266990108 shutdown_connections 2024-03-19T01:12:42.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.567+0000 7f748be00640 1 -- 172.21.15.69:0/266990108 wait complete. 2024-03-19T01:12:42.593 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:12:42.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.785+0000 7f4297400640 1 -- 172.21.15.69:0/2129485640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4290068a20 msgr2=0x7f4290068e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.785+0000 7f4297400640 1 --2- 172.21.15.69:0/2129485640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4290068a20 0x7f4290068e00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4284009900 tx=0x7f428402bcc0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.785+0000 7f4297400640 1 -- 172.21.15.69:0/2129485640 shutdown_connections 2024-03-19T01:12:42.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.785+0000 7f4297400640 1 --2- 172.21.15.69:0/2129485640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429010e1c0 0x7f42901105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.785+0000 7f4297400640 1 --2- 172.21.15.69:0/2129485640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290069340 0x7f429010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.785+0000 7f4297400640 1 --2- 172.21.15.69:0/2129485640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4290068a20 0x7f4290068e00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.785+0000 7f4297400640 1 -- 172.21.15.69:0/2129485640 >> 172.21.15.69:0/2129485640 conn(0x7f4290077820 msgr2=0x7f4290077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.785+0000 7f4297400640 1 -- 172.21.15.69:0/2129485640 shutdown_connections 2024-03-19T01:12:42.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.785+0000 7f4297400640 1 -- 172.21.15.69:0/2129485640 wait complete. 2024-03-19T01:12:42.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.786+0000 7f4297400640 1 Processor -- start 2024-03-19T01:12:42.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.786+0000 7f4297400640 1 -- start start 2024-03-19T01:12:42.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.786+0000 7f4297400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4290068a20 0x7f429019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.786+0000 7f4297400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290069340 0x7f429019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.786+0000 7f4297400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429010e1c0 0x7f429019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.786+0000 7f4297400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4290113d90 con 0x7f429010e1c0 2024-03-19T01:12:42.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.786+0000 7f4297400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4290113c10 con 0x7f4290069340 2024-03-19T01:12:42.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.786+0000 7f4297400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4290113f10 con 0x7f4290068a20 2024-03-19T01:12:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f428fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290069340 0x7f429019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f4294a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4290068a20 0x7f429019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f428fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290069340 0x7f429019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36472/0 (socket says 172.21.15.69:36472) 2024-03-19T01:12:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f428fe00640 1 -- 172.21.15.69:0/3839307146 learned_addr learned my addr 172.21.15.69:0/3839307146 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f4295400640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429010e1c0 0x7f429019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f428fe00640 1 -- 172.21.15.69:0/3839307146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4290068a20 msgr2=0x7f429019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f428fe00640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4290068a20 0x7f429019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f428fe00640 1 -- 172.21.15.69:0/3839307146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429010e1c0 msgr2=0x7f429019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f428fe00640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429010e1c0 0x7f429019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f428fe00640 1 -- 172.21.15.69:0/3839307146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429019f2a0 con 0x7f4290069340 2024-03-19T01:12:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f4295400640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429010e1c0 0x7f429019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.787+0000 7f428fe00640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290069340 0x7f429019a7f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f427c00e9c0 tx=0x7f427c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.788+0000 7f428d600640 1 -- 172.21.15.69:0/3839307146 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c00ccc0 con 0x7f4290069340 2024-03-19T01:12:42.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.788+0000 7f428d600640 1 -- 172.21.15.69:0/3839307146 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f427c0035f0 con 0x7f4290069340 2024-03-19T01:12:42.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.788+0000 7f4297400640 1 -- 172.21.15.69:0/3839307146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429019f590 con 0x7f4290069340 2024-03-19T01:12:42.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.788+0000 7f428d600640 1 -- 172.21.15.69:0/3839307146 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c00ce60 con 0x7f4290069340 2024-03-19T01:12:42.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.788+0000 7f4297400640 1 -- 172.21.15.69:0/3839307146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42901a6e20 con 0x7f4290069340 2024-03-19T01:12:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.791+0000 7f428d600640 1 -- 172.21.15.69:0/3839307146 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f427c0037a0 con 0x7f4290069340 2024-03-19T01:12:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.791+0000 7f429a182740 1 Processor -- start 2024-03-19T01:12:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.792+0000 7f429a182740 1 -- start start 2024-03-19T01:12:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.792+0000 7f429a182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc05de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.792+0000 7f429a182740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc0669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.792+0000 7f429a182740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc069310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.792+0000 7f429a182740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5616fbfc2380 con 0x5616fc066f20 2024-03-19T01:12:42.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.792+0000 7f429a182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616fbf6ad20 con 0x5616fc05da20 2024-03-19T01:12:42.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.792+0000 7f429a182740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5616fbfbe420 con 0x5616fc05e340 2024-03-19T01:12:42.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.792+0000 7f4295400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc05de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.792+0000 7f4295400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc05de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36478/0 (socket says 172.21.15.69:36478) 2024-03-19T01:12:42.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.792+0000 7f4295400640 1 -- 172.21.15.69:0/945326565 learned_addr learned my addr 172.21.15.69:0/945326565 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:42.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.792+0000 7f4294a00640 1 --2- 172.21.15.69:0/945326565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc0669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f428fe00640 1 --2- 172.21.15.69:0/945326565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f4295400640 1 -- 172.21.15.69:0/945326565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 msgr2=0x5616fc0669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f4295400640 1 --2- 172.21.15.69:0/945326565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc0669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f4295400640 1 -- 172.21.15.69:0/945326565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 msgr2=0x5616fc069310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f4295400640 1 --2- 172.21.15.69:0/945326565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc069310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f4295400640 1 -- 172.21.15.69:0/945326565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616fbcd63f0 con 0x5616fc05da20 2024-03-19T01:12:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f4294a00640 1 --2- 172.21.15.69:0/945326565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc0669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:42.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f428fe00640 1 --2- 172.21.15.69:0/945326565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc069310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:12:42.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f428d600640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42780a29f0 0x7f42780a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f4295400640 1 --2- 172.21.15.69:0/945326565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc05de00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4288011810 tx=0x7f4288011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b4f1f4981e35e10 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f428d600640 1 -- 172.21.15.69:0/3839307146 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f427c0cf560 con 0x7f4290069340 2024-03-19T01:12:42.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f4281c00640 1 -- 172.21.15.69:0/945326565 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288011310 con 0x5616fc05da20 2024-03-19T01:12:42.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f4281c00640 1 -- 172.21.15.69:0/945326565 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f428800f9a0 con 0x5616fc05da20 2024-03-19T01:12:42.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f4281c00640 1 -- 172.21.15.69:0/945326565 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288005720 con 0x5616fc05da20 2024-03-19T01:12:42.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.793+0000 7f4294a00640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42780a29f0 0x7f42780a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.794+0000 7f429a182740 1 -- 172.21.15.69:0/945326565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 msgr2=0x5616fc05de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.794+0000 7f429a182740 1 --2- 172.21.15.69:0/945326565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc05de00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4288011810 tx=0x7f4288011cd0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.794+0000 7f429a182740 1 -- 172.21.15.69:0/945326565 shutdown_connections 2024-03-19T01:12:42.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.794+0000 7f429a182740 1 --2- 172.21.15.69:0/945326565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc069310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.794+0000 7f429a182740 1 --2- 172.21.15.69:0/945326565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc0669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.794+0000 7f429a182740 1 --2- 172.21.15.69:0/945326565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc05de00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.794+0000 7f429a182740 1 -- 172.21.15.69:0/945326565 >> 172.21.15.69:0/945326565 conn(0x5616fc058150 msgr2=0x5616fc058530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.794+0000 7f429a182740 1 -- 172.21.15.69:0/945326565 shutdown_connections 2024-03-19T01:12:42.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.794+0000 7f429a182740 1 -- 172.21.15.69:0/945326565 wait complete. 2024-03-19T01:12:42.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.794+0000 7f4294a00640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42780a29f0 0x7f42780a4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4284006010 tx=0x7f4284005dd0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.795+0000 7f429a182740 1 -- start start 2024-03-19T01:12:42.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.795+0000 7f429a182740 1 Processor -- start 2024-03-19T01:12:42.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f429a182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc063f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f429a182740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc064440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f429a182740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc064980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f429a182740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5616fc069d10 con 0x5616fc066f20 2024-03-19T01:12:42.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f429a182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616fbf7e160 con 0x5616fc05da20 2024-03-19T01:12:42.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f429a182740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5616fc069e90 con 0x5616fc05e340 2024-03-19T01:12:42.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f429a182740 10 client.0 _ll_register_callbacks cb 0x5616fbfba170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:42.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f4294a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc063f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f4294a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc063f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36488/0 (socket says 172.21.15.69:36488) 2024-03-19T01:12:42.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f4294a00640 1 -- 172.21.15.69:0/2561340447 learned_addr learned my addr 172.21.15.69:0/2561340447 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:42.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f4295400640 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc064980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f428fe00640 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc064440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f428fe00640 1 -- 172.21.15.69:0/2561340447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 msgr2=0x5616fc063f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f428fe00640 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc063f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f428fe00640 1 -- 172.21.15.69:0/2561340447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 msgr2=0x5616fc064980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f428fe00640 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc064980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f428fe00640 1 -- 172.21.15.69:0/2561340447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616fbda07d0 con 0x5616fc05e340 2024-03-19T01:12:42.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.796+0000 7f4294a00640 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc063f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:42.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f428fe00640 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc064440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f427c0cc890 tx=0x7f427c09b1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1891d52cba3e00dd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f4276000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:42.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f4276000640 1 -- 172.21.15.69:0/2561340447 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c0a3f40 con 0x5616fc05e340 2024-03-19T01:12:42.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f4276000640 1 -- 172.21.15.69:0/2561340447 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f427c0cf170 con 0x5616fc05e340 2024-03-19T01:12:42.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f429a182740 1 -- 172.21.15.69:0/2561340447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f428800f9a0 con 0x5616fc05e340 2024-03-19T01:12:42.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f4276000640 1 -- 172.21.15.69:0/2561340447 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f427c0ccb10 con 0x5616fc05e340 2024-03-19T01:12:42.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f429a182740 10 client.26179 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:42.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f429a182740 1 -- 172.21.15.69:0/2561340447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5616fc07e7c0 con 0x5616fc05e340 2024-03-19T01:12:42.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f4276000640 1 -- 172.21.15.69:0/2561340447 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f428800f9a0 con 0x5616fc05e340 2024-03-19T01:12:42.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f4276000640 1 -- 172.21.15.69:0/2561340447 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f427c084ce0 con 0x5616fc05e340 2024-03-19T01:12:42.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f429a182740 10 client.26179 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:42.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.797+0000 7f429a182740 10 client.26179 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:12:42.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.798+0000 7f429a182740 20 client.26179 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:42.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.798+0000 7f429a182740 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5616fc07fc80 0x5616fc082070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.798+0000 7f429a182740 4 client.26179 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:42.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.798+0000 7f429a182740 1 -- 172.21.15.69:0/2561340447 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616fbfcf8c0 con 0x5616fc07fc80 2024-03-19T01:12:42.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.798+0000 7f4294a00640 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5616fc07fc80 0x5616fc082070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.798+0000 7f4294a00640 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5616fc07fc80 0x5616fc082070 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f08db2380b3ead7 server_cookie=cd98fa04b62bf62e in_seq=0 out_seq=0 2024-03-19T01:12:42.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.799+0000 7f4276000640 10 client.26179 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:12:42.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.799+0000 7f4276000640 1 -- 172.21.15.69:0/2561340447 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5616fbfcf8c0 con 0x5616fc07fc80 2024-03-19T01:12:42.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.799+0000 7f4276000640 10 client.26179 handle_command_reply: tid=0 2024-03-19T01:12:42.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.799+0000 7f429a182740 1 client.26179 shutdown 2024-03-19T01:12:42.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.799+0000 7f429a182740 1 -- 172.21.15.69:0/2561340447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 msgr2=0x5616fc064440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.799+0000 7f429a182740 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc064440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f427c0cc890 tx=0x7f427c09b1c0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.800+0000 7f429a182740 1 -- 172.21.15.69:0/2561340447 shutdown_connections 2024-03-19T01:12:42.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.800+0000 7f429a182740 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc064980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.800+0000 7f429a182740 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc064440 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.800+0000 7f429a182740 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5616fc07fc80 0x5616fc082070 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.800+0000 7f429a182740 1 --2- 172.21.15.69:0/2561340447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05da20 0x5616fc063f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.800+0000 7f4276000640 0 client.26179 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:12:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.800+0000 7f429a182740 1 -- 172.21.15.69:0/2561340447 >> 172.21.15.69:0/2561340447 conn(0x5616fc058150 msgr2=0x5616fc059ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.800+0000 7f429a182740 1 -- 172.21.15.69:0/2561340447 shutdown_connections 2024-03-19T01:12:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.800+0000 7f429a182740 1 -- 172.21.15.69:0/2561340447 wait complete. 2024-03-19T01:12:42.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.801+0000 7f429a182740 20 client.26179 trim_cache size 0 max 16384 2024-03-19T01:12:42.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.820+0000 7f429a182740 1 Processor -- start 2024-03-19T01:12:42.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.820+0000 7f429a182740 1 -- start start 2024-03-19T01:12:42.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.820+0000 7f429a182740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05da20 0x5616fc066630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.820+0000 7f429a182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05e340 0x5616fc072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.820+0000 7f429a182740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc059c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.820+0000 7f429a182740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f42901acbb0 con 0x5616fc066f20 2024-03-19T01:12:42.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.820+0000 7f429a182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42901ae7b0 con 0x5616fc05e340 2024-03-19T01:12:42.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f429a182740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f42901ad280 con 0x5616fc05da20 2024-03-19T01:12:42.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f4295400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc059c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f428fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05e340 0x5616fc072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f4295400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc059c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48678/0 (socket says 172.21.15.69:48678) 2024-03-19T01:12:42.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f4295400640 1 -- 172.21.15.69:0/371457764 learned_addr learned my addr 172.21.15.69:0/371457764 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:42.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f4294a00640 1 --2- 172.21.15.69:0/371457764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05da20 0x5616fc066630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f4295400640 1 -- 172.21.15.69:0/371457764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05da20 msgr2=0x5616fc066630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f4295400640 1 --2- 172.21.15.69:0/371457764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05da20 0x5616fc066630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f4295400640 1 -- 172.21.15.69:0/371457764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05e340 msgr2=0x5616fc072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f4295400640 1 --2- 172.21.15.69:0/371457764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05e340 0x5616fc072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f4295400640 1 -- 172.21.15.69:0/371457764 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616fc073020 con 0x5616fc066f20 2024-03-19T01:12:42.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f4294a00640 1 --2- 172.21.15.69:0/371457764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05da20 0x5616fc066630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:42.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.821+0000 7f428fe00640 1 --2- 172.21.15.69:0/371457764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05e340 0x5616fc072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:42.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.822+0000 7f4295400640 1 --2- 172.21.15.69:0/371457764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc059c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4288005c40 tx=0x7f4288005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173ef0a16d548cae server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.822+0000 7f4281200640 1 -- 172.21.15.69:0/371457764 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42880050d0 con 0x5616fc066f20 2024-03-19T01:12:42.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.822+0000 7f4281200640 1 -- 172.21.15.69:0/371457764 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f428801f040 con 0x5616fc066f20 2024-03-19T01:12:42.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.822+0000 7f4281200640 1 -- 172.21.15.69:0/371457764 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42880053b0 con 0x5616fc066f20 2024-03-19T01:12:42.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.823+0000 7f429a182740 1 -- 172.21.15.69:0/371457764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 msgr2=0x5616fc059c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.823+0000 7f429a182740 1 --2- 172.21.15.69:0/371457764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc059c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4288005c40 tx=0x7f4288005bd0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.823+0000 7f429a182740 1 -- 172.21.15.69:0/371457764 shutdown_connections 2024-03-19T01:12:42.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.823+0000 7f429a182740 1 --2- 172.21.15.69:0/371457764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc066f20 0x5616fc059c90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.823+0000 7f429a182740 1 --2- 172.21.15.69:0/371457764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc05e340 0x5616fc072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.823+0000 7f429a182740 1 --2- 172.21.15.69:0/371457764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05da20 0x5616fc066630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.823+0000 7f429a182740 1 -- 172.21.15.69:0/371457764 >> 172.21.15.69:0/371457764 conn(0x5616fc058150 msgr2=0x5616fc059580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.823+0000 7f429a182740 1 -- 172.21.15.69:0/371457764 shutdown_connections 2024-03-19T01:12:42.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.823+0000 7f429a182740 1 -- 172.21.15.69:0/371457764 wait complete. 2024-03-19T01:12:42.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.823+0000 7f429a182740 1 -- start start 2024-03-19T01:12:42.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.823+0000 7f429a182740 1 Processor -- start 2024-03-19T01:12:42.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.824+0000 7f429a182740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc05da20 0x5616fc061140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.824+0000 7f429a182740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.824+0000 7f429a182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc066f20 0x5616fc0e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.824+0000 7f429a182740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f42901ae020 con 0x5616fc05da20 2024-03-19T01:12:42.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.824+0000 7f429a182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42901ad950 con 0x5616fc066f20 2024-03-19T01:12:42.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.824+0000 7f429a182740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f42901aab90 con 0x5616fc05e340 2024-03-19T01:12:42.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.824+0000 7f429a182740 10 client.0 _ll_register_callbacks cb 0x7f427c003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:42.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.824+0000 7f4295400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc066f20 0x5616fc0e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.824+0000 7f4295400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc066f20 0x5616fc0e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36504/0 (socket says 172.21.15.69:36504) 2024-03-19T01:12:42.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.824+0000 7f4295400640 1 -- 172.21.15.69:0/2767347372 learned_addr learned my addr 172.21.15.69:0/2767347372 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:42.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.824+0000 7f4294a00640 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc05da20 0x5616fc061140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f428fe00640 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f4295400640 1 -- 172.21.15.69:0/2767347372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 msgr2=0x5616fc061680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f4295400640 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc061680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f4295400640 1 -- 172.21.15.69:0/2767347372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc05da20 msgr2=0x5616fc061140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:42.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f4295400640 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc05da20 0x5616fc061140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f4295400640 1 -- 172.21.15.69:0/2767347372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616fc0e54b0 con 0x5616fc066f20 2024-03-19T01:12:42.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f4294a00640 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc05da20 0x5616fc061140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:42.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f4295400640 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc066f20 0x5616fc0e4f70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f428800f790 tx=0x7f428800fe80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f8f5db01c9dd42 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:42.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f4276000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:42.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f4276000640 1 -- 172.21.15.69:0/2767347372 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288031020 con 0x5616fc066f20 2024-03-19T01:12:42.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f429a182740 1 -- 172.21.15.69:0/2767347372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f428801f040 con 0x5616fc066f20 2024-03-19T01:12:42.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.825+0000 7f4276000640 1 -- 172.21.15.69:0/2767347372 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4288003970 con 0x5616fc066f20 2024-03-19T01:12:42.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f4276000640 1 -- 172.21.15.69:0/2767347372 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f428803c050 con 0x5616fc066f20 2024-03-19T01:12:42.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f429a182740 10 client.26204 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:42.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f429a182740 1 -- 172.21.15.69:0/2767347372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5616fc0e66b0 con 0x5616fc066f20 2024-03-19T01:12:42.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f4276000640 1 -- 172.21.15.69:0/2767347372 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f428801f040 con 0x5616fc066f20 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f4276000640 1 -- 172.21.15.69:0/2767347372 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f4288018070 con 0x5616fc066f20 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f429a182740 10 client.26204 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f429a182740 10 client.26204 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f429a182740 20 client.26204 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f429a182740 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5616fc0e6be0 0x5616fc0e9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f429a182740 4 client.26204 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f429a182740 1 -- 172.21.15.69:0/2767347372 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616fbfc9c10 con 0x5616fc0e6be0 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.826+0000 7f4294a00640 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5616fc0e6be0 0x5616fc0e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f4294a00640 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5616fc0e6be0 0x5616fc0e9000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db392b0f1f92d257 server_cookie=f6ce95e60e6fd5c8 in_seq=0 out_seq=0 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f4276000640 10 client.26204 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f4276000640 1 -- 172.21.15.69:0/2767347372 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616fbfc9c10 con 0x5616fc0e6be0 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f4276000640 10 client.26204 handle_command_reply: tid=0 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f429a182740 1 client.26204 shutdown 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f429a182740 1 -- 172.21.15.69:0/2767347372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc066f20 msgr2=0x5616fc0e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f429a182740 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc066f20 0x5616fc0e4f70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f428800f790 tx=0x7f428800fe80 comp rx=0 tx=0).stop 2024-03-19T01:12:42.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f429a182740 1 -- 172.21.15.69:0/2767347372 shutdown_connections 2024-03-19T01:12:42.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f429a182740 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5616fc0e6be0 0x5616fc0e9000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f429a182740 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616fc066f20 0x5616fc0e4f70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f429a182740 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5616fc05e340 0x5616fc061680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f429a182740 1 --2- 172.21.15.69:0/2767347372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616fc05da20 0x5616fc061140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f4276000640 0 client.26204 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:12:42.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.827+0000 7f429a182740 1 -- 172.21.15.69:0/2767347372 >> 172.21.15.69:0/2767347372 conn(0x5616fc058150 msgr2=0x5616fc0571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f429a182740 1 -- 172.21.15.69:0/2767347372 shutdown_connections 2024-03-19T01:12:42.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f429a182740 1 -- 172.21.15.69:0/2767347372 wait complete. 2024-03-19T01:12:42.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f429a182740 20 client.26204 trim_cache size 0 max 16384 2024-03-19T01:12:42.840 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:12:42.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f4297400640 1 -- 172.21.15.69:0/3839307146 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42780a29f0 msgr2=0x7f42780a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f4297400640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42780a29f0 0x7f42780a4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4284006010 tx=0x7f4284005dd0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f4297400640 1 -- 172.21.15.69:0/3839307146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290069340 msgr2=0x7f429019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:42.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f4297400640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290069340 0x7f429019a7f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f427c00e9c0 tx=0x7f427c00edc0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f4297400640 1 -- 172.21.15.69:0/3839307146 shutdown_connections 2024-03-19T01:12:42.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f4297400640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f429010e1c0 0x7f429019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f4297400640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42780a29f0 0x7f42780a4eb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f4297400640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4290069340 0x7f429019a7f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f4297400640 1 --2- 172.21.15.69:0/3839307146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4290068a20 0x7f429019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:42.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.828+0000 7f4297400640 1 -- 172.21.15.69:0/3839307146 >> 172.21.15.69:0/3839307146 conn(0x7f4290077820 msgr2=0x7f429010f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:42.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.829+0000 7f4297400640 1 -- 172.21.15.69:0/3839307146 shutdown_connections 2024-03-19T01:12:42.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:42.829+0000 7f4297400640 1 -- 172.21.15.69:0/3839307146 wait complete. 2024-03-19T01:12:42.846 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.030+0000 7f142fa00640 1 -- 172.21.15.69:0/1164372472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142810e1e0 msgr2=0x7f14281105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.030+0000 7f142fa00640 1 --2- 172.21.15.69:0/1164372472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142810e1e0 0x7f14281105d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1420009ca0 tx=0x7f142002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.030+0000 7f142fa00640 1 -- 172.21.15.69:0/1164372472 shutdown_connections 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.030+0000 7f142fa00640 1 --2- 172.21.15.69:0/1164372472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142810e1e0 0x7f14281105d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.030+0000 7f142fa00640 1 --2- 172.21.15.69:0/1164372472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1428100f50 0x7f142810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.030+0000 7f142fa00640 1 --2- 172.21.15.69:0/1164372472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1428100630 0x7f1428100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.030+0000 7f142fa00640 1 -- 172.21.15.69:0/1164372472 >> 172.21.15.69:0/1164372472 conn(0x7f1428077880 msgr2=0x7f1428077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.030+0000 7f142fa00640 1 -- 172.21.15.69:0/1164372472 shutdown_connections 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.030+0000 7f142fa00640 1 -- 172.21.15.69:0/1164372472 wait complete. 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.032+0000 7f142fa00640 1 Processor -- start 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.032+0000 7f142fa00640 1 -- start start 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.032+0000 7f142fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428100630 0x7f142819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.032+0000 7f142fa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1428100f50 0x7f142819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.032+0000 7f142fa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f142810e1e0 0x7f142819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.032+0000 7f142fa00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1428113df0 con 0x7f1428100f50 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.032+0000 7f142fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1428113c70 con 0x7f1428100630 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.032+0000 7f142fa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1428113f70 con 0x7f142810e1e0 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f142d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428100630 0x7f142819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f142d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428100630 0x7f142819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36526/0 (socket says 172.21.15.69:36526) 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f142d000640 1 -- 172.21.15.69:0/2849614972 learned_addr learned my addr 172.21.15.69:0/2849614972 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f1427e00640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1428100f50 0x7f142819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f142da00640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f142810e1e0 0x7f142819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f142d000640 1 -- 172.21.15.69:0/2849614972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f142810e1e0 msgr2=0x7f142819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f142d000640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f142810e1e0 0x7f142819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f142d000640 1 -- 172.21.15.69:0/2849614972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1428100f50 msgr2=0x7f142819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f142d000640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1428100f50 0x7f142819a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f142d000640 1 -- 172.21.15.69:0/2849614972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142819f2c0 con 0x7f1428100630 2024-03-19T01:12:43.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f142d000640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428100630 0x7f142819a2d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f141800d950 tx=0x7f141800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.033+0000 7f1425600640 1 -- 172.21.15.69:0/2849614972 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418017030 con 0x7f1428100630 2024-03-19T01:12:43.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.034+0000 7f1425600640 1 -- 172.21.15.69:0/2849614972 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1418004020 con 0x7f1428100630 2024-03-19T01:12:43.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.034+0000 7f142fa00640 1 -- 172.21.15.69:0/2849614972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142819f5b0 con 0x7f1428100630 2024-03-19T01:12:43.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.034+0000 7f1425600640 1 -- 172.21.15.69:0/2849614972 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180047a0 con 0x7f1428100630 2024-03-19T01:12:43.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.034+0000 7f142fa00640 1 -- 172.21.15.69:0/2849614972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14281a6e40 con 0x7f1428100630 2024-03-19T01:12:43.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.037+0000 7f1425600640 1 -- 172.21.15.69:0/2849614972 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f14180041d0 con 0x7f1428100630 2024-03-19T01:12:43.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f1432621740 1 Processor -- start 2024-03-19T01:12:43.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f1432621740 1 -- start start 2024-03-19T01:12:43.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f1425600640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f140c0a29f0 0x7f140c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f1432621740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd4ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f1432621740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4b340 0x56162cd539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f1432621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 0x56162cd56310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f1432621740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56162ccaf380 con 0x56162cd4b340 2024-03-19T01:12:43.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f1432621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56162cc57d20 con 0x56162cd53f20 2024-03-19T01:12:43.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f1432621740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56162ccab420 con 0x56162cd4aa20 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f1425600640 1 -- 172.21.15.69:0/2849614972 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f14180ceb20 con 0x7f1428100630 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f142d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd4ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.038+0000 7f142d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd4ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53976/0 (socket says 172.21.15.69:53976) 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f142d000640 1 -- 172.21.15.69:0/178560756 learned_addr learned my addr 172.21.15.69:0/178560756 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f1427e00640 1 --2- 172.21.15.69:0/178560756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4b340 0x56162cd539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f142da00640 1 --2- 172.21.15.69:0/178560756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 0x56162cd56310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f142d000640 1 -- 172.21.15.69:0/178560756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 msgr2=0x56162cd56310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f142d000640 1 --2- 172.21.15.69:0/178560756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 0x56162cd56310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f142d000640 1 -- 172.21.15.69:0/178560756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4b340 msgr2=0x56162cd539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f142d000640 1 --2- 172.21.15.69:0/178560756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4b340 0x56162cd539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f142d000640 1 -- 172.21.15.69:0/178560756 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162c9c33f0 con 0x56162cd4aa20 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f142da00640 1 --2- 172.21.15.69:0/178560756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 0x56162cd56310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f142da00640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f140c0a29f0 0x7f140c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f142d000640 1 --2- 172.21.15.69:0/178560756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd4ae00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1418005fd0 tx=0x7f141809a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dec1bcef87b572e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f141e000640 1 -- 172.21.15.69:0/178560756 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180a43c0 con 0x56162cd4aa20 2024-03-19T01:12:43.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f141e000640 1 -- 172.21.15.69:0/178560756 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141809ad60 con 0x56162cd4aa20 2024-03-19T01:12:43.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.039+0000 7f141e000640 1 -- 172.21.15.69:0/178560756 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180a3500 con 0x56162cd4aa20 2024-03-19T01:12:43.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.040+0000 7f1432621740 1 -- 172.21.15.69:0/178560756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 msgr2=0x56162cd4ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.040+0000 7f142da00640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f140c0a29f0 0x7f140c0a4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1420009bf0 tx=0x7f142002c0f0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.040+0000 7f1432621740 1 --2- 172.21.15.69:0/178560756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd4ae00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1418005fd0 tx=0x7f141809a6c0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.040+0000 7f1432621740 1 -- 172.21.15.69:0/178560756 shutdown_connections 2024-03-19T01:12:43.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.040+0000 7f1432621740 1 --2- 172.21.15.69:0/178560756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 0x56162cd56310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.040+0000 7f1432621740 1 --2- 172.21.15.69:0/178560756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4b340 0x56162cd539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.040+0000 7f1432621740 1 --2- 172.21.15.69:0/178560756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd4ae00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.040+0000 7f1432621740 1 -- 172.21.15.69:0/178560756 >> 172.21.15.69:0/178560756 conn(0x56162cd45150 msgr2=0x56162cd45530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.040+0000 7f1432621740 1 -- 172.21.15.69:0/178560756 shutdown_connections 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.040+0000 7f1432621740 1 -- 172.21.15.69:0/178560756 wait complete. 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.041+0000 7f1432621740 1 -- start start 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.041+0000 7f1432621740 1 Processor -- start 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.041+0000 7f1432621740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4aa20 0x56162cd51030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.041+0000 7f1432621740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4b340 0x56162cd51570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.041+0000 7f1432621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 0x56162cd51ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.041+0000 7f1432621740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56162cd56cb0 con 0x56162cd4aa20 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.041+0000 7f1432621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56162cc6b160 con 0x56162cd53f20 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.041+0000 7f1432621740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56162cd56e30 con 0x56162cd4b340 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.041+0000 7f1432621740 10 client.0 _ll_register_callbacks cb 0x56162cca7170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.041+0000 7f142d000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4aa20 0x56162cd51030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.041+0000 7f1427e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4b340 0x56162cd51570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f1427e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4b340 0x56162cd51570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53986/0 (socket says 172.21.15.69:53986) 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f1427e00640 1 -- 172.21.15.69:0/1746255089 learned_addr learned my addr 172.21.15.69:0/1746255089 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f142da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 0x56162cd51ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f142d000640 1 -- 172.21.15.69:0/1746255089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4b340 msgr2=0x56162cd51570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f142d000640 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4b340 0x56162cd51570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f142d000640 1 -- 172.21.15.69:0/1746255089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 msgr2=0x56162cd51ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f142d000640 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 0x56162cd51ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f142d000640 1 -- 172.21.15.69:0/1746255089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162ca8d7d0 con 0x56162cd4aa20 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f142da00640 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 0x56162cd51ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f142d000640 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4aa20 0x56162cd51030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1418009610 tx=0x7f1418097600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72b90cb1db27458 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f1412a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.042+0000 7f1412a00640 1 -- 172.21.15.69:0/1746255089 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180cc2e0 con 0x56162cd4aa20 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1412a00640 1 -- 172.21.15.69:0/1746255089 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f14180aa9f0 con 0x56162cd4aa20 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1432621740 1 -- 172.21.15.69:0/1746255089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141809ad60 con 0x56162cd4aa20 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1412a00640 1 -- 172.21.15.69:0/1746255089 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f14180b8050 con 0x56162cd4aa20 2024-03-19T01:12:43.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1432621740 10 client.16026 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1432621740 1 -- 172.21.15.69:0/1746255089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56162cd62fa0 con 0x56162cd4aa20 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1412a00640 1 -- 172.21.15.69:0/1746255089 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141809ad60 con 0x56162cd4aa20 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1412a00640 1 -- 172.21.15.69:0/1746255089 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1418099c20 con 0x56162cd4aa20 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1432621740 10 client.16026 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1432621740 10 client.16026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1432621740 20 client.16026 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1432621740 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56162cd642f0 0x56162cd666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1432621740 4 client.16026 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.043+0000 7f1432621740 1 -- 172.21.15.69:0/1746255089 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162ccbc8c0 con 0x56162cd642f0 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.044+0000 7f1427e00640 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56162cd642f0 0x56162cd666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.044+0000 7f1427e00640 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56162cd642f0 0x56162cd666e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af08563c5c8ca3b8 server_cookie=7066a756054f1c66 in_seq=0 out_seq=0 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.044+0000 7f1412a00640 10 client.16026 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1412a00640 1 -- 172.21.15.69:0/1746255089 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56162ccbc8c0 con 0x56162cd642f0 2024-03-19T01:12:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1412a00640 10 client.16026 handle_command_reply: tid=0 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1432621740 1 client.16026 shutdown 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1432621740 1 -- 172.21.15.69:0/1746255089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4aa20 msgr2=0x56162cd51030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1432621740 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4aa20 0x56162cd51030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1418009610 tx=0x7f1418097600 comp rx=0 tx=0).stop 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1432621740 1 -- 172.21.15.69:0/1746255089 shutdown_connections 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1432621740 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56162cd642f0 0x56162cd666e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1432621740 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd53f20 0x56162cd51ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1432621740 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4b340 0x56162cd51570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1432621740 1 --2- 172.21.15.69:0/1746255089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4aa20 0x56162cd51030 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1432621740 1 -- 172.21.15.69:0/1746255089 >> 172.21.15.69:0/1746255089 conn(0x56162cd45150 msgr2=0x56162cd46ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1412a00640 0 client.16026 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1432621740 1 -- 172.21.15.69:0/1746255089 shutdown_connections 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.045+0000 7f1432621740 1 -- 172.21.15.69:0/1746255089 wait complete. 2024-03-19T01:12:43.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.046+0000 7f1432621740 20 client.16026 trim_cache size 0 max 16384 2024-03-19T01:12:43.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.059+0000 7f1432621740 1 Processor -- start 2024-03-19T01:12:43.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.059+0000 7f1432621740 1 -- start start 2024-03-19T01:12:43.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f1432621740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd53760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f1432621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4b340 0x56162cd4f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f1432621740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd53f20 0x56162cd4fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f1432621740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b0fd0 con 0x56162cd53f20 2024-03-19T01:12:43.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f1432621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b2bd0 con 0x56162cd4b340 2024-03-19T01:12:43.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f1432621740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b16a0 con 0x56162cd4aa20 2024-03-19T01:12:43.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f1427e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4b340 0x56162cd4f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f142d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd53760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f142da00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd53f20 0x56162cd4fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f142d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd53760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53998/0 (socket says 172.21.15.69:53998) 2024-03-19T01:12:43.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f1427e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4b340 0x56162cd4f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36556/0 (socket says 172.21.15.69:36556) 2024-03-19T01:12:43.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f142d000640 1 -- 172.21.15.69:0/3489509523 learned_addr learned my addr 172.21.15.69:0/3489509523 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f142d000640 1 -- 172.21.15.69:0/3489509523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4b340 msgr2=0x56162cd4f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f142d000640 1 --2- 172.21.15.69:0/3489509523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4b340 0x56162cd4f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f142d000640 1 -- 172.21.15.69:0/3489509523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd53f20 msgr2=0x56162cd4fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f142d000640 1 --2- 172.21.15.69:0/3489509523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd53f20 0x56162cd4fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f142d000640 1 -- 172.21.15.69:0/3489509523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162cd44c80 con 0x56162cd4aa20 2024-03-19T01:12:43.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.060+0000 7f1427e00640 1 --2- 172.21.15.69:0/3489509523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4b340 0x56162cd4f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:43.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.061+0000 7f142d000640 1 --2- 172.21.15.69:0/3489509523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd53760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f141809aa60 tx=0x7f141800b510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60cde328d6c30ebf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.061+0000 7f141d600640 1 -- 172.21.15.69:0/3489509523 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180cc2e0 con 0x56162cd4aa20 2024-03-19T01:12:43.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.061+0000 7f141d600640 1 -- 172.21.15.69:0/3489509523 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14180a9070 con 0x56162cd4aa20 2024-03-19T01:12:43.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.061+0000 7f141d600640 1 -- 172.21.15.69:0/3489509523 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141809a1b0 con 0x56162cd4aa20 2024-03-19T01:12:43.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.062+0000 7f1432621740 1 -- 172.21.15.69:0/3489509523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 msgr2=0x56162cd53760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.062+0000 7f1432621740 1 --2- 172.21.15.69:0/3489509523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd53760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f141809aa60 tx=0x7f141800b510 comp rx=0 tx=0).stop 2024-03-19T01:12:43.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.062+0000 7f1432621740 1 -- 172.21.15.69:0/3489509523 shutdown_connections 2024-03-19T01:12:43.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.062+0000 7f1432621740 1 --2- 172.21.15.69:0/3489509523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd53f20 0x56162cd4fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.062+0000 7f1432621740 1 --2- 172.21.15.69:0/3489509523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4b340 0x56162cd4f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.062+0000 7f1432621740 1 --2- 172.21.15.69:0/3489509523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd4aa20 0x56162cd53760 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.062+0000 7f1432621740 1 -- 172.21.15.69:0/3489509523 >> 172.21.15.69:0/3489509523 conn(0x56162cd45150 msgr2=0x56162cd4c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.062+0000 7f1432621740 1 -- 172.21.15.69:0/3489509523 shutdown_connections 2024-03-19T01:12:43.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.062+0000 7f1432621740 1 -- 172.21.15.69:0/3489509523 wait complete. 2024-03-19T01:12:43.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.062+0000 7f1432621740 1 -- start start 2024-03-19T01:12:43.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.063+0000 7f1432621740 1 Processor -- start 2024-03-19T01:12:43.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.063+0000 7f1432621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4aa20 0x56162cd663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.063+0000 7f1432621740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4b340 0x56162cd668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.063+0000 7f1432621740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd53f20 0x56162cdc8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.063+0000 7f1432621740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b2440 con 0x56162cd4b340 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.063+0000 7f1432621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b1d70 con 0x56162cd4aa20 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.063+0000 7f1432621740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f14281aefb0 con 0x56162cd53f20 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.063+0000 7f1432621740 10 client.0 _ll_register_callbacks cb 0x7f141809ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.063+0000 7f142d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4aa20 0x56162cd663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.063+0000 7f142d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4aa20 0x56162cd663a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36560/0 (socket says 172.21.15.69:36560) 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.063+0000 7f142d000640 1 -- 172.21.15.69:0/203950561 learned_addr learned my addr 172.21.15.69:0/203950561 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f142da00640 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd53f20 0x56162cdc8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f1427e00640 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4b340 0x56162cd668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f142d000640 1 -- 172.21.15.69:0/203950561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd53f20 msgr2=0x56162cdc8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f142d000640 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd53f20 0x56162cdc8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f142d000640 1 -- 172.21.15.69:0/203950561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4b340 msgr2=0x56162cd668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f142d000640 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4b340 0x56162cd668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f142d000640 1 -- 172.21.15.69:0/203950561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162cdc87f0 con 0x56162cd4aa20 2024-03-19T01:12:43.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f142d000640 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4aa20 0x56162cd663a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f141809acf0 tx=0x7f141809a3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d080505f6e3b8077 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f1412a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:43.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f1412a00640 1 -- 172.21.15.69:0/203950561 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141807e060 con 0x56162cd4aa20 2024-03-19T01:12:43.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f1412a00640 1 -- 172.21.15.69:0/203950561 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1418079b80 con 0x56162cd4aa20 2024-03-19T01:12:43.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f1432621740 1 -- 172.21.15.69:0/203950561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14180a9070 con 0x56162cd4aa20 2024-03-19T01:12:43.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.064+0000 7f1412a00640 1 -- 172.21.15.69:0/203950561 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f14180cc2e0 con 0x56162cd4aa20 2024-03-19T01:12:43.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.065+0000 7f1432621740 10 client.26219 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:43.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.065+0000 7f1432621740 1 -- 172.21.15.69:0/203950561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56162cdc87f0 con 0x56162cd4aa20 2024-03-19T01:12:43.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.065+0000 7f1412a00640 1 -- 172.21.15.69:0/203950561 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14180a9070 con 0x56162cd4aa20 2024-03-19T01:12:43.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.065+0000 7f1412a00640 1 -- 172.21.15.69:0/203950561 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f14180aaa10 con 0x56162cd4aa20 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.065+0000 7f1432621740 10 client.26219 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.065+0000 7f1432621740 10 client.26219 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.065+0000 7f1432621740 20 client.26219 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.065+0000 7f1432621740 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56162cdc9d40 0x56162cdcc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.065+0000 7f1432621740 4 client.26219 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.065+0000 7f1432621740 1 -- 172.21.15.69:0/203950561 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56162ccb6c10 con 0x56162cdc9d40 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.065+0000 7f1427e00640 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56162cdc9d40 0x56162cdcc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1427e00640 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56162cdc9d40 0x56162cdcc130 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e554996294f57b58 server_cookie=64a51810130ad41b in_seq=0 out_seq=0 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1412a00640 10 client.26219 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1412a00640 1 -- 172.21.15.69:0/203950561 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56162ccb6c10 con 0x56162cdc9d40 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1412a00640 10 client.26219 handle_command_reply: tid=0 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1432621740 1 client.26219 shutdown 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1432621740 1 -- 172.21.15.69:0/203950561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4aa20 msgr2=0x56162cd663a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1432621740 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4aa20 0x56162cd663a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f141809acf0 tx=0x7f141809a3e0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1432621740 1 -- 172.21.15.69:0/203950561 shutdown_connections 2024-03-19T01:12:43.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1432621740 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56162cdc9d40 0x56162cdcc130 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1432621740 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56162cd53f20 0x56162cdc8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1432621740 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56162cd4b340 0x56162cd668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1432621740 1 --2- 172.21.15.69:0/203950561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cd4aa20 0x56162cd663a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1412a00640 0 client.26219 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:12:43.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1432621740 1 -- 172.21.15.69:0/203950561 >> 172.21.15.69:0/203950561 conn(0x56162cd45150 msgr2=0x56162cd4f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1432621740 1 -- 172.21.15.69:0/203950561 shutdown_connections 2024-03-19T01:12:43.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.066+0000 7f1432621740 1 -- 172.21.15.69:0/203950561 wait complete. 2024-03-19T01:12:43.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f1432621740 20 client.26219 trim_cache size 0 max 16384 2024-03-19T01:12:43.072 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:12:43.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 -- 172.21.15.69:0/2849614972 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f140c0a29f0 msgr2=0x7f140c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f140c0a29f0 0x7f140c0a4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1420009bf0 tx=0x7f142002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 -- 172.21.15.69:0/2849614972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428100630 msgr2=0x7f142819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428100630 0x7f142819a2d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f141800d950 tx=0x7f141800de10 comp rx=0 tx=0).stop 2024-03-19T01:12:43.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 -- 172.21.15.69:0/2849614972 shutdown_connections 2024-03-19T01:12:43.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f142810e1e0 0x7f142819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1428100f50 0x7f142819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f140c0a29f0 0x7f140c0a4eb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 --2- 172.21.15.69:0/2849614972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428100630 0x7f142819a2d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 -- 172.21.15.69:0/2849614972 >> 172.21.15.69:0/2849614972 conn(0x7f1428077880 msgr2=0x7f142810f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 -- 172.21.15.69:0/2849614972 shutdown_connections 2024-03-19T01:12:43.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.067+0000 7f142fa00640 1 -- 172.21.15.69:0/2849614972 wait complete. 2024-03-19T01:12:43.085 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:12:43.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.296+0000 7fac0e200640 1 -- 172.21.15.69:0/697851396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac0810bcc0 msgr2=0x7fac0810e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.296+0000 7fac0e200640 1 --2- 172.21.15.69:0/697851396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac0810bcc0 0x7fac0810e110 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fac00009900 tx=0x7fac0002bcc0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.296+0000 7fac0e200640 1 -- 172.21.15.69:0/697851396 shutdown_connections 2024-03-19T01:12:43.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.296+0000 7fac0e200640 1 --2- 172.21.15.69:0/697851396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac0810bcc0 0x7fac0810e110 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.296+0000 7fac0e200640 1 --2- 172.21.15.69:0/697851396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac08103210 0x7fac0810b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.296+0000 7fac0e200640 1 --2- 172.21.15.69:0/697851396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac081028f0 0x7fac08102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.296+0000 7fac0e200640 1 -- 172.21.15.69:0/697851396 >> 172.21.15.69:0/697851396 conn(0x7fac080fc070 msgr2=0x7fac080fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.296+0000 7fac0e200640 1 -- 172.21.15.69:0/697851396 shutdown_connections 2024-03-19T01:12:43.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.296+0000 7fac0e200640 1 -- 172.21.15.69:0/697851396 wait complete. 2024-03-19T01:12:43.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.298+0000 7fac0e200640 1 Processor -- start 2024-03-19T01:12:43.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.298+0000 7fac0e200640 1 -- start start 2024-03-19T01:12:43.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.298+0000 7fac0e200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac081028f0 0x7fac0819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.298+0000 7fac0e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac08103210 0x7fac0819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.298+0000 7fac0e200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac0810bcc0 0x7fac0819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.298+0000 7fac0e200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac08111ab0 con 0x7fac0810bcc0 2024-03-19T01:12:43.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.298+0000 7fac0e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac08111930 con 0x7fac08103210 2024-03-19T01:12:43.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.298+0000 7fac0e200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fac08111c30 con 0x7fac081028f0 2024-03-19T01:12:43.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac0ce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac081028f0 0x7fac0819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac08103210 0x7fac0819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac0d800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac0810bcc0 0x7fac0819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac0ce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac081028f0 0x7fac0819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:54026/0 (socket says 172.21.15.69:54026) 2024-03-19T01:12:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac0ce00640 1 -- 172.21.15.69:0/1400197473 learned_addr learned my addr 172.21.15.69:0/1400197473 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac07e00640 1 -- 172.21.15.69:0/1400197473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac081028f0 msgr2=0x7fac0819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac07e00640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac081028f0 0x7fac0819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac07e00640 1 -- 172.21.15.69:0/1400197473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac0810bcc0 msgr2=0x7fac0819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac07e00640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac0810bcc0 0x7fac0819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac07e00640 1 -- 172.21.15.69:0/1400197473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0819f180 con 0x7fac08103210 2024-03-19T01:12:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac0ce00640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac081028f0 0x7fac0819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac0d800640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac0810bcc0 0x7fac0819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.299+0000 7fac07e00640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac08103210 0x7fac0819a6d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fabf400e9f0 tx=0x7fabf400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.300+0000 7fac05600640 1 -- 172.21.15.69:0/1400197473 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf400ccc0 con 0x7fac08103210 2024-03-19T01:12:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.300+0000 7fac05600640 1 -- 172.21.15.69:0/1400197473 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf4010910 con 0x7fac08103210 2024-03-19T01:12:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.300+0000 7fac0e200640 1 -- 172.21.15.69:0/1400197473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0819f470 con 0x7fac08103210 2024-03-19T01:12:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.300+0000 7fac0e200640 1 -- 172.21.15.69:0/1400197473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac081a6d00 con 0x7fac08103210 2024-03-19T01:12:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.300+0000 7fac05600640 1 -- 172.21.15.69:0/1400197473 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4018960 con 0x7fac08103210 2024-03-19T01:12:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.304+0000 7fac125c1740 1 Processor -- start 2024-03-19T01:12:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac125c1740 1 -- start start 2024-03-19T01:12:43.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac125c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b85672fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac125c1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b8567bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac125c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 0x560b8567e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac125c1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560b855d7380 con 0x560b85673510 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac125c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b8557fd20 con 0x560b85672bf0 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac125c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560b855d3420 con 0x560b8567c0f0 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac0d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b85672fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac0d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b85672fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36592/0 (socket says 172.21.15.69:36592) 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac0d800640 1 -- 172.21.15.69:0/1710729387 learned_addr learned my addr 172.21.15.69:0/1710729387 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac0ce00640 1 --2- 172.21.15.69:0/1710729387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b8567bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.305+0000 7fac05600640 1 -- 172.21.15.69:0/1400197473 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fabf4010ac0 con 0x7fac08103210 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.306+0000 7fac07e00640 1 --2- 172.21.15.69:0/1710729387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 0x560b8567e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.306+0000 7fac0d800640 1 -- 172.21.15.69:0/1710729387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 msgr2=0x560b8567e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.306+0000 7fac0d800640 1 --2- 172.21.15.69:0/1710729387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 0x560b8567e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.306+0000 7fac0d800640 1 -- 172.21.15.69:0/1710729387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 msgr2=0x560b8567bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.306+0000 7fac0d800640 1 --2- 172.21.15.69:0/1710729387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b8567bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.306+0000 7fac0d800640 1 -- 172.21.15.69:0/1710729387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b852eb3f0 con 0x560b85672bf0 2024-03-19T01:12:43.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.306+0000 7fac07e00640 1 --2- 172.21.15.69:0/1710729387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 0x560b8567e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:12:43.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.306+0000 7fac0d800640 1 --2- 172.21.15.69:0/1710729387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b85672fd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fac0002c2d0 tx=0x7fac00005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf1b73f78a83d10 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.306+0000 7fabfa000640 1 -- 172.21.15.69:0/1710729387 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00043070 con 0x560b85672bf0 2024-03-19T01:12:43.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.306+0000 7fabfa000640 1 -- 172.21.15.69:0/1710729387 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0002c6e0 con 0x560b85672bf0 2024-03-19T01:12:43.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.306+0000 7fabfa000640 1 -- 172.21.15.69:0/1710729387 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00004f60 con 0x560b85672bf0 2024-03-19T01:12:43.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.307+0000 7fac05600640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fabf00a29f0 0x7fabf00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.307+0000 7fac05600640 1 -- 172.21.15.69:0/1400197473 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fabf40cf3f0 con 0x7fac08103210 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.307+0000 7fac0ce00640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fabf00a29f0 0x7fabf00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.307+0000 7fac125c1740 1 -- 172.21.15.69:0/1710729387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 msgr2=0x560b85672fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.307+0000 7fac125c1740 1 --2- 172.21.15.69:0/1710729387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b85672fd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fac0002c2d0 tx=0x7fac00005be0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.308+0000 7fac125c1740 1 -- 172.21.15.69:0/1710729387 shutdown_connections 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.308+0000 7fac125c1740 1 --2- 172.21.15.69:0/1710729387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 0x560b8567e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.308+0000 7fac125c1740 1 --2- 172.21.15.69:0/1710729387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b8567bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.308+0000 7fac125c1740 1 --2- 172.21.15.69:0/1710729387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b85672fd0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.308+0000 7fac125c1740 1 -- 172.21.15.69:0/1710729387 >> 172.21.15.69:0/1710729387 conn(0x560b8566d320 msgr2=0x560b8566d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.308+0000 7fac125c1740 1 -- 172.21.15.69:0/1710729387 shutdown_connections 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.308+0000 7fac125c1740 1 -- 172.21.15.69:0/1710729387 wait complete. 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.308+0000 7fac0ce00640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fabf00a29f0 0x7fabf00a4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x560b856726c0 tx=0x7fabfc009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.308+0000 7fac125c1740 1 -- start start 2024-03-19T01:12:43.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.309+0000 7fac125c1740 1 Processor -- start 2024-03-19T01:12:43.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.309+0000 7fac125c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b856790a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.309+0000 7fac125c1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b856795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.309+0000 7fac125c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 0x560b85679b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.309+0000 7fac125c1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560b8567ee80 con 0x560b85673510 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.309+0000 7fac125c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b85593160 con 0x560b85672bf0 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.309+0000 7fac125c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560b8567f000 con 0x560b8567c0f0 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.309+0000 7fac0ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b856790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.309+0000 7fac0ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b856790a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36600/0 (socket says 172.21.15.69:36600) 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.309+0000 7fac0ce00640 1 -- 172.21.15.69:0/2354916541 learned_addr learned my addr 172.21.15.69:0/2354916541 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.309+0000 7fac125c1740 10 client.0 _ll_register_callbacks cb 0x560b855cf170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.310+0000 7fac0d800640 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 0x560b85679b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.310+0000 7fac07e00640 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b856795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.310+0000 7fac0ce00640 1 -- 172.21.15.69:0/2354916541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 msgr2=0x560b85679b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.310+0000 7fac0ce00640 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 0x560b85679b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.310+0000 7fac0ce00640 1 -- 172.21.15.69:0/2354916541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 msgr2=0x560b856795e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.310+0000 7fac0ce00640 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b856795e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.310+0000 7fac0ce00640 1 -- 172.21.15.69:0/2354916541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b853b57d0 con 0x560b85672bf0 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.310+0000 7fac0d800640 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 0x560b85679b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.310+0000 7fac0ce00640 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b856790a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00fcb0 tx=0x7fabfc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1558dcf2a4c04d25 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.310+0000 7fabeea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.310+0000 7fabeea00640 1 -- 172.21.15.69:0/2354916541 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc011e50 con 0x560b85672bf0 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fac125c1740 1 -- 172.21.15.69:0/2354916541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac0002c6e0 con 0x560b85672bf0 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fabeea00640 1 -- 172.21.15.69:0/2354916541 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fabfc029440 con 0x560b85672bf0 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fabeea00640 1 -- 172.21.15.69:0/2354916541 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fabfc032050 con 0x560b85672bf0 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fac125c1740 10 client.26231 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fac125c1740 1 -- 172.21.15.69:0/2354916541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b8568b150 con 0x560b85672bf0 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fabeea00640 1 -- 172.21.15.69:0/2354916541 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0002c6e0 con 0x560b85672bf0 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fabeea00640 1 -- 172.21.15.69:0/2354916541 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fabfc027020 con 0x560b85672bf0 2024-03-19T01:12:43.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fac125c1740 10 client.26231 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fac125c1740 10 client.26231 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fac125c1740 20 client.26231 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fac125c1740 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b8568c600 0x560b8568e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fac125c1740 4 client.26231 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.311+0000 7fac125c1740 1 -- 172.21.15.69:0/2354916541 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b855e48c0 con 0x560b8568c600 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.312+0000 7fac07e00640 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b8568c600 0x560b8568e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.312+0000 7fabeea00640 10 client.26231 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.312+0000 7fac07e00640 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b8568c600 0x560b8568e9f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6930b94a56e82188 server_cookie=bdbcdb83fceecff8 in_seq=0 out_seq=0 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fabeea00640 1 -- 172.21.15.69:0/2354916541 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560b855e48c0 con 0x560b8568c600 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fabeea00640 10 client.26231 handle_command_reply: tid=0 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 1 client.26231 shutdown 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 1 -- 172.21.15.69:0/2354916541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 msgr2=0x560b856790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b856790a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00fcb0 tx=0x7fabfc004ab0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 1 -- 172.21.15.69:0/2354916541 shutdown_connections 2024-03-19T01:12:43.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b8567c0f0 0x560b85679b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b856795e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b8568c600 0x560b8568e9f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 1 --2- 172.21.15.69:0/2354916541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b85672bf0 0x560b856790a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fabeea00640 0 client.26231 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:12:43.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 1 -- 172.21.15.69:0/2354916541 >> 172.21.15.69:0/2354916541 conn(0x560b8566d320 msgr2=0x560b8566f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 1 -- 172.21.15.69:0/2354916541 shutdown_connections 2024-03-19T01:12:43.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 1 -- 172.21.15.69:0/2354916541 wait complete. 2024-03-19T01:12:43.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.313+0000 7fac125c1740 20 client.26231 trim_cache size 0 max 16384 2024-03-19T01:12:43.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.327+0000 7fac125c1740 1 Processor -- start 2024-03-19T01:12:43.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.327+0000 7fac125c1740 1 -- start start 2024-03-19T01:12:43.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac125c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85672bf0 0x560b8567b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac125c1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b85687cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac125c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b8567c0f0 0x560b856942e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac125c1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fabe800ff40 con 0x560b85673510 2024-03-19T01:12:43.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac125c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8011b40 con 0x560b8567c0f0 2024-03-19T01:12:43.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac125c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8010610 con 0x560b85672bf0 2024-03-19T01:12:43.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac0ce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85672bf0 0x560b8567b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac0ce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85672bf0 0x560b8567b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:54042/0 (socket says 172.21.15.69:54042) 2024-03-19T01:12:43.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac0ce00640 1 -- 172.21.15.69:0/3039722142 learned_addr learned my addr 172.21.15.69:0/3039722142 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac0d800640 1 --2- 172.21.15.69:0/3039722142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b8567c0f0 0x560b856942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac07e00640 1 --2- 172.21.15.69:0/3039722142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b85687cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac0ce00640 1 -- 172.21.15.69:0/3039722142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b8567c0f0 msgr2=0x560b856942e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac0ce00640 1 --2- 172.21.15.69:0/3039722142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b8567c0f0 0x560b856942e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac0ce00640 1 -- 172.21.15.69:0/3039722142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 msgr2=0x560b85687cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac0ce00640 1 --2- 172.21.15.69:0/3039722142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b85687cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.328+0000 7fac0ce00640 1 -- 172.21.15.69:0/3039722142 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b8568ed60 con 0x560b85672bf0 2024-03-19T01:12:43.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.329+0000 7fac0ce00640 1 --2- 172.21.15.69:0/3039722142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85672bf0 0x560b8567b7d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fabfc01f8a0 tx=0x7fabfc00f8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63ccab3014c4ba7d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.329+0000 7fabf9600640 1 -- 172.21.15.69:0/3039722142 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc03b070 con 0x560b85672bf0 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.329+0000 7fabf9600640 1 -- 172.21.15.69:0/3039722142 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabfc01f420 con 0x560b85672bf0 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.329+0000 7fabf9600640 1 -- 172.21.15.69:0/3039722142 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc01fe60 con 0x560b85672bf0 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.329+0000 7fac125c1740 1 -- 172.21.15.69:0/3039722142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85672bf0 msgr2=0x560b8567b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.329+0000 7fac125c1740 1 --2- 172.21.15.69:0/3039722142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85672bf0 0x560b8567b7d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fabfc01f8a0 tx=0x7fabfc00f8b0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.329+0000 7fac125c1740 1 -- 172.21.15.69:0/3039722142 shutdown_connections 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.329+0000 7fac125c1740 1 --2- 172.21.15.69:0/3039722142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b8567c0f0 0x560b856942e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.329+0000 7fac125c1740 1 --2- 172.21.15.69:0/3039722142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85673510 0x560b85687cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.329+0000 7fac125c1740 1 --2- 172.21.15.69:0/3039722142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85672bf0 0x560b8567b7d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.329+0000 7fac125c1740 1 -- 172.21.15.69:0/3039722142 >> 172.21.15.69:0/3039722142 conn(0x560b8566d320 msgr2=0x560b8566d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.330+0000 7fac125c1740 1 -- 172.21.15.69:0/3039722142 shutdown_connections 2024-03-19T01:12:43.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.330+0000 7fac125c1740 1 -- 172.21.15.69:0/3039722142 wait complete. 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.330+0000 7fac125c1740 1 -- start start 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.330+0000 7fac125c1740 1 Processor -- start 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.330+0000 7fac125c1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85672bf0 0x560b856764c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.330+0000 7fac125c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85673510 0x560b85676a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac125c1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b8567c0f0 0x560b856f1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac125c1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fabe80113b0 con 0x560b85672bf0 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac125c1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8010ce0 con 0x560b8567c0f0 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac125c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fabe800df20 con 0x560b85673510 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac125c1740 10 client.0 _ll_register_callbacks cb 0x7fabfc012790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac0ce00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85672bf0 0x560b856764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac07e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85673510 0x560b85676a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac0d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b8567c0f0 0x560b856f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac0ce00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85672bf0 0x560b856764c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48814/0 (socket says 172.21.15.69:48814) 2024-03-19T01:12:43.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac07e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85673510 0x560b85676a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:54050/0 (socket says 172.21.15.69:54050) 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac0ce00640 1 -- 172.21.15.69:0/333303878 learned_addr learned my addr 172.21.15.69:0/333303878 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac07e00640 1 -- 172.21.15.69:0/333303878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b8567c0f0 msgr2=0x560b856f1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac07e00640 1 --2- 172.21.15.69:0/333303878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b8567c0f0 0x560b856f1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac07e00640 1 -- 172.21.15.69:0/333303878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85672bf0 msgr2=0x560b856764c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac07e00640 1 --2- 172.21.15.69:0/333303878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85672bf0 0x560b856764c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac07e00640 1 -- 172.21.15.69:0/333303878 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b856f1f20 con 0x560b85673510 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fac07e00640 1 --2- 172.21.15.69:0/333303878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85673510 0x560b85676a00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fabf40cc720 tx=0x7fabf4086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57d7a0593b638713 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.331+0000 7fabeea00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fabeea00640 1 -- 172.21.15.69:0/333303878 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf409f780 con 0x560b85673510 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fabeea00640 1 -- 172.21.15.69:0/333303878 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fabf4097d50 con 0x560b85673510 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fac125c1740 1 -- 172.21.15.69:0/333303878 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabfc01f420 con 0x560b85673510 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fabeea00640 1 -- 172.21.15.69:0/333303878 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fabf40a3dd0 con 0x560b85673510 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fac125c1740 10 client.26218 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fac125c1740 1 -- 172.21.15.69:0/333303878 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b856f3250 con 0x560b85673510 2024-03-19T01:12:43.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fabeea00640 1 -- 172.21.15.69:0/333303878 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabfc01f420 con 0x560b85673510 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fabeea00640 1 -- 172.21.15.69:0/333303878 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fabf409ab30 con 0x560b85673510 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fac125c1740 10 client.26218 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fac125c1740 10 client.26218 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fac125c1740 20 client.26218 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fac125c1740 1 --2- 172.21.15.69:0/333303878 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b856f3a20 0x560b856f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fac125c1740 4 client.26218 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.332+0000 7fac125c1740 1 -- 172.21.15.69:0/333303878 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b855dec10 con 0x560b856f3a20 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.333+0000 7fac0ce00640 1 --2- 172.21.15.69:0/333303878 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b856f3a20 0x560b856f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.333+0000 7fac0ce00640 1 --2- 172.21.15.69:0/333303878 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b856f3a20 0x560b856f5e40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0b7de0e8bca4dbd server_cookie=362b5f26da3b3123 in_seq=0 out_seq=0 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.333+0000 7fabeea00640 10 client.26218 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.333+0000 7fabeea00640 1 -- 172.21.15.69:0/333303878 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b855dec10 con 0x560b856f3a20 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.333+0000 7fabeea00640 10 client.26218 handle_command_reply: tid=0 2024-03-19T01:12:43.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.333+0000 7fac125c1740 1 client.26218 shutdown 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fac125c1740 1 -- 172.21.15.69:0/333303878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85673510 msgr2=0x560b85676a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fac125c1740 1 --2- 172.21.15.69:0/333303878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85673510 0x560b85676a00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fabf40cc720 tx=0x7fabf4086040 comp rx=0 tx=0).stop 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fac125c1740 1 -- 172.21.15.69:0/333303878 shutdown_connections 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fac125c1740 1 --2- 172.21.15.69:0/333303878 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b856f3a20 0x560b856f5e40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fac125c1740 1 --2- 172.21.15.69:0/333303878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b8567c0f0 0x560b856f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fabeea00640 0 client.26218 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fac125c1740 1 --2- 172.21.15.69:0/333303878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b85673510 0x560b85676a00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fac125c1740 1 --2- 172.21.15.69:0/333303878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b85672bf0 0x560b856764c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fac125c1740 1 -- 172.21.15.69:0/333303878 >> 172.21.15.69:0/333303878 conn(0x560b8566d320 msgr2=0x560b85679040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fac125c1740 1 -- 172.21.15.69:0/333303878 shutdown_connections 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fac125c1740 1 -- 172.21.15.69:0/333303878 wait complete. 2024-03-19T01:12:43.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.334+0000 7fac125c1740 20 client.26218 trim_cache size 0 max 16384 2024-03-19T01:12:43.340 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:12:43.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 -- 172.21.15.69:0/1400197473 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fabf00a29f0 msgr2=0x7fabf00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fabf00a29f0 0x7fabf00a4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x560b856726c0 tx=0x7fabfc009500 comp rx=0 tx=0).stop 2024-03-19T01:12:43.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 -- 172.21.15.69:0/1400197473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac08103210 msgr2=0x7fac0819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac08103210 0x7fac0819a6d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fabf400e9f0 tx=0x7fabf400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 -- 172.21.15.69:0/1400197473 shutdown_connections 2024-03-19T01:12:43.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac0810bcc0 0x7fac0819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fabf00a29f0 0x7fabf00a4eb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac08103210 0x7fac0819a6d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 --2- 172.21.15.69:0/1400197473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac081028f0 0x7fac0819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 -- 172.21.15.69:0/1400197473 >> 172.21.15.69:0/1400197473 conn(0x7fac080fc070 msgr2=0x7fac0810def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 -- 172.21.15.69:0/1400197473 shutdown_connections 2024-03-19T01:12:43.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.335+0000 7fac0e200640 1 -- 172.21.15.69:0/1400197473 wait complete. 2024-03-19T01:12:43.351 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:12:43.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.559+0000 7f4149c00640 1 -- 172.21.15.69:0/1958222199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41441017b0 msgr2=0x7f4144101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.559+0000 7f4149c00640 1 --2- 172.21.15.69:0/1958222199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41441017b0 0x7f4144101c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4130009930 tx=0x7f413002bc20 comp rx=0 tx=0).stop 2024-03-19T01:12:43.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.559+0000 7f4149c00640 1 -- 172.21.15.69:0/1958222199 shutdown_connections 2024-03-19T01:12:43.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.559+0000 7f4149c00640 1 --2- 172.21.15.69:0/1958222199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4144102170 0x7f414410e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.559+0000 7f4149c00640 1 --2- 172.21.15.69:0/1958222199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41441017b0 0x7f4144101c30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.559+0000 7f4149c00640 1 --2- 172.21.15.69:0/1958222199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41441077c0 0x7f4144107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.559+0000 7f4149c00640 1 -- 172.21.15.69:0/1958222199 >> 172.21.15.69:0/1958222199 conn(0x7f41440fd260 msgr2=0x7f41440ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.559+0000 7f4149c00640 1 -- 172.21.15.69:0/1958222199 shutdown_connections 2024-03-19T01:12:43.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.559+0000 7f4149c00640 1 -- 172.21.15.69:0/1958222199 wait complete. 2024-03-19T01:12:43.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.560+0000 7f4149c00640 1 Processor -- start 2024-03-19T01:12:43.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.560+0000 7f4149c00640 1 -- start start 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.560+0000 7f4149c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41441017b0 0x7f4144073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.560+0000 7f4149c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4144102170 0x7f4144074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4149c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41441077c0 0x7f41440745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4149c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4144111c20 con 0x7f41441017b0 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4149c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4144111aa0 con 0x7f4144102170 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4149c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4144111da0 con 0x7f41441077c0 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4142a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41441017b0 0x7f4144073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4142000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4144102170 0x7f4144074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4143400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41441077c0 0x7f41440745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4142000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4144102170 0x7f4144074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36634/0 (socket says 172.21.15.69:36634) 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4142000640 1 -- 172.21.15.69:0/2348597276 learned_addr learned my addr 172.21.15.69:0/2348597276 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4142a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41441017b0 0x7f4144073b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48834/0 (socket says 172.21.15.69:48834) 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4143400640 1 -- 172.21.15.69:0/2348597276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4144102170 msgr2=0x7f4144074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4143400640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4144102170 0x7f4144074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4143400640 1 -- 172.21.15.69:0/2348597276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41441017b0 msgr2=0x7f4144073b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4143400640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41441017b0 0x7f4144073b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4143400640 1 -- 172.21.15.69:0/2348597276 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41441a6600 con 0x7f41441077c0 2024-03-19T01:12:43.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.561+0000 7f4143400640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41441077c0 0x7f41440745a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f413c009fd0 tx=0x7f413c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.562+0000 7f413b400640 1 -- 172.21.15.69:0/2348597276 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c00ce20 con 0x7f41441077c0 2024-03-19T01:12:43.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.562+0000 7f413b400640 1 -- 172.21.15.69:0/2348597276 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413c005020 con 0x7f41441077c0 2024-03-19T01:12:43.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.562+0000 7f4149c00640 1 -- 172.21.15.69:0/2348597276 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41441a68f0 con 0x7f41441077c0 2024-03-19T01:12:43.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.562+0000 7f413b400640 1 -- 172.21.15.69:0/2348597276 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c010720 con 0x7f41441077c0 2024-03-19T01:12:43.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.562+0000 7f4149c00640 1 -- 172.21.15.69:0/2348597276 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41441a6e40 con 0x7f41441077c0 2024-03-19T01:12:43.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.565+0000 7f414c7f1740 1 Processor -- start 2024-03-19T01:12:43.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.565+0000 7f413b400640 1 -- 172.21.15.69:0/2348597276 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f413c005540 con 0x7f41441077c0 2024-03-19T01:12:43.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.565+0000 7f414c7f1740 1 -- start start 2024-03-19T01:12:43.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.565+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 0x55d0b4e7de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.565+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7e340 0x55d0b4e869e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.565+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e89310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.565+0000 7f414c7f1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4de2380 con 0x55d0b4e7e340 2024-03-19T01:12:43.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.565+0000 7f414c7f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4d8ad20 con 0x55d0b4e86f20 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.565+0000 7f414c7f1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4dde420 con 0x55d0b4e7da20 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.566+0000 7f4142000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 0x55d0b4e7de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.566+0000 7f4142000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 0x55d0b4e7de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:54078/0 (socket says 172.21.15.69:54078) 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.566+0000 7f4142000640 1 -- 172.21.15.69:0/1165447672 learned_addr learned my addr 172.21.15.69:0/1165447672 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.566+0000 7f4142a00640 1 --2- 172.21.15.69:0/1165447672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7e340 0x55d0b4e869e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.566+0000 7f4142000640 1 --2- 172.21.15.69:0/1165447672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e89310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.566+0000 7f4142a00640 1 -- 172.21.15.69:0/1165447672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 msgr2=0x55d0b4e7de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.566+0000 7f4142a00640 1 --2- 172.21.15.69:0/1165447672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 0x55d0b4e7de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.566+0000 7f4142a00640 1 -- 172.21.15.69:0/1165447672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 msgr2=0x55d0b4e89310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.566+0000 7f4142a00640 1 --2- 172.21.15.69:0/1165447672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e89310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.566+0000 7f4142a00640 1 -- 172.21.15.69:0/1165447672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b4af63f0 con 0x55d0b4e7e340 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.566+0000 7f413b400640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f41280a2970 0x7f41280a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.567+0000 7f4142a00640 1 --2- 172.21.15.69:0/1165447672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7e340 0x55d0b4e869e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4134009c20 tx=0x7f4134011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56a66f2e372e285 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.567+0000 7f413b400640 1 -- 172.21.15.69:0/2348597276 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f413c0cf840 con 0x7f41441077c0 2024-03-19T01:12:43.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.567+0000 7f4142000640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f41280a2970 0x7f41280a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.567+0000 7f412fe00640 1 -- 172.21.15.69:0/1165447672 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134004050 con 0x55d0b4e7e340 2024-03-19T01:12:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.567+0000 7f412fe00640 1 -- 172.21.15.69:0/1165447672 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4134005500 con 0x55d0b4e7e340 2024-03-19T01:12:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.567+0000 7f412fe00640 1 -- 172.21.15.69:0/1165447672 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134004d40 con 0x55d0b4e7e340 2024-03-19T01:12:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.568+0000 7f414c7f1740 1 -- 172.21.15.69:0/1165447672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7e340 msgr2=0x55d0b4e869e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.568+0000 7f4142000640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f41280a2970 0x7f41280a4e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55d0b4e792e0 tx=0x7f413002bc20 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.568+0000 7f414c7f1740 1 --2- 172.21.15.69:0/1165447672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7e340 0x55d0b4e869e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4134009c20 tx=0x7f4134011810 comp rx=0 tx=0).stop 2024-03-19T01:12:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.568+0000 7f414c7f1740 1 -- 172.21.15.69:0/1165447672 shutdown_connections 2024-03-19T01:12:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.568+0000 7f414c7f1740 1 --2- 172.21.15.69:0/1165447672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e89310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.568+0000 7f414c7f1740 1 --2- 172.21.15.69:0/1165447672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7e340 0x55d0b4e869e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.568+0000 7f414c7f1740 1 --2- 172.21.15.69:0/1165447672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 0x55d0b4e7de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.568+0000 7f414c7f1740 1 -- 172.21.15.69:0/1165447672 >> 172.21.15.69:0/1165447672 conn(0x55d0b4e78150 msgr2=0x55d0b4e78530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.568+0000 7f414c7f1740 1 -- 172.21.15.69:0/1165447672 shutdown_connections 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.568+0000 7f414c7f1740 1 -- 172.21.15.69:0/1165447672 wait complete. 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.568+0000 7f414c7f1740 1 -- start start 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f414c7f1740 1 Processor -- start 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 0x55d0b4e83f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7e340 0x55d0b4e844d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e84a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f414c7f1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4e89d10 con 0x55d0b4e7e340 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f414c7f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4d9e160 con 0x55d0b4e86f20 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f414c7f1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4e89e90 con 0x55d0b4e7da20 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f414c7f1740 10 client.0 _ll_register_callbacks cb 0x55d0b4dda170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f4142a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 0x55d0b4e83f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f4142a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 0x55d0b4e83f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:54086/0 (socket says 172.21.15.69:54086) 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f4142a00640 1 -- 172.21.15.69:0/1890091018 learned_addr learned my addr 172.21.15.69:0/1890091018 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f4143400640 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e84a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.569+0000 7f4142000640 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7e340 0x55d0b4e844d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.570+0000 7f4142a00640 1 -- 172.21.15.69:0/1890091018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 msgr2=0x55d0b4e84a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.570+0000 7f4142a00640 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e84a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.570+0000 7f4142a00640 1 -- 172.21.15.69:0/1890091018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7e340 msgr2=0x55d0b4e844d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.570+0000 7f4142a00640 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7e340 0x55d0b4e844d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.570+0000 7f4142a00640 1 -- 172.21.15.69:0/1890091018 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b4bc07d0 con 0x55d0b4e7da20 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.570+0000 7f4142a00640 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 0x55d0b4e83f90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f413400a000 tx=0x7f4134004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd4bf3c035b31892 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.570+0000 7f412cc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:12:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.570+0000 7f412cc00640 1 -- 172.21.15.69:0/1890091018 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134029070 con 0x55d0b4e7da20 2024-03-19T01:12:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.570+0000 7f412cc00640 1 -- 172.21.15.69:0/1890091018 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4134003a00 con 0x55d0b4e7da20 2024-03-19T01:12:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.570+0000 7f414c7f1740 1 -- 172.21.15.69:0/1890091018 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4134005500 con 0x55d0b4e7da20 2024-03-19T01:12:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.571+0000 7f412cc00640 1 -- 172.21.15.69:0/1890091018 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4134035050 con 0x55d0b4e7da20 2024-03-19T01:12:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.571+0000 7f414c7f1740 10 client.26230 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.571+0000 7f414c7f1740 1 -- 172.21.15.69:0/1890091018 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d0b4e96030 con 0x55d0b4e7da20 2024-03-19T01:12:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.571+0000 7f412cc00640 1 -- 172.21.15.69:0/1890091018 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4134005500 con 0x55d0b4e7da20 2024-03-19T01:12:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.571+0000 7f412cc00640 1 -- 172.21.15.69:0/1890091018 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f4134023430 con 0x55d0b4e7da20 2024-03-19T01:12:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.571+0000 7f414c7f1740 10 client.26230 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.571+0000 7f414c7f1740 10 client.26230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.571+0000 7f414c7f1740 20 client.26230 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.571+0000 7f414c7f1740 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d0b4e973e0 0x55d0b4e997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.571+0000 7f414c7f1740 4 client.26230 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.571+0000 7f414c7f1740 1 -- 172.21.15.69:0/1890091018 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0b4def8c0 con 0x55d0b4e973e0 2024-03-19T01:12:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.572+0000 7f4142000640 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d0b4e973e0 0x55d0b4e997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.572+0000 7f4142000640 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d0b4e973e0 0x55d0b4e997d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c87adf18d3c3168 server_cookie=e730160e6f338af8 in_seq=0 out_seq=0 2024-03-19T01:12:43.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.572+0000 7f412cc00640 10 client.26230 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:43.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f412cc00640 1 -- 172.21.15.69:0/1890091018 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d0b4def8c0 con 0x55d0b4e973e0 2024-03-19T01:12:43.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f412cc00640 10 client.26230 handle_command_reply: tid=0 2024-03-19T01:12:43.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f414c7f1740 1 client.26230 shutdown 2024-03-19T01:12:43.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f414c7f1740 1 -- 172.21.15.69:0/1890091018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 msgr2=0x55d0b4e83f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f414c7f1740 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 0x55d0b4e83f90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f413400a000 tx=0x7f4134004580 comp rx=0 tx=0).stop 2024-03-19T01:12:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f414c7f1740 1 -- 172.21.15.69:0/1890091018 shutdown_connections 2024-03-19T01:12:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f414c7f1740 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d0b4e973e0 0x55d0b4e997d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f414c7f1740 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e84a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f414c7f1740 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7e340 0x55d0b4e844d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f414c7f1740 1 --2- 172.21.15.69:0/1890091018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7da20 0x55d0b4e83f90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f414c7f1740 1 -- 172.21.15.69:0/1890091018 >> 172.21.15.69:0/1890091018 conn(0x55d0b4e78150 msgr2=0x55d0b4e79ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.575+0000 7f412cc00640 0 client.26230 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.576+0000 7f414c7f1740 1 -- 172.21.15.69:0/1890091018 shutdown_connections 2024-03-19T01:12:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.576+0000 7f414c7f1740 1 -- 172.21.15.69:0/1890091018 wait complete. 2024-03-19T01:12:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.576+0000 7f414c7f1740 20 client.26230 trim_cache size 0 max 16384 2024-03-19T01:12:43.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.603+0000 7f414c7f1740 1 Processor -- start 2024-03-19T01:12:43.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.603+0000 7f414c7f1740 1 -- start start 2024-03-19T01:12:43.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.603+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7da20 0x55d0b4e866c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.603+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 0x55d0b4e92ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.603+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e9ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.603+0000 7f414c7f1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f41441b0f00 con 0x55d0b4e7da20 2024-03-19T01:12:43.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.603+0000 7f414c7f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41441b2b00 con 0x55d0b4e86f20 2024-03-19T01:12:43.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.603+0000 7f414c7f1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f41441b15d0 con 0x55d0b4e7e340 2024-03-19T01:12:43.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.604+0000 7f4142a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7da20 0x55d0b4e866c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.604+0000 7f4142000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 0x55d0b4e92ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.604+0000 7f4143400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e9ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.604+0000 7f4142a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7da20 0x55d0b4e866c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48868/0 (socket says 172.21.15.69:48868) 2024-03-19T01:12:43.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.604+0000 7f4142a00640 1 -- 172.21.15.69:0/2410803161 learned_addr learned my addr 172.21.15.69:0/2410803161 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.604+0000 7f4142000640 1 -- 172.21.15.69:0/2410803161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 msgr2=0x55d0b4e9ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.604+0000 7f4142000640 1 --2- 172.21.15.69:0/2410803161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e9ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.604+0000 7f4142000640 1 -- 172.21.15.69:0/2410803161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7da20 msgr2=0x55d0b4e866c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.604+0000 7f4142000640 1 --2- 172.21.15.69:0/2410803161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7da20 0x55d0b4e866c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.604+0000 7f4142000640 1 -- 172.21.15.69:0/2410803161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b4e89bf0 con 0x55d0b4e7e340 2024-03-19T01:12:43.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.604+0000 7f4142000640 1 --2- 172.21.15.69:0/2410803161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 0x55d0b4e92ae0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4130005c40 tx=0x7f4130005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b8d545d29074dfa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f412f400640 1 -- 172.21.15.69:0/2410803161 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413004a070 con 0x55d0b4e7e340 2024-03-19T01:12:43.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f412f400640 1 -- 172.21.15.69:0/2410803161 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4130060070 con 0x55d0b4e7e340 2024-03-19T01:12:43.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f412f400640 1 -- 172.21.15.69:0/2410803161 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130004ee0 con 0x55d0b4e7e340 2024-03-19T01:12:43.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f414c7f1740 1 -- 172.21.15.69:0/2410803161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 msgr2=0x55d0b4e92ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f414c7f1740 1 --2- 172.21.15.69:0/2410803161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 0x55d0b4e92ae0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4130005c40 tx=0x7f4130005a20 comp rx=0 tx=0).stop 2024-03-19T01:12:43.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f414c7f1740 1 -- 172.21.15.69:0/2410803161 shutdown_connections 2024-03-19T01:12:43.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f414c7f1740 1 --2- 172.21.15.69:0/2410803161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e9ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f414c7f1740 1 --2- 172.21.15.69:0/2410803161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 0x55d0b4e92ae0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f414c7f1740 1 --2- 172.21.15.69:0/2410803161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7da20 0x55d0b4e866c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f414c7f1740 1 -- 172.21.15.69:0/2410803161 >> 172.21.15.69:0/2410803161 conn(0x55d0b4e78150 msgr2=0x55d0b4e993f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f414c7f1740 1 -- 172.21.15.69:0/2410803161 shutdown_connections 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.605+0000 7f414c7f1740 1 -- 172.21.15.69:0/2410803161 wait complete. 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.606+0000 7f414c7f1740 1 -- start start 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.606+0000 7f414c7f1740 1 Processor -- start 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.606+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7da20 0x55d0b4ef5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.606+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 0x55d0b4ef5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.606+0000 7f414c7f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e8a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.606+0000 7f414c7f1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f41441b2370 con 0x55d0b4e7da20 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.606+0000 7f414c7f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41441b1ca0 con 0x55d0b4e86f20 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.606+0000 7f414c7f1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f41441aeee0 con 0x55d0b4e7e340 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.606+0000 7f414c7f1740 10 client.0 _ll_register_callbacks cb 0x7f4130009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4143400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e8a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4142000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 0x55d0b4ef5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4143400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e8a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36680/0 (socket says 172.21.15.69:36680) 2024-03-19T01:12:43.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4143400640 1 -- 172.21.15.69:0/2038807232 learned_addr learned my addr 172.21.15.69:0/2038807232 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4142a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7da20 0x55d0b4ef5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4142000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 0x55d0b4ef5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:54102/0 (socket says 172.21.15.69:54102) 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4143400640 1 -- 172.21.15.69:0/2038807232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 msgr2=0x55d0b4ef5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4143400640 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 0x55d0b4ef5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4143400640 1 -- 172.21.15.69:0/2038807232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7da20 msgr2=0x55d0b4ef5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4143400640 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7da20 0x55d0b4ef5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4143400640 1 -- 172.21.15.69:0/2038807232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b4e8a960 con 0x55d0b4e86f20 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4142000640 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 0x55d0b4ef5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f4143400640 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e8a270 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f413c09b7e0 tx=0x7f413c09fcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e01c89c962dbb69 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f412cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.607+0000 7f412cc00640 1 -- 172.21.15.69:0/2038807232 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c0a3df0 con 0x55d0b4e86f20 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f414c7f1740 1 -- 172.21.15.69:0/2038807232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4130060070 con 0x55d0b4e86f20 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f412cc00640 1 -- 172.21.15.69:0/2038807232 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f413c0a3f90 con 0x55d0b4e86f20 2024-03-19T01:12:43.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f412cc00640 1 -- 172.21.15.69:0/2038807232 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f413c0ba2b0 con 0x55d0b4e86f20 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f414c7f1740 10 client.26255 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f414c7f1740 1 -- 172.21.15.69:0/2038807232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d0b4efd0e0 con 0x55d0b4e86f20 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f412cc00640 1 -- 172.21.15.69:0/2038807232 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4130060070 con 0x55d0b4e86f20 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f412cc00640 1 -- 172.21.15.69:0/2038807232 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f413c0cf4c0 con 0x55d0b4e86f20 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f414c7f1740 10 client.26255 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f414c7f1740 10 client.26255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f414c7f1740 20 client.26255 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f414c7f1740 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d0b4efd890 0x55d0b4effcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f414c7f1740 4 client.26255 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.608+0000 7f414c7f1740 1 -- 172.21.15.69:0/2038807232 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0b4de9c10 con 0x55d0b4efd890 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.609+0000 7f4142a00640 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d0b4efd890 0x55d0b4effcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.609+0000 7f412cc00640 10 client.26255 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.609+0000 7f4142a00640 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d0b4efd890 0x55d0b4effcb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47f060eacd4f89e server_cookie=bd93f66f03e2d8bc in_seq=0 out_seq=0 2024-03-19T01:12:43.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f412cc00640 1 -- 172.21.15.69:0/2038807232 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0b4de9c10 con 0x55d0b4efd890 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f412cc00640 10 client.26255 handle_command_reply: tid=0 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f414c7f1740 1 client.26255 shutdown 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f414c7f1740 1 -- 172.21.15.69:0/2038807232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 msgr2=0x55d0b4e8a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f414c7f1740 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e8a270 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f413c09b7e0 tx=0x7f413c09fcf0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f414c7f1740 1 -- 172.21.15.69:0/2038807232 shutdown_connections 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f414c7f1740 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d0b4efd890 0x55d0b4effcb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f414c7f1740 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0b4e86f20 0x55d0b4e8a270 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f414c7f1740 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d0b4e7e340 0x55d0b4ef5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f414c7f1740 1 --2- 172.21.15.69:0/2038807232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d0b4e7da20 0x55d0b4ef5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f412cc00640 0 client.26255 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f414c7f1740 1 -- 172.21.15.69:0/2038807232 >> 172.21.15.69:0/2038807232 conn(0x55d0b4e78150 msgr2=0x55d0b4e76940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.610+0000 7f414c7f1740 1 -- 172.21.15.69:0/2038807232 shutdown_connections 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f414c7f1740 1 -- 172.21.15.69:0/2038807232 wait complete. 2024-03-19T01:12:43.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f414c7f1740 20 client.26255 trim_cache size 0 max 16384 2024-03-19T01:12:43.617 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:12:43.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f4149c00640 1 -- 172.21.15.69:0/2348597276 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f41280a2970 msgr2=0x7f41280a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f4149c00640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f41280a2970 0x7f41280a4e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55d0b4e792e0 tx=0x7f413002bc20 comp rx=0 tx=0).stop 2024-03-19T01:12:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f4149c00640 1 -- 172.21.15.69:0/2348597276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41441077c0 msgr2=0x7f41440745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f4149c00640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41441077c0 0x7f41440745a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f413c009fd0 tx=0x7f413c00ef10 comp rx=0 tx=0).stop 2024-03-19T01:12:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f4149c00640 1 -- 172.21.15.69:0/2348597276 shutdown_connections 2024-03-19T01:12:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f4149c00640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41441077c0 0x7f41440745a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f4149c00640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f41280a2970 0x7f41280a4e30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f4149c00640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4144102170 0x7f4144074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f4149c00640 1 --2- 172.21.15.69:0/2348597276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41441017b0 0x7f4144073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.611+0000 7f4149c00640 1 -- 172.21.15.69:0/2348597276 >> 172.21.15.69:0/2348597276 conn(0x7f41440fd260 msgr2=0x7f41440fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.612+0000 7f4149c00640 1 -- 172.21.15.69:0/2348597276 shutdown_connections 2024-03-19T01:12:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:43.612+0000 7f4149c00640 1 -- 172.21.15.69:0/2348597276 wait complete. 2024-03-19T01:12:44.629 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:12:44.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:12:44.629 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:12:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.889+0000 7fc02ce00640 1 -- 172.21.15.69:0/2505090218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0280744f0 msgr2=0x7fc02810db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.889+0000 7fc02ce00640 1 --2- 172.21.15.69:0/2505090218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0280744f0 0x7fc02810db30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc014009900 tx=0x7fc01402bcd0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.889+0000 7fc02ce00640 1 -- 172.21.15.69:0/2505090218 shutdown_connections 2024-03-19T01:12:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.889+0000 7fc02ce00640 1 --2- 172.21.15.69:0/2505090218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc02810e070 0x7fc028110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.889+0000 7fc02ce00640 1 --2- 172.21.15.69:0/2505090218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0280744f0 0x7fc02810db30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.889+0000 7fc02ce00640 1 --2- 172.21.15.69:0/2505090218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc028073bd0 0x7fc028073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.889+0000 7fc02ce00640 1 -- 172.21.15.69:0/2505090218 >> 172.21.15.69:0/2505090218 conn(0x7fc028078c20 msgr2=0x7fc028079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:44.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.889+0000 7fc02ce00640 1 -- 172.21.15.69:0/2505090218 shutdown_connections 2024-03-19T01:12:44.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.889+0000 7fc02ce00640 1 -- 172.21.15.69:0/2505090218 wait complete. 2024-03-19T01:12:44.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.890+0000 7fc02ce00640 1 Processor -- start 2024-03-19T01:12:44.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.890+0000 7fc02ce00640 1 -- start start 2024-03-19T01:12:44.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc02ce00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc028073bd0 0x7fc02819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc02ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0280744f0 0x7fc02819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc02ce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc02810e070 0x7fc02819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc02ce00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc028113d90 con 0x7fc028073bd0 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc02ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc028113c10 con 0x7fc0280744f0 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc02ce00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc028113f10 con 0x7fc02810e070 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc025400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0280744f0 0x7fc02819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc025400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0280744f0 0x7fc02819a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36708/0 (socket says 172.21.15.69:36708) 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc025400640 1 -- 172.21.15.69:0/4220742099 learned_addr learned my addr 172.21.15.69:0/4220742099 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc025e00640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc028073bd0 0x7fc02819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc026800640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc02810e070 0x7fc02819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc025400640 1 -- 172.21.15.69:0/4220742099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc02810e070 msgr2=0x7fc02819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc025400640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc02810e070 0x7fc02819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc025400640 1 -- 172.21.15.69:0/4220742099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc028073bd0 msgr2=0x7fc02819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc025400640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc028073bd0 0x7fc02819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.891+0000 7fc025400640 1 -- 172.21.15.69:0/4220742099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02819f320 con 0x7fc0280744f0 2024-03-19T01:12:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.892+0000 7fc025400640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0280744f0 0x7fc02819a870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc014005940 tx=0x7fc01402cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:44.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.892+0000 7fc01ea00640 1 -- 172.21.15.69:0/4220742099 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014041070 con 0x7fc0280744f0 2024-03-19T01:12:44.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.892+0000 7fc01ea00640 1 -- 172.21.15.69:0/4220742099 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc014004450 con 0x7fc0280744f0 2024-03-19T01:12:44.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.892+0000 7fc02ce00640 1 -- 172.21.15.69:0/4220742099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02819f5b0 con 0x7fc0280744f0 2024-03-19T01:12:44.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.892+0000 7fc01ea00640 1 -- 172.21.15.69:0/4220742099 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014003d80 con 0x7fc0280744f0 2024-03-19T01:12:44.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.892+0000 7fc02ce00640 1 -- 172.21.15.69:0/4220742099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0281a6ea0 con 0x7fc0280744f0 2024-03-19T01:12:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.896+0000 7fc02fa2e740 1 Processor -- start 2024-03-19T01:12:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.896+0000 7fc02fa2e740 1 -- start start 2024-03-19T01:12:44.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.896+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d4b00 0x55afe90d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.896+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90ddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.896+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 0x55afe90e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.896+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55afe9037fb0 con 0x55afe90d4b00 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.896+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afe8fe0ab0 con 0x55afe90d5420 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.896+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55afe9034050 con 0x55afe90de000 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.896+0000 7fc025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90ddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.896+0000 7fc025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90ddac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36712/0 (socket says 172.21.15.69:36712) 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.896+0000 7fc025e00640 1 -- 172.21.15.69:0/761110714 learned_addr learned my addr 172.21.15.69:0/761110714 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.897+0000 7fc026800640 1 --2- 172.21.15.69:0/761110714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d4b00 0x55afe90d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.897+0000 7fc01ea00640 1 -- 172.21.15.69:0/4220742099 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fc014044070 con 0x7fc0280744f0 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.897+0000 7fc025e00640 1 -- 172.21.15.69:0/761110714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 msgr2=0x55afe90e03f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.897+0000 7fc025e00640 1 --2- 172.21.15.69:0/761110714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 0x55afe90e03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.897+0000 7fc025e00640 1 -- 172.21.15.69:0/761110714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d4b00 msgr2=0x55afe90d4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.897+0000 7fc025e00640 1 --2- 172.21.15.69:0/761110714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d4b00 0x55afe90d4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.897+0000 7fc025e00640 1 -- 172.21.15.69:0/761110714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe8ffee00 con 0x55afe90d5420 2024-03-19T01:12:44.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.897+0000 7fc025e00640 1 --2- 172.21.15.69:0/761110714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90ddac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc018009cf0 tx=0x7fc018011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a03c5e6335ce6f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:44.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.897+0000 7fc013400640 1 -- 172.21.15.69:0/761110714 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01801e070 con 0x55afe90d5420 2024-03-19T01:12:44.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.897+0000 7fc013400640 1 -- 172.21.15.69:0/761110714 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc018019b10 con 0x55afe90d5420 2024-03-19T01:12:44.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.897+0000 7fc013400640 1 -- 172.21.15.69:0/761110714 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01800ea00 con 0x55afe90d5420 2024-03-19T01:12:44.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.898+0000 7fc02fa2e740 1 -- 172.21.15.69:0/761110714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 msgr2=0x55afe90ddac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:44.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.898+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/761110714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90ddac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc018009cf0 tx=0x7fc018011810 comp rx=0 tx=0).stop 2024-03-19T01:12:44.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.898+0000 7fc02fa2e740 1 -- 172.21.15.69:0/761110714 shutdown_connections 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.898+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/761110714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 0x55afe90e03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.898+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/761110714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90ddac0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.898+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/761110714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d4b00 0x55afe90d4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.898+0000 7fc02fa2e740 1 -- 172.21.15.69:0/761110714 >> 172.21.15.69:0/761110714 conn(0x55afe90cf230 msgr2=0x55afe90cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.898+0000 7fc02fa2e740 1 -- 172.21.15.69:0/761110714 shutdown_connections 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.898+0000 7fc02fa2e740 1 -- 172.21.15.69:0/761110714 wait complete. 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.898+0000 7fc01ea00640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc00c0a2890 0x7fc00c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.899+0000 7fc01ea00640 1 -- 172.21.15.69:0/4220742099 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc0140f0bc0 con 0x7fc0280744f0 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.899+0000 7fc025e00640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc00c0a2890 0x7fc00c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.899+0000 7fc02fa2e740 1 -- start start 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.899+0000 7fc02fa2e740 1 Processor -- start 2024-03-19T01:12:44.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc025e00640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc00c0a2890 0x7fc00c0a4d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc0180093f0 tx=0x7fc01800e5a0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:44.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90db020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90d5420 0x55afe90db560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90de000 0x55afe90dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55afe90e0df0 con 0x55afe90de000 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afe8ff3f00 con 0x55afe90d4b00 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55afe90e0f70 con 0x55afe90d5420 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc02fa2e740 10 client.0 _ll_register_callbacks cb 0x55afe8ff38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90db020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36722/0 (socket says 172.21.15.69:36722) 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc025e00640 1 -- 172.21.15.69:0/24150220 learned_addr learned my addr 172.21.15.69:0/24150220 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.900+0000 7fc025400640 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90d5420 0x55afe90db560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.901+0000 7fc026800640 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90de000 0x55afe90dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.901+0000 7fc025e00640 1 -- 172.21.15.69:0/24150220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90d5420 msgr2=0x55afe90db560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.901+0000 7fc025e00640 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90d5420 0x55afe90db560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.901+0000 7fc025e00640 1 -- 172.21.15.69:0/24150220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90de000 msgr2=0x55afe90dbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.901+0000 7fc025e00640 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90de000 0x55afe90dbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.901+0000 7fc025e00640 1 -- 172.21.15.69:0/24150220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe8e147e0 con 0x55afe90d4b00 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.901+0000 7fc025e00640 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90db020 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc018000c00 tx=0x7fc018011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb9aae7682007317 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:44.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.901+0000 7fc007e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.901+0000 7fc007e00640 1 -- 172.21.15.69:0/24150220 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018035060 con 0x55afe90d4b00 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.901+0000 7fc007e00640 1 -- 172.21.15.69:0/24150220 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc01800ed30 con 0x55afe90d4b00 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.901+0000 7fc02fa2e740 1 -- 172.21.15.69:0/24150220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc018019b10 con 0x55afe90d4b00 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.902+0000 7fc007e00640 1 -- 172.21.15.69:0/24150220 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc01803c050 con 0x55afe90d4b00 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.902+0000 7fc02fa2e740 10 client.26267 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.902+0000 7fc02fa2e740 1 -- 172.21.15.69:0/24150220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55afe8e74f70 con 0x55afe90d4b00 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.902+0000 7fc007e00640 1 -- 172.21.15.69:0/24150220 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc018019b10 con 0x55afe90d4b00 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.902+0000 7fc007e00640 1 -- 172.21.15.69:0/24150220 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fc018025840 con 0x55afe90d4b00 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.902+0000 7fc02fa2e740 10 client.26267 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.902+0000 7fc02fa2e740 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.902+0000 7fc02fa2e740 20 client.26267 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.902+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afe90ee2c0 0x55afe90f0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.902+0000 7fc02fa2e740 4 client.26267 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:44.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.902+0000 7fc02fa2e740 1 -- 172.21.15.69:0/24150220 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afe90454f0 con 0x55afe90ee2c0 2024-03-19T01:12:44.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.903+0000 7fc025400640 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afe90ee2c0 0x55afe90f0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.903+0000 7fc025400640 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afe90ee2c0 0x55afe90f0710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a41963f06d7f762 server_cookie=37752ddb7e575022 in_seq=0 out_seq=0 2024-03-19T01:12:44.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.903+0000 7fc007e00640 10 client.26267 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:44.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc007e00640 1 -- 172.21.15.69:0/24150220 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55afe90454f0 con 0x55afe90ee2c0 2024-03-19T01:12:44.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc007e00640 10 client.26267 handle_command_reply: tid=0 2024-03-19T01:12:44.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 1 client.26267 shutdown 2024-03-19T01:12:44.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 1 -- 172.21.15.69:0/24150220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 msgr2=0x55afe90db020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:44.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90db020 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc018000c00 tx=0x7fc018011810 comp rx=0 tx=0).stop 2024-03-19T01:12:44.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 1 -- 172.21.15.69:0/24150220 shutdown_connections 2024-03-19T01:12:44.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afe90ee2c0 0x55afe90f0710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90de000 0x55afe90dbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90d5420 0x55afe90db560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc007e00640 0 client.26267 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:44.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/24150220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90db020 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 1 -- 172.21.15.69:0/24150220 >> 172.21.15.69:0/24150220 conn(0x55afe90cf230 msgr2=0x55afe90d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:44.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 1 -- 172.21.15.69:0/24150220 shutdown_connections 2024-03-19T01:12:44.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 1 -- 172.21.15.69:0/24150220 wait complete. 2024-03-19T01:12:44.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.905+0000 7fc02fa2e740 20 client.26267 trim_cache size 0 max 16384 2024-03-19T01:12:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.926+0000 7fc02fa2e740 1 Processor -- start 2024-03-19T01:12:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.926+0000 7fc02fa2e740 1 -- start start 2024-03-19T01:12:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.926+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90dd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.926+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d5420 0x55afe90e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.926+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 0x55afe90d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.926+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281b0fe0 con 0x55afe90d5420 2024-03-19T01:12:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.926+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281b2be0 con 0x55afe90d4b00 2024-03-19T01:12:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.926+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281b16b0 con 0x55afe90de000 2024-03-19T01:12:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90dd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36726/0 (socket says 172.21.15.69:36726) 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc025e00640 1 -- 172.21.15.69:0/2117750299 learned_addr learned my addr 172.21.15.69:0/2117750299 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc025400640 1 --2- 172.21.15.69:0/2117750299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d5420 0x55afe90e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc026800640 1 --2- 172.21.15.69:0/2117750299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 0x55afe90d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc025e00640 1 -- 172.21.15.69:0/2117750299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 msgr2=0x55afe90d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc025e00640 1 --2- 172.21.15.69:0/2117750299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 0x55afe90d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc025e00640 1 -- 172.21.15.69:0/2117750299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d5420 msgr2=0x55afe90e9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc025e00640 1 --2- 172.21.15.69:0/2117750299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d5420 0x55afe90e9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc025e00640 1 -- 172.21.15.69:0/2117750299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe8d0c170 con 0x55afe90d4b00 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc025400640 1 --2- 172.21.15.69:0/2117750299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d5420 0x55afe90e9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.927+0000 7fc025e00640 1 --2- 172.21.15.69:0/2117750299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90dd750 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc018025c10 tx=0x7fc018005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee738e6a0f5bda9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc012a00640 1 -- 172.21.15.69:0/2117750299 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018035050 con 0x55afe90d4b00 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc012a00640 1 -- 172.21.15.69:0/2117750299 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01801f040 con 0x55afe90d4b00 2024-03-19T01:12:44.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc012a00640 1 -- 172.21.15.69:0/2117750299 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018004880 con 0x55afe90d4b00 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc02fa2e740 1 -- 172.21.15.69:0/2117750299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 msgr2=0x55afe90dd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/2117750299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90dd750 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc018025c10 tx=0x7fc018005f90 comp rx=0 tx=0).stop 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc02fa2e740 1 -- 172.21.15.69:0/2117750299 shutdown_connections 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/2117750299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 0x55afe90d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/2117750299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d5420 0x55afe90e9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/2117750299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d4b00 0x55afe90dd750 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc02fa2e740 1 -- 172.21.15.69:0/2117750299 >> 172.21.15.69:0/2117750299 conn(0x55afe90cf230 msgr2=0x55afe90d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc02fa2e740 1 -- 172.21.15.69:0/2117750299 shutdown_connections 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.928+0000 7fc02fa2e740 1 -- 172.21.15.69:0/2117750299 wait complete. 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.929+0000 7fc02fa2e740 1 -- start start 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.929+0000 7fc02fa2e740 1 Processor -- start 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.929+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d4b00 0x55afe90e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.929+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90e31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc02fa2e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 0x55afe90e3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281b2450 con 0x55afe90d4b00 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281b1d80 con 0x55afe90d5420 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc02fa2e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281aefc0 con 0x55afe90de000 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc02fa2e740 10 client.0 _ll_register_callbacks cb 0x7fc018019760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc025400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90e31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc025400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90e31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36732/0 (socket says 172.21.15.69:36732) 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc026800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 0x55afe90e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc025400640 1 -- 172.21.15.69:0/914932077 learned_addr learned my addr 172.21.15.69:0/914932077 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc025e00640 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d4b00 0x55afe90e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc025400640 1 -- 172.21.15.69:0/914932077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 msgr2=0x55afe90e3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc025400640 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 0x55afe90e3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc025400640 1 -- 172.21.15.69:0/914932077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d4b00 msgr2=0x55afe90e61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc025400640 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d4b00 0x55afe90e61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc025400640 1 -- 172.21.15.69:0/914932077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe90e6930 con 0x55afe90d5420 2024-03-19T01:12:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc025e00640 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d4b00 0x55afe90e61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc025400640 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90e31c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc0140bcfb0 tx=0x7fc01402ce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2138ae7f9bda2ea server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.930+0000 7fc007e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc007e00640 1 -- 172.21.15.69:0/914932077 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01404b040 con 0x55afe90d5420 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc02fa2e740 1 -- 172.21.15.69:0/914932077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01801f040 con 0x55afe90d5420 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc007e00640 1 -- 172.21.15.69:0/914932077 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc0140ee1d0 con 0x55afe90d5420 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc007e00640 1 -- 172.21.15.69:0/914932077 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc0140bc980 con 0x55afe90d5420 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc02fa2e740 10 client.26273 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc02fa2e740 1 -- 172.21.15.69:0/914932077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55afe9155180 con 0x55afe90d5420 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc007e00640 1 -- 172.21.15.69:0/914932077 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01801f040 con 0x55afe90d5420 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc007e00640 1 -- 172.21.15.69:0/914932077 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fc0140c5280 con 0x55afe90d5420 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc02fa2e740 10 client.26273 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc02fa2e740 10 client.26273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc02fa2e740 20 client.26273 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.931+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afe91558e0 0x55afe9157d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.932+0000 7fc02fa2e740 4 client.26273 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:12:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.932+0000 7fc02fa2e740 1 -- 172.21.15.69:0/914932077 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55afe903f840 con 0x55afe91558e0 2024-03-19T01:12:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.932+0000 7fc025e00640 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afe91558e0 0x55afe9157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.932+0000 7fc025e00640 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afe91558e0 0x55afe9157d00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560d96e05423dd43 server_cookie=dfaafcb226bfd904 in_seq=0 out_seq=0 2024-03-19T01:12:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.932+0000 7fc007e00640 10 client.26273 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc007e00640 1 -- 172.21.15.69:0/914932077 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55afe903f840 con 0x55afe91558e0 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc007e00640 10 client.26273 handle_command_reply: tid=0 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc02fa2e740 1 client.26273 shutdown 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc02fa2e740 1 -- 172.21.15.69:0/914932077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 msgr2=0x55afe90e31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90e31c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc0140bcfb0 tx=0x7fc01402ce50 comp rx=0 tx=0).stop 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc02fa2e740 1 -- 172.21.15.69:0/914932077 shutdown_connections 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afe91558e0 0x55afe9157d00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afe90de000 0x55afe90e3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe90d5420 0x55afe90e31c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc02fa2e740 1 --2- 172.21.15.69:0/914932077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afe90d4b00 0x55afe90e61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc02fa2e740 1 -- 172.21.15.69:0/914932077 >> 172.21.15.69:0/914932077 conn(0x55afe90cf230 msgr2=0x55afe90d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:44.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.934+0000 7fc007e00640 0 client.26273 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.935+0000 7fc02fa2e740 1 -- 172.21.15.69:0/914932077 shutdown_connections 2024-03-19T01:12:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.935+0000 7fc02fa2e740 1 -- 172.21.15.69:0/914932077 wait complete. 2024-03-19T01:12:44.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.935+0000 7fc02fa2e740 20 client.26273 trim_cache size 0 max 16384 2024-03-19T01:12:44.942 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:12:44.942 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:12:44.942 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "0f9f87ec-19d1-49b2-9f6e-4665befd93cb", 2024-03-19T01:12:44.942 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:12:44.942 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:12:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.935+0000 7fc02ce00640 1 -- 172.21.15.69:0/4220742099 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc00c0a2890 msgr2=0x7fc00c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.935+0000 7fc02ce00640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc00c0a2890 0x7fc00c0a4d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc0180093f0 tx=0x7fc01800e5a0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.935+0000 7fc02ce00640 1 -- 172.21.15.69:0/4220742099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0280744f0 msgr2=0x7fc02819a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.935+0000 7fc02ce00640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0280744f0 0x7fc02819a870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc014005940 tx=0x7fc01402cbd0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.936+0000 7fc02ce00640 1 -- 172.21.15.69:0/4220742099 shutdown_connections 2024-03-19T01:12:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.936+0000 7fc02ce00640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc02810e070 0x7fc02819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.936+0000 7fc02ce00640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc00c0a2890 0x7fc00c0a4d50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.936+0000 7fc02ce00640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0280744f0 0x7fc02819a870 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.936+0000 7fc02ce00640 1 --2- 172.21.15.69:0/4220742099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc028073bd0 0x7fc02819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:44.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.936+0000 7fc02ce00640 1 -- 172.21.15.69:0/4220742099 >> 172.21.15.69:0/4220742099 conn(0x7fc028078c20 msgr2=0x7fc0280fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:44.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.936+0000 7fc02ce00640 1 -- 172.21.15.69:0/4220742099 shutdown_connections 2024-03-19T01:12:44.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:44.936+0000 7fc02ce00640 1 -- 172.21.15.69:0/4220742099 wait complete. 2024-03-19T01:12:44.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:12:44.957 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:12:45.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.165+0000 7f457ac00640 1 -- 172.21.15.69:0/3902905613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45741098b0 msgr2=0x7f4574109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:45.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.165+0000 7f457ac00640 1 --2- 172.21.15.69:0/3902905613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45741098b0 0x7f4574109c90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4568009930 tx=0x7f456802bc30 comp rx=0 tx=0).stop 2024-03-19T01:12:45.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.165+0000 7f457ac00640 1 -- 172.21.15.69:0/3902905613 shutdown_connections 2024-03-19T01:12:45.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.165+0000 7f457ac00640 1 --2- 172.21.15.69:0/3902905613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4574104260 0x7f4574110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.165+0000 7f457ac00640 1 --2- 172.21.15.69:0/3902905613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45741038a0 0x7f4574103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.165+0000 7f457ac00640 1 --2- 172.21.15.69:0/3902905613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45741098b0 0x7f4574109c90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.165+0000 7f457ac00640 1 -- 172.21.15.69:0/3902905613 >> 172.21.15.69:0/3902905613 conn(0x7f45740ff330 msgr2=0x7f4574101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:45.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.165+0000 7f457ac00640 1 -- 172.21.15.69:0/3902905613 shutdown_connections 2024-03-19T01:12:45.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.165+0000 7f457ac00640 1 -- 172.21.15.69:0/3902905613 wait complete. 2024-03-19T01:12:45.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.166+0000 7f457ac00640 1 Processor -- start 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.166+0000 7f457ac00640 1 -- start start 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.166+0000 7f457ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45741038a0 0x7f4574075d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.166+0000 7f457ac00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4574104260 0x7f4574076270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f457ac00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45741098b0 0x7f45740767b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f457ac00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4574113d10 con 0x7f45741098b0 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f457ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4574113b90 con 0x7f45741038a0 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f457ac00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4574113e90 con 0x7f4574104260 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f4579800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45741038a0 0x7f4574075d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f4579800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45741038a0 0x7f4574075d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36760/0 (socket says 172.21.15.69:36760) 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f4579800640 1 -- 172.21.15.69:0/2001959106 learned_addr learned my addr 172.21.15.69:0/2001959106 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f4578e00640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4574104260 0x7f4574076270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f457a200640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45741098b0 0x7f45740767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f4579800640 1 -- 172.21.15.69:0/2001959106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4574104260 msgr2=0x7f4574076270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:45.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f4579800640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4574104260 0x7f4574076270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f4579800640 1 -- 172.21.15.69:0/2001959106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45741098b0 msgr2=0x7f45740767b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:45.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f4579800640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45741098b0 0x7f45740767b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f4579800640 1 -- 172.21.15.69:0/2001959106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45741aa850 con 0x7f45741038a0 2024-03-19T01:12:45.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.167+0000 7f457a200640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45741098b0 0x7f45740767b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:45.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.168+0000 7f4579800640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45741038a0 0x7f4574075d30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f456802c170 tx=0x7f4568004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:45.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.168+0000 7f4572000640 1 -- 172.21.15.69:0/2001959106 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568043020 con 0x7f45741038a0 2024-03-19T01:12:45.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.168+0000 7f457ac00640 1 -- 172.21.15.69:0/2001959106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45741aaae0 con 0x7f45741038a0 2024-03-19T01:12:45.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.168+0000 7f4572000640 1 -- 172.21.15.69:0/2001959106 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45680054e0 con 0x7f45741038a0 2024-03-19T01:12:45.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.168+0000 7f4572000640 1 -- 172.21.15.69:0/2001959106 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568003be0 con 0x7f45741038a0 2024-03-19T01:12:45.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.168+0000 7f457ac00640 1 -- 172.21.15.69:0/2001959106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45741ab080 con 0x7f45741038a0 2024-03-19T01:12:45.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.172+0000 7f4572000640 1 -- 172.21.15.69:0/2001959106 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f4568046020 con 0x7f45741038a0 2024-03-19T01:12:45.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.173+0000 7f457ef8d740 1 Processor -- start 2024-03-19T01:12:45.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.173+0000 7f457ef8d740 1 -- start start 2024-03-19T01:12:45.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.173+0000 7f4572000640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f455c0a2970 0x7f455c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.173+0000 7f4572000640 1 -- 172.21.15.69:0/2001959106 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f45680f0c50 con 0x7f45741038a0 2024-03-19T01:12:45.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.173+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3576f30 0x55efe3577310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.173+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe357fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.173+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 0x55efe3582820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.173+0000 7f457ef8d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55efe34db2c0 con 0x55efe3577850 2024-03-19T01:12:45.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.173+0000 7f457ef8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55efe3483b60 con 0x55efe3580430 2024-03-19T01:12:45.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.173+0000 7f457ef8d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55efe34d7360 con 0x55efe3576f30 2024-03-19T01:12:45.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.173+0000 7f457a200640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f455c0a2970 0x7f455c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.174+0000 7f457a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 0x55efe3582820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.174+0000 7f4579800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3576f30 0x55efe3577310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.174+0000 7f457a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 0x55efe3582820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36772/0 (socket says 172.21.15.69:36772) 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.174+0000 7f457a200640 1 -- 172.21.15.69:0/654409931 learned_addr learned my addr 172.21.15.69:0/654409931 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.174+0000 7f4578e00640 1 --2- 172.21.15.69:0/654409931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe357fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.174+0000 7f4579800640 1 -- 172.21.15.69:0/654409931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 msgr2=0x55efe3582820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.174+0000 7f4579800640 1 --2- 172.21.15.69:0/654409931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 0x55efe3582820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.174+0000 7f4579800640 1 -- 172.21.15.69:0/654409931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 msgr2=0x55efe357fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.174+0000 7f4579800640 1 --2- 172.21.15.69:0/654409931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe357fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.174+0000 7f4579800640 1 -- 172.21.15.69:0/654409931 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe32b97d0 con 0x55efe3576f30 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.175+0000 7f457a200640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f455c0a2970 0x7f455c0a4e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f456c005c40 tx=0x7f456c009ed0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.175+0000 7f4579800640 1 --2- 172.21.15.69:0/654409931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3576f30 0x55efe3577310 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f45680ee090 tx=0x7f45680035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b4ac4666629837 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.175+0000 7f4566a00640 1 -- 172.21.15.69:0/654409931 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568004c80 con 0x55efe3576f30 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.175+0000 7f4566a00640 1 -- 172.21.15.69:0/654409931 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4568005690 con 0x55efe3576f30 2024-03-19T01:12:45.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.175+0000 7f4566a00640 1 -- 172.21.15.69:0/654409931 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680b9610 con 0x55efe3576f30 2024-03-19T01:12:45.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.176+0000 7f457ef8d740 1 -- 172.21.15.69:0/654409931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3576f30 msgr2=0x55efe3577310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:45.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.176+0000 7f457ef8d740 1 --2- 172.21.15.69:0/654409931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3576f30 0x55efe3577310 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f45680ee090 tx=0x7f45680035f0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.176+0000 7f457ef8d740 1 -- 172.21.15.69:0/654409931 shutdown_connections 2024-03-19T01:12:45.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.176+0000 7f457ef8d740 1 --2- 172.21.15.69:0/654409931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 0x55efe3582820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.176+0000 7f457ef8d740 1 --2- 172.21.15.69:0/654409931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe357fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.176+0000 7f457ef8d740 1 --2- 172.21.15.69:0/654409931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3576f30 0x55efe3577310 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f45680ee090 tx=0x7f45680035f0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.176+0000 7f457ef8d740 1 -- 172.21.15.69:0/654409931 >> 172.21.15.69:0/654409931 conn(0x55efe3571660 msgr2=0x55efe3571a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.176+0000 7f457ef8d740 1 -- 172.21.15.69:0/654409931 shutdown_connections 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.176+0000 7f457ef8d740 1 -- 172.21.15.69:0/654409931 wait complete. 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.177+0000 7f457ef8d740 1 -- start start 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.177+0000 7f457ef8d740 1 Processor -- start 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe357d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3580430 0x55efe357db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 0x55efe358c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f457ef8d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55efe34d7360 con 0x55efe3577850 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f457ef8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55efe3496fc0 con 0x55efe357e100 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f457ef8d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55efe3582fe0 con 0x55efe3580430 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f457ef8d740 10 client.0 _ll_register_callbacks cb 0x55efe3496990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f4578e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3580430 0x55efe357db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f457a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 0x55efe358c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f4578e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3580430 0x55efe357db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:54206/0 (socket says 172.21.15.69:54206) 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f4578e00640 1 -- 172.21.15.69:0/11336870 learned_addr learned my addr 172.21.15.69:0/11336870 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.178+0000 7f457a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 0x55efe358c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36776/0 (socket says 172.21.15.69:36776) 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f457a200640 1 -- 172.21.15.69:0/11336870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3580430 msgr2=0x55efe357db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f457a200640 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3580430 0x55efe357db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f4579800640 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe357d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f457a200640 1 -- 172.21.15.69:0/11336870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 msgr2=0x55efe357d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f457a200640 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe357d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f457a200640 1 -- 172.21.15.69:0/11336870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe31ef3f0 con 0x55efe357e100 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f4578e00640 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3580430 0x55efe357db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f457a200640 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 0x55efe358c1c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55efe35727f0 tx=0x7f456c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d69a7d3f96c31f79 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f455b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f455b400640 1 -- 172.21.15.69:0/11336870 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c029470 con 0x55efe357e100 2024-03-19T01:12:45.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f455b400640 1 -- 172.21.15.69:0/11336870 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f456c010430 con 0x55efe357e100 2024-03-19T01:12:45.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.179+0000 7f457ef8d740 1 -- 172.21.15.69:0/11336870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55efe347c610 con 0x55efe357e100 2024-03-19T01:12:45.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.180+0000 7f455b400640 1 -- 172.21.15.69:0/11336870 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f456c010630 con 0x55efe357e100 2024-03-19T01:12:45.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.180+0000 7f457ef8d740 10 client.26285 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:45.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.180+0000 7f457ef8d740 1 -- 172.21.15.69:0/11336870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55efe31af0f0 con 0x55efe357e100 2024-03-19T01:12:45.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.180+0000 7f455b400640 1 -- 172.21.15.69:0/11336870 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55efe347c610 con 0x55efe357e100 2024-03-19T01:12:45.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.180+0000 7f455b400640 1 -- 172.21.15.69:0/11336870 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f456c012ce0 con 0x55efe357e100 2024-03-19T01:12:45.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.180+0000 7f457ef8d740 10 client.26285 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:45.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.180+0000 7f457ef8d740 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:45.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.180+0000 7f457ef8d740 20 client.26285 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:45.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.180+0000 7f457ef8d740 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55efe3590a30 0x55efe3592e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.180+0000 7f457ef8d740 4 client.26285 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:12:45.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.180+0000 7f457ef8d740 1 -- 172.21.15.69:0/11336870 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efe34e8800 con 0x55efe3590a30 2024-03-19T01:12:45.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.181+0000 7f4579800640 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55efe3590a30 0x55efe3592e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.181+0000 7f4579800640 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55efe3590a30 0x55efe3592e90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=961445ed41a724a4 server_cookie=99e9338fd2ce010b in_seq=0 out_seq=0 2024-03-19T01:12:45.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.181+0000 7f455b400640 10 client.26285 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:45.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.184+0000 7f455b400640 1 -- 172.21.15.69:0/11336870 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55efe34e8800 con 0x55efe3590a30 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.184+0000 7f455b400640 10 client.26285 handle_command_reply: tid=0 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.184+0000 7f457ef8d740 1 client.26285 shutdown 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.184+0000 7f457ef8d740 1 -- 172.21.15.69:0/11336870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 msgr2=0x55efe358c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.184+0000 7f457ef8d740 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 0x55efe358c1c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55efe35727f0 tx=0x7f456c004ab0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.184+0000 7f457ef8d740 1 -- 172.21.15.69:0/11336870 shutdown_connections 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.184+0000 7f457ef8d740 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55efe3590a30 0x55efe3592e90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.184+0000 7f457ef8d740 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 0x55efe358c1c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.184+0000 7f457ef8d740 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3580430 0x55efe357db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.184+0000 7f455b400640 0 client.26285 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.184+0000 7f457ef8d740 1 --2- 172.21.15.69:0/11336870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe357d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.185+0000 7f457ef8d740 1 -- 172.21.15.69:0/11336870 >> 172.21.15.69:0/11336870 conn(0x55efe3571660 msgr2=0x55efe35733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.185+0000 7f457ef8d740 1 -- 172.21.15.69:0/11336870 shutdown_connections 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.185+0000 7f457ef8d740 1 -- 172.21.15.69:0/11336870 wait complete. 2024-03-19T01:12:45.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.185+0000 7f457ef8d740 20 client.26285 trim_cache size 0 max 16384 2024-03-19T01:12:45.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.204+0000 7f457ef8d740 1 Processor -- start 2024-03-19T01:12:45.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f457ef8d740 1 -- start start 2024-03-19T01:12:45.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3577850 0x55efe357c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 0x55efe35731a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3580430 0x55efe35987a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f457ef8d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45741b5130 con 0x55efe3580430 2024-03-19T01:12:45.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f457ef8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45741b6d30 con 0x55efe357e100 2024-03-19T01:12:45.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f457ef8d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f45741b5800 con 0x55efe3577850 2024-03-19T01:12:45.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f4579800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3577850 0x55efe357c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f4578e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 0x55efe35731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f4579800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3577850 0x55efe357c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:54208/0 (socket says 172.21.15.69:54208) 2024-03-19T01:12:45.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f4579800640 1 -- 172.21.15.69:0/3962679485 learned_addr learned my addr 172.21.15.69:0/3962679485 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:45.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.205+0000 7f457a200640 1 --2- 172.21.15.69:0/3962679485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3580430 0x55efe35987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.206+0000 7f4579800640 1 -- 172.21.15.69:0/3962679485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 msgr2=0x55efe35731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:45.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.206+0000 7f4579800640 1 --2- 172.21.15.69:0/3962679485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 0x55efe35731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.206+0000 7f4579800640 1 -- 172.21.15.69:0/3962679485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3580430 msgr2=0x55efe35987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:45.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.206+0000 7f4579800640 1 --2- 172.21.15.69:0/3962679485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3580430 0x55efe35987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.206+0000 7f4579800640 1 -- 172.21.15.69:0/3962679485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe3577680 con 0x55efe3577850 2024-03-19T01:12:45.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.206+0000 7f457a200640 1 --2- 172.21.15.69:0/3962679485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3580430 0x55efe35987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:12:45.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.206+0000 7f4579800640 1 --2- 172.21.15.69:0/3962679485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3577850 0x55efe357c790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f45680b93b0 tx=0x7f45680b9240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb0a0912d38e779a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:45.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.206+0000 7f4566000640 1 -- 172.21.15.69:0/3962679485 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680bd6a0 con 0x55efe3577850 2024-03-19T01:12:45.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.206+0000 7f4566000640 1 -- 172.21.15.69:0/3962679485 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45680c5740 con 0x55efe3577850 2024-03-19T01:12:45.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.206+0000 7f4566000640 1 -- 172.21.15.69:0/3962679485 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680b9790 con 0x55efe3577850 2024-03-19T01:12:45.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.207+0000 7f457ef8d740 1 -- 172.21.15.69:0/3962679485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3577850 msgr2=0x55efe357c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.207+0000 7f457ef8d740 1 --2- 172.21.15.69:0/3962679485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3577850 0x55efe357c790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f45680b93b0 tx=0x7f45680b9240 comp rx=0 tx=0).stop 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.207+0000 7f457ef8d740 1 -- 172.21.15.69:0/3962679485 shutdown_connections 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.207+0000 7f457ef8d740 1 --2- 172.21.15.69:0/3962679485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3580430 0x55efe35987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.207+0000 7f457ef8d740 1 --2- 172.21.15.69:0/3962679485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe357e100 0x55efe35731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.207+0000 7f457ef8d740 1 --2- 172.21.15.69:0/3962679485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe3577850 0x55efe357c790 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.207+0000 7f457ef8d740 1 -- 172.21.15.69:0/3962679485 >> 172.21.15.69:0/3962679485 conn(0x55efe3571660 msgr2=0x55efe3572a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.208+0000 7f457ef8d740 1 -- 172.21.15.69:0/3962679485 shutdown_connections 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.208+0000 7f457ef8d740 1 -- 172.21.15.69:0/3962679485 wait complete. 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.208+0000 7f457ef8d740 1 -- start start 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.208+0000 7f457ef8d740 1 Processor -- start 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe3583120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe357e100 0x55efe3583660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f457ef8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 0x55efe3583ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f457ef8d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45741b65a0 con 0x55efe3577850 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f457ef8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45741b5ed0 con 0x55efe3580430 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f457ef8d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f45741b3110 con 0x55efe357e100 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f457ef8d740 10 client.0 _ll_register_callbacks cb 0x7f456c009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f4579800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe3583120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f4578e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe357e100 0x55efe3583660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f457a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 0x55efe3583ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f4579800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe3583120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48988/0 (socket says 172.21.15.69:48988) 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.209+0000 7f4579800640 1 -- 172.21.15.69:0/3618496002 learned_addr learned my addr 172.21.15.69:0/3618496002 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f4579800640 1 -- 172.21.15.69:0/3618496002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe357e100 msgr2=0x55efe3583660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f4579800640 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe357e100 0x55efe3583660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f4579800640 1 -- 172.21.15.69:0/3618496002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 msgr2=0x55efe3583ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f4579800640 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 0x55efe3583ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f4579800640 1 -- 172.21.15.69:0/3618496002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe35f4fc0 con 0x55efe3577850 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f4578e00640 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe357e100 0x55efe3583660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f457a200640 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 0x55efe3583ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f4579800640 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe3583120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f45680bcf30 tx=0x7f4568005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0972ed548ae4114 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f455b400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f455b400640 1 -- 172.21.15.69:0/3618496002 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680db020 con 0x55efe3577850 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f455b400640 1 -- 172.21.15.69:0/3618496002 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f45680c52b0 con 0x55efe3577850 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.210+0000 7f457ef8d740 1 -- 172.21.15.69:0/3618496002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45680c5740 con 0x55efe3577850 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.211+0000 7f455b400640 1 -- 172.21.15.69:0/3618496002 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f45680b9790 con 0x55efe3577850 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.211+0000 7f457ef8d740 10 client.16101 fetch_fsmap learned FSMap version 31 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.211+0000 7f457ef8d740 1 -- 172.21.15.69:0/3618496002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55efe35f62b0 con 0x55efe3577850 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.211+0000 7f455b400640 1 -- 172.21.15.69:0/3618496002 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45680c5740 con 0x55efe3577850 2024-03-19T01:12:45.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.211+0000 7f455b400640 1 -- 172.21.15.69:0/3618496002 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f45680bc3f0 con 0x55efe3577850 2024-03-19T01:12:45.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.211+0000 7f457ef8d740 10 client.16101 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:12:45.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.211+0000 7f457ef8d740 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:12:45.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.211+0000 7f457ef8d740 20 client.16101 populate_metadata read hostname 'smithi069' 2024-03-19T01:12:45.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.211+0000 7f457ef8d740 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55efe3590a30 0x55efe35f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:12:45.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.211+0000 7f457ef8d740 4 client.16101 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:12:45.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.212+0000 7f457ef8d740 1 -- 172.21.15.69:0/3618496002 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55efe34e2b50 con 0x55efe3590a30 2024-03-19T01:12:45.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.212+0000 7f4578e00640 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55efe3590a30 0x55efe35f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:12:45.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.212+0000 7f4578e00640 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55efe3590a30 0x55efe35f87e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a428715425c804f server_cookie=881013f08edc2c21 in_seq=0 out_seq=0 2024-03-19T01:12:45.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.212+0000 7f455b400640 10 client.16101 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:45.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.215+0000 7f455b400640 1 -- 172.21.15.69:0/3618496002 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55efe34e2b50 con 0x55efe3590a30 2024-03-19T01:12:45.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.215+0000 7f455b400640 10 client.16101 handle_command_reply: tid=0 2024-03-19T01:12:45.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.215+0000 7f457ef8d740 1 client.16101 shutdown 2024-03-19T01:12:45.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f457ef8d740 1 -- 172.21.15.69:0/3618496002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 msgr2=0x55efe3583120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:45.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f457ef8d740 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe3583120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f45680bcf30 tx=0x7f4568005b50 comp rx=0 tx=0).stop 2024-03-19T01:12:45.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f457ef8d740 1 -- 172.21.15.69:0/3618496002 shutdown_connections 2024-03-19T01:12:45.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f457ef8d740 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55efe3590a30 0x55efe35f87e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f457ef8d740 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efe3580430 0x55efe3583ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f457ef8d740 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55efe357e100 0x55efe3583660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f457ef8d740 1 --2- 172.21.15.69:0/3618496002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55efe3577850 0x55efe3583120 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f455b400640 0 client.16101 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:12:45.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f457ef8d740 1 -- 172.21.15.69:0/3618496002 >> 172.21.15.69:0/3618496002 conn(0x55efe3571660 msgr2=0x55efe35706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:45.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f457ef8d740 1 -- 172.21.15.69:0/3618496002 shutdown_connections 2024-03-19T01:12:45.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f457ef8d740 1 -- 172.21.15.69:0/3618496002 wait complete. 2024-03-19T01:12:45.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.216+0000 7f457ef8d740 20 client.16101 trim_cache size 0 max 16384 2024-03-19T01:12:45.219 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:12:45.219 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (10 inodes in the stack)", 2024-03-19T01:12:45.219 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:12:45.219 INFO:teuthology.orchestra.run.smithi069.stdout: "0f9f87ec-19d1-49b2-9f6e-4665befd93cb": { 2024-03-19T01:12:45.219 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:12:45.219 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "0f9f87ec-19d1-49b2-9f6e-4665befd93cb", 2024-03-19T01:12:45.220 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:12:45.220 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:12:45.220 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:12:45.220 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:12:45.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 -- 172.21.15.69:0/2001959106 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f455c0a2970 msgr2=0x7f455c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:45.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f455c0a2970 0x7f455c0a4e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f456c005c40 tx=0x7f456c009ed0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 -- 172.21.15.69:0/2001959106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45741038a0 msgr2=0x7f4574075d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:12:45.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45741038a0 0x7f4574075d30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f456802c170 tx=0x7f4568004360 comp rx=0 tx=0).stop 2024-03-19T01:12:45.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 -- 172.21.15.69:0/2001959106 shutdown_connections 2024-03-19T01:12:45.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45741098b0 0x7f45740767b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4574104260 0x7f4574076270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f455c0a2970 0x7f455c0a4e30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 --2- 172.21.15.69:0/2001959106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45741038a0 0x7f4574075d30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:12:45.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 -- 172.21.15.69:0/2001959106 >> 172.21.15.69:0/2001959106 conn(0x7f45740ff330 msgr2=0x7f45740ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:12:45.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 -- 172.21.15.69:0/2001959106 shutdown_connections 2024-03-19T01:12:45.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:12:45.217+0000 7f457ac00640 1 -- 172.21.15.69:0/2001959106 wait complete. 2024-03-19T01:12:45.236 INFO:tasks.cephfs.filesystem:scrub status for tag:0f9f87ec-19d1-49b2-9f6e-4665befd93cb - {'path': '/', 'tag': '0f9f87ec-19d1-49b2-9f6e-4665befd93cb', 'options': 'recursive,force'} 2024-03-19T01:12:45.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:45 smithi069 ceph-mon[31727]: pgmap v65: 129 pgs: 129 active+clean; 188 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-19T01:12:45.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:45 smithi086 ceph-mon[66829]: pgmap v65: 129 pgs: 129 active+clean; 188 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-19T01:12:45.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:45 smithi163 ceph-mon[67428]: pgmap v65: 129 pgs: 129 active+clean; 188 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-19T01:12:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:12:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:12:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:12:46.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: pgmap v66: 129 pgs: 129 active+clean; 198 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 253 op/s 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: pgmap v66: 129 pgs: 129 active+clean; 198 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 253 op/s 2024-03-19T01:12:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:47.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:47.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:47.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:47.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:47.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: pgmap v66: 129 pgs: 129 active+clean; 198 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 253 op/s 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:49.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:49 smithi086 ceph-mon[66829]: pgmap v67: 129 pgs: 129 active+clean; 201 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-03-19T01:12:49.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:49.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:49.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:49 smithi069 ceph-mon[31727]: pgmap v67: 129 pgs: 129 active+clean; 201 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-03-19T01:12:49.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:49.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:49.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:49 smithi163 ceph-mon[67428]: pgmap v67: 129 pgs: 129 active+clean; 201 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-03-19T01:12:49.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:49.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:50.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:50 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:12:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:50.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:50 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:12:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:50 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:12:51.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: pgmap v68: 129 pgs: 129 active+clean; 201 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 305 op/s 2024-03-19T01:12:51.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: pgmap v68: 129 pgs: 129 active+clean; 201 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 305 op/s 2024-03-19T01:12:51.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: pgmap v68: 129 pgs: 129 active+clean; 201 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 305 op/s 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:12:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:12:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:12:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:52.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:52.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:52.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:12:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:12:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:12:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:52.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:12:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:12:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:12:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:53 smithi086 ceph-mon[66829]: pgmap v69: 129 pgs: 129 active+clean; 216 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 520 op/s 2024-03-19T01:12:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:53.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:53.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:53.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:53 smithi069 ceph-mon[31727]: pgmap v69: 129 pgs: 129 active+clean; 216 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 520 op/s 2024-03-19T01:12:53.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:53.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:53.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:53.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:53 smithi163 ceph-mon[67428]: pgmap v69: 129 pgs: 129 active+clean; 216 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 520 op/s 2024-03-19T01:12:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:54.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:54.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:54.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:54.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:54.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:54.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:54.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:55.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:55 smithi086 ceph-mon[66829]: pgmap v70: 129 pgs: 129 active+clean; 216 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 398 op/s 2024-03-19T01:12:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:55.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:55 smithi069 ceph-mon[31727]: pgmap v70: 129 pgs: 129 active+clean; 216 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 398 op/s 2024-03-19T01:12:55.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:55.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:55.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:55 smithi163 ceph-mon[67428]: pgmap v70: 129 pgs: 129 active+clean; 216 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 398 op/s 2024-03-19T01:12:55.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:55.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:56.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:12:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:56.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:56.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:56.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:12:56.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:12:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:57.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:57 smithi086 ceph-mon[66829]: pgmap v71: 129 pgs: 129 active+clean; 226 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 468 op/s 2024-03-19T01:12:57.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:57 smithi069 ceph-mon[31727]: pgmap v71: 129 pgs: 129 active+clean; 226 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 468 op/s 2024-03-19T01:12:57.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:57 smithi163 ceph-mon[67428]: pgmap v71: 129 pgs: 129 active+clean; 226 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 468 op/s 2024-03-19T01:12:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:58.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:12:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:12:59 smithi086 ceph-mon[66829]: pgmap v72: 129 pgs: 129 active+clean; 241 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 528 op/s 2024-03-19T01:12:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:12:59 smithi069 ceph-mon[31727]: pgmap v72: 129 pgs: 129 active+clean; 241 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 528 op/s 2024-03-19T01:12:59.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:12:59 smithi163 ceph-mon[67428]: pgmap v72: 129 pgs: 129 active+clean; 241 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 528 op/s 2024-03-19T01:13:01.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:01 smithi086 ceph-mon[66829]: pgmap v73: 129 pgs: 129 active+clean; 241 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-03-19T01:13:01.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:01 smithi069 ceph-mon[31727]: pgmap v73: 129 pgs: 129 active+clean; 241 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-03-19T01:13:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:01 smithi163 ceph-mon[67428]: pgmap v73: 129 pgs: 129 active+clean; 241 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-03-19T01:13:03.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:03 smithi086 ceph-mon[66829]: pgmap v74: 129 pgs: 129 active+clean; 257 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 558 op/s 2024-03-19T01:13:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:03 smithi069 ceph-mon[31727]: pgmap v74: 129 pgs: 129 active+clean; 257 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 558 op/s 2024-03-19T01:13:03.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:03 smithi163 ceph-mon[67428]: pgmap v74: 129 pgs: 129 active+clean; 257 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 558 op/s 2024-03-19T01:13:04.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:04 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:13:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:04 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:13:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:04 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:13:04.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:04 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:13:04.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:04 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:13:04.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:04 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:13:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:04 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:13:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:04 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:13:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:04 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:13:05.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:05 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:13:05.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:05 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:13:05.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:05 smithi086 ceph-mon[66829]: pgmap v75: 129 pgs: 129 active+clean; 257 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-03-19T01:13:05.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:05 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:13:05.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:05 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:13:05.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:05 smithi069 ceph-mon[31727]: pgmap v75: 129 pgs: 129 active+clean; 257 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-03-19T01:13:05.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:05 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:13:05.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:05 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:13:05.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:05 smithi163 ceph-mon[67428]: pgmap v75: 129 pgs: 129 active+clean; 257 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-03-19T01:13:07.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:07 smithi086 ceph-mon[66829]: pgmap v76: 129 pgs: 129 active+clean; 261 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 370 op/s 2024-03-19T01:13:07.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:07 smithi069 ceph-mon[31727]: pgmap v76: 129 pgs: 129 active+clean; 261 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 370 op/s 2024-03-19T01:13:07.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:07 smithi163 ceph-mon[67428]: pgmap v76: 129 pgs: 129 active+clean; 261 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 370 op/s 2024-03-19T01:13:09.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:09 smithi086 ceph-mon[66829]: pgmap v77: 129 pgs: 129 active+clean; 270 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 350 op/s 2024-03-19T01:13:09.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:09 smithi069 ceph-mon[31727]: pgmap v77: 129 pgs: 129 active+clean; 270 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 350 op/s 2024-03-19T01:13:09.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:09 smithi163 ceph-mon[67428]: pgmap v77: 129 pgs: 129 active+clean; 270 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 350 op/s 2024-03-19T01:13:11.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:11 smithi069 ceph-mon[31727]: pgmap v78: 129 pgs: 129 active+clean; 270 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 222 op/s 2024-03-19T01:13:11.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:11 smithi163 ceph-mon[67428]: pgmap v78: 129 pgs: 129 active+clean; 270 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 222 op/s 2024-03-19T01:13:11.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:11 smithi086 ceph-mon[66829]: pgmap v78: 129 pgs: 129 active+clean; 270 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 222 op/s 2024-03-19T01:13:13.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:13 smithi069 ceph-mon[31727]: pgmap v79: 129 pgs: 129 active+clean; 290 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2024-03-19T01:13:13.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:13 smithi163 ceph-mon[67428]: pgmap v79: 129 pgs: 129 active+clean; 290 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2024-03-19T01:13:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:13 smithi086 ceph-mon[66829]: pgmap v79: 129 pgs: 129 active+clean; 290 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 361 op/s 2024-03-19T01:13:15.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:13:15.237 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:13:15.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.449+0000 7f3402a00640 1 -- 172.21.15.69:0/1574955113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33fc108030 msgr2=0x7f33fc108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.449+0000 7f3402a00640 1 --2- 172.21.15.69:0/1574955113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33fc108030 0x7f33fc108410 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009900 tx=0x7f33ec02bc40 comp rx=0 tx=0).stop 2024-03-19T01:13:15.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.450+0000 7f3402a00640 1 -- 172.21.15.69:0/1574955113 shutdown_connections 2024-03-19T01:13:15.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.450+0000 7f3402a00640 1 --2- 172.21.15.69:0/1574955113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33fc10d100 0x7f33fc114cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.450+0000 7f3402a00640 1 --2- 172.21.15.69:0/1574955113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33fc1089e0 0x7f33fc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.450+0000 7f3402a00640 1 --2- 172.21.15.69:0/1574955113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33fc108030 0x7f33fc108410 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.450+0000 7f3402a00640 1 -- 172.21.15.69:0/1574955113 >> 172.21.15.69:0/1574955113 conn(0x7f33fc075d60 msgr2=0x7f33fc078180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:15.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.450+0000 7f3402a00640 1 -- 172.21.15.69:0/1574955113 shutdown_connections 2024-03-19T01:13:15.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.450+0000 7f3402a00640 1 -- 172.21.15.69:0/1574955113 wait complete. 2024-03-19T01:13:15.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.451+0000 7f3402a00640 1 Processor -- start 2024-03-19T01:13:15.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.451+0000 7f3402a00640 1 -- start start 2024-03-19T01:13:15.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.451+0000 7f3402a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33fc108030 0x7f33fc19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.451+0000 7f3402a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33fc10d100 0x7f33fc19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.451+0000 7f3402a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33fc1a2f30 0x7f33fc1a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.451+0000 7f3402a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc118190 con 0x7f33fc108030 2024-03-19T01:13:15.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.451+0000 7f3402a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc118010 con 0x7f33fc10d100 2024-03-19T01:13:15.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.451+0000 7f3402a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc118310 con 0x7f33fc1a2f30 2024-03-19T01:13:15.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.451+0000 7f33fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33fc10d100 0x7f33fc19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.451+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33fc108030 0x7f33fc19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.452+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33fc108030 0x7f33fc19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37526/0 (socket says 172.21.15.69:37526) 2024-03-19T01:13:15.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.452+0000 7f33fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33fc10d100 0x7f33fc19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42010/0 (socket says 172.21.15.69:42010) 2024-03-19T01:13:15.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.452+0000 7f33fbe00640 1 -- 172.21.15.69:0/573378425 learned_addr learned my addr 172.21.15.69:0/573378425 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:15.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.452+0000 7f3400a00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33fc1a2f30 0x7f33fc1a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.452+0000 7f33fbe00640 1 -- 172.21.15.69:0/573378425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33fc1a2f30 msgr2=0x7f33fc1a33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.452+0000 7f33fbe00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33fc1a2f30 0x7f33fc1a33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.452+0000 7f33fbe00640 1 -- 172.21.15.69:0/573378425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33fc10d100 msgr2=0x7f33fc19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.452+0000 7f33fbe00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33fc10d100 0x7f33fc19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.452+0000 7f33fbe00640 1 -- 172.21.15.69:0/573378425 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc1a3a20 con 0x7f33fc108030 2024-03-19T01:13:15.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.452+0000 7f33fb400640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33fc10d100 0x7f33fc19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:15.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.452+0000 7f33fbe00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33fc108030 0x7f33fc19e670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006010 tx=0x7f33ec02cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:15.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.453+0000 7f33f8c00640 1 -- 172.21.15.69:0/573378425 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec041070 con 0x7f33fc108030 2024-03-19T01:13:15.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.453+0000 7f3402a00640 1 -- 172.21.15.69:0/573378425 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33fc1a3cb0 con 0x7f33fc108030 2024-03-19T01:13:15.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.453+0000 7f33f8c00640 1 -- 172.21.15.69:0/573378425 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33ec004450 con 0x7f33fc108030 2024-03-19T01:13:15.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.453+0000 7f3402a00640 1 -- 172.21.15.69:0/573378425 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33fc1ab4d0 con 0x7f33fc108030 2024-03-19T01:13:15.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.453+0000 7f33f8c00640 1 -- 172.21.15.69:0/573378425 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec04b060 con 0x7f33fc108030 2024-03-19T01:13:15.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.456+0000 7f34057dc740 1 Processor -- start 2024-03-19T01:13:15.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.456+0000 7f34057dc740 1 -- start start 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f34057dc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f34057dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d1660 0x561c198d9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f34057dc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198dc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f34057dc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561c198352c0 con 0x561c198da240 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f34057dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c197ddb60 con 0x561c198d1660 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f34057dc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561c19831360 con 0x561c198d0d40 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f3400a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f33fb400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198dc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f3400a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45582/0 (socket says 172.21.15.69:45582) 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f3400a00640 1 -- 172.21.15.69:0/2373690498 learned_addr learned my addr 172.21.15.69:0/2373690498 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f33f8c00640 1 -- 172.21.15.69:0/573378425 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f33ec044070 con 0x7f33fc108030 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.457+0000 7f3400a00640 1 -- 172.21.15.69:0/2373690498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d1660 msgr2=0x561c198d9d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:13:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.458+0000 7f3400a00640 1 --2- 172.21.15.69:0/2373690498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d1660 0x561c198d9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.458+0000 7f3400a00640 1 -- 172.21.15.69:0/2373690498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 msgr2=0x561c198dc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:15.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.458+0000 7f3400a00640 1 --2- 172.21.15.69:0/2373690498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198dc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.458+0000 7f3400a00640 1 -- 172.21.15.69:0/2373690498 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c196137d0 con 0x561c198d0d40 2024-03-19T01:13:15.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.458+0000 7f33fb400640 1 --2- 172.21.15.69:0/2373690498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198dc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:15.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.458+0000 7f3400a00640 1 --2- 172.21.15.69:0/2373690498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d1120 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f33f4007ec0 tx=0x7f33f400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59297e6e81b2797f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:15.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.458+0000 7f33f1200640 1 -- 172.21.15.69:0/2373690498 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4010840 con 0x561c198d0d40 2024-03-19T01:13:15.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.458+0000 7f33f1200640 1 -- 172.21.15.69:0/2373690498 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f40183f0 con 0x561c198d0d40 2024-03-19T01:13:15.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.458+0000 7f33f1200640 1 -- 172.21.15.69:0/2373690498 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4017600 con 0x561c198d0d40 2024-03-19T01:13:15.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.459+0000 7f33f8c00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f33e00a2970 0x7f33e00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.459+0000 7f33f8c00640 1 -- 172.21.15.69:0/573378425 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f33ec0f1010 con 0x7f33fc108030 2024-03-19T01:13:15.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.459+0000 7f33fbe00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f33e00a2970 0x7f33e00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.460+0000 7f34057dc740 1 -- 172.21.15.69:0/2373690498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 msgr2=0x561c198d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.460+0000 7f34057dc740 1 --2- 172.21.15.69:0/2373690498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d1120 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f33f4007ec0 tx=0x7f33f400f790 comp rx=0 tx=0).stop 2024-03-19T01:13:15.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.460+0000 7f34057dc740 1 -- 172.21.15.69:0/2373690498 shutdown_connections 2024-03-19T01:13:15.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.460+0000 7f34057dc740 1 --2- 172.21.15.69:0/2373690498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198dc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.460+0000 7f34057dc740 1 --2- 172.21.15.69:0/2373690498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d1660 0x561c198d9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.460+0000 7f34057dc740 1 --2- 172.21.15.69:0/2373690498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d1120 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.460+0000 7f34057dc740 1 -- 172.21.15.69:0/2373690498 >> 172.21.15.69:0/2373690498 conn(0x561c198cb470 msgr2=0x561c198cb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:15.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.460+0000 7f33fbe00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f33e00a2970 0x7f33e00a4e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x561c198d0810 tx=0x7f33ec009600 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:15.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.460+0000 7f34057dc740 1 -- 172.21.15.69:0/2373690498 shutdown_connections 2024-03-19T01:13:15.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.460+0000 7f34057dc740 1 -- 172.21.15.69:0/2373690498 wait complete. 2024-03-19T01:13:15.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.460+0000 7f34057dc740 1 -- start start 2024-03-19T01:13:15.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.461+0000 7f34057dc740 1 Processor -- start 2024-03-19T01:13:15.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.461+0000 7f34057dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d0d40 0x561c198d7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.461+0000 7f34057dc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198d1660 0x561c198d7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.461+0000 7f34057dc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198da240 0x561c198d7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.461+0000 7f34057dc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561c198dd030 con 0x561c198d1660 2024-03-19T01:13:15.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.461+0000 7f34057dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c197f0fc0 con 0x561c198d0d40 2024-03-19T01:13:15.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.461+0000 7f34057dc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561c198dd1b0 con 0x561c198da240 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.461+0000 7f3400a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d0d40 0x561c198d7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.461+0000 7f3400a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d0d40 0x561c198d7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42018/0 (socket says 172.21.15.69:42018) 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.461+0000 7f3400a00640 1 -- 172.21.15.69:0/474422810 learned_addr learned my addr 172.21.15.69:0/474422810 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.462+0000 7f3400a00640 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198da240 0x561c198d7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.462+0000 7f34057dc740 10 client.0 _ll_register_callbacks cb 0x561c197f0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.462+0000 7f3400a00640 1 -- 172.21.15.69:0/474422810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198da240 msgr2=0x561c198d7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.462+0000 7f3400a00640 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198da240 0x561c198d7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.462+0000 7f3400a00640 1 -- 172.21.15.69:0/474422810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198d1660 msgr2=0x561c198d7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.462+0000 7f3400a00640 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198d1660 0x561c198d7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.462+0000 7f3400a00640 1 -- 172.21.15.69:0/474422810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c195493f0 con 0x561c198d0d40 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.462+0000 7f3400a00640 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d0d40 0x561c198d7300 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f33f4007c50 tx=0x7f33f4012040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c127e9d14249712 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.462+0000 7f33e5600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:15.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.462+0000 7f33e5600640 1 -- 172.21.15.69:0/474422810 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f402c060 con 0x561c198d0d40 2024-03-19T01:13:15.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.462+0000 7f33e5600640 1 -- 172.21.15.69:0/474422810 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f33f4003b70 con 0x561c198d0d40 2024-03-19T01:13:15.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.463+0000 7f34057dc740 1 -- 172.21.15.69:0/474422810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f40183f0 con 0x561c198d0d40 2024-03-19T01:13:15.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.463+0000 7f33e5600640 1 -- 172.21.15.69:0/474422810 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f33f4003d70 con 0x561c198d0d40 2024-03-19T01:13:15.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.463+0000 7f34057dc740 10 client.26297 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:15.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.463+0000 7f34057dc740 1 -- 172.21.15.69:0/474422810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561c195090f0 con 0x561c198d0d40 2024-03-19T01:13:15.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.463+0000 7f33e5600640 1 -- 172.21.15.69:0/474422810 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f40183f0 con 0x561c198d0d40 2024-03-19T01:13:15.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.463+0000 7f33e5600640 1 -- 172.21.15.69:0/474422810 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f33f4024040 con 0x561c198d0d40 2024-03-19T01:13:15.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.463+0000 7f34057dc740 10 client.26297 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:15.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.463+0000 7f34057dc740 10 client.26297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:15.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.464+0000 7f34057dc740 20 client.26297 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:15.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.464+0000 7f34057dc740 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561c198fb620 0x561c198fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.464+0000 7f34057dc740 4 client.26297 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:15.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.464+0000 7f34057dc740 1 -- 172.21.15.69:0/474422810 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c197d6610 con 0x561c198fb620 2024-03-19T01:13:15.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.465+0000 7f33fb400640 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561c198fb620 0x561c198fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.467+0000 7f33fb400640 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561c198fb620 0x561c198fda10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56abb2df6a2b729b server_cookie=1accd747115fecaa in_seq=0 out_seq=0 2024-03-19T01:13:15.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.467+0000 7f33e5600640 10 client.26297 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:15.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f33e5600640 1 -- 172.21.15.69:0/474422810 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561c197d6610 con 0x561c198fb620 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f33e5600640 10 client.26297 handle_command_reply: tid=0 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f34057dc740 1 client.26297 shutdown 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f34057dc740 1 -- 172.21.15.69:0/474422810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d0d40 msgr2=0x561c198d7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f34057dc740 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d0d40 0x561c198d7300 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f33f4007c50 tx=0x7f33f4012040 comp rx=0 tx=0).stop 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f34057dc740 1 -- 172.21.15.69:0/474422810 shutdown_connections 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f34057dc740 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561c198fb620 0x561c198fda10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f34057dc740 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198da240 0x561c198d7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f34057dc740 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198d1660 0x561c198d7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f34057dc740 1 --2- 172.21.15.69:0/474422810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d0d40 0x561c198d7300 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f34057dc740 1 -- 172.21.15.69:0/474422810 >> 172.21.15.69:0/474422810 conn(0x561c198cb470 msgr2=0x561c198cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.469+0000 7f33e5600640 0 client.26297 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.470+0000 7f34057dc740 1 -- 172.21.15.69:0/474422810 shutdown_connections 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.470+0000 7f34057dc740 1 -- 172.21.15.69:0/474422810 wait complete. 2024-03-19T01:13:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.470+0000 7f34057dc740 20 client.26297 trim_cache size 0 max 16384 2024-03-19T01:13:15.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.490+0000 7f34057dc740 1 Processor -- start 2024-03-19T01:13:15.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.491+0000 7f34057dc740 1 -- start start 2024-03-19T01:13:15.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.491+0000 7f34057dc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.491+0000 7f34057dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d1660 0x561c198e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.491+0000 7f34057dc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198ca530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.491+0000 7f34057dc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc07c570 con 0x561c198da240 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.491+0000 7f34057dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc07e170 con 0x561c198d1660 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.491+0000 7f34057dc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc07cc40 con 0x561c198d0d40 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.491+0000 7f33fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d1660 0x561c198e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.491+0000 7f33fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d1660 0x561c198e5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42020/0 (socket says 172.21.15.69:42020) 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.491+0000 7f33fb400640 1 -- 172.21.15.69:0/1349153435 learned_addr learned my addr 172.21.15.69:0/1349153435 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.491+0000 7f3400a00640 1 --2- 172.21.15.69:0/1349153435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.492+0000 7f3400a00640 1 --2- 172.21.15.69:0/1349153435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.492+0000 7f33fb400640 1 -- 172.21.15.69:0/1349153435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 msgr2=0x561c198d9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.492+0000 7f33fb400640 1 --2- 172.21.15.69:0/1349153435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.492+0000 7f33fb400640 1 -- 172.21.15.69:0/1349153435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 msgr2=0x561c198ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.492+0000 7f33fb400640 1 --2- 172.21.15.69:0/1349153435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.492+0000 7f33fb400640 1 -- 172.21.15.69:0/1349153435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c198dc9a0 con 0x561c198d1660 2024-03-19T01:13:15.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.492+0000 7f33fb400640 1 --2- 172.21.15.69:0/1349153435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d1660 0x561c198e5e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f33e8004700 tx=0x7f33e80119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5abafa7ae83abc38 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:15.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.492+0000 7f33e7e00640 1 -- 172.21.15.69:0/1349153435 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e801c430 con 0x561c198d1660 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.493+0000 7f33e7e00640 1 -- 172.21.15.69:0/1349153435 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e8005500 con 0x561c198d1660 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.493+0000 7f33e7e00640 1 -- 172.21.15.69:0/1349153435 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8010430 con 0x561c198d1660 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.493+0000 7f34057dc740 1 -- 172.21.15.69:0/1349153435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d1660 msgr2=0x561c198e5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.493+0000 7f34057dc740 1 --2- 172.21.15.69:0/1349153435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d1660 0x561c198e5e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f33e8004700 tx=0x7f33e80119b0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.493+0000 7f34057dc740 1 -- 172.21.15.69:0/1349153435 shutdown_connections 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.493+0000 7f34057dc740 1 --2- 172.21.15.69:0/1349153435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.493+0000 7f34057dc740 1 --2- 172.21.15.69:0/1349153435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c198d1660 0x561c198e5e00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.493+0000 7f34057dc740 1 --2- 172.21.15.69:0/1349153435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.493+0000 7f34057dc740 1 -- 172.21.15.69:0/1349153435 >> 172.21.15.69:0/1349153435 conn(0x561c198cb470 msgr2=0x561c198c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.493+0000 7f34057dc740 1 -- 172.21.15.69:0/1349153435 shutdown_connections 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.493+0000 7f34057dc740 1 -- 172.21.15.69:0/1349153435 wait complete. 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.494+0000 7f34057dc740 1 -- start start 2024-03-19T01:13:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.494+0000 7f34057dc740 1 Processor -- start 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.494+0000 7f34057dc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.494+0000 7f34057dc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.494+0000 7f34057dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c1995e6c0 0x561c19960ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.494+0000 7f34057dc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc07d9e0 con 0x561c198da240 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.494+0000 7f34057dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc07d310 con 0x561c1995e6c0 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.494+0000 7f34057dc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc07a550 con 0x561c198d0d40 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.494+0000 7f34057dc740 10 client.0 _ll_register_callbacks cb 0x7f33f4018700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f3400a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f3400a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45618/0 (socket says 172.21.15.69:45618) 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f3400a00640 1 -- 172.21.15.69:0/832488358 learned_addr learned my addr 172.21.15.69:0/832488358 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f3400a00640 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c1995e6c0 0x561c19960ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f33fb400640 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f3400a00640 1 -- 172.21.15.69:0/832488358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c1995e6c0 msgr2=0x561c19960ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f3400a00640 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c1995e6c0 0x561c19960ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f3400a00640 1 -- 172.21.15.69:0/832488358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 msgr2=0x561c198d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f3400a00640 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198d4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f3400a00640 1 -- 172.21.15.69:0/832488358 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c19961020 con 0x561c198d0d40 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f3400a00640 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d4540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33f4004630 tx=0x7f33f4018e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17542e655213b36c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f33e5600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.495+0000 7f33e5600640 1 -- 172.21.15.69:0/832488358 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f402c040 con 0x561c198d0d40 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f34057dc740 1 -- 172.21.15.69:0/832488358 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e8005500 con 0x561c198d0d40 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f33e5600640 1 -- 172.21.15.69:0/832488358 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f33f4004bb0 con 0x561c198d0d40 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f33e5600640 1 -- 172.21.15.69:0/832488358 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f33f4036080 con 0x561c198d0d40 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f34057dc740 10 client.26287 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f34057dc740 1 -- 172.21.15.69:0/832488358 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561c199622f0 con 0x561c198d0d40 2024-03-19T01:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f33e5600640 1 -- 172.21.15.69:0/832488358 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e8005500 con 0x561c198d0d40 2024-03-19T01:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f33e5600640 1 -- 172.21.15.69:0/832488358 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f33f4017c50 con 0x561c198d0d40 2024-03-19T01:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f34057dc740 10 client.26287 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f34057dc740 10 client.26287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f34057dc740 20 client.26287 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f34057dc740 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561c199629e0 0x561c19964e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f34057dc740 4 client.26287 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.496+0000 7f34057dc740 1 -- 172.21.15.69:0/832488358 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c19842800 con 0x561c199629e0 2024-03-19T01:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.497+0000 7f33fb400640 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561c199629e0 0x561c19964e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.497+0000 7f33fb400640 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561c199629e0 0x561c19964e00 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb3bda906030f6a server_cookie=60a31c1ae6895a87 in_seq=0 out_seq=0 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.497+0000 7f33e5600640 10 client.26287 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.499+0000 7f33e5600640 1 -- 172.21.15.69:0/832488358 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561c19842800 con 0x561c199629e0 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f33e5600640 10 client.26287 handle_command_reply: tid=0 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 1 client.26287 shutdown 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 1 -- 172.21.15.69:0/832488358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 msgr2=0x561c198d4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d4540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33f4004630 tx=0x7f33f4018e30 comp rx=0 tx=0).stop 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 1 -- 172.21.15.69:0/832488358 shutdown_connections 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561c199629e0 0x561c19964e00 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c1995e6c0 0x561c19960ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c198da240 0x561c198d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f33e5600640 0 client.26287 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 1 --2- 172.21.15.69:0/832488358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561c198d0d40 0x561c198d4540 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 1 -- 172.21.15.69:0/832488358 >> 172.21.15.69:0/832488358 conn(0x561c198cb470 msgr2=0x561c198ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:15.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 1 -- 172.21.15.69:0/832488358 shutdown_connections 2024-03-19T01:13:15.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 1 -- 172.21.15.69:0/832488358 wait complete. 2024-03-19T01:13:15.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.500+0000 7f34057dc740 20 client.26287 trim_cache size 0 max 16384 2024-03-19T01:13:15.503 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:13:15.503 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:13:15.503 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:13:15.503 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:13:15.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.501+0000 7f33f1c00640 1 -- 172.21.15.69:0/573378425 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f33e00a2970 msgr2=0x7f33e00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.501+0000 7f33f1c00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f33e00a2970 0x7f33e00a4e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x561c198d0810 tx=0x7f33ec009600 comp rx=0 tx=0).stop 2024-03-19T01:13:15.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.501+0000 7f33f1c00640 1 -- 172.21.15.69:0/573378425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33fc108030 msgr2=0x7f33fc19e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.501+0000 7f33f1c00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33fc108030 0x7f33fc19e670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006010 tx=0x7f33ec02cc80 comp rx=0 tx=0).stop 2024-03-19T01:13:15.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.501+0000 7f33f1c00640 1 -- 172.21.15.69:0/573378425 shutdown_connections 2024-03-19T01:13:15.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.501+0000 7f33f1c00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33fc1a2f30 0x7f33fc1a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.501+0000 7f33f1c00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f33e00a2970 0x7f33e00a4e30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.501+0000 7f33f1c00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33fc10d100 0x7f33fc19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.501+0000 7f33f1c00640 1 --2- 172.21.15.69:0/573378425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33fc108030 0x7f33fc19e670 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006010 tx=0x7f33ec02cc80 comp rx=0 tx=0).stop 2024-03-19T01:13:15.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.501+0000 7f33f1c00640 1 -- 172.21.15.69:0/573378425 >> 172.21.15.69:0/573378425 conn(0x7f33fc075d60 msgr2=0x7f33fc0780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:15.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.502+0000 7f33f1c00640 1 -- 172.21.15.69:0/573378425 shutdown_connections 2024-03-19T01:13:15.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.502+0000 7f33f1c00640 1 -- 172.21.15.69:0/573378425 wait complete. 2024-03-19T01:13:15.525 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:13:15.526 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:13:15.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:15 smithi069 ceph-mon[31727]: pgmap v80: 129 pgs: 129 active+clean; 290 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2024-03-19T01:13:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:15 smithi163 ceph-mon[67428]: pgmap v80: 129 pgs: 129 active+clean; 290 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2024-03-19T01:13:15.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.740+0000 7f5056400640 1 -- 172.21.15.69:0/1508652088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5050073c50 msgr2=0x7f5050074030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.740+0000 7f5056400640 1 --2- 172.21.15.69:0/1508652088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5050073c50 0x7f5050074030 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5044009ca0 tx=0x7f504402c190 comp rx=0 tx=0).stop 2024-03-19T01:13:15.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.740+0000 7f5056400640 1 -- 172.21.15.69:0/1508652088 shutdown_connections 2024-03-19T01:13:15.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.740+0000 7f5056400640 1 --2- 172.21.15.69:0/1508652088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f505010de50 0x7f50501102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.740+0000 7f5056400640 1 --2- 172.21.15.69:0/1508652088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5050074570 0x7f505010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.740+0000 7f5056400640 1 --2- 172.21.15.69:0/1508652088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5050073c50 0x7f5050074030 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.740+0000 7f5056400640 1 -- 172.21.15.69:0/1508652088 >> 172.21.15.69:0/1508652088 conn(0x7f50500fd120 msgr2=0x7f50500ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:15.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.740+0000 7f5056400640 1 -- 172.21.15.69:0/1508652088 shutdown_connections 2024-03-19T01:13:15.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.741+0000 7f5056400640 1 -- 172.21.15.69:0/1508652088 wait complete. 2024-03-19T01:13:15.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.741+0000 7f5056400640 1 Processor -- start 2024-03-19T01:13:15.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f5056400640 1 -- start start 2024-03-19T01:13:15.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f5056400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5050073c50 0x7f505019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f5056400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5050074570 0x7f505019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f5056400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505010de50 0x7f505019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f5056400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5050113bb0 con 0x7f5050073c50 2024-03-19T01:13:15.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f5056400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5050113a30 con 0x7f505010de50 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f5056400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5050113d30 con 0x7f5050074570 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f5055a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505010de50 0x7f505019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f5055000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5050073c50 0x7f505019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f5055a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505010de50 0x7f505019e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:42036/0 (socket says 172.21.15.69:42036) 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f5055a00640 1 -- 172.21.15.69:0/123999163 learned_addr learned my addr 172.21.15.69:0/123999163 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.742+0000 7f504fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5050074570 0x7f505019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.743+0000 7f5055a00640 1 -- 172.21.15.69:0/123999163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5050074570 msgr2=0x7f505019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.743+0000 7f5055a00640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5050074570 0x7f505019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.743+0000 7f5055a00640 1 -- 172.21.15.69:0/123999163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5050073c50 msgr2=0x7f505019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.743+0000 7f5055a00640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5050073c50 0x7f505019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.743+0000 7f5055a00640 1 -- 172.21.15.69:0/123999163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505019f120 con 0x7f505010de50 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.743+0000 7f5055a00640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505010de50 0x7f505019e9f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f504800e970 tx=0x7f504800ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.743+0000 7f504d600640 1 -- 172.21.15.69:0/123999163 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504800ccc0 con 0x7f505010de50 2024-03-19T01:13:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.743+0000 7f504d600640 1 -- 172.21.15.69:0/123999163 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f504800ce60 con 0x7f505010de50 2024-03-19T01:13:15.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.743+0000 7f5056400640 1 -- 172.21.15.69:0/123999163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505019f410 con 0x7f505010de50 2024-03-19T01:13:15.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.743+0000 7f504d600640 1 -- 172.21.15.69:0/123999163 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048010490 con 0x7f505010de50 2024-03-19T01:13:15.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.743+0000 7f5056400640 1 -- 172.21.15.69:0/123999163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50501a6ca0 con 0x7f505010de50 2024-03-19T01:13:15.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.744+0000 7f5056400640 1 -- 172.21.15.69:0/123999163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5050068530 con 0x7f505010de50 2024-03-19T01:13:15.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.747+0000 7f504d600640 1 -- 172.21.15.69:0/123999163 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f5048005000 con 0x7f505010de50 2024-03-19T01:13:15.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.749+0000 7f504d600640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f50380a2970 0x7f50380a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:15.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.749+0000 7f504d600640 1 -- 172.21.15.69:0/123999163 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f50480cf850 con 0x7f505010de50 2024-03-19T01:13:15.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.749+0000 7f5055000640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f50380a2970 0x7f50380a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:15.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.750+0000 7f5055000640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f50380a2970 0x7f50380a4e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5044000f80 tx=0x7f5044009360 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:15.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:15.754+0000 7f504d600640 1 -- 172.21.15.69:0/123999163 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5048053740 con 0x7f505010de50 2024-03-19T01:13:15.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:15 smithi086 ceph-mon[66829]: pgmap v80: 129 pgs: 129 active+clean; 290 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2024-03-19T01:13:16.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.018+0000 7f5056400640 1 -- 172.21.15.69:0/123999163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f505002d1f0 con 0x7f505010de50 2024-03-19T01:13:16.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.020+0000 7f504d600640 1 -- 172.21.15.69:0/123999163 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11937 (secure 0 0 0) 0x7f504809c190 con 0x7f505010de50 2024-03-19T01:13:16.021 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:13:16.021 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:12:18.556169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:13:16.021 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 31 2024-03-19T01:13:16.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 -- 172.21.15.69:0/123999163 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f50380a2970 msgr2=0x7f50380a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f50380a2970 0x7f50380a4e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5044000f80 tx=0x7f5044009360 comp rx=0 tx=0).stop 2024-03-19T01:13:16.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 -- 172.21.15.69:0/123999163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505010de50 msgr2=0x7f505019e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505010de50 0x7f505019e9f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f504800e970 tx=0x7f504800ed70 comp rx=0 tx=0).stop 2024-03-19T01:13:16.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 -- 172.21.15.69:0/123999163 shutdown_connections 2024-03-19T01:13:16.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f50380a2970 0x7f50380a4e30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f505010de50 0x7f505019e9f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5050074570 0x7f505019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 --2- 172.21.15.69:0/123999163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5050073c50 0x7f505019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 -- 172.21.15.69:0/123999163 >> 172.21.15.69:0/123999163 conn(0x7f50500fd120 msgr2=0x7f50500fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 -- 172.21.15.69:0/123999163 shutdown_connections 2024-03-19T01:13:16.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.023+0000 7f5056400640 1 -- 172.21.15.69:0/123999163 wait complete. 2024-03-19T01:13:16.045 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:13:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.343+0000 7fb37d000640 1 -- 172.21.15.69:0/39509148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb378102050 msgr2=0x7fb37810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.343+0000 7fb37d000640 1 --2- 172.21.15.69:0/39509148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb378102050 0x7fb37810e560 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb370009cd0 tx=0x7fb37002c0e0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.344+0000 7fb37d000640 1 -- 172.21.15.69:0/39509148 shutdown_connections 2024-03-19T01:13:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.344+0000 7fb37d000640 1 --2- 172.21.15.69:0/39509148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb378102050 0x7fb37810e560 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.344+0000 7fb37d000640 1 --2- 172.21.15.69:0/39509148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb378101690 0x7fb378101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.344+0000 7fb37d000640 1 --2- 172.21.15.69:0/39509148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3781076a0 0x7fb378107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.344+0000 7fb37d000640 1 -- 172.21.15.69:0/39509148 >> 172.21.15.69:0/39509148 conn(0x7fb3780fd1c0 msgr2=0x7fb3780ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.344+0000 7fb37d000640 1 -- 172.21.15.69:0/39509148 shutdown_connections 2024-03-19T01:13:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.344+0000 7fb37d000640 1 -- 172.21.15.69:0/39509148 wait complete. 2024-03-19T01:13:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.345+0000 7fb37d000640 1 Processor -- start 2024-03-19T01:13:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.345+0000 7fb37d000640 1 -- start start 2024-03-19T01:13:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.345+0000 7fb37d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb378101690 0x7fb37819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.345+0000 7fb37d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb378102050 0x7fb37819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.345+0000 7fb37d000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3781076a0 0x7fb37819e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.345+0000 7fb37d000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb378111ab0 con 0x7fb3781076a0 2024-03-19T01:13:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.345+0000 7fb37d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb378111930 con 0x7fb378102050 2024-03-19T01:13:16.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.345+0000 7fb37d000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb378111c30 con 0x7fb378101690 2024-03-19T01:13:16.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb377400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb378101690 0x7fb37819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb376a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb378102050 0x7fb37819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb377400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb378101690 0x7fb37819a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45844/0 (socket says 172.21.15.69:45844) 2024-03-19T01:13:16.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb377400640 1 -- 172.21.15.69:0/3755059705 learned_addr learned my addr 172.21.15.69:0/3755059705 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb377e00640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3781076a0 0x7fb37819e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb377400640 1 -- 172.21.15.69:0/3755059705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb378102050 msgr2=0x7fb37819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb377400640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb378102050 0x7fb37819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb377400640 1 -- 172.21.15.69:0/3755059705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3781076a0 msgr2=0x7fb37819e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb377400640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3781076a0 0x7fb37819e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb377400640 1 -- 172.21.15.69:0/3755059705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37819f120 con 0x7fb378101690 2024-03-19T01:13:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb376a00640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb378102050 0x7fb37819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.346+0000 7fb377400640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb378101690 0x7fb37819a130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb36800ea60 tx=0x7fb36800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.347+0000 7fb36fe00640 1 -- 172.21.15.69:0/3755059705 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36800cde0 con 0x7fb378101690 2024-03-19T01:13:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.347+0000 7fb36fe00640 1 -- 172.21.15.69:0/3755059705 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3680035f0 con 0x7fb378101690 2024-03-19T01:13:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.347+0000 7fb37d000640 1 -- 172.21.15.69:0/3755059705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37819f410 con 0x7fb378101690 2024-03-19T01:13:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.347+0000 7fb36fe00640 1 -- 172.21.15.69:0/3755059705 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368010670 con 0x7fb378101690 2024-03-19T01:13:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.347+0000 7fb37d000640 1 -- 172.21.15.69:0/3755059705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3781a6ca0 con 0x7fb378101690 2024-03-19T01:13:16.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.350+0000 7fb36fe00640 1 -- 172.21.15.69:0/3755059705 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fb3680037a0 con 0x7fb378101690 2024-03-19T01:13:16.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.350+0000 7fb381215740 1 Processor -- start 2024-03-19T01:13:16.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.350+0000 7fb381215740 1 -- start start 2024-03-19T01:13:16.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.351+0000 7fb381215740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805400 0x559fe78057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.351+0000 7fb381215740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805d20 0x559fe780e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.351+0000 7fb381215740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe7810cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.351+0000 7fb381215740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559fe7761380 con 0x559fe780e900 2024-03-19T01:13:16.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.351+0000 7fb381215740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559fe7709d20 con 0x559fe7805d20 2024-03-19T01:13:16.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.351+0000 7fb381215740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559fe775d420 con 0x559fe7805400 2024-03-19T01:13:16.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.351+0000 7fb377400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805d20 0x559fe780e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.351+0000 7fb377e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805400 0x559fe78057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.351+0000 7fb377e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805400 0x559fe78057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45850/0 (socket says 172.21.15.69:45850) 2024-03-19T01:13:16.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.351+0000 7fb377e00640 1 -- 172.21.15.69:0/1867156345 learned_addr learned my addr 172.21.15.69:0/1867156345 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb376a00640 1 --2- 172.21.15.69:0/1867156345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe7810cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb377400640 1 -- 172.21.15.69:0/1867156345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805400 msgr2=0x559fe78057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb377400640 1 --2- 172.21.15.69:0/1867156345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805400 0x559fe78057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb377400640 1 -- 172.21.15.69:0/1867156345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 msgr2=0x559fe7810cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb377400640 1 --2- 172.21.15.69:0/1867156345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe7810cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb377400640 1 -- 172.21.15.69:0/1867156345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe74753f0 con 0x559fe7805d20 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb36fe00640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb35c0a29f0 0x7fb35c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb377e00640 1 --2- 172.21.15.69:0/1867156345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805400 0x559fe78057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb376a00640 1 --2- 172.21.15.69:0/1867156345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe7810cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb36fe00640 1 -- 172.21.15.69:0/3755059705 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fb3680cf440 con 0x7fb378101690 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb377400640 1 --2- 172.21.15.69:0/1867156345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805d20 0x559fe780e3c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb3680cc7d0 tx=0x7fb3680d5920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ee62cb917a0feb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb377e00640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb35c0a29f0 0x7fb35c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.352+0000 7fb36cc00640 1 -- 172.21.15.69:0/1867156345 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368097980 con 0x559fe7805d20 2024-03-19T01:13:16.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.353+0000 7fb36cc00640 1 -- 172.21.15.69:0/1867156345 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3680860c0 con 0x559fe7805d20 2024-03-19T01:13:16.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.353+0000 7fb36cc00640 1 -- 172.21.15.69:0/1867156345 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36809b040 con 0x559fe7805d20 2024-03-19T01:13:16.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.357+0000 7fb381215740 1 -- 172.21.15.69:0/1867156345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805d20 msgr2=0x559fe780e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.357+0000 7fb381215740 1 --2- 172.21.15.69:0/1867156345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805d20 0x559fe780e3c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb3680cc7d0 tx=0x7fb3680d5920 comp rx=0 tx=0).stop 2024-03-19T01:13:16.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.357+0000 7fb377e00640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb35c0a29f0 0x7fb35c0a4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb370005e30 tx=0x7fb370005d80 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.358+0000 7fb381215740 1 -- 172.21.15.69:0/1867156345 shutdown_connections 2024-03-19T01:13:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.358+0000 7fb381215740 1 --2- 172.21.15.69:0/1867156345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe7810cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.358+0000 7fb381215740 1 --2- 172.21.15.69:0/1867156345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805d20 0x559fe780e3c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.358+0000 7fb381215740 1 --2- 172.21.15.69:0/1867156345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805400 0x559fe78057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.358+0000 7fb381215740 1 -- 172.21.15.69:0/1867156345 >> 172.21.15.69:0/1867156345 conn(0x559fe77ffb30 msgr2=0x559fe77fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.358+0000 7fb381215740 1 -- 172.21.15.69:0/1867156345 shutdown_connections 2024-03-19T01:13:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.358+0000 7fb381215740 1 -- 172.21.15.69:0/1867156345 wait complete. 2024-03-19T01:13:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.358+0000 7fb381215740 1 -- start start 2024-03-19T01:13:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.358+0000 7fb381215740 1 Processor -- start 2024-03-19T01:13:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb381215740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe780b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb381215740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 0x559fe780bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb381215740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe780c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb381215740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559fe78116f0 con 0x559fe780e900 2024-03-19T01:13:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb381215740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559fe771d160 con 0x559fe7805400 2024-03-19T01:13:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb381215740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559fe7811870 con 0x559fe7805d20 2024-03-19T01:13:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb381215740 10 client.0 _ll_register_callbacks cb 0x559fe7759170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb377400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe780b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb376a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 0x559fe780bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb377400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe780b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52692/0 (socket says 172.21.15.69:52692) 2024-03-19T01:13:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb377400640 1 -- 172.21.15.69:0/2175148570 learned_addr learned my addr 172.21.15.69:0/2175148570 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb376a00640 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 0x559fe780bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45864/0 (socket says 172.21.15.69:45864) 2024-03-19T01:13:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.359+0000 7fb377e00640 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe780c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.360+0000 7fb377400640 1 -- 172.21.15.69:0/2175148570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 msgr2=0x559fe780bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.360+0000 7fb377400640 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 0x559fe780bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.360+0000 7fb377400640 1 -- 172.21.15.69:0/2175148570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 msgr2=0x559fe780c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.360+0000 7fb377400640 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe780c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.360+0000 7fb377400640 1 -- 172.21.15.69:0/2175148570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe753f7d0 con 0x559fe7805400 2024-03-19T01:13:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.360+0000 7fb377e00640 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe780c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.360+0000 7fb377400640 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe780b9d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb368009230 tx=0x7fb368099bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c544a81827e039c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.360+0000 7fb361600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.360+0000 7fb361600640 1 -- 172.21.15.69:0/2175148570 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36809b220 con 0x559fe7805400 2024-03-19T01:13:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.360+0000 7fb361600640 1 -- 172.21.15.69:0/2175148570 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fb3680cca50 con 0x559fe7805400 2024-03-19T01:13:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.360+0000 7fb381215740 1 -- 172.21.15.69:0/2175148570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559fe776e8c0 con 0x559fe7805400 2024-03-19T01:13:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.361+0000 7fb361600640 1 -- 172.21.15.69:0/2175148570 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb36809b4b0 con 0x559fe7805400 2024-03-19T01:13:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.361+0000 7fb381215740 10 client.26321 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:16.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.361+0000 7fb381215740 1 -- 172.21.15.69:0/2175148570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559fe78116f0 con 0x559fe7805400 2024-03-19T01:13:16.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.361+0000 7fb361600640 1 -- 172.21.15.69:0/2175148570 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559fe776e8c0 con 0x559fe7805400 2024-03-19T01:13:16.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.361+0000 7fb361600640 1 -- 172.21.15.69:0/2175148570 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb368097d50 con 0x559fe7805400 2024-03-19T01:13:16.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.361+0000 7fb381215740 10 client.26321 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:16.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.361+0000 7fb381215740 10 client.26321 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:13:16.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.361+0000 7fb381215740 20 client.26321 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.361+0000 7fb381215740 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559fe781eb10 0x559fe7820f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.361+0000 7fb381215740 4 client.26321 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.361+0000 7fb381215740 1 -- 172.21.15.69:0/2175148570 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fe7768c10 con 0x559fe781eb10 2024-03-19T01:13:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.362+0000 7fb376a00640 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559fe781eb10 0x559fe7820f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.362+0000 7fb376a00640 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559fe781eb10 0x559fe7820f00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1e903bb0a477eeb server_cookie=84a41c5051ef72b9 in_seq=0 out_seq=0 2024-03-19T01:13:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.362+0000 7fb361600640 10 client.26321 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:13:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.363+0000 7fb361600640 1 -- 172.21.15.69:0/2175148570 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559fe7768c10 con 0x559fe781eb10 2024-03-19T01:13:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.363+0000 7fb361600640 10 client.26321 handle_command_reply: tid=0 2024-03-19T01:13:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.363+0000 7fb381215740 1 client.26321 shutdown 2024-03-19T01:13:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb381215740 1 -- 172.21.15.69:0/2175148570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 msgr2=0x559fe780b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb381215740 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe780b9d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb368009230 tx=0x7fb368099bf0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb381215740 1 -- 172.21.15.69:0/2175148570 shutdown_connections 2024-03-19T01:13:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb381215740 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559fe781eb10 0x559fe7820f00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb381215740 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe780c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb381215740 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 0x559fe780bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb381215740 1 --2- 172.21.15.69:0/2175148570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe780b9d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb361600640 0 client.26321 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:13:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb381215740 1 -- 172.21.15.69:0/2175148570 >> 172.21.15.69:0/2175148570 conn(0x559fe77ffb30 msgr2=0x559fe7801880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb381215740 1 -- 172.21.15.69:0/2175148570 shutdown_connections 2024-03-19T01:13:16.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb381215740 1 -- 172.21.15.69:0/2175148570 wait complete. 2024-03-19T01:13:16.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.364+0000 7fb381215740 20 client.26321 trim_cache size 0 max 16384 2024-03-19T01:13:16.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.379+0000 7fb381215740 1 Processor -- start 2024-03-19T01:13:16.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.379+0000 7fb381215740 1 -- start start 2024-03-19T01:13:16.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.379+0000 7fb381215740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe781a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb381215740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 0x559fe780c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb381215740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe7826670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb381215740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781ac9f0 con 0x559fe780e900 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb381215740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781ae5f0 con 0x559fe7805400 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb381215740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781ad0c0 con 0x559fe7805d20 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb377400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe781a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb377e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe7826670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb377400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe781a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52698/0 (socket says 172.21.15.69:52698) 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb377400640 1 -- 172.21.15.69:0/1600427741 learned_addr learned my addr 172.21.15.69:0/1600427741 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb376a00640 1 --2- 172.21.15.69:0/1600427741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 0x559fe780c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb377400640 1 -- 172.21.15.69:0/1600427741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 msgr2=0x559fe780c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb377400640 1 --2- 172.21.15.69:0/1600427741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 0x559fe780c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb377400640 1 -- 172.21.15.69:0/1600427741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 msgr2=0x559fe7826670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb377400640 1 --2- 172.21.15.69:0/1600427741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe7826670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb377400640 1 -- 172.21.15.69:0/1600427741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe7809f20 con 0x559fe7805400 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.380+0000 7fb376a00640 1 --2- 172.21.15.69:0/1600427741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 0x559fe780c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.381+0000 7fb377400640 1 --2- 172.21.15.69:0/1600427741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe781a490 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb3680cc880 tx=0x7fb3680ba0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e3eef4ff2abd9e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.381+0000 7fb363e00640 1 -- 172.21.15.69:0/1600427741 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36809b220 con 0x559fe7805400 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.381+0000 7fb363e00640 1 -- 172.21.15.69:0/1600427741 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36809bde0 con 0x559fe7805400 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.381+0000 7fb363e00640 1 -- 172.21.15.69:0/1600427741 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368097980 con 0x559fe7805400 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.381+0000 7fb381215740 1 -- 172.21.15.69:0/1600427741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 msgr2=0x559fe781a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.381+0000 7fb381215740 1 --2- 172.21.15.69:0/1600427741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe781a490 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb3680cc880 tx=0x7fb3680ba0f0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.382+0000 7fb381215740 1 -- 172.21.15.69:0/1600427741 shutdown_connections 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.382+0000 7fb381215740 1 --2- 172.21.15.69:0/1600427741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe780e900 0x559fe7826670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.382+0000 7fb381215740 1 --2- 172.21.15.69:0/1600427741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe7805d20 0x559fe780c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.382+0000 7fb381215740 1 --2- 172.21.15.69:0/1600427741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe781a490 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.382+0000 7fb381215740 1 -- 172.21.15.69:0/1600427741 >> 172.21.15.69:0/1600427741 conn(0x559fe77ffb30 msgr2=0x559fe77f5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.382+0000 7fb381215740 1 -- 172.21.15.69:0/1600427741 shutdown_connections 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.382+0000 7fb381215740 1 -- 172.21.15.69:0/1600427741 wait complete. 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.382+0000 7fb381215740 1 -- start start 2024-03-19T01:13:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.383+0000 7fb381215740 1 Processor -- start 2024-03-19T01:13:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.383+0000 7fb381215740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe7816e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.383+0000 7fb381215740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe7805d20 0x559fe78121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.383+0000 7fb381215740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe780e900 0x559fe78127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.383+0000 7fb381215740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781ade60 con 0x559fe7805d20 2024-03-19T01:13:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.383+0000 7fb381215740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781ad790 con 0x559fe7805400 2024-03-19T01:13:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.383+0000 7fb381215740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781aa9f0 con 0x559fe780e900 2024-03-19T01:13:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.383+0000 7fb381215740 10 client.0 _ll_register_callbacks cb 0x7fb368005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.383+0000 7fb377400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe7816e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.383+0000 7fb377400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe7816e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52704/0 (socket says 172.21.15.69:52704) 2024-03-19T01:13:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.383+0000 7fb377400640 1 -- 172.21.15.69:0/1400303146 learned_addr learned my addr 172.21.15.69:0/1400303146 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.384+0000 7fb376a00640 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe7805d20 0x559fe78121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.384+0000 7fb377e00640 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe780e900 0x559fe78127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.384+0000 7fb377400640 1 -- 172.21.15.69:0/1400303146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe780e900 msgr2=0x559fe78127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.384+0000 7fb377400640 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe780e900 0x559fe78127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.384+0000 7fb377400640 1 -- 172.21.15.69:0/1400303146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe7805d20 msgr2=0x559fe78121d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.384+0000 7fb377400640 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe7805d20 0x559fe78121d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.384+0000 7fb377400640 1 -- 172.21.15.69:0/1400303146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe7812fe0 con 0x559fe7805400 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.384+0000 7fb376a00640 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe7805d20 0x559fe78121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.384+0000 7fb377400640 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe7816e60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb36800d010 tx=0x7fb368097f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c97b22a2405447fc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.384+0000 7fb361600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb361600640 1 -- 172.21.15.69:0/1400303146 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36809b040 con 0x559fe7805400 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb361600640 1 -- 172.21.15.69:0/1400303146 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fb3680a39f0 con 0x559fe7805400 2024-03-19T01:13:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb381215740 1 -- 172.21.15.69:0/1400303146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb36809bde0 con 0x559fe7805400 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb361600640 1 -- 172.21.15.69:0/1400303146 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb368097980 con 0x559fe7805400 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb381215740 10 client.26327 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb381215740 1 -- 172.21.15.69:0/1400303146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559fe7812fe0 con 0x559fe7805400 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb361600640 1 -- 172.21.15.69:0/1400303146 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36807e020 con 0x559fe7805400 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb361600640 1 -- 172.21.15.69:0/1400303146 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb36809b940 con 0x559fe7805400 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb381215740 10 client.26327 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb381215740 10 client.26327 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb381215740 20 client.26327 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb381215740 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559fe7884fa0 0x559fe7887390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb381215740 4 client.26327 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.385+0000 7fb381215740 1 -- 172.21.15.69:0/1400303146 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fe76e6560 con 0x559fe7884fa0 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.386+0000 7fb376a00640 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559fe7884fa0 0x559fe7887390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb376a00640 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559fe7884fa0 0x559fe7887390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=969d71039a59db8c server_cookie=260e07c3ae07f358 in_seq=0 out_seq=0 2024-03-19T01:13:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb361600640 10 client.26327 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb361600640 1 -- 172.21.15.69:0/1400303146 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fe76e6560 con 0x559fe7884fa0 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb361600640 10 client.26327 handle_command_reply: tid=0 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb381215740 1 client.26327 shutdown 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb381215740 1 -- 172.21.15.69:0/1400303146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 msgr2=0x559fe7816e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb381215740 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe7816e60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb36800d010 tx=0x7fb368097f80 comp rx=0 tx=0).stop 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb381215740 1 -- 172.21.15.69:0/1400303146 shutdown_connections 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb381215740 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559fe7884fa0 0x559fe7887390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb381215740 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559fe780e900 0x559fe78127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb381215740 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559fe7805d20 0x559fe78121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb361600640 0 client.26327 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb381215740 1 --2- 172.21.15.69:0/1400303146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe7805400 0x559fe7816e60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb381215740 1 -- 172.21.15.69:0/1400303146 >> 172.21.15.69:0/1400303146 conn(0x559fe77ffb30 msgr2=0x559fe77f6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb381215740 1 -- 172.21.15.69:0/1400303146 shutdown_connections 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.387+0000 7fb381215740 1 -- 172.21.15.69:0/1400303146 wait complete. 2024-03-19T01:13:16.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb381215740 20 client.26327 trim_cache size 0 max 16384 2024-03-19T01:13:16.393 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:13:16.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb37d000640 1 -- 172.21.15.69:0/3755059705 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb35c0a29f0 msgr2=0x7fb35c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb37d000640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb35c0a29f0 0x7fb35c0a4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb370005e30 tx=0x7fb370005d80 comp rx=0 tx=0).stop 2024-03-19T01:13:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb37d000640 1 -- 172.21.15.69:0/3755059705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb378101690 msgr2=0x7fb37819a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb37d000640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb378101690 0x7fb37819a130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb36800ea60 tx=0x7fb36800ef20 comp rx=0 tx=0).stop 2024-03-19T01:13:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb37d000640 1 -- 172.21.15.69:0/3755059705 shutdown_connections 2024-03-19T01:13:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb37d000640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3781076a0 0x7fb37819e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb37d000640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb35c0a29f0 0x7fb35c0a4eb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb37d000640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb378102050 0x7fb37819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb37d000640 1 --2- 172.21.15.69:0/3755059705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb378101690 0x7fb37819a130 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb37d000640 1 -- 172.21.15.69:0/3755059705 >> 172.21.15.69:0/3755059705 conn(0x7fb3780fd1c0 msgr2=0x7fb37810e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.388+0000 7fb37d000640 1 -- 172.21.15.69:0/3755059705 shutdown_connections 2024-03-19T01:13:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.389+0000 7fb37d000640 1 -- 172.21.15.69:0/3755059705 wait complete. 2024-03-19T01:13:16.411 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:16 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/123999163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:13:16.414 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:13:16.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:16 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/123999163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:13:16.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.615+0000 7f8d7f000640 1 -- 172.21.15.69:0/996193825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d78100630 msgr2=0x7f8d78100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.615+0000 7f8d7f000640 1 --2- 172.21.15.69:0/996193825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d78100630 0x7f8d78100a10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009900 tx=0x7f8d6802bc40 comp rx=0 tx=0).stop 2024-03-19T01:13:16.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.615+0000 7f8d7f000640 1 -- 172.21.15.69:0/996193825 shutdown_connections 2024-03-19T01:13:16.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.615+0000 7f8d7f000640 1 --2- 172.21.15.69:0/996193825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d7810e1e0 0x7f8d781105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.615+0000 7f8d7f000640 1 --2- 172.21.15.69:0/996193825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d78100f50 0x7f8d7810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.615+0000 7f8d7f000640 1 --2- 172.21.15.69:0/996193825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d78100630 0x7f8d78100a10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.615+0000 7f8d7f000640 1 -- 172.21.15.69:0/996193825 >> 172.21.15.69:0/996193825 conn(0x7f8d78077880 msgr2=0x7f8d78077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.615+0000 7f8d7f000640 1 -- 172.21.15.69:0/996193825 shutdown_connections 2024-03-19T01:13:16.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.615+0000 7f8d7f000640 1 -- 172.21.15.69:0/996193825 wait complete. 2024-03-19T01:13:16.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.616+0000 7f8d7f000640 1 Processor -- start 2024-03-19T01:13:16.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.616+0000 7f8d7f000640 1 -- start start 2024-03-19T01:13:16.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.616+0000 7f8d7f000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d78100630 0x7f8d7819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d7f000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d78100f50 0x7f8d7819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d7f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d7810e1e0 0x7f8d7819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d7f000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78113e40 con 0x7f8d78100f50 2024-03-19T01:13:16.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d7f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78113cc0 con 0x7f8d7810e1e0 2024-03-19T01:13:16.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d7f000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78113fc0 con 0x7f8d78100630 2024-03-19T01:13:16.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d77400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d78100f50 0x7f8d7819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d78100630 0x7f8d7819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d7d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d7810e1e0 0x7f8d7819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d78100630 0x7f8d7819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45910/0 (socket says 172.21.15.69:45910) 2024-03-19T01:13:16.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d77e00640 1 -- 172.21.15.69:0/3953542274 learned_addr learned my addr 172.21.15.69:0/3953542274 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d7d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d7810e1e0 0x7f8d7819ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52722/0 (socket says 172.21.15.69:52722) 2024-03-19T01:13:16.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d77e00640 1 -- 172.21.15.69:0/3953542274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d7810e1e0 msgr2=0x7f8d7819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d77e00640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d7810e1e0 0x7f8d7819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d77e00640 1 -- 172.21.15.69:0/3953542274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d78100f50 msgr2=0x7f8d7819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d77e00640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d78100f50 0x7f8d7819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d77e00640 1 -- 172.21.15.69:0/3953542274 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7819f2f0 con 0x7f8d78100630 2024-03-19T01:13:16.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.617+0000 7f8d7d000640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d7810e1e0 0x7f8d7819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:16.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.618+0000 7f8d77e00640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d78100630 0x7f8d7819a300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8d68005ab0 tx=0x7f8d680042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.618+0000 7f8d74c00640 1 -- 172.21.15.69:0/3953542274 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68041070 con 0x7f8d78100630 2024-03-19T01:13:16.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.618+0000 7f8d74c00640 1 -- 172.21.15.69:0/3953542274 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6802cca0 con 0x7f8d78100630 2024-03-19T01:13:16.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.618+0000 7f8d7f000640 1 -- 172.21.15.69:0/3953542274 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7819f580 con 0x7f8d78100630 2024-03-19T01:13:16.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.618+0000 7f8d74c00640 1 -- 172.21.15.69:0/3953542274 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68003d90 con 0x7f8d78100630 2024-03-19T01:13:16.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.618+0000 7f8d7f000640 1 -- 172.21.15.69:0/3953542274 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d781a6e70 con 0x7f8d78100630 2024-03-19T01:13:16.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.621+0000 7f8d74c00640 1 -- 172.21.15.69:0/3953542274 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f8d68044070 con 0x7f8d78100630 2024-03-19T01:13:16.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.621+0000 7f8d81c53740 1 Processor -- start 2024-03-19T01:13:16.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.621+0000 7f8d81c53740 1 -- start start 2024-03-19T01:13:16.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.622+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x563353336e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.622+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353337340 0x56335333f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.622+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56335333ff20 0x563353342310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.622+0000 7f8d81c53740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56335329b380 con 0x563353337340 2024-03-19T01:13:16.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.622+0000 7f8d81c53740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563353243d20 con 0x56335333ff20 2024-03-19T01:13:16.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.622+0000 7f8d81c53740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563353297420 con 0x563353336a20 2024-03-19T01:13:16.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.622+0000 7f8d7d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x563353336e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.622+0000 7f8d7d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x563353336e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45926/0 (socket says 172.21.15.69:45926) 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.622+0000 7f8d7d000640 1 -- 172.21.15.69:0/3185407894 learned_addr learned my addr 172.21.15.69:0/3185407894 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.622+0000 7f8d77400640 1 --2- 172.21.15.69:0/3185407894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56335333ff20 0x563353342310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.622+0000 7f8d77e00640 1 --2- 172.21.15.69:0/3185407894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353337340 0x56335333f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d7d000640 1 -- 172.21.15.69:0/3185407894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56335333ff20 msgr2=0x563353342310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d7d000640 1 --2- 172.21.15.69:0/3185407894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56335333ff20 0x563353342310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d7d000640 1 -- 172.21.15.69:0/3185407894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353337340 msgr2=0x56335333f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d7d000640 1 --2- 172.21.15.69:0/3185407894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353337340 0x56335333f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d7d000640 1 -- 172.21.15.69:0/3185407894 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563352faf3f0 con 0x563353336a20 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d77e00640 1 --2- 172.21.15.69:0/3185407894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353337340 0x56335333f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d7d000640 1 --2- 172.21.15.69:0/3185407894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x563353336e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8d70011790 tx=0x7f8d70011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34166a8f8702f4b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d6d600640 1 -- 172.21.15.69:0/3185407894 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70003c30 con 0x563353336a20 2024-03-19T01:13:16.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d74c00640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8d5c0a2970 0x7f8d5c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d6d600640 1 -- 172.21.15.69:0/3185407894 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7000fe80 con 0x563353336a20 2024-03-19T01:13:16.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d6d600640 1 -- 172.21.15.69:0/3185407894 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d700193e0 con 0x563353336a20 2024-03-19T01:13:16.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.623+0000 7f8d74c00640 1 -- 172.21.15.69:0/3953542274 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f8d680f0ad0 con 0x7f8d78100630 2024-03-19T01:13:16.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.624+0000 7f8d77e00640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8d5c0a2970 0x7f8d5c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.624+0000 7f8d81c53740 1 -- 172.21.15.69:0/3185407894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 msgr2=0x563353336e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.624+0000 7f8d81c53740 1 --2- 172.21.15.69:0/3185407894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x563353336e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8d70011790 tx=0x7f8d70011c50 comp rx=0 tx=0).stop 2024-03-19T01:13:16.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.624+0000 7f8d81c53740 1 -- 172.21.15.69:0/3185407894 shutdown_connections 2024-03-19T01:13:16.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.624+0000 7f8d81c53740 1 --2- 172.21.15.69:0/3185407894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56335333ff20 0x563353342310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.624+0000 7f8d81c53740 1 --2- 172.21.15.69:0/3185407894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353337340 0x56335333f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.624+0000 7f8d81c53740 1 --2- 172.21.15.69:0/3185407894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x563353336e00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.624+0000 7f8d81c53740 1 -- 172.21.15.69:0/3185407894 >> 172.21.15.69:0/3185407894 conn(0x563353331150 msgr2=0x563353331530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.624+0000 7f8d77e00640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8d5c0a2970 0x7f8d5c0a4e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5633533364f0 tx=0x7f8d68009260 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.624+0000 7f8d81c53740 1 -- 172.21.15.69:0/3185407894 shutdown_connections 2024-03-19T01:13:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.624+0000 7f8d81c53740 1 -- 172.21.15.69:0/3185407894 wait complete. 2024-03-19T01:13:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.625+0000 7f8d81c53740 1 -- start start 2024-03-19T01:13:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.625+0000 7f8d81c53740 1 Processor -- start 2024-03-19T01:13:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.626+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x56335333cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.626+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353337340 0x56335333d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.626+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56335333ff20 0x56335333da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.626+0000 7f8d81c53740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563353342d10 con 0x56335333ff20 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.626+0000 7f8d81c53740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563353257160 con 0x563353337340 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.626+0000 7f8d81c53740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563353342e90 con 0x563353336a20 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.626+0000 7f8d81c53740 10 client.0 _ll_register_callbacks cb 0x563353293170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.626+0000 7f8d7d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x56335333cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.626+0000 7f8d7d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x56335333cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45930/0 (socket says 172.21.15.69:45930) 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.626+0000 7f8d7d000640 1 -- 172.21.15.69:0/555241870 learned_addr learned my addr 172.21.15.69:0/555241870 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.626+0000 7f8d7d000640 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56335333ff20 0x56335333da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.627+0000 7f8d7d000640 1 -- 172.21.15.69:0/555241870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353337340 msgr2=0x56335333d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.627+0000 7f8d7d000640 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353337340 0x56335333d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.627+0000 7f8d7d000640 1 -- 172.21.15.69:0/555241870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56335333ff20 msgr2=0x56335333da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.627+0000 7f8d7d000640 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56335333ff20 0x56335333da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.627+0000 7f8d7d000640 1 -- 172.21.15.69:0/555241870 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633530797d0 con 0x563353336a20 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.627+0000 7f8d7d000640 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x56335333cfb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8d70004660 tx=0x7f8d700044c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10cff433463c4c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.627+0000 7f8d62000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.627+0000 7f8d62000640 1 -- 172.21.15.69:0/555241870 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70003d00 con 0x563353336a20 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.628+0000 7f8d62000640 1 -- 172.21.15.69:0/555241870 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f8d7001f070 con 0x563353336a20 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.628+0000 7f8d81c53740 1 -- 172.21.15.69:0/555241870 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5633532a88c0 con 0x563353336a20 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.628+0000 7f8d62000640 1 -- 172.21.15.69:0/555241870 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8d70004a40 con 0x563353336a20 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.628+0000 7f8d81c53740 10 client.26323 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.628+0000 7f8d81c53740 1 -- 172.21.15.69:0/555241870 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56335334f040 con 0x563353336a20 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.628+0000 7f8d62000640 1 -- 172.21.15.69:0/555241870 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5633532a88c0 con 0x563353336a20 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.628+0000 7f8d62000640 1 -- 172.21.15.69:0/555241870 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f8d70018070 con 0x563353336a20 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.628+0000 7f8d81c53740 10 client.26323 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.628+0000 7f8d81c53740 10 client.26323 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.628+0000 7f8d81c53740 20 client.26323 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:16.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.629+0000 7f8d81c53740 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x563353350500 0x5633533528f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.629+0000 7f8d81c53740 4 client.26323 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:16.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.629+0000 7f8d81c53740 1 -- 172.21.15.69:0/555241870 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633532a2c10 con 0x563353350500 2024-03-19T01:13:16.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.629+0000 7f8d77400640 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x563353350500 0x5633533528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.629+0000 7f8d77400640 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x563353350500 0x5633533528f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68b7664c1e5eeea server_cookie=b65b4fad702d98e5 in_seq=0 out_seq=0 2024-03-19T01:13:16.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.629+0000 7f8d62000640 10 client.26323 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:13:16.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.630+0000 7f8d62000640 1 -- 172.21.15.69:0/555241870 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5633532a2c10 con 0x563353350500 2024-03-19T01:13:16.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.630+0000 7f8d62000640 10 client.26323 handle_command_reply: tid=0 2024-03-19T01:13:16.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.630+0000 7f8d81c53740 1 client.26323 shutdown 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d81c53740 1 -- 172.21.15.69:0/555241870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 msgr2=0x56335333cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d81c53740 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x56335333cfb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8d70004660 tx=0x7f8d700044c0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d81c53740 1 -- 172.21.15.69:0/555241870 shutdown_connections 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d81c53740 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56335333ff20 0x56335333da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d81c53740 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x563353350500 0x5633533528f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d81c53740 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353337340 0x56335333d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d81c53740 1 --2- 172.21.15.69:0/555241870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353336a20 0x56335333cfb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d81c53740 1 -- 172.21.15.69:0/555241870 >> 172.21.15.69:0/555241870 conn(0x563353331150 msgr2=0x563353332ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d62000640 0 client.26323 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d81c53740 1 -- 172.21.15.69:0/555241870 shutdown_connections 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d81c53740 1 -- 172.21.15.69:0/555241870 wait complete. 2024-03-19T01:13:16.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.631+0000 7f8d81c53740 20 client.26323 trim_cache size 0 max 16384 2024-03-19T01:13:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.645+0000 7f8d81c53740 1 Processor -- start 2024-03-19T01:13:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.645+0000 7f8d81c53740 1 -- start start 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.645+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353336a20 0x56335333f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353337340 0x56335334bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56335333ff20 0x5633533580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d81c53740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781b0f60 con 0x563353336a20 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d81c53740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781b2b60 con 0x563353337340 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d81c53740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781b1630 con 0x56335333ff20 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d77400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353337340 0x56335334bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d77400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353337340 0x56335334bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52746/0 (socket says 172.21.15.69:52746) 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d77400640 1 -- 172.21.15.69:0/1216146001 learned_addr learned my addr 172.21.15.69:0/1216146001 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d7d000640 1 --2- 172.21.15.69:0/1216146001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353336a20 0x56335333f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d7d000640 1 --2- 172.21.15.69:0/1216146001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56335333ff20 0x5633533580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d77400640 1 -- 172.21.15.69:0/1216146001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56335333ff20 msgr2=0x5633533580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d77400640 1 --2- 172.21.15.69:0/1216146001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56335333ff20 0x5633533580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d77400640 1 -- 172.21.15.69:0/1216146001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353336a20 msgr2=0x56335333f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d77400640 1 --2- 172.21.15.69:0/1216146001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353336a20 0x56335333f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d77400640 1 -- 172.21.15.69:0/1216146001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563353342310 con 0x563353337340 2024-03-19T01:13:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d7d000640 1 --2- 172.21.15.69:0/1216146001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353336a20 0x56335333f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.646+0000 7f8d77400640 1 --2- 172.21.15.69:0/1216146001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353337340 0x56335334bae0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x56335333c810 tx=0x7f8d640044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33935c85d0f208f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.647+0000 7f8d6cc00640 1 -- 172.21.15.69:0/1216146001 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64021070 con 0x563353337340 2024-03-19T01:13:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.647+0000 7f8d6cc00640 1 -- 172.21.15.69:0/1216146001 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d64003dc0 con 0x563353337340 2024-03-19T01:13:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.647+0000 7f8d6cc00640 1 -- 172.21.15.69:0/1216146001 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64010960 con 0x563353337340 2024-03-19T01:13:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.647+0000 7f8d81c53740 1 -- 172.21.15.69:0/1216146001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353337340 msgr2=0x56335334bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.647+0000 7f8d81c53740 1 --2- 172.21.15.69:0/1216146001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353337340 0x56335334bae0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x56335333c810 tx=0x7f8d640044d0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.647+0000 7f8d81c53740 1 -- 172.21.15.69:0/1216146001 shutdown_connections 2024-03-19T01:13:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.647+0000 7f8d81c53740 1 --2- 172.21.15.69:0/1216146001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56335333ff20 0x5633533580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.647+0000 7f8d81c53740 1 --2- 172.21.15.69:0/1216146001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353337340 0x56335334bae0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.647+0000 7f8d81c53740 1 --2- 172.21.15.69:0/1216146001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563353336a20 0x56335333f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.647+0000 7f8d81c53740 1 -- 172.21.15.69:0/1216146001 >> 172.21.15.69:0/1216146001 conn(0x563353331150 msgr2=0x56335332f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.648+0000 7f8d81c53740 1 -- 172.21.15.69:0/1216146001 shutdown_connections 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.648+0000 7f8d81c53740 1 -- 172.21.15.69:0/1216146001 wait complete. 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.648+0000 7f8d81c53740 1 -- start start 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.648+0000 7f8d81c53740 1 Processor -- start 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353336a20 0x563353343c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353337340 0x563353344140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d81c53740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56335333ff20 0x563353344680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d81c53740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781b23d0 con 0x56335333ff20 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d81c53740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781b1d00 con 0x563353336a20 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d81c53740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781aef40 con 0x563353337340 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d81c53740 10 client.0 _ll_register_callbacks cb 0x7f8d640097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d77400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353337340 0x563353344140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d7d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353336a20 0x563353343c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d77400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353337340 0x563353344140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45938/0 (socket says 172.21.15.69:45938) 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d77400640 1 -- 172.21.15.69:0/1102209998 learned_addr learned my addr 172.21.15.69:0/1102209998 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.649+0000 7f8d7d000640 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56335333ff20 0x563353344680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.650+0000 7f8d77400640 1 -- 172.21.15.69:0/1102209998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353336a20 msgr2=0x563353343c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.650+0000 7f8d77400640 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353336a20 0x563353343c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.650+0000 7f8d77400640 1 -- 172.21.15.69:0/1102209998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56335333ff20 msgr2=0x563353344680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.650+0000 7f8d77400640 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56335333ff20 0x563353344680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.650+0000 7f8d77400640 1 -- 172.21.15.69:0/1102209998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633533b5910 con 0x563353337340 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.650+0000 7f8d77400640 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353337340 0x563353344140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009cd0 tx=0x7f8d64009df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88760726d7c91c25 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.650+0000 7f8d62000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.650+0000 7f8d62000640 1 -- 172.21.15.69:0/1102209998 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6402d060 con 0x563353337340 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.650+0000 7f8d81c53740 1 -- 172.21.15.69:0/1102209998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d64003dc0 con 0x563353337340 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.650+0000 7f8d62000640 1 -- 172.21.15.69:0/1102209998 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f8d64003bc0 con 0x563353337340 2024-03-19T01:13:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d62000640 1 -- 172.21.15.69:0/1102209998 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8d64034050 con 0x563353337340 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d81c53740 10 client.26329 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d81c53740 1 -- 172.21.15.69:0/1102209998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5633533b6bf0 con 0x563353337340 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d62000640 1 -- 172.21.15.69:0/1102209998 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d64003dc0 con 0x563353337340 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d62000640 1 -- 172.21.15.69:0/1102209998 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f8d64027070 con 0x563353337340 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d81c53740 10 client.26329 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d81c53740 10 client.26329 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d81c53740 20 client.26329 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d81c53740 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5633533b71c0 0x5633533b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d81c53740 4 client.26329 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d81c53740 1 -- 172.21.15.69:0/1102209998 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563353220560 con 0x5633533b71c0 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.651+0000 7f8d7d000640 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5633533b71c0 0x5633533b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d7d000640 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5633533b71c0 0x5633533b95e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c27221b29c44cfb server_cookie=fe9e77ffb13a8750 in_seq=0 out_seq=0 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d62000640 10 client.26329 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:13:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d62000640 1 -- 172.21.15.69:0/1102209998 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563353220560 con 0x5633533b71c0 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d62000640 10 client.26329 handle_command_reply: tid=0 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d81c53740 1 client.26329 shutdown 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d81c53740 1 -- 172.21.15.69:0/1102209998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353337340 msgr2=0x563353344140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d81c53740 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353337340 0x563353344140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009cd0 tx=0x7f8d64009df0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d81c53740 1 -- 172.21.15.69:0/1102209998 shutdown_connections 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d81c53740 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56335333ff20 0x563353344680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d81c53740 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563353337340 0x563353344140 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d81c53740 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5633533b71c0 0x5633533b95e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d81c53740 1 --2- 172.21.15.69:0/1102209998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563353336a20 0x563353343c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d62000640 0 client.26329 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.652+0000 7f8d81c53740 1 -- 172.21.15.69:0/1102209998 >> 172.21.15.69:0/1102209998 conn(0x563353331150 msgr2=0x56335333d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d81c53740 1 -- 172.21.15.69:0/1102209998 shutdown_connections 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d81c53740 1 -- 172.21.15.69:0/1102209998 wait complete. 2024-03-19T01:13:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d81c53740 20 client.26329 trim_cache size 0 max 16384 2024-03-19T01:13:16.659 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:13:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d7f000640 1 -- 172.21.15.69:0/3953542274 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8d5c0a2970 msgr2=0x7f8d5c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d7f000640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8d5c0a2970 0x7f8d5c0a4e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5633533364f0 tx=0x7f8d68009260 comp rx=0 tx=0).stop 2024-03-19T01:13:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d7f000640 1 -- 172.21.15.69:0/3953542274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d78100630 msgr2=0x7f8d7819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d7f000640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d78100630 0x7f8d7819a300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8d68005ab0 tx=0x7f8d680042f0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d7f000640 1 -- 172.21.15.69:0/3953542274 shutdown_connections 2024-03-19T01:13:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d7f000640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8d5c0a2970 0x7f8d5c0a4e30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d7f000640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d7810e1e0 0x7f8d7819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d7f000640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d78100f50 0x7f8d7819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d7f000640 1 --2- 172.21.15.69:0/3953542274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d78100630 0x7f8d7819a300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.653+0000 7f8d7f000640 1 -- 172.21.15.69:0/3953542274 >> 172.21.15.69:0/3953542274 conn(0x7f8d78077880 msgr2=0x7f8d7810f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.654+0000 7f8d7f000640 1 -- 172.21.15.69:0/3953542274 shutdown_connections 2024-03-19T01:13:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.654+0000 7f8d7f000640 1 -- 172.21.15.69:0/3953542274 wait complete. 2024-03-19T01:13:16.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:13:16.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:16 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/123999163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:13:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.881+0000 7fcfd5a00640 1 -- 172.21.15.69:0/1702530305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd0101680 msgr2=0x7fcfd0101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.881+0000 7fcfd5a00640 1 --2- 172.21.15.69:0/1702530305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd0101680 0x7fcfd0101b00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009860 tx=0x7fcfb802bb70 comp rx=0 tx=0).stop 2024-03-19T01:13:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.881+0000 7fcfd5a00640 1 -- 172.21.15.69:0/1702530305 shutdown_connections 2024-03-19T01:13:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.881+0000 7fcfd5a00640 1 --2- 172.21.15.69:0/1702530305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfd0102040 0x7fcfd010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.881+0000 7fcfd5a00640 1 --2- 172.21.15.69:0/1702530305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd0101680 0x7fcfd0101b00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.881+0000 7fcfd5a00640 1 --2- 172.21.15.69:0/1702530305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfd0107690 0x7fcfd0107a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.881+0000 7fcfd5a00640 1 -- 172.21.15.69:0/1702530305 >> 172.21.15.69:0/1702530305 conn(0x7fcfd00fd190 msgr2=0x7fcfd00ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.881+0000 7fcfd5a00640 1 -- 172.21.15.69:0/1702530305 shutdown_connections 2024-03-19T01:13:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.881+0000 7fcfd5a00640 1 -- 172.21.15.69:0/1702530305 wait complete. 2024-03-19T01:13:16.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.882+0000 7fcfd5a00640 1 Processor -- start 2024-03-19T01:13:16.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.882+0000 7fcfd5a00640 1 -- start start 2024-03-19T01:13:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.882+0000 7fcfd5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd0101680 0x7fcfd019a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.882+0000 7fcfd5a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfd0102040 0x7fcfd019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.882+0000 7fcfd5a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfd0107690 0x7fcfd019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.882+0000 7fcfd5a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd0111a60 con 0x7fcfd0102040 2024-03-19T01:13:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.882+0000 7fcfd5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01118e0 con 0x7fcfd0101680 2024-03-19T01:13:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.882+0000 7fcfd5a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd0111be0 con 0x7fcfd0107690 2024-03-19T01:13:16.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd0101680 0x7fcfd019a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd0101680 0x7fcfd019a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52774/0 (socket says 172.21.15.69:52774) 2024-03-19T01:13:16.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfcfe00640 1 -- 172.21.15.69:0/2887666822 learned_addr learned my addr 172.21.15.69:0/2887666822 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfc7400640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfd0102040 0x7fcfd019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfd5000640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfd0107690 0x7fcfd019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfcfe00640 1 -- 172.21.15.69:0/2887666822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfd0107690 msgr2=0x7fcfd019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfcfe00640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfd0107690 0x7fcfd019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfcfe00640 1 -- 172.21.15.69:0/2887666822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfd0102040 msgr2=0x7fcfd019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfcfe00640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfd0102040 0x7fcfd019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfcfe00640 1 -- 172.21.15.69:0/2887666822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd019f130 con 0x7fcfd0101680 2024-03-19T01:13:16.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfc7400640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfd0102040 0x7fcfd019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:16.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfd5000640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfd0107690 0x7fcfd019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:16.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.883+0000 7fcfcfe00640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd0101680 0x7fcfd019a140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c420 tx=0x7fcfc0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.884+0000 7fcfcd600640 1 -- 172.21.15.69:0/2887666822 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0003970 con 0x7fcfd0101680 2024-03-19T01:13:16.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.884+0000 7fcfcd600640 1 -- 172.21.15.69:0/2887666822 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc0004020 con 0x7fcfd0101680 2024-03-19T01:13:16.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.884+0000 7fcfd5a00640 1 -- 172.21.15.69:0/2887666822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd019f420 con 0x7fcfd0101680 2024-03-19T01:13:16.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.884+0000 7fcfcd600640 1 -- 172.21.15.69:0/2887666822 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000bad0 con 0x7fcfd0101680 2024-03-19T01:13:16.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.884+0000 7fcfd5a00640 1 -- 172.21.15.69:0/2887666822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd01a6cb0 con 0x7fcfd0101680 2024-03-19T01:13:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.888+0000 7fcfd9dfe740 1 Processor -- start 2024-03-19T01:13:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.888+0000 7fcfd9dfe740 1 -- start start 2024-03-19T01:13:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.888+0000 7fcfcd600640 1 -- 172.21.15.69:0/2887666822 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fcfc001d020 con 0x7fcfd0101680 2024-03-19T01:13:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.889+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf0fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.889+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf10530 0x55df0cf18bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.889+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf19110 0x55df0cf1b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.889+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df0ce74380 con 0x55df0cf19110 2024-03-19T01:13:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.889+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df0ce1cd20 con 0x55df0cf10530 2024-03-19T01:13:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.889+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55df0ce70420 con 0x55df0cf0fc10 2024-03-19T01:13:16.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.889+0000 7fcfd5000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf0fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.889+0000 7fcfd5000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf0fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45974/0 (socket says 172.21.15.69:45974) 2024-03-19T01:13:16.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.889+0000 7fcfcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf10530 0x55df0cf18bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.889+0000 7fcfd5000640 1 -- 172.21.15.69:0/3690903277 learned_addr learned my addr 172.21.15.69:0/3690903277 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.889+0000 7fcfc7400640 1 --2- 172.21.15.69:0/3690903277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf19110 0x55df0cf1b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.890+0000 7fcfd5000640 1 -- 172.21.15.69:0/3690903277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf10530 msgr2=0x55df0cf18bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.890+0000 7fcfd5000640 1 --2- 172.21.15.69:0/3690903277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf10530 0x55df0cf18bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.890+0000 7fcfd5000640 1 -- 172.21.15.69:0/3690903277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf19110 msgr2=0x55df0cf1b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.890+0000 7fcfd5000640 1 --2- 172.21.15.69:0/3690903277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf19110 0x55df0cf1b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.890+0000 7fcfd5000640 1 -- 172.21.15.69:0/3690903277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0cb883f0 con 0x55df0cf0fc10 2024-03-19T01:13:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.890+0000 7fcfc7400640 1 --2- 172.21.15.69:0/3690903277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf19110 0x55df0cf1b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:13:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.890+0000 7fcfd5000640 1 --2- 172.21.15.69:0/3690903277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf0fff0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800bcf0 tx=0x7fcfc800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d4810aa375d2502 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.890+0000 7fcfc5600640 1 -- 172.21.15.69:0/3690903277 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8015070 con 0x55df0cf0fc10 2024-03-19T01:13:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.890+0000 7fcfcd600640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcfb40a2970 0x7fcfb40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.890+0000 7fcfc5600640 1 -- 172.21.15.69:0/3690903277 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc8011920 con 0x55df0cf0fc10 2024-03-19T01:13:16.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.890+0000 7fcfc5600640 1 -- 172.21.15.69:0/3690903277 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8010910 con 0x55df0cf0fc10 2024-03-19T01:13:16.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.891+0000 7fcfcd600640 1 -- 172.21.15.69:0/2887666822 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fcfc00cf190 con 0x7fcfd0101680 2024-03-19T01:13:16.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.891+0000 7fcfcfe00640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcfb40a2970 0x7fcfb40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.891+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3690903277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 msgr2=0x55df0cf0fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.891+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3690903277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf0fff0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800bcf0 tx=0x7fcfc800f730 comp rx=0 tx=0).stop 2024-03-19T01:13:16.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.891+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3690903277 shutdown_connections 2024-03-19T01:13:16.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.891+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3690903277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf19110 0x55df0cf1b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.891+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3690903277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf10530 0x55df0cf18bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.891+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3690903277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf0fff0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.891+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3690903277 >> 172.21.15.69:0/3690903277 conn(0x55df0cf0a340 msgr2=0x55df0cf0a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.892+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3690903277 shutdown_connections 2024-03-19T01:13:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.892+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3690903277 wait complete. 2024-03-19T01:13:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.892+0000 7fcfcfe00640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcfb40a2970 0x7fcfb40a4e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00cc5d0 tx=0x7fcfc0097040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.892+0000 7fcfd9dfe740 1 -- start start 2024-03-19T01:13:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.893+0000 7fcfd9dfe740 1 Processor -- start 2024-03-19T01:13:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.893+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 0x55df0cf16160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.893+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf166a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.893+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf19110 0x55df0cf16be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.893+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df0cf1bf00 con 0x55df0cf10530 2024-03-19T01:13:16.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.893+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df0ce30160 con 0x55df0cf0fc10 2024-03-19T01:13:16.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.893+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55df0cf1c080 con 0x55df0cf19110 2024-03-19T01:13:16.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.893+0000 7fcfd9dfe740 10 client.0 _ll_register_callbacks cb 0x55df0ce6c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:16.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.894+0000 7fcfd5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 0x55df0cf16160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.894+0000 7fcfc7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.894+0000 7fcfd5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 0x55df0cf16160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52792/0 (socket says 172.21.15.69:52792) 2024-03-19T01:13:16.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.894+0000 7fcfc7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf166a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36908/0 (socket says 172.21.15.69:36908) 2024-03-19T01:13:16.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.894+0000 7fcfc7400640 1 -- 172.21.15.69:0/1114655074 learned_addr learned my addr 172.21.15.69:0/1114655074 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.894+0000 7fcfd5000640 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf19110 0x55df0cf16be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.894+0000 7fcfc7400640 1 -- 172.21.15.69:0/1114655074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf19110 msgr2=0x55df0cf16be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.894+0000 7fcfc7400640 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf19110 0x55df0cf16be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.894+0000 7fcfc7400640 1 -- 172.21.15.69:0/1114655074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 msgr2=0x55df0cf16160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.894+0000 7fcfc7400640 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 0x55df0cf16160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.894+0000 7fcfc7400640 1 -- 172.21.15.69:0/1114655074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0cc527d0 con 0x55df0cf10530 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.895+0000 7fcfc7400640 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf166a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009860 tx=0x7fcfb8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a13babac24acff server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.895+0000 7fcfbe000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.895+0000 7fcfbe000640 1 -- 172.21.15.69:0/1114655074 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8039040 con 0x55df0cf10530 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.895+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/1114655074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc8011920 con 0x55df0cf10530 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.895+0000 7fcfbe000640 1 -- 172.21.15.69:0/1114655074 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fcfb8004430 con 0x55df0cf10530 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.895+0000 7fcfbe000640 1 -- 172.21.15.69:0/1114655074 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcfb804d050 con 0x55df0cf10530 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.895+0000 7fcfd9dfe740 10 client.16164 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.895+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/1114655074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55df0cf28180 con 0x55df0cf10530 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.896+0000 7fcfbe000640 1 -- 172.21.15.69:0/1114655074 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc8011920 con 0x55df0cf10530 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.896+0000 7fcfbe000640 1 -- 172.21.15.69:0/1114655074 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fcfb802c510 con 0x55df0cf10530 2024-03-19T01:13:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.896+0000 7fcfd9dfe740 10 client.16164 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.896+0000 7fcfd9dfe740 10 client.16164 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.896+0000 7fcfd9dfe740 20 client.16164 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.896+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55df0cf29590 0x55df0cf2b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.896+0000 7fcfd9dfe740 4 client.16164 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.896+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/1114655074 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df0ce818c0 con 0x55df0cf29590 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.897+0000 7fcfd5000640 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55df0cf29590 0x55df0cf2b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.897+0000 7fcfd5000640 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55df0cf29590 0x55df0cf2b9d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84af2f8bfffab61d server_cookie=9b34bf27db13244f in_seq=0 out_seq=0 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.897+0000 7fcfbe000640 10 client.16164 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfbe000640 1 -- 172.21.15.69:0/1114655074 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55df0ce818c0 con 0x55df0cf29590 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfbe000640 10 client.16164 handle_command_reply: tid=0 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfd9dfe740 1 client.16164 shutdown 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/1114655074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 msgr2=0x55df0cf166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf166a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009860 tx=0x7fcfb8004040 comp rx=0 tx=0).stop 2024-03-19T01:13:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/1114655074 shutdown_connections 2024-03-19T01:13:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55df0cf29590 0x55df0cf2b9d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf19110 0x55df0cf16be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf166a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/1114655074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 0x55df0cf16160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/1114655074 >> 172.21.15.69:0/1114655074 conn(0x55df0cf0a340 msgr2=0x55df0cf0c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfbe000640 0 client.16164 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:13:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.898+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/1114655074 shutdown_connections 2024-03-19T01:13:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.899+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/1114655074 wait complete. 2024-03-19T01:13:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.899+0000 7fcfd9dfe740 20 client.16164 trim_cache size 0 max 16384 2024-03-19T01:13:16.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.919+0000 7fcfd9dfe740 1 Processor -- start 2024-03-19T01:13:16.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.919+0000 7fcfd9dfe740 1 -- start start 2024-03-19T01:13:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf18890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf24ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf19110 0x55df0cf312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01b0cf0 con 0x55df0cf10530 2024-03-19T01:13:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01b28f0 con 0x55df0cf19110 2024-03-19T01:13:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01b13c0 con 0x55df0cf0fc10 2024-03-19T01:13:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd5000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf18890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfc7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf24ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd5000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf18890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45992/0 (socket says 172.21.15.69:45992) 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd5000640 1 -- 172.21.15.69:0/3897442229 learned_addr learned my addr 172.21.15.69:0/3897442229 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd5000640 1 --2- 172.21.15.69:0/3897442229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf19110 0x55df0cf312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd5000640 1 -- 172.21.15.69:0/3897442229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf19110 msgr2=0x55df0cf312a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.920+0000 7fcfd5000640 1 --2- 172.21.15.69:0/3897442229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf19110 0x55df0cf312a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.921+0000 7fcfd5000640 1 -- 172.21.15.69:0/3897442229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 msgr2=0x55df0cf24ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.921+0000 7fcfd5000640 1 --2- 172.21.15.69:0/3897442229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf24ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.921+0000 7fcfd5000640 1 -- 172.21.15.69:0/3897442229 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0cf047f0 con 0x55df0cf0fc10 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.921+0000 7fcfc7400640 1 --2- 172.21.15.69:0/3897442229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf24ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.921+0000 7fcfd5000640 1 --2- 172.21.15.69:0/3897442229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf18890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8010580 tx=0x7fcfc8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ed148ebc108c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.921+0000 7fcfc4c00640 1 -- 172.21.15.69:0/3897442229 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8015040 con 0x55df0cf0fc10 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.921+0000 7fcfc4c00640 1 -- 172.21.15.69:0/3897442229 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc8010620 con 0x55df0cf0fc10 2024-03-19T01:13:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.921+0000 7fcfc4c00640 1 -- 172.21.15.69:0/3897442229 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8011820 con 0x55df0cf0fc10 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.922+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3897442229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 msgr2=0x55df0cf18890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.922+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3897442229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf18890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8010580 tx=0x7fcfc8000bc0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.922+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3897442229 shutdown_connections 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.922+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3897442229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf19110 0x55df0cf312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.922+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3897442229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf24ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.922+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3897442229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf0fc10 0x55df0cf18890 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.922+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3897442229 >> 172.21.15.69:0/3897442229 conn(0x55df0cf0a340 msgr2=0x55df0cf11490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.922+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3897442229 shutdown_connections 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.922+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3897442229 wait complete. 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.922+0000 7fcfd9dfe740 1 -- start start 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.923+0000 7fcfd9dfe740 1 Processor -- start 2024-03-19T01:13:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.923+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 0x55df0cf216c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.923+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf12b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.923+0000 7fcfd9dfe740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf19110 0x55df0cf13290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.923+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01b2160 con 0x55df0cf10530 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.923+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01b1a90 con 0x55df0cf0fc10 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.923+0000 7fcfd9dfe740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01aecd0 con 0x55df0cf19110 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.923+0000 7fcfd9dfe740 10 client.0 _ll_register_callbacks cb 0x7fcfb8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfd5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 0x55df0cf216c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfc7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf12b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfc7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf12b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36934/0 (socket says 172.21.15.69:36934) 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfc7400640 1 -- 172.21.15.69:0/3439407598 learned_addr learned my addr 172.21.15.69:0/3439407598 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfd5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 0x55df0cf216c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52820/0 (socket says 172.21.15.69:52820) 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfd5000640 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf19110 0x55df0cf13290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfc7400640 1 -- 172.21.15.69:0/3439407598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf19110 msgr2=0x55df0cf13290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfc7400640 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf19110 0x55df0cf13290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfc7400640 1 -- 172.21.15.69:0/3439407598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 msgr2=0x55df0cf216c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfc7400640 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 0x55df0cf216c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfc7400640 1 -- 172.21.15.69:0/3439407598 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0cf8e8d0 con 0x55df0cf10530 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfc7400640 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf12b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcfb802c960 tx=0x7fcfb8005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8228211428c0c29 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.924+0000 7fcfbe000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.925+0000 7fcfbe000640 1 -- 172.21.15.69:0/3439407598 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8055070 con 0x55df0cf10530 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.925+0000 7fcfbe000640 1 -- 172.21.15.69:0/3439407598 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fcfb80042b0 con 0x55df0cf10530 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.925+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3439407598 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc8010620 con 0x55df0cf10530 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.925+0000 7fcfbe000640 1 -- 172.21.15.69:0/3439407598 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcfb805d050 con 0x55df0cf10530 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.925+0000 7fcfd9dfe740 10 client.16170 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.925+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3439407598 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55df0cf8fa00 con 0x55df0cf10530 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.925+0000 7fcfbe000640 1 -- 172.21.15.69:0/3439407598 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc8010620 con 0x55df0cf10530 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.925+0000 7fcfbe000640 1 -- 172.21.15.69:0/3439407598 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fcfb8058030 con 0x55df0cf10530 2024-03-19T01:13:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.926+0000 7fcfd9dfe740 10 client.16170 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.926+0000 7fcfd9dfe740 10 client.16170 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.926+0000 7fcfd9dfe740 20 client.16170 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.926+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55df0cf90180 0x55df0cf925a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.926+0000 7fcfd9dfe740 4 client.16170 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.926+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3439407598 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df0ce7bc10 con 0x55df0cf90180 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.926+0000 7fcfd5000640 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55df0cf90180 0x55df0cf925a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.926+0000 7fcfd5000640 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55df0cf90180 0x55df0cf925a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1cfd145ee294dfd9 server_cookie=ed860c3941047940 in_seq=0 out_seq=0 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.926+0000 7fcfbe000640 10 client.16170 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfbe000640 1 -- 172.21.15.69:0/3439407598 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df0ce7bc10 con 0x55df0cf90180 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfbe000640 10 client.16170 handle_command_reply: tid=0 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 1 client.16170 shutdown 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3439407598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 msgr2=0x55df0cf12b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf12b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcfb802c960 tx=0x7fcfb8005a90 comp rx=0 tx=0).stop 2024-03-19T01:13:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3439407598 shutdown_connections 2024-03-19T01:13:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55df0cf90180 0x55df0cf925a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0cf19110 0x55df0cf13290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0cf10530 0x55df0cf12b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 1 --2- 172.21.15.69:0/3439407598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0cf0fc10 0x55df0cf216c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfbe000640 0 client.16170 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:13:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3439407598 >> 172.21.15.69:0/3439407598 conn(0x55df0cf0a340 msgr2=0x55df0cf16100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3439407598 shutdown_connections 2024-03-19T01:13:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 1 -- 172.21.15.69:0/3439407598 wait complete. 2024-03-19T01:13:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.927+0000 7fcfd9dfe740 20 client.16170 trim_cache size 0 max 16384 2024-03-19T01:13:16.932 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:13:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 -- 172.21.15.69:0/2887666822 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcfb40a2970 msgr2=0x7fcfb40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcfb40a2970 0x7fcfb40a4e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00cc5d0 tx=0x7fcfc0097040 comp rx=0 tx=0).stop 2024-03-19T01:13:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 -- 172.21.15.69:0/2887666822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd0101680 msgr2=0x7fcfd019a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd0101680 0x7fcfd019a140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c420 tx=0x7fcfc0009f90 comp rx=0 tx=0).stop 2024-03-19T01:13:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 -- 172.21.15.69:0/2887666822 shutdown_connections 2024-03-19T01:13:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfd0107690 0x7fcfd019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfd0102040 0x7fcfd019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcfb40a2970 0x7fcfb40a4e30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 --2- 172.21.15.69:0/2887666822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd0101680 0x7fcfd019a140 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 -- 172.21.15.69:0/2887666822 >> 172.21.15.69:0/2887666822 conn(0x7fcfd00fd190 msgr2=0x7fcfd00fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 -- 172.21.15.69:0/2887666822 shutdown_connections 2024-03-19T01:13:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:16.928+0000 7fcfd5a00640 1 -- 172.21.15.69:0/2887666822 wait complete. 2024-03-19T01:13:16.947 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:13:17.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.151+0000 7f0bed800640 1 -- 172.21.15.69:0/880516273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8102050 msgr2=0x7f0be810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.151+0000 7f0bed800640 1 --2- 172.21.15.69:0/880516273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8102050 0x7f0be810e560 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009930 tx=0x7f0be002bc30 comp rx=0 tx=0).stop 2024-03-19T01:13:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.151+0000 7f0bed800640 1 -- 172.21.15.69:0/880516273 shutdown_connections 2024-03-19T01:13:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.151+0000 7f0bed800640 1 --2- 172.21.15.69:0/880516273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8102050 0x7f0be810e560 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.151+0000 7f0bed800640 1 --2- 172.21.15.69:0/880516273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be8101690 0x7f0be8101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.151+0000 7f0bed800640 1 --2- 172.21.15.69:0/880516273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be81076a0 0x7f0be8107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.151+0000 7f0bed800640 1 -- 172.21.15.69:0/880516273 >> 172.21.15.69:0/880516273 conn(0x7f0be80fd180 msgr2=0x7f0be80ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.151+0000 7f0bed800640 1 -- 172.21.15.69:0/880516273 shutdown_connections 2024-03-19T01:13:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.151+0000 7f0bed800640 1 -- 172.21.15.69:0/880516273 wait complete. 2024-03-19T01:13:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.152+0000 7f0bed800640 1 Processor -- start 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.152+0000 7f0bed800640 1 -- start start 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0bed800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8101690 0x7f0be819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0bed800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be8102050 0x7f0be819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0bed800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be81076a0 0x7f0be819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0bed800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8111b00 con 0x7f0be8102050 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0bed800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8111980 con 0x7f0be81076a0 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0bed800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8111c80 con 0x7f0be8101690 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8101690 0x7f0be819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8101690 0x7f0be819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46026/0 (socket says 172.21.15.69:46026) 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0bece00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be81076a0 0x7f0be819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0be7e00640 1 -- 172.21.15.69:0/4061576355 learned_addr learned my addr 172.21.15.69:0/4061576355 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be8102050 0x7f0be819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0be7e00640 1 -- 172.21.15.69:0/4061576355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be81076a0 msgr2=0x7f0be819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0be7e00640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be81076a0 0x7f0be819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0be7e00640 1 -- 172.21.15.69:0/4061576355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be8102050 msgr2=0x7f0be819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.153+0000 7f0be7e00640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be8102050 0x7f0be819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.154+0000 7f0be7e00640 1 -- 172.21.15.69:0/4061576355 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be819f190 con 0x7f0be8101690 2024-03-19T01:13:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.154+0000 7f0bece00640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be81076a0 0x7f0be819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.154+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be8102050 0x7f0be819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.154+0000 7f0be7e00640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8101690 0x7f0be819a1a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800eb90 tx=0x7f0bd800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.154+0000 7f0be5600640 1 -- 172.21.15.69:0/4061576355 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8004d20 con 0x7f0be8101690 2024-03-19T01:13:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.154+0000 7f0be5600640 1 -- 172.21.15.69:0/4061576355 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd800ee90 con 0x7f0be8101690 2024-03-19T01:13:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.154+0000 7f0be5600640 1 -- 172.21.15.69:0/4061576355 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8010830 con 0x7f0be8101690 2024-03-19T01:13:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.154+0000 7f0bed800640 1 -- 172.21.15.69:0/4061576355 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be819f480 con 0x7f0be8101690 2024-03-19T01:13:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.154+0000 7f0bed800640 1 -- 172.21.15.69:0/4061576355 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be81a6d10 con 0x7f0be8101690 2024-03-19T01:13:17.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.157+0000 7f0bf1a33740 1 Processor -- start 2024-03-19T01:13:17.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.158+0000 7f0bf1a33740 1 -- start start 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.158+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c52089afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.158+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c5208a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.158+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5208a40f0 0x55c5208a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.158+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c5207ff380 con 0x55c5208a40f0 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.158+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5207a7d20 con 0x55c52089abf0 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.158+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c5207fb420 con 0x55c52089b510 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.158+0000 7f0be5600640 1 -- 172.21.15.69:0/4061576355 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f0bd8005020 con 0x7f0be8101690 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.158+0000 7f0bece00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c52089afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.159+0000 7f0bece00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c52089afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52842/0 (socket says 172.21.15.69:52842) 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.159+0000 7f0bece00640 1 -- 172.21.15.69:0/1845077509 learned_addr learned my addr 172.21.15.69:0/1845077509 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.159+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/1845077509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5208a40f0 0x55c5208a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.159+0000 7f0be7e00640 1 --2- 172.21.15.69:0/1845077509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c5208a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.159+0000 7f0bece00640 1 -- 172.21.15.69:0/1845077509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 msgr2=0x55c5208a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.159+0000 7f0bece00640 1 --2- 172.21.15.69:0/1845077509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c5208a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.159+0000 7f0bece00640 1 -- 172.21.15.69:0/1845077509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5208a40f0 msgr2=0x55c5208a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.159+0000 7f0bece00640 1 --2- 172.21.15.69:0/1845077509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5208a40f0 0x55c5208a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.159+0000 7f0bece00640 1 -- 172.21.15.69:0/1845077509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5205133f0 con 0x55c52089abf0 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.159+0000 7f0bece00640 1 --2- 172.21.15.69:0/1845077509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c52089afd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009900 tx=0x7f0be0009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d63c4f1dc9eea4ea server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.160+0000 7f0bdd600640 1 -- 172.21.15.69:0/1845077509 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0045070 con 0x55c52089abf0 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.160+0000 7f0bdd600640 1 -- 172.21.15.69:0/1845077509 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be002c5d0 con 0x55c52089abf0 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.160+0000 7f0bdd600640 1 -- 172.21.15.69:0/1845077509 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0004f80 con 0x55c52089abf0 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.160+0000 7f0be5600640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0bcc0a29f0 0x7f0bcc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.160+0000 7f0be5600640 1 -- 172.21.15.69:0/4061576355 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f0bd80cf7a0 con 0x7f0be8101690 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.160+0000 7f0be7e00640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0bcc0a29f0 0x7f0bcc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.160+0000 7f0bf1a33740 1 -- 172.21.15.69:0/1845077509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 msgr2=0x55c52089afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.160+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/1845077509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c52089afd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009900 tx=0x7f0be0009560 comp rx=0 tx=0).stop 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.161+0000 7f0bf1a33740 1 -- 172.21.15.69:0/1845077509 shutdown_connections 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.161+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/1845077509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5208a40f0 0x55c5208a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.161+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/1845077509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c5208a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.161+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/1845077509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c52089afd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.161+0000 7f0bf1a33740 1 -- 172.21.15.69:0/1845077509 >> 172.21.15.69:0/1845077509 conn(0x55c520895320 msgr2=0x55c520895700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.161+0000 7f0bf1a33740 1 -- 172.21.15.69:0/1845077509 shutdown_connections 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.161+0000 7f0bf1a33740 1 -- 172.21.15.69:0/1845077509 wait complete. 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.161+0000 7f0be7e00640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0bcc0a29f0 0x7f0bcc0a4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55c52089a6c0 tx=0x7f0bd800cf90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.161+0000 7f0bf1a33740 1 -- start start 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bf1a33740 1 Processor -- start 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c5208a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c5208a1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5208a40f0 0x55c5208a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c5208a6e80 con 0x55c5208a40f0 2024-03-19T01:13:17.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5207bb160 con 0x55c52089abf0 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c5208a7000 con 0x55c52089b510 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bf1a33740 10 client.0 _ll_register_callbacks cb 0x55c5207f7170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c5208a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c5208a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46054/0 (socket says 172.21.15.69:46054) 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bdfe00640 1 -- 172.21.15.69:0/2424771873 learned_addr learned my addr 172.21.15.69:0/2424771873 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.162+0000 7f0bece00640 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c5208a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bece00640 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5208a40f0 0x55c5208a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bdfe00640 1 -- 172.21.15.69:0/2424771873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 msgr2=0x55c5208a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c5208a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bdfe00640 1 -- 172.21.15.69:0/2424771873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5208a40f0 msgr2=0x55c5208a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5208a40f0 0x55c5208a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bdfe00640 1 -- 172.21.15.69:0/2424771873 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5205dd7d0 con 0x55c52089b510 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bece00640 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5208a40f0 0x55c5208a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:13:17.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c5208a1680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0012d30 tx=0x7f0bd000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f49df7f28b8e90a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bd6000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bd6000640 1 -- 172.21.15.69:0/2424771873 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000fe20 con 0x55c52089b510 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bd6000640 1 -- 172.21.15.69:0/2424771873 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f0bd0012110 con 0x55c52089b510 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2424771873 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be002c5d0 con 0x55c52089b510 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.163+0000 7f0bd6000640 1 -- 172.21.15.69:0/2424771873 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0bd0025050 con 0x55c52089b510 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.164+0000 7f0bf1a33740 10 client.26359 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.164+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2424771873 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c5208b31a0 con 0x55c52089b510 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.164+0000 7f0bd6000640 1 -- 172.21.15.69:0/2424771873 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be002c5d0 con 0x55c52089b510 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.164+0000 7f0bd6000640 1 -- 172.21.15.69:0/2424771873 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f0bd0010430 con 0x55c52089b510 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.164+0000 7f0bf1a33740 10 client.26359 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.164+0000 7f0bf1a33740 10 client.26359 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.164+0000 7f0bf1a33740 20 client.26359 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.164+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c5208b4700 0x55c5208b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.164+0000 7f0bf1a33740 4 client.26359 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:17.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.164+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2424771873 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c52080c8c0 con 0x55c5208b4700 2024-03-19T01:13:17.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.165+0000 7f0bece00640 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c5208b4700 0x55c5208b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.165+0000 7f0bece00640 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c5208b4700 0x55c5208b6af0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32303f8ab276f450 server_cookie=8fc4cd2243de527d in_seq=0 out_seq=0 2024-03-19T01:13:17.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.165+0000 7f0bd6000640 10 client.26359 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:13:17.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.166+0000 7f0bd6000640 1 -- 172.21.15.69:0/2424771873 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c52080c8c0 con 0x55c5208b4700 2024-03-19T01:13:17.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.166+0000 7f0bd6000640 10 client.26359 handle_command_reply: tid=0 2024-03-19T01:13:17.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.166+0000 7f0bf1a33740 1 client.26359 shutdown 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.166+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2424771873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 msgr2=0x55c5208a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.166+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c5208a1680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0012d30 tx=0x7f0bd000f790 comp rx=0 tx=0).stop 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.166+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2424771873 shutdown_connections 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.167+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5208a40f0 0x55c5208a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.167+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c5208a1680 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.167+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c5208b4700 0x55c5208b6af0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.167+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2424771873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c5208a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.167+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2424771873 >> 172.21.15.69:0/2424771873 conn(0x55c520895320 msgr2=0x55c520897070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.167+0000 7f0bd6000640 0 client.26359 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.167+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2424771873 shutdown_connections 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.167+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2424771873 wait complete. 2024-03-19T01:13:17.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.167+0000 7f0bf1a33740 20 client.26359 trim_cache size 0 max 16384 2024-03-19T01:13:17.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bf1a33740 1 Processor -- start 2024-03-19T01:13:17.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bf1a33740 1 -- start start 2024-03-19T01:13:17.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c5208a3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089b510 0x55c5208afc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5208a40f0 0x55c5208bc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0be81b0cd0 con 0x55c52089b510 2024-03-19T01:13:17.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0be81b28d0 con 0x55c52089abf0 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0be81b13a0 con 0x55c5208a40f0 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089b510 0x55c5208afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089b510 0x55c5208afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36996/0 (socket says 172.21.15.69:36996) 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bdfe00640 1 -- 172.21.15.69:0/3799209275 learned_addr learned my addr 172.21.15.69:0/3799209275 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.182+0000 7f0bece00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c5208a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.183+0000 7f0bece00640 1 --2- 172.21.15.69:0/3799209275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5208a40f0 0x55c5208bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.183+0000 7f0bdfe00640 1 -- 172.21.15.69:0/3799209275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5208a40f0 msgr2=0x55c5208bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.183+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/3799209275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5208a40f0 0x55c5208bc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.183+0000 7f0bdfe00640 1 -- 172.21.15.69:0/3799209275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 msgr2=0x55c5208a3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.183+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/3799209275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c5208a3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.183+0000 7f0bdfe00640 1 -- 172.21.15.69:0/3799209275 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5208b6e60 con 0x55c52089b510 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.183+0000 7f0bece00640 1 --2- 172.21.15.69:0/3799209275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c5208a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.183+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/3799209275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089b510 0x55c5208afc20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00107d0 tx=0x7f0bd0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e7c4bb90ad32ee server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.183+0000 7f0bdcc00640 1 -- 172.21.15.69:0/3799209275 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd002e070 con 0x55c52089b510 2024-03-19T01:13:17.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.183+0000 7f0bdcc00640 1 -- 172.21.15.69:0/3799209275 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd0005590 con 0x55c52089b510 2024-03-19T01:13:17.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.183+0000 7f0bdcc00640 1 -- 172.21.15.69:0/3799209275 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00035f0 con 0x55c52089b510 2024-03-19T01:13:17.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.184+0000 7f0bf1a33740 1 -- 172.21.15.69:0/3799209275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089b510 msgr2=0x55c5208afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.184+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/3799209275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089b510 0x55c5208afc20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00107d0 tx=0x7f0bd0005db0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.184+0000 7f0bf1a33740 1 -- 172.21.15.69:0/3799209275 shutdown_connections 2024-03-19T01:13:17.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.184+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/3799209275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5208a40f0 0x55c5208bc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.184+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/3799209275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089b510 0x55c5208afc20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.184+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/3799209275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c52089abf0 0x55c5208a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.184+0000 7f0bf1a33740 1 -- 172.21.15.69:0/3799209275 >> 172.21.15.69:0/3799209275 conn(0x55c520895320 msgr2=0x55c52089c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.184+0000 7f0bf1a33740 1 -- 172.21.15.69:0/3799209275 shutdown_connections 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.184+0000 7f0bf1a33740 1 -- 172.21.15.69:0/3799209275 wait complete. 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.184+0000 7f0bf1a33740 1 -- start start 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bf1a33740 1 Processor -- start 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089abf0 0x55c52089dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c52089e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bf1a33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5208a40f0 0x55c52089e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0be81b2140 con 0x55c52089abf0 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0be81b1a70 con 0x55c5208a40f0 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bf1a33740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0be81aecb0 con 0x55c52089b510 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bf1a33740 10 client.0 _ll_register_callbacks cb 0x7f0bd0009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bece00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089abf0 0x55c52089dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c52089e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bece00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089abf0 0x55c52089dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37012/0 (socket says 172.21.15.69:37012) 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c52089e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46066/0 (socket says 172.21.15.69:46066) 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.185+0000 7f0bece00640 1 -- 172.21.15.69:0/2744250075 learned_addr learned my addr 172.21.15.69:0/2744250075 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bece00640 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5208a40f0 0x55c52089e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bdfe00640 1 -- 172.21.15.69:0/2744250075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5208a40f0 msgr2=0x55c52089e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5208a40f0 0x55c52089e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bdfe00640 1 -- 172.21.15.69:0/2744250075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089abf0 msgr2=0x55c52089dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089abf0 0x55c52089dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bdfe00640 1 -- 172.21.15.69:0/2744250075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c520919990 con 0x55c52089b510 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bece00640 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5208a40f0 0x55c52089e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bdfe00640 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c52089e110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55c5208b6d90 tx=0x7f0bd00096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b47582cb4be8b79f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bd6000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bd6000640 1 -- 172.21.15.69:0/2744250075 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd002e040 con 0x55c52089b510 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bd6000640 1 -- 172.21.15.69:0/2744250075 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f0bd0010bc0 con 0x55c52089b510 2024-03-19T01:13:17.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.186+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2744250075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd0005590 con 0x55c52089b510 2024-03-19T01:13:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.187+0000 7f0bd6000640 1 -- 172.21.15.69:0/2744250075 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0bd0012110 con 0x55c52089b510 2024-03-19T01:13:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.187+0000 7f0bf1a33740 10 client.26365 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.187+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2744250075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c52091ab50 con 0x55c52089b510 2024-03-19T01:13:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.187+0000 7f0bd6000640 1 -- 172.21.15.69:0/2744250075 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd0005590 con 0x55c52089b510 2024-03-19T01:13:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.187+0000 7f0bd6000640 1 -- 172.21.15.69:0/2744250075 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f0bd0029430 con 0x55c52089b510 2024-03-19T01:13:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.187+0000 7f0bf1a33740 10 client.26365 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.187+0000 7f0bf1a33740 10 client.26365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:13:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.187+0000 7f0bf1a33740 20 client.26365 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.187+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c52091b200 0x55c52091d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.187+0000 7f0bf1a33740 4 client.26365 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.187+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2744250075 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c520806c10 con 0x55c52091b200 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.188+0000 7f0bece00640 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c52091b200 0x55c52091d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.188+0000 7f0bece00640 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c52091b200 0x55c52091d620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1610aded69d32770 server_cookie=aa73548fc99e9c02 in_seq=0 out_seq=0 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.188+0000 7f0bd6000640 10 client.26365 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.188+0000 7f0bd6000640 1 -- 172.21.15.69:0/2744250075 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c520806c10 con 0x55c52091b200 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.188+0000 7f0bd6000640 10 client.26365 handle_command_reply: tid=0 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.188+0000 7f0bf1a33740 1 client.26365 shutdown 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2744250075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 msgr2=0x55c52089e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c52089e110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55c5208b6d90 tx=0x7f0bd00096e0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2744250075 shutdown_connections 2024-03-19T01:13:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c52091b200 0x55c52091d620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5208a40f0 0x55c52089e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c52089b510 0x55c52089e110 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bf1a33740 1 --2- 172.21.15.69:0/2744250075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c52089abf0 0x55c52089dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bd6000640 0 client.26365 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:13:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2744250075 >> 172.21.15.69:0/2744250075 conn(0x55c520895320 msgr2=0x55c5208a10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2744250075 shutdown_connections 2024-03-19T01:13:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bf1a33740 1 -- 172.21.15.69:0/2744250075 wait complete. 2024-03-19T01:13:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.189+0000 7f0bf1a33740 20 client.26365 trim_cache size 0 max 16384 2024-03-19T01:13:17.195 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:13:17.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 -- 172.21.15.69:0/4061576355 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0bcc0a29f0 msgr2=0x7f0bcc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0bcc0a29f0 0x7f0bcc0a4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55c52089a6c0 tx=0x7f0bd800cf90 comp rx=0 tx=0).stop 2024-03-19T01:13:17.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 -- 172.21.15.69:0/4061576355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8101690 msgr2=0x7f0be819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8101690 0x7f0be819a1a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800eb90 tx=0x7f0bd800c6a0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 -- 172.21.15.69:0/4061576355 shutdown_connections 2024-03-19T01:13:17.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0bcc0a29f0 0x7f0bcc0a4eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be81076a0 0x7f0be819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0be8102050 0x7f0be819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 --2- 172.21.15.69:0/4061576355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8101690 0x7f0be819a1a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 -- 172.21.15.69:0/4061576355 >> 172.21.15.69:0/4061576355 conn(0x7f0be80fd180 msgr2=0x7f0be810cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 -- 172.21.15.69:0/4061576355 shutdown_connections 2024-03-19T01:13:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.190+0000 7f0bed800640 1 -- 172.21.15.69:0/4061576355 wait complete. 2024-03-19T01:13:17.210 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:13:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.410+0000 7f9392800640 1 -- 172.21.15.69:0/1475899195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c103170 msgr2=0x7f938c10b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.410+0000 7f9392800640 1 --2- 172.21.15.69:0/1475899195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c103170 0x7f938c10b650 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9378009900 tx=0x7f937802bcd0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.410+0000 7f9392800640 1 -- 172.21.15.69:0/1475899195 shutdown_connections 2024-03-19T01:13:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.410+0000 7f9392800640 1 --2- 172.21.15.69:0/1475899195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938c10bc70 0x7f938c10e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.410+0000 7f9392800640 1 --2- 172.21.15.69:0/1475899195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c103170 0x7f938c10b650 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.410+0000 7f9392800640 1 --2- 172.21.15.69:0/1475899195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c102850 0x7f938c102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.410+0000 7f9392800640 1 -- 172.21.15.69:0/1475899195 >> 172.21.15.69:0/1475899195 conn(0x7f938c0fc070 msgr2=0x7f938c0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.410+0000 7f9392800640 1 -- 172.21.15.69:0/1475899195 shutdown_connections 2024-03-19T01:13:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.410+0000 7f9392800640 1 -- 172.21.15.69:0/1475899195 wait complete. 2024-03-19T01:13:17.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.411+0000 7f9392800640 1 Processor -- start 2024-03-19T01:13:17.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9392800640 1 -- start start 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9392800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938c102850 0x7f938c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9392800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c103170 0x7f938c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9392800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c10bc70 0x7f938c19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9392800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f938c111ac0 con 0x7f938c102850 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9392800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f938c111940 con 0x7f938c103170 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9392800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f938c111c40 con 0x7f938c10bc70 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9391400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938c102850 0x7f938c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9390a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c103170 0x7f938c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9391e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c10bc70 0x7f938c19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9390a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c103170 0x7f938c19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52882/0 (socket says 172.21.15.69:52882) 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.412+0000 7f9390a00640 1 -- 172.21.15.69:0/926178228 learned_addr learned my addr 172.21.15.69:0/926178228 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.413+0000 7f9390a00640 1 -- 172.21.15.69:0/926178228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c10bc70 msgr2=0x7f938c19ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.413+0000 7f9390a00640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c10bc70 0x7f938c19ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.413+0000 7f9390a00640 1 -- 172.21.15.69:0/926178228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938c102850 msgr2=0x7f938c19a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.413+0000 7f9390a00640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938c102850 0x7f938c19a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.413+0000 7f9390a00640 1 -- 172.21.15.69:0/926178228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938c19f160 con 0x7f938c103170 2024-03-19T01:13:17.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.413+0000 7f9390a00640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c103170 0x7f938c19a6b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9378000c00 tx=0x7f9378004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.413+0000 7f938a000640 1 -- 172.21.15.69:0/926178228 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378042020 con 0x7f938c103170 2024-03-19T01:13:17.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.413+0000 7f938a000640 1 -- 172.21.15.69:0/926178228 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93780054e0 con 0x7f938c103170 2024-03-19T01:13:17.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.413+0000 7f9392800640 1 -- 172.21.15.69:0/926178228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938c19f3f0 con 0x7f938c103170 2024-03-19T01:13:17.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.413+0000 7f938a000640 1 -- 172.21.15.69:0/926178228 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378003af0 con 0x7f938c103170 2024-03-19T01:13:17.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.414+0000 7f9392800640 1 -- 172.21.15.69:0/926178228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f938c1a6ce0 con 0x7f938c103170 2024-03-19T01:13:17.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.417+0000 7f9396b8e740 1 Processor -- start 2024-03-19T01:13:17.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.417+0000 7f9396b8e740 1 -- start start 2024-03-19T01:13:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x560741637fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560741638510 0x560741640bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5607416410f0 0x5607416434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9396b8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56074159c380 con 0x560741638510 2024-03-19T01:13:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9396b8e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560741544d20 con 0x560741637bf0 2024-03-19T01:13:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9396b8e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560741598420 con 0x5607416410f0 2024-03-19T01:13:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9391e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x560741637fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9391400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560741638510 0x560741640bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9391e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x560741637fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52894/0 (socket says 172.21.15.69:52894) 2024-03-19T01:13:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9391e00640 1 -- 172.21.15.69:0/125383918 learned_addr learned my addr 172.21.15.69:0/125383918 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:17.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f938a000640 1 -- 172.21.15.69:0/926178228 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9378045020 con 0x7f938c103170 2024-03-19T01:13:17.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9391e00640 1 -- 172.21.15.69:0/125383918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5607416410f0 msgr2=0x5607416434e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:13:17.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9391e00640 1 --2- 172.21.15.69:0/125383918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5607416410f0 0x5607416434e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9391e00640 1 -- 172.21.15.69:0/125383918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560741638510 msgr2=0x560741640bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9391e00640 1 --2- 172.21.15.69:0/125383918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560741638510 0x560741640bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.418+0000 7f9391e00640 1 -- 172.21.15.69:0/125383918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607412b03f0 con 0x560741637bf0 2024-03-19T01:13:17.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.419+0000 7f9391e00640 1 --2- 172.21.15.69:0/125383918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x560741637fd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9384009f70 tx=0x7f9384010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a31fd0ae6f0852c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.419+0000 7f937e600640 1 -- 172.21.15.69:0/125383918 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384016070 con 0x560741637bf0 2024-03-19T01:13:17.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.419+0000 7f937e600640 1 -- 172.21.15.69:0/125383918 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9384004d20 con 0x560741637bf0 2024-03-19T01:13:17.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.419+0000 7f937e600640 1 -- 172.21.15.69:0/125383918 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384012950 con 0x560741637bf0 2024-03-19T01:13:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.420+0000 7f9396b8e740 1 -- 172.21.15.69:0/125383918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 msgr2=0x560741637fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.420+0000 7f9396b8e740 1 --2- 172.21.15.69:0/125383918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x560741637fd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9384009f70 tx=0x7f9384010800 comp rx=0 tx=0).stop 2024-03-19T01:13:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.420+0000 7f9396b8e740 1 -- 172.21.15.69:0/125383918 shutdown_connections 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.420+0000 7f9396b8e740 1 --2- 172.21.15.69:0/125383918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5607416410f0 0x5607416434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.420+0000 7f9396b8e740 1 --2- 172.21.15.69:0/125383918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560741638510 0x560741640bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.420+0000 7f9396b8e740 1 --2- 172.21.15.69:0/125383918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x560741637fd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.420+0000 7f9396b8e740 1 -- 172.21.15.69:0/125383918 >> 172.21.15.69:0/125383918 conn(0x560741632320 msgr2=0x560741632700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.420+0000 7f9396b8e740 1 -- 172.21.15.69:0/125383918 shutdown_connections 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.420+0000 7f9396b8e740 1 -- 172.21.15.69:0/125383918 wait complete. 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.420+0000 7f9396b8e740 1 -- start start 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9396b8e740 1 Processor -- start 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f938a000640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2900 0x7f93740a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f938a000640 1 -- 172.21.15.69:0/926178228 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f93780f0c50 con 0x7f938c103170 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9391e00640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2900 0x7f93740a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741637bf0 0x56074163e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741638510 0x56074163e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 0x56074163ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9396b8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560741643ee0 con 0x5607416410f0 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9396b8e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560741558160 con 0x560741638510 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9396b8e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560741644060 con 0x560741637bf0 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9396b8e740 10 client.0 _ll_register_callbacks cb 0x560741594170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9391400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741637bf0 0x56074163e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9391400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741637bf0 0x56074163e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46102/0 (socket says 172.21.15.69:46102) 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.421+0000 7f9391400640 1 -- 172.21.15.69:0/530976031 learned_addr learned my addr 172.21.15.69:0/530976031 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.422+0000 7f9390a00640 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741638510 0x56074163e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.422+0000 7f9391400640 1 -- 172.21.15.69:0/530976031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741638510 msgr2=0x56074163e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.422+0000 7f9391400640 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741638510 0x56074163e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.422+0000 7f9391e00640 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 0x56074163ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.422+0000 7f9391400640 1 -- 172.21.15.69:0/530976031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 msgr2=0x56074163ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.422+0000 7f9391400640 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 0x56074163ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.422+0000 7f9391400640 1 -- 172.21.15.69:0/530976031 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56074137a7d0 con 0x560741637bf0 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.422+0000 7f9391e00640 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 0x56074163ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.422+0000 7f9391e00640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2900 0x7f93740a4dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9384005ab0 tx=0x7f93840124f0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.422+0000 7f9391400640 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741637bf0 0x56074163e160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9380012710 tx=0x7f9380012bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac97a9d4ffc1d4cc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.422+0000 7f9372a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.423+0000 7f9372a00640 1 -- 172.21.15.69:0/530976031 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938000f920 con 0x560741637bf0 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.423+0000 7f9396b8e740 1 -- 172.21.15.69:0/530976031 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9384004d20 con 0x560741637bf0 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.423+0000 7f9372a00640 1 -- 172.21.15.69:0/530976031 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f938000fac0 con 0x560741637bf0 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.423+0000 7f9372a00640 1 -- 172.21.15.69:0/530976031 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f938000fcc0 con 0x560741637bf0 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.423+0000 7f9396b8e740 10 client.26374 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.423+0000 7f9396b8e740 1 -- 172.21.15.69:0/530976031 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560741650290 con 0x560741637bf0 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.423+0000 7f9372a00640 1 -- 172.21.15.69:0/530976031 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9384004d20 con 0x560741637bf0 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.423+0000 7f9372a00640 1 -- 172.21.15.69:0/530976031 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f938001c420 con 0x560741637bf0 2024-03-19T01:13:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.424+0000 7f9396b8e740 10 client.26374 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:17.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.424+0000 7f9396b8e740 10 client.26374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:17.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.424+0000 7f9396b8e740 20 client.26374 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:17.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.424+0000 7f9396b8e740 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560741651760 0x560741653b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.424+0000 7f9396b8e740 4 client.26374 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:17.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.424+0000 7f9396b8e740 1 -- 172.21.15.69:0/530976031 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607415a98c0 con 0x560741651760 2024-03-19T01:13:17.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.424+0000 7f9390a00640 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560741651760 0x560741653b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.424+0000 7f9372a00640 10 client.26374 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:17.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.424+0000 7f9390a00640 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560741651760 0x560741653b70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49d5343c715d6152 server_cookie=2e6515c3235dd38 in_seq=0 out_seq=0 2024-03-19T01:13:17.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.425+0000 7f9372a00640 1 -- 172.21.15.69:0/530976031 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5607415a98c0 con 0x560741651760 2024-03-19T01:13:17.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.425+0000 7f9372a00640 10 client.26374 handle_command_reply: tid=0 2024-03-19T01:13:17.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.425+0000 7f9396b8e740 1 client.26374 shutdown 2024-03-19T01:13:17.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9396b8e740 1 -- 172.21.15.69:0/530976031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741637bf0 msgr2=0x56074163e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9396b8e740 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741637bf0 0x56074163e160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9380012710 tx=0x7f9380012bd0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9396b8e740 1 -- 172.21.15.69:0/530976031 shutdown_connections 2024-03-19T01:13:17.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9396b8e740 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560741651760 0x560741653b70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9396b8e740 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 0x56074163ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9396b8e740 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741638510 0x56074163e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9396b8e740 1 --2- 172.21.15.69:0/530976031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741637bf0 0x56074163e160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9396b8e740 1 -- 172.21.15.69:0/530976031 >> 172.21.15.69:0/530976031 conn(0x560741632320 msgr2=0x560741634070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9372a00640 0 client.26374 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:17.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9396b8e740 1 -- 172.21.15.69:0/530976031 shutdown_connections 2024-03-19T01:13:17.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9396b8e740 1 -- 172.21.15.69:0/530976031 wait complete. 2024-03-19T01:13:17.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.426+0000 7f9396b8e740 20 client.26374 trim_cache size 0 max 16384 2024-03-19T01:13:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.443+0000 7f9396b8e740 1 Processor -- start 2024-03-19T01:13:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.443+0000 7f9396b8e740 1 -- start start 2024-03-19T01:13:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x56074163d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560741638510 0x56074164ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5607416410f0 0x560741633e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9396b8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1b0d70 con 0x560741638510 2024-03-19T01:13:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9396b8e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1b2970 con 0x560741637bf0 2024-03-19T01:13:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9396b8e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1b1440 con 0x5607416410f0 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9391400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x56074163d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9391400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x56074163d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52904/0 (socket says 172.21.15.69:52904) 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9391400640 1 -- 172.21.15.69:0/485309037 learned_addr learned my addr 172.21.15.69:0/485309037 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9391e00640 1 --2- 172.21.15.69:0/485309037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5607416410f0 0x560741633e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9390a00640 1 --2- 172.21.15.69:0/485309037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560741638510 0x56074164ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9391400640 1 -- 172.21.15.69:0/485309037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5607416410f0 msgr2=0x560741633e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9391400640 1 --2- 172.21.15.69:0/485309037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5607416410f0 0x560741633e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9391400640 1 -- 172.21.15.69:0/485309037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560741638510 msgr2=0x56074164ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9391400640 1 --2- 172.21.15.69:0/485309037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560741638510 0x56074164ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9391400640 1 -- 172.21.15.69:0/485309037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56074163c6f0 con 0x560741637bf0 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.444+0000 7f9390a00640 1 --2- 172.21.15.69:0/485309037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560741638510 0x56074164ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.445+0000 7f9391400640 1 --2- 172.21.15.69:0/485309037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x56074163d300 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9380012420 tx=0x7f93800096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=579ba2497709e635 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.445+0000 7f937dc00640 1 -- 172.21.15.69:0/485309037 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938000f920 con 0x560741637bf0 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.445+0000 7f937dc00640 1 -- 172.21.15.69:0/485309037 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f938000fac0 con 0x560741637bf0 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.445+0000 7f937dc00640 1 -- 172.21.15.69:0/485309037 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380004640 con 0x560741637bf0 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.446+0000 7f9396b8e740 1 -- 172.21.15.69:0/485309037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 msgr2=0x56074163d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.446+0000 7f9396b8e740 1 --2- 172.21.15.69:0/485309037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x56074163d300 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9380012420 tx=0x7f93800096c0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.446+0000 7f9396b8e740 1 -- 172.21.15.69:0/485309037 shutdown_connections 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.446+0000 7f9396b8e740 1 --2- 172.21.15.69:0/485309037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5607416410f0 0x560741633e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.446+0000 7f9396b8e740 1 --2- 172.21.15.69:0/485309037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560741638510 0x56074164ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.446+0000 7f9396b8e740 1 --2- 172.21.15.69:0/485309037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x56074163d300 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.446+0000 7f9396b8e740 1 -- 172.21.15.69:0/485309037 >> 172.21.15.69:0/485309037 conn(0x560741632320 msgr2=0x560741633750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.446+0000 7f9396b8e740 1 -- 172.21.15.69:0/485309037 shutdown_connections 2024-03-19T01:13:17.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.446+0000 7f9396b8e740 1 -- 172.21.15.69:0/485309037 wait complete. 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.446+0000 7f9396b8e740 1 -- start start 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.447+0000 7f9396b8e740 1 Processor -- start 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.447+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x5607416492e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.447+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741638510 0x56074163a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.447+0000 7f9396b8e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 0x56074163a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.447+0000 7f9396b8e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1b21e0 con 0x5607416410f0 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.447+0000 7f9396b8e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1b1b10 con 0x560741637bf0 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.447+0000 7f9396b8e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1aed50 con 0x560741638510 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.447+0000 7f9396b8e740 10 client.0 _ll_register_callbacks cb 0x7f9380009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9391400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x5607416492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9391400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x5607416492e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52920/0 (socket says 172.21.15.69:52920) 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9391400640 1 -- 172.21.15.69:0/1049088062 learned_addr learned my addr 172.21.15.69:0/1049088062 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:17.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9390a00640 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741638510 0x56074163a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9391e00640 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 0x56074163a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9391400640 1 -- 172.21.15.69:0/1049088062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741638510 msgr2=0x56074163a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9391400640 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741638510 0x56074163a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9391400640 1 -- 172.21.15.69:0/1049088062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 msgr2=0x56074163a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9391400640 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 0x56074163a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9391400640 1 -- 172.21.15.69:0/1049088062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56074163bf60 con 0x560741637bf0 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9391e00640 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 0x56074163a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9391400640 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x5607416492e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9380009e00 tx=0x7f9380009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cbed28574e38a3a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.448+0000 7f9372a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.449+0000 7f9372a00640 1 -- 172.21.15.69:0/1049088062 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938000f920 con 0x560741637bf0 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.449+0000 7f9396b8e740 1 -- 172.21.15.69:0/1049088062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938000fac0 con 0x560741637bf0 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.449+0000 7f9372a00640 1 -- 172.21.15.69:0/1049088062 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f938002f070 con 0x560741637bf0 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.449+0000 7f9372a00640 1 -- 172.21.15.69:0/1049088062 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f938003c050 con 0x560741637bf0 2024-03-19T01:13:17.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.449+0000 7f9396b8e740 10 client.26396 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.449+0000 7f9396b8e740 1 -- 172.21.15.69:0/1049088062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5607416b7310 con 0x560741637bf0 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.449+0000 7f9372a00640 1 -- 172.21.15.69:0/1049088062 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f938000fac0 con 0x560741637bf0 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.450+0000 7f9372a00640 1 -- 172.21.15.69:0/1049088062 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f9380033460 con 0x560741637bf0 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.450+0000 7f9396b8e740 10 client.26396 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.450+0000 7f9396b8e740 10 client.26396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.450+0000 7f9396b8e740 20 client.26396 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.450+0000 7f9396b8e740 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5607416b7a00 0x5607416b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.450+0000 7f9396b8e740 4 client.26396 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.450+0000 7f9396b8e740 1 -- 172.21.15.69:0/1049088062 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607415a3c10 con 0x5607416b7a00 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.450+0000 7f9390a00640 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5607416b7a00 0x5607416b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.450+0000 7f9390a00640 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5607416b7a00 0x5607416b9e20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcafa16baa626014 server_cookie=35e6c054a920d8d in_seq=0 out_seq=0 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.450+0000 7f9372a00640 10 client.26396 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9372a00640 1 -- 172.21.15.69:0/1049088062 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607415a3c10 con 0x5607416b7a00 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9372a00640 10 client.26396 handle_command_reply: tid=0 2024-03-19T01:13:17.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 1 client.26396 shutdown 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 1 -- 172.21.15.69:0/1049088062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 msgr2=0x5607416492e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x5607416492e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9380009e00 tx=0x7f9380009e30 comp rx=0 tx=0).stop 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 1 -- 172.21.15.69:0/1049088062 shutdown_connections 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5607416b7a00 0x5607416b9e20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5607416410f0 0x56074163a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560741638510 0x56074163a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 1 --2- 172.21.15.69:0/1049088062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560741637bf0 0x5607416492e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9372a00640 0 client.26396 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 1 -- 172.21.15.69:0/1049088062 >> 172.21.15.69:0/1049088062 conn(0x560741632320 msgr2=0x560741633fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 1 -- 172.21.15.69:0/1049088062 shutdown_connections 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 1 -- 172.21.15.69:0/1049088062 wait complete. 2024-03-19T01:13:17.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.451+0000 7f9396b8e740 20 client.26396 trim_cache size 0 max 16384 2024-03-19T01:13:17.457 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:13:17.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 -- 172.21.15.69:0/926178228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2900 msgr2=0x7f93740a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2900 0x7f93740a4dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9384005ab0 tx=0x7f93840124f0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 -- 172.21.15.69:0/926178228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c103170 msgr2=0x7f938c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:17.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c103170 0x7f938c19a6b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9378000c00 tx=0x7f9378004360 comp rx=0 tx=0).stop 2024-03-19T01:13:17.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 -- 172.21.15.69:0/926178228 shutdown_connections 2024-03-19T01:13:17.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f938c10bc70 0x7f938c19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2900 0x7f93740a4dc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f938c103170 0x7f938c19a6b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 --2- 172.21.15.69:0/926178228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f938c102850 0x7f938c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:17.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 -- 172.21.15.69:0/926178228 >> 172.21.15.69:0/926178228 conn(0x7f938c0fc070 msgr2=0x7f938c0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:17.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 -- 172.21.15.69:0/926178228 shutdown_connections 2024-03-19T01:13:17.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:17.452+0000 7f9392800640 1 -- 172.21.15.69:0/926178228 wait complete. 2024-03-19T01:13:17.474 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:17 smithi069 ceph-mon[31727]: pgmap v81: 129 pgs: 129 active+clean; 302 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2024-03-19T01:13:17.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:17 smithi163 ceph-mon[67428]: pgmap v81: 129 pgs: 129 active+clean; 302 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2024-03-19T01:13:17.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:17 smithi086 ceph-mon[66829]: pgmap v81: 129 pgs: 129 active+clean; 302 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2024-03-19T01:13:18.478 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:13:18.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:13:18.478 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:13:18.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.753+0000 7ff34a200640 1 -- 172.21.15.69:0/4027871005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3441091c0 msgr2=0x7ff3441154d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:18.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.753+0000 7ff34a200640 1 --2- 172.21.15.69:0/4027871005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3441091c0 0x7ff3441154d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff330009900 tx=0x7ff33002ce00 comp rx=0 tx=0).stop 2024-03-19T01:13:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.753+0000 7ff34a200640 1 -- 172.21.15.69:0/4027871005 shutdown_connections 2024-03-19T01:13:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.753+0000 7ff34a200640 1 --2- 172.21.15.69:0/4027871005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3441091c0 0x7ff3441154d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.753+0000 7ff34a200640 1 --2- 172.21.15.69:0/4027871005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff344108820 0x7ff344108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.753+0000 7ff34a200640 1 --2- 172.21.15.69:0/4027871005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff34410e680 0x7ff34410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.753+0000 7ff34a200640 1 -- 172.21.15.69:0/4027871005 >> 172.21.15.69:0/4027871005 conn(0x7ff34406d170 msgr2=0x7ff34406d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.754+0000 7ff34a200640 1 -- 172.21.15.69:0/4027871005 shutdown_connections 2024-03-19T01:13:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.754+0000 7ff34a200640 1 -- 172.21.15.69:0/4027871005 wait complete. 2024-03-19T01:13:18.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.756+0000 7ff34a200640 1 Processor -- start 2024-03-19T01:13:18.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.759+0000 7ff34a200640 1 -- start start 2024-03-19T01:13:18.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.759+0000 7ff34a200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff344108820 0x7ff34410a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.759+0000 7ff34a200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3441091c0 0x7ff34410a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.759+0000 7ff34a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34410e680 0x7ff34410acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.759+0000 7ff34a200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff344118270 con 0x7ff344108820 2024-03-19T01:13:18.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.759+0000 7ff34a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3441180f0 con 0x7ff34410e680 2024-03-19T01:13:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.759+0000 7ff34a200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff3441183f0 con 0x7ff3441091c0 2024-03-19T01:13:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.760+0000 7ff343400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff344108820 0x7ff34410a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.760+0000 7ff343e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34410e680 0x7ff34410acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.760+0000 7ff343e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34410e680 0x7ff34410acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52940/0 (socket says 172.21.15.69:52940) 2024-03-19T01:13:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.760+0000 7ff343e00640 1 -- 172.21.15.69:0/2488885145 learned_addr learned my addr 172.21.15.69:0/2488885145 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.760+0000 7ff342a00640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3441091c0 0x7ff34410a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.761+0000 7ff343400640 1 -- 172.21.15.69:0/2488885145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3441091c0 msgr2=0x7ff34410a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.761+0000 7ff343400640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3441091c0 0x7ff34410a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.761+0000 7ff343400640 1 -- 172.21.15.69:0/2488885145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34410e680 msgr2=0x7ff34410acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.761+0000 7ff343400640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34410e680 0x7ff34410acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.761+0000 7ff343400640 1 -- 172.21.15.69:0/2488885145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3441aee50 con 0x7ff344108820 2024-03-19T01:13:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.761+0000 7ff342a00640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3441091c0 0x7ff34410a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.762+0000 7ff343400640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff344108820 0x7ff34410a230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00aec0 tx=0x7ff33c007220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.763+0000 7ff33be00640 1 -- 172.21.15.69:0/2488885145 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c004d20 con 0x7ff344108820 2024-03-19T01:13:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.763+0000 7ff33be00640 1 -- 172.21.15.69:0/2488885145 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff33c007e10 con 0x7ff344108820 2024-03-19T01:13:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.763+0000 7ff34a200640 1 -- 172.21.15.69:0/2488885145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3441a0700 con 0x7ff344108820 2024-03-19T01:13:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.763+0000 7ff34a200640 1 -- 172.21.15.69:0/2488885145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3441a0c50 con 0x7ff344108820 2024-03-19T01:13:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.763+0000 7ff33be00640 1 -- 172.21.15.69:0/2488885145 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0059b0 con 0x7ff344108820 2024-03-19T01:13:18.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.771+0000 7ff34cdfb740 1 Processor -- start 2024-03-19T01:13:18.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.773+0000 7ff34cdfb740 1 -- start start 2024-03-19T01:13:18.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.773+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 0x561bd54e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.773+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54e9420 0x561bd54f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.773+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54f2000 0x561bd54f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.773+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561bd544bfb0 con 0x561bd54e9420 2024-03-19T01:13:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.773+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bd53f4ab0 con 0x561bd54e8b00 2024-03-19T01:13:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.773+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561bd5448050 con 0x561bd54f2000 2024-03-19T01:13:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.774+0000 7ff342a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54f2000 0x561bd54f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.774+0000 7ff342a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54f2000 0x561bd54f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46168/0 (socket says 172.21.15.69:46168) 2024-03-19T01:13:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.774+0000 7ff342a00640 1 -- 172.21.15.69:0/596334956 learned_addr learned my addr 172.21.15.69:0/596334956 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.775+0000 7ff343400640 1 --2- 172.21.15.69:0/596334956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54e9420 0x561bd54f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.775+0000 7ff342a00640 1 -- 172.21.15.69:0/596334956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 msgr2=0x561bd54e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.775+0000 7ff343e00640 1 --2- 172.21.15.69:0/596334956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 0x561bd54e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.775+0000 7ff342a00640 1 --2- 172.21.15.69:0/596334956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 0x561bd54e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.775+0000 7ff342a00640 1 -- 172.21.15.69:0/596334956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54e9420 msgr2=0x561bd54f1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.775+0000 7ff342a00640 1 --2- 172.21.15.69:0/596334956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54e9420 0x561bd54f1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.776+0000 7ff342a00640 1 -- 172.21.15.69:0/596334956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd5412e00 con 0x561bd54f2000 2024-03-19T01:13:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.776+0000 7ff343e00640 1 --2- 172.21.15.69:0/596334956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 0x561bd54e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:13:18.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.781+0000 7ff342a00640 1 --2- 172.21.15.69:0/596334956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54f2000 0x561bd54f43f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff334009de0 tx=0x7ff33400e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a38fa1f34766de34 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:18.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.781+0000 7ff33be00640 1 -- 172.21.15.69:0/2488885145 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ff33c024020 con 0x7ff344108820 2024-03-19T01:13:18.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.782+0000 7ff32fe00640 1 -- 172.21.15.69:0/596334956 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334017070 con 0x561bd54f2000 2024-03-19T01:13:18.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.782+0000 7ff32fe00640 1 -- 172.21.15.69:0/596334956 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff334013950 con 0x561bd54f2000 2024-03-19T01:13:18.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.782+0000 7ff32fe00640 1 -- 172.21.15.69:0/596334956 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334012a70 con 0x561bd54f2000 2024-03-19T01:13:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.783+0000 7ff34cdfb740 1 -- 172.21.15.69:0/596334956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54f2000 msgr2=0x561bd54f43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.784+0000 7ff33be00640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3280a2b40 0x7ff3280a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.784+0000 7ff33be00640 1 -- 172.21.15.69:0/2488885145 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7ff33c0dce00 con 0x7ff344108820 2024-03-19T01:13:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.784+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/596334956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54f2000 0x561bd54f43f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff334009de0 tx=0x7ff33400e5a0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.785+0000 7ff34cdfb740 1 -- 172.21.15.69:0/596334956 shutdown_connections 2024-03-19T01:13:18.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.785+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/596334956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54f2000 0x561bd54f43f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.785+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/596334956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54e9420 0x561bd54f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.785+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/596334956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 0x561bd54e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.785+0000 7ff34cdfb740 1 -- 172.21.15.69:0/596334956 >> 172.21.15.69:0/596334956 conn(0x561bd54e3230 msgr2=0x561bd54e3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:18.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.786+0000 7ff34cdfb740 1 -- 172.21.15.69:0/596334956 shutdown_connections 2024-03-19T01:13:18.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.786+0000 7ff34cdfb740 1 -- 172.21.15.69:0/596334956 wait complete. 2024-03-19T01:13:18.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.787+0000 7ff34cdfb740 1 -- start start 2024-03-19T01:13:18.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.787+0000 7ff34cdfb740 1 Processor -- start 2024-03-19T01:13:18.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.788+0000 7ff343e00640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3280a2b40 0x7ff3280a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.789+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.789+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e9420 0x561bd54ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.789+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd54efb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.789+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561bd54f4df0 con 0x561bd54f2000 2024-03-19T01:13:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.789+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bd5407f00 con 0x561bd54e9420 2024-03-19T01:13:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.789+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561bd54f4f70 con 0x561bd54e8b00 2024-03-19T01:13:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.789+0000 7ff34cdfb740 10 client.0 _ll_register_callbacks cb 0x561bd54078d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.789+0000 7ff343e00640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3280a2b40 0x7ff3280a5000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff330006010 tx=0x7ff330005d50 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.790+0000 7ff343400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.790+0000 7ff343400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ef0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46172/0 (socket says 172.21.15.69:46172) 2024-03-19T01:13:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.790+0000 7ff343400640 1 -- 172.21.15.69:0/2262009641 learned_addr learned my addr 172.21.15.69:0/2262009641 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.790+0000 7ff343e00640 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd54efb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.791+0000 7ff342a00640 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e9420 0x561bd54ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.792+0000 7ff342a00640 1 -- 172.21.15.69:0/2262009641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 msgr2=0x561bd54ef0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:18.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.792+0000 7ff342a00640 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ef0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.792+0000 7ff342a00640 1 -- 172.21.15.69:0/2262009641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 msgr2=0x561bd54efb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:18.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.792+0000 7ff342a00640 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd54efb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.792+0000 7ff342a00640 1 -- 172.21.15.69:0/2262009641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd52287e0 con 0x561bd54e9420 2024-03-19T01:13:18.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.792+0000 7ff343e00640 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd54efb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:13:18.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.792+0000 7ff343400640 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ef0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:18.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.793+0000 7ff342a00640 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e9420 0x561bd54ef5f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff334000c00 tx=0x7ff334014d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbbd0caba224cd8c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:18.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.794+0000 7ff32cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:18.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.795+0000 7ff32cc00640 1 -- 172.21.15.69:0/2262009641 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334029060 con 0x561bd54e9420 2024-03-19T01:13:18.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.796+0000 7ff32cc00640 1 -- 172.21.15.69:0/2262009641 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7ff3340129c0 con 0x561bd54e9420 2024-03-19T01:13:18.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.796+0000 7ff34cdfb740 1 -- 172.21.15.69:0/2262009641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff334013950 con 0x561bd54e9420 2024-03-19T01:13:18.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.798+0000 7ff32cc00640 1 -- 172.21.15.69:0/2262009641 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff334017040 con 0x561bd54e9420 2024-03-19T01:13:18.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.799+0000 7ff34cdfb740 10 client.26405 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:18.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.799+0000 7ff34cdfb740 1 -- 172.21.15.69:0/2262009641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561bd5288f70 con 0x561bd54e9420 2024-03-19T01:13:18.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.799+0000 7ff32cc00640 1 -- 172.21.15.69:0/2262009641 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff334013950 con 0x561bd54e9420 2024-03-19T01:13:18.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.800+0000 7ff32cc00640 1 -- 172.21.15.69:0/2262009641 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ff33401c930 con 0x561bd54e9420 2024-03-19T01:13:18.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.800+0000 7ff34cdfb740 10 client.26405 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:18.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.800+0000 7ff34cdfb740 10 client.26405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:18.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.801+0000 7ff34cdfb740 20 client.26405 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:18.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.801+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561bd55022a0 0x561bd5504690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.801+0000 7ff34cdfb740 4 client.26405 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:18.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.801+0000 7ff34cdfb740 1 -- 172.21.15.69:0/2262009641 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd54594f0 con 0x561bd55022a0 2024-03-19T01:13:18.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.803+0000 7ff343400640 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561bd55022a0 0x561bd5504690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.810+0000 7ff32cc00640 10 client.26405 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:18.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.810+0000 7ff343400640 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561bd55022a0 0x561bd5504690 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b318cb257521ae3d server_cookie=9a03c1a2f12ecba4 in_seq=0 out_seq=0 2024-03-19T01:13:18.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.817+0000 7ff32cc00640 1 -- 172.21.15.69:0/2262009641 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561bd54594f0 con 0x561bd55022a0 2024-03-19T01:13:18.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.817+0000 7ff32cc00640 10 client.26405 handle_command_reply: tid=0 2024-03-19T01:13:18.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.817+0000 7ff34cdfb740 1 client.26405 shutdown 2024-03-19T01:13:18.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.819+0000 7ff34cdfb740 1 -- 172.21.15.69:0/2262009641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e9420 msgr2=0x561bd54ef5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:18.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.819+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e9420 0x561bd54ef5f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff334000c00 tx=0x7ff334014d40 comp rx=0 tx=0).stop 2024-03-19T01:13:18.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.820+0000 7ff34cdfb740 1 -- 172.21.15.69:0/2262009641 shutdown_connections 2024-03-19T01:13:18.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.820+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561bd55022a0 0x561bd5504690 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.820+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd54efb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.820+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e9420 0x561bd54ef5f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.820+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/2262009641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ef0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.820+0000 7ff32cc00640 0 client.26405 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:18.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.820+0000 7ff34cdfb740 1 -- 172.21.15.69:0/2262009641 >> 172.21.15.69:0/2262009641 conn(0x561bd54e3230 msgr2=0x561bd54e4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:18.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.820+0000 7ff34cdfb740 1 -- 172.21.15.69:0/2262009641 shutdown_connections 2024-03-19T01:13:18.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.820+0000 7ff34cdfb740 1 -- 172.21.15.69:0/2262009641 wait complete. 2024-03-19T01:13:18.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.820+0000 7ff34cdfb740 20 client.26405 trim_cache size 0 max 16384 2024-03-19T01:13:18.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.875+0000 7ff34cdfb740 1 Processor -- start 2024-03-19T01:13:18.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.876+0000 7ff34cdfb740 1 -- start start 2024-03-19T01:13:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.877+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 0x561bd54f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.877+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e9420 0x561bd54fdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.877+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd5509db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.877+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff324010060 con 0x561bd54f2000 2024-03-19T01:13:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.877+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff324011c60 con 0x561bd54e8b00 2024-03-19T01:13:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.877+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff324010730 con 0x561bd54e9420 2024-03-19T01:13:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.877+0000 7ff342a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e9420 0x561bd54fdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.877+0000 7ff343400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 0x561bd54f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.877+0000 7ff343400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 0x561bd54f17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52978/0 (socket says 172.21.15.69:52978) 2024-03-19T01:13:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.878+0000 7ff343400640 1 -- 172.21.15.69:0/3663375193 learned_addr learned my addr 172.21.15.69:0/3663375193 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.877+0000 7ff342a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e9420 0x561bd54fdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46174/0 (socket says 172.21.15.69:46174) 2024-03-19T01:13:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.878+0000 7ff343e00640 1 --2- 172.21.15.69:0/3663375193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd5509db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.878+0000 7ff342a00640 1 -- 172.21.15.69:0/3663375193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 msgr2=0x561bd54f17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.878+0000 7ff342a00640 1 --2- 172.21.15.69:0/3663375193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 0x561bd54f17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.878+0000 7ff342a00640 1 -- 172.21.15.69:0/3663375193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 msgr2=0x561bd5509db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:18.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.879+0000 7ff342a00640 1 --2- 172.21.15.69:0/3663375193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd5509db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.879+0000 7ff342a00640 1 -- 172.21.15.69:0/3663375193 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd5120170 con 0x561bd54e9420 2024-03-19T01:13:18.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.879+0000 7ff343400640 1 --2- 172.21.15.69:0/3663375193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 0x561bd54f17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:18.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.879+0000 7ff343e00640 1 --2- 172.21.15.69:0/3663375193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd5509db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:18.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.879+0000 7ff342a00640 1 --2- 172.21.15.69:0/3663375193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e9420 0x561bd54fdc20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff33400e5e0 tx=0x7ff33402e460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f3f7ac0a42bb780 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:18.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.881+0000 7ff32f400640 1 -- 172.21.15.69:0/3663375193 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334029040 con 0x561bd54e9420 2024-03-19T01:13:18.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.881+0000 7ff32f400640 1 -- 172.21.15.69:0/3663375193 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff334012ea0 con 0x561bd54e9420 2024-03-19T01:13:18.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.881+0000 7ff32f400640 1 -- 172.21.15.69:0/3663375193 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334033080 con 0x561bd54e9420 2024-03-19T01:13:18.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.883+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3663375193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e9420 msgr2=0x561bd54fdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:18.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.883+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/3663375193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e9420 0x561bd54fdc20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff33400e5e0 tx=0x7ff33402e460 comp rx=0 tx=0).stop 2024-03-19T01:13:18.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.884+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3663375193 shutdown_connections 2024-03-19T01:13:18.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.884+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/3663375193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd5509db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.884+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/3663375193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e9420 0x561bd54fdc20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.884+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/3663375193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e8b00 0x561bd54f17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.884+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3663375193 >> 172.21.15.69:0/3663375193 conn(0x561bd54e3230 msgr2=0x561bd54e22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:18.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.885+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3663375193 shutdown_connections 2024-03-19T01:13:18.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.885+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3663375193 wait complete. 2024-03-19T01:13:18.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.886+0000 7ff34cdfb740 1 -- start start 2024-03-19T01:13:18.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.886+0000 7ff34cdfb740 1 Processor -- start 2024-03-19T01:13:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.889+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.889+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e9420 0x561bd54ec7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.889+0000 7ff34cdfb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd55673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.889+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3240114d0 con 0x561bd54f2000 2024-03-19T01:13:18.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.889+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff324010e00 con 0x561bd54e9420 2024-03-19T01:13:18.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.889+0000 7ff34cdfb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff32400e040 con 0x561bd54e8b00 2024-03-19T01:13:18.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.889+0000 7ff34cdfb740 10 client.0 _ll_register_callbacks cb 0x7ff334013850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:18.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.889+0000 7ff343400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.889+0000 7ff343400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ec260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46188/0 (socket says 172.21.15.69:46188) 2024-03-19T01:13:18.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.889+0000 7ff343400640 1 -- 172.21.15.69:0/3734087043 learned_addr learned my addr 172.21.15.69:0/3734087043 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:18.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.890+0000 7ff343e00640 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd55673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.890+0000 7ff342a00640 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e9420 0x561bd54ec7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.891+0000 7ff343400640 1 -- 172.21.15.69:0/3734087043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e9420 msgr2=0x561bd54ec7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:18.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.893+0000 7ff343400640 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e9420 0x561bd54ec7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.893+0000 7ff343400640 1 -- 172.21.15.69:0/3734087043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 msgr2=0x561bd55673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:18.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.893+0000 7ff343400640 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd55673e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.893+0000 7ff343400640 1 -- 172.21.15.69:0/3734087043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd5567920 con 0x561bd54e8b00 2024-03-19T01:13:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.893+0000 7ff343e00640 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd55673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.894+0000 7ff343400640 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ec260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0da1e0 tx=0x7ff33c015e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb19602a118e173f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.894+0000 7ff32cc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.894+0000 7ff32cc00640 1 -- 172.21.15.69:0/3734087043 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0a9150 con 0x561bd54e8b00 2024-03-19T01:13:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.894+0000 7ff32cc00640 1 -- 172.21.15.69:0/3734087043 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7ff33c0da3b0 con 0x561bd54e8b00 2024-03-19T01:13:18.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.895+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3734087043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff334012ea0 con 0x561bd54e8b00 2024-03-19T01:13:18.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.895+0000 7ff32cc00640 1 -- 172.21.15.69:0/3734087043 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff33c0da5b0 con 0x561bd54e8b00 2024-03-19T01:13:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.898+0000 7ff34cdfb740 10 client.26395 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.898+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3734087043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561bd5568b00 con 0x561bd54e8b00 2024-03-19T01:13:18.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.898+0000 7ff32cc00640 1 -- 172.21.15.69:0/3734087043 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff334012ea0 con 0x561bd54e8b00 2024-03-19T01:13:18.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.899+0000 7ff32cc00640 1 -- 172.21.15.69:0/3734087043 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ff33c0b1460 con 0x561bd54e8b00 2024-03-19T01:13:18.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.899+0000 7ff34cdfb740 10 client.26395 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:18.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.899+0000 7ff34cdfb740 10 client.26395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.899+0000 7ff34cdfb740 20 client.26395 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.899+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561bd55692a0 0x561bd556b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.899+0000 7ff34cdfb740 4 client.26395 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:13:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.899+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3734087043 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561bd5453840 con 0x561bd55692a0 2024-03-19T01:13:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.901+0000 7ff342a00640 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561bd55692a0 0x561bd556b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.902+0000 7ff342a00640 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561bd55692a0 0x561bd556b6c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e595ab0173c8f091 server_cookie=8242ad27c5e39513 in_seq=0 out_seq=0 2024-03-19T01:13:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.902+0000 7ff32cc00640 10 client.26395 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:18.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.964+0000 7ff32cc00640 1 -- 172.21.15.69:0/3734087043 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561bd5453840 con 0x561bd55692a0 2024-03-19T01:13:18.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.964+0000 7ff32cc00640 10 client.26395 handle_command_reply: tid=0 2024-03-19T01:13:18.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.964+0000 7ff34cdfb740 1 client.26395 shutdown 2024-03-19T01:13:18.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.964+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3734087043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 msgr2=0x561bd54ec260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:18.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.964+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ec260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0da1e0 tx=0x7ff33c015e80 comp rx=0 tx=0).stop 2024-03-19T01:13:18.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.965+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3734087043 shutdown_connections 2024-03-19T01:13:18.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.965+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561bd55692a0 0x561bd556b6c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.965+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561bd54f2000 0x561bd55673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.965+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd54e9420 0x561bd54ec7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.965+0000 7ff34cdfb740 1 --2- 172.21.15.69:0/3734087043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561bd54e8b00 0x561bd54ec260 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.965+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3734087043 >> 172.21.15.69:0/3734087043 conn(0x561bd54e3230 msgr2=0x561bd54ef050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:18.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.965+0000 7ff32cc00640 0 client.26395 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:18.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.965+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3734087043 shutdown_connections 2024-03-19T01:13:18.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.965+0000 7ff34cdfb740 1 -- 172.21.15.69:0/3734087043 wait complete. 2024-03-19T01:13:18.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.965+0000 7ff34cdfb740 20 client.26395 trim_cache size 0 max 16384 2024-03-19T01:13:18.968 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:13:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:13:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "a8a05487-d7f5-40c6-a03c-ff8a245e048a", 2024-03-19T01:13:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:13:18.969 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:13:18.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.966+0000 7ff34a200640 1 -- 172.21.15.69:0/2488885145 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3280a2b40 msgr2=0x7ff3280a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:18.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.966+0000 7ff34a200640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3280a2b40 0x7ff3280a5000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff330006010 tx=0x7ff330005d50 comp rx=0 tx=0).stop 2024-03-19T01:13:18.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.966+0000 7ff34a200640 1 -- 172.21.15.69:0/2488885145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff344108820 msgr2=0x7ff34410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:18.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.966+0000 7ff34a200640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff344108820 0x7ff34410a230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00aec0 tx=0x7ff33c007220 comp rx=0 tx=0).stop 2024-03-19T01:13:18.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.967+0000 7ff34a200640 1 -- 172.21.15.69:0/2488885145 shutdown_connections 2024-03-19T01:13:18.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.967+0000 7ff34a200640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff3280a2b40 0x7ff3280a5000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.967+0000 7ff34a200640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34410e680 0x7ff34410acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.967+0000 7ff34a200640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3441091c0 0x7ff34410a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.967+0000 7ff34a200640 1 --2- 172.21.15.69:0/2488885145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff344108820 0x7ff34410a230 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:18.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.967+0000 7ff34a200640 1 -- 172.21.15.69:0/2488885145 >> 172.21.15.69:0/2488885145 conn(0x7ff34406d170 msgr2=0x7ff3440725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:18.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.967+0000 7ff34a200640 1 -- 172.21.15.69:0/2488885145 shutdown_connections 2024-03-19T01:13:18.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:18.967+0000 7ff34a200640 1 -- 172.21.15.69:0/2488885145 wait complete. 2024-03-19T01:13:19.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:13:19.004 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:13:19.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.322+0000 7fea45e00640 1 -- 172.21.15.69:0/1788523695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea401099d0 msgr2=0x7fea40109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:19.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.322+0000 7fea45e00640 1 --2- 172.21.15.69:0/1788523695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea401099d0 0x7fea40109db0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fea30009890 tx=0x7fea3002bb70 comp rx=0 tx=0).stop 2024-03-19T01:13:19.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.323+0000 7fea45e00640 1 -- 172.21.15.69:0/1788523695 shutdown_connections 2024-03-19T01:13:19.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.323+0000 7fea45e00640 1 --2- 172.21.15.69:0/1788523695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea40104380 0x7fea40110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.323+0000 7fea45e00640 1 --2- 172.21.15.69:0/1788523695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea401039c0 0x7fea40103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.323+0000 7fea45e00640 1 --2- 172.21.15.69:0/1788523695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea401099d0 0x7fea40109db0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.323+0000 7fea45e00640 1 -- 172.21.15.69:0/1788523695 >> 172.21.15.69:0/1788523695 conn(0x7fea400ff470 msgr2=0x7fea40101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:19.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.323+0000 7fea45e00640 1 -- 172.21.15.69:0/1788523695 shutdown_connections 2024-03-19T01:13:19.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.323+0000 7fea45e00640 1 -- 172.21.15.69:0/1788523695 wait complete. 2024-03-19T01:13:19.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.324+0000 7fea45e00640 1 Processor -- start 2024-03-19T01:13:19.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.325+0000 7fea45e00640 1 -- start start 2024-03-19T01:13:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.325+0000 7fea45e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea401039c0 0x7fea4019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.325+0000 7fea45e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea40104380 0x7fea4019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.325+0000 7fea45e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea401099d0 0x7fea4019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.325+0000 7fea45e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea40113e00 con 0x7fea401099d0 2024-03-19T01:13:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.325+0000 7fea45e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea40113c80 con 0x7fea40104380 2024-03-19T01:13:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.325+0000 7fea45e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea40113f80 con 0x7fea401039c0 2024-03-19T01:13:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.326+0000 7fea37e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea40104380 0x7fea4019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.326+0000 7fea37e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea40104380 0x7fea4019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53002/0 (socket says 172.21.15.69:53002) 2024-03-19T01:13:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.326+0000 7fea3f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea401099d0 0x7fea4019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.326+0000 7fea37e00640 1 -- 172.21.15.69:0/1126761231 learned_addr learned my addr 172.21.15.69:0/1126761231 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.326+0000 7fea3ea00640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea401039c0 0x7fea4019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.327+0000 7fea3f400640 1 -- 172.21.15.69:0/1126761231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea401039c0 msgr2=0x7fea4019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.327+0000 7fea3f400640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea401039c0 0x7fea4019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.327+0000 7fea3f400640 1 -- 172.21.15.69:0/1126761231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea40104380 msgr2=0x7fea4019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:19.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.327+0000 7fea3f400640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea40104380 0x7fea4019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.327+0000 7fea3f400640 1 -- 172.21.15.69:0/1126761231 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4019f350 con 0x7fea401099d0 2024-03-19T01:13:19.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.327+0000 7fea37e00640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea40104380 0x7fea4019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:19.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.328+0000 7fea3f400640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea401099d0 0x7fea4019ec20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea3800e940 tx=0x7fea3800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:19.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.328+0000 7fea44a00640 1 -- 172.21.15.69:0/1126761231 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38004d20 con 0x7fea401099d0 2024-03-19T01:13:19.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.328+0000 7fea44a00640 1 -- 172.21.15.69:0/1126761231 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea38005020 con 0x7fea401099d0 2024-03-19T01:13:19.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.328+0000 7fea45e00640 1 -- 172.21.15.69:0/1126761231 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4019f640 con 0x7fea401099d0 2024-03-19T01:13:19.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.329+0000 7fea45e00640 1 -- 172.21.15.69:0/1126761231 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea401a6ed0 con 0x7fea401099d0 2024-03-19T01:13:19.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.329+0000 7fea44a00640 1 -- 172.21.15.69:0/1126761231 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea380116b0 con 0x7fea401099d0 2024-03-19T01:13:19.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.336+0000 7fea48bc1740 1 Processor -- start 2024-03-19T01:13:19.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.336+0000 7fea44a00640 1 -- 172.21.15.69:0/1126761231 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fea380054d0 con 0x7fea401099d0 2024-03-19T01:13:19.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.338+0000 7fea48bc1740 1 -- start start 2024-03-19T01:13:19.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.338+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 0x564da8c08120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.338+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 0x564da8c10d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.338+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c13630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.338+0000 7fea48bc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564da8b6c2c0 con 0x564da8c11240 2024-03-19T01:13:19.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.338+0000 7fea48bc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564da8b14b60 con 0x564da8c07d40 2024-03-19T01:13:19.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.338+0000 7fea48bc1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564da8b68360 con 0x564da8c08660 2024-03-19T01:13:19.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.339+0000 7fea37e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c13630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.339+0000 7fea37e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c13630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37122/0 (socket says 172.21.15.69:37122) 2024-03-19T01:13:19.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.339+0000 7fea37e00640 1 -- 172.21.15.69:0/1037004983 learned_addr learned my addr 172.21.15.69:0/1037004983 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:19.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.339+0000 7fea37e00640 1 --2- 172.21.15.69:0/1037004983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 0x564da8c08120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.339+0000 7fea3ea00640 1 --2- 172.21.15.69:0/1037004983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 0x564da8c10d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.339+0000 7fea44a00640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea240a2b70 0x7fea240a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.340+0000 7fea44a00640 1 -- 172.21.15.69:0/1126761231 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fea380cf460 con 0x7fea401099d0 2024-03-19T01:13:19.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.340+0000 7fea3ea00640 1 -- 172.21.15.69:0/1037004983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 msgr2=0x564da8c08120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:19.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.340+0000 7fea3ea00640 1 --2- 172.21.15.69:0/1037004983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 0x564da8c08120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.340+0000 7fea3ea00640 1 -- 172.21.15.69:0/1037004983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 msgr2=0x564da8c13630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:19.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.340+0000 7fea3ea00640 1 --2- 172.21.15.69:0/1037004983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c13630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.340+0000 7fea3ea00640 1 -- 172.21.15.69:0/1037004983 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564da894a7d0 con 0x564da8c08660 2024-03-19T01:13:19.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.340+0000 7fea37e00640 1 --2- 172.21.15.69:0/1037004983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c13630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:19.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.340+0000 7fea3f400640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea240a2b70 0x7fea240a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.341+0000 7fea3f400640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea240a2b70 0x7fea240a5030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea3809b450 tx=0x7fea380cf6b0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:19.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.341+0000 7fea3ea00640 1 --2- 172.21.15.69:0/1037004983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 0x564da8c10d00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fea30009860 tx=0x7fea30009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e80fca53291cb52a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:19.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.341+0000 7fea2fe00640 1 -- 172.21.15.69:0/1037004983 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30043020 con 0x564da8c08660 2024-03-19T01:13:19.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.342+0000 7fea2fe00640 1 -- 172.21.15.69:0/1037004983 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea30034cb0 con 0x564da8c08660 2024-03-19T01:13:19.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.342+0000 7fea2fe00640 1 -- 172.21.15.69:0/1037004983 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30039040 con 0x564da8c08660 2024-03-19T01:13:19.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.342+0000 7fea48bc1740 1 -- 172.21.15.69:0/1037004983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 msgr2=0x564da8c10d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:19.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.342+0000 7fea48bc1740 1 --2- 172.21.15.69:0/1037004983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 0x564da8c10d00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fea30009860 tx=0x7fea30009590 comp rx=0 tx=0).stop 2024-03-19T01:13:19.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.342+0000 7fea48bc1740 1 -- 172.21.15.69:0/1037004983 shutdown_connections 2024-03-19T01:13:19.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.342+0000 7fea48bc1740 1 --2- 172.21.15.69:0/1037004983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c13630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.342+0000 7fea48bc1740 1 --2- 172.21.15.69:0/1037004983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 0x564da8c10d00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.342+0000 7fea48bc1740 1 --2- 172.21.15.69:0/1037004983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 0x564da8c08120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.342+0000 7fea48bc1740 1 -- 172.21.15.69:0/1037004983 >> 172.21.15.69:0/1037004983 conn(0x564da8c02470 msgr2=0x564da8c02850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:19.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.343+0000 7fea48bc1740 1 -- 172.21.15.69:0/1037004983 shutdown_connections 2024-03-19T01:13:19.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.343+0000 7fea48bc1740 1 -- 172.21.15.69:0/1037004983 wait complete. 2024-03-19T01:13:19.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.343+0000 7fea48bc1740 1 -- start start 2024-03-19T01:13:19.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.343+0000 7fea48bc1740 1 Processor -- start 2024-03-19T01:13:19.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.344+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 0x564da8c0e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.344+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 0x564da8c0e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.344+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c0ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.344+0000 7fea48bc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564da8c13fd0 con 0x564da8c11240 2024-03-19T01:13:19.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.344+0000 7fea48bc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564da8b27fc0 con 0x564da8c07d40 2024-03-19T01:13:19.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.344+0000 7fea48bc1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564da8c14150 con 0x564da8c08660 2024-03-19T01:13:19.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.344+0000 7fea37e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 0x564da8c0e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.344+0000 7fea48bc1740 10 client.0 _ll_register_callbacks cb 0x564da8b27990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.344+0000 7fea37e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 0x564da8c0e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53014/0 (socket says 172.21.15.69:53014) 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.344+0000 7fea37e00640 1 -- 172.21.15.69:0/3739878720 learned_addr learned my addr 172.21.15.69:0/3739878720 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.344+0000 7fea37e00640 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c0ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea3ea00640 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 0x564da8c0e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea37e00640 1 -- 172.21.15.69:0/3739878720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 msgr2=0x564da8c0e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea37e00640 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 0x564da8c0e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea37e00640 1 -- 172.21.15.69:0/3739878720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 msgr2=0x564da8c0ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea37e00640 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c0ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea37e00640 1 -- 172.21.15.69:0/3739878720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564da88803f0 con 0x564da8c07d40 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea3ea00640 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 0x564da8c0e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea37e00640 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 0x564da8c0e2b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fea28010750 tx=0x7fea28009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ea4d52234b6a48f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea2cc00640 1 -- 172.21.15.69:0/3739878720 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28004260 con 0x564da8c07d40 2024-03-19T01:13:19.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea2cc00640 1 -- 172.21.15.69:0/3739878720 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fea28004400 con 0x564da8c07d40 2024-03-19T01:13:19.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea48bc1740 1 -- 172.21.15.69:0/3739878720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea30034cb0 con 0x564da8c07d40 2024-03-19T01:13:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea2cc00640 1 -- 172.21.15.69:0/3739878720 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fea2802a050 con 0x564da8c07d40 2024-03-19T01:13:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea48bc1740 10 client.26423 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.345+0000 7fea48bc1740 1 -- 172.21.15.69:0/3739878720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564da88400f0 con 0x564da8c07d40 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.346+0000 7fea2cc00640 1 -- 172.21.15.69:0/3739878720 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea30034cb0 con 0x564da8c07d40 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.346+0000 7fea2cc00640 1 -- 172.21.15.69:0/3739878720 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fea28003a00 con 0x564da8c07d40 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.346+0000 7fea48bc1740 10 client.26423 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.346+0000 7fea48bc1740 10 client.26423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.346+0000 7fea48bc1740 20 client.26423 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.346+0000 7fea48bc1740 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564da8c21610 0x564da8c23a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.346+0000 7fea48bc1740 4 client.26423 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.346+0000 7fea48bc1740 1 -- 172.21.15.69:0/3739878720 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564da8b0d610 con 0x564da8c21610 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.346+0000 7fea3ea00640 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564da8c21610 0x564da8c23a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.347+0000 7fea3ea00640 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564da8c21610 0x564da8c23a90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f252b758612e4d server_cookie=e646b1143d102a1f in_seq=0 out_seq=0 2024-03-19T01:13:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.347+0000 7fea2cc00640 10 client.26423 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:19.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea2cc00640 1 -- 172.21.15.69:0/3739878720 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564da8b0d610 con 0x564da8c21610 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea2cc00640 10 client.26423 handle_command_reply: tid=0 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea48bc1740 1 client.26423 shutdown 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea48bc1740 1 -- 172.21.15.69:0/3739878720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 msgr2=0x564da8c0e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea48bc1740 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 0x564da8c0e2b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fea28010750 tx=0x7fea28009260 comp rx=0 tx=0).stop 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea48bc1740 1 -- 172.21.15.69:0/3739878720 shutdown_connections 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea48bc1740 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564da8c21610 0x564da8c23a90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea48bc1740 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c0ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea2cc00640 0 client.26423 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea48bc1740 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c08660 0x564da8c0e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea48bc1740 1 --2- 172.21.15.69:0/3739878720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c07d40 0x564da8c0e2b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea48bc1740 1 -- 172.21.15.69:0/3739878720 >> 172.21.15.69:0/3739878720 conn(0x564da8c02470 msgr2=0x564da8c041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.349+0000 7fea48bc1740 1 -- 172.21.15.69:0/3739878720 shutdown_connections 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.350+0000 7fea48bc1740 1 -- 172.21.15.69:0/3739878720 wait complete. 2024-03-19T01:13:19.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.350+0000 7fea48bc1740 20 client.26423 trim_cache size 0 max 16384 2024-03-19T01:13:19.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.368+0000 7fea48bc1740 1 Processor -- start 2024-03-19T01:13:19.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.368+0000 7fea48bc1740 1 -- start start 2024-03-19T01:13:19.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.368+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c07d40 0x564da8c109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.368+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c1cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.368+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c11240 0x564da8c01530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea48bc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea401b1040 con 0x564da8c07d40 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea48bc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea401b2c40 con 0x564da8c08660 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea48bc1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea401b1710 con 0x564da8c11240 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea37e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c07d40 0x564da8c109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea3ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c1cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea37e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c07d40 0x564da8c109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37136/0 (socket says 172.21.15.69:37136) 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea3ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c1cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53024/0 (socket says 172.21.15.69:53024) 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea37e00640 1 -- 172.21.15.69:0/2375682359 learned_addr learned my addr 172.21.15.69:0/2375682359 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea3ea00640 1 -- 172.21.15.69:0/2375682359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c11240 msgr2=0x564da8c01530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea3ea00640 1 --2- 172.21.15.69:0/2375682359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c11240 0x564da8c01530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea3ea00640 1 -- 172.21.15.69:0/2375682359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c07d40 msgr2=0x564da8c109e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea3ea00640 1 --2- 172.21.15.69:0/2375682359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c07d40 0x564da8c109e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea3ea00640 1 -- 172.21.15.69:0/2375682359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564da8c08490 con 0x564da8c08660 2024-03-19T01:13:19.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea37e00640 1 --2- 172.21.15.69:0/2375682359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c07d40 0x564da8c109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:19.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.369+0000 7fea3ea00640 1 --2- 172.21.15.69:0/2375682359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c1cd70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x564da8c0f880 tx=0x7fea30009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cced34411e51c07b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:19.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea2f400640 1 -- 172.21.15.69:0/2375682359 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30054070 con 0x564da8c08660 2024-03-19T01:13:19.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea2f400640 1 -- 172.21.15.69:0/2375682359 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea30034ab0 con 0x564da8c08660 2024-03-19T01:13:19.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea2f400640 1 -- 172.21.15.69:0/2375682359 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3004f070 con 0x564da8c08660 2024-03-19T01:13:19.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea48bc1740 1 -- 172.21.15.69:0/2375682359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 msgr2=0x564da8c1cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:19.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea48bc1740 1 --2- 172.21.15.69:0/2375682359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c1cd70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x564da8c0f880 tx=0x7fea30009590 comp rx=0 tx=0).stop 2024-03-19T01:13:19.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea48bc1740 1 -- 172.21.15.69:0/2375682359 shutdown_connections 2024-03-19T01:13:19.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea48bc1740 1 --2- 172.21.15.69:0/2375682359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c11240 0x564da8c01530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea48bc1740 1 --2- 172.21.15.69:0/2375682359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c1cd70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea48bc1740 1 --2- 172.21.15.69:0/2375682359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c07d40 0x564da8c109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea48bc1740 1 -- 172.21.15.69:0/2375682359 >> 172.21.15.69:0/2375682359 conn(0x564da8c02470 msgr2=0x564da8c00e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea48bc1740 1 -- 172.21.15.69:0/2375682359 shutdown_connections 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.370+0000 7fea48bc1740 1 -- 172.21.15.69:0/2375682359 wait complete. 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.371+0000 7fea48bc1740 1 -- start start 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.371+0000 7fea48bc1740 1 Processor -- start 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.371+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c07d40 0x564da8c19490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.371+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c14920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.371+0000 7fea48bc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c14e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.371+0000 7fea48bc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea401b24b0 con 0x564da8c11240 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.371+0000 7fea48bc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea401b1de0 con 0x564da8c08660 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.371+0000 7fea48bc1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea401af020 con 0x564da8c07d40 2024-03-19T01:13:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.371+0000 7fea48bc1740 10 client.0 _ll_register_callbacks cb 0x7fea28013ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea3ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c14920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea3ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c14920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53036/0 (socket says 172.21.15.69:53036) 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea3ea00640 1 -- 172.21.15.69:0/2133724839 learned_addr learned my addr 172.21.15.69:0/2133724839 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea37e00640 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c07d40 0x564da8c19490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea37e00640 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c14e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea3ea00640 1 -- 172.21.15.69:0/2133724839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c07d40 msgr2=0x564da8c19490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea3ea00640 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c07d40 0x564da8c19490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea3ea00640 1 -- 172.21.15.69:0/2133724839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 msgr2=0x564da8c14e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea3ea00640 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c14e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea3ea00640 1 -- 172.21.15.69:0/2133724839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564da8c154d0 con 0x564da8c08660 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea37e00640 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c07d40 0x564da8c19490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea3ea00640 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c14920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fea30034920 tx=0x7fea30005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6574f7b42eca8f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.372+0000 7fea2cc00640 1 -- 172.21.15.69:0/2133724839 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30054070 con 0x564da8c08660 2024-03-19T01:13:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea48bc1740 1 -- 172.21.15.69:0/2133724839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea30034ab0 con 0x564da8c08660 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea2cc00640 1 -- 172.21.15.69:0/2133724839 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fea3002c510 con 0x564da8c08660 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea2cc00640 1 -- 172.21.15.69:0/2133724839 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fea3004f040 con 0x564da8c08660 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea48bc1740 10 client.26429 fetch_fsmap learned FSMap version 31 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea48bc1740 1 -- 172.21.15.69:0/2133724839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564da8c87830 con 0x564da8c08660 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea2cc00640 1 -- 172.21.15.69:0/2133724839 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea30034ab0 con 0x564da8c08660 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea2cc00640 1 -- 172.21.15.69:0/2133724839 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fea30032ab0 con 0x564da8c08660 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea48bc1740 10 client.26429 fetch_fsmap finished waiting for FSMap version 31 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea48bc1740 10 client.26429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea48bc1740 20 client.26429 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea48bc1740 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564da8c87ca0 0x564da8c8a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea48bc1740 4 client.26429 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea48bc1740 1 -- 172.21.15.69:0/2133724839 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564da8b79800 con 0x564da8c87ca0 2024-03-19T01:13:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.373+0000 7fea37e00640 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564da8c87ca0 0x564da8c8a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:19.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.374+0000 7fea37e00640 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564da8c87ca0 0x564da8c8a090 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef4f2c92b0b5a174 server_cookie=482335765e8a26b3 in_seq=0 out_seq=0 2024-03-19T01:13:19.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.374+0000 7fea2cc00640 10 client.26429 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea2cc00640 1 -- 172.21.15.69:0/2133724839 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x564da8b79800 con 0x564da8c87ca0 2024-03-19T01:13:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea2cc00640 10 client.26429 handle_command_reply: tid=0 2024-03-19T01:13:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 1 client.26429 shutdown 2024-03-19T01:13:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 1 -- 172.21.15.69:0/2133724839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 msgr2=0x564da8c14920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c14920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fea30034920 tx=0x7fea30005d30 comp rx=0 tx=0).stop 2024-03-19T01:13:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 1 -- 172.21.15.69:0/2133724839 shutdown_connections 2024-03-19T01:13:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564da8c87ca0 0x564da8c8a090 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564da8c11240 0x564da8c14e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564da8c08660 0x564da8c14920 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 1 --2- 172.21.15.69:0/2133724839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564da8c07d40 0x564da8c19490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea2cc00640 0 client.26429 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 1 -- 172.21.15.69:0/2133724839 >> 172.21.15.69:0/2133724839 conn(0x564da8c02470 msgr2=0x564da8c0eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 1 -- 172.21.15.69:0/2133724839 shutdown_connections 2024-03-19T01:13:19.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 1 -- 172.21.15.69:0/2133724839 wait complete. 2024-03-19T01:13:19.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.377+0000 7fea48bc1740 20 client.26429 trim_cache size 0 max 16384 2024-03-19T01:13:19.382 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:13:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (997 inodes in the stack)", 2024-03-19T01:13:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:13:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "a8a05487-d7f5-40c6-a03c-ff8a245e048a": { 2024-03-19T01:13:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:13:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a8a05487-d7f5-40c6-a03c-ff8a245e048a", 2024-03-19T01:13:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:13:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:13:19.383 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:13:19.383 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:13:19.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 -- 172.21.15.69:0/1126761231 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea240a2b70 msgr2=0x7fea240a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:19.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea240a2b70 0x7fea240a5030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea3809b450 tx=0x7fea380cf6b0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 -- 172.21.15.69:0/1126761231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea401099d0 msgr2=0x7fea4019ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:19.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea401099d0 0x7fea4019ec20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea3800e940 tx=0x7fea3800ee00 comp rx=0 tx=0).stop 2024-03-19T01:13:19.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 -- 172.21.15.69:0/1126761231 shutdown_connections 2024-03-19T01:13:19.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea401099d0 0x7fea4019ec20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea240a2b70 0x7fea240a5030 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea40104380 0x7fea4019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 --2- 172.21.15.69:0/1126761231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea401039c0 0x7fea4019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:19.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 -- 172.21.15.69:0/1126761231 >> 172.21.15.69:0/1126761231 conn(0x7fea400ff470 msgr2=0x7fea400ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:19.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 -- 172.21.15.69:0/1126761231 shutdown_connections 2024-03-19T01:13:19.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:19.378+0000 7fea45e00640 1 -- 172.21.15.69:0/1126761231 wait complete. 2024-03-19T01:13:19.394 INFO:tasks.cephfs.filesystem:scrub status for tag:a8a05487-d7f5-40c6-a03c-ff8a245e048a - {'path': '/', 'tag': 'a8a05487-d7f5-40c6-a03c-ff8a245e048a', 'options': 'recursive,force'} 2024-03-19T01:13:19.557 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:19 smithi086 ceph-mon[66829]: pgmap v82: 129 pgs: 129 active+clean; 306 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 331 op/s 2024-03-19T01:13:19.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:19 smithi163 ceph-mon[67428]: pgmap v82: 129 pgs: 129 active+clean; 306 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 331 op/s 2024-03-19T01:13:19.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:19 smithi069 ceph-mon[31727]: pgmap v82: 129 pgs: 129 active+clean; 306 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 331 op/s 2024-03-19T01:13:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:13:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:13:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:13:20.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:20.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:20.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:20.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:21.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:21 smithi163 ceph-mon[67428]: pgmap v83: 129 pgs: 129 active+clean; 306 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2024-03-19T01:13:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:21.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:21 smithi086 ceph-mon[66829]: pgmap v83: 129 pgs: 129 active+clean; 306 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2024-03-19T01:13:21.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:21.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:21 smithi069 ceph-mon[31727]: pgmap v83: 129 pgs: 129 active+clean; 306 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2024-03-19T01:13:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: pgmap v84: 129 pgs: 129 active+clean; 324 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 430 op/s 2024-03-19T01:13:23.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:23.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: pgmap v84: 129 pgs: 129 active+clean; 324 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 430 op/s 2024-03-19T01:13:23.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:23.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: pgmap v84: 129 pgs: 129 active+clean; 324 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 430 op/s 2024-03-19T01:13:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:23.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:24.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:24.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:24.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:24.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:24.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:24.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:24.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:25.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:25 smithi086 ceph-mon[66829]: pgmap v85: 129 pgs: 129 active+clean; 324 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 290 op/s 2024-03-19T01:13:25.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:25.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:25.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:25 smithi069 ceph-mon[31727]: pgmap v85: 129 pgs: 129 active+clean; 324 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 290 op/s 2024-03-19T01:13:25.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:25.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:25.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:25 smithi163 ceph-mon[67428]: pgmap v85: 129 pgs: 129 active+clean; 324 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 290 op/s 2024-03-19T01:13:25.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:25.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:26.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:26.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:26.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:26.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:26.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:26.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:26.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:26.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:26.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:27.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:27 smithi086 ceph-mon[66829]: pgmap v86: 129 pgs: 129 active+clean; 338 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 355 op/s 2024-03-19T01:13:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:27.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:27 smithi069 ceph-mon[31727]: pgmap v86: 129 pgs: 129 active+clean; 338 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 355 op/s 2024-03-19T01:13:27.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:27.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:27.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:27.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:27 smithi163 ceph-mon[67428]: pgmap v86: 129 pgs: 129 active+clean; 338 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 355 op/s 2024-03-19T01:13:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:28.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:28.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:28.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:28.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:28.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:28.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:28.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: pgmap v87: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 392 op/s 2024-03-19T01:13:29.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: pgmap v87: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 392 op/s 2024-03-19T01:13:29.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: pgmap v87: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 392 op/s 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:30.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:30.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:31.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:31 smithi086 ceph-mon[66829]: pgmap v88: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 308 op/s 2024-03-19T01:13:31.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:31 smithi069 ceph-mon[31727]: pgmap v88: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 308 op/s 2024-03-19T01:13:31.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:31 smithi163 ceph-mon[67428]: pgmap v88: 129 pgs: 129 active+clean; 347 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 308 op/s 2024-03-19T01:13:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: pgmap v89: 129 pgs: 129 active+clean; 366 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 470 op/s 2024-03-19T01:13:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:33.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:33.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:33.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: pgmap v89: 129 pgs: 129 active+clean; 366 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 470 op/s 2024-03-19T01:13:33.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:33.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:33.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:33.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:33.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:33.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:33.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:33.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: pgmap v89: 129 pgs: 129 active+clean; 366 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 470 op/s 2024-03-19T01:13:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:34.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:34.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:34.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:34.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:34.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:34.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:34.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:34.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:34.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:34.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:35.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:35.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: pgmap v90: 129 pgs: 129 active+clean; 366 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 321 op/s 2024-03-19T01:13:35.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:35.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:35.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:35.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:35.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: pgmap v90: 129 pgs: 129 active+clean; 366 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 321 op/s 2024-03-19T01:13:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: pgmap v90: 129 pgs: 129 active+clean; 366 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 321 op/s 2024-03-19T01:13:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:35.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:35.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:36.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:36.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:36.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:36.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:36.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:36.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:36.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:36.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:36.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:36.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:36.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:36.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:37.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:37 smithi086 ceph-mon[66829]: pgmap v91: 129 pgs: 129 active+clean; 375 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-03-19T01:13:37.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:37.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:37.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:37.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:37.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:37 smithi069 ceph-mon[31727]: pgmap v91: 129 pgs: 129 active+clean; 375 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-03-19T01:13:37.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:37.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:37.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:37.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:37 smithi163 ceph-mon[67428]: pgmap v91: 129 pgs: 129 active+clean; 375 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-03-19T01:13:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:38.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:38.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:38.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:38.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:38.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:38.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:38.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:38.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:38.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:38.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:38.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:38.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:39.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:39 smithi086 ceph-mon[66829]: pgmap v92: 129 pgs: 129 active+clean; 385 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 391 op/s 2024-03-19T01:13:39.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:39 smithi069 ceph-mon[31727]: pgmap v92: 129 pgs: 129 active+clean; 385 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 391 op/s 2024-03-19T01:13:39.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:39 smithi163 ceph-mon[67428]: pgmap v92: 129 pgs: 129 active+clean; 385 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 391 op/s 2024-03-19T01:13:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:40.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:40.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:40.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:40.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:40.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:40.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:40.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:40.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:41.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:41 smithi086 ceph-mon[66829]: pgmap v93: 129 pgs: 129 active+clean; 385 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-03-19T01:13:41.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:41 smithi069 ceph-mon[31727]: pgmap v93: 129 pgs: 129 active+clean; 385 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-03-19T01:13:41.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:41 smithi163 ceph-mon[67428]: pgmap v93: 129 pgs: 129 active+clean; 385 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-03-19T01:13:42.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:42.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:42.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:43.712 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:43 smithi163 ceph-mon[67428]: pgmap v94: 129 pgs: 129 active+clean; 399 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 444 op/s 2024-03-19T01:13:43.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:43 smithi086 ceph-mon[66829]: pgmap v94: 129 pgs: 129 active+clean; 399 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 444 op/s 2024-03-19T01:13:43.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:43 smithi069 ceph-mon[31727]: pgmap v94: 129 pgs: 129 active+clean; 399 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 444 op/s 2024-03-19T01:13:44.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:44 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:13:44.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:44 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:13:44.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:44 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:13:45.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:45 smithi086 ceph-mon[66829]: pgmap v95: 129 pgs: 129 active+clean; 399 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-19T01:13:45.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:45 smithi069 ceph-mon[31727]: pgmap v95: 129 pgs: 129 active+clean; 399 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-19T01:13:45.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:45 smithi163 ceph-mon[67428]: pgmap v95: 129 pgs: 129 active+clean; 399 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-19T01:13:47.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:47 smithi086 ceph-mon[66829]: pgmap v96: 129 pgs: 129 active+clean; 403 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 324 op/s 2024-03-19T01:13:47.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:47 smithi069 ceph-mon[31727]: pgmap v96: 129 pgs: 129 active+clean; 403 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 324 op/s 2024-03-19T01:13:47.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:47 smithi163 ceph-mon[67428]: pgmap v96: 129 pgs: 129 active+clean; 403 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 324 op/s 2024-03-19T01:13:49.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:13:49.395 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:13:49.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.634+0000 7f1c3f400640 1 -- 172.21.15.69:0/1951260224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c38102ab0 msgr2=0x7f1c38102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.634+0000 7f1c3f400640 1 --2- 172.21.15.69:0/1951260224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c38102ab0 0x7f1c38102e90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009900 tx=0x7f1c2c02bcd0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.634+0000 7f1c3f400640 1 -- 172.21.15.69:0/1951260224 shutdown_connections 2024-03-19T01:13:49.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.634+0000 7f1c3f400640 1 --2- 172.21.15.69:0/1951260224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c3810bdf0 0x7f1c3810e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.634+0000 7f1c3f400640 1 --2- 172.21.15.69:0/1951260224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c381033d0 0x7f1c3810b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.634+0000 7f1c3f400640 1 --2- 172.21.15.69:0/1951260224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c38102ab0 0x7f1c38102e90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.634+0000 7f1c3f400640 1 -- 172.21.15.69:0/1951260224 >> 172.21.15.69:0/1951260224 conn(0x7f1c38077880 msgr2=0x7f1c38077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:49.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.634+0000 7f1c3f400640 1 -- 172.21.15.69:0/1951260224 shutdown_connections 2024-03-19T01:13:49.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.635+0000 7f1c3f400640 1 -- 172.21.15.69:0/1951260224 wait complete. 2024-03-19T01:13:49.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.636+0000 7f1c3f400640 1 Processor -- start 2024-03-19T01:13:49.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.636+0000 7f1c3f400640 1 -- start start 2024-03-19T01:13:49.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c3f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38102ab0 0x7f1c3819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c3f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c381033d0 0x7f1c3819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c3f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3810bdf0 0x7f1c3819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c3f400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38111c10 con 0x7f1c38102ab0 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c3f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38111a90 con 0x7f1c3810bdf0 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c3f400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38111d90 con 0x7f1c381033d0 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c381033d0 0x7f1c3819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c381033d0 0x7f1c3819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37576/0 (socket says 172.21.15.69:37576) 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c37e00640 1 -- 172.21.15.69:0/3572120435 learned_addr learned my addr 172.21.15.69:0/3572120435 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c3ca00640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38102ab0 0x7f1c3819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c3d400640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3810bdf0 0x7f1c3819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c37e00640 1 -- 172.21.15.69:0/3572120435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3810bdf0 msgr2=0x7f1c3819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c37e00640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3810bdf0 0x7f1c3819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c37e00640 1 -- 172.21.15.69:0/3572120435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38102ab0 msgr2=0x7f1c3819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c37e00640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38102ab0 0x7f1c3819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c37e00640 1 -- 172.21.15.69:0/3572120435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3819f260 con 0x7f1c381033d0 2024-03-19T01:13:49.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.637+0000 7f1c3ca00640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38102ab0 0x7f1c3819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:49.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.638+0000 7f1c37e00640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c381033d0 0x7f1c3819a7b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400e9f0 tx=0x7f1c2400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:49.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.638+0000 7f1c35600640 1 -- 172.21.15.69:0/3572120435 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400cd80 con 0x7f1c381033d0 2024-03-19T01:13:49.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.638+0000 7f1c35600640 1 -- 172.21.15.69:0/3572120435 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c24010910 con 0x7f1c381033d0 2024-03-19T01:13:49.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.638+0000 7f1c35600640 1 -- 172.21.15.69:0/3572120435 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24018990 con 0x7f1c381033d0 2024-03-19T01:13:49.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.638+0000 7f1c3f400640 1 -- 172.21.15.69:0/3572120435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3819f550 con 0x7f1c381033d0 2024-03-19T01:13:49.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.638+0000 7f1c3f400640 1 -- 172.21.15.69:0/3572120435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c381a6de0 con 0x7f1c381033d0 2024-03-19T01:13:49.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.641+0000 7f1c35600640 1 -- 172.21.15.69:0/3572120435 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f1c24010ac0 con 0x7f1c381033d0 2024-03-19T01:13:49.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.643+0000 7f1c42227740 1 Processor -- start 2024-03-19T01:13:49.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.643+0000 7f1c35600640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1c200a29f0 0x7f1c200a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.643+0000 7f1c35600640 1 -- 172.21.15.69:0/3572120435 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1c240cf4d0 con 0x7f1c381033d0 2024-03-19T01:13:49.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.644+0000 7f1c42227740 1 -- start start 2024-03-19T01:13:49.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.644+0000 7f1c3d400640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1c200a29f0 0x7f1c200a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.644+0000 7f1c42227740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 0x55fea5343120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.644+0000 7f1c42227740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea534bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.644+0000 7f1c42227740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea534c240 0x55fea534e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.644+0000 7f1c42227740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fea52a72c0 con 0x55fea5342d40 2024-03-19T01:13:49.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.644+0000 7f1c42227740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fea524fb60 con 0x55fea534c240 2024-03-19T01:13:49.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.644+0000 7f1c42227740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fea52a3360 con 0x55fea5343660 2024-03-19T01:13:49.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.645+0000 7f1c3ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 0x55fea5343120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.645+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea534bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.645+0000 7f1c3ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 0x55fea5343120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47738/0 (socket says 172.21.15.69:47738) 2024-03-19T01:13:49.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.645+0000 7f1c3ca00640 1 -- 172.21.15.69:0/2739235197 learned_addr learned my addr 172.21.15.69:0/2739235197 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:49.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.645+0000 7f1c3d400640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1c200a29f0 0x7f1c200a4eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c30005fd0 tx=0x7f1c30009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:49.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.645+0000 7f1c3ca00640 1 -- 172.21.15.69:0/2739235197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 msgr2=0x55fea534bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:49.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.645+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea534bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37584/0 (socket says 172.21.15.69:37584) 2024-03-19T01:13:49.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.645+0000 7f1c3ca00640 1 --2- 172.21.15.69:0/2739235197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea534bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.645+0000 7f1c3ca00640 1 -- 172.21.15.69:0/2739235197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea534c240 msgr2=0x55fea534e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:49.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.645+0000 7f1c3ca00640 1 --2- 172.21.15.69:0/2739235197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea534c240 0x55fea534e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.645+0000 7f1c3ca00640 1 -- 172.21.15.69:0/2739235197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea50857d0 con 0x55fea5342d40 2024-03-19T01:13:49.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.646+0000 7f1c37e00640 1 --2- 172.21.15.69:0/2739235197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea534bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:13:49.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.646+0000 7f1c3ca00640 1 --2- 172.21.15.69:0/2739235197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 0x55fea5343120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c000c00 tx=0x7f1c2c02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dcdeb10dcd4937 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:49.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.646+0000 7f1c29c00640 1 -- 172.21.15.69:0/2739235197 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c004240 con 0x55fea5342d40 2024-03-19T01:13:49.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.646+0000 7f1c29c00640 1 -- 172.21.15.69:0/2739235197 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2c02ce00 con 0x55fea5342d40 2024-03-19T01:13:49.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.646+0000 7f1c29c00640 1 -- 172.21.15.69:0/2739235197 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c004920 con 0x55fea5342d40 2024-03-19T01:13:49.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.647+0000 7f1c42227740 1 -- 172.21.15.69:0/2739235197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 msgr2=0x55fea5343120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.647+0000 7f1c42227740 1 --2- 172.21.15.69:0/2739235197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 0x55fea5343120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c000c00 tx=0x7f1c2c02c670 comp rx=0 tx=0).stop 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.647+0000 7f1c42227740 1 -- 172.21.15.69:0/2739235197 shutdown_connections 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.647+0000 7f1c42227740 1 --2- 172.21.15.69:0/2739235197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea534c240 0x55fea534e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.647+0000 7f1c42227740 1 --2- 172.21.15.69:0/2739235197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea534bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.647+0000 7f1c42227740 1 --2- 172.21.15.69:0/2739235197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 0x55fea5343120 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.647+0000 7f1c42227740 1 -- 172.21.15.69:0/2739235197 >> 172.21.15.69:0/2739235197 conn(0x55fea533d470 msgr2=0x55fea533d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.649+0000 7f1c42227740 1 -- 172.21.15.69:0/2739235197 shutdown_connections 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.649+0000 7f1c42227740 1 -- 172.21.15.69:0/2739235197 wait complete. 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.649+0000 7f1c42227740 1 -- start start 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.649+0000 7f1c42227740 1 Processor -- start 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.650+0000 7f1c42227740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5342d40 0x55fea5349330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.650+0000 7f1c42227740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea5349870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.650+0000 7f1c42227740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea534c240 0x55fea5349db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.650+0000 7f1c42227740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fea534efd0 con 0x55fea534c240 2024-03-19T01:13:49.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.650+0000 7f1c42227740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fea5262fc0 con 0x55fea5342d40 2024-03-19T01:13:49.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.650+0000 7f1c42227740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fea534f150 con 0x55fea5343660 2024-03-19T01:13:49.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.650+0000 7f1c42227740 10 client.0 _ll_register_callbacks cb 0x55fea5262990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:49.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.651+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea5349870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.651+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea5349870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37598/0 (socket says 172.21.15.69:37598) 2024-03-19T01:13:49.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.651+0000 7f1c37e00640 1 -- 172.21.15.69:0/2712444686 learned_addr learned my addr 172.21.15.69:0/2712444686 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:49.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.651+0000 7f1c3ca00640 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5342d40 0x55fea5349330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.651+0000 7f1c3d400640 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea534c240 0x55fea5349db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.651+0000 7f1c3ca00640 1 -- 172.21.15.69:0/2712444686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 msgr2=0x55fea5349870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:49.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.651+0000 7f1c3ca00640 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea5349870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.651+0000 7f1c3ca00640 1 -- 172.21.15.69:0/2712444686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea534c240 msgr2=0x55fea5349db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:49.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.651+0000 7f1c3ca00640 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea534c240 0x55fea5349db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.651+0000 7f1c3ca00640 1 -- 172.21.15.69:0/2712444686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea4fbb3f0 con 0x55fea5342d40 2024-03-19T01:13:49.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.651+0000 7f1c3ca00640 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5342d40 0x55fea5349330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c005ec0 tx=0x7f1c2c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb90659efbf8a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:49.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.652+0000 7f1c1e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.652+0000 7f1c1e000640 1 -- 172.21.15.69:0/2712444686 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c04e020 con 0x55fea5342d40 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.652+0000 7f1c1e000640 1 -- 172.21.15.69:0/2712444686 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1c2c0037f0 con 0x55fea5342d40 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.652+0000 7f1c42227740 1 -- 172.21.15.69:0/2712444686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c2c02ce00 con 0x55fea5342d40 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.652+0000 7f1c1e000640 1 -- 172.21.15.69:0/2712444686 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1c2c059050 con 0x55fea5342d40 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.652+0000 7f1c42227740 10 client.26438 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.652+0000 7f1c42227740 1 -- 172.21.15.69:0/2712444686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fea4f7b0f0 con 0x55fea5342d40 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.652+0000 7f1c1e000640 1 -- 172.21.15.69:0/2712444686 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2c034440 con 0x55fea5342d40 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.653+0000 7f1c1e000640 1 -- 172.21.15.69:0/2712444686 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f1c2c03e420 con 0x55fea5342d40 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.653+0000 7f1c42227740 10 client.26438 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.653+0000 7f1c42227740 10 client.26438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.653+0000 7f1c42227740 20 client.26438 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:49.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.653+0000 7f1c42227740 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fea535c4c0 0x55fea535e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.653+0000 7f1c42227740 4 client.26438 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:49.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.653+0000 7f1c42227740 1 -- 172.21.15.69:0/2712444686 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1c2c02ce00 con 0x55fea535c4c0 2024-03-19T01:13:49.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.653+0000 7f1c37e00640 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fea535c4c0 0x55fea535e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.654+0000 7f1c37e00640 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fea535c4c0 0x55fea535e8b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af1da661cab8ff25 server_cookie=bd0ede8cc2314ae0 in_seq=0 out_seq=0 2024-03-19T01:13:49.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.654+0000 7f1c1e000640 10 client.26438 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:49.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.654+0000 7f1c1e000640 1 -- 172.21.15.69:0/2712444686 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f1c2c02ce00 con 0x55fea535c4c0 2024-03-19T01:13:49.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.654+0000 7f1c1e000640 10 client.26438 handle_command_reply: tid=0 2024-03-19T01:13:49.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.654+0000 7f1c42227740 1 client.26438 shutdown 2024-03-19T01:13:49.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c42227740 1 -- 172.21.15.69:0/2712444686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5342d40 msgr2=0x55fea5349330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c42227740 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5342d40 0x55fea5349330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c005ec0 tx=0x7f1c2c004270 comp rx=0 tx=0).stop 2024-03-19T01:13:49.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c42227740 1 -- 172.21.15.69:0/2712444686 shutdown_connections 2024-03-19T01:13:49.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c42227740 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fea535c4c0 0x55fea535e8b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c42227740 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea534c240 0x55fea5349db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c42227740 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea5349870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c42227740 1 --2- 172.21.15.69:0/2712444686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5342d40 0x55fea5349330 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c1e000640 0 client.26438 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:49.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c42227740 1 -- 172.21.15.69:0/2712444686 >> 172.21.15.69:0/2712444686 conn(0x55fea533d470 msgr2=0x55fea533f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:49.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c42227740 1 -- 172.21.15.69:0/2712444686 shutdown_connections 2024-03-19T01:13:49.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c42227740 1 -- 172.21.15.69:0/2712444686 wait complete. 2024-03-19T01:13:49.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.655+0000 7f1c42227740 20 client.26438 trim_cache size 0 max 16384 2024-03-19T01:13:49.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.676+0000 7f1c42227740 1 Processor -- start 2024-03-19T01:13:49.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.676+0000 7f1c42227740 1 -- start start 2024-03-19T01:13:49.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c42227740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5342d40 0x55fea5357d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c42227740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5343660 0x55fea5363f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c42227740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea534c240 0x55fea5354390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c42227740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381acb90 con 0x55fea534c240 2024-03-19T01:13:49.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c42227740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381ae790 con 0x55fea5343660 2024-03-19T01:13:49.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c42227740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381ad260 con 0x55fea5342d40 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c3ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5342d40 0x55fea5357d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5343660 0x55fea5363f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5343660 0x55fea5363f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37298/0 (socket says 172.21.15.69:37298) 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c3ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5342d40 0x55fea5357d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37602/0 (socket says 172.21.15.69:37602) 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c37e00640 1 -- 172.21.15.69:0/2714780393 learned_addr learned my addr 172.21.15.69:0/2714780393 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c3d400640 1 --2- 172.21.15.69:0/2714780393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea534c240 0x55fea5354390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c37e00640 1 -- 172.21.15.69:0/2714780393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5342d40 msgr2=0x55fea5357d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c37e00640 1 --2- 172.21.15.69:0/2714780393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5342d40 0x55fea5357d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c37e00640 1 -- 172.21.15.69:0/2714780393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea534c240 msgr2=0x55fea5354390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c37e00640 1 --2- 172.21.15.69:0/2714780393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea534c240 0x55fea5354390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c37e00640 1 -- 172.21.15.69:0/2714780393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea532e950 con 0x55fea5343660 2024-03-19T01:13:49.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.677+0000 7f1c3d400640 1 --2- 172.21.15.69:0/2714780393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea534c240 0x55fea5354390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:49.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.678+0000 7f1c37e00640 1 --2- 172.21.15.69:0/2714780393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5343660 0x55fea5363f10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1c24097c70 tx=0x7f1c240cf040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c398b8c430b50dc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:49.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.678+0000 7f1c29200640 1 -- 172.21.15.69:0/2714780393 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24099b10 con 0x55fea5343660 2024-03-19T01:13:49.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.678+0000 7f1c29200640 1 -- 172.21.15.69:0/2714780393 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c240860b0 con 0x55fea5343660 2024-03-19T01:13:49.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.678+0000 7f1c29200640 1 -- 172.21.15.69:0/2714780393 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c240a3d80 con 0x55fea5343660 2024-03-19T01:13:49.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.679+0000 7f1c42227740 1 -- 172.21.15.69:0/2714780393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5343660 msgr2=0x55fea5363f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.679+0000 7f1c42227740 1 --2- 172.21.15.69:0/2714780393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5343660 0x55fea5363f10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1c24097c70 tx=0x7f1c240cf040 comp rx=0 tx=0).stop 2024-03-19T01:13:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.679+0000 7f1c42227740 1 -- 172.21.15.69:0/2714780393 shutdown_connections 2024-03-19T01:13:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.679+0000 7f1c42227740 1 --2- 172.21.15.69:0/2714780393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea534c240 0x55fea5354390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.679+0000 7f1c42227740 1 --2- 172.21.15.69:0/2714780393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea5343660 0x55fea5363f10 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.679+0000 7f1c42227740 1 --2- 172.21.15.69:0/2714780393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5342d40 0x55fea5357d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.679+0000 7f1c42227740 1 -- 172.21.15.69:0/2714780393 >> 172.21.15.69:0/2714780393 conn(0x55fea533d470 msgr2=0x55fea533c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.679+0000 7f1c42227740 1 -- 172.21.15.69:0/2714780393 shutdown_connections 2024-03-19T01:13:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.679+0000 7f1c42227740 1 -- 172.21.15.69:0/2714780393 wait complete. 2024-03-19T01:13:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.680+0000 7f1c42227740 1 -- start start 2024-03-19T01:13:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.680+0000 7f1c42227740 1 Processor -- start 2024-03-19T01:13:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c42227740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 0x55fea5351cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c42227740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea5352200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c42227740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea534c240 0x55fea5352740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c42227740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381ae000 con 0x55fea5342d40 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c42227740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381ad930 con 0x55fea534c240 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c42227740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1c381aab70 con 0x55fea5343660 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c42227740 10 client.0 _ll_register_callbacks cb 0x7f1c24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c3ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 0x55fea5351cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea5352200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c3ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 0x55fea5351cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47756/0 (socket says 172.21.15.69:47756) 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c3ca00640 1 -- 172.21.15.69:0/2212580937 learned_addr learned my addr 172.21.15.69:0/2212580937 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c3d400640 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea534c240 0x55fea5352740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c37e00640 1 -- 172.21.15.69:0/2212580937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea534c240 msgr2=0x55fea5352740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c37e00640 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea534c240 0x55fea5352740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c37e00640 1 -- 172.21.15.69:0/2212580937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 msgr2=0x55fea5351cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c37e00640 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 0x55fea5351cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.681+0000 7f1c37e00640 1 -- 172.21.15.69:0/2212580937 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea53c1550 con 0x55fea5343660 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.682+0000 7f1c37e00640 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea5352200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1c2409b7a0 tx=0x7f1c24097e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d83568e7d4227954 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.682+0000 7f1c1e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.682+0000 7f1c1e000640 1 -- 172.21.15.69:0/2212580937 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c240a3b20 con 0x55fea5343660 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.682+0000 7f1c1e000640 1 -- 172.21.15.69:0/2212580937 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1c240a3cc0 con 0x55fea5343660 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.682+0000 7f1c42227740 1 -- 172.21.15.69:0/2212580937 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c240860b0 con 0x55fea5343660 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.682+0000 7f1c1e000640 1 -- 172.21.15.69:0/2212580937 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1c2409cb40 con 0x55fea5343660 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.682+0000 7f1c42227740 10 client.26422 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.682+0000 7f1c42227740 1 -- 172.21.15.69:0/2212580937 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fea53c2800 con 0x55fea5343660 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.683+0000 7f1c1e000640 1 -- 172.21.15.69:0/2212580937 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c240860b0 con 0x55fea5343660 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.683+0000 7f1c1e000640 1 -- 172.21.15.69:0/2212580937 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f1c2409ac60 con 0x55fea5343660 2024-03-19T01:13:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.683+0000 7f1c42227740 10 client.26422 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.683+0000 7f1c42227740 10 client.26422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.683+0000 7f1c42227740 20 client.26422 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.683+0000 7f1c42227740 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fea53c2ea0 0x55fea53c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.683+0000 7f1c42227740 4 client.26422 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:13:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.683+0000 7f1c42227740 1 -- 172.21.15.69:0/2212580937 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fea5248610 con 0x55fea53c2ea0 2024-03-19T01:13:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.683+0000 7f1c3ca00640 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fea53c2ea0 0x55fea53c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c3ca00640 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fea53c2ea0 0x55fea53c52c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=700abc326169f945 server_cookie=47a58662e99ed1b5 in_seq=0 out_seq=0 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c1e000640 10 client.26422 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c1e000640 1 -- 172.21.15.69:0/2212580937 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fea5248610 con 0x55fea53c2ea0 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c1e000640 10 client.26422 handle_command_reply: tid=0 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c42227740 1 client.26422 shutdown 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c42227740 1 -- 172.21.15.69:0/2212580937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 msgr2=0x55fea5352200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c42227740 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea5352200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1c2409b7a0 tx=0x7f1c24097e80 comp rx=0 tx=0).stop 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c42227740 1 -- 172.21.15.69:0/2212580937 shutdown_connections 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c42227740 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fea53c2ea0 0x55fea53c52c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c42227740 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fea534c240 0x55fea5352740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c42227740 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fea5343660 0x55fea5352200 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c42227740 1 --2- 172.21.15.69:0/2212580937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fea5342d40 0x55fea5351cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c42227740 1 -- 172.21.15.69:0/2212580937 >> 172.21.15.69:0/2212580937 conn(0x55fea533d470 msgr2=0x55fea533e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.684+0000 7f1c1e000640 0 client.26422 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.685+0000 7f1c42227740 1 -- 172.21.15.69:0/2212580937 shutdown_connections 2024-03-19T01:13:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.685+0000 7f1c42227740 1 -- 172.21.15.69:0/2212580937 wait complete. 2024-03-19T01:13:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.685+0000 7f1c42227740 20 client.26422 trim_cache size 0 max 16384 2024-03-19T01:13:49.690 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:13:49.690 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:13:49.690 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:13:49.690 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:13:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.685+0000 7f1c3f400640 1 -- 172.21.15.69:0/3572120435 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1c200a29f0 msgr2=0x7f1c200a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.685+0000 7f1c3f400640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1c200a29f0 0x7f1c200a4eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c30005fd0 tx=0x7f1c30009500 comp rx=0 tx=0).stop 2024-03-19T01:13:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.685+0000 7f1c3f400640 1 -- 172.21.15.69:0/3572120435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c381033d0 msgr2=0x7f1c3819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.685+0000 7f1c3f400640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c381033d0 0x7f1c3819a7b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400e9f0 tx=0x7f1c2400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.686+0000 7f1c3f400640 1 -- 172.21.15.69:0/3572120435 shutdown_connections 2024-03-19T01:13:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.686+0000 7f1c3f400640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1c200a29f0 0x7f1c200a4eb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.686+0000 7f1c3f400640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3810bdf0 0x7f1c3819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.686+0000 7f1c3f400640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c381033d0 0x7f1c3819a7b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.686+0000 7f1c3f400640 1 --2- 172.21.15.69:0/3572120435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c38102ab0 0x7f1c3819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.686+0000 7f1c3f400640 1 -- 172.21.15.69:0/3572120435 >> 172.21.15.69:0/3572120435 conn(0x7f1c38077880 msgr2=0x7f1c38101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.686+0000 7f1c3f400640 1 -- 172.21.15.69:0/3572120435 shutdown_connections 2024-03-19T01:13:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.686+0000 7f1c3f400640 1 -- 172.21.15.69:0/3572120435 wait complete. 2024-03-19T01:13:49.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:49 smithi069 ceph-mon[31727]: pgmap v97: 129 pgs: 129 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 350 op/s 2024-03-19T01:13:49.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:49 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:13:49.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:49 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:13:49.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:49 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:13:49.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:49 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:13:49.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:49 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:13:49.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:49 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:13:49.711 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:13:49.711 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:13:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:49 smithi086 ceph-mon[66829]: pgmap v97: 129 pgs: 129 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 350 op/s 2024-03-19T01:13:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:49 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:13:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:49 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:13:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:49 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:13:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:49 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:13:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:49 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:13:49.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:49 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:13:49.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:49 smithi163 ceph-mon[67428]: pgmap v97: 129 pgs: 129 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 350 op/s 2024-03-19T01:13:49.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:49 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:13:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:49 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:13:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:49 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:13:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:49 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:13:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:49 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:13:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:49 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:13:49.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.975+0000 7fd49be00640 1 -- 172.21.15.69:0/1647876053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd49410e180 msgr2=0x7fd494110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.975+0000 7fd49be00640 1 --2- 172.21.15.69:0/1647876053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd49410e180 0x7fd494110570 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd480009860 tx=0x7fd48002bb90 comp rx=0 tx=0).stop 2024-03-19T01:13:49.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.975+0000 7fd49be00640 1 -- 172.21.15.69:0/1647876053 shutdown_connections 2024-03-19T01:13:49.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.975+0000 7fd49be00640 1 --2- 172.21.15.69:0/1647876053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd49410e180 0x7fd494110570 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.975+0000 7fd49be00640 1 --2- 172.21.15.69:0/1647876053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd494100ef0 0x7fd49410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.975+0000 7fd49be00640 1 --2- 172.21.15.69:0/1647876053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4941005d0 0x7fd4941009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.975+0000 7fd49be00640 1 -- 172.21.15.69:0/1647876053 >> 172.21.15.69:0/1647876053 conn(0x7fd494076aa0 msgr2=0x7fd494076eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:49.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.976+0000 7fd49be00640 1 -- 172.21.15.69:0/1647876053 shutdown_connections 2024-03-19T01:13:49.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.976+0000 7fd49be00640 1 -- 172.21.15.69:0/1647876053 wait complete. 2024-03-19T01:13:49.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.978+0000 7fd49be00640 1 Processor -- start 2024-03-19T01:13:49.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.978+0000 7fd49be00640 1 -- start start 2024-03-19T01:13:49.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.979+0000 7fd49be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4941005d0 0x7fd494073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.979+0000 7fd49be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494100ef0 0x7fd49406c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.979+0000 7fd49be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd49410e180 0x7fd49406c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.979+0000 7fd499e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd49410e180 0x7fd49406c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.979+0000 7fd499e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd49410e180 0x7fd49406c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37324/0 (socket says 172.21.15.69:37324) 2024-03-19T01:13:49.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.980+0000 7fd499400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4941005d0 0x7fd494073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.980+0000 7fd499e00640 1 -- 172.21.15.69:0/1923582776 learned_addr learned my addr 172.21.15.69:0/1923582776 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:49.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.980+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd494113de0 con 0x7fd494100ef0 2024-03-19T01:13:49.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.980+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd494113c60 con 0x7fd49410e180 2024-03-19T01:13:49.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.980+0000 7fd498a00640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494100ef0 0x7fd49406c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.980+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd494113f60 con 0x7fd4941005d0 2024-03-19T01:13:49.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.981+0000 7fd499e00640 1 -- 172.21.15.69:0/1923582776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4941005d0 msgr2=0x7fd494073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.981+0000 7fd499e00640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4941005d0 0x7fd494073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.981+0000 7fd499e00640 1 -- 172.21.15.69:0/1923582776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494100ef0 msgr2=0x7fd49406c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:49.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.981+0000 7fd499e00640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494100ef0 0x7fd49406c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:49.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.981+0000 7fd499e00640 1 -- 172.21.15.69:0/1923582776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49406ce10 con 0x7fd49410e180 2024-03-19T01:13:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.982+0000 7fd499e00640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd49410e180 0x7fd49406c6e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd48002c1a0 tx=0x7fd4800045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.982+0000 7fd492000640 1 -- 172.21.15.69:0/1923582776 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480041020 con 0x7fd49410e180 2024-03-19T01:13:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.982+0000 7fd492000640 1 -- 172.21.15.69:0/1923582776 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd48002ca10 con 0x7fd49410e180 2024-03-19T01:13:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.982+0000 7fd492000640 1 -- 172.21.15.69:0/1923582776 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480039040 con 0x7fd49410e180 2024-03-19T01:13:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.982+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd494070c40 con 0x7fd49410e180 2024-03-19T01:13:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.982+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd494071130 con 0x7fd49410e180 2024-03-19T01:13:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.986+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49406cfe0 con 0x7fd49410e180 2024-03-19T01:13:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.991+0000 7fd492000640 1 -- 172.21.15.69:0/1923582776 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fd480044020 con 0x7fd49410e180 2024-03-19T01:13:49.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.994+0000 7fd492000640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd47c0a29b0 0x7fd47c0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:49.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.994+0000 7fd492000640 1 -- 172.21.15.69:0/1923582776 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fd4800f0ad0 con 0x7fd49410e180 2024-03-19T01:13:49.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.995+0000 7fd499400640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd47c0a29b0 0x7fd47c0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:49.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:49.997+0000 7fd499400640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd47c0a29b0 0x7fd47c0a4e70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd48c005c40 tx=0x7fd48c00a780 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:50.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.003+0000 7fd492000640 1 -- 172.21.15.69:0/1923582776 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd48004b020 con 0x7fd49410e180 2024-03-19T01:13:50.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.452+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd494108d80 con 0x7fd49410e180 2024-03-19T01:13:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.453+0000 7fd492000640 1 -- 172.21.15.69:0/1923582776 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11944 (secure 0 0 0) 0x7fd4800bd580 con 0x7fd49410e180 2024-03-19T01:13:50.454 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:13:50.454 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:13:48.258761+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24550,24494,24580,24581,24476]},"id":1}]} 2024-03-19T01:13:50.455 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 33 2024-03-19T01:13:50.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.459+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd47c0a29b0 msgr2=0x7fd47c0a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.459+0000 7fd49be00640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd47c0a29b0 0x7fd47c0a4e70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd48c005c40 tx=0x7fd48c00a780 comp rx=0 tx=0).stop 2024-03-19T01:13:50.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.459+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd49410e180 msgr2=0x7fd49406c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.459+0000 7fd49be00640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd49410e180 0x7fd49406c6e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd48002c1a0 tx=0x7fd4800045e0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.460+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 shutdown_connections 2024-03-19T01:13:50.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.460+0000 7fd49be00640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd47c0a29b0 0x7fd47c0a4e70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.460+0000 7fd49be00640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd49410e180 0x7fd49406c6e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.460+0000 7fd49be00640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494100ef0 0x7fd49406c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.460+0000 7fd49be00640 1 --2- 172.21.15.69:0/1923582776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4941005d0 0x7fd494073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.460+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 >> 172.21.15.69:0/1923582776 conn(0x7fd494076aa0 msgr2=0x7fd49410f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:50.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.460+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 shutdown_connections 2024-03-19T01:13:50.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.460+0000 7fd49be00640 1 -- 172.21.15.69:0/1923582776 wait complete. 2024-03-19T01:13:50.482 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:13:50.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.700+0000 7fbf7f200640 1 -- 172.21.15.69:0/2901834658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf800ffff0 msgr2=0x7fbf80100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.700+0000 7fbf7f200640 1 --2- 172.21.15.69:0/2901834658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf800ffff0 0x7fbf80100450 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009900 tx=0x7fbf6802bc40 comp rx=0 tx=0).stop 2024-03-19T01:13:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.701+0000 7fbf7f200640 1 -- 172.21.15.69:0/2901834658 shutdown_connections 2024-03-19T01:13:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.701+0000 7fbf7f200640 1 --2- 172.21.15.69:0/2901834658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf80100990 0x7fbf80110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.701+0000 7fbf7f200640 1 --2- 172.21.15.69:0/2901834658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf800ffff0 0x7fbf80100450 secure :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009900 tx=0x7fbf6802bc40 comp rx=0 tx=0).stop 2024-03-19T01:13:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.701+0000 7fbf7f200640 1 --2- 172.21.15.69:0/2901834658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf801024b0 0x7fbf80102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.701+0000 7fbf7f200640 1 -- 172.21.15.69:0/2901834658 >> 172.21.15.69:0/2901834658 conn(0x7fbf80075940 msgr2=0x7fbf80075d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.701+0000 7fbf7f200640 1 -- 172.21.15.69:0/2901834658 shutdown_connections 2024-03-19T01:13:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.701+0000 7fbf7f200640 1 -- 172.21.15.69:0/2901834658 wait complete. 2024-03-19T01:13:50.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.702+0000 7fbf7f200640 1 Processor -- start 2024-03-19T01:13:50.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.702+0000 7fbf7f200640 1 -- start start 2024-03-19T01:13:50.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.703+0000 7fbf7f200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf80100990 0x7fbf8019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.703+0000 7fbf7f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf801024b0 0x7fbf8019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.703+0000 7fbf7f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf8019ec60 0x7fbf8019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.703+0000 7fbf7f200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80113de0 con 0x7fbf80100990 2024-03-19T01:13:50.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.703+0000 7fbf7f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80113c60 con 0x7fbf8019ec60 2024-03-19T01:13:50.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.703+0000 7fbf7f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80113f60 con 0x7fbf801024b0 2024-03-19T01:13:50.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.703+0000 7fbf77e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf801024b0 0x7fbf8019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.703+0000 7fbf77e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf801024b0 0x7fbf8019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37630/0 (socket says 172.21.15.69:37630) 2024-03-19T01:13:50.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.703+0000 7fbf77e00640 1 -- 172.21.15.69:0/3013232045 learned_addr learned my addr 172.21.15.69:0/3013232045 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:50.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.703+0000 7fbf7de00640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf80100990 0x7fbf8019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.704+0000 7fbf7e800640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf8019ec60 0x7fbf8019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.704+0000 7fbf7de00640 1 -- 172.21.15.69:0/3013232045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf801024b0 msgr2=0x7fbf8019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.704+0000 7fbf7de00640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf801024b0 0x7fbf8019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.704+0000 7fbf7de00640 1 -- 172.21.15.69:0/3013232045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf8019ec60 msgr2=0x7fbf8019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.704+0000 7fbf7de00640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf8019ec60 0x7fbf8019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.704+0000 7fbf7de00640 1 -- 172.21.15.69:0/3013232045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8019f750 con 0x7fbf80100990 2024-03-19T01:13:50.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.704+0000 7fbf77e00640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf801024b0 0x7fbf8019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:50.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.704+0000 7fbf7de00640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf80100990 0x7fbf8019a3a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000e9f0 tx=0x7fbf7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:50.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.705+0000 7fbf76a00640 1 -- 172.21.15.69:0/3013232045 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000cd80 con 0x7fbf80100990 2024-03-19T01:13:50.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.705+0000 7fbf76a00640 1 -- 172.21.15.69:0/3013232045 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf700035f0 con 0x7fbf80100990 2024-03-19T01:13:50.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.705+0000 7fbf7f200640 1 -- 172.21.15.69:0/3013232045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8019fa40 con 0x7fbf80100990 2024-03-19T01:13:50.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.705+0000 7fbf76a00640 1 -- 172.21.15.69:0/3013232045 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70010690 con 0x7fbf80100990 2024-03-19T01:13:50.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.705+0000 7fbf7f200640 1 -- 172.21.15.69:0/3013232045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf801a71b0 con 0x7fbf80100990 2024-03-19T01:13:50.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.709+0000 7fbf76a00640 1 -- 172.21.15.69:0/3013232045 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fbf70003b30 con 0x7fbf80100990 2024-03-19T01:13:50.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.709+0000 7fbf879f8740 1 Processor -- start 2024-03-19T01:13:50.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.710+0000 7fbf879f8740 1 -- start start 2024-03-19T01:13:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.710+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d51679bf0 0x559d51679fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.710+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d51682bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.710+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d516830f0 0x559d516854e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.710+0000 7fbf879f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559d515de380 con 0x559d5167a510 2024-03-19T01:13:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.710+0000 7fbf879f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d51586d20 con 0x559d51679bf0 2024-03-19T01:13:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.710+0000 7fbf879f8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559d515da420 con 0x559d516830f0 2024-03-19T01:13:50.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.710+0000 7fbf77e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d516830f0 0x559d516854e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.710+0000 7fbf77e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d516830f0 0x559d516854e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37644/0 (socket says 172.21.15.69:37644) 2024-03-19T01:13:50.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.710+0000 7fbf77e00640 1 -- 172.21.15.69:0/799542925 learned_addr learned my addr 172.21.15.69:0/799542925 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:50.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.710+0000 7fbf7de00640 1 --2- 172.21.15.69:0/799542925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d51682bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.711+0000 7fbf77e00640 1 -- 172.21.15.69:0/799542925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d51679bf0 msgr2=0x559d51679fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:13:50.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.711+0000 7fbf77e00640 1 --2- 172.21.15.69:0/799542925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d51679bf0 0x559d51679fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.711+0000 7fbf77e00640 1 -- 172.21.15.69:0/799542925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 msgr2=0x559d51682bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:50.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.711+0000 7fbf77e00640 1 --2- 172.21.15.69:0/799542925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d51682bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.711+0000 7fbf77e00640 1 -- 172.21.15.69:0/799542925 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d512f23f0 con 0x559d516830f0 2024-03-19T01:13:50.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.711+0000 7fbf77e00640 1 --2- 172.21.15.69:0/799542925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d516830f0 0x559d516854e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009230 tx=0x7fbf68005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9614deda108576 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:50.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.711+0000 7fbf6f000640 1 -- 172.21.15.69:0/799542925 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68050070 con 0x559d516830f0 2024-03-19T01:13:50.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.711+0000 7fbf6f000640 1 -- 172.21.15.69:0/799542925 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf6802c660 con 0x559d516830f0 2024-03-19T01:13:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.711+0000 7fbf6f000640 1 -- 172.21.15.69:0/799542925 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6802caf0 con 0x559d516830f0 2024-03-19T01:13:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.711+0000 7fbf76a00640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf640a29f0 0x7fbf640a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.712+0000 7fbf76a00640 1 -- 172.21.15.69:0/3013232045 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fbf700cf550 con 0x7fbf80100990 2024-03-19T01:13:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.712+0000 7fbf879f8740 1 -- 172.21.15.69:0/799542925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d516830f0 msgr2=0x559d516854e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.712+0000 7fbf879f8740 1 --2- 172.21.15.69:0/799542925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d516830f0 0x559d516854e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009230 tx=0x7fbf68005a20 comp rx=0 tx=0).stop 2024-03-19T01:13:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.712+0000 7fbf879f8740 1 -- 172.21.15.69:0/799542925 shutdown_connections 2024-03-19T01:13:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.712+0000 7fbf879f8740 1 --2- 172.21.15.69:0/799542925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d516830f0 0x559d516854e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.712+0000 7fbf879f8740 1 --2- 172.21.15.69:0/799542925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d51682bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.712+0000 7fbf879f8740 1 --2- 172.21.15.69:0/799542925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d51679bf0 0x559d51679fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.712+0000 7fbf879f8740 1 -- 172.21.15.69:0/799542925 >> 172.21.15.69:0/799542925 conn(0x559d51674320 msgr2=0x559d51674700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.713+0000 7fbf7e800640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf640a29f0 0x7fbf640a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.713+0000 7fbf879f8740 1 -- 172.21.15.69:0/799542925 shutdown_connections 2024-03-19T01:13:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.713+0000 7fbf879f8740 1 -- 172.21.15.69:0/799542925 wait complete. 2024-03-19T01:13:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.713+0000 7fbf879f8740 1 -- start start 2024-03-19T01:13:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.714+0000 7fbf879f8740 1 Processor -- start 2024-03-19T01:13:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.715+0000 7fbf7e800640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf640a29f0 0x7fbf640a4eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf78005fd0 tx=0x7fbf78009600 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.715+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d51680130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.715+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d51680670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.715+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d51680bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.715+0000 7fbf879f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559d51685e80 con 0x559d5167a510 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.715+0000 7fbf879f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d5159a160 con 0x559d516830f0 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.715+0000 7fbf879f8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559d51686000 con 0x559d51679bf0 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.715+0000 7fbf879f8740 10 client.0 _ll_register_callbacks cb 0x559d515d6170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf7de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d51680130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf7de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d51680130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37660/0 (socket says 172.21.15.69:37660) 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf7de00640 1 -- 172.21.15.69:0/899196766 learned_addr learned my addr 172.21.15.69:0/899196766 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf7e800640 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d51680bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf7de00640 1 -- 172.21.15.69:0/899196766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 msgr2=0x559d51680bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf77e00640 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d51680670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf7de00640 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d51680bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf7de00640 1 -- 172.21.15.69:0/899196766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 msgr2=0x559d51680670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf7de00640 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d51680670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf7de00640 1 -- 172.21.15.69:0/899196766 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d513bc7d0 con 0x559d51679bf0 2024-03-19T01:13:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf7de00640 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d51680130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000e9c0 tx=0x7fbf7009b2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39a98253e8b166ba server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf63400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf63400640 1 -- 172.21.15.69:0/899196766 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7009b790 con 0x559d51679bf0 2024-03-19T01:13:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.716+0000 7fbf879f8740 1 -- 172.21.15.69:0/899196766 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf6802c660 con 0x559d51679bf0 2024-03-19T01:13:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf63400640 1 -- 172.21.15.69:0/899196766 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fbf700037a0 con 0x559d51679bf0 2024-03-19T01:13:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf63400640 1 -- 172.21.15.69:0/899196766 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbf7009f8e0 con 0x559d51679bf0 2024-03-19T01:13:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf879f8740 10 client.26440 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf879f8740 1 -- 172.21.15.69:0/899196766 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559d516921f0 con 0x559d51679bf0 2024-03-19T01:13:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf63400640 1 -- 172.21.15.69:0/899196766 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf6802c660 con 0x559d51679bf0 2024-03-19T01:13:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf63400640 1 -- 172.21.15.69:0/899196766 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fbf700cf170 con 0x559d51679bf0 2024-03-19T01:13:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf879f8740 10 client.26440 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:50.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf879f8740 10 client.26440 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:13:50.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf879f8740 20 client.26440 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:50.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf879f8740 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559d516936b0 0x559d51695ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf879f8740 4 client.26440 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:50.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf879f8740 1 -- 172.21.15.69:0/899196766 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d515eb8c0 con 0x559d516936b0 2024-03-19T01:13:50.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.717+0000 7fbf77e00640 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559d516936b0 0x559d51695ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.718+0000 7fbf77e00640 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559d516936b0 0x559d51695ae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3c5d5f9f3aed03d3 server_cookie=690f98295962f9c in_seq=0 out_seq=0 2024-03-19T01:13:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.718+0000 7fbf63400640 10 client.26440 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:13:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.720+0000 7fbf63400640 1 -- 172.21.15.69:0/899196766 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559d515eb8c0 con 0x559d516936b0 2024-03-19T01:13:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.720+0000 7fbf63400640 10 client.26440 handle_command_reply: tid=0 2024-03-19T01:13:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.720+0000 7fbf879f8740 1 client.26440 shutdown 2024-03-19T01:13:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf879f8740 1 -- 172.21.15.69:0/899196766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 msgr2=0x559d51680130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf879f8740 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d51680130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000e9c0 tx=0x7fbf7009b2d0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf879f8740 1 -- 172.21.15.69:0/899196766 shutdown_connections 2024-03-19T01:13:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf879f8740 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559d516936b0 0x559d51695ae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf879f8740 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d51680bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf879f8740 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d51680670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf879f8740 1 --2- 172.21.15.69:0/899196766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d51680130 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf879f8740 1 -- 172.21.15.69:0/899196766 >> 172.21.15.69:0/899196766 conn(0x559d51674320 msgr2=0x559d51676070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf63400640 0 client.26440 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:13:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf879f8740 1 -- 172.21.15.69:0/899196766 shutdown_connections 2024-03-19T01:13:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf879f8740 1 -- 172.21.15.69:0/899196766 wait complete. 2024-03-19T01:13:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.721+0000 7fbf879f8740 20 client.26440 trim_cache size 0 max 16384 2024-03-19T01:13:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.736+0000 7fbf879f8740 1 Processor -- start 2024-03-19T01:13:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.736+0000 7fbf879f8740 1 -- start start 2024-03-19T01:13:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.736+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.736+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d516809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.736+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d516731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.736+0000 7fbf879f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801acd00 con 0x559d5167a510 2024-03-19T01:13:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.736+0000 7fbf879f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801ae900 con 0x559d516830f0 2024-03-19T01:13:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.736+0000 7fbf879f8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801ad3d0 con 0x559d51679bf0 2024-03-19T01:13:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.737+0000 7fbf7de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.737+0000 7fbf7de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37676/0 (socket says 172.21.15.69:37676) 2024-03-19T01:13:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.737+0000 7fbf7de00640 1 -- 172.21.15.69:0/3390772002 learned_addr learned my addr 172.21.15.69:0/3390772002 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.737+0000 7fbf7e800640 1 --2- 172.21.15.69:0/3390772002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d516731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.737+0000 7fbf77e00640 1 --2- 172.21.15.69:0/3390772002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d516809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.737+0000 7fbf7de00640 1 -- 172.21.15.69:0/3390772002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 msgr2=0x559d516731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:50.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.737+0000 7fbf7de00640 1 --2- 172.21.15.69:0/3390772002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d516731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.737+0000 7fbf7de00640 1 -- 172.21.15.69:0/3390772002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 msgr2=0x559d516809f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:50.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.737+0000 7fbf7de00640 1 --2- 172.21.15.69:0/3390772002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d516809f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.737+0000 7fbf7de00640 1 -- 172.21.15.69:0/3390772002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d5167a340 con 0x559d51679bf0 2024-03-19T01:13:50.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.737+0000 7fbf77e00640 1 --2- 172.21.15.69:0/3390772002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d516809f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:13:50.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.738+0000 7fbf7de00640 1 --2- 172.21.15.69:0/3390772002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168ec20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf7009b490 tx=0x7fbf70099ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b55f01d60b69d117 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:50.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.738+0000 7fbf6e600640 1 -- 172.21.15.69:0/3390772002 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7009b790 con 0x559d51679bf0 2024-03-19T01:13:50.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.738+0000 7fbf6e600640 1 -- 172.21.15.69:0/3390772002 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf700a3f30 con 0x559d51679bf0 2024-03-19T01:13:50.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.738+0000 7fbf6e600640 1 -- 172.21.15.69:0/3390772002 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf700cf170 con 0x559d51679bf0 2024-03-19T01:13:50.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.739+0000 7fbf879f8740 1 -- 172.21.15.69:0/3390772002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 msgr2=0x559d5168ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.739+0000 7fbf879f8740 1 --2- 172.21.15.69:0/3390772002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168ec20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf7009b490 tx=0x7fbf70099ca0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.740+0000 7fbf879f8740 1 -- 172.21.15.69:0/3390772002 shutdown_connections 2024-03-19T01:13:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.740+0000 7fbf879f8740 1 --2- 172.21.15.69:0/3390772002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d516731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.740+0000 7fbf879f8740 1 --2- 172.21.15.69:0/3390772002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d516809f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.740+0000 7fbf879f8740 1 --2- 172.21.15.69:0/3390772002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168ec20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.740+0000 7fbf879f8740 1 -- 172.21.15.69:0/3390772002 >> 172.21.15.69:0/3390772002 conn(0x559d51674320 msgr2=0x559d51672aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.740+0000 7fbf879f8740 1 -- 172.21.15.69:0/3390772002 shutdown_connections 2024-03-19T01:13:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.740+0000 7fbf879f8740 1 -- 172.21.15.69:0/3390772002 wait complete. 2024-03-19T01:13:50.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.741+0000 7fbf879f8740 1 -- start start 2024-03-19T01:13:50.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.741+0000 7fbf879f8740 1 Processor -- start 2024-03-19T01:13:50.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.741+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.741+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d5167c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.741+0000 7fbf879f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d5167cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.741+0000 7fbf879f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801ae170 con 0x559d5167a510 2024-03-19T01:13:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.741+0000 7fbf879f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801adaa0 con 0x559d516830f0 2024-03-19T01:13:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.741+0000 7fbf879f8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801aace0 con 0x559d51679bf0 2024-03-19T01:13:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.742+0000 7fbf879f8740 10 client.0 _ll_register_callbacks cb 0x7fbf70097a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.742+0000 7fbf7de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.742+0000 7fbf7e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d5167cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.742+0000 7fbf7de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37688/0 (socket says 172.21.15.69:37688) 2024-03-19T01:13:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.742+0000 7fbf7de00640 1 -- 172.21.15.69:0/1705728834 learned_addr learned my addr 172.21.15.69:0/1705728834 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.742+0000 7fbf7de00640 1 -- 172.21.15.69:0/1705728834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 msgr2=0x559d5167cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.742+0000 7fbf7de00640 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d5167cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.742+0000 7fbf7de00640 1 -- 172.21.15.69:0/1705728834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 msgr2=0x559d5167c5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.742+0000 7fbf7de00640 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d5167c5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.742+0000 7fbf7de00640 1 -- 172.21.15.69:0/1705728834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d5167d060 con 0x559d51679bf0 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.742+0000 7fbf7e800640 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d5167cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.743+0000 7fbf7de00640 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168b250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf700ccb00 tx=0x7fbf70005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e49f95263f774ba1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.743+0000 7fbf63400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.743+0000 7fbf63400640 1 -- 172.21.15.69:0/1705728834 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7009b0f0 con 0x559d51679bf0 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.743+0000 7fbf879f8740 1 -- 172.21.15.69:0/1705728834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf700a3f30 con 0x559d51679bf0 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.743+0000 7fbf63400640 1 -- 172.21.15.69:0/1705728834 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fbf700b4a50 con 0x559d51679bf0 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.743+0000 7fbf63400640 1 -- 172.21.15.69:0/1705728834 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbf70009430 con 0x559d51679bf0 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.743+0000 7fbf879f8740 10 client.26446 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.743+0000 7fbf879f8740 1 -- 172.21.15.69:0/1705728834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559d516f95d0 con 0x559d51679bf0 2024-03-19T01:13:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.744+0000 7fbf63400640 1 -- 172.21.15.69:0/1705728834 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf700a3f30 con 0x559d51679bf0 2024-03-19T01:13:50.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.744+0000 7fbf63400640 1 -- 172.21.15.69:0/1705728834 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fbf700afce0 con 0x559d51679bf0 2024-03-19T01:13:50.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.744+0000 7fbf879f8740 10 client.26446 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:50.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.744+0000 7fbf879f8740 10 client.26446 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:13:50.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.744+0000 7fbf879f8740 20 client.26446 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:50.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.744+0000 7fbf879f8740 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559d516f99a0 0x559d516fbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.744+0000 7fbf879f8740 4 client.26446 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:13:50.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.744+0000 7fbf879f8740 1 -- 172.21.15.69:0/1705728834 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d515e5c10 con 0x559d516f99a0 2024-03-19T01:13:50.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.745+0000 7fbf77e00640 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559d516f99a0 0x559d516fbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.745+0000 7fbf77e00640 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559d516f99a0 0x559d516fbdc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b6a7d8123018b566 server_cookie=3a22ab4e8baf6f1f in_seq=0 out_seq=0 2024-03-19T01:13:50.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.745+0000 7fbf63400640 10 client.26446 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:13:50.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.747+0000 7fbf63400640 1 -- 172.21.15.69:0/1705728834 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d515e5c10 con 0x559d516f99a0 2024-03-19T01:13:50.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.747+0000 7fbf63400640 10 client.26446 handle_command_reply: tid=0 2024-03-19T01:13:50.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.747+0000 7fbf879f8740 1 client.26446 shutdown 2024-03-19T01:13:50.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.748+0000 7fbf879f8740 1 -- 172.21.15.69:0/1705728834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 msgr2=0x559d5168b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.748+0000 7fbf879f8740 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168b250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf700ccb00 tx=0x7fbf70005bb0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.748+0000 7fbf879f8740 1 -- 172.21.15.69:0/1705728834 shutdown_connections 2024-03-19T01:13:50.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.748+0000 7fbf879f8740 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x559d516f99a0 0x559d516fbdc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.748+0000 7fbf879f8740 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d516830f0 0x559d5167cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.748+0000 7fbf879f8740 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559d5167a510 0x559d5167c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.748+0000 7fbf879f8740 1 --2- 172.21.15.69:0/1705728834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d51679bf0 0x559d5168b250 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.748+0000 7fbf879f8740 1 -- 172.21.15.69:0/1705728834 >> 172.21.15.69:0/1705728834 conn(0x559d51674320 msgr2=0x559d51673330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:50.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.748+0000 7fbf63400640 0 client.26446 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:13:50.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.748+0000 7fbf879f8740 1 -- 172.21.15.69:0/1705728834 shutdown_connections 2024-03-19T01:13:50.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.748+0000 7fbf879f8740 1 -- 172.21.15.69:0/1705728834 wait complete. 2024-03-19T01:13:50.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.749+0000 7fbf879f8740 20 client.26446 trim_cache size 0 max 16384 2024-03-19T01:13:50.751 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:13:50.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.749+0000 7fbf7f200640 1 -- 172.21.15.69:0/3013232045 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf640a29f0 msgr2=0x7fbf640a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.749+0000 7fbf7f200640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf640a29f0 0x7fbf640a4eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf78005fd0 tx=0x7fbf78009600 comp rx=0 tx=0).stop 2024-03-19T01:13:50.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.749+0000 7fbf7f200640 1 -- 172.21.15.69:0/3013232045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf80100990 msgr2=0x7fbf8019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.749+0000 7fbf7f200640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf80100990 0x7fbf8019a3a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000e9f0 tx=0x7fbf7000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.750+0000 7fbf7f200640 1 -- 172.21.15.69:0/3013232045 shutdown_connections 2024-03-19T01:13:50.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.750+0000 7fbf7f200640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf640a29f0 0x7fbf640a4eb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.750+0000 7fbf7f200640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf8019ec60 0x7fbf8019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.750+0000 7fbf7f200640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf801024b0 0x7fbf8019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.750+0000 7fbf7f200640 1 --2- 172.21.15.69:0/3013232045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf80100990 0x7fbf8019a3a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.750+0000 7fbf7f200640 1 -- 172.21.15.69:0/3013232045 >> 172.21.15.69:0/3013232045 conn(0x7fbf80075940 msgr2=0x7fbf800fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:50.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.750+0000 7fbf7f200640 1 -- 172.21.15.69:0/3013232045 shutdown_connections 2024-03-19T01:13:50.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.750+0000 7fbf7f200640 1 -- 172.21.15.69:0/3013232045 wait complete. 2024-03-19T01:13:50.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:50 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1923582776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:13:50.767 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:50 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1923582776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:13:50.769 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:13:50.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:50 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1923582776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:13:50.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.990+0000 7f131ea00640 1 -- 172.21.15.69:0/3622306308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1318103880 msgr2=0x7f1318103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.990+0000 7f131ea00640 1 --2- 172.21.15.69:0/3622306308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1318103880 0x7f1318103d00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1304009930 tx=0x7f130402bc20 comp rx=0 tx=0).stop 2024-03-19T01:13:50.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.990+0000 7f131ea00640 1 -- 172.21.15.69:0/3622306308 shutdown_connections 2024-03-19T01:13:50.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.990+0000 7f131ea00640 1 --2- 172.21.15.69:0/3622306308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1318104240 0x7f1318110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.990+0000 7f131ea00640 1 --2- 172.21.15.69:0/3622306308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1318103880 0x7f1318103d00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.990+0000 7f131ea00640 1 --2- 172.21.15.69:0/3622306308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1318109890 0x7f1318109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.990+0000 7f131ea00640 1 -- 172.21.15.69:0/3622306308 >> 172.21.15.69:0/3622306308 conn(0x7f13180ff330 msgr2=0x7f1318101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:50.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.990+0000 7f131ea00640 1 -- 172.21.15.69:0/3622306308 shutdown_connections 2024-03-19T01:13:50.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.990+0000 7f131ea00640 1 -- 172.21.15.69:0/3622306308 wait complete. 2024-03-19T01:13:50.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.991+0000 7f131ea00640 1 Processor -- start 2024-03-19T01:13:50.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.991+0000 7f131ea00640 1 -- start start 2024-03-19T01:13:50.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.991+0000 7f131ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1318103880 0x7f131819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.991+0000 7f131ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1318104240 0x7f131819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.991+0000 7f131ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1318109890 0x7f131819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:50.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.991+0000 7f131ea00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1318113cf0 con 0x7f1318109890 2024-03-19T01:13:50.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.991+0000 7f131ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1318113b70 con 0x7f1318104240 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.991+0000 7f131ea00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1318113e70 con 0x7f1318103880 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1318103880 0x7f131819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1318104240 0x7f131819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1318103880 0x7f131819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37712/0 (socket says 172.21.15.69:37712) 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1318104240 0x7f131819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37376/0 (socket says 172.21.15.69:37376) 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131d600640 1 -- 172.21.15.69:0/4108983514 learned_addr learned my addr 172.21.15.69:0/4108983514 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131e000640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1318109890 0x7f131819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131cc00640 1 -- 172.21.15.69:0/4108983514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1318103880 msgr2=0x7f131819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131cc00640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1318103880 0x7f131819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131cc00640 1 -- 172.21.15.69:0/4108983514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1318109890 msgr2=0x7f131819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131cc00640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1318109890 0x7f131819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131cc00640 1 -- 172.21.15.69:0/4108983514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131819f180 con 0x7f1318104240 2024-03-19T01:13:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.992+0000 7f131d600640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1318103880 0x7f131819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.993+0000 7f131cc00640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1318104240 0x7f131819a6d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1304036040 tx=0x7f1304004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.993+0000 7f1316000640 1 -- 172.21.15.69:0/4108983514 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304043020 con 0x7f1318104240 2024-03-19T01:13:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.993+0000 7f131ea00640 1 -- 172.21.15.69:0/4108983514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131819f410 con 0x7f1318104240 2024-03-19T01:13:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.993+0000 7f1316000640 1 -- 172.21.15.69:0/4108983514 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1304005000 con 0x7f1318104240 2024-03-19T01:13:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.993+0000 7f1316000640 1 -- 172.21.15.69:0/4108983514 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304003a80 con 0x7f1318104240 2024-03-19T01:13:50.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.993+0000 7f131ea00640 1 -- 172.21.15.69:0/4108983514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13181a6d00 con 0x7f1318104240 2024-03-19T01:13:50.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.996+0000 7f1322c6a740 1 Processor -- start 2024-03-19T01:13:51.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.996+0000 7f1322c6a740 1 -- start start 2024-03-19T01:13:51.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.996+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e32bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.996+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e334bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.996+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e3350f0 0x55d01e3374e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.996+0000 7f1322c6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d01e290380 con 0x55d01e32c510 2024-03-19T01:13:51.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.996+0000 7f1322c6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d01e238d20 con 0x55d01e32bbf0 2024-03-19T01:13:51.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.996+0000 7f1322c6a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d01e28c420 con 0x55d01e3350f0 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f131e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e32bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f131e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e32bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37382/0 (socket says 172.21.15.69:37382) 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f131e000640 1 -- 172.21.15.69:0/1963572207 learned_addr learned my addr 172.21.15.69:0/1963572207 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f131d600640 1 --2- 172.21.15.69:0/1963572207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e334bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f131e000640 1 -- 172.21.15.69:0/1963572207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e3350f0 msgr2=0x55d01e3374e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f131e000640 1 --2- 172.21.15.69:0/1963572207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e3350f0 0x55d01e3374e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f131e000640 1 -- 172.21.15.69:0/1963572207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 msgr2=0x55d01e334bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f131e000640 1 --2- 172.21.15.69:0/1963572207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e334bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f131e000640 1 -- 172.21.15.69:0/1963572207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d01dfa43f0 con 0x55d01e32bbf0 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f131e000640 1 --2- 172.21.15.69:0/1963572207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e32bfd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1310009fd0 tx=0x7f13100118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3efe86eef4a01200 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f1316000640 1 -- 172.21.15.69:0/4108983514 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f1304046020 con 0x7f1318104240 2024-03-19T01:13:51.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f130a600640 1 -- 172.21.15.69:0/1963572207 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1310004d20 con 0x55d01e32bbf0 2024-03-19T01:13:51.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f130a600640 1 -- 172.21.15.69:0/1963572207 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1310004050 con 0x55d01e32bbf0 2024-03-19T01:13:51.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.997+0000 7f130a600640 1 -- 172.21.15.69:0/1963572207 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1310005240 con 0x55d01e32bbf0 2024-03-19T01:13:51.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.998+0000 7f1322c6a740 1 -- 172.21.15.69:0/1963572207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 msgr2=0x55d01e32bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.998+0000 7f1322c6a740 1 --2- 172.21.15.69:0/1963572207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e32bfd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1310009fd0 tx=0x7f13100118d0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.998+0000 7f1322c6a740 1 -- 172.21.15.69:0/1963572207 shutdown_connections 2024-03-19T01:13:51.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.998+0000 7f1322c6a740 1 --2- 172.21.15.69:0/1963572207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e3350f0 0x55d01e3374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.998+0000 7f1322c6a740 1 --2- 172.21.15.69:0/1963572207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e334bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.998+0000 7f1322c6a740 1 --2- 172.21.15.69:0/1963572207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e32bfd0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.998+0000 7f1322c6a740 1 -- 172.21.15.69:0/1963572207 >> 172.21.15.69:0/1963572207 conn(0x55d01e326320 msgr2=0x55d01e326700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.999+0000 7f1322c6a740 1 -- 172.21.15.69:0/1963572207 shutdown_connections 2024-03-19T01:13:51.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.999+0000 7f1322c6a740 1 -- 172.21.15.69:0/1963572207 wait complete. 2024-03-19T01:13:51.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.999+0000 7f1322c6a740 1 -- start start 2024-03-19T01:13:51.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.999+0000 7f1316000640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f13000a2880 0x7f13000a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:50.999+0000 7f1322c6a740 1 Processor -- start 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f1316000640 1 -- 172.21.15.69:0/4108983514 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f13040f0fd0 con 0x7f1318104240 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f131e000640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f13000a2880 0x7f13000a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32bbf0 0x55d01e3321e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e32c510 0x55d01e332720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e3350f0 0x55d01e332c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f1322c6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d01e337ee0 con 0x55d01e32bbf0 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f1322c6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d01e24c160 con 0x55d01e3350f0 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f1322c6a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d01e338060 con 0x55d01e32c510 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f1322c6a740 10 client.0 _ll_register_callbacks cb 0x55d01e288170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f131cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e32c510 0x55d01e332720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f131cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e32c510 0x55d01e332720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37726/0 (socket says 172.21.15.69:37726) 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f131cc00640 1 -- 172.21.15.69:0/3462252251 learned_addr learned my addr 172.21.15.69:0/3462252251 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.000+0000 7f131d600640 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32bbf0 0x55d01e3321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f131e000640 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e3350f0 0x55d01e332c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f131cc00640 1 -- 172.21.15.69:0/3462252251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e3350f0 msgr2=0x55d01e332c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f131cc00640 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e3350f0 0x55d01e332c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f131e000640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f13000a2880 0x7f13000a4d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1310009de0 tx=0x7f1310005fd0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f131cc00640 1 -- 172.21.15.69:0/3462252251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32bbf0 msgr2=0x55d01e3321e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f131cc00640 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32bbf0 0x55d01e3321e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f131cc00640 1 -- 172.21.15.69:0/3462252251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d01e06e7d0 con 0x55d01e32c510 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f131cc00640 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e32c510 0x55d01e332720 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1304006010 tx=0x7f13040bcda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f25c9133ed135fc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f12fea00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f12fea00640 1 -- 172.21.15.69:0/3462252251 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13040c1590 con 0x55d01e32c510 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f12fea00640 1 -- 172.21.15.69:0/3462252251 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f13040d6260 con 0x55d01e32c510 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f1322c6a740 1 -- 172.21.15.69:0/3462252251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1310004050 con 0x55d01e32c510 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f12fea00640 1 -- 172.21.15.69:0/3462252251 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f13040c5be0 con 0x55d01e32c510 2024-03-19T01:13:51.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f1322c6a740 10 client.26455 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.001+0000 7f1322c6a740 1 -- 172.21.15.69:0/3462252251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d01e337f40 con 0x55d01e32c510 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.002+0000 7f12fea00640 1 -- 172.21.15.69:0/3462252251 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1310004050 con 0x55d01e32c510 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.002+0000 7f12fea00640 1 -- 172.21.15.69:0/3462252251 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f13040bd7a0 con 0x55d01e32c510 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.002+0000 7f1322c6a740 10 client.26455 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.002+0000 7f1322c6a740 10 client.26455 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.002+0000 7f1322c6a740 20 client.26455 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.002+0000 7f1322c6a740 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55d01e345600 0x55d01e347a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.002+0000 7f1322c6a740 4 client.26455 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.002+0000 7f1322c6a740 1 -- 172.21.15.69:0/3462252251 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d01e29d8c0 con 0x55d01e345600 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.002+0000 7f131d600640 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55d01e345600 0x55d01e347a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.003+0000 7f131d600640 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55d01e345600 0x55d01e347a20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17093dfce364bb6a server_cookie=ce287211ca139329 in_seq=0 out_seq=0 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.003+0000 7f12fea00640 10 client.26455 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:13:51.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.003+0000 7f12fea00640 1 -- 172.21.15.69:0/3462252251 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d01e29d8c0 con 0x55d01e345600 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.003+0000 7f12fea00640 10 client.26455 handle_command_reply: tid=0 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.004+0000 7f1322c6a740 1 client.26455 shutdown 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.004+0000 7f1322c6a740 1 -- 172.21.15.69:0/3462252251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e32c510 msgr2=0x55d01e332720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.004+0000 7f1322c6a740 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e32c510 0x55d01e332720 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1304006010 tx=0x7f13040bcda0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.005+0000 7f1322c6a740 1 -- 172.21.15.69:0/3462252251 shutdown_connections 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.005+0000 7f1322c6a740 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55d01e345600 0x55d01e347a20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.005+0000 7f1322c6a740 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e3350f0 0x55d01e332c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.005+0000 7f1322c6a740 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e32c510 0x55d01e332720 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.005+0000 7f1322c6a740 1 --2- 172.21.15.69:0/3462252251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32bbf0 0x55d01e3321e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.005+0000 7f1322c6a740 1 -- 172.21.15.69:0/3462252251 >> 172.21.15.69:0/3462252251 conn(0x55d01e326320 msgr2=0x55d01e328070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.005+0000 7f12fea00640 0 client.26455 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.005+0000 7f1322c6a740 1 -- 172.21.15.69:0/3462252251 shutdown_connections 2024-03-19T01:13:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.005+0000 7f1322c6a740 1 -- 172.21.15.69:0/3462252251 wait complete. 2024-03-19T01:13:51.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.005+0000 7f1322c6a740 20 client.26455 trim_cache size 0 max 16384 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.019+0000 7f1322c6a740 1 Processor -- start 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.019+0000 7f1322c6a740 1 -- start start 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.019+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e340c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.019+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e3328f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.019+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e34d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.019+0000 7f1322c6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13181b0c20 con 0x55d01e32c510 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.019+0000 7f1322c6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13181b2820 con 0x55d01e32bbf0 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.019+0000 7f1322c6a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f13181b12f0 con 0x55d01e326320 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.019+0000 7f131d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e340c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.019+0000 7f131d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e340c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37732/0 (socket says 172.21.15.69:37732) 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f131d600640 1 -- 172.21.15.69:0/4290298105 learned_addr learned my addr 172.21.15.69:0/4290298105 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f131cc00640 1 --2- 172.21.15.69:0/4290298105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e3328f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f131e000640 1 --2- 172.21.15.69:0/4290298105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e34d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f131d600640 1 -- 172.21.15.69:0/4290298105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 msgr2=0x55d01e3328f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f131d600640 1 --2- 172.21.15.69:0/4290298105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e3328f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f131d600640 1 -- 172.21.15.69:0/4290298105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 msgr2=0x55d01e34d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f131d600640 1 --2- 172.21.15.69:0/4290298105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e34d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f131d600640 1 -- 172.21.15.69:0/4290298105 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d01e3480a0 con 0x55d01e326320 2024-03-19T01:13:51.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f131cc00640 1 --2- 172.21.15.69:0/4290298105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e3328f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:51.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f131d600640 1 --2- 172.21.15.69:0/4290298105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e340c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f130c005e00 tx=0x7f130c012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=352db77d441cfca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f1309c00640 1 -- 172.21.15.69:0/4290298105 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c024070 con 0x55d01e326320 2024-03-19T01:13:51.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.020+0000 7f1309c00640 1 -- 172.21.15.69:0/4290298105 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130c005020 con 0x55d01e326320 2024-03-19T01:13:51.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.021+0000 7f1309c00640 1 -- 172.21.15.69:0/4290298105 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c004c40 con 0x55d01e326320 2024-03-19T01:13:51.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.021+0000 7f1322c6a740 1 -- 172.21.15.69:0/4290298105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 msgr2=0x55d01e340c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.021+0000 7f1322c6a740 1 --2- 172.21.15.69:0/4290298105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e340c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f130c005e00 tx=0x7f130c012be0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.021+0000 7f1322c6a740 1 -- 172.21.15.69:0/4290298105 shutdown_connections 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.021+0000 7f1322c6a740 1 --2- 172.21.15.69:0/4290298105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e34d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.021+0000 7f1322c6a740 1 --2- 172.21.15.69:0/4290298105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e3328f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.021+0000 7f1322c6a740 1 --2- 172.21.15.69:0/4290298105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e340c80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.021+0000 7f1322c6a740 1 -- 172.21.15.69:0/4290298105 >> 172.21.15.69:0/4290298105 conn(0x55d01e331de0 msgr2=0x55d01e3321e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.021+0000 7f1322c6a740 1 -- 172.21.15.69:0/4290298105 shutdown_connections 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.021+0000 7f1322c6a740 1 -- 172.21.15.69:0/4290298105 wait complete. 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.022+0000 7f1322c6a740 1 -- start start 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.022+0000 7f1322c6a740 1 Processor -- start 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.022+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e32f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.022+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e32f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.022+0000 7f1322c6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e3aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.022+0000 7f1322c6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13181b2090 con 0x55d01e32c510 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.022+0000 7f1322c6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13181b19c0 con 0x55d01e32bbf0 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.022+0000 7f1322c6a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f13181aec00 con 0x55d01e326320 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.022+0000 7f1322c6a740 10 client.0 _ll_register_callbacks cb 0x7f13040bd210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.022+0000 7f131d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e32f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f131d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e32f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37740/0 (socket says 172.21.15.69:37740) 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f131d600640 1 -- 172.21.15.69:0/4120686569 learned_addr learned my addr 172.21.15.69:0/4120686569 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f131cc00640 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e32f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f131e000640 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e3aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f131cc00640 1 -- 172.21.15.69:0/4120686569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 msgr2=0x55d01e32f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f131cc00640 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e32f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f131cc00640 1 -- 172.21.15.69:0/4120686569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 msgr2=0x55d01e3aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f131cc00640 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e3aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f131cc00640 1 -- 172.21.15.69:0/4120686569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d01e337850 con 0x55d01e32bbf0 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f131d600640 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e32f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f131cc00640 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e32f9d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f13040b9e10 tx=0x7f130402bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8ec05f43b49a7a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f12fea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f12fea00640 1 -- 172.21.15.69:0/4120686569 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13040bdb40 con 0x55d01e32bbf0 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f12fea00640 1 -- 172.21.15.69:0/4120686569 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f13040d6060 con 0x55d01e32bbf0 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.023+0000 7f1322c6a740 1 -- 172.21.15.69:0/4120686569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f130c005020 con 0x55d01e32bbf0 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f12fea00640 1 -- 172.21.15.69:0/4120686569 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f13040c1590 con 0x55d01e32bbf0 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f1322c6a740 10 client.26483 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f1322c6a740 1 -- 172.21.15.69:0/4120686569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d01e3ac000 con 0x55d01e32bbf0 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f12fea00640 1 -- 172.21.15.69:0/4120686569 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130c005020 con 0x55d01e32bbf0 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f12fea00640 1 -- 172.21.15.69:0/4120686569 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f13040bca20 con 0x55d01e32bbf0 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f1322c6a740 10 client.26483 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f1322c6a740 10 client.26483 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:13:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f1322c6a740 20 client.26483 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f1322c6a740 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55d01e3350f0 0x55d01e3ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f1322c6a740 4 client.26483 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f1322c6a740 1 -- 172.21.15.69:0/4120686569 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d01e297c10 con 0x55d01e3350f0 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.024+0000 7f131d600640 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55d01e3350f0 0x55d01e3ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f131d600640 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55d01e3350f0 0x55d01e3ae630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2611e13c276d79b1 server_cookie=f727534110fc2371 in_seq=0 out_seq=0 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f12fea00640 10 client.26483 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f12fea00640 1 -- 172.21.15.69:0/4120686569 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d01e297c10 con 0x55d01e3350f0 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f12fea00640 10 client.26483 handle_command_reply: tid=0 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 1 client.26483 shutdown 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 1 -- 172.21.15.69:0/4120686569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 msgr2=0x55d01e32f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e32f9d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f13040b9e10 tx=0x7f130402bc20 comp rx=0 tx=0).stop 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 1 -- 172.21.15.69:0/4120686569 shutdown_connections 2024-03-19T01:13:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d01e32c510 0x55d01e3aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55d01e3350f0 0x55d01e3ae630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d01e32bbf0 0x55d01e32f9d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 1 --2- 172.21.15.69:0/4120686569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d01e326320 0x55d01e32f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f12fea00640 0 client.26483 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:13:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 1 -- 172.21.15.69:0/4120686569 >> 172.21.15.69:0/4120686569 conn(0x55d01e331de0 msgr2=0x55d01e325330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 1 -- 172.21.15.69:0/4120686569 shutdown_connections 2024-03-19T01:13:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 1 -- 172.21.15.69:0/4120686569 wait complete. 2024-03-19T01:13:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.025+0000 7f1322c6a740 20 client.26483 trim_cache size 0 max 16384 2024-03-19T01:13:51.031 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:13:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 -- 172.21.15.69:0/4108983514 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f13000a2880 msgr2=0x7f13000a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f13000a2880 0x7f13000a4d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1310009de0 tx=0x7f1310005fd0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 -- 172.21.15.69:0/4108983514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1318104240 msgr2=0x7f131819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1318104240 0x7f131819a6d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1304036040 tx=0x7f1304004250 comp rx=0 tx=0).stop 2024-03-19T01:13:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 -- 172.21.15.69:0/4108983514 shutdown_connections 2024-03-19T01:13:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1318109890 0x7f131819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f13000a2880 0x7f13000a4d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1318104240 0x7f131819a6d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 --2- 172.21.15.69:0/4108983514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1318103880 0x7f131819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 -- 172.21.15.69:0/4108983514 >> 172.21.15.69:0/4108983514 conn(0x7f13180ff330 msgr2=0x7f13180ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 -- 172.21.15.69:0/4108983514 shutdown_connections 2024-03-19T01:13:51.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.026+0000 7f131ea00640 1 -- 172.21.15.69:0/4108983514 wait complete. 2024-03-19T01:13:51.045 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:13:51.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.266+0000 7f9377e00640 1 -- 172.21.15.69:0/2472964677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9378104260 msgr2=0x7f9378110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.266+0000 7f9377e00640 1 --2- 172.21.15.69:0/2472964677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9378104260 0x7f9378110770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9370009c10 tx=0x7f937002bee0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.267+0000 7f9377e00640 1 -- 172.21.15.69:0/2472964677 shutdown_connections 2024-03-19T01:13:51.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.267+0000 7f9377e00640 1 --2- 172.21.15.69:0/2472964677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9378104260 0x7f9378110770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.267+0000 7f9377e00640 1 --2- 172.21.15.69:0/2472964677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93781038a0 0x7f9378103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.267+0000 7f9377e00640 1 --2- 172.21.15.69:0/2472964677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93781098b0 0x7f9378109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.267+0000 7f9377e00640 1 -- 172.21.15.69:0/2472964677 >> 172.21.15.69:0/2472964677 conn(0x7f93780ff330 msgr2=0x7f9378101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.267+0000 7f9377e00640 1 -- 172.21.15.69:0/2472964677 shutdown_connections 2024-03-19T01:13:51.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.267+0000 7f9377e00640 1 -- 172.21.15.69:0/2472964677 wait complete. 2024-03-19T01:13:51.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9377e00640 1 Processor -- start 2024-03-19T01:13:51.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9377e00640 1 -- start start 2024-03-19T01:13:51.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9377e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93781038a0 0x7f937819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9377e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9378104260 0x7f937819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9377e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93781098b0 0x7f937819e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9377e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9378113d10 con 0x7f93781098b0 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9377e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9378113b90 con 0x7f93781038a0 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9377e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9378113e90 con 0x7f9378104260 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9376a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93781038a0 0x7f937819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9376a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93781038a0 0x7f937819a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37430/0 (socket says 172.21.15.69:37430) 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9376a00640 1 -- 172.21.15.69:0/2068127193 learned_addr learned my addr 172.21.15.69:0/2068127193 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9376000640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9378104260 0x7f937819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.268+0000 7f9377400640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93781098b0 0x7f937819e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f9376000640 1 -- 172.21.15.69:0/2068127193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93781038a0 msgr2=0x7f937819a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f9376000640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93781038a0 0x7f937819a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f9376000640 1 -- 172.21.15.69:0/2068127193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93781098b0 msgr2=0x7f937819e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f9376000640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93781098b0 0x7f937819e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f9376000640 1 -- 172.21.15.69:0/2068127193 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937819f120 con 0x7f9378104260 2024-03-19T01:13:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f9376a00640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93781038a0 0x7f937819a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f9376000640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9378104260 0x7f937819a670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f936400e9f0 tx=0x7f936400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f937ca00640 1 -- 172.21.15.69:0/2068127193 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936400ccc0 con 0x7f9378104260 2024-03-19T01:13:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f937ca00640 1 -- 172.21.15.69:0/2068127193 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9364010910 con 0x7f9378104260 2024-03-19T01:13:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f937ca00640 1 -- 172.21.15.69:0/2068127193 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364018960 con 0x7f9378104260 2024-03-19T01:13:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f9377e00640 1 -- 172.21.15.69:0/2068127193 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937819f410 con 0x7f9378104260 2024-03-19T01:13:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.269+0000 7f9377e00640 1 -- 172.21.15.69:0/2068127193 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93781a6ca0 con 0x7f9378104260 2024-03-19T01:13:51.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.273+0000 7f9380a06740 1 Processor -- start 2024-03-19T01:13:51.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.273+0000 7f9380a06740 1 -- start start 2024-03-19T01:13:51.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.273+0000 7f9380a06740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998eb0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.273+0000 7f9380a06740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998eb9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.273+0000 7f9380a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eba0f0 0x562998ebc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.273+0000 7f9380a06740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562998e15380 con 0x562998eb0bf0 2024-03-19T01:13:51.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.273+0000 7f9380a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562998dbdd20 con 0x562998eba0f0 2024-03-19T01:13:51.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.273+0000 7f9380a06740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562998e11420 con 0x562998eb1510 2024-03-19T01:13:51.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.273+0000 7f9376a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998eb9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.273+0000 7f9377400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998eb0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f9376a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998eb9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37786/0 (socket says 172.21.15.69:37786) 2024-03-19T01:13:51.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f9376a00640 1 -- 172.21.15.69:0/1410785130 learned_addr learned my addr 172.21.15.69:0/1410785130 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f9376a00640 1 -- 172.21.15.69:0/1410785130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eba0f0 msgr2=0x562998ebc4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:13:51.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f9376a00640 1 --2- 172.21.15.69:0/1410785130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eba0f0 0x562998ebc4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f9376a00640 1 -- 172.21.15.69:0/1410785130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 msgr2=0x562998eb0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f9376a00640 1 --2- 172.21.15.69:0/1410785130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998eb0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f9376a00640 1 -- 172.21.15.69:0/1410785130 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562998b293f0 con 0x562998eb1510 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f937ca00640 1 -- 172.21.15.69:0/2068127193 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9364010ac0 con 0x7f9378104260 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f9376a00640 1 --2- 172.21.15.69:0/1410785130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998eb9bb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9368011540 tx=0x7f9368011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d310106e3b236cd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f9363e00640 1 -- 172.21.15.69:0/1410785130 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936800f700 con 0x562998eb1510 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f9363e00640 1 -- 172.21.15.69:0/1410785130 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936800fd80 con 0x562998eb1510 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.274+0000 7f9363e00640 1 -- 172.21.15.69:0/1410785130 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93680183b0 con 0x562998eb1510 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.275+0000 7f9380a06740 1 -- 172.21.15.69:0/1410785130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 msgr2=0x562998eb9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.275+0000 7f9380a06740 1 --2- 172.21.15.69:0/1410785130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998eb9bb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9368011540 tx=0x7f9368011a00 comp rx=0 tx=0).stop 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.275+0000 7f9380a06740 1 -- 172.21.15.69:0/1410785130 shutdown_connections 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.275+0000 7f9380a06740 1 --2- 172.21.15.69:0/1410785130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eba0f0 0x562998ebc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.275+0000 7f9380a06740 1 --2- 172.21.15.69:0/1410785130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998eb9bb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.275+0000 7f9380a06740 1 --2- 172.21.15.69:0/1410785130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998eb0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.275+0000 7f9380a06740 1 -- 172.21.15.69:0/1410785130 >> 172.21.15.69:0/1410785130 conn(0x562998eab320 msgr2=0x562998eab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.275+0000 7f9380a06740 1 -- 172.21.15.69:0/1410785130 shutdown_connections 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.275+0000 7f9380a06740 1 -- 172.21.15.69:0/1410785130 wait complete. 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.276+0000 7f937ca00640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f935c0a2880 0x7f935c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.276+0000 7f937ca00640 1 -- 172.21.15.69:0/2068127193 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f93640cf5e0 con 0x7f9378104260 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.276+0000 7f9380a06740 1 -- start start 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.276+0000 7f9380a06740 1 Processor -- start 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9376a00640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f935c0a2880 0x7f935c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9380a06740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb0bf0 0x562998eb7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9380a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 0x562998eb76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9380a06740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eba0f0 0x562998eb7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9380a06740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562998ebcee0 con 0x562998eba0f0 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9380a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562998dd1160 con 0x562998eb1510 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9380a06740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562998ebd060 con 0x562998eb0bf0 2024-03-19T01:13:51.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9380a06740 10 client.0 _ll_register_callbacks cb 0x562998e0d170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9376a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb0bf0 0x562998eb7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9376000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 0x562998eb76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9376a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb0bf0 0x562998eb7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37800/0 (socket says 172.21.15.69:37800) 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9376a00640 1 -- 172.21.15.69:0/3074828090 learned_addr learned my addr 172.21.15.69:0/3074828090 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9377400640 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eba0f0 0x562998eb7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9376000640 1 -- 172.21.15.69:0/3074828090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb0bf0 msgr2=0x562998eb7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9376000640 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb0bf0 0x562998eb7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9376000640 1 -- 172.21.15.69:0/3074828090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eba0f0 msgr2=0x562998eb7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9376000640 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eba0f0 0x562998eb7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.277+0000 7f9376000640 1 -- 172.21.15.69:0/3074828090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562998bf37d0 con 0x562998eb1510 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.278+0000 7f9376a00640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f935c0a2880 0x7f935c0a4d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9368009e90 tx=0x7f93680199e0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.278+0000 7f9376000640 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 0x562998eb76c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f93640cc940 tx=0x7f936409b350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec1d6097c8ef369f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.278+0000 7f9360c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:51.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.278+0000 7f9360c00640 1 -- 172.21.15.69:0/3074828090 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93640862d0 con 0x562998eb1510 2024-03-19T01:13:51.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.278+0000 7f9360c00640 1 -- 172.21.15.69:0/3074828090 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f93640cf1e0 con 0x562998eb1510 2024-03-19T01:13:51.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.278+0000 7f9380a06740 1 -- 172.21.15.69:0/3074828090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936800fd80 con 0x562998eb1510 2024-03-19T01:13:51.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.278+0000 7f9360c00640 1 -- 172.21.15.69:0/3074828090 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f93640a4020 con 0x562998eb1510 2024-03-19T01:13:51.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.278+0000 7f9380a06740 10 client.26492 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:51.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.278+0000 7f9380a06740 1 -- 172.21.15.69:0/3074828090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562998ec9200 con 0x562998eb1510 2024-03-19T01:13:51.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.278+0000 7f9360c00640 1 -- 172.21.15.69:0/3074828090 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936800fd80 con 0x562998eb1510 2024-03-19T01:13:51.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.279+0000 7f9360c00640 1 -- 172.21.15.69:0/3074828090 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f936409af10 con 0x562998eb1510 2024-03-19T01:13:51.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.279+0000 7f9380a06740 10 client.26492 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:51.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.279+0000 7f9380a06740 10 client.26492 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:13:51.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.279+0000 7f9380a06740 20 client.26492 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.279+0000 7f9380a06740 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562998eca550 0x562998ecc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.279+0000 7f9380a06740 4 client.26492 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.279+0000 7f9380a06740 1 -- 172.21.15.69:0/3074828090 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562998e228c0 con 0x562998eca550 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.279+0000 7f9376a00640 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562998eca550 0x562998ecc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.279+0000 7f9376a00640 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562998eca550 0x562998ecc970 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba78f9864eaac0d8 server_cookie=34a189cde0377c3 in_seq=0 out_seq=0 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.279+0000 7f9360c00640 10 client.26492 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.280+0000 7f9360c00640 1 -- 172.21.15.69:0/3074828090 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562998e228c0 con 0x562998eca550 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.280+0000 7f9360c00640 10 client.26492 handle_command_reply: tid=0 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.280+0000 7f9380a06740 1 client.26492 shutdown 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9380a06740 1 -- 172.21.15.69:0/3074828090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 msgr2=0x562998eb76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9380a06740 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 0x562998eb76c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f93640cc940 tx=0x7f936409b350 comp rx=0 tx=0).stop 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9380a06740 1 -- 172.21.15.69:0/3074828090 shutdown_connections 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9380a06740 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562998eca550 0x562998ecc970 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9380a06740 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eba0f0 0x562998eb7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9380a06740 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 0x562998eb76c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9380a06740 1 --2- 172.21.15.69:0/3074828090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb0bf0 0x562998eb7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9360c00640 0 client.26492 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:13:51.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9380a06740 1 -- 172.21.15.69:0/3074828090 >> 172.21.15.69:0/3074828090 conn(0x562998eab320 msgr2=0x562998ead070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9380a06740 1 -- 172.21.15.69:0/3074828090 shutdown_connections 2024-03-19T01:13:51.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9380a06740 1 -- 172.21.15.69:0/3074828090 wait complete. 2024-03-19T01:13:51.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.281+0000 7f9380a06740 20 client.26492 trim_cache size 0 max 16384 2024-03-19T01:13:51.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.295+0000 7f9380a06740 1 Processor -- start 2024-03-19T01:13:51.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9380a06740 1 -- start start 2024-03-19T01:13:51.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9380a06740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998ec5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9380a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 0x562998eb7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9380a06740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eba0f0 0x562998ed2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9380a06740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93781b0d60 con 0x562998eb0bf0 2024-03-19T01:13:51.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9380a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93781b2960 con 0x562998eb1510 2024-03-19T01:13:51.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9380a06740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f93781b1430 con 0x562998eba0f0 2024-03-19T01:13:51.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9376000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 0x562998eb7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9377400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eba0f0 0x562998ed2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9376000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 0x562998eb7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37442/0 (socket says 172.21.15.69:37442) 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9377400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eba0f0 0x562998ed2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37806/0 (socket says 172.21.15.69:37806) 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9376000640 1 -- 172.21.15.69:0/3067326304 learned_addr learned my addr 172.21.15.69:0/3067326304 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9376a00640 1 --2- 172.21.15.69:0/3067326304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998ec5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9376000640 1 -- 172.21.15.69:0/3067326304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eba0f0 msgr2=0x562998ed2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9376000640 1 --2- 172.21.15.69:0/3067326304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eba0f0 0x562998ed2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9376000640 1 -- 172.21.15.69:0/3067326304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 msgr2=0x562998ec5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9376000640 1 --2- 172.21.15.69:0/3067326304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998ec5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.296+0000 7f9376000640 1 -- 172.21.15.69:0/3067326304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562998ebc810 con 0x562998eb1510 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.297+0000 7f9376a00640 1 --2- 172.21.15.69:0/3067326304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998ec5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.297+0000 7f9376000640 1 --2- 172.21.15.69:0/3067326304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 0x562998eb7a20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f936409b8c0 tx=0x7f936409fb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e9a3ebb46b17c56 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.297+0000 7f9363400640 1 -- 172.21.15.69:0/3067326304 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936409b470 con 0x562998eb1510 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.297+0000 7f9363400640 1 -- 172.21.15.69:0/3067326304 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93640cf1e0 con 0x562998eb1510 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.297+0000 7f9363400640 1 -- 172.21.15.69:0/3067326304 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936409afe0 con 0x562998eb1510 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.298+0000 7f9380a06740 1 -- 172.21.15.69:0/3067326304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 msgr2=0x562998eb7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.298+0000 7f9380a06740 1 --2- 172.21.15.69:0/3067326304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 0x562998eb7a20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f936409b8c0 tx=0x7f936409fb50 comp rx=0 tx=0).stop 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.298+0000 7f9380a06740 1 -- 172.21.15.69:0/3067326304 shutdown_connections 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.298+0000 7f9380a06740 1 --2- 172.21.15.69:0/3067326304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eba0f0 0x562998ed2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.298+0000 7f9380a06740 1 --2- 172.21.15.69:0/3067326304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eb1510 0x562998eb7a20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.298+0000 7f9380a06740 1 --2- 172.21.15.69:0/3067326304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998ec5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.298+0000 7f9380a06740 1 -- 172.21.15.69:0/3067326304 >> 172.21.15.69:0/3067326304 conn(0x562998eab320 msgr2=0x562998eacfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.298+0000 7f9380a06740 1 -- 172.21.15.69:0/3067326304 shutdown_connections 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.298+0000 7f9380a06740 1 -- 172.21.15.69:0/3067326304 wait complete. 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.298+0000 7f9380a06740 1 -- start start 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9380a06740 1 Processor -- start 2024-03-19T01:13:51.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9380a06740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998ebdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9380a06740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998ebe420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9380a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eba0f0 0x562998f2f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9380a06740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93781b21d0 con 0x562998eb0bf0 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9380a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93781b1b00 con 0x562998eba0f0 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9380a06740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f93781aed40 con 0x562998eb1510 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9380a06740 10 client.0 _ll_register_callbacks cb 0x7f9364010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9376000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998ebe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9376a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998ebdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9376000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998ebe420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37810/0 (socket says 172.21.15.69:37810) 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.299+0000 7f9376000640 1 -- 172.21.15.69:0/2487269390 learned_addr learned my addr 172.21.15.69:0/2487269390 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.300+0000 7f9377400640 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eba0f0 0x562998f2f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.300+0000 7f9376000640 1 -- 172.21.15.69:0/2487269390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eba0f0 msgr2=0x562998f2f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.300+0000 7f9376000640 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eba0f0 0x562998f2f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.300+0000 7f9376000640 1 -- 172.21.15.69:0/2487269390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 msgr2=0x562998ebdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.300+0000 7f9376000640 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998ebdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.300+0000 7f9376000640 1 -- 172.21.15.69:0/2487269390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562998ebecb0 con 0x562998eb1510 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.300+0000 7f9376000640 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998ebe420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f93640ccf70 tx=0x7f9364097bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=491631b07bd90016 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.300+0000 7f9360c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.300+0000 7f9360c00640 1 -- 172.21.15.69:0/2487269390 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936409b470 con 0x562998eb1510 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.300+0000 7f9360c00640 1 -- 172.21.15.69:0/2487269390 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f9364097040 con 0x562998eb1510 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.300+0000 7f9380a06740 1 -- 172.21.15.69:0/2487269390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93640cf1e0 con 0x562998eb1510 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.301+0000 7f9360c00640 1 -- 172.21.15.69:0/2487269390 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f93640ab050 con 0x562998eb1510 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.301+0000 7f9380a06740 10 client.26479 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.301+0000 7f9380a06740 1 -- 172.21.15.69:0/2487269390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562998ebecb0 con 0x562998eb1510 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.301+0000 7f9360c00640 1 -- 172.21.15.69:0/2487269390 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93640cf1e0 con 0x562998eb1510 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.301+0000 7f9360c00640 1 -- 172.21.15.69:0/2487269390 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f936409a7f0 con 0x562998eb1510 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.301+0000 7f9380a06740 10 client.26479 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:51.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.301+0000 7f9380a06740 10 client.26479 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.301+0000 7f9380a06740 20 client.26479 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.301+0000 7f9380a06740 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562998f31190 0x562998f335b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.301+0000 7f9380a06740 4 client.26479 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.301+0000 7f9380a06740 1 -- 172.21.15.69:0/2487269390 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562998e1cc10 con 0x562998f31190 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9376a00640 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562998f31190 0x562998f335b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9376a00640 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562998f31190 0x562998f335b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b16582ac395e80e server_cookie=d0fa3eac766b055 in_seq=0 out_seq=0 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9360c00640 10 client.26479 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9360c00640 1 -- 172.21.15.69:0/2487269390 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562998e1cc10 con 0x562998f31190 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9360c00640 10 client.26479 handle_command_reply: tid=0 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9380a06740 1 client.26479 shutdown 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9380a06740 1 -- 172.21.15.69:0/2487269390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 msgr2=0x562998ebe420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9380a06740 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998ebe420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f93640ccf70 tx=0x7f9364097bd0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9380a06740 1 -- 172.21.15.69:0/2487269390 shutdown_connections 2024-03-19T01:13:51.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9380a06740 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562998f31190 0x562998f335b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9380a06740 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562998eba0f0 0x562998f2f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9360c00640 0 client.26479 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:13:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9380a06740 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562998eb1510 0x562998ebe420 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9380a06740 1 --2- 172.21.15.69:0/2487269390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562998eb0bf0 0x562998ebdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.302+0000 7f9380a06740 1 -- 172.21.15.69:0/2487269390 >> 172.21.15.69:0/2487269390 conn(0x562998eab320 msgr2=0x562998ea9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.303+0000 7f9380a06740 1 -- 172.21.15.69:0/2487269390 shutdown_connections 2024-03-19T01:13:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.303+0000 7f9380a06740 1 -- 172.21.15.69:0/2487269390 wait complete. 2024-03-19T01:13:51.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.303+0000 7f9380a06740 20 client.26479 trim_cache size 0 max 16384 2024-03-19T01:13:51.311 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:13:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.303+0000 7f9377e00640 1 -- 172.21.15.69:0/2068127193 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f935c0a2880 msgr2=0x7f935c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.303+0000 7f9377e00640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f935c0a2880 0x7f935c0a4d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9368009e90 tx=0x7f93680199e0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.303+0000 7f9377e00640 1 -- 172.21.15.69:0/2068127193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9378104260 msgr2=0x7f937819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.303+0000 7f9377e00640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9378104260 0x7f937819a670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f936400e9f0 tx=0x7f936400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.303+0000 7f9377e00640 1 -- 172.21.15.69:0/2068127193 shutdown_connections 2024-03-19T01:13:51.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.304+0000 7f9377e00640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93781098b0 0x7f937819e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.304+0000 7f9377e00640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9378104260 0x7f937819a670 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.304+0000 7f9377e00640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f935c0a2880 0x7f935c0a4d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.304+0000 7f9377e00640 1 --2- 172.21.15.69:0/2068127193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93781038a0 0x7f937819a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.304+0000 7f9377e00640 1 -- 172.21.15.69:0/2068127193 >> 172.21.15.69:0/2068127193 conn(0x7f93780ff330 msgr2=0x7f93780ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.304+0000 7f9377e00640 1 -- 172.21.15.69:0/2068127193 shutdown_connections 2024-03-19T01:13:51.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.304+0000 7f9377e00640 1 -- 172.21.15.69:0/2068127193 wait complete. 2024-03-19T01:13:51.324 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:13:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.518+0000 7f0141800640 1 -- 172.21.15.69:0/3470081765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c10e010 msgr2=0x7f013c110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.518+0000 7f0141800640 1 --2- 172.21.15.69:0/3470081765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c10e010 0x7f013c110400 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0134009ca0 tx=0x7f013402c100 comp rx=0 tx=0).stop 2024-03-19T01:13:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.518+0000 7f0141800640 1 -- 172.21.15.69:0/3470081765 shutdown_connections 2024-03-19T01:13:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.518+0000 7f0141800640 1 --2- 172.21.15.69:0/3470081765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c10e010 0x7f013c110400 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.518+0000 7f0141800640 1 --2- 172.21.15.69:0/3470081765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f013c100e10 0x7f013c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.518+0000 7f0141800640 1 --2- 172.21.15.69:0/3470081765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f013c100460 0x7f013c100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.518+0000 7f0141800640 1 -- 172.21.15.69:0/3470081765 >> 172.21.15.69:0/3470081765 conn(0x7f013c0fc070 msgr2=0x7f013c0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.518+0000 7f0141800640 1 -- 172.21.15.69:0/3470081765 shutdown_connections 2024-03-19T01:13:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.518+0000 7f0141800640 1 -- 172.21.15.69:0/3470081765 wait complete. 2024-03-19T01:13:51.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.519+0000 7f0141800640 1 Processor -- start 2024-03-19T01:13:51.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.519+0000 7f0141800640 1 -- start start 2024-03-19T01:13:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f0141800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c100460 0x7f013c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f0141800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f013c100e10 0x7f013c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f0141800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f013c10e010 0x7f013c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f0141800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f013c113c80 con 0x7f013c100e10 2024-03-19T01:13:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f0141800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f013c113b00 con 0x7f013c100460 2024-03-19T01:13:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f0141800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f013c113e00 con 0x7f013c10e010 2024-03-19T01:13:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f013be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c100460 0x7f013c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f013be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c100460 0x7f013c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37470/0 (socket says 172.21.15.69:37470) 2024-03-19T01:13:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f013be00640 1 -- 172.21.15.69:0/2283877405 learned_addr learned my addr 172.21.15.69:0/2283877405 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f0140e00640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f013c10e010 0x7f013c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f0133e00640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f013c100e10 0x7f013c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f013be00640 1 -- 172.21.15.69:0/2283877405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f013c10e010 msgr2=0x7f013c19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f013be00640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f013c10e010 0x7f013c19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f013be00640 1 -- 172.21.15.69:0/2283877405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f013c100e10 msgr2=0x7f013c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f013be00640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f013c100e10 0x7f013c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f013be00640 1 -- 172.21.15.69:0/2283877405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c19f170 con 0x7f013c100460 2024-03-19T01:13:51.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.520+0000 7f0140e00640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f013c10e010 0x7f013c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:51.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.521+0000 7f013be00640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c100460 0x7f013c19a180 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f012c00e9f0 tx=0x7f012c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.521+0000 7f0139600640 1 -- 172.21.15.69:0/2283877405 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c00cd80 con 0x7f013c100460 2024-03-19T01:13:51.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.521+0000 7f0139600640 1 -- 172.21.15.69:0/2283877405 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f012c005500 con 0x7f013c100460 2024-03-19T01:13:51.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.521+0000 7f0141800640 1 -- 172.21.15.69:0/2283877405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c19f460 con 0x7f013c100460 2024-03-19T01:13:51.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.521+0000 7f0139600640 1 -- 172.21.15.69:0/2283877405 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c010690 con 0x7f013c100460 2024-03-19T01:13:51.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.521+0000 7f0141800640 1 -- 172.21.15.69:0/2283877405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c1a6cf0 con 0x7f013c100460 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.525+0000 7f0145c03740 1 Processor -- start 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.525+0000 7f0145c03740 1 -- start start 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.525+0000 7f0139600640 1 -- 172.21.15.69:0/2283877405 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f012c0046c0 con 0x7f013c100460 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0145c03740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab71bf0 0x55c17ab71fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0145c03740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 0x55c17ab7abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0145c03740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab7b0f0 0x55c17ab7d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0145c03740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c17aad6380 con 0x55c17ab72510 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0145c03740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c17aa7ed20 con 0x55c17ab71bf0 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0145c03740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c17aad2420 con 0x55c17ab7b0f0 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0140e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab71bf0 0x55c17ab71fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0140e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab71bf0 0x55c17ab71fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37482/0 (socket says 172.21.15.69:37482) 2024-03-19T01:13:51.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0140e00640 1 -- 172.21.15.69:0/98987561 learned_addr learned my addr 172.21.15.69:0/98987561 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f013be00640 1 --2- 172.21.15.69:0/98987561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 0x55c17ab7abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0133e00640 1 --2- 172.21.15.69:0/98987561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab7b0f0 0x55c17ab7d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0140e00640 1 -- 172.21.15.69:0/98987561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab7b0f0 msgr2=0x55c17ab7d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0140e00640 1 --2- 172.21.15.69:0/98987561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab7b0f0 0x55c17ab7d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0140e00640 1 -- 172.21.15.69:0/98987561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 msgr2=0x55c17ab7abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0140e00640 1 --2- 172.21.15.69:0/98987561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 0x55c17ab7abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f0140e00640 1 -- 172.21.15.69:0/98987561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c17a7ea3f0 con 0x55c17ab71bf0 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.526+0000 7f013be00640 1 --2- 172.21.15.69:0/98987561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 0x55c17ab7abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.527+0000 7f0140e00640 1 --2- 172.21.15.69:0/98987561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab71bf0 0x55c17ab71fd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f013c1013b0 tx=0x7f0134005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b3c5a91a205a94 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.527+0000 7f0131600640 1 -- 172.21.15.69:0/98987561 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134043070 con 0x55c17ab71bf0 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.527+0000 7f0131600640 1 -- 172.21.15.69:0/98987561 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0134032c90 con 0x55c17ab71bf0 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.527+0000 7f0131600640 1 -- 172.21.15.69:0/98987561 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134039040 con 0x55c17ab71bf0 2024-03-19T01:13:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.527+0000 7f0139600640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01200a2970 0x7f01200a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.527+0000 7f0139600640 1 -- 172.21.15.69:0/2283877405 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f012c0cf610 con 0x7f013c100460 2024-03-19T01:13:51.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f013be00640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01200a2970 0x7f01200a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f0145c03740 1 -- 172.21.15.69:0/98987561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab71bf0 msgr2=0x55c17ab71fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f0145c03740 1 --2- 172.21.15.69:0/98987561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab71bf0 0x55c17ab71fd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f013c1013b0 tx=0x7f0134005fa0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f0145c03740 1 -- 172.21.15.69:0/98987561 shutdown_connections 2024-03-19T01:13:51.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f0145c03740 1 --2- 172.21.15.69:0/98987561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab7b0f0 0x55c17ab7d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f0145c03740 1 --2- 172.21.15.69:0/98987561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 0x55c17ab7abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f0145c03740 1 --2- 172.21.15.69:0/98987561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab71bf0 0x55c17ab71fd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f0145c03740 1 -- 172.21.15.69:0/98987561 >> 172.21.15.69:0/98987561 conn(0x55c17ab6c320 msgr2=0x55c17ab6c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f0145c03740 1 -- 172.21.15.69:0/98987561 shutdown_connections 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f0145c03740 1 -- 172.21.15.69:0/98987561 wait complete. 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f013be00640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01200a2970 0x7f01200a4e30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f012c0cca50 tx=0x7f012c004a40 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.528+0000 7f0145c03740 1 -- start start 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0145c03740 1 Processor -- start 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0145c03740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab71bf0 0x55c17ab781e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0145c03740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 0x55c17ab78720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0145c03740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab7b0f0 0x55c17ab78c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0145c03740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c17ab7de80 con 0x55c17ab7b0f0 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0145c03740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c17aa92160 con 0x55c17ab72510 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0145c03740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c17ab7e000 con 0x55c17ab71bf0 2024-03-19T01:13:51.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0145c03740 10 client.0 _ll_register_callbacks cb 0x55c17aace170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0133e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 0x55c17ab78720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0133e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 0x55c17ab78720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37484/0 (socket says 172.21.15.69:37484) 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0133e00640 1 -- 172.21.15.69:0/2983012791 learned_addr learned my addr 172.21.15.69:0/2983012791 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.529+0000 7f0140e00640 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab71bf0 0x55c17ab781e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f0140e00640 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab7b0f0 0x55c17ab78c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f0133e00640 1 -- 172.21.15.69:0/2983012791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab71bf0 msgr2=0x55c17ab781e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f0133e00640 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab71bf0 0x55c17ab781e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f0133e00640 1 -- 172.21.15.69:0/2983012791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab7b0f0 msgr2=0x55c17ab78c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f0133e00640 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab7b0f0 0x55c17ab78c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f0133e00640 1 -- 172.21.15.69:0/2983012791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c17a8b47d0 con 0x55c17ab72510 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f0140e00640 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab7b0f0 0x55c17ab78c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f0133e00640 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 0x55c17ab78720 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f012400b9f0 tx=0x7f012400beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b806eb3cbde220 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f012a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f012a000640 1 -- 172.21.15.69:0/2983012791 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124003e20 con 0x55c17ab72510 2024-03-19T01:13:51.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f012a000640 1 -- 172.21.15.69:0/2983012791 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f0124005020 con 0x55c17ab72510 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.530+0000 7f0145c03740 1 -- 172.21.15.69:0/2983012791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0134032c90 con 0x55c17ab72510 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.531+0000 7f012a000640 1 -- 172.21.15.69:0/2983012791 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0124005220 con 0x55c17ab72510 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.531+0000 7f0145c03740 10 client.26510 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.531+0000 7f0145c03740 1 -- 172.21.15.69:0/2983012791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c17ab8a240 con 0x55c17ab72510 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.531+0000 7f012a000640 1 -- 172.21.15.69:0/2983012791 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0134032c90 con 0x55c17ab72510 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.531+0000 7f012a000640 1 -- 172.21.15.69:0/2983012791 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f012401b5f0 con 0x55c17ab72510 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.531+0000 7f0145c03740 10 client.26510 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.531+0000 7f0145c03740 10 client.26510 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.531+0000 7f0145c03740 20 client.26510 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.531+0000 7f0145c03740 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c17ab8b700 0x55c17ab8db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.531+0000 7f0145c03740 4 client.26510 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.531+0000 7f0145c03740 1 -- 172.21.15.69:0/2983012791 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c17aae38c0 con 0x55c17ab8b700 2024-03-19T01:13:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.532+0000 7f0140e00640 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c17ab8b700 0x55c17ab8db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.532+0000 7f012a000640 10 client.26510 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.532+0000 7f0140e00640 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c17ab8b700 0x55c17ab8db20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51c051060b0ed830 server_cookie=fbc5bbaef636a1fa in_seq=0 out_seq=0 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f012a000640 1 -- 172.21.15.69:0/2983012791 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c17aae38c0 con 0x55c17ab8b700 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f012a000640 10 client.26510 handle_command_reply: tid=0 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f0145c03740 1 client.26510 shutdown 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f0145c03740 1 -- 172.21.15.69:0/2983012791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 msgr2=0x55c17ab78720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f0145c03740 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 0x55c17ab78720 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f012400b9f0 tx=0x7f012400beb0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f0145c03740 1 -- 172.21.15.69:0/2983012791 shutdown_connections 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f0145c03740 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab7b0f0 0x55c17ab78c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f0145c03740 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c17ab8b700 0x55c17ab8db20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f0145c03740 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 0x55c17ab78720 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f0145c03740 1 --2- 172.21.15.69:0/2983012791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab71bf0 0x55c17ab781e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f012a000640 0 client.26510 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.533+0000 7f0145c03740 1 -- 172.21.15.69:0/2983012791 >> 172.21.15.69:0/2983012791 conn(0x55c17ab6c320 msgr2=0x55c17ab6e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.534+0000 7f0145c03740 1 -- 172.21.15.69:0/2983012791 shutdown_connections 2024-03-19T01:13:51.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.534+0000 7f0145c03740 1 -- 172.21.15.69:0/2983012791 wait complete. 2024-03-19T01:13:51.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.534+0000 7f0145c03740 20 client.26510 trim_cache size 0 max 16384 2024-03-19T01:13:51.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0145c03740 1 Processor -- start 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0145c03740 1 -- start start 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0145c03740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab71bf0 0x55c17ab7a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0145c03740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 0x55c17ab86c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0145c03740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab7b0f0 0x55c17ab933d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0145c03740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b0d60 con 0x55c17ab71bf0 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0145c03740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b2960 con 0x55c17ab72510 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0145c03740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b1430 con 0x55c17ab7b0f0 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0140e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab71bf0 0x55c17ab7a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0133e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 0x55c17ab86c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0140e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab71bf0 0x55c17ab7a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47972/0 (socket says 172.21.15.69:47972) 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0140e00640 1 -- 172.21.15.69:0/1072468571 learned_addr learned my addr 172.21.15.69:0/1072468571 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.548+0000 7f0140e00640 1 --2- 172.21.15.69:0/1072468571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab7b0f0 0x55c17ab933d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.549+0000 7f0133e00640 1 -- 172.21.15.69:0/1072468571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab7b0f0 msgr2=0x55c17ab933d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.549+0000 7f0133e00640 1 --2- 172.21.15.69:0/1072468571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab7b0f0 0x55c17ab933d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.549+0000 7f0133e00640 1 -- 172.21.15.69:0/1072468571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab71bf0 msgr2=0x55c17ab7a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.549+0000 7f0133e00640 1 --2- 172.21.15.69:0/1072468571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab71bf0 0x55c17ab7a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.549+0000 7f0133e00640 1 -- 172.21.15.69:0/1072468571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c17ab72340 con 0x55c17ab72510 2024-03-19T01:13:51.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.549+0000 7f0133e00640 1 --2- 172.21.15.69:0/1072468571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 0x55c17ab86c20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f012401ba90 tx=0x7f012400b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec398ee23cc4b7c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.549+0000 7f0130c00640 1 -- 172.21.15.69:0/1072468571 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012402c020 con 0x55c17ab72510 2024-03-19T01:13:51.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.549+0000 7f0130c00640 1 -- 172.21.15.69:0/1072468571 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0124003ea0 con 0x55c17ab72510 2024-03-19T01:13:51.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.549+0000 7f0130c00640 1 -- 172.21.15.69:0/1072468571 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240116a0 con 0x55c17ab72510 2024-03-19T01:13:51.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.550+0000 7f0145c03740 1 -- 172.21.15.69:0/1072468571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 msgr2=0x55c17ab86c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.550+0000 7f0145c03740 1 --2- 172.21.15.69:0/1072468571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 0x55c17ab86c20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f012401ba90 tx=0x7f012400b4a0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.550+0000 7f0145c03740 1 -- 172.21.15.69:0/1072468571 shutdown_connections 2024-03-19T01:13:51.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.550+0000 7f0145c03740 1 --2- 172.21.15.69:0/1072468571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab7b0f0 0x55c17ab933d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.550+0000 7f0145c03740 1 --2- 172.21.15.69:0/1072468571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab72510 0x55c17ab86c20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.550+0000 7f0145c03740 1 --2- 172.21.15.69:0/1072468571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab71bf0 0x55c17ab7a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.550+0000 7f0145c03740 1 -- 172.21.15.69:0/1072468571 >> 172.21.15.69:0/1072468571 conn(0x55c17ab6c320 msgr2=0x55c17ab6b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.550+0000 7f0145c03740 1 -- 172.21.15.69:0/1072468571 shutdown_connections 2024-03-19T01:13:51.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.550+0000 7f0145c03740 1 -- 172.21.15.69:0/1072468571 wait complete. 2024-03-19T01:13:51.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.550+0000 7f0145c03740 1 -- start start 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.550+0000 7f0145c03740 1 Processor -- start 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0145c03740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab71bf0 0x55c17ab75520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0145c03740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 0x55c17ab75a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0145c03740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab7b0f0 0x55c17abf0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0145c03740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b21d0 con 0x55c17ab72510 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0145c03740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b1b00 con 0x55c17ab7b0f0 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0145c03740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1aed40 con 0x55c17ab71bf0 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0145c03740 10 client.0 _ll_register_callbacks cb 0x7f0124004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0133e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 0x55c17ab75a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0140e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab7b0f0 0x55c17abf0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0133e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 0x55c17ab75a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47980/0 (socket says 172.21.15.69:47980) 2024-03-19T01:13:51.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0133e00640 1 -- 172.21.15.69:0/717375437 learned_addr learned my addr 172.21.15.69:0/717375437 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0140e00640 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab71bf0 0x55c17ab75520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.551+0000 7f0133e00640 1 -- 172.21.15.69:0/717375437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab71bf0 msgr2=0x55c17ab75520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f0133e00640 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab71bf0 0x55c17ab75520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f0133e00640 1 -- 172.21.15.69:0/717375437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab7b0f0 msgr2=0x55c17abf0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f0133e00640 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab7b0f0 0x55c17abf0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f0133e00640 1 -- 172.21.15.69:0/717375437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c17abf0fc0 con 0x55c17ab72510 2024-03-19T01:13:51.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f0140e00640 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab7b0f0 0x55c17abf0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:51.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f0133e00640 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 0x55c17ab75a60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f012401b6a0 tx=0x7f012401b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2194ca9f58b294b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f012a000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:13:51.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f012a000640 1 -- 172.21.15.69:0/717375437 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124011650 con 0x55c17ab72510 2024-03-19T01:13:51.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f012a000640 1 -- 172.21.15.69:0/717375437 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f01240117f0 con 0x55c17ab72510 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f0145c03740 1 -- 172.21.15.69:0/717375437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0124003ea0 con 0x55c17ab72510 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f012a000640 1 -- 172.21.15.69:0/717375437 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f01240119f0 con 0x55c17ab72510 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f0145c03740 10 client.16311 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.552+0000 7f0145c03740 1 -- 172.21.15.69:0/717375437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c17abf22a0 con 0x55c17ab72510 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.553+0000 7f012a000640 1 -- 172.21.15.69:0/717375437 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0124003ea0 con 0x55c17ab72510 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.553+0000 7f012a000640 1 -- 172.21.15.69:0/717375437 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f0124031b40 con 0x55c17ab72510 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.553+0000 7f0145c03740 10 client.16311 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.553+0000 7f0145c03740 10 client.16311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.553+0000 7f0145c03740 20 client.16311 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.553+0000 7f0145c03740 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c17abf2940 0x55c17abf4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.553+0000 7f0145c03740 4 client.16311 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.553+0000 7f0145c03740 1 -- 172.21.15.69:0/717375437 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c17aaddc10 con 0x55c17abf2940 2024-03-19T01:13:51.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.553+0000 7f0140e00640 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c17abf2940 0x55c17abf4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0140e00640 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c17abf2940 0x55c17abf4d30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=722f4a4434ae0106 server_cookie=a0fe777340b11292 in_seq=0 out_seq=0 2024-03-19T01:13:51.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f012a000640 10 client.16311 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:13:51.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f012a000640 1 -- 172.21.15.69:0/717375437 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c17aaddc10 con 0x55c17abf2940 2024-03-19T01:13:51.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f012a000640 10 client.16311 handle_command_reply: tid=0 2024-03-19T01:13:51.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0145c03740 1 client.16311 shutdown 2024-03-19T01:13:51.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0145c03740 1 -- 172.21.15.69:0/717375437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 msgr2=0x55c17ab75a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0145c03740 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 0x55c17ab75a60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f012401b6a0 tx=0x7f012401b6d0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0145c03740 1 -- 172.21.15.69:0/717375437 shutdown_connections 2024-03-19T01:13:51.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0145c03740 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c17abf2940 0x55c17abf4d30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0145c03740 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c17ab7b0f0 0x55c17abf0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0145c03740 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c17ab72510 0x55c17ab75a60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0145c03740 1 --2- 172.21.15.69:0/717375437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c17ab71bf0 0x55c17ab75520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0145c03740 1 -- 172.21.15.69:0/717375437 >> 172.21.15.69:0/717375437 conn(0x55c17ab6c320 msgr2=0x55c17ab6b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f012a000640 0 client.16311 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:13:51.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0145c03740 1 -- 172.21.15.69:0/717375437 shutdown_connections 2024-03-19T01:13:51.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.554+0000 7f0145c03740 1 -- 172.21.15.69:0/717375437 wait complete. 2024-03-19T01:13:51.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0145c03740 20 client.16311 trim_cache size 0 max 16384 2024-03-19T01:13:51.563 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:13:51.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 -- 172.21.15.69:0/2283877405 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01200a2970 msgr2=0x7f01200a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01200a2970 0x7f01200a4e30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f012c0cca50 tx=0x7f012c004a40 comp rx=0 tx=0).stop 2024-03-19T01:13:51.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 -- 172.21.15.69:0/2283877405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c100460 msgr2=0x7f013c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c100460 0x7f013c19a180 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f012c00e9f0 tx=0x7f012c00edf0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 -- 172.21.15.69:0/2283877405 shutdown_connections 2024-03-19T01:13:51.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f013c10e010 0x7f013c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f013c100e10 0x7f013c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01200a2970 0x7f01200a4e30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 --2- 172.21.15.69:0/2283877405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f013c100460 0x7f013c19a180 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 -- 172.21.15.69:0/2283877405 >> 172.21.15.69:0/2283877405 conn(0x7f013c0fc070 msgr2=0x7f013c10f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 -- 172.21.15.69:0/2283877405 shutdown_connections 2024-03-19T01:13:51.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.555+0000 7f0141800640 1 -- 172.21.15.69:0/2283877405 wait complete. 2024-03-19T01:13:51.573 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:13:51.633 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:51 smithi069 ceph-mon[31727]: pgmap v98: 129 pgs: 129 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 265 op/s 2024-03-19T01:13:51.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:51 smithi086 ceph-mon[66829]: pgmap v98: 129 pgs: 129 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 265 op/s 2024-03-19T01:13:51.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.777+0000 7f8914c00640 1 -- 172.21.15.69:0/2883801380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89101099d0 msgr2=0x7f8910109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.777+0000 7f8914c00640 1 --2- 172.21.15.69:0/2883801380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89101099d0 0x7f8910109db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8900009900 tx=0x7f890002bc30 comp rx=0 tx=0).stop 2024-03-19T01:13:51.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.777+0000 7f8914c00640 1 -- 172.21.15.69:0/2883801380 shutdown_connections 2024-03-19T01:13:51.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.777+0000 7f8914c00640 1 --2- 172.21.15.69:0/2883801380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8910104380 0x7f8910110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.777+0000 7f8914c00640 1 --2- 172.21.15.69:0/2883801380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89101039c0 0x7f8910103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.777+0000 7f8914c00640 1 --2- 172.21.15.69:0/2883801380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89101099d0 0x7f8910109db0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.777+0000 7f8914c00640 1 -- 172.21.15.69:0/2883801380 >> 172.21.15.69:0/2883801380 conn(0x7f89100ff490 msgr2=0x7f89101018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.777+0000 7f8914c00640 1 -- 172.21.15.69:0/2883801380 shutdown_connections 2024-03-19T01:13:51.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.777+0000 7f8914c00640 1 -- 172.21.15.69:0/2883801380 wait complete. 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.778+0000 7f8914c00640 1 Processor -- start 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.778+0000 7f8914c00640 1 -- start start 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f8914c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89101039c0 0x7f891019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f8914c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8910104380 0x7f891019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f8914c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89101099d0 0x7f891019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f8914c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8910113e00 con 0x7f89101099d0 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f8914c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8910113c80 con 0x7f89101039c0 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f8914c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8910113f80 con 0x7f8910104380 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f890de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89101039c0 0x7f891019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f890de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89101039c0 0x7f891019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37512/0 (socket says 172.21.15.69:37512) 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f890de00640 1 -- 172.21.15.69:0/1223470628 learned_addr learned my addr 172.21.15.69:0/1223470628 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f890e800640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89101099d0 0x7f891019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f890d400640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8910104380 0x7f891019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f890de00640 1 -- 172.21.15.69:0/1223470628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8910104380 msgr2=0x7f891019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f890de00640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8910104380 0x7f891019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f890de00640 1 -- 172.21.15.69:0/1223470628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89101099d0 msgr2=0x7f891019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f890de00640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89101099d0 0x7f891019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.779+0000 7f890de00640 1 -- 172.21.15.69:0/1223470628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891019f2b0 con 0x7f89101039c0 2024-03-19T01:13:51.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.780+0000 7f890d400640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8910104380 0x7f891019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:51.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.780+0000 7f890de00640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89101039c0 0x7f891019a2c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f890002efd0 tx=0x7f8900004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.781+0000 7f8906a00640 1 -- 172.21.15.69:0/1223470628 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900041070 con 0x7f89101039c0 2024-03-19T01:13:51.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.781+0000 7f8914c00640 1 -- 172.21.15.69:0/1223470628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891019f540 con 0x7f89101039c0 2024-03-19T01:13:51.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.781+0000 7f8906a00640 1 -- 172.21.15.69:0/1223470628 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890002ccc0 con 0x7f89101039c0 2024-03-19T01:13:51.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.781+0000 7f8906a00640 1 -- 172.21.15.69:0/1223470628 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900003ca0 con 0x7f89101039c0 2024-03-19T01:13:51.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.781+0000 7f8914c00640 1 -- 172.21.15.69:0/1223470628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89101a6e30 con 0x7f89101039c0 2024-03-19T01:13:51.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.785+0000 7f8906a00640 1 -- 172.21.15.69:0/1223470628 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f8900044070 con 0x7f89101039c0 2024-03-19T01:13:51.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.785+0000 7f8917816740 1 Processor -- start 2024-03-19T01:13:51.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.785+0000 7f8917816740 1 -- start start 2024-03-19T01:13:51.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f8917816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 0x556d35a47e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f8917816740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 0x556d35a509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f8917816740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a50f20 0x556d35a53310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f8917816740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556d359ac380 con 0x556d35a50f20 2024-03-19T01:13:51.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f8917816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d35954d20 con 0x556d35a47a20 2024-03-19T01:13:51.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f8917816740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556d359a8420 con 0x556d35a48340 2024-03-19T01:13:51.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f890de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 0x556d35a509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f890e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 0x556d35a47e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f890de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 0x556d35a509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37892/0 (socket says 172.21.15.69:37892) 2024-03-19T01:13:51.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f890de00640 1 -- 172.21.15.69:0/3662544616 learned_addr learned my addr 172.21.15.69:0/3662544616 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f890e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 0x556d35a47e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37526/0 (socket says 172.21.15.69:37526) 2024-03-19T01:13:51.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.786+0000 7f890d400640 1 --2- 172.21.15.69:0/3662544616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a50f20 0x556d35a53310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.787+0000 7f8906a00640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88f40a29f0 0x7f88f40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.787+0000 7f890de00640 1 -- 172.21.15.69:0/3662544616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 msgr2=0x556d35a47e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.787+0000 7f890de00640 1 --2- 172.21.15.69:0/3662544616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 0x556d35a47e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.787+0000 7f890de00640 1 -- 172.21.15.69:0/3662544616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a50f20 msgr2=0x556d35a53310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.787+0000 7f890de00640 1 --2- 172.21.15.69:0/3662544616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a50f20 0x556d35a53310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.787+0000 7f890de00640 1 -- 172.21.15.69:0/3662544616 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d356c03f0 con 0x556d35a48340 2024-03-19T01:13:51.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.787+0000 7f8906a00640 1 -- 172.21.15.69:0/1223470628 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f89000f0c20 con 0x7f89101039c0 2024-03-19T01:13:51.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.787+0000 7f890e800640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88f40a29f0 0x7f88f40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.788+0000 7f890de00640 1 --2- 172.21.15.69:0/3662544616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 0x556d35a509e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f89000edfa0 tx=0x7f89000b2400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20cc9c2d2f10d0f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.788+0000 7f88fb000640 1 -- 172.21.15.69:0/3662544616 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89000bcab0 con 0x556d35a48340 2024-03-19T01:13:51.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.788+0000 7f88fb000640 1 -- 172.21.15.69:0/3662544616 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8900004c80 con 0x556d35a48340 2024-03-19T01:13:51.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.788+0000 7f88fb000640 1 -- 172.21.15.69:0/3662544616 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89000bc8d0 con 0x556d35a48340 2024-03-19T01:13:51.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.789+0000 7f890e800640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88f40a29f0 0x7f88f40a4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8908009b80 tx=0x7f8908004a40 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.789+0000 7f8917816740 1 -- 172.21.15.69:0/3662544616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 msgr2=0x556d35a509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.789+0000 7f8917816740 1 --2- 172.21.15.69:0/3662544616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 0x556d35a509e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f89000edfa0 tx=0x7f89000b2400 comp rx=0 tx=0).stop 2024-03-19T01:13:51.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.790+0000 7f8917816740 1 -- 172.21.15.69:0/3662544616 shutdown_connections 2024-03-19T01:13:51.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.790+0000 7f8917816740 1 --2- 172.21.15.69:0/3662544616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a50f20 0x556d35a53310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.790+0000 7f8917816740 1 --2- 172.21.15.69:0/3662544616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 0x556d35a509e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.790+0000 7f8917816740 1 --2- 172.21.15.69:0/3662544616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 0x556d35a47e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.790+0000 7f8917816740 1 -- 172.21.15.69:0/3662544616 >> 172.21.15.69:0/3662544616 conn(0x556d35a42150 msgr2=0x556d35a42530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.790+0000 7f8917816740 1 -- 172.21.15.69:0/3662544616 shutdown_connections 2024-03-19T01:13:51.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.790+0000 7f8917816740 1 -- 172.21.15.69:0/3662544616 wait complete. 2024-03-19T01:13:51.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.791+0000 7f8917816740 1 -- start start 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.791+0000 7f8917816740 1 Processor -- start 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f8917816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 0x556d35a4df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f8917816740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 0x556d35a4e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f8917816740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a50f20 0x556d35a4e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f8917816740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556d359a8420 con 0x556d35a50f20 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f8917816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d35968160 con 0x556d35a47a20 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f8917816740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556d35a53ad0 con 0x556d35a48340 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f8917816740 10 client.0 _ll_register_callbacks cb 0x556d359a4170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f890de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 0x556d35a4df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f890de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 0x556d35a4df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37538/0 (socket says 172.21.15.69:37538) 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f890de00640 1 -- 172.21.15.69:0/3348142582 learned_addr learned my addr 172.21.15.69:0/3348142582 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f890d400640 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 0x556d35a4e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f890de00640 1 -- 172.21.15.69:0/3348142582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 msgr2=0x556d35a4e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f890de00640 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 0x556d35a4e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f890de00640 1 -- 172.21.15.69:0/3348142582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a50f20 msgr2=0x556d35a4e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f890de00640 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a50f20 0x556d35a4e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.792+0000 7f890de00640 1 -- 172.21.15.69:0/3348142582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d3578a7d0 con 0x556d35a47a20 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.793+0000 7f890de00640 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 0x556d35a4df30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8900005ab0 tx=0x7f8900009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=951bb3bc46fdce7b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.793+0000 7f88ef400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.793+0000 7f88ef400640 1 -- 172.21.15.69:0/3348142582 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890004b050 con 0x556d35a47a20 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.793+0000 7f88ef400640 1 -- 172.21.15.69:0/3348142582 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f89000f4220 con 0x556d35a47a20 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.793+0000 7f8917816740 1 -- 172.21.15.69:0/3348142582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8900004c80 con 0x556d35a47a20 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.793+0000 7f88ef400640 1 -- 172.21.15.69:0/3348142582 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f89000bc8d0 con 0x556d35a47a20 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.793+0000 7f8917816740 10 client.26528 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.793+0000 7f8917816740 1 -- 172.21.15.69:0/3348142582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556d35a5fc00 con 0x556d35a47a20 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.794+0000 7f88ef400640 1 -- 172.21.15.69:0/3348142582 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8900004c80 con 0x556d35a47a20 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.794+0000 7f88ef400640 1 -- 172.21.15.69:0/3348142582 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f89000f7020 con 0x556d35a47a20 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.794+0000 7f8917816740 10 client.26528 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.794+0000 7f8917816740 10 client.26528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:51.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.794+0000 7f8917816740 20 client.26528 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:51.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.794+0000 7f8917816740 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x556d35a60e70 0x556d35a63310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.794+0000 7f8917816740 4 client.26528 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:51.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.794+0000 7f8917816740 1 -- 172.21.15.69:0/3348142582 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d359b98c0 con 0x556d35a60e70 2024-03-19T01:13:51.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.794+0000 7f890d400640 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x556d35a60e70 0x556d35a63310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.795+0000 7f890d400640 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x556d35a60e70 0x556d35a63310 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea88594f915408a server_cookie=99f0bbca2fb1517c in_seq=0 out_seq=0 2024-03-19T01:13:51.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.795+0000 7f88ef400640 10 client.26528 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.795+0000 7f88ef400640 1 -- 172.21.15.69:0/3348142582 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556d359b98c0 con 0x556d35a60e70 2024-03-19T01:13:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.795+0000 7f88ef400640 10 client.26528 handle_command_reply: tid=0 2024-03-19T01:13:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.795+0000 7f8917816740 1 client.26528 shutdown 2024-03-19T01:13:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.796+0000 7f8917816740 1 -- 172.21.15.69:0/3348142582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 msgr2=0x556d35a4df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.796+0000 7f8917816740 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 0x556d35a4df30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8900005ab0 tx=0x7f8900009260 comp rx=0 tx=0).stop 2024-03-19T01:13:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.796+0000 7f8917816740 1 -- 172.21.15.69:0/3348142582 shutdown_connections 2024-03-19T01:13:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.796+0000 7f8917816740 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x556d35a60e70 0x556d35a63310 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.796+0000 7f8917816740 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a50f20 0x556d35a4e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.796+0000 7f8917816740 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a48340 0x556d35a4e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.796+0000 7f88ef400640 0 client.26528 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:51.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.796+0000 7f8917816740 1 --2- 172.21.15.69:0/3348142582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a47a20 0x556d35a4df30 secure :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8900005ab0 tx=0x7f8900009260 comp rx=0 tx=0).stop 2024-03-19T01:13:51.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.796+0000 7f8917816740 1 -- 172.21.15.69:0/3348142582 >> 172.21.15.69:0/3348142582 conn(0x556d35a42150 msgr2=0x556d35a43ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.797+0000 7f8917816740 1 -- 172.21.15.69:0/3348142582 shutdown_connections 2024-03-19T01:13:51.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.797+0000 7f8917816740 1 -- 172.21.15.69:0/3348142582 wait complete. 2024-03-19T01:13:51.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.797+0000 7f8917816740 20 client.26528 trim_cache size 0 max 16384 2024-03-19T01:13:51.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:51 smithi163 ceph-mon[67428]: pgmap v98: 129 pgs: 129 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 265 op/s 2024-03-19T01:13:51.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.815+0000 7f8917816740 1 Processor -- start 2024-03-19T01:13:51.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.815+0000 7f8917816740 1 -- start start 2024-03-19T01:13:51.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.815+0000 7f8917816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a48340 0x556d35a50660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.815+0000 7f8917816740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 0x556d35a5c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.815+0000 7f8917816740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 0x556d35a4b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.815+0000 7f8917816740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f89101b0f50 con 0x556d35a68be0 2024-03-19T01:13:51.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.815+0000 7f8917816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89101b2b50 con 0x556d35a48340 2024-03-19T01:13:51.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.815+0000 7f8917816740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f89101b1620 con 0x556d35a50f20 2024-03-19T01:13:51.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 0x556d35a5c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a48340 0x556d35a50660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 0x556d35a5c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37904/0 (socket says 172.21.15.69:37904) 2024-03-19T01:13:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890d400640 1 -- 172.21.15.69:0/2968204404 learned_addr learned my addr 172.21.15.69:0/2968204404 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890e800640 1 --2- 172.21.15.69:0/2968204404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 0x556d35a4b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890d400640 1 -- 172.21.15.69:0/2968204404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a48340 msgr2=0x556d35a50660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890d400640 1 --2- 172.21.15.69:0/2968204404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a48340 0x556d35a50660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890d400640 1 -- 172.21.15.69:0/2968204404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 msgr2=0x556d35a4b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890d400640 1 --2- 172.21.15.69:0/2968204404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 0x556d35a4b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890d400640 1 -- 172.21.15.69:0/2968204404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d35a48170 con 0x556d35a50f20 2024-03-19T01:13:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890de00640 1 --2- 172.21.15.69:0/2968204404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a48340 0x556d35a50660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890e800640 1 --2- 172.21.15.69:0/2968204404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 0x556d35a4b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:51.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.816+0000 7f890d400640 1 --2- 172.21.15.69:0/2968204404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 0x556d35a5c720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00bec0 tx=0x7f88fc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c4972a742e8402e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.817+0000 7f88fa600640 1 -- 172.21.15.69:0/2968204404 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc0130d0 con 0x556d35a50f20 2024-03-19T01:13:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.817+0000 7f88fa600640 1 -- 172.21.15.69:0/2968204404 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc013270 con 0x556d35a50f20 2024-03-19T01:13:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.817+0000 7f88fa600640 1 -- 172.21.15.69:0/2968204404 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc003ab0 con 0x556d35a50f20 2024-03-19T01:13:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.817+0000 7f8917816740 1 -- 172.21.15.69:0/2968204404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 msgr2=0x556d35a5c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.817+0000 7f8917816740 1 --2- 172.21.15.69:0/2968204404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 0x556d35a5c720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00bec0 tx=0x7f88fc0046c0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.817+0000 7f8917816740 1 -- 172.21.15.69:0/2968204404 shutdown_connections 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.817+0000 7f8917816740 1 --2- 172.21.15.69:0/2968204404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 0x556d35a4b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.817+0000 7f8917816740 1 --2- 172.21.15.69:0/2968204404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 0x556d35a5c720 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.817+0000 7f8917816740 1 --2- 172.21.15.69:0/2968204404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a48340 0x556d35a50660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.817+0000 7f8917816740 1 -- 172.21.15.69:0/2968204404 >> 172.21.15.69:0/2968204404 conn(0x556d35a42150 msgr2=0x556d35a411d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.818+0000 7f8917816740 1 -- 172.21.15.69:0/2968204404 shutdown_connections 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.818+0000 7f8917816740 1 -- 172.21.15.69:0/2968204404 wait complete. 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.818+0000 7f8917816740 1 -- start start 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.818+0000 7f8917816740 1 Processor -- start 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f8917816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a48340 0x556d35a58d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f8917816740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 0x556d35a55d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f8917816740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 0x556d35a56260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f8917816740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f89101b23c0 con 0x556d35a68be0 2024-03-19T01:13:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f8917816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89101b1cf0 con 0x556d35a48340 2024-03-19T01:13:51.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f8917816740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f89101aef30 con 0x556d35a50f20 2024-03-19T01:13:51.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f8917816740 10 client.0 _ll_register_callbacks cb 0x7f88fc00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:51.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f890e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 0x556d35a56260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f890e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 0x556d35a56260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48026/0 (socket says 172.21.15.69:48026) 2024-03-19T01:13:51.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f890e800640 1 -- 172.21.15.69:0/1532182403 learned_addr learned my addr 172.21.15.69:0/1532182403 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:51.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f890d400640 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 0x556d35a55d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.819+0000 7f890de00640 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a48340 0x556d35a58d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.820+0000 7f890e800640 1 -- 172.21.15.69:0/1532182403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 msgr2=0x556d35a55d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.820+0000 7f890e800640 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 0x556d35a55d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.820+0000 7f890e800640 1 -- 172.21.15.69:0/1532182403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a48340 msgr2=0x556d35a58d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.820+0000 7f890e800640 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a48340 0x556d35a58d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.820+0000 7f890e800640 1 -- 172.21.15.69:0/1532182403 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d35a567d0 con 0x556d35a68be0 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.820+0000 7f890e800640 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 0x556d35a56260 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f890800fbf0 tx=0x7f8908004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acbc1e4831e56b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.820+0000 7f88ef400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.820+0000 7f88ef400640 1 -- 172.21.15.69:0/1532182403 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890802e060 con 0x556d35a68be0 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.820+0000 7f88ef400640 1 -- 172.21.15.69:0/1532182403 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f8908029480 con 0x556d35a68be0 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.820+0000 7f8917816740 1 -- 172.21.15.69:0/1532182403 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88fc013270 con 0x556d35a68be0 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.821+0000 7f88ef400640 1 -- 172.21.15.69:0/1532182403 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8908035050 con 0x556d35a68be0 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.821+0000 7f8917816740 10 client.16326 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.821+0000 7f8917816740 1 -- 172.21.15.69:0/1532182403 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556d35ac7fe0 con 0x556d35a68be0 2024-03-19T01:13:51.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.821+0000 7f88ef400640 1 -- 172.21.15.69:0/1532182403 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc013270 con 0x556d35a68be0 2024-03-19T01:13:51.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.821+0000 7f88ef400640 1 -- 172.21.15.69:0/1532182403 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7f8908027680 con 0x556d35a68be0 2024-03-19T01:13:51.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.821+0000 7f8917816740 10 client.16326 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:51.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.821+0000 7f8917816740 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:51.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.821+0000 7f8917816740 20 client.16326 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:51.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.821+0000 7f8917816740 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x556d35ac8880 0x556d35acaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:51.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.821+0000 7f8917816740 4 client.16326 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:13:51.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.821+0000 7f8917816740 1 -- 172.21.15.69:0/1532182403 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d359b3c10 con 0x556d35ac8880 2024-03-19T01:13:51.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.822+0000 7f890de00640 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x556d35ac8880 0x556d35acaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:51.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.822+0000 7f88ef400640 10 client.16326 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:51.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.822+0000 7f890de00640 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x556d35ac8880 0x556d35acaca0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba82e258e3abc53b server_cookie=efd10c7ecbd661d7 in_seq=0 out_seq=0 2024-03-19T01:13:51.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.822+0000 7f88ef400640 1 -- 172.21.15.69:0/1532182403 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d359b3c10 con 0x556d35ac8880 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.822+0000 7f88ef400640 10 client.16326 handle_command_reply: tid=0 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.822+0000 7f8917816740 1 client.16326 shutdown 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f8917816740 1 -- 172.21.15.69:0/1532182403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 msgr2=0x556d35a56260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f8917816740 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 0x556d35a56260 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f890800fbf0 tx=0x7f8908004ab0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f8917816740 1 -- 172.21.15.69:0/1532182403 shutdown_connections 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f8917816740 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x556d35ac8880 0x556d35acaca0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f8917816740 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d35a68be0 0x556d35a56260 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f8917816740 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556d35a50f20 0x556d35a55d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f8917816740 1 --2- 172.21.15.69:0/1532182403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d35a48340 0x556d35a58d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f8917816740 1 -- 172.21.15.69:0/1532182403 >> 172.21.15.69:0/1532182403 conn(0x556d35a42150 msgr2=0x556d35a4ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f88ef400640 0 client.16326 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f8917816740 1 -- 172.21.15.69:0/1532182403 shutdown_connections 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f8917816740 1 -- 172.21.15.69:0/1532182403 wait complete. 2024-03-19T01:13:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.823+0000 7f8917816740 20 client.16326 trim_cache size 0 max 16384 2024-03-19T01:13:51.829 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:13:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 -- 172.21.15.69:0/1223470628 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88f40a29f0 msgr2=0x7f88f40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88f40a29f0 0x7f88f40a4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8908009b80 tx=0x7f8908004a40 comp rx=0 tx=0).stop 2024-03-19T01:13:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 -- 172.21.15.69:0/1223470628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89101039c0 msgr2=0x7f891019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89101039c0 0x7f891019a2c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f890002efd0 tx=0x7f8900004430 comp rx=0 tx=0).stop 2024-03-19T01:13:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 -- 172.21.15.69:0/1223470628 shutdown_connections 2024-03-19T01:13:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89101099d0 0x7f891019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8910104380 0x7f891019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88f40a29f0 0x7f88f40a4eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 --2- 172.21.15.69:0/1223470628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89101039c0 0x7f891019a2c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 -- 172.21.15.69:0/1223470628 >> 172.21.15.69:0/1223470628 conn(0x7f89100ff490 msgr2=0x7f89100ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 -- 172.21.15.69:0/1223470628 shutdown_connections 2024-03-19T01:13:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:51.824+0000 7f8914c00640 1 -- 172.21.15.69:0/1223470628 wait complete. 2024-03-19T01:13:52.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:52 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:13:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:52 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:13:52.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:52 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:13:52.843 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:13:52.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:13:52.844 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:13:53.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.061+0000 7fc57c000640 1 -- 172.21.15.69:0/3746864326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc574104380 msgr2=0x7fc574110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.061+0000 7fc57c000640 1 --2- 172.21.15.69:0/3746864326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc574104380 0x7fc574110890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009930 tx=0x7fc56c02bc20 comp rx=0 tx=0).stop 2024-03-19T01:13:53.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.061+0000 7fc57c000640 1 -- 172.21.15.69:0/3746864326 shutdown_connections 2024-03-19T01:13:53.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.061+0000 7fc57c000640 1 --2- 172.21.15.69:0/3746864326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc574104380 0x7fc574110890 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.061+0000 7fc57c000640 1 --2- 172.21.15.69:0/3746864326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5741039c0 0x7fc574103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.061+0000 7fc57c000640 1 --2- 172.21.15.69:0/3746864326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5741099d0 0x7fc574109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.061+0000 7fc57c000640 1 -- 172.21.15.69:0/3746864326 >> 172.21.15.69:0/3746864326 conn(0x7fc5740ff490 msgr2=0x7fc5741018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.061+0000 7fc57c000640 1 -- 172.21.15.69:0/3746864326 shutdown_connections 2024-03-19T01:13:53.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.061+0000 7fc57c000640 1 -- 172.21.15.69:0/3746864326 wait complete. 2024-03-19T01:13:53.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.063+0000 7fc57c000640 1 Processor -- start 2024-03-19T01:13:53.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.063+0000 7fc57c000640 1 -- start start 2024-03-19T01:13:53.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.063+0000 7fc57c000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5741039c0 0x7fc57419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57c000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc574104380 0x7fc57419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5741099d0 0x7fc57419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57c000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc574113e30 con 0x7fc5741039c0 2024-03-19T01:13:53.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc574113cb0 con 0x7fc5741099d0 2024-03-19T01:13:53.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57c000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc574113fb0 con 0x7fc574104380 2024-03-19T01:13:53.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc579600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5741039c0 0x7fc57419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc578c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc574104380 0x7fc57419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5741099d0 0x7fc57419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5741099d0 0x7fc57419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37582/0 (socket says 172.21.15.69:37582) 2024-03-19T01:13:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57a000640 1 -- 172.21.15.69:0/2054384802 learned_addr learned my addr 172.21.15.69:0/2054384802 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc578c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc574104380 0x7fc57419a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:37948/0 (socket says 172.21.15.69:37948) 2024-03-19T01:13:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57a000640 1 -- 172.21.15.69:0/2054384802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc574104380 msgr2=0x7fc57419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57a000640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc574104380 0x7fc57419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57a000640 1 -- 172.21.15.69:0/2054384802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5741039c0 msgr2=0x7fc57419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57a000640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5741039c0 0x7fc57419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc57a000640 1 -- 172.21.15.69:0/2054384802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57419f290 con 0x7fc5741099d0 2024-03-19T01:13:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.064+0000 7fc578c00640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc574104380 0x7fc57419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.065+0000 7fc57a000640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5741099d0 0x7fc57419eb60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc56c02c160 tx=0x7fc56c02c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.065+0000 7fc572000640 1 -- 172.21.15.69:0/2054384802 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c043020 con 0x7fc5741099d0 2024-03-19T01:13:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.065+0000 7fc57c000640 1 -- 172.21.15.69:0/2054384802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57419f520 con 0x7fc5741099d0 2024-03-19T01:13:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.065+0000 7fc572000640 1 -- 172.21.15.69:0/2054384802 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc56c005000 con 0x7fc5741099d0 2024-03-19T01:13:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.065+0000 7fc572000640 1 -- 172.21.15.69:0/2054384802 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c003af0 con 0x7fc5741099d0 2024-03-19T01:13:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.065+0000 7fc57c000640 1 -- 172.21.15.69:0/2054384802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5741a6e10 con 0x7fc5741099d0 2024-03-19T01:13:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.068+0000 7fc572000640 1 -- 172.21.15.69:0/2054384802 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fc56c046020 con 0x7fc5741099d0 2024-03-19T01:13:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.070+0000 7fc57ec4f740 1 Processor -- start 2024-03-19T01:13:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.070+0000 7fc57ec4f740 1 -- start start 2024-03-19T01:13:53.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.070+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216914b00 0x563216914ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.070+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563216915420 0x56321691dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.070+0000 7fc572000640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc55c0a29f0 0x7fc55c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.070+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321691e000 0x5632169203f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.070+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563216877fb0 con 0x563216915420 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.070+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563216820ab0 con 0x56321691e000 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.070+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563216874050 con 0x563216914b00 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc572000640 1 -- 172.21.15.69:0/2054384802 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc56c0f0ff0 con 0x7fc5741099d0 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc578c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216914b00 0x563216914ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc578c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563216915420 0x56321691dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc57a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321691e000 0x5632169203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc579600640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc55c0a29f0 0x7fc55c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc578c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563216915420 0x56321691dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48058/0 (socket says 172.21.15.69:48058) 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc578c00640 1 -- 172.21.15.69:0/1147639686 learned_addr learned my addr 172.21.15.69:0/1147639686 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc57a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321691e000 0x5632169203f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37594/0 (socket says 172.21.15.69:37594) 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc57a000640 1 -- 172.21.15.69:0/1147639686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216914b00 msgr2=0x563216914ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc57a000640 1 --2- 172.21.15.69:0/1147639686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216914b00 0x563216914ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc57a000640 1 -- 172.21.15.69:0/1147639686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563216915420 msgr2=0x56321691dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc57a000640 1 --2- 172.21.15.69:0/1147639686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563216915420 0x56321691dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc57a000640 1 -- 172.21.15.69:0/1147639686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321683ee00 con 0x56321691e000 2024-03-19T01:13:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc57a000640 1 --2- 172.21.15.69:0/1147639686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321691e000 0x5632169203f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0ee470 tx=0x7fc56c0bcf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c48e79d5bd6a79 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.071+0000 7fc579600640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc55c0a29f0 0x7fc55c0a4eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc568005fd0 tx=0x7fc568009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.072+0000 7fc566600640 1 -- 172.21.15.69:0/1147639686 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c03b940 con 0x56321691e000 2024-03-19T01:13:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.072+0000 7fc566600640 1 -- 172.21.15.69:0/1147639686 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc56c03bb80 con 0x56321691e000 2024-03-19T01:13:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.072+0000 7fc566600640 1 -- 172.21.15.69:0/1147639686 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c03bdc0 con 0x56321691e000 2024-03-19T01:13:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.072+0000 7fc57ec4f740 1 -- 172.21.15.69:0/1147639686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321691e000 msgr2=0x5632169203f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.072+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/1147639686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321691e000 0x5632169203f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0ee470 tx=0x7fc56c0bcf80 comp rx=0 tx=0).stop 2024-03-19T01:13:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.072+0000 7fc57ec4f740 1 -- 172.21.15.69:0/1147639686 shutdown_connections 2024-03-19T01:13:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.072+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/1147639686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56321691e000 0x5632169203f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.072+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/1147639686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563216915420 0x56321691dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.072+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/1147639686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216914b00 0x563216914ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.072+0000 7fc57ec4f740 1 -- 172.21.15.69:0/1147639686 >> 172.21.15.69:0/1147639686 conn(0x56321690f230 msgr2=0x56321690f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.072+0000 7fc57ec4f740 1 -- 172.21.15.69:0/1147639686 shutdown_connections 2024-03-19T01:13:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.073+0000 7fc57ec4f740 1 -- 172.21.15.69:0/1147639686 wait complete. 2024-03-19T01:13:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.073+0000 7fc57ec4f740 1 -- start start 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.073+0000 7fc57ec4f740 1 Processor -- start 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x56321691b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x56321691b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x56321691ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563216920bb0 con 0x56321691e000 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563216833f00 con 0x563216914b00 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563216920d30 con 0x563216915420 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc57ec4f740 10 client.0 _ll_register_callbacks cb 0x5632168338d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc579600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x56321691b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc579600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x56321691b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37598/0 (socket says 172.21.15.69:37598) 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc579600640 1 -- 172.21.15.69:0/3127783364 learned_addr learned my addr 172.21.15.69:0/3127783364 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc57a000640 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x56321691ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.074+0000 7fc578c00640 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x56321691b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.075+0000 7fc579600640 1 -- 172.21.15.69:0/3127783364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 msgr2=0x56321691b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.075+0000 7fc579600640 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x56321691b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.075+0000 7fc579600640 1 -- 172.21.15.69:0/3127783364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 msgr2=0x56321691ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.075+0000 7fc579600640 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x56321691ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.075+0000 7fc579600640 1 -- 172.21.15.69:0/3127783364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632166547e0 con 0x563216914b00 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.075+0000 7fc579600640 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x56321691b010 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc568004280 tx=0x7fc568004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ff463132b909c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.075+0000 7fc55aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.075+0000 7fc55aa00640 1 -- 172.21.15.69:0/3127783364 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568004800 con 0x563216914b00 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.076+0000 7fc55aa00640 1 -- 172.21.15.69:0/3127783364 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc56800eac0 con 0x563216914b00 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.076+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3127783364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc56c03bb80 con 0x563216914b00 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.076+0000 7fc55aa00640 1 -- 172.21.15.69:0/3127783364 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc56802f050 con 0x563216914b00 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.076+0000 7fc57ec4f740 10 client.26543 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.076+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3127783364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5632166b4f70 con 0x563216914b00 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.076+0000 7fc55aa00640 1 -- 172.21.15.69:0/3127783364 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc56c03bb80 con 0x563216914b00 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.076+0000 7fc55aa00640 1 -- 172.21.15.69:0/3127783364 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fc56801c3b0 con 0x563216914b00 2024-03-19T01:13:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.076+0000 7fc57ec4f740 10 client.26543 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.076+0000 7fc57ec4f740 10 client.26543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.077+0000 7fc57ec4f740 20 client.26543 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.077+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56321692e130 0x563216930540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.077+0000 7fc57ec4f740 4 client.26543 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.077+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3127783364 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632168854f0 con 0x56321692e130 2024-03-19T01:13:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.077+0000 7fc578c00640 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56321692e130 0x563216930540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.077+0000 7fc578c00640 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56321692e130 0x563216930540 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be719e7330efb86d server_cookie=cf3bc740fe4140d9 in_seq=0 out_seq=0 2024-03-19T01:13:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.077+0000 7fc55aa00640 10 client.26543 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.078+0000 7fc55aa00640 1 -- 172.21.15.69:0/3127783364 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5632168854f0 con 0x56321692e130 2024-03-19T01:13:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.078+0000 7fc55aa00640 10 client.26543 handle_command_reply: tid=0 2024-03-19T01:13:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.079+0000 7fc57ec4f740 1 client.26543 shutdown 2024-03-19T01:13:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.079+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3127783364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 msgr2=0x56321691b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.079+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x56321691b010 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc568004280 tx=0x7fc568004ab0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.079+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3127783364 shutdown_connections 2024-03-19T01:13:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.079+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56321692e130 0x563216930540 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.079+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x56321691ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.079+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x56321691b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.079+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3127783364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x56321691b010 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.079+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3127783364 >> 172.21.15.69:0/3127783364 conn(0x56321690f230 msgr2=0x563216910f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.080+0000 7fc55aa00640 0 client.26543 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.080+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3127783364 shutdown_connections 2024-03-19T01:13:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.080+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3127783364 wait complete. 2024-03-19T01:13:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.080+0000 7fc57ec4f740 20 client.26543 trim_cache size 0 max 16384 2024-03-19T01:13:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.099+0000 7fc57ec4f740 1 Processor -- start 2024-03-19T01:13:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.099+0000 7fc57ec4f740 1 -- start start 2024-03-19T01:13:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.099+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216929830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.099+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x563216935e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.099+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x5632169182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.099+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b0f00 con 0x56321691e000 2024-03-19T01:13:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.099+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b2b00 con 0x563216914b00 2024-03-19T01:13:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.099+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b15d0 con 0x563216915420 2024-03-19T01:13:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc579600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216929830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc579600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216929830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37608/0 (socket says 172.21.15.69:37608) 2024-03-19T01:13:53.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc579600640 1 -- 172.21.15.69:0/3149590393 learned_addr learned my addr 172.21.15.69:0/3149590393 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:53.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc57a000640 1 --2- 172.21.15.69:0/3149590393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x5632169182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc578c00640 1 --2- 172.21.15.69:0/3149590393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x563216935e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc579600640 1 -- 172.21.15.69:0/3149590393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 msgr2=0x563216935e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc579600640 1 --2- 172.21.15.69:0/3149590393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x563216935e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc579600640 1 -- 172.21.15.69:0/3149590393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 msgr2=0x5632169182d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc579600640 1 --2- 172.21.15.69:0/3149590393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x5632169182d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc579600640 1 -- 172.21.15.69:0/3149590393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321654c170 con 0x563216914b00 2024-03-19T01:13:53.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc57a000640 1 --2- 172.21.15.69:0/3149590393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x5632169182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:53.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc578c00640 1 --2- 172.21.15.69:0/3149590393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x563216935e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:53.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.100+0000 7fc579600640 1 --2- 172.21.15.69:0/3149590393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216929830 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc568009810 tx=0x7fc568003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=132382f63448540a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.101+0000 7fc565c00640 1 -- 172.21.15.69:0/3149590393 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568039030 con 0x563216914b00 2024-03-19T01:13:53.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.101+0000 7fc565c00640 1 -- 172.21.15.69:0/3149590393 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc568004800 con 0x563216914b00 2024-03-19T01:13:53.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.101+0000 7fc565c00640 1 -- 172.21.15.69:0/3149590393 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56801cdf0 con 0x563216914b00 2024-03-19T01:13:53.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.102+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3149590393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 msgr2=0x563216929830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.102+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3149590393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216929830 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc568009810 tx=0x7fc568003e30 comp rx=0 tx=0).stop 2024-03-19T01:13:53.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.102+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3149590393 shutdown_connections 2024-03-19T01:13:53.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.102+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3149590393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x5632169182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.102+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3149590393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x563216935e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.102+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3149590393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216929830 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.102+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3149590393 >> 172.21.15.69:0/3149590393 conn(0x56321690f230 msgr2=0x563216910f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.102+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3149590393 shutdown_connections 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.102+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3149590393 wait complete. 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.102+0000 7fc57ec4f740 1 -- start start 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.103+0000 7fc57ec4f740 1 Processor -- start 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.103+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216926120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.103+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x563216922d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.103+0000 7fc57ec4f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x5632169232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.103+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b2370 con 0x56321691e000 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.103+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b1ca0 con 0x563216914b00 2024-03-19T01:13:53.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.103+0000 7fc57ec4f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741aeee0 con 0x563216915420 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.103+0000 7fc57ec4f740 10 client.0 _ll_register_callbacks cb 0x7fc568003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc579600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216926120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc579600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216926120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37614/0 (socket says 172.21.15.69:37614) 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc579600640 1 -- 172.21.15.69:0/3522525488 learned_addr learned my addr 172.21.15.69:0/3522525488 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc578c00640 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x563216922d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc57a000640 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x5632169232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc579600640 1 -- 172.21.15.69:0/3522525488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 msgr2=0x563216922d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc579600640 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x563216922d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc579600640 1 -- 172.21.15.69:0/3522525488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 msgr2=0x5632169232d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc579600640 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x5632169232d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc579600640 1 -- 172.21.15.69:0/3522525488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563216923ad0 con 0x563216914b00 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc57a000640 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x5632169232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc579600640 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216926120 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc568004020 tx=0x7fc56801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac7bb0f9278c3f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc55aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:13:53.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc55aa00640 1 -- 172.21.15.69:0/3522525488 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568043060 con 0x563216914b00 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.104+0000 7fc55aa00640 1 -- 172.21.15.69:0/3522525488 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fc568009040 con 0x563216914b00 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3522525488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc568004800 con 0x563216914b00 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc55aa00640 1 -- 172.21.15.69:0/3522525488 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc56801ce80 con 0x563216914b00 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc57ec4f740 10 client.26549 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3522525488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563216995500 con 0x563216914b00 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc55aa00640 1 -- 172.21.15.69:0/3522525488 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc568004800 con 0x563216914b00 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc55aa00640 1 -- 172.21.15.69:0/3522525488 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fc568033430 con 0x563216914b00 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc57ec4f740 10 client.26549 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc57ec4f740 10 client.26549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc57ec4f740 20 client.26549 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563216995b60 0x563216997f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc57ec4f740 4 client.26549 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:13:53.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.105+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3522525488 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56321687f840 con 0x563216995b60 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.106+0000 7fc578c00640 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563216995b60 0x563216997f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.106+0000 7fc55aa00640 10 client.26549 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.106+0000 7fc578c00640 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563216995b60 0x563216997f80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc359dd75747b50a server_cookie=e32f27134ca3bfa9 in_seq=0 out_seq=0 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.107+0000 7fc55aa00640 1 -- 172.21.15.69:0/3522525488 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56321687f840 con 0x563216995b60 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.107+0000 7fc55aa00640 10 client.26549 handle_command_reply: tid=0 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.107+0000 7fc57ec4f740 1 client.26549 shutdown 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3522525488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 msgr2=0x563216926120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216926120 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc568004020 tx=0x7fc56801c3b0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3522525488 shutdown_connections 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563216995b60 0x563216997f80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56321691e000 0x5632169232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563216915420 0x563216922d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc55aa00640 0 client.26549 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:53.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc57ec4f740 1 --2- 172.21.15.69:0/3522525488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563216914b00 0x563216926120 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3522525488 >> 172.21.15.69:0/3522525488 conn(0x56321690f230 msgr2=0x56321691af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3522525488 shutdown_connections 2024-03-19T01:13:53.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc57ec4f740 1 -- 172.21.15.69:0/3522525488 wait complete. 2024-03-19T01:13:53.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.108+0000 7fc57ec4f740 20 client.26549 trim_cache size 0 max 16384 2024-03-19T01:13:53.112 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:13:53.112 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:13:53.112 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "50ca2af7-6cba-4162-8735-f8a498363dbf", 2024-03-19T01:13:53.112 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:13:53.112 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:13:53.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.109+0000 7fc57c000640 1 -- 172.21.15.69:0/2054384802 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc55c0a29f0 msgr2=0x7fc55c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.109+0000 7fc57c000640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc55c0a29f0 0x7fc55c0a4eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc568005fd0 tx=0x7fc568009500 comp rx=0 tx=0).stop 2024-03-19T01:13:53.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.109+0000 7fc57c000640 1 -- 172.21.15.69:0/2054384802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5741099d0 msgr2=0x7fc57419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.109+0000 7fc57c000640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5741099d0 0x7fc57419eb60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc56c02c160 tx=0x7fc56c02c7d0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.109+0000 7fc57c000640 1 -- 172.21.15.69:0/2054384802 shutdown_connections 2024-03-19T01:13:53.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.109+0000 7fc57c000640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc55c0a29f0 0x7fc55c0a4eb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.109+0000 7fc57c000640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5741099d0 0x7fc57419eb60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.109+0000 7fc57c000640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc574104380 0x7fc57419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.109+0000 7fc57c000640 1 --2- 172.21.15.69:0/2054384802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5741039c0 0x7fc57419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.109+0000 7fc57c000640 1 -- 172.21.15.69:0/2054384802 >> 172.21.15.69:0/2054384802 conn(0x7fc5740ff490 msgr2=0x7fc5740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.109+0000 7fc57c000640 1 -- 172.21.15.69:0/2054384802 shutdown_connections 2024-03-19T01:13:53.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.110+0000 7fc57c000640 1 -- 172.21.15.69:0/2054384802 wait complete. 2024-03-19T01:13:53.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:13:53.137 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:13:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.413+0000 7fbf63e00640 1 -- 172.21.15.69:0/1530804421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf64073bf0 msgr2=0x7fbf64073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.413+0000 7fbf63e00640 1 --2- 172.21.15.69:0/1530804421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf64073bf0 0x7fbf64073fd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009900 tx=0x7fbf5402bcd0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.413+0000 7fbf63e00640 1 -- 172.21.15.69:0/1530804421 shutdown_connections 2024-03-19T01:13:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.413+0000 7fbf63e00640 1 --2- 172.21.15.69:0/1530804421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf6410df30 0x7fbf64110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.413+0000 7fbf63e00640 1 --2- 172.21.15.69:0/1530804421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf64074510 0x7fbf6410d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.413+0000 7fbf63e00640 1 --2- 172.21.15.69:0/1530804421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf64073bf0 0x7fbf64073fd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.413+0000 7fbf63e00640 1 -- 172.21.15.69:0/1530804421 >> 172.21.15.69:0/1530804421 conn(0x7fbf640fd1c0 msgr2=0x7fbf640ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.413+0000 7fbf63e00640 1 -- 172.21.15.69:0/1530804421 shutdown_connections 2024-03-19T01:13:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.413+0000 7fbf63e00640 1 -- 172.21.15.69:0/1530804421 wait complete. 2024-03-19T01:13:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.414+0000 7fbf63e00640 1 Processor -- start 2024-03-19T01:13:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.415+0000 7fbf63e00640 1 -- start start 2024-03-19T01:13:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.415+0000 7fbf63e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf64073bf0 0x7fbf6419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.415+0000 7fbf63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf64074510 0x7fbf6419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.415+0000 7fbf63e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf6410df30 0x7fbf6419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.415+0000 7fbf63e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64113ce0 con 0x7fbf6410df30 2024-03-19T01:13:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.415+0000 7fbf63e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64113b60 con 0x7fbf64074510 2024-03-19T01:13:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.415+0000 7fbf63e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64113e60 con 0x7fbf64073bf0 2024-03-19T01:13:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.415+0000 7fbf62a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf64073bf0 0x7fbf6419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.415+0000 7fbf63400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf6410df30 0x7fbf6419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.415+0000 7fbf62a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf64073bf0 0x7fbf6419a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38000/0 (socket says 172.21.15.69:38000) 2024-03-19T01:13:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.415+0000 7fbf62a00640 1 -- 172.21.15.69:0/506968587 learned_addr learned my addr 172.21.15.69:0/506968587 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:53.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.416+0000 7fbf62000640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf64074510 0x7fbf6419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.416+0000 7fbf62a00640 1 -- 172.21.15.69:0/506968587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf64074510 msgr2=0x7fbf6419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.416+0000 7fbf62a00640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf64074510 0x7fbf6419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.416+0000 7fbf62a00640 1 -- 172.21.15.69:0/506968587 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf6410df30 msgr2=0x7fbf6419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.416+0000 7fbf62a00640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf6410df30 0x7fbf6419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.416+0000 7fbf62a00640 1 -- 172.21.15.69:0/506968587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6419f160 con 0x7fbf64073bf0 2024-03-19T01:13:53.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.416+0000 7fbf62a00640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf64073bf0 0x7fbf6419a1a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbf5402c2e0 tx=0x7fbf540043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.417+0000 7fbf5b400640 1 -- 172.21.15.69:0/506968587 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54041020 con 0x7fbf64073bf0 2024-03-19T01:13:53.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.417+0000 7fbf5b400640 1 -- 172.21.15.69:0/506968587 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf540054e0 con 0x7fbf64073bf0 2024-03-19T01:13:53.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.417+0000 7fbf5b400640 1 -- 172.21.15.69:0/506968587 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54003b20 con 0x7fbf64073bf0 2024-03-19T01:13:53.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.417+0000 7fbf63e00640 1 -- 172.21.15.69:0/506968587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6419f3f0 con 0x7fbf64073bf0 2024-03-19T01:13:53.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.417+0000 7fbf63e00640 1 -- 172.21.15.69:0/506968587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6419f890 con 0x7fbf64073bf0 2024-03-19T01:13:53.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.421+0000 7fbf5b400640 1 -- 172.21.15.69:0/506968587 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fbf54044020 con 0x7fbf64073bf0 2024-03-19T01:13:53.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.424+0000 7fbf5b400640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf480a29f0 0x7fbf480a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.424+0000 7fbf5b400640 1 -- 172.21.15.69:0/506968587 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fbf540f01b0 con 0x7fbf64073bf0 2024-03-19T01:13:53.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.425+0000 7fbf62000640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf480a29f0 0x7fbf480a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.426+0000 7fbf62000640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf480a29f0 0x7fbf480a4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbf6410f200 tx=0x7fbf50005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.428+0000 7fbf6c5ca740 1 Processor -- start 2024-03-19T01:13:53.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.428+0000 7fbf6c5ca740 1 -- start start 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.428+0000 7fbf6c5ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfc4f30 0x55756bfc5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.428+0000 7fbf6c5ca740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfc5850 0x55756bfcdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.428+0000 7fbf6c5ca740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfce430 0x55756bfd0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.428+0000 7fbf6c5ca740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55756bf292c0 con 0x55756bfc5850 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.428+0000 7fbf6c5ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55756bed1b60 con 0x55756bfc4f30 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.428+0000 7fbf6c5ca740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55756bf25360 con 0x55756bfce430 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.429+0000 7fbf62000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfc5850 0x55756bfcdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.429+0000 7fbf62000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfc5850 0x55756bfcdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48098/0 (socket says 172.21.15.69:48098) 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.429+0000 7fbf62000640 1 -- 172.21.15.69:0/2046442556 learned_addr learned my addr 172.21.15.69:0/2046442556 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.429+0000 7fbf62000640 1 -- 172.21.15.69:0/2046442556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfce430 msgr2=0x55756bfd0820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.429+0000 7fbf62000640 1 --2- 172.21.15.69:0/2046442556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfce430 0x55756bfd0820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.429+0000 7fbf62000640 1 -- 172.21.15.69:0/2046442556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfc4f30 msgr2=0x55756bfc5310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:13:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.429+0000 7fbf62000640 1 --2- 172.21.15.69:0/2046442556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfc4f30 0x55756bfc5310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.429+0000 7fbf62000640 1 -- 172.21.15.69:0/2046442556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756bd077d0 con 0x55756bfc5850 2024-03-19T01:13:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.430+0000 7fbf62000640 1 --2- 172.21.15.69:0/2046442556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfc5850 0x55756bfcdef0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbf50010dd0 tx=0x7fbf50013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2db1ca67325f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.430+0000 7fbf4fe00640 1 -- 172.21.15.69:0/2046442556 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50023070 con 0x55756bfc5850 2024-03-19T01:13:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.431+0000 7fbf4fe00640 1 -- 172.21.15.69:0/2046442556 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf5001f930 con 0x55756bfc5850 2024-03-19T01:13:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.431+0000 7fbf4fe00640 1 -- 172.21.15.69:0/2046442556 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50011980 con 0x55756bfc5850 2024-03-19T01:13:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.432+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2046442556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfc5850 msgr2=0x55756bfcdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.432+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2046442556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfc5850 0x55756bfcdef0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbf50010dd0 tx=0x7fbf50013000 comp rx=0 tx=0).stop 2024-03-19T01:13:53.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.433+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2046442556 shutdown_connections 2024-03-19T01:13:53.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.433+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2046442556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfce430 0x55756bfd0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.433+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2046442556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfc5850 0x55756bfcdef0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.433+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2046442556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfc4f30 0x55756bfc5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.433+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2046442556 >> 172.21.15.69:0/2046442556 conn(0x55756bfbf660 msgr2=0x55756bfbfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.434+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2046442556 shutdown_connections 2024-03-19T01:13:53.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.434+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2046442556 wait complete. 2024-03-19T01:13:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.434+0000 7fbf6c5ca740 1 -- start start 2024-03-19T01:13:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.435+0000 7fbf6c5ca740 1 Processor -- start 2024-03-19T01:13:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.436+0000 7fbf6c5ca740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc4f30 0x55756bfcb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.437+0000 7fbf62a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc4f30 0x55756bfcb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.437+0000 7fbf62a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc4f30 0x55756bfcb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38022/0 (socket says 172.21.15.69:38022) 2024-03-19T01:13:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.437+0000 7fbf62a00640 1 -- 172.21.15.69:0/2941503575 learned_addr learned my addr 172.21.15.69:0/2941503575 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.437+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfce430 0x55756bfcbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.437+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfcc120 0x55756bfe2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.437+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2941503575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55756bfd9a30 con 0x55756bfce430 2024-03-19T01:13:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.437+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2941503575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55756bee4fc0 con 0x55756bfcc120 2024-03-19T01:13:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.437+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2941503575 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55756bfd9bb0 con 0x55756bfc4f30 2024-03-19T01:13:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.438+0000 7fbf63400640 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfcc120 0x55756bfe2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.438+0000 7fbf62a00640 1 -- 172.21.15.69:0/2941503575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfcc120 msgr2=0x55756bfe2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.438+0000 7fbf62a00640 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfcc120 0x55756bfe2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.438+0000 7fbf62a00640 1 -- 172.21.15.69:0/2941503575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfce430 msgr2=0x55756bfcbb90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:13:53.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.438+0000 7fbf62a00640 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfce430 0x55756bfcbb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.438+0000 7fbf6c5ca740 10 client.0 _ll_register_callbacks cb 0x55756bee4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:53.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.438+0000 7fbf62a00640 1 -- 172.21.15.69:0/2941503575 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756bc3d3f0 con 0x55756bfc4f30 2024-03-19T01:13:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.439+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2941503575 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf5001f930 con 0x55756bfc4f30 2024-03-19T01:13:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.439+0000 7fbf62a00640 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc4f30 0x55756bfcb650 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbf540b8ed0 tx=0x7fbf540bc100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621e9e4787da040f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.439+0000 7fbf4cc00640 10 client.26542 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.439+0000 7fbf4cc00640 1 -- 172.21.15.69:0/2941503575 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54004c80 con 0x55756bfc4f30 2024-03-19T01:13:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.440+0000 7fbf4cc00640 1 -- 172.21.15.69:0/2941503575 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fbf5402cc30 con 0x55756bfc4f30 2024-03-19T01:13:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.440+0000 7fbf4cc00640 1 -- 172.21.15.69:0/2941503575 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbf540bcc00 con 0x55756bfc4f30 2024-03-19T01:13:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.440+0000 7fbf6c5ca740 10 client.26542 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.440+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2941503575 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55756bbfd0f0 con 0x55756bfc4f30 2024-03-19T01:13:53.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.440+0000 7fbf4cc00640 1 -- 172.21.15.69:0/2941503575 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fbf540d5230 con 0x55756bfc4f30 2024-03-19T01:13:53.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.440+0000 7fbf6c5ca740 10 client.26542 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:53.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.440+0000 7fbf6c5ca740 10 client.26542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:53.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.440+0000 7fbf6c5ca740 20 client.26542 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:53.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.441+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55756bfe7470 0x55756bfe9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.441+0000 7fbf6c5ca740 4 client.26542 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:13:53.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.441+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2941503575 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55756beca610 con 0x55756bfe7470 2024-03-19T01:13:53.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.442+0000 7fbf62000640 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55756bfe7470 0x55756bfe9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.444+0000 7fbf4cc00640 10 client.26542 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:53.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.444+0000 7fbf62000640 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55756bfe7470 0x55756bfe9910 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a5f8b93e40d6e43 server_cookie=8f697745ec87aae2 in_seq=0 out_seq=0 2024-03-19T01:13:53.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.447+0000 7fbf4cc00640 1 -- 172.21.15.69:0/2941503575 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55756beca610 con 0x55756bfe7470 2024-03-19T01:13:53.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.447+0000 7fbf4cc00640 10 client.26542 handle_command_reply: tid=0 2024-03-19T01:13:53.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.447+0000 7fbf6c5ca740 1 client.26542 shutdown 2024-03-19T01:13:53.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2941503575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc4f30 msgr2=0x55756bfcb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc4f30 0x55756bfcb650 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbf540b8ed0 tx=0x7fbf540bc100 comp rx=0 tx=0).stop 2024-03-19T01:13:53.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2941503575 shutdown_connections 2024-03-19T01:13:53.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55756bfe7470 0x55756bfe9910 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfcc120 0x55756bfe2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfce430 0x55756bfcbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/2941503575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc4f30 0x55756bfcb650 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2941503575 >> 172.21.15.69:0/2941503575 conn(0x55756bfbf660 msgr2=0x55756bfc13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf4cc00640 0 client.26542 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:53.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2941503575 shutdown_connections 2024-03-19T01:13:53.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/2941503575 wait complete. 2024-03-19T01:13:53.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.448+0000 7fbf6c5ca740 20 client.26542 trim_cache size 0 max 16384 2024-03-19T01:13:53.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.491+0000 7fbf6c5ca740 1 Processor -- start 2024-03-19T01:13:53.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.491+0000 7fbf6c5ca740 1 -- start start 2024-03-19T01:13:53.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.492+0000 7fbf6c5ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfc4f30 0x55756bfef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.492+0000 7fbf6c5ca740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfcc120 0x55756bfe27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.492+0000 7fbf6c5ca740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfce430 0x55756bfe2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.492+0000 7fbf6c5ca740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64070e90 con 0x55756bfcc120 2024-03-19T01:13:53.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.492+0000 7fbf6c5ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64072a90 con 0x55756bfc4f30 2024-03-19T01:13:53.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.492+0000 7fbf6c5ca740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64071560 con 0x55756bfce430 2024-03-19T01:13:53.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.492+0000 7fbf62a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfc4f30 0x55756bfef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.492+0000 7fbf62a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfc4f30 0x55756bfef220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37656/0 (socket says 172.21.15.69:37656) 2024-03-19T01:13:53.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.492+0000 7fbf62a00640 1 -- 172.21.15.69:0/4223013092 learned_addr learned my addr 172.21.15.69:0/4223013092 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:53.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.492+0000 7fbf63400640 1 --2- 172.21.15.69:0/4223013092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfce430 0x55756bfe2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.492+0000 7fbf62000640 1 --2- 172.21.15.69:0/4223013092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfcc120 0x55756bfe27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.493+0000 7fbf62a00640 1 -- 172.21.15.69:0/4223013092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfce430 msgr2=0x55756bfe2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.493+0000 7fbf62a00640 1 --2- 172.21.15.69:0/4223013092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfce430 0x55756bfe2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.493+0000 7fbf62a00640 1 -- 172.21.15.69:0/4223013092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfcc120 msgr2=0x55756bfe27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.493+0000 7fbf62a00640 1 --2- 172.21.15.69:0/4223013092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfcc120 0x55756bfe27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.493+0000 7fbf62a00640 1 -- 172.21.15.69:0/4223013092 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756bfd0b50 con 0x55756bfc4f30 2024-03-19T01:13:53.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.493+0000 7fbf63400640 1 --2- 172.21.15.69:0/4223013092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfce430 0x55756bfe2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:13:53.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.493+0000 7fbf62a00640 1 --2- 172.21.15.69:0/4223013092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfc4f30 0x55756bfef220 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbf540c4770 tx=0x7fbf540ef000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9deea19043aa12d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.493+0000 7fbf4f400640 1 -- 172.21.15.69:0/4223013092 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54004c80 con 0x55756bfc4f30 2024-03-19T01:13:53.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.494+0000 7fbf4f400640 1 -- 172.21.15.69:0/4223013092 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf5001f930 con 0x55756bfc4f30 2024-03-19T01:13:53.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.495+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/4223013092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfc4f30 msgr2=0x55756bfef220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.495+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/4223013092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfc4f30 0x55756bfef220 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbf540c4770 tx=0x7fbf540ef000 comp rx=0 tx=0).stop 2024-03-19T01:13:53.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.495+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/4223013092 shutdown_connections 2024-03-19T01:13:53.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.495+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/4223013092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfce430 0x55756bfe2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.495+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/4223013092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfcc120 0x55756bfe27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.495+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/4223013092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfc4f30 0x55756bfef220 secure :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbf540c4770 tx=0x7fbf540ef000 comp rx=0 tx=0).stop 2024-03-19T01:13:53.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.495+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/4223013092 >> 172.21.15.69:0/4223013092 conn(0x55756bfbf660 msgr2=0x55756bfc9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.495+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/4223013092 shutdown_connections 2024-03-19T01:13:53.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.495+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/4223013092 wait complete. 2024-03-19T01:13:53.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.496+0000 7fbf6c5ca740 1 -- start start 2024-03-19T01:13:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.496+0000 7fbf6c5ca740 1 Processor -- start 2024-03-19T01:13:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.497+0000 7fbf6c5ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfcc120 0x55756bfc8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.497+0000 7fbf6c5ca740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfce430 0x55756bfc8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.497+0000 7fbf6c5ca740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc9160 0x55756c04c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.497+0000 7fbf6c5ca740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64072300 con 0x55756bfce430 2024-03-19T01:13:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.497+0000 7fbf6c5ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64071c30 con 0x55756bfcc120 2024-03-19T01:13:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.497+0000 7fbf6c5ca740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6406ee70 con 0x55756bfc9160 2024-03-19T01:13:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.497+0000 7fbf6c5ca740 10 client.0 _ll_register_callbacks cb 0x7fbf540bc2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:13:53.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.497+0000 7fbf62000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfce430 0x55756bfc8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.497+0000 7fbf62000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfce430 0x55756bfc8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48106/0 (socket says 172.21.15.69:48106) 2024-03-19T01:13:53.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.497+0000 7fbf62000640 1 -- 172.21.15.69:0/1605646751 learned_addr learned my addr 172.21.15.69:0/1605646751 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:13:53.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.498+0000 7fbf63400640 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc9160 0x55756c04c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.498+0000 7fbf62a00640 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfcc120 0x55756bfc8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.499+0000 7fbf63400640 1 -- 172.21.15.69:0/1605646751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfcc120 msgr2=0x55756bfc8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.499+0000 7fbf63400640 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfcc120 0x55756bfc8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.499+0000 7fbf63400640 1 -- 172.21.15.69:0/1605646751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfce430 msgr2=0x55756bfc8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:13:53.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.499+0000 7fbf63400640 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfce430 0x55756bfc8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.499+0000 7fbf63400640 1 -- 172.21.15.69:0/1605646751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756c04cfe0 con 0x55756bfc9160 2024-03-19T01:13:53.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.499+0000 7fbf62000640 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfce430 0x55756bfc8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:13:53.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.500+0000 7fbf63400640 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc9160 0x55756c04c850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55756bfc07f0 tx=0x7fbf5c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d551d83f0b542a83 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:13:53.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.500+0000 7fbf4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:13:53.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.500+0000 7fbf4cc00640 1 -- 172.21.15.69:0/1605646751 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c003b80 con 0x55756bfc9160 2024-03-19T01:13:53.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.500+0000 7fbf4cc00640 1 -- 172.21.15.69:0/1605646751 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fbf5c003d20 con 0x55756bfc9160 2024-03-19T01:13:53.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.500+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/1605646751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf5001f930 con 0x55756bfc9160 2024-03-19T01:13:53.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.501+0000 7fbf4cc00640 1 -- 172.21.15.69:0/1605646751 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbf5c0108e0 con 0x55756bfc9160 2024-03-19T01:13:53.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.502+0000 7fbf6c5ca740 10 client.26548 fetch_fsmap learned FSMap version 33 2024-03-19T01:13:53.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.502+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/1605646751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55756c04e350 con 0x55756bfc9160 2024-03-19T01:13:53.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.502+0000 7fbf4cc00640 1 -- 172.21.15.69:0/1605646751 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf5001f930 con 0x55756bfc9160 2024-03-19T01:13:53.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.502+0000 7fbf4cc00640 1 -- 172.21.15.69:0/1605646751 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 33) ==== 8394+0+0 (secure 0 0 0) 0x7fbf5c026470 con 0x55756bfc9160 2024-03-19T01:13:53.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.503+0000 7fbf6c5ca740 10 client.26548 fetch_fsmap finished waiting for FSMap version 33 2024-03-19T01:13:53.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.503+0000 7fbf6c5ca740 10 client.26548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:13:53.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.503+0000 7fbf6c5ca740 20 client.26548 populate_metadata read hostname 'smithi069' 2024-03-19T01:13:53.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.503+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55756c04eb00 0x55756c050f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:13:53.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.503+0000 7fbf6c5ca740 4 client.26548 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:13:53.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.503+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/1605646751 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55756bf36800 con 0x55756c04eb00 2024-03-19T01:13:53.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.503+0000 7fbf62a00640 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55756c04eb00 0x55756c050f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:13:53.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.504+0000 7fbf4cc00640 10 client.26548 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:53.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.504+0000 7fbf62a00640 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55756c04eb00 0x55756c050f20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b412ec13bc493faf server_cookie=2e72e335b77ac0e8 in_seq=0 out_seq=0 2024-03-19T01:13:53.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.508+0000 7fbf4cc00640 1 -- 172.21.15.69:0/1605646751 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55756bf36800 con 0x55756c04eb00 2024-03-19T01:13:53.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.508+0000 7fbf4cc00640 10 client.26548 handle_command_reply: tid=0 2024-03-19T01:13:53.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.508+0000 7fbf6c5ca740 1 client.26548 shutdown 2024-03-19T01:13:53.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.510+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/1605646751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc9160 msgr2=0x55756c04c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.510+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc9160 0x55756c04c850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55756bfc07f0 tx=0x7fbf5c012b60 comp rx=0 tx=0).stop 2024-03-19T01:13:53.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.510+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/1605646751 shutdown_connections 2024-03-19T01:13:53.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.510+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55756c04eb00 0x55756c050f20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.510+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55756bfc9160 0x55756c04c850 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.510+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55756bfce430 0x55756bfc8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.510+0000 7fbf6c5ca740 1 --2- 172.21.15.69:0/1605646751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55756bfcc120 0x55756bfc8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.510+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/1605646751 >> 172.21.15.69:0/1605646751 conn(0x55756bfbf660 msgr2=0x55756bfc0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.510+0000 7fbf4cc00640 0 client.26548 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:13:53.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.510+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/1605646751 shutdown_connections 2024-03-19T01:13:53.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.511+0000 7fbf6c5ca740 1 -- 172.21.15.69:0/1605646751 wait complete. 2024-03-19T01:13:53.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.511+0000 7fbf6c5ca740 20 client.26548 trim_cache size 0 max 16384 2024-03-19T01:13:53.512 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:13:53.512 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (7099 inodes in the stack)", 2024-03-19T01:13:53.513 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:13:53.513 INFO:teuthology.orchestra.run.smithi069.stdout: "50ca2af7-6cba-4162-8735-f8a498363dbf": { 2024-03-19T01:13:53.513 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:13:53.513 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "50ca2af7-6cba-4162-8735-f8a498363dbf", 2024-03-19T01:13:53.513 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:13:53.513 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:13:53.513 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:13:53.513 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.512+0000 7fbf63e00640 1 -- 172.21.15.69:0/506968587 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf480a29f0 msgr2=0x7fbf480a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.512+0000 7fbf63e00640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf480a29f0 0x7fbf480a4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbf6410f200 tx=0x7fbf50005f90 comp rx=0 tx=0).stop 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.512+0000 7fbf63e00640 1 -- 172.21.15.69:0/506968587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf64073bf0 msgr2=0x7fbf6419a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.512+0000 7fbf63e00640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf64073bf0 0x7fbf6419a1a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbf5402c2e0 tx=0x7fbf540043c0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.512+0000 7fbf63e00640 1 -- 172.21.15.69:0/506968587 shutdown_connections 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.512+0000 7fbf63e00640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf6410df30 0x7fbf6419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.512+0000 7fbf63e00640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbf480a29f0 0x7fbf480a4eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.512+0000 7fbf63e00640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf64074510 0x7fbf6419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.512+0000 7fbf63e00640 1 --2- 172.21.15.69:0/506968587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf64073bf0 0x7fbf6419a1a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.512+0000 7fbf63e00640 1 -- 172.21.15.69:0/506968587 >> 172.21.15.69:0/506968587 conn(0x7fbf640fd1c0 msgr2=0x7fbf640fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.512+0000 7fbf63e00640 1 -- 172.21.15.69:0/506968587 shutdown_connections 2024-03-19T01:13:53.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:13:53.513+0000 7fbf63e00640 1 -- 172.21.15.69:0/506968587 wait complete. 2024-03-19T01:13:53.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:53 smithi069 ceph-mon[31727]: pgmap v99: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-03-19T01:13:53.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:53 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:13:53.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:53 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:13:53.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:53 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:13:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:53.540 INFO:tasks.cephfs.filesystem:scrub status for tag:50ca2af7-6cba-4162-8735-f8a498363dbf - {'path': '/', 'tag': '50ca2af7-6cba-4162-8735-f8a498363dbf', 'options': 'recursive,force'} 2024-03-19T01:13:53.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:53 smithi086 ceph-mon[66829]: pgmap v99: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-03-19T01:13:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:53 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:13:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:53 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:13:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:53 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:13:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:53 smithi163 ceph-mon[67428]: pgmap v99: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-03-19T01:13:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:53 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:13:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:53 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:13:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:53 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:13:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:13:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:54.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:13:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:54.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:13:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.818 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:54.818 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:54.818 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: pgmap v100: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-03-19T01:13:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:55.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: pgmap v100: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:55.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: pgmap v100: 129 pgs: 129 active+clean; 423 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:55.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:56.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:13:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:56.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:57 smithi069 ceph-mon[31727]: pgmap v101: 129 pgs: 129 active+clean; 437 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 316 op/s 2024-03-19T01:13:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:57 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:13:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:57 smithi163 ceph-mon[67428]: pgmap v101: 129 pgs: 129 active+clean; 437 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 316 op/s 2024-03-19T01:13:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:57 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:13:58.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:57 smithi086 ceph-mon[66829]: pgmap v101: 129 pgs: 129 active+clean; 437 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 316 op/s 2024-03-19T01:13:58.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:57 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:13:59.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:59 smithi069 ceph-mon[31727]: pgmap v102: 129 pgs: 129 active+clean; 457 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 458 op/s 2024-03-19T01:13:59.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:13:59.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:13:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:13:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:59 smithi163 ceph-mon[67428]: pgmap v102: 129 pgs: 129 active+clean; 457 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 458 op/s 2024-03-19T01:13:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:13:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:13:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:59 smithi086 ceph-mon[66829]: pgmap v102: 129 pgs: 129 active+clean; 457 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 458 op/s 2024-03-19T01:14:00.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:00.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:13:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:00.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:00.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:00.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:01.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:01.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:01.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: pgmap v103: 129 pgs: 129 active+clean; 457 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-03-19T01:14:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:01.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:01.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:01.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:01.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: pgmap v103: 129 pgs: 129 active+clean; 457 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-03-19T01:14:01.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:01.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:02.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:02.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:02.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:02.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:02.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:02.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:02.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:02.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:02.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:02.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:02.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:02.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:02.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: pgmap v103: 129 pgs: 129 active+clean; 457 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-03-19T01:14:02.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:02.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:02 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:14:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:03.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:02 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:14:03.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:03.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:02 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:14:03.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:03 smithi163 ceph-mon[67428]: pgmap v104: 129 pgs: 129 active+clean; 544 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 845 op/s 2024-03-19T01:14:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:04.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:03 smithi086 ceph-mon[66829]: pgmap v104: 129 pgs: 129 active+clean; 544 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 845 op/s 2024-03-19T01:14:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:04.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:04.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:03 smithi069 ceph-mon[31727]: pgmap v104: 129 pgs: 129 active+clean; 544 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 845 op/s 2024-03-19T01:14:04.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:04.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:04.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:04.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:04.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:04.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:05.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:05.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:05.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:06.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:05 smithi086 ceph-mon[66829]: pgmap v105: 129 pgs: 129 active+clean; 544 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 10 MiB/s wr, 767 op/s 2024-03-19T01:14:06.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:05 smithi069 ceph-mon[31727]: pgmap v105: 129 pgs: 129 active+clean; 544 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 10 MiB/s wr, 767 op/s 2024-03-19T01:14:06.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:05 smithi163 ceph-mon[67428]: pgmap v105: 129 pgs: 129 active+clean; 544 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 10 MiB/s wr, 767 op/s 2024-03-19T01:14:08.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:07 smithi086 ceph-mon[66829]: pgmap v106: 129 pgs: 129 active+clean; 552 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 776 op/s 2024-03-19T01:14:08.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:07 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:14:08.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:07 smithi069 ceph-mon[31727]: pgmap v106: 129 pgs: 129 active+clean; 552 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 776 op/s 2024-03-19T01:14:08.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:07 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:14:08.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:07 smithi163 ceph-mon[67428]: pgmap v106: 129 pgs: 129 active+clean; 552 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 776 op/s 2024-03-19T01:14:08.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:07 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:14:09.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:08 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:14:09.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:08 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:14:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:08 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:14:10.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:09 smithi086 ceph-mon[66829]: pgmap v107: 129 pgs: 129 active+clean; 587 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 13 MiB/s wr, 751 op/s 2024-03-19T01:14:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:09 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:14:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:09 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:14:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:09 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:14:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:09 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:14:10.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:09 smithi069 ceph-mon[31727]: pgmap v107: 129 pgs: 129 active+clean; 587 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 13 MiB/s wr, 751 op/s 2024-03-19T01:14:10.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:09 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:14:10.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:09 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:14:10.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:09 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:14:10.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:09 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:14:10.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:09 smithi163 ceph-mon[67428]: pgmap v107: 129 pgs: 129 active+clean; 587 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 13 MiB/s wr, 751 op/s 2024-03-19T01:14:10.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:09 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:14:10.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:09 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:14:10.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:09 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:14:10.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:09 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:14:12.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:11 smithi086 ceph-mon[66829]: pgmap v108: 129 pgs: 129 active+clean; 587 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-03-19T01:14:12.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:11 smithi069 ceph-mon[31727]: pgmap v108: 129 pgs: 129 active+clean; 587 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-03-19T01:14:12.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:11 smithi163 ceph-mon[67428]: pgmap v108: 129 pgs: 129 active+clean; 587 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-03-19T01:14:14.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:13 smithi086 ceph-mon[66829]: pgmap v109: 129 pgs: 129 active+clean; 689 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 20 MiB/s wr, 648 op/s 2024-03-19T01:14:14.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:13 smithi069 ceph-mon[31727]: pgmap v109: 129 pgs: 129 active+clean; 689 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 20 MiB/s wr, 648 op/s 2024-03-19T01:14:14.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:13 smithi163 ceph-mon[67428]: pgmap v109: 129 pgs: 129 active+clean; 689 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 20 MiB/s wr, 648 op/s 2024-03-19T01:14:16.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:15 smithi086 ceph-mon[66829]: pgmap v110: 129 pgs: 129 active+clean; 689 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 186 op/s 2024-03-19T01:14:16.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:15 smithi069 ceph-mon[31727]: pgmap v110: 129 pgs: 129 active+clean; 689 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 186 op/s 2024-03-19T01:14:16.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:15 smithi163 ceph-mon[67428]: pgmap v110: 129 pgs: 129 active+clean; 689 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 186 op/s 2024-03-19T01:14:18.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:17 smithi086 ceph-mon[66829]: pgmap v111: 129 pgs: 129 active+clean; 722 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 199 op/s 2024-03-19T01:14:18.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:17 smithi069 ceph-mon[31727]: pgmap v111: 129 pgs: 129 active+clean; 722 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 199 op/s 2024-03-19T01:14:18.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:17 smithi163 ceph-mon[67428]: pgmap v111: 129 pgs: 129 active+clean; 722 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 199 op/s 2024-03-19T01:14:20.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:19 smithi086 ceph-mon[66829]: pgmap v112: 129 pgs: 129 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 37 MiB/s wr, 274 op/s 2024-03-19T01:14:20.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:19 smithi069 ceph-mon[31727]: pgmap v112: 129 pgs: 129 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 37 MiB/s wr, 274 op/s 2024-03-19T01:14:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:19 smithi163 ceph-mon[67428]: pgmap v112: 129 pgs: 129 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 37 MiB/s wr, 274 op/s 2024-03-19T01:14:22.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:21 smithi086 ceph-mon[66829]: pgmap v113: 129 pgs: 129 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 34 MiB/s wr, 178 op/s 2024-03-19T01:14:22.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:21 smithi069 ceph-mon[31727]: pgmap v113: 129 pgs: 129 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 34 MiB/s wr, 178 op/s 2024-03-19T01:14:22.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:21 smithi163 ceph-mon[67428]: pgmap v113: 129 pgs: 129 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 34 MiB/s wr, 178 op/s 2024-03-19T01:14:23.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:14:23.541 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:14:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.799+0000 7f6f99400640 1 -- 172.21.15.69:0/976151908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f94101690 msgr2=0x7f6f94101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.799+0000 7f6f99400640 1 --2- 172.21.15.69:0/976151908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f94101690 0x7f6f94101b10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009890 tx=0x7f6f8002bb70 comp rx=0 tx=0).stop 2024-03-19T01:14:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.800+0000 7f6f99400640 1 -- 172.21.15.69:0/976151908 shutdown_connections 2024-03-19T01:14:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.800+0000 7f6f99400640 1 --2- 172.21.15.69:0/976151908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f94102050 0x7f6f9410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.800+0000 7f6f99400640 1 --2- 172.21.15.69:0/976151908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f94101690 0x7f6f94101b10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.800+0000 7f6f99400640 1 --2- 172.21.15.69:0/976151908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f941076a0 0x7f6f94107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.800+0000 7f6f99400640 1 -- 172.21.15.69:0/976151908 >> 172.21.15.69:0/976151908 conn(0x7f6f940fd1c0 msgr2=0x7f6f940ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.800+0000 7f6f99400640 1 -- 172.21.15.69:0/976151908 shutdown_connections 2024-03-19T01:14:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.800+0000 7f6f99400640 1 -- 172.21.15.69:0/976151908 wait complete. 2024-03-19T01:14:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.801+0000 7f6f99400640 1 Processor -- start 2024-03-19T01:14:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.801+0000 7f6f99400640 1 -- start start 2024-03-19T01:14:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.802+0000 7f6f99400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f94101690 0x7f6f94197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.802+0000 7f6f99400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f94102050 0x7f6f941984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.802+0000 7f6f99400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f941076a0 0x7f6f9419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.802+0000 7f6f99400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94111ab0 con 0x7f6f94102050 2024-03-19T01:14:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.802+0000 7f6f99400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94111930 con 0x7f6f94101690 2024-03-19T01:14:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.802+0000 7f6f99400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94111c30 con 0x7f6f941076a0 2024-03-19T01:14:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.802+0000 7f6f93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f94101690 0x7f6f94197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.802+0000 7f6f93400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f94102050 0x7f6f941984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.802+0000 7f6f93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f94101690 0x7f6f94197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33460/0 (socket says 172.21.15.69:33460) 2024-03-19T01:14:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.802+0000 7f6f93e00640 1 -- 172.21.15.69:0/906247768 learned_addr learned my addr 172.21.15.69:0/906247768 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.802+0000 7f6f98a00640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f941076a0 0x7f6f9419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.803+0000 7f6f93e00640 1 -- 172.21.15.69:0/906247768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f941076a0 msgr2=0x7f6f9419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.803+0000 7f6f93e00640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f941076a0 0x7f6f9419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.803+0000 7f6f93e00640 1 -- 172.21.15.69:0/906247768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f94102050 msgr2=0x7f6f941984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.803+0000 7f6f93e00640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f94102050 0x7f6f941984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.803+0000 7f6f93e00640 1 -- 172.21.15.69:0/906247768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9419cf80 con 0x7f6f94101690 2024-03-19T01:14:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.803+0000 7f6f98a00640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f941076a0 0x7f6f9419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.803+0000 7f6f93e00640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f94101690 0x7f6f94197f90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400e9c0 tx=0x7f6f8400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.804+0000 7f6f90c00640 1 -- 172.21.15.69:0/906247768 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8400cde0 con 0x7f6f94101690 2024-03-19T01:14:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.804+0000 7f6f90c00640 1 -- 172.21.15.69:0/906247768 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f840035f0 con 0x7f6f94101690 2024-03-19T01:14:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.804+0000 7f6f99400640 1 -- 172.21.15.69:0/906247768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9419d270 con 0x7f6f94101690 2024-03-19T01:14:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.804+0000 7f6f99400640 1 -- 172.21.15.69:0/906247768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f94073a50 con 0x7f6f94101690 2024-03-19T01:14:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.804+0000 7f6f90c00640 1 -- 172.21.15.69:0/906247768 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84010690 con 0x7f6f94101690 2024-03-19T01:14:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.809+0000 7f6f9d76a740 1 Processor -- start 2024-03-19T01:14:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.809+0000 7f6f9d76a740 1 -- start start 2024-03-19T01:14:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.810+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 0x55b2b9084310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.810+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b908cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.810+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b908f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.810+0000 7f6f9d76a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b2b8fe82c0 con 0x55b2b9083f30 2024-03-19T01:14:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.811+0000 7f6f98a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 0x55b2b9084310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.811+0000 7f6f98a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 0x55b2b9084310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52084/0 (socket says 172.21.15.69:52084) 2024-03-19T01:14:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.811+0000 7f6f98a00640 1 -- 172.21.15.69:0/2081203600 learned_addr learned my addr 172.21.15.69:0/2081203600 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.811+0000 7f6f9d76a740 1 -- 172.21.15.69:0/2081203600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2b8f90b60 con 0x55b2b9084850 2024-03-19T01:14:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.811+0000 7f6f9d76a740 1 -- 172.21.15.69:0/2081203600 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b2b8fe4360 con 0x55b2b908d430 2024-03-19T01:14:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.811+0000 7f6f90c00640 1 -- 172.21.15.69:0/906247768 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f6f84010830 con 0x7f6f94101690 2024-03-19T01:14:23.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.812+0000 7f6f93400640 1 --2- 172.21.15.69:0/2081203600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b908f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.812+0000 7f6f98a00640 1 -- 172.21.15.69:0/2081203600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 msgr2=0x55b2b908f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:23.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.812+0000 7f6f98a00640 1 --2- 172.21.15.69:0/2081203600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b908f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.812+0000 7f6f98a00640 1 -- 172.21.15.69:0/2081203600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 msgr2=0x55b2b908cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:23.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.814+0000 7f6f90c00640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6f780a29f0 0x7f6f780a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.814+0000 7f6f93e00640 1 --2- 172.21.15.69:0/2081203600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b908cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.814+0000 7f6f98a00640 1 --2- 172.21.15.69:0/2081203600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b908cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.814+0000 7f6f98a00640 1 -- 172.21.15.69:0/2081203600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b8dc67d0 con 0x55b2b9083f30 2024-03-19T01:14:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.814+0000 7f6f93400640 1 --2- 172.21.15.69:0/2081203600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b908f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:14:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.814+0000 7f6f93e00640 1 --2- 172.21.15.69:0/2081203600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b908cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:14:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.814+0000 7f6f90c00640 1 -- 172.21.15.69:0/906247768 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f6f840cf450 con 0x7f6f94101690 2024-03-19T01:14:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.816+0000 7f6f93400640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6f780a29f0 0x7f6f780a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.816+0000 7f6f98a00640 1 --2- 172.21.15.69:0/2081203600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 0x55b2b9084310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00f410 tx=0x7f6f8c00f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b83259d3029ece8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.817+0000 7f6f93400640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6f780a29f0 0x7f6f780a4eb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6f8002c180 tx=0x7f6f800056d0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:23.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.818+0000 7f6f89200640 1 -- 172.21.15.69:0/2081203600 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c015070 con 0x55b2b9083f30 2024-03-19T01:14:23.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.818+0000 7f6f89200640 1 -- 172.21.15.69:0/2081203600 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8c011be0 con 0x55b2b9083f30 2024-03-19T01:14:23.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.818+0000 7f6f89200640 1 -- 172.21.15.69:0/2081203600 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c010d30 con 0x55b2b9083f30 2024-03-19T01:14:23.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.819+0000 7f6f9d76a740 1 -- 172.21.15.69:0/2081203600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 msgr2=0x55b2b9084310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:23.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.819+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/2081203600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 0x55b2b9084310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00f410 tx=0x7f6f8c00f8d0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.820+0000 7f6f9d76a740 1 -- 172.21.15.69:0/2081203600 shutdown_connections 2024-03-19T01:14:23.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.820+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/2081203600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b908f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.820+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/2081203600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b908cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.820+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/2081203600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 0x55b2b9084310 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.820+0000 7f6f9d76a740 1 -- 172.21.15.69:0/2081203600 >> 172.21.15.69:0/2081203600 conn(0x55b2b907e660 msgr2=0x55b2b907ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.820+0000 7f6f9d76a740 1 -- 172.21.15.69:0/2081203600 shutdown_connections 2024-03-19T01:14:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.820+0000 7f6f9d76a740 1 -- 172.21.15.69:0/2081203600 wait complete. 2024-03-19T01:14:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.821+0000 7f6f9d76a740 1 -- start start 2024-03-19T01:14:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.821+0000 7f6f9d76a740 1 Processor -- start 2024-03-19T01:14:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.822+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9083f30 0x55b2b908a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.822+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9084850 0x55b2b908a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.822+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b908ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.822+0000 7f6f9d76a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b2b9090220 con 0x55b2b9084850 2024-03-19T01:14:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.822+0000 7f6f9d76a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2b8fa3fc0 con 0x55b2b9083f30 2024-03-19T01:14:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.822+0000 7f6f9d76a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b2b90903a0 con 0x55b2b908d430 2024-03-19T01:14:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.822+0000 7f6f9d76a740 10 client.0 _ll_register_callbacks cb 0x55b2b8fa3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.822+0000 7f6f93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9083f30 0x55b2b908a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.822+0000 7f6f93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9083f30 0x55b2b908a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33478/0 (socket says 172.21.15.69:33478) 2024-03-19T01:14:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.822+0000 7f6f93e00640 1 -- 172.21.15.69:0/574790851 learned_addr learned my addr 172.21.15.69:0/574790851 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.823+0000 7f6f93400640 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9084850 0x55b2b908a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.823+0000 7f6f93e00640 1 -- 172.21.15.69:0/574790851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 msgr2=0x55b2b908ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.823+0000 7f6f98a00640 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b908ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.823+0000 7f6f93e00640 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b908ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.823+0000 7f6f93e00640 1 -- 172.21.15.69:0/574790851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9084850 msgr2=0x55b2b908a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:23.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.823+0000 7f6f93e00640 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9084850 0x55b2b908a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.823+0000 7f6f93e00640 1 -- 172.21.15.69:0/574790851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b8cfc3f0 con 0x55b2b9083f30 2024-03-19T01:14:23.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.823+0000 7f6f93400640 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9084850 0x55b2b908a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:14:23.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.823+0000 7f6f98a00640 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b908ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:14:23.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.823+0000 7f6f93e00640 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9083f30 0x55b2b908a390 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400c6a0 tx=0x7f6f8409b2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ff9b60486559c16 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:23.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.823+0000 7f6f7d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.824+0000 7f6f7d600640 1 -- 172.21.15.69:0/574790851 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8409f840 con 0x55b2b9083f30 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.824+0000 7f6f9d76a740 1 -- 172.21.15.69:0/574790851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f8c011be0 con 0x55b2b9083f30 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.824+0000 7f6f7d600640 1 -- 172.21.15.69:0/574790851 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f6f84097d60 con 0x55b2b9083f30 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.824+0000 7f6f7d600640 1 -- 172.21.15.69:0/574790851 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6f8409f840 con 0x55b2b9083f30 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.824+0000 7f6f9d76a740 10 client.26570 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.824+0000 7f6f9d76a740 1 -- 172.21.15.69:0/574790851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b2b8cbc0f0 con 0x55b2b9083f30 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.824+0000 7f6f7d600640 1 -- 172.21.15.69:0/574790851 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f840a3e90 con 0x55b2b9083f30 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.824+0000 7f6f7d600640 1 -- 172.21.15.69:0/574790851 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f6f84004090 con 0x55b2b9083f30 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.825+0000 7f6f9d76a740 10 client.26570 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.825+0000 7f6f9d76a740 10 client.26570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.825+0000 7f6f9d76a740 20 client.26570 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.825+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2b909d6d0 0x55b2b909fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.825+0000 7f6f9d76a740 4 client.26570 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:14:23.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.825+0000 7f6f9d76a740 1 -- 172.21.15.69:0/574790851 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c011be0 con 0x55b2b909d6d0 2024-03-19T01:14:23.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.825+0000 7f6f93400640 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2b909d6d0 0x55b2b909fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.826+0000 7f6f93400640 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2b909d6d0 0x55b2b909fb90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=584dc437d1e4c15d server_cookie=8f198890ee460b3e in_seq=0 out_seq=0 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.826+0000 7f6f7d600640 10 client.26570 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.826+0000 7f6f7d600640 1 -- 172.21.15.69:0/574790851 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f6f8c011be0 con 0x55b2b909d6d0 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.826+0000 7f6f7d600640 10 client.26570 handle_command_reply: tid=0 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.826+0000 7f6f9d76a740 1 client.26570 shutdown 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.827+0000 7f6f9d76a740 1 -- 172.21.15.69:0/574790851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9083f30 msgr2=0x55b2b908a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.827+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9083f30 0x55b2b908a390 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400c6a0 tx=0x7f6f8409b2b0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.827+0000 7f6f9d76a740 1 -- 172.21.15.69:0/574790851 shutdown_connections 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.827+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2b909d6d0 0x55b2b909fb90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.827+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b908ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.827+0000 7f6f7d600640 0 client.26570 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.827+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9084850 0x55b2b908a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.827+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/574790851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9083f30 0x55b2b908a390 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.827+0000 7f6f9d76a740 1 -- 172.21.15.69:0/574790851 >> 172.21.15.69:0/574790851 conn(0x55b2b907e660 msgr2=0x55b2b90803b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:23.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.827+0000 7f6f9d76a740 1 -- 172.21.15.69:0/574790851 shutdown_connections 2024-03-19T01:14:23.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.827+0000 7f6f9d76a740 1 -- 172.21.15.69:0/574790851 wait complete. 2024-03-19T01:14:23.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.828+0000 7f6f9d76a740 20 client.26570 trim_cache size 0 max 16384 2024-03-19T01:14:23.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.856+0000 7f6f9d76a740 1 Processor -- start 2024-03-19T01:14:23.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.856+0000 7f6f9d76a740 1 -- start start 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.856+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 0x55b2b9098ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.856+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b9089b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.856+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b90801a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.856+0000 7f6f9d76a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7400ff60 con 0x55b2b9083f30 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.856+0000 7f6f9d76a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74011c20 con 0x55b2b9084850 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.856+0000 7f6f9d76a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74010630 con 0x55b2b908d430 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.857+0000 7f6f93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b9089b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.857+0000 7f6f93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b9089b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33482/0 (socket says 172.21.15.69:33482) 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.857+0000 7f6f93400640 1 -- 172.21.15.69:0/3406205058 learned_addr learned my addr 172.21.15.69:0/3406205058 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.857+0000 7f6f98a00640 1 --2- 172.21.15.69:0/3406205058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b90801a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.857+0000 7f6f93e00640 1 --2- 172.21.15.69:0/3406205058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 0x55b2b9098ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.857+0000 7f6f93400640 1 -- 172.21.15.69:0/3406205058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 msgr2=0x55b2b90801a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.857+0000 7f6f93400640 1 --2- 172.21.15.69:0/3406205058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b90801a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.857+0000 7f6f93400640 1 -- 172.21.15.69:0/3406205058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 msgr2=0x55b2b9098ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:23.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.857+0000 7f6f93400640 1 --2- 172.21.15.69:0/3406205058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 0x55b2b9098ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.857+0000 7f6f93400640 1 -- 172.21.15.69:0/3406205058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b909c240 con 0x55b2b9084850 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.857+0000 7f6f93400640 1 --2- 172.21.15.69:0/3406205058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b9089b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009890 tx=0x7f6f80041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e2fd64594a14eb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.858+0000 7f6f7fe00640 1 -- 172.21.15.69:0/3406205058 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80052af0 con 0x55b2b9084850 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.858+0000 7f6f7fe00640 1 -- 172.21.15.69:0/3406205058 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f80052c90 con 0x55b2b9084850 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.858+0000 7f6f7fe00640 1 -- 172.21.15.69:0/3406205058 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80052260 con 0x55b2b9084850 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.859+0000 7f6f9d76a740 1 -- 172.21.15.69:0/3406205058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 msgr2=0x55b2b9089b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.859+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/3406205058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b9089b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009890 tx=0x7f6f80041000 comp rx=0 tx=0).stop 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.859+0000 7f6f9d76a740 1 -- 172.21.15.69:0/3406205058 shutdown_connections 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.859+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/3406205058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b908d430 0x55b2b90801a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.859+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/3406205058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b9089b40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.859+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/3406205058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b9083f30 0x55b2b9098ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.859+0000 7f6f9d76a740 1 -- 172.21.15.69:0/3406205058 >> 172.21.15.69:0/3406205058 conn(0x55b2b907e660 msgr2=0x55b2b907fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.860+0000 7f6f9d76a740 1 -- 172.21.15.69:0/3406205058 shutdown_connections 2024-03-19T01:14:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.860+0000 7f6f9d76a740 1 -- 172.21.15.69:0/3406205058 wait complete. 2024-03-19T01:14:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.860+0000 7f6f9d76a740 1 -- start start 2024-03-19T01:14:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.860+0000 7f6f9d76a740 1 Processor -- start 2024-03-19T01:14:23.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.861+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b9083f30 0x55b2b90915e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.861+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b9091b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.861+0000 7f6f9d76a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b908d430 0x55b2b910b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f9d76a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74011490 con 0x55b2b908d430 2024-03-19T01:14:23.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f93e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b9083f30 0x55b2b90915e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f9d76a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74010d00 con 0x55b2b9084850 2024-03-19T01:14:23.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f9d76a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7400df40 con 0x55b2b9083f30 2024-03-19T01:14:23.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f9d76a740 10 client.0 _ll_register_callbacks cb 0x7f6f8409b4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:23.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f98a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b908d430 0x55b2b910b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f93e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b9083f30 0x55b2b90915e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55884/0 (socket says 172.21.15.69:55884) 2024-03-19T01:14:23.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f93e00640 1 -- 172.21.15.69:0/4248500064 learned_addr learned my addr 172.21.15.69:0/4248500064 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:23.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b9091b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f93e00640 1 -- 172.21.15.69:0/4248500064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 msgr2=0x55b2b9091b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:23.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f93e00640 1 --2- 172.21.15.69:0/4248500064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b9091b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f93e00640 1 -- 172.21.15.69:0/4248500064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b908d430 msgr2=0x55b2b910b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:23.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f93e00640 1 --2- 172.21.15.69:0/4248500064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b908d430 0x55b2b910b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f93e00640 1 -- 172.21.15.69:0/4248500064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b910b7e0 con 0x55b2b9083f30 2024-03-19T01:14:23.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f93e00640 1 --2- 172.21.15.69:0/4248500064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b9083f30 0x55b2b90915e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6f840cf7b0 tx=0x7f6f840975e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2edf6571ddb1296 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:23.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.862+0000 7f6f7d600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:14:23.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.863+0000 7f6f7d600640 1 -- 172.21.15.69:0/4248500064 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84097990 con 0x55b2b9083f30 2024-03-19T01:14:23.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.863+0000 7f6f9d76a740 1 -- 172.21.15.69:0/4248500064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f80052c90 con 0x55b2b9083f30 2024-03-19T01:14:23.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.863+0000 7f6f7d600640 1 -- 172.21.15.69:0/4248500064 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f6f8409ad40 con 0x55b2b9083f30 2024-03-19T01:14:23.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.863+0000 7f6f7d600640 1 -- 172.21.15.69:0/4248500064 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6f840976d0 con 0x55b2b9083f30 2024-03-19T01:14:23.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.863+0000 7f6f9d76a740 10 client.26560 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:23.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.863+0000 7f6f9d76a740 1 -- 172.21.15.69:0/4248500064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b2b910ca90 con 0x55b2b9083f30 2024-03-19T01:14:23.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.863+0000 7f6f7d600640 1 -- 172.21.15.69:0/4248500064 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f84004210 con 0x55b2b9083f30 2024-03-19T01:14:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.864+0000 7f6f7d600640 1 -- 172.21.15.69:0/4248500064 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f6f84010b20 con 0x55b2b9083f30 2024-03-19T01:14:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.864+0000 7f6f9d76a740 10 client.26560 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.864+0000 7f6f9d76a740 10 client.26560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:14:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.864+0000 7f6f9d76a740 20 client.26560 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.864+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/4248500064 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2b910d210 0x55b2b910f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.864+0000 7f6f9d76a740 4 client.26560 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:14:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.864+0000 7f6f9d76a740 1 -- 172.21.15.69:0/4248500064 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6f80052c90 con 0x55b2b910d210 2024-03-19T01:14:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.864+0000 7f6f93400640 1 --2- 172.21.15.69:0/4248500064 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2b910d210 0x55b2b910f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.865+0000 7f6f7d600640 10 client.26560 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.865+0000 7f6f93400640 1 --2- 172.21.15.69:0/4248500064 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2b910d210 0x55b2b910f630 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33884e302569e208 server_cookie=32b08a0d7dad87b2 in_seq=0 out_seq=0 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.865+0000 7f6f7d600640 1 -- 172.21.15.69:0/4248500064 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f6f80052c90 con 0x55b2b910d210 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.865+0000 7f6f7d600640 10 client.26560 handle_command_reply: tid=0 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.865+0000 7f6f9d76a740 1 client.26560 shutdown 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.865+0000 7f6f9d76a740 1 -- 172.21.15.69:0/4248500064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b9083f30 msgr2=0x55b2b90915e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.865+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/4248500064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b9083f30 0x55b2b90915e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6f840cf7b0 tx=0x7f6f840975e0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.865+0000 7f6f9d76a740 1 -- 172.21.15.69:0/4248500064 shutdown_connections 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.866+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/4248500064 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2b910d210 0x55b2b910f630 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.866+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/4248500064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2b908d430 0x55b2b910b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.866+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/4248500064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2b9084850 0x55b2b9091b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.866+0000 7f6f9d76a740 1 --2- 172.21.15.69:0/4248500064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2b9083f30 0x55b2b90915e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.866+0000 7f6f7d600640 0 client.26560 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.866+0000 7f6f9d76a740 1 -- 172.21.15.69:0/4248500064 >> 172.21.15.69:0/4248500064 conn(0x55b2b907e660 msgr2=0x55b2b9080320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.866+0000 7f6f9d76a740 1 -- 172.21.15.69:0/4248500064 shutdown_connections 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.866+0000 7f6f9d76a740 1 -- 172.21.15.69:0/4248500064 wait complete. 2024-03-19T01:14:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.866+0000 7f6f9d76a740 20 client.26560 trim_cache size 0 max 16384 2024-03-19T01:14:23.869 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:14:23.869 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:14:23.869 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:14:23.869 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:14:23.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.867+0000 7f6f99400640 1 -- 172.21.15.69:0/906247768 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6f780a29f0 msgr2=0x7f6f780a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:23.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.867+0000 7f6f99400640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6f780a29f0 0x7f6f780a4eb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6f8002c180 tx=0x7f6f800056d0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.867+0000 7f6f99400640 1 -- 172.21.15.69:0/906247768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f94101690 msgr2=0x7f6f94197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:23.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.867+0000 7f6f99400640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f94101690 0x7f6f94197f90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400e9c0 tx=0x7f6f8400ee80 comp rx=0 tx=0).stop 2024-03-19T01:14:23.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.867+0000 7f6f99400640 1 -- 172.21.15.69:0/906247768 shutdown_connections 2024-03-19T01:14:23.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.867+0000 7f6f99400640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f941076a0 0x7f6f9419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.867+0000 7f6f99400640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f94102050 0x7f6f941984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.867+0000 7f6f99400640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6f780a29f0 0x7f6f780a4eb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.867+0000 7f6f99400640 1 --2- 172.21.15.69:0/906247768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f94101690 0x7f6f94197f90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:23.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.867+0000 7f6f99400640 1 -- 172.21.15.69:0/906247768 >> 172.21.15.69:0/906247768 conn(0x7f6f940fd1c0 msgr2=0x7f6f940fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:23.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.868+0000 7f6f99400640 1 -- 172.21.15.69:0/906247768 shutdown_connections 2024-03-19T01:14:23.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:23.868+0000 7f6f99400640 1 -- 172.21.15.69:0/906247768 wait complete. 2024-03-19T01:14:23.891 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:14:23.891 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:14:24.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.138+0000 7ffa46c00640 1 -- 172.21.15.69:0/63219034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa40101fd0 msgr2=0x7ffa4010e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.138+0000 7ffa46c00640 1 --2- 172.21.15.69:0/63219034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa40101fd0 0x7ffa4010e4e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009ca0 tx=0x7ffa3802c100 comp rx=0 tx=0).stop 2024-03-19T01:14:24.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.138+0000 7ffa46c00640 1 -- 172.21.15.69:0/63219034 shutdown_connections 2024-03-19T01:14:24.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.138+0000 7ffa46c00640 1 --2- 172.21.15.69:0/63219034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa40101fd0 0x7ffa4010e4e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.138+0000 7ffa46c00640 1 --2- 172.21.15.69:0/63219034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa40101610 0x7ffa40101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.138+0000 7ffa46c00640 1 --2- 172.21.15.69:0/63219034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa401076b0 0x7ffa40107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.138+0000 7ffa46c00640 1 -- 172.21.15.69:0/63219034 >> 172.21.15.69:0/63219034 conn(0x7ffa400fd120 msgr2=0x7ffa400ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:24.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.139+0000 7ffa46c00640 1 -- 172.21.15.69:0/63219034 shutdown_connections 2024-03-19T01:14:24.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.139+0000 7ffa46c00640 1 -- 172.21.15.69:0/63219034 wait complete. 2024-03-19T01:14:24.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.140+0000 7ffa46c00640 1 Processor -- start 2024-03-19T01:14:24.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa46c00640 1 -- start start 2024-03-19T01:14:24.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa46c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa40101610 0x7ffa40108540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa46c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa40101fd0 0x7ffa40108a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa46c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa401076b0 0x7ffa40108fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa46c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401119f0 con 0x7ffa40101fd0 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa46c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa40111870 con 0x7ffa40101610 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa46c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffa40111b70 con 0x7ffa401076b0 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa45800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa40101610 0x7ffa40108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa46200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa401076b0 0x7ffa40108fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa45800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa40101610 0x7ffa40108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33514/0 (socket says 172.21.15.69:33514) 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa45800640 1 -- 172.21.15.69:0/3797553451 learned_addr learned my addr 172.21.15.69:0/3797553451 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.141+0000 7ffa44e00640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa40101fd0 0x7ffa40108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.142+0000 7ffa46200640 1 -- 172.21.15.69:0/3797553451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa40101610 msgr2=0x7ffa40108540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.142+0000 7ffa46200640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa40101610 0x7ffa40108540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.142+0000 7ffa46200640 1 -- 172.21.15.69:0/3797553451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa40101fd0 msgr2=0x7ffa40108a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.142+0000 7ffa46200640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa40101fd0 0x7ffa40108a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.142+0000 7ffa46200640 1 -- 172.21.15.69:0/3797553451 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa40199e70 con 0x7ffa401076b0 2024-03-19T01:14:24.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.142+0000 7ffa45800640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa40101610 0x7ffa40108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:14:24.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.142+0000 7ffa46200640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa401076b0 0x7ffa40108fc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffa38037040 tx=0x7ffa38034670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:24.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.143+0000 7ffa3e000640 1 -- 172.21.15.69:0/3797553451 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38039050 con 0x7ffa401076b0 2024-03-19T01:14:24.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.143+0000 7ffa3e000640 1 -- 172.21.15.69:0/3797553451 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa38045070 con 0x7ffa401076b0 2024-03-19T01:14:24.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.143+0000 7ffa3e000640 1 -- 172.21.15.69:0/3797553451 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa380044c0 con 0x7ffa401076b0 2024-03-19T01:14:24.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.143+0000 7ffa46c00640 1 -- 172.21.15.69:0/3797553451 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4019a100 con 0x7ffa401076b0 2024-03-19T01:14:24.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.143+0000 7ffa46c00640 1 -- 172.21.15.69:0/3797553451 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4019a570 con 0x7ffa401076b0 2024-03-19T01:14:24.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.144+0000 7ffa46c00640 1 -- 172.21.15.69:0/3797553451 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa40102d50 con 0x7ffa401076b0 2024-03-19T01:14:24.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.147+0000 7ffa3e000640 1 -- 172.21.15.69:0/3797553451 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ffa38004660 con 0x7ffa401076b0 2024-03-19T01:14:24.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.148+0000 7ffa3e000640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa280a29f0 0x7ffa280a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.148+0000 7ffa3e000640 1 -- 172.21.15.69:0/3797553451 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7ffa380f15d0 con 0x7ffa401076b0 2024-03-19T01:14:24.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.149+0000 7ffa45800640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa280a29f0 0x7ffa280a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.150+0000 7ffa45800640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa280a29f0 0x7ffa280a4eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffa340058c0 tx=0x7ffa34005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:24.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.153+0000 7ffa3e000640 1 -- 172.21.15.69:0/3797553451 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffa380be080 con 0x7ffa401076b0 2024-03-19T01:14:24.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:23 smithi086 ceph-mon[66829]: pgmap v114: 129 pgs: 129 active+clean; 1000 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 35 MiB/s wr, 196 op/s 2024-03-19T01:14:24.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:23 smithi069 ceph-mon[31727]: pgmap v114: 129 pgs: 129 active+clean; 1000 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 35 MiB/s wr, 196 op/s 2024-03-19T01:14:24.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:23 smithi163 ceph-mon[67428]: pgmap v114: 129 pgs: 129 active+clean; 1000 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 35 MiB/s wr, 196 op/s 2024-03-19T01:14:24.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.576+0000 7ffa46c00640 1 -- 172.21.15.69:0/3797553451 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa4019a9b0 con 0x7ffa401076b0 2024-03-19T01:14:24.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.577+0000 7ffa3e000640 1 -- 172.21.15.69:0/3797553451 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11951 (secure 0 0 0) 0x7ffa380bd980 con 0x7ffa401076b0 2024-03-19T01:14:24.578 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:14:24.578 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:14:05.695054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581,24476,24580,24494,24550]},"id":1}]} 2024-03-19T01:14:24.578 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 36 2024-03-19T01:14:24.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.579+0000 7ffa46c00640 1 -- 172.21.15.69:0/3797553451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa280a29f0 msgr2=0x7ffa280a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.579+0000 7ffa46c00640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa280a29f0 0x7ffa280a4eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffa340058c0 tx=0x7ffa34005f90 comp rx=0 tx=0).stop 2024-03-19T01:14:24.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.579+0000 7ffa46c00640 1 -- 172.21.15.69:0/3797553451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa401076b0 msgr2=0x7ffa40108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.579+0000 7ffa46c00640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa401076b0 0x7ffa40108fc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffa38037040 tx=0x7ffa38034670 comp rx=0 tx=0).stop 2024-03-19T01:14:24.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.579+0000 7ffa46c00640 1 -- 172.21.15.69:0/3797553451 shutdown_connections 2024-03-19T01:14:24.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.579+0000 7ffa46c00640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa401076b0 0x7ffa40108fc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.579+0000 7ffa46c00640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa40101fd0 0x7ffa40108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.579+0000 7ffa46c00640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa280a29f0 0x7ffa280a4eb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.579+0000 7ffa46c00640 1 --2- 172.21.15.69:0/3797553451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa40101610 0x7ffa40108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.579+0000 7ffa46c00640 1 -- 172.21.15.69:0/3797553451 >> 172.21.15.69:0/3797553451 conn(0x7ffa400fd120 msgr2=0x7ffa400fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:24.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.580+0000 7ffa46c00640 1 -- 172.21.15.69:0/3797553451 shutdown_connections 2024-03-19T01:14:24.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.580+0000 7ffa46c00640 1 -- 172.21.15.69:0/3797553451 wait complete. 2024-03-19T01:14:24.603 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:14:24.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.841+0000 7f88bfe00640 1 -- 172.21.15.69:0/2498272717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88b81039c0 msgr2=0x7f88b8103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.841+0000 7f88bfe00640 1 --2- 172.21.15.69:0/2498272717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88b81039c0 0x7f88b8103e40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009890 tx=0x7f88a402bb90 comp rx=0 tx=0).stop 2024-03-19T01:14:24.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.841+0000 7f88bfe00640 1 -- 172.21.15.69:0/2498272717 shutdown_connections 2024-03-19T01:14:24.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.841+0000 7f88bfe00640 1 --2- 172.21.15.69:0/2498272717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b8104380 0x7f88b8110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.841+0000 7f88bfe00640 1 --2- 172.21.15.69:0/2498272717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88b81039c0 0x7f88b8103e40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.841+0000 7f88bfe00640 1 --2- 172.21.15.69:0/2498272717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88b81099d0 0x7f88b8109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.841+0000 7f88bfe00640 1 -- 172.21.15.69:0/2498272717 >> 172.21.15.69:0/2498272717 conn(0x7f88b80ff470 msgr2=0x7f88b8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.841+0000 7f88bfe00640 1 -- 172.21.15.69:0/2498272717 shutdown_connections 2024-03-19T01:14:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.841+0000 7f88bfe00640 1 -- 172.21.15.69:0/2498272717 wait complete. 2024-03-19T01:14:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.842+0000 7f88bfe00640 1 Processor -- start 2024-03-19T01:14:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bfe00640 1 -- start start 2024-03-19T01:14:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88b81039c0 0x7f88b810f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b8104380 0x7f88b810fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88b81099d0 0x7f88b81a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bfe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88b80771d0 con 0x7f88b81039c0 2024-03-19T01:14:24.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8077050 con 0x7f88b8104380 2024-03-19T01:14:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bfe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8077350 con 0x7f88b81099d0 2024-03-19T01:14:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b8104380 0x7f88b810fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b8104380 0x7f88b810fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33536/0 (socket says 172.21.15.69:33536) 2024-03-19T01:14:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bd400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88b81039c0 0x7f88b810f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bca00640 1 -- 172.21.15.69:0/1559532974 learned_addr learned my addr 172.21.15.69:0/1559532974 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.843+0000 7f88bde00640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88b81099d0 0x7f88b81a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.844+0000 7f88bca00640 1 -- 172.21.15.69:0/1559532974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88b81099d0 msgr2=0x7f88b81a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.844+0000 7f88bca00640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88b81099d0 0x7f88b81a2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.844+0000 7f88bca00640 1 -- 172.21.15.69:0/1559532974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88b81039c0 msgr2=0x7f88b810f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.844+0000 7f88bca00640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88b81039c0 0x7f88b810f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.844+0000 7f88bca00640 1 -- 172.21.15.69:0/1559532974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b81a3720 con 0x7f88b8104380 2024-03-19T01:14:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.844+0000 7f88bca00640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b8104380 0x7f88b810fc80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f88a402c1a0 tx=0x7f88a4034930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.844+0000 7f88b6000640 1 -- 172.21.15.69:0/1559532974 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40044d0 con 0x7f88b8104380 2024-03-19T01:14:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.844+0000 7f88bfe00640 1 -- 172.21.15.69:0/1559532974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b81a39b0 con 0x7f88b8104380 2024-03-19T01:14:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.844+0000 7f88b6000640 1 -- 172.21.15.69:0/1559532974 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a4055070 con 0x7f88b8104380 2024-03-19T01:14:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.844+0000 7f88b6000640 1 -- 172.21.15.69:0/1559532974 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40324a0 con 0x7f88b8104380 2024-03-19T01:14:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.844+0000 7f88bfe00640 1 -- 172.21.15.69:0/1559532974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b806b240 con 0x7f88b8104380 2024-03-19T01:14:24.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.848+0000 7f88b6000640 1 -- 172.21.15.69:0/1559532974 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f88a40036f0 con 0x7f88b8104380 2024-03-19T01:14:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.849+0000 7f88b6000640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88a00a2b90 0x7f88a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.849+0000 7f88b6000640 1 -- 172.21.15.69:0/1559532974 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f88a4102110 con 0x7f88b8104380 2024-03-19T01:14:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.850+0000 7f88bd400640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88a00a2b90 0x7f88a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.850+0000 7f88c2ba6740 1 Processor -- start 2024-03-19T01:14:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.851+0000 7f88c2ba6740 1 -- start start 2024-03-19T01:14:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.851+0000 7f88bd400640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88a00a2b90 0x7f88a00a5050 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009b50 tx=0x7f88ac005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.851+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30c6a20 0x55f3c30c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.851+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.851+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30cff20 0x55f3c30d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.851+0000 7f88c2ba6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f3c302b380 con 0x55f3c30c7340 2024-03-19T01:14:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.851+0000 7f88c2ba6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2fd3d20 con 0x55f3c30cff20 2024-03-19T01:14:24.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.851+0000 7f88c2ba6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f3c3027420 con 0x55f3c30c6a20 2024-03-19T01:14:24.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bd400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30c6a20 0x55f3c30c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52148/0 (socket says 172.21.15.69:52148) 2024-03-19T01:14:24.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bca00640 1 -- 172.21.15.69:0/315839484 learned_addr learned my addr 172.21.15.69:0/315839484 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:24.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bd400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30c6a20 0x55f3c30c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55954/0 (socket says 172.21.15.69:55954) 2024-03-19T01:14:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bd400640 1 -- 172.21.15.69:0/315839484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30cff20 msgr2=0x55f3c30d2310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:14:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bde00640 1 --2- 172.21.15.69:0/315839484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30cff20 0x55f3c30d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bd400640 1 --2- 172.21.15.69:0/315839484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30cff20 0x55f3c30d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bd400640 1 -- 172.21.15.69:0/315839484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 msgr2=0x55f3c30cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bd400640 1 --2- 172.21.15.69:0/315839484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bd400640 1 -- 172.21.15.69:0/315839484 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c2d3f3f0 con 0x55f3c30c6a20 2024-03-19T01:14:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.852+0000 7f88bca00640 1 --2- 172.21.15.69:0/315839484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:14:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.853+0000 7f88bde00640 1 --2- 172.21.15.69:0/315839484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30cff20 0x55f3c30d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:14:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.853+0000 7f88bd400640 1 --2- 172.21.15.69:0/315839484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30c6a20 0x55f3c30c6e00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0118d0 tx=0x7f88ac014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9924101cb8ade803 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.853+0000 7f88aa600640 1 -- 172.21.15.69:0/315839484 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac01e420 con 0x55f3c30c6a20 2024-03-19T01:14:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.853+0000 7f88aa600640 1 -- 172.21.15.69:0/315839484 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ac01eaa0 con 0x55f3c30c6a20 2024-03-19T01:14:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.853+0000 7f88aa600640 1 -- 172.21.15.69:0/315839484 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac01db70 con 0x55f3c30c6a20 2024-03-19T01:14:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.854+0000 7f88c2ba6740 1 -- 172.21.15.69:0/315839484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30c6a20 msgr2=0x55f3c30c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.854+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/315839484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30c6a20 0x55f3c30c6e00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0118d0 tx=0x7f88ac014000 comp rx=0 tx=0).stop 2024-03-19T01:14:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.854+0000 7f88c2ba6740 1 -- 172.21.15.69:0/315839484 shutdown_connections 2024-03-19T01:14:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.854+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/315839484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30cff20 0x55f3c30d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.855+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/315839484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.855+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/315839484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30c6a20 0x55f3c30c6e00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.855+0000 7f88c2ba6740 1 -- 172.21.15.69:0/315839484 >> 172.21.15.69:0/315839484 conn(0x55f3c30c1150 msgr2=0x55f3c30c1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.855+0000 7f88c2ba6740 1 -- 172.21.15.69:0/315839484 shutdown_connections 2024-03-19T01:14:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.855+0000 7f88c2ba6740 1 -- 172.21.15.69:0/315839484 wait complete. 2024-03-19T01:14:24.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.855+0000 7f88c2ba6740 1 -- start start 2024-03-19T01:14:24.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.856+0000 7f88c2ba6740 1 Processor -- start 2024-03-19T01:14:24.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.856+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 0x55f3c30ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.856+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.856+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 0x55f3c30ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88bca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88bca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52152/0 (socket says 172.21.15.69:52152) 2024-03-19T01:14:24.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88bca00640 1 -- 172.21.15.69:0/626434804 learned_addr learned my addr 172.21.15.69:0/626434804 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:24.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88c2ba6740 1 -- 172.21.15.69:0/626434804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f3c30d2d10 con 0x55f3c30c7340 2024-03-19T01:14:24.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88c2ba6740 1 -- 172.21.15.69:0/626434804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2fe7160 con 0x55f3c30c6a20 2024-03-19T01:14:24.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88c2ba6740 1 -- 172.21.15.69:0/626434804 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f3c30d2e90 con 0x55f3c30cff20 2024-03-19T01:14:24.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88bca00640 1 -- 172.21.15.69:0/626434804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 msgr2=0x55f3c30cda30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:14:24.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88bca00640 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30cda30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88bca00640 1 -- 172.21.15.69:0/626434804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 msgr2=0x55f3c30ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:24.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88bca00640 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 0x55f3c30ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.857+0000 7f88bca00640 1 -- 172.21.15.69:0/626434804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c2e097d0 con 0x55f3c30c7340 2024-03-19T01:14:24.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.858+0000 7f88bd400640 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 0x55f3c30ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:14:24.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.858+0000 7f88bca00640 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cd4f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f88a40ffaa0 tx=0x7f88a4009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419793a3065e5fff server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:24.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.858+0000 7f889ea00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:14:24.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.858+0000 7f88c2ba6740 10 client.0 _ll_register_callbacks cb 0x55f3c3023170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:24.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.858+0000 7f88c2ba6740 1 -- 172.21.15.69:0/626434804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88ac01eaa0 con 0x55f3c30c7340 2024-03-19T01:14:24.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.858+0000 7f889ea00640 1 -- 172.21.15.69:0/626434804 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40d2610 con 0x55f3c30c7340 2024-03-19T01:14:24.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.858+0000 7f889ea00640 1 -- 172.21.15.69:0/626434804 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f88a40d67d0 con 0x55f3c30c7340 2024-03-19T01:14:24.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.859+0000 7f889ea00640 1 -- 172.21.15.69:0/626434804 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f88a40d69d0 con 0x55f3c30c7340 2024-03-19T01:14:24.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.859+0000 7f88c2ba6740 10 client.16392 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:24.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.859+0000 7f88c2ba6740 1 -- 172.21.15.69:0/626434804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55f3c30dea90 con 0x55f3c30c7340 2024-03-19T01:14:24.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.859+0000 7f889ea00640 1 -- 172.21.15.69:0/626434804 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f88a40d2120 con 0x55f3c30c7340 2024-03-19T01:14:24.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.859+0000 7f88c2ba6740 10 client.16392 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:24.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.859+0000 7f88c2ba6740 10 client.16392 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:14:24.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.859+0000 7f88c2ba6740 20 client.16392 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:24.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.859+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55f3c30e02b0 0x55f3c30e2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.859+0000 7f88c2ba6740 4 client.16392 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:14:24.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.859+0000 7f88c2ba6740 1 -- 172.21.15.69:0/626434804 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3c30388c0 con 0x55f3c30e02b0 2024-03-19T01:14:24.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.860+0000 7f88bd400640 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55f3c30e02b0 0x55f3c30e2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.861+0000 7f88bd400640 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55f3c30e02b0 0x55f3c30e2770 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f559cdf96060046d server_cookie=43e8cf13c3456ab in_seq=0 out_seq=0 2024-03-19T01:14:24.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.861+0000 7f889ea00640 10 client.16392 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:14:24.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f889ea00640 1 -- 172.21.15.69:0/626434804 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f3c30388c0 con 0x55f3c30e02b0 2024-03-19T01:14:24.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f889ea00640 10 client.16392 handle_command_reply: tid=0 2024-03-19T01:14:24.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f88c2ba6740 1 client.16392 shutdown 2024-03-19T01:14:24.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f88c2ba6740 1 -- 172.21.15.69:0/626434804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 msgr2=0x55f3c30cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cd4f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f88a40ffaa0 tx=0x7f88a4009440 comp rx=0 tx=0).stop 2024-03-19T01:14:24.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f88c2ba6740 1 -- 172.21.15.69:0/626434804 shutdown_connections 2024-03-19T01:14:24.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55f3c30e02b0 0x55f3c30e2770 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cd4f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/626434804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 0x55f3c30ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f88c2ba6740 1 -- 172.21.15.69:0/626434804 >> 172.21.15.69:0/626434804 conn(0x55f3c30c1150 msgr2=0x55f3c30c2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:24.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.862+0000 7f889ea00640 0 client.16392 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:14:24.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.863+0000 7f88c2ba6740 1 -- 172.21.15.69:0/626434804 shutdown_connections 2024-03-19T01:14:24.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.863+0000 7f88c2ba6740 1 -- 172.21.15.69:0/626434804 wait complete. 2024-03-19T01:14:24.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.863+0000 7f88c2ba6740 20 client.16392 trim_cache size 0 max 16384 2024-03-19T01:14:24.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.889+0000 7f88c2ba6740 1 Processor -- start 2024-03-19T01:14:24.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.889+0000 7f88c2ba6740 1 -- start start 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 0x55f3c30dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88c2ba6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81acc00 con 0x55f3c30c7340 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88c2ba6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81ae8c0 con 0x55f3c30c6a20 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88c2ba6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81ad2d0 con 0x55f3c30cff20 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88bde00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 0x55f3c30dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88bde00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30e8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55974/0 (socket says 172.21.15.69:55974) 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 0x55f3c30dbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33562/0 (socket says 172.21.15.69:33562) 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88bde00640 1 -- 172.21.15.69:0/2860723061 learned_addr learned my addr 172.21.15.69:0/2860723061 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88bca00640 1 --2- 172.21.15.69:0/2860723061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88bde00640 1 -- 172.21.15.69:0/2860723061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 msgr2=0x55f3c30dbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88bde00640 1 --2- 172.21.15.69:0/2860723061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 0x55f3c30dbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88bde00640 1 -- 172.21.15.69:0/2860723061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 msgr2=0x55f3c30cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88bde00640 1 --2- 172.21.15.69:0/2860723061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.890+0000 7f88bde00640 1 -- 172.21.15.69:0/2860723061 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c30e2df0 con 0x55f3c30cff20 2024-03-19T01:14:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.891+0000 7f88bde00640 1 --2- 172.21.15.69:0/2860723061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30e8050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f88b000e4d0 tx=0x7f88b000e8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de0a848b10e12713 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:24.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.894+0000 7f88a9c00640 1 -- 172.21.15.69:0/2860723061 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0004d20 con 0x55f3c30cff20 2024-03-19T01:14:24.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.894+0000 7f88a9c00640 1 -- 172.21.15.69:0/2860723061 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b0003e50 con 0x55f3c30cff20 2024-03-19T01:14:24.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.894+0000 7f88a9c00640 1 -- 172.21.15.69:0/2860723061 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0004270 con 0x55f3c30cff20 2024-03-19T01:14:24.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.895+0000 7f88c2ba6740 1 -- 172.21.15.69:0/2860723061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 msgr2=0x55f3c30e8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.895+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/2860723061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30e8050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f88b000e4d0 tx=0x7f88b000e8d0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.895+0000 7f88c2ba6740 1 -- 172.21.15.69:0/2860723061 shutdown_connections 2024-03-19T01:14:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.895+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/2860723061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30e8050 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.895+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/2860723061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c7340 0x55f3c30cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.895+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/2860723061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c6a20 0x55f3c30dbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.895+0000 7f88c2ba6740 1 -- 172.21.15.69:0/2860723061 >> 172.21.15.69:0/2860723061 conn(0x55f3c30c1150 msgr2=0x55f3c30c01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.895+0000 7f88c2ba6740 1 -- 172.21.15.69:0/2860723061 shutdown_connections 2024-03-19T01:14:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.895+0000 7f88c2ba6740 1 -- 172.21.15.69:0/2860723061 wait complete. 2024-03-19T01:14:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.896+0000 7f88c2ba6740 1 -- start start 2024-03-19T01:14:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.896+0000 7f88c2ba6740 1 Processor -- start 2024-03-19T01:14:24.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c6a20 0x55f3c30d8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c7340 0x55f3c30d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88c2ba6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88c2ba6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81ae130 con 0x55f3c30c6a20 2024-03-19T01:14:24.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88c2ba6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81ad9a0 con 0x55f3c30c7340 2024-03-19T01:14:24.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88c2ba6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8068aa0 con 0x55f3c30cff20 2024-03-19T01:14:24.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88c2ba6740 10 client.0 _ll_register_callbacks cb 0x7f88a40ce090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:24.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c7340 0x55f3c30d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c7340 0x55f3c30d37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33566/0 (socket says 172.21.15.69:33566) 2024-03-19T01:14:24.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88bca00640 1 -- 172.21.15.69:0/4093121338 learned_addr learned my addr 172.21.15.69:0/4093121338 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:24.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88bd400640 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c6a20 0x55f3c30d8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.897+0000 7f88bca00640 1 -- 172.21.15.69:0/4093121338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 msgr2=0x55f3c30d3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:24.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.898+0000 7f88bca00640 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.898+0000 7f88bca00640 1 -- 172.21.15.69:0/4093121338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c6a20 msgr2=0x55f3c30d8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:24.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.898+0000 7f88bca00640 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c6a20 0x55f3c30d8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.898+0000 7f88bca00640 1 -- 172.21.15.69:0/4093121338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c30d4590 con 0x55f3c30c7340 2024-03-19T01:14:24.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.898+0000 7f88bd400640 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c6a20 0x55f3c30d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:14:24.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.898+0000 7f889ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:14:24.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.898+0000 7f88bca00640 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c7340 0x55f3c30d37e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f88a40d2580 tx=0x7f88a4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ef46458c2005732 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.898+0000 7f889ea00640 1 -- 172.21.15.69:0/4093121338 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40d2610 con 0x55f3c30c7340 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.898+0000 7f889ea00640 1 -- 172.21.15.69:0/4093121338 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f88a40b76d0 con 0x55f3c30c7340 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.898+0000 7f88c2ba6740 1 -- 172.21.15.69:0/4093121338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b0003e50 con 0x55f3c30c7340 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.899+0000 7f889ea00640 1 -- 172.21.15.69:0/4093121338 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f88a40d6c60 con 0x55f3c30c7340 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.899+0000 7f88c2ba6740 10 client.26594 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.899+0000 7f88c2ba6740 1 -- 172.21.15.69:0/4093121338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f3c3146720 con 0x55f3c30c7340 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.899+0000 7f889ea00640 1 -- 172.21.15.69:0/4093121338 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b0003e50 con 0x55f3c30c7340 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.900+0000 7f889ea00640 1 -- 172.21.15.69:0/4093121338 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f88a40cd910 con 0x55f3c30c7340 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.900+0000 7f88c2ba6740 10 client.26594 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.900+0000 7f88c2ba6740 10 client.26594 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.900+0000 7f88c2ba6740 20 client.26594 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.900+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55f3c3146de0 0x55f3c3149220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:24.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.900+0000 7f88c2ba6740 4 client.26594 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:14:24.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.900+0000 7f88c2ba6740 1 -- 172.21.15.69:0/4093121338 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3c3032c10 con 0x55f3c3146de0 2024-03-19T01:14:24.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.900+0000 7f88bd400640 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55f3c3146de0 0x55f3c3149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:24.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.901+0000 7f88bd400640 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55f3c3146de0 0x55f3c3149220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad02b636e6098f8e server_cookie=738f3b8fb2e3a668 in_seq=0 out_seq=0 2024-03-19T01:14:24.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.901+0000 7f889ea00640 10 client.26594 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:14:24.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.902+0000 7f889ea00640 1 -- 172.21.15.69:0/4093121338 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3c3032c10 con 0x55f3c3146de0 2024-03-19T01:14:24.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.902+0000 7f889ea00640 10 client.26594 handle_command_reply: tid=0 2024-03-19T01:14:24.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.902+0000 7f88c2ba6740 1 client.26594 shutdown 2024-03-19T01:14:24.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.902+0000 7f88c2ba6740 1 -- 172.21.15.69:0/4093121338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c7340 msgr2=0x55f3c30d37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.902+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c7340 0x55f3c30d37e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f88a40d2580 tx=0x7f88a4005f90 comp rx=0 tx=0).stop 2024-03-19T01:14:24.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.903+0000 7f88c2ba6740 1 -- 172.21.15.69:0/4093121338 shutdown_connections 2024-03-19T01:14:24.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.903+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55f3c3146de0 0x55f3c3149220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.903+0000 7f889ea00640 0 client.26594 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:14:24.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.903+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f3c30cff20 0x55f3c30d3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.903+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c30c7340 0x55f3c30d37e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.903+0000 7f88c2ba6740 1 --2- 172.21.15.69:0/4093121338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3c30c6a20 0x55f3c30d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.903+0000 7f88c2ba6740 1 -- 172.21.15.69:0/4093121338 >> 172.21.15.69:0/4093121338 conn(0x55f3c30c1150 msgr2=0x55f3c30c22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:24.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.903+0000 7f88c2ba6740 1 -- 172.21.15.69:0/4093121338 shutdown_connections 2024-03-19T01:14:24.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.903+0000 7f88c2ba6740 1 -- 172.21.15.69:0/4093121338 wait complete. 2024-03-19T01:14:24.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.903+0000 7f88c2ba6740 20 client.26594 trim_cache size 0 max 16384 2024-03-19T01:14:24.905 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:14:24.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.904+0000 7f88bfe00640 1 -- 172.21.15.69:0/1559532974 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88a00a2b90 msgr2=0x7f88a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.904+0000 7f88bfe00640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88a00a2b90 0x7f88a00a5050 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009b50 tx=0x7f88ac005f90 comp rx=0 tx=0).stop 2024-03-19T01:14:24.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.904+0000 7f88bfe00640 1 -- 172.21.15.69:0/1559532974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b8104380 msgr2=0x7f88b810fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:24.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.904+0000 7f88bfe00640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b8104380 0x7f88b810fc80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f88a402c1a0 tx=0x7f88a4034930 comp rx=0 tx=0).stop 2024-03-19T01:14:24.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.905+0000 7f88bfe00640 1 -- 172.21.15.69:0/1559532974 shutdown_connections 2024-03-19T01:14:24.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.905+0000 7f88bfe00640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88b81099d0 0x7f88b81a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.905+0000 7f88bfe00640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88a00a2b90 0x7f88a00a5050 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.905+0000 7f88bfe00640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b8104380 0x7f88b810fc80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.905+0000 7f88bfe00640 1 --2- 172.21.15.69:0/1559532974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88b81039c0 0x7f88b810f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:24.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.905+0000 7f88bfe00640 1 -- 172.21.15.69:0/1559532974 >> 172.21.15.69:0/1559532974 conn(0x7f88b80ff470 msgr2=0x7f88b8077b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:24.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.905+0000 7f88bfe00640 1 -- 172.21.15.69:0/1559532974 shutdown_connections 2024-03-19T01:14:24.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:24.905+0000 7f88bfe00640 1 -- 172.21.15.69:0/1559532974 wait complete. 2024-03-19T01:14:24.926 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:14:25.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.135+0000 7f98eb000640 1 -- 172.21.15.69:0/3086345822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e4109a40 msgr2=0x7f98e4109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.135+0000 7f98eb000640 1 --2- 172.21.15.69:0/3086345822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e4109a40 0x7f98e4109e20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009cd0 tx=0x7f98d402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.135+0000 7f98eb000640 1 -- 172.21.15.69:0/3086345822 shutdown_connections 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.135+0000 7f98eb000640 1 --2- 172.21.15.69:0/3086345822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98e41043f0 0x7f98e4110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.135+0000 7f98eb000640 1 --2- 172.21.15.69:0/3086345822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98e4103a30 0x7f98e4103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.135+0000 7f98eb000640 1 --2- 172.21.15.69:0/3086345822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e4109a40 0x7f98e4109e20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.135+0000 7f98eb000640 1 -- 172.21.15.69:0/3086345822 >> 172.21.15.69:0/3086345822 conn(0x7f98e40ff4e0 msgr2=0x7f98e4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.135+0000 7f98eb000640 1 -- 172.21.15.69:0/3086345822 shutdown_connections 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.135+0000 7f98eb000640 1 -- 172.21.15.69:0/3086345822 wait complete. 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.136+0000 7f98eb000640 1 Processor -- start 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.136+0000 7f98eb000640 1 -- start start 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.136+0000 7f98eb000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e4103a30 0x7f98e419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.136+0000 7f98eb000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98e41043f0 0x7f98e419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.136+0000 7f98eb000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98e4109a40 0x7f98e419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.136+0000 7f98eb000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4113e50 con 0x7f98e4109a40 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.136+0000 7f98eb000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4113cd0 con 0x7f98e41043f0 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.136+0000 7f98eb000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4113fd0 con 0x7f98e4103a30 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.136+0000 7f98e3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e4103a30 0x7f98e419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.136+0000 7f98db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98e41043f0 0x7f98e419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98e3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e4103a30 0x7f98e419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:56000/0 (socket says 172.21.15.69:56000) 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98e3e00640 1 -- 172.21.15.69:0/1415471249 learned_addr learned my addr 172.21.15.69:0/1415471249 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98e9000640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98e4109a40 0x7f98e419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98e3e00640 1 -- 172.21.15.69:0/1415471249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98e41043f0 msgr2=0x7f98e419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98e3e00640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98e41043f0 0x7f98e419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98e3e00640 1 -- 172.21.15.69:0/1415471249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98e4109a40 msgr2=0x7f98e419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98e3e00640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98e4109a40 0x7f98e419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98e3e00640 1 -- 172.21.15.69:0/1415471249 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e419f3a0 con 0x7f98e4103a30 2024-03-19T01:14:25.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98e9000640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98e4109a40 0x7f98e419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:25.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98e3e00640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e4103a30 0x7f98e419a3b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f98d4004af0 tx=0x7f98d4032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98e1600640 1 -- 172.21.15.69:0/1415471249 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4041020 con 0x7f98e4103a30 2024-03-19T01:14:25.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.137+0000 7f98eb000640 1 -- 172.21.15.69:0/1415471249 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e419f630 con 0x7f98e4103a30 2024-03-19T01:14:25.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.138+0000 7f98e1600640 1 -- 172.21.15.69:0/1415471249 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98d4034cd0 con 0x7f98e4103a30 2024-03-19T01:14:25.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.138+0000 7f98e1600640 1 -- 172.21.15.69:0/1415471249 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4039040 con 0x7f98e4103a30 2024-03-19T01:14:25.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.138+0000 7f98eb000640 1 -- 172.21.15.69:0/1415471249 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41a6f20 con 0x7f98e4103a30 2024-03-19T01:14:25.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.141+0000 7f98e1600640 1 -- 172.21.15.69:0/1415471249 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f98d4044020 con 0x7f98e4103a30 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.142+0000 7f98e1600640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98c80a29f0 0x7f98c80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.142+0000 7f98e1600640 1 -- 172.21.15.69:0/1415471249 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f98d40f0e30 con 0x7f98e4103a30 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.142+0000 7f98db400640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98c80a29f0 0x7f98c80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.142+0000 7f98edc4f740 1 Processor -- start 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98edc4f740 1 -- start start 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98db400640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98c80a29f0 0x7f98c80a4eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f98e4109130 tx=0x7f98cc00b410 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e03a20 0x55bb77e03e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e04340 0x55bb77e0c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e0cf20 0x55bb77e0f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98edc4f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb77d68380 con 0x55bb77e0cf20 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98edc4f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb77d10d20 con 0x55bb77e03a20 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98edc4f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bb77d64420 con 0x55bb77e04340 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e03a20 0x55bb77e03e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e03a20 0x55bb77e03e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33598/0 (socket says 172.21.15.69:33598) 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98e3e00640 1 -- 172.21.15.69:0/856479359 learned_addr learned my addr 172.21.15.69:0/856479359 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98db400640 1 --2- 172.21.15.69:0/856479359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e04340 0x55bb77e0c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.143+0000 7f98e9000640 1 --2- 172.21.15.69:0/856479359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e0cf20 0x55bb77e0f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.144+0000 7f98e3e00640 1 -- 172.21.15.69:0/856479359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e04340 msgr2=0x55bb77e0c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.144+0000 7f98e3e00640 1 --2- 172.21.15.69:0/856479359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e04340 0x55bb77e0c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.144+0000 7f98e3e00640 1 -- 172.21.15.69:0/856479359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e0cf20 msgr2=0x55bb77e0f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.144+0000 7f98e3e00640 1 --2- 172.21.15.69:0/856479359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e0cf20 0x55bb77e0f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.144+0000 7f98e3e00640 1 -- 172.21.15.69:0/856479359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb77a7c3f0 con 0x55bb77e03a20 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.144+0000 7f98e3e00640 1 --2- 172.21.15.69:0/856479359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e03a20 0x55bb77e03e00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f98d40bce80 tx=0x7f98d402c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f798b61ab75844ea server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.144+0000 7f98d9600640 1 -- 172.21.15.69:0/856479359 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d40bccc0 con 0x55bb77e03a20 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.144+0000 7f98d9600640 1 -- 172.21.15.69:0/856479359 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98d4034920 con 0x55bb77e03a20 2024-03-19T01:14:25.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.144+0000 7f98d9600640 1 -- 172.21.15.69:0/856479359 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d40bcae0 con 0x55bb77e03a20 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.145+0000 7f98edc4f740 1 -- 172.21.15.69:0/856479359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e03a20 msgr2=0x55bb77e03e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.145+0000 7f98edc4f740 1 --2- 172.21.15.69:0/856479359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e03a20 0x55bb77e03e00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f98d40bce80 tx=0x7f98d402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.145+0000 7f98edc4f740 1 -- 172.21.15.69:0/856479359 shutdown_connections 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.145+0000 7f98edc4f740 1 --2- 172.21.15.69:0/856479359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e0cf20 0x55bb77e0f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.145+0000 7f98edc4f740 1 --2- 172.21.15.69:0/856479359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e04340 0x55bb77e0c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.145+0000 7f98edc4f740 1 --2- 172.21.15.69:0/856479359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e03a20 0x55bb77e03e00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.145+0000 7f98edc4f740 1 -- 172.21.15.69:0/856479359 >> 172.21.15.69:0/856479359 conn(0x55bb77dfe150 msgr2=0x55bb77dfe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.145+0000 7f98edc4f740 1 -- 172.21.15.69:0/856479359 shutdown_connections 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.145+0000 7f98edc4f740 1 -- 172.21.15.69:0/856479359 wait complete. 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.146+0000 7f98edc4f740 1 -- start start 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.146+0000 7f98edc4f740 1 Processor -- start 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e03a20 0x55bb77e09fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e04340 0x55bb77e0a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e0cf20 0x55bb77e0aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98edc4f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb77e0fad0 con 0x55bb77e0cf20 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98edc4f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb77d24160 con 0x55bb77e04340 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98edc4f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bb77e0fc50 con 0x55bb77e03a20 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e04340 0x55bb77e0a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98e3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e03a20 0x55bb77e09fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e04340 0x55bb77e0a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33600/0 (socket says 172.21.15.69:33600) 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98db400640 1 -- 172.21.15.69:0/2598761519 learned_addr learned my addr 172.21.15.69:0/2598761519 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98e3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e03a20 0x55bb77e09fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:56012/0 (socket says 172.21.15.69:56012) 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98e9000640 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e0cf20 0x55bb77e0aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98e3e00640 1 -- 172.21.15.69:0/2598761519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e04340 msgr2=0x55bb77e0a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98e3e00640 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e04340 0x55bb77e0a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98e3e00640 1 -- 172.21.15.69:0/2598761519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e0cf20 msgr2=0x55bb77e0aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98e3e00640 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e0cf20 0x55bb77e0aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98e3e00640 1 -- 172.21.15.69:0/2598761519 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb77b467d0 con 0x55bb77e03a20 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98e9000640 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e0cf20 0x55bb77e0aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98edc4f740 10 client.0 _ll_register_callbacks cb 0x55bb77d60170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.147+0000 7f98edc4f740 1 -- 172.21.15.69:0/2598761519 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d4034920 con 0x55bb77e03a20 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98e3e00640 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e03a20 0x55bb77e09fb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f98d40bce80 tx=0x7f98d4003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c35fa090e8d1b429 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98d2000640 10 client.26590 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98d2000640 1 -- 172.21.15.69:0/2598761519 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d40bccc0 con 0x55bb77e03a20 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98d2000640 1 -- 172.21.15.69:0/2598761519 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f98d40c5690 con 0x55bb77e03a20 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98d2000640 1 -- 172.21.15.69:0/2598761519 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f98d40bd180 con 0x55bb77e03a20 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98edc4f740 10 client.26590 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98edc4f740 1 -- 172.21.15.69:0/2598761519 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55bb77e0fbc0 con 0x55bb77e03a20 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98d2000640 1 -- 172.21.15.69:0/2598761519 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f98d403c030 con 0x55bb77e03a20 2024-03-19T01:14:25.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98edc4f740 10 client.26590 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98edc4f740 10 client.26590 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:14:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98edc4f740 20 client.26590 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bb77e1cdc0 0x55bb77e1f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98edc4f740 4 client.26590 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:14:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.148+0000 7f98edc4f740 1 -- 172.21.15.69:0/2598761519 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb77d758c0 con 0x55bb77e1cdc0 2024-03-19T01:14:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.149+0000 7f98db400640 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bb77e1cdc0 0x55bb77e1f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.149+0000 7f98db400640 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bb77e1cdc0 0x55bb77e1f230 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fe0506ae0005f0e server_cookie=389f134ec4ddbf37 in_seq=0 out_seq=0 2024-03-19T01:14:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.149+0000 7f98d2000640 10 client.26590 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:14:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.150+0000 7f98d2000640 1 -- 172.21.15.69:0/2598761519 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bb77d758c0 con 0x55bb77e1cdc0 2024-03-19T01:14:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.150+0000 7f98d2000640 10 client.26590 handle_command_reply: tid=0 2024-03-19T01:14:25.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.150+0000 7f98edc4f740 1 client.26590 shutdown 2024-03-19T01:14:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98edc4f740 1 -- 172.21.15.69:0/2598761519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e03a20 msgr2=0x55bb77e09fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e03a20 0x55bb77e09fb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f98d40bce80 tx=0x7f98d4003c30 comp rx=0 tx=0).stop 2024-03-19T01:14:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98edc4f740 1 -- 172.21.15.69:0/2598761519 shutdown_connections 2024-03-19T01:14:25.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e0cf20 0x55bb77e0aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bb77e1cdc0 0x55bb77e1f230 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e04340 0x55bb77e0a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2598761519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e03a20 0x55bb77e09fb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98edc4f740 1 -- 172.21.15.69:0/2598761519 >> 172.21.15.69:0/2598761519 conn(0x55bb77dfe150 msgr2=0x55bb77dffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98d2000640 0 client.26590 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:14:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98edc4f740 1 -- 172.21.15.69:0/2598761519 shutdown_connections 2024-03-19T01:14:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98edc4f740 1 -- 172.21.15.69:0/2598761519 wait complete. 2024-03-19T01:14:25.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.151+0000 7f98edc4f740 20 client.26590 trim_cache size 0 max 16384 2024-03-19T01:14:25.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.173+0000 7f98edc4f740 1 Processor -- start 2024-03-19T01:14:25.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.173+0000 7f98edc4f740 1 -- start start 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.173+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77dfe150 0x55bb77e0c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.173+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e03a20 0x55bb77e08880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.173+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e04340 0x55bb77e187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.173+0000 7f98edc4f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41b0e30 con 0x55bb77e03a20 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.173+0000 7f98edc4f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41b2af0 con 0x55bb77dfe150 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.173+0000 7f98edc4f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41b1500 con 0x55bb77e04340 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.173+0000 7f98e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77dfe150 0x55bb77e0c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98e9000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e04340 0x55bb77e187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77dfe150 0x55bb77e0c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33606/0 (socket says 172.21.15.69:33606) 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.173+0000 7f98db400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e03a20 0x55bb77e08880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98e3e00640 1 -- 172.21.15.69:0/3399062263 learned_addr learned my addr 172.21.15.69:0/3399062263 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98e3e00640 1 -- 172.21.15.69:0/3399062263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e04340 msgr2=0x55bb77e187e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98e3e00640 1 --2- 172.21.15.69:0/3399062263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e04340 0x55bb77e187e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98e3e00640 1 -- 172.21.15.69:0/3399062263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e03a20 msgr2=0x55bb77e08880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98e3e00640 1 --2- 172.21.15.69:0/3399062263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e03a20 0x55bb77e08880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98e3e00640 1 -- 172.21.15.69:0/3399062263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb77e08f60 con 0x55bb77dfe150 2024-03-19T01:14:25.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98e3e00640 1 --2- 172.21.15.69:0/3399062263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77dfe150 0x55bb77e0c6e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f98d4037040 tx=0x7f98d4009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9bd2551bac92f8a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98d8c00640 1 -- 172.21.15.69:0/3399062263 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d40bccc0 con 0x55bb77dfe150 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98d8c00640 1 -- 172.21.15.69:0/3399062263 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98d4034920 con 0x55bb77dfe150 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.174+0000 7f98d8c00640 1 -- 172.21.15.69:0/3399062263 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d40b9420 con 0x55bb77dfe150 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.175+0000 7f98edc4f740 1 -- 172.21.15.69:0/3399062263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77dfe150 msgr2=0x55bb77e0c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.175+0000 7f98edc4f740 1 --2- 172.21.15.69:0/3399062263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77dfe150 0x55bb77e0c6e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f98d4037040 tx=0x7f98d4009800 comp rx=0 tx=0).stop 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.175+0000 7f98edc4f740 1 -- 172.21.15.69:0/3399062263 shutdown_connections 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.175+0000 7f98edc4f740 1 --2- 172.21.15.69:0/3399062263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77e04340 0x55bb77e187e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.175+0000 7f98edc4f740 1 --2- 172.21.15.69:0/3399062263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e03a20 0x55bb77e08880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.175+0000 7f98edc4f740 1 --2- 172.21.15.69:0/3399062263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77dfe150 0x55bb77e0c6e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.175+0000 7f98edc4f740 1 -- 172.21.15.69:0/3399062263 >> 172.21.15.69:0/3399062263 conn(0x55bb77e05980 msgr2=0x55bb77dfd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.175+0000 7f98edc4f740 1 -- 172.21.15.69:0/3399062263 shutdown_connections 2024-03-19T01:14:25.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.175+0000 7f98edc4f740 1 -- 172.21.15.69:0/3399062263 wait complete. 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98edc4f740 1 -- start start 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98edc4f740 1 Processor -- start 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77dfe150 0x55bb77e10d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e03a20 0x55bb77e112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98edc4f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e04340 0x55bb77e82130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98edc4f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41b2360 con 0x55bb77e03a20 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98edc4f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41b1bd0 con 0x55bb77e04340 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98edc4f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41aee10 con 0x55bb77dfe150 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98edc4f740 10 client.0 _ll_register_callbacks cb 0x7f98d4009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98e3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77dfe150 0x55bb77e10d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98db400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e03a20 0x55bb77e112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98e3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77dfe150 0x55bb77e10d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:56032/0 (socket says 172.21.15.69:56032) 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98e3e00640 1 -- 172.21.15.69:0/2422958145 learned_addr learned my addr 172.21.15.69:0/2422958145 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.176+0000 7f98e9000640 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e04340 0x55bb77e82130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98e3e00640 1 -- 172.21.15.69:0/2422958145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e04340 msgr2=0x55bb77e82130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98e3e00640 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e04340 0x55bb77e82130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98e3e00640 1 -- 172.21.15.69:0/2422958145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e03a20 msgr2=0x55bb77e112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98e3e00640 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e03a20 0x55bb77e112c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98e3e00640 1 -- 172.21.15.69:0/2422958145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb77e82670 con 0x55bb77dfe150 2024-03-19T01:14:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98db400640 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e03a20 0x55bb77e112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:14:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98e3e00640 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77dfe150 0x55bb77e10d80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f98d40bd110 tx=0x7f98d4032b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e49ecdc6d937cb85 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98d2000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:14:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98d2000640 1 -- 172.21.15.69:0/2422958145 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d40b9420 con 0x55bb77dfe150 2024-03-19T01:14:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98d2000640 1 -- 172.21.15.69:0/2422958145 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f98d40b8ed0 con 0x55bb77dfe150 2024-03-19T01:14:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98edc4f740 1 -- 172.21.15.69:0/2422958145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d4034920 con 0x55bb77dfe150 2024-03-19T01:14:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98d2000640 1 -- 172.21.15.69:0/2422958145 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f98d40bccc0 con 0x55bb77dfe150 2024-03-19T01:14:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98edc4f740 10 client.26596 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.177+0000 7f98edc4f740 1 -- 172.21.15.69:0/2422958145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bb77e82670 con 0x55bb77dfe150 2024-03-19T01:14:25.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.178+0000 7f98d2000640 1 -- 172.21.15.69:0/2422958145 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98d4034920 con 0x55bb77dfe150 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.178+0000 7f98d2000640 1 -- 172.21.15.69:0/2422958145 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f98d4032590 con 0x55bb77dfe150 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.178+0000 7f98edc4f740 10 client.26596 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.178+0000 7f98edc4f740 10 client.26596 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.178+0000 7f98edc4f740 20 client.26596 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.178+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bb77e0cf20 0x55bb77e85b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.178+0000 7f98edc4f740 4 client.26596 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.178+0000 7f98edc4f740 1 -- 172.21.15.69:0/2422958145 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb77d6fc10 con 0x55bb77e0cf20 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.179+0000 7f98db400640 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bb77e0cf20 0x55bb77e85b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.179+0000 7f98db400640 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bb77e0cf20 0x55bb77e85b80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e41da2729ab548b9 server_cookie=18c9ad2eddeb77aa in_seq=0 out_seq=0 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.179+0000 7f98d2000640 10 client.26596 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.179+0000 7f98d2000640 1 -- 172.21.15.69:0/2422958145 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb77d6fc10 con 0x55bb77e0cf20 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.179+0000 7f98d2000640 10 client.26596 handle_command_reply: tid=0 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.179+0000 7f98edc4f740 1 client.26596 shutdown 2024-03-19T01:14:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98edc4f740 1 -- 172.21.15.69:0/2422958145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77dfe150 msgr2=0x55bb77e10d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77dfe150 0x55bb77e10d80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f98d40bd110 tx=0x7f98d4032b00 comp rx=0 tx=0).stop 2024-03-19T01:14:25.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98edc4f740 1 -- 172.21.15.69:0/2422958145 shutdown_connections 2024-03-19T01:14:25.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bb77e0cf20 0x55bb77e85b80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb77e04340 0x55bb77e82130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb77e03a20 0x55bb77e112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98d2000640 0 client.26596 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:14:25.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98edc4f740 1 --2- 172.21.15.69:0/2422958145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb77dfe150 0x55bb77e10d80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98edc4f740 1 -- 172.21.15.69:0/2422958145 >> 172.21.15.69:0/2422958145 conn(0x55bb77e05980 msgr2=0x55bb77dfd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98edc4f740 1 -- 172.21.15.69:0/2422958145 shutdown_connections 2024-03-19T01:14:25.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98edc4f740 1 -- 172.21.15.69:0/2422958145 wait complete. 2024-03-19T01:14:25.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98edc4f740 20 client.26596 trim_cache size 0 max 16384 2024-03-19T01:14:25.185 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:14:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98eb000640 1 -- 172.21.15.69:0/1415471249 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98c80a29f0 msgr2=0x7f98c80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98eb000640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98c80a29f0 0x7f98c80a4eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f98e4109130 tx=0x7f98cc00b410 comp rx=0 tx=0).stop 2024-03-19T01:14:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98eb000640 1 -- 172.21.15.69:0/1415471249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e4103a30 msgr2=0x7f98e419a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.180+0000 7f98eb000640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e4103a30 0x7f98e419a3b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f98d4004af0 tx=0x7f98d4032ea0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.181+0000 7f98eb000640 1 -- 172.21.15.69:0/1415471249 shutdown_connections 2024-03-19T01:14:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.181+0000 7f98eb000640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98e4109a40 0x7f98e419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.181+0000 7f98eb000640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98c80a29f0 0x7f98c80a4eb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.181+0000 7f98eb000640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98e41043f0 0x7f98e419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.181+0000 7f98eb000640 1 --2- 172.21.15.69:0/1415471249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e4103a30 0x7f98e419a3b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.181+0000 7f98eb000640 1 -- 172.21.15.69:0/1415471249 >> 172.21.15.69:0/1415471249 conn(0x7f98e40ff4e0 msgr2=0x7f98e40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.181+0000 7f98eb000640 1 -- 172.21.15.69:0/1415471249 shutdown_connections 2024-03-19T01:14:25.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.181+0000 7f98eb000640 1 -- 172.21.15.69:0/1415471249 wait complete. 2024-03-19T01:14:25.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:24 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3797553451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:14:25.198 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:14:25.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:24 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3797553451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:14:25.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:24 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3797553451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:14:25.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.390+0000 7f64bdc00640 1 -- 172.21.15.69:0/346577328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b81017b0 msgr2=0x7f64b8101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.390+0000 7f64bdc00640 1 --2- 172.21.15.69:0/346577328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b81017b0 0x7f64b8101c30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009900 tx=0x7f64a002bc30 comp rx=0 tx=0).stop 2024-03-19T01:14:25.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.390+0000 7f64bdc00640 1 -- 172.21.15.69:0/346577328 shutdown_connections 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.390+0000 7f64bdc00640 1 --2- 172.21.15.69:0/346577328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64b8102170 0x7f64b810e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.390+0000 7f64bdc00640 1 --2- 172.21.15.69:0/346577328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b81017b0 0x7f64b8101c30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.390+0000 7f64bdc00640 1 --2- 172.21.15.69:0/346577328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64b81077c0 0x7f64b8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.390+0000 7f64bdc00640 1 -- 172.21.15.69:0/346577328 >> 172.21.15.69:0/346577328 conn(0x7f64b8078c20 msgr2=0x7f64b8079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.390+0000 7f64bdc00640 1 -- 172.21.15.69:0/346577328 shutdown_connections 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.390+0000 7f64bdc00640 1 -- 172.21.15.69:0/346577328 wait complete. 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.391+0000 7f64bdc00640 1 Processor -- start 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.391+0000 7f64bdc00640 1 -- start start 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64bdc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64b81017b0 0x7f64b8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64bdc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64b8102170 0x7f64b81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64bdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b81077c0 0x7f64b819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64bdc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8111be0 con 0x7f64b8102170 2024-03-19T01:14:25.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64bdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8111a60 con 0x7f64b81077c0 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64bdc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8111d60 con 0x7f64b81017b0 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64b6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64b81017b0 0x7f64b8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64b6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64b81017b0 0x7f64b8198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:56048/0 (socket says 172.21.15.69:56048) 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64b6a00640 1 -- 172.21.15.69:0/118759672 learned_addr learned my addr 172.21.15.69:0/118759672 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64b7400640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b81077c0 0x7f64b819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64ae000640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64b8102170 0x7f64b81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64b6a00640 1 -- 172.21.15.69:0/118759672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b81077c0 msgr2=0x7f64b819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64b6a00640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b81077c0 0x7f64b819c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64b6a00640 1 -- 172.21.15.69:0/118759672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64b8102170 msgr2=0x7f64b81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64b6a00640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64b8102170 0x7f64b81985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64b6a00640 1 -- 172.21.15.69:0/118759672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b819d020 con 0x7f64b81017b0 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.392+0000 7f64ae000640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64b8102170 0x7f64b81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.393+0000 7f64b6a00640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64b81017b0 0x7f64b8198080 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009d50 tx=0x7f64a800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.393+0000 7f64afe00640 1 -- 172.21.15.69:0/118759672 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8003b70 con 0x7f64b81017b0 2024-03-19T01:14:25.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.393+0000 7f64bdc00640 1 -- 172.21.15.69:0/118759672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b819d310 con 0x7f64b81017b0 2024-03-19T01:14:25.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.393+0000 7f64afe00640 1 -- 172.21.15.69:0/118759672 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a8003d10 con 0x7f64b81017b0 2024-03-19T01:14:25.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.393+0000 7f64afe00640 1 -- 172.21.15.69:0/118759672 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8010a20 con 0x7f64b81017b0 2024-03-19T01:14:25.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.393+0000 7f64bdc00640 1 -- 172.21.15.69:0/118759672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b81a4ba0 con 0x7f64b81017b0 2024-03-19T01:14:25.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.396+0000 7f64afe00640 1 -- 172.21.15.69:0/118759672 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f64a8003ec0 con 0x7f64b81017b0 2024-03-19T01:14:25.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.397+0000 7f64afe00640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f649c0a2970 0x7f649c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.397+0000 7f64c09f2740 1 Processor -- start 2024-03-19T01:14:25.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.398+0000 7f64afe00640 1 -- 172.21.15.69:0/118759672 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f64a80cf970 con 0x7f64b81017b0 2024-03-19T01:14:25.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.398+0000 7f64c09f2740 1 -- start start 2024-03-19T01:14:25.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.398+0000 7f64ae000640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f649c0a2970 0x7f649c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.398+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f0fa20 0x55ca12f0fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.398+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.398+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 0x55ca12f1b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.398+0000 7f64c09f2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ca12e74380 con 0x55ca12f0fa20 2024-03-19T01:14:25.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.398+0000 7f64c09f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca12e1cd20 con 0x55ca12f18f20 2024-03-19T01:14:25.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.398+0000 7f64c09f2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ca12e70420 con 0x55ca12f10340 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.398+0000 7f64b6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f0fa20 0x55ca12f0fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 0x55ca12f1b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 0x55ca12f1b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33630/0 (socket says 172.21.15.69:33630) 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64b7400640 1 -- 172.21.15.69:0/3712450955 learned_addr learned my addr 172.21.15.69:0/3712450955 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64ae000640 1 --2- 172.21.15.69:0/3712450955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64ae000640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f649c0a2970 0x7f649c0a4e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f64a0006010 tx=0x7f64a0005a20 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64b7400640 1 -- 172.21.15.69:0/3712450955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 msgr2=0x55ca12f189e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64b7400640 1 --2- 172.21.15.69:0/3712450955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f189e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64b7400640 1 -- 172.21.15.69:0/3712450955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f0fa20 msgr2=0x55ca12f0fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64b7400640 1 --2- 172.21.15.69:0/3712450955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f0fa20 0x55ca12f0fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64b7400640 1 -- 172.21.15.69:0/3712450955 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca12b883f0 con 0x55ca12f18f20 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64ae000640 1 --2- 172.21.15.69:0/3712450955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64b7400640 1 --2- 172.21.15.69:0/3712450955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 0x55ca12f1b310 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009ca0 tx=0x7f64b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94972fd65130fb77 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64a7e00640 1 -- 172.21.15.69:0/3712450955 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b001e070 con 0x55ca12f18f20 2024-03-19T01:14:25.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64a7e00640 1 -- 172.21.15.69:0/3712450955 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64b0009e50 con 0x55ca12f18f20 2024-03-19T01:14:25.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.399+0000 7f64a7e00640 1 -- 172.21.15.69:0/3712450955 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b000eac0 con 0x55ca12f18f20 2024-03-19T01:14:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.400+0000 7f64c09f2740 1 -- 172.21.15.69:0/3712450955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 msgr2=0x55ca12f1b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.400+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3712450955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 0x55ca12f1b310 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009ca0 tx=0x7f64b0011810 comp rx=0 tx=0).stop 2024-03-19T01:14:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.400+0000 7f64c09f2740 1 -- 172.21.15.69:0/3712450955 shutdown_connections 2024-03-19T01:14:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.400+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3712450955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 0x55ca12f1b310 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.400+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3712450955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.400+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3712450955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f0fa20 0x55ca12f0fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.400+0000 7f64c09f2740 1 -- 172.21.15.69:0/3712450955 >> 172.21.15.69:0/3712450955 conn(0x55ca12f0a150 msgr2=0x55ca12f0a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.400+0000 7f64c09f2740 1 -- 172.21.15.69:0/3712450955 shutdown_connections 2024-03-19T01:14:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.400+0000 7f64c09f2740 1 -- 172.21.15.69:0/3712450955 wait complete. 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.401+0000 7f64c09f2740 1 -- start start 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.401+0000 7f64c09f2740 1 Processor -- start 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.401+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f15fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.401+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f10340 0x55ca12f164f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.401+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f18f20 0x55ca12f16a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.401+0000 7f64c09f2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ca12f20120 con 0x55ca12f10340 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.401+0000 7f64c09f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca12e30160 con 0x55ca12f0fa20 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.401+0000 7f64c09f2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ca12f202a0 con 0x55ca12f18f20 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.401+0000 7f64c09f2740 10 client.0 _ll_register_callbacks cb 0x55ca12e6c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f15fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f15fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33644/0 (socket says 172.21.15.69:33644) 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64b6a00640 1 -- 172.21.15.69:0/1544508538 learned_addr learned my addr 172.21.15.69:0/1544508538 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64b7400640 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f18f20 0x55ca12f16a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64ae000640 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f10340 0x55ca12f164f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64b6a00640 1 -- 172.21.15.69:0/1544508538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f18f20 msgr2=0x55ca12f16a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64b6a00640 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f18f20 0x55ca12f16a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64b6a00640 1 -- 172.21.15.69:0/1544508538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f10340 msgr2=0x55ca12f164f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64b6a00640 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f10340 0x55ca12f164f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64b6a00640 1 -- 172.21.15.69:0/1544508538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca12c527d0 con 0x55ca12f0fa20 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64ae000640 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f10340 0x55ca12f164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64b6a00640 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f15fb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f64a809bb40 tx=0x7f64a8009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d0535410dd7501 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64a4c00640 1 -- 172.21.15.69:0/1544508538 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a80b3030 con 0x55ca12f0fa20 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64a4c00640 1 -- 172.21.15.69:0/1544508538 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f64a80ccf20 con 0x55ca12f0fa20 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.402+0000 7f64c09f2740 1 -- 172.21.15.69:0/1544508538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64b0009e50 con 0x55ca12f0fa20 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.403+0000 7f64a4c00640 1 -- 172.21.15.69:0/1544508538 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f64a80a4350 con 0x55ca12f0fa20 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.403+0000 7f64c09f2740 10 client.26624 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.403+0000 7f64c09f2740 1 -- 172.21.15.69:0/1544508538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ca12f2c450 con 0x55ca12f0fa20 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.403+0000 7f64a4c00640 1 -- 172.21.15.69:0/1544508538 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64b0009e50 con 0x55ca12f0fa20 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.403+0000 7f64a4c00640 1 -- 172.21.15.69:0/1544508538 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f64a80041b0 con 0x55ca12f0fa20 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.403+0000 7f64c09f2740 10 client.26624 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.403+0000 7f64c09f2740 10 client.26624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.403+0000 7f64c09f2740 20 client.26624 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.403+0000 7f64c09f2740 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ca12f2d910 0x55ca12f2fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.403+0000 7f64c09f2740 4 client.26624 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.403+0000 7f64c09f2740 1 -- 172.21.15.69:0/1544508538 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca12e818c0 con 0x55ca12f2d910 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.404+0000 7f64ae000640 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ca12f2d910 0x55ca12f2fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.404+0000 7f64ae000640 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ca12f2d910 0x55ca12f2fd80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ed2b2acba589b77 server_cookie=b7fe031cab07874e in_seq=0 out_seq=0 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.404+0000 7f64a4c00640 10 client.26624 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64a4c00640 1 -- 172.21.15.69:0/1544508538 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ca12e818c0 con 0x55ca12f2d910 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64a4c00640 10 client.26624 handle_command_reply: tid=0 2024-03-19T01:14:25.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 1 client.26624 shutdown 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 1 -- 172.21.15.69:0/1544508538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 msgr2=0x55ca12f15fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f15fb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f64a809bb40 tx=0x7f64a8009960 comp rx=0 tx=0).stop 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 1 -- 172.21.15.69:0/1544508538 shutdown_connections 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ca12f2d910 0x55ca12f2fd80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f18f20 0x55ca12f16a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f10340 0x55ca12f164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 1 --2- 172.21.15.69:0/1544508538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f15fb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 1 -- 172.21.15.69:0/1544508538 >> 172.21.15.69:0/1544508538 conn(0x55ca12f0a150 msgr2=0x55ca12f0bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64a4c00640 0 client.26624 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 1 -- 172.21.15.69:0/1544508538 shutdown_connections 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 1 -- 172.21.15.69:0/1544508538 wait complete. 2024-03-19T01:14:25.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.405+0000 7f64c09f2740 20 client.26624 trim_cache size 0 max 16384 2024-03-19T01:14:25.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64c09f2740 1 Processor -- start 2024-03-19T01:14:25.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64c09f2740 1 -- start start 2024-03-19T01:14:25.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f28ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f18f20 0x55ca12f35650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64c09f2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81acb70 con 0x55ca12f18f20 2024-03-19T01:14:25.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64c09f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81ae830 con 0x55ca12f0fa20 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64c09f2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81ad240 con 0x55ca12f10340 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f186e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33656/0 (socket says 172.21.15.69:33656) 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64ae000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f28ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64b6a00640 1 -- 172.21.15.69:0/3721297973 learned_addr learned my addr 172.21.15.69:0/3721297973 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64b7400640 1 --2- 172.21.15.69:0/3721297973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f18f20 0x55ca12f35650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.420+0000 7f64b6a00640 1 -- 172.21.15.69:0/3721297973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 msgr2=0x55ca12f28ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.421+0000 7f64b6a00640 1 --2- 172.21.15.69:0/3721297973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f28ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.421+0000 7f64b6a00640 1 -- 172.21.15.69:0/3721297973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f18f20 msgr2=0x55ca12f35650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.421+0000 7f64b6a00640 1 --2- 172.21.15.69:0/3721297973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f18f20 0x55ca12f35650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.421+0000 7f64b6a00640 1 -- 172.21.15.69:0/3721297973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca12f10170 con 0x55ca12f0fa20 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.421+0000 7f64ae000640 1 --2- 172.21.15.69:0/3721297973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f28ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:14:25.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.421+0000 7f64b6a00640 1 --2- 172.21.15.69:0/3721297973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f186e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f64a800cba0 tx=0x7f64a8097820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=166b8b1f8347ae5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.421+0000 7f64a7400640 1 -- 172.21.15.69:0/3721297973 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a80b3030 con 0x55ca12f0fa20 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.421+0000 7f64a7400640 1 -- 172.21.15.69:0/3721297973 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a809b2d0 con 0x55ca12f0fa20 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.421+0000 7f64a7400640 1 -- 172.21.15.69:0/3721297973 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8098220 con 0x55ca12f0fa20 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.421+0000 7f64c09f2740 1 -- 172.21.15.69:0/3721297973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 msgr2=0x55ca12f186e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.421+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3721297973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f186e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f64a800cba0 tx=0x7f64a8097820 comp rx=0 tx=0).stop 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.422+0000 7f64c09f2740 1 -- 172.21.15.69:0/3721297973 shutdown_connections 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.422+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3721297973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f18f20 0x55ca12f35650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.422+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3721297973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f28ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.422+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3721297973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f0fa20 0x55ca12f186e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.422+0000 7f64c09f2740 1 -- 172.21.15.69:0/3721297973 >> 172.21.15.69:0/3721297973 conn(0x55ca12f0a150 msgr2=0x55ca12f0be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.422+0000 7f64c09f2740 1 -- 172.21.15.69:0/3721297973 shutdown_connections 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.422+0000 7f64c09f2740 1 -- 172.21.15.69:0/3721297973 wait complete. 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.422+0000 7f64c09f2740 1 -- start start 2024-03-19T01:14:25.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.422+0000 7f64c09f2740 1 Processor -- start 2024-03-19T01:14:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f0fa20 0x55ca12f12b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f13040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64c09f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 0x55ca12f13580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64c09f2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81ae0a0 con 0x55ca12f0fa20 2024-03-19T01:14:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64c09f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81ad910 con 0x55ca12f18f20 2024-03-19T01:14:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64c09f2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81aab70 con 0x55ca12f10340 2024-03-19T01:14:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64c09f2740 10 client.0 _ll_register_callbacks cb 0x7f64a809b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 0x55ca12f13580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64ae000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f13040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 0x55ca12f13580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33664/0 (socket says 172.21.15.69:33664) 2024-03-19T01:14:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64b7400640 1 -- 172.21.15.69:0/3588888699 learned_addr learned my addr 172.21.15.69:0/3588888699 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64b6a00640 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f0fa20 0x55ca12f12b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64ae000640 1 -- 172.21.15.69:0/3588888699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 msgr2=0x55ca12f13580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64ae000640 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 0x55ca12f13580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64ae000640 1 -- 172.21.15.69:0/3588888699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f0fa20 msgr2=0x55ca12f12b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64ae000640 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f0fa20 0x55ca12f12b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.423+0000 7f64ae000640 1 -- 172.21.15.69:0/3588888699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca12f92c80 con 0x55ca12f10340 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.424+0000 7f64ae000640 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f13040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f64a002c170 tx=0x7f64a0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7c696bb5e8ca076 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.424+0000 7f64a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.424+0000 7f64a4c00640 1 -- 172.21.15.69:0/3588888699 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a00055d0 con 0x55ca12f10340 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.424+0000 7f64c09f2740 1 -- 172.21.15.69:0/3588888699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a809b2d0 con 0x55ca12f10340 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.424+0000 7f64a4c00640 1 -- 172.21.15.69:0/3588888699 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f64a00501e0 con 0x55ca12f10340 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.424+0000 7f64a4c00640 1 -- 172.21.15.69:0/3588888699 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f64a0060050 con 0x55ca12f10340 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.424+0000 7f64c09f2740 10 client.26614 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.424+0000 7f64c09f2740 1 -- 172.21.15.69:0/3588888699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ca12f93ea0 con 0x55ca12f10340 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.424+0000 7f64a4c00640 1 -- 172.21.15.69:0/3588888699 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a809b2d0 con 0x55ca12f10340 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.425+0000 7f64a4c00640 1 -- 172.21.15.69:0/3588888699 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f64a00035f0 con 0x55ca12f10340 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.425+0000 7f64c09f2740 10 client.26614 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.425+0000 7f64c09f2740 10 client.26614 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.425+0000 7f64c09f2740 20 client.26614 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.425+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ca12f945d0 0x55ca12f969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.425+0000 7f64c09f2740 4 client.26614 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.425+0000 7f64c09f2740 1 -- 172.21.15.69:0/3588888699 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca12e7bc10 con 0x55ca12f945d0 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.425+0000 7f64b6a00640 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ca12f945d0 0x55ca12f969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.425+0000 7f64a4c00640 10 client.26614 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.425+0000 7f64b6a00640 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ca12f945d0 0x55ca12f969f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e026aabe9493a27 server_cookie=8085af018f36344 in_seq=0 out_seq=0 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64a4c00640 1 -- 172.21.15.69:0/3588888699 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca12e7bc10 con 0x55ca12f945d0 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64a4c00640 10 client.26614 handle_command_reply: tid=0 2024-03-19T01:14:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 1 client.26614 shutdown 2024-03-19T01:14:25.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 1 -- 172.21.15.69:0/3588888699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 msgr2=0x55ca12f13040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f13040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f64a002c170 tx=0x7f64a0009260 comp rx=0 tx=0).stop 2024-03-19T01:14:25.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 1 -- 172.21.15.69:0/3588888699 shutdown_connections 2024-03-19T01:14:25.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ca12f945d0 0x55ca12f969f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca12f18f20 0x55ca12f13580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64a4c00640 0 client.26614 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:14:25.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca12f10340 0x55ca12f13040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 1 --2- 172.21.15.69:0/3588888699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ca12f0fa20 0x55ca12f12b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 1 -- 172.21.15.69:0/3588888699 >> 172.21.15.69:0/3588888699 conn(0x55ca12f0a150 msgr2=0x55ca12f15f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 1 -- 172.21.15.69:0/3588888699 shutdown_connections 2024-03-19T01:14:25.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 1 -- 172.21.15.69:0/3588888699 wait complete. 2024-03-19T01:14:25.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.426+0000 7f64c09f2740 20 client.26614 trim_cache size 0 max 16384 2024-03-19T01:14:25.433 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:14:25.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 -- 172.21.15.69:0/118759672 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f649c0a2970 msgr2=0x7f649c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f649c0a2970 0x7f649c0a4e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f64a0006010 tx=0x7f64a0005a20 comp rx=0 tx=0).stop 2024-03-19T01:14:25.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 -- 172.21.15.69:0/118759672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64b81017b0 msgr2=0x7f64b8198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64b81017b0 0x7f64b8198080 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009d50 tx=0x7f64a800c6a0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 -- 172.21.15.69:0/118759672 shutdown_connections 2024-03-19T01:14:25.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f649c0a2970 0x7f649c0a4e30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b81077c0 0x7f64b819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64b8102170 0x7f64b81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 --2- 172.21.15.69:0/118759672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64b81017b0 0x7f64b8198080 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 -- 172.21.15.69:0/118759672 >> 172.21.15.69:0/118759672 conn(0x7f64b8078c20 msgr2=0x7f64b8104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 -- 172.21.15.69:0/118759672 shutdown_connections 2024-03-19T01:14:25.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.427+0000 7f64bdc00640 1 -- 172.21.15.69:0/118759672 wait complete. 2024-03-19T01:14:25.445 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:14:25.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.650+0000 7f98f0e00640 1 -- 172.21.15.69:0/323255419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ec104380 msgr2=0x7f98ec110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.650+0000 7f98f0e00640 1 --2- 172.21.15.69:0/323255419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ec104380 0x7f98ec110890 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009900 tx=0x7f98e402bc40 comp rx=0 tx=0).stop 2024-03-19T01:14:25.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.650+0000 7f98f0e00640 1 -- 172.21.15.69:0/323255419 shutdown_connections 2024-03-19T01:14:25.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.650+0000 7f98f0e00640 1 --2- 172.21.15.69:0/323255419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ec104380 0x7f98ec110890 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.650+0000 7f98f0e00640 1 --2- 172.21.15.69:0/323255419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec1039c0 0x7f98ec103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.650+0000 7f98f0e00640 1 --2- 172.21.15.69:0/323255419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98ec1099d0 0x7f98ec109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.650+0000 7f98f0e00640 1 -- 172.21.15.69:0/323255419 >> 172.21.15.69:0/323255419 conn(0x7f98ec0ff4d0 msgr2=0x7f98ec1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.650+0000 7f98f0e00640 1 -- 172.21.15.69:0/323255419 shutdown_connections 2024-03-19T01:14:25.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.650+0000 7f98f0e00640 1 -- 172.21.15.69:0/323255419 wait complete. 2024-03-19T01:14:25.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.651+0000 7f98f0e00640 1 Processor -- start 2024-03-19T01:14:25.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.651+0000 7f98f0e00640 1 -- start start 2024-03-19T01:14:25.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.652+0000 7f98f0e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98ec1039c0 0x7f98ec19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.652+0000 7f98f0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ec104380 0x7f98ec19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.652+0000 7f98f0e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec1099d0 0x7f98ec19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.652+0000 7f98f0e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec113e30 con 0x7f98ec1099d0 2024-03-19T01:14:25.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.652+0000 7f98f0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec113cb0 con 0x7f98ec104380 2024-03-19T01:14:25.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.652+0000 7f98f0e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec113fb0 con 0x7f98ec1039c0 2024-03-19T01:14:25.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.652+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98ec1039c0 0x7f98ec19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.652+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98ec1039c0 0x7f98ec19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:56112/0 (socket says 172.21.15.69:56112) 2024-03-19T01:14:25.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.652+0000 7f98e9e00640 1 -- 172.21.15.69:0/3105929139 learned_addr learned my addr 172.21.15.69:0/3105929139 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.652+0000 7f98ea800640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec1099d0 0x7f98ec19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.652+0000 7f98e9400640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ec104380 0x7f98ec19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98e9e00640 1 -- 172.21.15.69:0/3105929139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ec104380 msgr2=0x7f98ec19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98e9e00640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ec104380 0x7f98ec19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98e9e00640 1 -- 172.21.15.69:0/3105929139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec1099d0 msgr2=0x7f98ec19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98e9e00640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec1099d0 0x7f98ec19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98e9e00640 1 -- 172.21.15.69:0/3105929139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec19f2e0 con 0x7f98ec1039c0 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98e9400640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ec104380 0x7f98ec19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98e9e00640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98ec1039c0 0x7f98ec19a2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00ebc0 tx=0x7f98dc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98e2a00640 1 -- 172.21.15.69:0/3105929139 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc010630 con 0x7f98ec1039c0 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98e2a00640 1 -- 172.21.15.69:0/3105929139 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98dc010cb0 con 0x7f98ec1039c0 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98e2a00640 1 -- 172.21.15.69:0/3105929139 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc018bd0 con 0x7f98ec1039c0 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98f0e00640 1 -- 172.21.15.69:0/3105929139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec19f5d0 con 0x7f98ec1039c0 2024-03-19T01:14:25.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.653+0000 7f98f0e00640 1 -- 172.21.15.69:0/3105929139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec1a6e60 con 0x7f98ec1039c0 2024-03-19T01:14:25.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.658+0000 7f98f3c1f740 1 Processor -- start 2024-03-19T01:14:25.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.658+0000 7f98f3c1f740 1 -- start start 2024-03-19T01:14:25.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.658+0000 7f98e2a00640 1 -- 172.21.15.69:0/3105929139 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f98dc0107d0 con 0x7f98ec1039c0 2024-03-19T01:14:25.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 0x55cf56e84e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e85340 0x55cf56e8d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e8df20 0x55cf56e90310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cf56de9380 con 0x55cf56e84a20 2024-03-19T01:14:25.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf56d91d20 con 0x55cf56e8df20 2024-03-19T01:14:25.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55cf56de5420 con 0x55cf56e85340 2024-03-19T01:14:25.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98ea800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 0x55cf56e84e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e85340 0x55cf56e8d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e8df20 0x55cf56e90310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e85340 0x55cf56e8d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:56116/0 (socket says 172.21.15.69:56116) 2024-03-19T01:14:25.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98e9e00640 1 -- 172.21.15.69:0/2640457942 learned_addr learned my addr 172.21.15.69:0/2640457942 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98ea800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 0x55cf56e84e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52310/0 (socket says 172.21.15.69:52310) 2024-03-19T01:14:25.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98e9e00640 1 -- 172.21.15.69:0/2640457942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e8df20 msgr2=0x55cf56e90310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98e9e00640 1 --2- 172.21.15.69:0/2640457942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e8df20 0x55cf56e90310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98e9e00640 1 -- 172.21.15.69:0/2640457942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 msgr2=0x55cf56e84e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98e9e00640 1 --2- 172.21.15.69:0/2640457942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 0x55cf56e84e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.659+0000 7f98e9e00640 1 -- 172.21.15.69:0/2640457942 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf56afd3f0 con 0x55cf56e85340 2024-03-19T01:14:25.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.660+0000 7f98e9e00640 1 --2- 172.21.15.69:0/2640457942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e85340 0x55cf56e8d9e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0cca10 tx=0x7f98dc0d7690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=512f5b54b36dfdcc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.660+0000 7f98e2a00640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98d00a29f0 0x7f98d00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.660+0000 7f98db400640 1 -- 172.21.15.69:0/2640457942 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0de070 con 0x55cf56e85340 2024-03-19T01:14:25.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.660+0000 7f98db400640 1 -- 172.21.15.69:0/2640457942 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98dc096350 con 0x55cf56e85340 2024-03-19T01:14:25.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.660+0000 7f98db400640 1 -- 172.21.15.69:0/2640457942 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc072de0 con 0x55cf56e85340 2024-03-19T01:14:25.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.660+0000 7f98e2a00640 1 -- 172.21.15.69:0/3105929139 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f98dc0cf710 con 0x7f98ec1039c0 2024-03-19T01:14:25.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.660+0000 7f98ea800640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98d00a29f0 0x7f98d00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.661+0000 7f98f3c1f740 1 -- 172.21.15.69:0/2640457942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e85340 msgr2=0x55cf56e8d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.661+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/2640457942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e85340 0x55cf56e8d9e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0cca10 tx=0x7f98dc0d7690 comp rx=0 tx=0).stop 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.661+0000 7f98f3c1f740 1 -- 172.21.15.69:0/2640457942 shutdown_connections 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.661+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/2640457942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e8df20 0x55cf56e90310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.661+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/2640457942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e85340 0x55cf56e8d9e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.661+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/2640457942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 0x55cf56e84e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.661+0000 7f98f3c1f740 1 -- 172.21.15.69:0/2640457942 >> 172.21.15.69:0/2640457942 conn(0x55cf56e7f150 msgr2=0x55cf56e7f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.661+0000 7f98f3c1f740 1 -- 172.21.15.69:0/2640457942 shutdown_connections 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.661+0000 7f98ea800640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98d00a29f0 0x7f98d00a4eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98e4005ab0 tx=0x7f98e4005a20 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.661+0000 7f98f3c1f740 1 -- 172.21.15.69:0/2640457942 wait complete. 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.662+0000 7f98f3c1f740 1 -- start start 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.662+0000 7f98f3c1f740 1 Processor -- start 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.662+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 0x55cf56e8afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.662+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 0x55cf56e8b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.662+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e8df20 0x55cf56e8ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.662+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cf56e90d10 con 0x55cf56e84a20 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.662+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf56da5160 con 0x55cf56e85340 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.662+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55cf56e90e90 con 0x55cf56e8df20 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.662+0000 7f98f3c1f740 10 client.0 _ll_register_callbacks cb 0x55cf56de1170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 0x55cf56e8b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e8df20 0x55cf56e8ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 0x55cf56e8b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33708/0 (socket says 172.21.15.69:33708) 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98e9400640 1 -- 172.21.15.69:0/969140343 learned_addr learned my addr 172.21.15.69:0/969140343 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98e9e00640 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 0x55cf56e8afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98e9400640 1 -- 172.21.15.69:0/969140343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e8df20 msgr2=0x55cf56e8ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98e9400640 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e8df20 0x55cf56e8ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98e9400640 1 -- 172.21.15.69:0/969140343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 msgr2=0x55cf56e8afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98e9400640 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 0x55cf56e8afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98e9400640 1 -- 172.21.15.69:0/969140343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf56bc77d0 con 0x55cf56e85340 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98e9e00640 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 0x55cf56e8afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98e9400640 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 0x55cf56e8b4f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f98d4011830 tx=0x7f98d4011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba4866eee5c76c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.663+0000 7f98cfe00640 1 -- 172.21.15.69:0/969140343 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d401f060 con 0x55cf56e85340 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98f3c1f740 1 -- 172.21.15.69:0/969140343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98dc096350 con 0x55cf56e85340 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98cfe00640 1 -- 172.21.15.69:0/969140343 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f98d4010c60 con 0x55cf56e85340 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98cfe00640 1 -- 172.21.15.69:0/969140343 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f98d4026050 con 0x55cf56e85340 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98f3c1f740 10 client.26642 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98f3c1f740 1 -- 172.21.15.69:0/969140343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cf56e9d0c0 con 0x55cf56e85340 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98cfe00640 1 -- 172.21.15.69:0/969140343 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98dc096350 con 0x55cf56e85340 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98cfe00640 1 -- 172.21.15.69:0/969140343 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f98d400f450 con 0x55cf56e85340 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98f3c1f740 10 client.26642 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98f3c1f740 10 client.26642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:14:25.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98f3c1f740 20 client.26642 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:25.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cf56e9e5d0 0x55cf56ea0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98f3c1f740 4 client.26642 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:14:25.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.664+0000 7f98f3c1f740 1 -- 172.21.15.69:0/969140343 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf56df68c0 con 0x55cf56e9e5d0 2024-03-19T01:14:25.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.665+0000 7f98e9e00640 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cf56e9e5d0 0x55cf56ea0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.665+0000 7f98e9e00640 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cf56e9e5d0 0x55cf56ea0a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5266873048fd13f server_cookie=d50f1649cec7ba6d in_seq=0 out_seq=0 2024-03-19T01:14:25.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.665+0000 7f98cfe00640 10 client.26642 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:14:25.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98cfe00640 1 -- 172.21.15.69:0/969140343 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cf56df68c0 con 0x55cf56e9e5d0 2024-03-19T01:14:25.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98cfe00640 10 client.26642 handle_command_reply: tid=0 2024-03-19T01:14:25.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98f3c1f740 1 client.26642 shutdown 2024-03-19T01:14:25.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98f3c1f740 1 -- 172.21.15.69:0/969140343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 msgr2=0x55cf56e8b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 0x55cf56e8b4f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f98d4011830 tx=0x7f98d4011cf0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98f3c1f740 1 -- 172.21.15.69:0/969140343 shutdown_connections 2024-03-19T01:14:25.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e8df20 0x55cf56e8ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cf56e9e5d0 0x55cf56ea0a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 0x55cf56e8b4f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/969140343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e84a20 0x55cf56e8afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98cfe00640 0 client.26642 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:14:25.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98f3c1f740 1 -- 172.21.15.69:0/969140343 >> 172.21.15.69:0/969140343 conn(0x55cf56e7f150 msgr2=0x55cf56e80ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98f3c1f740 1 -- 172.21.15.69:0/969140343 shutdown_connections 2024-03-19T01:14:25.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.666+0000 7f98f3c1f740 1 -- 172.21.15.69:0/969140343 wait complete. 2024-03-19T01:14:25.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.667+0000 7f98f3c1f740 20 client.26642 trim_cache size 0 max 16384 2024-03-19T01:14:25.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.687+0000 7f98f3c1f740 1 Processor -- start 2024-03-19T01:14:25.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.687+0000 7f98f3c1f740 1 -- start start 2024-03-19T01:14:25.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.687+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e84a20 0x55cf56e8d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.687+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 0x55cf56e99ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.687+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e8df20 0x55cf56ea6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.687+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1b0f70 con 0x55cf56e8df20 2024-03-19T01:14:25.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.687+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1b2c30 con 0x55cf56e85340 2024-03-19T01:14:25.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.687+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1b1640 con 0x55cf56e84a20 2024-03-19T01:14:25.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.687+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e84a20 0x55cf56e8d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.687+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e84a20 0x55cf56e8d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:56138/0 (socket says 172.21.15.69:56138) 2024-03-19T01:14:25.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.687+0000 7f98e9e00640 1 -- 172.21.15.69:0/696369074 learned_addr learned my addr 172.21.15.69:0/696369074 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.688+0000 7f98e9400640 1 --2- 172.21.15.69:0/696369074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 0x55cf56e99ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.688+0000 7f98ea800640 1 --2- 172.21.15.69:0/696369074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e8df20 0x55cf56ea6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.688+0000 7f98e9400640 1 -- 172.21.15.69:0/696369074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e84a20 msgr2=0x55cf56e8d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.688+0000 7f98e9400640 1 --2- 172.21.15.69:0/696369074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e84a20 0x55cf56e8d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.688+0000 7f98e9400640 1 -- 172.21.15.69:0/696369074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e8df20 msgr2=0x55cf56ea6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.688+0000 7f98e9400640 1 --2- 172.21.15.69:0/696369074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e8df20 0x55cf56ea6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.688+0000 7f98e9400640 1 -- 172.21.15.69:0/696369074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf56e85170 con 0x55cf56e85340 2024-03-19T01:14:25.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.688+0000 7f98e9400640 1 --2- 172.21.15.69:0/696369074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 0x55cf56e99ab0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55cf56e8c580 tx=0x7f98d400b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56cf01ae85b42795 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.689+0000 7f98daa00640 1 -- 172.21.15.69:0/696369074 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4010c60 con 0x55cf56e85340 2024-03-19T01:14:25.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.689+0000 7f98daa00640 1 -- 172.21.15.69:0/696369074 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98d4010e00 con 0x55cf56e85340 2024-03-19T01:14:25.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.689+0000 7f98daa00640 1 -- 172.21.15.69:0/696369074 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d400f620 con 0x55cf56e85340 2024-03-19T01:14:25.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.690+0000 7f98f3c1f740 1 -- 172.21.15.69:0/696369074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 msgr2=0x55cf56e99ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.690+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/696369074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 0x55cf56e99ab0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55cf56e8c580 tx=0x7f98d400b540 comp rx=0 tx=0).stop 2024-03-19T01:14:25.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.690+0000 7f98f3c1f740 1 -- 172.21.15.69:0/696369074 shutdown_connections 2024-03-19T01:14:25.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.690+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/696369074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e8df20 0x55cf56ea6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.690+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/696369074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e85340 0x55cf56e99ab0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.690+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/696369074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e84a20 0x55cf56e8d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.690+0000 7f98f3c1f740 1 -- 172.21.15.69:0/696369074 >> 172.21.15.69:0/696369074 conn(0x55cf56e7f150 msgr2=0x55cf56e7e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.690+0000 7f98f3c1f740 1 -- 172.21.15.69:0/696369074 shutdown_connections 2024-03-19T01:14:25.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.690+0000 7f98f3c1f740 1 -- 172.21.15.69:0/696369074 wait complete. 2024-03-19T01:14:25.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.691+0000 7f98f3c1f740 1 -- start start 2024-03-19T01:14:25.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.691+0000 7f98f3c1f740 1 Processor -- start 2024-03-19T01:14:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.692+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e84a20 0x55cf56e91930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.692+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e85340 0x55cf56e91e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.692+0000 7f98f3c1f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e8df20 0x55cf56e923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.692+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1b24a0 con 0x55cf56e8df20 2024-03-19T01:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.692+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1b1d10 con 0x55cf56e84a20 2024-03-19T01:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.692+0000 7f98f3c1f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1aef50 con 0x55cf56e85340 2024-03-19T01:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.692+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e84a20 0x55cf56e91930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.693+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e84a20 0x55cf56e91930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33716/0 (socket says 172.21.15.69:33716) 2024-03-19T01:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.693+0000 7f98e9e00640 1 -- 172.21.15.69:0/3106502736 learned_addr learned my addr 172.21.15.69:0/3106502736 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.693+0000 7f98f3c1f740 10 client.0 _ll_register_callbacks cb 0x7f98d400bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.693+0000 7f98e9e00640 1 -- 172.21.15.69:0/3106502736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e85340 msgr2=0x55cf56e91e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.693+0000 7f98e9e00640 1 --2- 172.21.15.69:0/3106502736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e85340 0x55cf56e91e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.693+0000 7f98e9e00640 1 -- 172.21.15.69:0/3106502736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e8df20 msgr2=0x55cf56e923b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.693+0000 7f98e9e00640 1 --2- 172.21.15.69:0/3106502736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e8df20 0x55cf56e923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.693+0000 7f98e9e00640 1 -- 172.21.15.69:0/3106502736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf56efc930 con 0x55cf56e84a20 2024-03-19T01:14:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.693+0000 7f98f3c1f740 1 -- 172.21.15.69:0/3106502736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d4010e00 con 0x55cf56e84a20 2024-03-19T01:14:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.694+0000 7f98e9e00640 1 --2- 172.21.15.69:0/3106502736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e84a20 0x55cf56e91930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00cba0 tx=0x7f98dc01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=141c05bea70a7ad8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.694+0000 7f98cfe00640 10 client.26648 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:14:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.694+0000 7f98cfe00640 1 -- 172.21.15.69:0/3106502736 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc072010 con 0x55cf56e84a20 2024-03-19T01:14:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.694+0000 7f98cfe00640 1 -- 172.21.15.69:0/3106502736 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f98dc099b00 con 0x55cf56e84a20 2024-03-19T01:14:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.694+0000 7f98cfe00640 1 -- 172.21.15.69:0/3106502736 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f98dc072210 con 0x55cf56e84a20 2024-03-19T01:14:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.694+0000 7f98f3c1f740 10 client.26648 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.694+0000 7f98f3c1f740 1 -- 172.21.15.69:0/3106502736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55cf56efdb50 con 0x55cf56e84a20 2024-03-19T01:14:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.695+0000 7f98cfe00640 1 -- 172.21.15.69:0/3106502736 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f98dc09cce0 con 0x55cf56e84a20 2024-03-19T01:14:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.695+0000 7f98f3c1f740 10 client.26648 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.695+0000 7f98f3c1f740 10 client.26648 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.695+0000 7f98f3c1f740 20 client.26648 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.695+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/3106502736 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cf56efdd70 0x55cf56f001b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.695+0000 7f98f3c1f740 4 client.26648 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.695+0000 7f98f3c1f740 1 -- 172.21.15.69:0/3106502736 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf56df0c10 con 0x55cf56efdd70 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.695+0000 7f98e9400640 1 --2- 172.21.15.69:0/3106502736 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cf56efdd70 0x55cf56f001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98e9400640 1 --2- 172.21.15.69:0/3106502736 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cf56efdd70 0x55cf56f001b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=915e570a4bd1e931 server_cookie=f03d08f3ad3cf972 in_seq=0 out_seq=0 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98cfe00640 10 client.26648 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98cfe00640 1 -- 172.21.15.69:0/3106502736 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf56df0c10 con 0x55cf56efdd70 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98cfe00640 10 client.26648 handle_command_reply: tid=0 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98f3c1f740 1 client.26648 shutdown 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98f3c1f740 1 -- 172.21.15.69:0/3106502736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e84a20 msgr2=0x55cf56e91930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/3106502736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e84a20 0x55cf56e91930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00cba0 tx=0x7f98dc01f000 comp rx=0 tx=0).stop 2024-03-19T01:14:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98f3c1f740 1 -- 172.21.15.69:0/3106502736 shutdown_connections 2024-03-19T01:14:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/3106502736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf56e8df20 0x55cf56e923b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/3106502736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf56e85340 0x55cf56e91e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/3106502736 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cf56efdd70 0x55cf56f001b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98f3c1f740 1 --2- 172.21.15.69:0/3106502736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf56e84a20 0x55cf56e91930 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98f3c1f740 1 -- 172.21.15.69:0/3106502736 >> 172.21.15.69:0/3106502736 conn(0x55cf56e7f150 msgr2=0x55cf56e804d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.696+0000 7f98cfe00640 0 client.26648 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:14:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.697+0000 7f98f3c1f740 1 -- 172.21.15.69:0/3106502736 shutdown_connections 2024-03-19T01:14:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.697+0000 7f98f3c1f740 1 -- 172.21.15.69:0/3106502736 wait complete. 2024-03-19T01:14:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.697+0000 7f98f3c1f740 20 client.26648 trim_cache size 0 max 16384 2024-03-19T01:14:25.701 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:14:25.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.698+0000 7f98f0e00640 1 -- 172.21.15.69:0/3105929139 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98d00a29f0 msgr2=0x7f98d00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.698+0000 7f98f0e00640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98d00a29f0 0x7f98d00a4eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98e4005ab0 tx=0x7f98e4005a20 comp rx=0 tx=0).stop 2024-03-19T01:14:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.698+0000 7f98f0e00640 1 -- 172.21.15.69:0/3105929139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98ec1039c0 msgr2=0x7f98ec19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.698+0000 7f98f0e00640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98ec1039c0 0x7f98ec19a2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00ebc0 tx=0x7f98dc00c6a0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.698+0000 7f98f0e00640 1 -- 172.21.15.69:0/3105929139 shutdown_connections 2024-03-19T01:14:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.698+0000 7f98f0e00640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98ec1099d0 0x7f98ec19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.698+0000 7f98f0e00640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f98d00a29f0 0x7f98d00a4eb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.698+0000 7f98f0e00640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ec104380 0x7f98ec19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.698+0000 7f98f0e00640 1 --2- 172.21.15.69:0/3105929139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98ec1039c0 0x7f98ec19a2f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.698+0000 7f98f0e00640 1 -- 172.21.15.69:0/3105929139 >> 172.21.15.69:0/3105929139 conn(0x7f98ec0ff4d0 msgr2=0x7f98ec0ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.699+0000 7f98f0e00640 1 -- 172.21.15.69:0/3105929139 shutdown_connections 2024-03-19T01:14:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.699+0000 7f98f0e00640 1 -- 172.21.15.69:0/3105929139 wait complete. 2024-03-19T01:14:25.723 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:14:25.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.950+0000 7f4267200640 1 -- 172.21.15.69:0/2101390746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42681038a0 msgr2=0x7f4268103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.950+0000 7f4267200640 1 --2- 172.21.15.69:0/2101390746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42681038a0 0x7f4268103d20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4254009900 tx=0x7f425402bc30 comp rx=0 tx=0).stop 2024-03-19T01:14:25.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.950+0000 7f4267200640 1 -- 172.21.15.69:0/2101390746 shutdown_connections 2024-03-19T01:14:25.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.950+0000 7f4267200640 1 --2- 172.21.15.69:0/2101390746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4268104260 0x7f4268110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.950+0000 7f4267200640 1 --2- 172.21.15.69:0/2101390746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42681038a0 0x7f4268103d20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.950+0000 7f4267200640 1 --2- 172.21.15.69:0/2101390746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42681098b0 0x7f4268109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.950+0000 7f4267200640 1 -- 172.21.15.69:0/2101390746 >> 172.21.15.69:0/2101390746 conn(0x7f42680ff330 msgr2=0x7f4268101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.951+0000 7f4267200640 1 -- 172.21.15.69:0/2101390746 shutdown_connections 2024-03-19T01:14:25.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.951+0000 7f4267200640 1 -- 172.21.15.69:0/2101390746 wait complete. 2024-03-19T01:14:25.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.952+0000 7f4267200640 1 Processor -- start 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.952+0000 7f4267200640 1 -- start start 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.952+0000 7f4267200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42681038a0 0x7f426819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.952+0000 7f4267200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4268104260 0x7f426819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.952+0000 7f4267200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42681098b0 0x7f426819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.952+0000 7f4267200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4268113d10 con 0x7f42681098b0 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.952+0000 7f4267200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4268113b90 con 0x7f4268104260 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.952+0000 7f4267200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4268113e90 con 0x7f42681038a0 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4265e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42681038a0 0x7f426819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4265e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42681038a0 0x7f426819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52884/0 (socket says 172.21.15.69:52884) 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4265e00640 1 -- 172.21.15.69:0/2957799429 learned_addr learned my addr 172.21.15.69:0/2957799429 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4265400640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4268104260 0x7f426819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4266800640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42681098b0 0x7f426819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4265e00640 1 -- 172.21.15.69:0/2957799429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4268104260 msgr2=0x7f426819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4265e00640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4268104260 0x7f426819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4265e00640 1 -- 172.21.15.69:0/2957799429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42681098b0 msgr2=0x7f426819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4265e00640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42681098b0 0x7f426819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4265e00640 1 -- 172.21.15.69:0/2957799429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426819f1e0 con 0x7f42681038a0 2024-03-19T01:14:25.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4266800640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42681098b0 0x7f426819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:25.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.953+0000 7f4265e00640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42681038a0 0x7f426819a1f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f425800e9f0 tx=0x7f425800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.954+0000 7f425ea00640 1 -- 172.21.15.69:0/2957799429 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425800cd80 con 0x7f42681038a0 2024-03-19T01:14:25.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.954+0000 7f425ea00640 1 -- 172.21.15.69:0/2957799429 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4258010910 con 0x7f42681038a0 2024-03-19T01:14:25.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.954+0000 7f425ea00640 1 -- 172.21.15.69:0/2957799429 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258018980 con 0x7f42681038a0 2024-03-19T01:14:25.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.954+0000 7f4267200640 1 -- 172.21.15.69:0/2957799429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426819f4d0 con 0x7f42681038a0 2024-03-19T01:14:25.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.955+0000 7f4267200640 1 -- 172.21.15.69:0/2957799429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42681a6d60 con 0x7f42681038a0 2024-03-19T01:14:25.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.958+0000 7f426f980740 1 Processor -- start 2024-03-19T01:14:25.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.958+0000 7f426f980740 1 -- start start 2024-03-19T01:14:25.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f426f980740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78a4a00 0x55e9b78a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f426f980740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a53b0 0x55e9b78b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f426f980740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78b6900 0x55e9b78b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f426f980740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e9b7809380 con 0x55e9b78a4a00 2024-03-19T01:14:25.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f426f980740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e9b77b1d20 con 0x55e9b78a53b0 2024-03-19T01:14:25.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f426f980740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e9b7805420 con 0x55e9b78b6900 2024-03-19T01:14:25.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f425ea00640 1 -- 172.21.15.69:0/2957799429 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f4258010ac0 con 0x7f42681038a0 2024-03-19T01:14:25.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f4266800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78a4a00 0x55e9b78a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f4265400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78b6900 0x55e9b78b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f4266800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78a4a00 0x55e9b78a4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36464/0 (socket says 172.21.15.69:36464) 2024-03-19T01:14:25.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f4265400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78b6900 0x55e9b78b8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52894/0 (socket says 172.21.15.69:52894) 2024-03-19T01:14:25.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.959+0000 7f4266800640 1 -- 172.21.15.69:0/3389909523 learned_addr learned my addr 172.21.15.69:0/3389909523 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.960+0000 7f4265400640 1 -- 172.21.15.69:0/3389909523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a53b0 msgr2=0x55e9b78b63c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:14:25.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.960+0000 7f4265400640 1 --2- 172.21.15.69:0/3389909523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a53b0 0x55e9b78b63c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.960+0000 7f4265400640 1 -- 172.21.15.69:0/3389909523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78a4a00 msgr2=0x55e9b78a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.960+0000 7f4265400640 1 --2- 172.21.15.69:0/3389909523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78a4a00 0x55e9b78a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.960+0000 7f4265400640 1 -- 172.21.15.69:0/3389909523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b751d3f0 con 0x55e9b78b6900 2024-03-19T01:14:25.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.960+0000 7f4265400640 1 --2- 172.21.15.69:0/3389909523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78b6900 0x55e9b78b8cf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4254036040 tx=0x7f4254005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f21f3ebffa40075 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.960+0000 7f4253000640 1 -- 172.21.15.69:0/3389909523 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254041070 con 0x55e9b78b6900 2024-03-19T01:14:25.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.960+0000 7f4253000640 1 -- 172.21.15.69:0/3389909523 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f425402c6d0 con 0x55e9b78b6900 2024-03-19T01:14:25.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.960+0000 7f4253000640 1 -- 172.21.15.69:0/3389909523 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42540046c0 con 0x55e9b78b6900 2024-03-19T01:14:25.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.961+0000 7f426f980740 1 -- 172.21.15.69:0/3389909523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78b6900 msgr2=0x55e9b78b8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.961+0000 7f426f980740 1 --2- 172.21.15.69:0/3389909523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78b6900 0x55e9b78b8cf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4254036040 tx=0x7f4254005dd0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.961+0000 7f425ea00640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f424c0a29f0 0x7f424c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.962+0000 7f425ea00640 1 -- 172.21.15.69:0/2957799429 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4258099ad0 con 0x7f42681038a0 2024-03-19T01:14:25.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.962+0000 7f426f980740 1 -- 172.21.15.69:0/3389909523 shutdown_connections 2024-03-19T01:14:25.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.962+0000 7f426f980740 1 --2- 172.21.15.69:0/3389909523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78b6900 0x55e9b78b8cf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.962+0000 7f426f980740 1 --2- 172.21.15.69:0/3389909523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a53b0 0x55e9b78b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.962+0000 7f426f980740 1 --2- 172.21.15.69:0/3389909523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78a4a00 0x55e9b78a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.962+0000 7f426f980740 1 -- 172.21.15.69:0/3389909523 >> 172.21.15.69:0/3389909523 conn(0x55e9b789f320 msgr2=0x55e9b789f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.962+0000 7f4266800640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f424c0a29f0 0x7f424c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.962+0000 7f426f980740 1 -- 172.21.15.69:0/3389909523 shutdown_connections 2024-03-19T01:14:25.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.962+0000 7f426f980740 1 -- 172.21.15.69:0/3389909523 wait complete. 2024-03-19T01:14:25.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.963+0000 7f426f980740 1 -- start start 2024-03-19T01:14:25.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.963+0000 7f4266800640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f424c0a29f0 0x7f424c0a4eb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4260005fd0 tx=0x7f4260009940 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.963+0000 7f426f980740 1 Processor -- start 2024-03-19T01:14:25.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f426f980740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 0x55e9b78b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f426f980740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a53b0 0x55e9b78b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f426f980740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78b6900 0x55e9b78b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f426f980740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e9b78b9660 con 0x55e9b78b6900 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f426f980740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e9b77c5160 con 0x55e9b78a53b0 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f426f980740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e9b78b97e0 con 0x55e9b78a4a00 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f426f980740 10 client.0 _ll_register_callbacks cb 0x55e9b7801170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f4265400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a53b0 0x55e9b78b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f4265400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a53b0 0x55e9b78b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38656/0 (socket says 172.21.15.69:38656) 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f4265400640 1 -- 172.21.15.69:0/2729800766 learned_addr learned my addr 172.21.15.69:0/2729800766 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f4265e00640 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 0x55e9b78b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f4265400640 1 -- 172.21.15.69:0/2729800766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 msgr2=0x55e9b78b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f4265400640 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 0x55e9b78b38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f4266800640 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78b6900 0x55e9b78b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f4265400640 1 -- 172.21.15.69:0/2729800766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78b6900 msgr2=0x55e9b78b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f4265400640 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78b6900 0x55e9b78b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.964+0000 7f4265400640 1 -- 172.21.15.69:0/2729800766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b75e77d0 con 0x55e9b78a53b0 2024-03-19T01:14:25.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.965+0000 7f4265400640 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a53b0 0x55e9b78b3e30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4254005940 tx=0x7f425402c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f3010bfdc77db9d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.965+0000 7f4247400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:14:25.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.965+0000 7f4247400640 1 -- 172.21.15.69:0/2729800766 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254041040 con 0x55e9b78a53b0 2024-03-19T01:14:25.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.965+0000 7f4247400640 1 -- 172.21.15.69:0/2729800766 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4254004240 con 0x55e9b78a53b0 2024-03-19T01:14:25.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.965+0000 7f426f980740 1 -- 172.21.15.69:0/2729800766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f425402c6d0 con 0x55e9b78a53b0 2024-03-19T01:14:25.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.965+0000 7f4247400640 1 -- 172.21.15.69:0/2729800766 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4254057050 con 0x55e9b78a53b0 2024-03-19T01:14:25.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.965+0000 7f426f980740 10 client.26657 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:25.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.965+0000 7f426f980740 1 -- 172.21.15.69:0/2729800766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e9b78c58f0 con 0x55e9b78a53b0 2024-03-19T01:14:25.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.965+0000 7f4247400640 1 -- 172.21.15.69:0/2729800766 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4254004440 con 0x55e9b78a53b0 2024-03-19T01:14:25.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.966+0000 7f4247400640 1 -- 172.21.15.69:0/2729800766 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f4254003ae0 con 0x55e9b78a53b0 2024-03-19T01:14:25.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.966+0000 7f426f980740 10 client.26657 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:25.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.966+0000 7f426f980740 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:14:25.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.966+0000 7f426f980740 20 client.26657 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.966+0000 7f426f980740 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55e9b78c6c40 0x55e9b78c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.966+0000 7f426f980740 4 client.26657 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.966+0000 7f426f980740 1 -- 172.21.15.69:0/2729800766 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f425402c6d0 con 0x55e9b78c6c40 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.966+0000 7f4265e00640 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55e9b78c6c40 0x55e9b78c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.967+0000 7f4247400640 10 client.26657 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.967+0000 7f4265e00640 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55e9b78c6c40 0x55e9b78c9030 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e60f58845adeee6 server_cookie=5b1168ab3b996816 in_seq=0 out_seq=0 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.967+0000 7f4247400640 1 -- 172.21.15.69:0/2729800766 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f425402c6d0 con 0x55e9b78c6c40 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.967+0000 7f4247400640 10 client.26657 handle_command_reply: tid=0 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.967+0000 7f426f980740 1 client.26657 shutdown 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.967+0000 7f426f980740 1 -- 172.21.15.69:0/2729800766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a53b0 msgr2=0x55e9b78b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.967+0000 7f426f980740 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a53b0 0x55e9b78b3e30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4254005940 tx=0x7f425402c950 comp rx=0 tx=0).stop 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.968+0000 7f426f980740 1 -- 172.21.15.69:0/2729800766 shutdown_connections 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.968+0000 7f426f980740 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55e9b78c6c40 0x55e9b78c9030 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.968+0000 7f426f980740 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78b6900 0x55e9b78b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.968+0000 7f426f980740 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a53b0 0x55e9b78b3e30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.968+0000 7f4247400640 0 client.26657 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:25.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.968+0000 7f426f980740 1 --2- 172.21.15.69:0/2729800766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 0x55e9b78b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.968+0000 7f426f980740 1 -- 172.21.15.69:0/2729800766 >> 172.21.15.69:0/2729800766 conn(0x55e9b789f320 msgr2=0x55e9b78a1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.968+0000 7f426f980740 1 -- 172.21.15.69:0/2729800766 shutdown_connections 2024-03-19T01:14:25.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.968+0000 7f426f980740 1 -- 172.21.15.69:0/2729800766 wait complete. 2024-03-19T01:14:25.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.968+0000 7f426f980740 20 client.26657 trim_cache size 0 max 16384 2024-03-19T01:14:25.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f426f980740 1 Processor -- start 2024-03-19T01:14:25.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f426f980740 1 -- start start 2024-03-19T01:14:25.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f426f980740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 0x55e9b78c2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f426f980740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78a53b0 0x55e9b78b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f426f980740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78b6900 0x55e9b789e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f426f980740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f42681ac8f0 con 0x55e9b78a53b0 2024-03-19T01:14:25.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f426f980740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42681ae5b0 con 0x55e9b78b6900 2024-03-19T01:14:25.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f426f980740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f42681acfc0 con 0x55e9b78a4a00 2024-03-19T01:14:25.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f4265e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 0x55e9b78c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f4265400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78a53b0 0x55e9b78b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f4265e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 0x55e9b78c2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52914/0 (socket says 172.21.15.69:52914) 2024-03-19T01:14:25.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f4265e00640 1 -- 172.21.15.69:0/546351290 learned_addr learned my addr 172.21.15.69:0/546351290 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.984+0000 7f4266800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78b6900 0x55e9b789e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.985+0000 7f4265e00640 1 -- 172.21.15.69:0/546351290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78b6900 msgr2=0x55e9b789e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.985+0000 7f4265e00640 1 --2- 172.21.15.69:0/546351290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78b6900 0x55e9b789e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.985+0000 7f4265e00640 1 -- 172.21.15.69:0/546351290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78a53b0 msgr2=0x55e9b78b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.985+0000 7f4265e00640 1 --2- 172.21.15.69:0/546351290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78a53b0 0x55e9b78b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.985+0000 7f4265e00640 1 -- 172.21.15.69:0/546351290 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b789ac10 con 0x55e9b78a4a00 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.985+0000 7f4265e00640 1 --2- 172.21.15.69:0/546351290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 0x55e9b78c2440 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f425800c6a0 tx=0x7f42580a0a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1046f17782124564 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.985+0000 7f4252600640 1 -- 172.21.15.69:0/546351290 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258098460 con 0x55e9b78a4a00 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.985+0000 7f4252600640 1 -- 172.21.15.69:0/546351290 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4258011e50 con 0x55e9b78a4a00 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.985+0000 7f4252600640 1 -- 172.21.15.69:0/546351290 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258098600 con 0x55e9b78a4a00 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.986+0000 7f426f980740 1 -- 172.21.15.69:0/546351290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 msgr2=0x55e9b78c2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.986+0000 7f426f980740 1 --2- 172.21.15.69:0/546351290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 0x55e9b78c2440 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f425800c6a0 tx=0x7f42580a0a30 comp rx=0 tx=0).stop 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.986+0000 7f426f980740 1 -- 172.21.15.69:0/546351290 shutdown_connections 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.986+0000 7f426f980740 1 --2- 172.21.15.69:0/546351290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78b6900 0x55e9b789e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.986+0000 7f426f980740 1 --2- 172.21.15.69:0/546351290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78a53b0 0x55e9b78b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.986+0000 7f426f980740 1 --2- 172.21.15.69:0/546351290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a4a00 0x55e9b78c2440 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.986+0000 7f426f980740 1 -- 172.21.15.69:0/546351290 >> 172.21.15.69:0/546351290 conn(0x55e9b789f320 msgr2=0x55e9b789daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.987+0000 7f426f980740 1 -- 172.21.15.69:0/546351290 shutdown_connections 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.987+0000 7f426f980740 1 -- 172.21.15.69:0/546351290 wait complete. 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.988+0000 7f426f980740 1 -- start start 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.988+0000 7f426f980740 1 Processor -- start 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f426f980740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a4a00 0x55e9b78ba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f426f980740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a53b0 0x55e9b78bac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f426f980740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78b6900 0x55e9b792bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f426f980740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f42681ade20 con 0x55e9b78b6900 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f426f980740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42681ad690 con 0x55e9b78a4a00 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f426f980740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f42681aa8f0 con 0x55e9b78a53b0 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f426f980740 10 client.0 _ll_register_callbacks cb 0x7f42600097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f4265e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a4a00 0x55e9b78ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f4265e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a4a00 0x55e9b78ba740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38666/0 (socket says 172.21.15.69:38666) 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f4265e00640 1 -- 172.21.15.69:0/2004992806 learned_addr learned my addr 172.21.15.69:0/2004992806 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f4265400640 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a53b0 0x55e9b78bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f4266800640 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78b6900 0x55e9b792bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f4265e00640 1 -- 172.21.15.69:0/2004992806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a53b0 msgr2=0x55e9b78bac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f4265e00640 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a53b0 0x55e9b78bac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f4265e00640 1 -- 172.21.15.69:0/2004992806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78b6900 msgr2=0x55e9b792bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f4265e00640 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78b6900 0x55e9b792bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.989+0000 7f4265e00640 1 -- 172.21.15.69:0/2004992806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b792c290 con 0x55e9b78a4a00 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.990+0000 7f4265e00640 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a4a00 0x55e9b78ba740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4258013040 tx=0x7f425800cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad6d39b2861bc42e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.990+0000 7f4247400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.990+0000 7f4247400640 1 -- 172.21.15.69:0/2004992806 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42580b9060 con 0x55e9b78a4a00 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.990+0000 7f4247400640 1 -- 172.21.15.69:0/2004992806 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4258010430 con 0x55e9b78a4a00 2024-03-19T01:14:25.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.990+0000 7f426f980740 1 -- 172.21.15.69:0/2004992806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4258011e50 con 0x55e9b78a4a00 2024-03-19T01:14:25.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.990+0000 7f4247400640 1 -- 172.21.15.69:0/2004992806 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f42580b4040 con 0x55e9b78a4a00 2024-03-19T01:14:25.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.990+0000 7f426f980740 10 client.26663 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:25.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.990+0000 7f426f980740 1 -- 172.21.15.69:0/2004992806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e9b792d430 con 0x55e9b78a4a00 2024-03-19T01:14:25.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.991+0000 7f4247400640 1 -- 172.21.15.69:0/2004992806 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4258011e50 con 0x55e9b78a4a00 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.991+0000 7f4247400640 1 -- 172.21.15.69:0/2004992806 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f425809b390 con 0x55e9b78a4a00 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.991+0000 7f426f980740 10 client.26663 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.991+0000 7f426f980740 10 client.26663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.991+0000 7f426f980740 20 client.26663 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.991+0000 7f426f980740 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55e9b792db40 0x55e9b792ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.991+0000 7f426f980740 4 client.26663 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.991+0000 7f426f980740 1 -- 172.21.15.69:0/2004992806 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9b78168c0 con 0x55e9b792db40 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.991+0000 7f4265400640 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55e9b792db40 0x55e9b792ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f4247400640 10 client.26663 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f4265400640 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55e9b792db40 0x55e9b792ff60 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c84e1892dd296fcc server_cookie=4cec896f46b4dc96 in_seq=0 out_seq=0 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f4247400640 1 -- 172.21.15.69:0/2004992806 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9b78168c0 con 0x55e9b792db40 2024-03-19T01:14:25.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f4247400640 10 client.26663 handle_command_reply: tid=0 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f426f980740 1 client.26663 shutdown 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f426f980740 1 -- 172.21.15.69:0/2004992806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a4a00 msgr2=0x55e9b78ba740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f426f980740 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a4a00 0x55e9b78ba740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4258013040 tx=0x7f425800cf20 comp rx=0 tx=0).stop 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f426f980740 1 -- 172.21.15.69:0/2004992806 shutdown_connections 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f426f980740 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55e9b792db40 0x55e9b792ff60 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f426f980740 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9b78b6900 0x55e9b792bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f426f980740 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9b78a53b0 0x55e9b78bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f426f980740 1 --2- 172.21.15.69:0/2004992806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9b78a4a00 0x55e9b78ba740 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f426f980740 1 -- 172.21.15.69:0/2004992806 >> 172.21.15.69:0/2004992806 conn(0x55e9b789f320 msgr2=0x55e9b78a0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.992+0000 7f4247400640 0 client.26663 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.993+0000 7f426f980740 1 -- 172.21.15.69:0/2004992806 shutdown_connections 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.993+0000 7f426f980740 1 -- 172.21.15.69:0/2004992806 wait complete. 2024-03-19T01:14:25.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.993+0000 7f426f980740 20 client.26663 trim_cache size 0 max 16384 2024-03-19T01:14:25.999 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:14:25.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.994+0000 7f4267200640 1 -- 172.21.15.69:0/2957799429 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f424c0a29f0 msgr2=0x7f424c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:25.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.994+0000 7f4267200640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f424c0a29f0 0x7f424c0a4eb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4260005fd0 tx=0x7f4260009940 comp rx=0 tx=0).stop 2024-03-19T01:14:26.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.994+0000 7f4267200640 1 -- 172.21.15.69:0/2957799429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42681038a0 msgr2=0x7f426819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:26.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.994+0000 7f4267200640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42681038a0 0x7f426819a1f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f425800e9f0 tx=0x7f425800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:14:26.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.994+0000 7f4267200640 1 -- 172.21.15.69:0/2957799429 shutdown_connections 2024-03-19T01:14:26.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.994+0000 7f4267200640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42681098b0 0x7f426819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:26.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.994+0000 7f4267200640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f424c0a29f0 0x7f424c0a4eb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:26.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.994+0000 7f4267200640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4268104260 0x7f426819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:26.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.994+0000 7f4267200640 1 --2- 172.21.15.69:0/2957799429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42681038a0 0x7f426819a1f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:26.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.994+0000 7f4267200640 1 -- 172.21.15.69:0/2957799429 >> 172.21.15.69:0/2957799429 conn(0x7f42680ff330 msgr2=0x7f42680ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:26.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.994+0000 7f4267200640 1 -- 172.21.15.69:0/2957799429 shutdown_connections 2024-03-19T01:14:26.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:25.995+0000 7f4267200640 1 -- 172.21.15.69:0/2957799429 wait complete. 2024-03-19T01:14:26.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:25 smithi086 ceph-mon[66829]: pgmap v115: 129 pgs: 129 active+clean; 1000 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 26 MiB/s wr, 116 op/s 2024-03-19T01:14:26.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:25 smithi069 ceph-mon[31727]: pgmap v115: 129 pgs: 129 active+clean; 1000 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 26 MiB/s wr, 116 op/s 2024-03-19T01:14:26.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:25 smithi163 ceph-mon[67428]: pgmap v115: 129 pgs: 129 active+clean; 1000 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 26 MiB/s wr, 116 op/s 2024-03-19T01:14:27.018 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:14:27.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:14:27.018 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:14:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.231+0000 7f0591200640 1 -- 172.21.15.69:0/4226964697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f058c103380 msgr2=0x7f058c10b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.231+0000 7f0591200640 1 --2- 172.21.15.69:0/4226964697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f058c103380 0x7f058c10b860 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0578009900 tx=0x7f057802bcd0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.231+0000 7f0591200640 1 -- 172.21.15.69:0/4226964697 shutdown_connections 2024-03-19T01:14:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.231+0000 7f0591200640 1 --2- 172.21.15.69:0/4226964697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f058c10be30 0x7f058c10e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.231+0000 7f0591200640 1 --2- 172.21.15.69:0/4226964697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f058c103380 0x7f058c10b860 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.231+0000 7f0591200640 1 --2- 172.21.15.69:0/4226964697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058c102a60 0x7f058c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.231+0000 7f0591200640 1 -- 172.21.15.69:0/4226964697 >> 172.21.15.69:0/4226964697 conn(0x7f058c076aa0 msgr2=0x7f058c076eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.231+0000 7f0591200640 1 -- 172.21.15.69:0/4226964697 shutdown_connections 2024-03-19T01:14:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.231+0000 7f0591200640 1 -- 172.21.15.69:0/4226964697 wait complete. 2024-03-19T01:14:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.232+0000 7f0591200640 1 Processor -- start 2024-03-19T01:14:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.232+0000 7f0591200640 1 -- start start 2024-03-19T01:14:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0591200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f058c102a60 0x7f058c1038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0591200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058c103380 0x7f058c103e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0591200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f058c10be30 0x7f058c104350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0591200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f058c111c10 con 0x7f058c10be30 2024-03-19T01:14:27.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0591200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f058c111a90 con 0x7f058c103380 2024-03-19T01:14:27.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0591200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f058c111d90 con 0x7f058c102a60 2024-03-19T01:14:27.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0589400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058c103380 0x7f058c103e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0589e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f058c102a60 0x7f058c1038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0589e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f058c102a60 0x7f058c1038d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52932/0 (socket says 172.21.15.69:52932) 2024-03-19T01:14:27.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0589400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058c103380 0x7f058c103e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38682/0 (socket says 172.21.15.69:38682) 2024-03-19T01:14:27.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0589e00640 1 -- 172.21.15.69:0/3161219317 learned_addr learned my addr 172.21.15.69:0/3161219317 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:27.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f058a800640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f058c10be30 0x7f058c104350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0589400640 1 -- 172.21.15.69:0/3161219317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f058c102a60 msgr2=0x7f058c1038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0589400640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f058c102a60 0x7f058c1038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0589400640 1 -- 172.21.15.69:0/3161219317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f058c10be30 msgr2=0x7f058c104350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0589400640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f058c10be30 0x7f058c104350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.233+0000 7f0589400640 1 -- 172.21.15.69:0/3161219317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058c1aa9c0 con 0x7f058c103380 2024-03-19T01:14:27.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.234+0000 7f0589e00640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f058c102a60 0x7f058c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:14:27.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.234+0000 7f0589400640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058c103380 0x7f058c103e10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f057802c210 tx=0x7f0578005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.234+0000 7f0582a00640 1 -- 172.21.15.69:0/3161219317 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578041020 con 0x7f058c103380 2024-03-19T01:14:27.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.234+0000 7f0582a00640 1 -- 172.21.15.69:0/3161219317 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f057802ce00 con 0x7f058c103380 2024-03-19T01:14:27.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.234+0000 7f0582a00640 1 -- 172.21.15.69:0/3161219317 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578003ab0 con 0x7f058c103380 2024-03-19T01:14:27.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.234+0000 7f0591200640 1 -- 172.21.15.69:0/3161219317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058c1aac50 con 0x7f058c103380 2024-03-19T01:14:27.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.234+0000 7f0591200640 1 -- 172.21.15.69:0/3161219317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058c1ab140 con 0x7f058c103380 2024-03-19T01:14:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.238+0000 7f0593e0c740 1 Processor -- start 2024-03-19T01:14:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.238+0000 7f0593e0c740 1 -- start start 2024-03-19T01:14:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.238+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480bab00 0x55d8480baee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.238+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 0x55d8480c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.238+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480c4000 0x55d8480c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.238+0000 7f0593e0c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d84801dfb0 con 0x55d8480bab00 2024-03-19T01:14:27.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.238+0000 7f0593e0c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d847fc6ab0 con 0x55d8480bb420 2024-03-19T01:14:27.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.238+0000 7f0593e0c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d84801a050 con 0x55d8480c4000 2024-03-19T01:14:27.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.238+0000 7f0582a00640 1 -- 172.21.15.69:0/3161219317 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f0578044020 con 0x7f058c103380 2024-03-19T01:14:27.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.238+0000 7f0589e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 0x55d8480c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.239+0000 7f0589e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 0x55d8480c3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38686/0 (socket says 172.21.15.69:38686) 2024-03-19T01:14:27.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.239+0000 7f0589e00640 1 -- 172.21.15.69:0/156962832 learned_addr learned my addr 172.21.15.69:0/156962832 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.239+0000 7f0589400640 1 --2- 172.21.15.69:0/156962832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480c4000 0x55d8480c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.239+0000 7f058a800640 1 --2- 172.21.15.69:0/156962832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480bab00 0x55d8480baee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.239+0000 7f0589e00640 1 -- 172.21.15.69:0/156962832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480c4000 msgr2=0x55d8480c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.239+0000 7f0589e00640 1 --2- 172.21.15.69:0/156962832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480c4000 0x55d8480c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.239+0000 7f0589e00640 1 -- 172.21.15.69:0/156962832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480bab00 msgr2=0x55d8480baee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.239+0000 7f0589e00640 1 --2- 172.21.15.69:0/156962832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480bab00 0x55d8480baee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.239+0000 7f0589e00640 1 -- 172.21.15.69:0/156962832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d847fe4e00 con 0x55d8480bb420 2024-03-19T01:14:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.239+0000 7f0589e00640 1 --2- 172.21.15.69:0/156962832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 0x55d8480c3ac0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f057c009fd0 tx=0x7f057c011ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbb39127a53341e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.240+0000 7f0577400640 1 -- 172.21.15.69:0/156962832 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c01e070 con 0x55d8480bb420 2024-03-19T01:14:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.240+0000 7f0577400640 1 -- 172.21.15.69:0/156962832 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f057c00fbd0 con 0x55d8480bb420 2024-03-19T01:14:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.240+0000 7f0577400640 1 -- 172.21.15.69:0/156962832 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c019d80 con 0x55d8480bb420 2024-03-19T01:14:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.240+0000 7f0582a00640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f05700a2970 0x7f05700a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.240+0000 7f0582a00640 1 -- 172.21.15.69:0/3161219317 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f05780f0810 con 0x7f058c103380 2024-03-19T01:14:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.240+0000 7f0593e0c740 1 -- 172.21.15.69:0/156962832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 msgr2=0x55d8480c3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.240+0000 7f0593e0c740 1 --2- 172.21.15.69:0/156962832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 0x55d8480c3ac0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f057c009fd0 tx=0x7f057c011ae0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.240+0000 7f058a800640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f05700a2970 0x7f05700a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.241+0000 7f0593e0c740 1 -- 172.21.15.69:0/156962832 shutdown_connections 2024-03-19T01:14:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.241+0000 7f0593e0c740 1 --2- 172.21.15.69:0/156962832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480c4000 0x55d8480c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.241+0000 7f0593e0c740 1 --2- 172.21.15.69:0/156962832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 0x55d8480c3ac0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.241+0000 7f0593e0c740 1 --2- 172.21.15.69:0/156962832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480bab00 0x55d8480baee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.241+0000 7f0593e0c740 1 -- 172.21.15.69:0/156962832 >> 172.21.15.69:0/156962832 conn(0x55d8480b5230 msgr2=0x55d8480b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.241+0000 7f0593e0c740 1 -- 172.21.15.69:0/156962832 shutdown_connections 2024-03-19T01:14:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.241+0000 7f0593e0c740 1 -- 172.21.15.69:0/156962832 wait complete. 2024-03-19T01:14:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.241+0000 7f058a800640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f05700a2970 0x7f05700a4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f058c06aed0 tx=0x7f0584009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.241+0000 7f0593e0c740 1 -- start start 2024-03-19T01:14:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.241+0000 7f0593e0c740 1 Processor -- start 2024-03-19T01:14:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bab00 0x55d8480c1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 0x55d8480c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480c4000 0x55d8480c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f0593e0c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d8480c6df0 con 0x55d8480c4000 2024-03-19T01:14:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f0593e0c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d847fd9f00 con 0x55d8480bb420 2024-03-19T01:14:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f0593e0c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d8480c6f70 con 0x55d8480bab00 2024-03-19T01:14:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f0593e0c740 10 client.0 _ll_register_callbacks cb 0x55d847fd98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:27.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f0589e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bab00 0x55d8480c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f0589400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 0x55d8480c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f0589e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bab00 0x55d8480c1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52952/0 (socket says 172.21.15.69:52952) 2024-03-19T01:14:27.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f0589e00640 1 -- 172.21.15.69:0/1529809250 learned_addr learned my addr 172.21.15.69:0/1529809250 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:27.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.242+0000 7f058a800640 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480c4000 0x55d8480c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.243+0000 7f0589e00640 1 -- 172.21.15.69:0/1529809250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 msgr2=0x55d8480c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.243+0000 7f0589e00640 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 0x55d8480c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.243+0000 7f0589e00640 1 -- 172.21.15.69:0/1529809250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480c4000 msgr2=0x55d8480c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.243+0000 7f0589e00640 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480c4000 0x55d8480c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.243+0000 7f0589e00640 1 -- 172.21.15.69:0/1529809250 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d847dfa7e0 con 0x55d8480bab00 2024-03-19T01:14:27.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.243+0000 7f0589400640 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 0x55d8480c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:14:27.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.243+0000 7f058a800640 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480c4000 0x55d8480c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:27.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.243+0000 7f0589e00640 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bab00 0x55d8480c1000 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f057c000c00 tx=0x7f057c00ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd72d11bc46b13f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.243+0000 7f056be00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:14:27.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.243+0000 7f056be00640 1 -- 172.21.15.69:0/1529809250 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c027060 con 0x55d8480bab00 2024-03-19T01:14:27.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.243+0000 7f056be00640 1 -- 172.21.15.69:0/1529809250 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f057c019a80 con 0x55d8480bab00 2024-03-19T01:14:27.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f0593e0c740 1 -- 172.21.15.69:0/1529809250 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f057c00fbd0 con 0x55d8480bab00 2024-03-19T01:14:27.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f056be00640 1 -- 172.21.15.69:0/1529809250 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f057c01e040 con 0x55d8480bab00 2024-03-19T01:14:27.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f0593e0c740 10 client.26656 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:27.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f0593e0c740 1 -- 172.21.15.69:0/1529809250 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d847e5af70 con 0x55d8480bab00 2024-03-19T01:14:27.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f056be00640 1 -- 172.21.15.69:0/1529809250 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f057c00fbd0 con 0x55d8480bab00 2024-03-19T01:14:27.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f056be00640 1 -- 172.21.15.69:0/1529809250 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f057c021030 con 0x55d8480bab00 2024-03-19T01:14:27.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f0593e0c740 10 client.26656 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:27.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f0593e0c740 10 client.26656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:14:27.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f0593e0c740 20 client.26656 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:27.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8480d42c0 0x55d8480d66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f0593e0c740 4 client.26656 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:14:27.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.244+0000 7f0593e0c740 1 -- 172.21.15.69:0/1529809250 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d84802b4f0 con 0x55d8480d42c0 2024-03-19T01:14:27.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.245+0000 7f0589400640 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8480d42c0 0x55d8480d66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.245+0000 7f0589400640 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8480d42c0 0x55d8480d66b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a020279abd99202 server_cookie=3abf35e953dcd18e in_seq=0 out_seq=0 2024-03-19T01:14:27.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.245+0000 7f056be00640 10 client.26656 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:27.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f056be00640 1 -- 172.21.15.69:0/1529809250 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d84802b4f0 con 0x55d8480d42c0 2024-03-19T01:14:27.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f056be00640 10 client.26656 handle_command_reply: tid=0 2024-03-19T01:14:27.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 1 client.26656 shutdown 2024-03-19T01:14:27.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 1 -- 172.21.15.69:0/1529809250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bab00 msgr2=0x55d8480c1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bab00 0x55d8480c1000 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f057c000c00 tx=0x7f057c00ff90 comp rx=0 tx=0).stop 2024-03-19T01:14:27.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 1 -- 172.21.15.69:0/1529809250 shutdown_connections 2024-03-19T01:14:27.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8480d42c0 0x55d8480d66b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480c4000 0x55d8480c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bb420 0x55d8480c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1529809250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bab00 0x55d8480c1000 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f056be00640 0 client.26656 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:27.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 1 -- 172.21.15.69:0/1529809250 >> 172.21.15.69:0/1529809250 conn(0x55d8480b5230 msgr2=0x55d8480b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 1 -- 172.21.15.69:0/1529809250 shutdown_connections 2024-03-19T01:14:27.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 1 -- 172.21.15.69:0/1529809250 wait complete. 2024-03-19T01:14:27.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.246+0000 7f0593e0c740 20 client.26656 trim_cache size 0 max 16384 2024-03-19T01:14:27.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.276+0000 7f0593e0c740 1 Processor -- start 2024-03-19T01:14:27.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f0593e0c740 1 -- start start 2024-03-19T01:14:27.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480bb420 0x55d8480b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480c4000 0x55d8480dbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f0593e0c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f058c1b5340 con 0x55d8480bb420 2024-03-19T01:14:27.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f0593e0c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f058c1b7000 con 0x55d8480bab00 2024-03-19T01:14:27.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f0593e0c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f058c1b5a10 con 0x55d8480c4000 2024-03-19T01:14:27.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f0589e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f058a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480c4000 0x55d8480dbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f058a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480c4000 0x55d8480dbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52962/0 (socket says 172.21.15.69:52962) 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f0589400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480bb420 0x55d8480b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f058a800640 1 -- 172.21.15.69:0/3574953878 learned_addr learned my addr 172.21.15.69:0/3574953878 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.277+0000 7f0589e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38696/0 (socket says 172.21.15.69:38696) 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.278+0000 7f0589e00640 1 -- 172.21.15.69:0/3574953878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480c4000 msgr2=0x55d8480dbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.278+0000 7f0589e00640 1 --2- 172.21.15.69:0/3574953878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480c4000 0x55d8480dbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.278+0000 7f0589e00640 1 -- 172.21.15.69:0/3574953878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480bb420 msgr2=0x55d8480b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.278+0000 7f0589e00640 1 --2- 172.21.15.69:0/3574953878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480bb420 0x55d8480b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.278+0000 7f0589e00640 1 -- 172.21.15.69:0/3574953878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8480bb250 con 0x55d8480bab00 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.278+0000 7f0589400640 1 --2- 172.21.15.69:0/3574953878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480bb420 0x55d8480b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.278+0000 7f0589e00640 1 --2- 172.21.15.69:0/3574953878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cfc20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f057c0115a0 tx=0x7f057c005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9dc3123e6ffea7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.278+0000 7f0576a00640 1 -- 172.21.15.69:0/3574953878 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c027070 con 0x55d8480bab00 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.278+0000 7f0576a00640 1 -- 172.21.15.69:0/3574953878 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f057c004840 con 0x55d8480bab00 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.278+0000 7f0576a00640 1 -- 172.21.15.69:0/3574953878 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c031080 con 0x55d8480bab00 2024-03-19T01:14:27.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.279+0000 7f0593e0c740 1 -- 172.21.15.69:0/3574953878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 msgr2=0x55d8480cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.279+0000 7f0593e0c740 1 --2- 172.21.15.69:0/3574953878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cfc20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f057c0115a0 tx=0x7f057c005c50 comp rx=0 tx=0).stop 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.279+0000 7f0593e0c740 1 -- 172.21.15.69:0/3574953878 shutdown_connections 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.279+0000 7f0593e0c740 1 --2- 172.21.15.69:0/3574953878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480c4000 0x55d8480dbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.279+0000 7f0593e0c740 1 --2- 172.21.15.69:0/3574953878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480bb420 0x55d8480b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.279+0000 7f0593e0c740 1 --2- 172.21.15.69:0/3574953878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cfc20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.279+0000 7f0593e0c740 1 -- 172.21.15.69:0/3574953878 >> 172.21.15.69:0/3574953878 conn(0x55d8480b5230 msgr2=0x55d8480b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.279+0000 7f0593e0c740 1 -- 172.21.15.69:0/3574953878 shutdown_connections 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.279+0000 7f0593e0c740 1 -- 172.21.15.69:0/3574953878 wait complete. 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.280+0000 7f0593e0c740 1 -- start start 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.280+0000 7f0593e0c740 1 Processor -- start 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bb420 0x55d8480cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0593e0c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480c4000 0x55d8480c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0593e0c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f058c1b6870 con 0x55d8480c4000 2024-03-19T01:14:27.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0593e0c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f058c1b60e0 con 0x55d8480bab00 2024-03-19T01:14:27.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0593e0c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f058c1b3320 con 0x55d8480bb420 2024-03-19T01:14:27.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0589e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0589e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38708/0 (socket says 172.21.15.69:38708) 2024-03-19T01:14:27.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0589e00640 1 -- 172.21.15.69:0/1184890302 learned_addr learned my addr 172.21.15.69:0/1184890302 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:27.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0589400640 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bb420 0x55d8480cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0593e0c740 10 client.0 _ll_register_callbacks cb 0x7f05780f6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:27.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f058a800640 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480c4000 0x55d8480c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0589e00640 1 -- 172.21.15.69:0/1184890302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bb420 msgr2=0x55d8480cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0589e00640 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bb420 0x55d8480cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0589e00640 1 -- 172.21.15.69:0/1184890302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480c4000 msgr2=0x55d8480c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0589e00640 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480c4000 0x55d8480c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.281+0000 7f0589e00640 1 -- 172.21.15.69:0/1184890302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d847cf2170 con 0x55d8480bab00 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.282+0000 7f0589e00640 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cc110 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f057c00a010 tx=0x7f057c009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f61a0e4e78f0938 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.282+0000 7f056be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.282+0000 7f056be00640 1 -- 172.21.15.69:0/1184890302 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c027030 con 0x55d8480bab00 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.282+0000 7f056be00640 1 -- 172.21.15.69:0/1184890302 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f057c036460 con 0x55d8480bab00 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.282+0000 7f0593e0c740 1 -- 172.21.15.69:0/1184890302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f057c004840 con 0x55d8480bab00 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.282+0000 7f056be00640 1 -- 172.21.15.69:0/1184890302 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f057c046050 con 0x55d8480bab00 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.282+0000 7f0593e0c740 10 client.26681 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.282+0000 7f0593e0c740 1 -- 172.21.15.69:0/1184890302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d84813a020 con 0x55d8480bab00 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.283+0000 7f056be00640 1 -- 172.21.15.69:0/1184890302 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f057c004840 con 0x55d8480bab00 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.283+0000 7f056be00640 1 -- 172.21.15.69:0/1184890302 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f057c021050 con 0x55d8480bab00 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.283+0000 7f0593e0c740 10 client.26681 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:27.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.283+0000 7f0593e0c740 10 client.26681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.283+0000 7f0593e0c740 20 client.26681 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.283+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d84813a710 0x55d84813cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.283+0000 7f0593e0c740 4 client.26681 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.283+0000 7f0593e0c740 1 -- 172.21.15.69:0/1184890302 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d848025840 con 0x55d84813a710 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.283+0000 7f0589400640 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d84813a710 0x55d84813cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.284+0000 7f056be00640 10 client.26681 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.284+0000 7f0589400640 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d84813a710 0x55d84813cb30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bcd6416a3991f02 server_cookie=ab5376e53bfa4a1f in_seq=0 out_seq=0 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f056be00640 1 -- 172.21.15.69:0/1184890302 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d848025840 con 0x55d84813a710 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f056be00640 10 client.26681 handle_command_reply: tid=0 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f0593e0c740 1 client.26681 shutdown 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f0593e0c740 1 -- 172.21.15.69:0/1184890302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 msgr2=0x55d8480cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cc110 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f057c00a010 tx=0x7f057c009870 comp rx=0 tx=0).stop 2024-03-19T01:14:27.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f0593e0c740 1 -- 172.21.15.69:0/1184890302 shutdown_connections 2024-03-19T01:14:27.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d84813a710 0x55d84813cb30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8480c4000 0x55d8480c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8480bb420 0x55d8480cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f056be00640 0 client.26681 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:27.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f0593e0c740 1 --2- 172.21.15.69:0/1184890302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8480bab00 0x55d8480cc110 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.285+0000 7f0593e0c740 1 -- 172.21.15.69:0/1184890302 >> 172.21.15.69:0/1184890302 conn(0x55d8480b5230 msgr2=0x55d8480b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.286+0000 7f0593e0c740 1 -- 172.21.15.69:0/1184890302 shutdown_connections 2024-03-19T01:14:27.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.286+0000 7f0593e0c740 1 -- 172.21.15.69:0/1184890302 wait complete. 2024-03-19T01:14:27.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.286+0000 7f0593e0c740 20 client.26681 trim_cache size 0 max 16384 2024-03-19T01:14:27.292 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:14:27.292 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:14:27.292 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "f2ea6c40-85ce-485a-b25a-468e07ee8e08", 2024-03-19T01:14:27.292 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:14:27.292 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:14:27.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.286+0000 7f0591200640 1 -- 172.21.15.69:0/3161219317 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f05700a2970 msgr2=0x7f05700a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.286+0000 7f0591200640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f05700a2970 0x7f05700a4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f058c06aed0 tx=0x7f0584009500 comp rx=0 tx=0).stop 2024-03-19T01:14:27.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.287+0000 7f0591200640 1 -- 172.21.15.69:0/3161219317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058c103380 msgr2=0x7f058c103e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.287+0000 7f0591200640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058c103380 0x7f058c103e10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f057802c210 tx=0x7f0578005e80 comp rx=0 tx=0).stop 2024-03-19T01:14:27.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.287+0000 7f0591200640 1 -- 172.21.15.69:0/3161219317 shutdown_connections 2024-03-19T01:14:27.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.287+0000 7f0591200640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f058c10be30 0x7f058c104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.287+0000 7f0591200640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f05700a2970 0x7f05700a4e30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.287+0000 7f0591200640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058c103380 0x7f058c103e10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.287+0000 7f0591200640 1 --2- 172.21.15.69:0/3161219317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f058c102a60 0x7f058c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.287+0000 7f0591200640 1 -- 172.21.15.69:0/3161219317 >> 172.21.15.69:0/3161219317 conn(0x7f058c076aa0 msgr2=0x7f058c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.287+0000 7f0591200640 1 -- 172.21.15.69:0/3161219317 shutdown_connections 2024-03-19T01:14:27.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.287+0000 7f0591200640 1 -- 172.21.15.69:0/3161219317 wait complete. 2024-03-19T01:14:27.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:14:27.312 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:14:27.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.612+0000 7f1b05e00640 1 -- 172.21.15.69:0/3481513113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0010e050 msgr2=0x7f1b00110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.612+0000 7f1b05e00640 1 --2- 172.21.15.69:0/3481513113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0010e050 0x7f1b00110460 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009ca0 tx=0x7f1af802c190 comp rx=0 tx=0).stop 2024-03-19T01:14:27.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.612+0000 7f1b05e00640 1 -- 172.21.15.69:0/3481513113 shutdown_connections 2024-03-19T01:14:27.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.612+0000 7f1b05e00640 1 --2- 172.21.15.69:0/3481513113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0010e050 0x7f1b00110460 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.612+0000 7f1b05e00640 1 --2- 172.21.15.69:0/3481513113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b00074510 0x7f1b0010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.612+0000 7f1b05e00640 1 --2- 172.21.15.69:0/3481513113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00073bf0 0x7f1b00073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.612+0000 7f1b05e00640 1 -- 172.21.15.69:0/3481513113 >> 172.21.15.69:0/3481513113 conn(0x7f1b00078c20 msgr2=0x7f1b00079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.613+0000 7f1b05e00640 1 -- 172.21.15.69:0/3481513113 shutdown_connections 2024-03-19T01:14:27.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.613+0000 7f1b05e00640 1 -- 172.21.15.69:0/3481513113 wait complete. 2024-03-19T01:14:27.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.614+0000 7f1b05e00640 1 Processor -- start 2024-03-19T01:14:27.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.614+0000 7f1b05e00640 1 -- start start 2024-03-19T01:14:27.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.615+0000 7f1b05e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b00073bf0 0x7f1b0010d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.615+0000 7f1b05e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00074510 0x7f1b0010aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.615+0000 7f1b05e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0010e050 0x7f1b0010b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.615+0000 7f1afea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b00073bf0 0x7f1b0010d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.615+0000 7f1afea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b00073bf0 0x7f1b0010d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:52992/0 (socket says 172.21.15.69:52992) 2024-03-19T01:14:27.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.615+0000 7f1afea00640 1 -- 172.21.15.69:0/349910088 learned_addr learned my addr 172.21.15.69:0/349910088 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:27.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.615+0000 7f1b05e00640 1 -- 172.21.15.69:0/349910088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00113db0 con 0x7f1b00074510 2024-03-19T01:14:27.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.616+0000 7f1b05e00640 1 -- 172.21.15.69:0/349910088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00113c30 con 0x7f1b0010e050 2024-03-19T01:14:27.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.616+0000 7f1b05e00640 1 -- 172.21.15.69:0/349910088 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00113f30 con 0x7f1b00073bf0 2024-03-19T01:14:27.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.616+0000 7f1aff400640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0010e050 0x7f1b0010b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.616+0000 7f1afe000640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00074510 0x7f1b0010aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.616+0000 7f1aff400640 1 -- 172.21.15.69:0/349910088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b00073bf0 msgr2=0x7f1b0010d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.616+0000 7f1aff400640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b00073bf0 0x7f1b0010d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.616+0000 7f1aff400640 1 -- 172.21.15.69:0/349910088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00074510 msgr2=0x7f1b0010aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.616+0000 7f1aff400640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00074510 0x7f1b0010aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.616+0000 7f1aff400640 1 -- 172.21.15.69:0/349910088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0010b8d0 con 0x7f1b0010e050 2024-03-19T01:14:27.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.616+0000 7f1afea00640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b00073bf0 0x7f1b0010d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:14:27.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.617+0000 7f1afe000640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00074510 0x7f1b0010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:27.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.618+0000 7f1aff400640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0010e050 0x7f1b0010b010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1af802c7a0 tx=0x7f1af8035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.618+0000 7f1b04a00640 1 -- 172.21.15.69:0/349910088 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8039040 con 0x7f1b0010e050 2024-03-19T01:14:27.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.618+0000 7f1b04a00640 1 -- 172.21.15.69:0/349910088 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af8044070 con 0x7f1b0010e050 2024-03-19T01:14:27.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.618+0000 7f1b04a00640 1 -- 172.21.15.69:0/349910088 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af80328d0 con 0x7f1b0010e050 2024-03-19T01:14:27.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.618+0000 7f1b05e00640 1 -- 172.21.15.69:0/349910088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b001aab70 con 0x7f1b0010e050 2024-03-19T01:14:27.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.618+0000 7f1b05e00640 1 -- 172.21.15.69:0/349910088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b001aafe0 con 0x7f1b0010e050 2024-03-19T01:14:27.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.624+0000 7f1b08a09740 1 Processor -- start 2024-03-19T01:14:27.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.624+0000 7f1b04a00640 1 -- 172.21.15.69:0/349910088 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f1af8032a70 con 0x7f1b0010e050 2024-03-19T01:14:27.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.625+0000 7f1b08a09740 1 -- start start 2024-03-19T01:14:27.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.627+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff68444120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.627+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff68444660 0x55ff6844cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.627+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6844d240 0x55ff6844f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.627+0000 7f1b08a09740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ff683a82c0 con 0x55ff68443d40 2024-03-19T01:14:27.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.627+0000 7f1b08a09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff68350b60 con 0x55ff6844d240 2024-03-19T01:14:27.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.627+0000 7f1b08a09740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ff683a4360 con 0x55ff68444660 2024-03-19T01:14:27.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.627+0000 7f1b04a00640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1ae40a2970 0x7f1ae40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.627+0000 7f1b04a00640 1 -- 172.21.15.69:0/349910088 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1af80f15c0 con 0x7f1b0010e050 2024-03-19T01:14:27.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1afe000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff68444120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1afe000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff68444120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36564/0 (socket says 172.21.15.69:36564) 2024-03-19T01:14:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1afea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff68444660 0x55ff6844cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1afea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff68444660 0x55ff6844cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:53004/0 (socket says 172.21.15.69:53004) 2024-03-19T01:14:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1afe000640 1 -- 172.21.15.69:0/532139664 learned_addr learned my addr 172.21.15.69:0/532139664 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1aff400640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1ae40a2970 0x7f1ae40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1afe000640 1 --2- 172.21.15.69:0/532139664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6844d240 0x55ff6844f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1afea00640 1 -- 172.21.15.69:0/532139664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6844d240 msgr2=0x55ff6844f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1afea00640 1 --2- 172.21.15.69:0/532139664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6844d240 0x55ff6844f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1afea00640 1 -- 172.21.15.69:0/532139664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 msgr2=0x55ff68444120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1afea00640 1 --2- 172.21.15.69:0/532139664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff68444120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.628+0000 7f1afea00640 1 -- 172.21.15.69:0/532139664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff681867d0 con 0x55ff68444660 2024-03-19T01:14:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.629+0000 7f1afea00640 1 --2- 172.21.15.69:0/532139664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff68444660 0x55ff6844cd00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005fd0 tx=0x7f1aec00f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99ec78c7002b27f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.629+0000 7f1aff400640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1ae40a2970 0x7f1ae40a4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1af8004af0 tx=0x7f1af8004690 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.629+0000 7f1aebe00640 1 -- 172.21.15.69:0/532139664 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec01d070 con 0x55ff68444660 2024-03-19T01:14:27.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.629+0000 7f1aebe00640 1 -- 172.21.15.69:0/532139664 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aec010de0 con 0x55ff68444660 2024-03-19T01:14:27.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.629+0000 7f1aebe00640 1 -- 172.21.15.69:0/532139664 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec016340 con 0x55ff68444660 2024-03-19T01:14:27.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.631+0000 7f1b08a09740 1 -- 172.21.15.69:0/532139664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff68444660 msgr2=0x55ff6844cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.631+0000 7f1b08a09740 1 --2- 172.21.15.69:0/532139664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff68444660 0x55ff6844cd00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005fd0 tx=0x7f1aec00f8d0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.632+0000 7f1b08a09740 1 -- 172.21.15.69:0/532139664 shutdown_connections 2024-03-19T01:14:27.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.632+0000 7f1b08a09740 1 --2- 172.21.15.69:0/532139664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff6844d240 0x55ff6844f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.632+0000 7f1b08a09740 1 --2- 172.21.15.69:0/532139664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff68444660 0x55ff6844cd00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.632+0000 7f1b08a09740 1 --2- 172.21.15.69:0/532139664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff68444120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.632+0000 7f1b08a09740 1 -- 172.21.15.69:0/532139664 >> 172.21.15.69:0/532139664 conn(0x55ff6843e470 msgr2=0x55ff6843e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.632+0000 7f1b08a09740 1 -- 172.21.15.69:0/532139664 shutdown_connections 2024-03-19T01:14:27.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.632+0000 7f1b08a09740 1 -- 172.21.15.69:0/532139664 wait complete. 2024-03-19T01:14:27.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.632+0000 7f1b08a09740 1 -- start start 2024-03-19T01:14:27.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.633+0000 7f1b08a09740 1 Processor -- start 2024-03-19T01:14:27.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.634+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff6844a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.634+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff6844a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.634+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 0x55ff6844ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.634+0000 7f1b08a09740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ff68450030 con 0x55ff68443d40 2024-03-19T01:14:27.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.634+0000 7f1b08a09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff68363fc0 con 0x55ff68444660 2024-03-19T01:14:27.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.634+0000 7f1b08a09740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ff684501b0 con 0x55ff6844d240 2024-03-19T01:14:27.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.634+0000 7f1b08a09740 10 client.0 _ll_register_callbacks cb 0x55ff68363990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.634+0000 7f1afea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff6844a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.634+0000 7f1afea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff6844a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38738/0 (socket says 172.21.15.69:38738) 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.634+0000 7f1afea00640 1 -- 172.21.15.69:0/3202864735 learned_addr learned my addr 172.21.15.69:0/3202864735 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.635+0000 7f1afe000640 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff6844a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.635+0000 7f1afea00640 1 -- 172.21.15.69:0/3202864735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 msgr2=0x55ff6844ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.635+0000 7f1afea00640 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 0x55ff6844ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.635+0000 7f1afea00640 1 -- 172.21.15.69:0/3202864735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 msgr2=0x55ff6844a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.635+0000 7f1afea00640 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff6844a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.635+0000 7f1afea00640 1 -- 172.21.15.69:0/3202864735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff680bc3f0 con 0x55ff68444660 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.635+0000 7f1afe000640 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff6844a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.635+0000 7f1afea00640 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff6844a7d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1aec019f60 tx=0x7f1aec00abe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc0dc7c5e939e5ea server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.635+0000 7f1ae8c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.636+0000 7f1ae8c00640 1 -- 172.21.15.69:0/3202864735 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec01d040 con 0x55ff68444660 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.636+0000 7f1b08a09740 1 -- 172.21.15.69:0/3202864735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aec010de0 con 0x55ff68444660 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.636+0000 7f1ae8c00640 1 -- 172.21.15.69:0/3202864735 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1aec010900 con 0x55ff68444660 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.636+0000 7f1ae8c00640 1 -- 172.21.15.69:0/3202864735 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1aec031050 con 0x55ff68444660 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.636+0000 7f1b08a09740 10 client.26693 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.636+0000 7f1b08a09740 1 -- 172.21.15.69:0/3202864735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ff6807c0f0 con 0x55ff68444660 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.637+0000 7f1ae8c00640 1 -- 172.21.15.69:0/3202864735 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aec010de0 con 0x55ff68444660 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.637+0000 7f1ae8c00640 1 -- 172.21.15.69:0/3202864735 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f1aec004830 con 0x55ff68444660 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.637+0000 7f1b08a09740 10 client.26693 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.637+0000 7f1b08a09740 10 client.26693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.637+0000 7f1b08a09740 20 client.26693 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.637+0000 7f1b08a09740 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff6845d580 0x55ff6845f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.637+0000 7f1b08a09740 4 client.26693 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:14:27.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.637+0000 7f1b08a09740 1 -- 172.21.15.69:0/3202864735 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff68349610 con 0x55ff6845d580 2024-03-19T01:14:27.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.637+0000 7f1afe000640 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff6845d580 0x55ff6845f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.638+0000 7f1afe000640 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff6845d580 0x55ff6845f970 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e0e49253fc73ac server_cookie=ec42d409fdd62d5f in_seq=0 out_seq=0 2024-03-19T01:14:27.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.638+0000 7f1ae8c00640 10 client.26693 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:27.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.641+0000 7f1ae8c00640 1 -- 172.21.15.69:0/3202864735 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ff68349610 con 0x55ff6845d580 2024-03-19T01:14:27.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.641+0000 7f1ae8c00640 10 client.26693 handle_command_reply: tid=0 2024-03-19T01:14:27.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.641+0000 7f1b08a09740 1 client.26693 shutdown 2024-03-19T01:14:27.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.641+0000 7f1b08a09740 1 -- 172.21.15.69:0/3202864735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 msgr2=0x55ff6844a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.641+0000 7f1b08a09740 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff6844a7d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1aec019f60 tx=0x7f1aec00abe0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.641+0000 7f1b08a09740 1 -- 172.21.15.69:0/3202864735 shutdown_connections 2024-03-19T01:14:27.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.641+0000 7f1b08a09740 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff6845d580 0x55ff6845f970 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.642+0000 7f1b08a09740 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 0x55ff6844ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.642+0000 7f1b08a09740 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff6844a7d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.642+0000 7f1b08a09740 1 --2- 172.21.15.69:0/3202864735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff6844a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.642+0000 7f1b08a09740 1 -- 172.21.15.69:0/3202864735 >> 172.21.15.69:0/3202864735 conn(0x55ff6843e470 msgr2=0x55ff684401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.642+0000 7f1ae8c00640 0 client.26693 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:27.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.642+0000 7f1b08a09740 1 -- 172.21.15.69:0/3202864735 shutdown_connections 2024-03-19T01:14:27.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.643+0000 7f1b08a09740 1 -- 172.21.15.69:0/3202864735 wait complete. 2024-03-19T01:14:27.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.643+0000 7f1b08a09740 20 client.26693 trim_cache size 0 max 16384 2024-03-19T01:14:27.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.685+0000 7f1b08a09740 1 Processor -- start 2024-03-19T01:14:27.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.685+0000 7f1b08a09740 1 -- start start 2024-03-19T01:14:27.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.686+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff6844c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.686+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff684493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.686+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 0x55ff68458dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.686+0000 7f1b08a09740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001b51e0 con 0x55ff68443d40 2024-03-19T01:14:27.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.686+0000 7f1b08a09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001b6ea0 con 0x55ff68444660 2024-03-19T01:14:27.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.686+0000 7f1b08a09740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001b58b0 con 0x55ff6844d240 2024-03-19T01:14:27.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.686+0000 7f1afea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff684493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.686+0000 7f1afea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff684493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38750/0 (socket says 172.21.15.69:38750) 2024-03-19T01:14:27.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.687+0000 7f1afea00640 1 -- 172.21.15.69:0/3752829608 learned_addr learned my addr 172.21.15.69:0/3752829608 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:27.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.687+0000 7f1afea00640 1 -- 172.21.15.69:0/3752829608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 msgr2=0x55ff68458dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:14:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.687+0000 7f1afea00640 1 --2- 172.21.15.69:0/3752829608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 0x55ff68458dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.687+0000 7f1afea00640 1 -- 172.21.15.69:0/3752829608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 msgr2=0x55ff6844c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.687+0000 7f1afea00640 1 --2- 172.21.15.69:0/3752829608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff6844c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.687+0000 7f1afea00640 1 -- 172.21.15.69:0/3752829608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff6843a0c0 con 0x55ff68444660 2024-03-19T01:14:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.687+0000 7f1afea00640 1 --2- 172.21.15.69:0/3752829608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff684493f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1aec004c00 tx=0x7f1aec004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cedce520d51d8338 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.690+0000 7f1aeb400640 1 -- 172.21.15.69:0/3752829608 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec039020 con 0x55ff68444660 2024-03-19T01:14:27.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.690+0000 7f1aeb400640 1 -- 172.21.15.69:0/3752829608 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aec028070 con 0x55ff68444660 2024-03-19T01:14:27.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.690+0000 7f1aeb400640 1 -- 172.21.15.69:0/3752829608 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec01d040 con 0x55ff68444660 2024-03-19T01:14:27.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.692+0000 7f1b08a09740 1 -- 172.21.15.69:0/3752829608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 msgr2=0x55ff684493f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.692+0000 7f1b08a09740 1 --2- 172.21.15.69:0/3752829608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff684493f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1aec004c00 tx=0x7f1aec004ef0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.692+0000 7f1b08a09740 1 -- 172.21.15.69:0/3752829608 shutdown_connections 2024-03-19T01:14:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.692+0000 7f1b08a09740 1 --2- 172.21.15.69:0/3752829608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 0x55ff68458dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.692+0000 7f1b08a09740 1 --2- 172.21.15.69:0/3752829608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff684493f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.692+0000 7f1b08a09740 1 --2- 172.21.15.69:0/3752829608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff6844c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.692+0000 7f1b08a09740 1 -- 172.21.15.69:0/3752829608 >> 172.21.15.69:0/3752829608 conn(0x55ff6843e470 msgr2=0x55ff6843ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.693+0000 7f1b08a09740 1 -- 172.21.15.69:0/3752829608 shutdown_connections 2024-03-19T01:14:27.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.693+0000 7f1b08a09740 1 -- 172.21.15.69:0/3752829608 wait complete. 2024-03-19T01:14:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.693+0000 7f1b08a09740 1 -- start start 2024-03-19T01:14:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.694+0000 7f1b08a09740 1 Processor -- start 2024-03-19T01:14:27.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.695+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff684555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.695+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff68446b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.696+0000 7f1b08a09740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 0x55ff68447060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.696+0000 7f1b08a09740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001b6710 con 0x55ff68443d40 2024-03-19T01:14:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.696+0000 7f1b08a09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001b5f80 con 0x55ff68444660 2024-03-19T01:14:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.696+0000 7f1b08a09740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001b31c0 con 0x55ff6844d240 2024-03-19T01:14:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.696+0000 7f1b08a09740 10 client.0 _ll_register_callbacks cb 0x7f1aec018770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.696+0000 7f1afea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff68446b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.696+0000 7f1afea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff68446b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38764/0 (socket says 172.21.15.69:38764) 2024-03-19T01:14:27.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.696+0000 7f1afea00640 1 -- 172.21.15.69:0/1591812358 learned_addr learned my addr 172.21.15.69:0/1591812358 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:27.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.696+0000 7f1afe000640 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff684555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.697+0000 7f1afea00640 1 -- 172.21.15.69:0/1591812358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 msgr2=0x55ff68447060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.697+0000 7f1afea00640 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 0x55ff68447060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.697+0000 7f1afea00640 1 -- 172.21.15.69:0/1591812358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 msgr2=0x55ff684555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:27.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.697+0000 7f1afea00640 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff684555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.697+0000 7f1afea00640 1 -- 172.21.15.69:0/1591812358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff68447890 con 0x55ff68444660 2024-03-19T01:14:27.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.697+0000 7f1afea00640 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff68446b20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00f910 tx=0x7f1aec00aea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6be4c7bbc258659d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:27.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.697+0000 7f1ae8c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:14:27.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.697+0000 7f1ae8c00640 1 -- 172.21.15.69:0/1591812358 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec01d040 con 0x55ff68444660 2024-03-19T01:14:27.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.698+0000 7f1ae8c00640 1 -- 172.21.15.69:0/1591812358 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1aec004860 con 0x55ff68444660 2024-03-19T01:14:27.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.698+0000 7f1b08a09740 1 -- 172.21.15.69:0/1591812358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aec028070 con 0x55ff68444660 2024-03-19T01:14:27.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.698+0000 7f1ae8c00640 1 -- 172.21.15.69:0/1591812358 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1aec039040 con 0x55ff68444660 2024-03-19T01:14:27.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.699+0000 7f1b08a09740 10 client.26699 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:27.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.699+0000 7f1b08a09740 1 -- 172.21.15.69:0/1591812358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ff684c35d0 con 0x55ff68444660 2024-03-19T01:14:27.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.700+0000 7f1ae8c00640 1 -- 172.21.15.69:0/1591812358 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aec028070 con 0x55ff68444660 2024-03-19T01:14:27.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.700+0000 7f1ae8c00640 1 -- 172.21.15.69:0/1591812358 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f1aec010700 con 0x55ff68444660 2024-03-19T01:14:27.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.700+0000 7f1b08a09740 10 client.26699 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:27.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.700+0000 7f1b08a09740 10 client.26699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:14:27.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.700+0000 7f1b08a09740 20 client.26699 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:27.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.700+0000 7f1b08a09740 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff684c3d50 0x55ff684c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:27.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.700+0000 7f1b08a09740 4 client.26699 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:14:27.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.700+0000 7f1b08a09740 1 -- 172.21.15.69:0/1591812358 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff683b5800 con 0x55ff684c3d50 2024-03-19T01:14:27.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.700+0000 7f1afe000640 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff684c3d50 0x55ff684c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:27.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.702+0000 7f1afe000640 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff684c3d50 0x55ff684c6170 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=264d240d0bbfbd58 server_cookie=c4b9d0bf738bb023 in_seq=0 out_seq=0 2024-03-19T01:14:27.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.702+0000 7f1ae8c00640 10 client.26699 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:27.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.702+0000 7f1ae8c00640 1 -- 172.21.15.69:0/1591812358 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55ff683b5800 con 0x55ff684c3d50 2024-03-19T01:14:27.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.702+0000 7f1ae8c00640 10 client.26699 handle_command_reply: tid=0 2024-03-19T01:14:27.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.703+0000 7f1b08a09740 1 client.26699 shutdown 2024-03-19T01:14:27.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.703+0000 7f1b08a09740 1 -- 172.21.15.69:0/1591812358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 msgr2=0x55ff68446b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.703+0000 7f1b08a09740 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff68446b20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00f910 tx=0x7f1aec00aea0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.704+0000 7f1b08a09740 1 -- 172.21.15.69:0/1591812358 shutdown_connections 2024-03-19T01:14:27.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.704+0000 7f1b08a09740 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff684c3d50 0x55ff684c6170 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.704+0000 7f1ae8c00640 0 client.26699 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:27.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.704+0000 7f1b08a09740 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff6844d240 0x55ff68447060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.704+0000 7f1b08a09740 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff68444660 0x55ff68446b20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.704+0000 7f1b08a09740 1 --2- 172.21.15.69:0/1591812358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff68443d40 0x55ff684555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.704+0000 7f1b08a09740 1 -- 172.21.15.69:0/1591812358 >> 172.21.15.69:0/1591812358 conn(0x55ff6843e470 msgr2=0x55ff6843ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.704+0000 7f1b08a09740 1 -- 172.21.15.69:0/1591812358 shutdown_connections 2024-03-19T01:14:27.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.704+0000 7f1b08a09740 1 -- 172.21.15.69:0/1591812358 wait complete. 2024-03-19T01:14:27.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.705+0000 7f1b08a09740 20 client.26699 trim_cache size 0 max 16384 2024-03-19T01:14:27.706 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:14:27.706 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (7342 inodes in the stack)", 2024-03-19T01:14:27.706 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:14:27.706 INFO:teuthology.orchestra.run.smithi069.stdout: "f2ea6c40-85ce-485a-b25a-468e07ee8e08": { 2024-03-19T01:14:27.706 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:14:27.707 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "f2ea6c40-85ce-485a-b25a-468e07ee8e08", 2024-03-19T01:14:27.707 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:14:27.707 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:14:27.707 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:14:27.707 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:14:27.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.706+0000 7f1b05e00640 1 -- 172.21.15.69:0/349910088 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1ae40a2970 msgr2=0x7f1ae40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.706+0000 7f1b05e00640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1ae40a2970 0x7f1ae40a4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1af8004af0 tx=0x7f1af8004690 comp rx=0 tx=0).stop 2024-03-19T01:14:27.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.706+0000 7f1b05e00640 1 -- 172.21.15.69:0/349910088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0010e050 msgr2=0x7f1b0010b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:27.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.706+0000 7f1b05e00640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0010e050 0x7f1b0010b010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1af802c7a0 tx=0x7f1af8035d10 comp rx=0 tx=0).stop 2024-03-19T01:14:27.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.706+0000 7f1b05e00640 1 -- 172.21.15.69:0/349910088 shutdown_connections 2024-03-19T01:14:27.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.706+0000 7f1b05e00640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1ae40a2970 0x7f1ae40a4e30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.706+0000 7f1b05e00640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0010e050 0x7f1b0010b010 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.706+0000 7f1b05e00640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b00074510 0x7f1b0010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.706+0000 7f1b05e00640 1 --2- 172.21.15.69:0/349910088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b00073bf0 0x7f1b0010d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:27.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.706+0000 7f1b05e00640 1 -- 172.21.15.69:0/349910088 >> 172.21.15.69:0/349910088 conn(0x7f1b00078c20 msgr2=0x7f1b000ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:27.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.707+0000 7f1b05e00640 1 -- 172.21.15.69:0/349910088 shutdown_connections 2024-03-19T01:14:27.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:27.707+0000 7f1b05e00640 1 -- 172.21.15.69:0/349910088 wait complete. 2024-03-19T01:14:27.734 INFO:tasks.cephfs.filesystem:scrub status for tag:f2ea6c40-85ce-485a-b25a-468e07ee8e08 - {'path': '/', 'tag': 'f2ea6c40-85ce-485a-b25a-468e07ee8e08', 'options': 'recursive,force'} 2024-03-19T01:14:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: pgmap v116: 129 pgs: 129 active+clean; 1000 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 26 MiB/s wr, 119 op/s 2024-03-19T01:14:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:14:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:28.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:28.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:28.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:28.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: pgmap v116: 129 pgs: 129 active+clean; 1000 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 26 MiB/s wr, 119 op/s 2024-03-19T01:14:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:14:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:28.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:28.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:28.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:28.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: pgmap v116: 129 pgs: 129 active+clean; 1000 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 26 MiB/s wr, 119 op/s 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:28.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:28.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:28.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:28.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:29.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:29.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:29.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:29.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: pgmap v117: 129 pgs: 129 active+clean; 1004 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 24 MiB/s wr, 119 op/s 2024-03-19T01:14:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:30.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: pgmap v117: 129 pgs: 129 active+clean; 1004 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 24 MiB/s wr, 119 op/s 2024-03-19T01:14:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:30.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: pgmap v117: 129 pgs: 129 active+clean; 1004 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 24 MiB/s wr, 119 op/s 2024-03-19T01:14:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:30.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:30.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:31.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:31.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:31.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:31.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:31.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:31.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: pgmap v118: 129 pgs: 129 active+clean; 1004 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 832 KiB/s wr, 34 op/s 2024-03-19T01:14:32.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:32.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: pgmap v118: 129 pgs: 129 active+clean; 1004 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 832 KiB/s wr, 34 op/s 2024-03-19T01:14:32.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:32.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: pgmap v118: 129 pgs: 129 active+clean; 1004 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 832 KiB/s wr, 34 op/s 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:32.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:33.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:33.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:33.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:33.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: pgmap v119: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 363 op/s 2024-03-19T01:14:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: pgmap v119: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 363 op/s 2024-03-19T01:14:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:34.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:34.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:34.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:34.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: pgmap v119: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 363 op/s 2024-03-19T01:14:34.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:34.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:34.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:34.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:36.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:35 smithi086 ceph-mon[66829]: pgmap v120: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 4.2 MiB/s wr, 344 op/s 2024-03-19T01:14:36.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:35 smithi069 ceph-mon[31727]: pgmap v120: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 4.2 MiB/s wr, 344 op/s 2024-03-19T01:14:36.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:35 smithi163 ceph-mon[67428]: pgmap v120: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 4.2 MiB/s wr, 344 op/s 2024-03-19T01:14:38.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:37 smithi086 ceph-mon[66829]: pgmap v121: 129 pgs: 129 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 396 op/s 2024-03-19T01:14:38.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:37 smithi069 ceph-mon[31727]: pgmap v121: 129 pgs: 129 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 396 op/s 2024-03-19T01:14:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:37 smithi163 ceph-mon[67428]: pgmap v121: 129 pgs: 129 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 396 op/s 2024-03-19T01:14:40.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:39 smithi086 ceph-mon[66829]: pgmap v122: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-03-19T01:14:40.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:39 smithi069 ceph-mon[31727]: pgmap v122: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-03-19T01:14:40.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:39 smithi163 ceph-mon[67428]: pgmap v122: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-03-19T01:14:41.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:41.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:41.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:41.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:41.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:41.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:41.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:41.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:41.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:41.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:41.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:41.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:41.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:41.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:41.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:41.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:41.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:41.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:41.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:41.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:41.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:41.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:41.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:41.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:42.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:41 smithi086 ceph-mon[66829]: pgmap v123: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 10 MiB/s wr, 710 op/s 2024-03-19T01:14:42.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:42.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:42.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:42.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:42.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:42.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:42.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:41 smithi069 ceph-mon[31727]: pgmap v123: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 10 MiB/s wr, 710 op/s 2024-03-19T01:14:42.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:42.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:42.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:42.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:42.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:42.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:42.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:41 smithi163 ceph-mon[67428]: pgmap v123: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 10 MiB/s wr, 710 op/s 2024-03-19T01:14:42.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:42.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:42.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:42.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:42.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:42.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:43.212 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:43.212 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:43.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:43.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:43.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:43.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:43.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:43.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:43.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:43.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:43.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:43.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:43.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:43.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:43.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:43.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:43.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:43.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:43 smithi086 ceph-mon[66829]: pgmap v124: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-03-19T01:14:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:43 smithi069 ceph-mon[31727]: pgmap v124: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-03-19T01:14:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:44.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:43 smithi163 ceph-mon[67428]: pgmap v124: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-03-19T01:14:44.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:46.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:45 smithi086 ceph-mon[66829]: pgmap v125: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.2 MiB/s wr, 403 op/s 2024-03-19T01:14:46.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:45 smithi069 ceph-mon[31727]: pgmap v125: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.2 MiB/s wr, 403 op/s 2024-03-19T01:14:46.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:45 smithi163 ceph-mon[67428]: pgmap v125: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.2 MiB/s wr, 403 op/s 2024-03-19T01:14:47.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:47.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:47.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:47.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:47.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:47.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:47.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:47.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:47.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:47.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:47.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:47.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:47.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:47.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:47.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: pgmap v126: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 409 op/s 2024-03-19T01:14:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: pgmap v126: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 409 op/s 2024-03-19T01:14:48.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:48.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:48.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:48.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:48.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:48.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:48.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:48.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:48.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: pgmap v126: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 409 op/s 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:49.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:49.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:49.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:49.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:49.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:49.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:49.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:50.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:49 smithi086 ceph-mon[66829]: pgmap v127: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.5 MiB/s wr, 365 op/s 2024-03-19T01:14:50.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:49 smithi069 ceph-mon[31727]: pgmap v127: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.5 MiB/s wr, 365 op/s 2024-03-19T01:14:50.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:49 smithi163 ceph-mon[67428]: pgmap v127: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.5 MiB/s wr, 365 op/s 2024-03-19T01:14:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:51.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:51.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:51.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:51.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:52.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:51 smithi086 ceph-mon[66829]: pgmap v128: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 35 op/s 2024-03-19T01:14:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:51 smithi069 ceph-mon[31727]: pgmap v128: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 35 op/s 2024-03-19T01:14:52.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:51 smithi163 ceph-mon[67428]: pgmap v128: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 35 op/s 2024-03-19T01:14:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:53 smithi086 ceph-mon[66829]: pgmap v129: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 164 op/s 2024-03-19T01:14:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:53 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:14:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:14:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:53 smithi069 ceph-mon[31727]: pgmap v129: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 164 op/s 2024-03-19T01:14:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:53 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:14:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:14:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:53 smithi163 ceph-mon[67428]: pgmap v129: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 164 op/s 2024-03-19T01:14:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:53 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:14:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:14:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:55.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:55.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:55 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:14:55.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:55 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:14:55.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:55 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:14:55.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:55.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:55 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:14:55.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:55 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:14:55.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:55 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:14:55.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:14:55.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:55 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:14:55.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:55 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:14:55.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:55 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:14:56.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:56 smithi069 ceph-mon[31727]: pgmap v130: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 143 op/s 2024-03-19T01:14:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:56 smithi163 ceph-mon[67428]: pgmap v130: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 143 op/s 2024-03-19T01:14:56.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:56 smithi086 ceph-mon[66829]: pgmap v130: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 143 op/s 2024-03-19T01:14:57.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:14:57.735 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:14:57.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.992+0000 7fa315a00640 1 -- 172.21.15.69:0/11452821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa310100430 msgr2=0x7fa310100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:57.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.992+0000 7fa315a00640 1 --2- 172.21.15.69:0/11452821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa310100430 0x7fa310100810 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa300009900 tx=0x7fa30002bc60 comp rx=0 tx=0).stop 2024-03-19T01:14:57.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.992+0000 7fa315a00640 1 -- 172.21.15.69:0/11452821 shutdown_connections 2024-03-19T01:14:57.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.992+0000 7fa315a00640 1 --2- 172.21.15.69:0/11452821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa31010dfe0 0x7fa3101103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:57.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.992+0000 7fa315a00640 1 --2- 172.21.15.69:0/11452821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310100de0 0x7fa31010daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:57.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.992+0000 7fa315a00640 1 --2- 172.21.15.69:0/11452821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa310100430 0x7fa310100810 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:57.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.992+0000 7fa315a00640 1 -- 172.21.15.69:0/11452821 >> 172.21.15.69:0/11452821 conn(0x7fa3100fc070 msgr2=0x7fa3100fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:57.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.992+0000 7fa315a00640 1 -- 172.21.15.69:0/11452821 shutdown_connections 2024-03-19T01:14:57.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.992+0000 7fa315a00640 1 -- 172.21.15.69:0/11452821 wait complete. 2024-03-19T01:14:57.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.993+0000 7fa315a00640 1 Processor -- start 2024-03-19T01:14:57.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.994+0000 7fa315a00640 1 -- start start 2024-03-19T01:14:57.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.994+0000 7fa315a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa310100430 0x7fa31019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:57.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.994+0000 7fa315a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310100de0 0x7fa31019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:57.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.994+0000 7fa315a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa31010dfe0 0x7fa31019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:57.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.994+0000 7fa315a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa310113d20 con 0x7fa310100de0 2024-03-19T01:14:57.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.994+0000 7fa315a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa310113ba0 con 0x7fa310100430 2024-03-19T01:14:57.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.994+0000 7fa315a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa310113ea0 con 0x7fa31010dfe0 2024-03-19T01:14:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.994+0000 7fa30fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa310100430 0x7fa31019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.994+0000 7fa30fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa310100430 0x7fa31019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34416/0 (socket says 172.21.15.69:34416) 2024-03-19T01:14:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.994+0000 7fa30fe00640 1 -- 172.21.15.69:0/1506028451 learned_addr learned my addr 172.21.15.69:0/1506028451 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.994+0000 7fa315000640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa31010dfe0 0x7fa31019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.995+0000 7fa30fe00640 1 -- 172.21.15.69:0/1506028451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa31010dfe0 msgr2=0x7fa31019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.995+0000 7fa30fe00640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa31010dfe0 0x7fa31019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.995+0000 7fa30fe00640 1 -- 172.21.15.69:0/1506028451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310100de0 msgr2=0x7fa31019a740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:14:57.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.995+0000 7fa30f400640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310100de0 0x7fa31019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:57.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.995+0000 7fa30fe00640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310100de0 0x7fa31019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:57.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.995+0000 7fa30fe00640 1 -- 172.21.15.69:0/1506028451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31019f1c0 con 0x7fa310100430 2024-03-19T01:14:57.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.995+0000 7fa315000640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa31010dfe0 0x7fa31019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:14:57.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.995+0000 7fa30f400640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310100de0 0x7fa31019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:14:57.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.996+0000 7fa30fe00640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa310100430 0x7fa31019a200 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa300005ab0 tx=0x7fa3000043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:57.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.996+0000 7fa30cc00640 1 -- 172.21.15.69:0/1506028451 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300041070 con 0x7fa310100430 2024-03-19T01:14:57.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.996+0000 7fa30cc00640 1 -- 172.21.15.69:0/1506028451 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30002ccc0 con 0x7fa310100430 2024-03-19T01:14:57.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.996+0000 7fa315a00640 1 -- 172.21.15.69:0/1506028451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31019f450 con 0x7fa310100430 2024-03-19T01:14:57.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.996+0000 7fa315a00640 1 -- 172.21.15.69:0/1506028451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31006b240 con 0x7fa310100430 2024-03-19T01:14:57.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:57.996+0000 7fa30cc00640 1 -- 172.21.15.69:0/1506028451 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300034440 con 0x7fa310100430 2024-03-19T01:14:58.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.001+0000 7fa319ddd740 1 Processor -- start 2024-03-19T01:14:58.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.001+0000 7fa30cc00640 1 -- 172.21.15.69:0/1506028451 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fa300044070 con 0x7fa310100430 2024-03-19T01:14:58.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.001+0000 7fa319ddd740 1 -- start start 2024-03-19T01:14:58.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.002+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eadb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.002+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eae060 0x557131eb6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.002+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 0x557131eb9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.002+0000 7fa319ddd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557131e092c0 con 0x557131eb6c40 2024-03-19T01:14:58.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.002+0000 7fa319ddd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557131db1b60 con 0x557131eae060 2024-03-19T01:14:58.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.002+0000 7fa319ddd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557131e05360 con 0x557131ead740 2024-03-19T01:14:58.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.002+0000 7fa315000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eadb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.003+0000 7fa315000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eadb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:34114/0 (socket says 172.21.15.69:34114) 2024-03-19T01:14:58.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.003+0000 7fa315000640 1 -- 172.21.15.69:0/407919400 learned_addr learned my addr 172.21.15.69:0/407919400 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:58.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.003+0000 7fa30cc00640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2f40a29f0 0x7fa2f40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.003+0000 7fa30cc00640 1 -- 172.21.15.69:0/1506028451 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa3000f0c10 con 0x7fa310100430 2024-03-19T01:14:58.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.003+0000 7fa30f400640 1 --2- 172.21.15.69:0/407919400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 0x557131eb9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.003+0000 7fa30fe00640 1 --2- 172.21.15.69:0/407919400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eae060 0x557131eb6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.004+0000 7fa30fe00640 1 -- 172.21.15.69:0/407919400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 msgr2=0x557131eadb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:58.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.004+0000 7fa30fe00640 1 --2- 172.21.15.69:0/407919400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eadb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.004+0000 7fa30fe00640 1 -- 172.21.15.69:0/407919400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 msgr2=0x557131eb9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:58.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.004+0000 7fa30fe00640 1 --2- 172.21.15.69:0/407919400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 0x557131eb9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.004+0000 7fa30fe00640 1 -- 172.21.15.69:0/407919400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557131be77d0 con 0x557131eae060 2024-03-19T01:14:58.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.004+0000 7fa315000640 1 --2- 172.21.15.69:0/407919400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eadb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:14:58.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.004+0000 7fa30f400640 1 --2- 172.21.15.69:0/407919400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 0x557131eb9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:58.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.004+0000 7fa30fe00640 1 --2- 172.21.15.69:0/407919400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eae060 0x557131eb6700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa3000ee050 tx=0x7fa300009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d20a65e20cfdab server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:58.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.005+0000 7fa305600640 1 -- 172.21.15.69:0/407919400 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30004b040 con 0x557131eae060 2024-03-19T01:14:58.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.005+0000 7fa305600640 1 -- 172.21.15.69:0/407919400 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa300004c80 con 0x557131eae060 2024-03-19T01:14:58.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.005+0000 7fa305600640 1 -- 172.21.15.69:0/407919400 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3000035f0 con 0x557131eae060 2024-03-19T01:14:58.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.005+0000 7fa315000640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2f40a29f0 0x7fa2f40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.006+0000 7fa319ddd740 1 -- 172.21.15.69:0/407919400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eae060 msgr2=0x557131eb6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:58.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.006+0000 7fa319ddd740 1 --2- 172.21.15.69:0/407919400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eae060 0x557131eb6700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa3000ee050 tx=0x7fa300009680 comp rx=0 tx=0).stop 2024-03-19T01:14:58.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.006+0000 7fa315000640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2f40a29f0 0x7fa2f40a4eb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa308009fd0 tx=0x7fa308004af0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.006+0000 7fa319ddd740 1 -- 172.21.15.69:0/407919400 shutdown_connections 2024-03-19T01:14:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.006+0000 7fa319ddd740 1 --2- 172.21.15.69:0/407919400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 0x557131eb9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.006+0000 7fa319ddd740 1 --2- 172.21.15.69:0/407919400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eae060 0x557131eb6700 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.006+0000 7fa319ddd740 1 --2- 172.21.15.69:0/407919400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eadb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.006+0000 7fa319ddd740 1 -- 172.21.15.69:0/407919400 >> 172.21.15.69:0/407919400 conn(0x557131ea7e70 msgr2=0x557131ea8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.007+0000 7fa319ddd740 1 -- 172.21.15.69:0/407919400 shutdown_connections 2024-03-19T01:14:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.007+0000 7fa319ddd740 1 -- 172.21.15.69:0/407919400 wait complete. 2024-03-19T01:14:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.008+0000 7fa319ddd740 1 -- start start 2024-03-19T01:14:58.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.008+0000 7fa319ddd740 1 Processor -- start 2024-03-19T01:14:58.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.009+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eb3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.010+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 0x557131eb40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.010+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 0x557131eb4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.010+0000 7fa30fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eb3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.010+0000 7fa30fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eb3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:34120/0 (socket says 172.21.15.69:34120) 2024-03-19T01:14:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.010+0000 7fa30f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 0x557131eb40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.010+0000 7fa30f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 0x557131eb40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56268/0 (socket says 172.21.15.69:56268) 2024-03-19T01:14:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.010+0000 7fa315000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 0x557131eb4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.010+0000 7fa315000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 0x557131eb4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34446/0 (socket says 172.21.15.69:34446) 2024-03-19T01:14:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.010+0000 7fa319ddd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557131eb97f0 con 0x557131eae060 2024-03-19T01:14:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.010+0000 7fa30fe00640 1 -- 172.21.15.69:0/2774682851 learned_addr learned my addr 172.21.15.69:0/2774682851 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:58.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.011+0000 7fa319ddd740 1 -- 172.21.15.69:0/2774682851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557131dc4fc0 con 0x557131eb6c40 2024-03-19T01:14:58.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.011+0000 7fa319ddd740 1 -- 172.21.15.69:0/2774682851 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557131eb9970 con 0x557131ead740 2024-03-19T01:14:58.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.011+0000 7fa30fe00640 1 -- 172.21.15.69:0/2774682851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 msgr2=0x557131eb4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:58.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.011+0000 7fa30fe00640 1 --2- 172.21.15.69:0/2774682851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 0x557131eb4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.011+0000 7fa30fe00640 1 -- 172.21.15.69:0/2774682851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 msgr2=0x557131eb40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:58.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.011+0000 7fa30fe00640 1 --2- 172.21.15.69:0/2774682851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 0x557131eb40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.011+0000 7fa30fe00640 1 -- 172.21.15.69:0/2774682851 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557131b1d3f0 con 0x557131ead740 2024-03-19T01:14:58.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.011+0000 7fa319ddd740 10 client.0 _ll_register_callbacks cb 0x557131dc4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:58.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.011+0000 7fa319ddd740 1 -- 172.21.15.69:0/2774682851 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa300004c80 con 0x557131ead740 2024-03-19T01:14:58.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.011+0000 7fa30fe00640 1 --2- 172.21.15.69:0/2774682851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eb3b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa300003aa0 tx=0x7fa30002ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=641771ceee05401a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:58.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.011+0000 7fa2fa000640 10 client.26680 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:14:58.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.012+0000 7fa2fa000640 1 -- 172.21.15.69:0/2774682851 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3000da070 con 0x557131ead740 2024-03-19T01:14:58.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.012+0000 7fa2fa000640 1 -- 172.21.15.69:0/2774682851 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa3000c5480 con 0x557131ead740 2024-03-19T01:14:58.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.013+0000 7fa2fa000640 1 -- 172.21.15.69:0/2774682851 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa30004b030 con 0x557131ead740 2024-03-19T01:14:58.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.014+0000 7fa319ddd740 10 client.26680 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:58.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.014+0000 7fa319ddd740 1 -- 172.21.15.69:0/2774682851 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=36}) -- 0x557131add0f0 con 0x557131ead740 2024-03-19T01:14:58.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.014+0000 7fa2fa000640 1 -- 172.21.15.69:0/2774682851 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7fa3000c0070 con 0x557131ead740 2024-03-19T01:14:58.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.014+0000 7fa319ddd740 10 client.26680 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:58.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.014+0000 7fa319ddd740 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:14:58.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.015+0000 7fa319ddd740 20 client.26680 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:58.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.015+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2774682851 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557131ec68d0 0x557131ec8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.015+0000 7fa319ddd740 4 client.26680 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:14:58.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.015+0000 7fa319ddd740 1 -- 172.21.15.69:0/2774682851 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557131daa610 con 0x557131ec68d0 2024-03-19T01:14:58.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.015+0000 7fa30f400640 1 --2- 172.21.15.69:0/2774682851 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557131ec68d0 0x557131ec8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.016+0000 7fa30f400640 1 --2- 172.21.15.69:0/2774682851 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557131ec68d0 0x557131ec8d20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f0c9dd26e68b7d9 server_cookie=dae3246e015042cf in_seq=0 out_seq=0 2024-03-19T01:14:58.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.016+0000 7fa2fa000640 10 client.26680 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:58.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.017+0000 7fa2fa000640 1 -- 172.21.15.69:0/2774682851 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557131daa610 con 0x557131ec68d0 2024-03-19T01:14:58.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.017+0000 7fa2fa000640 10 client.26680 handle_command_reply: tid=0 2024-03-19T01:14:58.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.017+0000 7fa319ddd740 1 client.26680 shutdown 2024-03-19T01:14:58.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.018+0000 7fa319ddd740 1 -- 172.21.15.69:0/2774682851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 msgr2=0x557131eb3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:58.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.018+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2774682851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eb3b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa300003aa0 tx=0x7fa30002ce70 comp rx=0 tx=0).stop 2024-03-19T01:14:58.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.018+0000 7fa319ddd740 1 -- 172.21.15.69:0/2774682851 shutdown_connections 2024-03-19T01:14:58.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.018+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2774682851 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557131ec68d0 0x557131ec8d20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.018+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2774682851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 0x557131eb4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.018+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2774682851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 0x557131eb40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.018+0000 7fa2fa000640 0 client.26680 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:58.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.018+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2774682851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eb3b90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.018+0000 7fa319ddd740 1 -- 172.21.15.69:0/2774682851 >> 172.21.15.69:0/2774682851 conn(0x557131ea7e70 msgr2=0x557131ea9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:58.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.019+0000 7fa319ddd740 1 -- 172.21.15.69:0/2774682851 shutdown_connections 2024-03-19T01:14:58.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.019+0000 7fa319ddd740 1 -- 172.21.15.69:0/2774682851 wait complete. 2024-03-19T01:14:58.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.019+0000 7fa319ddd740 20 client.26680 trim_cache size 0 max 16384 2024-03-19T01:14:58.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.057+0000 7fa319ddd740 1 Processor -- start 2024-03-19T01:14:58.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.057+0000 7fa319ddd740 1 -- start start 2024-03-19T01:14:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.057+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eb62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.057+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 0x557131ec2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.057+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 0x557131e9e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.057+0000 7fa319ddd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f000ff80 con 0x557131eae060 2024-03-19T01:14:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa319ddd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f0011c40 con 0x557131eb6c40 2024-03-19T01:14:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa319ddd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f0010650 con 0x557131ead740 2024-03-19T01:14:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa315000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 0x557131e9e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa30fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eb62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa315000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 0x557131e9e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34458/0 (socket says 172.21.15.69:34458) 2024-03-19T01:14:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa315000640 1 -- 172.21.15.69:0/2807254696 learned_addr learned my addr 172.21.15.69:0/2807254696 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa30f400640 1 --2- 172.21.15.69:0/2807254696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 0x557131ec2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa30fe00640 1 -- 172.21.15.69:0/2807254696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 msgr2=0x557131e9e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa30fe00640 1 --2- 172.21.15.69:0/2807254696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 0x557131e9e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa30fe00640 1 -- 172.21.15.69:0/2807254696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 msgr2=0x557131ec2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa30fe00640 1 --2- 172.21.15.69:0/2807254696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 0x557131ec2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa30fe00640 1 -- 172.21.15.69:0/2807254696 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557131eaf790 con 0x557131ead740 2024-03-19T01:14:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.058+0000 7fa30f400640 1 --2- 172.21.15.69:0/2807254696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 0x557131ec2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:14:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.059+0000 7fa30fe00640 1 --2- 172.21.15.69:0/2807254696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eb62c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa30002c270 tx=0x7fa3000ef000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=460d62591933472e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.059+0000 7fa304c00640 1 -- 172.21.15.69:0/2807254696 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3000da040 con 0x557131ead740 2024-03-19T01:14:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.059+0000 7fa304c00640 1 -- 172.21.15.69:0/2807254696 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa300004c80 con 0x557131ead740 2024-03-19T01:14:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.059+0000 7fa304c00640 1 -- 172.21.15.69:0/2807254696 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3000ee110 con 0x557131ead740 2024-03-19T01:14:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.060+0000 7fa319ddd740 1 -- 172.21.15.69:0/2807254696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 msgr2=0x557131eb62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.060+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2807254696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eb62c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa30002c270 tx=0x7fa3000ef000 comp rx=0 tx=0).stop 2024-03-19T01:14:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.060+0000 7fa319ddd740 1 -- 172.21.15.69:0/2807254696 shutdown_connections 2024-03-19T01:14:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.060+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2807254696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131eb6c40 0x557131e9e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.060+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2807254696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eae060 0x557131ec2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.060+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2807254696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131ead740 0x557131eb62c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.060+0000 7fa319ddd740 1 -- 172.21.15.69:0/2807254696 >> 172.21.15.69:0/2807254696 conn(0x557131ea7e70 msgr2=0x557131e9de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.061+0000 7fa319ddd740 1 -- 172.21.15.69:0/2807254696 shutdown_connections 2024-03-19T01:14:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.061+0000 7fa319ddd740 1 -- 172.21.15.69:0/2807254696 wait complete. 2024-03-19T01:14:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.061+0000 7fa319ddd740 1 -- start start 2024-03-19T01:14:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.062+0000 7fa319ddd740 1 Processor -- start 2024-03-19T01:14:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131ead740 0x557131eb0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131eae060 0x557131eb0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa319ddd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 0x557131eb13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa319ddd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f00114b0 con 0x557131eb6c40 2024-03-19T01:14:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa319ddd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f0010d20 con 0x557131ead740 2024-03-19T01:14:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa319ddd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f000df60 con 0x557131eae060 2024-03-19T01:14:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa30fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131ead740 0x557131eb0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa319ddd740 10 client.0 _ll_register_callbacks cb 0x7fa300009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:14:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa30fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131ead740 0x557131eb0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34468/0 (socket says 172.21.15.69:34468) 2024-03-19T01:14:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa30fe00640 1 -- 172.21.15.69:0/2072138971 learned_addr learned my addr 172.21.15.69:0/2072138971 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:14:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa30f400640 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131eae060 0x557131eb0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa30fe00640 1 -- 172.21.15.69:0/2072138971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131eae060 msgr2=0x557131eb0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:14:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa30fe00640 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131eae060 0x557131eb0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa30fe00640 1 -- 172.21.15.69:0/2072138971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 msgr2=0x557131eb13c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.063+0000 7fa315000640 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 0x557131eb13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.064+0000 7fa30fe00640 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 0x557131eb13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.064+0000 7fa30fe00640 1 -- 172.21.15.69:0/2072138971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557131f2be20 con 0x557131ead740 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.064+0000 7fa315000640 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 0x557131eb13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.064+0000 7fa30fe00640 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131ead740 0x557131eb0940 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa300005820 tx=0x7fa3000b95d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31af02f76b1c384e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.064+0000 7fa2fa000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.064+0000 7fa2fa000640 1 -- 172.21.15.69:0/2072138971 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3000c5750 con 0x557131ead740 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.064+0000 7fa319ddd740 1 -- 172.21.15.69:0/2072138971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa300004c80 con 0x557131ead740 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.064+0000 7fa2fa000640 1 -- 172.21.15.69:0/2072138971 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa3000bc3b0 con 0x557131ead740 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.064+0000 7fa2fa000640 1 -- 172.21.15.69:0/2072138971 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa3000bc5b0 con 0x557131ead740 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.064+0000 7fa319ddd740 10 client.26717 fetch_fsmap learned FSMap version 36 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.064+0000 7fa319ddd740 1 -- 172.21.15.69:0/2072138971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557131f2be20 con 0x557131ead740 2024-03-19T01:14:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.065+0000 7fa2fa000640 1 -- 172.21.15.69:0/2072138971 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa300004c80 con 0x557131ead740 2024-03-19T01:14:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.065+0000 7fa2fa000640 1 -- 172.21.15.69:0/2072138971 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7fa3000c0040 con 0x557131ead740 2024-03-19T01:14:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.065+0000 7fa319ddd740 10 client.26717 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:14:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.065+0000 7fa319ddd740 10 client.26717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:14:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.065+0000 7fa319ddd740 20 client.26717 populate_metadata read hostname 'smithi069' 2024-03-19T01:14:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.065+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557131f2d4a0 0x557131f2f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:14:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.065+0000 7fa319ddd740 4 client.26717 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:14:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.065+0000 7fa319ddd740 1 -- 172.21.15.69:0/2072138971 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557131e16800 con 0x557131f2d4a0 2024-03-19T01:14:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.065+0000 7fa30f400640 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557131f2d4a0 0x557131f2f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:14:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa30f400640 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557131f2d4a0 0x557131f2f8c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b27bd512f5fd8c1 server_cookie=3aa09d2322d75633 in_seq=0 out_seq=0 2024-03-19T01:14:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa2fa000640 10 client.26717 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa2fa000640 1 -- 172.21.15.69:0/2072138971 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557131e16800 con 0x557131f2d4a0 2024-03-19T01:14:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa2fa000640 10 client.26717 handle_command_reply: tid=0 2024-03-19T01:14:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa319ddd740 1 client.26717 shutdown 2024-03-19T01:14:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa319ddd740 1 -- 172.21.15.69:0/2072138971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131ead740 msgr2=0x557131eb0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131ead740 0x557131eb0940 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa300005820 tx=0x7fa3000b95d0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa319ddd740 1 -- 172.21.15.69:0/2072138971 shutdown_connections 2024-03-19T01:14:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557131f2d4a0 0x557131f2f8c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557131eb6c40 0x557131eb13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557131eae060 0x557131eb0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa319ddd740 1 --2- 172.21.15.69:0/2072138971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557131ead740 0x557131eb0940 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa2fa000640 0 client.26717 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:14:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa319ddd740 1 -- 172.21.15.69:0/2072138971 >> 172.21.15.69:0/2072138971 conn(0x557131ea7e70 msgr2=0x557131eb2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.066+0000 7fa319ddd740 1 -- 172.21.15.69:0/2072138971 shutdown_connections 2024-03-19T01:14:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.067+0000 7fa319ddd740 1 -- 172.21.15.69:0/2072138971 wait complete. 2024-03-19T01:14:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.067+0000 7fa319ddd740 20 client.26717 trim_cache size 0 max 16384 2024-03-19T01:14:58.073 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:14:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T01:14:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:14:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "f2ea6c40-85ce-485a-b25a-468e07ee8e08": { 2024-03-19T01:14:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:14:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "f2ea6c40-85ce-485a-b25a-468e07ee8e08", 2024-03-19T01:14:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:14:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:14:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:14:58.073 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:14:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.067+0000 7fa315a00640 1 -- 172.21.15.69:0/1506028451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2f40a29f0 msgr2=0x7fa2f40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.067+0000 7fa315a00640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2f40a29f0 0x7fa2f40a4eb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa308009fd0 tx=0x7fa308004af0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.067+0000 7fa315a00640 1 -- 172.21.15.69:0/1506028451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa310100430 msgr2=0x7fa31019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:14:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.067+0000 7fa315a00640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa310100430 0x7fa31019a200 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa300005ab0 tx=0x7fa3000043c0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.068+0000 7fa315a00640 1 -- 172.21.15.69:0/1506028451 shutdown_connections 2024-03-19T01:14:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.068+0000 7fa315a00640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa31010dfe0 0x7fa31019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.068+0000 7fa315a00640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310100de0 0x7fa31019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.068+0000 7fa315a00640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2f40a29f0 0x7fa2f40a4eb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.068+0000 7fa315a00640 1 --2- 172.21.15.69:0/1506028451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa310100430 0x7fa31019a200 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:14:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.068+0000 7fa315a00640 1 -- 172.21.15.69:0/1506028451 >> 172.21.15.69:0/1506028451 conn(0x7fa3100fc070 msgr2=0x7fa31010f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:14:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.068+0000 7fa315a00640 1 -- 172.21.15.69:0/1506028451 shutdown_connections 2024-03-19T01:14:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:14:58.068+0000 7fa315a00640 1 -- 172.21.15.69:0/1506028451 wait complete. 2024-03-19T01:14:58.092 INFO:tasks.cephfs.filesystem:scrub status for tag:f2ea6c40-85ce-485a-b25a-468e07ee8e08 - {'path': '/', 'tag': 'f2ea6c40-85ce-485a-b25a-468e07ee8e08', 'options': 'recursive,force'} 2024-03-19T01:14:58.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:14:58 smithi069 ceph-mon[31727]: pgmap v131: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 167 op/s 2024-03-19T01:14:58.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:14:58 smithi163 ceph-mon[67428]: pgmap v131: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 167 op/s 2024-03-19T01:14:58.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:14:58 smithi086 ceph-mon[66829]: pgmap v131: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 167 op/s 2024-03-19T01:15:00.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:00 smithi069 ceph-mon[31727]: pgmap v132: 129 pgs: 129 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 307 op/s 2024-03-19T01:15:00.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:00 smithi163 ceph-mon[67428]: pgmap v132: 129 pgs: 129 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 307 op/s 2024-03-19T01:15:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:00 smithi086 ceph-mon[66829]: pgmap v132: 129 pgs: 129 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 307 op/s 2024-03-19T01:15:02.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:02 smithi069 ceph-mon[31727]: pgmap v133: 129 pgs: 129 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-03-19T01:15:02.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:02 smithi163 ceph-mon[67428]: pgmap v133: 129 pgs: 129 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-03-19T01:15:02.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:02 smithi086 ceph-mon[66829]: pgmap v133: 129 pgs: 129 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-03-19T01:15:04.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:04 smithi069 ceph-mon[31727]: pgmap v134: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 331 op/s 2024-03-19T01:15:04.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:04 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:15:04.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:04 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:15:04.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:04 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:15:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:04 smithi163 ceph-mon[67428]: pgmap v134: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 331 op/s 2024-03-19T01:15:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:04 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:15:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:04 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:15:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:04 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:15:04.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:04 smithi086 ceph-mon[66829]: pgmap v134: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 331 op/s 2024-03-19T01:15:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:04 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:15:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:04 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:15:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:04 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:15:05.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:05 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:15:05.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:05 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:15:05.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:05 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:15:05.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:05 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:15:05.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:05 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:15:05.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:05 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:15:06.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:06 smithi069 ceph-mon[31727]: pgmap v135: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 201 op/s 2024-03-19T01:15:06.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:06 smithi163 ceph-mon[67428]: pgmap v135: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 201 op/s 2024-03-19T01:15:06.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:06 smithi086 ceph-mon[66829]: pgmap v135: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 201 op/s 2024-03-19T01:15:08.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:08 smithi163 ceph-mon[67428]: pgmap v136: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 212 op/s 2024-03-19T01:15:08.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:08 smithi086 ceph-mon[66829]: pgmap v136: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 212 op/s 2024-03-19T01:15:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:08 smithi069 ceph-mon[31727]: pgmap v136: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 212 op/s 2024-03-19T01:15:10.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:10 smithi163 ceph-mon[67428]: pgmap v137: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 232 op/s 2024-03-19T01:15:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:10 smithi086 ceph-mon[66829]: pgmap v137: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 232 op/s 2024-03-19T01:15:10.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:10 smithi069 ceph-mon[31727]: pgmap v137: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 232 op/s 2024-03-19T01:15:12.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:12 smithi163 ceph-mon[67428]: pgmap v138: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 86 op/s 2024-03-19T01:15:12.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:12 smithi086 ceph-mon[66829]: pgmap v138: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 86 op/s 2024-03-19T01:15:12.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:12 smithi069 ceph-mon[31727]: pgmap v138: 129 pgs: 129 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 86 op/s 2024-03-19T01:15:14.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:14 smithi163 ceph-mon[67428]: pgmap v139: 129 pgs: 129 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.2 MiB/s wr, 269 op/s 2024-03-19T01:15:14.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:14 smithi086 ceph-mon[66829]: pgmap v139: 129 pgs: 129 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.2 MiB/s wr, 269 op/s 2024-03-19T01:15:14.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:14 smithi069 ceph-mon[31727]: pgmap v139: 129 pgs: 129 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.2 MiB/s wr, 269 op/s 2024-03-19T01:15:16.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:16 smithi163 ceph-mon[67428]: pgmap v140: 129 pgs: 129 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 279 op/s 2024-03-19T01:15:16.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:16 smithi086 ceph-mon[66829]: pgmap v140: 129 pgs: 129 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 279 op/s 2024-03-19T01:15:16.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:16 smithi069 ceph-mon[31727]: pgmap v140: 129 pgs: 129 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 279 op/s 2024-03-19T01:15:18.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:18 smithi163 ceph-mon[67428]: pgmap v141: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.3 MiB/s wr, 320 op/s 2024-03-19T01:15:18.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:18 smithi086 ceph-mon[66829]: pgmap v141: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.3 MiB/s wr, 320 op/s 2024-03-19T01:15:18.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:18 smithi069 ceph-mon[31727]: pgmap v141: 129 pgs: 129 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.3 MiB/s wr, 320 op/s 2024-03-19T01:15:20.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:20 smithi086 ceph-mon[66829]: pgmap v142: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 502 op/s 2024-03-19T01:15:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:20 smithi163 ceph-mon[67428]: pgmap v142: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 502 op/s 2024-03-19T01:15:20.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:20 smithi069 ceph-mon[31727]: pgmap v142: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 502 op/s 2024-03-19T01:15:22.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:22 smithi163 ceph-mon[67428]: pgmap v143: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 458 op/s 2024-03-19T01:15:22.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:22 smithi086 ceph-mon[66829]: pgmap v143: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 458 op/s 2024-03-19T01:15:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:22 smithi069 ceph-mon[31727]: pgmap v143: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 458 op/s 2024-03-19T01:15:24.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:24 smithi086 ceph-mon[66829]: pgmap v144: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 481 op/s 2024-03-19T01:15:24.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:24 smithi069 ceph-mon[31727]: pgmap v144: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 481 op/s 2024-03-19T01:15:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:24 smithi163 ceph-mon[67428]: pgmap v144: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 481 op/s 2024-03-19T01:15:26.334 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:26 smithi069 ceph-mon[31727]: pgmap v145: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 301 op/s 2024-03-19T01:15:26.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:26 smithi086 ceph-mon[66829]: pgmap v145: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 301 op/s 2024-03-19T01:15:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:26 smithi163 ceph-mon[67428]: pgmap v145: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 301 op/s 2024-03-19T01:15:28.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:15:28.093 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:15:28.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.454+0000 7f3eae400640 1 -- 172.21.15.69:0/2130430118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea8109320 msgr2=0x7f3ea8114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.454+0000 7f3eae400640 1 --2- 172.21.15.69:0/2130430118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea8109320 0x7f3ea8114430 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0009be0 tx=0x7f3ea002bef0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.458+0000 7f3eae400640 1 -- 172.21.15.69:0/2130430118 shutdown_connections 2024-03-19T01:15:28.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.458+0000 7f3eae400640 1 --2- 172.21.15.69:0/2130430118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ea8114970 0x7f3ea8116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.458+0000 7f3eae400640 1 --2- 172.21.15.69:0/2130430118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea8109320 0x7f3ea8114430 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.458+0000 7f3eae400640 1 --2- 172.21.15.69:0/2130430118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ea8108a00 0x7f3ea8108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.458+0000 7f3eae400640 1 -- 172.21.15.69:0/2130430118 >> 172.21.15.69:0/2130430118 conn(0x7f3ea806c900 msgr2=0x7f3ea806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:28.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.458+0000 7f3eae400640 1 -- 172.21.15.69:0/2130430118 shutdown_connections 2024-03-19T01:15:28.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.458+0000 7f3eae400640 1 -- 172.21.15.69:0/2130430118 wait complete. 2024-03-19T01:15:28.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.463+0000 7f3eae400640 1 Processor -- start 2024-03-19T01:15:28.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.463+0000 7f3eae400640 1 -- start start 2024-03-19T01:15:28.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.464+0000 7f3eae400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ea8108a00 0x7f3ea81a8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.465+0000 7f3eae400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea8109320 0x7f3ea81a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.465+0000 7f3eae400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ea8114970 0x7f3ea81a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.465+0000 7f3eae400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea811a3b0 con 0x7f3ea8108a00 2024-03-19T01:15:28.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.465+0000 7f3eae400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea811a230 con 0x7f3ea8109320 2024-03-19T01:15:28.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.465+0000 7f3eae400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea811a530 con 0x7f3ea8114970 2024-03-19T01:15:28.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.465+0000 7f3ea6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea8109320 0x7f3ea81a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.465+0000 7f3ea6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea8109320 0x7f3ea81a8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51112/0 (socket says 172.21.15.69:51112) 2024-03-19T01:15:28.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.465+0000 7f3ea6a00640 1 -- 172.21.15.69:0/2164234815 learned_addr learned my addr 172.21.15.69:0/2164234815 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:28.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.465+0000 7f3ea7e00640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ea8114970 0x7f3ea81a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.465+0000 7f3ea7400640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ea8108a00 0x7f3ea81a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.466+0000 7f3ea7e00640 1 -- 172.21.15.69:0/2164234815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea8109320 msgr2=0x7f3ea81a8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.466+0000 7f3ea7e00640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea8109320 0x7f3ea81a8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.466+0000 7f3ea7e00640 1 -- 172.21.15.69:0/2164234815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ea8108a00 msgr2=0x7f3ea81a8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.466+0000 7f3ea7e00640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ea8108a00 0x7f3ea81a8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.466+0000 7f3ea7e00640 1 -- 172.21.15.69:0/2164234815 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea81a3190 con 0x7f3ea8114970 2024-03-19T01:15:28.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.466+0000 7f3ea7400640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ea8108a00 0x7f3ea81a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:15:28.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.466+0000 7f3ea7e00640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ea8114970 0x7f3ea81a2a10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400e940 tx=0x7f3e9400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:28.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.467+0000 7f3e9fe00640 1 -- 172.21.15.69:0/2164234815 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9400ccc0 con 0x7f3ea8114970 2024-03-19T01:15:28.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.467+0000 7f3eae400640 1 -- 172.21.15.69:0/2164234815 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea81a3480 con 0x7f3ea8114970 2024-03-19T01:15:28.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.467+0000 7f3e9fe00640 1 -- 172.21.15.69:0/2164234815 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e94005020 con 0x7f3ea8114970 2024-03-19T01:15:28.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.467+0000 7f3e9fe00640 1 -- 172.21.15.69:0/2164234815 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9401c070 con 0x7f3ea8114970 2024-03-19T01:15:28.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.467+0000 7f3eae400640 1 -- 172.21.15.69:0/2164234815 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea81af6d0 con 0x7f3ea8114970 2024-03-19T01:15:28.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.474+0000 7f3eb1210740 1 Processor -- start 2024-03-19T01:15:28.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.477+0000 7f3eb1210740 1 -- start start 2024-03-19T01:15:28.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.477+0000 7f3e9fe00640 1 -- 172.21.15.69:0/2164234815 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f3e940051d0 con 0x7f3ea8114970 2024-03-19T01:15:28.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.478+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 0x563a4fe39120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.478+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe39660 0x563a4fe41d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.478+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe42240 0x563a4fe44630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.478+0000 7f3eb1210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563a4fd9d2c0 con 0x563a4fe39660 2024-03-19T01:15:28.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.478+0000 7f3eb1210740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a4fd45b60 con 0x563a4fe42240 2024-03-19T01:15:28.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.478+0000 7f3eb1210740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563a4fd99360 con 0x563a4fe38d40 2024-03-19T01:15:28.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.479+0000 7f3ea7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe39660 0x563a4fe41d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.479+0000 7f3ea7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe39660 0x563a4fe41d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41370/0 (socket says 172.21.15.69:41370) 2024-03-19T01:15:28.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.479+0000 7f3ea7400640 1 -- 172.21.15.69:0/3872498685 learned_addr learned my addr 172.21.15.69:0/3872498685 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:28.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.480+0000 7f3ea7400640 1 -- 172.21.15.69:0/3872498685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 msgr2=0x563a4fe39120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:15:28.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.480+0000 7f3ea7400640 1 --2- 172.21.15.69:0/3872498685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 0x563a4fe39120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.480+0000 7f3ea7400640 1 -- 172.21.15.69:0/3872498685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe42240 msgr2=0x563a4fe44630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:15:28.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.480+0000 7f3ea7400640 1 --2- 172.21.15.69:0/3872498685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe42240 0x563a4fe44630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.480+0000 7f3ea7400640 1 -- 172.21.15.69:0/3872498685 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a4fb7b7d0 con 0x563a4fe39660 2024-03-19T01:15:28.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.481+0000 7f3ea7400640 1 --2- 172.21.15.69:0/3872498685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe39660 0x563a4fe41d00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009ce0 tx=0x7f3e980117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04fa2e0d7614c25 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:28.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.482+0000 7f3e9fe00640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f3e8c0a29f0 0x7f3e8c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.482+0000 7f3e9cc00640 1 -- 172.21.15.69:0/3872498685 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98012450 con 0x563a4fe39660 2024-03-19T01:15:28.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.482+0000 7f3e9cc00640 1 -- 172.21.15.69:0/3872498685 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e98012ad0 con 0x563a4fe39660 2024-03-19T01:15:28.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.482+0000 7f3e9cc00640 1 -- 172.21.15.69:0/3872498685 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800fbc0 con 0x563a4fe39660 2024-03-19T01:15:28.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.483+0000 7f3ea6a00640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f3e8c0a29f0 0x7f3e8c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.483+0000 7f3eb1210740 1 -- 172.21.15.69:0/3872498685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe39660 msgr2=0x563a4fe41d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.483+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3872498685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe39660 0x563a4fe41d00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009ce0 tx=0x7f3e980117b0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.486+0000 7f3eb1210740 1 -- 172.21.15.69:0/3872498685 shutdown_connections 2024-03-19T01:15:28.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.486+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3872498685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe42240 0x563a4fe44630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.486+0000 7f3ea6a00640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f3e8c0a29f0 0x7f3e8c0a4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0000f80 tx=0x7f3ea00098a0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:28.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.486+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3872498685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe39660 0x563a4fe41d00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.486+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3872498685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 0x563a4fe39120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.486+0000 7f3eb1210740 1 -- 172.21.15.69:0/3872498685 >> 172.21.15.69:0/3872498685 conn(0x563a4fe33470 msgr2=0x563a4fe33850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:28.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.486+0000 7f3e9fe00640 1 -- 172.21.15.69:0/2164234815 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f3e940cf1e0 con 0x7f3ea8114970 2024-03-19T01:15:28.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.486+0000 7f3eb1210740 1 -- 172.21.15.69:0/3872498685 shutdown_connections 2024-03-19T01:15:28.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.487+0000 7f3eb1210740 1 -- 172.21.15.69:0/3872498685 wait complete. 2024-03-19T01:15:28.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.487+0000 7f3eb1210740 1 -- start start 2024-03-19T01:15:28.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.488+0000 7f3eb1210740 1 Processor -- start 2024-03-19T01:15:28.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.489+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 0x563a4fe3f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.489+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.489+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe42240 0x563a4fe3fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.489+0000 7f3eb1210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563a4fe45080 con 0x563a4fe42240 2024-03-19T01:15:28.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.489+0000 7f3eb1210740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a4fd58fc0 con 0x563a4fe39660 2024-03-19T01:15:28.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.489+0000 7f3eb1210740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563a4fe45200 con 0x563a4fe38d40 2024-03-19T01:15:28.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.489+0000 7f3eb1210740 10 client.0 _ll_register_callbacks cb 0x563a4fd58990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:28.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.490+0000 7f3ea6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.490+0000 7f3ea6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51122/0 (socket says 172.21.15.69:51122) 2024-03-19T01:15:28.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.490+0000 7f3ea6a00640 1 -- 172.21.15.69:0/3198853704 learned_addr learned my addr 172.21.15.69:0/3198853704 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:28.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.490+0000 7f3ea6a00640 1 -- 172.21.15.69:0/3198853704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 msgr2=0x563a4fe3f290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:15:28.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.490+0000 7f3ea6a00640 1 --2- 172.21.15.69:0/3198853704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 0x563a4fe3f290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.490+0000 7f3ea6a00640 1 -- 172.21.15.69:0/3198853704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe42240 msgr2=0x563a4fe3fd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:15:28.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.490+0000 7f3ea6a00640 1 --2- 172.21.15.69:0/3198853704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe42240 0x563a4fe3fd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.490+0000 7f3ea6a00640 1 -- 172.21.15.69:0/3198853704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a4fab13f0 con 0x563a4fe39660 2024-03-19T01:15:28.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.493+0000 7f3ea6a00640 1 --2- 172.21.15.69:0/3198853704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3f7d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0036040 tx=0x7f3ea0009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9afe9817156e67d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:28.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.493+0000 7f3e91600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:15:28.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.493+0000 7f3e91600640 1 -- 172.21.15.69:0/3198853704 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0045070 con 0x563a4fe39660 2024-03-19T01:15:28.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.494+0000 7f3e91600640 1 -- 172.21.15.69:0/3198853704 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f3ea002c890 con 0x563a4fe39660 2024-03-19T01:15:28.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.494+0000 7f3eb1210740 1 -- 172.21.15.69:0/3198853704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e98012ad0 con 0x563a4fe39660 2024-03-19T01:15:28.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.495+0000 7f3e91600640 1 -- 172.21.15.69:0/3198853704 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3ea002ca90 con 0x563a4fe39660 2024-03-19T01:15:28.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.495+0000 7f3eb1210740 10 client.26726 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:28.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.495+0000 7f3eb1210740 1 -- 172.21.15.69:0/3198853704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563a4fa710f0 con 0x563a4fe39660 2024-03-19T01:15:28.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.495+0000 7f3e91600640 1 -- 172.21.15.69:0/3198853704 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e98012ad0 con 0x563a4fe39660 2024-03-19T01:15:28.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.496+0000 7f3e91600640 1 -- 172.21.15.69:0/3198853704 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f3ea0003e70 con 0x563a4fe39660 2024-03-19T01:15:28.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.497+0000 7f3eb1210740 10 client.26726 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:28.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.498+0000 7f3eb1210740 10 client.26726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:15:28.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.498+0000 7f3eb1210740 20 client.26726 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:28.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.498+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3198853704 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563a4fe526f0 0x563a4fe54b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.498+0000 7f3eb1210740 4 client.26726 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:15:28.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.498+0000 7f3eb1210740 1 -- 172.21.15.69:0/3198853704 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a4fd3e610 con 0x563a4fe526f0 2024-03-19T01:15:28.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.498+0000 7f3ea7400640 1 --2- 172.21.15.69:0/3198853704 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563a4fe526f0 0x563a4fe54b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.500+0000 7f3ea7400640 1 --2- 172.21.15.69:0/3198853704 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563a4fe526f0 0x563a4fe54b80 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d851763bb7104fb9 server_cookie=d1dd0978c5c2166 in_seq=0 out_seq=0 2024-03-19T01:15:28.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.500+0000 7f3e91600640 10 client.26726 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.501+0000 7f3e91600640 1 -- 172.21.15.69:0/3198853704 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563a4fd3e610 con 0x563a4fe526f0 2024-03-19T01:15:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.501+0000 7f3e91600640 10 client.26726 handle_command_reply: tid=0 2024-03-19T01:15:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.501+0000 7f3eb1210740 1 client.26726 shutdown 2024-03-19T01:15:28.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.502+0000 7f3eb1210740 1 -- 172.21.15.69:0/3198853704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 msgr2=0x563a4fe3f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.502+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3198853704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3f7d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0036040 tx=0x7f3ea0009500 comp rx=0 tx=0).stop 2024-03-19T01:15:28.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.502+0000 7f3eb1210740 1 -- 172.21.15.69:0/3198853704 shutdown_connections 2024-03-19T01:15:28.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.502+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3198853704 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563a4fe526f0 0x563a4fe54b80 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.502+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3198853704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe42240 0x563a4fe3fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.502+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3198853704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3f7d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.502+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3198853704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 0x563a4fe3f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.502+0000 7f3eb1210740 1 -- 172.21.15.69:0/3198853704 >> 172.21.15.69:0/3198853704 conn(0x563a4fe33470 msgr2=0x563a4fe351c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.502+0000 7f3e91600640 0 client.26726 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.503+0000 7f3eb1210740 1 -- 172.21.15.69:0/3198853704 shutdown_connections 2024-03-19T01:15:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.503+0000 7f3eb1210740 1 -- 172.21.15.69:0/3198853704 wait complete. 2024-03-19T01:15:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.503+0000 7f3eb1210740 20 client.26726 trim_cache size 0 max 16384 2024-03-19T01:15:28.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:28 smithi086 ceph-mon[66829]: pgmap v146: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 266 op/s 2024-03-19T01:15:28.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:28 smithi069 ceph-mon[31727]: pgmap v146: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 266 op/s 2024-03-19T01:15:28.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.544+0000 7f3eb1210740 1 Processor -- start 2024-03-19T01:15:28.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.544+0000 7f3eb1210740 1 -- start start 2024-03-19T01:15:28.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.544+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe38d40 0x563a4fe419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe42240 0x563a4fe4de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3eb1210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81b95f0 con 0x563a4fe38d40 2024-03-19T01:15:28.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3eb1210740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81bb2b0 con 0x563a4fe39660 2024-03-19T01:15:28.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3eb1210740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81b9cc0 con 0x563a4fe42240 2024-03-19T01:15:28.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3ea6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3ea6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51138/0 (socket says 172.21.15.69:51138) 2024-03-19T01:15:28.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3ea7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe42240 0x563a4fe4de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3ea6a00640 1 -- 172.21.15.69:0/1333271392 learned_addr learned my addr 172.21.15.69:0/1333271392 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3ea6a00640 1 -- 172.21.15.69:0/1333271392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe42240 msgr2=0x563a4fe4de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3ea6a00640 1 --2- 172.21.15.69:0/1333271392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe42240 0x563a4fe4de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.545+0000 7f3ea6a00640 1 -- 172.21.15.69:0/1333271392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe38d40 msgr2=0x563a4fe419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.546+0000 7f3ea7400640 1 --2- 172.21.15.69:0/1333271392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe38d40 0x563a4fe419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.546+0000 7f3ea6a00640 1 --2- 172.21.15.69:0/1333271392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe38d40 0x563a4fe419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.546+0000 7f3ea6a00640 1 -- 172.21.15.69:0/1333271392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a4fe39490 con 0x563a4fe39660 2024-03-19T01:15:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.546+0000 7f3ea7e00640 1 --2- 172.21.15.69:0/1333271392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe42240 0x563a4fe4de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:15:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.546+0000 7f3ea7400640 1 --2- 172.21.15.69:0/1333271392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe38d40 0x563a4fe419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:15:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.546+0000 7f3ea6a00640 1 --2- 172.21.15.69:0/1333271392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3e3f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0004310 tx=0x7f3ea002c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4affd1369602b0c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.549+0000 7f3e93e00640 1 -- 172.21.15.69:0/1333271392 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0064070 con 0x563a4fe39660 2024-03-19T01:15:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.549+0000 7f3e93e00640 1 -- 172.21.15.69:0/1333271392 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea0051070 con 0x563a4fe39660 2024-03-19T01:15:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.549+0000 7f3e93e00640 1 -- 172.21.15.69:0/1333271392 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0045040 con 0x563a4fe39660 2024-03-19T01:15:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.550+0000 7f3eb1210740 1 -- 172.21.15.69:0/1333271392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 msgr2=0x563a4fe3e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.550+0000 7f3eb1210740 1 --2- 172.21.15.69:0/1333271392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3e3f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0004310 tx=0x7f3ea002c940 comp rx=0 tx=0).stop 2024-03-19T01:15:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.550+0000 7f3eb1210740 1 -- 172.21.15.69:0/1333271392 shutdown_connections 2024-03-19T01:15:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.550+0000 7f3eb1210740 1 --2- 172.21.15.69:0/1333271392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe42240 0x563a4fe4de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.550+0000 7f3eb1210740 1 --2- 172.21.15.69:0/1333271392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3e3f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.550+0000 7f3eb1210740 1 --2- 172.21.15.69:0/1333271392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe38d40 0x563a4fe419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.550+0000 7f3eb1210740 1 -- 172.21.15.69:0/1333271392 >> 172.21.15.69:0/1333271392 conn(0x563a4fe33470 msgr2=0x563a4fe348a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.550+0000 7f3eb1210740 1 -- 172.21.15.69:0/1333271392 shutdown_connections 2024-03-19T01:15:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.550+0000 7f3eb1210740 1 -- 172.21.15.69:0/1333271392 wait complete. 2024-03-19T01:15:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.551+0000 7f3eb1210740 1 -- start start 2024-03-19T01:15:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.551+0000 7f3eb1210740 1 Processor -- start 2024-03-19T01:15:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.552+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 0x563a4fe4a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.552+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.552+0000 7f3eb1210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe42240 0x563a4fe3c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.552+0000 7f3eb1210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81bab20 con 0x563a4fe42240 2024-03-19T01:15:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.552+0000 7f3eb1210740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81ba390 con 0x563a4fe39660 2024-03-19T01:15:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.552+0000 7f3eb1210740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea81b7680 con 0x563a4fe38d40 2024-03-19T01:15:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.552+0000 7f3eb1210740 10 client.0 _ll_register_callbacks cb 0x7f3ea0033820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:28.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.553+0000 7f3ea6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.553+0000 7f3ea7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 0x563a4fe4a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.553+0000 7f3ea6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51152/0 (socket says 172.21.15.69:51152) 2024-03-19T01:15:28.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.553+0000 7f3ea6a00640 1 -- 172.21.15.69:0/3132672876 learned_addr learned my addr 172.21.15.69:0/3132672876 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:28.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.553+0000 7f3ea7e00640 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe42240 0x563a4fe3c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.553+0000 7f3ea6a00640 1 -- 172.21.15.69:0/3132672876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 msgr2=0x563a4fe4a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:28.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.553+0000 7f3ea6a00640 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 0x563a4fe4a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.553+0000 7f3ea6a00640 1 -- 172.21.15.69:0/3132672876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe42240 msgr2=0x563a4fe3c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:28.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.553+0000 7f3ea6a00640 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe42240 0x563a4fe3c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.553+0000 7f3ea6a00640 1 -- 172.21.15.69:0/3132672876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a4fe3c820 con 0x563a4fe39660 2024-03-19T01:15:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.553+0000 7f3ea7e00640 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe42240 0x563a4fe3c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:15:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.555+0000 7f3ea6a00640 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3bae0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3ea002c500 tx=0x7f3ea003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5c48b105dfd1008 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.555+0000 7f3e91600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:15:28.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.556+0000 7f3e91600640 1 -- 172.21.15.69:0/3132672876 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0064070 con 0x563a4fe39660 2024-03-19T01:15:28.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.556+0000 7f3e91600640 1 -- 172.21.15.69:0/3132672876 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f3ea004c880 con 0x563a4fe39660 2024-03-19T01:15:28.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.556+0000 7f3eb1210740 1 -- 172.21.15.69:0/3132672876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea0051070 con 0x563a4fe39660 2024-03-19T01:15:28.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.556+0000 7f3e91600640 1 -- 172.21.15.69:0/3132672876 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3ea0045040 con 0x563a4fe39660 2024-03-19T01:15:28.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.556+0000 7f3eb1210740 10 client.26732 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:28.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.556+0000 7f3eb1210740 1 -- 172.21.15.69:0/3132672876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563a4fec1120 con 0x563a4fe39660 2024-03-19T01:15:28.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.557+0000 7f3e91600640 1 -- 172.21.15.69:0/3132672876 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea0051070 con 0x563a4fe39660 2024-03-19T01:15:28.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.557+0000 7f3e91600640 1 -- 172.21.15.69:0/3132672876 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f3ea0056450 con 0x563a4fe39660 2024-03-19T01:15:28.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.557+0000 7f3eb1210740 10 client.26732 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:28.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.557+0000 7f3eb1210740 10 client.26732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:15:28.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.557+0000 7f3eb1210740 20 client.26732 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:28.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.557+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563a4fec1680 0x563a4fec3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.557+0000 7f3eb1210740 4 client.26732 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:15:28.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.557+0000 7f3eb1210740 1 -- 172.21.15.69:0/3132672876 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a4fdaa800 con 0x563a4fec1680 2024-03-19T01:15:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:28 smithi163 ceph-mon[67428]: pgmap v146: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 266 op/s 2024-03-19T01:15:28.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.558+0000 7f3ea7400640 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563a4fec1680 0x563a4fec3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.560+0000 7f3ea7400640 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563a4fec1680 0x563a4fec3aa0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef3b242d6a9f852e server_cookie=fe7247cc87ca7f65 in_seq=0 out_seq=0 2024-03-19T01:15:28.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.560+0000 7f3e91600640 10 client.26732 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:28.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.561+0000 7f3e91600640 1 -- 172.21.15.69:0/3132672876 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563a4fdaa800 con 0x563a4fec1680 2024-03-19T01:15:28.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.561+0000 7f3e91600640 10 client.26732 handle_command_reply: tid=0 2024-03-19T01:15:28.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.562+0000 7f3eb1210740 1 client.26732 shutdown 2024-03-19T01:15:28.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.562+0000 7f3eb1210740 1 -- 172.21.15.69:0/3132672876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 msgr2=0x563a4fe3bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.562+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3bae0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3ea002c500 tx=0x7f3ea003f000 comp rx=0 tx=0).stop 2024-03-19T01:15:28.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.562+0000 7f3eb1210740 1 -- 172.21.15.69:0/3132672876 shutdown_connections 2024-03-19T01:15:28.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.562+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563a4fec1680 0x563a4fec3aa0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.562+0000 7f3e91600640 0 client.26732 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:28.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.562+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a4fe42240 0x563a4fe3c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.562+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a4fe39660 0x563a4fe3bae0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.562+0000 7f3eb1210740 1 --2- 172.21.15.69:0/3132672876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563a4fe38d40 0x563a4fe4a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.562+0000 7f3eb1210740 1 -- 172.21.15.69:0/3132672876 >> 172.21.15.69:0/3132672876 conn(0x563a4fe33470 msgr2=0x563a4fe31e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:28.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.563+0000 7f3eb1210740 1 -- 172.21.15.69:0/3132672876 shutdown_connections 2024-03-19T01:15:28.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.563+0000 7f3eb1210740 1 -- 172.21.15.69:0/3132672876 wait complete. 2024-03-19T01:15:28.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.563+0000 7f3eb1210740 20 client.26732 trim_cache size 0 max 16384 2024-03-19T01:15:28.565 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:15:28.565 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:15:28.565 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:15:28.565 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:15:28.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.566+0000 7f3eae400640 1 -- 172.21.15.69:0/2164234815 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f3e8c0a29f0 msgr2=0x7f3e8c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.566+0000 7f3eae400640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f3e8c0a29f0 0x7f3e8c0a4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0000f80 tx=0x7f3ea00098a0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.566+0000 7f3eae400640 1 -- 172.21.15.69:0/2164234815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ea8114970 msgr2=0x7f3ea81a2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.566+0000 7f3eae400640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ea8114970 0x7f3ea81a2a10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400e940 tx=0x7f3e9400ee00 comp rx=0 tx=0).stop 2024-03-19T01:15:28.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.567+0000 7f3eae400640 1 -- 172.21.15.69:0/2164234815 shutdown_connections 2024-03-19T01:15:28.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.567+0000 7f3eae400640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ea8114970 0x7f3ea81a2a10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.567+0000 7f3eae400640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f3e8c0a29f0 0x7f3e8c0a4eb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.567+0000 7f3eae400640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ea8109320 0x7f3ea81a8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.567+0000 7f3eae400640 1 --2- 172.21.15.69:0/2164234815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ea8108a00 0x7f3ea81a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.567+0000 7f3eae400640 1 -- 172.21.15.69:0/2164234815 >> 172.21.15.69:0/2164234815 conn(0x7f3ea806c900 msgr2=0x7f3ea810a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:28.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.567+0000 7f3eae400640 1 -- 172.21.15.69:0/2164234815 shutdown_connections 2024-03-19T01:15:28.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.568+0000 7f3eae400640 1 -- 172.21.15.69:0/2164234815 wait complete. 2024-03-19T01:15:28.598 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:15:28.599 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:15:28.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.936+0000 7f8933400640 1 -- 172.21.15.69:0/1495290949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f892c074510 msgr2=0x7f892c10db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.936+0000 7f8933400640 1 --2- 172.21.15.69:0/1495290949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f892c074510 0x7f892c10db10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f891c009860 tx=0x7f891c02bc10 comp rx=0 tx=0).stop 2024-03-19T01:15:28.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.937+0000 7f8933400640 1 -- 172.21.15.69:0/1495290949 shutdown_connections 2024-03-19T01:15:28.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.937+0000 7f8933400640 1 --2- 172.21.15.69:0/1495290949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f892c10e050 0x7f892c110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.937+0000 7f8933400640 1 --2- 172.21.15.69:0/1495290949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f892c074510 0x7f892c10db10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.937+0000 7f8933400640 1 --2- 172.21.15.69:0/1495290949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c073bf0 0x7f892c073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.937+0000 7f8933400640 1 -- 172.21.15.69:0/1495290949 >> 172.21.15.69:0/1495290949 conn(0x7f892c0fd2c0 msgr2=0x7f892c0ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:28.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.937+0000 7f8933400640 1 -- 172.21.15.69:0/1495290949 shutdown_connections 2024-03-19T01:15:28.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.937+0000 7f8933400640 1 -- 172.21.15.69:0/1495290949 wait complete. 2024-03-19T01:15:28.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.939+0000 7f8933400640 1 Processor -- start 2024-03-19T01:15:28.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.941+0000 7f8933400640 1 -- start start 2024-03-19T01:15:28.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.942+0000 7f8933400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f892c073bf0 0x7f892c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.942+0000 7f8930a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f892c073bf0 0x7f892c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.942+0000 7f8933400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f892c074510 0x7f892c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.942+0000 7f8930a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f892c073bf0 0x7f892c19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:49832/0 (socket says 172.21.15.69:49832) 2024-03-19T01:15:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.942+0000 7f8930a00640 1 -- 172.21.15.69:0/2503345755 learned_addr learned my addr 172.21.15.69:0/2503345755 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.942+0000 7f8933400640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c10e050 0x7f892c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.943+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f892c113db0 con 0x7f892c074510 2024-03-19T01:15:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.943+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f892c113c30 con 0x7f892c10e050 2024-03-19T01:15:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.943+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f892c113f30 con 0x7f892c073bf0 2024-03-19T01:15:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.943+0000 7f8931400640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c10e050 0x7f892c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.943+0000 7f8931400640 1 -- 172.21.15.69:0/2503345755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f892c073bf0 msgr2=0x7f892c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.943+0000 7f8931400640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f892c073bf0 0x7f892c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.943+0000 7f8931400640 1 -- 172.21.15.69:0/2503345755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f892c074510 msgr2=0x7f892c19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.943+0000 7f8931400640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f892c074510 0x7f892c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.943+0000 7f8931400640 1 -- 172.21.15.69:0/2503345755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c19f300 con 0x7f892c10e050 2024-03-19T01:15:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.945+0000 7f8931400640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c10e050 0x7f892c19ebd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f892c074b40 tx=0x7f891800edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.945+0000 7f8929600640 1 -- 172.21.15.69:0/2503345755 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891800ccc0 con 0x7f892c10e050 2024-03-19T01:15:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.945+0000 7f8929600640 1 -- 172.21.15.69:0/2503345755 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8918005500 con 0x7f892c10e050 2024-03-19T01:15:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.945+0000 7f8929600640 1 -- 172.21.15.69:0/2503345755 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8918010430 con 0x7f892c10e050 2024-03-19T01:15:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.945+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c19f5f0 con 0x7f892c10e050 2024-03-19T01:15:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.946+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c1a6e80 con 0x7f892c10e050 2024-03-19T01:15:28.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.947+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f892c068a60 con 0x7f892c10e050 2024-03-19T01:15:28.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.954+0000 7f8929600640 1 -- 172.21.15.69:0/2503345755 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f8918005020 con 0x7f892c10e050 2024-03-19T01:15:28.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.956+0000 7f8929600640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f89100a2b40 0x7f89100a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:28.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.956+0000 7f8929600640 1 -- 172.21.15.69:0/2503345755 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f89180cf460 con 0x7f892c10e050 2024-03-19T01:15:28.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.957+0000 7f8930a00640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f89100a2b40 0x7f89100a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:28.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.959+0000 7f8930a00640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f89100a2b40 0x7f89100a5000 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8924009fd0 tx=0x7f8924010040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:28.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:28.960+0000 7f8929600640 1 -- 172.21.15.69:0/2503345755 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f891809be20 con 0x7f892c10e050 2024-03-19T01:15:29.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.412+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f892c19b660 con 0x7f892c10e050 2024-03-19T01:15:29.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.413+0000 7f8929600640 1 -- 172.21.15.69:0/2503345755 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11951 (secure 0 0 0) 0x7f891809b720 con 0x7f892c10e050 2024-03-19T01:15:29.414 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:15:29.414 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:14:05.695054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581,24476,24580,24494,24550]},"id":1}]} 2024-03-19T01:15:29.414 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 36 2024-03-19T01:15:29.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.415+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f89100a2b40 msgr2=0x7f89100a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.415+0000 7f8933400640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f89100a2b40 0x7f89100a5000 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8924009fd0 tx=0x7f8924010040 comp rx=0 tx=0).stop 2024-03-19T01:15:29.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.416+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c10e050 msgr2=0x7f892c19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.416+0000 7f8933400640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c10e050 0x7f892c19ebd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f892c074b40 tx=0x7f891800edd0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.416+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 shutdown_connections 2024-03-19T01:15:29.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.416+0000 7f8933400640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f89100a2b40 0x7f89100a5000 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.416+0000 7f8933400640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c10e050 0x7f892c19ebd0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.416+0000 7f8933400640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f892c074510 0x7f892c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.416+0000 7f8933400640 1 --2- 172.21.15.69:0/2503345755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f892c073bf0 0x7f892c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.416+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 >> 172.21.15.69:0/2503345755 conn(0x7f892c0fd2c0 msgr2=0x7f892c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:29.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.416+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 shutdown_connections 2024-03-19T01:15:29.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.416+0000 7f8933400640 1 -- 172.21.15.69:0/2503345755 wait complete. 2024-03-19T01:15:29.435 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:15:29.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.660+0000 7ff312800640 1 -- 172.21.15.69:0/4287908427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102850 msgr2=0x7ff30c102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.660+0000 7ff312800640 1 --2- 172.21.15.69:0/4287908427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102850 0x7ff30c102c30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff300009900 tx=0x7ff30002bcd0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.661+0000 7ff312800640 1 -- 172.21.15.69:0/4287908427 shutdown_connections 2024-03-19T01:15:29.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.661+0000 7ff312800640 1 --2- 172.21.15.69:0/4287908427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c10bc70 0x7ff30c10e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.661+0000 7ff312800640 1 --2- 172.21.15.69:0/4287908427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff30c103170 0x7ff30c10b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.661+0000 7ff312800640 1 --2- 172.21.15.69:0/4287908427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102850 0x7ff30c102c30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.661+0000 7ff312800640 1 -- 172.21.15.69:0/4287908427 >> 172.21.15.69:0/4287908427 conn(0x7ff30c0fc070 msgr2=0x7ff30c0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:29.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.661+0000 7ff312800640 1 -- 172.21.15.69:0/4287908427 shutdown_connections 2024-03-19T01:15:29.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.661+0000 7ff312800640 1 -- 172.21.15.69:0/4287908427 wait complete. 2024-03-19T01:15:29.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.662+0000 7ff312800640 1 Processor -- start 2024-03-19T01:15:29.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.662+0000 7ff312800640 1 -- start start 2024-03-19T01:15:29.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff312800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102850 0x7ff30c1a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff312800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c103170 0x7ff30c1a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff312800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff30c10bc70 0x7ff30c10a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff312800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c111a70 con 0x7ff30c103170 2024-03-19T01:15:29.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff312800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1118f0 con 0x7ff30c102850 2024-03-19T01:15:29.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff312800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c111bf0 con 0x7ff30c10bc70 2024-03-19T01:15:29.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff311400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102850 0x7ff30c1a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff311400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102850 0x7ff30c1a31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51182/0 (socket says 172.21.15.69:51182) 2024-03-19T01:15:29.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff311400640 1 -- 172.21.15.69:0/319714870 learned_addr learned my addr 172.21.15.69:0/319714870 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:29.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff311e00640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff30c10bc70 0x7ff30c10a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff311400640 1 -- 172.21.15.69:0/319714870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff30c10bc70 msgr2=0x7ff30c10a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff311400640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff30c10bc70 0x7ff30c10a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff311400640 1 -- 172.21.15.69:0/319714870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c103170 msgr2=0x7ff30c1a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff311400640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c103170 0x7ff30c1a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.663+0000 7ff311400640 1 -- 172.21.15.69:0/319714870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c10b090 con 0x7ff30c102850 2024-03-19T01:15:29.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.664+0000 7ff311400640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102850 0x7ff30c1a31c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff300005940 tx=0x7ff30002cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:29.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.664+0000 7ff30a000640 1 -- 172.21.15.69:0/319714870 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300041070 con 0x7ff30c102850 2024-03-19T01:15:29.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.664+0000 7ff30a000640 1 -- 172.21.15.69:0/319714870 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff300004450 con 0x7ff30c102850 2024-03-19T01:15:29.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.664+0000 7ff312800640 1 -- 172.21.15.69:0/319714870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c10b320 con 0x7ff30c102850 2024-03-19T01:15:29.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.664+0000 7ff30a000640 1 -- 172.21.15.69:0/319714870 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300003d80 con 0x7ff30c102850 2024-03-19T01:15:29.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.664+0000 7ff312800640 1 -- 172.21.15.69:0/319714870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c1ab0e0 con 0x7ff30c102850 2024-03-19T01:15:29.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.668+0000 7ff30a000640 1 -- 172.21.15.69:0/319714870 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ff300044070 con 0x7ff30c102850 2024-03-19T01:15:29.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.668+0000 7ff316b86740 1 Processor -- start 2024-03-19T01:15:29.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.669+0000 7ff316b86740 1 -- start start 2024-03-19T01:15:29.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.669+0000 7ff316b86740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 0x56448d366fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.669+0000 7ff316b86740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 0x56448d36fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.669+0000 7ff316b86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d3724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.669+0000 7ff316b86740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56448d2cb380 con 0x56448d367510 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.669+0000 7ff316b86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56448d273d20 con 0x56448d3700f0 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.669+0000 7ff316b86740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56448d2c7420 con 0x56448d366bf0 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.669+0000 7ff310a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d3724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.669+0000 7ff310a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d3724e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51186/0 (socket says 172.21.15.69:51186) 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.669+0000 7ff310a00640 1 -- 172.21.15.69:0/840419586 learned_addr learned my addr 172.21.15.69:0/840419586 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.669+0000 7ff311400640 1 --2- 172.21.15.69:0/840419586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 0x56448d36fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff311e00640 1 --2- 172.21.15.69:0/840419586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 0x56448d366fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff30a000640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2f40a29f0 0x7ff2f40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff310a00640 1 -- 172.21.15.69:0/840419586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 msgr2=0x56448d366fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff310a00640 1 --2- 172.21.15.69:0/840419586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 0x56448d366fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff310a00640 1 -- 172.21.15.69:0/840419586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 msgr2=0x56448d36fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff310a00640 1 --2- 172.21.15.69:0/840419586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 0x56448d36fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff310a00640 1 -- 172.21.15.69:0/840419586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56448cfdf3f0 con 0x56448d3700f0 2024-03-19T01:15:29.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff311400640 1 --2- 172.21.15.69:0/840419586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 0x56448d36fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:15:29.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff30a000640 1 -- 172.21.15.69:0/319714870 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7ff3000f0d30 con 0x7ff30c102850 2024-03-19T01:15:29.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff310a00640 1 --2- 172.21.15.69:0/840419586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d3724e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009ac0 tx=0x7ff2f8009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a3fbabd92fc4609 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:29.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff311e00640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2f40a29f0 0x7ff2f40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff2fe600640 1 -- 172.21.15.69:0/840419586 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f800ed10 con 0x56448d3700f0 2024-03-19T01:15:29.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.670+0000 7ff2fe600640 1 -- 172.21.15.69:0/840419586 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f8005500 con 0x56448d3700f0 2024-03-19T01:15:29.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.671+0000 7ff2fe600640 1 -- 172.21.15.69:0/840419586 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8010690 con 0x56448d3700f0 2024-03-19T01:15:29.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.671+0000 7ff316b86740 1 -- 172.21.15.69:0/840419586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 msgr2=0x56448d3724e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.671+0000 7ff311e00640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2f40a29f0 0x7ff2f40a4eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff3040094d0 tx=0x7ff304004a40 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:29.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.671+0000 7ff316b86740 1 --2- 172.21.15.69:0/840419586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d3724e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009ac0 tx=0x7ff2f8009f80 comp rx=0 tx=0).stop 2024-03-19T01:15:29.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.671+0000 7ff316b86740 1 -- 172.21.15.69:0/840419586 shutdown_connections 2024-03-19T01:15:29.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.671+0000 7ff316b86740 1 --2- 172.21.15.69:0/840419586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d3724e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.671+0000 7ff316b86740 1 --2- 172.21.15.69:0/840419586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 0x56448d36fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.671+0000 7ff316b86740 1 --2- 172.21.15.69:0/840419586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 0x56448d366fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.671+0000 7ff316b86740 1 -- 172.21.15.69:0/840419586 >> 172.21.15.69:0/840419586 conn(0x56448d361320 msgr2=0x56448d361700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:29.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.672+0000 7ff316b86740 1 -- 172.21.15.69:0/840419586 shutdown_connections 2024-03-19T01:15:29.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.672+0000 7ff316b86740 1 -- 172.21.15.69:0/840419586 wait complete. 2024-03-19T01:15:29.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.672+0000 7ff316b86740 1 -- start start 2024-03-19T01:15:29.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.672+0000 7ff316b86740 1 Processor -- start 2024-03-19T01:15:29.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff316b86740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d366bf0 0x56448d36d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff316b86740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d367510 0x56448d36d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff316b86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d36db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff316b86740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56448d372ee0 con 0x56448d366bf0 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff316b86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56448d287160 con 0x56448d3700f0 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff316b86740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56448d373060 con 0x56448d367510 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff316b86740 10 client.0 _ll_register_callbacks cb 0x56448d2c3170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff311e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d36db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff311e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d36db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51194/0 (socket says 172.21.15.69:51194) 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff311e00640 1 -- 172.21.15.69:0/2798044746 learned_addr learned my addr 172.21.15.69:0/2798044746 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff311400640 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d366bf0 0x56448d36d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff311e00640 1 -- 172.21.15.69:0/2798044746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d367510 msgr2=0x56448d36d650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff311e00640 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d367510 0x56448d36d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff311e00640 1 -- 172.21.15.69:0/2798044746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d366bf0 msgr2=0x56448d36d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff311e00640 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d366bf0 0x56448d36d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.673+0000 7ff311e00640 1 -- 172.21.15.69:0/2798044746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56448d0a97d0 con 0x56448d3700f0 2024-03-19T01:15:29.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.674+0000 7ff311400640 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d366bf0 0x56448d36d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.674+0000 7ff311e00640 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d36db90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff304011c80 tx=0x7ff304004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc4906df0ba38be8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.674+0000 7ff2f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.674+0000 7ff2f2a00640 1 -- 172.21.15.69:0/2798044746 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304027640 con 0x56448d3700f0 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.674+0000 7ff316b86740 1 -- 172.21.15.69:0/2798044746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f8005500 con 0x56448d3700f0 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.674+0000 7ff2f2a00640 1 -- 172.21.15.69:0/2798044746 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7ff30401b410 con 0x56448d3700f0 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.674+0000 7ff2f2a00640 1 -- 172.21.15.69:0/2798044746 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff3040277e0 con 0x56448d3700f0 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.674+0000 7ff316b86740 10 client.26750 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.674+0000 7ff316b86740 1 -- 172.21.15.69:0/2798044746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56448d37f240 con 0x56448d3700f0 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.675+0000 7ff2f2a00640 1 -- 172.21.15.69:0/2798044746 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f8005500 con 0x56448d3700f0 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.675+0000 7ff2f2a00640 1 -- 172.21.15.69:0/2798044746 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7ff304025070 con 0x56448d3700f0 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.675+0000 7ff316b86740 10 client.26750 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.675+0000 7ff316b86740 10 client.26750 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.675+0000 7ff316b86740 20 client.26750 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:29.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.675+0000 7ff316b86740 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56448d3806f0 0x56448d382b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.675+0000 7ff316b86740 4 client.26750 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:15:29.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.675+0000 7ff316b86740 1 -- 172.21.15.69:0/2798044746 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56448d2d88c0 con 0x56448d3806f0 2024-03-19T01:15:29.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.675+0000 7ff311400640 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56448d3806f0 0x56448d382b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.676+0000 7ff311400640 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56448d3806f0 0x56448d382b90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb3d99eac594de3f server_cookie=d7aacb02bd44a5a in_seq=0 out_seq=0 2024-03-19T01:15:29.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.676+0000 7ff2f2a00640 10 client.26750 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:15:29.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.679+0000 7ff2f2a00640 1 -- 172.21.15.69:0/2798044746 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56448d2d88c0 con 0x56448d3806f0 2024-03-19T01:15:29.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.679+0000 7ff2f2a00640 10 client.26750 handle_command_reply: tid=0 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.679+0000 7ff316b86740 1 client.26750 shutdown 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.679+0000 7ff316b86740 1 -- 172.21.15.69:0/2798044746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 msgr2=0x56448d36db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.679+0000 7ff316b86740 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d36db90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff304011c80 tx=0x7ff304004ab0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.680+0000 7ff316b86740 1 -- 172.21.15.69:0/2798044746 shutdown_connections 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.680+0000 7ff316b86740 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56448d3806f0 0x56448d382b90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.680+0000 7ff316b86740 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d3700f0 0x56448d36db90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.680+0000 7ff316b86740 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d367510 0x56448d36d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.680+0000 7ff316b86740 1 --2- 172.21.15.69:0/2798044746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d366bf0 0x56448d36d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.680+0000 7ff316b86740 1 -- 172.21.15.69:0/2798044746 >> 172.21.15.69:0/2798044746 conn(0x56448d361320 msgr2=0x56448d363070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.680+0000 7ff2f2a00640 0 client.26750 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.680+0000 7ff316b86740 1 -- 172.21.15.69:0/2798044746 shutdown_connections 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.680+0000 7ff316b86740 1 -- 172.21.15.69:0/2798044746 wait complete. 2024-03-19T01:15:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.680+0000 7ff316b86740 20 client.26750 trim_cache size 0 max 16384 2024-03-19T01:15:29.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.697+0000 7ff316b86740 1 Processor -- start 2024-03-19T01:15:29.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.697+0000 7ff316b86740 1 -- start start 2024-03-19T01:15:29.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.697+0000 7ff316b86740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 0x56448d36f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.697+0000 7ff316b86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d367510 0x56448d37bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.697+0000 7ff316b86740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d3700f0 0x56448d36c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff316b86740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b50c0 con 0x56448d3700f0 2024-03-19T01:15:29.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff316b86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b6d80 con 0x56448d367510 2024-03-19T01:15:29.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff316b86740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b5790 con 0x56448d366bf0 2024-03-19T01:15:29.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff311400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 0x56448d36f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff311e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d3700f0 0x56448d36c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff311400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 0x56448d36f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:49864/0 (socket says 172.21.15.69:49864) 2024-03-19T01:15:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff311400640 1 -- 172.21.15.69:0/1086178699 learned_addr learned my addr 172.21.15.69:0/1086178699 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff310a00640 1 --2- 172.21.15.69:0/1086178699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d367510 0x56448d37bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff311400640 1 -- 172.21.15.69:0/1086178699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d367510 msgr2=0x56448d37bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff311400640 1 --2- 172.21.15.69:0/1086178699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d367510 0x56448d37bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff311400640 1 -- 172.21.15.69:0/1086178699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d3700f0 msgr2=0x56448d36c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff311400640 1 --2- 172.21.15.69:0/1086178699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d3700f0 0x56448d36c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff311400640 1 -- 172.21.15.69:0/1086178699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56448d3724e0 con 0x56448d366bf0 2024-03-19T01:15:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff310a00640 1 --2- 172.21.15.69:0/1086178699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d367510 0x56448d37bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:15:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.698+0000 7ff311e00640 1 --2- 172.21.15.69:0/1086178699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d3700f0 0x56448d36c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:15:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.699+0000 7ff311400640 1 --2- 172.21.15.69:0/1086178699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 0x56448d36f840 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff3000bcd80 tx=0x7ff3000f3180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43beb69fac95b4b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.699+0000 7ff2fdc00640 1 -- 172.21.15.69:0/1086178699 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30004b040 con 0x56448d366bf0 2024-03-19T01:15:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.699+0000 7ff2fdc00640 1 -- 172.21.15.69:0/1086178699 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff300004c80 con 0x56448d366bf0 2024-03-19T01:15:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.699+0000 7ff2fdc00640 1 -- 172.21.15.69:0/1086178699 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3000c5820 con 0x56448d366bf0 2024-03-19T01:15:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.700+0000 7ff316b86740 1 -- 172.21.15.69:0/1086178699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 msgr2=0x56448d36f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.700+0000 7ff316b86740 1 --2- 172.21.15.69:0/1086178699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 0x56448d36f840 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff3000bcd80 tx=0x7ff3000f3180 comp rx=0 tx=0).stop 2024-03-19T01:15:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.700+0000 7ff316b86740 1 -- 172.21.15.69:0/1086178699 shutdown_connections 2024-03-19T01:15:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.700+0000 7ff316b86740 1 --2- 172.21.15.69:0/1086178699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d3700f0 0x56448d36c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.700+0000 7ff316b86740 1 --2- 172.21.15.69:0/1086178699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d367510 0x56448d37bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.700+0000 7ff316b86740 1 --2- 172.21.15.69:0/1086178699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d366bf0 0x56448d36f840 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.700+0000 7ff316b86740 1 -- 172.21.15.69:0/1086178699 >> 172.21.15.69:0/1086178699 conn(0x56448d361320 msgr2=0x56448d362750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.700+0000 7ff316b86740 1 -- 172.21.15.69:0/1086178699 shutdown_connections 2024-03-19T01:15:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.701+0000 7ff316b86740 1 -- 172.21.15.69:0/1086178699 wait complete. 2024-03-19T01:15:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.701+0000 7ff316b86740 1 -- start start 2024-03-19T01:15:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.702+0000 7ff316b86740 1 Processor -- start 2024-03-19T01:15:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.702+0000 7ff316b86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d366bf0 0x56448d378280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.702+0000 7ff316b86740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 0x56448d3736b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.702+0000 7ff316b86740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d3700f0 0x56448d373bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.702+0000 7ff316b86740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b65f0 con 0x56448d367510 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.702+0000 7ff316b86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b5e60 con 0x56448d366bf0 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.702+0000 7ff316b86740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b30a0 con 0x56448d3700f0 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.702+0000 7ff316b86740 10 client.0 _ll_register_callbacks cb 0x7ff304009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff311400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d366bf0 0x56448d378280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff311400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d366bf0 0x56448d378280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51202/0 (socket says 172.21.15.69:51202) 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff311400640 1 -- 172.21.15.69:0/3888374619 learned_addr learned my addr 172.21.15.69:0/3888374619 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff310a00640 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 0x56448d3736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff311e00640 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d3700f0 0x56448d373bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff311400640 1 -- 172.21.15.69:0/3888374619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d3700f0 msgr2=0x56448d373bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff311400640 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d3700f0 0x56448d373bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff311400640 1 -- 172.21.15.69:0/3888374619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 msgr2=0x56448d3736b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff311400640 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 0x56448d3736b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff311400640 1 -- 172.21.15.69:0/3888374619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56448d374130 con 0x56448d366bf0 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff310a00640 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 0x56448d3736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff311400640 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d366bf0 0x56448d378280 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff30002cfd0 tx=0x7ff3000c12b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7919bceaad1ea165 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff2f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff2f2a00640 1 -- 172.21.15.69:0/3888374619 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3000d5070 con 0x56448d366bf0 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff2f2a00640 1 -- 172.21.15.69:0/3888374619 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7ff3000b96f0 con 0x56448d366bf0 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.703+0000 7ff316b86740 1 -- 172.21.15.69:0/3888374619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff300004c80 con 0x56448d366bf0 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff2f2a00640 1 -- 172.21.15.69:0/3888374619 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff30004b040 con 0x56448d366bf0 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff316b86740 10 client.26756 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff316b86740 1 -- 172.21.15.69:0/3888374619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56448d3e6800 con 0x56448d366bf0 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff2f2a00640 1 -- 172.21.15.69:0/3888374619 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff300004c80 con 0x56448d366bf0 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff2f2a00640 1 -- 172.21.15.69:0/3888374619 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7ff3000ee2e0 con 0x56448d366bf0 2024-03-19T01:15:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff316b86740 10 client.26756 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:29.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff316b86740 10 client.26756 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:15:29.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff316b86740 20 client.26756 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:29.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff316b86740 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56448d3e6dd0 0x56448d3e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff316b86740 4 client.26756 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:15:29.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff316b86740 1 -- 172.21.15.69:0/3888374619 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56448d2d2c10 con 0x56448d3e6dd0 2024-03-19T01:15:29.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.704+0000 7ff310a00640 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56448d3e6dd0 0x56448d3e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.705+0000 7ff310a00640 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56448d3e6dd0 0x56448d3e91f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e59780a764a462e9 server_cookie=855dd0b30eec4116 in_seq=0 out_seq=0 2024-03-19T01:15:29.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.705+0000 7ff2f2a00640 10 client.26756 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:15:29.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.709+0000 7ff2f2a00640 1 -- 172.21.15.69:0/3888374619 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56448d2d2c10 con 0x56448d3e6dd0 2024-03-19T01:15:29.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.709+0000 7ff2f2a00640 10 client.26756 handle_command_reply: tid=0 2024-03-19T01:15:29.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.709+0000 7ff316b86740 1 client.26756 shutdown 2024-03-19T01:15:29.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.710+0000 7ff316b86740 1 -- 172.21.15.69:0/3888374619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d366bf0 msgr2=0x56448d378280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.710+0000 7ff316b86740 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d366bf0 0x56448d378280 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff30002cfd0 tx=0x7ff3000c12b0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.710+0000 7ff316b86740 1 -- 172.21.15.69:0/3888374619 shutdown_connections 2024-03-19T01:15:29.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.710+0000 7ff316b86740 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56448d3e6dd0 0x56448d3e91f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.710+0000 7ff316b86740 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56448d3700f0 0x56448d373bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.710+0000 7ff316b86740 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56448d367510 0x56448d3736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.710+0000 7ff316b86740 1 --2- 172.21.15.69:0/3888374619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56448d366bf0 0x56448d378280 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.710+0000 7ff2f2a00640 0 client.26756 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:15:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.710+0000 7ff316b86740 1 -- 172.21.15.69:0/3888374619 >> 172.21.15.69:0/3888374619 conn(0x56448d361320 msgr2=0x56448d360330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.710+0000 7ff316b86740 1 -- 172.21.15.69:0/3888374619 shutdown_connections 2024-03-19T01:15:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.710+0000 7ff316b86740 1 -- 172.21.15.69:0/3888374619 wait complete. 2024-03-19T01:15:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.711+0000 7ff316b86740 20 client.26756 trim_cache size 0 max 16384 2024-03-19T01:15:29.713 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:15:29.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.711+0000 7ff312800640 1 -- 172.21.15.69:0/319714870 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2f40a29f0 msgr2=0x7ff2f40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.711+0000 7ff312800640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2f40a29f0 0x7ff2f40a4eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff3040094d0 tx=0x7ff304004a40 comp rx=0 tx=0).stop 2024-03-19T01:15:29.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.711+0000 7ff312800640 1 -- 172.21.15.69:0/319714870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102850 msgr2=0x7ff30c1a31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.711+0000 7ff312800640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102850 0x7ff30c1a31c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff300005940 tx=0x7ff30002cbd0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.712+0000 7ff312800640 1 -- 172.21.15.69:0/319714870 shutdown_connections 2024-03-19T01:15:29.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.712+0000 7ff312800640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff30c10bc70 0x7ff30c10a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.712+0000 7ff312800640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c103170 0x7ff30c1a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.712+0000 7ff312800640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2f40a29f0 0x7ff2f40a4eb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.712+0000 7ff312800640 1 --2- 172.21.15.69:0/319714870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c102850 0x7ff30c1a31c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.712+0000 7ff312800640 1 -- 172.21.15.69:0/319714870 >> 172.21.15.69:0/319714870 conn(0x7ff30c0fc070 msgr2=0x7ff30c0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:29.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.712+0000 7ff312800640 1 -- 172.21.15.69:0/319714870 shutdown_connections 2024-03-19T01:15:29.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.712+0000 7ff312800640 1 -- 172.21.15.69:0/319714870 wait complete. 2024-03-19T01:15:29.732 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:15:29.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.965+0000 7efd02000640 1 -- 172.21.15.69:0/3680216082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc073c50 msgr2=0x7efcfc074030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.965+0000 7efd02000640 1 --2- 172.21.15.69:0/3680216082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc073c50 0x7efcfc074030 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009900 tx=0x7efcf002bcd0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.965+0000 7efd02000640 1 -- 172.21.15.69:0/3680216082 shutdown_connections 2024-03-19T01:15:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.965+0000 7efd02000640 1 --2- 172.21.15.69:0/3680216082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcfc10dec0 0x7efcfc110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.965+0000 7efd02000640 1 --2- 172.21.15.69:0/3680216082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc074570 0x7efcfc10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.965+0000 7efd02000640 1 --2- 172.21.15.69:0/3680216082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc073c50 0x7efcfc074030 secure :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009900 tx=0x7efcf002bcd0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.965+0000 7efd02000640 1 -- 172.21.15.69:0/3680216082 >> 172.21.15.69:0/3680216082 conn(0x7efcfc0fd190 msgr2=0x7efcfc0ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.965+0000 7efd02000640 1 -- 172.21.15.69:0/3680216082 shutdown_connections 2024-03-19T01:15:29.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.965+0000 7efd02000640 1 -- 172.21.15.69:0/3680216082 wait complete. 2024-03-19T01:15:29.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.966+0000 7efd02000640 1 Processor -- start 2024-03-19T01:15:29.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.967+0000 7efd02000640 1 -- start start 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.967+0000 7efd02000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc074570 0x7efcfc19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.967+0000 7efd02000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcfc10dec0 0x7efcfc19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.967+0000 7efd02000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc19ecc0 0x7efcfc19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.967+0000 7efd02000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc113cb0 con 0x7efcfc074570 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.967+0000 7efd02000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc113b30 con 0x7efcfc19ecc0 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.967+0000 7efd02000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc113e30 con 0x7efcfc10dec0 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efd00c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc074570 0x7efcfc19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcfc10dec0 0x7efcfc19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcfc10dec0 0x7efcfc19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:49890/0 (socket says 172.21.15.69:49890) 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efcfbe00640 1 -- 172.21.15.69:0/638377470 learned_addr learned my addr 172.21.15.69:0/638377470 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efd01600640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc19ecc0 0x7efcfc19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efcfbe00640 1 -- 172.21.15.69:0/638377470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc19ecc0 msgr2=0x7efcfc19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efcfbe00640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc19ecc0 0x7efcfc19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efcfbe00640 1 -- 172.21.15.69:0/638377470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc074570 msgr2=0x7efcfc19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efcfbe00640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc074570 0x7efcfc19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efcfbe00640 1 -- 172.21.15.69:0/638377470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc19f7b0 con 0x7efcfc10dec0 2024-03-19T01:15:29.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efd00c00640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc074570 0x7efcfc19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:15:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.968+0000 7efcfbe00640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcfc10dec0 0x7efcfc19a990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efcec00e9f0 tx=0x7efcec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.969+0000 7efcf9600640 1 -- 172.21.15.69:0/638377470 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec00cd80 con 0x7efcfc10dec0 2024-03-19T01:15:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.969+0000 7efcf9600640 1 -- 172.21.15.69:0/638377470 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcec010910 con 0x7efcfc10dec0 2024-03-19T01:15:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.969+0000 7efcf9600640 1 -- 172.21.15.69:0/638377470 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec018990 con 0x7efcfc10dec0 2024-03-19T01:15:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.969+0000 7efd02000640 1 -- 172.21.15.69:0/638377470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc19faa0 con 0x7efcfc10dec0 2024-03-19T01:15:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.969+0000 7efd02000640 1 -- 172.21.15.69:0/638377470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc1a71e0 con 0x7efcfc10dec0 2024-03-19T01:15:29.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.973+0000 7efd0642b740 1 Processor -- start 2024-03-19T01:15:29.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.973+0000 7efd0642b740 1 -- start start 2024-03-19T01:15:29.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.973+0000 7efcf9600640 1 -- 172.21.15.69:0/638377470 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7efcec018b30 con 0x7efcfc10dec0 2024-03-19T01:15:29.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.974+0000 7efd0642b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 0x55bf93fcffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.974+0000 7efd0642b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fd0510 0x55bf93fd8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.974+0000 7efd0642b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd90f0 0x55bf93fdb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.974+0000 7efd0642b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bf93f34380 con 0x55bf93fd0510 2024-03-19T01:15:29.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.974+0000 7efd0642b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bf93edcd20 con 0x55bf93fd90f0 2024-03-19T01:15:29.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.974+0000 7efd0642b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bf93f30420 con 0x55bf93fcfbf0 2024-03-19T01:15:29.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.974+0000 7efd01600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 0x55bf93fcffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.974+0000 7efd01600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 0x55bf93fcffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:49892/0 (socket says 172.21.15.69:49892) 2024-03-19T01:15:29.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.974+0000 7efd01600640 1 -- 172.21.15.69:0/2691383994 learned_addr learned my addr 172.21.15.69:0/2691383994 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:29.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.974+0000 7efd00c00640 1 --2- 172.21.15.69:0/2691383994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fd0510 0x55bf93fd8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.975+0000 7efd01600640 1 -- 172.21.15.69:0/2691383994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd90f0 msgr2=0x55bf93fdb4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:15:29.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.975+0000 7efd01600640 1 --2- 172.21.15.69:0/2691383994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd90f0 0x55bf93fdb4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.975+0000 7efd01600640 1 -- 172.21.15.69:0/2691383994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fd0510 msgr2=0x55bf93fd8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:29.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.975+0000 7efd01600640 1 --2- 172.21.15.69:0/2691383994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fd0510 0x55bf93fd8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.975+0000 7efd01600640 1 -- 172.21.15.69:0/2691383994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf93c483f0 con 0x55bf93fcfbf0 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.975+0000 7efd01600640 1 --2- 172.21.15.69:0/2691383994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 0x55bf93fcffd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009f50 tx=0x7efcf40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9766947f41a69233 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.975+0000 7efce9c00640 1 -- 172.21.15.69:0/2691383994 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4004b20 con 0x55bf93fcfbf0 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.975+0000 7efce9c00640 1 -- 172.21.15.69:0/2691383994 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf4004cc0 con 0x55bf93fcfbf0 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.975+0000 7efce9c00640 1 -- 172.21.15.69:0/2691383994 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4005450 con 0x55bf93fcfbf0 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.975+0000 7efcf9600640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efce40a2970 0x7efce40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.976+0000 7efcf9600640 1 -- 172.21.15.69:0/638377470 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7efcec0cf460 con 0x7efcfc10dec0 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.976+0000 7efd00c00640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efce40a2970 0x7efce40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.976+0000 7efd0642b740 1 -- 172.21.15.69:0/2691383994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 msgr2=0x55bf93fcffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.976+0000 7efd0642b740 1 --2- 172.21.15.69:0/2691383994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 0x55bf93fcffd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009f50 tx=0x7efcf40118d0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.976+0000 7efd0642b740 1 -- 172.21.15.69:0/2691383994 shutdown_connections 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.976+0000 7efd0642b740 1 --2- 172.21.15.69:0/2691383994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd90f0 0x55bf93fdb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.976+0000 7efd0642b740 1 --2- 172.21.15.69:0/2691383994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fd0510 0x55bf93fd8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.976+0000 7efd0642b740 1 --2- 172.21.15.69:0/2691383994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 0x55bf93fcffd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.976+0000 7efd0642b740 1 -- 172.21.15.69:0/2691383994 >> 172.21.15.69:0/2691383994 conn(0x55bf93fca320 msgr2=0x55bf93fca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:29.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.977+0000 7efd0642b740 1 -- 172.21.15.69:0/2691383994 shutdown_connections 2024-03-19T01:15:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.977+0000 7efd0642b740 1 -- 172.21.15.69:0/2691383994 wait complete. 2024-03-19T01:15:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.977+0000 7efd00c00640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efce40a2970 0x7efce40a4e30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efcf0036040 tx=0x7efcf0005b50 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.977+0000 7efd0642b740 1 -- start start 2024-03-19T01:15:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.978+0000 7efd0642b740 1 Processor -- start 2024-03-19T01:15:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.978+0000 7efd0642b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 0x55bf93fd6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.978+0000 7efd0642b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fd6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.978+0000 7efd0642b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fd90f0 0x55bf93fd6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.978+0000 7efd0642b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bf93fdbee0 con 0x55bf93fd90f0 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.978+0000 7efd0642b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bf93ef0160 con 0x55bf93fd0510 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.978+0000 7efd0642b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bf93fdc060 con 0x55bf93fcfbf0 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.978+0000 7efd0642b740 10 client.0 _ll_register_callbacks cb 0x55bf93f2c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.978+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fd6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.978+0000 7efd00c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 0x55bf93fd6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.979+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fd6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51250/0 (socket says 172.21.15.69:51250) 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.979+0000 7efd00c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 0x55bf93fd6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:49896/0 (socket says 172.21.15.69:49896) 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.979+0000 7efcfbe00640 1 -- 172.21.15.69:0/3013549723 learned_addr learned my addr 172.21.15.69:0/3013549723 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.979+0000 7efd01600640 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fd90f0 0x55bf93fd6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.979+0000 7efcfbe00640 1 -- 172.21.15.69:0/3013549723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 msgr2=0x55bf93fd6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.979+0000 7efcfbe00640 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 0x55bf93fd6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.979+0000 7efcfbe00640 1 -- 172.21.15.69:0/3013549723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fd90f0 msgr2=0x55bf93fd6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.979+0000 7efcfbe00640 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fd90f0 0x55bf93fd6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.979+0000 7efcfbe00640 1 -- 172.21.15.69:0/3013549723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf93d127d0 con 0x55bf93fd0510 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.979+0000 7efcfbe00640 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fd6670 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efcec0ccdf0 tx=0x7efcec09b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b24b54bd96263b5a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efcde000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efcde000640 1 -- 172.21.15.69:0/3013549723 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec09b700 con 0x55bf93fd0510 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efcde000640 1 -- 172.21.15.69:0/3013549723 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7efcec09ab50 con 0x55bf93fd0510 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efd0642b740 1 -- 172.21.15.69:0/3013549723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf4004cc0 con 0x55bf93fd0510 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efcde000640 1 -- 172.21.15.69:0/3013549723 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efcec09f850 con 0x55bf93fd0510 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efd0642b740 10 client.26765 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efd0642b740 1 -- 172.21.15.69:0/3013549723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bf93fe8200 con 0x55bf93fd0510 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efcde000640 1 -- 172.21.15.69:0/3013549723 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf4004cc0 con 0x55bf93fd0510 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efcde000640 1 -- 172.21.15.69:0/3013549723 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7efcec0afae0 con 0x55bf93fd0510 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efd0642b740 10 client.26765 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efd0642b740 10 client.26765 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:15:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.980+0000 7efd0642b740 20 client.26765 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.981+0000 7efd0642b740 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bf93fe96c0 0x55bf93febae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.981+0000 7efd0642b740 4 client.26765 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:15:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.981+0000 7efd0642b740 1 -- 172.21.15.69:0/3013549723 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf93f418c0 con 0x55bf93fe96c0 2024-03-19T01:15:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.981+0000 7efd00c00640 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bf93fe96c0 0x55bf93febae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.981+0000 7efd00c00640 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bf93fe96c0 0x55bf93febae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd0c5b6e985f5c42 server_cookie=32ff0af44054ffc9 in_seq=0 out_seq=0 2024-03-19T01:15:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.981+0000 7efcde000640 10 client.26765 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:15:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.982+0000 7efcde000640 1 -- 172.21.15.69:0/3013549723 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bf93f418c0 con 0x55bf93fe96c0 2024-03-19T01:15:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.982+0000 7efcde000640 10 client.26765 handle_command_reply: tid=0 2024-03-19T01:15:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.982+0000 7efd0642b740 1 client.26765 shutdown 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.982+0000 7efd0642b740 1 -- 172.21.15.69:0/3013549723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 msgr2=0x55bf93fd6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.982+0000 7efd0642b740 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fd6670 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efcec0ccdf0 tx=0x7efcec09b5c0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.982+0000 7efd0642b740 1 -- 172.21.15.69:0/3013549723 shutdown_connections 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.982+0000 7efd0642b740 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fd90f0 0x55bf93fd6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.982+0000 7efd0642b740 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bf93fe96c0 0x55bf93febae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.982+0000 7efd0642b740 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fd6670 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.982+0000 7efd0642b740 1 --2- 172.21.15.69:0/3013549723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fcfbf0 0x55bf93fd6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.982+0000 7efd0642b740 1 -- 172.21.15.69:0/3013549723 >> 172.21.15.69:0/3013549723 conn(0x55bf93fca320 msgr2=0x55bf93fcc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.983+0000 7efcde000640 0 client.26765 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.983+0000 7efd0642b740 1 -- 172.21.15.69:0/3013549723 shutdown_connections 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.983+0000 7efd0642b740 1 -- 172.21.15.69:0/3013549723 wait complete. 2024-03-19T01:15:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:29.983+0000 7efd0642b740 20 client.26765 trim_cache size 0 max 16384 2024-03-19T01:15:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.005+0000 7efd0642b740 1 Processor -- start 2024-03-19T01:15:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.005+0000 7efd0642b740 1 -- start start 2024-03-19T01:15:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.005+0000 7efd0642b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fcfbf0 0x55bf93fd8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.005+0000 7efd0642b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fe4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.005+0000 7efd0642b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 0x55bf93fc91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.005+0000 7efd0642b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efce000ffa0 con 0x55bf93fcfbf0 2024-03-19T01:15:30.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.005+0000 7efd0642b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efce0011c60 con 0x55bf93fd0510 2024-03-19T01:15:30.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.005+0000 7efd0642b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efce0010670 con 0x55bf93fd90f0 2024-03-19T01:15:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fe4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efd01600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 0x55bf93fc91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fe4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51260/0 (socket says 172.21.15.69:51260) 2024-03-19T01:15:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efcfbe00640 1 -- 172.21.15.69:0/2381410929 learned_addr learned my addr 172.21.15.69:0/2381410929 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efd01600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 0x55bf93fc91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:49898/0 (socket says 172.21.15.69:49898) 2024-03-19T01:15:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efd00c00640 1 --2- 172.21.15.69:0/2381410929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fcfbf0 0x55bf93fd8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efcfbe00640 1 -- 172.21.15.69:0/2381410929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 msgr2=0x55bf93fc91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efcfbe00640 1 --2- 172.21.15.69:0/2381410929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 0x55bf93fc91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efcfbe00640 1 -- 172.21.15.69:0/2381410929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fcfbf0 msgr2=0x55bf93fd8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efcfbe00640 1 --2- 172.21.15.69:0/2381410929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fcfbf0 0x55bf93fd8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efcfbe00640 1 -- 172.21.15.69:0/2381410929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf93fd4680 con 0x55bf93fd0510 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efd01600640 1 --2- 172.21.15.69:0/2381410929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 0x55bf93fc91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.006+0000 7efcfbe00640 1 --2- 172.21.15.69:0/2381410929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fe4ce0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efcec0cc8a0 tx=0x7efcec018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7da0b19f8b8f16a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.007+0000 7efce9200640 1 -- 172.21.15.69:0/2381410929 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec09b700 con 0x55bf93fd0510 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.007+0000 7efce9200640 1 -- 172.21.15.69:0/2381410929 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcec0a3ea0 con 0x55bf93fd0510 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.007+0000 7efce9200640 1 -- 172.21.15.69:0/2381410929 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec09ab50 con 0x55bf93fd0510 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.007+0000 7efd0642b740 1 -- 172.21.15.69:0/2381410929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 msgr2=0x55bf93fe4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.007+0000 7efd0642b740 1 --2- 172.21.15.69:0/2381410929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fe4ce0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efcec0cc8a0 tx=0x7efcec018420 comp rx=0 tx=0).stop 2024-03-19T01:15:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.008+0000 7efd0642b740 1 -- 172.21.15.69:0/2381410929 shutdown_connections 2024-03-19T01:15:30.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.008+0000 7efd0642b740 1 --2- 172.21.15.69:0/2381410929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 0x55bf93fc91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.008+0000 7efd0642b740 1 --2- 172.21.15.69:0/2381410929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fe4ce0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.008+0000 7efd0642b740 1 --2- 172.21.15.69:0/2381410929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fcfbf0 0x55bf93fd8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.008+0000 7efd0642b740 1 -- 172.21.15.69:0/2381410929 >> 172.21.15.69:0/2381410929 conn(0x55bf93fca320 msgr2=0x55bf93fc8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.008+0000 7efd0642b740 1 -- 172.21.15.69:0/2381410929 shutdown_connections 2024-03-19T01:15:30.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.008+0000 7efd0642b740 1 -- 172.21.15.69:0/2381410929 wait complete. 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.008+0000 7efd0642b740 1 -- start start 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.008+0000 7efd0642b740 1 Processor -- start 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.009+0000 7efd0642b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fcfbf0 0x55bf93fe15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.009+0000 7efd0642b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fdc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.009+0000 7efd0642b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 0x55bf93fdce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.009+0000 7efd0642b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efce00114d0 con 0x55bf93fcfbf0 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.009+0000 7efd0642b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efce0010d40 con 0x55bf93fd0510 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.009+0000 7efd0642b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efce000df80 con 0x55bf93fd90f0 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.009+0000 7efd0642b740 10 client.0 _ll_register_callbacks cb 0x7efcec00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.009+0000 7efd00c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fcfbf0 0x55bf93fe15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.010+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fdc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.010+0000 7efd01600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 0x55bf93fdce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.010+0000 7efd00c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fcfbf0 0x55bf93fe15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41530/0 (socket says 172.21.15.69:41530) 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.010+0000 7efd00c00640 1 -- 172.21.15.69:0/1663457495 learned_addr learned my addr 172.21.15.69:0/1663457495 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.010+0000 7efcfbe00640 1 -- 172.21.15.69:0/1663457495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 msgr2=0x55bf93fdce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.010+0000 7efcfbe00640 1 --2- 172.21.15.69:0/1663457495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 0x55bf93fdce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.010+0000 7efcfbe00640 1 -- 172.21.15.69:0/1663457495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fcfbf0 msgr2=0x55bf93fe15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.010+0000 7efcfbe00640 1 --2- 172.21.15.69:0/1663457495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fcfbf0 0x55bf93fe15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.010+0000 7efcfbe00640 1 -- 172.21.15.69:0/1663457495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf93fdd670 con 0x55bf93fd0510 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.010+0000 7efcfbe00640 1 --2- 172.21.15.69:0/1663457495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fdc950 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efcec097620 tx=0x7efcec01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ce57cf0494f8ea server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.010+0000 7efcde000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efcde000640 1 -- 172.21.15.69:0/1663457495 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec09b060 con 0x55bf93fd0510 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efcde000640 1 -- 172.21.15.69:0/1663457495 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7efcec09ad50 con 0x55bf93fd0510 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efd0642b740 1 -- 172.21.15.69:0/1663457495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcec0a3ea0 con 0x55bf93fd0510 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efcde000640 1 -- 172.21.15.69:0/1663457495 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efcec097450 con 0x55bf93fd0510 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efd0642b740 10 client.26771 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:30.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efd0642b740 1 -- 172.21.15.69:0/1663457495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bf93fdd670 con 0x55bf93fd0510 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efcde000640 1 -- 172.21.15.69:0/1663457495 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcec0a3ea0 con 0x55bf93fd0510 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efcde000640 1 -- 172.21.15.69:0/1663457495 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7efcec09a650 con 0x55bf93fd0510 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efd0642b740 10 client.26771 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efd0642b740 10 client.26771 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efd0642b740 20 client.26771 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efd0642b740 1 --2- 172.21.15.69:0/1663457495 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bf940584d0 0x55bf9405a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efd0642b740 4 client.26771 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.011+0000 7efd0642b740 1 -- 172.21.15.69:0/1663457495 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf93f3bc10 con 0x55bf940584d0 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.012+0000 7efd00c00640 1 --2- 172.21.15.69:0/1663457495 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bf940584d0 0x55bf9405a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.012+0000 7efd00c00640 1 --2- 172.21.15.69:0/1663457495 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bf940584d0 0x55bf9405a8c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b227c481d927af3 server_cookie=619431cd8a543a3d in_seq=0 out_seq=0 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.012+0000 7efcde000640 10 client.26771 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.012+0000 7efcde000640 1 -- 172.21.15.69:0/1663457495 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf93f3bc10 con 0x55bf940584d0 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.012+0000 7efcde000640 10 client.26771 handle_command_reply: tid=0 2024-03-19T01:15:30.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.012+0000 7efd0642b740 1 client.26771 shutdown 2024-03-19T01:15:30.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efd0642b740 1 -- 172.21.15.69:0/1663457495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 msgr2=0x55bf93fdc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efd0642b740 1 --2- 172.21.15.69:0/1663457495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fdc950 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efcec097620 tx=0x7efcec01f000 comp rx=0 tx=0).stop 2024-03-19T01:15:30.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efd0642b740 1 -- 172.21.15.69:0/1663457495 shutdown_connections 2024-03-19T01:15:30.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efd0642b740 1 --2- 172.21.15.69:0/1663457495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bf93fd90f0 0x55bf93fdce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efd0642b740 1 --2- 172.21.15.69:0/1663457495 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bf940584d0 0x55bf9405a8c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efd0642b740 1 --2- 172.21.15.69:0/1663457495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf93fd0510 0x55bf93fdc950 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efcde000640 0 client.26771 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:15:30.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efd0642b740 1 --2- 172.21.15.69:0/1663457495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bf93fcfbf0 0x55bf93fe15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efd0642b740 1 -- 172.21.15.69:0/1663457495 >> 172.21.15.69:0/1663457495 conn(0x55bf93fca320 msgr2=0x55bf93fc9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efd0642b740 1 -- 172.21.15.69:0/1663457495 shutdown_connections 2024-03-19T01:15:30.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efd0642b740 1 -- 172.21.15.69:0/1663457495 wait complete. 2024-03-19T01:15:30.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.013+0000 7efd0642b740 20 client.26771 trim_cache size 0 max 16384 2024-03-19T01:15:30.017 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:15:30.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.014+0000 7efd02000640 1 -- 172.21.15.69:0/638377470 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efce40a2970 msgr2=0x7efce40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.014+0000 7efd02000640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efce40a2970 0x7efce40a4e30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efcf0036040 tx=0x7efcf0005b50 comp rx=0 tx=0).stop 2024-03-19T01:15:30.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.014+0000 7efd02000640 1 -- 172.21.15.69:0/638377470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcfc10dec0 msgr2=0x7efcfc19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.014+0000 7efd02000640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcfc10dec0 0x7efcfc19a990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efcec00e9f0 tx=0x7efcec00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.015+0000 7efd02000640 1 -- 172.21.15.69:0/638377470 shutdown_connections 2024-03-19T01:15:30.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.015+0000 7efd02000640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efce40a2970 0x7efce40a4e30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.015+0000 7efd02000640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcfc19ecc0 0x7efcfc19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.015+0000 7efd02000640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcfc10dec0 0x7efcfc19a990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.015+0000 7efd02000640 1 --2- 172.21.15.69:0/638377470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc074570 0x7efcfc19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.015+0000 7efd02000640 1 -- 172.21.15.69:0/638377470 >> 172.21.15.69:0/638377470 conn(0x7efcfc0fd190 msgr2=0x7efcfc0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.015+0000 7efd02000640 1 -- 172.21.15.69:0/638377470 shutdown_connections 2024-03-19T01:15:30.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.015+0000 7efd02000640 1 -- 172.21.15.69:0/638377470 wait complete. 2024-03-19T01:15:30.034 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:15:30.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.278+0000 7f342dc00640 1 -- 172.21.15.69:0/123014422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428105a40 msgr2=0x7f3428105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.278+0000 7f342dc00640 1 --2- 172.21.15.69:0/123014422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428105a40 0x7f3428105ed0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3420009be0 tx=0x7f342002bf80 comp rx=0 tx=0).stop 2024-03-19T01:15:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.279+0000 7f342dc00640 1 -- 172.21.15.69:0/123014422 shutdown_connections 2024-03-19T01:15:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.279+0000 7f342dc00640 1 --2- 172.21.15.69:0/123014422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428105a40 0x7f3428105ed0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.279+0000 7f342dc00640 1 --2- 172.21.15.69:0/123014422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3428100f60 0x7f3428105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.279+0000 7f342dc00640 1 --2- 172.21.15.69:0/123014422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3428100640 0x7f3428100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.279+0000 7f342dc00640 1 -- 172.21.15.69:0/123014422 >> 172.21.15.69:0/123014422 conn(0x7f34280fc070 msgr2=0x7f34280fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.279+0000 7f342dc00640 1 -- 172.21.15.69:0/123014422 shutdown_connections 2024-03-19T01:15:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.279+0000 7f342dc00640 1 -- 172.21.15.69:0/123014422 wait complete. 2024-03-19T01:15:30.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.280+0000 7f342dc00640 1 Processor -- start 2024-03-19T01:15:30.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.280+0000 7f342dc00640 1 -- start start 2024-03-19T01:15:30.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.280+0000 7f342dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3428100640 0x7f342819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.280+0000 7f342dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428100f60 0x7f342819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.280+0000 7f342dc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3428105a40 0x7f34281a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.280+0000 7f342dc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3428118010 con 0x7f3428100640 2024-03-19T01:15:30.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.280+0000 7f342dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3428117e90 con 0x7f3428100f60 2024-03-19T01:15:30.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.280+0000 7f342dc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3428118190 con 0x7f3428105a40 2024-03-19T01:15:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:30 smithi069 ceph-mon[31727]: pgmap v147: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 243 op/s 2024-03-19T01:15:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:30 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2503345755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:15:30.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f3427e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3428100640 0x7f342819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f3427400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428100f60 0x7f342819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f3427400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428100f60 0x7f342819ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51284/0 (socket says 172.21.15.69:51284) 2024-03-19T01:15:30.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f3427e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3428100640 0x7f342819e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41542/0 (socket says 172.21.15.69:41542) 2024-03-19T01:15:30.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f3427400640 1 -- 172.21.15.69:0/2095707048 learned_addr learned my addr 172.21.15.69:0/2095707048 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f342d200640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3428105a40 0x7f34281a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f3427400640 1 -- 172.21.15.69:0/2095707048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3428105a40 msgr2=0x7f34281a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f3427400640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3428105a40 0x7f34281a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f3427400640 1 -- 172.21.15.69:0/2095707048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3428100640 msgr2=0x7f342819e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f3427400640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3428100640 0x7f342819e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f3427400640 1 -- 172.21.15.69:0/2095707048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34281a3500 con 0x7f3428100f60 2024-03-19T01:15:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.281+0000 7f3427400640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428100f60 0x7f342819ea50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f341400e9f0 tx=0x7f341400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.282+0000 7f3424c00640 1 -- 172.21.15.69:0/2095707048 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341400cab0 con 0x7f3428100f60 2024-03-19T01:15:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.282+0000 7f3424c00640 1 -- 172.21.15.69:0/2095707048 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f341400cc50 con 0x7f3428100f60 2024-03-19T01:15:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.282+0000 7f3424c00640 1 -- 172.21.15.69:0/2095707048 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140187e0 con 0x7f3428100f60 2024-03-19T01:15:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.282+0000 7f342dc00640 1 -- 172.21.15.69:0/2095707048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34281a37f0 con 0x7f3428100f60 2024-03-19T01:15:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.282+0000 7f342dc00640 1 -- 172.21.15.69:0/2095707048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34281ab080 con 0x7f3428100f60 2024-03-19T01:15:30.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.286+0000 7f3424c00640 1 -- 172.21.15.69:0/2095707048 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f341401f020 con 0x7f3428100f60 2024-03-19T01:15:30.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.286+0000 7f3431fc1740 1 Processor -- start 2024-03-19T01:15:30.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.287+0000 7f3431fc1740 1 -- start start 2024-03-19T01:15:30.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.287+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356e8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.287+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.287+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 0x557d356f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.287+0000 7f3431fc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557d3564d380 con 0x557d356f20f0 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.287+0000 7f3431fc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d355f5d20 con 0x557d356e8bf0 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.287+0000 7f3431fc1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557d35649420 con 0x557d356e9510 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.287+0000 7f342d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356e8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.287+0000 7f342d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356e8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51298/0 (socket says 172.21.15.69:51298) 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.287+0000 7f342d200640 1 -- 172.21.15.69:0/2547899884 learned_addr learned my addr 172.21.15.69:0/2547899884 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.288+0000 7f3427400640 1 --2- 172.21.15.69:0/2547899884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 0x557d356f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.288+0000 7f342d200640 1 -- 172.21.15.69:0/2547899884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 msgr2=0x557d356f1bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.288+0000 7f342d200640 1 --2- 172.21.15.69:0/2547899884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356f1bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.288+0000 7f342d200640 1 -- 172.21.15.69:0/2547899884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 msgr2=0x557d356f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.288+0000 7f342d200640 1 --2- 172.21.15.69:0/2547899884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 0x557d356f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.288+0000 7f342d200640 1 -- 172.21.15.69:0/2547899884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d353613f0 con 0x557d356e8bf0 2024-03-19T01:15:30.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.288+0000 7f3424c00640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f340c0a29f0 0x7f340c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.288+0000 7f3424c00640 1 -- 172.21.15.69:0/2095707048 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f34140cf2f0 con 0x7f3428100f60 2024-03-19T01:15:30.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.288+0000 7f342d200640 1 --2- 172.21.15.69:0/2547899884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356e8fd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3420035000 tx=0x7f3420009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d726f0651e52db5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.289+0000 7f341d600640 1 -- 172.21.15.69:0/2547899884 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420043030 con 0x557d356e8bf0 2024-03-19T01:15:30.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.289+0000 7f341d600640 1 -- 172.21.15.69:0/2547899884 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3420003e50 con 0x557d356e8bf0 2024-03-19T01:15:30.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.289+0000 7f341d600640 1 -- 172.21.15.69:0/2547899884 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34200042e0 con 0x557d356e8bf0 2024-03-19T01:15:30.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.289+0000 7f3427e00640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f340c0a29f0 0x7f340c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.290+0000 7f3427e00640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f340c0a29f0 0x7f340c0a4eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x557d356e86c0 tx=0x7f3418009260 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.290+0000 7f3431fc1740 1 -- 172.21.15.69:0/2547899884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 msgr2=0x557d356e8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.290+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2547899884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356e8fd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3420035000 tx=0x7f3420009840 comp rx=0 tx=0).stop 2024-03-19T01:15:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.290+0000 7f3431fc1740 1 -- 172.21.15.69:0/2547899884 shutdown_connections 2024-03-19T01:15:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.290+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2547899884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 0x557d356f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.290+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2547899884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.290+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2547899884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356e8fd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.290+0000 7f3431fc1740 1 -- 172.21.15.69:0/2547899884 >> 172.21.15.69:0/2547899884 conn(0x557d356e3320 msgr2=0x557d356e3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.291+0000 7f3431fc1740 1 -- 172.21.15.69:0/2547899884 shutdown_connections 2024-03-19T01:15:30.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.291+0000 7f3431fc1740 1 -- 172.21.15.69:0/2547899884 wait complete. 2024-03-19T01:15:30.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.292+0000 7f3431fc1740 1 -- start start 2024-03-19T01:15:30.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.292+0000 7f3431fc1740 1 Processor -- start 2024-03-19T01:15:30.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.292+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356ef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 0x557d356efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f3431fc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557d356f4e80 con 0x557d356f20f0 2024-03-19T01:15:30.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f3431fc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d35609160 con 0x557d356e8bf0 2024-03-19T01:15:30.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f3431fc1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557d356f5000 con 0x557d356e9510 2024-03-19T01:15:30.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f3431fc1740 10 client.0 _ll_register_callbacks cb 0x557d35645170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f342d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 0x557d356efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f342d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 0x557d356efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41568/0 (socket says 172.21.15.69:41568) 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f342d200640 1 -- 172.21.15.69:0/2704320030 learned_addr learned my addr 172.21.15.69:0/2704320030 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f3427e00640 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f3427400640 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f342d200640 1 -- 172.21.15.69:0/2704320030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 msgr2=0x557d356ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f342d200640 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f342d200640 1 -- 172.21.15.69:0/2704320030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 msgr2=0x557d356ef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f342d200640 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356ef1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.293+0000 7f342d200640 1 -- 172.21.15.69:0/2704320030 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d3542b7d0 con 0x557d356f20f0 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.294+0000 7f3427400640 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:15:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.294+0000 7f342d200640 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 0x557d356efc60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f342002c4c0 tx=0x7f3420009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad947384a408f10a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.294+0000 7f3412000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:15:30.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.295+0000 7f3412000640 1 -- 172.21.15.69:0/2704320030 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420043040 con 0x557d356f20f0 2024-03-19T01:15:30.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.295+0000 7f3412000640 1 -- 172.21.15.69:0/2704320030 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f3420004460 con 0x557d356f20f0 2024-03-19T01:15:30.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.295+0000 7f3431fc1740 1 -- 172.21.15.69:0/2704320030 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3420003e50 con 0x557d356f20f0 2024-03-19T01:15:30.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.295+0000 7f3412000640 1 -- 172.21.15.69:0/2704320030 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3420004660 con 0x557d356f20f0 2024-03-19T01:15:30.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.295+0000 7f3431fc1740 10 client.16557 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:30.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.295+0000 7f3431fc1740 1 -- 172.21.15.69:0/2704320030 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557d356f4e80 con 0x557d356f20f0 2024-03-19T01:15:30.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.296+0000 7f3412000640 1 -- 172.21.15.69:0/2704320030 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3420003e50 con 0x557d356f20f0 2024-03-19T01:15:30.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.296+0000 7f3412000640 1 -- 172.21.15.69:0/2704320030 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f3420033ae0 con 0x557d356f20f0 2024-03-19T01:15:30.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.297+0000 7f3431fc1740 10 client.16557 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:30.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.297+0000 7f3431fc1740 10 client.16557 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:15:30.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.297+0000 7f3431fc1740 20 client.16557 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:30.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.297+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x557d35702360 0x557d35704750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.297+0000 7f3431fc1740 4 client.16557 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:15:30.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.297+0000 7f3431fc1740 1 -- 172.21.15.69:0/2704320030 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d3565a8c0 con 0x557d35702360 2024-03-19T01:15:30.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.297+0000 7f3427e00640 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x557d35702360 0x557d35704750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.298+0000 7f3412000640 10 client.16557 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:15:30.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.298+0000 7f3427e00640 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x557d35702360 0x557d35704750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3378baaa286bab3c server_cookie=242bf6da7407d310 in_seq=0 out_seq=0 2024-03-19T01:15:30.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3412000640 1 -- 172.21.15.69:0/2704320030 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557d3565a8c0 con 0x557d35702360 2024-03-19T01:15:30.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3412000640 10 client.16557 handle_command_reply: tid=0 2024-03-19T01:15:30.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3431fc1740 1 client.16557 shutdown 2024-03-19T01:15:30.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3431fc1740 1 -- 172.21.15.69:0/2704320030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 msgr2=0x557d356efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 0x557d356efc60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f342002c4c0 tx=0x7f3420009960 comp rx=0 tx=0).stop 2024-03-19T01:15:30.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3431fc1740 1 -- 172.21.15.69:0/2704320030 shutdown_connections 2024-03-19T01:15:30.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x557d35702360 0x557d35704750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356f20f0 0x557d356efc60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2704320030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3431fc1740 1 -- 172.21.15.69:0/2704320030 >> 172.21.15.69:0/2704320030 conn(0x557d356e3320 msgr2=0x557d356e5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.299+0000 7f3412000640 0 client.16557 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:15:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.300+0000 7f3431fc1740 1 -- 172.21.15.69:0/2704320030 shutdown_connections 2024-03-19T01:15:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.300+0000 7f3431fc1740 1 -- 172.21.15.69:0/2704320030 wait complete. 2024-03-19T01:15:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.300+0000 7f3431fc1740 20 client.16557 trim_cache size 0 max 16384 2024-03-19T01:15:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.321+0000 7f3431fc1740 1 Processor -- start 2024-03-19T01:15:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.321+0000 7f3431fc1740 1 -- start start 2024-03-19T01:15:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.321+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e8bf0 0x557d356f1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.321+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356fdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.321+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356f20f0 0x557d35709f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.321+0000 7f3431fc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f34281b5140 con 0x557d356e8bf0 2024-03-19T01:15:30.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.321+0000 7f3431fc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34281b6e00 con 0x557d356f20f0 2024-03-19T01:15:30.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.321+0000 7f3431fc1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f34281b5810 con 0x557d356e9510 2024-03-19T01:15:30.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.322+0000 7f342d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356f20f0 0x557d35709f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.322+0000 7f3427e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e8bf0 0x557d356f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.322+0000 7f342d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356f20f0 0x557d35709f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51324/0 (socket says 172.21.15.69:51324) 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.322+0000 7f342d200640 1 -- 172.21.15.69:0/2165001566 learned_addr learned my addr 172.21.15.69:0/2165001566 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.322+0000 7f3427400640 1 --2- 172.21.15.69:0/2165001566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356fdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.322+0000 7f342d200640 1 -- 172.21.15.69:0/2165001566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 msgr2=0x557d356fdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.322+0000 7f342d200640 1 --2- 172.21.15.69:0/2165001566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356fdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.322+0000 7f342d200640 1 -- 172.21.15.69:0/2165001566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e8bf0 msgr2=0x557d356f1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.322+0000 7f342d200640 1 --2- 172.21.15.69:0/2165001566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e8bf0 0x557d356f1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.322+0000 7f342d200640 1 -- 172.21.15.69:0/2165001566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d356e9340 con 0x557d356f20f0 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.322+0000 7f342d200640 1 --2- 172.21.15.69:0/2165001566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356f20f0 0x557d35709f30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f34200344d0 tx=0x7f3420005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b4b44b267e990bf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.323+0000 7f341cc00640 1 -- 172.21.15.69:0/2165001566 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420043040 con 0x557d356f20f0 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.323+0000 7f341cc00640 1 -- 172.21.15.69:0/2165001566 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3420004800 con 0x557d356f20f0 2024-03-19T01:15:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.323+0000 7f341cc00640 1 -- 172.21.15.69:0/2165001566 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420033c60 con 0x557d356f20f0 2024-03-19T01:15:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.324+0000 7f3431fc1740 1 -- 172.21.15.69:0/2165001566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356f20f0 msgr2=0x557d35709f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.324+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2165001566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356f20f0 0x557d35709f30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f34200344d0 tx=0x7f3420005e30 comp rx=0 tx=0).stop 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.324+0000 7f3431fc1740 1 -- 172.21.15.69:0/2165001566 shutdown_connections 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.324+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2165001566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356f20f0 0x557d35709f30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.324+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2165001566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356e9510 0x557d356fdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.324+0000 7f3431fc1740 1 --2- 172.21.15.69:0/2165001566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e8bf0 0x557d356f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.324+0000 7f3431fc1740 1 -- 172.21.15.69:0/2165001566 >> 172.21.15.69:0/2165001566 conn(0x557d356e3320 msgr2=0x557d356e2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.324+0000 7f3431fc1740 1 -- 172.21.15.69:0/2165001566 shutdown_connections 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.324+0000 7f3431fc1740 1 -- 172.21.15.69:0/2165001566 wait complete. 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.324+0000 7f3431fc1740 1 -- start start 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.324+0000 7f3431fc1740 1 Processor -- start 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356ec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e9510 0x557d356ec650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f3431fc1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356f20f0 0x557d35767560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f3431fc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f34281b6670 con 0x557d356e9510 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f3431fc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34281b5ee0 con 0x557d356e8bf0 2024-03-19T01:15:30.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f3431fc1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f34281b3120 con 0x557d356f20f0 2024-03-19T01:15:30.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f3431fc1740 10 client.0 _ll_register_callbacks cb 0x7f3420009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f3427400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e9510 0x557d356ec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f3427e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f3427400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e9510 0x557d356ec650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41586/0 (socket says 172.21.15.69:41586) 2024-03-19T01:15:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f3427400640 1 -- 172.21.15.69:0/588685988 learned_addr learned my addr 172.21.15.69:0/588685988 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.325+0000 7f342d200640 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356f20f0 0x557d35767560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f342d200640 1 -- 172.21.15.69:0/588685988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 msgr2=0x557d356ec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f342d200640 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356ec110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f342d200640 1 -- 172.21.15.69:0/588685988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e9510 msgr2=0x557d356ec650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f342d200640 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e9510 0x557d356ec650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f342d200640 1 -- 172.21.15.69:0/588685988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d35767aa0 con 0x557d356f20f0 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f3427e00640 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f3427400640 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e9510 0x557d356ec650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f342d200640 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356f20f0 0x557d35767560 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3420041040 tx=0x7f3420009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17a07061ee1d327 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f3412000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f3412000640 1 -- 172.21.15.69:0/588685988 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420043040 con 0x557d356f20f0 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f3431fc1740 1 -- 172.21.15.69:0/588685988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3420004800 con 0x557d356f20f0 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.326+0000 7f3412000640 1 -- 172.21.15.69:0/588685988 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f3420004060 con 0x557d356f20f0 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.327+0000 7f3412000640 1 -- 172.21.15.69:0/588685988 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f342006a050 con 0x557d356f20f0 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.327+0000 7f3431fc1740 10 client.26749 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:30.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.327+0000 7f3431fc1740 1 -- 172.21.15.69:0/588685988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557d35768be0 con 0x557d356f20f0 2024-03-19T01:15:30.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.327+0000 7f3412000640 1 -- 172.21.15.69:0/588685988 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3420004800 con 0x557d356f20f0 2024-03-19T01:15:30.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.327+0000 7f3412000640 1 -- 172.21.15.69:0/588685988 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f3420034620 con 0x557d356f20f0 2024-03-19T01:15:30.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.327+0000 7f3431fc1740 10 client.26749 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:30.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.327+0000 7f3431fc1740 10 client.26749 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:15:30.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.327+0000 7f3431fc1740 20 client.26749 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:30.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.327+0000 7f3431fc1740 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x557d357691f0 0x557d3576b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.327+0000 7f3431fc1740 4 client.26749 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:15:30.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.327+0000 7f3431fc1740 1 -- 172.21.15.69:0/588685988 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d35654c10 con 0x557d357691f0 2024-03-19T01:15:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.328+0000 7f3427e00640 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x557d357691f0 0x557d3576b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.329+0000 7f3412000640 10 client.26749 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:15:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.329+0000 7f3427e00640 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x557d357691f0 0x557d3576b5e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e48e9e090c1c062e server_cookie=955d1ae2d1ae535f in_seq=0 out_seq=0 2024-03-19T01:15:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.329+0000 7f3412000640 1 -- 172.21.15.69:0/588685988 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d35654c10 con 0x557d357691f0 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.329+0000 7f3412000640 10 client.26749 handle_command_reply: tid=0 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.329+0000 7f3431fc1740 1 client.26749 shutdown 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.329+0000 7f3431fc1740 1 -- 172.21.15.69:0/588685988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356f20f0 msgr2=0x557d35767560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.329+0000 7f3431fc1740 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356f20f0 0x557d35767560 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3420041040 tx=0x7f3420009800 comp rx=0 tx=0).stop 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.330+0000 7f3431fc1740 1 -- 172.21.15.69:0/588685988 shutdown_connections 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.330+0000 7f3431fc1740 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x557d357691f0 0x557d3576b5e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.330+0000 7f3431fc1740 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557d356f20f0 0x557d35767560 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.330+0000 7f3431fc1740 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557d356e9510 0x557d356ec650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.330+0000 7f3431fc1740 1 --2- 172.21.15.69:0/588685988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d356e8bf0 0x557d356ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.330+0000 7f3431fc1740 1 -- 172.21.15.69:0/588685988 >> 172.21.15.69:0/588685988 conn(0x557d356e3320 msgr2=0x557d356ef1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.330+0000 7f3412000640 0 client.26749 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.330+0000 7f3431fc1740 1 -- 172.21.15.69:0/588685988 shutdown_connections 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.330+0000 7f3431fc1740 1 -- 172.21.15.69:0/588685988 wait complete. 2024-03-19T01:15:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.330+0000 7f3431fc1740 20 client.26749 trim_cache size 0 max 16384 2024-03-19T01:15:30.333 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:15:30.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.331+0000 7f342dc00640 1 -- 172.21.15.69:0/2095707048 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f340c0a29f0 msgr2=0x7f340c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.331+0000 7f342dc00640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f340c0a29f0 0x7f340c0a4eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x557d356e86c0 tx=0x7f3418009260 comp rx=0 tx=0).stop 2024-03-19T01:15:30.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.331+0000 7f342dc00640 1 -- 172.21.15.69:0/2095707048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428100f60 msgr2=0x7f342819ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.331+0000 7f342dc00640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428100f60 0x7f342819ea50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f341400e9f0 tx=0x7f341400edf0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.331+0000 7f342dc00640 1 -- 172.21.15.69:0/2095707048 shutdown_connections 2024-03-19T01:15:30.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.331+0000 7f342dc00640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3428105a40 0x7f34281a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.331+0000 7f342dc00640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f340c0a29f0 0x7f340c0a4eb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.331+0000 7f342dc00640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428100f60 0x7f342819ea50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.331+0000 7f342dc00640 1 --2- 172.21.15.69:0/2095707048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3428100640 0x7f342819e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.331+0000 7f342dc00640 1 -- 172.21.15.69:0/2095707048 >> 172.21.15.69:0/2095707048 conn(0x7f34280fc070 msgr2=0x7f34281033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.332+0000 7f342dc00640 1 -- 172.21.15.69:0/2095707048 shutdown_connections 2024-03-19T01:15:30.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.332+0000 7f342dc00640 1 -- 172.21.15.69:0/2095707048 wait complete. 2024-03-19T01:15:30.349 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:15:30.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:30 smithi086 ceph-mon[66829]: pgmap v147: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 243 op/s 2024-03-19T01:15:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:30 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2503345755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:15:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:30 smithi163 ceph-mon[67428]: pgmap v147: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 243 op/s 2024-03-19T01:15:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:30 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2503345755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:15:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.578+0000 7f9c1ca00640 1 -- 172.21.15.69:0/865958392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1810bc70 msgr2=0x7f9c1810e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.578+0000 7f9c1ca00640 1 --2- 172.21.15.69:0/865958392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1810bc70 0x7f9c1810e110 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009ca0 tx=0x7f9c1002c190 comp rx=0 tx=0).stop 2024-03-19T01:15:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.578+0000 7f9c1ca00640 1 -- 172.21.15.69:0/865958392 shutdown_connections 2024-03-19T01:15:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.578+0000 7f9c1ca00640 1 --2- 172.21.15.69:0/865958392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1810bc70 0x7f9c1810e110 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.578+0000 7f9c1ca00640 1 --2- 172.21.15.69:0/865958392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c18103170 0x7f9c1810b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.578+0000 7f9c1ca00640 1 --2- 172.21.15.69:0/865958392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c18102850 0x7f9c18102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.578+0000 7f9c1ca00640 1 -- 172.21.15.69:0/865958392 >> 172.21.15.69:0/865958392 conn(0x7f9c180fc070 msgr2=0x7f9c180fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.579+0000 7f9c1ca00640 1 -- 172.21.15.69:0/865958392 shutdown_connections 2024-03-19T01:15:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.579+0000 7f9c1ca00640 1 -- 172.21.15.69:0/865958392 wait complete. 2024-03-19T01:15:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.579+0000 7f9c1ca00640 1 Processor -- start 2024-03-19T01:15:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c1ca00640 1 -- start start 2024-03-19T01:15:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c1ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c18102850 0x7f9c1819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c1ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c18103170 0x7f9c1819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c1ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c1810bc70 0x7f9c1819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c1ca00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c18111a70 con 0x7f9c1810bc70 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c1ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c181118f0 con 0x7f9c18103170 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c1ca00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9c18111bf0 con 0x7f9c18102850 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c17400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c18102850 0x7f9c1819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c17400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c18102850 0x7f9c1819a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:49990/0 (socket says 172.21.15.69:49990) 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c17400640 1 -- 172.21.15.69:0/2611723174 learned_addr learned my addr 172.21.15.69:0/2611723174 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c16a00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c18103170 0x7f9c1819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.580+0000 7f9c17e00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c1810bc70 0x7f9c1819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.581+0000 7f9c17400640 1 -- 172.21.15.69:0/2611723174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c18103170 msgr2=0x7f9c1819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.581+0000 7f9c17400640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c18103170 0x7f9c1819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.581+0000 7f9c17400640 1 -- 172.21.15.69:0/2611723174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c1810bc70 msgr2=0x7f9c1819e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.581+0000 7f9c17400640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c1810bc70 0x7f9c1819e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.581+0000 7f9c17400640 1 -- 172.21.15.69:0/2611723174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1819f110 con 0x7f9c18102850 2024-03-19T01:15:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.581+0000 7f9c17400640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c18102850 0x7f9c1819a120 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800e9f0 tx=0x7f9c0800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.581+0000 7f9c16a00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c18103170 0x7f9c1819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:15:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.581+0000 7f9c17e00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c1810bc70 0x7f9c1819e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:15:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.581+0000 7f9c0fe00640 1 -- 172.21.15.69:0/2611723174 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800cd80 con 0x7f9c18102850 2024-03-19T01:15:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.582+0000 7f9c1ca00640 1 -- 172.21.15.69:0/2611723174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1819f400 con 0x7f9c18102850 2024-03-19T01:15:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.582+0000 7f9c0fe00640 1 -- 172.21.15.69:0/2611723174 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c08010910 con 0x7f9c18102850 2024-03-19T01:15:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.582+0000 7f9c1ca00640 1 -- 172.21.15.69:0/2611723174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c181a6c90 con 0x7f9c18102850 2024-03-19T01:15:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.582+0000 7f9c0fe00640 1 -- 172.21.15.69:0/2611723174 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08018980 con 0x7f9c18102850 2024-03-19T01:15:30.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.585+0000 7f9c20d8b740 1 Processor -- start 2024-03-19T01:15:30.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.586+0000 7f9c20d8b740 1 -- start start 2024-03-19T01:15:30.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.586+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcf400 0x5594c2bcf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.586+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcfd20 0x5594c2bd83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.586+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 0x5594c2bdacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.586+0000 7f9c20d8b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5594c2b2b380 con 0x5594c2bcf400 2024-03-19T01:15:30.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.586+0000 7f9c20d8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5594c2ad3d20 con 0x5594c2bcfd20 2024-03-19T01:15:30.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.586+0000 7f9c20d8b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5594c2b27420 con 0x5594c2bd8900 2024-03-19T01:15:30.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.586+0000 7f9c16a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 0x5594c2bdacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.587+0000 7f9c16a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 0x5594c2bdacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50000/0 (socket says 172.21.15.69:50000) 2024-03-19T01:15:30.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.587+0000 7f9c16a00640 1 -- 172.21.15.69:0/4166940995 learned_addr learned my addr 172.21.15.69:0/4166940995 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.587+0000 7f9c17e00640 1 --2- 172.21.15.69:0/4166940995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcf400 0x5594c2bcf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.587+0000 7f9c16a00640 1 -- 172.21.15.69:0/4166940995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcfd20 msgr2=0x5594c2bd83c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:15:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.587+0000 7f9c16a00640 1 --2- 172.21.15.69:0/4166940995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcfd20 0x5594c2bd83c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.587+0000 7f9c16a00640 1 -- 172.21.15.69:0/4166940995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcf400 msgr2=0x5594c2bcf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.587+0000 7f9c0fe00640 1 -- 172.21.15.69:0/2611723174 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9c08010430 con 0x7f9c18102850 2024-03-19T01:15:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.587+0000 7f9c16a00640 1 --2- 172.21.15.69:0/4166940995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcf400 0x5594c2bcf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.587+0000 7f9c16a00640 1 -- 172.21.15.69:0/4166940995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c283f3f0 con 0x5594c2bd8900 2024-03-19T01:15:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.587+0000 7f9c17e00640 1 --2- 172.21.15.69:0/4166940995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcf400 0x5594c2bcf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:15:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.587+0000 7f9c16a00640 1 --2- 172.21.15.69:0/4166940995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 0x5594c2bdacf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9c04011440 tx=0x7f9c04011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97149893b60a3d4c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.588+0000 7f9c03e00640 1 -- 172.21.15.69:0/4166940995 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0400f600 con 0x5594c2bd8900 2024-03-19T01:15:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.589+0000 7f9c03e00640 1 -- 172.21.15.69:0/4166940995 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c0400fc80 con 0x5594c2bd8900 2024-03-19T01:15:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.589+0000 7f9c03e00640 1 -- 172.21.15.69:0/4166940995 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04019d40 con 0x5594c2bd8900 2024-03-19T01:15:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.589+0000 7f9c0fe00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bfc0a2a70 0x7f9bfc0a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.589+0000 7f9c0fe00640 1 -- 172.21.15.69:0/2611723174 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f9c080cf5c0 con 0x7f9c18102850 2024-03-19T01:15:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.590+0000 7f9c17e00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bfc0a2a70 0x7f9bfc0a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.590+0000 7f9c20d8b740 1 -- 172.21.15.69:0/4166940995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 msgr2=0x5594c2bdacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.590+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/4166940995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 0x5594c2bdacf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9c04011440 tx=0x7f9c04011900 comp rx=0 tx=0).stop 2024-03-19T01:15:30.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.591+0000 7f9c20d8b740 1 -- 172.21.15.69:0/4166940995 shutdown_connections 2024-03-19T01:15:30.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.591+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/4166940995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 0x5594c2bdacf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.591+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/4166940995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcfd20 0x5594c2bd83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.591+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/4166940995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcf400 0x5594c2bcf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.591+0000 7f9c20d8b740 1 -- 172.21.15.69:0/4166940995 >> 172.21.15.69:0/4166940995 conn(0x5594c2bc9b30 msgr2=0x5594c2bc9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.591+0000 7f9c17e00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bfc0a2a70 0x7f9bfc0a4f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9c10004af0 tx=0x7f9c1002e040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.591+0000 7f9c20d8b740 1 -- 172.21.15.69:0/4166940995 shutdown_connections 2024-03-19T01:15:30.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.592+0000 7f9c20d8b740 1 -- 172.21.15.69:0/4166940995 wait complete. 2024-03-19T01:15:30.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.592+0000 7f9c20d8b740 1 -- start start 2024-03-19T01:15:30.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.593+0000 7f9c20d8b740 1 Processor -- start 2024-03-19T01:15:30.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.593+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bcf400 0x5594c2bd59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.593+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcfd20 0x5594c2bd5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.593+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bd8900 0x5594c2bd6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.593+0000 7f9c20d8b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5594c2bdb6f0 con 0x5594c2bd8900 2024-03-19T01:15:30.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.593+0000 7f9c20d8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5594c2ae7160 con 0x5594c2bcfd20 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.593+0000 7f9c20d8b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5594c2bdb870 con 0x5594c2bcf400 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.593+0000 7f9c20d8b740 10 client.0 _ll_register_callbacks cb 0x5594c2b23170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.594+0000 7f9c16a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcfd20 0x5594c2bd5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.594+0000 7f9c16a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcfd20 0x5594c2bd5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51358/0 (socket says 172.21.15.69:51358) 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.594+0000 7f9c16a00640 1 -- 172.21.15.69:0/1729647949 learned_addr learned my addr 172.21.15.69:0/1729647949 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.594+0000 7f9c17e00640 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bd8900 0x5594c2bd6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.594+0000 7f9c16a00640 1 -- 172.21.15.69:0/1729647949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bcf400 msgr2=0x5594c2bd59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.594+0000 7f9c16a00640 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bcf400 0x5594c2bd59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.594+0000 7f9c16a00640 1 -- 172.21.15.69:0/1729647949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bd8900 msgr2=0x5594c2bd6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.594+0000 7f9c16a00640 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bd8900 0x5594c2bd6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.594+0000 7f9c16a00640 1 -- 172.21.15.69:0/1729647949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c29097d0 con 0x5594c2bcfd20 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.594+0000 7f9c16a00640 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcfd20 0x5594c2bd5f10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9c0401b040 tx=0x7f9c040195b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4756e25e14f4d90 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.594+0000 7f9c00c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:15:30.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.595+0000 7f9c00c00640 1 -- 172.21.15.69:0/1729647949 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04026020 con 0x5594c2bcfd20 2024-03-19T01:15:30.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.595+0000 7f9c00c00640 1 -- 172.21.15.69:0/1729647949 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f9c04016040 con 0x5594c2bcfd20 2024-03-19T01:15:30.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.595+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1729647949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c0400fc80 con 0x5594c2bcfd20 2024-03-19T01:15:30.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.595+0000 7f9c00c00640 1 -- 172.21.15.69:0/1729647949 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9c04004250 con 0x5594c2bcfd20 2024-03-19T01:15:30.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.595+0000 7f9c20d8b740 10 client.26798 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:30.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.595+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1729647949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5594c2bf8af0 con 0x5594c2bcfd20 2024-03-19T01:15:30.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.595+0000 7f9c00c00640 1 -- 172.21.15.69:0/1729647949 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c0400fc80 con 0x5594c2bcfd20 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.596+0000 7f9c00c00640 1 -- 172.21.15.69:0/1729647949 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f9c04005290 con 0x5594c2bcfd20 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.596+0000 7f9c20d8b740 10 client.26798 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.596+0000 7f9c20d8b740 10 client.26798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.596+0000 7f9c20d8b740 20 client.26798 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.596+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5594c2bf9f00 0x5594c2bfc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.596+0000 7f9c20d8b740 4 client.26798 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.596+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1729647949 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594c2b388c0 con 0x5594c2bf9f00 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.596+0000 7f9c17400640 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5594c2bf9f00 0x5594c2bfc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.596+0000 7f9c17400640 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5594c2bf9f00 0x5594c2bfc2f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b20361e1301be9e server_cookie=e549697258318e47 in_seq=0 out_seq=0 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.596+0000 7f9c00c00640 10 client.26798 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c00c00640 1 -- 172.21.15.69:0/1729647949 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5594c2b388c0 con 0x5594c2bf9f00 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c00c00640 10 client.26798 handle_command_reply: tid=0 2024-03-19T01:15:30.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c20d8b740 1 client.26798 shutdown 2024-03-19T01:15:30.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1729647949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcfd20 msgr2=0x5594c2bd5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcfd20 0x5594c2bd5f10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9c0401b040 tx=0x7f9c040195b0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1729647949 shutdown_connections 2024-03-19T01:15:30.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bd8900 0x5594c2bd6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5594c2bf9f00 0x5594c2bfc2f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcfd20 0x5594c2bd5f10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1729647949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bcf400 0x5594c2bd59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1729647949 >> 172.21.15.69:0/1729647949 conn(0x5594c2bc9b30 msgr2=0x5594c2bcb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c00c00640 0 client.26798 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:15:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1729647949 shutdown_connections 2024-03-19T01:15:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.598+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1729647949 wait complete. 2024-03-19T01:15:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.599+0000 7f9c20d8b740 20 client.26798 trim_cache size 0 max 16384 2024-03-19T01:15:30.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c20d8b740 1 Processor -- start 2024-03-19T01:15:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c20d8b740 1 -- start start 2024-03-19T01:15:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcf400 0x5594c2bd8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bcfd20 0x5594c2be4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bd8900 0x5594c2c01b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c20d8b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf800ff40 con 0x5594c2bcf400 2024-03-19T01:15:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c20d8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8011c00 con 0x5594c2bd8900 2024-03-19T01:15:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c20d8b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8010610 con 0x5594c2bcfd20 2024-03-19T01:15:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c16a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bcfd20 0x5594c2be4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c16a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bcfd20 0x5594c2be4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50020/0 (socket says 172.21.15.69:50020) 2024-03-19T01:15:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c16a00640 1 -- 172.21.15.69:0/450217562 learned_addr learned my addr 172.21.15.69:0/450217562 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c17e00640 1 --2- 172.21.15.69:0/450217562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bd8900 0x5594c2c01b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.613+0000 7f9c17400640 1 --2- 172.21.15.69:0/450217562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcf400 0x5594c2bd8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.614+0000 7f9c16a00640 1 -- 172.21.15.69:0/450217562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bd8900 msgr2=0x5594c2c01b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.614+0000 7f9c16a00640 1 --2- 172.21.15.69:0/450217562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bd8900 0x5594c2c01b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.614+0000 7f9c16a00640 1 -- 172.21.15.69:0/450217562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcf400 msgr2=0x5594c2bd8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.614+0000 7f9c16a00640 1 --2- 172.21.15.69:0/450217562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcf400 0x5594c2bd8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.614+0000 7f9c16a00640 1 -- 172.21.15.69:0/450217562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c2bc9660 con 0x5594c2bcfd20 2024-03-19T01:15:30.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.614+0000 7f9c16a00640 1 --2- 172.21.15.69:0/450217562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bcfd20 0x5594c2be4520 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9c04004710 tx=0x7f9c04012f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b96856e2fca211fd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.614+0000 7f9c03400640 1 -- 172.21.15.69:0/450217562 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04026040 con 0x5594c2bcfd20 2024-03-19T01:15:30.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.615+0000 7f9c03400640 1 -- 172.21.15.69:0/450217562 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c04004790 con 0x5594c2bcfd20 2024-03-19T01:15:30.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.615+0000 7f9c03400640 1 -- 172.21.15.69:0/450217562 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04004e90 con 0x5594c2bcfd20 2024-03-19T01:15:30.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.616+0000 7f9c20d8b740 1 -- 172.21.15.69:0/450217562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bcfd20 msgr2=0x5594c2be4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.616+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/450217562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bcfd20 0x5594c2be4520 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9c04004710 tx=0x7f9c04012f40 comp rx=0 tx=0).stop 2024-03-19T01:15:30.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.616+0000 7f9c20d8b740 1 -- 172.21.15.69:0/450217562 shutdown_connections 2024-03-19T01:15:30.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.616+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/450217562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bd8900 0x5594c2c01b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.616+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/450217562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bcfd20 0x5594c2be4520 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.616+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/450217562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcf400 0x5594c2bd8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.616+0000 7f9c20d8b740 1 -- 172.21.15.69:0/450217562 >> 172.21.15.69:0/450217562 conn(0x5594c2bc9b30 msgr2=0x5594c2bd0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.616+0000 7f9c20d8b740 1 -- 172.21.15.69:0/450217562 shutdown_connections 2024-03-19T01:15:30.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.616+0000 7f9c20d8b740 1 -- 172.21.15.69:0/450217562 wait complete. 2024-03-19T01:15:30.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.617+0000 7f9c20d8b740 1 -- start start 2024-03-19T01:15:30.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.617+0000 7f9c20d8b740 1 Processor -- start 2024-03-19T01:15:30.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcf400 0x5594c2bdbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcfd20 0x5594c2bdc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c20d8b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 0x5594c2c5e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcf400 0x5594c2bdbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcf400 0x5594c2bdbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51376/0 (socket says 172.21.15.69:51376) 2024-03-19T01:15:30.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c17400640 1 -- 172.21.15.69:0/1034360634 learned_addr learned my addr 172.21.15.69:0/1034360634 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1034360634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8011470 con 0x5594c2bcfd20 2024-03-19T01:15:30.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1034360634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8010ce0 con 0x5594c2bcf400 2024-03-19T01:15:30.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1034360634 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf800df20 con 0x5594c2bd8900 2024-03-19T01:15:30.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c17e00640 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 0x5594c2c5e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c20d8b740 10 client.0 _ll_register_callbacks cb 0x7f9c04009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.621+0000 7f9c16a00640 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcfd20 0x5594c2bdc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.622+0000 7f9c17400640 1 -- 172.21.15.69:0/1034360634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 msgr2=0x5594c2c5e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.622+0000 7f9c17400640 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 0x5594c2c5e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.622+0000 7f9c17400640 1 -- 172.21.15.69:0/1034360634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcfd20 msgr2=0x5594c2bdc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.622+0000 7f9c17400640 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcfd20 0x5594c2bdc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.622+0000 7f9c17400640 1 -- 172.21.15.69:0/1034360634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c2c5e6d0 con 0x5594c2bcf400 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.622+0000 7f9c16a00640 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcfd20 0x5594c2bdc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.622+0000 7f9c17400640 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcf400 0x5594c2bdbd00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9c0809b500 tx=0x7f9c08009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14f4d474d77d9ff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.622+0000 7f9c00c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.622+0000 7f9c00c00640 1 -- 172.21.15.69:0/1034360634 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080a3fa0 con 0x5594c2bcf400 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.622+0000 7f9c00c00640 1 -- 172.21.15.69:0/1034360634 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f9c08086070 con 0x5594c2bcf400 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.622+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1034360634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c04004790 con 0x5594c2bcf400 2024-03-19T01:15:30.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.623+0000 7f9c00c00640 1 -- 172.21.15.69:0/1034360634 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9c08010720 con 0x5594c2bcf400 2024-03-19T01:15:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.623+0000 7f9c20d8b740 10 client.26804 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.623+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1034360634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5594c2c5f930 con 0x5594c2bcf400 2024-03-19T01:15:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.623+0000 7f9c00c00640 1 -- 172.21.15.69:0/1034360634 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c04004790 con 0x5594c2bcf400 2024-03-19T01:15:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.624+0000 7f9c00c00640 1 -- 172.21.15.69:0/1034360634 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f9c080afbe0 con 0x5594c2bcf400 2024-03-19T01:15:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.624+0000 7f9c20d8b740 10 client.26804 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.624+0000 7f9c20d8b740 10 client.26804 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:15:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.624+0000 7f9c20d8b740 20 client.26804 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.624+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5594c2c60120 0x5594c2c62540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.624+0000 7f9c20d8b740 4 client.26804 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:15:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.624+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1034360634 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594c2b32c10 con 0x5594c2c60120 2024-03-19T01:15:30.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.625+0000 7f9c16a00640 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5594c2c60120 0x5594c2c62540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.625+0000 7f9c16a00640 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5594c2c60120 0x5594c2c62540 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f0e2c839ae286cb server_cookie=91b38a04fd88d571 in_seq=0 out_seq=0 2024-03-19T01:15:30.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.625+0000 7f9c00c00640 10 client.26804 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:15:30.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.626+0000 7f9c00c00640 1 -- 172.21.15.69:0/1034360634 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594c2b32c10 con 0x5594c2c60120 2024-03-19T01:15:30.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.626+0000 7f9c00c00640 10 client.26804 handle_command_reply: tid=0 2024-03-19T01:15:30.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.626+0000 7f9c20d8b740 1 client.26804 shutdown 2024-03-19T01:15:30.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.626+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1034360634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcf400 msgr2=0x5594c2bdbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.626+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcf400 0x5594c2bdbd00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9c0809b500 tx=0x7f9c08009760 comp rx=0 tx=0).stop 2024-03-19T01:15:30.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.626+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1034360634 shutdown_connections 2024-03-19T01:15:30.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.626+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5594c2bd8900 0x5594c2c5e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.626+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5594c2bcfd20 0x5594c2bdc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.626+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5594c2c60120 0x5594c2c62540 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.626+0000 7f9c20d8b740 1 --2- 172.21.15.69:0/1034360634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594c2bcf400 0x5594c2bdbd00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.626+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1034360634 >> 172.21.15.69:0/1034360634 conn(0x5594c2bc9b30 msgr2=0x5594c2bd4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.627+0000 7f9c00c00640 0 client.26804 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:15:30.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.627+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1034360634 shutdown_connections 2024-03-19T01:15:30.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.627+0000 7f9c20d8b740 1 -- 172.21.15.69:0/1034360634 wait complete. 2024-03-19T01:15:30.629 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:15:30.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.627+0000 7f9c20d8b740 20 client.26804 trim_cache size 0 max 16384 2024-03-19T01:15:30.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 -- 172.21.15.69:0/2611723174 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bfc0a2a70 msgr2=0x7f9bfc0a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bfc0a2a70 0x7f9bfc0a4f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9c10004af0 tx=0x7f9c1002e040 comp rx=0 tx=0).stop 2024-03-19T01:15:30.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 -- 172.21.15.69:0/2611723174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c18102850 msgr2=0x7f9c1819a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c18102850 0x7f9c1819a120 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800e9f0 tx=0x7f9c0800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 -- 172.21.15.69:0/2611723174 shutdown_connections 2024-03-19T01:15:30.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c1810bc70 0x7f9c1819e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bfc0a2a70 0x7f9bfc0a4f30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c18103170 0x7f9c1819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 --2- 172.21.15.69:0/2611723174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c18102850 0x7f9c1819a120 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 -- 172.21.15.69:0/2611723174 >> 172.21.15.69:0/2611723174 conn(0x7f9c180fc070 msgr2=0x7f9c180fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 -- 172.21.15.69:0/2611723174 shutdown_connections 2024-03-19T01:15:30.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.628+0000 7f9c1ca00640 1 -- 172.21.15.69:0/2611723174 wait complete. 2024-03-19T01:15:30.652 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:15:30.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.889+0000 7f1e78a00640 1 -- 172.21.15.69:0/3413555706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e74100f60 msgr2=0x7f1e74105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.889+0000 7f1e78a00640 1 --2- 172.21.15.69:0/3413555706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e74100f60 0x7f1e74105420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009900 tx=0x7f1e6002bcd0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.889+0000 7f1e78a00640 1 -- 172.21.15.69:0/3413555706 shutdown_connections 2024-03-19T01:15:30.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.889+0000 7f1e78a00640 1 --2- 172.21.15.69:0/3413555706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74105a40 0x7f1e74105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.889+0000 7f1e78a00640 1 --2- 172.21.15.69:0/3413555706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e74100f60 0x7f1e74105420 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.889+0000 7f1e78a00640 1 --2- 172.21.15.69:0/3413555706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e74100640 0x7f1e74100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.889+0000 7f1e78a00640 1 -- 172.21.15.69:0/3413555706 >> 172.21.15.69:0/3413555706 conn(0x7f1e740fc070 msgr2=0x7f1e740fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.889+0000 7f1e78a00640 1 -- 172.21.15.69:0/3413555706 shutdown_connections 2024-03-19T01:15:30.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.889+0000 7f1e78a00640 1 -- 172.21.15.69:0/3413555706 wait complete. 2024-03-19T01:15:30.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.891+0000 7f1e78a00640 1 Processor -- start 2024-03-19T01:15:30.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.891+0000 7f1e78a00640 1 -- start start 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.891+0000 7f1e78a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e74100640 0x7f1e7419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.891+0000 7f1e78a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74100f60 0x7f1e7419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.891+0000 7f1e78a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e74105a40 0x7f1e741a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.891+0000 7f1e78a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1e74118010 con 0x7f1e74100640 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.891+0000 7f1e78a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e74117e90 con 0x7f1e74105a40 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.891+0000 7f1e78a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1e74118190 con 0x7f1e74100f60 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e72a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74100f60 0x7f1e7419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e74105a40 0x7f1e741a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e72a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74100f60 0x7f1e7419ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50052/0 (socket says 172.21.15.69:50052) 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e72a00640 1 -- 172.21.15.69:0/1709414489 learned_addr learned my addr 172.21.15.69:0/1709414489 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e73400640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e74100640 0x7f1e7419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e72a00640 1 -- 172.21.15.69:0/1709414489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e74105a40 msgr2=0x7f1e741a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e72a00640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e74105a40 0x7f1e741a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e72a00640 1 -- 172.21.15.69:0/1709414489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e74100640 msgr2=0x7f1e7419e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e72a00640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e74100640 0x7f1e7419e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e72a00640 1 -- 172.21.15.69:0/1709414489 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e741a34f0 con 0x7f1e74100f60 2024-03-19T01:15:30.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e73e00640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e74105a40 0x7f1e741a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:15:30.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.892+0000 7f1e73400640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e74100640 0x7f1e7419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:15:30.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.893+0000 7f1e72a00640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74100f60 0x7f1e7419ea90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1e60005ab0 tx=0x7f1e60004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.893+0000 7f1e6be00640 1 -- 172.21.15.69:0/1709414489 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60043020 con 0x7f1e74100f60 2024-03-19T01:15:30.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.893+0000 7f1e6be00640 1 -- 172.21.15.69:0/1709414489 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e600054e0 con 0x7f1e74100f60 2024-03-19T01:15:30.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.893+0000 7f1e6be00640 1 -- 172.21.15.69:0/1709414489 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60003af0 con 0x7f1e74100f60 2024-03-19T01:15:30.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.893+0000 7f1e78a00640 1 -- 172.21.15.69:0/1709414489 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e741a3780 con 0x7f1e74100f60 2024-03-19T01:15:30.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.893+0000 7f1e78a00640 1 -- 172.21.15.69:0/1709414489 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e741ab070 con 0x7f1e74100f60 2024-03-19T01:15:30.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.897+0000 7f1e7cde4740 1 Processor -- start 2024-03-19T01:15:30.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.897+0000 7f1e7cde4740 1 -- start start 2024-03-19T01:15:30.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.897+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef79bf0 0x55668ef79fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.897+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef7a510 0x55668ef82bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.897+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef830f0 0x55668ef854e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.897+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55668eede380 con 0x55668ef7a510 2024-03-19T01:15:30.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.897+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55668ee86d20 con 0x55668ef830f0 2024-03-19T01:15:30.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.897+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55668eeda420 con 0x55668ef79bf0 2024-03-19T01:15:30.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.898+0000 7f1e73e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef79bf0 0x55668ef79fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.898+0000 7f1e73400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef7a510 0x55668ef82bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.898+0000 7f1e73400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef7a510 0x55668ef82bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41656/0 (socket says 172.21.15.69:41656) 2024-03-19T01:15:30.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.898+0000 7f1e73400640 1 -- 172.21.15.69:0/2237790094 learned_addr learned my addr 172.21.15.69:0/2237790094 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.898+0000 7f1e73e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef79bf0 0x55668ef79fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50064/0 (socket says 172.21.15.69:50064) 2024-03-19T01:15:30.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.898+0000 7f1e73e00640 1 -- 172.21.15.69:0/2237790094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef830f0 msgr2=0x55668ef854e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:15:30.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.898+0000 7f1e73e00640 1 --2- 172.21.15.69:0/2237790094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef830f0 0x55668ef854e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.898+0000 7f1e73e00640 1 -- 172.21.15.69:0/2237790094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef7a510 msgr2=0x55668ef82bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.898+0000 7f1e73e00640 1 --2- 172.21.15.69:0/2237790094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef7a510 0x55668ef82bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.898+0000 7f1e73e00640 1 -- 172.21.15.69:0/2237790094 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55668ebf23f0 con 0x55668ef79bf0 2024-03-19T01:15:30.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.898+0000 7f1e6be00640 1 -- 172.21.15.69:0/1709414489 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f1e60046020 con 0x7f1e74100f60 2024-03-19T01:15:30.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.899+0000 7f1e73e00640 1 --2- 172.21.15.69:0/2237790094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef79bf0 0x55668ef79fd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c005fd0 tx=0x7f1e6c011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=365dbe692f89c210 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.899+0000 7f1e5fe00640 1 -- 172.21.15.69:0/2237790094 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c004d20 con 0x55668ef79bf0 2024-03-19T01:15:30.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.899+0000 7f1e5fe00640 1 -- 172.21.15.69:0/2237790094 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e6c00f9a0 con 0x55668ef79bf0 2024-03-19T01:15:30.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.899+0000 7f1e5fe00640 1 -- 172.21.15.69:0/2237790094 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c0045b0 con 0x55668ef79bf0 2024-03-19T01:15:30.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.900+0000 7f1e7cde4740 1 -- 172.21.15.69:0/2237790094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef79bf0 msgr2=0x55668ef79fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.900+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/2237790094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef79bf0 0x55668ef79fd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c005fd0 tx=0x7f1e6c011bf0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.900+0000 7f1e7cde4740 1 -- 172.21.15.69:0/2237790094 shutdown_connections 2024-03-19T01:15:30.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.900+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/2237790094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef830f0 0x55668ef854e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.900+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/2237790094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef7a510 0x55668ef82bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.900+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/2237790094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef79bf0 0x55668ef79fd0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.900+0000 7f1e7cde4740 1 -- 172.21.15.69:0/2237790094 >> 172.21.15.69:0/2237790094 conn(0x55668ef74320 msgr2=0x55668ef74700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.901+0000 7f1e6be00640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e580a29f0 0x7f1e580a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.901+0000 7f1e6be00640 1 -- 172.21.15.69:0/1709414489 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1e600f1160 con 0x7f1e74100f60 2024-03-19T01:15:30.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.901+0000 7f1e73400640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e580a29f0 0x7f1e580a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.901+0000 7f1e7cde4740 1 -- 172.21.15.69:0/2237790094 shutdown_connections 2024-03-19T01:15:30.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.902+0000 7f1e7cde4740 1 -- 172.21.15.69:0/2237790094 wait complete. 2024-03-19T01:15:30.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.902+0000 7f1e73400640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e580a29f0 0x7f1e580a4eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55668ef796c0 tx=0x7f1e64005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.902+0000 7f1e7cde4740 1 -- start start 2024-03-19T01:15:30.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.902+0000 7f1e7cde4740 1 Processor -- start 2024-03-19T01:15:30.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef80190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef830f0 0x55668ef80c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55668ef85ee0 con 0x55668ef79bf0 2024-03-19T01:15:30.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55668ee9a160 con 0x55668ef7a510 2024-03-19T01:15:30.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55668ef86060 con 0x55668ef830f0 2024-03-19T01:15:30.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e7cde4740 10 client.0 _ll_register_callbacks cb 0x55668eed6170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e73400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef80190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e72a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e72a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef806d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51420/0 (socket says 172.21.15.69:51420) 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e72a00640 1 -- 172.21.15.69:0/3634586961 learned_addr learned my addr 172.21.15.69:0/3634586961 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.903+0000 7f1e73e00640 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef830f0 0x55668ef80c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e72a00640 1 -- 172.21.15.69:0/3634586961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef830f0 msgr2=0x55668ef80c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e72a00640 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef830f0 0x55668ef80c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e72a00640 1 -- 172.21.15.69:0/3634586961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 msgr2=0x55668ef80190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e72a00640 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef80190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e72a00640 1 -- 172.21.15.69:0/3634586961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55668ecbc7d0 con 0x55668ef7a510 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e72a00640 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef806d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009a20 tx=0x7f1e600bcff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d4bcc3201686538 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e5cc00640 1 -- 172.21.15.69:0/3634586961 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e600bdbb0 con 0x55668ef7a510 2024-03-19T01:15:30.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3634586961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e6c00f9a0 con 0x55668ef7a510 2024-03-19T01:15:30.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e5cc00640 1 -- 172.21.15.69:0/3634586961 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1e600ee710 con 0x55668ef7a510 2024-03-19T01:15:30.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e5cc00640 1 -- 172.21.15.69:0/3634586961 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1e600ee910 con 0x55668ef7a510 2024-03-19T01:15:30.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e7cde4740 10 client.26813 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:30.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.904+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3634586961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55668ef921e0 con 0x55668ef7a510 2024-03-19T01:15:30.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.905+0000 7f1e5cc00640 1 -- 172.21.15.69:0/3634586961 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e6c00f9a0 con 0x55668ef7a510 2024-03-19T01:15:30.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.905+0000 7f1e5cc00640 1 -- 172.21.15.69:0/3634586961 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f1e600eec70 con 0x55668ef7a510 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.905+0000 7f1e7cde4740 10 client.26813 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.905+0000 7f1e7cde4740 10 client.26813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.905+0000 7f1e7cde4740 20 client.26813 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.905+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55668ef935b0 0x55668ef959d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.905+0000 7f1e7cde4740 4 client.26813 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.905+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3634586961 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55668eeeb8c0 con 0x55668ef935b0 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.905+0000 7f1e73400640 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55668ef935b0 0x55668ef959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.906+0000 7f1e73400640 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55668ef935b0 0x55668ef959d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88768b9e1bb88d31 server_cookie=3edca44793d91898 in_seq=0 out_seq=0 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.906+0000 7f1e5cc00640 10 client.26813 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.907+0000 7f1e5cc00640 1 -- 172.21.15.69:0/3634586961 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55668eeeb8c0 con 0x55668ef935b0 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.907+0000 7f1e5cc00640 10 client.26813 handle_command_reply: tid=0 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.907+0000 7f1e7cde4740 1 client.26813 shutdown 2024-03-19T01:15:30.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3634586961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 msgr2=0x55668ef806d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef806d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009a20 tx=0x7f1e600bcff0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3634586961 shutdown_connections 2024-03-19T01:15:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55668ef935b0 0x55668ef959d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef830f0 0x55668ef80c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef806d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/3634586961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef80190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3634586961 >> 172.21.15.69:0/3634586961 conn(0x55668ef74320 msgr2=0x55668ef76070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e5cc00640 0 client.26813 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3634586961 shutdown_connections 2024-03-19T01:15:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3634586961 wait complete. 2024-03-19T01:15:30.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.908+0000 7f1e7cde4740 20 client.26813 trim_cache size 0 max 16384 2024-03-19T01:15:30.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.926+0000 7f1e7cde4740 1 Processor -- start 2024-03-19T01:15:30.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.926+0000 7f1e7cde4740 1 -- start start 2024-03-19T01:15:30.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef8ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef7a510 0x55668ef80a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef830f0 0x55668ef9b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741b50e0 con 0x55668ef79bf0 2024-03-19T01:15:30.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741b6da0 con 0x55668ef830f0 2024-03-19T01:15:30.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741b57b0 con 0x55668ef7a510 2024-03-19T01:15:30.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef830f0 0x55668ef9b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef830f0 0x55668ef9b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51434/0 (socket says 172.21.15.69:51434) 2024-03-19T01:15:30.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e73e00640 1 -- 172.21.15.69:0/3484478013 learned_addr learned my addr 172.21.15.69:0/3484478013 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e73400640 1 --2- 172.21.15.69:0/3484478013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef8ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e72a00640 1 --2- 172.21.15.69:0/3484478013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef7a510 0x55668ef80a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e73e00640 1 -- 172.21.15.69:0/3484478013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef7a510 msgr2=0x55668ef80a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e73e00640 1 --2- 172.21.15.69:0/3484478013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef7a510 0x55668ef80a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.927+0000 7f1e73e00640 1 -- 172.21.15.69:0/3484478013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 msgr2=0x55668ef8ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.928+0000 7f1e73e00640 1 --2- 172.21.15.69:0/3484478013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef8ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.928+0000 7f1e73e00640 1 -- 172.21.15.69:0/3484478013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55668ef78d90 con 0x55668ef830f0 2024-03-19T01:15:30.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.928+0000 7f1e72a00640 1 --2- 172.21.15.69:0/3484478013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef7a510 0x55668ef80a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:15:30.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.928+0000 7f1e73400640 1 --2- 172.21.15.69:0/3484478013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef8ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.928+0000 7f1e73e00640 1 --2- 172.21.15.69:0/3484478013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef830f0 0x55668ef9b280 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c004100 tx=0x7f1e6c011210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de1700c3f82927b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.928+0000 7f1e5f400640 1 -- 172.21.15.69:0/3484478013 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c029070 con 0x55668ef830f0 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.928+0000 7f1e5f400640 1 -- 172.21.15.69:0/3484478013 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e6c00fe80 con 0x55668ef830f0 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.928+0000 7f1e5f400640 1 -- 172.21.15.69:0/3484478013 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c01e040 con 0x55668ef830f0 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.929+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3484478013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef830f0 msgr2=0x55668ef9b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.929+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/3484478013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef830f0 0x55668ef9b280 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c004100 tx=0x7f1e6c011210 comp rx=0 tx=0).stop 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.929+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3484478013 shutdown_connections 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.929+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/3484478013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef830f0 0x55668ef9b280 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.929+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/3484478013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef7a510 0x55668ef80a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.929+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/3484478013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef8ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.929+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3484478013 >> 172.21.15.69:0/3484478013 conn(0x55668ef74320 msgr2=0x55668ef73330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.929+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3484478013 shutdown_connections 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.929+0000 7f1e7cde4740 1 -- 172.21.15.69:0/3484478013 wait complete. 2024-03-19T01:15:30.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.930+0000 7f1e7cde4740 1 -- start start 2024-03-19T01:15:30.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.930+0000 7f1e7cde4740 1 Processor -- start 2024-03-19T01:15:30.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.930+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef7d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.930+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef7d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e7cde4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef830f0 0x55668eff88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741b6610 con 0x55668ef79bf0 2024-03-19T01:15:30.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741b5e80 con 0x55668ef7a510 2024-03-19T01:15:30.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e7cde4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741b30c0 con 0x55668ef830f0 2024-03-19T01:15:30.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e7cde4740 10 client.0 _ll_register_callbacks cb 0x7f1e600b9750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:30.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e72a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef7d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e72a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef7d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51442/0 (socket says 172.21.15.69:51442) 2024-03-19T01:15:30.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e72a00640 1 -- 172.21.15.69:0/248892646 learned_addr learned my addr 172.21.15.69:0/248892646 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e73400640 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef7d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e73e00640 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef830f0 0x55668eff88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e72a00640 1 -- 172.21.15.69:0/248892646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef830f0 msgr2=0x55668eff88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e72a00640 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef830f0 0x55668eff88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e72a00640 1 -- 172.21.15.69:0/248892646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 msgr2=0x55668ef7d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e72a00640 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef7d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.931+0000 7f1e72a00640 1 -- 172.21.15.69:0/248892646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55668eff8df0 con 0x55668ef7a510 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.932+0000 7f1e72a00640 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef7d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1e6002c8c0 tx=0x7f1e600bd120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83468e960d5e4b8f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.932+0000 7f1e5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.932+0000 7f1e5cc00640 1 -- 172.21.15.69:0/248892646 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e600bce10 con 0x55668ef7a510 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.932+0000 7f1e5cc00640 1 -- 172.21.15.69:0/248892646 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1e600c1120 con 0x55668ef7a510 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.932+0000 7f1e7cde4740 1 -- 172.21.15.69:0/248892646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55668eee5c10 con 0x55668ef7a510 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.932+0000 7f1e5cc00640 1 -- 172.21.15.69:0/248892646 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1e600bd4b0 con 0x55668ef7a510 2024-03-19T01:15:30.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.932+0000 7f1e7cde4740 10 client.26819 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:30.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.932+0000 7f1e7cde4740 1 -- 172.21.15.69:0/248892646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55668effa0f0 con 0x55668ef7a510 2024-03-19T01:15:30.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.932+0000 7f1e5cc00640 1 -- 172.21.15.69:0/248892646 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55668eee5c10 con 0x55668ef7a510 2024-03-19T01:15:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.933+0000 7f1e5cc00640 1 -- 172.21.15.69:0/248892646 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f1e600bc900 con 0x55668ef7a510 2024-03-19T01:15:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.933+0000 7f1e7cde4740 10 client.26819 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.933+0000 7f1e7cde4740 10 client.26819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:15:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.933+0000 7f1e7cde4740 20 client.26819 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.933+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55668effa7b0 0x55668effcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.933+0000 7f1e7cde4740 4 client.26819 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:15:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.933+0000 7f1e7cde4740 1 -- 172.21.15.69:0/248892646 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55668ee63560 con 0x55668effa7b0 2024-03-19T01:15:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.933+0000 7f1e73400640 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55668effa7b0 0x55668effcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:30.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.933+0000 7f1e73400640 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55668effa7b0 0x55668effcbd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7014460ac5e8203 server_cookie=4716fd6bd0549317 in_seq=0 out_seq=0 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.933+0000 7f1e5cc00640 10 client.26819 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e5cc00640 1 -- 172.21.15.69:0/248892646 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55668ee63560 con 0x55668effa7b0 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e5cc00640 10 client.26819 handle_command_reply: tid=0 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e7cde4740 1 client.26819 shutdown 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e7cde4740 1 -- 172.21.15.69:0/248892646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 msgr2=0x55668ef7d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef7d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1e6002c8c0 tx=0x7f1e600bd120 comp rx=0 tx=0).stop 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e7cde4740 1 -- 172.21.15.69:0/248892646 shutdown_connections 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55668effa7b0 0x55668effcbd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55668ef830f0 0x55668eff88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55668ef7a510 0x55668ef7d990 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e5cc00640 0 client.26819 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e7cde4740 1 --2- 172.21.15.69:0/248892646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55668ef79bf0 0x55668ef7d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e7cde4740 1 -- 172.21.15.69:0/248892646 >> 172.21.15.69:0/248892646 conn(0x55668ef74320 msgr2=0x55668ef754b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e7cde4740 1 -- 172.21.15.69:0/248892646 shutdown_connections 2024-03-19T01:15:30.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.934+0000 7f1e7cde4740 1 -- 172.21.15.69:0/248892646 wait complete. 2024-03-19T01:15:30.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.935+0000 7f1e7cde4740 20 client.26819 trim_cache size 0 max 16384 2024-03-19T01:15:30.938 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.935+0000 7f1e78a00640 1 -- 172.21.15.69:0/1709414489 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e580a29f0 msgr2=0x7f1e580a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.935+0000 7f1e78a00640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e580a29f0 0x7f1e580a4eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55668ef796c0 tx=0x7f1e64005f90 comp rx=0 tx=0).stop 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.935+0000 7f1e78a00640 1 -- 172.21.15.69:0/1709414489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74100f60 msgr2=0x7f1e7419ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.935+0000 7f1e78a00640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74100f60 0x7f1e7419ea90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1e60005ab0 tx=0x7f1e60004970 comp rx=0 tx=0).stop 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.936+0000 7f1e78a00640 1 -- 172.21.15.69:0/1709414489 shutdown_connections 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.936+0000 7f1e78a00640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e580a29f0 0x7f1e580a4eb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.936+0000 7f1e78a00640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e74105a40 0x7f1e741a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.936+0000 7f1e78a00640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74100f60 0x7f1e7419ea90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.936+0000 7f1e78a00640 1 --2- 172.21.15.69:0/1709414489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e74100640 0x7f1e7419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.936+0000 7f1e78a00640 1 -- 172.21.15.69:0/1709414489 >> 172.21.15.69:0/1709414489 conn(0x7f1e740fc070 msgr2=0x7f1e741033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.936+0000 7f1e78a00640 1 -- 172.21.15.69:0/1709414489 shutdown_connections 2024-03-19T01:15:30.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:30.936+0000 7f1e78a00640 1 -- 172.21.15.69:0/1709414489 wait complete. 2024-03-19T01:15:31.960 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:15:31.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:15:31.961 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:15:32.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.158+0000 7f80f9a00640 1 -- 172.21.15.69:0/1870890596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f41017b0 msgr2=0x7f80f4101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.158+0000 7f80f9a00640 1 --2- 172.21.15.69:0/1870890596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f41017b0 0x7f80f4101c30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009900 tx=0x7f80e002bc60 comp rx=0 tx=0).stop 2024-03-19T01:15:32.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.158+0000 7f80f9a00640 1 -- 172.21.15.69:0/1870890596 shutdown_connections 2024-03-19T01:15:32.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.158+0000 7f80f9a00640 1 --2- 172.21.15.69:0/1870890596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80f4102170 0x7f80f410e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.158+0000 7f80f9a00640 1 --2- 172.21.15.69:0/1870890596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f41017b0 0x7f80f4101c30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.158+0000 7f80f9a00640 1 --2- 172.21.15.69:0/1870890596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f41077c0 0x7f80f4107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.158+0000 7f80f9a00640 1 -- 172.21.15.69:0/1870890596 >> 172.21.15.69:0/1870890596 conn(0x7f80f40fd280 msgr2=0x7f80f40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.158+0000 7f80f9a00640 1 -- 172.21.15.69:0/1870890596 shutdown_connections 2024-03-19T01:15:32.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.159+0000 7f80f9a00640 1 -- 172.21.15.69:0/1870890596 wait complete. 2024-03-19T01:15:32.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.159+0000 7f80f9a00640 1 Processor -- start 2024-03-19T01:15:32.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.160+0000 7f80f9a00640 1 -- start start 2024-03-19T01:15:32.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.160+0000 7f80f9a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f41017b0 0x7f80f41a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.160+0000 7f80f9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4102170 0x7f80f41a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.160+0000 7f80f9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80f41077c0 0x7f80f419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.160+0000 7f80f9a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4074190 con 0x7f80f41017b0 2024-03-19T01:15:32.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.160+0000 7f80f9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4074010 con 0x7f80f4102170 2024-03-19T01:15:32.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.160+0000 7f80f9a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4074310 con 0x7f80f41077c0 2024-03-19T01:15:32.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.160+0000 7f80f2a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f41017b0 0x7f80f41a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.160+0000 7f80f2a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f41017b0 0x7f80f41a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41696/0 (socket says 172.21.15.69:41696) 2024-03-19T01:15:32.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.161+0000 7f80f2a00640 1 -- 172.21.15.69:0/3826364554 learned_addr learned my addr 172.21.15.69:0/3826364554 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:32.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.160+0000 7f80f2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4102170 0x7f80f41a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.161+0000 7f80f3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80f41077c0 0x7f80f419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.161+0000 7f80f2a00640 1 -- 172.21.15.69:0/3826364554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80f41077c0 msgr2=0x7f80f419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.161+0000 7f80f2a00640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80f41077c0 0x7f80f419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.161+0000 7f80f2a00640 1 -- 172.21.15.69:0/3826364554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4102170 msgr2=0x7f80f41a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.161+0000 7f80f2a00640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4102170 0x7f80f41a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.161+0000 7f80f2a00640 1 -- 172.21.15.69:0/3826364554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f419adf0 con 0x7f80f41017b0 2024-03-19T01:15:32.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.162+0000 7f80f2a00640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f41017b0 0x7f80f41a0090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80e400e9f0 tx=0x7f80e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.162+0000 7f80eb400640 1 -- 172.21.15.69:0/3826364554 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e400cd80 con 0x7f80f41017b0 2024-03-19T01:15:32.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.162+0000 7f80eb400640 1 -- 172.21.15.69:0/3826364554 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e4010910 con 0x7f80f41017b0 2024-03-19T01:15:32.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.162+0000 7f80f9a00640 1 -- 172.21.15.69:0/3826364554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f419b080 con 0x7f80f41017b0 2024-03-19T01:15:32.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.162+0000 7f80eb400640 1 -- 172.21.15.69:0/3826364554 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e40188f0 con 0x7f80f41017b0 2024-03-19T01:15:32.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.162+0000 7f80f9a00640 1 -- 172.21.15.69:0/3826364554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f41a6ea0 con 0x7f80f41017b0 2024-03-19T01:15:32.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.167+0000 7f80fc785740 1 Processor -- start 2024-03-19T01:15:32.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.167+0000 7f80fc785740 1 -- start start 2024-03-19T01:15:32.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.167+0000 7f80fc785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07aa6040 0x55ba07aa6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.167+0000 7f80fc785740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07aa6960 0x55ba07a9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.167+0000 7f80fc785740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07ab8280 0x55ba07aba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.167+0000 7f80fc785740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ba07a00fb0 con 0x55ba07aa6960 2024-03-19T01:15:32.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.167+0000 7f80fc785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba079a9ab0 con 0x55ba07aa6040 2024-03-19T01:15:32.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.167+0000 7f80fc785740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ba079fd050 con 0x55ba07ab8280 2024-03-19T01:15:32.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.168+0000 7f80f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07aa6040 0x55ba07aa6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.168+0000 7f80eb400640 1 -- 172.21.15.69:0/3826364554 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f80e4010ac0 con 0x7f80f41017b0 2024-03-19T01:15:32.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.168+0000 7f80f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07aa6040 0x55ba07aa6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51480/0 (socket says 172.21.15.69:51480) 2024-03-19T01:15:32.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.168+0000 7f80f3400640 1 -- 172.21.15.69:0/177172172 learned_addr learned my addr 172.21.15.69:0/177172172 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:32.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.168+0000 7f80f2000640 1 --2- 172.21.15.69:0/177172172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07ab8280 0x55ba07aba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.168+0000 7f80f3400640 1 -- 172.21.15.69:0/177172172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07ab8280 msgr2=0x55ba07aba670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.168+0000 7f80f3400640 1 --2- 172.21.15.69:0/177172172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07ab8280 0x55ba07aba670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.168+0000 7f80f3400640 1 -- 172.21.15.69:0/177172172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07aa6960 msgr2=0x55ba07a9fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.168+0000 7f80f3400640 1 --2- 172.21.15.69:0/177172172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07aa6960 0x55ba07a9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.168+0000 7f80f3400640 1 -- 172.21.15.69:0/177172172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba079c7e00 con 0x55ba07aa6040 2024-03-19T01:15:32.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.169+0000 7f80f3400640 1 --2- 172.21.15.69:0/177172172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07aa6040 0x55ba07aa6420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00a430 tx=0x7f80ec009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df7b943cd6ab03b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.169+0000 7f80dfe00640 1 -- 172.21.15.69:0/177172172 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec013790 con 0x55ba07aa6040 2024-03-19T01:15:32.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.169+0000 7f80dfe00640 1 -- 172.21.15.69:0/177172172 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec019070 con 0x55ba07aa6040 2024-03-19T01:15:32.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.169+0000 7f80dfe00640 1 -- 172.21.15.69:0/177172172 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec0122e0 con 0x55ba07aa6040 2024-03-19T01:15:32.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.170+0000 7f80fc785740 1 -- 172.21.15.69:0/177172172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07aa6040 msgr2=0x55ba07aa6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.170+0000 7f80fc785740 1 --2- 172.21.15.69:0/177172172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07aa6040 0x55ba07aa6420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00a430 tx=0x7f80ec009650 comp rx=0 tx=0).stop 2024-03-19T01:15:32.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.170+0000 7f80fc785740 1 -- 172.21.15.69:0/177172172 shutdown_connections 2024-03-19T01:15:32.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.170+0000 7f80fc785740 1 --2- 172.21.15.69:0/177172172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07ab8280 0x55ba07aba670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.170+0000 7f80fc785740 1 --2- 172.21.15.69:0/177172172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07aa6960 0x55ba07a9fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.170+0000 7f80fc785740 1 --2- 172.21.15.69:0/177172172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07aa6040 0x55ba07aa6420 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.170+0000 7f80fc785740 1 -- 172.21.15.69:0/177172172 >> 172.21.15.69:0/177172172 conn(0x55ba07aa0a40 msgr2=0x55ba07aa0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.170+0000 7f80fc785740 1 -- 172.21.15.69:0/177172172 shutdown_connections 2024-03-19T01:15:32.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.170+0000 7f80eb400640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80d80a2900 0x7f80d80a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.170+0000 7f80fc785740 1 -- 172.21.15.69:0/177172172 wait complete. 2024-03-19T01:15:32.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.170+0000 7f80f2000640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80d80a2900 0x7f80d80a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.171+0000 7f80fc785740 1 -- start start 2024-03-19T01:15:32.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.171+0000 7f80eb400640 1 -- 172.21.15.69:0/3826364554 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f80e40cf4b0 con 0x7f80f41017b0 2024-03-19T01:15:32.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.172+0000 7f80fc785740 1 Processor -- start 2024-03-19T01:15:32.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.172+0000 7f80f2000640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80d80a2900 0x7f80d80a4dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f80e0005ab0 tx=0x7f80e0005a00 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.172+0000 7f80fc785740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07aa6040 0x55ba07a9d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.172+0000 7f80fc785740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6960 0x55ba07a9d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.172+0000 7f80fc785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07a9dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.172+0000 7f80fc785740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ba07abadf0 con 0x55ba07aa6040 2024-03-19T01:15:32.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.172+0000 7f80fc785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba079bcf00 con 0x55ba07ab8280 2024-03-19T01:15:32.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.172+0000 7f80fc785740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ba07abaf70 con 0x55ba07aa6960 2024-03-19T01:15:32.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80fc785740 10 client.0 _ll_register_callbacks cb 0x55ba079bc8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f2000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6960 0x55ba07a9d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07a9dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07a9dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51490/0 (socket says 172.21.15.69:51490) 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f3400640 1 -- 172.21.15.69:0/3032730911 learned_addr learned my addr 172.21.15.69:0/3032730911 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f2000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6960 0x55ba07a9d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50134/0 (socket says 172.21.15.69:50134) 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f2a00640 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07aa6040 0x55ba07a9d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f2000640 1 -- 172.21.15.69:0/3032730911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 msgr2=0x55ba07a9dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f2000640 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07a9dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f2000640 1 -- 172.21.15.69:0/3032730911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07aa6040 msgr2=0x55ba07a9d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f2000640 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07aa6040 0x55ba07a9d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f2000640 1 -- 172.21.15.69:0/3032730911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba077dd7e0 con 0x55ba07aa6960 2024-03-19T01:15:32.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.173+0000 7f80f2a00640 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07aa6040 0x55ba07a9d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:15:32.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.174+0000 7f80f2000640 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6960 0x55ba07a9d750 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f80e002c1a0 tx=0x7f80e0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d4bbd8b59e4e52e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.174+0000 7f80dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.174+0000 7f80dcc00640 1 -- 172.21.15.69:0/3032730911 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0056060 con 0x55ba07aa6960 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.174+0000 7f80dcc00640 1 -- 172.21.15.69:0/3032730911 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f80e0004ee0 con 0x55ba07aa6960 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.174+0000 7f80fc785740 1 -- 172.21.15.69:0/3032730911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ec019070 con 0x55ba07aa6960 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.174+0000 7f80dcc00640 1 -- 172.21.15.69:0/3032730911 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f80e00050e0 con 0x55ba07aa6960 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.174+0000 7f80fc785740 10 client.26791 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.174+0000 7f80fc785740 1 -- 172.21.15.69:0/3032730911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ba0783df70 con 0x55ba07aa6960 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.175+0000 7f80dcc00640 1 -- 172.21.15.69:0/3032730911 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec019070 con 0x55ba07aa6960 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.175+0000 7f80dcc00640 1 -- 172.21.15.69:0/3032730911 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f80e0046040 con 0x55ba07aa6960 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.175+0000 7f80fc785740 10 client.26791 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.175+0000 7f80fc785740 10 client.26791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.175+0000 7f80fc785740 20 client.26791 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.175+0000 7f80fc785740 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba07ac8450 0x55ba07aca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.175+0000 7f80fc785740 4 client.26791 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:15:32.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.175+0000 7f80fc785740 1 -- 172.21.15.69:0/3032730911 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba07a0e4f0 con 0x55ba07ac8450 2024-03-19T01:15:32.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.176+0000 7f80f2a00640 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba07ac8450 0x55ba07aca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.176+0000 7f80f2a00640 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba07ac8450 0x55ba07aca870 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8182d21761e94b server_cookie=a3c024bca859c925 in_seq=0 out_seq=0 2024-03-19T01:15:32.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.176+0000 7f80dcc00640 10 client.26791 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:32.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80dcc00640 1 -- 172.21.15.69:0/3032730911 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ba07a0e4f0 con 0x55ba07ac8450 2024-03-19T01:15:32.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80dcc00640 10 client.26791 handle_command_reply: tid=0 2024-03-19T01:15:32.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80fc785740 1 client.26791 shutdown 2024-03-19T01:15:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80fc785740 1 -- 172.21.15.69:0/3032730911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6960 msgr2=0x55ba07a9d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80fc785740 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6960 0x55ba07a9d750 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f80e002c1a0 tx=0x7f80e0009260 comp rx=0 tx=0).stop 2024-03-19T01:15:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80fc785740 1 -- 172.21.15.69:0/3032730911 shutdown_connections 2024-03-19T01:15:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80fc785740 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba07ac8450 0x55ba07aca870 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80fc785740 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07a9dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80fc785740 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6960 0x55ba07a9d750 secure :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f80e002c1a0 tx=0x7f80e0009260 comp rx=0 tx=0).stop 2024-03-19T01:15:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80fc785740 1 --2- 172.21.15.69:0/3032730911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07aa6040 0x55ba07a9d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80dcc00640 0 client.26791 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.177+0000 7f80fc785740 1 -- 172.21.15.69:0/3032730911 >> 172.21.15.69:0/3032730911 conn(0x55ba07aa0a40 msgr2=0x55ba07aa2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.178+0000 7f80fc785740 1 -- 172.21.15.69:0/3032730911 shutdown_connections 2024-03-19T01:15:32.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.178+0000 7f80fc785740 1 -- 172.21.15.69:0/3032730911 wait complete. 2024-03-19T01:15:32.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.178+0000 7f80fc785740 20 client.26791 trim_cache size 0 max 16384 2024-03-19T01:15:32.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.214+0000 7f80fc785740 1 Processor -- start 2024-03-19T01:15:32.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.214+0000 7f80fc785740 1 -- start start 2024-03-19T01:15:32.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.214+0000 7f80fc785740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6040 0x55ba07a9f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.214+0000 7f80fc785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07a972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.214+0000 7f80fc785740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07a9dab0 0x55ba07aa2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.214+0000 7f80fc785740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41acbf0 con 0x55ba07a9dab0 2024-03-19T01:15:32.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.214+0000 7f80fc785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41ae8b0 con 0x55ba07ab8280 2024-03-19T01:15:32.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.214+0000 7f80fc785740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41ad2c0 con 0x55ba07aa6040 2024-03-19T01:15:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6040 0x55ba07a9f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6040 0x55ba07a9f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50144/0 (socket says 172.21.15.69:50144) 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f2a00640 1 -- 172.21.15.69:0/246037968 learned_addr learned my addr 172.21.15.69:0/246037968 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f2000640 1 --2- 172.21.15.69:0/246037968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07a972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f3400640 1 --2- 172.21.15.69:0/246037968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07a9dab0 0x55ba07aa2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f2a00640 1 -- 172.21.15.69:0/246037968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 msgr2=0x55ba07a972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f2a00640 1 --2- 172.21.15.69:0/246037968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07a972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f2a00640 1 -- 172.21.15.69:0/246037968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07a9dab0 msgr2=0x55ba07aa2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f2a00640 1 --2- 172.21.15.69:0/246037968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07a9dab0 0x55ba07aa2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f2a00640 1 -- 172.21.15.69:0/246037968 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba07aa6790 con 0x55ba07aa6040 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f3400640 1 --2- 172.21.15.69:0/246037968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07a9dab0 0x55ba07aa2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.215+0000 7f80f2a00640 1 --2- 172.21.15.69:0/246037968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6040 0x55ba07a9f940 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f80e409b5a0 tx=0x7f80e401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8dd25311a09d31 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.216+0000 7f80df400640 1 -- 172.21.15.69:0/246037968 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4099ba0 con 0x55ba07aa6040 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.216+0000 7f80df400640 1 -- 172.21.15.69:0/246037968 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e4097f10 con 0x55ba07aa6040 2024-03-19T01:15:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.216+0000 7f80df400640 1 -- 172.21.15.69:0/246037968 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e409acf0 con 0x55ba07aa6040 2024-03-19T01:15:32.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.217+0000 7f80fc785740 1 -- 172.21.15.69:0/246037968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6040 msgr2=0x55ba07a9f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.217+0000 7f80fc785740 1 --2- 172.21.15.69:0/246037968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6040 0x55ba07a9f940 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f80e409b5a0 tx=0x7f80e401a040 comp rx=0 tx=0).stop 2024-03-19T01:15:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.217+0000 7f80fc785740 1 -- 172.21.15.69:0/246037968 shutdown_connections 2024-03-19T01:15:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.217+0000 7f80fc785740 1 --2- 172.21.15.69:0/246037968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07a9dab0 0x55ba07aa2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.217+0000 7f80fc785740 1 --2- 172.21.15.69:0/246037968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07a972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.217+0000 7f80fc785740 1 --2- 172.21.15.69:0/246037968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07aa6040 0x55ba07a9f940 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f80e409b5a0 tx=0x7f80e401a040 comp rx=0 tx=0).stop 2024-03-19T01:15:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.217+0000 7f80fc785740 1 -- 172.21.15.69:0/246037968 >> 172.21.15.69:0/246037968 conn(0x55ba07aa0a40 msgr2=0x55ba07aa1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.217+0000 7f80fc785740 1 -- 172.21.15.69:0/246037968 shutdown_connections 2024-03-19T01:15:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.217+0000 7f80fc785740 1 -- 172.21.15.69:0/246037968 wait complete. 2024-03-19T01:15:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.218+0000 7f80fc785740 1 -- start start 2024-03-19T01:15:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.218+0000 7f80fc785740 1 Processor -- start 2024-03-19T01:15:32.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80fc785740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07a9dab0 0x55ba07abcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80fc785740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07abd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80fc785740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07abd710 0x55ba07b2e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80fc785740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41ae120 con 0x55ba07abd710 2024-03-19T01:15:32.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80fc785740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41ad990 con 0x55ba07ab8280 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80fc785740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41aabd0 con 0x55ba07a9dab0 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80fc785740 10 client.0 _ll_register_callbacks cb 0x7f80e0004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80f2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07a9dab0 0x55ba07abcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80f2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07a9dab0 0x55ba07abcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50156/0 (socket says 172.21.15.69:50156) 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80f2a00640 1 -- 172.21.15.69:0/1607974441 learned_addr learned my addr 172.21.15.69:0/1607974441 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80f2000640 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07abd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80f3400640 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07abd710 0x55ba07b2e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80f2a00640 1 -- 172.21.15.69:0/1607974441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 msgr2=0x55ba07abd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80f2a00640 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07abd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80f2a00640 1 -- 172.21.15.69:0/1607974441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07abd710 msgr2=0x55ba07b2e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80f2a00640 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07abd710 0x55ba07b2e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.219+0000 7f80f2a00640 1 -- 172.21.15.69:0/1607974441 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba076d5170 con 0x55ba07a9dab0 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.220+0000 7f80f2a00640 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07a9dab0 0x55ba07abcc90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f80e4097d00 tx=0x7f80e400c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5265090b850e80b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.220+0000 7f80dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:15:32.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.220+0000 7f80dcc00640 1 -- 172.21.15.69:0/1607974441 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e40b3070 con 0x55ba07a9dab0 2024-03-19T01:15:32.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.220+0000 7f80dcc00640 1 -- 172.21.15.69:0/1607974441 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f80e40860b0 con 0x55ba07a9dab0 2024-03-19T01:15:32.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.220+0000 7f80fc785740 1 -- 172.21.15.69:0/1607974441 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e4097f10 con 0x55ba07a9dab0 2024-03-19T01:15:32.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.220+0000 7f80dcc00640 1 -- 172.21.15.69:0/1607974441 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f80e40cf150 con 0x55ba07a9dab0 2024-03-19T01:15:32.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.220+0000 7f80fc785740 10 client.26797 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:32.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.220+0000 7f80fc785740 1 -- 172.21.15.69:0/1607974441 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ba07b2fe90 con 0x55ba07a9dab0 2024-03-19T01:15:32.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.220+0000 7f80dcc00640 1 -- 172.21.15.69:0/1607974441 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e4097f10 con 0x55ba07a9dab0 2024-03-19T01:15:32.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.221+0000 7f80dcc00640 1 -- 172.21.15.69:0/1607974441 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f80e40bb040 con 0x55ba07a9dab0 2024-03-19T01:15:32.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.221+0000 7f80fc785740 10 client.26797 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:32.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.221+0000 7f80fc785740 10 client.26797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:15:32.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.221+0000 7f80fc785740 20 client.26797 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:32.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.221+0000 7f80fc785740 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba07b30510 0x55ba07b32930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.221+0000 7f80fc785740 4 client.26797 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:15:32.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.221+0000 7f80fc785740 1 -- 172.21.15.69:0/1607974441 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ba07a08840 con 0x55ba07b30510 2024-03-19T01:15:32.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.222+0000 7f80f2000640 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba07b30510 0x55ba07b32930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.222+0000 7f80dcc00640 10 client.26797 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:32.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.222+0000 7f80f2000640 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba07b30510 0x55ba07b32930 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea837a3911c86b3 server_cookie=17e71291274c2cbb in_seq=0 out_seq=0 2024-03-19T01:15:32.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80dcc00640 1 -- 172.21.15.69:0/1607974441 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ba07a08840 con 0x55ba07b30510 2024-03-19T01:15:32.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80dcc00640 10 client.26797 handle_command_reply: tid=0 2024-03-19T01:15:32.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80fc785740 1 client.26797 shutdown 2024-03-19T01:15:32.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80fc785740 1 -- 172.21.15.69:0/1607974441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07a9dab0 msgr2=0x55ba07abcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80fc785740 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07a9dab0 0x55ba07abcc90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f80e4097d00 tx=0x7f80e400c7d0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80fc785740 1 -- 172.21.15.69:0/1607974441 shutdown_connections 2024-03-19T01:15:32.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80fc785740 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba07b30510 0x55ba07b32930 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80fc785740 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba07abd710 0x55ba07b2e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80fc785740 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba07ab8280 0x55ba07abd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80fc785740 1 --2- 172.21.15.69:0/1607974441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba07a9dab0 0x55ba07abcc90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80fc785740 1 -- 172.21.15.69:0/1607974441 >> 172.21.15.69:0/1607974441 conn(0x55ba07aa0a40 msgr2=0x55ba07a97120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80dcc00640 0 client.26797 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:32.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.224+0000 7f80fc785740 1 -- 172.21.15.69:0/1607974441 shutdown_connections 2024-03-19T01:15:32.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.225+0000 7f80fc785740 1 -- 172.21.15.69:0/1607974441 wait complete. 2024-03-19T01:15:32.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.225+0000 7f80fc785740 20 client.26797 trim_cache size 0 max 16384 2024-03-19T01:15:32.227 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:15:32.227 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:15:32.227 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "5eed2f48-f4ed-4b79-9053-8383f3a95ca5", 2024-03-19T01:15:32.227 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:15:32.228 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:15:32.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 -- 172.21.15.69:0/3826364554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80d80a2900 msgr2=0x7f80d80a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80d80a2900 0x7f80d80a4dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f80e0005ab0 tx=0x7f80e0005a00 comp rx=0 tx=0).stop 2024-03-19T01:15:32.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 -- 172.21.15.69:0/3826364554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f41017b0 msgr2=0x7f80f41a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f41017b0 0x7f80f41a0090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80e400e9f0 tx=0x7f80e400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 -- 172.21.15.69:0/3826364554 shutdown_connections 2024-03-19T01:15:32.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80f41077c0 0x7f80f419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80d80a2900 0x7f80d80a4dc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80f4102170 0x7f80f41a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 --2- 172.21.15.69:0/3826364554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80f41017b0 0x7f80f41a0090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 -- 172.21.15.69:0/3826364554 >> 172.21.15.69:0/3826364554 conn(0x7f80f40fd280 msgr2=0x7f80f4074b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 -- 172.21.15.69:0/3826364554 shutdown_connections 2024-03-19T01:15:32.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.226+0000 7f80f9a00640 1 -- 172.21.15.69:0/3826364554 wait complete. 2024-03-19T01:15:32.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:15:32.252 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:15:32.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:32 smithi069 ceph-mon[31727]: pgmap v148: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 922 KiB/s wr, 51 op/s 2024-03-19T01:15:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:32 smithi086 ceph-mon[66829]: pgmap v148: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 922 KiB/s wr, 51 op/s 2024-03-19T01:15:32.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.517+0000 7f03ce000640 1 -- 172.21.15.69:0/454523337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c8103170 msgr2=0x7f03c810b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.517+0000 7f03ce000640 1 --2- 172.21.15.69:0/454523337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c8103170 0x7f03c810b650 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009860 tx=0x7f03bc02bc10 comp rx=0 tx=0).stop 2024-03-19T01:15:32.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.517+0000 7f03ce000640 1 -- 172.21.15.69:0/454523337 shutdown_connections 2024-03-19T01:15:32.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.517+0000 7f03ce000640 1 --2- 172.21.15.69:0/454523337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c810bc70 0x7f03c810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.517+0000 7f03ce000640 1 --2- 172.21.15.69:0/454523337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c8103170 0x7f03c810b650 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.517+0000 7f03ce000640 1 --2- 172.21.15.69:0/454523337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c8102850 0x7f03c8102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.517+0000 7f03ce000640 1 -- 172.21.15.69:0/454523337 >> 172.21.15.69:0/454523337 conn(0x7f03c80fc070 msgr2=0x7f03c80fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.518+0000 7f03ce000640 1 -- 172.21.15.69:0/454523337 shutdown_connections 2024-03-19T01:15:32.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.518+0000 7f03ce000640 1 -- 172.21.15.69:0/454523337 wait complete. 2024-03-19T01:15:32.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.519+0000 7f03ce000640 1 Processor -- start 2024-03-19T01:15:32.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.520+0000 7f03ce000640 1 -- start start 2024-03-19T01:15:32.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.520+0000 7f03ce000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c8102850 0x7f03c810b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.520+0000 7f03ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c8103170 0x7f03c81085e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.520+0000 7f03ce000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c810bc70 0x7f03c8108b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.520+0000 7f03ce000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03c8113cb0 con 0x7f03c8102850 2024-03-19T01:15:32.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.520+0000 7f03ce000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03c8113b30 con 0x7f03c8103170 2024-03-19T01:15:32.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.520+0000 7f03ce000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f03c8113e30 con 0x7f03c810bc70 2024-03-19T01:15:32.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.521+0000 7f03cd600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c810bc70 0x7f03c8108b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.521+0000 7f03cd600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c810bc70 0x7f03c8108b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50168/0 (socket says 172.21.15.69:50168) 2024-03-19T01:15:32.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.521+0000 7f03cd600640 1 -- 172.21.15.69:0/1452080083 learned_addr learned my addr 172.21.15.69:0/1452080083 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:32.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.521+0000 7f03c7e00640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c8103170 0x7f03c81085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.521+0000 7f03cd600640 1 -- 172.21.15.69:0/1452080083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c8103170 msgr2=0x7f03c81085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.521+0000 7f03cd600640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c8103170 0x7f03c81085e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.521+0000 7f03cd600640 1 -- 172.21.15.69:0/1452080083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c8102850 msgr2=0x7f03c810b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.521+0000 7f03cd600640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c8102850 0x7f03c810b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.521+0000 7f03cd600640 1 -- 172.21.15.69:0/1452080083 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c8109420 con 0x7f03c810bc70 2024-03-19T01:15:32.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.522+0000 7f03cd600640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c810bc70 0x7f03c8108b60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f03c000cad0 tx=0x7f03c000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.522+0000 7f03c5600640 1 -- 172.21.15.69:0/1452080083 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c000ae20 con 0x7f03c810bc70 2024-03-19T01:15:32.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.522+0000 7f03ce000640 1 -- 172.21.15.69:0/1452080083 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c81aaa90 con 0x7f03c810bc70 2024-03-19T01:15:32.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.522+0000 7f03ce000640 1 -- 172.21.15.69:0/1452080083 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c81aafb0 con 0x7f03c810bc70 2024-03-19T01:15:32.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.522+0000 7f03c5600640 1 -- 172.21.15.69:0/1452080083 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c0005020 con 0x7f03c810bc70 2024-03-19T01:15:32.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.522+0000 7f03c5600640 1 -- 172.21.15.69:0/1452080083 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c000f6f0 con 0x7f03c810bc70 2024-03-19T01:15:32.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.527+0000 7f03d2417740 1 Processor -- start 2024-03-19T01:15:32.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.529+0000 7f03c5600640 1 -- 172.21.15.69:0/1452080083 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f03c000f890 con 0x7f03c810bc70 2024-03-19T01:15:32.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.529+0000 7f03d2417740 1 -- start start 2024-03-19T01:15:32.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.529+0000 7f03d2417740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc21310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.530+0000 7f03d2417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 0x5593fdc29ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.530+0000 7f03d2417740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc2a430 0x5593fdc2c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.530+0000 7f03d2417740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5593fdb852c0 con 0x5593fdc20f30 2024-03-19T01:15:32.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.530+0000 7f03d2417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593fdb2db60 con 0x5593fdc21850 2024-03-19T01:15:32.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.530+0000 7f03d2417740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5593fdb81360 con 0x5593fdc2a430 2024-03-19T01:15:32.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.530+0000 7f03ccc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 0x5593fdc29ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.530+0000 7f03c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc21310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.530+0000 7f03ccc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 0x5593fdc29ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51530/0 (socket says 172.21.15.69:51530) 2024-03-19T01:15:32.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.530+0000 7f03ccc00640 1 -- 172.21.15.69:0/3153267692 learned_addr learned my addr 172.21.15.69:0/3153267692 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:32.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.532+0000 7f03ccc00640 1 -- 172.21.15.69:0/3153267692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc2a430 msgr2=0x5593fdc2c820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:15:32.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.532+0000 7f03ccc00640 1 --2- 172.21.15.69:0/3153267692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc2a430 0x5593fdc2c820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.532+0000 7f03ccc00640 1 -- 172.21.15.69:0/3153267692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 msgr2=0x5593fdc21310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.533+0000 7f03ccc00640 1 --2- 172.21.15.69:0/3153267692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc21310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.533+0000 7f03ccc00640 1 -- 172.21.15.69:0/3153267692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593fd9637d0 con 0x5593fdc21850 2024-03-19T01:15:32.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.533+0000 7f03ccc00640 1 --2- 172.21.15.69:0/3153267692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 0x5593fdc29ef0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f03b400bb10 tx=0x7f03b400d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc81b90b7f59747 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.534+0000 7f03b9c00640 1 -- 172.21.15.69:0/3153267692 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b400dce0 con 0x5593fdc21850 2024-03-19T01:15:32.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.534+0000 7f03b9c00640 1 -- 172.21.15.69:0/3153267692 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b40035f0 con 0x5593fdc21850 2024-03-19T01:15:32.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.534+0000 7f03b9c00640 1 -- 172.21.15.69:0/3153267692 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b40106c0 con 0x5593fdc21850 2024-03-19T01:15:32.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.535+0000 7f03d2417740 1 -- 172.21.15.69:0/3153267692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 msgr2=0x5593fdc29ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.535+0000 7f03d2417740 1 --2- 172.21.15.69:0/3153267692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 0x5593fdc29ef0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f03b400bb10 tx=0x7f03b400d2e0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.535+0000 7f03d2417740 1 -- 172.21.15.69:0/3153267692 shutdown_connections 2024-03-19T01:15:32.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.535+0000 7f03d2417740 1 --2- 172.21.15.69:0/3153267692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc2a430 0x5593fdc2c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.535+0000 7f03d2417740 1 --2- 172.21.15.69:0/3153267692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 0x5593fdc29ef0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.535+0000 7f03d2417740 1 --2- 172.21.15.69:0/3153267692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc21310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.535+0000 7f03d2417740 1 -- 172.21.15.69:0/3153267692 >> 172.21.15.69:0/3153267692 conn(0x5593fdc1b660 msgr2=0x5593fdc1ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.535+0000 7f03d2417740 1 -- 172.21.15.69:0/3153267692 shutdown_connections 2024-03-19T01:15:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.535+0000 7f03d2417740 1 -- 172.21.15.69:0/3153267692 wait complete. 2024-03-19T01:15:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.536+0000 7f03d2417740 1 -- start start 2024-03-19T01:15:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.537+0000 7f03d2417740 1 Processor -- start 2024-03-19T01:15:32.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.538+0000 7f03c5600640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f03b00a2900 0x7f03b00a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.538+0000 7f03c5600640 1 -- 172.21.15.69:0/1452080083 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f03c00cf0a0 con 0x7f03c810bc70 2024-03-19T01:15:32.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.538+0000 7f03d2417740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc27410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.539+0000 7f03d2417740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc21850 0x5593fdc27950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.539+0000 7f03d2417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc2a430 0x5593fdc27e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.539+0000 7f03d2417740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5593fdc3e230 con 0x5593fdc20f30 2024-03-19T01:15:32.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.539+0000 7f03d2417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593fdb40fc0 con 0x5593fdc2a430 2024-03-19T01:15:32.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.539+0000 7f03d2417740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5593fdc3e3b0 con 0x5593fdc21850 2024-03-19T01:15:32.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.539+0000 7f03d2417740 10 client.0 _ll_register_callbacks cb 0x5593fdb40990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:32.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.539+0000 7f03cd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc2a430 0x5593fdc27e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.539+0000 7f03c7e00640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f03b00a2900 0x7f03b00a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.539+0000 7f03cd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc2a430 0x5593fdc27e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:51544/0 (socket says 172.21.15.69:51544) 2024-03-19T01:15:32.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.539+0000 7f03cd600640 1 -- 172.21.15.69:0/100435593 learned_addr learned my addr 172.21.15.69:0/100435593 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:32.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.540+0000 7f03cd600640 1 -- 172.21.15.69:0/100435593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc21850 msgr2=0x5593fdc27950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.540+0000 7f03cd600640 1 --2- 172.21.15.69:0/100435593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc21850 0x5593fdc27950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.540+0000 7f03cd600640 1 -- 172.21.15.69:0/100435593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 msgr2=0x5593fdc27410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:15:32.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.541+0000 7f03cd600640 1 --2- 172.21.15.69:0/100435593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc27410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.541+0000 7f03cd600640 1 -- 172.21.15.69:0/100435593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593fd8993f0 con 0x5593fdc2a430 2024-03-19T01:15:32.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.541+0000 7f03cd600640 1 --2- 172.21.15.69:0/100435593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc2a430 0x5593fdc27e90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f03c000caa0 tx=0x7f03c009f570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ddc90729fad0af0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.541+0000 7f03ae000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:15:32.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.541+0000 7f03ae000640 1 -- 172.21.15.69:0/100435593 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c001d040 con 0x5593fdc2a430 2024-03-19T01:15:32.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.541+0000 7f03ae000640 1 -- 172.21.15.69:0/100435593 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f03c009f120 con 0x5593fdc2a430 2024-03-19T01:15:32.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.542+0000 7f03c7e00640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f03b00a2900 0x7f03b00a4dc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f03bc02c220 tx=0x7f03bc0056d0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.542+0000 7f03d2417740 1 -- 172.21.15.69:0/100435593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b40035f0 con 0x5593fdc2a430 2024-03-19T01:15:32.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.542+0000 7f03ae000640 1 -- 172.21.15.69:0/100435593 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f03c000fb80 con 0x5593fdc2a430 2024-03-19T01:15:32.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.542+0000 7f03d2417740 10 client.26849 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:32.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.542+0000 7f03d2417740 1 -- 172.21.15.69:0/100435593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5593fd8590f0 con 0x5593fdc2a430 2024-03-19T01:15:32.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.543+0000 7f03ae000640 1 -- 172.21.15.69:0/100435593 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b40035f0 con 0x5593fdc2a430 2024-03-19T01:15:32.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.543+0000 7f03ae000640 1 -- 172.21.15.69:0/100435593 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f03c0097030 con 0x5593fdc2a430 2024-03-19T01:15:32.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.543+0000 7f03d2417740 10 client.26849 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.543+0000 7f03d2417740 10 client.26849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:15:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.543+0000 7f03d2417740 20 client.26849 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.543+0000 7f03d2417740 1 --2- 172.21.15.69:0/100435593 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5593fdc5c830 0x5593fdc5ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.543+0000 7f03d2417740 4 client.26849 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:15:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.543+0000 7f03d2417740 1 -- 172.21.15.69:0/100435593 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593fdb26610 con 0x5593fdc5c830 2024-03-19T01:15:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.544+0000 7f03ccc00640 1 --2- 172.21.15.69:0/100435593 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5593fdc5c830 0x5593fdc5ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.546+0000 7f03ae000640 10 client.26849 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:32.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.546+0000 7f03ccc00640 1 --2- 172.21.15.69:0/100435593 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5593fdc5c830 0x5593fdc5ec60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c5b55b36f60000 server_cookie=1bc6705212d70d7e in_seq=0 out_seq=0 2024-03-19T01:15:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.548+0000 7f03ae000640 1 -- 172.21.15.69:0/100435593 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5593fdb26610 con 0x5593fdc5c830 2024-03-19T01:15:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.548+0000 7f03ae000640 10 client.26849 handle_command_reply: tid=0 2024-03-19T01:15:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.548+0000 7f03d2417740 1 client.26849 shutdown 2024-03-19T01:15:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.549+0000 7f03d2417740 1 -- 172.21.15.69:0/100435593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc2a430 msgr2=0x5593fdc27e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.549+0000 7f03d2417740 1 --2- 172.21.15.69:0/100435593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc2a430 0x5593fdc27e90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f03c000caa0 tx=0x7f03c009f570 comp rx=0 tx=0).stop 2024-03-19T01:15:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.549+0000 7f03d2417740 1 -- 172.21.15.69:0/100435593 shutdown_connections 2024-03-19T01:15:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.549+0000 7f03d2417740 1 --2- 172.21.15.69:0/100435593 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5593fdc5c830 0x5593fdc5ec60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.549+0000 7f03d2417740 1 --2- 172.21.15.69:0/100435593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc2a430 0x5593fdc27e90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.549+0000 7f03d2417740 1 --2- 172.21.15.69:0/100435593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc21850 0x5593fdc27950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.549+0000 7f03d2417740 1 --2- 172.21.15.69:0/100435593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc27410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.549+0000 7f03d2417740 1 -- 172.21.15.69:0/100435593 >> 172.21.15.69:0/100435593 conn(0x5593fdc1b660 msgr2=0x5593fdc1d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.549+0000 7f03ae000640 0 client.26849 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:32.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.549+0000 7f03d2417740 1 -- 172.21.15.69:0/100435593 shutdown_connections 2024-03-19T01:15:32.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.549+0000 7f03d2417740 1 -- 172.21.15.69:0/100435593 wait complete. 2024-03-19T01:15:32.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.550+0000 7f03d2417740 20 client.26849 trim_cache size 0 max 16384 2024-03-19T01:15:32.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:32 smithi163 ceph-mon[67428]: pgmap v148: 129 pgs: 129 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 922 KiB/s wr, 51 op/s 2024-03-19T01:15:32.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.578+0000 7f03d2417740 1 Processor -- start 2024-03-19T01:15:32.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.579+0000 7f03d2417740 1 -- start start 2024-03-19T01:15:32.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.579+0000 7f03d2417740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc1b660 0x5593fdc29b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.579+0000 7f03d2417740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc47030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.579+0000 7f03d2417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 0x5593fdc1d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.579+0000 7f03d2417740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03a800ff40 con 0x5593fdc20f30 2024-03-19T01:15:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.579+0000 7f03d2417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03a8011c00 con 0x5593fdc21850 2024-03-19T01:15:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.579+0000 7f03d2417740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f03a8010610 con 0x5593fdc1b660 2024-03-19T01:15:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.579+0000 7f03ccc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc1b660 0x5593fdc29b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.579+0000 7f03ccc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc1b660 0x5593fdc29b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50210/0 (socket says 172.21.15.69:50210) 2024-03-19T01:15:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.579+0000 7f03ccc00640 1 -- 172.21.15.69:0/1783673327 learned_addr learned my addr 172.21.15.69:0/1783673327 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.580+0000 7f03c7e00640 1 --2- 172.21.15.69:0/1783673327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc47030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.580+0000 7f03ccc00640 1 -- 172.21.15.69:0/1783673327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 msgr2=0x5593fdc1d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.580+0000 7f03cd600640 1 --2- 172.21.15.69:0/1783673327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 0x5593fdc1d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.580+0000 7f03ccc00640 1 --2- 172.21.15.69:0/1783673327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 0x5593fdc1d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.580+0000 7f03ccc00640 1 -- 172.21.15.69:0/1783673327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 msgr2=0x5593fdc47030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.580+0000 7f03ccc00640 1 --2- 172.21.15.69:0/1783673327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc47030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.580+0000 7f03ccc00640 1 -- 172.21.15.69:0/1783673327 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593fdc25960 con 0x5593fdc1b660 2024-03-19T01:15:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.580+0000 7f03c7e00640 1 --2- 172.21.15.69:0/1783673327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc47030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:15:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.580+0000 7f03cd600640 1 --2- 172.21.15.69:0/1783673327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 0x5593fdc1d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:15:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.581+0000 7f03ccc00640 1 --2- 172.21.15.69:0/1783673327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc1b660 0x5593fdc29b40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f03b4005dc0 tx=0x7f03b4005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e93d4049c3b5078 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.581+0000 7f03b9200640 1 -- 172.21.15.69:0/1783673327 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b400dae0 con 0x5593fdc1b660 2024-03-19T01:15:32.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.581+0000 7f03b9200640 1 -- 172.21.15.69:0/1783673327 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b400dc80 con 0x5593fdc1b660 2024-03-19T01:15:32.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.581+0000 7f03b9200640 1 -- 172.21.15.69:0/1783673327 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b40041f0 con 0x5593fdc1b660 2024-03-19T01:15:32.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.582+0000 7f03d2417740 1 -- 172.21.15.69:0/1783673327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc1b660 msgr2=0x5593fdc29b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.582+0000 7f03d2417740 1 --2- 172.21.15.69:0/1783673327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc1b660 0x5593fdc29b40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f03b4005dc0 tx=0x7f03b4005d50 comp rx=0 tx=0).stop 2024-03-19T01:15:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.582+0000 7f03d2417740 1 -- 172.21.15.69:0/1783673327 shutdown_connections 2024-03-19T01:15:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.582+0000 7f03d2417740 1 --2- 172.21.15.69:0/1783673327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc21850 0x5593fdc1d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.582+0000 7f03d2417740 1 --2- 172.21.15.69:0/1783673327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc47030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.582+0000 7f03d2417740 1 --2- 172.21.15.69:0/1783673327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc1b660 0x5593fdc29b40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.582+0000 7f03d2417740 1 -- 172.21.15.69:0/1783673327 >> 172.21.15.69:0/1783673327 conn(0x5593fdc26050 msgr2=0x5593fdc1ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.582+0000 7f03d2417740 1 -- 172.21.15.69:0/1783673327 shutdown_connections 2024-03-19T01:15:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.582+0000 7f03d2417740 1 -- 172.21.15.69:0/1783673327 wait complete. 2024-03-19T01:15:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.583+0000 7f03d2417740 1 -- start start 2024-03-19T01:15:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.583+0000 7f03d2417740 1 Processor -- start 2024-03-19T01:15:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.584+0000 7f03d2417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc1b660 0x5593fdc43660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.584+0000 7f03d2417740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc40630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.584+0000 7f03d2417740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc21850 0x5593fdc40b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.584+0000 7f03d2417740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03a8011470 con 0x5593fdc20f30 2024-03-19T01:15:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.584+0000 7f03d2417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03a8010ce0 con 0x5593fdc1b660 2024-03-19T01:15:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.584+0000 7f03d2417740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f03a800df20 con 0x5593fdc21850 2024-03-19T01:15:32.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.584+0000 7f03d2417740 10 client.0 _ll_register_callbacks cb 0x7f03c009b350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:15:32.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.584+0000 7f03c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc40630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.584+0000 7f03cd600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc21850 0x5593fdc40b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.585+0000 7f03c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc40630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41774/0 (socket says 172.21.15.69:41774) 2024-03-19T01:15:32.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.585+0000 7f03c7e00640 1 -- 172.21.15.69:0/3493662369 learned_addr learned my addr 172.21.15.69:0/3493662369 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:15:32.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.585+0000 7f03ccc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc1b660 0x5593fdc43660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.585+0000 7f03c7e00640 1 -- 172.21.15.69:0/3493662369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc21850 msgr2=0x5593fdc40b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.585+0000 7f03c7e00640 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc21850 0x5593fdc40b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.585+0000 7f03c7e00640 1 -- 172.21.15.69:0/3493662369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc1b660 msgr2=0x5593fdc43660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:15:32.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.585+0000 7f03c7e00640 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc1b660 0x5593fdc43660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.585+0000 7f03c7e00640 1 -- 172.21.15.69:0/3493662369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593fdc43ca0 con 0x5593fdc20f30 2024-03-19T01:15:32.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.586+0000 7f03ccc00640 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc1b660 0x5593fdc43660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:15:32.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.586+0000 7f03c7e00640 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc40630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009250 tx=0x7f03bc02bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93ad569e63a5df4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:15:32.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.586+0000 7f03ae000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:15:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.587+0000 7f03ae000640 1 -- 172.21.15.69:0/3493662369 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc049040 con 0x5593fdc20f30 2024-03-19T01:15:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.587+0000 7f03ae000640 1 -- 172.21.15.69:0/3493662369 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f03bc02c870 con 0x5593fdc20f30 2024-03-19T01:15:32.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.587+0000 7f03d2417740 1 -- 172.21.15.69:0/3493662369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b400dc80 con 0x5593fdc20f30 2024-03-19T01:15:32.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.587+0000 7f03ae000640 1 -- 172.21.15.69:0/3493662369 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f03bc05b050 con 0x5593fdc20f30 2024-03-19T01:15:32.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.587+0000 7f03d2417740 10 client.16611 fetch_fsmap learned FSMap version 36 2024-03-19T01:15:32.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.588+0000 7f03d2417740 1 -- 172.21.15.69:0/3493662369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5593fdcc3560 con 0x5593fdc20f30 2024-03-19T01:15:32.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.589+0000 7f03ae000640 1 -- 172.21.15.69:0/3493662369 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b400dc80 con 0x5593fdc20f30 2024-03-19T01:15:32.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.589+0000 7f03ae000640 1 -- 172.21.15.69:0/3493662369 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 36) ==== 8410+0+0 (secure 0 0 0) 0x7f03bc03f040 con 0x5593fdc20f30 2024-03-19T01:15:32.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.589+0000 7f03d2417740 10 client.16611 fetch_fsmap finished waiting for FSMap version 36 2024-03-19T01:15:32.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.589+0000 7f03d2417740 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:15:32.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.589+0000 7f03d2417740 20 client.16611 populate_metadata read hostname 'smithi069' 2024-03-19T01:15:32.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.589+0000 7f03d2417740 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5593fdc2a430 0x5593fdcc5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:15:32.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.589+0000 7f03d2417740 4 client.16611 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:15:32.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.589+0000 7f03d2417740 1 -- 172.21.15.69:0/3493662369 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593fdb92800 con 0x5593fdc2a430 2024-03-19T01:15:32.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.589+0000 7f03ccc00640 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5593fdc2a430 0x5593fdcc5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:15:32.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.590+0000 7f03ccc00640 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5593fdc2a430 0x5593fdcc5e10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5251f99d3e6f1b11 server_cookie=8fbf9496966c55ff in_seq=0 out_seq=0 2024-03-19T01:15:32.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.591+0000 7f03ae000640 10 client.16611 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:32.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.595+0000 7f03ae000640 1 -- 172.21.15.69:0/3493662369 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5593fdb92800 con 0x5593fdc2a430 2024-03-19T01:15:32.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.595+0000 7f03ae000640 10 client.16611 handle_command_reply: tid=0 2024-03-19T01:15:32.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.595+0000 7f03d2417740 1 client.16611 shutdown 2024-03-19T01:15:32.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.595+0000 7f03d2417740 1 -- 172.21.15.69:0/3493662369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 msgr2=0x5593fdc40630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.595+0000 7f03d2417740 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc40630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009250 tx=0x7f03bc02bc10 comp rx=0 tx=0).stop 2024-03-19T01:15:32.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.595+0000 7f03d2417740 1 -- 172.21.15.69:0/3493662369 shutdown_connections 2024-03-19T01:15:32.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.595+0000 7f03d2417740 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5593fdc2a430 0x5593fdcc5e10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.595+0000 7f03d2417740 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5593fdc21850 0x5593fdc40b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.595+0000 7f03d2417740 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593fdc20f30 0x5593fdc40630 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.595+0000 7f03d2417740 1 --2- 172.21.15.69:0/3493662369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593fdc1b660 0x5593fdc43660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.595+0000 7f03d2417740 1 -- 172.21.15.69:0/3493662369 >> 172.21.15.69:0/3493662369 conn(0x5593fdc26050 msgr2=0x5593fdc1d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.596+0000 7f03ae000640 0 client.16611 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:15:32.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.596+0000 7f03d2417740 1 -- 172.21.15.69:0/3493662369 shutdown_connections 2024-03-19T01:15:32.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.596+0000 7f03d2417740 1 -- 172.21.15.69:0/3493662369 wait complete. 2024-03-19T01:15:32.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.597+0000 7f03d2417740 20 client.16611 trim_cache size 0 max 16384 2024-03-19T01:15:32.598 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:15:32.598 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (7265 inodes in the stack)", 2024-03-19T01:15:32.598 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:15:32.598 INFO:teuthology.orchestra.run.smithi069.stdout: "5eed2f48-f4ed-4b79-9053-8383f3a95ca5": { 2024-03-19T01:15:32.598 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:15:32.599 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "5eed2f48-f4ed-4b79-9053-8383f3a95ca5", 2024-03-19T01:15:32.599 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:15:32.599 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:15:32.599 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:15:32.599 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:15:32.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 -- 172.21.15.69:0/1452080083 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f03b00a2900 msgr2=0x7f03b00a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f03b00a2900 0x7f03b00a4dc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f03bc02c220 tx=0x7f03bc0056d0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 -- 172.21.15.69:0/1452080083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c810bc70 msgr2=0x7f03c8108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:15:32.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c810bc70 0x7f03c8108b60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f03c000cad0 tx=0x7f03c000cf90 comp rx=0 tx=0).stop 2024-03-19T01:15:32.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 -- 172.21.15.69:0/1452080083 shutdown_connections 2024-03-19T01:15:32.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c810bc70 0x7f03c8108b60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f03b00a2900 0x7f03b00a4dc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03c8103170 0x7f03c81085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 --2- 172.21.15.69:0/1452080083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c8102850 0x7f03c810b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:15:32.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 -- 172.21.15.69:0/1452080083 >> 172.21.15.69:0/1452080083 conn(0x7f03c80fc070 msgr2=0x7f03c80fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:15:32.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 -- 172.21.15.69:0/1452080083 shutdown_connections 2024-03-19T01:15:32.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:15:32.598+0000 7f03ce000640 1 -- 172.21.15.69:0/1452080083 wait complete. 2024-03-19T01:15:32.621 INFO:tasks.cephfs.filesystem:scrub status for tag:5eed2f48-f4ed-4b79-9053-8383f3a95ca5 - {'path': '/', 'tag': '5eed2f48-f4ed-4b79-9053-8383f3a95ca5', 'options': 'recursive,force'} 2024-03-19T01:15:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:15:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:33.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:15:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: pgmap v149: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 231 op/s 2024-03-19T01:15:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: pgmap v149: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 231 op/s 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: pgmap v149: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 231 op/s 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:35.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: pgmap v150: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 4.9 MiB/s wr, 251 op/s 2024-03-19T01:15:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:36.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: pgmap v150: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 4.9 MiB/s wr, 251 op/s 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:36.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: pgmap v150: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 4.9 MiB/s wr, 251 op/s 2024-03-19T01:15:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:36.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:36.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:37.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:37.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:37.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:37.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: pgmap v151: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.4 MiB/s wr, 318 op/s 2024-03-19T01:15:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:38.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: pgmap v151: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.4 MiB/s wr, 318 op/s 2024-03-19T01:15:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: pgmap v151: 129 pgs: 129 active+clean; 1.4 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.4 MiB/s wr, 318 op/s 2024-03-19T01:15:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:39 smithi086 ceph-mon[66829]: pgmap v152: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.4 MiB/s wr, 459 op/s 2024-03-19T01:15:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:39.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:39 smithi069 ceph-mon[31727]: pgmap v152: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.4 MiB/s wr, 459 op/s 2024-03-19T01:15:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:39 smithi163 ceph-mon[67428]: pgmap v152: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.4 MiB/s wr, 459 op/s 2024-03-19T01:15:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: pgmap v153: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.0 MiB/s wr, 441 op/s 2024-03-19T01:15:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: pgmap v153: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.0 MiB/s wr, 441 op/s 2024-03-19T01:15:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: pgmap v153: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.0 MiB/s wr, 441 op/s 2024-03-19T01:15:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:42.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:43.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:43 smithi086 ceph-mon[66829]: pgmap v154: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 457 op/s 2024-03-19T01:15:43.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:43 smithi069 ceph-mon[31727]: pgmap v154: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 457 op/s 2024-03-19T01:15:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:43 smithi163 ceph-mon[67428]: pgmap v154: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 457 op/s 2024-03-19T01:15:44.462 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:44.462 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:44.462 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:44.462 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:44.462 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:44.463 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:44.463 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:44.463 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:44.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:45.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:45 smithi086 ceph-mon[66829]: pgmap v155: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 284 op/s 2024-03-19T01:15:45.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:45 smithi069 ceph-mon[31727]: pgmap v155: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 284 op/s 2024-03-19T01:15:45.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:45.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:45.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:45 smithi163 ceph-mon[67428]: pgmap v155: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 284 op/s 2024-03-19T01:15:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: pgmap v156: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 279 op/s 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:47.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: pgmap v156: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 279 op/s 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: pgmap v156: 129 pgs: 129 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 279 op/s 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:47.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:47.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:48.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:48.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.853 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:49.853 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.853 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:49.853 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:15:49.853 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.853 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:49 smithi086 ceph-mon[66829]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 308 op/s 2024-03-19T01:15:49.853 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.853 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.854 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.854 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:49.854 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.854 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:49.854 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:15:49.854 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.854 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:49 smithi163 ceph-mon[67428]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 308 op/s 2024-03-19T01:15:49.854 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.855 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:49.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:49.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:15:49.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:49 smithi069 ceph-mon[31727]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 308 op/s 2024-03-19T01:15:49.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:49.855 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:15:51.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:51 smithi086 ceph-mon[66829]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-03-19T01:15:51.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:51 smithi069 ceph-mon[31727]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-03-19T01:15:51.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:51 smithi163 ceph-mon[67428]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-03-19T01:15:53.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:53 smithi086 ceph-mon[66829]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.5 MiB/s wr, 358 op/s 2024-03-19T01:15:53.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:53 smithi069 ceph-mon[31727]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.5 MiB/s wr, 358 op/s 2024-03-19T01:15:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:53 smithi163 ceph-mon[67428]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.5 MiB/s wr, 358 op/s 2024-03-19T01:15:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:55 smithi086 ceph-mon[66829]: pgmap v160: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.8 MiB/s wr, 390 op/s 2024-03-19T01:15:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:55 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:15:55.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:55 smithi069 ceph-mon[31727]: pgmap v160: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.8 MiB/s wr, 390 op/s 2024-03-19T01:15:55.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:55 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:15:55.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:55 smithi163 ceph-mon[67428]: pgmap v160: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.8 MiB/s wr, 390 op/s 2024-03-19T01:15:55.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:55 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:15:56.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:56 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:15:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:56 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:15:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:56 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:15:56.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:56 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:15:56.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:56 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:15:56.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:56 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:15:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:56 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:15:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:56 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:15:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:56 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:15:57.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:57 smithi086 ceph-mon[66829]: pgmap v161: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 393 op/s 2024-03-19T01:15:57.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:57 smithi069 ceph-mon[31727]: pgmap v161: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 393 op/s 2024-03-19T01:15:57.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:57 smithi163 ceph-mon[67428]: pgmap v161: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 393 op/s 2024-03-19T01:15:59.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:59 smithi086 ceph-mon[66829]: pgmap v162: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 364 op/s 2024-03-19T01:15:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:59 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:15:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:59 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:15:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:59 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:15:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:15:59 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:15:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:59 smithi069 ceph-mon[31727]: pgmap v162: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 364 op/s 2024-03-19T01:15:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:59 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:15:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:59 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:15:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:59 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:15:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:15:59 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:15:59.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:59 smithi163 ceph-mon[67428]: pgmap v162: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 364 op/s 2024-03-19T01:15:59.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:59 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:15:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:59 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:15:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:59 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:15:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:15:59 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:16:00.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:00 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:16:00.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:00 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:16:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:00 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:16:01.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:01 smithi086 ceph-mon[66829]: pgmap v163: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2024-03-19T01:16:01.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:01 smithi069 ceph-mon[31727]: pgmap v163: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2024-03-19T01:16:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:01 smithi163 ceph-mon[67428]: pgmap v163: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2024-03-19T01:16:02.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:02 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:02.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:02 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:02 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:02.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:16:02.622 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:16:02.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.839+0000 7f37eb200640 1 -- 172.21.15.69:0/358234031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37ec103170 msgr2=0x7f37ec10b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:02.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.839+0000 7f37eb200640 1 --2- 172.21.15.69:0/358234031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37ec103170 0x7f37ec10b650 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009900 tx=0x7f37d802bcd0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.839+0000 7f37eb200640 1 -- 172.21.15.69:0/358234031 shutdown_connections 2024-03-19T01:16:02.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.839+0000 7f37eb200640 1 --2- 172.21.15.69:0/358234031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec10bc70 0x7f37ec10e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.839+0000 7f37eb200640 1 --2- 172.21.15.69:0/358234031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37ec103170 0x7f37ec10b650 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.839+0000 7f37eb200640 1 --2- 172.21.15.69:0/358234031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec102850 0x7f37ec102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.839+0000 7f37eb200640 1 -- 172.21.15.69:0/358234031 >> 172.21.15.69:0/358234031 conn(0x7f37ec0fc070 msgr2=0x7f37ec0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:02.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.840+0000 7f37eb200640 1 -- 172.21.15.69:0/358234031 shutdown_connections 2024-03-19T01:16:02.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.840+0000 7f37eb200640 1 -- 172.21.15.69:0/358234031 wait complete. 2024-03-19T01:16:02.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37eb200640 1 Processor -- start 2024-03-19T01:16:02.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37eb200640 1 -- start start 2024-03-19T01:16:02.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37eb200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec102850 0x7f37ec19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37eb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec103170 0x7f37ec19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37eb200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37ec10bc70 0x7f37ec19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37eb200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec111ac0 con 0x7f37ec102850 2024-03-19T01:16:02.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37eb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec111940 con 0x7f37ec103170 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37eb200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec111c40 con 0x7f37ec10bc70 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37e9e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec102850 0x7f37ec19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37ec10bc70 0x7f37ec19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec103170 0x7f37ec19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec103170 0x7f37ec19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55248/0 (socket says 172.21.15.69:55248) 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37e9e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec102850 0x7f37ec19a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54082/0 (socket says 172.21.15.69:54082) 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37e9400640 1 -- 172.21.15.69:0/3943866641 learned_addr learned my addr 172.21.15.69:0/3943866641 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.841+0000 7f37ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37ec10bc70 0x7f37ec19ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60612/0 (socket says 172.21.15.69:60612) 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.842+0000 7f37e9400640 1 -- 172.21.15.69:0/3943866641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37ec10bc70 msgr2=0x7f37ec19ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.842+0000 7f37e9400640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37ec10bc70 0x7f37ec19ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.842+0000 7f37e9400640 1 -- 172.21.15.69:0/3943866641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec102850 msgr2=0x7f37ec19a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.842+0000 7f37e9400640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec102850 0x7f37ec19a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.842+0000 7f37e9400640 1 -- 172.21.15.69:0/3943866641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec19f160 con 0x7f37ec103170 2024-03-19T01:16:02.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.842+0000 7f37e9400640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec103170 0x7f37ec19a6b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f37d8000c00 tx=0x7f37d80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:02.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.842+0000 7f37e2a00640 1 -- 172.21.15.69:0/3943866641 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8042020 con 0x7f37ec103170 2024-03-19T01:16:02.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.842+0000 7f37e2a00640 1 -- 172.21.15.69:0/3943866641 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d80054e0 con 0x7f37ec103170 2024-03-19T01:16:02.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.842+0000 7f37eb200640 1 -- 172.21.15.69:0/3943866641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec19f3f0 con 0x7f37ec103170 2024-03-19T01:16:02.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.843+0000 7f37e2a00640 1 -- 172.21.15.69:0/3943866641 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8003a80 con 0x7f37ec103170 2024-03-19T01:16:02.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.843+0000 7f37eb200640 1 -- 172.21.15.69:0/3943866641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec1a6ce0 con 0x7f37ec103170 2024-03-19T01:16:02.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.846+0000 7f37e2a00640 1 -- 172.21.15.69:0/3943866641 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f37d8045020 con 0x7f37ec103170 2024-03-19T01:16:02.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.846+0000 7f37f3e19740 1 Processor -- start 2024-03-19T01:16:02.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.846+0000 7f37f3e19740 1 -- start start 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f15b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15b850 0x55df0f163ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f164430 0x55df0f166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37f3e19740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df0f0bf2c0 con 0x55df0f164430 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37f3e19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df0f067b60 con 0x55df0f15b850 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37f3e19740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55df0f0bb360 con 0x55df0f15af30 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f15b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60614/0 (socket says 172.21.15.69:60614) 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37ea800640 1 -- 172.21.15.69:0/3097672915 learned_addr learned my addr 172.21.15.69:0/3097672915 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37e9400640 1 --2- 172.21.15.69:0/3097672915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f164430 0x55df0f166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37ea800640 1 -- 172.21.15.69:0/3097672915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15b850 msgr2=0x55df0f163ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:16:02.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37ea800640 1 --2- 172.21.15.69:0/3097672915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15b850 0x55df0f163ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37ea800640 1 -- 172.21.15.69:0/3097672915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f164430 msgr2=0x55df0f166820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:02.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37ea800640 1 --2- 172.21.15.69:0/3097672915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f164430 0x55df0f166820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.847+0000 7f37ea800640 1 -- 172.21.15.69:0/3097672915 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0ee9d7d0 con 0x55df0f15af30 2024-03-19T01:16:02.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.848+0000 7f37ea800640 1 --2- 172.21.15.69:0/3097672915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f15b310 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009f50 tx=0x7f37e40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=463c97c5cf2f3e3b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:02.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.848+0000 7f37e2a00640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37d00a29f0 0x7f37d00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.848+0000 7f37d7400640 1 -- 172.21.15.69:0/3097672915 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4004b20 con 0x55df0f15af30 2024-03-19T01:16:02.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.848+0000 7f37d7400640 1 -- 172.21.15.69:0/3097672915 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e4004cc0 con 0x55df0f15af30 2024-03-19T01:16:02.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.848+0000 7f37d7400640 1 -- 172.21.15.69:0/3097672915 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4005450 con 0x55df0f15af30 2024-03-19T01:16:02.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.848+0000 7f37e2a00640 1 -- 172.21.15.69:0/3943866641 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f37d80f0bd0 con 0x7f37ec103170 2024-03-19T01:16:02.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.849+0000 7f37e9e00640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37d00a29f0 0x7f37d00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.849+0000 7f37f3e19740 1 -- 172.21.15.69:0/3097672915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 msgr2=0x55df0f15b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:02.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.849+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3097672915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f15b310 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009f50 tx=0x7f37e40118d0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.849+0000 7f37f3e19740 1 -- 172.21.15.69:0/3097672915 shutdown_connections 2024-03-19T01:16:02.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.849+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3097672915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f164430 0x55df0f166820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.849+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3097672915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15b850 0x55df0f163ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.849+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3097672915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f15b310 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.849+0000 7f37f3e19740 1 -- 172.21.15.69:0/3097672915 >> 172.21.15.69:0/3097672915 conn(0x55df0f155660 msgr2=0x55df0f155a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:02.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.849+0000 7f37f3e19740 1 -- 172.21.15.69:0/3097672915 shutdown_connections 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.850+0000 7f37f3e19740 1 -- 172.21.15.69:0/3097672915 wait complete. 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.850+0000 7f37e9e00640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37d00a29f0 0x7f37d00a4eb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0095f0 tx=0x7f37dc005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.850+0000 7f37f3e19740 1 -- start start 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.850+0000 7f37f3e19740 1 Processor -- start 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15b850 0x55df0f161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f164430 0x55df0f161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37f3e19740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df0f167220 con 0x55df0f164430 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37f3e19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df0f07afc0 con 0x55df0f15b850 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37f3e19740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55df0f1673a0 con 0x55df0f15af30 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37f3e19740 10 client.0 _ll_register_callbacks cb 0x55df0f07a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37e9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37e9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60624/0 (socket says 172.21.15.69:60624) 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37e9e00640 1 -- 172.21.15.69:0/3134834024 learned_addr learned my addr 172.21.15.69:0/3134834024 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37e9400640 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15b850 0x55df0f161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37ea800640 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f164430 0x55df0f161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37e9e00640 1 -- 172.21.15.69:0/3134834024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15b850 msgr2=0x55df0f161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37e9e00640 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15b850 0x55df0f161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.851+0000 7f37e9e00640 1 -- 172.21.15.69:0/3134834024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f164430 msgr2=0x55df0f161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.852+0000 7f37e9e00640 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f164430 0x55df0f161ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.852+0000 7f37e9e00640 1 -- 172.21.15.69:0/3134834024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0edd33f0 con 0x55df0f15af30 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.852+0000 7f37ea800640 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f164430 0x55df0f161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.852+0000 7f37e9e00640 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f161440 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00ebc0 tx=0x7f37dc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a92d6957241fbb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.852+0000 7f37cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.852+0000 7f37cbe00640 1 -- 172.21.15.69:0/3134834024 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc026630 con 0x55df0f15af30 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.852+0000 7f37cbe00640 1 -- 172.21.15.69:0/3134834024 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f37dc01d410 con 0x55df0f15af30 2024-03-19T01:16:02.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.852+0000 7f37f3e19740 1 -- 172.21.15.69:0/3134834024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e4004cc0 con 0x55df0f15af30 2024-03-19T01:16:02.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.852+0000 7f37cbe00640 1 -- 172.21.15.69:0/3134834024 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f37dc0267d0 con 0x55df0f15af30 2024-03-19T01:16:02.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.852+0000 7f37f3e19740 10 client.26821 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:02.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.852+0000 7f37f3e19740 1 -- 172.21.15.69:0/3134834024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55df0ed930f0 con 0x55df0f15af30 2024-03-19T01:16:02.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.853+0000 7f37cbe00640 1 -- 172.21.15.69:0/3134834024 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e4004cc0 con 0x55df0f15af30 2024-03-19T01:16:02.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.853+0000 7f37cbe00640 1 -- 172.21.15.69:0/3134834024 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7f37dc0253d0 con 0x55df0f15af30 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.853+0000 7f37f3e19740 10 client.26821 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.853+0000 7f37f3e19740 10 client.26821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.853+0000 7f37f3e19740 20 client.26821 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.853+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55df0f174910 0x55df0f176db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.853+0000 7f37f3e19740 4 client.26821 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.853+0000 7f37f3e19740 1 -- 172.21.15.69:0/3134834024 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df0f060610 con 0x55df0f174910 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.853+0000 7f37e9400640 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55df0f174910 0x55df0f176db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.854+0000 7f37cbe00640 10 client.26821 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.854+0000 7f37e9400640 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55df0f174910 0x55df0f176db0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f91b71c90dfe97 server_cookie=f7f063fa2ea9e65f in_seq=0 out_seq=0 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37cbe00640 1 -- 172.21.15.69:0/3134834024 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55df0f060610 con 0x55df0f174910 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37cbe00640 10 client.26821 handle_command_reply: tid=0 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37f3e19740 1 client.26821 shutdown 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37f3e19740 1 -- 172.21.15.69:0/3134834024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 msgr2=0x55df0f161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:02.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f161440 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00ebc0 tx=0x7f37dc004730 comp rx=0 tx=0).stop 2024-03-19T01:16:02.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37f3e19740 1 -- 172.21.15.69:0/3134834024 shutdown_connections 2024-03-19T01:16:02.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55df0f174910 0x55df0f176db0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f164430 0x55df0f161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15b850 0x55df0f161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37cbe00640 0 client.26821 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:02.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3134834024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f15af30 0x55df0f161440 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37f3e19740 1 -- 172.21.15.69:0/3134834024 >> 172.21.15.69:0/3134834024 conn(0x55df0f155660 msgr2=0x55df0f1573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:02.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.855+0000 7f37f3e19740 1 -- 172.21.15.69:0/3134834024 shutdown_connections 2024-03-19T01:16:02.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.856+0000 7f37f3e19740 1 -- 172.21.15.69:0/3134834024 wait complete. 2024-03-19T01:16:02.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.856+0000 7f37f3e19740 20 client.26821 trim_cache size 0 max 16384 2024-03-19T01:16:02.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.874+0000 7f37f3e19740 1 Processor -- start 2024-03-19T01:16:02.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.874+0000 7f37f3e19740 1 -- start start 2024-03-19T01:16:02.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f163b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f15b850 0x55df0f16fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f164430 0x55df0f1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37f3e19740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b0d60 con 0x55df0f15b850 2024-03-19T01:16:02.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37f3e19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b2a20 con 0x55df0f15af30 2024-03-19T01:16:02.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37f3e19740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b1430 con 0x55df0f164430 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f163b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55276/0 (socket says 172.21.15.69:55276) 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37e9e00640 1 -- 172.21.15.69:0/1428524717 learned_addr learned my addr 172.21.15.69:0/1428524717 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37e9400640 1 --2- 172.21.15.69:0/1428524717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f15b850 0x55df0f16fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37ea800640 1 --2- 172.21.15.69:0/1428524717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f164430 0x55df0f1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37e9e00640 1 -- 172.21.15.69:0/1428524717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f164430 msgr2=0x55df0f1571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37e9e00640 1 --2- 172.21.15.69:0/1428524717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f164430 0x55df0f1571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37e9e00640 1 -- 172.21.15.69:0/1428524717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f15b850 msgr2=0x55df0f16fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37e9e00640 1 --2- 172.21.15.69:0/1428524717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f15b850 0x55df0f16fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37e9e00640 1 -- 172.21.15.69:0/1428524717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0f15f990 con 0x55df0f15af30 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.875+0000 7f37e9400640 1 --2- 172.21.15.69:0/1428524717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f15b850 0x55df0f16fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:16:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.876+0000 7f37e9e00640 1 --2- 172.21.15.69:0/1428524717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f163b70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f37dc011d10 tx=0x7f37dc009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fe577d3e4a6e41f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:02.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.876+0000 7f37d6a00640 1 -- 172.21.15.69:0/1428524717 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc038070 con 0x55df0f15af30 2024-03-19T01:16:02.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.876+0000 7f37d6a00640 1 -- 172.21.15.69:0/1428524717 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37dc026430 con 0x55df0f15af30 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.876+0000 7f37d6a00640 1 -- 172.21.15.69:0/1428524717 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc026d70 con 0x55df0f15af30 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.876+0000 7f37f3e19740 1 -- 172.21.15.69:0/1428524717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 msgr2=0x55df0f163b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.876+0000 7f37f3e19740 1 --2- 172.21.15.69:0/1428524717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f163b70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f37dc011d10 tx=0x7f37dc009960 comp rx=0 tx=0).stop 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.877+0000 7f37f3e19740 1 -- 172.21.15.69:0/1428524717 shutdown_connections 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.877+0000 7f37f3e19740 1 --2- 172.21.15.69:0/1428524717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f164430 0x55df0f1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.877+0000 7f37f3e19740 1 --2- 172.21.15.69:0/1428524717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f15b850 0x55df0f16fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.877+0000 7f37f3e19740 1 --2- 172.21.15.69:0/1428524717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f163b70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.877+0000 7f37f3e19740 1 -- 172.21.15.69:0/1428524717 >> 172.21.15.69:0/1428524717 conn(0x55df0f155660 msgr2=0x55df0f156a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.877+0000 7f37f3e19740 1 -- 172.21.15.69:0/1428524717 shutdown_connections 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.877+0000 7f37f3e19740 1 -- 172.21.15.69:0/1428524717 wait complete. 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.877+0000 7f37f3e19740 1 -- start start 2024-03-19T01:16:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.877+0000 7f37f3e19740 1 Processor -- start 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f167f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f15b850 0x55df0f1684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37f3e19740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f164430 0x55df0f168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37f3e19740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b2290 con 0x55df0f15b850 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37f3e19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b1b00 con 0x55df0f15af30 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37f3e19740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1aed10 con 0x55df0f164430 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37f3e19740 10 client.0 _ll_register_callbacks cb 0x7f37dc00e840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f167f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f167f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55282/0 (socket says 172.21.15.69:55282) 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37e9e00640 1 -- 172.21.15.69:0/3309579434 learned_addr learned my addr 172.21.15.69:0/3309579434 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f164430 0x55df0f168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37e9400640 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f15b850 0x55df0f1684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37e9e00640 1 -- 172.21.15.69:0/3309579434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f164430 msgr2=0x55df0f168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37e9e00640 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f164430 0x55df0f168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37e9e00640 1 -- 172.21.15.69:0/3309579434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f15b850 msgr2=0x55df0f1684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37e9e00640 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f15b850 0x55df0f1684c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.878+0000 7f37e9e00640 1 -- 172.21.15.69:0/3309579434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0f1d9d40 con 0x55df0f15af30 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.879+0000 7f37e9e00640 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f167f80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f37dc025630 tx=0x7f37dc025660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc4c9e91234d18e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.879+0000 7f37cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.879+0000 7f37cbe00640 1 -- 172.21.15.69:0/3309579434 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc038040 con 0x55df0f15af30 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.879+0000 7f37f3e19740 1 -- 172.21.15.69:0/3309579434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37dc026430 con 0x55df0f15af30 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.879+0000 7f37cbe00640 1 -- 172.21.15.69:0/3309579434 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f37dc027ca0 con 0x55df0f15af30 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.879+0000 7f37cbe00640 1 -- 172.21.15.69:0/3309579434 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f37dc050050 con 0x55df0f15af30 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.879+0000 7f37f3e19740 10 client.26867 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.879+0000 7f37f3e19740 1 -- 172.21.15.69:0/3309579434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55df0f1db050 con 0x55df0f15af30 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.879+0000 7f37cbe00640 1 -- 172.21.15.69:0/3309579434 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37dc026430 con 0x55df0f15af30 2024-03-19T01:16:02.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.879+0000 7f37cbe00640 1 -- 172.21.15.69:0/3309579434 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7f37dc004700 con 0x55df0f15af30 2024-03-19T01:16:02.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.880+0000 7f37f3e19740 10 client.26867 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:02.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.880+0000 7f37f3e19740 10 client.26867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:16:02.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.880+0000 7f37f3e19740 20 client.26867 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:02.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.880+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55df0f1db720 0x55df0f1ddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:02.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.880+0000 7f37f3e19740 4 client.26867 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:16:02.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.880+0000 7f37f3e19740 1 -- 172.21.15.69:0/3309579434 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df0f0cc800 con 0x55df0f1db720 2024-03-19T01:16:02.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.883+0000 7f37e9400640 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55df0f1db720 0x55df0f1ddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:02.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37e9400640 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55df0f1db720 0x55df0f1ddb40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f56f5446ece0ad server_cookie=654cd77c9a7cafa4 in_seq=0 out_seq=0 2024-03-19T01:16:02.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37cbe00640 10 client.26867 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:02.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37cbe00640 1 -- 172.21.15.69:0/3309579434 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df0f0cc800 con 0x55df0f1db720 2024-03-19T01:16:02.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37cbe00640 10 client.26867 handle_command_reply: tid=0 2024-03-19T01:16:02.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37f3e19740 1 client.26867 shutdown 2024-03-19T01:16:02.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37f3e19740 1 -- 172.21.15.69:0/3309579434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 msgr2=0x55df0f167f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:02.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f167f80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f37dc025630 tx=0x7f37dc025660 comp rx=0 tx=0).stop 2024-03-19T01:16:02.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37f3e19740 1 -- 172.21.15.69:0/3309579434 shutdown_connections 2024-03-19T01:16:02.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55df0f1db720 0x55df0f1ddb40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55df0f164430 0x55df0f168a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df0f15b850 0x55df0f1684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37f3e19740 1 --2- 172.21.15.69:0/3309579434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0f15af30 0x55df0f167f80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37cbe00640 0 client.26867 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:02.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.884+0000 7f37f3e19740 1 -- 172.21.15.69:0/3309579434 >> 172.21.15.69:0/3309579434 conn(0x55df0f155660 msgr2=0x55df0f1546e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:02.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37f3e19740 1 -- 172.21.15.69:0/3309579434 shutdown_connections 2024-03-19T01:16:02.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37f3e19740 1 -- 172.21.15.69:0/3309579434 wait complete. 2024-03-19T01:16:02.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37f3e19740 20 client.26867 trim_cache size 0 max 16384 2024-03-19T01:16:02.888 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:16:02.888 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:16:02.889 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:16:02.889 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:16:02.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37eb200640 1 -- 172.21.15.69:0/3943866641 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37d00a29f0 msgr2=0x7f37d00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:02.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37eb200640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37d00a29f0 0x7f37d00a4eb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0095f0 tx=0x7f37dc005f90 comp rx=0 tx=0).stop 2024-03-19T01:16:02.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37eb200640 1 -- 172.21.15.69:0/3943866641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec103170 msgr2=0x7f37ec19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:02.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37eb200640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec103170 0x7f37ec19a6b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f37d8000c00 tx=0x7f37d80042f0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37eb200640 1 -- 172.21.15.69:0/3943866641 shutdown_connections 2024-03-19T01:16:02.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37eb200640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37ec10bc70 0x7f37ec19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37eb200640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37d00a29f0 0x7f37d00a4eb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37eb200640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec103170 0x7f37ec19a6b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37eb200640 1 --2- 172.21.15.69:0/3943866641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec102850 0x7f37ec19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:02.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.885+0000 7f37eb200640 1 -- 172.21.15.69:0/3943866641 >> 172.21.15.69:0/3943866641 conn(0x7f37ec0fc070 msgr2=0x7f37ec0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:02.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.886+0000 7f37eb200640 1 -- 172.21.15.69:0/3943866641 shutdown_connections 2024-03-19T01:16:02.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:02.886+0000 7f37eb200640 1 -- 172.21.15.69:0/3943866641 wait complete. 2024-03-19T01:16:02.906 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:16:02.906 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:16:03.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.114+0000 7fea19e00640 1 -- 172.21.15.69:0/2622542656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea14102a50 msgr2=0x7fea14102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.114+0000 7fea19e00640 1 --2- 172.21.15.69:0/2622542656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea14102a50 0x7fea14102e30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fea04009900 tx=0x7fea0402bcd0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.114+0000 7fea19e00640 1 -- 172.21.15.69:0/2622542656 shutdown_connections 2024-03-19T01:16:03.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.114+0000 7fea19e00640 1 --2- 172.21.15.69:0/2622542656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea1410be20 0x7fea1410e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.114+0000 7fea19e00640 1 --2- 172.21.15.69:0/2622542656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea14103370 0x7fea1410b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.114+0000 7fea19e00640 1 --2- 172.21.15.69:0/2622542656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea14102a50 0x7fea14102e30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.114+0000 7fea19e00640 1 -- 172.21.15.69:0/2622542656 >> 172.21.15.69:0/2622542656 conn(0x7fea14076aa0 msgr2=0x7fea14076eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.114+0000 7fea19e00640 1 -- 172.21.15.69:0/2622542656 shutdown_connections 2024-03-19T01:16:03.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.114+0000 7fea19e00640 1 -- 172.21.15.69:0/2622542656 wait complete. 2024-03-19T01:16:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.115+0000 7fea19e00640 1 Processor -- start 2024-03-19T01:16:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.115+0000 7fea19e00640 1 -- start start 2024-03-19T01:16:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.115+0000 7fea19e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea14102a50 0x7fea141980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea19e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea14103370 0x7fea14198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea19e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1410be20 0x7fea1419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea19e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea14111bb0 con 0x7fea1410be20 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea19e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea14111a30 con 0x7fea14102a50 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea19e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea14111d30 con 0x7fea14103370 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea12000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea14103370 0x7fea14198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea12a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea14102a50 0x7fea141980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea13400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1410be20 0x7fea1419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea12000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea14103370 0x7fea14198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60676/0 (socket says 172.21.15.69:60676) 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea12a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea14102a50 0x7fea141980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55286/0 (socket says 172.21.15.69:55286) 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea12000640 1 -- 172.21.15.69:0/98542759 learned_addr learned my addr 172.21.15.69:0/98542759 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea12000640 1 -- 172.21.15.69:0/98542759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea14102a50 msgr2=0x7fea141980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea12000640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea14102a50 0x7fea141980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea12000640 1 -- 172.21.15.69:0/98542759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1410be20 msgr2=0x7fea1419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea12000640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1410be20 0x7fea1419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea12000640 1 -- 172.21.15.69:0/98542759 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1419d0e0 con 0x7fea14103370 2024-03-19T01:16:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.116+0000 7fea12a00640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea14102a50 0x7fea141980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.117+0000 7fea12000640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea14103370 0x7fea14198630 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea0000e9f0 tx=0x7fea0000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.117+0000 7fea18a00640 1 -- 172.21.15.69:0/98542759 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0000cd80 con 0x7fea14103370 2024-03-19T01:16:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.117+0000 7fea18a00640 1 -- 172.21.15.69:0/98542759 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea00010910 con 0x7fea14103370 2024-03-19T01:16:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.117+0000 7fea18a00640 1 -- 172.21.15.69:0/98542759 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00018990 con 0x7fea14103370 2024-03-19T01:16:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.117+0000 7fea19e00640 1 -- 172.21.15.69:0/98542759 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1419d3d0 con 0x7fea14103370 2024-03-19T01:16:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.117+0000 7fea19e00640 1 -- 172.21.15.69:0/98542759 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea141a4c60 con 0x7fea14103370 2024-03-19T01:16:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.118+0000 7fea19e00640 1 -- 172.21.15.69:0/98542759 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea1406a670 con 0x7fea14103370 2024-03-19T01:16:03.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.120+0000 7fea18a00640 1 -- 172.21.15.69:0/98542759 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fea00010ac0 con 0x7fea14103370 2024-03-19T01:16:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.122+0000 7fea18a00640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9f80a2970 0x7fe9f80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.122+0000 7fea18a00640 1 -- 172.21.15.69:0/98542759 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fea000cf560 con 0x7fea14103370 2024-03-19T01:16:03.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.122+0000 7fea12a00640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9f80a2970 0x7fe9f80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.123+0000 7fea12a00640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9f80a2970 0x7fe9f80a4e30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea0402efd0 tx=0x7fea0402cf90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.125+0000 7fea18a00640 1 -- 172.21.15.69:0/98542759 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fea0009b8b0 con 0x7fea14103370 2024-03-19T01:16:03.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:03 smithi069 ceph-mon[31727]: pgmap v164: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 292 op/s 2024-03-19T01:16:03.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.411+0000 7fea19e00640 1 -- 172.21.15.69:0/98542759 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea14199440 con 0x7fea14103370 2024-03-19T01:16:03.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.412+0000 7fea18a00640 1 -- 172.21.15.69:0/98542759 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11952 (secure 0 0 0) 0x7fea0009f950 con 0x7fea14103370 2024-03-19T01:16:03.413 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:16:03.413 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:16:01.082720+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24550,24494,24580,24581,24476]},"id":1}]} 2024-03-19T01:16:03.413 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 37 2024-03-19T01:16:03.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 -- 172.21.15.69:0/98542759 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9f80a2970 msgr2=0x7fe9f80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9f80a2970 0x7fe9f80a4e30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea0402efd0 tx=0x7fea0402cf90 comp rx=0 tx=0).stop 2024-03-19T01:16:03.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 -- 172.21.15.69:0/98542759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea14103370 msgr2=0x7fea14198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea14103370 0x7fea14198630 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea0000e9f0 tx=0x7fea0000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 -- 172.21.15.69:0/98542759 shutdown_connections 2024-03-19T01:16:03.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1410be20 0x7fea1419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea14103370 0x7fea14198630 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9f80a2970 0x7fe9f80a4e30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 --2- 172.21.15.69:0/98542759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea14102a50 0x7fea141980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 -- 172.21.15.69:0/98542759 >> 172.21.15.69:0/98542759 conn(0x7fea14076aa0 msgr2=0x7fea140fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 -- 172.21.15.69:0/98542759 shutdown_connections 2024-03-19T01:16:03.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.415+0000 7fea19e00640 1 -- 172.21.15.69:0/98542759 wait complete. 2024-03-19T01:16:03.433 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:16:03.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:03 smithi086 ceph-mon[66829]: pgmap v164: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 292 op/s 2024-03-19T01:16:03.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:03 smithi163 ceph-mon[67428]: pgmap v164: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 292 op/s 2024-03-19T01:16:03.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.650+0000 7f1248c00640 1 -- 172.21.15.69:0/977880910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1244101680 msgr2=0x7f1244101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.650+0000 7f1248c00640 1 --2- 172.21.15.69:0/977880910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1244101680 0x7f1244101b00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1230009900 tx=0x7f123002bc50 comp rx=0 tx=0).stop 2024-03-19T01:16:03.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.650+0000 7f1248c00640 1 -- 172.21.15.69:0/977880910 shutdown_connections 2024-03-19T01:16:03.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.650+0000 7f1248c00640 1 --2- 172.21.15.69:0/977880910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244102040 0x7f124410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.650+0000 7f1248c00640 1 --2- 172.21.15.69:0/977880910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1244101680 0x7f1244101b00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.650+0000 7f1248c00640 1 --2- 172.21.15.69:0/977880910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1244107690 0x7f1244107a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.650+0000 7f1248c00640 1 -- 172.21.15.69:0/977880910 >> 172.21.15.69:0/977880910 conn(0x7f12440fd190 msgr2=0x7f12440ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.650+0000 7f1248c00640 1 -- 172.21.15.69:0/977880910 shutdown_connections 2024-03-19T01:16:03.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.650+0000 7f1248c00640 1 -- 172.21.15.69:0/977880910 wait complete. 2024-03-19T01:16:03.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.651+0000 7f1248c00640 1 Processor -- start 2024-03-19T01:16:03.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1248c00640 1 -- start start 2024-03-19T01:16:03.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1248c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244101680 0x7f1244197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1248c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1244102040 0x7f12441984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1248c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1244107690 0x7f124419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1248c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1244111ab0 con 0x7f1244102040 2024-03-19T01:16:03.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1248c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1244111930 con 0x7f1244107690 2024-03-19T01:16:03.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1248c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1244111c30 con 0x7f1244101680 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1243400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244101680 0x7f1244197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1243400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244101680 0x7f1244197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60696/0 (socket says 172.21.15.69:60696) 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1243400640 1 -- 172.21.15.69:0/4237605013 learned_addr learned my addr 172.21.15.69:0/4237605013 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1243e00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1244107690 0x7f124419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.652+0000 7f1242a00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1244102040 0x7f12441984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.653+0000 7f1243400640 1 -- 172.21.15.69:0/4237605013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1244107690 msgr2=0x7f124419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.653+0000 7f1243400640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1244107690 0x7f124419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.653+0000 7f1243400640 1 -- 172.21.15.69:0/4237605013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1244102040 msgr2=0x7f12441984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.653+0000 7f1243400640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1244102040 0x7f12441984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.653+0000 7f1243400640 1 -- 172.21.15.69:0/4237605013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124419cf80 con 0x7f1244101680 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.653+0000 7f1243e00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1244107690 0x7f124419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.653+0000 7f1243400640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244101680 0x7f1244197f90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f123400ea60 tx=0x7f123400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.653+0000 7f123be00640 1 -- 172.21.15.69:0/4237605013 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123400cde0 con 0x7f1244101680 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.653+0000 7f1248c00640 1 -- 172.21.15.69:0/4237605013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124419d270 con 0x7f1244101680 2024-03-19T01:16:03.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.653+0000 7f123be00640 1 -- 172.21.15.69:0/4237605013 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12340035f0 con 0x7f1244101680 2024-03-19T01:16:03.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.653+0000 7f123be00640 1 -- 172.21.15.69:0/4237605013 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234010670 con 0x7f1244101680 2024-03-19T01:16:03.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.654+0000 7f1248c00640 1 -- 172.21.15.69:0/4237605013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12441a4b00 con 0x7f1244101680 2024-03-19T01:16:03.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f124cfd2740 1 Processor -- start 2024-03-19T01:16:03.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f124cfd2740 1 -- start start 2024-03-19T01:16:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d245d0 0x55a8e5d249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d24f80 0x55a8e5d31fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d32500 0x55a8e5d348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f124cfd2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8e5c89380 con 0x55a8e5d24f80 2024-03-19T01:16:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f124cfd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8e5c31d20 con 0x55a8e5d32500 2024-03-19T01:16:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f124cfd2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a8e5c85420 con 0x55a8e5d245d0 2024-03-19T01:16:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f1243e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d245d0 0x55a8e5d249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f1243e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d245d0 0x55a8e5d249b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60710/0 (socket says 172.21.15.69:60710) 2024-03-19T01:16:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f1243e00640 1 -- 172.21.15.69:0/81159412 learned_addr learned my addr 172.21.15.69:0/81159412 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.657+0000 7f1242a00640 1 --2- 172.21.15.69:0/81159412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d32500 0x55a8e5d348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.658+0000 7f1243e00640 1 -- 172.21.15.69:0/81159412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d32500 msgr2=0x55a8e5d348f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.658+0000 7f1243e00640 1 --2- 172.21.15.69:0/81159412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d32500 0x55a8e5d348f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.658+0000 7f1243e00640 1 -- 172.21.15.69:0/81159412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d24f80 msgr2=0x55a8e5d31fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:16:03.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.658+0000 7f1243e00640 1 --2- 172.21.15.69:0/81159412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d24f80 0x55a8e5d31fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.658+0000 7f1243e00640 1 -- 172.21.15.69:0/81159412 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e599d3f0 con 0x55a8e5d245d0 2024-03-19T01:16:03.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.658+0000 7f1243e00640 1 --2- 172.21.15.69:0/81159412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d245d0 0x55a8e5d249b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f123c0117e0 tx=0x7f123c011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b171068ea10993e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.658+0000 7f122fe00640 1 -- 172.21.15.69:0/81159412 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c011310 con 0x55a8e5d245d0 2024-03-19T01:16:03.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.658+0000 7f122fe00640 1 -- 172.21.15.69:0/81159412 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f123c003e50 con 0x55a8e5d245d0 2024-03-19T01:16:03.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.658+0000 7f122fe00640 1 -- 172.21.15.69:0/81159412 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c004590 con 0x55a8e5d245d0 2024-03-19T01:16:03.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.658+0000 7f123be00640 1 -- 172.21.15.69:0/4237605013 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f12340037a0 con 0x7f1244101680 2024-03-19T01:16:03.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.659+0000 7f124cfd2740 1 -- 172.21.15.69:0/81159412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d245d0 msgr2=0x55a8e5d249b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.659+0000 7f124cfd2740 1 --2- 172.21.15.69:0/81159412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d245d0 0x55a8e5d249b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f123c0117e0 tx=0x7f123c011ca0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.659+0000 7f124cfd2740 1 -- 172.21.15.69:0/81159412 shutdown_connections 2024-03-19T01:16:03.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.659+0000 7f124cfd2740 1 --2- 172.21.15.69:0/81159412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d32500 0x55a8e5d348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.659+0000 7f124cfd2740 1 --2- 172.21.15.69:0/81159412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d24f80 0x55a8e5d31fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.659+0000 7f124cfd2740 1 --2- 172.21.15.69:0/81159412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d245d0 0x55a8e5d249b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.659+0000 7f124cfd2740 1 -- 172.21.15.69:0/81159412 >> 172.21.15.69:0/81159412 conn(0x55a8e5d1f320 msgr2=0x55a8e5d1f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.659+0000 7f124cfd2740 1 -- 172.21.15.69:0/81159412 shutdown_connections 2024-03-19T01:16:03.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.659+0000 7f124cfd2740 1 -- 172.21.15.69:0/81159412 wait complete. 2024-03-19T01:16:03.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.660+0000 7f123be00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f12280a2880 0x7f12280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.660+0000 7f124cfd2740 1 -- start start 2024-03-19T01:16:03.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.660+0000 7f123be00640 1 -- 172.21.15.69:0/4237605013 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f12340cf4a0 con 0x7f1244101680 2024-03-19T01:16:03.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.660+0000 7f124cfd2740 1 Processor -- start 2024-03-19T01:16:03.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.660+0000 7f1243e00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f12280a2880 0x7f12280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d245d0 0x55a8e5d2f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 0x55a8e5d2fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f1243e00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f12280a2880 0x7f12280a4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f123c00f790 tx=0x7f123c009fd0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 0x55a8e5d30010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f124cfd2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8e5d352c0 con 0x55a8e5d245d0 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f124cfd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8e5c45160 con 0x55a8e5d24f80 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f124cfd2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a8e5d35440 con 0x55a8e5d32500 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f124cfd2740 10 client.0 _ll_register_callbacks cb 0x55a8e5c81170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f1243e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 0x55a8e5d30010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f1242a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 0x55a8e5d2fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f1243400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d245d0 0x55a8e5d2f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f1242a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 0x55a8e5d2fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55332/0 (socket says 172.21.15.69:55332) 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.661+0000 7f1242a00640 1 -- 172.21.15.69:0/2356576023 learned_addr learned my addr 172.21.15.69:0/2356576023 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:03.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.662+0000 7f1242a00640 1 -- 172.21.15.69:0/2356576023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 msgr2=0x55a8e5d30010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.662+0000 7f1242a00640 1 --2- 172.21.15.69:0/2356576023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 0x55a8e5d30010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.662+0000 7f1242a00640 1 -- 172.21.15.69:0/2356576023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d245d0 msgr2=0x55a8e5d2f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.662+0000 7f1242a00640 1 --2- 172.21.15.69:0/2356576023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d245d0 0x55a8e5d2f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.662+0000 7f1242a00640 1 -- 172.21.15.69:0/2356576023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e5a677d0 con 0x55a8e5d24f80 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.662+0000 7f1242a00640 1 --2- 172.21.15.69:0/2356576023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 0x55a8e5d2fad0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f123002cfd0 tx=0x7f123002c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26e41e83f98f0959 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.662+0000 7f122cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.662+0000 7f122cc00640 1 -- 172.21.15.69:0/2356576023 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12300046c0 con 0x55a8e5d24f80 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.662+0000 7f122cc00640 1 -- 172.21.15.69:0/2356576023 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1230004860 con 0x55a8e5d24f80 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.663+0000 7f124cfd2740 1 -- 172.21.15.69:0/2356576023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f123c003e50 con 0x55a8e5d24f80 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.663+0000 7f122cc00640 1 -- 172.21.15.69:0/2356576023 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1230004a60 con 0x55a8e5d24f80 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.663+0000 7f124cfd2740 10 client.26885 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.663+0000 7f124cfd2740 1 -- 172.21.15.69:0/2356576023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a8e5d415f0 con 0x55a8e5d24f80 2024-03-19T01:16:03.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.663+0000 7f122cc00640 1 -- 172.21.15.69:0/2356576023 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f123c003e50 con 0x55a8e5d24f80 2024-03-19T01:16:03.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.663+0000 7f122cc00640 1 -- 172.21.15.69:0/2356576023 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7f1230046070 con 0x55a8e5d24f80 2024-03-19T01:16:03.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.663+0000 7f124cfd2740 10 client.26885 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:03.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.663+0000 7f124cfd2740 10 client.26885 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:16:03.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.663+0000 7f124cfd2740 20 client.26885 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:03.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.664+0000 7f124cfd2740 1 --2- 172.21.15.69:0/2356576023 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55a8e5d42a50 0x55a8e5d44e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.664+0000 7f124cfd2740 4 client.26885 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:16:03.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.664+0000 7f124cfd2740 1 -- 172.21.15.69:0/2356576023 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8e5c968c0 con 0x55a8e5d42a50 2024-03-19T01:16:03.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.664+0000 7f1243400640 1 --2- 172.21.15.69:0/2356576023 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55a8e5d42a50 0x55a8e5d44e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.664+0000 7f1243400640 1 --2- 172.21.15.69:0/2356576023 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55a8e5d42a50 0x55a8e5d44e60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=73898f11c9fb4ff6 server_cookie=1bfbc573ecd1cde3 in_seq=0 out_seq=0 2024-03-19T01:16:03.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.664+0000 7f122cc00640 10 client.26885 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:16:03.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.665+0000 7f122cc00640 1 -- 172.21.15.69:0/2356576023 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a8e5c968c0 con 0x55a8e5d42a50 2024-03-19T01:16:03.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.665+0000 7f122cc00640 10 client.26885 handle_command_reply: tid=0 2024-03-19T01:16:03.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.665+0000 7f124cfd2740 1 client.26885 shutdown 2024-03-19T01:16:03.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.665+0000 7f124cfd2740 1 -- 172.21.15.69:0/2356576023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 msgr2=0x55a8e5d2fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.665+0000 7f124cfd2740 1 --2- 172.21.15.69:0/2356576023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 0x55a8e5d2fad0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f123002cfd0 tx=0x7f123002c800 comp rx=0 tx=0).stop 2024-03-19T01:16:03.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.666+0000 7f124cfd2740 1 -- 172.21.15.69:0/2356576023 shutdown_connections 2024-03-19T01:16:03.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.666+0000 7f124cfd2740 1 --2- 172.21.15.69:0/2356576023 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55a8e5d42a50 0x55a8e5d44e60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.666+0000 7f122cc00640 0 client.26885 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:16:03.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.666+0000 7f124cfd2740 1 --2- 172.21.15.69:0/2356576023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 0x55a8e5d30010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.666+0000 7f124cfd2740 1 --2- 172.21.15.69:0/2356576023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 0x55a8e5d2fad0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.666+0000 7f124cfd2740 1 --2- 172.21.15.69:0/2356576023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d245d0 0x55a8e5d2f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.666+0000 7f124cfd2740 1 -- 172.21.15.69:0/2356576023 >> 172.21.15.69:0/2356576023 conn(0x55a8e5d1f320 msgr2=0x55a8e5d21070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.666+0000 7f124cfd2740 1 -- 172.21.15.69:0/2356576023 shutdown_connections 2024-03-19T01:16:03.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.666+0000 7f124cfd2740 1 -- 172.21.15.69:0/2356576023 wait complete. 2024-03-19T01:16:03.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.666+0000 7f124cfd2740 20 client.26885 trim_cache size 0 max 16384 2024-03-19T01:16:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.692+0000 7f124cfd2740 1 Processor -- start 2024-03-19T01:16:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.692+0000 7f124cfd2740 1 -- start start 2024-03-19T01:16:03.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.692+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d245d0 0x55a8e5d31cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 0x55a8e5d3e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 0x55a8e5d4a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f124cfd2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f12441ac960 con 0x55a8e5d245d0 2024-03-19T01:16:03.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f124cfd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12441ae620 con 0x55a8e5d24f80 2024-03-19T01:16:03.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f124cfd2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f12441ad030 con 0x55a8e5d32500 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f1243e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 0x55a8e5d4a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f1243e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 0x55a8e5d4a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60718/0 (socket says 172.21.15.69:60718) 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f1243e00640 1 -- 172.21.15.69:0/3489219390 learned_addr learned my addr 172.21.15.69:0/3489219390 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f1242a00640 1 --2- 172.21.15.69:0/3489219390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 0x55a8e5d3e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f1243400640 1 --2- 172.21.15.69:0/3489219390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d245d0 0x55a8e5d31cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f1243e00640 1 -- 172.21.15.69:0/3489219390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 msgr2=0x55a8e5d3e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f1243e00640 1 --2- 172.21.15.69:0/3489219390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 0x55a8e5d3e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f1243e00640 1 -- 172.21.15.69:0/3489219390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d245d0 msgr2=0x55a8e5d31cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f1243e00640 1 --2- 172.21.15.69:0/3489219390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d245d0 0x55a8e5d31cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f1243e00640 1 -- 172.21.15.69:0/3489219390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e5d24d20 con 0x55a8e5d32500 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.693+0000 7f1243400640 1 --2- 172.21.15.69:0/3489219390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d245d0 0x55a8e5d31cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:16:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.694+0000 7f1243e00640 1 --2- 172.21.15.69:0/3489219390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 0x55a8e5d4a770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f123c0041e0 tx=0x7f123c011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d622c56bb12bc89 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.694+0000 7f122f400640 1 -- 172.21.15.69:0/3489219390 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c037070 con 0x55a8e5d32500 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.694+0000 7f122f400640 1 -- 172.21.15.69:0/3489219390 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f123c0111d0 con 0x55a8e5d32500 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.694+0000 7f122f400640 1 -- 172.21.15.69:0/3489219390 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c01a680 con 0x55a8e5d32500 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.695+0000 7f124cfd2740 1 -- 172.21.15.69:0/3489219390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 msgr2=0x55a8e5d4a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.695+0000 7f124cfd2740 1 --2- 172.21.15.69:0/3489219390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 0x55a8e5d4a770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f123c0041e0 tx=0x7f123c011ca0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.695+0000 7f124cfd2740 1 -- 172.21.15.69:0/3489219390 shutdown_connections 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.695+0000 7f124cfd2740 1 --2- 172.21.15.69:0/3489219390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d32500 0x55a8e5d4a770 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.695+0000 7f124cfd2740 1 --2- 172.21.15.69:0/3489219390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d24f80 0x55a8e5d3e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.695+0000 7f124cfd2740 1 --2- 172.21.15.69:0/3489219390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d245d0 0x55a8e5d31cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.695+0000 7f124cfd2740 1 -- 172.21.15.69:0/3489219390 >> 172.21.15.69:0/3489219390 conn(0x55a8e5d1f320 msgr2=0x55a8e5d2fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.695+0000 7f124cfd2740 1 -- 172.21.15.69:0/3489219390 shutdown_connections 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.695+0000 7f124cfd2740 1 -- 172.21.15.69:0/3489219390 wait complete. 2024-03-19T01:16:03.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.696+0000 7f124cfd2740 1 -- start start 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.696+0000 7f124cfd2740 1 Processor -- start 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.696+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d245d0 0x55a8e5d36250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.696+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d24f80 0x55a8e5d36790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.696+0000 7f124cfd2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d32500 0x55a8e5d36cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.696+0000 7f124cfd2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f12441ade90 con 0x55a8e5d32500 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.696+0000 7f124cfd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12441ad700 con 0x55a8e5d245d0 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.696+0000 7f124cfd2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f12441aa910 con 0x55a8e5d24f80 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.696+0000 7f124cfd2740 10 client.0 _ll_register_callbacks cb 0x7f1230009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f1243400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d245d0 0x55a8e5d36250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f1242a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d24f80 0x55a8e5d36790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f1243400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d245d0 0x55a8e5d36250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55352/0 (socket says 172.21.15.69:55352) 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f1243400640 1 -- 172.21.15.69:0/168467425 learned_addr learned my addr 172.21.15.69:0/168467425 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:03.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f1243e00640 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d32500 0x55a8e5d36cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f1243400640 1 -- 172.21.15.69:0/168467425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d24f80 msgr2=0x55a8e5d36790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f1243400640 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d24f80 0x55a8e5d36790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f1243400640 1 -- 172.21.15.69:0/168467425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d32500 msgr2=0x55a8e5d36cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f1243400640 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d32500 0x55a8e5d36cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f1243400640 1 -- 172.21.15.69:0/168467425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e5da8040 con 0x55a8e5d245d0 2024-03-19T01:16:03.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f1243400640 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d245d0 0x55a8e5d36250 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1234098110 tx=0x7f1234003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5916843b462e0dae server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f122cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:16:03.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.697+0000 7f122cc00640 1 -- 172.21.15.69:0/168467425 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12340a3ee0 con 0x55a8e5d245d0 2024-03-19T01:16:03.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f122cc00640 1 -- 172.21.15.69:0/168467425 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f1234003b30 con 0x55a8e5d245d0 2024-03-19T01:16:03.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f124cfd2740 1 -- 172.21.15.69:0/168467425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f123c0111d0 con 0x55a8e5d245d0 2024-03-19T01:16:03.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f122cc00640 1 -- 172.21.15.69:0/168467425 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f12340ccab0 con 0x55a8e5d245d0 2024-03-19T01:16:03.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f124cfd2740 10 client.26891 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:03.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f124cfd2740 1 -- 172.21.15.69:0/168467425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a8e5da92f0 con 0x55a8e5d245d0 2024-03-19T01:16:03.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f122cc00640 1 -- 172.21.15.69:0/168467425 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f123c0111d0 con 0x55a8e5d245d0 2024-03-19T01:16:03.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f122cc00640 1 -- 172.21.15.69:0/168467425 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7f123409ab90 con 0x55a8e5d245d0 2024-03-19T01:16:03.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f124cfd2740 10 client.26891 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:03.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f124cfd2740 10 client.26891 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f124cfd2740 20 client.26891 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f124cfd2740 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55a8e5da9a30 0x55a8e5dabe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f124cfd2740 4 client.26891 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.698+0000 7f124cfd2740 1 -- 172.21.15.69:0/168467425 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8e5c90c10 con 0x55a8e5da9a30 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.699+0000 7f1242a00640 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55a8e5da9a30 0x55a8e5dabe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.699+0000 7f122cc00640 10 client.26891 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.699+0000 7f1242a00640 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55a8e5da9a30 0x55a8e5dabe50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cff88fb7cf681901 server_cookie=fc35ac075e8f6582 in_seq=0 out_seq=0 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.699+0000 7f122cc00640 1 -- 172.21.15.69:0/168467425 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8e5c90c10 con 0x55a8e5da9a30 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.699+0000 7f122cc00640 10 client.26891 handle_command_reply: tid=0 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 1 client.26891 shutdown 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 1 -- 172.21.15.69:0/168467425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d245d0 msgr2=0x55a8e5d36250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d245d0 0x55a8e5d36250 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1234098110 tx=0x7f1234003e70 comp rx=0 tx=0).stop 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 1 -- 172.21.15.69:0/168467425 shutdown_connections 2024-03-19T01:16:03.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55a8e5da9a30 0x55a8e5dabe50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e5d32500 0x55a8e5d36cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8e5d24f80 0x55a8e5d36790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f122cc00640 0 client.26891 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:16:03.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 1 --2- 172.21.15.69:0/168467425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8e5d245d0 0x55a8e5d36250 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 1 -- 172.21.15.69:0/168467425 >> 172.21.15.69:0/168467425 conn(0x55a8e5d1f320 msgr2=0x55a8e5d2f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 1 -- 172.21.15.69:0/168467425 shutdown_connections 2024-03-19T01:16:03.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 1 -- 172.21.15.69:0/168467425 wait complete. 2024-03-19T01:16:03.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.700+0000 7f124cfd2740 20 client.26891 trim_cache size 0 max 16384 2024-03-19T01:16:03.705 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:16:03.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 -- 172.21.15.69:0/4237605013 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f12280a2880 msgr2=0x7f12280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f12280a2880 0x7f12280a4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f123c00f790 tx=0x7f123c009fd0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 -- 172.21.15.69:0/4237605013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244101680 msgr2=0x7f1244197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244101680 0x7f1244197f90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f123400ea60 tx=0x7f123400ef20 comp rx=0 tx=0).stop 2024-03-19T01:16:03.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 -- 172.21.15.69:0/4237605013 shutdown_connections 2024-03-19T01:16:03.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f12280a2880 0x7f12280a4d40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1244107690 0x7f124419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1244102040 0x7f12441984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 --2- 172.21.15.69:0/4237605013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1244101680 0x7f1244197f90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 -- 172.21.15.69:0/4237605013 >> 172.21.15.69:0/4237605013 conn(0x7f12440fd190 msgr2=0x7f12440fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 -- 172.21.15.69:0/4237605013 shutdown_connections 2024-03-19T01:16:03.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.701+0000 7f1248c00640 1 -- 172.21.15.69:0/4237605013 wait complete. 2024-03-19T01:16:03.720 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:16:03.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.926+0000 7f431ae00640 1 -- 172.21.15.69:0/1861692644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43141031b0 msgr2=0x7f431410b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.926+0000 7f431ae00640 1 --2- 172.21.15.69:0/1861692644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43141031b0 0x7f431410b690 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4300009900 tx=0x7f430002bcd0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.927+0000 7f431ae00640 1 -- 172.21.15.69:0/1861692644 shutdown_connections 2024-03-19T01:16:03.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.927+0000 7f431ae00640 1 --2- 172.21.15.69:0/1861692644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f431410bc60 0x7f431410e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.927+0000 7f431ae00640 1 --2- 172.21.15.69:0/1861692644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43141031b0 0x7f431410b690 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.927+0000 7f431ae00640 1 --2- 172.21.15.69:0/1861692644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4314102890 0x7f4314102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.927+0000 7f431ae00640 1 -- 172.21.15.69:0/1861692644 >> 172.21.15.69:0/1861692644 conn(0x7f43140fc070 msgr2=0x7f43140fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.927+0000 7f431ae00640 1 -- 172.21.15.69:0/1861692644 shutdown_connections 2024-03-19T01:16:03.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.927+0000 7f431ae00640 1 -- 172.21.15.69:0/1861692644 wait complete. 2024-03-19T01:16:03.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.928+0000 7f431ae00640 1 Processor -- start 2024-03-19T01:16:03.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.928+0000 7f431ae00640 1 -- start start 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.929+0000 7f431ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4314102890 0x7f431419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.929+0000 7f431ae00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43141031b0 0x7f431419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.929+0000 7f431ae00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f431410bc60 0x7f431419e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.929+0000 7f431ae00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4314113cb0 con 0x7f43141031b0 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.929+0000 7f431ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4314113b30 con 0x7f4314102890 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.929+0000 7f431ae00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4314113e30 con 0x7f431410bc60 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.929+0000 7f4319a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4314102890 0x7f431419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.929+0000 7f4319a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4314102890 0x7f431419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55372/0 (socket says 172.21.15.69:55372) 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.929+0000 7f4319a00640 1 -- 172.21.15.69:0/4282076869 learned_addr learned my addr 172.21.15.69:0/4282076869 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.929+0000 7f431a400640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f431410bc60 0x7f431419e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.929+0000 7f4319000640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43141031b0 0x7f431419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.930+0000 7f431a400640 1 -- 172.21.15.69:0/4282076869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4314102890 msgr2=0x7f431419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.930+0000 7f431a400640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4314102890 0x7f431419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.930+0000 7f431a400640 1 -- 172.21.15.69:0/4282076869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43141031b0 msgr2=0x7f431419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.930+0000 7f431a400640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43141031b0 0x7f431419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.930+0000 7f431a400640 1 -- 172.21.15.69:0/4282076869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431419f110 con 0x7f431410bc60 2024-03-19T01:16:03.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.930+0000 7f4319a00640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4314102890 0x7f431419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:03.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.930+0000 7f431a400640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f431410bc60 0x7f431419e9e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f430c009fd0 tx=0x7f430c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.930+0000 7f4312000640 1 -- 172.21.15.69:0/4282076869 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00ce20 con 0x7f431410bc60 2024-03-19T01:16:03.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.930+0000 7f4312000640 1 -- 172.21.15.69:0/4282076869 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430c005020 con 0x7f431410bc60 2024-03-19T01:16:03.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.930+0000 7f4312000640 1 -- 172.21.15.69:0/4282076869 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c010720 con 0x7f431410bc60 2024-03-19T01:16:03.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.930+0000 7f431ae00640 1 -- 172.21.15.69:0/4282076869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431419f400 con 0x7f431410bc60 2024-03-19T01:16:03.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.931+0000 7f431ae00640 1 -- 172.21.15.69:0/4282076869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43141a6c90 con 0x7f431410bc60 2024-03-19T01:16:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.934+0000 7f431f1f9740 1 Processor -- start 2024-03-19T01:16:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.934+0000 7f4312000640 1 -- 172.21.15.69:0/4282076869 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f430c005540 con 0x7f431410bc60 2024-03-19T01:16:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.934+0000 7f431f1f9740 1 -- start start 2024-03-19T01:16:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.935+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a0bf0 0x562f3b3a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.935+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.935+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.935+0000 7f431f1f9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562f3b305380 con 0x562f3b3aa0f0 2024-03-19T01:16:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.935+0000 7f431f1f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562f3b2add20 con 0x562f3b3a1510 2024-03-19T01:16:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.935+0000 7f431f1f9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562f3b301420 con 0x562f3b3a0bf0 2024-03-19T01:16:03.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.935+0000 7f4319a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.935+0000 7f4319a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55378/0 (socket says 172.21.15.69:55378) 2024-03-19T01:16:03.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.935+0000 7f4319a00640 1 -- 172.21.15.69:0/650412816 learned_addr learned my addr 172.21.15.69:0/650412816 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:03.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.936+0000 7f4319a00640 1 -- 172.21.15.69:0/650412816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a0bf0 msgr2=0x562f3b3a0fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:16:03.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.936+0000 7f4319a00640 1 --2- 172.21.15.69:0/650412816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a0bf0 0x562f3b3a0fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.936+0000 7f4319a00640 1 -- 172.21.15.69:0/650412816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 msgr2=0x562f3b3ac4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:16:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.937+0000 7f4319000640 1 --2- 172.21.15.69:0/650412816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.937+0000 7f4319a00640 1 --2- 172.21.15.69:0/650412816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3ac4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.937+0000 7f4319a00640 1 -- 172.21.15.69:0/650412816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3b0193f0 con 0x562f3b3a1510 2024-03-19T01:16:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.937+0000 7f4312000640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42fc0a29f0 0x7f42fc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.937+0000 7f4312000640 1 -- 172.21.15.69:0/4282076869 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f430c0cf7e0 con 0x7f431410bc60 2024-03-19T01:16:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.937+0000 7f4319a00640 1 --2- 172.21.15.69:0/650412816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a9bb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4308009fd0 tx=0x7f4308011ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7cb94b2adf30c67 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.937+0000 7f4306a00640 1 -- 172.21.15.69:0/650412816 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430801e070 con 0x562f3b3a1510 2024-03-19T01:16:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.938+0000 7f4306a00640 1 -- 172.21.15.69:0/650412816 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430800fbd0 con 0x562f3b3a1510 2024-03-19T01:16:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.938+0000 7f4306a00640 1 -- 172.21.15.69:0/650412816 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308019d80 con 0x562f3b3a1510 2024-03-19T01:16:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.938+0000 7f4319000640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42fc0a29f0 0x7f42fc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.939+0000 7f431f1f9740 1 -- 172.21.15.69:0/650412816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 msgr2=0x562f3b3a9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.939+0000 7f431f1f9740 1 --2- 172.21.15.69:0/650412816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a9bb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4308009fd0 tx=0x7f4308011ae0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.940+0000 7f4319000640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42fc0a29f0 0x7f42fc0a4eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4300005ab0 tx=0x7f4300005a20 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.940+0000 7f431f1f9740 1 -- 172.21.15.69:0/650412816 shutdown_connections 2024-03-19T01:16:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.940+0000 7f431f1f9740 1 --2- 172.21.15.69:0/650412816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.940+0000 7f431f1f9740 1 --2- 172.21.15.69:0/650412816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a9bb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.940+0000 7f431f1f9740 1 --2- 172.21.15.69:0/650412816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a0bf0 0x562f3b3a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.940+0000 7f431f1f9740 1 -- 172.21.15.69:0/650412816 >> 172.21.15.69:0/650412816 conn(0x562f3b39b320 msgr2=0x562f3b39b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.941+0000 7f431f1f9740 1 -- 172.21.15.69:0/650412816 shutdown_connections 2024-03-19T01:16:03.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.941+0000 7f431f1f9740 1 -- 172.21.15.69:0/650412816 wait complete. 2024-03-19T01:16:03.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.941+0000 7f431f1f9740 1 -- start start 2024-03-19T01:16:03.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.942+0000 7f431f1f9740 1 Processor -- start 2024-03-19T01:16:03.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.942+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3a7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.942+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a1510 0x562f3b3a7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.942+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.942+0000 7f431f1f9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562f3b3acee0 con 0x562f3b3aa0f0 2024-03-19T01:16:03.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.942+0000 7f431f1f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562f3b2c1160 con 0x562f3b3a0bf0 2024-03-19T01:16:03.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.942+0000 7f431f1f9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562f3b3ad060 con 0x562f3b3a1510 2024-03-19T01:16:03.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.942+0000 7f431f1f9740 10 client.0 _ll_register_callbacks cb 0x562f3b2fd170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:03.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.943+0000 7f4319a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.943+0000 7f4319a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3a7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55386/0 (socket says 172.21.15.69:55386) 2024-03-19T01:16:03.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.943+0000 7f4319a00640 1 -- 172.21.15.69:0/1798314921 learned_addr learned my addr 172.21.15.69:0/1798314921 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:03.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.943+0000 7f4319000640 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a1510 0x562f3b3a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.943+0000 7f431a400640 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.944+0000 7f4319a00640 1 -- 172.21.15.69:0/1798314921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a1510 msgr2=0x562f3b3a7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.944+0000 7f4319a00640 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a1510 0x562f3b3a7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.944+0000 7f4319a00640 1 -- 172.21.15.69:0/1798314921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 msgr2=0x562f3b3a7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.944+0000 7f4319a00640 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3a7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.944+0000 7f4319a00640 1 -- 172.21.15.69:0/1798314921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3b0e37d0 con 0x562f3b3a0bf0 2024-03-19T01:16:03.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.945+0000 7f4319a00640 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3a7100 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4308019880 tx=0x7f430800ffb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab41c73c29fa084 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.945+0000 7f42fb400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:16:03.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.945+0000 7f42fb400640 1 -- 172.21.15.69:0/1798314921 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430801e040 con 0x562f3b3a0bf0 2024-03-19T01:16:03.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.945+0000 7f42fb400640 1 -- 172.21.15.69:0/1798314921 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4308019a80 con 0x562f3b3a0bf0 2024-03-19T01:16:03.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.945+0000 7f431f1f9740 1 -- 172.21.15.69:0/1798314921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430800fbd0 con 0x562f3b3a0bf0 2024-03-19T01:16:03.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.946+0000 7f42fb400640 1 -- 172.21.15.69:0/1798314921 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f430802c050 con 0x562f3b3a0bf0 2024-03-19T01:16:03.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.946+0000 7f431f1f9740 10 client.26903 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:03.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.946+0000 7f431f1f9740 1 -- 172.21.15.69:0/1798314921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562f3b3c1970 con 0x562f3b3a0bf0 2024-03-19T01:16:03.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.946+0000 7f42fb400640 1 -- 172.21.15.69:0/1798314921 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430800fbd0 con 0x562f3b3a0bf0 2024-03-19T01:16:03.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.946+0000 7f42fb400640 1 -- 172.21.15.69:0/1798314921 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7f4308021020 con 0x562f3b3a0bf0 2024-03-19T01:16:03.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.947+0000 7f431f1f9740 10 client.26903 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:03.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.947+0000 7f431f1f9740 10 client.26903 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:16:03.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.947+0000 7f431f1f9740 20 client.26903 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:03.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.947+0000 7f431f1f9740 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x562f3b3c2dd0 0x562f3b3c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.947+0000 7f431f1f9740 4 client.26903 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:16:03.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.947+0000 7f431f1f9740 1 -- 172.21.15.69:0/1798314921 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f3b3128c0 con 0x562f3b3c2dd0 2024-03-19T01:16:03.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.947+0000 7f4319000640 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x562f3b3c2dd0 0x562f3b3c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.947+0000 7f4319000640 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x562f3b3c2dd0 0x562f3b3c51c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0590d042b6a24f2 server_cookie=ad81244b3ce99dd7 in_seq=0 out_seq=0 2024-03-19T01:16:03.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.947+0000 7f42fb400640 10 client.26903 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:16:03.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.948+0000 7f42fb400640 1 -- 172.21.15.69:0/1798314921 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562f3b3128c0 con 0x562f3b3c2dd0 2024-03-19T01:16:03.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.948+0000 7f42fb400640 10 client.26903 handle_command_reply: tid=0 2024-03-19T01:16:03.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.948+0000 7f431f1f9740 1 client.26903 shutdown 2024-03-19T01:16:03.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f431f1f9740 1 -- 172.21.15.69:0/1798314921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 msgr2=0x562f3b3a7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f431f1f9740 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3a7100 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4308019880 tx=0x7f430800ffb0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f431f1f9740 1 -- 172.21.15.69:0/1798314921 shutdown_connections 2024-03-19T01:16:03.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f431f1f9740 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3a7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f431f1f9740 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a1510 0x562f3b3a7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f431f1f9740 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x562f3b3c2dd0 0x562f3b3c51c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f431f1f9740 1 --2- 172.21.15.69:0/1798314921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3a7100 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f431f1f9740 1 -- 172.21.15.69:0/1798314921 >> 172.21.15.69:0/1798314921 conn(0x562f3b39b320 msgr2=0x562f3b39d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f42fb400640 0 client.26903 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:16:03.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f431f1f9740 1 -- 172.21.15.69:0/1798314921 shutdown_connections 2024-03-19T01:16:03.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f431f1f9740 1 -- 172.21.15.69:0/1798314921 wait complete. 2024-03-19T01:16:03.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.949+0000 7f431f1f9740 20 client.26903 trim_cache size 0 max 16384 2024-03-19T01:16:03.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.967+0000 7f431f1f9740 1 Processor -- start 2024-03-19T01:16:03.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.967+0000 7f431f1f9740 1 -- start start 2024-03-19T01:16:03.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.967+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.967+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3a1510 0x562f3b3a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.967+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3aa0f0 0x562f3b39a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.967+0000 7f431f1f9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f43141ac8d0 con 0x562f3b3a1510 2024-03-19T01:16:03.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.967+0000 7f431f1f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43141ae590 con 0x562f3b3a0bf0 2024-03-19T01:16:03.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.967+0000 7f431f1f9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43141acfa0 con 0x562f3b3aa0f0 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f4319a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f4319a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3be510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55392/0 (socket says 172.21.15.69:55392) 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f4319a00640 1 -- 172.21.15.69:0/1673169112 learned_addr learned my addr 172.21.15.69:0/1673169112 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f431a400640 1 --2- 172.21.15.69:0/1673169112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3aa0f0 0x562f3b39a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f4319000640 1 --2- 172.21.15.69:0/1673169112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3a1510 0x562f3b3a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f4319a00640 1 -- 172.21.15.69:0/1673169112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3aa0f0 msgr2=0x562f3b39a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f4319a00640 1 --2- 172.21.15.69:0/1673169112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3aa0f0 0x562f3b39a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f4319a00640 1 -- 172.21.15.69:0/1673169112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3a1510 msgr2=0x562f3b3a79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f4319a00640 1 --2- 172.21.15.69:0/1673169112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3a1510 0x562f3b3a79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f4319a00640 1 -- 172.21.15.69:0/1673169112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3b3c5750 con 0x562f3b3a0bf0 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f4319000640 1 --2- 172.21.15.69:0/1673169112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3a1510 0x562f3b3a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:16:03.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.968+0000 7f4319a00640 1 --2- 172.21.15.69:0/1673169112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3be510 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4308006010 tx=0x7f4308009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8ef887308131521 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.969+0000 7f4306000640 1 -- 172.21.15.69:0/1673169112 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308015030 con 0x562f3b3a0bf0 2024-03-19T01:16:03.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.969+0000 7f4306000640 1 -- 172.21.15.69:0/1673169112 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4308005c50 con 0x562f3b3a0bf0 2024-03-19T01:16:03.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.969+0000 7f4306000640 1 -- 172.21.15.69:0/1673169112 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430801e030 con 0x562f3b3a0bf0 2024-03-19T01:16:03.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.970+0000 7f431f1f9740 1 -- 172.21.15.69:0/1673169112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 msgr2=0x562f3b3be510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.970+0000 7f431f1f9740 1 --2- 172.21.15.69:0/1673169112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3be510 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4308006010 tx=0x7f4308009d00 comp rx=0 tx=0).stop 2024-03-19T01:16:03.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.970+0000 7f431f1f9740 1 -- 172.21.15.69:0/1673169112 shutdown_connections 2024-03-19T01:16:03.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.970+0000 7f431f1f9740 1 --2- 172.21.15.69:0/1673169112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3aa0f0 0x562f3b39a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.970+0000 7f431f1f9740 1 --2- 172.21.15.69:0/1673169112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3a1510 0x562f3b3a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.970+0000 7f431f1f9740 1 --2- 172.21.15.69:0/1673169112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a0bf0 0x562f3b3be510 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.970+0000 7f431f1f9740 1 -- 172.21.15.69:0/1673169112 >> 172.21.15.69:0/1673169112 conn(0x562f3b39b320 msgr2=0x562f3b399aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.970+0000 7f431f1f9740 1 -- 172.21.15.69:0/1673169112 shutdown_connections 2024-03-19T01:16:03.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.970+0000 7f431f1f9740 1 -- 172.21.15.69:0/1673169112 wait complete. 2024-03-19T01:16:03.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.970+0000 7f431f1f9740 1 -- start start 2024-03-19T01:16:03.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.971+0000 7f431f1f9740 1 Processor -- start 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.971+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a0bf0 0x562f3b3bae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.971+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.971+0000 7f431f1f9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.971+0000 7f431f1f9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f43141ade00 con 0x562f3b3aa0f0 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.971+0000 7f431f1f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43141ad670 con 0x562f3b3a1510 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.971+0000 7f431f1f9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43141aa8a0 con 0x562f3b3a0bf0 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.971+0000 7f431f1f9740 10 client.0 _ll_register_callbacks cb 0x7f430801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a0bf0 0x562f3b3bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a0bf0 0x562f3b3bae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60754/0 (socket says 172.21.15.69:60754) 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55402/0 (socket says 172.21.15.69:55402) 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319a00640 1 -- 172.21.15.69:0/3121807420 learned_addr learned my addr 172.21.15.69:0/3121807420 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:03.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f431a400640 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319a00640 1 -- 172.21.15.69:0/3121807420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 msgr2=0x562f3b3a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319a00640 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319a00640 1 -- 172.21.15.69:0/3121807420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 msgr2=0x562f3b3a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319a00640 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319a00640 1 -- 172.21.15.69:0/3121807420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3b3a47d0 con 0x562f3b3a0bf0 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f431a400640 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319000640 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f4319a00640 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a0bf0 0x562f3b3bae00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4308000c00 tx=0x7f4308019410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3746fb774d5f6cf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.972+0000 7f42fb400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.973+0000 7f42fb400640 1 -- 172.21.15.69:0/3121807420 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430801e070 con 0x562f3b3a0bf0 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.973+0000 7f431f1f9740 1 -- 172.21.15.69:0/3121807420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4308005c50 con 0x562f3b3a0bf0 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.973+0000 7f42fb400640 1 -- 172.21.15.69:0/3121807420 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4308005470 con 0x562f3b3a0bf0 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.973+0000 7f42fb400640 1 -- 172.21.15.69:0/3121807420 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4308042050 con 0x562f3b3a0bf0 2024-03-19T01:16:03.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.973+0000 7f431f1f9740 10 client.26866 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:03.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.973+0000 7f431f1f9740 1 -- 172.21.15.69:0/3121807420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562f3b429050 con 0x562f3b3a0bf0 2024-03-19T01:16:03.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.973+0000 7f42fb400640 1 -- 172.21.15.69:0/3121807420 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4308005c50 con 0x562f3b3a0bf0 2024-03-19T01:16:03.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.973+0000 7f42fb400640 1 -- 172.21.15.69:0/3121807420 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7f4308021050 con 0x562f3b3a0bf0 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.974+0000 7f431f1f9740 10 client.26866 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.974+0000 7f431f1f9740 10 client.26866 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.974+0000 7f431f1f9740 20 client.26866 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.974+0000 7f431f1f9740 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x562f3b429660 0x562f3b42ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.974+0000 7f431f1f9740 4 client.26866 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.974+0000 7f431f1f9740 1 -- 172.21.15.69:0/3121807420 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f3b30cc10 con 0x562f3b429660 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.974+0000 7f4319000640 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x562f3b429660 0x562f3b42ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.974+0000 7f42fb400640 10 client.26866 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.974+0000 7f4319000640 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x562f3b429660 0x562f3b42ba80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8effa0cea530856a server_cookie=b817e891f2eeba3a in_seq=0 out_seq=0 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f42fb400640 1 -- 172.21.15.69:0/3121807420 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f3b30cc10 con 0x562f3b429660 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f42fb400640 10 client.26866 handle_command_reply: tid=0 2024-03-19T01:16:03.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 1 client.26866 shutdown 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 1 -- 172.21.15.69:0/3121807420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a0bf0 msgr2=0x562f3b3bae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a0bf0 0x562f3b3bae00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4308000c00 tx=0x7f4308019410 comp rx=0 tx=0).stop 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 1 -- 172.21.15.69:0/3121807420 shutdown_connections 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562f3b3aa0f0 0x562f3b3a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x562f3b429660 0x562f3b42ba80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f3b3a1510 0x562f3b3a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 1 --2- 172.21.15.69:0/3121807420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562f3b3a0bf0 0x562f3b3bae00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f42fb400640 0 client.26866 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 1 -- 172.21.15.69:0/3121807420 >> 172.21.15.69:0/3121807420 conn(0x562f3b39b320 msgr2=0x562f3b39a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 1 -- 172.21.15.69:0/3121807420 shutdown_connections 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 1 -- 172.21.15.69:0/3121807420 wait complete. 2024-03-19T01:16:03.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.975+0000 7f431f1f9740 20 client.26866 trim_cache size 0 max 16384 2024-03-19T01:16:03.981 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.976+0000 7f431ae00640 1 -- 172.21.15.69:0/4282076869 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42fc0a29f0 msgr2=0x7f42fc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.976+0000 7f431ae00640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42fc0a29f0 0x7f42fc0a4eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4300005ab0 tx=0x7f4300005a20 comp rx=0 tx=0).stop 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.976+0000 7f431ae00640 1 -- 172.21.15.69:0/4282076869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f431410bc60 msgr2=0x7f431419e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.976+0000 7f431ae00640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f431410bc60 0x7f431419e9e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f430c009fd0 tx=0x7f430c00ef10 comp rx=0 tx=0).stop 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.976+0000 7f431ae00640 1 -- 172.21.15.69:0/4282076869 shutdown_connections 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.976+0000 7f431ae00640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f431410bc60 0x7f431419e9e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.976+0000 7f431ae00640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43141031b0 0x7f431419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.976+0000 7f431ae00640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f42fc0a29f0 0x7f42fc0a4eb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.976+0000 7f431ae00640 1 --2- 172.21.15.69:0/4282076869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4314102890 0x7f431419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.976+0000 7f431ae00640 1 -- 172.21.15.69:0/4282076869 >> 172.21.15.69:0/4282076869 conn(0x7f43140fc070 msgr2=0x7f43140fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.977+0000 7f431ae00640 1 -- 172.21.15.69:0/4282076869 shutdown_connections 2024-03-19T01:16:03.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:03.977+0000 7f431ae00640 1 -- 172.21.15.69:0/4282076869 wait complete. 2024-03-19T01:16:03.996 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:16:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.228+0000 7fa338a00640 1 -- 172.21.15.69:0/1411543777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa334073bd0 msgr2=0x7fa334073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.228+0000 7fa338a00640 1 --2- 172.21.15.69:0/1411543777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa334073bd0 0x7fa334073fb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa324009ca0 tx=0x7fa32402c190 comp rx=0 tx=0).stop 2024-03-19T01:16:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.228+0000 7fa338a00640 1 -- 172.21.15.69:0/1411543777 shutdown_connections 2024-03-19T01:16:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.228+0000 7fa338a00640 1 --2- 172.21.15.69:0/1411543777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa33410e070 0x7fa334110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.228+0000 7fa338a00640 1 --2- 172.21.15.69:0/1411543777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3340744f0 0x7fa33410db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.228+0000 7fa338a00640 1 --2- 172.21.15.69:0/1411543777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa334073bd0 0x7fa334073fb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.228+0000 7fa338a00640 1 -- 172.21.15.69:0/1411543777 >> 172.21.15.69:0/1411543777 conn(0x7fa334078c20 msgr2=0x7fa334079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.228+0000 7fa338a00640 1 -- 172.21.15.69:0/1411543777 shutdown_connections 2024-03-19T01:16:04.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.228+0000 7fa338a00640 1 -- 172.21.15.69:0/1411543777 wait complete. 2024-03-19T01:16:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.229+0000 7fa338a00640 1 Processor -- start 2024-03-19T01:16:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.229+0000 7fa338a00640 1 -- start start 2024-03-19T01:16:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.230+0000 7fa338a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa334073bd0 0x7fa33419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.230+0000 7fa338a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3340744f0 0x7fa33419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.230+0000 7fa338a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa33410e070 0x7fa33419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.230+0000 7fa338a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa334113d90 con 0x7fa334073bd0 2024-03-19T01:16:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.230+0000 7fa338a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa334113c10 con 0x7fa33410e070 2024-03-19T01:16:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.230+0000 7fa338a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa334113f10 con 0x7fa3340744f0 2024-03-19T01:16:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.230+0000 7fa331400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3340744f0 0x7fa33419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.230+0000 7fa331400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3340744f0 0x7fa33419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60782/0 (socket says 172.21.15.69:60782) 2024-03-19T01:16:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.230+0000 7fa331400640 1 -- 172.21.15.69:0/2203021471 learned_addr learned my addr 172.21.15.69:0/2203021471 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.230+0000 7fa332800640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa33410e070 0x7fa33419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.230+0000 7fa331e00640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa334073bd0 0x7fa33419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa331400640 1 -- 172.21.15.69:0/2203021471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa33410e070 msgr2=0x7fa33419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa331400640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa33410e070 0x7fa33419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa331400640 1 -- 172.21.15.69:0/2203021471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa334073bd0 msgr2=0x7fa33419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa331400640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa334073bd0 0x7fa33419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa331400640 1 -- 172.21.15.69:0/2203021471 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33419f300 con 0x7fa3340744f0 2024-03-19T01:16:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa331e00640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa334073bd0 0x7fa33419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:16:04.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa331400640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3340744f0 0x7fa33419a850 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa32000e9f0 tx=0x7fa32000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa32aa00640 1 -- 172.21.15.69:0/2203021471 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32000ccc0 con 0x7fa3340744f0 2024-03-19T01:16:04.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa32aa00640 1 -- 172.21.15.69:0/2203021471 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa320010910 con 0x7fa3340744f0 2024-03-19T01:16:04.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa338a00640 1 -- 172.21.15.69:0/2203021471 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33419f5f0 con 0x7fa3340744f0 2024-03-19T01:16:04.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa32aa00640 1 -- 172.21.15.69:0/2203021471 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200188a0 con 0x7fa3340744f0 2024-03-19T01:16:04.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.231+0000 7fa338a00640 1 -- 172.21.15.69:0/2203021471 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3341a6e80 con 0x7fa3340744f0 2024-03-19T01:16:04.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.236+0000 7fa33b61c740 1 Processor -- start 2024-03-19T01:16:04.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.236+0000 7fa33b61c740 1 -- start start 2024-03-19T01:16:04.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.237+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d53830 0x561921d53c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.237+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 0x561921d651f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.237+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d65730 0x561921d67b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.237+0000 7fa33b61c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561921cb8380 con 0x561921d53830 2024-03-19T01:16:04.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.237+0000 7fa33b61c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561921c60d20 con 0x561921d541e0 2024-03-19T01:16:04.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.237+0000 7fa33b61c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561921cb4420 con 0x561921d65730 2024-03-19T01:16:04.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.237+0000 7fa332800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d53830 0x561921d53c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.237+0000 7fa331e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 0x561921d651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.237+0000 7fa331e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 0x561921d651f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55428/0 (socket says 172.21.15.69:55428) 2024-03-19T01:16:04.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.237+0000 7fa332800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d53830 0x561921d53c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54238/0 (socket says 172.21.15.69:54238) 2024-03-19T01:16:04.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.237+0000 7fa331e00640 1 -- 172.21.15.69:0/1665575783 learned_addr learned my addr 172.21.15.69:0/1665575783 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.238+0000 7fa331e00640 1 -- 172.21.15.69:0/1665575783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d65730 msgr2=0x561921d67b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:16:04.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.238+0000 7fa331e00640 1 --2- 172.21.15.69:0/1665575783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d65730 0x561921d67b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.238+0000 7fa331e00640 1 -- 172.21.15.69:0/1665575783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d53830 msgr2=0x561921d53c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.238+0000 7fa331e00640 1 --2- 172.21.15.69:0/1665575783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d53830 0x561921d53c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.238+0000 7fa331e00640 1 -- 172.21.15.69:0/1665575783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619219cc3f0 con 0x561921d541e0 2024-03-19T01:16:04.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.238+0000 7fa32aa00640 1 -- 172.21.15.69:0/2203021471 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fa320010ac0 con 0x7fa3340744f0 2024-03-19T01:16:04.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.238+0000 7fa331e00640 1 --2- 172.21.15.69:0/1665575783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 0x561921d651f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa324009ca0 tx=0x7fa324009910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab42de66b92579fd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.240+0000 7fa32aa00640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa3180a29f0 0x7fa3180a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.240+0000 7fa32aa00640 1 -- 172.21.15.69:0/2203021471 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa3200cf6b0 con 0x7fa3340744f0 2024-03-19T01:16:04.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.240+0000 7fa332800640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa3180a29f0 0x7fa3180a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.240+0000 7fa31f000640 1 -- 172.21.15.69:0/1665575783 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324042020 con 0x561921d541e0 2024-03-19T01:16:04.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.240+0000 7fa31f000640 1 -- 172.21.15.69:0/1665575783 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa324003e50 con 0x561921d541e0 2024-03-19T01:16:04.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.240+0000 7fa31f000640 1 -- 172.21.15.69:0/1665575783 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324039040 con 0x561921d541e0 2024-03-19T01:16:04.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.241+0000 7fa33b61c740 1 -- 172.21.15.69:0/1665575783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 msgr2=0x561921d651f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.241+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1665575783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 0x561921d651f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa324009ca0 tx=0x7fa324009910 comp rx=0 tx=0).stop 2024-03-19T01:16:04.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.241+0000 7fa332800640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa3180a29f0 0x7fa3180a4eb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009d30 tx=0x7fa32c005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.242+0000 7fa33b61c740 1 -- 172.21.15.69:0/1665575783 shutdown_connections 2024-03-19T01:16:04.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.242+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1665575783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d65730 0x561921d67b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.242+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1665575783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 0x561921d651f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.242+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1665575783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d53830 0x561921d53c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.242+0000 7fa33b61c740 1 -- 172.21.15.69:0/1665575783 >> 172.21.15.69:0/1665575783 conn(0x561921d4e150 msgr2=0x561921d4e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.242+0000 7fa33b61c740 1 -- 172.21.15.69:0/1665575783 shutdown_connections 2024-03-19T01:16:04.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.242+0000 7fa33b61c740 1 -- 172.21.15.69:0/1665575783 wait complete. 2024-03-19T01:16:04.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.242+0000 7fa33b61c740 1 -- start start 2024-03-19T01:16:04.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.243+0000 7fa33b61c740 1 Processor -- start 2024-03-19T01:16:04.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.243+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.243+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d541e0 0x561921d62d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.243+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d65730 0x561921d63240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.243+0000 7fa33b61c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561921d68490 con 0x561921d541e0 2024-03-19T01:16:04.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.243+0000 7fa33b61c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561921c74160 con 0x561921d65730 2024-03-19T01:16:04.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.243+0000 7fa33b61c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561921d68610 con 0x561921d53830 2024-03-19T01:16:04.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa33b61c740 10 client.0 _ll_register_callbacks cb 0x561921cb0170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:04.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa331e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa332800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d65730 0x561921d63240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa331e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d627c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60796/0 (socket says 172.21.15.69:60796) 2024-03-19T01:16:04.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa332800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d65730 0x561921d63240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55442/0 (socket says 172.21.15.69:55442) 2024-03-19T01:16:04.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa331e00640 1 -- 172.21.15.69:0/1380293199 learned_addr learned my addr 172.21.15.69:0/1380293199 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa331400640 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d541e0 0x561921d62d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa331e00640 1 -- 172.21.15.69:0/1380293199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d65730 msgr2=0x561921d63240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa331e00640 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d65730 0x561921d63240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa331e00640 1 -- 172.21.15.69:0/1380293199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d541e0 msgr2=0x561921d62d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa331e00640 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d541e0 0x561921d62d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa331e00640 1 -- 172.21.15.69:0/1380293199 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561921a967d0 con 0x561921d53830 2024-03-19T01:16:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa331400640 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d541e0 0x561921d62d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:16:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.244+0000 7fa331e00640 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d627c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa32402c6d0 tx=0x7fa324004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92df8387d4ecff83 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.245+0000 7fa313400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:16:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.245+0000 7fa313400640 1 -- 172.21.15.69:0/1380293199 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32404b070 con 0x561921d53830 2024-03-19T01:16:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.245+0000 7fa313400640 1 -- 172.21.15.69:0/1380293199 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa324003b80 con 0x561921d53830 2024-03-19T01:16:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.245+0000 7fa33b61c740 1 -- 172.21.15.69:0/1380293199 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa324003e50 con 0x561921d53830 2024-03-19T01:16:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.245+0000 7fa313400640 1 -- 172.21.15.69:0/1380293199 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa324056050 con 0x561921d53830 2024-03-19T01:16:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.245+0000 7fa33b61c740 10 client.26875 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.245+0000 7fa33b61c740 1 -- 172.21.15.69:0/1380293199 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561921d684e0 con 0x561921d53830 2024-03-19T01:16:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.245+0000 7fa313400640 1 -- 172.21.15.69:0/1380293199 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa324003e50 con 0x561921d53830 2024-03-19T01:16:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.245+0000 7fa313400640 1 -- 172.21.15.69:0/1380293199 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7fa324047570 con 0x561921d53830 2024-03-19T01:16:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.246+0000 7fa33b61c740 10 client.26875 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.246+0000 7fa33b61c740 10 client.26875 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:16:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.246+0000 7fa33b61c740 20 client.26875 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.246+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x561921d75950 0x561921d77d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.246+0000 7fa33b61c740 4 client.26875 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:16:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.246+0000 7fa33b61c740 1 -- 172.21.15.69:0/1380293199 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561921cc58c0 con 0x561921d75950 2024-03-19T01:16:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.246+0000 7fa331400640 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x561921d75950 0x561921d77d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.247+0000 7fa313400640 10 client.26875 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:16:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.247+0000 7fa331400640 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x561921d75950 0x561921d77d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7dbf7f9fd7ab8d7b server_cookie=c5a05458ab850649 in_seq=0 out_seq=0 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.247+0000 7fa313400640 1 -- 172.21.15.69:0/1380293199 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561921cc58c0 con 0x561921d75950 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.247+0000 7fa313400640 10 client.26875 handle_command_reply: tid=0 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.247+0000 7fa33b61c740 1 client.26875 shutdown 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa33b61c740 1 -- 172.21.15.69:0/1380293199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 msgr2=0x561921d627c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d627c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa32402c6d0 tx=0x7fa324004970 comp rx=0 tx=0).stop 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa33b61c740 1 -- 172.21.15.69:0/1380293199 shutdown_connections 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x561921d75950 0x561921d77d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d65730 0x561921d63240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d541e0 0x561921d62d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1380293199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d627c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa313400640 0 client.26875 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa33b61c740 1 -- 172.21.15.69:0/1380293199 >> 172.21.15.69:0/1380293199 conn(0x561921d4e150 msgr2=0x561921d4fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa33b61c740 1 -- 172.21.15.69:0/1380293199 shutdown_connections 2024-03-19T01:16:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa33b61c740 1 -- 172.21.15.69:0/1380293199 wait complete. 2024-03-19T01:16:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.248+0000 7fa33b61c740 20 client.26875 trim_cache size 0 max 16384 2024-03-19T01:16:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.262+0000 7fa33b61c740 1 Processor -- start 2024-03-19T01:16:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa33b61c740 1 -- start start 2024-03-19T01:16:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d53830 0x561921d64ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d541e0 0x561921d71270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d65730 0x561921d7d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa33b61c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341acb40 con 0x561921d541e0 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa33b61c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341ae800 con 0x561921d53830 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa33b61c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341ad210 con 0x561921d65730 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa331e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d53830 0x561921d64ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa331e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d53830 0x561921d64ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55456/0 (socket says 172.21.15.69:55456) 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa331e00640 1 -- 172.21.15.69:0/1665445739 learned_addr learned my addr 172.21.15.69:0/1665445739 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa331400640 1 --2- 172.21.15.69:0/1665445739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d541e0 0x561921d71270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.263+0000 7fa332800640 1 --2- 172.21.15.69:0/1665445739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d65730 0x561921d7d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.264+0000 7fa331e00640 1 -- 172.21.15.69:0/1665445739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d65730 msgr2=0x561921d7d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.264+0000 7fa331e00640 1 --2- 172.21.15.69:0/1665445739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d65730 0x561921d7d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.264+0000 7fa331e00640 1 -- 172.21.15.69:0/1665445739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d541e0 msgr2=0x561921d71270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.264+0000 7fa331e00640 1 --2- 172.21.15.69:0/1665445739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d541e0 0x561921d71270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.264+0000 7fa331e00640 1 -- 172.21.15.69:0/1665445739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561921d53f80 con 0x561921d53830 2024-03-19T01:16:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.264+0000 7fa331e00640 1 --2- 172.21.15.69:0/1665445739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d53830 0x561921d64ef0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa324000f80 tx=0x7fa324009a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc1152405a1c29f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.264+0000 7fa31e600640 1 -- 172.21.15.69:0/1665445739 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32404b050 con 0x561921d53830 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.264+0000 7fa31e600640 1 -- 172.21.15.69:0/1665445739 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa324032340 con 0x561921d53830 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.264+0000 7fa31e600640 1 -- 172.21.15.69:0/1665445739 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324042050 con 0x561921d53830 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.265+0000 7fa33b61c740 1 -- 172.21.15.69:0/1665445739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d53830 msgr2=0x561921d64ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.265+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1665445739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d53830 0x561921d64ef0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa324000f80 tx=0x7fa324009a00 comp rx=0 tx=0).stop 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.265+0000 7fa33b61c740 1 -- 172.21.15.69:0/1665445739 shutdown_connections 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.265+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1665445739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d65730 0x561921d7d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.265+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1665445739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d541e0 0x561921d71270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.265+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1665445739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d53830 0x561921d64ef0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.265+0000 7fa33b61c740 1 -- 172.21.15.69:0/1665445739 >> 172.21.15.69:0/1665445739 conn(0x561921d4e150 msgr2=0x561921d4d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.265+0000 7fa33b61c740 1 -- 172.21.15.69:0/1665445739 shutdown_connections 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.265+0000 7fa33b61c740 1 -- 172.21.15.69:0/1665445739 wait complete. 2024-03-19T01:16:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.265+0000 7fa33b61c740 1 -- start start 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.266+0000 7fa33b61c740 1 Processor -- start 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.266+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d6dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.266+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 0x561921d68fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.266+0000 7fa33b61c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d65730 0x561921d695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.266+0000 7fa33b61c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341ae070 con 0x561921d65730 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.266+0000 7fa33b61c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341ad8e0 con 0x561921d541e0 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.266+0000 7fa33b61c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341aaaf0 con 0x561921d53830 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa33b61c740 10 client.0 _ll_register_callbacks cb 0x7fa32c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa331e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d6dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa331e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d6dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60826/0 (socket says 172.21.15.69:60826) 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa331e00640 1 -- 172.21.15.69:0/1464172150 learned_addr learned my addr 172.21.15.69:0/1464172150 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa331400640 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 0x561921d68fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa332800640 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d65730 0x561921d695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa331400640 1 -- 172.21.15.69:0/1464172150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 msgr2=0x561921d6dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa331400640 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d6dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa331400640 1 -- 172.21.15.69:0/1464172150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d65730 msgr2=0x561921d695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa331400640 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d65730 0x561921d695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa331400640 1 -- 172.21.15.69:0/1464172150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561921d69de0 con 0x561921d541e0 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa331e00640 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d6dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa331400640 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 0x561921d68fa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa3200cc9e0 tx=0x7fa32000cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65384ec09294f537 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.267+0000 7fa313400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa313400640 1 -- 172.21.15.69:0/1464172150 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200b4060 con 0x561921d541e0 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa313400640 1 -- 172.21.15.69:0/1464172150 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa3200ccc60 con 0x561921d541e0 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa33b61c740 1 -- 172.21.15.69:0/1464172150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa324032340 con 0x561921d541e0 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa313400640 1 -- 172.21.15.69:0/1464172150 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa3200a4090 con 0x561921d541e0 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa33b61c740 10 client.26927 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa33b61c740 1 -- 172.21.15.69:0/1464172150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561921ddbc10 con 0x561921d541e0 2024-03-19T01:16:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa313400640 1 -- 172.21.15.69:0/1464172150 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa324032340 con 0x561921d541e0 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa313400640 1 -- 172.21.15.69:0/1464172150 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7fa3200cf2a0 con 0x561921d541e0 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa33b61c740 10 client.26927 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa33b61c740 10 client.26927 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa33b61c740 20 client.26927 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x561921ddc340 0x561921dde760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa33b61c740 4 client.26927 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.268+0000 7fa33b61c740 1 -- 172.21.15.69:0/1464172150 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561921cbfc10 con 0x561921ddc340 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.269+0000 7fa331e00640 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x561921ddc340 0x561921dde760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.269+0000 7fa331e00640 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x561921ddc340 0x561921dde760 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17d5c0f2c5867958 server_cookie=71c0aa9c790123aa in_seq=0 out_seq=0 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.269+0000 7fa313400640 10 client.26927 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.269+0000 7fa313400640 1 -- 172.21.15.69:0/1464172150 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561921cbfc10 con 0x561921ddc340 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.269+0000 7fa313400640 10 client.26927 handle_command_reply: tid=0 2024-03-19T01:16:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.269+0000 7fa33b61c740 1 client.26927 shutdown 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa33b61c740 1 -- 172.21.15.69:0/1464172150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 msgr2=0x561921d68fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 0x561921d68fa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa3200cc9e0 tx=0x7fa32000cef0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa33b61c740 1 -- 172.21.15.69:0/1464172150 shutdown_connections 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x561921ddc340 0x561921dde760 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561921d65730 0x561921d695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561921d541e0 0x561921d68fa0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa33b61c740 1 --2- 172.21.15.69:0/1464172150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561921d53830 0x561921d6dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa313400640 0 client.26927 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa33b61c740 1 -- 172.21.15.69:0/1464172150 >> 172.21.15.69:0/1464172150 conn(0x561921d4e150 msgr2=0x561921d4f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa33b61c740 1 -- 172.21.15.69:0/1464172150 shutdown_connections 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa33b61c740 1 -- 172.21.15.69:0/1464172150 wait complete. 2024-03-19T01:16:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.270+0000 7fa33b61c740 20 client.26927 trim_cache size 0 max 16384 2024-03-19T01:16:04.276 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:16:04.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 -- 172.21.15.69:0/2203021471 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa3180a29f0 msgr2=0x7fa3180a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa3180a29f0 0x7fa3180a4eb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009d30 tx=0x7fa32c005f90 comp rx=0 tx=0).stop 2024-03-19T01:16:04.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 -- 172.21.15.69:0/2203021471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3340744f0 msgr2=0x7fa33419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3340744f0 0x7fa33419a850 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa32000e9f0 tx=0x7fa32000edf0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 -- 172.21.15.69:0/2203021471 shutdown_connections 2024-03-19T01:16:04.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa3180a29f0 0x7fa3180a4eb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa33410e070 0x7fa33419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3340744f0 0x7fa33419a850 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 --2- 172.21.15.69:0/2203021471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa334073bd0 0x7fa33419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 -- 172.21.15.69:0/2203021471 >> 172.21.15.69:0/2203021471 conn(0x7fa334078c20 msgr2=0x7fa3340fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 -- 172.21.15.69:0/2203021471 shutdown_connections 2024-03-19T01:16:04.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.271+0000 7fa338a00640 1 -- 172.21.15.69:0/2203021471 wait complete. 2024-03-19T01:16:04.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:04 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/98542759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:16:04.290 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:16:04.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.491+0000 7fa681e00640 1 -- 172.21.15.69:0/3438173979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa67c1076a0 msgr2=0x7fa67c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.491+0000 7fa681e00640 1 --2- 172.21.15.69:0/3438173979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa67c1076a0 0x7fa67c107a80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa670009930 tx=0x7fa67002bc20 comp rx=0 tx=0).stop 2024-03-19T01:16:04.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.491+0000 7fa681e00640 1 -- 172.21.15.69:0/3438173979 shutdown_connections 2024-03-19T01:16:04.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.491+0000 7fa681e00640 1 --2- 172.21.15.69:0/3438173979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa67c102050 0x7fa67c10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.491+0000 7fa681e00640 1 --2- 172.21.15.69:0/3438173979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c101690 0x7fa67c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.491+0000 7fa681e00640 1 --2- 172.21.15.69:0/3438173979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa67c1076a0 0x7fa67c107a80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.491+0000 7fa681e00640 1 -- 172.21.15.69:0/3438173979 >> 172.21.15.69:0/3438173979 conn(0x7fa67c0fd180 msgr2=0x7fa67c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.491+0000 7fa681e00640 1 -- 172.21.15.69:0/3438173979 shutdown_connections 2024-03-19T01:16:04.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.491+0000 7fa681e00640 1 -- 172.21.15.69:0/3438173979 wait complete. 2024-03-19T01:16:04.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.493+0000 7fa681e00640 1 Processor -- start 2024-03-19T01:16:04.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.493+0000 7fa681e00640 1 -- start start 2024-03-19T01:16:04.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.493+0000 7fa681e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c101690 0x7fa67c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.493+0000 7fa681e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa67c102050 0x7fa67c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.493+0000 7fa681e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa67c1076a0 0x7fa67c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.493+0000 7fa681e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c111b00 con 0x7fa67c101690 2024-03-19T01:16:04.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.493+0000 7fa681e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c111980 con 0x7fa67c102050 2024-03-19T01:16:04.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.493+0000 7fa681e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c111c80 con 0x7fa67c1076a0 2024-03-19T01:16:04.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.494+0000 7fa680a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c101690 0x7fa67c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.494+0000 7fa680a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c101690 0x7fa67c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54292/0 (socket says 172.21.15.69:54292) 2024-03-19T01:16:04.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.494+0000 7fa67be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa67c102050 0x7fa67c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.494+0000 7fa680a00640 1 -- 172.21.15.69:0/3406767846 learned_addr learned my addr 172.21.15.69:0/3406767846 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.494+0000 7fa681400640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa67c1076a0 0x7fa67c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.494+0000 7fa67be00640 1 -- 172.21.15.69:0/3406767846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa67c1076a0 msgr2=0x7fa67c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.494+0000 7fa67be00640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa67c1076a0 0x7fa67c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.494+0000 7fa67be00640 1 -- 172.21.15.69:0/3406767846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c101690 msgr2=0x7fa67c19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.494+0000 7fa67be00640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c101690 0x7fa67c19a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.494+0000 7fa67be00640 1 -- 172.21.15.69:0/3406767846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c19f1f0 con 0x7fa67c102050 2024-03-19T01:16:04.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.495+0000 7fa67be00640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa67c102050 0x7fa67c19a740 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa66800e9f0 tx=0x7fa66800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.495+0000 7fa679600640 1 -- 172.21.15.69:0/3406767846 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66800ccc0 con 0x7fa67c102050 2024-03-19T01:16:04.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.495+0000 7fa679600640 1 -- 172.21.15.69:0/3406767846 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa668010910 con 0x7fa67c102050 2024-03-19T01:16:04.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.495+0000 7fa681e00640 1 -- 172.21.15.69:0/3406767846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67c19f4e0 con 0x7fa67c102050 2024-03-19T01:16:04.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.495+0000 7fa679600640 1 -- 172.21.15.69:0/3406767846 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa668018960 con 0x7fa67c102050 2024-03-19T01:16:04.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.495+0000 7fa681e00640 1 -- 172.21.15.69:0/3406767846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67c1a6d70 con 0x7fa67c102050 2024-03-19T01:16:04.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.499+0000 7fa679600640 1 -- 172.21.15.69:0/3406767846 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fa668010ac0 con 0x7fa67c102050 2024-03-19T01:16:04.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.500+0000 7fa686229740 1 Processor -- start 2024-03-19T01:16:04.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.500+0000 7fa679600640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6640a29f0 0x7fa6640a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.500+0000 7fa686229740 1 -- start start 2024-03-19T01:16:04.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.500+0000 7fa679600640 1 -- 172.21.15.69:0/3406767846 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa6680cf580 con 0x7fa67c102050 2024-03-19T01:16:04.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.500+0000 7fa681400640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6640a29f0 0x7fa6640a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa686229740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa686229740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa686229740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa686229740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d3b6f31380 con 0x55d3b6fcd510 2024-03-19T01:16:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa686229740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3b6ed9d20 con 0x55d3b6fd60f0 2024-03-19T01:16:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa686229740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d3b6f2d420 con 0x55d3b6fccbf0 2024-03-19T01:16:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa680a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa680a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60868/0 (socket says 172.21.15.69:60868) 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa680a00640 1 -- 172.21.15.69:0/517269434 learned_addr learned my addr 172.21.15.69:0/517269434 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa681400640 1 --2- 172.21.15.69:0/517269434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa67be00640 1 --2- 172.21.15.69:0/517269434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa681400640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6640a29f0 0x7fa6640a4eb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa674005fd0 tx=0x7fa674009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa680a00640 1 -- 172.21.15.69:0/517269434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fd60f0 msgr2=0x55d3b6fd84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa680a00640 1 --2- 172.21.15.69:0/517269434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa680a00640 1 -- 172.21.15.69:0/517269434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fcd510 msgr2=0x55d3b6fd5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa680a00640 1 --2- 172.21.15.69:0/517269434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa680a00640 1 -- 172.21.15.69:0/517269434 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b6c453f0 con 0x55d3b6fccbf0 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.501+0000 7fa681400640 1 --2- 172.21.15.69:0/517269434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.502+0000 7fa680a00640 1 --2- 172.21.15.69:0/517269434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fccfd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa670036040 tx=0x7fa670005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25b2a83551b86849 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.502+0000 7fa66dc00640 1 -- 172.21.15.69:0/517269434 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670045070 con 0x55d3b6fccbf0 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.502+0000 7fa66dc00640 1 -- 172.21.15.69:0/517269434 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa67002c640 con 0x55d3b6fccbf0 2024-03-19T01:16:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.502+0000 7fa66dc00640 1 -- 172.21.15.69:0/517269434 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670004c60 con 0x55d3b6fccbf0 2024-03-19T01:16:04.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.503+0000 7fa686229740 1 -- 172.21.15.69:0/517269434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 msgr2=0x55d3b6fccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.503+0000 7fa686229740 1 --2- 172.21.15.69:0/517269434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fccfd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa670036040 tx=0x7fa670005e40 comp rx=0 tx=0).stop 2024-03-19T01:16:04.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.503+0000 7fa686229740 1 -- 172.21.15.69:0/517269434 shutdown_connections 2024-03-19T01:16:04.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.503+0000 7fa686229740 1 --2- 172.21.15.69:0/517269434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.503+0000 7fa686229740 1 --2- 172.21.15.69:0/517269434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.503+0000 7fa686229740 1 --2- 172.21.15.69:0/517269434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fccfd0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.503+0000 7fa686229740 1 -- 172.21.15.69:0/517269434 >> 172.21.15.69:0/517269434 conn(0x55d3b6fc7320 msgr2=0x55d3b6fc7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.503+0000 7fa686229740 1 -- 172.21.15.69:0/517269434 shutdown_connections 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.503+0000 7fa686229740 1 -- 172.21.15.69:0/517269434 wait complete. 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.504+0000 7fa686229740 1 -- start start 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.504+0000 7fa686229740 1 Processor -- start 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.504+0000 7fa686229740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fd3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.504+0000 7fa686229740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.504+0000 7fa686229740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.504+0000 7fa686229740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d3b6fd8e80 con 0x55d3b6fd60f0 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.504+0000 7fa686229740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3b6eed160 con 0x55d3b6fcd510 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.504+0000 7fa686229740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d3b6fd9000 con 0x55d3b6fccbf0 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.504+0000 7fa686229740 10 client.0 _ll_register_callbacks cb 0x55d3b6f29170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa67be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa680a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fd3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa680a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fd3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60876/0 (socket says 172.21.15.69:60876) 2024-03-19T01:16:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa67be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55488/0 (socket says 172.21.15.69:55488) 2024-03-19T01:16:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa680a00640 1 -- 172.21.15.69:0/1681821692 learned_addr learned my addr 172.21.15.69:0/1681821692 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa681400640 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa680a00640 1 -- 172.21.15.69:0/1681821692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 msgr2=0x55d3b6fd35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa680a00640 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa680a00640 1 -- 172.21.15.69:0/1681821692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fd60f0 msgr2=0x55d3b6fd3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa680a00640 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa680a00640 1 -- 172.21.15.69:0/1681821692 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b6d0f7d0 con 0x55d3b6fccbf0 2024-03-19T01:16:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa67be00640 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa680a00640 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fd3090 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa67002caa0 tx=0x7fa670009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6131d6c919529d0b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.505+0000 7fa662000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.506+0000 7fa662000640 1 -- 172.21.15.69:0/1681821692 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670045040 con 0x55d3b6fccbf0 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.506+0000 7fa662000640 1 -- 172.21.15.69:0/1681821692 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa6700052f0 con 0x55d3b6fccbf0 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.506+0000 7fa686229740 1 -- 172.21.15.69:0/1681821692 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa67002c640 con 0x55d3b6fccbf0 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.506+0000 7fa662000640 1 -- 172.21.15.69:0/1681821692 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa67005b050 con 0x55d3b6fccbf0 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.506+0000 7fa686229740 10 client.26893 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.506+0000 7fa686229740 1 -- 172.21.15.69:0/1681821692 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d3b6fd8ed0 con 0x55d3b6fccbf0 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.506+0000 7fa662000640 1 -- 172.21.15.69:0/1681821692 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa67002c640 con 0x55d3b6fccbf0 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.507+0000 7fa662000640 1 -- 172.21.15.69:0/1681821692 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7fa67003f440 con 0x55d3b6fccbf0 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.507+0000 7fa686229740 10 client.26893 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.507+0000 7fa686229740 10 client.26893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.507+0000 7fa686229740 20 client.26893 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.507+0000 7fa686229740 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d3b6fe62b0 0x55d3b6fe86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.507+0000 7fa686229740 4 client.26893 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:16:04.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.507+0000 7fa686229740 1 -- 172.21.15.69:0/1681821692 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3b6f3e8c0 con 0x55d3b6fe62b0 2024-03-19T01:16:04.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.507+0000 7fa67be00640 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d3b6fe62b0 0x55d3b6fe86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.508+0000 7fa67be00640 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d3b6fe62b0 0x55d3b6fe86a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7100ce5613d36304 server_cookie=6c18f22803222eb0 in_seq=0 out_seq=0 2024-03-19T01:16:04.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.508+0000 7fa662000640 10 client.26893 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:16:04.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:04 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/98542759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:16:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.508+0000 7fa662000640 1 -- 172.21.15.69:0/1681821692 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d3b6f3e8c0 con 0x55d3b6fe62b0 2024-03-19T01:16:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.508+0000 7fa662000640 10 client.26893 handle_command_reply: tid=0 2024-03-19T01:16:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.508+0000 7fa686229740 1 client.26893 shutdown 2024-03-19T01:16:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa686229740 1 -- 172.21.15.69:0/1681821692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 msgr2=0x55d3b6fd3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa686229740 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fd3090 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa67002caa0 tx=0x7fa670009260 comp rx=0 tx=0).stop 2024-03-19T01:16:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa686229740 1 -- 172.21.15.69:0/1681821692 shutdown_connections 2024-03-19T01:16:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa686229740 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa686229740 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d3b6fe62b0 0x55d3b6fe86a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa686229740 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa686229740 1 --2- 172.21.15.69:0/1681821692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fd3090 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa662000640 0 client.26893 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:16:04.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa686229740 1 -- 172.21.15.69:0/1681821692 >> 172.21.15.69:0/1681821692 conn(0x55d3b6fc7320 msgr2=0x55d3b6fc9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa686229740 1 -- 172.21.15.69:0/1681821692 shutdown_connections 2024-03-19T01:16:04.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa686229740 1 -- 172.21.15.69:0/1681821692 wait complete. 2024-03-19T01:16:04.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.509+0000 7fa686229740 20 client.26893 trim_cache size 0 max 16384 2024-03-19T01:16:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.538+0000 7fa686229740 1 Processor -- start 2024-03-19T01:16:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.538+0000 7fa686229740 1 -- start start 2024-03-19T01:16:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.539+0000 7fa686229740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fd57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.539+0000 7fa686229740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fe1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.539+0000 7fa686229740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fc61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.539+0000 7fa686229740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b0ce0 con 0x55d3b6fd60f0 2024-03-19T01:16:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.539+0000 7fa686229740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b29a0 con 0x55d3b6fcd510 2024-03-19T01:16:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.539+0000 7fa686229740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b13b0 con 0x55d3b6fccbf0 2024-03-19T01:16:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.539+0000 7fa67be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fe1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.539+0000 7fa67be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fe1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55490/0 (socket says 172.21.15.69:55490) 2024-03-19T01:16:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.539+0000 7fa67be00640 1 -- 172.21.15.69:0/1933173046 learned_addr learned my addr 172.21.15.69:0/1933173046 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.539+0000 7fa680a00640 1 --2- 172.21.15.69:0/1933173046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fd57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.539+0000 7fa681400640 1 --2- 172.21.15.69:0/1933173046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fc61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.540+0000 7fa680a00640 1 -- 172.21.15.69:0/1933173046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 msgr2=0x55d3b6fe1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.540+0000 7fa680a00640 1 --2- 172.21.15.69:0/1933173046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fe1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.540+0000 7fa680a00640 1 -- 172.21.15.69:0/1933173046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fd60f0 msgr2=0x55d3b6fc61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.540+0000 7fa680a00640 1 --2- 172.21.15.69:0/1933173046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fc61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.540+0000 7fa680a00640 1 -- 172.21.15.69:0/1933173046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b6fc7150 con 0x55d3b6fccbf0 2024-03-19T01:16:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.540+0000 7fa67be00640 1 --2- 172.21.15.69:0/1933173046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fe1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.540+0000 7fa680a00640 1 --2- 172.21.15.69:0/1933173046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fd57c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa67003f7e0 tx=0x7fa670005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfe63569c5e380ac server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.540+0000 7fa66d200640 1 -- 172.21.15.69:0/1933173046 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670045070 con 0x55d3b6fccbf0 2024-03-19T01:16:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.540+0000 7fa66d200640 1 -- 172.21.15.69:0/1933173046 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6700055d0 con 0x55d3b6fccbf0 2024-03-19T01:16:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.540+0000 7fa66d200640 1 -- 172.21.15.69:0/1933173046 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67002cd30 con 0x55d3b6fccbf0 2024-03-19T01:16:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.541+0000 7fa686229740 1 -- 172.21.15.69:0/1933173046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 msgr2=0x55d3b6fd57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.541+0000 7fa686229740 1 --2- 172.21.15.69:0/1933173046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fd57c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa67003f7e0 tx=0x7fa670005fd0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.541+0000 7fa686229740 1 -- 172.21.15.69:0/1933173046 shutdown_connections 2024-03-19T01:16:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.541+0000 7fa686229740 1 --2- 172.21.15.69:0/1933173046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fc61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.541+0000 7fa686229740 1 --2- 172.21.15.69:0/1933173046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fcd510 0x55d3b6fe1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.541+0000 7fa686229740 1 --2- 172.21.15.69:0/1933173046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fd57c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.541+0000 7fa686229740 1 -- 172.21.15.69:0/1933173046 >> 172.21.15.69:0/1933173046 conn(0x55d3b6fc7320 msgr2=0x55d3b6fc5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.541+0000 7fa686229740 1 -- 172.21.15.69:0/1933173046 shutdown_connections 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.541+0000 7fa686229740 1 -- 172.21.15.69:0/1933173046 wait complete. 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.541+0000 7fa686229740 1 -- start start 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa686229740 1 Processor -- start 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa686229740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fde4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa686229740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa686229740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa686229740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b2210 con 0x55d3b6fcd510 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa686229740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1b1a80 con 0x55d3b6fd60f0 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa686229740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1aec90 con 0x55d3b6fccbf0 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa686229740 10 client.0 _ll_register_callbacks cb 0x7fa674009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa680a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fde4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa680a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fde4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60894/0 (socket says 172.21.15.69:60894) 2024-03-19T01:16:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa681400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa680a00640 1 -- 172.21.15.69:0/2503063495 learned_addr learned my addr 172.21.15.69:0/2503063495 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.542+0000 7fa67be00640 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.543+0000 7fa680a00640 1 -- 172.21.15.69:0/2503063495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fd60f0 msgr2=0x55d3b6fd9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.543+0000 7fa680a00640 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.543+0000 7fa680a00640 1 -- 172.21.15.69:0/2503063495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fcd510 msgr2=0x55d3b6fd9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.543+0000 7fa680a00640 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.543+0000 7fa680a00640 1 -- 172.21.15.69:0/2503063495 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b6fda5b0 con 0x55d3b6fccbf0 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.543+0000 7fa680a00640 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fde4b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa670036040 tx=0x7fa67002bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f20c6568523f4d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.543+0000 7fa662000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.543+0000 7fa662000640 1 -- 172.21.15.69:0/2503063495 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670045040 con 0x55d3b6fccbf0 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.543+0000 7fa662000640 1 -- 172.21.15.69:0/2503063495 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7fa67002ccd0 con 0x55d3b6fccbf0 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.543+0000 7fa686229740 1 -- 172.21.15.69:0/2503063495 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6700055d0 con 0x55d3b6fccbf0 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.543+0000 7fa662000640 1 -- 172.21.15.69:0/2503063495 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa67006d050 con 0x55d3b6fccbf0 2024-03-19T01:16:04.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.544+0000 7fa686229740 10 client.26899 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:04.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.544+0000 7fa686229740 1 -- 172.21.15.69:0/2503063495 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d3b704c570 con 0x55d3b6fccbf0 2024-03-19T01:16:04.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.544+0000 7fa662000640 1 -- 172.21.15.69:0/2503063495 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6700055d0 con 0x55d3b6fccbf0 2024-03-19T01:16:04.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.544+0000 7fa662000640 1 -- 172.21.15.69:0/2503063495 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7fa67004a6f0 con 0x55d3b6fccbf0 2024-03-19T01:16:04.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.544+0000 7fa686229740 10 client.26899 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.544+0000 7fa686229740 10 client.26899 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.544+0000 7fa686229740 20 client.26899 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.544+0000 7fa686229740 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d3b704cce0 0x55d3b704f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.544+0000 7fa686229740 4 client.26899 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.544+0000 7fa686229740 1 -- 172.21.15.69:0/2503063495 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3b6f38c10 con 0x55d3b704cce0 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.544+0000 7fa67be00640 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d3b704cce0 0x55d3b704f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa67be00640 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d3b704cce0 0x55d3b704f100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1edcfc8e011e0c5b server_cookie=41e92747dc803b5f in_seq=0 out_seq=0 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa662000640 10 client.26899 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa662000640 1 -- 172.21.15.69:0/2503063495 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3b6f38c10 con 0x55d3b704cce0 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa662000640 10 client.26899 handle_command_reply: tid=0 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa686229740 1 client.26899 shutdown 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa686229740 1 -- 172.21.15.69:0/2503063495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 msgr2=0x55d3b6fde4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa686229740 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fde4b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa670036040 tx=0x7fa67002bc20 comp rx=0 tx=0).stop 2024-03-19T01:16:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa686229740 1 -- 172.21.15.69:0/2503063495 shutdown_connections 2024-03-19T01:16:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa686229740 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55d3b704cce0 0x55d3b704f100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa686229740 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b6fd60f0 0x55d3b6fd9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa686229740 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3b6fcd510 0x55d3b6fd9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa686229740 1 --2- 172.21.15.69:0/2503063495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d3b6fccbf0 0x55d3b6fde4b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa662000640 0 client.26899 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:16:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.545+0000 7fa686229740 1 -- 172.21.15.69:0/2503063495 >> 172.21.15.69:0/2503063495 conn(0x55d3b6fc7320 msgr2=0x55d3b6fc6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.546+0000 7fa686229740 1 -- 172.21.15.69:0/2503063495 shutdown_connections 2024-03-19T01:16:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.546+0000 7fa686229740 1 -- 172.21.15.69:0/2503063495 wait complete. 2024-03-19T01:16:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.546+0000 7fa686229740 20 client.26899 trim_cache size 0 max 16384 2024-03-19T01:16:04.550 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:16:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.546+0000 7fa681e00640 1 -- 172.21.15.69:0/3406767846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6640a29f0 msgr2=0x7fa6640a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.546+0000 7fa681e00640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6640a29f0 0x7fa6640a4eb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa674005fd0 tx=0x7fa674009500 comp rx=0 tx=0).stop 2024-03-19T01:16:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.546+0000 7fa681e00640 1 -- 172.21.15.69:0/3406767846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa67c102050 msgr2=0x7fa67c19a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.546+0000 7fa681e00640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa67c102050 0x7fa67c19a740 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa66800e9f0 tx=0x7fa66800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.547+0000 7fa681e00640 1 -- 172.21.15.69:0/3406767846 shutdown_connections 2024-03-19T01:16:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.547+0000 7fa681e00640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa67c1076a0 0x7fa67c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.547+0000 7fa681e00640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa6640a29f0 0x7fa6640a4eb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.547+0000 7fa681e00640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa67c102050 0x7fa67c19a740 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.547+0000 7fa681e00640 1 --2- 172.21.15.69:0/3406767846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c101690 0x7fa67c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.547+0000 7fa681e00640 1 -- 172.21.15.69:0/3406767846 >> 172.21.15.69:0/3406767846 conn(0x7fa67c0fd180 msgr2=0x7fa67c10ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.547+0000 7fa681e00640 1 -- 172.21.15.69:0/3406767846 shutdown_connections 2024-03-19T01:16:04.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.547+0000 7fa681e00640 1 -- 172.21.15.69:0/3406767846 wait complete. 2024-03-19T01:16:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:04 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/98542759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:16:04.565 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:16:04.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.789+0000 7f4ab4600640 1 -- 172.21.15.69:0/470421293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aac068a20 msgr2=0x7f4aac068ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.789+0000 7f4ab4600640 1 --2- 172.21.15.69:0/470421293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aac068a20 0x7f4aac068ea0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009960 tx=0x7f4a9802bc10 comp rx=0 tx=0).stop 2024-03-19T01:16:04.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.789+0000 7f4ab4600640 1 -- 172.21.15.69:0/470421293 shutdown_connections 2024-03-19T01:16:04.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.789+0000 7f4ab4600640 1 --2- 172.21.15.69:0/470421293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4aac0693e0 0x7f4aac10e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.789+0000 7f4ab4600640 1 --2- 172.21.15.69:0/470421293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aac068a20 0x7f4aac068ea0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.789+0000 7f4ab4600640 1 --2- 172.21.15.69:0/470421293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aac107870 0x7f4aac107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.789+0000 7f4ab4600640 1 -- 172.21.15.69:0/470421293 >> 172.21.15.69:0/470421293 conn(0x7f4aac077820 msgr2=0x7f4aac077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.789+0000 7f4ab4600640 1 -- 172.21.15.69:0/470421293 shutdown_connections 2024-03-19T01:16:04.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.789+0000 7f4ab4600640 1 -- 172.21.15.69:0/470421293 wait complete. 2024-03-19T01:16:04.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.791+0000 7f4ab4600640 1 Processor -- start 2024-03-19T01:16:04.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.791+0000 7f4ab4600640 1 -- start start 2024-03-19T01:16:04.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.791+0000 7f4ab4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aac068a20 0x7f4aac198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab4600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4aac0693e0 0x7f4aac198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab4600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aac107870 0x7f4aac19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab4600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac111bc0 con 0x7f4aac0693e0 2024-03-19T01:16:04.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac111a40 con 0x7f4aac068a20 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab4600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac111d40 con 0x7f4aac107870 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aac068a20 0x7f4aac198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aac068a20 0x7f4aac198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55514/0 (socket says 172.21.15.69:55514) 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab1c00640 1 -- 172.21.15.69:0/2477033240 learned_addr learned my addr 172.21.15.69:0/2477033240 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab2600640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aac107870 0x7f4aac19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab1200640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4aac0693e0 0x7f4aac198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab1c00640 1 -- 172.21.15.69:0/2477033240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aac107870 msgr2=0x7f4aac19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab1c00640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aac107870 0x7f4aac19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab1c00640 1 -- 172.21.15.69:0/2477033240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4aac0693e0 msgr2=0x7f4aac198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab1c00640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4aac0693e0 0x7f4aac198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab1c00640 1 -- 172.21.15.69:0/2477033240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac19d0c0 con 0x7f4aac068a20 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.792+0000 7f4ab2600640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aac107870 0x7f4aac19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.793+0000 7f4ab1c00640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aac068a20 0x7f4aac198120 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000e9f0 tx=0x7f4aa000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.793+0000 7f4aaa000640 1 -- 172.21.15.69:0/2477033240 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000cd80 con 0x7f4aac068a20 2024-03-19T01:16:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.793+0000 7f4ab4600640 1 -- 172.21.15.69:0/2477033240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aac19d3b0 con 0x7f4aac068a20 2024-03-19T01:16:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.793+0000 7f4aaa000640 1 -- 172.21.15.69:0/2477033240 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa0010910 con 0x7f4aac068a20 2024-03-19T01:16:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.793+0000 7f4aaa000640 1 -- 172.21.15.69:0/2477033240 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa00188f0 con 0x7f4aac068a20 2024-03-19T01:16:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.793+0000 7f4ab4600640 1 -- 172.21.15.69:0/2477033240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aac1a4c40 con 0x7f4aac068a20 2024-03-19T01:16:04.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.797+0000 7f4ab7385740 1 Processor -- start 2024-03-19T01:16:04.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.797+0000 7f4ab7385740 1 -- start start 2024-03-19T01:16:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.797+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd98210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.797+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efda0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.797+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 0x5560efda3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.797+0000 7f4ab7385740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5560efcf8380 con 0x5560efd97e30 2024-03-19T01:16:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.797+0000 7f4ab7385740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560efca0d20 con 0x5560efda1330 2024-03-19T01:16:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.797+0000 7f4ab7385740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5560efcf4420 con 0x5560efd98750 2024-03-19T01:16:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.798+0000 7f4ab2600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd98210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.798+0000 7f4ab2600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd98210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54356/0 (socket says 172.21.15.69:54356) 2024-03-19T01:16:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.798+0000 7f4ab2600640 1 -- 172.21.15.69:0/1318905433 learned_addr learned my addr 172.21.15.69:0/1318905433 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.798+0000 7f4ab1200640 1 --2- 172.21.15.69:0/1318905433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 0x5560efda3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.798+0000 7f4ab2600640 1 -- 172.21.15.69:0/1318905433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 msgr2=0x5560efda0df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:16:04.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.798+0000 7f4ab2600640 1 --2- 172.21.15.69:0/1318905433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efda0df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.798+0000 7f4ab2600640 1 -- 172.21.15.69:0/1318905433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 msgr2=0x5560efda3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.798+0000 7f4ab2600640 1 --2- 172.21.15.69:0/1318905433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 0x5560efda3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.798+0000 7f4ab2600640 1 -- 172.21.15.69:0/1318905433 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560efa0c3f0 con 0x5560efd97e30 2024-03-19T01:16:04.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.798+0000 7f4aaa000640 1 -- 172.21.15.69:0/2477033240 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f4aa0010ac0 con 0x7f4aac068a20 2024-03-19T01:16:04.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.798+0000 7f4ab2600640 1 --2- 172.21.15.69:0/1318905433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd98210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4010750 tx=0x7f4aa4010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf9ba0a1466ab21 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.799+0000 7f4a9ea00640 1 -- 172.21.15.69:0/1318905433 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa400ead0 con 0x5560efd97e30 2024-03-19T01:16:04.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.799+0000 7f4a9ea00640 1 -- 172.21.15.69:0/1318905433 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa4010e60 con 0x5560efd97e30 2024-03-19T01:16:04.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.799+0000 7f4a9ea00640 1 -- 172.21.15.69:0/1318905433 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4011640 con 0x5560efd97e30 2024-03-19T01:16:04.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.800+0000 7f4ab7385740 1 -- 172.21.15.69:0/1318905433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 msgr2=0x5560efd98210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.800+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1318905433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd98210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4010750 tx=0x7f4aa4010c10 comp rx=0 tx=0).stop 2024-03-19T01:16:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.800+0000 7f4ab7385740 1 -- 172.21.15.69:0/1318905433 shutdown_connections 2024-03-19T01:16:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.800+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1318905433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 0x5560efda3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.800+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1318905433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efda0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.800+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1318905433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd98210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.800+0000 7f4ab7385740 1 -- 172.21.15.69:0/1318905433 >> 172.21.15.69:0/1318905433 conn(0x5560efd92560 msgr2=0x5560efd92940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.800+0000 7f4ab7385740 1 -- 172.21.15.69:0/1318905433 shutdown_connections 2024-03-19T01:16:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.800+0000 7f4ab7385740 1 -- 172.21.15.69:0/1318905433 wait complete. 2024-03-19T01:16:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.800+0000 7f4aaa000640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4a940a2900 0x7f4a940a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.801+0000 7f4aaa000640 1 -- 172.21.15.69:0/2477033240 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4aa00cf5d0 con 0x7f4aac068a20 2024-03-19T01:16:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.801+0000 7f4ab2600640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4a940a2900 0x7f4a940a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.801+0000 7f4ab7385740 1 -- start start 2024-03-19T01:16:04.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.801+0000 7f4ab7385740 1 Processor -- start 2024-03-19T01:16:04.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.801+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd9e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd98750 0x5560efd9e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab2600640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4a940a2900 0x7f4a940a4dc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4000f80 tx=0x7f4aa4012d30 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efda1330 0x5560efd9eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab7385740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5560efda4120 con 0x5560efd97e30 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab7385740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560efcb4160 con 0x5560efd98750 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab7385740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5560efda42a0 con 0x5560efda1330 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab7385740 10 client.0 _ll_register_callbacks cb 0x5560efcf0170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab1c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd9e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab2600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efda1330 0x5560efd9eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd98750 0x5560efd9e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab2600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efda1330 0x5560efd9eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60918/0 (socket says 172.21.15.69:60918) 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab2600640 1 -- 172.21.15.69:0/1913707482 learned_addr learned my addr 172.21.15.69:0/1913707482 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab1c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd9e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54364/0 (socket says 172.21.15.69:54364) 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab1200640 1 -- 172.21.15.69:0/1913707482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efda1330 msgr2=0x5560efd9eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab1200640 1 --2- 172.21.15.69:0/1913707482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efda1330 0x5560efd9eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab1200640 1 -- 172.21.15.69:0/1913707482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 msgr2=0x5560efd9e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab1200640 1 --2- 172.21.15.69:0/1913707482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd9e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.802+0000 7f4ab1200640 1 -- 172.21.15.69:0/1913707482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560efad67d0 con 0x5560efd98750 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.803+0000 7f4ab1200640 1 --2- 172.21.15.69:0/1913707482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd98750 0x5560efd9e9b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009230 tx=0x7f4a98009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce8e879c249b6b75 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.803+0000 7f4a93400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.803+0000 7f4a93400640 1 -- 172.21.15.69:0/1913707482 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98004450 con 0x5560efd98750 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.803+0000 7f4ab7385740 1 -- 172.21.15.69:0/1913707482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa4010e60 con 0x5560efd98750 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.803+0000 7f4a93400640 1 -- 172.21.15.69:0/1913707482 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4a98005030 con 0x5560efd98750 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.803+0000 7f4a93400640 1 -- 172.21.15.69:0/1913707482 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4a98005230 con 0x5560efd98750 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.803+0000 7f4ab7385740 10 client.26957 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.803+0000 7f4ab7385740 1 -- 172.21.15.69:0/1913707482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5560efdb04f0 con 0x5560efd98750 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.804+0000 7f4a93400640 1 -- 172.21.15.69:0/1913707482 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa4010e60 con 0x5560efd98750 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.804+0000 7f4a93400640 1 -- 172.21.15.69:0/1913707482 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7f4a98003680 con 0x5560efd98750 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.804+0000 7f4ab7385740 10 client.26957 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:04.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.804+0000 7f4ab7385740 10 client.26957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:16:04.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.804+0000 7f4ab7385740 20 client.26957 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:04.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.804+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1913707482 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560efdb1950 0x5560efdb3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.804+0000 7f4ab7385740 4 client.26957 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:16:04.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.804+0000 7f4ab7385740 1 -- 172.21.15.69:0/1913707482 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560efd058c0 con 0x5560efdb1950 2024-03-19T01:16:04.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.805+0000 7f4ab1c00640 1 --2- 172.21.15.69:0/1913707482 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560efdb1950 0x5560efdb3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.805+0000 7f4a93400640 10 client.26957 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:04.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.805+0000 7f4ab1c00640 1 --2- 172.21.15.69:0/1913707482 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560efdb1950 0x5560efdb3d90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff780ab2464bf95 server_cookie=f73ffce636ecdde2 in_seq=0 out_seq=0 2024-03-19T01:16:04.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.807+0000 7f4a93400640 1 -- 172.21.15.69:0/1913707482 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5560efd058c0 con 0x5560efdb1950 2024-03-19T01:16:04.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.808+0000 7f4a93400640 10 client.26957 handle_command_reply: tid=0 2024-03-19T01:16:04.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.808+0000 7f4ab7385740 1 client.26957 shutdown 2024-03-19T01:16:04.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.808+0000 7f4ab7385740 1 -- 172.21.15.69:0/1913707482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd98750 msgr2=0x5560efd9e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.808+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1913707482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd98750 0x5560efd9e9b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009230 tx=0x7f4a98009670 comp rx=0 tx=0).stop 2024-03-19T01:16:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.809+0000 7f4ab7385740 1 -- 172.21.15.69:0/1913707482 shutdown_connections 2024-03-19T01:16:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.809+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1913707482 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560efdb1950 0x5560efdb3d90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.809+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1913707482 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efda1330 0x5560efd9eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.809+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1913707482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd98750 0x5560efd9e9b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.809+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1913707482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd9e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.809+0000 7f4ab7385740 1 -- 172.21.15.69:0/1913707482 >> 172.21.15.69:0/1913707482 conn(0x5560efd92560 msgr2=0x5560efd942b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.809+0000 7f4a93400640 0 client.26957 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.809+0000 7f4ab7385740 1 -- 172.21.15.69:0/1913707482 shutdown_connections 2024-03-19T01:16:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.809+0000 7f4ab7385740 1 -- 172.21.15.69:0/1913707482 wait complete. 2024-03-19T01:16:04.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.810+0000 7f4ab7385740 20 client.26957 trim_cache size 0 max 16384 2024-03-19T01:16:04.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.827+0000 7f4ab7385740 1 Processor -- start 2024-03-19T01:16:04.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.827+0000 7f4ab7385740 1 -- start start 2024-03-19T01:16:04.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.827+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd9ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.827+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efdacec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.827+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 0x5560efd8d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.827+0000 7f4ab7385740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1acb80 con 0x5560efd97e30 2024-03-19T01:16:04.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.827+0000 7f4ab7385740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1ae840 con 0x5560efda1330 2024-03-19T01:16:04.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.827+0000 7f4ab7385740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1ad250 con 0x5560efd98750 2024-03-19T01:16:04.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 0x5560efd8d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab1200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efdacec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 0x5560efd8d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55538/0 (socket says 172.21.15.69:55538) 2024-03-19T01:16:04.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab2600640 1 -- 172.21.15.69:0/2650893534 learned_addr learned my addr 172.21.15.69:0/2650893534 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab1c00640 1 --2- 172.21.15.69:0/2650893534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd9ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab2600640 1 -- 172.21.15.69:0/2650893534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 msgr2=0x5560efdacec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab2600640 1 --2- 172.21.15.69:0/2650893534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efdacec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab2600640 1 -- 172.21.15.69:0/2650893534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 msgr2=0x5560efd9ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab2600640 1 --2- 172.21.15.69:0/2650893534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd9ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab2600640 1 -- 172.21.15.69:0/2650893534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560efd98580 con 0x5560efda1330 2024-03-19T01:16:04.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab1200640 1 --2- 172.21.15.69:0/2650893534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efdacec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:04.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.828+0000 7f4ab2600640 1 --2- 172.21.15.69:0/2650893534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 0x5560efd8d050 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400e6c0 tx=0x7f4aa4025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc082f778aaaad1b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.829+0000 7f4a9e000640 1 -- 172.21.15.69:0/2650893534 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4012450 con 0x5560efda1330 2024-03-19T01:16:04.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.829+0000 7f4a9e000640 1 -- 172.21.15.69:0/2650893534 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa4012ad0 con 0x5560efda1330 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.829+0000 7f4a9e000640 1 -- 172.21.15.69:0/2650893534 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4029ce0 con 0x5560efda1330 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.830+0000 7f4ab7385740 1 -- 172.21.15.69:0/2650893534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 msgr2=0x5560efd8d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.830+0000 7f4ab7385740 1 --2- 172.21.15.69:0/2650893534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 0x5560efd8d050 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400e6c0 tx=0x7f4aa4025000 comp rx=0 tx=0).stop 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.830+0000 7f4ab7385740 1 -- 172.21.15.69:0/2650893534 shutdown_connections 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.830+0000 7f4ab7385740 1 --2- 172.21.15.69:0/2650893534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efda1330 0x5560efd8d050 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.830+0000 7f4ab7385740 1 --2- 172.21.15.69:0/2650893534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efdacec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.830+0000 7f4ab7385740 1 --2- 172.21.15.69:0/2650893534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efd97e30 0x5560efd9ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.830+0000 7f4ab7385740 1 -- 172.21.15.69:0/2650893534 >> 172.21.15.69:0/2650893534 conn(0x5560efd92560 msgr2=0x5560efd8c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.830+0000 7f4ab7385740 1 -- 172.21.15.69:0/2650893534 shutdown_connections 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.830+0000 7f4ab7385740 1 -- 172.21.15.69:0/2650893534 wait complete. 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.830+0000 7f4ab7385740 1 -- start start 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.831+0000 7f4ab7385740 1 Processor -- start 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.831+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd97e30 0x5560efd9ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.831+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efd9b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.831+0000 7f4ab7385740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efda1330 0x5560efd9b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.831+0000 7f4ab7385740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1ae0b0 con 0x5560efda1330 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.831+0000 7f4ab7385740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1ad920 con 0x5560efd97e30 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.831+0000 7f4ab7385740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1aab30 con 0x5560efd98750 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.831+0000 7f4ab7385740 10 client.0 _ll_register_callbacks cb 0x7f4a98009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.831+0000 7f4ab1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd97e30 0x5560efd9ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4ab1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd97e30 0x5560efd9ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:55544/0 (socket says 172.21.15.69:55544) 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4ab1c00640 1 -- 172.21.15.69:0/1279866558 learned_addr learned my addr 172.21.15.69:0/1279866558 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4ab1200640 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efd9b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4ab2600640 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efda1330 0x5560efd9b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4ab1200640 1 -- 172.21.15.69:0/1279866558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd97e30 msgr2=0x5560efd9ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4ab1200640 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd97e30 0x5560efd9ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4ab1200640 1 -- 172.21.15.69:0/1279866558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efda1330 msgr2=0x5560efd9b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4ab1200640 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efda1330 0x5560efd9b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4ab1200640 1 -- 172.21.15.69:0/1279866558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560efda8750 con 0x5560efd98750 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4ab2600640 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efda1330 0x5560efd9b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4ab1200640 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efd9b220 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4a98001eb0 tx=0x7f4a98005600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5740174307fdbf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.832+0000 7f4a93400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.833+0000 7f4a93400640 1 -- 172.21.15.69:0/1279866558 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98005d50 con 0x5560efd98750 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.833+0000 7f4ab7385740 1 -- 172.21.15.69:0/1279866558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa4012ad0 con 0x5560efd98750 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.833+0000 7f4a93400640 1 -- 172.21.15.69:0/1279866558 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f4a98004450 con 0x5560efd98750 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.833+0000 7f4a93400640 1 -- 172.21.15.69:0/1279866558 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4a9802cd40 con 0x5560efd98750 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.833+0000 7f4ab7385740 10 client.26914 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.833+0000 7f4ab7385740 1 -- 172.21.15.69:0/1279866558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5560efda9a50 con 0x5560efd98750 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.833+0000 7f4a93400640 1 -- 172.21.15.69:0/1279866558 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa4012ad0 con 0x5560efd98750 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.833+0000 7f4a93400640 1 -- 172.21.15.69:0/1279866558 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7f4a98053440 con 0x5560efd98750 2024-03-19T01:16:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.833+0000 7f4ab7385740 10 client.26914 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:04.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.833+0000 7f4ab7385740 10 client.26914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:16:04.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.833+0000 7f4ab7385740 20 client.26914 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:04.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.834+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560efe13fe0 0x5560efe16400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:04.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.834+0000 7f4ab7385740 4 client.26914 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:16:04.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.834+0000 7f4ab7385740 1 -- 172.21.15.69:0/1279866558 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560efcffc10 con 0x5560efe13fe0 2024-03-19T01:16:04.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.834+0000 7f4ab1c00640 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560efe13fe0 0x5560efe16400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:04.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.834+0000 7f4ab1c00640 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560efe13fe0 0x5560efe16400 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e7e091acf18da54 server_cookie=5ab2bd182e6f9637 in_seq=0 out_seq=0 2024-03-19T01:16:04.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.834+0000 7f4a93400640 10 client.26914 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:04.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4a93400640 1 -- 172.21.15.69:0/1279866558 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560efcffc10 con 0x5560efe13fe0 2024-03-19T01:16:04.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4a93400640 10 client.26914 handle_command_reply: tid=0 2024-03-19T01:16:04.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 1 client.26914 shutdown 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 1 -- 172.21.15.69:0/1279866558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 msgr2=0x5560efd9b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efd9b220 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4a98001eb0 tx=0x7f4a98005600 comp rx=0 tx=0).stop 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 1 -- 172.21.15.69:0/1279866558 shutdown_connections 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560efe13fe0 0x5560efe16400 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560efda1330 0x5560efd9b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560efd98750 0x5560efd9b220 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 1 --2- 172.21.15.69:0/1279866558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560efd97e30 0x5560efd9ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 1 -- 172.21.15.69:0/1279866558 >> 172.21.15.69:0/1279866558 conn(0x5560efd92560 msgr2=0x5560efd8d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4a93400640 0 client.26914 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 1 -- 172.21.15.69:0/1279866558 shutdown_connections 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 1 -- 172.21.15.69:0/1279866558 wait complete. 2024-03-19T01:16:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.837+0000 7f4ab7385740 20 client.26914 trim_cache size 0 max 16384 2024-03-19T01:16:04.841 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:16:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.838+0000 7f4ab4600640 1 -- 172.21.15.69:0/2477033240 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4a940a2900 msgr2=0x7f4a940a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.838+0000 7f4ab4600640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4a940a2900 0x7f4a940a4dc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4000f80 tx=0x7f4aa4012d30 comp rx=0 tx=0).stop 2024-03-19T01:16:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.838+0000 7f4ab4600640 1 -- 172.21.15.69:0/2477033240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aac068a20 msgr2=0x7f4aac198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:04.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.838+0000 7f4ab4600640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aac068a20 0x7f4aac198120 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000e9f0 tx=0x7f4aa000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.838+0000 7f4ab4600640 1 -- 172.21.15.69:0/2477033240 shutdown_connections 2024-03-19T01:16:04.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.838+0000 7f4ab4600640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aac107870 0x7f4aac19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.838+0000 7f4ab4600640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4aac0693e0 0x7f4aac198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.838+0000 7f4ab4600640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4a940a2900 0x7f4a940a4dc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.838+0000 7f4ab4600640 1 --2- 172.21.15.69:0/2477033240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4aac068a20 0x7f4aac198120 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:04.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.838+0000 7f4ab4600640 1 -- 172.21.15.69:0/2477033240 >> 172.21.15.69:0/2477033240 conn(0x7f4aac077820 msgr2=0x7f4aac0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:04.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.839+0000 7f4ab4600640 1 -- 172.21.15.69:0/2477033240 shutdown_connections 2024-03-19T01:16:04.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:04.839+0000 7f4ab4600640 1 -- 172.21.15.69:0/2477033240 wait complete. 2024-03-19T01:16:05.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:05 smithi086 ceph-mon[66829]: pgmap v165: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 108 op/s 2024-03-19T01:16:05.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:05 smithi069 ceph-mon[31727]: pgmap v165: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 108 op/s 2024-03-19T01:16:05.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:05 smithi163 ceph-mon[67428]: pgmap v165: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 108 op/s 2024-03-19T01:16:05.859 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:16:05.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:16:05.860 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:16:06.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.090+0000 7f8029a00640 1 -- 172.21.15.69:0/3091884708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8024101680 msgr2=0x7f8024101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.090+0000 7f8029a00640 1 --2- 172.21.15.69:0/3091884708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8024101680 0x7f8024101b00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8010009890 tx=0x7f801002bb70 comp rx=0 tx=0).stop 2024-03-19T01:16:06.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.090+0000 7f8029a00640 1 -- 172.21.15.69:0/3091884708 shutdown_connections 2024-03-19T01:16:06.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.090+0000 7f8029a00640 1 --2- 172.21.15.69:0/3091884708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024102040 0x7f802410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.090+0000 7f8029a00640 1 --2- 172.21.15.69:0/3091884708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8024101680 0x7f8024101b00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.090+0000 7f8029a00640 1 --2- 172.21.15.69:0/3091884708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024107690 0x7f8024107a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.090+0000 7f8029a00640 1 -- 172.21.15.69:0/3091884708 >> 172.21.15.69:0/3091884708 conn(0x7f80240fd190 msgr2=0x7f80240ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.090+0000 7f8029a00640 1 -- 172.21.15.69:0/3091884708 shutdown_connections 2024-03-19T01:16:06.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.091+0000 7f8029a00640 1 -- 172.21.15.69:0/3091884708 wait complete. 2024-03-19T01:16:06.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.092+0000 7f8029a00640 1 Processor -- start 2024-03-19T01:16:06.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.092+0000 7f8029a00640 1 -- start start 2024-03-19T01:16:06.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.092+0000 7f8029a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8024101680 0x7f802419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.092+0000 7f8029a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024102040 0x7f802419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.092+0000 7f8029a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024107690 0x7f802419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.092+0000 7f8029a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8024111aa0 con 0x7f8024101680 2024-03-19T01:16:06.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.092+0000 7f8029a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8024111920 con 0x7f8024102040 2024-03-19T01:16:06.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.092+0000 7f8029a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8024111c20 con 0x7f8024107690 2024-03-19T01:16:06.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.093+0000 7f8029000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024107690 0x7f802419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.093+0000 7f8023e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8024101680 0x7f802419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.093+0000 7f8023400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024102040 0x7f802419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.093+0000 7f8029000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024107690 0x7f802419ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48282/0 (socket says 172.21.15.69:48282) 2024-03-19T01:16:06.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.093+0000 7f8029000640 1 -- 172.21.15.69:0/3054114758 learned_addr learned my addr 172.21.15.69:0/3054114758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:06.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.093+0000 7f8029000640 1 -- 172.21.15.69:0/3054114758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024102040 msgr2=0x7f802419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.093+0000 7f8029000640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024102040 0x7f802419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.093+0000 7f8029000640 1 -- 172.21.15.69:0/3054114758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8024101680 msgr2=0x7f802419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.093+0000 7f8029000640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8024101680 0x7f802419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.093+0000 7f8029000640 1 -- 172.21.15.69:0/3054114758 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802419f170 con 0x7f8024107690 2024-03-19T01:16:06.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.093+0000 7f8023400640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024102040 0x7f802419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:06.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.094+0000 7f8029000640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024107690 0x7f802419ea40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f801c00eb10 tx=0x7f801c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.094+0000 7f8020c00640 1 -- 172.21.15.69:0/3054114758 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c00ce20 con 0x7f8024107690 2024-03-19T01:16:06.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.094+0000 7f8029a00640 1 -- 172.21.15.69:0/3054114758 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802419f460 con 0x7f8024107690 2024-03-19T01:16:06.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.094+0000 7f8020c00640 1 -- 172.21.15.69:0/3054114758 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f801c005020 con 0x7f8024107690 2024-03-19T01:16:06.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.094+0000 7f8020c00640 1 -- 172.21.15.69:0/3054114758 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c010690 con 0x7f8024107690 2024-03-19T01:16:06.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.094+0000 7f8029a00640 1 -- 172.21.15.69:0/3054114758 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80241a6cf0 con 0x7f8024107690 2024-03-19T01:16:06.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.098+0000 7f802de1e740 1 Processor -- start 2024-03-19T01:16:06.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.098+0000 7f8020c00640 1 -- 172.21.15.69:0/3054114758 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f801c005540 con 0x7f8024107690 2024-03-19T01:16:06.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.099+0000 7f802de1e740 1 -- start start 2024-03-19T01:16:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.099+0000 7f802de1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 0x55c3015030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.099+0000 7f802de1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c30150bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.099+0000 7f802de1e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 0x55c30150e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.099+0000 7f802de1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c301465fb0 con 0x55c301502ce0 2024-03-19T01:16:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.099+0000 7f802de1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c30140eab0 con 0x55c301503600 2024-03-19T01:16:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.099+0000 7f802de1e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c301462050 con 0x55c30150c1e0 2024-03-19T01:16:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.099+0000 7f8023e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c30150bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8023e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c30150bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36958/0 (socket says 172.21.15.69:36958) 2024-03-19T01:16:06.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8023e00640 1 -- 172.21.15.69:0/977998112 learned_addr learned my addr 172.21.15.69:0/977998112 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8023400640 1 --2- 172.21.15.69:0/977998112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 0x55c3015030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8023400640 1 --2- 172.21.15.69:0/977998112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 0x55c30150e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8023e00640 1 -- 172.21.15.69:0/977998112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 msgr2=0x55c30150e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8023e00640 1 --2- 172.21.15.69:0/977998112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 0x55c30150e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8023e00640 1 -- 172.21.15.69:0/977998112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 msgr2=0x55c3015030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8023e00640 1 --2- 172.21.15.69:0/977998112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 0x55c3015030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8023e00640 1 -- 172.21.15.69:0/977998112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c30142ce00 con 0x55c301503600 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8023400640 1 --2- 172.21.15.69:0/977998112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 0x55c30150e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8023e00640 1 --2- 172.21.15.69:0/977998112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c30150bca0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8014009c50 tx=0x7f8014011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90010b2629040a3d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.100+0000 7f8020c00640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80080a2970 0x7f80080a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.101+0000 7f8019600640 1 -- 172.21.15.69:0/977998112 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801401e070 con 0x55c301503600 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.101+0000 7f8020c00640 1 -- 172.21.15.69:0/3054114758 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f801c0cf950 con 0x7f8024107690 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.101+0000 7f8019600640 1 -- 172.21.15.69:0/977998112 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8014019b10 con 0x55c301503600 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.101+0000 7f8019600640 1 -- 172.21.15.69:0/977998112 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801400eb10 con 0x55c301503600 2024-03-19T01:16:06.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.101+0000 7f8023400640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80080a2970 0x7f80080a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.102+0000 7f802de1e740 1 -- 172.21.15.69:0/977998112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 msgr2=0x55c30150bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.102+0000 7f802de1e740 1 --2- 172.21.15.69:0/977998112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c30150bca0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8014009c50 tx=0x7f8014011810 comp rx=0 tx=0).stop 2024-03-19T01:16:06.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.102+0000 7f8023400640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80080a2970 0x7f80080a4e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8010000c00 tx=0x7f801002e040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.102+0000 7f802de1e740 1 -- 172.21.15.69:0/977998112 shutdown_connections 2024-03-19T01:16:06.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.102+0000 7f802de1e740 1 --2- 172.21.15.69:0/977998112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 0x55c30150e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.102+0000 7f802de1e740 1 --2- 172.21.15.69:0/977998112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c30150bca0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.102+0000 7f802de1e740 1 --2- 172.21.15.69:0/977998112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 0x55c3015030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.102+0000 7f802de1e740 1 -- 172.21.15.69:0/977998112 >> 172.21.15.69:0/977998112 conn(0x55c3014fd410 msgr2=0x55c3014fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.102+0000 7f802de1e740 1 -- 172.21.15.69:0/977998112 shutdown_connections 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.102+0000 7f802de1e740 1 -- 172.21.15.69:0/977998112 wait complete. 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.102+0000 7f802de1e740 1 -- start start 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.103+0000 7f802de1e740 1 Processor -- start 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.103+0000 7f802de1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 0x55c3015091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.103+0000 7f802de1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c301509730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.103+0000 7f802de1e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 0x55c301509c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.103+0000 7f802de1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c30150efd0 con 0x55c301502ce0 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.103+0000 7f802de1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c301421f00 con 0x55c301503600 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.103+0000 7f802de1e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c30150f150 con 0x55c30150c1e0 2024-03-19T01:16:06.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.103+0000 7f802de1e740 10 client.0 _ll_register_callbacks cb 0x55c3014218d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.103+0000 7f8023400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c301509730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.103+0000 7f8023e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 0x55c3015091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8023e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 0x55c3015091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46662/0 (socket says 172.21.15.69:46662) 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8023e00640 1 -- 172.21.15.69:0/4274055666 learned_addr learned my addr 172.21.15.69:0/4274055666 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8023400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c301509730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36964/0 (socket says 172.21.15.69:36964) 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8029000640 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 0x55c301509c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8023400640 1 -- 172.21.15.69:0/4274055666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 msgr2=0x55c301509c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8023400640 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 0x55c301509c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8023400640 1 -- 172.21.15.69:0/4274055666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 msgr2=0x55c3015091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8023400640 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 0x55c3015091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8023400640 1 -- 172.21.15.69:0/4274055666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3012427e0 con 0x55c301503600 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8023e00640 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 0x55c3015091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8029000640 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 0x55c301509c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f8023400640 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c301509730 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8010005940 tx=0x7f8010005890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34bf7d1a3c5f7291 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.104+0000 7f800e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.105+0000 7f800e000640 1 -- 172.21.15.69:0/4274055666 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010052020 con 0x55c301503600 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.105+0000 7f800e000640 1 -- 172.21.15.69:0/4274055666 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f801002c7d0 con 0x55c301503600 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.105+0000 7f802de1e740 1 -- 172.21.15.69:0/4274055666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8014019b10 con 0x55c301503600 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.105+0000 7f800e000640 1 -- 172.21.15.69:0/4274055666 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f801004a040 con 0x55c301503600 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.105+0000 7f802de1e740 10 client.26975 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.105+0000 7f802de1e740 1 -- 172.21.15.69:0/4274055666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c3012a2f70 con 0x55c301503600 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.105+0000 7f800e000640 1 -- 172.21.15.69:0/4274055666 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8014019b10 con 0x55c301503600 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.105+0000 7f800e000640 1 -- 172.21.15.69:0/4274055666 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7f801004d030 con 0x55c301503600 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.106+0000 7f802de1e740 10 client.26975 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.106+0000 7f802de1e740 10 client.26975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.106+0000 7f802de1e740 20 client.26975 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.106+0000 7f802de1e740 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c30151c530 0x55c30151e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.106+0000 7f802de1e740 4 client.26975 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:16:06.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.106+0000 7f802de1e740 1 -- 172.21.15.69:0/4274055666 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3014734f0 con 0x55c30151c530 2024-03-19T01:16:06.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.106+0000 7f8023e00640 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c30151c530 0x55c30151e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.106+0000 7f8023e00640 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c30151c530 0x55c30151e9c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d30009d6047a65 server_cookie=db09cd4854e4e1a0 in_seq=0 out_seq=0 2024-03-19T01:16:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.106+0000 7f800e000640 10 client.26975 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f800e000640 1 -- 172.21.15.69:0/4274055666 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c3014734f0 con 0x55c30151c530 2024-03-19T01:16:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f800e000640 10 client.26975 handle_command_reply: tid=0 2024-03-19T01:16:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f802de1e740 1 client.26975 shutdown 2024-03-19T01:16:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f802de1e740 1 -- 172.21.15.69:0/4274055666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 msgr2=0x55c301509730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f802de1e740 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c301509730 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8010005940 tx=0x7f8010005890 comp rx=0 tx=0).stop 2024-03-19T01:16:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f802de1e740 1 -- 172.21.15.69:0/4274055666 shutdown_connections 2024-03-19T01:16:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f802de1e740 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c30151c530 0x55c30151e9c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f802de1e740 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c30150c1e0 0x55c301509c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f802de1e740 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c301503600 0x55c301509730 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f802de1e740 1 --2- 172.21.15.69:0/4274055666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301502ce0 0x55c3015091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f800e000640 0 client.26975 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:06.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f802de1e740 1 -- 172.21.15.69:0/4274055666 >> 172.21.15.69:0/4274055666 conn(0x55c3014fd410 msgr2=0x55c3014ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.110+0000 7f802de1e740 1 -- 172.21.15.69:0/4274055666 shutdown_connections 2024-03-19T01:16:06.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.111+0000 7f802de1e740 1 -- 172.21.15.69:0/4274055666 wait complete. 2024-03-19T01:16:06.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.111+0000 7f802de1e740 20 client.26975 trim_cache size 0 max 16384 2024-03-19T01:16:06.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.147+0000 7f802de1e740 1 Processor -- start 2024-03-19T01:16:06.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.147+0000 7f802de1e740 1 -- start start 2024-03-19T01:16:06.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f802de1e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c30150b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f802de1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301503600 0x55c301517d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f802de1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 0x55c3015242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f802de1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80241b0d30 con 0x55c301503600 2024-03-19T01:16:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f802de1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80241b29f0 con 0x55c30150c1e0 2024-03-19T01:16:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f802de1e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f80241b1400 con 0x55c301502ce0 2024-03-19T01:16:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f8023e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c30150b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f8023e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c30150b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48294/0 (socket says 172.21.15.69:48294) 2024-03-19T01:16:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f8023e00640 1 -- 172.21.15.69:0/3264547646 learned_addr learned my addr 172.21.15.69:0/3264547646 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f8023400640 1 --2- 172.21.15.69:0/3264547646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301503600 0x55c301517d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f8029000640 1 --2- 172.21.15.69:0/3264547646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 0x55c3015242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f8023e00640 1 -- 172.21.15.69:0/3264547646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 msgr2=0x55c3015242d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f8023e00640 1 --2- 172.21.15.69:0/3264547646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 0x55c3015242d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f8023e00640 1 -- 172.21.15.69:0/3264547646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301503600 msgr2=0x55c301517d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f8023e00640 1 --2- 172.21.15.69:0/3264547646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301503600 0x55c301517d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.148+0000 7f8023e00640 1 -- 172.21.15.69:0/3264547646 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3015081a0 con 0x55c301502ce0 2024-03-19T01:16:06.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.149+0000 7f8023e00640 1 --2- 172.21.15.69:0/3264547646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c30150b920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8014013010 tx=0x7f801400e6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25f4dbbcc539a438 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.149+0000 7f8018c00640 1 -- 172.21.15.69:0/3264547646 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014031070 con 0x55c301502ce0 2024-03-19T01:16:06.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.149+0000 7f8018c00640 1 -- 172.21.15.69:0/3264547646 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8014019740 con 0x55c301502ce0 2024-03-19T01:16:06.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.149+0000 7f8018c00640 1 -- 172.21.15.69:0/3264547646 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801401e040 con 0x55c301502ce0 2024-03-19T01:16:06.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.150+0000 7f802de1e740 1 -- 172.21.15.69:0/3264547646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 msgr2=0x55c30150b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.150+0000 7f802de1e740 1 --2- 172.21.15.69:0/3264547646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c30150b920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8014013010 tx=0x7f801400e6e0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.150+0000 7f802de1e740 1 -- 172.21.15.69:0/3264547646 shutdown_connections 2024-03-19T01:16:06.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.150+0000 7f802de1e740 1 --2- 172.21.15.69:0/3264547646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 0x55c3015242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.150+0000 7f802de1e740 1 --2- 172.21.15.69:0/3264547646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301503600 0x55c301517d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.150+0000 7f802de1e740 1 --2- 172.21.15.69:0/3264547646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c30150b920 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.150+0000 7f802de1e740 1 -- 172.21.15.69:0/3264547646 >> 172.21.15.69:0/3264547646 conn(0x55c3014fd410 msgr2=0x55c3014ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.150+0000 7f802de1e740 1 -- 172.21.15.69:0/3264547646 shutdown_connections 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.150+0000 7f802de1e740 1 -- 172.21.15.69:0/3264547646 wait complete. 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f802de1e740 1 -- start start 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f802de1e740 1 Processor -- start 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f802de1e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c301514590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f802de1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301503600 0x55c301511370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f802de1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 0x55c3015118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f802de1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80241b2260 con 0x55c301503600 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f802de1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80241b1ad0 con 0x55c30150c1e0 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f802de1e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f80241aece0 con 0x55c301502ce0 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f802de1e740 10 client.0 _ll_register_callbacks cb 0x7f80100096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:06.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f8023e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c301514590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f8029000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 0x55c3015118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f8023e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c301514590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48302/0 (socket says 172.21.15.69:48302) 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f8029000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 0x55c3015118b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36988/0 (socket says 172.21.15.69:36988) 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.151+0000 7f8023e00640 1 -- 172.21.15.69:0/2836426585 learned_addr learned my addr 172.21.15.69:0/2836426585 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f8023400640 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301503600 0x55c301511370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f8029000640 1 -- 172.21.15.69:0/2836426585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 msgr2=0x55c301514590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f8029000640 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c301514590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f8029000640 1 -- 172.21.15.69:0/2836426585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301503600 msgr2=0x55c301511370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f8029000640 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301503600 0x55c301511370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f8029000640 1 -- 172.21.15.69:0/2836426585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c30113a170 con 0x55c30150c1e0 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f8023e00640 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c301514590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f8029000640 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 0x55c3015118b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f801c0ccc80 tx=0x7f801c09b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d50f812e86d8d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f800e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f800e000640 1 -- 172.21.15.69:0/2836426585 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c0b9060 con 0x55c30150c1e0 2024-03-19T01:16:06.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f802de1e740 1 -- 172.21.15.69:0/2836426585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8014019740 con 0x55c30150c1e0 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.152+0000 7f800e000640 1 -- 172.21.15.69:0/2836426585 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f801c09b1e0 con 0x55c30150c1e0 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f800e000640 1 -- 172.21.15.69:0/2836426585 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f801c0b3070 con 0x55c30150c1e0 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f802de1e740 10 client.26981 fetch_fsmap learned FSMap version 37 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f802de1e740 1 -- 172.21.15.69:0/2836426585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c3015836b0 con 0x55c30150c1e0 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f800e000640 1 -- 172.21.15.69:0/2836426585 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8014019740 con 0x55c30150c1e0 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f800e000640 1 -- 172.21.15.69:0/2836426585 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8414+0+0 (secure 0 0 0) 0x7f801c09aa50 con 0x55c30150c1e0 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f802de1e740 10 client.26981 fetch_fsmap finished waiting for FSMap version 37 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f802de1e740 10 client.26981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f802de1e740 20 client.26981 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f802de1e740 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c301583e90 0x55c3015862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f802de1e740 4 client.26981 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f802de1e740 1 -- 172.21.15.69:0/2836426585 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c30146d840 con 0x55c301583e90 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.153+0000 7f8023e00640 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c301583e90 0x55c3015862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.154+0000 7f8023e00640 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c301583e90 0x55c3015862b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=132ba3775f43bc5c server_cookie=392742771c5d1af8 in_seq=0 out_seq=0 2024-03-19T01:16:06.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.154+0000 7f800e000640 10 client.26981 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:06.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.540+0000 7f800e000640 1 -- 172.21.15.69:0/2836426585 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c30146d840 con 0x55c301583e90 2024-03-19T01:16:06.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.540+0000 7f800e000640 10 client.26981 handle_command_reply: tid=0 2024-03-19T01:16:06.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.541+0000 7f802de1e740 1 client.26981 shutdown 2024-03-19T01:16:06.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.541+0000 7f802de1e740 1 -- 172.21.15.69:0/2836426585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 msgr2=0x55c3015118b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.541+0000 7f802de1e740 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 0x55c3015118b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f801c0ccc80 tx=0x7f801c09b610 comp rx=0 tx=0).stop 2024-03-19T01:16:06.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.541+0000 7f802de1e740 1 -- 172.21.15.69:0/2836426585 shutdown_connections 2024-03-19T01:16:06.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.541+0000 7f802de1e740 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c301583e90 0x55c3015862b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.541+0000 7f802de1e740 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c30150c1e0 0x55c3015118b0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.541+0000 7f802de1e740 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c301503600 0x55c301511370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.541+0000 7f802de1e740 1 --2- 172.21.15.69:0/2836426585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c301502ce0 0x55c301514590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.541+0000 7f802de1e740 1 -- 172.21.15.69:0/2836426585 >> 172.21.15.69:0/2836426585 conn(0x55c3014fd410 msgr2=0x55c3015091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.541+0000 7f800e000640 0 client.26981 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:06.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.542+0000 7f802de1e740 1 -- 172.21.15.69:0/2836426585 shutdown_connections 2024-03-19T01:16:06.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.542+0000 7f802de1e740 1 -- 172.21.15.69:0/2836426585 wait complete. 2024-03-19T01:16:06.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.542+0000 7f802de1e740 20 client.26981 trim_cache size 0 max 16384 2024-03-19T01:16:06.544 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:16:06.544 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:16:06.544 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "712b3e5e-ade1-40d7-9117-0b98fdf40a46", 2024-03-19T01:16:06.544 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:16:06.544 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:16:06.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.543+0000 7f8029a00640 1 -- 172.21.15.69:0/3054114758 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80080a2970 msgr2=0x7f80080a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.543+0000 7f8029a00640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80080a2970 0x7f80080a4e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8010000c00 tx=0x7f801002e040 comp rx=0 tx=0).stop 2024-03-19T01:16:06.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.543+0000 7f8029a00640 1 -- 172.21.15.69:0/3054114758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024107690 msgr2=0x7f802419ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.543+0000 7f8029a00640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024107690 0x7f802419ea40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f801c00eb10 tx=0x7f801c00ef10 comp rx=0 tx=0).stop 2024-03-19T01:16:06.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.543+0000 7f8029a00640 1 -- 172.21.15.69:0/3054114758 shutdown_connections 2024-03-19T01:16:06.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.543+0000 7f8029a00640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024107690 0x7f802419ea40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.543+0000 7f8029a00640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80080a2970 0x7f80080a4e30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.543+0000 7f8029a00640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8024102040 0x7f802419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.543+0000 7f8029a00640 1 --2- 172.21.15.69:0/3054114758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8024101680 0x7f802419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.543+0000 7f8029a00640 1 -- 172.21.15.69:0/3054114758 >> 172.21.15.69:0/3054114758 conn(0x7f80240fd190 msgr2=0x7f80240fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.544+0000 7f8029a00640 1 -- 172.21.15.69:0/3054114758 shutdown_connections 2024-03-19T01:16:06.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.544+0000 7f8029a00640 1 -- 172.21.15.69:0/3054114758 wait complete. 2024-03-19T01:16:06.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:16:06.572 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:16:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.857+0000 7f24c3c00640 1 -- 172.21.15.69:0/975420720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24bc109b40 msgr2=0x7f24bc109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.857+0000 7f24c3c00640 1 --2- 172.21.15.69:0/975420720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24bc109b40 0x7f24bc109f20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009930 tx=0x7f24b002bc30 comp rx=0 tx=0).stop 2024-03-19T01:16:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.857+0000 7f24c3c00640 1 -- 172.21.15.69:0/975420720 shutdown_connections 2024-03-19T01:16:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.857+0000 7f24c3c00640 1 --2- 172.21.15.69:0/975420720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24bc0ffe30 0x7f24bc1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.857+0000 7f24c3c00640 1 --2- 172.21.15.69:0/975420720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24bc10a4f0 0x7f24bc0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.857+0000 7f24c3c00640 1 --2- 172.21.15.69:0/975420720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24bc109b40 0x7f24bc109f20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.857+0000 7f24c3c00640 1 -- 172.21.15.69:0/975420720 >> 172.21.15.69:0/975420720 conn(0x7f24bc076870 msgr2=0x7f24bc0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.857+0000 7f24c3c00640 1 -- 172.21.15.69:0/975420720 shutdown_connections 2024-03-19T01:16:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.858+0000 7f24c3c00640 1 -- 172.21.15.69:0/975420720 wait complete. 2024-03-19T01:16:06.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.859+0000 7f24c3c00640 1 Processor -- start 2024-03-19T01:16:06.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.859+0000 7f24c3c00640 1 -- start start 2024-03-19T01:16:06.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.860+0000 7f24c3c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24bc0ffe30 0x7f24bc19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.860+0000 7f24c3c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24bc109b40 0x7f24bc19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.860+0000 7f24c3c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24bc1a2fd0 0x7f24bc1a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.860+0000 7f24c3c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc118210 con 0x7f24bc109b40 2024-03-19T01:16:06.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.860+0000 7f24c3c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc118090 con 0x7f24bc1a2fd0 2024-03-19T01:16:06.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.860+0000 7f24c3c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc118390 con 0x7f24bc0ffe30 2024-03-19T01:16:06.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.861+0000 7f24bbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24bc109b40 0x7f24bc19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.861+0000 7f24bbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24bc109b40 0x7f24bc19ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46688/0 (socket says 172.21.15.69:46688) 2024-03-19T01:16:06.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.861+0000 7f24c1200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24bc0ffe30 0x7f24bc19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.861+0000 7f24bbe00640 1 -- 172.21.15.69:0/1408117846 learned_addr learned my addr 172.21.15.69:0/1408117846 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:06.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.861+0000 7f24c1c00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24bc1a2fd0 0x7f24bc1a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.861+0000 7f24bbe00640 1 -- 172.21.15.69:0/1408117846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24bc0ffe30 msgr2=0x7f24bc19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.861+0000 7f24bbe00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24bc0ffe30 0x7f24bc19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.861+0000 7f24bbe00640 1 -- 172.21.15.69:0/1408117846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24bc1a2fd0 msgr2=0x7f24bc1a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.861+0000 7f24bbe00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24bc1a2fd0 0x7f24bc1a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.861+0000 7f24bbe00640 1 -- 172.21.15.69:0/1408117846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc1a3ac0 con 0x7f24bc109b40 2024-03-19T01:16:06.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.862+0000 7f24c1c00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24bc1a2fd0 0x7f24bc1a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:06.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.862+0000 7f24bbe00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24bc109b40 0x7f24bc19ec50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24a800e9c0 tx=0x7f24a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.862+0000 7f24b9600640 1 -- 172.21.15.69:0/1408117846 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a800cd80 con 0x7f24bc109b40 2024-03-19T01:16:06.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.862+0000 7f24b9600640 1 -- 172.21.15.69:0/1408117846 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a8005500 con 0x7f24bc109b40 2024-03-19T01:16:06.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.862+0000 7f24b9600640 1 -- 172.21.15.69:0/1408117846 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8010690 con 0x7f24bc109b40 2024-03-19T01:16:06.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.862+0000 7f24c3c00640 1 -- 172.21.15.69:0/1408117846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24bc1a3db0 con 0x7f24bc109b40 2024-03-19T01:16:06.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.863+0000 7f24c3c00640 1 -- 172.21.15.69:0/1408117846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24bc1ab550 con 0x7f24bc109b40 2024-03-19T01:16:06.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.867+0000 7f24b9600640 1 -- 172.21.15.69:0/1408117846 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f24a80046c0 con 0x7f24bc109b40 2024-03-19T01:16:06.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.869+0000 7f24c69c7740 1 Processor -- start 2024-03-19T01:16:06.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.870+0000 7f24c69c7740 1 -- start start 2024-03-19T01:16:06.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.870+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7857d40 0x55ffc7858120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.870+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7858660 0x55ffc7860d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.870+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc7863630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.870+0000 7f24c69c7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ffc77bc2c0 con 0x55ffc7861240 2024-03-19T01:16:06.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.870+0000 7f24c69c7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7764b60 con 0x55ffc7858660 2024-03-19T01:16:06.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.870+0000 7f24c69c7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ffc77b8360 con 0x55ffc7857d40 2024-03-19T01:16:06.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.871+0000 7f24c1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7857d40 0x55ffc7858120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.871+0000 7f24c1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7857d40 0x55ffc7858120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48332/0 (socket says 172.21.15.69:48332) 2024-03-19T01:16:06.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.871+0000 7f24c1c00640 1 -- 172.21.15.69:0/91960023 learned_addr learned my addr 172.21.15.69:0/91960023 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:06.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.871+0000 7f24bbe00640 1 --2- 172.21.15.69:0/91960023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc7863630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.871+0000 7f24c1200640 1 --2- 172.21.15.69:0/91960023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7858660 0x55ffc7860d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.871+0000 7f24b9600640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f24a40a2a30 0x7f24a40a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.871+0000 7f24c1c00640 1 -- 172.21.15.69:0/91960023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7858660 msgr2=0x55ffc7860d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.871+0000 7f24c1c00640 1 --2- 172.21.15.69:0/91960023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7858660 0x55ffc7860d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.871+0000 7f24c1c00640 1 -- 172.21.15.69:0/91960023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 msgr2=0x55ffc7863630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.871+0000 7f24c1c00640 1 --2- 172.21.15.69:0/91960023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc7863630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.872+0000 7f24c1c00640 1 -- 172.21.15.69:0/91960023 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc759a7d0 con 0x55ffc7857d40 2024-03-19T01:16:06.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.872+0000 7f24b9600640 1 -- 172.21.15.69:0/1408117846 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f24a80cf590 con 0x7f24bc109b40 2024-03-19T01:16:06.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.872+0000 7f24c1c00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f24a40a2a30 0x7f24a40a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.872+0000 7f24c1c00640 1 --2- 172.21.15.69:0/91960023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7857d40 0x55ffc7858120 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f24b400f790 tx=0x7f24b400fc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71f3726083e82e86 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.873+0000 7f24ae000640 1 -- 172.21.15.69:0/91960023 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b400de00 con 0x55ffc7857d40 2024-03-19T01:16:06.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.873+0000 7f24ae000640 1 -- 172.21.15.69:0/91960023 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b4010910 con 0x55ffc7857d40 2024-03-19T01:16:06.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.873+0000 7f24ae000640 1 -- 172.21.15.69:0/91960023 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b401abf0 con 0x55ffc7857d40 2024-03-19T01:16:06.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.873+0000 7f24c1c00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f24a40a2a30 0x7f24a40a4ef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24b401e000 tx=0x7f24b401af20 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.875+0000 7f24c69c7740 1 -- 172.21.15.69:0/91960023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7857d40 msgr2=0x55ffc7858120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.875+0000 7f24c69c7740 1 --2- 172.21.15.69:0/91960023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7857d40 0x55ffc7858120 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f24b400f790 tx=0x7f24b400fc50 comp rx=0 tx=0).stop 2024-03-19T01:16:06.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.875+0000 7f24c69c7740 1 -- 172.21.15.69:0/91960023 shutdown_connections 2024-03-19T01:16:06.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.875+0000 7f24c69c7740 1 --2- 172.21.15.69:0/91960023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc7863630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.875+0000 7f24c69c7740 1 --2- 172.21.15.69:0/91960023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7858660 0x55ffc7860d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.875+0000 7f24c69c7740 1 --2- 172.21.15.69:0/91960023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7857d40 0x55ffc7858120 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.875+0000 7f24c69c7740 1 -- 172.21.15.69:0/91960023 >> 172.21.15.69:0/91960023 conn(0x55ffc7852470 msgr2=0x55ffc7852850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.875+0000 7f24c69c7740 1 -- 172.21.15.69:0/91960023 shutdown_connections 2024-03-19T01:16:06.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.876+0000 7f24c69c7740 1 -- 172.21.15.69:0/91960023 wait complete. 2024-03-19T01:16:06.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.877+0000 7f24c69c7740 1 -- start start 2024-03-19T01:16:06.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.877+0000 7f24c69c7740 1 Processor -- start 2024-03-19T01:16:06.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.879+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7857d40 0x55ffc785e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.879+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.879+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7861240 0x55ffc785ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24bbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24bbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48348/0 (socket says 172.21.15.69:48348) 2024-03-19T01:16:06.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24bbe00640 1 -- 172.21.15.69:0/2688487277 learned_addr learned my addr 172.21.15.69:0/2688487277 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:06.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24c69c7740 1 -- 172.21.15.69:0/2688487277 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7864030 con 0x55ffc7857d40 2024-03-19T01:16:06.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24c1c00640 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7861240 0x55ffc785ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24c1200640 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7857d40 0x55ffc785e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24c69c7740 1 -- 172.21.15.69:0/2688487277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7777fc0 con 0x55ffc7861240 2024-03-19T01:16:06.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24c69c7740 1 -- 172.21.15.69:0/2688487277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ffc78641b0 con 0x55ffc7858660 2024-03-19T01:16:06.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24bbe00640 1 -- 172.21.15.69:0/2688487277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7861240 msgr2=0x55ffc785ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24bbe00640 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7861240 0x55ffc785ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24bbe00640 1 -- 172.21.15.69:0/2688487277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7857d40 msgr2=0x55ffc785e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24bbe00640 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7857d40 0x55ffc785e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.880+0000 7f24bbe00640 1 -- 172.21.15.69:0/2688487277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc74d03f0 con 0x55ffc7858660 2024-03-19T01:16:06.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.881+0000 7f24bbe00640 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785e810 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f24a8097c30 tx=0x7f24a80d2af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=282c314939ff469b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.881+0000 7f24c69c7740 10 client.0 _ll_register_callbacks cb 0x55ffc7777990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.881+0000 7f24a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.881+0000 7f24a2a00640 1 -- 172.21.15.69:0/2688487277 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a809b5a0 con 0x55ffc7858660 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.881+0000 7f24a2a00640 1 -- 172.21.15.69:0/2688487277 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f24a80cf110 con 0x55ffc7858660 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.882+0000 7f24c69c7740 1 -- 172.21.15.69:0/2688487277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b4010910 con 0x55ffc7858660 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.882+0000 7f24a2a00640 1 -- 172.21.15.69:0/2688487277 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f24a80b4020 con 0x55ffc7858660 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.882+0000 7f24c69c7740 10 client.26941 fetch_fsmap learned FSMap version 38 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.882+0000 7f24c69c7740 1 -- 172.21.15.69:0/2688487277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=38}) -- 0x55ffc74900f0 con 0x55ffc7858660 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.882+0000 7f24a2a00640 1 -- 172.21.15.69:0/2688487277 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7f24a80a3ae0 con 0x55ffc7858660 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.882+0000 7f24c69c7740 10 client.26941 fetch_fsmap finished waiting for FSMap version 38 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.883+0000 7f24c69c7740 10 client.26941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.883+0000 7f24c69c7740 20 client.26941 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:06.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.883+0000 7f24c69c7740 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ffc78713f0 0x55ffc78738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.883+0000 7f24c69c7740 4 client.26941 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.883+0000 7f24c69c7740 1 -- 172.21.15.69:0/2688487277 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffc775d610 con 0x55ffc78713f0 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.883+0000 7f24c1200640 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ffc78713f0 0x55ffc78738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.884+0000 7f24c1200640 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ffc78713f0 0x55ffc78738b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c85951619ea081a server_cookie=7243434a72e2507c in_seq=0 out_seq=0 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.884+0000 7f24a2a00640 10 client.26941 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.886+0000 7f24a2a00640 1 -- 172.21.15.69:0/2688487277 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ffc775d610 con 0x55ffc78713f0 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.886+0000 7f24a2a00640 10 client.26941 handle_command_reply: tid=0 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.886+0000 7f24c69c7740 1 client.26941 shutdown 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.887+0000 7f24c69c7740 1 -- 172.21.15.69:0/2688487277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 msgr2=0x55ffc785e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.887+0000 7f24c69c7740 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785e810 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f24a8097c30 tx=0x7f24a80d2af0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.887+0000 7f24c69c7740 1 -- 172.21.15.69:0/2688487277 shutdown_connections 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.887+0000 7f24c69c7740 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ffc78713f0 0x55ffc78738b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.887+0000 7f24c69c7740 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7861240 0x55ffc785ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.887+0000 7f24c69c7740 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785e810 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.887+0000 7f24c69c7740 1 --2- 172.21.15.69:0/2688487277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7857d40 0x55ffc785e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.887+0000 7f24a2a00640 0 client.26941 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:06.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.887+0000 7f24c69c7740 1 -- 172.21.15.69:0/2688487277 >> 172.21.15.69:0/2688487277 conn(0x55ffc7852470 msgr2=0x55ffc78541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.887+0000 7f24c69c7740 1 -- 172.21.15.69:0/2688487277 shutdown_connections 2024-03-19T01:16:06.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.887+0000 7f24c69c7740 1 -- 172.21.15.69:0/2688487277 wait complete. 2024-03-19T01:16:06.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.888+0000 7f24c69c7740 20 client.26941 trim_cache size 0 max 16384 2024-03-19T01:16:06.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.929+0000 7f24c69c7740 1 Processor -- start 2024-03-19T01:16:06.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.929+0000 7f24c69c7740 1 -- start start 2024-03-19T01:16:06.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.929+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7857d40 0x55ffc7860990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.929+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc78516b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.929+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc785eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.929+0000 7f24c69c7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1b53a0 con 0x55ffc7861240 2024-03-19T01:16:06.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.929+0000 7f24c69c7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1b7060 con 0x55ffc7857d40 2024-03-19T01:16:06.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.929+0000 7f24c69c7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1b5a70 con 0x55ffc7858660 2024-03-19T01:16:06.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.930+0000 7f24c1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7857d40 0x55ffc7860990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.930+0000 7f24c1c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc785eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.930+0000 7f24c1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7857d40 0x55ffc7860990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37026/0 (socket says 172.21.15.69:37026) 2024-03-19T01:16:06.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.930+0000 7f24c1200640 1 -- 172.21.15.69:0/3481501815 learned_addr learned my addr 172.21.15.69:0/3481501815 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:06.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.930+0000 7f24bbe00640 1 --2- 172.21.15.69:0/3481501815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc78516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.931+0000 7f24c1c00640 1 -- 172.21.15.69:0/3481501815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 msgr2=0x55ffc78516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.931+0000 7f24c1c00640 1 --2- 172.21.15.69:0/3481501815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc78516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.931+0000 7f24c1c00640 1 -- 172.21.15.69:0/3481501815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7857d40 msgr2=0x55ffc7860990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.931+0000 7f24c1c00640 1 --2- 172.21.15.69:0/3481501815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7857d40 0x55ffc7860990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.931+0000 7f24c1c00640 1 -- 172.21.15.69:0/3481501815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc7858490 con 0x55ffc7861240 2024-03-19T01:16:06.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.932+0000 7f24c1c00640 1 --2- 172.21.15.69:0/3481501815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc785eb70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f24b400d710 tx=0x7f24b4026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28182c00f581e7b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.933+0000 7f24ad600640 1 -- 172.21.15.69:0/3481501815 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b402c040 con 0x55ffc7861240 2024-03-19T01:16:06.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.933+0000 7f24ad600640 1 -- 172.21.15.69:0/3481501815 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b401a400 con 0x55ffc7861240 2024-03-19T01:16:06.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.933+0000 7f24ad600640 1 -- 172.21.15.69:0/3481501815 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4010d20 con 0x55ffc7861240 2024-03-19T01:16:06.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.934+0000 7f24c69c7740 1 -- 172.21.15.69:0/3481501815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 msgr2=0x55ffc785eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.934+0000 7f24c69c7740 1 --2- 172.21.15.69:0/3481501815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc785eb70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f24b400d710 tx=0x7f24b4026000 comp rx=0 tx=0).stop 2024-03-19T01:16:06.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.935+0000 7f24c69c7740 1 -- 172.21.15.69:0/3481501815 shutdown_connections 2024-03-19T01:16:06.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.935+0000 7f24c69c7740 1 --2- 172.21.15.69:0/3481501815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc785eb70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.935+0000 7f24c69c7740 1 --2- 172.21.15.69:0/3481501815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc78516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.935+0000 7f24c69c7740 1 --2- 172.21.15.69:0/3481501815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7857d40 0x55ffc7860990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.935+0000 7f24c69c7740 1 -- 172.21.15.69:0/3481501815 >> 172.21.15.69:0/3481501815 conn(0x55ffc7852470 msgr2=0x55ffc78538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.935+0000 7f24c69c7740 1 -- 172.21.15.69:0/3481501815 shutdown_connections 2024-03-19T01:16:06.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.936+0000 7f24c69c7740 1 -- 172.21.15.69:0/3481501815 wait complete. 2024-03-19T01:16:06.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.936+0000 7f24c69c7740 1 -- start start 2024-03-19T01:16:06.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.937+0000 7f24c69c7740 1 Processor -- start 2024-03-19T01:16:06.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.937+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7857d40 0x55ffc785b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.937+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.937+0000 7f24c69c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc78d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.937+0000 7f24c69c7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1b68d0 con 0x55ffc7861240 2024-03-19T01:16:06.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.937+0000 7f24c69c7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1b6140 con 0x55ffc7857d40 2024-03-19T01:16:06.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.937+0000 7f24c69c7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc1b3320 con 0x55ffc7858660 2024-03-19T01:16:06.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.937+0000 7f24c69c7740 10 client.0 _ll_register_callbacks cb 0x7f24a8009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:06.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.938+0000 7f24bbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.938+0000 7f24bbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48366/0 (socket says 172.21.15.69:48366) 2024-03-19T01:16:06.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.938+0000 7f24bbe00640 1 -- 172.21.15.69:0/669130419 learned_addr learned my addr 172.21.15.69:0/669130419 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:06.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.938+0000 7f24c1c00640 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc78d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.938+0000 7f24c1200640 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7857d40 0x55ffc785b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.938+0000 7f24bbe00640 1 -- 172.21.15.69:0/669130419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7857d40 msgr2=0x55ffc785b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.938+0000 7f24bbe00640 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7857d40 0x55ffc785b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.938+0000 7f24bbe00640 1 -- 172.21.15.69:0/669130419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 msgr2=0x55ffc78d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:06.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.938+0000 7f24bbe00640 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc78d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.938+0000 7f24bbe00640 1 -- 172.21.15.69:0/669130419 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc78d0d70 con 0x55ffc7858660 2024-03-19T01:16:06.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.939+0000 7f24bbe00640 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785bc00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f24a809b4d0 tx=0x7f24a80d1960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d453646c0ca2054 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:06.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.939+0000 7f24a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:16:06.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.939+0000 7f24a2a00640 1 -- 172.21.15.69:0/669130419 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a809b130 con 0x55ffc7858660 2024-03-19T01:16:06.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.939+0000 7f24a2a00640 1 -- 172.21.15.69:0/669130419 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f24a80d6070 con 0x55ffc7858660 2024-03-19T01:16:06.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.939+0000 7f24c69c7740 1 -- 172.21.15.69:0/669130419 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b401a400 con 0x55ffc7858660 2024-03-19T01:16:06.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.940+0000 7f24a2a00640 1 -- 172.21.15.69:0/669130419 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f24a809b310 con 0x55ffc7858660 2024-03-19T01:16:06.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.940+0000 7f24c69c7740 10 client.26947 fetch_fsmap learned FSMap version 38 2024-03-19T01:16:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.940+0000 7f24c69c7740 1 -- 172.21.15.69:0/669130419 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ffc78d8100 con 0x55ffc7858660 2024-03-19T01:16:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.940+0000 7f24a2a00640 1 -- 172.21.15.69:0/669130419 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b401a400 con 0x55ffc7858660 2024-03-19T01:16:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.941+0000 7f24a2a00640 1 -- 172.21.15.69:0/669130419 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7f24a809ac20 con 0x55ffc7858660 2024-03-19T01:16:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.941+0000 7f24c69c7740 10 client.26947 fetch_fsmap finished waiting for FSMap version 38 2024-03-19T01:16:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.941+0000 7f24c69c7740 10 client.26947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:16:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.941+0000 7f24c69c7740 20 client.26947 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.941+0000 7f24c69c7740 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ffc78d86d0 0x55ffc78daaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.941+0000 7f24c69c7740 4 client.26947 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:16:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.941+0000 7f24c69c7740 1 -- 172.21.15.69:0/669130419 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ffc77c9800 con 0x55ffc78d86d0 2024-03-19T01:16:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.941+0000 7f24c1200640 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ffc78d86d0 0x55ffc78daaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:06.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.943+0000 7f24c1200640 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ffc78d86d0 0x55ffc78daaf0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77701d3ae5970c9 server_cookie=de9d312ec99eb96c in_seq=0 out_seq=0 2024-03-19T01:16:06.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.943+0000 7f24a2a00640 10 client.26947 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:06.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.947+0000 7f24a2a00640 1 -- 172.21.15.69:0/669130419 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55ffc77c9800 con 0x55ffc78d86d0 2024-03-19T01:16:06.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.947+0000 7f24a2a00640 10 client.26947 handle_command_reply: tid=0 2024-03-19T01:16:06.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.947+0000 7f24c69c7740 1 client.26947 shutdown 2024-03-19T01:16:06.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.947+0000 7f24c69c7740 1 -- 172.21.15.69:0/669130419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 msgr2=0x55ffc785bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.947+0000 7f24c69c7740 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785bc00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f24a809b4d0 tx=0x7f24a80d1960 comp rx=0 tx=0).stop 2024-03-19T01:16:06.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.948+0000 7f24c69c7740 1 -- 172.21.15.69:0/669130419 shutdown_connections 2024-03-19T01:16:06.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.948+0000 7f24c69c7740 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ffc78d86d0 0x55ffc78daaf0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.948+0000 7f24c69c7740 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc7861240 0x55ffc78d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.948+0000 7f24c69c7740 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc7858660 0x55ffc785bc00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.948+0000 7f24c69c7740 1 --2- 172.21.15.69:0/669130419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffc7857d40 0x55ffc785b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.948+0000 7f24c69c7740 1 -- 172.21.15.69:0/669130419 >> 172.21.15.69:0/669130419 conn(0x55ffc7852470 msgr2=0x55ffc7851530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.948+0000 7f24a2a00640 0 client.26947 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:06.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.949+0000 7f24c69c7740 1 -- 172.21.15.69:0/669130419 shutdown_connections 2024-03-19T01:16:06.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.949+0000 7f24c69c7740 1 -- 172.21.15.69:0/669130419 wait complete. 2024-03-19T01:16:06.950 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:16:06.950 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3988 inodes in the stack)", 2024-03-19T01:16:06.950 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:16:06.950 INFO:teuthology.orchestra.run.smithi069.stdout: "712b3e5e-ade1-40d7-9117-0b98fdf40a46": { 2024-03-19T01:16:06.951 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:16:06.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "712b3e5e-ade1-40d7-9117-0b98fdf40a46", 2024-03-19T01:16:06.951 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:16:06.951 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:16:06.951 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:16:06.951 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.949+0000 7f24c69c7740 20 client.26947 trim_cache size 0 max 16384 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.950+0000 7f24c3c00640 1 -- 172.21.15.69:0/1408117846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f24a40a2a30 msgr2=0x7f24a40a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.950+0000 7f24c3c00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f24a40a2a30 0x7f24a40a4ef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24b401e000 tx=0x7f24b401af20 comp rx=0 tx=0).stop 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.950+0000 7f24c3c00640 1 -- 172.21.15.69:0/1408117846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24bc109b40 msgr2=0x7f24bc19ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.950+0000 7f24c3c00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24bc109b40 0x7f24bc19ec50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24a800e9c0 tx=0x7f24a800ee80 comp rx=0 tx=0).stop 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.951+0000 7f24c3c00640 1 -- 172.21.15.69:0/1408117846 shutdown_connections 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.951+0000 7f24c3c00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f24a40a2a30 0x7f24a40a4ef0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.951+0000 7f24c3c00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24bc1a2fd0 0x7f24bc1a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.951+0000 7f24c3c00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24bc109b40 0x7f24bc19ec50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.951+0000 7f24c3c00640 1 --2- 172.21.15.69:0/1408117846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24bc0ffe30 0x7f24bc19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.951+0000 7f24c3c00640 1 -- 172.21.15.69:0/1408117846 >> 172.21.15.69:0/1408117846 conn(0x7f24bc076870 msgr2=0x7f24bc1078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:06.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.951+0000 7f24c3c00640 1 -- 172.21.15.69:0/1408117846 shutdown_connections 2024-03-19T01:16:06.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:06.951+0000 7f24c3c00640 1 -- 172.21.15.69:0/1408117846 wait complete. 2024-03-19T01:16:06.977 INFO:tasks.cephfs.filesystem:scrub status for tag:712b3e5e-ade1-40d7-9117-0b98fdf40a46 - {'path': '/', 'tag': '712b3e5e-ade1-40d7-9117-0b98fdf40a46', 'options': 'recursive,force'} 2024-03-19T01:16:07.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: pgmap v166: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 76 op/s 2024-03-19T01:16:07.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:07.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:07.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: pgmap v166: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 76 op/s 2024-03-19T01:16:07.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:07.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:16:07.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:07.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: pgmap v166: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 76 op/s 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:07.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:07.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:08.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:08.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:08.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: pgmap v167: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 77 op/s 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:09.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: pgmap v167: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 77 op/s 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:09.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: pgmap v167: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 77 op/s 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:09.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:10.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:10.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:10.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:10.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: pgmap v168: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:11.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: pgmap v168: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:11.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: pgmap v168: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:11.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:11.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:12.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:12.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: pgmap v169: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 106 op/s 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:13.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: pgmap v169: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 106 op/s 2024-03-19T01:16:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: pgmap v169: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 106 op/s 2024-03-19T01:16:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:13.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:13.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:13.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:13.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:13.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:13.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.463 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:14.463 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.463 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:14.463 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.464 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:14.464 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.464 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:14.464 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.464 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:14.464 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:14.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: pgmap v170: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: pgmap v170: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-03-19T01:16:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: pgmap v170: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-03-19T01:16:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:15.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:15.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:16.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:16.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: pgmap v171: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 95 op/s 2024-03-19T01:16:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:17.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: pgmap v171: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 95 op/s 2024-03-19T01:16:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:17.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:17.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:17.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: pgmap v171: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 95 op/s 2024-03-19T01:16:17.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:17.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:17.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:18.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:18.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:18.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:18.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:18.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:18.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:18.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: pgmap v172: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 129 op/s 2024-03-19T01:16:19.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:19.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:19.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: pgmap v172: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 129 op/s 2024-03-19T01:16:19.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:19.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:19.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:19.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:19.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: pgmap v172: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 129 op/s 2024-03-19T01:16:19.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:19.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:19.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:20.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:20.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:20.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:20.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:20.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:20.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:20.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.665 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:21.665 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.665 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:21.665 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.665 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:21.665 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:21.666 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.666 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.666 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: pgmap v173: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.2 MiB/s wr, 117 op/s 2024-03-19T01:16:21.666 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:21.666 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.666 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:21.666 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: pgmap v173: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.2 MiB/s wr, 117 op/s 2024-03-19T01:16:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:21.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:21.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:21.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:21.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: pgmap v173: 129 pgs: 129 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.2 MiB/s wr, 117 op/s 2024-03-19T01:16:21.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:21.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:21.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:21.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:23.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:23 smithi086 ceph-mon[66829]: pgmap v174: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 319 op/s 2024-03-19T01:16:23.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:23 smithi069 ceph-mon[31727]: pgmap v174: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 319 op/s 2024-03-19T01:16:23.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:23 smithi163 ceph-mon[67428]: pgmap v174: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 319 op/s 2024-03-19T01:16:25.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:25 smithi086 ceph-mon[66829]: pgmap v175: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.0 MiB/s wr, 299 op/s 2024-03-19T01:16:25.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:25 smithi069 ceph-mon[31727]: pgmap v175: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.0 MiB/s wr, 299 op/s 2024-03-19T01:16:25.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:25 smithi163 ceph-mon[67428]: pgmap v175: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.0 MiB/s wr, 299 op/s 2024-03-19T01:16:27.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:27 smithi086 ceph-mon[66829]: pgmap v176: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 339 op/s 2024-03-19T01:16:27.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:27 smithi069 ceph-mon[31727]: pgmap v176: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 339 op/s 2024-03-19T01:16:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:27 smithi163 ceph-mon[67428]: pgmap v176: 129 pgs: 129 active+clean; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.5 MiB/s wr, 339 op/s 2024-03-19T01:16:29.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:29 smithi086 ceph-mon[66829]: pgmap v177: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 859 op/s 2024-03-19T01:16:29.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:29 smithi069 ceph-mon[31727]: pgmap v177: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 859 op/s 2024-03-19T01:16:29.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:29 smithi163 ceph-mon[67428]: pgmap v177: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 859 op/s 2024-03-19T01:16:31.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:31 smithi086 ceph-mon[66829]: pgmap v178: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 810 op/s 2024-03-19T01:16:31.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:31 smithi069 ceph-mon[31727]: pgmap v178: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 810 op/s 2024-03-19T01:16:31.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:31 smithi163 ceph-mon[67428]: pgmap v178: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 810 op/s 2024-03-19T01:16:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:32 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:32.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:32 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:32.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:32 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:33.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:33 smithi086 ceph-mon[66829]: pgmap v179: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 832 op/s 2024-03-19T01:16:33.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:33 smithi069 ceph-mon[31727]: pgmap v179: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 832 op/s 2024-03-19T01:16:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:33 smithi163 ceph-mon[67428]: pgmap v179: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 832 op/s 2024-03-19T01:16:35.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:35.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:35.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:35.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:35.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:35.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:35.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:35.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:35.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:35.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:35.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:35.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: pgmap v180: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.8 MiB/s wr, 637 op/s 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:36.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: pgmap v180: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.8 MiB/s wr, 637 op/s 2024-03-19T01:16:36.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:36.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: pgmap v180: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.8 MiB/s wr, 637 op/s 2024-03-19T01:16:36.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:36.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:36.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:36.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:36.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:36.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:36.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:36.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:36.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:16:36.978 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:16:37.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.206+0000 7f304be00640 1 -- 172.21.15.69:0/338321846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f304c073bd0 msgr2=0x7f304c073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:37.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.206+0000 7f304be00640 1 --2- 172.21.15.69:0/338321846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f304c073bd0 0x7f304c073fb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f303c009900 tx=0x7f303c02bcd0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.206+0000 7f304be00640 1 -- 172.21.15.69:0/338321846 shutdown_connections 2024-03-19T01:16:37.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.206+0000 7f304be00640 1 --2- 172.21.15.69:0/338321846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f304c10df50 0x7f304c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.206+0000 7f304be00640 1 --2- 172.21.15.69:0/338321846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f304c0744f0 0x7f304c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.206+0000 7f304be00640 1 --2- 172.21.15.69:0/338321846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f304c073bd0 0x7f304c073fb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.206+0000 7f304be00640 1 -- 172.21.15.69:0/338321846 >> 172.21.15.69:0/338321846 conn(0x7f304c0fd1c0 msgr2=0x7f304c0ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:37.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.207+0000 7f304be00640 1 -- 172.21.15.69:0/338321846 shutdown_connections 2024-03-19T01:16:37.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.207+0000 7f304be00640 1 -- 172.21.15.69:0/338321846 wait complete. 2024-03-19T01:16:37.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.208+0000 7f304be00640 1 Processor -- start 2024-03-19T01:16:37.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.208+0000 7f304be00640 1 -- start start 2024-03-19T01:16:37.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.209+0000 7f304be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f304c073bd0 0x7f304c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.209+0000 7f304be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f304c0744f0 0x7f304c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.209+0000 7f304be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f304c10df50 0x7f304c19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.209+0000 7f304be00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f304c113cc0 con 0x7f304c10df50 2024-03-19T01:16:37.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.209+0000 7f304be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f304c113b40 con 0x7f304c073bd0 2024-03-19T01:16:37.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.209+0000 7f304be00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f304c113e40 con 0x7f304c0744f0 2024-03-19T01:16:37.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.209+0000 7f304a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f304c0744f0 0x7f304c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.209+0000 7f304aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f304c073bd0 0x7f304c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.209+0000 7f304a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f304c0744f0 0x7f304c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48368/0 (socket says 172.21.15.69:48368) 2024-03-19T01:16:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.209+0000 7f304a000640 1 -- 172.21.15.69:0/3546147578 learned_addr learned my addr 172.21.15.69:0/3546147578 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.209+0000 7f304b400640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f304c10df50 0x7f304c19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.210+0000 7f304a000640 1 -- 172.21.15.69:0/3546147578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f304c073bd0 msgr2=0x7f304c19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.210+0000 7f304a000640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f304c073bd0 0x7f304c19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.210+0000 7f304a000640 1 -- 172.21.15.69:0/3546147578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f304c10df50 msgr2=0x7f304c19ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:37.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.210+0000 7f304a000640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f304c10df50 0x7f304c19ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.210+0000 7f304a000640 1 -- 172.21.15.69:0/3546147578 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c19f1b0 con 0x7f304c0744f0 2024-03-19T01:16:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.210+0000 7f304b400640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f304c10df50 0x7f304c19ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:16:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.210+0000 7f304a000640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f304c0744f0 0x7f304c19a700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f303800e9f0 tx=0x7f303800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.211+0000 7f3050a00640 1 -- 172.21.15.69:0/3546147578 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303800cd80 con 0x7f304c0744f0 2024-03-19T01:16:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.211+0000 7f3050a00640 1 -- 172.21.15.69:0/3546147578 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3038005500 con 0x7f304c0744f0 2024-03-19T01:16:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.211+0000 7f3050a00640 1 -- 172.21.15.69:0/3546147578 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038010690 con 0x7f304c0744f0 2024-03-19T01:16:37.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.211+0000 7f304be00640 1 -- 172.21.15.69:0/3546147578 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c19f4a0 con 0x7f304c0744f0 2024-03-19T01:16:37.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.211+0000 7f304be00640 1 -- 172.21.15.69:0/3546147578 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c1a6d30 con 0x7f304c0744f0 2024-03-19T01:16:37.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.215+0000 7f3050a00640 1 -- 172.21.15.69:0/3546147578 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f30380046c0 con 0x7f304c0744f0 2024-03-19T01:16:37.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.216+0000 7f3054bec740 1 Processor -- start 2024-03-19T01:16:37.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.217+0000 7f3054bec740 1 -- start start 2024-03-19T01:16:37.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.217+0000 7f3054bec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c6061f30 0x55b8c6062310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.217+0000 7f3054bec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6062850 0x55b8c606aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.217+0000 7f3054bec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c606b430 0x55b8c606d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.217+0000 7f3054bec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5fc62c0 con 0x55b8c6061f30 2024-03-19T01:16:37.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.217+0000 7f3054bec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5f6eb60 con 0x55b8c606b430 2024-03-19T01:16:37.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.217+0000 7f3054bec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5fc2360 con 0x55b8c6062850 2024-03-19T01:16:37.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.217+0000 7f3050a00640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f30300a2a30 0x7f30300a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.217+0000 7f3050a00640 1 -- 172.21.15.69:0/3546147578 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f30380cf590 con 0x7f304c0744f0 2024-03-19T01:16:37.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.217+0000 7f304a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c606b430 0x55b8c606d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.218+0000 7f304a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c606b430 0x55b8c606d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:59312/0 (socket says 172.21.15.69:59312) 2024-03-19T01:16:37.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.218+0000 7f304a000640 1 -- 172.21.15.69:0/1997873026 learned_addr learned my addr 172.21.15.69:0/1997873026 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:37.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.218+0000 7f304b400640 1 --2- 172.21.15.69:0/1997873026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c6061f30 0x55b8c6062310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.218+0000 7f304aa00640 1 --2- 172.21.15.69:0/1997873026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6062850 0x55b8c606aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.218+0000 7f304a000640 1 -- 172.21.15.69:0/1997873026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6062850 msgr2=0x55b8c606aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:37.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.218+0000 7f304a000640 1 --2- 172.21.15.69:0/1997873026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6062850 0x55b8c606aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.218+0000 7f304a000640 1 -- 172.21.15.69:0/1997873026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c6061f30 msgr2=0x55b8c6062310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:37.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.218+0000 7f304a000640 1 --2- 172.21.15.69:0/1997873026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c6061f30 0x55b8c6062310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.218+0000 7f304a000640 1 -- 172.21.15.69:0/1997873026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8c5da47d0 con 0x55b8c606b430 2024-03-19T01:16:37.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.218+0000 7f304b400640 1 --2- 172.21.15.69:0/1997873026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c6061f30 0x55b8c6062310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:16:37.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.218+0000 7f304b400640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f30300a2a30 0x7f30300a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.219+0000 7f304a000640 1 --2- 172.21.15.69:0/1997873026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c606b430 0x55b8c606d820 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f30380cc980 tx=0x7f3038004bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a474dac7d5be0a0c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:37.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.219+0000 7f3037e00640 1 -- 172.21.15.69:0/1997873026 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30380b3020 con 0x55b8c606b430 2024-03-19T01:16:37.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.219+0000 7f3037e00640 1 -- 172.21.15.69:0/1997873026 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f303809bf30 con 0x55b8c606b430 2024-03-19T01:16:37.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.219+0000 7f3037e00640 1 -- 172.21.15.69:0/1997873026 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303809cd90 con 0x55b8c606b430 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.220+0000 7f3054bec740 1 -- 172.21.15.69:0/1997873026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c606b430 msgr2=0x55b8c606d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.220+0000 7f304b400640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f30300a2a30 0x7f30300a4ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f30440094d0 tx=0x7f3044004a40 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.220+0000 7f3054bec740 1 --2- 172.21.15.69:0/1997873026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c606b430 0x55b8c606d820 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f30380cc980 tx=0x7f3038004bb0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.220+0000 7f3054bec740 1 -- 172.21.15.69:0/1997873026 shutdown_connections 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.220+0000 7f3054bec740 1 --2- 172.21.15.69:0/1997873026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c606b430 0x55b8c606d820 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.220+0000 7f3054bec740 1 --2- 172.21.15.69:0/1997873026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6062850 0x55b8c606aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.220+0000 7f3054bec740 1 --2- 172.21.15.69:0/1997873026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c6061f30 0x55b8c6062310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.220+0000 7f3054bec740 1 -- 172.21.15.69:0/1997873026 >> 172.21.15.69:0/1997873026 conn(0x55b8c605c660 msgr2=0x55b8c605ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.220+0000 7f3054bec740 1 -- 172.21.15.69:0/1997873026 shutdown_connections 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.221+0000 7f3054bec740 1 -- 172.21.15.69:0/1997873026 wait complete. 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.221+0000 7f3054bec740 1 -- start start 2024-03-19T01:16:37.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.221+0000 7f3054bec740 1 Processor -- start 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f3054bec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c6068460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f3054bec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6062850 0x55b8c60689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f3054bec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6068ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f3054bec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b8c606e220 con 0x55b8c606b430 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f3054bec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5f81fc0 con 0x55b8c6062850 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f3054bec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b8c606e3a0 con 0x55b8c6061f30 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f3054bec740 10 client.0 _ll_register_callbacks cb 0x55b8c5f81990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f304aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c6068460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f304aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c6068460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48392/0 (socket says 172.21.15.69:48392) 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f304aa00640 1 -- 172.21.15.69:0/4256020680 learned_addr learned my addr 172.21.15.69:0/4256020680 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f304a000640 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6062850 0x55b8c60689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.222+0000 7f304b400640 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6068ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.223+0000 7f304aa00640 1 -- 172.21.15.69:0/4256020680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6062850 msgr2=0x55b8c60689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.223+0000 7f304aa00640 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6062850 0x55b8c60689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.223+0000 7f304aa00640 1 -- 172.21.15.69:0/4256020680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 msgr2=0x55b8c6068ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.223+0000 7f304aa00640 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6068ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.223+0000 7f304aa00640 1 -- 172.21.15.69:0/4256020680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8c5cda3f0 con 0x55b8c6061f30 2024-03-19T01:16:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.223+0000 7f304a000640 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6062850 0x55b8c60689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:16:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.223+0000 7f304b400640 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6068ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:16:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.223+0000 7f304aa00640 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c6068460 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f303c009230 tx=0x7f303c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab2fdf1a7b03c9f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.223+0000 7f3034c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:16:37.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.224+0000 7f3034c00640 1 -- 172.21.15.69:0/4256020680 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c044070 con 0x55b8c6061f30 2024-03-19T01:16:37.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.224+0000 7f3034c00640 1 -- 172.21.15.69:0/4256020680 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f303c0046c0 con 0x55b8c6061f30 2024-03-19T01:16:37.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.224+0000 7f3054bec740 1 -- 172.21.15.69:0/4256020680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f303809bf30 con 0x55b8c6061f30 2024-03-19T01:16:37.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.224+0000 7f3034c00640 1 -- 172.21.15.69:0/4256020680 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f303c0048c0 con 0x55b8c6061f30 2024-03-19T01:16:37.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.224+0000 7f3054bec740 10 client.26959 fetch_fsmap learned FSMap version 40 2024-03-19T01:16:37.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.224+0000 7f3054bec740 1 -- 172.21.15.69:0/4256020680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b8c5c9a0f0 con 0x55b8c6061f30 2024-03-19T01:16:37.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.224+0000 7f3034c00640 1 -- 172.21.15.69:0/4256020680 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f303809bf30 con 0x55b8c6061f30 2024-03-19T01:16:37.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.225+0000 7f3034c00640 1 -- 172.21.15.69:0/4256020680 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) ==== 8410+0+0 (secure 0 0 0) 0x7f303c003710 con 0x55b8c6061f30 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.225+0000 7f3054bec740 10 client.26959 fetch_fsmap finished waiting for FSMap version 40 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.225+0000 7f3054bec740 10 client.26959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.225+0000 7f3054bec740 20 client.26959 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.225+0000 7f3054bec740 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b8c607b7b0 0x55b8c607dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.225+0000 7f3054bec740 4 client.26959 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.225+0000 7f3054bec740 1 -- 172.21.15.69:0/4256020680 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8c5f67610 con 0x55b8c607b7b0 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.225+0000 7f304a000640 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b8c607b7b0 0x55b8c607dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.226+0000 7f304a000640 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b8c607b7b0 0x55b8c607dba0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52500892bf870e86 server_cookie=e4aefb4e25314b41 in_seq=0 out_seq=0 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.226+0000 7f3034c00640 10 client.26959 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.226+0000 7f3034c00640 1 -- 172.21.15.69:0/4256020680 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b8c5f67610 con 0x55b8c607b7b0 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.226+0000 7f3034c00640 10 client.26959 handle_command_reply: tid=0 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 1 client.26959 shutdown 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 1 -- 172.21.15.69:0/4256020680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 msgr2=0x55b8c6068460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:37.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c6068460 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f303c009230 tx=0x7f303c004610 comp rx=0 tx=0).stop 2024-03-19T01:16:37.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 1 -- 172.21.15.69:0/4256020680 shutdown_connections 2024-03-19T01:16:37.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b8c607b7b0 0x55b8c607dba0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6068ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6062850 0x55b8c60689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 1 --2- 172.21.15.69:0/4256020680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c6068460 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3034c00640 0 client.26959 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:37.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 1 -- 172.21.15.69:0/4256020680 >> 172.21.15.69:0/4256020680 conn(0x55b8c605c660 msgr2=0x55b8c605e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:37.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 1 -- 172.21.15.69:0/4256020680 shutdown_connections 2024-03-19T01:16:37.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 1 -- 172.21.15.69:0/4256020680 wait complete. 2024-03-19T01:16:37.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.227+0000 7f3054bec740 20 client.26959 trim_cache size 0 max 16384 2024-03-19T01:16:37.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.254+0000 7f3054bec740 1 Processor -- start 2024-03-19T01:16:37.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.254+0000 7f3054bec740 1 -- start start 2024-03-19T01:16:37.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.255+0000 7f3054bec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c606ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.255+0000 7f3054bec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6062850 0x55b8c60673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.255+0000 7f3054bec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6076fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.255+0000 7f3054bec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f304c1ac9f0 con 0x55b8c606b430 2024-03-19T01:16:37.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.255+0000 7f3054bec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f304c1ae650 con 0x55b8c6062850 2024-03-19T01:16:37.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.255+0000 7f3054bec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f304c1ad0c0 con 0x55b8c6061f30 2024-03-19T01:16:37.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:37.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:37.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:37.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.255+0000 7f304aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c606ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.255+0000 7f304aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c606ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48404/0 (socket says 172.21.15.69:48404) 2024-03-19T01:16:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.255+0000 7f304aa00640 1 -- 172.21.15.69:0/1816866893 learned_addr learned my addr 172.21.15.69:0/1816866893 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.255+0000 7f304a000640 1 --2- 172.21.15.69:0/1816866893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6062850 0x55b8c60673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.255+0000 7f304b400640 1 --2- 172.21.15.69:0/1816866893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6076fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.256+0000 7f304aa00640 1 -- 172.21.15.69:0/1816866893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6062850 msgr2=0x55b8c60673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.256+0000 7f304aa00640 1 --2- 172.21.15.69:0/1816866893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6062850 0x55b8c60673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.256+0000 7f304aa00640 1 -- 172.21.15.69:0/1816866893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 msgr2=0x55b8c6076fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.256+0000 7f304aa00640 1 --2- 172.21.15.69:0/1816866893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6076fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.256+0000 7f304aa00640 1 -- 172.21.15.69:0/1816866893 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8c607df10 con 0x55b8c6061f30 2024-03-19T01:16:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.256+0000 7f304aa00640 1 --2- 172.21.15.69:0/1816866893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c606ab90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f303c0092a0 tx=0x7f303c02c1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7798ec76d3370252 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.256+0000 7f3037400640 1 -- 172.21.15.69:0/1816866893 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c05a070 con 0x55b8c6061f30 2024-03-19T01:16:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.256+0000 7f3037400640 1 -- 172.21.15.69:0/1816866893 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f303c009520 con 0x55b8c6061f30 2024-03-19T01:16:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.256+0000 7f3037400640 1 -- 172.21.15.69:0/1816866893 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c044070 con 0x55b8c6061f30 2024-03-19T01:16:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.258+0000 7f3054bec740 1 -- 172.21.15.69:0/1816866893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 msgr2=0x55b8c606ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.258+0000 7f3054bec740 1 --2- 172.21.15.69:0/1816866893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c606ab90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f303c0092a0 tx=0x7f303c02c1f0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.258+0000 7f3054bec740 1 -- 172.21.15.69:0/1816866893 shutdown_connections 2024-03-19T01:16:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.258+0000 7f3054bec740 1 --2- 172.21.15.69:0/1816866893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6076fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.258+0000 7f3054bec740 1 --2- 172.21.15.69:0/1816866893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6062850 0x55b8c60673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.258+0000 7f3054bec740 1 --2- 172.21.15.69:0/1816866893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6061f30 0x55b8c606ab90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.258+0000 7f3054bec740 1 -- 172.21.15.69:0/1816866893 >> 172.21.15.69:0/1816866893 conn(0x55b8c605c660 msgr2=0x55b8c605b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.258+0000 7f3054bec740 1 -- 172.21.15.69:0/1816866893 shutdown_connections 2024-03-19T01:16:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.258+0000 7f3054bec740 1 -- 172.21.15.69:0/1816866893 wait complete. 2024-03-19T01:16:37.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.259+0000 7f3054bec740 1 -- start start 2024-03-19T01:16:37.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.259+0000 7f3054bec740 1 Processor -- start 2024-03-19T01:16:37.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f3054bec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6061f30 0x55b8c6065320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f3054bec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6062850 0x55b8c6065860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f3054bec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6065dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f3054bec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f304c1adec0 con 0x55b8c606b430 2024-03-19T01:16:37.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f3054bec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f304c1ad790 con 0x55b8c6061f30 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f3054bec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f304c1aa970 con 0x55b8c6062850 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f3054bec740 10 client.0 _ll_register_callbacks cb 0x7f303c0044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f304aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6061f30 0x55b8c6065320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f304aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6061f30 0x55b8c6065320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:59328/0 (socket says 172.21.15.69:59328) 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f304aa00640 1 -- 172.21.15.69:0/3717164363 learned_addr learned my addr 172.21.15.69:0/3717164363 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f304a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6062850 0x55b8c6065860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.260+0000 7f304b400640 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6065dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.261+0000 7f304a000640 1 -- 172.21.15.69:0/3717164363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6061f30 msgr2=0x55b8c6065320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.261+0000 7f304a000640 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6061f30 0x55b8c6065320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.261+0000 7f304a000640 1 -- 172.21.15.69:0/3717164363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 msgr2=0x55b8c6065dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.261+0000 7f304a000640 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6065dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.261+0000 7f304a000640 1 -- 172.21.15.69:0/3717164363 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8c60e0d90 con 0x55b8c6062850 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.261+0000 7f304a000640 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6062850 0x55b8c6065860 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f303809cad0 tx=0x7f303800c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7dec21103b68be4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:16:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.261+0000 7f3034c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:16:37.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.261+0000 7f3034c00640 1 -- 172.21.15.69:0/3717164363 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30380b3040 con 0x55b8c6062850 2024-03-19T01:16:37.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.261+0000 7f3034c00640 1 -- 172.21.15.69:0/3717164363 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9480+0+0 (secure 0 0 0) 0x7f30380a08a0 con 0x55b8c6062850 2024-03-19T01:16:37.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.261+0000 7f3054bec740 1 -- 172.21.15.69:0/3717164363 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f303c009520 con 0x55b8c6062850 2024-03-19T01:16:37.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.262+0000 7f3034c00640 1 -- 172.21.15.69:0/3717164363 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f303809bf30 con 0x55b8c6062850 2024-03-19T01:16:37.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.262+0000 7f3054bec740 10 client.26965 fetch_fsmap learned FSMap version 40 2024-03-19T01:16:37.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.262+0000 7f3054bec740 1 -- 172.21.15.69:0/3717164363 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b8c60e20a0 con 0x55b8c6062850 2024-03-19T01:16:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.262+0000 7f3034c00640 1 -- 172.21.15.69:0/3717164363 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f303c009520 con 0x55b8c6062850 2024-03-19T01:16:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.262+0000 7f3034c00640 1 -- 172.21.15.69:0/3717164363 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) ==== 8410+0+0 (secure 0 0 0) 0x7f30380ccba0 con 0x55b8c6062850 2024-03-19T01:16:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.262+0000 7f3054bec740 10 client.26965 fetch_fsmap finished waiting for FSMap version 40 2024-03-19T01:16:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.262+0000 7f3054bec740 10 client.26965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:16:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.262+0000 7f3054bec740 20 client.26965 populate_metadata read hostname 'smithi069' 2024-03-19T01:16:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.262+0000 7f3054bec740 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b8c60e24f0 0x55b8c60e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.262+0000 7f3054bec740 4 client.26965 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.262+0000 7f3054bec740 1 -- 172.21.15.69:0/3717164363 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b8c5fd3800 con 0x55b8c60e24f0 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.263+0000 7f304aa00640 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b8c60e24f0 0x55b8c60e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.263+0000 7f304aa00640 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b8c60e24f0 0x55b8c60e48e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e1113a92e14b26 server_cookie=8eab073d4226ac84 in_seq=0 out_seq=0 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.263+0000 7f3034c00640 10 client.26965 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3034c00640 1 -- 172.21.15.69:0/3717164363 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55b8c5fd3800 con 0x55b8c60e24f0 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3034c00640 10 client.26965 handle_command_reply: tid=0 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 1 client.26965 shutdown 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 1 -- 172.21.15.69:0/3717164363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6062850 msgr2=0x55b8c6065860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6062850 0x55b8c6065860 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f303809cad0 tx=0x7f303800c7d0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 1 -- 172.21.15.69:0/3717164363 shutdown_connections 2024-03-19T01:16:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b8c60e24f0 0x55b8c60e48e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b8c606b430 0x55b8c6065dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b8c6062850 0x55b8c6065860 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 1 --2- 172.21.15.69:0/3717164363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c6061f30 0x55b8c6065320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 1 -- 172.21.15.69:0/3717164363 >> 172.21.15.69:0/3717164363 conn(0x55b8c605c660 msgr2=0x55b8c6068420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:37.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3034c00640 0 client.26965 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:16:37.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 1 -- 172.21.15.69:0/3717164363 shutdown_connections 2024-03-19T01:16:37.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 1 -- 172.21.15.69:0/3717164363 wait complete. 2024-03-19T01:16:37.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.264+0000 7f3054bec740 20 client.26965 trim_cache size 0 max 16384 2024-03-19T01:16:37.268 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:16:37.268 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (2618 inodes in the stack)", 2024-03-19T01:16:37.269 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:16:37.269 INFO:teuthology.orchestra.run.smithi069.stdout: "712b3e5e-ade1-40d7-9117-0b98fdf40a46": { 2024-03-19T01:16:37.269 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:16:37.269 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "712b3e5e-ade1-40d7-9117-0b98fdf40a46", 2024-03-19T01:16:37.269 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:16:37.269 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:16:37.269 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:16:37.269 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:16:37.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.265+0000 7f304be00640 1 -- 172.21.15.69:0/3546147578 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f30300a2a30 msgr2=0x7f30300a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:37.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.265+0000 7f304be00640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f30300a2a30 0x7f30300a4ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f30440094d0 tx=0x7f3044004a40 comp rx=0 tx=0).stop 2024-03-19T01:16:37.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.265+0000 7f304be00640 1 -- 172.21.15.69:0/3546147578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f304c0744f0 msgr2=0x7f304c19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:16:37.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.265+0000 7f304be00640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f304c0744f0 0x7f304c19a700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f303800e9f0 tx=0x7f303800edf0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.265+0000 7f304be00640 1 -- 172.21.15.69:0/3546147578 shutdown_connections 2024-03-19T01:16:37.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.265+0000 7f304be00640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f304c10df50 0x7f304c19ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.265+0000 7f304be00640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f304c0744f0 0x7f304c19a700 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.265+0000 7f304be00640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f30300a2a30 0x7f30300a4ef0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.265+0000 7f304be00640 1 --2- 172.21.15.69:0/3546147578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f304c073bd0 0x7f304c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:16:37.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.265+0000 7f304be00640 1 -- 172.21.15.69:0/3546147578 >> 172.21.15.69:0/3546147578 conn(0x7f304c0fd1c0 msgr2=0x7f304c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:16:37.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.266+0000 7f304be00640 1 -- 172.21.15.69:0/3546147578 shutdown_connections 2024-03-19T01:16:37.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:16:37.266+0000 7f304be00640 1 -- 172.21.15.69:0/3546147578 wait complete. 2024-03-19T01:16:37.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:37.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:37.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:37.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:37.286 INFO:tasks.cephfs.filesystem:scrub status for tag:712b3e5e-ade1-40d7-9117-0b98fdf40a46 - {'path': '/', 'tag': '712b3e5e-ade1-40d7-9117-0b98fdf40a46', 'options': 'recursive,force'} 2024-03-19T01:16:37.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:37.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:37.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:37.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:38.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:37 smithi086 ceph-mon[66829]: pgmap v181: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.9 MiB/s wr, 628 op/s 2024-03-19T01:16:38.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:16:38.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:38.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:38.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:37 smithi069 ceph-mon[31727]: pgmap v181: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.9 MiB/s wr, 628 op/s 2024-03-19T01:16:38.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:16:38.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:38.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:37 smithi163 ceph-mon[67428]: pgmap v181: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.9 MiB/s wr, 628 op/s 2024-03-19T01:16:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:16:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:38.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:16:40.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:39 smithi086 ceph-mon[66829]: pgmap v182: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.8 MiB/s wr, 603 op/s 2024-03-19T01:16:40.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:39 smithi069 ceph-mon[31727]: pgmap v182: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.8 MiB/s wr, 603 op/s 2024-03-19T01:16:40.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:39 smithi163 ceph-mon[67428]: pgmap v182: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.8 MiB/s wr, 603 op/s 2024-03-19T01:16:42.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:41 smithi086 ceph-mon[66829]: pgmap v183: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 56 op/s 2024-03-19T01:16:42.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:41 smithi069 ceph-mon[31727]: pgmap v183: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 56 op/s 2024-03-19T01:16:42.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:41 smithi163 ceph-mon[67428]: pgmap v183: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 56 op/s 2024-03-19T01:16:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:43 smithi086 ceph-mon[66829]: pgmap v184: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 259 op/s 2024-03-19T01:16:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:43 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:16:44.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:43 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:16:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:43 smithi069 ceph-mon[31727]: pgmap v184: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 259 op/s 2024-03-19T01:16:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:43 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:16:44.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:43 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:16:44.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:43 smithi163 ceph-mon[67428]: pgmap v184: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 259 op/s 2024-03-19T01:16:44.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:43 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:16:44.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:43 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:16:45.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:44 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:16:45.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:44 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:16:45.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:44 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:16:45.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:44 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:16:45.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:44 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:16:45.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:44 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:16:45.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:44 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:16:45.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:44 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:16:45.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:44 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:16:46.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:45 smithi086 ceph-mon[66829]: pgmap v185: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 325 op/s 2024-03-19T01:16:46.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:45 smithi069 ceph-mon[31727]: pgmap v185: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 325 op/s 2024-03-19T01:16:46.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:45 smithi163 ceph-mon[67428]: pgmap v185: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 325 op/s 2024-03-19T01:16:47.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:46 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:47.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:46 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:47.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:46 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:48.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:47 smithi086 ceph-mon[66829]: pgmap v186: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 391 op/s 2024-03-19T01:16:48.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:47 smithi069 ceph-mon[31727]: pgmap v186: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 391 op/s 2024-03-19T01:16:48.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:47 smithi163 ceph-mon[67428]: pgmap v186: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 391 op/s 2024-03-19T01:16:50.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:49 smithi086 ceph-mon[66829]: pgmap v187: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 843 op/s 2024-03-19T01:16:50.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:49 smithi069 ceph-mon[31727]: pgmap v187: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 843 op/s 2024-03-19T01:16:50.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:49 smithi163 ceph-mon[67428]: pgmap v187: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 843 op/s 2024-03-19T01:16:52.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:51 smithi086 ceph-mon[66829]: pgmap v188: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.6 MiB/s wr, 824 op/s 2024-03-19T01:16:52.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:51 smithi069 ceph-mon[31727]: pgmap v188: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.6 MiB/s wr, 824 op/s 2024-03-19T01:16:52.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:51 smithi163 ceph-mon[67428]: pgmap v188: 129 pgs: 129 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.6 MiB/s wr, 824 op/s 2024-03-19T01:16:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:53 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:53 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:53 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:16:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:54 smithi069 ceph-mon[31727]: pgmap v189: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 1.25k op/s 2024-03-19T01:16:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:54 smithi163 ceph-mon[67428]: pgmap v189: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 1.25k op/s 2024-03-19T01:16:54.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:54 smithi086 ceph-mon[66829]: pgmap v189: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 1.25k op/s 2024-03-19T01:16:56.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:56 smithi069 ceph-mon[31727]: pgmap v190: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 1.14k op/s 2024-03-19T01:16:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:56 smithi163 ceph-mon[67428]: pgmap v190: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 1.14k op/s 2024-03-19T01:16:56.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:56 smithi086 ceph-mon[66829]: pgmap v190: 129 pgs: 129 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 1.14k op/s 2024-03-19T01:16:57.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:57 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:16:57.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:57 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:16:57.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:57 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:16:58.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:58 smithi086 ceph-mon[66829]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 1.34k op/s 2024-03-19T01:16:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:58 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:16:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:58 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:16:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:16:58 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:16:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:58 smithi069 ceph-mon[31727]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 1.34k op/s 2024-03-19T01:16:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:58 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:16:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:58 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:16:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:16:58 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:16:58.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:58 smithi163 ceph-mon[67428]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 1.34k op/s 2024-03-19T01:16:58.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:58 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:16:58.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:58 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:16:58.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:16:58 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:17:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:00 smithi086 ceph-mon[66829]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 2.00k op/s 2024-03-19T01:17:00.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:00 smithi069 ceph-mon[31727]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 2.00k op/s 2024-03-19T01:17:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:00 smithi163 ceph-mon[67428]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 2.00k op/s 2024-03-19T01:17:02.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:02 smithi086 ceph-mon[66829]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.0 MiB/s wr, 1.54k op/s 2024-03-19T01:17:02.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:02 smithi069 ceph-mon[31727]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.0 MiB/s wr, 1.54k op/s 2024-03-19T01:17:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:02 smithi163 ceph-mon[67428]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.0 MiB/s wr, 1.54k op/s 2024-03-19T01:17:04.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:04 smithi086 ceph-mon[66829]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.7 MiB/s wr, 2.10k op/s 2024-03-19T01:17:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:04 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:04.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:04 smithi069 ceph-mon[31727]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.7 MiB/s wr, 2.10k op/s 2024-03-19T01:17:04.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:04 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:04 smithi163 ceph-mon[67428]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.7 MiB/s wr, 2.10k op/s 2024-03-19T01:17:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:04 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:05.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:05 smithi086 ceph-mon[66829]: osdmap e95: 12 total, 12 up, 12 in 2024-03-19T01:17:05.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:05 smithi069 ceph-mon[31727]: osdmap e95: 12 total, 12 up, 12 in 2024-03-19T01:17:05.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:05 smithi163 ceph-mon[67428]: osdmap e95: 12 total, 12 up, 12 in 2024-03-19T01:17:06.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:06 smithi086 ceph-mon[66829]: pgmap v196: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.9 MiB/s wr, 2.13k op/s 2024-03-19T01:17:06.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:06 smithi069 ceph-mon[31727]: pgmap v196: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.9 MiB/s wr, 2.13k op/s 2024-03-19T01:17:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:06 smithi163 ceph-mon[67428]: pgmap v196: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.9 MiB/s wr, 2.13k op/s 2024-03-19T01:17:07.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:17:07.288 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:17:07.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:07 smithi086 ceph-mon[66829]: pgmap v197: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.4 MiB/s wr, 2.05k op/s 2024-03-19T01:17:07.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.509+0000 7f82fa200640 1 -- 172.21.15.69:0/1240935192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f4102850 msgr2=0x7f82f4102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.509+0000 7f82fa200640 1 --2- 172.21.15.69:0/1240935192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f4102850 0x7f82f4102c30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009900 tx=0x7f82e802bcd0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.509+0000 7f82fa200640 1 -- 172.21.15.69:0/1240935192 shutdown_connections 2024-03-19T01:17:07.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.509+0000 7f82fa200640 1 --2- 172.21.15.69:0/1240935192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f410bc70 0x7f82f410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.509+0000 7f82fa200640 1 --2- 172.21.15.69:0/1240935192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f4103170 0x7f82f410b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.509+0000 7f82fa200640 1 --2- 172.21.15.69:0/1240935192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f4102850 0x7f82f4102c30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.509+0000 7f82fa200640 1 -- 172.21.15.69:0/1240935192 >> 172.21.15.69:0/1240935192 conn(0x7f82f40fc070 msgr2=0x7f82f40fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:07.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.509+0000 7f82fa200640 1 -- 172.21.15.69:0/1240935192 shutdown_connections 2024-03-19T01:17:07.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.509+0000 7f82fa200640 1 -- 172.21.15.69:0/1240935192 wait complete. 2024-03-19T01:17:07.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.511+0000 7f82fa200640 1 Processor -- start 2024-03-19T01:17:07.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.511+0000 7f82fa200640 1 -- start start 2024-03-19T01:17:07.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.511+0000 7f82fa200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f4102850 0x7f82f4198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.511+0000 7f82fa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f4103170 0x7f82f41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.511+0000 7f82fa200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f410bc70 0x7f82f419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.511+0000 7f82fa200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4111ac0 con 0x7f82f410bc70 2024-03-19T01:17:07.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.511+0000 7f82fa200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4111940 con 0x7f82f4103170 2024-03-19T01:17:07.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.511+0000 7f82fa200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4111c40 con 0x7f82f4102850 2024-03-19T01:17:07.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f8e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f4102850 0x7f82f4198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f8e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f4102850 0x7f82f4198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46788/0 (socket says 172.21.15.69:46788) 2024-03-19T01:17:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f8e00640 1 -- 172.21.15.69:0/1758166552 learned_addr learned my addr 172.21.15.69:0/1758166552 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f3e00640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f4103170 0x7f82f41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f9800640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f410bc70 0x7f82f419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f3e00640 1 -- 172.21.15.69:0/1758166552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f4102850 msgr2=0x7f82f4198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f3e00640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f4102850 0x7f82f4198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f3e00640 1 -- 172.21.15.69:0/1758166552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f410bc70 msgr2=0x7f82f419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f3e00640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f410bc70 0x7f82f419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f3e00640 1 -- 172.21.15.69:0/1758166552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f419d070 con 0x7f82f4103170 2024-03-19T01:17:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f8e00640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f4102850 0x7f82f4198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f9800640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f410bc70 0x7f82f419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.512+0000 7f82f3e00640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f4103170 0x7f82f41985c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f82e000e7c0 tx=0x7f82e000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:07.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.513+0000 7f82f1600640 1 -- 172.21.15.69:0/1758166552 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0009530 con 0x7f82f4103170 2024-03-19T01:17:07.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.513+0000 7f82f1600640 1 -- 172.21.15.69:0/1758166552 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e000ccc0 con 0x7f82f4103170 2024-03-19T01:17:07.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.513+0000 7f82fa200640 1 -- 172.21.15.69:0/1758166552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f419d360 con 0x7f82f4103170 2024-03-19T01:17:07.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.513+0000 7f82f1600640 1 -- 172.21.15.69:0/1758166552 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0004a00 con 0x7f82f4103170 2024-03-19T01:17:07.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.513+0000 7f82fa200640 1 -- 172.21.15.69:0/1758166552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f41a4bf0 con 0x7f82f4103170 2024-03-19T01:17:07.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.517+0000 7f82fe5a3740 1 Processor -- start 2024-03-19T01:17:07.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82fe5a3740 1 -- start start 2024-03-19T01:17:07.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d1577310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 0x5588d157fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 0x5588d1582820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5588d14db2c0 con 0x5588d1580430 2024-03-19T01:17:07.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5588d1483b60 con 0x5588d1577850 2024-03-19T01:17:07.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5588d14d7360 con 0x5588d1576f30 2024-03-19T01:17:07.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82f1600640 1 -- 172.21.15.69:0/1758166552 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f82e0004ba0 con 0x7f82f4103170 2024-03-19T01:17:07.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82f9800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d1577310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82f9800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d1577310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46802/0 (socket says 172.21.15.69:46802) 2024-03-19T01:17:07.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82f9800640 1 -- 172.21.15.69:0/542503577 learned_addr learned my addr 172.21.15.69:0/542503577 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:07.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.518+0000 7f82f8e00640 1 --2- 172.21.15.69:0/542503577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 0x5588d157fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.519+0000 7f82f9800640 1 -- 172.21.15.69:0/542503577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 msgr2=0x5588d157fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:07.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.519+0000 7f82f3e00640 1 --2- 172.21.15.69:0/542503577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 0x5588d1582820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.519+0000 7f82f9800640 1 --2- 172.21.15.69:0/542503577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 0x5588d157fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.519+0000 7f82f9800640 1 -- 172.21.15.69:0/542503577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 msgr2=0x5588d1582820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:07.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.519+0000 7f82f9800640 1 --2- 172.21.15.69:0/542503577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 0x5588d1582820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.519+0000 7f82f9800640 1 -- 172.21.15.69:0/542503577 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d12b97d0 con 0x5588d1576f30 2024-03-19T01:17:07.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.519+0000 7f82f8e00640 1 --2- 172.21.15.69:0/542503577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 0x5588d157fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:07.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.519+0000 7f82f3e00640 1 --2- 172.21.15.69:0/542503577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 0x5588d1582820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:07.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.519+0000 7f82f9800640 1 --2- 172.21.15.69:0/542503577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d1577310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009fd0 tx=0x7f82ec010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59f3d5364697e81f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:07.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.520+0000 7f82e6000640 1 -- 172.21.15.69:0/542503577 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec011700 con 0x5588d1576f30 2024-03-19T01:17:07.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.520+0000 7f82e6000640 1 -- 172.21.15.69:0/542503577 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ec011d80 con 0x5588d1576f30 2024-03-19T01:17:07.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.520+0000 7f82e6000640 1 -- 172.21.15.69:0/542503577 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00e3c0 con 0x5588d1576f30 2024-03-19T01:17:07.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.521+0000 7f82f1600640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82dc0a29f0 0x7f82dc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.521+0000 7f82fe5a3740 1 -- 172.21.15.69:0/542503577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 msgr2=0x5588d1577310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.521+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/542503577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d1577310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009fd0 tx=0x7f82ec010860 comp rx=0 tx=0).stop 2024-03-19T01:17:07.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.521+0000 7f82f1600640 1 -- 172.21.15.69:0/1758166552 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f82e00cf1d0 con 0x7f82f4103170 2024-03-19T01:17:07.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.521+0000 7f82f8e00640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82dc0a29f0 0x7f82dc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.521+0000 7f82fe5a3740 1 -- 172.21.15.69:0/542503577 shutdown_connections 2024-03-19T01:17:07.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.521+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/542503577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 0x5588d1582820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.521+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/542503577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 0x5588d157fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.521+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/542503577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d1577310 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.521+0000 7f82fe5a3740 1 -- 172.21.15.69:0/542503577 >> 172.21.15.69:0/542503577 conn(0x5588d1571660 msgr2=0x5588d1571a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:07.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.522+0000 7f82fe5a3740 1 -- 172.21.15.69:0/542503577 shutdown_connections 2024-03-19T01:17:07.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.522+0000 7f82fe5a3740 1 -- 172.21.15.69:0/542503577 wait complete. 2024-03-19T01:17:07.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.522+0000 7f82f8e00640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82dc0a29f0 0x7f82dc0a4eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5588d1576a00 tx=0x7f82e802e040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:07.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.522+0000 7f82fe5a3740 1 -- start start 2024-03-19T01:17:07.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.523+0000 7f82fe5a3740 1 Processor -- start 2024-03-19T01:17:07.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.523+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1576f30 0x5588d157d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.523+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1577850 0x5588d157d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.523+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1580430 0x5588d157def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.523+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5588d1583220 con 0x5588d1576f30 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.523+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5588d1496fc0 con 0x5588d1580430 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.523+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5588d15833a0 con 0x5588d1577850 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.523+0000 7f82fe5a3740 10 client.0 _ll_register_callbacks cb 0x5588d1496990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.523+0000 7f82f3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1577850 0x5588d157d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.523+0000 7f82f3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1577850 0x5588d157d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46808/0 (socket says 172.21.15.69:46808) 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.523+0000 7f82f3e00640 1 -- 172.21.15.69:0/1245180549 learned_addr learned my addr 172.21.15.69:0/1245180549 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.524+0000 7f82f9800640 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1580430 0x5588d157def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.524+0000 7f82f8e00640 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1576f30 0x5588d157d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.524+0000 7f82f3e00640 1 -- 172.21.15.69:0/1245180549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1580430 msgr2=0x5588d157def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.524+0000 7f82f3e00640 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1580430 0x5588d157def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.524+0000 7f82f3e00640 1 -- 172.21.15.69:0/1245180549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1576f30 msgr2=0x5588d157d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.524+0000 7f82f3e00640 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1576f30 0x5588d157d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.524+0000 7f82f3e00640 1 -- 172.21.15.69:0/1245180549 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d11ef3f0 con 0x5588d1577850 2024-03-19T01:17:07.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.524+0000 7f82f3e00640 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1577850 0x5588d157d9b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f82e000e790 tx=0x7f82e009af00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d7dbb1fbd1b9248 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.524+0000 7f82daa00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.524+0000 7f82daa00640 1 -- 172.21.15.69:0/1245180549 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e001c030 con 0x5588d1577850 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.524+0000 7f82fe5a3740 1 -- 172.21.15.69:0/1245180549 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82ec011d80 con 0x5588d1577850 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82daa00640 1 -- 172.21.15.69:0/1245180549 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f82e009f120 con 0x5588d1577850 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82daa00640 1 -- 172.21.15.69:0/1245180549 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f82e009b3c0 con 0x5588d1577850 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82fe5a3740 10 client.26977 fetch_fsmap learned FSMap version 43 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82fe5a3740 1 -- 172.21.15.69:0/1245180549 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5588d11af0f0 con 0x5588d1577850 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82daa00640 1 -- 172.21.15.69:0/1245180549 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ec011d80 con 0x5588d1577850 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82daa00640 1 -- 172.21.15.69:0/1245180549 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 43) ==== 8414+0+0 (secure 0 0 0) 0x7f82e00a36d0 con 0x5588d1577850 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82fe5a3740 10 client.26977 fetch_fsmap finished waiting for FSMap version 43 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82fe5a3740 10 client.26977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82fe5a3740 20 client.26977 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5588d1594b90 0x5588d1597010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82fe5a3740 4 client.26977 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.525+0000 7f82fe5a3740 1 -- 172.21.15.69:0/1245180549 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588d147c610 con 0x5588d1594b90 2024-03-19T01:17:07.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.526+0000 7f82f8e00640 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5588d1594b90 0x5588d1597010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.526+0000 7f82daa00640 10 client.26977 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:07.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.526+0000 7f82f8e00640 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5588d1594b90 0x5588d1597010 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec9f47b567c05fd1 server_cookie=c6639017cb870947 in_seq=0 out_seq=0 2024-03-19T01:17:07.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.527+0000 7f82daa00640 1 -- 172.21.15.69:0/1245180549 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5588d147c610 con 0x5588d1594b90 2024-03-19T01:17:07.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.527+0000 7f82daa00640 10 client.26977 handle_command_reply: tid=0 2024-03-19T01:17:07.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.527+0000 7f82fe5a3740 1 client.26977 shutdown 2024-03-19T01:17:07.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.527+0000 7f82fe5a3740 1 -- 172.21.15.69:0/1245180549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1577850 msgr2=0x5588d157d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.527+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1577850 0x5588d157d9b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f82e000e790 tx=0x7f82e009af00 comp rx=0 tx=0).stop 2024-03-19T01:17:07.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.528+0000 7f82fe5a3740 1 -- 172.21.15.69:0/1245180549 shutdown_connections 2024-03-19T01:17:07.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.528+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5588d1594b90 0x5588d1597010 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.528+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1580430 0x5588d157def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.528+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1577850 0x5588d157d9b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.528+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/1245180549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1576f30 0x5588d157d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.528+0000 7f82fe5a3740 1 -- 172.21.15.69:0/1245180549 >> 172.21.15.69:0/1245180549 conn(0x5588d1571660 msgr2=0x5588d15733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:07.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.528+0000 7f82daa00640 0 client.26977 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:07.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.528+0000 7f82fe5a3740 1 -- 172.21.15.69:0/1245180549 shutdown_connections 2024-03-19T01:17:07.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.528+0000 7f82fe5a3740 1 -- 172.21.15.69:0/1245180549 wait complete. 2024-03-19T01:17:07.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.528+0000 7f82fe5a3740 20 client.26977 trim_cache size 0 max 16384 2024-03-19T01:17:07.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:07 smithi069 ceph-mon[31727]: pgmap v197: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.4 MiB/s wr, 2.05k op/s 2024-03-19T01:17:07.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.551+0000 7f82fe5a3740 1 Processor -- start 2024-03-19T01:17:07.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82fe5a3740 1 -- start start 2024-03-19T01:17:07.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1576f30 0x5588d157fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 0x5588d1590450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1580430 0x5588d157c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41acc90 con 0x5588d1576f30 2024-03-19T01:17:07.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41ae8f0 con 0x5588d1577850 2024-03-19T01:17:07.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41ad360 con 0x5588d1580430 2024-03-19T01:17:07.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 0x5588d1590450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 0x5588d1590450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43528/0 (socket says 172.21.15.69:43528) 2024-03-19T01:17:07.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82f3e00640 1 -- 172.21.15.69:0/3956579360 learned_addr learned my addr 172.21.15.69:0/3956579360 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:07.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82f8e00640 1 --2- 172.21.15.69:0/3956579360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1576f30 0x5588d157fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.552+0000 7f82f9800640 1 --2- 172.21.15.69:0/3956579360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1580430 0x5588d157c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.553+0000 7f82f3e00640 1 -- 172.21.15.69:0/3956579360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1580430 msgr2=0x5588d157c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:07.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.553+0000 7f82f3e00640 1 --2- 172.21.15.69:0/3956579360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1580430 0x5588d157c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.553+0000 7f82f3e00640 1 -- 172.21.15.69:0/3956579360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1576f30 msgr2=0x5588d157fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:07.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.553+0000 7f82f3e00640 1 --2- 172.21.15.69:0/3956579360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1576f30 0x5588d157fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.553+0000 7f82f3e00640 1 -- 172.21.15.69:0/3956579360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d157b9c0 con 0x5588d1577850 2024-03-19T01:17:07.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.553+0000 7f82f9800640 1 --2- 172.21.15.69:0/3956579360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1580430 0x5588d157c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:07.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.553+0000 7f82f3e00640 1 --2- 172.21.15.69:0/3956579360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 0x5588d1590450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f82e009b0c0 tx=0x7f82e0005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b16315fe7ae85a36 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:07.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.553+0000 7f82e5600640 1 -- 172.21.15.69:0/3956579360 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e001c070 con 0x5588d1577850 2024-03-19T01:17:07.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.553+0000 7f82e5600640 1 -- 172.21.15.69:0/3956579360 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e00a3b60 con 0x5588d1577850 2024-03-19T01:17:07.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.553+0000 7f82e5600640 1 -- 172.21.15.69:0/3956579360 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e009f120 con 0x5588d1577850 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.554+0000 7f82fe5a3740 1 -- 172.21.15.69:0/3956579360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 msgr2=0x5588d1590450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.554+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/3956579360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 0x5588d1590450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f82e009b0c0 tx=0x7f82e0005230 comp rx=0 tx=0).stop 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.554+0000 7f82fe5a3740 1 -- 172.21.15.69:0/3956579360 shutdown_connections 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.554+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/3956579360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1580430 0x5588d157c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.554+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/3956579360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1577850 0x5588d1590450 secure :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f82e009b0c0 tx=0x7f82e0005230 comp rx=0 tx=0).stop 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.554+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/3956579360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1576f30 0x5588d157fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.554+0000 7f82fe5a3740 1 -- 172.21.15.69:0/3956579360 >> 172.21.15.69:0/3956579360 conn(0x5588d1571660 msgr2=0x5588d156fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.554+0000 7f82fe5a3740 1 -- 172.21.15.69:0/3956579360 shutdown_connections 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.554+0000 7f82fe5a3740 1 -- 172.21.15.69:0/3956579360 wait complete. 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.555+0000 7f82fe5a3740 1 -- start start 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.555+0000 7f82fe5a3740 1 Processor -- start 2024-03-19T01:17:07.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d15841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 0x5588d1584730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82fe5a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1584c70 0x5588d1588a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41ae160 con 0x5588d1580430 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41ada60 con 0x5588d1584c70 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82fe5a3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41aac00 con 0x5588d1576f30 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82fe5a3740 10 client.0 _ll_register_callbacks cb 0x7f82e0097660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82f8e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d15841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82f8e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d15841f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46832/0 (socket says 172.21.15.69:46832) 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82f8e00640 1 -- 172.21.15.69:0/2076768413 learned_addr learned my addr 172.21.15.69:0/2076768413 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82f3e00640 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 0x5588d1584730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.556+0000 7f82f9800640 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1584c70 0x5588d1588a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.557+0000 7f82f8e00640 1 -- 172.21.15.69:0/2076768413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1584c70 msgr2=0x5588d1588a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.557+0000 7f82f8e00640 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1584c70 0x5588d1588a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.557+0000 7f82f8e00640 1 -- 172.21.15.69:0/2076768413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 msgr2=0x5588d1584730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.557+0000 7f82f8e00640 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 0x5588d1584730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.557+0000 7f82f8e00640 1 -- 172.21.15.69:0/2076768413 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d1588f80 con 0x5588d1576f30 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.557+0000 7f82f3e00640 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 0x5588d1584730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.557+0000 7f82f8e00640 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d15841f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009850 tx=0x7f82e8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0fa5deda673680f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.557+0000 7f82daa00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.557+0000 7f82daa00640 1 -- 172.21.15.69:0/2076768413 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8004480 con 0x5588d1576f30 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.558+0000 7f82daa00640 1 -- 172.21.15.69:0/2076768413 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f82e8054a30 con 0x5588d1576f30 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.558+0000 7f82fe5a3740 1 -- 172.21.15.69:0/2076768413 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e00a3b60 con 0x5588d1576f30 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.558+0000 7f82daa00640 1 -- 172.21.15.69:0/2076768413 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f82e8065050 con 0x5588d1576f30 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.558+0000 7f82fe5a3740 10 client.26983 fetch_fsmap learned FSMap version 43 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.558+0000 7f82fe5a3740 1 -- 172.21.15.69:0/2076768413 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5588d15ffd60 con 0x5588d1576f30 2024-03-19T01:17:07.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.558+0000 7f82daa00640 1 -- 172.21.15.69:0/2076768413 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e00a3b60 con 0x5588d1576f30 2024-03-19T01:17:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:07 smithi163 ceph-mon[67428]: pgmap v197: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.4 MiB/s wr, 2.05k op/s 2024-03-19T01:17:07.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.559+0000 7f82daa00640 1 -- 172.21.15.69:0/2076768413 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 43) ==== 8414+0+0 (secure 0 0 0) 0x7f82e80544c0 con 0x5588d1576f30 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.559+0000 7f82fe5a3740 10 client.26983 fetch_fsmap finished waiting for FSMap version 43 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.559+0000 7f82fe5a3740 10 client.26983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.559+0000 7f82fe5a3740 20 client.26983 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.559+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5588d1600550 0x5588d1602970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.559+0000 7f82fe5a3740 4 client.26983 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.559+0000 7f82fe5a3740 1 -- 172.21.15.69:0/2076768413 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5588d14e8800 con 0x5588d1600550 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.559+0000 7f82f3e00640 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5588d1600550 0x5588d1602970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.559+0000 7f82f3e00640 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5588d1600550 0x5588d1602970 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6199b1d1dc6d476a server_cookie=d97f28767079c557 in_seq=0 out_seq=0 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.559+0000 7f82daa00640 10 client.26983 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82daa00640 1 -- 172.21.15.69:0/2076768413 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5588d14e8800 con 0x5588d1600550 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82daa00640 10 client.26983 handle_command_reply: tid=0 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82fe5a3740 1 client.26983 shutdown 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82fe5a3740 1 -- 172.21.15.69:0/2076768413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 msgr2=0x5588d15841f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d15841f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009850 tx=0x7f82e8009260 comp rx=0 tx=0).stop 2024-03-19T01:17:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82fe5a3740 1 -- 172.21.15.69:0/2076768413 shutdown_connections 2024-03-19T01:17:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5588d1600550 0x5588d1602970 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d1584c70 0x5588d1588a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588d1580430 0x5588d1584730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82daa00640 0 client.26983 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82fe5a3740 1 --2- 172.21.15.69:0/2076768413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5588d1576f30 0x5588d15841f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82fe5a3740 1 -- 172.21.15.69:0/2076768413 >> 172.21.15.69:0/2076768413 conn(0x5588d1571660 msgr2=0x5588d15729b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82fe5a3740 1 -- 172.21.15.69:0/2076768413 shutdown_connections 2024-03-19T01:17:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.560+0000 7f82fe5a3740 1 -- 172.21.15.69:0/2076768413 wait complete. 2024-03-19T01:17:07.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.561+0000 7f82fe5a3740 20 client.26983 trim_cache size 0 max 16384 2024-03-19T01:17:07.565 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:17:07.566 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:17:07.566 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:17:07.566 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:17:07.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.561+0000 7f82fa200640 1 -- 172.21.15.69:0/1758166552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82dc0a29f0 msgr2=0x7f82dc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.561+0000 7f82fa200640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82dc0a29f0 0x7f82dc0a4eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5588d1576a00 tx=0x7f82e802e040 comp rx=0 tx=0).stop 2024-03-19T01:17:07.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.561+0000 7f82fa200640 1 -- 172.21.15.69:0/1758166552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f4103170 msgr2=0x7f82f41985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.561+0000 7f82fa200640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f4103170 0x7f82f41985c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f82e000e7c0 tx=0x7f82e000ec80 comp rx=0 tx=0).stop 2024-03-19T01:17:07.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.561+0000 7f82fa200640 1 -- 172.21.15.69:0/1758166552 shutdown_connections 2024-03-19T01:17:07.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.561+0000 7f82fa200640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f410bc70 0x7f82f419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.561+0000 7f82fa200640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82dc0a29f0 0x7f82dc0a4eb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.561+0000 7f82fa200640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f4103170 0x7f82f41985c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.561+0000 7f82fa200640 1 --2- 172.21.15.69:0/1758166552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f4102850 0x7f82f4198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.561+0000 7f82fa200640 1 -- 172.21.15.69:0/1758166552 >> 172.21.15.69:0/1758166552 conn(0x7f82f40fc070 msgr2=0x7f82f40fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:07.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.562+0000 7f82fa200640 1 -- 172.21.15.69:0/1758166552 shutdown_connections 2024-03-19T01:17:07.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.562+0000 7f82fa200640 1 -- 172.21.15.69:0/1758166552 wait complete. 2024-03-19T01:17:07.581 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:17:07.581 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:17:07.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.788+0000 7f5b9b200640 1 -- 172.21.15.69:0/1534446819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b9c10df50 msgr2=0x7f5b9c110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.788+0000 7f5b9b200640 1 --2- 172.21.15.69:0/1534446819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b9c10df50 0x7f5b9c110340 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009ca0 tx=0x7f5b9402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.788+0000 7f5b9b200640 1 -- 172.21.15.69:0/1534446819 shutdown_connections 2024-03-19T01:17:07.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.788+0000 7f5b9b200640 1 --2- 172.21.15.69:0/1534446819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b9c10df50 0x7f5b9c110340 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.788+0000 7f5b9b200640 1 --2- 172.21.15.69:0/1534446819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9c100d50 0x7f5b9c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.788+0000 7f5b9b200640 1 --2- 172.21.15.69:0/1534446819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b9c1003a0 0x7f5b9c100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.788+0000 7f5b9b200640 1 -- 172.21.15.69:0/1534446819 >> 172.21.15.69:0/1534446819 conn(0x7f5b9c0fbfe0 msgr2=0x7f5b9c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:07.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.788+0000 7f5b9b200640 1 -- 172.21.15.69:0/1534446819 shutdown_connections 2024-03-19T01:17:07.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.788+0000 7f5b9b200640 1 -- 172.21.15.69:0/1534446819 wait complete. 2024-03-19T01:17:07.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.789+0000 7f5b9b200640 1 Processor -- start 2024-03-19T01:17:07.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.789+0000 7f5b9b200640 1 -- start start 2024-03-19T01:17:07.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.790+0000 7f5b9b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b9c1003a0 0x7f5b9c19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.790+0000 7f5b9b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b9c100d50 0x7f5b9c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.790+0000 7f5b9b200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9c10df50 0x7f5b9c19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.790+0000 7f5b9b200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c113bf0 con 0x7f5b9c10df50 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.790+0000 7f5b9b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c113a70 con 0x7f5b9c1003a0 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.790+0000 7f5b9b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c113d70 con 0x7f5b9c100d50 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.790+0000 7f5b99e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b9c1003a0 0x7f5b9c19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.790+0000 7f5b99e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b9c1003a0 0x7f5b9c19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43554/0 (socket says 172.21.15.69:43554) 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.790+0000 7f5b99e00640 1 -- 172.21.15.69:0/1706869110 learned_addr learned my addr 172.21.15.69:0/1706869110 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.790+0000 7f5b99400640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b9c100d50 0x7f5b9c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.790+0000 7f5b9a800640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9c10df50 0x7f5b9c19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.791+0000 7f5b99e00640 1 -- 172.21.15.69:0/1706869110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b9c100d50 msgr2=0x7f5b9c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.791+0000 7f5b99e00640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b9c100d50 0x7f5b9c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.791+0000 7f5b99e00640 1 -- 172.21.15.69:0/1706869110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9c10df50 msgr2=0x7f5b9c19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:07.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.791+0000 7f5b99e00640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9c10df50 0x7f5b9c19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:07.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.791+0000 7f5b99e00640 1 -- 172.21.15.69:0/1706869110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c19f0f0 con 0x7f5b9c1003a0 2024-03-19T01:17:07.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.791+0000 7f5b99e00640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b9c1003a0 0x7f5b9c19a150 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00e9f0 tx=0x7f5b8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:07.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.791+0000 7f5b92a00640 1 -- 172.21.15.69:0/1706869110 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00cd80 con 0x7f5b9c1003a0 2024-03-19T01:17:07.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.791+0000 7f5b92a00640 1 -- 172.21.15.69:0/1706869110 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b8c010910 con 0x7f5b9c1003a0 2024-03-19T01:17:07.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.791+0000 7f5b92a00640 1 -- 172.21.15.69:0/1706869110 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c018990 con 0x7f5b9c1003a0 2024-03-19T01:17:07.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.791+0000 7f5b9b200640 1 -- 172.21.15.69:0/1706869110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c19f3e0 con 0x7f5b9c1003a0 2024-03-19T01:17:07.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.791+0000 7f5b9b200640 1 -- 172.21.15.69:0/1706869110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c1a6c70 con 0x7f5b9c1003a0 2024-03-19T01:17:07.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.793+0000 7f5b9b200640 1 -- 172.21.15.69:0/1706869110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c068980 con 0x7f5b9c1003a0 2024-03-19T01:17:07.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.795+0000 7f5b92a00640 1 -- 172.21.15.69:0/1706869110 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f5b8c010ac0 con 0x7f5b9c1003a0 2024-03-19T01:17:07.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.796+0000 7f5b92a00640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5b800a29f0 0x7f5b800a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:07.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.797+0000 7f5b92a00640 1 -- 172.21.15.69:0/1706869110 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f5b8c0cf570 con 0x7f5b9c1003a0 2024-03-19T01:17:07.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.797+0000 7f5b99400640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5b800a29f0 0x7f5b800a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:07.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.797+0000 7f5b99400640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5b800a29f0 0x7f5b800a4eb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0ff610 tx=0x7f5b88005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:07.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:07.800+0000 7f5b92a00640 1 -- 172.21.15.69:0/1706869110 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5b8c09b810 con 0x7f5b9c1003a0 2024-03-19T01:17:08.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.122+0000 7f5b9b200640 1 -- 172.21.15.69:0/1706869110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b9c19b450 con 0x7f5b9c1003a0 2024-03-19T01:17:08.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.123+0000 7f5b92a00640 1 -- 172.21.15.69:0/1706869110 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11951 (secure 0 0 0) 0x7f5b8c09f8b0 con 0x7f5b9c1003a0 2024-03-19T01:17:08.134 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:17:08.134 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:17:02.899621+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:17:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 43 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.134+0000 7f5b9b200640 1 -- 172.21.15.69:0/1706869110 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5b800a29f0 msgr2=0x7f5b800a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.134+0000 7f5b9b200640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5b800a29f0 0x7f5b800a4eb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0ff610 tx=0x7f5b88005f90 comp rx=0 tx=0).stop 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.134+0000 7f5b9b200640 1 -- 172.21.15.69:0/1706869110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b9c1003a0 msgr2=0x7f5b9c19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.134+0000 7f5b9b200640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b9c1003a0 0x7f5b9c19a150 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00e9f0 tx=0x7f5b8c00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.135+0000 7f5b9b200640 1 -- 172.21.15.69:0/1706869110 shutdown_connections 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.135+0000 7f5b9b200640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9c10df50 0x7f5b9c19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.135+0000 7f5b9b200640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b9c100d50 0x7f5b9c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.135+0000 7f5b9b200640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5b800a29f0 0x7f5b800a4eb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.135+0000 7f5b9b200640 1 --2- 172.21.15.69:0/1706869110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b9c1003a0 0x7f5b9c19a150 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.135+0000 7f5b9b200640 1 -- 172.21.15.69:0/1706869110 >> 172.21.15.69:0/1706869110 conn(0x7f5b9c0fbfe0 msgr2=0x7f5b9c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.135+0000 7f5b9b200640 1 -- 172.21.15.69:0/1706869110 shutdown_connections 2024-03-19T01:17:08.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.135+0000 7f5b9b200640 1 -- 172.21.15.69:0/1706869110 wait complete. 2024-03-19T01:17:08.167 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:17:08.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:08 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1706869110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:17:08.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:08 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1706869110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:17:08.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.551+0000 7f709ce00640 1 -- 172.21.15.69:0/1004708739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981038a0 msgr2=0x7f7098103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:08.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.551+0000 7f709ce00640 1 --2- 172.21.15.69:0/1004708739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981038a0 0x7f7098103d20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7088009930 tx=0x7f708802bc30 comp rx=0 tx=0).stop 2024-03-19T01:17:08.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.554+0000 7f709ce00640 1 -- 172.21.15.69:0/1004708739 shutdown_connections 2024-03-19T01:17:08.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.554+0000 7f709ce00640 1 --2- 172.21.15.69:0/1004708739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7098104260 0x7f7098110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.554+0000 7f709ce00640 1 --2- 172.21.15.69:0/1004708739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981038a0 0x7f7098103d20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.554+0000 7f709ce00640 1 --2- 172.21.15.69:0/1004708739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70981098b0 0x7f7098109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.554+0000 7f709ce00640 1 -- 172.21.15.69:0/1004708739 >> 172.21.15.69:0/1004708739 conn(0x7f70980ff330 msgr2=0x7f7098101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:08.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.556+0000 7f709ce00640 1 -- 172.21.15.69:0/1004708739 shutdown_connections 2024-03-19T01:17:08.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.556+0000 7f709ce00640 1 -- 172.21.15.69:0/1004708739 wait complete. 2024-03-19T01:17:08.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.560+0000 7f709ce00640 1 Processor -- start 2024-03-19T01:17:08.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.560+0000 7f709ce00640 1 -- start start 2024-03-19T01:17:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.560+0000 7f709ce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981038a0 0x7f7098110640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.560+0000 7f709ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098104260 0x7f709810d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.560+0000 7f709ce00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70981098b0 0x7f709810dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.560+0000 7f709ce00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7098113cd0 con 0x7f70981098b0 2024-03-19T01:17:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.560+0000 7f709ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7098113b50 con 0x7f7098104260 2024-03-19T01:17:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.560+0000 7f709ce00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7098113e50 con 0x7f70981038a0 2024-03-19T01:17:08.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.561+0000 7f7096a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098104260 0x7f709810d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:08 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1706869110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:17:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.561+0000 7f7096a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098104260 0x7f709810d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43558/0 (socket says 172.21.15.69:43558) 2024-03-19T01:17:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.561+0000 7f7096a00640 1 -- 172.21.15.69:0/1955560057 learned_addr learned my addr 172.21.15.69:0/1955560057 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.561+0000 7f7097400640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981038a0 0x7f7098110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.561+0000 7f7096a00640 1 -- 172.21.15.69:0/1955560057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981038a0 msgr2=0x7f7098110640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.561+0000 7f7096a00640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981038a0 0x7f7098110640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.561+0000 7f7096a00640 1 -- 172.21.15.69:0/1955560057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70981098b0 msgr2=0x7f709810dc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:17:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.561+0000 7f7096a00640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70981098b0 0x7f709810dc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.561+0000 7f7096a00640 1 -- 172.21.15.69:0/1955560057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709810e500 con 0x7f7098104260 2024-03-19T01:17:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.562+0000 7f7096a00640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098104260 0x7f709810d6e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7088009900 tx=0x7f70880042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:08.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.563+0000 7f708fe00640 1 -- 172.21.15.69:0/1955560057 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088043020 con 0x7f7098104260 2024-03-19T01:17:08.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.563+0000 7f708fe00640 1 -- 172.21.15.69:0/1955560057 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70880044d0 con 0x7f7098104260 2024-03-19T01:17:08.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.563+0000 7f708fe00640 1 -- 172.21.15.69:0/1955560057 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088003be0 con 0x7f7098104260 2024-03-19T01:17:08.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.563+0000 7f709ce00640 1 -- 172.21.15.69:0/1955560057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70981a0270 con 0x7f7098104260 2024-03-19T01:17:08.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.564+0000 7f709ce00640 1 -- 172.21.15.69:0/1955560057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70981a0810 con 0x7f7098104260 2024-03-19T01:17:08.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.571+0000 7f70a1178740 1 Processor -- start 2024-03-19T01:17:08.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.571+0000 7f70a1178740 1 -- start start 2024-03-19T01:17:08.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.572+0000 7f70a1178740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f48186fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.572+0000 7f70a1178740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f48187510 0x555f4818fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.572+0000 7f70a1178740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f481900f0 0x555f481924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.572+0000 7f70a1178740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555f480eb380 con 0x555f481900f0 2024-03-19T01:17:08.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.572+0000 7f70a1178740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f48093d20 con 0x555f48187510 2024-03-19T01:17:08.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.572+0000 7f70a1178740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555f480e7420 con 0x555f48186bf0 2024-03-19T01:17:08.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.573+0000 7f7097400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f48187510 0x555f4818fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.574+0000 7f7097e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f48186fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.574+0000 7f7097e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f48186fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46882/0 (socket says 172.21.15.69:46882) 2024-03-19T01:17:08.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.574+0000 7f7097e00640 1 -- 172.21.15.69:0/2465070836 learned_addr learned my addr 172.21.15.69:0/2465070836 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:08.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.574+0000 7f7097e00640 1 -- 172.21.15.69:0/2465070836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f48187510 msgr2=0x555f4818fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:08.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.574+0000 7f7097e00640 1 --2- 172.21.15.69:0/2465070836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f48187510 0x555f4818fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.574+0000 7f7097e00640 1 -- 172.21.15.69:0/2465070836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f481900f0 msgr2=0x555f481924e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:17:08.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.574+0000 7f7097e00640 1 --2- 172.21.15.69:0/2465070836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f481900f0 0x555f481924e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.574+0000 7f7097e00640 1 -- 172.21.15.69:0/2465070836 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f47dff3f0 con 0x555f48186bf0 2024-03-19T01:17:08.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.575+0000 7f7097e00640 1 --2- 172.21.15.69:0/2465070836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f48186fd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7084009bd0 tx=0x7f708400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=863361319598c202 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:08.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.575+0000 7f708cc00640 1 -- 172.21.15.69:0/2465070836 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7084005020 con 0x555f48186bf0 2024-03-19T01:17:08.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.575+0000 7f708cc00640 1 -- 172.21.15.69:0/2465070836 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70840047a0 con 0x555f48186bf0 2024-03-19T01:17:08.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.575+0000 7f708cc00640 1 -- 172.21.15.69:0/2465070836 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7084010690 con 0x555f48186bf0 2024-03-19T01:17:08.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.576+0000 7f70a1178740 1 -- 172.21.15.69:0/2465070836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 msgr2=0x555f48186fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:08.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.576+0000 7f70a1178740 1 --2- 172.21.15.69:0/2465070836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f48186fd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7084009bd0 tx=0x7f708400e720 comp rx=0 tx=0).stop 2024-03-19T01:17:08.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.577+0000 7f70a1178740 1 -- 172.21.15.69:0/2465070836 shutdown_connections 2024-03-19T01:17:08.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.577+0000 7f70a1178740 1 --2- 172.21.15.69:0/2465070836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f481900f0 0x555f481924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.577+0000 7f70a1178740 1 --2- 172.21.15.69:0/2465070836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f48187510 0x555f4818fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.577+0000 7f70a1178740 1 --2- 172.21.15.69:0/2465070836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f48186fd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.577+0000 7f70a1178740 1 -- 172.21.15.69:0/2465070836 >> 172.21.15.69:0/2465070836 conn(0x555f48181320 msgr2=0x555f48181700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:08.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.578+0000 7f708fe00640 1 -- 172.21.15.69:0/1955560057 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f7088046020 con 0x7f7098104260 2024-03-19T01:17:08.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.579+0000 7f70a1178740 1 -- 172.21.15.69:0/2465070836 shutdown_connections 2024-03-19T01:17:08.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.580+0000 7f708fe00640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f707c0a2940 0x7f707c0a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.580+0000 7f7097400640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f707c0a2940 0x7f707c0a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.581+0000 7f70a1178740 1 -- 172.21.15.69:0/2465070836 wait complete. 2024-03-19T01:17:08.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.581+0000 7f708fe00640 1 -- 172.21.15.69:0/1955560057 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f70880f0fa0 con 0x7f7098104260 2024-03-19T01:17:08.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.582+0000 7f70a1178740 1 -- start start 2024-03-19T01:17:08.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.582+0000 7f70a1178740 1 Processor -- start 2024-03-19T01:17:08.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.583+0000 7f7097400640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f707c0a2940 0x7f707c0a4e00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x555f481866c0 tx=0x7f7090009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:08.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.585+0000 7f70a1178740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 0x555f4818d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.586+0000 7f70a1178740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f48187510 0x555f4818d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.586+0000 7f70a1178740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f4818dbf0 0x555f4819c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.586+0000 7f7097400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 0x555f4818d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.587+0000 7f70a1178740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555f48192ee0 con 0x555f48186bf0 2024-03-19T01:17:08.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.587+0000 7f70a1178740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f480a7160 con 0x555f48187510 2024-03-19T01:17:08.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.587+0000 7f70a1178740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555f48193060 con 0x555f4818dbf0 2024-03-19T01:17:08.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.587+0000 7f70a1178740 10 client.0 _ll_register_callbacks cb 0x555f480e3170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:08.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.587+0000 7f7097e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f4818dbf0 0x555f4819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.587+0000 7f7097e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f4818dbf0 0x555f4819c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46886/0 (socket says 172.21.15.69:46886) 2024-03-19T01:17:08.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.587+0000 7f7097e00640 1 -- 172.21.15.69:0/1628113166 learned_addr learned my addr 172.21.15.69:0/1628113166 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:08.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.588+0000 7f7096a00640 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f48187510 0x555f4818d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.588+0000 7f7097400640 1 -- 172.21.15.69:0/1628113166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f4818dbf0 msgr2=0x555f4819c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:08.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.588+0000 7f7097400640 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f4818dbf0 0x555f4819c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.588+0000 7f7097400640 1 -- 172.21.15.69:0/1628113166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f48187510 msgr2=0x555f4818d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:08.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.588+0000 7f7097400640 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f48187510 0x555f4818d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.588+0000 7f7097400640 1 -- 172.21.15.69:0/1628113166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f47ec97d0 con 0x555f48186bf0 2024-03-19T01:17:08.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.588+0000 7f7097e00640 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f4818dbf0 0x555f4819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:08.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.590+0000 7f7097400640 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 0x555f4818d170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7090012660 tx=0x7f7090004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03eb423a8519272 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:08.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.590+0000 7f7081600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:17:08.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.590+0000 7f7081600640 1 -- 172.21.15.69:0/1628113166 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70900119b0 con 0x555f48186bf0 2024-03-19T01:17:08.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.591+0000 7f7081600640 1 -- 172.21.15.69:0/1628113166 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f7090011b50 con 0x555f48186bf0 2024-03-19T01:17:08.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.593+0000 7f70a1178740 1 -- 172.21.15.69:0/1628113166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70840047a0 con 0x555f48186bf0 2024-03-19T01:17:08.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.593+0000 7f7081600640 1 -- 172.21.15.69:0/1628113166 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7090011d50 con 0x555f48186bf0 2024-03-19T01:17:08.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.593+0000 7f70a1178740 10 client.16791 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:08.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.593+0000 7f70a1178740 1 -- 172.21.15.69:0/1628113166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555f4819f6d0 con 0x555f48186bf0 2024-03-19T01:17:08.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.594+0000 7f7081600640 1 -- 172.21.15.69:0/1628113166 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70840047a0 con 0x555f48186bf0 2024-03-19T01:17:08.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.595+0000 7f7081600640 1 -- 172.21.15.69:0/1628113166 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f709001f620 con 0x555f48186bf0 2024-03-19T01:17:08.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.595+0000 7f70a1178740 10 client.16791 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:08.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.595+0000 7f70a1178740 10 client.16791 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:17:08.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.595+0000 7f70a1178740 20 client.16791 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:08.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.596+0000 7f70a1178740 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555f481a0ae0 0x555f481a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.596+0000 7f70a1178740 4 client.16791 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:08.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.596+0000 7f70a1178740 1 -- 172.21.15.69:0/1628113166 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f480f88c0 con 0x555f481a0ae0 2024-03-19T01:17:08.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.596+0000 7f7096a00640 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555f481a0ae0 0x555f481a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.598+0000 7f7096a00640 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555f481a0ae0 0x555f481a2f10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f75c4aa63ae4977f server_cookie=b4f4bd5a8ba6d277 in_seq=0 out_seq=0 2024-03-19T01:17:08.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.598+0000 7f7081600640 10 client.16791 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:17:08.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.600+0000 7f7081600640 1 -- 172.21.15.69:0/1628113166 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555f480f88c0 con 0x555f481a0ae0 2024-03-19T01:17:08.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.600+0000 7f7081600640 10 client.16791 handle_command_reply: tid=0 2024-03-19T01:17:08.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.600+0000 7f70a1178740 1 client.16791 shutdown 2024-03-19T01:17:08.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.601+0000 7f70a1178740 1 -- 172.21.15.69:0/1628113166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 msgr2=0x555f4818d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:08.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.601+0000 7f70a1178740 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 0x555f4818d170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7090012660 tx=0x7f7090004a40 comp rx=0 tx=0).stop 2024-03-19T01:17:08.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.601+0000 7f70a1178740 1 -- 172.21.15.69:0/1628113166 shutdown_connections 2024-03-19T01:17:08.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.601+0000 7f70a1178740 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555f481a0ae0 0x555f481a2f10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.601+0000 7f70a1178740 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f4818dbf0 0x555f4819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.601+0000 7f70a1178740 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f48187510 0x555f4818d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.601+0000 7f70a1178740 1 --2- 172.21.15.69:0/1628113166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 0x555f4818d170 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.601+0000 7f70a1178740 1 -- 172.21.15.69:0/1628113166 >> 172.21.15.69:0/1628113166 conn(0x555f48181320 msgr2=0x555f48183070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.602+0000 7f7081600640 0 client.16791 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:17:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.602+0000 7f70a1178740 1 -- 172.21.15.69:0/1628113166 shutdown_connections 2024-03-19T01:17:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.602+0000 7f70a1178740 1 -- 172.21.15.69:0/1628113166 wait complete. 2024-03-19T01:17:08.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.602+0000 7f70a1178740 20 client.16791 trim_cache size 0 max 16384 2024-03-19T01:17:08.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.634+0000 7f70a1178740 1 Processor -- start 2024-03-19T01:17:08.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.635+0000 7f70a1178740 1 -- start start 2024-03-19T01:17:08.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.635+0000 7f70a1178740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f481a8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.635+0000 7f70a1178740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48187510 0x555f4819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.635+0000 7f70a1178740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4818dbf0 0x555f4819c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.635+0000 7f70a1178740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f70981bdc30 con 0x555f48187510 2024-03-19T01:17:08.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.635+0000 7f70a1178740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70981bf8f0 con 0x555f4818dbf0 2024-03-19T01:17:08.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.635+0000 7f70a1178740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f70981be300 con 0x555f48186bf0 2024-03-19T01:17:08.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.635+0000 7f7097400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f481a8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.636+0000 7f7097400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f481a8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46896/0 (socket says 172.21.15.69:46896) 2024-03-19T01:17:08.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.636+0000 7f7097400640 1 -- 172.21.15.69:0/115420969 learned_addr learned my addr 172.21.15.69:0/115420969 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:08.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.636+0000 7f7096a00640 1 --2- 172.21.15.69:0/115420969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48187510 0x555f4819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.636+0000 7f7097400640 1 -- 172.21.15.69:0/115420969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4818dbf0 msgr2=0x555f4819c1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:17:08.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.636+0000 7f7097e00640 1 --2- 172.21.15.69:0/115420969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4818dbf0 0x555f4819c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.637+0000 7f7097400640 1 --2- 172.21.15.69:0/115420969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4818dbf0 0x555f4819c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.637+0000 7f7097400640 1 -- 172.21.15.69:0/115420969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48187510 msgr2=0x555f4819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:08.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.637+0000 7f7097400640 1 --2- 172.21.15.69:0/115420969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48187510 0x555f4819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.637+0000 7f7097400640 1 -- 172.21.15.69:0/115420969 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f4817cbe0 con 0x555f48186bf0 2024-03-19T01:17:08.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.637+0000 7f7097400640 1 --2- 172.21.15.69:0/115420969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f481a8820 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7090006010 tx=0x7f7090011fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67723f91dd8f86e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:08.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.638+0000 7f7083e00640 1 -- 172.21.15.69:0/115420969 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090037040 con 0x555f48186bf0 2024-03-19T01:17:08.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.638+0000 7f7083e00640 1 -- 172.21.15.69:0/115420969 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f709001f420 con 0x555f48186bf0 2024-03-19T01:17:08.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.638+0000 7f7083e00640 1 -- 172.21.15.69:0/115420969 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090005290 con 0x555f48186bf0 2024-03-19T01:17:08.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.639+0000 7f70a1178740 1 -- 172.21.15.69:0/115420969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 msgr2=0x555f481a8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:08.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.639+0000 7f70a1178740 1 --2- 172.21.15.69:0/115420969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f481a8820 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7090006010 tx=0x7f7090011fa0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.640+0000 7f70a1178740 1 -- 172.21.15.69:0/115420969 shutdown_connections 2024-03-19T01:17:08.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.640+0000 7f70a1178740 1 --2- 172.21.15.69:0/115420969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4818dbf0 0x555f4819c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.640+0000 7f70a1178740 1 --2- 172.21.15.69:0/115420969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48187510 0x555f4819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.640+0000 7f70a1178740 1 --2- 172.21.15.69:0/115420969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48186bf0 0x555f481a8820 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.640+0000 7f70a1178740 1 -- 172.21.15.69:0/115420969 >> 172.21.15.69:0/115420969 conn(0x555f48181320 msgr2=0x555f48188450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:08.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.640+0000 7f70a1178740 1 -- 172.21.15.69:0/115420969 shutdown_connections 2024-03-19T01:17:08.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.640+0000 7f70a1178740 1 -- 172.21.15.69:0/115420969 wait complete. 2024-03-19T01:17:08.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.641+0000 7f70a1178740 1 -- start start 2024-03-19T01:17:08.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.641+0000 7f70a1178740 1 Processor -- start 2024-03-19T01:17:08.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.643+0000 7f70a1178740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 0x555f48189de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.643+0000 7f70a1178740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48187510 0x555f4818a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.643+0000 7f70a1178740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4818dbf0 0x555f4818a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.644+0000 7f7096a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48187510 0x555f4818a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.644+0000 7f7096a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48187510 0x555f4818a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46912/0 (socket says 172.21.15.69:46912) 2024-03-19T01:17:08.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.644+0000 7f70a1178740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f70981bf160 con 0x555f48186bf0 2024-03-19T01:17:08.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.644+0000 7f70a1178740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70981bea60 con 0x555f4818dbf0 2024-03-19T01:17:08.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.644+0000 7f70a1178740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f70981bbba0 con 0x555f48187510 2024-03-19T01:17:08.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.644+0000 7f70a1178740 10 client.0 _ll_register_callbacks cb 0x7f7090009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:08.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.644+0000 7f7097e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4818dbf0 0x555f4818a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.644+0000 7f7096a00640 1 -- 172.21.15.69:0/3583849662 learned_addr learned my addr 172.21.15.69:0/3583849662 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:08.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.645+0000 7f7097e00640 1 -- 172.21.15.69:0/3583849662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48187510 msgr2=0x555f4818a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:08.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.645+0000 7f7097e00640 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48187510 0x555f4818a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.645+0000 7f7097e00640 1 -- 172.21.15.69:0/3583849662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 msgr2=0x555f48189de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:08.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.645+0000 7f7097400640 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 0x555f48189de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.645+0000 7f7097e00640 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 0x555f48189de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.646+0000 7f7097e00640 1 -- 172.21.15.69:0/3583849662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f48205ed0 con 0x555f4818dbf0 2024-03-19T01:17:08.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.646+0000 7f7097400640 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 0x555f48189de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:08.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.646+0000 7f7097e00640 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4818dbf0 0x555f4818a860 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7084009bd0 tx=0x7f7084009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=498b8073250f4c47 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:08.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.647+0000 7f7081600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:17:08.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.647+0000 7f7081600640 1 -- 172.21.15.69:0/3583849662 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708401d040 con 0x555f4818dbf0 2024-03-19T01:17:08.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.647+0000 7f70a1178740 1 -- 172.21.15.69:0/3583849662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f709001f420 con 0x555f4818dbf0 2024-03-19T01:17:08.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.647+0000 7f7081600640 1 -- 172.21.15.69:0/3583849662 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f70840054d0 con 0x555f4818dbf0 2024-03-19T01:17:08.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.648+0000 7f7081600640 1 -- 172.21.15.69:0/3583849662 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f708402f050 con 0x555f4818dbf0 2024-03-19T01:17:08.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.648+0000 7f70a1178740 10 client.27050 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:08.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.648+0000 7f70a1178740 1 -- 172.21.15.69:0/3583849662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555f48207200 con 0x555f4818dbf0 2024-03-19T01:17:08.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.649+0000 7f7081600640 1 -- 172.21.15.69:0/3583849662 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f709001f420 con 0x555f4818dbf0 2024-03-19T01:17:08.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.651+0000 7f7081600640 1 -- 172.21.15.69:0/3583849662 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f7084018830 con 0x555f4818dbf0 2024-03-19T01:17:08.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.651+0000 7f70a1178740 10 client.27050 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.651+0000 7f70a1178740 10 client.27050 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:17:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.651+0000 7f70a1178740 20 client.27050 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.651+0000 7f70a1178740 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555f482076a0 0x555f48209ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.651+0000 7f70a1178740 4 client.27050 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:17:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.651+0000 7f70a1178740 1 -- 172.21.15.69:0/3583849662 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f480f2c10 con 0x555f482076a0 2024-03-19T01:17:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.652+0000 7f7097400640 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555f482076a0 0x555f48209ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:08.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.654+0000 7f7097400640 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555f482076a0 0x555f48209ac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a26b95bd80bbf512 server_cookie=a9f8bb5f69cac3a5 in_seq=0 out_seq=0 2024-03-19T01:17:08.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.654+0000 7f7081600640 10 client.27050 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:17:08.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.655+0000 7f7081600640 1 -- 172.21.15.69:0/3583849662 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f480f2c10 con 0x555f482076a0 2024-03-19T01:17:08.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.655+0000 7f7081600640 10 client.27050 handle_command_reply: tid=0 2024-03-19T01:17:08.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.655+0000 7f70a1178740 1 client.27050 shutdown 2024-03-19T01:17:08.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.656+0000 7f70a1178740 1 -- 172.21.15.69:0/3583849662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4818dbf0 msgr2=0x555f4818a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:08.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.656+0000 7f70a1178740 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4818dbf0 0x555f4818a860 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7084009bd0 tx=0x7f7084009830 comp rx=0 tx=0).stop 2024-03-19T01:17:08.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.657+0000 7f70a1178740 1 -- 172.21.15.69:0/3583849662 shutdown_connections 2024-03-19T01:17:08.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.657+0000 7f70a1178740 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x555f482076a0 0x555f48209ac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.657+0000 7f70a1178740 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4818dbf0 0x555f4818a860 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.657+0000 7f7081600640 0 client.27050 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:17:08.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.657+0000 7f70a1178740 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555f48187510 0x555f4818a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.657+0000 7f70a1178740 1 --2- 172.21.15.69:0/3583849662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555f48186bf0 0x555f48189de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.657+0000 7f70a1178740 1 -- 172.21.15.69:0/3583849662 >> 172.21.15.69:0/3583849662 conn(0x555f48181320 msgr2=0x555f4818b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:08.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.658+0000 7f70a1178740 1 -- 172.21.15.69:0/3583849662 shutdown_connections 2024-03-19T01:17:08.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.660+0000 7f70a1178740 1 -- 172.21.15.69:0/3583849662 wait complete. 2024-03-19T01:17:08.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.660+0000 7f70a1178740 20 client.27050 trim_cache size 0 max 16384 2024-03-19T01:17:08.662 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:17:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.662+0000 7f709ce00640 1 -- 172.21.15.69:0/1955560057 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f707c0a2940 msgr2=0x7f707c0a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.662+0000 7f709ce00640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f707c0a2940 0x7f707c0a4e00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x555f481866c0 tx=0x7f7090009500 comp rx=0 tx=0).stop 2024-03-19T01:17:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.662+0000 7f709ce00640 1 -- 172.21.15.69:0/1955560057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098104260 msgr2=0x7f709810d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:08.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.662+0000 7f709ce00640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098104260 0x7f709810d6e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7088009900 tx=0x7f70880042e0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.663+0000 7f709ce00640 1 -- 172.21.15.69:0/1955560057 shutdown_connections 2024-03-19T01:17:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.663+0000 7f709ce00640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70981098b0 0x7f709810dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.663+0000 7f709ce00640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f707c0a2940 0x7f707c0a4e00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.663+0000 7f709ce00640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7098104260 0x7f709810d6e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.663+0000 7f709ce00640 1 --2- 172.21.15.69:0/1955560057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981038a0 0x7f7098110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.663+0000 7f709ce00640 1 -- 172.21.15.69:0/1955560057 >> 172.21.15.69:0/1955560057 conn(0x7f70980ff330 msgr2=0x7f70980ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.663+0000 7f709ce00640 1 -- 172.21.15.69:0/1955560057 shutdown_connections 2024-03-19T01:17:08.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:08.663+0000 7f709ce00640 1 -- 172.21.15.69:0/1955560057 wait complete. 2024-03-19T01:17:08.692 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:17:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.029+0000 7f5a59e00640 1 -- 172.21.15.69:0/693837884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a54075d50 msgr2=0x7f5a54076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.029+0000 7f5a59e00640 1 --2- 172.21.15.69:0/693837884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a54075d50 0x7f5a54076130 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009930 tx=0x7f5a4402bc50 comp rx=0 tx=0).stop 2024-03-19T01:17:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.029+0000 7f5a59e00640 1 -- 172.21.15.69:0/693837884 shutdown_connections 2024-03-19T01:17:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.029+0000 7f5a59e00640 1 --2- 172.21.15.69:0/693837884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a54112660 0x7f5a54114a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.029+0000 7f5a59e00640 1 --2- 172.21.15.69:0/693837884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54076670 0x7f5a54112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.029+0000 7f5a59e00640 1 --2- 172.21.15.69:0/693837884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a54075d50 0x7f5a54076130 secure :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009930 tx=0x7f5a4402bc50 comp rx=0 tx=0).stop 2024-03-19T01:17:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.029+0000 7f5a59e00640 1 -- 172.21.15.69:0/693837884 >> 172.21.15.69:0/693837884 conn(0x7f5a540ff4d0 msgr2=0x7f5a541018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.029+0000 7f5a59e00640 1 -- 172.21.15.69:0/693837884 shutdown_connections 2024-03-19T01:17:09.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.029+0000 7f5a59e00640 1 -- 172.21.15.69:0/693837884 wait complete. 2024-03-19T01:17:09.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.030+0000 7f5a59e00640 1 Processor -- start 2024-03-19T01:17:09.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.031+0000 7f5a59e00640 1 -- start start 2024-03-19T01:17:09.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.031+0000 7f5a59e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a54076670 0x7f5a541a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.031+0000 7f5a59e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54112660 0x7f5a541a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.031+0000 7f5a59e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419e800 0x7f5a5419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.031+0000 7f5a59e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54118230 con 0x7f5a54112660 2024-03-19T01:17:09.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.031+0000 7f5a59e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541180b0 con 0x7f5a5419e800 2024-03-19T01:17:09.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.031+0000 7f5a59e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541183b0 con 0x7f5a54076670 2024-03-19T01:17:09.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.032+0000 7f5a53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419e800 0x7f5a5419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.032+0000 7f5a53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419e800 0x7f5a5419ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43612/0 (socket says 172.21.15.69:43612) 2024-03-19T01:17:09.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.032+0000 7f5a53400640 1 -- 172.21.15.69:0/2914797072 learned_addr learned my addr 172.21.15.69:0/2914797072 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.032+0000 7f5a52000640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54112660 0x7f5a541a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.032+0000 7f5a53400640 1 -- 172.21.15.69:0/2914797072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a54076670 msgr2=0x7f5a541a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.032+0000 7f5a53400640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a54076670 0x7f5a541a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.032+0000 7f5a53400640 1 -- 172.21.15.69:0/2914797072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54112660 msgr2=0x7f5a541a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.032+0000 7f5a53400640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54112660 0x7f5a541a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.032+0000 7f5a53400640 1 -- 172.21.15.69:0/2914797072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5419f390 con 0x7f5a5419e800 2024-03-19T01:17:09.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.033+0000 7f5a53400640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419e800 0x7f5a5419ecb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00dac0 tx=0x7f5a4c00df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.033+0000 7f5a58a00640 1 -- 172.21.15.69:0/2914797072 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c017070 con 0x7f5a5419e800 2024-03-19T01:17:09.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.033+0000 7f5a59e00640 1 -- 172.21.15.69:0/2914797072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5419f680 con 0x7f5a5419e800 2024-03-19T01:17:09.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.033+0000 7f5a59e00640 1 -- 172.21.15.69:0/2914797072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a541ab810 con 0x7f5a5419e800 2024-03-19T01:17:09.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.034+0000 7f5a58a00640 1 -- 172.21.15.69:0/2914797072 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a4c003e50 con 0x7f5a5419e800 2024-03-19T01:17:09.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.034+0000 7f5a58a00640 1 -- 172.21.15.69:0/2914797072 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c0106c0 con 0x7f5a5419e800 2024-03-19T01:17:09.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.039+0000 7f5a5cbcd740 1 Processor -- start 2024-03-19T01:17:09.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.041+0000 7f5a58a00640 1 -- 172.21.15.69:0/2914797072 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f5a4c010860 con 0x7f5a5419e800 2024-03-19T01:17:09.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.041+0000 7f5a5cbcd740 1 -- start start 2024-03-19T01:17:09.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.041+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b2a20 0x5635ba5b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.041+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5b3340 0x5635ba5bb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.041+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5bbf20 0x5635ba5be310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.041+0000 7f5a5cbcd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5635ba517380 con 0x5635ba5b3340 2024-03-19T01:17:09.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.041+0000 7f5a5cbcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635ba4bfd20 con 0x5635ba5bbf20 2024-03-19T01:17:09.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.041+0000 7f5a5cbcd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5635ba513420 con 0x5635ba5b2a20 2024-03-19T01:17:09.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.042+0000 7f5a52a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5b3340 0x5635ba5bb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.042+0000 7f5a52a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5b3340 0x5635ba5bb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51734/0 (socket says 172.21.15.69:51734) 2024-03-19T01:17:09.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.042+0000 7f5a52a00640 1 -- 172.21.15.69:0/1151955469 learned_addr learned my addr 172.21.15.69:0/1151955469 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.042+0000 7f5a52000640 1 --2- 172.21.15.69:0/1151955469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b2a20 0x5635ba5b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.042+0000 7f5a52000640 1 --2- 172.21.15.69:0/1151955469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5bbf20 0x5635ba5be310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.043+0000 7f5a52a00640 1 -- 172.21.15.69:0/1151955469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b2a20 msgr2=0x5635ba5b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.043+0000 7f5a52a00640 1 --2- 172.21.15.69:0/1151955469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b2a20 0x5635ba5b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.043+0000 7f5a52a00640 1 -- 172.21.15.69:0/1151955469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5bbf20 msgr2=0x5635ba5be310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.043+0000 7f5a52a00640 1 --2- 172.21.15.69:0/1151955469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5bbf20 0x5635ba5be310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.043+0000 7f5a52a00640 1 -- 172.21.15.69:0/1151955469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ba22b3f0 con 0x5635ba5b3340 2024-03-19T01:17:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.043+0000 7f5a58a00640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5a380a2a70 0x7f5a380a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.044+0000 7f5a58a00640 1 -- 172.21.15.69:0/2914797072 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f5a4c0cec80 con 0x7f5a5419e800 2024-03-19T01:17:09.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.046+0000 7f5a52000640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5a380a2a70 0x7f5a380a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.046+0000 7f5a52a00640 1 --2- 172.21.15.69:0/1151955469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5b3340 0x5635ba5bb9e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5a44036040 tx=0x7f5a44005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03882c9bb46a2e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.047+0000 7f5a52000640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5a380a2a70 0x7f5a380a4f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x5635ba5ae2e0 tx=0x7f5a400046c0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.047+0000 7f5a3fe00640 1 -- 172.21.15.69:0/1151955469 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4402c870 con 0x5635ba5b3340 2024-03-19T01:17:09.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.048+0000 7f5a3fe00640 1 -- 172.21.15.69:0/1151955469 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a4402ca10 con 0x5635ba5b3340 2024-03-19T01:17:09.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.048+0000 7f5a3fe00640 1 -- 172.21.15.69:0/1151955469 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44003cb0 con 0x5635ba5b3340 2024-03-19T01:17:09.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.048+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/1151955469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5b3340 msgr2=0x5635ba5bb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.048+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/1151955469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5b3340 0x5635ba5bb9e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5a44036040 tx=0x7f5a44005760 comp rx=0 tx=0).stop 2024-03-19T01:17:09.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.050+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/1151955469 shutdown_connections 2024-03-19T01:17:09.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.050+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/1151955469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5bbf20 0x5635ba5be310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.050+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/1151955469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5b3340 0x5635ba5bb9e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.050+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/1151955469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b2a20 0x5635ba5b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.050+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/1151955469 >> 172.21.15.69:0/1151955469 conn(0x5635ba5ad150 msgr2=0x5635ba5ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.050+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/1151955469 shutdown_connections 2024-03-19T01:17:09.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.050+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/1151955469 wait complete. 2024-03-19T01:17:09.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.051+0000 7f5a5cbcd740 1 -- start start 2024-03-19T01:17:09.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.051+0000 7f5a5cbcd740 1 Processor -- start 2024-03-19T01:17:09.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.052+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.052+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b3340 0x5635ba5b9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.053+0000 7f5a52a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.053+0000 7f5a52a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5b8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43632/0 (socket says 172.21.15.69:43632) 2024-03-19T01:17:09.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.054+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5bbf20 0x5635ba5b9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.054+0000 7f5a52000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b3340 0x5635ba5b9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.054+0000 7f5a52a00640 1 -- 172.21.15.69:0/3762439444 learned_addr learned my addr 172.21.15.69:0/3762439444 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.054+0000 7f5a53400640 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5bbf20 0x5635ba5b9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.054+0000 7f5a5cbcd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5635ba5becb0 con 0x5635ba5bbf20 2024-03-19T01:17:09.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.054+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/3762439444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635ba4d3160 con 0x5635ba5b2a20 2024-03-19T01:17:09.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.054+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/3762439444 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5635ba5bee30 con 0x5635ba5b3340 2024-03-19T01:17:09.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.055+0000 7f5a5cbcd740 10 client.0 _ll_register_callbacks cb 0x5635ba50f170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:09.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.055+0000 7f5a52a00640 1 -- 172.21.15.69:0/3762439444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b3340 msgr2=0x5635ba5b9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.055+0000 7f5a52a00640 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b3340 0x5635ba5b9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.055+0000 7f5a52a00640 1 -- 172.21.15.69:0/3762439444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5bbf20 msgr2=0x5635ba5b9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.055+0000 7f5a52a00640 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5bbf20 0x5635ba5b9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.055+0000 7f5a52a00640 1 -- 172.21.15.69:0/3762439444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ba2f57d0 con 0x5635ba5b2a20 2024-03-19T01:17:09.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.055+0000 7f5a52a00640 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5b8ef0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5a4403ea90 tx=0x7f5a4403eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa3a6dcc1b15dcc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.055+0000 7f5a3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:17:09.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.056+0000 7f5a3cc00640 1 -- 172.21.15.69:0/3762439444 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4402c870 con 0x5635ba5b2a20 2024-03-19T01:17:09.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.056+0000 7f5a3cc00640 1 -- 172.21.15.69:0/3762439444 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f5a4403ece0 con 0x5635ba5b2a20 2024-03-19T01:17:09.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.056+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/3762439444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a4402ca10 con 0x5635ba5b2a20 2024-03-19T01:17:09.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.056+0000 7f5a3cc00640 1 -- 172.21.15.69:0/3762439444 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5a44057050 con 0x5635ba5b2a20 2024-03-19T01:17:09.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.057+0000 7f5a5cbcd740 10 client.27059 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:09.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.057+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/3762439444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=44}) -- 0x5635ba5caea0 con 0x5635ba5b2a20 2024-03-19T01:17:09.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.060+0000 7f5a3cc00640 1 -- 172.21.15.69:0/3762439444 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f5a44048450 con 0x5635ba5b2a20 2024-03-19T01:17:09.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.060+0000 7f5a5cbcd740 10 client.27059 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:09.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.060+0000 7f5a5cbcd740 10 client.27059 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:17:09.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.060+0000 7f5a5cbcd740 20 client.27059 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:09.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.060+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5635ba5cc1f0 0x5635ba5ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.061+0000 7f5a52000640 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5635ba5cc1f0 0x5635ba5ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.061+0000 7f5a5cbcd740 4 client.27059 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:09.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.061+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/3762439444 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635ba5248c0 con 0x5635ba5cc1f0 2024-03-19T01:17:09.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.063+0000 7f5a3cc00640 10 client.27059 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:17:09.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.063+0000 7f5a52000640 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5635ba5cc1f0 0x5635ba5ce5e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c205a002f684dfba server_cookie=aadd0858ff2cb12e in_seq=0 out_seq=0 2024-03-19T01:17:09.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.064+0000 7f5a3cc00640 1 -- 172.21.15.69:0/3762439444 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5635ba5248c0 con 0x5635ba5cc1f0 2024-03-19T01:17:09.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.064+0000 7f5a3cc00640 10 client.27059 handle_command_reply: tid=0 2024-03-19T01:17:09.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.064+0000 7f5a5cbcd740 1 client.27059 shutdown 2024-03-19T01:17:09.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.065+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/3762439444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 msgr2=0x5635ba5b8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.065+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5b8ef0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5a4403ea90 tx=0x7f5a4403eac0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.065+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/3762439444 shutdown_connections 2024-03-19T01:17:09.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.066+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5bbf20 0x5635ba5b9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.066+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b3340 0x5635ba5b9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.066+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5635ba5cc1f0 0x5635ba5ce5e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.066+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/3762439444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5b8ef0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.066+0000 7f5a3cc00640 0 client.27059 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:17:09.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.066+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/3762439444 >> 172.21.15.69:0/3762439444 conn(0x5635ba5ad150 msgr2=0x5635ba5aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.067+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/3762439444 shutdown_connections 2024-03-19T01:17:09.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.067+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/3762439444 wait complete. 2024-03-19T01:17:09.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.068+0000 7f5a5cbcd740 20 client.27059 trim_cache size 0 max 16384 2024-03-19T01:17:09.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.108+0000 7f5a5cbcd740 1 Processor -- start 2024-03-19T01:17:09.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.109+0000 7f5a5cbcd740 1 -- start start 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.109+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5bb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.109+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b3340 0x5635ba5c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.109+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5bbf20 0x5635ba5aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.109+0000 7f5a5cbcd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34010130 con 0x5635ba5bbf20 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.109+0000 7f5a5cbcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34011df0 con 0x5635ba5b2a20 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.109+0000 7f5a5cbcd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34010800 con 0x5635ba5b3340 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.109+0000 7f5a52a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.109+0000 7f5a52a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5bb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43644/0 (socket says 172.21.15.69:43644) 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.109+0000 7f5a52a00640 1 -- 172.21.15.69:0/2123521692 learned_addr learned my addr 172.21.15.69:0/2123521692 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.110+0000 7f5a52a00640 1 -- 172.21.15.69:0/2123521692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b3340 msgr2=0x5635ba5c7ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.110+0000 7f5a52a00640 1 --2- 172.21.15.69:0/2123521692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b3340 0x5635ba5c7ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.110+0000 7f5a52a00640 1 -- 172.21.15.69:0/2123521692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5bbf20 msgr2=0x5635ba5aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.110+0000 7f5a52a00640 1 --2- 172.21.15.69:0/2123521692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5bbf20 0x5635ba5aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.110+0000 7f5a52a00640 1 -- 172.21.15.69:0/2123521692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ba5c8020 con 0x5635ba5b2a20 2024-03-19T01:17:09.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.111+0000 7f5a52a00640 1 --2- 172.21.15.69:0/2123521692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5bb620 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5a4403e800 tx=0x7f5a44049000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=774ed034d15510f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.112+0000 7f5a3f400640 1 -- 172.21.15.69:0/2123521692 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44000e60 con 0x5635ba5b2a20 2024-03-19T01:17:09.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.112+0000 7f5a3f400640 1 -- 172.21.15.69:0/2123521692 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a4402ca10 con 0x5635ba5b2a20 2024-03-19T01:17:09.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.112+0000 7f5a3f400640 1 -- 172.21.15.69:0/2123521692 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44048b60 con 0x5635ba5b2a20 2024-03-19T01:17:09.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.114+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/2123521692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 msgr2=0x5635ba5bb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.114+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/2123521692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5bb620 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5a4403e800 tx=0x7f5a44049000 comp rx=0 tx=0).stop 2024-03-19T01:17:09.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.114+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/2123521692 shutdown_connections 2024-03-19T01:17:09.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.114+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/2123521692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5bbf20 0x5635ba5aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.115+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/2123521692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b3340 0x5635ba5c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.115+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/2123521692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5b2a20 0x5635ba5bb620 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.115+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/2123521692 >> 172.21.15.69:0/2123521692 conn(0x5635ba5ad150 msgr2=0x5635ba5ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.115+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/2123521692 shutdown_connections 2024-03-19T01:17:09.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.115+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/2123521692 wait complete. 2024-03-19T01:17:09.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.116+0000 7f5a5cbcd740 1 -- start start 2024-03-19T01:17:09.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.116+0000 7f5a5cbcd740 1 Processor -- start 2024-03-19T01:17:09.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.117+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b2a20 0x5635ba5bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.117+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5b3340 0x5635ba5bff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.117+0000 7f5a5cbcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5bbf20 0x5635ba5c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.118+0000 7f5a52a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b2a20 0x5635ba5bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.118+0000 7f5a52a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b2a20 0x5635ba5bfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46940/0 (socket says 172.21.15.69:46940) 2024-03-19T01:17:09.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.118+0000 7f5a52a00640 1 -- 172.21.15.69:0/4257525617 learned_addr learned my addr 172.21.15.69:0/4257525617 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.118+0000 7f5a53400640 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5bbf20 0x5635ba5c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.118+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/4257525617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34011660 con 0x5635ba5b3340 2024-03-19T01:17:09.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.118+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/4257525617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34010f60 con 0x5635ba5bbf20 2024-03-19T01:17:09.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.118+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/4257525617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3400e0a0 con 0x5635ba5b2a20 2024-03-19T01:17:09.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.118+0000 7f5a5cbcd740 10 client.0 _ll_register_callbacks cb 0x7f5a4000e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:09.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.118+0000 7f5a53400640 1 -- 172.21.15.69:0/4257525617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b2a20 msgr2=0x5635ba5bfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.118+0000 7f5a53400640 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b2a20 0x5635ba5bfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.119+0000 7f5a53400640 1 -- 172.21.15.69:0/4257525617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5b3340 msgr2=0x5635ba5bff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.119+0000 7f5a53400640 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5b3340 0x5635ba5bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.119+0000 7f5a53400640 1 -- 172.21.15.69:0/4257525617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ba631270 con 0x5635ba5bbf20 2024-03-19T01:17:09.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.120+0000 7f5a53400640 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5bbf20 0x5635ba5c04b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0cbf00 tx=0x7f5a4c09a860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=619f5695c15f3418 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.120+0000 7f5a3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:17:09.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.120+0000 7f5a3cc00640 1 -- 172.21.15.69:0/4257525617 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c020070 con 0x5635ba5bbf20 2024-03-19T01:17:09.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.120+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/4257525617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a4402ca10 con 0x5635ba5bbf20 2024-03-19T01:17:09.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.120+0000 7f5a3cc00640 1 -- 172.21.15.69:0/4257525617 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f5a4c097480 con 0x5635ba5bbf20 2024-03-19T01:17:09.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.121+0000 7f5a3cc00640 1 -- 172.21.15.69:0/4257525617 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5a4c020070 con 0x5635ba5bbf20 2024-03-19T01:17:09.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.121+0000 7f5a5cbcd740 10 client.27065 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:09.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.121+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/4257525617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5635ba632600 con 0x5635ba5bbf20 2024-03-19T01:17:09.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.121+0000 7f5a3cc00640 1 -- 172.21.15.69:0/4257525617 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a4402ca10 con 0x5635ba5bbf20 2024-03-19T01:17:09.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.122+0000 7f5a3cc00640 1 -- 172.21.15.69:0/4257525617 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f5a4c023060 con 0x5635ba5bbf20 2024-03-19T01:17:09.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.122+0000 7f5a5cbcd740 10 client.27065 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:09.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.122+0000 7f5a5cbcd740 10 client.27065 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:17:09.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.122+0000 7f5a5cbcd740 20 client.27065 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:09.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.122+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5635ba632e80 0x5635ba6352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.122+0000 7f5a5cbcd740 4 client.27065 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:17:09.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.122+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/4257525617 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635ba51ec10 con 0x5635ba632e80 2024-03-19T01:17:09.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.123+0000 7f5a52a00640 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5635ba632e80 0x5635ba6352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.125+0000 7f5a52a00640 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5635ba632e80 0x5635ba6352a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=204e8f858a277187 server_cookie=729087d5134b0054 in_seq=0 out_seq=0 2024-03-19T01:17:09.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.125+0000 7f5a3cc00640 10 client.27065 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:17:09.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.127+0000 7f5a3cc00640 1 -- 172.21.15.69:0/4257525617 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635ba51ec10 con 0x5635ba632e80 2024-03-19T01:17:09.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.128+0000 7f5a3cc00640 10 client.27065 handle_command_reply: tid=0 2024-03-19T01:17:09.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.128+0000 7f5a5cbcd740 1 client.27065 shutdown 2024-03-19T01:17:09.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.130+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/4257525617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5bbf20 msgr2=0x5635ba5c04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.130+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5bbf20 0x5635ba5c04b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0cbf00 tx=0x7f5a4c09a860 comp rx=0 tx=0).stop 2024-03-19T01:17:09.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.131+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/4257525617 shutdown_connections 2024-03-19T01:17:09.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.131+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5635ba632e80 0x5635ba6352a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.131+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ba5bbf20 0x5635ba5c04b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.131+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635ba5b3340 0x5635ba5bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.131+0000 7f5a5cbcd740 1 --2- 172.21.15.69:0/4257525617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635ba5b2a20 0x5635ba5bfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.131+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/4257525617 >> 172.21.15.69:0/4257525617 conn(0x5635ba5ad150 msgr2=0x5635ba5ac1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.131+0000 7f5a3cc00640 0 client.27065 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:17:09.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.131+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/4257525617 shutdown_connections 2024-03-19T01:17:09.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.131+0000 7f5a5cbcd740 1 -- 172.21.15.69:0/4257525617 wait complete. 2024-03-19T01:17:09.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.131+0000 7f5a5cbcd740 20 client.27065 trim_cache size 0 max 16384 2024-03-19T01:17:09.133 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:17:09.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.133+0000 7f5a59e00640 1 -- 172.21.15.69:0/2914797072 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5a380a2a70 msgr2=0x7f5a380a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.133+0000 7f5a59e00640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5a380a2a70 0x7f5a380a4f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x5635ba5ae2e0 tx=0x7f5a400046c0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.133+0000 7f5a59e00640 1 -- 172.21.15.69:0/2914797072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419e800 msgr2=0x7f5a5419ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.133+0000 7f5a59e00640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419e800 0x7f5a5419ecb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00dac0 tx=0x7f5a4c00df80 comp rx=0 tx=0).stop 2024-03-19T01:17:09.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.133+0000 7f5a59e00640 1 -- 172.21.15.69:0/2914797072 shutdown_connections 2024-03-19T01:17:09.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.133+0000 7f5a59e00640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5a380a2a70 0x7f5a380a4f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.133+0000 7f5a59e00640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5419e800 0x7f5a5419ecb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.133+0000 7f5a59e00640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54112660 0x7f5a541a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.133+0000 7f5a59e00640 1 --2- 172.21.15.69:0/2914797072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a54076670 0x7f5a541a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.133+0000 7f5a59e00640 1 -- 172.21.15.69:0/2914797072 >> 172.21.15.69:0/2914797072 conn(0x7f5a540ff4d0 msgr2=0x7f5a541018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.134+0000 7f5a59e00640 1 -- 172.21.15.69:0/2914797072 shutdown_connections 2024-03-19T01:17:09.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.134+0000 7f5a59e00640 1 -- 172.21.15.69:0/2914797072 wait complete. 2024-03-19T01:17:09.164 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:17:09.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.420+0000 7f5c32e00640 1 -- 172.21.15.69:0/3870297214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1038a0 msgr2=0x7f5c2c103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.420+0000 7f5c32e00640 1 --2- 172.21.15.69:0/3870297214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1038a0 0x7f5c2c103d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009930 tx=0x7f5c1802bc30 comp rx=0 tx=0).stop 2024-03-19T01:17:09.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.421+0000 7f5c32e00640 1 -- 172.21.15.69:0/3870297214 shutdown_connections 2024-03-19T01:17:09.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.421+0000 7f5c32e00640 1 --2- 172.21.15.69:0/3870297214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c2c104260 0x7f5c2c110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.421+0000 7f5c32e00640 1 --2- 172.21.15.69:0/3870297214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1038a0 0x7f5c2c103d20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.421+0000 7f5c32e00640 1 --2- 172.21.15.69:0/3870297214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c2c1098b0 0x7f5c2c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.421+0000 7f5c32e00640 1 -- 172.21.15.69:0/3870297214 >> 172.21.15.69:0/3870297214 conn(0x7f5c2c0ff330 msgr2=0x7f5c2c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.421+0000 7f5c32e00640 1 -- 172.21.15.69:0/3870297214 shutdown_connections 2024-03-19T01:17:09.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.421+0000 7f5c32e00640 1 -- 172.21.15.69:0/3870297214 wait complete. 2024-03-19T01:17:09.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c32e00640 1 Processor -- start 2024-03-19T01:17:09.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c32e00640 1 -- start start 2024-03-19T01:17:09.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c32e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c2c1038a0 0x7f5c2c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c32e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c2c104260 0x7f5c2c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c32e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1098b0 0x7f5c2c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c32e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c113cc0 con 0x7f5c2c1098b0 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c32e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c113b40 con 0x7f5c2c1038a0 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c32e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c113e40 con 0x7f5c2c104260 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c31a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c2c1038a0 0x7f5c2c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c31a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c2c1038a0 0x7f5c2c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43656/0 (socket says 172.21.15.69:43656) 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c31a00640 1 -- 172.21.15.69:0/136045069 learned_addr learned my addr 172.21.15.69:0/136045069 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c31000640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c2c104260 0x7f5c2c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.422+0000 7f5c32400640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1098b0 0x7f5c2c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.423+0000 7f5c31a00640 1 -- 172.21.15.69:0/136045069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c2c104260 msgr2=0x7f5c2c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.423+0000 7f5c31a00640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c2c104260 0x7f5c2c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.423+0000 7f5c31a00640 1 -- 172.21.15.69:0/136045069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1098b0 msgr2=0x7f5c2c19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.423+0000 7f5c31a00640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1098b0 0x7f5c2c19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.423+0000 7f5c31a00640 1 -- 172.21.15.69:0/136045069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c19f1a0 con 0x7f5c2c1038a0 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.423+0000 7f5c31a00640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c2c1038a0 0x7f5c2c19a200 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000e9f0 tx=0x7f5c2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.423+0000 7f5c2a000640 1 -- 172.21.15.69:0/136045069 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000cd80 con 0x7f5c2c1038a0 2024-03-19T01:17:09.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.423+0000 7f5c32e00640 1 -- 172.21.15.69:0/136045069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2c19f490 con 0x7f5c2c1038a0 2024-03-19T01:17:09.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.423+0000 7f5c2a000640 1 -- 172.21.15.69:0/136045069 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c20010910 con 0x7f5c2c1038a0 2024-03-19T01:17:09.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.423+0000 7f5c2a000640 1 -- 172.21.15.69:0/136045069 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c200188f0 con 0x7f5c2c1038a0 2024-03-19T01:17:09.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.423+0000 7f5c32e00640 1 -- 172.21.15.69:0/136045069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2c1a6d20 con 0x7f5c2c1038a0 2024-03-19T01:17:09.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.426+0000 7f5c37186740 1 Processor -- start 2024-03-19T01:17:09.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.426+0000 7f5c37186740 1 -- start start 2024-03-19T01:17:09.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c37186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2ca9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c37186740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2caa510 0x55a1b2cb2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c37186740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c37186740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2c0e380 con 0x55a1b2cb30f0 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c37186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2bb6d20 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c37186740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2c0a420 con 0x55a1b2caa510 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c32400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2ca9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c32400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2ca9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43666/0 (socket says 172.21.15.69:43666) 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c32400640 1 -- 172.21.15.69:0/3053747914 learned_addr learned my addr 172.21.15.69:0/3053747914 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c31000640 1 --2- 172.21.15.69:0/3053747914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c2a000640 1 -- 172.21.15.69:0/136045069 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f5c20010ac0 con 0x7f5c2c1038a0 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c32400640 1 -- 172.21.15.69:0/3053747914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2caa510 msgr2=0x55a1b2cb2bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c32400640 1 --2- 172.21.15.69:0/3053747914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2caa510 0x55a1b2cb2bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c32400640 1 -- 172.21.15.69:0/3053747914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2cb30f0 msgr2=0x55a1b2cb54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c32400640 1 --2- 172.21.15.69:0/3053747914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.427+0000 7f5c32400640 1 -- 172.21.15.69:0/3053747914 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b29223f0 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.428+0000 7f5c32400640 1 --2- 172.21.15.69:0/3053747914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2ca9fd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009fd0 tx=0x7f5c240118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f018c20e128180e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.428+0000 7f5c1ea00640 1 -- 172.21.15.69:0/3053747914 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24004d20 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.428+0000 7f5c1ea00640 1 -- 172.21.15.69:0/3053747914 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c24004050 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.428+0000 7f5c1ea00640 1 -- 172.21.15.69:0/3053747914 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24005280 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.428+0000 7f5c37186740 1 -- 172.21.15.69:0/3053747914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2ca9bf0 msgr2=0x55a1b2ca9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.428+0000 7f5c37186740 1 --2- 172.21.15.69:0/3053747914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2ca9fd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009fd0 tx=0x7f5c240118d0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.429+0000 7f5c37186740 1 -- 172.21.15.69:0/3053747914 shutdown_connections 2024-03-19T01:17:09.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.429+0000 7f5c37186740 1 --2- 172.21.15.69:0/3053747914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.429+0000 7f5c37186740 1 --2- 172.21.15.69:0/3053747914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2caa510 0x55a1b2cb2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.429+0000 7f5c37186740 1 --2- 172.21.15.69:0/3053747914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2ca9fd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.429+0000 7f5c37186740 1 -- 172.21.15.69:0/3053747914 >> 172.21.15.69:0/3053747914 conn(0x55a1b2ca4320 msgr2=0x55a1b2ca4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.429+0000 7f5c37186740 1 -- 172.21.15.69:0/3053747914 shutdown_connections 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.429+0000 7f5c37186740 1 -- 172.21.15.69:0/3053747914 wait complete. 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.429+0000 7f5c37186740 1 -- start start 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.429+0000 7f5c37186740 1 Processor -- start 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c2a000640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5c140a2900 0x7f5c140a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c2a000640 1 -- 172.21.15.69:0/136045069 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f5c200cf580 con 0x7f5c2c1038a0 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c32400640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5c140a2900 0x7f5c140a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c37186740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cb0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c37186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2caa510 0x55a1b2cb0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c37186740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c37186740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2cb5ee0 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c37186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2bca160 con 0x55a1b2caa510 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c37186740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2cb6060 con 0x55a1b2cb30f0 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c37186740 10 client.0 _ll_register_callbacks cb 0x55a1b2c06170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c32400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c31a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cb0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c32400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46966/0 (socket says 172.21.15.69:46966) 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c32400640 1 -- 172.21.15.69:0/2563160586 learned_addr learned my addr 172.21.15.69:0/2563160586 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.430+0000 7f5c31000640 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2caa510 0x55a1b2cb0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c31a00640 1 -- 172.21.15.69:0/2563160586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2cb30f0 msgr2=0x55a1b2cb0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c31a00640 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c31a00640 1 -- 172.21.15.69:0/2563160586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2caa510 msgr2=0x55a1b2cb0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c31a00640 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2caa510 0x55a1b2cb0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c31a00640 1 -- 172.21.15.69:0/2563160586 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b29ec7d0 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c31000640 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2caa510 0x55a1b2cb0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c32400640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5c140a2900 0x7f5c140a4dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009fd0 tx=0x7f5c24004cb0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c31a00640 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cb0100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c200cc860 tx=0x7f5c2009b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff791c55d2860b0c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c13400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c13400640 1 -- 172.21.15.69:0/2563160586 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c200a3f10 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c13400640 1 -- 172.21.15.69:0/2563160586 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f5c200cf170 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.431+0000 7f5c37186740 1 -- 172.21.15.69:0/2563160586 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c24004050 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c13400640 1 -- 172.21.15.69:0/2563160586 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5c200cf370 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c37186740 10 client.16809 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c37186740 1 -- 172.21.15.69:0/2563160586 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a1b2cc2140 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c13400640 1 -- 172.21.15.69:0/2563160586 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c24004050 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c13400640 1 -- 172.21.15.69:0/2563160586 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f5c20097700 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c37186740 10 client.16809 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c37186740 10 client.16809 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c37186740 20 client.16809 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c37186740 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a1b2cc3600 0x55a1b2cc59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c37186740 4 client.16809 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c37186740 1 -- 172.21.15.69:0/2563160586 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1b2c1b8c0 con 0x55a1b2cc3600 2024-03-19T01:17:09.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.432+0000 7f5c31000640 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a1b2cc3600 0x55a1b2cc59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.433+0000 7f5c31000640 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a1b2cc3600 0x55a1b2cc59f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e8dd06086a05072 server_cookie=8c902fc9ea248a1e in_seq=0 out_seq=0 2024-03-19T01:17:09.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.433+0000 7f5c13400640 10 client.16809 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c13400640 1 -- 172.21.15.69:0/2563160586 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a1b2c1b8c0 con 0x55a1b2cc3600 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c13400640 10 client.16809 handle_command_reply: tid=0 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 1 client.16809 shutdown 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 1 -- 172.21.15.69:0/2563160586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 msgr2=0x55a1b2cb0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cb0100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c200cc860 tx=0x7f5c2009b460 comp rx=0 tx=0).stop 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 1 -- 172.21.15.69:0/2563160586 shutdown_connections 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a1b2cc3600 0x55a1b2cc59f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2caa510 0x55a1b2cb0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c13400640 0 client.16809 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 1 --2- 172.21.15.69:0/2563160586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cb0100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 1 -- 172.21.15.69:0/2563160586 >> 172.21.15.69:0/2563160586 conn(0x55a1b2ca4320 msgr2=0x55a1b2ca6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 1 -- 172.21.15.69:0/2563160586 shutdown_connections 2024-03-19T01:17:09.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 1 -- 172.21.15.69:0/2563160586 wait complete. 2024-03-19T01:17:09.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.434+0000 7f5c37186740 20 client.16809 trim_cache size 0 max 16384 2024-03-19T01:17:09.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.448+0000 7f5c37186740 1 Processor -- start 2024-03-19T01:17:09.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.448+0000 7f5c37186740 1 -- start start 2024-03-19T01:17:09.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.448+0000 7f5c37186740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cb2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.448+0000 7f5c37186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2caa510 0x55a1b2cbec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.448+0000 7f5c37186740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2cb30f0 0x55a1b2ca31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.448+0000 7f5c37186740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1b10a0 con 0x55a1b2cb30f0 2024-03-19T01:17:09.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.448+0000 7f5c37186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1b2d60 con 0x55a1b2caa510 2024-03-19T01:17:09.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.448+0000 7f5c37186740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1b1770 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.449+0000 7f5c31a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cb2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.449+0000 7f5c31a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cb2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46970/0 (socket says 172.21.15.69:46970) 2024-03-19T01:17:09.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.449+0000 7f5c31a00640 1 -- 172.21.15.69:0/871540647 learned_addr learned my addr 172.21.15.69:0/871540647 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.449+0000 7f5c31000640 1 --2- 172.21.15.69:0/871540647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2caa510 0x55a1b2cbec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.449+0000 7f5c32400640 1 --2- 172.21.15.69:0/871540647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2cb30f0 0x55a1b2ca31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.449+0000 7f5c31a00640 1 -- 172.21.15.69:0/871540647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2caa510 msgr2=0x55a1b2cbec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.449+0000 7f5c31a00640 1 --2- 172.21.15.69:0/871540647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2caa510 0x55a1b2cbec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.449+0000 7f5c31a00640 1 -- 172.21.15.69:0/871540647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2cb30f0 msgr2=0x55a1b2ca31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.449+0000 7f5c31a00640 1 --2- 172.21.15.69:0/871540647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2cb30f0 0x55a1b2ca31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.449+0000 7f5c31a00640 1 -- 172.21.15.69:0/871540647 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b2ca3b60 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.449+0000 7f5c31a00640 1 --2- 172.21.15.69:0/871540647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cb2830 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5c2009b4d0 tx=0x7f5c200cf510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddc6e99f650f9a95 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c1e000640 1 -- 172.21.15.69:0/871540647 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20097e90 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c1e000640 1 -- 172.21.15.69:0/871540647 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c20097570 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c1e000640 1 -- 172.21.15.69:0/871540647 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20098030 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c37186740 1 -- 172.21.15.69:0/871540647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2ca9bf0 msgr2=0x55a1b2cb2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c37186740 1 --2- 172.21.15.69:0/871540647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cb2830 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5c2009b4d0 tx=0x7f5c200cf510 comp rx=0 tx=0).stop 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c37186740 1 -- 172.21.15.69:0/871540647 shutdown_connections 2024-03-19T01:17:09.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c37186740 1 --2- 172.21.15.69:0/871540647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2cb30f0 0x55a1b2ca31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c37186740 1 --2- 172.21.15.69:0/871540647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2caa510 0x55a1b2cbec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c37186740 1 --2- 172.21.15.69:0/871540647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cb2830 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c37186740 1 -- 172.21.15.69:0/871540647 >> 172.21.15.69:0/871540647 conn(0x55a1b2ca4320 msgr2=0x55a1b2ca2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c37186740 1 -- 172.21.15.69:0/871540647 shutdown_connections 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.450+0000 7f5c37186740 1 -- 172.21.15.69:0/871540647 wait complete. 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.451+0000 7f5c37186740 1 -- start start 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.451+0000 7f5c37186740 1 Processor -- start 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.451+0000 7f5c37186740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cbb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.451+0000 7f5c37186740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2caa510 0x55a1b2cb68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.451+0000 7f5c37186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.451+0000 7f5c37186740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1b25d0 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.451+0000 7f5c37186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1b1ed0 con 0x55a1b2cb30f0 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.451+0000 7f5c37186740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1af010 con 0x55a1b2caa510 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.451+0000 7f5c37186740 10 client.0 _ll_register_callbacks cb 0x7f5c2000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c32400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cbb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2caa510 0x55a1b2cb68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2caa510 0x55a1b2cb68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46972/0 (socket says 172.21.15.69:46972) 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cbb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51820/0 (socket says 172.21.15.69:51820) 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31000640 1 -- 172.21.15.69:0/3874121084 learned_addr learned my addr 172.21.15.69:0/3874121084 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c32400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43700/0 (socket says 172.21.15.69:43700) 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31a00640 1 -- 172.21.15.69:0/3874121084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2caa510 msgr2=0x55a1b2cb68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31a00640 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2caa510 0x55a1b2cb68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31a00640 1 -- 172.21.15.69:0/3874121084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2cb30f0 msgr2=0x55a1b2cb6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31a00640 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31a00640 1 -- 172.21.15.69:0/3874121084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b2cb7630 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31000640 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2caa510 0x55a1b2cb68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:09.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c31a00640 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cbb580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5c200981f0 tx=0x7f5c2001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec700fe1b0ba803 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c13400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:17:09.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.452+0000 7f5c13400640 1 -- 172.21.15.69:0/3874121084 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2009a730 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c13400640 1 -- 172.21.15.69:0/3874121084 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f5c200afb50 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c37186740 1 -- 172.21.15.69:0/3874121084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c20097570 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c13400640 1 -- 172.21.15.69:0/3874121084 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5c200a3f10 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c37186740 10 client.16815 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c37186740 1 -- 172.21.15.69:0/3874121084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a1b2cb7630 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c13400640 1 -- 172.21.15.69:0/3874121084 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c20097570 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c13400640 1 -- 172.21.15.69:0/3874121084 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f5c2009cb30 con 0x55a1b2ca9bf0 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c37186740 10 client.16815 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c37186740 10 client.16815 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c37186740 20 client.16815 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c37186740 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a1b2d29bb0 0x55a1b2d2bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c37186740 4 client.16815 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c37186740 1 -- 172.21.15.69:0/3874121084 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1b2c15c10 con 0x55a1b2d29bb0 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.453+0000 7f5c31000640 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a1b2d29bb0 0x55a1b2d2bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c31000640 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a1b2d29bb0 0x55a1b2d2bfa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a74f9aa619285ecb server_cookie=ac72eefba9868547 in_seq=0 out_seq=0 2024-03-19T01:17:09.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c13400640 10 client.16815 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c13400640 1 -- 172.21.15.69:0/3874121084 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1b2c15c10 con 0x55a1b2d29bb0 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c13400640 10 client.16815 handle_command_reply: tid=0 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c37186740 1 client.16815 shutdown 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c37186740 1 -- 172.21.15.69:0/3874121084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 msgr2=0x55a1b2cbb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c37186740 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cbb580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5c200981f0 tx=0x7f5c2001f000 comp rx=0 tx=0).stop 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c37186740 1 -- 172.21.15.69:0/3874121084 shutdown_connections 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c37186740 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55a1b2d29bb0 0x55a1b2d2bfa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c37186740 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1b2cb30f0 0x55a1b2cb6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c37186740 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1b2caa510 0x55a1b2cb68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c37186740 1 --2- 172.21.15.69:0/3874121084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2ca9bf0 0x55a1b2cbb580 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c37186740 1 -- 172.21.15.69:0/3874121084 >> 172.21.15.69:0/3874121084 conn(0x55a1b2ca4320 msgr2=0x55a1b2ca5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.454+0000 7f5c13400640 0 client.16815 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c37186740 1 -- 172.21.15.69:0/3874121084 shutdown_connections 2024-03-19T01:17:09.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c37186740 1 -- 172.21.15.69:0/3874121084 wait complete. 2024-03-19T01:17:09.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c37186740 20 client.16815 trim_cache size 0 max 16384 2024-03-19T01:17:09.461 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:17:09.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c32e00640 1 -- 172.21.15.69:0/136045069 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5c140a2900 msgr2=0x7f5c140a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c32e00640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5c140a2900 0x7f5c140a4dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009fd0 tx=0x7f5c24004cb0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c32e00640 1 -- 172.21.15.69:0/136045069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c2c1038a0 msgr2=0x7f5c2c19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c32e00640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c2c1038a0 0x7f5c2c19a200 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000e9f0 tx=0x7f5c2000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c32e00640 1 -- 172.21.15.69:0/136045069 shutdown_connections 2024-03-19T01:17:09.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c32e00640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c2c1098b0 0x7f5c2c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c32e00640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c2c104260 0x7f5c2c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c32e00640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5c140a2900 0x7f5c140a4dc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c32e00640 1 --2- 172.21.15.69:0/136045069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c2c1038a0 0x7f5c2c19a200 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.455+0000 7f5c32e00640 1 -- 172.21.15.69:0/136045069 >> 172.21.15.69:0/136045069 conn(0x7f5c2c0ff330 msgr2=0x7f5c2c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.456+0000 7f5c32e00640 1 -- 172.21.15.69:0/136045069 shutdown_connections 2024-03-19T01:17:09.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.456+0000 7f5c32e00640 1 -- 172.21.15.69:0/136045069 wait complete. 2024-03-19T01:17:09.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:09 smithi069 ceph-mon[31727]: pgmap v198: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 1.69k op/s 2024-03-19T01:17:09.471 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:09 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:09.473 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:17:09.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:09 smithi086 ceph-mon[66829]: pgmap v198: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 1.69k op/s 2024-03-19T01:17:09.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:09 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:09.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:09 smithi163 ceph-mon[67428]: pgmap v198: 129 pgs: 129 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 1.69k op/s 2024-03-19T01:17:09.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:09 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.670+0000 7f1502c00640 1 -- 172.21.15.69:0/4065861058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14fc102170 msgr2=0x7f14fc10e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.670+0000 7f1502c00640 1 --2- 172.21.15.69:0/4065861058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14fc102170 0x7f14fc10e680 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009c10 tx=0x7f14f402bee0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.670+0000 7f1502c00640 1 -- 172.21.15.69:0/4065861058 shutdown_connections 2024-03-19T01:17:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.670+0000 7f1502c00640 1 --2- 172.21.15.69:0/4065861058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14fc102170 0x7f14fc10e680 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.670+0000 7f1502c00640 1 --2- 172.21.15.69:0/4065861058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14fc1017b0 0x7f14fc101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.670+0000 7f1502c00640 1 --2- 172.21.15.69:0/4065861058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc1077c0 0x7f14fc107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.670+0000 7f1502c00640 1 -- 172.21.15.69:0/4065861058 >> 172.21.15.69:0/4065861058 conn(0x7f14fc0fd260 msgr2=0x7f14fc0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.670+0000 7f1502c00640 1 -- 172.21.15.69:0/4065861058 shutdown_connections 2024-03-19T01:17:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.670+0000 7f1502c00640 1 -- 172.21.15.69:0/4065861058 wait complete. 2024-03-19T01:17:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.671+0000 7f1502c00640 1 Processor -- start 2024-03-19T01:17:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.671+0000 7f1502c00640 1 -- start start 2024-03-19T01:17:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f1502c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc1017b0 0x7f14fc19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f1502c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14fc102170 0x7f14fc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f1502c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14fc1077c0 0x7f14fc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f1502c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc111c20 con 0x7f14fc1077c0 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f1502c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc111aa0 con 0x7f14fc1017b0 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f1502c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc111da0 con 0x7f14fc102170 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f14fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc1017b0 0x7f14fc19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f14fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14fc102170 0x7f14fc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f14fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc1017b0 0x7f14fc19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43726/0 (socket says 172.21.15.69:43726) 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f14fbe00640 1 -- 172.21.15.69:0/750256218 learned_addr learned my addr 172.21.15.69:0/750256218 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f14fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14fc102170 0x7f14fc19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46994/0 (socket says 172.21.15.69:46994) 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f1500c00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14fc1077c0 0x7f14fc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f14fbe00640 1 -- 172.21.15.69:0/750256218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14fc102170 msgr2=0x7f14fc19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f14fbe00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14fc102170 0x7f14fc19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f14fbe00640 1 -- 172.21.15.69:0/750256218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14fc1077c0 msgr2=0x7f14fc19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.672+0000 7f14fbe00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14fc1077c0 0x7f14fc19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.673+0000 7f14fbe00640 1 -- 172.21.15.69:0/750256218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc19f270 con 0x7f14fc1017b0 2024-03-19T01:17:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.673+0000 7f14fb400640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14fc102170 0x7f14fc19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.673+0000 7f14fbe00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc1017b0 0x7f14fc19a280 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00e9c0 tx=0x7f14ec00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.673+0000 7f14f8c00640 1 -- 172.21.15.69:0/750256218 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec00cde0 con 0x7f14fc1017b0 2024-03-19T01:17:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.673+0000 7f1502c00640 1 -- 172.21.15.69:0/750256218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc19f560 con 0x7f14fc1017b0 2024-03-19T01:17:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.673+0000 7f14f8c00640 1 -- 172.21.15.69:0/750256218 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ec0035f0 con 0x7f14fc1017b0 2024-03-19T01:17:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.673+0000 7f14f8c00640 1 -- 172.21.15.69:0/750256218 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec010650 con 0x7f14fc1017b0 2024-03-19T01:17:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.673+0000 7f1502c00640 1 -- 172.21.15.69:0/750256218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc1a6df0 con 0x7f14fc1017b0 2024-03-19T01:17:09.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.678+0000 7f15059a7740 1 Processor -- start 2024-03-19T01:17:09.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.678+0000 7f15059a7740 1 -- start start 2024-03-19T01:17:09.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.678+0000 7f14f8c00640 1 -- 172.21.15.69:0/750256218 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f14ec0037a0 con 0x7f14fc1017b0 2024-03-19T01:17:09.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.678+0000 7f15059a7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a1a20 0x5577266a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.678+0000 7f15059a7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a2340 0x5577266aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.678+0000 7f15059a7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266aaf20 0x5577266ad310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.678+0000 7f15059a7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557726606380 con 0x5577266a2340 2024-03-19T01:17:09.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.678+0000 7f15059a7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577265aed20 con 0x5577266aaf20 2024-03-19T01:17:09.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.678+0000 7f15059a7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557726602420 con 0x5577266a1a20 2024-03-19T01:17:09.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f1500c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a1a20 0x5577266a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f1500c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a1a20 0x5577266a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:47004/0 (socket says 172.21.15.69:47004) 2024-03-19T01:17:09.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f1500c00640 1 -- 172.21.15.69:0/3410181080 learned_addr learned my addr 172.21.15.69:0/3410181080 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f14fb400640 1 --2- 172.21.15.69:0/3410181080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266aaf20 0x5577266ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f14fbe00640 1 --2- 172.21.15.69:0/3410181080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a2340 0x5577266aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f1500c00640 1 -- 172.21.15.69:0/3410181080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266aaf20 msgr2=0x5577266ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f1500c00640 1 --2- 172.21.15.69:0/3410181080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266aaf20 0x5577266ad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f1500c00640 1 -- 172.21.15.69:0/3410181080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a2340 msgr2=0x5577266aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f1500c00640 1 --2- 172.21.15.69:0/3410181080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a2340 0x5577266aa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f1500c00640 1 -- 172.21.15.69:0/3410181080 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55772631a3f0 con 0x5577266a1a20 2024-03-19T01:17:09.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f14fbe00640 1 --2- 172.21.15.69:0/3410181080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a2340 0x5577266aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:09.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.679+0000 7f1500c00640 1 --2- 172.21.15.69:0/3410181080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a1a20 0x5577266a1e00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009be0 tx=0x7f14f4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4da91e21fd79807 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.680+0000 7f14f8c00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f14e00a2970 0x7f14e00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.680+0000 7f14f1200640 1 -- 172.21.15.69:0/3410181080 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4043030 con 0x5577266a1a20 2024-03-19T01:17:09.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.680+0000 7f14f1200640 1 -- 172.21.15.69:0/3410181080 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f4033c40 con 0x5577266a1a20 2024-03-19T01:17:09.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.680+0000 7f14f1200640 1 -- 172.21.15.69:0/3410181080 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f402c880 con 0x5577266a1a20 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.680+0000 7f14f8c00640 1 -- 172.21.15.69:0/750256218 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f14ec0cf540 con 0x7f14fc1017b0 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.680+0000 7f14fbe00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f14e00a2970 0x7f14e00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.680+0000 7f15059a7740 1 -- 172.21.15.69:0/3410181080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a1a20 msgr2=0x5577266a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.680+0000 7f15059a7740 1 --2- 172.21.15.69:0/3410181080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a1a20 0x5577266a1e00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009be0 tx=0x7f14f4009840 comp rx=0 tx=0).stop 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.681+0000 7f15059a7740 1 -- 172.21.15.69:0/3410181080 shutdown_connections 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.681+0000 7f15059a7740 1 --2- 172.21.15.69:0/3410181080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266aaf20 0x5577266ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.681+0000 7f15059a7740 1 --2- 172.21.15.69:0/3410181080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a2340 0x5577266aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.681+0000 7f15059a7740 1 --2- 172.21.15.69:0/3410181080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a1a20 0x5577266a1e00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.681+0000 7f14fbe00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f14e00a2970 0x7f14e00a4e30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0cc8d0 tx=0x7f14ec005c80 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.681+0000 7f15059a7740 1 -- 172.21.15.69:0/3410181080 >> 172.21.15.69:0/3410181080 conn(0x55772669c150 msgr2=0x55772669c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.681+0000 7f15059a7740 1 -- 172.21.15.69:0/3410181080 shutdown_connections 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.681+0000 7f15059a7740 1 -- 172.21.15.69:0/3410181080 wait complete. 2024-03-19T01:17:09.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.681+0000 7f15059a7740 1 -- start start 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.682+0000 7f15059a7740 1 Processor -- start 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.682+0000 7f15059a7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a1a20 0x5577266a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.682+0000 7f15059a7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a2340 0x5577266a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.682+0000 7f15059a7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266aaf20 0x5577266a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.682+0000 7f15059a7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5577266adcb0 con 0x5577266aaf20 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.682+0000 7f15059a7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577265c2160 con 0x5577266a1a20 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.682+0000 7f15059a7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5577266ade30 con 0x5577266a2340 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.682+0000 7f15059a7740 10 client.0 _ll_register_callbacks cb 0x5577265fe170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.682+0000 7f1500c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a1a20 0x5577266a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.682+0000 7f1500c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a1a20 0x5577266a7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43744/0 (socket says 172.21.15.69:43744) 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.682+0000 7f1500c00640 1 -- 172.21.15.69:0/1006194724 learned_addr learned my addr 172.21.15.69:0/1006194724 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f14fb400640 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a2340 0x5577266a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f1500c00640 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266aaf20 0x5577266a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f1500c00640 1 -- 172.21.15.69:0/1006194724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a2340 msgr2=0x5577266a84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f1500c00640 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a2340 0x5577266a84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f1500c00640 1 -- 172.21.15.69:0/1006194724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266aaf20 msgr2=0x5577266a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f1500c00640 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266aaf20 0x5577266a8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f1500c00640 1 -- 172.21.15.69:0/1006194724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577263e47d0 con 0x5577266a1a20 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f1500c00640 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a1a20 0x5577266a7fa0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009b30 tx=0x7f14f4003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1824f5dde4a7c62 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f14e5600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f14e5600640 1 -- 172.21.15.69:0/1006194724 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4053060 con 0x5577266a1a20 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f14e5600640 1 -- 172.21.15.69:0/1006194724 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f14f4033680 con 0x5577266a1a20 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.683+0000 7f15059a7740 1 -- 172.21.15.69:0/1006194724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14f4033c40 con 0x5577266a1a20 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.684+0000 7f14e5600640 1 -- 172.21.15.69:0/1006194724 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f14f4043040 con 0x5577266a1a20 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.684+0000 7f15059a7740 10 client.27095 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.684+0000 7f15059a7740 1 -- 172.21.15.69:0/1006194724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5577266b9f80 con 0x5577266a1a20 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.684+0000 7f14e5600640 1 -- 172.21.15.69:0/1006194724 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f4033c40 con 0x5577266a1a20 2024-03-19T01:17:09.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.684+0000 7f14e5600640 1 -- 172.21.15.69:0/1006194724 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f14f403f560 con 0x5577266a1a20 2024-03-19T01:17:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.684+0000 7f15059a7740 10 client.27095 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.684+0000 7f15059a7740 10 client.27095 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:17:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.684+0000 7f15059a7740 20 client.27095 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.684+0000 7f15059a7740 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5577266bb2d0 0x5577266bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.684+0000 7f15059a7740 4 client.27095 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.684+0000 7f15059a7740 1 -- 172.21.15.69:0/1006194724 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577266138c0 con 0x5577266bb2d0 2024-03-19T01:17:09.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.685+0000 7f14fb400640 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5577266bb2d0 0x5577266bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.685+0000 7f14fb400640 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5577266bb2d0 0x5577266bd6c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcd42c1527e68d99 server_cookie=5c5fa11923160339 in_seq=0 out_seq=0 2024-03-19T01:17:09.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.685+0000 7f14e5600640 10 client.27095 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:17:09.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.686+0000 7f14e5600640 1 -- 172.21.15.69:0/1006194724 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5577266138c0 con 0x5577266bb2d0 2024-03-19T01:17:09.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.686+0000 7f14e5600640 10 client.27095 handle_command_reply: tid=0 2024-03-19T01:17:09.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 1 client.27095 shutdown 2024-03-19T01:17:09.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 1 -- 172.21.15.69:0/1006194724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a1a20 msgr2=0x5577266a7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a1a20 0x5577266a7fa0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009b30 tx=0x7f14f4003d70 comp rx=0 tx=0).stop 2024-03-19T01:17:09.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 1 -- 172.21.15.69:0/1006194724 shutdown_connections 2024-03-19T01:17:09.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266aaf20 0x5577266a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266a2340 0x5577266a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5577266bb2d0 0x5577266bd6c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 1 --2- 172.21.15.69:0/1006194724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a1a20 0x5577266a7fa0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f14e5600640 0 client.27095 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:17:09.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 1 -- 172.21.15.69:0/1006194724 >> 172.21.15.69:0/1006194724 conn(0x55772669c150 msgr2=0x55772669dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 1 -- 172.21.15.69:0/1006194724 shutdown_connections 2024-03-19T01:17:09.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 1 -- 172.21.15.69:0/1006194724 wait complete. 2024-03-19T01:17:09.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.687+0000 7f15059a7740 20 client.27095 trim_cache size 0 max 16384 2024-03-19T01:17:09.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f15059a7740 1 Processor -- start 2024-03-19T01:17:09.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f15059a7740 1 -- start start 2024-03-19T01:17:09.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f15059a7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a1a20 0x5577266aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f15059a7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a2340 0x55772669b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f15059a7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 0x5577266c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f15059a7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b1130 con 0x5577266a1a20 2024-03-19T01:17:09.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f15059a7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b2df0 con 0x5577266a2340 2024-03-19T01:17:09.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f15059a7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b1800 con 0x5577266aaf20 2024-03-19T01:17:09.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f14fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a2340 0x55772669b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f14fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a2340 0x55772669b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43750/0 (socket says 172.21.15.69:43750) 2024-03-19T01:17:09.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f14fb400640 1 -- 172.21.15.69:0/1167673774 learned_addr learned my addr 172.21.15.69:0/1167673774 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.703+0000 7f1500c00640 1 --2- 172.21.15.69:0/1167673774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a1a20 0x5577266aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.704+0000 7f1500c00640 1 --2- 172.21.15.69:0/1167673774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 0x5577266c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.704+0000 7f14fb400640 1 -- 172.21.15.69:0/1167673774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 msgr2=0x5577266c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.704+0000 7f14fb400640 1 --2- 172.21.15.69:0/1167673774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 0x5577266c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.704+0000 7f14fb400640 1 -- 172.21.15.69:0/1167673774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a1a20 msgr2=0x5577266aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.704+0000 7f14fb400640 1 --2- 172.21.15.69:0/1167673774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a1a20 0x5577266aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.704+0000 7f14fb400640 1 -- 172.21.15.69:0/1167673774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577266a2170 con 0x5577266a2340 2024-03-19T01:17:09.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.704+0000 7f1500c00640 1 --2- 172.21.15.69:0/1167673774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a1a20 0x5577266aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:09.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.704+0000 7f14fb400640 1 --2- 172.21.15.69:0/1167673774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a2340 0x55772669b050 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x5577266a7800 tx=0x7f14e8012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef41b13cc6b33a30 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.705+0000 7f14e7e00640 1 -- 172.21.15.69:0/1167673774 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8023030 con 0x5577266a2340 2024-03-19T01:17:09.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.705+0000 7f14e7e00640 1 -- 172.21.15.69:0/1167673774 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e8005500 con 0x5577266a2340 2024-03-19T01:17:09.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.705+0000 7f14e7e00640 1 -- 172.21.15.69:0/1167673774 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e801f2b0 con 0x5577266a2340 2024-03-19T01:17:09.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.706+0000 7f15059a7740 1 -- 172.21.15.69:0/1167673774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a2340 msgr2=0x55772669b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.706+0000 7f15059a7740 1 --2- 172.21.15.69:0/1167673774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a2340 0x55772669b050 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x5577266a7800 tx=0x7f14e8012b00 comp rx=0 tx=0).stop 2024-03-19T01:17:09.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.706+0000 7f15059a7740 1 -- 172.21.15.69:0/1167673774 shutdown_connections 2024-03-19T01:17:09.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.706+0000 7f15059a7740 1 --2- 172.21.15.69:0/1167673774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 0x5577266c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.706+0000 7f15059a7740 1 --2- 172.21.15.69:0/1167673774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266a2340 0x55772669b050 secure :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0x5577266a7800 tx=0x7f14e8012b00 comp rx=0 tx=0).stop 2024-03-19T01:17:09.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.706+0000 7f15059a7740 1 --2- 172.21.15.69:0/1167673774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a1a20 0x5577266aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.706+0000 7f15059a7740 1 -- 172.21.15.69:0/1167673774 >> 172.21.15.69:0/1167673774 conn(0x55772669c150 msgr2=0x55772669a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.707+0000 7f15059a7740 1 -- 172.21.15.69:0/1167673774 shutdown_connections 2024-03-19T01:17:09.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.707+0000 7f15059a7740 1 -- 172.21.15.69:0/1167673774 wait complete. 2024-03-19T01:17:09.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.708+0000 7f15059a7740 1 -- start start 2024-03-19T01:17:09.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.708+0000 7f15059a7740 1 Processor -- start 2024-03-19T01:17:09.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.708+0000 7f15059a7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a1a20 0x5577266c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.708+0000 7f15059a7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 0x5577266c0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.708+0000 7f15059a7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266be6f0 0x5577266beb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.708+0000 7f15059a7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b2660 con 0x5577266a1a20 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.708+0000 7f15059a7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1b1f60 con 0x5577266be6f0 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.708+0000 7f15059a7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1af0a0 con 0x5577266aaf20 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.708+0000 7f15059a7740 10 client.0 _ll_register_callbacks cb 0x7f14e8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f14fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 0x5577266c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f1500c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a1a20 0x5577266c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f14fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 0x5577266c0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:47030/0 (socket says 172.21.15.69:47030) 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f14fb400640 1 -- 172.21.15.69:0/4198640733 learned_addr learned my addr 172.21.15.69:0/4198640733 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f1500c00640 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266be6f0 0x5577266beb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f14fb400640 1 -- 172.21.15.69:0/4198640733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266be6f0 msgr2=0x5577266beb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f14fb400640 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266be6f0 0x5577266beb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f14fb400640 1 -- 172.21.15.69:0/4198640733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a1a20 msgr2=0x5577266c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f14fb400640 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a1a20 0x5577266c02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f14fb400640 1 -- 172.21.15.69:0/4198640733 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577266bf260 con 0x5577266aaf20 2024-03-19T01:17:09.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f14fb400640 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 0x5577266c0830 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f14e80097b0 tx=0x7f14e8012050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e374bbf47bfa6f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.709+0000 7f14e5600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:09.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.710+0000 7f14e5600640 1 -- 172.21.15.69:0/4198640733 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8023040 con 0x5577266aaf20 2024-03-19T01:17:09.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.710+0000 7f14e5600640 1 -- 172.21.15.69:0/4198640733 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f14e80052a0 con 0x5577266aaf20 2024-03-19T01:17:09.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.710+0000 7f15059a7740 1 -- 172.21.15.69:0/4198640733 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e8005500 con 0x5577266aaf20 2024-03-19T01:17:09.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.710+0000 7f14e5600640 1 -- 172.21.15.69:0/4198640733 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f14e8004700 con 0x5577266aaf20 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.710+0000 7f15059a7740 10 client.27052 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.710+0000 7f15059a7740 1 -- 172.21.15.69:0/4198640733 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5577267220c0 con 0x5577266aaf20 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.710+0000 7f14e5600640 1 -- 172.21.15.69:0/4198640733 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e8005500 con 0x5577266aaf20 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.711+0000 7f14e5600640 1 -- 172.21.15.69:0/4198640733 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f14e803a470 con 0x5577266aaf20 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.711+0000 7f15059a7740 10 client.27052 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.711+0000 7f15059a7740 10 client.27052 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.711+0000 7f15059a7740 20 client.27052 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.711+0000 7f15059a7740 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x557726722680 0x557726724aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.711+0000 7f15059a7740 4 client.27052 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.711+0000 7f15059a7740 1 -- 172.21.15.69:0/4198640733 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55772660dc10 con 0x557726722680 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.711+0000 7f1500c00640 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x557726722680 0x557726724aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.711+0000 7f1500c00640 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x557726722680 0x557726724aa0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f0d32b11ac291d9 server_cookie=4dc2d097ab0f28a7 in_seq=0 out_seq=0 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f14e5600640 10 client.27052 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:17:09.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f14e5600640 1 -- 172.21.15.69:0/4198640733 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55772660dc10 con 0x557726722680 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f14e5600640 10 client.27052 handle_command_reply: tid=0 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 1 client.27052 shutdown 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 1 -- 172.21.15.69:0/4198640733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 msgr2=0x5577266c0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 0x5577266c0830 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f14e80097b0 tx=0x7f14e8012050 comp rx=0 tx=0).stop 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 1 -- 172.21.15.69:0/4198640733 shutdown_connections 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x557726722680 0x557726724aa0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577266be6f0 0x5577266beb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5577266aaf20 0x5577266c0830 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 1 --2- 172.21.15.69:0/4198640733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577266a1a20 0x5577266c02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f14e5600640 0 client.27052 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 1 -- 172.21.15.69:0/4198640733 >> 172.21.15.69:0/4198640733 conn(0x55772669c150 msgr2=0x5577266b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 1 -- 172.21.15.69:0/4198640733 shutdown_connections 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 1 -- 172.21.15.69:0/4198640733 wait complete. 2024-03-19T01:17:09.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.712+0000 7f15059a7740 20 client.27052 trim_cache size 0 max 16384 2024-03-19T01:17:09.716 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:17:09.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 -- 172.21.15.69:0/750256218 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f14e00a2970 msgr2=0x7f14e00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f14e00a2970 0x7f14e00a4e30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0cc8d0 tx=0x7f14ec005c80 comp rx=0 tx=0).stop 2024-03-19T01:17:09.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 -- 172.21.15.69:0/750256218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc1017b0 msgr2=0x7f14fc19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc1017b0 0x7f14fc19a280 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00e9c0 tx=0x7f14ec00ee80 comp rx=0 tx=0).stop 2024-03-19T01:17:09.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 -- 172.21.15.69:0/750256218 shutdown_connections 2024-03-19T01:17:09.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14fc1077c0 0x7f14fc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14fc102170 0x7f14fc19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f14e00a2970 0x7f14e00a4e30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 --2- 172.21.15.69:0/750256218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc1017b0 0x7f14fc19a280 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 -- 172.21.15.69:0/750256218 >> 172.21.15.69:0/750256218 conn(0x7f14fc0fd260 msgr2=0x7f14fc10e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 -- 172.21.15.69:0/750256218 shutdown_connections 2024-03-19T01:17:09.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.713+0000 7f1502c00640 1 -- 172.21.15.69:0/750256218 wait complete. 2024-03-19T01:17:09.730 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:17:09.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.927+0000 7f07d5a00640 1 -- 172.21.15.69:0/1440315674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07d0102a60 msgr2=0x7f07d0102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.927+0000 7f07d5a00640 1 --2- 172.21.15.69:0/1440315674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07d0102a60 0x7f07d0102e40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009900 tx=0x7f07c002bcd0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.927+0000 7f07d5a00640 1 -- 172.21.15.69:0/1440315674 shutdown_connections 2024-03-19T01:17:09.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.927+0000 7f07d5a00640 1 --2- 172.21.15.69:0/1440315674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d010be30 0x7f07d010e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.927+0000 7f07d5a00640 1 --2- 172.21.15.69:0/1440315674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07d0103380 0x7f07d010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.927+0000 7f07d5a00640 1 --2- 172.21.15.69:0/1440315674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07d0102a60 0x7f07d0102e40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.927+0000 7f07d5a00640 1 -- 172.21.15.69:0/1440315674 >> 172.21.15.69:0/1440315674 conn(0x7f07d0076aa0 msgr2=0x7f07d0076eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.927+0000 7f07d5a00640 1 -- 172.21.15.69:0/1440315674 shutdown_connections 2024-03-19T01:17:09.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.928+0000 7f07d5a00640 1 -- 172.21.15.69:0/1440315674 wait complete. 2024-03-19T01:17:09.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.928+0000 7f07d5a00640 1 Processor -- start 2024-03-19T01:17:09.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07d5a00640 1 -- start start 2024-03-19T01:17:09.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07d5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d0102a60 0x7f07d019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07d5a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07d0103380 0x7f07d019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07d5a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07d010be30 0x7f07d019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07d5a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07d0111c10 con 0x7f07d0103380 2024-03-19T01:17:09.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07d5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07d0111a90 con 0x7f07d0102a60 2024-03-19T01:17:09.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07d5a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f07d0111d90 con 0x7f07d010be30 2024-03-19T01:17:09.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d0102a60 0x7f07d019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d0102a60 0x7f07d019a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43772/0 (socket says 172.21.15.69:43772) 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07cea00640 1 -- 172.21.15.69:0/2919017023 learned_addr learned my addr 172.21.15.69:0/2919017023 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07cf400640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07d010be30 0x7f07d019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.929+0000 7f07c7e00640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07d0103380 0x7f07d019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.930+0000 7f07cea00640 1 -- 172.21.15.69:0/2919017023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07d010be30 msgr2=0x7f07d019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.930+0000 7f07cea00640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07d010be30 0x7f07d019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.930+0000 7f07cea00640 1 -- 172.21.15.69:0/2919017023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07d0103380 msgr2=0x7f07d019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.930+0000 7f07cea00640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07d0103380 0x7f07d019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.930+0000 7f07cea00640 1 -- 172.21.15.69:0/2919017023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d019f300 con 0x7f07d0102a60 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.930+0000 7f07cea00640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d0102a60 0x7f07d019a310 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f07c0000c00 tx=0x7f07c0004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.930+0000 7f07c7400640 1 -- 172.21.15.69:0/2919017023 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0042020 con 0x7f07d0102a60 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.930+0000 7f07d5a00640 1 -- 172.21.15.69:0/2919017023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d019f590 con 0x7f07d0102a60 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.930+0000 7f07c7400640 1 -- 172.21.15.69:0/2919017023 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c00054e0 con 0x7f07d0102a60 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.930+0000 7f07c7400640 1 -- 172.21.15.69:0/2919017023 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0003a80 con 0x7f07d0102a60 2024-03-19T01:17:09.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.930+0000 7f07d5a00640 1 -- 172.21.15.69:0/2919017023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d01a6e80 con 0x7f07d0102a60 2024-03-19T01:17:09.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.933+0000 7f07d8829740 1 Processor -- start 2024-03-19T01:17:09.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.933+0000 7f07d8829740 1 -- start start 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.934+0000 7f07d8829740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 0x561d4320fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.934+0000 7f07d8829740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d432189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.934+0000 7f07d8829740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 0x561d4321b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.934+0000 7f07d8829740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561d43174380 con 0x561d4320fa20 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.934+0000 7f07d8829740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561d4311cd20 con 0x561d43218f20 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.934+0000 7f07d8829740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561d43170420 con 0x561d43210340 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.934+0000 7f07cf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 0x561d4320fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.934+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 0x561d4321b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.934+0000 7f07cf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 0x561d4320fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51874/0 (socket says 172.21.15.69:51874) 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.934+0000 7f07c7e00640 1 --2- 172.21.15.69:0/2607542935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 0x561d4321b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43776/0 (socket says 172.21.15.69:43776) 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.934+0000 7f07cf400640 1 -- 172.21.15.69:0/2607542935 learned_addr learned my addr 172.21.15.69:0/2607542935 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.935+0000 7f07c7e00640 1 -- 172.21.15.69:0/2607542935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 msgr2=0x561d432189e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:17:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.935+0000 7f07c7e00640 1 --2- 172.21.15.69:0/2607542935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d432189e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.935+0000 7f07c7e00640 1 -- 172.21.15.69:0/2607542935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 msgr2=0x561d4320fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.935+0000 7f07c7e00640 1 --2- 172.21.15.69:0/2607542935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 0x561d4320fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.935+0000 7f07c7e00640 1 -- 172.21.15.69:0/2607542935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d42e883f0 con 0x561d43218f20 2024-03-19T01:17:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.935+0000 7f07c7e00640 1 --2- 172.21.15.69:0/2607542935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 0x561d4321b310 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009b40 tx=0x7f07b80113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bc1036cca8d2939 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.935+0000 7f07bfe00640 1 -- 172.21.15.69:0/2607542935 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8011e10 con 0x561d43218f20 2024-03-19T01:17:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.935+0000 7f07bfe00640 1 -- 172.21.15.69:0/2607542935 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b8012930 con 0x561d43218f20 2024-03-19T01:17:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.935+0000 7f07bfe00640 1 -- 172.21.15.69:0/2607542935 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b800fb30 con 0x561d43218f20 2024-03-19T01:17:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.935+0000 7f07c7400640 1 -- 172.21.15.69:0/2919017023 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f07c0045020 con 0x7f07d0102a60 2024-03-19T01:17:09.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.936+0000 7f07d8829740 1 -- 172.21.15.69:0/2607542935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 msgr2=0x561d4321b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.936+0000 7f07d8829740 1 --2- 172.21.15.69:0/2607542935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 0x561d4321b310 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009b40 tx=0x7f07b80113e0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.936+0000 7f07d8829740 1 -- 172.21.15.69:0/2607542935 shutdown_connections 2024-03-19T01:17:09.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.936+0000 7f07d8829740 1 --2- 172.21.15.69:0/2607542935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 0x561d4321b310 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.936+0000 7f07d8829740 1 --2- 172.21.15.69:0/2607542935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d432189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.936+0000 7f07d8829740 1 --2- 172.21.15.69:0/2607542935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 0x561d4320fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.936+0000 7f07d8829740 1 -- 172.21.15.69:0/2607542935 >> 172.21.15.69:0/2607542935 conn(0x561d4320a150 msgr2=0x561d4320a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.936+0000 7f07d8829740 1 -- 172.21.15.69:0/2607542935 shutdown_connections 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.936+0000 7f07d8829740 1 -- 172.21.15.69:0/2607542935 wait complete. 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.937+0000 7f07d8829740 1 -- start start 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.937+0000 7f07d8829740 1 Processor -- start 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.937+0000 7f07c7400640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07b40a2880 0x7f07b40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.937+0000 7f07d8829740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 0x561d43215f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.937+0000 7f07d8829740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d432164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.937+0000 7f07d8829740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 0x561d432169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.937+0000 7f07c7400640 1 -- 172.21.15.69:0/2919017023 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f07c00f0870 con 0x7f07d0102a60 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.937+0000 7f07d8829740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561d4321bd10 con 0x561d4320fa20 2024-03-19T01:17:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.937+0000 7f07d8829740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561d43130160 con 0x561d43218f20 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.937+0000 7f07d8829740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561d4321be90 con 0x561d43210340 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07d8829740 10 client.0 _ll_register_callbacks cb 0x561d4316c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07cf400640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07b40a2880 0x7f07b40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d432164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d432164b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:47042/0 (socket says 172.21.15.69:47042) 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07c7e00640 1 -- 172.21.15.69:0/2261579289 learned_addr learned my addr 172.21.15.69:0/2261579289 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07cea00640 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 0x561d43215f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07cf400640 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 0x561d432169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07c7e00640 1 -- 172.21.15.69:0/2261579289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 msgr2=0x561d432169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07c7e00640 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 0x561d432169f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07c7e00640 1 -- 172.21.15.69:0/2261579289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 msgr2=0x561d43215f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07c7e00640 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 0x561d43215f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07c7e00640 1 -- 172.21.15.69:0/2261579289 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d42f527d0 con 0x561d43210340 2024-03-19T01:17:09.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07cf400640 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 0x561d432169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:09.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.938+0000 7f07cea00640 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 0x561d43215f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:09.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.939+0000 7f07c7e00640 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d432164b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f07b8012700 tx=0x7f07b8012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed352be3c6d7dfb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.939+0000 7f07cf400640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07b40a2880 0x7f07b40a4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009fd0 tx=0x7f07c8009c40 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.939+0000 7f07bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:09.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.939+0000 7f07bcc00640 1 -- 172.21.15.69:0/2261579289 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8028020 con 0x561d43210340 2024-03-19T01:17:09.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.939+0000 7f07bcc00640 1 -- 172.21.15.69:0/2261579289 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f07b800f910 con 0x561d43210340 2024-03-19T01:17:09.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.939+0000 7f07d8829740 1 -- 172.21.15.69:0/2261579289 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8012930 con 0x561d43210340 2024-03-19T01:17:09.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.939+0000 7f07bcc00640 1 -- 172.21.15.69:0/2261579289 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f07b8033050 con 0x561d43210340 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.939+0000 7f07d8829740 10 client.27061 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.939+0000 7f07d8829740 1 -- 172.21.15.69:0/2261579289 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561d43227fd0 con 0x561d43210340 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.940+0000 7f07bcc00640 1 -- 172.21.15.69:0/2261579289 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b8012930 con 0x561d43210340 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.940+0000 7f07bcc00640 1 -- 172.21.15.69:0/2261579289 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f07b80035f0 con 0x561d43210340 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.940+0000 7f07d8829740 10 client.27061 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.940+0000 7f07d8829740 10 client.27061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.940+0000 7f07d8829740 20 client.27061 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.940+0000 7f07d8829740 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561d43229430 0x561d4322b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.940+0000 7f07d8829740 4 client.27061 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.940+0000 7f07d8829740 1 -- 172.21.15.69:0/2261579289 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d431818c0 con 0x561d43229430 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.941+0000 7f07cea00640 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561d43229430 0x561d4322b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.941+0000 7f07cea00640 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561d43229430 0x561d4322b820 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd74cbe9882fef5 server_cookie=9a065f32c8910747 in_seq=0 out_seq=0 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.941+0000 7f07bcc00640 10 client.27061 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.942+0000 7f07bcc00640 1 -- 172.21.15.69:0/2261579289 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561d431818c0 con 0x561d43229430 2024-03-19T01:17:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.942+0000 7f07bcc00640 10 client.27061 handle_command_reply: tid=0 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.942+0000 7f07d8829740 1 client.27061 shutdown 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.942+0000 7f07d8829740 1 -- 172.21.15.69:0/2261579289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 msgr2=0x561d432164b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.942+0000 7f07d8829740 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d432164b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f07b8012700 tx=0x7f07b8012730 comp rx=0 tx=0).stop 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.943+0000 7f07d8829740 1 -- 172.21.15.69:0/2261579289 shutdown_connections 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.943+0000 7f07d8829740 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561d43229430 0x561d4322b820 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.943+0000 7f07d8829740 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43218f20 0x561d432169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.943+0000 7f07d8829740 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d432164b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.943+0000 7f07d8829740 1 --2- 172.21.15.69:0/2261579289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d4320fa20 0x561d43215f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.943+0000 7f07bcc00640 0 client.27061 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.943+0000 7f07d8829740 1 -- 172.21.15.69:0/2261579289 >> 172.21.15.69:0/2261579289 conn(0x561d4320a150 msgr2=0x561d4320bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.943+0000 7f07d8829740 1 -- 172.21.15.69:0/2261579289 shutdown_connections 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.943+0000 7f07d8829740 1 -- 172.21.15.69:0/2261579289 wait complete. 2024-03-19T01:17:09.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.943+0000 7f07d8829740 20 client.27061 trim_cache size 0 max 16384 2024-03-19T01:17:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.971+0000 7f07d8829740 1 Processor -- start 2024-03-19T01:17:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.972+0000 7f07d8829740 1 -- start start 2024-03-19T01:17:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.972+0000 7f07d8829740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4320fa20 0x561d432186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.972+0000 7f07d8829740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d43224ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.972+0000 7f07d8829740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d43218f20 0x561d43230f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.972+0000 7f07d8829740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07d01b11b0 con 0x561d43218f20 2024-03-19T01:17:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.972+0000 7f07d8829740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07d01b2e70 con 0x561d4320fa20 2024-03-19T01:17:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.972+0000 7f07d8829740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f07d01b1880 con 0x561d43210340 2024-03-19T01:17:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.973+0000 7f07cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4320fa20 0x561d432186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.973+0000 7f07cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4320fa20 0x561d432186a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43804/0 (socket says 172.21.15.69:43804) 2024-03-19T01:17:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.973+0000 7f07cea00640 1 -- 172.21.15.69:0/1983358769 learned_addr learned my addr 172.21.15.69:0/1983358769 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.973+0000 7f07cea00640 1 -- 172.21.15.69:0/1983358769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 msgr2=0x561d43224ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.973+0000 7f07cea00640 1 --2- 172.21.15.69:0/1983358769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d43224ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.974+0000 7f07cea00640 1 -- 172.21.15.69:0/1983358769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d43218f20 msgr2=0x561d43230f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.974+0000 7f07cea00640 1 --2- 172.21.15.69:0/1983358769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d43218f20 0x561d43230f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.974+0000 7f07cea00640 1 -- 172.21.15.69:0/1983358769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d43209c80 con 0x561d4320fa20 2024-03-19T01:17:09.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.975+0000 7f07cea00640 1 --2- 172.21.15.69:0/1983358769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4320fa20 0x561d432186a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f07c00bc810 tx=0x7f07c002bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2ae0be0d666a893 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.976+0000 7f07bf400640 1 -- 172.21.15.69:0/1983358769 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00da070 con 0x561d4320fa20 2024-03-19T01:17:09.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.976+0000 7f07bf400640 1 -- 172.21.15.69:0/1983358769 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c00bcb10 con 0x561d4320fa20 2024-03-19T01:17:09.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.976+0000 7f07bf400640 1 -- 172.21.15.69:0/1983358769 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00b9310 con 0x561d4320fa20 2024-03-19T01:17:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.981+0000 7f07d8829740 1 -- 172.21.15.69:0/1983358769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4320fa20 msgr2=0x561d432186a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.981+0000 7f07d8829740 1 --2- 172.21.15.69:0/1983358769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4320fa20 0x561d432186a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f07c00bc810 tx=0x7f07c002bcd0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.981+0000 7f07d8829740 1 -- 172.21.15.69:0/1983358769 shutdown_connections 2024-03-19T01:17:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.981+0000 7f07d8829740 1 --2- 172.21.15.69:0/1983358769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d43218f20 0x561d43230f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.981+0000 7f07d8829740 1 --2- 172.21.15.69:0/1983358769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d43210340 0x561d43224ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.981+0000 7f07d8829740 1 --2- 172.21.15.69:0/1983358769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d4320fa20 0x561d432186a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.981+0000 7f07d8829740 1 -- 172.21.15.69:0/1983358769 >> 172.21.15.69:0/1983358769 conn(0x561d4320a150 msgr2=0x561d432112a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:09.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.984+0000 7f07d8829740 1 -- 172.21.15.69:0/1983358769 shutdown_connections 2024-03-19T01:17:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.984+0000 7f07d8829740 1 -- 172.21.15.69:0/1983358769 wait complete. 2024-03-19T01:17:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.984+0000 7f07d8829740 1 -- start start 2024-03-19T01:17:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.985+0000 7f07d8829740 1 Processor -- start 2024-03-19T01:17:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.986+0000 7f07d8829740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d4320fa20 0x561d43212a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.987+0000 7f07d8829740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43210340 0x561d43212f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.987+0000 7f07d8829740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d43218f20 0x561d43213480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.988+0000 7f07cf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d43218f20 0x561d43213480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.988+0000 7f07cf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d43218f20 0x561d43213480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51902/0 (socket says 172.21.15.69:51902) 2024-03-19T01:17:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.988+0000 7f07cf400640 1 -- 172.21.15.69:0/2887312485 learned_addr learned my addr 172.21.15.69:0/2887312485 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.988+0000 7f07cea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d4320fa20 0x561d43212a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.988+0000 7f07d8829740 1 -- 172.21.15.69:0/2887312485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07d01b26e0 con 0x561d43218f20 2024-03-19T01:17:09.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.991+0000 7f07c7e00640 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43210340 0x561d43212f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:09.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.991+0000 7f07d8829740 1 -- 172.21.15.69:0/2887312485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07d01b1fe0 con 0x561d43210340 2024-03-19T01:17:09.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.991+0000 7f07d8829740 1 -- 172.21.15.69:0/2887312485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f07d01af120 con 0x561d4320fa20 2024-03-19T01:17:09.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.991+0000 7f07d8829740 10 client.0 _ll_register_callbacks cb 0x7f07c8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:09.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.991+0000 7f07cea00640 1 -- 172.21.15.69:0/2887312485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43210340 msgr2=0x561d43212f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.991+0000 7f07cea00640 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43210340 0x561d43212f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.991+0000 7f07cea00640 1 -- 172.21.15.69:0/2887312485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d43218f20 msgr2=0x561d43213480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:09.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.991+0000 7f07cea00640 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d43218f20 0x561d43213480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:09.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.991+0000 7f07cea00640 1 -- 172.21.15.69:0/2887312485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d4321f0d0 con 0x561d4320fa20 2024-03-19T01:17:09.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.993+0000 7f07cea00640 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d4320fa20 0x561d43212a00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f07c00bcaa0 tx=0x7f07c0009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da1c39c8fdbdff51 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:09.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.995+0000 7f07bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:09.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.995+0000 7f07bcc00640 1 -- 172.21.15.69:0/2887312485 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00da070 con 0x561d4320fa20 2024-03-19T01:17:09.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.995+0000 7f07d8829740 1 -- 172.21.15.69:0/2887312485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07c00bcb10 con 0x561d4320fa20 2024-03-19T01:17:09.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.996+0000 7f07bcc00640 1 -- 172.21.15.69:0/2887312485 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f07c00edd70 con 0x561d4320fa20 2024-03-19T01:17:09.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.996+0000 7f07bcc00640 1 -- 172.21.15.69:0/2887312485 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f07c00c0c60 con 0x561d4320fa20 2024-03-19T01:17:09.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.996+0000 7f07d8829740 10 client.27064 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:09.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.996+0000 7f07d8829740 1 -- 172.21.15.69:0/2887312485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561d432a06f0 con 0x561d4320fa20 2024-03-19T01:17:09.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.997+0000 7f07bcc00640 1 -- 172.21.15.69:0/2887312485 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c00bcb10 con 0x561d4320fa20 2024-03-19T01:17:09.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.997+0000 7f07bcc00640 1 -- 172.21.15.69:0/2887312485 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f07c00c4eb0 con 0x561d4320fa20 2024-03-19T01:17:09.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.997+0000 7f07d8829740 10 client.27064 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:09.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.997+0000 7f07d8829740 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:09.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.997+0000 7f07d8829740 20 client.27064 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:09.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.997+0000 7f07d8829740 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561d432a0de0 0x561d432a3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:09.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.998+0000 7f07d8829740 4 client.27064 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:17:09.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:09.998+0000 7f07d8829740 1 -- 172.21.15.69:0/2887312485 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d4317bc10 con 0x561d432a0de0 2024-03-19T01:17:10.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.002+0000 7f07c7e00640 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561d432a0de0 0x561d432a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.008+0000 7f07c7e00640 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561d432a0de0 0x561d432a3200 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15d3d0e1b3526e1 server_cookie=bb69fb5568458a2 in_seq=0 out_seq=0 2024-03-19T01:17:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.008+0000 7f07bcc00640 10 client.27064 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:10.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.024+0000 7f07bcc00640 1 -- 172.21.15.69:0/2887312485 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d4317bc10 con 0x561d432a0de0 2024-03-19T01:17:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.024+0000 7f07bcc00640 10 client.27064 handle_command_reply: tid=0 2024-03-19T01:17:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.024+0000 7f07d8829740 1 client.27064 shutdown 2024-03-19T01:17:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.024+0000 7f07d8829740 1 -- 172.21.15.69:0/2887312485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d4320fa20 msgr2=0x561d43212a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.024+0000 7f07d8829740 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d4320fa20 0x561d43212a00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f07c00bcaa0 tx=0x7f07c0009680 comp rx=0 tx=0).stop 2024-03-19T01:17:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.025+0000 7f07d8829740 1 -- 172.21.15.69:0/2887312485 shutdown_connections 2024-03-19T01:17:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.025+0000 7f07d8829740 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561d432a0de0 0x561d432a3200 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.025+0000 7f07d8829740 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561d43218f20 0x561d43213480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.025+0000 7f07bcc00640 0 client.27064 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.025+0000 7f07d8829740 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561d43210340 0x561d43212f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.025+0000 7f07d8829740 1 --2- 172.21.15.69:0/2887312485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561d4320fa20 0x561d43212a00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.025+0000 7f07d8829740 1 -- 172.21.15.69:0/2887312485 >> 172.21.15.69:0/2887312485 conn(0x561d4320a150 msgr2=0x561d43215f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:10.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.025+0000 7f07d8829740 1 -- 172.21.15.69:0/2887312485 shutdown_connections 2024-03-19T01:17:10.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.026+0000 7f07d8829740 1 -- 172.21.15.69:0/2887312485 wait complete. 2024-03-19T01:17:10.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.026+0000 7f07d8829740 20 client.27064 trim_cache size 0 max 16384 2024-03-19T01:17:10.030 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:17:10.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.029+0000 7f07d5a00640 1 -- 172.21.15.69:0/2919017023 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07b40a2880 msgr2=0x7f07b40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:10.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.029+0000 7f07d5a00640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07b40a2880 0x7f07b40a4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009fd0 tx=0x7f07c8009c40 comp rx=0 tx=0).stop 2024-03-19T01:17:10.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.030+0000 7f07d5a00640 1 -- 172.21.15.69:0/2919017023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d0102a60 msgr2=0x7f07d019a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.030+0000 7f07d5a00640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d0102a60 0x7f07d019a310 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f07c0000c00 tx=0x7f07c0004360 comp rx=0 tx=0).stop 2024-03-19T01:17:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.030+0000 7f07d5a00640 1 -- 172.21.15.69:0/2919017023 shutdown_connections 2024-03-19T01:17:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.030+0000 7f07d5a00640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07d010be30 0x7f07d019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.030+0000 7f07d5a00640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07d0103380 0x7f07d019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.030+0000 7f07d5a00640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f07b40a2880 0x7f07b40a4d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.030+0000 7f07d5a00640 1 --2- 172.21.15.69:0/2919017023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07d0102a60 0x7f07d019a310 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.030+0000 7f07d5a00640 1 -- 172.21.15.69:0/2919017023 >> 172.21.15.69:0/2919017023 conn(0x7f07d0076aa0 msgr2=0x7f07d00ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:10.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.030+0000 7f07d5a00640 1 -- 172.21.15.69:0/2919017023 shutdown_connections 2024-03-19T01:17:10.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:10.031+0000 7f07d5a00640 1 -- 172.21.15.69:0/2919017023 wait complete. 2024-03-19T01:17:11.063 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:17:11.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:17:11.064 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:17:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.278+0000 7f7b89c00640 1 -- 172.21.15.69:0/2434915133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b84101680 msgr2=0x7f7b84101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.278+0000 7f7b89c00640 1 --2- 172.21.15.69:0/2434915133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b84101680 0x7f7b84101b00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009900 tx=0x7f7b7002bc30 comp rx=0 tx=0).stop 2024-03-19T01:17:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.278+0000 7f7b89c00640 1 -- 172.21.15.69:0/2434915133 shutdown_connections 2024-03-19T01:17:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.278+0000 7f7b89c00640 1 --2- 172.21.15.69:0/2434915133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b84102040 0x7f7b8410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.278+0000 7f7b89c00640 1 --2- 172.21.15.69:0/2434915133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b84101680 0x7f7b84101b00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.278+0000 7f7b89c00640 1 --2- 172.21.15.69:0/2434915133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b84107690 0x7f7b84107a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.278+0000 7f7b89c00640 1 -- 172.21.15.69:0/2434915133 >> 172.21.15.69:0/2434915133 conn(0x7f7b840fd190 msgr2=0x7f7b840ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.278+0000 7f7b89c00640 1 -- 172.21.15.69:0/2434915133 shutdown_connections 2024-03-19T01:17:11.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.278+0000 7f7b89c00640 1 -- 172.21.15.69:0/2434915133 wait complete. 2024-03-19T01:17:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.279+0000 7f7b89c00640 1 Processor -- start 2024-03-19T01:17:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.279+0000 7f7b89c00640 1 -- start start 2024-03-19T01:17:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.279+0000 7f7b89c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b84101680 0x7f7b84197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.279+0000 7f7b89c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b84102040 0x7f7b841984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.279+0000 7f7b89c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b84107690 0x7f7b8419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.279+0000 7f7b89c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84111af0 con 0x7f7b84102040 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.279+0000 7f7b89c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84111970 con 0x7f7b84107690 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.279+0000 7f7b89c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84111c70 con 0x7f7b84101680 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.280+0000 7f7b83e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b84101680 0x7f7b84197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.280+0000 7f7b83400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b84102040 0x7f7b841984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.280+0000 7f7b83e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b84101680 0x7f7b84197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:47082/0 (socket says 172.21.15.69:47082) 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.280+0000 7f7b83e00640 1 -- 172.21.15.69:0/2216147215 learned_addr learned my addr 172.21.15.69:0/2216147215 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.280+0000 7f7b89200640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b84107690 0x7f7b8419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.280+0000 7f7b83e00640 1 -- 172.21.15.69:0/2216147215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b84107690 msgr2=0x7f7b8419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.280+0000 7f7b83e00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b84107690 0x7f7b8419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.280+0000 7f7b83e00640 1 -- 172.21.15.69:0/2216147215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b84102040 msgr2=0x7f7b841984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.280+0000 7f7b83e00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b84102040 0x7f7b841984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.280+0000 7f7b83e00640 1 -- 172.21.15.69:0/2216147215 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8419cf30 con 0x7f7b84101680 2024-03-19T01:17:11.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.280+0000 7f7b83e00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b84101680 0x7f7b84197f90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400ebc0 tx=0x7f7b7400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.281+0000 7f7b80c00640 1 -- 172.21.15.69:0/2216147215 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74010630 con 0x7f7b84101680 2024-03-19T01:17:11.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.281+0000 7f7b89c00640 1 -- 172.21.15.69:0/2216147215 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8419d220 con 0x7f7b84101680 2024-03-19T01:17:11.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.281+0000 7f7b80c00640 1 -- 172.21.15.69:0/2216147215 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b74010cb0 con 0x7f7b84101680 2024-03-19T01:17:11.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.281+0000 7f7b80c00640 1 -- 172.21.15.69:0/2216147215 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74018bd0 con 0x7f7b84101680 2024-03-19T01:17:11.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.281+0000 7f7b89c00640 1 -- 172.21.15.69:0/2216147215 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b841a4ab0 con 0x7f7b84101680 2024-03-19T01:17:11.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b8de3c740 1 Processor -- start 2024-03-19T01:17:11.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b8de3c740 1 -- start start 2024-03-19T01:17:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 0x5581650b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b0a10 0x5581650b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b95f0 0x5581650bb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55816500efb0 con 0x5581650b0a10 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558164fb7ab0 con 0x5581650b00f0 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55816500b050 con 0x5581650b95f0 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b80c00640 1 -- 172.21.15.69:0/2216147215 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f7b740107d0 con 0x7f7b84101680 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b89200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 0x5581650b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b89200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 0x5581650b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43844/0 (socket says 172.21.15.69:43844) 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b89200640 1 -- 172.21.15.69:0/1304871981 learned_addr learned my addr 172.21.15.69:0/1304871981 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.284+0000 7f7b83e00640 1 --2- 172.21.15.69:0/1304871981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b0a10 0x5581650b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.285+0000 7f7b83400640 1 --2- 172.21.15.69:0/1304871981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b95f0 0x5581650bb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.285+0000 7f7b89200640 1 -- 172.21.15.69:0/1304871981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b95f0 msgr2=0x5581650bb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.285+0000 7f7b89200640 1 --2- 172.21.15.69:0/1304871981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b95f0 0x5581650bb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.285+0000 7f7b89200640 1 -- 172.21.15.69:0/1304871981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b0a10 msgr2=0x5581650b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.285+0000 7f7b89200640 1 --2- 172.21.15.69:0/1304871981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b0a10 0x5581650b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.285+0000 7f7b89200640 1 -- 172.21.15.69:0/1304871981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558164fd5e00 con 0x5581650b00f0 2024-03-19T01:17:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.285+0000 7f7b83400640 1 --2- 172.21.15.69:0/1304871981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b95f0 0x5581650bb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:11.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.285+0000 7f7b89200640 1 --2- 172.21.15.69:0/1304871981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 0x5581650b04d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009ca0 tx=0x7f7b7c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=153f939a2ce9119c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.285+0000 7f7b79600640 1 -- 172.21.15.69:0/1304871981 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c01e070 con 0x5581650b00f0 2024-03-19T01:17:11.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b79600640 1 -- 172.21.15.69:0/1304871981 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7c009e50 con 0x5581650b00f0 2024-03-19T01:17:11.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b79600640 1 -- 172.21.15.69:0/1304871981 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c00eac0 con 0x5581650b00f0 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b80c00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7b680a29f0 0x7f7b680a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b80c00640 1 -- 172.21.15.69:0/2216147215 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f7b740cf8d0 con 0x7f7b84101680 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b83e00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7b680a29f0 0x7f7b680a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b8de3c740 1 -- 172.21.15.69:0/1304871981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 msgr2=0x5581650b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/1304871981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 0x5581650b04d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009ca0 tx=0x7f7b7c011810 comp rx=0 tx=0).stop 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b8de3c740 1 -- 172.21.15.69:0/1304871981 shutdown_connections 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/1304871981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b95f0 0x5581650bb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/1304871981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b0a10 0x5581650b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/1304871981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 0x5581650b04d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.286+0000 7f7b8de3c740 1 -- 172.21.15.69:0/1304871981 >> 172.21.15.69:0/1304871981 conn(0x5581650aa820 msgr2=0x5581650aac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.287+0000 7f7b8de3c740 1 -- 172.21.15.69:0/1304871981 shutdown_connections 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.287+0000 7f7b83e00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7b680a29f0 0x7f7b680a4eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5581650afbc0 tx=0x7f7b7400cf90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.287+0000 7f7b8de3c740 1 -- 172.21.15.69:0/1304871981 wait complete. 2024-03-19T01:17:11.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.287+0000 7f7b8de3c740 1 -- start start 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.287+0000 7f7b8de3c740 1 Processor -- start 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.288+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 0x5581650b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.288+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b0a10 0x5581650b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.288+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b95f0 0x5581650b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.288+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5581650bc3e0 con 0x5581650b00f0 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.288+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558164fcaf00 con 0x5581650b95f0 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.288+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5581650bc560 con 0x5581650b0a10 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.288+0000 7f7b8de3c740 10 client.0 _ll_register_callbacks cb 0x558164fca8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.288+0000 7f7b89200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 0x5581650b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.288+0000 7f7b89200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 0x5581650b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51944/0 (socket says 172.21.15.69:51944) 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.288+0000 7f7b89200640 1 -- 172.21.15.69:0/103727264 learned_addr learned my addr 172.21.15.69:0/103727264 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.289+0000 7f7b89200640 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b95f0 0x5581650b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.289+0000 7f7b89200640 1 -- 172.21.15.69:0/103727264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b0a10 msgr2=0x5581650b6b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.289+0000 7f7b89200640 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b0a10 0x5581650b6b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.289+0000 7f7b89200640 1 -- 172.21.15.69:0/103727264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b95f0 msgr2=0x5581650b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.289+0000 7f7b89200640 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b95f0 0x5581650b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.289+0000 7f7b89200640 1 -- 172.21.15.69:0/103727264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558164deb7e0 con 0x5581650b00f0 2024-03-19T01:17:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.289+0000 7f7b89200640 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 0x5581650b6630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0057a0 tx=0x7f7b7c00e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb8953ac1bac05b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.289+0000 7f7b6e000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:17:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.289+0000 7f7b6e000640 1 -- 172.21.15.69:0/103727264 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c02d060 con 0x5581650b00f0 2024-03-19T01:17:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.289+0000 7f7b6e000640 1 -- 172.21.15.69:0/103727264 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f7b7c004250 con 0x5581650b00f0 2024-03-19T01:17:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.289+0000 7f7b8de3c740 1 -- 172.21.15.69:0/103727264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b7c009e50 con 0x5581650b00f0 2024-03-19T01:17:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.290+0000 7f7b6e000640 1 -- 172.21.15.69:0/103727264 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7b7c034050 con 0x5581650b00f0 2024-03-19T01:17:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.290+0000 7f7b8de3c740 10 client.16860 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:11.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.290+0000 7f7b8de3c740 1 -- 172.21.15.69:0/103727264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558164e4bf70 con 0x5581650b00f0 2024-03-19T01:17:11.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.290+0000 7f7b6e000640 1 -- 172.21.15.69:0/103727264 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7c009e50 con 0x5581650b00f0 2024-03-19T01:17:11.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.290+0000 7f7b6e000640 1 -- 172.21.15.69:0/103727264 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f7b7c0048c0 con 0x5581650b00f0 2024-03-19T01:17:11.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.290+0000 7f7b8de3c740 10 client.16860 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:11.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.290+0000 7f7b8de3c740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:11.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.290+0000 7f7b8de3c740 20 client.16860 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:11.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.290+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5581650c9a60 0x5581650cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.290+0000 7f7b8de3c740 4 client.16860 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:11.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.290+0000 7f7b8de3c740 1 -- 172.21.15.69:0/103727264 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55816501c4f0 con 0x5581650c9a60 2024-03-19T01:17:11.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.291+0000 7f7b83400640 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5581650c9a60 0x5581650cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.291+0000 7f7b6e000640 10 client.16860 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.291+0000 7f7b83400640 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5581650c9a60 0x5581650cbe50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b20ec65407c5141d server_cookie=150f6ff77cc25b4c in_seq=0 out_seq=0 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b6e000640 1 -- 172.21.15.69:0/103727264 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55816501c4f0 con 0x5581650c9a60 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b6e000640 10 client.16860 handle_command_reply: tid=0 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b8de3c740 1 client.16860 shutdown 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b8de3c740 1 -- 172.21.15.69:0/103727264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 msgr2=0x5581650b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 0x5581650b6630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0057a0 tx=0x7f7b7c00e700 comp rx=0 tx=0).stop 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b8de3c740 1 -- 172.21.15.69:0/103727264 shutdown_connections 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5581650c9a60 0x5581650cbe50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b95f0 0x5581650b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b0a10 0x5581650b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/103727264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 0x5581650b6630 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b8de3c740 1 -- 172.21.15.69:0/103727264 >> 172.21.15.69:0/103727264 conn(0x5581650aa820 msgr2=0x5581650ac570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.292+0000 7f7b6e000640 0 client.16860 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:11.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.293+0000 7f7b8de3c740 1 -- 172.21.15.69:0/103727264 shutdown_connections 2024-03-19T01:17:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.293+0000 7f7b8de3c740 1 -- 172.21.15.69:0/103727264 wait complete. 2024-03-19T01:17:11.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.293+0000 7f7b8de3c740 20 client.16860 trim_cache size 0 max 16384 2024-03-19T01:17:11.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.311+0000 7f7b8de3c740 1 Processor -- start 2024-03-19T01:17:11.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.311+0000 7f7b8de3c740 1 -- start start 2024-03-19T01:17:11.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.311+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 0x5581650b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b0a10 0x5581650b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b95f0 0x5581650d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841accc0 con 0x5581650b00f0 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841ae980 con 0x5581650b0a10 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841ad390 con 0x5581650b95f0 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b0a10 0x5581650b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b0a10 0x5581650b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43874/0 (socket says 172.21.15.69:43874) 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b83400640 1 -- 172.21.15.69:0/2483897197 learned_addr learned my addr 172.21.15.69:0/2483897197 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b89200640 1 --2- 172.21.15.69:0/2483897197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 0x5581650b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b89200640 1 --2- 172.21.15.69:0/2483897197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b95f0 0x5581650d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b83400640 1 -- 172.21.15.69:0/2483897197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b95f0 msgr2=0x5581650d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b83400640 1 --2- 172.21.15.69:0/2483897197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b95f0 0x5581650d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b83400640 1 -- 172.21.15.69:0/2483897197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 msgr2=0x5581650b8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b83400640 1 --2- 172.21.15.69:0/2483897197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 0x5581650b8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b83400640 1 -- 172.21.15.69:0/2483897197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581650b0840 con 0x5581650b0a10 2024-03-19T01:17:11.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.312+0000 7f7b83400640 1 --2- 172.21.15.69:0/2483897197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b0a10 0x5581650b5de0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009230 tx=0x7f7b7002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd7ea925572e8907 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.313+0000 7f7b78c00640 1 -- 172.21.15.69:0/2483897197 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70048d10 con 0x5581650b0a10 2024-03-19T01:17:11.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.313+0000 7f7b78c00640 1 -- 172.21.15.69:0/2483897197 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7004d070 con 0x5581650b0a10 2024-03-19T01:17:11.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.313+0000 7f7b78c00640 1 -- 172.21.15.69:0/2483897197 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7002cb10 con 0x5581650b0a10 2024-03-19T01:17:11.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.313+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2483897197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b0a10 msgr2=0x5581650b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.313+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/2483897197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b0a10 0x5581650b5de0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009230 tx=0x7f7b7002e040 comp rx=0 tx=0).stop 2024-03-19T01:17:11.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.314+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2483897197 shutdown_connections 2024-03-19T01:17:11.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.314+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/2483897197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b95f0 0x5581650d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.314+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/2483897197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b0a10 0x5581650b5de0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.314+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/2483897197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b00f0 0x5581650b8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.314+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2483897197 >> 172.21.15.69:0/2483897197 conn(0x5581650aa820 msgr2=0x5581650abb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.314+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2483897197 shutdown_connections 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.314+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2483897197 wait complete. 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.314+0000 7f7b8de3c740 1 -- start start 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.314+0000 7f7b8de3c740 1 Processor -- start 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 0x5581650bd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b0a10 0x5581650bd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b8de3c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b95f0 0x55816512eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841ae1f0 con 0x5581650b95f0 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841adaf0 con 0x5581650b00f0 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b8de3c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841aac30 con 0x5581650b0a10 2024-03-19T01:17:11.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b8de3c740 10 client.0 _ll_register_callbacks cb 0x7f7b7c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b0a10 0x5581650bd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b0a10 0x5581650bd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:47126/0 (socket says 172.21.15.69:47126) 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b83400640 1 -- 172.21.15.69:0/2582948968 learned_addr learned my addr 172.21.15.69:0/2582948968 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b89200640 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 0x5581650bd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b89200640 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b95f0 0x55816512eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b83400640 1 -- 172.21.15.69:0/2582948968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 msgr2=0x5581650bd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b83400640 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 0x5581650bd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b83400640 1 -- 172.21.15.69:0/2582948968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b95f0 msgr2=0x55816512eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b83400640 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b95f0 0x55816512eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.315+0000 7f7b83400640 1 -- 172.21.15.69:0/2582948968 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558164ce3170 con 0x5581650b0a10 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.316+0000 7f7b89200640 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b95f0 0x55816512eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.316+0000 7f7b83400640 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b0a10 0x5581650bd9d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7b70036040 tx=0x7f7b70000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b480ebb625e50af3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.316+0000 7f7b6e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:11.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.316+0000 7f7b6e000640 1 -- 172.21.15.69:0/2582948968 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b700483f0 con 0x5581650b0a10 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.316+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2582948968 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b7004d070 con 0x5581650b0a10 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.316+0000 7f7b6e000640 1 -- 172.21.15.69:0/2582948968 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f7b70048590 con 0x5581650b0a10 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.316+0000 7f7b6e000640 1 -- 172.21.15.69:0/2582948968 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7b7002c8b0 con 0x5581650b0a10 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.316+0000 7f7b8de3c740 10 client.27079 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.316+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2582948968 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558165130060 con 0x5581650b0a10 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.317+0000 7f7b6e000640 1 -- 172.21.15.69:0/2582948968 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7004d070 con 0x5581650b0a10 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.317+0000 7f7b6e000640 1 -- 172.21.15.69:0/2582948968 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f7b70004f80 con 0x5581650b0a10 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.317+0000 7f7b8de3c740 10 client.27079 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.317+0000 7f7b8de3c740 10 client.27079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.317+0000 7f7b8de3c740 20 client.27079 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.317+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5581651308d0 0x558165132cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.317+0000 7f7b8de3c740 4 client.27079 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.317+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2582948968 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558165016840 con 0x5581651308d0 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.317+0000 7f7b89200640 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5581651308d0 0x558165132cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.318+0000 7f7b89200640 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5581651308d0 0x558165132cf0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb7ecd6395c55f04 server_cookie=d38255add58837be in_seq=0 out_seq=0 2024-03-19T01:17:11.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.318+0000 7f7b6e000640 10 client.27079 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:11.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.318+0000 7f7b6e000640 1 -- 172.21.15.69:0/2582948968 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558165016840 con 0x5581651308d0 2024-03-19T01:17:11.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.318+0000 7f7b6e000640 10 client.27079 handle_command_reply: tid=0 2024-03-19T01:17:11.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.318+0000 7f7b8de3c740 1 client.27079 shutdown 2024-03-19T01:17:11.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.318+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2582948968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b0a10 msgr2=0x5581650bd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.318+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b0a10 0x5581650bd9d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7b70036040 tx=0x7f7b70000bc0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.319+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2582948968 shutdown_connections 2024-03-19T01:17:11.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.319+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5581651308d0 0x558165132cf0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.319+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581650b95f0 0x55816512eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.319+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5581650b0a10 0x5581650bd9d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.319+0000 7f7b8de3c740 1 --2- 172.21.15.69:0/2582948968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581650b00f0 0x5581650bd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.319+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2582948968 >> 172.21.15.69:0/2582948968 conn(0x5581650aa820 msgr2=0x5581650ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.319+0000 7f7b6e000640 0 client.27079 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:11.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.319+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2582948968 shutdown_connections 2024-03-19T01:17:11.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.319+0000 7f7b8de3c740 1 -- 172.21.15.69:0/2582948968 wait complete. 2024-03-19T01:17:11.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.319+0000 7f7b8de3c740 20 client.27079 trim_cache size 0 max 16384 2024-03-19T01:17:11.324 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:17:11.324 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:17:11.324 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "3f8feed6-b2ce-4d19-9573-2666cedcf652", 2024-03-19T01:17:11.324 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:17:11.324 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:17:11.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.320+0000 7f7b89c00640 1 -- 172.21.15.69:0/2216147215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7b680a29f0 msgr2=0x7f7b680a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.320+0000 7f7b89c00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7b680a29f0 0x7f7b680a4eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5581650afbc0 tx=0x7f7b7400cf90 comp rx=0 tx=0).stop 2024-03-19T01:17:11.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.320+0000 7f7b89c00640 1 -- 172.21.15.69:0/2216147215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b84101680 msgr2=0x7f7b84197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.320+0000 7f7b89c00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b84101680 0x7f7b84197f90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400ebc0 tx=0x7f7b7400c6a0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.320+0000 7f7b89c00640 1 -- 172.21.15.69:0/2216147215 shutdown_connections 2024-03-19T01:17:11.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.320+0000 7f7b89c00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7b680a29f0 0x7f7b680a4eb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.320+0000 7f7b89c00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b84107690 0x7f7b8419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.320+0000 7f7b89c00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b84102040 0x7f7b841984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.320+0000 7f7b89c00640 1 --2- 172.21.15.69:0/2216147215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b84101680 0x7f7b84197f90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.320+0000 7f7b89c00640 1 -- 172.21.15.69:0/2216147215 >> 172.21.15.69:0/2216147215 conn(0x7f7b840fd190 msgr2=0x7f7b840fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.321+0000 7f7b89c00640 1 -- 172.21.15.69:0/2216147215 shutdown_connections 2024-03-19T01:17:11.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.321+0000 7f7b89c00640 1 -- 172.21.15.69:0/2216147215 wait complete. 2024-03-19T01:17:11.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:17:11.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:17:11.435 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:11 smithi069 ceph-mon[31727]: pgmap v199: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 1.76k op/s 2024-03-19T01:17:11.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:11 smithi086 ceph-mon[66829]: pgmap v199: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 1.76k op/s 2024-03-19T01:17:11.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:11 smithi163 ceph-mon[67428]: pgmap v199: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 1.76k op/s 2024-03-19T01:17:11.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.622+0000 7f165bc00640 1 -- 172.21.15.69:0/3081686931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1654108030 msgr2=0x7f1654108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.622+0000 7f165bc00640 1 --2- 172.21.15.69:0/3081686931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1654108030 0x7f1654108410 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1648009900 tx=0x7f164802bc30 comp rx=0 tx=0).stop 2024-03-19T01:17:11.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.622+0000 7f165bc00640 1 -- 172.21.15.69:0/3081686931 shutdown_connections 2024-03-19T01:17:11.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.622+0000 7f165bc00640 1 --2- 172.21.15.69:0/3081686931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f165410d100 0x7f1654114cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.622+0000 7f165bc00640 1 --2- 172.21.15.69:0/3081686931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16541089e0 0x7f165410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.622+0000 7f165bc00640 1 --2- 172.21.15.69:0/3081686931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1654108030 0x7f1654108410 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.622+0000 7f165bc00640 1 -- 172.21.15.69:0/3081686931 >> 172.21.15.69:0/3081686931 conn(0x7f1654075d60 msgr2=0x7f1654078180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.623+0000 7f165bc00640 1 -- 172.21.15.69:0/3081686931 shutdown_connections 2024-03-19T01:17:11.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.623+0000 7f165bc00640 1 -- 172.21.15.69:0/3081686931 wait complete. 2024-03-19T01:17:11.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.624+0000 7f165bc00640 1 Processor -- start 2024-03-19T01:17:11.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.624+0000 7f165bc00640 1 -- start start 2024-03-19T01:17:11.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.625+0000 7f165bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1654108030 0x7f16541a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.625+0000 7f165bc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f165410d100 0x7f16541a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.625+0000 7f165bc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f165419e5c0 0x7f165419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.625+0000 7f165bc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1654118200 con 0x7f165419e5c0 2024-03-19T01:17:11.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.625+0000 7f165bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1654118080 con 0x7f1654108030 2024-03-19T01:17:11.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.625+0000 7f165bc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1654118380 con 0x7f165410d100 2024-03-19T01:17:11.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.625+0000 7f1659200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1654108030 0x7f16541a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.626+0000 7f1659200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1654108030 0x7f16541a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43914/0 (socket says 172.21.15.69:43914) 2024-03-19T01:17:11.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.626+0000 7f1659200640 1 -- 172.21.15.69:0/146884228 learned_addr learned my addr 172.21.15.69:0/146884228 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:11.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.626+0000 7f1653e00640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f165410d100 0x7f16541a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.626+0000 7f1659c00640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f165419e5c0 0x7f165419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.626+0000 7f1659200640 1 -- 172.21.15.69:0/146884228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f165410d100 msgr2=0x7f16541a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.626+0000 7f1659200640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f165410d100 0x7f16541a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.626+0000 7f1659200640 1 -- 172.21.15.69:0/146884228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f165419e5c0 msgr2=0x7f165419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.626+0000 7f1659200640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f165419e5c0 0x7f165419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.626+0000 7f1659200640 1 -- 172.21.15.69:0/146884228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165419f0e0 con 0x7f1654108030 2024-03-19T01:17:11.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.626+0000 7f1659200640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1654108030 0x7f16541a4410 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f164802c240 tx=0x7f16480042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.627+0000 7f1651600640 1 -- 172.21.15.69:0/146884228 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648043020 con 0x7f1654108030 2024-03-19T01:17:11.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.627+0000 7f1651600640 1 -- 172.21.15.69:0/146884228 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16480054e0 con 0x7f1654108030 2024-03-19T01:17:11.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.627+0000 7f1651600640 1 -- 172.21.15.69:0/146884228 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648003b50 con 0x7f1654108030 2024-03-19T01:17:11.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.627+0000 7f165bc00640 1 -- 172.21.15.69:0/146884228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165419f370 con 0x7f1654108030 2024-03-19T01:17:11.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.627+0000 7f165bc00640 1 -- 172.21.15.69:0/146884228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16541ab570 con 0x7f1654108030 2024-03-19T01:17:11.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.631+0000 7f1651600640 1 -- 172.21.15.69:0/146884228 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f1648046020 con 0x7f1654108030 2024-03-19T01:17:11.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.636+0000 7f165e836740 1 Processor -- start 2024-03-19T01:17:11.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.637+0000 7f1651600640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f163c0a29f0 0x7f163c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.637+0000 7f1651600640 1 -- 172.21.15.69:0/146884228 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f16480f10d0 con 0x7f1654108030 2024-03-19T01:17:11.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.638+0000 7f1659c00640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f163c0a29f0 0x7f163c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.638+0000 7f165e836740 1 -- start start 2024-03-19T01:17:11.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.639+0000 7f165e836740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 0x5648f27db120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.639+0000 7f165e836740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27db660 0x5648f27e3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.639+0000 7f165e836740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 0x5648f27e6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.639+0000 7f165e836740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5648f273f2c0 con 0x5648f27e4240 2024-03-19T01:17:11.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.639+0000 7f165e836740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648f26e7b60 con 0x5648f27db660 2024-03-19T01:17:11.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.639+0000 7f165e836740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5648f273b360 con 0x5648f27dad40 2024-03-19T01:17:11.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.639+0000 7f1653e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27db660 0x5648f27e3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.639+0000 7f1659c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 0x5648f27e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.639+0000 7f1653e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27db660 0x5648f27e3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43920/0 (socket says 172.21.15.69:43920) 2024-03-19T01:17:11.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.639+0000 7f1653e00640 1 -- 172.21.15.69:0/3013205199 learned_addr learned my addr 172.21.15.69:0/3013205199 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:11.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.639+0000 7f1659200640 1 --2- 172.21.15.69:0/3013205199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 0x5648f27db120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.640+0000 7f1659200640 1 -- 172.21.15.69:0/3013205199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27db660 msgr2=0x5648f27e3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.640+0000 7f1659200640 1 --2- 172.21.15.69:0/3013205199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27db660 0x5648f27e3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.640+0000 7f1659200640 1 -- 172.21.15.69:0/3013205199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 msgr2=0x5648f27e6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.640+0000 7f1659200640 1 --2- 172.21.15.69:0/3013205199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 0x5648f27e6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.640+0000 7f1659200640 1 -- 172.21.15.69:0/3013205199 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f251d7d0 con 0x5648f27dad40 2024-03-19T01:17:11.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.640+0000 7f1659c00640 1 --2- 172.21.15.69:0/3013205199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 0x5648f27e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:11.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.641+0000 7f1659200640 1 --2- 172.21.15.69:0/3013205199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 0x5648f27db120 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f16480ee4c0 tx=0x7f16480bcf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4de54a18d95e1e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.641+0000 7f1646000640 1 -- 172.21.15.69:0/3013205199 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648005690 con 0x5648f27dad40 2024-03-19T01:17:11.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.641+0000 7f1646000640 1 -- 172.21.15.69:0/3013205199 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16480bdad0 con 0x5648f27dad40 2024-03-19T01:17:11.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.641+0000 7f1646000640 1 -- 172.21.15.69:0/3013205199 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16480b9a40 con 0x5648f27dad40 2024-03-19T01:17:11.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.641+0000 7f1659c00640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f163c0a29f0 0x7f163c0a4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f164c009370 tx=0x7f164c009df0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.642+0000 7f165e836740 1 -- 172.21.15.69:0/3013205199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 msgr2=0x5648f27db120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.642+0000 7f165e836740 1 --2- 172.21.15.69:0/3013205199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 0x5648f27db120 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f16480ee4c0 tx=0x7f16480bcf10 comp rx=0 tx=0).stop 2024-03-19T01:17:11.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.643+0000 7f165e836740 1 -- 172.21.15.69:0/3013205199 shutdown_connections 2024-03-19T01:17:11.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.643+0000 7f165e836740 1 --2- 172.21.15.69:0/3013205199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 0x5648f27e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.643+0000 7f165e836740 1 --2- 172.21.15.69:0/3013205199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27db660 0x5648f27e3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.643+0000 7f165e836740 1 --2- 172.21.15.69:0/3013205199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 0x5648f27db120 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.643+0000 7f165e836740 1 -- 172.21.15.69:0/3013205199 >> 172.21.15.69:0/3013205199 conn(0x5648f27d5470 msgr2=0x5648f27d5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.643+0000 7f165e836740 1 -- 172.21.15.69:0/3013205199 shutdown_connections 2024-03-19T01:17:11.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.643+0000 7f165e836740 1 -- 172.21.15.69:0/3013205199 wait complete. 2024-03-19T01:17:11.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.644+0000 7f165e836740 1 -- start start 2024-03-19T01:17:11.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.644+0000 7f165e836740 1 Processor -- start 2024-03-19T01:17:11.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.645+0000 7f165e836740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 0x5648f27e12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.645+0000 7f165e836740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27db660 0x5648f27e1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.645+0000 7f165e836740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27e4240 0x5648f27e1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.645+0000 7f165e836740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5648f27e6df0 con 0x5648f27db660 2024-03-19T01:17:11.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.645+0000 7f165e836740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648f26fafc0 con 0x5648f27e4240 2024-03-19T01:17:11.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.645+0000 7f165e836740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5648f27e6f70 con 0x5648f27dad40 2024-03-19T01:17:11.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.645+0000 7f165e836740 10 client.0 _ll_register_callbacks cb 0x5648f26fa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:11.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.645+0000 7f1659200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 0x5648f27e12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.645+0000 7f1659200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 0x5648f27e12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:47164/0 (socket says 172.21.15.69:47164) 2024-03-19T01:17:11.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.645+0000 7f1659200640 1 -- 172.21.15.69:0/2650013624 learned_addr learned my addr 172.21.15.69:0/2650013624 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:11.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.645+0000 7f1659c00640 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27e4240 0x5648f27e1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.646+0000 7f1653e00640 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27db660 0x5648f27e1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.646+0000 7f1659200640 1 -- 172.21.15.69:0/2650013624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27e4240 msgr2=0x5648f27e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.646+0000 7f1659200640 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27e4240 0x5648f27e1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.646+0000 7f1659200640 1 -- 172.21.15.69:0/2650013624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27db660 msgr2=0x5648f27e1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.646+0000 7f1659200640 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27db660 0x5648f27e1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.646+0000 7f1659200640 1 -- 172.21.15.69:0/2650013624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f24533f0 con 0x5648f27dad40 2024-03-19T01:17:11.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.646+0000 7f1653e00640 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27db660 0x5648f27e1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:11.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.646+0000 7f1659200640 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 0x5648f27e12e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f16480ee570 tx=0x7f164802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f42625f018874c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.646+0000 7f163aa00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:11.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.647+0000 7f163aa00640 1 -- 172.21.15.69:0/2650013624 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16480da020 con 0x5648f27dad40 2024-03-19T01:17:11.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.647+0000 7f163aa00640 1 -- 172.21.15.69:0/2650013624 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f16480bd630 con 0x5648f27dad40 2024-03-19T01:17:11.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.647+0000 7f165e836740 1 -- 172.21.15.69:0/2650013624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16480bdad0 con 0x5648f27dad40 2024-03-19T01:17:11.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.647+0000 7f163aa00640 1 -- 172.21.15.69:0/2650013624 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f16480a3050 con 0x5648f27dad40 2024-03-19T01:17:11.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.647+0000 7f165e836740 10 client.27091 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:11.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.647+0000 7f165e836740 1 -- 172.21.15.69:0/2650013624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5648f24130f0 con 0x5648f27dad40 2024-03-19T01:17:11.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.648+0000 7f163aa00640 1 -- 172.21.15.69:0/2650013624 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16480bdad0 con 0x5648f27dad40 2024-03-19T01:17:11.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.648+0000 7f163aa00640 1 -- 172.21.15.69:0/2650013624 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f16480c56e0 con 0x5648f27dad40 2024-03-19T01:17:11.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.648+0000 7f165e836740 10 client.27091 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:11.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.648+0000 7f165e836740 10 client.27091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:11.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.648+0000 7f165e836740 20 client.27091 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:11.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.648+0000 7f165e836740 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5648f27f3fd0 0x5648f27f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.648+0000 7f165e836740 4 client.27091 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:11.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.648+0000 7f165e836740 1 -- 172.21.15.69:0/2650013624 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648f26e0610 con 0x5648f27f3fd0 2024-03-19T01:17:11.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.648+0000 7f1653e00640 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5648f27f3fd0 0x5648f27f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.649+0000 7f1653e00640 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5648f27f3fd0 0x5648f27f63c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d44eac3ba109585 server_cookie=5dd52af3b49aad61 in_seq=0 out_seq=0 2024-03-19T01:17:11.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.649+0000 7f163aa00640 10 client.27091 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:11.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.652+0000 7f163aa00640 1 -- 172.21.15.69:0/2650013624 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5648f26e0610 con 0x5648f27f3fd0 2024-03-19T01:17:11.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.652+0000 7f163aa00640 10 client.27091 handle_command_reply: tid=0 2024-03-19T01:17:11.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.652+0000 7f165e836740 1 client.27091 shutdown 2024-03-19T01:17:11.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.653+0000 7f165e836740 1 -- 172.21.15.69:0/2650013624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 msgr2=0x5648f27e12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.653+0000 7f165e836740 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 0x5648f27e12e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f16480ee570 tx=0x7f164802bc30 comp rx=0 tx=0).stop 2024-03-19T01:17:11.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.653+0000 7f165e836740 1 -- 172.21.15.69:0/2650013624 shutdown_connections 2024-03-19T01:17:11.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.653+0000 7f165e836740 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5648f27f3fd0 0x5648f27f63c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.653+0000 7f165e836740 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27e4240 0x5648f27e1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.653+0000 7f165e836740 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27db660 0x5648f27e1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.653+0000 7f165e836740 1 --2- 172.21.15.69:0/2650013624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27dad40 0x5648f27e12e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.653+0000 7f163aa00640 0 client.27091 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:11.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.653+0000 7f165e836740 1 -- 172.21.15.69:0/2650013624 >> 172.21.15.69:0/2650013624 conn(0x5648f27d5470 msgr2=0x5648f27d71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.653+0000 7f165e836740 1 -- 172.21.15.69:0/2650013624 shutdown_connections 2024-03-19T01:17:11.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.653+0000 7f165e836740 1 -- 172.21.15.69:0/2650013624 wait complete. 2024-03-19T01:17:11.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.654+0000 7f165e836740 20 client.27091 trim_cache size 0 max 16384 2024-03-19T01:17:11.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.688+0000 7f165e836740 1 Processor -- start 2024-03-19T01:17:11.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.688+0000 7f165e836740 1 -- start start 2024-03-19T01:17:11.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.688+0000 7f165e836740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27dad40 0x5648f27efb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.688+0000 7f165e836740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27db660 0x5648f27e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.688+0000 7f165e836740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27e4240 0x5648f27fbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.688+0000 7f165e836740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16340102e0 con 0x5648f27dad40 2024-03-19T01:17:11.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.688+0000 7f165e836740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1634011fa0 con 0x5648f27e4240 2024-03-19T01:17:11.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.688+0000 7f165e836740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f16340109b0 con 0x5648f27db660 2024-03-19T01:17:11.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.689+0000 7f1653e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27db660 0x5648f27e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.689+0000 7f1653e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27db660 0x5648f27e1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:47170/0 (socket says 172.21.15.69:47170) 2024-03-19T01:17:11.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.689+0000 7f1653e00640 1 -- 172.21.15.69:0/298296347 learned_addr learned my addr 172.21.15.69:0/298296347 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:11.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.689+0000 7f1659c00640 1 --2- 172.21.15.69:0/298296347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27e4240 0x5648f27fbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.689+0000 7f1659200640 1 --2- 172.21.15.69:0/298296347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27dad40 0x5648f27efb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.689+0000 7f1653e00640 1 -- 172.21.15.69:0/298296347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27e4240 msgr2=0x5648f27fbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.689+0000 7f1653e00640 1 --2- 172.21.15.69:0/298296347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27e4240 0x5648f27fbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.689+0000 7f1653e00640 1 -- 172.21.15.69:0/298296347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27dad40 msgr2=0x5648f27efb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:11.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.689+0000 7f1653e00640 1 --2- 172.21.15.69:0/298296347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27dad40 0x5648f27efb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.689+0000 7f1653e00640 1 -- 172.21.15.69:0/298296347 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f27e69a0 con 0x5648f27db660 2024-03-19T01:17:11.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.689+0000 7f1659c00640 1 --2- 172.21.15.69:0/298296347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27e4240 0x5648f27fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:11.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.690+0000 7f1653e00640 1 --2- 172.21.15.69:0/298296347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27db660 0x5648f27e1b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5648f27e0b40 tx=0x7f1640004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98b353d8e7c0afe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.690+0000 7f1645600640 1 -- 172.21.15.69:0/298296347 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640023020 con 0x5648f27db660 2024-03-19T01:17:11.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.690+0000 7f1645600640 1 -- 172.21.15.69:0/298296347 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1640003d20 con 0x5648f27db660 2024-03-19T01:17:11.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.690+0000 7f1645600640 1 -- 172.21.15.69:0/298296347 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640010b50 con 0x5648f27db660 2024-03-19T01:17:11.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.691+0000 7f165e836740 1 -- 172.21.15.69:0/298296347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27db660 msgr2=0x5648f27e1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.691+0000 7f165e836740 1 --2- 172.21.15.69:0/298296347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27db660 0x5648f27e1b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5648f27e0b40 tx=0x7f1640004410 comp rx=0 tx=0).stop 2024-03-19T01:17:11.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.692+0000 7f165e836740 1 -- 172.21.15.69:0/298296347 shutdown_connections 2024-03-19T01:17:11.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.692+0000 7f165e836740 1 --2- 172.21.15.69:0/298296347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27e4240 0x5648f27fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.692+0000 7f165e836740 1 --2- 172.21.15.69:0/298296347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27db660 0x5648f27e1b80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.692+0000 7f165e836740 1 --2- 172.21.15.69:0/298296347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27dad40 0x5648f27efb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.692+0000 7f165e836740 1 -- 172.21.15.69:0/298296347 >> 172.21.15.69:0/298296347 conn(0x5648f27d5470 msgr2=0x5648f27d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.692+0000 7f165e836740 1 -- 172.21.15.69:0/298296347 shutdown_connections 2024-03-19T01:17:11.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.692+0000 7f165e836740 1 -- 172.21.15.69:0/298296347 wait complete. 2024-03-19T01:17:11.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.692+0000 7f165e836740 1 -- start start 2024-03-19T01:17:11.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.693+0000 7f165e836740 1 Processor -- start 2024-03-19T01:17:11.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.693+0000 7f165e836740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27dad40 0x5648f27e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.693+0000 7f165e836740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27db660 0x5648f27e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.693+0000 7f165e836740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 0x5648f27e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.693+0000 7f165e836740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1634011810 con 0x5648f27e4240 2024-03-19T01:17:11.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.693+0000 7f165e836740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1634011110 con 0x5648f27dad40 2024-03-19T01:17:11.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.694+0000 7f165e836740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f163400e250 con 0x5648f27db660 2024-03-19T01:17:11.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.694+0000 7f165e836740 10 client.0 _ll_register_callbacks cb 0x7f16480b9730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:11.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.694+0000 7f1659200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27dad40 0x5648f27e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.694+0000 7f1659200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27dad40 0x5648f27e7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:43946/0 (socket says 172.21.15.69:43946) 2024-03-19T01:17:11.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.694+0000 7f1659200640 1 -- 172.21.15.69:0/252802791 learned_addr learned my addr 172.21.15.69:0/252802791 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:11.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.694+0000 7f1659200640 1 -- 172.21.15.69:0/252802791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27db660 msgr2=0x5648f27e7fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:17:11.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.694+0000 7f1659200640 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27db660 0x5648f27e7fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.694+0000 7f1659200640 1 -- 172.21.15.69:0/252802791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 msgr2=0x5648f27e84e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:17:11.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.695+0000 7f1659c00640 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 0x5648f27e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.695+0000 7f1659200640 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 0x5648f27e84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.695+0000 7f1659200640 1 -- 172.21.15.69:0/252802791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f286a260 con 0x5648f27dad40 2024-03-19T01:17:11.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.695+0000 7f1659c00640 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 0x5648f27e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:11.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.695+0000 7f1659200640 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27dad40 0x5648f27e7a60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f16480b9da0 tx=0x7f16480c5910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c5153686d5c9d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:11.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.695+0000 7f163aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:17:11.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.695+0000 7f163aa00640 1 -- 172.21.15.69:0/252802791 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16480da070 con 0x5648f27dad40 2024-03-19T01:17:11.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.696+0000 7f163aa00640 1 -- 172.21.15.69:0/252802791 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9656+0+0 (secure 0 0 0) 0x7f16480bd840 con 0x5648f27dad40 2024-03-19T01:17:11.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.696+0000 7f165e836740 1 -- 172.21.15.69:0/252802791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1640003d20 con 0x5648f27dad40 2024-03-19T01:17:11.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.697+0000 7f163aa00640 1 -- 172.21.15.69:0/252802791 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f16480bd1a0 con 0x5648f27dad40 2024-03-19T01:17:11.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.697+0000 7f165e836740 10 client.27152 fetch_fsmap learned FSMap version 44 2024-03-19T01:17:11.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.697+0000 7f165e836740 1 -- 172.21.15.69:0/252802791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5648f286a260 con 0x5648f27dad40 2024-03-19T01:17:11.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.698+0000 7f163aa00640 1 -- 172.21.15.69:0/252802791 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1640003d20 con 0x5648f27dad40 2024-03-19T01:17:11.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.699+0000 7f163aa00640 1 -- 172.21.15.69:0/252802791 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 44) ==== 8422+0+0 (secure 0 0 0) 0x7f16480bc820 con 0x5648f27dad40 2024-03-19T01:17:11.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.699+0000 7f165e836740 10 client.27152 fetch_fsmap finished waiting for FSMap version 44 2024-03-19T01:17:11.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.699+0000 7f165e836740 10 client.27152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:11.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.700+0000 7f165e836740 20 client.27152 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:11.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.700+0000 7f165e836740 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5648f286b930 0x5648f286dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:11.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.700+0000 7f165e836740 4 client.27152 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:17:11.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.700+0000 7f165e836740 1 -- 172.21.15.69:0/252802791 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648f274c800 con 0x5648f286b930 2024-03-19T01:17:11.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.702+0000 7f1653e00640 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5648f286b930 0x5648f286dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:11.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.704+0000 7f1653e00640 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5648f286b930 0x5648f286dd70 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=425b59626d2646fd server_cookie=cb095bd8fab7f87b in_seq=0 out_seq=0 2024-03-19T01:17:11.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.704+0000 7f163aa00640 10 client.27152 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:11.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.707+0000 7f163aa00640 1 -- 172.21.15.69:0/252802791 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5648f274c800 con 0x5648f286b930 2024-03-19T01:17:11.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.707+0000 7f163aa00640 10 client.27152 handle_command_reply: tid=0 2024-03-19T01:17:11.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.707+0000 7f165e836740 1 client.27152 shutdown 2024-03-19T01:17:11.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.707+0000 7f165e836740 1 -- 172.21.15.69:0/252802791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27dad40 msgr2=0x5648f27e7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.707+0000 7f165e836740 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27dad40 0x5648f27e7a60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f16480b9da0 tx=0x7f16480c5910 comp rx=0 tx=0).stop 2024-03-19T01:17:11.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.708+0000 7f165e836740 1 -- 172.21.15.69:0/252802791 shutdown_connections 2024-03-19T01:17:11.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.708+0000 7f165e836740 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5648f286b930 0x5648f286dd70 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.708+0000 7f165e836740 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f27e4240 0x5648f27e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.708+0000 7f165e836740 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5648f27db660 0x5648f27e7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.708+0000 7f163aa00640 0 client.27152 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:11.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.708+0000 7f165e836740 1 --2- 172.21.15.69:0/252802791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f27dad40 0x5648f27e7a60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.708+0000 7f165e836740 1 -- 172.21.15.69:0/252802791 >> 172.21.15.69:0/252802791 conn(0x5648f27d5470 msgr2=0x5648f27d6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.708+0000 7f165e836740 1 -- 172.21.15.69:0/252802791 shutdown_connections 2024-03-19T01:17:11.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.708+0000 7f165e836740 1 -- 172.21.15.69:0/252802791 wait complete. 2024-03-19T01:17:11.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.709+0000 7f165e836740 20 client.27152 trim_cache size 0 max 16384 2024-03-19T01:17:11.710 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:17:11.710 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (7712 inodes in the stack)", 2024-03-19T01:17:11.710 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:17:11.711 INFO:teuthology.orchestra.run.smithi069.stdout: "3f8feed6-b2ce-4d19-9573-2666cedcf652": { 2024-03-19T01:17:11.711 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:17:11.711 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "3f8feed6-b2ce-4d19-9573-2666cedcf652", 2024-03-19T01:17:11.711 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:17:11.711 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:17:11.711 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:17:11.711 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.709+0000 7f165bc00640 1 -- 172.21.15.69:0/146884228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f163c0a29f0 msgr2=0x7f163c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.709+0000 7f165bc00640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f163c0a29f0 0x7f163c0a4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f164c009370 tx=0x7f164c009df0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.709+0000 7f165bc00640 1 -- 172.21.15.69:0/146884228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1654108030 msgr2=0x7f16541a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.709+0000 7f165bc00640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1654108030 0x7f16541a4410 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f164802c240 tx=0x7f16480042f0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.710+0000 7f165bc00640 1 -- 172.21.15.69:0/146884228 shutdown_connections 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.710+0000 7f165bc00640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f165419e5c0 0x7f165419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.710+0000 7f165bc00640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f165410d100 0x7f16541a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.710+0000 7f165bc00640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f163c0a29f0 0x7f163c0a4eb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.710+0000 7f165bc00640 1 --2- 172.21.15.69:0/146884228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1654108030 0x7f16541a4410 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.710+0000 7f165bc00640 1 -- 172.21.15.69:0/146884228 >> 172.21.15.69:0/146884228 conn(0x7f1654075d60 msgr2=0x7f16540780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.710+0000 7f165bc00640 1 -- 172.21.15.69:0/146884228 shutdown_connections 2024-03-19T01:17:11.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:11.710+0000 7f165bc00640 1 -- 172.21.15.69:0/146884228 wait complete. 2024-03-19T01:17:11.736 INFO:tasks.cephfs.filesystem:scrub status for tag:3f8feed6-b2ce-4d19-9573-2666cedcf652 - {'path': '/', 'tag': '3f8feed6-b2ce-4d19-9573-2666cedcf652', 'options': 'recursive,force'} 2024-03-19T01:17:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:12.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:12.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:17:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:12.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: pgmap v200: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 1.56k op/s 2024-03-19T01:17:13.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.542 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.542 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.542 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.542 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.542 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.542 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.545 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.545 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.545 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.545 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.545 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.546 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.546 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.546 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:13.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: pgmap v200: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 1.56k op/s 2024-03-19T01:17:13.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: pgmap v200: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 1.56k op/s 2024-03-19T01:17:13.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.765 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.765 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.765 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.765 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.765 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.765 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.766 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.766 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.766 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.766 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.766 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.767 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.768 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.768 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:13.768 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:13.768 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:14.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:14 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:14 smithi086 ceph-mon[66829]: osdmap e96: 12 total, 12 up, 12 in 2024-03-19T01:17:14.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:14.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:14.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:14.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:14.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:14.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:14.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:14 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:14.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:14 smithi069 ceph-mon[31727]: osdmap e96: 12 total, 12 up, 12 in 2024-03-19T01:17:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:14 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:14.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:14 smithi163 ceph-mon[67428]: osdmap e96: 12 total, 12 up, 12 in 2024-03-19T01:17:15.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:15 smithi069 ceph-mon[31727]: pgmap v202: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 1.43k op/s 2024-03-19T01:17:15.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:15.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:15.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:15.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:15.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:15 smithi163 ceph-mon[67428]: pgmap v202: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 1.43k op/s 2024-03-19T01:17:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:15.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:15 smithi086 ceph-mon[66829]: pgmap v202: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 1.43k op/s 2024-03-19T01:17:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.481 INFO:tasks.workunit.client.0.smithi069.stderr:+ cd linux-6.5.11 2024-03-19T01:17:16.481 INFO:tasks.workunit.client.0.smithi069.stderr:+ make defconfig 2024-03-19T01:17:16.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:16.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:16.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:16.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:16.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: pgmap v203: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 1.37k op/s 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:17.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: pgmap v203: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 1.37k op/s 2024-03-19T01:17:17.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:17.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.639 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/basic/fixdep 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: pgmap v203: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 1.37k op/s 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:17.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:17.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.031 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/kconfig/conf.o 2024-03-19T01:17:18.436 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/kconfig/confdata.o 2024-03-19T01:17:18.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:18.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:18.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:18.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:18.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:18.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:18.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:18.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:18.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:18.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:18.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:18.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:18.881 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/kconfig/expr.o 2024-03-19T01:17:19.276 INFO:tasks.workunit.client.0.smithi069.stdout: LEX scripts/kconfig/lexer.lex.c 2024-03-19T01:17:19.308 INFO:tasks.workunit.client.0.smithi069.stdout: YACC scripts/kconfig/parser.tab.[ch] 2024-03-19T01:17:19.451 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/kconfig/lexer.lex.o 2024-03-19T01:17:19.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: pgmap v204: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 1.28k op/s 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:19.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: pgmap v204: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 1.28k op/s 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:19.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: pgmap v204: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 1.28k op/s 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:19.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:19.973 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/kconfig/menu.o 2024-03-19T01:17:20.308 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/kconfig/parser.tab.o 2024-03-19T01:17:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:20.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:20.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:20.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:20.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.165 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/kconfig/preprocess.o 2024-03-19T01:17:21.471 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/kconfig/symbol.o 2024-03-19T01:17:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: pgmap v205: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 1.30k op/s 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:21.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: pgmap v205: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 1.30k op/s 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:21.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: pgmap v205: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 1.30k op/s 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:21.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:21.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.471 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/kconfig/util.o 2024-03-19T01:17:22.558 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTLD scripts/kconfig/conf 2024-03-19T01:17:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:22.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.596 INFO:tasks.workunit.client.0.smithi069.stdout:*** Default configuration is based on 'x86_64_defconfig' 2024-03-19T01:17:22.656 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.656 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.656 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:22.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:23.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:23.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:23.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:23.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:23 smithi163 ceph-mon[67428]: pgmap v206: 129 pgs: 129 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 1.75k op/s 2024-03-19T01:17:23.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:23.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:23.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:23.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:23 smithi086 ceph-mon[66829]: pgmap v206: 129 pgs: 129 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 1.75k op/s 2024-03-19T01:17:23.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:23.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:23.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:23.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:23 smithi069 ceph-mon[31727]: pgmap v206: 129 pgs: 129 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 1.75k op/s 2024-03-19T01:17:25.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:25 smithi163 ceph-mon[67428]: pgmap v207: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 1.88k op/s 2024-03-19T01:17:25.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:25 smithi086 ceph-mon[66829]: pgmap v207: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 1.88k op/s 2024-03-19T01:17:25.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:25 smithi069 ceph-mon[31727]: pgmap v207: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 1.88k op/s 2024-03-19T01:17:27.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:27 smithi163 ceph-mon[67428]: pgmap v208: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 1.78k op/s 2024-03-19T01:17:27.583 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:27 smithi069 ceph-mon[31727]: pgmap v208: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 1.78k op/s 2024-03-19T01:17:27.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:27 smithi086 ceph-mon[66829]: pgmap v208: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 1.78k op/s 2024-03-19T01:17:29.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:29 smithi086 ceph-mon[66829]: pgmap v209: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 1.65k op/s 2024-03-19T01:17:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:29 smithi069 ceph-mon[31727]: pgmap v209: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 1.65k op/s 2024-03-19T01:17:29.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:29 smithi163 ceph-mon[67428]: pgmap v209: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 1.65k op/s 2024-03-19T01:17:31.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:31 smithi086 ceph-mon[66829]: pgmap v210: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 696 KiB/s wr, 1.30k op/s 2024-03-19T01:17:31.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:31 smithi069 ceph-mon[31727]: pgmap v210: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 696 KiB/s wr, 1.30k op/s 2024-03-19T01:17:31.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:31 smithi163 ceph-mon[67428]: pgmap v210: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 696 KiB/s wr, 1.30k op/s 2024-03-19T01:17:32.225 INFO:tasks.workunit.client.0.smithi069.stdout:# 2024-03-19T01:17:32.225 INFO:tasks.workunit.client.0.smithi069.stdout:# configuration written to .config 2024-03-19T01:17:32.225 INFO:tasks.workunit.client.0.smithi069.stdout:# 2024-03-19T01:17:32.240 INFO:tasks.workunit.client.0.smithi069.stderr:++ grep -c processor /proc/cpuinfo 2024-03-19T01:17:32.246 INFO:tasks.workunit.client.0.smithi069.stderr:+ make -j8 2024-03-19T01:17:33.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:33 smithi086 ceph-mon[66829]: pgmap v211: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 1.40k op/s 2024-03-19T01:17:33.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:33 smithi069 ceph-mon[31727]: pgmap v211: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 1.40k op/s 2024-03-19T01:17:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:33 smithi163 ceph-mon[67428]: pgmap v211: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 1.40k op/s 2024-03-19T01:17:34.475 INFO:tasks.workunit.client.0.smithi069.stdout: GEN arch/x86/include/generated/asm/orc_hash.h 2024-03-19T01:17:34.475 INFO:tasks.workunit.client.0.smithi069.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_32.h 2024-03-19T01:17:34.476 INFO:tasks.workunit.client.0.smithi069.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_64.h 2024-03-19T01:17:34.477 INFO:tasks.workunit.client.0.smithi069.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_x32.h 2024-03-19T01:17:34.480 INFO:tasks.workunit.client.0.smithi069.stdout: SYSTBL arch/x86/include/generated/asm/syscalls_32.h 2024-03-19T01:17:34.517 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/bpf_perf_event.h 2024-03-19T01:17:34.520 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC arch/x86/tools/relocs_32.o 2024-03-19T01:17:34.525 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/errno.h 2024-03-19T01:17:34.531 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/fcntl.h 2024-03-19T01:17:34.538 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/ioctl.h 2024-03-19T01:17:34.545 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/ioctls.h 2024-03-19T01:17:34.551 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/ipcbuf.h 2024-03-19T01:17:34.555 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC arch/x86/tools/relocs_64.o 2024-03-19T01:17:34.558 INFO:tasks.workunit.client.0.smithi069.stdout: SYSHDR arch/x86/include/generated/asm/unistd_32_ia32.h 2024-03-19T01:17:34.560 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC arch/x86/tools/relocs_common.o 2024-03-19T01:17:34.560 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/param.h 2024-03-19T01:17:34.564 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/poll.h 2024-03-19T01:17:34.569 INFO:tasks.workunit.client.0.smithi069.stdout: SYSHDR arch/x86/include/generated/asm/unistd_64_x32.h 2024-03-19T01:17:34.572 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/resource.h 2024-03-19T01:17:34.579 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/socket.h 2024-03-19T01:17:34.589 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/sockios.h 2024-03-19T01:17:34.597 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/termbits.h 2024-03-19T01:17:34.605 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/termios.h 2024-03-19T01:17:34.612 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/uapi/asm/types.h 2024-03-19T01:17:34.626 INFO:tasks.workunit.client.0.smithi069.stdout: SYSTBL arch/x86/include/generated/asm/syscalls_64.h 2024-03-19T01:17:34.634 INFO:tasks.workunit.client.0.smithi069.stdout: UPD include/generated/uapi/linux/version.h 2024-03-19T01:17:34.641 INFO:tasks.workunit.client.0.smithi069.stdout: UPD include/config/kernel.release 2024-03-19T01:17:34.705 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/asm/early_ioremap.h 2024-03-19T01:17:34.712 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/asm/export.h 2024-03-19T01:17:34.718 INFO:tasks.workunit.client.0.smithi069.stdout: UPD include/generated/compile.h 2024-03-19T01:17:34.721 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/asm/mcs_spinlock.h 2024-03-19T01:17:34.728 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/asm/irq_regs.h 2024-03-19T01:17:34.735 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/asm/kmap_size.h 2024-03-19T01:17:34.742 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/asm/local64.h 2024-03-19T01:17:34.750 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/asm/mmiowb.h 2024-03-19T01:17:34.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:34 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:17:34.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:34 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:17:34.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:34 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:17:34.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:34 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:17:34.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:34 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:17:34.760 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/asm/module.lds.h 2024-03-19T01:17:34.767 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/asm/rwonce.h 2024-03-19T01:17:34.772 INFO:tasks.workunit.client.0.smithi069.stdout: WRAP arch/x86/include/generated/asm/unaligned.h 2024-03-19T01:17:34.780 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/selinux/genheaders/genheaders 2024-03-19T01:17:34.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:34 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:17:34.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:34 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:17:34.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:34 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:17:34.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:34 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:17:34.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:34 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:17:34.792 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/selinux/mdp/mdp 2024-03-19T01:17:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:34 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:17:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:34 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:17:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:34 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:17:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:34 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:17:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:34 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:17:35.016 INFO:tasks.workunit.client.0.smithi069.stdout: DESCEND objtool 2024-03-19T01:17:35.031 INFO:tasks.workunit.client.0.smithi069.stdout: UPD include/generated/utsrelease.h 2024-03-19T01:17:35.035 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/kallsyms 2024-03-19T01:17:35.045 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTLD arch/x86/tools/relocs 2024-03-19T01:17:35.045 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/sorttable 2024-03-19T01:17:35.045 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/asn1_compiler 2024-03-19T01:17:35.338 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep.o 2024-03-19T01:17:35.427 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTLD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep-in.o 2024-03-19T01:17:35.442 INFO:tasks.workunit.client.0.smithi069.stdout: LINK /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep 2024-03-19T01:17:35.539 INFO:tasks.workunit.client.0.smithi069.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/exec-cmd.h 2024-03-19T01:17:35.541 INFO:tasks.workunit.client.0.smithi069.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/help.h 2024-03-19T01:17:35.541 INFO:tasks.workunit.client.0.smithi069.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/pager.h 2024-03-19T01:17:35.542 INFO:tasks.workunit.client.0.smithi069.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/parse-options.h 2024-03-19T01:17:35.543 INFO:tasks.workunit.client.0.smithi069.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/run-command.h 2024-03-19T01:17:35.547 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/exec-cmd.o 2024-03-19T01:17:35.569 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/help.o 2024-03-19T01:17:35.572 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/pager.o 2024-03-19T01:17:35.573 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/parse-options.o 2024-03-19T01:17:35.581 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/run-command.o 2024-03-19T01:17:35.585 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/sigchain.o 2024-03-19T01:17:35.610 INFO:tasks.workunit.client.0.smithi069.stdout: INSTALL libsubcmd_headers 2024-03-19T01:17:35.615 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/subcmd-config.o 2024-03-19T01:17:35.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:35 smithi086 ceph-mon[66829]: pgmap v212: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 698 op/s 2024-03-19T01:17:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:35 smithi069 ceph-mon[31727]: pgmap v212: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 698 op/s 2024-03-19T01:17:35.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:35 smithi163 ceph-mon[67428]: pgmap v212: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 698 op/s 2024-03-19T01:17:36.219 INFO:tasks.workunit.client.0.smithi069.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/libsubcmd-in.o 2024-03-19T01:17:36.272 INFO:tasks.workunit.client.0.smithi069.stdout: AR /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/libsubcmd.a 2024-03-19T01:17:37.499 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/weak.o 2024-03-19T01:17:37.500 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/check.o 2024-03-19T01:17:37.502 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/special.o 2024-03-19T01:17:37.503 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/builtin-check.o 2024-03-19T01:17:37.505 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/elf.o 2024-03-19T01:17:37.509 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool.o 2024-03-19T01:17:37.514 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/special.o 2024-03-19T01:17:37.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:37 smithi086 ceph-mon[66829]: pgmap v213: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 586 op/s 2024-03-19T01:17:37.777 INFO:tasks.workunit.client.0.smithi069.stdout: MKDIR /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/lib/ 2024-03-19T01:17:37.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:37 smithi069 ceph-mon[31727]: pgmap v213: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 586 op/s 2024-03-19T01:17:37.800 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/orc_gen.o 2024-03-19T01:17:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:37 smithi163 ceph-mon[67428]: pgmap v213: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 586 op/s 2024-03-19T01:17:37.822 INFO:tasks.workunit.client.0.smithi069.stdout: GEN /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/lib/inat-tables.c 2024-03-19T01:17:37.837 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/orc_dump.o 2024-03-19T01:17:37.877 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libstring.o 2024-03-19T01:17:37.895 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/decode.o 2024-03-19T01:17:38.118 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libctype.o 2024-03-19T01:17:38.162 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/str_error_r.o 2024-03-19T01:17:38.173 INFO:tasks.workunit.client.0.smithi069.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/librbtree.o 2024-03-19T01:17:38.692 INFO:tasks.workunit.client.0.smithi069.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/objtool-in.o 2024-03-19T01:17:39.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:39 smithi086 ceph-mon[66829]: pgmap v214: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 637 op/s 2024-03-19T01:17:39.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:39 smithi069 ceph-mon[31727]: pgmap v214: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 637 op/s 2024-03-19T01:17:39.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:39 smithi163 ceph-mon[67428]: pgmap v214: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 637 op/s 2024-03-19T01:17:40.140 INFO:tasks.workunit.client.0.smithi069.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool-in.o 2024-03-19T01:17:40.177 INFO:tasks.workunit.client.0.smithi069.stdout: LINK /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool 2024-03-19T01:17:41.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:17:41.737 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:17:41.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:41 smithi086 ceph-mon[66829]: pgmap v215: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.8 MiB/s wr, 615 op/s 2024-03-19T01:17:41.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:41 smithi069 ceph-mon[31727]: pgmap v215: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.8 MiB/s wr, 615 op/s 2024-03-19T01:17:41.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:41 smithi163 ceph-mon[67428]: pgmap v215: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.8 MiB/s wr, 615 op/s 2024-03-19T01:17:41.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.969+0000 7fe5e6e00640 1 -- 172.21.15.69:0/4123963332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e010bdf0 msgr2=0x7fe5e010e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:41.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.969+0000 7fe5e6e00640 1 --2- 172.21.15.69:0/4123963332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e010bdf0 0x7fe5e010e260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009be0 tx=0x7fe5d802bf80 comp rx=0 tx=0).stop 2024-03-19T01:17:41.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.969+0000 7fe5e6e00640 1 -- 172.21.15.69:0/4123963332 shutdown_connections 2024-03-19T01:17:41.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.969+0000 7fe5e6e00640 1 --2- 172.21.15.69:0/4123963332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e010bdf0 0x7fe5e010e260 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.969+0000 7fe5e6e00640 1 --2- 172.21.15.69:0/4123963332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e01033d0 0x7fe5e010b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.969+0000 7fe5e6e00640 1 --2- 172.21.15.69:0/4123963332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e0102ab0 0x7fe5e0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.969+0000 7fe5e6e00640 1 -- 172.21.15.69:0/4123963332 >> 172.21.15.69:0/4123963332 conn(0x7fe5e0077880 msgr2=0x7fe5e0077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:41.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.969+0000 7fe5e6e00640 1 -- 172.21.15.69:0/4123963332 shutdown_connections 2024-03-19T01:17:41.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.969+0000 7fe5e6e00640 1 -- 172.21.15.69:0/4123963332 wait complete. 2024-03-19T01:17:41.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.971+0000 7fe5e6e00640 1 Processor -- start 2024-03-19T01:17:41.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.971+0000 7fe5e6e00640 1 -- start start 2024-03-19T01:17:41.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.971+0000 7fe5e6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e0102ab0 0x7fe5e019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:41.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.971+0000 7fe5e6e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e01033d0 0x7fe5e019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:41.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.971+0000 7fe5e6e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e010bdf0 0x7fe5e019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:41.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.971+0000 7fe5e6e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e0111c10 con 0x7fe5e01033d0 2024-03-19T01:17:41.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.971+0000 7fe5e6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e0111a90 con 0x7fe5e0102ab0 2024-03-19T01:17:41.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.971+0000 7fe5e6e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e0111d90 con 0x7fe5e010bdf0 2024-03-19T01:17:41.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e0102ab0 0x7fe5e019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:41.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e0102ab0 0x7fe5e019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40270/0 (socket says 172.21.15.69:40270) 2024-03-19T01:17:41.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5dfe00640 1 -- 172.21.15.69:0/1899887893 learned_addr learned my addr 172.21.15.69:0/1899887893 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:41.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5df400640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e01033d0 0x7fe5e019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:41.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5e4e00640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e010bdf0 0x7fe5e019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:41.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5dfe00640 1 -- 172.21.15.69:0/1899887893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e010bdf0 msgr2=0x7fe5e019ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:41.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e010bdf0 0x7fe5e019ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5dfe00640 1 -- 172.21.15.69:0/1899887893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e01033d0 msgr2=0x7fe5e019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:41.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e01033d0 0x7fe5e019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5dfe00640 1 -- 172.21.15.69:0/1899887893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e019f1c0 con 0x7fe5e0102ab0 2024-03-19T01:17:41.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5df400640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e01033d0 0x7fe5e019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:41.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.972+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e0102ab0 0x7fe5e019a1d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e9c0 tx=0x7fe5d000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:41.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.973+0000 7fe5dcc00640 1 -- 172.21.15.69:0/1899887893 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000ccc0 con 0x7fe5e0102ab0 2024-03-19T01:17:41.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.973+0000 7fe5dcc00640 1 -- 172.21.15.69:0/1899887893 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d0005500 con 0x7fe5e0102ab0 2024-03-19T01:17:41.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.973+0000 7fe5dcc00640 1 -- 172.21.15.69:0/1899887893 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0010430 con 0x7fe5e0102ab0 2024-03-19T01:17:41.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.973+0000 7fe5e6e00640 1 -- 172.21.15.69:0/1899887893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e019f4b0 con 0x7fe5e0102ab0 2024-03-19T01:17:41.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.973+0000 7fe5e6e00640 1 -- 172.21.15.69:0/1899887893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e01a6d40 con 0x7fe5e0102ab0 2024-03-19T01:17:41.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.976+0000 7fe5dcc00640 1 -- 172.21.15.69:0/1899887893 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fe5d00046c0 con 0x7fe5e0102ab0 2024-03-19T01:17:41.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.979+0000 7fe5dcc00640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5c40a29f0 0x7fe5c40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:41.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.979+0000 7fe5e9b7e740 1 Processor -- start 2024-03-19T01:17:41.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.979+0000 7fe5dcc00640 1 -- 172.21.15.69:0/1899887893 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7fe5d00cf4e0 con 0x7fe5e0102ab0 2024-03-19T01:17:41.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.979+0000 7fe5e9b7e740 1 -- start start 2024-03-19T01:17:41.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.979+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 0x55d722e41930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:41.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.980+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41e70 0x55d722e4a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:41.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.980+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 0x55d722e4ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:41.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.980+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d722d9d2c0 con 0x55d722e41550 2024-03-19T01:17:41.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.980+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d722d45b60 con 0x55d722e41e70 2024-03-19T01:17:41.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.980+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d722d99360 con 0x55d722e4aa50 2024-03-19T01:17:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.980+0000 7fe5e4e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 0x55d722e4ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.980+0000 7fe5e4e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 0x55d722e4ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44582/0 (socket says 172.21.15.69:44582) 2024-03-19T01:17:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.980+0000 7fe5e4e00640 1 -- 172.21.15.69:0/3479801841 learned_addr learned my addr 172.21.15.69:0/3479801841 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.980+0000 7fe5df400640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5c40a29f0 0x7fe5c40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.980+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/3479801841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 0x55d722e41930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.980+0000 7fe5df400640 1 --2- 172.21.15.69:0/3479801841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41e70 0x55d722e4a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.981+0000 7fe5e4e00640 1 -- 172.21.15.69:0/3479801841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41e70 msgr2=0x55d722e4a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.981+0000 7fe5e4e00640 1 --2- 172.21.15.69:0/3479801841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41e70 0x55d722e4a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.981+0000 7fe5e4e00640 1 -- 172.21.15.69:0/3479801841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 msgr2=0x55d722e41930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.981+0000 7fe5e4e00640 1 --2- 172.21.15.69:0/3479801841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 0x55d722e41930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.981+0000 7fe5e4e00640 1 -- 172.21.15.69:0/3479801841 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d722b7b7d0 con 0x55d722e4aa50 2024-03-19T01:17:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.981+0000 7fe5df400640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5c40a29f0 0x7fe5c40a4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc005940 tx=0x7fe5cc00b9b0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.981+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/3479801841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 0x55d722e41930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.981+0000 7fe5e4e00640 1 --2- 172.21.15.69:0/3479801841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 0x55d722e4ce40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009be0 tx=0x7fe5d8009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a853ef471bb73960 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.982+0000 7fe5d5600640 1 -- 172.21.15.69:0/3479801841 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8043030 con 0x55d722e4aa50 2024-03-19T01:17:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.982+0000 7fe5d5600640 1 -- 172.21.15.69:0/3479801841 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d802c920 con 0x55d722e4aa50 2024-03-19T01:17:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.982+0000 7fe5d5600640 1 -- 172.21.15.69:0/3479801841 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8004480 con 0x55d722e4aa50 2024-03-19T01:17:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.983+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3479801841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 msgr2=0x55d722e4ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.983+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/3479801841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 0x55d722e4ce40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009be0 tx=0x7fe5d8009910 comp rx=0 tx=0).stop 2024-03-19T01:17:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.984+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3479801841 shutdown_connections 2024-03-19T01:17:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.984+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/3479801841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 0x55d722e4ce40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.984+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/3479801841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41e70 0x55d722e4a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.984+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/3479801841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 0x55d722e41930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.984+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3479801841 >> 172.21.15.69:0/3479801841 conn(0x55d722e3bc80 msgr2=0x55d722e3c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.984+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3479801841 shutdown_connections 2024-03-19T01:17:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.985+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3479801841 wait complete. 2024-03-19T01:17:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.985+0000 7fe5e9b7e740 1 -- start start 2024-03-19T01:17:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.985+0000 7fe5e9b7e740 1 Processor -- start 2024-03-19T01:17:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.986+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 0x55d722e47af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.986+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41e70 0x55d722e48030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.986+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 0x55d722e48570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.986+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d722e4d7e0 con 0x55d722e41550 2024-03-19T01:17:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.986+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d722d58fc0 con 0x55d722e4aa50 2024-03-19T01:17:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.986+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d722e4d960 con 0x55d722e41e70 2024-03-19T01:17:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.986+0000 7fe5e9b7e740 10 client.0 _ll_register_callbacks cb 0x55d722d58990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.986+0000 7fe5df400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41e70 0x55d722e48030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.986+0000 7fe5df400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41e70 0x55d722e48030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44586/0 (socket says 172.21.15.69:44586) 2024-03-19T01:17:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.986+0000 7fe5df400640 1 -- 172.21.15.69:0/3392684803 learned_addr learned my addr 172.21.15.69:0/3392684803 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.986+0000 7fe5e4e00640 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 0x55d722e48570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.987+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 0x55d722e47af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.987+0000 7fe5df400640 1 -- 172.21.15.69:0/3392684803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 msgr2=0x55d722e48570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.987+0000 7fe5df400640 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 0x55d722e48570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.987+0000 7fe5df400640 1 -- 172.21.15.69:0/3392684803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 msgr2=0x55d722e47af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.987+0000 7fe5df400640 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 0x55d722e47af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.987+0000 7fe5df400640 1 -- 172.21.15.69:0/3392684803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d722ab13f0 con 0x55d722e41e70 2024-03-19T01:17:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.987+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 0x55d722e47af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.987+0000 7fe5e4e00640 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 0x55d722e48570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.988+0000 7fe5df400640 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41e70 0x55d722e48030 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00fbb0 tx=0x7fe5cc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be33734a4ee14353 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.988+0000 7fe5ca000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.988+0000 7fe5ca000640 1 -- 172.21.15.69:0/3392684803 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc028430 con 0x55d722e41e70 2024-03-19T01:17:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.988+0000 7fe5ca000640 1 -- 172.21.15.69:0/3392684803 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7fe5cc01f400 con 0x55d722e41e70 2024-03-19T01:17:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.988+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3392684803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d802c920 con 0x55d722e41e70 2024-03-19T01:17:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.988+0000 7fe5ca000640 1 -- 172.21.15.69:0/3392684803 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe5cc01f600 con 0x55d722e41e70 2024-03-19T01:17:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.988+0000 7fe5e9b7e740 10 client.27106 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.988+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3392684803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d722a710f0 con 0x55d722e41e70 2024-03-19T01:17:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.989+0000 7fe5ca000640 1 -- 172.21.15.69:0/3392684803 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d802c920 con 0x55d722e41e70 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.989+0000 7fe5ca000640 1 -- 172.21.15.69:0/3392684803 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7fe5cc028790 con 0x55d722e41e70 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.989+0000 7fe5e9b7e740 10 client.27106 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.989+0000 7fe5e9b7e740 10 client.27106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.989+0000 7fe5e9b7e740 20 client.27106 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.989+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d722e36f50 0x55d722e39410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.989+0000 7fe5e9b7e740 4 client.27106 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.989+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3392684803 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d722d3e610 con 0x55d722e36f50 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.990+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d722e36f50 0x55d722e39410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.990+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d722e36f50 0x55d722e39410 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a211eb040631e3 server_cookie=29c465a292c8369d in_seq=0 out_seq=0 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.990+0000 7fe5ca000640 10 client.27106 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5ca000640 1 -- 172.21.15.69:0/3392684803 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d722d3e610 con 0x55d722e36f50 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5ca000640 10 client.27106 handle_command_reply: tid=0 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5e9b7e740 1 client.27106 shutdown 2024-03-19T01:17:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3392684803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41e70 msgr2=0x55d722e48030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41e70 0x55d722e48030 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00fbb0 tx=0x7fe5cc004730 comp rx=0 tx=0).stop 2024-03-19T01:17:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3392684803 shutdown_connections 2024-03-19T01:17:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d722e36f50 0x55d722e39410 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 0x55d722e48570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41e70 0x55d722e48030 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/3392684803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e41550 0x55d722e47af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3392684803 >> 172.21.15.69:0/3392684803 conn(0x55d722e3bc80 msgr2=0x55d722e3d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.991+0000 7fe5ca000640 0 client.27106 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.992+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3392684803 shutdown_connections 2024-03-19T01:17:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.992+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/3392684803 wait complete. 2024-03-19T01:17:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:41.992+0000 7fe5e9b7e740 20 client.27106 trim_cache size 0 max 16384 2024-03-19T01:17:42.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.019+0000 7fe5e9b7e740 1 Processor -- start 2024-03-19T01:17:42.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.019+0000 7fe5e9b7e740 1 -- start start 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.019+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41550 0x55d722e4a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.019+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 0x55d722e565d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.019+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e46c50 0x55d722e481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.019+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01acbb0 con 0x55d722e46c50 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.019+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01ae8a0 con 0x55d722e4aa50 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.019+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01ad280 con 0x55d722e41550 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.020+0000 7fe5df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 0x55d722e565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.020+0000 7fe5df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 0x55d722e565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40304/0 (socket says 172.21.15.69:40304) 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.020+0000 7fe5df400640 1 -- 172.21.15.69:0/2932065271 learned_addr learned my addr 172.21.15.69:0/2932065271 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.020+0000 7fe5e4e00640 1 --2- 172.21.15.69:0/2932065271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e46c50 0x55d722e481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.020+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/2932065271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41550 0x55d722e4a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.020+0000 7fe5df400640 1 -- 172.21.15.69:0/2932065271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41550 msgr2=0x55d722e4a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:42.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.020+0000 7fe5df400640 1 --2- 172.21.15.69:0/2932065271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41550 0x55d722e4a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.020+0000 7fe5df400640 1 -- 172.21.15.69:0/2932065271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e46c50 msgr2=0x55d722e481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:42.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.020+0000 7fe5df400640 1 --2- 172.21.15.69:0/2932065271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e46c50 0x55d722e481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.020+0000 7fe5df400640 1 -- 172.21.15.69:0/2932065271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d722e41ca0 con 0x55d722e4aa50 2024-03-19T01:17:42.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.020+0000 7fe5df400640 1 --2- 172.21.15.69:0/2932065271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 0x55d722e565d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c010 tx=0x7fe5cc01efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbdf5f08dc3685a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:42.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.021+0000 7fe5d4c00640 1 -- 172.21.15.69:0/2932065271 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc03b070 con 0x55d722e4aa50 2024-03-19T01:17:42.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.021+0000 7fe5d4c00640 1 -- 172.21.15.69:0/2932065271 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5cc00b3e0 con 0x55d722e4aa50 2024-03-19T01:17:42.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.021+0000 7fe5d4c00640 1 -- 172.21.15.69:0/2932065271 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc005280 con 0x55d722e4aa50 2024-03-19T01:17:42.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.022+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/2932065271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 msgr2=0x55d722e565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.022+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/2932065271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 0x55d722e565d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c010 tx=0x7fe5cc01efa0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.022+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/2932065271 shutdown_connections 2024-03-19T01:17:42.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.022+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/2932065271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e46c50 0x55d722e481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.022+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/2932065271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e4aa50 0x55d722e565d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.022+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/2932065271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e41550 0x55d722e4a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.022+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/2932065271 >> 172.21.15.69:0/2932065271 conn(0x55d722e3bc80 msgr2=0x55d722e47ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:42.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.022+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/2932065271 shutdown_connections 2024-03-19T01:17:42.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.022+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/2932065271 wait complete. 2024-03-19T01:17:42.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.023+0000 7fe5e9b7e740 1 -- start start 2024-03-19T01:17:42.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.023+0000 7fe5e9b7e740 1 Processor -- start 2024-03-19T01:17:42.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.023+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41550 0x55d722e52c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e46c50 0x55d722e4dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5e9b7e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 0x55d722e4e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01ae110 con 0x55d722e46c50 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01ada10 con 0x55d722e41550 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5e9b7e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01aab20 con 0x55d722e4aa50 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5e9b7e740 10 client.0 _ll_register_callbacks cb 0x7fe5cc00f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41550 0x55d722e52c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41550 0x55d722e52c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40308/0 (socket says 172.21.15.69:40308) 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5dfe00640 1 -- 172.21.15.69:0/1125295279 learned_addr learned my addr 172.21.15.69:0/1125295279 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5e4e00640 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 0x55d722e4e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5df400640 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e46c50 0x55d722e4dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5dfe00640 1 -- 172.21.15.69:0/1125295279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 msgr2=0x55d722e4e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 0x55d722e4e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5dfe00640 1 -- 172.21.15.69:0/1125295279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e46c50 msgr2=0x55d722e4dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:42.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e46c50 0x55d722e4dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.024+0000 7fe5dfe00640 1 -- 172.21.15.69:0/1125295279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d722e53300 con 0x55d722e41550 2024-03-19T01:17:42.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.025+0000 7fe5df400640 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e46c50 0x55d722e4dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:42.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.025+0000 7fe5dfe00640 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41550 0x55d722e52c10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe5d009b4d0 tx=0x7fe5d0086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919987bba6d912f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:42.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.025+0000 7fe5ca000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:17:42.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.025+0000 7fe5ca000640 1 -- 172.21.15.69:0/1125295279 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00b8020 con 0x55d722e41550 2024-03-19T01:17:42.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.025+0000 7fe5ca000640 1 -- 172.21.15.69:0/1125295279 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7fe5d00ccb40 con 0x55d722e41550 2024-03-19T01:17:42.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.026+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/1125295279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d722daa800 con 0x55d722e41550 2024-03-19T01:17:42.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.026+0000 7fe5ca000640 1 -- 172.21.15.69:0/1125295279 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe5d009f920 con 0x55d722e41550 2024-03-19T01:17:42.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.026+0000 7fe5e9b7e740 10 client.27167 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.026+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/1125295279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d722ec9e80 con 0x55d722e41550 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.026+0000 7fe5ca000640 1 -- 172.21.15.69:0/1125295279 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d722daa800 con 0x55d722e41550 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.026+0000 7fe5ca000640 1 -- 172.21.15.69:0/1125295279 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7fe5d00a3ae0 con 0x55d722e41550 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.027+0000 7fe5e9b7e740 10 client.27167 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.027+0000 7fe5e9b7e740 10 client.27167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.027+0000 7fe5e9b7e740 20 client.27167 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.027+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d722eca5f0 0x55d722ecca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.027+0000 7fe5e9b7e740 4 client.27167 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.027+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/1125295279 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d722da4b50 con 0x55d722eca5f0 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.027+0000 7fe5df400640 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d722eca5f0 0x55d722ecca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.028+0000 7fe5df400640 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d722eca5f0 0x55d722ecca10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a046bec48da91 server_cookie=3333bd00c6acdd1c in_seq=0 out_seq=0 2024-03-19T01:17:42.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.028+0000 7fe5ca000640 10 client.27167 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:42.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5ca000640 1 -- 172.21.15.69:0/1125295279 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d722da4b50 con 0x55d722eca5f0 2024-03-19T01:17:42.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5ca000640 10 client.27167 handle_command_reply: tid=0 2024-03-19T01:17:42.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 1 client.27167 shutdown 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/1125295279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41550 msgr2=0x55d722e52c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41550 0x55d722e52c10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe5d009b4d0 tx=0x7fe5d0086040 comp rx=0 tx=0).stop 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/1125295279 shutdown_connections 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d722eca5f0 0x55d722ecca10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d722e4aa50 0x55d722e4e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d722e46c50 0x55d722e4dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 1 --2- 172.21.15.69:0/1125295279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d722e41550 0x55d722e52c10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5ca000640 0 client.27167 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/1125295279 >> 172.21.15.69:0/1125295279 conn(0x55d722e3bc80 msgr2=0x55d722e48340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/1125295279 shutdown_connections 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 1 -- 172.21.15.69:0/1125295279 wait complete. 2024-03-19T01:17:42.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.029+0000 7fe5e9b7e740 20 client.27167 trim_cache size 0 max 16384 2024-03-19T01:17:42.032 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:17:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:17:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:17:42.032 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.030+0000 7fe5e6e00640 1 -- 172.21.15.69:0/1899887893 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5c40a29f0 msgr2=0x7fe5c40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.030+0000 7fe5e6e00640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5c40a29f0 0x7fe5c40a4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc005940 tx=0x7fe5cc00b9b0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.030+0000 7fe5e6e00640 1 -- 172.21.15.69:0/1899887893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e0102ab0 msgr2=0x7fe5e019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.030+0000 7fe5e6e00640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e0102ab0 0x7fe5e019a1d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e9c0 tx=0x7fe5d000ee80 comp rx=0 tx=0).stop 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.030+0000 7fe5e6e00640 1 -- 172.21.15.69:0/1899887893 shutdown_connections 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.030+0000 7fe5e6e00640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e010bdf0 0x7fe5e019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.030+0000 7fe5e6e00640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5e01033d0 0x7fe5e019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.030+0000 7fe5e6e00640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5c40a29f0 0x7fe5c40a4eb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.030+0000 7fe5e6e00640 1 --2- 172.21.15.69:0/1899887893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e0102ab0 0x7fe5e019a1d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.030+0000 7fe5e6e00640 1 -- 172.21.15.69:0/1899887893 >> 172.21.15.69:0/1899887893 conn(0x7fe5e0077880 msgr2=0x7fe5e00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.031+0000 7fe5e6e00640 1 -- 172.21.15.69:0/1899887893 shutdown_connections 2024-03-19T01:17:42.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.031+0000 7fe5e6e00640 1 -- 172.21.15.69:0/1899887893 wait complete. 2024-03-19T01:17:42.052 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:17:42.052 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:17:42.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.321+0000 7f2e4a200640 1 -- 172.21.15.69:0/3533000659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e441076b0 msgr2=0x7f2e44107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.321+0000 7f2e4a200640 1 --2- 172.21.15.69:0/3533000659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e441076b0 0x7f2e44107a90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009930 tx=0x7f2e3802bc20 comp rx=0 tx=0).stop 2024-03-19T01:17:42.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.321+0000 7f2e4a200640 1 -- 172.21.15.69:0/3533000659 shutdown_connections 2024-03-19T01:17:42.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.321+0000 7f2e4a200640 1 --2- 172.21.15.69:0/3533000659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44101fd0 0x7f2e4410e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.321+0000 7f2e4a200640 1 --2- 172.21.15.69:0/3533000659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101610 0x7f2e44101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.321+0000 7f2e4a200640 1 --2- 172.21.15.69:0/3533000659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e441076b0 0x7f2e44107a90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.321+0000 7f2e4a200640 1 -- 172.21.15.69:0/3533000659 >> 172.21.15.69:0/3533000659 conn(0x7f2e440fd100 msgr2=0x7f2e440ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:42.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.322+0000 7f2e4a200640 1 -- 172.21.15.69:0/3533000659 shutdown_connections 2024-03-19T01:17:42.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.322+0000 7f2e4a200640 1 -- 172.21.15.69:0/3533000659 wait complete. 2024-03-19T01:17:42.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.323+0000 7f2e4a200640 1 Processor -- start 2024-03-19T01:17:42.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.323+0000 7f2e4a200640 1 -- start start 2024-03-19T01:17:42.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.323+0000 7f2e4a200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e44101610 0x7f2e44073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.323+0000 7f2e4a200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101fd0 0x7f2e44074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.323+0000 7f2e4a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e441076b0 0x7f2e440745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.323+0000 7f2e4a200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44111a40 con 0x7f2e44101fd0 2024-03-19T01:17:42.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.323+0000 7f2e4a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441118c0 con 0x7f2e441076b0 2024-03-19T01:17:42.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.323+0000 7f2e4a200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44111bc0 con 0x7f2e44101610 2024-03-19T01:17:42.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.323+0000 7f2e48e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e44101610 0x7f2e44073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.324+0000 7f2e48e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e44101610 0x7f2e44073b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44628/0 (socket says 172.21.15.69:44628) 2024-03-19T01:17:42.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.324+0000 7f2e48e00640 1 -- 172.21.15.69:0/4229946781 learned_addr learned my addr 172.21.15.69:0/4229946781 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:42.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.324+0000 7f2e48e00640 1 -- 172.21.15.69:0/4229946781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e441076b0 msgr2=0x7f2e440745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.324+0000 7f2e48e00640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e441076b0 0x7f2e440745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.324+0000 7f2e48e00640 1 -- 172.21.15.69:0/4229946781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101fd0 msgr2=0x7f2e44074060 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-19T01:17:42.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.324+0000 7f2e43e00640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101fd0 0x7f2e44074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.324+0000 7f2e48e00640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101fd0 0x7f2e44074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.324+0000 7f2e48e00640 1 -- 172.21.15.69:0/4229946781 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e441a6420 con 0x7f2e44101610 2024-03-19T01:17:42.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.325+0000 7f2e48e00640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e44101610 0x7f2e44073b20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005ab0 tx=0x7f2e380042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:42.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.325+0000 7f2e41600640 1 -- 172.21.15.69:0/4229946781 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38043020 con 0x7f2e44101610 2024-03-19T01:17:42.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.325+0000 7f2e41600640 1 -- 172.21.15.69:0/4229946781 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e380044d0 con 0x7f2e44101610 2024-03-19T01:17:42.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.325+0000 7f2e41600640 1 -- 172.21.15.69:0/4229946781 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38003b50 con 0x7f2e44101610 2024-03-19T01:17:42.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.325+0000 7f2e4a200640 1 -- 172.21.15.69:0/4229946781 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e441a66b0 con 0x7f2e44101610 2024-03-19T01:17:42.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.325+0000 7f2e4a200640 1 -- 172.21.15.69:0/4229946781 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a6bb0 con 0x7f2e44101610 2024-03-19T01:17:42.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.327+0000 7f2e4a200640 1 -- 172.21.15.69:0/4229946781 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e44068530 con 0x7f2e44101610 2024-03-19T01:17:42.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.329+0000 7f2e41600640 1 -- 172.21.15.69:0/4229946781 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f2e38046020 con 0x7f2e44101610 2024-03-19T01:17:42.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.331+0000 7f2e41600640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e2c0a2a20 0x7f2e2c0a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.331+0000 7f2e41600640 1 -- 172.21.15.69:0/4229946781 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f2e380f16e0 con 0x7f2e44101610 2024-03-19T01:17:42.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.332+0000 7f2e43e00640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e2c0a2a20 0x7f2e2c0a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.332+0000 7f2e41600640 1 -- 172.21.15.69:0/4229946781 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2e3804d020 con 0x7f2e44101610 2024-03-19T01:17:42.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.333+0000 7f2e43e00640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e2c0a2a20 0x7f2e2c0a4ee0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2e30005fd0 tx=0x7f2e30005850 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:42.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.666+0000 7f2e4a200640 1 -- 172.21.15.69:0/4229946781 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e44075000 con 0x7f2e44101610 2024-03-19T01:17:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.667+0000 7f2e41600640 1 -- 172.21.15.69:0/4229946781 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11957 (secure 0 0 0) 0x7f2e380be100 con 0x7f2e44101610 2024-03-19T01:17:42.668 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:17:42.668 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:17:12.906962+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24550,24494,24580,24581,24476]},"id":1}]} 2024-03-19T01:17:42.668 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 45 2024-03-19T01:17:42.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 -- 172.21.15.69:0/4229946781 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e2c0a2a20 msgr2=0x7f2e2c0a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e2c0a2a20 0x7f2e2c0a4ee0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2e30005fd0 tx=0x7f2e30005850 comp rx=0 tx=0).stop 2024-03-19T01:17:42.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 -- 172.21.15.69:0/4229946781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e44101610 msgr2=0x7f2e44073b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e44101610 0x7f2e44073b20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005ab0 tx=0x7f2e380042e0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 -- 172.21.15.69:0/4229946781 shutdown_connections 2024-03-19T01:17:42.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e2c0a2a20 0x7f2e2c0a4ee0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e441076b0 0x7f2e440745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44101fd0 0x7f2e44074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 --2- 172.21.15.69:0/4229946781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e44101610 0x7f2e44073b20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 -- 172.21.15.69:0/4229946781 >> 172.21.15.69:0/4229946781 conn(0x7f2e440fd100 msgr2=0x7f2e440fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 -- 172.21.15.69:0/4229946781 shutdown_connections 2024-03-19T01:17:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.670+0000 7f2e4a200640 1 -- 172.21.15.69:0/4229946781 wait complete. 2024-03-19T01:17:42.692 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:17:42.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.937+0000 7ffaa1400640 1 -- 172.21.15.69:0/3568333208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c1099d0 msgr2=0x7ffa9c109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.937+0000 7ffaa1400640 1 --2- 172.21.15.69:0/3568333208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c1099d0 0x7ffa9c109db0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009900 tx=0x7ffa8c02bc50 comp rx=0 tx=0).stop 2024-03-19T01:17:42.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.937+0000 7ffaa1400640 1 -- 172.21.15.69:0/3568333208 shutdown_connections 2024-03-19T01:17:42.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.937+0000 7ffaa1400640 1 --2- 172.21.15.69:0/3568333208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa9c104380 0x7ffa9c110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.937+0000 7ffaa1400640 1 --2- 172.21.15.69:0/3568333208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa9c1039c0 0x7ffa9c103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.937+0000 7ffaa1400640 1 --2- 172.21.15.69:0/3568333208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c1099d0 0x7ffa9c109db0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.937+0000 7ffaa1400640 1 -- 172.21.15.69:0/3568333208 >> 172.21.15.69:0/3568333208 conn(0x7ffa9c0ff4d0 msgr2=0x7ffa9c1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:42.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.938+0000 7ffaa1400640 1 -- 172.21.15.69:0/3568333208 shutdown_connections 2024-03-19T01:17:42.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.938+0000 7ffaa1400640 1 -- 172.21.15.69:0/3568333208 wait complete. 2024-03-19T01:17:42.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.939+0000 7ffaa1400640 1 Processor -- start 2024-03-19T01:17:42.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.939+0000 7ffaa1400640 1 -- start start 2024-03-19T01:17:42.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.939+0000 7ffaa1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa9c1039c0 0x7ffa9c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.939+0000 7ffaa1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa9c104380 0x7ffa9c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.939+0000 7ffaa1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c1099d0 0x7ffa9c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.939+0000 7ffaa1400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c113e30 con 0x7ffa9c104380 2024-03-19T01:17:42.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.939+0000 7ffaa1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c113cb0 con 0x7ffa9c1099d0 2024-03-19T01:17:42.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.939+0000 7ffaa1400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c113fb0 con 0x7ffa9c1039c0 2024-03-19T01:17:42.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.939+0000 7ffa9a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa9c1039c0 0x7ffa9c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.940+0000 7ffa9a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa9c1039c0 0x7ffa9c19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44654/0 (socket says 172.21.15.69:44654) 2024-03-19T01:17:42.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.940+0000 7ffa9a800640 1 -- 172.21.15.69:0/564385876 learned_addr learned my addr 172.21.15.69:0/564385876 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.940+0000 7ffa99e00640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa9c104380 0x7ffa9c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.940+0000 7ffa9b200640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c1099d0 0x7ffa9c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.940+0000 7ffa9a800640 1 -- 172.21.15.69:0/564385876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c1099d0 msgr2=0x7ffa9c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.940+0000 7ffa9a800640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c1099d0 0x7ffa9c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.940+0000 7ffa9a800640 1 -- 172.21.15.69:0/564385876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa9c104380 msgr2=0x7ffa9c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.940+0000 7ffa9a800640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa9c104380 0x7ffa9c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.940+0000 7ffa9a800640 1 -- 172.21.15.69:0/564385876 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9c19f2c0 con 0x7ffa9c1039c0 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.940+0000 7ffa99e00640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa9c104380 0x7ffa9c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.940+0000 7ffa9a800640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa9c1039c0 0x7ffa9c19a2d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c005ab0 tx=0x7ffa8c02e980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.941+0000 7ffa93400640 1 -- 172.21.15.69:0/564385876 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c041020 con 0x7ffa9c1039c0 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.941+0000 7ffa93400640 1 -- 172.21.15.69:0/564385876 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa8c02ed80 con 0x7ffa9c1039c0 2024-03-19T01:17:42.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.941+0000 7ffa93400640 1 -- 172.21.15.69:0/564385876 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c02ce60 con 0x7ffa9c1039c0 2024-03-19T01:17:42.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.941+0000 7ffaa1400640 1 -- 172.21.15.69:0/564385876 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9c19f550 con 0x7ffa9c1039c0 2024-03-19T01:17:42.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.941+0000 7ffaa1400640 1 -- 172.21.15.69:0/564385876 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa9c1a6e40 con 0x7ffa9c1039c0 2024-03-19T01:17:42.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.945+0000 7ffaa400a740 1 Processor -- start 2024-03-19T01:17:42.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.946+0000 7ffaa400a740 1 -- start start 2024-03-19T01:17:42.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.946+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff1830 0x561665ff1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.946+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x5616660031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.946+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561666003730 0x561666005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.946+0000 7ffaa400a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561665f56380 con 0x561665ff21e0 2024-03-19T01:17:42.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.946+0000 7ffaa400a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561665efed20 con 0x561666003730 2024-03-19T01:17:42.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.946+0000 7ffaa400a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561665f52420 con 0x561665ff1830 2024-03-19T01:17:42.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.946+0000 7ffa93400640 1 -- 172.21.15.69:0/564385876 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ffa8c044020 con 0x7ffa9c1039c0 2024-03-19T01:17:42.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.947+0000 7ffa9b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff1830 0x561665ff1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.947+0000 7ffa9a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x5616660031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.947+0000 7ffa9b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff1830 0x561665ff1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44666/0 (socket says 172.21.15.69:44666) 2024-03-19T01:17:42.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.947+0000 7ffa9b200640 1 -- 172.21.15.69:0/2163721280 learned_addr learned my addr 172.21.15.69:0/2163721280 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:42.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.947+0000 7ffa99e00640 1 --2- 172.21.15.69:0/2163721280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561666003730 0x561666005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.947+0000 7ffa9b200640 1 -- 172.21.15.69:0/2163721280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561666003730 msgr2=0x561666005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:42.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.948+0000 7ffa93400640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa800a2a70 0x7ffa800a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.948+0000 7ffa93400640 1 -- 172.21.15.69:0/564385876 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7ffa8c0f0190 con 0x7ffa9c1039c0 2024-03-19T01:17:42.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.948+0000 7ffa9b200640 1 --2- 172.21.15.69:0/2163721280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561666003730 0x561666005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.949+0000 7ffa9b200640 1 -- 172.21.15.69:0/2163721280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 msgr2=0x5616660031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:42.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.949+0000 7ffa9b200640 1 --2- 172.21.15.69:0/2163721280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x5616660031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.949+0000 7ffa9b200640 1 -- 172.21.15.69:0/2163721280 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561665c6a3f0 con 0x561665ff1830 2024-03-19T01:17:42.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.949+0000 7ffa9a800640 1 --2- 172.21.15.69:0/2163721280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x5616660031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:42.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.949+0000 7ffa99e00640 1 --2- 172.21.15.69:0/2163721280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561666003730 0x561666005b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:42.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.949+0000 7ffa9b200640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa800a2a70 0x7ffa800a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.949+0000 7ffa9b200640 1 --2- 172.21.15.69:0/2163721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff1830 0x561665ff1c10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009f70 tx=0x7ffa94010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb132bdc630056ea server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:42.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.950+0000 7ffa87400640 1 -- 172.21.15.69:0/2163721280 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94022070 con 0x561665ff1830 2024-03-19T01:17:42.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.950+0000 7ffa87400640 1 -- 172.21.15.69:0/2163721280 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa9401c450 con 0x561665ff1830 2024-03-19T01:17:42.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.950+0000 7ffa87400640 1 -- 172.21.15.69:0/2163721280 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9401b6e0 con 0x561665ff1830 2024-03-19T01:17:42.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.951+0000 7ffaa400a740 1 -- 172.21.15.69:0/2163721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff1830 msgr2=0x561665ff1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.951+0000 7ffa9b200640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa800a2a70 0x7ffa800a4f30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa9401f000 tx=0x7ffa9401bd90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:42.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.951+0000 7ffaa400a740 1 --2- 172.21.15.69:0/2163721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff1830 0x561665ff1c10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009f70 tx=0x7ffa94010860 comp rx=0 tx=0).stop 2024-03-19T01:17:42.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.951+0000 7ffaa400a740 1 -- 172.21.15.69:0/2163721280 shutdown_connections 2024-03-19T01:17:42.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.951+0000 7ffaa400a740 1 --2- 172.21.15.69:0/2163721280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561666003730 0x561666005b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.951+0000 7ffaa400a740 1 --2- 172.21.15.69:0/2163721280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x5616660031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.951+0000 7ffaa400a740 1 --2- 172.21.15.69:0/2163721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff1830 0x561665ff1c10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.951+0000 7ffaa400a740 1 -- 172.21.15.69:0/2163721280 >> 172.21.15.69:0/2163721280 conn(0x561665fec150 msgr2=0x561665fec530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:42.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.951+0000 7ffaa400a740 1 -- 172.21.15.69:0/2163721280 shutdown_connections 2024-03-19T01:17:42.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.951+0000 7ffaa400a740 1 -- 172.21.15.69:0/2163721280 wait complete. 2024-03-19T01:17:42.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.952+0000 7ffaa400a740 1 -- start start 2024-03-19T01:17:42.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.952+0000 7ffaa400a740 1 Processor -- start 2024-03-19T01:17:42.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff1830 0x5616660007a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff21e0 0x561666000ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561666003730 0x561666001220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffaa400a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5616660064f0 con 0x561665ff1830 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffaa400a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561665f12160 con 0x561666003730 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffaa400a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561666006670 con 0x561665ff21e0 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffaa400a740 10 client.0 _ll_register_callbacks cb 0x561665f4e170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffa99e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff21e0 0x561666000ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffa99e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff21e0 0x561666000ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44668/0 (socket says 172.21.15.69:44668) 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffa99e00640 1 -- 172.21.15.69:0/3538696016 learned_addr learned my addr 172.21.15.69:0/3538696016 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffa9a800640 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff1830 0x5616660007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffa99e00640 1 -- 172.21.15.69:0/3538696016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561666003730 msgr2=0x561666001220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffa99e00640 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561666003730 0x561666001220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffa99e00640 1 -- 172.21.15.69:0/3538696016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff1830 msgr2=0x5616660007a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffa99e00640 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff1830 0x5616660007a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffa99e00640 1 -- 172.21.15.69:0/3538696016 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561665d347d0 con 0x561665ff21e0 2024-03-19T01:17:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffa99e00640 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff21e0 0x561666000ce0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffa880137c0 tx=0x7ffa88013c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c84ee6c4497544 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:42.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.953+0000 7ffa7be00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:42.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.954+0000 7ffa7be00640 1 -- 172.21.15.69:0/3538696016 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88011dd0 con 0x561665ff21e0 2024-03-19T01:17:42.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.954+0000 7ffa7be00640 1 -- 172.21.15.69:0/3538696016 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7ffa88005020 con 0x561665ff21e0 2024-03-19T01:17:42.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.954+0000 7ffaa400a740 1 -- 172.21.15.69:0/3538696016 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa9401c450 con 0x561665ff21e0 2024-03-19T01:17:42.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.954+0000 7ffa7be00640 1 -- 172.21.15.69:0/3538696016 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ffa88026050 con 0x561665ff21e0 2024-03-19T01:17:42.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.954+0000 7ffaa400a740 10 client.27130 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:42.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.954+0000 7ffaa400a740 1 -- 172.21.15.69:0/3538696016 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561666012900 con 0x561665ff21e0 2024-03-19T01:17:42.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.954+0000 7ffa7be00640 1 -- 172.21.15.69:0/3538696016 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa9401c450 con 0x561665ff21e0 2024-03-19T01:17:42.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.955+0000 7ffa7be00640 1 -- 172.21.15.69:0/3538696016 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7ffa8801a030 con 0x561665ff21e0 2024-03-19T01:17:42.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.955+0000 7ffaa400a740 10 client.27130 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:42.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.955+0000 7ffaa400a740 10 client.27130 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:17:42.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.955+0000 7ffaa400a740 20 client.27130 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:42.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.955+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561666013e80 0x5616660162c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.956+0000 7ffaa400a740 4 client.27130 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:42.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.956+0000 7ffaa400a740 1 -- 172.21.15.69:0/3538696016 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561665f638c0 con 0x561666013e80 2024-03-19T01:17:42.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.956+0000 7ffa9a800640 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561666013e80 0x5616660162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.956+0000 7ffa9a800640 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561666013e80 0x5616660162c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=68b6f97be6904580 server_cookie=764ca501ea997a2b in_seq=0 out_seq=0 2024-03-19T01:17:42.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.956+0000 7ffa7be00640 10 client.27130 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:17:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.957+0000 7ffa7be00640 1 -- 172.21.15.69:0/3538696016 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561665f638c0 con 0x561666013e80 2024-03-19T01:17:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.957+0000 7ffa7be00640 10 client.27130 handle_command_reply: tid=0 2024-03-19T01:17:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.957+0000 7ffaa400a740 1 client.27130 shutdown 2024-03-19T01:17:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.957+0000 7ffaa400a740 1 -- 172.21.15.69:0/3538696016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff21e0 msgr2=0x561666000ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.957+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff21e0 0x561666000ce0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffa880137c0 tx=0x7ffa88013c80 comp rx=0 tx=0).stop 2024-03-19T01:17:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.958+0000 7ffaa400a740 1 -- 172.21.15.69:0/3538696016 shutdown_connections 2024-03-19T01:17:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.958+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561666013e80 0x5616660162c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.958+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561666003730 0x561666001220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.958+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561665ff21e0 0x561666000ce0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.958+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3538696016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff1830 0x5616660007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.958+0000 7ffaa400a740 1 -- 172.21.15.69:0/3538696016 >> 172.21.15.69:0/3538696016 conn(0x561665fec150 msgr2=0x561665fedea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:42.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.958+0000 7ffa7be00640 0 client.27130 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:17:42.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.958+0000 7ffaa400a740 1 -- 172.21.15.69:0/3538696016 shutdown_connections 2024-03-19T01:17:42.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.958+0000 7ffaa400a740 1 -- 172.21.15.69:0/3538696016 wait complete. 2024-03-19T01:17:42.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.958+0000 7ffaa400a740 20 client.27130 trim_cache size 0 max 16384 2024-03-19T01:17:42.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.977+0000 7ffaa400a740 1 Processor -- start 2024-03-19T01:17:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.978+0000 7ffaa400a740 1 -- start start 2024-03-19T01:17:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.978+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x561666002ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.978+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x561665fff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.978+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561666003730 0x56166600f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.978+0000 7ffaa400a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1acda0 con 0x561665ff21e0 2024-03-19T01:17:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.978+0000 7ffaa400a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1aea90 con 0x561665ff1830 2024-03-19T01:17:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.978+0000 7ffaa400a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1ad470 con 0x561666003730 2024-03-19T01:17:42.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.978+0000 7ffa9a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x561666002ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.978+0000 7ffa9a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x561666002ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40382/0 (socket says 172.21.15.69:40382) 2024-03-19T01:17:42.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.978+0000 7ffa99e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x561665fff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.978+0000 7ffa9a800640 1 -- 172.21.15.69:0/542661801 learned_addr learned my addr 172.21.15.69:0/542661801 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:42.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.979+0000 7ffa9b200640 1 --2- 172.21.15.69:0/542661801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561666003730 0x56166600f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.979+0000 7ffa99e00640 1 -- 172.21.15.69:0/542661801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561666003730 msgr2=0x56166600f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:42.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.979+0000 7ffa99e00640 1 --2- 172.21.15.69:0/542661801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561666003730 0x56166600f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.979+0000 7ffa99e00640 1 -- 172.21.15.69:0/542661801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 msgr2=0x561666002ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:42.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.979+0000 7ffa99e00640 1 --2- 172.21.15.69:0/542661801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x561666002ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.979+0000 7ffa99e00640 1 -- 172.21.15.69:0/542661801 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561665ff1f80 con 0x561665ff21e0 2024-03-19T01:17:42.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.979+0000 7ffa9a800640 1 --2- 172.21.15.69:0/542661801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x561666002ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:42.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.979+0000 7ffa99e00640 1 --2- 172.21.15.69:0/542661801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x561665fff900 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800f450 tx=0x7ffa88011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9cd35e717fd85c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:42.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.979+0000 7ffa86a00640 1 -- 172.21.15.69:0/542661801 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88011bd0 con 0x561665ff21e0 2024-03-19T01:17:42.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.980+0000 7ffa86a00640 1 -- 172.21.15.69:0/542661801 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa88011d70 con 0x561665ff21e0 2024-03-19T01:17:42.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.980+0000 7ffa86a00640 1 -- 172.21.15.69:0/542661801 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88005680 con 0x561665ff21e0 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.980+0000 7ffaa400a740 1 -- 172.21.15.69:0/542661801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 msgr2=0x561665fff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.980+0000 7ffaa400a740 1 --2- 172.21.15.69:0/542661801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x561665fff900 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800f450 tx=0x7ffa88011880 comp rx=0 tx=0).stop 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.980+0000 7ffaa400a740 1 -- 172.21.15.69:0/542661801 shutdown_connections 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.980+0000 7ffaa400a740 1 --2- 172.21.15.69:0/542661801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561666003730 0x56166600f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.980+0000 7ffaa400a740 1 --2- 172.21.15.69:0/542661801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x561665fff900 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.980+0000 7ffaa400a740 1 --2- 172.21.15.69:0/542661801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x561666002ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.980+0000 7ffaa400a740 1 -- 172.21.15.69:0/542661801 >> 172.21.15.69:0/542661801 conn(0x561665fec150 msgr2=0x561665fed4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.981+0000 7ffaa400a740 1 -- 172.21.15.69:0/542661801 shutdown_connections 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.981+0000 7ffaa400a740 1 -- 172.21.15.69:0/542661801 wait complete. 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.981+0000 7ffaa400a740 1 -- start start 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.981+0000 7ffaa400a740 1 Processor -- start 2024-03-19T01:17:42.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x5616660071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x561666007720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffaa400a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561666003730 0x561666007c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffaa400a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1ae300 con 0x561665ff21e0 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffaa400a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1adc00 con 0x561665ff1830 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffaa400a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1aacf0 con 0x561666003730 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffaa400a740 10 client.0 _ll_register_callbacks cb 0x7ffa88009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffa9a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x5616660071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffa9a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x5616660071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40390/0 (socket says 172.21.15.69:40390) 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffa9a800640 1 -- 172.21.15.69:0/3647081688 learned_addr learned my addr 172.21.15.69:0/3647081688 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffa9b200640 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561666003730 0x561666007c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffa99e00640 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x561666007720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffa9a800640 1 -- 172.21.15.69:0/3647081688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561666003730 msgr2=0x561666007c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffa9a800640 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561666003730 0x561666007c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffa9a800640 1 -- 172.21.15.69:0/3647081688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 msgr2=0x561666007720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffa9a800640 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x561666007720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.982+0000 7ffa9a800640 1 -- 172.21.15.69:0/3647081688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561666079200 con 0x561665ff1830 2024-03-19T01:17:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.983+0000 7ffa9a800640 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x5616660071e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c02c230 tx=0x7ffa8c02bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e6e743db1700da server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.983+0000 7ffa7be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:17:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.983+0000 7ffa7be00640 1 -- 172.21.15.69:0/3647081688 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0bbe10 con 0x561665ff1830 2024-03-19T01:17:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.983+0000 7ffa7be00640 1 -- 172.21.15.69:0/3647081688 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7ffa8c0ed710 con 0x561665ff1830 2024-03-19T01:17:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.983+0000 7ffaa400a740 1 -- 172.21.15.69:0/3647081688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa88011d70 con 0x561665ff1830 2024-03-19T01:17:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.983+0000 7ffa7be00640 1 -- 172.21.15.69:0/3647081688 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ffa8c0bcbb0 con 0x561665ff1830 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.983+0000 7ffaa400a740 10 client.27185 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.983+0000 7ffaa400a740 1 -- 172.21.15.69:0/3647081688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56166607a400 con 0x561665ff1830 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.983+0000 7ffa7be00640 1 -- 172.21.15.69:0/3647081688 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa88011d70 con 0x561665ff1830 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.984+0000 7ffa7be00640 1 -- 172.21.15.69:0/3647081688 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7ffa8c0b8b20 con 0x561665ff1830 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.984+0000 7ffaa400a740 10 client.27185 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.984+0000 7ffaa400a740 10 client.27185 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.984+0000 7ffaa400a740 20 client.27185 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.984+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56166607aba0 0x56166607cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.984+0000 7ffaa400a740 4 client.27185 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.984+0000 7ffaa400a740 1 -- 172.21.15.69:0/3647081688 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561665f5dc10 con 0x56166607aba0 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.984+0000 7ffa99e00640 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56166607aba0 0x56166607cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffa99e00640 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56166607aba0 0x56166607cfc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb6727e2753d3c64 server_cookie=571a3e7b9b91ea7 in_seq=0 out_seq=0 2024-03-19T01:17:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffa7be00640 10 client.27185 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffa7be00640 1 -- 172.21.15.69:0/3647081688 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561665f5dc10 con 0x56166607aba0 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffa7be00640 10 client.27185 handle_command_reply: tid=0 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffaa400a740 1 client.27185 shutdown 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffaa400a740 1 -- 172.21.15.69:0/3647081688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 msgr2=0x5616660071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x5616660071e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c02c230 tx=0x7ffa8c02bc50 comp rx=0 tx=0).stop 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffaa400a740 1 -- 172.21.15.69:0/3647081688 shutdown_connections 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x56166607aba0 0x56166607cfc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561666003730 0x561666007c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561665ff21e0 0x561666007720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.985+0000 7ffaa400a740 1 --2- 172.21.15.69:0/3647081688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561665ff1830 0x5616660071e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.986+0000 7ffaa400a740 1 -- 172.21.15.69:0/3647081688 >> 172.21.15.69:0/3647081688 conn(0x561665fec150 msgr2=0x561665feb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.986+0000 7ffa7be00640 0 client.27185 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.986+0000 7ffaa400a740 1 -- 172.21.15.69:0/3647081688 shutdown_connections 2024-03-19T01:17:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.986+0000 7ffaa400a740 1 -- 172.21.15.69:0/3647081688 wait complete. 2024-03-19T01:17:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.986+0000 7ffaa400a740 20 client.27185 trim_cache size 0 max 16384 2024-03-19T01:17:42.990 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 -- 172.21.15.69:0/564385876 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa800a2a70 msgr2=0x7ffa800a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa800a2a70 0x7ffa800a4f30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa9401f000 tx=0x7ffa9401bd90 comp rx=0 tx=0).stop 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 -- 172.21.15.69:0/564385876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa9c1039c0 msgr2=0x7ffa9c19a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa9c1039c0 0x7ffa9c19a2d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c005ab0 tx=0x7ffa8c02e980 comp rx=0 tx=0).stop 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 -- 172.21.15.69:0/564385876 shutdown_connections 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffa800a2a70 0x7ffa800a4f30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c1099d0 0x7ffa9c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa9c104380 0x7ffa9c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 --2- 172.21.15.69:0/564385876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa9c1039c0 0x7ffa9c19a2d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 -- 172.21.15.69:0/564385876 >> 172.21.15.69:0/564385876 conn(0x7ffa9c0ff4d0 msgr2=0x7ffa9c0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 -- 172.21.15.69:0/564385876 shutdown_connections 2024-03-19T01:17:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:42.987+0000 7ffaa1400640 1 -- 172.21.15.69:0/564385876 wait complete. 2024-03-19T01:17:43.008 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:17:43.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.228+0000 7f4c73200640 1 -- 172.21.15.69:0/991649153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c74103170 msgr2=0x7f4c7410b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.228+0000 7f4c73200640 1 --2- 172.21.15.69:0/991649153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c74103170 0x7f4c7410b650 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009900 tx=0x7f4c6002bcc0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.228+0000 7f4c73200640 1 -- 172.21.15.69:0/991649153 shutdown_connections 2024-03-19T01:17:43.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.228+0000 7f4c73200640 1 --2- 172.21.15.69:0/991649153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c7410bc70 0x7f4c7410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.228+0000 7f4c73200640 1 --2- 172.21.15.69:0/991649153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c74103170 0x7f4c7410b650 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.228+0000 7f4c73200640 1 --2- 172.21.15.69:0/991649153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c74102850 0x7f4c74102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.228+0000 7f4c73200640 1 -- 172.21.15.69:0/991649153 >> 172.21.15.69:0/991649153 conn(0x7f4c740fc070 msgr2=0x7f4c740fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.228+0000 7f4c73200640 1 -- 172.21.15.69:0/991649153 shutdown_connections 2024-03-19T01:17:43.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.228+0000 7f4c73200640 1 -- 172.21.15.69:0/991649153 wait complete. 2024-03-19T01:17:43.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.229+0000 7f4c73200640 1 Processor -- start 2024-03-19T01:17:43.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.229+0000 7f4c73200640 1 -- start start 2024-03-19T01:17:43.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.230+0000 7f4c73200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c74102850 0x7f4c7419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.230+0000 7f4c73200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74103170 0x7f4c7419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.230+0000 7f4c73200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c7410bc70 0x7f4c7419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.230+0000 7f4c73200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74111ac0 con 0x7f4c7410bc70 2024-03-19T01:17:43.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.230+0000 7f4c73200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74111940 con 0x7f4c74103170 2024-03-19T01:17:43.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.230+0000 7f4c73200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74111c40 con 0x7f4c74102850 2024-03-19T01:17:43.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.230+0000 7f4c71400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74103170 0x7f4c7419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.230+0000 7f4c71400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74103170 0x7f4c7419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40418/0 (socket says 172.21.15.69:40418) 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.230+0000 7f4c71400640 1 -- 172.21.15.69:0/431523062 learned_addr learned my addr 172.21.15.69:0/431523062 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.230+0000 7f4c72800640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c7410bc70 0x7f4c7419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.230+0000 7f4c71e00640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c74102850 0x7f4c7419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.231+0000 7f4c71400640 1 -- 172.21.15.69:0/431523062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c74102850 msgr2=0x7f4c7419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.231+0000 7f4c71400640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c74102850 0x7f4c7419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.231+0000 7f4c71400640 1 -- 172.21.15.69:0/431523062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c7410bc70 msgr2=0x7f4c7419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.231+0000 7f4c71400640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c7410bc70 0x7f4c7419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.231+0000 7f4c71400640 1 -- 172.21.15.69:0/431523062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7419f190 con 0x7f4c74103170 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.231+0000 7f4c72800640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c7410bc70 0x7f4c7419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.231+0000 7f4c71400640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74103170 0x7f4c7419a6e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4c60000c00 tx=0x7f4c600042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.231+0000 7f4c6aa00640 1 -- 172.21.15.69:0/431523062 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60042020 con 0x7f4c74103170 2024-03-19T01:17:43.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.231+0000 7f4c73200640 1 -- 172.21.15.69:0/431523062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7419f420 con 0x7f4c74103170 2024-03-19T01:17:43.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.231+0000 7f4c6aa00640 1 -- 172.21.15.69:0/431523062 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c60005000 con 0x7f4c74103170 2024-03-19T01:17:43.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.231+0000 7f4c6aa00640 1 -- 172.21.15.69:0/431523062 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60003a10 con 0x7f4c74103170 2024-03-19T01:17:43.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.232+0000 7f4c73200640 1 -- 172.21.15.69:0/431523062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c741a6d10 con 0x7f4c74103170 2024-03-19T01:17:43.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.236+0000 7f4c7b7b9740 1 Processor -- start 2024-03-19T01:17:43.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.236+0000 7f4c7b7b9740 1 -- start start 2024-03-19T01:17:43.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55738533fbf0 0x55738533ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c6aa00640 1 -- 172.21.15.69:0/431523062 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f4c60045020 con 0x7f4c74103170 2024-03-19T01:17:43.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557385340510 0x557385348bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573853490f0 0x55738534b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c7b7b9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5573852a4380 con 0x55738533fbf0 2024-03-19T01:17:43.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c72800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55738533fbf0 0x55738533ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c72800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55738533fbf0 0x55738533ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44942/0 (socket says 172.21.15.69:44942) 2024-03-19T01:17:43.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c72800640 1 -- 172.21.15.69:0/270764941 learned_addr learned my addr 172.21.15.69:0/270764941 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/270764941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55738524cd20 con 0x557385340510 2024-03-19T01:17:43.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/270764941 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5573852a0420 con 0x5573853490f0 2024-03-19T01:17:43.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c71400640 1 --2- 172.21.15.69:0/270764941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573853490f0 0x55738534b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.237+0000 7f4c71e00640 1 --2- 172.21.15.69:0/270764941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557385340510 0x557385348bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.238+0000 7f4c72800640 1 -- 172.21.15.69:0/270764941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573853490f0 msgr2=0x55738534b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.238+0000 7f4c72800640 1 --2- 172.21.15.69:0/270764941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573853490f0 0x55738534b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.238+0000 7f4c72800640 1 -- 172.21.15.69:0/270764941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557385340510 msgr2=0x557385348bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.238+0000 7f4c72800640 1 --2- 172.21.15.69:0/270764941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557385340510 0x557385348bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.238+0000 7f4c72800640 1 -- 172.21.15.69:0/270764941 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557384fb83f0 con 0x55738533fbf0 2024-03-19T01:17:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.238+0000 7f4c72800640 1 --2- 172.21.15.69:0/270764941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55738533fbf0 0x55738533ffd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009fd0 tx=0x7f4c6c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3777f6be6c29308 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.238+0000 7f4c5f000640 1 -- 172.21.15.69:0/270764941 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c011700 con 0x55738533fbf0 2024-03-19T01:17:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.238+0000 7f4c5f000640 1 -- 172.21.15.69:0/270764941 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c011d80 con 0x55738533fbf0 2024-03-19T01:17:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.238+0000 7f4c5f000640 1 -- 172.21.15.69:0/270764941 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00e3c0 con 0x55738533fbf0 2024-03-19T01:17:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.239+0000 7f4c6aa00640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c580a29f0 0x7f4c580a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.239+0000 7f4c6aa00640 1 -- 172.21.15.69:0/431523062 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f4c600f0c60 con 0x7f4c74103170 2024-03-19T01:17:43.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.239+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/270764941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55738533fbf0 msgr2=0x55738533ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.239+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/270764941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55738533fbf0 0x55738533ffd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009fd0 tx=0x7f4c6c010860 comp rx=0 tx=0).stop 2024-03-19T01:17:43.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.240+0000 7f4c71e00640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c580a29f0 0x7f4c580a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.240+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/270764941 shutdown_connections 2024-03-19T01:17:43.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.240+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/270764941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573853490f0 0x55738534b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.240+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/270764941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557385340510 0x557385348bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.240+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/270764941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55738533fbf0 0x55738533ffd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.240+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/270764941 >> 172.21.15.69:0/270764941 conn(0x55738533a320 msgr2=0x55738533a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.240+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/270764941 shutdown_connections 2024-03-19T01:17:43.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.240+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/270764941 wait complete. 2024-03-19T01:17:43.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.240+0000 7f4c71e00640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c580a29f0 0x7f4c580a4eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55738533f6c0 tx=0x7f4c64005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.241+0000 7f4c7b7b9740 1 -- start start 2024-03-19T01:17:43.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.241+0000 7f4c7b7b9740 1 Processor -- start 2024-03-19T01:17:43.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385346700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573853490f0 0x557385346c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c7b7b9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55738534bee0 con 0x557385340510 2024-03-19T01:17:43.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c7b7b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557385260160 con 0x5573853490f0 2024-03-19T01:17:43.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c7b7b9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55738534c060 con 0x55738533fbf0 2024-03-19T01:17:43.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c7b7b9740 10 client.0 _ll_register_callbacks cb 0x55738529c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:43.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c71e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c71e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44740/0 (socket says 172.21.15.69:44740) 2024-03-19T01:17:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c71e00640 1 -- 172.21.15.69:0/2039663283 learned_addr learned my addr 172.21.15.69:0/2039663283 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c72800640 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573853490f0 0x557385346c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.242+0000 7f4c71400640 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385346700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.243+0000 7f4c71400640 1 -- 172.21.15.69:0/2039663283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 msgr2=0x5573853461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.243+0000 7f4c71400640 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.243+0000 7f4c71400640 1 -- 172.21.15.69:0/2039663283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573853490f0 msgr2=0x557385346c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.243+0000 7f4c71400640 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573853490f0 0x557385346c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.243+0000 7f4c71400640 1 -- 172.21.15.69:0/2039663283 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573850827d0 con 0x557385340510 2024-03-19T01:17:43.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.243+0000 7f4c72800640 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573853490f0 0x557385346c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:43.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.243+0000 7f4c71400640 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385346700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c600ee0b0 tx=0x7f4c60009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d90eeae1634b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.243+0000 7f4c53400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:17:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.243+0000 7f4c71e00640 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.244+0000 7f4c53400640 1 -- 172.21.15.69:0/2039663283 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600bcd90 con 0x557385340510 2024-03-19T01:17:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.244+0000 7f4c53400640 1 -- 172.21.15.69:0/2039663283 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f4c600051b0 con 0x557385340510 2024-03-19T01:17:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.244+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/2039663283 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c6c011d80 con 0x557385340510 2024-03-19T01:17:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.244+0000 7f4c53400640 1 -- 172.21.15.69:0/2039663283 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4c600bc920 con 0x557385340510 2024-03-19T01:17:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.245+0000 7f4c7b7b9740 10 client.16920 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.245+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/2039663283 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557385358240 con 0x557385340510 2024-03-19T01:17:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.245+0000 7f4c53400640 1 -- 172.21.15.69:0/2039663283 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c011d80 con 0x557385340510 2024-03-19T01:17:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.245+0000 7f4c53400640 1 -- 172.21.15.69:0/2039663283 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7f4c600bf070 con 0x557385340510 2024-03-19T01:17:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.245+0000 7f4c7b7b9740 10 client.16920 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.245+0000 7f4c7b7b9740 10 client.16920 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:17:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.245+0000 7f4c7b7b9740 20 client.16920 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.245+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5573853596b0 0x55738535bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.245+0000 7f4c7b7b9740 4 client.16920 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.245+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/2039663283 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573852b18c0 con 0x5573853596b0 2024-03-19T01:17:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.246+0000 7f4c71e00640 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5573853596b0 0x55738535bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.246+0000 7f4c71e00640 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5573853596b0 0x55738535bb50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=48e5dae4b17584ec server_cookie=f9653adce465ecd6 in_seq=0 out_seq=0 2024-03-19T01:17:43.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.246+0000 7f4c53400640 10 client.16920 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.247+0000 7f4c53400640 1 -- 172.21.15.69:0/2039663283 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5573852b18c0 con 0x5573853596b0 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.247+0000 7f4c53400640 10 client.16920 handle_command_reply: tid=0 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.247+0000 7f4c7b7b9740 1 client.16920 shutdown 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.247+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/2039663283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 msgr2=0x557385346700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.247+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385346700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c600ee0b0 tx=0x7f4c60009620 comp rx=0 tx=0).stop 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.247+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/2039663283 shutdown_connections 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.247+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5573853596b0 0x55738535bb50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.247+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573853490f0 0x557385346c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.248+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385346700 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.248+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/2039663283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.248+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/2039663283 >> 172.21.15.69:0/2039663283 conn(0x55738533a320 msgr2=0x55738533c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.248+0000 7f4c53400640 0 client.16920 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:17:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.248+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/2039663283 shutdown_connections 2024-03-19T01:17:43.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.248+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/2039663283 wait complete. 2024-03-19T01:17:43.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.248+0000 7f4c7b7b9740 20 client.16920 trim_cache size 0 max 16384 2024-03-19T01:17:43.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.267+0000 7f4c7b7b9740 1 Processor -- start 2024-03-19T01:17:43.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.267+0000 7f4c7b7b9740 1 -- start start 2024-03-19T01:17:43.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.267+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55738533fbf0 0x557385354cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.267+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385361460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.267+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573853490f0 0x557385344a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.267+0000 7f4c7b7b9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b0ef0 con 0x557385340510 2024-03-19T01:17:43.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.267+0000 7f4c7b7b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b2be0 con 0x55738533fbf0 2024-03-19T01:17:43.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.267+0000 7f4c7b7b9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b15c0 con 0x5573853490f0 2024-03-19T01:17:43.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c71400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385361460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c71e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55738533fbf0 0x557385354cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c72800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573853490f0 0x557385344a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c71400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385361460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44954/0 (socket says 172.21.15.69:44954) 2024-03-19T01:17:43.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c71400640 1 -- 172.21.15.69:0/3980958528 learned_addr learned my addr 172.21.15.69:0/3980958528 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c71e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55738533fbf0 0x557385354cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40452/0 (socket says 172.21.15.69:40452) 2024-03-19T01:17:43.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c71400640 1 -- 172.21.15.69:0/3980958528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573853490f0 msgr2=0x557385344a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c71400640 1 --2- 172.21.15.69:0/3980958528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573853490f0 0x557385344a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c71400640 1 -- 172.21.15.69:0/3980958528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55738533fbf0 msgr2=0x557385354cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c71400640 1 --2- 172.21.15.69:0/3980958528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55738533fbf0 0x557385354cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c71400640 1 -- 172.21.15.69:0/3980958528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557385356350 con 0x557385340510 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c72800640 1 --2- 172.21.15.69:0/3980958528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573853490f0 0x557385344a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.268+0000 7f4c71400640 1 --2- 172.21.15.69:0/3980958528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385361460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c600b9420 tx=0x7f4c60009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b414e2f53e300160 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.269+0000 7f4c5e600640 1 -- 172.21.15.69:0/3980958528 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600a0070 con 0x557385340510 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.269+0000 7f4c5e600640 1 -- 172.21.15.69:0/3980958528 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c600d9020 con 0x557385340510 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.269+0000 7f4c5e600640 1 -- 172.21.15.69:0/3980958528 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600bcb00 con 0x557385340510 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.269+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/3980958528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 msgr2=0x557385361460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.269+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/3980958528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385361460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c600b9420 tx=0x7f4c60009260 comp rx=0 tx=0).stop 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.270+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/3980958528 shutdown_connections 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.270+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/3980958528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573853490f0 0x557385344a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.270+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/3980958528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385361460 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.270+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/3980958528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55738533fbf0 0x557385354cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.270+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/3980958528 >> 172.21.15.69:0/3980958528 conn(0x55738533a320 msgr2=0x557385341530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.270+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/3980958528 shutdown_connections 2024-03-19T01:17:43.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.271+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/3980958528 wait complete. 2024-03-19T01:17:43.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.271+0000 7f4c7b7b9740 1 -- start start 2024-03-19T01:17:43.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.271+0000 7f4c7b7b9740 1 Processor -- start 2024-03-19T01:17:43.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853514d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385342800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c7b7b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573853490f0 0x557385342d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c7b7b9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b2450 con 0x557385340510 2024-03-19T01:17:43.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c7b7b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b1d50 con 0x5573853490f0 2024-03-19T01:17:43.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c7b7b9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741aee40 con 0x55738533fbf0 2024-03-19T01:17:43.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c7b7b9740 10 client.0 _ll_register_callbacks cb 0x7f4c60005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c71e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c71e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44756/0 (socket says 172.21.15.69:44756) 2024-03-19T01:17:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c71e00640 1 -- 172.21.15.69:0/369547397 learned_addr learned my addr 172.21.15.69:0/369547397 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c71e00640 1 -- 172.21.15.69:0/369547397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573853490f0 msgr2=0x557385342d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c71e00640 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573853490f0 0x557385342d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.272+0000 7f4c71400640 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385342800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.273+0000 7f4c71e00640 1 -- 172.21.15.69:0/369547397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 msgr2=0x557385342800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.273+0000 7f4c71e00640 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385342800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.273+0000 7f4c71e00640 1 -- 172.21.15.69:0/369547397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573853432b0 con 0x55738533fbf0 2024-03-19T01:17:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.273+0000 7f4c71400640 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385342800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.273+0000 7f4c71e00640 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853514d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4c64004290 tx=0x7f4c640046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70a90a2b9112eee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.273+0000 7f4c53400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.273+0000 7f4c53400640 1 -- 172.21.15.69:0/369547397 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6402b410 con 0x55738533fbf0 2024-03-19T01:17:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.273+0000 7f4c53400640 1 -- 172.21.15.69:0/369547397 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f4c6402b5b0 con 0x55738533fbf0 2024-03-19T01:17:43.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.274+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/369547397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c600d9020 con 0x55738533fbf0 2024-03-19T01:17:43.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.274+0000 7f4c53400640 1 -- 172.21.15.69:0/369547397 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4c64038050 con 0x55738533fbf0 2024-03-19T01:17:43.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.274+0000 7f4c7b7b9740 10 client.27154 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:43.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.274+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/369547397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5573853bf830 con 0x55738533fbf0 2024-03-19T01:17:43.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.275+0000 7f4c53400640 1 -- 172.21.15.69:0/369547397 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c600d9020 con 0x55738533fbf0 2024-03-19T01:17:43.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.275+0000 7f4c53400640 1 -- 172.21.15.69:0/369547397 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7f4c6401d530 con 0x55738533fbf0 2024-03-19T01:17:43.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.275+0000 7f4c7b7b9740 10 client.27154 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:43.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.275+0000 7f4c7b7b9740 10 client.27154 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:17:43.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.275+0000 7f4c7b7b9740 20 client.27154 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:43.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.275+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5573853c00c0 0x5573853c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.275+0000 7f4c7b7b9740 4 client.27154 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:17:43.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.275+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/369547397 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573852abc10 con 0x5573853c00c0 2024-03-19T01:17:43.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.275+0000 7f4c71400640 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5573853c00c0 0x5573853c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c71400640 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5573853c00c0 0x5573853c2520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c6a09abd199ecddd server_cookie=76b8f6177766eb51 in_seq=0 out_seq=0 2024-03-19T01:17:43.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c53400640 10 client.27154 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:17:43.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c53400640 1 -- 172.21.15.69:0/369547397 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573852abc10 con 0x5573853c00c0 2024-03-19T01:17:43.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c53400640 10 client.27154 handle_command_reply: tid=0 2024-03-19T01:17:43.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c7b7b9740 1 client.27154 shutdown 2024-03-19T01:17:43.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/369547397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 msgr2=0x5573853514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853514d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4c64004290 tx=0x7f4c640046c0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/369547397 shutdown_connections 2024-03-19T01:17:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5573853c00c0 0x5573853c2520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5573853490f0 0x557385342d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557385340510 0x557385342800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c53400640 0 client.27154 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:17:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.276+0000 7f4c7b7b9740 1 --2- 172.21.15.69:0/369547397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55738533fbf0 0x5573853514d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.277+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/369547397 >> 172.21.15.69:0/369547397 conn(0x55738533a320 msgr2=0x557385344a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.277+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/369547397 shutdown_connections 2024-03-19T01:17:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.277+0000 7f4c7b7b9740 1 -- 172.21.15.69:0/369547397 wait complete. 2024-03-19T01:17:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.277+0000 7f4c7b7b9740 20 client.27154 trim_cache size 0 max 16384 2024-03-19T01:17:43.280 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:17:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.278+0000 7f4c73200640 1 -- 172.21.15.69:0/431523062 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c580a29f0 msgr2=0x7f4c580a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.278+0000 7f4c73200640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c580a29f0 0x7f4c580a4eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55738533f6c0 tx=0x7f4c64005f90 comp rx=0 tx=0).stop 2024-03-19T01:17:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.278+0000 7f4c73200640 1 -- 172.21.15.69:0/431523062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74103170 msgr2=0x7f4c7419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.278+0000 7f4c73200640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74103170 0x7f4c7419a6e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4c60000c00 tx=0x7f4c600042f0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.278+0000 7f4c73200640 1 -- 172.21.15.69:0/431523062 shutdown_connections 2024-03-19T01:17:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.278+0000 7f4c73200640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c7410bc70 0x7f4c7419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.278+0000 7f4c73200640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4c580a29f0 0x7f4c580a4eb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.278+0000 7f4c73200640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74103170 0x7f4c7419a6e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.278+0000 7f4c73200640 1 --2- 172.21.15.69:0/431523062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c74102850 0x7f4c7419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.278+0000 7f4c73200640 1 -- 172.21.15.69:0/431523062 >> 172.21.15.69:0/431523062 conn(0x7f4c740fc070 msgr2=0x7f4c740fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.278+0000 7f4c73200640 1 -- 172.21.15.69:0/431523062 shutdown_connections 2024-03-19T01:17:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.279+0000 7f4c73200640 1 -- 172.21.15.69:0/431523062 wait complete. 2024-03-19T01:17:43.299 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:17:43.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.531+0000 7fd02be00640 1 -- 172.21.15.69:0/699143676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd02c1098b0 msgr2=0x7fd02c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.531+0000 7fd02be00640 1 --2- 172.21.15.69:0/699143676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd02c1098b0 0x7fd02c109c90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009900 tx=0x7fd01c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:17:43.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.532+0000 7fd02be00640 1 -- 172.21.15.69:0/699143676 shutdown_connections 2024-03-19T01:17:43.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.532+0000 7fd02be00640 1 --2- 172.21.15.69:0/699143676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd02c104260 0x7fd02c110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.532+0000 7fd02be00640 1 --2- 172.21.15.69:0/699143676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02c1038a0 0x7fd02c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.532+0000 7fd02be00640 1 --2- 172.21.15.69:0/699143676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd02c1098b0 0x7fd02c109c90 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.532+0000 7fd02be00640 1 -- 172.21.15.69:0/699143676 >> 172.21.15.69:0/699143676 conn(0x7fd02c0ff330 msgr2=0x7fd02c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.532+0000 7fd02be00640 1 -- 172.21.15.69:0/699143676 shutdown_connections 2024-03-19T01:17:43.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.532+0000 7fd02be00640 1 -- 172.21.15.69:0/699143676 wait complete. 2024-03-19T01:17:43.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.534+0000 7fd02be00640 1 Processor -- start 2024-03-19T01:17:43.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.534+0000 7fd02be00640 1 -- start start 2024-03-19T01:17:43.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.534+0000 7fd02be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02c1038a0 0x7fd02c10f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.534+0000 7fd02be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd02c104260 0x7fd02c10f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.534+0000 7fd02be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd02c1098b0 0x7fd02c1a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.534+0000 7fd02be00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c076e00 con 0x7fd02c1098b0 2024-03-19T01:17:43.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.534+0000 7fd02be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c076c80 con 0x7fd02c104260 2024-03-19T01:17:43.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.534+0000 7fd02be00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c076f80 con 0x7fd02c1038a0 2024-03-19T01:17:43.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.535+0000 7fd02a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd02c104260 0x7fd02c10f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.535+0000 7fd02aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02c1038a0 0x7fd02c10f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.535+0000 7fd02aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02c1038a0 0x7fd02c10f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44780/0 (socket says 172.21.15.69:44780) 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.535+0000 7fd02aa00640 1 -- 172.21.15.69:0/3788487617 learned_addr learned my addr 172.21.15.69:0/3788487617 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.535+0000 7fd02b400640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd02c1098b0 0x7fd02c1a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.535+0000 7fd02aa00640 1 -- 172.21.15.69:0/3788487617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd02c104260 msgr2=0x7fd02c10f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.535+0000 7fd02aa00640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd02c104260 0x7fd02c10f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.535+0000 7fd02aa00640 1 -- 172.21.15.69:0/3788487617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd02c1098b0 msgr2=0x7fd02c1a40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.535+0000 7fd02aa00640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd02c1098b0 0x7fd02c1a40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.535+0000 7fd02aa00640 1 -- 172.21.15.69:0/3788487617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02c1a4800 con 0x7fd02c1038a0 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.536+0000 7fd02aa00640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02c1038a0 0x7fd02c10f1d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd01c000c00 tx=0x7fd01c02c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.536+0000 7fd023400640 1 -- 172.21.15.69:0/3788487617 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c041070 con 0x7fd02c1038a0 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.536+0000 7fd023400640 1 -- 172.21.15.69:0/3788487617 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c02cd40 con 0x7fd02c1038a0 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.536+0000 7fd02be00640 1 -- 172.21.15.69:0/3788487617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02c1a4a90 con 0x7fd02c1038a0 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.536+0000 7fd023400640 1 -- 172.21.15.69:0/3788487617 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c034440 con 0x7fd02c1038a0 2024-03-19T01:17:43.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.536+0000 7fd02be00640 1 -- 172.21.15.69:0/3788487617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02c1ab180 con 0x7fd02c1038a0 2024-03-19T01:17:43.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.539+0000 7fd023400640 1 -- 172.21.15.69:0/3788487617 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fd01c044070 con 0x7fd02c1038a0 2024-03-19T01:17:43.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.540+0000 7fd0347dc740 1 Processor -- start 2024-03-19T01:17:43.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.540+0000 7fd0347dc740 1 -- start start 2024-03-19T01:17:43.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.540+0000 7fd023400640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0100a2970 0x7fd0100a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375915bf0 0x55f375915fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd023400640 1 -- 172.21.15.69:0/3788487617 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7fd01c0f0ea0 con 0x7fd02c1038a0 2024-03-19T01:17:43.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375916510 0x55f37591ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f3759214e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd0347dc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f37587a380 con 0x55f375915bf0 2024-03-19T01:17:43.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd0347dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f375822d20 con 0x55f375916510 2024-03-19T01:17:43.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd0347dc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f375876420 con 0x55f37591f0f0 2024-03-19T01:17:43.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd02b400640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0100a2970 0x7fd0100a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd02b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f3759214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd02b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f3759214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44794/0 (socket says 172.21.15.69:44794) 2024-03-19T01:17:43.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd02b400640 1 -- 172.21.15.69:0/833375185 learned_addr learned my addr 172.21.15.69:0/833375185 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd02a000640 1 --2- 172.21.15.69:0/833375185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375916510 0x55f37591ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.541+0000 7fd02aa00640 1 --2- 172.21.15.69:0/833375185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375915bf0 0x55f375915fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.542+0000 7fd02b400640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0100a2970 0x7fd0100a4e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd02400c7b0 tx=0x7fd02400cbb0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.542+0000 7fd02aa00640 1 -- 172.21.15.69:0/833375185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 msgr2=0x55f3759214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.542+0000 7fd02aa00640 1 --2- 172.21.15.69:0/833375185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f3759214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.542+0000 7fd02aa00640 1 -- 172.21.15.69:0/833375185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375916510 msgr2=0x55f37591ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.542+0000 7fd02aa00640 1 --2- 172.21.15.69:0/833375185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375916510 0x55f37591ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.542+0000 7fd02aa00640 1 -- 172.21.15.69:0/833375185 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f37558e3f0 con 0x55f375915bf0 2024-03-19T01:17:43.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.542+0000 7fd02b400640 1 --2- 172.21.15.69:0/833375185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f3759214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:43.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.542+0000 7fd02aa00640 1 --2- 172.21.15.69:0/833375185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375915bf0 0x55f375915fd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0ee2b0 tx=0x7fd01c003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895aad3d1adea311 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.543+0000 7fd017e00640 1 -- 172.21.15.69:0/833375185 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c04b070 con 0x55f375915bf0 2024-03-19T01:17:43.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.543+0000 7fd017e00640 1 -- 172.21.15.69:0/833375185 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c0bd1c0 con 0x55f375915bf0 2024-03-19T01:17:43.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.543+0000 7fd017e00640 1 -- 172.21.15.69:0/833375185 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0b9830 con 0x55f375915bf0 2024-03-19T01:17:43.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.544+0000 7fd0347dc740 1 -- 172.21.15.69:0/833375185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375915bf0 msgr2=0x55f375915fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.544+0000 7fd0347dc740 1 --2- 172.21.15.69:0/833375185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375915bf0 0x55f375915fd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0ee2b0 tx=0x7fd01c003990 comp rx=0 tx=0).stop 2024-03-19T01:17:43.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.544+0000 7fd0347dc740 1 -- 172.21.15.69:0/833375185 shutdown_connections 2024-03-19T01:17:43.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.544+0000 7fd0347dc740 1 --2- 172.21.15.69:0/833375185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f3759214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.544+0000 7fd0347dc740 1 --2- 172.21.15.69:0/833375185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375916510 0x55f37591ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.544+0000 7fd0347dc740 1 --2- 172.21.15.69:0/833375185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375915bf0 0x55f375915fd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.544+0000 7fd0347dc740 1 -- 172.21.15.69:0/833375185 >> 172.21.15.69:0/833375185 conn(0x55f375910320 msgr2=0x55f375910700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.544+0000 7fd0347dc740 1 -- 172.21.15.69:0/833375185 shutdown_connections 2024-03-19T01:17:43.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.544+0000 7fd0347dc740 1 -- 172.21.15.69:0/833375185 wait complete. 2024-03-19T01:17:43.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.545+0000 7fd0347dc740 1 -- start start 2024-03-19T01:17:43.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.545+0000 7fd0347dc740 1 Processor -- start 2024-03-19T01:17:43.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.545+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 0x55f37591c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.545+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f37591c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.545+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f37591cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.545+0000 7fd0347dc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f375921ca0 con 0x55f375916510 2024-03-19T01:17:43.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.545+0000 7fd0347dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f375836160 con 0x55f375915bf0 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.545+0000 7fd0347dc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f375921e20 con 0x55f37591f0f0 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.545+0000 7fd0347dc740 10 client.0 _ll_register_callbacks cb 0x55f375872170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 0x55f37591c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f37591cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f37591cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44800/0 (socket says 172.21.15.69:44800) 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02b400640 1 -- 172.21.15.69:0/2943818952 learned_addr learned my addr 172.21.15.69:0/2943818952 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02a000640 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f37591c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02b400640 1 -- 172.21.15.69:0/2943818952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 msgr2=0x55f37591c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02b400640 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 0x55f37591c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02b400640 1 -- 172.21.15.69:0/2943818952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 msgr2=0x55f37591c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02b400640 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f37591c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02b400640 1 -- 172.21.15.69:0/2943818952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3756587d0 con 0x55f37591f0f0 2024-03-19T01:17:43.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02aa00640 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 0x55f37591c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:43.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.546+0000 7fd02a000640 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f37591c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:43.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.547+0000 7fd02b400640 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f37591cc90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd02401c660 tx=0x7fd024004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81583672875ea4d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.547+0000 7fd014c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:43.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.547+0000 7fd014c00640 1 -- 172.21.15.69:0/2943818952 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02402a060 con 0x55f37591f0f0 2024-03-19T01:17:43.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.547+0000 7fd0347dc740 1 -- 172.21.15.69:0/2943818952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd01c0bd1c0 con 0x55f37591f0f0 2024-03-19T01:17:43.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.547+0000 7fd014c00640 1 -- 172.21.15.69:0/2943818952 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7fd02401a5e0 con 0x55f37591f0f0 2024-03-19T01:17:43.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.547+0000 7fd014c00640 1 -- 172.21.15.69:0/2943818952 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd024031050 con 0x55f37591f0f0 2024-03-19T01:17:43.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.547+0000 7fd0347dc740 10 client.27163 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:43.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.547+0000 7fd0347dc740 1 -- 172.21.15.69:0/2943818952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f37592dff0 con 0x55f37591f0f0 2024-03-19T01:17:43.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.548+0000 7fd014c00640 1 -- 172.21.15.69:0/2943818952 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c0bd1c0 con 0x55f37591f0f0 2024-03-19T01:17:43.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.548+0000 7fd014c00640 1 -- 172.21.15.69:0/2943818952 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7fd02401b950 con 0x55f37591f0f0 2024-03-19T01:17:43.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.549+0000 7fd0347dc740 10 client.27163 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:43.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.549+0000 7fd0347dc740 10 client.27163 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:17:43.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.549+0000 7fd0347dc740 20 client.27163 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:43.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.549+0000 7fd0347dc740 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f37592f510 0x55f375931900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.549+0000 7fd0347dc740 4 client.27163 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:43.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.549+0000 7fd0347dc740 1 -- 172.21.15.69:0/2943818952 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3758878c0 con 0x55f37592f510 2024-03-19T01:17:43.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.549+0000 7fd02aa00640 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f37592f510 0x55f375931900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.549+0000 7fd014c00640 10 client.27163 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.549+0000 7fd02aa00640 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f37592f510 0x55f375931900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1f6784930cf778c server_cookie=b3fd8c6cc6292453 in_seq=0 out_seq=0 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.550+0000 7fd014c00640 1 -- 172.21.15.69:0/2943818952 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f3758878c0 con 0x55f37592f510 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.550+0000 7fd014c00640 10 client.27163 handle_command_reply: tid=0 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.550+0000 7fd0347dc740 1 client.27163 shutdown 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.550+0000 7fd0347dc740 1 -- 172.21.15.69:0/2943818952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 msgr2=0x55f37591cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.550+0000 7fd0347dc740 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f37591cc90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd02401c660 tx=0x7fd024004a40 comp rx=0 tx=0).stop 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.551+0000 7fd0347dc740 1 -- 172.21.15.69:0/2943818952 shutdown_connections 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.551+0000 7fd0347dc740 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f37592f510 0x55f375931900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.551+0000 7fd0347dc740 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f37591cc90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.551+0000 7fd0347dc740 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f37591c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.551+0000 7fd0347dc740 1 --2- 172.21.15.69:0/2943818952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 0x55f37591c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.551+0000 7fd0347dc740 1 -- 172.21.15.69:0/2943818952 >> 172.21.15.69:0/2943818952 conn(0x55f375910320 msgr2=0x55f375912070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.551+0000 7fd014c00640 0 client.27163 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:17:43.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.551+0000 7fd0347dc740 1 -- 172.21.15.69:0/2943818952 shutdown_connections 2024-03-19T01:17:43.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.551+0000 7fd0347dc740 1 -- 172.21.15.69:0/2943818952 wait complete. 2024-03-19T01:17:43.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.551+0000 7fd0347dc740 20 client.27163 trim_cache size 0 max 16384 2024-03-19T01:17:43.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.569+0000 7fd0347dc740 1 Processor -- start 2024-03-19T01:17:43.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.570+0000 7fd0347dc740 1 -- start start 2024-03-19T01:17:43.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.570+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 0x55f37591e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.570+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f37592a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.570+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f375911e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.570+0000 7fd0347dc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1b5280 con 0x55f375916510 2024-03-19T01:17:43.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.570+0000 7fd0347dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1b6f70 con 0x55f375915bf0 2024-03-19T01:17:43.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.570+0000 7fd0347dc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1b5950 con 0x55f37591f0f0 2024-03-19T01:17:43.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.570+0000 7fd02aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 0x55f37591e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.570+0000 7fd02b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f375911e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.570+0000 7fd02aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 0x55f37591e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40498/0 (socket says 172.21.15.69:40498) 2024-03-19T01:17:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.570+0000 7fd02aa00640 1 -- 172.21.15.69:0/951311837 learned_addr learned my addr 172.21.15.69:0/951311837 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.571+0000 7fd02aa00640 1 -- 172.21.15.69:0/951311837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 msgr2=0x55f375911e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.571+0000 7fd02aa00640 1 --2- 172.21.15.69:0/951311837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f375911e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.571+0000 7fd02aa00640 1 -- 172.21.15.69:0/951311837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 msgr2=0x55f37592a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.571+0000 7fd02a000640 1 --2- 172.21.15.69:0/951311837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f37592a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.571+0000 7fd02aa00640 1 --2- 172.21.15.69:0/951311837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f37592a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.571+0000 7fd02aa00640 1 -- 172.21.15.69:0/951311837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f375921810 con 0x55f375915bf0 2024-03-19T01:17:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.571+0000 7fd02a000640 1 --2- 172.21.15.69:0/951311837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f37592a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.571+0000 7fd02aa00640 1 --2- 172.21.15.69:0/951311837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 0x55f37591e940 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0ee280 tx=0x7fd01c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa471596b89c5582 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.571+0000 7fd017400640 1 -- 172.21.15.69:0/951311837 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c04b040 con 0x55f375915bf0 2024-03-19T01:17:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.571+0000 7fd017400640 1 -- 172.21.15.69:0/951311837 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c0d5020 con 0x55f375915bf0 2024-03-19T01:17:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.571+0000 7fd017400640 1 -- 172.21.15.69:0/951311837 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0035f0 con 0x55f375915bf0 2024-03-19T01:17:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.572+0000 7fd0347dc740 1 -- 172.21.15.69:0/951311837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 msgr2=0x55f37591e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.572+0000 7fd0347dc740 1 --2- 172.21.15.69:0/951311837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 0x55f37591e940 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0ee280 tx=0x7fd01c009680 comp rx=0 tx=0).stop 2024-03-19T01:17:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.573+0000 7fd0347dc740 1 -- 172.21.15.69:0/951311837 shutdown_connections 2024-03-19T01:17:43.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.573+0000 7fd0347dc740 1 --2- 172.21.15.69:0/951311837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f37591f0f0 0x55f375911e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.573+0000 7fd0347dc740 1 --2- 172.21.15.69:0/951311837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f37592a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.573+0000 7fd0347dc740 1 --2- 172.21.15.69:0/951311837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f375915bf0 0x55f37591e940 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.573+0000 7fd0347dc740 1 -- 172.21.15.69:0/951311837 >> 172.21.15.69:0/951311837 conn(0x55f375910320 msgr2=0x55f375911750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.573+0000 7fd0347dc740 1 -- 172.21.15.69:0/951311837 shutdown_connections 2024-03-19T01:17:43.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.573+0000 7fd0347dc740 1 -- 172.21.15.69:0/951311837 wait complete. 2024-03-19T01:17:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.573+0000 7fd0347dc740 1 -- start start 2024-03-19T01:17:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.574+0000 7fd0347dc740 1 Processor -- start 2024-03-19T01:17:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.574+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f375915bf0 0x55f3759195a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.574+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f375919ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.574+0000 7fd0347dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f37591f0f0 0x55f375994840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.574+0000 7fd0347dc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1b67e0 con 0x55f375916510 2024-03-19T01:17:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.574+0000 7fd0347dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1b60e0 con 0x55f37591f0f0 2024-03-19T01:17:43.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.574+0000 7fd0347dc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1b31d0 con 0x55f375915bf0 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.574+0000 7fd0347dc740 10 client.0 _ll_register_callbacks cb 0x7fd024009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.574+0000 7fd02aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f375915bf0 0x55f3759195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.574+0000 7fd02aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f375915bf0 0x55f3759195a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44826/0 (socket says 172.21.15.69:44826) 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.574+0000 7fd02aa00640 1 -- 172.21.15.69:0/3801591735 learned_addr learned my addr 172.21.15.69:0/3801591735 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.575+0000 7fd02a000640 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f375919ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.575+0000 7fd02b400640 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f37591f0f0 0x55f375994840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.575+0000 7fd02aa00640 1 -- 172.21.15.69:0/3801591735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f37591f0f0 msgr2=0x55f375994840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.575+0000 7fd02aa00640 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f37591f0f0 0x55f375994840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.575+0000 7fd02aa00640 1 -- 172.21.15.69:0/3801591735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 msgr2=0x55f375919ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.575+0000 7fd02aa00640 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f375919ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.575+0000 7fd02aa00640 1 -- 172.21.15.69:0/3801591735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f375994e00 con 0x55f375915bf0 2024-03-19T01:17:43.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.575+0000 7fd02b400640 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f37591f0f0 0x55f375994840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.575+0000 7fd02aa00640 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f375915bf0 0x55f3759195a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0b9b90 tx=0x7fd01c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6681b16dff70fe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.575+0000 7fd014c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.575+0000 7fd014c00640 1 -- 172.21.15.69:0/3801591735 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c087060 con 0x55f375915bf0 2024-03-19T01:17:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.576+0000 7fd014c00640 1 -- 172.21.15.69:0/3801591735 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7fd01c0035f0 con 0x55f375915bf0 2024-03-19T01:17:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.576+0000 7fd0347dc740 1 -- 172.21.15.69:0/3801591735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd01c0d5020 con 0x55f375915bf0 2024-03-19T01:17:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.576+0000 7fd014c00640 1 -- 172.21.15.69:0/3801591735 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd01c04b070 con 0x55f375915bf0 2024-03-19T01:17:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.576+0000 7fd0347dc740 10 client.27169 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.576+0000 7fd0347dc740 1 -- 172.21.15.69:0/3801591735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f375996070 con 0x55f375915bf0 2024-03-19T01:17:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.576+0000 7fd014c00640 1 -- 172.21.15.69:0/3801591735 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c0d5020 con 0x55f375915bf0 2024-03-19T01:17:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.576+0000 7fd014c00640 1 -- 172.21.15.69:0/3801591735 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7fd01c0c54d0 con 0x55f375915bf0 2024-03-19T01:17:43.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.577+0000 7fd0347dc740 10 client.27169 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:43.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.577+0000 7fd0347dc740 10 client.27169 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:17:43.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.577+0000 7fd0347dc740 20 client.27169 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:43.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.577+0000 7fd0347dc740 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f375996180 0x55f3759985e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.577+0000 7fd0347dc740 4 client.27169 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:17:43.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.577+0000 7fd0347dc740 1 -- 172.21.15.69:0/3801591735 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f375881c10 con 0x55f375996180 2024-03-19T01:17:43.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.577+0000 7fd02a000640 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f375996180 0x55f3759985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.578+0000 7fd02a000640 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f375996180 0x55f3759985e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1034a2a79e730d31 server_cookie=f90f68a3e103fd04 in_seq=0 out_seq=0 2024-03-19T01:17:43.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.578+0000 7fd014c00640 10 client.27169 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:17:43.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.578+0000 7fd014c00640 1 -- 172.21.15.69:0/3801591735 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f375881c10 con 0x55f375996180 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.578+0000 7fd014c00640 10 client.27169 handle_command_reply: tid=0 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.578+0000 7fd0347dc740 1 client.27169 shutdown 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd0347dc740 1 -- 172.21.15.69:0/3801591735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f375915bf0 msgr2=0x55f3759195a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd0347dc740 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f375915bf0 0x55f3759195a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0b9b90 tx=0x7fd01c009260 comp rx=0 tx=0).stop 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd0347dc740 1 -- 172.21.15.69:0/3801591735 shutdown_connections 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd0347dc740 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55f375996180 0x55f3759985e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd014c00640 0 client.27169 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd0347dc740 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f37591f0f0 0x55f375994840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd0347dc740 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f375916510 0x55f375919ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd0347dc740 1 --2- 172.21.15.69:0/3801591735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f375915bf0 0x55f3759195a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd0347dc740 1 -- 172.21.15.69:0/3801591735 >> 172.21.15.69:0/3801591735 conn(0x55f375910320 msgr2=0x55f375911fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd0347dc740 1 -- 172.21.15.69:0/3801591735 shutdown_connections 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd0347dc740 1 -- 172.21.15.69:0/3801591735 wait complete. 2024-03-19T01:17:43.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.579+0000 7fd0347dc740 20 client.27169 trim_cache size 0 max 16384 2024-03-19T01:17:43.582 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.580+0000 7fd02be00640 1 -- 172.21.15.69:0/3788487617 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0100a2970 msgr2=0x7fd0100a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.580+0000 7fd02be00640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0100a2970 0x7fd0100a4e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd02400c7b0 tx=0x7fd02400cbb0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.580+0000 7fd02be00640 1 -- 172.21.15.69:0/3788487617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02c1038a0 msgr2=0x7fd02c10f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.580+0000 7fd02be00640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02c1038a0 0x7fd02c10f1d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd01c000c00 tx=0x7fd01c02c710 comp rx=0 tx=0).stop 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.581+0000 7fd02be00640 1 -- 172.21.15.69:0/3788487617 shutdown_connections 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.581+0000 7fd02be00640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd02c1098b0 0x7fd02c1a40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.581+0000 7fd02be00640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0100a2970 0x7fd0100a4e30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.581+0000 7fd02be00640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd02c104260 0x7fd02c10f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.581+0000 7fd02be00640 1 --2- 172.21.15.69:0/3788487617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02c1038a0 0x7fd02c10f1d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.581+0000 7fd02be00640 1 -- 172.21.15.69:0/3788487617 >> 172.21.15.69:0/3788487617 conn(0x7fd02c0ff330 msgr2=0x7fd02c079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.581+0000 7fd02be00640 1 -- 172.21.15.69:0/3788487617 shutdown_connections 2024-03-19T01:17:43.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.581+0000 7fd02be00640 1 -- 172.21.15.69:0/3788487617 wait complete. 2024-03-19T01:17:43.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:43 smithi069 ceph-mon[31727]: pgmap v216: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.8 MiB/s wr, 624 op/s 2024-03-19T01:17:43.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:43 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/4229946781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:17:43.605 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:17:43.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:43 smithi086 ceph-mon[66829]: pgmap v216: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.8 MiB/s wr, 624 op/s 2024-03-19T01:17:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:43 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/4229946781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:17:43.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:43 smithi163 ceph-mon[67428]: pgmap v216: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.8 MiB/s wr, 624 op/s 2024-03-19T01:17:43.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:43 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/4229946781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:17:43.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.841+0000 7f7f5b200640 1 -- 172.21.15.69:0/1747958258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5c102850 msgr2=0x7f7f5c102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.841+0000 7f7f5b200640 1 --2- 172.21.15.69:0/1747958258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5c102850 0x7f7f5c102c30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009900 tx=0x7f7f4c02bcc0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.841+0000 7f7f5b200640 1 -- 172.21.15.69:0/1747958258 shutdown_connections 2024-03-19T01:17:43.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.841+0000 7f7f5b200640 1 --2- 172.21.15.69:0/1747958258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f5c10bc70 0x7f7f5c10e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.841+0000 7f7f5b200640 1 --2- 172.21.15.69:0/1747958258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c103170 0x7f7f5c10b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.841+0000 7f7f5b200640 1 --2- 172.21.15.69:0/1747958258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5c102850 0x7f7f5c102c30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.841+0000 7f7f5b200640 1 -- 172.21.15.69:0/1747958258 >> 172.21.15.69:0/1747958258 conn(0x7f7f5c0fc070 msgr2=0x7f7f5c0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.842+0000 7f7f5b200640 1 -- 172.21.15.69:0/1747958258 shutdown_connections 2024-03-19T01:17:43.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.842+0000 7f7f5b200640 1 -- 172.21.15.69:0/1747958258 wait complete. 2024-03-19T01:17:43.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.843+0000 7f7f5b200640 1 Processor -- start 2024-03-19T01:17:43.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.843+0000 7f7f5b200640 1 -- start start 2024-03-19T01:17:43.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.843+0000 7f7f5b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f5c102850 0x7f7f5c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.843+0000 7f7f5b200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c103170 0x7f7f5c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f5b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5c10bc70 0x7f7f5c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f5b200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c111ac0 con 0x7f7f5c103170 2024-03-19T01:17:43.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f5b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c111940 con 0x7f7f5c10bc70 2024-03-19T01:17:43.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f5b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c111c40 con 0x7f7f5c102850 2024-03-19T01:17:43.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f59e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f5c102850 0x7f7f5c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f5a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5c10bc70 0x7f7f5c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f59e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f5c102850 0x7f7f5c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44854/0 (socket says 172.21.15.69:44854) 2024-03-19T01:17:43.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f59e00640 1 -- 172.21.15.69:0/1930105408 learned_addr learned my addr 172.21.15.69:0/1930105408 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f59400640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c103170 0x7f7f5c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f5a800640 1 -- 172.21.15.69:0/1930105408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f5c102850 msgr2=0x7f7f5c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f5a800640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f5c102850 0x7f7f5c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f5a800640 1 -- 172.21.15.69:0/1930105408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c103170 msgr2=0x7f7f5c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f5a800640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c103170 0x7f7f5c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f5a800640 1 -- 172.21.15.69:0/1930105408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5c19cf50 con 0x7f7f5c10bc70 2024-03-19T01:17:43.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.844+0000 7f7f59400640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c103170 0x7f7f5c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:43.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.845+0000 7f7f5a800640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5c10bc70 0x7f7f5c19c820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009fd0 tx=0x7f7f5400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.845+0000 7f7f52a00640 1 -- 172.21.15.69:0/1930105408 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400ce20 con 0x7f7f5c10bc70 2024-03-19T01:17:43.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.845+0000 7f7f52a00640 1 -- 172.21.15.69:0/1930105408 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f540137d0 con 0x7f7f5c10bc70 2024-03-19T01:17:43.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.845+0000 7f7f52a00640 1 -- 172.21.15.69:0/1930105408 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f540055e0 con 0x7f7f5c10bc70 2024-03-19T01:17:43.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.845+0000 7f7f5b200640 1 -- 172.21.15.69:0/1930105408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5c19d240 con 0x7f7f5c10bc70 2024-03-19T01:17:43.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.845+0000 7f7f5b200640 1 -- 172.21.15.69:0/1930105408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5c1003d0 con 0x7f7f5c10bc70 2024-03-19T01:17:43.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.848+0000 7f7f52a00640 1 -- 172.21.15.69:0/1930105408 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f7f54013980 con 0x7f7f5c10bc70 2024-03-19T01:17:43.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.850+0000 7f7f52a00640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f400a2a70 0x7f7f400a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.850+0000 7f7f639db740 1 Processor -- start 2024-03-19T01:17:43.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.851+0000 7f7f52a00640 1 -- 172.21.15.69:0/1930105408 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f7f540d3870 con 0x7f7f5c10bc70 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.851+0000 7f7f639db740 1 -- start start 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.851+0000 7f7f59e00640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f400a2a70 0x7f7f400a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.851+0000 7f7f639db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 0x5557f3bb49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.851+0000 7f7f639db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bb4f80 0x5557f3bc63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.851+0000 7f7f639db740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bc6900 0x5557f3bc8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.851+0000 7f7f639db740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5557f3b19380 con 0x5557f3bb4f80 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.851+0000 7f7f639db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557f3ac1d20 con 0x5557f3bb45d0 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.851+0000 7f7f639db740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5557f3b15420 con 0x5557f3bc6900 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.852+0000 7f7f59e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 0x5557f3bb49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.852+0000 7f7f5a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bc6900 0x5557f3bc8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.852+0000 7f7f5a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bc6900 0x5557f3bc8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44870/0 (socket says 172.21.15.69:44870) 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.852+0000 7f7f5a800640 1 -- 172.21.15.69:0/457701430 learned_addr learned my addr 172.21.15.69:0/457701430 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.852+0000 7f7f59e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 0x5557f3bb49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40540/0 (socket says 172.21.15.69:40540) 2024-03-19T01:17:43.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.852+0000 7f7f59e00640 1 -- 172.21.15.69:0/457701430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bc6900 msgr2=0x5557f3bc8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.852+0000 7f7f59e00640 1 --2- 172.21.15.69:0/457701430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bc6900 0x5557f3bc8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.852+0000 7f7f59e00640 1 -- 172.21.15.69:0/457701430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bb4f80 msgr2=0x5557f3bc63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.852+0000 7f7f59e00640 1 --2- 172.21.15.69:0/457701430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bb4f80 0x5557f3bc63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.852+0000 7f7f59e00640 1 -- 172.21.15.69:0/457701430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f382d3f0 con 0x5557f3bb45d0 2024-03-19T01:17:43.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.853+0000 7f7f59e00640 1 --2- 172.21.15.69:0/457701430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 0x5557f3bb49b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c006010 tx=0x7f7f4c02c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e74feeeec47c80b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.853+0000 7f7f47000640 1 -- 172.21.15.69:0/457701430 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0042f0 con 0x5557f3bb45d0 2024-03-19T01:17:43.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.853+0000 7f7f47000640 1 -- 172.21.15.69:0/457701430 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c004b50 con 0x5557f3bb45d0 2024-03-19T01:17:43.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.853+0000 7f7f47000640 1 -- 172.21.15.69:0/457701430 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0037b0 con 0x5557f3bb45d0 2024-03-19T01:17:43.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.854+0000 7f7f639db740 1 -- 172.21.15.69:0/457701430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 msgr2=0x5557f3bb49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.854+0000 7f7f639db740 1 --2- 172.21.15.69:0/457701430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 0x5557f3bb49b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c006010 tx=0x7f7f4c02c990 comp rx=0 tx=0).stop 2024-03-19T01:17:43.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.854+0000 7f7f59e00640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f400a2a70 0x7f7f400a4f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c02c8b0 tx=0x7f7f4c03c000 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.854+0000 7f7f639db740 1 -- 172.21.15.69:0/457701430 shutdown_connections 2024-03-19T01:17:43.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.854+0000 7f7f639db740 1 --2- 172.21.15.69:0/457701430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bc6900 0x5557f3bc8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.854+0000 7f7f639db740 1 --2- 172.21.15.69:0/457701430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bb4f80 0x5557f3bc63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.854+0000 7f7f639db740 1 --2- 172.21.15.69:0/457701430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 0x5557f3bb49b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.854+0000 7f7f639db740 1 -- 172.21.15.69:0/457701430 >> 172.21.15.69:0/457701430 conn(0x5557f3baf320 msgr2=0x5557f3baf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.854+0000 7f7f639db740 1 -- 172.21.15.69:0/457701430 shutdown_connections 2024-03-19T01:17:43.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.854+0000 7f7f639db740 1 -- 172.21.15.69:0/457701430 wait complete. 2024-03-19T01:17:43.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.855+0000 7f7f639db740 1 -- start start 2024-03-19T01:17:43.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.855+0000 7f7f639db740 1 Processor -- start 2024-03-19T01:17:43.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f639db740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 0x5557f3bc0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f639db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bb4f80 0x5557f3bc62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f639db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bc6900 0x5557f3bc0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f639db740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5557f3bb5ef0 con 0x5557f3bb4f80 2024-03-19T01:17:43.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f639db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557f3ad5160 con 0x5557f3bc6900 2024-03-19T01:17:43.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f639db740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5557f3bb6070 con 0x5557f3bb45d0 2024-03-19T01:17:43.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f5a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bc6900 0x5557f3bc0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f59e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 0x5557f3bc0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f5a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bc6900 0x5557f3bc0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40552/0 (socket says 172.21.15.69:40552) 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f59e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 0x5557f3bc0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44884/0 (socket says 172.21.15.69:44884) 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f5a800640 1 -- 172.21.15.69:0/4213272788 learned_addr learned my addr 172.21.15.69:0/4213272788 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f639db740 10 client.0 _ll_register_callbacks cb 0x5557f3b11170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f5a800640 1 -- 172.21.15.69:0/4213272788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 msgr2=0x5557f3bc0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f59400640 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bb4f80 0x5557f3bc62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f5a800640 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 0x5557f3bc0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f5a800640 1 -- 172.21.15.69:0/4213272788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bb4f80 msgr2=0x5557f3bc62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f5a800640 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bb4f80 0x5557f3bc62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.856+0000 7f7f5a800640 1 -- 172.21.15.69:0/4213272788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f38f77d0 con 0x5557f3bc6900 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.857+0000 7f7f59400640 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bb4f80 0x5557f3bc62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.857+0000 7f7f5a800640 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bc6900 0x5557f3bc0bb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400c760 tx=0x7f7f540a0350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f0cb9b33cb652f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.857+0000 7f7f3b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:17:43.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.857+0000 7f7f3b400640 1 -- 172.21.15.69:0/4213272788 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f540b8060 con 0x5557f3bc6900 2024-03-19T01:17:43.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.857+0000 7f7f3b400640 1 -- 172.21.15.69:0/4213272788 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f7f540a7ea0 con 0x5557f3bc6900 2024-03-19T01:17:43.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.857+0000 7f7f639db740 1 -- 172.21.15.69:0/4213272788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f4c004b50 con 0x5557f3bc6900 2024-03-19T01:17:43.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.857+0000 7f7f3b400640 1 -- 172.21.15.69:0/4213272788 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7f540d0ff0 con 0x5557f3bc6900 2024-03-19T01:17:43.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.857+0000 7f7f639db740 10 client.27227 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:43.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.857+0000 7f7f639db740 1 -- 172.21.15.69:0/4213272788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5557f3bde260 con 0x5557f3bc6900 2024-03-19T01:17:43.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.857+0000 7f7f3b400640 1 -- 172.21.15.69:0/4213272788 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c004b50 con 0x5557f3bc6900 2024-03-19T01:17:43.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.858+0000 7f7f3b400640 1 -- 172.21.15.69:0/4213272788 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7f7f540057a0 con 0x5557f3bc6900 2024-03-19T01:17:43.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.858+0000 7f7f639db740 10 client.27227 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:43.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.858+0000 7f7f639db740 10 client.27227 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:17:43.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.859+0000 7f7f639db740 20 client.27227 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.859+0000 7f7f639db740 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557f3bdf6c0 0x5557f3be1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.859+0000 7f7f639db740 4 client.27227 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.859+0000 7f7f639db740 1 -- 172.21.15.69:0/4213272788 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557f3b268c0 con 0x5557f3bdf6c0 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.859+0000 7f7f59e00640 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557f3bdf6c0 0x5557f3be1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.860+0000 7f7f59e00640 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557f3bdf6c0 0x5557f3be1b80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7aa611efef30c56d server_cookie=b6c626133ac5c4cf in_seq=0 out_seq=0 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.860+0000 7f7f3b400640 10 client.27227 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.860+0000 7f7f3b400640 1 -- 172.21.15.69:0/4213272788 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5557f3b268c0 con 0x5557f3bdf6c0 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.860+0000 7f7f3b400640 10 client.27227 handle_command_reply: tid=0 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.860+0000 7f7f639db740 1 client.27227 shutdown 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f639db740 1 -- 172.21.15.69:0/4213272788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bc6900 msgr2=0x5557f3bc0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f639db740 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bc6900 0x5557f3bc0bb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400c760 tx=0x7f7f540a0350 comp rx=0 tx=0).stop 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f639db740 1 -- 172.21.15.69:0/4213272788 shutdown_connections 2024-03-19T01:17:43.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f639db740 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557f3bdf6c0 0x5557f3be1b80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f639db740 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bc6900 0x5557f3bc0bb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f639db740 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bb4f80 0x5557f3bc62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f639db740 1 --2- 172.21.15.69:0/4213272788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 0x5557f3bc0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f3b400640 0 client.27227 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:17:43.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f639db740 1 -- 172.21.15.69:0/4213272788 >> 172.21.15.69:0/4213272788 conn(0x5557f3baf320 msgr2=0x5557f3bb1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f639db740 1 -- 172.21.15.69:0/4213272788 shutdown_connections 2024-03-19T01:17:43.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f639db740 1 -- 172.21.15.69:0/4213272788 wait complete. 2024-03-19T01:17:43.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.861+0000 7f7f639db740 20 client.27227 trim_cache size 0 max 16384 2024-03-19T01:17:43.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.875+0000 7f7f639db740 1 Processor -- start 2024-03-19T01:17:43.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.875+0000 7f7f639db740 1 -- start start 2024-03-19T01:17:43.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f639db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 0x5557f3bae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f639db740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb4f80 0x5557f3be7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f639db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bc6900 0x5557f3bf27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f639db740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1acb90 con 0x5557f3bc6900 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f639db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1ae880 con 0x5557f3bb45d0 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f639db740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1ad260 con 0x5557f3bb4f80 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f5a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bc6900 0x5557f3bf27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f59400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb4f80 0x5557f3be7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f59400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb4f80 0x5557f3be7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44896/0 (socket says 172.21.15.69:44896) 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f59400640 1 -- 172.21.15.69:0/393136987 learned_addr learned my addr 172.21.15.69:0/393136987 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f59e00640 1 --2- 172.21.15.69:0/393136987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 0x5557f3bae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f59400640 1 -- 172.21.15.69:0/393136987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 msgr2=0x5557f3bae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f59400640 1 --2- 172.21.15.69:0/393136987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 0x5557f3bae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f59400640 1 -- 172.21.15.69:0/393136987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bc6900 msgr2=0x5557f3bf27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f59400640 1 --2- 172.21.15.69:0/393136987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bc6900 0x5557f3bf27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f59400640 1 -- 172.21.15.69:0/393136987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f3bb4d20 con 0x5557f3bb4f80 2024-03-19T01:17:43.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.876+0000 7f7f5a800640 1 --2- 172.21.15.69:0/393136987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bc6900 0x5557f3bf27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:43.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.877+0000 7f7f59e00640 1 --2- 172.21.15.69:0/393136987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 0x5557f3bae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:43.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.877+0000 7f7f59400640 1 --2- 172.21.15.69:0/393136987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb4f80 0x5557f3be7450 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009a70 tx=0x7f7f48009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de3d11577d8a3e3e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.877+0000 7f7f46600640 1 -- 172.21.15.69:0/393136987 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800fcc0 con 0x5557f3bb4f80 2024-03-19T01:17:43.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.877+0000 7f7f46600640 1 -- 172.21.15.69:0/393136987 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4800fe60 con 0x5557f3bb4f80 2024-03-19T01:17:43.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.877+0000 7f7f46600640 1 -- 172.21.15.69:0/393136987 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48004b20 con 0x5557f3bb4f80 2024-03-19T01:17:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.878+0000 7f7f639db740 1 -- 172.21.15.69:0/393136987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb4f80 msgr2=0x5557f3be7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.878+0000 7f7f639db740 1 --2- 172.21.15.69:0/393136987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb4f80 0x5557f3be7450 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009a70 tx=0x7f7f48009f30 comp rx=0 tx=0).stop 2024-03-19T01:17:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.878+0000 7f7f639db740 1 -- 172.21.15.69:0/393136987 shutdown_connections 2024-03-19T01:17:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.878+0000 7f7f639db740 1 --2- 172.21.15.69:0/393136987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bc6900 0x5557f3bf27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.878+0000 7f7f639db740 1 --2- 172.21.15.69:0/393136987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb4f80 0x5557f3be7450 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.878+0000 7f7f639db740 1 --2- 172.21.15.69:0/393136987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb45d0 0x5557f3bae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.878+0000 7f7f639db740 1 -- 172.21.15.69:0/393136987 >> 172.21.15.69:0/393136987 conn(0x5557f3baf320 msgr2=0x5557f3badaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.878+0000 7f7f639db740 1 -- 172.21.15.69:0/393136987 shutdown_connections 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.878+0000 7f7f639db740 1 -- 172.21.15.69:0/393136987 wait complete. 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.878+0000 7f7f639db740 1 -- start start 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.878+0000 7f7f639db740 1 Processor -- start 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f639db740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 0x5557f3bf2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f639db740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb4f80 0x5557f3bef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f639db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bc6900 0x5557f3befe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f639db740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1ae0f0 con 0x5557f3bc6900 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f639db740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1ad9f0 con 0x5557f3bb4f80 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f639db740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1aaae0 con 0x5557f3bb45d0 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f639db740 10 client.0 _ll_register_callbacks cb 0x7f7f54013cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f59e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 0x5557f3bf2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f59400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb4f80 0x5557f3bef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f59e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 0x5557f3bf2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44910/0 (socket says 172.21.15.69:44910) 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f59e00640 1 -- 172.21.15.69:0/1945728425 learned_addr learned my addr 172.21.15.69:0/1945728425 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.879+0000 7f7f5a800640 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bc6900 0x5557f3befe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.880+0000 7f7f59e00640 1 -- 172.21.15.69:0/1945728425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb4f80 msgr2=0x5557f3bef7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.880+0000 7f7f59e00640 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb4f80 0x5557f3bef7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.880+0000 7f7f59e00640 1 -- 172.21.15.69:0/1945728425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bc6900 msgr2=0x5557f3befe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.880+0000 7f7f59e00640 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bc6900 0x5557f3befe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.880+0000 7f7f59e00640 1 -- 172.21.15.69:0/1945728425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f3c45450 con 0x5557f3bb45d0 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.880+0000 7f7f59e00640 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 0x5557f3bf2bb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c02c6d0 tx=0x7f7f4c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89906bc82ae8d876 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.880+0000 7f7f3b400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.880+0000 7f7f3b400640 1 -- 172.21.15.69:0/1945728425 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c051040 con 0x5557f3bb45d0 2024-03-19T01:17:43.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.880+0000 7f7f3b400640 1 -- 172.21.15.69:0/1945728425 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f7f4c0042f0 con 0x5557f3bb45d0 2024-03-19T01:17:43.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.880+0000 7f7f639db740 1 -- 172.21.15.69:0/1945728425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f4800fe60 con 0x5557f3bb45d0 2024-03-19T01:17:43.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.881+0000 7f7f3b400640 1 -- 172.21.15.69:0/1945728425 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7f4c069050 con 0x5557f3bb45d0 2024-03-19T01:17:43.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.881+0000 7f7f639db740 10 client.27187 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:43.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.881+0000 7f7f639db740 1 -- 172.21.15.69:0/1945728425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5557f3c464f0 con 0x5557f3bb45d0 2024-03-19T01:17:43.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.881+0000 7f7f3b400640 1 -- 172.21.15.69:0/1945728425 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4800fe60 con 0x5557f3bb45d0 2024-03-19T01:17:43.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.881+0000 7f7f3b400640 1 -- 172.21.15.69:0/1945728425 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7f7f4c059360 con 0x5557f3bb45d0 2024-03-19T01:17:43.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.881+0000 7f7f639db740 10 client.27187 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:43.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.881+0000 7f7f639db740 10 client.27187 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.881+0000 7f7f639db740 20 client.27187 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.881+0000 7f7f639db740 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557f3c46820 0x5557f3c48c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.881+0000 7f7f639db740 4 client.27187 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.881+0000 7f7f639db740 1 -- 172.21.15.69:0/1945728425 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557f3b20c10 con 0x5557f3c46820 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.882+0000 7f7f59400640 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557f3c46820 0x5557f3c48c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.882+0000 7f7f59400640 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557f3c46820 0x5557f3c48c10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b1f36ac9000d34e server_cookie=a4757693a5a47705 in_seq=0 out_seq=0 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.882+0000 7f7f3b400640 10 client.27187 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f3b400640 1 -- 172.21.15.69:0/1945728425 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557f3b20c10 con 0x5557f3c46820 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f3b400640 10 client.27187 handle_command_reply: tid=0 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 1 client.27187 shutdown 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 1 -- 172.21.15.69:0/1945728425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 msgr2=0x5557f3bf2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 0x5557f3bf2bb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c02c6d0 tx=0x7f7f4c009260 comp rx=0 tx=0).stop 2024-03-19T01:17:43.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 1 -- 172.21.15.69:0/1945728425 shutdown_connections 2024-03-19T01:17:43.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557f3bc6900 0x5557f3befe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557f3c46820 0x5557f3c48c10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557f3bb4f80 0x5557f3bef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 1 --2- 172.21.15.69:0/1945728425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557f3bb45d0 0x5557f3bf2bb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f3b400640 0 client.27187 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:17:43.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 1 -- 172.21.15.69:0/1945728425 >> 172.21.15.69:0/1945728425 conn(0x5557f3baf320 msgr2=0x5557f3bae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 1 -- 172.21.15.69:0/1945728425 shutdown_connections 2024-03-19T01:17:43.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 1 -- 172.21.15.69:0/1945728425 wait complete. 2024-03-19T01:17:43.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.883+0000 7f7f639db740 20 client.27187 trim_cache size 0 max 16384 2024-03-19T01:17:43.887 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:17:43.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.884+0000 7f7f5b200640 1 -- 172.21.15.69:0/1930105408 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f400a2a70 msgr2=0x7f7f400a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.884+0000 7f7f5b200640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f400a2a70 0x7f7f400a4f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c02c8b0 tx=0x7f7f4c03c000 comp rx=0 tx=0).stop 2024-03-19T01:17:43.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.884+0000 7f7f5b200640 1 -- 172.21.15.69:0/1930105408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5c10bc70 msgr2=0x7f7f5c19c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:43.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.884+0000 7f7f5b200640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5c10bc70 0x7f7f5c19c820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009fd0 tx=0x7f7f5400ef10 comp rx=0 tx=0).stop 2024-03-19T01:17:43.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.884+0000 7f7f5b200640 1 -- 172.21.15.69:0/1930105408 shutdown_connections 2024-03-19T01:17:43.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.884+0000 7f7f5b200640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f400a2a70 0x7f7f400a4f30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.884+0000 7f7f5b200640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5c10bc70 0x7f7f5c19c820 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.884+0000 7f7f5b200640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c103170 0x7f7f5c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.884+0000 7f7f5b200640 1 --2- 172.21.15.69:0/1930105408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f5c102850 0x7f7f5c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:43.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.884+0000 7f7f5b200640 1 -- 172.21.15.69:0/1930105408 >> 172.21.15.69:0/1930105408 conn(0x7f7f5c0fc070 msgr2=0x7f7f5c0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:43.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.884+0000 7f7f5b200640 1 -- 172.21.15.69:0/1930105408 shutdown_connections 2024-03-19T01:17:43.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:43.885+0000 7f7f5b200640 1 -- 172.21.15.69:0/1930105408 wait complete. 2024-03-19T01:17:43.908 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:17:44.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.136+0000 7f7c6a400640 1 -- 172.21.15.69:0/3612382164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64100460 msgr2=0x7f7c64100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:44.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.136+0000 7f7c6a400640 1 --2- 172.21.15.69:0/3612382164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64100460 0x7f7c64100840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009900 tx=0x7f7c5802bc30 comp rx=0 tx=0).stop 2024-03-19T01:17:44.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.136+0000 7f7c6a400640 1 -- 172.21.15.69:0/3612382164 shutdown_connections 2024-03-19T01:17:44.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.136+0000 7f7c6a400640 1 --2- 172.21.15.69:0/3612382164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c6410e010 0x7f7c64110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.136+0000 7f7c6a400640 1 --2- 172.21.15.69:0/3612382164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c64100e10 0x7f7c6410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.136+0000 7f7c6a400640 1 --2- 172.21.15.69:0/3612382164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64100460 0x7f7c64100840 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.136+0000 7f7c6a400640 1 -- 172.21.15.69:0/3612382164 >> 172.21.15.69:0/3612382164 conn(0x7f7c640fc070 msgr2=0x7f7c640fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:44.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.136+0000 7f7c6a400640 1 -- 172.21.15.69:0/3612382164 shutdown_connections 2024-03-19T01:17:44.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.136+0000 7f7c6a400640 1 -- 172.21.15.69:0/3612382164 wait complete. 2024-03-19T01:17:44.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.138+0000 7f7c6a400640 1 Processor -- start 2024-03-19T01:17:44.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.138+0000 7f7c6a400640 1 -- start start 2024-03-19T01:17:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.138+0000 7f7c6a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c64100460 0x7f7c6419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.138+0000 7f7c6a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c64100e10 0x7f7c6419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.138+0000 7f7c6a400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c6410e010 0x7f7c6419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.138+0000 7f7c6a400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c64113cd0 con 0x7f7c64100e10 2024-03-19T01:17:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.138+0000 7f7c6a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c64113b50 con 0x7f7c64100460 2024-03-19T01:17:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.138+0000 7f7c6a400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7c64113e50 con 0x7f7c6410e010 2024-03-19T01:17:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.139+0000 7f7c69000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c64100460 0x7f7c6419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.139+0000 7f7c69000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c64100460 0x7f7c6419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40576/0 (socket says 172.21.15.69:40576) 2024-03-19T01:17:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.139+0000 7f7c69000640 1 -- 172.21.15.69:0/1784829935 learned_addr learned my addr 172.21.15.69:0/1784829935 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.139+0000 7f7c69a00640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c6410e010 0x7f7c6419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.139+0000 7f7c63e00640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c64100e10 0x7f7c6419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.139+0000 7f7c69000640 1 -- 172.21.15.69:0/1784829935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c6410e010 msgr2=0x7f7c6419ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.139+0000 7f7c69000640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c6410e010 0x7f7c6419ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.139+0000 7f7c69000640 1 -- 172.21.15.69:0/1784829935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c64100e10 msgr2=0x7f7c6419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.139+0000 7f7c69000640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c64100e10 0x7f7c6419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.139+0000 7f7c69000640 1 -- 172.21.15.69:0/1784829935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6419f140 con 0x7f7c64100460 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.139+0000 7f7c63e00640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c64100e10 0x7f7c6419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.140+0000 7f7c69000640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c64100460 0x7f7c6419a150 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7c5802c240 tx=0x7f7c580042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.140+0000 7f7c61600640 1 -- 172.21.15.69:0/1784829935 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58043020 con 0x7f7c64100460 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.140+0000 7f7c61600640 1 -- 172.21.15.69:0/1784829935 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c580054e0 con 0x7f7c64100460 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.140+0000 7f7c61600640 1 -- 172.21.15.69:0/1784829935 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58003b30 con 0x7f7c64100460 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.140+0000 7f7c6a400640 1 -- 172.21.15.69:0/1784829935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6419f3d0 con 0x7f7c64100460 2024-03-19T01:17:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.140+0000 7f7c6a400640 1 -- 172.21.15.69:0/1784829935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c641a6cc0 con 0x7f7c64100460 2024-03-19T01:17:44.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.144+0000 7f7c6e81e740 1 Processor -- start 2024-03-19T01:17:44.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.144+0000 7f7c6e81e740 1 -- start start 2024-03-19T01:17:44.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.144+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f61fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.144+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f62510 0x558a64f6abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.144+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f6b0f0 0x558a64f6d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.144+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558a64ec6380 con 0x558a64f6b0f0 2024-03-19T01:17:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.144+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558a64e6ed20 con 0x558a64f61bf0 2024-03-19T01:17:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.144+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558a64ec2420 con 0x558a64f62510 2024-03-19T01:17:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.145+0000 7f7c69a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f61fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.145+0000 7f7c69a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f61fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40590/0 (socket says 172.21.15.69:40590) 2024-03-19T01:17:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.145+0000 7f7c69a00640 1 -- 172.21.15.69:0/2796160758 learned_addr learned my addr 172.21.15.69:0/2796160758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:44.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.145+0000 7f7c61600640 1 -- 172.21.15.69:0/1784829935 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f7c58046020 con 0x7f7c64100460 2024-03-19T01:17:44.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.145+0000 7f7c63e00640 1 --2- 172.21.15.69:0/2796160758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f6b0f0 0x558a64f6d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.145+0000 7f7c69a00640 1 -- 172.21.15.69:0/2796160758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f62510 msgr2=0x558a64f6abb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.145+0000 7f7c69a00640 1 --2- 172.21.15.69:0/2796160758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f62510 0x558a64f6abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.145+0000 7f7c69a00640 1 -- 172.21.15.69:0/2796160758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f6b0f0 msgr2=0x558a64f6d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.145+0000 7f7c69a00640 1 --2- 172.21.15.69:0/2796160758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f6b0f0 0x558a64f6d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.145+0000 7f7c69a00640 1 -- 172.21.15.69:0/2796160758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a64bda3f0 con 0x558a64f61bf0 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.146+0000 7f7c69a00640 1 --2- 172.21.15.69:0/2796160758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f61fd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009ec0 tx=0x7f7c5c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b184d30a4aef4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.146+0000 7f7c56000640 1 -- 172.21.15.69:0/2796160758 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c004d20 con 0x558a64f61bf0 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.146+0000 7f7c56000640 1 -- 172.21.15.69:0/2796160758 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c5c003e50 con 0x558a64f61bf0 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.146+0000 7f7c56000640 1 -- 172.21.15.69:0/2796160758 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c005280 con 0x558a64f61bf0 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.146+0000 7f7c61600640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7c4c0a29f0 0x7f7c4c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.147+0000 7f7c6e81e740 1 -- 172.21.15.69:0/2796160758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 msgr2=0x558a64f61fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.147+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/2796160758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f61fd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009ec0 tx=0x7f7c5c0107a0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.147+0000 7f7c6e81e740 1 -- 172.21.15.69:0/2796160758 shutdown_connections 2024-03-19T01:17:44.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.147+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/2796160758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f6b0f0 0x558a64f6d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.147+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/2796160758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f62510 0x558a64f6abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.147+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/2796160758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f61fd0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.147+0000 7f7c6e81e740 1 -- 172.21.15.69:0/2796160758 >> 172.21.15.69:0/2796160758 conn(0x558a64f5c320 msgr2=0x558a64f5c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:44.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.147+0000 7f7c69000640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7c4c0a29f0 0x7f7c4c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.147+0000 7f7c6e81e740 1 -- 172.21.15.69:0/2796160758 shutdown_connections 2024-03-19T01:17:44.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.147+0000 7f7c61600640 1 -- 172.21.15.69:0/1784829935 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f7c580f1260 con 0x7f7c64100460 2024-03-19T01:17:44.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.148+0000 7f7c6e81e740 1 -- 172.21.15.69:0/2796160758 wait complete. 2024-03-19T01:17:44.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.148+0000 7f7c6e81e740 1 -- start start 2024-03-19T01:17:44.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.148+0000 7f7c6e81e740 1 Processor -- start 2024-03-19T01:17:44.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.149+0000 7f7c69000640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7c4c0a29f0 0x7f7c4c0a4eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x558a64f616c0 tx=0x7f7c5802bc30 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:44.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.149+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f61bf0 0x558a64f68120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.149+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 0x558a64f68660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.149+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f6b0f0 0x558a64f68ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.149+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558a64f6dee0 con 0x558a64f62510 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.149+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558a64e82160 con 0x558a64f6b0f0 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.149+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558a64f6e060 con 0x558a64f61bf0 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.149+0000 7f7c6e81e740 10 client.0 _ll_register_callbacks cb 0x558a64ebe170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 0x558a64f68660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c69a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f61bf0 0x558a64f68120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 0x558a64f68660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45082/0 (socket says 172.21.15.69:45082) 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c63e00640 1 -- 172.21.15.69:0/265468712 learned_addr learned my addr 172.21.15.69:0/265468712 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c69a00640 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f6b0f0 0x558a64f68ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c63e00640 1 -- 172.21.15.69:0/265468712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f61bf0 msgr2=0x558a64f68120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c63e00640 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f61bf0 0x558a64f68120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c63e00640 1 -- 172.21.15.69:0/265468712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f6b0f0 msgr2=0x558a64f68ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c63e00640 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f6b0f0 0x558a64f68ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c63e00640 1 -- 172.21.15.69:0/265468712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a64ca47d0 con 0x558a64f62510 2024-03-19T01:17:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c63e00640 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 0x558a64f68660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7c50011830 tx=0x7f7c50011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7aff5e703b60990 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:44.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.150+0000 7f7c4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:17:44.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.151+0000 7f7c4aa00640 1 -- 172.21.15.69:0/265468712 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50004440 con 0x558a64f62510 2024-03-19T01:17:44.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.151+0000 7f7c4aa00640 1 -- 172.21.15.69:0/265468712 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f7c5000f9a0 con 0x558a64f62510 2024-03-19T01:17:44.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.151+0000 7f7c6e81e740 1 -- 172.21.15.69:0/265468712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c5c003e50 con 0x558a64f62510 2024-03-19T01:17:44.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.152+0000 7f7c4aa00640 1 -- 172.21.15.69:0/265468712 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7c5000fbe0 con 0x558a64f62510 2024-03-19T01:17:44.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.152+0000 7f7c6e81e740 10 client.16953 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:44.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.152+0000 7f7c6e81e740 1 -- 172.21.15.69:0/265468712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558a64f7a1e0 con 0x558a64f62510 2024-03-19T01:17:44.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.152+0000 7f7c4aa00640 1 -- 172.21.15.69:0/265468712 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c5c003e50 con 0x558a64f62510 2024-03-19T01:17:44.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.152+0000 7f7c4aa00640 1 -- 172.21.15.69:0/265468712 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7f7c50018030 con 0x558a64f62510 2024-03-19T01:17:44.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.152+0000 7f7c6e81e740 10 client.16953 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:44.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.152+0000 7f7c6e81e740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:44.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.153+0000 7f7c6e81e740 20 client.16953 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:44.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.153+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558a64f7b7c0 0x558a64f7dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.153+0000 7f7c6e81e740 4 client.16953 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:44.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.153+0000 7f7c6e81e740 1 -- 172.21.15.69:0/265468712 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a64ed38c0 con 0x558a64f7b7c0 2024-03-19T01:17:44.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.153+0000 7f7c69a00640 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558a64f7b7c0 0x558a64f7dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.154+0000 7f7c4aa00640 10 client.16953 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:44.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.154+0000 7f7c69a00640 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558a64f7b7c0 0x558a64f7dbb0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8f7758e18b26ff7 server_cookie=6cf6d58ccea299c6 in_seq=0 out_seq=0 2024-03-19T01:17:44.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c4aa00640 1 -- 172.21.15.69:0/265468712 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558a64ed38c0 con 0x558a64f7b7c0 2024-03-19T01:17:44.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c4aa00640 10 client.16953 handle_command_reply: tid=0 2024-03-19T01:17:44.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c6e81e740 1 client.16953 shutdown 2024-03-19T01:17:44.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c6e81e740 1 -- 172.21.15.69:0/265468712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 msgr2=0x558a64f68660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:44.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 0x558a64f68660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7c50011830 tx=0x7f7c50011cf0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c6e81e740 1 -- 172.21.15.69:0/265468712 shutdown_connections 2024-03-19T01:17:44.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558a64f7b7c0 0x558a64f7dbb0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f6b0f0 0x558a64f68ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 0x558a64f68660 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/265468712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f61bf0 0x558a64f68120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c6e81e740 1 -- 172.21.15.69:0/265468712 >> 172.21.15.69:0/265468712 conn(0x558a64f5c320 msgr2=0x558a64f5e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:44.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.155+0000 7f7c4aa00640 0 client.16953 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:44.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.156+0000 7f7c6e81e740 1 -- 172.21.15.69:0/265468712 shutdown_connections 2024-03-19T01:17:44.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.156+0000 7f7c6e81e740 1 -- 172.21.15.69:0/265468712 wait complete. 2024-03-19T01:17:44.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.156+0000 7f7c6e81e740 20 client.16953 trim_cache size 0 max 16384 2024-03-19T01:17:44.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.178+0000 7f7c6e81e740 1 Processor -- start 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.178+0000 7f7c6e81e740 1 -- start start 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.178+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f76c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.178+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f62510 0x558a64f68a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f6b0f0 0x558a64f83460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641bdb20 con 0x558a64f6b0f0 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641bf810 con 0x558a64f61bf0 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641be1f0 con 0x558a64f62510 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f62510 0x558a64f68a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c69a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f76c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f62510 0x558a64f68a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44964/0 (socket says 172.21.15.69:44964) 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c63e00640 1 -- 172.21.15.69:0/278804698 learned_addr learned my addr 172.21.15.69:0/278804698 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:44.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c69a00640 1 --2- 172.21.15.69:0/278804698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f6b0f0 0x558a64f83460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c63e00640 1 -- 172.21.15.69:0/278804698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 msgr2=0x558a64f76c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:44.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c63e00640 1 --2- 172.21.15.69:0/278804698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f76c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.179+0000 7f7c63e00640 1 -- 172.21.15.69:0/278804698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f6b0f0 msgr2=0x558a64f83460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:44.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.180+0000 7f7c63e00640 1 --2- 172.21.15.69:0/278804698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f6b0f0 0x558a64f83460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.180+0000 7f7c63e00640 1 -- 172.21.15.69:0/278804698 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a64f6d850 con 0x558a64f62510 2024-03-19T01:17:44.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.180+0000 7f7c69a00640 1 --2- 172.21.15.69:0/278804698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f6b0f0 0x558a64f83460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:44.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.180+0000 7f7c63e00640 1 --2- 172.21.15.69:0/278804698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f62510 0x558a64f68a40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000f790 tx=0x7f7c50008c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c35521fb8702de0d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:44.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.180+0000 7f7c55600640 1 -- 172.21.15.69:0/278804698 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000fbe0 con 0x558a64f62510 2024-03-19T01:17:44.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.180+0000 7f7c55600640 1 -- 172.21.15.69:0/278804698 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c5000f9a0 con 0x558a64f62510 2024-03-19T01:17:44.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.181+0000 7f7c55600640 1 -- 172.21.15.69:0/278804698 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50003b40 con 0x558a64f62510 2024-03-19T01:17:44.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.182+0000 7f7c6e81e740 1 -- 172.21.15.69:0/278804698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f62510 msgr2=0x558a64f68a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:44.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.182+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/278804698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f62510 0x558a64f68a40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000f790 tx=0x7f7c50008c80 comp rx=0 tx=0).stop 2024-03-19T01:17:44.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.182+0000 7f7c6e81e740 1 -- 172.21.15.69:0/278804698 shutdown_connections 2024-03-19T01:17:44.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.182+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/278804698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f6b0f0 0x558a64f83460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.182+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/278804698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f62510 0x558a64f68a40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.182+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/278804698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f76c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.182+0000 7f7c6e81e740 1 -- 172.21.15.69:0/278804698 >> 172.21.15.69:0/278804698 conn(0x558a64f5c320 msgr2=0x558a64f5e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:44.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.182+0000 7f7c6e81e740 1 -- 172.21.15.69:0/278804698 shutdown_connections 2024-03-19T01:17:44.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.183+0000 7f7c6e81e740 1 -- 172.21.15.69:0/278804698 wait complete. 2024-03-19T01:17:44.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.183+0000 7f7c6e81e740 1 -- start start 2024-03-19T01:17:44.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.183+0000 7f7c6e81e740 1 Processor -- start 2024-03-19T01:17:44.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.184+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f64f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.184+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 0x558a64f654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.184+0000 7f7c6e81e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f6b0f0 0x558a64f65a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.184+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641bf080 con 0x558a64f62510 2024-03-19T01:17:44.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.184+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641be980 con 0x558a64f61bf0 2024-03-19T01:17:44.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.184+0000 7f7c6e81e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641bba50 con 0x558a64f6b0f0 2024-03-19T01:17:44.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.184+0000 7f7c6e81e740 10 client.0 _ll_register_callbacks cb 0x7f7c50011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:44.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.184+0000 7f7c69a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f64f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.184+0000 7f7c69a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f64f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40610/0 (socket says 172.21.15.69:40610) 2024-03-19T01:17:44.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.184+0000 7f7c69a00640 1 -- 172.21.15.69:0/3997177969 learned_addr learned my addr 172.21.15.69:0/3997177969 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:44.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.184+0000 7f7c69a00640 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f6b0f0 0x558a64f65a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.185+0000 7f7c63e00640 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 0x558a64f654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.185+0000 7f7c69a00640 1 -- 172.21.15.69:0/3997177969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f6b0f0 msgr2=0x558a64f65a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:44.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.185+0000 7f7c69a00640 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f6b0f0 0x558a64f65a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.185+0000 7f7c69a00640 1 -- 172.21.15.69:0/3997177969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 msgr2=0x558a64f654c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:44.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.185+0000 7f7c69a00640 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 0x558a64f654c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.185+0000 7f7c69a00640 1 -- 172.21.15.69:0/3997177969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a64fe0c90 con 0x558a64f61bf0 2024-03-19T01:17:44.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.185+0000 7f7c69a00640 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f64f80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c004490 tx=0x7f7c5c013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=843d4f0e192f573c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:44.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.185+0000 7f7c4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:17:44.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.186+0000 7f7c4aa00640 1 -- 172.21.15.69:0/3997177969 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c02f380 con 0x558a64f61bf0 2024-03-19T01:17:44.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.186+0000 7f7c4aa00640 1 -- 172.21.15.69:0/3997177969 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f7c5c02f520 con 0x558a64f61bf0 2024-03-19T01:17:44.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.186+0000 7f7c6e81e740 1 -- 172.21.15.69:0/3997177969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c5000f9a0 con 0x558a64f61bf0 2024-03-19T01:17:44.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.186+0000 7f7c4aa00640 1 -- 172.21.15.69:0/3997177969 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7c5c042050 con 0x558a64f61bf0 2024-03-19T01:17:44.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.186+0000 7f7c6e81e740 10 client.27245 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:44.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.186+0000 7f7c6e81e740 1 -- 172.21.15.69:0/3997177969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558a64fe1f70 con 0x558a64f61bf0 2024-03-19T01:17:44.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.187+0000 7f7c4aa00640 1 -- 172.21.15.69:0/3997177969 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c5000f9a0 con 0x558a64f61bf0 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.187+0000 7f7c4aa00640 1 -- 172.21.15.69:0/3997177969 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7f7c5c019160 con 0x558a64f61bf0 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.187+0000 7f7c6e81e740 10 client.27245 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.187+0000 7f7c6e81e740 10 client.27245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.187+0000 7f7c6e81e740 20 client.27245 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.187+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558a64fe2590 0x558a64fe49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.187+0000 7f7c6e81e740 4 client.27245 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.187+0000 7f7c6e81e740 1 -- 172.21.15.69:0/3997177969 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a64ecdc10 con 0x558a64fe2590 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.187+0000 7f7c63e00640 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558a64fe2590 0x558a64fe49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.188+0000 7f7c63e00640 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558a64fe2590 0x558a64fe49b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b3387259b14396 server_cookie=40a16cd7495cc40a in_seq=0 out_seq=0 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.188+0000 7f7c4aa00640 10 client.27245 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.188+0000 7f7c4aa00640 1 -- 172.21.15.69:0/3997177969 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a64ecdc10 con 0x558a64fe2590 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.188+0000 7f7c4aa00640 10 client.27245 handle_command_reply: tid=0 2024-03-19T01:17:44.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.188+0000 7f7c6e81e740 1 client.27245 shutdown 2024-03-19T01:17:44.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.188+0000 7f7c6e81e740 1 -- 172.21.15.69:0/3997177969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 msgr2=0x558a64f64f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:44.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.188+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f64f80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c004490 tx=0x7f7c5c013040 comp rx=0 tx=0).stop 2024-03-19T01:17:44.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.189+0000 7f7c6e81e740 1 -- 172.21.15.69:0/3997177969 shutdown_connections 2024-03-19T01:17:44.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.189+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558a64fe2590 0x558a64fe49b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.189+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a64f6b0f0 0x558a64f65a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.189+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558a64f62510 0x558a64f654c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.189+0000 7f7c6e81e740 1 --2- 172.21.15.69:0/3997177969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558a64f61bf0 0x558a64f64f80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.189+0000 7f7c6e81e740 1 -- 172.21.15.69:0/3997177969 >> 172.21.15.69:0/3997177969 conn(0x558a64f5c320 msgr2=0x558a64f5aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:44.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.190+0000 7f7c4aa00640 0 client.27245 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:44.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.190+0000 7f7c6e81e740 1 -- 172.21.15.69:0/3997177969 shutdown_connections 2024-03-19T01:17:44.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.190+0000 7f7c6e81e740 1 -- 172.21.15.69:0/3997177969 wait complete. 2024-03-19T01:17:44.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.190+0000 7f7c6e81e740 20 client.27245 trim_cache size 0 max 16384 2024-03-19T01:17:44.192 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:17:44.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 -- 172.21.15.69:0/1784829935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7c4c0a29f0 msgr2=0x7f7c4c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:44.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7c4c0a29f0 0x7f7c4c0a4eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x558a64f616c0 tx=0x7f7c5802bc30 comp rx=0 tx=0).stop 2024-03-19T01:17:44.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 -- 172.21.15.69:0/1784829935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c64100460 msgr2=0x7f7c6419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:44.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c64100460 0x7f7c6419a150 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7c5802c240 tx=0x7f7c580042f0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 -- 172.21.15.69:0/1784829935 shutdown_connections 2024-03-19T01:17:44.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c6410e010 0x7f7c6419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c64100e10 0x7f7c6419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7c4c0a29f0 0x7f7c4c0a4eb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 --2- 172.21.15.69:0/1784829935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c64100460 0x7f7c6419a150 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:44.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 -- 172.21.15.69:0/1784829935 >> 172.21.15.69:0/1784829935 conn(0x7f7c640fc070 msgr2=0x7f7c640ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:44.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 -- 172.21.15.69:0/1784829935 shutdown_connections 2024-03-19T01:17:44.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:44.191+0000 7f7c6a400640 1 -- 172.21.15.69:0/1784829935 wait complete. 2024-03-19T01:17:45.213 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:17:45.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:17:45.214 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:17:45.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.442+0000 7f6b8f600640 1 -- 172.21.15.69:0/327934281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b881077c0 msgr2=0x7f6b88107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.442+0000 7f6b8f600640 1 --2- 172.21.15.69:0/327934281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b881077c0 0x7f6b88107ba0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009930 tx=0x7f6b7c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:17:45.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.442+0000 7f6b8f600640 1 -- 172.21.15.69:0/327934281 shutdown_connections 2024-03-19T01:17:45.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.442+0000 7f6b8f600640 1 --2- 172.21.15.69:0/327934281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b88102170 0x7f6b8810e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.442+0000 7f6b8f600640 1 --2- 172.21.15.69:0/327934281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b881017b0 0x7f6b88101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.442+0000 7f6b8f600640 1 --2- 172.21.15.69:0/327934281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b881077c0 0x7f6b88107ba0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.442+0000 7f6b8f600640 1 -- 172.21.15.69:0/327934281 >> 172.21.15.69:0/327934281 conn(0x7f6b88078c20 msgr2=0x7f6b88079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.442+0000 7f6b8f600640 1 -- 172.21.15.69:0/327934281 shutdown_connections 2024-03-19T01:17:45.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.442+0000 7f6b8f600640 1 -- 172.21.15.69:0/327934281 wait complete. 2024-03-19T01:17:45.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.443+0000 7f6b8f600640 1 Processor -- start 2024-03-19T01:17:45.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.443+0000 7f6b8f600640 1 -- start start 2024-03-19T01:17:45.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.444+0000 7f6b8f600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b881017b0 0x7f6b8819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.444+0000 7f6b8f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b88102170 0x7f6b8819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.444+0000 7f6b8f600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b881077c0 0x7f6b8819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.444+0000 7f6b8f600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88111c20 con 0x7f6b881017b0 2024-03-19T01:17:45.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.444+0000 7f6b8f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88111aa0 con 0x7f6b88102170 2024-03-19T01:17:45.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.444+0000 7f6b8f600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88111da0 con 0x7f6b881077c0 2024-03-19T01:17:45.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.444+0000 7f6b8cc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b881017b0 0x7f6b8819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.444+0000 7f6b87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b88102170 0x7f6b8819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.444+0000 7f6b8cc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b881017b0 0x7f6b8819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45124/0 (socket says 172.21.15.69:45124) 2024-03-19T01:17:45.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.444+0000 7f6b8cc00640 1 -- 172.21.15.69:0/640371554 learned_addr learned my addr 172.21.15.69:0/640371554 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:45.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.444+0000 7f6b8d600640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b881077c0 0x7f6b8819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.445+0000 7f6b8d600640 1 -- 172.21.15.69:0/640371554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b88102170 msgr2=0x7f6b8819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.445+0000 7f6b8d600640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b88102170 0x7f6b8819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.445+0000 7f6b8d600640 1 -- 172.21.15.69:0/640371554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b881017b0 msgr2=0x7f6b8819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.445+0000 7f6b8d600640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b881017b0 0x7f6b8819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.445+0000 7f6b8d600640 1 -- 172.21.15.69:0/640371554 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8819f2f0 con 0x7f6b881077c0 2024-03-19T01:17:45.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.445+0000 7f6b87e00640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b88102170 0x7f6b8819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:45.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.445+0000 7f6b8cc00640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b881017b0 0x7f6b8819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:17:45.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.445+0000 7f6b8d600640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b881077c0 0x7f6b8819ebc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009fd0 tx=0x7f6b8000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.445+0000 7f6b85600640 1 -- 172.21.15.69:0/640371554 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000ce20 con 0x7f6b881077c0 2024-03-19T01:17:45.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.446+0000 7f6b85600640 1 -- 172.21.15.69:0/640371554 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b80005020 con 0x7f6b881077c0 2024-03-19T01:17:45.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.446+0000 7f6b85600640 1 -- 172.21.15.69:0/640371554 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80010720 con 0x7f6b881077c0 2024-03-19T01:17:45.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.446+0000 7f6b8f600640 1 -- 172.21.15.69:0/640371554 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8819f5e0 con 0x7f6b881077c0 2024-03-19T01:17:45.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.446+0000 7f6b8f600640 1 -- 172.21.15.69:0/640371554 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b881a6e70 con 0x7f6b881077c0 2024-03-19T01:17:45.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.449+0000 7f6b85600640 1 -- 172.21.15.69:0/640371554 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f6b80004b20 con 0x7f6b881077c0 2024-03-19T01:17:45.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.451+0000 7f6b85600640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6b700ab350 0x7f6b700ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.451+0000 7f6b9237c740 1 Processor -- start 2024-03-19T01:17:45.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.451+0000 7f6b85600640 1 -- 172.21.15.69:0/640371554 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f6b800cf780 con 0x7f6b881077c0 2024-03-19T01:17:45.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.451+0000 7f6b8cc00640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6b700ab350 0x7f6b700ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.451+0000 7f6b9237c740 1 -- start start 2024-03-19T01:17:45.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc76aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 0x55afbc773ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 0x55afbc7763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b9237c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55afbc6cdfb0 con 0x55afbc76b420 2024-03-19T01:17:45.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b9237c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afbc676ab0 con 0x55afbc774000 2024-03-19T01:17:45.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b9237c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55afbc6ca050 con 0x55afbc76ab00 2024-03-19T01:17:45.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b8d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 0x55afbc7763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b8d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 0x55afbc7763f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40634/0 (socket says 172.21.15.69:40634) 2024-03-19T01:17:45.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b8d600640 1 -- 172.21.15.69:0/1735797795 learned_addr learned my addr 172.21.15.69:0/1735797795 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:45.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b87e00640 1 --2- 172.21.15.69:0/1735797795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 0x55afbc773ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b8d600640 1 -- 172.21.15.69:0/1735797795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 msgr2=0x55afbc76aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b8cc00640 1 --2- 172.21.15.69:0/1735797795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc76aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b8d600640 1 --2- 172.21.15.69:0/1735797795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc76aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b8d600640 1 -- 172.21.15.69:0/1735797795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 msgr2=0x55afbc773ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b8d600640 1 --2- 172.21.15.69:0/1735797795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 0x55afbc773ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b8d600640 1 -- 172.21.15.69:0/1735797795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afbc694e00 con 0x55afbc774000 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b87e00640 1 --2- 172.21.15.69:0/1735797795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 0x55afbc773ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b8cc00640 1 --2- 172.21.15.69:0/1735797795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc76aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.452+0000 7f6b8cc00640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6b700ab350 0x7f6b700ad810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0096c0 tx=0x7f6b7c02df90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.453+0000 7f6b8d600640 1 --2- 172.21.15.69:0/1735797795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 0x55afbc7763f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b80098360 tx=0x7f6b8009b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12251af23bbe2a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.453+0000 7f6b79c00640 1 -- 172.21.15.69:0/1735797795 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b800b3070 con 0x55afbc774000 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.453+0000 7f6b79c00640 1 -- 172.21.15.69:0/1735797795 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b80004e10 con 0x55afbc774000 2024-03-19T01:17:45.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.453+0000 7f6b79c00640 1 -- 172.21.15.69:0/1735797795 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b800ccf00 con 0x55afbc774000 2024-03-19T01:17:45.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.454+0000 7f6b9237c740 1 -- 172.21.15.69:0/1735797795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 msgr2=0x55afbc7763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.454+0000 7f6b9237c740 1 --2- 172.21.15.69:0/1735797795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 0x55afbc7763f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b80098360 tx=0x7f6b8009b990 comp rx=0 tx=0).stop 2024-03-19T01:17:45.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.454+0000 7f6b9237c740 1 -- 172.21.15.69:0/1735797795 shutdown_connections 2024-03-19T01:17:45.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.454+0000 7f6b9237c740 1 --2- 172.21.15.69:0/1735797795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 0x55afbc7763f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.454+0000 7f6b9237c740 1 --2- 172.21.15.69:0/1735797795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 0x55afbc773ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.454+0000 7f6b9237c740 1 --2- 172.21.15.69:0/1735797795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc76aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.454+0000 7f6b9237c740 1 -- 172.21.15.69:0/1735797795 >> 172.21.15.69:0/1735797795 conn(0x55afbc765230 msgr2=0x55afbc765610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.454+0000 7f6b9237c740 1 -- 172.21.15.69:0/1735797795 shutdown_connections 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.454+0000 7f6b9237c740 1 -- 172.21.15.69:0/1735797795 wait complete. 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.455+0000 7f6b9237c740 1 -- start start 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.455+0000 7f6b9237c740 1 Processor -- start 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.456+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76ab00 0x55afbc770fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.456+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76b420 0x55afbc771510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.456+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc774000 0x55afbc771a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.456+0000 7f6b9237c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55afbc776df0 con 0x55afbc774000 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.456+0000 7f6b9237c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afbc689f00 con 0x55afbc76ab00 2024-03-19T01:17:45.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.456+0000 7f6b9237c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55afbc776f70 con 0x55afbc76b420 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.456+0000 7f6b9237c740 10 client.0 _ll_register_callbacks cb 0x55afbc6898d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.456+0000 7f6b8cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76ab00 0x55afbc770fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.456+0000 7f6b8cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76ab00 0x55afbc770fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40638/0 (socket says 172.21.15.69:40638) 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.456+0000 7f6b8cc00640 1 -- 172.21.15.69:0/1241092489 learned_addr learned my addr 172.21.15.69:0/1241092489 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.456+0000 7f6b8d600640 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc774000 0x55afbc771a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b87e00640 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76b420 0x55afbc771510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b8cc00640 1 -- 172.21.15.69:0/1241092489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76b420 msgr2=0x55afbc771510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b8cc00640 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76b420 0x55afbc771510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b8cc00640 1 -- 172.21.15.69:0/1241092489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc774000 msgr2=0x55afbc771a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b8cc00640 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc774000 0x55afbc771a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b8cc00640 1 -- 172.21.15.69:0/1241092489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afbc4aa7e0 con 0x55afbc76ab00 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b8d600640 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc774000 0x55afbc771a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b8cc00640 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76ab00 0x55afbc770fd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0044f0 tx=0x7f6b7c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f373a03f0a705b11 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b6e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:17:45.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b6e000640 1 -- 172.21.15.69:0/1241092489 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c050030 con 0x55afbc76ab00 2024-03-19T01:17:45.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b9237c740 1 -- 172.21.15.69:0/1241092489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b80004e10 con 0x55afbc76ab00 2024-03-19T01:17:45.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.457+0000 7f6b6e000640 1 -- 172.21.15.69:0/1241092489 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f6b7c03a040 con 0x55afbc76ab00 2024-03-19T01:17:45.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.458+0000 7f6b6e000640 1 -- 172.21.15.69:0/1241092489 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6b7c02cc50 con 0x55afbc76ab00 2024-03-19T01:17:45.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.458+0000 7f6b9237c740 10 client.27257 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:45.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.458+0000 7f6b9237c740 1 -- 172.21.15.69:0/1241092489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55afbc50af70 con 0x55afbc76ab00 2024-03-19T01:17:45.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.458+0000 7f6b6e000640 1 -- 172.21.15.69:0/1241092489 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b80004e10 con 0x55afbc76ab00 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.458+0000 7f6b6e000640 1 -- 172.21.15.69:0/1241092489 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7f6b7c04b040 con 0x55afbc76ab00 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.458+0000 7f6b9237c740 10 client.27257 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.458+0000 7f6b9237c740 10 client.27257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.458+0000 7f6b9237c740 20 client.27257 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.458+0000 7f6b9237c740 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afbc784320 0x55afbc786790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.458+0000 7f6b9237c740 4 client.27257 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.458+0000 7f6b9237c740 1 -- 172.21.15.69:0/1241092489 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afbc6db4f0 con 0x55afbc784320 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.459+0000 7f6b87e00640 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afbc784320 0x55afbc786790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.459+0000 7f6b87e00640 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afbc784320 0x55afbc786790 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c48a1c257c7b593 server_cookie=1debdfa3599fb01b in_seq=0 out_seq=0 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.459+0000 7f6b6e000640 10 client.27257 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.460+0000 7f6b6e000640 1 -- 172.21.15.69:0/1241092489 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55afbc6db4f0 con 0x55afbc784320 2024-03-19T01:17:45.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.460+0000 7f6b6e000640 10 client.27257 handle_command_reply: tid=0 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.460+0000 7f6b9237c740 1 client.27257 shutdown 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b9237c740 1 -- 172.21.15.69:0/1241092489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76ab00 msgr2=0x55afbc770fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b9237c740 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76ab00 0x55afbc770fd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0044f0 tx=0x7f6b7c009260 comp rx=0 tx=0).stop 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b9237c740 1 -- 172.21.15.69:0/1241092489 shutdown_connections 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b9237c740 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afbc784320 0x55afbc786790 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b6e000640 0 client.27257 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b9237c740 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc774000 0x55afbc771a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b9237c740 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76b420 0x55afbc771510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b9237c740 1 --2- 172.21.15.69:0/1241092489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76ab00 0x55afbc770fd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b9237c740 1 -- 172.21.15.69:0/1241092489 >> 172.21.15.69:0/1241092489 conn(0x55afbc765230 msgr2=0x55afbc766f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b9237c740 1 -- 172.21.15.69:0/1241092489 shutdown_connections 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b9237c740 1 -- 172.21.15.69:0/1241092489 wait complete. 2024-03-19T01:17:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.461+0000 7f6b9237c740 20 client.27257 trim_cache size 0 max 16384 2024-03-19T01:17:45.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.484+0000 7f6b9237c740 1 Processor -- start 2024-03-19T01:17:45.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.484+0000 7f6b9237c740 1 -- start start 2024-03-19T01:17:45.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc773700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76b420 0x55afbc77fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc774000 0x55afbc78c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b9237c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b881acd50 con 0x55afbc774000 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b9237c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b881aea40 con 0x55afbc76b420 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b9237c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6b881ad420 con 0x55afbc76ab00 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b8cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc773700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76b420 0x55afbc77fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b8cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc773700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45018/0 (socket says 172.21.15.69:45018) 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b8cc00640 1 -- 172.21.15.69:0/3774985850 learned_addr learned my addr 172.21.15.69:0/3774985850 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b8d600640 1 --2- 172.21.15.69:0/3774985850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc774000 0x55afbc78c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b87e00640 1 -- 172.21.15.69:0/3774985850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 msgr2=0x55afbc773700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.485+0000 7f6b87e00640 1 --2- 172.21.15.69:0/3774985850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc773700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.486+0000 7f6b87e00640 1 -- 172.21.15.69:0/3774985850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc774000 msgr2=0x55afbc78c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.486+0000 7f6b87e00640 1 --2- 172.21.15.69:0/3774985850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc774000 0x55afbc78c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.486+0000 7f6b87e00640 1 -- 172.21.15.69:0/3774985850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afbc3a2170 con 0x55afbc76b420 2024-03-19T01:17:45.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.486+0000 7f6b87e00640 1 --2- 172.21.15.69:0/3774985850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76b420 0x55afbc77fbc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55afbc770830 tx=0x7f6b7400b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c75d4094d9040569 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.486+0000 7f6b79200640 1 -- 172.21.15.69:0/3774985850 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74004c60 con 0x55afbc76b420 2024-03-19T01:17:45.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.486+0000 7f6b79200640 1 -- 172.21.15.69:0/3774985850 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b7401f030 con 0x55afbc76b420 2024-03-19T01:17:45.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.486+0000 7f6b79200640 1 -- 172.21.15.69:0/3774985850 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74013610 con 0x55afbc76b420 2024-03-19T01:17:45.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.487+0000 7f6b9237c740 1 -- 172.21.15.69:0/3774985850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76b420 msgr2=0x55afbc77fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.487+0000 7f6b9237c740 1 --2- 172.21.15.69:0/3774985850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76b420 0x55afbc77fbc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55afbc770830 tx=0x7f6b7400b410 comp rx=0 tx=0).stop 2024-03-19T01:17:45.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.487+0000 7f6b9237c740 1 -- 172.21.15.69:0/3774985850 shutdown_connections 2024-03-19T01:17:45.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.487+0000 7f6b9237c740 1 --2- 172.21.15.69:0/3774985850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc774000 0x55afbc78c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.487+0000 7f6b9237c740 1 --2- 172.21.15.69:0/3774985850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc76b420 0x55afbc77fbc0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.487+0000 7f6b9237c740 1 --2- 172.21.15.69:0/3774985850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc773700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.487+0000 7f6b9237c740 1 -- 172.21.15.69:0/3774985850 >> 172.21.15.69:0/3774985850 conn(0x55afbc765230 msgr2=0x55afbc7642a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.488+0000 7f6b9237c740 1 -- 172.21.15.69:0/3774985850 shutdown_connections 2024-03-19T01:17:45.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.488+0000 7f6b9237c740 1 -- 172.21.15.69:0/3774985850 wait complete. 2024-03-19T01:17:45.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.488+0000 7f6b9237c740 1 -- start start 2024-03-19T01:17:45.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.488+0000 7f6b9237c740 1 Processor -- start 2024-03-19T01:17:45.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.489+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc77c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.489+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 0x55afbc77c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.489+0000 7f6b9237c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 0x55afbc7786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.489+0000 7f6b9237c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b881ae2b0 con 0x55afbc76b420 2024-03-19T01:17:45.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.489+0000 7f6b9237c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b881adbb0 con 0x55afbc774000 2024-03-19T01:17:45.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.489+0000 7f6b9237c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6b881aaca0 con 0x55afbc76ab00 2024-03-19T01:17:45.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.489+0000 7f6b9237c740 10 client.0 _ll_register_callbacks cb 0x7f6b7c0041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:45.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.489+0000 7f6b87e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 0x55afbc77c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.489+0000 7f6b87e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 0x55afbc77c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45166/0 (socket says 172.21.15.69:45166) 2024-03-19T01:17:45.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.489+0000 7f6b87e00640 1 -- 172.21.15.69:0/4064041642 learned_addr learned my addr 172.21.15.69:0/4064041642 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:45.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.490+0000 7f6b8d600640 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 0x55afbc7786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.490+0000 7f6b8cc00640 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc77c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.490+0000 7f6b87e00640 1 -- 172.21.15.69:0/4064041642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 msgr2=0x55afbc77c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.490+0000 7f6b87e00640 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc77c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.490+0000 7f6b87e00640 1 -- 172.21.15.69:0/4064041642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 msgr2=0x55afbc7786a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.490+0000 7f6b87e00640 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 0x55afbc7786a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.490+0000 7f6b87e00640 1 -- 172.21.15.69:0/4064041642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afbc778ea0 con 0x55afbc76b420 2024-03-19T01:17:45.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.490+0000 7f6b87e00640 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 0x55afbc77c660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6b74000c00 tx=0x7f6b74005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777261978f027f0f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.490+0000 7f6b6e000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.491+0000 7f6b6e000640 1 -- 172.21.15.69:0/4064041642 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74004020 con 0x55afbc76b420 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.491+0000 7f6b6e000640 1 -- 172.21.15.69:0/4064041642 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7f6b740041c0 con 0x55afbc76b420 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.491+0000 7f6b9237c740 1 -- 172.21.15.69:0/4064041642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b7401f030 con 0x55afbc76b420 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.492+0000 7f6b6e000640 1 -- 172.21.15.69:0/4064041642 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6b740043e0 con 0x55afbc76b420 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.492+0000 7f6b9237c740 10 client.16974 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.492+0000 7f6b9237c740 1 -- 172.21.15.69:0/4064041642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55afbc7eb3b0 con 0x55afbc76b420 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.492+0000 7f6b6e000640 1 -- 172.21.15.69:0/4064041642 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b7401f030 con 0x55afbc76b420 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.492+0000 7f6b6e000640 1 -- 172.21.15.69:0/4064041642 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7f6b7402b070 con 0x55afbc76b420 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.492+0000 7f6b9237c740 10 client.16974 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.492+0000 7f6b9237c740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.492+0000 7f6b9237c740 20 client.16974 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.492+0000 7f6b9237c740 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afbc784320 0x55afbc7edab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.492+0000 7f6b9237c740 4 client.16974 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:17:45.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.492+0000 7f6b9237c740 1 -- 172.21.15.69:0/4064041642 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55afbc6d5840 con 0x55afbc784320 2024-03-19T01:17:45.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.493+0000 7f6b8cc00640 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afbc784320 0x55afbc7edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.493+0000 7f6b8cc00640 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afbc784320 0x55afbc7edab0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb153c7d721a81de server_cookie=4c0a995de78cc3c5 in_seq=0 out_seq=0 2024-03-19T01:17:45.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.493+0000 7f6b6e000640 10 client.16974 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:45.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.494+0000 7f6b6e000640 1 -- 172.21.15.69:0/4064041642 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55afbc6d5840 con 0x55afbc784320 2024-03-19T01:17:45.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.494+0000 7f6b6e000640 10 client.16974 handle_command_reply: tid=0 2024-03-19T01:17:45.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.495+0000 7f6b9237c740 1 client.16974 shutdown 2024-03-19T01:17:45.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.495+0000 7f6b9237c740 1 -- 172.21.15.69:0/4064041642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 msgr2=0x55afbc77c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.495+0000 7f6b9237c740 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 0x55afbc77c660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6b74000c00 tx=0x7f6b74005870 comp rx=0 tx=0).stop 2024-03-19T01:17:45.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.496+0000 7f6b9237c740 1 -- 172.21.15.69:0/4064041642 shutdown_connections 2024-03-19T01:17:45.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.496+0000 7f6b9237c740 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55afbc784320 0x55afbc7edab0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.496+0000 7f6b9237c740 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afbc774000 0x55afbc7786a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.496+0000 7f6b9237c740 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55afbc76b420 0x55afbc77c660 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.496+0000 7f6b6e000640 0 client.16974 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:45.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.496+0000 7f6b9237c740 1 --2- 172.21.15.69:0/4064041642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afbc76ab00 0x55afbc77c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.496+0000 7f6b9237c740 1 -- 172.21.15.69:0/4064041642 >> 172.21.15.69:0/4064041642 conn(0x55afbc765230 msgr2=0x55afbc770f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.496+0000 7f6b9237c740 1 -- 172.21.15.69:0/4064041642 shutdown_connections 2024-03-19T01:17:45.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.496+0000 7f6b9237c740 1 -- 172.21.15.69:0/4064041642 wait complete. 2024-03-19T01:17:45.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.496+0000 7f6b9237c740 20 client.16974 trim_cache size 0 max 16384 2024-03-19T01:17:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:17:45.498 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:17:45.498 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "3a1f6545-5eb4-4e74-87c1-e0cdce4d577e", 2024-03-19T01:17:45.498 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:17:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:17:45.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 -- 172.21.15.69:0/640371554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6b700ab350 msgr2=0x7f6b700ad810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6b700ab350 0x7f6b700ad810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0096c0 tx=0x7f6b7c02df90 comp rx=0 tx=0).stop 2024-03-19T01:17:45.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 -- 172.21.15.69:0/640371554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b881077c0 msgr2=0x7f6b8819ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b881077c0 0x7f6b8819ebc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009fd0 tx=0x7f6b8000ef10 comp rx=0 tx=0).stop 2024-03-19T01:17:45.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 -- 172.21.15.69:0/640371554 shutdown_connections 2024-03-19T01:17:45.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b881077c0 0x7f6b8819ebc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6b700ab350 0x7f6b700ad810 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b88102170 0x7f6b8819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 --2- 172.21.15.69:0/640371554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b881017b0 0x7f6b8819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 -- 172.21.15.69:0/640371554 >> 172.21.15.69:0/640371554 conn(0x7f6b88078c20 msgr2=0x7f6b88103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 -- 172.21.15.69:0/640371554 shutdown_connections 2024-03-19T01:17:45.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.497+0000 7f6b8f600640 1 -- 172.21.15.69:0/640371554 wait complete. 2024-03-19T01:17:45.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:17:45.518 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:17:45.613 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:45 smithi163 ceph-mon[67428]: pgmap v217: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1023 KiB/s wr, 480 op/s 2024-03-19T01:17:45.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:45 smithi069 ceph-mon[31727]: pgmap v217: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1023 KiB/s wr, 480 op/s 2024-03-19T01:17:45.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:45 smithi086 ceph-mon[66829]: pgmap v217: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1023 KiB/s wr, 480 op/s 2024-03-19T01:17:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.787+0000 7fe503000640 1 -- 172.21.15.69:0/1659042045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4fc1039c0 msgr2=0x7fe4fc103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.787+0000 7fe503000640 1 --2- 172.21.15.69:0/1659042045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4fc1039c0 0x7fe4fc103e40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009890 tx=0x7fe4e802bb70 comp rx=0 tx=0).stop 2024-03-19T01:17:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.787+0000 7fe503000640 1 -- 172.21.15.69:0/1659042045 shutdown_connections 2024-03-19T01:17:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.787+0000 7fe503000640 1 --2- 172.21.15.69:0/1659042045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc104380 0x7fe4fc110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.787+0000 7fe503000640 1 --2- 172.21.15.69:0/1659042045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4fc1039c0 0x7fe4fc103e40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.787+0000 7fe503000640 1 --2- 172.21.15.69:0/1659042045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4fc1099d0 0x7fe4fc109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.787+0000 7fe503000640 1 -- 172.21.15.69:0/1659042045 >> 172.21.15.69:0/1659042045 conn(0x7fe4fc0ff490 msgr2=0x7fe4fc1018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.788+0000 7fe503000640 1 -- 172.21.15.69:0/1659042045 shutdown_connections 2024-03-19T01:17:45.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.788+0000 7fe503000640 1 -- 172.21.15.69:0/1659042045 wait complete. 2024-03-19T01:17:45.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.789+0000 7fe503000640 1 Processor -- start 2024-03-19T01:17:45.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.790+0000 7fe503000640 1 -- start start 2024-03-19T01:17:45.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.790+0000 7fe503000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4fc1039c0 0x7fe4fc110800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.790+0000 7fe503000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4fc104380 0x7fe4fc10d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.790+0000 7fe503000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc1099d0 0x7fe4fc10e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.790+0000 7fe503000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc113de0 con 0x7fe4fc1039c0 2024-03-19T01:17:45.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.791+0000 7fe503000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc113c60 con 0x7fe4fc1099d0 2024-03-19T01:17:45.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.791+0000 7fe503000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc113f60 con 0x7fe4fc104380 2024-03-19T01:17:45.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.791+0000 7fe4fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4fc104380 0x7fe4fc10d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.791+0000 7fe4fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4fc104380 0x7fe4fc10d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45040/0 (socket says 172.21.15.69:45040) 2024-03-19T01:17:45.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.791+0000 7fe4fb400640 1 -- 172.21.15.69:0/1413247305 learned_addr learned my addr 172.21.15.69:0/1413247305 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:45.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.791+0000 7fe501000640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc1099d0 0x7fe4fc10e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.792+0000 7fe4fb400640 1 -- 172.21.15.69:0/1413247305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc1099d0 msgr2=0x7fe4fc10e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.792+0000 7fe4fb400640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc1099d0 0x7fe4fc10e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.792+0000 7fe4fb400640 1 -- 172.21.15.69:0/1413247305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4fc1039c0 msgr2=0x7fe4fc110800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.792+0000 7fe4fb400640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4fc1039c0 0x7fe4fc110800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.792+0000 7fe4fb400640 1 -- 172.21.15.69:0/1413247305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc06b240 con 0x7fe4fc104380 2024-03-19T01:17:45.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.792+0000 7fe4fb400640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4fc104380 0x7fe4fc10d8f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8004320 tx=0x7fe4e8004350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.793+0000 7fe4f8c00640 1 -- 172.21.15.69:0/1413247305 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8041020 con 0x7fe4fc104380 2024-03-19T01:17:45.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.793+0000 7fe503000640 1 -- 172.21.15.69:0/1413247305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4fc06b4d0 con 0x7fe4fc104380 2024-03-19T01:17:45.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.793+0000 7fe503000640 1 -- 172.21.15.69:0/1413247305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4fc06ba10 con 0x7fe4fc104380 2024-03-19T01:17:45.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.793+0000 7fe4f8c00640 1 -- 172.21.15.69:0/1413247305 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e802c9f0 con 0x7fe4fc104380 2024-03-19T01:17:45.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.794+0000 7fe4f8c00640 1 -- 172.21.15.69:0/1413247305 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8039040 con 0x7fe4fc104380 2024-03-19T01:17:45.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.801+0000 7fe505d63740 1 Processor -- start 2024-03-19T01:17:45.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.801+0000 7fe505d63740 1 -- start start 2024-03-19T01:17:45.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.801+0000 7fe4f8c00640 1 -- 172.21.15.69:0/1413247305 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fe4e8044020 con 0x7fe4fc104380 2024-03-19T01:17:45.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe505d63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 0x55bb3eb73120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe505d63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb7bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe505d63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb7c240 0x55bb3eb7e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe505d63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ead72c0 con 0x55bb3eb7c240 2024-03-19T01:17:45.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe505d63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ea7fb60 con 0x55bb3eb72d40 2024-03-19T01:17:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe505d63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ead3360 con 0x55bb3eb73660 2024-03-19T01:17:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe501000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 0x55bb3eb73120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe501000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 0x55bb3eb73120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:40668/0 (socket says 172.21.15.69:40668) 2024-03-19T01:17:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe501000640 1 -- 172.21.15.69:0/822982229 learned_addr learned my addr 172.21.15.69:0/822982229 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe501000640 1 -- 172.21.15.69:0/822982229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 msgr2=0x55bb3eb7bd00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:17:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe501000640 1 --2- 172.21.15.69:0/822982229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb7bd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe501000640 1 -- 172.21.15.69:0/822982229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb7c240 msgr2=0x55bb3eb7e630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:17:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe501000640 1 --2- 172.21.15.69:0/822982229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb7c240 0x55bb3eb7e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.802+0000 7fe501000640 1 -- 172.21.15.69:0/822982229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3e8b57d0 con 0x55bb3eb72d40 2024-03-19T01:17:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.803+0000 7fe501000640 1 --2- 172.21.15.69:0/822982229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 0x55bb3eb73120 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009fd0 tx=0x7fe4f40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f62f8d1b7efc48d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.804+0000 7fe4f1600640 1 -- 172.21.15.69:0/822982229 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4004d20 con 0x55bb3eb72d40 2024-03-19T01:17:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.804+0000 7fe4f1600640 1 -- 172.21.15.69:0/822982229 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f4004050 con 0x55bb3eb72d40 2024-03-19T01:17:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.804+0000 7fe4f1600640 1 -- 172.21.15.69:0/822982229 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4005260 con 0x55bb3eb72d40 2024-03-19T01:17:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.804+0000 7fe4f8c00640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe4e00a2b70 0x7fe4e00a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.804+0000 7fe4f8c00640 1 -- 172.21.15.69:0/1413247305 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7fe4e802cba0 con 0x7fe4fc104380 2024-03-19T01:17:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.804+0000 7fe4fbe00640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe4e00a2b70 0x7fe4e00a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.806+0000 7fe505d63740 1 -- 172.21.15.69:0/822982229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 msgr2=0x55bb3eb73120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.806+0000 7fe505d63740 1 --2- 172.21.15.69:0/822982229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 0x55bb3eb73120 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009fd0 tx=0x7fe4f40118d0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.806+0000 7fe4fbe00640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe4e00a2b70 0x7fe4e00a5030 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55bb3eb72810 tx=0x7fe4ec009600 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.806+0000 7fe505d63740 1 -- 172.21.15.69:0/822982229 shutdown_connections 2024-03-19T01:17:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.806+0000 7fe505d63740 1 --2- 172.21.15.69:0/822982229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb7c240 0x55bb3eb7e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.806+0000 7fe505d63740 1 --2- 172.21.15.69:0/822982229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb7bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.806+0000 7fe505d63740 1 --2- 172.21.15.69:0/822982229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 0x55bb3eb73120 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.806+0000 7fe505d63740 1 -- 172.21.15.69:0/822982229 >> 172.21.15.69:0/822982229 conn(0x55bb3eb6d470 msgr2=0x55bb3eb6d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.807+0000 7fe505d63740 1 -- 172.21.15.69:0/822982229 shutdown_connections 2024-03-19T01:17:45.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.807+0000 7fe505d63740 1 -- 172.21.15.69:0/822982229 wait complete. 2024-03-19T01:17:45.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.807+0000 7fe505d63740 1 -- start start 2024-03-19T01:17:45.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.808+0000 7fe505d63740 1 Processor -- start 2024-03-19T01:17:45.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.809+0000 7fe505d63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb72d40 0x55bb3eb79300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.809+0000 7fe505d63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb79840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.809+0000 7fe505d63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 0x55bb3eb79d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.809+0000 7fe505d63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb3eb90040 con 0x55bb3eb72d40 2024-03-19T01:17:45.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.809+0000 7fe505d63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb3ea92fc0 con 0x55bb3eb7c240 2024-03-19T01:17:45.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.809+0000 7fe505d63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bb3eb901c0 con 0x55bb3eb73660 2024-03-19T01:17:45.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.809+0000 7fe505d63740 10 client.0 _ll_register_callbacks cb 0x55bb3ea92990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:45.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.810+0000 7fe4fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb79840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.810+0000 7fe4fbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb72d40 0x55bb3eb79300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.810+0000 7fe4fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb79840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45054/0 (socket says 172.21.15.69:45054) 2024-03-19T01:17:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.810+0000 7fe4fb400640 1 -- 172.21.15.69:0/2131749204 learned_addr learned my addr 172.21.15.69:0/2131749204 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.810+0000 7fe501000640 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 0x55bb3eb79d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.810+0000 7fe4fb400640 1 -- 172.21.15.69:0/2131749204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 msgr2=0x55bb3eb79d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.810+0000 7fe4fb400640 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 0x55bb3eb79d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.810+0000 7fe4fb400640 1 -- 172.21.15.69:0/2131749204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb72d40 msgr2=0x55bb3eb79300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.810+0000 7fe4fb400640 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb72d40 0x55bb3eb79300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.810+0000 7fe4fb400640 1 -- 172.21.15.69:0/2131749204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3e7eb3f0 con 0x55bb3eb73660 2024-03-19T01:17:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.810+0000 7fe501000640 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 0x55bb3eb79d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:17:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.811+0000 7fe4fb400640 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb79840 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80043c0 tx=0x7fe4e80bbe90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23e5946a5b4fccc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.811+0000 7fe4e6000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.811+0000 7fe4e6000640 1 -- 172.21.15.69:0/2131749204 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8039050 con 0x55bb3eb73660 2024-03-19T01:17:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.811+0000 7fe4e6000640 1 -- 172.21.15.69:0/2131749204 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7fe4e80c4840 con 0x55bb3eb73660 2024-03-19T01:17:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.811+0000 7fe505d63740 1 -- 172.21.15.69:0/2131749204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f4004050 con 0x55bb3eb73660 2024-03-19T01:17:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.811+0000 7fe4e6000640 1 -- 172.21.15.69:0/2131749204 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe4e80c0680 con 0x55bb3eb73660 2024-03-19T01:17:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.811+0000 7fe505d63740 10 client.27226 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.811+0000 7fe505d63740 1 -- 172.21.15.69:0/2131749204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bb3e7ab0f0 con 0x55bb3eb73660 2024-03-19T01:17:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.812+0000 7fe4e6000640 1 -- 172.21.15.69:0/2131749204 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f4004050 con 0x55bb3eb73660 2024-03-19T01:17:45.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.812+0000 7fe4e6000640 1 -- 172.21.15.69:0/2131749204 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7fe4e803c030 con 0x55bb3eb73660 2024-03-19T01:17:45.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.812+0000 7fe505d63740 10 client.27226 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:45.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.812+0000 7fe505d63740 10 client.27226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:45.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.812+0000 7fe505d63740 20 client.27226 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:45.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.812+0000 7fe505d63740 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb3eb9d6c0 0x55bb3eb9fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.812+0000 7fe505d63740 4 client.27226 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:17:45.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.812+0000 7fe505d63740 1 -- 172.21.15.69:0/2131749204 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb3ea78610 con 0x55bb3eb9d6c0 2024-03-19T01:17:45.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.812+0000 7fe4fbe00640 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb3eb9d6c0 0x55bb3eb9fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.813+0000 7fe4fbe00640 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb3eb9d6c0 0x55bb3eb9fb00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb1e98fd5f5a67e server_cookie=34e8ad4cdbf138d4 in_seq=0 out_seq=0 2024-03-19T01:17:45.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.813+0000 7fe4e6000640 10 client.27226 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:45.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.820+0000 7fe4e6000640 1 -- 172.21.15.69:0/2131749204 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bb3ea78610 con 0x55bb3eb9d6c0 2024-03-19T01:17:45.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.820+0000 7fe4e6000640 10 client.27226 handle_command_reply: tid=0 2024-03-19T01:17:45.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.820+0000 7fe505d63740 1 client.27226 shutdown 2024-03-19T01:17:45.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.821+0000 7fe505d63740 1 -- 172.21.15.69:0/2131749204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 msgr2=0x55bb3eb79840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.821+0000 7fe505d63740 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb79840 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80043c0 tx=0x7fe4e80bbe90 comp rx=0 tx=0).stop 2024-03-19T01:17:45.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.821+0000 7fe505d63740 1 -- 172.21.15.69:0/2131749204 shutdown_connections 2024-03-19T01:17:45.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.821+0000 7fe505d63740 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb3eb9d6c0 0x55bb3eb9fb00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.821+0000 7fe505d63740 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 0x55bb3eb79d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.821+0000 7fe505d63740 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb79840 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.821+0000 7fe505d63740 1 --2- 172.21.15.69:0/2131749204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb72d40 0x55bb3eb79300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.821+0000 7fe505d63740 1 -- 172.21.15.69:0/2131749204 >> 172.21.15.69:0/2131749204 conn(0x55bb3eb6d470 msgr2=0x55bb3eb6f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.821+0000 7fe4e6000640 0 client.27226 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:45.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.822+0000 7fe505d63740 1 -- 172.21.15.69:0/2131749204 shutdown_connections 2024-03-19T01:17:45.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.822+0000 7fe505d63740 1 -- 172.21.15.69:0/2131749204 wait complete. 2024-03-19T01:17:45.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.822+0000 7fe505d63740 20 client.27226 trim_cache size 0 max 16384 2024-03-19T01:17:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.857+0000 7fe505d63740 1 Processor -- start 2024-03-19T01:17:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.857+0000 7fe505d63740 1 -- start start 2024-03-19T01:17:45.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.857+0000 7fe505d63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb72d40 0x55bb3eb7ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.857+0000 7fe505d63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb73660 0x55bb3eb98e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.857+0000 7fe505d63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 0x55bb3eb6c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.857+0000 7fe505d63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc0100c0 con 0x55bb3eb73660 2024-03-19T01:17:45.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.857+0000 7fe505d63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc011db0 con 0x55bb3eb7c240 2024-03-19T01:17:45.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.857+0000 7fe505d63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc010790 con 0x55bb3eb72d40 2024-03-19T01:17:45.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.858+0000 7fe501000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 0x55bb3eb6c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.858+0000 7fe501000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 0x55bb3eb6c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34056/0 (socket says 172.21.15.69:34056) 2024-03-19T01:17:45.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.858+0000 7fe501000640 1 -- 172.21.15.69:0/3543832309 learned_addr learned my addr 172.21.15.69:0/3543832309 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:45.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.858+0000 7fe4fb400640 1 --2- 172.21.15.69:0/3543832309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb73660 0x55bb3eb98e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.858+0000 7fe4fbe00640 1 --2- 172.21.15.69:0/3543832309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb72d40 0x55bb3eb7ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.858+0000 7fe501000640 1 -- 172.21.15.69:0/3543832309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb72d40 msgr2=0x55bb3eb7ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.858+0000 7fe501000640 1 --2- 172.21.15.69:0/3543832309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb72d40 0x55bb3eb7ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.858+0000 7fe501000640 1 -- 172.21.15.69:0/3543832309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb73660 msgr2=0x55bb3eb98e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.858+0000 7fe501000640 1 --2- 172.21.15.69:0/3543832309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb73660 0x55bb3eb98e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.858+0000 7fe501000640 1 -- 172.21.15.69:0/3543832309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3eb6e6a0 con 0x55bb3eb7c240 2024-03-19T01:17:45.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.858+0000 7fe4fbe00640 1 --2- 172.21.15.69:0/3543832309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb72d40 0x55bb3eb7ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:17:45.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.859+0000 7fe501000640 1 --2- 172.21.15.69:0/3543832309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 0x55bb3eb6c530 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe4f401b040 tx=0x7fe4f40097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f03fa2a202a81aa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.860+0000 7fe4f0c00640 1 -- 172.21.15.69:0/3543832309 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4003b70 con 0x55bb3eb7c240 2024-03-19T01:17:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.860+0000 7fe4f0c00640 1 -- 172.21.15.69:0/3543832309 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f4005520 con 0x55bb3eb7c240 2024-03-19T01:17:45.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.861+0000 7fe4f0c00640 1 -- 172.21.15.69:0/3543832309 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f400fbc0 con 0x55bb3eb7c240 2024-03-19T01:17:45.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.861+0000 7fe505d63740 1 -- 172.21.15.69:0/3543832309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 msgr2=0x55bb3eb6c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.861+0000 7fe505d63740 1 --2- 172.21.15.69:0/3543832309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 0x55bb3eb6c530 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe4f401b040 tx=0x7fe4f40097a0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.861+0000 7fe505d63740 1 -- 172.21.15.69:0/3543832309 shutdown_connections 2024-03-19T01:17:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.861+0000 7fe505d63740 1 --2- 172.21.15.69:0/3543832309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb7c240 0x55bb3eb6c530 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.861+0000 7fe505d63740 1 --2- 172.21.15.69:0/3543832309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb73660 0x55bb3eb98e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.861+0000 7fe505d63740 1 --2- 172.21.15.69:0/3543832309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb72d40 0x55bb3eb7ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.861+0000 7fe505d63740 1 -- 172.21.15.69:0/3543832309 >> 172.21.15.69:0/3543832309 conn(0x55bb3eb6d470 msgr2=0x55bb3eb6be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.862+0000 7fe505d63740 1 -- 172.21.15.69:0/3543832309 shutdown_connections 2024-03-19T01:17:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.862+0000 7fe505d63740 1 -- 172.21.15.69:0/3543832309 wait complete. 2024-03-19T01:17:45.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.863+0000 7fe505d63740 1 -- start start 2024-03-19T01:17:45.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.863+0000 7fe505d63740 1 Processor -- start 2024-03-19T01:17:45.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.865+0000 7fe505d63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 0x55bb3eb95690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.865+0000 7fe505d63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb75af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.865+0000 7fe505d63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb7c240 0x55bb3eb76030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.865+0000 7fe4fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb75af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.865+0000 7fe4fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb75af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:39988/0 (socket says 172.21.15.69:39988) 2024-03-19T01:17:45.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.865+0000 7fe4fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 0x55bb3eb95690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.865+0000 7fe4fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 0x55bb3eb95690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34062/0 (socket says 172.21.15.69:34062) 2024-03-19T01:17:45.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.866+0000 7fe501000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb7c240 0x55bb3eb76030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.866+0000 7fe501000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb7c240 0x55bb3eb76030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55698/0 (socket says 172.21.15.69:55698) 2024-03-19T01:17:45.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.866+0000 7fe4fb400640 1 -- 172.21.15.69:0/1341516421 learned_addr learned my addr 172.21.15.69:0/1341516421 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:17:45.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.866+0000 7fe505d63740 1 -- 172.21.15.69:0/1341516421 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc011620 con 0x55bb3eb7c240 2024-03-19T01:17:45.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.867+0000 7fe505d63740 1 -- 172.21.15.69:0/1341516421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc010f20 con 0x55bb3eb72d40 2024-03-19T01:17:45.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.867+0000 7fe505d63740 1 -- 172.21.15.69:0/1341516421 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc00e010 con 0x55bb3eb73660 2024-03-19T01:17:45.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.868+0000 7fe505d63740 10 client.0 _ll_register_callbacks cb 0x7fe4e80b87d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:17:45.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.868+0000 7fe4fb400640 1 -- 172.21.15.69:0/1341516421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 msgr2=0x55bb3eb95690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.868+0000 7fe4fb400640 1 --2- 172.21.15.69:0/1341516421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 0x55bb3eb95690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.868+0000 7fe4fb400640 1 -- 172.21.15.69:0/1341516421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb7c240 msgr2=0x55bb3eb76030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:17:45.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.868+0000 7fe4fb400640 1 --2- 172.21.15.69:0/1341516421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb7c240 0x55bb3eb76030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.868+0000 7fe4fb400640 1 -- 172.21.15.69:0/1341516421 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3eb76720 con 0x55bb3eb73660 2024-03-19T01:17:45.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.869+0000 7fe4fb400640 1 --2- 172.21.15.69:0/1341516421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb75af0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80bba80 tx=0x7fe4e80bc6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e1680697637ddfb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:17:45.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.869+0000 7fe4e6000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:17:45.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.869+0000 7fe4e6000640 1 -- 172.21.15.69:0/1341516421 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8039070 con 0x55bb3eb73660 2024-03-19T01:17:45.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.869+0000 7fe4e6000640 1 -- 172.21.15.69:0/1341516421 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9568+0+0 (secure 0 0 0) 0x7fe4e80c4a50 con 0x55bb3eb73660 2024-03-19T01:17:45.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.870+0000 7fe505d63740 1 -- 172.21.15.69:0/1341516421 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f4005520 con 0x55bb3eb73660 2024-03-19T01:17:45.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.870+0000 7fe4e6000640 1 -- 172.21.15.69:0/1341516421 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe4e80bcc30 con 0x55bb3eb73660 2024-03-19T01:17:45.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.870+0000 7fe505d63740 10 client.27232 fetch_fsmap learned FSMap version 45 2024-03-19T01:17:45.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.870+0000 7fe505d63740 1 -- 172.21.15.69:0/1341516421 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bb3eb76720 con 0x55bb3eb73660 2024-03-19T01:17:45.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.870+0000 7fe4e6000640 1 -- 172.21.15.69:0/1341516421 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f4005520 con 0x55bb3eb73660 2024-03-19T01:17:45.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.872+0000 7fe4e6000640 1 -- 172.21.15.69:0/1341516421 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 45) ==== 8426+0+0 (secure 0 0 0) 0x7fe4e8032b30 con 0x55bb3eb73660 2024-03-19T01:17:45.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.872+0000 7fe505d63740 10 client.27232 fetch_fsmap finished waiting for FSMap version 45 2024-03-19T01:17:45.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.872+0000 7fe505d63740 10 client.27232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:17:45.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.872+0000 7fe505d63740 20 client.27232 populate_metadata read hostname 'smithi069' 2024-03-19T01:17:45.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.872+0000 7fe505d63740 1 --2- 172.21.15.69:0/1341516421 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb3ec03dd0 0x55bb3ec061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:17:45.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.872+0000 7fe505d63740 4 client.27232 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:17:45.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.872+0000 7fe505d63740 1 -- 172.21.15.69:0/1341516421 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb3eae4800 con 0x55bb3ec03dd0 2024-03-19T01:17:45.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.872+0000 7fe4fbe00640 1 --2- 172.21.15.69:0/1341516421 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb3ec03dd0 0x55bb3ec061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:17:45.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.874+0000 7fe4fbe00640 1 --2- 172.21.15.69:0/1341516421 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb3ec03dd0 0x55bb3ec061c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de5c857b9f151c2 server_cookie=3867bd3163cdc1f7 in_seq=0 out_seq=0 2024-03-19T01:17:45.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.874+0000 7fe4e6000640 10 client.27232 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:45.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.879+0000 7fe4e6000640 1 -- 172.21.15.69:0/1341516421 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55bb3eae4800 con 0x55bb3ec03dd0 2024-03-19T01:17:45.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.879+0000 7fe4e6000640 10 client.27232 handle_command_reply: tid=0 2024-03-19T01:17:45.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.879+0000 7fe505d63740 1 client.27232 shutdown 2024-03-19T01:17:45.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.879+0000 7fe505d63740 1 -- 172.21.15.69:0/1341516421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 msgr2=0x55bb3eb75af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.879+0000 7fe505d63740 1 --2- 172.21.15.69:0/1341516421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb75af0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80bba80 tx=0x7fe4e80bc6b0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.880+0000 7fe505d63740 1 -- 172.21.15.69:0/1341516421 shutdown_connections 2024-03-19T01:17:45.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.880+0000 7fe505d63740 1 --2- 172.21.15.69:0/1341516421 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55bb3ec03dd0 0x55bb3ec061c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.880+0000 7fe505d63740 1 --2- 172.21.15.69:0/1341516421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb3eb7c240 0x55bb3eb76030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.880+0000 7fe505d63740 1 --2- 172.21.15.69:0/1341516421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bb3eb73660 0x55bb3eb75af0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.880+0000 7fe4e6000640 0 client.27232 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:17:45.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.880+0000 7fe505d63740 1 --2- 172.21.15.69:0/1341516421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb3eb72d40 0x55bb3eb95690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.880+0000 7fe505d63740 1 -- 172.21.15.69:0/1341516421 >> 172.21.15.69:0/1341516421 conn(0x55bb3eb6d470 msgr2=0x55bb3eb6c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.880+0000 7fe505d63740 1 -- 172.21.15.69:0/1341516421 shutdown_connections 2024-03-19T01:17:45.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.880+0000 7fe505d63740 1 -- 172.21.15.69:0/1341516421 wait complete. 2024-03-19T01:17:45.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.881+0000 7fe505d63740 20 client.27232 trim_cache size 0 max 16384 2024-03-19T01:17:45.882 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:17:45.882 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (8107 inodes in the stack)", 2024-03-19T01:17:45.882 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:17:45.882 INFO:teuthology.orchestra.run.smithi069.stdout: "3a1f6545-5eb4-4e74-87c1-e0cdce4d577e": { 2024-03-19T01:17:45.882 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:17:45.882 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "3a1f6545-5eb4-4e74-87c1-e0cdce4d577e", 2024-03-19T01:17:45.882 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:17:45.882 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:17:45.883 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:17:45.883 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:17:45.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 -- 172.21.15.69:0/1413247305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe4e00a2b70 msgr2=0x7fe4e00a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe4e00a2b70 0x7fe4e00a5030 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55bb3eb72810 tx=0x7fe4ec009600 comp rx=0 tx=0).stop 2024-03-19T01:17:45.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 -- 172.21.15.69:0/1413247305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4fc104380 msgr2=0x7fe4fc10d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:17:45.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4fc104380 0x7fe4fc10d8f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8004320 tx=0x7fe4e8004350 comp rx=0 tx=0).stop 2024-03-19T01:17:45.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 -- 172.21.15.69:0/1413247305 shutdown_connections 2024-03-19T01:17:45.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe4e00a2b70 0x7fe4e00a5030 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55bb3eb72810 tx=0x7fe4ec009600 comp rx=0 tx=0).stop 2024-03-19T01:17:45.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc1099d0 0x7fe4fc10e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4fc104380 0x7fe4fc10d8f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 --2- 172.21.15.69:0/1413247305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4fc1039c0 0x7fe4fc110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:17:45.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 -- 172.21.15.69:0/1413247305 >> 172.21.15.69:0/1413247305 conn(0x7fe4fc0ff490 msgr2=0x7fe4fc0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:17:45.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 -- 172.21.15.69:0/1413247305 shutdown_connections 2024-03-19T01:17:45.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:17:45.882+0000 7fe503000640 1 -- 172.21.15.69:0/1413247305 wait complete. 2024-03-19T01:17:45.908 INFO:tasks.cephfs.filesystem:scrub status for tag:3a1f6545-5eb4-4e74-87c1-e0cdce4d577e - {'path': '/', 'tag': '3a1f6545-5eb4-4e74-87c1-e0cdce4d577e', 'options': 'recursive,force'} 2024-03-19T01:17:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:17:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:46.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:17:46.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:17:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:46.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:46.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: pgmap v218: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.1 MiB/s wr, 483 op/s 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:47.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: pgmap v218: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.1 MiB/s wr, 483 op/s 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:47.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: pgmap v218: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.1 MiB/s wr, 483 op/s 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:47.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:47.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:48.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:48.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.818 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.818 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.818 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.818 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.818 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:48.818 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:48.818 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.382 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.382 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.382 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.382 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.382 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.382 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.382 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.382 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.382 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.382 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.382 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: pgmap v219: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 944 KiB/s wr, 413 op/s 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.385 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:49.386 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: pgmap v219: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 944 KiB/s wr, 413 op/s 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:49.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: pgmap v219: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 944 KiB/s wr, 413 op/s 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:49.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:49.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:50.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: pgmap v220: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 969 KiB/s wr, 258 op/s 2024-03-19T01:17:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:51.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: pgmap v220: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 969 KiB/s wr, 258 op/s 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: pgmap v220: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 969 KiB/s wr, 258 op/s 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:51.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:51.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:52.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:52.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:52.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:52.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: pgmap v221: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 650 op/s 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:53.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: pgmap v221: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 650 op/s 2024-03-19T01:17:53.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:53.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: pgmap v221: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 650 op/s 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:53.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:53.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:54.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:54.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: pgmap v222: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 762 op/s 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: pgmap v222: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 762 op/s 2024-03-19T01:17:55.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: pgmap v222: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 762 op/s 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:55.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:56.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:56.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:56.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:57.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:57 smithi069 ceph-mon[31727]: pgmap v223: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 1.08k op/s 2024-03-19T01:17:57.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:57.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:57.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:57.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:57.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:17:57.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:57 smithi086 ceph-mon[66829]: pgmap v223: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 1.08k op/s 2024-03-19T01:17:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:17:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:57 smithi163 ceph-mon[67428]: pgmap v223: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 1.08k op/s 2024-03-19T01:17:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:17:57.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:58.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:58.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:58 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:17:58.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:58.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:58 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:17:58.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:17:58.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:58 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:17:59.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:59 smithi086 ceph-mon[66829]: pgmap v224: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 1.70k op/s 2024-03-19T01:17:59.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:59 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:17:59.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:17:59 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:17:59.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:59 smithi069 ceph-mon[31727]: pgmap v224: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 1.70k op/s 2024-03-19T01:17:59.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:59 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:17:59.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:17:59 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:17:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:59 smithi163 ceph-mon[67428]: pgmap v224: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 1.70k op/s 2024-03-19T01:17:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:59 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:17:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:17:59 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:18:00.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:00 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:18:00.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:00 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:18:00.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:00 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:18:01.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:01 smithi086 ceph-mon[66829]: pgmap v225: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 1.72k op/s 2024-03-19T01:18:01.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:01 smithi069 ceph-mon[31727]: pgmap v225: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 1.72k op/s 2024-03-19T01:18:01.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:01 smithi163 ceph-mon[67428]: pgmap v225: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 1.72k op/s 2024-03-19T01:18:03.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:03 smithi086 ceph-mon[66829]: pgmap v226: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 1.91k op/s 2024-03-19T01:18:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:03 smithi069 ceph-mon[31727]: pgmap v226: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 1.91k op/s 2024-03-19T01:18:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:03 smithi163 ceph-mon[67428]: pgmap v226: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 1.91k op/s 2024-03-19T01:18:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:05 smithi086 ceph-mon[66829]: pgmap v227: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 1.55k op/s 2024-03-19T01:18:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:05 smithi086 ceph-mon[66829]: osdmap e97: 12 total, 12 up, 12 in 2024-03-19T01:18:05.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:05 smithi069 ceph-mon[31727]: pgmap v227: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 1.55k op/s 2024-03-19T01:18:05.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:05 smithi069 ceph-mon[31727]: osdmap e97: 12 total, 12 up, 12 in 2024-03-19T01:18:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:05 smithi163 ceph-mon[67428]: pgmap v227: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 1.55k op/s 2024-03-19T01:18:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:05 smithi163 ceph-mon[67428]: osdmap e97: 12 total, 12 up, 12 in 2024-03-19T01:18:05.959 INFO:tasks.workunit.client.0.smithi069.stdout: CC scripts/mod/empty.o 2024-03-19T01:18:05.961 INFO:tasks.workunit.client.0.smithi069.stdout: CC scripts/mod/devicetable-offsets.s 2024-03-19T01:18:05.961 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/mod/mk_elfconfig 2024-03-19T01:18:06.093 INFO:tasks.workunit.client.0.smithi069.stdout: MKELF scripts/mod/elfconfig.h 2024-03-19T01:18:06.098 INFO:tasks.workunit.client.0.smithi069.stdout: UPD scripts/mod/devicetable-offsets.h 2024-03-19T01:18:06.106 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/mod/modpost.o 2024-03-19T01:18:06.106 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/mod/sumversion.o 2024-03-19T01:18:06.125 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC scripts/mod/file2alias.o 2024-03-19T01:18:06.831 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTLD scripts/mod/modpost 2024-03-19T01:18:06.886 INFO:tasks.workunit.client.0.smithi069.stdout: CHKSHA1 include/linux/atomic/atomic-arch-fallback.h 2024-03-19T01:18:06.886 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/bounds.s 2024-03-19T01:18:06.888 INFO:tasks.workunit.client.0.smithi069.stdout: CHKSHA1 include/linux/atomic/atomic-instrumented.h 2024-03-19T01:18:06.889 INFO:tasks.workunit.client.0.smithi069.stdout: CHKSHA1 include/linux/atomic/atomic-long.h 2024-03-19T01:18:06.910 INFO:tasks.workunit.client.0.smithi069.stdout: UPD include/generated/timeconst.h 2024-03-19T01:18:06.986 INFO:tasks.workunit.client.0.smithi069.stdout: UPD include/generated/bounds.h 2024-03-19T01:18:06.997 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/asm-offsets.s 2024-03-19T01:18:07.609 INFO:tasks.workunit.client.0.smithi069.stdout: UPD include/generated/asm-offsets.h 2024-03-19T01:18:07.620 INFO:tasks.workunit.client.0.smithi069.stdout: CALL scripts/checksyscalls.sh 2024-03-19T01:18:07.722 INFO:tasks.workunit.client.0.smithi069.stdout: LDS scripts/module.lds 2024-03-19T01:18:07.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:07 smithi086 ceph-mon[66829]: pgmap v229: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 1.37k op/s 2024-03-19T01:18:07.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:07 smithi069 ceph-mon[31727]: pgmap v229: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 1.37k op/s 2024-03-19T01:18:07.807 INFO:tasks.workunit.client.0.smithi069.stdout: CC init/main.o 2024-03-19T01:18:07.807 INFO:tasks.workunit.client.0.smithi069.stdout: CC certs/system_keyring.o 2024-03-19T01:18:07.810 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC usr/gen_init_cpio 2024-03-19T01:18:07.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:07 smithi163 ceph-mon[67428]: pgmap v229: 129 pgs: 129 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 1.37k op/s 2024-03-19T01:18:07.817 INFO:tasks.workunit.client.0.smithi069.stdout: CC ipc/compat.o 2024-03-19T01:18:07.822 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/filemap.o 2024-03-19T01:18:07.860 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/notify/dnotify/dnotify.o 2024-03-19T01:18:07.954 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/vdso/vma.o 2024-03-19T01:18:08.075 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/sched/core.o 2024-03-19T01:18:08.253 INFO:tasks.workunit.client.0.smithi069.stdout: GEN usr/initramfs_data.cpio 2024-03-19T01:18:08.373 INFO:tasks.workunit.client.0.smithi069.stdout: COPY usr/initramfs_inc_data 2024-03-19T01:18:08.423 INFO:tasks.workunit.client.0.smithi069.stdout: AS usr/initramfs_data.o 2024-03-19T01:18:08.445 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC certs/extract-cert 2024-03-19T01:18:08.516 INFO:tasks.workunit.client.0.smithi069.stdout: AR usr/built-in.a 2024-03-19T01:18:08.591 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mempool.o 2024-03-19T01:18:08.832 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/notify/dnotify/built-in.a 2024-03-19T01:18:08.891 INFO:tasks.workunit.client.0.smithi069.stdout: CC ipc/util.o 2024-03-19T01:18:08.906 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/notify/inotify/inotify_fsnotify.o 2024-03-19T01:18:08.953 INFO:tasks.workunit.client.0.smithi069.stdout: CERT certs/x509_certificate_list 2024-03-19T01:18:09.003 INFO:tasks.workunit.client.0.smithi069.stdout: CERT certs/signing_key.x509 2024-03-19T01:18:09.031 INFO:tasks.workunit.client.0.smithi069.stdout: AS certs/system_certificates.o 2024-03-19T01:18:09.205 INFO:tasks.workunit.client.0.smithi069.stdout: AR certs/built-in.a 2024-03-19T01:18:09.210 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/vdso/extable.o 2024-03-19T01:18:09.422 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs_common/nfsacl.o 2024-03-19T01:18:09.632 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/notify/inotify/inotify_user.o 2024-03-19T01:18:09.705 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs_common/grace.o 2024-03-19T01:18:09.736 INFO:tasks.workunit.client.0.smithi069.stdout: UPD init/utsversion-tmp.h 2024-03-19T01:18:09.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:09 smithi086 ceph-mon[66829]: pgmap v230: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 678 op/s 2024-03-19T01:18:09.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:09 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:18:09.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:09 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:18:09.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:09 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:18:09.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:09 smithi086 ceph-mon[66829]: osdmap e98: 12 total, 12 up, 12 in 2024-03-19T01:18:09.756 INFO:tasks.workunit.client.0.smithi069.stdout: CC init/do_mounts.o 2024-03-19T01:18:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:09 smithi069 ceph-mon[31727]: pgmap v230: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 678 op/s 2024-03-19T01:18:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:09 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:18:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:09 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:18:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:09 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:18:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:09 smithi069 ceph-mon[31727]: osdmap e98: 12 total, 12 up, 12 in 2024-03-19T01:18:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:09 smithi163 ceph-mon[67428]: pgmap v230: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 678 op/s 2024-03-19T01:18:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:09 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:18:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:09 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:18:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:09 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:18:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:09 smithi163 ceph-mon[67428]: osdmap e98: 12 total, 12 up, 12 in 2024-03-19T01:18:10.293 INFO:tasks.workunit.client.0.smithi069.stdout: CC ipc/msgutil.o 2024-03-19T01:18:10.329 INFO:tasks.workunit.client.0.smithi069.stdout: CC init/do_mounts_initrd.o 2024-03-19T01:18:10.359 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/vdso/vdso32-setup.o 2024-03-19T01:18:10.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:10 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:18:10.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:10 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:18:10.761 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/nfs_common/built-in.a 2024-03-19T01:18:10.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:10 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:18:10.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:10 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:18:10.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:10 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:18:10.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:10 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:18:10.857 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/notify/inotify/built-in.a 2024-03-19T01:18:10.867 INFO:tasks.workunit.client.0.smithi069.stdout: CC init/initramfs.o 2024-03-19T01:18:10.954 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/oom_kill.o 2024-03-19T01:18:10.976 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/notify/fanotify/built-in.a 2024-03-19T01:18:10.984 INFO:tasks.workunit.client.0.smithi069.stdout: LDS arch/x86/entry/vdso/vdso.lds 2024-03-19T01:18:11.029 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/notify/fsnotify.o 2024-03-19T01:18:11.043 INFO:tasks.workunit.client.0.smithi069.stdout: CC ipc/msg.o 2024-03-19T01:18:11.171 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/entry/vdso/vdso-note.o 2024-03-19T01:18:11.390 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/vdso/vclock_gettime.o 2024-03-19T01:18:11.425 INFO:tasks.workunit.client.0.smithi069.stdout: CC ipc/sem.o 2024-03-19T01:18:11.472 INFO:tasks.workunit.client.0.smithi069.stdout: CC ipc/shm.o 2024-03-19T01:18:11.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:11 smithi086 ceph-mon[66829]: pgmap v232: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 822 KiB/s wr, 352 op/s 2024-03-19T01:18:11.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:11 smithi069 ceph-mon[31727]: pgmap v232: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 822 KiB/s wr, 352 op/s 2024-03-19T01:18:11.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:11 smithi163 ceph-mon[67428]: pgmap v232: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 822 KiB/s wr, 352 op/s 2024-03-19T01:18:12.035 INFO:tasks.workunit.client.0.smithi069.stdout: CC init/calibrate.o 2024-03-19T01:18:12.201 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/vdso/vgetcpu.o 2024-03-19T01:18:12.342 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/gc.o 2024-03-19T01:18:12.415 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC arch/x86/entry/vdso/vdso2c 2024-03-19T01:18:12.608 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/notify/notification.o 2024-03-19T01:18:12.622 INFO:tasks.workunit.client.0.smithi069.stdout: CC init/init_task.o 2024-03-19T01:18:12.669 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/fadvise.o 2024-03-19T01:18:12.999 INFO:tasks.workunit.client.0.smithi069.stdout: LDS arch/x86/entry/vdso/vdso32/vdso32.lds 2024-03-19T01:18:13.096 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/entry/vdso/vdso32/note.o 2024-03-19T01:18:13.159 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/key.o 2024-03-19T01:18:13.216 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/entry/vdso/vdso32/system_call.o 2024-03-19T01:18:13.343 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/entry/vdso/vdso32/sigreturn.o 2024-03-19T01:18:13.371 INFO:tasks.workunit.client.0.smithi069.stdout: CC ipc/syscall.o 2024-03-19T01:18:13.422 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/vdso/vdso32/vclock_gettime.o 2024-03-19T01:18:13.492 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/notify/group.o 2024-03-19T01:18:13.694 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/maccess.o 2024-03-19T01:18:13.748 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/asymmetric_type.o 2024-03-19T01:18:13.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:13 smithi086 ceph-mon[66829]: pgmap v233: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 313 op/s 2024-03-19T01:18:13.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:13 smithi069 ceph-mon[31727]: pgmap v233: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 313 op/s 2024-03-19T01:18:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:13 smithi163 ceph-mon[67428]: pgmap v233: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1004 KiB/s wr, 313 op/s 2024-03-19T01:18:13.827 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/sched/fair.o 2024-03-19T01:18:13.931 INFO:tasks.workunit.client.0.smithi069.stdout: CC init/version.o 2024-03-19T01:18:14.464 INFO:tasks.workunit.client.0.smithi069.stdout: CC ipc/ipc_sysctl.o 2024-03-19T01:18:14.531 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/keyring.o 2024-03-19T01:18:14.608 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/vdso/vdso32/vgetcpu.o 2024-03-19T01:18:14.727 INFO:tasks.workunit.client.0.smithi069.stdout: AR init/built-in.a 2024-03-19T01:18:14.750 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/notify/mark.o 2024-03-19T01:18:14.814 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/keyctl.o 2024-03-19T01:18:14.908 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/page-writeback.o 2024-03-19T01:18:14.914 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/restrict.o 2024-03-19T01:18:15.126 INFO:tasks.workunit.client.0.smithi069.stdout: VDSO arch/x86/entry/vdso/vdso64.so.dbg 2024-03-19T01:18:15.540 INFO:tasks.workunit.client.0.smithi069.stdout: VDSO arch/x86/entry/vdso/vdso32.so.dbg 2024-03-19T01:18:15.586 INFO:tasks.workunit.client.0.smithi069.stdout: CC ipc/mqueue.o 2024-03-19T01:18:15.629 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/signature.o 2024-03-19T01:18:15.733 INFO:tasks.workunit.client.0.smithi069.stdout: OBJCOPY arch/x86/entry/vdso/vdso64.so 2024-03-19T01:18:15.752 INFO:tasks.workunit.client.0.smithi069.stdout: OBJCOPY arch/x86/entry/vdso/vdso32.so 2024-03-19T01:18:15.788 INFO:tasks.workunit.client.0.smithi069.stdout: VDSO2C arch/x86/entry/vdso/vdso-image-64.c 2024-03-19T01:18:15.812 INFO:tasks.workunit.client.0.smithi069.stdout: VDSO2C arch/x86/entry/vdso/vdso-image-32.c 2024-03-19T01:18:15.827 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/vsyscall/vsyscall_64.o 2024-03-19T01:18:15.845 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/vdso/vdso-image-64.o 2024-03-19T01:18:15.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:18:15.909 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:18:16.098 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/notify/fdinfo.o 2024-03-19T01:18:16.192 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/permission.o 2024-03-19T01:18:16.229 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:15 smithi069 ceph-mon[31727]: pgmap v234: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 825 KiB/s wr, 260 op/s 2024-03-19T01:18:16.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:15 smithi086 ceph-mon[66829]: pgmap v234: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 825 KiB/s wr, 260 op/s 2024-03-19T01:18:16.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:15 smithi163 ceph-mon[67428]: pgmap v234: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 825 KiB/s wr, 260 op/s 2024-03-19T01:18:16.433 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/public_key.o 2024-03-19T01:18:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.603+0000 7f6047800640 1 -- 172.21.15.69:0/2308405047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f604011b000 msgr2=0x7f604011d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.603+0000 7f6047800640 1 --2- 172.21.15.69:0/2308405047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f604011b000 0x7f604011d3f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f602c009900 tx=0x7f602c02bc60 comp rx=0 tx=0).stop 2024-03-19T01:18:16.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.606+0000 7f6047800640 1 -- 172.21.15.69:0/2308405047 shutdown_connections 2024-03-19T01:18:16.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.606+0000 7f6047800640 1 --2- 172.21.15.69:0/2308405047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f604011b000 0x7f604011d3f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.606+0000 7f6047800640 1 --2- 172.21.15.69:0/2308405047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f604010e070 0x7f604010e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.606+0000 7f6047800640 1 --2- 172.21.15.69:0/2308405047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6040110040 0x7f604010db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.606+0000 7f6047800640 1 -- 172.21.15.69:0/2308405047 >> 172.21.15.69:0/2308405047 conn(0x7f604006c900 msgr2=0x7f604006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:16.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.606+0000 7f6047800640 1 -- 172.21.15.69:0/2308405047 shutdown_connections 2024-03-19T01:18:16.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.606+0000 7f6047800640 1 -- 172.21.15.69:0/2308405047 wait complete. 2024-03-19T01:18:16.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.608+0000 7f6047800640 1 Processor -- start 2024-03-19T01:18:16.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.612+0000 7f6047800640 1 -- start start 2024-03-19T01:18:16.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.612+0000 7f6047800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f604010e070 0x7f6040118540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.612+0000 7f6047800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6040110040 0x7f6040118a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.612+0000 7f6047800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f604011b000 0x7f60401117c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.612+0000 7f6047800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f60401209c0 con 0x7f604010e070 2024-03-19T01:18:16.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.612+0000 7f6047800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6040120840 con 0x7f604011b000 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.612+0000 7f6047800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6040120b40 con 0x7f6040110040 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.613+0000 7f603fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6040110040 0x7f6040118a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.613+0000 7f6045800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f604011b000 0x7f60401117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.613+0000 7f603fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6040110040 0x7f6040118a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48294/0 (socket says 172.21.15.69:48294) 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.613+0000 7f603fe00640 1 -- 172.21.15.69:0/1538889600 learned_addr learned my addr 172.21.15.69:0/1538889600 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.613+0000 7f6044e00640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f604010e070 0x7f6040118540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.613+0000 7f603fe00640 1 -- 172.21.15.69:0/1538889600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f604011b000 msgr2=0x7f60401117c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.613+0000 7f603fe00640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f604011b000 0x7f60401117c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.613+0000 7f603fe00640 1 -- 172.21.15.69:0/1538889600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f604010e070 msgr2=0x7f6040118540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.613+0000 7f603fe00640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f604010e070 0x7f6040118540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.613+0000 7f603fe00640 1 -- 172.21.15.69:0/1538889600 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6040112080 con 0x7f6040110040 2024-03-19T01:18:16.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.614+0000 7f603fe00640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6040110040 0x7f6040118a80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f603000c920 tx=0x7f603000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:16.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.614+0000 7f603d600640 1 -- 172.21.15.69:0/1538889600 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030007c70 con 0x7f6040110040 2024-03-19T01:18:16.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.614+0000 7f6047800640 1 -- 172.21.15.69:0/1538889600 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6040112370 con 0x7f6040110040 2024-03-19T01:18:16.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.614+0000 7f6047800640 1 -- 172.21.15.69:0/1538889600 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6040116350 con 0x7f6040110040 2024-03-19T01:18:16.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.614+0000 7f603d600640 1 -- 172.21.15.69:0/1538889600 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6030007e10 con 0x7f6040110040 2024-03-19T01:18:16.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.615+0000 7f603d600640 1 -- 172.21.15.69:0/1538889600 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030018890 con 0x7f6040110040 2024-03-19T01:18:16.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.626+0000 7f604a580740 1 Processor -- start 2024-03-19T01:18:16.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.626+0000 7f603d600640 1 -- 172.21.15.69:0/1538889600 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f6030018b10 con 0x7f6040110040 2024-03-19T01:18:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.628+0000 7f603d600640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f60240a2a30 0x7f60240a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.629+0000 7f604a580740 1 -- start start 2024-03-19T01:18:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.629+0000 7f604a580740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 0x558839719120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.629+0000 7f603d600640 1 -- 172.21.15.69:0/1538889600 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f6030018e00 con 0x7f6040110040 2024-03-19T01:18:16.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.630+0000 7f6045800640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f60240a2a30 0x7f60240a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.632+0000 7f6045800640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f60240a2a30 0x7f60240a4ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f602c02efd0 tx=0x7f602c004f10 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:16.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.634+0000 7f604a580740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 0x558839721d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.634+0000 7f604a580740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839722240 0x558839724630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.634+0000 7f604a580740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55883967d2c0 con 0x558839718d40 2024-03-19T01:18:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.634+0000 7f604a580740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558839625b60 con 0x558839722240 2024-03-19T01:18:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.634+0000 7f604a580740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558839679360 con 0x558839719660 2024-03-19T01:18:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.634+0000 7f6044e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 0x558839719120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.634+0000 7f6045800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839722240 0x558839724630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.634+0000 7f603fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 0x558839721d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.634+0000 7f6044e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 0x558839719120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41268/0 (socket says 172.21.15.69:41268) 2024-03-19T01:18:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.634+0000 7f6044e00640 1 -- 172.21.15.69:0/4015983746 learned_addr learned my addr 172.21.15.69:0/4015983746 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.635+0000 7f603fe00640 1 -- 172.21.15.69:0/4015983746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839722240 msgr2=0x558839724630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.635+0000 7f603fe00640 1 --2- 172.21.15.69:0/4015983746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839722240 0x558839724630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.635+0000 7f603fe00640 1 -- 172.21.15.69:0/4015983746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 msgr2=0x558839719120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:16.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.635+0000 7f603fe00640 1 --2- 172.21.15.69:0/4015983746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 0x558839719120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.635+0000 7f603fe00640 1 -- 172.21.15.69:0/4015983746 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883945b7d0 con 0x558839719660 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.635+0000 7f6044e00640 1 --2- 172.21.15.69:0/4015983746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 0x558839719120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.635+0000 7f603fe00640 1 --2- 172.21.15.69:0/4015983746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 0x558839721d00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f60300cd710 tx=0x7f60300a1620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f4f69bdb0123349 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.636+0000 7f6036000640 1 -- 172.21.15.69:0/4015983746 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030099050 con 0x558839719660 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.636+0000 7f6036000640 1 -- 172.21.15.69:0/4015983746 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f603009cd20 con 0x558839719660 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.636+0000 7f6036000640 1 -- 172.21.15.69:0/4015983746 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300991f0 con 0x558839719660 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.637+0000 7f604a580740 1 -- 172.21.15.69:0/4015983746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 msgr2=0x558839721d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.637+0000 7f604a580740 1 --2- 172.21.15.69:0/4015983746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 0x558839721d00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f60300cd710 tx=0x7f60300a1620 comp rx=0 tx=0).stop 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.637+0000 7f604a580740 1 -- 172.21.15.69:0/4015983746 shutdown_connections 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.637+0000 7f604a580740 1 --2- 172.21.15.69:0/4015983746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839722240 0x558839724630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.637+0000 7f604a580740 1 --2- 172.21.15.69:0/4015983746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 0x558839721d00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.637+0000 7f604a580740 1 --2- 172.21.15.69:0/4015983746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 0x558839719120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.637+0000 7f604a580740 1 -- 172.21.15.69:0/4015983746 >> 172.21.15.69:0/4015983746 conn(0x558839713470 msgr2=0x558839713850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:16.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.641+0000 7f604a580740 1 -- 172.21.15.69:0/4015983746 shutdown_connections 2024-03-19T01:18:16.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.641+0000 7f604a580740 1 -- 172.21.15.69:0/4015983746 wait complete. 2024-03-19T01:18:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.642+0000 7f604a580740 1 -- start start 2024-03-19T01:18:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.642+0000 7f604a580740 1 Processor -- start 2024-03-19T01:18:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.643+0000 7f604a580740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 0x55883971f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.643+0000 7f604a580740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839719660 0x55883971f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.643+0000 7f604a580740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839722240 0x55883971fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.644+0000 7f6044e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 0x55883971f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.644+0000 7f6044e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 0x55883971f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41280/0 (socket says 172.21.15.69:41280) 2024-03-19T01:18:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.644+0000 7f603fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839719660 0x55883971f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.644+0000 7f603fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839719660 0x55883971f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34882/0 (socket says 172.21.15.69:34882) 2024-03-19T01:18:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.645+0000 7f604a580740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558839724fd0 con 0x558839718d40 2024-03-19T01:18:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.645+0000 7f604a580740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558839638fc0 con 0x558839719660 2024-03-19T01:18:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.645+0000 7f604a580740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558839725150 con 0x558839722240 2024-03-19T01:18:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.645+0000 7f604a580740 10 client.0 _ll_register_callbacks cb 0x558839638990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.645+0000 7f6044e00640 1 -- 172.21.15.69:0/556375695 learned_addr learned my addr 172.21.15.69:0/556375695 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.645+0000 7f603fe00640 1 -- 172.21.15.69:0/556375695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839722240 msgr2=0x55883971fdc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.645+0000 7f603fe00640 1 --2- 172.21.15.69:0/556375695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839722240 0x55883971fdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.645+0000 7f603fe00640 1 -- 172.21.15.69:0/556375695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 msgr2=0x55883971f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.645+0000 7f603fe00640 1 --2- 172.21.15.69:0/556375695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 0x55883971f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.645+0000 7f603fe00640 1 -- 172.21.15.69:0/556375695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588393913f0 con 0x558839719660 2024-03-19T01:18:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.648+0000 7f603fe00640 1 --2- 172.21.15.69:0/556375695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839719660 0x55883971f880 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x558839718810 tx=0x7f603009ab00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a57697be03f0cdd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.648+0000 7f602aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:18:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.648+0000 7f602aa00640 1 -- 172.21.15.69:0/556375695 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030099390 con 0x558839719660 2024-03-19T01:18:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.648+0000 7f602aa00640 1 -- 172.21.15.69:0/556375695 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f603009c590 con 0x558839719660 2024-03-19T01:18:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.650+0000 7f604a580740 1 -- 172.21.15.69:0/556375695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f603009cd20 con 0x558839719660 2024-03-19T01:18:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.650+0000 7f602aa00640 1 -- 172.21.15.69:0/556375695 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f60300ac050 con 0x558839719660 2024-03-19T01:18:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.650+0000 7f604a580740 10 client.27287 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.650+0000 7f604a580740 1 -- 172.21.15.69:0/556375695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5588393510f0 con 0x558839719660 2024-03-19T01:18:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.654+0000 7f602aa00640 1 -- 172.21.15.69:0/556375695 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f603009cd20 con 0x558839719660 2024-03-19T01:18:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.654+0000 7f602aa00640 1 -- 172.21.15.69:0/556375695 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f60300cd990 con 0x558839719660 2024-03-19T01:18:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.654+0000 7f604a580740 10 client.27287 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.654+0000 7f604a580740 10 client.27287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:18:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.654+0000 7f604a580740 20 client.27287 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.654+0000 7f604a580740 1 --2- 172.21.15.69:0/556375695 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558839732500 0x5588397348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.654+0000 7f604a580740 4 client.27287 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.654+0000 7f604a580740 1 -- 172.21.15.69:0/556375695 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55883961e610 con 0x558839732500 2024-03-19T01:18:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.655+0000 7f6044e00640 1 --2- 172.21.15.69:0/556375695 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558839732500 0x5588397348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.655+0000 7f6044e00640 1 --2- 172.21.15.69:0/556375695 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558839732500 0x5588397348f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abd59323f60ab1db server_cookie=dcc58c2967d73371 in_seq=0 out_seq=0 2024-03-19T01:18:16.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.655+0000 7f602aa00640 10 client.27287 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f602aa00640 1 -- 172.21.15.69:0/556375695 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55883961e610 con 0x558839732500 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f602aa00640 10 client.27287 handle_command_reply: tid=0 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f604a580740 1 client.27287 shutdown 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f604a580740 1 -- 172.21.15.69:0/556375695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839719660 msgr2=0x55883971f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f604a580740 1 --2- 172.21.15.69:0/556375695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839719660 0x55883971f880 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x558839718810 tx=0x7f603009ab00 comp rx=0 tx=0).stop 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f604a580740 1 -- 172.21.15.69:0/556375695 shutdown_connections 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f604a580740 1 --2- 172.21.15.69:0/556375695 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558839732500 0x5588397348f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f604a580740 1 --2- 172.21.15.69:0/556375695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839722240 0x55883971fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f604a580740 1 --2- 172.21.15.69:0/556375695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839719660 0x55883971f880 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f604a580740 1 --2- 172.21.15.69:0/556375695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839718d40 0x55883971f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f604a580740 1 -- 172.21.15.69:0/556375695 >> 172.21.15.69:0/556375695 conn(0x558839713470 msgr2=0x5588397151c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.661+0000 7f602aa00640 0 client.27287 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.662+0000 7f604a580740 1 -- 172.21.15.69:0/556375695 shutdown_connections 2024-03-19T01:18:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.662+0000 7f604a580740 1 -- 172.21.15.69:0/556375695 wait complete. 2024-03-19T01:18:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.662+0000 7f604a580740 20 client.27287 trim_cache size 0 max 16384 2024-03-19T01:18:16.681 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/vdso/vdso-image-32.o 2024-03-19T01:18:16.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.708+0000 7f604a580740 1 Processor -- start 2024-03-19T01:18:16.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.709+0000 7f604a580740 1 -- start start 2024-03-19T01:18:16.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.709+0000 7f604a580740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 0x558839721a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.709+0000 7f604a580740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839719660 0x55883972dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.709+0000 7f604a580740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839722240 0x558839739ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.710+0000 7f6044e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 0x558839721a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.710+0000 7f6044e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 0x558839721a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34898/0 (socket says 172.21.15.69:34898) 2024-03-19T01:18:16.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.710+0000 7f603fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839719660 0x55883972dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.710+0000 7f603fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839719660 0x55883972dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41296/0 (socket says 172.21.15.69:41296) 2024-03-19T01:18:16.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.712+0000 7f6044e00640 1 -- 172.21.15.69:0/3658208391 learned_addr learned my addr 172.21.15.69:0/3658208391 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.712+0000 7f604a580740 1 -- 172.21.15.69:0/3658208391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6020010190 con 0x558839719660 2024-03-19T01:18:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.712+0000 7f604a580740 1 -- 172.21.15.69:0/3658208391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6020011e50 con 0x558839718d40 2024-03-19T01:18:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.712+0000 7f604a580740 1 -- 172.21.15.69:0/3658208391 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6020010860 con 0x558839722240 2024-03-19T01:18:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.713+0000 7f6044e00640 1 -- 172.21.15.69:0/3658208391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839722240 msgr2=0x558839739ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:16.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.713+0000 7f6044e00640 1 --2- 172.21.15.69:0/3658208391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839722240 0x558839739ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.713+0000 7f6044e00640 1 -- 172.21.15.69:0/3658208391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839719660 msgr2=0x55883972dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:16.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.713+0000 7f6044e00640 1 --2- 172.21.15.69:0/3658208391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839719660 0x55883972dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.713+0000 7f6044e00640 1 -- 172.21.15.69:0/3658208391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558839719490 con 0x558839718d40 2024-03-19T01:18:16.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.714+0000 7f6044e00640 1 --2- 172.21.15.69:0/3658208391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 0x558839721a70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6038007d40 tx=0x7f6038007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56baf3253c77d8e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:16.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.714+0000 7f603fe00640 1 --2- 172.21.15.69:0/3658208391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839719660 0x55883972dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:16.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.714+0000 7f6035600640 1 -- 172.21.15.69:0/3658208391 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038020070 con 0x558839718d40 2024-03-19T01:18:16.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.714+0000 7f6035600640 1 -- 172.21.15.69:0/3658208391 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f603801c2d0 con 0x558839718d40 2024-03-19T01:18:16.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.714+0000 7f6035600640 1 -- 172.21.15.69:0/3658208391 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603800d040 con 0x558839718d40 2024-03-19T01:18:16.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.715+0000 7f604a580740 1 -- 172.21.15.69:0/3658208391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 msgr2=0x558839721a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:16.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.715+0000 7f604a580740 1 --2- 172.21.15.69:0/3658208391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 0x558839721a70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6038007d40 tx=0x7f6038007700 comp rx=0 tx=0).stop 2024-03-19T01:18:16.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.716+0000 7f604a580740 1 -- 172.21.15.69:0/3658208391 shutdown_connections 2024-03-19T01:18:16.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.716+0000 7f604a580740 1 --2- 172.21.15.69:0/3658208391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839722240 0x558839739ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.716+0000 7f604a580740 1 --2- 172.21.15.69:0/3658208391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839719660 0x55883972dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.716+0000 7f604a580740 1 --2- 172.21.15.69:0/3658208391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 0x558839721a70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.716+0000 7f604a580740 1 -- 172.21.15.69:0/3658208391 >> 172.21.15.69:0/3658208391 conn(0x558839713470 msgr2=0x558839713850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:16.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.722+0000 7f604a580740 1 -- 172.21.15.69:0/3658208391 shutdown_connections 2024-03-19T01:18:16.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.722+0000 7f604a580740 1 -- 172.21.15.69:0/3658208391 wait complete. 2024-03-19T01:18:16.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.722+0000 7f604a580740 1 -- start start 2024-03-19T01:18:16.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.723+0000 7f604a580740 1 Processor -- start 2024-03-19T01:18:16.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.723+0000 7f604a580740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 0x55883972a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.723+0000 7f604a580740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 0x558839725990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.724+0000 7f604a580740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839722240 0x558839725ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.724+0000 7f6044e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 0x55883972a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.724+0000 7f6044e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 0x55883972a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34912/0 (socket says 172.21.15.69:34912) 2024-03-19T01:18:16.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.724+0000 7f603fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 0x558839725990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.724+0000 7f603fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 0x558839725990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48312/0 (socket says 172.21.15.69:48312) 2024-03-19T01:18:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.724+0000 7f6045800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839722240 0x558839725ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.724+0000 7f6045800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839722240 0x558839725ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41300/0 (socket says 172.21.15.69:41300) 2024-03-19T01:18:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.724+0000 7f604a580740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f60200116c0 con 0x558839722240 2024-03-19T01:18:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.724+0000 7f604a580740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6020010ff0 con 0x558839718d40 2024-03-19T01:18:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.724+0000 7f604a580740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f602000e110 con 0x558839719660 2024-03-19T01:18:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.725+0000 7f604a580740 10 client.0 _ll_register_callbacks cb 0x7f603009cf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.725+0000 7f6044e00640 1 -- 172.21.15.69:0/3016134688 learned_addr learned my addr 172.21.15.69:0/3016134688 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:16.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.725+0000 7f603fe00640 1 -- 172.21.15.69:0/3016134688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 msgr2=0x55883972a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:16.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.725+0000 7f603fe00640 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 0x55883972a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.725+0000 7f603fe00640 1 -- 172.21.15.69:0/3016134688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839722240 msgr2=0x558839725ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:16.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.725+0000 7f603fe00640 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839722240 0x558839725ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.725+0000 7f603fe00640 1 -- 172.21.15.69:0/3016134688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558839726570 con 0x558839719660 2024-03-19T01:18:16.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.725+0000 7f6045800640 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839722240 0x558839725ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:16.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.726+0000 7f603fe00640 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 0x558839725990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f603009ab70 tx=0x7f6030007620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94a5e6e75d1f9970 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:16.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.726+0000 7f602aa00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:18:16.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.726+0000 7f602aa00640 1 -- 172.21.15.69:0/3016134688 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030099560 con 0x558839719660 2024-03-19T01:18:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.726+0000 7f602aa00640 1 -- 172.21.15.69:0/3016134688 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f6030099330 con 0x558839719660 2024-03-19T01:18:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.726+0000 7f604a580740 1 -- 172.21.15.69:0/3016134688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f603801c2d0 con 0x558839719660 2024-03-19T01:18:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.727+0000 7f602aa00640 1 -- 172.21.15.69:0/3016134688 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6030084050 con 0x558839719660 2024-03-19T01:18:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.727+0000 7f604a580740 10 client.27244 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.727+0000 7f604a580740 1 -- 172.21.15.69:0/3016134688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558839798390 con 0x558839719660 2024-03-19T01:18:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.728+0000 7f602aa00640 1 -- 172.21.15.69:0/3016134688 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f603801c2d0 con 0x558839719660 2024-03-19T01:18:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.728+0000 7f602aa00640 1 -- 172.21.15.69:0/3016134688 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f60300986f0 con 0x558839719660 2024-03-19T01:18:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.728+0000 7f604a580740 10 client.27244 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.728+0000 7f604a580740 10 client.27244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:18:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.728+0000 7f604a580740 20 client.27244 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.728+0000 7f604a580740 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558839798910 0x55883979ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:16.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.728+0000 7f604a580740 4 client.27244 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:18:16.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.729+0000 7f604a580740 1 -- 172.21.15.69:0/3016134688 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55883968a800 con 0x558839798910 2024-03-19T01:18:16.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.729+0000 7f6044e00640 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558839798910 0x55883979ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:16.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.729+0000 7f6044e00640 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558839798910 0x55883979ad30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29bd13efd1dfd8a server_cookie=29b1871b6574ce84 in_seq=0 out_seq=0 2024-03-19T01:18:16.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.729+0000 7f602aa00640 10 client.27244 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:16.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.729+0000 7f602aa00640 1 -- 172.21.15.69:0/3016134688 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55883968a800 con 0x558839798910 2024-03-19T01:18:16.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.729+0000 7f602aa00640 10 client.27244 handle_command_reply: tid=0 2024-03-19T01:18:16.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.730+0000 7f604a580740 1 client.27244 shutdown 2024-03-19T01:18:16.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.742+0000 7f604a580740 1 -- 172.21.15.69:0/3016134688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 msgr2=0x558839725990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:16.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.742+0000 7f604a580740 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 0x558839725990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f603009ab70 tx=0x7f6030007620 comp rx=0 tx=0).stop 2024-03-19T01:18:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.742+0000 7f604a580740 1 -- 172.21.15.69:0/3016134688 shutdown_connections 2024-03-19T01:18:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.742+0000 7f604a580740 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558839798910 0x55883979ad30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.742+0000 7f604a580740 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558839722240 0x558839725ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.742+0000 7f604a580740 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558839719660 0x558839725990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.742+0000 7f604a580740 1 --2- 172.21.15.69:0/3016134688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558839718d40 0x55883972a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.742+0000 7f602aa00640 0 client.27244 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.742+0000 7f604a580740 1 -- 172.21.15.69:0/3016134688 >> 172.21.15.69:0/3016134688 conn(0x558839713470 msgr2=0x55883971a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.743+0000 7f604a580740 1 -- 172.21.15.69:0/3016134688 shutdown_connections 2024-03-19T01:18:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.743+0000 7f604a580740 1 -- 172.21.15.69:0/3016134688 wait complete. 2024-03-19T01:18:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.743+0000 7f604a580740 20 client.27244 trim_cache size 0 max 16384 2024-03-19T01:18:16.747 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:18:16.748 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:18:16.748 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:18:16.748 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:18:16.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.750+0000 7f6047800640 1 -- 172.21.15.69:0/1538889600 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f60240a2a30 msgr2=0x7f60240a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:16.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.750+0000 7f6047800640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f60240a2a30 0x7f60240a4ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f602c02efd0 tx=0x7f602c004f10 comp rx=0 tx=0).stop 2024-03-19T01:18:16.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.750+0000 7f6047800640 1 -- 172.21.15.69:0/1538889600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6040110040 msgr2=0x7f6040118a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:16.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.750+0000 7f6047800640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6040110040 0x7f6040118a80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f603000c920 tx=0x7f603000cde0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.750+0000 7f6047800640 1 -- 172.21.15.69:0/1538889600 shutdown_connections 2024-03-19T01:18:16.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.750+0000 7f6047800640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f60240a2a30 0x7f60240a4ef0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.750+0000 7f6047800640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f604011b000 0x7f60401117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.750+0000 7f6047800640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6040110040 0x7f6040118a80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.750+0000 7f6047800640 1 --2- 172.21.15.69:0/1538889600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f604010e070 0x7f6040118540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:16.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.750+0000 7f6047800640 1 -- 172.21.15.69:0/1538889600 >> 172.21.15.69:0/1538889600 conn(0x7f604006c900 msgr2=0x7f60400722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:16.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.752+0000 7f6047800640 1 -- 172.21.15.69:0/1538889600 shutdown_connections 2024-03-19T01:18:16.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:16.752+0000 7f6047800640 1 -- 172.21.15.69:0/1538889600 wait complete. 2024-03-19T01:18:16.813 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:18:16.813 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:18:16.966 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/folio-compat.o 2024-03-19T01:18:17.055 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:16 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:17.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:16 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:17.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:16 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:17.325 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/entry/vsyscall/vsyscall_emu_64.o 2024-03-19T01:18:17.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.341+0000 7fa263e00640 1 -- 172.21.15.69:0/4188666044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa26410fdc0 msgr2=0x7fa2641101a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:17.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.341+0000 7fa263e00640 1 --2- 172.21.15.69:0/4188666044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa26410fdc0 0x7fa2641101a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa25401a910 tx=0x7fa25403cc70 comp rx=0 tx=0).stop 2024-03-19T01:18:17.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.341+0000 7fa263e00640 1 -- 172.21.15.69:0/4188666044 shutdown_connections 2024-03-19T01:18:17.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.341+0000 7fa263e00640 1 --2- 172.21.15.69:0/4188666044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26410e4b0 0x7fa26411d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:17.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.341+0000 7fa263e00640 1 --2- 172.21.15.69:0/4188666044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa26410d920 0x7fa26410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:17.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.341+0000 7fa263e00640 1 --2- 172.21.15.69:0/4188666044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa26410fdc0 0x7fa2641101a0 secure :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa25401a910 tx=0x7fa25403cc70 comp rx=0 tx=0).stop 2024-03-19T01:18:17.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.341+0000 7fa263e00640 1 -- 172.21.15.69:0/4188666044 >> 172.21.15.69:0/4188666044 conn(0x7fa26406c700 msgr2=0x7fa26406cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:17.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.347+0000 7fa263e00640 1 -- 172.21.15.69:0/4188666044 shutdown_connections 2024-03-19T01:18:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.347+0000 7fa263e00640 1 -- 172.21.15.69:0/4188666044 wait complete. 2024-03-19T01:18:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.350+0000 7fa263e00640 1 Processor -- start 2024-03-19T01:18:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.353+0000 7fa263e00640 1 -- start start 2024-03-19T01:18:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.353+0000 7fa263e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa26410d920 0x7fa26411ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.353+0000 7fa263e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa26410e4b0 0x7fa264118ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.353+0000 7fa263e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2641193e0 0x7fa264119890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.353+0000 7fa263e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2641131a0 con 0x7fa26410d920 2024-03-19T01:18:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.353+0000 7fa263e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa264113020 con 0x7fa26410e4b0 2024-03-19T01:18:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.353+0000 7fa263e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa264113320 con 0x7fa2641193e0 2024-03-19T01:18:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.354+0000 7fa25a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa26410e4b0 0x7fa264118ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.354+0000 7fa25a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa26410e4b0 0x7fa264118ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34930/0 (socket says 172.21.15.69:34930) 2024-03-19T01:18:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.354+0000 7fa25a000640 1 -- 172.21.15.69:0/667508359 learned_addr learned my addr 172.21.15.69:0/667508359 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.354+0000 7fa263400640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2641193e0 0x7fa264119890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.354+0000 7fa262a00640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa26410d920 0x7fa26411ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.354+0000 7fa263400640 1 -- 172.21.15.69:0/667508359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa26410e4b0 msgr2=0x7fa264118ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.354+0000 7fa263400640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa26410e4b0 0x7fa264118ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.354+0000 7fa263400640 1 -- 172.21.15.69:0/667508359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa26410d920 msgr2=0x7fa26411ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.354+0000 7fa263400640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa26410d920 0x7fa26411ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.354+0000 7fa263400640 1 -- 172.21.15.69:0/667508359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2641aadb0 con 0x7fa2641193e0 2024-03-19T01:18:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.355+0000 7fa262a00640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa26410d920 0x7fa26411ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:18:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.355+0000 7fa263400640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2641193e0 0x7fa264119890 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009fd0 tx=0x7fa25c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.356+0000 7fa25be00640 1 -- 172.21.15.69:0/667508359 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00ce20 con 0x7fa2641193e0 2024-03-19T01:18:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.356+0000 7fa263e00640 1 -- 172.21.15.69:0/667508359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2641ab0a0 con 0x7fa2641193e0 2024-03-19T01:18:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.360+0000 7fa263e00640 1 -- 172.21.15.69:0/667508359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2641ab590 con 0x7fa2641193e0 2024-03-19T01:18:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.356+0000 7fa25be00640 1 -- 172.21.15.69:0/667508359 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa25c005020 con 0x7fa2641193e0 2024-03-19T01:18:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.361+0000 7fa263e00640 1 -- 172.21.15.69:0/667508359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26406d970 con 0x7fa2641193e0 2024-03-19T01:18:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.362+0000 7fa25be00640 1 -- 172.21.15.69:0/667508359 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c010720 con 0x7fa2641193e0 2024-03-19T01:18:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.365+0000 7fa25be00640 1 -- 172.21.15.69:0/667508359 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fa25c03eb30 con 0x7fa2641193e0 2024-03-19T01:18:17.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.370+0000 7fa25be00640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2480a2ab0 0x7fa2480a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:17.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.370+0000 7fa25be00640 1 -- 172.21.15.69:0/667508359 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7fa25c005520 con 0x7fa2641193e0 2024-03-19T01:18:17.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.371+0000 7fa262a00640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2480a2ab0 0x7fa2480a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:17.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.375+0000 7fa262a00640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2480a2ab0 0x7fa2480a4f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa25401a860 tx=0x7fa254000bc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:17.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:17.375+0000 7fa25be00640 1 -- 172.21.15.69:0/667508359 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa25c09c8f0 con 0x7fa2641193e0 2024-03-19T01:18:17.445 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/entry/vsyscall/built-in.a 2024-03-19T01:18:17.454 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/entry/vdso/built-in.a 2024-03-19T01:18:17.480 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/process_keys.o 2024-03-19T01:18:17.495 INFO:tasks.workunit.client.0.smithi069.stdout: ASN.1 crypto/asymmetric_keys/x509.asn1.[ch] 2024-03-19T01:18:17.516 INFO:tasks.workunit.client.0.smithi069.stdout: ASN.1 crypto/asymmetric_keys/x509_akid.asn1.[ch] 2024-03-19T01:18:17.537 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/x509_loader.o 2024-03-19T01:18:17.565 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/notify/built-in.a 2024-03-19T01:18:17.624 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/entry/entry.o 2024-03-19T01:18:17.643 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/locking/mutex.o 2024-03-19T01:18:17.659 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/iomap/trace.o 2024-03-19T01:18:17.845 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/entry/entry_64.o 2024-03-19T01:18:17.983 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/x509_public_key.o 2024-03-19T01:18:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.083+0000 7fa263e00640 1 -- 172.21.15.69:0/667508359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa26402d1f0 con 0x7fa2641193e0 2024-03-19T01:18:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.085+0000 7fa25be00640 1 -- 172.21.15.69:0/667508359 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11954 (secure 0 0 0) 0x7fa25c09c1f0 con 0x7fa2641193e0 2024-03-19T01:18:18.089 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:18:18.090 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:18:15.707759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:18:18.090 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 47 2024-03-19T01:18:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.098+0000 7fa263e00640 1 -- 172.21.15.69:0/667508359 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2480a2ab0 msgr2=0x7fa2480a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.098+0000 7fa263e00640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2480a2ab0 0x7fa2480a4f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa25401a860 tx=0x7fa254000bc0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.098+0000 7fa263e00640 1 -- 172.21.15.69:0/667508359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2641193e0 msgr2=0x7fa264119890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.098+0000 7fa263e00640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2641193e0 0x7fa264119890 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009fd0 tx=0x7fa25c00ef10 comp rx=0 tx=0).stop 2024-03-19T01:18:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.098+0000 7fa263e00640 1 -- 172.21.15.69:0/667508359 shutdown_connections 2024-03-19T01:18:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.098+0000 7fa263e00640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2641193e0 0x7fa264119890 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.098+0000 7fa263e00640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa2480a2ab0 0x7fa2480a4f70 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa25401a860 tx=0x7fa254000bc0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.098+0000 7fa263e00640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa26410e4b0 0x7fa264118ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.098+0000 7fa263e00640 1 --2- 172.21.15.69:0/667508359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa26410d920 0x7fa26411ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.098+0000 7fa263e00640 1 -- 172.21.15.69:0/667508359 >> 172.21.15.69:0/667508359 conn(0x7fa26406c700 msgr2=0x7fa264070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:18.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.104+0000 7fa263e00640 1 -- 172.21.15.69:0/667508359 shutdown_connections 2024-03-19T01:18:18.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.106+0000 7fa263e00640 1 -- 172.21.15.69:0/667508359 wait complete. 2024-03-19T01:18:18.190 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:18:18.207 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/readahead.o 2024-03-19T01:18:18.207 INFO:tasks.workunit.client.0.smithi069.stdout: CC ipc/namespace.o 2024-03-19T01:18:18.207 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/syscall_64.o 2024-03-19T01:18:18.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:17 smithi086 ceph-mon[66829]: pgmap v235: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 547 KiB/s wr, 197 op/s 2024-03-19T01:18:18.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:17 smithi069 ceph-mon[31727]: pgmap v235: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 547 KiB/s wr, 197 op/s 2024-03-19T01:18:18.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:17 smithi163 ceph-mon[67428]: pgmap v235: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 547 KiB/s wr, 197 op/s 2024-03-19T01:18:18.491 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/sched/build_policy.o 2024-03-19T01:18:18.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.743+0000 7f9479c00640 1 -- 172.21.15.69:0/755001616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f947410fe30 msgr2=0x7f9474110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:18.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.743+0000 7f9479c00640 1 --2- 172.21.15.69:0/755001616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f947410fe30 0x7f9474110210 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f946c009cd0 tx=0x7f946c02c100 comp rx=0 tx=0).stop 2024-03-19T01:18:18.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.743+0000 7f9479c00640 1 -- 172.21.15.69:0/755001616 shutdown_connections 2024-03-19T01:18:18.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.743+0000 7f9479c00640 1 --2- 172.21.15.69:0/755001616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f947410e4b0 0x7f947411d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.743+0000 7f9479c00640 1 --2- 172.21.15.69:0/755001616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947410d900 0x7f947410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.743+0000 7f9479c00640 1 --2- 172.21.15.69:0/755001616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f947410fe30 0x7f9474110210 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.743+0000 7f9479c00640 1 -- 172.21.15.69:0/755001616 >> 172.21.15.69:0/755001616 conn(0x7f947406c700 msgr2=0x7f947406cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:18.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.744+0000 7f9479c00640 1 -- 172.21.15.69:0/755001616 shutdown_connections 2024-03-19T01:18:18.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.744+0000 7f9479c00640 1 -- 172.21.15.69:0/755001616 wait complete. 2024-03-19T01:18:18.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.745+0000 7f9479c00640 1 Processor -- start 2024-03-19T01:18:18.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.745+0000 7f9479c00640 1 -- start start 2024-03-19T01:18:18.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.746+0000 7f9479c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f947410d900 0x7f94741a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.746+0000 7f9479c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947410e4b0 0x7f94741a7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.746+0000 7f9479c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f947410fe30 0x7f94741ab690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.746+0000 7f9479c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9474120890 con 0x7f947410fe30 2024-03-19T01:18:18.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.746+0000 7f9479c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9474120710 con 0x7f947410e4b0 2024-03-19T01:18:18.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.746+0000 7f9479c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9474120a10 con 0x7f947410d900 2024-03-19T01:18:18.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.746+0000 7f9472a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f947410d900 0x7f94741a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.746+0000 7f9472a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f947410d900 0x7f94741a6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48340/0 (socket says 172.21.15.69:48340) 2024-03-19T01:18:18.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.746+0000 7f9472a00640 1 -- 172.21.15.69:0/2251926064 learned_addr learned my addr 172.21.15.69:0/2251926064 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:18.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.746+0000 7f9473400640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f947410fe30 0x7f94741ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.747+0000 7f9472a00640 1 -- 172.21.15.69:0/2251926064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947410e4b0 msgr2=0x7f94741a7310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-19T01:18:18.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.747+0000 7f9472a00640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947410e4b0 0x7f94741a7310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.747+0000 7f9472a00640 1 -- 172.21.15.69:0/2251926064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f947410fe30 msgr2=0x7f94741ab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:18.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.747+0000 7f9472a00640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f947410fe30 0x7f94741ab690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.747+0000 7f9472a00640 1 -- 172.21.15.69:0/2251926064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94741abd90 con 0x7f947410d900 2024-03-19T01:18:18.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.747+0000 7f9473400640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f947410fe30 0x7f94741ab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:18.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.747+0000 7f9472a00640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f947410d900 0x7f94741a6dd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f946c009ca0 tx=0x7f946c035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:18.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.748+0000 7f946b400640 1 -- 172.21.15.69:0/2251926064 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c039040 con 0x7f947410d900 2024-03-19T01:18:18.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.748+0000 7f946b400640 1 -- 172.21.15.69:0/2251926064 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f946c044070 con 0x7f947410d900 2024-03-19T01:18:18.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.748+0000 7f946b400640 1 -- 172.21.15.69:0/2251926064 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c004380 con 0x7f947410d900 2024-03-19T01:18:18.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.752+0000 7f9479c00640 1 -- 172.21.15.69:0/2251926064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94741ac020 con 0x7f947410d900 2024-03-19T01:18:18.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.752+0000 7f9479c00640 1 -- 172.21.15.69:0/2251926064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94741107e0 con 0x7f947410d900 2024-03-19T01:18:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.759+0000 7f947c9ac740 1 Processor -- start 2024-03-19T01:18:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.760+0000 7f946b400640 1 -- 172.21.15.69:0/2251926064 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f946c004520 con 0x7f947410d900 2024-03-19T01:18:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.762+0000 7f947c9ac740 1 -- start start 2024-03-19T01:18:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.762+0000 7f947c9ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10089c10 0x562b10089ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f947c9ac740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1008a530 0x562b10092bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f947c9ac740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b10095500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f947c9ac740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b0ffee380 con 0x562b1008a530 2024-03-19T01:18:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f947c9ac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b0ff96d20 con 0x562b10089c10 2024-03-19T01:18:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f947c9ac740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562b0ffea420 con 0x562b10093110 2024-03-19T01:18:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f9473400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10089c10 0x562b10089ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f9472000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b10095500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f9473400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10089c10 0x562b10089ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34946/0 (socket says 172.21.15.69:34946) 2024-03-19T01:18:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f9473400640 1 -- 172.21.15.69:0/3732565369 learned_addr learned my addr 172.21.15.69:0/3732565369 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f9472000640 1 -- 172.21.15.69:0/3732565369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10089c10 msgr2=0x562b10089ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:18.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f9472000640 1 --2- 172.21.15.69:0/3732565369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10089c10 0x562b10089ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f9472000640 1 -- 172.21.15.69:0/3732565369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1008a530 msgr2=0x562b10092bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:18.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f9472000640 1 --2- 172.21.15.69:0/3732565369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1008a530 0x562b10092bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.763+0000 7f9472000640 1 -- 172.21.15.69:0/3732565369 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0fd023f0 con 0x562b10093110 2024-03-19T01:18:18.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.764+0000 7f9472000640 1 --2- 172.21.15.69:0/3732565369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b10095500 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9474072860 tx=0x7f9464009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6a6f2d69a88ada3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:18.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.764+0000 7f945fe00640 1 -- 172.21.15.69:0/3732565369 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464015020 con 0x562b10093110 2024-03-19T01:18:18.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.764+0000 7f945fe00640 1 -- 172.21.15.69:0/3732565369 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f946400bda0 con 0x562b10093110 2024-03-19T01:18:18.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.764+0000 7f945fe00640 1 -- 172.21.15.69:0/3732565369 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464010380 con 0x562b10093110 2024-03-19T01:18:18.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.765+0000 7f947c9ac740 1 -- 172.21.15.69:0/3732565369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 msgr2=0x562b10095500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:18.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.765+0000 7f947c9ac740 1 --2- 172.21.15.69:0/3732565369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b10095500 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9474072860 tx=0x7f9464009b20 comp rx=0 tx=0).stop 2024-03-19T01:18:18.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.765+0000 7f947c9ac740 1 -- 172.21.15.69:0/3732565369 shutdown_connections 2024-03-19T01:18:18.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.765+0000 7f947c9ac740 1 --2- 172.21.15.69:0/3732565369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b10095500 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.765+0000 7f947c9ac740 1 --2- 172.21.15.69:0/3732565369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1008a530 0x562b10092bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.765+0000 7f947c9ac740 1 --2- 172.21.15.69:0/3732565369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10089c10 0x562b10089ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.765+0000 7f947c9ac740 1 -- 172.21.15.69:0/3732565369 >> 172.21.15.69:0/3732565369 conn(0x562b10084340 msgr2=0x562b10084720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:18.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.767+0000 7f947c9ac740 1 -- 172.21.15.69:0/3732565369 shutdown_connections 2024-03-19T01:18:18.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.767+0000 7f947c9ac740 1 -- 172.21.15.69:0/3732565369 wait complete. 2024-03-19T01:18:18.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.767+0000 7f946b400640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f94580a2940 0x7f94580a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.767+0000 7f946b400640 1 -- 172.21.15.69:0/2251926064 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f946c0f0ea0 con 0x7f947410d900 2024-03-19T01:18:18.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.767+0000 7f947c9ac740 1 -- start start 2024-03-19T01:18:18.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.768+0000 7f947c9ac740 1 Processor -- start 2024-03-19T01:18:18.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.771+0000 7f9472000640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f94580a2940 0x7f94580a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.772+0000 7f9472000640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f94580a2940 0x7f94580a4e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9464009450 tx=0x7f946400bf50 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:18.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.772+0000 7f947c9ac740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 0x562b10090190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.772+0000 7f947c9ac740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b1008a530 0x562b100906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.772+0000 7f947c9ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10093110 0x562b10090c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.772+0000 7f947c9ac740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b10095f00 con 0x562b10089c10 2024-03-19T01:18:18.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.772+0000 7f947c9ac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b0ffaa160 con 0x562b10093110 2024-03-19T01:18:18.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.772+0000 7f947c9ac740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562b10096080 con 0x562b1008a530 2024-03-19T01:18:18.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.772+0000 7f947c9ac740 10 client.0 _ll_register_callbacks cb 0x562b0ffe6170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:18.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.773+0000 7f9472000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b1008a530 0x562b100906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.773+0000 7f9472000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b1008a530 0x562b100906d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48362/0 (socket says 172.21.15.69:48362) 2024-03-19T01:18:18.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.773+0000 7f9472000640 1 -- 172.21.15.69:0/1940745111 learned_addr learned my addr 172.21.15.69:0/1940745111 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:18.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.773+0000 7f9472a00640 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 0x562b10090190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.773+0000 7f9473400640 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10093110 0x562b10090c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.773+0000 7f9472000640 1 -- 172.21.15.69:0/1940745111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10093110 msgr2=0x562b10090c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.773+0000 7f9472000640 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10093110 0x562b10090c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.773+0000 7f9472000640 1 -- 172.21.15.69:0/1940745111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 msgr2=0x562b10090190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.773+0000 7f9472000640 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 0x562b10090190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.774+0000 7f9472000640 1 -- 172.21.15.69:0/1940745111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b0fdcc7d0 con 0x562b1008a530 2024-03-19T01:18:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.774+0000 7f9472a00640 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 0x562b10090190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.774+0000 7f9473400640 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10093110 0x562b10090c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.774+0000 7f9472000640 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b1008a530 0x562b100906d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9464016000 tx=0x7f9464011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dc582230cb1e415 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.774+0000 7f945cc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:18:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.774+0000 7f945cc00640 1 -- 172.21.15.69:0/1940745111 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946402a070 con 0x562b1008a530 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.775+0000 7f945cc00640 1 -- 172.21.15.69:0/1940745111 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f94640036b0 con 0x562b1008a530 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.775+0000 7f947c9ac740 1 -- 172.21.15.69:0/1940745111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f946400bda0 con 0x562b1008a530 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.775+0000 7f945cc00640 1 -- 172.21.15.69:0/1940745111 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9464038050 con 0x562b1008a530 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.775+0000 7f947c9ac740 10 client.27262 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.775+0000 7f947c9ac740 1 -- 172.21.15.69:0/1940745111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562b100a2270 con 0x562b1008a530 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.775+0000 7f945cc00640 1 -- 172.21.15.69:0/1940745111 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f946400bda0 con 0x562b1008a530 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.776+0000 7f945cc00640 1 -- 172.21.15.69:0/1940745111 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f946402f7f0 con 0x562b1008a530 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.776+0000 7f947c9ac740 10 client.27262 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.776+0000 7f947c9ac740 10 client.27262 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.776+0000 7f947c9ac740 20 client.27262 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.776+0000 7f947c9ac740 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x562b100a36d0 0x562b100a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.776+0000 7f947c9ac740 4 client.27262 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.776+0000 7f947c9ac740 1 -- 172.21.15.69:0/1940745111 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b0fffb8c0 con 0x562b100a36d0 2024-03-19T01:18:18.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.776+0000 7f9472a00640 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x562b100a36d0 0x562b100a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.777+0000 7f9472a00640 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x562b100a36d0 0x562b100a5ac0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f315afe1dd4948cf server_cookie=a668da6342128180 in_seq=0 out_seq=0 2024-03-19T01:18:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f945cc00640 10 client.27262 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:18:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f945cc00640 1 -- 172.21.15.69:0/1940745111 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562b0fffb8c0 con 0x562b100a36d0 2024-03-19T01:18:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f945cc00640 10 client.27262 handle_command_reply: tid=0 2024-03-19T01:18:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f947c9ac740 1 client.27262 shutdown 2024-03-19T01:18:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f947c9ac740 1 -- 172.21.15.69:0/1940745111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b1008a530 msgr2=0x562b100906d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f947c9ac740 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b1008a530 0x562b100906d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9464016000 tx=0x7f9464011b50 comp rx=0 tx=0).stop 2024-03-19T01:18:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f947c9ac740 1 -- 172.21.15.69:0/1940745111 shutdown_connections 2024-03-19T01:18:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f947c9ac740 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x562b100a36d0 0x562b100a5ac0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f947c9ac740 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10093110 0x562b10090c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f947c9ac740 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b1008a530 0x562b100906d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f947c9ac740 1 --2- 172.21.15.69:0/1940745111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 0x562b10090190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f947c9ac740 1 -- 172.21.15.69:0/1940745111 >> 172.21.15.69:0/1940745111 conn(0x562b10084340 msgr2=0x562b10086090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.779+0000 7f945cc00640 0 client.27262 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:18:18.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.782+0000 7f947c9ac740 1 -- 172.21.15.69:0/1940745111 shutdown_connections 2024-03-19T01:18:18.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.782+0000 7f947c9ac740 1 -- 172.21.15.69:0/1940745111 wait complete. 2024-03-19T01:18:18.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.782+0000 7f947c9ac740 20 client.27262 trim_cache size 0 max 16384 2024-03-19T01:18:18.837 INFO:tasks.workunit.client.0.smithi069.stdout: ASN.1 crypto/asymmetric_keys/pkcs7.asn1.[ch] 2024-03-19T01:18:18.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.840+0000 7f947c9ac740 1 Processor -- start 2024-03-19T01:18:18.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.840+0000 7f947c9ac740 1 -- start start 2024-03-19T01:18:18.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.840+0000 7f947c9ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10089c10 0x562b100928c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.840+0000 7f947c9ac740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1008a530 0x562b1009ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.840+0000 7f947c9ac740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b100ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.840+0000 7f947c9ac740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f94741c6450 con 0x562b1008a530 2024-03-19T01:18:18.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.840+0000 7f947c9ac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94741c8110 con 0x562b10089c10 2024-03-19T01:18:18.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.840+0000 7f947c9ac740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f94741c6b20 con 0x562b10093110 2024-03-19T01:18:18.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.841+0000 7f9472000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1008a530 0x562b1009ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.841+0000 7f9472000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1008a530 0x562b1009ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41362/0 (socket says 172.21.15.69:41362) 2024-03-19T01:18:18.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.841+0000 7f9472000640 1 -- 172.21.15.69:0/1205835041 learned_addr learned my addr 172.21.15.69:0/1205835041 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:18.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.842+0000 7f9472000640 1 -- 172.21.15.69:0/1205835041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 msgr2=0x562b100ab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:18.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.842+0000 7f9472000640 1 --2- 172.21.15.69:0/1205835041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b100ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.842+0000 7f9472000640 1 -- 172.21.15.69:0/1205835041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10089c10 msgr2=0x562b100928c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:18:18.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.842+0000 7f9472000640 1 --2- 172.21.15.69:0/1205835041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10089c10 0x562b100928c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.842+0000 7f9472000640 1 -- 172.21.15.69:0/1205835041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b100a5e30 con 0x562b1008a530 2024-03-19T01:18:18.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.842+0000 7f9472000640 1 --2- 172.21.15.69:0/1205835041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1008a530 0x562b1009ed20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f94640047b0 tx=0x7f9464011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97395e8d11f9e26b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:18.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.843+0000 7f945f400640 1 -- 172.21.15.69:0/1205835041 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946402a040 con 0x562b1008a530 2024-03-19T01:18:18.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.843+0000 7f945f400640 1 -- 172.21.15.69:0/1205835041 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9464041070 con 0x562b1008a530 2024-03-19T01:18:18.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.843+0000 7f945f400640 1 -- 172.21.15.69:0/1205835041 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464015040 con 0x562b1008a530 2024-03-19T01:18:18.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.844+0000 7f947c9ac740 1 -- 172.21.15.69:0/1205835041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1008a530 msgr2=0x562b1009ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:18.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.844+0000 7f947c9ac740 1 --2- 172.21.15.69:0/1205835041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1008a530 0x562b1009ed20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f94640047b0 tx=0x7f9464011860 comp rx=0 tx=0).stop 2024-03-19T01:18:18.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.845+0000 7f947c9ac740 1 -- 172.21.15.69:0/1205835041 shutdown_connections 2024-03-19T01:18:18.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.845+0000 7f947c9ac740 1 --2- 172.21.15.69:0/1205835041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b100ab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.845+0000 7f947c9ac740 1 --2- 172.21.15.69:0/1205835041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1008a530 0x562b1009ed20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.845+0000 7f947c9ac740 1 --2- 172.21.15.69:0/1205835041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b10089c10 0x562b100928c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.845+0000 7f947c9ac740 1 -- 172.21.15.69:0/1205835041 >> 172.21.15.69:0/1205835041 conn(0x562b10084340 msgr2=0x562b1007f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:18.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.845+0000 7f947c9ac740 1 -- 172.21.15.69:0/1205835041 shutdown_connections 2024-03-19T01:18:18.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.847+0000 7f947c9ac740 1 -- 172.21.15.69:0/1205835041 wait complete. 2024-03-19T01:18:18.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.847+0000 7f947c9ac740 1 -- start start 2024-03-19T01:18:18.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.847+0000 7f947c9ac740 1 Processor -- start 2024-03-19T01:18:18.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.848+0000 7f947c9ac740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 0x562b10101c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.848+0000 7f947c9ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b1008a530 0x562b101021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.849+0000 7f9472a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 0x562b10101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.849+0000 7f9472a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 0x562b10101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41372/0 (socket says 172.21.15.69:41372) 2024-03-19T01:18:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f9472a00640 1 -- 172.21.15.69:0/194114464 learned_addr learned my addr 172.21.15.69:0/194114464 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f947c9ac740 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b10096430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f947c9ac740 1 -- 172.21.15.69:0/194114464 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f94741c7980 con 0x562b10089c10 2024-03-19T01:18:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f947c9ac740 1 -- 172.21.15.69:0/194114464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94741c72b0 con 0x562b1008a530 2024-03-19T01:18:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f947c9ac740 1 -- 172.21.15.69:0/194114464 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9474068530 con 0x562b10093110 2024-03-19T01:18:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f947c9ac740 10 client.0 _ll_register_callbacks cb 0x7f946401e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f9472000640 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b1008a530 0x562b101021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f9473400640 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b10096430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f9472000640 1 -- 172.21.15.69:0/194114464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 msgr2=0x562b10096430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f9472000640 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b10096430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f9472000640 1 -- 172.21.15.69:0/194114464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 msgr2=0x562b10101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f9472000640 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 0x562b10101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.852+0000 7f9472000640 1 -- 172.21.15.69:0/194114464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b10096b20 con 0x562b1008a530 2024-03-19T01:18:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.853+0000 7f9472a00640 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 0x562b10101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.854+0000 7f9472000640 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b1008a530 0x562b101021c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x562b10084e90 tx=0x7f9464011d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5d893b9443515c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.854+0000 7f945cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:18:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.854+0000 7f945cc00640 1 -- 172.21.15.69:0/194114464 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946404a060 con 0x562b1008a530 2024-03-19T01:18:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.854+0000 7f945cc00640 1 -- 172.21.15.69:0/194114464 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f946402f660 con 0x562b1008a530 2024-03-19T01:18:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.854+0000 7f947c9ac740 1 -- 172.21.15.69:0/194114464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9464041070 con 0x562b1008a530 2024-03-19T01:18:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.855+0000 7f945cc00640 1 -- 172.21.15.69:0/194114464 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9464015070 con 0x562b1008a530 2024-03-19T01:18:18.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.858+0000 7f947c9ac740 10 client.27305 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:18.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.858+0000 7f947c9ac740 1 -- 172.21.15.69:0/194114464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562b101093d0 con 0x562b1008a530 2024-03-19T01:18:18.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.858+0000 7f945cc00640 1 -- 172.21.15.69:0/194114464 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9464041070 con 0x562b1008a530 2024-03-19T01:18:18.859 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/pkcs7_trust.o 2024-03-19T01:18:18.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.859+0000 7f945cc00640 1 -- 172.21.15.69:0/194114464 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f9464045450 con 0x562b1008a530 2024-03-19T01:18:18.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.859+0000 7f947c9ac740 10 client.27305 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:18.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.859+0000 7f947c9ac740 10 client.27305 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:18:18.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.859+0000 7f947c9ac740 20 client.27305 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:18.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.859+0000 7f947c9ac740 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x562b10109910 0x562b1010bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:18.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.859+0000 7f947c9ac740 4 client.27305 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:18:18.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.859+0000 7f947c9ac740 1 -- 172.21.15.69:0/194114464 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b0fff5c10 con 0x562b10109910 2024-03-19T01:18:18.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.860+0000 7f9472a00640 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x562b10109910 0x562b1010bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:18.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.861+0000 7f945cc00640 10 client.27305 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:18:18.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.861+0000 7f9472a00640 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x562b10109910 0x562b1010bd30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5bb3a683704d140 server_cookie=3efcc3ba8d3a08f7 in_seq=0 out_seq=0 2024-03-19T01:18:18.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f945cc00640 1 -- 172.21.15.69:0/194114464 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b0fff5c10 con 0x562b10109910 2024-03-19T01:18:18.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f945cc00640 10 client.27305 handle_command_reply: tid=0 2024-03-19T01:18:18.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f947c9ac740 1 client.27305 shutdown 2024-03-19T01:18:18.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f947c9ac740 1 -- 172.21.15.69:0/194114464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b1008a530 msgr2=0x562b101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:18.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f947c9ac740 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b1008a530 0x562b101021c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x562b10084e90 tx=0x7f9464011d80 comp rx=0 tx=0).stop 2024-03-19T01:18:18.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f947c9ac740 1 -- 172.21.15.69:0/194114464 shutdown_connections 2024-03-19T01:18:18.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f947c9ac740 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x562b10109910 0x562b1010bd30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f947c9ac740 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562b10093110 0x562b10096430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f947c9ac740 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b1008a530 0x562b101021c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f947c9ac740 1 --2- 172.21.15.69:0/194114464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b10089c10 0x562b10101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f947c9ac740 1 -- 172.21.15.69:0/194114464 >> 172.21.15.69:0/194114464 conn(0x562b10084340 msgr2=0x562b10085770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:18.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.867+0000 7f945cc00640 0 client.27305 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:18:18.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.869+0000 7f947c9ac740 1 -- 172.21.15.69:0/194114464 shutdown_connections 2024-03-19T01:18:18.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.869+0000 7f947c9ac740 1 -- 172.21.15.69:0/194114464 wait complete. 2024-03-19T01:18:18.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.869+0000 7f947c9ac740 20 client.27305 trim_cache size 0 max 16384 2024-03-19T01:18:18.871 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:18:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 -- 172.21.15.69:0/2251926064 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f94580a2940 msgr2=0x7f94580a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f94580a2940 0x7f94580a4e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9464009450 tx=0x7f946400bf50 comp rx=0 tx=0).stop 2024-03-19T01:18:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 -- 172.21.15.69:0/2251926064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f947410d900 msgr2=0x7f94741a6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f947410d900 0x7f94741a6dd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f946c009ca0 tx=0x7f946c035d10 comp rx=0 tx=0).stop 2024-03-19T01:18:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 -- 172.21.15.69:0/2251926064 shutdown_connections 2024-03-19T01:18:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f947410fe30 0x7f94741ab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f94580a2940 0x7f94580a4e00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947410e4b0 0x7f94741a7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 --2- 172.21.15.69:0/2251926064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f947410d900 0x7f94741a6dd0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:18.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 -- 172.21.15.69:0/2251926064 >> 172.21.15.69:0/2251926064 conn(0x7f947406c700 msgr2=0x7f9474070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:18.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 -- 172.21.15.69:0/2251926064 shutdown_connections 2024-03-19T01:18:18.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:18.874+0000 7f9479c00640 1 -- 172.21.15.69:0/2251926064 wait complete. 2024-03-19T01:18:18.926 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:18:19.157 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/common.o 2024-03-19T01:18:19.180 INFO:tasks.workunit.client.0.smithi069.stdout: CC ipc/mq_sysctl.o 2024-03-19T01:18:19.197 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/request_key.o 2024-03-19T01:18:19.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:18 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/667508359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:18:19.274 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/iomap/iter.o 2024-03-19T01:18:19.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:18 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/667508359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:18:19.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.303+0000 7f4905600640 1 -- 172.21.15.69:0/3864373055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49001144c0 msgr2=0x7f49001168b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:19.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.303+0000 7f4905600640 1 --2- 172.21.15.69:0/3864373055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49001144c0 0x7f49001168b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009900 tx=0x7f48f002bc30 comp rx=0 tx=0).stop 2024-03-19T01:18:19.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.306+0000 7f4905600640 1 -- 172.21.15.69:0/3864373055 shutdown_connections 2024-03-19T01:18:19.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.306+0000 7f4905600640 1 --2- 172.21.15.69:0/3864373055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49001144c0 0x7f49001168b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.306+0000 7f4905600640 1 --2- 172.21.15.69:0/3864373055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4900108d60 0x7f49001091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.306+0000 7f4905600640 1 --2- 172.21.15.69:0/3864373055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f490006dab0 0x7f4900108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.306+0000 7f4905600640 1 -- 172.21.15.69:0/3864373055 >> 172.21.15.69:0/3864373055 conn(0x7f490006c6d0 msgr2=0x7f490006cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:19.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.307+0000 7f4905600640 1 -- 172.21.15.69:0/3864373055 shutdown_connections 2024-03-19T01:18:19.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.307+0000 7f4905600640 1 -- 172.21.15.69:0/3864373055 wait complete. 2024-03-19T01:18:19.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.308+0000 7f4905600640 1 Processor -- start 2024-03-19T01:18:19.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:18 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/667508359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:18:19.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.312+0000 7f4905600640 1 -- start start 2024-03-19T01:18:19.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.312+0000 7f4905600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f490006dab0 0x7f49001a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.312+0000 7f4905600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4900108d60 0x7f49001a8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.312+0000 7f4905600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49001144c0 0x7f49001a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.312+0000 7f4905600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f490011a140 con 0x7f4900108d60 2024-03-19T01:18:19.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.312+0000 7f4905600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4900119fc0 con 0x7f49001144c0 2024-03-19T01:18:19.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.312+0000 7f4905600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f490011a2c0 con 0x7f490006dab0 2024-03-19T01:18:19.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.313+0000 7f48ffe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f490006dab0 0x7f49001a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.313+0000 7f4904c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49001144c0 0x7f49001a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.313+0000 7f4904c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49001144c0 0x7f49001a2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34986/0 (socket says 172.21.15.69:34986) 2024-03-19T01:18:19.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.313+0000 7f48ffe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f490006dab0 0x7f49001a8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48390/0 (socket says 172.21.15.69:48390) 2024-03-19T01:18:19.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.313+0000 7f4904c00640 1 -- 172.21.15.69:0/4232690330 learned_addr learned my addr 172.21.15.69:0/4232690330 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:19.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.313+0000 7f4904c00640 1 -- 172.21.15.69:0/4232690330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f490006dab0 msgr2=0x7f49001a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:19.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.313+0000 7f4904c00640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f490006dab0 0x7f49001a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.313+0000 7f4904c00640 1 -- 172.21.15.69:0/4232690330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4900108d60 msgr2=0x7f49001a8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:19.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.313+0000 7f4904c00640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4900108d60 0x7f49001a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.313+0000 7f4904c00640 1 -- 172.21.15.69:0/4232690330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49001a2d80 con 0x7f49001144c0 2024-03-19T01:18:19.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.314+0000 7f4904c00640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49001144c0 0x7f49001a2600 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009850 tx=0x7f48f00045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:19.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.314+0000 7f48fcc00640 1 -- 172.21.15.69:0/4232690330 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0043020 con 0x7f49001144c0 2024-03-19T01:18:19.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.314+0000 7f48fcc00640 1 -- 172.21.15.69:0/4232690330 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f00054e0 con 0x7f49001144c0 2024-03-19T01:18:19.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.314+0000 7f48fcc00640 1 -- 172.21.15.69:0/4232690330 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0003de0 con 0x7f49001144c0 2024-03-19T01:18:19.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.314+0000 7f4905600640 1 -- 172.21.15.69:0/4232690330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49001a3010 con 0x7f49001144c0 2024-03-19T01:18:19.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.314+0000 7f4905600640 1 -- 172.21.15.69:0/4232690330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490010c3d0 con 0x7f49001144c0 2024-03-19T01:18:19.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.324+0000 7f49099eb740 1 Processor -- start 2024-03-19T01:18:19.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.328+0000 7f49099eb740 1 -- start start 2024-03-19T01:18:19.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.330+0000 7f49099eb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.330+0000 7f49099eb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.330+0000 7f49099eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749e7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.330+0000 7f49099eb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560d74940380 con 0x560d749dbc30 2024-03-19T01:18:19.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.330+0000 7f49099eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d748e8d20 con 0x560d749e5130 2024-03-19T01:18:19.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.330+0000 7f48ffe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.330+0000 7f48ffe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48400/0 (socket says 172.21.15.69:48400) 2024-03-19T01:18:19.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.330+0000 7f48ffe00640 1 -- 172.21.15.69:0/2168742993 learned_addr learned my addr 172.21.15.69:0/2168742993 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:19.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.331+0000 7f49099eb740 1 -- 172.21.15.69:0/2168742993 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560d7493c420 con 0x560d749dc550 2024-03-19T01:18:19.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.331+0000 7f48fcc00640 1 -- 172.21.15.69:0/4232690330 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f48f0046020 con 0x7f49001144c0 2024-03-19T01:18:19.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.332+0000 7f48ff400640 1 --2- 172.21.15.69:0/2168742993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.333+0000 7f48ff400640 1 -- 172.21.15.69:0/2168742993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 msgr2=0x560d749e4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:19.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.333+0000 7f48ff400640 1 --2- 172.21.15.69:0/2168742993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.333+0000 7f48ff400640 1 -- 172.21.15.69:0/2168742993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 msgr2=0x560d749e7520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:19.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.333+0000 7f48ff400640 1 --2- 172.21.15.69:0/2168742993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749e7520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.333+0000 7f48ff400640 1 -- 172.21.15.69:0/2168742993 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d746543f0 con 0x560d749dbc30 2024-03-19T01:18:19.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.333+0000 7f48fcc00640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f48e40a2ab0 0x7f48e40a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.333+0000 7f48fcc00640 1 -- 172.21.15.69:0/4232690330 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f48f00f12f0 con 0x7f49001144c0 2024-03-19T01:18:19.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.334+0000 7f48ff400640 1 --2- 172.21.15.69:0/2168742993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749dc010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f490010b5a0 tx=0x7f48ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabd507a3aeb742f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:19.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.334+0000 7f48ffe00640 1 --2- 172.21.15.69:0/2168742993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:19.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.334+0000 7f48f5200640 1 -- 172.21.15.69:0/2168742993 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00ccd0 con 0x560d749dbc30 2024-03-19T01:18:19.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.334+0000 7f48f5200640 1 -- 172.21.15.69:0/2168742993 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48ec0035f0 con 0x560d749dbc30 2024-03-19T01:18:19.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.334+0000 7f48f5200640 1 -- 172.21.15.69:0/2168742993 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00f690 con 0x560d749dbc30 2024-03-19T01:18:19.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.335+0000 7f49099eb740 1 -- 172.21.15.69:0/2168742993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 msgr2=0x560d749dc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:19.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.335+0000 7f49099eb740 1 --2- 172.21.15.69:0/2168742993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749dc010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f490010b5a0 tx=0x7f48ec00c2d0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.336+0000 7f48ffe00640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f48e40a2ab0 0x7f48e40a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.337+0000 7f48ffe00640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f48e40a2ab0 0x7f48e40a4f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x560d749db700 tx=0x7f48f800a590 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:19.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.339+0000 7f49099eb740 1 -- 172.21.15.69:0/2168742993 shutdown_connections 2024-03-19T01:18:19.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.339+0000 7f49099eb740 1 --2- 172.21.15.69:0/2168742993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.339+0000 7f49099eb740 1 --2- 172.21.15.69:0/2168742993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.339+0000 7f49099eb740 1 --2- 172.21.15.69:0/2168742993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749dc010 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.339+0000 7f49099eb740 1 -- 172.21.15.69:0/2168742993 >> 172.21.15.69:0/2168742993 conn(0x560d749d6360 msgr2=0x560d749d6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:19.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.342+0000 7f49099eb740 1 -- 172.21.15.69:0/2168742993 shutdown_connections 2024-03-19T01:18:19.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.342+0000 7f49099eb740 1 -- 172.21.15.69:0/2168742993 wait complete. 2024-03-19T01:18:19.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.343+0000 7f49099eb740 1 -- start start 2024-03-19T01:18:19.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.344+0000 7f49099eb740 1 Processor -- start 2024-03-19T01:18:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.347+0000 7f49099eb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.347+0000 7f49099eb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.347+0000 7f49099eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749e2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.347+0000 7f49099eb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560d749e7f20 con 0x560d749dbc30 2024-03-19T01:18:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.347+0000 7f49099eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d748fc160 con 0x560d749e5130 2024-03-19T01:18:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.347+0000 7f49099eb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560d749e80a0 con 0x560d749dc550 2024-03-19T01:18:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.347+0000 7f49099eb740 10 client.0 _ll_register_callbacks cb 0x560d74938170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.347+0000 7f48ff400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.348+0000 7f48ff400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48402/0 (socket says 172.21.15.69:48402) 2024-03-19T01:18:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.348+0000 7f48ff400640 1 -- 172.21.15.69:0/1443811567 learned_addr learned my addr 172.21.15.69:0/1443811567 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.348+0000 7f48ffe00640 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.348+0000 7f4904c00640 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749e2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.348+0000 7f48ff400640 1 -- 172.21.15.69:0/1443811567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 msgr2=0x560d749e2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.348+0000 7f48ff400640 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749e2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.348+0000 7f48ff400640 1 -- 172.21.15.69:0/1443811567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 msgr2=0x560d749e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.348+0000 7f48ff400640 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.348+0000 7f48ff400640 1 -- 172.21.15.69:0/1443811567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7471e7d0 con 0x560d749dc550 2024-03-19T01:18:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.348+0000 7f48ffe00640 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:19.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.349+0000 7f48ff400640 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e25d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006010 tx=0x7f48ec00cd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d672504e7ca228 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:19.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.352+0000 7f48e9600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:18:19.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.352+0000 7f48e9600640 1 -- 172.21.15.69:0/1443811567 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0037a0 con 0x560d749dc550 2024-03-19T01:18:19.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.352+0000 7f48e9600640 1 -- 172.21.15.69:0/1443811567 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f48ec003940 con 0x560d749dc550 2024-03-19T01:18:19.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.352+0000 7f49099eb740 1 -- 172.21.15.69:0/1443811567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48ec0035f0 con 0x560d749dc550 2024-03-19T01:18:19.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.352+0000 7f48e9600640 1 -- 172.21.15.69:0/1443811567 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f48ec003b40 con 0x560d749dc550 2024-03-19T01:18:19.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.352+0000 7f49099eb740 10 client.27271 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:19.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.352+0000 7f49099eb740 1 -- 172.21.15.69:0/1443811567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560d749f4120 con 0x560d749dc550 2024-03-19T01:18:19.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.353+0000 7f48e9600640 1 -- 172.21.15.69:0/1443811567 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48ec0035f0 con 0x560d749dc550 2024-03-19T01:18:19.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.354+0000 7f48e9600640 1 -- 172.21.15.69:0/1443811567 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f48ec004ba0 con 0x560d749dc550 2024-03-19T01:18:19.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.354+0000 7f49099eb740 10 client.27271 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:19.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.354+0000 7f49099eb740 10 client.27271 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:18:19.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.354+0000 7f49099eb740 20 client.27271 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:19.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.354+0000 7f49099eb740 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x560d749f54d0 0x560d749f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.354+0000 7f49099eb740 4 client.27271 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:19.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.354+0000 7f49099eb740 1 -- 172.21.15.69:0/1443811567 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d7494d8c0 con 0x560d749f54d0 2024-03-19T01:18:19.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.356+0000 7f48ffe00640 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x560d749f54d0 0x560d749f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.356+0000 7f48ffe00640 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x560d749f54d0 0x560d749f78c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b5cd64c2a80de66 server_cookie=6e6d295c47ba03c1 in_seq=0 out_seq=0 2024-03-19T01:18:19.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.356+0000 7f48e9600640 10 client.27271 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:18:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.357+0000 7f48e9600640 1 -- 172.21.15.69:0/1443811567 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560d7494d8c0 con 0x560d749f54d0 2024-03-19T01:18:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.357+0000 7f48e9600640 10 client.27271 handle_command_reply: tid=0 2024-03-19T01:18:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.357+0000 7f49099eb740 1 client.27271 shutdown 2024-03-19T01:18:19.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.360+0000 7f49099eb740 1 -- 172.21.15.69:0/1443811567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 msgr2=0x560d749e25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:19.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.360+0000 7f49099eb740 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e25d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006010 tx=0x7f48ec00cd00 comp rx=0 tx=0).stop 2024-03-19T01:18:19.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.360+0000 7f49099eb740 1 -- 172.21.15.69:0/1443811567 shutdown_connections 2024-03-19T01:18:19.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.360+0000 7f49099eb740 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x560d749f54d0 0x560d749f78c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.360+0000 7f49099eb740 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749e2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.360+0000 7f49099eb740 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dc550 0x560d749e25d0 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006010 tx=0x7f48ec00cd00 comp rx=0 tx=0).stop 2024-03-19T01:18:19.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.360+0000 7f49099eb740 1 --2- 172.21.15.69:0/1443811567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.360+0000 7f48e9600640 0 client.27271 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:18:19.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.360+0000 7f49099eb740 1 -- 172.21.15.69:0/1443811567 >> 172.21.15.69:0/1443811567 conn(0x560d749d6360 msgr2=0x560d749d80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:19.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.360+0000 7f49099eb740 1 -- 172.21.15.69:0/1443811567 shutdown_connections 2024-03-19T01:18:19.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.363+0000 7f49099eb740 1 -- 172.21.15.69:0/1443811567 wait complete. 2024-03-19T01:18:19.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.363+0000 7f49099eb740 20 client.27271 trim_cache size 0 max 16384 2024-03-19T01:18:19.387 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/locking/semaphore.o 2024-03-19T01:18:19.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.398+0000 7f49099eb740 1 Processor -- start 2024-03-19T01:18:19.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.398+0000 7f49099eb740 1 -- start start 2024-03-19T01:18:19.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.398+0000 7f49099eb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.398+0000 7f49099eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.398+0000 7f49099eb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749fcec0 0x560d749d4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.398+0000 7f49099eb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f48e00101b0 con 0x560d749dbc30 2024-03-19T01:18:19.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.398+0000 7f49099eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48e0011e70 con 0x560d749e5130 2024-03-19T01:18:19.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.398+0000 7f49099eb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f48e0010880 con 0x560d749fcec0 2024-03-19T01:18:19.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f4904c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749fcec0 0x560d749d4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f4904c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749fcec0 0x560d749d4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48408/0 (socket says 172.21.15.69:48408) 2024-03-19T01:18:19.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f4904c00640 1 -- 172.21.15.69:0/1489229005 learned_addr learned my addr 172.21.15.69:0/1489229005 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:19.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f48ffe00640 1 --2- 172.21.15.69:0/1489229005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f48ff400640 1 --2- 172.21.15.69:0/1489229005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f4904c00640 1 -- 172.21.15.69:0/1489229005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 msgr2=0x560d749f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:19.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f4904c00640 1 --2- 172.21.15.69:0/1489229005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749f0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f4904c00640 1 -- 172.21.15.69:0/1489229005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 msgr2=0x560d749e47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:19.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f4904c00640 1 --2- 172.21.15.69:0/1489229005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749e47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f4904c00640 1 -- 172.21.15.69:0/1489229005 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d749f7c30 con 0x560d749fcec0 2024-03-19T01:18:19.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f48ff400640 1 --2- 172.21.15.69:0/1489229005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:19.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.399+0000 7f48ffe00640 1 --2- 172.21.15.69:0/1489229005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:19.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.400+0000 7f4904c00640 1 --2- 172.21.15.69:0/1489229005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749fcec0 0x560d749d4b50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f48f0004650 tx=0x7f48f00bd210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66e6e1c5e4470562 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:19.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.400+0000 7f48ebe00640 1 -- 172.21.15.69:0/1489229005 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f00d6070 con 0x560d749fcec0 2024-03-19T01:18:19.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.400+0000 7f48ebe00640 1 -- 172.21.15.69:0/1489229005 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f0005690 con 0x560d749fcec0 2024-03-19T01:18:19.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.400+0000 7f48ebe00640 1 -- 172.21.15.69:0/1489229005 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f00ee870 con 0x560d749fcec0 2024-03-19T01:18:19.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.401+0000 7f49099eb740 1 -- 172.21.15.69:0/1489229005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749fcec0 msgr2=0x560d749d4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:19.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.401+0000 7f49099eb740 1 --2- 172.21.15.69:0/1489229005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749fcec0 0x560d749d4b50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f48f0004650 tx=0x7f48f00bd210 comp rx=0 tx=0).stop 2024-03-19T01:18:19.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.401+0000 7f49099eb740 1 -- 172.21.15.69:0/1489229005 shutdown_connections 2024-03-19T01:18:19.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.401+0000 7f49099eb740 1 --2- 172.21.15.69:0/1489229005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749fcec0 0x560d749d4b50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.401+0000 7f49099eb740 1 --2- 172.21.15.69:0/1489229005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.401+0000 7f49099eb740 1 --2- 172.21.15.69:0/1489229005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749dbc30 0x560d749e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.401+0000 7f49099eb740 1 -- 172.21.15.69:0/1489229005 >> 172.21.15.69:0/1489229005 conn(0x560d749d6360 msgr2=0x560d749dd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:19.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.403+0000 7f49099eb740 1 -- 172.21.15.69:0/1489229005 shutdown_connections 2024-03-19T01:18:19.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.403+0000 7f49099eb740 1 -- 172.21.15.69:0/1489229005 wait complete. 2024-03-19T01:18:19.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.403+0000 7f49099eb740 1 -- start start 2024-03-19T01:18:19.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.404+0000 7f49099eb740 1 Processor -- start 2024-03-19T01:18:19.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.408+0000 7f49099eb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dbc30 0x560d749e8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.408+0000 7f49099eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.408+0000 7f49099eb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749fcec0 0x560d74a5a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.408+0000 7f49099eb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f48e00116e0 con 0x560d749fcec0 2024-03-19T01:18:19.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.408+0000 7f49099eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48e0011010 con 0x560d749e5130 2024-03-19T01:18:19.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.408+0000 7f49099eb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f48e000e130 con 0x560d749dbc30 2024-03-19T01:18:19.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.408+0000 7f49099eb740 10 client.0 _ll_register_callbacks cb 0x7f48ec00a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:19.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.409+0000 7f48ffe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dbc30 0x560d749e8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.409+0000 7f48ffe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dbc30 0x560d749e8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48422/0 (socket says 172.21.15.69:48422) 2024-03-19T01:18:19.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.409+0000 7f48ffe00640 1 -- 172.21.15.69:0/3985679173 learned_addr learned my addr 172.21.15.69:0/3985679173 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:19.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.409+0000 7f48ff400640 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.409+0000 7f4904c00640 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749fcec0 0x560d74a5a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.409+0000 7f48ffe00640 1 -- 172.21.15.69:0/3985679173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 msgr2=0x560d749e94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:19.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.409+0000 7f48ffe00640 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749e94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.409+0000 7f48ffe00640 1 -- 172.21.15.69:0/3985679173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749fcec0 msgr2=0x560d74a5a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:19.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.409+0000 7f48ffe00640 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749fcec0 0x560d74a5a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.409+0000 7f48ffe00640 1 -- 172.21.15.69:0/3985679173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d749e9d60 con 0x560d749dbc30 2024-03-19T01:18:19.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.410+0000 7f4904c00640 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749fcec0 0x560d74a5a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:19.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.414+0000 7f48ffe00640 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dbc30 0x560d749e8f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x560d749f7e60 tx=0x7f48f8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e39dccf98c3cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:19.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.414+0000 7f48e9600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:18:19.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.414+0000 7f48e9600640 1 -- 172.21.15.69:0/3985679173 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f802b070 con 0x560d749dbc30 2024-03-19T01:18:19.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.414+0000 7f48e9600640 1 -- 172.21.15.69:0/3985679173 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f48f8027c40 con 0x560d749dbc30 2024-03-19T01:18:19.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.414+0000 7f49099eb740 1 -- 172.21.15.69:0/3985679173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560d74947c10 con 0x560d749dbc30 2024-03-19T01:18:19.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.415+0000 7f48e9600640 1 -- 172.21.15.69:0/3985679173 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f48f8036050 con 0x560d749dbc30 2024-03-19T01:18:19.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.415+0000 7f49099eb740 10 client.27277 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:19.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.415+0000 7f49099eb740 1 -- 172.21.15.69:0/3985679173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560d74a6cae0 con 0x560d749dbc30 2024-03-19T01:18:19.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.415+0000 7f48e9600640 1 -- 172.21.15.69:0/3985679173 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560d74947c10 con 0x560d749dbc30 2024-03-19T01:18:19.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.415+0000 7f48e9600640 1 -- 172.21.15.69:0/3985679173 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f48f800e880 con 0x560d749dbc30 2024-03-19T01:18:19.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.415+0000 7f49099eb740 10 client.27277 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:19.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.415+0000 7f49099eb740 10 client.27277 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:18:19.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.415+0000 7f49099eb740 20 client.27277 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:19.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.415+0000 7f49099eb740 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x560d74a6d270 0x560d74a6f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:19.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.415+0000 7f49099eb740 4 client.27277 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:18:19.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.415+0000 7f49099eb740 1 -- 172.21.15.69:0/3985679173 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d748c5560 con 0x560d74a6d270 2024-03-19T01:18:19.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.416+0000 7f48ff400640 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x560d74a6d270 0x560d74a6f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:19.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.416+0000 7f48e9600640 10 client.27277 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:18:19.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.419+0000 7f48ff400640 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x560d74a6d270 0x560d74a6f690 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=57bda88fea15ca39 server_cookie=38ad39bc869ee36c in_seq=0 out_seq=0 2024-03-19T01:18:19.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.420+0000 7f48e9600640 1 -- 172.21.15.69:0/3985679173 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d748c5560 con 0x560d74a6d270 2024-03-19T01:18:19.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.420+0000 7f48e9600640 10 client.27277 handle_command_reply: tid=0 2024-03-19T01:18:19.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.420+0000 7f49099eb740 1 client.27277 shutdown 2024-03-19T01:18:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.420+0000 7f49099eb740 1 -- 172.21.15.69:0/3985679173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dbc30 msgr2=0x560d749e8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.420+0000 7f49099eb740 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dbc30 0x560d749e8f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x560d749f7e60 tx=0x7f48f8004a40 comp rx=0 tx=0).stop 2024-03-19T01:18:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.420+0000 7f49099eb740 1 -- 172.21.15.69:0/3985679173 shutdown_connections 2024-03-19T01:18:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.420+0000 7f49099eb740 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d749fcec0 0x560d74a5a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.420+0000 7f49099eb740 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x560d74a6d270 0x560d74a6f690 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.420+0000 7f49099eb740 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d749e5130 0x560d749e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.420+0000 7f49099eb740 1 --2- 172.21.15.69:0/3985679173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d749dbc30 0x560d749e8f90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.420+0000 7f49099eb740 1 -- 172.21.15.69:0/3985679173 >> 172.21.15.69:0/3985679173 conn(0x560d749d6360 msgr2=0x560d749d4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.422+0000 7f48e9600640 0 client.27277 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:18:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.422+0000 7f49099eb740 1 -- 172.21.15.69:0/3985679173 shutdown_connections 2024-03-19T01:18:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.422+0000 7f49099eb740 1 -- 172.21.15.69:0/3985679173 wait complete. 2024-03-19T01:18:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.422+0000 7f49099eb740 20 client.27277 trim_cache size 0 max 16384 2024-03-19T01:18:19.425 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:18:19.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.428+0000 7f4905600640 1 -- 172.21.15.69:0/4232690330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f48e40a2ab0 msgr2=0x7f48e40a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:19.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.428+0000 7f4905600640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f48e40a2ab0 0x7f48e40a4f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x560d749db700 tx=0x7f48f800a590 comp rx=0 tx=0).stop 2024-03-19T01:18:19.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.428+0000 7f4905600640 1 -- 172.21.15.69:0/4232690330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49001144c0 msgr2=0x7f49001a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:19.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.428+0000 7f4905600640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49001144c0 0x7f49001a2600 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009850 tx=0x7f48f00045e0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.428+0000 7f4905600640 1 -- 172.21.15.69:0/4232690330 shutdown_connections 2024-03-19T01:18:19.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.428+0000 7f4905600640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f48e40a2ab0 0x7f48e40a4f70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.428+0000 7f4905600640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49001144c0 0x7f49001a2600 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.428+0000 7f4905600640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4900108d60 0x7f49001a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.428+0000 7f4905600640 1 --2- 172.21.15.69:0/4232690330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f490006dab0 0x7f49001a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:19.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.429+0000 7f4905600640 1 -- 172.21.15.69:0/4232690330 >> 172.21.15.69:0/4232690330 conn(0x7f490006c6d0 msgr2=0x7f490010a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:19.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.431+0000 7f4905600640 1 -- 172.21.15.69:0/4232690330 shutdown_connections 2024-03-19T01:18:19.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:19.431+0000 7f4905600640 1 -- 172.21.15.69:0/4232690330 wait complete. 2024-03-19T01:18:19.487 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/pkcs7_verify.o 2024-03-19T01:18:19.532 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:18:19.734 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/swap.o 2024-03-19T01:18:20.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:19 smithi069 ceph-mon[31727]: pgmap v236: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 640 KiB/s wr, 152 op/s 2024-03-19T01:18:20.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.080+0000 7fdd65800640 1 -- 172.21.15.69:0/3086509165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd601103f0 msgr2=0x7fdd6011d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:20.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.080+0000 7fdd65800640 1 --2- 172.21.15.69:0/3086509165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd601103f0 0x7fdd6011d020 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009cd0 tx=0x7fdd5802c0e0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.084+0000 7fdd65800640 1 -- 172.21.15.69:0/3086509165 shutdown_connections 2024-03-19T01:18:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.084+0000 7fdd65800640 1 --2- 172.21.15.69:0/3086509165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd601103f0 0x7fdd6011d020 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.084+0000 7fdd65800640 1 --2- 172.21.15.69:0/3086509165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60108df0 0x7fdd60109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.084+0000 7fdd65800640 1 --2- 172.21.15.69:0/3086509165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6010be00 0x7fdd60108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.084+0000 7fdd65800640 1 -- 172.21.15.69:0/3086509165 >> 172.21.15.69:0/3086509165 conn(0x7fdd6006c770 msgr2=0x7fdd6006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.084+0000 7fdd65800640 1 -- 172.21.15.69:0/3086509165 shutdown_connections 2024-03-19T01:18:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.084+0000 7fdd65800640 1 -- 172.21.15.69:0/3086509165 wait complete. 2024-03-19T01:18:20.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.086+0000 7fdd65800640 1 Processor -- start 2024-03-19T01:18:20.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.086+0000 7fdd65800640 1 -- start start 2024-03-19T01:18:20.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.086+0000 7fdd65800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60108df0 0x7fdd601a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd65800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd6010be00 0x7fdd601a7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd65800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd601ab5e0 0x7fdd601aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd65800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdd60120840 con 0x7fdd601ab5e0 2024-03-19T01:18:20.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd65800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd601206c0 con 0x7fdd6010be00 2024-03-19T01:18:20.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd65800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdd601209c0 con 0x7fdd60108df0 2024-03-19T01:18:20.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd5fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60108df0 0x7fdd601a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd5fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60108df0 0x7fdd601a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48436/0 (socket says 172.21.15.69:48436) 2024-03-19T01:18:20.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd5fe00640 1 -- 172.21.15.69:0/3840797531 learned_addr learned my addr 172.21.15.69:0/3840797531 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:20.092 INFO:tasks.workunit.client.0.smithi069.stdout: AR ipc/built-in.a 2024-03-19T01:18:20.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd5f400640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd6010be00 0x7fdd601a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd5fe00640 1 -- 172.21.15.69:0/3840797531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd6010be00 msgr2=0x7fdd601a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd5fe00640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd6010be00 0x7fdd601a7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.089+0000 7fdd5fe00640 1 -- 172.21.15.69:0/3840797531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd601ab5e0 msgr2=0x7fdd601aba90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.090+0000 7fdd64e00640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd601ab5e0 0x7fdd601aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.090+0000 7fdd5fe00640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd601ab5e0 0x7fdd601aba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.090+0000 7fdd5fe00640 1 -- 172.21.15.69:0/3840797531 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd601ac0d0 con 0x7fdd60108df0 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.090+0000 7fdd5f400640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd6010be00 0x7fdd601a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.090+0000 7fdd64e00640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd601ab5e0 0x7fdd601aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.090+0000 7fdd5fe00640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60108df0 0x7fdd601a6d20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000d950 tx=0x7fdd5000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.090+0000 7fdd5cc00640 1 -- 172.21.15.69:0/3840797531 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50017030 con 0x7fdd60108df0 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.090+0000 7fdd5cc00640 1 -- 172.21.15.69:0/3840797531 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd50004020 con 0x7fdd60108df0 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.090+0000 7fdd5cc00640 1 -- 172.21.15.69:0/3840797531 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50003d60 con 0x7fdd60108df0 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.091+0000 7fdd65800640 1 -- 172.21.15.69:0/3840797531 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd601ac3c0 con 0x7fdd60108df0 2024-03-19T01:18:20.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.091+0000 7fdd65800640 1 -- 172.21.15.69:0/3840797531 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd601b3b00 con 0x7fdd60108df0 2024-03-19T01:18:20.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.098+0000 7fdd5cc00640 1 -- 172.21.15.69:0/3840797531 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fdd5001a070 con 0x7fdd60108df0 2024-03-19T01:18:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.100+0000 7fdd5cc00640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdd440a29f0 0x7fdd440a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.100+0000 7fdd5cc00640 1 -- 172.21.15.69:0/3840797531 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7fdd500cedc0 con 0x7fdd60108df0 2024-03-19T01:18:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.101+0000 7fdd5f400640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdd440a29f0 0x7fdd440a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.104+0000 7fdd5f400640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdd440a29f0 0x7fdd440a4eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd6010afc0 tx=0x7fdd48005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.104+0000 7fdd69bc4740 1 Processor -- start 2024-03-19T01:18:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.105+0000 7fdd69bc4740 1 -- start start 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.105+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d71c10 0x562ec3d71ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.105+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d72530 0x562ec3d7abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.105+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d7d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.105+0000 7fdd69bc4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562ec3cd6380 con 0x562ec3d72530 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.105+0000 7fdd69bc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ec3c7ed20 con 0x562ec3d71c10 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.105+0000 7fdd69bc4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562ec3cd2420 con 0x562ec3d7b110 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.105+0000 7fdd64e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d7d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.106+0000 7fdd64e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d7d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48446/0 (socket says 172.21.15.69:48446) 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.106+0000 7fdd64e00640 1 -- 172.21.15.69:0/3901581132 learned_addr learned my addr 172.21.15.69:0/3901581132 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.106+0000 7fdd5f400640 1 --2- 172.21.15.69:0/3901581132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d72530 0x562ec3d7abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.106+0000 7fdd64e00640 1 -- 172.21.15.69:0/3901581132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d71c10 msgr2=0x562ec3d71ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.106+0000 7fdd64e00640 1 --2- 172.21.15.69:0/3901581132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d71c10 0x562ec3d71ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.106+0000 7fdd64e00640 1 -- 172.21.15.69:0/3901581132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d72530 msgr2=0x562ec3d7abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.106+0000 7fdd64e00640 1 --2- 172.21.15.69:0/3901581132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d72530 0x562ec3d7abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.106+0000 7fdd64e00640 1 -- 172.21.15.69:0/3901581132 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ec39ea3f0 con 0x562ec3d7b110 2024-03-19T01:18:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.106+0000 7fdd5f400640 1 --2- 172.21.15.69:0/3901581132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d72530 0x562ec3d7abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:18:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.108+0000 7fdd64e00640 1 --2- 172.21.15.69:0/3901581132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d7d500 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdd58004af0 tx=0x7fdd580323d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54e06539dcab4e6d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.109+0000 7fdd55600640 1 -- 172.21.15.69:0/3901581132 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58039050 con 0x562ec3d7b110 2024-03-19T01:18:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.111+0000 7fdd55600640 1 -- 172.21.15.69:0/3901581132 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd58045070 con 0x562ec3d7b110 2024-03-19T01:18:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.111+0000 7fdd55600640 1 -- 172.21.15.69:0/3901581132 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58004040 con 0x562ec3d7b110 2024-03-19T01:18:20.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.112+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3901581132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 msgr2=0x562ec3d7d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:20.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.112+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/3901581132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d7d500 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdd58004af0 tx=0x7fdd580323d0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.114+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3901581132 shutdown_connections 2024-03-19T01:18:20.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.114+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/3901581132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d7d500 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.114+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/3901581132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d72530 0x562ec3d7abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.114+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/3901581132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d71c10 0x562ec3d71ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.114+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3901581132 >> 172.21.15.69:0/3901581132 conn(0x562ec3d6c340 msgr2=0x562ec3d6c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.114+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3901581132 shutdown_connections 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.114+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3901581132 wait complete. 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.115+0000 7fdd69bc4740 1 -- start start 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.115+0000 7fdd69bc4740 1 Processor -- start 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.116+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d71c10 0x562ec3d78240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.116+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 0x562ec3d78780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.116+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d7b110 0x562ec3d78cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.116+0000 7fdd69bc4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562ec3d7dea0 con 0x562ec3d71c10 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.116+0000 7fdd5f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 0x562ec3d78780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.117+0000 7fdd5f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 0x562ec3d78780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48450/0 (socket says 172.21.15.69:48450) 2024-03-19T01:18:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.117+0000 7fdd64e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d7b110 0x562ec3d78cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.117+0000 7fdd5f400640 1 -- 172.21.15.69:0/3955400935 learned_addr learned my addr 172.21.15.69:0/3955400935 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.116+0000 7fdd69bc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ec3c92160 con 0x562ec3d7b110 2024-03-19T01:18:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.117+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3955400935 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562ec3d7e020 con 0x562ec3d72530 2024-03-19T01:18:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.117+0000 7fdd69bc4740 10 client.0 _ll_register_callbacks cb 0x562ec3cce170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.117+0000 7fdd5fe00640 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d71c10 0x562ec3d78240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.117+0000 7fdd5fe00640 1 -- 172.21.15.69:0/3955400935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 msgr2=0x562ec3d78780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.117+0000 7fdd5fe00640 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 0x562ec3d78780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.117+0000 7fdd5fe00640 1 -- 172.21.15.69:0/3955400935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d7b110 msgr2=0x562ec3d78cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.117+0000 7fdd5fe00640 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d7b110 0x562ec3d78cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.117+0000 7fdd5fe00640 1 -- 172.21.15.69:0/3955400935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ec3ab47d0 con 0x562ec3d71c10 2024-03-19T01:18:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.118+0000 7fdd5f400640 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 0x562ec3d78780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.118+0000 7fdd64e00640 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d7b110 0x562ec3d78cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.122+0000 7fdd5fe00640 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d71c10 0x562ec3d78240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdd500cc100 tx=0x7fdd5009ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321b910e1732cebe server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.122+0000 7fdd4e000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:18:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.122+0000 7fdd4e000640 1 -- 172.21.15.69:0/3955400935 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5009b010 con 0x562ec3d71c10 2024-03-19T01:18:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.122+0000 7fdd4e000640 1 -- 172.21.15.69:0/3955400935 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7fdd50097680 con 0x562ec3d71c10 2024-03-19T01:18:20.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.125+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3955400935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd58045070 con 0x562ec3d71c10 2024-03-19T01:18:20.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.125+0000 7fdd4e000640 1 -- 172.21.15.69:0/3955400935 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fdd5009f160 con 0x562ec3d71c10 2024-03-19T01:18:20.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.125+0000 7fdd69bc4740 10 client.17028 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:20.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.125+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3955400935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562ec3d8a2a0 con 0x562ec3d71c10 2024-03-19T01:18:20.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.128+0000 7fdd4e000640 1 -- 172.21.15.69:0/3955400935 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd58045070 con 0x562ec3d71c10 2024-03-19T01:18:20.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.128+0000 7fdd4e000640 1 -- 172.21.15.69:0/3955400935 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7fdd50024060 con 0x562ec3d71c10 2024-03-19T01:18:20.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.129+0000 7fdd69bc4740 10 client.17028 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:20.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.129+0000 7fdd69bc4740 10 client.17028 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:18:20.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.129+0000 7fdd69bc4740 20 client.17028 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:20.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.129+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562ec3d8b700 0x562ec3d8db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.129+0000 7fdd69bc4740 4 client.17028 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:20.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.129+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3955400935 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ec3ce38c0 con 0x562ec3d8b700 2024-03-19T01:18:20.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.129+0000 7fdd5f400640 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562ec3d8b700 0x562ec3d8db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.133+0000 7fdd5f400640 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562ec3d8b700 0x562ec3d8db10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=269c43cd1ff7d1 server_cookie=d5bd0c82060f8f4f in_seq=0 out_seq=0 2024-03-19T01:18:20.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.133+0000 7fdd4e000640 10 client.17028 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:18:20.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.133+0000 7fdd4e000640 1 -- 172.21.15.69:0/3955400935 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562ec3ce38c0 con 0x562ec3d8b700 2024-03-19T01:18:20.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.133+0000 7fdd4e000640 10 client.17028 handle_command_reply: tid=0 2024-03-19T01:18:20.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.134+0000 7fdd69bc4740 1 client.17028 shutdown 2024-03-19T01:18:20.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.134+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3955400935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d71c10 msgr2=0x562ec3d78240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:20.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.134+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d71c10 0x562ec3d78240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdd500cc100 tx=0x7fdd5009ac10 comp rx=0 tx=0).stop 2024-03-19T01:18:20.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.135+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3955400935 shutdown_connections 2024-03-19T01:18:20.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.135+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562ec3d8b700 0x562ec3d8db10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.135+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d7b110 0x562ec3d78cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.135+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 0x562ec3d78780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.135+0000 7fdd4e000640 0 client.17028 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:18:20.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.135+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/3955400935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d71c10 0x562ec3d78240 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.135+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3955400935 >> 172.21.15.69:0/3955400935 conn(0x562ec3d6c340 msgr2=0x562ec3d6e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:20.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.135+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3955400935 shutdown_connections 2024-03-19T01:18:20.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.135+0000 7fdd69bc4740 1 -- 172.21.15.69:0/3955400935 wait complete. 2024-03-19T01:18:20.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.135+0000 7fdd69bc4740 20 client.17028 trim_cache size 0 max 16384 2024-03-19T01:18:20.184 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/entry/thunk_64.o 2024-03-19T01:18:20.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.194+0000 7fdd69bc4740 1 Processor -- start 2024-03-19T01:18:20.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.194+0000 7fdd69bc4740 1 -- start start 2024-03-19T01:18:20.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.195+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d71c10 0x562ec3d7a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.195+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d72530 0x562ec3d86c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.195+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.195+0000 7fdd69bc4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdd601b5250 con 0x562ec3d71c10 2024-03-19T01:18:20.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.195+0000 7fdd69bc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd601b6f10 con 0x562ec3d72530 2024-03-19T01:18:20.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.195+0000 7fdd69bc4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdd601b5920 con 0x562ec3d7b110 2024-03-19T01:18:20.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.195+0000 7fdd64e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.195+0000 7fdd5f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d72530 0x562ec3d86c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.195+0000 7fdd64e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d773a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48464/0 (socket says 172.21.15.69:48464) 2024-03-19T01:18:20.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.195+0000 7fdd64e00640 1 -- 172.21.15.69:0/2936853001 learned_addr learned my addr 172.21.15.69:0/2936853001 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:20.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.196+0000 7fdd64e00640 1 -- 172.21.15.69:0/2936853001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d72530 msgr2=0x562ec3d86c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:20.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.196+0000 7fdd64e00640 1 --2- 172.21.15.69:0/2936853001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d72530 0x562ec3d86c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.196+0000 7fdd64e00640 1 -- 172.21.15.69:0/2936853001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d71c10 msgr2=0x562ec3d7a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:20.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.196+0000 7fdd64e00640 1 --2- 172.21.15.69:0/2936853001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d71c10 0x562ec3d7a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.196+0000 7fdd64e00640 1 -- 172.21.15.69:0/2936853001 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ec3d8e190 con 0x562ec3d7b110 2024-03-19T01:18:20.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.196+0000 7fdd64e00640 1 --2- 172.21.15.69:0/2936853001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d773a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdd5802c7e0 tx=0x7fdd580095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2781d09bd9468b45 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:20.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.197+0000 7fdd54c00640 1 -- 172.21.15.69:0/2936853001 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5804f070 con 0x562ec3d7b110 2024-03-19T01:18:20.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.197+0000 7fdd54c00640 1 -- 172.21.15.69:0/2936853001 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd58045040 con 0x562ec3d7b110 2024-03-19T01:18:20.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.197+0000 7fdd54c00640 1 -- 172.21.15.69:0/2936853001 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58004040 con 0x562ec3d7b110 2024-03-19T01:18:20.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.198+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2936853001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 msgr2=0x562ec3d773a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:20.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.198+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/2936853001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d773a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdd5802c7e0 tx=0x7fdd580095c0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.200+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2936853001 shutdown_connections 2024-03-19T01:18:20.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.200+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/2936853001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d7b110 0x562ec3d773a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.200+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/2936853001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d72530 0x562ec3d86c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.200+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/2936853001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d71c10 0x562ec3d7a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.200+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2936853001 >> 172.21.15.69:0/2936853001 conn(0x562ec3d6c340 msgr2=0x562ec3d6d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:20.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.200+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2936853001 shutdown_connections 2024-03-19T01:18:20.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.200+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2936853001 wait complete. 2024-03-19T01:18:20.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.200+0000 7fdd69bc4740 1 -- start start 2024-03-19T01:18:20.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.201+0000 7fdd69bc4740 1 Processor -- start 2024-03-19T01:18:20.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.205+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d71c10 0x562ec3d83520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.205+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 0x562ec3d7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.205+0000 7fdd69bc4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d7b110 0x562ec3d7edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.205+0000 7fdd69bc4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdd601b6780 con 0x562ec3d7b110 2024-03-19T01:18:20.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.205+0000 7fdd69bc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd601b60b0 con 0x562ec3d71c10 2024-03-19T01:18:20.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.205+0000 7fdd69bc4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdd601a8070 con 0x562ec3d72530 2024-03-19T01:18:20.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.205+0000 7fdd69bc4740 10 client.0 _ll_register_callbacks cb 0x7fdd50005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:20.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.205+0000 7fdd5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d71c10 0x562ec3d83520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.205+0000 7fdd5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d71c10 0x562ec3d83520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35034/0 (socket says 172.21.15.69:35034) 2024-03-19T01:18:20.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.205+0000 7fdd5fe00640 1 -- 172.21.15.69:0/2180152594 learned_addr learned my addr 172.21.15.69:0/2180152594 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:20.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.205+0000 7fdd5f400640 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 0x562ec3d7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.206+0000 7fdd5f400640 1 -- 172.21.15.69:0/2180152594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d71c10 msgr2=0x562ec3d83520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:20.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.206+0000 7fdd5f400640 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d71c10 0x562ec3d83520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.206+0000 7fdd5f400640 1 -- 172.21.15.69:0/2180152594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d7b110 msgr2=0x562ec3d7edd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:20.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.206+0000 7fdd64e00640 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d7b110 0x562ec3d7edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.206+0000 7fdd5f400640 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d7b110 0x562ec3d7edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.207+0000 7fdd5f400640 1 -- 172.21.15.69:0/2180152594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ec3d7d870 con 0x562ec3d72530 2024-03-19T01:18:20.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.207+0000 7fdd64e00640 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d7b110 0x562ec3d7edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:20.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.207+0000 7fdd4e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:18:20.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.207+0000 7fdd5f400640 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 0x562ec3d7e890 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x562ec3d8e110 tx=0x7fdd48004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d719c2005bddde1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:20.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.208+0000 7fdd4e000640 1 -- 172.21.15.69:0/2180152594 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48004230 con 0x562ec3d72530 2024-03-19T01:18:20.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.208+0000 7fdd4e000640 1 -- 172.21.15.69:0/2180152594 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7fdd480043d0 con 0x562ec3d72530 2024-03-19T01:18:20.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.208+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2180152594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd58045040 con 0x562ec3d72530 2024-03-19T01:18:20.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.209+0000 7fdd4e000640 1 -- 172.21.15.69:0/2180152594 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fdd48003610 con 0x562ec3d72530 2024-03-19T01:18:20.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.209+0000 7fdd69bc4740 10 client.27292 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:20.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.209+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2180152594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562ec3df1900 con 0x562ec3d72530 2024-03-19T01:18:20.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.209+0000 7fdd4e000640 1 -- 172.21.15.69:0/2180152594 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd58045040 con 0x562ec3d72530 2024-03-19T01:18:20.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.209+0000 7fdd4e000640 1 -- 172.21.15.69:0/2180152594 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7fdd48040070 con 0x562ec3d72530 2024-03-19T01:18:20.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.210+0000 7fdd69bc4740 10 client.27292 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:20.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.210+0000 7fdd69bc4740 10 client.27292 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:18:20.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.210+0000 7fdd69bc4740 20 client.27292 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:20.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.210+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562ec3df20f0 0x562ec3df4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:20.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.210+0000 7fdd5fe00640 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562ec3df20f0 0x562ec3df4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:20.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.210+0000 7fdd69bc4740 4 client.27292 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:18:20.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.210+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2180152594 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ec3cddc10 con 0x562ec3df20f0 2024-03-19T01:18:20.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.211+0000 7fdd5fe00640 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562ec3df20f0 0x562ec3df4510 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3462d061b5331b5d server_cookie=67abd0ba637b77ff in_seq=0 out_seq=0 2024-03-19T01:18:20.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.212+0000 7fdd4e000640 10 client.27292 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:18:20.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.212+0000 7fdd4e000640 1 -- 172.21.15.69:0/2180152594 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ec3cddc10 con 0x562ec3df20f0 2024-03-19T01:18:20.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.212+0000 7fdd4e000640 10 client.27292 handle_command_reply: tid=0 2024-03-19T01:18:20.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.214+0000 7fdd69bc4740 1 client.27292 shutdown 2024-03-19T01:18:20.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.214+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2180152594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 msgr2=0x562ec3d7e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:20.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.214+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 0x562ec3d7e890 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x562ec3d8e110 tx=0x7fdd48004730 comp rx=0 tx=0).stop 2024-03-19T01:18:20.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.214+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2180152594 shutdown_connections 2024-03-19T01:18:20.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.214+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x562ec3df20f0 0x562ec3df4510 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.214+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562ec3d7b110 0x562ec3d7edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.214+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562ec3d72530 0x562ec3d7e890 secure :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0x562ec3d8e110 tx=0x7fdd48004730 comp rx=0 tx=0).stop 2024-03-19T01:18:20.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.214+0000 7fdd4e000640 0 client.27292 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:18:20.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.214+0000 7fdd69bc4740 1 --2- 172.21.15.69:0/2180152594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ec3d71c10 0x562ec3d83520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.214+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2180152594 >> 172.21.15.69:0/2180152594 conn(0x562ec3d6c340 msgr2=0x562ec3d6e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:20.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.214+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2180152594 shutdown_connections 2024-03-19T01:18:20.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.215+0000 7fdd69bc4740 1 -- 172.21.15.69:0/2180152594 wait complete. 2024-03-19T01:18:20.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.215+0000 7fdd69bc4740 20 client.27292 trim_cache size 0 max 16384 2024-03-19T01:18:20.218 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/amd/core.o 2024-03-19T01:18:20.219 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:18:20.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.231+0000 7fdd65800640 1 -- 172.21.15.69:0/3840797531 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdd440a29f0 msgr2=0x7fdd440a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:20.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.231+0000 7fdd65800640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdd440a29f0 0x7fdd440a4eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd6010afc0 tx=0x7fdd48005f90 comp rx=0 tx=0).stop 2024-03-19T01:18:20.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.231+0000 7fdd65800640 1 -- 172.21.15.69:0/3840797531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60108df0 msgr2=0x7fdd601a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:20.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.231+0000 7fdd65800640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60108df0 0x7fdd601a6d20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000d950 tx=0x7fdd5000de10 comp rx=0 tx=0).stop 2024-03-19T01:18:20.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.231+0000 7fdd65800640 1 -- 172.21.15.69:0/3840797531 shutdown_connections 2024-03-19T01:18:20.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.231+0000 7fdd65800640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd601ab5e0 0x7fdd601aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.231+0000 7fdd65800640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdd440a29f0 0x7fdd440a4eb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.231+0000 7fdd65800640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd6010be00 0x7fdd601a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.231+0000 7fdd65800640 1 --2- 172.21.15.69:0/3840797531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60108df0 0x7fdd601a6d20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:20.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.231+0000 7fdd65800640 1 -- 172.21.15.69:0/3840797531 >> 172.21.15.69:0/3840797531 conn(0x7fdd6006c770 msgr2=0x7fdd601098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:20.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.232+0000 7fdd65800640 1 -- 172.21.15.69:0/3840797531 shutdown_connections 2024-03-19T01:18:20.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:20.232+0000 7fdd65800640 1 -- 172.21.15.69:0/3840797531 wait complete. 2024-03-19T01:18:20.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:19 smithi086 ceph-mon[66829]: pgmap v236: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 640 KiB/s wr, 152 op/s 2024-03-19T01:18:20.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:18:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:19 smithi163 ceph-mon[67428]: pgmap v236: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 640 KiB/s wr, 152 op/s 2024-03-19T01:18:20.313 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/x509.asn1.o 2024-03-19T01:18:20.326 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/iomap/buffered-io.o 2024-03-19T01:18:20.355 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/entry/entry_64_compat.o 2024-03-19T01:18:20.380 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/request_key_auth.o 2024-03-19T01:18:20.482 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/x509_akid.asn1.o 2024-03-19T01:18:20.527 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/locking/rwsem.o 2024-03-19T01:18:20.563 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/entry/syscall_32.o 2024-03-19T01:18:20.706 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/x509_cert_parser.o 2024-03-19T01:18:21.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.134+0000 7f0dc5400640 1 -- 172.21.15.69:0/2977615161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dc0110040 msgr2=0x7f0dc010db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.134+0000 7f0dc5400640 1 --2- 172.21.15.69:0/2977615161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dc0110040 0x7f0dc010db30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0db801acb0 tx=0x7f0db803d130 comp rx=0 tx=0).stop 2024-03-19T01:18:21.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.135+0000 7f0dc5400640 1 -- 172.21.15.69:0/2977615161 shutdown_connections 2024-03-19T01:18:21.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.135+0000 7f0dc5400640 1 --2- 172.21.15.69:0/2977615161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dc011b000 0x7f0dc011d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.135+0000 7f0dc5400640 1 --2- 172.21.15.69:0/2977615161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dc010e070 0x7f0dc010e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.135+0000 7f0dc5400640 1 --2- 172.21.15.69:0/2977615161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dc0110040 0x7f0dc010db30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.135+0000 7f0dc5400640 1 -- 172.21.15.69:0/2977615161 >> 172.21.15.69:0/2977615161 conn(0x7f0dc006c900 msgr2=0x7f0dc006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.137+0000 7f0dc5400640 1 -- 172.21.15.69:0/2977615161 shutdown_connections 2024-03-19T01:18:21.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.137+0000 7f0dc5400640 1 -- 172.21.15.69:0/2977615161 wait complete. 2024-03-19T01:18:21.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.143+0000 7f0dc5400640 1 Processor -- start 2024-03-19T01:18:21.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.143+0000 7f0dc5400640 1 -- start start 2024-03-19T01:18:21.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.143+0000 7f0dc5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dc010e070 0x7f0dc011a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.143+0000 7f0dc5400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dc0110040 0x7f0dc01166e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.143+0000 7f0dc5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dc011b000 0x7f0dc0116c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.143+0000 7f0dc5400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc0113170 con 0x7f0dc0110040 2024-03-19T01:18:21.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.143+0000 7f0dc5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc0112ff0 con 0x7f0dc010e070 2024-03-19T01:18:21.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.143+0000 7f0dc5400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01132f0 con 0x7f0dc011b000 2024-03-19T01:18:21.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.144+0000 7f0dbde00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dc0110040 0x7f0dc01166e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.144+0000 7f0dbde00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dc0110040 0x7f0dc01166e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41458/0 (socket says 172.21.15.69:41458) 2024-03-19T01:18:21.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.144+0000 7f0dbde00640 1 -- 172.21.15.69:0/1216740204 learned_addr learned my addr 172.21.15.69:0/1216740204 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:21.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.144+0000 7f0dbf200640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dc011b000 0x7f0dc0116c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.144+0000 7f0dbe800640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dc010e070 0x7f0dc011a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.145+0000 7f0dbf200640 1 -- 172.21.15.69:0/1216740204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dc010e070 msgr2=0x7f0dc011a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.145+0000 7f0dbf200640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dc010e070 0x7f0dc011a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.145+0000 7f0dbf200640 1 -- 172.21.15.69:0/1216740204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dc0110040 msgr2=0x7f0dc01166e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.145+0000 7f0dbf200640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dc0110040 0x7f0dc01166e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.145+0000 7f0dbf200640 1 -- 172.21.15.69:0/1216740204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc0117210 con 0x7f0dc011b000 2024-03-19T01:18:21.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.145+0000 7f0dbe800640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dc010e070 0x7f0dc011a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:21.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.145+0000 7f0dbf200640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dc011b000 0x7f0dc0116c20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00d6e0 tx=0x7f0dac00dba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.145+0000 7f0db7400640 1 -- 172.21.15.69:0/1216740204 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac004230 con 0x7f0dc011b000 2024-03-19T01:18:21.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.145+0000 7f0db7400640 1 -- 172.21.15.69:0/1216740204 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dac0043d0 con 0x7f0dc011b000 2024-03-19T01:18:21.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.145+0000 7f0db7400640 1 -- 172.21.15.69:0/1216740204 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac010460 con 0x7f0dc011b000 2024-03-19T01:18:21.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.146+0000 7f0dc5400640 1 -- 172.21.15.69:0/1216740204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc0117500 con 0x7f0dc011b000 2024-03-19T01:18:21.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.146+0000 7f0dc5400640 1 -- 172.21.15.69:0/1216740204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc01ab0a0 con 0x7f0dc011b000 2024-03-19T01:18:21.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.153+0000 7f0dc8185740 1 Processor -- start 2024-03-19T01:18:21.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.159+0000 7f0db7400640 1 -- 172.21.15.69:0/1216740204 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f0dac009820 con 0x7f0dc011b000 2024-03-19T01:18:21.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.161+0000 7f0db7400640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0da40a29f0 0x7f0da40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.161+0000 7f0db7400640 1 -- 172.21.15.69:0/1216740204 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f0dac0cec00 con 0x7f0dc011b000 2024-03-19T01:18:21.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.162+0000 7f0dbde00640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0da40a29f0 0x7f0da40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.163+0000 7f0dbde00640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0da40a29f0 0x7f0da40a4eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0117ea0 tx=0x7f0db0005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.163+0000 7f0dc8185740 1 -- start start 2024-03-19T01:18:21.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.163+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff1fa20 0x560b1ff1fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.163+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff20340 0x560b1ff289e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.163+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff28f20 0x560b1ff2b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.163+0000 7f0dc8185740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560b1fe84380 con 0x560b1ff1fa20 2024-03-19T01:18:21.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.163+0000 7f0dc8185740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b1fe2cd20 con 0x560b1ff28f20 2024-03-19T01:18:21.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.163+0000 7f0dc8185740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560b1fe80420 con 0x560b1ff20340 2024-03-19T01:18:21.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.164+0000 7f0dbe800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff1fa20 0x560b1ff1fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.164+0000 7f0dbe800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff1fa20 0x560b1ff1fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41466/0 (socket says 172.21.15.69:41466) 2024-03-19T01:18:21.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.164+0000 7f0dbe800640 1 -- 172.21.15.69:0/2317000889 learned_addr learned my addr 172.21.15.69:0/2317000889 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:21.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.165+0000 7f0dbe800640 1 -- 172.21.15.69:0/2317000889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff20340 msgr2=0x560b1ff289e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:21.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.165+0000 7f0dbe800640 1 --2- 172.21.15.69:0/2317000889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff20340 0x560b1ff289e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.165+0000 7f0dbe800640 1 -- 172.21.15.69:0/2317000889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff28f20 msgr2=0x560b1ff2b310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:21.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.165+0000 7f0dbe800640 1 --2- 172.21.15.69:0/2317000889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff28f20 0x560b1ff2b310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.165+0000 7f0dbe800640 1 -- 172.21.15.69:0/2317000889 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b1fb983f0 con 0x560b1ff1fa20 2024-03-19T01:18:21.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.165+0000 7f0dbe800640 1 --2- 172.21.15.69:0/2317000889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff1fa20 0x560b1ff1fe00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0db801acb0 tx=0x7f0db8005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e76810832496954 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.166+0000 7f0dab400640 1 -- 172.21.15.69:0/2317000889 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8054030 con 0x560b1ff1fa20 2024-03-19T01:18:21.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.167+0000 7f0dab400640 1 -- 172.21.15.69:0/2317000889 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db801a810 con 0x560b1ff1fa20 2024-03-19T01:18:21.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.167+0000 7f0dab400640 1 -- 172.21.15.69:0/2317000889 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db804a070 con 0x560b1ff1fa20 2024-03-19T01:18:21.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.168+0000 7f0dc8185740 1 -- 172.21.15.69:0/2317000889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff1fa20 msgr2=0x560b1ff1fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.168+0000 7f0dc8185740 1 --2- 172.21.15.69:0/2317000889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff1fa20 0x560b1ff1fe00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0db801acb0 tx=0x7f0db8005f70 comp rx=0 tx=0).stop 2024-03-19T01:18:21.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.171+0000 7f0dc8185740 1 -- 172.21.15.69:0/2317000889 shutdown_connections 2024-03-19T01:18:21.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.171+0000 7f0dc8185740 1 --2- 172.21.15.69:0/2317000889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff28f20 0x560b1ff2b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.171+0000 7f0dc8185740 1 --2- 172.21.15.69:0/2317000889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff20340 0x560b1ff289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.171+0000 7f0dc8185740 1 --2- 172.21.15.69:0/2317000889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff1fa20 0x560b1ff1fe00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.171+0000 7f0dc8185740 1 -- 172.21.15.69:0/2317000889 >> 172.21.15.69:0/2317000889 conn(0x560b1ff1a150 msgr2=0x560b1ff1a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.175+0000 7f0dc8185740 1 -- 172.21.15.69:0/2317000889 shutdown_connections 2024-03-19T01:18:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.175+0000 7f0dc8185740 1 -- 172.21.15.69:0/2317000889 wait complete. 2024-03-19T01:18:21.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.175+0000 7f0dc8185740 1 -- start start 2024-03-19T01:18:21.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.176+0000 7f0dc8185740 1 Processor -- start 2024-03-19T01:18:21.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.179+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 0x560b1ff25ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.179+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 0x560b1ff26530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.179+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff26a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.179+0000 7f0dc8185740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560b1ff2bcb0 con 0x560b1ff20340 2024-03-19T01:18:21.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.179+0000 7f0dc8185740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b1fe40160 con 0x560b1ff1fa20 2024-03-19T01:18:21.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.179+0000 7f0dc8185740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560b1ff2be30 con 0x560b1ff28f20 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbf200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff26a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbf200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff26a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48504/0 (socket says 172.21.15.69:48504) 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbf200640 1 -- 172.21.15.69:0/4050065966 learned_addr learned my addr 172.21.15.69:0/4050065966 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dc8185740 10 client.0 _ll_register_callbacks cb 0x560b1fe7c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbde00640 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 0x560b1ff26530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbe800640 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 0x560b1ff25ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbf200640 1 -- 172.21.15.69:0/4050065966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 msgr2=0x560b1ff25ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbf200640 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 0x560b1ff25ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbf200640 1 -- 172.21.15.69:0/4050065966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 msgr2=0x560b1ff26530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbf200640 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 0x560b1ff26530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbf200640 1 -- 172.21.15.69:0/4050065966 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1fc627d0 con 0x560b1ff28f20 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbde00640 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 0x560b1ff26530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0dbf200640 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff26a70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0cbfa0 tx=0x7f0dac09a920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f3b067abb61323e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.180+0000 7f0d9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:18:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.181+0000 7f0d9fe00640 1 -- 172.21.15.69:0/4050065966 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac020040 con 0x560b1ff28f20 2024-03-19T01:18:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.181+0000 7f0d9fe00640 1 -- 172.21.15.69:0/4050065966 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f0dac097570 con 0x560b1ff28f20 2024-03-19T01:18:21.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.184+0000 7f0dc8185740 1 -- 172.21.15.69:0/4050065966 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db801a810 con 0x560b1ff28f20 2024-03-19T01:18:21.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.184+0000 7f0d9fe00640 1 -- 172.21.15.69:0/4050065966 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0dac0b8050 con 0x560b1ff28f20 2024-03-19T01:18:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.188+0000 7f0dc8185740 10 client.27301 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.188+0000 7f0dc8185740 1 -- 172.21.15.69:0/4050065966 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560b1ff38040 con 0x560b1ff28f20 2024-03-19T01:18:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.188+0000 7f0d9fe00640 1 -- 172.21.15.69:0/4050065966 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db801a810 con 0x560b1ff28f20 2024-03-19T01:18:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.188+0000 7f0d9fe00640 1 -- 172.21.15.69:0/4050065966 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f0dac09a350 con 0x560b1ff28f20 2024-03-19T01:18:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.188+0000 7f0dc8185740 10 client.27301 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.188+0000 7f0dc8185740 10 client.27301 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:18:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.188+0000 7f0dc8185740 20 client.27301 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.188+0000 7f0dc8185740 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b1ff39550 0x560b1ff3b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.188+0000 7f0dc8185740 4 client.27301 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.188+0000 7f0dc8185740 1 -- 172.21.15.69:0/4050065966 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b1fe918c0 con 0x560b1ff39550 2024-03-19T01:18:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.189+0000 7f0dbe800640 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b1ff39550 0x560b1ff3b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.190+0000 7f0dbe800640 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b1ff39550 0x560b1ff3b940 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e22c595e12b3cb3 server_cookie=a5cfe75d51802f27 in_seq=0 out_seq=0 2024-03-19T01:18:21.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.190+0000 7f0d9fe00640 10 client.27301 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:18:21.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0d9fe00640 1 -- 172.21.15.69:0/4050065966 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560b1fe918c0 con 0x560b1ff39550 2024-03-19T01:18:21.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0d9fe00640 10 client.27301 handle_command_reply: tid=0 2024-03-19T01:18:21.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0dc8185740 1 client.27301 shutdown 2024-03-19T01:18:21.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0dc8185740 1 -- 172.21.15.69:0/4050065966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 msgr2=0x560b1ff26a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0dc8185740 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff26a70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0cbfa0 tx=0x7f0dac09a920 comp rx=0 tx=0).stop 2024-03-19T01:18:21.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0dc8185740 1 -- 172.21.15.69:0/4050065966 shutdown_connections 2024-03-19T01:18:21.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0dc8185740 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff26a70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0dc8185740 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 0x560b1ff26530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0dc8185740 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b1ff39550 0x560b1ff3b940 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0dc8185740 1 --2- 172.21.15.69:0/4050065966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 0x560b1ff25ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0d9fe00640 0 client.27301 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:18:21.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.194+0000 7f0dc8185740 1 -- 172.21.15.69:0/4050065966 >> 172.21.15.69:0/4050065966 conn(0x560b1ff1a150 msgr2=0x560b1ff1bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.195+0000 7f0dc8185740 1 -- 172.21.15.69:0/4050065966 shutdown_connections 2024-03-19T01:18:21.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.195+0000 7f0dc8185740 1 -- 172.21.15.69:0/4050065966 wait complete. 2024-03-19T01:18:21.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.195+0000 7f0dc8185740 20 client.27301 trim_cache size 0 max 16384 2024-03-19T01:18:21.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.248+0000 7f0dc8185740 1 Processor -- start 2024-03-19T01:18:21.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.248+0000 7f0dc8185740 1 -- start start 2024-03-19T01:18:21.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.248+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff1fa20 0x560b1ff28720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.248+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff20340 0x560b1ff34a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.248+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff28f20 0x560b1ff1bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.248+0000 7f0dc8185740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b5390 con 0x560b1ff28f20 2024-03-19T01:18:21.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.248+0000 7f0dc8185740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b7050 con 0x560b1ff20340 2024-03-19T01:18:21.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.248+0000 7f0dc8185740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b5a60 con 0x560b1ff1fa20 2024-03-19T01:18:21.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.249+0000 7f0dbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff20340 0x560b1ff34a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.249+0000 7f0dbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff20340 0x560b1ff34a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35076/0 (socket says 172.21.15.69:35076) 2024-03-19T01:18:21.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.249+0000 7f0dbe800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff1fa20 0x560b1ff28720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.249+0000 7f0dbde00640 1 -- 172.21.15.69:0/3900879689 learned_addr learned my addr 172.21.15.69:0/3900879689 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:21.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.249+0000 7f0dbf200640 1 --2- 172.21.15.69:0/3900879689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff28f20 0x560b1ff1bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.249+0000 7f0dbde00640 1 -- 172.21.15.69:0/3900879689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff1fa20 msgr2=0x560b1ff28720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.249+0000 7f0dbde00640 1 --2- 172.21.15.69:0/3900879689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff1fa20 0x560b1ff28720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.249+0000 7f0dbde00640 1 -- 172.21.15.69:0/3900879689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff28f20 msgr2=0x560b1ff1bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.249+0000 7f0dbde00640 1 --2- 172.21.15.69:0/3900879689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff28f20 0x560b1ff1bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.249+0000 7f0dbde00640 1 -- 172.21.15.69:0/3900879689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b1ff2b680 con 0x560b1ff20340 2024-03-19T01:18:21.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.249+0000 7f0dbf200640 1 --2- 172.21.15.69:0/3900879689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff28f20 0x560b1ff1bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:21.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.250+0000 7f0dbde00640 1 --2- 172.21.15.69:0/3900879689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff20340 0x560b1ff34a50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009460 tx=0x7f0db0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e74b9f780e7a4013 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.254+0000 7f0daaa00640 1 -- 172.21.15.69:0/3900879689 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db002b070 con 0x560b1ff20340 2024-03-19T01:18:21.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.254+0000 7f0daaa00640 1 -- 172.21.15.69:0/3900879689 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db00263f0 con 0x560b1ff20340 2024-03-19T01:18:21.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.254+0000 7f0daaa00640 1 -- 172.21.15.69:0/3900879689 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db001c670 con 0x560b1ff20340 2024-03-19T01:18:21.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.255+0000 7f0dc8185740 1 -- 172.21.15.69:0/3900879689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff20340 msgr2=0x560b1ff34a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.255+0000 7f0dc8185740 1 --2- 172.21.15.69:0/3900879689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff20340 0x560b1ff34a50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009460 tx=0x7f0db0014000 comp rx=0 tx=0).stop 2024-03-19T01:18:21.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.258+0000 7f0dc8185740 1 -- 172.21.15.69:0/3900879689 shutdown_connections 2024-03-19T01:18:21.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.258+0000 7f0dc8185740 1 --2- 172.21.15.69:0/3900879689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff28f20 0x560b1ff1bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.258+0000 7f0dc8185740 1 --2- 172.21.15.69:0/3900879689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff20340 0x560b1ff34a50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.258+0000 7f0dc8185740 1 --2- 172.21.15.69:0/3900879689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff1fa20 0x560b1ff28720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.258+0000 7f0dc8185740 1 -- 172.21.15.69:0/3900879689 >> 172.21.15.69:0/3900879689 conn(0x560b1ff1a150 msgr2=0x560b1ff1b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.258+0000 7f0dc8185740 1 -- 172.21.15.69:0/3900879689 shutdown_connections 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.259+0000 7f0dc8185740 1 -- 172.21.15.69:0/3900879689 wait complete. 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.259+0000 7f0dc8185740 1 -- start start 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.259+0000 7f0dc8185740 1 Processor -- start 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.262+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 0x560b1ff2cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.262+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 0x560b1ff2d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.262+0000 7f0dc8185740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff2d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dbe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 0x560b1ff2cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dbe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 0x560b1ff2cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35086/0 (socket says 172.21.15.69:35086) 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dbf200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff2d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dbde00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 0x560b1ff2d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dbf200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff2d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48524/0 (socket says 172.21.15.69:48524) 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dbe800640 1 -- 172.21.15.69:0/442517863 learned_addr learned my addr 172.21.15.69:0/442517863 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dc8185740 1 -- 172.21.15.69:0/442517863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b68c0 con 0x560b1ff20340 2024-03-19T01:18:21.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dc8185740 1 -- 172.21.15.69:0/442517863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b61f0 con 0x560b1ff1fa20 2024-03-19T01:18:21.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dc8185740 1 -- 172.21.15.69:0/442517863 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b3310 con 0x560b1ff28f20 2024-03-19T01:18:21.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dbf200640 1 -- 172.21.15.69:0/442517863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 msgr2=0x560b1ff2cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dbf200640 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 0x560b1ff2cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.263+0000 7f0dbf200640 1 -- 172.21.15.69:0/442517863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 msgr2=0x560b1ff2d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.264+0000 7f0dbf200640 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 0x560b1ff2d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.264+0000 7f0dc8185740 10 client.0 _ll_register_callbacks cb 0x7f0dac09ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:21.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.264+0000 7f0dbf200640 1 -- 172.21.15.69:0/442517863 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1ffc0a50 con 0x560b1ff28f20 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.264+0000 7f0dbde00640 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 0x560b1ff2d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.264+0000 7f0dbe800640 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 0x560b1ff2cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.264+0000 7f0dc8185740 1 -- 172.21.15.69:0/442517863 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db00263f0 con 0x560b1ff28f20 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.264+0000 7f0dbf200640 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff2d5c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0dac09a6f0 tx=0x7f0dac003980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d95dd105d717ed30 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.264+0000 7f0d9fe00640 10 client.27307 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.265+0000 7f0d9fe00640 1 -- 172.21.15.69:0/442517863 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac014070 con 0x560b1ff28f20 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.265+0000 7f0d9fe00640 1 -- 172.21.15.69:0/442517863 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f0dac0840c0 con 0x560b1ff28f20 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.265+0000 7f0d9fe00640 1 -- 172.21.15.69:0/442517863 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0dac020070 con 0x560b1ff28f20 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.265+0000 7f0dc8185740 10 client.27307 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.265+0000 7f0dc8185740 1 -- 172.21.15.69:0/442517863 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=47}) -- 0x560b1ff97800 con 0x560b1ff28f20 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.266+0000 7f0d9fe00640 1 -- 172.21.15.69:0/442517863 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f0dac009b10 con 0x560b1ff28f20 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.266+0000 7f0dc8185740 10 client.27307 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.266+0000 7f0dc8185740 10 client.27307 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:18:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.266+0000 7f0dc8185740 20 client.27307 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:21.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.266+0000 7f0dc8185740 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b1ff97c80 0x560b1ff9a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.266+0000 7f0dc8185740 4 client.27307 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:18:21.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.266+0000 7f0dc8185740 1 -- 172.21.15.69:0/442517863 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b1fe8bc10 con 0x560b1ff97c80 2024-03-19T01:18:21.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.267+0000 7f0dbe800640 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b1ff97c80 0x560b1ff9a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.268+0000 7f0dbe800640 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b1ff97c80 0x560b1ff9a140 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2274eb11ca51cb0 server_cookie=5129e79fe7d565 in_seq=0 out_seq=0 2024-03-19T01:18:21.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.268+0000 7f0d9fe00640 10 client.27307 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:18:21.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.268+0000 7f0d9fe00640 1 -- 172.21.15.69:0/442517863 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b1fe8bc10 con 0x560b1ff97c80 2024-03-19T01:18:21.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.268+0000 7f0d9fe00640 10 client.27307 handle_command_reply: tid=0 2024-03-19T01:18:21.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.268+0000 7f0dc8185740 1 client.27307 shutdown 2024-03-19T01:18:21.273 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/user_defined.o 2024-03-19T01:18:21.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.279+0000 7f0dc8185740 1 -- 172.21.15.69:0/442517863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 msgr2=0x560b1ff2d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.279+0000 7f0dc8185740 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff2d5c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0dac09a6f0 tx=0x7f0dac003980 comp rx=0 tx=0).stop 2024-03-19T01:18:21.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.279+0000 7f0dc8185740 1 -- 172.21.15.69:0/442517863 shutdown_connections 2024-03-19T01:18:21.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.279+0000 7f0dc8185740 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b1ff28f20 0x560b1ff2d5c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.279+0000 7f0dc8185740 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b1ff20340 0x560b1ff2d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.279+0000 7f0dc8185740 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x560b1ff97c80 0x560b1ff9a140 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.279+0000 7f0dc8185740 1 --2- 172.21.15.69:0/442517863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b1ff1fa20 0x560b1ff2cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.279+0000 7f0dc8185740 1 -- 172.21.15.69:0/442517863 >> 172.21.15.69:0/442517863 conn(0x560b1ff1a150 msgr2=0x560b1ff1be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.279+0000 7f0d9fe00640 0 client.27307 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:18:21.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.281+0000 7f0dc8185740 1 -- 172.21.15.69:0/442517863 shutdown_connections 2024-03-19T01:18:21.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.281+0000 7f0dc8185740 1 -- 172.21.15.69:0/442517863 wait complete. 2024-03-19T01:18:21.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.281+0000 7f0dc8185740 20 client.27307 trim_cache size 0 max 16384 2024-03-19T01:18:21.284 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:18:21.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.289+0000 7f0dc5400640 1 -- 172.21.15.69:0/1216740204 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0da40a29f0 msgr2=0x7f0da40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.289+0000 7f0dc5400640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0da40a29f0 0x7f0da40a4eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0117ea0 tx=0x7f0db0005f90 comp rx=0 tx=0).stop 2024-03-19T01:18:21.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.289+0000 7f0dc5400640 1 -- 172.21.15.69:0/1216740204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dc011b000 msgr2=0x7f0dc0116c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.289+0000 7f0dc5400640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dc011b000 0x7f0dc0116c20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00d6e0 tx=0x7f0dac00dba0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.289+0000 7f0dc5400640 1 -- 172.21.15.69:0/1216740204 shutdown_connections 2024-03-19T01:18:21.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.289+0000 7f0dc5400640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dc011b000 0x7f0dc0116c20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.289+0000 7f0dc5400640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dc0110040 0x7f0dc01166e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.289+0000 7f0dc5400640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0da40a29f0 0x7f0da40a4eb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.289+0000 7f0dc5400640 1 --2- 172.21.15.69:0/1216740204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dc010e070 0x7f0dc011a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.289+0000 7f0dc5400640 1 -- 172.21.15.69:0/1216740204 >> 172.21.15.69:0/1216740204 conn(0x7f0dc006c900 msgr2=0x7f0dc011c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.293+0000 7f0dc5400640 1 -- 172.21.15.69:0/1216740204 shutdown_connections 2024-03-19T01:18:21.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.295+0000 7f0dc5400640 1 -- 172.21.15.69:0/1216740204 wait complete. 2024-03-19T01:18:21.345 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:18:21.412 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/truncate.o 2024-03-19T01:18:21.576 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/amd/lbr.o 2024-03-19T01:18:21.577 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/pkcs7.asn1.o 2024-03-19T01:18:21.679 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/asymmetric_keys/pkcs7_parser.o 2024-03-19T01:18:21.684 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/locking/percpu-rwsem.o 2024-03-19T01:18:21.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.864+0000 7f0236e00640 1 -- 172.21.15.69:0/970558801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023010bf40 msgr2=0x7f0230108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.864+0000 7f0236e00640 1 --2- 172.21.15.69:0/970558801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023010bf40 0x7f0230108820 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0220009900 tx=0x7f022002bce0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.865+0000 7f0236e00640 1 -- 172.21.15.69:0/970558801 shutdown_connections 2024-03-19T01:18:21.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.865+0000 7f0236e00640 1 --2- 172.21.15.69:0/970558801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023011b010 0x7f023011d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.865+0000 7f0236e00640 1 --2- 172.21.15.69:0/970558801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0230108d60 0x7f02301091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.865+0000 7f0236e00640 1 --2- 172.21.15.69:0/970558801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023010bf40 0x7f0230108820 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.865+0000 7f0236e00640 1 -- 172.21.15.69:0/970558801 >> 172.21.15.69:0/970558801 conn(0x7f023006cd60 msgr2=0x7f023006d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.865+0000 7f0236e00640 1 -- 172.21.15.69:0/970558801 shutdown_connections 2024-03-19T01:18:21.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.865+0000 7f0236e00640 1 -- 172.21.15.69:0/970558801 wait complete. 2024-03-19T01:18:21.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.866+0000 7f0236e00640 1 Processor -- start 2024-03-19T01:18:21.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.867+0000 7f0236e00640 1 -- start start 2024-03-19T01:18:21.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.867+0000 7f0236e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d60 0x7f023011a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.868+0000 7f0236e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023010bf40 0x7f0230116830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.868+0000 7f0236e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023011b010 0x7f0230116d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.868+0000 7f022f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023010bf40 0x7f0230116830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.868+0000 7f022f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023010bf40 0x7f0230116830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48548/0 (socket says 172.21.15.69:48548) 2024-03-19T01:18:21.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.868+0000 7f022fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d60 0x7f023011a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.868+0000 7f0234e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023011b010 0x7f0230116d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.868+0000 7f0234e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023011b010 0x7f0230116d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41498/0 (socket says 172.21.15.69:41498) 2024-03-19T01:18:21.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.869+0000 7f022fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d60 0x7f023011a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35110/0 (socket says 172.21.15.69:35110) 2024-03-19T01:18:21.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.869+0000 7f0236e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02301151e0 con 0x7f023011b010 2024-03-19T01:18:21.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.869+0000 7f022f400640 1 -- 172.21.15.69:0/2635330939 learned_addr learned my addr 172.21.15.69:0/2635330939 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:21.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.869+0000 7f0236e00640 1 -- 172.21.15.69:0/2635330939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0230115060 con 0x7f0230108d60 2024-03-19T01:18:21.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.869+0000 7f0236e00640 1 -- 172.21.15.69:0/2635330939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0230115360 con 0x7f023010bf40 2024-03-19T01:18:21.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.870+0000 7f022fe00640 1 -- 172.21.15.69:0/2635330939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023010bf40 msgr2=0x7f0230116830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.870+0000 7f022fe00640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023010bf40 0x7f0230116830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.870+0000 7f022fe00640 1 -- 172.21.15.69:0/2635330939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023011b010 msgr2=0x7f0230116d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.870+0000 7f022fe00640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023011b010 0x7f0230116d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.870+0000 7f022fe00640 1 -- 172.21.15.69:0/2635330939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02301174a0 con 0x7f0230108d60 2024-03-19T01:18:21.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.870+0000 7f022f400640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023010bf40 0x7f0230116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:21.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.870+0000 7f0234e00640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023011b010 0x7f0230116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:21.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.871+0000 7f022fe00640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d60 0x7f023011a750 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0220009230 tx=0x7f02200043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.871+0000 7f022cc00640 1 -- 172.21.15.69:0/2635330939 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220042020 con 0x7f0230108d60 2024-03-19T01:18:21.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.871+0000 7f022cc00640 1 -- 172.21.15.69:0/2635330939 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02200044a0 con 0x7f0230108d60 2024-03-19T01:18:21.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.871+0000 7f022cc00640 1 -- 172.21.15.69:0/2635330939 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220003a10 con 0x7f0230108d60 2024-03-19T01:18:21.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.871+0000 7f0236e00640 1 -- 172.21.15.69:0/2635330939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02301aad60 con 0x7f0230108d60 2024-03-19T01:18:21.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.871+0000 7f0236e00640 1 -- 172.21.15.69:0/2635330939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02301ab110 con 0x7f0230108d60 2024-03-19T01:18:21.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.876+0000 7f0239baf740 1 Processor -- start 2024-03-19T01:18:21.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.876+0000 7f0239baf740 1 -- start start 2024-03-19T01:18:21.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.877+0000 7f0239baf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cac6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.877+0000 7f0239baf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac7340 0x55a21cacf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.877+0000 7f0239baf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cad2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.877+0000 7f0239baf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a21ca2b380 con 0x55a21cac7340 2024-03-19T01:18:21.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.877+0000 7f0239baf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a21c9d3d20 con 0x55a21cacff20 2024-03-19T01:18:21.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.877+0000 7f0239baf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a21ca27420 con 0x55a21cac6a20 2024-03-19T01:18:21.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.877+0000 7f022f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cad2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.877+0000 7f022f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cad2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35122/0 (socket says 172.21.15.69:35122) 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.877+0000 7f022f400640 1 -- 172.21.15.69:0/2006540797 learned_addr learned my addr 172.21.15.69:0/2006540797 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.877+0000 7f0234e00640 1 --2- 172.21.15.69:0/2006540797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cac6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.878+0000 7f022f400640 1 -- 172.21.15.69:0/2006540797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 msgr2=0x55a21cac6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.878+0000 7f022f400640 1 --2- 172.21.15.69:0/2006540797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cac6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.878+0000 7f022f400640 1 -- 172.21.15.69:0/2006540797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac7340 msgr2=0x55a21cacf9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.878+0000 7f022f400640 1 --2- 172.21.15.69:0/2006540797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac7340 0x55a21cacf9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.878+0000 7f022f400640 1 -- 172.21.15.69:0/2006540797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a21c73f3f0 con 0x55a21cacff20 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.878+0000 7f0234e00640 1 --2- 172.21.15.69:0/2006540797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cac6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.878+0000 7f022f400640 1 --2- 172.21.15.69:0/2006540797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cad2310 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f021c0225b0 tx=0x7f021c022a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a660adb39483a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.879+0000 7f0225600640 1 -- 172.21.15.69:0/2006540797 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c020940 con 0x55a21cacff20 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.879+0000 7f0225600640 1 -- 172.21.15.69:0/2006540797 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f021c005020 con 0x55a21cacff20 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.879+0000 7f0225600640 1 -- 172.21.15.69:0/2006540797 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c020940 con 0x55a21cacff20 2024-03-19T01:18:21.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.879+0000 7f022cc00640 1 -- 172.21.15.69:0/2635330939 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f0220045020 con 0x7f0230108d60 2024-03-19T01:18:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.880+0000 7f0239baf740 1 -- 172.21.15.69:0/2006540797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 msgr2=0x55a21cad2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.880+0000 7f0239baf740 1 --2- 172.21.15.69:0/2006540797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cad2310 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f021c0225b0 tx=0x7f021c022a70 comp rx=0 tx=0).stop 2024-03-19T01:18:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.880+0000 7f0239baf740 1 -- 172.21.15.69:0/2006540797 shutdown_connections 2024-03-19T01:18:21.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.880+0000 7f0239baf740 1 --2- 172.21.15.69:0/2006540797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cad2310 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.880+0000 7f0239baf740 1 --2- 172.21.15.69:0/2006540797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac7340 0x55a21cacf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.880+0000 7f0239baf740 1 --2- 172.21.15.69:0/2006540797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cac6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.880+0000 7f0239baf740 1 -- 172.21.15.69:0/2006540797 >> 172.21.15.69:0/2006540797 conn(0x55a21cac1150 msgr2=0x55a21cac1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.881+0000 7f022cc00640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02140a2950 0x7f02140a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.881+0000 7f022cc00640 1 -- 172.21.15.69:0/2635330939 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f02200f0c10 con 0x7f0230108d60 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.882+0000 7f022f400640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02140a2950 0x7f02140a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.883+0000 7f022f400640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02140a2950 0x7f02140a4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f021c005e00 tx=0x7f021c01a8b0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.884+0000 7f0239baf740 1 -- 172.21.15.69:0/2006540797 shutdown_connections 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.884+0000 7f0239baf740 1 -- 172.21.15.69:0/2006540797 wait complete. 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.884+0000 7f0239baf740 1 -- start start 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.885+0000 7f0239baf740 1 Processor -- start 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.885+0000 7f0239baf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac6a20 0x55a21caccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.885+0000 7f0239baf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cac7340 0x55a21cacd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.885+0000 7f0239baf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cacff20 0x55a21cacda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.886+0000 7f022fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac6a20 0x55a21caccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.886+0000 7f022f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cac7340 0x55a21cacd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.886+0000 7f022fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac6a20 0x55a21caccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41500/0 (socket says 172.21.15.69:41500) 2024-03-19T01:18:21.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.886+0000 7f022f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cac7340 0x55a21cacd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35138/0 (socket says 172.21.15.69:35138) 2024-03-19T01:18:21.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.886+0000 7f0234e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cacff20 0x55a21cacda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.886+0000 7f0234e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cacff20 0x55a21cacda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48556/0 (socket says 172.21.15.69:48556) 2024-03-19T01:18:21.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.886+0000 7f022fe00640 1 -- 172.21.15.69:0/2464984370 learned_addr learned my addr 172.21.15.69:0/2464984370 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:21.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.886+0000 7f0239baf740 1 -- 172.21.15.69:0/2464984370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a21cad2d10 con 0x55a21cac6a20 2024-03-19T01:18:21.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.886+0000 7f0239baf740 1 -- 172.21.15.69:0/2464984370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a21c9e7160 con 0x55a21cac7340 2024-03-19T01:18:21.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.886+0000 7f0239baf740 1 -- 172.21.15.69:0/2464984370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a21cad2e90 con 0x55a21cacff20 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.886+0000 7f0239baf740 10 client.0 _ll_register_callbacks cb 0x55a21ca23170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.887+0000 7f022fe00640 1 -- 172.21.15.69:0/2464984370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cacff20 msgr2=0x55a21cacda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.887+0000 7f022fe00640 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cacff20 0x55a21cacda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.887+0000 7f022fe00640 1 -- 172.21.15.69:0/2464984370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cac7340 msgr2=0x55a21cacd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.887+0000 7f022fe00640 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cac7340 0x55a21cacd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.887+0000 7f022fe00640 1 -- 172.21.15.69:0/2464984370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a21c8097d0 con 0x55a21cac6a20 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.887+0000 7f022f400640 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cac7340 0x55a21cacd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.888+0000 7f022fe00640 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac6a20 0x55a21caccfa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f02200bcec0 tx=0x7f02200bcb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6824ea47fe9043e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.888+0000 7f021a000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.888+0000 7f021a000640 1 -- 172.21.15.69:0/2464984370 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200bcf30 con 0x55a21cac6a20 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.888+0000 7f021a000640 1 -- 172.21.15.69:0/2464984370 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f02200c5240 con 0x55a21cac6a20 2024-03-19T01:18:21.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.888+0000 7f0239baf740 1 -- 172.21.15.69:0/2464984370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f021c005020 con 0x55a21cac6a20 2024-03-19T01:18:21.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.889+0000 7f021a000640 1 -- 172.21.15.69:0/2464984370 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f02200c1080 con 0x55a21cac6a20 2024-03-19T01:18:21.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.889+0000 7f0239baf740 10 client.17052 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:21.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.889+0000 7f0239baf740 1 -- 172.21.15.69:0/2464984370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a21cb00e30 con 0x55a21cac6a20 2024-03-19T01:18:21.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.889+0000 7f021a000640 1 -- 172.21.15.69:0/2464984370 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f021c005020 con 0x55a21cac6a20 2024-03-19T01:18:21.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.891+0000 7f021a000640 1 -- 172.21.15.69:0/2464984370 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f02200bc380 con 0x55a21cac6a20 2024-03-19T01:18:21.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.891+0000 7f0239baf740 10 client.17052 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:21.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.891+0000 7f0239baf740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:18:21.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.891+0000 7f0239baf740 20 client.17052 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:21.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.891+0000 7f0239baf740 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a21cb02220 0x55a21cb046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.892+0000 7f0239baf740 4 client.17052 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:21.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.892+0000 7f0239baf740 1 -- 172.21.15.69:0/2464984370 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a21ca388c0 con 0x55a21cb02220 2024-03-19T01:18:21.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.892+0000 7f022f400640 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a21cb02220 0x55a21cb046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.892+0000 7f022f400640 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a21cb02220 0x55a21cb046a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd374217f8cf38cf server_cookie=774070d4725f1d7e in_seq=0 out_seq=0 2024-03-19T01:18:21.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.892+0000 7f021a000640 10 client.17052 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:21.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.894+0000 7f021a000640 1 -- 172.21.15.69:0/2464984370 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a21ca388c0 con 0x55a21cb02220 2024-03-19T01:18:21.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.894+0000 7f021a000640 10 client.17052 handle_command_reply: tid=0 2024-03-19T01:18:21.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.894+0000 7f0239baf740 1 client.17052 shutdown 2024-03-19T01:18:21.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.894+0000 7f0239baf740 1 -- 172.21.15.69:0/2464984370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac6a20 msgr2=0x55a21caccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.894+0000 7f0239baf740 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac6a20 0x55a21caccfa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f02200bcec0 tx=0x7f02200bcb60 comp rx=0 tx=0).stop 2024-03-19T01:18:21.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.898+0000 7f0239baf740 1 -- 172.21.15.69:0/2464984370 shutdown_connections 2024-03-19T01:18:21.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.898+0000 7f0239baf740 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a21cb02220 0x55a21cb046a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.898+0000 7f0239baf740 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cacff20 0x55a21cacda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.898+0000 7f0239baf740 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cac7340 0x55a21cacd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.898+0000 7f0239baf740 1 --2- 172.21.15.69:0/2464984370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac6a20 0x55a21caccfa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.898+0000 7f0239baf740 1 -- 172.21.15.69:0/2464984370 >> 172.21.15.69:0/2464984370 conn(0x55a21cac1150 msgr2=0x55a21cac2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.898+0000 7f021a000640 0 client.17052 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:21.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.906+0000 7f0239baf740 1 -- 172.21.15.69:0/2464984370 shutdown_connections 2024-03-19T01:18:21.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.906+0000 7f0239baf740 1 -- 172.21.15.69:0/2464984370 wait complete. 2024-03-19T01:18:21.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.906+0000 7f0239baf740 20 client.17052 trim_cache size 0 max 16384 2024-03-19T01:18:21.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.946+0000 7f0239baf740 1 Processor -- start 2024-03-19T01:18:21.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.946+0000 7f0239baf740 1 -- start start 2024-03-19T01:18:21.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.946+0000 7f0239baf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cacf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.946+0000 7f0239baf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac7340 0x55a21cacc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.946+0000 7f0239baf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cb09fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.946+0000 7f0239baf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02301b52e0 con 0x55a21cac7340 2024-03-19T01:18:21.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.946+0000 7f0239baf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02301b6fa0 con 0x55a21cacff20 2024-03-19T01:18:21.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.946+0000 7f0239baf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f02301b59b0 con 0x55a21cac6a20 2024-03-19T01:18:21.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.947+0000 7f0234e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cb09fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.947+0000 7f0234e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cb09fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35146/0 (socket says 172.21.15.69:35146) 2024-03-19T01:18:21.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.947+0000 7f0234e00640 1 -- 172.21.15.69:0/1263494245 learned_addr learned my addr 172.21.15.69:0/1263494245 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:21.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.947+0000 7f0234e00640 1 -- 172.21.15.69:0/1263494245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 msgr2=0x55a21cacf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.947+0000 7f022fe00640 1 --2- 172.21.15.69:0/1263494245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cacf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.947+0000 7f0234e00640 1 --2- 172.21.15.69:0/1263494245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cacf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.947+0000 7f0234e00640 1 -- 172.21.15.69:0/1263494245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac7340 msgr2=0x55a21cacc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.947+0000 7f0234e00640 1 --2- 172.21.15.69:0/1263494245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac7340 0x55a21cacc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.947+0000 7f0234e00640 1 -- 172.21.15.69:0/1263494245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a21cac63f0 con 0x55a21cacff20 2024-03-19T01:18:21.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.951+0000 7f0234e00640 1 --2- 172.21.15.69:0/1263494245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cb09fb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0228013920 tx=0x7f0228013de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e82d937defa9dcb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.951+0000 7f0224c00640 1 -- 172.21.15.69:0/1263494245 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228021070 con 0x55a21cacff20 2024-03-19T01:18:21.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.951+0000 7f0224c00640 1 -- 172.21.15.69:0/1263494245 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f022801c410 con 0x55a21cacff20 2024-03-19T01:18:21.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.951+0000 7f0224c00640 1 -- 172.21.15.69:0/1263494245 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228010610 con 0x55a21cacff20 2024-03-19T01:18:21.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.953+0000 7f0239baf740 1 -- 172.21.15.69:0/1263494245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 msgr2=0x55a21cb09fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.953+0000 7f0239baf740 1 --2- 172.21.15.69:0/1263494245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cb09fb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0228013920 tx=0x7f0228013de0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.959+0000 7f0239baf740 1 -- 172.21.15.69:0/1263494245 shutdown_connections 2024-03-19T01:18:21.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.959+0000 7f0239baf740 1 --2- 172.21.15.69:0/1263494245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cacff20 0x55a21cb09fb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.959+0000 7f0239baf740 1 --2- 172.21.15.69:0/1263494245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cac7340 0x55a21cacc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.959+0000 7f0239baf740 1 --2- 172.21.15.69:0/1263494245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cacf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.959+0000 7f0239baf740 1 -- 172.21.15.69:0/1263494245 >> 172.21.15.69:0/1263494245 conn(0x55a21cac1150 msgr2=0x55a21cac1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.959+0000 7f0239baf740 1 -- 172.21.15.69:0/1263494245 shutdown_connections 2024-03-19T01:18:21.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.959+0000 7f0239baf740 1 -- 172.21.15.69:0/1263494245 wait complete. 2024-03-19T01:18:21.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.960+0000 7f0239baf740 1 -- start start 2024-03-19T01:18:21.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.960+0000 7f0239baf740 1 Processor -- start 2024-03-19T01:18:21.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.962+0000 7f0239baf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cb10dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.962+0000 7f022fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cb10dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.962+0000 7f022fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cb10dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48562/0 (socket says 172.21.15.69:48562) 2024-03-19T01:18:21.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.962+0000 7f0239baf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cac7340 0x55a21cb11300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.962+0000 7f022fe00640 1 -- 172.21.15.69:0/2291691749 learned_addr learned my addr 172.21.15.69:0/2291691749 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:21.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.963+0000 7f0239baf740 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cacff20 0x55a21cb11840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.963+0000 7f0239baf740 1 -- 172.21.15.69:0/2291691749 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02301b6810 con 0x55a21cacff20 2024-03-19T01:18:21.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.963+0000 7f0239baf740 1 -- 172.21.15.69:0/2291691749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02301b6140 con 0x55a21cac7340 2024-03-19T01:18:21.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.963+0000 7f0239baf740 1 -- 172.21.15.69:0/2291691749 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f02301b3310 con 0x55a21cac6a20 2024-03-19T01:18:21.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.964+0000 7f0239baf740 10 client.0 _ll_register_callbacks cb 0x7f0220005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:21.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.964+0000 7f0234e00640 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cacff20 0x55a21cb11840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.964+0000 7f0234e00640 1 -- 172.21.15.69:0/2291691749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 msgr2=0x55a21cb10dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:21.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.964+0000 7f0234e00640 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cb10dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.965+0000 7f0234e00640 1 -- 172.21.15.69:0/2291691749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cac7340 msgr2=0x55a21cb11300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:18:21.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.965+0000 7f0234e00640 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cac7340 0x55a21cb11300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.965+0000 7f0234e00640 1 -- 172.21.15.69:0/2291691749 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a21cb67070 con 0x55a21cacff20 2024-03-19T01:18:21.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.965+0000 7f0234e00640 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cacff20 0x55a21cb11840 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55a21caff100 tx=0x7f0228004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd24a0f6eaf7bb9a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.965+0000 7f021a000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:18:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.966+0000 7f021a000640 1 -- 172.21.15.69:0/2291691749 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228021040 con 0x55a21cacff20 2024-03-19T01:18:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.966+0000 7f021a000640 1 -- 172.21.15.69:0/2291691749 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f022801c5c0 con 0x55a21cacff20 2024-03-19T01:18:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.967+0000 7f0239baf740 1 -- 172.21.15.69:0/2291691749 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f022801c410 con 0x55a21cacff20 2024-03-19T01:18:21.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.970+0000 7f021a000640 1 -- 172.21.15.69:0/2291691749 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f022801c7c0 con 0x55a21cacff20 2024-03-19T01:18:21.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.970+0000 7f0239baf740 10 client.17055 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:21.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.970+0000 7f0239baf740 1 -- 172.21.15.69:0/2291691749 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a21cb68370 con 0x55a21cacff20 2024-03-19T01:18:21.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.971+0000 7f021a000640 1 -- 172.21.15.69:0/2291691749 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f022801c410 con 0x55a21cacff20 2024-03-19T01:18:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.971+0000 7f021a000640 1 -- 172.21.15.69:0/2291691749 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f022802f030 con 0x55a21cacff20 2024-03-19T01:18:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.971+0000 7f0239baf740 10 client.17055 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.971+0000 7f0239baf740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:18:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.971+0000 7f0239baf740 20 client.17055 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.971+0000 7f0239baf740 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a21cb68ab0 0x55a21cb6aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.971+0000 7f0239baf740 4 client.17055 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:18:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.971+0000 7f0239baf740 1 -- 172.21.15.69:0/2291691749 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a21ca32c10 con 0x55a21cb68ab0 2024-03-19T01:18:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.971+0000 7f022fe00640 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a21cb68ab0 0x55a21cb6aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.972+0000 7f022fe00640 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a21cb68ab0 0x55a21cb6aed0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c334bbfbe97361eb server_cookie=c53c8e048030b59b in_seq=0 out_seq=0 2024-03-19T01:18:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.972+0000 7f021a000640 10 client.17055 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.972+0000 7f021a000640 1 -- 172.21.15.69:0/2291691749 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a21ca32c10 con 0x55a21cb68ab0 2024-03-19T01:18:21.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.972+0000 7f021a000640 10 client.17055 handle_command_reply: tid=0 2024-03-19T01:18:21.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.972+0000 7f0239baf740 1 client.17055 shutdown 2024-03-19T01:18:21.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.976+0000 7f0239baf740 1 -- 172.21.15.69:0/2291691749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cacff20 msgr2=0x55a21cb11840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.976+0000 7f0239baf740 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cacff20 0x55a21cb11840 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55a21caff100 tx=0x7f0228004c70 comp rx=0 tx=0).stop 2024-03-19T01:18:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.978+0000 7f0239baf740 1 -- 172.21.15.69:0/2291691749 shutdown_connections 2024-03-19T01:18:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.978+0000 7f0239baf740 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a21cb68ab0 0x55a21cb6aed0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.978+0000 7f0239baf740 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a21cacff20 0x55a21cb11840 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.978+0000 7f0239baf740 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a21cac7340 0x55a21cb11300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.978+0000 7f0239baf740 1 --2- 172.21.15.69:0/2291691749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a21cac6a20 0x55a21cb10dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.978+0000 7f021a000640 0 client.17055 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:21.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.978+0000 7f0239baf740 1 -- 172.21.15.69:0/2291691749 >> 172.21.15.69:0/2291691749 conn(0x55a21cac1150 msgr2=0x55a21caccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.980 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:18:21.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.978+0000 7f0239baf740 1 -- 172.21.15.69:0/2291691749 shutdown_connections 2024-03-19T01:18:21.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.978+0000 7f0239baf740 1 -- 172.21.15.69:0/2291691749 wait complete. 2024-03-19T01:18:21.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.978+0000 7f0239baf740 20 client.17055 trim_cache size 0 max 16384 2024-03-19T01:18:21.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.983+0000 7f0236e00640 1 -- 172.21.15.69:0/2635330939 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02140a2950 msgr2=0x7f02140a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.983+0000 7f0236e00640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02140a2950 0x7f02140a4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f021c005e00 tx=0x7f021c01a8b0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.983+0000 7f0236e00640 1 -- 172.21.15.69:0/2635330939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d60 msgr2=0x7f023011a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:21.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.983+0000 7f0236e00640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d60 0x7f023011a750 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0220009230 tx=0x7f02200043b0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.983+0000 7f0236e00640 1 -- 172.21.15.69:0/2635330939 shutdown_connections 2024-03-19T01:18:21.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.983+0000 7f0236e00640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023011b010 0x7f0230116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.983+0000 7f0236e00640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023010bf40 0x7f0230116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.983+0000 7f0236e00640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02140a2950 0x7f02140a4e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.983+0000 7f0236e00640 1 --2- 172.21.15.69:0/2635330939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230108d60 0x7f023011a750 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:21.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.983+0000 7f0236e00640 1 -- 172.21.15.69:0/2635330939 >> 172.21.15.69:0/2635330939 conn(0x7f023006cd60 msgr2=0x7f02301107c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:21.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.984+0000 7f0236e00640 1 -- 172.21.15.69:0/2635330939 shutdown_connections 2024-03-19T01:18:21.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:21.985+0000 7f0236e00640 1 -- 172.21.15.69:0/2635330939 wait complete. 2024-03-19T01:18:22.020 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/entry/built-in.a 2024-03-19T01:18:22.144 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/amd/ibs.o 2024-03-19T01:18:22.211 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/compat.o 2024-03-19T01:18:22.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:22 smithi069 ceph-mon[31727]: pgmap v237: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 640 KiB/s wr, 140 op/s 2024-03-19T01:18:22.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:22 smithi163 ceph-mon[67428]: pgmap v237: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 640 KiB/s wr, 140 op/s 2024-03-19T01:18:22.399 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/iomap/direct-io.o 2024-03-19T01:18:22.490 INFO:tasks.workunit.client.0.smithi069.stdout: AR crypto/asymmetric_keys/built-in.a 2024-03-19T01:18:22.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:22 smithi086 ceph-mon[66829]: pgmap v237: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 640 KiB/s wr, 140 op/s 2024-03-19T01:18:22.550 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/locking/spinlock.o 2024-03-19T01:18:22.574 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/api.o 2024-03-19T01:18:22.632 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/vmscan.o 2024-03-19T01:18:22.637 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/sched/build_utility.o 2024-03-19T01:18:22.823 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/power/qos.o 2024-03-19T01:18:23.035 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:18:23.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:18:23.036 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:18:23.092 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/locking/osq_lock.o 2024-03-19T01:18:23.218 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/proc.o 2024-03-19T01:18:23.524 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/amd/uncore.o 2024-03-19T01:18:23.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.556+0000 7f56cbc00640 1 -- 172.21.15.69:0/2636255915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56bc0ad2a0 msgr2=0x7f56bc0b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:23.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.556+0000 7f56cbc00640 1 --2- 172.21.15.69:0/2636255915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56bc0ad2a0 0x7f56bc0b3c20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f56c404f850 tx=0x7f56c408f430 comp rx=0 tx=0).stop 2024-03-19T01:18:23.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.556+0000 7f56cbc00640 1 -- 172.21.15.69:0/2636255915 shutdown_connections 2024-03-19T01:18:23.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.556+0000 7f56cbc00640 1 --2- 172.21.15.69:0/2636255915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56bc0ad2a0 0x7f56bc0b3c20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.556+0000 7f56cbc00640 1 --2- 172.21.15.69:0/2636255915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56bc0a5b30 0x7f56bc0acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.556+0000 7f56cbc00640 1 --2- 172.21.15.69:0/2636255915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc0a5180 0x7f56bc0a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.556+0000 7f56cbc00640 1 -- 172.21.15.69:0/2636255915 >> 172.21.15.69:0/2636255915 conn(0x7f56bc01a800 msgr2=0x7f56bc01ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:23.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.559+0000 7f56cbc00640 1 -- 172.21.15.69:0/2636255915 shutdown_connections 2024-03-19T01:18:23.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.559+0000 7f56cbc00640 1 -- 172.21.15.69:0/2636255915 wait complete. 2024-03-19T01:18:23.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.560+0000 7f56cbc00640 1 Processor -- start 2024-03-19T01:18:23.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.562+0000 7f56cbc00640 1 -- start start 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.562+0000 7f56cbc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56bc0a5180 0x7f56bc0bb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.562+0000 7f56cbc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc0ad2a0 0x7f56bc0b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.562+0000 7f56cbc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56bc0bb810 0x7f56bc0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.562+0000 7f56cbc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc0b7060 con 0x7f56bc0bb810 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.562+0000 7f56cbc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc0b6ee0 con 0x7f56bc0ad2a0 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.562+0000 7f56cbc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc0b71e0 con 0x7f56bc0a5180 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc0ad2a0 0x7f56bc0b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56bc0a5180 0x7f56bc0bb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc0ad2a0 0x7f56bc0b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35174/0 (socket says 172.21.15.69:35174) 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c3400640 1 -- 172.21.15.69:0/2331309957 learned_addr learned my addr 172.21.15.69:0/2331309957 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c9200640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56bc0bb810 0x7f56bc0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c3400640 1 -- 172.21.15.69:0/2331309957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56bc0a5180 msgr2=0x7f56bc0bb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:23.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c3400640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56bc0a5180 0x7f56bc0bb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c3400640 1 -- 172.21.15.69:0/2331309957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56bc0bb810 msgr2=0x7f56bc0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:23.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c3400640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56bc0bb810 0x7f56bc0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c3400640 1 -- 172.21.15.69:0/2331309957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc140920 con 0x7f56bc0ad2a0 2024-03-19T01:18:23.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c3e00640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56bc0a5180 0x7f56bc0bb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:23.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.563+0000 7f56c9200640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56bc0bb810 0x7f56bc0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:18:23.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.564+0000 7f56c3400640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc0ad2a0 0x7f56bc0b2c20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f56b000c8c0 tx=0x7f56b000cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:23.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.564+0000 7f56c1600640 1 -- 172.21.15.69:0/2331309957 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0007bb0 con 0x7f56bc0ad2a0 2024-03-19T01:18:23.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.564+0000 7f56c1600640 1 -- 172.21.15.69:0/2331309957 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b0007d50 con 0x7f56bc0ad2a0 2024-03-19T01:18:23.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.564+0000 7f56c1600640 1 -- 172.21.15.69:0/2331309957 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0007bb0 con 0x7f56bc0ad2a0 2024-03-19T01:18:23.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.568+0000 7f56cbc00640 1 -- 172.21.15.69:0/2331309957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc140c10 con 0x7f56bc0ad2a0 2024-03-19T01:18:23.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.568+0000 7f56cbc00640 1 -- 172.21.15.69:0/2331309957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc1410e0 con 0x7f56bc0ad2a0 2024-03-19T01:18:23.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.575+0000 7f56ce99e740 1 Processor -- start 2024-03-19T01:18:23.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.579+0000 7f56ce99e740 1 -- start start 2024-03-19T01:18:23.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.579+0000 7f56c1600640 1 -- 172.21.15.69:0/2331309957 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f56b000f450 con 0x7f56bc0ad2a0 2024-03-19T01:18:23.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.579+0000 7f56ce99e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 0x558491f5b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.579+0000 7f56ce99e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 0x558491f63ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.579+0000 7f56ce99e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.579+0000 7f56ce99e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558491ebdfb0 con 0x558491f5b600 2024-03-19T01:18:23.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.579+0000 7f56ce99e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558491e66ab0 con 0x558491f641e0 2024-03-19T01:18:23.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.579+0000 7f56ce99e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558491eba050 con 0x558491f5ace0 2024-03-19T01:18:23.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.580+0000 7f56c9200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 0x558491f5b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.580+0000 7f56c9200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 0x558491f5b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48586/0 (socket says 172.21.15.69:48586) 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.580+0000 7f56c9200640 1 -- 172.21.15.69:0/1933994564 learned_addr learned my addr 172.21.15.69:0/1933994564 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.580+0000 7f56c9200640 1 -- 172.21.15.69:0/1933994564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 msgr2=0x558491f665d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.580+0000 7f56c9200640 1 --2- 172.21.15.69:0/1933994564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f665d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.580+0000 7f56c9200640 1 -- 172.21.15.69:0/1933994564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 msgr2=0x558491f63ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.580+0000 7f56c3e00640 1 --2- 172.21.15.69:0/1933994564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 0x558491f63ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.580+0000 7f56c9200640 1 --2- 172.21.15.69:0/1933994564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 0x558491f63ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.580+0000 7f56c9200640 1 -- 172.21.15.69:0/1933994564 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558491e84e00 con 0x558491f5ace0 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.580+0000 7f56c3e00640 1 --2- 172.21.15.69:0/1933994564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 0x558491f63ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.581+0000 7f56c9200640 1 --2- 172.21.15.69:0/1933994564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 0x558491f5b0c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f56c4066f50 tx=0x7f56c4096530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=261cbe6f39c2837a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.581+0000 7f56b6000640 1 -- 172.21.15.69:0/1933994564 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c40a7020 con 0x558491f5ace0 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.581+0000 7f56b6000640 1 -- 172.21.15.69:0/1933994564 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c409d040 con 0x558491f5ace0 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.581+0000 7f56b6000640 1 -- 172.21.15.69:0/1933994564 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4097630 con 0x558491f5ace0 2024-03-19T01:18:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.581+0000 7f56c1600640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f56ac0a2df0 0x7f56ac0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.582+0000 7f56c1600640 1 -- 172.21.15.69:0/2331309957 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f56b00079b0 con 0x7f56bc0ad2a0 2024-03-19T01:18:23.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.582+0000 7f56ce99e740 1 -- 172.21.15.69:0/1933994564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 msgr2=0x558491f5b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:23.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.582+0000 7f56ce99e740 1 --2- 172.21.15.69:0/1933994564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 0x558491f5b0c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f56c4066f50 tx=0x7f56c4096530 comp rx=0 tx=0).stop 2024-03-19T01:18:23.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.584+0000 7f56c3e00640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f56ac0a2df0 0x7f56ac0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.585+0000 7f56ce99e740 1 -- 172.21.15.69:0/1933994564 shutdown_connections 2024-03-19T01:18:23.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.585+0000 7f56ce99e740 1 --2- 172.21.15.69:0/1933994564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f665d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.585+0000 7f56ce99e740 1 --2- 172.21.15.69:0/1933994564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 0x558491f63ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.585+0000 7f56ce99e740 1 --2- 172.21.15.69:0/1933994564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 0x558491f5b0c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.585+0000 7f56ce99e740 1 -- 172.21.15.69:0/1933994564 >> 172.21.15.69:0/1933994564 conn(0x558491f55410 msgr2=0x558491f557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:23.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.585+0000 7f56c3e00640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f56ac0a2df0 0x7f56ac0a52b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x558491f5a7b0 tx=0x7f56b800a300 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:23.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.585+0000 7f56ce99e740 1 -- 172.21.15.69:0/1933994564 shutdown_connections 2024-03-19T01:18:23.587 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/locking/qspinlock.o 2024-03-19T01:18:23.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.586+0000 7f56ce99e740 1 -- 172.21.15.69:0/1933994564 wait complete. 2024-03-19T01:18:23.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.586+0000 7f56ce99e740 1 -- start start 2024-03-19T01:18:23.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.587+0000 7f56ce99e740 1 Processor -- start 2024-03-19T01:18:23.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.594+0000 7f56ce99e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 0x558491f61240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.594+0000 7f56ce99e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 0x558491f61780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.594+0000 7f56ce99e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f61cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.594+0000 7f56ce99e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558491f66f70 con 0x558491f5b600 2024-03-19T01:18:23.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.594+0000 7f56ce99e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558491e79f00 con 0x558491f641e0 2024-03-19T01:18:23.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.594+0000 7f56ce99e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558491f670f0 con 0x558491f5ace0 2024-03-19T01:18:23.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.594+0000 7f56ce99e740 10 client.0 _ll_register_callbacks cb 0x558491e798d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:23.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.594+0000 7f56c3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 0x558491f61780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.595+0000 7f56c9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f61cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.595+0000 7f56c3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 0x558491f61780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41560/0 (socket says 172.21.15.69:41560) 2024-03-19T01:18:23.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.595+0000 7f56c3400640 1 -- 172.21.15.69:0/2376101794 learned_addr learned my addr 172.21.15.69:0/2376101794 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:23.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.595+0000 7f56c3e00640 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 0x558491f61240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.595+0000 7f56c9200640 1 -- 172.21.15.69:0/2376101794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 msgr2=0x558491f61240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:23.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.595+0000 7f56c9200640 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 0x558491f61240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.595+0000 7f56c9200640 1 -- 172.21.15.69:0/2376101794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 msgr2=0x558491f61780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:23.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.595+0000 7f56c9200640 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 0x558491f61780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.595+0000 7f56c9200640 1 -- 172.21.15.69:0/2376101794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558491c9a7e0 con 0x558491f641e0 2024-03-19T01:18:23.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.595+0000 7f56c3e00640 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 0x558491f61240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:18:23.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.595+0000 7f56c3400640 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 0x558491f61780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:23.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.596+0000 7f56c9200640 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f61cc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f56c40507f0 tx=0x7f56c4098f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff987b9e6959afa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:23.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.596+0000 7f56aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:18:23.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.596+0000 7f56aaa00640 1 -- 172.21.15.69:0/2376101794 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c40a7070 con 0x558491f641e0 2024-03-19T01:18:23.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.596+0000 7f56aaa00640 1 -- 172.21.15.69:0/2376101794 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f56c4097430 con 0x558491f641e0 2024-03-19T01:18:23.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.596+0000 7f56ce99e740 1 -- 172.21.15.69:0/2376101794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c409d040 con 0x558491f641e0 2024-03-19T01:18:23.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.596+0000 7f56aaa00640 1 -- 172.21.15.69:0/2376101794 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f56c40bd050 con 0x558491f641e0 2024-03-19T01:18:23.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.597+0000 7f56ce99e740 10 client.27359 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:23.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.597+0000 7f56ce99e740 1 -- 172.21.15.69:0/2376101794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558491cfaf70 con 0x558491f641e0 2024-03-19T01:18:23.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.597+0000 7f56aaa00640 1 -- 172.21.15.69:0/2376101794 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c409d040 con 0x558491f641e0 2024-03-19T01:18:23.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.598+0000 7f56aaa00640 1 -- 172.21.15.69:0/2376101794 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f56c40a2040 con 0x558491f641e0 2024-03-19T01:18:23.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.598+0000 7f56ce99e740 10 client.27359 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:23.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.598+0000 7f56ce99e740 10 client.27359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:18:23.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.598+0000 7f56ce99e740 20 client.27359 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:23.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.598+0000 7f56ce99e740 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558491f74390 0x558491f76780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.598+0000 7f56ce99e740 4 client.27359 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:23.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.598+0000 7f56ce99e740 1 -- 172.21.15.69:0/2376101794 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558491ecb4f0 con 0x558491f74390 2024-03-19T01:18:23.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.599+0000 7f56c3e00640 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558491f74390 0x558491f76780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.599+0000 7f56c3e00640 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558491f74390 0x558491f76780 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ec159f1d5049a6 server_cookie=4d95b97e0b346e9a in_seq=0 out_seq=0 2024-03-19T01:18:23.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.599+0000 7f56aaa00640 10 client.27359 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:23.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.600+0000 7f56aaa00640 1 -- 172.21.15.69:0/2376101794 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558491ecb4f0 con 0x558491f74390 2024-03-19T01:18:23.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.600+0000 7f56aaa00640 10 client.27359 handle_command_reply: tid=0 2024-03-19T01:18:23.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.600+0000 7f56ce99e740 1 client.27359 shutdown 2024-03-19T01:18:23.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56ce99e740 1 -- 172.21.15.69:0/2376101794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 msgr2=0x558491f61cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:23.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56ce99e740 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f61cc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f56c40507f0 tx=0x7f56c4098f70 comp rx=0 tx=0).stop 2024-03-19T01:18:23.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56ce99e740 1 -- 172.21.15.69:0/2376101794 shutdown_connections 2024-03-19T01:18:23.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56ce99e740 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558491f74390 0x558491f76780 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56ce99e740 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f61cc0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56ce99e740 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5b600 0x558491f61780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56ce99e740 1 --2- 172.21.15.69:0/2376101794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5ace0 0x558491f61240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56ce99e740 1 -- 172.21.15.69:0/2376101794 >> 172.21.15.69:0/2376101794 conn(0x558491f55410 msgr2=0x558491f57160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:23.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56aaa00640 0 client.27359 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:23.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56ce99e740 1 -- 172.21.15.69:0/2376101794 shutdown_connections 2024-03-19T01:18:23.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56ce99e740 1 -- 172.21.15.69:0/2376101794 wait complete. 2024-03-19T01:18:23.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.601+0000 7f56ce99e740 20 client.27359 trim_cache size 0 max 16384 2024-03-19T01:18:23.612 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/iomap/fiemap.o 2024-03-19T01:18:23.722 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/cipher.o 2024-03-19T01:18:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.835+0000 7f56ce99e740 1 Processor -- start 2024-03-19T01:18:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.835+0000 7f56ce99e740 1 -- start start 2024-03-19T01:18:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.836+0000 7f56ce99e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 0x558491f63970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.837+0000 7f56c3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 0x558491f63970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.837+0000 7f56c3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 0x558491f63970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41568/0 (socket says 172.21.15.69:41568) 2024-03-19T01:18:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.837+0000 7f56c3e00640 1 -- 172.21.15.69:0/621667827 learned_addr learned my addr 172.21.15.69:0/621667827 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.837+0000 7f56ce99e740 1 --2- 172.21.15.69:0/621667827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f5b600 0x558491f6fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.837+0000 7f56ce99e740 1 --2- 172.21.15.69:0/621667827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f641e0 0x558491f7be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.838+0000 7f56c3400640 1 --2- 172.21.15.69:0/621667827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f5b600 0x558491f6fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.838+0000 7f56ce99e740 1 -- 172.21.15.69:0/621667827 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc162ea0 con 0x558491f5ace0 2024-03-19T01:18:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.838+0000 7f56ce99e740 1 -- 172.21.15.69:0/621667827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc164b60 con 0x558491f5b600 2024-03-19T01:18:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.838+0000 7f56ce99e740 1 -- 172.21.15.69:0/621667827 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc163570 con 0x558491f641e0 2024-03-19T01:18:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.838+0000 7f56c3400640 1 -- 172.21.15.69:0/621667827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f641e0 msgr2=0x558491f7be10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:18:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.838+0000 7f56c3400640 1 --2- 172.21.15.69:0/621667827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f641e0 0x558491f7be10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.838+0000 7f56c3400640 1 -- 172.21.15.69:0/621667827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 msgr2=0x558491f63970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.838+0000 7f56c3400640 1 --2- 172.21.15.69:0/621667827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 0x558491f63970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.838+0000 7f56c3400640 1 -- 172.21.15.69:0/621667827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558491f601f0 con 0x558491f5b600 2024-03-19T01:18:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.839+0000 7f56c3e00640 1 --2- 172.21.15.69:0/621667827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 0x558491f63970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.839+0000 7f56c3400640 1 --2- 172.21.15.69:0/621667827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f5b600 0x558491f6fd70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007590 tx=0x7f56b0007f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea0a27b0d942ce5e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.839+0000 7f56b5600640 1 -- 172.21.15.69:0/621667827 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b00b8070 con 0x558491f5b600 2024-03-19T01:18:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.839+0000 7f56b5600640 1 -- 172.21.15.69:0/621667827 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b009ab00 con 0x558491f5b600 2024-03-19T01:18:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.839+0000 7f56b5600640 1 -- 172.21.15.69:0/621667827 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b00b3070 con 0x558491f5b600 2024-03-19T01:18:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.840+0000 7f56ce99e740 1 -- 172.21.15.69:0/621667827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f5b600 msgr2=0x558491f6fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.840+0000 7f56ce99e740 1 --2- 172.21.15.69:0/621667827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f5b600 0x558491f6fd70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007590 tx=0x7f56b0007f00 comp rx=0 tx=0).stop 2024-03-19T01:18:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.841+0000 7f56ce99e740 1 -- 172.21.15.69:0/621667827 shutdown_connections 2024-03-19T01:18:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.841+0000 7f56ce99e740 1 --2- 172.21.15.69:0/621667827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f641e0 0x558491f7be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.841+0000 7f56ce99e740 1 --2- 172.21.15.69:0/621667827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f5b600 0x558491f6fd70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.841+0000 7f56ce99e740 1 --2- 172.21.15.69:0/621667827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 0x558491f63970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.841+0000 7f56ce99e740 1 -- 172.21.15.69:0/621667827 >> 172.21.15.69:0/621667827 conn(0x558491f55410 msgr2=0x558491f570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.841+0000 7f56ce99e740 1 -- 172.21.15.69:0/621667827 shutdown_connections 2024-03-19T01:18:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.841+0000 7f56ce99e740 1 -- 172.21.15.69:0/621667827 wait complete. 2024-03-19T01:18:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.842+0000 7f56ce99e740 1 -- start start 2024-03-19T01:18:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.842+0000 7f56ce99e740 1 Processor -- start 2024-03-19T01:18:23.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.844+0000 7f56ce99e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 0x558491f6c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.844+0000 7f56ce99e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5b600 0x558491f68b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.844+0000 7f56ce99e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.844+0000 7f56c3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5b600 0x558491f68b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.844+0000 7f56c3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5b600 0x558491f68b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48602/0 (socket says 172.21.15.69:48602) 2024-03-19T01:18:23.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.844+0000 7f56ce99e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1643d0 con 0x558491f5ace0 2024-03-19T01:18:23.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.844+0000 7f56c3400640 1 -- 172.21.15.69:0/349112029 learned_addr learned my addr 172.21.15.69:0/349112029 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:23.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.844+0000 7f56c3e00640 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 0x558491f6c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.844+0000 7f56c9200640 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.844+0000 7f56ce99e740 1 -- 172.21.15.69:0/349112029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc163d00 con 0x558491f641e0 2024-03-19T01:18:23.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.844+0000 7f56ce99e740 1 -- 172.21.15.69:0/349112029 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc005250 con 0x558491f5b600 2024-03-19T01:18:23.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.845+0000 7f56ce99e740 10 client.0 _ll_register_callbacks cb 0x7f56c4098e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:23.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.845+0000 7f56c9200640 1 -- 172.21.15.69:0/349112029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5b600 msgr2=0x558491f68b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.845+0000 7f56c9200640 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5b600 0x558491f68b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.845+0000 7f56c9200640 1 -- 172.21.15.69:0/349112029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 msgr2=0x558491f6c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.845+0000 7f56c9200640 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 0x558491f6c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.845+0000 7f56c9200640 1 -- 172.21.15.69:0/349112029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558491b92170 con 0x558491f641e0 2024-03-19T01:18:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.846+0000 7f56c3e00640 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 0x558491f6c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:18:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.846+0000 7f56c9200640 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f692c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f56c40adbb0 tx=0x7f56c408f9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56394b2e3e09e5fc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.846+0000 7f56aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:18:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.846+0000 7f56aaa00640 1 -- 172.21.15.69:0/349112029 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c40a7040 con 0x558491f641e0 2024-03-19T01:18:23.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.847+0000 7f56ce99e740 1 -- 172.21.15.69:0/349112029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b009ab00 con 0x558491f641e0 2024-03-19T01:18:23.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.848+0000 7f56aaa00640 1 -- 172.21.15.69:0/349112029 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f56c4096d50 con 0x558491f641e0 2024-03-19T01:18:23.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.851+0000 7f56aaa00640 1 -- 172.21.15.69:0/349112029 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f56c409c040 con 0x558491f641e0 2024-03-19T01:18:23.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.853+0000 7f56ce99e740 10 client.27365 fetch_fsmap learned FSMap version 47 2024-03-19T01:18:23.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.853+0000 7f56ce99e740 1 -- 172.21.15.69:0/349112029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558491fdb480 con 0x558491f641e0 2024-03-19T01:18:23.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.854+0000 7f56aaa00640 1 -- 172.21.15.69:0/349112029 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b009ab00 con 0x558491f641e0 2024-03-19T01:18:23.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.854+0000 7f56aaa00640 1 -- 172.21.15.69:0/349112029 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 47) ==== 8418+0+0 (secure 0 0 0) 0x7f56c4097430 con 0x558491f641e0 2024-03-19T01:18:23.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.854+0000 7f56ce99e740 10 client.27365 fetch_fsmap finished waiting for FSMap version 47 2024-03-19T01:18:23.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.854+0000 7f56ce99e740 10 client.27365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:18:23.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.854+0000 7f56ce99e740 20 client.27365 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:23.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.854+0000 7f56ce99e740 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558491fdbc10 0x558491fde030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:23.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.854+0000 7f56ce99e740 4 client.27365 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:18:23.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.854+0000 7f56ce99e740 1 -- 172.21.15.69:0/349112029 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558491ec5840 con 0x558491fdbc10 2024-03-19T01:18:23.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.857+0000 7f56c3e00640 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558491fdbc10 0x558491fde030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.857+0000 7f56c3e00640 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558491fdbc10 0x558491fde030 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51f17a46c161082d server_cookie=71abdd70ce009282 in_seq=0 out_seq=0 2024-03-19T01:18:23.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.857+0000 7f56aaa00640 10 client.27365 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:23.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.858+0000 7f56aaa00640 1 -- 172.21.15.69:0/349112029 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558491ec5840 con 0x558491fdbc10 2024-03-19T01:18:23.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.858+0000 7f56aaa00640 10 client.27365 handle_command_reply: tid=0 2024-03-19T01:18:23.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.859+0000 7f56ce99e740 1 client.27365 shutdown 2024-03-19T01:18:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.859+0000 7f56ce99e740 1 -- 172.21.15.69:0/349112029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 msgr2=0x558491f692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.859+0000 7f56ce99e740 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f692c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f56c40adbb0 tx=0x7f56c408f9f0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.862+0000 7f56ce99e740 1 -- 172.21.15.69:0/349112029 shutdown_connections 2024-03-19T01:18:23.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.862+0000 7f56ce99e740 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x558491fdbc10 0x558491fde030 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.862+0000 7f56ce99e740 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558491f641e0 0x558491f692c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.862+0000 7f56ce99e740 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558491f5b600 0x558491f68b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.862+0000 7f56ce99e740 1 --2- 172.21.15.69:0/349112029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558491f5ace0 0x558491f6c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.862+0000 7f56ce99e740 1 -- 172.21.15.69:0/349112029 >> 172.21.15.69:0/349112029 conn(0x558491f55410 msgr2=0x558491f56710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:23.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.862+0000 7f56aaa00640 0 client.27365 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:23.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.866+0000 7f56ce99e740 1 -- 172.21.15.69:0/349112029 shutdown_connections 2024-03-19T01:18:23.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.866+0000 7f56ce99e740 1 -- 172.21.15.69:0/349112029 wait complete. 2024-03-19T01:18:23.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.869+0000 7f56ce99e740 20 client.27365 trim_cache size 0 max 16384 2024-03-19T01:18:23.871 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:18:23.871 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:18:23.871 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "7213bbdf-dd79-4eae-8bf8-c9193b2bf430", 2024-03-19T01:18:23.871 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:18:23.871 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:18:23.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.872+0000 7f56cbc00640 1 -- 172.21.15.69:0/2331309957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f56ac0a2df0 msgr2=0x7f56ac0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:23.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.872+0000 7f56cbc00640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f56ac0a2df0 0x7f56ac0a52b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x558491f5a7b0 tx=0x7f56b800a300 comp rx=0 tx=0).stop 2024-03-19T01:18:23.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.874+0000 7f56cbc00640 1 -- 172.21.15.69:0/2331309957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc0ad2a0 msgr2=0x7f56bc0b2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:23.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.874+0000 7f56cbc00640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc0ad2a0 0x7f56bc0b2c20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f56b000c8c0 tx=0x7f56b000cd80 comp rx=0 tx=0).stop 2024-03-19T01:18:23.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.874+0000 7f56cbc00640 1 -- 172.21.15.69:0/2331309957 shutdown_connections 2024-03-19T01:18:23.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.874+0000 7f56cbc00640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56bc0bb810 0x7f56bc0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.874+0000 7f56cbc00640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f56ac0a2df0 0x7f56ac0a52b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.874+0000 7f56cbc00640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56bc0ad2a0 0x7f56bc0b2c20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.874+0000 7f56cbc00640 1 --2- 172.21.15.69:0/2331309957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56bc0a5180 0x7f56bc0bb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:23.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.874+0000 7f56cbc00640 1 -- 172.21.15.69:0/2331309957 >> 172.21.15.69:0/2331309957 conn(0x7f56bc01a800 msgr2=0x7f56bc0a1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:23.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.875+0000 7f56cbc00640 1 -- 172.21.15.69:0/2331309957 shutdown_connections 2024-03-19T01:18:23.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:23.875+0000 7f56cbc00640 1 -- 172.21.15.69:0/2331309957 wait complete. 2024-03-19T01:18:23.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:18:23.931 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:18:24.060 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/power/main.o 2024-03-19T01:18:24.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:24 smithi069 ceph-mon[31727]: pgmap v238: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 679 KiB/s wr, 140 op/s 2024-03-19T01:18:24.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:24.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:24.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:24 smithi163 ceph-mon[67428]: pgmap v238: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 679 KiB/s wr, 140 op/s 2024-03-19T01:18:24.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:24.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:24.398 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/locking/rtmutex_api.o 2024-03-19T01:18:24.426 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/sysctl.o 2024-03-19T01:18:24.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:24 smithi086 ceph-mon[66829]: pgmap v238: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 679 KiB/s wr, 140 op/s 2024-03-19T01:18:24.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:24.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:24.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.515+0000 7f82ef200640 1 -- 172.21.15.69:0/4189926442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f010fe40 msgr2=0x7f82f0110220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:24.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.515+0000 7f82ef200640 1 --2- 172.21.15.69:0/4189926442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f010fe40 0x7f82f0110220 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009860 tx=0x7f82e002bb90 comp rx=0 tx=0).stop 2024-03-19T01:18:24.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.519+0000 7f82ef200640 1 -- 172.21.15.69:0/4189926442 shutdown_connections 2024-03-19T01:18:24.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.519+0000 7f82ef200640 1 --2- 172.21.15.69:0/4189926442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f011b000 0x7f82f011d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.519+0000 7f82ef200640 1 --2- 172.21.15.69:0/4189926442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f01088b0 0x7f82f0108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.519+0000 7f82ef200640 1 --2- 172.21.15.69:0/4189926442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f010fe40 0x7f82f0110220 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.519+0000 7f82ef200640 1 -- 172.21.15.69:0/4189926442 >> 172.21.15.69:0/4189926442 conn(0x7f82f006c770 msgr2=0x7f82f006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:24.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.524+0000 7f82ef200640 1 -- 172.21.15.69:0/4189926442 shutdown_connections 2024-03-19T01:18:24.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.524+0000 7f82ef200640 1 -- 172.21.15.69:0/4189926442 wait complete. 2024-03-19T01:18:24.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.526+0000 7f82ef200640 1 Processor -- start 2024-03-19T01:18:24.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.527+0000 7f82ef200640 1 -- start start 2024-03-19T01:18:24.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.527+0000 7f82ef200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f01088b0 0x7f82f011a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.529+0000 7f82ef200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f010fe40 0x7f82f0116830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.529+0000 7f82ef200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f011b000 0x7f82f0116d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.529+0000 7f82ef200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01131e0 con 0x7f82f011b000 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.529+0000 7f82ef200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0113060 con 0x7f82f01088b0 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.529+0000 7f82ef200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0113360 con 0x7f82f010fe40 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.529+0000 7f82ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f01088b0 0x7f82f011a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.529+0000 7f82ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f01088b0 0x7f82f011a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35234/0 (socket says 172.21.15.69:35234) 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.529+0000 7f82ede00640 1 -- 172.21.15.69:0/195880678 learned_addr learned my addr 172.21.15.69:0/195880678 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.529+0000 7f82ed400640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f010fe40 0x7f82f0116830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.530+0000 7f82ee800640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f011b000 0x7f82f0116d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.530+0000 7f82ede00640 1 -- 172.21.15.69:0/195880678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f010fe40 msgr2=0x7f82f0116830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.530+0000 7f82ede00640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f010fe40 0x7f82f0116830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.530+0000 7f82ede00640 1 -- 172.21.15.69:0/195880678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f011b000 msgr2=0x7f82f0116d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.530+0000 7f82ede00640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f011b000 0x7f82f0116d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.530+0000 7f82ede00640 1 -- 172.21.15.69:0/195880678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f01174a0 con 0x7f82f01088b0 2024-03-19T01:18:24.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.530+0000 7f82ee800640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f011b000 0x7f82f0116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:24.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.531+0000 7f82ede00640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f01088b0 0x7f82f011a7a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f82e002c120 tx=0x7f82e0005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:24.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.533+0000 7f82e6a00640 1 -- 172.21.15.69:0/195880678 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0039040 con 0x7f82f01088b0 2024-03-19T01:18:24.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.533+0000 7f82e6a00640 1 -- 172.21.15.69:0/195880678 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e0042070 con 0x7f82f01088b0 2024-03-19T01:18:24.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.533+0000 7f82e6a00640 1 -- 172.21.15.69:0/195880678 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e00046c0 con 0x7f82f01088b0 2024-03-19T01:18:24.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.535+0000 7f82ef200640 1 -- 172.21.15.69:0/195880678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f01aac50 con 0x7f82f01088b0 2024-03-19T01:18:24.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.535+0000 7f82ef200640 1 -- 172.21.15.69:0/195880678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f01ab000 con 0x7f82f01088b0 2024-03-19T01:18:24.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.543+0000 7f82e6a00640 1 -- 172.21.15.69:0/195880678 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f82e002c530 con 0x7f82f01088b0 2024-03-19T01:18:24.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.547+0000 7f82f7c0e740 1 Processor -- start 2024-03-19T01:18:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.550+0000 7f82e6a00640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82d40a2af0 0x7f82d40a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.551+0000 7f82e6a00640 1 -- 172.21.15.69:0/195880678 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f82e00f0350 con 0x7f82f01088b0 2024-03-19T01:18:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.551+0000 7f82f7c0e740 1 -- start start 2024-03-19T01:18:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.551+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 0x5635792e5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.551+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 0x5635792edef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.551+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 0x5635792f0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.551+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5635792492c0 con 0x5635792ee430 2024-03-19T01:18:24.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.551+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635791f1b60 con 0x5635792e5850 2024-03-19T01:18:24.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.551+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563579245360 con 0x5635792e4f30 2024-03-19T01:18:24.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.553+0000 7f82ee800640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82d40a2af0 0x7f82d40a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.553+0000 7f82ee800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 0x5635792f0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.553+0000 7f82ee800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 0x5635792f0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41608/0 (socket says 172.21.15.69:41608) 2024-03-19T01:18:24.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.553+0000 7f82ee800640 1 -- 172.21.15.69:0/3810024019 learned_addr learned my addr 172.21.15.69:0/3810024019 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:24.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.555+0000 7f82ed400640 1 --2- 172.21.15.69:0/3810024019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 0x5635792edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.556+0000 7f82ed400640 1 -- 172.21.15.69:0/3810024019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 msgr2=0x5635792e5310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:24.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.557+0000 7f82ede00640 1 --2- 172.21.15.69:0/3810024019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 0x5635792e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.557+0000 7f82ed400640 1 --2- 172.21.15.69:0/3810024019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 0x5635792e5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.557+0000 7f82ed400640 1 -- 172.21.15.69:0/3810024019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 msgr2=0x5635792f0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:24.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.557+0000 7f82ed400640 1 --2- 172.21.15.69:0/3810024019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 0x5635792f0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.557+0000 7f82ed400640 1 -- 172.21.15.69:0/3810024019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635790277d0 con 0x5635792e5850 2024-03-19T01:18:24.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.557+0000 7f82ee800640 1 --2- 172.21.15.69:0/3810024019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 0x5635792f0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:24.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.558+0000 7f82ee800640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82d40a2af0 0x7f82d40a4fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82e80118e0 tx=0x7f82e8007eb0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:24.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.558+0000 7f82ede00640 1 --2- 172.21.15.69:0/3810024019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 0x5635792e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:24.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.560+0000 7f82ed400640 1 --2- 172.21.15.69:0/3810024019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 0x5635792edef0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f82f0118080 tx=0x7f82dc022820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27e02d84be18a200 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:24.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.561+0000 7f82db400640 1 -- 172.21.15.69:0/3810024019 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc02f070 con 0x5635792e5850 2024-03-19T01:18:24.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.561+0000 7f82db400640 1 -- 172.21.15.69:0/3810024019 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82dc0035f0 con 0x5635792e5850 2024-03-19T01:18:24.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.561+0000 7f82db400640 1 -- 172.21.15.69:0/3810024019 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc02a410 con 0x5635792e5850 2024-03-19T01:18:24.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.562+0000 7f82f7c0e740 1 -- 172.21.15.69:0/3810024019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 msgr2=0x5635792edef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:24.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.562+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/3810024019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 0x5635792edef0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f82f0118080 tx=0x7f82dc022820 comp rx=0 tx=0).stop 2024-03-19T01:18:24.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.562+0000 7f82f7c0e740 1 -- 172.21.15.69:0/3810024019 shutdown_connections 2024-03-19T01:18:24.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.562+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/3810024019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 0x5635792f0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.562+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/3810024019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 0x5635792edef0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.562+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/3810024019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 0x5635792e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.562+0000 7f82f7c0e740 1 -- 172.21.15.69:0/3810024019 >> 172.21.15.69:0/3810024019 conn(0x5635792df660 msgr2=0x5635792dfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:24.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.565+0000 7f82f7c0e740 1 -- 172.21.15.69:0/3810024019 shutdown_connections 2024-03-19T01:18:24.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.565+0000 7f82f7c0e740 1 -- 172.21.15.69:0/3810024019 wait complete. 2024-03-19T01:18:24.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.570+0000 7f82f7c0e740 1 -- start start 2024-03-19T01:18:24.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.570+0000 7f82f7c0e740 1 Processor -- start 2024-03-19T01:18:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.571+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 0x5635792eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.571+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 0x5635792eb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.571+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792ee430 0x5635792ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.571+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5635792f1270 con 0x5635792e5850 2024-03-19T01:18:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.571+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563579204fc0 con 0x5635792ee430 2024-03-19T01:18:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.571+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5635792f13f0 con 0x5635792e4f30 2024-03-19T01:18:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.571+0000 7f82f7c0e740 10 client.0 _ll_register_callbacks cb 0x563579204990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.572+0000 7f82ed400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 0x5635792eb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.572+0000 7f82ed400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 0x5635792eb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41614/0 (socket says 172.21.15.69:41614) 2024-03-19T01:18:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.572+0000 7f82ed400640 1 -- 172.21.15.69:0/387608850 learned_addr learned my addr 172.21.15.69:0/387608850 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:24.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.573+0000 7f82ee800640 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792ee430 0x5635792ebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.574+0000 7f82ede00640 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 0x5635792eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.574+0000 7f82ede00640 1 -- 172.21.15.69:0/387608850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792ee430 msgr2=0x5635792ebeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:24.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.574+0000 7f82ede00640 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792ee430 0x5635792ebeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.574+0000 7f82ede00640 1 -- 172.21.15.69:0/387608850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 msgr2=0x5635792eb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:24.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.574+0000 7f82ede00640 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 0x5635792eb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.574+0000 7f82ede00640 1 -- 172.21.15.69:0/387608850 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563578f5d3f0 con 0x5635792e4f30 2024-03-19T01:18:24.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.580+0000 7f82ede00640 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 0x5635792eb430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f82e00ed7a0 tx=0x7f82e0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c099612e2926f16 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:24.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.580+0000 7f82cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:18:24.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.581+0000 7f82cfe00640 1 -- 172.21.15.69:0/387608850 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e00d5060 con 0x5635792e4f30 2024-03-19T01:18:24.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.581+0000 7f82cfe00640 1 -- 172.21.15.69:0/387608850 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f82e00b8da0 con 0x5635792e4f30 2024-03-19T01:18:24.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.582+0000 7f82f7c0e740 1 -- 172.21.15.69:0/387608850 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82dc0035f0 con 0x5635792e4f30 2024-03-19T01:18:24.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.584+0000 7f82cfe00640 1 -- 172.21.15.69:0/387608850 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f82e00da050 con 0x5635792e4f30 2024-03-19T01:18:24.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.584+0000 7f82f7c0e740 10 client.27346 fetch_fsmap learned FSMap version 48 2024-03-19T01:18:24.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.584+0000 7f82f7c0e740 1 -- 172.21.15.69:0/387608850 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563578f1d0f0 con 0x5635792e4f30 2024-03-19T01:18:24.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.584+0000 7f82cfe00640 1 -- 172.21.15.69:0/387608850 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82dc0035f0 con 0x5635792e4f30 2024-03-19T01:18:24.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.585+0000 7f82cfe00640 1 -- 172.21.15.69:0/387608850 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 48) ==== 8410+0+0 (secure 0 0 0) 0x7f82e003c070 con 0x5635792e4f30 2024-03-19T01:18:24.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.586+0000 7f82f7c0e740 10 client.27346 fetch_fsmap finished waiting for FSMap version 48 2024-03-19T01:18:24.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.586+0000 7f82f7c0e740 10 client.27346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:18:24.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.586+0000 7f82f7c0e740 20 client.27346 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:24.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.586+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5635792fe880 0x563579300cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.586+0000 7f82f7c0e740 4 client.27346 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:24.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.586+0000 7f82f7c0e740 1 -- 172.21.15.69:0/387608850 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635791ea610 con 0x5635792fe880 2024-03-19T01:18:24.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.587+0000 7f82ed400640 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5635792fe880 0x563579300cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.588+0000 7f82ed400640 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5635792fe880 0x563579300cc0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8454b8099b47ca server_cookie=46ecd432b10c7d72 in_seq=0 out_seq=0 2024-03-19T01:18:24.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.588+0000 7f82cfe00640 10 client.27346 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.590+0000 7f82cfe00640 1 -- 172.21.15.69:0/387608850 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5635791ea610 con 0x5635792fe880 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.590+0000 7f82cfe00640 10 client.27346 handle_command_reply: tid=0 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.590+0000 7f82f7c0e740 1 client.27346 shutdown 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.591+0000 7f82f7c0e740 1 -- 172.21.15.69:0/387608850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 msgr2=0x5635792eb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.591+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 0x5635792eb430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f82e00ed7a0 tx=0x7f82e0005f90 comp rx=0 tx=0).stop 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.591+0000 7f82f7c0e740 1 -- 172.21.15.69:0/387608850 shutdown_connections 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.591+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5635792fe880 0x563579300cc0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.591+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792ee430 0x5635792ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.591+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 0x5635792eb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.591+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/387608850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792e4f30 0x5635792eb430 secure :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f82e00ed7a0 tx=0x7f82e0005f90 comp rx=0 tx=0).stop 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.591+0000 7f82f7c0e740 1 -- 172.21.15.69:0/387608850 >> 172.21.15.69:0/387608850 conn(0x5635792df660 msgr2=0x5635792e13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.591+0000 7f82cfe00640 0 client.27346 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:24.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.591+0000 7f82f7c0e740 1 -- 172.21.15.69:0/387608850 shutdown_connections 2024-03-19T01:18:24.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.592+0000 7f82f7c0e740 1 -- 172.21.15.69:0/387608850 wait complete. 2024-03-19T01:18:24.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.593+0000 7f82f7c0e740 20 client.27346 trim_cache size 0 max 16384 2024-03-19T01:18:24.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.673+0000 7f82f7c0e740 1 Processor -- start 2024-03-19T01:18:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.673+0000 7f82f7c0e740 1 -- start start 2024-03-19T01:18:24.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.675+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792df660 0x5635792fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.675+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 0x5635792ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.675+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 0x563579306590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.675+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01b5310 con 0x5635792ee430 2024-03-19T01:18:24.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.675+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01b6fd0 con 0x5635792e5850 2024-03-19T01:18:24.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.675+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01b59e0 con 0x5635792df660 2024-03-19T01:18:24.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.675+0000 7f82ed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 0x5635792ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.675+0000 7f82ed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 0x5635792ebd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:35266/0 (socket says 172.21.15.69:35266) 2024-03-19T01:18:24.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.675+0000 7f82ed400640 1 -- 172.21.15.69:0/3079154579 learned_addr learned my addr 172.21.15.69:0/3079154579 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:24.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.676+0000 7f82ed400640 1 -- 172.21.15.69:0/3079154579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792df660 msgr2=0x5635792fa040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:18:24.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.676+0000 7f82ede00640 1 --2- 172.21.15.69:0/3079154579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792df660 0x5635792fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.677+0000 7f82ed400640 1 --2- 172.21.15.69:0/3079154579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792df660 0x5635792fa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.677+0000 7f82ed400640 1 -- 172.21.15.69:0/3079154579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 msgr2=0x563579306590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.678+0000 7f82ee800640 1 --2- 172.21.15.69:0/3079154579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 0x563579306590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.678+0000 7f82ed400640 1 --2- 172.21.15.69:0/3079154579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 0x563579306590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.678+0000 7f82ed400640 1 -- 172.21.15.69:0/3079154579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635792e6200 con 0x5635792e5850 2024-03-19T01:18:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.678+0000 7f82ede00640 1 --2- 172.21.15.69:0/3079154579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792df660 0x5635792fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.678+0000 7f82ee800640 1 --2- 172.21.15.69:0/3079154579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 0x563579306590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:24.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.681+0000 7f82ed400640 1 --2- 172.21.15.69:0/3079154579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 0x5635792ebd00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f82dc01aed0 tx=0x7f82dc01af70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c4f6d313404e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:24.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.686+0000 7f82daa00640 1 -- 172.21.15.69:0/3079154579 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc02f040 con 0x5635792e5850 2024-03-19T01:18:24.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.686+0000 7f82daa00640 1 -- 172.21.15.69:0/3079154579 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82dc003de0 con 0x5635792e5850 2024-03-19T01:18:24.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.687+0000 7f82daa00640 1 -- 172.21.15.69:0/3079154579 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc03b2a0 con 0x5635792e5850 2024-03-19T01:18:24.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.691+0000 7f82f7c0e740 1 -- 172.21.15.69:0/3079154579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 msgr2=0x5635792ebd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:24.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.691+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/3079154579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 0x5635792ebd00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f82dc01aed0 tx=0x7f82dc01af70 comp rx=0 tx=0).stop 2024-03-19T01:18:24.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.691+0000 7f82f7c0e740 1 -- 172.21.15.69:0/3079154579 shutdown_connections 2024-03-19T01:18:24.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.691+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/3079154579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792ee430 0x563579306590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.691+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/3079154579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792e5850 0x5635792ebd00 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.691+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/3079154579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792df660 0x5635792fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.691+0000 7f82f7c0e740 1 -- 172.21.15.69:0/3079154579 >> 172.21.15.69:0/3079154579 conn(0x5635792ea070 msgr2=0x5635792dde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:24.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.691+0000 7f82f7c0e740 1 -- 172.21.15.69:0/3079154579 shutdown_connections 2024-03-19T01:18:24.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.691+0000 7f82f7c0e740 1 -- 172.21.15.69:0/3079154579 wait complete. 2024-03-19T01:18:24.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.692+0000 7f82f7c0e740 1 -- start start 2024-03-19T01:18:24.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.692+0000 7f82f7c0e740 1 Processor -- start 2024-03-19T01:18:24.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.696+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792df660 0x5635792f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.696+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 0x5635792f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.696+0000 7f82f7c0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792ee430 0x5635792f2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.696+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01b6840 con 0x5635792e5850 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.696+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01b6170 con 0x5635792ee430 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.696+0000 7f82f7c0e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01b3290 con 0x5635792df660 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.696+0000 7f82f7c0e740 10 client.0 _ll_register_callbacks cb 0x7f82e00bc500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.697+0000 7f82ed400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 0x5635792f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.697+0000 7f82ed400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 0x5635792f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41636/0 (socket says 172.21.15.69:41636) 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.697+0000 7f82ed400640 1 -- 172.21.15.69:0/2619876282 learned_addr learned my addr 172.21.15.69:0/2619876282 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.698+0000 7f82ed400640 1 -- 172.21.15.69:0/2619876282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792df660 msgr2=0x5635792f6a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.698+0000 7f82ed400640 1 --2- 172.21.15.69:0/2619876282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792df660 0x5635792f6a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.698+0000 7f82ed400640 1 -- 172.21.15.69:0/2619876282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792ee430 msgr2=0x5635792f2300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.698+0000 7f82ed400640 1 --2- 172.21.15.69:0/2619876282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792ee430 0x5635792f2300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.698+0000 7f82ed400640 1 -- 172.21.15.69:0/2619876282 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635792f2b30 con 0x5635792e5850 2024-03-19T01:18:24.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.699+0000 7f82ed400640 1 --2- 172.21.15.69:0/2619876282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 0x5635792f1d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f82dc005650 tx=0x7f82dc01a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14157a9acc2dd19 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:24.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.699+0000 7f82cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:18:24.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.703+0000 7f82cfe00640 1 -- 172.21.15.69:0/2619876282 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc02f070 con 0x5635792e5850 2024-03-19T01:18:24.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.703+0000 7f82cfe00640 1 -- 172.21.15.69:0/2619876282 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f82dc027040 con 0x5635792e5850 2024-03-19T01:18:24.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.704+0000 7f82f7c0e740 1 -- 172.21.15.69:0/2619876282 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82dc003de0 con 0x5635792e5850 2024-03-19T01:18:24.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.707+0000 7f82cfe00640 1 -- 172.21.15.69:0/2619876282 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f82dc003a00 con 0x5635792e5850 2024-03-19T01:18:24.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.707+0000 7f82f7c0e740 10 client.17076 fetch_fsmap learned FSMap version 48 2024-03-19T01:18:24.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.707+0000 7f82f7c0e740 1 -- 172.21.15.69:0/2619876282 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563579364c20 con 0x5635792e5850 2024-03-19T01:18:24.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.708+0000 7f82cfe00640 1 -- 172.21.15.69:0/2619876282 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82dc003de0 con 0x5635792e5850 2024-03-19T01:18:24.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.708+0000 7f82cfe00640 1 -- 172.21.15.69:0/2619876282 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 48) ==== 8410+0+0 (secure 0 0 0) 0x7f82dc038440 con 0x5635792e5850 2024-03-19T01:18:24.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.709+0000 7f82f7c0e740 10 client.17076 fetch_fsmap finished waiting for FSMap version 48 2024-03-19T01:18:24.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.709+0000 7f82f7c0e740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:18:24.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.709+0000 7f82f7c0e740 20 client.17076 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:24.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.709+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/2619876282 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5635793650d0 0x5635793674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:24.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.709+0000 7f82f7c0e740 4 client.17076 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:18:24.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.709+0000 7f82f7c0e740 1 -- 172.21.15.69:0/2619876282 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563579256800 con 0x5635793650d0 2024-03-19T01:18:24.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.710+0000 7f82ede00640 1 --2- 172.21.15.69:0/2619876282 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5635793650d0 0x5635793674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:24.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.710+0000 7f82ede00640 1 --2- 172.21.15.69:0/2619876282 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5635793650d0 0x5635793674f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476da0bd3ef27d7f server_cookie=94720c01451ea0bf in_seq=0 out_seq=0 2024-03-19T01:18:24.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.710+0000 7f82cfe00640 10 client.17076 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:24.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.711+0000 7f82cfe00640 1 -- 172.21.15.69:0/2619876282 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x563579256800 con 0x5635793650d0 2024-03-19T01:18:24.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.711+0000 7f82cfe00640 10 client.17076 handle_command_reply: tid=0 2024-03-19T01:18:24.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.711+0000 7f82f7c0e740 1 client.17076 shutdown 2024-03-19T01:18:24.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.718+0000 7f82f7c0e740 1 -- 172.21.15.69:0/2619876282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 msgr2=0x5635792f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:24.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.718+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/2619876282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 0x5635792f1d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f82dc005650 tx=0x7f82dc01a2f0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.721+0000 7f82f7c0e740 1 -- 172.21.15.69:0/2619876282 shutdown_connections 2024-03-19T01:18:24.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.721+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/2619876282 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5635793650d0 0x5635793674f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.721+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/2619876282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635792ee430 0x5635792f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.721+0000 7f82cfe00640 0 client.17076 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:24.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.721+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/2619876282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5635792e5850 0x5635792f1d40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.721+0000 7f82f7c0e740 1 --2- 172.21.15.69:0/2619876282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5635792df660 0x5635792f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.721+0000 7f82f7c0e740 1 -- 172.21.15.69:0/2619876282 >> 172.21.15.69:0/2619876282 conn(0x5635792ea070 msgr2=0x5635792dde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:24.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.723+0000 7f82f7c0e740 1 -- 172.21.15.69:0/2619876282 shutdown_connections 2024-03-19T01:18:24.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.723+0000 7f82f7c0e740 1 -- 172.21.15.69:0/2619876282 wait complete. 2024-03-19T01:18:24.728 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:18:24.728 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (10716 inodes in the stack)", 2024-03-19T01:18:24.728 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:18:24.728 INFO:teuthology.orchestra.run.smithi069.stdout: "7213bbdf-dd79-4eae-8bf8-c9193b2bf430": { 2024-03-19T01:18:24.728 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:18:24.728 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "7213bbdf-dd79-4eae-8bf8-c9193b2bf430", 2024-03-19T01:18:24.728 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:18:24.728 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:18:24.729 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:18:24.729 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:18:24.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.724+0000 7f82f7c0e740 20 client.17076 trim_cache size 0 max 16384 2024-03-19T01:18:24.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.735+0000 7f82ef200640 1 -- 172.21.15.69:0/195880678 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82d40a2af0 msgr2=0x7f82d40a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:24.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.735+0000 7f82ef200640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82d40a2af0 0x7f82d40a4fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82e80118e0 tx=0x7f82e8007eb0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.735+0000 7f82ef200640 1 -- 172.21.15.69:0/195880678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f01088b0 msgr2=0x7f82f011a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:24.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.735+0000 7f82ef200640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f01088b0 0x7f82f011a7a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f82e002c120 tx=0x7f82e0005730 comp rx=0 tx=0).stop 2024-03-19T01:18:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.736+0000 7f82ef200640 1 -- 172.21.15.69:0/195880678 shutdown_connections 2024-03-19T01:18:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.736+0000 7f82ef200640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f011b000 0x7f82f0116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.736+0000 7f82ef200640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f010fe40 0x7f82f0116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.736+0000 7f82ef200640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f82d40a2af0 0x7f82d40a4fb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.736+0000 7f82ef200640 1 --2- 172.21.15.69:0/195880678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f01088b0 0x7f82f011a7a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.736+0000 7f82ef200640 1 -- 172.21.15.69:0/195880678 >> 172.21.15.69:0/195880678 conn(0x7f82f006c770 msgr2=0x7f82f011b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.737+0000 7f82ef200640 1 -- 172.21.15.69:0/195880678 shutdown_connections 2024-03-19T01:18:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:24.739+0000 7f82ef200640 1 -- 172.21.15.69:0/195880678 wait complete. 2024-03-19T01:18:24.799 INFO:tasks.cephfs.filesystem:scrub status for tag:7213bbdf-dd79-4eae-8bf8-c9193b2bf430 - {'path': '/', 'tag': '7213bbdf-dd79-4eae-8bf8-c9193b2bf430', 'options': 'recursive,force'} 2024-03-19T01:18:25.025 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/compress.o 2024-03-19T01:18:25.192 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/amd/iommu.o 2024-03-19T01:18:25.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:18:25.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:25.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.330 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/iomap/seek.o 2024-03-19T01:18:25.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:25.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:18:25.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:25.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:25.759 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/keys/keyctl_pkey.o 2024-03-19T01:18:25.957 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/locking/qrwlock.o 2024-03-19T01:18:26.159 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/algapi.o 2024-03-19T01:18:26.211 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/power/console.o 2024-03-19T01:18:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: pgmap v239: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 737 KiB/s wr, 130 op/s 2024-03-19T01:18:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:26.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: pgmap v239: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 737 KiB/s wr, 130 op/s 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:26.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: pgmap v239: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 737 KiB/s wr, 130 op/s 2024-03-19T01:18:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:26.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:26.761 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/events/amd/built-in.a 2024-03-19T01:18:26.763 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/iomap/swapfile.o 2024-03-19T01:18:26.805 INFO:tasks.workunit.client.0.smithi069.stdout: AR security/keys/built-in.a 2024-03-19T01:18:26.910 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/core.o 2024-03-19T01:18:26.911 INFO:tasks.workunit.client.0.smithi069.stdout: GEN security/selinux/flask.h security/selinux/av_permissions.h 2024-03-19T01:18:26.956 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/avc.o 2024-03-19T01:18:26.985 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/locking/built-in.a 2024-03-19T01:18:27.060 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/shmem.o 2024-03-19T01:18:27.192 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/quota/dquot.o 2024-03-19T01:18:27.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:27.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.521 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/power/process.o 2024-03-19T01:18:27.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:27.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:27.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:27.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.027 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/scatterwalk.o 2024-03-19T01:18:28.211 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/iomap/built-in.a 2024-03-19T01:18:28.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: pgmap v240: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 737 KiB/s wr, 137 op/s 2024-03-19T01:18:28.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:28.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.514 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/partitions/core.o 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: pgmap v240: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 737 KiB/s wr, 137 op/s 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:28.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: pgmap v240: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 737 KiB/s wr, 137 op/s 2024-03-19T01:18:28.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:28.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:29.087 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/sched/built-in.a 2024-03-19T01:18:29.235 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/proc.o 2024-03-19T01:18:29.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:29.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:29.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:29.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:29.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:29.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:29.521 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/power/suspend.o 2024-03-19T01:18:29.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:29.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:29.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:29.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:29.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:29.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:29.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:29.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:29.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:29.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:29.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:29.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:29.594 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/hooks.o 2024-03-19T01:18:29.816 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/realmode/init.o 2024-03-19T01:18:29.918 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/quota/quota_v2.o 2024-03-19T01:18:29.940 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/partitions/msdos.o 2024-03-19T01:18:30.410 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/util.o 2024-03-19T01:18:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: pgmap v241: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 808 KiB/s wr, 186 op/s 2024-03-19T01:18:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:30.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: pgmap v241: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 808 KiB/s wr, 186 op/s 2024-03-19T01:18:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:30.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: pgmap v241: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 808 KiB/s wr, 186 op/s 2024-03-19T01:18:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:30.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:30.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:30.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:30.921 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/quota/quota_tree.o 2024-03-19T01:18:30.958 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/aead.o 2024-03-19T01:18:31.111 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/realmode/rm/header.o 2024-03-19T01:18:31.312 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/realmode/rm/trampoline_64.o 2024-03-19T01:18:31.348 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/bts.o 2024-03-19T01:18:31.353 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/power/hibernate.o 2024-03-19T01:18:31.362 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/partitions/efi.o 2024-03-19T01:18:31.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:31.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:31.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:31.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:31.564 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/realmode/rm/stack.o 2024-03-19T01:18:31.740 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/realmode/rm/reboot.o 2024-03-19T01:18:31.900 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/realmode/rm/wakeup_asm.o 2024-03-19T01:18:32.082 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/realmode/rm/wakemain.o 2024-03-19T01:18:32.316 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/realmode/rm/video-mode.o 2024-03-19T01:18:32.404 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/quota/quota.o 2024-03-19T01:18:32.407 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mmzone.o 2024-03-19T01:18:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: pgmap v242: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 449 KiB/s wr, 144 op/s 2024-03-19T01:18:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:32.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.529 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/geniv.o 2024-03-19T01:18:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: pgmap v242: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 449 KiB/s wr, 144 op/s 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:32.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: pgmap v242: 129 pgs: 129 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 449 KiB/s wr, 144 op/s 2024-03-19T01:18:32.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:32.604 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/realmode/rm/copy.o 2024-03-19T01:18:32.758 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/realmode/rm/bioscall.o 2024-03-19T01:18:32.885 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/realmode/rm/regs.o 2024-03-19T01:18:32.917 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/ds.o 2024-03-19T01:18:33.016 INFO:tasks.workunit.client.0.smithi069.stdout: AR block/partitions/built-in.a 2024-03-19T01:18:33.130 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/realmode/rm/video-vga.o 2024-03-19T01:18:33.153 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/bdev.o 2024-03-19T01:18:33.374 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/power/snapshot.o 2024-03-19T01:18:33.445 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/realmode/rm/video-vesa.o 2024-03-19T01:18:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:33.643 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/vmstat.o 2024-03-19T01:18:33.740 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/realmode/rm/video-bios.o 2024-03-19T01:18:34.043 INFO:tasks.workunit.client.0.smithi069.stdout: PASYMS arch/x86/realmode/rm/pasyms.h 2024-03-19T01:18:34.088 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/skcipher.o 2024-03-19T01:18:34.182 INFO:tasks.workunit.client.0.smithi069.stdout: LDS arch/x86/realmode/rm/realmode.lds 2024-03-19T01:18:34.316 INFO:tasks.workunit.client.0.smithi069.stdout: LD arch/x86/realmode/rm/realmode.elf 2024-03-19T01:18:34.367 INFO:tasks.workunit.client.0.smithi069.stdout: RELOCS arch/x86/realmode/rm/realmode.relocs 2024-03-19T01:18:34.398 INFO:tasks.workunit.client.0.smithi069.stdout: OBJCOPY arch/x86/realmode/rm/realmode.bin 2024-03-19T01:18:34.499 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/quota/kqid.o 2024-03-19T01:18:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: pgmap v243: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 622 KiB/s wr, 589 op/s 2024-03-19T01:18:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.514 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/realmode/rmpiggy.o 2024-03-19T01:18:34.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: pgmap v243: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 622 KiB/s wr, 589 op/s 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: pgmap v243: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 622 KiB/s wr, 589 op/s 2024-03-19T01:18:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:34.697 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/realmode/built-in.a 2024-03-19T01:18:34.957 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/seqiv.o 2024-03-19T01:18:35.228 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/fops.o 2024-03-19T01:18:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: pgmap v244: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 647 KiB/s wr, 721 op/s 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:35.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: pgmap v244: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 647 KiB/s wr, 721 op/s 2024-03-19T01:18:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:35.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: pgmap v244: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 647 KiB/s wr, 721 op/s 2024-03-19T01:18:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:35.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:35.579 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/quota/netlink.o 2024-03-19T01:18:35.805 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/knc.o 2024-03-19T01:18:35.817 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/selinuxfs.o 2024-03-19T01:18:35.825 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/backing-dev.o 2024-03-19T01:18:35.993 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/power/swap.o 2024-03-19T01:18:36.014 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/integrity/iint.o 2024-03-19T01:18:36.254 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/echainiv.o 2024-03-19T01:18:36.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:36.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:36.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:36.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:36.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:36.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:36.754 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/quota/built-in.a 2024-03-19T01:18:37.029 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/integrity/integrity_audit.o 2024-03-19T01:18:37.047 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/bio.o 2024-03-19T01:18:37.069 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/lbr.o 2024-03-19T01:18:37.313 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/task_mmu.o 2024-03-19T01:18:37.318 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mm_init.o 2024-03-19T01:18:37.318 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/ahash.o 2024-03-19T01:18:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: pgmap v245: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 731 KiB/s wr, 1.06k op/s 2024-03-19T01:18:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: pgmap v245: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 731 KiB/s wr, 1.06k op/s 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:37.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: pgmap v245: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 731 KiB/s wr, 1.06k op/s 2024-03-19T01:18:37.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:37.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.055 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/power/user.o 2024-03-19T01:18:38.156 INFO:tasks.workunit.client.0.smithi069.stdout: AR security/integrity/built-in.a 2024-03-19T01:18:38.253 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/elevator.o 2024-03-19T01:18:38.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:38.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:38.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:38.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:38.731 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/shash.o 2024-03-19T01:18:38.750 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/netlink.o 2024-03-19T01:18:39.235 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/p4.o 2024-03-19T01:18:39.468 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/power/poweroff.o 2024-03-19T01:18:39.498 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/percpu.o 2024-03-19T01:18:39.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:39.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:39 smithi086 ceph-mon[66829]: pgmap v246: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 KiB/s wr, 1.71k op/s 2024-03-19T01:18:39.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:39 smithi069 ceph-mon[31727]: pgmap v246: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 KiB/s wr, 1.71k op/s 2024-03-19T01:18:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:39 smithi163 ceph-mon[67428]: pgmap v246: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 KiB/s wr, 1.71k op/s 2024-03-19T01:18:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:39.768 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/inode.o 2024-03-19T01:18:39.873 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/nlmsgtab.o 2024-03-19T01:18:39.957 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/power/built-in.a 2024-03-19T01:18:39.965 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/io_uring.o 2024-03-19T01:18:39.965 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-core.o 2024-03-19T01:18:40.048 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/printk/printk.o 2024-03-19T01:18:40.316 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/akcipher.o 2024-03-19T01:18:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:40.738 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/p6.o 2024-03-19T01:18:41.504 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/root.o 2024-03-19T01:18:41.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: pgmap v247: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 854 KiB/s wr, 1.69k op/s 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: pgmap v247: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 854 KiB/s wr, 1.69k op/s 2024-03-19T01:18:41.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: pgmap v247: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 854 KiB/s wr, 1.69k op/s 2024-03-19T01:18:41.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:41.737 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/netif.o 2024-03-19T01:18:41.913 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/sig.o 2024-03-19T01:18:42.311 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/pt.o 2024-03-19T01:18:42.452 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/base.o 2024-03-19T01:18:42.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:42.663 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/printk/printk_safe.o 2024-03-19T01:18:42.668 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/slab_common.o 2024-03-19T01:18:42.862 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-sysfs.o 2024-03-19T01:18:43.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:43.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:43.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:43.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:43.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:43 smithi069 ceph-mon[31727]: pgmap v248: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 938 KiB/s wr, 1.72k op/s 2024-03-19T01:18:43.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:43.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:43 smithi163 ceph-mon[67428]: pgmap v248: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 938 KiB/s wr, 1.72k op/s 2024-03-19T01:18:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:43.569 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/netnode.o 2024-03-19T01:18:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:43 smithi086 ceph-mon[66829]: pgmap v248: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 938 KiB/s wr, 1.72k op/s 2024-03-19T01:18:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:43.805 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/kpp.o 2024-03-19T01:18:43.907 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/printk/printk_ringbuffer.o 2024-03-19T01:18:44.170 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/uncore.o 2024-03-19T01:18:44.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:44.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:44.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:44.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:18:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.541 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-flush.o 2024-03-19T01:18:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:18:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.675 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/printk/sysctl.o 2024-03-19T01:18:44.746 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/generic.o 2024-03-19T01:18:44.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:44.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:44.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:18:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:18:44.804 INFO:tasks.workunit.client.0.smithi069.stdout: ASN.1 crypto/rsapubkey.asn1.[ch] 2024-03-19T01:18:44.820 INFO:tasks.workunit.client.0.smithi069.stdout: ASN.1 crypto/rsaprivkey.asn1.[ch] 2024-03-19T01:18:44.885 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/rsa.o 2024-03-19T01:18:44.939 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/xattr.o 2024-03-19T01:18:44.986 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/compaction.o 2024-03-19T01:18:45.224 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/printk/built-in.a 2024-03-19T01:18:45.315 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/irqdesc.o 2024-03-19T01:18:45.317 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/netport.o 2024-03-19T01:18:45.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:45 smithi069 ceph-mon[31727]: pgmap v249: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 698 KiB/s wr, 1.27k op/s 2024-03-19T01:18:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:45 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:45.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:45 smithi163 ceph-mon[67428]: pgmap v249: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 698 KiB/s wr, 1.27k op/s 2024-03-19T01:18:45.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:45 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:45.565 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-settings.o 2024-03-19T01:18:45.734 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/array.o 2024-03-19T01:18:45.739 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/uncore_nhmex.o 2024-03-19T01:18:45.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:45 smithi086 ceph-mon[66829]: pgmap v249: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 698 KiB/s wr, 1.27k op/s 2024-03-19T01:18:45.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:45 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:45.895 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/rsa_helper.o 2024-03-19T01:18:46.129 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/rsa-pkcs1pad.o 2024-03-19T01:18:46.156 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/nop.o 2024-03-19T01:18:46.199 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/handle.o 2024-03-19T01:18:46.667 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-ioc.o 2024-03-19T01:18:46.793 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/status.o 2024-03-19T01:18:47.050 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/manage.o 2024-03-19T01:18:47.081 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/acompress.o 2024-03-19T01:18:47.084 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/fd.o 2024-03-19T01:18:47.091 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/fs.o 2024-03-19T01:18:47.262 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/uncore_snb.o 2024-03-19T01:18:47.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:47 smithi069 ceph-mon[31727]: pgmap v250: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 592 KiB/s wr, 1.13k op/s 2024-03-19T01:18:47.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:47 smithi163 ceph-mon[67428]: pgmap v250: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 592 KiB/s wr, 1.13k op/s 2024-03-19T01:18:47.715 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-map.o 2024-03-19T01:18:47.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:47 smithi086 ceph-mon[66829]: pgmap v250: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 592 KiB/s wr, 1.13k op/s 2024-03-19T01:18:48.009 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/show_mem.o 2024-03-19T01:18:48.292 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/scompress.o 2024-03-19T01:18:48.450 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/proc_tty.o 2024-03-19T01:18:48.578 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/ss/ebitmap.o 2024-03-19T01:18:48.754 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/splice.o 2024-03-19T01:18:49.077 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/uncore_snbep.o 2024-03-19T01:18:49.077 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/spurious.o 2024-03-19T01:18:49.411 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/algboss.o 2024-03-19T01:18:49.509 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/interval_tree.o 2024-03-19T01:18:49.520 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-merge.o 2024-03-19T01:18:49.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:49 smithi163 ceph-mon[67428]: pgmap v251: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 415 KiB/s wr, 791 op/s 2024-03-19T01:18:49.624 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/cmdline.o 2024-03-19T01:18:49.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:49 smithi086 ceph-mon[66829]: pgmap v251: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 415 KiB/s wr, 791 op/s 2024-03-19T01:18:49.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:49 smithi069 ceph-mon[31727]: pgmap v251: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 415 KiB/s wr, 791 op/s 2024-03-19T01:18:49.836 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/sync.o 2024-03-19T01:18:49.887 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/resend.o 2024-03-19T01:18:50.321 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/ss/hashtab.o 2024-03-19T01:18:50.446 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/testmgr.o 2024-03-19T01:18:50.559 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/chip.o 2024-03-19T01:18:50.590 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/list_lru.o 2024-03-19T01:18:50.810 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/advise.o 2024-03-19T01:18:50.849 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/consoles.o 2024-03-19T01:18:50.883 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/ss/symtab.o 2024-03-19T01:18:51.138 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-timeout.o 2024-03-19T01:18:51.323 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/cmac.o 2024-03-19T01:18:51.398 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/ss/sidtab.o 2024-03-19T01:18:51.442 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/uncore_discovery.o 2024-03-19T01:18:51.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:51 smithi163 ceph-mon[67428]: pgmap v252: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 303 KiB/s wr, 137 op/s 2024-03-19T01:18:51.620 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/cpuinfo.o 2024-03-19T01:18:51.712 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/workingset.o 2024-03-19T01:18:51.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:51 smithi086 ceph-mon[66829]: pgmap v252: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 303 KiB/s wr, 137 op/s 2024-03-19T01:18:51.756 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/filetable.o 2024-03-19T01:18:51.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:51 smithi069 ceph-mon[31727]: pgmap v252: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 303 KiB/s wr, 137 op/s 2024-03-19T01:18:52.044 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/dummychip.o 2024-03-19T01:18:52.091 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-lib.o 2024-03-19T01:18:52.265 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/hmac.o 2024-03-19T01:18:52.623 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/devices.o 2024-03-19T01:18:52.668 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/intel/cstate.o 2024-03-19T01:18:52.685 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/devres.o 2024-03-19T01:18:52.841 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/debug.o 2024-03-19T01:18:52.864 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/ss/avtab.o 2024-03-19T01:18:53.131 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-mq.o 2024-03-19T01:18:53.193 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/crypto_null.o 2024-03-19T01:18:53.334 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/openclose.o 2024-03-19T01:18:53.343 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/autoprobe.o 2024-03-19T01:18:53.396 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/interrupts.o 2024-03-19T01:18:53.557 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:53 smithi086 ceph-mon[66829]: pgmap v253: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 773 KiB/s wr, 133 op/s 2024-03-19T01:18:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:53 smithi163 ceph-mon[67428]: pgmap v253: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 773 KiB/s wr, 133 op/s 2024-03-19T01:18:53.684 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/events/intel/built-in.a 2024-03-19T01:18:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:53 smithi069 ceph-mon[31727]: pgmap v253: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 773 KiB/s wr, 133 op/s 2024-03-19T01:18:53.802 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/zhaoxin/core.o 2024-03-19T01:18:53.931 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/gup.o 2024-03-19T01:18:53.950 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/md5.o 2024-03-19T01:18:54.053 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/irqdomain.o 2024-03-19T01:18:54.272 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/loadavg.o 2024-03-19T01:18:54.434 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/ss/policydb.o 2024-03-19T01:18:54.694 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/sha256_generic.o 2024-03-19T01:18:54.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:54 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:18:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:54 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:18:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:54 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:18:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:54 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:18:54.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:54 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:18:54.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:54 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:18:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:54 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:18:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:54 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:18:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:54 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:18:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:54 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:18:54.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:18:54.800 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:18:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:54 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:18:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:54 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:18:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:54 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:18:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:54 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:18:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:54 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:18:54.896 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/events/zhaoxin/built-in.a 2024-03-19T01:18:54.977 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/core.o 2024-03-19T01:18:55.175 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/meminfo.o 2024-03-19T01:18:55.198 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/uring_cmd.o 2024-03-19T01:18:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.526+0000 7fafb8200640 1 -- 172.21.15.69:0/936965787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafb0114970 msgr2=0x7fafb0116d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.526+0000 7fafb8200640 1 --2- 172.21.15.69:0/936965787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafb0114970 0x7fafb0116d60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009900 tx=0x7faf9c02bc60 comp rx=0 tx=0).stop 2024-03-19T01:18:55.532 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/sha512_generic.o 2024-03-19T01:18:55.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:55 smithi069 ceph-mon[31727]: pgmap v254: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 937 KiB/s wr, 93 op/s 2024-03-19T01:18:55.547 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/proc.o 2024-03-19T01:18:55.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.562+0000 7fafb8200640 1 -- 172.21.15.69:0/936965787 shutdown_connections 2024-03-19T01:18:55.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.562+0000 7fafb8200640 1 --2- 172.21.15.69:0/936965787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafb0114970 0x7fafb0116d60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.562+0000 7fafb8200640 1 --2- 172.21.15.69:0/936965787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb0109320 0x7fafb0114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.562+0000 7fafb8200640 1 --2- 172.21.15.69:0/936965787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb0108a00 0x7fafb0108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.562+0000 7fafb8200640 1 -- 172.21.15.69:0/936965787 >> 172.21.15.69:0/936965787 conn(0x7fafb006c900 msgr2=0x7fafb006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:55.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.562+0000 7fafb8200640 1 -- 172.21.15.69:0/936965787 shutdown_connections 2024-03-19T01:18:55.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.562+0000 7fafb8200640 1 -- 172.21.15.69:0/936965787 wait complete. 2024-03-19T01:18:55.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.564+0000 7fafb8200640 1 Processor -- start 2024-03-19T01:18:55.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.564+0000 7fafb8200640 1 -- start start 2024-03-19T01:18:55.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.565+0000 7fafb8200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafb0108a00 0x7fafb010be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.565+0000 7fafb8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb0109320 0x7fafb010a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.565+0000 7fafb8200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb0114970 0x7fafb010a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.565+0000 7fafb8200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fafb011a420 con 0x7fafb0114970 2024-03-19T01:18:55.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.565+0000 7fafb8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafb011a2a0 con 0x7fafb0109320 2024-03-19T01:18:55.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.565+0000 7fafb8200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fafb011a5a0 con 0x7fafb0108a00 2024-03-19T01:18:55.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.565+0000 7fafb6200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb0114970 0x7fafb010a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.565+0000 7fafb6200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb0114970 0x7fafb010a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42860/0 (socket says 172.21.15.69:42860) 2024-03-19T01:18:55.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.565+0000 7fafb6200640 1 -- 172.21.15.69:0/925495998 learned_addr learned my addr 172.21.15.69:0/925495998 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:55.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.566+0000 7fafb4e00640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb0109320 0x7fafb010a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.566+0000 7fafb6200640 1 -- 172.21.15.69:0/925495998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafb0108a00 msgr2=0x7fafb010be90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:18:55.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.566+0000 7fafb6200640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafb0108a00 0x7fafb010be90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.566+0000 7fafb6200640 1 -- 172.21.15.69:0/925495998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb0109320 msgr2=0x7fafb010a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:55.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.566+0000 7fafb6200640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb0109320 0x7fafb010a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.566+0000 7fafb6200640 1 -- 172.21.15.69:0/925495998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb01a49c0 con 0x7fafb0114970 2024-03-19T01:18:55.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.566+0000 7fafb6200640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb0114970 0x7fafb010a8d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf9c006010 tx=0x7faf9c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:55.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.567+0000 7fafae000640 1 -- 172.21.15.69:0/925495998 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c041070 con 0x7fafb0114970 2024-03-19T01:18:55.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.567+0000 7fafae000640 1 -- 172.21.15.69:0/925495998 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf9c005410 con 0x7fafb0114970 2024-03-19T01:18:55.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.567+0000 7fafae000640 1 -- 172.21.15.69:0/925495998 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0348d0 con 0x7fafb0114970 2024-03-19T01:18:55.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.567+0000 7fafb8200640 1 -- 172.21.15.69:0/925495998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb01a4c50 con 0x7fafb0114970 2024-03-19T01:18:55.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.569+0000 7fafb8200640 1 -- 172.21.15.69:0/925495998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb01a5030 con 0x7fafb0114970 2024-03-19T01:18:55.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.576+0000 7fafae000640 1 -- 172.21.15.69:0/925495998 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7faf9c044070 con 0x7fafb0114970 2024-03-19T01:18:55.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.578+0000 7fafae000640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faf980a2a30 0x7faf980a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.579+0000 7fafbae61740 1 Processor -- start 2024-03-19T01:18:55.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.579+0000 7fafb5800640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faf980a2a30 0x7faf980a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.584+0000 7fafae000640 1 -- 172.21.15.69:0/925495998 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7faf9c02ca50 con 0x7fafb0114970 2024-03-19T01:18:55.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.584+0000 7fafb5800640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faf980a2a30 0x7faf980a4ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fafa800ac00 tx=0x7fafa8009250 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:55.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.581+0000 7fafbae61740 1 -- start start 2024-03-19T01:18:55.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.584+0000 7fafbae61740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.584+0000 7fafbae61740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1dbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.585+0000 7fafb5800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.585+0000 7fafb5800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:34454/0 (socket says 172.21.15.69:34454) 2024-03-19T01:18:55.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.585+0000 7fafb5800640 1 -- 172.21.15.69:0/1476433910 learned_addr learned my addr 172.21.15.69:0/1476433910 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:55.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.585+0000 7fafbae61740 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 0x55b2dc1de920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.585+0000 7fafb4e00640 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1dbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.585+0000 7fafbae61740 1 -- 172.21.15.69:0/1476433910 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b2dc1372c0 con 0x55b2dc1dc530 2024-03-19T01:18:55.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.585+0000 7fafbae61740 1 -- 172.21.15.69:0/1476433910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2dc0dfb60 con 0x55b2dc1d3760 2024-03-19T01:18:55.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.585+0000 7fafbae61740 1 -- 172.21.15.69:0/1476433910 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b2dc133360 con 0x55b2dc1d2e40 2024-03-19T01:18:55.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.585+0000 7fafb6200640 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 0x55b2dc1de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.586+0000 7fafb5800640 1 -- 172.21.15.69:0/1476433910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 msgr2=0x55b2dc1dbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:55.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.586+0000 7fafb5800640 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1dbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.586+0000 7fafb5800640 1 -- 172.21.15.69:0/1476433910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 msgr2=0x55b2dc1de920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:55.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.586+0000 7fafb5800640 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 0x55b2dc1de920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.586+0000 7fafb5800640 1 -- 172.21.15.69:0/1476433910 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2dbf157d0 con 0x55b2dc1d2e40 2024-03-19T01:18:55.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.586+0000 7fafb5800640 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d3220 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fafa80096c0 tx=0x7fafa800f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf524a706a17f62 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:55.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.587+0000 7fafb8200640 1 -- 172.21.15.69:0/1476433910 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8019830 con 0x55b2dc1d2e40 2024-03-19T01:18:55.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.587+0000 7fafb8200640 1 -- 172.21.15.69:0/1476433910 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa8021400 con 0x55b2dc1d2e40 2024-03-19T01:18:55.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.587+0000 7fafb8200640 1 -- 172.21.15.69:0/1476433910 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8017610 con 0x55b2dc1d2e40 2024-03-19T01:18:55.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.587+0000 7fafb6200640 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 0x55b2dc1de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:18:55.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.587+0000 7fafb4e00640 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1dbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:55.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.588+0000 7fafbae61740 1 -- 172.21.15.69:0/1476433910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 msgr2=0x55b2dc1d3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:55.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.588+0000 7fafbae61740 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d3220 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fafa80096c0 tx=0x7fafa800f000 comp rx=0 tx=0).stop 2024-03-19T01:18:55.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.591+0000 7fafbae61740 1 -- 172.21.15.69:0/1476433910 shutdown_connections 2024-03-19T01:18:55.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.591+0000 7fafbae61740 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 0x55b2dc1de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.591+0000 7fafbae61740 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1dbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.591+0000 7fafbae61740 1 --2- 172.21.15.69:0/1476433910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d3220 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.591+0000 7fafbae61740 1 -- 172.21.15.69:0/1476433910 >> 172.21.15.69:0/1476433910 conn(0x55b2dc1cd760 msgr2=0x55b2dc1cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:55.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.591+0000 7fafbae61740 1 -- 172.21.15.69:0/1476433910 shutdown_connections 2024-03-19T01:18:55.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.591+0000 7fafbae61740 1 -- 172.21.15.69:0/1476433910 wait complete. 2024-03-19T01:18:55.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.591+0000 7fafbae61740 1 -- start start 2024-03-19T01:18:55.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.592+0000 7fafbae61740 1 Processor -- start 2024-03-19T01:18:55.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.593+0000 7fafbae61740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.593+0000 7fafbae61740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d3760 0x55b2dc1d9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.593+0000 7fafbae61740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1dc530 0x55b2dc1da070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.593+0000 7fafb5800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.594+0000 7fafb5800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:34464/0 (socket says 172.21.15.69:34464) 2024-03-19T01:18:55.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.594+0000 7fafb4e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d3760 0x55b2dc1d9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.594+0000 7fafb4e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d3760 0x55b2dc1d9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42886/0 (socket says 172.21.15.69:42886) 2024-03-19T01:18:55.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.598+0000 7fafb5800640 1 -- 172.21.15.69:0/4195247064 learned_addr learned my addr 172.21.15.69:0/4195247064 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:55.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.598+0000 7fafbae61740 1 -- 172.21.15.69:0/4195247064 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b2dc1df040 con 0x55b2dc1d3760 2024-03-19T01:18:55.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.598+0000 7fafbae61740 1 -- 172.21.15.69:0/4195247064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2dc0f2fc0 con 0x55b2dc1dc530 2024-03-19T01:18:55.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.598+0000 7fafbae61740 1 -- 172.21.15.69:0/4195247064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b2dc1df1c0 con 0x55b2dc1d2e40 2024-03-19T01:18:55.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.598+0000 7fafbae61740 10 client.0 _ll_register_callbacks cb 0x55b2dc0f2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:55.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.598+0000 7fafb6200640 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1dc530 0x55b2dc1da070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.598+0000 7fafb6200640 1 -- 172.21.15.69:0/4195247064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 msgr2=0x55b2dc1d95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:55.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.598+0000 7fafb6200640 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.598+0000 7fafb6200640 1 -- 172.21.15.69:0/4195247064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d3760 msgr2=0x55b2dc1d9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:55.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.599+0000 7fafb6200640 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d3760 0x55b2dc1d9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.599+0000 7fafb6200640 1 -- 172.21.15.69:0/4195247064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2dbe4b3f0 con 0x55b2dc1dc530 2024-03-19T01:18:55.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.599+0000 7fafb5800640 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:55.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.599+0000 7fafb4e00640 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d3760 0x55b2dc1d9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:55.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.599+0000 7fafb6200640 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1dc530 0x55b2dc1da070 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0ee5b0 tx=0x7faf9c003d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=156d07f2fc95fcdf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:55.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.599+0000 7faf97400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:18:55.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.600+0000 7faf97400640 1 -- 172.21.15.69:0/4195247064 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c04b030 con 0x55b2dc1dc530 2024-03-19T01:18:55.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.600+0000 7faf97400640 1 -- 172.21.15.69:0/4195247064 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7faf9c0d6260 con 0x55b2dc1dc530 2024-03-19T01:18:55.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.600+0000 7fafbae61740 1 -- 172.21.15.69:0/4195247064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa8021400 con 0x55b2dc1dc530 2024-03-19T01:18:55.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.600+0000 7faf97400640 1 -- 172.21.15.69:0/4195247064 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7faf9c0ee930 con 0x55b2dc1dc530 2024-03-19T01:18:55.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.600+0000 7fafbae61740 10 client.27389 fetch_fsmap learned FSMap version 50 2024-03-19T01:18:55.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.600+0000 7fafbae61740 1 -- 172.21.15.69:0/4195247064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55b2dbe0b0f0 con 0x55b2dc1dc530 2024-03-19T01:18:55.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.601+0000 7faf97400640 1 -- 172.21.15.69:0/4195247064 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa8021400 con 0x55b2dc1dc530 2024-03-19T01:18:55.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.601+0000 7faf97400640 1 -- 172.21.15.69:0/4195247064 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7faf9c0c57d0 con 0x55b2dc1dc530 2024-03-19T01:18:55.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.601+0000 7fafbae61740 10 client.27389 fetch_fsmap finished waiting for FSMap version 50 2024-03-19T01:18:55.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.601+0000 7fafbae61740 10 client.27389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:18:55.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.601+0000 7fafbae61740 20 client.27389 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:55.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.601+0000 7fafbae61740 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2dc1ec6a0 0x55b2dc1eeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.601+0000 7fafbae61740 4 client.27389 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:55.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.601+0000 7fafbae61740 1 -- 172.21.15.69:0/4195247064 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2dc0d8610 con 0x55b2dc1ec6a0 2024-03-19T01:18:55.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.601+0000 7fafb5800640 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2dc1ec6a0 0x55b2dc1eeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.602+0000 7fafb5800640 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2dc1ec6a0 0x55b2dc1eeae0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=849722caaa800195 server_cookie=1c7d18c2bfd5503e in_seq=0 out_seq=0 2024-03-19T01:18:55.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.602+0000 7faf97400640 10 client.27389 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:55.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.606+0000 7faf97400640 1 -- 172.21.15.69:0/4195247064 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b2dc0d8610 con 0x55b2dc1ec6a0 2024-03-19T01:18:55.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.606+0000 7faf97400640 10 client.27389 handle_command_reply: tid=0 2024-03-19T01:18:55.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.606+0000 7fafbae61740 1 client.27389 shutdown 2024-03-19T01:18:55.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7fafbae61740 1 -- 172.21.15.69:0/4195247064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1dc530 msgr2=0x55b2dc1da070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:55.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7fafbae61740 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1dc530 0x55b2dc1da070 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0ee5b0 tx=0x7faf9c003d90 comp rx=0 tx=0).stop 2024-03-19T01:18:55.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7fafbae61740 1 -- 172.21.15.69:0/4195247064 shutdown_connections 2024-03-19T01:18:55.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7fafbae61740 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2dc1ec6a0 0x55b2dc1eeae0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7fafbae61740 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1dc530 0x55b2dc1da070 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7fafbae61740 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d3760 0x55b2dc1d9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7fafbae61740 1 --2- 172.21.15.69:0/4195247064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1d95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7faf97400640 0 client.27389 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:55.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7fafbae61740 1 -- 172.21.15.69:0/4195247064 >> 172.21.15.69:0/4195247064 conn(0x55b2dc1cd760 msgr2=0x55b2dc1cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:55.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7fafbae61740 1 -- 172.21.15.69:0/4195247064 shutdown_connections 2024-03-19T01:18:55.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7fafbae61740 1 -- 172.21.15.69:0/4195247064 wait complete. 2024-03-19T01:18:55.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.607+0000 7fafbae61740 20 client.27389 trim_cache size 0 max 16384 2024-03-19T01:18:55.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.727+0000 7fafbae61740 1 Processor -- start 2024-03-19T01:18:55.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.727+0000 7fafbae61740 1 -- start start 2024-03-19T01:18:55.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.732+0000 7fafbae61740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1dbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.732+0000 7fafbae61740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.732+0000 7fafbae61740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1dc530 0x55b2dc1d8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.732+0000 7fafbae61740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01c65b0 con 0x55b2dc1d2e40 2024-03-19T01:18:55.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.732+0000 7fafbae61740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01c8270 con 0x55b2dc1d3760 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.732+0000 7fafbae61740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01c6c80 con 0x55b2dc1dc530 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.732+0000 7fafb6200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1dc530 0x55b2dc1d8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.733+0000 7fafb4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.733+0000 7fafb4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1e7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36200/0 (socket says 172.21.15.69:36200) 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.733+0000 7fafb4e00640 1 -- 172.21.15.69:0/3907181624 learned_addr learned my addr 172.21.15.69:0/3907181624 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.733+0000 7fafb4e00640 1 -- 172.21.15.69:0/3907181624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1dc530 msgr2=0x55b2dc1d8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.733+0000 7fafb4e00640 1 --2- 172.21.15.69:0/3907181624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1dc530 0x55b2dc1d8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.733+0000 7fafb4e00640 1 -- 172.21.15.69:0/3907181624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d2e40 msgr2=0x55b2dc1dbd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.734+0000 7fafb5800640 1 --2- 172.21.15.69:0/3907181624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1dbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.734+0000 7fafb4e00640 1 --2- 172.21.15.69:0/3907181624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1dbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.734+0000 7fafb4e00640 1 -- 172.21.15.69:0/3907181624 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2dc1ef160 con 0x55b2dc1d3760 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.735+0000 7fafb4e00640 1 --2- 172.21.15.69:0/3907181624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1e7e30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fafb010ba90 tx=0x7fafa400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a55f97e8386057e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.735+0000 7fafb5800640 1 --2- 172.21.15.69:0/3907181624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1dbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.735+0000 7fafa2000640 1 -- 172.21.15.69:0/3907181624 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4003e30 con 0x55b2dc1d3760 2024-03-19T01:18:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.735+0000 7fafa2000640 1 -- 172.21.15.69:0/3907181624 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa4010910 con 0x55b2dc1d3760 2024-03-19T01:18:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.735+0000 7fafa2000640 1 -- 172.21.15.69:0/3907181624 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa401aa60 con 0x55b2dc1d3760 2024-03-19T01:18:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.736+0000 7fafbae61740 1 -- 172.21.15.69:0/3907181624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 msgr2=0x55b2dc1e7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.736+0000 7fafbae61740 1 --2- 172.21.15.69:0/3907181624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1e7e30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fafb010ba90 tx=0x7fafa400cf60 comp rx=0 tx=0).stop 2024-03-19T01:18:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.737+0000 7fafbae61740 1 -- 172.21.15.69:0/3907181624 shutdown_connections 2024-03-19T01:18:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.737+0000 7fafbae61740 1 --2- 172.21.15.69:0/3907181624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1dc530 0x55b2dc1d8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.737+0000 7fafbae61740 1 --2- 172.21.15.69:0/3907181624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1e7e30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.737+0000 7fafbae61740 1 --2- 172.21.15.69:0/3907181624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1dbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.737+0000 7fafbae61740 1 -- 172.21.15.69:0/3907181624 >> 172.21.15.69:0/3907181624 conn(0x55b2dc1cd760 msgr2=0x55b2dc1cbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.737+0000 7fafbae61740 1 -- 172.21.15.69:0/3907181624 shutdown_connections 2024-03-19T01:18:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.737+0000 7fafbae61740 1 -- 172.21.15.69:0/3907181624 wait complete. 2024-03-19T01:18:55.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.739+0000 7fafbae61740 1 -- start start 2024-03-19T01:18:55.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.739+0000 7fafbae61740 1 Processor -- start 2024-03-19T01:18:55.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.743+0000 7fafbae61740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1e4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.743+0000 7fafbae61740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1dfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.743+0000 7fafbae61740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 0x55b2dc1dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.744+0000 7fafb6200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 0x55b2dc1dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.744+0000 7fafb6200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 0x55b2dc1dffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42910/0 (socket says 172.21.15.69:42910) 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.744+0000 7fafb6200640 1 -- 172.21.15.69:0/3313467070 learned_addr learned my addr 172.21.15.69:0/3313467070 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.744+0000 7fafbae61740 1 -- 172.21.15.69:0/3313467070 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01c7ae0 con 0x55b2dc1dc530 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.744+0000 7fafbae61740 1 -- 172.21.15.69:0/3313467070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01c7410 con 0x55b2dc1d3760 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.744+0000 7fafbae61740 1 -- 172.21.15.69:0/3313467070 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010b770 con 0x55b2dc1d2e40 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.744+0000 7fafb6200640 1 -- 172.21.15.69:0/3313467070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 msgr2=0x55b2dc1e4730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.744+0000 7fafb6200640 1 --2- 172.21.15.69:0/3313467070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1e4730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.744+0000 7fafb6200640 1 -- 172.21.15.69:0/3313467070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 msgr2=0x55b2dc1dfaa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.744+0000 7fafb6200640 1 --2- 172.21.15.69:0/3313467070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1dfaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.744+0000 7fafb6200640 1 -- 172.21.15.69:0/3313467070 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2dc1e07e0 con 0x55b2dc1dc530 2024-03-19T01:18:55.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.745+0000 7fafbae61740 10 client.0 _ll_register_callbacks cb 0x7faf9c0b8fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:55.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.745+0000 7fafbae61740 1 -- 172.21.15.69:0/3313467070 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa4010910 con 0x55b2dc1dc530 2024-03-19T01:18:55.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.745+0000 7fafb6200640 1 --2- 172.21.15.69:0/3313467070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 0x55b2dc1dffe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0c16f0 tx=0x7faf9c0d62f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270040c0673939f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:55.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.746+0000 7faf97400640 10 client.17088 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:18:55.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.748+0000 7faf97400640 1 -- 172.21.15.69:0/3313467070 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c04b070 con 0x55b2dc1dc530 2024-03-19T01:18:55.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.748+0000 7faf97400640 1 -- 172.21.15.69:0/3313467070 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7faf9c0bcb10 con 0x55b2dc1dc530 2024-03-19T01:18:55.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.748+0000 7faf97400640 1 -- 172.21.15.69:0/3313467070 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7faf9c0bce20 con 0x55b2dc1dc530 2024-03-19T01:18:55.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.751+0000 7fafbae61740 10 client.17088 fetch_fsmap learned FSMap version 50 2024-03-19T01:18:55.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.751+0000 7fafbae61740 1 -- 172.21.15.69:0/3313467070 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=50}) -- 0x55b2dc252860 con 0x55b2dc1dc530 2024-03-19T01:18:55.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.751+0000 7faf97400640 1 -- 172.21.15.69:0/3313467070 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 50) ==== 8402+0+0 (secure 0 0 0) 0x7faf9c0b9180 con 0x55b2dc1dc530 2024-03-19T01:18:55.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.751+0000 7fafbae61740 10 client.17088 fetch_fsmap finished waiting for FSMap version 50 2024-03-19T01:18:55.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.751+0000 7fafbae61740 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:18:55.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.752+0000 7fafbae61740 20 client.17088 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:55.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.752+0000 7fafbae61740 1 --2- 172.21.15.69:0/3313467070 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2dc252d10 0x55b2dc255190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:55.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.752+0000 7fafbae61740 4 client.17088 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:18:55.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.752+0000 7fafbae61740 1 -- 172.21.15.69:0/3313467070 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2dc144800 con 0x55b2dc252d10 2024-03-19T01:18:55.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.752+0000 7fafb5800640 1 --2- 172.21.15.69:0/3313467070 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2dc252d10 0x55b2dc255190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:55.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.753+0000 7fafb5800640 1 --2- 172.21.15.69:0/3313467070 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2dc252d10 0x55b2dc255190 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4f085d36204b06d server_cookie=4bf48b9998213b61 in_seq=0 out_seq=0 2024-03-19T01:18:55.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.753+0000 7faf97400640 10 client.17088 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:55.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.753+0000 7faf97400640 1 -- 172.21.15.69:0/3313467070 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b2dc144800 con 0x55b2dc252d10 2024-03-19T01:18:55.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.753+0000 7faf97400640 10 client.17088 handle_command_reply: tid=0 2024-03-19T01:18:55.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.753+0000 7fafbae61740 1 client.17088 shutdown 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.753+0000 7fafbae61740 1 -- 172.21.15.69:0/3313467070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 msgr2=0x55b2dc1dffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.753+0000 7fafbae61740 1 --2- 172.21.15.69:0/3313467070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 0x55b2dc1dffe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0c16f0 tx=0x7faf9c0d62f0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.754+0000 7fafbae61740 1 -- 172.21.15.69:0/3313467070 shutdown_connections 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.754+0000 7fafbae61740 1 --2- 172.21.15.69:0/3313467070 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b2dc252d10 0x55b2dc255190 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.754+0000 7fafbae61740 1 --2- 172.21.15.69:0/3313467070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2dc1dc530 0x55b2dc1dffe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.754+0000 7faf97400640 0 client.17088 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.754+0000 7fafbae61740 1 --2- 172.21.15.69:0/3313467070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2dc1d3760 0x55b2dc1dfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.754+0000 7fafbae61740 1 --2- 172.21.15.69:0/3313467070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b2dc1d2e40 0x55b2dc1e4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.754+0000 7fafbae61740 1 -- 172.21.15.69:0/3313467070 >> 172.21.15.69:0/3313467070 conn(0x55b2dc1cd760 msgr2=0x55b2dc1d95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.754+0000 7fafbae61740 1 -- 172.21.15.69:0/3313467070 shutdown_connections 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.754+0000 7fafbae61740 1 -- 172.21.15.69:0/3313467070 wait complete. 2024-03-19T01:18:55.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.755+0000 7fafbae61740 20 client.17088 trim_cache size 0 max 16384 2024-03-19T01:18:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:55 smithi086 ceph-mon[66829]: pgmap v254: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 937 KiB/s wr, 93 op/s 2024-03-19T01:18:55.757 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:18:55.757 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:18:55.757 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:18:55.757 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:18:55.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.760+0000 7fafa3400640 1 -- 172.21.15.69:0/925495998 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faf980a2a30 msgr2=0x7faf980a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:55.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.760+0000 7fafa3400640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faf980a2a30 0x7faf980a4ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fafa800ac00 tx=0x7fafa8009250 comp rx=0 tx=0).stop 2024-03-19T01:18:55.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.760+0000 7fafa3400640 1 -- 172.21.15.69:0/925495998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb0114970 msgr2=0x7fafb010a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:55.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.760+0000 7fafa3400640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb0114970 0x7fafb010a8d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf9c006010 tx=0x7faf9c004340 comp rx=0 tx=0).stop 2024-03-19T01:18:55.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.760+0000 7fafa3400640 1 -- 172.21.15.69:0/925495998 shutdown_connections 2024-03-19T01:18:55.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.760+0000 7fafa3400640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb0114970 0x7fafb010a8d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.760+0000 7fafa3400640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7faf980a2a30 0x7faf980a4ef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.760+0000 7fafa3400640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb0109320 0x7fafb010a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.760+0000 7fafa3400640 1 --2- 172.21.15.69:0/925495998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafb0108a00 0x7fafb010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:55.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.760+0000 7fafa3400640 1 -- 172.21.15.69:0/925495998 >> 172.21.15.69:0/925495998 conn(0x7fafb006c900 msgr2=0x7fafb00703c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:55.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.761+0000 7fafa3400640 1 -- 172.21.15.69:0/925495998 shutdown_connections 2024-03-19T01:18:55.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:55.761+0000 7fafa3400640 1 -- 172.21.15.69:0/925495998 wait complete. 2024-03-19T01:18:55.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:55 smithi163 ceph-mon[67428]: pgmap v254: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 937 KiB/s wr, 93 op/s 2024-03-19T01:18:55.823 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:18:55.823 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:18:56.333 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/stat.o 2024-03-19T01:18:56.464 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mmap_lock.o 2024-03-19T01:18:56.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.538+0000 7fe7cf800640 1 -- 172.21.15.69:0/3604545182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c80823a0 msgr2=0x7fe7c807a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:56.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.538+0000 7fe7cf800640 1 --2- 172.21.15.69:0/3604545182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c80823a0 0x7fe7c807a180 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009860 tx=0x7fe7b402bc00 comp rx=0 tx=0).stop 2024-03-19T01:18:56.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.542+0000 7fe7cf800640 1 -- 172.21.15.69:0/3604545182 shutdown_connections 2024-03-19T01:18:56.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.542+0000 7fe7cf800640 1 --2- 172.21.15.69:0/3604545182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7c811ace0 0x7fe7c811d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:56.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.542+0000 7fe7cf800640 1 --2- 172.21.15.69:0/3604545182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7c807a6c0 0x7fe7c807ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:56.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.542+0000 7fe7cf800640 1 --2- 172.21.15.69:0/3604545182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c80823a0 0x7fe7c807a180 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:56.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.542+0000 7fe7cf800640 1 -- 172.21.15.69:0/3604545182 >> 172.21.15.69:0/3604545182 conn(0x7fe7c8075d70 msgr2=0x7fe7c8078190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:56.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.542+0000 7fe7cf800640 1 -- 172.21.15.69:0/3604545182 shutdown_connections 2024-03-19T01:18:56.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.542+0000 7fe7cf800640 1 -- 172.21.15.69:0/3604545182 wait complete. 2024-03-19T01:18:56.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.543+0000 7fe7cf800640 1 Processor -- start 2024-03-19T01:18:56.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.547+0000 7fe7cf800640 1 -- start start 2024-03-19T01:18:56.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.547+0000 7fe7cf800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7c807a6c0 0x7fe7c81a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:56.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.547+0000 7fe7cf800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7c80823a0 0x7fe7c81a7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:56.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.547+0000 7fe7cf800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c811ace0 0x7fe7c81ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:56.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.547+0000 7fe7cf800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81209b0 con 0x7fe7c80823a0 2024-03-19T01:18:56.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.547+0000 7fe7cf800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8120830 con 0x7fe7c811ace0 2024-03-19T01:18:56.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.547+0000 7fe7cf800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8120b30 con 0x7fe7c807a6c0 2024-03-19T01:18:56.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.548+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7c80823a0 0x7fe7c81a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:56.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.548+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7c80823a0 0x7fe7c81a7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43888/0 (socket says 172.21.15.69:43888) 2024-03-19T01:18:56.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.548+0000 7fe7c7e00640 1 -- 172.21.15.69:0/2173498374 learned_addr learned my addr 172.21.15.69:0/2173498374 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:56.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.548+0000 7fe7cce00640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7c807a6c0 0x7fe7c81a6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:56.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.548+0000 7fe7cd800640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c811ace0 0x7fe7c81ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:56.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.548+0000 7fe7cce00640 1 -- 172.21.15.69:0/2173498374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c811ace0 msgr2=0x7fe7c81ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:56.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.548+0000 7fe7cce00640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c811ace0 0x7fe7c81ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:56.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.548+0000 7fe7cce00640 1 -- 172.21.15.69:0/2173498374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7c80823a0 msgr2=0x7fe7c81a7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:56.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.548+0000 7fe7cce00640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7c80823a0 0x7fe7c81a7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:56.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.549+0000 7fe7cce00640 1 -- 172.21.15.69:0/2173498374 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c81abee0 con 0x7fe7c807a6c0 2024-03-19T01:18:56.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.549+0000 7fe7cd800640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c811ace0 0x7fe7c81ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:56.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.549+0000 7fe7cce00640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7c807a6c0 0x7fe7c81a6ef0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe7b402c140 tx=0x7fe7b40346f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:56.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.549+0000 7fe7c5600640 1 -- 172.21.15.69:0/2173498374 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4041020 con 0x7fe7c807a6c0 2024-03-19T01:18:56.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.549+0000 7fe7c5600640 1 -- 172.21.15.69:0/2173498374 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b40044d0 con 0x7fe7c807a6c0 2024-03-19T01:18:56.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.549+0000 7fe7cf800640 1 -- 172.21.15.69:0/2173498374 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c81ac170 con 0x7fe7c807a6c0 2024-03-19T01:18:56.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.549+0000 7fe7c5600640 1 -- 172.21.15.69:0/2173498374 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4039040 con 0x7fe7c807a6c0 2024-03-19T01:18:56.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.550+0000 7fe7cf800640 1 -- 172.21.15.69:0/2173498374 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81b3a60 con 0x7fe7c807a6c0 2024-03-19T01:18:56.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.553+0000 7fe7cf800640 1 -- 172.21.15.69:0/2173498374 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c8068a60 con 0x7fe7c807a6c0 2024-03-19T01:18:56.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.554+0000 7fe7c5600640 1 -- 172.21.15.69:0/2173498374 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fe7b4044020 con 0x7fe7c807a6c0 2024-03-19T01:18:56.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.556+0000 7fe7c5600640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe7ac0a29f0 0x7fe7ac0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:56.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.556+0000 7fe7c5600640 1 -- 172.21.15.69:0/2173498374 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7fe7b40f0880 con 0x7fe7c807a6c0 2024-03-19T01:18:56.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.558+0000 7fe7c7e00640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe7ac0a29f0 0x7fe7ac0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:56.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.563+0000 7fe7c7e00640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe7ac0a29f0 0x7fe7ac0a4eb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe7c811d1b0 tx=0x7fe7b80095c0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:56.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:56.564+0000 7fe7c5600640 1 -- 172.21.15.69:0/2173498374 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe7b404b020 con 0x7fe7c807a6c0 2024-03-19T01:18:56.910 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/sha3_generic.o 2024-03-19T01:18:56.993 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/migration.o 2024-03-19T01:18:57.028 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-mq-tag.o 2024-03-19T01:18:57.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.105+0000 7fe7cf800640 1 -- 172.21.15.69:0/2173498374 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7c8077860 con 0x7fe7c807a6c0 2024-03-19T01:18:57.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.106+0000 7fe7c5600640 1 -- 172.21.15.69:0/2173498374 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11945 (secure 0 0 0) 0x7fe7b40bd2e0 con 0x7fe7c807a6c0 2024-03-19T01:18:57.110 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:18:57.110 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":51,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:18:56.101949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:18:57.110 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 51 2024-03-19T01:18:57.116 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/epoll.o 2024-03-19T01:18:57.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.125+0000 7fe7cf800640 1 -- 172.21.15.69:0/2173498374 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe7ac0a29f0 msgr2=0x7fe7ac0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:57.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.125+0000 7fe7cf800640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe7ac0a29f0 0x7fe7ac0a4eb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe7c811d1b0 tx=0x7fe7b80095c0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.125+0000 7fe7cf800640 1 -- 172.21.15.69:0/2173498374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7c807a6c0 msgr2=0x7fe7c81a6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:57.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.125+0000 7fe7cf800640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7c807a6c0 0x7fe7c81a6ef0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe7b402c140 tx=0x7fe7b40346f0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.137+0000 7fe7cf800640 1 -- 172.21.15.69:0/2173498374 shutdown_connections 2024-03-19T01:18:57.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.137+0000 7fe7cf800640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe7ac0a29f0 0x7fe7ac0a4eb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.137+0000 7fe7cf800640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c811ace0 0x7fe7c81ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.137+0000 7fe7cf800640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7c80823a0 0x7fe7c81a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.137+0000 7fe7cf800640 1 --2- 172.21.15.69:0/2173498374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7c807a6c0 0x7fe7c81a6ef0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.137+0000 7fe7cf800640 1 -- 172.21.15.69:0/2173498374 >> 172.21.15.69:0/2173498374 conn(0x7fe7c8075d70 msgr2=0x7fe7c80814e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:57.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.141+0000 7fe7cf800640 1 -- 172.21.15.69:0/2173498374 shutdown_connections 2024-03-19T01:18:57.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.142+0000 7fe7cf800640 1 -- 172.21.15.69:0/2173498374 wait complete. 2024-03-19T01:18:57.176 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/probe.o 2024-03-19T01:18:57.246 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:18:57.401 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/ss/services.o 2024-03-19T01:18:57.435 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/uptime.o 2024-03-19T01:18:57.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:57 smithi069 ceph-mon[31727]: pgmap v255: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 943 KiB/s wr, 96 op/s 2024-03-19T01:18:57.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:57 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:57.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:57 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2173498374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:18:57.602 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/highmem.o 2024-03-19T01:18:57.657 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/cbc.o 2024-03-19T01:18:57.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:57 smithi086 ceph-mon[66829]: pgmap v255: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 943 KiB/s wr, 96 op/s 2024-03-19T01:18:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:57 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:57 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2173498374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:18:57.764 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/utils.o 2024-03-19T01:18:57.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.801+0000 7f73ed400640 1 -- 172.21.15.69:0/3393577217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8108700 msgr2=0x7f73e8114ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:57.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.801+0000 7f73ed400640 1 --2- 172.21.15.69:0/3393577217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8108700 0x7f73e8114ad0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009be0 tx=0x7f73e002bf30 comp rx=0 tx=0).stop 2024-03-19T01:18:57.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.804+0000 7f73ed400640 1 -- 172.21.15.69:0/3393577217 shutdown_connections 2024-03-19T01:18:57.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.804+0000 7f73ed400640 1 --2- 172.21.15.69:0/3393577217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8108700 0x7f73e8114ad0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.804+0000 7f73ed400640 1 --2- 172.21.15.69:0/3393577217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e8107d60 0x7f73e81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.804+0000 7f73ed400640 1 --2- 172.21.15.69:0/3393577217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73e810dca0 0x7f73e810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.804+0000 7f73ed400640 1 -- 172.21.15.69:0/3393577217 >> 172.21.15.69:0/3393577217 conn(0x7f73e80775f0 msgr2=0x7f73e8106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:57.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.808+0000 7f73ed400640 1 -- 172.21.15.69:0/3393577217 shutdown_connections 2024-03-19T01:18:57.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.808+0000 7f73ed400640 1 -- 172.21.15.69:0/3393577217 wait complete. 2024-03-19T01:18:57.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.809+0000 7f73ed400640 1 Processor -- start 2024-03-19T01:18:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:57 smithi163 ceph-mon[67428]: pgmap v255: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 943 KiB/s wr, 96 op/s 2024-03-19T01:18:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:57 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:18:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:57 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2173498374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:18:57.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.812+0000 7f73ed400640 1 -- start start 2024-03-19T01:18:57.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.812+0000 7f73ed400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e8107d60 0x7f73e8081d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.812+0000 7f73ed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8108700 0x7f73e80822a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.812+0000 7f73ed400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73e810dca0 0x7f73e807af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.812+0000 7f73ed400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8118050 con 0x7f73e810dca0 2024-03-19T01:18:57.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.812+0000 7f73ed400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8117ed0 con 0x7f73e8108700 2024-03-19T01:18:57.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.812+0000 7f73ed400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81181d0 con 0x7f73e8107d60 2024-03-19T01:18:57.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.813+0000 7f73e7200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73e810dca0 0x7f73e807af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.813+0000 7f73e5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8108700 0x7f73e80822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.813+0000 7f73e7200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73e810dca0 0x7f73e807af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43912/0 (socket says 172.21.15.69:43912) 2024-03-19T01:18:57.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.813+0000 7f73e7200640 1 -- 172.21.15.69:0/2385702638 learned_addr learned my addr 172.21.15.69:0/2385702638 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:57.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.813+0000 7f73e6800640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e8107d60 0x7f73e8081d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.813+0000 7f73e6800640 1 -- 172.21.15.69:0/2385702638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8108700 msgr2=0x7f73e80822a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:57.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.813+0000 7f73e6800640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8108700 0x7f73e80822a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.814+0000 7f73e6800640 1 -- 172.21.15.69:0/2385702638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73e810dca0 msgr2=0x7f73e807af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:57.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.814+0000 7f73e6800640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73e810dca0 0x7f73e807af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.814+0000 7f73e6800640 1 -- 172.21.15.69:0/2385702638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e807b7e0 con 0x7f73e8107d60 2024-03-19T01:18:57.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.814+0000 7f73e7200640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73e810dca0 0x7f73e807af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:57.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.817+0000 7f73e6800640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e8107d60 0x7f73e8081d60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f73d800d6d0 tx=0x7f73d800dad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:57.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.817+0000 7f73df400640 1 -- 172.21.15.69:0/2385702638 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8017070 con 0x7f73e8107d60 2024-03-19T01:18:57.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.817+0000 7f73df400640 1 -- 172.21.15.69:0/2385702638 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d80047a0 con 0x7f73e8107d60 2024-03-19T01:18:57.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.817+0000 7f73df400640 1 -- 172.21.15.69:0/2385702638 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8009c80 con 0x7f73e8107d60 2024-03-19T01:18:57.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.817+0000 7f73ed400640 1 -- 172.21.15.69:0/2385702638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e807bad0 con 0x7f73e8107d60 2024-03-19T01:18:57.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.818+0000 7f73ed400640 1 -- 172.21.15.69:0/2385702638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e807fa60 con 0x7f73e8107d60 2024-03-19T01:18:57.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.822+0000 7f73f002f740 1 Processor -- start 2024-03-19T01:18:57.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.824+0000 7f73f002f740 1 -- start start 2024-03-19T01:18:57.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.824+0000 7f73df400640 1 -- 172.21.15.69:0/2385702638 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f73d8004950 con 0x7f73e8107d60 2024-03-19T01:18:57.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.825+0000 7f73f002f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc3c10 0x55b129cc3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.825+0000 7f73f002f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc4530 0x55b129cccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.825+0000 7f73f002f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129ccd110 0x55b129ccf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.825+0000 7f73f002f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b129c28380 con 0x55b129cc4530 2024-03-19T01:18:57.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.825+0000 7f73f002f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b129bd0d20 con 0x55b129ccd110 2024-03-19T01:18:57.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.825+0000 7f73f002f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b129c24420 con 0x55b129cc3c10 2024-03-19T01:18:57.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.826+0000 7f73e7200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc3c10 0x55b129cc3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.826+0000 7f73e7200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc3c10 0x55b129cc3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44800/0 (socket says 172.21.15.69:44800) 2024-03-19T01:18:57.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.826+0000 7f73e7200640 1 -- 172.21.15.69:0/1534910614 learned_addr learned my addr 172.21.15.69:0/1534910614 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:57.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.826+0000 7f73e5e00640 1 --2- 172.21.15.69:0/1534910614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129ccd110 0x55b129ccf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.826+0000 7f73e7200640 1 -- 172.21.15.69:0/1534910614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129ccd110 msgr2=0x55b129ccf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:57.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.826+0000 7f73e7200640 1 --2- 172.21.15.69:0/1534910614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129ccd110 0x55b129ccf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.826+0000 7f73e7200640 1 -- 172.21.15.69:0/1534910614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc4530 msgr2=0x55b129cccbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:57.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.827+0000 7f73df400640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f73cc0a2a30 0x7f73cc0a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.827+0000 7f73df400640 1 -- 172.21.15.69:0/2385702638 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f73d80ce9c0 con 0x7f73e8107d60 2024-03-19T01:18:57.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.828+0000 7f73e5e00640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f73cc0a2a30 0x7f73cc0a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.829+0000 7f73e5e00640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f73cc0a2a30 0x7f73cc0a4ef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55b129cbf4d0 tx=0x7f73d4009260 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:57.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.829+0000 7f73e6800640 1 --2- 172.21.15.69:0/1534910614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc4530 0x55b129cccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.829+0000 7f73e7200640 1 --2- 172.21.15.69:0/1534910614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc4530 0x55b129cccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.829+0000 7f73e7200640 1 -- 172.21.15.69:0/1534910614 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12993c3f0 con 0x55b129cc3c10 2024-03-19T01:18:57.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.829+0000 7f73e6800640 1 --2- 172.21.15.69:0/1534910614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc4530 0x55b129cccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:57.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.829+0000 7f73e7200640 1 --2- 172.21.15.69:0/1534910614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc3c10 0x55b129cc3ff0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009920 tx=0x7f73e0009950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f17a534fe601cef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:57.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.829+0000 7f73d3400640 1 -- 172.21.15.69:0/1534910614 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0045070 con 0x55b129cc3c10 2024-03-19T01:18:57.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.830+0000 7f73d3400640 1 -- 172.21.15.69:0/1534910614 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e0034d60 con 0x55b129cc3c10 2024-03-19T01:18:57.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.830+0000 7f73d3400640 1 -- 172.21.15.69:0/1534910614 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e003fce0 con 0x55b129cc3c10 2024-03-19T01:18:57.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.831+0000 7f73f002f740 1 -- 172.21.15.69:0/1534910614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc3c10 msgr2=0x55b129cc3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:57.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.831+0000 7f73f002f740 1 --2- 172.21.15.69:0/1534910614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc3c10 0x55b129cc3ff0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009920 tx=0x7f73e0009950 comp rx=0 tx=0).stop 2024-03-19T01:18:57.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.833+0000 7f73f002f740 1 -- 172.21.15.69:0/1534910614 shutdown_connections 2024-03-19T01:18:57.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.834+0000 7f73f002f740 1 --2- 172.21.15.69:0/1534910614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129ccd110 0x55b129ccf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.834+0000 7f73f002f740 1 --2- 172.21.15.69:0/1534910614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc4530 0x55b129cccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.834+0000 7f73f002f740 1 --2- 172.21.15.69:0/1534910614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc3c10 0x55b129cc3ff0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.834+0000 7f73f002f740 1 -- 172.21.15.69:0/1534910614 >> 172.21.15.69:0/1534910614 conn(0x55b129cbe340 msgr2=0x55b129cbe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.834+0000 7f73f002f740 1 -- 172.21.15.69:0/1534910614 shutdown_connections 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.834+0000 7f73f002f740 1 -- 172.21.15.69:0/1534910614 wait complete. 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.835+0000 7f73f002f740 1 -- start start 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.835+0000 7f73f002f740 1 Processor -- start 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.836+0000 7f73f002f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 0x55b129cca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.836+0000 7f73f002f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.836+0000 7f73f002f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 0x55b129ccac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.837+0000 7f73e5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.837+0000 7f73e5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44808/0 (socket says 172.21.15.69:44808) 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.837+0000 7f73e7200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 0x55b129ccac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.837+0000 7f73e7200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 0x55b129ccac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43930/0 (socket says 172.21.15.69:43930) 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.837+0000 7f73e6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 0x55b129cca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.837+0000 7f73e6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 0x55b129cca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58536/0 (socket says 172.21.15.69:58536) 2024-03-19T01:18:57.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.837+0000 7f73e5e00640 1 -- 172.21.15.69:0/3511043044 learned_addr learned my addr 172.21.15.69:0/3511043044 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:57.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.838+0000 7f73f002f740 1 -- 172.21.15.69:0/3511043044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b129ccfea0 con 0x55b129ccd110 2024-03-19T01:18:57.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.838+0000 7f73f002f740 1 -- 172.21.15.69:0/3511043044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b129be4160 con 0x55b129cc3c10 2024-03-19T01:18:57.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.838+0000 7f73f002f740 1 -- 172.21.15.69:0/3511043044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b129cd0020 con 0x55b129cc4530 2024-03-19T01:18:57.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.838+0000 7f73e7200640 1 -- 172.21.15.69:0/3511043044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 msgr2=0x55b129cca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:57.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.838+0000 7f73e7200640 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.838+0000 7f73e7200640 1 -- 172.21.15.69:0/3511043044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 msgr2=0x55b129cca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:57.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.838+0000 7f73e7200640 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 0x55b129cca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.838+0000 7f73e7200640 1 -- 172.21.15.69:0/3511043044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b129a067d0 con 0x55b129ccd110 2024-03-19T01:18:57.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.838+0000 7f73e5e00640 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:18:57.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.839+0000 7f73f002f740 10 client.0 _ll_register_callbacks cb 0x55b129c20170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:57.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.839+0000 7f73f002f740 1 -- 172.21.15.69:0/3511043044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e0034d60 con 0x55b129ccd110 2024-03-19T01:18:57.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.839+0000 7f73e7200640 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 0x55b129ccac60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73e0036040 tx=0x7f73e0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb678060c2a126d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:57.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.846+0000 7f73c7e00640 10 client.17103 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:18:57.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.846+0000 7f73c7e00640 1 -- 172.21.15.69:0/3511043044 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e004e030 con 0x55b129ccd110 2024-03-19T01:18:57.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.847+0000 7f73c7e00640 1 -- 172.21.15.69:0/3511043044 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f73e003fc30 con 0x55b129ccd110 2024-03-19T01:18:57.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.847+0000 7f73c7e00640 1 -- 172.21.15.69:0/3511043044 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f73e0045040 con 0x55b129ccd110 2024-03-19T01:18:57.849 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/cpuhotplug.o 2024-03-19T01:18:57.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.848+0000 7f73f002f740 10 client.17103 fetch_fsmap learned FSMap version 51 2024-03-19T01:18:57.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.848+0000 7f73f002f740 1 -- 172.21.15.69:0/3511043044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=51}) -- 0x55b129cdbdc0 con 0x55b129ccd110 2024-03-19T01:18:57.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.849+0000 7f73c7e00640 1 -- 172.21.15.69:0/3511043044 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f73e0003970 con 0x55b129ccd110 2024-03-19T01:18:57.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.849+0000 7f73f002f740 10 client.17103 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:18:57.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.849+0000 7f73f002f740 10 client.17103 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:18:57.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.849+0000 7f73f002f740 20 client.17103 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:57.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.849+0000 7f73f002f740 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b129cdd110 0x55b129cdf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.849+0000 7f73f002f740 4 client.17103 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:57.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.849+0000 7f73f002f740 1 -- 172.21.15.69:0/3511043044 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b129c358c0 con 0x55b129cdd110 2024-03-19T01:18:57.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.850+0000 7f73e6800640 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b129cdd110 0x55b129cdf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.851+0000 7f73e6800640 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b129cdd110 0x55b129cdf500 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a07f858fa63a0ab server_cookie=14445f55d88ce74d in_seq=0 out_seq=0 2024-03-19T01:18:57.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.851+0000 7f73c7e00640 10 client.17103 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:18:57.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.852+0000 7f73c7e00640 1 -- 172.21.15.69:0/3511043044 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b129c358c0 con 0x55b129cdd110 2024-03-19T01:18:57.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.852+0000 7f73c7e00640 10 client.17103 handle_command_reply: tid=0 2024-03-19T01:18:57.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.852+0000 7f73f002f740 1 client.17103 shutdown 2024-03-19T01:18:57.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.854+0000 7f73f002f740 1 -- 172.21.15.69:0/3511043044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 msgr2=0x55b129ccac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:57.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.854+0000 7f73f002f740 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 0x55b129ccac60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73e0036040 tx=0x7f73e0009500 comp rx=0 tx=0).stop 2024-03-19T01:18:57.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.854+0000 7f73f002f740 1 -- 172.21.15.69:0/3511043044 shutdown_connections 2024-03-19T01:18:57.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.854+0000 7f73f002f740 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b129cdd110 0x55b129cdf500 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.854+0000 7f73f002f740 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 0x55b129ccac60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.854+0000 7f73c7e00640 0 client.17103 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:18:57.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.854+0000 7f73f002f740 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.854+0000 7f73f002f740 1 --2- 172.21.15.69:0/3511043044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 0x55b129cca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.854+0000 7f73f002f740 1 -- 172.21.15.69:0/3511043044 >> 172.21.15.69:0/3511043044 conn(0x55b129cbe340 msgr2=0x55b129cc0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:57.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.858+0000 7f73f002f740 1 -- 172.21.15.69:0/3511043044 shutdown_connections 2024-03-19T01:18:57.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.858+0000 7f73f002f740 1 -- 172.21.15.69:0/3511043044 wait complete. 2024-03-19T01:18:57.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.858+0000 7f73f002f740 20 client.17103 trim_cache size 0 max 16384 2024-03-19T01:18:57.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.919+0000 7f73f002f740 1 Processor -- start 2024-03-19T01:18:57.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.919+0000 7f73f002f740 1 -- start start 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.919+0000 7f73f002f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 0x55b129cd8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73f002f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129ce4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73f002f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 0x55b129ccaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73f002f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81c62e0 con 0x55b129ccd110 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73f002f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81c7fa0 con 0x55b129cc3c10 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73f002f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81c69b0 con 0x55b129cc4530 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73e6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 0x55b129cd8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73e6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 0x55b129cd8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58544/0 (socket says 172.21.15.69:58544) 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73e6800640 1 -- 172.21.15.69:0/4131276891 learned_addr learned my addr 172.21.15.69:0/4131276891 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73e6800640 1 -- 172.21.15.69:0/4131276891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 msgr2=0x55b129ce4c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73e6800640 1 --2- 172.21.15.69:0/4131276891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129ce4c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73e6800640 1 -- 172.21.15.69:0/4131276891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 msgr2=0x55b129ccaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73e7200640 1 --2- 172.21.15.69:0/4131276891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 0x55b129ccaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73e6800640 1 --2- 172.21.15.69:0/4131276891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 0x55b129ccaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73e6800640 1 -- 172.21.15.69:0/4131276891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b129cc9190 con 0x55b129cc3c10 2024-03-19T01:18:57.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.920+0000 7f73e7200640 1 --2- 172.21.15.69:0/4131276891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 0x55b129ccaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:57.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.921+0000 7f73e6800640 1 --2- 172.21.15.69:0/4131276891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 0x55b129cd8c40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f73d809a910 tx=0x7f73d80ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b825c3068dd3e74 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:57.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.921+0000 7f73d2a00640 1 -- 172.21.15.69:0/4131276891 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8020040 con 0x55b129cc3c10 2024-03-19T01:18:57.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.921+0000 7f73d2a00640 1 -- 172.21.15.69:0/4131276891 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d809ed60 con 0x55b129cc3c10 2024-03-19T01:18:57.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.921+0000 7f73d2a00640 1 -- 172.21.15.69:0/4131276891 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8097410 con 0x55b129cc3c10 2024-03-19T01:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.922+0000 7f73f002f740 1 -- 172.21.15.69:0/4131276891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 msgr2=0x55b129cd8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.922+0000 7f73f002f740 1 --2- 172.21.15.69:0/4131276891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 0x55b129cd8c40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f73d809a910 tx=0x7f73d80ce040 comp rx=0 tx=0).stop 2024-03-19T01:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.922+0000 7f73f002f740 1 -- 172.21.15.69:0/4131276891 shutdown_connections 2024-03-19T01:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.922+0000 7f73f002f740 1 --2- 172.21.15.69:0/4131276891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129ccd110 0x55b129ccaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.922+0000 7f73f002f740 1 --2- 172.21.15.69:0/4131276891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129ce4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.922+0000 7f73f002f740 1 --2- 172.21.15.69:0/4131276891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129cc3c10 0x55b129cd8c40 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.922+0000 7f73f002f740 1 -- 172.21.15.69:0/4131276891 >> 172.21.15.69:0/4131276891 conn(0x55b129cbe340 msgr2=0x55b129cbfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.923+0000 7f73f002f740 1 -- 172.21.15.69:0/4131276891 shutdown_connections 2024-03-19T01:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.923+0000 7f73f002f740 1 -- 172.21.15.69:0/4131276891 wait complete. 2024-03-19T01:18:57.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.923+0000 7f73f002f740 1 -- start start 2024-03-19T01:18:57.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.924+0000 7f73f002f740 1 Processor -- start 2024-03-19T01:18:57.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.927+0000 7f73f002f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc3c10 0x55b129cd53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73f002f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cd1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73f002f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129ccd110 0x55b129cd1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73f002f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81c7810 con 0x55b129cc3c10 2024-03-19T01:18:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73f002f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81c7140 con 0x55b129ccd110 2024-03-19T01:18:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73f002f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f73e80727c0 con 0x55b129cc4530 2024-03-19T01:18:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73f002f740 10 client.0 _ll_register_callbacks cb 0x7f73d40097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73e5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cd1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73e5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cd1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44820/0 (socket says 172.21.15.69:44820) 2024-03-19T01:18:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73e5e00640 1 -- 172.21.15.69:0/1016693249 learned_addr learned my addr 172.21.15.69:0/1016693249 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73e7200640 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129ccd110 0x55b129cd1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73e5e00640 1 -- 172.21.15.69:0/1016693249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129ccd110 msgr2=0x55b129cd1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73e5e00640 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129ccd110 0x55b129cd1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.931+0000 7f73e5e00640 1 -- 172.21.15.69:0/1016693249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc3c10 msgr2=0x55b129cd53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:57.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.932+0000 7f73e6800640 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc3c10 0x55b129cd53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.932+0000 7f73e5e00640 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc3c10 0x55b129cd53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.932+0000 7f73e5e00640 1 -- 172.21.15.69:0/1016693249 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b129ccf870 con 0x55b129cc4530 2024-03-19T01:18:57.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.932+0000 7f73e7200640 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129ccd110 0x55b129cd1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:57.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.932+0000 7f73e5e00640 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cd1730 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f73d4005c50 tx=0x7f73d4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d1ad36343ee7bc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:57.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.932+0000 7f73c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:18:57.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.932+0000 7f73e6800640 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc3c10 0x55b129cd53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:57.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.933+0000 7f73c7e00640 1 -- 172.21.15.69:0/1016693249 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d402d060 con 0x55b129cc4530 2024-03-19T01:18:57.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.933+0000 7f73f002f740 1 -- 172.21.15.69:0/1016693249 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d809ed60 con 0x55b129cc4530 2024-03-19T01:18:57.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.933+0000 7f73c7e00640 1 -- 172.21.15.69:0/1016693249 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f73d4018030 con 0x55b129cc4530 2024-03-19T01:18:57.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.933+0000 7f73c7e00640 1 -- 172.21.15.69:0/1016693249 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f73d4034050 con 0x55b129cc4530 2024-03-19T01:18:57.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.933+0000 7f73f002f740 10 client.27373 fetch_fsmap learned FSMap version 51 2024-03-19T01:18:57.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.933+0000 7f73f002f740 1 -- 172.21.15.69:0/1016693249 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55b129d43f90 con 0x55b129cc4530 2024-03-19T01:18:57.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.934+0000 7f73c7e00640 1 -- 172.21.15.69:0/1016693249 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d809ed60 con 0x55b129cc4530 2024-03-19T01:18:57.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.935+0000 7f73c7e00640 1 -- 172.21.15.69:0/1016693249 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f73d401e420 con 0x55b129cc4530 2024-03-19T01:18:57.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.935+0000 7f73f002f740 10 client.27373 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:18:57.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.935+0000 7f73f002f740 10 client.27373 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:18:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.935+0000 7f73f002f740 20 client.27373 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.935+0000 7f73f002f740 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b129d44670 0x55b129d46a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.935+0000 7f73f002f740 4 client.27373 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:18:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.935+0000 7f73f002f740 1 -- 172.21.15.69:0/1016693249 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b129c2fc10 con 0x55b129d44670 2024-03-19T01:18:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.935+0000 7f73e6800640 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b129d44670 0x55b129d46a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:57.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.936+0000 7f73e6800640 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b129d44670 0x55b129d46a90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad3a6b84f7eae253 server_cookie=c428caeb93a48446 in_seq=0 out_seq=0 2024-03-19T01:18:57.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.936+0000 7f73c7e00640 10 client.27373 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:18:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73c7e00640 1 -- 172.21.15.69:0/1016693249 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b129c2fc10 con 0x55b129d44670 2024-03-19T01:18:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73c7e00640 10 client.27373 handle_command_reply: tid=0 2024-03-19T01:18:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73f002f740 1 client.27373 shutdown 2024-03-19T01:18:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73f002f740 1 -- 172.21.15.69:0/1016693249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 msgr2=0x55b129cd1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73f002f740 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cd1730 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f73d4005c50 tx=0x7f73d4004730 comp rx=0 tx=0).stop 2024-03-19T01:18:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73f002f740 1 -- 172.21.15.69:0/1016693249 shutdown_connections 2024-03-19T01:18:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73f002f740 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55b129d44670 0x55b129d46a90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73f002f740 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b129ccd110 0x55b129cd1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73f002f740 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b129cc4530 0x55b129cd1730 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73f002f740 1 --2- 172.21.15.69:0/1016693249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b129cc3c10 0x55b129cd53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73c7e00640 0 client.27373 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:18:57.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.937+0000 7f73f002f740 1 -- 172.21.15.69:0/1016693249 >> 172.21.15.69:0/1016693249 conn(0x55b129cbe340 msgr2=0x55b129cbf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:57.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.938+0000 7f73f002f740 1 -- 172.21.15.69:0/1016693249 shutdown_connections 2024-03-19T01:18:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.940+0000 7f73f002f740 1 -- 172.21.15.69:0/1016693249 wait complete. 2024-03-19T01:18:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.940+0000 7f73f002f740 20 client.27373 trim_cache size 0 max 16384 2024-03-19T01:18:57.942 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:18:57.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 -- 172.21.15.69:0/2385702638 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f73cc0a2a30 msgr2=0x7f73cc0a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:57.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f73cc0a2a30 0x7f73cc0a4ef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55b129cbf4d0 tx=0x7f73d4009260 comp rx=0 tx=0).stop 2024-03-19T01:18:57.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 -- 172.21.15.69:0/2385702638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e8107d60 msgr2=0x7f73e8081d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:57.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e8107d60 0x7f73e8081d60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f73d800d6d0 tx=0x7f73d800dad0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 -- 172.21.15.69:0/2385702638 shutdown_connections 2024-03-19T01:18:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73e810dca0 0x7f73e807af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f73cc0a2a30 0x7f73cc0a4ef0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e8108700 0x7f73e80822a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 --2- 172.21.15.69:0/2385702638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e8107d60 0x7f73e8081d60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 -- 172.21.15.69:0/2385702638 >> 172.21.15.69:0/2385702638 conn(0x7f73e80775f0 msgr2=0x7f73e80785d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 -- 172.21.15.69:0/2385702638 shutdown_connections 2024-03-19T01:18:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:57.948+0000 7f73ed400640 1 -- 172.21.15.69:0/2385702638 wait complete. 2024-03-19T01:18:58.004 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:18:58.190 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/statx.o 2024-03-19T01:18:58.309 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/ctr.o 2024-03-19T01:18:58.339 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/util.o 2024-03-19T01:18:58.521 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-stat.o 2024-03-19T01:18:58.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.558+0000 7f7461e00640 1 -- 172.21.15.69:0/3689085930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f745c109320 msgr2=0x7f745c114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:58.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.558+0000 7f7461e00640 1 --2- 172.21.15.69:0/3689085930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f745c109320 0x7f745c114430 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7448009900 tx=0x7f744802bc60 comp rx=0 tx=0).stop 2024-03-19T01:18:58.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.566+0000 7f7461e00640 1 -- 172.21.15.69:0/3689085930 shutdown_connections 2024-03-19T01:18:58.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.566+0000 7f7461e00640 1 --2- 172.21.15.69:0/3689085930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c114970 0x7f745c116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.566+0000 7f7461e00640 1 --2- 172.21.15.69:0/3689085930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f745c109320 0x7f745c114430 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.566+0000 7f7461e00640 1 --2- 172.21.15.69:0/3689085930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f745c108a00 0x7f745c108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.566+0000 7f7461e00640 1 -- 172.21.15.69:0/3689085930 >> 172.21.15.69:0/3689085930 conn(0x7f745c06c900 msgr2=0x7f745c06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.567+0000 7f7461e00640 1 -- 172.21.15.69:0/3689085930 shutdown_connections 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.567+0000 7f7461e00640 1 -- 172.21.15.69:0/3689085930 wait complete. 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.568+0000 7f7461e00640 1 Processor -- start 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.568+0000 7f7461e00640 1 -- start start 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.569+0000 7f7461e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f745c108a00 0x7f745c113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.569+0000 7f7461e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f745c109320 0x7f745c1143e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.569+0000 7f7461e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c114970 0x7f745c10b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.569+0000 7f7461e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f745c11a420 con 0x7f745c108a00 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.569+0000 7f7461e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f745c11a2a0 con 0x7f745c114970 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.569+0000 7f7461e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f745c11a5a0 con 0x7f745c109320 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.569+0000 7f745b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c114970 0x7f745c10b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.570+0000 7f745b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c114970 0x7f745c10b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58556/0 (socket says 172.21.15.69:58556) 2024-03-19T01:18:58.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.570+0000 7f745b400640 1 -- 172.21.15.69:0/3807128635 learned_addr learned my addr 172.21.15.69:0/3807128635 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:58.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.570+0000 7f745aa00640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f745c108a00 0x7f745c113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.570+0000 7f745a000640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f745c109320 0x7f745c1143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.570+0000 7f745b400640 1 -- 172.21.15.69:0/3807128635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f745c109320 msgr2=0x7f745c1143e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:58.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.570+0000 7f745b400640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f745c109320 0x7f745c1143e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.570+0000 7f745b400640 1 -- 172.21.15.69:0/3807128635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f745c108a00 msgr2=0x7f745c113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:58.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.570+0000 7f745b400640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f745c108a00 0x7f745c113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.570+0000 7f745b400640 1 -- 172.21.15.69:0/3807128635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c10b940 con 0x7f745c114970 2024-03-19T01:18:58.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.571+0000 7f745aa00640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f745c108a00 0x7f745c113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:58.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.571+0000 7f745b400640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c114970 0x7f745c10b080 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f745400b9e0 tx=0x7f745400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:58.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.571+0000 7f7460a00640 1 -- 172.21.15.69:0/3807128635 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454019070 con 0x7f745c114970 2024-03-19T01:18:58.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.571+0000 7f7460a00640 1 -- 172.21.15.69:0/3807128635 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7454014440 con 0x7f745c114970 2024-03-19T01:18:58.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.571+0000 7f7460a00640 1 -- 172.21.15.69:0/3807128635 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454012670 con 0x7f745c114970 2024-03-19T01:18:58.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.571+0000 7f7461e00640 1 -- 172.21.15.69:0/3807128635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c10bc30 con 0x7f745c114970 2024-03-19T01:18:58.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.571+0000 7f7461e00640 1 -- 172.21.15.69:0/3807128635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c10fb70 con 0x7f745c114970 2024-03-19T01:18:58.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.579+0000 7f7460a00640 1 -- 172.21.15.69:0/3807128635 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f745401c070 con 0x7f745c114970 2024-03-19T01:18:58.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.581+0000 7f7460a00640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74400a29f0 0x7f74400a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.581+0000 7f7464a1b740 1 Processor -- start 2024-03-19T01:18:58.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.581+0000 7f7460a00640 1 -- 172.21.15.69:0/3807128635 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f74540ce600 con 0x7f745c114970 2024-03-19T01:18:58.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.582+0000 7f7464a1b740 1 -- start start 2024-03-19T01:18:58.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.582+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a972ca20 0x5619a972ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.582+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a97359e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.582+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a9735f20 0x5619a9738310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.582+0000 7f7464a1b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5619a9691380 con 0x5619a972d340 2024-03-19T01:18:58.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.582+0000 7f7464a1b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5619a9639d20 con 0x5619a972ca20 2024-03-19T01:18:58.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.582+0000 7f7464a1b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5619a968d420 con 0x5619a9735f20 2024-03-19T01:18:58.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.582+0000 7f745b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a9735f20 0x5619a9738310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.582+0000 7f745b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a9735f20 0x5619a9738310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44856/0 (socket says 172.21.15.69:44856) 2024-03-19T01:18:58.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.582+0000 7f745b400640 1 -- 172.21.15.69:0/835515925 learned_addr learned my addr 172.21.15.69:0/835515925 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:58.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.583+0000 7f745b400640 1 -- 172.21.15.69:0/835515925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a972ca20 msgr2=0x5619a972ce00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:18:58.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.583+0000 7f745b400640 1 --2- 172.21.15.69:0/835515925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a972ca20 0x5619a972ce00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.583+0000 7f745b400640 1 -- 172.21.15.69:0/835515925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 msgr2=0x5619a97359e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:58.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.583+0000 7f745b400640 1 --2- 172.21.15.69:0/835515925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a97359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.583+0000 7f745b400640 1 -- 172.21.15.69:0/835515925 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a93a53f0 con 0x5619a9735f20 2024-03-19T01:18:58.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.585+0000 7f745aa00640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74400a29f0 0x7f74400a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.586+0000 7f745aa00640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74400a29f0 0x7f74400a4eb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f744c005d00 tx=0x7f744c00b410 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:58.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.587+0000 7f745b400640 1 --2- 172.21.15.69:0/835515925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a9735f20 0x5619a9738310 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f74540a2bd0 tx=0x7f745409a8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf247ee76d7ff6ac server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:58.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.587+0000 7f7447e00640 1 -- 172.21.15.69:0/835515925 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745409aa10 con 0x5619a9735f20 2024-03-19T01:18:58.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.587+0000 7f7447e00640 1 -- 172.21.15.69:0/835515925 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f745409eab0 con 0x5619a9735f20 2024-03-19T01:18:58.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.587+0000 7f7447e00640 1 -- 172.21.15.69:0/835515925 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74540a2ed0 con 0x5619a9735f20 2024-03-19T01:18:58.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.588+0000 7f7464a1b740 1 -- 172.21.15.69:0/835515925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a9735f20 msgr2=0x5619a9738310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:58.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.588+0000 7f7464a1b740 1 --2- 172.21.15.69:0/835515925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a9735f20 0x5619a9738310 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f74540a2bd0 tx=0x7f745409a8a0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.588+0000 7f7464a1b740 1 -- 172.21.15.69:0/835515925 shutdown_connections 2024-03-19T01:18:58.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.588+0000 7f7464a1b740 1 --2- 172.21.15.69:0/835515925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a9735f20 0x5619a9738310 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.588+0000 7f7464a1b740 1 --2- 172.21.15.69:0/835515925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a97359e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.588+0000 7f7464a1b740 1 --2- 172.21.15.69:0/835515925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a972ca20 0x5619a972ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.588+0000 7f7464a1b740 1 -- 172.21.15.69:0/835515925 >> 172.21.15.69:0/835515925 conn(0x5619a9727150 msgr2=0x5619a9727530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:58.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.589+0000 7f7464a1b740 1 -- 172.21.15.69:0/835515925 shutdown_connections 2024-03-19T01:18:58.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.592+0000 7f7464a1b740 1 -- 172.21.15.69:0/835515925 wait complete. 2024-03-19T01:18:58.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.592+0000 7f7464a1b740 1 -- start start 2024-03-19T01:18:58.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.593+0000 7f7464a1b740 1 Processor -- start 2024-03-19T01:18:58.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.593+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a9733000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.593+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a972d340 0x5619a9733540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.593+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a9735f20 0x5619a9733a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.594+0000 7f745a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a972d340 0x5619a9733540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.594+0000 7f745a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a972d340 0x5619a9733540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58564/0 (socket says 172.21.15.69:58564) 2024-03-19T01:18:58.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.594+0000 7f745aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a9733000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.594+0000 7f745aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a9733000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44860/0 (socket says 172.21.15.69:44860) 2024-03-19T01:18:58.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.598+0000 7f745a000640 1 -- 172.21.15.69:0/804504049 learned_addr learned my addr 172.21.15.69:0/804504049 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:58.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.598+0000 7f7464a1b740 1 -- 172.21.15.69:0/804504049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5619a9738c50 con 0x5619a9735f20 2024-03-19T01:18:58.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.598+0000 7f7464a1b740 1 -- 172.21.15.69:0/804504049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5619a964d160 con 0x5619a972d340 2024-03-19T01:18:58.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.598+0000 7f7464a1b740 1 -- 172.21.15.69:0/804504049 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5619a9738dd0 con 0x5619a972ca20 2024-03-19T01:18:58.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.598+0000 7f7464a1b740 10 client.0 _ll_register_callbacks cb 0x5619a9689170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:58.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.598+0000 7f745b400640 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a9735f20 0x5619a9733a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.599+0000 7f745b400640 1 -- 172.21.15.69:0/804504049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 msgr2=0x5619a9733000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:58.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.599+0000 7f745b400640 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a9733000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.599+0000 7f745b400640 1 -- 172.21.15.69:0/804504049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a972d340 msgr2=0x5619a9733540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:58.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.599+0000 7f745b400640 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a972d340 0x5619a9733540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.599+0000 7f745b400640 1 -- 172.21.15.69:0/804504049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a946f7d0 con 0x5619a9735f20 2024-03-19T01:18:58.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.599+0000 7f745a000640 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a972d340 0x5619a9733540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:58.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.600+0000 7f745b400640 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a9735f20 0x5619a9733a80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f745409a9a0 tx=0x7f7454014c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df6eb25e3b6c826 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:58.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.600+0000 7f7444c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:18:58.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.600+0000 7f745aa00640 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a9733000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:58.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.600+0000 7f7444c00640 1 -- 172.21.15.69:0/804504049 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74540b8060 con 0x5619a9735f20 2024-03-19T01:18:58.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.600+0000 7f7444c00640 1 -- 172.21.15.69:0/804504049 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f74540ce3a0 con 0x5619a9735f20 2024-03-19T01:18:58.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.600+0000 7f7464a1b740 1 -- 172.21.15.69:0/804504049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f745409eab0 con 0x5619a9735f20 2024-03-19T01:18:58.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.601+0000 7f7444c00640 1 -- 172.21.15.69:0/804504049 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f745409aa10 con 0x5619a9735f20 2024-03-19T01:18:58.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.601+0000 7f7464a1b740 10 client.17112 fetch_fsmap learned FSMap version 51 2024-03-19T01:18:58.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.601+0000 7f7464a1b740 1 -- 172.21.15.69:0/804504049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5619a9744da0 con 0x5619a9735f20 2024-03-19T01:18:58.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.604+0000 7f7444c00640 1 -- 172.21.15.69:0/804504049 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f745409eab0 con 0x5619a9735f20 2024-03-19T01:18:58.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.606+0000 7f7444c00640 1 -- 172.21.15.69:0/804504049 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f74540851c0 con 0x5619a9735f20 2024-03-19T01:18:58.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.606+0000 7f7464a1b740 10 client.17112 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:18:58.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.606+0000 7f7464a1b740 10 client.17112 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:18:58.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.606+0000 7f7464a1b740 20 client.17112 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:58.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.606+0000 7f7464a1b740 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5619a97460f0 0x5619a97484e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.606+0000 7f7464a1b740 4 client.17112 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:58.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.606+0000 7f7464a1b740 1 -- 172.21.15.69:0/804504049 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619a969e8c0 con 0x5619a97460f0 2024-03-19T01:18:58.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.607+0000 7f745aa00640 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5619a97460f0 0x5619a97484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.608+0000 7f745aa00640 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5619a97460f0 0x5619a97484e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53397d8779dbfba2 server_cookie=1db552f85fdf8575 in_seq=0 out_seq=0 2024-03-19T01:18:58.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.608+0000 7f7444c00640 10 client.17112 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:18:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.610+0000 7f7444c00640 1 -- 172.21.15.69:0/804504049 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5619a969e8c0 con 0x5619a97460f0 2024-03-19T01:18:58.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.610+0000 7f7444c00640 10 client.17112 handle_command_reply: tid=0 2024-03-19T01:18:58.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.610+0000 7f7464a1b740 1 client.17112 shutdown 2024-03-19T01:18:58.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.610+0000 7f7464a1b740 1 -- 172.21.15.69:0/804504049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a9735f20 msgr2=0x5619a9733a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:58.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.610+0000 7f7464a1b740 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a9735f20 0x5619a9733a80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f745409a9a0 tx=0x7f7454014c40 comp rx=0 tx=0).stop 2024-03-19T01:18:58.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.611+0000 7f7464a1b740 1 -- 172.21.15.69:0/804504049 shutdown_connections 2024-03-19T01:18:58.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.611+0000 7f7464a1b740 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a9735f20 0x5619a9733a80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.611+0000 7f7464a1b740 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5619a97460f0 0x5619a97484e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.611+0000 7f7444c00640 0 client.17112 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:18:58.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.611+0000 7f7464a1b740 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a972d340 0x5619a9733540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.611+0000 7f7464a1b740 1 --2- 172.21.15.69:0/804504049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a9733000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.611+0000 7f7464a1b740 1 -- 172.21.15.69:0/804504049 >> 172.21.15.69:0/804504049 conn(0x5619a9727150 msgr2=0x5619a9728ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:58.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.612+0000 7f7464a1b740 1 -- 172.21.15.69:0/804504049 shutdown_connections 2024-03-19T01:18:58.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.613+0000 7f7464a1b740 1 -- 172.21.15.69:0/804504049 wait complete. 2024-03-19T01:18:58.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.614+0000 7f7464a1b740 20 client.17112 trim_cache size 0 max 16384 2024-03-19T01:18:58.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.656+0000 7f7464a1b740 1 Processor -- start 2024-03-19T01:18:58.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.658+0000 7f7464a1b740 1 -- start start 2024-03-19T01:18:58.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.659+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a9735730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.659+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9730260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.659+0000 7f745aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a9735730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.659+0000 7f745aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a9735730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44876/0 (socket says 172.21.15.69:44876) 2024-03-19T01:18:58.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.660+0000 7f745a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9730260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.660+0000 7f745a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9730260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44008/0 (socket says 172.21.15.69:44008) 2024-03-19T01:18:58.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.660+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 0x5619a97307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.660+0000 7f7464a1b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1b5380 con 0x5619a972d340 2024-03-19T01:18:58.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.660+0000 7f7464a1b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1b7040 con 0x5619a9735f20 2024-03-19T01:18:58.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.660+0000 7f7464a1b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1b5a50 con 0x5619a972ca20 2024-03-19T01:18:58.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.660+0000 7f745b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 0x5619a97307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.660+0000 7f745b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 0x5619a97307a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58570/0 (socket says 172.21.15.69:58570) 2024-03-19T01:18:58.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.660+0000 7f745b400640 1 -- 172.21.15.69:0/3610190293 learned_addr learned my addr 172.21.15.69:0/3610190293 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:58.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.661+0000 7f745b400640 1 -- 172.21.15.69:0/3610190293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 msgr2=0x5619a9735730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:58.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.661+0000 7f745b400640 1 --2- 172.21.15.69:0/3610190293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a9735730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.661+0000 7f745b400640 1 -- 172.21.15.69:0/3610190293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 msgr2=0x5619a9730260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:58.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.661+0000 7f745b400640 1 --2- 172.21.15.69:0/3610190293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9730260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.661+0000 7f745b400640 1 -- 172.21.15.69:0/3610190293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a972d170 con 0x5619a9735f20 2024-03-19T01:18:58.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.661+0000 7f745a000640 1 --2- 172.21.15.69:0/3610190293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9730260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:58.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.662+0000 7f745b400640 1 --2- 172.21.15.69:0/3610190293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 0x5619a97307a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f74540973e0 tx=0x7f7454014960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d937ad0fc7aaa5ed server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:58.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.662+0000 7f7447400640 1 -- 172.21.15.69:0/3610190293 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74540b8070 con 0x5619a9735f20 2024-03-19T01:18:58.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.663+0000 7f7447400640 1 -- 172.21.15.69:0/3610190293 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f745409eab0 con 0x5619a9735f20 2024-03-19T01:18:58.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.664+0000 7f7464a1b740 1 -- 172.21.15.69:0/3610190293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 msgr2=0x5619a97307a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:58.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.664+0000 7f7464a1b740 1 --2- 172.21.15.69:0/3610190293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 0x5619a97307a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f74540973e0 tx=0x7f7454014960 comp rx=0 tx=0).stop 2024-03-19T01:18:58.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.664+0000 7f7464a1b740 1 -- 172.21.15.69:0/3610190293 shutdown_connections 2024-03-19T01:18:58.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.664+0000 7f7464a1b740 1 --2- 172.21.15.69:0/3610190293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 0x5619a97307a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.664+0000 7f7464a1b740 1 --2- 172.21.15.69:0/3610190293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9730260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.664+0000 7f7464a1b740 1 --2- 172.21.15.69:0/3610190293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a9735730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.664+0000 7f7464a1b740 1 -- 172.21.15.69:0/3610190293 >> 172.21.15.69:0/3610190293 conn(0x5619a9727150 msgr2=0x5619a9727530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:58.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.664+0000 7f7464a1b740 1 -- 172.21.15.69:0/3610190293 shutdown_connections 2024-03-19T01:18:58.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.664+0000 7f7464a1b740 1 -- 172.21.15.69:0/3610190293 wait complete. 2024-03-19T01:18:58.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.664+0000 7f7464a1b740 1 -- start start 2024-03-19T01:18:58.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.665+0000 7f7464a1b740 1 Processor -- start 2024-03-19T01:18:58.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.666+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a974b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.666+0000 7f745aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a974b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.668+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9748a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.668+0000 7f7464a1b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 0x5619a9748f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.668+0000 7f7464a1b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1b68b0 con 0x5619a972d340 2024-03-19T01:18:58.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.668+0000 7f7464a1b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1b61e0 con 0x5619a9735f20 2024-03-19T01:18:58.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.668+0000 7f7464a1b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1b33e0 con 0x5619a972ca20 2024-03-19T01:18:58.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.668+0000 7f745a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9748a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.669+0000 7f745a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9748a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44024/0 (socket says 172.21.15.69:44024) 2024-03-19T01:18:58.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.669+0000 7f745a000640 1 -- 172.21.15.69:0/560129118 learned_addr learned my addr 172.21.15.69:0/560129118 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:58.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.669+0000 7f745b400640 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 0x5619a9748f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.670+0000 7f745a000640 1 -- 172.21.15.69:0/560129118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 msgr2=0x5619a974b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:58.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.670+0000 7f745a000640 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a974b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.670+0000 7f745a000640 1 -- 172.21.15.69:0/560129118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 msgr2=0x5619a9748f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:58.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.670+0000 7f745a000640 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 0x5619a9748f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.670+0000 7f745a000640 1 -- 172.21.15.69:0/560129118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a9749660 con 0x5619a972d340 2024-03-19T01:18:58.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.670+0000 7f745b400640 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 0x5619a9748f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:58.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.670+0000 7f745a000640 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9748a30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f744802c270 tx=0x7f744802ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51c022c0f37db45 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:58.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.670+0000 7f7444c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:18:58.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.671+0000 7f7444c00640 1 -- 172.21.15.69:0/560129118 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448005380 con 0x5619a972d340 2024-03-19T01:18:58.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.671+0000 7f7464a1b740 10 client.0 _ll_register_callbacks cb 0x7f7454083f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:58.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.671+0000 7f7444c00640 1 -- 172.21.15.69:0/560129118 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f7448005520 con 0x5619a972d340 2024-03-19T01:18:58.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.671+0000 7f7464a1b740 1 -- 172.21.15.69:0/560129118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f745409eab0 con 0x5619a972d340 2024-03-19T01:18:58.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.671+0000 7f745aa00640 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a974b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:58.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.672+0000 7f7444c00640 1 -- 172.21.15.69:0/560129118 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f74480046c0 con 0x5619a972d340 2024-03-19T01:18:58.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.673+0000 7f7464a1b740 10 client.17118 fetch_fsmap learned FSMap version 51 2024-03-19T01:18:58.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.673+0000 7f7464a1b740 1 -- 172.21.15.69:0/560129118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=51}) -- 0x5619a97abe20 con 0x5619a972d340 2024-03-19T01:18:58.676 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/rapl.o 2024-03-19T01:18:58.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.677+0000 7f7444c00640 1 -- 172.21.15.69:0/560129118 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f7448045070 con 0x5619a972d340 2024-03-19T01:18:58.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.677+0000 7f7464a1b740 10 client.17118 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:18:58.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.677+0000 7f7464a1b740 10 client.17118 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:18:58.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.677+0000 7f7464a1b740 20 client.17118 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:58.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.677+0000 7f7464a1b740 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5619a97ac470 0x5619a97ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:58.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.677+0000 7f7464a1b740 4 client.17118 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:18:58.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.677+0000 7f7464a1b740 1 -- 172.21.15.69:0/560129118 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619a9698c10 con 0x5619a97ac470 2024-03-19T01:18:58.679 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/pm.o 2024-03-19T01:18:58.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.679+0000 7f745aa00640 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5619a97ac470 0x5619a97ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:58.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.679+0000 7f745aa00640 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5619a97ac470 0x5619a97ae8f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=df36f9f1a7cf0ec server_cookie=9e0ff86fdcc3dcf in_seq=0 out_seq=0 2024-03-19T01:18:58.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.681+0000 7f7444c00640 10 client.17118 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:18:58.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.681+0000 7f7444c00640 1 -- 172.21.15.69:0/560129118 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619a9698c10 con 0x5619a97ac470 2024-03-19T01:18:58.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.681+0000 7f7444c00640 10 client.17118 handle_command_reply: tid=0 2024-03-19T01:18:58.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.681+0000 7f7464a1b740 1 client.17118 shutdown 2024-03-19T01:18:58.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.681+0000 7f7464a1b740 1 -- 172.21.15.69:0/560129118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 msgr2=0x5619a9748a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:58.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.681+0000 7f7464a1b740 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9748a30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f744802c270 tx=0x7f744802ce50 comp rx=0 tx=0).stop 2024-03-19T01:18:58.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.682+0000 7f7464a1b740 1 -- 172.21.15.69:0/560129118 shutdown_connections 2024-03-19T01:18:58.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.682+0000 7f7464a1b740 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5619a97ac470 0x5619a97ae8f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.682+0000 7f7464a1b740 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619a9735f20 0x5619a9748f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.682+0000 7f7464a1b740 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619a972d340 0x5619a9748a30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.682+0000 7f7464a1b740 1 --2- 172.21.15.69:0/560129118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619a972ca20 0x5619a974b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.682+0000 7f7444c00640 0 client.17118 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:18:58.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.682+0000 7f7464a1b740 1 -- 172.21.15.69:0/560129118 >> 172.21.15.69:0/560129118 conn(0x5619a9727150 msgr2=0x5619a9725940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:58.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.683+0000 7f7464a1b740 1 -- 172.21.15.69:0/560129118 shutdown_connections 2024-03-19T01:18:58.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.683+0000 7f7464a1b740 1 -- 172.21.15.69:0/560129118 wait complete. 2024-03-19T01:18:58.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.683+0000 7f7464a1b740 20 client.17118 trim_cache size 0 max 16384 2024-03-19T01:18:58.686 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:18:58.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.685+0000 7f7461e00640 1 -- 172.21.15.69:0/3807128635 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74400a29f0 msgr2=0x7f74400a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:58.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.685+0000 7f7461e00640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74400a29f0 0x7f74400a4eb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f744c005d00 tx=0x7f744c00b410 comp rx=0 tx=0).stop 2024-03-19T01:18:58.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.685+0000 7f7461e00640 1 -- 172.21.15.69:0/3807128635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c114970 msgr2=0x7f745c10b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:58.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.685+0000 7f7461e00640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c114970 0x7f745c10b080 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f745400b9e0 tx=0x7f745400bea0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.687+0000 7f7461e00640 1 -- 172.21.15.69:0/3807128635 shutdown_connections 2024-03-19T01:18:58.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.687+0000 7f7461e00640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f74400a29f0 0x7f74400a4eb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.687+0000 7f7461e00640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c114970 0x7f745c10b080 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.687+0000 7f7461e00640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f745c109320 0x7f745c1143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.687+0000 7f7461e00640 1 --2- 172.21.15.69:0/3807128635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f745c108a00 0x7f745c113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:58.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.687+0000 7f7461e00640 1 -- 172.21.15.69:0/3807128635 >> 172.21.15.69:0/3807128635 conn(0x7f745c06c900 msgr2=0x7f745c071870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:58.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.691+0000 7f7461e00640 1 -- 172.21.15.69:0/3807128635 shutdown_connections 2024-03-19T01:18:58.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:58.692+0000 7f7461e00640 1 -- 172.21.15.69:0/3807128635 wait complete. 2024-03-19T01:18:58.745 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:18:59.030 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/gcm.o 2024-03-19T01:18:59.042 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/memory.o 2024-03-19T01:18:59.220 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/net.o 2024-03-19T01:18:59.246 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/version.o 2024-03-19T01:18:59.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.448+0000 7f09d9600640 1 -- 172.21.15.69:0/1151625336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09d4114970 msgr2=0x7f09d4116d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:59.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.448+0000 7f09d9600640 1 --2- 172.21.15.69:0/1151625336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09d4114970 0x7f09d4116d60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009900 tx=0x7f09c002bc30 comp rx=0 tx=0).stop 2024-03-19T01:18:59.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.462+0000 7f09d9600640 1 -- 172.21.15.69:0/1151625336 shutdown_connections 2024-03-19T01:18:59.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.462+0000 7f09d9600640 1 --2- 172.21.15.69:0/1151625336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09d4114970 0x7f09d4116d60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.462+0000 7f09d9600640 1 --2- 172.21.15.69:0/1151625336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09d4109320 0x7f09d4114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.462+0000 7f09d9600640 1 --2- 172.21.15.69:0/1151625336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4108a00 0x7f09d4108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.462+0000 7f09d9600640 1 -- 172.21.15.69:0/1151625336 >> 172.21.15.69:0/1151625336 conn(0x7f09d406c900 msgr2=0x7f09d406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:59.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.462+0000 7f09d9600640 1 -- 172.21.15.69:0/1151625336 shutdown_connections 2024-03-19T01:18:59.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.462+0000 7f09d9600640 1 -- 172.21.15.69:0/1151625336 wait complete. 2024-03-19T01:18:59.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.463+0000 7f09d9600640 1 Processor -- start 2024-03-19T01:18:59.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.464+0000 7f09d9600640 1 -- start start 2024-03-19T01:18:59.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.464+0000 7f09d9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4108a00 0x7f09d41a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.464+0000 7f09d9600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09d4109320 0x7f09d41a7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.464+0000 7f09d9600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09d4114970 0x7f09d41ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.464+0000 7f09d9600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4118160 con 0x7f09d4109320 2024-03-19T01:18:59.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.464+0000 7f09d9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4117fe0 con 0x7f09d4108a00 2024-03-19T01:18:59.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.464+0000 7f09d9600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41182e0 con 0x7f09d4114970 2024-03-19T01:18:59.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.465+0000 7f09d2000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09d4109320 0x7f09d41a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.465+0000 7f09d2000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09d4109320 0x7f09d41a7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44038/0 (socket says 172.21.15.69:44038) 2024-03-19T01:18:59.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.465+0000 7f09d2000640 1 -- 172.21.15.69:0/3269783225 learned_addr learned my addr 172.21.15.69:0/3269783225 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:59.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.466+0000 7f09d2000640 1 -- 172.21.15.69:0/3269783225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09d4114970 msgr2=0x7f09d41ab7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-19T01:18:59.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.466+0000 7f09d2000640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09d4114970 0x7f09d41ab7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.466+0000 7f09d2000640 1 -- 172.21.15.69:0/3269783225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4108a00 msgr2=0x7f09d41a6ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-19T01:18:59.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.466+0000 7f09d2000640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4108a00 0x7f09d41a6ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.466+0000 7f09d2000640 1 -- 172.21.15.69:0/3269783225 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d41abeb0 con 0x7f09d4109320 2024-03-19T01:18:59.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.466+0000 7f09d2000640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09d4109320 0x7f09d41a7430 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f09c400d950 tx=0x7f09c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:59.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.467+0000 7f09cb400640 1 -- 172.21.15.69:0/3269783225 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4017030 con 0x7f09d4109320 2024-03-19T01:18:59.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.467+0000 7f09cb400640 1 -- 172.21.15.69:0/3269783225 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c4004020 con 0x7f09d4109320 2024-03-19T01:18:59.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.467+0000 7f09cb400640 1 -- 172.21.15.69:0/3269783225 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4003c70 con 0x7f09d4109320 2024-03-19T01:18:59.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.468+0000 7f09d9600640 1 -- 172.21.15.69:0/3269783225 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d41ac1a0 con 0x7f09d4109320 2024-03-19T01:18:59.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.473+0000 7f09d9600640 1 -- 172.21.15.69:0/3269783225 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d419e430 con 0x7f09d4109320 2024-03-19T01:18:59.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.478+0000 7f09dc3ad740 1 Processor -- start 2024-03-19T01:18:59.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.485+0000 7f09dc3ad740 1 -- start start 2024-03-19T01:18:59.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.485+0000 7f09dc3ad740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 0x55ddea506e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.485+0000 7f09dc3ad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea507340 0x55ddea50f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.485+0000 7f09dc3ad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea50ff20 0x55ddea512310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.485+0000 7f09dc3ad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ddea46b380 con 0x55ddea507340 2024-03-19T01:18:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.485+0000 7f09dc3ad740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ddea413d20 con 0x55ddea50ff20 2024-03-19T01:18:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.485+0000 7f09dc3ad740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ddea467420 con 0x55ddea506a20 2024-03-19T01:18:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.486+0000 7f09d3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 0x55ddea506e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.486+0000 7f09d3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 0x55ddea506e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44920/0 (socket says 172.21.15.69:44920) 2024-03-19T01:18:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.486+0000 7f09d3400640 1 -- 172.21.15.69:0/3061156236 learned_addr learned my addr 172.21.15.69:0/3061156236 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.486+0000 7f09d2000640 1 --2- 172.21.15.69:0/3061156236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea50ff20 0x55ddea512310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.486+0000 7f09d2a00640 1 --2- 172.21.15.69:0/3061156236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea507340 0x55ddea50f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.486+0000 7f09cb400640 1 -- 172.21.15.69:0/3269783225 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f09c401a070 con 0x7f09d4109320 2024-03-19T01:18:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.486+0000 7f09d3400640 1 -- 172.21.15.69:0/3061156236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea50ff20 msgr2=0x55ddea512310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.486+0000 7f09d3400640 1 --2- 172.21.15.69:0/3061156236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea50ff20 0x55ddea512310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.486+0000 7f09d3400640 1 -- 172.21.15.69:0/3061156236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea507340 msgr2=0x55ddea50f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.487+0000 7f09d3400640 1 --2- 172.21.15.69:0/3061156236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea507340 0x55ddea50f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.487+0000 7f09d3400640 1 -- 172.21.15.69:0/3061156236 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddea17f3f0 con 0x55ddea506a20 2024-03-19T01:18:59.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.487+0000 7f09d2a00640 1 --2- 172.21.15.69:0/3061156236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea507340 0x55ddea50f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:18:59.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.487+0000 7f09d3400640 1 --2- 172.21.15.69:0/3061156236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 0x55ddea506e00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f09c002c1c0 tx=0x7f09c00095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eb47fc29d8fd3a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:59.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.488+0000 7f09bf400640 1 -- 172.21.15.69:0/3061156236 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0042020 con 0x55ddea506a20 2024-03-19T01:18:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.488+0000 7f09bf400640 1 -- 172.21.15.69:0/3061156236 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c002c880 con 0x55ddea506a20 2024-03-19T01:18:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.488+0000 7f09bf400640 1 -- 172.21.15.69:0/3061156236 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0003c20 con 0x55ddea506a20 2024-03-19T01:18:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.488+0000 7f09cb400640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f09b80a2b80 0x7f09b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.488+0000 7f09cb400640 1 -- 172.21.15.69:0/3269783225 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f09c40d1170 con 0x7f09d4109320 2024-03-19T01:18:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.489+0000 7f09d2a00640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f09b80a2b80 0x7f09b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.489+0000 7f09dc3ad740 1 -- 172.21.15.69:0/3061156236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 msgr2=0x55ddea506e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.489+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/3061156236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 0x55ddea506e00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f09c002c1c0 tx=0x7f09c00095f0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.490+0000 7f09d2a00640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f09b80a2b80 0x7f09b80a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55ddea5064f0 tx=0x7f09cc009600 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.490+0000 7f09dc3ad740 1 -- 172.21.15.69:0/3061156236 shutdown_connections 2024-03-19T01:18:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.490+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/3061156236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea50ff20 0x55ddea512310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.490+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/3061156236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea507340 0x55ddea50f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.490+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/3061156236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 0x55ddea506e00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.490+0000 7f09dc3ad740 1 -- 172.21.15.69:0/3061156236 >> 172.21.15.69:0/3061156236 conn(0x55ddea501150 msgr2=0x55ddea501530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:59.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.491+0000 7f09dc3ad740 1 -- 172.21.15.69:0/3061156236 shutdown_connections 2024-03-19T01:18:59.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.491+0000 7f09dc3ad740 1 -- 172.21.15.69:0/3061156236 wait complete. 2024-03-19T01:18:59.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.492+0000 7f09dc3ad740 1 -- start start 2024-03-19T01:18:59.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.493+0000 7f09dc3ad740 1 Processor -- start 2024-03-19T01:18:59.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.496+0000 7f09dc3ad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea506a20 0x55ddea50cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.496+0000 7f09dc3ad740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea507340 0x55ddea50d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.496+0000 7f09dc3ad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea50ff20 0x55ddea50da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.496+0000 7f09dc3ad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ddea512cb0 con 0x55ddea506a20 2024-03-19T01:18:59.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.496+0000 7f09dc3ad740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ddea427160 con 0x55ddea50ff20 2024-03-19T01:18:59.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.496+0000 7f09dc3ad740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ddea512e30 con 0x55ddea507340 2024-03-19T01:18:59.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.496+0000 7f09dc3ad740 10 client.0 _ll_register_callbacks cb 0x55ddea463170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:59.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.496+0000 7f09d2000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea507340 0x55ddea50d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.496+0000 7f09d2000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea507340 0x55ddea50d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44924/0 (socket says 172.21.15.69:44924) 2024-03-19T01:18:59.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.496+0000 7f09d2000640 1 -- 172.21.15.69:0/2082401758 learned_addr learned my addr 172.21.15.69:0/2082401758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:59.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.496+0000 7f09d2a00640 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea506a20 0x55ddea50cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.497+0000 7f09d2000640 1 -- 172.21.15.69:0/2082401758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea50ff20 msgr2=0x55ddea50da40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:18:59.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.497+0000 7f09d2000640 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea50ff20 0x55ddea50da40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.497+0000 7f09d2000640 1 -- 172.21.15.69:0/2082401758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea506a20 msgr2=0x55ddea50cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:59.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.501+0000 7f09d2000640 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea506a20 0x55ddea50cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.501+0000 7f09d2000640 1 -- 172.21.15.69:0/2082401758 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddea2497d0 con 0x55ddea507340 2024-03-19T01:18:59.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.501+0000 7f09d2a00640 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea506a20 0x55ddea50cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:18:59.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.501+0000 7f09d2000640 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea507340 0x55ddea50d500 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f09c409a300 tx=0x7f09c409a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a74cb37a9c0649b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:59.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.501+0000 7f09b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:18:59.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.501+0000 7f09b3e00640 1 -- 172.21.15.69:0/2082401758 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c40d2060 con 0x55ddea507340 2024-03-19T01:18:59.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.503+0000 7f09b3e00640 1 -- 172.21.15.69:0/2082401758 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f09c40041d0 con 0x55ddea507340 2024-03-19T01:18:59.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.503+0000 7f09dc3ad740 1 -- 172.21.15.69:0/2082401758 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09c002c880 con 0x55ddea507340 2024-03-19T01:18:59.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.507+0000 7f09b3e00640 1 -- 172.21.15.69:0/2082401758 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f09c409aa70 con 0x55ddea507340 2024-03-19T01:18:59.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.507+0000 7f09dc3ad740 10 client.27397 fetch_fsmap learned FSMap version 51 2024-03-19T01:18:59.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.507+0000 7f09dc3ad740 1 -- 172.21.15.69:0/2082401758 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55ddea51efb0 con 0x55ddea507340 2024-03-19T01:18:59.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.507+0000 7f09b3e00640 1 -- 172.21.15.69:0/2082401758 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c40b3220 con 0x55ddea507340 2024-03-19T01:18:59.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.507+0000 7f09b3e00640 1 -- 172.21.15.69:0/2082401758 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f09c40cb660 con 0x55ddea507340 2024-03-19T01:18:59.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.508+0000 7f09dc3ad740 10 client.27397 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:18:59.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.508+0000 7f09dc3ad740 10 client.27397 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:18:59.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.508+0000 7f09dc3ad740 20 client.27397 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:59.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.508+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ddea520410 0x55ddea522870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.508+0000 7f09dc3ad740 4 client.27397 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:18:59.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.508+0000 7f09dc3ad740 1 -- 172.21.15.69:0/2082401758 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f09c002c880 con 0x55ddea520410 2024-03-19T01:18:59.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.508+0000 7f09d2a00640 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ddea520410 0x55ddea522870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.509+0000 7f09d2a00640 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ddea520410 0x55ddea522870 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b5f2c933d53d1df server_cookie=b6538c60f4feff40 in_seq=0 out_seq=0 2024-03-19T01:18:59.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.509+0000 7f09b3e00640 10 client.27397 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:18:59.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.510+0000 7f09b3e00640 1 -- 172.21.15.69:0/2082401758 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f09c002c880 con 0x55ddea520410 2024-03-19T01:18:59.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.510+0000 7f09b3e00640 10 client.27397 handle_command_reply: tid=0 2024-03-19T01:18:59.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.510+0000 7f09dc3ad740 1 client.27397 shutdown 2024-03-19T01:18:59.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.512+0000 7f09dc3ad740 1 -- 172.21.15.69:0/2082401758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea507340 msgr2=0x55ddea50d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:59.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.512+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea507340 0x55ddea50d500 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f09c409a300 tx=0x7f09c409a140 comp rx=0 tx=0).stop 2024-03-19T01:18:59.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.512+0000 7f09dc3ad740 1 -- 172.21.15.69:0/2082401758 shutdown_connections 2024-03-19T01:18:59.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.512+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ddea520410 0x55ddea522870 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.512+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea50ff20 0x55ddea50da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.512+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea507340 0x55ddea50d500 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.512+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/2082401758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea506a20 0x55ddea50cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.512+0000 7f09b3e00640 0 client.27397 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:18:59.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.512+0000 7f09dc3ad740 1 -- 172.21.15.69:0/2082401758 >> 172.21.15.69:0/2082401758 conn(0x55ddea501150 msgr2=0x55ddea502ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:59.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.513+0000 7f09dc3ad740 1 -- 172.21.15.69:0/2082401758 shutdown_connections 2024-03-19T01:18:59.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.513+0000 7f09dc3ad740 1 -- 172.21.15.69:0/2082401758 wait complete. 2024-03-19T01:18:59.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.513+0000 7f09dc3ad740 20 client.27397 trim_cache size 0 max 16384 2024-03-19T01:18:59.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.589+0000 7f09dc3ad740 1 Processor -- start 2024-03-19T01:18:59.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.589+0000 7f09dc3ad740 1 -- start start 2024-03-19T01:18:59.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.589+0000 7f09dc3ad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea506a20 0x55ddea50f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.589+0000 7f09dc3ad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 0x55ddea51ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.589+0000 7f09dc3ad740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea50ff20 0x55ddea528140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.589+0000 7f09dc3ad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41c6630 con 0x55ddea506a20 2024-03-19T01:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.589+0000 7f09dc3ad740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41c82f0 con 0x55ddea507340 2024-03-19T01:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.589+0000 7f09dc3ad740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41c6d00 con 0x55ddea50ff20 2024-03-19T01:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.590+0000 7f09d3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea50ff20 0x55ddea528140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.590+0000 7f09d2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 0x55ddea51ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.590+0000 7f09d2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 0x55ddea51ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58630/0 (socket says 172.21.15.69:58630) 2024-03-19T01:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.590+0000 7f09d3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea50ff20 0x55ddea528140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44940/0 (socket says 172.21.15.69:44940) 2024-03-19T01:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.590+0000 7f09d2000640 1 -- 172.21.15.69:0/3826564539 learned_addr learned my addr 172.21.15.69:0/3826564539 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.590+0000 7f09d3400640 1 -- 172.21.15.69:0/3826564539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 msgr2=0x55ddea51ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.590+0000 7f09d3400640 1 --2- 172.21.15.69:0/3826564539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 0x55ddea51ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.590+0000 7f09d2a00640 1 --2- 172.21.15.69:0/3826564539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea506a20 0x55ddea50f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.590+0000 7f09d3400640 1 -- 172.21.15.69:0/3826564539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea506a20 msgr2=0x55ddea50f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.590+0000 7f09d3400640 1 --2- 172.21.15.69:0/3826564539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea506a20 0x55ddea50f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.591+0000 7f09d3400640 1 -- 172.21.15.69:0/3826564539 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddea51bf90 con 0x55ddea50ff20 2024-03-19T01:18:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.591+0000 7f09d3400640 1 --2- 172.21.15.69:0/3826564539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea50ff20 0x55ddea528140 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f09c002c1c0 tx=0x7f09c0005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0935c5752dceea4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.592+0000 7f09bea00640 1 -- 172.21.15.69:0/3826564539 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0042050 con 0x55ddea50ff20 2024-03-19T01:18:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.592+0000 7f09bea00640 1 -- 172.21.15.69:0/3826564539 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c002c880 con 0x55ddea50ff20 2024-03-19T01:18:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.592+0000 7f09bea00640 1 -- 172.21.15.69:0/3826564539 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c00039f0 con 0x55ddea50ff20 2024-03-19T01:18:59.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.593+0000 7f09dc3ad740 1 -- 172.21.15.69:0/3826564539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea50ff20 msgr2=0x55ddea528140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:59.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.593+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/3826564539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea50ff20 0x55ddea528140 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f09c002c1c0 tx=0x7f09c0005850 comp rx=0 tx=0).stop 2024-03-19T01:18:59.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.594+0000 7f09dc3ad740 1 -- 172.21.15.69:0/3826564539 shutdown_connections 2024-03-19T01:18:59.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.594+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/3826564539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea50ff20 0x55ddea528140 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.594+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/3826564539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 0x55ddea51ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.594+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/3826564539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea506a20 0x55ddea50f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.594+0000 7f09dc3ad740 1 -- 172.21.15.69:0/3826564539 >> 172.21.15.69:0/3826564539 conn(0x55ddea501150 msgr2=0x55ddea50cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:59.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.594+0000 7f09dc3ad740 1 -- 172.21.15.69:0/3826564539 shutdown_connections 2024-03-19T01:18:59.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.594+0000 7f09dc3ad740 1 -- 172.21.15.69:0/3826564539 wait complete. 2024-03-19T01:18:59.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.596+0000 7f09dc3ad740 1 -- start start 2024-03-19T01:18:59.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.596+0000 7f09dc3ad740 1 Processor -- start 2024-03-19T01:18:59.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.599+0000 7f09dc3ad740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 0x55ddea514230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.599+0000 7f09dc3ad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 0x55ddea514770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.599+0000 7f09d2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 0x55ddea514770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.599+0000 7f09d2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 0x55ddea514770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58640/0 (socket says 172.21.15.69:58640) 2024-03-19T01:18:59.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.599+0000 7f09d2000640 1 -- 172.21.15.69:0/1058774312 learned_addr learned my addr 172.21.15.69:0/1058774312 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:18:59.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.599+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea50ff20 0x55ddea57f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.600+0000 7f09d2a00640 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 0x55ddea514230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.600+0000 7f09d3400640 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea50ff20 0x55ddea57f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.601+0000 7f09dc3ad740 1 -- 172.21.15.69:0/1058774312 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41c7b60 con 0x55ddea50ff20 2024-03-19T01:18:59.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.601+0000 7f09dc3ad740 1 -- 172.21.15.69:0/1058774312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41c7490 con 0x55ddea507340 2024-03-19T01:18:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.601+0000 7f09dc3ad740 1 -- 172.21.15.69:0/1058774312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41a67b0 con 0x55ddea506a20 2024-03-19T01:18:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.601+0000 7f09dc3ad740 10 client.0 _ll_register_callbacks cb 0x7f09c4005d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:18:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.602+0000 7f09d3400640 1 -- 172.21.15.69:0/1058774312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 msgr2=0x55ddea514230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.602+0000 7f09d3400640 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 0x55ddea514230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.602+0000 7f09d3400640 1 -- 172.21.15.69:0/1058774312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 msgr2=0x55ddea514770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:18:59.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.602+0000 7f09d3400640 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 0x55ddea514770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.602+0000 7f09d3400640 1 -- 172.21.15.69:0/1058774312 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddea57fd70 con 0x55ddea50ff20 2024-03-19T01:18:59.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.603+0000 7f09d3400640 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea50ff20 0x55ddea57f7b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f09c0000c00 tx=0x7f09c0049690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691e3037d5680a9c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:18:59.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.603+0000 7f09b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:18:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.602+0000 7f09d2000640 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 0x55ddea514770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:18:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.604+0000 7f09b3e00640 1 -- 172.21.15.69:0/1058774312 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0056020 con 0x55ddea50ff20 2024-03-19T01:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.604+0000 7f09b3e00640 1 -- 172.21.15.69:0/1058774312 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f09c0049a10 con 0x55ddea50ff20 2024-03-19T01:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.604+0000 7f09dc3ad740 1 -- 172.21.15.69:0/1058774312 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09c002c880 con 0x55ddea50ff20 2024-03-19T01:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.606+0000 7f09b3e00640 1 -- 172.21.15.69:0/1058774312 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f09c0061050 con 0x55ddea50ff20 2024-03-19T01:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.606+0000 7f09dc3ad740 10 client.17133 fetch_fsmap learned FSMap version 51 2024-03-19T01:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.606+0000 7f09dc3ad740 1 -- 172.21.15.69:0/1058774312 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55ddea587120 con 0x55ddea50ff20 2024-03-19T01:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.607+0000 7f09b3e00640 1 -- 172.21.15.69:0/1058774312 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c002c880 con 0x55ddea50ff20 2024-03-19T01:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.607+0000 7f09b3e00640 1 -- 172.21.15.69:0/1058774312 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f09c0034440 con 0x55ddea50ff20 2024-03-19T01:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.607+0000 7f09dc3ad740 10 client.17133 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.607+0000 7f09dc3ad740 10 client.17133 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.607+0000 7f09dc3ad740 20 client.17133 populate_metadata read hostname 'smithi069' 2024-03-19T01:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.607+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ddea5873f0 0x55ddea5897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.607+0000 7f09dc3ad740 4 client.17133 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.607+0000 7f09dc3ad740 1 -- 172.21.15.69:0/1058774312 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddea4788c0 con 0x55ddea5873f0 2024-03-19T01:18:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.608+0000 7f09d2a00640 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ddea5873f0 0x55ddea5897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.610+0000 7f09b3e00640 10 client.17133 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.610+0000 7f09d2a00640 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ddea5873f0 0x55ddea5897e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75a131367cc8de28 server_cookie=751d288e46306511 in_seq=0 out_seq=0 2024-03-19T01:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.610+0000 7f09b3e00640 1 -- 172.21.15.69:0/1058774312 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddea4788c0 con 0x55ddea5873f0 2024-03-19T01:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.610+0000 7f09b3e00640 10 client.17133 handle_command_reply: tid=0 2024-03-19T01:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.610+0000 7f09dc3ad740 1 client.17133 shutdown 2024-03-19T01:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.610+0000 7f09dc3ad740 1 -- 172.21.15.69:0/1058774312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea50ff20 msgr2=0x55ddea57f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.610+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea50ff20 0x55ddea57f7b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f09c0000c00 tx=0x7f09c0049690 comp rx=0 tx=0).stop 2024-03-19T01:18:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.614+0000 7f09dc3ad740 1 -- 172.21.15.69:0/1058774312 shutdown_connections 2024-03-19T01:18:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.614+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55ddea5873f0 0x55ddea5897e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.614+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ddea50ff20 0x55ddea57f7b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.614+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddea507340 0x55ddea514770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.614+0000 7f09b3e00640 0 client.17133 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.614+0000 7f09dc3ad740 1 --2- 172.21.15.69:0/1058774312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ddea506a20 0x55ddea514230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.614+0000 7f09dc3ad740 1 -- 172.21.15.69:0/1058774312 >> 172.21.15.69:0/1058774312 conn(0x55ddea501150 msgr2=0x55ddea502d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.614+0000 7f09dc3ad740 1 -- 172.21.15.69:0/1058774312 shutdown_connections 2024-03-19T01:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.614+0000 7f09dc3ad740 1 -- 172.21.15.69:0/1058774312 wait complete. 2024-03-19T01:18:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.615+0000 7f09dc3ad740 20 client.17133 trim_cache size 0 max 16384 2024-03-19T01:18:59.617 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:18:59.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 -- 172.21.15.69:0/3269783225 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f09b80a2b80 msgr2=0x7f09b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:59.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f09b80a2b80 0x7f09b80a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55ddea5064f0 tx=0x7f09cc009600 comp rx=0 tx=0).stop 2024-03-19T01:18:59.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 -- 172.21.15.69:0/3269783225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09d4109320 msgr2=0x7f09d41a7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:18:59.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09d4109320 0x7f09d41a7430 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f09c400d950 tx=0x7f09c400de10 comp rx=0 tx=0).stop 2024-03-19T01:18:59.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 -- 172.21.15.69:0/3269783225 shutdown_connections 2024-03-19T01:18:59.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09d4114970 0x7f09d41ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09d4109320 0x7f09d41a7430 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f09b80a2b80 0x7f09b80a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 --2- 172.21.15.69:0/3269783225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4108a00 0x7f09d41a6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:18:59.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 -- 172.21.15.69:0/3269783225 >> 172.21.15.69:0/3269783225 conn(0x7f09d406c900 msgr2=0x7f09d4071870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:18:59.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 -- 172.21.15.69:0/3269783225 shutdown_connections 2024-03-19T01:18:59.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:18:59.624+0000 7f09d9600640 1 -- 172.21.15.69:0/3269783225 wait complete. 2024-03-19T01:18:59.630 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-mq-sysfs.o 2024-03-19T01:18:59.668 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:18:59.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:18:59 smithi086 ceph-mon[66829]: pgmap v256: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-03-19T01:18:59.762 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/msi.o 2024-03-19T01:18:59.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:18:59 smithi069 ceph-mon[31727]: pgmap v256: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-03-19T01:18:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:18:59 smithi163 ceph-mon[67428]: pgmap v256: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-03-19T01:18:59.845 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/events/msr.o 2024-03-19T01:19:00.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.305+0000 7f20ed600640 1 -- 172.21.15.69:0/3222187181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20e8114970 msgr2=0x7f20e8116d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:00.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.305+0000 7f20ed600640 1 --2- 172.21.15.69:0/3222187181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20e8114970 0x7f20e8116d60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009930 tx=0x7f20d402bc30 comp rx=0 tx=0).stop 2024-03-19T01:19:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.317+0000 7f20ed600640 1 -- 172.21.15.69:0/3222187181 shutdown_connections 2024-03-19T01:19:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.317+0000 7f20ed600640 1 --2- 172.21.15.69:0/3222187181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20e8114970 0x7f20e8116d60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.317+0000 7f20ed600640 1 --2- 172.21.15.69:0/3222187181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8109140 0x7f20e8114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.317+0000 7f20ed600640 1 --2- 172.21.15.69:0/3222187181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e8108820 0x7f20e8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.317+0000 7f20ed600640 1 -- 172.21.15.69:0/3222187181 >> 172.21.15.69:0/3222187181 conn(0x7f20e806c770 msgr2=0x7f20e806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:00.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.319+0000 7f20ed600640 1 -- 172.21.15.69:0/3222187181 shutdown_connections 2024-03-19T01:19:00.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.320+0000 7f20ed600640 1 -- 172.21.15.69:0/3222187181 wait complete. 2024-03-19T01:19:00.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.321+0000 7f20ed600640 1 Processor -- start 2024-03-19T01:19:00.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.324+0000 7f20ed600640 1 -- start start 2024-03-19T01:19:00.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.324+0000 7f20ed600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8108820 0x7f20e819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.325+0000 7f20e7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8108820 0x7f20e819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.325+0000 7f20e7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8108820 0x7f20e819e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44098/0 (socket says 172.21.15.69:44098) 2024-03-19T01:19:00.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.325+0000 7f20ed600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20e8109140 0x7f20e819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.325+0000 7f20ed600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e8114970 0x7f20e81a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.325+0000 7f20ed600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8118050 con 0x7f20e8108820 2024-03-19T01:19:00.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.325+0000 7f20e7e00640 1 -- 172.21.15.69:0/3732607935 learned_addr learned my addr 172.21.15.69:0/3732607935 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:00.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.326+0000 7f20ed600640 1 -- 172.21.15.69:0/3732607935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8117ed0 con 0x7f20e8114970 2024-03-19T01:19:00.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.326+0000 7f20ed600640 1 -- 172.21.15.69:0/3732607935 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81181d0 con 0x7f20e8109140 2024-03-19T01:19:00.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.326+0000 7f20ecc00640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e8114970 0x7f20e81a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.326+0000 7f20e7400640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20e8109140 0x7f20e819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.326+0000 7f20e7e00640 1 -- 172.21.15.69:0/3732607935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20e8109140 msgr2=0x7f20e819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:00.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.326+0000 7f20e7e00640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20e8109140 0x7f20e819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.327+0000 7f20e7e00640 1 -- 172.21.15.69:0/3732607935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e8114970 msgr2=0x7f20e81a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:00.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.327+0000 7f20e7e00640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e8114970 0x7f20e81a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.327+0000 7f20e7e00640 1 -- 172.21.15.69:0/3732607935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e81a34d0 con 0x7f20e8108820 2024-03-19T01:19:00.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.329+0000 7f20e7e00640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8108820 0x7f20e819e530 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f20e000edf0 tx=0x7f20e000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:00.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.330+0000 7f20e4c00640 1 -- 172.21.15.69:0/3732607935 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0004b20 con 0x7f20e8108820 2024-03-19T01:19:00.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.330+0000 7f20e4c00640 1 -- 172.21.15.69:0/3732607935 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e0004cc0 con 0x7f20e8108820 2024-03-19T01:19:00.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.330+0000 7f20e4c00640 1 -- 172.21.15.69:0/3732607935 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00108d0 con 0x7f20e8108820 2024-03-19T01:19:00.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.330+0000 7f20ed600640 1 -- 172.21.15.69:0/3732607935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e81a37c0 con 0x7f20e8108820 2024-03-19T01:19:00.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.330+0000 7f20ed600640 1 -- 172.21.15.69:0/3732607935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e81ab050 con 0x7f20e8108820 2024-03-19T01:19:00.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.331+0000 7f20ecc00640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e8114970 0x7f20e81a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:19:00.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.337+0000 7f20f1865740 1 Processor -- start 2024-03-19T01:19:00.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.340+0000 7f20e4c00640 1 -- 172.21.15.69:0/3732607935 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f20e0005020 con 0x7f20e8108820 2024-03-19T01:19:00.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.340+0000 7f20f1865740 1 -- start start 2024-03-19T01:19:00.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.340+0000 7f20f1865740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c09ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.340+0000 7f20f1865740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c0a530 0x55e387c12bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.340+0000 7f20f1865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c13110 0x55e387c15500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.340+0000 7f20f1865740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e387b6e380 con 0x55e387c0a530 2024-03-19T01:19:00.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.340+0000 7f20f1865740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e387b16d20 con 0x55e387c13110 2024-03-19T01:19:00.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.340+0000 7f20f1865740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e387b6a420 con 0x55e387c09c10 2024-03-19T01:19:00.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.345+0000 7f20e4c00640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f20cc0a6ec0 0x7f20cc0a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.344+0000 7f20e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c13110 0x55e387c15500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.345+0000 7f20e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c13110 0x55e387c15500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58658/0 (socket says 172.21.15.69:58658) 2024-03-19T01:19:00.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.345+0000 7f20e7400640 1 -- 172.21.15.69:0/2171427514 learned_addr learned my addr 172.21.15.69:0/2171427514 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:00.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.346+0000 7f20e7400640 1 -- 172.21.15.69:0/2171427514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 msgr2=0x55e387c09ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:00.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.346+0000 7f20e4c00640 1 -- 172.21.15.69:0/3732607935 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f20e00cf930 con 0x7f20e8108820 2024-03-19T01:19:00.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.346+0000 7f20e7e00640 1 --2- 172.21.15.69:0/2171427514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c0a530 0x55e387c12bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.346+0000 7f20e7400640 1 --2- 172.21.15.69:0/2171427514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c09ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.346+0000 7f20e7400640 1 -- 172.21.15.69:0/2171427514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c0a530 msgr2=0x55e387c12bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:00.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.346+0000 7f20e7400640 1 --2- 172.21.15.69:0/2171427514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c0a530 0x55e387c12bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.346+0000 7f20e7400640 1 -- 172.21.15.69:0/2171427514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3878823f0 con 0x55e387c13110 2024-03-19T01:19:00.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.346+0000 7f20ecc00640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f20cc0a6ec0 0x7f20cc0a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.347+0000 7f20e7e00640 1 --2- 172.21.15.69:0/2171427514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c0a530 0x55e387c12bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:00.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.347+0000 7f20e7400640 1 --2- 172.21.15.69:0/2171427514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c13110 0x55e387c15500 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f20d800bc90 tx=0x7f20d80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf00ca5dbdcbbc59 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:00.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.347+0000 7f20dd200640 1 -- 172.21.15.69:0/2171427514 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8005220 con 0x55e387c13110 2024-03-19T01:19:00.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.348+0000 7f20dd200640 1 -- 172.21.15.69:0/2171427514 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d800be40 con 0x55e387c13110 2024-03-19T01:19:00.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.348+0000 7f20dd200640 1 -- 172.21.15.69:0/2171427514 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8011840 con 0x55e387c13110 2024-03-19T01:19:00.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.349+0000 7f20ecc00640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f20cc0a6ec0 0x7f20cc0a9380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009850 tx=0x7f20d402df90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:00.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.349+0000 7f20f1865740 1 -- 172.21.15.69:0/2171427514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c13110 msgr2=0x55e387c15500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:00.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.349+0000 7f20f1865740 1 --2- 172.21.15.69:0/2171427514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c13110 0x55e387c15500 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f20d800bc90 tx=0x7f20d80107a0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.349+0000 7f20f1865740 1 -- 172.21.15.69:0/2171427514 shutdown_connections 2024-03-19T01:19:00.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.349+0000 7f20f1865740 1 --2- 172.21.15.69:0/2171427514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c13110 0x55e387c15500 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.349+0000 7f20f1865740 1 --2- 172.21.15.69:0/2171427514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c0a530 0x55e387c12bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.349+0000 7f20f1865740 1 --2- 172.21.15.69:0/2171427514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c09ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.349+0000 7f20f1865740 1 -- 172.21.15.69:0/2171427514 >> 172.21.15.69:0/2171427514 conn(0x55e387c04340 msgr2=0x55e387c04720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:00.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.349+0000 7f20f1865740 1 -- 172.21.15.69:0/2171427514 shutdown_connections 2024-03-19T01:19:00.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.352+0000 7f20f1865740 1 -- 172.21.15.69:0/2171427514 wait complete. 2024-03-19T01:19:00.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.353+0000 7f20f1865740 1 -- start start 2024-03-19T01:19:00.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.353+0000 7f20f1865740 1 Processor -- start 2024-03-19T01:19:00.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.356+0000 7f20f1865740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.357+0000 7f20f1865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c10700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.357+0000 7f20e7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.357+0000 7f20e7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c101c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44964/0 (socket says 172.21.15.69:44964) 2024-03-19T01:19:00.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.357+0000 7f20e7e00640 1 -- 172.21.15.69:0/3925426987 learned_addr learned my addr 172.21.15.69:0/3925426987 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:00.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.357+0000 7f20f1865740 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c13110 0x55e387c10c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.357+0000 7f20f1865740 1 -- 172.21.15.69:0/3925426987 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e387c15f00 con 0x55e387c13110 2024-03-19T01:19:00.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.358+0000 7f20f1865740 1 -- 172.21.15.69:0/3925426987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e387b2a160 con 0x55e387c0a530 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.358+0000 7f20f1865740 1 -- 172.21.15.69:0/3925426987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e387c16080 con 0x55e387c09c10 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.358+0000 7f20e7400640 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c10700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.358+0000 7f20f1865740 10 client.0 _ll_register_callbacks cb 0x55e387b66170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.358+0000 7f20e7e00640 1 -- 172.21.15.69:0/3925426987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 msgr2=0x55e387c10700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.358+0000 7f20e7e00640 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c10700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.358+0000 7f20e7e00640 1 -- 172.21.15.69:0/3925426987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c13110 msgr2=0x55e387c10c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.358+0000 7f20e7e00640 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c13110 0x55e387c10c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.358+0000 7f20e7e00640 1 -- 172.21.15.69:0/3925426987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e38794c7d0 con 0x55e387c09c10 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.358+0000 7f20e7e00640 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c101c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f20e0097fa0 tx=0x7f20e009c240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef1640f5ffb0707d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.359+0000 7f20f1865740 1 -- 172.21.15.69:0/3925426987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e387b7b8c0 con 0x55e387c09c10 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.359+0000 7f20d1600640 10 client.27412 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.359+0000 7f20d1600640 1 -- 172.21.15.69:0/3925426987 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00a42e0 con 0x55e387c09c10 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.359+0000 7f20d1600640 1 -- 172.21.15.69:0/3925426987 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f20e00981b0 con 0x55e387c09c10 2024-03-19T01:19:00.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.359+0000 7f20d1600640 1 -- 172.21.15.69:0/3925426987 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f20e00bdb60 con 0x55e387c09c10 2024-03-19T01:19:00.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.359+0000 7f20f1865740 10 client.27412 fetch_fsmap learned FSMap version 51 2024-03-19T01:19:00.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.359+0000 7f20f1865740 1 -- 172.21.15.69:0/3925426987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=51}) -- 0x55e387c32f20 con 0x55e387c09c10 2024-03-19T01:19:00.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.362+0000 7f20d1600640 1 -- 172.21.15.69:0/3925426987 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f20e00a3e50 con 0x55e387c09c10 2024-03-19T01:19:00.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.365+0000 7f20f1865740 10 client.27412 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:19:00.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.365+0000 7f20f1865740 10 client.27412 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:19:00.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.365+0000 7f20f1865740 20 client.27412 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:00.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.365+0000 7f20f1865740 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55e387c34270 0x55e387c36660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.365+0000 7f20f1865740 4 client.27412 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:00.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.365+0000 7f20f1865740 1 -- 172.21.15.69:0/3925426987 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e387b7b8c0 con 0x55e387c34270 2024-03-19T01:19:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.367+0000 7f20e7400640 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55e387c34270 0x55e387c36660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.368+0000 7f20e7400640 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55e387c34270 0x55e387c36660 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=436821c9c5677b6 server_cookie=13bf5d992236af22 in_seq=0 out_seq=0 2024-03-19T01:19:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.368+0000 7f20d1600640 10 client.27412 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:19:00.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.370+0000 7f20d1600640 1 -- 172.21.15.69:0/3925426987 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e387b7b8c0 con 0x55e387c34270 2024-03-19T01:19:00.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.370+0000 7f20d1600640 10 client.27412 handle_command_reply: tid=0 2024-03-19T01:19:00.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.371+0000 7f20f1865740 1 client.27412 shutdown 2024-03-19T01:19:00.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.371+0000 7f20f1865740 1 -- 172.21.15.69:0/3925426987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 msgr2=0x55e387c101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:00.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.371+0000 7f20f1865740 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c101c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f20e0097fa0 tx=0x7f20e009c240 comp rx=0 tx=0).stop 2024-03-19T01:19:00.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.371+0000 7f20f1865740 1 -- 172.21.15.69:0/3925426987 shutdown_connections 2024-03-19T01:19:00.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.371+0000 7f20f1865740 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c13110 0x55e387c10c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.371+0000 7f20f1865740 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55e387c34270 0x55e387c36660 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.371+0000 7f20f1865740 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c10700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.371+0000 7f20f1865740 1 --2- 172.21.15.69:0/3925426987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c101c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.371+0000 7f20f1865740 1 -- 172.21.15.69:0/3925426987 >> 172.21.15.69:0/3925426987 conn(0x55e387c04340 msgr2=0x55e387c06090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:00.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.372+0000 7f20d1600640 0 client.27412 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:19:00.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.372+0000 7f20f1865740 1 -- 172.21.15.69:0/3925426987 shutdown_connections 2024-03-19T01:19:00.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.372+0000 7f20f1865740 1 -- 172.21.15.69:0/3925426987 wait complete. 2024-03-19T01:19:00.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.372+0000 7f20f1865740 20 client.27412 trim_cache size 0 max 16384 2024-03-19T01:19:00.399 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/softirqs.o 2024-03-19T01:19:00.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.428+0000 7f20f1865740 1 Processor -- start 2024-03-19T01:19:00.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.429+0000 7f20f1865740 1 -- start start 2024-03-19T01:19:00.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.429+0000 7f20f1865740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c09c10 0x55e387c1eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.430+0000 7f20f1865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c10a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.430+0000 7f20f1865740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c13110 0x55e387c03260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.430+0000 7f20e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c10a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.430+0000 7f20e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c10a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58688/0 (socket says 172.21.15.69:58688) 2024-03-19T01:19:00.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.431+0000 7f20e7400640 1 -- 172.21.15.69:0/425093622 learned_addr learned my addr 172.21.15.69:0/425093622 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:00.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.431+0000 7f20ecc00640 1 --2- 172.21.15.69:0/425093622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c13110 0x55e387c03260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.431+0000 7f20f1865740 1 -- 172.21.15.69:0/425093622 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81b5250 con 0x55e387c09c10 2024-03-19T01:19:00.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.431+0000 7f20f1865740 1 -- 172.21.15.69:0/425093622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81b6f10 con 0x55e387c0a530 2024-03-19T01:19:00.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.431+0000 7f20f1865740 1 -- 172.21.15.69:0/425093622 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81b5920 con 0x55e387c13110 2024-03-19T01:19:00.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.432+0000 7f20ecc00640 1 -- 172.21.15.69:0/425093622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 msgr2=0x55e387c10a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:00.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.432+0000 7f20ecc00640 1 --2- 172.21.15.69:0/425093622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c10a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.432+0000 7f20ecc00640 1 -- 172.21.15.69:0/425093622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c09c10 msgr2=0x55e387c1eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:00.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.432+0000 7f20ecc00640 1 --2- 172.21.15.69:0/425093622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c09c10 0x55e387c1eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.432+0000 7f20ecc00640 1 -- 172.21.15.69:0/425093622 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e387c0bda0 con 0x55e387c13110 2024-03-19T01:19:00.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.432+0000 7f20e7400640 1 --2- 172.21.15.69:0/425093622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c10a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:00.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.432+0000 7f20ecc00640 1 --2- 172.21.15.69:0/425093622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c13110 0x55e387c03260 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009900 tx=0x7f20d403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d3db108e56f2406 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:00.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.434+0000 7f20d3e00640 1 -- 172.21.15.69:0/425093622 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4049040 con 0x55e387c13110 2024-03-19T01:19:00.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.434+0000 7f20d3e00640 1 -- 172.21.15.69:0/425093622 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d4052070 con 0x55e387c13110 2024-03-19T01:19:00.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.434+0000 7f20d3e00640 1 -- 172.21.15.69:0/425093622 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4003d20 con 0x55e387c13110 2024-03-19T01:19:00.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.435+0000 7f20f1865740 1 -- 172.21.15.69:0/425093622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c13110 msgr2=0x55e387c03260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:00.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.436+0000 7f20f1865740 1 --2- 172.21.15.69:0/425093622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c13110 0x55e387c03260 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009900 tx=0x7f20d403f000 comp rx=0 tx=0).stop 2024-03-19T01:19:00.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.438+0000 7f20f1865740 1 -- 172.21.15.69:0/425093622 shutdown_connections 2024-03-19T01:19:00.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.438+0000 7f20f1865740 1 --2- 172.21.15.69:0/425093622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c13110 0x55e387c03260 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.438+0000 7f20f1865740 1 --2- 172.21.15.69:0/425093622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c10a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.438+0000 7f20f1865740 1 --2- 172.21.15.69:0/425093622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c09c10 0x55e387c1eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.438+0000 7f20f1865740 1 -- 172.21.15.69:0/425093622 >> 172.21.15.69:0/425093622 conn(0x55e387c04340 msgr2=0x55e387c02b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:00.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.440+0000 7f20f1865740 1 -- 172.21.15.69:0/425093622 shutdown_connections 2024-03-19T01:19:00.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.442+0000 7f20f1865740 1 -- 172.21.15.69:0/425093622 wait complete. 2024-03-19T01:19:00.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.442+0000 7f20f1865740 1 -- start start 2024-03-19T01:19:00.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.442+0000 7f20f1865740 1 Processor -- start 2024-03-19T01:19:00.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.448+0000 7f20f1865740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c1b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.448+0000 7f20f1865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c16930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.448+0000 7f20f1865740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c13110 0x55e387c16e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.448+0000 7f20f1865740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81b6780 con 0x55e387c13110 2024-03-19T01:19:00.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.448+0000 7f20f1865740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81b60b0 con 0x55e387c0a530 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.448+0000 7f20f1865740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81b3200 con 0x55e387c09c10 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.448+0000 7f20f1865740 10 client.0 _ll_register_callbacks cb 0x7f20e00058d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.448+0000 7f20e7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c1b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.448+0000 7f20e7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c1b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44984/0 (socket says 172.21.15.69:44984) 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.448+0000 7f20e7e00640 1 -- 172.21.15.69:0/937026695 learned_addr learned my addr 172.21.15.69:0/937026695 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.449+0000 7f20e7400640 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c16930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.449+0000 7f20e7e00640 1 -- 172.21.15.69:0/937026695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 msgr2=0x55e387c16930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.449+0000 7f20e7e00640 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c16930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.449+0000 7f20e7e00640 1 -- 172.21.15.69:0/937026695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c13110 msgr2=0x55e387c16e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.449+0000 7f20e7e00640 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c13110 0x55e387c16e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.449+0000 7f20e7e00640 1 -- 172.21.15.69:0/937026695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e387c176a0 con 0x55e387c09c10 2024-03-19T01:19:00.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.450+0000 7f20e7e00640 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c1b520 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f20e009bd00 tx=0x7f20e00059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f79b622c18164d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.450+0000 7f20d1600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:19:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.450+0000 7f20d1600640 1 -- 172.21.15.69:0/937026695 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00a42e0 con 0x55e387c09c10 2024-03-19T01:19:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.450+0000 7f20d1600640 1 -- 172.21.15.69:0/937026695 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f20e00ba320 con 0x55e387c09c10 2024-03-19T01:19:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.450+0000 7f20f1865740 1 -- 172.21.15.69:0/937026695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20d4052070 con 0x55e387c09c10 2024-03-19T01:19:00.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.451+0000 7f20d1600640 1 -- 172.21.15.69:0/937026695 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f20e009af90 con 0x55e387c09c10 2024-03-19T01:19:00.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.453+0000 7f20f1865740 10 client.27418 fetch_fsmap learned FSMap version 51 2024-03-19T01:19:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.453+0000 7f20f1865740 1 -- 172.21.15.69:0/937026695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e387c9a420 con 0x55e387c09c10 2024-03-19T01:19:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.453+0000 7f20d1600640 1 -- 172.21.15.69:0/937026695 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d4052070 con 0x55e387c09c10 2024-03-19T01:19:00.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.453+0000 7f20d1600640 1 -- 172.21.15.69:0/937026695 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f20e009aa00 con 0x55e387c09c10 2024-03-19T01:19:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.454+0000 7f20f1865740 10 client.27418 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:19:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.454+0000 7f20f1865740 10 client.27418 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:19:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.454+0000 7f20f1865740 20 client.27418 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.454+0000 7f20f1865740 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55e387c9a960 0x55e387c9cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.454+0000 7f20f1865740 4 client.27418 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:19:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.454+0000 7f20f1865740 1 -- 172.21.15.69:0/937026695 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e387b75c10 con 0x55e387c9a960 2024-03-19T01:19:00.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.454+0000 7f20e7400640 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55e387c9a960 0x55e387c9cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:00.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.458+0000 7f20d1600640 10 client.27418 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:19:00.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.458+0000 7f20e7400640 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55e387c9a960 0x55e387c9cd50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d754708c740e5f1 server_cookie=17bfd59287be0ddc in_seq=0 out_seq=0 2024-03-19T01:19:00.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.463+0000 7f20d1600640 1 -- 172.21.15.69:0/937026695 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e387b75c10 con 0x55e387c9a960 2024-03-19T01:19:00.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.463+0000 7f20d1600640 10 client.27418 handle_command_reply: tid=0 2024-03-19T01:19:00.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.463+0000 7f20f1865740 1 client.27418 shutdown 2024-03-19T01:19:00.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.464+0000 7f20f1865740 1 -- 172.21.15.69:0/937026695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 msgr2=0x55e387c1b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:00.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.464+0000 7f20f1865740 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c1b520 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f20e009bd00 tx=0x7f20e00059d0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.464+0000 7f20f1865740 1 -- 172.21.15.69:0/937026695 shutdown_connections 2024-03-19T01:19:00.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.464+0000 7f20f1865740 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e387c13110 0x55e387c16e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.464+0000 7f20f1865740 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55e387c9a960 0x55e387c9cd50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.464+0000 7f20f1865740 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e387c0a530 0x55e387c16930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.464+0000 7f20f1865740 1 --2- 172.21.15.69:0/937026695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e387c09c10 0x55e387c1b520 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.464+0000 7f20f1865740 1 -- 172.21.15.69:0/937026695 >> 172.21.15.69:0/937026695 conn(0x55e387c04340 msgr2=0x55e387c033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:00.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.464+0000 7f20d1600640 0 client.27418 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:19:00.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.466+0000 7f20f1865740 1 -- 172.21.15.69:0/937026695 shutdown_connections 2024-03-19T01:19:00.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.466+0000 7f20f1865740 1 -- 172.21.15.69:0/937026695 wait complete. 2024-03-19T01:19:00.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.466+0000 7f20f1865740 20 client.27418 trim_cache size 0 max 16384 2024-03-19T01:19:00.469 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:19:00.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.469+0000 7f20ed600640 1 -- 172.21.15.69:0/3732607935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f20cc0a6ec0 msgr2=0x7f20cc0a9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:00.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.469+0000 7f20ed600640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f20cc0a6ec0 0x7f20cc0a9380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009850 tx=0x7f20d402df90 comp rx=0 tx=0).stop 2024-03-19T01:19:00.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.469+0000 7f20ed600640 1 -- 172.21.15.69:0/3732607935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8108820 msgr2=0x7f20e819e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:00.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.469+0000 7f20ed600640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8108820 0x7f20e819e530 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f20e000edf0 tx=0x7f20e000c6a0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.470+0000 7f20ed600640 1 -- 172.21.15.69:0/3732607935 shutdown_connections 2024-03-19T01:19:00.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.470+0000 7f20ed600640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f20cc0a6ec0 0x7f20cc0a9380 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.470+0000 7f20ed600640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e8114970 0x7f20e81a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.470+0000 7f20ed600640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20e8109140 0x7f20e819ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.470+0000 7f20ed600640 1 --2- 172.21.15.69:0/3732607935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8108820 0x7f20e819e530 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:00.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.470+0000 7f20ed600640 1 -- 172.21.15.69:0/3732607935 >> 172.21.15.69:0/3732607935 conn(0x7f20e806c770 msgr2=0x7f20e8070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:00.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.470+0000 7f20ed600640 1 -- 172.21.15.69:0/3732607935 shutdown_connections 2024-03-19T01:19:00.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:00.470+0000 7f20ed600640 1 -- 172.21.15.69:0/3732607935 wait complete. 2024-03-19T01:19:00.518 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:19:00.531 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/ccm.o 2024-03-19T01:19:00.752 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:00 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:19:00.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:00 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:19:00.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:00 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:19:00.928 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-mq-cpumap.o 2024-03-19T01:19:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.101+0000 7f7b10400640 1 -- 172.21.15.69:0/3578830723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b0806dab0 msgr2=0x7f7b08108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:01.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.101+0000 7f7b10400640 1 --2- 172.21.15.69:0/3578830723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b0806dab0 0x7f7b08108820 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009900 tx=0x7f7af402bc40 comp rx=0 tx=0).stop 2024-03-19T01:19:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.103+0000 7f7b10400640 1 -- 172.21.15.69:0/3578830723 shutdown_connections 2024-03-19T01:19:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.103+0000 7f7b10400640 1 --2- 172.21.15.69:0/3578830723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b081144c0 0x7f7b081168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.103+0000 7f7b10400640 1 --2- 172.21.15.69:0/3578830723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b08108d60 0x7f7b081091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.103+0000 7f7b10400640 1 --2- 172.21.15.69:0/3578830723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b0806dab0 0x7f7b08108820 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.103+0000 7f7b10400640 1 -- 172.21.15.69:0/3578830723 >> 172.21.15.69:0/3578830723 conn(0x7f7b0806c6d0 msgr2=0x7f7b0806cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:01.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.103+0000 7f7b10400640 1 -- 172.21.15.69:0/3578830723 shutdown_connections 2024-03-19T01:19:01.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.103+0000 7f7b10400640 1 -- 172.21.15.69:0/3578830723 wait complete. 2024-03-19T01:19:01.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.104+0000 7f7b10400640 1 Processor -- start 2024-03-19T01:19:01.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.106+0000 7f7b10400640 1 -- start start 2024-03-19T01:19:01.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.107+0000 7f7b10400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b0806dab0 0x7f7b081a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.107+0000 7f7b10400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b08108d60 0x7f7b081a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.107+0000 7f7b10400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b081144c0 0x7f7b081a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.107+0000 7f7b10400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0811a1c0 con 0x7f7b08108d60 2024-03-19T01:19:01.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.107+0000 7f7b10400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0811a040 con 0x7f7b0806dab0 2024-03-19T01:19:01.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.107+0000 7f7b10400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0811a340 con 0x7f7b081144c0 2024-03-19T01:19:01.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.107+0000 7f7b0e400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b081144c0 0x7f7b081a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.107+0000 7f7b0da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b0806dab0 0x7f7b081a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.107+0000 7f7b0e400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b081144c0 0x7f7b081a4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44990/0 (socket says 172.21.15.69:44990) 2024-03-19T01:19:01.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.107+0000 7f7b0e400640 1 -- 172.21.15.69:0/3197723703 learned_addr learned my addr 172.21.15.69:0/3197723703 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:01.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.108+0000 7f7b0e400640 1 -- 172.21.15.69:0/3197723703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b0806dab0 msgr2=0x7f7b081a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:01.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.108+0000 7f7b0e400640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b0806dab0 0x7f7b081a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.108+0000 7f7b0e400640 1 -- 172.21.15.69:0/3197723703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b08108d60 msgr2=0x7f7b081a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:01.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.108+0000 7f7b0d000640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b08108d60 0x7f7b081a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.108+0000 7f7b0e400640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b08108d60 0x7f7b081a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.108+0000 7f7b0e400640 1 -- 172.21.15.69:0/3197723703 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b081a55f0 con 0x7f7b081144c0 2024-03-19T01:19:01.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.108+0000 7f7b0da00640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b0806dab0 0x7f7b081a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:19:01.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.108+0000 7f7b0d000640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b08108d60 0x7f7b081a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:01.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.108+0000 7f7b0e400640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b081144c0 0x7f7b081a4ec0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00be00 tx=0x7f7afc00dd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:01.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.109+0000 7f7b06000640 1 -- 172.21.15.69:0/3197723703 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc018070 con 0x7f7b081144c0 2024-03-19T01:19:01.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.109+0000 7f7b10400640 1 -- 172.21.15.69:0/3197723703 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b081a58e0 con 0x7f7b081144c0 2024-03-19T01:19:01.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.109+0000 7f7b06000640 1 -- 172.21.15.69:0/3197723703 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7afc009c30 con 0x7f7b081144c0 2024-03-19T01:19:01.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.109+0000 7f7b06000640 1 -- 172.21.15.69:0/3197723703 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0035f0 con 0x7f7b081144c0 2024-03-19T01:19:01.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.109+0000 7f7b10400640 1 -- 172.21.15.69:0/3197723703 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0810a1c0 con 0x7f7b081144c0 2024-03-19T01:19:01.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.119+0000 7f7b06000640 1 -- 172.21.15.69:0/3197723703 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f7afc025440 con 0x7f7b081144c0 2024-03-19T01:19:01.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.119+0000 7f7b1321d740 1 Processor -- start 2024-03-19T01:19:01.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.121+0000 7f7b06000640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7af00a2a70 0x7f7af00a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.121+0000 7f7b06000640 1 -- 172.21.15.69:0/3197723703 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f7afc0d2d00 con 0x7f7b081144c0 2024-03-19T01:19:01.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.123+0000 7f7b0d000640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7af00a2a70 0x7f7af00a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.124+0000 7f7b0d000640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7af00a2a70 0x7f7af00a4f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b000016d0 tx=0x7f7b000093d0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:01.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.126+0000 7f7b1321d740 1 -- start start 2024-03-19T01:19:01.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.126+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d15dc10 0x55d96d15dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.126+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 0x55d96d166bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.126+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d169500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.126+0000 7f7b1321d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d96d0c2380 con 0x55d96d15e530 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.126+0000 7f7b1321d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d96d06ad20 con 0x55d96d167110 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.126+0000 7f7b1321d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d96d0be420 con 0x55d96d15dc10 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.127+0000 7f7b0da00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d15dc10 0x55d96d15dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.127+0000 7f7b0da00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d15dc10 0x55d96d15dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45002/0 (socket says 172.21.15.69:45002) 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.127+0000 7f7b0da00640 1 -- 172.21.15.69:0/2300645204 learned_addr learned my addr 172.21.15.69:0/2300645204 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.127+0000 7f7b0e400640 1 --2- 172.21.15.69:0/2300645204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d169500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.127+0000 7f7b0da00640 1 -- 172.21.15.69:0/2300645204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 msgr2=0x55d96d169500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.127+0000 7f7b0da00640 1 --2- 172.21.15.69:0/2300645204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d169500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.127+0000 7f7b0da00640 1 -- 172.21.15.69:0/2300645204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 msgr2=0x55d96d166bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.128+0000 7f7b0d000640 1 --2- 172.21.15.69:0/2300645204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 0x55d96d166bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.128+0000 7f7b0da00640 1 --2- 172.21.15.69:0/2300645204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 0x55d96d166bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.128+0000 7f7b0da00640 1 -- 172.21.15.69:0/2300645204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96cdd63f0 con 0x55d96d15dc10 2024-03-19T01:19:01.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.128+0000 7f7b0d000640 1 --2- 172.21.15.69:0/2300645204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 0x55d96d166bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:01.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.132+0000 7f7b0da00640 1 --2- 172.21.15.69:0/2300645204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d15dc10 0x55d96d15dff0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009230 tx=0x7f7af4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b7b12d32c4dc20 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:01.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.132+0000 7f7afaa00640 1 -- 172.21.15.69:0/2300645204 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4042070 con 0x55d96d15dc10 2024-03-19T01:19:01.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.132+0000 7f7afaa00640 1 -- 172.21.15.69:0/2300645204 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af4004c70 con 0x55d96d15dc10 2024-03-19T01:19:01.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.132+0000 7f7afaa00640 1 -- 172.21.15.69:0/2300645204 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4005320 con 0x55d96d15dc10 2024-03-19T01:19:01.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.133+0000 7f7b1321d740 1 -- 172.21.15.69:0/2300645204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d15dc10 msgr2=0x55d96d15dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:01.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.133+0000 7f7b1321d740 1 --2- 172.21.15.69:0/2300645204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d15dc10 0x55d96d15dff0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009230 tx=0x7f7af4000bc0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.135+0000 7f7b1321d740 1 -- 172.21.15.69:0/2300645204 shutdown_connections 2024-03-19T01:19:01.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.135+0000 7f7b1321d740 1 --2- 172.21.15.69:0/2300645204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d169500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.135+0000 7f7b1321d740 1 --2- 172.21.15.69:0/2300645204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 0x55d96d166bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.135+0000 7f7b1321d740 1 --2- 172.21.15.69:0/2300645204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d15dc10 0x55d96d15dff0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.135+0000 7f7b1321d740 1 -- 172.21.15.69:0/2300645204 >> 172.21.15.69:0/2300645204 conn(0x55d96d158340 msgr2=0x55d96d158720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:01.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.135+0000 7f7b1321d740 1 -- 172.21.15.69:0/2300645204 shutdown_connections 2024-03-19T01:19:01.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.135+0000 7f7b1321d740 1 -- 172.21.15.69:0/2300645204 wait complete. 2024-03-19T01:19:01.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.135+0000 7f7b1321d740 1 -- start start 2024-03-19T01:19:01.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.136+0000 7f7b1321d740 1 Processor -- start 2024-03-19T01:19:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.138+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d15dc10 0x55d96d164190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.138+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 0x55d96d1646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.138+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d167110 0x55d96d164c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.138+0000 7f7b1321d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d96d169ea0 con 0x55d96d15e530 2024-03-19T01:19:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.138+0000 7f7b0e400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d167110 0x55d96d164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.138+0000 7f7b0e400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d167110 0x55d96d164c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45010/0 (socket says 172.21.15.69:45010) 2024-03-19T01:19:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.138+0000 7f7b0e400640 1 -- 172.21.15.69:0/2868266546 learned_addr learned my addr 172.21.15.69:0/2868266546 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.138+0000 7f7b1321d740 1 -- 172.21.15.69:0/2868266546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d96d07e160 con 0x55d96d15dc10 2024-03-19T01:19:01.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.138+0000 7f7b1321d740 1 -- 172.21.15.69:0/2868266546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d96d16a020 con 0x55d96d167110 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.139+0000 7f7b0d000640 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 0x55d96d1646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.139+0000 7f7b1321d740 10 client.0 _ll_register_callbacks cb 0x55d96d0ba170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.139+0000 7f7b0d000640 1 -- 172.21.15.69:0/2868266546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d167110 msgr2=0x55d96d164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.139+0000 7f7b0d000640 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d167110 0x55d96d164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.139+0000 7f7b0d000640 1 -- 172.21.15.69:0/2868266546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d15dc10 msgr2=0x55d96d164190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.139+0000 7f7b0d000640 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d15dc10 0x55d96d164190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.139+0000 7f7b0d000640 1 -- 172.21.15.69:0/2868266546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96cea07d0 con 0x55d96d15e530 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.139+0000 7f7b1321d740 1 -- 172.21.15.69:0/2868266546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d96d0cf8c0 con 0x55d96d15e530 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.140+0000 7f7b0d000640 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 0x55d96d1646d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b00000b80 tx=0x7f7b00001350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb65bc21bab8051 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.140+0000 7f7aef400640 10 client.17145 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.141+0000 7f7aef400640 1 -- 172.21.15.69:0/2868266546 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00024640 con 0x55d96d15e530 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.141+0000 7f7aef400640 1 -- 172.21.15.69:0/2868266546 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f7b000247e0 con 0x55d96d15e530 2024-03-19T01:19:01.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.141+0000 7f7b0e400640 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d167110 0x55d96d164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:19:01.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.141+0000 7f7aef400640 1 -- 172.21.15.69:0/2868266546 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f7b00024a20 con 0x55d96d15e530 2024-03-19T01:19:01.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.141+0000 7f7b1321d740 10 client.17145 fetch_fsmap learned FSMap version 51 2024-03-19T01:19:01.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.141+0000 7f7b1321d740 1 -- 172.21.15.69:0/2868266546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=51}) -- 0x55d96d17e940 con 0x55d96d15e530 2024-03-19T01:19:01.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.142+0000 7f7aef400640 1 -- 172.21.15.69:0/2868266546 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f7b00022020 con 0x55d96d15e530 2024-03-19T01:19:01.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.142+0000 7f7b1321d740 10 client.17145 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:19:01.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.142+0000 7f7b1321d740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:01.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.142+0000 7f7b1321d740 20 client.17145 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:01.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.142+0000 7f7b1321d740 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d96d17fd70 0x55d96d1821b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.142+0000 7f7b1321d740 4 client.17145 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:01.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.142+0000 7f7b1321d740 1 -- 172.21.15.69:0/2868266546 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d96d0c9c10 con 0x55d96d17fd70 2024-03-19T01:19:01.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.142+0000 7f7b0da00640 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d96d17fd70 0x55d96d1821b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.143+0000 7f7b0da00640 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d96d17fd70 0x55d96d1821b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f781d0a8fadc4c server_cookie=85a3b7b1685500c9 in_seq=0 out_seq=0 2024-03-19T01:19:01.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.145+0000 7f7aef400640 10 client.17145 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:01.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.145+0000 7f7aef400640 1 -- 172.21.15.69:0/2868266546 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d96d0c9c10 con 0x55d96d17fd70 2024-03-19T01:19:01.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.145+0000 7f7aef400640 10 client.17145 handle_command_reply: tid=0 2024-03-19T01:19:01.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.145+0000 7f7b1321d740 1 client.17145 shutdown 2024-03-19T01:19:01.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.148+0000 7f7b1321d740 1 -- 172.21.15.69:0/2868266546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 msgr2=0x55d96d1646d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:01.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.148+0000 7f7b1321d740 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 0x55d96d1646d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b00000b80 tx=0x7f7b00001350 comp rx=0 tx=0).stop 2024-03-19T01:19:01.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.148+0000 7f7b1321d740 1 -- 172.21.15.69:0/2868266546 shutdown_connections 2024-03-19T01:19:01.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.148+0000 7f7b1321d740 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d96d17fd70 0x55d96d1821b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.148+0000 7f7b1321d740 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d167110 0x55d96d164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.148+0000 7f7b1321d740 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15e530 0x55d96d1646d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.148+0000 7f7b1321d740 1 --2- 172.21.15.69:0/2868266546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d15dc10 0x55d96d164190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.148+0000 7f7aef400640 0 client.17145 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:01.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.148+0000 7f7b1321d740 1 -- 172.21.15.69:0/2868266546 >> 172.21.15.69:0/2868266546 conn(0x55d96d158340 msgr2=0x55d96d15a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:01.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.150+0000 7f7b1321d740 1 -- 172.21.15.69:0/2868266546 shutdown_connections 2024-03-19T01:19:01.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.150+0000 7f7b1321d740 1 -- 172.21.15.69:0/2868266546 wait complete. 2024-03-19T01:19:01.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.150+0000 7f7b1321d740 20 client.17145 trim_cache size 0 max 16384 2024-03-19T01:19:01.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.205+0000 7f7b1321d740 1 Processor -- start 2024-03-19T01:19:01.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.205+0000 7f7b1321d740 1 -- start start 2024-03-19T01:19:01.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.205+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d15dc10 0x55d96d17b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.205+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d164a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.206+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d163530 0x55d96d187a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.206+0000 7f7b1321d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b5190 con 0x55d96d163530 2024-03-19T01:19:01.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.206+0000 7f7b1321d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b6e50 con 0x55d96d167110 2024-03-19T01:19:01.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.206+0000 7f7b1321d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b5860 con 0x55d96d15dc10 2024-03-19T01:19:01.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.206+0000 7f7b0e400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d163530 0x55d96d187a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.206+0000 7f7b0d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.206+0000 7f7b0d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d164a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58726/0 (socket says 172.21.15.69:58726) 2024-03-19T01:19:01.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.206+0000 7f7b0d000640 1 -- 172.21.15.69:0/783292790 learned_addr learned my addr 172.21.15.69:0/783292790 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:01.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.207+0000 7f7b0e400640 1 -- 172.21.15.69:0/783292790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d15dc10 msgr2=0x55d96d17b4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:19:01.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.207+0000 7f7b0e400640 1 --2- 172.21.15.69:0/783292790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d15dc10 0x55d96d17b4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.207+0000 7f7b0e400640 1 -- 172.21.15.69:0/783292790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 msgr2=0x55d96d164a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:01.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.207+0000 7f7b0e400640 1 --2- 172.21.15.69:0/783292790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d164a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.207+0000 7f7b0e400640 1 -- 172.21.15.69:0/783292790 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96d17ba10 con 0x55d96d163530 2024-03-19T01:19:01.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.208+0000 7f7b0e400640 1 --2- 172.21.15.69:0/783292790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d163530 0x55d96d187a80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7afc09ed20 tx=0x7f7afc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23545cdf61e99cff server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:01.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.210+0000 7f7afa000640 1 -- 172.21.15.69:0/783292790 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc09f020 con 0x55d96d163530 2024-03-19T01:19:01.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.210+0000 7f7afa000640 1 -- 172.21.15.69:0/783292790 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7afc0a3170 con 0x55d96d163530 2024-03-19T01:19:01.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.210+0000 7f7afa000640 1 -- 172.21.15.69:0/783292790 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0a7590 con 0x55d96d163530 2024-03-19T01:19:01.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.212+0000 7f7b1321d740 1 -- 172.21.15.69:0/783292790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d163530 msgr2=0x55d96d187a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:01.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.212+0000 7f7b1321d740 1 --2- 172.21.15.69:0/783292790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d163530 0x55d96d187a80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7afc09ed20 tx=0x7f7afc005bd0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.214+0000 7f7b1321d740 1 -- 172.21.15.69:0/783292790 shutdown_connections 2024-03-19T01:19:01.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.214+0000 7f7b1321d740 1 --2- 172.21.15.69:0/783292790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d163530 0x55d96d187a80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.214+0000 7f7b1321d740 1 --2- 172.21.15.69:0/783292790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d164a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.214+0000 7f7b1321d740 1 --2- 172.21.15.69:0/783292790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d15dc10 0x55d96d17b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.214+0000 7f7b1321d740 1 -- 172.21.15.69:0/783292790 >> 172.21.15.69:0/783292790 conn(0x55d96d158340 msgr2=0x55d96d15a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:01.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.214+0000 7f7b1321d740 1 -- 172.21.15.69:0/783292790 shutdown_connections 2024-03-19T01:19:01.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.214+0000 7f7b1321d740 1 -- 172.21.15.69:0/783292790 wait complete. 2024-03-19T01:19:01.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.214+0000 7f7b1321d740 1 -- start start 2024-03-19T01:19:01.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.215+0000 7f7b1321d740 1 Processor -- start 2024-03-19T01:19:01.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.219+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15dc10 0x55d96d177b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.219+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d163530 0x55d96d174ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.220+0000 7f7b1321d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d175010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.220+0000 7f7b0d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d163530 0x55d96d174ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.220+0000 7f7b0d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d163530 0x55d96d174ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45028/0 (socket says 172.21.15.69:45028) 2024-03-19T01:19:01.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.220+0000 7f7b0e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d175010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.220+0000 7f7b0e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d175010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58742/0 (socket says 172.21.15.69:58742) 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.220+0000 7f7b0e400640 1 -- 172.21.15.69:0/917758429 learned_addr learned my addr 172.21.15.69:0/917758429 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.220+0000 7f7b1321d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b66c0 con 0x55d96d15dc10 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.220+0000 7f7b1321d740 1 -- 172.21.15.69:0/917758429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b5ff0 con 0x55d96d167110 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.220+0000 7f7b1321d740 1 -- 172.21.15.69:0/917758429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7b081b3140 con 0x55d96d163530 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.221+0000 7f7b0da00640 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15dc10 0x55d96d177b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.221+0000 7f7b0da00640 1 -- 172.21.15.69:0/917758429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d163530 msgr2=0x55d96d174ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.221+0000 7f7b0da00640 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d163530 0x55d96d174ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.222+0000 7f7b0da00640 1 -- 172.21.15.69:0/917758429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 msgr2=0x55d96d175010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.222+0000 7f7b0da00640 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d175010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.222+0000 7f7b0da00640 1 -- 172.21.15.69:0/917758429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96d175550 con 0x55d96d15dc10 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.222+0000 7f7b0e400640 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d175010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.222+0000 7f7b1321d740 10 client.0 _ll_register_callbacks cb 0x7f7b00009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:01.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.222+0000 7f7b1321d740 1 -- 172.21.15.69:0/917758429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7afc0a3170 con 0x55d96d15dc10 2024-03-19T01:19:01.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.226+0000 7f7b0da00640 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15dc10 0x55d96d177b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009900 tx=0x7f7af4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09010986225e9f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:01.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.226+0000 7f7aef400640 10 client.17151 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:19:01.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.226+0000 7f7aef400640 1 -- 172.21.15.69:0/917758429 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4042040 con 0x55d96d15dc10 2024-03-19T01:19:01.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.226+0000 7f7aef400640 1 -- 172.21.15.69:0/917758429 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f7af4050d20 con 0x55d96d15dc10 2024-03-19T01:19:01.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.226+0000 7f7aef400640 1 -- 172.21.15.69:0/917758429 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f7af4009520 con 0x55d96d15dc10 2024-03-19T01:19:01.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.227+0000 7f7b1321d740 10 client.17151 fetch_fsmap learned FSMap version 51 2024-03-19T01:19:01.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.227+0000 7f7b1321d740 1 -- 172.21.15.69:0/917758429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=51}) -- 0x55d96d1e6ad0 con 0x55d96d15dc10 2024-03-19T01:19:01.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.227+0000 7f7aef400640 1 -- 172.21.15.69:0/917758429 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f7af4004040 con 0x55d96d15dc10 2024-03-19T01:19:01.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.227+0000 7f7b1321d740 10 client.17151 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:19:01.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.227+0000 7f7b1321d740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:01.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.227+0000 7f7b1321d740 20 client.17151 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:01.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.227+0000 7f7b1321d740 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d96d1e6d50 0x55d96d1e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:01.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.228+0000 7f7b1321d740 4 client.17151 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:19:01.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.228+0000 7f7b1321d740 1 -- 172.21.15.69:0/917758429 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d96d047560 con 0x55d96d1e6d50 2024-03-19T01:19:01.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.231+0000 7f7b0d000640 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d96d1e6d50 0x55d96d1e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:01.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.231+0000 7f7b0d000640 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d96d1e6d50 0x55d96d1e91e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ad083bbb3e7ef6 server_cookie=6e360c4c83ec6ce8 in_seq=0 out_seq=0 2024-03-19T01:19:01.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.231+0000 7f7aef400640 10 client.17151 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:01.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.231+0000 7f7aef400640 1 -- 172.21.15.69:0/917758429 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d96d047560 con 0x55d96d1e6d50 2024-03-19T01:19:01.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.231+0000 7f7aef400640 10 client.17151 handle_command_reply: tid=0 2024-03-19T01:19:01.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.235+0000 7f7b1321d740 1 client.17151 shutdown 2024-03-19T01:19:01.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.237+0000 7f7b1321d740 1 -- 172.21.15.69:0/917758429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15dc10 msgr2=0x55d96d177b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:01.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.237+0000 7f7b1321d740 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15dc10 0x55d96d177b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009900 tx=0x7f7af4009260 comp rx=0 tx=0).stop 2024-03-19T01:19:01.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.237+0000 7f7b1321d740 1 -- 172.21.15.69:0/917758429 shutdown_connections 2024-03-19T01:19:01.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.237+0000 7f7b1321d740 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d96d1e6d50 0x55d96d1e91e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.237+0000 7f7b1321d740 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d96d167110 0x55d96d175010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.237+0000 7f7b1321d740 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d96d163530 0x55d96d174ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.237+0000 7f7b1321d740 1 --2- 172.21.15.69:0/917758429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d96d15dc10 0x55d96d177b00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.239 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:19:01.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.237+0000 7f7aef400640 0 client.17151 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:01.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.237+0000 7f7b1321d740 1 -- 172.21.15.69:0/917758429 >> 172.21.15.69:0/917758429 conn(0x55d96d158340 msgr2=0x55d96d156b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:01.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.238+0000 7f7b1321d740 1 -- 172.21.15.69:0/917758429 shutdown_connections 2024-03-19T01:19:01.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.238+0000 7f7b1321d740 1 -- 172.21.15.69:0/917758429 wait complete. 2024-03-19T01:19:01.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.238+0000 7f7b1321d740 20 client.17151 trim_cache size 0 max 16384 2024-03-19T01:19:01.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.239+0000 7f7b10400640 1 -- 172.21.15.69:0/3197723703 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7af00a2a70 msgr2=0x7f7af00a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:01.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.239+0000 7f7b10400640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7af00a2a70 0x7f7af00a4f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b000016d0 tx=0x7f7b000093d0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.239+0000 7f7b10400640 1 -- 172.21.15.69:0/3197723703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b081144c0 msgr2=0x7f7b081a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:01.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.239+0000 7f7b10400640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b081144c0 0x7f7b081a4ec0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00be00 tx=0x7f7afc00dd20 comp rx=0 tx=0).stop 2024-03-19T01:19:01.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.241+0000 7f7b10400640 1 -- 172.21.15.69:0/3197723703 shutdown_connections 2024-03-19T01:19:01.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.241+0000 7f7b10400640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b081144c0 0x7f7b081a4ec0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.241+0000 7f7b10400640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b08108d60 0x7f7b081a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.241+0000 7f7b10400640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7af00a2a70 0x7f7af00a4f30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.241+0000 7f7b10400640 1 --2- 172.21.15.69:0/3197723703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b0806dab0 0x7f7b081a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:01.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.241+0000 7f7b10400640 1 -- 172.21.15.69:0/3197723703 >> 172.21.15.69:0/3197723703 conn(0x7f7b0806c6d0 msgr2=0x7f7b080700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:01.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.242+0000 7f7b10400640 1 -- 172.21.15.69:0/3197723703 shutdown_connections 2024-03-19T01:19:01.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:01.245+0000 7f7b10400640 1 -- 172.21.15.69:0/3197723703 wait complete. 2024-03-19T01:19:01.500 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/msg_ring.o 2024-03-19T01:19:01.598 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/events/built-in.a 2024-03-19T01:19:01.726 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/affinity.o 2024-03-19T01:19:01.735 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/namespaces.o 2024-03-19T01:19:01.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:01 smithi086 ceph-mon[66829]: pgmap v257: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 929 KiB/s wr, 94 op/s 2024-03-19T01:19:01.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:01 smithi069 ceph-mon[31727]: pgmap v257: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 929 KiB/s wr, 94 op/s 2024-03-19T01:19:01.808 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/fpu/init.o 2024-03-19T01:19:01.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:01 smithi163 ceph-mon[67428]: pgmap v257: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 929 KiB/s wr, 94 op/s 2024-03-19T01:19:01.848 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/ss/conditional.o 2024-03-19T01:19:02.024 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/aes_generic.o 2024-03-19T01:19:02.260 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-mq-sched.o 2024-03-19T01:19:02.368 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:19:02.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:19:02.369 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:19:02.464 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/fpu/bugs.o 2024-03-19T01:19:02.782 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/self.o 2024-03-19T01:19:02.856 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/fpu/core.o 2024-03-19T01:19:02.931 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/timeout.o 2024-03-19T01:19:02.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.990+0000 7f1a7ee00640 1 -- 172.21.15.69:0/436632484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a700ac100 msgr2=0x7f1a700b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.990+0000 7f1a7ee00640 1 --2- 172.21.15.69:0/436632484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a700ac100 0x7f1a700b3c60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009900 tx=0x7f1a6c02bcb0 comp rx=0 tx=0).stop 2024-03-19T01:19:02.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.992+0000 7f1a7ee00640 1 -- 172.21.15.69:0/436632484 shutdown_connections 2024-03-19T01:19:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.992+0000 7f1a7ee00640 1 --2- 172.21.15.69:0/436632484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a700ac100 0x7f1a700b3c60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.992+0000 7f1a7ee00640 1 --2- 172.21.15.69:0/436632484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a700a5970 0x7f1a700ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.992+0000 7f1a7ee00640 1 --2- 172.21.15.69:0/436632484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a700a4fc0 0x7f1a700a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.992+0000 7f1a7ee00640 1 -- 172.21.15.69:0/436632484 >> 172.21.15.69:0/436632484 conn(0x7f1a7001a640 msgr2=0x7f1a7001aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.992+0000 7f1a7ee00640 1 -- 172.21.15.69:0/436632484 shutdown_connections 2024-03-19T01:19:02.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.994+0000 7f1a7ee00640 1 -- 172.21.15.69:0/436632484 wait complete. 2024-03-19T01:19:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.995+0000 7f1a7ee00640 1 Processor -- start 2024-03-19T01:19:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.995+0000 7f1a7ee00640 1 -- start start 2024-03-19T01:19:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:02.996+0000 7f1a7ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a700a4fc0 0x7f1a700a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.000+0000 7f1a7ee00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a700ac100 0x7f1a700a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.000+0000 7f1a7ee00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a700a8480 0x7f1a700a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.000+0000 7f1a7ee00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a700b7210 con 0x7f1a700a8480 2024-03-19T01:19:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.000+0000 7f1a7ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a700b7090 con 0x7f1a700a4fc0 2024-03-19T01:19:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.000+0000 7f1a7ee00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1a700b7390 con 0x7f1a700ac100 2024-03-19T01:19:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.000+0000 7f1a7d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a700ac100 0x7f1a700a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.000+0000 7f1a7d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a700ac100 0x7f1a700a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45042/0 (socket says 172.21.15.69:45042) 2024-03-19T01:19:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.000+0000 7f1a7d000640 1 -- 172.21.15.69:0/2046959495 learned_addr learned my addr 172.21.15.69:0/2046959495 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.000+0000 7f1a7da00640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a700a4fc0 0x7f1a700a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.000+0000 7f1a7e400640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a700a8480 0x7f1a700a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.001+0000 7f1a7da00640 1 -- 172.21.15.69:0/2046959495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a700ac100 msgr2=0x7f1a700a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.001+0000 7f1a7da00640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a700ac100 0x7f1a700a6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.001+0000 7f1a7da00640 1 -- 172.21.15.69:0/2046959495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a700a8480 msgr2=0x7f1a700a7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.001+0000 7f1a7da00640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a700a8480 0x7f1a700a7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.001+0000 7f1a7da00640 1 -- 172.21.15.69:0/2046959495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a701501f0 con 0x7f1a700a4fc0 2024-03-19T01:19:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.001+0000 7f1a7e400640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a700a8480 0x7f1a700a7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:19:03.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.001+0000 7f1a7da00640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a700a4fc0 0x7f1a700a67e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009fd0 tx=0x7f1a6400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:03.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.004+0000 7f1a76000640 1 -- 172.21.15.69:0/2046959495 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64015020 con 0x7f1a700a4fc0 2024-03-19T01:19:03.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.006+0000 7f1a76000640 1 -- 172.21.15.69:0/2046959495 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a64011d10 con 0x7f1a700a4fc0 2024-03-19T01:19:03.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.006+0000 7f1a76000640 1 -- 172.21.15.69:0/2046959495 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400f2e0 con 0x7f1a700a4fc0 2024-03-19T01:19:03.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.006+0000 7f1a7ee00640 1 -- 172.21.15.69:0/2046959495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a701504e0 con 0x7f1a700a4fc0 2024-03-19T01:19:03.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.006+0000 7f1a7ee00640 1 -- 172.21.15.69:0/2046959495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a701509a0 con 0x7f1a700a4fc0 2024-03-19T01:19:03.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.018+0000 7f1a76000640 1 -- 172.21.15.69:0/2046959495 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f1a64018020 con 0x7f1a700a4fc0 2024-03-19T01:19:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.020+0000 7f1a83033740 1 Processor -- start 2024-03-19T01:19:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.021+0000 7f1a76000640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1a600a2ac0 0x7f1a600a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.021+0000 7f1a76000640 1 -- 172.21.15.69:0/2046959495 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f1a640ce900 con 0x7f1a700a4fc0 2024-03-19T01:19:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.021+0000 7f1a7e400640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1a600a2ac0 0x7f1a600a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.022+0000 7f1a83033740 1 -- start start 2024-03-19T01:19:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.025+0000 7f1a83033740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 0x5595929c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.025+0000 7f1a83033740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c9600 0x5595929d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.025+0000 7f1a83033740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 0x5595929d45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.025+0000 7f1a83033740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55959292bfb0 con 0x5595929c9600 2024-03-19T01:19:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.025+0000 7f1a83033740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595928d4ab0 con 0x5595929c8ce0 2024-03-19T01:19:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.025+0000 7f1a83033740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559592928050 con 0x5595929d21e0 2024-03-19T01:19:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.025+0000 7f1a7da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 0x5595929c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.025+0000 7f1a7da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 0x5595929c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58766/0 (socket says 172.21.15.69:58766) 2024-03-19T01:19:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.025+0000 7f1a7da00640 1 -- 172.21.15.69:0/2857207049 learned_addr learned my addr 172.21.15.69:0/2857207049 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.025+0000 7f1a7d000640 1 --2- 172.21.15.69:0/2857207049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c9600 0x5595929d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.028+0000 7f1a7da00640 1 -- 172.21.15.69:0/2857207049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 msgr2=0x5595929d45d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:03.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.028+0000 7f1a7da00640 1 --2- 172.21.15.69:0/2857207049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 0x5595929d45d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.028+0000 7f1a7da00640 1 -- 172.21.15.69:0/2857207049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c9600 msgr2=0x5595929d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:03.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.028+0000 7f1a7e400640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1a600a2ac0 0x7f1a600a4f80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1a700a77c0 tx=0x7f1a6c005a00 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:03.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.030+0000 7f1a7da00640 1 --2- 172.21.15.69:0/2857207049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c9600 0x5595929d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.030+0000 7f1a7da00640 1 -- 172.21.15.69:0/2857207049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595928f2e00 con 0x5595929c8ce0 2024-03-19T01:19:03.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.030+0000 7f1a7d000640 1 --2- 172.21.15.69:0/2857207049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c9600 0x5595929d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:19:03.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.030+0000 7f1a7da00640 1 --2- 172.21.15.69:0/2857207049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 0x5595929c90c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1a64097630 tx=0x7f1a6409a690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8332cf656cd2f450 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:03.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.030+0000 7f1a6aa00640 1 -- 172.21.15.69:0/2857207049 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6409edb0 con 0x5595929c8ce0 2024-03-19T01:19:03.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.030+0000 7f1a6aa00640 1 -- 172.21.15.69:0/2857207049 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a640a3400 con 0x5595929c8ce0 2024-03-19T01:19:03.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.030+0000 7f1a6aa00640 1 -- 172.21.15.69:0/2857207049 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6409a0b0 con 0x5595929c8ce0 2024-03-19T01:19:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.032+0000 7f1a83033740 1 -- 172.21.15.69:0/2857207049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 msgr2=0x5595929c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.032+0000 7f1a83033740 1 --2- 172.21.15.69:0/2857207049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 0x5595929c90c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1a64097630 tx=0x7f1a6409a690 comp rx=0 tx=0).stop 2024-03-19T01:19:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.033+0000 7f1a83033740 1 -- 172.21.15.69:0/2857207049 shutdown_connections 2024-03-19T01:19:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.033+0000 7f1a83033740 1 --2- 172.21.15.69:0/2857207049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 0x5595929d45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.033+0000 7f1a83033740 1 --2- 172.21.15.69:0/2857207049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c9600 0x5595929d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.033+0000 7f1a83033740 1 --2- 172.21.15.69:0/2857207049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 0x5595929c90c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.033+0000 7f1a83033740 1 -- 172.21.15.69:0/2857207049 >> 172.21.15.69:0/2857207049 conn(0x5595929c3410 msgr2=0x5595929c37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.033+0000 7f1a83033740 1 -- 172.21.15.69:0/2857207049 shutdown_connections 2024-03-19T01:19:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.033+0000 7f1a83033740 1 -- 172.21.15.69:0/2857207049 wait complete. 2024-03-19T01:19:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.033+0000 7f1a83033740 1 -- start start 2024-03-19T01:19:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.034+0000 7f1a83033740 1 Processor -- start 2024-03-19T01:19:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.034+0000 7f1a83033740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 0x5595929cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.034+0000 7f1a83033740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c9600 0x5595929cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a83033740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 0x5595929cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a83033740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5595929d4f70 con 0x5595929c8ce0 2024-03-19T01:19:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a83033740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595928e7f00 con 0x5595929c9600 2024-03-19T01:19:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a7d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c9600 0x5595929cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a7d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c9600 0x5595929cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58772/0 (socket says 172.21.15.69:58772) 2024-03-19T01:19:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a7d000640 1 -- 172.21.15.69:0/2752025786 learned_addr learned my addr 172.21.15.69:0/2752025786 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a83033740 1 -- 172.21.15.69:0/2752025786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5595929d50f0 con 0x5595929d21e0 2024-03-19T01:19:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a7e400640 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 0x5595929cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a83033740 10 client.0 _ll_register_callbacks cb 0x5595928e78d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a7da00640 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 0x5595929cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a7d000640 1 -- 172.21.15.69:0/2752025786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 msgr2=0x5595929cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a7d000640 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 0x5595929cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a7d000640 1 -- 172.21.15.69:0/2752025786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 msgr2=0x5595929cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a7d000640 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 0x5595929cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a7d000640 1 -- 172.21.15.69:0/2752025786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595927087e0 con 0x5595929c9600 2024-03-19T01:19:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.035+0000 7f1a7da00640 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 0x5595929cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:19:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.036+0000 7f1a7d000640 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c9600 0x5595929cf790 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1a7804f850 tx=0x7f1a78074af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f070a829a6a1a143 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.036+0000 7f1a5f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:19:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.036+0000 7f1a5f400640 1 -- 172.21.15.69:0/2752025786 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78071670 con 0x5595929c9600 2024-03-19T01:19:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.036+0000 7f1a5f400640 1 -- 172.21.15.69:0/2752025786 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f1a78071810 con 0x5595929c9600 2024-03-19T01:19:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.036+0000 7f1a83033740 1 -- 172.21.15.69:0/2752025786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a640a3400 con 0x5595929c9600 2024-03-19T01:19:03.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.040+0000 7f1a5f400640 1 -- 172.21.15.69:0/2752025786 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1a78071a10 con 0x5595929c9600 2024-03-19T01:19:03.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.040+0000 7f1a83033740 10 client.27473 fetch_fsmap learned FSMap version 51 2024-03-19T01:19:03.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.040+0000 7f1a83033740 1 -- 172.21.15.69:0/2752025786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x559592768f70 con 0x5595929c9600 2024-03-19T01:19:03.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.040+0000 7f1a5f400640 1 -- 172.21.15.69:0/2752025786 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a640a3400 con 0x5595929c9600 2024-03-19T01:19:03.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.040+0000 7f1a5f400640 1 -- 172.21.15.69:0/2752025786 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f1a78088420 con 0x5595929c9600 2024-03-19T01:19:03.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.041+0000 7f1a83033740 10 client.27473 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:19:03.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.041+0000 7f1a83033740 10 client.27473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:03.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.041+0000 7f1a83033740 20 client.27473 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:03.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.041+0000 7f1a83033740 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595929e2500 0x5595929e48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.041+0000 7f1a83033740 4 client.27473 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:03.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.041+0000 7f1a83033740 1 -- 172.21.15.69:0/2752025786 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595929394f0 con 0x5595929e2500 2024-03-19T01:19:03.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.041+0000 7f1a7da00640 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595929e2500 0x5595929e48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.042+0000 7f1a7da00640 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595929e2500 0x5595929e48f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3041e4f1ab34771b server_cookie=3bd5324a645d69b5 in_seq=0 out_seq=0 2024-03-19T01:19:03.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.042+0000 7f1a5f400640 10 client.27473 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:03.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a5f400640 1 -- 172.21.15.69:0/2752025786 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5595929394f0 con 0x5595929e2500 2024-03-19T01:19:03.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a5f400640 10 client.27473 handle_command_reply: tid=0 2024-03-19T01:19:03.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a83033740 1 client.27473 shutdown 2024-03-19T01:19:03.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a83033740 1 -- 172.21.15.69:0/2752025786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c9600 msgr2=0x5595929cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a83033740 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c9600 0x5595929cf790 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1a7804f850 tx=0x7f1a78074af0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a83033740 1 -- 172.21.15.69:0/2752025786 shutdown_connections 2024-03-19T01:19:03.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a83033740 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5595929e2500 0x5595929e48f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a83033740 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 0x5595929cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a83033740 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c9600 0x5595929cf790 secure :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1a7804f850 tx=0x7f1a78074af0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a83033740 1 --2- 172.21.15.69:0/2752025786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 0x5595929cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a5f400640 0 client.27473 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:03.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.043+0000 7f1a83033740 1 -- 172.21.15.69:0/2752025786 >> 172.21.15.69:0/2752025786 conn(0x5595929c3410 msgr2=0x5595929c5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:03.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.044+0000 7f1a83033740 1 -- 172.21.15.69:0/2752025786 shutdown_connections 2024-03-19T01:19:03.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.046+0000 7f1a83033740 1 -- 172.21.15.69:0/2752025786 wait complete. 2024-03-19T01:19:03.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.046+0000 7f1a83033740 20 client.27473 trim_cache size 0 max 16384 2024-03-19T01:19:03.106 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mincore.o 2024-03-19T01:19:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.116+0000 7f1a83033740 1 Processor -- start 2024-03-19T01:19:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.116+0000 7f1a83033740 1 -- start start 2024-03-19T01:19:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.117+0000 7f1a83033740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 0x5595929d1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.117+0000 7f1a83033740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929d21e0 0x5595929ddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.118+0000 7f1a7d000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929d21e0 0x5595929ddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.118+0000 7f1a7d000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929d21e0 0x5595929ddd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44222/0 (socket says 172.21.15.69:44222) 2024-03-19T01:19:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.118+0000 7f1a7d000640 1 -- 172.21.15.69:0/924131593 learned_addr learned my addr 172.21.15.69:0/924131593 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.119+0000 7f1a83033740 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929ea200 0x5595929ea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.119+0000 7f1a83033740 1 -- 172.21.15.69:0/924131593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7015a8e0 con 0x5595929d21e0 2024-03-19T01:19:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.119+0000 7f1a7e400640 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929ea200 0x5595929ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.119+0000 7f1a83033740 1 -- 172.21.15.69:0/924131593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7015c5a0 con 0x5595929c8ce0 2024-03-19T01:19:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.119+0000 7f1a83033740 1 -- 172.21.15.69:0/924131593 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7015afb0 con 0x5595929ea200 2024-03-19T01:19:03.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.120+0000 7f1a7e400640 1 -- 172.21.15.69:0/924131593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 msgr2=0x5595929d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.121+0000 7f1a7da00640 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 0x5595929d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.121+0000 7f1a7e400640 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 0x5595929d1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.121+0000 7f1a7e400640 1 -- 172.21.15.69:0/924131593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929d21e0 msgr2=0x5595929ddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.121+0000 7f1a7e400640 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929d21e0 0x5595929ddd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.121+0000 7f1a7e400640 1 -- 172.21.15.69:0/924131593 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595929d4940 con 0x5595929ea200 2024-03-19T01:19:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.121+0000 7f1a7da00640 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 0x5595929d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.121+0000 7f1a7d000640 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929d21e0 0x5595929ddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:19:03.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.121+0000 7f1a7e400640 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929ea200 0x5595929ea640 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c036040 tx=0x7f1a6c03e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=596f6edcdff28538 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:03.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.122+0000 7f1a6a000640 1 -- 172.21.15.69:0/924131593 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c005240 con 0x5595929ea200 2024-03-19T01:19:03.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.122+0000 7f1a6a000640 1 -- 172.21.15.69:0/924131593 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a6c044070 con 0x5595929ea200 2024-03-19T01:19:03.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.122+0000 7f1a6a000640 1 -- 172.21.15.69:0/924131593 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c0035f0 con 0x5595929ea200 2024-03-19T01:19:03.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.123+0000 7f1a83033740 1 -- 172.21.15.69:0/924131593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929ea200 msgr2=0x5595929ea640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.123+0000 7f1a83033740 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929ea200 0x5595929ea640 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c036040 tx=0x7f1a6c03e000 comp rx=0 tx=0).stop 2024-03-19T01:19:03.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.123+0000 7f1a83033740 1 -- 172.21.15.69:0/924131593 shutdown_connections 2024-03-19T01:19:03.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.124+0000 7f1a83033740 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929ea200 0x5595929ea640 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.124+0000 7f1a83033740 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929d21e0 0x5595929ddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.124+0000 7f1a83033740 1 --2- 172.21.15.69:0/924131593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929c8ce0 0x5595929d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.124+0000 7f1a83033740 1 -- 172.21.15.69:0/924131593 >> 172.21.15.69:0/924131593 conn(0x5595929c3410 msgr2=0x5595929cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.124+0000 7f1a83033740 1 -- 172.21.15.69:0/924131593 shutdown_connections 2024-03-19T01:19:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.124+0000 7f1a83033740 1 -- 172.21.15.69:0/924131593 wait complete. 2024-03-19T01:19:03.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.125+0000 7f1a83033740 1 -- start start 2024-03-19T01:19:03.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.126+0000 7f1a83033740 1 Processor -- start 2024-03-19T01:19:03.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.127+0000 7f1a83033740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 0x5595929d7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.127+0000 7f1a7da00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 0x5595929d7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.128+0000 7f1a7da00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 0x5595929d7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44224/0 (socket says 172.21.15.69:44224) 2024-03-19T01:19:03.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.128+0000 7f1a7da00640 1 -- 172.21.15.69:0/695288623 learned_addr learned my addr 172.21.15.69:0/695288623 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:03.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.128+0000 7f1a83033740 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 0x5595929d7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.128+0000 7f1a83033740 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929ea200 0x5595929d7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.128+0000 7f1a7d000640 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 0x5595929d7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.129+0000 7f1a83033740 1 -- 172.21.15.69:0/695288623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7015be10 con 0x5595929c8ce0 2024-03-19T01:19:03.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.129+0000 7f1a7e400640 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929ea200 0x5595929d7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.129+0000 7f1a83033740 1 -- 172.21.15.69:0/695288623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7015b740 con 0x5595929ea200 2024-03-19T01:19:03.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.129+0000 7f1a83033740 1 -- 172.21.15.69:0/695288623 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70158890 con 0x5595929d21e0 2024-03-19T01:19:03.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.130+0000 7f1a7da00640 1 -- 172.21.15.69:0/695288623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 msgr2=0x5595929d7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:03.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.130+0000 7f1a7da00640 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 0x5595929d7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.130+0000 7f1a7da00640 1 -- 172.21.15.69:0/695288623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929ea200 msgr2=0x5595929d7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:03.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.130+0000 7f1a7da00640 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929ea200 0x5595929d7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.130+0000 7f1a7da00640 1 -- 172.21.15.69:0/695288623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559592600170 con 0x5595929c8ce0 2024-03-19T01:19:03.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.130+0000 7f1a7e400640 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929ea200 0x5595929d7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:03.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.131+0000 7f1a7da00640 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 0x5595929d7310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1a64006dc0 tx=0x7f1a640109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb15e13e26d452b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:03.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.131+0000 7f1a5f400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:19:03.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.132+0000 7f1a83033740 10 client.0 _ll_register_callbacks cb 0x7f1a78073d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:03.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.132+0000 7f1a5f400640 1 -- 172.21.15.69:0/695288623 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a640a3400 con 0x5595929c8ce0 2024-03-19T01:19:03.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.132+0000 7f1a5f400640 1 -- 172.21.15.69:0/695288623 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7f1a640cbec0 con 0x5595929c8ce0 2024-03-19T01:19:03.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.132+0000 7f1a83033740 1 -- 172.21.15.69:0/695288623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a6c044070 con 0x5595929c8ce0 2024-03-19T01:19:03.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.132+0000 7f1a5f400640 1 -- 172.21.15.69:0/695288623 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1a640cc0c0 con 0x5595929c8ce0 2024-03-19T01:19:03.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.132+0000 7f1a83033740 10 client.17166 fetch_fsmap learned FSMap version 51 2024-03-19T01:19:03.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.133+0000 7f1a83033740 1 -- 172.21.15.69:0/695288623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=51}) -- 0x559592a497f0 con 0x5595929c8ce0 2024-03-19T01:19:03.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.133+0000 7f1a5f400640 1 -- 172.21.15.69:0/695288623 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7f1a640b92b0 con 0x5595929c8ce0 2024-03-19T01:19:03.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.133+0000 7f1a83033740 10 client.17166 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:19:03.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.133+0000 7f1a83033740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:03.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.133+0000 7f1a83033740 20 client.17166 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:03.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.133+0000 7f1a83033740 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559592a4a060 0x559592a4c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.133+0000 7f1a83033740 4 client.17166 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:19:03.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.133+0000 7f1a83033740 1 -- 172.21.15.69:0/695288623 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559592933840 con 0x559592a4a060 2024-03-19T01:19:03.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.134+0000 7f1a7d000640 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559592a4a060 0x559592a4c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.134+0000 7f1a5f400640 10 client.17166 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:03.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.134+0000 7f1a7d000640 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559592a4a060 0x559592a4c520 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50faefbf5139a585 server_cookie=e6b09d4b0f8e9772 in_seq=0 out_seq=0 2024-03-19T01:19:03.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a5f400640 1 -- 172.21.15.69:0/695288623 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559592933840 con 0x559592a4a060 2024-03-19T01:19:03.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a5f400640 10 client.17166 handle_command_reply: tid=0 2024-03-19T01:19:03.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a83033740 1 client.17166 shutdown 2024-03-19T01:19:03.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a83033740 1 -- 172.21.15.69:0/695288623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 msgr2=0x5595929d7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a83033740 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 0x5595929d7310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1a64006dc0 tx=0x7f1a640109e0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a83033740 1 -- 172.21.15.69:0/695288623 shutdown_connections 2024-03-19T01:19:03.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a83033740 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559592a4a060 0x559592a4c520 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a83033740 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595929ea200 0x5595929d7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a83033740 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595929d21e0 0x5595929d7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a83033740 1 --2- 172.21.15.69:0/695288623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5595929c8ce0 0x5595929d7310 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a83033740 1 -- 172.21.15.69:0/695288623 >> 172.21.15.69:0/695288623 conn(0x5595929c3410 msgr2=0x5595929e91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:03.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.136+0000 7f1a5f400640 0 client.17166 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:03.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.140+0000 7f1a83033740 1 -- 172.21.15.69:0/695288623 shutdown_connections 2024-03-19T01:19:03.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.140+0000 7f1a83033740 1 -- 172.21.15.69:0/695288623 wait complete. 2024-03-19T01:19:03.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.140+0000 7f1a83033740 20 client.17166 trim_cache size 0 max 16384 2024-03-19T01:19:03.141 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:19:03.141 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:19:03.141 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "253a821b-e147-4849-9883-b34c16f98f02", 2024-03-19T01:19:03.141 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:19:03.141 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:19:03.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.144+0000 7f1a7ee00640 1 -- 172.21.15.69:0/2046959495 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1a600a2ac0 msgr2=0x7f1a600a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.144+0000 7f1a7ee00640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1a600a2ac0 0x7f1a600a4f80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1a700a77c0 tx=0x7f1a6c005a00 comp rx=0 tx=0).stop 2024-03-19T01:19:03.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.144+0000 7f1a7ee00640 1 -- 172.21.15.69:0/2046959495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a700a4fc0 msgr2=0x7f1a700a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.144+0000 7f1a7ee00640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a700a4fc0 0x7f1a700a67e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009fd0 tx=0x7f1a6400b870 comp rx=0 tx=0).stop 2024-03-19T01:19:03.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.145+0000 7f1a7ee00640 1 -- 172.21.15.69:0/2046959495 shutdown_connections 2024-03-19T01:19:03.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.145+0000 7f1a7ee00640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a700a8480 0x7f1a700a7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.145+0000 7f1a7ee00640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a700ac100 0x7f1a700a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.145+0000 7f1a7ee00640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1a600a2ac0 0x7f1a600a4f80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.145+0000 7f1a7ee00640 1 --2- 172.21.15.69:0/2046959495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a700a4fc0 0x7f1a700a67e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.145+0000 7f1a7ee00640 1 -- 172.21.15.69:0/2046959495 >> 172.21.15.69:0/2046959495 conn(0x7f1a7001a640 msgr2=0x7f1a700a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:03.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.146+0000 7f1a7ee00640 1 -- 172.21.15.69:0/2046959495 shutdown_connections 2024-03-19T01:19:03.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.146+0000 7f1a7ee00640 1 -- 172.21.15.69:0/2046959495 wait complete. 2024-03-19T01:19:03.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:19:03.192 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:19:03.487 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/crc32c_generic.o 2024-03-19T01:19:03.647 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/ss/mls.o 2024-03-19T01:19:03.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:03 smithi086 ceph-mon[66829]: pgmap v258: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 872 KiB/s wr, 92 op/s 2024-03-19T01:19:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:03 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:19:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:03 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:19:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:03 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:19:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:03 smithi069 ceph-mon[31727]: pgmap v258: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 872 KiB/s wr, 92 op/s 2024-03-19T01:19:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:03 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:19:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:03 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:19:03.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:03 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:19:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:03 smithi163 ceph-mon[67428]: pgmap v258: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 872 KiB/s wr, 92 op/s 2024-03-19T01:19:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:03 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:19:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:03 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:19:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:03 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:19:03.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.888+0000 7fbeda200640 1 -- 172.21.15.69:0/4160452487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed411b000 msgr2=0x7fbed411d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.888+0000 7fbeda200640 1 --2- 172.21.15.69:0/4160452487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed411b000 0x7fbed411d3f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009ca0 tx=0x7fbecc02c100 comp rx=0 tx=0).stop 2024-03-19T01:19:03.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.888+0000 7fbeda200640 1 -- 172.21.15.69:0/4160452487 shutdown_connections 2024-03-19T01:19:03.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.888+0000 7fbeda200640 1 --2- 172.21.15.69:0/4160452487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed411b000 0x7fbed411d3f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.888+0000 7fbeda200640 1 --2- 172.21.15.69:0/4160452487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed410e070 0x7fbed410e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.888+0000 7fbeda200640 1 --2- 172.21.15.69:0/4160452487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed4110040 0x7fbed410db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.888+0000 7fbeda200640 1 -- 172.21.15.69:0/4160452487 >> 172.21.15.69:0/4160452487 conn(0x7fbed406c900 msgr2=0x7fbed406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.889+0000 7fbeda200640 1 -- 172.21.15.69:0/4160452487 shutdown_connections 2024-03-19T01:19:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.889+0000 7fbeda200640 1 -- 172.21.15.69:0/4160452487 wait complete. 2024-03-19T01:19:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.890+0000 7fbeda200640 1 Processor -- start 2024-03-19T01:19:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.894+0000 7fbeda200640 1 -- start start 2024-03-19T01:19:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.894+0000 7fbeda200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed410e070 0x7fbed41a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.894+0000 7fbeda200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed4110040 0x7fbed41a7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.894+0000 7fbeda200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed411b000 0x7fbed41ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.894+0000 7fbeda200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbed4120960 con 0x7fbed411b000 2024-03-19T01:19:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.894+0000 7fbeda200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbed41207e0 con 0x7fbed410e070 2024-03-19T01:19:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.894+0000 7fbeda200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbed4120ae0 con 0x7fbed4110040 2024-03-19T01:19:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.895+0000 7fbed2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed4110040 0x7fbed41a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.895+0000 7fbed2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed4110040 0x7fbed41a7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45094/0 (socket says 172.21.15.69:45094) 2024-03-19T01:19:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.895+0000 7fbed2a00640 1 -- 172.21.15.69:0/3923618271 learned_addr learned my addr 172.21.15.69:0/3923618271 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:03.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.895+0000 7fbed3e00640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed411b000 0x7fbed41ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.895+0000 7fbed2a00640 1 -- 172.21.15.69:0/3923618271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed410e070 msgr2=0x7fbed41a6ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-19T01:19:03.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.895+0000 7fbed2a00640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed410e070 0x7fbed41a6ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.895+0000 7fbed2a00640 1 -- 172.21.15.69:0/3923618271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed411b000 msgr2=0x7fbed41ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.896+0000 7fbed2a00640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed411b000 0x7fbed41ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.896+0000 7fbed2a00640 1 -- 172.21.15.69:0/3923618271 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed41abe80 con 0x7fbed4110040 2024-03-19T01:19:03.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.897+0000 7fbed3e00640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed411b000 0x7fbed41ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:03.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.897+0000 7fbed2a00640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed4110040 0x7fbed41a7420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbec000d950 tx=0x7fbec000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:03.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.898+0000 7fbecbe00640 1 -- 172.21.15.69:0/3923618271 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0017070 con 0x7fbed4110040 2024-03-19T01:19:03.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.898+0000 7fbecbe00640 1 -- 172.21.15.69:0/3923618271 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec0009c30 con 0x7fbed4110040 2024-03-19T01:19:03.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.898+0000 7fbecbe00640 1 -- 172.21.15.69:0/3923618271 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec000f690 con 0x7fbed4110040 2024-03-19T01:19:03.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.898+0000 7fbeda200640 1 -- 172.21.15.69:0/3923618271 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed41ac170 con 0x7fbed4110040 2024-03-19T01:19:03.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.898+0000 7fbeda200640 1 -- 172.21.15.69:0/3923618271 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed41b3a00 con 0x7fbed4110040 2024-03-19T01:19:03.902 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/thread_self.o 2024-03-19T01:19:03.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.911+0000 7fbedcb22740 1 Processor -- start 2024-03-19T01:19:03.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.912+0000 7fbedcb22740 1 -- start start 2024-03-19T01:19:03.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.913+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd93fd40 0x55effd940120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.913+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 0x55effd948d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.913+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd949240 0x55effd94b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.913+0000 7fbedcb22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55effd8a42c0 con 0x55effd949240 2024-03-19T01:19:03.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.913+0000 7fbedcb22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55effd84cb60 con 0x55effd93fd40 2024-03-19T01:19:03.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.913+0000 7fbedcb22740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55effd8a0360 con 0x55effd940660 2024-03-19T01:19:03.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.914+0000 7fbed3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd93fd40 0x55effd940120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.914+0000 7fbed3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 0x55effd948d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.914+0000 7fbed3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 0x55effd948d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45098/0 (socket says 172.21.15.69:45098) 2024-03-19T01:19:03.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.914+0000 7fbed3400640 1 -- 172.21.15.69:0/977741134 learned_addr learned my addr 172.21.15.69:0/977741134 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:03.916 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq/matrix.o 2024-03-19T01:19:03.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.915+0000 7fbecbe00640 1 -- 172.21.15.69:0/3923618271 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fbec000f830 con 0x7fbed4110040 2024-03-19T01:19:03.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.916+0000 7fbed3400640 1 -- 172.21.15.69:0/977741134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd93fd40 msgr2=0x55effd940120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:03.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.916+0000 7fbed3400640 1 --2- 172.21.15.69:0/977741134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd93fd40 0x55effd940120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.916+0000 7fbed3400640 1 -- 172.21.15.69:0/977741134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd949240 msgr2=0x55effd94b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:03.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.916+0000 7fbed3400640 1 --2- 172.21.15.69:0/977741134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd949240 0x55effd94b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.916+0000 7fbed3400640 1 -- 172.21.15.69:0/977741134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55effd6827d0 con 0x55effd940660 2024-03-19T01:19:03.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.917+0000 7fbed3400640 1 --2- 172.21.15.69:0/977741134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 0x55effd948d00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbec400a8f0 tx=0x7fbec400adb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50a028c7432b3978 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:03.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.917+0000 7fbed3e00640 1 --2- 172.21.15.69:0/977741134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd93fd40 0x55effd940120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:03.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.917+0000 7fbebfe00640 1 -- 172.21.15.69:0/977741134 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec400ccc0 con 0x55effd940660 2024-03-19T01:19:03.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.918+0000 7fbebfe00640 1 -- 172.21.15.69:0/977741134 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec40035f0 con 0x55effd940660 2024-03-19T01:19:03.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.918+0000 7fbebfe00640 1 -- 172.21.15.69:0/977741134 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec400f6b0 con 0x55effd940660 2024-03-19T01:19:03.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.919+0000 7fbecbe00640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbeb80a2ab0 0x7fbeb80a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.919+0000 7fbecbe00640 1 -- 172.21.15.69:0/3923618271 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7fbec00cea60 con 0x7fbed4110040 2024-03-19T01:19:03.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.919+0000 7fbedcb22740 1 -- 172.21.15.69:0/977741134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 msgr2=0x55effd948d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.919+0000 7fbedcb22740 1 --2- 172.21.15.69:0/977741134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 0x55effd948d00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbec400a8f0 tx=0x7fbec400adb0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.921+0000 7fbed3e00640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbeb80a2ab0 0x7fbeb80a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.925+0000 7fbed3e00640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbeb80a2ab0 0x7fbeb80a4f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009a60 tx=0x7fbecc032470 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:03.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.925+0000 7fbedcb22740 1 -- 172.21.15.69:0/977741134 shutdown_connections 2024-03-19T01:19:03.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.925+0000 7fbedcb22740 1 --2- 172.21.15.69:0/977741134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd949240 0x55effd94b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.925+0000 7fbedcb22740 1 --2- 172.21.15.69:0/977741134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 0x55effd948d00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.925+0000 7fbedcb22740 1 --2- 172.21.15.69:0/977741134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd93fd40 0x55effd940120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.925+0000 7fbedcb22740 1 -- 172.21.15.69:0/977741134 >> 172.21.15.69:0/977741134 conn(0x55effd93a470 msgr2=0x55effd93a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:03.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.925+0000 7fbedcb22740 1 -- 172.21.15.69:0/977741134 shutdown_connections 2024-03-19T01:19:03.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.926+0000 7fbedcb22740 1 -- 172.21.15.69:0/977741134 wait complete. 2024-03-19T01:19:03.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.927+0000 7fbedcb22740 1 -- start start 2024-03-19T01:19:03.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.927+0000 7fbedcb22740 1 Processor -- start 2024-03-19T01:19:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.930+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd93fd40 0x55effd946240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.931+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd940660 0x55effd946780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.931+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd949240 0x55effd946cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.931+0000 7fbed2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd940660 0x55effd946780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.931+0000 7fbed2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd940660 0x55effd946780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58832/0 (socket says 172.21.15.69:58832) 2024-03-19T01:19:03.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.931+0000 7fbed3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd93fd40 0x55effd946240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.931+0000 7fbed3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd93fd40 0x55effd946240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45110/0 (socket says 172.21.15.69:45110) 2024-03-19T01:19:03.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.932+0000 7fbed3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd949240 0x55effd946cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.932+0000 7fbed3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd949240 0x55effd946cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44264/0 (socket says 172.21.15.69:44264) 2024-03-19T01:19:03.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.932+0000 7fbed2a00640 1 -- 172.21.15.69:0/526836430 learned_addr learned my addr 172.21.15.69:0/526836430 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:03.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.932+0000 7fbedcb22740 1 -- 172.21.15.69:0/526836430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55effd94c030 con 0x55effd949240 2024-03-19T01:19:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.932+0000 7fbedcb22740 1 -- 172.21.15.69:0/526836430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55effd85ffc0 con 0x55effd940660 2024-03-19T01:19:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.932+0000 7fbedcb22740 1 -- 172.21.15.69:0/526836430 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55effd94c1b0 con 0x55effd93fd40 2024-03-19T01:19:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.932+0000 7fbedcb22740 10 client.0 _ll_register_callbacks cb 0x55effd85f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.933+0000 7fbed2a00640 1 -- 172.21.15.69:0/526836430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd93fd40 msgr2=0x55effd946240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.933+0000 7fbed2a00640 1 --2- 172.21.15.69:0/526836430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd93fd40 0x55effd946240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.933+0000 7fbed2a00640 1 -- 172.21.15.69:0/526836430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd949240 msgr2=0x55effd946cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.933+0000 7fbed2a00640 1 --2- 172.21.15.69:0/526836430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd949240 0x55effd946cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.933+0000 7fbed2a00640 1 -- 172.21.15.69:0/526836430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55effd5b83f0 con 0x55effd940660 2024-03-19T01:19:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.934+0000 7fbed2a00640 1 --2- 172.21.15.69:0/526836430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd940660 0x55effd946780 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbec009ace0 tx=0x7fbec0009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4eaa9e920cac33d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.935+0000 7fbebcc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:19:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.935+0000 7fbebcc00640 1 -- 172.21.15.69:0/526836430 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0020040 con 0x55effd940660 2024-03-19T01:19:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.935+0000 7fbebcc00640 1 -- 172.21.15.69:0/526836430 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7fbec000fb20 con 0x55effd940660 2024-03-19T01:19:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.935+0000 7fbedcb22740 1 -- 172.21.15.69:0/526836430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbec40035f0 con 0x55effd940660 2024-03-19T01:19:03.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.937+0000 7fbebcc00640 1 -- 172.21.15.69:0/526836430 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fbec00b8050 con 0x55effd940660 2024-03-19T01:19:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.937+0000 7fbedcb22740 10 client.27476 fetch_fsmap learned FSMap version 51 2024-03-19T01:19:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.937+0000 7fbedcb22740 1 -- 172.21.15.69:0/526836430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55effd5780f0 con 0x55effd940660 2024-03-19T01:19:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.937+0000 7fbebcc00640 1 -- 172.21.15.69:0/526836430 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec00cc1e0 con 0x55effd940660 2024-03-19T01:19:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.937+0000 7fbebcc00640 1 -- 172.21.15.69:0/526836430 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7fbec0023060 con 0x55effd940660 2024-03-19T01:19:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.937+0000 7fbedcb22740 10 client.27476 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:19:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.937+0000 7fbedcb22740 10 client.27476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.937+0000 7fbedcb22740 20 client.27476 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.937+0000 7fbedcb22740 1 --2- 172.21.15.69:0/526836430 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55effd959620 0x55effd95ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.938+0000 7fbedcb22740 4 client.27476 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.938+0000 7fbedcb22740 1 -- 172.21.15.69:0/526836430 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbec40035f0 con 0x55effd959620 2024-03-19T01:19:03.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.938+0000 7fbed3400640 1 --2- 172.21.15.69:0/526836430 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55effd959620 0x55effd95ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:03.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.941+0000 7fbed3400640 1 --2- 172.21.15.69:0/526836430 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55effd959620 0x55effd95ba10 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e82045381cc36a6d server_cookie=48426d3ce11b1b5a in_seq=0 out_seq=0 2024-03-19T01:19:03.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.941+0000 7fbebcc00640 10 client.27476 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:03.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.942+0000 7fbebcc00640 1 -- 172.21.15.69:0/526836430 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fbec40035f0 con 0x55effd959620 2024-03-19T01:19:03.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.942+0000 7fbebcc00640 10 client.27476 handle_command_reply: tid=0 2024-03-19T01:19:03.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.943+0000 7fbedcb22740 1 client.27476 shutdown 2024-03-19T01:19:03.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.944+0000 7fbedcb22740 1 -- 172.21.15.69:0/526836430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd940660 msgr2=0x55effd946780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:03.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.944+0000 7fbedcb22740 1 --2- 172.21.15.69:0/526836430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd940660 0x55effd946780 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbec009ace0 tx=0x7fbec0009de0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.952+0000 7fbedcb22740 1 -- 172.21.15.69:0/526836430 shutdown_connections 2024-03-19T01:19:03.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.952+0000 7fbedcb22740 1 --2- 172.21.15.69:0/526836430 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55effd959620 0x55effd95ba10 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.952+0000 7fbedcb22740 1 --2- 172.21.15.69:0/526836430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd949240 0x55effd946cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.952+0000 7fbedcb22740 1 --2- 172.21.15.69:0/526836430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd940660 0x55effd946780 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.952+0000 7fbedcb22740 1 --2- 172.21.15.69:0/526836430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd93fd40 0x55effd946240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:03.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.952+0000 7fbedcb22740 1 -- 172.21.15.69:0/526836430 >> 172.21.15.69:0/526836430 conn(0x55effd93a470 msgr2=0x55effd93c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:03.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.953+0000 7fbebcc00640 0 client.27476 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:03.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.955+0000 7fbedcb22740 1 -- 172.21.15.69:0/526836430 shutdown_connections 2024-03-19T01:19:03.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.955+0000 7fbedcb22740 1 -- 172.21.15.69:0/526836430 wait complete. 2024-03-19T01:19:03.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:03.955+0000 7fbedcb22740 20 client.27476 trim_cache size 0 max 16384 2024-03-19T01:19:04.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.038+0000 7fbedcb22740 1 Processor -- start 2024-03-19T01:19:04.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.041+0000 7fbedcb22740 1 -- start start 2024-03-19T01:19:04.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.041+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd93fd40 0x55effd954e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:04.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.041+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd940660 0x55effd9453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:04.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.041+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd949240 0x55effd93bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:04.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.042+0000 7fbedcb22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4010180 con 0x55effd940660 2024-03-19T01:19:04.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.042+0000 7fbedcb22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4011e40 con 0x55effd93fd40 2024-03-19T01:19:04.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.042+0000 7fbedcb22740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4010850 con 0x55effd949240 2024-03-19T01:19:04.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.042+0000 7fbed3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd949240 0x55effd93bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:04.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.042+0000 7fbed3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd949240 0x55effd93bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45112/0 (socket says 172.21.15.69:45112) 2024-03-19T01:19:04.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.042+0000 7fbed3e00640 1 -- 172.21.15.69:0/3985206869 learned_addr learned my addr 172.21.15.69:0/3985206869 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:04.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.043+0000 7fbed3e00640 1 -- 172.21.15.69:0/3985206869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd93fd40 msgr2=0x55effd954e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:19:04.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.043+0000 7fbed3e00640 1 --2- 172.21.15.69:0/3985206869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd93fd40 0x55effd954e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.043+0000 7fbed3e00640 1 -- 172.21.15.69:0/3985206869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd940660 msgr2=0x55effd9453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:04.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.043+0000 7fbed3e00640 1 --2- 172.21.15.69:0/3985206869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd940660 0x55effd9453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.043+0000 7fbed3e00640 1 -- 172.21.15.69:0/3985206869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55effd93a2a0 con 0x55effd949240 2024-03-19T01:19:04.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.045+0000 7fbed3e00640 1 --2- 172.21.15.69:0/3985206869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd949240 0x55effd93bfd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbecc036000 tx=0x7fbecc03d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e755ba333d13ab2b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:04.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.046+0000 7fbebf400640 1 -- 172.21.15.69:0/3985206869 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc046070 con 0x55effd949240 2024-03-19T01:19:04.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.046+0000 7fbebf400640 1 -- 172.21.15.69:0/3985206869 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbecc051070 con 0x55effd949240 2024-03-19T01:19:04.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.046+0000 7fbebf400640 1 -- 172.21.15.69:0/3985206869 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc039040 con 0x55effd949240 2024-03-19T01:19:04.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.048+0000 7fbedcb22740 1 -- 172.21.15.69:0/3985206869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd949240 msgr2=0x55effd93bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:04.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.048+0000 7fbedcb22740 1 --2- 172.21.15.69:0/3985206869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd949240 0x55effd93bfd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbecc036000 tx=0x7fbecc03d000 comp rx=0 tx=0).stop 2024-03-19T01:19:04.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.051+0000 7fbedcb22740 1 -- 172.21.15.69:0/3985206869 shutdown_connections 2024-03-19T01:19:04.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.051+0000 7fbedcb22740 1 --2- 172.21.15.69:0/3985206869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd949240 0x55effd93bfd0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.051+0000 7fbedcb22740 1 --2- 172.21.15.69:0/3985206869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd940660 0x55effd9453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.051+0000 7fbedcb22740 1 --2- 172.21.15.69:0/3985206869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd93fd40 0x55effd954e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.051+0000 7fbedcb22740 1 -- 172.21.15.69:0/3985206869 >> 172.21.15.69:0/3985206869 conn(0x55effd93a470 msgr2=0x55effd93b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:04.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.055+0000 7fbedcb22740 1 -- 172.21.15.69:0/3985206869 shutdown_connections 2024-03-19T01:19:04.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.055+0000 7fbedcb22740 1 -- 172.21.15.69:0/3985206869 wait complete. 2024-03-19T01:19:04.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.056+0000 7fbedcb22740 1 -- start start 2024-03-19T01:19:04.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.056+0000 7fbedcb22740 1 Processor -- start 2024-03-19T01:19:04.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.059+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd93fd40 0x55effd951570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:04.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.059+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 0x55effd94ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:04.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.059+0000 7fbedcb22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd949240 0x55effd94cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:04.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.059+0000 7fbed2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 0x55effd94ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:04.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.059+0000 7fbed3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd93fd40 0x55effd951570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:04.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbedcb22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb40116b0 con 0x55effd93fd40 2024-03-19T01:19:04.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbedcb22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4010fe0 con 0x55effd949240 2024-03-19T01:19:04.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbedcb22740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb400e130 con 0x55effd940660 2024-03-19T01:19:04.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbedcb22740 10 client.0 _ll_register_callbacks cb 0x7fbec009ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:04.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbed3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd949240 0x55effd94cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:04.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbed3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd949240 0x55effd94cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58848/0 (socket says 172.21.15.69:58848) 2024-03-19T01:19:04.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbed3e00640 1 -- 172.21.15.69:0/824118124 learned_addr learned my addr 172.21.15.69:0/824118124 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:04.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbed3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd93fd40 0x55effd951570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44282/0 (socket says 172.21.15.69:44282) 2024-03-19T01:19:04.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.059+0000 7fbed2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 0x55effd94ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45114/0 (socket says 172.21.15.69:45114) 2024-03-19T01:19:04.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbed3e00640 1 -- 172.21.15.69:0/824118124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 msgr2=0x55effd94ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:04.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbed3e00640 1 --2- 172.21.15.69:0/824118124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 0x55effd94ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbed3e00640 1 -- 172.21.15.69:0/824118124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd93fd40 msgr2=0x55effd951570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:04.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbed3e00640 1 --2- 172.21.15.69:0/824118124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd93fd40 0x55effd951570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.060+0000 7fbed3e00640 1 -- 172.21.15.69:0/824118124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55effd94d5d0 con 0x55effd949240 2024-03-19T01:19:04.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.061+0000 7fbed3e00640 1 --2- 172.21.15.69:0/824118124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd949240 0x55effd94cf90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbecc032960 tx=0x7fbecc02c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b987bd47971b18aa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:04.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.061+0000 7fbebcc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:19:04.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.062+0000 7fbebcc00640 1 -- 172.21.15.69:0/824118124 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc059070 con 0x55effd949240 2024-03-19T01:19:04.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.062+0000 7fbebcc00640 1 -- 172.21.15.69:0/824118124 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9568+0+0 (secure 0 0 0) 0x7fbecc004200 con 0x55effd949240 2024-03-19T01:19:04.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.062+0000 7fbedcb22740 1 -- 172.21.15.69:0/824118124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbecc051070 con 0x55effd949240 2024-03-19T01:19:04.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.062+0000 7fbebcc00640 1 -- 172.21.15.69:0/824118124 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fbecc063050 con 0x55effd949240 2024-03-19T01:19:04.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.063+0000 7fbedcb22740 10 client.27479 fetch_fsmap learned FSMap version 51 2024-03-19T01:19:04.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.063+0000 7fbedcb22740 1 -- 172.21.15.69:0/824118124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55effd9bf750 con 0x55effd949240 2024-03-19T01:19:04.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.063+0000 7fbebcc00640 1 -- 172.21.15.69:0/824118124 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbecc051070 con 0x55effd949240 2024-03-19T01:19:04.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.063+0000 7fbebcc00640 1 -- 172.21.15.69:0/824118124 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 51) ==== 8398+0+0 (secure 0 0 0) 0x7fbecc05e440 con 0x55effd949240 2024-03-19T01:19:04.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.063+0000 7fbedcb22740 10 client.27479 fetch_fsmap finished waiting for FSMap version 51 2024-03-19T01:19:04.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.063+0000 7fbedcb22740 10 client.27479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:04.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.063+0000 7fbedcb22740 20 client.27479 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:04.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.063+0000 7fbedcb22740 1 --2- 172.21.15.69:0/824118124 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55effd9bff70 0x55effd9c2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:04.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.063+0000 7fbedcb22740 4 client.27479 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:19:04.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.063+0000 7fbedcb22740 1 -- 172.21.15.69:0/824118124 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55effd845610 con 0x55effd9bff70 2024-03-19T01:19:04.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.065+0000 7fbed3400640 1 --2- 172.21.15.69:0/824118124 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55effd9bff70 0x55effd9c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:04.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.070+0000 7fbed3400640 1 --2- 172.21.15.69:0/824118124 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55effd9bff70 0x55effd9c2390 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f03bcea18c0a27bd server_cookie=905770ddf80a99c6 in_seq=0 out_seq=0 2024-03-19T01:19:04.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.070+0000 7fbebcc00640 10 client.27479 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:04.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.071+0000 7fbebcc00640 1 -- 172.21.15.69:0/824118124 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55effd845610 con 0x55effd9bff70 2024-03-19T01:19:04.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.071+0000 7fbebcc00640 10 client.27479 handle_command_reply: tid=0 2024-03-19T01:19:04.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.071+0000 7fbedcb22740 1 client.27479 shutdown 2024-03-19T01:19:04.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.072+0000 7fbedcb22740 1 -- 172.21.15.69:0/824118124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd949240 msgr2=0x55effd94cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:04.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.072+0000 7fbedcb22740 1 --2- 172.21.15.69:0/824118124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd949240 0x55effd94cf90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbecc032960 tx=0x7fbecc02c100 comp rx=0 tx=0).stop 2024-03-19T01:19:04.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.074+0000 7fbedcb22740 1 -- 172.21.15.69:0/824118124 shutdown_connections 2024-03-19T01:19:04.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.074+0000 7fbedcb22740 1 --2- 172.21.15.69:0/824118124 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55effd9bff70 0x55effd9c2390 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.074+0000 7fbedcb22740 1 --2- 172.21.15.69:0/824118124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55effd949240 0x55effd94cf90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.074+0000 7fbedcb22740 1 --2- 172.21.15.69:0/824118124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55effd940660 0x55effd94ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.074+0000 7fbedcb22740 1 --2- 172.21.15.69:0/824118124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55effd93fd40 0x55effd951570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.074+0000 7fbebcc00640 0 client.27479 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:04.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.074+0000 7fbedcb22740 1 -- 172.21.15.69:0/824118124 >> 172.21.15.69:0/824118124 conn(0x55effd93a470 msgr2=0x55effd9396b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:04.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.074+0000 7fbedcb22740 1 -- 172.21.15.69:0/824118124 shutdown_connections 2024-03-19T01:19:04.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.075+0000 7fbedcb22740 1 -- 172.21.15.69:0/824118124 wait complete. 2024-03-19T01:19:04.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.075+0000 7fbedcb22740 20 client.27479 trim_cache size 0 max 16384 2024-03-19T01:19:04.080 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/ioctl.o 2024-03-19T01:19:04.080 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:19:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (10951 inodes in the stack)", 2024-03-19T01:19:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:19:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "253a821b-e147-4849-9883-b34c16f98f02": { 2024-03-19T01:19:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:19:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "253a821b-e147-4849-9883-b34c16f98f02", 2024-03-19T01:19:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:19:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:19:04.081 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:19:04.081 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:19:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.081+0000 7fbeda200640 1 -- 172.21.15.69:0/3923618271 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbeb80a2ab0 msgr2=0x7fbeb80a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.081+0000 7fbeda200640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbeb80a2ab0 0x7fbeb80a4f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009a60 tx=0x7fbecc032470 comp rx=0 tx=0).stop 2024-03-19T01:19:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.081+0000 7fbeda200640 1 -- 172.21.15.69:0/3923618271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed4110040 msgr2=0x7fbed41a7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.081+0000 7fbeda200640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed4110040 0x7fbed41a7420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbec000d950 tx=0x7fbec000de10 comp rx=0 tx=0).stop 2024-03-19T01:19:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.082+0000 7fbeda200640 1 -- 172.21.15.69:0/3923618271 shutdown_connections 2024-03-19T01:19:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.082+0000 7fbeda200640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed411b000 0x7fbed41ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.082+0000 7fbeda200640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed4110040 0x7fbed41a7420 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.082+0000 7fbeda200640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbeb80a2ab0 0x7fbeb80a4f70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.082+0000 7fbeda200640 1 --2- 172.21.15.69:0/3923618271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed410e070 0x7fbed41a6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.082+0000 7fbeda200640 1 -- 172.21.15.69:0/3923618271 >> 172.21.15.69:0/3923618271 conn(0x7fbed406c900 msgr2=0x7fbed40722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:04.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.087+0000 7fbeda200640 1 -- 172.21.15.69:0/3923618271 shutdown_connections 2024-03-19T01:19:04.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:04.087+0000 7fbeda200640 1 -- 172.21.15.69:0/3923618271 wait complete. 2024-03-19T01:19:04.158 INFO:tasks.cephfs.filesystem:scrub status for tag:253a821b-e147-4849-9883-b34c16f98f02 - {'path': '/', 'tag': '253a821b-e147-4849-9883-b34c16f98f02', 'options': 'recursive,force'} 2024-03-19T01:19:04.525 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/authenc.o 2024-03-19T01:19:04.631 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/sqpoll.o 2024-03-19T01:19:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:19:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:04.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:19:04.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:04.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:19:04.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:04.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:04.835 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/fpu/regset.o 2024-03-19T01:19:05.052 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mlock.o 2024-03-19T01:19:05.165 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/proc_sysctl.o 2024-03-19T01:19:05.753 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/genhd.o 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: pgmap v259: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 494 KiB/s wr, 43 op/s 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: osdmap e99: 12 total, 12 up, 12 in 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: pgmap v259: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 494 KiB/s wr, 43 op/s 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: osdmap e99: 12 total, 12 up, 12 in 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:05.817 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:05.966 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/ss/context.o 2024-03-19T01:19:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: pgmap v259: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 494 KiB/s wr, 43 op/s 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: osdmap e99: 12 total, 12 up, 12 in 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:06.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.014 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/irq/built-in.a 2024-03-19T01:19:06.082 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/fpu/signal.o 2024-03-19T01:19:06.167 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/fdinfo.o 2024-03-19T01:19:06.192 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/rcu/update.o 2024-03-19T01:19:06.346 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/authencesn.o 2024-03-19T01:19:06.548 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.550 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.550 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.550 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.550 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.550 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.550 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.550 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.552 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.552 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.552 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.552 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:06.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:06.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:06.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:07.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.553 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mmap.o 2024-03-19T01:19:07.568 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/selinux/netlabel.o 2024-03-19T01:19:07.600 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/fpu/xstate.o 2024-03-19T01:19:07.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: pgmap v261: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 311 KiB/s wr, 63 op/s 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:07.856 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/proc_net.o 2024-03-19T01:19:08.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: pgmap v261: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 311 KiB/s wr, 63 op/s 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:08.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: pgmap v261: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 311 KiB/s wr, 63 op/s 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:08.060 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/tctx.o 2024-03-19T01:19:08.174 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/ioprio.o 2024-03-19T01:19:08.336 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/rng.o 2024-03-19T01:19:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:09.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:09.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:09.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:09.340 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/rcu/sync.o 2024-03-19T01:19:09.606 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/kcore.o 2024-03-19T01:19:09.678 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/poll.o 2024-03-19T01:19:09.804 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/badblocks.o 2024-03-19T01:19:09.890 INFO:tasks.workunit.client.0.smithi069.stdout: AR security/selinux/built-in.a 2024-03-19T01:19:09.902 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/drbg.o 2024-03-19T01:19:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:10.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:10.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:10.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: pgmap v262: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 298 KiB/s wr, 174 op/s 2024-03-19T01:19:10.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:10.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: osdmap e100: 12 total, 12 up, 12 in 2024-03-19T01:19:10.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:10.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:10.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:10.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:10.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:10.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: pgmap v262: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 298 KiB/s wr, 174 op/s 2024-03-19T01:19:10.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:10.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: osdmap e100: 12 total, 12 up, 12 in 2024-03-19T01:19:10.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:10.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:10.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:10.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:10.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:10.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: pgmap v262: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 298 KiB/s wr, 174 op/s 2024-03-19T01:19:10.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:10.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: osdmap e100: 12 total, 12 up, 12 in 2024-03-19T01:19:10.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:10.067 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/rcu/srcutree.o 2024-03-19T01:19:10.089 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/commoncap.o 2024-03-19T01:19:10.141 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/kernel/fpu/built-in.a 2024-03-19T01:19:10.282 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/mce/core.o 2024-03-19T01:19:11.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:11.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:11.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:11.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:11.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:11.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:11.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:11.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:11.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:11.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:11.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:11.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:11.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:11.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:11.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:11.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:11.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:11.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:11.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:11.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:11.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:11.063 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-rq-qos.o 2024-03-19T01:19:11.362 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/vmcore.o 2024-03-19T01:19:11.375 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/jitterentropy.o 2024-03-19T01:19:11.448 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mmu_gather.o 2024-03-19T01:19:11.535 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/cancel.o 2024-03-19T01:19:11.583 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/jitterentropy-kcapi.o 2024-03-19T01:19:12.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: pgmap v264: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 461 KiB/s wr, 261 op/s 2024-03-19T01:19:12.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:12.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: pgmap v264: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 461 KiB/s wr, 261 op/s 2024-03-19T01:19:12.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:12.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: pgmap v264: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 461 KiB/s wr, 261 op/s 2024-03-19T01:19:12.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:12.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:12.087 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/rcu/tree.o 2024-03-19T01:19:12.137 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/min_addr.o 2024-03-19T01:19:12.609 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/ghash-generic.o 2024-03-19T01:19:12.678 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/disk-events.o 2024-03-19T01:19:13.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:13.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.041 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.041 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.041 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.041 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.041 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.041 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:13.041 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:13.066 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:13.091 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/kmsg.o 2024-03-19T01:19:13.143 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mprotect.o 2024-03-19T01:19:13.322 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/mce/severity.o 2024-03-19T01:19:13.414 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/security.o 2024-03-19T01:19:13.433 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/kbuf.o 2024-03-19T01:19:13.520 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/hash_info.o 2024-03-19T01:19:13.906 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc/page.o 2024-03-19T01:19:13.913 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/rsapubkey.asn1.o 2024-03-19T01:19:14.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: pgmap v265: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 834 op/s 2024-03-19T01:19:14.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:14.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:14.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:14.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:14.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:13 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: pgmap v265: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 834 op/s 2024-03-19T01:19:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:14.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:14.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:13 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.045 INFO:tasks.workunit.client.0.smithi069.stdout: CC crypto/rsaprivkey.asn1.o 2024-03-19T01:19:14.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: pgmap v265: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 834 op/s 2024-03-19T01:19:14.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:14.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:14.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:14.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:14.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:13 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:14.167 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-ia-ranges.o 2024-03-19T01:19:14.208 INFO:tasks.workunit.client.0.smithi069.stdout: AR crypto/built-in.a 2024-03-19T01:19:14.342 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/lsm_audit.o 2024-03-19T01:19:14.871 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/mce/genpool.o 2024-03-19T01:19:14.991 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mremap.o 2024-03-19T01:19:15.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:15.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:14 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:15.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:14 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:15.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:14 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:15.315 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/rsrc.o 2024-03-19T01:19:15.408 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/early-lookup.o 2024-03-19T01:19:15.449 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/proc/built-in.a 2024-03-19T01:19:15.600 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/kernfs/mount.o 2024-03-19T01:19:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: pgmap v266: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 866 op/s 2024-03-19T01:19:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:16.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:16.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:15 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: pgmap v266: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 866 op/s 2024-03-19T01:19:16.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:16.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:16.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:16.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:16.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:16.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:15 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: pgmap v266: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 866 op/s 2024-03-19T01:19:16.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:16.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:16.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:16.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:16.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:16.064 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:15 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:16.127 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/mce/intel.o 2024-03-19T01:19:16.362 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/mce/amd.o 2024-03-19T01:19:16.680 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/kernfs/inode.o 2024-03-19T01:19:16.737 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/bsg.o 2024-03-19T01:19:16.985 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/msync.o 2024-03-19T01:19:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:17.008 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:16 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:17.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:16 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.063 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:16 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.073 INFO:tasks.workunit.client.0.smithi069.stdout: CC security/device_cgroup.o 2024-03-19T01:19:17.444 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/sysfs/file.o 2024-03-19T01:19:17.737 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/rcu/rcu_segcblist.o 2024-03-19T01:19:17.962 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:17 smithi163 ceph-mon[67428]: pgmap v267: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.25k op/s 2024-03-19T01:19:17.962 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.962 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.962 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.962 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:17.962 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:17 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:17.963 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:17 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:18.004 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/rw.o 2024-03-19T01:19:18.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:17 smithi086 ceph-mon[66829]: pgmap v267: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.25k op/s 2024-03-19T01:19:18.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:18.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:18.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:18.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:18.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:17 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:18.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:17 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:17 smithi069 ceph-mon[31727]: pgmap v267: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.25k op/s 2024-03-19T01:19:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:17 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:18.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:17 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:18.249 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/kernfs/dir.o 2024-03-19T01:19:18.310 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/page_vma_mapped.o 2024-03-19T01:19:18.343 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-cgroup.o 2024-03-19T01:19:18.612 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/mce/threshold.o 2024-03-19T01:19:18.980 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/rcu/built-in.a 2024-03-19T01:19:19.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:18 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:19.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:18 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:18 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:19.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:18 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.056 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/livepatch/built-in.a 2024-03-19T01:19:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:18 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:19.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:18 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:19.140 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/dma/mapping.o 2024-03-19T01:19:19.467 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/sysfs/dir.o 2024-03-19T01:19:19.539 INFO:tasks.workunit.client.0.smithi069.stdout: AR security/built-in.a 2024-03-19T01:19:19.644 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/kernfs/file.o 2024-03-19T01:19:19.653 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/opdef.o 2024-03-19T01:19:19.861 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/dma/direct.o 2024-03-19T01:19:19.866 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/pagewalk.o 2024-03-19T01:19:20.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:19 smithi086 ceph-mon[66829]: pgmap v268: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 1.85k op/s 2024-03-19T01:19:20.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:19 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:20.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:19 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:20.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:19 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:20.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:19 smithi069 ceph-mon[31727]: pgmap v268: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 1.85k op/s 2024-03-19T01:19:20.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:19 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:20.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:19 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:20.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:19 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:20.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:19 smithi163 ceph-mon[67428]: pgmap v268: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 1.85k op/s 2024-03-19T01:19:20.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:19 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:20.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:19 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:20.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:19 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:20.216 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/kernel/cpu/mce/built-in.a 2024-03-19T01:19:20.316 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/mtrr/mtrr.o 2024-03-19T01:19:20.452 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-ioprio.o 2024-03-19T01:19:20.517 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/sysfs/symlink.o 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:21.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:20 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.014 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/microcode/core.o 2024-03-19T01:19:21.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:21.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:21.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:21.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:21.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:21.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:21.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:20 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:21.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:20 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:21.117 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/kernfs/symlink.o 2024-03-19T01:19:21.200 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/pgtable-generic.o 2024-03-19T01:19:21.281 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/dma/ops_helpers.o 2024-03-19T01:19:21.658 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/mtrr/if.o 2024-03-19T01:19:21.691 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/sysfs/mount.o 2024-03-19T01:19:21.723 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/notif.o 2024-03-19T01:19:21.897 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-iolatency.o 2024-03-19T01:19:22.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:22.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:22.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:22.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:22.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:22.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: pgmap v269: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.67k op/s 2024-03-19T01:19:22.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:22.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:22.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:21 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:22.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:22.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:22.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: pgmap v269: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.67k op/s 2024-03-19T01:19:22.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:22.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:22.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:21 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:22.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:22.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:22.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:22.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:22.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: pgmap v269: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.67k op/s 2024-03-19T01:19:22.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:22.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:22.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:21 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:22.142 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/kernfs/built-in.a 2024-03-19T01:19:22.225 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/cacheinfo.o 2024-03-19T01:19:22.415 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/dma/dummy.o 2024-03-19T01:19:22.457 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/rmap.o 2024-03-19T01:19:22.520 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/microcode/intel.o 2024-03-19T01:19:22.824 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/mtrr/generic.o 2024-03-19T01:19:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:23.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:23.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:23.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:23.174 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/sysfs/group.o 2024-03-19T01:19:23.831 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/microcode/amd.o 2024-03-19T01:19:23.841 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-iocost.o 2024-03-19T01:19:23.845 INFO:tasks.workunit.client.0.smithi069.stdout: CC io_uring/io-wq.o 2024-03-19T01:19:23.849 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/mq-deadline.o 2024-03-19T01:19:23.854 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/dma/swiotlb.o 2024-03-19T01:19:23.871 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/sysfs/built-in.a 2024-03-19T01:19:23.939 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/mtrr/cleanup.o 2024-03-19T01:19:23.957 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/devpts/inode.o 2024-03-19T01:19:24.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:24.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:24.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:24.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:24.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:23 smithi086 ceph-mon[66829]: pgmap v270: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.60k op/s 2024-03-19T01:19:24.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:24.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:24.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:23 smithi069 ceph-mon[31727]: pgmap v270: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.60k op/s 2024-03-19T01:19:24.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:24.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:24.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:23 smithi163 ceph-mon[67428]: pgmap v270: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 1.60k op/s 2024-03-19T01:19:24.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:24.345 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/vmalloc.o 2024-03-19T01:19:24.882 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/kernel/cpu/microcode/built-in.a 2024-03-19T01:19:25.021 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/process_vm_access.o 2024-03-19T01:19:25.021 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/kernel/cpu/mtrr/built-in.a 2024-03-19T01:19:25.142 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/scattered.o 2024-03-19T01:19:25.194 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/dma/remap.o 2024-03-19T01:19:25.460 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/devpts/built-in.a 2024-03-19T01:19:25.524 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/netfs/buffered_read.o 2024-03-19T01:19:25.541 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/topology.o 2024-03-19T01:19:25.592 INFO:tasks.workunit.client.0.smithi069.stdout: AR io_uring/built-in.a 2024-03-19T01:19:25.748 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/pat/set_memory.o 2024-03-19T01:19:25.846 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/pat/memtype.o 2024-03-19T01:19:26.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:25 smithi086 ceph-mon[66829]: pgmap v271: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 400 KiB/s wr, 1.24k op/s 2024-03-19T01:19:26.010 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/dma/built-in.a 2024-03-19T01:19:26.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:25 smithi069 ceph-mon[31727]: pgmap v271: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 400 KiB/s wr, 1.24k op/s 2024-03-19T01:19:26.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:25 smithi163 ceph-mon[67428]: pgmap v271: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 400 KiB/s wr, 1.24k op/s 2024-03-19T01:19:26.074 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/entry/common.o 2024-03-19T01:19:26.094 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/balloc.o 2024-03-19T01:19:26.285 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/common.o 2024-03-19T01:19:26.816 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/kyber-iosched.o 2024-03-19T01:19:27.006 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/netfs/io.o 2024-03-19T01:19:27.057 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/bitmap.o 2024-03-19T01:19:27.618 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/entry/syscall_user_dispatch.o 2024-03-19T01:19:27.929 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/pat/memtype_interval.o 2024-03-19T01:19:27.947 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/page_alloc.o 2024-03-19T01:19:28.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:27 smithi086 ceph-mon[66829]: pgmap v272: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 368 KiB/s wr, 1.10k op/s 2024-03-19T01:19:28.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:27 smithi069 ceph-mon[31727]: pgmap v272: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 368 KiB/s wr, 1.10k op/s 2024-03-19T01:19:28.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:27 smithi163 ceph-mon[67428]: pgmap v272: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 368 KiB/s wr, 1.10k op/s 2024-03-19T01:19:28.064 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/crypto/built-in.a 2024-03-19T01:19:28.158 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/jbd2/transaction.o 2024-03-19T01:19:28.282 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/rdrand.o 2024-03-19T01:19:28.346 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/netfs/iterator.o 2024-03-19T01:19:28.412 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/entry/built-in.a 2024-03-19T01:19:28.412 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/block_validity.o 2024-03-19T01:19:28.540 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/module/main.o 2024-03-19T01:19:28.540 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-mq-pci.o 2024-03-19T01:19:28.658 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/match.o 2024-03-19T01:19:28.829 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/mm/pat/built-in.a 2024-03-19T01:19:28.905 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/init.o 2024-03-19T01:19:29.404 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/netfs/main.o 2024-03-19T01:19:29.512 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/bugs.o 2024-03-19T01:19:29.733 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/dir.o 2024-03-19T01:19:29.733 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-mq-virtio.o 2024-03-19T01:19:29.949 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/jbd2/commit.o 2024-03-19T01:19:30.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:29 smithi069 ceph-mon[31727]: pgmap v273: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 826 KiB/s wr, 757 op/s 2024-03-19T01:19:30.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:29 smithi163 ceph-mon[67428]: pgmap v273: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 826 KiB/s wr, 757 op/s 2024-03-19T01:19:30.089 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/init_64.o 2024-03-19T01:19:30.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:29 smithi086 ceph-mon[66829]: pgmap v273: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 826 KiB/s wr, 757 op/s 2024-03-19T01:19:30.558 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/netfs/objects.o 2024-03-19T01:19:30.739 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/module/strict_rwx.o 2024-03-19T01:19:30.822 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-mq-debugfs.o 2024-03-19T01:19:30.905 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/ext4_jbd2.o 2024-03-19T01:19:31.267 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/aperfmperf.o 2024-03-19T01:19:31.333 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/init-mm.o 2024-03-19T01:19:31.441 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/jbd2/recovery.o 2024-03-19T01:19:31.690 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/netfs/built-in.a 2024-03-19T01:19:31.772 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/module/kmod.o 2024-03-19T01:19:31.782 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/cpuid-deps.o 2024-03-19T01:19:31.825 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/fault.o 2024-03-19T01:19:32.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:31 smithi069 ceph-mon[31727]: pgmap v274: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 864 KiB/s wr, 163 op/s 2024-03-19T01:19:32.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:31 smithi163 ceph-mon[67428]: pgmap v274: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 864 KiB/s wr, 163 op/s 2024-03-19T01:19:32.192 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/blk-pm.o 2024-03-19T01:19:32.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:31 smithi086 ceph-mon[66829]: pgmap v274: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 864 KiB/s wr, 163 op/s 2024-03-19T01:19:32.315 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/memblock.o 2024-03-19T01:19:32.568 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/umwait.o 2024-03-19T01:19:32.594 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/math/div64.o 2024-03-19T01:19:32.662 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/extents.o 2024-03-19T01:19:32.863 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/math/gcd.o 2024-03-19T01:19:32.974 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/module/tree_lookup.o 2024-03-19T01:19:32.991 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/jbd2/checkpoint.o 2024-03-19T01:19:33.053 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/math/lcm.o 2024-03-19T01:19:33.236 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/math/int_pow.o 2024-03-19T01:19:33.447 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/math/int_sqrt.o 2024-03-19T01:19:33.532 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/cpu/proc.o 2024-03-19T01:19:33.558 INFO:tasks.workunit.client.0.smithi069.stdout: CC block/holder.o 2024-03-19T01:19:33.660 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/math/reciprocal_div.o 2024-03-19T01:19:33.814 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/ioremap.o 2024-03-19T01:19:33.921 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/math/rational.o 2024-03-19T01:19:34.034 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/module/kallsyms.o 2024-03-19T01:19:34.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:33 smithi069 ceph-mon[31727]: pgmap v275: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 958 KiB/s wr, 139 op/s 2024-03-19T01:19:34.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:33 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:19:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:33 smithi163 ceph-mon[67428]: pgmap v275: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 958 KiB/s wr, 139 op/s 2024-03-19T01:19:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:33 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:19:34.098 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/jbd2/revoke.o 2024-03-19T01:19:34.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:19:34.160 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:19:34.201 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/madvise.o 2024-03-19T01:19:34.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:33 smithi086 ceph-mon[66829]: pgmap v275: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 958 KiB/s wr, 139 op/s 2024-03-19T01:19:34.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:33 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:19:34.313 INFO:tasks.workunit.client.0.smithi069.stdout: MKCAP arch/x86/kernel/cpu/capflags.c 2024-03-19T01:19:34.623 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/math/built-in.a 2024-03-19T01:19:34.752 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crypto/memneq.o 2024-03-19T01:19:34.757 INFO:tasks.workunit.client.0.smithi069.stdout: AR block/built-in.a 2024-03-19T01:19:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.793+0000 7fc60f600640 1 -- 172.21.15.69:0/3747525698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc608114970 msgr2=0x7fc608116d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.793+0000 7fc60f600640 1 --2- 172.21.15.69:0/3747525698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc608114970 0x7fc608116d60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009950 tx=0x7fc5f802bca0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.796+0000 7fc60f600640 1 -- 172.21.15.69:0/3747525698 shutdown_connections 2024-03-19T01:19:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.796+0000 7fc60f600640 1 --2- 172.21.15.69:0/3747525698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc608114970 0x7fc608116d60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.796+0000 7fc60f600640 1 --2- 172.21.15.69:0/3747525698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc608109320 0x7fc608114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.796+0000 7fc60f600640 1 --2- 172.21.15.69:0/3747525698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc608108a00 0x7fc608108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.796+0000 7fc60f600640 1 -- 172.21.15.69:0/3747525698 >> 172.21.15.69:0/3747525698 conn(0x7fc60806c900 msgr2=0x7fc60806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:34.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.801+0000 7fc60f600640 1 -- 172.21.15.69:0/3747525698 shutdown_connections 2024-03-19T01:19:34.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.801+0000 7fc60f600640 1 -- 172.21.15.69:0/3747525698 wait complete. 2024-03-19T01:19:34.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.804+0000 7fc60f600640 1 Processor -- start 2024-03-19T01:19:34.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.805+0000 7fc60f600640 1 -- start start 2024-03-19T01:19:34.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.805+0000 7fc60f600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc608108a00 0x7fc60811d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.805+0000 7fc60f600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc608109320 0x7fc608119dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.806+0000 7fc60cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc608108a00 0x7fc60811d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.806+0000 7fc60cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc608108a00 0x7fc60811d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55958/0 (socket says 172.21.15.69:55958) 2024-03-19T01:19:34.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.809+0000 7fc60cc00640 1 -- 172.21.15.69:0/2062112895 learned_addr learned my addr 172.21.15.69:0/2062112895 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:34.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.809+0000 7fc60f600640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc608114970 0x7fc60811a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.809+0000 7fc60f600640 1 -- 172.21.15.69:0/2062112895 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc60810a7d0 con 0x7fc608109320 2024-03-19T01:19:34.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.809+0000 7fc60f600640 1 -- 172.21.15.69:0/2062112895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc60810a650 con 0x7fc608114970 2024-03-19T01:19:34.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.809+0000 7fc60f600640 1 -- 172.21.15.69:0/2062112895 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc60810a950 con 0x7fc608108a00 2024-03-19T01:19:34.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.809+0000 7fc60d600640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc608114970 0x7fc60811a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.810+0000 7fc607e00640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc608109320 0x7fc608119dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.810+0000 7fc607e00640 1 -- 172.21.15.69:0/2062112895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc608108a00 msgr2=0x7fc60811d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:34.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.810+0000 7fc607e00640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc608108a00 0x7fc60811d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.810+0000 7fc607e00640 1 -- 172.21.15.69:0/2062112895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc608114970 msgr2=0x7fc60811a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:34.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.810+0000 7fc607e00640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc608114970 0x7fc60811a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.810+0000 7fc607e00640 1 -- 172.21.15.69:0/2062112895 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60811a840 con 0x7fc608109320 2024-03-19T01:19:34.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.811+0000 7fc607e00640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc608109320 0x7fc608119dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400cb00 tx=0x7fc5f40071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:34.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.812+0000 7fc60cc00640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc608108a00 0x7fc60811d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:34.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.812+0000 7fc605600640 1 -- 172.21.15.69:0/2062112895 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4007bf0 con 0x7fc608109320 2024-03-19T01:19:34.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.812+0000 7fc60f600640 1 -- 172.21.15.69:0/2062112895 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60811ab30 con 0x7fc608109320 2024-03-19T01:19:34.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.812+0000 7fc60f600640 1 -- 172.21.15.69:0/2062112895 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6081b3a60 con 0x7fc608109320 2024-03-19T01:19:34.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.819+0000 7fc605600640 1 -- 172.21.15.69:0/2062112895 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f4007d90 con 0x7fc608109320 2024-03-19T01:19:34.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.819+0000 7fc605600640 1 -- 172.21.15.69:0/2062112895 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40188c0 con 0x7fc608109320 2024-03-19T01:19:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.826+0000 7fc6123ef740 1 Processor -- start 2024-03-19T01:19:34.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.826+0000 7fc6123ef740 1 -- start start 2024-03-19T01:19:34.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.827+0000 7fc6123ef740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f5486120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.827+0000 7fc6123ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5486660 0x5564f548ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.827+0000 7fc6123ef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f5491630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.827+0000 7fc6123ef740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5564f53ea2c0 con 0x5564f548f240 2024-03-19T01:19:34.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.827+0000 7fc6123ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564f5392b60 con 0x5564f5486660 2024-03-19T01:19:34.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.827+0000 7fc6123ef740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5564f53e6360 con 0x5564f5485d40 2024-03-19T01:19:34.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.827+0000 7fc60d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f5486120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.827+0000 7fc60d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f5486120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55964/0 (socket says 172.21.15.69:55964) 2024-03-19T01:19:34.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.827+0000 7fc60d600640 1 -- 172.21.15.69:0/3622680896 learned_addr learned my addr 172.21.15.69:0/3622680896 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:34.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.828+0000 7fc60cc00640 1 --2- 172.21.15.69:0/3622680896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5486660 0x5564f548ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.828+0000 7fc60d600640 1 -- 172.21.15.69:0/3622680896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5486660 msgr2=0x5564f548ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:34.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.828+0000 7fc60d600640 1 --2- 172.21.15.69:0/3622680896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5486660 0x5564f548ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.828+0000 7fc60d600640 1 -- 172.21.15.69:0/3622680896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 msgr2=0x5564f5491630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:34.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.828+0000 7fc60d600640 1 --2- 172.21.15.69:0/3622680896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f5491630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.828+0000 7fc60d600640 1 -- 172.21.15.69:0/3622680896 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564f51c87d0 con 0x5564f5485d40 2024-03-19T01:19:34.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.829+0000 7fc60d600640 1 --2- 172.21.15.69:0/3622680896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f5486120 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc60806ac40 tx=0x7fc5f802c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3069a58c8e417512 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:34.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.830+0000 7fc5fdc00640 1 -- 172.21.15.69:0/3622680896 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80042c0 con 0x5564f5485d40 2024-03-19T01:19:34.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.830+0000 7fc5fdc00640 1 -- 172.21.15.69:0/3622680896 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8004460 con 0x5564f5485d40 2024-03-19T01:19:34.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.830+0000 7fc5fdc00640 1 -- 172.21.15.69:0/3622680896 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8005260 con 0x5564f5485d40 2024-03-19T01:19:34.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.832+0000 7fc605600640 1 -- 172.21.15.69:0/2062112895 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fc5f4018a60 con 0x7fc608109320 2024-03-19T01:19:34.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.836+0000 7fc6123ef740 1 -- 172.21.15.69:0/3622680896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 msgr2=0x5564f5486120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:34.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.836+0000 7fc6123ef740 1 --2- 172.21.15.69:0/3622680896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f5486120 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc60806ac40 tx=0x7fc5f802c780 comp rx=0 tx=0).stop 2024-03-19T01:19:34.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.840+0000 7fc605600640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5ec0a2ab0 0x7fc5ec0a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.840+0000 7fc605600640 1 -- 172.21.15.69:0/2062112895 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fc5f40cf140 con 0x7fc608109320 2024-03-19T01:19:34.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.841+0000 7fc60d600640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5ec0a2ab0 0x7fc5ec0a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.842+0000 7fc6123ef740 1 -- 172.21.15.69:0/3622680896 shutdown_connections 2024-03-19T01:19:34.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.842+0000 7fc6123ef740 1 --2- 172.21.15.69:0/3622680896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f5491630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.842+0000 7fc6123ef740 1 --2- 172.21.15.69:0/3622680896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5486660 0x5564f548ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.842+0000 7fc6123ef740 1 --2- 172.21.15.69:0/3622680896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f5486120 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.842+0000 7fc6123ef740 1 -- 172.21.15.69:0/3622680896 >> 172.21.15.69:0/3622680896 conn(0x5564f5480470 msgr2=0x5564f5480850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:34.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.843+0000 7fc60d600640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5ec0a2ab0 0x7fc5ec0a4f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc60806ac40 tx=0x7fc5f8005520 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.843+0000 7fc6123ef740 1 -- 172.21.15.69:0/3622680896 shutdown_connections 2024-03-19T01:19:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.843+0000 7fc6123ef740 1 -- 172.21.15.69:0/3622680896 wait complete. 2024-03-19T01:19:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.844+0000 7fc6123ef740 1 -- start start 2024-03-19T01:19:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.844+0000 7fc6123ef740 1 Processor -- start 2024-03-19T01:19:34.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.849+0000 7fc6123ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5485d40 0x5564f548c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.849+0000 7fc6123ef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f5486660 0x5564f548c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.849+0000 7fc6123ef740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f548f240 0x5564f548cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.849+0000 7fc6123ef740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5564f54a2fe0 con 0x5564f5486660 2024-03-19T01:19:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.849+0000 7fc6123ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564f53a5fc0 con 0x5564f5485d40 2024-03-19T01:19:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.849+0000 7fc6123ef740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5564f54a3160 con 0x5564f548f240 2024-03-19T01:19:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.849+0000 7fc6123ef740 10 client.0 _ll_register_callbacks cb 0x5564f53a5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.850+0000 7fc60d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f548f240 0x5564f548cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.850+0000 7fc60d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f548f240 0x5564f548cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55966/0 (socket says 172.21.15.69:55966) 2024-03-19T01:19:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.850+0000 7fc60d600640 1 -- 172.21.15.69:0/1164336655 learned_addr learned my addr 172.21.15.69:0/1164336655 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.850+0000 7fc60cc00640 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5485d40 0x5564f548c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.850+0000 7fc607e00640 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f5486660 0x5564f548c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.851+0000 7fc607e00640 1 -- 172.21.15.69:0/1164336655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f548f240 msgr2=0x5564f548cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.851+0000 7fc607e00640 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f548f240 0x5564f548cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.851+0000 7fc607e00640 1 -- 172.21.15.69:0/1164336655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5485d40 msgr2=0x5564f548c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.851+0000 7fc607e00640 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5485d40 0x5564f548c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.851+0000 7fc607e00640 1 -- 172.21.15.69:0/1164336655 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564f50fe3f0 con 0x5564f5486660 2024-03-19T01:19:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.852+0000 7fc607e00640 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f5486660 0x5564f548c7f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40cc440 tx=0x7fc5f409bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a59baca888807e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:34.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.852+0000 7fc5f2000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:19:34.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.853+0000 7fc5f2000640 1 -- 172.21.15.69:0/1164336655 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40b3020 con 0x5564f5486660 2024-03-19T01:19:34.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.853+0000 7fc6123ef740 1 -- 172.21.15.69:0/1164336655 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f8004460 con 0x5564f5486660 2024-03-19T01:19:34.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.853+0000 7fc5f2000640 1 -- 172.21.15.69:0/1164336655 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fc5f40cc6c0 con 0x5564f5486660 2024-03-19T01:19:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.856+0000 7fc5f2000640 1 -- 172.21.15.69:0/1164336655 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fc5f407b050 con 0x5564f5486660 2024-03-19T01:19:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.856+0000 7fc6123ef740 10 client.17184 fetch_fsmap learned FSMap version 53 2024-03-19T01:19:34.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.856+0000 7fc6123ef740 1 -- 172.21.15.69:0/1164336655 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5564f50be0f0 con 0x5564f5486660 2024-03-19T01:19:34.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.856+0000 7fc5f2000640 1 -- 172.21.15.69:0/1164336655 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8004460 con 0x5564f5486660 2024-03-19T01:19:34.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.856+0000 7fc5f2000640 1 -- 172.21.15.69:0/1164336655 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 53) ==== 8390+0+0 (secure 0 0 0) 0x7fc5f4020030 con 0x5564f5486660 2024-03-19T01:19:34.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.856+0000 7fc6123ef740 10 client.17184 fetch_fsmap finished waiting for FSMap version 53 2024-03-19T01:19:34.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.856+0000 7fc6123ef740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:34.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.856+0000 7fc6123ef740 20 client.17184 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:34.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.856+0000 7fc6123ef740 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5564f5492f70 0x5564f5495390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.856+0000 7fc6123ef740 4 client.17184 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:34.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.856+0000 7fc6123ef740 1 -- 172.21.15.69:0/1164336655 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564f538b610 con 0x5564f5492f70 2024-03-19T01:19:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.857+0000 7fc60cc00640 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5564f5492f70 0x5564f5495390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.857+0000 7fc60cc00640 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5564f5492f70 0x5564f5495390 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890e1df4d76b1e4d server_cookie=dc28895d6205ffd8 in_seq=0 out_seq=0 2024-03-19T01:19:34.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.857+0000 7fc5f2000640 10 client.17184 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.858+0000 7fc5f2000640 1 -- 172.21.15.69:0/1164336655 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5564f538b610 con 0x5564f5492f70 2024-03-19T01:19:34.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.858+0000 7fc5f2000640 10 client.17184 handle_command_reply: tid=0 2024-03-19T01:19:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.858+0000 7fc6123ef740 1 client.17184 shutdown 2024-03-19T01:19:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.859+0000 7fc6123ef740 1 -- 172.21.15.69:0/1164336655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f5486660 msgr2=0x5564f548c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.859+0000 7fc6123ef740 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f5486660 0x5564f548c7f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40cc440 tx=0x7fc5f409bb90 comp rx=0 tx=0).stop 2024-03-19T01:19:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.859+0000 7fc6123ef740 1 -- 172.21.15.69:0/1164336655 shutdown_connections 2024-03-19T01:19:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.859+0000 7fc6123ef740 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5564f5492f70 0x5564f5495390 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.859+0000 7fc6123ef740 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f548f240 0x5564f548cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.859+0000 7fc6123ef740 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f5486660 0x5564f548c7f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.859+0000 7fc6123ef740 1 --2- 172.21.15.69:0/1164336655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5485d40 0x5564f548c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.859+0000 7fc6123ef740 1 -- 172.21.15.69:0/1164336655 >> 172.21.15.69:0/1164336655 conn(0x5564f5480470 msgr2=0x5564f54821c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.859+0000 7fc5f2000640 0 client.17184 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.859+0000 7fc6123ef740 1 -- 172.21.15.69:0/1164336655 shutdown_connections 2024-03-19T01:19:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.863+0000 7fc6123ef740 1 -- 172.21.15.69:0/1164336655 wait complete. 2024-03-19T01:19:34.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.863+0000 7fc6123ef740 20 client.17184 trim_cache size 0 max 16384 2024-03-19T01:19:34.913 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/page_io.o 2024-03-19T01:19:34.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.921+0000 7fc6123ef740 1 Processor -- start 2024-03-19T01:19:34.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.921+0000 7fc6123ef740 1 -- start start 2024-03-19T01:19:34.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.923+0000 7fc6123ef740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f548e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.923+0000 7fc6123ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5486660 0x5564f547f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.924+0000 7fc6123ef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f548cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.924+0000 7fc607e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5486660 0x5564f547f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.924+0000 7fc607e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5486660 0x5564f547f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37690/0 (socket says 172.21.15.69:37690) 2024-03-19T01:19:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.924+0000 7fc60cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f548e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.924+0000 7fc60cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f548e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55968/0 (socket says 172.21.15.69:55968) 2024-03-19T01:19:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.924+0000 7fc607e00640 1 -- 172.21.15.69:0/4091971136 learned_addr learned my addr 172.21.15.69:0/4091971136 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.925+0000 7fc6123ef740 1 -- 172.21.15.69:0/4091971136 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e800ffe0 con 0x5564f548f240 2024-03-19T01:19:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.925+0000 7fc6123ef740 1 -- 172.21.15.69:0/4091971136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e8011c70 con 0x5564f5486660 2024-03-19T01:19:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.925+0000 7fc6123ef740 1 -- 172.21.15.69:0/4091971136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e80106b0 con 0x5564f5485d40 2024-03-19T01:19:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.925+0000 7fc60cc00640 1 -- 172.21.15.69:0/4091971136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5486660 msgr2=0x5564f547f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.925+0000 7fc60cc00640 1 --2- 172.21.15.69:0/4091971136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5486660 0x5564f547f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.925+0000 7fc60cc00640 1 -- 172.21.15.69:0/4091971136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 msgr2=0x5564f548cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.925+0000 7fc60cc00640 1 --2- 172.21.15.69:0/4091971136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f548cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.925+0000 7fc60cc00640 1 -- 172.21.15.69:0/4091971136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564f5471950 con 0x5564f5485d40 2024-03-19T01:19:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.926+0000 7fc60cc00640 1 --2- 172.21.15.69:0/4091971136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f548e9e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5564f5493350 tx=0x7fc600007940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a5f0737a02acf6d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.926+0000 7fc5fd200640 1 -- 172.21.15.69:0/4091971136 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60001a620 con 0x5564f5485d40 2024-03-19T01:19:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.926+0000 7fc5fd200640 1 -- 172.21.15.69:0/4091971136 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc60001a7c0 con 0x5564f5485d40 2024-03-19T01:19:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.926+0000 7fc5fd200640 1 -- 172.21.15.69:0/4091971136 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600019b70 con 0x5564f5485d40 2024-03-19T01:19:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.927+0000 7fc6123ef740 1 -- 172.21.15.69:0/4091971136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 msgr2=0x5564f548e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.927+0000 7fc6123ef740 1 --2- 172.21.15.69:0/4091971136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f548e9e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5564f5493350 tx=0x7fc600007940 comp rx=0 tx=0).stop 2024-03-19T01:19:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.928+0000 7fc6123ef740 1 -- 172.21.15.69:0/4091971136 shutdown_connections 2024-03-19T01:19:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.928+0000 7fc6123ef740 1 --2- 172.21.15.69:0/4091971136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f548cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.928+0000 7fc6123ef740 1 --2- 172.21.15.69:0/4091971136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f5486660 0x5564f547f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.928+0000 7fc6123ef740 1 --2- 172.21.15.69:0/4091971136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5485d40 0x5564f548e9e0 secure :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5564f5493350 tx=0x7fc600007940 comp rx=0 tx=0).stop 2024-03-19T01:19:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.928+0000 7fc6123ef740 1 -- 172.21.15.69:0/4091971136 >> 172.21.15.69:0/4091971136 conn(0x5564f5480470 msgr2=0x5564f5481770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.929+0000 7fc6123ef740 1 -- 172.21.15.69:0/4091971136 shutdown_connections 2024-03-19T01:19:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.929+0000 7fc6123ef740 1 -- 172.21.15.69:0/4091971136 wait complete. 2024-03-19T01:19:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.930+0000 7fc6123ef740 1 -- start start 2024-03-19T01:19:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.930+0000 7fc6123ef740 1 Processor -- start 2024-03-19T01:19:34.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.931+0000 7fc6123ef740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5486660 0x5564f54a8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.932+0000 7fc6123ef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f54a8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.932+0000 7fc60cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5486660 0x5564f54a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.932+0000 7fc60cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5486660 0x5564f54a8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55972/0 (socket says 172.21.15.69:55972) 2024-03-19T01:19:34.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.933+0000 7fc607e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f54a8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.933+0000 7fc607e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f54a8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49898/0 (socket says 172.21.15.69:49898) 2024-03-19T01:19:34.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.933+0000 7fc60cc00640 1 -- 172.21.15.69:0/3356070830 learned_addr learned my addr 172.21.15.69:0/3356070830 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:34.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.933+0000 7fc6123ef740 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f54a1c10 0x5564f54a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.934+0000 7fc60d600640 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f54a1c10 0x5564f54a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.934+0000 7fc6123ef740 1 -- 172.21.15.69:0/3356070830 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e80114e0 con 0x5564f548f240 2024-03-19T01:19:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.934+0000 7fc6123ef740 1 -- 172.21.15.69:0/3356070830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e8010e10 con 0x5564f54a1c10 2024-03-19T01:19:34.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.934+0000 7fc6123ef740 1 -- 172.21.15.69:0/3356070830 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e800dfc0 con 0x5564f5486660 2024-03-19T01:19:34.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.935+0000 7fc607e00640 1 -- 172.21.15.69:0/3356070830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5486660 msgr2=0x5564f54a8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:34.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.935+0000 7fc607e00640 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5486660 0x5564f54a8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.935+0000 7fc607e00640 1 -- 172.21.15.69:0/3356070830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f54a1c10 msgr2=0x5564f54a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:34.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.935+0000 7fc607e00640 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f54a1c10 0x5564f54a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.935+0000 7fc607e00640 1 -- 172.21.15.69:0/3356070830 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564f54a2730 con 0x5564f548f240 2024-03-19T01:19:34.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.935+0000 7fc60d600640 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f54a1c10 0x5564f54a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:34.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.936+0000 7fc6123ef740 10 client.0 _ll_register_callbacks cb 0x7fc5f409ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:34.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.936+0000 7fc6123ef740 1 -- 172.21.15.69:0/3356070830 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc60001a7c0 con 0x5564f548f240 2024-03-19T01:19:34.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.936+0000 7fc607e00640 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f54a8590 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc5f409b2e0 tx=0x7fc5f40cce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6316d33e52b377ce server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:34.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.936+0000 7fc5f2000640 10 client.17187 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:19:34.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.937+0000 7fc5f2000640 1 -- 172.21.15.69:0/3356070830 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40b3070 con 0x5564f548f240 2024-03-19T01:19:34.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.937+0000 7fc5f2000640 1 -- 172.21.15.69:0/3356070830 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fc5f409a9f0 con 0x5564f548f240 2024-03-19T01:19:34.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.937+0000 7fc5f2000640 1 -- 172.21.15.69:0/3356070830 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fc5f40ccbe0 con 0x5564f548f240 2024-03-19T01:19:34.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.937+0000 7fc6123ef740 10 client.17187 fetch_fsmap learned FSMap version 53 2024-03-19T01:19:34.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.938+0000 7fc6123ef740 1 -- 172.21.15.69:0/3356070830 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=53}) -- 0x5564f5506060 con 0x5564f548f240 2024-03-19T01:19:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.938+0000 7fc5f2000640 1 -- 172.21.15.69:0/3356070830 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 53) ==== 8390+0+0 (secure 0 0 0) 0x7fc5f407e060 con 0x5564f548f240 2024-03-19T01:19:34.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.939+0000 7fc6123ef740 10 client.17187 fetch_fsmap finished waiting for FSMap version 53 2024-03-19T01:19:34.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.939+0000 7fc6123ef740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:34.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.939+0000 7fc6123ef740 20 client.17187 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:34.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.939+0000 7fc6123ef740 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5564f55068b0 0x5564f5508d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:34.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.939+0000 7fc6123ef740 4 client.17187 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:19:34.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.939+0000 7fc6123ef740 1 -- 172.21.15.69:0/3356070830 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5564f53f7800 con 0x5564f55068b0 2024-03-19T01:19:34.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.939+0000 7fc60cc00640 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5564f55068b0 0x5564f5508d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:34.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.941+0000 7fc60cc00640 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5564f55068b0 0x5564f5508d70 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c20ca4838fda300c server_cookie=666e74c6a1b10631 in_seq=0 out_seq=0 2024-03-19T01:19:34.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.941+0000 7fc5f2000640 10 client.17187 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:34.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.942+0000 7fc5f2000640 1 -- 172.21.15.69:0/3356070830 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5564f53f7800 con 0x5564f55068b0 2024-03-19T01:19:34.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.942+0000 7fc5f2000640 10 client.17187 handle_command_reply: tid=0 2024-03-19T01:19:34.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.942+0000 7fc6123ef740 1 client.17187 shutdown 2024-03-19T01:19:34.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.942+0000 7fc6123ef740 1 -- 172.21.15.69:0/3356070830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 msgr2=0x5564f54a8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:34.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.942+0000 7fc6123ef740 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f54a8590 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc5f409b2e0 tx=0x7fc5f40cce00 comp rx=0 tx=0).stop 2024-03-19T01:19:34.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.946+0000 7fc6123ef740 1 -- 172.21.15.69:0/3356070830 shutdown_connections 2024-03-19T01:19:34.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.946+0000 7fc6123ef740 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5564f55068b0 0x5564f5508d70 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.946+0000 7fc5f2000640 0 client.17187 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:34.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.946+0000 7fc6123ef740 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564f54a1c10 0x5564f54a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.946+0000 7fc6123ef740 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5564f548f240 0x5564f54a8590 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.947+0000 7fc6123ef740 1 --2- 172.21.15.69:0/3356070830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5564f5486660 0x5564f54a8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.947+0000 7fc6123ef740 1 -- 172.21.15.69:0/3356070830 >> 172.21.15.69:0/3356070830 conn(0x5564f5480470 msgr2=0x5564f547f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:34.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.947+0000 7fc6123ef740 1 -- 172.21.15.69:0/3356070830 shutdown_connections 2024-03-19T01:19:34.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.947+0000 7fc6123ef740 1 -- 172.21.15.69:0/3356070830 wait complete. 2024-03-19T01:19:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.948+0000 7fc6123ef740 20 client.17187 trim_cache size 0 max 16384 2024-03-19T01:19:34.963 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:19:34.963 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:19:34.963 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:19:34.963 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:19:34.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.957+0000 7fc60f600640 1 -- 172.21.15.69:0/2062112895 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5ec0a2ab0 msgr2=0x7fc5ec0a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:34.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.957+0000 7fc60f600640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5ec0a2ab0 0x7fc5ec0a4f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc60806ac40 tx=0x7fc5f8005520 comp rx=0 tx=0).stop 2024-03-19T01:19:34.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.957+0000 7fc60f600640 1 -- 172.21.15.69:0/2062112895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc608109320 msgr2=0x7fc608119dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:34.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.957+0000 7fc60f600640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc608109320 0x7fc608119dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400cb00 tx=0x7fc5f40071c0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.963+0000 7fc60f600640 1 -- 172.21.15.69:0/2062112895 shutdown_connections 2024-03-19T01:19:34.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.963+0000 7fc60f600640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc5ec0a2ab0 0x7fc5ec0a4f70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.963+0000 7fc60f600640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc608114970 0x7fc60811a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.963+0000 7fc60f600640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc608109320 0x7fc608119dc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.963+0000 7fc60f600640 1 --2- 172.21.15.69:0/2062112895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc608108a00 0x7fc60811d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:34.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.963+0000 7fc60f600640 1 -- 172.21.15.69:0/2062112895 >> 172.21.15.69:0/2062112895 conn(0x7fc60806c900 msgr2=0x7fc608070480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:34.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.973+0000 7fc60f600640 1 -- 172.21.15.69:0/2062112895 shutdown_connections 2024-03-19T01:19:34.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:34.973+0000 7fc60f600640 1 -- 172.21.15.69:0/2062112895 wait complete. 2024-03-19T01:19:35.023 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:19:35.024 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:19:35.029 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/extable.o 2024-03-19T01:19:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:34 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:19:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:34 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:19:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:34 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:19:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:34 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:19:35.202 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/module/procfs.o 2024-03-19T01:19:35.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:34 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:19:35.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:34 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:19:35.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:34 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:19:35.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:34 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:19:35.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:34 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:19:35.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:34 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:19:35.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:34 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:19:35.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:34 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:19:35.292 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/jbd2/journal.o 2024-03-19T01:19:35.371 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crypto/utils.o 2024-03-19T01:19:35.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.532+0000 7f9f24000640 1 -- 172.21.15.69:0/1582133234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f1c114970 msgr2=0x7f9f1c116d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:35.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.532+0000 7f9f24000640 1 --2- 172.21.15.69:0/1582133234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f1c114970 0x7f9f1c116d60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9f140098b0 tx=0x7f9f1402bc90 comp rx=0 tx=0).stop 2024-03-19T01:19:35.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.533+0000 7f9f24000640 1 -- 172.21.15.69:0/1582133234 shutdown_connections 2024-03-19T01:19:35.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.533+0000 7f9f24000640 1 --2- 172.21.15.69:0/1582133234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f1c114970 0x7f9f1c116d60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:35.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.533+0000 7f9f24000640 1 --2- 172.21.15.69:0/1582133234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f1c109330 0x7f9f1c114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:35.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.533+0000 7f9f24000640 1 --2- 172.21.15.69:0/1582133234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c108a10 0x7f9f1c108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:35.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.533+0000 7f9f24000640 1 -- 172.21.15.69:0/1582133234 >> 172.21.15.69:0/1582133234 conn(0x7f9f1c06c910 msgr2=0x7f9f1c06cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:35.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.533+0000 7f9f24000640 1 -- 172.21.15.69:0/1582133234 shutdown_connections 2024-03-19T01:19:35.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.533+0000 7f9f24000640 1 -- 172.21.15.69:0/1582133234 wait complete. 2024-03-19T01:19:35.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.535+0000 7f9f24000640 1 Processor -- start 2024-03-19T01:19:35.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.538+0000 7f9f24000640 1 -- start start 2024-03-19T01:19:35.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.538+0000 7f9f24000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f1c108a10 0x7f9f1c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:35.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.538+0000 7f9f24000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c109330 0x7f9f1c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:35.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.539+0000 7f9f24000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f1c114970 0x7f9f1c1a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:35.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.539+0000 7f9f24000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c1181f0 con 0x7f9f1c114970 2024-03-19T01:19:35.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.539+0000 7f9f21600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f1c108a10 0x7f9f1c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:35.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.539+0000 7f9f21600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f1c108a10 0x7f9f1c19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:37714/0 (socket says 172.21.15.69:37714) 2024-03-19T01:19:35.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.539+0000 7f9f21600640 1 -- 172.21.15.69:0/3217806618 learned_addr learned my addr 172.21.15.69:0/3217806618 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:35.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.539+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c118070 con 0x7f9f1c108a10 2024-03-19T01:19:35.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.539+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c118370 con 0x7f9f1c109330 2024-03-19T01:19:35.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.539+0000 7f9f20c00640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c109330 0x7f9f1c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:35.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.540+0000 7f9f22000640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f1c114970 0x7f9f1c1a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:35.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.540+0000 7f9f21600640 1 -- 172.21.15.69:0/3217806618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c109330 msgr2=0x7f9f1c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:35.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.540+0000 7f9f21600640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c109330 0x7f9f1c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:35.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.540+0000 7f9f21600640 1 -- 172.21.15.69:0/3217806618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f1c114970 msgr2=0x7f9f1c1a2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:35.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.540+0000 7f9f21600640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f1c114970 0x7f9f1c1a2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:35.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.540+0000 7f9f21600640 1 -- 172.21.15.69:0/3217806618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c1a36d0 con 0x7f9f1c108a10 2024-03-19T01:19:35.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.540+0000 7f9f22000640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f1c114970 0x7f9f1c1a2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:19:35.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.540+0000 7f9f21600640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f1c108a10 0x7f9f1c19e730 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000efb0 tx=0x7f9f1000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:35.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.541+0000 7f9f1a000640 1 -- 172.21.15.69:0/3217806618 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1001b070 con 0x7f9f1c108a10 2024-03-19T01:19:35.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.541+0000 7f9f1a000640 1 -- 172.21.15.69:0/3217806618 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1000ba50 con 0x7f9f1c108a10 2024-03-19T01:19:35.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.541+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1c1a39c0 con 0x7f9f1c108a10 2024-03-19T01:19:35.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.542+0000 7f9f1a000640 1 -- 172.21.15.69:0/3217806618 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10016c00 con 0x7f9f1c108a10 2024-03-19T01:19:35.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.543+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1c1ab250 con 0x7f9f1c108a10 2024-03-19T01:19:35.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.548+0000 7f9f1a000640 1 -- 172.21.15.69:0/3217806618 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9f1001e070 con 0x7f9f1c108a10 2024-03-19T01:19:35.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.553+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f1c06d4d0 con 0x7f9f1c108a10 2024-03-19T01:19:35.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.553+0000 7f9f1a000640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9f040a6c60 0x7f9f040a9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:35.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.554+0000 7f9f20c00640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9f040a6c60 0x7f9f040a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:35.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.554+0000 7f9f1a000640 1 -- 172.21.15.69:0/3217806618 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7f9f100d2170 con 0x7f9f1c108a10 2024-03-19T01:19:35.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.555+0000 7f9f20c00640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9f040a6c60 0x7f9f040a9120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c072ed0 tx=0x7f9f080090e0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:35.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:35.558+0000 7f9f1a000640 1 -- 172.21.15.69:0/3217806618 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9f1009b8d0 con 0x7f9f1c108a10 2024-03-19T01:19:36.017 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/extents_status.o 2024-03-19T01:19:36.030 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crypto/chacha.o 2024-03-19T01:19:36.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:35 smithi163 ceph-mon[67428]: pgmap v276: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 847 KiB/s wr, 101 op/s 2024-03-19T01:19:36.092 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/module/sysfs.o 2024-03-19T01:19:36.126 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/mmap.o 2024-03-19T01:19:36.188 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/swap_state.o 2024-03-19T01:19:36.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.199+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f1c19fa60 con 0x7f9f1c108a10 2024-03-19T01:19:36.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.218+0000 7f9f1a000640 1 -- 172.21.15.69:0/3217806618 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11942 (secure 0 0 0) 0x7f9f1c19fa60 con 0x7f9f1c108a10 2024-03-19T01:19:36.221 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:19:36.222 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":53,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:19:17.903865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24550,24494,24580,24581,24476]},"id":1}]} 2024-03-19T01:19:36.222 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 53 2024-03-19T01:19:36.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.228+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9f040a6c60 msgr2=0x7f9f040a9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:36.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.228+0000 7f9f24000640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9f040a6c60 0x7f9f040a9120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c072ed0 tx=0x7f9f080090e0 comp rx=0 tx=0).stop 2024-03-19T01:19:36.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.228+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f1c108a10 msgr2=0x7f9f1c19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:36.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.228+0000 7f9f24000640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f1c108a10 0x7f9f1c19e730 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000efb0 tx=0x7f9f1000c800 comp rx=0 tx=0).stop 2024-03-19T01:19:36.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.228+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 shutdown_connections 2024-03-19T01:19:36.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.228+0000 7f9f24000640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f1c114970 0x7f9f1c1a2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:36.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.228+0000 7f9f24000640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c109330 0x7f9f1c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:36.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.228+0000 7f9f24000640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9f040a6c60 0x7f9f040a9120 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:36.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.228+0000 7f9f24000640 1 --2- 172.21.15.69:0/3217806618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f1c108a10 0x7f9f1c19e730 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:36.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.228+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 >> 172.21.15.69:0/3217806618 conn(0x7f9f1c06c910 msgr2=0x7f9f1c070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:36.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.229+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 shutdown_connections 2024-03-19T01:19:36.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:36.229+0000 7f9f24000640 1 -- 172.21.15.69:0/3217806618 wait complete. 2024-03-19T01:19:36.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:35 smithi086 ceph-mon[66829]: pgmap v276: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 847 KiB/s wr, 101 op/s 2024-03-19T01:19:36.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:35 smithi069 ceph-mon[31727]: pgmap v276: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 847 KiB/s wr, 101 op/s 2024-03-19T01:19:36.283 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:19:36.294 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crypto/aes.o 2024-03-19T01:19:36.660 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/pgtable.o 2024-03-19T01:19:37.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:36 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3217806618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:19:37.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:36 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3217806618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:19:37.090 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crypto/arc4.o 2024-03-19T01:19:37.103 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/clear_page_64.o 2024-03-19T01:19:37.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.147+0000 7f1636400640 1 -- 172.21.15.69:0/2469799813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630108a00 msgr2=0x7f1630108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.147+0000 7f1636400640 1 --2- 172.21.15.69:0/2469799813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630108a00 0x7f1630108de0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f161c009860 tx=0x7f161c02bb70 comp rx=0 tx=0).stop 2024-03-19T01:19:37.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.147+0000 7f1636400640 1 -- 172.21.15.69:0/2469799813 shutdown_connections 2024-03-19T01:19:37.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.147+0000 7f1636400640 1 --2- 172.21.15.69:0/2469799813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1630114970 0x7f1630116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.147+0000 7f1636400640 1 --2- 172.21.15.69:0/2469799813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630109320 0x7f1630114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.147+0000 7f1636400640 1 --2- 172.21.15.69:0/2469799813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630108a00 0x7f1630108de0 secure :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f161c009860 tx=0x7f161c02bb70 comp rx=0 tx=0).stop 2024-03-19T01:19:37.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.147+0000 7f1636400640 1 -- 172.21.15.69:0/2469799813 >> 172.21.15.69:0/2469799813 conn(0x7f163006c900 msgr2=0x7f163006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:37.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.148+0000 7f1636400640 1 -- 172.21.15.69:0/2469799813 shutdown_connections 2024-03-19T01:19:37.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.149+0000 7f1636400640 1 -- 172.21.15.69:0/2469799813 wait complete. 2024-03-19T01:19:37.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.150+0000 7f1636400640 1 Processor -- start 2024-03-19T01:19:37.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.150+0000 7f1636400640 1 -- start start 2024-03-19T01:19:37.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f1636400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109320 0x7f16301a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f1636400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630114970 0x7f16301a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f1636400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16301a0920 0x7f16301a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f1636400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f163011a3b0 con 0x7f16301a0920 2024-03-19T01:19:37.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f1636400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f163011a230 con 0x7f1630114970 2024-03-19T01:19:37.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f1636400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f163011a530 con 0x7f1630109320 2024-03-19T01:19:37.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f162f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109320 0x7f16301a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f162f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109320 0x7f16301a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:43776/0 (socket says 172.21.15.69:43776) 2024-03-19T01:19:37.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f162f400640 1 -- 172.21.15.69:0/364772786 learned_addr learned my addr 172.21.15.69:0/364772786 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:37.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f162ea00640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630114970 0x7f16301a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f162f400640 1 -- 172.21.15.69:0/364772786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630114970 msgr2=0x7f16301a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f162fe00640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16301a0920 0x7f16301a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f162f400640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630114970 0x7f16301a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.151+0000 7f162f400640 1 -- 172.21.15.69:0/364772786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16301a0920 msgr2=0x7f16301a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.152+0000 7f162f400640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16301a0920 0x7f16301a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.152+0000 7f162f400640 1 -- 172.21.15.69:0/364772786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16301a1440 con 0x7f1630109320 2024-03-19T01:19:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.152+0000 7f162fe00640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16301a0920 0x7f16301a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.152+0000 7f162f400640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109320 0x7f16301a6770 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f161c02c100 tx=0x7f161c0346e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.152+0000 7f1627e00640 1 -- 172.21.15.69:0/364772786 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c041020 con 0x7f1630109320 2024-03-19T01:19:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.152+0000 7f1627e00640 1 -- 172.21.15.69:0/364772786 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f161c02c9f0 con 0x7f1630109320 2024-03-19T01:19:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.152+0000 7f1627e00640 1 -- 172.21.15.69:0/364772786 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c039040 con 0x7f1630109320 2024-03-19T01:19:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.157+0000 7f1636400640 1 -- 172.21.15.69:0/364772786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16301a16d0 con 0x7f1630109320 2024-03-19T01:19:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.157+0000 7f1636400640 1 -- 172.21.15.69:0/364772786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163010a400 con 0x7f1630109320 2024-03-19T01:19:37.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.165+0000 7f1627e00640 1 -- 172.21.15.69:0/364772786 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f161c044020 con 0x7f1630109320 2024-03-19T01:19:37.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.167+0000 7f1627e00640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f16140a2b70 0x7f16140a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.168+0000 7f1627e00640 1 -- 172.21.15.69:0/364772786 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7f161c0efdf0 con 0x7f1630109320 2024-03-19T01:19:37.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.168+0000 7f163918a740 1 Processor -- start 2024-03-19T01:19:37.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.168+0000 7f163918a740 1 -- start start 2024-03-19T01:19:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.168+0000 7f163918a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415e230 0x55603415e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.168+0000 7f163918a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55603415eb50 0x5560341671f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.168+0000 7f163918a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556034167730 0x556034169b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.168+0000 7f163918a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5560340ba380 con 0x556034167730 2024-03-19T01:19:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.168+0000 7f163918a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556034062d20 con 0x55603415e230 2024-03-19T01:19:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.168+0000 7f163918a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5560340b6420 con 0x55603415eb50 2024-03-19T01:19:37.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.169+0000 7f162fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556034167730 0x556034169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.169+0000 7f162fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556034167730 0x556034169b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59452/0 (socket says 172.21.15.69:59452) 2024-03-19T01:19:37.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.169+0000 7f162fe00640 1 -- 172.21.15.69:0/188932410 learned_addr learned my addr 172.21.15.69:0/188932410 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:37.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.170+0000 7f162f400640 1 --2- 172.21.15.69:0/188932410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415e230 0x55603415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.170+0000 7f162fe00640 1 -- 172.21.15.69:0/188932410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55603415eb50 msgr2=0x5560341671f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:37.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.170+0000 7f162fe00640 1 --2- 172.21.15.69:0/188932410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55603415eb50 0x5560341671f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.170+0000 7f162fe00640 1 -- 172.21.15.69:0/188932410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415e230 msgr2=0x55603415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:37.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.170+0000 7f162fe00640 1 --2- 172.21.15.69:0/188932410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415e230 0x55603415e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.170+0000 7f162fe00640 1 -- 172.21.15.69:0/188932410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556033dce3f0 con 0x556034167730 2024-03-19T01:19:37.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.170+0000 7f162ea00640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f16140a2b70 0x7f16140a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.171+0000 7f162f400640 1 --2- 172.21.15.69:0/188932410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415e230 0x55603415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:37.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.172+0000 7f162fe00640 1 --2- 172.21.15.69:0/188932410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556034167730 0x556034169b20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1620005fd0 tx=0x7f162000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ee04550f0515aa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.172+0000 7f162ea00640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f16140a2b70 0x7f16140a5030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16301167e0 tx=0x7f1628009480 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.175+0000 7f1624c00640 1 -- 172.21.15.69:0/188932410 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620016070 con 0x556034167730 2024-03-19T01:19:37.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.175+0000 7f1624c00640 1 -- 172.21.15.69:0/188932410 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1620005500 con 0x556034167730 2024-03-19T01:19:37.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.175+0000 7f1624c00640 1 -- 172.21.15.69:0/188932410 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620011470 con 0x556034167730 2024-03-19T01:19:37.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.176+0000 7f163918a740 1 -- 172.21.15.69:0/188932410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556034167730 msgr2=0x556034169b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.176+0000 7f163918a740 1 --2- 172.21.15.69:0/188932410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556034167730 0x556034169b20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1620005fd0 tx=0x7f162000e6c0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.178+0000 7f163918a740 1 -- 172.21.15.69:0/188932410 shutdown_connections 2024-03-19T01:19:37.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.178+0000 7f163918a740 1 --2- 172.21.15.69:0/188932410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556034167730 0x556034169b20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.178+0000 7f163918a740 1 --2- 172.21.15.69:0/188932410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55603415eb50 0x5560341671f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.178+0000 7f163918a740 1 --2- 172.21.15.69:0/188932410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415e230 0x55603415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.178+0000 7f163918a740 1 -- 172.21.15.69:0/188932410 >> 172.21.15.69:0/188932410 conn(0x556034158960 msgr2=0x556034158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:37.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.178+0000 7f163918a740 1 -- 172.21.15.69:0/188932410 shutdown_connections 2024-03-19T01:19:37.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.178+0000 7f163918a740 1 -- 172.21.15.69:0/188932410 wait complete. 2024-03-19T01:19:37.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.180+0000 7f163918a740 1 -- start start 2024-03-19T01:19:37.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.180+0000 7f163918a740 1 Processor -- start 2024-03-19T01:19:37.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.181+0000 7f163918a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415e230 0x5560341648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.181+0000 7f163918a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55603415eb50 0x556034164e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.181+0000 7f163918a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556034167730 0x556034165350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.181+0000 7f163918a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55603416a570 con 0x55603415e230 2024-03-19T01:19:37.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.181+0000 7f162f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415e230 0x5560341648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.182+0000 7f163918a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556034076160 con 0x556034167730 2024-03-19T01:19:37.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.182+0000 7f162f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415e230 0x5560341648d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59464/0 (socket says 172.21.15.69:59464) 2024-03-19T01:19:37.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.182+0000 7f162f400640 1 -- 172.21.15.69:0/4289603283 learned_addr learned my addr 172.21.15.69:0/4289603283 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:37.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.182+0000 7f163918a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55603416a6f0 con 0x55603415eb50 2024-03-19T01:19:37.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.182+0000 7f163918a740 10 client.0 _ll_register_callbacks cb 0x5560340b2170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:37.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.182+0000 7f162f400640 1 -- 172.21.15.69:0/4289603283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55603415eb50 msgr2=0x556034164e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:37.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.182+0000 7f162f400640 1 --2- 172.21.15.69:0/4289603283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55603415eb50 0x556034164e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.182+0000 7f162f400640 1 -- 172.21.15.69:0/4289603283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556034167730 msgr2=0x556034165350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:19:37.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.182+0000 7f162f400640 1 --2- 172.21.15.69:0/4289603283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556034167730 0x556034165350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.182+0000 7f162f400640 1 -- 172.21.15.69:0/4289603283 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556033e987d0 con 0x55603415e230 2024-03-19T01:19:37.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.183+0000 7f162f400640 1 --2- 172.21.15.69:0/4289603283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415e230 0x5560341648d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f161c0c4380 tx=0x7f161c0bbbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8aad71b6d789ec6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.183+0000 7f1619600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:19:37.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.183+0000 7f1619600640 1 -- 172.21.15.69:0/4289603283 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0c02f0 con 0x55603415e230 2024-03-19T01:19:37.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.183+0000 7f1619600640 1 -- 172.21.15.69:0/4289603283 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7f161c003940 con 0x55603415e230 2024-03-19T01:19:37.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.184+0000 7f163918a740 1 -- 172.21.15.69:0/4289603283 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1620005500 con 0x55603415e230 2024-03-19T01:19:37.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.185+0000 7f1619600640 1 -- 172.21.15.69:0/4289603283 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f161c0c48b0 con 0x55603415e230 2024-03-19T01:19:37.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.185+0000 7f163918a740 10 client.17202 fetch_fsmap learned FSMap version 53 2024-03-19T01:19:37.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.185+0000 7f163918a740 1 -- 172.21.15.69:0/4289603283 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x556034176900 con 0x55603415e230 2024-03-19T01:19:37.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.187+0000 7f1619600640 1 -- 172.21.15.69:0/4289603283 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1620005500 con 0x55603415e230 2024-03-19T01:19:37.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.188+0000 7f1619600640 1 -- 172.21.15.69:0/4289603283 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 53) ==== 8390+0+0 (secure 0 0 0) 0x7f161c03c030 con 0x55603415e230 2024-03-19T01:19:37.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.188+0000 7f163918a740 10 client.17202 fetch_fsmap finished waiting for FSMap version 53 2024-03-19T01:19:37.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.188+0000 7f163918a740 10 client.17202 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:19:37.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.188+0000 7f163918a740 20 client.17202 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:37.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.188+0000 7f163918a740 1 --2- 172.21.15.69:0/4289603283 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556034177da0 0x55603417a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.189+0000 7f163918a740 4 client.17202 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:37.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.189+0000 7f162ea00640 1 --2- 172.21.15.69:0/4289603283 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556034177da0 0x55603417a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.189+0000 7f163918a740 1 -- 172.21.15.69:0/4289603283 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560340c78c0 con 0x556034177da0 2024-03-19T01:19:37.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.190+0000 7f162ea00640 1 --2- 172.21.15.69:0/4289603283 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556034177da0 0x55603417a210 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aa30247554952494 server_cookie=c5112f414229466b in_seq=0 out_seq=0 2024-03-19T01:19:37.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.194+0000 7f1619600640 10 client.17202 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:19:37.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.195+0000 7f1619600640 1 -- 172.21.15.69:0/4289603283 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5560340c78c0 con 0x556034177da0 2024-03-19T01:19:37.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.195+0000 7f1619600640 10 client.17202 handle_command_reply: tid=0 2024-03-19T01:19:37.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.195+0000 7f163918a740 1 client.17202 shutdown 2024-03-19T01:19:37.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.195+0000 7f163918a740 1 -- 172.21.15.69:0/4289603283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415e230 msgr2=0x5560341648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.195+0000 7f163918a740 1 --2- 172.21.15.69:0/4289603283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415e230 0x5560341648d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f161c0c4380 tx=0x7f161c0bbbb0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.196+0000 7f163918a740 1 -- 172.21.15.69:0/4289603283 shutdown_connections 2024-03-19T01:19:37.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.196+0000 7f163918a740 1 --2- 172.21.15.69:0/4289603283 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x556034177da0 0x55603417a210 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.196+0000 7f163918a740 1 --2- 172.21.15.69:0/4289603283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556034167730 0x556034165350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.196+0000 7f163918a740 1 --2- 172.21.15.69:0/4289603283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55603415eb50 0x556034164e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.196+0000 7f163918a740 1 --2- 172.21.15.69:0/4289603283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415e230 0x5560341648d0 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f161c0c4380 tx=0x7f161c0bbbb0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.196+0000 7f163918a740 1 -- 172.21.15.69:0/4289603283 >> 172.21.15.69:0/4289603283 conn(0x556034158960 msgr2=0x55603415a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:37.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.196+0000 7f1619600640 0 client.17202 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:19:37.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.197+0000 7f163918a740 1 -- 172.21.15.69:0/4289603283 shutdown_connections 2024-03-19T01:19:37.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.197+0000 7f163918a740 1 -- 172.21.15.69:0/4289603283 wait complete. 2024-03-19T01:19:37.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.197+0000 7f163918a740 20 client.17202 trim_cache size 0 max 16384 2024-03-19T01:19:37.222 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/cmdline.o 2024-03-19T01:19:37.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:36 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3217806618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.255+0000 7f163918a740 1 Processor -- start 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.255+0000 7f163918a740 1 -- start start 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f163918a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415eb50 0x556034167000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f163918a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556034167730 0x556034173310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f163918a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415f4d0 0x55603417fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f163918a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b5900 con 0x55603415eb50 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f163918a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b7590 con 0x55603415f4d0 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f163918a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b5fd0 con 0x556034167730 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f162fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415f4d0 0x55603417fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f162ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556034167730 0x556034173310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f162fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415f4d0 0x55603417fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60844/0 (socket says 172.21.15.69:60844) 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f162fe00640 1 -- 172.21.15.69:0/968193094 learned_addr learned my addr 172.21.15.69:0/968193094 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:37.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f162f400640 1 --2- 172.21.15.69:0/968193094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415eb50 0x556034167000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f162ea00640 1 -- 172.21.15.69:0/968193094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415f4d0 msgr2=0x55603417fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f162ea00640 1 --2- 172.21.15.69:0/968193094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415f4d0 0x55603417fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f162ea00640 1 -- 172.21.15.69:0/968193094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415eb50 msgr2=0x556034167000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f162ea00640 1 --2- 172.21.15.69:0/968193094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415eb50 0x556034167000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f162ea00640 1 -- 172.21.15.69:0/968193094 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55603417a890 con 0x556034167730 2024-03-19T01:19:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.256+0000 7f162f400640 1 --2- 172.21.15.69:0/968193094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415eb50 0x556034167000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.257+0000 7f162ea00640 1 --2- 172.21.15.69:0/968193094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556034167730 0x556034173310 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55603417aac0 tx=0x7f1628012040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70c52da4ff5c5c9d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.257+0000 7f161be00640 1 -- 172.21.15.69:0/968193094 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628009610 con 0x556034167730 2024-03-19T01:19:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.257+0000 7f161be00640 1 -- 172.21.15.69:0/968193094 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f162801d8d0 con 0x556034167730 2024-03-19T01:19:37.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.257+0000 7f161be00640 1 -- 172.21.15.69:0/968193094 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162801ca30 con 0x556034167730 2024-03-19T01:19:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.258+0000 7f163918a740 1 -- 172.21.15.69:0/968193094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556034167730 msgr2=0x556034173310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.258+0000 7f163918a740 1 --2- 172.21.15.69:0/968193094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556034167730 0x556034173310 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55603417aac0 tx=0x7f1628012040 comp rx=0 tx=0).stop 2024-03-19T01:19:37.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.262+0000 7f163918a740 1 -- 172.21.15.69:0/968193094 shutdown_connections 2024-03-19T01:19:37.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.262+0000 7f163918a740 1 --2- 172.21.15.69:0/968193094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415f4d0 0x55603417fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.262+0000 7f163918a740 1 --2- 172.21.15.69:0/968193094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556034167730 0x556034173310 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.262+0000 7f163918a740 1 --2- 172.21.15.69:0/968193094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415eb50 0x556034167000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.262+0000 7f163918a740 1 -- 172.21.15.69:0/968193094 >> 172.21.15.69:0/968193094 conn(0x556034158960 msgr2=0x55603415a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.262+0000 7f163918a740 1 -- 172.21.15.69:0/968193094 shutdown_connections 2024-03-19T01:19:37.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.262+0000 7f163918a740 1 -- 172.21.15.69:0/968193094 wait complete. 2024-03-19T01:19:37.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.262+0000 7f163918a740 1 -- start start 2024-03-19T01:19:37.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.263+0000 7f163918a740 1 Processor -- start 2024-03-19T01:19:37.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.263+0000 7f163918a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415eb50 0x55603416f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.263+0000 7f163918a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415f4d0 0x55603416c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.263+0000 7f163918a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556034167730 0x55603416ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.263+0000 7f163918a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b6e00 con 0x55603415f4d0 2024-03-19T01:19:37.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.263+0000 7f163918a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b6730 con 0x55603415eb50 2024-03-19T01:19:37.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f163918a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b38e0 con 0x556034167730 2024-03-19T01:19:37.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f163918a740 10 client.0 _ll_register_callbacks cb 0x7f161c0bc020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:37.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f162f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415eb50 0x55603416f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f162f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415eb50 0x55603416f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60852/0 (socket says 172.21.15.69:60852) 2024-03-19T01:19:37.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f162f400640 1 -- 172.21.15.69:0/2111557912 learned_addr learned my addr 172.21.15.69:0/2111557912 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f162ea00640 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415f4d0 0x55603416c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f162f400640 1 -- 172.21.15.69:0/2111557912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556034167730 msgr2=0x55603416ce50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f162f400640 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556034167730 0x55603416ce50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f162f400640 1 -- 172.21.15.69:0/2111557912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415f4d0 msgr2=0x55603416c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f162f400640 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415f4d0 0x55603416c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f162f400640 1 -- 172.21.15.69:0/2111557912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55603416ff30 con 0x55603415eb50 2024-03-19T01:19:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.264+0000 7f162ea00640 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415f4d0 0x55603416c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:37.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.265+0000 7f162f400640 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415eb50 0x55603416f940 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f161c000c00 tx=0x7f161c0bbce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55cdc7191a61476 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.265+0000 7f1619600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:19:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.265+0000 7f1619600640 1 -- 172.21.15.69:0/2111557912 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0dc0b0 con 0x55603415eb50 2024-03-19T01:19:37.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.265+0000 7f1619600640 1 -- 172.21.15.69:0/2111557912 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7f161c02cba0 con 0x55603415eb50 2024-03-19T01:19:37.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.268+0000 7f163918a740 1 -- 172.21.15.69:0/2111557912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f162801d8d0 con 0x55603415eb50 2024-03-19T01:19:37.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.270+0000 7f1619600640 1 -- 172.21.15.69:0/2111557912 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f161c0c02f0 con 0x55603415eb50 2024-03-19T01:19:37.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.270+0000 7f163918a740 10 client.27506 fetch_fsmap learned FSMap version 53 2024-03-19T01:19:37.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.270+0000 7f163918a740 1 -- 172.21.15.69:0/2111557912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5560341503d0 con 0x55603415eb50 2024-03-19T01:19:37.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.270+0000 7f1619600640 1 -- 172.21.15.69:0/2111557912 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f162801d8d0 con 0x55603415eb50 2024-03-19T01:19:37.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.270+0000 7f1619600640 1 -- 172.21.15.69:0/2111557912 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 8390+0+0 (secure 0 0 0) 0x7f161c0c4b00 con 0x55603415eb50 2024-03-19T01:19:37.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.270+0000 7f163918a740 10 client.27506 fetch_fsmap finished waiting for FSMap version 53 2024-03-19T01:19:37.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.270+0000 7f163918a740 10 client.27506 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:19:37.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.270+0000 7f163918a740 20 client.27506 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:37.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.270+0000 7f163918a740 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5560341509c0 0x556034152de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.270+0000 7f163918a740 4 client.27506 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:19:37.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.270+0000 7f163918a740 1 -- 172.21.15.69:0/2111557912 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560340c1c10 con 0x5560341509c0 2024-03-19T01:19:37.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.271+0000 7f162ea00640 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5560341509c0 0x556034152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.273+0000 7f162ea00640 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5560341509c0 0x556034152de0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=19c27168723d9dc8 server_cookie=f745c672aebc0605 in_seq=0 out_seq=0 2024-03-19T01:19:37.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.273+0000 7f1619600640 10 client.27506 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:19:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.275+0000 7f1619600640 1 -- 172.21.15.69:0/2111557912 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560340c1c10 con 0x5560341509c0 2024-03-19T01:19:37.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.275+0000 7f1619600640 10 client.27506 handle_command_reply: tid=0 2024-03-19T01:19:37.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.275+0000 7f163918a740 1 client.27506 shutdown 2024-03-19T01:19:37.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.275+0000 7f163918a740 1 -- 172.21.15.69:0/2111557912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415eb50 msgr2=0x55603416f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.275+0000 7f163918a740 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415eb50 0x55603416f940 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f161c000c00 tx=0x7f161c0bbce0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.275+0000 7f163918a740 1 -- 172.21.15.69:0/2111557912 shutdown_connections 2024-03-19T01:19:37.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.275+0000 7f163918a740 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5560341509c0 0x556034152de0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.275+0000 7f163918a740 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556034167730 0x55603416ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.275+0000 7f163918a740 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55603415f4d0 0x55603416c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.275+0000 7f163918a740 1 --2- 172.21.15.69:0/2111557912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603415eb50 0x55603416f940 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.275+0000 7f163918a740 1 -- 172.21.15.69:0/2111557912 >> 172.21.15.69:0/2111557912 conn(0x556034158960 msgr2=0x556034159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:37.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.276+0000 7f1619600640 0 client.27506 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:19:37.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.282+0000 7f163918a740 1 -- 172.21.15.69:0/2111557912 shutdown_connections 2024-03-19T01:19:37.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.282+0000 7f163918a740 1 -- 172.21.15.69:0/2111557912 wait complete. 2024-03-19T01:19:37.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.282+0000 7f163918a740 20 client.27506 trim_cache size 0 max 16384 2024-03-19T01:19:37.284 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:19:37.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.312+0000 7f1636400640 1 -- 172.21.15.69:0/364772786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f16140a2b70 msgr2=0x7f16140a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.312+0000 7f1636400640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f16140a2b70 0x7f16140a5030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16301167e0 tx=0x7f1628009480 comp rx=0 tx=0).stop 2024-03-19T01:19:37.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.312+0000 7f1636400640 1 -- 172.21.15.69:0/364772786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109320 msgr2=0x7f16301a6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.312+0000 7f1636400640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109320 0x7f16301a6770 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f161c02c100 tx=0x7f161c0346e0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.312+0000 7f1636400640 1 -- 172.21.15.69:0/364772786 shutdown_connections 2024-03-19T01:19:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.312+0000 7f1636400640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16301a0920 0x7f16301a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.312+0000 7f1636400640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f16140a2b70 0x7f16140a5030 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.312+0000 7f1636400640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630114970 0x7f16301a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.312+0000 7f1636400640 1 --2- 172.21.15.69:0/364772786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1630109320 0x7f16301a6770 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.312+0000 7f1636400640 1 -- 172.21.15.69:0/364772786 >> 172.21.15.69:0/364772786 conn(0x7f163006c900 msgr2=0x7f16300705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.312+0000 7f1636400640 1 -- 172.21.15.69:0/364772786 shutdown_connections 2024-03-19T01:19:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.313+0000 7f1636400640 1 -- 172.21.15.69:0/364772786 wait complete. 2024-03-19T01:19:37.320 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/module/built-in.a 2024-03-19T01:19:37.368 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:19:37.509 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/time.o 2024-03-19T01:19:37.525 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/cmpxchg16b_emu.o 2024-03-19T01:19:37.655 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/copy_mc.o 2024-03-19T01:19:37.800 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/swapfile.o 2024-03-19T01:19:37.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.821+0000 7fe580a00640 1 -- 172.21.15.69:0/2486516836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe57c110040 msgr2=0x7fe57c10db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.821+0000 7fe580a00640 1 --2- 172.21.15.69:0/2486516836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe57c110040 0x7fe57c10db30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe568009860 tx=0x7fe56802bbb0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.822+0000 7fe580a00640 1 -- 172.21.15.69:0/2486516836 shutdown_connections 2024-03-19T01:19:37.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.822+0000 7fe580a00640 1 --2- 172.21.15.69:0/2486516836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe57c11b000 0x7fe57c11d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.822+0000 7fe580a00640 1 --2- 172.21.15.69:0/2486516836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe57c10e070 0x7fe57c10e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.822+0000 7fe580a00640 1 --2- 172.21.15.69:0/2486516836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe57c110040 0x7fe57c10db30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.822+0000 7fe580a00640 1 -- 172.21.15.69:0/2486516836 >> 172.21.15.69:0/2486516836 conn(0x7fe57c06c900 msgr2=0x7fe57c06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:37.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.826+0000 7fe580a00640 1 -- 172.21.15.69:0/2486516836 shutdown_connections 2024-03-19T01:19:37.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.826+0000 7fe580a00640 1 -- 172.21.15.69:0/2486516836 wait complete. 2024-03-19T01:19:37.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.827+0000 7fe580a00640 1 Processor -- start 2024-03-19T01:19:37.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.827+0000 7fe580a00640 1 -- start start 2024-03-19T01:19:37.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.828+0000 7fe580a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe57c10e070 0x7fe57c1186c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.828+0000 7fe580a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe57c110040 0x7fe57c111740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe580a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe57c11b000 0x7fe57c111c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe580a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1209c0 con 0x7fe57c110040 2024-03-19T01:19:37.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe580a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c120840 con 0x7fe57c11b000 2024-03-19T01:19:37.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe580a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c120b40 con 0x7fe57c10e070 2024-03-19T01:19:37.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe57a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe57c11b000 0x7fe57c111c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe579400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe57c110040 0x7fe57c111740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe57a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe57c11b000 0x7fe57c111c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60880/0 (socket says 172.21.15.69:60880) 2024-03-19T01:19:37.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe57a800640 1 -- 172.21.15.69:0/3741643728 learned_addr learned my addr 172.21.15.69:0/3741643728 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:37.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe57a800640 1 -- 172.21.15.69:0/3741643728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe57c10e070 msgr2=0x7fe57c1186c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe57a800640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe57c10e070 0x7fe57c1186c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe57a800640 1 -- 172.21.15.69:0/3741643728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe57c110040 msgr2=0x7fe57c111740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe57a800640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe57c110040 0x7fe57c111740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.829+0000 7fe57a800640 1 -- 172.21.15.69:0/3741643728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c1123b0 con 0x7fe57c11b000 2024-03-19T01:19:37.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.830+0000 7fe57a800640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe57c11b000 0x7fe57c111c80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00b3f0 tx=0x7fe56c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.830+0000 7fe572a00640 1 -- 172.21.15.69:0/3741643728 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c015020 con 0x7fe57c11b000 2024-03-19T01:19:37.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.830+0000 7fe572a00640 1 -- 172.21.15.69:0/3741643728 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c011d10 con 0x7fe57c11b000 2024-03-19T01:19:37.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.830+0000 7fe572a00640 1 -- 172.21.15.69:0/3741643728 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00f2e0 con 0x7fe57c11b000 2024-03-19T01:19:37.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.830+0000 7fe580a00640 1 -- 172.21.15.69:0/3741643728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57c1161e0 con 0x7fe57c11b000 2024-03-19T01:19:37.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.830+0000 7fe580a00640 1 -- 172.21.15.69:0/3741643728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57c116680 con 0x7fe57c11b000 2024-03-19T01:19:37.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.840+0000 7fe572a00640 1 -- 172.21.15.69:0/3741643728 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fe56c018020 con 0x7fe57c11b000 2024-03-19T01:19:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.846+0000 7fe572a00640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5600a2a70 0x7fe5600a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.846+0000 7fe5837bc740 1 Processor -- start 2024-03-19T01:19:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.846+0000 7fe572a00640 1 -- 172.21.15.69:0/3741643728 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fe56c0ce9a0 con 0x7fe57c11b000 2024-03-19T01:19:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.846+0000 7fe5837bc740 1 -- start start 2024-03-19T01:19:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.847+0000 7fe579e00640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5600a2a70 0x7fe5600a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.847+0000 7fe5837bc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2d8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.847+0000 7fe5837bc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d9340 0x55ab2c2e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.847+0000 7fe5837bc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2e4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.847+0000 7fe5837bc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ab2c23d380 con 0x55ab2c2e1f20 2024-03-19T01:19:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.847+0000 7fe5837bc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab2c1e5d20 con 0x55ab2c2d8a20 2024-03-19T01:19:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.847+0000 7fe5837bc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ab2c239420 con 0x55ab2c2d9340 2024-03-19T01:19:37.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.847+0000 7fe579400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d9340 0x55ab2c2e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.847+0000 7fe579400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d9340 0x55ab2c2e19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:43832/0 (socket says 172.21.15.69:43832) 2024-03-19T01:19:37.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.847+0000 7fe579400640 1 -- 172.21.15.69:0/479358904 learned_addr learned my addr 172.21.15.69:0/479358904 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:37.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.848+0000 7fe579e00640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5600a2a70 0x7fe5600a4f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe568037040 tx=0x7fe568005850 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.848+0000 7fe579e00640 1 --2- 172.21.15.69:0/479358904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2d8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.848+0000 7fe579e00640 1 -- 172.21.15.69:0/479358904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d9340 msgr2=0x55ab2c2e19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:37.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.848+0000 7fe579e00640 1 --2- 172.21.15.69:0/479358904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d9340 0x55ab2c2e19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.848+0000 7fe579e00640 1 -- 172.21.15.69:0/479358904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2e1f20 msgr2=0x55ab2c2e4310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:37.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.849+0000 7fe57a800640 1 --2- 172.21.15.69:0/479358904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.849+0000 7fe579e00640 1 --2- 172.21.15.69:0/479358904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2e4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.849+0000 7fe579e00640 1 -- 172.21.15.69:0/479358904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab2bf513f0 con 0x55ab2c2d8a20 2024-03-19T01:19:37.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.849+0000 7fe57a800640 1 --2- 172.21.15.69:0/479358904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:37.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.850+0000 7fe579e00640 1 --2- 172.21.15.69:0/479358904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2d8e00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe56802c0f0 tx=0x7fe56803b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddce8c80cf9e144c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.850+0000 7fe567000640 1 -- 172.21.15.69:0/479358904 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56804d020 con 0x55ab2c2d8a20 2024-03-19T01:19:37.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.850+0000 7fe567000640 1 -- 172.21.15.69:0/479358904 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe568032640 con 0x55ab2c2d8a20 2024-03-19T01:19:37.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.850+0000 7fe567000640 1 -- 172.21.15.69:0/479358904 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568045040 con 0x55ab2c2d8a20 2024-03-19T01:19:37.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.851+0000 7fe5837bc740 1 -- 172.21.15.69:0/479358904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2d8a20 msgr2=0x55ab2c2d8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.851+0000 7fe5837bc740 1 --2- 172.21.15.69:0/479358904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2d8e00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe56802c0f0 tx=0x7fe56803b000 comp rx=0 tx=0).stop 2024-03-19T01:19:37.852 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crypto/gf128mul.o 2024-03-19T01:19:37.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.852+0000 7fe5837bc740 1 -- 172.21.15.69:0/479358904 shutdown_connections 2024-03-19T01:19:37.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.852+0000 7fe5837bc740 1 --2- 172.21.15.69:0/479358904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.852+0000 7fe5837bc740 1 --2- 172.21.15.69:0/479358904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d9340 0x55ab2c2e19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.852+0000 7fe5837bc740 1 --2- 172.21.15.69:0/479358904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2d8e00 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.852+0000 7fe5837bc740 1 -- 172.21.15.69:0/479358904 >> 172.21.15.69:0/479358904 conn(0x55ab2c2d3150 msgr2=0x55ab2c2d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:37.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.853+0000 7fe5837bc740 1 -- 172.21.15.69:0/479358904 shutdown_connections 2024-03-19T01:19:37.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.853+0000 7fe5837bc740 1 -- 172.21.15.69:0/479358904 wait complete. 2024-03-19T01:19:37.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.854+0000 7fe5837bc740 1 -- start start 2024-03-19T01:19:37.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.854+0000 7fe5837bc740 1 Processor -- start 2024-03-19T01:19:37.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.854+0000 7fe5837bc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2defc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.857+0000 7fe5837bc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 0x55ab2c2df500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.857+0000 7fe5837bc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2dfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.857+0000 7fe5837bc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ab2c2e4cb0 con 0x55ab2c2d9340 2024-03-19T01:19:37.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.857+0000 7fe5837bc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab2c1f9160 con 0x55ab2c2e1f20 2024-03-19T01:19:37.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.857+0000 7fe5837bc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ab2c2e4e30 con 0x55ab2c2d8a20 2024-03-19T01:19:37.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.857+0000 7fe5837bc740 10 client.0 _ll_register_callbacks cb 0x55ab2c235170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:37.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.857+0000 7fe579400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 0x55ab2c2df500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.857+0000 7fe579400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 0x55ab2c2df500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59488/0 (socket says 172.21.15.69:59488) 2024-03-19T01:19:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.857+0000 7fe579400640 1 -- 172.21.15.69:0/2895608291 learned_addr learned my addr 172.21.15.69:0/2895608291 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.858+0000 7fe57a800640 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2dfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.858+0000 7fe57a800640 1 -- 172.21.15.69:0/2895608291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d8a20 msgr2=0x55ab2c2defc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:19:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.858+0000 7fe57a800640 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2defc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.858+0000 7fe57a800640 1 -- 172.21.15.69:0/2895608291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 msgr2=0x55ab2c2df500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.858+0000 7fe57a800640 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 0x55ab2c2df500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.858+0000 7fe57a800640 1 -- 172.21.15.69:0/2895608291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab2c01b7d0 con 0x55ab2c2e1f20 2024-03-19T01:19:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.860+0000 7fe579400640 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 0x55ab2c2df500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:19:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.860+0000 7fe57a800640 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2dfa40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0cbdb0 tx=0x7fe56c09a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1515b22a32a66ab server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.860+0000 7fe55b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:19:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.861+0000 7fe55b400640 1 -- 172.21.15.69:0/2895608291 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c09ee10 con 0x55ab2c2e1f20 2024-03-19T01:19:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.861+0000 7fe55b400640 1 -- 172.21.15.69:0/2895608291 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fe56c097330 con 0x55ab2c2e1f20 2024-03-19T01:19:37.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.864+0000 7fe5837bc740 1 -- 172.21.15.69:0/2895608291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe568032640 con 0x55ab2c2e1f20 2024-03-19T01:19:37.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.864+0000 7fe55b400640 1 -- 172.21.15.69:0/2895608291 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fe56c0a3460 con 0x55ab2c2e1f20 2024-03-19T01:19:37.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.864+0000 7fe5837bc740 10 client.27518 fetch_fsmap learned FSMap version 53 2024-03-19T01:19:37.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.864+0000 7fe5837bc740 1 -- 172.21.15.69:0/2895608291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55ab2c2f0f20 con 0x55ab2c2e1f20 2024-03-19T01:19:37.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.864+0000 7fe55b400640 1 -- 172.21.15.69:0/2895608291 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe568032640 con 0x55ab2c2e1f20 2024-03-19T01:19:37.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.865+0000 7fe55b400640 1 -- 172.21.15.69:0/2895608291 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 8390+0+0 (secure 0 0 0) 0x7fe56c09a110 con 0x55ab2c2e1f20 2024-03-19T01:19:37.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.865+0000 7fe5837bc740 10 client.27518 fetch_fsmap finished waiting for FSMap version 53 2024-03-19T01:19:37.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.865+0000 7fe5837bc740 10 client.27518 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:19:37.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.865+0000 7fe5837bc740 20 client.27518 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:37.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.865+0000 7fe5837bc740 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55ab2c2f2380 0x55ab2c2f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.865+0000 7fe5837bc740 4 client.27518 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:37.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.865+0000 7fe5837bc740 1 -- 172.21.15.69:0/2895608291 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab2c24a8c0 con 0x55ab2c2f2380 2024-03-19T01:19:37.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.865+0000 7fe579e00640 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55ab2c2f2380 0x55ab2c2f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.866+0000 7fe579e00640 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55ab2c2f2380 0x55ab2c2f47e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=65e257cb465bf26e server_cookie=8cbb1080e7872430 in_seq=0 out_seq=0 2024-03-19T01:19:37.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.866+0000 7fe55b400640 10 client.27518 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:19:37.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.867+0000 7fe55b400640 1 -- 172.21.15.69:0/2895608291 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ab2c24a8c0 con 0x55ab2c2f2380 2024-03-19T01:19:37.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.867+0000 7fe55b400640 10 client.27518 handle_command_reply: tid=0 2024-03-19T01:19:37.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.867+0000 7fe5837bc740 1 client.27518 shutdown 2024-03-19T01:19:37.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.871+0000 7fe5837bc740 1 -- 172.21.15.69:0/2895608291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 msgr2=0x55ab2c2dfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.871+0000 7fe5837bc740 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2dfa40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0cbdb0 tx=0x7fe56c09a8f0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.871+0000 7fe5837bc740 1 -- 172.21.15.69:0/2895608291 shutdown_connections 2024-03-19T01:19:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.871+0000 7fe5837bc740 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55ab2c2f2380 0x55ab2c2f47e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.872+0000 7fe5837bc740 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2dfa40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.872+0000 7fe5837bc740 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 0x55ab2c2df500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.872+0000 7fe5837bc740 1 --2- 172.21.15.69:0/2895608291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2defc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.872+0000 7fe55b400640 0 client.27518 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:19:37.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.872+0000 7fe5837bc740 1 -- 172.21.15.69:0/2895608291 >> 172.21.15.69:0/2895608291 conn(0x55ab2c2d3150 msgr2=0x55ab2c2d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:37.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.879+0000 7fe5837bc740 1 -- 172.21.15.69:0/2895608291 shutdown_connections 2024-03-19T01:19:37.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.881+0000 7fe5837bc740 1 -- 172.21.15.69:0/2895608291 wait complete. 2024-03-19T01:19:37.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.881+0000 7fe5837bc740 20 client.27518 trim_cache size 0 max 16384 2024-03-19T01:19:37.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.942+0000 7fe5837bc740 1 Processor -- start 2024-03-19T01:19:37.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.942+0000 7fe5837bc740 1 -- start start 2024-03-19T01:19:37.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.942+0000 7fe5837bc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2e16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.942+0000 7fe5837bc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d9340 0x55ab2c2eda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.942+0000 7fe5837bc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2df860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.942+0000 7fe5837bc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1b5650 con 0x55ab2c2d8a20 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.942+0000 7fe5837bc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1d92f0 con 0x55ab2c2e1f20 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.942+0000 7fe5837bc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1b5d20 con 0x55ab2c2d9340 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.943+0000 7fe57a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.943+0000 7fe57a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2df860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60902/0 (socket says 172.21.15.69:60902) 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.943+0000 7fe57a800640 1 -- 172.21.15.69:0/773106986 learned_addr learned my addr 172.21.15.69:0/773106986 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.945+0000 7fe579400640 1 --2- 172.21.15.69:0/773106986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d9340 0x55ab2c2eda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.945+0000 7fe57a800640 1 -- 172.21.15.69:0/773106986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d9340 msgr2=0x55ab2c2eda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.945+0000 7fe57a800640 1 --2- 172.21.15.69:0/773106986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d9340 0x55ab2c2eda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.945+0000 7fe57a800640 1 -- 172.21.15.69:0/773106986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d8a20 msgr2=0x55ab2c2e16f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.946+0000 7fe579e00640 1 --2- 172.21.15.69:0/773106986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2e16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.946+0000 7fe57a800640 1 --2- 172.21.15.69:0/773106986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2e16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.946+0000 7fe57a800640 1 -- 172.21.15.69:0/773106986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab2c2ce800 con 0x55ab2c2e1f20 2024-03-19T01:19:37.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.946+0000 7fe579e00640 1 --2- 172.21.15.69:0/773106986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2e16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:37.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.946+0000 7fe57a800640 1 --2- 172.21.15.69:0/773106986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2df860 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe56c097690 tx=0x7fe56c09b4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71f0db5fafe30539 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.950 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/physaddr.o 2024-03-19T01:19:37.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.955+0000 7fe566600640 1 -- 172.21.15.69:0/773106986 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c09ee10 con 0x55ab2c2e1f20 2024-03-19T01:19:37.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.956+0000 7fe566600640 1 -- 172.21.15.69:0/773106986 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c0b9250 con 0x55ab2c2e1f20 2024-03-19T01:19:37.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.960+0000 7fe5837bc740 1 -- 172.21.15.69:0/773106986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 msgr2=0x55ab2c2df860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.960+0000 7fe5837bc740 1 --2- 172.21.15.69:0/773106986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2df860 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe56c097690 tx=0x7fe56c09b4c0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.961+0000 7fe5837bc740 1 -- 172.21.15.69:0/773106986 shutdown_connections 2024-03-19T01:19:37.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.961+0000 7fe5837bc740 1 --2- 172.21.15.69:0/773106986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e1f20 0x55ab2c2df860 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.961+0000 7fe5837bc740 1 --2- 172.21.15.69:0/773106986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d9340 0x55ab2c2eda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.961+0000 7fe5837bc740 1 --2- 172.21.15.69:0/773106986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2e16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.961+0000 7fe5837bc740 1 -- 172.21.15.69:0/773106986 >> 172.21.15.69:0/773106986 conn(0x55ab2c2d3150 msgr2=0x55ab2c2dd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:37.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.962+0000 7fe5837bc740 1 -- 172.21.15.69:0/773106986 shutdown_connections 2024-03-19T01:19:37.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.962+0000 7fe5837bc740 1 -- 172.21.15.69:0/773106986 wait complete. 2024-03-19T01:19:37.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.963+0000 7fe5837bc740 1 -- start start 2024-03-19T01:19:37.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.963+0000 7fe5837bc740 1 Processor -- start 2024-03-19T01:19:37.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.964+0000 7fe5837bc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2ea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.964+0000 7fe5837bc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 0x55ab2c2e5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.964+0000 7fe579e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2ea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.964+0000 7fe579e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2ea480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:43864/0 (socket says 172.21.15.69:43864) 2024-03-19T01:19:37.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.965+0000 7fe579e00640 1 -- 172.21.15.69:0/1962115458 learned_addr learned my addr 172.21.15.69:0/1962115458 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:37.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.965+0000 7fe579400640 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 0x55ab2c2e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.965+0000 7fe5837bc740 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e56e0 0x55ab2c3566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.982+0000 7fe5837bc740 1 -- 172.21.15.69:0/1962115458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1b6b50 con 0x55ab2c2d9340 2024-03-19T01:19:37.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.982+0000 7fe5837bc740 1 -- 172.21.15.69:0/1962115458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1b6480 con 0x55ab2c2e56e0 2024-03-19T01:19:37.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.982+0000 7fe5837bc740 1 -- 172.21.15.69:0/1962115458 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c068610 con 0x55ab2c2d8a20 2024-03-19T01:19:37.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.982+0000 7fe5837bc740 10 client.0 _ll_register_callbacks cb 0x7fe56c0046e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:37.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.982+0000 7fe57a800640 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e56e0 0x55ab2c3566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.982+0000 7fe579400640 1 -- 172.21.15.69:0/1962115458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d8a20 msgr2=0x55ab2c2ea480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:37.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.982+0000 7fe579400640 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2ea480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.982+0000 7fe579400640 1 -- 172.21.15.69:0/1962115458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e56e0 msgr2=0x55ab2c3566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:37.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.982+0000 7fe579400640 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e56e0 0x55ab2c3566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.983+0000 7fe579400640 1 -- 172.21.15.69:0/1962115458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab2c2e5e20 con 0x55ab2c2d9340 2024-03-19T01:19:37.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.983+0000 7fe579400640 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 0x55ab2c2e5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe5740098d0 tx=0x7fe57400daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6048b62e0a3fde4e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:37.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.983+0000 7fe55b400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:19:37.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.983+0000 7fe5837bc740 1 -- 172.21.15.69:0/1962115458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe56c09a2b0 con 0x55ab2c2d9340 2024-03-19T01:19:37.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.984+0000 7fe55b400640 1 -- 172.21.15.69:0/1962115458 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574028050 con 0x55ab2c2d9340 2024-03-19T01:19:37.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.984+0000 7fe55b400640 1 -- 172.21.15.69:0/1962115458 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fe574004030 con 0x55ab2c2d9340 2024-03-19T01:19:37.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.984+0000 7fe55b400640 1 -- 172.21.15.69:0/1962115458 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fe57401e070 con 0x55ab2c2d9340 2024-03-19T01:19:37.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.987+0000 7fe5837bc740 10 client.17214 fetch_fsmap learned FSMap version 53 2024-03-19T01:19:37.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.987+0000 7fe5837bc740 1 -- 172.21.15.69:0/1962115458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=53}) -- 0x55ab2c357bf0 con 0x55ab2c2d9340 2024-03-19T01:19:37.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.989+0000 7fe55b400640 1 -- 172.21.15.69:0/1962115458 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 53) ==== 8390+0+0 (secure 0 0 0) 0x7fe57402b020 con 0x55ab2c2d9340 2024-03-19T01:19:37.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.989+0000 7fe5837bc740 10 client.17214 fetch_fsmap finished waiting for FSMap version 53 2024-03-19T01:19:37.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.989+0000 7fe5837bc740 10 client.17214 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:19:37.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.989+0000 7fe5837bc740 20 client.17214 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:37.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.989+0000 7fe5837bc740 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55ab2c358200 0x55ab2c35a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:37.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.989+0000 7fe5837bc740 4 client.17214 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:19:37.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.989+0000 7fe5837bc740 1 -- 172.21.15.69:0/1962115458 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab2c244c10 con 0x55ab2c358200 2024-03-19T01:19:37.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.989+0000 7fe579e00640 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55ab2c358200 0x55ab2c35a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:37.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.990+0000 7fe55b400640 10 client.17214 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:19:37.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.990+0000 7fe579e00640 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55ab2c358200 0x55ab2c35a5f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64a73945aaa6ce14 server_cookie=22b7c42f1283697f in_seq=0 out_seq=0 2024-03-19T01:19:37.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.990+0000 7fe55b400640 1 -- 172.21.15.69:0/1962115458 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab2c244c10 con 0x55ab2c358200 2024-03-19T01:19:37.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.990+0000 7fe55b400640 10 client.17214 handle_command_reply: tid=0 2024-03-19T01:19:37.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.990+0000 7fe5837bc740 1 client.17214 shutdown 2024-03-19T01:19:37.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.990+0000 7fe5837bc740 1 -- 172.21.15.69:0/1962115458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 msgr2=0x55ab2c2e5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:37.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.991+0000 7fe5837bc740 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 0x55ab2c2e5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe5740098d0 tx=0x7fe57400daf0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.991+0000 7fe5837bc740 1 -- 172.21.15.69:0/1962115458 shutdown_connections 2024-03-19T01:19:37.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.992+0000 7fe5837bc740 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55ab2c358200 0x55ab2c35a5f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.992+0000 7fe5837bc740 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab2c2e56e0 0x55ab2c3566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.992+0000 7fe5837bc740 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab2c2d9340 0x55ab2c2e5040 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.992+0000 7fe5837bc740 1 --2- 172.21.15.69:0/1962115458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ab2c2d8a20 0x55ab2c2ea480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:37.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.992+0000 7fe5837bc740 1 -- 172.21.15.69:0/1962115458 >> 172.21.15.69:0/1962115458 conn(0x55ab2c2d3150 msgr2=0x55ab2c2d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:37.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.992+0000 7fe55b400640 0 client.17214 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:19:37.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.994+0000 7fe5837bc740 1 -- 172.21.15.69:0/1962115458 shutdown_connections 2024-03-19T01:19:37.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.997+0000 7fe5837bc740 1 -- 172.21.15.69:0/1962115458 wait complete. 2024-03-19T01:19:37.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:37.997+0000 7fe5837bc740 20 client.17214 trim_cache size 0 max 16384 2024-03-19T01:19:38.002 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:19:38.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.004+0000 7fe580a00640 1 -- 172.21.15.69:0/3741643728 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5600a2a70 msgr2=0x7fe5600a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:38.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.004+0000 7fe580a00640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5600a2a70 0x7fe5600a4f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe568037040 tx=0x7fe568005850 comp rx=0 tx=0).stop 2024-03-19T01:19:38.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.004+0000 7fe580a00640 1 -- 172.21.15.69:0/3741643728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe57c11b000 msgr2=0x7fe57c111c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:38.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.004+0000 7fe580a00640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe57c11b000 0x7fe57c111c80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00b3f0 tx=0x7fe56c00b8b0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.004+0000 7fe580a00640 1 -- 172.21.15.69:0/3741643728 shutdown_connections 2024-03-19T01:19:38.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.004+0000 7fe580a00640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe5600a2a70 0x7fe5600a4f30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.004+0000 7fe580a00640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe57c11b000 0x7fe57c111c80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.004+0000 7fe580a00640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe57c110040 0x7fe57c111740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.004+0000 7fe580a00640 1 --2- 172.21.15.69:0/3741643728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe57c10e070 0x7fe57c1186c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.004+0000 7fe580a00640 1 -- 172.21.15.69:0/3741643728 >> 172.21.15.69:0/3741643728 conn(0x7fe57c06c900 msgr2=0x7fe57c11c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:38.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.005+0000 7fe580a00640 1 -- 172.21.15.69:0/3741643728 shutdown_connections 2024-03-19T01:19:38.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.005+0000 7fe580a00640 1 -- 172.21.15.69:0/3741643728 wait complete. 2024-03-19T01:19:38.058 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:19:38.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:37 smithi163 ceph-mon[67428]: pgmap v277: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 867 KiB/s wr, 136 op/s 2024-03-19T01:19:38.189 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:37 smithi069 ceph-mon[31727]: pgmap v277: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 867 KiB/s wr, 136 op/s 2024-03-19T01:19:38.225 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/copy_mc_64.o 2024-03-19T01:19:38.251 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/file.o 2024-03-19T01:19:38.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:37 smithi086 ceph-mon[66829]: pgmap v277: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 867 KiB/s wr, 136 op/s 2024-03-19T01:19:38.352 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/copy_page_64.o 2024-03-19T01:19:38.440 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/jbd2/built-in.a 2024-03-19T01:19:38.473 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/copy_user_64.o 2024-03-19T01:19:38.500 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/fsmap.o 2024-03-19T01:19:38.571 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/copy_user_uncached_64.o 2024-03-19T01:19:38.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.678+0000 7f807be00640 1 -- 172.21.15.69:0/3965074176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f807c10e380 msgr2=0x7f807c11d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:38.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.678+0000 7f807be00640 1 --2- 172.21.15.69:0/3965074176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f807c10e380 0x7f807c11d010 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8068009890 tx=0x7f806802bb70 comp rx=0 tx=0).stop 2024-03-19T01:19:38.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.678+0000 7f807be00640 1 -- 172.21.15.69:0/3965074176 shutdown_connections 2024-03-19T01:19:38.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.678+0000 7f807be00640 1 --2- 172.21.15.69:0/3965074176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f807c10e380 0x7f807c11d010 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.678+0000 7f807be00640 1 --2- 172.21.15.69:0/3965074176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f807c10d900 0x7f807c10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.678+0000 7f807be00640 1 --2- 172.21.15.69:0/3965074176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c10fd40 0x7f807c110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.678+0000 7f807be00640 1 -- 172.21.15.69:0/3965074176 >> 172.21.15.69:0/3965074176 conn(0x7f807c06c6d0 msgr2=0x7f807c06cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:38.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.688+0000 7f807be00640 1 -- 172.21.15.69:0/3965074176 shutdown_connections 2024-03-19T01:19:38.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.688+0000 7f807be00640 1 -- 172.21.15.69:0/3965074176 wait complete. 2024-03-19T01:19:38.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.693+0000 7f807be00640 1 Processor -- start 2024-03-19T01:19:38.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.693+0000 7f807be00640 1 -- start start 2024-03-19T01:19:38.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.694+0000 7f807be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f807c10d900 0x7f807c1ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.694+0000 7f807be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c10e380 0x7f807c1acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.694+0000 7f807be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f807c10fd40 0x7f807c1a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.694+0000 7f807be00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f807c120770 con 0x7f807c10d900 2024-03-19T01:19:38.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.694+0000 7f807be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1205f0 con 0x7f807c10e380 2024-03-19T01:19:38.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.694+0000 7f807be00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1208f0 con 0x7f807c10fd40 2024-03-19T01:19:38.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.694+0000 7f807a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c10e380 0x7f807c1acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.694+0000 7f807a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c10e380 0x7f807c1acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60932/0 (socket says 172.21.15.69:60932) 2024-03-19T01:19:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.694+0000 7f807a000640 1 -- 172.21.15.69:0/464762752 learned_addr learned my addr 172.21.15.69:0/464762752 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.694+0000 7f807aa00640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f807c10d900 0x7f807c1ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.695+0000 7f807b400640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f807c10fd40 0x7f807c1a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.695+0000 7f807a000640 1 -- 172.21.15.69:0/464762752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f807c10fd40 msgr2=0x7f807c1a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.695+0000 7f807a000640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f807c10fd40 0x7f807c1a69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.695+0000 7f807a000640 1 -- 172.21.15.69:0/464762752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f807c10d900 msgr2=0x7f807c1ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.697+0000 7f807a000640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f807c10d900 0x7f807c1ac840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.697+0000 7f807a000640 1 -- 172.21.15.69:0/464762752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c1a7170 con 0x7f807c10e380 2024-03-19T01:19:38.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.697+0000 7f807aa00640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f807c10d900 0x7f807c1ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:19:38.700 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/cpu.o 2024-03-19T01:19:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.697+0000 7f807a000640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c10e380 0x7f807c1acd80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8074009d50 tx=0x7f807400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.698+0000 7f8080a00640 1 -- 172.21.15.69:0/464762752 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074003b70 con 0x7f807c10e380 2024-03-19T01:19:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.698+0000 7f8080a00640 1 -- 172.21.15.69:0/464762752 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8074003d10 con 0x7f807c10e380 2024-03-19T01:19:38.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.698+0000 7f8080a00640 1 -- 172.21.15.69:0/464762752 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074010a60 con 0x7f807c10e380 2024-03-19T01:19:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.700+0000 7f807be00640 1 -- 172.21.15.69:0/464762752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c1a7460 con 0x7f807c10e380 2024-03-19T01:19:38.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.700+0000 7f807be00640 1 -- 172.21.15.69:0/464762752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c1b3700 con 0x7f807c10e380 2024-03-19T01:19:38.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.705+0000 7f8084b71740 1 Processor -- start 2024-03-19T01:19:38.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.705+0000 7f8084b71740 1 -- start start 2024-03-19T01:19:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.706+0000 7f8084b71740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a71cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.706+0000 7f8084b71740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a72200 0x564f88a8bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.706+0000 7f8084b71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a8c140 0x564f88a8e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.706+0000 7f8084b71740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564f889d6380 con 0x564f88a72200 2024-03-19T01:19:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.706+0000 7f8084b71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f8897ed20 con 0x564f88a8c140 2024-03-19T01:19:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.706+0000 7f8084b71740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564f889d2420 con 0x564f88a718e0 2024-03-19T01:19:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.706+0000 7f807b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a71cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.706+0000 7f807b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a71cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:43894/0 (socket says 172.21.15.69:43894) 2024-03-19T01:19:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.706+0000 7f807b400640 1 -- 172.21.15.69:0/2075067824 learned_addr learned my addr 172.21.15.69:0/2075067824 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.707+0000 7f807b400640 1 -- 172.21.15.69:0/2075067824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a8c140 msgr2=0x564f88a8e530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.707+0000 7f807b400640 1 --2- 172.21.15.69:0/2075067824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a8c140 0x564f88a8e530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.707+0000 7f807b400640 1 -- 172.21.15.69:0/2075067824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a72200 msgr2=0x564f88a8bc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.707+0000 7f807b400640 1 --2- 172.21.15.69:0/2075067824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a72200 0x564f88a8bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.707+0000 7f807b400640 1 -- 172.21.15.69:0/2075067824 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f886ea3f0 con 0x564f88a718e0 2024-03-19T01:19:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.707+0000 7f807b400640 1 --2- 172.21.15.69:0/2075067824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a71cc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8068009780 tx=0x7f8068032340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c868146930b1fa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.707+0000 7f8067e00640 1 -- 172.21.15.69:0/2075067824 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068039070 con 0x564f88a718e0 2024-03-19T01:19:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.708+0000 7f8067e00640 1 -- 172.21.15.69:0/2075067824 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f806803a040 con 0x564f88a718e0 2024-03-19T01:19:38.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.708+0000 7f8067e00640 1 -- 172.21.15.69:0/2075067824 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80680052a0 con 0x564f88a718e0 2024-03-19T01:19:38.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.710+0000 7f8084b71740 1 -- 172.21.15.69:0/2075067824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 msgr2=0x564f88a71cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:38.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.710+0000 7f8084b71740 1 --2- 172.21.15.69:0/2075067824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a71cc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8068009780 tx=0x7f8068032340 comp rx=0 tx=0).stop 2024-03-19T01:19:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.711+0000 7f8080a00640 1 -- 172.21.15.69:0/464762752 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f8074003ec0 con 0x7f807c10e380 2024-03-19T01:19:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.713+0000 7f8080a00640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80600a2b70 0x7f80600a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.713+0000 7f8080a00640 1 -- 172.21.15.69:0/464762752 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7f80740cffd0 con 0x7f807c10e380 2024-03-19T01:19:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.714+0000 7f8084b71740 1 -- 172.21.15.69:0/2075067824 shutdown_connections 2024-03-19T01:19:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.714+0000 7f8084b71740 1 --2- 172.21.15.69:0/2075067824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a8c140 0x564f88a8e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.714+0000 7f8084b71740 1 --2- 172.21.15.69:0/2075067824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a72200 0x564f88a8bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.714+0000 7f8084b71740 1 --2- 172.21.15.69:0/2075067824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a71cc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.714+0000 7f8084b71740 1 -- 172.21.15.69:0/2075067824 >> 172.21.15.69:0/2075067824 conn(0x564f88a6c360 msgr2=0x564f88a6c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.714+0000 7f807b400640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80600a2b70 0x7f80600a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.715+0000 7f807b400640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80600a2b70 0x7f80600a5030 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8068034690 tx=0x7f806802bb70 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.716+0000 7f8084b71740 1 -- 172.21.15.69:0/2075067824 shutdown_connections 2024-03-19T01:19:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.716+0000 7f8084b71740 1 -- 172.21.15.69:0/2075067824 wait complete. 2024-03-19T01:19:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.716+0000 7f8084b71740 1 -- start start 2024-03-19T01:19:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.717+0000 7f8084b71740 1 Processor -- start 2024-03-19T01:19:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.717+0000 7f8084b71740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a89130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.718 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/timer.o 2024-03-19T01:19:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.719+0000 7f8084b71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a89670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.719+0000 7f8084b71740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a89bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.719+0000 7f8084b71740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564f88a8ef20 con 0x564f88a8c140 2024-03-19T01:19:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.720+0000 7f8084b71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f88992160 con 0x564f88a72200 2024-03-19T01:19:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.720+0000 7f8084b71740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564f88a8f0a0 con 0x564f88a718e0 2024-03-19T01:19:38.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.720+0000 7f807b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a89bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.720+0000 7f807b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a89bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59536/0 (socket says 172.21.15.69:59536) 2024-03-19T01:19:38.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.720+0000 7f807b400640 1 -- 172.21.15.69:0/2473024972 learned_addr learned my addr 172.21.15.69:0/2473024972 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:38.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.720+0000 7f807a000640 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a89670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.720+0000 7f807aa00640 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a89130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.720+0000 7f8084b71740 10 client.0 _ll_register_callbacks cb 0x564f889ce170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:38.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.721+0000 7f807b400640 1 -- 172.21.15.69:0/2473024972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 msgr2=0x564f88a89130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:38.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.721+0000 7f807b400640 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a89130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.721+0000 7f807b400640 1 -- 172.21.15.69:0/2473024972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 msgr2=0x564f88a89670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:38.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.721+0000 7f807b400640 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a89670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.721+0000 7f807b400640 1 -- 172.21.15.69:0/2473024972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f887b47d0 con 0x564f88a8c140 2024-03-19T01:19:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.722+0000 7f807b400640 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a89bb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8068005d80 tx=0x7f80680346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7601ab3177179219 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.722+0000 7f8064c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:19:38.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.722+0000 7f807a000640 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a89670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:38.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.722+0000 7f8064c00640 1 -- 172.21.15.69:0/2473024972 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068055020 con 0x564f88a8c140 2024-03-19T01:19:38.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.722+0000 7f8064c00640 1 -- 172.21.15.69:0/2473024972 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7f80680043c0 con 0x564f88a8c140 2024-03-19T01:19:38.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.722+0000 7f807aa00640 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a89130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:38.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.725+0000 7f8084b71740 1 -- 172.21.15.69:0/2473024972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f806803a040 con 0x564f88a8c140 2024-03-19T01:19:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.729+0000 7f8064c00640 1 -- 172.21.15.69:0/2473024972 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f8068039070 con 0x564f88a8c140 2024-03-19T01:19:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.729+0000 7f8084b71740 10 client.17223 fetch_fsmap learned FSMap version 53 2024-03-19T01:19:38.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.729+0000 7f8084b71740 1 -- 172.21.15.69:0/2473024972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x564f88a9b210 con 0x564f88a8c140 2024-03-19T01:19:38.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.731+0000 7f8064c00640 1 -- 172.21.15.69:0/2473024972 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f806803a040 con 0x564f88a8c140 2024-03-19T01:19:38.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.732+0000 7f8064c00640 1 -- 172.21.15.69:0/2473024972 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 53) ==== 8390+0+0 (secure 0 0 0) 0x7f8068003950 con 0x564f88a8c140 2024-03-19T01:19:38.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.732+0000 7f8084b71740 10 client.17223 fetch_fsmap finished waiting for FSMap version 53 2024-03-19T01:19:38.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.732+0000 7f8084b71740 10 client.17223 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:19:38.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.732+0000 7f8084b71740 20 client.17223 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:38.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.732+0000 7f8084b71740 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564f88a9c560 0x564f88a9e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.732+0000 7f8084b71740 4 client.17223 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:38.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.732+0000 7f8084b71740 1 -- 172.21.15.69:0/2473024972 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f889e38c0 con 0x564f88a9c560 2024-03-19T01:19:38.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.732+0000 7f807aa00640 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564f88a9c560 0x564f88a9e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.733+0000 7f807aa00640 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564f88a9c560 0x564f88a9e950 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73ef828d930be491 server_cookie=951dbb27515fb46b in_seq=0 out_seq=0 2024-03-19T01:19:38.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.736+0000 7f8064c00640 10 client.17223 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:19:38.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.736+0000 7f8064c00640 1 -- 172.21.15.69:0/2473024972 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564f889e38c0 con 0x564f88a9c560 2024-03-19T01:19:38.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.736+0000 7f8064c00640 10 client.17223 handle_command_reply: tid=0 2024-03-19T01:19:38.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.736+0000 7f8084b71740 1 client.17223 shutdown 2024-03-19T01:19:38.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.736+0000 7f8084b71740 1 -- 172.21.15.69:0/2473024972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 msgr2=0x564f88a89bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:38.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.736+0000 7f8084b71740 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a89bb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8068005d80 tx=0x7f80680346f0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.737+0000 7f8084b71740 1 -- 172.21.15.69:0/2473024972 shutdown_connections 2024-03-19T01:19:38.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.737+0000 7f8084b71740 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564f88a9c560 0x564f88a9e950 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.737+0000 7f8084b71740 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a89bb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.737+0000 7f8084b71740 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a89670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.737+0000 7f8084b71740 1 --2- 172.21.15.69:0/2473024972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a89130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.737+0000 7f8084b71740 1 -- 172.21.15.69:0/2473024972 >> 172.21.15.69:0/2473024972 conn(0x564f88a6c360 msgr2=0x564f88a6e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:38.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.737+0000 7f8064c00640 0 client.17223 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:19:38.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.739+0000 7f8084b71740 1 -- 172.21.15.69:0/2473024972 shutdown_connections 2024-03-19T01:19:38.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.739+0000 7f8084b71740 1 -- 172.21.15.69:0/2473024972 wait complete. 2024-03-19T01:19:38.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.739+0000 7f8084b71740 20 client.17223 trim_cache size 0 max 16384 2024-03-19T01:19:38.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.839+0000 7f8084b71740 1 Processor -- start 2024-03-19T01:19:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.850+0000 7f8084b71740 1 -- start start 2024-03-19T01:19:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.850+0000 7f8084b71740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a8b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.854+0000 7f8084b71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a97d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.856+0000 7f807aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a8b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.858+0000 7f807a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a97d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.858+0000 7f807a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a97d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60946/0 (socket says 172.21.15.69:60946) 2024-03-19T01:19:38.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.859+0000 7f807a000640 1 -- 172.21.15.69:0/2691253105 learned_addr learned my addr 172.21.15.69:0/2691253105 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:38.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.859+0000 7f8084b71740 1 --2- 172.21.15.69:0/2691253105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.859+0000 7f8084b71740 1 -- 172.21.15.69:0/2691253105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1b50c0 con 0x564f88a8c140 2024-03-19T01:19:38.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.859+0000 7f8084b71740 1 -- 172.21.15.69:0/2691253105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1b6d50 con 0x564f88a72200 2024-03-19T01:19:38.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.859+0000 7f8084b71740 1 -- 172.21.15.69:0/2691253105 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1b5790 con 0x564f88a718e0 2024-03-19T01:19:38.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.859+0000 7f807a000640 1 -- 172.21.15.69:0/2691253105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 msgr2=0x564f88a8b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:38.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.859+0000 7f807a000640 1 --2- 172.21.15.69:0/2691253105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a8b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.859+0000 7f807a000640 1 -- 172.21.15.69:0/2691253105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 msgr2=0x564f88a899d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:38.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.859+0000 7f807a000640 1 --2- 172.21.15.69:0/2691253105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a899d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.859+0000 7f807a000640 1 -- 172.21.15.69:0/2691253105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f88a71710 con 0x564f88a72200 2024-03-19T01:19:38.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.860+0000 7f807a000640 1 --2- 172.21.15.69:0/2691253105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a97d60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f807409c170 tx=0x7f807409bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c34fe66e581f603 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:38.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.875+0000 7f8067400640 1 -- 172.21.15.69:0/2691253105 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807409c1e0 con 0x564f88a72200 2024-03-19T01:19:38.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.875+0000 7f8067400640 1 -- 172.21.15.69:0/2691253105 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80740a0330 con 0x564f88a72200 2024-03-19T01:19:38.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.875+0000 7f8067400640 1 -- 172.21.15.69:0/2691253105 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80740cd550 con 0x564f88a72200 2024-03-19T01:19:38.887 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crypto/blake2s.o 2024-03-19T01:19:38.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.889+0000 7f8084b71740 1 -- 172.21.15.69:0/2691253105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 msgr2=0x564f88a97d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:38.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.889+0000 7f8084b71740 1 --2- 172.21.15.69:0/2691253105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a97d60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f807409c170 tx=0x7f807409bc00 comp rx=0 tx=0).stop 2024-03-19T01:19:38.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.890+0000 7f8084b71740 1 -- 172.21.15.69:0/2691253105 shutdown_connections 2024-03-19T01:19:38.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.890+0000 7f8084b71740 1 --2- 172.21.15.69:0/2691253105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a899d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.890+0000 7f8084b71740 1 --2- 172.21.15.69:0/2691253105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a97d60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.890+0000 7f8084b71740 1 --2- 172.21.15.69:0/2691253105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a8b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.890+0000 7f8084b71740 1 -- 172.21.15.69:0/2691253105 >> 172.21.15.69:0/2691253105 conn(0x564f88a6c360 msgr2=0x564f88a9e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:38.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.896+0000 7f8084b71740 1 -- 172.21.15.69:0/2691253105 shutdown_connections 2024-03-19T01:19:38.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.896+0000 7f8084b71740 1 -- 172.21.15.69:0/2691253105 wait complete. 2024-03-19T01:19:38.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.897+0000 7f8084b71740 1 -- start start 2024-03-19T01:19:38.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.897+0000 7f8084b71740 1 Processor -- start 2024-03-19T01:19:38.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.898+0000 7f8084b71740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a942e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.898+0000 7f8084b71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.898+0000 7f8084b71740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a8fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.898+0000 7f8084b71740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1b65c0 con 0x564f88a8c140 2024-03-19T01:19:38.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.898+0000 7f8084b71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1b5ef0 con 0x564f88a72200 2024-03-19T01:19:38.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.898+0000 7f8084b71740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f807c068570 con 0x564f88a718e0 2024-03-19T01:19:38.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.898+0000 7f8084b71740 10 client.0 _ll_register_callbacks cb 0x7f80740093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:38.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.900+0000 7f807aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.900+0000 7f807a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.900+0000 7f807a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a8f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60962/0 (socket says 172.21.15.69:60962) 2024-03-19T01:19:38.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.900+0000 7f807a000640 1 -- 172.21.15.69:0/2292080106 learned_addr learned my addr 172.21.15.69:0/2292080106 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:38.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.901+0000 7f807b400640 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a8fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.901+0000 7f807a000640 1 -- 172.21.15.69:0/2292080106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 msgr2=0x564f88a942e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:38.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.901+0000 7f807a000640 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a942e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.901+0000 7f807a000640 1 -- 172.21.15.69:0/2292080106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 msgr2=0x564f88a8fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:38.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.901+0000 7f807a000640 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a8fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.901+0000 7f807a000640 1 -- 172.21.15.69:0/2292080106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f88a901e0 con 0x564f88a72200 2024-03-19T01:19:38.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.901+0000 7f807b400640 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a8fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:19:38.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.902+0000 7f807a000640 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a8f760 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f807409bee0 tx=0x7f807409bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=236fc82471bc780 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:38.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.902+0000 7f8064c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:19:38.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.902+0000 7f8064c00640 1 -- 172.21.15.69:0/2292080106 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80740bc050 con 0x564f88a72200 2024-03-19T01:19:38.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.902+0000 7f8064c00640 1 -- 172.21.15.69:0/2292080106 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7f80740041b0 con 0x564f88a72200 2024-03-19T01:19:38.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.908+0000 7f8084b71740 1 -- 172.21.15.69:0/2292080106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80740a0330 con 0x564f88a72200 2024-03-19T01:19:38.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.909+0000 7f8064c00640 1 -- 172.21.15.69:0/2292080106 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f80740043f0 con 0x564f88a72200 2024-03-19T01:19:38.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.909+0000 7f8084b71740 10 client.27533 fetch_fsmap learned FSMap version 54 2024-03-19T01:19:38.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.909+0000 7f8084b71740 1 -- 172.21.15.69:0/2292080106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564f88b02370 con 0x564f88a72200 2024-03-19T01:19:38.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.910+0000 7f8064c00640 1 -- 172.21.15.69:0/2292080106 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80740a0330 con 0x564f88a72200 2024-03-19T01:19:38.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.912+0000 7f8064c00640 1 -- 172.21.15.69:0/2292080106 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7f80740bace0 con 0x564f88a72200 2024-03-19T01:19:38.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.912+0000 7f8084b71740 10 client.27533 fetch_fsmap finished waiting for FSMap version 54 2024-03-19T01:19:38.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.912+0000 7f8084b71740 10 client.27533 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:19:38.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.913+0000 7f8084b71740 20 client.27533 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:38.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.913+0000 7f8084b71740 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564f88b02b70 0x564f88b04f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:38.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.913+0000 7f8084b71740 4 client.27533 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:19:38.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.913+0000 7f8084b71740 1 -- 172.21.15.69:0/2292080106 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f889ddc10 con 0x564f88b02b70 2024-03-19T01:19:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.913+0000 7f807aa00640 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564f88b02b70 0x564f88b04f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.913+0000 7f807aa00640 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564f88b02b70 0x564f88b04f90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=710f3a6f794faef2 server_cookie=7c5dd1aef56cf166 in_seq=0 out_seq=0 2024-03-19T01:19:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.913+0000 7f8064c00640 10 client.27533 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:19:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.914+0000 7f8064c00640 1 -- 172.21.15.69:0/2292080106 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f889ddc10 con 0x564f88b02b70 2024-03-19T01:19:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.914+0000 7f8064c00640 10 client.27533 handle_command_reply: tid=0 2024-03-19T01:19:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.914+0000 7f8084b71740 1 client.27533 shutdown 2024-03-19T01:19:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.914+0000 7f8084b71740 1 -- 172.21.15.69:0/2292080106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 msgr2=0x564f88a8f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:38.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.914+0000 7f8084b71740 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a8f760 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f807409bee0 tx=0x7f807409bc70 comp rx=0 tx=0).stop 2024-03-19T01:19:38.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.918+0000 7f8084b71740 1 -- 172.21.15.69:0/2292080106 shutdown_connections 2024-03-19T01:19:38.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.918+0000 7f8084b71740 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x564f88b02b70 0x564f88b04f90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.918+0000 7f8064c00640 0 client.27533 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:19:38.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.918+0000 7f8084b71740 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f88a8c140 0x564f88a8fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.918+0000 7f8084b71740 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f88a72200 0x564f88a8f760 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.918+0000 7f8084b71740 1 --2- 172.21.15.69:0/2292080106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564f88a718e0 0x564f88a942e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.918+0000 7f8084b71740 1 -- 172.21.15.69:0/2292080106 >> 172.21.15.69:0/2292080106 conn(0x564f88a6c360 msgr2=0x564f88a6ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:38.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.923+0000 7f8084b71740 1 -- 172.21.15.69:0/2292080106 shutdown_connections 2024-03-19T01:19:38.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.923+0000 7f8084b71740 1 -- 172.21.15.69:0/2292080106 wait complete. 2024-03-19T01:19:38.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.923+0000 7f8084b71740 20 client.27533 trim_cache size 0 max 16384 2024-03-19T01:19:38.924 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:19:38.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.928+0000 7f807be00640 1 -- 172.21.15.69:0/464762752 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80600a2b70 msgr2=0x7f80600a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:38.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.928+0000 7f807be00640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80600a2b70 0x7f80600a5030 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8068034690 tx=0x7f806802bb70 comp rx=0 tx=0).stop 2024-03-19T01:19:38.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.928+0000 7f807be00640 1 -- 172.21.15.69:0/464762752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c10e380 msgr2=0x7f807c1acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:38.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.928+0000 7f807be00640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c10e380 0x7f807c1acd80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8074009d50 tx=0x7f807400c6a0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.929+0000 7f807be00640 1 -- 172.21.15.69:0/464762752 shutdown_connections 2024-03-19T01:19:38.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.929+0000 7f807be00640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f807c10fd40 0x7f807c1a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.929+0000 7f807be00640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f80600a2b70 0x7f80600a5030 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.929+0000 7f807be00640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c10e380 0x7f807c1acd80 secure :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8074009d50 tx=0x7f807400c6a0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.929+0000 7f807be00640 1 --2- 172.21.15.69:0/464762752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f807c10d900 0x7f807c1ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:38.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.929+0000 7f807be00640 1 -- 172.21.15.69:0/464762752 >> 172.21.15.69:0/464762752 conn(0x7f807c06c6d0 msgr2=0x7f807c070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:38.930 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/tlb.o 2024-03-19T01:19:38.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.930+0000 7f807be00640 1 -- 172.21.15.69:0/464762752 shutdown_connections 2024-03-19T01:19:38.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:38.930+0000 7f807be00640 1 -- 172.21.15.69:0/464762752 wait complete. 2024-03-19T01:19:38.961 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:19:39.321 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/csum-copy_64.o 2024-03-19T01:19:39.498 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/csum-partial_64.o 2024-03-19T01:19:39.510 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/cpu_entry_area.o 2024-03-19T01:19:39.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.523+0000 7fdeda200640 1 -- 172.21.15.69:0/2383385826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fded4110040 msgr2=0x7fded410db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:39.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.523+0000 7fdeda200640 1 --2- 172.21.15.69:0/2383385826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fded4110040 0x7fded410db30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009900 tx=0x7fdec002bc40 comp rx=0 tx=0).stop 2024-03-19T01:19:39.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.525+0000 7fdeda200640 1 -- 172.21.15.69:0/2383385826 shutdown_connections 2024-03-19T01:19:39.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.525+0000 7fdeda200640 1 --2- 172.21.15.69:0/2383385826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fded411b000 0x7fded411d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.525+0000 7fdeda200640 1 --2- 172.21.15.69:0/2383385826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fded410e070 0x7fded410e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.525+0000 7fdeda200640 1 --2- 172.21.15.69:0/2383385826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fded4110040 0x7fded410db30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.525+0000 7fdeda200640 1 -- 172.21.15.69:0/2383385826 >> 172.21.15.69:0/2383385826 conn(0x7fded406c900 msgr2=0x7fded406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:39.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.529+0000 7fdeda200640 1 -- 172.21.15.69:0/2383385826 shutdown_connections 2024-03-19T01:19:39.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.529+0000 7fdeda200640 1 -- 172.21.15.69:0/2383385826 wait complete. 2024-03-19T01:19:39.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.530+0000 7fdeda200640 1 Processor -- start 2024-03-19T01:19:39.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.531+0000 7fdeda200640 1 -- start start 2024-03-19T01:19:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.531+0000 7fdeda200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fded410e070 0x7fded4118460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.531+0000 7fdeda200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fded4110040 0x7fded41189a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.531+0000 7fdeda200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fded411b000 0x7fded4111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.531+0000 7fded2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fded4110040 0x7fded41189a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.531+0000 7fded2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fded4110040 0x7fded41189a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60968/0 (socket says 172.21.15.69:60968) 2024-03-19T01:19:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.531+0000 7fded2a00640 1 -- 172.21.15.69:0/3608992623 learned_addr learned my addr 172.21.15.69:0/3608992623 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.532+0000 7fded3e00640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fded411b000 0x7fded4111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.532+0000 7fded3400640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fded410e070 0x7fded4118460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.532+0000 7fdeda200640 1 -- 172.21.15.69:0/3608992623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fded41209b0 con 0x7fded410e070 2024-03-19T01:19:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.532+0000 7fdeda200640 1 -- 172.21.15.69:0/3608992623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fded4120830 con 0x7fded4110040 2024-03-19T01:19:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.532+0000 7fdeda200640 1 -- 172.21.15.69:0/3608992623 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fded4120b30 con 0x7fded411b000 2024-03-19T01:19:39.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.537+0000 7fded3400640 1 -- 172.21.15.69:0/3608992623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fded411b000 msgr2=0x7fded4111600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:39.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.537+0000 7fded3400640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fded411b000 0x7fded4111600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.537+0000 7fded3400640 1 -- 172.21.15.69:0/3608992623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fded4110040 msgr2=0x7fded41189a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:39.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.537+0000 7fded3400640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fded4110040 0x7fded41189a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.537+0000 7fded3400640 1 -- 172.21.15.69:0/3608992623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded4111ec0 con 0x7fded410e070 2024-03-19T01:19:39.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.538+0000 7fded3400640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fded410e070 0x7fded4118460 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000c00 tx=0x7fdec0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:39.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.539+0000 7fdecbe00640 1 -- 172.21.15.69:0/3608992623 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0043020 con 0x7fded410e070 2024-03-19T01:19:39.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.539+0000 7fdecbe00640 1 -- 172.21.15.69:0/3608992623 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdec0005560 con 0x7fded410e070 2024-03-19T01:19:39.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.539+0000 7fdeda200640 1 -- 172.21.15.69:0/3608992623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded4112150 con 0x7fded410e070 2024-03-19T01:19:39.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.539+0000 7fded3e00640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fded411b000 0x7fded4111600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:19:39.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.539+0000 7fded2a00640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fded4110040 0x7fded41189a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:19:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.545+0000 7fdecbe00640 1 -- 172.21.15.69:0/3608992623 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec003e6b0 con 0x7fded410e070 2024-03-19T01:19:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.545+0000 7fdeda200640 1 -- 172.21.15.69:0/3608992623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded4116160 con 0x7fded410e070 2024-03-19T01:19:39.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.549+0000 7fdecbe00640 1 -- 172.21.15.69:0/3608992623 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fdec0046020 con 0x7fded410e070 2024-03-19T01:19:39.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.551+0000 7fdecbe00640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdeb80a29f0 0x7fdeb80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.551+0000 7fdecbe00640 1 -- 172.21.15.69:0/3608992623 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fdec00f5060 con 0x7fded410e070 2024-03-19T01:19:39.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.555+0000 7fded2a00640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdeb80a29f0 0x7fdeb80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.556+0000 7fded2a00640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdeb80a29f0 0x7fdeb80a4eb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fded4112ca0 tx=0x7fdec400b710 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:39.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.558+0000 7fdedcfaa740 1 Processor -- start 2024-03-19T01:19:39.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.558+0000 7fdedcfaa740 1 -- start start 2024-03-19T01:19:39.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.559+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44da20 0x55c2eb44de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.559+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44e340 0x55c2eb4569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.559+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb459310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.559+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c2eb3b2380 con 0x55c2eb456f20 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.559+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c2eb35ad20 con 0x55c2eb44e340 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.559+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c2eb3ae420 con 0x55c2eb44da20 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.559+0000 7fded3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb459310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.560+0000 7fded3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb459310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59574/0 (socket says 172.21.15.69:59574) 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.560+0000 7fded3e00640 1 -- 172.21.15.69:0/3380035217 learned_addr learned my addr 172.21.15.69:0/3380035217 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.560+0000 7fded2a00640 1 --2- 172.21.15.69:0/3380035217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44e340 0x55c2eb4569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.560+0000 7fded3e00640 1 -- 172.21.15.69:0/3380035217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44da20 msgr2=0x55c2eb44de00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.560+0000 7fded3e00640 1 --2- 172.21.15.69:0/3380035217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44da20 0x55c2eb44de00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.560+0000 7fded3e00640 1 -- 172.21.15.69:0/3380035217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44e340 msgr2=0x55c2eb4569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.560+0000 7fded3e00640 1 --2- 172.21.15.69:0/3380035217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44e340 0x55c2eb4569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.560+0000 7fded3e00640 1 -- 172.21.15.69:0/3380035217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2eb0c63f0 con 0x55c2eb456f20 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.561+0000 7fded3e00640 1 --2- 172.21.15.69:0/3380035217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb459310 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdecc005f40 tx=0x7fdecc010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bcca77842e570d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.561+0000 7fdebfe00640 1 -- 172.21.15.69:0/3380035217 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc011700 con 0x55c2eb456f20 2024-03-19T01:19:39.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.561+0000 7fdebfe00640 1 -- 172.21.15.69:0/3380035217 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdecc011d80 con 0x55c2eb456f20 2024-03-19T01:19:39.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.561+0000 7fdebfe00640 1 -- 172.21.15.69:0/3380035217 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc011700 con 0x55c2eb456f20 2024-03-19T01:19:39.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.562+0000 7fdedcfaa740 1 -- 172.21.15.69:0/3380035217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 msgr2=0x55c2eb459310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:39.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.562+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/3380035217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb459310 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdecc005f40 tx=0x7fdecc010860 comp rx=0 tx=0).stop 2024-03-19T01:19:39.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.563+0000 7fdedcfaa740 1 -- 172.21.15.69:0/3380035217 shutdown_connections 2024-03-19T01:19:39.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.563+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/3380035217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb459310 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.563+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/3380035217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44e340 0x55c2eb4569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.563+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/3380035217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44da20 0x55c2eb44de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.563+0000 7fdedcfaa740 1 -- 172.21.15.69:0/3380035217 >> 172.21.15.69:0/3380035217 conn(0x55c2eb448150 msgr2=0x55c2eb448530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:39.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.563+0000 7fdedcfaa740 1 -- 172.21.15.69:0/3380035217 shutdown_connections 2024-03-19T01:19:39.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.563+0000 7fdedcfaa740 1 -- 172.21.15.69:0/3380035217 wait complete. 2024-03-19T01:19:39.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.564+0000 7fdedcfaa740 1 -- start start 2024-03-19T01:19:39.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.564+0000 7fdedcfaa740 1 Processor -- start 2024-03-19T01:19:39.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.565+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44da20 0x55c2eb453f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.565+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44e340 0x55c2eb4544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.565+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb454a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.565+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c2eb459d10 con 0x55c2eb456f20 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.565+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c2eb36e160 con 0x55c2eb44e340 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.565+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c2eb459e90 con 0x55c2eb44da20 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.565+0000 7fdedcfaa740 10 client.0 _ll_register_callbacks cb 0x55c2eb3aa170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.566+0000 7fded3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb454a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.566+0000 7fded3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb454a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59590/0 (socket says 172.21.15.69:59590) 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.566+0000 7fded3e00640 1 -- 172.21.15.69:0/224584245 learned_addr learned my addr 172.21.15.69:0/224584245 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.566+0000 7fded3e00640 1 -- 172.21.15.69:0/224584245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44da20 msgr2=0x55c2eb453f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.566+0000 7fded3400640 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44da20 0x55c2eb453f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.567+0000 7fded2a00640 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44e340 0x55c2eb4544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.567+0000 7fded3e00640 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44da20 0x55c2eb453f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.567+0000 7fded3e00640 1 -- 172.21.15.69:0/224584245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44e340 msgr2=0x55c2eb4544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.567+0000 7fded3e00640 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44e340 0x55c2eb4544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.567+0000 7fded3e00640 1 -- 172.21.15.69:0/224584245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2eb1907d0 con 0x55c2eb456f20 2024-03-19T01:19:39.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.567+0000 7fded3400640 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44da20 0x55c2eb453f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:39.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.567+0000 7fded2a00640 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44e340 0x55c2eb4544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:39.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.568+0000 7fded3e00640 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb454a10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0099a0 tx=0x7fdecc010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d7feca01d253a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:39.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.568+0000 7fdebcc00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:19:39.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.569+0000 7fdebcc00640 1 -- 172.21.15.69:0/224584245 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc004e50 con 0x55c2eb456f20 2024-03-19T01:19:39.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.569+0000 7fdebcc00640 1 -- 172.21.15.69:0/224584245 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fdecc011500 con 0x55c2eb456f20 2024-03-19T01:19:39.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.569+0000 7fdedcfaa740 1 -- 172.21.15.69:0/224584245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdecc011d80 con 0x55c2eb456f20 2024-03-19T01:19:39.571 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crypto/blake2s-generic.o 2024-03-19T01:19:39.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.574+0000 7fdebcc00640 1 -- 172.21.15.69:0/224584245 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdecc011700 con 0x55c2eb456f20 2024-03-19T01:19:39.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.574+0000 7fdedcfaa740 10 client.17238 fetch_fsmap learned FSMap version 54 2024-03-19T01:19:39.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.574+0000 7fdedcfaa740 1 -- 172.21.15.69:0/224584245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c2eb465dd0 con 0x55c2eb456f20 2024-03-19T01:19:39.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.575+0000 7fdebcc00640 1 -- 172.21.15.69:0/224584245 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdecc011d80 con 0x55c2eb456f20 2024-03-19T01:19:39.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.575+0000 7fdebcc00640 1 -- 172.21.15.69:0/224584245 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7fdecc01b430 con 0x55c2eb456f20 2024-03-19T01:19:39.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.578+0000 7fdedcfaa740 10 client.17238 fetch_fsmap finished waiting for FSMap version 54 2024-03-19T01:19:39.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.578+0000 7fdedcfaa740 10 client.17238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:19:39.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.578+0000 7fdedcfaa740 20 client.17238 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:39.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.578+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c2eb467180 0x55c2eb469570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.578+0000 7fdedcfaa740 4 client.17238 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:39.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.578+0000 7fdedcfaa740 1 -- 172.21.15.69:0/224584245 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2eb3bf8c0 con 0x55c2eb467180 2024-03-19T01:19:39.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.578+0000 7fded3400640 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c2eb467180 0x55c2eb469570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.580+0000 7fdebcc00640 10 client.17238 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:19:39.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.580+0000 7fded3400640 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c2eb467180 0x55c2eb469570 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ae74e4963ecfd77 server_cookie=6321c21bee17dc60 in_seq=0 out_seq=0 2024-03-19T01:19:39.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.583+0000 7fdebcc00640 1 -- 172.21.15.69:0/224584245 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fdec00a39c0 con 0x55c2eb467180 2024-03-19T01:19:39.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.583+0000 7fdebcc00640 10 client.17238 handle_command_reply: tid=0 2024-03-19T01:19:39.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.583+0000 7fdedcfaa740 1 client.17238 shutdown 2024-03-19T01:19:39.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.583+0000 7fdedcfaa740 1 -- 172.21.15.69:0/224584245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 msgr2=0x55c2eb454a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:39.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.583+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb454a10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0099a0 tx=0x7fdecc010fd0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.584+0000 7fdedcfaa740 1 -- 172.21.15.69:0/224584245 shutdown_connections 2024-03-19T01:19:39.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.584+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb454a10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.584+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c2eb467180 0x55c2eb469570 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.584+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44e340 0x55c2eb4544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.584+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/224584245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44da20 0x55c2eb453f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.584+0000 7fdedcfaa740 1 -- 172.21.15.69:0/224584245 >> 172.21.15.69:0/224584245 conn(0x55c2eb448150 msgr2=0x55c2eb449ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:39.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.584+0000 7fdebcc00640 0 client.17238 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:19:39.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.584+0000 7fdedcfaa740 1 -- 172.21.15.69:0/224584245 shutdown_connections 2024-03-19T01:19:39.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.584+0000 7fdedcfaa740 1 -- 172.21.15.69:0/224584245 wait complete. 2024-03-19T01:19:39.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.584+0000 7fdedcfaa740 20 client.17238 trim_cache size 0 max 16384 2024-03-19T01:19:39.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.646+0000 7fdedcfaa740 1 Processor -- start 2024-03-19T01:19:39.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.646+0000 7fdedcfaa740 1 -- start start 2024-03-19T01:19:39.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.646+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb44da20 0x55c2eb4566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.647+0000 7fded3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb44da20 0x55c2eb4566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.647+0000 7fded3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb44da20 0x55c2eb4566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59594/0 (socket says 172.21.15.69:59594) 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.647+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44e340 0x55c2eb462ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.647+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb456f20 0x55c2eb46ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.647+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fded41c63b0 con 0x55c2eb44da20 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.647+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fded41c8010 con 0x55c2eb456f20 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.647+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fded41c6a80 con 0x55c2eb44e340 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.647+0000 7fded3400640 1 -- 172.21.15.69:0/530643336 learned_addr learned my addr 172.21.15.69:0/530643336 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.648+0000 7fded3400640 1 -- 172.21.15.69:0/530643336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44e340 msgr2=0x55c2eb462ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.648+0000 7fded3400640 1 --2- 172.21.15.69:0/530643336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44e340 0x55c2eb462ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.648+0000 7fded3400640 1 -- 172.21.15.69:0/530643336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb456f20 msgr2=0x55c2eb46ec00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.648+0000 7fded3400640 1 --2- 172.21.15.69:0/530643336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb456f20 0x55c2eb46ec00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.648+0000 7fded3400640 1 -- 172.21.15.69:0/530643336 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2eb44ed90 con 0x55c2eb44da20 2024-03-19T01:19:39.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.648+0000 7fded3400640 1 --2- 172.21.15.69:0/530643336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb44da20 0x55c2eb4566c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdec00be060 tx=0x7fdec00de040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae19b7017c18b30 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:39.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.649+0000 7fdebf400640 1 -- 172.21.15.69:0/530643336 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00d6020 con 0x55c2eb44da20 2024-03-19T01:19:39.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.649+0000 7fdebf400640 1 -- 172.21.15.69:0/530643336 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c2eb3bf8c0 con 0x55c2eb44da20 2024-03-19T01:19:39.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.649+0000 7fdebf400640 1 -- 172.21.15.69:0/530643336 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00dce60 con 0x55c2eb44da20 2024-03-19T01:19:39.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.650+0000 7fdedcfaa740 1 -- 172.21.15.69:0/530643336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb44da20 msgr2=0x55c2eb4566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:39.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.650+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/530643336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb44da20 0x55c2eb4566c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdec00be060 tx=0x7fdec00de040 comp rx=0 tx=0).stop 2024-03-19T01:19:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.660+0000 7fdedcfaa740 1 -- 172.21.15.69:0/530643336 shutdown_connections 2024-03-19T01:19:39.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.660+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/530643336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb456f20 0x55c2eb46ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.660+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/530643336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44e340 0x55c2eb462ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.660+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/530643336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb44da20 0x55c2eb4566c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.660+0000 7fdedcfaa740 1 -- 172.21.15.69:0/530643336 >> 172.21.15.69:0/530643336 conn(0x55c2eb448150 msgr2=0x55c2eb44f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:39.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.660+0000 7fdedcfaa740 1 -- 172.21.15.69:0/530643336 shutdown_connections 2024-03-19T01:19:39.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.660+0000 7fdedcfaa740 1 -- 172.21.15.69:0/530643336 wait complete. 2024-03-19T01:19:39.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.661+0000 7fdedcfaa740 1 -- start start 2024-03-19T01:19:39.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.661+0000 7fdedcfaa740 1 Processor -- start 2024-03-19T01:19:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.663+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44da20 0x55c2eb45ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.663+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44e340 0x55c2eb45b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.663+0000 7fdedcfaa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb45b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.663+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fded41c7880 con 0x55c2eb456f20 2024-03-19T01:19:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.663+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fded41c71b0 con 0x55c2eb44da20 2024-03-19T01:19:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.663+0000 7fdedcfaa740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fded41c4390 con 0x55c2eb44e340 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.663+0000 7fdedcfaa740 10 client.0 _ll_register_callbacks cb 0x7fdecc009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.664+0000 7fded3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44da20 0x55c2eb45ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.664+0000 7fded3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44da20 0x55c2eb45ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60988/0 (socket says 172.21.15.69:60988) 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.664+0000 7fded3400640 1 -- 172.21.15.69:0/2375896476 learned_addr learned my addr 172.21.15.69:0/2375896476 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.664+0000 7fded3e00640 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb45b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.664+0000 7fded3400640 1 -- 172.21.15.69:0/2375896476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44e340 msgr2=0x55c2eb45b050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.664+0000 7fded3400640 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44e340 0x55c2eb45b050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.664+0000 7fded3400640 1 -- 172.21.15.69:0/2375896476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 msgr2=0x55c2eb45b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.664+0000 7fded3400640 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb45b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.664+0000 7fded3400640 1 -- 172.21.15.69:0/2375896476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2eb4cc360 con 0x55c2eb44da20 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.664+0000 7fded3e00640 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb45b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.665+0000 7fded3400640 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44da20 0x55c2eb45ab10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdec002c4f0 tx=0x7fdec0003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b60732c04d0ff50d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.665+0000 7fdebcc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:19:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.665+0000 7fdebcc00640 1 -- 172.21.15.69:0/2375896476 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00d6030 con 0x55c2eb44da20 2024-03-19T01:19:39.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.665+0000 7fdebcc00640 1 -- 172.21.15.69:0/2375896476 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fdec00c2170 con 0x55c2eb44da20 2024-03-19T01:19:39.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.668+0000 7fdedcfaa740 1 -- 172.21.15.69:0/2375896476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c2eb3bf8c0 con 0x55c2eb44da20 2024-03-19T01:19:39.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.668+0000 7fdebcc00640 1 -- 172.21.15.69:0/2375896476 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdec00dce60 con 0x55c2eb44da20 2024-03-19T01:19:39.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.668+0000 7fdedcfaa740 10 client.27545 fetch_fsmap learned FSMap version 54 2024-03-19T01:19:39.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.668+0000 7fdedcfaa740 1 -- 172.21.15.69:0/2375896476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c2eb4cd640 con 0x55c2eb44da20 2024-03-19T01:19:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.668+0000 7fdebcc00640 1 -- 172.21.15.69:0/2375896476 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c2eb3bf8c0 con 0x55c2eb44da20 2024-03-19T01:19:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.668+0000 7fdebcc00640 1 -- 172.21.15.69:0/2375896476 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7fdec00ee9c0 con 0x55c2eb44da20 2024-03-19T01:19:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.668+0000 7fdedcfaa740 10 client.27545 fetch_fsmap finished waiting for FSMap version 54 2024-03-19T01:19:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.668+0000 7fdedcfaa740 10 client.27545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:19:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.668+0000 7fdedcfaa740 20 client.27545 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.669+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c2eb4cdb80 0x55c2eb4cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.669+0000 7fdedcfaa740 4 client.27545 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:19:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.669+0000 7fdedcfaa740 1 -- 172.21.15.69:0/2375896476 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2eb3b9c10 con 0x55c2eb4cdb80 2024-03-19T01:19:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.669+0000 7fded2a00640 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c2eb4cdb80 0x55c2eb4cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:39.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.679+0000 7fdebcc00640 10 client.27545 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:19:39.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.679+0000 7fded2a00640 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c2eb4cdb80 0x55c2eb4cffa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=322061258f12fc0e server_cookie=c463451f9114738a in_seq=0 out_seq=0 2024-03-19T01:19:39.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.679+0000 7fdebcc00640 1 -- 172.21.15.69:0/2375896476 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2eb3b9c10 con 0x55c2eb4cdb80 2024-03-19T01:19:39.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.679+0000 7fdebcc00640 10 client.27545 handle_command_reply: tid=0 2024-03-19T01:19:39.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.680+0000 7fdedcfaa740 1 client.27545 shutdown 2024-03-19T01:19:39.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.680+0000 7fdedcfaa740 1 -- 172.21.15.69:0/2375896476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44da20 msgr2=0x55c2eb45ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:39.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.680+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44da20 0x55c2eb45ab10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdec002c4f0 tx=0x7fdec0003c20 comp rx=0 tx=0).stop 2024-03-19T01:19:39.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.681+0000 7fdedcfaa740 1 -- 172.21.15.69:0/2375896476 shutdown_connections 2024-03-19T01:19:39.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.681+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2eb456f20 0x55c2eb45b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.681+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c2eb44e340 0x55c2eb45b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.681+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c2eb4cdb80 0x55c2eb4cffa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.681+0000 7fdebcc00640 0 client.27545 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:19:39.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.681+0000 7fdedcfaa740 1 --2- 172.21.15.69:0/2375896476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2eb44da20 0x55c2eb45ab10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.681+0000 7fdedcfaa740 1 -- 172.21.15.69:0/2375896476 >> 172.21.15.69:0/2375896476 conn(0x55c2eb448150 msgr2=0x55c2eb449580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:39.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.682+0000 7fdedcfaa740 1 -- 172.21.15.69:0/2375896476 shutdown_connections 2024-03-19T01:19:39.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.682+0000 7fdedcfaa740 1 -- 172.21.15.69:0/2375896476 wait complete. 2024-03-19T01:19:39.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.682+0000 7fdedcfaa740 20 client.27545 trim_cache size 0 max 16384 2024-03-19T01:19:39.688 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:19:39.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.692+0000 7fdeda200640 1 -- 172.21.15.69:0/3608992623 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdeb80a29f0 msgr2=0x7fdeb80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:39.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.692+0000 7fdeda200640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdeb80a29f0 0x7fdeb80a4eb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fded4112ca0 tx=0x7fdec400b710 comp rx=0 tx=0).stop 2024-03-19T01:19:39.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.692+0000 7fdeda200640 1 -- 172.21.15.69:0/3608992623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fded410e070 msgr2=0x7fded4118460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:39.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.692+0000 7fdeda200640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fded410e070 0x7fded4118460 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000c00 tx=0x7fdec0005d50 comp rx=0 tx=0).stop 2024-03-19T01:19:39.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.692+0000 7fdeda200640 1 -- 172.21.15.69:0/3608992623 shutdown_connections 2024-03-19T01:19:39.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.692+0000 7fdeda200640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fded411b000 0x7fded4111600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.692+0000 7fdeda200640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdeb80a29f0 0x7fdeb80a4eb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.692+0000 7fdeda200640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fded4110040 0x7fded41189a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.692+0000 7fdeda200640 1 --2- 172.21.15.69:0/3608992623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fded410e070 0x7fded4118460 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:39.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.692+0000 7fdeda200640 1 -- 172.21.15.69:0/3608992623 >> 172.21.15.69:0/3608992623 conn(0x7fded406c900 msgr2=0x7fded411bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:39.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.694+0000 7fdeda200640 1 -- 172.21.15.69:0/3608992623 shutdown_connections 2024-03-19T01:19:39.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:39.697+0000 7fdeda200640 1 -- 172.21.15.69:0/3608992623 wait complete. 2024-03-19T01:19:39.722 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/csum-wrappers_64.o 2024-03-19T01:19:39.749 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:19:40.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:39 smithi069 ceph-mon[31727]: pgmap v278: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 761 KiB/s wr, 173 op/s 2024-03-19T01:19:40.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:39 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:40.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:39 smithi086 ceph-mon[66829]: pgmap v278: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 761 KiB/s wr, 173 op/s 2024-03-19T01:19:40.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:39 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:40.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.277+0000 7f9395c00640 1 -- 172.21.15.69:0/2497716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93901088b0 msgr2=0x7f9390108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:40.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.277+0000 7f9395c00640 1 --2- 172.21.15.69:0/2497716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93901088b0 0x7f9390108d30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9380009900 tx=0x7f938002bc60 comp rx=0 tx=0).stop 2024-03-19T01:19:40.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.281+0000 7f9395c00640 1 -- 172.21.15.69:0/2497716 shutdown_connections 2024-03-19T01:19:40.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.281+0000 7f9395c00640 1 --2- 172.21.15.69:0/2497716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f939011b000 0x7f939011d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.281+0000 7f9395c00640 1 --2- 172.21.15.69:0/2497716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93901088b0 0x7f9390108d30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.281+0000 7f9395c00640 1 --2- 172.21.15.69:0/2497716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939010fe40 0x7f9390110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.281+0000 7f9395c00640 1 -- 172.21.15.69:0/2497716 >> 172.21.15.69:0/2497716 conn(0x7f939006c770 msgr2=0x7f939006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:40.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.281+0000 7f9395c00640 1 -- 172.21.15.69:0/2497716 shutdown_connections 2024-03-19T01:19:40.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.281+0000 7f9395c00640 1 -- 172.21.15.69:0/2497716 wait complete. 2024-03-19T01:19:40.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.283+0000 7f9395c00640 1 Processor -- start 2024-03-19T01:19:40.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.284+0000 7f9395c00640 1 -- start start 2024-03-19T01:19:40.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.284+0000 7f9395c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93901088b0 0x7f93901a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.284+0000 7f9395c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939010fe40 0x7f93901a7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.284+0000 7f9395c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f939011b000 0x7f93901ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.285+0000 7f9395c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9390120990 con 0x7f93901088b0 2024-03-19T01:19:40.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.285+0000 7f9395c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9390120810 con 0x7f939011b000 2024-03-19T01:19:40.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.285+0000 7f9395c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9390120b10 con 0x7f939010fe40 2024-03-19T01:19:40.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.285+0000 7f938f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939010fe40 0x7f93901a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.285+0000 7f938f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939010fe40 0x7f93901a7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44004/0 (socket says 172.21.15.69:44004) 2024-03-19T01:19:40.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.285+0000 7f938f400640 1 -- 172.21.15.69:0/3607906757 learned_addr learned my addr 172.21.15.69:0/3607906757 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:40.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.285+0000 7f938f400640 1 -- 172.21.15.69:0/3607906757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f939011b000 msgr2=0x7f93901ab6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:19:40.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.285+0000 7f938f400640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f939011b000 0x7f93901ab6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.285+0000 7f938f400640 1 -- 172.21.15.69:0/3607906757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93901088b0 msgr2=0x7f93901a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:40.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.285+0000 7f938f400640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93901088b0 0x7f93901a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.285+0000 7f938f400640 1 -- 172.21.15.69:0/3607906757 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93901abe20 con 0x7f939010fe40 2024-03-19T01:19:40.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.286+0000 7f938f400640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939010fe40 0x7f93901a7370 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9380000c00 tx=0x7f93800045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:40.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.286+0000 7f938cc00640 1 -- 172.21.15.69:0/3607906757 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380041070 con 0x7f939010fe40 2024-03-19T01:19:40.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.286+0000 7f938cc00640 1 -- 172.21.15.69:0/3607906757 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f938002cd40 con 0x7f939010fe40 2024-03-19T01:19:40.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.286+0000 7f938cc00640 1 -- 172.21.15.69:0/3607906757 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380034440 con 0x7f939010fe40 2024-03-19T01:19:40.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.287+0000 7f9395c00640 1 -- 172.21.15.69:0/3607906757 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93901ac0b0 con 0x7f939010fe40 2024-03-19T01:19:40.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.288+0000 7f9395c00640 1 -- 172.21.15.69:0/3607906757 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93901b39a0 con 0x7f939010fe40 2024-03-19T01:19:40.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.295+0000 7f9399e35740 1 Processor -- start 2024-03-19T01:19:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.296+0000 7f9399e35740 1 -- start start 2024-03-19T01:19:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.296+0000 7f9399e35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4947c10 0x564cb4947ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.300+0000 7f9399e35740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb4950bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.300+0000 7f9399e35740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb4953500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.300+0000 7f9399e35740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564cb48ac380 con 0x564cb4948530 2024-03-19T01:19:40.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.300+0000 7f9399e35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564cb4854d20 con 0x564cb4947c10 2024-03-19T01:19:40.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.300+0000 7f9399e35740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564cb48a8420 con 0x564cb4951110 2024-03-19T01:19:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.300+0000 7f938fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb4950bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.301+0000 7f938fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb4950bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59624/0 (socket says 172.21.15.69:59624) 2024-03-19T01:19:40.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.301+0000 7f938fe00640 1 -- 172.21.15.69:0/3006218936 learned_addr learned my addr 172.21.15.69:0/3006218936 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:40.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.301+0000 7f938fe00640 1 -- 172.21.15.69:0/3006218936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 msgr2=0x564cb4953500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:40.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.301+0000 7f938fe00640 1 --2- 172.21.15.69:0/3006218936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb4953500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.301+0000 7f938fe00640 1 -- 172.21.15.69:0/3006218936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4947c10 msgr2=0x564cb4947ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:19:40.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.301+0000 7f938fe00640 1 --2- 172.21.15.69:0/3006218936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4947c10 0x564cb4947ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.301+0000 7f938fe00640 1 -- 172.21.15.69:0/3006218936 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb45c03f0 con 0x564cb4948530 2024-03-19T01:19:40.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.302+0000 7f938fe00640 1 --2- 172.21.15.69:0/3006218936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb4950bd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9388009da0 tx=0x7f938800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd54e1532a104b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:40.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.303+0000 7f9385600640 1 -- 172.21.15.69:0/3006218936 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938800bd40 con 0x564cb4948530 2024-03-19T01:19:40.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.303+0000 7f9385600640 1 -- 172.21.15.69:0/3006218936 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9388017070 con 0x564cb4948530 2024-03-19T01:19:40.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.303+0000 7f9385600640 1 -- 172.21.15.69:0/3006218936 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388011790 con 0x564cb4948530 2024-03-19T01:19:40.305 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crypto/sha1.o 2024-03-19T01:19:40.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.304+0000 7f938cc00640 1 -- 172.21.15.69:0/3607906757 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9380044070 con 0x7f939010fe40 2024-03-19T01:19:40.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.304+0000 7f9399e35740 1 -- 172.21.15.69:0/3006218936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 msgr2=0x564cb4950bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:40.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.304+0000 7f9399e35740 1 --2- 172.21.15.69:0/3006218936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb4950bd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9388009da0 tx=0x7f938800b420 comp rx=0 tx=0).stop 2024-03-19T01:19:40.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.309+0000 7f938cc00640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2ab0 0x7f93740a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.310+0000 7f938cc00640 1 -- 172.21.15.69:0/3607906757 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7f93800f0b60 con 0x7f939010fe40 2024-03-19T01:19:40.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:39 smithi163 ceph-mon[67428]: pgmap v278: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 761 KiB/s wr, 173 op/s 2024-03-19T01:19:40.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:39 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:40.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.313+0000 7f9399e35740 1 -- 172.21.15.69:0/3006218936 shutdown_connections 2024-03-19T01:19:40.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.313+0000 7f9399e35740 1 --2- 172.21.15.69:0/3006218936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb4953500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.313+0000 7f9399e35740 1 --2- 172.21.15.69:0/3006218936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb4950bd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.313+0000 7f9399e35740 1 --2- 172.21.15.69:0/3006218936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4947c10 0x564cb4947ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.313+0000 7f9399e35740 1 -- 172.21.15.69:0/3006218936 >> 172.21.15.69:0/3006218936 conn(0x564cb4942340 msgr2=0x564cb4942720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:40.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.314+0000 7f9399e35740 1 -- 172.21.15.69:0/3006218936 shutdown_connections 2024-03-19T01:19:40.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.314+0000 7f9395200640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2ab0 0x7f93740a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.316+0000 7f9399e35740 1 -- 172.21.15.69:0/3006218936 wait complete. 2024-03-19T01:19:40.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.316+0000 7f9399e35740 1 -- start start 2024-03-19T01:19:40.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.317+0000 7f9399e35740 1 Processor -- start 2024-03-19T01:19:40.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.318+0000 7f9395200640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2ab0 0x7f93740a4f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f937c0095b0 tx=0x7f937c006c20 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:40.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.319+0000 7f9399e35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4947c10 0x564cb494e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.319+0000 7f9399e35740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb494e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.319+0000 7f9399e35740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb494ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.319+0000 7f9399e35740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564cb4953f00 con 0x564cb4948530 2024-03-19T01:19:40.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.319+0000 7f9399e35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564cb4868160 con 0x564cb4947c10 2024-03-19T01:19:40.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.319+0000 7f9399e35740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564cb4954080 con 0x564cb4951110 2024-03-19T01:19:40.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.319+0000 7f9399e35740 10 client.0 _ll_register_callbacks cb 0x564cb48a4170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:40.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.319+0000 7f9395200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb494ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.319+0000 7f9395200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb494ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44018/0 (socket says 172.21.15.69:44018) 2024-03-19T01:19:40.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.319+0000 7f9395200640 1 -- 172.21.15.69:0/935986593 learned_addr learned my addr 172.21.15.69:0/935986593 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:40.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.320+0000 7f9395200640 1 -- 172.21.15.69:0/935986593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4947c10 msgr2=0x564cb494e140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:40.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.320+0000 7f9395200640 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4947c10 0x564cb494e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.320+0000 7f938f400640 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb494e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.320+0000 7f9395200640 1 -- 172.21.15.69:0/935986593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 msgr2=0x564cb494e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:40.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.321+0000 7f9395200640 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb494e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.321+0000 7f9395200640 1 -- 172.21.15.69:0/935986593 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb468a7d0 con 0x564cb4951110 2024-03-19T01:19:40.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.321+0000 7f938f400640 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb494e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:40.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.322+0000 7f9395200640 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb494ebc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f937c013f80 tx=0x7f937c0047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70192b52c873ba39 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:40.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.322+0000 7f937a000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:19:40.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.325+0000 7f937a000640 1 -- 172.21.15.69:0/935986593 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c010770 con 0x564cb4951110 2024-03-19T01:19:40.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.325+0000 7f937a000640 1 -- 172.21.15.69:0/935986593 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7f937c010570 con 0x564cb4951110 2024-03-19T01:19:40.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.325+0000 7f9399e35740 1 -- 172.21.15.69:0/935986593 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9388017070 con 0x564cb4951110 2024-03-19T01:19:40.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.326+0000 7f937a000640 1 -- 172.21.15.69:0/935986593 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f937c010770 con 0x564cb4951110 2024-03-19T01:19:40.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.326+0000 7f9399e35740 10 client.27520 fetch_fsmap learned FSMap version 54 2024-03-19T01:19:40.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.326+0000 7f9399e35740 1 -- 172.21.15.69:0/935986593 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564cb4971270 con 0x564cb4951110 2024-03-19T01:19:40.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.327+0000 7f937a000640 1 -- 172.21.15.69:0/935986593 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9388017070 con 0x564cb4951110 2024-03-19T01:19:40.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.327+0000 7f937a000640 1 -- 172.21.15.69:0/935986593 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7f937c012070 con 0x564cb4951110 2024-03-19T01:19:40.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.328+0000 7f9399e35740 10 client.27520 fetch_fsmap finished waiting for FSMap version 54 2024-03-19T01:19:40.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.328+0000 7f9399e35740 10 client.27520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:40.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.328+0000 7f9399e35740 20 client.27520 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:40.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.328+0000 7f9399e35740 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564cb4972850 0x564cb4974cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.328+0000 7f9399e35740 4 client.27520 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:40.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.328+0000 7f9399e35740 1 -- 172.21.15.69:0/935986593 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cb48b98c0 con 0x564cb4972850 2024-03-19T01:19:40.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.330+0000 7f938fe00640 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564cb4972850 0x564cb4974cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.331+0000 7f938fe00640 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564cb4972850 0x564cb4974cf0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5355f0da11188918 server_cookie=f32cde2dc0ad8edf in_seq=0 out_seq=0 2024-03-19T01:19:40.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.331+0000 7f937a000640 10 client.27520 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:40.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.332+0000 7f937a000640 1 -- 172.21.15.69:0/935986593 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564cb48b98c0 con 0x564cb4972850 2024-03-19T01:19:40.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.332+0000 7f937a000640 10 client.27520 handle_command_reply: tid=0 2024-03-19T01:19:40.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.332+0000 7f9399e35740 1 client.27520 shutdown 2024-03-19T01:19:40.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.334+0000 7f9399e35740 1 -- 172.21.15.69:0/935986593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 msgr2=0x564cb494ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:40.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.334+0000 7f9399e35740 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb494ebc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f937c013f80 tx=0x7f937c0047d0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.335+0000 7f9399e35740 1 -- 172.21.15.69:0/935986593 shutdown_connections 2024-03-19T01:19:40.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.335+0000 7f9399e35740 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564cb4972850 0x564cb4974cf0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.335+0000 7f9399e35740 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb494ebc0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.335+0000 7f9399e35740 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb494e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.335+0000 7f9399e35740 1 --2- 172.21.15.69:0/935986593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4947c10 0x564cb494e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.335+0000 7f9399e35740 1 -- 172.21.15.69:0/935986593 >> 172.21.15.69:0/935986593 conn(0x564cb4942340 msgr2=0x564cb4944090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:40.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.335+0000 7f937a000640 0 client.27520 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:40.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.335+0000 7f9399e35740 1 -- 172.21.15.69:0/935986593 shutdown_connections 2024-03-19T01:19:40.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.337+0000 7f9399e35740 1 -- 172.21.15.69:0/935986593 wait complete. 2024-03-19T01:19:40.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.337+0000 7f9399e35740 20 client.27520 trim_cache size 0 max 16384 2024-03-19T01:19:40.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.389+0000 7f9399e35740 1 Processor -- start 2024-03-19T01:19:40.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.391+0000 7f9399e35740 1 -- start start 2024-03-19T01:19:40.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.392+0000 7f9399e35740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4947c10 0x564cb4950870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.392+0000 7f938fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4947c10 0x564cb4950870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.393+0000 7f9399e35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4948530 0x564cb495cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.393+0000 7f9399e35740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb497a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.393+0000 7f9399e35740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93901b52d0 con 0x564cb4947c10 2024-03-19T01:19:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.393+0000 7f9399e35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93901b6f30 con 0x564cb4948530 2024-03-19T01:19:40.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.393+0000 7f9399e35740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f93901b59a0 con 0x564cb4951110 2024-03-19T01:19:40.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.394+0000 7f938fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4947c10 0x564cb4950870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59648/0 (socket says 172.21.15.69:59648) 2024-03-19T01:19:40.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.394+0000 7f938fe00640 1 -- 172.21.15.69:0/2318779901 learned_addr learned my addr 172.21.15.69:0/2318779901 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:40.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.394+0000 7f938f400640 1 --2- 172.21.15.69:0/2318779901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4948530 0x564cb495cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.394+0000 7f938fe00640 1 -- 172.21.15.69:0/2318779901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 msgr2=0x564cb497a5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:19:40.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.394+0000 7f938fe00640 1 --2- 172.21.15.69:0/2318779901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb497a5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.394+0000 7f938fe00640 1 -- 172.21.15.69:0/2318779901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4948530 msgr2=0x564cb495cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:40.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.394+0000 7f938fe00640 1 --2- 172.21.15.69:0/2318779901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4948530 0x564cb495cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.394+0000 7f938fe00640 1 -- 172.21.15.69:0/2318779901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb4948360 con 0x564cb4947c10 2024-03-19T01:19:40.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.398+0000 7f938fe00640 1 --2- 172.21.15.69:0/2318779901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4947c10 0x564cb4950870 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9388000f80 tx=0x7f938800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1b925a07deb539 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:40.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.399+0000 7f9384c00640 1 -- 172.21.15.69:0/2318779901 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388024c30 con 0x564cb4947c10 2024-03-19T01:19:40.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.399+0000 7f9384c00640 1 -- 172.21.15.69:0/2318779901 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9388017040 con 0x564cb4947c10 2024-03-19T01:19:40.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.399+0000 7f9384c00640 1 -- 172.21.15.69:0/2318779901 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93880126b0 con 0x564cb4947c10 2024-03-19T01:19:40.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.400+0000 7f9399e35740 1 -- 172.21.15.69:0/2318779901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4947c10 msgr2=0x564cb4950870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:40.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.400+0000 7f9399e35740 1 --2- 172.21.15.69:0/2318779901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4947c10 0x564cb4950870 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9388000f80 tx=0x7f938800b420 comp rx=0 tx=0).stop 2024-03-19T01:19:40.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.402+0000 7f9399e35740 1 -- 172.21.15.69:0/2318779901 shutdown_connections 2024-03-19T01:19:40.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.402+0000 7f9399e35740 1 --2- 172.21.15.69:0/2318779901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4951110 0x564cb497a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.402+0000 7f9399e35740 1 --2- 172.21.15.69:0/2318779901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4948530 0x564cb495cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.402+0000 7f9399e35740 1 --2- 172.21.15.69:0/2318779901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4947c10 0x564cb4950870 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.402+0000 7f9399e35740 1 -- 172.21.15.69:0/2318779901 >> 172.21.15.69:0/2318779901 conn(0x564cb4942340 msgr2=0x564cb494ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:40.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.403+0000 7f9399e35740 1 -- 172.21.15.69:0/2318779901 shutdown_connections 2024-03-19T01:19:40.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.403+0000 7f9399e35740 1 -- 172.21.15.69:0/2318779901 wait complete. 2024-03-19T01:19:40.404 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/delay.o 2024-03-19T01:19:40.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.403+0000 7f9399e35740 1 -- start start 2024-03-19T01:19:40.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.404+0000 7f9399e35740 1 Processor -- start 2024-03-19T01:19:40.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.406+0000 7f9399e35740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4947c10 0x564cb4959520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.406+0000 7f9399e35740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb494a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.406+0000 7f9399e35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4951110 0x564cb494aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.408+0000 7f9399e35740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93901b67a0 con 0x564cb4948530 2024-03-19T01:19:40.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.408+0000 7f938f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb494a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.408+0000 7f938f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb494a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59654/0 (socket says 172.21.15.69:59654) 2024-03-19T01:19:40.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.408+0000 7f9395200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4951110 0x564cb494aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.408+0000 7f9395200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4951110 0x564cb494aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:32798/0 (socket says 172.21.15.69:32798) 2024-03-19T01:19:40.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.408+0000 7f9395200640 1 -- 172.21.15.69:0/3921369175 learned_addr learned my addr 172.21.15.69:0/3921369175 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:40.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.409+0000 7f938fe00640 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4947c10 0x564cb4959520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.409+0000 7f9399e35740 1 -- 172.21.15.69:0/3921369175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93901b60d0 con 0x564cb4951110 2024-03-19T01:19:40.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.409+0000 7f9399e35740 1 -- 172.21.15.69:0/3921369175 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f93901a8180 con 0x564cb4947c10 2024-03-19T01:19:40.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.411+0000 7f9399e35740 10 client.0 _ll_register_callbacks cb 0x7f937c0092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:40.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.413+0000 7f9395200640 1 -- 172.21.15.69:0/3921369175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4947c10 msgr2=0x564cb4959520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:40.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.413+0000 7f9395200640 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4947c10 0x564cb4959520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.413+0000 7f9395200640 1 -- 172.21.15.69:0/3921369175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 msgr2=0x564cb494a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:40.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.413+0000 7f9395200640 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb494a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.413+0000 7f9395200640 1 -- 172.21.15.69:0/3921369175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb494b520 con 0x564cb4951110 2024-03-19T01:19:40.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.413+0000 7f9399e35740 1 -- 172.21.15.69:0/3921369175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9388017040 con 0x564cb4951110 2024-03-19T01:19:40.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.413+0000 7f938f400640 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb494a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:19:40.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.413+0000 7f937a000640 10 client.27554 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:19:40.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.413+0000 7f9395200640 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4951110 0x564cb494aeb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f937c027c30 tx=0x7f937c010570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f91d2c076ffa6255 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:40.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.414+0000 7f937a000640 1 -- 172.21.15.69:0/3921369175 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c034020 con 0x564cb4951110 2024-03-19T01:19:40.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.418+0000 7f937a000640 1 -- 172.21.15.69:0/3921369175 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7f937c027660 con 0x564cb4951110 2024-03-19T01:19:40.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.418+0000 7f937a000640 1 -- 172.21.15.69:0/3921369175 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f937c034020 con 0x564cb4951110 2024-03-19T01:19:40.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.425+0000 7f9399e35740 10 client.27554 fetch_fsmap learned FSMap version 54 2024-03-19T01:19:40.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.425+0000 7f9399e35740 1 -- 172.21.15.69:0/3921369175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=54}) -- 0x564cb49d89f0 con 0x564cb4951110 2024-03-19T01:19:40.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.426+0000 7f937a000640 1 -- 172.21.15.69:0/3921369175 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7f937c02f050 con 0x564cb4951110 2024-03-19T01:19:40.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.426+0000 7f9399e35740 10 client.27554 fetch_fsmap finished waiting for FSMap version 54 2024-03-19T01:19:40.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.426+0000 7f9399e35740 10 client.27554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:40.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.426+0000 7f9399e35740 20 client.27554 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:40.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.426+0000 7f9399e35740 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564cb49d9000 0x564cb49db3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:40.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.426+0000 7f9399e35740 4 client.27554 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:19:40.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.426+0000 7f9399e35740 1 -- 172.21.15.69:0/3921369175 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564cb48b3c10 con 0x564cb49d9000 2024-03-19T01:19:40.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.427+0000 7f938fe00640 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564cb49d9000 0x564cb49db3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:40.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.427+0000 7f938fe00640 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564cb49d9000 0x564cb49db3f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e73ef1780e7bc5 server_cookie=e4e20cced5e91c1a in_seq=0 out_seq=0 2024-03-19T01:19:40.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.427+0000 7f937a000640 10 client.27554 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:40.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.429+0000 7f937a000640 1 -- 172.21.15.69:0/3921369175 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564cb48b3c10 con 0x564cb49d9000 2024-03-19T01:19:40.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.429+0000 7f937a000640 10 client.27554 handle_command_reply: tid=0 2024-03-19T01:19:40.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.429+0000 7f9399e35740 1 client.27554 shutdown 2024-03-19T01:19:40.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.430+0000 7f9399e35740 1 -- 172.21.15.69:0/3921369175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4951110 msgr2=0x564cb494aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:40.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.430+0000 7f9399e35740 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4951110 0x564cb494aeb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f937c027c30 tx=0x7f937c010570 comp rx=0 tx=0).stop 2024-03-19T01:19:40.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.432+0000 7f9399e35740 1 -- 172.21.15.69:0/3921369175 shutdown_connections 2024-03-19T01:19:40.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.432+0000 7f9399e35740 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564cb49d9000 0x564cb49db3f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.432+0000 7f9399e35740 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cb4951110 0x564cb494aeb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.432+0000 7f9399e35740 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564cb4948530 0x564cb494a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.432+0000 7f9399e35740 1 --2- 172.21.15.69:0/3921369175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cb4947c10 0x564cb4959520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.432+0000 7f9399e35740 1 -- 172.21.15.69:0/3921369175 >> 172.21.15.69:0/3921369175 conn(0x564cb4942340 msgr2=0x564cb4943770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:40.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.433+0000 7f937a000640 0 client.27554 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:40.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.433+0000 7f9399e35740 1 -- 172.21.15.69:0/3921369175 shutdown_connections 2024-03-19T01:19:40.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.433+0000 7f9399e35740 1 -- 172.21.15.69:0/3921369175 wait complete. 2024-03-19T01:19:40.434 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:19:40.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.433+0000 7f9399e35740 20 client.27554 trim_cache size 0 max 16384 2024-03-19T01:19:40.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.436+0000 7f9395c00640 1 -- 172.21.15.69:0/3607906757 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2ab0 msgr2=0x7f93740a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:40.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.436+0000 7f9395c00640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2ab0 0x7f93740a4f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f937c0095b0 tx=0x7f937c006c20 comp rx=0 tx=0).stop 2024-03-19T01:19:40.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.438+0000 7f9395c00640 1 -- 172.21.15.69:0/3607906757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939010fe40 msgr2=0x7f93901a7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:40.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.438+0000 7f9395c00640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939010fe40 0x7f93901a7370 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9380000c00 tx=0x7f93800045e0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.440+0000 7f9395c00640 1 -- 172.21.15.69:0/3607906757 shutdown_connections 2024-03-19T01:19:40.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.440+0000 7f9395c00640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f93740a2ab0 0x7f93740a4f70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.440+0000 7f9395c00640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f939011b000 0x7f93901ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.440+0000 7f9395c00640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939010fe40 0x7f93901a7370 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.440+0000 7f9395c00640 1 --2- 172.21.15.69:0/3607906757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93901088b0 0x7f93901a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:40.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.440+0000 7f9395c00640 1 -- 172.21.15.69:0/3607906757 >> 172.21.15.69:0/3607906757 conn(0x7f939006c770 msgr2=0x7f9390118ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:40.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.442+0000 7f9395c00640 1 -- 172.21.15.69:0/3607906757 shutdown_connections 2024-03-19T01:19:40.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:40.443+0000 7f9395c00640 1 -- 172.21.15.69:0/3607906757 wait complete. 2024-03-19T01:19:40.473 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/fsync.o 2024-03-19T01:19:40.632 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/maccess.o 2024-03-19T01:19:40.776 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/futex/core.o 2024-03-19T01:19:40.866 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/swap_slots.o 2024-03-19T01:19:40.900 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/hrtimer.o 2024-03-19T01:19:40.997 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/getuser.o 2024-03-19T01:19:41.115 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/pgprot.o 2024-03-19T01:19:41.125 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crypto/sha256.o 2024-03-19T01:19:41.128 INFO:tasks.workunit.client.0.smithi069.stdout: GEN arch/x86/lib/inat-tables.c 2024-03-19T01:19:41.227 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/insn-eval.o 2024-03-19T01:19:41.508 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:19:41.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:19:41.509 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:19:41.874 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/dmapool.o 2024-03-19T01:19:42.085 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/futex/syscalls.o 2024-03-19T01:19:42.133 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/crypto/built-in.a 2024-03-19T01:19:42.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.154+0000 7fc639a00640 1 -- 172.21.15.69:0/1596929775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc634109140 msgr2=0x7fc634114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.154+0000 7fc639a00640 1 --2- 172.21.15.69:0/1596929775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc634109140 0x7fc634114430 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc624009860 tx=0x7fc62402bb70 comp rx=0 tx=0).stop 2024-03-19T01:19:42.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.155+0000 7fc639a00640 1 -- 172.21.15.69:0/1596929775 shutdown_connections 2024-03-19T01:19:42.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.155+0000 7fc639a00640 1 --2- 172.21.15.69:0/1596929775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc634114970 0x7fc634116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.155+0000 7fc639a00640 1 --2- 172.21.15.69:0/1596929775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc634109140 0x7fc634114430 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.155+0000 7fc639a00640 1 --2- 172.21.15.69:0/1596929775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc634108820 0x7fc634108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.155+0000 7fc639a00640 1 -- 172.21.15.69:0/1596929775 >> 172.21.15.69:0/1596929775 conn(0x7fc63406c770 msgr2=0x7fc63406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:42.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.160+0000 7fc639a00640 1 -- 172.21.15.69:0/1596929775 shutdown_connections 2024-03-19T01:19:42.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.160+0000 7fc639a00640 1 -- 172.21.15.69:0/1596929775 wait complete. 2024-03-19T01:19:42.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.162+0000 7fc639a00640 1 Processor -- start 2024-03-19T01:19:42.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.165+0000 7fc639a00640 1 -- start start 2024-03-19T01:19:42.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.166+0000 7fc639a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc634108820 0x7fc63419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.166+0000 7fc639a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc634109140 0x7fc63419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.166+0000 7fc639a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc634114970 0x7fc6341a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.166+0000 7fc639a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc634118050 con 0x7fc634109140 2024-03-19T01:19:42.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.166+0000 7fc639a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc634117ed0 con 0x7fc634108820 2024-03-19T01:19:42.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.166+0000 7fc639a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341181d0 con 0x7fc634114970 2024-03-19T01:19:42.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.166+0000 7fc639000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc634114970 0x7fc6341a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.166+0000 7fc639000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc634114970 0x7fc6341a2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44056/0 (socket says 172.21.15.69:44056) 2024-03-19T01:19:42.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.166+0000 7fc639000640 1 -- 172.21.15.69:0/2382086901 learned_addr learned my addr 172.21.15.69:0/2382086901 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:42.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.166+0000 7fc633e00640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc634108820 0x7fc63419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.166+0000 7fc633400640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc634109140 0x7fc63419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.168+0000 7fc639000640 1 -- 172.21.15.69:0/2382086901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc634108820 msgr2=0x7fc63419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.168+0000 7fc639000640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc634108820 0x7fc63419e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.168+0000 7fc639000640 1 -- 172.21.15.69:0/2382086901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc634109140 msgr2=0x7fc63419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.168+0000 7fc639000640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc634109140 0x7fc63419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.168+0000 7fc639000640 1 -- 172.21.15.69:0/2382086901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6341a34b0 con 0x7fc634114970 2024-03-19T01:19:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.168+0000 7fc639000640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc634114970 0x7fc6341a2d80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc62000d6a0 tx=0x7fc62000db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.168+0000 7fc630c00640 1 -- 172.21.15.69:0/2382086901 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620017070 con 0x7fc634114970 2024-03-19T01:19:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.169+0000 7fc630c00640 1 -- 172.21.15.69:0/2382086901 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc620009820 con 0x7fc634114970 2024-03-19T01:19:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.169+0000 7fc630c00640 1 -- 172.21.15.69:0/2382086901 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620004d00 con 0x7fc634114970 2024-03-19T01:19:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.169+0000 7fc639a00640 1 -- 172.21.15.69:0/2382086901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6341a37a0 con 0x7fc634114970 2024-03-19T01:19:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.169+0000 7fc639a00640 1 -- 172.21.15.69:0/2382086901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6341ab030 con 0x7fc634114970 2024-03-19T01:19:42.182 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/hash.o 2024-03-19T01:19:42.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.181+0000 7fc63dda9740 1 Processor -- start 2024-03-19T01:19:42.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.182+0000 7fc630c00640 1 -- 172.21.15.69:0/2382086901 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fc6200099d0 con 0x7fc634114970 2024-03-19T01:19:42.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.183+0000 7fc63dda9740 1 -- start start 2024-03-19T01:19:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.184+0000 7fc63dda9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3b9e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.184+0000 7fc63dda9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9ede0 0x564ae3ba7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.184+0000 7fc63dda9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3ba79c0 0x564ae3ba9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.184+0000 7fc63dda9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564ae3af8fb0 con 0x564ae3b9e4c0 2024-03-19T01:19:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.184+0000 7fc63dda9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ae3aa1ab0 con 0x564ae3b9ede0 2024-03-19T01:19:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.184+0000 7fc63dda9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564ae3af5050 con 0x564ae3ba79c0 2024-03-19T01:19:42.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.185+0000 7fc633400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3b9e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.185+0000 7fc633400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3b9e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59682/0 (socket says 172.21.15.69:59682) 2024-03-19T01:19:42.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.185+0000 7fc633400640 1 -- 172.21.15.69:0/2347673448 learned_addr learned my addr 172.21.15.69:0/2347673448 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:42.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.186+0000 7fc633400640 1 --2- 172.21.15.69:0/2347673448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3ba79c0 0x564ae3ba9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.186+0000 7fc633400640 1 -- 172.21.15.69:0/2347673448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3ba79c0 msgr2=0x564ae3ba9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:42.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.186+0000 7fc633400640 1 --2- 172.21.15.69:0/2347673448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3ba79c0 0x564ae3ba9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.186+0000 7fc633400640 1 -- 172.21.15.69:0/2347673448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9ede0 msgr2=0x564ae3ba7480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:19:42.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.186+0000 7fc633400640 1 --2- 172.21.15.69:0/2347673448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9ede0 0x564ae3ba7480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.186+0000 7fc633400640 1 -- 172.21.15.69:0/2347673448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae3abfe00 con 0x564ae3b9e4c0 2024-03-19T01:19:42.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.186+0000 7fc633400640 1 --2- 172.21.15.69:0/2347673448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3b9e8a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc62402c0b0 tx=0x7fc624009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a68bad7b9e3edc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:42.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.188+0000 7fc630c00640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc6180a2a60 0x7fc6180a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.188+0000 7fc630c00640 1 -- 172.21.15.69:0/2382086901 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fc6200cec60 con 0x7fc634114970 2024-03-19T01:19:42.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.189+0000 7fc633e00640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc6180a2a60 0x7fc6180a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.189+0000 7fc629600640 1 -- 172.21.15.69:0/2347673448 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624048070 con 0x564ae3b9e4c0 2024-03-19T01:19:42.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.189+0000 7fc629600640 1 -- 172.21.15.69:0/2347673448 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc624034600 con 0x564ae3b9e4c0 2024-03-19T01:19:42.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.189+0000 7fc629600640 1 -- 172.21.15.69:0/2347673448 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624039070 con 0x564ae3b9e4c0 2024-03-19T01:19:42.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.190+0000 7fc633e00640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc6180a2a60 0x7fc6180a4f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x564ae3b9df90 tx=0x7fc62c009680 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:42.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.191+0000 7fc63dda9740 1 -- 172.21.15.69:0/2347673448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 msgr2=0x564ae3b9e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.191+0000 7fc63dda9740 1 --2- 172.21.15.69:0/2347673448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3b9e8a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc62402c0b0 tx=0x7fc624009560 comp rx=0 tx=0).stop 2024-03-19T01:19:42.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.191+0000 7fc63dda9740 1 -- 172.21.15.69:0/2347673448 shutdown_connections 2024-03-19T01:19:42.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.191+0000 7fc63dda9740 1 --2- 172.21.15.69:0/2347673448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3ba79c0 0x564ae3ba9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.191+0000 7fc63dda9740 1 --2- 172.21.15.69:0/2347673448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9ede0 0x564ae3ba7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.191+0000 7fc63dda9740 1 --2- 172.21.15.69:0/2347673448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3b9e8a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.191+0000 7fc63dda9740 1 -- 172.21.15.69:0/2347673448 >> 172.21.15.69:0/2347673448 conn(0x564ae3b98bf0 msgr2=0x564ae3b98fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:42.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.193+0000 7fc63dda9740 1 -- 172.21.15.69:0/2347673448 shutdown_connections 2024-03-19T01:19:42.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.195+0000 7fc63dda9740 1 -- 172.21.15.69:0/2347673448 wait complete. 2024-03-19T01:19:42.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.196+0000 7fc63dda9740 1 -- start start 2024-03-19T01:19:42.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.196+0000 7fc63dda9740 1 Processor -- start 2024-03-19T01:19:42.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.201+0000 7fc63dda9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3b95db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.201+0000 7fc63dda9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b9ede0 0x564ae3b962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.201+0000 7fc63dda9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3ba79c0 0x564ae3ba2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.201+0000 7fc63dda9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564ae3b90050 con 0x564ae3b9e4c0 2024-03-19T01:19:42.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.201+0000 7fc63dda9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ae3ab4f00 con 0x564ae3ba79c0 2024-03-19T01:19:42.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.201+0000 7fc63dda9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564ae3b901d0 con 0x564ae3b9ede0 2024-03-19T01:19:42.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.201+0000 7fc63dda9740 10 client.0 _ll_register_callbacks cb 0x564ae3ab48d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:42.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.202+0000 7fc633e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3b95db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.202+0000 7fc639000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3ba79c0 0x564ae3ba2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.202+0000 7fc639000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3ba79c0 0x564ae3ba2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:32828/0 (socket says 172.21.15.69:32828) 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.202+0000 7fc639000640 1 -- 172.21.15.69:0/1756531289 learned_addr learned my addr 172.21.15.69:0/1756531289 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.202+0000 7fc639000640 1 -- 172.21.15.69:0/1756531289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b9ede0 msgr2=0x564ae3b962f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.202+0000 7fc639000640 1 --2- 172.21.15.69:0/1756531289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b9ede0 0x564ae3b962f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.202+0000 7fc639000640 1 -- 172.21.15.69:0/1756531289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 msgr2=0x564ae3b95db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.202+0000 7fc639000640 1 --2- 172.21.15.69:0/1756531289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3b95db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.202+0000 7fc639000640 1 -- 172.21.15.69:0/1756531289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae38d57e0 con 0x564ae3ba79c0 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.203+0000 7fc639000640 1 --2- 172.21.15.69:0/1756531289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3ba79c0 0x564ae3ba2450 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc620097840 tx=0x7fc62009a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92925fc54554217b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.203+0000 7fc61e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.203+0000 7fc61e000640 1 -- 172.21.15.69:0/1756531289 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620020040 con 0x564ae3ba79c0 2024-03-19T01:19:42.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.203+0000 7fc61e000640 1 -- 172.21.15.69:0/1756531289 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fc62009a2c0 con 0x564ae3ba79c0 2024-03-19T01:19:42.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.203+0000 7fc63dda9740 1 -- 172.21.15.69:0/1756531289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc624034600 con 0x564ae3ba79c0 2024-03-19T01:19:42.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.204+0000 7fc61e000640 1 -- 172.21.15.69:0/1756531289 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc6200cc3e0 con 0x564ae3ba79c0 2024-03-19T01:19:42.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.204+0000 7fc63dda9740 10 client.27563 fetch_fsmap learned FSMap version 54 2024-03-19T01:19:42.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.204+0000 7fc63dda9740 1 -- 172.21.15.69:0/1756531289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564ae3935f70 con 0x564ae3ba79c0 2024-03-19T01:19:42.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.204+0000 7fc61e000640 1 -- 172.21.15.69:0/1756531289 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc624034600 con 0x564ae3ba79c0 2024-03-19T01:19:42.206 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/hugetlbpage.o 2024-03-19T01:19:42.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.206+0000 7fc61e000640 1 -- 172.21.15.69:0/1756531289 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7fc6200b1020 con 0x564ae3ba79c0 2024-03-19T01:19:42.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.206+0000 7fc63dda9740 10 client.27563 fetch_fsmap finished waiting for FSMap version 54 2024-03-19T01:19:42.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.206+0000 7fc63dda9740 10 client.27563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:42.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.206+0000 7fc63dda9740 20 client.27563 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:42.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.206+0000 7fc63dda9740 1 --2- 172.21.15.69:0/1756531289 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564ae3bc02f0 0x564ae3bc2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.206+0000 7fc63dda9740 4 client.27563 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:42.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.206+0000 7fc63dda9740 1 -- 172.21.15.69:0/1756531289 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ae3b064f0 con 0x564ae3bc02f0 2024-03-19T01:19:42.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.206+0000 7fc633e00640 1 --2- 172.21.15.69:0/1756531289 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564ae3bc02f0 0x564ae3bc2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.208+0000 7fc61e000640 10 client.27563 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:42.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.208+0000 7fc633e00640 1 --2- 172.21.15.69:0/1756531289 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564ae3bc02f0 0x564ae3bc2770 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b63fb3a3477032a server_cookie=3cb94994de3745a8 in_seq=0 out_seq=0 2024-03-19T01:19:42.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.209+0000 7fc61e000640 1 -- 172.21.15.69:0/1756531289 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564ae3b064f0 con 0x564ae3bc02f0 2024-03-19T01:19:42.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.209+0000 7fc61e000640 10 client.27563 handle_command_reply: tid=0 2024-03-19T01:19:42.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.209+0000 7fc63dda9740 1 client.27563 shutdown 2024-03-19T01:19:42.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.209+0000 7fc63dda9740 1 -- 172.21.15.69:0/1756531289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3ba79c0 msgr2=0x564ae3ba2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.209+0000 7fc63dda9740 1 --2- 172.21.15.69:0/1756531289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3ba79c0 0x564ae3ba2450 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc620097840 tx=0x7fc62009a8c0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.210+0000 7fc63dda9740 1 -- 172.21.15.69:0/1756531289 shutdown_connections 2024-03-19T01:19:42.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.210+0000 7fc63dda9740 1 --2- 172.21.15.69:0/1756531289 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564ae3bc02f0 0x564ae3bc2770 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.210+0000 7fc61e000640 0 client.27563 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:42.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.210+0000 7fc63dda9740 1 --2- 172.21.15.69:0/1756531289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3ba79c0 0x564ae3ba2450 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.210+0000 7fc63dda9740 1 --2- 172.21.15.69:0/1756531289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b9ede0 0x564ae3b962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.210+0000 7fc63dda9740 1 --2- 172.21.15.69:0/1756531289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3b95db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.210+0000 7fc63dda9740 1 -- 172.21.15.69:0/1756531289 >> 172.21.15.69:0/1756531289 conn(0x564ae3b98bf0 msgr2=0x564ae3b9a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:42.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.210+0000 7fc63dda9740 1 -- 172.21.15.69:0/1756531289 shutdown_connections 2024-03-19T01:19:42.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.210+0000 7fc63dda9740 1 -- 172.21.15.69:0/1756531289 wait complete. 2024-03-19T01:19:42.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.210+0000 7fc63dda9740 20 client.27563 trim_cache size 0 max 16384 2024-03-19T01:19:42.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:41 smithi086 ceph-mon[66829]: pgmap v279: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 234 KiB/s wr, 150 op/s 2024-03-19T01:19:42.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.260+0000 7fc63dda9740 1 Processor -- start 2024-03-19T01:19:42.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.263+0000 7fc63dda9740 1 -- start start 2024-03-19T01:19:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.263+0000 7fc63dda9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 0x564ae3ba21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.263+0000 7fc63dda9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9e4c0 0x564ae3bc8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.264+0000 7fc633400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9e4c0 0x564ae3bc8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.264+0000 7fc633400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9e4c0 0x564ae3bc8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:32842/0 (socket says 172.21.15.69:32842) 2024-03-19T01:19:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.264+0000 7fc633e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 0x564ae3ba21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.264+0000 7fc633400640 1 -- 172.21.15.69:0/2212830090 learned_addr learned my addr 172.21.15.69:0/2212830090 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:42.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.264+0000 7fc63dda9740 1 --2- 172.21.15.69:0/2212830090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9ede0 0x564ae3b92e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.265+0000 7fc639000640 1 --2- 172.21.15.69:0/2212830090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9ede0 0x564ae3b92e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.265+0000 7fc63dda9740 1 -- 172.21.15.69:0/2212830090 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b51e0 con 0x564ae3b9ede0 2024-03-19T01:19:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.265+0000 7fc63dda9740 1 -- 172.21.15.69:0/2212830090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b6e40 con 0x564ae3b9e4c0 2024-03-19T01:19:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.265+0000 7fc63dda9740 1 -- 172.21.15.69:0/2212830090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b58b0 con 0x564ae3b98bf0 2024-03-19T01:19:42.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.265+0000 7fc639000640 1 -- 172.21.15.69:0/2212830090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 msgr2=0x564ae3ba21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.265+0000 7fc639000640 1 --2- 172.21.15.69:0/2212830090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 0x564ae3ba21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.265+0000 7fc639000640 1 -- 172.21.15.69:0/2212830090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9e4c0 msgr2=0x564ae3bc8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.265+0000 7fc639000640 1 --2- 172.21.15.69:0/2212830090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9e4c0 0x564ae3bc8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.265+0000 7fc639000640 1 -- 172.21.15.69:0/2212830090 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae37cd170 con 0x564ae3b9ede0 2024-03-19T01:19:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.266+0000 7fc639000640 1 --2- 172.21.15.69:0/2212830090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9ede0 0x564ae3b92e40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc62009ae00 tx=0x7fc620009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b9dd5ce6019d47 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.266+0000 7fc628c00640 1 -- 172.21.15.69:0/2212830090 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62009efc0 con 0x564ae3b9ede0 2024-03-19T01:19:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.267+0000 7fc628c00640 1 -- 172.21.15.69:0/2212830090 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62007e020 con 0x564ae3b9ede0 2024-03-19T01:19:42.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.267+0000 7fc628c00640 1 -- 172.21.15.69:0/2212830090 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62009a2c0 con 0x564ae3b9ede0 2024-03-19T01:19:42.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.267+0000 7fc633e00640 1 --2- 172.21.15.69:0/2212830090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 0x564ae3ba21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:42.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.268+0000 7fc633400640 1 --2- 172.21.15.69:0/2212830090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9e4c0 0x564ae3bc8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:42.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.268+0000 7fc63dda9740 1 -- 172.21.15.69:0/2212830090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9ede0 msgr2=0x564ae3b92e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.268+0000 7fc63dda9740 1 --2- 172.21.15.69:0/2212830090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9ede0 0x564ae3b92e40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc62009ae00 tx=0x7fc620009f50 comp rx=0 tx=0).stop 2024-03-19T01:19:42.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.269+0000 7fc63dda9740 1 -- 172.21.15.69:0/2212830090 shutdown_connections 2024-03-19T01:19:42.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.269+0000 7fc63dda9740 1 --2- 172.21.15.69:0/2212830090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9ede0 0x564ae3b92e40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.269+0000 7fc63dda9740 1 --2- 172.21.15.69:0/2212830090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9e4c0 0x564ae3bc8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.269+0000 7fc63dda9740 1 --2- 172.21.15.69:0/2212830090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 0x564ae3ba21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.269+0000 7fc63dda9740 1 -- 172.21.15.69:0/2212830090 >> 172.21.15.69:0/2212830090 conn(0x564ae3b8b4f0 msgr2=0x564ae3b8f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:42.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.269+0000 7fc63dda9740 1 -- 172.21.15.69:0/2212830090 shutdown_connections 2024-03-19T01:19:42.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.269+0000 7fc63dda9740 1 -- 172.21.15.69:0/2212830090 wait complete. 2024-03-19T01:19:42.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.270+0000 7fc63dda9740 1 -- start start 2024-03-19T01:19:42.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.271+0000 7fc63dda9740 1 Processor -- start 2024-03-19T01:19:42.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.272+0000 7fc63dda9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 0x564ae3bd0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.272+0000 7fc63dda9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3bd0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.273+0000 7fc633400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3bd0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.274+0000 7fc633e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 0x564ae3bd0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.274+0000 7fc633e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 0x564ae3bd0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44086/0 (socket says 172.21.15.69:44086) 2024-03-19T01:19:42.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.274+0000 7fc63dda9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9ede0 0x564ae3bd0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.274+0000 7fc63dda9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b66b0 con 0x564ae3b9e4c0 2024-03-19T01:19:42.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.274+0000 7fc63dda9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b5fe0 con 0x564ae3b9ede0 2024-03-19T01:19:42.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.274+0000 7fc63dda9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b31c0 con 0x564ae3b98bf0 2024-03-19T01:19:42.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.274+0000 7fc63dda9740 10 client.0 _ll_register_callbacks cb 0x7fc62009aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:42.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.274+0000 7fc633400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3bd0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59710/0 (socket says 172.21.15.69:59710) 2024-03-19T01:19:42.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.274+0000 7fc633400640 1 -- 172.21.15.69:0/3829467029 learned_addr learned my addr 172.21.15.69:0/3829467029 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:42.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.275+0000 7fc639000640 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9ede0 0x564ae3bd0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.276+0000 7fc633e00640 1 -- 172.21.15.69:0/3829467029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9ede0 msgr2=0x564ae3bd0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:42.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.276+0000 7fc633e00640 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9ede0 0x564ae3bd0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.276+0000 7fc633e00640 1 -- 172.21.15.69:0/3829467029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 msgr2=0x564ae3bd0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:42.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.276+0000 7fc633e00640 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3bd0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.276+0000 7fc633e00640 1 -- 172.21.15.69:0/3829467029 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae3c26100 con 0x564ae3b98bf0 2024-03-19T01:19:42.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.276+0000 7fc633400640 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3bd0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:19:42.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.277+0000 7fc639000640 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9ede0 0x564ae3bd0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:19:42.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.277+0000 7fc633e00640 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 0x564ae3bd0400 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x564ae3bc2fc0 tx=0x7fc62c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2fc6b7247f51953 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:42.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.277+0000 7fc61e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:19:42.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.279+0000 7fc61e000640 1 -- 172.21.15.69:0/3829467029 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c027e40 con 0x564ae3b98bf0 2024-03-19T01:19:42.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.280+0000 7fc61e000640 1 -- 172.21.15.69:0/3829467029 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fc62c0271c0 con 0x564ae3b98bf0 2024-03-19T01:19:42.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.280+0000 7fc63dda9740 1 -- 172.21.15.69:0/3829467029 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc62007e020 con 0x564ae3b98bf0 2024-03-19T01:19:42.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:41 smithi069 ceph-mon[31727]: pgmap v279: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 234 KiB/s wr, 150 op/s 2024-03-19T01:19:42.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.281+0000 7fc61e000640 1 -- 172.21.15.69:0/3829467029 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc62c009cc0 con 0x564ae3b98bf0 2024-03-19T01:19:42.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.282+0000 7fc63dda9740 10 client.27535 fetch_fsmap learned FSMap version 54 2024-03-19T01:19:42.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.282+0000 7fc63dda9740 1 -- 172.21.15.69:0/3829467029 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564ae3c27420 con 0x564ae3b98bf0 2024-03-19T01:19:42.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.282+0000 7fc61e000640 1 -- 172.21.15.69:0/3829467029 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62007e020 con 0x564ae3b98bf0 2024-03-19T01:19:42.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.282+0000 7fc61e000640 1 -- 172.21.15.69:0/3829467029 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7fc62c00f040 con 0x564ae3b98bf0 2024-03-19T01:19:42.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.283+0000 7fc63dda9740 10 client.27535 fetch_fsmap finished waiting for FSMap version 54 2024-03-19T01:19:42.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.283+0000 7fc63dda9740 10 client.27535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:42.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.283+0000 7fc63dda9740 20 client.27535 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:42.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.283+0000 7fc63dda9740 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564ae3ba79c0 0x564ae3c29bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.283+0000 7fc63dda9740 4 client.27535 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:19:42.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.283+0000 7fc63dda9740 1 -- 172.21.15.69:0/3829467029 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564ae3b00840 con 0x564ae3ba79c0 2024-03-19T01:19:42.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.284+0000 7fc633400640 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564ae3ba79c0 0x564ae3c29bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.287+0000 7fc633400640 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564ae3ba79c0 0x564ae3c29bf0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0dd323df0a95498 server_cookie=7e9c423a0f10de87 in_seq=0 out_seq=0 2024-03-19T01:19:42.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.287+0000 7fc61e000640 10 client.27535 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:42.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.288+0000 7fc61e000640 1 -- 172.21.15.69:0/3829467029 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564ae3b00840 con 0x564ae3ba79c0 2024-03-19T01:19:42.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.288+0000 7fc61e000640 10 client.27535 handle_command_reply: tid=0 2024-03-19T01:19:42.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.289+0000 7fc63dda9740 1 client.27535 shutdown 2024-03-19T01:19:42.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.289+0000 7fc63dda9740 1 -- 172.21.15.69:0/3829467029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 msgr2=0x564ae3bd0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.289+0000 7fc63dda9740 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 0x564ae3bd0400 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x564ae3bc2fc0 tx=0x7fc62c004ab0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.290+0000 7fc63dda9740 1 -- 172.21.15.69:0/3829467029 shutdown_connections 2024-03-19T01:19:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.290+0000 7fc63dda9740 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564ae3ba79c0 0x564ae3c29bf0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.290+0000 7fc63dda9740 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ae3b9ede0 0x564ae3bd0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.290+0000 7fc63dda9740 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ae3b9e4c0 0x564ae3bd0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.290+0000 7fc63dda9740 1 --2- 172.21.15.69:0/3829467029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ae3b98bf0 0x564ae3bd0400 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.290+0000 7fc63dda9740 1 -- 172.21.15.69:0/3829467029 >> 172.21.15.69:0/3829467029 conn(0x564ae3b8b4f0 msgr2=0x564ae3b8eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.290+0000 7fc61e000640 0 client.27535 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.293+0000 7fc63dda9740 1 -- 172.21.15.69:0/3829467029 shutdown_connections 2024-03-19T01:19:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.294+0000 7fc63dda9740 1 -- 172.21.15.69:0/3829467029 wait complete. 2024-03-19T01:19:42.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.294+0000 7fc63dda9740 20 client.27535 trim_cache size 0 max 16384 2024-03-19T01:19:42.296 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:19:42.296 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:19:42.296 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "8833b1af-6426-4537-8ef8-fa88fda3c48d", 2024-03-19T01:19:42.296 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:19:42.296 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:19:42.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.299+0000 7fc639a00640 1 -- 172.21.15.69:0/2382086901 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc6180a2a60 msgr2=0x7fc6180a4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.299+0000 7fc639a00640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc6180a2a60 0x7fc6180a4f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x564ae3b9df90 tx=0x7fc62c009680 comp rx=0 tx=0).stop 2024-03-19T01:19:42.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.299+0000 7fc639a00640 1 -- 172.21.15.69:0/2382086901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc634114970 msgr2=0x7fc6341a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.299+0000 7fc639a00640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc634114970 0x7fc6341a2d80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc62000d6a0 tx=0x7fc62000db60 comp rx=0 tx=0).stop 2024-03-19T01:19:42.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.301+0000 7fc639a00640 1 -- 172.21.15.69:0/2382086901 shutdown_connections 2024-03-19T01:19:42.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.301+0000 7fc639a00640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc634114970 0x7fc6341a2d80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.301+0000 7fc639a00640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc634109140 0x7fc63419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.301+0000 7fc639a00640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc6180a2a60 0x7fc6180a4f20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.301+0000 7fc639a00640 1 --2- 172.21.15.69:0/2382086901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc634108820 0x7fc63419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.301+0000 7fc639a00640 1 -- 172.21.15.69:0/2382086901 >> 172.21.15.69:0/2382086901 conn(0x7fc63406c770 msgr2=0x7fc634070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:42.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.301+0000 7fc639a00640 1 -- 172.21.15.69:0/2382086901 shutdown_connections 2024-03-19T01:19:42.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.302+0000 7fc639a00640 1 -- 172.21.15.69:0/2382086901 wait complete. 2024-03-19T01:19:42.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:41 smithi163 ceph-mon[67428]: pgmap v279: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 234 KiB/s wr, 150 op/s 2024-03-19T01:19:42.321 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zlib_inflate/inffast.o 2024-03-19T01:19:42.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:19:42.365 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:19:42.529 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/insn.o 2024-03-19T01:19:42.860 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zlib_inflate/inflate.o 2024-03-19T01:19:42.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.910+0000 7fb0dcc00640 1 -- 172.21.15.69:0/815403518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d8109440 msgr2=0x7fb0d810cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.910+0000 7fb0dcc00640 1 --2- 172.21.15.69:0/815403518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d8109440 0x7fb0d810cc20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009900 tx=0x7fb0d002cf20 comp rx=0 tx=0).stop 2024-03-19T01:19:42.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.912+0000 7fb0dcc00640 1 -- 172.21.15.69:0/815403518 shutdown_connections 2024-03-19T01:19:42.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.912+0000 7fb0dcc00640 1 --2- 172.21.15.69:0/815403518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0d810d300 0x7fb0d8114f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.912+0000 7fb0dcc00640 1 --2- 172.21.15.69:0/815403518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d8109440 0x7fb0d810cc20 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.912+0000 7fb0dcc00640 1 --2- 172.21.15.69:0/815403518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0d8108a90 0x7fb0d8108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.912+0000 7fb0dcc00640 1 -- 172.21.15.69:0/815403518 >> 172.21.15.69:0/815403518 conn(0x7fb0d806cb50 msgr2=0x7fb0d806cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:42.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.915+0000 7fb0dcc00640 1 -- 172.21.15.69:0/815403518 shutdown_connections 2024-03-19T01:19:42.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.915+0000 7fb0dcc00640 1 -- 172.21.15.69:0/815403518 wait complete. 2024-03-19T01:19:42.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.919+0000 7fb0dcc00640 1 Processor -- start 2024-03-19T01:19:42.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.919+0000 7fb0dcc00640 1 -- start start 2024-03-19T01:19:42.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.919+0000 7fb0dcc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0d8108a90 0x7fb0d811e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.919+0000 7fb0dcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d810d300 0x7fb0d811ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.923+0000 7fb0dcc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0d8124230 0x7fb0d81246e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.923+0000 7fb0dcc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8117cd0 con 0x7fb0d8124230 2024-03-19T01:19:42.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.923+0000 7fb0dcc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8117b50 con 0x7fb0d810d300 2024-03-19T01:19:42.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.923+0000 7fb0dcc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8117e50 con 0x7fb0d8108a90 2024-03-19T01:19:42.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.923+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d810d300 0x7fb0d811ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.923+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d810d300 0x7fb0d811ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:32878/0 (socket says 172.21.15.69:32878) 2024-03-19T01:19:42.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.923+0000 7fb0d5400640 1 -- 172.21.15.69:0/2441486361 learned_addr learned my addr 172.21.15.69:0/2441486361 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:42.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.923+0000 7fb0d6800640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0d8124230 0x7fb0d81246e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.923+0000 7fb0d5400640 1 -- 172.21.15.69:0/2441486361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0d8108a90 msgr2=0x7fb0d811e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.924+0000 7fb0d5e00640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0d8108a90 0x7fb0d811e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.925+0000 7fb0d5400640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0d8108a90 0x7fb0d811e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.925+0000 7fb0d5400640 1 -- 172.21.15.69:0/2441486361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0d8124230 msgr2=0x7fb0d81246e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.925+0000 7fb0d5400640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0d8124230 0x7fb0d81246e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.925+0000 7fb0d5400640 1 -- 172.21.15.69:0/2441486361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d8124fa0 con 0x7fb0d810d300 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.925+0000 7fb0d5e00640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0d8108a90 0x7fb0d811e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.925+0000 7fb0d5400640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d810d300 0x7fb0d811ea60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb0d002d530 tx=0x7fb0d00338e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.925+0000 7fb0cea00640 1 -- 172.21.15.69:0/2441486361 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d003a040 con 0x7fb0d810d300 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.926+0000 7fb0cea00640 1 -- 172.21.15.69:0/2441486361 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d0045070 con 0x7fb0d810d300 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.926+0000 7fb0dcc00640 1 -- 172.21.15.69:0/2441486361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d8119630 con 0x7fb0d810d300 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.926+0000 7fb0cea00640 1 -- 172.21.15.69:0/2441486361 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0040b90 con 0x7fb0d810d300 2024-03-19T01:19:42.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.926+0000 7fb0dcc00640 1 -- 172.21.15.69:0/2441486361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d8119bd0 con 0x7fb0d810d300 2024-03-19T01:19:42.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.933+0000 7fb0df826740 1 Processor -- start 2024-03-19T01:19:42.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.936+0000 7fb0df826740 1 -- start start 2024-03-19T01:19:42.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.937+0000 7fb0df826740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea21c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.937+0000 7fb0df826740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea224ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.937+0000 7fb0df826740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 0x5641ea227800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.937+0000 7fb0df826740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5641ea1802c0 con 0x5641ea21bf10 2024-03-19T01:19:42.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.937+0000 7fb0df826740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5641ea128b60 con 0x5641ea225410 2024-03-19T01:19:42.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.937+0000 7fb0df826740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5641ea17c360 con 0x5641ea21c830 2024-03-19T01:19:42.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.938+0000 7fb0d5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea224ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.938+0000 7fb0d5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea224ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44106/0 (socket says 172.21.15.69:44106) 2024-03-19T01:19:42.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.938+0000 7fb0d5e00640 1 -- 172.21.15.69:0/3453750981 learned_addr learned my addr 172.21.15.69:0/3453750981 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:42.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.938+0000 7fb0d6800640 1 --2- 172.21.15.69:0/3453750981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea21c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.940 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/timekeeping.o 2024-03-19T01:19:42.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.942+0000 7fb0cea00640 1 -- 172.21.15.69:0/2441486361 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fb0d0035640 con 0x7fb0d810d300 2024-03-19T01:19:42.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.943+0000 7fb0d5400640 1 --2- 172.21.15.69:0/3453750981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 0x5641ea227800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.944+0000 7fb0d5400640 1 -- 172.21.15.69:0/3453750981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 msgr2=0x5641ea224ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:42.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.944+0000 7fb0d5400640 1 --2- 172.21.15.69:0/3453750981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea224ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.944+0000 7fb0d5400640 1 -- 172.21.15.69:0/3453750981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 msgr2=0x5641ea21c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:42.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.944+0000 7fb0d5400640 1 --2- 172.21.15.69:0/3453750981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea21c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.944+0000 7fb0d5400640 1 -- 172.21.15.69:0/3453750981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e9f5e7d0 con 0x5641ea225410 2024-03-19T01:19:42.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.945+0000 7fb0d6800640 1 --2- 172.21.15.69:0/3453750981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea21c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:42.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.946+0000 7fb0cea00640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb0bc0a2ab0 0x7fb0bc0a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.946+0000 7fb0cea00640 1 -- 172.21.15.69:0/2441486361 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fb0d00f1820 con 0x7fb0d810d300 2024-03-19T01:19:42.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.947+0000 7fb0d6800640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb0bc0a2ab0 0x7fb0bc0a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.949+0000 7fb0d5400640 1 --2- 172.21.15.69:0/3453750981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 0x5641ea227800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00eec30 tx=0x7fb0d00f7b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=555a1fb04a45d8ab server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:42.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.950+0000 7fb0c3000640 1 -- 172.21.15.69:0/3453750981 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00b9d50 con 0x5641ea225410 2024-03-19T01:19:42.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.950+0000 7fb0c3000640 1 -- 172.21.15.69:0/3453750981 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d0040450 con 0x5641ea225410 2024-03-19T01:19:42.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.950+0000 7fb0c3000640 1 -- 172.21.15.69:0/3453750981 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00bbc80 con 0x5641ea225410 2024-03-19T01:19:42.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.951+0000 7fb0df826740 1 -- 172.21.15.69:0/3453750981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 msgr2=0x5641ea227800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.951+0000 7fb0df826740 1 --2- 172.21.15.69:0/3453750981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 0x5641ea227800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00eec30 tx=0x7fb0d00f7b20 comp rx=0 tx=0).stop 2024-03-19T01:19:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.954+0000 7fb0d6800640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb0bc0a2ab0 0x7fb0bc0a4f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800ab70 tx=0x7fb0c8005e10 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.954+0000 7fb0df826740 1 -- 172.21.15.69:0/3453750981 shutdown_connections 2024-03-19T01:19:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.954+0000 7fb0df826740 1 --2- 172.21.15.69:0/3453750981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 0x5641ea227800 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.954+0000 7fb0df826740 1 --2- 172.21.15.69:0/3453750981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea224ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.954+0000 7fb0df826740 1 --2- 172.21.15.69:0/3453750981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea21c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.954+0000 7fb0df826740 1 -- 172.21.15.69:0/3453750981 >> 172.21.15.69:0/3453750981 conn(0x5641ea216640 msgr2=0x5641ea216a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:42.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.955+0000 7fb0df826740 1 -- 172.21.15.69:0/3453750981 shutdown_connections 2024-03-19T01:19:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.956+0000 7fb0df826740 1 -- 172.21.15.69:0/3453750981 wait complete. 2024-03-19T01:19:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.956+0000 7fb0df826740 1 -- start start 2024-03-19T01:19:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.957+0000 7fb0df826740 1 Processor -- start 2024-03-19T01:19:42.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.959+0000 7fb0df826740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea222500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.960+0000 7fb0d5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea222500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.960+0000 7fb0df826740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea222a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.961+0000 7fb0d5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea222500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59746/0 (socket says 172.21.15.69:59746) 2024-03-19T01:19:42.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.961+0000 7fb0d5e00640 1 -- 172.21.15.69:0/3726044131 learned_addr learned my addr 172.21.15.69:0/3726044131 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:42.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.961+0000 7fb0df826740 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 0x5641ea222f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.961+0000 7fb0d5400640 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea222a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.961+0000 7fb0df826740 1 -- 172.21.15.69:0/3726044131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5641ea128b60 con 0x5641ea21bf10 2024-03-19T01:19:42.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.961+0000 7fb0df826740 1 -- 172.21.15.69:0/3726044131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5641ea13bfc0 con 0x5641ea225410 2024-03-19T01:19:42.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.961+0000 7fb0df826740 1 -- 172.21.15.69:0/3726044131 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5641ea227fc0 con 0x5641ea21c830 2024-03-19T01:19:42.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.961+0000 7fb0d6800640 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 0x5641ea222f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.962+0000 7fb0d6800640 1 -- 172.21.15.69:0/3726044131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 msgr2=0x5641ea222a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:42.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.962+0000 7fb0d6800640 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea222a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.962+0000 7fb0d6800640 1 -- 172.21.15.69:0/3726044131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 msgr2=0x5641ea222500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:42.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.962+0000 7fb0d6800640 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea222500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.962+0000 7fb0d6800640 1 -- 172.21.15.69:0/3726044131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e9e943f0 con 0x5641ea225410 2024-03-19T01:19:42.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.962+0000 7fb0d5400640 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea222a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:19:42.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.962+0000 7fb0d5e00640 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea222500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:19:42.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.962+0000 7fb0d6800640 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 0x5641ea222f80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800f7f0 tx=0x7fb0c8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8181952cbecd34bf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:42.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.962+0000 7fb0b7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:19:42.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.962+0000 7fb0b7400640 1 -- 172.21.15.69:0/3726044131 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8022070 con 0x5641ea225410 2024-03-19T01:19:42.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.963+0000 7fb0b7400640 1 -- 172.21.15.69:0/3726044131 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fb0c8010920 con 0x5641ea225410 2024-03-19T01:19:42.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.963+0000 7fb0df826740 10 client.0 _ll_register_callbacks cb 0x5641ea13b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:42.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.963+0000 7fb0df826740 1 -- 172.21.15.69:0/3726044131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641ea121610 con 0x5641ea225410 2024-03-19T01:19:42.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.966+0000 7fb0b7400640 1 -- 172.21.15.69:0/3726044131 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb0c802c050 con 0x5641ea225410 2024-03-19T01:19:42.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.966+0000 7fb0df826740 10 client.27578 fetch_fsmap learned FSMap version 54 2024-03-19T01:19:42.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.966+0000 7fb0df826740 1 -- 172.21.15.69:0/3726044131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=54}) -- 0x5641e9e540f0 con 0x5641ea225410 2024-03-19T01:19:42.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.967+0000 7fb0b7400640 1 -- 172.21.15.69:0/3726044131 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7fb0c8010b20 con 0x5641ea225410 2024-03-19T01:19:42.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.967+0000 7fb0df826740 10 client.27578 fetch_fsmap finished waiting for FSMap version 54 2024-03-19T01:19:42.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.967+0000 7fb0df826740 10 client.27578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:42.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.967+0000 7fb0df826740 20 client.27578 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:42.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.967+0000 7fb0df826740 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5641ea234f80 0x5641ea237440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:42.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.970+0000 7fb0df826740 4 client.27578 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:19:42.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.970+0000 7fb0df826740 1 -- 172.21.15.69:0/3726044131 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641ea18d800 con 0x5641ea234f80 2024-03-19T01:19:42.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.971+0000 7fb0d5e00640 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5641ea234f80 0x5641ea237440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:42.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.971+0000 7fb0d5e00640 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5641ea234f80 0x5641ea237440 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43411669bf83758 server_cookie=683b01bfd2f53303 in_seq=0 out_seq=0 2024-03-19T01:19:42.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.971+0000 7fb0b7400640 10 client.27578 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:42.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.974+0000 7fb0b7400640 1 -- 172.21.15.69:0/3726044131 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5641ea18d800 con 0x5641ea234f80 2024-03-19T01:19:42.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.974+0000 7fb0b7400640 10 client.27578 handle_command_reply: tid=0 2024-03-19T01:19:42.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.974+0000 7fb0df826740 1 client.27578 shutdown 2024-03-19T01:19:42.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.975+0000 7fb0df826740 1 -- 172.21.15.69:0/3726044131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 msgr2=0x5641ea222f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:42.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.976+0000 7fb0df826740 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 0x5641ea222f80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800f7f0 tx=0x7fb0c8004730 comp rx=0 tx=0).stop 2024-03-19T01:19:42.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.978+0000 7fb0df826740 1 -- 172.21.15.69:0/3726044131 shutdown_connections 2024-03-19T01:19:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.978+0000 7fb0df826740 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5641ea234f80 0x5641ea237440 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.978+0000 7fb0df826740 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea225410 0x5641ea222f80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.978+0000 7fb0df826740 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea222a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.978+0000 7fb0df826740 1 --2- 172.21.15.69:0/3726044131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea222500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.978+0000 7fb0df826740 1 -- 172.21.15.69:0/3726044131 >> 172.21.15.69:0/3726044131 conn(0x5641ea216640 msgr2=0x5641ea215900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:42.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.978+0000 7fb0b7400640 0 client.27578 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:42.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.979+0000 7fb0df826740 1 -- 172.21.15.69:0/3726044131 shutdown_connections 2024-03-19T01:19:42.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.980+0000 7fb0df826740 1 -- 172.21.15.69:0/3726044131 wait complete. 2024-03-19T01:19:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:42.983+0000 7fb0df826740 20 client.27578 trim_cache size 0 max 16384 2024-03-19T01:19:43.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.078+0000 7fb0df826740 1 Processor -- start 2024-03-19T01:19:43.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.082+0000 7fb0df826740 1 -- start start 2024-03-19T01:19:43.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.083+0000 7fb0df826740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea230ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:43.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.083+0000 7fb0df826740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21c830 0x5641ea215780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:43.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.083+0000 7fb0df826740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea225410 0x5641ea23cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:43.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.083+0000 7fb0df826740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81c61e0 con 0x5641ea21bf10 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.083+0000 7fb0df826740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81c7e40 con 0x5641ea21c830 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.083+0000 7fb0df826740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81c68b0 con 0x5641ea225410 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.083+0000 7fb0d6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea225410 0x5641ea23cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.084+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21c830 0x5641ea215780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.084+0000 7fb0d6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea225410 0x5641ea23cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44122/0 (socket says 172.21.15.69:44122) 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.084+0000 7fb0d6800640 1 -- 172.21.15.69:0/3925450513 learned_addr learned my addr 172.21.15.69:0/3925450513 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.084+0000 7fb0d5400640 1 -- 172.21.15.69:0/3925450513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea225410 msgr2=0x5641ea23cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.084+0000 7fb0d5400640 1 --2- 172.21.15.69:0/3925450513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea225410 0x5641ea23cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.084+0000 7fb0d5e00640 1 --2- 172.21.15.69:0/3925450513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea230ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.084+0000 7fb0d5400640 1 -- 172.21.15.69:0/3925450513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 msgr2=0x5641ea230ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.084+0000 7fb0d5400640 1 --2- 172.21.15.69:0/3925450513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea230ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.084+0000 7fb0d5400640 1 -- 172.21.15.69:0/3925450513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ea216170 con 0x5641ea21c830 2024-03-19T01:19:43.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.084+0000 7fb0d5e00640 1 --2- 172.21.15.69:0/3925450513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea230ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:19:43.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.085+0000 7fb0d5400640 1 --2- 172.21.15.69:0/3925450513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21c830 0x5641ea215780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00eec30 tx=0x7fb0d0009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=374914b5e0b9da9e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:43.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.088+0000 7fb0c2600640 1 -- 172.21.15.69:0/3925450513 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00bd4a0 con 0x5641ea21c830 2024-03-19T01:19:43.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.088+0000 7fb0c2600640 1 -- 172.21.15.69:0/3925450513 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d00be240 con 0x5641ea21c830 2024-03-19T01:19:43.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.088+0000 7fb0c2600640 1 -- 172.21.15.69:0/3925450513 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00bd680 con 0x5641ea21c830 2024-03-19T01:19:43.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.092+0000 7fb0df826740 1 -- 172.21.15.69:0/3925450513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21c830 msgr2=0x5641ea215780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:43.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.092+0000 7fb0df826740 1 --2- 172.21.15.69:0/3925450513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21c830 0x5641ea215780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00eec30 tx=0x7fb0d0009f60 comp rx=0 tx=0).stop 2024-03-19T01:19:43.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.093+0000 7fb0df826740 1 -- 172.21.15.69:0/3925450513 shutdown_connections 2024-03-19T01:19:43.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.093+0000 7fb0df826740 1 --2- 172.21.15.69:0/3925450513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea225410 0x5641ea23cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.093+0000 7fb0df826740 1 --2- 172.21.15.69:0/3925450513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21c830 0x5641ea215780 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.093+0000 7fb0df826740 1 --2- 172.21.15.69:0/3925450513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea21bf10 0x5641ea230ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.093+0000 7fb0df826740 1 -- 172.21.15.69:0/3925450513 >> 172.21.15.69:0/3925450513 conn(0x5641ea216640 msgr2=0x5641ea215070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:43.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.093+0000 7fb0df826740 1 -- 172.21.15.69:0/3925450513 shutdown_connections 2024-03-19T01:19:43.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.094+0000 7fb0df826740 1 -- 172.21.15.69:0/3925450513 wait complete. 2024-03-19T01:19:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.100+0000 7fb0df826740 1 -- start start 2024-03-19T01:19:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.100+0000 7fb0df826740 1 Processor -- start 2024-03-19T01:19:43.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.101+0000 7fb0df826740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21bf10 0x5641ea22d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:43.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.102+0000 7fb0d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21bf10 0x5641ea22d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:43.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.102+0000 7fb0d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21bf10 0x5641ea22d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:32920/0 (socket says 172.21.15.69:32920) 2024-03-19T01:19:43.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.104+0000 7fb0df826740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea21e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:43.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.105+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea21e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:43.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.105+0000 7fb0d5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea21e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:44126/0 (socket says 172.21.15.69:44126) 2024-03-19T01:19:43.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.105+0000 7fb0df826740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea225410 0x5641ea21edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:43.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.105+0000 7fb0d6800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea225410 0x5641ea21edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:43.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.105+0000 7fb0d6800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea225410 0x5641ea21edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59758/0 (socket says 172.21.15.69:59758) 2024-03-19T01:19:43.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.106+0000 7fb0d5400640 1 -- 172.21.15.69:0/1317700650 learned_addr learned my addr 172.21.15.69:0/1317700650 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:19:43.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.107+0000 7fb0df826740 1 -- 172.21.15.69:0/1317700650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81c76b0 con 0x5641ea225410 2024-03-19T01:19:43.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.107+0000 7fb0df826740 1 -- 172.21.15.69:0/1317700650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81c6fe0 con 0x5641ea21bf10 2024-03-19T01:19:43.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.107+0000 7fb0df826740 1 -- 172.21.15.69:0/1317700650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81c41c0 con 0x5641ea21c830 2024-03-19T01:19:43.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.107+0000 7fb0df826740 10 client.0 _ll_register_callbacks cb 0x7fb0c800fe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:19:43.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.108+0000 7fb0d5400640 1 -- 172.21.15.69:0/1317700650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21bf10 msgr2=0x5641ea22d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:43.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.108+0000 7fb0d5400640 1 --2- 172.21.15.69:0/1317700650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21bf10 0x5641ea22d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.108+0000 7fb0d5400640 1 -- 172.21.15.69:0/1317700650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea225410 msgr2=0x5641ea21edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:19:43.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.108+0000 7fb0d5400640 1 --2- 172.21.15.69:0/1317700650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea225410 0x5641ea21edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.108+0000 7fb0d5400640 1 -- 172.21.15.69:0/1317700650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641ea22d8a0 con 0x5641ea21c830 2024-03-19T01:19:43.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.110+0000 7fb0d5400640 1 --2- 172.21.15.69:0/1317700650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea21e8b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00b9f10 tx=0x7fb0d00f7fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3920f76a6223c22f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:19:43.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.110+0000 7fb0b7400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:19:43.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.111+0000 7fb0b7400640 1 -- 172.21.15.69:0/1317700650 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00bd910 con 0x5641ea21c830 2024-03-19T01:19:43.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.111+0000 7fb0b7400640 1 -- 172.21.15.69:0/1317700650 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9568+0+0 (secure 0 0 0) 0x7fb0d00bdfb0 con 0x5641ea21c830 2024-03-19T01:19:43.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.112+0000 7fb0df826740 1 -- 172.21.15.69:0/1317700650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d00be240 con 0x5641ea21c830 2024-03-19T01:19:43.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.113+0000 7fb0b7400640 1 -- 172.21.15.69:0/1317700650 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb0d00b9d50 con 0x5641ea21c830 2024-03-19T01:19:43.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.114+0000 7fb0df826740 10 client.27547 fetch_fsmap learned FSMap version 54 2024-03-19T01:19:43.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.114+0000 7fb0df826740 1 -- 172.21.15.69:0/1317700650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5641ea29ad60 con 0x5641ea21c830 2024-03-19T01:19:43.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.116+0000 7fb0b7400640 1 -- 172.21.15.69:0/1317700650 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d00be240 con 0x5641ea21c830 2024-03-19T01:19:43.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.116+0000 7fb0b7400640 1 -- 172.21.15.69:0/1317700650 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7fb0d00c5e50 con 0x5641ea21c830 2024-03-19T01:19:43.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.117+0000 7fb0df826740 10 client.27547 fetch_fsmap finished waiting for FSMap version 54 2024-03-19T01:19:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.117+0000 7fb0df826740 10 client.27547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:19:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.117+0000 7fb0df826740 20 client.27547 populate_metadata read hostname 'smithi069' 2024-03-19T01:19:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.117+0000 7fb0df826740 1 --2- 172.21.15.69:0/1317700650 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5641ea29b110 0x5641ea29d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:19:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.117+0000 7fb0df826740 4 client.27547 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:19:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.117+0000 7fb0df826740 1 -- 172.21.15.69:0/1317700650 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641ea187b50 con 0x5641ea29b110 2024-03-19T01:19:43.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.119+0000 7fb0d5e00640 1 --2- 172.21.15.69:0/1317700650 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5641ea29b110 0x5641ea29d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:19:43.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.121+0000 7fb0b7400640 10 client.27547 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:43.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.121+0000 7fb0d5e00640 1 --2- 172.21.15.69:0/1317700650 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5641ea29b110 0x5641ea29d530 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0330a986d5269b server_cookie=8fa2a7dc40ec7878 in_seq=0 out_seq=0 2024-03-19T01:19:43.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.122+0000 7fb0b7400640 1 -- 172.21.15.69:0/1317700650 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x5641ea187b50 con 0x5641ea29b110 2024-03-19T01:19:43.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.122+0000 7fb0b7400640 10 client.27547 handle_command_reply: tid=0 2024-03-19T01:19:43.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.122+0000 7fb0df826740 1 client.27547 shutdown 2024-03-19T01:19:43.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.123+0000 7fb0df826740 1 -- 172.21.15.69:0/1317700650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 msgr2=0x5641ea21e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:43.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.123+0000 7fb0df826740 1 --2- 172.21.15.69:0/1317700650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea21e8b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00b9f10 tx=0x7fb0d00f7fd0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.123+0000 7fb0df826740 1 -- 172.21.15.69:0/1317700650 shutdown_connections 2024-03-19T01:19:43.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.123+0000 7fb0df826740 1 --2- 172.21.15.69:0/1317700650 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5641ea29b110 0x5641ea29d530 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.123+0000 7fb0df826740 1 --2- 172.21.15.69:0/1317700650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641ea225410 0x5641ea21edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.123+0000 7fb0df826740 1 --2- 172.21.15.69:0/1317700650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5641ea21c830 0x5641ea21e8b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.123+0000 7fb0df826740 1 --2- 172.21.15.69:0/1317700650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641ea21bf10 0x5641ea22d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.123+0000 7fb0df826740 1 -- 172.21.15.69:0/1317700650 >> 172.21.15.69:0/1317700650 conn(0x5641ea216640 msgr2=0x5641ea215900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:43.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.124+0000 7fb0b7400640 0 client.27547 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:19:43.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.125+0000 7fb0df826740 1 -- 172.21.15.69:0/1317700650 shutdown_connections 2024-03-19T01:19:43.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.125+0000 7fb0df826740 1 -- 172.21.15.69:0/1317700650 wait complete. 2024-03-19T01:19:43.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.127+0000 7fb0df826740 20 client.27547 trim_cache size 0 max 16384 2024-03-19T01:19:43.135 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:19:43.135 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (11448 inodes in the stack)", 2024-03-19T01:19:43.135 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:19:43.135 INFO:teuthology.orchestra.run.smithi069.stdout: "8833b1af-6426-4537-8ef8-fa88fda3c48d": { 2024-03-19T01:19:43.135 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:19:43.135 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "8833b1af-6426-4537-8ef8-fa88fda3c48d", 2024-03-19T01:19:43.135 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:19:43.135 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:19:43.136 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:19:43.136 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:19:43.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.136+0000 7fb0dcc00640 1 -- 172.21.15.69:0/2441486361 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb0bc0a2ab0 msgr2=0x7fb0bc0a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:43.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.136+0000 7fb0dcc00640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb0bc0a2ab0 0x7fb0bc0a4f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800ab70 tx=0x7fb0c8005e10 comp rx=0 tx=0).stop 2024-03-19T01:19:43.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.137+0000 7fb0dcc00640 1 -- 172.21.15.69:0/2441486361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d810d300 msgr2=0x7fb0d811ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:19:43.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.137+0000 7fb0dcc00640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d810d300 0x7fb0d811ea60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb0d002d530 tx=0x7fb0d00338e0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.138+0000 7fb0dcc00640 1 -- 172.21.15.69:0/2441486361 shutdown_connections 2024-03-19T01:19:43.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.138+0000 7fb0dcc00640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0d8124230 0x7fb0d81246e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.138+0000 7fb0dcc00640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb0bc0a2ab0 0x7fb0bc0a4f70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.138+0000 7fb0dcc00640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d810d300 0x7fb0d811ea60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.138+0000 7fb0dcc00640 1 --2- 172.21.15.69:0/2441486361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0d8108a90 0x7fb0d811e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:19:43.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.138+0000 7fb0dcc00640 1 -- 172.21.15.69:0/2441486361 >> 172.21.15.69:0/2441486361 conn(0x7fb0d806cb50 msgr2=0x7fb0d8071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:19:43.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.142+0000 7fb0dcc00640 1 -- 172.21.15.69:0/2441486361 shutdown_connections 2024-03-19T01:19:43.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:19:43.142+0000 7fb0dcc00640 1 -- 172.21.15.69:0/2441486361 wait complete. 2024-03-19T01:19:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:42 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:19:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:43.232 INFO:tasks.cephfs.filesystem:scrub status for tag:8833b1af-6426-4537-8ef8-fa88fda3c48d - {'path': '/', 'tag': '8833b1af-6426-4537-8ef8-fa88fda3c48d', 'options': 'recursive,force'} 2024-03-19T01:19:43.255 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:42 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:19:43.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:43.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:43.265 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/hugetlb.o 2024-03-19T01:19:43.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:42 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:19:43.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:43.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:43.389 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/kaslr.o 2024-03-19T01:19:43.471 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/dump_pagetables.o 2024-03-19T01:19:43.682 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/futex/pi.o 2024-03-19T01:19:43.847 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/ialloc.o 2024-03-19T01:19:43.937 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/memcpy_64.o 2024-03-19T01:19:44.039 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zlib_inflate/infutil.o 2024-03-19T01:19:44.069 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/memmove_64.o 2024-03-19T01:19:44.198 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/memset_64.o 2024-03-19T01:19:44.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: pgmap v280: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 721 KiB/s wr, 172 op/s 2024-03-19T01:19:44.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.262 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.262 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.262 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.262 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.262 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.262 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.262 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.262 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:44.262 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: pgmap v280: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 721 KiB/s wr, 172 op/s 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:44.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: pgmap v280: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 721 KiB/s wr, 172 op/s 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:44.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:44.316 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/misc.o 2024-03-19T01:19:44.438 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/pc-conf-reg.o 2024-03-19T01:19:44.734 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/futex/requeue.o 2024-03-19T01:19:44.845 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zlib_inflate/inftrees.o 2024-03-19T01:19:44.901 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/putuser.o 2024-03-19T01:19:44.937 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/numa.o 2024-03-19T01:19:45.063 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/retpoline.o 2024-03-19T01:19:45.194 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/ntp.o 2024-03-19T01:19:45.195 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zlib_inflate/inflate_syms.o 2024-03-19T01:19:45.227 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/usercopy.o 2024-03-19T01:19:45.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:45.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:45.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:45.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:45.819 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/futex/waitwake.o 2024-03-19T01:19:46.068 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/usercopy_64.o 2024-03-19T01:19:46.084 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/zlib_inflate/built-in.a 2024-03-19T01:19:46.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: pgmap v281: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 820 KiB/s wr, 179 op/s 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:46.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: pgmap v281: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 820 KiB/s wr, 179 op/s 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:46.299 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zlib_deflate/deflate.o 2024-03-19T01:19:46.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: pgmap v281: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 820 KiB/s wr, 179 op/s 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:46.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:46.668 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/clocksource.o 2024-03-19T01:19:46.840 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/numa_64.o 2024-03-19T01:19:46.936 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/futex/built-in.a 2024-03-19T01:19:47.091 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/indirect.o 2024-03-19T01:19:47.212 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.213 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.214 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:47.215 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:47.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:47.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:47.406 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/msr-smp.o 2024-03-19T01:19:47.671 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zlib_deflate/deftree.o 2024-03-19T01:19:47.847 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/amdtopology.o 2024-03-19T01:19:47.879 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/irqchip/built-in.a 2024-03-19T01:19:47.967 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/bus/mhi/built-in.a 2024-03-19T01:19:48.021 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/bus/built-in.a 2024-03-19T01:19:48.110 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/cache-smp.o 2024-03-19T01:19:48.121 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pwm/built-in.a 2024-03-19T01:19:48.138 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/jiffies.o 2024-03-19T01:19:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: pgmap v282: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 924 KiB/s wr, 215 op/s 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:48.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.271 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/msi/pcidev_msi.o 2024-03-19T01:19:48.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: pgmap v282: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 924 KiB/s wr, 215 op/s 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:48.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: pgmap v282: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 924 KiB/s wr, 215 op/s 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:48.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:48.463 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/hugetlb_vmemmap.o 2024-03-19T01:19:48.501 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/msr.o 2024-03-19T01:19:48.553 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zlib_deflate/deflate_syms.o 2024-03-19T01:19:49.082 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/timer_list.o 2024-03-19T01:19:49.244 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/srat.o 2024-03-19T01:19:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:49.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:49.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:49.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:49.363 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/zlib_deflate/built-in.a 2024-03-19T01:19:49.558 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/lzo/lzo1x_compress.o 2024-03-19T01:19:49.816 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/msr-reg.o 2024-03-19T01:19:49.820 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/msi/api.o 2024-03-19T01:19:49.823 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/inline.o 2024-03-19T01:19:50.001 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/msr-reg-export.o 2024-03-19T01:19:50.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: pgmap v283: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 943 KiB/s wr, 303 op/s 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:50.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.260 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mempolicy.o 2024-03-19T01:19:50.274 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/hweight.o 2024-03-19T01:19:50.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: pgmap v283: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 943 KiB/s wr, 303 op/s 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:50.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.291 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/timeconv.o 2024-03-19T01:19:50.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: pgmap v283: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 943 KiB/s wr, 303 op/s 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:50.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:50.369 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/iomem.o 2024-03-19T01:19:50.380 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/pkeys.o 2024-03-19T01:19:50.678 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/lzo/lzo1x_decompress_safe.o 2024-03-19T01:19:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:51.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:51.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.306 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/timecounter.o 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:51.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:51.357 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/msi/msi.o 2024-03-19T01:19:51.378 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/lib/iomap_copy_64.o 2024-03-19T01:19:51.480 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/alarmtimer.o 2024-03-19T01:19:51.490 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/kaslr.o 2024-03-19T01:19:51.503 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/lib/inat.o 2024-03-19T01:19:51.564 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/lzo/built-in.a 2024-03-19T01:19:51.727 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/lz4/lz4_decompress.o 2024-03-19T01:19:51.766 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/lib/built-in.a 2024-03-19T01:19:51.867 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/lib/lib.a 2024-03-19T01:19:52.096 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/mm/pti.o 2024-03-19T01:19:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: pgmap v284: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 919 KiB/s wr, 322 op/s 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:52.259 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: pgmap v284: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 919 KiB/s wr, 322 op/s 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:52.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: pgmap v284: 129 pgs: 129 active+clean; 2.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 919 KiB/s wr, 322 op/s 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:52.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:52.546 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/inode.o 2024-03-19T01:19:52.650 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/acpi/boot.o 2024-03-19T01:19:53.156 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/msi/irqdomain.o 2024-03-19T01:19:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:53.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:53.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.288 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/sparse.o 2024-03-19T01:19:53.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:53.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:53.423 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/posix-timers.o 2024-03-19T01:19:53.469 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/mm/built-in.a 2024-03-19T01:19:53.480 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/lz4/built-in.a 2024-03-19T01:19:53.572 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/ioctl.o 2024-03-19T01:19:53.648 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zstd/zstd_decompress_module.o 2024-03-19T01:19:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: pgmap v285: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 981 KiB/s wr, 904 op/s 2024-03-19T01:19:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:54.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:54.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:54.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:54.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:54.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.257 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:54.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:54.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:54.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:54.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.258 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: pgmap v285: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 981 KiB/s wr, 904 op/s 2024-03-19T01:19:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:54.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:54.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: pgmap v285: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 981 KiB/s wr, 904 op/s 2024-03-19T01:19:54.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:54.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:54.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:54.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:54.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:54.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:54.351 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zstd/decompress/huf_decompress.o 2024-03-19T01:19:54.396 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/acpi/sleep.o 2024-03-19T01:19:54.412 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pci/msi/built-in.a 2024-03-19T01:19:54.531 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/pcie/portdrv.o 2024-03-19T01:19:54.853 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/sparse-vmemmap.o 2024-03-19T01:19:55.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:55.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:55.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:55.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:55.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:55.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:19:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:55.632 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/posix-cpu-timers.o 2024-03-19T01:19:55.648 INFO:tasks.workunit.client.0.smithi069.stdout: AS arch/x86/kernel/acpi/wakeup_64.o 2024-03-19T01:19:55.835 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/acpi/cppc.o 2024-03-19T01:19:56.085 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/pcie/rcec.o 2024-03-19T01:19:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:56 smithi163 ceph-mon[67428]: pgmap v286: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 511 KiB/s wr, 1.30k op/s 2024-03-19T01:19:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:56.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:56.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:56.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.326 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/mmu_notifier.o 2024-03-19T01:19:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:56 smithi086 ceph-mon[66829]: pgmap v286: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 511 KiB/s wr, 1.30k op/s 2024-03-19T01:19:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:56 smithi069 ceph-mon[31727]: pgmap v286: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 511 KiB/s wr, 1.30k op/s 2024-03-19T01:19:56.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:56.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:56.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:56.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:56.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:56.664 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/hotplug/pci_hotplug_core.o 2024-03-19T01:19:56.812 INFO:tasks.workunit.client.0.smithi069.stdout: CC arch/x86/kernel/acpi/cstate.o 2024-03-19T01:19:57.189 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/pcie/aspm.o 2024-03-19T01:19:57.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:57.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:57.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:57.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:57.392 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/posix-clock.o 2024-03-19T01:19:57.435 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:57.435 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:57.435 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:57.435 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:57.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:57.594 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:57.594 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:57.594 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:57.727 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zstd/decompress/zstd_ddict.o 2024-03-19T01:19:57.737 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/slub.o 2024-03-19T01:19:57.841 INFO:tasks.workunit.client.0.smithi069.stdout: AR arch/x86/kernel/acpi/built-in.a 2024-03-19T01:19:58.031 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zstd/decompress/zstd_decompress.o 2024-03-19T01:19:58.042 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ramfs/inode.o 2024-03-19T01:19:58.139 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/hotplug/acpi_pcihp.o 2024-03-19T01:19:58.304 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/mballoc.o 2024-03-19T01:19:58.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:58 smithi163 ceph-mon[67428]: pgmap v287: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 420 KiB/s wr, 1.53k op/s 2024-03-19T01:19:58.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:58.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:58.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:58.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:58.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:58.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:58.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:58.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:58 smithi086 ceph-mon[66829]: pgmap v287: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 420 KiB/s wr, 1.53k op/s 2024-03-19T01:19:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:58.530 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/itimer.o 2024-03-19T01:19:58.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:58 smithi069 ceph-mon[31727]: pgmap v287: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 420 KiB/s wr, 1.53k op/s 2024-03-19T01:19:58.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.196 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zstd/decompress/zstd_decompress_block.o 2024-03-19T01:19:59.208 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/pcie/pme.o 2024-03-19T01:19:59.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:59.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:59.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:59.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:59.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:59.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:19:59.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:19:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.418 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ramfs/file-mmu.o 2024-03-19T01:19:59.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:19:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:19:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:19:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:19:59.556 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pci/hotplug/built-in.a 2024-03-19T01:19:59.711 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/video/console/dummycon.o 2024-03-19T01:20:00.321 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/clockevents.o 2024-03-19T01:20:00.367 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/ramfs/built-in.a 2024-03-19T01:20:00.372 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pci/pcie/built-in.a 2024-03-19T01:20:00.474 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/idle/built-in.a 2024-03-19T01:20:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: pgmap v288: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 683 KiB/s wr, 1.67k op/s 2024-03-19T01:20:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:20:00.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:00.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:00 smithi086 ceph-mon[66829]: overall HEALTH_OK 2024-03-19T01:20:00.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: pgmap v288: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 683 KiB/s wr, 1.67k op/s 2024-03-19T01:20:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:20:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:00.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:00 smithi069 ceph-mon[31727]: overall HEALTH_OK 2024-03-19T01:20:00.537 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/video/backlight/backlight.o 2024-03-19T01:20:00.546 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pci/controller/dwc/built-in.a 2024-03-19T01:20:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: pgmap v288: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 683 KiB/s wr, 1.67k op/s 2024-03-19T01:20:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:20:00.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:00.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:00.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:00 smithi163 ceph-mon[67428]: overall HEALTH_OK 2024-03-19T01:20:00.615 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pci/controller/mobiveil/built-in.a 2024-03-19T01:20:00.673 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pci/controller/built-in.a 2024-03-19T01:20:00.891 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pci/switch/built-in.a 2024-03-19T01:20:00.940 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/video/console/vgacon.o 2024-03-19T01:20:00.956 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/access.o 2024-03-19T01:20:01.174 INFO:tasks.workunit.client.0.smithi069.stderr:./arch/x86/kernel/cpu/mkcapflags.sh: line 46: printf: write error: Resource temporarily unavailable 2024-03-19T01:20:01.194 INFO:tasks.workunit.client.0.smithi069.stderr:make[5]: *** [arch/x86/kernel/cpu/Makefile:64: arch/x86/kernel/cpu/capflags.c] Error 1 2024-03-19T01:20:01.194 INFO:tasks.workunit.client.0.smithi069.stderr:make[4]: *** [scripts/Makefile.build:480: arch/x86/kernel/cpu] Error 2 2024-03-19T01:20:01.195 INFO:tasks.workunit.client.0.smithi069.stderr:make[3]: *** [scripts/Makefile.build:480: arch/x86/kernel] Error 2 2024-03-19T01:20:01.196 INFO:tasks.workunit.client.0.smithi069.stderr:make[2]: *** [scripts/Makefile.build:480: arch/x86] Error 2 2024-03-19T01:20:01.196 INFO:tasks.workunit.client.0.smithi069.stderr:make[2]: *** Waiting for unfinished jobs.... 2024-03-19T01:20:01.267 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/video/fbdev/core/built-in.a 2024-03-19T01:20:01.366 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/video/fbdev/omap/built-in.a 2024-03-19T01:20:01.481 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/tick-common.o 2024-03-19T01:20:01.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:01.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:01.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:01.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:01.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:01.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:01.548 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/video/fbdev/omap2/omapfb/dss/built-in.a 2024-03-19T01:20:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:01.657 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/video/fbdev/omap2/omapfb/displays/built-in.a 2024-03-19T01:20:01.719 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/video/fbdev/omap2/omapfb/built-in.a 2024-03-19T01:20:01.726 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/migrate.o 2024-03-19T01:20:01.767 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/video/fbdev/omap2/built-in.a 2024-03-19T01:20:01.898 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/video/fbdev/built-in.a 2024-03-19T01:20:02.073 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/video/backlight/built-in.a 2024-03-19T01:20:02.086 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/xz/xz_dec_syms.o 2024-03-19T01:20:02.174 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/bus.o 2024-03-19T01:20:02.224 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zstd/zstd_common_module.o 2024-03-19T01:20:02.253 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/video/console/built-in.a 2024-03-19T01:20:02.323 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/video/aperture.o 2024-03-19T01:20:02.484 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/hugetlbfs/inode.o 2024-03-19T01:20:02.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:02 smithi086 ceph-mon[66829]: pgmap v289: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 733 KiB/s wr, 1.53k op/s 2024-03-19T01:20:02.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:02.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:02.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:02.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:02.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:02 smithi069 ceph-mon[31727]: pgmap v289: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 733 KiB/s wr, 1.53k op/s 2024-03-19T01:20:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:02 smithi163 ceph-mon[67428]: pgmap v289: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 733 KiB/s wr, 1.53k op/s 2024-03-19T01:20:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:02.806 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zstd/common/debug.o 2024-03-19T01:20:02.878 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zstd/common/entropy_common.o 2024-03-19T01:20:02.902 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/tick-broadcast.o 2024-03-19T01:20:02.965 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/xz/xz_dec_stream.o 2024-03-19T01:20:03.276 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zstd/common/error_private.o 2024-03-19T01:20:03.317 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/migrate.o 2024-03-19T01:20:03.408 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zstd/common/fse_decompress.o 2024-03-19T01:20:03.459 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/video/cmdline.o 2024-03-19T01:20:03.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:03.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:03 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:20:03.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:03.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:03 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:20:03.536 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/probe.o 2024-03-19T01:20:03.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:03.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:03 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:20:03.644 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/memory-tiers.o 2024-03-19T01:20:03.870 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/xz/xz_dec_lzma2.o 2024-03-19T01:20:03.926 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/tick-broadcast-hrtimer.o 2024-03-19T01:20:04.268 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/hugetlbfs/built-in.a 2024-03-19T01:20:04.294 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/zstd/common/zstd_common.o 2024-03-19T01:20:04.329 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/host-bridge.o 2024-03-19T01:20:04.467 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/zstd/built-in.a 2024-03-19T01:20:04.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:04 smithi086 ceph-mon[66829]: pgmap v290: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 713 KiB/s wr, 1.48k op/s 2024-03-19T01:20:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:04 smithi069 ceph-mon[31727]: pgmap v290: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 713 KiB/s wr, 1.48k op/s 2024-03-19T01:20:04.542 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/tick-oneshot.o 2024-03-19T01:20:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:04 smithi163 ceph-mon[67428]: pgmap v290: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 713 KiB/s wr, 1.48k op/s 2024-03-19T01:20:04.593 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/video/nomodeset.o 2024-03-19T01:20:04.617 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/xz/xz_dec_bcj.o 2024-03-19T01:20:04.794 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/mmp.o 2024-03-19T01:20:05.072 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/page_counter.o 2024-03-19T01:20:05.121 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cgroup/cgroup.o 2024-03-19T01:20:05.129 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cgroup/rstat.o 2024-03-19T01:20:05.202 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/tick-sched.o 2024-03-19T01:20:05.288 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/video/hdmi.o 2024-03-19T01:20:05.302 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/xz/built-in.a 2024-03-19T01:20:05.434 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/generic_mpih-lshift.o 2024-03-19T01:20:05.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:05 smithi086 ceph-mon[66829]: osdmap e101: 12 total, 12 up, 12 in 2024-03-19T01:20:05.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:05 smithi069 ceph-mon[31727]: osdmap e101: 12 total, 12 up, 12 in 2024-03-19T01:20:05.546 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/remove.o 2024-03-19T01:20:05.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:05 smithi163 ceph-mon[67428]: osdmap e101: 12 total, 12 up, 12 in 2024-03-19T01:20:05.694 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/hugetlb_cgroup.o 2024-03-19T01:20:05.934 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/move_extent.o 2024-03-19T01:20:06.255 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/generic_mpih-mul1.o 2024-03-19T01:20:06.366 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/pci.o 2024-03-19T01:20:06.383 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/video/built-in.a 2024-03-19T01:20:06.494 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/vsyscall.o 2024-03-19T01:20:06.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:06 smithi086 ceph-mon[66829]: pgmap v292: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 894 KiB/s wr, 569 op/s 2024-03-19T01:20:06.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:06 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:20:06.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:06 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:20:06.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:06 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:20:06.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:06 smithi069 ceph-mon[31727]: pgmap v292: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 894 KiB/s wr, 569 op/s 2024-03-19T01:20:06.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:06 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:20:06.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:06 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:20:06.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:06 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:20:06.540 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/early_ioremap.o 2024-03-19T01:20:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:06 smithi163 ceph-mon[67428]: pgmap v292: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 894 KiB/s wr, 569 op/s 2024-03-19T01:20:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:06 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:20:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:06 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:20:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:06 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:20:06.586 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/timekeeping_debug.o 2024-03-19T01:20:07.145 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fat/cache.o 2024-03-19T01:20:07.169 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_clock.o 2024-03-19T01:20:07.182 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/generic_mpih-mul2.o 2024-03-19T01:20:07.348 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/namei.o 2024-03-19T01:20:07.599 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/secretmem.o 2024-03-19T01:20:07.740 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/time/namespace.o 2024-03-19T01:20:07.862 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/ring_buffer.o 2024-03-19T01:20:08.097 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/generic_mpih-mul3.o 2024-03-19T01:20:08.278 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fat/dir.o 2024-03-19T01:20:08.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:08 smithi086 ceph-mon[66829]: pgmap v293: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 875 KiB/s wr, 296 op/s 2024-03-19T01:20:08.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:08 smithi069 ceph-mon[31727]: pgmap v293: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 875 KiB/s wr, 296 op/s 2024-03-19T01:20:08.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:08 smithi163 ceph-mon[67428]: pgmap v293: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 875 KiB/s wr, 296 op/s 2024-03-19T01:20:08.833 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/memfd.o 2024-03-19T01:20:08.846 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/generic_mpih-rshift.o 2024-03-19T01:20:08.863 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/time/built-in.a 2024-03-19T01:20:08.952 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace.o 2024-03-19T01:20:09.482 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cgroup/namespace.o 2024-03-19T01:20:09.565 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/pci-driver.o 2024-03-19T01:20:09.957 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/generic_mpih-sub1.o 2024-03-19T01:20:10.090 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fat/fatent.o 2024-03-19T01:20:10.208 INFO:tasks.workunit.client.0.smithi069.stdout: CC mm/ptdump.o 2024-03-19T01:20:10.320 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cgroup/cgroup-v1.o 2024-03-19T01:20:10.464 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/page-io.o 2024-03-19T01:20:10.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:10 smithi086 ceph-mon[66829]: pgmap v294: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 495 KiB/s wr, 59 op/s 2024-03-19T01:20:10.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:10 smithi086 ceph-mon[66829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:20:10.506 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/generic_mpih-add1.o 2024-03-19T01:20:10.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:10 smithi069 ceph-mon[31727]: pgmap v294: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 495 KiB/s wr, 59 op/s 2024-03-19T01:20:10.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:10 smithi069 ceph-mon[31727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:20:10.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:10 smithi163 ceph-mon[67428]: pgmap v294: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 495 KiB/s wr, 59 op/s 2024-03-19T01:20:10.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:10 smithi163 ceph-mon[67428]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:20:11.413 INFO:tasks.workunit.client.0.smithi069.stdout: AR mm/built-in.a 2024-03-19T01:20:11.571 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/ec.o 2024-03-19T01:20:11.578 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/fonts/fonts.o 2024-03-19T01:20:11.586 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/search.o 2024-03-19T01:20:11.663 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/bpf/core.o 2024-03-19T01:20:11.719 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fat/file.o 2024-03-19T01:20:11.926 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cgroup/freezer.o 2024-03-19T01:20:11.974 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/readpage.o 2024-03-19T01:20:12.164 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/fonts/font_8x16.o 2024-03-19T01:20:12.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:12 smithi086 ceph-mon[66829]: pgmap v295: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 419 KiB/s wr, 56 op/s 2024-03-19T01:20:12.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:12 smithi069 ceph-mon[31727]: pgmap v295: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 419 KiB/s wr, 56 op/s 2024-03-19T01:20:12.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:12 smithi163 ceph-mon[67428]: pgmap v295: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 419 KiB/s wr, 56 op/s 2024-03-19T01:20:13.096 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cgroup/legacy_freezer.o 2024-03-19T01:20:13.096 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpicoder.o 2024-03-19T01:20:13.098 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fat/inode.o 2024-03-19T01:20:13.099 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/pci-sysfs.o 2024-03-19T01:20:13.107 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/resize.o 2024-03-19T01:20:13.122 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_output.o 2024-03-19T01:20:13.144 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/fonts/built-in.a 2024-03-19T01:20:13.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:20:13.233 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:20:13.584 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_seq.o 2024-03-19T01:20:13.924 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/bpf/built-in.a 2024-03-19T01:20:13.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.928+0000 7f9bfe600640 1 -- 172.21.15.69:0/2342834158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bf8109320 msgr2=0x7f9bf8114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:13.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.928+0000 7f9bfe600640 1 --2- 172.21.15.69:0/2342834158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bf8109320 0x7f9bf8114430 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9be40098b0 tx=0x7f9be402bc00 comp rx=0 tx=0).stop 2024-03-19T01:20:13.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.930+0000 7f9bfe600640 1 -- 172.21.15.69:0/2342834158 shutdown_connections 2024-03-19T01:20:13.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.930+0000 7f9bfe600640 1 --2- 172.21.15.69:0/2342834158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf8114970 0x7f9bf8116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.930+0000 7f9bfe600640 1 --2- 172.21.15.69:0/2342834158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bf8109320 0x7f9bf8114430 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.930+0000 7f9bfe600640 1 --2- 172.21.15.69:0/2342834158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bf8108a00 0x7f9bf8108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.930+0000 7f9bfe600640 1 -- 172.21.15.69:0/2342834158 >> 172.21.15.69:0/2342834158 conn(0x7f9bf806c900 msgr2=0x7f9bf806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:13.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.930+0000 7f9bfe600640 1 -- 172.21.15.69:0/2342834158 shutdown_connections 2024-03-19T01:20:13.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.930+0000 7f9bfe600640 1 -- 172.21.15.69:0/2342834158 wait complete. 2024-03-19T01:20:13.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.931+0000 7f9bfe600640 1 Processor -- start 2024-03-19T01:20:13.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.935+0000 7f9bfe600640 1 -- start start 2024-03-19T01:20:13.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.935+0000 7f9bfe600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bf8108a00 0x7f9bf8113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:13.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bf8108a00 0x7f9bf8113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bf8108a00 0x7f9bf8113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:40040/0 (socket says 172.21.15.69:40040) 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bfe600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bf8109320 0x7f9bf81143a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bf7400640 1 -- 172.21.15.69:0/3747043725 learned_addr learned my addr 172.21.15.69:0/3747043725 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bfe600640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf8114970 0x7f9bf810b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bfe600640 1 -- 172.21.15.69:0/3747043725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf811a3d0 con 0x7f9bf8109320 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bfe600640 1 -- 172.21.15.69:0/3747043725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf811a250 con 0x7f9bf8114970 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bfe600640 1 -- 172.21.15.69:0/3747043725 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf811a550 con 0x7f9bf8108a00 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bf7e00640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf8114970 0x7f9bf810b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bf8109320 0x7f9bf81143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bf7400640 1 -- 172.21.15.69:0/3747043725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf8114970 msgr2=0x7f9bf810b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bf7400640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf8114970 0x7f9bf810b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bf7400640 1 -- 172.21.15.69:0/3747043725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bf8109320 msgr2=0x7f9bf81143a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:13.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bf7400640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bf8109320 0x7f9bf81143a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bf7400640 1 -- 172.21.15.69:0/3747043725 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf810b870 con 0x7f9bf8108a00 2024-03-19T01:20:13.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.936+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bf8109320 0x7f9bf81143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:13.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.937+0000 7f9bf7e00640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf8114970 0x7f9bf810b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:13.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.937+0000 7f9bf7400640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bf8108a00 0x7f9bf8113e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9be800d950 tx=0x7f9be800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:13.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.937+0000 7f9befe00640 1 -- 172.21.15.69:0/3747043725 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8017070 con 0x7f9bf8108a00 2024-03-19T01:20:13.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.937+0000 7f9befe00640 1 -- 172.21.15.69:0/3747043725 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be8009c30 con 0x7f9bf8108a00 2024-03-19T01:20:13.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.937+0000 7f9befe00640 1 -- 172.21.15.69:0/3747043725 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be800f690 con 0x7f9bf8108a00 2024-03-19T01:20:13.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.937+0000 7f9bfe600640 1 -- 172.21.15.69:0/3747043725 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf810bb60 con 0x7f9bf8108a00 2024-03-19T01:20:13.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.938+0000 7f9bfe600640 1 -- 172.21.15.69:0/3747043725 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf810fa80 con 0x7f9bf8108a00 2024-03-19T01:20:13.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.943+0000 7f9befe00640 1 -- 172.21.15.69:0/3747043725 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9be80109d0 con 0x7f9bf8108a00 2024-03-19T01:20:13.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.948+0000 7f9befe00640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bdc0a29f0 0x7f9bdc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:13.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.949+0000 7f9befe00640 1 -- 172.21.15.69:0/3747043725 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9656+0+0 (secure 0 0 0) 0x7f9be80cef30 con 0x7f9bf8108a00 2024-03-19T01:20:13.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.949+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bdc0a29f0 0x7f9bdc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:13.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.950+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bdc0a29f0 0x7f9bdc0a4eb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097d0 tx=0x7f9be4000bc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:13.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.951+0000 7f9c01210740 1 Processor -- start 2024-03-19T01:20:13.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.951+0000 7f9c01210740 1 -- start start 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.951+0000 7f9c01210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 0x55c4a47f6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.951+0000 7f9c01210740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f6660 0x55c4a47fed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.951+0000 7f9c01210740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a4801630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.952+0000 7f9c01210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c4a475a2c0 con 0x55c4a47f5d40 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.952+0000 7f9c01210740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4a4702b60 con 0x55c4a47f6660 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.952+0000 7f9c01210740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c4a4756360 con 0x55c4a47ff240 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.952+0000 7f9bf6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f6660 0x55c4a47fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.952+0000 7f9bf6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f6660 0x55c4a47fed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:50498/0 (socket says 172.21.15.69:50498) 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.952+0000 7f9bf6a00640 1 -- 172.21.15.69:0/1815523264 learned_addr learned my addr 172.21.15.69:0/1815523264 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.952+0000 7f9bf6a00640 1 -- 172.21.15.69:0/1815523264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 msgr2=0x55c4a4801630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.952+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/1815523264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a4801630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.952+0000 7f9bf6a00640 1 -- 172.21.15.69:0/1815523264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 msgr2=0x55c4a47f6120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:13.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.952+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/1815523264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 0x55c4a47f6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.952+0000 7f9bf6a00640 1 -- 172.21.15.69:0/1815523264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a45387d0 con 0x55c4a47f6660 2024-03-19T01:20:13.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.953+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/1815523264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f6660 0x55c4a47fed00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9be4036000 tx=0x7f9be403d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f760c35c060f499 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:13.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.953+0000 7f9becc00640 1 -- 172.21.15.69:0/1815523264 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4050070 con 0x55c4a47f6660 2024-03-19T01:20:13.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.953+0000 7f9becc00640 1 -- 172.21.15.69:0/1815523264 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be4032340 con 0x55c4a47f6660 2024-03-19T01:20:13.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.954+0000 7f9c01210740 1 -- 172.21.15.69:0/1815523264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f6660 msgr2=0x55c4a47fed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:13.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.954+0000 7f9c01210740 1 --2- 172.21.15.69:0/1815523264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f6660 0x55c4a47fed00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9be4036000 tx=0x7f9be403d000 comp rx=0 tx=0).stop 2024-03-19T01:20:13.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.957+0000 7f9c01210740 1 -- 172.21.15.69:0/1815523264 shutdown_connections 2024-03-19T01:20:13.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.957+0000 7f9c01210740 1 --2- 172.21.15.69:0/1815523264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a4801630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.957+0000 7f9c01210740 1 --2- 172.21.15.69:0/1815523264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f6660 0x55c4a47fed00 secure :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9be4036000 tx=0x7f9be403d000 comp rx=0 tx=0).stop 2024-03-19T01:20:13.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.957+0000 7f9c01210740 1 --2- 172.21.15.69:0/1815523264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 0x55c4a47f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.957+0000 7f9c01210740 1 -- 172.21.15.69:0/1815523264 >> 172.21.15.69:0/1815523264 conn(0x55c4a47f0470 msgr2=0x55c4a47f0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:13.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.958+0000 7f9c01210740 1 -- 172.21.15.69:0/1815523264 shutdown_connections 2024-03-19T01:20:13.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.958+0000 7f9c01210740 1 -- 172.21.15.69:0/1815523264 wait complete. 2024-03-19T01:20:13.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.959+0000 7f9c01210740 1 -- start start 2024-03-19T01:20:13.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.959+0000 7f9c01210740 1 Processor -- start 2024-03-19T01:20:13.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.961+0000 7f9c01210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 0x55c4a47fc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:13.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9c01210740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47ff240 0x55c4a47fc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:13.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9c01210740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47fcd50 0x55c4a480b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:13.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9c01210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c4a4802020 con 0x55c4a47f5d40 2024-03-19T01:20:13.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9c01210740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4a4715fc0 con 0x55c4a47ff240 2024-03-19T01:20:13.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9bf7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47fcd50 0x55c4a480b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:13.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9bf6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47ff240 0x55c4a47fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:13.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9bf7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47fcd50 0x55c4a480b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:40050/0 (socket says 172.21.15.69:40050) 2024-03-19T01:20:13.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9bf7e00640 1 -- 172.21.15.69:0/51323229 learned_addr learned my addr 172.21.15.69:0/51323229 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:13.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9c01210740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c4a48021a0 con 0x55c4a47fcd50 2024-03-19T01:20:13.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9c01210740 10 client.0 _ll_register_callbacks cb 0x55c4a4715990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:13.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9bf6a00640 1 -- 172.21.15.69:0/51323229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47fcd50 msgr2=0x55c4a480b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:13.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.963+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/51323229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47fcd50 0x55c4a480b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.964+0000 7f9bf6a00640 1 -- 172.21.15.69:0/51323229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 msgr2=0x55c4a47fc2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:13.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.964+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/51323229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 0x55c4a47fc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.964+0000 7f9bf6a00640 1 -- 172.21.15.69:0/51323229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a446e3f0 con 0x55c4a47ff240 2024-03-19T01:20:13.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.964+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/51323229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47ff240 0x55c4a47fc810 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55c4a47f1370 tx=0x7f9be4034650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d9fffe5f03517b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:13.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.964+0000 7f9be1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:20:13.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.964+0000 7f9be1600640 1 -- 172.21.15.69:0/51323229 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4050040 con 0x55c4a47ff240 2024-03-19T01:20:13.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.965+0000 7f9c01210740 1 -- 172.21.15.69:0/51323229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9be4032340 con 0x55c4a47ff240 2024-03-19T01:20:13.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.966+0000 7f9be1600640 1 -- 172.21.15.69:0/51323229 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9656+0+0 (secure 0 0 0) 0x7f9be4032640 con 0x55c4a47ff240 2024-03-19T01:20:13.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.966+0000 7f9be1600640 1 -- 172.21.15.69:0/51323229 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9be4061050 con 0x55c4a47ff240 2024-03-19T01:20:13.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.966+0000 7f9c01210740 10 client.27590 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:13.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.966+0000 7f9c01210740 1 -- 172.21.15.69:0/51323229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55c4a442e0f0 con 0x55c4a47ff240 2024-03-19T01:20:13.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.966+0000 7f9be1600640 1 -- 172.21.15.69:0/51323229 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be4032340 con 0x55c4a47ff240 2024-03-19T01:20:13.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.966+0000 7f9be1600640 1 -- 172.21.15.69:0/51323229 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f9be405c030 con 0x55c4a47ff240 2024-03-19T01:20:13.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.967+0000 7f9c01210740 10 client.27590 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:13.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.967+0000 7f9c01210740 10 client.27590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:20:13.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.967+0000 7f9c01210740 20 client.27590 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:13.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.967+0000 7f9c01210740 1 --2- 172.21.15.69:0/51323229 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c4a480f940 0x55c4a4811d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:13.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.967+0000 7f9bf7400640 1 --2- 172.21.15.69:0/51323229 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c4a480f940 0x55c4a4811d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:13.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.967+0000 7f9c01210740 4 client.27590 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:20:13.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.967+0000 7f9c01210740 1 -- 172.21.15.69:0/51323229 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4a46fb610 con 0x55c4a480f940 2024-03-19T01:20:13.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.968+0000 7f9bf7400640 1 --2- 172.21.15.69:0/51323229 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c4a480f940 0x55c4a4811d30 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2392d4966c318f server_cookie=c6f710d7710c815f in_seq=0 out_seq=0 2024-03-19T01:20:13.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.968+0000 7f9be1600640 10 client.27590 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:13.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.970+0000 7f9be1600640 1 -- 172.21.15.69:0/51323229 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c4a46fb610 con 0x55c4a480f940 2024-03-19T01:20:13.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.970+0000 7f9be1600640 10 client.27590 handle_command_reply: tid=0 2024-03-19T01:20:13.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.970+0000 7f9c01210740 1 client.27590 shutdown 2024-03-19T01:20:13.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.970+0000 7f9c01210740 1 -- 172.21.15.69:0/51323229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47ff240 msgr2=0x55c4a47fc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:13.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.970+0000 7f9c01210740 1 --2- 172.21.15.69:0/51323229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47ff240 0x55c4a47fc810 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55c4a47f1370 tx=0x7f9be4034650 comp rx=0 tx=0).stop 2024-03-19T01:20:13.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.972+0000 7f9c01210740 1 -- 172.21.15.69:0/51323229 shutdown_connections 2024-03-19T01:20:13.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.972+0000 7f9c01210740 1 --2- 172.21.15.69:0/51323229 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c4a480f940 0x55c4a4811d30 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.972+0000 7f9c01210740 1 --2- 172.21.15.69:0/51323229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47fcd50 0x55c4a480b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.972+0000 7f9c01210740 1 --2- 172.21.15.69:0/51323229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47ff240 0x55c4a47fc810 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.972+0000 7f9c01210740 1 --2- 172.21.15.69:0/51323229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 0x55c4a47fc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:13.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.972+0000 7f9be1600640 0 client.27590 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:13.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.972+0000 7f9c01210740 1 -- 172.21.15.69:0/51323229 >> 172.21.15.69:0/51323229 conn(0x55c4a47f0470 msgr2=0x55c4a47f21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:13.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.974+0000 7f9c01210740 1 -- 172.21.15.69:0/51323229 shutdown_connections 2024-03-19T01:20:13.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.975+0000 7f9c01210740 1 -- 172.21.15.69:0/51323229 wait complete. 2024-03-19T01:20:13.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:13.976+0000 7f9c01210740 20 client.27590 trim_cache size 0 max 16384 2024-03-19T01:20:14.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.054+0000 7f9c01210740 1 Processor -- start 2024-03-19T01:20:14.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.059+0000 7f9c01210740 1 -- start start 2024-03-19T01:20:14.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.059+0000 7f9c01210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 0x55c4a480b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:14.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.059+0000 7f9c01210740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47fcd50 0x55c4a4814d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:14.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.059+0000 7f9c01210740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a47f9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:14.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.059+0000 7f9c01210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81c6670 con 0x55c4a47f5d40 2024-03-19T01:20:14.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.059+0000 7f9c01210740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81c81b0 con 0x55c4a47fcd50 2024-03-19T01:20:14.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.059+0000 7f9c01210740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81c6d40 con 0x55c4a47ff240 2024-03-19T01:20:14.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.060+0000 7f9bf6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47fcd50 0x55c4a4814d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:14.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.060+0000 7f9bf6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47fcd50 0x55c4a4814d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:50514/0 (socket says 172.21.15.69:50514) 2024-03-19T01:20:14.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.060+0000 7f9bf6a00640 1 -- 172.21.15.69:0/3164153584 learned_addr learned my addr 172.21.15.69:0/3164153584 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:14.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.060+0000 7f9bf7e00640 1 --2- 172.21.15.69:0/3164153584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a47f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:14.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.060+0000 7f9bf6a00640 1 -- 172.21.15.69:0/3164153584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 msgr2=0x55c4a47f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:14.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.060+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/3164153584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a47f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.060+0000 7f9bf6a00640 1 -- 172.21.15.69:0/3164153584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 msgr2=0x55c4a480b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:14.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.060+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/3164153584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 0x55c4a480b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.060+0000 7f9bf6a00640 1 -- 172.21.15.69:0/3164153584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a47f6480 con 0x55c4a47fcd50 2024-03-19T01:20:14.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.061+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/3164153584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47fcd50 0x55c4a4814d50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9be403f040 tx=0x7f9be402bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f88a596ba3c3ee4c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:14.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.061+0000 7f9be3e00640 1 -- 172.21.15.69:0/3164153584 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4050040 con 0x55c4a47fcd50 2024-03-19T01:20:14.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.061+0000 7f9be3e00640 1 -- 172.21.15.69:0/3164153584 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be4032340 con 0x55c4a47fcd50 2024-03-19T01:20:14.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.061+0000 7f9be3e00640 1 -- 172.21.15.69:0/3164153584 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be402de60 con 0x55c4a47fcd50 2024-03-19T01:20:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.063+0000 7f9c01210740 1 -- 172.21.15.69:0/3164153584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47fcd50 msgr2=0x55c4a4814d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.063+0000 7f9c01210740 1 --2- 172.21.15.69:0/3164153584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47fcd50 0x55c4a4814d50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9be403f040 tx=0x7f9be402bc00 comp rx=0 tx=0).stop 2024-03-19T01:20:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.063+0000 7f9c01210740 1 -- 172.21.15.69:0/3164153584 shutdown_connections 2024-03-19T01:20:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.063+0000 7f9c01210740 1 --2- 172.21.15.69:0/3164153584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a47f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.063+0000 7f9c01210740 1 --2- 172.21.15.69:0/3164153584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47fcd50 0x55c4a4814d50 secure :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9be403f040 tx=0x7f9be402bc00 comp rx=0 tx=0).stop 2024-03-19T01:20:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.063+0000 7f9c01210740 1 --2- 172.21.15.69:0/3164153584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a47f5d40 0x55c4a480b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.063+0000 7f9c01210740 1 -- 172.21.15.69:0/3164153584 >> 172.21.15.69:0/3164153584 conn(0x55c4a47f0470 msgr2=0x55c4a47fabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:14.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.064+0000 7f9c01210740 1 -- 172.21.15.69:0/3164153584 shutdown_connections 2024-03-19T01:20:14.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.064+0000 7f9c01210740 1 -- 172.21.15.69:0/3164153584 wait complete. 2024-03-19T01:20:14.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.064+0000 7f9c01210740 1 -- start start 2024-03-19T01:20:14.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.064+0000 7f9c01210740 1 Processor -- start 2024-03-19T01:20:14.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.065+0000 7f9c01210740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f5d40 0x55c4a4812710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:14.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.065+0000 7f9c01210740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a4812c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:14.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.065+0000 7f9c01210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a4813190 0x55c4a48748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:14.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.065+0000 7f9c01210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81c7ae0 con 0x55c4a4813190 2024-03-19T01:20:14.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.065+0000 7f9c01210740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81c7410 con 0x55c4a47f5d40 2024-03-19T01:20:14.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.065+0000 7f9c01210740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81c4670 con 0x55c4a47ff240 2024-03-19T01:20:14.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.065+0000 7f9c01210740 10 client.0 _ll_register_callbacks cb 0x7f9be40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:14.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.066+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f5d40 0x55c4a4812710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:14.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.066+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f5d40 0x55c4a4812710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:50526/0 (socket says 172.21.15.69:50526) 2024-03-19T01:20:14.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.066+0000 7f9bf7400640 1 -- 172.21.15.69:0/50792026 learned_addr learned my addr 172.21.15.69:0/50792026 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:14.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.066+0000 7f9bf7e00640 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a4813190 0x55c4a48748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:14.069 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpi-add.o 2024-03-19T01:20:14.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.069+0000 7f9bf7e00640 1 -- 172.21.15.69:0/50792026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 msgr2=0x55c4a4812c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:14.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.068+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a4812c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:14.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.069+0000 7f9bf7e00640 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a4812c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.069+0000 7f9bf7e00640 1 -- 172.21.15.69:0/50792026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f5d40 msgr2=0x55c4a4812710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:14.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.069+0000 7f9bf7e00640 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f5d40 0x55c4a4812710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.069+0000 7f9bf7e00640 1 -- 172.21.15.69:0/50792026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a4874e20 con 0x55c4a4813190 2024-03-19T01:20:14.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.069+0000 7f9bf7400640 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f5d40 0x55c4a4812710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:20:14.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.069+0000 7f9bf6a00640 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a4812c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:14.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.070+0000 7f9bf7e00640 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a4813190 0x55c4a48748e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55c4a47f63a0 tx=0x7f9bf0010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b116aae9ca1758a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:14.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.070+0000 7f9be1600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:20:14.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.074+0000 7f9be1600640 1 -- 172.21.15.69:0/50792026 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf000e900 con 0x55c4a4813190 2024-03-19T01:20:14.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.074+0000 7f9be1600640 1 -- 172.21.15.69:0/50792026 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f9bf000eaa0 con 0x55c4a4813190 2024-03-19T01:20:14.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.076+0000 7f9c01210740 1 -- 172.21.15.69:0/50792026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9be4032340 con 0x55c4a4813190 2024-03-19T01:20:14.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.076+0000 7f9be1600640 1 -- 172.21.15.69:0/50792026 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9bf000eca0 con 0x55c4a4813190 2024-03-19T01:20:14.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.076+0000 7f9c01210740 10 client.17286 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:14.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.076+0000 7f9c01210740 1 -- 172.21.15.69:0/50792026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55c4a4876140 con 0x55c4a4813190 2024-03-19T01:20:14.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.079+0000 7f9be1600640 1 -- 172.21.15.69:0/50792026 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be4032340 con 0x55c4a4813190 2024-03-19T01:20:14.080 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/events/core.o 2024-03-19T01:20:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.080+0000 7f9be1600640 1 -- 172.21.15.69:0/50792026 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f9bf00265c0 con 0x55c4a4813190 2024-03-19T01:20:14.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.083+0000 7f9c01210740 10 client.17286 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:14.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.083+0000 7f9c01210740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:20:14.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.083+0000 7f9c01210740 20 client.17286 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:14.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.083+0000 7f9c01210740 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c4a4876720 0x55c4a4878b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:14.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.083+0000 7f9c01210740 4 client.17286 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:20:14.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.083+0000 7f9c01210740 1 -- 172.21.15.69:0/50792026 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c4a4767800 con 0x55c4a4876720 2024-03-19T01:20:14.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.084+0000 7f9bf7400640 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c4a4876720 0x55c4a4878b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:14.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.085+0000 7f9bf7400640 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c4a4876720 0x55c4a4878b40 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=453a39964e4db4f7 server_cookie=3ab7bf750016f659 in_seq=0 out_seq=0 2024-03-19T01:20:14.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.085+0000 7f9be1600640 10 client.17286 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:14.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.085+0000 7f9be1600640 1 -- 172.21.15.69:0/50792026 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c4a4767800 con 0x55c4a4876720 2024-03-19T01:20:14.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.085+0000 7f9be1600640 10 client.17286 handle_command_reply: tid=0 2024-03-19T01:20:14.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.085+0000 7f9c01210740 1 client.17286 shutdown 2024-03-19T01:20:14.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.086+0000 7f9c01210740 1 -- 172.21.15.69:0/50792026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a4813190 msgr2=0x55c4a48748e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:14.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.086+0000 7f9c01210740 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a4813190 0x55c4a48748e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55c4a47f63a0 tx=0x7f9bf0010b50 comp rx=0 tx=0).stop 2024-03-19T01:20:14.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.087+0000 7f9c01210740 1 -- 172.21.15.69:0/50792026 shutdown_connections 2024-03-19T01:20:14.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.087+0000 7f9c01210740 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55c4a4876720 0x55c4a4878b40 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.087+0000 7f9c01210740 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4a4813190 0x55c4a48748e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.087+0000 7f9c01210740 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4a47ff240 0x55c4a4812c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.087+0000 7f9c01210740 1 --2- 172.21.15.69:0/50792026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a47f5d40 0x55c4a4812710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.087+0000 7f9c01210740 1 -- 172.21.15.69:0/50792026 >> 172.21.15.69:0/50792026 conn(0x55c4a47f0470 msgr2=0x55c4a47eee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:14.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.087+0000 7f9be1600640 0 client.17286 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:14.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.091+0000 7f9c01210740 1 -- 172.21.15.69:0/50792026 shutdown_connections 2024-03-19T01:20:14.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.091+0000 7f9c01210740 1 -- 172.21.15.69:0/50792026 wait complete. 2024-03-19T01:20:14.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.091+0000 7f9c01210740 20 client.17286 trim_cache size 0 max 16384 2024-03-19T01:20:14.097 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:20:14.097 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:20:14.097 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:20:14.097 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:20:14.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.099+0000 7f9bfe600640 1 -- 172.21.15.69:0/3747043725 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bdc0a29f0 msgr2=0x7f9bdc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:14.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.099+0000 7f9bfe600640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bdc0a29f0 0x7f9bdc0a4eb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097d0 tx=0x7f9be4000bc0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.099+0000 7f9bfe600640 1 -- 172.21.15.69:0/3747043725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bf8108a00 msgr2=0x7f9bf8113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:14.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.099+0000 7f9bfe600640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bf8108a00 0x7f9bf8113e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9be800d950 tx=0x7f9be800de10 comp rx=0 tx=0).stop 2024-03-19T01:20:14.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.099+0000 7f9bfe600640 1 -- 172.21.15.69:0/3747043725 shutdown_connections 2024-03-19T01:20:14.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.099+0000 7f9bfe600640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bdc0a29f0 0x7f9bdc0a4eb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.099+0000 7f9bfe600640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bf8114970 0x7f9bf810b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.099+0000 7f9bfe600640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bf8109320 0x7f9bf81143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.099+0000 7f9bfe600640 1 --2- 172.21.15.69:0/3747043725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bf8108a00 0x7f9bf8113e60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.099+0000 7f9bfe600640 1 -- 172.21.15.69:0/3747043725 >> 172.21.15.69:0/3747043725 conn(0x7f9bf806c900 msgr2=0x7f9bf8070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:14.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.099+0000 7f9bfe600640 1 -- 172.21.15.69:0/3747043725 shutdown_connections 2024-03-19T01:20:14.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.100+0000 7f9bfe600640 1 -- 172.21.15.69:0/3747043725 wait complete. 2024-03-19T01:20:14.153 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:20:14.153 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:20:14.398 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:14 smithi069 ceph-mon[31727]: pgmap v296: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 836 KiB/s wr, 70 op/s 2024-03-19T01:20:14.398 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:14 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:20:14.398 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:14 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:20:14.398 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:14 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:20:14.398 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:14 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:20:14.399 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:14 smithi069 ceph-mon[31727]: osdmap e102: 12 total, 12 up, 12 in 2024-03-19T01:20:14.428 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cgroup/pids.o 2024-03-19T01:20:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:14 smithi086 ceph-mon[66829]: pgmap v296: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 836 KiB/s wr, 70 op/s 2024-03-19T01:20:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:14 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:20:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:14 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:20:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:14 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:20:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:14 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:20:14.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:14 smithi086 ceph-mon[66829]: osdmap e102: 12 total, 12 up, 12 in 2024-03-19T01:20:14.555 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/isofs/namei.o 2024-03-19T01:20:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:14 smithi163 ceph-mon[67428]: pgmap v296: 129 pgs: 129 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 836 KiB/s wr, 70 op/s 2024-03-19T01:20:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:14 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:20:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:14 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:20:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:14 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:20:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:14 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:20:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:14 smithi163 ceph-mon[67428]: osdmap e102: 12 total, 12 up, 12 in 2024-03-19T01:20:14.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.642+0000 7f4ef1c00640 1 -- 172.21.15.69:0/2910641605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee40a59a0 msgr2=0x7f4ee40aba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.642+0000 7f4ef1c00640 1 --2- 172.21.15.69:0/2910641605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee40a59a0 0x7f4ee40aba00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009900 tx=0x7f4ed802bcd0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.642+0000 7f4ef1c00640 1 -- 172.21.15.69:0/2910641605 shutdown_connections 2024-03-19T01:20:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.642+0000 7f4ef1c00640 1 --2- 172.21.15.69:0/2910641605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ee40ac130 0x7f4ee40b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.642+0000 7f4ef1c00640 1 --2- 172.21.15.69:0/2910641605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee40a59a0 0x7f4ee40aba00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.642+0000 7f4ef1c00640 1 --2- 172.21.15.69:0/2910641605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee40a4ff0 0x7f4ee40a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.642+0000 7f4ef1c00640 1 -- 172.21.15.69:0/2910641605 >> 172.21.15.69:0/2910641605 conn(0x7f4ee401a670 msgr2=0x7f4ee401aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.642+0000 7f4ef1c00640 1 -- 172.21.15.69:0/2910641605 shutdown_connections 2024-03-19T01:20:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.642+0000 7f4ef1c00640 1 -- 172.21.15.69:0/2910641605 wait complete. 2024-03-19T01:20:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.644+0000 7f4ef1c00640 1 Processor -- start 2024-03-19T01:20:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.648+0000 7f4ef1c00640 1 -- start start 2024-03-19T01:20:14.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.648+0000 7f4ef1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee40a4ff0 0x7f4ee40a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:14.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.648+0000 7f4ef1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee40ac130 0x7f4ee40a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:14.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.648+0000 7f4ef1c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ee40a7270 0x7f4ee414b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:14.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.648+0000 7f4ef1c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40b7290 con 0x7f4ee40a7270 2024-03-19T01:20:14.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.648+0000 7f4ef1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40b7110 con 0x7f4ee40ac130 2024-03-19T01:20:14.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4ef1c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40b7410 con 0x7f4ee40a4ff0 2024-03-19T01:20:14.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eeaa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee40a4ff0 0x7f4ee40a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:14.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eeaa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee40a4ff0 0x7f4ee40a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:40098/0 (socket says 172.21.15.69:40098) 2024-03-19T01:20:14.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eeaa00640 1 -- 172.21.15.69:0/395653284 learned_addr learned my addr 172.21.15.69:0/395653284 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:14.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eeb400640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ee40a7270 0x7f4ee414b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:14.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eea000640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee40ac130 0x7f4ee40a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:14.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eeaa00640 1 -- 172.21.15.69:0/395653284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee40ac130 msgr2=0x7f4ee40a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:14.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eeaa00640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee40ac130 0x7f4ee40a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eeaa00640 1 -- 172.21.15.69:0/395653284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ee40a7270 msgr2=0x7f4ee414b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:14.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eeaa00640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ee40a7270 0x7f4ee414b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eeaa00640 1 -- 172.21.15.69:0/395653284 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee414c0a0 con 0x7f4ee40a4ff0 2024-03-19T01:20:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eea000640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee40ac130 0x7f4ee40a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.649+0000 7f4eeb400640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ee40a7270 0x7f4ee414b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.650+0000 7f4eeaa00640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee40a4ff0 0x7f4ee40a67f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00b450 tx=0x7f4edc00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.652+0000 7f4ee3400640 1 -- 172.21.15.69:0/395653284 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc015020 con 0x7f4ee40a4ff0 2024-03-19T01:20:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.652+0000 7f4ee3400640 1 -- 172.21.15.69:0/395653284 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4edc011d70 con 0x7f4ee40a4ff0 2024-03-19T01:20:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.652+0000 7f4ee3400640 1 -- 172.21.15.69:0/395653284 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00f340 con 0x7f4ee40a4ff0 2024-03-19T01:20:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.652+0000 7f4ef1c00640 1 -- 172.21.15.69:0/395653284 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee414c390 con 0x7f4ee40a4ff0 2024-03-19T01:20:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.652+0000 7f4ef1c00640 1 -- 172.21.15.69:0/395653284 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee414c8e0 con 0x7f4ee40a4ff0 2024-03-19T01:20:14.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.654+0000 7f4ef1c00640 1 -- 172.21.15.69:0/395653284 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee40050c0 con 0x7f4ee40a4ff0 2024-03-19T01:20:14.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.660+0000 7f4ee3400640 1 -- 172.21.15.69:0/395653284 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f4edc018020 con 0x7f4ee40a4ff0 2024-03-19T01:20:14.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.662+0000 7f4ee3400640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4ed00a2b20 0x7f4ed00a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:14.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.662+0000 7f4ee3400640 1 -- 172.21.15.69:0/395653284 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f4edc0cec00 con 0x7f4ee40a4ff0 2024-03-19T01:20:14.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.663+0000 7f4eea000640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4ed00a2b20 0x7f4ed00a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:14.664 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fat/misc.o 2024-03-19T01:20:14.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.666+0000 7f4eea000640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4ed00a2b20 0x7f4ed00a4fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009850 tx=0x7f4ed8000bc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:14.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:14.672+0000 7f4ee3400640 1 -- 172.21.15.69:0/395653284 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4edc01f020 con 0x7f4ee40a4ff0 2024-03-19T01:20:15.119 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/rom.o 2024-03-19T01:20:15.204 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_stat.o 2024-03-19T01:20:15.232 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cgroup/rdma.o 2024-03-19T01:20:15.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.333+0000 7f4ef1c00640 1 -- 172.21.15.69:0/395653284 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ee40a59a0 con 0x7f4ee40a4ff0 2024-03-19T01:20:15.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.334+0000 7f4ee3400640 1 -- 172.21.15.69:0/395653284 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11930 (secure 0 0 0) 0x7f4edc00d040 con 0x7f4ee40a4ff0 2024-03-19T01:20:15.335 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:20:15.335 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":59,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:20:08.600901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:20:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 59 2024-03-19T01:20:15.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.347+0000 7f4ef1c00640 1 -- 172.21.15.69:0/395653284 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4ed00a2b20 msgr2=0x7f4ed00a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:15.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.347+0000 7f4ef1c00640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4ed00a2b20 0x7f4ed00a4fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009850 tx=0x7f4ed8000bc0 comp rx=0 tx=0).stop 2024-03-19T01:20:15.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.347+0000 7f4ef1c00640 1 -- 172.21.15.69:0/395653284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee40a4ff0 msgr2=0x7f4ee40a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:15.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.347+0000 7f4ef1c00640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee40a4ff0 0x7f4ee40a67f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00b450 tx=0x7f4edc00b910 comp rx=0 tx=0).stop 2024-03-19T01:20:15.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.347+0000 7f4ef1c00640 1 -- 172.21.15.69:0/395653284 shutdown_connections 2024-03-19T01:20:15.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.347+0000 7f4ef1c00640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ee40a7270 0x7f4ee414b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:15.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.347+0000 7f4ef1c00640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f4ed00a2b20 0x7f4ed00a4fe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:15.349 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/super.o 2024-03-19T01:20:15.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.347+0000 7f4ef1c00640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee40ac130 0x7f4ee40a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:15.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.347+0000 7f4ef1c00640 1 --2- 172.21.15.69:0/395653284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee40a4ff0 0x7f4ee40a67f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:15.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.347+0000 7f4ef1c00640 1 -- 172.21.15.69:0/395653284 >> 172.21.15.69:0/395653284 conn(0x7f4ee401a670 msgr2=0x7f4ee40a3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:15.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.348+0000 7f4ef1c00640 1 -- 172.21.15.69:0/395653284 shutdown_connections 2024-03-19T01:20:15.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:15.348+0000 7f4ef1c00640 1 -- 172.21.15.69:0/395653284 wait complete. 2024-03-19T01:20:15.406 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:20:15.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:15 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:20:15.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:15 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:20:15.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:15 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:20:15.606 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/isofs/inode.o 2024-03-19T01:20:15.774 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpi-bit.o 2024-03-19T01:20:15.904 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fat/nfs.o 2024-03-19T01:20:16.207 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cgroup/cpuset.o 2024-03-19T01:20:16.236 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_printk.o 2024-03-19T01:20:16.236 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/setup-res.o 2024-03-19T01:20:16.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.232+0000 7f0209400640 1 -- 172.21.15.69:0/1741436474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f020410c2f0 msgr2=0x7f020410c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.232+0000 7f0209400640 1 --2- 172.21.15.69:0/1741436474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f020410c2f0 0x7f020410c6d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009900 tx=0x7f01f402bc80 comp rx=0 tx=0).stop 2024-03-19T01:20:16.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.236+0000 7f0209400640 1 -- 172.21.15.69:0/1741436474 shutdown_connections 2024-03-19T01:20:16.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.236+0000 7f0209400640 1 --2- 172.21.15.69:0/1741436474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02041113c0 0x7f0204118f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.236+0000 7f0209400640 1 --2- 172.21.15.69:0/1741436474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020410cca0 0x7f0204110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.236+0000 7f0209400640 1 --2- 172.21.15.69:0/1741436474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f020410c2f0 0x7f020410c6d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.236+0000 7f0209400640 1 -- 172.21.15.69:0/1741436474 >> 172.21.15.69:0/1741436474 conn(0x7f020406c680 msgr2=0x7f020410a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.237+0000 7f0209400640 1 -- 172.21.15.69:0/1741436474 shutdown_connections 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.237+0000 7f0209400640 1 -- 172.21.15.69:0/1741436474 wait complete. 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.241+0000 7f0209400640 1 Processor -- start 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.241+0000 7f0209400640 1 -- start start 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.241+0000 7f0209400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020410c2f0 0x7f0204117ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.241+0000 7f0209400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02041113c0 0x7f02041181e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.241+0000 7f0209400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020411e2b0 0x7f020411e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.242+0000 7f0203e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020410c2f0 0x7f0204117ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.242+0000 7f0203e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020410c2f0 0x7f0204117ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43688/0 (socket says 172.21.15.69:43688) 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.242+0000 7f0203400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02041113c0 0x7f02041181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.243+0000 7f0203400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02041113c0 0x7f02041181e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38522/0 (socket says 172.21.15.69:38522) 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.243+0000 7f0203e00640 1 -- 172.21.15.69:0/2263654951 learned_addr learned my addr 172.21.15.69:0/2263654951 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.243+0000 7f0209400640 1 -- 172.21.15.69:0/2263654951 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f020406e410 con 0x7f020410c2f0 2024-03-19T01:20:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.243+0000 7f0208a00640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020411e2b0 0x7f020411e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.243+0000 7f0209400640 1 -- 172.21.15.69:0/2263654951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f020406e290 con 0x7f020411e2b0 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.243+0000 7f0209400640 1 -- 172.21.15.69:0/2263654951 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f020406e590 con 0x7f02041113c0 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.244+0000 7f0203400640 1 -- 172.21.15.69:0/2263654951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020411e2b0 msgr2=0x7f020411e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.244+0000 7f0203400640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020411e2b0 0x7f020411e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.244+0000 7f0203400640 1 -- 172.21.15.69:0/2263654951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020410c2f0 msgr2=0x7f0204117ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.244+0000 7f0203400640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020410c2f0 0x7f0204117ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.244+0000 7f0203400640 1 -- 172.21.15.69:0/2263654951 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020411eda0 con 0x7f02041113c0 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.244+0000 7f0208a00640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020411e2b0 0x7f020411e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.245+0000 7f0203e00640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020410c2f0 0x7f0204117ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.245+0000 7f0203400640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02041113c0 0x7f02041181e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f01f004f460 tx=0x7f01f004f920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.245+0000 7f0200c00640 1 -- 172.21.15.69:0/2263654951 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0058070 con 0x7f02041113c0 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.245+0000 7f0200c00640 1 -- 172.21.15.69:0/2263654951 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f0054ce0 con 0x7f02041113c0 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.245+0000 7f0209400640 1 -- 172.21.15.69:0/2263654951 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020411f090 con 0x7f02041113c0 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.245+0000 7f0200c00640 1 -- 172.21.15.69:0/2263654951 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0053350 con 0x7f02041113c0 2024-03-19T01:20:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.246+0000 7f0209400640 1 -- 172.21.15.69:0/2263654951 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041267f0 con 0x7f02041113c0 2024-03-19T01:20:16.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.257+0000 7f020d80e740 1 Processor -- start 2024-03-19T01:20:16.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.257+0000 7f0200c00640 1 -- 172.21.15.69:0/2263654951 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f01f005b070 con 0x7f02041113c0 2024-03-19T01:20:16.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.258+0000 7f020d80e740 1 -- start start 2024-03-19T01:20:16.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.259+0000 7f020d80e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c144aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.259+0000 7f020d80e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c1453bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.259+0000 7f0208a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c144aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.260+0000 7f0208a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c144aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38526/0 (socket says 172.21.15.69:38526) 2024-03-19T01:20:16.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.260+0000 7f0200c00640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01e80a29f0 0x7f01e80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.260+0000 7f0200c00640 1 -- 172.21.15.69:0/2263654951 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f01f01125f0 con 0x7f02041113c0 2024-03-19T01:20:16.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.260+0000 7f0208a00640 1 -- 172.21.15.69:0/3011731654 learned_addr learned my addr 172.21.15.69:0/3011731654 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:16.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.260+0000 7f020d80e740 1 --2- 172.21.15.69:0/3011731654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c1456500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.260+0000 7f020d80e740 1 -- 172.21.15.69:0/3011731654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5634c13af380 con 0x5634c144b530 2024-03-19T01:20:16.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.263+0000 7f0203e00640 1 --2- 172.21.15.69:0/3011731654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c1453bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.263+0000 7f020d80e740 1 -- 172.21.15.69:0/3011731654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634c1357d20 con 0x5634c1454110 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.263+0000 7f020d80e740 1 -- 172.21.15.69:0/3011731654 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5634c13ab420 con 0x5634c144ac10 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.263+0000 7f0203400640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01e80a29f0 0x7f01e80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.263+0000 7f0208a00640 1 --2- 172.21.15.69:0/3011731654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c1456500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.264+0000 7f0208a00640 1 -- 172.21.15.69:0/3011731654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 msgr2=0x5634c1456500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.264+0000 7f0208a00640 1 --2- 172.21.15.69:0/3011731654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c1456500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.264+0000 7f0208a00640 1 -- 172.21.15.69:0/3011731654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 msgr2=0x5634c1453bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.264+0000 7f0208a00640 1 --2- 172.21.15.69:0/3011731654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c1453bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.264+0000 7f0208a00640 1 -- 172.21.15.69:0/3011731654 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c10c33f0 con 0x5634c144ac10 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.264+0000 7f0203e00640 1 --2- 172.21.15.69:0/3011731654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c1453bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.264+0000 7f0208a00640 1 --2- 172.21.15.69:0/3011731654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c144aff0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00bbb0 tx=0x7f01fc009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd205657d78b6056 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.264+0000 7f0203400640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01e80a29f0 0x7f01e80a4eb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f01f004d040 tx=0x7f01f005b800 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.265+0000 7f01f9200640 1 -- 172.21.15.69:0/3011731654 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc01a450 con 0x5634c144ac10 2024-03-19T01:20:16.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.265+0000 7f01f9200640 1 -- 172.21.15.69:0/3011731654 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01fc01aad0 con 0x5634c144ac10 2024-03-19T01:20:16.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.265+0000 7f01f9200640 1 -- 172.21.15.69:0/3011731654 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc018800 con 0x5634c144ac10 2024-03-19T01:20:16.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.266+0000 7f020d80e740 1 -- 172.21.15.69:0/3011731654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 msgr2=0x5634c144aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.266+0000 7f020d80e740 1 --2- 172.21.15.69:0/3011731654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c144aff0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00bbb0 tx=0x7f01fc009630 comp rx=0 tx=0).stop 2024-03-19T01:20:16.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.266+0000 7f020d80e740 1 -- 172.21.15.69:0/3011731654 shutdown_connections 2024-03-19T01:20:16.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.266+0000 7f020d80e740 1 --2- 172.21.15.69:0/3011731654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c1456500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.266+0000 7f020d80e740 1 --2- 172.21.15.69:0/3011731654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c1453bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.266+0000 7f020d80e740 1 --2- 172.21.15.69:0/3011731654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c144aff0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.266+0000 7f020d80e740 1 -- 172.21.15.69:0/3011731654 >> 172.21.15.69:0/3011731654 conn(0x5634c1445340 msgr2=0x5634c1445720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:16.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.266+0000 7f020d80e740 1 -- 172.21.15.69:0/3011731654 shutdown_connections 2024-03-19T01:20:16.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.266+0000 7f020d80e740 1 -- 172.21.15.69:0/3011731654 wait complete. 2024-03-19T01:20:16.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.267+0000 7f020d80e740 1 -- start start 2024-03-19T01:20:16.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.267+0000 7f020d80e740 1 Processor -- start 2024-03-19T01:20:16.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.270+0000 7f020d80e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1451250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.270+0000 7f020d80e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c1451790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f020d80e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c1451cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f020d80e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5634c1456f00 con 0x5634c144b530 2024-03-19T01:20:16.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f020d80e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634c136b160 con 0x5634c1454110 2024-03-19T01:20:16.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f020d80e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5634c1457080 con 0x5634c144ac10 2024-03-19T01:20:16.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f020d80e740 10 client.0 _ll_register_callbacks cb 0x5634c13a7170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:16.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f0208a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1451250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f0208a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1451250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38542/0 (socket says 172.21.15.69:38542) 2024-03-19T01:20:16.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f0208a00640 1 -- 172.21.15.69:0/3642057498 learned_addr learned my addr 172.21.15.69:0/3642057498 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:16.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f0208a00640 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c1451cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f0203e00640 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c1451790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f0208a00640 1 -- 172.21.15.69:0/3642057498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 msgr2=0x5634c1451cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f0208a00640 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c1451cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f0208a00640 1 -- 172.21.15.69:0/3642057498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 msgr2=0x5634c1451790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.271+0000 7f0208a00640 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c1451790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.272+0000 7f0208a00640 1 -- 172.21.15.69:0/3642057498 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c118d7d0 con 0x5634c144ac10 2024-03-19T01:20:16.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.272+0000 7f0203e00640 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c1451790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:20:16.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.272+0000 7f0208a00640 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1451250 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x5634c1449e60 tx=0x7f01fc009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6231660ba982157 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.272+0000 7f01ed600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:20:16.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.273+0000 7f01ed600640 1 -- 172.21.15.69:0/3642057498 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc02d060 con 0x5634c144ac10 2024-03-19T01:20:16.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.274+0000 7f01ed600640 1 -- 172.21.15.69:0/3642057498 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f01fc022040 con 0x5634c144ac10 2024-03-19T01:20:16.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.289+0000 7f020d80e740 1 -- 172.21.15.69:0/3642057498 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01fc01aad0 con 0x5634c144ac10 2024-03-19T01:20:16.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.289+0000 7f01ed600640 1 -- 172.21.15.69:0/3642057498 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f01fc018d20 con 0x5634c144ac10 2024-03-19T01:20:16.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.289+0000 7f020d80e740 10 client.27577 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:16.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.289+0000 7f020d80e740 1 -- 172.21.15.69:0/3642057498 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5634c14630a0 con 0x5634c144ac10 2024-03-19T01:20:16.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.290+0000 7f01ed600640 1 -- 172.21.15.69:0/3642057498 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01fc01aad0 con 0x5634c144ac10 2024-03-19T01:20:16.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.290+0000 7f01ed600640 1 -- 172.21.15.69:0/3642057498 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f01fc0313a0 con 0x5634c144ac10 2024-03-19T01:20:16.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.292+0000 7f020d80e740 10 client.27577 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:16.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.292+0000 7f020d80e740 10 client.27577 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:20:16.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.292+0000 7f020d80e740 20 client.27577 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:16.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.292+0000 7f020d80e740 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5634c14644b0 0x5634c14668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.292+0000 7f020d80e740 4 client.27577 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:20:16.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.292+0000 7f020d80e740 1 -- 172.21.15.69:0/3642057498 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634c13bc8c0 con 0x5634c14644b0 2024-03-19T01:20:16.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.292+0000 7f0203e00640 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5634c14644b0 0x5634c14668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.293+0000 7f01ed600640 10 client.27577 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:20:16.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.294+0000 7f0203e00640 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5634c14644b0 0x5634c14668a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1bc5e47ecb2167da server_cookie=c678daa4b6d4bd1f in_seq=0 out_seq=0 2024-03-19T01:20:16.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.300+0000 7f01ed600640 1 -- 172.21.15.69:0/3642057498 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5634c13bc8c0 con 0x5634c14644b0 2024-03-19T01:20:16.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.300+0000 7f01ed600640 10 client.27577 handle_command_reply: tid=0 2024-03-19T01:20:16.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.300+0000 7f020d80e740 1 client.27577 shutdown 2024-03-19T01:20:16.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.301+0000 7f020d80e740 1 -- 172.21.15.69:0/3642057498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 msgr2=0x5634c1451250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.301+0000 7f020d80e740 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1451250 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x5634c1449e60 tx=0x7f01fc009f60 comp rx=0 tx=0).stop 2024-03-19T01:20:16.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.301+0000 7f020d80e740 1 -- 172.21.15.69:0/3642057498 shutdown_connections 2024-03-19T01:20:16.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.301+0000 7f020d80e740 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5634c14644b0 0x5634c14668a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.301+0000 7f020d80e740 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c1451cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.301+0000 7f020d80e740 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c1451790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.301+0000 7f020d80e740 1 --2- 172.21.15.69:0/3642057498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1451250 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.301+0000 7f020d80e740 1 -- 172.21.15.69:0/3642057498 >> 172.21.15.69:0/3642057498 conn(0x5634c1445340 msgr2=0x5634c1447090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:16.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.301+0000 7f01ed600640 0 client.27577 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:20:16.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.302+0000 7f020d80e740 1 -- 172.21.15.69:0/3642057498 shutdown_connections 2024-03-19T01:20:16.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.302+0000 7f020d80e740 1 -- 172.21.15.69:0/3642057498 wait complete. 2024-03-19T01:20:16.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.302+0000 7f020d80e740 20 client.27577 trim_cache size 0 max 16384 2024-03-19T01:20:16.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.356+0000 7f020d80e740 1 Processor -- start 2024-03-19T01:20:16.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.360+0000 7f020d80e740 1 -- start start 2024-03-19T01:20:16.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.360+0000 7f020d80e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1453980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.361+0000 7f0208a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1453980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.361+0000 7f0208a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1453980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38558/0 (socket says 172.21.15.69:38558) 2024-03-19T01:20:16.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.361+0000 7f020d80e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c144e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.361+0000 7f0208a00640 1 -- 172.21.15.69:0/4069581921 learned_addr learned my addr 172.21.15.69:0/4069581921 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:16.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.361+0000 7f020d80e740 1 --2- 172.21.15.69:0/4069581921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c144ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.363+0000 7f020d80e740 1 -- 172.21.15.69:0/4069581921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02041305b0 con 0x5634c144b530 2024-03-19T01:20:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.363+0000 7f020d80e740 1 -- 172.21.15.69:0/4069581921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02041320f0 con 0x5634c1454110 2024-03-19T01:20:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.363+0000 7f020d80e740 1 -- 172.21.15.69:0/4069581921 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0204130c80 con 0x5634c144ac10 2024-03-19T01:20:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.362+0000 7f0203e00640 1 --2- 172.21.15.69:0/4069581921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c144e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.364+0000 7f0208a00640 1 --2- 172.21.15.69:0/4069581921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c144ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.364+0000 7f0208a00640 1 -- 172.21.15.69:0/4069581921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 msgr2=0x5634c144ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.364+0000 7f0208a00640 1 --2- 172.21.15.69:0/4069581921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c144ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.364+0000 7f0208a00640 1 -- 172.21.15.69:0/4069581921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 msgr2=0x5634c144e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.364+0000 7f0208a00640 1 --2- 172.21.15.69:0/4069581921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c144e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.364+0000 7f0208a00640 1 -- 172.21.15.69:0/4069581921 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c144f7c0 con 0x5634c144ac10 2024-03-19T01:20:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.364+0000 7f0208a00640 1 --2- 172.21.15.69:0/4069581921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1453980 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f01fc01a7e0 tx=0x7f01fc009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97c7bb20147fde7b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.364+0000 7f01efe00640 1 -- 172.21.15.69:0/4069581921 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc02d040 con 0x5634c144ac10 2024-03-19T01:20:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.364+0000 7f01efe00640 1 -- 172.21.15.69:0/4069581921 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01fc01aa90 con 0x5634c144ac10 2024-03-19T01:20:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.364+0000 7f01efe00640 1 -- 172.21.15.69:0/4069581921 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc037080 con 0x5634c144ac10 2024-03-19T01:20:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.365+0000 7f020d80e740 1 -- 172.21.15.69:0/4069581921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 msgr2=0x5634c1453980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.365+0000 7f020d80e740 1 --2- 172.21.15.69:0/4069581921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1453980 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f01fc01a7e0 tx=0x7f01fc009630 comp rx=0 tx=0).stop 2024-03-19T01:20:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.366+0000 7f020d80e740 1 -- 172.21.15.69:0/4069581921 shutdown_connections 2024-03-19T01:20:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.366+0000 7f020d80e740 1 --2- 172.21.15.69:0/4069581921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c1454110 0x5634c144ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.366+0000 7f020d80e740 1 --2- 172.21.15.69:0/4069581921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c144b530 0x5634c144e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.366+0000 7f020d80e740 1 --2- 172.21.15.69:0/4069581921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144ac10 0x5634c1453980 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.366+0000 7f020d80e740 1 -- 172.21.15.69:0/4069581921 >> 172.21.15.69:0/4069581921 conn(0x5634c1445340 msgr2=0x5634c144c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.366+0000 7f020d80e740 1 -- 172.21.15.69:0/4069581921 shutdown_connections 2024-03-19T01:20:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.366+0000 7f020d80e740 1 -- 172.21.15.69:0/4069581921 wait complete. 2024-03-19T01:20:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.367+0000 7f020d80e740 1 -- start start 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.367+0000 7f020d80e740 1 Processor -- start 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.369+0000 7f020d80e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c144ac10 0x5634c14698d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.369+0000 7f020d80e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144b530 0x5634c1466c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.369+0000 7f020d80e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c1454110 0x5634c14673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.369+0000 7f020d80e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0204131a20 con 0x5634c1454110 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.369+0000 7f020d80e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0204131350 con 0x5634c144ac10 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.369+0000 7f020d80e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f020412e590 con 0x5634c144b530 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.369+0000 7f020d80e740 10 client.0 _ll_register_callbacks cb 0x7f01f4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.370+0000 7f0208a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c1454110 0x5634c14673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.371+0000 7f0208a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c1454110 0x5634c14673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43730/0 (socket says 172.21.15.69:43730) 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.371+0000 7f0208a00640 1 -- 172.21.15.69:0/32077745 learned_addr learned my addr 172.21.15.69:0/32077745 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.371+0000 7f0203e00640 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144b530 0x5634c1466c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.371+0000 7f0203e00640 1 -- 172.21.15.69:0/32077745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c144ac10 msgr2=0x5634c14698d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:20:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.371+0000 7f0203e00640 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c144ac10 0x5634c14698d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.371+0000 7f0203e00640 1 -- 172.21.15.69:0/32077745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c1454110 msgr2=0x5634c14673a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.372+0000 7f0203e00640 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c1454110 0x5634c14673a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.372+0000 7f0203e00640 1 -- 172.21.15.69:0/32077745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c14c9450 con 0x5634c144b530 2024-03-19T01:20:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.372+0000 7f0208a00640 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c1454110 0x5634c14673a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:20:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.372+0000 7f0203e00640 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144b530 0x5634c1466c60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f01f4004f50 tx=0x7f01f4005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d14959b6081f77c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.373+0000 7f01ed600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:20:16.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.373+0000 7f01ed600640 1 -- 172.21.15.69:0/32077745 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f404f070 con 0x5634c144b530 2024-03-19T01:20:16.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.373+0000 7f01ed600640 1 -- 172.21.15.69:0/32077745 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f01f40040c0 con 0x5634c144b530 2024-03-19T01:20:16.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.373+0000 7f020d80e740 1 -- 172.21.15.69:0/32077745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01fc01aa90 con 0x5634c144b530 2024-03-19T01:20:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.374+0000 7f01ed600640 1 -- 172.21.15.69:0/32077745 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f01f4059050 con 0x5634c144b530 2024-03-19T01:20:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.376+0000 7f020d80e740 10 client.27583 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.376+0000 7f020d80e740 1 -- 172.21.15.69:0/32077745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5634c14ec5f0 con 0x5634c144b530 2024-03-19T01:20:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.377+0000 7f01ed600640 1 -- 172.21.15.69:0/32077745 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f40042c0 con 0x5634c144b530 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.378+0000 7f01ed600640 1 -- 172.21.15.69:0/32077745 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f01f4048070 con 0x5634c144b530 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.378+0000 7f020d80e740 10 client.27583 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.378+0000 7f020d80e740 10 client.27583 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.378+0000 7f020d80e740 20 client.27583 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.378+0000 7f020d80e740 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5634c14ecd20 0x5634c14ef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.378+0000 7f020d80e740 4 client.27583 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.378+0000 7f020d80e740 1 -- 172.21.15.69:0/32077745 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f01fc01aa90 con 0x5634c14ecd20 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.379+0000 7f0208a00640 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5634c14ecd20 0x5634c14ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.380+0000 7f0208a00640 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5634c14ecd20 0x5634c14ef140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a31993f13c1bcb27 server_cookie=77dcff883ca67a27 in_seq=0 out_seq=0 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.380+0000 7f01ed600640 10 client.27583 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f01ed600640 1 -- 172.21.15.69:0/32077745 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f01fc01aa90 con 0x5634c14ecd20 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f01ed600640 10 client.27583 handle_command_reply: tid=0 2024-03-19T01:20:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f020d80e740 1 client.27583 shutdown 2024-03-19T01:20:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f020d80e740 1 -- 172.21.15.69:0/32077745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144b530 msgr2=0x5634c1466c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f020d80e740 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144b530 0x5634c1466c60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f01f4004f50 tx=0x7f01f4005a20 comp rx=0 tx=0).stop 2024-03-19T01:20:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f020d80e740 1 -- 172.21.15.69:0/32077745 shutdown_connections 2024-03-19T01:20:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f020d80e740 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5634c14ecd20 0x5634c14ef140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f020d80e740 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c1454110 0x5634c14673a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f020d80e740 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5634c144b530 0x5634c1466c60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f020d80e740 1 --2- 172.21.15.69:0/32077745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634c144ac10 0x5634c14698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f020d80e740 1 -- 172.21.15.69:0/32077745 >> 172.21.15.69:0/32077745 conn(0x5634c1445340 msgr2=0x5634c14511f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.383+0000 7f01ed600640 0 client.27583 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:20:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.385+0000 7f020d80e740 1 -- 172.21.15.69:0/32077745 shutdown_connections 2024-03-19T01:20:16.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.385+0000 7f020d80e740 1 -- 172.21.15.69:0/32077745 wait complete. 2024-03-19T01:20:16.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.387+0000 7f020d80e740 20 client.27583 trim_cache size 0 max 16384 2024-03-19T01:20:16.388 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:20:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.389+0000 7f0209400640 1 -- 172.21.15.69:0/2263654951 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01e80a29f0 msgr2=0x7f01e80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.389+0000 7f0209400640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01e80a29f0 0x7f01e80a4eb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f01f004d040 tx=0x7f01f005b800 comp rx=0 tx=0).stop 2024-03-19T01:20:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.389+0000 7f0209400640 1 -- 172.21.15.69:0/2263654951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02041113c0 msgr2=0x7f02041181e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.389+0000 7f0209400640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02041113c0 0x7f02041181e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f01f004f460 tx=0x7f01f004f920 comp rx=0 tx=0).stop 2024-03-19T01:20:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.390+0000 7f0209400640 1 -- 172.21.15.69:0/2263654951 shutdown_connections 2024-03-19T01:20:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.390+0000 7f0209400640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f01e80a29f0 0x7f01e80a4eb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.390+0000 7f0209400640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020411e2b0 0x7f020411e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.390+0000 7f0209400640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02041113c0 0x7f02041181e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.390+0000 7f0209400640 1 --2- 172.21.15.69:0/2263654951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f020410c2f0 0x7f0204117ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.390+0000 7f0209400640 1 -- 172.21.15.69:0/2263654951 >> 172.21.15.69:0/2263654951 conn(0x7f020406c680 msgr2=0x7f0204070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.392+0000 7f0209400640 1 -- 172.21.15.69:0/2263654951 shutdown_connections 2024-03-19T01:20:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.392+0000 7f0209400640 1 -- 172.21.15.69:0/2263654951 wait complete. 2024-03-19T01:20:16.444 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:20:16.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:16 smithi086 ceph-mon[66829]: pgmap v298: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-03-19T01:20:16.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:16 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/395653284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:20:16.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:16 smithi069 ceph-mon[31727]: pgmap v298: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-03-19T01:20:16.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:16 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/395653284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:20:16.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:16 smithi163 ceph-mon[67428]: pgmap v298: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-03-19T01:20:16.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:16 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/395653284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:20:16.670 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpi-cmp.o 2024-03-19T01:20:16.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.864+0000 7f9fd9400640 1 -- 172.21.15.69:0/2136765097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd411b000 msgr2=0x7f9fd411d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.864+0000 7f9fd9400640 1 --2- 172.21.15.69:0/2136765097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd411b000 0x7f9fd411d3f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc01acb0 tx=0x7f9fcc03d170 comp rx=0 tx=0).stop 2024-03-19T01:20:16.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.866+0000 7f9fd9400640 1 -- 172.21.15.69:0/2136765097 shutdown_connections 2024-03-19T01:20:16.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.866+0000 7f9fd9400640 1 --2- 172.21.15.69:0/2136765097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd411b000 0x7f9fd411d3f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.866+0000 7f9fd9400640 1 --2- 172.21.15.69:0/2136765097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd41088b0 0x7f9fd4108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.866+0000 7f9fd9400640 1 --2- 172.21.15.69:0/2136765097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fd410fe40 0x7f9fd4110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.866+0000 7f9fd9400640 1 -- 172.21.15.69:0/2136765097 >> 172.21.15.69:0/2136765097 conn(0x7f9fd406c770 msgr2=0x7f9fd406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:16.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.866+0000 7f9fd9400640 1 -- 172.21.15.69:0/2136765097 shutdown_connections 2024-03-19T01:20:16.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.866+0000 7f9fd9400640 1 -- 172.21.15.69:0/2136765097 wait complete. 2024-03-19T01:20:16.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.867+0000 7f9fd9400640 1 Processor -- start 2024-03-19T01:20:16.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.871+0000 7f9fd9400640 1 -- start start 2024-03-19T01:20:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.871+0000 7f9fd9400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fd41088b0 0x7f9fd411a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.871+0000 7f9fd9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd410fe40 0x7f9fd41168b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.871+0000 7f9fd9400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd411b000 0x7f9fd4116df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.871+0000 7f9fd9400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4113250 con 0x7f9fd41088b0 2024-03-19T01:20:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.871+0000 7f9fd9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41130d0 con 0x7f9fd410fe40 2024-03-19T01:20:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.871+0000 7f9fd9400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41133d0 con 0x7f9fd411b000 2024-03-19T01:20:16.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.872+0000 7f9fd8a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd411b000 0x7f9fd4116df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.872+0000 7f9fd8a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd411b000 0x7f9fd4116df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38588/0 (socket says 172.21.15.69:38588) 2024-03-19T01:20:16.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.872+0000 7f9fd8a00640 1 -- 172.21.15.69:0/1421422269 learned_addr learned my addr 172.21.15.69:0/1421422269 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:16.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.872+0000 7f9fd3400640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd410fe40 0x7f9fd41168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.872+0000 7f9fd8a00640 1 -- 172.21.15.69:0/1421422269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd410fe40 msgr2=0x7f9fd41168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.872+0000 7f9fd8a00640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd410fe40 0x7f9fd41168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.872+0000 7f9fd8a00640 1 -- 172.21.15.69:0/1421422269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fd41088b0 msgr2=0x7f9fd411a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.872+0000 7f9fd8a00640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fd41088b0 0x7f9fd411a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.872+0000 7f9fd8a00640 1 -- 172.21.15.69:0/1421422269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd4117570 con 0x7f9fd411b000 2024-03-19T01:20:16.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.873+0000 7f9fd8a00640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd411b000 0x7f9fd4116df0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0433b0 tx=0x7f9fcc003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.874+0000 7f9fd0c00640 1 -- 172.21.15.69:0/1421422269 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc052020 con 0x7f9fd411b000 2024-03-19T01:20:16.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.874+0000 7f9fd0c00640 1 -- 172.21.15.69:0/1421422269 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fcc0438c0 con 0x7f9fd411b000 2024-03-19T01:20:16.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.874+0000 7f9fd9400640 1 -- 172.21.15.69:0/1421422269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd41aac50 con 0x7f9fd411b000 2024-03-19T01:20:16.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.874+0000 7f9fd9400640 1 -- 172.21.15.69:0/1421422269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd41ab050 con 0x7f9fd411b000 2024-03-19T01:20:16.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.874+0000 7f9fd0c00640 1 -- 172.21.15.69:0/1421422269 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc04a040 con 0x7f9fd411b000 2024-03-19T01:20:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.881+0000 7f9fd0c00640 1 -- 172.21.15.69:0/1421422269 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9fcc055020 con 0x7f9fd411b000 2024-03-19T01:20:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.882+0000 7f9fdd7e7740 1 Processor -- start 2024-03-19T01:20:16.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.886+0000 7f9fdd7e7740 1 -- start start 2024-03-19T01:20:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.886+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba8fc10 0x55f9aba8fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.886+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba90530 0x55f9aba98bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.886+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba99110 0x55f9aba9b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.886+0000 7f9fdd7e7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f9ab9f4380 con 0x55f9aba8fc10 2024-03-19T01:20:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.886+0000 7f9fdd7e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9ab99cd20 con 0x55f9aba90530 2024-03-19T01:20:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.886+0000 7f9fdd7e7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f9ab9f0420 con 0x55f9aba99110 2024-03-19T01:20:16.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.887+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba90530 0x55f9aba98bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.887+0000 7f9fd0c00640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9fb80a2ac0 0x7f9fb80a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.887+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba90530 0x55f9aba98bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60574/0 (socket says 172.21.15.69:60574) 2024-03-19T01:20:16.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.887+0000 7f9fd3e00640 1 -- 172.21.15.69:0/88226429 learned_addr learned my addr 172.21.15.69:0/88226429 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:16.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.887+0000 7f9fd3e00640 1 -- 172.21.15.69:0/88226429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba99110 msgr2=0x55f9aba9b500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:16.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.887+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/88226429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba99110 0x55f9aba9b500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.887+0000 7f9fd8a00640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9fb80a2ac0 0x7f9fb80a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.887+0000 7f9fd0c00640 1 -- 172.21.15.69:0/1421422269 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f9fcc101e90 con 0x7f9fd411b000 2024-03-19T01:20:16.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.887+0000 7f9fd3e00640 1 -- 172.21.15.69:0/88226429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba8fc10 msgr2=0x55f9aba8fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.888+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/88226429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba8fc10 0x55f9aba8fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.888+0000 7f9fd3e00640 1 -- 172.21.15.69:0/88226429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9ab7083f0 con 0x55f9aba90530 2024-03-19T01:20:16.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.888+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/88226429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba90530 0x55f9aba98bd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40097c0 tx=0x7f9fc4009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0f9d4265fca7de3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.888+0000 7f9fd8a00640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9fb80a2ac0 0x7f9fb80a4f80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0ca650 tx=0x7f9fcc059040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.888+0000 7f9fc9200640 1 -- 172.21.15.69:0/88226429 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4015020 con 0x55f9aba90530 2024-03-19T01:20:16.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.889+0000 7f9fc9200640 1 -- 172.21.15.69:0/88226429 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc4011420 con 0x55f9aba90530 2024-03-19T01:20:16.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.889+0000 7f9fc9200640 1 -- 172.21.15.69:0/88226429 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40105a0 con 0x55f9aba90530 2024-03-19T01:20:16.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.891+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/88226429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba90530 msgr2=0x55f9aba98bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.891+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/88226429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba90530 0x55f9aba98bd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40097c0 tx=0x7f9fc4009c80 comp rx=0 tx=0).stop 2024-03-19T01:20:16.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.893+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/88226429 shutdown_connections 2024-03-19T01:20:16.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.893+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/88226429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba99110 0x55f9aba9b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.893+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/88226429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba90530 0x55f9aba98bd0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.893+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/88226429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba8fc10 0x55f9aba8fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.893+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/88226429 >> 172.21.15.69:0/88226429 conn(0x55f9aba8a340 msgr2=0x55f9aba8a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:16.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.896+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/88226429 shutdown_connections 2024-03-19T01:20:16.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.896+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/88226429 wait complete. 2024-03-19T01:20:16.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.896+0000 7f9fdd7e7740 1 -- start start 2024-03-19T01:20:16.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.897+0000 7f9fdd7e7740 1 Processor -- start 2024-03-19T01:20:16.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.900+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba8fc10 0x55f9aba96250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.900+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 0x55f9aba96790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.900+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba99110 0x55f9aba96cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.900+0000 7f9fdd7e7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f9aba9bf00 con 0x55f9aba8fc10 2024-03-19T01:20:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.900+0000 7f9fdd7e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9ab9b0160 con 0x55f9aba99110 2024-03-19T01:20:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.900+0000 7f9fdd7e7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f9aba9c080 con 0x55f9aba90530 2024-03-19T01:20:16.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.900+0000 7f9fdd7e7740 10 client.0 _ll_register_callbacks cb 0x55f9ab9ec170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.901+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 0x55f9aba96790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.901+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 0x55f9aba96790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38602/0 (socket says 172.21.15.69:38602) 2024-03-19T01:20:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.901+0000 7f9fd3e00640 1 -- 172.21.15.69:0/2097194723 learned_addr learned my addr 172.21.15.69:0/2097194723 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.901+0000 7f9fd3400640 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba8fc10 0x55f9aba96250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.901+0000 7f9fd3e00640 1 -- 172.21.15.69:0/2097194723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba99110 msgr2=0x55f9aba96cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.901+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba99110 0x55f9aba96cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.901+0000 7f9fd3e00640 1 -- 172.21.15.69:0/2097194723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba8fc10 msgr2=0x55f9aba96250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.901+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba8fc10 0x55f9aba96250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.901+0000 7f9fd3e00640 1 -- 172.21.15.69:0/2097194723 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9ab7d27d0 con 0x55f9aba90530 2024-03-19T01:20:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.901+0000 7f9fd3400640 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba8fc10 0x55f9aba96250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:16.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.902+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 0x55f9aba96790 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4011c80 tx=0x7f9fc4011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80f9f5b8b100c0ac server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.903+0000 7f9fc1600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:20:16.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.903+0000 7f9fc1600640 1 -- 172.21.15.69:0/2097194723 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4020070 con 0x55f9aba90530 2024-03-19T01:20:16.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.903+0000 7f9fc1600640 1 -- 172.21.15.69:0/2097194723 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f9fc400bce0 con 0x55f9aba90530 2024-03-19T01:20:16.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.903+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/2097194723 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc4011420 con 0x55f9aba90530 2024-03-19T01:20:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.903+0000 7f9fc1600640 1 -- 172.21.15.69:0/2097194723 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9fc401a420 con 0x55f9aba90530 2024-03-19T01:20:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.904+0000 7f9fdd7e7740 10 client.27589 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.904+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/2097194723 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55f9abaa8370 con 0x55f9aba90530 2024-03-19T01:20:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.906+0000 7f9fc1600640 1 -- 172.21.15.69:0/2097194723 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc4011420 con 0x55f9aba90530 2024-03-19T01:20:16.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.906+0000 7f9fc1600640 1 -- 172.21.15.69:0/2097194723 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f9fc402e480 con 0x55f9aba90530 2024-03-19T01:20:16.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.906+0000 7f9fdd7e7740 10 client.27589 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.906+0000 7f9fdd7e7740 10 client.27589 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:20:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.906+0000 7f9fdd7e7740 20 client.27589 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.906+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f9abaa97d0 0x55f9abaabbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.906+0000 7f9fdd7e7740 4 client.27589 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:20:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.906+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/2097194723 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9aba018c0 con 0x55f9abaa97d0 2024-03-19T01:20:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.906+0000 7f9fd3400640 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f9abaa97d0 0x55f9abaabbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.907+0000 7f9fc1600640 10 client.27589 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:20:16.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.907+0000 7f9fd3400640 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f9abaa97d0 0x55f9abaabbc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=61168fd79c2286a0 server_cookie=7e21316cf1e4da21 in_seq=0 out_seq=0 2024-03-19T01:20:16.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.908+0000 7f9fc1600640 1 -- 172.21.15.69:0/2097194723 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f9aba018c0 con 0x55f9abaa97d0 2024-03-19T01:20:16.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.908+0000 7f9fc1600640 10 client.27589 handle_command_reply: tid=0 2024-03-19T01:20:16.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.908+0000 7f9fdd7e7740 1 client.27589 shutdown 2024-03-19T01:20:16.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.916+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/2097194723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 msgr2=0x55f9aba96790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.916+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 0x55f9aba96790 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4011c80 tx=0x7f9fc4011800 comp rx=0 tx=0).stop 2024-03-19T01:20:16.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.920+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/2097194723 shutdown_connections 2024-03-19T01:20:16.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.920+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f9abaa97d0 0x55f9abaabbc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.920+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba99110 0x55f9aba96cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.920+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 0x55f9aba96790 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.920+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/2097194723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba8fc10 0x55f9aba96250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.920+0000 7f9fc1600640 0 client.27589 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:20:16.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.920+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/2097194723 >> 172.21.15.69:0/2097194723 conn(0x55f9aba8a340 msgr2=0x55f9aba8c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.920+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/2097194723 shutdown_connections 2024-03-19T01:20:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.923+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/2097194723 wait complete. 2024-03-19T01:20:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.923+0000 7f9fdd7e7740 20 client.27589 trim_cache size 0 max 16384 2024-03-19T01:20:16.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.975+0000 7f9fdd7e7740 1 Processor -- start 2024-03-19T01:20:16.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.975+0000 7f9fdd7e7740 1 -- start start 2024-03-19T01:20:16.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.975+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba8fc10 0x55f9aba98980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.975+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba90530 0x55f9abaa4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.975+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba99110 0x55f9abab1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.975+0000 7f9fdd7e7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb0010140 con 0x55f9aba90530 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.975+0000 7f9fdd7e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb0011c80 con 0x55f9aba8fc10 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.975+0000 7f9fdd7e7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb0010810 con 0x55f9aba99110 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.976+0000 7f9fd3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba8fc10 0x55f9aba98980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.976+0000 7f9fd3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba8fc10 0x55f9aba98980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60588/0 (socket says 172.21.15.69:60588) 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.976+0000 7f9fd3400640 1 -- 172.21.15.69:0/3912898261 learned_addr learned my addr 172.21.15.69:0/3912898261 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.976+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/3912898261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba90530 0x55f9abaa4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.976+0000 7f9fd3400640 1 --2- 172.21.15.69:0/3912898261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba99110 0x55f9abab1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.976+0000 7f9fd3400640 1 -- 172.21.15.69:0/3912898261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba99110 msgr2=0x55f9abab1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.976+0000 7f9fd3400640 1 --2- 172.21.15.69:0/3912898261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba99110 0x55f9abab1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.976+0000 7f9fd3400640 1 -- 172.21.15.69:0/3912898261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba90530 msgr2=0x55f9abaa4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.976+0000 7f9fd3400640 1 --2- 172.21.15.69:0/3912898261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba90530 0x55f9abaa4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.977+0000 7f9fd3400640 1 -- 172.21.15.69:0/3912898261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9aba9b830 con 0x55f9aba8fc10 2024-03-19T01:20:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.977+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/3912898261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba90530 0x55f9abaa4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:20:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.977+0000 7f9fd3400640 1 --2- 172.21.15.69:0/3912898261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba8fc10 0x55f9aba98980 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4118020 tx=0x7f9fbc0090e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=173249627016f76b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.978+0000 7f9fc3e00640 1 -- 172.21.15.69:0/3912898261 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc00f040 con 0x55f9aba8fc10 2024-03-19T01:20:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.978+0000 7f9fc3e00640 1 -- 172.21.15.69:0/3912898261 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fbc01a810 con 0x55f9aba8fc10 2024-03-19T01:20:16.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.978+0000 7f9fc3e00640 1 -- 172.21.15.69:0/3912898261 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc016a60 con 0x55f9aba8fc10 2024-03-19T01:20:16.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.981+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/3912898261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba8fc10 msgr2=0x55f9aba98980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:16.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.981+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/3912898261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba8fc10 0x55f9aba98980 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4118020 tx=0x7f9fbc0090e0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.982+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/3912898261 shutdown_connections 2024-03-19T01:20:16.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.982+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/3912898261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba99110 0x55f9abab1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.982+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/3912898261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba90530 0x55f9abaa4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.982+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/3912898261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba8fc10 0x55f9aba98980 secure :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4118020 tx=0x7f9fbc0090e0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.982+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/3912898261 >> 172.21.15.69:0/3912898261 conn(0x55f9aba8a340 msgr2=0x55f9aba853b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:16.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.982+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/3912898261 shutdown_connections 2024-03-19T01:20:16.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.982+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/3912898261 wait complete. 2024-03-19T01:20:16.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.983+0000 7f9fdd7e7740 1 -- start start 2024-03-19T01:20:16.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.983+0000 7f9fdd7e7740 1 Processor -- start 2024-03-19T01:20:16.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.985+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 0x55f9aba9ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.985+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba99110 0x55f9aba9d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.985+0000 7f9fd3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 0x55f9aba9ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.985+0000 7f9fd3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 0x55f9aba9ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38626/0 (socket says 172.21.15.69:38626) 2024-03-19T01:20:16.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.986+0000 7f9fdd7e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba9d930 0x55f9abb0e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:16.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.986+0000 7f9fd3400640 1 -- 172.21.15.69:0/1997273537 learned_addr learned my addr 172.21.15.69:0/1997273537 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:16.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.986+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba99110 0x55f9aba9d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:16.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.986+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/1997273537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb00115b0 con 0x55f9aba99110 2024-03-19T01:20:16.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.986+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/1997273537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb0010ee0 con 0x55f9aba9d930 2024-03-19T01:20:16.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.986+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/1997273537 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb000e120 con 0x55f9aba90530 2024-03-19T01:20:16.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.987+0000 7f9fdd7e7740 10 client.0 _ll_register_callbacks cb 0x7f9fc4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:16.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.987+0000 7f9fd3e00640 1 -- 172.21.15.69:0/1997273537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 msgr2=0x55f9aba9ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.987+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 0x55f9aba9ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.987+0000 7f9fd3e00640 1 -- 172.21.15.69:0/1997273537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba9d930 msgr2=0x55f9abb0e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:16.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.987+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba9d930 0x55f9abb0e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:16.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.987+0000 7f9fd3e00640 1 -- 172.21.15.69:0/1997273537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9abb0f040 con 0x55f9aba99110 2024-03-19T01:20:16.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.987+0000 7f9fd3e00640 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba99110 0x55f9aba9d3f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9fc402e880 tx=0x7f9fc40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2fa8c5b3273c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:16.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.987+0000 7f9fc1600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:20:16.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.988+0000 7f9fc1600640 1 -- 172.21.15.69:0/1997273537 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4033060 con 0x55f9aba99110 2024-03-19T01:20:16.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.988+0000 7f9fc1600640 1 -- 172.21.15.69:0/1997273537 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f9fc400bce0 con 0x55f9aba99110 2024-03-19T01:20:16.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.992+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/1997273537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fbc01a810 con 0x55f9aba99110 2024-03-19T01:20:16.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.994+0000 7f9fc1600640 1 -- 172.21.15.69:0/1997273537 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9fc4020040 con 0x55f9aba99110 2024-03-19T01:20:16.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.994+0000 7f9fdd7e7740 10 client.17307 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:16.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.994+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/1997273537 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55f9abb102e0 con 0x55f9aba99110 2024-03-19T01:20:16.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.998+0000 7f9fc1600640 1 -- 172.21.15.69:0/1997273537 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fbc01a810 con 0x55f9aba99110 2024-03-19T01:20:16.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.998+0000 7f9fc1600640 1 -- 172.21.15.69:0/1997273537 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f9fc4038420 con 0x55f9aba99110 2024-03-19T01:20:16.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.999+0000 7f9fdd7e7740 10 client.17307 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:17.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.999+0000 7f9fdd7e7740 10 client.17307 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:20:17.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.999+0000 7f9fdd7e7740 20 client.17307 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:17.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.999+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f9abb10a10 0x55f9abb12e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.999+0000 7f9fdd7e7740 4 client.17307 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:20:17.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.999+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/1997273537 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9ab9fbc10 con 0x55f9abb10a10 2024-03-19T01:20:17.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:16.999+0000 7f9fd3400640 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f9abb10a10 0x55f9abb12e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.000+0000 7f9fc1600640 10 client.17307 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:20:17.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.000+0000 7f9fd3400640 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f9abb10a10 0x55f9abb12e30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=832de162e59ae012 server_cookie=bd5c34596b73bdcc in_seq=0 out_seq=0 2024-03-19T01:20:17.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.000+0000 7f9fc1600640 1 -- 172.21.15.69:0/1997273537 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9ab9fbc10 con 0x55f9abb10a10 2024-03-19T01:20:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.000+0000 7f9fc1600640 10 client.17307 handle_command_reply: tid=0 2024-03-19T01:20:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.000+0000 7f9fdd7e7740 1 client.17307 shutdown 2024-03-19T01:20:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.000+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/1997273537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba99110 msgr2=0x55f9aba9d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.000+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba99110 0x55f9aba9d3f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9fc402e880 tx=0x7f9fc40119b0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.001+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/1997273537 shutdown_connections 2024-03-19T01:20:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.001+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f9abb10a10 0x55f9abb12e30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.001+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9aba9d930 0x55f9abb0e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.001+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9aba99110 0x55f9aba9d3f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.001+0000 7f9fdd7e7740 1 --2- 172.21.15.69:0/1997273537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9aba90530 0x55f9aba9ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.001+0000 7f9fc1600640 0 client.17307 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:20:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.001+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/1997273537 >> 172.21.15.69:0/1997273537 conn(0x55f9aba8a340 msgr2=0x55f9aba961f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.001+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/1997273537 shutdown_connections 2024-03-19T01:20:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.001+0000 7f9fdd7e7740 1 -- 172.21.15.69:0/1997273537 wait complete. 2024-03-19T01:20:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.001+0000 7f9fdd7e7740 20 client.17307 trim_cache size 0 max 16384 2024-03-19T01:20:17.005 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:20:17.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.006+0000 7f9fd9400640 1 -- 172.21.15.69:0/1421422269 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9fb80a2ac0 msgr2=0x7f9fb80a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.006+0000 7f9fd9400640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9fb80a2ac0 0x7f9fb80a4f80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0ca650 tx=0x7f9fcc059040 comp rx=0 tx=0).stop 2024-03-19T01:20:17.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.006+0000 7f9fd9400640 1 -- 172.21.15.69:0/1421422269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd411b000 msgr2=0x7f9fd4116df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.006+0000 7f9fd9400640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd411b000 0x7f9fd4116df0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0433b0 tx=0x7f9fcc003ba0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.012+0000 7f9fd9400640 1 -- 172.21.15.69:0/1421422269 shutdown_connections 2024-03-19T01:20:17.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.012+0000 7f9fd9400640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd411b000 0x7f9fd4116df0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.012+0000 7f9fd9400640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9fb80a2ac0 0x7f9fb80a4f80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.012+0000 7f9fd9400640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd410fe40 0x7f9fd41168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.012+0000 7f9fd9400640 1 --2- 172.21.15.69:0/1421422269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fd41088b0 0x7f9fd411a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.012+0000 7f9fd9400640 1 -- 172.21.15.69:0/1421422269 >> 172.21.15.69:0/1421422269 conn(0x7f9fd406c770 msgr2=0x7f9fd411b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:17.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.017+0000 7f9fd9400640 1 -- 172.21.15.69:0/1421422269 shutdown_connections 2024-03-19T01:20:17.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.017+0000 7f9fd9400640 1 -- 172.21.15.69:0/1421422269 wait complete. 2024-03-19T01:20:17.066 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:20:17.310 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fat/namei_vfat.o 2024-03-19T01:20:17.365 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/isofs/dir.o 2024-03-19T01:20:17.399 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/pid_list.o 2024-03-19T01:20:17.462 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:17 smithi163 ceph-mon[67428]: pgmap v299: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 1.1 MiB/s wr, 69 op/s 2024-03-19T01:20:17.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:17 smithi086 ceph-mon[66829]: pgmap v299: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 1.1 MiB/s wr, 69 op/s 2024-03-19T01:20:17.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:17 smithi069 ceph-mon[31727]: pgmap v299: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 1.1 MiB/s wr, 69 op/s 2024-03-19T01:20:17.656 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/irq.o 2024-03-19T01:20:17.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.671+0000 7f9ab4a00640 1 -- 172.21.15.69:0/1942993911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab0108820 msgr2=0x7f9ab0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.671+0000 7f9ab4a00640 1 --2- 172.21.15.69:0/1942993911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab0108820 0x7f9ab0108c00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009930 tx=0x7f9aa002bc20 comp rx=0 tx=0).stop 2024-03-19T01:20:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.672+0000 7f9ab4a00640 1 -- 172.21.15.69:0/1942993911 shutdown_connections 2024-03-19T01:20:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.672+0000 7f9ab4a00640 1 --2- 172.21.15.69:0/1942993911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab0114970 0x7f9ab0116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.672+0000 7f9ab4a00640 1 --2- 172.21.15.69:0/1942993911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0109140 0x7f9ab0114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.672+0000 7f9ab4a00640 1 --2- 172.21.15.69:0/1942993911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab0108820 0x7f9ab0108c00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.672+0000 7f9ab4a00640 1 -- 172.21.15.69:0/1942993911 >> 172.21.15.69:0/1942993911 conn(0x7f9ab006c770 msgr2=0x7f9ab006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.672+0000 7f9ab4a00640 1 -- 172.21.15.69:0/1942993911 shutdown_connections 2024-03-19T01:20:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.672+0000 7f9ab4a00640 1 -- 172.21.15.69:0/1942993911 wait complete. 2024-03-19T01:20:17.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.674+0000 7f9ab4a00640 1 Processor -- start 2024-03-19T01:20:17.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.674+0000 7f9ab4a00640 1 -- start start 2024-03-19T01:20:17.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.675+0000 7f9ab4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab0108820 0x7f9ab010c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.675+0000 7f9ab4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab0109140 0x7f9ab010c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.675+0000 7f9ab4a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0114970 0x7f9ab010cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.675+0000 7f9ab4a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab011a2a0 con 0x7f9ab0114970 2024-03-19T01:20:17.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.675+0000 7f9ab4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab011a120 con 0x7f9ab0109140 2024-03-19T01:20:17.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.675+0000 7f9ab4a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab011a420 con 0x7f9ab0108820 2024-03-19T01:20:17.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.675+0000 7f9aae800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab0108820 0x7f9ab010c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.675+0000 7f9aae800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab0108820 0x7f9ab010c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38642/0 (socket says 172.21.15.69:38642) 2024-03-19T01:20:17.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.675+0000 7f9aae800640 1 -- 172.21.15.69:0/3887080455 learned_addr learned my addr 172.21.15.69:0/3887080455 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:17.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.676+0000 7f9aade00640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab0109140 0x7f9ab010c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.676+0000 7f9aaf200640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0114970 0x7f9ab010cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.676+0000 7f9aae800640 1 -- 172.21.15.69:0/3887080455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab0109140 msgr2=0x7f9ab010c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.676+0000 7f9aae800640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab0109140 0x7f9ab010c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.676+0000 7f9aae800640 1 -- 172.21.15.69:0/3887080455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0114970 msgr2=0x7f9ab010cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.676+0000 7f9aae800640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0114970 0x7f9ab010cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.676+0000 7f9aae800640 1 -- 172.21.15.69:0/3887080455 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab01a2700 con 0x7f9ab0108820 2024-03-19T01:20:17.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.676+0000 7f9aade00640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab0109140 0x7f9ab010c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:20:17.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.676+0000 7f9aaf200640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0114970 0x7f9ab010cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:17.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.676+0000 7f9aae800640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab0108820 0x7f9ab010c4a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0005ab0 tx=0x7f9aa00042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:17.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.677+0000 7f9aa7400640 1 -- 172.21.15.69:0/3887080455 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0043020 con 0x7f9ab0108820 2024-03-19T01:20:17.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.677+0000 7f9aa7400640 1 -- 172.21.15.69:0/3887080455 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aa00044d0 con 0x7f9ab0108820 2024-03-19T01:20:17.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.677+0000 7f9aa7400640 1 -- 172.21.15.69:0/3887080455 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0003bb0 con 0x7f9ab0108820 2024-03-19T01:20:17.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.678+0000 7f9ab4a00640 1 -- 172.21.15.69:0/3887080455 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab01a2990 con 0x7f9ab0108820 2024-03-19T01:20:17.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.678+0000 7f9ab4a00640 1 -- 172.21.15.69:0/3887080455 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab01a2e60 con 0x7f9ab0108820 2024-03-19T01:20:17.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.683+0000 7f9aa7400640 1 -- 172.21.15.69:0/3887080455 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9aa0046020 con 0x7f9ab0108820 2024-03-19T01:20:17.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.685+0000 7f9aa7400640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9a940a29f0 0x7f9a940a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.686+0000 7f9aa7400640 1 -- 172.21.15.69:0/3887080455 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f9aa00f0be0 con 0x7f9ab0108820 2024-03-19T01:20:17.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.688+0000 7f9aade00640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9a940a29f0 0x7f9a940a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.689+0000 7f9aade00640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9a940a29f0 0x7f9a940a4eb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ab010dc60 tx=0x7f9aa8007850 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:17.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.689+0000 7f9ab806f740 1 Processor -- start 2024-03-19T01:20:17.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.689+0000 7f9ab806f740 1 -- start start 2024-03-19T01:20:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.690+0000 7f9ab806f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cec3c10 0x556a8cec3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.690+0000 7f9ab806f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8ceccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.690+0000 7f9ab806f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cecd110 0x556a8cecf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.690+0000 7f9ab806f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556a8ce28380 con 0x556a8cecd110 2024-03-19T01:20:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.690+0000 7f9ab806f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a8cdd0d20 con 0x556a8cec3c10 2024-03-19T01:20:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.690+0000 7f9ab806f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556a8ce24420 con 0x556a8cec4530 2024-03-19T01:20:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.690+0000 7f9aaf200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cecd110 0x556a8cecf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.691+0000 7f9aaf200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cecd110 0x556a8cecf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43798/0 (socket says 172.21.15.69:43798) 2024-03-19T01:20:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.691+0000 7f9aaf200640 1 -- 172.21.15.69:0/1519373789 learned_addr learned my addr 172.21.15.69:0/1519373789 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.691+0000 7f9aade00640 1 --2- 172.21.15.69:0/1519373789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8ceccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.692+0000 7f9aae800640 1 --2- 172.21.15.69:0/1519373789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cec3c10 0x556a8cec3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.692+0000 7f9aaf200640 1 -- 172.21.15.69:0/1519373789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 msgr2=0x556a8ceccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.692+0000 7f9aaf200640 1 --2- 172.21.15.69:0/1519373789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8ceccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.692+0000 7f9aaf200640 1 -- 172.21.15.69:0/1519373789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cec3c10 msgr2=0x556a8cec3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.692+0000 7f9aaf200640 1 --2- 172.21.15.69:0/1519373789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cec3c10 0x556a8cec3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.692+0000 7f9aaf200640 1 -- 172.21.15.69:0/1519373789 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a8cb3c3f0 con 0x556a8cecd110 2024-03-19T01:20:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.692+0000 7f9aaf200640 1 --2- 172.21.15.69:0/1519373789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cecd110 0x556a8cecf500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009b50 tx=0x7f9a98010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e44d9ff1b753378 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.694+0000 7f9a9f400640 1 -- 172.21.15.69:0/1519373789 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98009e40 con 0x556a8cecd110 2024-03-19T01:20:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.694+0000 7f9a9f400640 1 -- 172.21.15.69:0/1519373789 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a9801b960 con 0x556a8cecd110 2024-03-19T01:20:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.694+0000 7f9a9f400640 1 -- 172.21.15.69:0/1519373789 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98012a40 con 0x556a8cecd110 2024-03-19T01:20:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.695+0000 7f9ab806f740 1 -- 172.21.15.69:0/1519373789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cecd110 msgr2=0x556a8cecf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.695+0000 7f9ab806f740 1 --2- 172.21.15.69:0/1519373789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cecd110 0x556a8cecf500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009b50 tx=0x7f9a98010740 comp rx=0 tx=0).stop 2024-03-19T01:20:17.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.698+0000 7f9ab806f740 1 -- 172.21.15.69:0/1519373789 shutdown_connections 2024-03-19T01:20:17.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.698+0000 7f9ab806f740 1 --2- 172.21.15.69:0/1519373789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cecd110 0x556a8cecf500 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.698+0000 7f9ab806f740 1 --2- 172.21.15.69:0/1519373789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8ceccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.698+0000 7f9ab806f740 1 --2- 172.21.15.69:0/1519373789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cec3c10 0x556a8cec3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.698+0000 7f9ab806f740 1 -- 172.21.15.69:0/1519373789 >> 172.21.15.69:0/1519373789 conn(0x556a8cebe340 msgr2=0x556a8cebe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.698+0000 7f9ab806f740 1 -- 172.21.15.69:0/1519373789 shutdown_connections 2024-03-19T01:20:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.698+0000 7f9ab806f740 1 -- 172.21.15.69:0/1519373789 wait complete. 2024-03-19T01:20:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.698+0000 7f9ab806f740 1 -- start start 2024-03-19T01:20:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.699+0000 7f9ab806f740 1 Processor -- start 2024-03-19T01:20:17.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.703+0000 7f9ab806f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ceca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.703+0000 7f9ab806f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8ceca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.703+0000 7f9ab806f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cecacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.703+0000 7f9ab806f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556a8cecff00 con 0x556a8cec3c10 2024-03-19T01:20:17.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.703+0000 7f9ab806f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a8cde4160 con 0x556a8cecd110 2024-03-19T01:20:17.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.703+0000 7f9ab806f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556a8ced0080 con 0x556a8cec4530 2024-03-19T01:20:17.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.703+0000 7f9ab806f740 10 client.0 _ll_register_callbacks cb 0x556a8ce20170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.704+0000 7f9aaf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cecacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.704+0000 7f9aaf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cecacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60630/0 (socket says 172.21.15.69:60630) 2024-03-19T01:20:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.704+0000 7f9aaf200640 1 -- 172.21.15.69:0/2228988385 learned_addr learned my addr 172.21.15.69:0/2228988385 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.704+0000 7f9aade00640 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8ceca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.705+0000 7f9aaf200640 1 -- 172.21.15.69:0/2228988385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 msgr2=0x556a8ceca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.705+0000 7f9aaf200640 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8ceca790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.705+0000 7f9aaf200640 1 -- 172.21.15.69:0/2228988385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 msgr2=0x556a8ceca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.705+0000 7f9aae800640 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ceca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.705+0000 7f9aaf200640 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ceca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.705+0000 7f9aaf200640 1 -- 172.21.15.69:0/2228988385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a8cc067d0 con 0x556a8cecd110 2024-03-19T01:20:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.705+0000 7f9aae800640 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ceca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.705+0000 7f9aaf200640 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cecacd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9a980125c0 tx=0x7f9a98012690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d8fd9fec822f119 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:17.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.705+0000 7f9a93e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:20:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.706+0000 7f9a93e00640 1 -- 172.21.15.69:0/2228988385 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98029060 con 0x556a8cecd110 2024-03-19T01:20:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.706+0000 7f9a93e00640 1 -- 172.21.15.69:0/2228988385 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f9a9801bd10 con 0x556a8cecd110 2024-03-19T01:20:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.706+0000 7f9ab806f740 1 -- 172.21.15.69:0/2228988385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a9801b960 con 0x556a8cecd110 2024-03-19T01:20:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.706+0000 7f9a93e00640 1 -- 172.21.15.69:0/2228988385 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9a98030050 con 0x556a8cecd110 2024-03-19T01:20:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.707+0000 7f9ab806f740 10 client.27632 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.707+0000 7f9ab806f740 1 -- 172.21.15.69:0/2228988385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x556a8cedc2b0 con 0x556a8cecd110 2024-03-19T01:20:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.707+0000 7f9a93e00640 1 -- 172.21.15.69:0/2228988385 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a9801b960 con 0x556a8cecd110 2024-03-19T01:20:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.707+0000 7f9a93e00640 1 -- 172.21.15.69:0/2228988385 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f9a98023070 con 0x556a8cecd110 2024-03-19T01:20:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.707+0000 7f9ab806f740 10 client.27632 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.707+0000 7f9ab806f740 10 client.27632 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:20:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.707+0000 7f9ab806f740 20 client.27632 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.708+0000 7f9ab806f740 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x556a8cedd710 0x556a8cedfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.708+0000 7f9ab806f740 4 client.27632 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:20:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.708+0000 7f9ab806f740 1 -- 172.21.15.69:0/2228988385 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a8ce358c0 con 0x556a8cedd710 2024-03-19T01:20:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.708+0000 7f9aae800640 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x556a8cedd710 0x556a8cedfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.708+0000 7f9aae800640 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x556a8cedd710 0x556a8cedfb00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ba5e0ff201a61a server_cookie=84554cdde6ff609 in_seq=0 out_seq=0 2024-03-19T01:20:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9a93e00640 10 client.27632 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:20:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9a93e00640 1 -- 172.21.15.69:0/2228988385 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556a8ce358c0 con 0x556a8cedd710 2024-03-19T01:20:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9a93e00640 10 client.27632 handle_command_reply: tid=0 2024-03-19T01:20:17.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9ab806f740 1 client.27632 shutdown 2024-03-19T01:20:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9ab806f740 1 -- 172.21.15.69:0/2228988385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 msgr2=0x556a8cecacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9ab806f740 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cecacd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9a980125c0 tx=0x7f9a98012690 comp rx=0 tx=0).stop 2024-03-19T01:20:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9ab806f740 1 -- 172.21.15.69:0/2228988385 shutdown_connections 2024-03-19T01:20:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9ab806f740 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x556a8cedd710 0x556a8cedfb00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9ab806f740 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cecacd0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9ab806f740 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8ceca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9ab806f740 1 --2- 172.21.15.69:0/2228988385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ceca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9a93e00640 0 client.27632 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:20:17.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.712+0000 7f9ab806f740 1 -- 172.21.15.69:0/2228988385 >> 172.21.15.69:0/2228988385 conn(0x556a8cebe340 msgr2=0x556a8cec0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:17.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.713+0000 7f9ab806f740 1 -- 172.21.15.69:0/2228988385 shutdown_connections 2024-03-19T01:20:17.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.713+0000 7f9ab806f740 1 -- 172.21.15.69:0/2228988385 wait complete. 2024-03-19T01:20:17.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.713+0000 7f9ab806f740 20 client.27632 trim_cache size 0 max 16384 2024-03-19T01:20:17.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.771+0000 7f9ab806f740 1 Processor -- start 2024-03-19T01:20:17.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.771+0000 7f9ab806f740 1 -- start start 2024-03-19T01:20:17.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.772+0000 7f9ab806f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ced8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.772+0000 7f9ab806f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8cecaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.772+0000 7f9ab806f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cebd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.772+0000 7f9ab806f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01c6220 con 0x556a8cec3c10 2024-03-19T01:20:17.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.772+0000 7f9ab806f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01c7d60 con 0x556a8cecd110 2024-03-19T01:20:17.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.772+0000 7f9ab806f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01c68f0 con 0x556a8cec4530 2024-03-19T01:20:17.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.772+0000 7f9aaf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cebd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.772+0000 7f9aae800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ced8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.772+0000 7f9aaf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cebd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60636/0 (socket says 172.21.15.69:60636) 2024-03-19T01:20:17.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.772+0000 7f9aaf200640 1 -- 172.21.15.69:0/608338822 learned_addr learned my addr 172.21.15.69:0/608338822 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:17.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.774+0000 7f9aaf200640 1 -- 172.21.15.69:0/608338822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 msgr2=0x556a8cecaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:17.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.773+0000 7f9aade00640 1 --2- 172.21.15.69:0/608338822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8cecaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.774+0000 7f9aaf200640 1 --2- 172.21.15.69:0/608338822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8cecaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.774+0000 7f9aaf200640 1 -- 172.21.15.69:0/608338822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 msgr2=0x556a8ced8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:17.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.774+0000 7f9aaf200640 1 --2- 172.21.15.69:0/608338822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ced8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.774+0000 7f9aaf200640 1 -- 172.21.15.69:0/608338822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a8ceb87f0 con 0x556a8cecd110 2024-03-19T01:20:17.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.774+0000 7f9aae800640 1 --2- 172.21.15.69:0/608338822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ced8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:20:17.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.774+0000 7f9aade00640 1 --2- 172.21.15.69:0/608338822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8cecaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:17.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.774+0000 7f9aaf200640 1 --2- 172.21.15.69:0/608338822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cebd260 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9a98010cc0 tx=0x7f9a98010d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e647a90b3c3b3c86 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:17.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.778+0000 7f9a9ea00640 1 -- 172.21.15.69:0/608338822 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98029050 con 0x556a8cecd110 2024-03-19T01:20:17.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.778+0000 7f9a9ea00640 1 -- 172.21.15.69:0/608338822 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a98039070 con 0x556a8cecd110 2024-03-19T01:20:17.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.778+0000 7f9a9ea00640 1 -- 172.21.15.69:0/608338822 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98034430 con 0x556a8cecd110 2024-03-19T01:20:17.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.779+0000 7f9ab806f740 1 -- 172.21.15.69:0/608338822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 msgr2=0x556a8cebd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.779+0000 7f9ab806f740 1 --2- 172.21.15.69:0/608338822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cebd260 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9a98010cc0 tx=0x7f9a98010d70 comp rx=0 tx=0).stop 2024-03-19T01:20:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.781+0000 7f9ab806f740 1 -- 172.21.15.69:0/608338822 shutdown_connections 2024-03-19T01:20:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.781+0000 7f9ab806f740 1 --2- 172.21.15.69:0/608338822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cecd110 0x556a8cebd260 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.781+0000 7f9ab806f740 1 --2- 172.21.15.69:0/608338822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cec4530 0x556a8cecaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.781+0000 7f9ab806f740 1 --2- 172.21.15.69:0/608338822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ced8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.781+0000 7f9ab806f740 1 -- 172.21.15.69:0/608338822 >> 172.21.15.69:0/608338822 conn(0x556a8cebe340 msgr2=0x556a8cebcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.781+0000 7f9ab806f740 1 -- 172.21.15.69:0/608338822 shutdown_connections 2024-03-19T01:20:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.781+0000 7f9ab806f740 1 -- 172.21.15.69:0/608338822 wait complete. 2024-03-19T01:20:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.782+0000 7f9ab806f740 1 -- start start 2024-03-19T01:20:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.782+0000 7f9ab806f740 1 Processor -- start 2024-03-19T01:20:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.784+0000 7f9ab806f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ced4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.784+0000 7f9aae800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ced4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.784+0000 7f9aae800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ced4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43832/0 (socket says 172.21.15.69:43832) 2024-03-19T01:20:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.785+0000 7f9aae800640 1 -- 172.21.15.69:0/3866778535 learned_addr learned my addr 172.21.15.69:0/3866778535 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9ab806f740 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cec4530 0x556a8ced5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9ab806f740 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cecd110 0x556a8ced03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9ab806f740 1 -- 172.21.15.69:0/3866778535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01c7690 con 0x556a8cec3c10 2024-03-19T01:20:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9ab806f740 1 -- 172.21.15.69:0/3866778535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01c6fc0 con 0x556a8cec4530 2024-03-19T01:20:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9ab806f740 1 -- 172.21.15.69:0/3866778535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab010d9d0 con 0x556a8cecd110 2024-03-19T01:20:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9ab806f740 10 client.0 _ll_register_callbacks cb 0x7f9aa8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9aaf200640 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cecd110 0x556a8ced03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9aaf200640 1 -- 172.21.15.69:0/3866778535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cec4530 msgr2=0x556a8ced5530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:20:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9aaf200640 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cec4530 0x556a8ced5530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9aaf200640 1 -- 172.21.15.69:0/3866778535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 msgr2=0x556a8ced4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9aaf200640 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ced4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9aaf200640 1 -- 172.21.15.69:0/3866778535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a8ced0bd0 con 0x556a8cecd110 2024-03-19T01:20:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.788+0000 7f9aae800640 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ced4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.789+0000 7f9aaf200640 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cecd110 0x556a8ced03d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009b50 tx=0x7f9a98012d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232206a4913cc063 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.789+0000 7f9a93e00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:20:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.789+0000 7f9a93e00640 1 -- 172.21.15.69:0/3866778535 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98044060 con 0x556a8cecd110 2024-03-19T01:20:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.789+0000 7f9a93e00640 1 -- 172.21.15.69:0/3866778535 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f9a9801b8d0 con 0x556a8cecd110 2024-03-19T01:20:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.789+0000 7f9ab806f740 1 -- 172.21.15.69:0/3866778535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a98039070 con 0x556a8cecd110 2024-03-19T01:20:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.790+0000 7f9a93e00640 1 -- 172.21.15.69:0/3866778535 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9a98029040 con 0x556a8cecd110 2024-03-19T01:20:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.790+0000 7f9ab806f740 10 client.27607 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.790+0000 7f9ab806f740 1 -- 172.21.15.69:0/3866778535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x556a8cf43210 con 0x556a8cecd110 2024-03-19T01:20:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.791+0000 7f9a93e00640 1 -- 172.21.15.69:0/3866778535 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a98039070 con 0x556a8cecd110 2024-03-19T01:20:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.791+0000 7f9a93e00640 1 -- 172.21.15.69:0/3866778535 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f9a98023040 con 0x556a8cecd110 2024-03-19T01:20:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.791+0000 7f9ab806f740 10 client.27607 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:17.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.791+0000 7f9ab806f740 10 client.27607 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:20:17.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.791+0000 7f9ab806f740 20 client.27607 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:17.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.791+0000 7f9ab806f740 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x556a8cf438b0 0x556a8cf45cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:17.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.791+0000 7f9ab806f740 4 client.27607 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:20:17.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.791+0000 7f9ab806f740 1 -- 172.21.15.69:0/3866778535 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a8ce2fc10 con 0x556a8cf438b0 2024-03-19T01:20:17.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.791+0000 7f9aae800640 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x556a8cf438b0 0x556a8cf45cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:17.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.792+0000 7f9a93e00640 10 client.27607 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:20:17.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.792+0000 7f9aae800640 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x556a8cf438b0 0x556a8cf45cf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20e781927a39cf40 server_cookie=3d4128810c90bc11 in_seq=0 out_seq=0 2024-03-19T01:20:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9a93e00640 1 -- 172.21.15.69:0/3866778535 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a8ce2fc10 con 0x556a8cf438b0 2024-03-19T01:20:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9a93e00640 10 client.27607 handle_command_reply: tid=0 2024-03-19T01:20:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9ab806f740 1 client.27607 shutdown 2024-03-19T01:20:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9ab806f740 1 -- 172.21.15.69:0/3866778535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cecd110 msgr2=0x556a8ced03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9ab806f740 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cecd110 0x556a8ced03d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009b50 tx=0x7f9a98012d20 comp rx=0 tx=0).stop 2024-03-19T01:20:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9ab806f740 1 -- 172.21.15.69:0/3866778535 shutdown_connections 2024-03-19T01:20:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9ab806f740 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x556a8cf438b0 0x556a8cf45cf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9ab806f740 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a8cecd110 0x556a8ced03d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9ab806f740 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8cec4530 0x556a8ced5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9ab806f740 1 --2- 172.21.15.69:0/3866778535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a8cec3c10 0x556a8ced4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9ab806f740 1 -- 172.21.15.69:0/3866778535 >> 172.21.15.69:0/3866778535 conn(0x556a8cebe340 msgr2=0x556a8cebd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.794+0000 7f9a93e00640 0 client.27607 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:20:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.795+0000 7f9ab806f740 1 -- 172.21.15.69:0/3866778535 shutdown_connections 2024-03-19T01:20:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.795+0000 7f9ab806f740 1 -- 172.21.15.69:0/3866778535 wait complete. 2024-03-19T01:20:17.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.795+0000 7f9ab806f740 20 client.27607 trim_cache size 0 max 16384 2024-03-19T01:20:17.797 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:20:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.803+0000 7f9ab4a00640 1 -- 172.21.15.69:0/3887080455 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9a940a29f0 msgr2=0x7f9a940a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.803+0000 7f9ab4a00640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9a940a29f0 0x7f9a940a4eb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ab010dc60 tx=0x7f9aa8007850 comp rx=0 tx=0).stop 2024-03-19T01:20:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.803+0000 7f9ab4a00640 1 -- 172.21.15.69:0/3887080455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab0108820 msgr2=0x7f9ab010c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.803+0000 7f9ab4a00640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab0108820 0x7f9ab010c4a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0005ab0 tx=0x7f9aa00042e0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.804+0000 7f9ab4a00640 1 -- 172.21.15.69:0/3887080455 shutdown_connections 2024-03-19T01:20:17.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.804+0000 7f9ab4a00640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0114970 0x7f9ab010cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.804+0000 7f9ab4a00640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9a940a29f0 0x7f9a940a4eb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.804+0000 7f9ab4a00640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab0109140 0x7f9ab010c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.804+0000 7f9ab4a00640 1 --2- 172.21.15.69:0/3887080455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ab0108820 0x7f9ab010c4a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:17.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.804+0000 7f9ab4a00640 1 -- 172.21.15.69:0/3887080455 >> 172.21.15.69:0/3887080455 conn(0x7f9ab006c770 msgr2=0x7f9ab010a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:17.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.809+0000 7f9ab4a00640 1 -- 172.21.15.69:0/3887080455 shutdown_connections 2024-03-19T01:20:17.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:17.811+0000 7f9ab4a00640 1 -- 172.21.15.69:0/3887080455 wait complete. 2024-03-19T01:20:17.871 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:20:17.908 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpi-sub-ui.o 2024-03-19T01:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.461+0000 7fd691200640 1 -- 172.21.15.69:0/3633329205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd68c109140 msgr2=0x7fd68c114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.462+0000 7fd691200640 1 --2- 172.21.15.69:0/3633329205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd68c109140 0x7fd68c114430 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0098b0 tx=0x7fd67c02bbc0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.462+0000 7fd691200640 1 -- 172.21.15.69:0/3633329205 shutdown_connections 2024-03-19T01:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.462+0000 7fd691200640 1 --2- 172.21.15.69:0/3633329205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd68c114970 0x7fd68c116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.462+0000 7fd691200640 1 --2- 172.21.15.69:0/3633329205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd68c109140 0x7fd68c114430 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.462+0000 7fd691200640 1 --2- 172.21.15.69:0/3633329205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68c108820 0x7fd68c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.462+0000 7fd691200640 1 -- 172.21.15.69:0/3633329205 >> 172.21.15.69:0/3633329205 conn(0x7fd68c06c770 msgr2=0x7fd68c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:18.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.463+0000 7fd691200640 1 -- 172.21.15.69:0/3633329205 shutdown_connections 2024-03-19T01:20:18.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.463+0000 7fd691200640 1 -- 172.21.15.69:0/3633329205 wait complete. 2024-03-19T01:20:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.466+0000 7fd691200640 1 Processor -- start 2024-03-19T01:20:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.466+0000 7fd691200640 1 -- start start 2024-03-19T01:20:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.467+0000 7fd691200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68c108820 0x7fd68c1a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.468+0000 7fd691200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd68c109140 0x7fd68c1a7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.468+0000 7fd68b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68c108820 0x7fd68c1a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.468+0000 7fd68b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68c108820 0x7fd68c1a6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60662/0 (socket says 172.21.15.69:60662) 2024-03-19T01:20:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.468+0000 7fd68b400640 1 -- 172.21.15.69:0/2502450864 learned_addr learned my addr 172.21.15.69:0/2502450864 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.468+0000 7fd691200640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd68c114970 0x7fd68c1ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.468+0000 7fd68aa00640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd68c109140 0x7fd68c1a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.469+0000 7fd68be00640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd68c114970 0x7fd68c1ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.469+0000 7fd691200640 1 -- 172.21.15.69:0/2502450864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1180c0 con 0x7fd68c114970 2024-03-19T01:20:18.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.469+0000 7fd691200640 1 -- 172.21.15.69:0/2502450864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c117f40 con 0x7fd68c108820 2024-03-19T01:20:18.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.469+0000 7fd691200640 1 -- 172.21.15.69:0/2502450864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c118240 con 0x7fd68c109140 2024-03-19T01:20:18.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.470+0000 7fd68aa00640 1 -- 172.21.15.69:0/2502450864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68c108820 msgr2=0x7fd68c1a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:18.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.470+0000 7fd68aa00640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68c108820 0x7fd68c1a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.470+0000 7fd68aa00640 1 -- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd68c114970 msgr2=0x7fd68c1ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:18.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.470+0000 7fd68aa00640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd68c114970 0x7fd68c1ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.470+0000 7fd68aa00640 1 -- 172.21.15.69:0/2502450864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c1abe20 con 0x7fd68c109140 2024-03-19T01:20:18.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.470+0000 7fd68b400640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68c108820 0x7fd68c1a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:20:18.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.470+0000 7fd68be00640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd68c114970 0x7fd68c1ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:18.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.470+0000 7fd68aa00640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd68c109140 0x7fd68c1a7370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd67c02c1d0 tx=0x7fd67c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:18.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.472+0000 7fd683e00640 1 -- 172.21.15.69:0/2502450864 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c039050 con 0x7fd68c109140 2024-03-19T01:20:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.472+0000 7fd691200640 1 -- 172.21.15.69:0/2502450864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c1ac0b0 con 0x7fd68c109140 2024-03-19T01:20:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.472+0000 7fd691200640 1 -- 172.21.15.69:0/2502450864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c1b39a0 con 0x7fd68c109140 2024-03-19T01:20:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.472+0000 7fd683e00640 1 -- 172.21.15.69:0/2502450864 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c03a040 con 0x7fd68c109140 2024-03-19T01:20:18.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.474+0000 7fd683e00640 1 -- 172.21.15.69:0/2502450864 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c039050 con 0x7fd68c109140 2024-03-19T01:20:18.479 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/isofs/util.o 2024-03-19T01:20:18.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.478+0000 7fd695588740 1 Processor -- start 2024-03-19T01:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.481+0000 7fd695588740 1 -- start start 2024-03-19T01:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.481+0000 7fd695588740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c3fc10 0x55cb51c3fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.481+0000 7fd695588740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c40530 0x55cb51c48bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.481+0000 7fd695588740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c49110 0x55cb51c4b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.481+0000 7fd695588740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cb51ba4380 con 0x55cb51c3fc10 2024-03-19T01:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.481+0000 7fd695588740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb51b4cd20 con 0x55cb51c49110 2024-03-19T01:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.481+0000 7fd695588740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55cb51ba0420 con 0x55cb51c40530 2024-03-19T01:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.482+0000 7fd68be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c3fc10 0x55cb51c3fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.482+0000 7fd68be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c3fc10 0x55cb51c3fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43862/0 (socket says 172.21.15.69:43862) 2024-03-19T01:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.482+0000 7fd68be00640 1 -- 172.21.15.69:0/3884070025 learned_addr learned my addr 172.21.15.69:0/3884070025 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.483+0000 7fd68be00640 1 -- 172.21.15.69:0/3884070025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c40530 msgr2=0x55cb51c48bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.483+0000 7fd68be00640 1 --2- 172.21.15.69:0/3884070025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c40530 0x55cb51c48bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.483+0000 7fd68be00640 1 -- 172.21.15.69:0/3884070025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c49110 msgr2=0x55cb51c4b500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.483+0000 7fd68be00640 1 --2- 172.21.15.69:0/3884070025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c49110 0x55cb51c4b500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.483+0000 7fd68be00640 1 -- 172.21.15.69:0/3884070025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb518b83f0 con 0x55cb51c3fc10 2024-03-19T01:20:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.483+0000 7fd68be00640 1 --2- 172.21.15.69:0/3884070025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c3fc10 0x55cb51c3fff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd67800bfa0 tx=0x7fd678010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708a1a09b3401eed server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:18.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.484+0000 7fd683e00640 1 -- 172.21.15.69:0/2502450864 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fd67c047030 con 0x7fd68c109140 2024-03-19T01:20:18.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.484+0000 7fd680c00640 1 -- 172.21.15.69:0/3884070025 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678011700 con 0x55cb51c3fc10 2024-03-19T01:20:18.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.484+0000 7fd680c00640 1 -- 172.21.15.69:0/3884070025 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd678011d80 con 0x55cb51c3fc10 2024-03-19T01:20:18.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.484+0000 7fd680c00640 1 -- 172.21.15.69:0/3884070025 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67801a380 con 0x55cb51c3fc10 2024-03-19T01:20:18.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.486+0000 7fd683e00640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd6700a2ba0 0x7fd6700a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.487+0000 7fd695588740 1 -- 172.21.15.69:0/3884070025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c3fc10 msgr2=0x55cb51c3fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:18.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.487+0000 7fd695588740 1 --2- 172.21.15.69:0/3884070025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c3fc10 0x55cb51c3fff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd67800bfa0 tx=0x7fd678010860 comp rx=0 tx=0).stop 2024-03-19T01:20:18.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.488+0000 7fd695588740 1 -- 172.21.15.69:0/3884070025 shutdown_connections 2024-03-19T01:20:18.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.488+0000 7fd695588740 1 --2- 172.21.15.69:0/3884070025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c49110 0x55cb51c4b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.488+0000 7fd695588740 1 --2- 172.21.15.69:0/3884070025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c40530 0x55cb51c48bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.488+0000 7fd695588740 1 --2- 172.21.15.69:0/3884070025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c3fc10 0x55cb51c3fff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.488+0000 7fd695588740 1 -- 172.21.15.69:0/3884070025 >> 172.21.15.69:0/3884070025 conn(0x55cb51c3a340 msgr2=0x55cb51c3a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:18.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.488+0000 7fd68b400640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd6700a2ba0 0x7fd6700a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.489+0000 7fd695588740 1 -- 172.21.15.69:0/3884070025 shutdown_connections 2024-03-19T01:20:18.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.489+0000 7fd695588740 1 -- 172.21.15.69:0/3884070025 wait complete. 2024-03-19T01:20:18.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.490+0000 7fd695588740 1 -- start start 2024-03-19T01:20:18.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.490+0000 7fd695588740 1 Processor -- start 2024-03-19T01:20:18.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.490+0000 7fd68b400640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd6700a2ba0 0x7fd6700a5060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55cb51c3f6e0 tx=0x7fd6840099f0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:18.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.491+0000 7fd695588740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c3fc10 0x55cb51c460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.491+0000 7fd695588740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 0x55cb51c46600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.491+0000 7fd695588740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 0x55cb51c46b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.491+0000 7fd68aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 0x55cb51c46600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.491+0000 7fd68b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c3fc10 0x55cb51c460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.491+0000 7fd68aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 0x55cb51c46600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60670/0 (socket says 172.21.15.69:60670) 2024-03-19T01:20:18.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.491+0000 7fd68b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c3fc10 0x55cb51c460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38722/0 (socket says 172.21.15.69:38722) 2024-03-19T01:20:18.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.492+0000 7fd68aa00640 1 -- 172.21.15.69:0/3534273774 learned_addr learned my addr 172.21.15.69:0/3534273774 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:18.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.492+0000 7fd695588740 1 -- 172.21.15.69:0/3534273774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cb51c4bf00 con 0x55cb51c49110 2024-03-19T01:20:18.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.492+0000 7fd695588740 1 -- 172.21.15.69:0/3534273774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb51b60160 con 0x55cb51c40530 2024-03-19T01:20:18.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.492+0000 7fd695588740 1 -- 172.21.15.69:0/3534273774 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55cb51c4c080 con 0x55cb51c3fc10 2024-03-19T01:20:18.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.492+0000 7fd695588740 10 client.0 _ll_register_callbacks cb 0x55cb51b9c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:18.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.492+0000 7fd683e00640 1 -- 172.21.15.69:0/2502450864 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7fd67c02de00 con 0x7fd68c109140 2024-03-19T01:20:18.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.492+0000 7fd68be00640 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 0x55cb51c46b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.493+0000 7fd68be00640 1 -- 172.21.15.69:0/3534273774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c3fc10 msgr2=0x55cb51c460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.493+0000 7fd68be00640 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c3fc10 0x55cb51c460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.493+0000 7fd68be00640 1 -- 172.21.15.69:0/3534273774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 msgr2=0x55cb51c46600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.493+0000 7fd68be00640 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 0x55cb51c46600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.493+0000 7fd68be00640 1 -- 172.21.15.69:0/3534273774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb519827d0 con 0x55cb51c49110 2024-03-19T01:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.494+0000 7fd675600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.494+0000 7fd68be00640 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 0x55cb51c46b40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd678005890 tx=0x7fd67801c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afecde69104c50f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:18.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.494+0000 7fd675600640 1 -- 172.21.15.69:0/3534273774 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678005240 con 0x55cb51c49110 2024-03-19T01:20:18.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.495+0000 7fd68b400640 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c3fc10 0x55cb51c460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-19T01:20:18.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.495+0000 7fd675600640 1 -- 172.21.15.69:0/3534273774 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7fd678005040 con 0x55cb51c49110 2024-03-19T01:20:18.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.495+0000 7fd695588740 1 -- 172.21.15.69:0/3534273774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd678011d80 con 0x55cb51c49110 2024-03-19T01:20:18.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.495+0000 7fd68aa00640 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 0x55cb51c46600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:18.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.495+0000 7fd675600640 1 -- 172.21.15.69:0/3534273774 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fd678005240 con 0x55cb51c49110 2024-03-19T01:20:18.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.495+0000 7fd695588740 10 client.17328 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:18.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.495+0000 7fd695588740 1 -- 172.21.15.69:0/3534273774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cb51c581e0 con 0x55cb51c49110 2024-03-19T01:20:18.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.496+0000 7fd675600640 1 -- 172.21.15.69:0/3534273774 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd678011d80 con 0x55cb51c49110 2024-03-19T01:20:18.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.496+0000 7fd675600640 1 -- 172.21.15.69:0/3534273774 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fd67802f030 con 0x55cb51c49110 2024-03-19T01:20:18.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.496+0000 7fd695588740 10 client.17328 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.496+0000 7fd695588740 10 client.17328 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.496+0000 7fd695588740 20 client.17328 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.496+0000 7fd695588740 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cb51c599e0 0x55cb51c5be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.497+0000 7fd695588740 4 client.17328 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.497+0000 7fd695588740 1 -- 172.21.15.69:0/3534273774 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb51bb18c0 con 0x55cb51c599e0 2024-03-19T01:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.498+0000 7fd68b400640 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cb51c599e0 0x55cb51c5be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.500+0000 7fd68b400640 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cb51c599e0 0x55cb51c5be40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f0f401248346b8c server_cookie=5c15e4e952790d0d in_seq=0 out_seq=0 2024-03-19T01:20:18.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.501+0000 7fd675600640 10 client.17328 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:20:18.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.502+0000 7fd675600640 1 -- 172.21.15.69:0/3534273774 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cb51bb18c0 con 0x55cb51c599e0 2024-03-19T01:20:18.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.502+0000 7fd675600640 10 client.17328 handle_command_reply: tid=0 2024-03-19T01:20:18.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.503+0000 7fd695588740 1 client.17328 shutdown 2024-03-19T01:20:18.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.503+0000 7fd695588740 1 -- 172.21.15.69:0/3534273774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 msgr2=0x55cb51c46b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:18.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.503+0000 7fd695588740 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 0x55cb51c46b40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd678005890 tx=0x7fd67801c6b0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.505+0000 7fd695588740 1 -- 172.21.15.69:0/3534273774 shutdown_connections 2024-03-19T01:20:18.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.505+0000 7fd695588740 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 0x55cb51c46b40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.505+0000 7fd695588740 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cb51c599e0 0x55cb51c5be40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.505+0000 7fd695588740 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 0x55cb51c46600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.505+0000 7fd695588740 1 --2- 172.21.15.69:0/3534273774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c3fc10 0x55cb51c460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.505+0000 7fd695588740 1 -- 172.21.15.69:0/3534273774 >> 172.21.15.69:0/3534273774 conn(0x55cb51c3a340 msgr2=0x55cb51c3c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:18.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.506+0000 7fd675600640 0 client.17328 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:20:18.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.507+0000 7fd695588740 1 -- 172.21.15.69:0/3534273774 shutdown_connections 2024-03-19T01:20:18.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.507+0000 7fd695588740 1 -- 172.21.15.69:0/3534273774 wait complete. 2024-03-19T01:20:18.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.507+0000 7fd695588740 20 client.17328 trim_cache size 0 max 16384 2024-03-19T01:20:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.570+0000 7fd695588740 1 Processor -- start 2024-03-19T01:20:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.570+0000 7fd695588740 1 -- start start 2024-03-19T01:20:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.570+0000 7fd695588740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c3fc10 0x55cb51c48780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.570+0000 7fd695588740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 0x55cb51c54d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd695588740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c49110 0x55cb51c46960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd695588740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c010350 con 0x55cb51c3fc10 2024-03-19T01:20:18.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd695588740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c011e90 con 0x55cb51c40530 2024-03-19T01:20:18.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd695588740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c010a20 con 0x55cb51c49110 2024-03-19T01:20:18.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd68aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 0x55cb51c54d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd68be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c49110 0x55cb51c46960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd68aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 0x55cb51c54d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60686/0 (socket says 172.21.15.69:60686) 2024-03-19T01:20:18.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd68aa00640 1 -- 172.21.15.69:0/943795009 learned_addr learned my addr 172.21.15.69:0/943795009 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd68be00640 1 -- 172.21.15.69:0/943795009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 msgr2=0x55cb51c54d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd68be00640 1 --2- 172.21.15.69:0/943795009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 0x55cb51c54d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd68be00640 1 -- 172.21.15.69:0/943795009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c3fc10 msgr2=0x55cb51c48780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd68be00640 1 --2- 172.21.15.69:0/943795009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c3fc10 0x55cb51c48780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.571+0000 7fd68be00640 1 -- 172.21.15.69:0/943795009 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb51c3fa00 con 0x55cb51c49110 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.572+0000 7fd68be00640 1 --2- 172.21.15.69:0/943795009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c49110 0x55cb51c46960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd67800ea50 tx=0x7fd67800b2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c39e63de98ef896 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.572+0000 7fd677e00640 1 -- 172.21.15.69:0/943795009 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678005240 con 0x55cb51c49110 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.573+0000 7fd677e00640 1 -- 172.21.15.69:0/943795009 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67800ee00 con 0x55cb51c49110 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.573+0000 7fd677e00640 1 -- 172.21.15.69:0/943795009 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678005240 con 0x55cb51c49110 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.574+0000 7fd695588740 1 -- 172.21.15.69:0/943795009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c49110 msgr2=0x55cb51c46960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.574+0000 7fd695588740 1 --2- 172.21.15.69:0/943795009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c49110 0x55cb51c46960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd67800ea50 tx=0x7fd67800b2e0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.575+0000 7fd695588740 1 -- 172.21.15.69:0/943795009 shutdown_connections 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.575+0000 7fd695588740 1 --2- 172.21.15.69:0/943795009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c49110 0x55cb51c46960 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.575+0000 7fd695588740 1 --2- 172.21.15.69:0/943795009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c40530 0x55cb51c54d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.575+0000 7fd695588740 1 --2- 172.21.15.69:0/943795009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c3fc10 0x55cb51c48780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.575+0000 7fd695588740 1 -- 172.21.15.69:0/943795009 >> 172.21.15.69:0/943795009 conn(0x55cb51c3a340 msgr2=0x55cb51c41530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:18.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.575+0000 7fd695588740 1 -- 172.21.15.69:0/943795009 shutdown_connections 2024-03-19T01:20:18.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.575+0000 7fd695588740 1 -- 172.21.15.69:0/943795009 wait complete. 2024-03-19T01:20:18.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.575+0000 7fd695588740 1 -- start start 2024-03-19T01:20:18.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.576+0000 7fd695588740 1 Processor -- start 2024-03-19T01:20:18.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.579+0000 7fd695588740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c3fc10 0x55cb51c43950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.579+0000 7fd68b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c3fc10 0x55cb51c43950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.579+0000 7fd68b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c3fc10 0x55cb51c43950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60692/0 (socket says 172.21.15.69:60692) 2024-03-19T01:20:18.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.579+0000 7fd695588740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c40530 0x55cb51c42370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.580+0000 7fd68b400640 1 -- 172.21.15.69:0/2755296198 learned_addr learned my addr 172.21.15.69:0/2755296198 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:18.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.580+0000 7fd695588740 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 0x55cb51c428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.580+0000 7fd68be00640 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 0x55cb51c428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.580+0000 7fd695588740 1 -- 172.21.15.69:0/2755296198 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c0117c0 con 0x55cb51c49110 2024-03-19T01:20:18.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.581+0000 7fd68aa00640 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c40530 0x55cb51c42370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.583+0000 7fd695588740 1 -- 172.21.15.69:0/2755296198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c0110f0 con 0x55cb51c3fc10 2024-03-19T01:20:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.583+0000 7fd695588740 1 -- 172.21.15.69:0/2755296198 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c00e2d0 con 0x55cb51c40530 2024-03-19T01:20:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.583+0000 7fd695588740 10 client.0 _ll_register_callbacks cb 0x7fd67800bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.583+0000 7fd68b400640 1 -- 172.21.15.69:0/2755296198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c40530 msgr2=0x55cb51c42370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.583+0000 7fd68b400640 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c40530 0x55cb51c42370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.583+0000 7fd68b400640 1 -- 172.21.15.69:0/2755296198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 msgr2=0x55cb51c428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.583+0000 7fd68b400640 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 0x55cb51c428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.583+0000 7fd68b400640 1 -- 172.21.15.69:0/2755296198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb51c517f0 con 0x55cb51c3fc10 2024-03-19T01:20:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.584+0000 7fd68be00640 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 0x55cb51c428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.584+0000 7fd68b400640 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c3fc10 0x55cb51c43950 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55cb51c5c6b0 tx=0x7fd684004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=715a9d12a2939b5d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.584+0000 7fd675600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:20:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.584+0000 7fd68aa00640 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c40530 0x55cb51c42370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.584+0000 7fd675600640 1 -- 172.21.15.69:0/2755296198 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684003970 con 0x55cb51c3fc10 2024-03-19T01:20:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.584+0000 7fd675600640 1 -- 172.21.15.69:0/2755296198 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7fd684025070 con 0x55cb51c3fc10 2024-03-19T01:20:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.584+0000 7fd695588740 1 -- 172.21.15.69:0/2755296198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67800ee00 con 0x55cb51c3fc10 2024-03-19T01:20:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.585+0000 7fd675600640 1 -- 172.21.15.69:0/2755296198 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fd684034050 con 0x55cb51c3fc10 2024-03-19T01:20:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.585+0000 7fd695588740 10 client.27647 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.585+0000 7fd695588740 1 -- 172.21.15.69:0/2755296198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55cb51cbf5a0 con 0x55cb51c3fc10 2024-03-19T01:20:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.586+0000 7fd675600640 1 -- 172.21.15.69:0/2755296198 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67800ee00 con 0x55cb51c3fc10 2024-03-19T01:20:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.586+0000 7fd675600640 1 -- 172.21.15.69:0/2755296198 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fd684004140 con 0x55cb51c3fc10 2024-03-19T01:20:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.586+0000 7fd695588740 10 client.27647 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.586+0000 7fd695588740 10 client.27647 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:20:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.586+0000 7fd695588740 20 client.27647 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:18.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.586+0000 7fd695588740 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cb51cbfd50 0x55cb51cc2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:18.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.586+0000 7fd695588740 4 client.27647 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:20:18.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.586+0000 7fd695588740 1 -- 172.21.15.69:0/2755296198 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb51babc10 con 0x55cb51cbfd50 2024-03-19T01:20:18.588 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_sched_switch.o 2024-03-19T01:20:18.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.586+0000 7fd68aa00640 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cb51cbfd50 0x55cb51cc2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:18.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.588+0000 7fd68aa00640 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cb51cbfd50 0x55cb51cc2170 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e73e794dd03fa06 server_cookie=3e4a54f58af7344e in_seq=0 out_seq=0 2024-03-19T01:20:18.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.588+0000 7fd675600640 10 client.27647 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:20:18.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.591+0000 7fd675600640 1 -- 172.21.15.69:0/2755296198 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb51babc10 con 0x55cb51cbfd50 2024-03-19T01:20:18.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.591+0000 7fd675600640 10 client.27647 handle_command_reply: tid=0 2024-03-19T01:20:18.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.591+0000 7fd695588740 1 client.27647 shutdown 2024-03-19T01:20:18.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.591+0000 7fd695588740 1 -- 172.21.15.69:0/2755296198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c3fc10 msgr2=0x55cb51c43950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:18.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.592+0000 7fd695588740 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c3fc10 0x55cb51c43950 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55cb51c5c6b0 tx=0x7fd684004ab0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.594+0000 7fd695588740 1 -- 172.21.15.69:0/2755296198 shutdown_connections 2024-03-19T01:20:18.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.594+0000 7fd695588740 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb51c49110 0x55cb51c428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.594+0000 7fd695588740 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb51c40530 0x55cb51c42370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.594+0000 7fd695588740 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55cb51cbfd50 0x55cb51cc2170 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.594+0000 7fd695588740 1 --2- 172.21.15.69:0/2755296198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb51c3fc10 0x55cb51c43950 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.594+0000 7fd675600640 0 client.27647 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:20:18.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.594+0000 7fd695588740 1 -- 172.21.15.69:0/2755296198 >> 172.21.15.69:0/2755296198 conn(0x55cb51c3a340 msgr2=0x55cb51c38b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:18.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.595+0000 7fd695588740 1 -- 172.21.15.69:0/2755296198 shutdown_connections 2024-03-19T01:20:18.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.595+0000 7fd695588740 1 -- 172.21.15.69:0/2755296198 wait complete. 2024-03-19T01:20:18.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.595+0000 7fd695588740 20 client.27647 trim_cache size 0 max 16384 2024-03-19T01:20:18.599 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:20:18.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.601+0000 7fd691200640 1 -- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd6700a2ba0 msgr2=0x7fd6700a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:18.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.601+0000 7fd691200640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd6700a2ba0 0x7fd6700a5060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55cb51c3f6e0 tx=0x7fd6840099f0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.601+0000 7fd691200640 1 -- 172.21.15.69:0/2502450864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd68c109140 msgr2=0x7fd68c1a7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:18.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.601+0000 7fd691200640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd68c109140 0x7fd68c1a7370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd67c02c1d0 tx=0x7fd67c0045e0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.605+0000 7fd691200640 1 -- 172.21.15.69:0/2502450864 shutdown_connections 2024-03-19T01:20:18.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.605+0000 7fd691200640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd68c114970 0x7fd68c1ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.605+0000 7fd691200640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd68c109140 0x7fd68c1a7370 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.605+0000 7fd691200640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd6700a2ba0 0x7fd6700a5060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.605+0000 7fd691200640 1 --2- 172.21.15.69:0/2502450864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68c108820 0x7fd68c1a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:18.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.605+0000 7fd691200640 1 -- 172.21.15.69:0/2502450864 >> 172.21.15.69:0/2502450864 conn(0x7fd68c06c770 msgr2=0x7fd68c0716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:18.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.605+0000 7fd691200640 1 -- 172.21.15.69:0/2502450864 shutdown_connections 2024-03-19T01:20:18.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:18.607+0000 7fd691200640 1 -- 172.21.15.69:0/2502450864 wait complete. 2024-03-19T01:20:18.660 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:20:18.759 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/vpd.o 2024-03-19T01:20:18.784 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fat/namei_msdos.o 2024-03-19T01:20:18.846 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cgroup/misc.o 2024-03-19T01:20:19.307 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpi-div.o 2024-03-19T01:20:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.349+0000 7f5421800640 1 -- 172.21.15.69:0/4177934463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f541c108df0 msgr2=0x7f541c109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.349+0000 7f5421800640 1 --2- 172.21.15.69:0/4177934463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f541c108df0 0x7f541c109250 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5414009ca0 tx=0x7f541402c100 comp rx=0 tx=0).stop 2024-03-19T01:20:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.349+0000 7f5421800640 1 -- 172.21.15.69:0/4177934463 shutdown_connections 2024-03-19T01:20:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.349+0000 7f5421800640 1 --2- 172.21.15.69:0/4177934463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f541c1144c0 0x7f541c1168d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.349+0000 7f5421800640 1 --2- 172.21.15.69:0/4177934463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f541c108df0 0x7f541c109250 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.349+0000 7f5421800640 1 --2- 172.21.15.69:0/4177934463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f541c06dae0 0x7f541c108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.349+0000 7f5421800640 1 -- 172.21.15.69:0/4177934463 >> 172.21.15.69:0/4177934463 conn(0x7f541c06c700 msgr2=0x7f541c06cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.349+0000 7f5421800640 1 -- 172.21.15.69:0/4177934463 shutdown_connections 2024-03-19T01:20:19.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.349+0000 7f5421800640 1 -- 172.21.15.69:0/4177934463 wait complete. 2024-03-19T01:20:19.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.351+0000 7f5421800640 1 Processor -- start 2024-03-19T01:20:19.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.355+0000 7f5421800640 1 -- start start 2024-03-19T01:20:19.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.356+0000 7f5421800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f541c06dae0 0x7f541c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.356+0000 7f5421800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f541c108df0 0x7f541c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.356+0000 7f5421800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f541c1144c0 0x7f541c1a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.356+0000 7f5421800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f541c11a230 con 0x7f541c1144c0 2024-03-19T01:20:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.356+0000 7f5421800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f541c11a0b0 con 0x7f541c06dae0 2024-03-19T01:20:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.356+0000 7f5421800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f541c11a3b0 con 0x7f541c108df0 2024-03-19T01:20:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.356+0000 7f541a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f541c108df0 0x7f541c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.357+0000 7f541aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f541c06dae0 0x7f541c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.357+0000 7f541a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f541c108df0 0x7f541c1a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38748/0 (socket says 172.21.15.69:38748) 2024-03-19T01:20:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.357+0000 7f541a000640 1 -- 172.21.15.69:0/2152695254 learned_addr learned my addr 172.21.15.69:0/2152695254 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.357+0000 7f541b400640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f541c1144c0 0x7f541c1a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.357+0000 7f541a000640 1 -- 172.21.15.69:0/2152695254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f541c06dae0 msgr2=0x7f541c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.357+0000 7f541a000640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f541c06dae0 0x7f541c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.357+0000 7f541a000640 1 -- 172.21.15.69:0/2152695254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f541c1144c0 msgr2=0x7f541c1a4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:19.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.357+0000 7f541a000640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f541c1144c0 0x7f541c1a4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.357+0000 7f541a000640 1 -- 172.21.15.69:0/2152695254 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c1a5490 con 0x7f541c108df0 2024-03-19T01:20:19.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.357+0000 7f541b400640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f541c1144c0 0x7f541c1a4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:19.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.358+0000 7f541a000640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f541c108df0 0x7f541c1a0bd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5414009430 tx=0x7f5414035e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:19.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.359+0000 7f5413400640 1 -- 172.21.15.69:0/2152695254 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414039040 con 0x7f541c108df0 2024-03-19T01:20:19.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.359+0000 7f5413400640 1 -- 172.21.15.69:0/2152695254 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5414044070 con 0x7f541c108df0 2024-03-19T01:20:19.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.359+0000 7f5421800640 1 -- 172.21.15.69:0/2152695254 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c1a5720 con 0x7f541c108df0 2024-03-19T01:20:19.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.359+0000 7f5413400640 1 -- 172.21.15.69:0/2152695254 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414004600 con 0x7f541c108df0 2024-03-19T01:20:19.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.359+0000 7f5421800640 1 -- 172.21.15.69:0/2152695254 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c1a5c90 con 0x7f541c108df0 2024-03-19T01:20:19.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.366+0000 7f5413400640 1 -- 172.21.15.69:0/2152695254 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f5414003970 con 0x7f541c108df0 2024-03-19T01:20:19.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.368+0000 7f5413400640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54000a2b40 0x7f54000a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.368+0000 7f5413400640 1 -- 172.21.15.69:0/2152695254 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f54140f1830 con 0x7f541c108df0 2024-03-19T01:20:19.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.369+0000 7f541aa00640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54000a2b40 0x7f54000a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.373+0000 7f541aa00640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54000a2b40 0x7f54000a5000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f540c00abe0 tx=0x7f540c00a2d0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:19.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.374+0000 7f54245eb740 1 Processor -- start 2024-03-19T01:20:19.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.375+0000 7f54245eb740 1 -- start start 2024-03-19T01:20:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.375+0000 7f54245eb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad612c10 0x5632ad612ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.375+0000 7f54245eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad613530 0x5632ad61bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.375+0000 7f54245eb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 0x5632ad61e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.375+0000 7f54245eb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5632ad577380 con 0x5632ad61c110 2024-03-19T01:20:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.375+0000 7f54245eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5632ad51fd20 con 0x5632ad613530 2024-03-19T01:20:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.375+0000 7f54245eb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5632ad573420 con 0x5632ad612c10 2024-03-19T01:20:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.375+0000 7f541aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad612c10 0x5632ad612ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.376+0000 7f541aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad612c10 0x5632ad612ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38752/0 (socket says 172.21.15.69:38752) 2024-03-19T01:20:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.376+0000 7f541aa00640 1 -- 172.21.15.69:0/1383300225 learned_addr learned my addr 172.21.15.69:0/1383300225 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.376+0000 7f541aa00640 1 -- 172.21.15.69:0/1383300225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad613530 msgr2=0x5632ad61bbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.376+0000 7f541aa00640 1 --2- 172.21.15.69:0/1383300225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad613530 0x5632ad61bbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.376+0000 7f541aa00640 1 -- 172.21.15.69:0/1383300225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 msgr2=0x5632ad61e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.376+0000 7f541b400640 1 --2- 172.21.15.69:0/1383300225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 0x5632ad61e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.376+0000 7f541aa00640 1 --2- 172.21.15.69:0/1383300225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 0x5632ad61e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.376+0000 7f541aa00640 1 -- 172.21.15.69:0/1383300225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ad28b3f0 con 0x5632ad612c10 2024-03-19T01:20:19.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.376+0000 7f541b400640 1 --2- 172.21.15.69:0/1383300225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 0x5632ad61e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:19.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.377+0000 7f541aa00640 1 --2- 172.21.15.69:0/1383300225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad612c10 0x5632ad612ff0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f540c00ffd0 tx=0x7f540c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3dbf0ea29debb08 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:19.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.377+0000 7f5407e00640 1 -- 172.21.15.69:0/1383300225 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c016070 con 0x5632ad612c10 2024-03-19T01:20:19.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.377+0000 7f5407e00640 1 -- 172.21.15.69:0/1383300225 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f540c01b900 con 0x5632ad612c10 2024-03-19T01:20:19.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.377+0000 7f5407e00640 1 -- 172.21.15.69:0/1383300225 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c01c9b0 con 0x5632ad612c10 2024-03-19T01:20:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.378+0000 7f54245eb740 1 -- 172.21.15.69:0/1383300225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad612c10 msgr2=0x5632ad612ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.378+0000 7f54245eb740 1 --2- 172.21.15.69:0/1383300225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad612c10 0x5632ad612ff0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f540c00ffd0 tx=0x7f540c012000 comp rx=0 tx=0).stop 2024-03-19T01:20:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.379+0000 7f54245eb740 1 -- 172.21.15.69:0/1383300225 shutdown_connections 2024-03-19T01:20:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.379+0000 7f54245eb740 1 --2- 172.21.15.69:0/1383300225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 0x5632ad61e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.379+0000 7f54245eb740 1 --2- 172.21.15.69:0/1383300225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad613530 0x5632ad61bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.379+0000 7f54245eb740 1 --2- 172.21.15.69:0/1383300225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad612c10 0x5632ad612ff0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.379+0000 7f54245eb740 1 -- 172.21.15.69:0/1383300225 >> 172.21.15.69:0/1383300225 conn(0x5632ad60d340 msgr2=0x5632ad60d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.381+0000 7f54245eb740 1 -- 172.21.15.69:0/1383300225 shutdown_connections 2024-03-19T01:20:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.381+0000 7f54245eb740 1 -- 172.21.15.69:0/1383300225 wait complete. 2024-03-19T01:20:19.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.381+0000 7f54245eb740 1 -- start start 2024-03-19T01:20:19.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.382+0000 7f54245eb740 1 Processor -- start 2024-03-19T01:20:19.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.386+0000 7f54245eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad612c10 0x5632ad619250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.386+0000 7f54245eb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 0x5632ad619790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.386+0000 7f54245eb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad619cd0 0x5632ad628370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.386+0000 7f54245eb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5632ad61ef00 con 0x5632ad61c110 2024-03-19T01:20:19.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.386+0000 7f54245eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5632ad533160 con 0x5632ad612c10 2024-03-19T01:20:19.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.386+0000 7f54245eb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5632ad61f080 con 0x5632ad619cd0 2024-03-19T01:20:19.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.386+0000 7f54245eb740 10 client.0 _ll_register_callbacks cb 0x5632ad56f170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:19.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad612c10 0x5632ad619250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad612c10 0x5632ad619250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60720/0 (socket says 172.21.15.69:60720) 2024-03-19T01:20:19.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541aa00640 1 -- 172.21.15.69:0/1407700105 learned_addr learned my addr 172.21.15.69:0/1407700105 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:19.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541a000640 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 0x5632ad619790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541aa00640 1 -- 172.21.15.69:0/1407700105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad619cd0 msgr2=0x5632ad628370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:19.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541b400640 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad619cd0 0x5632ad628370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541aa00640 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad619cd0 0x5632ad628370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541aa00640 1 -- 172.21.15.69:0/1407700105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 msgr2=0x5632ad619790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:19.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541aa00640 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 0x5632ad619790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541aa00640 1 -- 172.21.15.69:0/1407700105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ad3557d0 con 0x5632ad612c10 2024-03-19T01:20:19.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541a000640 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 0x5632ad619790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:19.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.387+0000 7f541b400640 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad619cd0 0x5632ad628370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:19.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.388+0000 7f541aa00640 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad612c10 0x5632ad619250 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f540c01bd60 tx=0x7f540c01bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c37c5a7a2441ee46 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:19.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.388+0000 7f5404c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:20:19.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.389+0000 7f5404c00640 1 -- 172.21.15.69:0/1407700105 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c033060 con 0x5632ad612c10 2024-03-19T01:20:19.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.389+0000 7f5404c00640 1 -- 172.21.15.69:0/1407700105 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f540c01b6a0 con 0x5632ad612c10 2024-03-19T01:20:19.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.389+0000 7f54245eb740 1 -- 172.21.15.69:0/1407700105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f540c01b900 con 0x5632ad612c10 2024-03-19T01:20:19.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.389+0000 7f5404c00640 1 -- 172.21.15.69:0/1407700105 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f540c03a050 con 0x5632ad612c10 2024-03-19T01:20:19.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.389+0000 7f54245eb740 10 client.27653 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:19.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.389+0000 7f54245eb740 1 -- 172.21.15.69:0/1407700105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5632ad62b6a0 con 0x5632ad612c10 2024-03-19T01:20:19.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.390+0000 7f5404c00640 1 -- 172.21.15.69:0/1407700105 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f540c01b900 con 0x5632ad612c10 2024-03-19T01:20:19.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.390+0000 7f5404c00640 1 -- 172.21.15.69:0/1407700105 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f540c02d030 con 0x5632ad612c10 2024-03-19T01:20:19.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.390+0000 7f54245eb740 10 client.27653 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:19.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.390+0000 7f54245eb740 10 client.27653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:20:19.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.390+0000 7f54245eb740 20 client.27653 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:19.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.390+0000 7f54245eb740 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5632ad62cb00 0x5632ad62eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.390+0000 7f54245eb740 4 client.27653 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.390+0000 7f54245eb740 1 -- 172.21.15.69:0/1407700105 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632ad5848c0 con 0x5632ad62cb00 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.391+0000 7f541a000640 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5632ad62cb00 0x5632ad62eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.391+0000 7f541a000640 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5632ad62cb00 0x5632ad62eef0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa017303e2806ad server_cookie=841ca498d883f0a3 in_seq=0 out_seq=0 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.391+0000 7f5404c00640 10 client.27653 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.392+0000 7f5404c00640 1 -- 172.21.15.69:0/1407700105 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5632ad5848c0 con 0x5632ad62cb00 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.392+0000 7f5404c00640 10 client.27653 handle_command_reply: tid=0 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.392+0000 7f54245eb740 1 client.27653 shutdown 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.392+0000 7f54245eb740 1 -- 172.21.15.69:0/1407700105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad612c10 msgr2=0x5632ad619250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.392+0000 7f54245eb740 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad612c10 0x5632ad619250 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f540c01bd60 tx=0x7f540c01bd90 comp rx=0 tx=0).stop 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.393+0000 7f54245eb740 1 -- 172.21.15.69:0/1407700105 shutdown_connections 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.393+0000 7f54245eb740 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5632ad62cb00 0x5632ad62eef0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.393+0000 7f54245eb740 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad619cd0 0x5632ad628370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.393+0000 7f54245eb740 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad61c110 0x5632ad619790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.393+0000 7f54245eb740 1 --2- 172.21.15.69:0/1407700105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad612c10 0x5632ad619250 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.393+0000 7f5404c00640 0 client.27653 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:19.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.393+0000 7f54245eb740 1 -- 172.21.15.69:0/1407700105 >> 172.21.15.69:0/1407700105 conn(0x5632ad60d340 msgr2=0x5632ad60f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:19.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.393+0000 7f54245eb740 1 -- 172.21.15.69:0/1407700105 shutdown_connections 2024-03-19T01:20:19.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.393+0000 7f54245eb740 1 -- 172.21.15.69:0/1407700105 wait complete. 2024-03-19T01:20:19.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.394+0000 7f54245eb740 20 client.27653 trim_cache size 0 max 16384 2024-03-19T01:20:19.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.452+0000 7f54245eb740 1 Processor -- start 2024-03-19T01:20:19.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.456+0000 7f54245eb740 1 -- start start 2024-03-19T01:20:19.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.456+0000 7f54245eb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad619cd0 0x5632ad6282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.456+0000 7f54245eb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad61c110 0x5632ad6183b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.456+0000 7f54245eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad608780 0x5632ad60c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.456+0000 7f54245eb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1bdb20 con 0x5632ad619cd0 2024-03-19T01:20:19.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.456+0000 7f54245eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1bf660 con 0x5632ad608780 2024-03-19T01:20:19.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.456+0000 7f54245eb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1be1f0 con 0x5632ad61c110 2024-03-19T01:20:19.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.456+0000 7f541b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad608780 0x5632ad60c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.457+0000 7f541b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad608780 0x5632ad60c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60730/0 (socket says 172.21.15.69:60730) 2024-03-19T01:20:19.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.457+0000 7f541b400640 1 -- 172.21.15.69:0/1796271036 learned_addr learned my addr 172.21.15.69:0/1796271036 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:19.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.457+0000 7f541aa00640 1 --2- 172.21.15.69:0/1796271036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad619cd0 0x5632ad6282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.457+0000 7f541a000640 1 --2- 172.21.15.69:0/1796271036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad61c110 0x5632ad6183b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.457+0000 7f541b400640 1 -- 172.21.15.69:0/1796271036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad61c110 msgr2=0x5632ad6183b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:19.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.457+0000 7f541b400640 1 --2- 172.21.15.69:0/1796271036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad61c110 0x5632ad6183b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.457+0000 7f541b400640 1 -- 172.21.15.69:0/1796271036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad619cd0 msgr2=0x5632ad6282e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:19.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.457+0000 7f541b400640 1 --2- 172.21.15.69:0/1796271036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad619cd0 0x5632ad6282e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.457+0000 7f541b400640 1 -- 172.21.15.69:0/1796271036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ad608d70 con 0x5632ad608780 2024-03-19T01:20:19.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.457+0000 7f541aa00640 1 --2- 172.21.15.69:0/1796271036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad619cd0 0x5632ad6282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:19.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.458+0000 7f541b400640 1 --2- 172.21.15.69:0/1796271036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad608780 0x5632ad60c260 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f540800c930 tx=0x7f540800cdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1376a7af585300f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:19.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.458+0000 7f5407400640 1 -- 172.21.15.69:0/1796271036 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540800ece0 con 0x5632ad608780 2024-03-19T01:20:19.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.458+0000 7f5407400640 1 -- 172.21.15.69:0/1796271036 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f540800ee80 con 0x5632ad608780 2024-03-19T01:20:19.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.459+0000 7f5407400640 1 -- 172.21.15.69:0/1796271036 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408011850 con 0x5632ad608780 2024-03-19T01:20:19.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.460+0000 7f54245eb740 1 -- 172.21.15.69:0/1796271036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad608780 msgr2=0x5632ad60c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:19.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.460+0000 7f54245eb740 1 --2- 172.21.15.69:0/1796271036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad608780 0x5632ad60c260 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f540800c930 tx=0x7f540800cdf0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.461+0000 7f54245eb740 1 -- 172.21.15.69:0/1796271036 shutdown_connections 2024-03-19T01:20:19.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.461+0000 7f54245eb740 1 --2- 172.21.15.69:0/1796271036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad608780 0x5632ad60c260 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.461+0000 7f54245eb740 1 --2- 172.21.15.69:0/1796271036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad61c110 0x5632ad6183b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.461+0000 7f54245eb740 1 --2- 172.21.15.69:0/1796271036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad619cd0 0x5632ad6282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.461+0000 7f54245eb740 1 -- 172.21.15.69:0/1796271036 >> 172.21.15.69:0/1796271036 conn(0x5632ad60d340 msgr2=0x5632ad60bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:19.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.464+0000 7f54245eb740 1 -- 172.21.15.69:0/1796271036 shutdown_connections 2024-03-19T01:20:19.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.464+0000 7f54245eb740 1 -- 172.21.15.69:0/1796271036 wait complete. 2024-03-19T01:20:19.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.464+0000 7f54245eb740 1 -- start start 2024-03-19T01:20:19.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.465+0000 7f54245eb740 1 Processor -- start 2024-03-19T01:20:19.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.467+0000 7f54245eb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad608780 0x5632ad631e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.467+0000 7f54245eb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad619cd0 0x5632ad62f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.467+0000 7f54245eb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad61c110 0x5632ad62f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.467+0000 7f54245eb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1bef90 con 0x5632ad608780 2024-03-19T01:20:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.467+0000 7f54245eb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1be8c0 con 0x5632ad61c110 2024-03-19T01:20:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.467+0000 7f54245eb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1bbb60 con 0x5632ad619cd0 2024-03-19T01:20:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.467+0000 7f54245eb740 10 client.0 _ll_register_callbacks cb 0x7f540c00fea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.468+0000 7f541aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad608780 0x5632ad631e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.468+0000 7f541aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad608780 0x5632ad631e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43934/0 (socket says 172.21.15.69:43934) 2024-03-19T01:20:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.468+0000 7f541aa00640 1 -- 172.21.15.69:0/2996116552 learned_addr learned my addr 172.21.15.69:0/2996116552 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.468+0000 7f541b400640 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad61c110 0x5632ad62f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.468+0000 7f541aa00640 1 -- 172.21.15.69:0/2996116552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad619cd0 msgr2=0x5632ad62f1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.468+0000 7f541aa00640 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad619cd0 0x5632ad62f1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.468+0000 7f541aa00640 1 -- 172.21.15.69:0/2996116552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad61c110 msgr2=0x5632ad62f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.468+0000 7f541aa00640 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad61c110 0x5632ad62f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.468+0000 7f541aa00640 1 -- 172.21.15.69:0/2996116552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ad62fc70 con 0x5632ad608780 2024-03-19T01:20:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.469+0000 7f541aa00640 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad608780 0x5632ad631e60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5632ad618ae0 tx=0x7f540c010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef28b11cf472d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.473+0000 7f5404c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:20:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.473+0000 7f5404c00640 1 -- 172.21.15.69:0/2996116552 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c043060 con 0x5632ad608780 2024-03-19T01:20:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.473+0000 7f5404c00640 1 -- 172.21.15.69:0/2996116552 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f540c01b630 con 0x5632ad608780 2024-03-19T01:20:19.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.473+0000 7f54245eb740 1 -- 172.21.15.69:0/2996116552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5632ad57ec10 con 0x5632ad608780 2024-03-19T01:20:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.473+0000 7f5404c00640 1 -- 172.21.15.69:0/2996116552 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f540c016040 con 0x5632ad608780 2024-03-19T01:20:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.474+0000 7f54245eb740 10 client.17340 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.474+0000 7f54245eb740 1 -- 172.21.15.69:0/2996116552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5632ad692750 con 0x5632ad608780 2024-03-19T01:20:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.474+0000 7f5404c00640 1 -- 172.21.15.69:0/2996116552 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5632ad57ec10 con 0x5632ad608780 2024-03-19T01:20:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.474+0000 7f5404c00640 1 -- 172.21.15.69:0/2996116552 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f540c02d040 con 0x5632ad608780 2024-03-19T01:20:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.474+0000 7f54245eb740 10 client.17340 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.474+0000 7f54245eb740 10 client.17340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:20:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.474+0000 7f54245eb740 20 client.17340 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.474+0000 7f54245eb740 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5632ad692e50 0x5632ad695270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.474+0000 7f54245eb740 4 client.17340 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.474+0000 7f54245eb740 1 -- 172.21.15.69:0/2996116552 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632ad4fc560 con 0x5632ad692e50 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.475+0000 7f541a000640 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5632ad692e50 0x5632ad695270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.476+0000 7f541a000640 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5632ad692e50 0x5632ad695270 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d2dd78be48fa0f server_cookie=a37670b4dbeb64f in_seq=0 out_seq=0 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.476+0000 7f5404c00640 10 client.17340 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.476+0000 7f5404c00640 1 -- 172.21.15.69:0/2996116552 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632ad4fc560 con 0x5632ad692e50 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.476+0000 7f5404c00640 10 client.17340 handle_command_reply: tid=0 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.476+0000 7f54245eb740 1 client.17340 shutdown 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.476+0000 7f54245eb740 1 -- 172.21.15.69:0/2996116552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad608780 msgr2=0x5632ad631e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.476+0000 7f54245eb740 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad608780 0x5632ad631e60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5632ad618ae0 tx=0x7f540c010f80 comp rx=0 tx=0).stop 2024-03-19T01:20:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.477+0000 7f54245eb740 1 -- 172.21.15.69:0/2996116552 shutdown_connections 2024-03-19T01:20:19.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.477+0000 7f54245eb740 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5632ad692e50 0x5632ad695270 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.477+0000 7f54245eb740 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632ad61c110 0x5632ad62f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.477+0000 7f54245eb740 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632ad619cd0 0x5632ad62f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.477+0000 7f54245eb740 1 --2- 172.21.15.69:0/2996116552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632ad608780 0x5632ad631e60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.477+0000 7f54245eb740 1 -- 172.21.15.69:0/2996116552 >> 172.21.15.69:0/2996116552 conn(0x5632ad60d340 msgr2=0x5632ad60e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:19.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.477+0000 7f5404c00640 0 client.17340 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:19.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.477+0000 7f54245eb740 1 -- 172.21.15.69:0/2996116552 shutdown_connections 2024-03-19T01:20:19.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.477+0000 7f54245eb740 1 -- 172.21.15.69:0/2996116552 wait complete. 2024-03-19T01:20:19.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.477+0000 7f54245eb740 20 client.17340 trim_cache size 0 max 16384 2024-03-19T01:20:19.481 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:20:19.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.483+0000 7f5421800640 1 -- 172.21.15.69:0/2152695254 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54000a2b40 msgr2=0x7f54000a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:19.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.483+0000 7f5421800640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54000a2b40 0x7f54000a5000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f540c00abe0 tx=0x7f540c00a2d0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.483+0000 7f5421800640 1 -- 172.21.15.69:0/2152695254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f541c108df0 msgr2=0x7f541c1a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:19.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.483+0000 7f5421800640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f541c108df0 0x7f541c1a0bd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5414009430 tx=0x7f5414035e50 comp rx=0 tx=0).stop 2024-03-19T01:20:19.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.483+0000 7f5421800640 1 -- 172.21.15.69:0/2152695254 shutdown_connections 2024-03-19T01:20:19.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.483+0000 7f5421800640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f541c1144c0 0x7f541c1a4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.483+0000 7f5421800640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f541c108df0 0x7f541c1a0bd0 secure :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5414009430 tx=0x7f5414035e50 comp rx=0 tx=0).stop 2024-03-19T01:20:19.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.483+0000 7f5421800640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54000a2b40 0x7f54000a5000 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.483+0000 7f5421800640 1 --2- 172.21.15.69:0/2152695254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f541c06dae0 0x7f541c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:19.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.483+0000 7f5421800640 1 -- 172.21.15.69:0/2152695254 >> 172.21.15.69:0/2152695254 conn(0x7f541c06c700 msgr2=0x7f541c0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:19.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.487+0000 7f5421800640 1 -- 172.21.15.69:0/2152695254 shutdown_connections 2024-03-19T01:20:19.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:19.487+0000 7f5421800640 1 -- 172.21.15.69:0/2152695254 wait complete. 2024-03-19T01:20:19.518 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/isofs/rock.o 2024-03-19T01:20:19.561 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cgroup/debug.o 2024-03-19T01:20:19.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:19 smithi086 ceph-mon[66829]: pgmap v300: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.0 MiB/s wr, 75 op/s 2024-03-19T01:20:19.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:19 smithi069 ceph-mon[31727]: pgmap v300: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.0 MiB/s wr, 75 op/s 2024-03-19T01:20:19.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:19 smithi163 ceph-mon[67428]: pgmap v300: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.0 MiB/s wr, 75 op/s 2024-03-19T01:20:19.823 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_nop.o 2024-03-19T01:20:20.020 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/setup-bus.o 2024-03-19T01:20:20.048 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/fat/built-in.a 2024-03-19T01:20:20.196 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/isofs/export.o 2024-03-19T01:20:20.428 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpi-inv.o 2024-03-19T01:20:20.544 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:20:20.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:20:20.545 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:20:20.623 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/cgroup/built-in.a 2024-03-19T01:20:20.717 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/symlink.o 2024-03-19T01:20:20.730 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/blktrace.o 2024-03-19T01:20:20.786 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/isofs/joliet.o 2024-03-19T01:20:20.990 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/events/ring_buffer.o 2024-03-19T01:20:21.175 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/vc.o 2024-03-19T01:20:21.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.262+0000 7f37d2000640 1 -- 172.21.15.69:0/2115570450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37cc108a00 msgr2=0x7f37cc108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:21.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.262+0000 7f37d2000640 1 --2- 172.21.15.69:0/2115570450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37cc108a00 0x7f37cc108de0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009c30 tx=0x7f37c402bf60 comp rx=0 tx=0).stop 2024-03-19T01:20:21.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.265+0000 7f37d2000640 1 -- 172.21.15.69:0/2115570450 shutdown_connections 2024-03-19T01:20:21.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.265+0000 7f37d2000640 1 --2- 172.21.15.69:0/2115570450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37cc114970 0x7f37cc116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.265+0000 7f37d2000640 1 --2- 172.21.15.69:0/2115570450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc109320 0x7f37cc114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.265+0000 7f37d2000640 1 --2- 172.21.15.69:0/2115570450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37cc108a00 0x7f37cc108de0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.265+0000 7f37d2000640 1 -- 172.21.15.69:0/2115570450 >> 172.21.15.69:0/2115570450 conn(0x7f37cc06c900 msgr2=0x7f37cc06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:21.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.265+0000 7f37d2000640 1 -- 172.21.15.69:0/2115570450 shutdown_connections 2024-03-19T01:20:21.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.265+0000 7f37d2000640 1 -- 172.21.15.69:0/2115570450 wait complete. 2024-03-19T01:20:21.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.268+0000 7f37d2000640 1 Processor -- start 2024-03-19T01:20:21.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.269+0000 7f37d2000640 1 -- start start 2024-03-19T01:20:21.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.269+0000 7f37d2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc108a00 0x7f37cc10be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.269+0000 7f37d2000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37cc109320 0x7f37cc10a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.270+0000 7f37caa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37cc109320 0x7f37cc10a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.270+0000 7f37caa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37cc109320 0x7f37cc10a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38770/0 (socket says 172.21.15.69:38770) 2024-03-19T01:20:21.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.273+0000 7f37cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc108a00 0x7f37cc10be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.273+0000 7f37cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc108a00 0x7f37cc10be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60768/0 (socket says 172.21.15.69:60768) 2024-03-19T01:20:21.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.274+0000 7f37d2000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37cc114970 0x7f37cc10a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.274+0000 7f37d2000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc11a3d0 con 0x7f37cc114970 2024-03-19T01:20:21.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.274+0000 7f37d2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc11a250 con 0x7f37cc108a00 2024-03-19T01:20:21.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.274+0000 7f37d2000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc11a550 con 0x7f37cc109320 2024-03-19T01:20:21.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.274+0000 7f37caa00640 1 -- 172.21.15.69:0/2645110297 learned_addr learned my addr 172.21.15.69:0/2645110297 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:21.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.274+0000 7f37cb400640 1 -- 172.21.15.69:0/2645110297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37cc109320 msgr2=0x7f37cc10a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:21.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.274+0000 7f37cb400640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37cc109320 0x7f37cc10a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.274+0000 7f37cb400640 1 -- 172.21.15.69:0/2645110297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37cc114970 msgr2=0x7f37cc10a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:21.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.274+0000 7f37cb400640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37cc114970 0x7f37cc10a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.274+0000 7f37cb400640 1 -- 172.21.15.69:0/2645110297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc1b3270 con 0x7f37cc108a00 2024-03-19T01:20:21.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.274+0000 7f37caa00640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37cc109320 0x7f37cc10a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:21.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.275+0000 7f37cb400640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc108a00 0x7f37cc10be90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009b80 tx=0x7f37c4033910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:21.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.275+0000 7f37c3e00640 1 -- 172.21.15.69:0/2645110297 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4043020 con 0x7f37cc108a00 2024-03-19T01:20:21.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.275+0000 7f37c3e00640 1 -- 172.21.15.69:0/2645110297 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37c4003e40 con 0x7f37cc108a00 2024-03-19T01:20:21.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.275+0000 7f37c3e00640 1 -- 172.21.15.69:0/2645110297 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c402de60 con 0x7f37cc108a00 2024-03-19T01:20:21.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.275+0000 7f37d2000640 1 -- 172.21.15.69:0/2645110297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc1b3440 con 0x7f37cc108a00 2024-03-19T01:20:21.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.276+0000 7f37d2000640 1 -- 172.21.15.69:0/2645110297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc1b3910 con 0x7f37cc108a00 2024-03-19T01:20:21.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.285+0000 7f37c3e00640 1 -- 172.21.15.69:0/2645110297 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f37c4046020 con 0x7f37cc108a00 2024-03-19T01:20:21.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.287+0000 7f37c3e00640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37b00a2a70 0x7f37b00a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.287+0000 7f37c3e00640 1 -- 172.21.15.69:0/2645110297 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f37c40f1cb0 con 0x7f37cc108a00 2024-03-19T01:20:21.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.287+0000 7f37caa00640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37b00a2a70 0x7f37b00a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.289+0000 7f37d4c65740 1 Processor -- start 2024-03-19T01:20:21.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.289+0000 7f37d4c65740 1 -- start start 2024-03-19T01:20:21.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.289+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b7dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.290+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b7e420 0x55b296b86ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.290+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b87000 0x55b296b893f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.290+0000 7f37d4c65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b296ae0fb0 con 0x55b296b87000 2024-03-19T01:20:21.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.290+0000 7f37d4c65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b296a89ab0 con 0x55b296b7e420 2024-03-19T01:20:21.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.290+0000 7f37d4c65740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b296add050 con 0x55b296b7db00 2024-03-19T01:20:21.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.290+0000 7f37cbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b87000 0x55b296b893f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.290+0000 7f37cbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b87000 0x55b296b893f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43968/0 (socket says 172.21.15.69:43968) 2024-03-19T01:20:21.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.290+0000 7f37cbe00640 1 -- 172.21.15.69:0/1773442335 learned_addr learned my addr 172.21.15.69:0/1773442335 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:21.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.291+0000 7f37caa00640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37b00a2a70 0x7f37b00a4f30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f37cc10ba00 tx=0x7f37bc009260 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:21.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.291+0000 7f37cbe00640 1 -- 172.21.15.69:0/1773442335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 msgr2=0x55b296b7dee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:21.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.291+0000 7f37cbe00640 1 --2- 172.21.15.69:0/1773442335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b7dee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.291+0000 7f37caa00640 1 --2- 172.21.15.69:0/1773442335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b7e420 0x55b296b86ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.291+0000 7f37cbe00640 1 -- 172.21.15.69:0/1773442335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b7e420 msgr2=0x55b296b86ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:21.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.291+0000 7f37cbe00640 1 --2- 172.21.15.69:0/1773442335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b7e420 0x55b296b86ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.291+0000 7f37cbe00640 1 -- 172.21.15.69:0/1773442335 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b296aa7e00 con 0x55b296b87000 2024-03-19T01:20:21.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.291+0000 7f37caa00640 1 --2- 172.21.15.69:0/1773442335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b7e420 0x55b296b86ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:21.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.292+0000 7f37cbe00640 1 --2- 172.21.15.69:0/1773442335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b87000 0x55b296b893f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009bb0 tx=0x7f37b800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c420f8f1945fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:21.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.293+0000 7f37b7e00640 1 -- 172.21.15.69:0/1773442335 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8005020 con 0x55b296b87000 2024-03-19T01:20:21.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.293+0000 7f37b7e00640 1 -- 172.21.15.69:0/1773442335 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b80047a0 con 0x55b296b87000 2024-03-19T01:20:21.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.293+0000 7f37b7e00640 1 -- 172.21.15.69:0/1773442335 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8010720 con 0x55b296b87000 2024-03-19T01:20:21.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.294+0000 7f37d4c65740 1 -- 172.21.15.69:0/1773442335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b87000 msgr2=0x55b296b893f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:21.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.294+0000 7f37d4c65740 1 --2- 172.21.15.69:0/1773442335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b87000 0x55b296b893f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009bb0 tx=0x7f37b800e720 comp rx=0 tx=0).stop 2024-03-19T01:20:21.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.299+0000 7f37d4c65740 1 -- 172.21.15.69:0/1773442335 shutdown_connections 2024-03-19T01:20:21.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.299+0000 7f37d4c65740 1 --2- 172.21.15.69:0/1773442335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b87000 0x55b296b893f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.299+0000 7f37d4c65740 1 --2- 172.21.15.69:0/1773442335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b7e420 0x55b296b86ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.299+0000 7f37d4c65740 1 --2- 172.21.15.69:0/1773442335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b7dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.299+0000 7f37d4c65740 1 -- 172.21.15.69:0/1773442335 >> 172.21.15.69:0/1773442335 conn(0x55b296b78230 msgr2=0x55b296b78610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:21.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.300+0000 7f37d4c65740 1 -- 172.21.15.69:0/1773442335 shutdown_connections 2024-03-19T01:20:21.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.300+0000 7f37d4c65740 1 -- 172.21.15.69:0/1773442335 wait complete. 2024-03-19T01:20:21.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.301+0000 7f37d4c65740 1 -- start start 2024-03-19T01:20:21.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.301+0000 7f37d4c65740 1 Processor -- start 2024-03-19T01:20:21.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.302+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b84190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.302+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 0x55b296b846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.302+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296b84c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.302+0000 7f37d4c65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b296b89df0 con 0x55b296b7e420 2024-03-19T01:20:21.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.302+0000 7f37d4c65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b296a9cf00 con 0x55b296b87000 2024-03-19T01:20:21.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.302+0000 7f37d4c65740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b296b89f70 con 0x55b296b7db00 2024-03-19T01:20:21.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.302+0000 7f37d4c65740 10 client.0 _ll_register_callbacks cb 0x55b296a9c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:21.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.303+0000 7f37caa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 0x55b296b846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.303+0000 7f37caa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 0x55b296b846d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43978/0 (socket says 172.21.15.69:43978) 2024-03-19T01:20:21.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.303+0000 7f37caa00640 1 -- 172.21.15.69:0/855461112 learned_addr learned my addr 172.21.15.69:0/855461112 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:21.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.303+0000 7f37cbe00640 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296b84c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.303+0000 7f37cb400640 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b84190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.303+0000 7f37cbe00640 1 -- 172.21.15.69:0/855461112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 msgr2=0x55b296b84190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:21.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.303+0000 7f37cbe00640 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b84190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.303+0000 7f37cbe00640 1 -- 172.21.15.69:0/855461112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 msgr2=0x55b296b846d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:21.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.303+0000 7f37cbe00640 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 0x55b296b846d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.304+0000 7f37cbe00640 1 -- 172.21.15.69:0/855461112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2968bd7e0 con 0x55b296b87000 2024-03-19T01:20:21.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.304+0000 7f37cb400640 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b84190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:20:21.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.304+0000 7f37caa00640 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 0x55b296b846d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:21.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.304+0000 7f37cbe00640 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296b84c10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009b80 tx=0x7f37b8009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c3bbd1c9b33541 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:21.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.304+0000 7f37b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:20:21.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.305+0000 7f37b4c00640 1 -- 172.21.15.69:0/855461112 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b801d040 con 0x55b296b87000 2024-03-19T01:20:21.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.305+0000 7f37b4c00640 1 -- 172.21.15.69:0/855461112 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f37b8005200 con 0x55b296b87000 2024-03-19T01:20:21.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.306+0000 7f37d4c65740 1 -- 172.21.15.69:0/855461112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b80047a0 con 0x55b296b87000 2024-03-19T01:20:21.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.307+0000 7f37b4c00640 1 -- 172.21.15.69:0/855461112 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f37b80097d0 con 0x55b296b87000 2024-03-19T01:20:21.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.307+0000 7f37d4c65740 10 client.27668 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:21.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.307+0000 7f37d4c65740 1 -- 172.21.15.69:0/855461112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55b29691df70 con 0x55b296b87000 2024-03-19T01:20:21.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.307+0000 7f37b4c00640 1 -- 172.21.15.69:0/855461112 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b80047a0 con 0x55b296b87000 2024-03-19T01:20:21.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.307+0000 7f37b4c00640 1 -- 172.21.15.69:0/855461112 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f37b8010cc0 con 0x55b296b87000 2024-03-19T01:20:21.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.308+0000 7f37d4c65740 10 client.27668 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:21.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.308+0000 7f37d4c65740 10 client.27668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:20:21.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.308+0000 7f37d4c65740 20 client.27668 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:21.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.308+0000 7f37d4c65740 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b296ba8350 0x55b296baa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.308+0000 7f37d4c65740 4 client.27668 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:20:21.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.308+0000 7f37d4c65740 1 -- 172.21.15.69:0/855461112 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b296aee4f0 con 0x55b296ba8350 2024-03-19T01:20:21.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.309+0000 7f37cb400640 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b296ba8350 0x55b296baa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.309+0000 7f37cb400640 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b296ba8350 0x55b296baa740 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb1da98d2459ed10 server_cookie=2322c907f4eddba7 in_seq=0 out_seq=0 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.309+0000 7f37b4c00640 10 client.27668 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.310+0000 7f37b4c00640 1 -- 172.21.15.69:0/855461112 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b296aee4f0 con 0x55b296ba8350 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.310+0000 7f37b4c00640 10 client.27668 handle_command_reply: tid=0 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.310+0000 7f37d4c65740 1 client.27668 shutdown 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.312+0000 7f37d4c65740 1 -- 172.21.15.69:0/855461112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 msgr2=0x55b296b84c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.312+0000 7f37d4c65740 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296b84c10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009b80 tx=0x7f37b8009f40 comp rx=0 tx=0).stop 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.312+0000 7f37d4c65740 1 -- 172.21.15.69:0/855461112 shutdown_connections 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.312+0000 7f37d4c65740 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b296ba8350 0x55b296baa740 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.312+0000 7f37d4c65740 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296b84c10 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.312+0000 7f37d4c65740 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 0x55b296b846d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.312+0000 7f37d4c65740 1 --2- 172.21.15.69:0/855461112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b84190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.312+0000 7f37b4c00640 0 client.27668 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:21.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.312+0000 7f37d4c65740 1 -- 172.21.15.69:0/855461112 >> 172.21.15.69:0/855461112 conn(0x55b296b78230 msgr2=0x55b296b79f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:21.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.314+0000 7f37d4c65740 1 -- 172.21.15.69:0/855461112 shutdown_connections 2024-03-19T01:20:21.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.314+0000 7f37d4c65740 1 -- 172.21.15.69:0/855461112 wait complete. 2024-03-19T01:20:21.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.314+0000 7f37d4c65740 20 client.27668 trim_cache size 0 max 16384 2024-03-19T01:20:21.335 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpi-mod.o 2024-03-19T01:20:21.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.400+0000 7f37d4c65740 1 Processor -- start 2024-03-19T01:20:21.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.400+0000 7f37d4c65740 1 -- start start 2024-03-19T01:20:21.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.400+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7db00 0x55b296b868c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.400+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7e420 0x55b296b92bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.400+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296bafda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.400+0000 7f37d4c65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1b53f0 con 0x55b296b7db00 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.400+0000 7f37d4c65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1b6f30 con 0x55b296b87000 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.400+0000 7f37d4c65740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1b5ac0 con 0x55b296b7e420 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.401+0000 7f37cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296bafda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.401+0000 7f37caa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7e420 0x55b296b92bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.401+0000 7f37caa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7e420 0x55b296b92bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38778/0 (socket says 172.21.15.69:38778) 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.401+0000 7f37caa00640 1 -- 172.21.15.69:0/2915495767 learned_addr learned my addr 172.21.15.69:0/2915495767 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.401+0000 7f37caa00640 1 -- 172.21.15.69:0/2915495767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 msgr2=0x55b296bafda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.401+0000 7f37caa00640 1 --2- 172.21.15.69:0/2915495767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296bafda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.401+0000 7f37caa00640 1 -- 172.21.15.69:0/2915495767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7db00 msgr2=0x55b296b868c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.401+0000 7f37cb400640 1 --2- 172.21.15.69:0/2915495767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7db00 0x55b296b868c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.401+0000 7f37caa00640 1 --2- 172.21.15.69:0/2915495767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7db00 0x55b296b868c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.401+0000 7f37caa00640 1 -- 172.21.15.69:0/2915495767 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2967b5170 con 0x55b296b7e420 2024-03-19T01:20:21.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.402+0000 7f37cb400640 1 --2- 172.21.15.69:0/2915495767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7db00 0x55b296b868c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:21.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.402+0000 7f37caa00640 1 --2- 172.21.15.69:0/2915495767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7e420 0x55b296b92bf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00fbb0 tx=0x7f37bc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bb9d68b9c820690 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:21.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.402+0000 7f37b7400640 1 -- 172.21.15.69:0/2915495767 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc02b070 con 0x55b296b7e420 2024-03-19T01:20:21.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.402+0000 7f37b7400640 1 -- 172.21.15.69:0/2915495767 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37bc026460 con 0x55b296b7e420 2024-03-19T01:20:21.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.403+0000 7f37b7400640 1 -- 172.21.15.69:0/2915495767 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00e5f0 con 0x55b296b7e420 2024-03-19T01:20:21.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.405+0000 7f37d4c65740 1 -- 172.21.15.69:0/2915495767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7e420 msgr2=0x55b296b92bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:21.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.405+0000 7f37d4c65740 1 --2- 172.21.15.69:0/2915495767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7e420 0x55b296b92bf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00fbb0 tx=0x7f37bc014000 comp rx=0 tx=0).stop 2024-03-19T01:20:21.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.407+0000 7f37d4c65740 1 -- 172.21.15.69:0/2915495767 shutdown_connections 2024-03-19T01:20:21.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.407+0000 7f37d4c65740 1 --2- 172.21.15.69:0/2915495767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296bafda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.407+0000 7f37d4c65740 1 --2- 172.21.15.69:0/2915495767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7e420 0x55b296b92bf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.407+0000 7f37d4c65740 1 --2- 172.21.15.69:0/2915495767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7db00 0x55b296b868c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.407+0000 7f37d4c65740 1 -- 172.21.15.69:0/2915495767 >> 172.21.15.69:0/2915495767 conn(0x55b296b78230 msgr2=0x55b296b772a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:21.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.407+0000 7f37d4c65740 1 -- 172.21.15.69:0/2915495767 shutdown_connections 2024-03-19T01:20:21.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.407+0000 7f37d4c65740 1 -- 172.21.15.69:0/2915495767 wait complete. 2024-03-19T01:20:21.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.407+0000 7f37d4c65740 1 -- start start 2024-03-19T01:20:21.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.408+0000 7f37d4c65740 1 Processor -- start 2024-03-19T01:20:21.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.410+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b80e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.410+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 0x55b296b813a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.410+0000 7f37d4c65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296b818e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.410+0000 7f37d4c65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1b6860 con 0x55b296b7e420 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.410+0000 7f37d4c65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1b6190 con 0x55b296b87000 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.410+0000 7f37d4c65740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1acb30 con 0x55b296b7db00 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.410+0000 7f37d4c65740 10 client.0 _ll_register_callbacks cb 0x7f37b8004c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.411+0000 7f37cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296b818e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.411+0000 7f37cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296b818e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60812/0 (socket says 172.21.15.69:60812) 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.411+0000 7f37cbe00640 1 -- 172.21.15.69:0/2251015329 learned_addr learned my addr 172.21.15.69:0/2251015329 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.411+0000 7f37cb400640 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b80e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.411+0000 7f37caa00640 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 0x55b296b813a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.411+0000 7f37cbe00640 1 -- 172.21.15.69:0/2251015329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 msgr2=0x55b296b80e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.411+0000 7f37cbe00640 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b80e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.411+0000 7f37cbe00640 1 -- 172.21.15.69:0/2251015329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 msgr2=0x55b296b813a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.411+0000 7f37cbe00640 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 0x55b296b813a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.411+0000 7f37cbe00640 1 -- 172.21.15.69:0/2251015329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b296c0d5d0 con 0x55b296b87000 2024-03-19T01:20:21.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.411+0000 7f37caa00640 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 0x55b296b813a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:21.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.412+0000 7f37cbe00640 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296b818e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f37b8000c00 tx=0x7f37b800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34b660108fe4452 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:21.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.414+0000 7f37b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:20:21.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.415+0000 7f37b4c00640 1 -- 172.21.15.69:0/2251015329 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b805a020 con 0x55b296b87000 2024-03-19T01:20:21.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.415+0000 7f37b4c00640 1 -- 172.21.15.69:0/2251015329 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f37b8018ad0 con 0x55b296b87000 2024-03-19T01:20:21.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.415+0000 7f37d4c65740 1 -- 172.21.15.69:0/2251015329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37bc026460 con 0x55b296b87000 2024-03-19T01:20:21.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.415+0000 7f37b4c00640 1 -- 172.21.15.69:0/2251015329 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f37b8065050 con 0x55b296b87000 2024-03-19T01:20:21.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.415+0000 7f37d4c65740 10 client.27674 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:21.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.415+0000 7f37d4c65740 1 -- 172.21.15.69:0/2251015329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55b296b93410 con 0x55b296b87000 2024-03-19T01:20:21.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.415+0000 7f37b4c00640 1 -- 172.21.15.69:0/2251015329 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b8027070 con 0x55b296b87000 2024-03-19T01:20:21.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.416+0000 7f37b4c00640 1 -- 172.21.15.69:0/2251015329 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f37b8005650 con 0x55b296b87000 2024-03-19T01:20:21.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.416+0000 7f37d4c65740 10 client.27674 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:21.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.416+0000 7f37d4c65740 10 client.27674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:20:21.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.416+0000 7f37d4c65740 20 client.27674 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:21.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.416+0000 7f37d4c65740 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b296b93be0 0x55b296b96000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:21.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.416+0000 7f37d4c65740 4 client.27674 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:20:21.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.416+0000 7f37d4c65740 1 -- 172.21.15.69:0/2251015329 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f37bc026460 con 0x55b296b93be0 2024-03-19T01:20:21.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.416+0000 7f37cb400640 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b296b93be0 0x55b296b96000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:21.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.418+0000 7f37cb400640 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b296b93be0 0x55b296b96000 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53744c4222a52d47 server_cookie=5789f692e70fdfe2 in_seq=0 out_seq=0 2024-03-19T01:20:21.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.418+0000 7f37b4c00640 10 client.27674 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:21.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.419+0000 7f37b4c00640 1 -- 172.21.15.69:0/2251015329 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f37bc026460 con 0x55b296b93be0 2024-03-19T01:20:21.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.419+0000 7f37b4c00640 10 client.27674 handle_command_reply: tid=0 2024-03-19T01:20:21.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.419+0000 7f37d4c65740 1 client.27674 shutdown 2024-03-19T01:20:21.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.421+0000 7f37d4c65740 1 -- 172.21.15.69:0/2251015329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 msgr2=0x55b296b818e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:21.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.421+0000 7f37d4c65740 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296b818e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f37b8000c00 tx=0x7f37b800ef70 comp rx=0 tx=0).stop 2024-03-19T01:20:21.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.421+0000 7f37d4c65740 1 -- 172.21.15.69:0/2251015329 shutdown_connections 2024-03-19T01:20:21.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.421+0000 7f37d4c65740 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55b296b93be0 0x55b296b96000 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.421+0000 7f37d4c65740 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b296b87000 0x55b296b818e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.421+0000 7f37d4c65740 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b296b7e420 0x55b296b813a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.421+0000 7f37b4c00640 0 client.27674 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:21.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.421+0000 7f37d4c65740 1 --2- 172.21.15.69:0/2251015329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b296b7db00 0x55b296b80e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.421+0000 7f37d4c65740 1 -- 172.21.15.69:0/2251015329 >> 172.21.15.69:0/2251015329 conn(0x55b296b78230 msgr2=0x55b296b76a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:21.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.421+0000 7f37d4c65740 1 -- 172.21.15.69:0/2251015329 shutdown_connections 2024-03-19T01:20:21.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.421+0000 7f37d4c65740 1 -- 172.21.15.69:0/2251015329 wait complete. 2024-03-19T01:20:21.426 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:20:21.426 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:20:21.426 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "b4529a67-55a4-4600-8621-8f60fd0b0d5a", 2024-03-19T01:20:21.426 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:20:21.426 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:20:21.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.425+0000 7f37d4c65740 20 client.27674 trim_cache size 0 max 16384 2024-03-19T01:20:21.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.427+0000 7f37d2000640 1 -- 172.21.15.69:0/2645110297 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37b00a2a70 msgr2=0x7f37b00a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:21.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.427+0000 7f37d2000640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37b00a2a70 0x7f37b00a4f30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f37cc10ba00 tx=0x7f37bc009260 comp rx=0 tx=0).stop 2024-03-19T01:20:21.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.427+0000 7f37d2000640 1 -- 172.21.15.69:0/2645110297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc108a00 msgr2=0x7f37cc10be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:21.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.427+0000 7f37d2000640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc108a00 0x7f37cc10be90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009b80 tx=0x7f37c4033910 comp rx=0 tx=0).stop 2024-03-19T01:20:21.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.429+0000 7f37d2000640 1 -- 172.21.15.69:0/2645110297 shutdown_connections 2024-03-19T01:20:21.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.429+0000 7f37d2000640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37cc114970 0x7f37cc10a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.429+0000 7f37d2000640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37cc109320 0x7f37cc10a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.429+0000 7f37d2000640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f37b00a2a70 0x7f37b00a4f30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.429+0000 7f37d2000640 1 --2- 172.21.15.69:0/2645110297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc108a00 0x7f37cc10be90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:21.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.429+0000 7f37d2000640 1 -- 172.21.15.69:0/2645110297 >> 172.21.15.69:0/2645110297 conn(0x7f37cc06c900 msgr2=0x7f37cc0704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:21.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.430+0000 7f37d2000640 1 -- 172.21.15.69:0/2645110297 shutdown_connections 2024-03-19T01:20:21.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:21.431+0000 7f37d2000640 1 -- 172.21.15.69:0/2645110297 wait complete. 2024-03-19T01:20:21.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:20:21.486 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:20:21.709 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:21 smithi069 ceph-mon[31727]: pgmap v301: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.0 MiB/s wr, 74 op/s 2024-03-19T01:20:21.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:21 smithi086 ceph-mon[66829]: pgmap v301: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.0 MiB/s wr, 74 op/s 2024-03-19T01:20:21.809 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/isofs/compress.o 2024-03-19T01:20:21.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:21 smithi163 ceph-mon[67428]: pgmap v301: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.0 MiB/s wr, 74 op/s 2024-03-19T01:20:21.958 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/char/ipmi/built-in.a 2024-03-19T01:20:22.102 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/client.o 2024-03-19T01:20:22.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.107+0000 7f95b5600640 1 -- 172.21.15.69:0/2938719913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95b01031f0 msgr2=0x7f95b010b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:22.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.107+0000 7f95b5600640 1 --2- 172.21.15.69:0/2938719913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95b01031f0 0x7f95b010b6d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009900 tx=0x7f95a402bcc0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.108+0000 7f95b5600640 1 -- 172.21.15.69:0/2938719913 shutdown_connections 2024-03-19T01:20:22.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.108+0000 7f95b5600640 1 --2- 172.21.15.69:0/2938719913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b010bca0 0x7f95b010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.108+0000 7f95b5600640 1 --2- 172.21.15.69:0/2938719913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95b01031f0 0x7f95b010b6d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.109+0000 7f95b5600640 1 --2- 172.21.15.69:0/2938719913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95b01028d0 0x7f95b0102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.109+0000 7f95b5600640 1 -- 172.21.15.69:0/2938719913 >> 172.21.15.69:0/2938719913 conn(0x7f95b00fc070 msgr2=0x7f95b00fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.113+0000 7f95b5600640 1 -- 172.21.15.69:0/2938719913 shutdown_connections 2024-03-19T01:20:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.114+0000 7f95b5600640 1 -- 172.21.15.69:0/2938719913 wait complete. 2024-03-19T01:20:22.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.120+0000 7f95b5600640 1 Processor -- start 2024-03-19T01:20:22.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.121+0000 7f95b5600640 1 -- start start 2024-03-19T01:20:22.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.121+0000 7f95b5600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95b01028d0 0x7f95b010b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.121+0000 7f95b5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01031f0 0x7f95b011b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.121+0000 7f95b5600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95b010bca0 0x7f95b0109350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.121+0000 7f95b5600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f95b010f220 con 0x7f95b01028d0 2024-03-19T01:20:22.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.121+0000 7f95b5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95b010f0a0 con 0x7f95b01031f0 2024-03-19T01:20:22.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.121+0000 7f95b5600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f95b010f3a0 con 0x7f95b010bca0 2024-03-19T01:20:22.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.122+0000 7f95af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01031f0 0x7f95b011b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.122+0000 7f95af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01031f0 0x7f95b011b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60830/0 (socket says 172.21.15.69:60830) 2024-03-19T01:20:22.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.122+0000 7f95af400640 1 -- 172.21.15.69:0/4291760880 learned_addr learned my addr 172.21.15.69:0/4291760880 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:22.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.123+0000 7f95af400640 1 -- 172.21.15.69:0/4291760880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95b010bca0 msgr2=0x7f95b0109350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:22.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.123+0000 7f95af400640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95b010bca0 0x7f95b0109350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.123+0000 7f95af400640 1 -- 172.21.15.69:0/4291760880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95b01028d0 msgr2=0x7f95b010b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:22.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.123+0000 7f95afe00640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95b01028d0 0x7f95b010b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.123+0000 7f95af400640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95b01028d0 0x7f95b010b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.123+0000 7f95af400640 1 -- 172.21.15.69:0/4291760880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b0109890 con 0x7f95b01031f0 2024-03-19T01:20:22.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.123+0000 7f95afe00640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95b01028d0 0x7f95b010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:22.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.126+0000 7f95af400640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01031f0 0x7f95b011b790 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f95a4000c00 tx=0x7f95a40043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:22.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.126+0000 7f95acc00640 1 -- 172.21.15.69:0/4291760880 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4042020 con 0x7f95b01031f0 2024-03-19T01:20:22.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.126+0000 7f95b5600640 1 -- 172.21.15.69:0/4291760880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b0109b20 con 0x7f95b01031f0 2024-03-19T01:20:22.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.127+0000 7f95b5600640 1 -- 172.21.15.69:0/4291760880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b010a0c0 con 0x7f95b01031f0 2024-03-19T01:20:22.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.127+0000 7f95acc00640 1 -- 172.21.15.69:0/4291760880 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a4005000 con 0x7f95b01031f0 2024-03-19T01:20:22.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.127+0000 7f95acc00640 1 -- 172.21.15.69:0/4291760880 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4003b20 con 0x7f95b01031f0 2024-03-19T01:20:22.139 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/mmap.o 2024-03-19T01:20:22.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.141+0000 7f95b997d740 1 Processor -- start 2024-03-19T01:20:22.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.141+0000 7f95acc00640 1 -- 172.21.15.69:0/4291760880 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f95a4045020 con 0x7f95b01031f0 2024-03-19T01:20:22.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.143+0000 7f95acc00640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f95980a2c20 0x7f95980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.143+0000 7f95acc00640 1 -- 172.21.15.69:0/4291760880 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f95a40038c0 con 0x7f95b01031f0 2024-03-19T01:20:22.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.146+0000 7f95b4c00640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f95980a2c20 0x7f95980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.147+0000 7f95b997d740 1 -- start start 2024-03-19T01:20:22.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.148+0000 7f95b4c00640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f95980a2c20 0x7f95980a50e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005fd0 tx=0x7f95a8009940 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:22.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.155+0000 7f95b997d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c94e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.155+0000 7f95b997d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 0x56026c956ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.155+0000 7f95b997d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c957430 0x56026c959820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.155+0000 7f95b997d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56026c8b22c0 con 0x56026c94e850 2024-03-19T01:20:22.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.155+0000 7f95b997d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56026c85ab60 con 0x56026c94df30 2024-03-19T01:20:22.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.155+0000 7f95b997d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56026c8ae360 con 0x56026c957430 2024-03-19T01:20:22.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.156+0000 7f95af400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 0x56026c956ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.156+0000 7f95af400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 0x56026c956ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44030/0 (socket says 172.21.15.69:44030) 2024-03-19T01:20:22.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.156+0000 7f95af400640 1 -- 172.21.15.69:0/4117434310 learned_addr learned my addr 172.21.15.69:0/4117434310 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:22.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.157+0000 7f95b4c00640 1 --2- 172.21.15.69:0/4117434310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c957430 0x56026c959820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.159+0000 7f95af400640 1 -- 172.21.15.69:0/4117434310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c957430 msgr2=0x56026c959820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:22.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.159+0000 7f95af400640 1 --2- 172.21.15.69:0/4117434310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c957430 0x56026c959820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.159+0000 7f95af400640 1 -- 172.21.15.69:0/4117434310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 msgr2=0x56026c94e310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:20:22.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.159+0000 7f95af400640 1 --2- 172.21.15.69:0/4117434310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c94e310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.159+0000 7f95af400640 1 -- 172.21.15.69:0/4117434310 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026c6907d0 con 0x56026c94e850 2024-03-19T01:20:22.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.160+0000 7f95af400640 1 --2- 172.21.15.69:0/4117434310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 0x56026c956ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f95a402c2d0 tx=0x7f95a40bd380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ee4858cfe836f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:22.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.160+0000 7f95a1200640 1 -- 172.21.15.69:0/4117434310 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a40d5020 con 0x56026c94e850 2024-03-19T01:20:22.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.160+0000 7f95a1200640 1 -- 172.21.15.69:0/4117434310 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a40c1b70 con 0x56026c94e850 2024-03-19T01:20:22.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.160+0000 7f95a1200640 1 -- 172.21.15.69:0/4117434310 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a40ba220 con 0x56026c94e850 2024-03-19T01:20:22.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.161+0000 7f95b997d740 1 -- 172.21.15.69:0/4117434310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 msgr2=0x56026c956ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:22.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.161+0000 7f95b997d740 1 --2- 172.21.15.69:0/4117434310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 0x56026c956ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f95a402c2d0 tx=0x7f95a40bd380 comp rx=0 tx=0).stop 2024-03-19T01:20:22.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.163+0000 7f95b997d740 1 -- 172.21.15.69:0/4117434310 shutdown_connections 2024-03-19T01:20:22.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.163+0000 7f95b997d740 1 --2- 172.21.15.69:0/4117434310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c957430 0x56026c959820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.163+0000 7f95b997d740 1 --2- 172.21.15.69:0/4117434310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 0x56026c956ef0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.163+0000 7f95b997d740 1 --2- 172.21.15.69:0/4117434310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c94e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.163+0000 7f95b997d740 1 -- 172.21.15.69:0/4117434310 >> 172.21.15.69:0/4117434310 conn(0x56026c948660 msgr2=0x56026c948a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:22.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.166+0000 7f95b997d740 1 -- 172.21.15.69:0/4117434310 shutdown_connections 2024-03-19T01:20:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.166+0000 7f95b997d740 1 -- 172.21.15.69:0/4117434310 wait complete. 2024-03-19T01:20:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.168+0000 7f95b997d740 1 -- start start 2024-03-19T01:20:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.169+0000 7f95b997d740 1 Processor -- start 2024-03-19T01:20:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.171+0000 7f95b997d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c9544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.172+0000 7f95b997d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 0x56026c954a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.172+0000 7f95afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c9544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.172+0000 7f95afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c9544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:60846/0 (socket says 172.21.15.69:60846) 2024-03-19T01:20:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.172+0000 7f95afe00640 1 -- 172.21.15.69:0/3276953897 learned_addr learned my addr 172.21.15.69:0/3276953897 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.172+0000 7f95af400640 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 0x56026c954a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.172+0000 7f95b997d740 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c957430 0x56026c954f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.173+0000 7f95b4c00640 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c957430 0x56026c954f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.173+0000 7f95b997d740 1 -- 172.21.15.69:0/3276953897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56026c959fe0 con 0x56026c94e850 2024-03-19T01:20:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.173+0000 7f95b997d740 1 -- 172.21.15.69:0/3276953897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56026c86dfc0 con 0x56026c94df30 2024-03-19T01:20:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.173+0000 7f95b997d740 1 -- 172.21.15.69:0/3276953897 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56026c95a160 con 0x56026c957430 2024-03-19T01:20:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.174+0000 7f95af400640 1 -- 172.21.15.69:0/3276953897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c957430 msgr2=0x56026c954f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.174+0000 7f95af400640 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c957430 0x56026c954f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.174+0000 7f95af400640 1 -- 172.21.15.69:0/3276953897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 msgr2=0x56026c9544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.174+0000 7f95af400640 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c9544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.174+0000 7f95af400640 1 -- 172.21.15.69:0/3276953897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026c5c63f0 con 0x56026c94e850 2024-03-19T01:20:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.176+0000 7f95b997d740 10 client.0 _ll_register_callbacks cb 0x56026c86d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.176+0000 7f95b997d740 1 -- 172.21.15.69:0/3276953897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a40c1b70 con 0x56026c94e850 2024-03-19T01:20:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.176+0000 7f95af400640 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 0x56026c954a30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x56026c94da00 tx=0x7f95a40eeb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688d6a7a165ee8c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.176+0000 7f9595600640 10 client.17358 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:20:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.181+0000 7f9595600640 1 -- 172.21.15.69:0/3276953897 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a40d5050 con 0x56026c94e850 2024-03-19T01:20:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.181+0000 7f9595600640 1 -- 172.21.15.69:0/3276953897 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f95a40d1e00 con 0x56026c94e850 2024-03-19T01:20:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.181+0000 7f9595600640 1 -- 172.21.15.69:0/3276953897 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f95a40d5050 con 0x56026c94e850 2024-03-19T01:20:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.182+0000 7f95b997d740 10 client.17358 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.182+0000 7f95b997d740 1 -- 172.21.15.69:0/3276953897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=59}) -- 0x56026c5860f0 con 0x56026c94e850 2024-03-19T01:20:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.183+0000 7f9595600640 1 -- 172.21.15.69:0/3276953897 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f95a402cb80 con 0x56026c94e850 2024-03-19T01:20:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.183+0000 7f95b997d740 10 client.17358 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.183+0000 7f95b997d740 10 client.17358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:20:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.183+0000 7f95b997d740 20 client.17358 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.184+0000 7f95b997d740 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56026c966e70 0x56026c969330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.184+0000 7f95b997d740 4 client.17358 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:20:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.184+0000 7f95b997d740 1 -- 172.21.15.69:0/3276953897 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56026c853610 con 0x56026c966e70 2024-03-19T01:20:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.184+0000 7f95afe00640 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56026c966e70 0x56026c969330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.189+0000 7f9595600640 10 client.17358 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:22.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.189+0000 7f95afe00640 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56026c966e70 0x56026c969330 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd10f01751c54303 server_cookie=cc4636aafa48b074 in_seq=0 out_seq=0 2024-03-19T01:20:22.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.190+0000 7f9595600640 1 -- 172.21.15.69:0/3276953897 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56026c853610 con 0x56026c966e70 2024-03-19T01:20:22.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.190+0000 7f9595600640 10 client.17358 handle_command_reply: tid=0 2024-03-19T01:20:22.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.191+0000 7f95b997d740 1 client.17358 shutdown 2024-03-19T01:20:22.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.195+0000 7f95b997d740 1 -- 172.21.15.69:0/3276953897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 msgr2=0x56026c954a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:22.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.195+0000 7f95b997d740 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 0x56026c954a30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x56026c94da00 tx=0x7f95a40eeb80 comp rx=0 tx=0).stop 2024-03-19T01:20:22.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.195+0000 7f95b997d740 1 -- 172.21.15.69:0/3276953897 shutdown_connections 2024-03-19T01:20:22.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.195+0000 7f95b997d740 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56026c966e70 0x56026c969330 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.195+0000 7f95b997d740 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c957430 0x56026c954f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.195+0000 7f95b997d740 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94e850 0x56026c954a30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.195+0000 7f95b997d740 1 --2- 172.21.15.69:0/3276953897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c9544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.195+0000 7f95b997d740 1 -- 172.21.15.69:0/3276953897 >> 172.21.15.69:0/3276953897 conn(0x56026c948660 msgr2=0x56026c94a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:22.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.195+0000 7f9595600640 0 client.17358 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:22.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.196+0000 7f95b997d740 1 -- 172.21.15.69:0/3276953897 shutdown_connections 2024-03-19T01:20:22.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.197+0000 7f95b997d740 1 -- 172.21.15.69:0/3276953897 wait complete. 2024-03-19T01:20:22.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.197+0000 7f95b997d740 20 client.17358 trim_cache size 0 max 16384 2024-03-19T01:20:22.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.286+0000 7f95b997d740 1 Processor -- start 2024-03-19T01:20:22.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.289+0000 7f95b997d740 1 -- start start 2024-03-19T01:20:22.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.289+0000 7f95b997d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c953430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.289+0000 7f95b997d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c957430 0x56026c962d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.290+0000 7f95b997d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c96ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.290+0000 7f95b997d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9590010080 con 0x56026c957430 2024-03-19T01:20:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.290+0000 7f95b997d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9590011bc0 con 0x56026c94df30 2024-03-19T01:20:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.290+0000 7f95b997d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9590010750 con 0x56026c954d90 2024-03-19T01:20:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.290+0000 7f95b4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c96ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.290+0000 7f95b4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c96ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38824/0 (socket says 172.21.15.69:38824) 2024-03-19T01:20:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.290+0000 7f95b4c00640 1 -- 172.21.15.69:0/1250866742 learned_addr learned my addr 172.21.15.69:0/1250866742 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.291+0000 7f95afe00640 1 --2- 172.21.15.69:0/1250866742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c953430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.290+0000 7f95af400640 1 --2- 172.21.15.69:0/1250866742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c957430 0x56026c962d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.291+0000 7f95b4c00640 1 -- 172.21.15.69:0/1250866742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 msgr2=0x56026c953430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.292+0000 7f95b4c00640 1 --2- 172.21.15.69:0/1250866742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c953430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.293+0000 7f95b4c00640 1 -- 172.21.15.69:0/1250866742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c957430 msgr2=0x56026c962d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:22.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.293+0000 7f95b4c00640 1 --2- 172.21.15.69:0/1250866742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c957430 0x56026c962d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.293+0000 7f95b4c00640 1 -- 172.21.15.69:0/1250866742 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026c94e680 con 0x56026c954d90 2024-03-19T01:20:22.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.294+0000 7f95af400640 1 --2- 172.21.15.69:0/1250866742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c957430 0x56026c962d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:20:22.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.295+0000 7f95b4c00640 1 --2- 172.21.15.69:0/1250866742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c96ec40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f95a801f580 tx=0x7f95a8011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e778889296be316a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:22.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.295+0000 7f9597e00640 1 -- 172.21.15.69:0/1250866742 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a802c070 con 0x56026c954d90 2024-03-19T01:20:22.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.295+0000 7f9597e00640 1 -- 172.21.15.69:0/1250866742 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a8010a90 con 0x56026c954d90 2024-03-19T01:20:22.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.295+0000 7f9597e00640 1 -- 172.21.15.69:0/1250866742 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8027c40 con 0x56026c954d90 2024-03-19T01:20:22.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.300+0000 7f95b997d740 1 -- 172.21.15.69:0/1250866742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 msgr2=0x56026c96ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:22.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.301+0000 7f95b997d740 1 --2- 172.21.15.69:0/1250866742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c96ec40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f95a801f580 tx=0x7f95a8011000 comp rx=0 tx=0).stop 2024-03-19T01:20:22.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.302+0000 7f95b997d740 1 -- 172.21.15.69:0/1250866742 shutdown_connections 2024-03-19T01:20:22.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.302+0000 7f95b997d740 1 --2- 172.21.15.69:0/1250866742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c96ec40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.302+0000 7f95b997d740 1 --2- 172.21.15.69:0/1250866742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c957430 0x56026c962d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.302+0000 7f95b997d740 1 --2- 172.21.15.69:0/1250866742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c94df30 0x56026c953430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.302+0000 7f95b997d740 1 -- 172.21.15.69:0/1250866742 >> 172.21.15.69:0/1250866742 conn(0x56026c948660 msgr2=0x56026c94a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:22.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.304+0000 7f95b997d740 1 -- 172.21.15.69:0/1250866742 shutdown_connections 2024-03-19T01:20:22.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.304+0000 7f95b997d740 1 -- 172.21.15.69:0/1250866742 wait complete. 2024-03-19T01:20:22.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.305+0000 7f95b997d740 1 -- start start 2024-03-19T01:20:22.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.305+0000 7f95b997d740 1 Processor -- start 2024-03-19T01:20:22.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.308+0000 7f95b997d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94df30 0x56026c95f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.308+0000 7f95b997d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c95c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.308+0000 7f95b997d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c957430 0x56026c95c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.308+0000 7f95b997d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f95900114f0 con 0x56026c94df30 2024-03-19T01:20:22.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.308+0000 7f95b997d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9590010e20 con 0x56026c957430 2024-03-19T01:20:22.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.308+0000 7f95b997d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f959000e060 con 0x56026c954d90 2024-03-19T01:20:22.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.308+0000 7f95b997d740 10 client.0 _ll_register_callbacks cb 0x7f95a40eeab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:22.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.309+0000 7f95af400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c95c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.309+0000 7f95af400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c95c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:38830/0 (socket says 172.21.15.69:38830) 2024-03-19T01:20:22.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.309+0000 7f95af400640 1 -- 172.21.15.69:0/4237701186 learned_addr learned my addr 172.21.15.69:0/4237701186 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:22.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.309+0000 7f95b4c00640 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c957430 0x56026c95c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.310+0000 7f95af400640 1 -- 172.21.15.69:0/4237701186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c957430 msgr2=0x56026c95c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:22.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.310+0000 7f95af400640 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c957430 0x56026c95c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.310+0000 7f95af400640 1 -- 172.21.15.69:0/4237701186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94df30 msgr2=0x56026c95f3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:22.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.310+0000 7f95af400640 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94df30 0x56026c95f3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.310+0000 7f95af400640 1 -- 172.21.15.69:0/4237701186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026c95fb60 con 0x56026c954d90 2024-03-19T01:20:22.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.310+0000 7f95b4c00640 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c957430 0x56026c95c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:22.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.313+0000 7f95af400640 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c95c380 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f95a402c2d0 tx=0x7f95a403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ef8bee01d962af2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:22.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.313+0000 7f9595600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:20:22.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.313+0000 7f9595600640 1 -- 172.21.15.69:0/4237701186 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a40a1060 con 0x56026c954d90 2024-03-19T01:20:22.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.314+0000 7f9595600640 1 -- 172.21.15.69:0/4237701186 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f95a40dc210 con 0x56026c954d90 2024-03-19T01:20:22.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.316+0000 7f95b997d740 1 -- 172.21.15.69:0/4237701186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a8010a90 con 0x56026c954d90 2024-03-19T01:20:22.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.316+0000 7f9595600640 1 -- 172.21.15.69:0/4237701186 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f95a40eee90 con 0x56026c954d90 2024-03-19T01:20:22.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.318+0000 7f95b997d740 10 client.27658 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:22.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.318+0000 7f95b997d740 1 -- 172.21.15.69:0/4237701186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x56026c95fb60 con 0x56026c954d90 2024-03-19T01:20:22.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.318+0000 7f9595600640 1 -- 172.21.15.69:0/4237701186 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a40c1b70 con 0x56026c954d90 2024-03-19T01:20:22.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.318+0000 7f9595600640 1 -- 172.21.15.69:0/4237701186 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f95a40a80b0 con 0x56026c954d90 2024-03-19T01:20:22.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.318+0000 7f95b997d740 10 client.27658 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:22.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.318+0000 7f95b997d740 10 client.27658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:20:22.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.318+0000 7f95b997d740 20 client.27658 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:22.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.318+0000 7f95b997d740 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56026c9ce120 0x56026c9d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:22.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.318+0000 7f95b997d740 4 client.27658 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:20:22.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.318+0000 7f95b997d740 1 -- 172.21.15.69:0/4237701186 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f95a8010a90 con 0x56026c9ce120 2024-03-19T01:20:22.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.321+0000 7f95afe00640 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56026c9ce120 0x56026c9d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:22.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.323+0000 7f95afe00640 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56026c9ce120 0x56026c9d0540 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e9ef491111b42c0 server_cookie=66c646fd2ba5ef8 in_seq=0 out_seq=0 2024-03-19T01:20:22.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.323+0000 7f9595600640 10 client.27658 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.327+0000 7f9595600640 1 -- 172.21.15.69:0/4237701186 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x7f95a8010a90 con 0x56026c9ce120 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.327+0000 7f9595600640 10 client.27658 handle_command_reply: tid=0 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.327+0000 7f95b997d740 1 client.27658 shutdown 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.328+0000 7f95b997d740 1 -- 172.21.15.69:0/4237701186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 msgr2=0x56026c95c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.328+0000 7f95b997d740 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c95c380 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f95a402c2d0 tx=0x7f95a403f000 comp rx=0 tx=0).stop 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.328+0000 7f95b997d740 1 -- 172.21.15.69:0/4237701186 shutdown_connections 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.328+0000 7f95b997d740 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x56026c9ce120 0x56026c9d0540 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.328+0000 7f95b997d740 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56026c957430 0x56026c95c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.328+0000 7f9595600640 0 client.27658 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.328+0000 7f95b997d740 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56026c954d90 0x56026c95c380 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.328+0000 7f95b997d740 1 --2- 172.21.15.69:0/4237701186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56026c94df30 0x56026c95f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.328+0000 7f95b997d740 1 -- 172.21.15.69:0/4237701186 >> 172.21.15.69:0/4237701186 conn(0x56026c948660 msgr2=0x56026c9499d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:22.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.331+0000 7f95b997d740 1 -- 172.21.15.69:0/4237701186 shutdown_connections 2024-03-19T01:20:22.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.331+0000 7f95b997d740 1 -- 172.21.15.69:0/4237701186 wait complete. 2024-03-19T01:20:22.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.331+0000 7f95b997d740 20 client.27658 trim_cache size 0 max 16384 2024-03-19T01:20:22.339 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:20:22.339 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (11314 inodes in the stack)", 2024-03-19T01:20:22.340 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:20:22.340 INFO:teuthology.orchestra.run.smithi069.stdout: "b4529a67-55a4-4600-8621-8f60fd0b0d5a": { 2024-03-19T01:20:22.340 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:20:22.340 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "b4529a67-55a4-4600-8621-8f60fd0b0d5a", 2024-03-19T01:20:22.340 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:20:22.340 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:20:22.340 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:20:22.340 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:20:22.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.344+0000 7f95b5600640 1 -- 172.21.15.69:0/4291760880 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f95980a2c20 msgr2=0x7f95980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:22.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.344+0000 7f95b5600640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f95980a2c20 0x7f95980a50e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005fd0 tx=0x7f95a8009940 comp rx=0 tx=0).stop 2024-03-19T01:20:22.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.344+0000 7f95b5600640 1 -- 172.21.15.69:0/4291760880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01031f0 msgr2=0x7f95b011b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:22.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.344+0000 7f95b5600640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01031f0 0x7f95b011b790 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f95a4000c00 tx=0x7f95a40043b0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.346+0000 7f95b5600640 1 -- 172.21.15.69:0/4291760880 shutdown_connections 2024-03-19T01:20:22.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.346+0000 7f95b5600640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95b010bca0 0x7f95b0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.346+0000 7f95b5600640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f95980a2c20 0x7f95980a50e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.346+0000 7f95b5600640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b01031f0 0x7f95b011b790 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.346+0000 7f95b5600640 1 --2- 172.21.15.69:0/4291760880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95b01028d0 0x7f95b010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:22.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.346+0000 7f95b5600640 1 -- 172.21.15.69:0/4291760880 >> 172.21.15.69:0/4291760880 conn(0x7f95b00fc070 msgr2=0x7f95b00fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:22.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.347+0000 7f95b5600640 1 -- 172.21.15.69:0/4291760880 shutdown_connections 2024-03-19T01:20:22.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:22.347+0000 7f95b5600640 1 -- 172.21.15.69:0/4291760880 wait complete. 2024-03-19T01:20:22.436 INFO:tasks.cephfs.filesystem:scrub status for tag:b4529a67-55a4-4600-8621-8f60fd0b0d5a - {'path': '/', 'tag': 'b4529a67-55a4-4600-8621-8f60fd0b0d5a', 'options': 'recursive,force'} 2024-03-19T01:20:22.622 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpi-mul.o 2024-03-19T01:20:22.673 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/events/callchain.o 2024-03-19T01:20:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:20:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:22 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:22.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:22 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:22.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:20:22.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:22.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:22 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:22.823 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/dir.o 2024-03-19T01:20:23.021 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_events.o 2024-03-19T01:20:23.400 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/setup-irq.o 2024-03-19T01:20:23.591 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/isofs/built-in.a 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: pgmap v302: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 631 KiB/s wr, 54 op/s 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.659 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.659 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.659 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:23.659 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:23 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.676 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/proc.o 2024-03-19T01:20:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: pgmap v302: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 631 KiB/s wr, 54 op/s 2024-03-19T01:20:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:23.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:23 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: pgmap v302: 129 pgs: 129 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 631 KiB/s wr, 54 op/s 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:23.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:23 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:23.887 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpih-cmp.o 2024-03-19T01:20:24.171 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/events/hw_breakpoint.o 2024-03-19T01:20:24.605 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/exportfs/expfs.o 2024-03-19T01:20:24.732 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/clntlock.o 2024-03-19T01:20:24.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:24.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:24 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:24.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:24 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:24.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:24.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:24 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.056 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/slot.o 2024-03-19T01:20:25.116 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpih-div.o 2024-03-19T01:20:25.598 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/sysfs.o 2024-03-19T01:20:25.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: pgmap v303: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 305 KiB/s wr, 42 op/s 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:25.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:25 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: pgmap v303: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 305 KiB/s wr, 42 op/s 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:25.793 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:25 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: pgmap v303: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 305 KiB/s wr, 42 op/s 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:25.816 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:25 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:25.846 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/exportfs/built-in.a 2024-03-19T01:20:25.929 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/pci-acpi.o 2024-03-19T01:20:26.038 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/events/uprobes.o 2024-03-19T01:20:26.163 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_export.o 2024-03-19T01:20:26.563 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpih-mul.o 2024-03-19T01:20:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:26.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:26 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.776 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/fork.o 2024-03-19T01:20:26.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:26.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:26 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:26.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:26.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:26 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.011 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/xattr.o 2024-03-19T01:20:27.119 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/file.o 2024-03-19T01:20:27.141 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/clntproc.o 2024-03-19T01:20:27.683 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:27.683 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: pgmap v304: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 303 KiB/s wr, 107 op/s 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:27.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:27 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.725 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_event_perf.o 2024-03-19T01:20:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: pgmap v304: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 303 KiB/s wr, 107 op/s 2024-03-19T01:20:27.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:27.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:27.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:27.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:27.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:27 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.792 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/quirks.o 2024-03-19T01:20:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:27.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: pgmap v304: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 303 KiB/s wr, 107 op/s 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:27.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:27 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:27.965 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpi-pow.o 2024-03-19T01:20:28.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:28.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:28 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:28.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:28 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:28.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:28 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:28.858 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/events/built-in.a 2024-03-19T01:20:29.009 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/xattr_hurd.o 2024-03-19T01:20:29.130 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_events_filter.o 2024-03-19T01:20:29.377 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/mpi/mpiutil.o 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: pgmap v305: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 538 KiB/s wr, 313 op/s 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:29.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:29 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: pgmap v305: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 538 KiB/s wr, 313 op/s 2024-03-19T01:20:29.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:29.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:29 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.798 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/clntxdr.o 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: pgmap v305: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 538 KiB/s wr, 313 op/s 2024-03-19T01:20:29.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:29.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:29.815 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:29 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.216 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/getroot.o 2024-03-19T01:20:30.453 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/xattr_trusted.o 2024-03-19T01:20:30.466 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nls/nls_base.o 2024-03-19T01:20:30.682 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/mpi/built-in.a 2024-03-19T01:20:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:30.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:30 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.771 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dsargs.o 2024-03-19T01:20:30.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:30.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:30 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:30.814 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:30 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:30.861 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/argv_split.o 2024-03-19T01:20:31.235 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_events_trigger.o 2024-03-19T01:20:31.247 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nls/nls_cp437.o 2024-03-19T01:20:31.318 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/ats.o 2024-03-19T01:20:31.670 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/bug.o 2024-03-19T01:20:31.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: pgmap v306: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 575 KiB/s wr, 360 op/s 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:31.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.772 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/xattr_user.o 2024-03-19T01:20:31.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: pgmap v306: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 575 KiB/s wr, 360 op/s 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:31.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: pgmap v306: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 575 KiB/s wr, 360 op/s 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:31.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:31.878 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dscontrol.o 2024-03-19T01:20:31.943 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/host.o 2024-03-19T01:20:32.125 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nls/nls_ascii.o 2024-03-19T01:20:32.182 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/inode.o 2024-03-19T01:20:32.614 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/pci-label.o 2024-03-19T01:20:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:32.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.758 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nls/nls_iso8859-1.o 2024-03-19T01:20:32.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:32.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:32.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:32.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:32.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:32.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:32.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:32.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:32.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:32.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:32.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:32.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:32.944 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dsdebug.o 2024-03-19T01:20:32.963 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/buildid.o 2024-03-19T01:20:33.210 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_eprobe.o 2024-03-19T01:20:33.298 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/fast_commit.o 2024-03-19T01:20:33.459 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nls/nls_utf8.o 2024-03-19T01:20:33.663 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dsfield.o 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: pgmap v307: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 868 KiB/s wr, 1.09k op/s 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:33.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:33.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:33.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:33.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: pgmap v307: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 868 KiB/s wr, 1.09k op/s 2024-03-19T01:20:33.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:33.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:33.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:33.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:33.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:33.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: pgmap v307: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 868 KiB/s wr, 1.09k op/s 2024-03-19T01:20:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:33.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:33.891 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pci/vgaarb.o 2024-03-19T01:20:33.967 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/svc.o 2024-03-19T01:20:34.086 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/nls/built-in.a 2024-03-19T01:20:34.120 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/clz_tab.o 2024-03-19T01:20:34.170 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/orphan.o 2024-03-19T01:20:34.278 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/cmdline.o 2024-03-19T01:20:34.507 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dsinit.o 2024-03-19T01:20:34.663 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/cpumask.o 2024-03-19T01:20:34.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:34.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:34.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:34.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:34.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:34.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:34.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:34.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:34.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:34.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:34.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:34.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:35.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:35.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.245 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dsmethod.o 2024-03-19T01:20:35.262 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_kprobe.o 2024-03-19T01:20:35.332 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/super.o 2024-03-19T01:20:35.517 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pci/built-in.a 2024-03-19T01:20:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:35.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: pgmap v308: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 874 KiB/s wr, 1.35k op/s 2024-03-19T01:20:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:35.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:35.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:35.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: pgmap v308: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 874 KiB/s wr, 1.35k op/s 2024-03-19T01:20:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:35.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:35.819 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/svclock.o 2024-03-19T01:20:35.835 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/ctype.o 2024-03-19T01:20:35.849 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pnp/pnpacpi/core.o 2024-03-19T01:20:35.983 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/exec_domain.o 2024-03-19T01:20:36.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: pgmap v308: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 874 KiB/s wr, 1.35k op/s 2024-03-19T01:20:36.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:36.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.010 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dsmthdat.o 2024-03-19T01:20:36.048 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/dec_and_lock.o 2024-03-19T01:20:36.154 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/acl.o 2024-03-19T01:20:36.661 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/decompress.o 2024-03-19T01:20:36.743 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dsobject.o 2024-03-19T01:20:36.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:36.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:36.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:36.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:36.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:36.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:36.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:36.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:36.906 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/decompress_bunzip2.o 2024-03-19T01:20:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:37.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:37.026 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pnp/pnpacpi/rsparser.o 2024-03-19T01:20:37.131 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ext4/xattr_security.o 2024-03-19T01:20:37.336 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/io.o 2024-03-19T01:20:37.683 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/svcshare.o 2024-03-19T01:20:37.713 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dsopcode.o 2024-03-19T01:20:37.729 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/direct.o 2024-03-19T01:20:37.804 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/error_report-traces.o 2024-03-19T01:20:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:37 smithi163 ceph-mon[67428]: pgmap v309: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 791 KiB/s wr, 1.58k op/s 2024-03-19T01:20:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:37.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:38.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:37 smithi086 ceph-mon[66829]: pgmap v309: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 791 KiB/s wr, 1.58k op/s 2024-03-19T01:20:38.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:38.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:38.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:38.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:37 smithi069 ceph-mon[31727]: pgmap v309: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 791 KiB/s wr, 1.58k op/s 2024-03-19T01:20:38.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:38.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:38.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.150 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/decompress_inflate.o 2024-03-19T01:20:38.506 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dspkginit.o 2024-03-19T01:20:38.582 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/ext4/built-in.a 2024-03-19T01:20:38.705 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pnp/core.o 2024-03-19T01:20:38.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:38.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:38.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:38.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:38 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:39.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:38 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:39.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:38 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.613 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/power-traces.o 2024-03-19T01:20:39.616 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/svcproc.o 2024-03-19T01:20:39.617 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pnp/pnpacpi/built-in.a 2024-03-19T01:20:39.634 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dsutils.o 2024-03-19T01:20:39.650 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/decompress_unlz4.o 2024-03-19T01:20:39.657 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/numa/srat.o 2024-03-19T01:20:39.663 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/panic.o 2024-03-19T01:20:39.730 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pnp/card.o 2024-03-19T01:20:39.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: pgmap v310: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 823 KiB/s wr, 1.66k op/s 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:39.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:39.874 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/pagelist.o 2024-03-19T01:20:40.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:40.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:40.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:40.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:40.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:40.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: pgmap v310: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 823 KiB/s wr, 1.66k op/s 2024-03-19T01:20:40.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:40.007 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:40.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:40.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:40.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:40.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:40.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: pgmap v310: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 823 KiB/s wr, 1.66k op/s 2024-03-19T01:20:40.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:40.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:40.609 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/decompress_unlzma.o 2024-03-19T01:20:40.746 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dswexec.o 2024-03-19T01:20:40.831 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/acpi/numa/built-in.a 2024-03-19T01:20:40.937 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pnp/driver.o 2024-03-19T01:20:40.970 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/read.o 2024-03-19T01:20:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:41.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:41.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:41.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:41.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:41.109 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/svcsubs.o 2024-03-19T01:20:41.438 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/decompress_unlzo.o 2024-03-19T01:20:41.590 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dswload.o 2024-03-19T01:20:41.762 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/unicode/built-in.a 2024-03-19T01:20:41.815 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pnp/resource.o 2024-03-19T01:20:41.847 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/rpm-traces.o 2024-03-19T01:20:41.883 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/amba/built-in.a 2024-03-19T01:20:42.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:42.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:42.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:20:42.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:42.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:41 smithi086 ceph-mon[66829]: pgmap v311: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 471 KiB/s wr, 1.45k op/s 2024-03-19T01:20:42.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:42.019 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dswload2.o 2024-03-19T01:20:42.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:42.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:42.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:20:42.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:42.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:41 smithi069 ceph-mon[31727]: pgmap v311: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 471 KiB/s wr, 1.45k op/s 2024-03-19T01:20:42.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:42.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:42.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:42.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:20:42.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:42.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:41 smithi163 ceph-mon[67428]: pgmap v311: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 471 KiB/s wr, 1.45k op/s 2024-03-19T01:20:42.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:20:42.341 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/decompress_unxz.o 2024-03-19T01:20:42.591 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cpu.o 2024-03-19T01:20:42.696 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/mon.o 2024-03-19T01:20:42.960 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/trace.o 2024-03-19T01:20:42.973 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dswscope.o 2024-03-19T01:20:43.156 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pnp/manager.o 2024-03-19T01:20:43.207 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_dynevent.o 2024-03-19T01:20:43.246 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/symlink.o 2024-03-19T01:20:43.472 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/decompress_unzstd.o 2024-03-19T01:20:43.844 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/dswstate.o 2024-03-19T01:20:43.942 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pnp/support.o 2024-03-19T01:20:44.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:43 smithi086 ceph-mon[66829]: pgmap v312: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 496 KiB/s wr, 1.41k op/s 2024-03-19T01:20:44.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:43 smithi069 ceph-mon[31727]: pgmap v312: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 496 KiB/s wr, 1.41k op/s 2024-03-19T01:20:44.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:43 smithi163 ceph-mon[67428]: pgmap v312: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 496 KiB/s wr, 1.41k op/s 2024-03-19T01:20:44.128 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pnp/interface.o 2024-03-19T01:20:44.308 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/dump_stack.o 2024-03-19T01:20:44.529 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/xdr.o 2024-03-19T01:20:44.553 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_probe.o 2024-03-19T01:20:44.662 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evevent.o 2024-03-19T01:20:44.708 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/unlink.o 2024-03-19T01:20:44.776 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evgpe.o 2024-03-19T01:20:44.887 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/clnt4xdr.o 2024-03-19T01:20:45.220 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pnp/quirks.o 2024-03-19T01:20:45.481 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/actions/built-in.a 2024-03-19T01:20:45.505 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/earlycpio.o 2024-03-19T01:20:45.564 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evgpeblk.o 2024-03-19T01:20:45.570 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/analogbits/built-in.a 2024-03-19T01:20:45.699 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/bcm/built-in.a 2024-03-19T01:20:45.754 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/imgtec/built-in.a 2024-03-19T01:20:45.812 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/imx/built-in.a 2024-03-19T01:20:45.870 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/ingenic/built-in.a 2024-03-19T01:20:45.927 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/extable.o 2024-03-19T01:20:45.932 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/mediatek/built-in.a 2024-03-19T01:20:45.951 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma/dw/core.o 2024-03-19T01:20:45.980 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/microchip/built-in.a 2024-03-19T01:20:46.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:45 smithi086 ceph-mon[66829]: pgmap v313: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 306 KiB/s wr, 679 op/s 2024-03-19T01:20:46.024 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/mstar/built-in.a 2024-03-19T01:20:46.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:45 smithi069 ceph-mon[31727]: pgmap v313: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 306 KiB/s wr, 679 op/s 2024-03-19T01:20:46.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:45 smithi163 ceph-mon[67428]: pgmap v313: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 306 KiB/s wr, 679 op/s 2024-03-19T01:20:46.098 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/mvebu/built-in.a 2024-03-19T01:20:46.130 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/xdr4.o 2024-03-19T01:20:46.165 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/trace_uprobe.o 2024-03-19T01:20:46.174 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pnp/system.o 2024-03-19T01:20:46.184 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/ralink/built-in.a 2024-03-19T01:20:46.243 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evgpeinit.o 2024-03-19T01:20:46.319 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/renesas/built-in.a 2024-03-19T01:20:46.458 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/socfpga/built-in.a 2024-03-19T01:20:46.561 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/sprd/built-in.a 2024-03-19T01:20:46.617 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/starfive/built-in.a 2024-03-19T01:20:46.663 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/flex_proportions.o 2024-03-19T01:20:46.685 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/sunxi-ng/built-in.a 2024-03-19T01:20:46.726 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/write.o 2024-03-19T01:20:46.745 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/ti/built-in.a 2024-03-19T01:20:46.844 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/versatile/built-in.a 2024-03-19T01:20:46.910 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evgpeutil.o 2024-03-19T01:20:46.911 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/xilinx/built-in.a 2024-03-19T01:20:46.930 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pnp/built-in.a 2024-03-19T01:20:46.989 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clk/built-in.a 2024-03-19T01:20:47.196 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/idr.o 2024-03-19T01:20:47.211 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/irq_regs.o 2024-03-19T01:20:47.360 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/acpi/pmic/built-in.a 2024-03-19T01:20:47.388 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma/dw/dw.o 2024-03-19T01:20:47.420 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma/dw/idma32.o 2024-03-19T01:20:47.642 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/svc4proc.o 2024-03-19T01:20:47.645 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/autofs/init.o 2024-03-19T01:20:47.662 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evglock.o 2024-03-19T01:20:47.863 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:47 smithi163 ceph-mon[67428]: pgmap v314: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 561 KiB/s wr, 431 op/s 2024-03-19T01:20:48.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:47 smithi086 ceph-mon[66829]: pgmap v314: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 561 KiB/s wr, 431 op/s 2024-03-19T01:20:48.023 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/is_single_threaded.o 2024-03-19T01:20:48.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:47 smithi069 ceph-mon[31727]: pgmap v314: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 561 KiB/s wr, 431 op/s 2024-03-19T01:20:48.300 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/trace/rethook.o 2024-03-19T01:20:48.335 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evhandler.o 2024-03-19T01:20:48.411 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evmisc.o 2024-03-19T01:20:48.422 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/autofs/inode.o 2024-03-19T01:20:48.438 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma/dw/acpi.o 2024-03-19T01:20:48.993 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/lockd/procfs.o 2024-03-19T01:20:49.038 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/klist.o 2024-03-19T01:20:49.109 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evregion.o 2024-03-19T01:20:49.110 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma/hsu/hsu.o 2024-03-19T01:20:49.313 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/trace/built-in.a 2024-03-19T01:20:49.349 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/autofs/root.o 2024-03-19T01:20:49.437 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/namespace.o 2024-03-19T01:20:49.465 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/dma/dw/built-in.a 2024-03-19T01:20:49.469 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/exit.o 2024-03-19T01:20:49.593 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/softirq.o 2024-03-19T01:20:49.593 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/kobject.o 2024-03-19T01:20:49.989 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evrgnini.o 2024-03-19T01:20:50.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:49 smithi086 ceph-mon[66829]: pgmap v315: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 871 KiB/s wr, 212 op/s 2024-03-19T01:20:50.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:49 smithi069 ceph-mon[31727]: pgmap v315: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 871 KiB/s wr, 212 op/s 2024-03-19T01:20:50.053 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/lockd/built-in.a 2024-03-19T01:20:50.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:49 smithi163 ceph-mon[67428]: pgmap v315: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 871 KiB/s wr, 212 op/s 2024-03-19T01:20:50.193 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evsci.o 2024-03-19T01:20:50.218 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/autofs/symlink.o 2024-03-19T01:20:50.359 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/dma/hsu/built-in.a 2024-03-19T01:20:50.522 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/dma/idxd/built-in.a 2024-03-19T01:20:50.545 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/kobject_uevent.o 2024-03-19T01:20:50.589 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/dma/mediatek/built-in.a 2024-03-19T01:20:50.644 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/dma/qcom/built-in.a 2024-03-19T01:20:50.659 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/resource.o 2024-03-19T01:20:50.705 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/dma/ti/built-in.a 2024-03-19T01:20:50.760 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evxface.o 2024-03-19T01:20:50.769 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/mount_clnt.o 2024-03-19T01:20:50.793 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/dma/xilinx/built-in.a 2024-03-19T01:20:50.834 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma/dmaengine.o 2024-03-19T01:20:51.049 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/autofs/waitq.o 2024-03-19T01:20:51.079 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/autofs/expire.o 2024-03-19T01:20:51.561 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evxfevnt.o 2024-03-19T01:20:51.691 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evxfgpe.o 2024-03-19T01:20:51.874 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/sysctl.o 2024-03-19T01:20:51.889 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma/virt-dma.o 2024-03-19T01:20:51.958 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/logic_pio.o 2024-03-19T01:20:52.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:51 smithi086 ceph-mon[66829]: pgmap v316: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 799 KiB/s wr, 57 op/s 2024-03-19T01:20:52.011 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfstrace.o 2024-03-19T01:20:52.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:51 smithi069 ceph-mon[31727]: pgmap v316: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 799 KiB/s wr, 57 op/s 2024-03-19T01:20:52.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:51 smithi163 ceph-mon[67428]: pgmap v316: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 799 KiB/s wr, 57 op/s 2024-03-19T01:20:52.202 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/autofs/dev-ioctl.o 2024-03-19T01:20:52.234 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/dptf/int340x_thermal.o 2024-03-19T01:20:52.242 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/apple/built-in.a 2024-03-19T01:20:52.326 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/aspeed/built-in.a 2024-03-19T01:20:52.414 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/bcm/bcm63xx/built-in.a 2024-03-19T01:20:52.434 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/evxfregn.o 2024-03-19T01:20:52.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:20:52.438 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:20:52.450 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/bcm/built-in.a 2024-03-19T01:20:52.568 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/fsl/built-in.a 2024-03-19T01:20:52.749 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/fujitsu/built-in.a 2024-03-19T01:20:52.798 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/imx/built-in.a 2024-03-19T01:20:52.845 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma/acpi-dma.o 2024-03-19T01:20:52.847 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/ixp4xx/built-in.a 2024-03-19T01:20:52.899 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/loongson/built-in.a 2024-03-19T01:20:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.956+0000 7f5e08e00640 1 -- 172.21.15.69:0/285475633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e04105a40 msgr2=0x7f5e04105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.956+0000 7f5e08e00640 1 --2- 172.21.15.69:0/285475633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e04105a40 0x7f5e04105ed0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc01acb0 tx=0x7f5dfc03d1c0 comp rx=0 tx=0).stop 2024-03-19T01:20:52.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.960+0000 7f5e08e00640 1 -- 172.21.15.69:0/285475633 shutdown_connections 2024-03-19T01:20:52.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.960+0000 7f5e08e00640 1 --2- 172.21.15.69:0/285475633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e04105a40 0x7f5e04105ed0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:52.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.960+0000 7f5e08e00640 1 --2- 172.21.15.69:0/285475633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04100f60 0x7f5e04105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:52.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.960+0000 7f5e08e00640 1 --2- 172.21.15.69:0/285475633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e04100640 0x7f5e04100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:52.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.960+0000 7f5e08e00640 1 -- 172.21.15.69:0/285475633 >> 172.21.15.69:0/285475633 conn(0x7f5e040fc070 msgr2=0x7f5e040fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:52.961 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/mediatek/built-in.a 2024-03-19T01:20:52.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.969+0000 7f5e08e00640 1 -- 172.21.15.69:0/285475633 shutdown_connections 2024-03-19T01:20:52.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.969+0000 7f5e08e00640 1 -- 172.21.15.69:0/285475633 wait complete. 2024-03-19T01:20:52.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.970+0000 7f5e08e00640 1 Processor -- start 2024-03-19T01:20:52.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.971+0000 7f5e08e00640 1 -- start start 2024-03-19T01:20:52.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.971+0000 7f5e08e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e04100640 0x7f5e04072300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:52.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.971+0000 7f5e08e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04100f60 0x7f5e04072840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:52.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.971+0000 7f5e08e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e04105a40 0x7f5e0410cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:52.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.971+0000 7f5e08e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e04109370 con 0x7f5e04100f60 2024-03-19T01:20:52.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.971+0000 7f5e08e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e041091f0 con 0x7f5e04100640 2024-03-19T01:20:52.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.971+0000 7f5e08e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5e041094f0 con 0x7f5e04105a40 2024-03-19T01:20:52.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.972+0000 7f5e03400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e04100640 0x7f5e04072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:52.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.972+0000 7f5e03400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e04100640 0x7f5e04072300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53408/0 (socket says 172.21.15.69:53408) 2024-03-19T01:20:52.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.972+0000 7f5e03400640 1 -- 172.21.15.69:0/2304940490 learned_addr learned my addr 172.21.15.69:0/2304940490 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:52.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.972+0000 7f5e03e00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e04105a40 0x7f5e0410cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:52.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.972+0000 7f5e03400640 1 -- 172.21.15.69:0/2304940490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e04105a40 msgr2=0x7f5e0410cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:52.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.972+0000 7f5e03400640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e04105a40 0x7f5e0410cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:52.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.972+0000 7f5e03400640 1 -- 172.21.15.69:0/2304940490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04100f60 msgr2=0x7f5e04072840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:20:52.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.972+0000 7f5e02a00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04100f60 0x7f5e04072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:52.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.972+0000 7f5e03400640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04100f60 0x7f5e04072840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:52.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.973+0000 7f5e03400640 1 -- 172.21.15.69:0/2304940490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0410d1e0 con 0x7f5e04100640 2024-03-19T01:20:52.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.973+0000 7f5e03e00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e04105a40 0x7f5e0410cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:52.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.973+0000 7f5e02a00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04100f60 0x7f5e04072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:20:52.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.973+0000 7f5e03400640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e04100640 0x7f5e04072300 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5df400e9f0 tx=0x7f5df400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:52.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.973+0000 7f5dfbe00640 1 -- 172.21.15.69:0/2304940490 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df400cd80 con 0x7f5e04100640 2024-03-19T01:20:52.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.973+0000 7f5dfbe00640 1 -- 172.21.15.69:0/2304940490 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df4010910 con 0x7f5e04100640 2024-03-19T01:20:52.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.973+0000 7f5dfbe00640 1 -- 172.21.15.69:0/2304940490 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40188f0 con 0x7f5e04100640 2024-03-19T01:20:52.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.973+0000 7f5e08e00640 1 -- 172.21.15.69:0/2304940490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0410d4d0 con 0x7f5e04100640 2024-03-19T01:20:52.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.973+0000 7f5e08e00640 1 -- 172.21.15.69:0/2304940490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0410d9d0 con 0x7f5e04100640 2024-03-19T01:20:52.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.985+0000 7f5dfbe00640 1 -- 172.21.15.69:0/2304940490 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f5df4010ac0 con 0x7f5e04100640 2024-03-19T01:20:52.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.988+0000 7f5dfbe00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5de80a2a70 0x7f5de80a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:52.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.989+0000 7f5dfbe00640 1 -- 172.21.15.69:0/2304940490 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f5df40cf870 con 0x7f5e04100640 2024-03-19T01:20:52.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.989+0000 7f5e0d174740 1 Processor -- start 2024-03-19T01:20:52.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.990+0000 7f5e0d174740 1 -- start start 2024-03-19T01:20:52.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.991+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6df30 0x5576bef6e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:52.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.991+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6e850 0x5576bef76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:52.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.991+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef79820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:52.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.991+0000 7f5e0d174740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5576beed22c0 con 0x5576bef6e850 2024-03-19T01:20:52.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.991+0000 7f5e0d174740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576bee7ab60 con 0x5576bef77430 2024-03-19T01:20:52.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.991+0000 7f5e0d174740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5576beece360 con 0x5576bef6df30 2024-03-19T01:20:52.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.991+0000 7f5e03e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef79820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:52.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.992+0000 7f5e03e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef79820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53416/0 (socket says 172.21.15.69:53416) 2024-03-19T01:20:52.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.992+0000 7f5e03e00640 1 -- 172.21.15.69:0/395749607 learned_addr learned my addr 172.21.15.69:0/395749607 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:52.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.993+0000 7f5e02a00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5de80a2a70 0x7f5de80a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:52.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.993+0000 7f5e03e00640 1 -- 172.21.15.69:0/395749607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6df30 msgr2=0x5576bef6e310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:20:52.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.993+0000 7f5e03e00640 1 --2- 172.21.15.69:0/395749607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6df30 0x5576bef6e310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:52.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.993+0000 7f5e03e00640 1 -- 172.21.15.69:0/395749607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6e850 msgr2=0x5576bef76ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:20:52.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.993+0000 7f5e03e00640 1 --2- 172.21.15.69:0/395749607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6e850 0x5576bef76ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:52.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.993+0000 7f5e03e00640 1 -- 172.21.15.69:0/395749607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576becb07d0 con 0x5576bef77430 2024-03-19T01:20:52.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.993+0000 7f5e03e00640 1 --2- 172.21.15.69:0/395749607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef79820 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc048040 tx=0x7f5dfc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10fa592883fdd246 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:52.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.994+0000 7f5e02a00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5de80a2a70 0x7f5de80a4f30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5df0005fd0 tx=0x7f5df0005dc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:52.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.995+0000 7f5df8c00640 1 -- 172.21.15.69:0/395749607 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc052020 con 0x5576bef77430 2024-03-19T01:20:52.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.996+0000 7f5df8c00640 1 -- 172.21.15.69:0/395749607 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dfc003e50 con 0x5576bef77430 2024-03-19T01:20:52.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.996+0000 7f5df8c00640 1 -- 172.21.15.69:0/395749607 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc052020 con 0x5576bef77430 2024-03-19T01:20:52.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.997+0000 7f5e0d174740 1 -- 172.21.15.69:0/395749607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 msgr2=0x5576bef79820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:52.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.997+0000 7f5e0d174740 1 --2- 172.21.15.69:0/395749607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef79820 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc048040 tx=0x7f5dfc004a40 comp rx=0 tx=0).stop 2024-03-19T01:20:52.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.998+0000 7f5e0d174740 1 -- 172.21.15.69:0/395749607 shutdown_connections 2024-03-19T01:20:52.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.998+0000 7f5e0d174740 1 --2- 172.21.15.69:0/395749607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef79820 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:52.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.998+0000 7f5e0d174740 1 --2- 172.21.15.69:0/395749607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6e850 0x5576bef76ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:52.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.998+0000 7f5e0d174740 1 --2- 172.21.15.69:0/395749607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6df30 0x5576bef6e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:52.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.998+0000 7f5e0d174740 1 -- 172.21.15.69:0/395749607 >> 172.21.15.69:0/395749607 conn(0x5576bef68660 msgr2=0x5576bef68a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:53.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.999+0000 7f5e0d174740 1 -- 172.21.15.69:0/395749607 shutdown_connections 2024-03-19T01:20:53.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:52.999+0000 7f5e0d174740 1 -- 172.21.15.69:0/395749607 wait complete. 2024-03-19T01:20:53.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.000+0000 7f5e0d174740 1 -- start start 2024-03-19T01:20:53.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.002+0000 7f5e0d174740 1 Processor -- start 2024-03-19T01:20:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.002+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef74570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.002+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6e850 0x5576bef74ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.002+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef74ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.002+0000 7f5e0d174740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5576bef7a1c0 con 0x5576bef6df30 2024-03-19T01:20:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.002+0000 7f5e0d174740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576bee8dfc0 con 0x5576bef77430 2024-03-19T01:20:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.002+0000 7f5e0d174740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5576bef7a340 con 0x5576bef6e850 2024-03-19T01:20:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.003+0000 7f5e0d174740 10 client.0 _ll_register_callbacks cb 0x5576bee8d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.003+0000 7f5e03400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef74570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.003+0000 7f5e03e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef74ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:53.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.003+0000 7f5e03400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef74570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34218/0 (socket says 172.21.15.69:34218) 2024-03-19T01:20:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.003+0000 7f5e03400640 1 -- 172.21.15.69:0/3757537093 learned_addr learned my addr 172.21.15.69:0/3757537093 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.004+0000 7f5e03400640 1 -- 172.21.15.69:0/3757537093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6e850 msgr2=0x5576bef74ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:20:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.004+0000 7f5e03400640 1 --2- 172.21.15.69:0/3757537093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6e850 0x5576bef74ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.004+0000 7f5e03400640 1 -- 172.21.15.69:0/3757537093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 msgr2=0x5576bef74ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.004+0000 7f5e03400640 1 --2- 172.21.15.69:0/3757537093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef74ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.004+0000 7f5e03400640 1 -- 172.21.15.69:0/3757537093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576bebe63f0 con 0x5576bef6df30 2024-03-19T01:20:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.004+0000 7f5e03400640 1 --2- 172.21.15.69:0/3757537093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef74570 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5df4097ee0 tx=0x7f5df409c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c1ec7b1413ac95 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.005+0000 7f5ded600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:20:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.005+0000 7f5ded600640 1 -- 172.21.15.69:0/3757537093 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40afe60 con 0x5576bef6df30 2024-03-19T01:20:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.005+0000 7f5ded600640 1 -- 172.21.15.69:0/3757537093 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f5df40cf3c0 con 0x5576bef6df30 2024-03-19T01:20:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.005+0000 7f5e0d174740 1 -- 172.21.15.69:0/3757537093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dfc003e50 con 0x5576bef6df30 2024-03-19T01:20:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.006+0000 7f5ded600640 1 -- 172.21.15.69:0/3757537093 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f5df40afe60 con 0x5576bef6df30 2024-03-19T01:20:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.006+0000 7f5e0d174740 10 client.17367 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.006+0000 7f5e0d174740 1 -- 172.21.15.69:0/3757537093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5576beba60f0 con 0x5576bef6df30 2024-03-19T01:20:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.006+0000 7f5ded600640 1 -- 172.21.15.69:0/3757537093 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dfc003e50 con 0x5576bef6df30 2024-03-19T01:20:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.006+0000 7f5ded600640 1 -- 172.21.15.69:0/3757537093 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f5df40b4b40 con 0x5576bef6df30 2024-03-19T01:20:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.007+0000 7f5e0d174740 10 client.17367 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.007+0000 7f5e0d174740 10 client.17367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:20:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.007+0000 7f5e0d174740 20 client.17367 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.007+0000 7f5e0d174740 1 --2- 172.21.15.69:0/3757537093 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5576bef87dd0 0x5576bef8a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.007+0000 7f5e0d174740 4 client.17367 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:20:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.007+0000 7f5e0d174740 1 -- 172.21.15.69:0/3757537093 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576bee73610 con 0x5576bef87dd0 2024-03-19T01:20:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.008+0000 7f5e02a00640 1 --2- 172.21.15.69:0/3757537093 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5576bef87dd0 0x5576bef8a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.009+0000 7f5e02a00640 1 --2- 172.21.15.69:0/3757537093 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5576bef87dd0 0x5576bef8a290 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75cdaa5c4ba25552 server_cookie=5646e5e5382443f0 in_seq=0 out_seq=0 2024-03-19T01:20:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.009+0000 7f5ded600640 10 client.17367 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:53.010 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/maple_tree.o 2024-03-19T01:20:53.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.009+0000 7f5ded600640 1 -- 172.21.15.69:0/3757537093 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5576bee73610 con 0x5576bef87dd0 2024-03-19T01:20:53.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.010+0000 7f5ded600640 10 client.17367 handle_command_reply: tid=0 2024-03-19T01:20:53.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.010+0000 7f5e0d174740 1 client.17367 shutdown 2024-03-19T01:20:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.010+0000 7f5e0d174740 1 -- 172.21.15.69:0/3757537093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 msgr2=0x5576bef74570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.010+0000 7f5e0d174740 1 --2- 172.21.15.69:0/3757537093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef74570 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5df4097ee0 tx=0x7f5df409c240 comp rx=0 tx=0).stop 2024-03-19T01:20:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.011+0000 7f5e0d174740 1 -- 172.21.15.69:0/3757537093 shutdown_connections 2024-03-19T01:20:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.011+0000 7f5e0d174740 1 --2- 172.21.15.69:0/3757537093 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5576bef87dd0 0x5576bef8a290 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.011+0000 7f5e0d174740 1 --2- 172.21.15.69:0/3757537093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef74ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.011+0000 7f5e0d174740 1 --2- 172.21.15.69:0/3757537093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6e850 0x5576bef74ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.011+0000 7f5ded600640 0 client.17367 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.011+0000 7f5e0d174740 1 --2- 172.21.15.69:0/3757537093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef74570 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.011+0000 7f5e0d174740 1 -- 172.21.15.69:0/3757537093 >> 172.21.15.69:0/3757537093 conn(0x5576bef68660 msgr2=0x5576bef6a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.013+0000 7f5e0d174740 1 -- 172.21.15.69:0/3757537093 shutdown_connections 2024-03-19T01:20:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.013+0000 7f5e0d174740 1 -- 172.21.15.69:0/3757537093 wait complete. 2024-03-19T01:20:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.013+0000 7f5e0d174740 20 client.17367 trim_cache size 0 max 16384 2024-03-19T01:20:53.025 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/acpi/dptf/built-in.a 2024-03-19T01:20:53.036 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/microchip/built-in.a 2024-03-19T01:20:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.073+0000 7f5e0d174740 1 Processor -- start 2024-03-19T01:20:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.073+0000 7f5e0d174740 1 -- start start 2024-03-19T01:20:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.073+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6df30 0x5576bef76ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.074+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6e850 0x5576bef734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.074+0000 7f5e03400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6df30 0x5576bef76ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:53.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.074+0000 7f5e03400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6df30 0x5576bef76ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:47368/0 (socket says 172.21.15.69:47368) 2024-03-19T01:20:53.081 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exconcat.o 2024-03-19T01:20:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.074+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef82f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.074+0000 7f5e0d174740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e041b54c0 con 0x5576bef6e850 2024-03-19T01:20:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.074+0000 7f5e0d174740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e041b7000 con 0x5576bef77430 2024-03-19T01:20:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.074+0000 7f5e0d174740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5e041b5b90 con 0x5576bef6df30 2024-03-19T01:20:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.075+0000 7f5e03400640 1 -- 172.21.15.69:0/253738786 learned_addr learned my addr 172.21.15.69:0/253738786 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.075+0000 7f5e03e00640 1 --2- 172.21.15.69:0/253738786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef82f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.075+0000 7f5e02a00640 1 --2- 172.21.15.69:0/253738786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6e850 0x5576bef734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.076+0000 7f5e03400640 1 -- 172.21.15.69:0/253738786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 msgr2=0x5576bef82f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.076+0000 7f5e03400640 1 --2- 172.21.15.69:0/253738786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef82f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.076+0000 7f5e03400640 1 -- 172.21.15.69:0/253738786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6e850 msgr2=0x5576bef734b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:20:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.076+0000 7f5e03400640 1 --2- 172.21.15.69:0/253738786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6e850 0x5576bef734b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.076+0000 7f5e03400640 1 -- 172.21.15.69:0/253738786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576bef8a910 con 0x5576bef6df30 2024-03-19T01:20:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.076+0000 7f5e02a00640 1 --2- 172.21.15.69:0/253738786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6e850 0x5576bef734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:20:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.077+0000 7f5e03e00640 1 --2- 172.21.15.69:0/253738786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef82f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:20:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.080+0000 7f5e03400640 1 --2- 172.21.15.69:0/253738786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6df30 0x5576bef76ca0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5df40057a0 tx=0x7f5df409bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b8da80fb3f48b9b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.080+0000 7f5defe00640 1 -- 172.21.15.69:0/253738786 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40afe60 con 0x5576bef6df30 2024-03-19T01:20:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.080+0000 7f5defe00640 1 -- 172.21.15.69:0/253738786 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df409fbd0 con 0x5576bef6df30 2024-03-19T01:20:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.080+0000 7f5defe00640 1 -- 172.21.15.69:0/253738786 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40b4b40 con 0x5576bef6df30 2024-03-19T01:20:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.081+0000 7f5e0d174740 1 -- 172.21.15.69:0/253738786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6df30 msgr2=0x5576bef76ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.081+0000 7f5e0d174740 1 --2- 172.21.15.69:0/253738786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6df30 0x5576bef76ca0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5df40057a0 tx=0x7f5df409bc00 comp rx=0 tx=0).stop 2024-03-19T01:20:53.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.083+0000 7f5e0d174740 1 -- 172.21.15.69:0/253738786 shutdown_connections 2024-03-19T01:20:53.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.083+0000 7f5e0d174740 1 --2- 172.21.15.69:0/253738786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef77430 0x5576bef82f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.083+0000 7f5e0d174740 1 --2- 172.21.15.69:0/253738786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6e850 0x5576bef734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.083+0000 7f5e0d174740 1 --2- 172.21.15.69:0/253738786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef6df30 0x5576bef76ca0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.083+0000 7f5e0d174740 1 -- 172.21.15.69:0/253738786 >> 172.21.15.69:0/253738786 conn(0x5576bef68660 msgr2=0x5576bef676e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.084+0000 7f5e0d174740 1 -- 172.21.15.69:0/253738786 shutdown_connections 2024-03-19T01:20:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.084+0000 7f5e0d174740 1 -- 172.21.15.69:0/253738786 wait complete. 2024-03-19T01:20:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.085+0000 7f5e0d174740 1 -- start start 2024-03-19T01:20:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.085+0000 7f5e0d174740 1 Processor -- start 2024-03-19T01:20:53.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.087+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef7ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:53.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.087+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef6e850 0x5576bef7b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:53.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.087+0000 7f5e0d174740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef77430 0x5576bef7b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:53.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.088+0000 7f5e0d174740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e041b6930 con 0x5576bef6df30 2024-03-19T01:20:53.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.088+0000 7f5e0d174740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e041b6260 con 0x5576bef6e850 2024-03-19T01:20:53.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.088+0000 7f5e0d174740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5e041b3460 con 0x5576bef77430 2024-03-19T01:20:53.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.088+0000 7f5e0d174740 10 client.0 _ll_register_callbacks cb 0x7f5df409b3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:20:53.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.088+0000 7f5e03400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef7ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:53.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.088+0000 7f5e03400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef7ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34234/0 (socket says 172.21.15.69:34234) 2024-03-19T01:20:53.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.088+0000 7f5e03400640 1 -- 172.21.15.69:0/1814482338 learned_addr learned my addr 172.21.15.69:0/1814482338 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:20:53.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.089+0000 7f5e03400640 1 -- 172.21.15.69:0/1814482338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef77430 msgr2=0x5576bef7b8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:20:53.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.089+0000 7f5e03400640 1 --2- 172.21.15.69:0/1814482338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef77430 0x5576bef7b8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.089+0000 7f5e03400640 1 -- 172.21.15.69:0/1814482338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef6e850 msgr2=0x5576bef7b390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:20:53.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.089+0000 7f5e03400640 1 --2- 172.21.15.69:0/1814482338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef6e850 0x5576bef7b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.089+0000 7f5e03400640 1 -- 172.21.15.69:0/1814482338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576bef864b0 con 0x5576bef6df30 2024-03-19T01:20:53.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.090+0000 7f5ded600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:20:53.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.091+0000 7f5e03400640 1 --2- 172.21.15.69:0/1814482338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef7ae50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5df409b780 tx=0x7f5df4097da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7947aa06baba7fac server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:20:53.092 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/nuvoton/built-in.a 2024-03-19T01:20:53.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.091+0000 7f5ded600640 1 -- 172.21.15.69:0/1814482338 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40afe60 con 0x5576bef6df30 2024-03-19T01:20:53.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.092+0000 7f5ded600640 1 -- 172.21.15.69:0/1814482338 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9568+0+0 (secure 0 0 0) 0x7f5df4010430 con 0x5576bef6df30 2024-03-19T01:20:53.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.092+0000 7f5e0d174740 1 -- 172.21.15.69:0/1814482338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df409fbd0 con 0x5576bef6df30 2024-03-19T01:20:53.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.092+0000 7f5ded600640 1 -- 172.21.15.69:0/1814482338 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f5df4080050 con 0x5576bef6df30 2024-03-19T01:20:53.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.093+0000 7f5e0d174740 10 client.17373 fetch_fsmap learned FSMap version 59 2024-03-19T01:20:53.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.093+0000 7f5e0d174740 1 -- 172.21.15.69:0/1814482338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5576bef864b0 con 0x5576bef6df30 2024-03-19T01:20:53.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.094+0000 7f5ded600640 1 -- 172.21.15.69:0/1814482338 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df409fbd0 con 0x5576bef6df30 2024-03-19T01:20:53.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.094+0000 7f5ded600640 1 -- 172.21.15.69:0/1814482338 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f5df409aab0 con 0x5576bef6df30 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.094+0000 7f5e0d174740 10 client.17373 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.094+0000 7f5e0d174740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.094+0000 7f5e0d174740 20 client.17373 populate_metadata read hostname 'smithi069' 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.094+0000 7f5e0d174740 1 --2- 172.21.15.69:0/1814482338 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5576befee5c0 0x5576beff09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.094+0000 7f5e0d174740 4 client.17373 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.094+0000 7f5e0d174740 1 -- 172.21.15.69:0/1814482338 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5576beedf800 con 0x5576befee5c0 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.095+0000 7f5e02a00640 1 --2- 172.21.15.69:0/1814482338 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5576befee5c0 0x5576beff09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.097+0000 7f5e02a00640 1 --2- 172.21.15.69:0/1814482338 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5576befee5c0 0x5576beff09b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7c74c73fbbf356a server_cookie=ab9fe901f0781dd0 in_seq=0 out_seq=0 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.097+0000 7f5ded600640 10 client.17373 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.097+0000 7f5ded600640 1 -- 172.21.15.69:0/1814482338 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5576beedf800 con 0x5576befee5c0 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.097+0000 7f5ded600640 10 client.17373 handle_command_reply: tid=0 2024-03-19T01:20:53.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.097+0000 7f5e0d174740 1 client.17373 shutdown 2024-03-19T01:20:53.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.097+0000 7f5e0d174740 1 -- 172.21.15.69:0/1814482338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 msgr2=0x5576bef7ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:53.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.097+0000 7f5e0d174740 1 --2- 172.21.15.69:0/1814482338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef7ae50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5df409b780 tx=0x7f5df4097da0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.098+0000 7f5e0d174740 1 -- 172.21.15.69:0/1814482338 shutdown_connections 2024-03-19T01:20:53.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.098+0000 7f5e0d174740 1 --2- 172.21.15.69:0/1814482338 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5576befee5c0 0x5576beff09b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.098+0000 7f5e0d174740 1 --2- 172.21.15.69:0/1814482338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5576bef77430 0x5576bef7b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.098+0000 7f5e0d174740 1 --2- 172.21.15.69:0/1814482338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576bef6e850 0x5576bef7b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.098+0000 7f5e0d174740 1 --2- 172.21.15.69:0/1814482338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576bef6df30 0x5576bef7ae50 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5df409b780 tx=0x7f5df4097da0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.098+0000 7f5ded600640 0 client.17373 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:20:53.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.098+0000 7f5e0d174740 1 -- 172.21.15.69:0/1814482338 >> 172.21.15.69:0/1814482338 conn(0x5576bef68660 msgr2=0x5576bef74530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:53.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.098+0000 7f5e0d174740 1 -- 172.21.15.69:0/1814482338 shutdown_connections 2024-03-19T01:20:53.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.098+0000 7f5e0d174740 1 -- 172.21.15.69:0/1814482338 wait complete. 2024-03-19T01:20:53.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.099+0000 7f5e0d174740 20 client.17373 trim_cache size 0 max 16384 2024-03-19T01:20:53.102 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:20:53.102 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T01:20:53.102 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:20:53.102 INFO:teuthology.orchestra.run.smithi069.stdout: "b4529a67-55a4-4600-8621-8f60fd0b0d5a": { 2024-03-19T01:20:53.102 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:20:53.102 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "b4529a67-55a4-4600-8621-8f60fd0b0d5a", 2024-03-19T01:20:53.102 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:20:53.102 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:20:53.102 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:20:53.102 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:20:53.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 -- 172.21.15.69:0/2304940490 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5de80a2a70 msgr2=0x7f5de80a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:53.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5de80a2a70 0x7f5de80a4f30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5df0005fd0 tx=0x7f5df0005dc0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 -- 172.21.15.69:0/2304940490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e04100640 msgr2=0x7f5e04072300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:20:53.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e04100640 0x7f5e04072300 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5df400e9f0 tx=0x7f5df400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 -- 172.21.15.69:0/2304940490 shutdown_connections 2024-03-19T01:20:53.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e04105a40 0x7f5e0410cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04100f60 0x7f5e04072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f5de80a2a70 0x7f5de80a4f30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 --2- 172.21.15.69:0/2304940490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e04100640 0x7f5e04072300 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:20:53.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 -- 172.21.15.69:0/2304940490 >> 172.21.15.69:0/2304940490 conn(0x7f5e040fc070 msgr2=0x7f5e041033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:20:53.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 -- 172.21.15.69:0/2304940490 shutdown_connections 2024-03-19T01:20:53.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:20:53.115+0000 7f5e08e00640 1 -- 172.21.15.69:0/2304940490 wait complete. 2024-03-19T01:20:53.173 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/pxa/built-in.a 2024-03-19T01:20:53.174 INFO:tasks.cephfs.filesystem:scrub status for tag:b4529a67-55a4-4600-8621-8f60fd0b0d5a - {'path': '/', 'tag': 'b4529a67-55a4-4600-8621-8f60fd0b0d5a', 'options': 'recursive,force'} 2024-03-19T01:20:53.218 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/memcat_p.o 2024-03-19T01:20:53.247 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/amlogic/built-in.a 2024-03-19T01:20:53.351 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/qcom/built-in.a 2024-03-19T01:20:53.408 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/autofs/built-in.a 2024-03-19T01:20:53.413 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/renesas/built-in.a 2024-03-19T01:20:53.459 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/rockchip/built-in.a 2024-03-19T01:20:53.501 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/capability.o 2024-03-19T01:20:53.528 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/sifive/built-in.a 2024-03-19T01:20:53.647 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/sunxi/built-in.a 2024-03-19T01:20:53.698 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/ti/built-in.a 2024-03-19T01:20:53.735 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/export.o 2024-03-19T01:20:53.739 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/nmi_backtrace.o 2024-03-19T01:20:53.762 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/xilinx/built-in.a 2024-03-19T01:20:53.804 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/soc/built-in.a 2024-03-19T01:20:53.806 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exconfig.o 2024-03-19T01:20:53.813 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/dma/built-in.a 2024-03-19T01:20:53.961 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/9p/vfs_super.o 2024-03-19T01:20:54.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:53 smithi086 ceph-mon[66829]: pgmap v317: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 893 KiB/s wr, 65 op/s 2024-03-19T01:20:54.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:53 smithi069 ceph-mon[31727]: pgmap v317: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 893 KiB/s wr, 65 op/s 2024-03-19T01:20:54.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:53 smithi163 ceph-mon[67428]: pgmap v317: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 893 KiB/s wr, 65 op/s 2024-03-19T01:20:54.268 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/plist.o 2024-03-19T01:20:54.397 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exconvrt.o 2024-03-19T01:20:54.517 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/ptrace.o 2024-03-19T01:20:54.527 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/tables.o 2024-03-19T01:20:54.689 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/sysfs.o 2024-03-19T01:20:54.989 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/9p/vfs_inode.o 2024-03-19T01:20:55.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:54 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:20:55.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:54 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:20:55.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:54 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:20:55.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:54 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:20:55.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:54 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:20:55.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:54 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:20:55.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:54 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:20:55.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:54 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:20:55.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:54 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:20:55.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:54 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:20:55.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:54 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:20:55.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:54 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:20:55.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:54 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:20:55.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:54 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:20:55.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:54 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:20:55.088 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/excreate.o 2024-03-19T01:20:55.865 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/hostfs/built-in.a 2024-03-19T01:20:55.928 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/9p/vfs_inode_dotl.o 2024-03-19T01:20:55.958 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/9p/vfs_addr.o 2024-03-19T01:20:56.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:55 smithi086 ceph-mon[66829]: pgmap v318: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 802 KiB/s wr, 54 op/s 2024-03-19T01:20:56.006 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exdebug.o 2024-03-19T01:20:56.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:55 smithi069 ceph-mon[31727]: pgmap v318: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 802 KiB/s wr, 54 op/s 2024-03-19T01:20:56.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:55 smithi163 ceph-mon[67428]: pgmap v318: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 802 KiB/s wr, 54 op/s 2024-03-19T01:20:56.097 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/virtio/virtio.o 2024-03-19T01:20:56.150 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/user.o 2024-03-19T01:20:56.219 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/fs_context.o 2024-03-19T01:20:56.258 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/virtio/virtio_ring.o 2024-03-19T01:20:56.568 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exdump.o 2024-03-19T01:20:57.066 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/signal.o 2024-03-19T01:20:57.162 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exfield.o 2024-03-19T01:20:57.179 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/9p/vfs_file.o 2024-03-19T01:20:57.185 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/radix-tree.o 2024-03-19T01:20:57.224 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/9p/vfs_dir.o 2024-03-19T01:20:58.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:57 smithi086 ceph-mon[66829]: pgmap v319: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 715 KiB/s wr, 53 op/s 2024-03-19T01:20:58.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:57 smithi069 ceph-mon[31727]: pgmap v319: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 715 KiB/s wr, 53 op/s 2024-03-19T01:20:58.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:57 smithi163 ceph-mon[67428]: pgmap v319: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 715 KiB/s wr, 53 op/s 2024-03-19T01:20:58.394 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/debugfs/inode.o 2024-03-19T01:20:58.488 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfsroot.o 2024-03-19T01:20:58.590 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/9p/vfs_dentry.o 2024-03-19T01:20:58.616 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exfldio.o 2024-03-19T01:20:58.622 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/blacklist.o 2024-03-19T01:20:58.644 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/virtio/virtio_anchor.o 2024-03-19T01:20:58.898 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/ratelimit.o 2024-03-19T01:20:59.325 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exmisc.o 2024-03-19T01:20:59.334 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/vt/vt_ioctl.o 2024-03-19T01:20:59.548 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/rbtree.o 2024-03-19T01:20:59.568 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/9p/v9fs.o 2024-03-19T01:20:59.814 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/debugfs/file.o 2024-03-19T01:20:59.941 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/sysctl.o 2024-03-19T01:20:59.957 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/virtio/virtio_pci_modern_dev.o 2024-03-19T01:21:00.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:20:59 smithi086 ceph-mon[66829]: pgmap v320: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 579 KiB/s wr, 46 op/s 2024-03-19T01:21:00.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:20:59 smithi069 ceph-mon[31727]: pgmap v320: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 579 KiB/s wr, 46 op/s 2024-03-19T01:21:00.049 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exmutex.o 2024-03-19T01:21:00.065 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:20:59 smithi163 ceph-mon[67428]: pgmap v320: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 579 KiB/s wr, 46 op/s 2024-03-19T01:21:00.137 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/seq_buf.o 2024-03-19T01:21:00.377 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/sys.o 2024-03-19T01:21:00.547 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/9p/fid.o 2024-03-19T01:21:00.681 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/vt/vc_screen.o 2024-03-19T01:21:00.906 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exnames.o 2024-03-19T01:21:01.233 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/virtio/virtio_pci_legacy_dev.o 2024-03-19T01:21:01.242 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/debugfs/built-in.a 2024-03-19T01:21:01.258 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/siphash.o 2024-03-19T01:21:01.322 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/hvc/hvc_console.o 2024-03-19T01:21:01.328 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs2super.o 2024-03-19T01:21:01.756 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exoparg1.o 2024-03-19T01:21:01.797 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/9p/xattr.o 2024-03-19T01:21:01.836 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/vt/selection.o 2024-03-19T01:21:02.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:01 smithi086 ceph-mon[66829]: pgmap v321: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 307 KiB/s wr, 32 op/s 2024-03-19T01:21:02.020 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/string.o 2024-03-19T01:21:02.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:01 smithi069 ceph-mon[31727]: pgmap v321: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 307 KiB/s wr, 32 op/s 2024-03-19T01:21:02.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:01 smithi163 ceph-mon[67428]: pgmap v321: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 307 KiB/s wr, 32 op/s 2024-03-19T01:21:02.080 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/virtio/virtio_pci_modern.o 2024-03-19T01:21:02.595 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/umh.o 2024-03-19T01:21:02.597 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exoparg2.o 2024-03-19T01:21:02.622 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/tty/hvc/built-in.a 2024-03-19T01:21:02.714 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/proc.o 2024-03-19T01:21:02.762 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exoparg3.o 2024-03-19T01:21:02.782 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/timerqueue.o 2024-03-19T01:21:03.074 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/vt/keyboard.o 2024-03-19T01:21:03.109 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/9p/built-in.a 2024-03-19T01:21:03.146 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/virtio/virtio_pci_common.o 2024-03-19T01:21:03.257 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exoparg6.o 2024-03-19T01:21:03.265 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/vsprintf.o 2024-03-19T01:21:03.293 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/vt/consolemap.o 2024-03-19T01:21:03.568 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs2xdr.o 2024-03-19T01:21:03.780 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/workqueue.o 2024-03-19T01:21:03.997 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exprep.o 2024-03-19T01:21:04.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:03 smithi086 ceph-mon[66829]: pgmap v322: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.0 MiB/s wr, 79 op/s 2024-03-19T01:21:04.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:03 smithi069 ceph-mon[31727]: pgmap v322: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.0 MiB/s wr, 79 op/s 2024-03-19T01:21:04.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:03 smithi163 ceph-mon[67428]: pgmap v322: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.0 MiB/s wr, 79 op/s 2024-03-19T01:21:04.291 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC drivers/tty/vt/conmakehash 2024-03-19T01:21:04.394 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/virtio/virtio_pci_legacy.o 2024-03-19T01:21:04.596 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exregion.o 2024-03-19T01:21:04.631 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exresnte.o 2024-03-19T01:21:04.736 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/vt/vt.o 2024-03-19T01:21:04.863 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_core.o 2024-03-19T01:21:05.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:04 smithi086 ceph-mon[66829]: osdmap e103: 12 total, 12 up, 12 in 2024-03-19T01:21:05.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:04 smithi069 ceph-mon[31727]: osdmap e103: 12 total, 12 up, 12 in 2024-03-19T01:21:05.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:04 smithi163 ceph-mon[67428]: osdmap e103: 12 total, 12 up, 12 in 2024-03-19T01:21:05.247 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/serial_core.o 2024-03-19T01:21:05.398 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/virtio/virtio_input.o 2024-03-19T01:21:05.444 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exresolv.o 2024-03-19T01:21:05.719 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs3super.o 2024-03-19T01:21:06.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:05 smithi086 ceph-mon[66829]: pgmap v324: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-03-19T01:21:06.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:05 smithi069 ceph-mon[31727]: pgmap v324: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-03-19T01:21:06.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:05 smithi163 ceph-mon[67428]: pgmap v324: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-03-19T01:21:06.087 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exresop.o 2024-03-19T01:21:06.202 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_pnp.o 2024-03-19T01:21:06.307 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/win_minmax.o 2024-03-19T01:21:06.429 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/virtio/virtio_dma_buf.o 2024-03-19T01:21:06.591 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/pid.o 2024-03-19T01:21:06.853 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exserial.o 2024-03-19T01:21:07.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:06 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:21:07.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:06 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:21:07.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:06 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:21:07.107 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/xarray.o 2024-03-19T01:21:07.210 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_port.o 2024-03-19T01:21:07.350 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs3client.o 2024-03-19T01:21:07.531 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/virtio/built-in.a 2024-03-19T01:21:07.585 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exstore.o 2024-03-19T01:21:07.652 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_dma.o 2024-03-19T01:21:07.659 INFO:tasks.workunit.client.0.smithi069.stdout: COPY drivers/tty/vt/defkeymap.c 2024-03-19T01:21:07.719 INFO:tasks.workunit.client.0.smithi069.stdout: CONMK drivers/tty/vt/consolemap_deftbl.c 2024-03-19T01:21:07.732 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/vt/defkeymap.o 2024-03-19T01:21:07.805 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/osi.o 2024-03-19T01:21:08.002 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:07 smithi086 ceph-mon[66829]: pgmap v325: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-19T01:21:08.002 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:07 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.002 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:07 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.002 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:07 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.002 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:07 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:07 smithi069 ceph-mon[31727]: pgmap v325: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-19T01:21:08.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:07 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:07 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:07 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:07 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:07 smithi163 ceph-mon[67428]: pgmap v325: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-19T01:21:08.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:07 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:07 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:07 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:07 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:08.092 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/vt/consolemap_deftbl.o 2024-03-19T01:21:08.172 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/tty/vt/built-in.a 2024-03-19T01:21:08.292 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/tracefs/inode.o 2024-03-19T01:21:08.318 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exstoren.o 2024-03-19T01:21:08.697 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/lockref.o 2024-03-19T01:21:08.784 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/osl.o 2024-03-19T01:21:08.785 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/task_work.o 2024-03-19T01:21:08.933 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_dwlib.o 2024-03-19T01:21:09.191 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/bcd.o 2024-03-19T01:21:09.223 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs3proc.o 2024-03-19T01:21:09.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:08 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:08 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:08 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:08 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:08 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.256 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:08 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.267 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/utils.o 2024-03-19T01:21:09.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:08 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:08 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:08 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:08 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:08 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:08 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.294 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exstorob.o 2024-03-19T01:21:09.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:08 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:08 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:08 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:08 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:08 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:08 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:09.510 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/sort.o 2024-03-19T01:21:09.587 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/tracefs/built-in.a 2024-03-19T01:21:09.656 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/parser.o 2024-03-19T01:21:09.859 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/serial_base_bus.o 2024-03-19T01:21:10.062 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/extable.o 2024-03-19T01:21:10.063 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exsystem.o 2024-03-19T01:21:10.075 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_pcilib.o 2024-03-19T01:21:10.178 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/params.o 2024-03-19T01:21:10.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: pgmap v326: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-03-19T01:21:10.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: osdmap e104: 12 total, 12 up, 12 in 2024-03-19T01:21:10.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:10 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: pgmap v326: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-03-19T01:21:10.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: osdmap e104: 12 total, 12 up, 12 in 2024-03-19T01:21:10.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.312 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.313 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.314 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:10 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: pgmap v326: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-03-19T01:21:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: osdmap e104: 12 total, 12 up, 12 in 2024-03-19T01:21:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:10 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:10.567 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_pci.o 2024-03-19T01:21:10.624 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/debug_locks.o 2024-03-19T01:21:10.819 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/serial_ctrl.o 2024-03-19T01:21:10.930 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/extrace.o 2024-03-19T01:21:11.010 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs3xdr.o 2024-03-19T01:21:11.081 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/random32.o 2024-03-19T01:21:11.117 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/hw_random/core.o 2024-03-19T01:21:11.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:11 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi163 to 174.1M 2024-03-19T01:21:11.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:11 smithi069 ceph-mon[31727]: Unable to set osd_memory_target on smithi163 to 182627737: error parsing value: Value '182627737' is below minimum 939524096 2024-03-19T01:21:11.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:11 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi086 to 174.1M 2024-03-19T01:21:11.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:11 smithi069 ceph-mon[31727]: Unable to set osd_memory_target on smithi086 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-19T01:21:11.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:11 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi163 to 174.1M 2024-03-19T01:21:11.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:11 smithi163 ceph-mon[67428]: Unable to set osd_memory_target on smithi163 to 182627737: error parsing value: Value '182627737' is below minimum 939524096 2024-03-19T01:21:11.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:11 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi086 to 174.1M 2024-03-19T01:21:11.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:11 smithi163 ceph-mon[67428]: Unable to set osd_memory_target on smithi086 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-19T01:21:11.492 INFO:tasks.workunit.client.0.smithi069.stdout: CC [M] fs/efivarfs/inode.o 2024-03-19T01:21:11.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:11 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi163 to 174.1M 2024-03-19T01:21:11.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:11 smithi086 ceph-mon[66829]: Unable to set osd_memory_target on smithi163 to 182627737: error parsing value: Value '182627737' is below minimum 939524096 2024-03-19T01:21:11.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:11 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi086 to 174.1M 2024-03-19T01:21:11.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:11 smithi086 ceph-mon[66829]: Unable to set osd_memory_target on smithi086 to 182632755: error parsing value: Value '182632755' is below minimum 939524096 2024-03-19T01:21:11.540 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/exutils.o 2024-03-19T01:21:11.552 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/kthread.o 2024-03-19T01:21:11.745 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/tty/ipwireless/built-in.a 2024-03-19T01:21:11.797 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/bust_spinlocks.o 2024-03-19T01:21:11.808 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/reboot.o 2024-03-19T01:21:12.074 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_exar.o 2024-03-19T01:21:12.097 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/hw_random/via-rng.o 2024-03-19T01:21:12.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:12 smithi069 ceph-mon[31727]: pgmap v328: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 326 KiB/s wr, 125 op/s 2024-03-19T01:21:12.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:12 smithi163 ceph-mon[67428]: pgmap v328: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 326 KiB/s wr, 125 op/s 2024-03-19T01:21:12.427 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/hwacpi.o 2024-03-19T01:21:12.485 INFO:tasks.workunit.client.0.smithi069.stdout: CC [M] fs/efivarfs/file.o 2024-03-19T01:21:12.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:12 smithi086 ceph-mon[66829]: pgmap v328: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 326 KiB/s wr, 125 op/s 2024-03-19T01:21:12.892 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/kasprintf.o 2024-03-19T01:21:13.037 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/bitmap.o 2024-03-19T01:21:13.169 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/hwesleep.o 2024-03-19T01:21:13.189 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_early.o 2024-03-19T01:21:13.265 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs3acl.o 2024-03-19T01:21:13.334 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/char/hw_random/built-in.a 2024-03-19T01:21:13.431 INFO:tasks.workunit.client.0.smithi069.stdout: CC [M] fs/efivarfs/super.o 2024-03-19T01:21:13.462 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/agp/backend.o 2024-03-19T01:21:13.587 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_lpss.o 2024-03-19T01:21:13.602 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/sys_ni.o 2024-03-19T01:21:13.751 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/hwgpe.o 2024-03-19T01:21:14.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:14 smithi163 ceph-mon[67428]: pgmap v329: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 280 KiB/s wr, 116 op/s 2024-03-19T01:21:14.354 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/tty_io.o 2024-03-19T01:21:14.465 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/nsproxy.o 2024-03-19T01:21:14.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:14 smithi086 ceph-mon[66829]: pgmap v329: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 280 KiB/s wr, 116 op/s 2024-03-19T01:21:14.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:14 smithi069 ceph-mon[31727]: pgmap v329: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 280 KiB/s wr, 116 op/s 2024-03-19T01:21:14.574 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/hwregs.o 2024-03-19T01:21:14.637 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/scatterlist.o 2024-03-19T01:21:14.649 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/agp/generic.o 2024-03-19T01:21:14.668 INFO:tasks.workunit.client.0.smithi069.stdout: CC [M] fs/efivarfs/vars.o 2024-03-19T01:21:14.678 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_mid.o 2024-03-19T01:21:15.027 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4proc.o 2024-03-19T01:21:15.418 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/hwsleep.o 2024-03-19T01:21:15.931 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/8250/8250_pericom.o 2024-03-19T01:21:16.197 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/list_sort.o 2024-03-19T01:21:16.210 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/notifier.o 2024-03-19T01:21:16.226 INFO:tasks.workunit.client.0.smithi069.stdout: LD [M] fs/efivarfs/efivarfs.o 2024-03-19T01:21:16.278 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/agp/isoch.o 2024-03-19T01:21:16.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:16 smithi163 ceph-mon[67428]: pgmap v330: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 246 KiB/s wr, 102 op/s 2024-03-19T01:21:16.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:16 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:16.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:16 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:16.335 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/hwvalid.o 2024-03-19T01:21:16.390 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/ksysfs.o 2024-03-19T01:21:16.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:16 smithi086 ceph-mon[66829]: pgmap v330: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 246 KiB/s wr, 102 op/s 2024-03-19T01:21:16.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:16 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:16.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:16 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:16.506 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/uuid.o 2024-03-19T01:21:16.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:16 smithi069 ceph-mon[31727]: pgmap v330: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 246 KiB/s wr, 102 op/s 2024-03-19T01:21:16.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:16 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:16.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:16 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:16.892 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/amd/iommu.o 2024-03-19T01:21:16.925 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/iov_iter.o 2024-03-19T01:21:17.008 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/tty/serial/8250/built-in.a 2024-03-19T01:21:17.047 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/hwxface.o 2024-03-19T01:21:17.212 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/serial_port.o 2024-03-19T01:21:17.381 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/host1x/built-in.a 2024-03-19T01:21:17.430 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/cred.o 2024-03-19T01:21:17.451 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/agp/amd64-agp.o 2024-03-19T01:21:17.493 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/tests/built-in.a 2024-03-19T01:21:17.569 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/arm/built-in.a 2024-03-19T01:21:17.647 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/display/drm_display_helper_mod.o 2024-03-19T01:21:17.685 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/hwxfsleep.o 2024-03-19T01:21:18.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:18 smithi163 ceph-mon[67428]: pgmap v331: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 257 KiB/s wr, 62 op/s 2024-03-19T01:21:18.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:18 smithi086 ceph-mon[66829]: pgmap v331: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 257 KiB/s wr, 62 op/s 2024-03-19T01:21:18.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:18 smithi069 ceph-mon[31727]: pgmap v331: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 257 KiB/s wr, 62 op/s 2024-03-19T01:21:19.041 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/reboot.o 2024-03-19T01:21:19.123 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/display/drm_dp_dual_mode_helper.o 2024-03-19T01:21:19.141 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/hwpci.o 2024-03-19T01:21:19.163 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/serial/earlycon.o 2024-03-19T01:21:19.184 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/agp/intel-agp.o 2024-03-19T01:21:19.280 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/amd/init.o 2024-03-19T01:21:19.470 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:19 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.470 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:19 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.470 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:19 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.470 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:19 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.470 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:19 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.470 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:19 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.470 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:19 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.471 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:19 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.471 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:19 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:21:19.471 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:19 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:21:19.471 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:19 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:19 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:19 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:19 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:19 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:19 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:19 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:19 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:19 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:19 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:21:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:19 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:21:19.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:19 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:19.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:19 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:19 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:19 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:19 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:19 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:19 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:19 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:19 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:21:19.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:19 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:21:19.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:19 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:21:19.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:19 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:21:19.710 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4xdr.o 2024-03-19T01:21:19.797 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/clz_ctz.o 2024-03-19T01:21:20.083 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsaccess.o 2024-03-19T01:21:20.113 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/bsearch.o 2024-03-19T01:21:20.252 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/tty/serial/built-in.a 2024-03-19T01:21:20.335 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/display/drm_dp_helper.o 2024-03-19T01:21:20.366 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/agp/intel-gtt.o 2024-03-19T01:21:20.478 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/async.o 2024-03-19T01:21:20.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:20 smithi086 ceph-mon[66829]: pgmap v332: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 310 KiB/s wr, 26 op/s 2024-03-19T01:21:20.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:20 smithi086 ceph-mon[66829]: Adjusting osd_memory_target on smithi069 to 176.7M 2024-03-19T01:21:20.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:20 smithi086 ceph-mon[66829]: Unable to set osd_memory_target on smithi069 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-19T01:21:20.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:20 smithi069 ceph-mon[31727]: pgmap v332: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 310 KiB/s wr, 26 op/s 2024-03-19T01:21:20.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:20 smithi069 ceph-mon[31727]: Adjusting osd_memory_target on smithi069 to 176.7M 2024-03-19T01:21:20.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:20 smithi069 ceph-mon[31727]: Unable to set osd_memory_target on smithi069 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-19T01:21:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:20 smithi163 ceph-mon[67428]: pgmap v332: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 310 KiB/s wr, 26 op/s 2024-03-19T01:21:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:20 smithi163 ceph-mon[67428]: Adjusting osd_memory_target on smithi069 to 176.7M 2024-03-19T01:21:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:20 smithi163 ceph-mon[67428]: Unable to set osd_memory_target on smithi069 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-19T01:21:20.625 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/n_tty.o 2024-03-19T01:21:20.978 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsalloc.o 2024-03-19T01:21:21.090 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/find_bit.o 2024-03-19T01:21:21.273 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/range.o 2024-03-19T01:21:21.516 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/llist.o 2024-03-19T01:21:21.535 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/smpboot.o 2024-03-19T01:21:21.633 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/amd/quirks.o 2024-03-19T01:21:21.664 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsarguments.o 2024-03-19T01:21:21.683 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/char/agp/built-in.a 2024-03-19T01:21:21.788 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/memweight.o 2024-03-19T01:21:21.836 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/mem.o 2024-03-19T01:21:22.259 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/tty_ioctl.o 2024-03-19T01:21:22.260 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsconvert.o 2024-03-19T01:21:22.262 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/kfifo.o 2024-03-19T01:21:22.307 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/display/drm_dp_mst_topology.o 2024-03-19T01:21:22.332 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/ucount.o 2024-03-19T01:21:22.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:22 smithi086 ceph-mon[66829]: pgmap v333: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 459 KiB/s wr, 34 op/s 2024-03-19T01:21:22.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:22 smithi069 ceph-mon[31727]: pgmap v333: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 459 KiB/s wr, 34 op/s 2024-03-19T01:21:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:22 smithi163 ceph-mon[67428]: pgmap v333: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 459 KiB/s wr, 34 op/s 2024-03-19T01:21:22.735 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/amd/io_pgtable.o 2024-03-19T01:21:22.880 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/random.o 2024-03-19T01:21:22.958 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsdump.o 2024-03-19T01:21:22.982 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/regset.o 2024-03-19T01:21:23.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:21:23.175 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:21:23.343 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4state.o 2024-03-19T01:21:23.372 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/tty_ldisc.o 2024-03-19T01:21:23.512 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nseval.o 2024-03-19T01:21:23.523 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/percpu-refcount.o 2024-03-19T01:21:23.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.714+0000 7fe4bca00640 1 -- 172.21.15.69:0/2875657196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4b8116820 msgr2=0x7fe4b8118c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:23.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.714+0000 7fe4bca00640 1 --2- 172.21.15.69:0/2875657196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4b8116820 0x7fe4b8118c30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009ca0 tx=0x7fe4b002c1a0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.718+0000 7fe4bca00640 1 -- 172.21.15.69:0/2875657196 shutdown_connections 2024-03-19T01:21:23.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.718+0000 7fe4bca00640 1 --2- 172.21.15.69:0/2875657196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4b8116820 0x7fe4b8118c30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.718+0000 7fe4bca00640 1 --2- 172.21.15.69:0/2875657196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4b8108d60 0x7fe4b81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.718+0000 7fe4bca00640 1 --2- 172.21.15.69:0/2875657196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b810bf40 0x7fe4b8108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.718+0000 7fe4bca00640 1 -- 172.21.15.69:0/2875657196 >> 172.21.15.69:0/2875657196 conn(0x7fe4b806cd60 msgr2=0x7fe4b806d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:23.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.720+0000 7fe4bca00640 1 -- 172.21.15.69:0/2875657196 shutdown_connections 2024-03-19T01:21:23.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.724+0000 7fe4bca00640 1 -- 172.21.15.69:0/2875657196 wait complete. 2024-03-19T01:21:23.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.725+0000 7fe4bca00640 1 Processor -- start 2024-03-19T01:21:23.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.725+0000 7fe4bca00640 1 -- start start 2024-03-19T01:21:23.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.726+0000 7fe4bca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4b8108d60 0x7fe4b810c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.726+0000 7fe4bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b810bf40 0x7fe4b810c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.726+0000 7fe4bca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4b8116820 0x7fe4b810cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.726+0000 7fe4bca00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b811c5f0 con 0x7fe4b8108d60 2024-03-19T01:21:23.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.726+0000 7fe4bca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b811c470 con 0x7fe4b810bf40 2024-03-19T01:21:23.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.726+0000 7fe4b5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4b8108d60 0x7fe4b810c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.726+0000 7fe4b6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4b8116820 0x7fe4b810cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.726+0000 7fe4b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b810bf40 0x7fe4b810c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.726+0000 7fe4b6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4b8116820 0x7fe4b810cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59426/0 (socket says 172.21.15.69:59426) 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.726+0000 7fe4b6800640 1 -- 172.21.15.69:0/1132950763 learned_addr learned my addr 172.21.15.69:0/1132950763 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.727+0000 7fe4bca00640 1 -- 172.21.15.69:0/1132950763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b811c770 con 0x7fe4b8116820 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.727+0000 7fe4b6800640 1 -- 172.21.15.69:0/1132950763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b810bf40 msgr2=0x7fe4b810c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.727+0000 7fe4b6800640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b810bf40 0x7fe4b810c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.727+0000 7fe4b6800640 1 -- 172.21.15.69:0/1132950763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4b8108d60 msgr2=0x7fe4b810c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.727+0000 7fe4b6800640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4b8108d60 0x7fe4b810c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.727+0000 7fe4b6800640 1 -- 172.21.15.69:0/1132950763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b81b3260 con 0x7fe4b8116820 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.727+0000 7fe4b6800640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4b8116820 0x7fe4b810cf20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0037040 tx=0x7fe4b0035e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.728+0000 7fe4aea00640 1 -- 172.21.15.69:0/1132950763 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0039040 con 0x7fe4b8116820 2024-03-19T01:21:23.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.728+0000 7fe4aea00640 1 -- 172.21.15.69:0/1132950763 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b0044070 con 0x7fe4b8116820 2024-03-19T01:21:23.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.728+0000 7fe4aea00640 1 -- 172.21.15.69:0/1132950763 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00328d0 con 0x7fe4b8116820 2024-03-19T01:21:23.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.728+0000 7fe4bca00640 1 -- 172.21.15.69:0/1132950763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b81b34f0 con 0x7fe4b8116820 2024-03-19T01:21:23.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.728+0000 7fe4bca00640 1 -- 172.21.15.69:0/1132950763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b81b3990 con 0x7fe4b8116820 2024-03-19T01:21:23.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.735+0000 7fe4aea00640 1 -- 172.21.15.69:0/1132950763 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fe4b0032a70 con 0x7fe4b8116820 2024-03-19T01:21:23.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.737+0000 7fe4aea00640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe49c0a2a30 0x7fe49c0a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.737+0000 7fe4aea00640 1 -- 172.21.15.69:0/1132950763 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fe4b00f14b0 con 0x7fe4b8116820 2024-03-19T01:21:23.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.738+0000 7fe4b5e00640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe49c0a2a30 0x7fe49c0a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.739+0000 7fe4bf612740 1 Processor -- start 2024-03-19T01:21:23.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.740+0000 7fe4b5e00640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe49c0a2a30 0x7fe49c0a4ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8006fd0 tx=0x7fe4a80090e0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:23.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4bf612740 1 -- start start 2024-03-19T01:21:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d176120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 0x55ff9d17ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 0x55ff9d181630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4bf612740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ff9d0da2c0 con 0x55ff9d176660 2024-03-19T01:21:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4bf612740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff9d082b60 con 0x55ff9d17f240 2024-03-19T01:21:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4bf612740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ff9d0d6360 con 0x55ff9d175d40 2024-03-19T01:21:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4b5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d176120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 0x55ff9d181630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 0x55ff9d181630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52806/0 (socket says 172.21.15.69:52806) 2024-03-19T01:21:23.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4b6800640 1 -- 172.21.15.69:0/2905236510 learned_addr learned my addr 172.21.15.69:0/2905236510 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:23.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.743+0000 7fe4b5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d176120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59430/0 (socket says 172.21.15.69:59430) 2024-03-19T01:21:23.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.744+0000 7fe4b6800640 1 -- 172.21.15.69:0/2905236510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 msgr2=0x55ff9d176120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:23.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.744+0000 7fe4b6800640 1 --2- 172.21.15.69:0/2905236510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d176120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.744+0000 7fe4b6800640 1 -- 172.21.15.69:0/2905236510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 msgr2=0x55ff9d17ed00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:21:23.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.745+0000 7fe4b5400640 1 --2- 172.21.15.69:0/2905236510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 0x55ff9d17ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.745+0000 7fe4b6800640 1 --2- 172.21.15.69:0/2905236510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 0x55ff9d17ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.745+0000 7fe4b6800640 1 -- 172.21.15.69:0/2905236510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff9ceb87d0 con 0x55ff9d17f240 2024-03-19T01:21:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.745+0000 7fe4b6800640 1 --2- 172.21.15.69:0/2905236510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 0x55ff9d181630 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00b9c70 tx=0x7fe4b002c1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f60d405067c1974b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.746+0000 7fe4a3000640 1 -- 172.21.15.69:0/2905236510 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00bdf10 con 0x55ff9d17f240 2024-03-19T01:21:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.746+0000 7fe4a3000640 1 -- 172.21.15.69:0/2905236510 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b00bd810 con 0x55ff9d17f240 2024-03-19T01:21:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.746+0000 7fe4b5400640 1 --2- 172.21.15.69:0/2905236510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 0x55ff9d17ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.747+0000 7fe4bf612740 1 -- 172.21.15.69:0/2905236510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 msgr2=0x55ff9d181630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.747+0000 7fe4bf612740 1 --2- 172.21.15.69:0/2905236510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 0x55ff9d181630 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00b9c70 tx=0x7fe4b002c1a0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.746+0000 7fe4a3000640 1 -- 172.21.15.69:0/2905236510 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 0 0 0) 0x7fe4b0004040 con 0x55ff9d17f240 2024-03-19T01:21:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.748+0000 7fe4b5e00640 1 --2- 172.21.15.69:0/2905236510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d176120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.749+0000 7fe4bf612740 1 -- 172.21.15.69:0/2905236510 shutdown_connections 2024-03-19T01:21:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.749+0000 7fe4bf612740 1 --2- 172.21.15.69:0/2905236510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 0x55ff9d181630 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.749+0000 7fe4bf612740 1 --2- 172.21.15.69:0/2905236510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 0x55ff9d17ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.749+0000 7fe4bf612740 1 --2- 172.21.15.69:0/2905236510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d176120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.749+0000 7fe4bf612740 1 -- 172.21.15.69:0/2905236510 >> 172.21.15.69:0/2905236510 conn(0x55ff9d170470 msgr2=0x55ff9d170850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:23.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.752+0000 7fe4bf612740 1 -- 172.21.15.69:0/2905236510 shutdown_connections 2024-03-19T01:21:23.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.752+0000 7fe4bf612740 1 -- 172.21.15.69:0/2905236510 wait complete. 2024-03-19T01:21:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.752+0000 7fe4bf612740 1 -- start start 2024-03-19T01:21:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.753+0000 7fe4bf612740 1 Processor -- start 2024-03-19T01:21:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.754+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d17c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.758+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 0x55ff9d17c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.758+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 0x55ff9d17cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.758+0000 7fe4bf612740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ff9d181df0 con 0x55ff9d176660 2024-03-19T01:21:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.758+0000 7fe4bf612740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff9d095fc0 con 0x55ff9d17f240 2024-03-19T01:21:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.758+0000 7fe4bf612740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ff9d181f70 con 0x55ff9d175d40 2024-03-19T01:21:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.758+0000 7fe4bf612740 10 client.0 _ll_register_callbacks cb 0x55ff9d095990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:23.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.758+0000 7fe4b5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d17c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.758+0000 7fe4b5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d17c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59434/0 (socket says 172.21.15.69:59434) 2024-03-19T01:21:23.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.758+0000 7fe4b5e00640 1 -- 172.21.15.69:0/1840917559 learned_addr learned my addr 172.21.15.69:0/1840917559 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:23.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.758+0000 7fe4b5400640 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 0x55ff9d17c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.758+0000 7fe4b6800640 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 0x55ff9d17cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.759+0000 7fe4b5e00640 1 -- 172.21.15.69:0/1840917559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 msgr2=0x55ff9d17cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:23.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.759+0000 7fe4b5e00640 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 0x55ff9d17cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.759+0000 7fe4b5e00640 1 -- 172.21.15.69:0/1840917559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 msgr2=0x55ff9d17c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:23.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.759+0000 7fe4b5e00640 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 0x55ff9d17c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.759+0000 7fe4b5e00640 1 -- 172.21.15.69:0/1840917559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9cdee3f0 con 0x55ff9d175d40 2024-03-19T01:21:23.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.759+0000 7fe4b5400640 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 0x55ff9d17c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:21:23.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.759+0000 7fe4b5e00640 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d17c1a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800f000 tx=0x7fe4a8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7d07979306ec14d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:23.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.759+0000 7fe497400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:21:23.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.760+0000 7fe497400640 1 -- 172.21.15.69:0/1840917559 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8024020 con 0x55ff9d175d40 2024-03-19T01:21:23.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.760+0000 7fe497400640 1 -- 172.21.15.69:0/1840917559 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fe4a800f040 con 0x55ff9d175d40 2024-03-19T01:21:23.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.760+0000 7fe4bf612740 1 -- 172.21.15.69:0/1840917559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b00bd810 con 0x55ff9d175d40 2024-03-19T01:21:23.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.760+0000 7fe497400640 1 -- 172.21.15.69:0/1840917559 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe4a802f050 con 0x55ff9d175d40 2024-03-19T01:21:23.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.760+0000 7fe4bf612740 10 client.27673 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:23.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.760+0000 7fe4bf612740 1 -- 172.21.15.69:0/1840917559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ff9cdae0f0 con 0x55ff9d175d40 2024-03-19T01:21:23.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.760+0000 7fe497400640 1 -- 172.21.15.69:0/1840917559 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b00bd810 con 0x55ff9d175d40 2024-03-19T01:21:23.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.761+0000 7fe497400640 1 -- 172.21.15.69:0/1840917559 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe4a8013870 con 0x55ff9d175d40 2024-03-19T01:21:23.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.761+0000 7fe4bf612740 10 client.27673 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:23.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.761+0000 7fe4bf612740 10 client.27673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:21:23.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.761+0000 7fe4bf612740 20 client.27673 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:23.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.761+0000 7fe4bf612740 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff9d18f2e0 0x55ff9d191780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.761+0000 7fe4bf612740 4 client.27673 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:21:23.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.761+0000 7fe4bf612740 1 -- 172.21.15.69:0/1840917559 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff9d07b610 con 0x55ff9d18f2e0 2024-03-19T01:21:23.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.761+0000 7fe4b5400640 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff9d18f2e0 0x55ff9d191780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.762+0000 7fe497400640 10 client.27673 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:23.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.762+0000 7fe4b5400640 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff9d18f2e0 0x55ff9d191780 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed4554df107c6afa server_cookie=4014c33a25e86518 in_seq=0 out_seq=0 2024-03-19T01:21:23.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.763+0000 7fe497400640 1 -- 172.21.15.69:0/1840917559 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ff9d07b610 con 0x55ff9d18f2e0 2024-03-19T01:21:23.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.763+0000 7fe497400640 10 client.27673 handle_command_reply: tid=0 2024-03-19T01:21:23.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.763+0000 7fe4bf612740 1 client.27673 shutdown 2024-03-19T01:21:23.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.763+0000 7fe4bf612740 1 -- 172.21.15.69:0/1840917559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 msgr2=0x55ff9d17c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:23.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.763+0000 7fe4bf612740 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d17c1a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800f000 tx=0x7fe4a8004750 comp rx=0 tx=0).stop 2024-03-19T01:21:23.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.763+0000 7fe4bf612740 1 -- 172.21.15.69:0/1840917559 shutdown_connections 2024-03-19T01:21:23.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.763+0000 7fe4bf612740 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff9d18f2e0 0x55ff9d191780 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.764+0000 7fe4bf612740 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d17f240 0x55ff9d17cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.764+0000 7fe4bf612740 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d176660 0x55ff9d17c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.764+0000 7fe4bf612740 1 --2- 172.21.15.69:0/1840917559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d175d40 0x55ff9d17c1a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.764+0000 7fe4bf612740 1 -- 172.21.15.69:0/1840917559 >> 172.21.15.69:0/1840917559 conn(0x55ff9d170470 msgr2=0x55ff9d1721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:23.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.764+0000 7fe497400640 0 client.27673 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:23.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.767+0000 7fe4bf612740 1 -- 172.21.15.69:0/1840917559 shutdown_connections 2024-03-19T01:21:23.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.767+0000 7fe4bf612740 1 -- 172.21.15.69:0/1840917559 wait complete. 2024-03-19T01:21:23.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.767+0000 7fe4bf612740 20 client.27673 trim_cache size 0 max 16384 2024-03-19T01:21:23.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.845+0000 7fe4bf612740 1 Processor -- start 2024-03-19T01:21:23.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.845+0000 7fe4bf612740 1 -- start start 2024-03-19T01:21:23.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.845+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d175d40 0x55ff9d17e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.846+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d176660 0x55ff9d18abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.846+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d17f240 0x55ff9d197050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.846+0000 7fe4bf612740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81bdd90 con 0x55ff9d175d40 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.846+0000 7fe4bf612740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81bf8d0 con 0x55ff9d176660 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.846+0000 7fe4bf612740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81be460 con 0x55ff9d17f240 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.846+0000 7fe4b5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d175d40 0x55ff9d17e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.846+0000 7fe4b5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d175d40 0x55ff9d17e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41940/0 (socket says 172.21.15.69:41940) 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.846+0000 7fe4b5e00640 1 -- 172.21.15.69:0/2224520597 learned_addr learned my addr 172.21.15.69:0/2224520597 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.846+0000 7fe4b6800640 1 --2- 172.21.15.69:0/2224520597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d17f240 0x55ff9d197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.846+0000 7fe4b5400640 1 --2- 172.21.15.69:0/2224520597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d176660 0x55ff9d18abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.847+0000 7fe4b6800640 1 -- 172.21.15.69:0/2224520597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d176660 msgr2=0x55ff9d18abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.847+0000 7fe4b6800640 1 --2- 172.21.15.69:0/2224520597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d176660 0x55ff9d18abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.847+0000 7fe4b6800640 1 -- 172.21.15.69:0/2224520597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d175d40 msgr2=0x55ff9d17e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.847+0000 7fe4b6800640 1 --2- 172.21.15.69:0/2224520597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d175d40 0x55ff9d17e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.847+0000 7fe4b6800640 1 -- 172.21.15.69:0/2224520597 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff9d176490 con 0x55ff9d17f240 2024-03-19T01:21:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.847+0000 7fe4b5e00640 1 --2- 172.21.15.69:0/2224520597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d175d40 0x55ff9d17e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:21:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.847+0000 7fe4b5400640 1 --2- 172.21.15.69:0/2224520597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d176660 0x55ff9d18abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:21:23.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.851+0000 7fe4b6800640 1 --2- 172.21.15.69:0/2224520597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d17f240 0x55ff9d197050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00b9c70 tx=0x7fe4b002c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6307926e00204b9f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.851+0000 7fe4a2600640 1 -- 172.21.15.69:0/2224520597 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00bb8b0 con 0x55ff9d17f240 2024-03-19T01:21:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.852+0000 7fe4a2600640 1 -- 172.21.15.69:0/2224520597 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b00bdf10 con 0x55ff9d17f240 2024-03-19T01:21:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.852+0000 7fe4a2600640 1 -- 172.21.15.69:0/2224520597 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00a80c0 con 0x55ff9d17f240 2024-03-19T01:21:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.853+0000 7fe4bf612740 1 -- 172.21.15.69:0/2224520597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d17f240 msgr2=0x55ff9d197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.853+0000 7fe4bf612740 1 --2- 172.21.15.69:0/2224520597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d17f240 0x55ff9d197050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00b9c70 tx=0x7fe4b002c9c0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.855+0000 7fe4bf612740 1 -- 172.21.15.69:0/2224520597 shutdown_connections 2024-03-19T01:21:23.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.855+0000 7fe4bf612740 1 --2- 172.21.15.69:0/2224520597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d17f240 0x55ff9d197050 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.855+0000 7fe4bf612740 1 --2- 172.21.15.69:0/2224520597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d176660 0x55ff9d18abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.855+0000 7fe4bf612740 1 --2- 172.21.15.69:0/2224520597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d175d40 0x55ff9d17e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.855+0000 7fe4bf612740 1 -- 172.21.15.69:0/2224520597 >> 172.21.15.69:0/2224520597 conn(0x55ff9d170470 msgr2=0x55ff9d16f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.855+0000 7fe4bf612740 1 -- 172.21.15.69:0/2224520597 shutdown_connections 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.855+0000 7fe4bf612740 1 -- 172.21.15.69:0/2224520597 wait complete. 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.855+0000 7fe4bf612740 1 -- start start 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.856+0000 7fe4bf612740 1 Processor -- start 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.856+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d175d40 0x55ff9d179460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.856+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d176660 0x55ff9d1799a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.856+0000 7fe4bf612740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d17f240 0x55ff9d1f4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.856+0000 7fe4bf612740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81bf200 con 0x55ff9d17f240 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.856+0000 7fe4bf612740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81beb30 con 0x55ff9d175d40 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4bf612740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81bbd70 con 0x55ff9d176660 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4b5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d176660 0x55ff9d1799a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4bf612740 10 client.0 _ll_register_callbacks cb 0x7fe4a800ed00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d175d40 0x55ff9d179460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d175d40 0x55ff9d179460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52846/0 (socket says 172.21.15.69:52846) 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4b5e00640 1 -- 172.21.15.69:0/341142167 learned_addr learned my addr 172.21.15.69:0/341142167 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4b5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d176660 0x55ff9d1799a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59460/0 (socket says 172.21.15.69:59460) 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4b6800640 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d17f240 0x55ff9d1f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4b5e00640 1 -- 172.21.15.69:0/341142167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d176660 msgr2=0x55ff9d1799a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4b5e00640 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d176660 0x55ff9d1799a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4b5e00640 1 -- 172.21.15.69:0/341142167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d17f240 msgr2=0x55ff9d1f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.857+0000 7fe4b5e00640 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d17f240 0x55ff9d1f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.858+0000 7fe4b5e00640 1 -- 172.21.15.69:0/341142167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9d1f4bc0 con 0x55ff9d175d40 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.858+0000 7fe4b6800640 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d17f240 0x55ff9d1f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.858+0000 7fe4b5e00640 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d175d40 0x55ff9d179460 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8013e80 tx=0x7fe4a8015d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f383a40d41042053 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.858+0000 7fe497400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.858+0000 7fe497400640 1 -- 172.21.15.69:0/341142167 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8024040 con 0x55ff9d175d40 2024-03-19T01:21:23.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.858+0000 7fe497400640 1 -- 172.21.15.69:0/341142167 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fe4a8013660 con 0x55ff9d175d40 2024-03-19T01:21:23.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.861+0000 7fe4bf612740 1 -- 172.21.15.69:0/341142167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b00bdf10 con 0x55ff9d175d40 2024-03-19T01:21:23.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.862+0000 7fe497400640 1 -- 172.21.15.69:0/341142167 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe4a8013860 con 0x55ff9d175d40 2024-03-19T01:21:23.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.863+0000 7fe4bf612740 10 client.27716 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:23.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.863+0000 7fe4bf612740 1 -- 172.21.15.69:0/341142167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ff9d217e90 con 0x55ff9d175d40 2024-03-19T01:21:23.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.863+0000 7fe497400640 1 -- 172.21.15.69:0/341142167 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b00bdf10 con 0x55ff9d175d40 2024-03-19T01:21:23.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.864+0000 7fe497400640 1 -- 172.21.15.69:0/341142167 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe4a8003690 con 0x55ff9d175d40 2024-03-19T01:21:23.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.865+0000 7fe4bf612740 10 client.27716 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:23.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.865+0000 7fe4bf612740 10 client.27716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:21:23.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.865+0000 7fe4bf612740 20 client.27716 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:23.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.865+0000 7fe4bf612740 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff9d2185e0 0x55ff9d21aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:23.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.865+0000 7fe4bf612740 4 client.27716 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:21:23.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.865+0000 7fe4bf612740 1 -- 172.21.15.69:0/341142167 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff9d0e7800 con 0x55ff9d2185e0 2024-03-19T01:21:23.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.865+0000 7fe4b5400640 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff9d2185e0 0x55ff9d21aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.866+0000 7fe4b5400640 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff9d2185e0 0x55ff9d21aa00 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b0eef04f6a103da server_cookie=5155480e2687582 in_seq=0 out_seq=0 2024-03-19T01:21:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.866+0000 7fe497400640 10 client.27716 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.866+0000 7fe497400640 1 -- 172.21.15.69:0/341142167 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ff9d0e7800 con 0x55ff9d2185e0 2024-03-19T01:21:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.866+0000 7fe497400640 10 client.27716 handle_command_reply: tid=0 2024-03-19T01:21:23.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.866+0000 7fe4bf612740 1 client.27716 shutdown 2024-03-19T01:21:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.867+0000 7fe4bf612740 1 -- 172.21.15.69:0/341142167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d175d40 msgr2=0x55ff9d179460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.867+0000 7fe4bf612740 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d175d40 0x55ff9d179460 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8013e80 tx=0x7fe4a8015d50 comp rx=0 tx=0).stop 2024-03-19T01:21:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.868+0000 7fe4bf612740 1 -- 172.21.15.69:0/341142167 shutdown_connections 2024-03-19T01:21:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.868+0000 7fe4bf612740 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ff9d2185e0 0x55ff9d21aa00 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.868+0000 7fe4bf612740 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ff9d17f240 0x55ff9d1f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.868+0000 7fe497400640 0 client.27716 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:23.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.868+0000 7fe4bf612740 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ff9d176660 0x55ff9d1799a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.868+0000 7fe4bf612740 1 --2- 172.21.15.69:0/341142167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9d175d40 0x55ff9d179460 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.868+0000 7fe4bf612740 1 -- 172.21.15.69:0/341142167 >> 172.21.15.69:0/341142167 conn(0x55ff9d170470 msgr2=0x55ff9d16ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:23.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.868+0000 7fe4bf612740 1 -- 172.21.15.69:0/341142167 shutdown_connections 2024-03-19T01:21:23.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.868+0000 7fe4bf612740 1 -- 172.21.15.69:0/341142167 wait complete. 2024-03-19T01:21:23.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.871+0000 7fe4bf612740 20 client.27716 trim_cache size 0 max 16384 2024-03-19T01:21:23.874 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:21:23.874 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:21:23.874 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:21:23.874 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:21:23.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.876+0000 7fe4bca00640 1 -- 172.21.15.69:0/1132950763 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe49c0a2a30 msgr2=0x7fe49c0a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:23.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.876+0000 7fe4bca00640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe49c0a2a30 0x7fe49c0a4ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8006fd0 tx=0x7fe4a80090e0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.876+0000 7fe4bca00640 1 -- 172.21.15.69:0/1132950763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4b8116820 msgr2=0x7fe4b810cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.876+0000 7fe4bca00640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4b8116820 0x7fe4b810cf20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0037040 tx=0x7fe4b0035e50 comp rx=0 tx=0).stop 2024-03-19T01:21:23.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.879+0000 7fe4bca00640 1 -- 172.21.15.69:0/1132950763 shutdown_connections 2024-03-19T01:21:23.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.879+0000 7fe4bca00640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4b8116820 0x7fe4b810cf20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.879+0000 7fe4bca00640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe49c0a2a30 0x7fe49c0a4ef0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.879+0000 7fe4bca00640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b810bf40 0x7fe4b810c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.879+0000 7fe4bca00640 1 --2- 172.21.15.69:0/1132950763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4b8108d60 0x7fe4b810c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:23.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.879+0000 7fe4bca00640 1 -- 172.21.15.69:0/1132950763 >> 172.21.15.69:0/1132950763 conn(0x7fe4b806cd60 msgr2=0x7fe4b8116c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:23.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.879+0000 7fe4bca00640 1 -- 172.21.15.69:0/1132950763 shutdown_connections 2024-03-19T01:21:23.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:23.879+0000 7fe4bca00640 1 -- 172.21.15.69:0/1132950763 wait complete. 2024-03-19T01:21:23.916 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/ksyms_common.o 2024-03-19T01:21:23.938 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:21:23.939 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:21:24.165 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsinit.o 2024-03-19T01:21:24.237 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/amd/io_pgtable_v2.o 2024-03-19T01:21:24.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:24 smithi069 ceph-mon[31727]: pgmap v334: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.2 MiB/s wr, 75 op/s 2024-03-19T01:21:24.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.475+0000 7f2f65800640 1 -- 172.21.15.69:0/658784170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f60114970 msgr2=0x7f2f60116d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:24.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.475+0000 7f2f65800640 1 --2- 172.21.15.69:0/658784170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f60114970 0x7f2f60116d60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009900 tx=0x7f2f4c02bd10 comp rx=0 tx=0).stop 2024-03-19T01:21:24.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.479+0000 7f2f65800640 1 -- 172.21.15.69:0/658784170 shutdown_connections 2024-03-19T01:21:24.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.479+0000 7f2f65800640 1 --2- 172.21.15.69:0/658784170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f60114970 0x7f2f60116d60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:24.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.479+0000 7f2f65800640 1 --2- 172.21.15.69:0/658784170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f60109330 0x7f2f60114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:24.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.479+0000 7f2f65800640 1 --2- 172.21.15.69:0/658784170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f60108a10 0x7f2f60108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:24.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.479+0000 7f2f65800640 1 -- 172.21.15.69:0/658784170 >> 172.21.15.69:0/658784170 conn(0x7f2f6006c910 msgr2=0x7f2f6006cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:24.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.482+0000 7f2f65800640 1 -- 172.21.15.69:0/658784170 shutdown_connections 2024-03-19T01:21:24.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.482+0000 7f2f65800640 1 -- 172.21.15.69:0/658784170 wait complete. 2024-03-19T01:21:24.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.483+0000 7f2f65800640 1 Processor -- start 2024-03-19T01:21:24.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.487+0000 7f2f65800640 1 -- start start 2024-03-19T01:21:24.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.487+0000 7f2f65800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f60108a10 0x7f2f601a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:24.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.488+0000 7f2f65800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f60109330 0x7f2f601a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:24.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.488+0000 7f2f65800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f60114970 0x7f2f601a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:24.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.488+0000 7f2f5ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f60108a10 0x7f2f601a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:24.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.488+0000 7f2f65800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6011a4f0 con 0x7f2f60108a10 2024-03-19T01:21:24.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.488+0000 7f2f5ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f60108a10 0x7f2f601a88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41966/0 (socket says 172.21.15.69:41966) 2024-03-19T01:21:24.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.488+0000 7f2f5ea00640 1 -- 172.21.15.69:0/3641611957 learned_addr learned my addr 172.21.15.69:0/3641611957 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:24.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.488+0000 7f2f65800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6011a370 con 0x7f2f60109330 2024-03-19T01:21:24.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.488+0000 7f2f65800640 1 -- 172.21.15.69:0/3641611957 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6011a670 con 0x7f2f60114970 2024-03-19T01:21:24.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.488+0000 7f2f5e000640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f60109330 0x7f2f601a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:24.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.488+0000 7f2f5f400640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f60114970 0x7f2f601a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:24.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.489+0000 7f2f5ea00640 1 -- 172.21.15.69:0/3641611957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f60114970 msgr2=0x7f2f601a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:24.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.489+0000 7f2f5ea00640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f60114970 0x7f2f601a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:24.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.489+0000 7f2f5ea00640 1 -- 172.21.15.69:0/3641611957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f60109330 msgr2=0x7f2f601a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:24.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.489+0000 7f2f5ea00640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f60109330 0x7f2f601a8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:24.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.489+0000 7f2f5ea00640 1 -- 172.21.15.69:0/3641611957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f601a31d0 con 0x7f2f60108a10 2024-03-19T01:21:24.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.489+0000 7f2f5e000640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f60109330 0x7f2f601a8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:21:24.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.489+0000 7f2f5f400640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f60114970 0x7f2f601a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:21:24.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.489+0000 7f2f5ea00640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f60108a10 0x7f2f601a88a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800ba50 tx=0x7f2f5800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:24.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.490+0000 7f2f57400640 1 -- 172.21.15.69:0/3641611957 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58019070 con 0x7f2f60108a10 2024-03-19T01:21:24.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.490+0000 7f2f57400640 1 -- 172.21.15.69:0/3641611957 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f5800cb80 con 0x7f2f60108a10 2024-03-19T01:21:24.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.490+0000 7f2f65800640 1 -- 172.21.15.69:0/3641611957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f601a34c0 con 0x7f2f60108a10 2024-03-19T01:21:24.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.490+0000 7f2f57400640 1 -- 172.21.15.69:0/3641611957 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58007630 con 0x7f2f60108a10 2024-03-19T01:21:24.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.490+0000 7f2f65800640 1 -- 172.21.15.69:0/3641611957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f601af760 con 0x7f2f60108a10 2024-03-19T01:21:24.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.493+0000 7f2f65800640 1 -- 172.21.15.69:0/3641611957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f60068610 con 0x7f2f60108a10 2024-03-19T01:21:24.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.496+0000 7f2f57400640 1 -- 172.21.15.69:0/3641611957 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f2f5801c070 con 0x7f2f60108a10 2024-03-19T01:21:24.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.499+0000 7f2f57400640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2f440a2a30 0x7f2f440a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:24.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.499+0000 7f2f57400640 1 -- 172.21.15.69:0/3641611957 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7f2f580ce760 con 0x7f2f60108a10 2024-03-19T01:21:24.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.500+0000 7f2f5e000640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2f440a2a30 0x7f2f440a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:24.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:24 smithi086 ceph-mon[66829]: pgmap v334: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.2 MiB/s wr, 75 op/s 2024-03-19T01:21:24.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.506+0000 7f2f5e000640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2f440a2a30 0x7f2f440a4ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2f6010bc90 tx=0x7f2f500090e0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:24.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:24.520+0000 7f2f57400640 1 -- 172.21.15.69:0/3641611957 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2f5809ab70 con 0x7f2f60108a10 2024-03-19T01:21:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:24 smithi163 ceph-mon[67428]: pgmap v334: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.2 MiB/s wr, 75 op/s 2024-03-19T01:21:24.632 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/rhashtable.o 2024-03-19T01:21:24.646 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/tty_buffer.o 2024-03-19T01:21:24.689 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/misc.o 2024-03-19T01:21:24.867 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/groups.o 2024-03-19T01:21:25.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.048+0000 7f2f65800640 1 -- 172.21.15.69:0/3641611957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f601afa50 con 0x7f2f60108a10 2024-03-19T01:21:25.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.053+0000 7f2f57400640 1 -- 172.21.15.69:0/3641611957 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11930 (secure 0 0 0) 0x7f2f5809ec10 con 0x7f2f60108a10 2024-03-19T01:21:25.057 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:21:25.058 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":59,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:20:08.600901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:21:25.058 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 59 2024-03-19T01:21:25.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.066+0000 7f2f65800640 1 -- 172.21.15.69:0/3641611957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2f440a2a30 msgr2=0x7f2f440a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:25.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.066+0000 7f2f65800640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2f440a2a30 0x7f2f440a4ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2f6010bc90 tx=0x7f2f500090e0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.066+0000 7f2f65800640 1 -- 172.21.15.69:0/3641611957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f60108a10 msgr2=0x7f2f601a88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:25.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.066+0000 7f2f65800640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f60108a10 0x7f2f601a88a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800ba50 tx=0x7f2f5800bf10 comp rx=0 tx=0).stop 2024-03-19T01:21:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.067+0000 7f2f65800640 1 -- 172.21.15.69:0/3641611957 shutdown_connections 2024-03-19T01:21:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.067+0000 7f2f65800640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f60114970 0x7f2f601a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.067+0000 7f2f65800640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2f440a2a30 0x7f2f440a4ef0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.067+0000 7f2f65800640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f60109330 0x7f2f601a8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.067+0000 7f2f65800640 1 --2- 172.21.15.69:0/3641611957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f60108a10 0x7f2f601a88a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.067+0000 7f2f65800640 1 -- 172.21.15.69:0/3641611957 >> 172.21.15.69:0/3641611957 conn(0x7f2f6006c910 msgr2=0x7f2f6010a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.067+0000 7f2f65800640 1 -- 172.21.15.69:0/3641611957 shutdown_connections 2024-03-19T01:21:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.067+0000 7f2f65800640 1 -- 172.21.15.69:0/3641611957 wait complete. 2024-03-19T01:21:25.124 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:21:25.320 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/iommu/amd/built-in.a 2024-03-19T01:21:25.483 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/intel/dmar.o 2024-03-19T01:21:25.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:25 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3641611957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:21:25.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:25 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3641611957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:21:25.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:25 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3641611957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:21:25.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.581+0000 7efe49200640 1 -- 172.21.15.69:0/379224065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe4411b000 msgr2=0x7efe4411d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:25.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.581+0000 7efe49200640 1 --2- 172.21.15.69:0/379224065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe4411b000 0x7efe4411d3f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efe34009900 tx=0x7efe3402bc50 comp rx=0 tx=0).stop 2024-03-19T01:21:25.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.584+0000 7efe49200640 1 -- 172.21.15.69:0/379224065 shutdown_connections 2024-03-19T01:21:25.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.584+0000 7efe49200640 1 --2- 172.21.15.69:0/379224065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe4411b000 0x7efe4411d3f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.584+0000 7efe49200640 1 --2- 172.21.15.69:0/379224065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe441088b0 0x7efe44108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.584+0000 7efe49200640 1 --2- 172.21.15.69:0/379224065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4410fe40 0x7efe44110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.584+0000 7efe49200640 1 -- 172.21.15.69:0/379224065 >> 172.21.15.69:0/379224065 conn(0x7efe4406c770 msgr2=0x7efe4406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.584+0000 7efe49200640 1 -- 172.21.15.69:0/379224065 shutdown_connections 2024-03-19T01:21:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.584+0000 7efe49200640 1 -- 172.21.15.69:0/379224065 wait complete. 2024-03-19T01:21:25.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.585+0000 7efe49200640 1 Processor -- start 2024-03-19T01:21:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.586+0000 7efe49200640 1 -- start start 2024-03-19T01:21:25.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.590+0000 7efe49200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe441088b0 0x7efe441ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.590+0000 7efe49200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4410fe40 0x7efe441ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.590+0000 7efe49200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe4411b000 0x7efe441a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.590+0000 7efe49200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe44120870 con 0x7efe4411b000 2024-03-19T01:21:25.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.590+0000 7efe49200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe441206f0 con 0x7efe4410fe40 2024-03-19T01:21:25.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.590+0000 7efe49200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efe441209f0 con 0x7efe441088b0 2024-03-19T01:21:25.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.590+0000 7efe43400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe441088b0 0x7efe441ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.591+0000 7efe43400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe441088b0 0x7efe441ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59496/0 (socket says 172.21.15.69:59496) 2024-03-19T01:21:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.591+0000 7efe43400640 1 -- 172.21.15.69:0/659759773 learned_addr learned my addr 172.21.15.69:0/659759773 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.591+0000 7efe43e00640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe4411b000 0x7efe441a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.591+0000 7efe42a00640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4410fe40 0x7efe441ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.591+0000 7efe43400640 1 -- 172.21.15.69:0/659759773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4410fe40 msgr2=0x7efe441ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.591+0000 7efe43400640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4410fe40 0x7efe441ace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.591+0000 7efe43400640 1 -- 172.21.15.69:0/659759773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe4411b000 msgr2=0x7efe441a6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.591+0000 7efe43400640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe4411b000 0x7efe441a6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.591+0000 7efe43400640 1 -- 172.21.15.69:0/659759773 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe441a71f0 con 0x7efe441088b0 2024-03-19T01:21:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.591+0000 7efe43e00640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe4411b000 0x7efe441a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:21:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.591+0000 7efe42a00640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4410fe40 0x7efe441ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.592+0000 7efe43400640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe441088b0 0x7efe441ac8c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00eab0 tx=0x7efe3c00ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.592+0000 7efe3be00640 1 -- 172.21.15.69:0/659759773 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00ce40 con 0x7efe441088b0 2024-03-19T01:21:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.592+0000 7efe3be00640 1 -- 172.21.15.69:0/659759773 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3c011950 con 0x7efe441088b0 2024-03-19T01:21:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.592+0000 7efe49200640 1 -- 172.21.15.69:0/659759773 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe441a74e0 con 0x7efe441088b0 2024-03-19T01:21:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.592+0000 7efe3be00640 1 -- 172.21.15.69:0/659759773 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c010ae0 con 0x7efe441088b0 2024-03-19T01:21:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.592+0000 7efe49200640 1 -- 172.21.15.69:0/659759773 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe441b3780 con 0x7efe441088b0 2024-03-19T01:21:25.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.601+0000 7efe4d3f2740 1 Processor -- start 2024-03-19T01:21:25.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.604+0000 7efe3be00640 1 -- 172.21.15.69:0/659759773 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7efe3c011b00 con 0x7efe441088b0 2024-03-19T01:21:25.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.605+0000 7efe4d3f2740 1 -- start start 2024-03-19T01:21:25.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.605+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434d5900 0x5569434d5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.605+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d6220 0x5569434efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.605+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 0x5569434f2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.605+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55694343a380 con 0x5569434d6220 2024-03-19T01:21:25.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.605+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5569433e2d20 con 0x5569434d5900 2024-03-19T01:21:25.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.605+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556943436420 con 0x5569434f0120 2024-03-19T01:21:25.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.606+0000 7efe42a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 0x5569434f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.606+0000 7efe42a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 0x5569434f2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59504/0 (socket says 172.21.15.69:59504) 2024-03-19T01:21:25.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.606+0000 7efe42a00640 1 -- 172.21.15.69:0/244954008 learned_addr learned my addr 172.21.15.69:0/244954008 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:25.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.606+0000 7efe3be00640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efe280a2a70 0x7efe280a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.606+0000 7efe43e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434d5900 0x5569434d5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.607+0000 7efe43400640 1 --2- 172.21.15.69:0/244954008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d6220 0x5569434efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.607+0000 7efe3be00640 1 -- 172.21.15.69:0/659759773 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7efe3c0cf910 con 0x7efe441088b0 2024-03-19T01:21:25.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.607+0000 7efe43e00640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efe280a2a70 0x7efe280a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.607+0000 7efe43400640 1 -- 172.21.15.69:0/244954008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 msgr2=0x5569434f2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:25.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.608+0000 7efe43400640 1 --2- 172.21.15.69:0/244954008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 0x5569434f2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.608+0000 7efe43400640 1 -- 172.21.15.69:0/244954008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434d5900 msgr2=0x5569434d5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:25.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.608+0000 7efe43400640 1 --2- 172.21.15.69:0/244954008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434d5900 0x5569434d5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.608+0000 7efe43400640 1 -- 172.21.15.69:0/244954008 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55694314e3f0 con 0x5569434d6220 2024-03-19T01:21:25.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.609+0000 7efe43e00640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efe280a2a70 0x7efe280a4f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efe34006010 tx=0x7efe340095e0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:25.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.609+0000 7efe43400640 1 --2- 172.21.15.69:0/244954008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d6220 0x5569434efbe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0ccc10 tx=0x7efe3c010540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a90e5b1f36e3102 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:25.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.609+0000 7efe38c00640 1 -- 172.21.15.69:0/244954008 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c09de60 con 0x5569434d6220 2024-03-19T01:21:25.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.609+0000 7efe38c00640 1 -- 172.21.15.69:0/244954008 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3c09fc70 con 0x5569434d6220 2024-03-19T01:21:25.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.609+0000 7efe38c00640 1 -- 172.21.15.69:0/244954008 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0202b0 con 0x5569434d6220 2024-03-19T01:21:25.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.610+0000 7efe4d3f2740 1 -- 172.21.15.69:0/244954008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d6220 msgr2=0x5569434efbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:25.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.610+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/244954008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d6220 0x5569434efbe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0ccc10 tx=0x7efe3c010540 comp rx=0 tx=0).stop 2024-03-19T01:21:25.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.611+0000 7efe4d3f2740 1 -- 172.21.15.69:0/244954008 shutdown_connections 2024-03-19T01:21:25.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.611+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/244954008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 0x5569434f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.611+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/244954008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d6220 0x5569434efbe0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.611+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/244954008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434d5900 0x5569434d5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.611+0000 7efe4d3f2740 1 -- 172.21.15.69:0/244954008 >> 172.21.15.69:0/244954008 conn(0x5569434d0340 msgr2=0x5569434d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:25.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.611+0000 7efe4d3f2740 1 -- 172.21.15.69:0/244954008 shutdown_connections 2024-03-19T01:21:25.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.611+0000 7efe4d3f2740 1 -- 172.21.15.69:0/244954008 wait complete. 2024-03-19T01:21:25.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.612+0000 7efe4d3f2740 1 -- start start 2024-03-19T01:21:25.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.612+0000 7efe4d3f2740 1 Processor -- start 2024-03-19T01:21:25.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.615+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434e1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.615+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x5569434e19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.615+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434dc7d0 0x5569434dcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.615+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5569434d6cf0 con 0x5569434d5900 2024-03-19T01:21:25.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.615+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5569433f6160 con 0x5569434f0120 2024-03-19T01:21:25.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.615+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5569434d6e70 con 0x5569434dc7d0 2024-03-19T01:21:25.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.615+0000 7efe4d3f2740 10 client.0 _ll_register_callbacks cb 0x556943432170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:25.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.616+0000 7efe42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x5569434e19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.616+0000 7efe42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x5569434e19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52876/0 (socket says 172.21.15.69:52876) 2024-03-19T01:21:25.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.616+0000 7efe42a00640 1 -- 172.21.15.69:0/2968250671 learned_addr learned my addr 172.21.15.69:0/2968250671 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:25.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.616+0000 7efe43400640 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434e1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.616+0000 7efe42a00640 1 -- 172.21.15.69:0/2968250671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434dc7d0 msgr2=0x5569434dcc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:21:25.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.616+0000 7efe42a00640 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434dc7d0 0x5569434dcc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.616+0000 7efe42a00640 1 -- 172.21.15.69:0/2968250671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 msgr2=0x5569434e1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:25.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.616+0000 7efe42a00640 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434e1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.617+0000 7efe42a00640 1 -- 172.21.15.69:0/2968250671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569432187d0 con 0x5569434f0120 2024-03-19T01:21:25.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.617+0000 7efe43400640 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434e1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:25.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.617+0000 7efe42a00640 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x5569434e19b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efe300327a0 tx=0x7efe30032c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a6497ec7ad342c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:25.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.618+0000 7efe2d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:21:25.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.618+0000 7efe2d600640 1 -- 172.21.15.69:0/2968250671 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3002dd40 con 0x5569434f0120 2024-03-19T01:21:25.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.618+0000 7efe2d600640 1 -- 172.21.15.69:0/2968250671 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7efe30005020 con 0x5569434f0120 2024-03-19T01:21:25.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.618+0000 7efe4d3f2740 1 -- 172.21.15.69:0/2968250671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe3c09fc70 con 0x5569434f0120 2024-03-19T01:21:25.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.618+0000 7efe2d600640 1 -- 172.21.15.69:0/2968250671 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7efe30046050 con 0x5569434f0120 2024-03-19T01:21:25.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.618+0000 7efe4d3f2740 10 client.27728 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:25.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.618+0000 7efe4d3f2740 1 -- 172.21.15.69:0/2968250671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5569434ebcc0 con 0x5569434f0120 2024-03-19T01:21:25.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.621+0000 7efe2d600640 1 -- 172.21.15.69:0/2968250671 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3c09fc70 con 0x5569434f0120 2024-03-19T01:21:25.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.621+0000 7efe2d600640 1 -- 172.21.15.69:0/2968250671 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7efe30033450 con 0x5569434f0120 2024-03-19T01:21:25.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.621+0000 7efe4d3f2740 10 client.27728 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:25.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.621+0000 7efe4d3f2740 10 client.27728 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:21:25.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.621+0000 7efe4d3f2740 20 client.27728 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:25.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.621+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5569434ed0d0 0x5569434ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.621+0000 7efe4d3f2740 4 client.27728 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:21:25.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.621+0000 7efe4d3f2740 1 -- 172.21.15.69:0/2968250671 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569434478c0 con 0x5569434ed0d0 2024-03-19T01:21:25.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.622+0000 7efe43400640 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5569434ed0d0 0x5569434ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.623+0000 7efe2d600640 10 client.27728 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:21:25.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.623+0000 7efe43400640 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5569434ed0d0 0x5569434ef550 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23b51356fb3dc413 server_cookie=3cf4c04bdd273bcb in_seq=0 out_seq=0 2024-03-19T01:21:25.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.624+0000 7efe2d600640 1 -- 172.21.15.69:0/2968250671 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5569434478c0 con 0x5569434ed0d0 2024-03-19T01:21:25.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.624+0000 7efe2d600640 10 client.27728 handle_command_reply: tid=0 2024-03-19T01:21:25.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.624+0000 7efe4d3f2740 1 client.27728 shutdown 2024-03-19T01:21:25.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.625+0000 7efe4d3f2740 1 -- 172.21.15.69:0/2968250671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 msgr2=0x5569434e19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:25.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.625+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x5569434e19b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efe300327a0 tx=0x7efe30032c60 comp rx=0 tx=0).stop 2024-03-19T01:21:25.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.628+0000 7efe4d3f2740 1 -- 172.21.15.69:0/2968250671 shutdown_connections 2024-03-19T01:21:25.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.628+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5569434ed0d0 0x5569434ef550 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.628+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434dc7d0 0x5569434dcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.628+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x5569434e19b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.628+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/2968250671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434e1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.628+0000 7efe2d600640 0 client.27728 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:21:25.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.628+0000 7efe4d3f2740 1 -- 172.21.15.69:0/2968250671 >> 172.21.15.69:0/2968250671 conn(0x5569434d0340 msgr2=0x5569434d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:25.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.628+0000 7efe4d3f2740 1 -- 172.21.15.69:0/2968250671 shutdown_connections 2024-03-19T01:21:25.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.628+0000 7efe4d3f2740 1 -- 172.21.15.69:0/2968250671 wait complete. 2024-03-19T01:21:25.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.628+0000 7efe4d3f2740 20 client.27728 trim_cache size 0 max 16384 2024-03-19T01:21:25.632 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/virtio_console.o 2024-03-19T01:21:25.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.692+0000 7efe4d3f2740 1 Processor -- start 2024-03-19T01:21:25.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.692+0000 7efe4d3f2740 1 -- start start 2024-03-19T01:21:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434dc7d0 0x5569434f8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 0x5569434e1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe441b5280 con 0x5569434d5900 2024-03-19T01:21:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe441b6dc0 con 0x5569434dc7d0 2024-03-19T01:21:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efe441b5950 con 0x5569434f0120 2024-03-19T01:21:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434dc7d0 0x5569434f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe43e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 0x5569434e1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434dc7d0 0x5569434f8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52878/0 (socket says 172.21.15.69:52878) 2024-03-19T01:21:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe42a00640 1 -- 172.21.15.69:0/129977204 learned_addr learned my addr 172.21.15.69:0/129977204 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe43e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 0x5569434e1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:59516/0 (socket says 172.21.15.69:59516) 2024-03-19T01:21:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe43400640 1 --2- 172.21.15.69:0/129977204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe42a00640 1 -- 172.21.15.69:0/129977204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 msgr2=0x5569434e1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe42a00640 1 --2- 172.21.15.69:0/129977204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 0x5569434e1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe42a00640 1 -- 172.21.15.69:0/129977204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 msgr2=0x5569434f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe42a00640 1 --2- 172.21.15.69:0/129977204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434f7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.693+0000 7efe42a00640 1 -- 172.21.15.69:0/129977204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569434d6050 con 0x5569434dc7d0 2024-03-19T01:21:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.694+0000 7efe42a00640 1 --2- 172.21.15.69:0/129977204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434dc7d0 0x5569434f8150 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efe30001eb0 tx=0x7efe3002d7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dd8c86d4f3abae4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.694+0000 7efe2fe00640 1 -- 172.21.15.69:0/129977204 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30030590 con 0x5569434dc7d0 2024-03-19T01:21:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.694+0000 7efe2fe00640 1 -- 172.21.15.69:0/129977204 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe30033a90 con 0x5569434dc7d0 2024-03-19T01:21:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.694+0000 7efe2fe00640 1 -- 172.21.15.69:0/129977204 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3003b7c0 con 0x5569434dc7d0 2024-03-19T01:21:25.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.695+0000 7efe4d3f2740 1 -- 172.21.15.69:0/129977204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434dc7d0 msgr2=0x5569434f8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:25.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.695+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/129977204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434dc7d0 0x5569434f8150 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efe30001eb0 tx=0x7efe3002d7d0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.695+0000 7efe4d3f2740 1 -- 172.21.15.69:0/129977204 shutdown_connections 2024-03-19T01:21:25.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.695+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/129977204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434f0120 0x5569434e1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.695+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/129977204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434dc7d0 0x5569434f8150 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.695+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/129977204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.695+0000 7efe4d3f2740 1 -- 172.21.15.69:0/129977204 >> 172.21.15.69:0/129977204 conn(0x5569434d0340 msgr2=0x5569434d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.697+0000 7efe4d3f2740 1 -- 172.21.15.69:0/129977204 shutdown_connections 2024-03-19T01:21:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.697+0000 7efe4d3f2740 1 -- 172.21.15.69:0/129977204 wait complete. 2024-03-19T01:21:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.698+0000 7efe4d3f2740 1 -- start start 2024-03-19T01:21:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.698+0000 7efe4d3f2740 1 Processor -- start 2024-03-19T01:21:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.699+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.699+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434dc7d0 0x5569434f6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.699+0000 7efe4d3f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x556943555340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.699+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe441b66f0 con 0x5569434d5900 2024-03-19T01:21:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.699+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe441b6020 con 0x5569434f0120 2024-03-19T01:21:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.699+0000 7efe43e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x556943555340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.699+0000 7efe43e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x556943555340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:52892/0 (socket says 172.21.15.69:52892) 2024-03-19T01:21:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.699+0000 7efe43e00640 1 -- 172.21.15.69:0/3329407843 learned_addr learned my addr 172.21.15.69:0/3329407843 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.699+0000 7efe43400640 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.699+0000 7efe4d3f2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efe44068530 con 0x5569434dc7d0 2024-03-19T01:21:25.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.700+0000 7efe42a00640 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434dc7d0 0x5569434f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.700+0000 7efe4d3f2740 10 client.0 _ll_register_callbacks cb 0x7efe300303c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:25.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.700+0000 7efe43e00640 1 -- 172.21.15.69:0/3329407843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434dc7d0 msgr2=0x5569434f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:25.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.700+0000 7efe43e00640 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434dc7d0 0x5569434f6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.700+0000 7efe43e00640 1 -- 172.21.15.69:0/3329407843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 msgr2=0x5569434f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:25.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.700+0000 7efe43e00640 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434f5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.700+0000 7efe43e00640 1 -- 172.21.15.69:0/3329407843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556943555880 con 0x5569434f0120 2024-03-19T01:21:25.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.701+0000 7efe2d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:21:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.701+0000 7efe43e00640 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x556943555340 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efe3402c260 tx=0x7efe34009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e48ce58c2a34bb0a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.702+0000 7efe2d600640 1 -- 172.21.15.69:0/3329407843 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34004ad0 con 0x5569434f0120 2024-03-19T01:21:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.702+0000 7efe2d600640 1 -- 172.21.15.69:0/3329407843 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7efe34004c70 con 0x5569434f0120 2024-03-19T01:21:25.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.702+0000 7efe4d3f2740 1 -- 172.21.15.69:0/3329407843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe30033a90 con 0x5569434f0120 2024-03-19T01:21:25.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.702+0000 7efe2d600640 1 -- 172.21.15.69:0/3329407843 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7efe3405e050 con 0x5569434f0120 2024-03-19T01:21:25.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.702+0000 7efe4d3f2740 10 client.27734 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:25.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.702+0000 7efe4d3f2740 1 -- 172.21.15.69:0/3329407843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x556943556b30 con 0x5569434f0120 2024-03-19T01:21:25.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.702+0000 7efe2d600640 1 -- 172.21.15.69:0/3329407843 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe30033a90 con 0x5569434f0120 2024-03-19T01:21:25.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.703+0000 7efe2d600640 1 -- 172.21.15.69:0/3329407843 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7efe34034560 con 0x5569434f0120 2024-03-19T01:21:25.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.703+0000 7efe4d3f2740 10 client.27734 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:25.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.703+0000 7efe4d3f2740 10 client.27734 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:21:25.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.703+0000 7efe4d3f2740 20 client.27734 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:25.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.703+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5569435572b0 0x5569435596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:25.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.703+0000 7efe4d3f2740 4 client.27734 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:21:25.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.703+0000 7efe4d3f2740 1 -- 172.21.15.69:0/3329407843 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556943441c10 con 0x5569435572b0 2024-03-19T01:21:25.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.707+0000 7efe43400640 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5569435572b0 0x5569435596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:25.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.707+0000 7efe43400640 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5569435572b0 0x5569435596d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5c5f91309aed7084 server_cookie=e1c4c5b3740f7875 in_seq=0 out_seq=0 2024-03-19T01:21:25.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.708+0000 7efe2d600640 10 client.27734 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:21:25.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.710+0000 7efe2d600640 1 -- 172.21.15.69:0/3329407843 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556943441c10 con 0x5569435572b0 2024-03-19T01:21:25.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.710+0000 7efe2d600640 10 client.27734 handle_command_reply: tid=0 2024-03-19T01:21:25.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.710+0000 7efe4d3f2740 1 client.27734 shutdown 2024-03-19T01:21:25.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.711+0000 7efe4d3f2740 1 -- 172.21.15.69:0/3329407843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 msgr2=0x556943555340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:25.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.711+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x556943555340 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efe3402c260 tx=0x7efe34009400 comp rx=0 tx=0).stop 2024-03-19T01:21:25.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.711+0000 7efe4d3f2740 1 -- 172.21.15.69:0/3329407843 shutdown_connections 2024-03-19T01:21:25.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.711+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5569435572b0 0x5569435596d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.711+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569434f0120 0x556943555340 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.711+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5569434dc7d0 0x5569434f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.711+0000 7efe4d3f2740 1 --2- 172.21.15.69:0/3329407843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569434d5900 0x5569434f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.711+0000 7efe2d600640 0 client.27734 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:21:25.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.711+0000 7efe4d3f2740 1 -- 172.21.15.69:0/3329407843 >> 172.21.15.69:0/3329407843 conn(0x5569434d0340 msgr2=0x5569434ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:25.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.714+0000 7efe4d3f2740 1 -- 172.21.15.69:0/3329407843 shutdown_connections 2024-03-19T01:21:25.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.714+0000 7efe4d3f2740 1 -- 172.21.15.69:0/3329407843 wait complete. 2024-03-19T01:21:25.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.714+0000 7efe4d3f2740 20 client.27734 trim_cache size 0 max 16384 2024-03-19T01:21:25.718 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 -- 172.21.15.69:0/659759773 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efe280a2a70 msgr2=0x7efe280a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efe280a2a70 0x7efe280a4f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efe34006010 tx=0x7efe340095e0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 -- 172.21.15.69:0/659759773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe441088b0 msgr2=0x7efe441ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe441088b0 0x7efe441ac8c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00eab0 tx=0x7efe3c00ef70 comp rx=0 tx=0).stop 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 -- 172.21.15.69:0/659759773 shutdown_connections 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe4411b000 0x7efe441a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7efe280a2a70 0x7efe280a4f30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe4410fe40 0x7efe441ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 --2- 172.21.15.69:0/659759773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe441088b0 0x7efe441ac8c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 -- 172.21.15.69:0/659759773 >> 172.21.15.69:0/659759773 conn(0x7efe4406c770 msgr2=0x7efe4410da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 -- 172.21.15.69:0/659759773 shutdown_connections 2024-03-19T01:21:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:25.722+0000 7efe49200640 1 -- 172.21.15.69:0/659759773 wait complete. 2024-03-19T01:21:25.778 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:21:25.874 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsload.o 2024-03-19T01:21:25.890 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/tty_port.o 2024-03-19T01:21:26.002 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/base64.o 2024-03-19T01:21:26.024 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/display/drm_dsc_helper.o 2024-03-19T01:21:26.028 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/kcmp.o 2024-03-19T01:21:26.113 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4renewd.o 2024-03-19T01:21:26.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:26 smithi069 ceph-mon[31727]: pgmap v335: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-03-19T01:21:26.311 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/once.o 2024-03-19T01:21:26.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.359+0000 7f2e49600640 1 -- 172.21.15.69:0/2295872728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44110040 msgr2=0x7f2e4410db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:26.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.359+0000 7f2e49600640 1 --2- 172.21.15.69:0/2295872728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44110040 0x7f2e4410db30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2e3001a910 tx=0x7f2e3003cca0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.360+0000 7f2e49600640 1 -- 172.21.15.69:0/2295872728 shutdown_connections 2024-03-19T01:21:26.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.360+0000 7f2e49600640 1 --2- 172.21.15.69:0/2295872728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e4411b000 0x7f2e4411d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.360+0000 7f2e49600640 1 --2- 172.21.15.69:0/2295872728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4410e070 0x7f2e4410e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.360+0000 7f2e49600640 1 --2- 172.21.15.69:0/2295872728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44110040 0x7f2e4410db30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.360+0000 7f2e49600640 1 -- 172.21.15.69:0/2295872728 >> 172.21.15.69:0/2295872728 conn(0x7f2e4406c900 msgr2=0x7f2e4406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:26.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.369+0000 7f2e49600640 1 -- 172.21.15.69:0/2295872728 shutdown_connections 2024-03-19T01:21:26.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.381+0000 7f2e49600640 1 -- 172.21.15.69:0/2295872728 wait complete. 2024-03-19T01:21:26.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.383+0000 7f2e49600640 1 Processor -- start 2024-03-19T01:21:26.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.383+0000 7f2e49600640 1 -- start start 2024-03-19T01:21:26.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.386+0000 7f2e49600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e4410e070 0x7f2e4411a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.386+0000 7f2e49600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44110040 0x7f2e44116830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.386+0000 7f2e49600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4411b000 0x7f2e44116d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.386+0000 7f2e49600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441131e0 con 0x7f2e44110040 2024-03-19T01:21:26.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.386+0000 7f2e49600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44113060 con 0x7f2e4410e070 2024-03-19T01:21:26.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.386+0000 7f2e49600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44113360 con 0x7f2e4411b000 2024-03-19T01:21:26.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.386+0000 7f2e43400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4411b000 0x7f2e44116d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:26.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.386+0000 7f2e43400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4411b000 0x7f2e44116d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60716/0 (socket says 172.21.15.69:60716) 2024-03-19T01:21:26.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.386+0000 7f2e43400640 1 -- 172.21.15.69:0/1621333087 learned_addr learned my addr 172.21.15.69:0/1621333087 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:26.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.387+0000 7f2e43400640 1 -- 172.21.15.69:0/1621333087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e4410e070 msgr2=0x7f2e4411a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:26.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.387+0000 7f2e43400640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e4410e070 0x7f2e4411a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.387+0000 7f2e43400640 1 -- 172.21.15.69:0/1621333087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44110040 msgr2=0x7f2e44116830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:26.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.387+0000 7f2e42000640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44110040 0x7f2e44116830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:26.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.387+0000 7f2e43400640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44110040 0x7f2e44116830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.387+0000 7f2e43400640 1 -- 172.21.15.69:0/1621333087 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e441174a0 con 0x7f2e4411b000 2024-03-19T01:21:26.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.387+0000 7f2e42000640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44110040 0x7f2e44116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:26.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.388+0000 7f2e43400640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4411b000 0x7f2e44116d70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009fd0 tx=0x7f2e3c00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:26.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.388+0000 7f2e3b400640 1 -- 172.21.15.69:0/1621333087 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c017070 con 0x7f2e4411b000 2024-03-19T01:21:26.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.388+0000 7f2e3b400640 1 -- 172.21.15.69:0/1621333087 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e3c003e50 con 0x7f2e4411b000 2024-03-19T01:21:26.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.388+0000 7f2e49600640 1 -- 172.21.15.69:0/1621333087 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e441aad70 con 0x7f2e4411b000 2024-03-19T01:21:26.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.389+0000 7f2e49600640 1 -- 172.21.15.69:0/1621333087 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441ab230 con 0x7f2e4411b000 2024-03-19T01:21:26.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.389+0000 7f2e3b400640 1 -- 172.21.15.69:0/1621333087 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0106c0 con 0x7f2e4411b000 2024-03-19T01:21:26.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.395+0000 7f2e4c394740 1 Processor -- start 2024-03-19T01:21:26.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.396+0000 7f2e4c394740 1 -- start start 2024-03-19T01:21:26.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.396+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 0x55bbdcb82e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.396+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb8b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.396+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb8b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb8b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58464/0 (socket says 172.21.15.69:58464) 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 0x55bbdcb8e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e4c394740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcae7380 con 0x55bbdcb82a20 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e4c394740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bbdca8fd20 con 0x55bbdcb83340 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e4c394740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcae3420 con 0x55bbdcb8bf20 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e42a00640 1 -- 172.21.15.69:0/197270128 learned_addr learned my addr 172.21.15.69:0/197270128 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e42a00640 1 -- 172.21.15.69:0/197270128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 msgr2=0x55bbdcb8e310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e42a00640 1 --2- 172.21.15.69:0/197270128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 0x55bbdcb8e310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e42a00640 1 -- 172.21.15.69:0/197270128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 msgr2=0x55bbdcb82e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e42a00640 1 --2- 172.21.15.69:0/197270128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 0x55bbdcb82e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e42a00640 1 -- 172.21.15.69:0/197270128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbdc7fb3f0 con 0x55bbdcb83340 2024-03-19T01:21:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.397+0000 7f2e42a00640 1 --2- 172.21.15.69:0/197270128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb8b9e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2e30047040 tx=0x7f2e30004800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc6bffcd5b217c14 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:26.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.399+0000 7f2e3b400640 1 -- 172.21.15.69:0/1621333087 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f2e3c010860 con 0x7f2e4411b000 2024-03-19T01:21:26.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.401+0000 7f2e3b400640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e280a2ab0 0x7f2e280a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.401+0000 7f2e3b400640 1 -- 172.21.15.69:0/1621333087 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7f2e3c0cf0b0 con 0x7f2e4411b000 2024-03-19T01:21:26.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.402+0000 7f2e42000640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e280a2ab0 0x7f2e280a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:26.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.402+0000 7f2e2f400640 1 -- 172.21.15.69:0/197270128 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30052020 con 0x55bbdcb83340 2024-03-19T01:21:26.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.403+0000 7f2e2f400640 1 -- 172.21.15.69:0/197270128 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e3001a510 con 0x55bbdcb83340 2024-03-19T01:21:26.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.403+0000 7f2e2f400640 1 -- 172.21.15.69:0/197270128 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3003dce0 con 0x55bbdcb83340 2024-03-19T01:21:26.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.403+0000 7f2e42000640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e280a2ab0 0x7f2e280a4f70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55bbdcb7e2e0 tx=0x7f2e34008040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.406+0000 7f2e4c394740 1 -- 172.21.15.69:0/197270128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 msgr2=0x55bbdcb8b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.406+0000 7f2e4c394740 1 --2- 172.21.15.69:0/197270128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb8b9e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2e30047040 tx=0x7f2e30004800 comp rx=0 tx=0).stop 2024-03-19T01:21:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.407+0000 7f2e4c394740 1 -- 172.21.15.69:0/197270128 shutdown_connections 2024-03-19T01:21:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.407+0000 7f2e4c394740 1 --2- 172.21.15.69:0/197270128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 0x55bbdcb8e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.407+0000 7f2e4c394740 1 --2- 172.21.15.69:0/197270128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb8b9e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.407+0000 7f2e4c394740 1 --2- 172.21.15.69:0/197270128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 0x55bbdcb82e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.407+0000 7f2e4c394740 1 -- 172.21.15.69:0/197270128 >> 172.21.15.69:0/197270128 conn(0x55bbdcb7d150 msgr2=0x55bbdcb7d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:26.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.407+0000 7f2e4c394740 1 -- 172.21.15.69:0/197270128 shutdown_connections 2024-03-19T01:21:26.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.407+0000 7f2e4c394740 1 -- 172.21.15.69:0/197270128 wait complete. 2024-03-19T01:21:26.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.408+0000 7f2e4c394740 1 -- start start 2024-03-19T01:21:26.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.408+0000 7f2e4c394740 1 Processor -- start 2024-03-19T01:21:26.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.412+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb82a20 0x55bbdcb88f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.412+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb83340 0x55bbdcb89460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.412+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 0x55bbdcb899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb82a20 0x55bbdcb88f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb82a20 0x55bbdcb88f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58480/0 (socket says 172.21.15.69:58480) 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e42a00640 1 -- 172.21.15.69:0/1871081299 learned_addr learned my addr 172.21.15.69:0/1871081299 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e4c394740 1 -- 172.21.15.69:0/1871081299 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcb8ecb0 con 0x55bbdcb83340 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e4c394740 1 -- 172.21.15.69:0/1871081299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcaa3160 con 0x55bbdcb82a20 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e4c394740 1 -- 172.21.15.69:0/1871081299 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcb8ee30 con 0x55bbdcb8bf20 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e42000640 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb83340 0x55bbdcb89460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e42a00640 1 -- 172.21.15.69:0/1871081299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 msgr2=0x55bbdcb899a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e42a00640 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 0x55bbdcb899a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e42a00640 1 -- 172.21.15.69:0/1871081299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb83340 msgr2=0x55bbdcb89460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e42a00640 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb83340 0x55bbdcb89460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e42a00640 1 -- 172.21.15.69:0/1871081299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbdc8c57d0 con 0x55bbdcb82a20 2024-03-19T01:21:26.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e42000640 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb83340 0x55bbdcb89460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:26.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.413+0000 7f2e4c394740 10 client.0 _ll_register_callbacks cb 0x55bbdcadf170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:26.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.414+0000 7f2e42a00640 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb82a20 0x55bbdcb88f20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2e3001a860 tx=0x7f2e3003dee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5deaafdebdd75a65 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:26.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.414+0000 7f2e23e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:21:26.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.414+0000 7f2e4c394740 1 -- 172.21.15.69:0/1871081299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e3001a510 con 0x55bbdcb82a20 2024-03-19T01:21:26.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.414+0000 7f2e23e00640 1 -- 172.21.15.69:0/1871081299 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30052070 con 0x55bbdcb82a20 2024-03-19T01:21:26.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.415+0000 7f2e23e00640 1 -- 172.21.15.69:0/1871081299 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7f2e300051c0 con 0x55bbdcb82a20 2024-03-19T01:21:26.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.415+0000 7f2e23e00640 1 -- 172.21.15.69:0/1871081299 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2e300053e0 con 0x55bbdcb82a20 2024-03-19T01:21:26.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.415+0000 7f2e4c394740 10 client.27743 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:26.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.415+0000 7f2e4c394740 1 -- 172.21.15.69:0/1871081299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=59}) -- 0x55bbdcb9ab80 con 0x55bbdcb82a20 2024-03-19T01:21:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.415+0000 7f2e23e00640 1 -- 172.21.15.69:0/1871081299 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f2e3004e480 con 0x55bbdcb82a20 2024-03-19T01:21:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.415+0000 7f2e4c394740 10 client.27743 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.415+0000 7f2e4c394740 10 client.27743 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:21:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.416+0000 7f2e4c394740 20 client.27743 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.416+0000 7f2e4c394740 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bbdcb9bed0 0x55bbdcb9e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.416+0000 7f2e4c394740 4 client.27743 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:21:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.416+0000 7f2e4c394740 1 -- 172.21.15.69:0/1871081299 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbdcaf48c0 con 0x55bbdcb9bed0 2024-03-19T01:21:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.416+0000 7f2e42000640 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bbdcb9bed0 0x55bbdcb9e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.417+0000 7f2e42000640 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bbdcb9bed0 0x55bbdcb9e2c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=50460e998dba1dcf server_cookie=8442960c4e09d6ad in_seq=0 out_seq=0 2024-03-19T01:21:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.417+0000 7f2e23e00640 10 client.27743 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:21:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.418+0000 7f2e23e00640 1 -- 172.21.15.69:0/1871081299 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bbdcaf48c0 con 0x55bbdcb9bed0 2024-03-19T01:21:26.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.418+0000 7f2e23e00640 10 client.27743 handle_command_reply: tid=0 2024-03-19T01:21:26.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.418+0000 7f2e4c394740 1 client.27743 shutdown 2024-03-19T01:21:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.421+0000 7f2e4c394740 1 -- 172.21.15.69:0/1871081299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb82a20 msgr2=0x55bbdcb88f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.421+0000 7f2e4c394740 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb82a20 0x55bbdcb88f20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2e3001a860 tx=0x7f2e3003dee0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.421+0000 7f2e4c394740 1 -- 172.21.15.69:0/1871081299 shutdown_connections 2024-03-19T01:21:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.421+0000 7f2e4c394740 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 0x55bbdcb899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.421+0000 7f2e4c394740 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb83340 0x55bbdcb89460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.421+0000 7f2e4c394740 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bbdcb9bed0 0x55bbdcb9e2c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.421+0000 7f2e4c394740 1 --2- 172.21.15.69:0/1871081299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb82a20 0x55bbdcb88f20 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.421+0000 7f2e4c394740 1 -- 172.21.15.69:0/1871081299 >> 172.21.15.69:0/1871081299 conn(0x55bbdcb7d150 msgr2=0x55bbdcb7eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.421+0000 7f2e23e00640 0 client.27743 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:21:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.424+0000 7f2e4c394740 1 -- 172.21.15.69:0/1871081299 shutdown_connections 2024-03-19T01:21:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.424+0000 7f2e4c394740 1 -- 172.21.15.69:0/1871081299 wait complete. 2024-03-19T01:21:26.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.425+0000 7f2e4c394740 20 client.27743 trim_cache size 0 max 16384 2024-03-19T01:21:26.494 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsnames.o 2024-03-19T01:21:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.493+0000 7f2e4c394740 1 Processor -- start 2024-03-19T01:21:26.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.496+0000 7f2e4c394740 1 -- start start 2024-03-19T01:21:26.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.497+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 0x55bbdcb8b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.497+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb83340 0x55bbdcb97a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.497+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb8bf20 0x55bbdcba38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.497+0000 7f2e4c394740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4407da00 con 0x55bbdcb82a20 2024-03-19T01:21:26.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.497+0000 7f2e4c394740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4407f540 con 0x55bbdcb8bf20 2024-03-19T01:21:26.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.497+0000 7f2e4c394740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4407e0d0 con 0x55bbdcb83340 2024-03-19T01:21:26.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.497+0000 7f2e43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb8bf20 0x55bbdcba38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:26.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.497+0000 7f2e43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb8bf20 0x55bbdcba38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58490/0 (socket says 172.21.15.69:58490) 2024-03-19T01:21:26.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.497+0000 7f2e43400640 1 -- 172.21.15.69:0/3209970613 learned_addr learned my addr 172.21.15.69:0/3209970613 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:26.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.498+0000 7f2e43400640 1 -- 172.21.15.69:0/3209970613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb83340 msgr2=0x55bbdcb97a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:21:26.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.498+0000 7f2e43400640 1 --2- 172.21.15.69:0/3209970613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb83340 0x55bbdcb97a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.498+0000 7f2e43400640 1 -- 172.21.15.69:0/3209970613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 msgr2=0x55bbdcb8b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:26.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.498+0000 7f2e43400640 1 --2- 172.21.15.69:0/3209970613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 0x55bbdcb8b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.498+0000 7f2e43400640 1 -- 172.21.15.69:0/3209970613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbdcb87ed0 con 0x55bbdcb8bf20 2024-03-19T01:21:26.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.499+0000 7f2e43400640 1 --2- 172.21.15.69:0/3209970613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb8bf20 0x55bbdcba38f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c098ce0 tx=0x7f2e3c09ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a80bc46c990876d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:26.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.499+0000 7f2e2ea00640 1 -- 172.21.15.69:0/3209970613 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c020040 con 0x55bbdcb8bf20 2024-03-19T01:21:26.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.499+0000 7f2e2ea00640 1 -- 172.21.15.69:0/3209970613 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e3c0a4ab0 con 0x55bbdcb8bf20 2024-03-19T01:21:26.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.499+0000 7f2e2ea00640 1 -- 172.21.15.69:0/3209970613 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c098980 con 0x55bbdcb8bf20 2024-03-19T01:21:26.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.500+0000 7f2e4c394740 1 -- 172.21.15.69:0/3209970613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb8bf20 msgr2=0x55bbdcba38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:26.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.500+0000 7f2e4c394740 1 --2- 172.21.15.69:0/3209970613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb8bf20 0x55bbdcba38f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c098ce0 tx=0x7f2e3c09ca10 comp rx=0 tx=0).stop 2024-03-19T01:21:26.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.500+0000 7f2e4c394740 1 -- 172.21.15.69:0/3209970613 shutdown_connections 2024-03-19T01:21:26.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.500+0000 7f2e4c394740 1 --2- 172.21.15.69:0/3209970613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb8bf20 0x55bbdcba38f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.500+0000 7f2e4c394740 1 --2- 172.21.15.69:0/3209970613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb83340 0x55bbdcb97a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.500+0000 7f2e4c394740 1 --2- 172.21.15.69:0/3209970613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 0x55bbdcb8b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.500+0000 7f2e4c394740 1 -- 172.21.15.69:0/3209970613 >> 172.21.15.69:0/3209970613 conn(0x55bbdcb7d150 msgr2=0x55bbdcb89770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:26.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.501+0000 7f2e4c394740 1 -- 172.21.15.69:0/3209970613 shutdown_connections 2024-03-19T01:21:26.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.501+0000 7f2e4c394740 1 -- 172.21.15.69:0/3209970613 wait complete. 2024-03-19T01:21:26.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.501+0000 7f2e4c394740 1 -- start start 2024-03-19T01:21:26.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.502+0000 7f2e4c394740 1 Processor -- start 2024-03-19T01:21:26.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:26 smithi086 ceph-mon[66829]: pgmap v335: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-03-19T01:21:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.506+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 0x55bbdcb943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.506+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb90740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.506+0000 7f2e4c394740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 0x55bbdcb90ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.506+0000 7f2e4c394740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4407ee70 con 0x55bbdcb82a20 2024-03-19T01:21:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.506+0000 7f2e4c394740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4407e7a0 con 0x55bbdcb83340 2024-03-19T01:21:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.506+0000 7f2e4c394740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4407b9e0 con 0x55bbdcb8bf20 2024-03-19T01:21:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.506+0000 7f2e4c394740 10 client.0 _ll_register_callbacks cb 0x7f2e30004750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.507+0000 7f2e42000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb90740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.507+0000 7f2e42000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb90740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58496/0 (socket says 172.21.15.69:58496) 2024-03-19T01:21:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.507+0000 7f2e42000640 1 -- 172.21.15.69:0/773492066 learned_addr learned my addr 172.21.15.69:0/773492066 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.507+0000 7f2e43400640 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 0x55bbdcb90ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.507+0000 7f2e42000640 1 -- 172.21.15.69:0/773492066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 msgr2=0x55bbdcb90ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.507+0000 7f2e42000640 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 0x55bbdcb90ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.507+0000 7f2e42000640 1 -- 172.21.15.69:0/773492066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 msgr2=0x55bbdcb943c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:21:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.507+0000 7f2e42000640 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 0x55bbdcb943c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.507+0000 7f2e42000640 1 -- 172.21.15.69:0/773492066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbdcb914d0 con 0x55bbdcb83340 2024-03-19T01:21:26.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.508+0000 7f2e42000640 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb90740 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c900 tx=0x7f2e34004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a647426e87be1c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:26.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.510+0000 7f2e23e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:21:26.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.510+0000 7f2e23e00640 1 -- 172.21.15.69:0/773492066 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340234e0 con 0x55bbdcb83340 2024-03-19T01:21:26.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.510+0000 7f2e23e00640 1 -- 172.21.15.69:0/773492066 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7f2e34023680 con 0x55bbdcb83340 2024-03-19T01:21:26.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.510+0000 7f2e4c394740 1 -- 172.21.15.69:0/773492066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e3c0a4ab0 con 0x55bbdcb83340 2024-03-19T01:21:26.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.510+0000 7f2e23e00640 1 -- 172.21.15.69:0/773492066 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2e3401f420 con 0x55bbdcb83340 2024-03-19T01:21:26.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.510+0000 7f2e4c394740 10 client.27749 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:26.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.510+0000 7f2e4c394740 1 -- 172.21.15.69:0/773492066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55bbdcc03070 con 0x55bbdcb83340 2024-03-19T01:21:26.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.511+0000 7f2e23e00640 1 -- 172.21.15.69:0/773492066 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e3c0a4ab0 con 0x55bbdcb83340 2024-03-19T01:21:26.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.511+0000 7f2e23e00640 1 -- 172.21.15.69:0/773492066 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f2e3402c5e0 con 0x55bbdcb83340 2024-03-19T01:21:26.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.511+0000 7f2e4c394740 10 client.27749 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:26.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.511+0000 7f2e4c394740 10 client.27749 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:21:26.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.511+0000 7f2e4c394740 20 client.27749 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:26.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.511+0000 7f2e4c394740 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bbdcc03790 0x55bbdcc05bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:26.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.511+0000 7f2e4c394740 4 client.27749 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:21:26.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.511+0000 7f2e4c394740 1 -- 172.21.15.69:0/773492066 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbdcaeec10 con 0x55bbdcc03790 2024-03-19T01:21:26.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.511+0000 7f2e42a00640 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bbdcc03790 0x55bbdcc05bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:26.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.512+0000 7f2e42a00640 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bbdcc03790 0x55bbdcc05bb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d5014f273de582e6 server_cookie=5247d7798c08d09e in_seq=0 out_seq=0 2024-03-19T01:21:26.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.512+0000 7f2e23e00640 10 client.27749 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:21:26.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.512+0000 7f2e23e00640 1 -- 172.21.15.69:0/773492066 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbdcaeec10 con 0x55bbdcc03790 2024-03-19T01:21:26.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.512+0000 7f2e23e00640 10 client.27749 handle_command_reply: tid=0 2024-03-19T01:21:26.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.512+0000 7f2e4c394740 1 client.27749 shutdown 2024-03-19T01:21:26.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.512+0000 7f2e4c394740 1 -- 172.21.15.69:0/773492066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 msgr2=0x55bbdcb90740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:26.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.513+0000 7f2e4c394740 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb90740 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c900 tx=0x7f2e34004700 comp rx=0 tx=0).stop 2024-03-19T01:21:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.513+0000 7f2e4c394740 1 -- 172.21.15.69:0/773492066 shutdown_connections 2024-03-19T01:21:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.513+0000 7f2e4c394740 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbdcb8bf20 0x55bbdcb90ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.513+0000 7f2e4c394740 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55bbdcc03790 0x55bbdcc05bb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.513+0000 7f2e4c394740 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbdcb83340 0x55bbdcb90740 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.513+0000 7f2e4c394740 1 --2- 172.21.15.69:0/773492066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdcb82a20 0x55bbdcb943c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.513+0000 7f2e23e00640 0 client.27749 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:21:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.513+0000 7f2e4c394740 1 -- 172.21.15.69:0/773492066 >> 172.21.15.69:0/773492066 conn(0x55bbdcb7d150 msgr2=0x55bbdcb7e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:26.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.515+0000 7f2e4c394740 1 -- 172.21.15.69:0/773492066 shutdown_connections 2024-03-19T01:21:26.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.515+0000 7f2e4c394740 1 -- 172.21.15.69:0/773492066 wait complete. 2024-03-19T01:21:26.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.515+0000 7f2e4c394740 20 client.27749 trim_cache size 0 max 16384 2024-03-19T01:21:26.517 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:21:26.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.522+0000 7f2e49600640 1 -- 172.21.15.69:0/1621333087 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e280a2ab0 msgr2=0x7f2e280a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.522+0000 7f2e49600640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e280a2ab0 0x7f2e280a4f70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55bbdcb7e2e0 tx=0x7f2e34008040 comp rx=0 tx=0).stop 2024-03-19T01:21:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.522+0000 7f2e49600640 1 -- 172.21.15.69:0/1621333087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4411b000 msgr2=0x7f2e44116d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.522+0000 7f2e49600640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4411b000 0x7f2e44116d70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009fd0 tx=0x7f2e3c00b260 comp rx=0 tx=0).stop 2024-03-19T01:21:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.522+0000 7f2e49600640 1 -- 172.21.15.69:0/1621333087 shutdown_connections 2024-03-19T01:21:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.522+0000 7f2e49600640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e4411b000 0x7f2e44116d70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.522+0000 7f2e49600640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e44110040 0x7f2e44116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.522+0000 7f2e49600640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2e280a2ab0 0x7f2e280a4f70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.522+0000 7f2e49600640 1 --2- 172.21.15.69:0/1621333087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e4410e070 0x7f2e4411a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.522+0000 7f2e49600640 1 -- 172.21.15.69:0/1621333087 >> 172.21.15.69:0/1621333087 conn(0x7f2e4406c900 msgr2=0x7f2e4411c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:26.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.522+0000 7f2e49600640 1 -- 172.21.15.69:0/1621333087 shutdown_connections 2024-03-19T01:21:26.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:26.523+0000 7f2e49600640 1 -- 172.21.15.69:0/1621333087 wait complete. 2024-03-19T01:21:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:26 smithi163 ceph-mon[67428]: pgmap v335: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-03-19T01:21:26.575 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:21:26.971 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/freezer.o 2024-03-19T01:21:26.993 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/refcount.o 2024-03-19T01:21:27.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.075+0000 7fea21a00640 1 -- 172.21.15.69:0/3992339337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea1c06b800 msgr2=0x7fea1c06bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.075+0000 7fea21a00640 1 --2- 172.21.15.69:0/3992339337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea1c06b800 0x7fea1c06bc60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fea08009900 tx=0x7fea0802bcd0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.079+0000 7fea21a00640 1 -- 172.21.15.69:0/3992339337 shutdown_connections 2024-03-19T01:21:27.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.079+0000 7fea21a00640 1 --2- 172.21.15.69:0/3992339337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea1c113630 0x7fea1c115a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.079+0000 7fea21a00640 1 --2- 172.21.15.69:0/3992339337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea1c06b800 0x7fea1c06bc60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.079+0000 7fea21a00640 1 --2- 172.21.15.69:0/3992339337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1c0734e0 0x7fea1c06b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.079+0000 7fea21a00640 1 -- 172.21.15.69:0/3992339337 >> 172.21.15.69:0/3992339337 conn(0x7fea1c077820 msgr2=0x7fea1c077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:27.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.081+0000 7fea21a00640 1 -- 172.21.15.69:0/3992339337 shutdown_connections 2024-03-19T01:21:27.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.081+0000 7fea21a00640 1 -- 172.21.15.69:0/3992339337 wait complete. 2024-03-19T01:21:27.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.082+0000 7fea21a00640 1 Processor -- start 2024-03-19T01:21:27.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.082+0000 7fea21a00640 1 -- start start 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.083+0000 7fea21a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea1c06b800 0x7fea1c1a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.083+0000 7fea21a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1c0734e0 0x7fea1c1a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.083+0000 7fea21a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea1c113630 0x7fea1c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.083+0000 7fea21a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c1192a0 con 0x7fea1c0734e0 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.083+0000 7fea21a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c119120 con 0x7fea1c113630 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.083+0000 7fea21a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c119420 con 0x7fea1c06b800 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.083+0000 7fea1aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea1c06b800 0x7fea1c1a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.083+0000 7fea1aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea1c06b800 0x7fea1c1a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60764/0 (socket says 172.21.15.69:60764) 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.083+0000 7fea1aa00640 1 -- 172.21.15.69:0/3556341590 learned_addr learned my addr 172.21.15.69:0/3556341590 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.083+0000 7fea1a000640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1c0734e0 0x7fea1c1a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.084+0000 7fea1aa00640 1 -- 172.21.15.69:0/3556341590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea1c113630 msgr2=0x7fea1c1a0750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:21:27.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.084+0000 7fea1aa00640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea1c113630 0x7fea1c1a0750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.084+0000 7fea1aa00640 1 -- 172.21.15.69:0/3556341590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1c0734e0 msgr2=0x7fea1c1a6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.084+0000 7fea1aa00640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1c0734e0 0x7fea1c1a6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.084+0000 7fea1aa00640 1 -- 172.21.15.69:0/3556341590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1c1a1010 con 0x7fea1c06b800 2024-03-19T01:21:27.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.084+0000 7fea1a000640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1c0734e0 0x7fea1c1a6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:21:27.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.084+0000 7fea1aa00640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea1c06b800 0x7fea1c1a6690 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00eab0 tx=0x7fea0c00ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:27.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.086+0000 7fea13400640 1 -- 172.21.15.69:0/3556341590 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c020070 con 0x7fea1c06b800 2024-03-19T01:21:27.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.086+0000 7fea13400640 1 -- 172.21.15.69:0/3556341590 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea0c00ce40 con 0x7fea1c06b800 2024-03-19T01:21:27.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.086+0000 7fea13400640 1 -- 172.21.15.69:0/3556341590 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c01bd00 con 0x7fea1c06b800 2024-03-19T01:21:27.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.086+0000 7fea21a00640 1 -- 172.21.15.69:0/3556341590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1c1a12a0 con 0x7fea1c06b800 2024-03-19T01:21:27.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.086+0000 7fea21a00640 1 -- 172.21.15.69:0/3556341590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea1c1a15e0 con 0x7fea1c06b800 2024-03-19T01:21:27.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.096+0000 7fea247ae740 1 Processor -- start 2024-03-19T01:21:27.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.095+0000 7fea13400640 1 -- 172.21.15.69:0/3556341590 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fea0c023070 con 0x7fea1c06b800 2024-03-19T01:21:27.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.098+0000 7fea13400640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea000a2ac0 0x7fea000a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.098+0000 7fea13400640 1 -- 172.21.15.69:0/3556341590 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fea0c0d3440 con 0x7fea1c06b800 2024-03-19T01:21:27.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.099+0000 7fea247ae740 1 -- start start 2024-03-19T01:21:27.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.099+0000 7fea247ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98230 0x55d44af98610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.099+0000 7fea247ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af98b50 0x55d44afa11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.099+0000 7fea247ae740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 0x55d44afa3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.099+0000 7fea247ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d44aef4380 con 0x55d44af98230 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.099+0000 7fea247ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d44ae9cd20 con 0x55d44af98b50 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.099+0000 7fea247ae740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d44aef0420 con 0x55d44afa1730 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.100+0000 7fea1b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 0x55d44afa3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.100+0000 7fea1b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 0x55d44afa3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60768/0 (socket says 172.21.15.69:60768) 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.100+0000 7fea1b400640 1 -- 172.21.15.69:0/2657511749 learned_addr learned my addr 172.21.15.69:0/2657511749 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.100+0000 7fea1aa00640 1 --2- 172.21.15.69:0/2657511749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98230 0x55d44af98610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.100+0000 7fea1a000640 1 --2- 172.21.15.69:0/2657511749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af98b50 0x55d44afa11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.100+0000 7fea1b400640 1 -- 172.21.15.69:0/2657511749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af98b50 msgr2=0x55d44afa11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.100+0000 7fea1b400640 1 --2- 172.21.15.69:0/2657511749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af98b50 0x55d44afa11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.101+0000 7fea1b400640 1 -- 172.21.15.69:0/2657511749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98230 msgr2=0x55d44af98610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.101+0000 7fea1b400640 1 --2- 172.21.15.69:0/2657511749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98230 0x55d44af98610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.101+0000 7fea1b400640 1 -- 172.21.15.69:0/2657511749 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d44ac083f0 con 0x55d44afa1730 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.101+0000 7fea1aa00640 1 --2- 172.21.15.69:0/2657511749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98230 0x55d44af98610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.101+0000 7fea1b400640 1 --2- 172.21.15.69:0/2657511749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 0x55d44afa3b20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fea14009c20 tx=0x7fea140107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26f3c8ba823020aa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.101+0000 7fea07e00640 1 -- 172.21.15.69:0/2657511749 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14009e50 con 0x55d44afa1730 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.101+0000 7fea07e00640 1 -- 172.21.15.69:0/2657511749 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea14015070 con 0x55d44afa1730 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.101+0000 7fea07e00640 1 -- 172.21.15.69:0/2657511749 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14011910 con 0x55d44afa1730 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.102+0000 7fea1b400640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea000a2ac0 0x7fea000a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.102+0000 7fea247ae740 1 -- 172.21.15.69:0/2657511749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 msgr2=0x55d44afa3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.102+0000 7fea247ae740 1 --2- 172.21.15.69:0/2657511749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 0x55d44afa3b20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fea14009c20 tx=0x7fea140107a0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.103+0000 7fea1b400640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea000a2ac0 0x7fea000a4f80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fea14000c00 tx=0x7fea14011ba0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:27.105 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/tty_mutex.o 2024-03-19T01:21:27.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.109+0000 7fea247ae740 1 -- 172.21.15.69:0/2657511749 shutdown_connections 2024-03-19T01:21:27.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.109+0000 7fea247ae740 1 --2- 172.21.15.69:0/2657511749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 0x55d44afa3b20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.109+0000 7fea247ae740 1 --2- 172.21.15.69:0/2657511749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af98b50 0x55d44afa11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.109+0000 7fea247ae740 1 --2- 172.21.15.69:0/2657511749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98230 0x55d44af98610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.109+0000 7fea247ae740 1 -- 172.21.15.69:0/2657511749 >> 172.21.15.69:0/2657511749 conn(0x55d44af92960 msgr2=0x55d44af92d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:27.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.109+0000 7fea247ae740 1 -- 172.21.15.69:0/2657511749 shutdown_connections 2024-03-19T01:21:27.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.109+0000 7fea247ae740 1 -- 172.21.15.69:0/2657511749 wait complete. 2024-03-19T01:21:27.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.110+0000 7fea247ae740 1 -- start start 2024-03-19T01:21:27.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.110+0000 7fea247ae740 1 Processor -- start 2024-03-19T01:21:27.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea247ae740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af98230 0x55d44af8fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea247ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 0x55d44af8ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea247ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44afa1730 0x55d44afa01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea1a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 0x55d44af8ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea1aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af98230 0x55d44af8fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea1aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af98230 0x55d44af8fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60770/0 (socket says 172.21.15.69:60770) 2024-03-19T01:21:27.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea1a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 0x55d44af8ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50920/0 (socket says 172.21.15.69:50920) 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44afa1730 0x55d44afa01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44afa1730 0x55d44afa01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58516/0 (socket says 172.21.15.69:58516) 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea247ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d44af89bd0 con 0x55d44af98b50 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea247ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d44aeb0160 con 0x55d44afa1730 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea247ae740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d44af89d50 con 0x55d44af98230 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.113+0000 7fea247ae740 10 client.0 _ll_register_callbacks cb 0x55d44aeec170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.114+0000 7fea1aa00640 1 -- 172.21.15.69:0/377401788 learned_addr learned my addr 172.21.15.69:0/377401788 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.114+0000 7fea1aa00640 1 -- 172.21.15.69:0/377401788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44afa1730 msgr2=0x55d44afa01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.114+0000 7fea1aa00640 1 --2- 172.21.15.69:0/377401788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44afa1730 0x55d44afa01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.114+0000 7fea1aa00640 1 -- 172.21.15.69:0/377401788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 msgr2=0x55d44af8ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.114+0000 7fea1aa00640 1 --2- 172.21.15.69:0/377401788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 0x55d44af8ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.114+0000 7fea1aa00640 1 -- 172.21.15.69:0/377401788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d44acd27d0 con 0x55d44af98230 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.114+0000 7fea04c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.114+0000 7fea1aa00640 1 --2- 172.21.15.69:0/377401788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af98230 0x55d44af8fa00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0a0720 tx=0x7fea0c0a0f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=542a5eb202b5d262 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.115+0000 7fea04c00640 1 -- 172.21.15.69:0/377401788 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0a8f30 con 0x55d44af98230 2024-03-19T01:21:27.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.115+0000 7fea247ae740 1 -- 172.21.15.69:0/377401788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea14015070 con 0x55d44af98230 2024-03-19T01:21:27.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.115+0000 7fea04c00640 1 -- 172.21.15.69:0/377401788 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fea0c0d3150 con 0x55d44af98230 2024-03-19T01:21:27.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.117+0000 7fea04c00640 1 -- 172.21.15.69:0/377401788 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fea0c0a8f30 con 0x55d44af98230 2024-03-19T01:21:27.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.119+0000 7fea247ae740 10 client.27715 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:27.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.119+0000 7fea247ae740 1 -- 172.21.15.69:0/377401788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55d44afca0c0 con 0x55d44af98230 2024-03-19T01:21:27.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.120+0000 7fea04c00640 1 -- 172.21.15.69:0/377401788 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea14015070 con 0x55d44af98230 2024-03-19T01:21:27.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.120+0000 7fea04c00640 1 -- 172.21.15.69:0/377401788 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fea0c00f040 con 0x55d44af98230 2024-03-19T01:21:27.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.120+0000 7fea247ae740 10 client.27715 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:27.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.120+0000 7fea247ae740 10 client.27715 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:21:27.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.120+0000 7fea247ae740 20 client.27715 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:27.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.120+0000 7fea247ae740 1 --2- 172.21.15.69:0/377401788 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55d44afcb900 0x55d44afcdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.120+0000 7fea247ae740 4 client.27715 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:21:27.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.120+0000 7fea247ae740 1 -- 172.21.15.69:0/377401788 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d44af018c0 con 0x55d44afcb900 2024-03-19T01:21:27.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.121+0000 7fea1a000640 1 --2- 172.21.15.69:0/377401788 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55d44afcb900 0x55d44afcdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.121+0000 7fea1a000640 1 --2- 172.21.15.69:0/377401788 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55d44afcb900 0x55d44afcdd50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56f6f1c5011b11bf server_cookie=faa8c405d11b8efc in_seq=0 out_seq=0 2024-03-19T01:21:27.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.121+0000 7fea04c00640 10 client.27715 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:21:27.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.124+0000 7fea04c00640 1 -- 172.21.15.69:0/377401788 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d44af018c0 con 0x55d44afcb900 2024-03-19T01:21:27.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.124+0000 7fea04c00640 10 client.27715 handle_command_reply: tid=0 2024-03-19T01:21:27.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.124+0000 7fea247ae740 1 client.27715 shutdown 2024-03-19T01:21:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.125+0000 7fea247ae740 1 -- 172.21.15.69:0/377401788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af98230 msgr2=0x55d44af8fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.125+0000 7fea247ae740 1 --2- 172.21.15.69:0/377401788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af98230 0x55d44af8fa00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0a0720 tx=0x7fea0c0a0f50 comp rx=0 tx=0).stop 2024-03-19T01:21:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.125+0000 7fea247ae740 1 -- 172.21.15.69:0/377401788 shutdown_connections 2024-03-19T01:21:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.125+0000 7fea247ae740 1 --2- 172.21.15.69:0/377401788 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55d44afcb900 0x55d44afcdd50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.125+0000 7fea247ae740 1 --2- 172.21.15.69:0/377401788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44afa1730 0x55d44afa01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.125+0000 7fea247ae740 1 --2- 172.21.15.69:0/377401788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 0x55d44af8ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.125+0000 7fea247ae740 1 --2- 172.21.15.69:0/377401788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af98230 0x55d44af8fa00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.125+0000 7fea04c00640 0 client.27715 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:21:27.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.125+0000 7fea247ae740 1 -- 172.21.15.69:0/377401788 >> 172.21.15.69:0/377401788 conn(0x55d44af92960 msgr2=0x55d44af946b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:27.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.126+0000 7fea247ae740 1 -- 172.21.15.69:0/377401788 shutdown_connections 2024-03-19T01:21:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.127+0000 7fea247ae740 1 -- 172.21.15.69:0/377401788 wait complete. 2024-03-19T01:21:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.127+0000 7fea247ae740 20 client.27715 trim_cache size 0 max 16384 2024-03-19T01:21:27.132 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/display/drm_hdcp_helper.o 2024-03-19T01:21:27.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.185+0000 7fea247ae740 1 Processor -- start 2024-03-19T01:21:27.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.190+0000 7fea247ae740 1 -- start start 2024-03-19T01:21:27.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.190+0000 7fea247ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af98b50 0x55d44af8fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.190+0000 7fea247ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44afa1730 0x55d44afa00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.190+0000 7fea247ae740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af944a0 0x55d44afd35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.190+0000 7fea247ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c1086d0 con 0x55d44afa1730 2024-03-19T01:21:27.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.190+0000 7fea247ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c10a210 con 0x55d44af98b50 2024-03-19T01:21:27.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.190+0000 7fea247ae740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c108da0 con 0x55d44af944a0 2024-03-19T01:21:27.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.191+0000 7fea1b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af944a0 0x55d44afd35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.191+0000 7fea1b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af944a0 0x55d44afd35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60776/0 (socket says 172.21.15.69:60776) 2024-03-19T01:21:27.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.191+0000 7fea1b400640 1 -- 172.21.15.69:0/272158217 learned_addr learned my addr 172.21.15.69:0/272158217 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:27.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.191+0000 7fea1a000640 1 --2- 172.21.15.69:0/272158217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44afa1730 0x55d44afa00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.191+0000 7fea1b400640 1 -- 172.21.15.69:0/272158217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af98b50 msgr2=0x55d44af8fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.191+0000 7fea1b400640 1 --2- 172.21.15.69:0/272158217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af98b50 0x55d44af8fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.191+0000 7fea1b400640 1 -- 172.21.15.69:0/272158217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44afa1730 msgr2=0x55d44afa00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.191+0000 7fea1b400640 1 --2- 172.21.15.69:0/272158217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44afa1730 0x55d44afa00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.191+0000 7fea1b400640 1 -- 172.21.15.69:0/272158217 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d44afa3e90 con 0x55d44af944a0 2024-03-19T01:21:27.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.192+0000 7fea1b400640 1 --2- 172.21.15.69:0/272158217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af944a0 0x55d44afd35f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fea14009c20 tx=0x7fea14026000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9518419d0298b2dc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:27.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.193+0000 7fea07400640 1 -- 172.21.15.69:0/272158217 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14009c50 con 0x55d44af944a0 2024-03-19T01:21:27.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.193+0000 7fea07400640 1 -- 172.21.15.69:0/272158217 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea14015040 con 0x55d44af944a0 2024-03-19T01:21:27.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.193+0000 7fea07400640 1 -- 172.21.15.69:0/272158217 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14004bc0 con 0x55d44af944a0 2024-03-19T01:21:27.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.194+0000 7fea247ae740 1 -- 172.21.15.69:0/272158217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af944a0 msgr2=0x55d44afd35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.194+0000 7fea247ae740 1 --2- 172.21.15.69:0/272158217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af944a0 0x55d44afd35f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fea14009c20 tx=0x7fea14026000 comp rx=0 tx=0).stop 2024-03-19T01:21:27.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.195+0000 7fea247ae740 1 -- 172.21.15.69:0/272158217 shutdown_connections 2024-03-19T01:21:27.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.195+0000 7fea247ae740 1 --2- 172.21.15.69:0/272158217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44af944a0 0x55d44afd35f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.195+0000 7fea247ae740 1 --2- 172.21.15.69:0/272158217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44afa1730 0x55d44afa00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.195+0000 7fea247ae740 1 --2- 172.21.15.69:0/272158217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af98b50 0x55d44af8fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.195+0000 7fea247ae740 1 -- 172.21.15.69:0/272158217 >> 172.21.15.69:0/272158217 conn(0x55d44af92960 msgr2=0x55d44af93d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:27.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.195+0000 7fea247ae740 1 -- 172.21.15.69:0/272158217 shutdown_connections 2024-03-19T01:21:27.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.195+0000 7fea247ae740 1 -- 172.21.15.69:0/272158217 wait complete. 2024-03-19T01:21:27.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.195+0000 7fea247ae740 1 -- start start 2024-03-19T01:21:27.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.196+0000 7fea247ae740 1 Processor -- start 2024-03-19T01:21:27.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.196+0000 7fea247ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af944a0 0x55d44af9ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.198+0000 7fea247ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 0x55d44af9b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.198+0000 7fea1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af944a0 0x55d44af9ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.198+0000 7fea1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af944a0 0x55d44af9ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58530/0 (socket says 172.21.15.69:58530) 2024-03-19T01:21:27.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.198+0000 7fea1a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 0x55d44af9b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.198+0000 7fea1a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 0x55d44af9b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50934/0 (socket says 172.21.15.69:50934) 2024-03-19T01:21:27.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.199+0000 7fea247ae740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 0x55d44af9b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.199+0000 7fea247ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c109b40 con 0x55d44af98b50 2024-03-19T01:21:27.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.199+0000 7fea247ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c109470 con 0x55d44af944a0 2024-03-19T01:21:27.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.199+0000 7fea247ae740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c106670 con 0x55d44afa1730 2024-03-19T01:21:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.199+0000 7fea247ae740 10 client.0 _ll_register_callbacks cb 0x7fea0c0a0560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.199+0000 7fea1aa00640 1 -- 172.21.15.69:0/4031566479 learned_addr learned my addr 172.21.15.69:0/4031566479 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.199+0000 7fea1b400640 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 0x55d44af9b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.200+0000 7fea1aa00640 1 -- 172.21.15.69:0/4031566479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 msgr2=0x55d44af9b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.200+0000 7fea1aa00640 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 0x55d44af9b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.200+0000 7fea1aa00640 1 -- 172.21.15.69:0/4031566479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 msgr2=0x55d44af9b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.200+0000 7fea1aa00640 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 0x55d44af9b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.200+0000 7fea1aa00640 1 -- 172.21.15.69:0/4031566479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d44af9c080 con 0x55d44af944a0 2024-03-19T01:21:27.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.200+0000 7fea1aa00640 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af944a0 0x55d44af9ff50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000f60 tx=0x7fea0c0d0970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90650b2af1ee9b16 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:27.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.200+0000 7fea04c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:21:27.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.200+0000 7fea04c00640 1 -- 172.21.15.69:0/4031566479 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0a00f0 con 0x55d44af944a0 2024-03-19T01:21:27.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.200+0000 7fea04c00640 1 -- 172.21.15.69:0/4031566479 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fea0c09e4e0 con 0x55d44af944a0 2024-03-19T01:21:27.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.203+0000 7fea247ae740 1 -- 172.21.15.69:0/4031566479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea14015040 con 0x55d44af944a0 2024-03-19T01:21:27.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.206+0000 7fea04c00640 1 -- 172.21.15.69:0/4031566479 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fea0c0a8aa0 con 0x55d44af944a0 2024-03-19T01:21:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.210+0000 7fea247ae740 10 client.27761 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.210+0000 7fea247ae740 1 -- 172.21.15.69:0/4031566479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55d44b031c10 con 0x55d44af944a0 2024-03-19T01:21:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.210+0000 7fea04c00640 1 -- 172.21.15.69:0/4031566479 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea0c0aa080 con 0x55d44af944a0 2024-03-19T01:21:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.210+0000 7fea04c00640 1 -- 172.21.15.69:0/4031566479 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fea0c00f040 con 0x55d44af944a0 2024-03-19T01:21:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.210+0000 7fea247ae740 10 client.27761 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.210+0000 7fea247ae740 10 client.27761 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:21:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.210+0000 7fea247ae740 20 client.27761 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.210+0000 7fea247ae740 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55d44b032140 0x55d44b034530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.210+0000 7fea247ae740 4 client.27761 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:21:27.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.210+0000 7fea247ae740 1 -- 172.21.15.69:0/4031566479 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d44aefbc10 con 0x55d44b032140 2024-03-19T01:21:27.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.211+0000 7fea1a000640 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55d44b032140 0x55d44b034530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.211+0000 7fea1a000640 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55d44b032140 0x55d44b034530 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c85069190d15d95 server_cookie=ce64aa872c5fc438 in_seq=0 out_seq=0 2024-03-19T01:21:27.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.211+0000 7fea04c00640 10 client.27761 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:21:27.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.212+0000 7fea04c00640 1 -- 172.21.15.69:0/4031566479 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d44aefbc10 con 0x55d44b032140 2024-03-19T01:21:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.212+0000 7fea04c00640 10 client.27761 handle_command_reply: tid=0 2024-03-19T01:21:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.212+0000 7fea247ae740 1 client.27761 shutdown 2024-03-19T01:21:27.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.214+0000 7fea247ae740 1 -- 172.21.15.69:0/4031566479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af944a0 msgr2=0x55d44af9ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.214+0000 7fea247ae740 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af944a0 0x55d44af9ff50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000f60 tx=0x7fea0c0d0970 comp rx=0 tx=0).stop 2024-03-19T01:21:27.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.214+0000 7fea247ae740 1 -- 172.21.15.69:0/4031566479 shutdown_connections 2024-03-19T01:21:27.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.214+0000 7fea247ae740 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x55d44b032140 0x55d44b034530 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.214+0000 7fea247ae740 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d44afa1730 0x55d44af9b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.214+0000 7fea247ae740 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d44af98b50 0x55d44af9b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.214+0000 7fea247ae740 1 --2- 172.21.15.69:0/4031566479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d44af944a0 0x55d44af9ff50 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.214+0000 7fea04c00640 0 client.27761 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:21:27.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.214+0000 7fea247ae740 1 -- 172.21.15.69:0/4031566479 >> 172.21.15.69:0/4031566479 conn(0x55d44af92960 msgr2=0x55d44af891d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:27.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.215+0000 7fea247ae740 1 -- 172.21.15.69:0/4031566479 shutdown_connections 2024-03-19T01:21:27.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.215+0000 7fea247ae740 1 -- 172.21.15.69:0/4031566479 wait complete. 2024-03-19T01:21:27.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.215+0000 7fea247ae740 20 client.27761 trim_cache size 0 max 16384 2024-03-19T01:21:27.217 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:21:27.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.221+0000 7fea21a00640 1 -- 172.21.15.69:0/3556341590 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea000a2ac0 msgr2=0x7fea000a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.221+0000 7fea21a00640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea000a2ac0 0x7fea000a4f80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fea14000c00 tx=0x7fea14011ba0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.221+0000 7fea21a00640 1 -- 172.21.15.69:0/3556341590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea1c06b800 msgr2=0x7fea1c1a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.221+0000 7fea21a00640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea1c06b800 0x7fea1c1a6690 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00eab0 tx=0x7fea0c00ef70 comp rx=0 tx=0).stop 2024-03-19T01:21:27.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.221+0000 7fea21a00640 1 -- 172.21.15.69:0/3556341590 shutdown_connections 2024-03-19T01:21:27.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.221+0000 7fea21a00640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea000a2ac0 0x7fea000a4f80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.221+0000 7fea21a00640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea1c113630 0x7fea1c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.221+0000 7fea21a00640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea1c0734e0 0x7fea1c1a6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.221+0000 7fea21a00640 1 --2- 172.21.15.69:0/3556341590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea1c06b800 0x7fea1c1a6690 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.221+0000 7fea21a00640 1 -- 172.21.15.69:0/3556341590 >> 172.21.15.69:0/3556341590 conn(0x7fea1c077820 msgr2=0x7fea1c0ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:27.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.223+0000 7fea21a00640 1 -- 172.21.15.69:0/3556341590 shutdown_connections 2024-03-19T01:21:27.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.223+0000 7fea21a00640 1 -- 172.21.15.69:0/3556341590 wait complete. 2024-03-19T01:21:27.270 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:21:27.308 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsobject.o 2024-03-19T01:21:27.339 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/hpet.o 2024-03-19T01:21:27.423 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/rcuref.o 2024-03-19T01:21:27.742 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/intel/iommu.o 2024-03-19T01:21:27.762 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/usercopy.o 2024-03-19T01:21:27.846 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4super.o 2024-03-19T01:21:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.894+0000 7fea97800640 1 -- 172.21.15.69:0/3354766719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea90109320 msgr2=0x7fea90114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.894+0000 7fea97800640 1 --2- 172.21.15.69:0/3354766719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea90109320 0x7fea90114430 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fea88009be0 tx=0x7fea8802bef0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.894+0000 7fea97800640 1 -- 172.21.15.69:0/3354766719 shutdown_connections 2024-03-19T01:21:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.894+0000 7fea97800640 1 --2- 172.21.15.69:0/3354766719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea90114970 0x7fea90116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.894+0000 7fea97800640 1 --2- 172.21.15.69:0/3354766719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea90109320 0x7fea90114430 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.894+0000 7fea97800640 1 --2- 172.21.15.69:0/3354766719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea90108a00 0x7fea90108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.894+0000 7fea97800640 1 -- 172.21.15.69:0/3354766719 >> 172.21.15.69:0/3354766719 conn(0x7fea9006c900 msgr2=0x7fea9006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.894+0000 7fea97800640 1 -- 172.21.15.69:0/3354766719 shutdown_connections 2024-03-19T01:21:27.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.894+0000 7fea97800640 1 -- 172.21.15.69:0/3354766719 wait complete. 2024-03-19T01:21:27.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.896+0000 7fea97800640 1 Processor -- start 2024-03-19T01:21:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.899+0000 7fea97800640 1 -- start start 2024-03-19T01:21:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.899+0000 7fea97800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea90108a00 0x7fea901a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.899+0000 7fea97800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea90109320 0x7fea901a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.899+0000 7fea97800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea90114970 0x7fea901a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.900+0000 7fea94e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea90108a00 0x7fea901a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.900+0000 7fea94e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea90108a00 0x7fea901a6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60818/0 (socket says 172.21.15.69:60818) 2024-03-19T01:21:27.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.900+0000 7fea95800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea90114970 0x7fea901a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.900+0000 7fea95800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea90114970 0x7fea901a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50948/0 (socket says 172.21.15.69:50948) 2024-03-19T01:21:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.900+0000 7fea95800640 1 -- 172.21.15.69:0/617425073 learned_addr learned my addr 172.21.15.69:0/617425073 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.900+0000 7fea97800640 1 -- 172.21.15.69:0/617425073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea9011a3b0 con 0x7fea90114970 2024-03-19T01:21:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.900+0000 7fea97800640 1 -- 172.21.15.69:0/617425073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea9011a230 con 0x7fea90109320 2024-03-19T01:21:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.900+0000 7fea97800640 1 -- 172.21.15.69:0/617425073 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea9011a530 con 0x7fea90108a00 2024-03-19T01:21:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.900+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea90109320 0x7fea901a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.901+0000 7fea95800640 1 -- 172.21.15.69:0/617425073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea90108a00 msgr2=0x7fea901a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.901+0000 7fea95800640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea90108a00 0x7fea901a6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.901+0000 7fea95800640 1 -- 172.21.15.69:0/617425073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea90109320 msgr2=0x7fea901a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.901+0000 7fea95800640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea90109320 0x7fea901a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.901+0000 7fea95800640 1 -- 172.21.15.69:0/617425073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea901a0f10 con 0x7fea90114970 2024-03-19T01:21:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.901+0000 7fea95800640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea90114970 0x7fea901a0650 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00c970 tx=0x7fea7c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.902+0000 7fea8d600640 1 -- 172.21.15.69:0/617425073 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c007bb0 con 0x7fea90114970 2024-03-19T01:21:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.902+0000 7fea8d600640 1 -- 172.21.15.69:0/617425073 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea7c007d50 con 0x7fea90114970 2024-03-19T01:21:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.902+0000 7fea8d600640 1 -- 172.21.15.69:0/617425073 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c018800 con 0x7fea90114970 2024-03-19T01:21:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.902+0000 7fea97800640 1 -- 172.21.15.69:0/617425073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea901a1200 con 0x7fea90114970 2024-03-19T01:21:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.903+0000 7fea94e00640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea90108a00 0x7fea901a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-19T01:21:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.903+0000 7fea97800640 1 -- 172.21.15.69:0/617425073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea901a14e0 con 0x7fea90114970 2024-03-19T01:21:27.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.912+0000 7fea9a5e3740 1 Processor -- start 2024-03-19T01:21:27.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.916+0000 7fea9a5e3740 1 -- start start 2024-03-19T01:21:27.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.917+0000 7fea9a5e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 0x561199d024a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.918+0000 7fea9a5e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199d02a70 0x561199ceec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.918+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 0x561199d024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.918+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 0x561199d024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58546/0 (socket says 172.21.15.69:58546) 2024-03-19T01:21:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.919+0000 7fea8fe00640 1 -- 172.21.15.69:0/1272837286 learned_addr learned my addr 172.21.15.69:0/1272837286 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.919+0000 7fea8d600640 1 -- 172.21.15.69:0/617425073 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fea7c00f450 con 0x7fea90114970 2024-03-19T01:21:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.919+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1272837286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199cef220 0x561199cef680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.919+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1272837286 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561199c59380 con 0x561199cef220 2024-03-19T01:21:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.919+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1272837286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561199c01d20 con 0x561199d020c0 2024-03-19T01:21:27.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.919+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1272837286 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561199c55420 con 0x561199d02a70 2024-03-19T01:21:27.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.919+0000 7fea94e00640 1 --2- 172.21.15.69:0/1272837286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199d02a70 0x561199ceec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.919+0000 7fea94e00640 1 -- 172.21.15.69:0/1272837286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 msgr2=0x561199d024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.919+0000 7fea94e00640 1 --2- 172.21.15.69:0/1272837286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 0x561199d024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.919+0000 7fea8fe00640 1 --2- 172.21.15.69:0/1272837286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199cef220 0x561199cef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.919+0000 7fea94e00640 1 -- 172.21.15.69:0/1272837286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199cef220 msgr2=0x561199cef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.921+0000 7fea94e00640 1 --2- 172.21.15.69:0/1272837286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199cef220 0x561199cef680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.921+0000 7fea94e00640 1 -- 172.21.15.69:0/1272837286 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119996d3f0 con 0x561199d02a70 2024-03-19T01:21:27.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.921+0000 7fea8fe00640 1 --2- 172.21.15.69:0/1272837286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199cef220 0x561199cef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:27.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.921+0000 7fea8d600640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea740a2b70 0x7fea740a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.921+0000 7fea8d600640 1 -- 172.21.15.69:0/617425073 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fea7c0cf310 con 0x7fea90114970 2024-03-19T01:21:27.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.922+0000 7fea94e00640 1 --2- 172.21.15.69:0/1272837286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199d02a70 0x561199ceec00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fea80009b70 tx=0x7fea80010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78bc92e365afab0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:27.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.923+0000 7fea8fe00640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea740a2b70 0x7fea740a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.923+0000 7fea86000640 1 -- 172.21.15.69:0/1272837286 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80009e60 con 0x561199d02a70 2024-03-19T01:21:27.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.923+0000 7fea86000640 1 -- 172.21.15.69:0/1272837286 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea80003610 con 0x561199d02a70 2024-03-19T01:21:27.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.923+0000 7fea86000640 1 -- 172.21.15.69:0/1272837286 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800126b0 con 0x561199d02a70 2024-03-19T01:21:27.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.929+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1272837286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199d02a70 msgr2=0x561199ceec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.929+0000 7fea8fe00640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea740a2b70 0x7fea740a5030 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x561199cefd40 tx=0x7fea8802bef0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:27.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.929+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1272837286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199d02a70 0x561199ceec00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fea80009b70 tx=0x7fea80010740 comp rx=0 tx=0).stop 2024-03-19T01:21:27.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.932+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1272837286 shutdown_connections 2024-03-19T01:21:27.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.932+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1272837286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199cef220 0x561199cef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.932+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1272837286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199d02a70 0x561199ceec00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.932+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1272837286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 0x561199d024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.932+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1272837286 >> 172.21.15.69:0/1272837286 conn(0x561199cf16c0 msgr2=0x561199cf1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:27.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.932+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1272837286 shutdown_connections 2024-03-19T01:21:27.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.932+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1272837286 wait complete. 2024-03-19T01:21:27.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.933+0000 7fea9a5e3740 1 -- start start 2024-03-19T01:21:27.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.933+0000 7fea9a5e3740 1 Processor -- start 2024-03-19T01:21:27.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.934+0000 7fea9a5e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d06070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.934+0000 7fea9a5e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 0x561199d065b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.934+0000 7fea9a5e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d02a70 0x561199d06af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.934+0000 7fea9a5e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561199d09500 con 0x561199d02a70 2024-03-19T01:21:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.934+0000 7fea9a5e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561199c15160 con 0x561199d020c0 2024-03-19T01:21:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.934+0000 7fea9a5e3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561199d09680 con 0x561199cef220 2024-03-19T01:21:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.934+0000 7fea9a5e3740 10 client.0 _ll_register_callbacks cb 0x561199c51170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.934+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 0x561199d065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.935+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 0x561199d065b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58548/0 (socket says 172.21.15.69:58548) 2024-03-19T01:21:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.935+0000 7fea8fe00640 1 -- 172.21.15.69:0/2793112586 learned_addr learned my addr 172.21.15.69:0/2793112586 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.935+0000 7fea95800640 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d02a70 0x561199d06af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.935+0000 7fea94e00640 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d06070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.935+0000 7fea8fe00640 1 -- 172.21.15.69:0/2793112586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 msgr2=0x561199d06070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.935+0000 7fea8fe00640 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d06070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.935+0000 7fea8fe00640 1 -- 172.21.15.69:0/2793112586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d02a70 msgr2=0x561199d06af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:27.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.935+0000 7fea8fe00640 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d02a70 0x561199d06af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.935+0000 7fea8fe00640 1 -- 172.21.15.69:0/2793112586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561199a377d0 con 0x561199d020c0 2024-03-19T01:21:27.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.935+0000 7fea95800640 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d02a70 0x561199d06af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:21:27.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.936+0000 7fea8fe00640 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 0x561199d065b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fea8802c500 tx=0x7fea88005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98a415a796856108 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:27.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.936+0000 7fea7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:21:27.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.936+0000 7fea94e00640 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d06070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:27.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.936+0000 7fea7aa00640 1 -- 172.21.15.69:0/2793112586 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88047070 con 0x561199d020c0 2024-03-19T01:21:27.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.936+0000 7fea7aa00640 1 -- 172.21.15.69:0/2793112586 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fea88003e00 con 0x561199d020c0 2024-03-19T01:21:27.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.937+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2793112586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea80003610 con 0x561199d020c0 2024-03-19T01:21:27.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.937+0000 7fea7aa00640 1 -- 172.21.15.69:0/2793112586 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fea88033900 con 0x561199d020c0 2024-03-19T01:21:27.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.937+0000 7fea9a5e3740 10 client.27773 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.937+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2793112586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x561199d157b0 con 0x561199d020c0 2024-03-19T01:21:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.937+0000 7fea7aa00640 1 -- 172.21.15.69:0/2793112586 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea80003610 con 0x561199d020c0 2024-03-19T01:21:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.938+0000 7fea7aa00640 1 -- 172.21.15.69:0/2793112586 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fea8804e950 con 0x561199d020c0 2024-03-19T01:21:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.938+0000 7fea9a5e3740 10 client.27773 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.938+0000 7fea9a5e3740 10 client.27773 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:21:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.938+0000 7fea9a5e3740 20 client.27773 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.938+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x561199d16cc0 0x561199d19160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:27.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.938+0000 7fea9a5e3740 4 client.27773 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:21:27.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.938+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2793112586 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561199c668c0 con 0x561199d16cc0 2024-03-19T01:21:27.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.939+0000 7fea94e00640 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x561199d16cc0 0x561199d19160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:27.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.940+0000 7fea94e00640 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x561199d16cc0 0x561199d19160 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9699f152ceffa104 server_cookie=44f26550f7eb09bc in_seq=0 out_seq=0 2024-03-19T01:21:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.943+0000 7fea7aa00640 10 client.27773 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:21:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.949+0000 7fea7aa00640 1 -- 172.21.15.69:0/2793112586 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561199c668c0 con 0x561199d16cc0 2024-03-19T01:21:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.949+0000 7fea7aa00640 10 client.27773 handle_command_reply: tid=0 2024-03-19T01:21:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.949+0000 7fea9a5e3740 1 client.27773 shutdown 2024-03-19T01:21:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.953+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2793112586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 msgr2=0x561199d065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.953+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 0x561199d065b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fea8802c500 tx=0x7fea88005fd0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.955+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2793112586 shutdown_connections 2024-03-19T01:21:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.955+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d02a70 0x561199d06af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.955+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x561199d16cc0 0x561199d19160 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.955+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d020c0 0x561199d065b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.955+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/2793112586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d06070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.955+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2793112586 >> 172.21.15.69:0/2793112586 conn(0x561199cf16c0 msgr2=0x561199cf0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.955+0000 7fea7aa00640 0 client.27773 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:21:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.957+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2793112586 shutdown_connections 2024-03-19T01:21:27.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.957+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2793112586 wait complete. 2024-03-19T01:21:27.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:27.958+0000 7fea9a5e3740 20 client.27773 trim_cache size 0 max 16384 2024-03-19T01:21:28.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.019+0000 7fea9a5e3740 1 Processor -- start 2024-03-19T01:21:28.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.019+0000 7fea9a5e3740 1 -- start start 2024-03-19T01:21:28.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.019+0000 7fea9a5e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.019+0000 7fea9a5e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d02a70 0x561199cf2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.019+0000 7fea9a5e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d12310 0x561199d06910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.019+0000 7fea9a5e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b5360 con 0x561199d12310 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.019+0000 7fea9a5e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b6ea0 con 0x561199d02a70 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.019+0000 7fea9a5e3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b5a30 con 0x561199cef220 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.020+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d02a70 0x561199cf2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.020+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d02a70 0x561199cf2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58550/0 (socket says 172.21.15.69:58550) 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.020+0000 7fea8fe00640 1 -- 172.21.15.69:0/2014126041 learned_addr learned my addr 172.21.15.69:0/2014126041 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.020+0000 7fea94e00640 1 --2- 172.21.15.69:0/2014126041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.021+0000 7fea94e00640 1 -- 172.21.15.69:0/2014126041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d02a70 msgr2=0x561199cf2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.021+0000 7fea94e00640 1 --2- 172.21.15.69:0/2014126041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d02a70 0x561199cf2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.021+0000 7fea94e00640 1 -- 172.21.15.69:0/2014126041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d12310 msgr2=0x561199d06910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:28.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.021+0000 7fea94e00640 1 --2- 172.21.15.69:0/2014126041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d12310 0x561199d06910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.021+0000 7fea94e00640 1 -- 172.21.15.69:0/2014126041 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561199d05040 con 0x561199cef220 2024-03-19T01:21:28.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.021+0000 7fea94e00640 1 --2- 172.21.15.69:0/2014126041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d087a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fea80000c00 tx=0x7fea80009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec8f1fe79ebf2c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.025+0000 7fea85600640 1 -- 172.21.15.69:0/2014126041 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80005110 con 0x561199cef220 2024-03-19T01:21:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.025+0000 7fea85600640 1 -- 172.21.15.69:0/2014126041 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea800052b0 con 0x561199cef220 2024-03-19T01:21:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.025+0000 7fea85600640 1 -- 172.21.15.69:0/2014126041 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80003a10 con 0x561199cef220 2024-03-19T01:21:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.027+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2014126041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 msgr2=0x561199d087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.027+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/2014126041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d087a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fea80000c00 tx=0x7fea80009460 comp rx=0 tx=0).stop 2024-03-19T01:21:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.027+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2014126041 shutdown_connections 2024-03-19T01:21:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.027+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/2014126041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d12310 0x561199d06910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.027+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/2014126041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d02a70 0x561199cf2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.027+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/2014126041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d087a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.027+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2014126041 >> 172.21.15.69:0/2014126041 conn(0x561199cf16c0 msgr2=0x561199d01990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.027+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2014126041 shutdown_connections 2024-03-19T01:21:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.027+0000 7fea9a5e3740 1 -- 172.21.15.69:0/2014126041 wait complete. 2024-03-19T01:21:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.028+0000 7fea9a5e3740 1 -- start start 2024-03-19T01:21:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.028+0000 7fea9a5e3740 1 Processor -- start 2024-03-19T01:21:28.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.029+0000 7fea9a5e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d76290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.030+0000 7fea94e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d76290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.030+0000 7fea94e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d76290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60840/0 (socket says 172.21.15.69:60840) 2024-03-19T01:21:28.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.030+0000 7fea94e00640 1 -- 172.21.15.69:0/1802926695 learned_addr learned my addr 172.21.15.69:0/1802926695 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:28.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.030+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d02a70 0x561199d767d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.030+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d12310 0x561199d1e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.030+0000 7fea8fe00640 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d02a70 0x561199d767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.031+0000 7fea95800640 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d12310 0x561199d1e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.035+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1802926695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b67d0 con 0x561199d12310 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.035+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1802926695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b6100 con 0x561199d02a70 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.035+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1802926695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b3340 con 0x561199cef220 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.035+0000 7fea9a5e3740 10 client.0 _ll_register_callbacks cb 0x7fea880335d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.035+0000 7fea94e00640 1 -- 172.21.15.69:0/1802926695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d02a70 msgr2=0x561199d767d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.035+0000 7fea94e00640 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d02a70 0x561199d767d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.035+0000 7fea94e00640 1 -- 172.21.15.69:0/1802926695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d12310 msgr2=0x561199d1e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.035+0000 7fea94e00640 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d12310 0x561199d1e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.035+0000 7fea94e00640 1 -- 172.21.15.69:0/1802926695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561199d1e9c0 con 0x561199cef220 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.036+0000 7fea95800640 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d12310 0x561199d1e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.036+0000 7fea94e00640 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d76290 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fea800096e0 tx=0x7fea80009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56ccf444f83f4a94 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:28.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.036+0000 7fea7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.037+0000 7fea7aa00640 1 -- 172.21.15.69:0/1802926695 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800274d0 con 0x561199cef220 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.037+0000 7fea7aa00640 1 -- 172.21.15.69:0/1802926695 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fea80027670 con 0x561199cef220 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.037+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1802926695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea800052b0 con 0x561199cef220 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.037+0000 7fea7aa00640 1 -- 172.21.15.69:0/1802926695 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fea8003d050 con 0x561199cef220 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.037+0000 7fea9a5e3740 10 client.27733 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.037+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1802926695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x561199d7da50 con 0x561199cef220 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.038+0000 7fea7aa00640 1 -- 172.21.15.69:0/1802926695 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea800052b0 con 0x561199cef220 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.038+0000 7fea7aa00640 1 -- 172.21.15.69:0/1802926695 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fea80038030 con 0x561199cef220 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.038+0000 7fea9a5e3740 10 client.27733 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.038+0000 7fea9a5e3740 10 client.27733 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.038+0000 7fea9a5e3740 20 client.27733 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.038+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x561199d7ded0 0x561199d802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.038+0000 7fea9a5e3740 4 client.27733 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:21:28.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.038+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1802926695 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561199c60c10 con 0x561199d7ded0 2024-03-19T01:21:28.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.042+0000 7fea8fe00640 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x561199d7ded0 0x561199d802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.046+0000 7fea8fe00640 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x561199d7ded0 0x561199d802c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32385ae8dad0392a server_cookie=1096e33081dfd030 in_seq=0 out_seq=0 2024-03-19T01:21:28.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.050+0000 7fea7aa00640 10 client.27733 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:21:28.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.050+0000 7fea7aa00640 1 -- 172.21.15.69:0/1802926695 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561199c60c10 con 0x561199d7ded0 2024-03-19T01:21:28.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.050+0000 7fea7aa00640 10 client.27733 handle_command_reply: tid=0 2024-03-19T01:21:28.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.050+0000 7fea9a5e3740 1 client.27733 shutdown 2024-03-19T01:21:28.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.053+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1802926695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 msgr2=0x561199d76290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.053+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d76290 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fea800096e0 tx=0x7fea80009f50 comp rx=0 tx=0).stop 2024-03-19T01:21:28.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.053+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1802926695 shutdown_connections 2024-03-19T01:21:28.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.053+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561199d12310 0x561199d1e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.053+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x561199d7ded0 0x561199d802c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.054+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561199d02a70 0x561199d767d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.054+0000 7fea7aa00640 0 client.27733 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:21:28.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.054+0000 7fea9a5e3740 1 --2- 172.21.15.69:0/1802926695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561199cef220 0x561199d76290 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.054+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1802926695 >> 172.21.15.69:0/1802926695 conn(0x561199cf16c0 msgr2=0x561199cf0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:28.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.057+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1802926695 shutdown_connections 2024-03-19T01:21:28.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.057+0000 7fea9a5e3740 1 -- 172.21.15.69:0/1802926695 wait complete. 2024-03-19T01:21:28.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.058+0000 7fea9a5e3740 20 client.27733 trim_cache size 0 max 16384 2024-03-19T01:21:28.058 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:21:28.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.065+0000 7fea97800640 1 -- 172.21.15.69:0/617425073 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea740a2b70 msgr2=0x7fea740a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.065+0000 7fea97800640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea740a2b70 0x7fea740a5030 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x561199cefd40 tx=0x7fea8802bef0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.065+0000 7fea97800640 1 -- 172.21.15.69:0/617425073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea90114970 msgr2=0x7fea901a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.065+0000 7fea97800640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea90114970 0x7fea901a0650 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00c970 tx=0x7fea7c00ce30 comp rx=0 tx=0).stop 2024-03-19T01:21:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.068+0000 7fea97800640 1 -- 172.21.15.69:0/617425073 shutdown_connections 2024-03-19T01:21:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.068+0000 7fea97800640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea90114970 0x7fea901a0650 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.068+0000 7fea97800640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fea740a2b70 0x7fea740a5030 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.068+0000 7fea97800640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea90109320 0x7fea901a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.068+0000 7fea97800640 1 --2- 172.21.15.69:0/617425073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea90108a00 0x7fea901a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.068+0000 7fea97800640 1 -- 172.21.15.69:0/617425073 >> 172.21.15.69:0/617425073 conn(0x7fea9006c900 msgr2=0x7fea90070480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:28.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.069+0000 7fea97800640 1 -- 172.21.15.69:0/617425073 shutdown_connections 2024-03-19T01:21:28.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.069+0000 7fea97800640 1 -- 172.21.15.69:0/617425073 wait complete. 2024-03-19T01:21:28.099 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/tty_ldsem.o 2024-03-19T01:21:28.111 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:21:28.170 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/display/drm_hdmi_helper.o 2024-03-19T01:21:28.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:28 smithi069 ceph-mon[31727]: pgmap v336: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-03-19T01:21:28.330 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/profile.o 2024-03-19T01:21:28.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:28 smithi086 ceph-mon[66829]: pgmap v336: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-03-19T01:21:28.527 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsparse.o 2024-03-19T01:21:28.549 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/errseq.o 2024-03-19T01:21:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:28 smithi163 ceph-mon[67428]: pgmap v336: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-03-19T01:21:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.711+0000 7fe931600640 1 -- 172.21.15.69:0/298065157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92c10e070 msgr2=0x7fe92c10e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.711+0000 7fe931600640 1 --2- 172.21.15.69:0/298065157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92c10e070 0x7fe92c10e4f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe924009ca0 tx=0x7fe92402c120 comp rx=0 tx=0).stop 2024-03-19T01:21:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.711+0000 7fe931600640 1 -- 172.21.15.69:0/298065157 shutdown_connections 2024-03-19T01:21:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.711+0000 7fe931600640 1 --2- 172.21.15.69:0/298065157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe92c11b000 0x7fe92c11d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.711+0000 7fe931600640 1 --2- 172.21.15.69:0/298065157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92c10e070 0x7fe92c10e4f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.711+0000 7fe931600640 1 --2- 172.21.15.69:0/298065157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe92c110040 0x7fe92c10db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.711+0000 7fe931600640 1 -- 172.21.15.69:0/298065157 >> 172.21.15.69:0/298065157 conn(0x7fe92c06c900 msgr2=0x7fe92c06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.711+0000 7fe931600640 1 -- 172.21.15.69:0/298065157 shutdown_connections 2024-03-19T01:21:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.715+0000 7fe931600640 1 -- 172.21.15.69:0/298065157 wait complete. 2024-03-19T01:21:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.716+0000 7fe931600640 1 Processor -- start 2024-03-19T01:21:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.717+0000 7fe931600640 1 -- start start 2024-03-19T01:21:28.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.718+0000 7fe931600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe92c10e070 0x7fe92c1185f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.718+0000 7fe931600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe92c110040 0x7fe92c111670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.718+0000 7fe931600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92c11b000 0x7fe92c111bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.718+0000 7fe931600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1209b0 con 0x7fe92c110040 2024-03-19T01:21:28.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.718+0000 7fe931600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c120830 con 0x7fe92c10e070 2024-03-19T01:21:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.718+0000 7fe92aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe92c10e070 0x7fe92c1185f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.718+0000 7fe92aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe92c10e070 0x7fe92c1185f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58568/0 (socket says 172.21.15.69:58568) 2024-03-19T01:21:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.718+0000 7fe92aa00640 1 -- 172.21.15.69:0/3031438714 learned_addr learned my addr 172.21.15.69:0/3031438714 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.718+0000 7fe92b400640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92c11b000 0x7fe92c111bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.719+0000 7fe931600640 1 -- 172.21.15.69:0/3031438714 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c120b30 con 0x7fe92c11b000 2024-03-19T01:21:28.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.719+0000 7fe92b400640 1 -- 172.21.15.69:0/3031438714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe92c10e070 msgr2=0x7fe92c1185f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.719+0000 7fe92b400640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe92c10e070 0x7fe92c1185f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.719+0000 7fe92b400640 1 -- 172.21.15.69:0/3031438714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe92c110040 msgr2=0x7fe92c111670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:21:28.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.720+0000 7fe92a000640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe92c110040 0x7fe92c111670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.720+0000 7fe92b400640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe92c110040 0x7fe92c111670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.720+0000 7fe92b400640 1 -- 172.21.15.69:0/3031438714 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c1121a0 con 0x7fe92c11b000 2024-03-19T01:21:28.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.720+0000 7fe92aa00640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe92c10e070 0x7fe92c1185f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:21:28.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.720+0000 7fe92a000640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe92c110040 0x7fe92c111670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.720+0000 7fe92b400640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92c11b000 0x7fe92c111bb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe91800e970 tx=0x7fe91800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.721+0000 7fe923400640 1 -- 172.21.15.69:0/3031438714 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91800cc60 con 0x7fe92c11b000 2024-03-19T01:21:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.721+0000 7fe923400640 1 -- 172.21.15.69:0/3031438714 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91800ce00 con 0x7fe92c11b000 2024-03-19T01:21:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.721+0000 7fe923400640 1 -- 172.21.15.69:0/3031438714 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918004a90 con 0x7fe92c11b000 2024-03-19T01:21:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.721+0000 7fe931600640 1 -- 172.21.15.69:0/3031438714 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c112490 con 0x7fe92c11b000 2024-03-19T01:21:28.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.721+0000 7fe931600640 1 -- 172.21.15.69:0/3031438714 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c1163e0 con 0x7fe92c11b000 2024-03-19T01:21:28.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.729+0000 7fe923400640 1 -- 172.21.15.69:0/3031438714 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fe918004c30 con 0x7fe92c11b000 2024-03-19T01:21:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.730+0000 7fe9343c0740 1 Processor -- start 2024-03-19T01:21:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.733+0000 7fe9343c0740 1 -- start start 2024-03-19T01:21:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe9343c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe9343c0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47e6340 0x560be47ee9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe9343c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47eef20 0x560be47f1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe9343c0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560be474a380 con 0x560be47e5a20 2024-03-19T01:21:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe9343c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560be46f2d20 con 0x560be47eef20 2024-03-19T01:21:28.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe9343c0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560be4746420 con 0x560be47e6340 2024-03-19T01:21:28.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe92aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47e6340 0x560be47ee9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe92aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47e6340 0x560be47ee9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60858/0 (socket says 172.21.15.69:60858) 2024-03-19T01:21:28.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe92aa00640 1 -- 172.21.15.69:0/833636589 learned_addr learned my addr 172.21.15.69:0/833636589 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:28.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe92a000640 1 --2- 172.21.15.69:0/833636589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe92aa00640 1 -- 172.21.15.69:0/833636589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47eef20 msgr2=0x560be47f1310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:21:28.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe92aa00640 1 --2- 172.21.15.69:0/833636589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47eef20 0x560be47f1310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.735+0000 7fe92aa00640 1 -- 172.21.15.69:0/833636589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 msgr2=0x560be47e5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:28.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.736+0000 7fe92aa00640 1 --2- 172.21.15.69:0/833636589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47e5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.736+0000 7fe92aa00640 1 -- 172.21.15.69:0/833636589 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be445e3f0 con 0x560be47e6340 2024-03-19T01:21:28.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.736+0000 7fe92a000640 1 --2- 172.21.15.69:0/833636589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47e5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:21:28.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.736+0000 7fe92aa00640 1 --2- 172.21.15.69:0/833636589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47e6340 0x560be47ee9e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00f4f0 tx=0x7fe91c00f9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b228c6ccdd872232 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:28.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.736+0000 7fe917400640 1 -- 172.21.15.69:0/833636589 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c005220 con 0x560be47e6340 2024-03-19T01:21:28.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.736+0000 7fe917400640 1 -- 172.21.15.69:0/833636589 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c004c80 con 0x560be47e6340 2024-03-19T01:21:28.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.736+0000 7fe917400640 1 -- 172.21.15.69:0/833636589 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c010d00 con 0x560be47e6340 2024-03-19T01:21:28.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.737+0000 7fe9343c0740 1 -- 172.21.15.69:0/833636589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47e6340 msgr2=0x560be47ee9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.737+0000 7fe9343c0740 1 --2- 172.21.15.69:0/833636589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47e6340 0x560be47ee9e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00f4f0 tx=0x7fe91c00f9b0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.738+0000 7fe9343c0740 1 -- 172.21.15.69:0/833636589 shutdown_connections 2024-03-19T01:21:28.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.738+0000 7fe9343c0740 1 --2- 172.21.15.69:0/833636589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47eef20 0x560be47f1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.738+0000 7fe9343c0740 1 --2- 172.21.15.69:0/833636589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47e6340 0x560be47ee9e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.738+0000 7fe9343c0740 1 --2- 172.21.15.69:0/833636589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47e5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.738+0000 7fe9343c0740 1 -- 172.21.15.69:0/833636589 >> 172.21.15.69:0/833636589 conn(0x560be47e0150 msgr2=0x560be47e0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:28.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.738+0000 7fe923400640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9100a2a30 0x7fe9100a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.738+0000 7fe92a000640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9100a2a30 0x7fe9100a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.738+0000 7fe923400640 1 -- 172.21.15.69:0/3031438714 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fe9180cf0d0 con 0x7fe92c11b000 2024-03-19T01:21:28.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.740+0000 7fe92a000640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9100a2a30 0x7fe9100a4ef0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe924009bf0 tx=0x7fe92402e040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:28.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.741+0000 7fe9343c0740 1 -- 172.21.15.69:0/833636589 shutdown_connections 2024-03-19T01:21:28.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.741+0000 7fe9343c0740 1 -- 172.21.15.69:0/833636589 wait complete. 2024-03-19T01:21:28.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.741+0000 7fe9343c0740 1 -- start start 2024-03-19T01:21:28.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.742+0000 7fe9343c0740 1 Processor -- start 2024-03-19T01:21:28.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.745+0000 7fe9343c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.745+0000 7fe9343c0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47e6340 0x560be47ec500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.745+0000 7fe9343c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47eef20 0x560be47eca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.745+0000 7fe9343c0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560be47f1d10 con 0x560be47e5a20 2024-03-19T01:21:28.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.745+0000 7fe9343c0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560be4706160 con 0x560be47eef20 2024-03-19T01:21:28.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.745+0000 7fe9343c0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560be47f1e90 con 0x560be47e6340 2024-03-19T01:21:28.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.745+0000 7fe9343c0740 10 client.0 _ll_register_callbacks cb 0x560be4742170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:28.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.745+0000 7fe92b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47eef20 0x560be47eca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.745+0000 7fe92b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47eef20 0x560be47eca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58582/0 (socket says 172.21.15.69:58582) 2024-03-19T01:21:28.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.745+0000 7fe92b400640 1 -- 172.21.15.69:0/1817713886 learned_addr learned my addr 172.21.15.69:0/1817713886 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:28.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.745+0000 7fe92aa00640 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.746+0000 7fe92b400640 1 -- 172.21.15.69:0/1817713886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47e6340 msgr2=0x560be47ec500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:21:28.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.746+0000 7fe92b400640 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47e6340 0x560be47ec500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.746+0000 7fe92b400640 1 -- 172.21.15.69:0/1817713886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 msgr2=0x560be47ebfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:28.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.746+0000 7fe92b400640 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47ebfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.746+0000 7fe92b400640 1 -- 172.21.15.69:0/1817713886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be45287d0 con 0x560be47eef20 2024-03-19T01:21:28.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.746+0000 7fe92aa00640 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:21:28.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.746+0000 7fe92b400640 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47eef20 0x560be47eca40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe9180a3590 tx=0x7fe91809adb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e1dd235d76e416 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:28.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.746+0000 7fe90be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:21:28.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.746+0000 7fe90be00640 1 -- 172.21.15.69:0/1817713886 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91801c040 con 0x560be47eef20 2024-03-19T01:21:28.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.747+0000 7fe90be00640 1 -- 172.21.15.69:0/1817713886 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fe918097990 con 0x560be47eef20 2024-03-19T01:21:28.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.747+0000 7fe9343c0740 1 -- 172.21.15.69:0/1817713886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe91c004c80 con 0x560be47eef20 2024-03-19T01:21:28.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.747+0000 7fe90be00640 1 -- 172.21.15.69:0/1817713886 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe9180a3ac0 con 0x560be47eef20 2024-03-19T01:21:28.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.751+0000 7fe9343c0740 10 client.27788 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:28.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.751+0000 7fe9343c0740 1 -- 172.21.15.69:0/1817713886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x560be47fe070 con 0x560be47eef20 2024-03-19T01:21:28.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.751+0000 7fe90be00640 1 -- 172.21.15.69:0/1817713886 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c004c80 con 0x560be47eef20 2024-03-19T01:21:28.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.752+0000 7fe90be00640 1 -- 172.21.15.69:0/1817713886 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe9180a36c0 con 0x560be47eef20 2024-03-19T01:21:28.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.752+0000 7fe9343c0740 10 client.27788 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:28.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.752+0000 7fe9343c0740 10 client.27788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:21:28.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.752+0000 7fe9343c0740 20 client.27788 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:28.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.752+0000 7fe9343c0740 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560be47ff580 0x560be4801a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.752+0000 7fe9343c0740 4 client.27788 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:21:28.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.752+0000 7fe9343c0740 1 -- 172.21.15.69:0/1817713886 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560be47578c0 con 0x560be47ff580 2024-03-19T01:21:28.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.755+0000 7fe92aa00640 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560be47ff580 0x560be4801a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.756+0000 7fe92aa00640 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560be47ff580 0x560be4801a10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=702a2f2482530959 server_cookie=6fe99812625927e3 in_seq=0 out_seq=0 2024-03-19T01:21:28.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.756+0000 7fe90be00640 10 client.27788 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:28.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.757+0000 7fe90be00640 1 -- 172.21.15.69:0/1817713886 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560be47578c0 con 0x560be47ff580 2024-03-19T01:21:28.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.757+0000 7fe90be00640 10 client.27788 handle_command_reply: tid=0 2024-03-19T01:21:28.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.757+0000 7fe9343c0740 1 client.27788 shutdown 2024-03-19T01:21:28.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.759+0000 7fe9343c0740 1 -- 172.21.15.69:0/1817713886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47eef20 msgr2=0x560be47eca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.759+0000 7fe9343c0740 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47eef20 0x560be47eca40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe9180a3590 tx=0x7fe91809adb0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.759+0000 7fe9343c0740 1 -- 172.21.15.69:0/1817713886 shutdown_connections 2024-03-19T01:21:28.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.759+0000 7fe9343c0740 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560be47ff580 0x560be4801a10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.759+0000 7fe9343c0740 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47eef20 0x560be47eca40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.759+0000 7fe9343c0740 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47e6340 0x560be47ec500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.759+0000 7fe9343c0740 1 --2- 172.21.15.69:0/1817713886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.759+0000 7fe90be00640 0 client.27788 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:28.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.759+0000 7fe9343c0740 1 -- 172.21.15.69:0/1817713886 >> 172.21.15.69:0/1817713886 conn(0x560be47e0150 msgr2=0x560be47e1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:28.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.767+0000 7fe9343c0740 1 -- 172.21.15.69:0/1817713886 shutdown_connections 2024-03-19T01:21:28.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.769+0000 7fe9343c0740 1 -- 172.21.15.69:0/1817713886 wait complete. 2024-03-19T01:21:28.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.769+0000 7fe9343c0740 20 client.27788 trim_cache size 0 max 16384 2024-03-19T01:21:28.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.806+0000 7fe9343c0740 1 Processor -- start 2024-03-19T01:21:28.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.806+0000 7fe9343c0740 1 -- start start 2024-03-19T01:21:28.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.807+0000 7fe9343c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47ee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.807+0000 7fe9343c0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 0x560be47eb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.807+0000 7fe9343c0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 0x560be47faab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.807+0000 7fe92a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 0x560be47eb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.807+0000 7fe92a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 0x560be47eb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58594/0 (socket says 172.21.15.69:58594) 2024-03-19T01:21:28.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.807+0000 7fe92a000640 1 -- 172.21.15.69:0/3131997832 learned_addr learned my addr 172.21.15.69:0/3131997832 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:28.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.807+0000 7fe9343c0740 1 -- 172.21.15.69:0/3131997832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1c6500 con 0x560be47e5a20 2024-03-19T01:21:28.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.807+0000 7fe9343c0740 1 -- 172.21.15.69:0/3131997832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1c8040 con 0x560be47e6340 2024-03-19T01:21:28.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.807+0000 7fe9343c0740 1 -- 172.21.15.69:0/3131997832 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1c6bd0 con 0x560be47eef20 2024-03-19T01:21:28.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.807+0000 7fe92b400640 1 --2- 172.21.15.69:0/3131997832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 0x560be47faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.808+0000 7fe92a000640 1 -- 172.21.15.69:0/3131997832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 msgr2=0x560be47faab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:28.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.808+0000 7fe92a000640 1 --2- 172.21.15.69:0/3131997832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 0x560be47faab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.808+0000 7fe92a000640 1 -- 172.21.15.69:0/3131997832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 msgr2=0x560be47ee6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:21:28.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.808+0000 7fe92a000640 1 --2- 172.21.15.69:0/3131997832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47ee6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.808+0000 7fe92a000640 1 -- 172.21.15.69:0/3131997832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be4802090 con 0x560be47e6340 2024-03-19T01:21:28.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.808+0000 7fe92a000640 1 --2- 172.21.15.69:0/3131997832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 0x560be47eb120 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe924009ca0 tx=0x7fe92403d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5d2fb4c05bd855 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.808+0000 7fe916a00640 1 -- 172.21.15.69:0/3131997832 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924039040 con 0x560be47e6340 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.809+0000 7fe916a00640 1 -- 172.21.15.69:0/3131997832 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe92404f070 con 0x560be47e6340 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.809+0000 7fe916a00640 1 -- 172.21.15.69:0/3131997832 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240347c0 con 0x560be47e6340 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.809+0000 7fe9343c0740 1 -- 172.21.15.69:0/3131997832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 msgr2=0x560be47eb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.809+0000 7fe9343c0740 1 --2- 172.21.15.69:0/3131997832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 0x560be47eb120 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe924009ca0 tx=0x7fe92403d000 comp rx=0 tx=0).stop 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.810+0000 7fe92b400640 1 --2- 172.21.15.69:0/3131997832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 0x560be47faab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.810+0000 7fe9343c0740 1 -- 172.21.15.69:0/3131997832 shutdown_connections 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.810+0000 7fe9343c0740 1 --2- 172.21.15.69:0/3131997832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 0x560be47faab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.810+0000 7fe9343c0740 1 --2- 172.21.15.69:0/3131997832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 0x560be47eb120 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.810+0000 7fe9343c0740 1 --2- 172.21.15.69:0/3131997832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47ee6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.810+0000 7fe9343c0740 1 -- 172.21.15.69:0/3131997832 >> 172.21.15.69:0/3131997832 conn(0x560be47e0150 msgr2=0x560be47de940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.810+0000 7fe9343c0740 1 -- 172.21.15.69:0/3131997832 shutdown_connections 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.810+0000 7fe9343c0740 1 -- 172.21.15.69:0/3131997832 wait complete. 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.811+0000 7fe9343c0740 1 -- start start 2024-03-19T01:21:28.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.811+0000 7fe9343c0740 1 Processor -- start 2024-03-19T01:21:28.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.812+0000 7fe9343c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.813+0000 7fe92aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.813+0000 7fe92aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51028/0 (socket says 172.21.15.69:51028) 2024-03-19T01:21:28.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.813+0000 7fe92aa00640 1 -- 172.21.15.69:0/2972184024 learned_addr learned my addr 172.21.15.69:0/2972184024 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:28.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.813+0000 7fe9343c0740 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 0x560be47f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.813+0000 7fe9343c0740 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 0x560be47f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.814+0000 7fe9343c0740 1 -- 172.21.15.69:0/2972184024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1c7970 con 0x560be47e5a20 2024-03-19T01:21:28.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.814+0000 7fe92b400640 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 0x560be47f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.814+0000 7fe9343c0740 1 -- 172.21.15.69:0/2972184024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1c72a0 con 0x560be47e6340 2024-03-19T01:21:28.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.814+0000 7fe9343c0740 1 -- 172.21.15.69:0/2972184024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1c44e0 con 0x560be47eef20 2024-03-19T01:21:28.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.815+0000 7fe92b400640 1 -- 172.21.15.69:0/2972184024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 msgr2=0x560be47f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:28.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.815+0000 7fe92a000640 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 0x560be47f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.815+0000 7fe92b400640 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 0x560be47f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.815+0000 7fe92b400640 1 -- 172.21.15.69:0/2972184024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 msgr2=0x560be47f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:28.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.815+0000 7fe92b400640 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.815+0000 7fe9343c0740 10 client.0 _ll_register_callbacks cb 0x7fe918005440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:28.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.815+0000 7fe92b400640 1 -- 172.21.15.69:0/2972184024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be47f3200 con 0x560be47eef20 2024-03-19T01:21:28.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.815+0000 7fe92aa00640 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:21:28.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.815+0000 7fe9343c0740 1 -- 172.21.15.69:0/2972184024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe92404f070 con 0x560be47eef20 2024-03-19T01:21:28.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.815+0000 7fe92b400640 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 0x560be47f2bc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe91809b2b0 tx=0x7fe918004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75e378ed7aa43e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.815+0000 7fe90be00640 10 client.27748 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:21:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.816+0000 7fe90be00640 1 -- 172.21.15.69:0/2972184024 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91801c070 con 0x560be47eef20 2024-03-19T01:21:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.816+0000 7fe90be00640 1 -- 172.21.15.69:0/2972184024 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fe91809a770 con 0x560be47eef20 2024-03-19T01:21:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.816+0000 7fe90be00640 1 -- 172.21.15.69:0/2972184024 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe91809a970 con 0x560be47eef20 2024-03-19T01:21:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.816+0000 7fe9343c0740 10 client.27748 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.816+0000 7fe9343c0740 1 -- 172.21.15.69:0/2972184024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=59}) -- 0x560be4865680 con 0x560be47eef20 2024-03-19T01:21:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.816+0000 7fe90be00640 1 -- 172.21.15.69:0/2972184024 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe91807e020 con 0x560be47eef20 2024-03-19T01:21:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.817+0000 7fe9343c0740 10 client.27748 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.817+0000 7fe9343c0740 10 client.27748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:21:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.817+0000 7fe9343c0740 20 client.27748 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:28.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.817+0000 7fe9343c0740 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560be4865c40 0x560be48680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:28.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.817+0000 7fe9343c0740 4 client.27748 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:21:28.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.817+0000 7fe9343c0740 1 -- 172.21.15.69:0/2972184024 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560be4751c10 con 0x560be4865c40 2024-03-19T01:21:28.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.817+0000 7fe92aa00640 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560be4865c40 0x560be48680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:28.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.818+0000 7fe92aa00640 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560be4865c40 0x560be48680c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=998b33959da51467 server_cookie=3ac2cef2ac36ead1 in_seq=0 out_seq=0 2024-03-19T01:21:28.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.821+0000 7fe90be00640 10 client.27748 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:28.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.821+0000 7fe90be00640 1 -- 172.21.15.69:0/2972184024 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560be4751c10 con 0x560be4865c40 2024-03-19T01:21:28.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.821+0000 7fe90be00640 10 client.27748 handle_command_reply: tid=0 2024-03-19T01:21:28.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.821+0000 7fe9343c0740 1 client.27748 shutdown 2024-03-19T01:21:28.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.822+0000 7fe9343c0740 1 -- 172.21.15.69:0/2972184024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 msgr2=0x560be47f2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.822+0000 7fe9343c0740 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 0x560be47f2bc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe91809b2b0 tx=0x7fe918004730 comp rx=0 tx=0).stop 2024-03-19T01:21:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.822+0000 7fe9343c0740 1 -- 172.21.15.69:0/2972184024 shutdown_connections 2024-03-19T01:21:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.822+0000 7fe9343c0740 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x560be4865c40 0x560be48680c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.822+0000 7fe9343c0740 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560be47eef20 0x560be47f2bc0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.822+0000 7fe9343c0740 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560be47e6340 0x560be47f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.822+0000 7fe9343c0740 1 --2- 172.21.15.69:0/2972184024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560be47e5a20 0x560be47f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.822+0000 7fe9343c0740 1 -- 172.21.15.69:0/2972184024 >> 172.21.15.69:0/2972184024 conn(0x560be47e0150 msgr2=0x560be47de940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.822+0000 7fe90be00640 0 client.27748 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.822+0000 7fe9343c0740 1 -- 172.21.15.69:0/2972184024 shutdown_connections 2024-03-19T01:21:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.822+0000 7fe9343c0740 1 -- 172.21.15.69:0/2972184024 wait complete. 2024-03-19T01:21:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.823+0000 7fe9343c0740 20 client.27748 trim_cache size 0 max 16384 2024-03-19T01:21:28.824 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:21:28.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 -- 172.21.15.69:0/3031438714 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9100a2a30 msgr2=0x7fe9100a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9100a2a30 0x7fe9100a4ef0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe924009bf0 tx=0x7fe92402e040 comp rx=0 tx=0).stop 2024-03-19T01:21:28.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 -- 172.21.15.69:0/3031438714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92c11b000 msgr2=0x7fe92c111bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:28.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92c11b000 0x7fe92c111bb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe91800e970 tx=0x7fe91800ee30 comp rx=0 tx=0).stop 2024-03-19T01:21:28.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 -- 172.21.15.69:0/3031438714 shutdown_connections 2024-03-19T01:21:28.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92c11b000 0x7fe92c111bb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe92c110040 0x7fe92c111670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe9100a2a30 0x7fe9100a4ef0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 --2- 172.21.15.69:0/3031438714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe92c10e070 0x7fe92c1185f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:28.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 -- 172.21.15.69:0/3031438714 >> 172.21.15.69:0/3031438714 conn(0x7fe92c06c900 msgr2=0x7fe92c0722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:28.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 -- 172.21.15.69:0/3031438714 shutdown_connections 2024-03-19T01:21:28.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:28.833+0000 7fe931600640 1 -- 172.21.15.69:0/3031438714 wait complete. 2024-03-19T01:21:28.944 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/bucket_locks.o 2024-03-19T01:21:29.061 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/char/nvram.o 2024-03-19T01:21:29.190 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/tty_baudrate.o 2024-03-19T01:21:29.198 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nspredef.o 2024-03-19T01:21:29.320 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/display/drm_scdc_helper.o 2024-03-19T01:21:29.381 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4file.o 2024-03-19T01:21:29.610 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/stacktrace.o 2024-03-19T01:21:29.813 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsprepkg.o 2024-03-19T01:21:29.892 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:21:29.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:21:29.892 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:21:29.908 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/generic-radix-tree.o 2024-03-19T01:21:30.281 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/tty_jobctrl.o 2024-03-19T01:21:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:30 smithi069 ceph-mon[31727]: pgmap v337: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-03-19T01:21:30.410 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/char/built-in.a 2024-03-19T01:21:30.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:30 smithi086 ceph-mon[66829]: pgmap v337: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-03-19T01:21:30.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.510+0000 7fb201400640 1 -- 172.21.15.69:0/1564014172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc1005f0 msgr2=0x7fb1fc1009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:30.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.510+0000 7fb201400640 1 --2- 172.21.15.69:0/1564014172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc1005f0 0x7fb1fc1009d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009cd0 tx=0x7fb1f402c180 comp rx=0 tx=0).stop 2024-03-19T01:21:30.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.513+0000 7fb201400640 1 -- 172.21.15.69:0/1564014172 shutdown_connections 2024-03-19T01:21:30.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.513+0000 7fb201400640 1 --2- 172.21.15.69:0/1564014172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1fc1059f0 0x7fb1fc105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.513+0000 7fb201400640 1 --2- 172.21.15.69:0/1564014172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc100f10 0x7fb1fc1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.513+0000 7fb201400640 1 --2- 172.21.15.69:0/1564014172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc1005f0 0x7fb1fc1009d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.513+0000 7fb201400640 1 -- 172.21.15.69:0/1564014172 >> 172.21.15.69:0/1564014172 conn(0x7fb1fc0772d0 msgr2=0x7fb1fc0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:30.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.518+0000 7fb201400640 1 -- 172.21.15.69:0/1564014172 shutdown_connections 2024-03-19T01:21:30.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.518+0000 7fb201400640 1 -- 172.21.15.69:0/1564014172 wait complete. 2024-03-19T01:21:30.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.525+0000 7fb201400640 1 Processor -- start 2024-03-19T01:21:30.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.525+0000 7fb201400640 1 -- start start 2024-03-19T01:21:30.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.525+0000 7fb201400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc1005f0 0x7fb1fc1148a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.525+0000 7fb201400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc100f10 0x7fb1fc10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.525+0000 7fb201400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1fc1059f0 0x7fb1fc10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.525+0000 7fb201400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc117fc0 con 0x7fb1fc1059f0 2024-03-19T01:21:30.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.525+0000 7fb201400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc117e40 con 0x7fb1fc1005f0 2024-03-19T01:21:30.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.525+0000 7fb201400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc118140 con 0x7fb1fc100f10 2024-03-19T01:21:30.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.526+0000 7fb1fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc100f10 0x7fb1fc10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.526+0000 7fb1fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc100f10 0x7fb1fc10d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60890/0 (socket says 172.21.15.69:60890) 2024-03-19T01:21:30.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.526+0000 7fb1fb400640 1 -- 172.21.15.69:0/1183714929 learned_addr learned my addr 172.21.15.69:0/1183714929 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:30.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.526+0000 7fb200a00640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1fc1059f0 0x7fb1fc10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.527+0000 7fb1fb400640 1 -- 172.21.15.69:0/1183714929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc1005f0 msgr2=0x7fb1fc1148a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:30.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.527+0000 7fb1fb400640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc1005f0 0x7fb1fc1148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.527+0000 7fb1fb400640 1 -- 172.21.15.69:0/1183714929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1fc1059f0 msgr2=0x7fb1fc10de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:30.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.527+0000 7fb1fb400640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1fc1059f0 0x7fb1fc10de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.527+0000 7fb1fb400640 1 -- 172.21.15.69:0/1183714929 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc10e500 con 0x7fb1fc100f10 2024-03-19T01:21:30.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.527+0000 7fb200a00640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1fc1059f0 0x7fb1fc10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:21:30.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.529+0000 7fb1fb400640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc100f10 0x7fb1fc10d920 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00e9f0 tx=0x7fb1ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:30.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.530+0000 7fb1f8c00640 1 -- 172.21.15.69:0/1183714929 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec00cd80 con 0x7fb1fc100f10 2024-03-19T01:21:30.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.530+0000 7fb1f8c00640 1 -- 172.21.15.69:0/1183714929 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1ec010910 con 0x7fb1fc100f10 2024-03-19T01:21:30.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.530+0000 7fb201400640 1 -- 172.21.15.69:0/1183714929 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1fc1123c0 con 0x7fb1fc100f10 2024-03-19T01:21:30.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.530+0000 7fb1f8c00640 1 -- 172.21.15.69:0/1183714929 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec0189a0 con 0x7fb1fc100f10 2024-03-19T01:21:30.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.530+0000 7fb201400640 1 -- 172.21.15.69:0/1183714929 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1fc112960 con 0x7fb1fc100f10 2024-03-19T01:21:30.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.540+0000 7fb205813740 1 Processor -- start 2024-03-19T01:21:30.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.540+0000 7fb1f8c00640 1 -- 172.21.15.69:0/1183714929 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fb1ec010ac0 con 0x7fb1fc100f10 2024-03-19T01:21:30.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.543+0000 7fb205813740 1 -- start start 2024-03-19T01:21:30.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.544+0000 7fb205813740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 0x5603c782d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.544+0000 7fb205813740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7835cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.545+0000 7fb205813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 0x5603c7838620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.545+0000 7fb205813740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5603c778ffb0 con 0x5603c782d650 2024-03-19T01:21:30.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.545+0000 7fb205813740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5603c7738ab0 con 0x5603c7836230 2024-03-19T01:21:30.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.545+0000 7fb205813740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5603c778c050 con 0x5603c782cd30 2024-03-19T01:21:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.545+0000 7fb1fbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7835cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.545+0000 7fb1fbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7835cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51058/0 (socket says 172.21.15.69:51058) 2024-03-19T01:21:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.545+0000 7fb1fbe00640 1 -- 172.21.15.69:0/2473650574 learned_addr learned my addr 172.21.15.69:0/2473650574 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.546+0000 7fb200a00640 1 --2- 172.21.15.69:0/2473650574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 0x5603c782d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.546+0000 7fb1fbe00640 1 -- 172.21.15.69:0/2473650574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 msgr2=0x5603c782d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.546+0000 7fb1fbe00640 1 --2- 172.21.15.69:0/2473650574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 0x5603c782d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.546+0000 7fb1fbe00640 1 -- 172.21.15.69:0/2473650574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 msgr2=0x5603c7838620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:21:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.546+0000 7fb1fbe00640 1 --2- 172.21.15.69:0/2473650574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 0x5603c7838620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.546+0000 7fb1fbe00640 1 -- 172.21.15.69:0/2473650574 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c7756e00 con 0x5603c782d650 2024-03-19T01:21:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.547+0000 7fb1f8c00640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb1e00a2a30 0x7fb1e00a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.548+0000 7fb1fbe00640 1 --2- 172.21.15.69:0/2473650574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7835cf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb1f402c710 tx=0x7fb1f4009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69178b336d48d71 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.548+0000 7fb200a00640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb1e00a2a30 0x7fb1e00a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.549+0000 7fb1f1200640 1 -- 172.21.15.69:0/2473650574 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4041020 con 0x5603c782d650 2024-03-19T01:21:30.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.549+0000 7fb1f1200640 1 -- 172.21.15.69:0/2473650574 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1f4003e50 con 0x5603c782d650 2024-03-19T01:21:30.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.549+0000 7fb1f1200640 1 -- 172.21.15.69:0/2473650574 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4039040 con 0x5603c782d650 2024-03-19T01:21:30.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.550+0000 7fb205813740 1 -- 172.21.15.69:0/2473650574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 msgr2=0x5603c7835cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:30.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.550+0000 7fb205813740 1 --2- 172.21.15.69:0/2473650574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7835cf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb1f402c710 tx=0x7fb1f4009890 comp rx=0 tx=0).stop 2024-03-19T01:21:30.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.551+0000 7fb1f8c00640 1 -- 172.21.15.69:0/1183714929 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fb1ec0d3060 con 0x7fb1fc100f10 2024-03-19T01:21:30.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.552+0000 7fb200a00640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb1e00a2a30 0x7fb1e00a4ef0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8005fd0 tx=0x7fb1e8005cc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:30.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.554+0000 7fb205813740 1 -- 172.21.15.69:0/2473650574 shutdown_connections 2024-03-19T01:21:30.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.554+0000 7fb205813740 1 --2- 172.21.15.69:0/2473650574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 0x5603c7838620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.554+0000 7fb205813740 1 --2- 172.21.15.69:0/2473650574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7835cf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.554+0000 7fb205813740 1 --2- 172.21.15.69:0/2473650574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 0x5603c782d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.554+0000 7fb205813740 1 -- 172.21.15.69:0/2473650574 >> 172.21.15.69:0/2473650574 conn(0x5603c7827460 msgr2=0x5603c7827840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.559+0000 7fb205813740 1 -- 172.21.15.69:0/2473650574 shutdown_connections 2024-03-19T01:21:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.560+0000 7fb205813740 1 -- 172.21.15.69:0/2473650574 wait complete. 2024-03-19T01:21:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.560+0000 7fb205813740 1 -- start start 2024-03-19T01:21:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.560+0000 7fb205813740 1 Processor -- start 2024-03-19T01:21:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:30 smithi163 ceph-mon[67428]: pgmap v337: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-03-19T01:21:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.564+0000 7fb205813740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 0x5603c78332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.565+0000 7fb205813740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7833830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.565+0000 7fb205813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 0x5603c7833d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.565+0000 7fb205813740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5603c7838fc0 con 0x5603c782d650 2024-03-19T01:21:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.565+0000 7fb205813740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5603c774bf00 con 0x5603c7836230 2024-03-19T01:21:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.565+0000 7fb205813740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5603c7839140 con 0x5603c782cd30 2024-03-19T01:21:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.565+0000 7fb205813740 10 client.0 _ll_register_callbacks cb 0x5603c774b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.566+0000 7fb1fbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 0x5603c78332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.566+0000 7fb1fbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 0x5603c78332f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60918/0 (socket says 172.21.15.69:60918) 2024-03-19T01:21:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.566+0000 7fb1fbe00640 1 -- 172.21.15.69:0/1940543558 learned_addr learned my addr 172.21.15.69:0/1940543558 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.566+0000 7fb200a00640 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 0x5603c7833d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.566+0000 7fb1fb400640 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7833830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.566+0000 7fb1fbe00640 1 -- 172.21.15.69:0/1940543558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 msgr2=0x5603c7833d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.566+0000 7fb1fbe00640 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 0x5603c7833d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.566+0000 7fb1fbe00640 1 -- 172.21.15.69:0/1940543558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 msgr2=0x5603c7833830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.566+0000 7fb1fbe00640 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7833830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.567+0000 7fb1fbe00640 1 -- 172.21.15.69:0/1940543558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c756c7e0 con 0x5603c782cd30 2024-03-19T01:21:30.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.567+0000 7fb1fbe00640 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 0x5603c78332f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4000f80 tx=0x7fb1f4009a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab9f64892c87e7d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:30.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.567+0000 7fb1e5600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:21:30.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.567+0000 7fb1e5600640 1 -- 172.21.15.69:0/1940543558 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4041030 con 0x5603c782cd30 2024-03-19T01:21:30.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.567+0000 7fb1e5600640 1 -- 172.21.15.69:0/1940543558 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fb1f4003970 con 0x5603c782cd30 2024-03-19T01:21:30.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.568+0000 7fb205813740 1 -- 172.21.15.69:0/1940543558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f4003e50 con 0x5603c782cd30 2024-03-19T01:21:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.568+0000 7fb1e5600640 1 -- 172.21.15.69:0/1940543558 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb1f4039050 con 0x5603c782cd30 2024-03-19T01:21:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.569+0000 7fb205813740 10 client.27760 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.569+0000 7fb205813740 1 -- 172.21.15.69:0/1940543558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5603c75ccf70 con 0x5603c782cd30 2024-03-19T01:21:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.569+0000 7fb1e5600640 1 -- 172.21.15.69:0/1940543558 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1f4003e50 con 0x5603c782cd30 2024-03-19T01:21:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.569+0000 7fb1e5600640 1 -- 172.21.15.69:0/1940543558 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fb1f403c030 con 0x5603c782cd30 2024-03-19T01:21:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.570+0000 7fb205813740 10 client.27760 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.570+0000 7fb205813740 10 client.27760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:21:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.570+0000 7fb205813740 20 client.27760 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.570+0000 7fb205813740 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5603c7857480 0x5603c7859870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.570+0000 7fb205813740 4 client.27760 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:21:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.570+0000 7fb205813740 1 -- 172.21.15.69:0/1940543558 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603c779d4f0 con 0x5603c7857480 2024-03-19T01:21:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.570+0000 7fb1fb400640 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5603c7857480 0x5603c7859870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.571+0000 7fb1fb400640 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5603c7857480 0x5603c7859870 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f90f0d753240d613 server_cookie=a8b1a4bdaaa95181 in_seq=0 out_seq=0 2024-03-19T01:21:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.571+0000 7fb1e5600640 10 client.27760 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.571+0000 7fb1e5600640 1 -- 172.21.15.69:0/1940543558 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5603c779d4f0 con 0x5603c7857480 2024-03-19T01:21:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.572+0000 7fb1e5600640 10 client.27760 handle_command_reply: tid=0 2024-03-19T01:21:30.573 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/intel/pasid.o 2024-03-19T01:21:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.573+0000 7fb205813740 1 client.27760 shutdown 2024-03-19T01:21:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.573+0000 7fb205813740 1 -- 172.21.15.69:0/1940543558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 msgr2=0x5603c78332f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.573+0000 7fb205813740 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 0x5603c78332f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4000f80 tx=0x7fb1f4009a00 comp rx=0 tx=0).stop 2024-03-19T01:21:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.573+0000 7fb205813740 1 -- 172.21.15.69:0/1940543558 shutdown_connections 2024-03-19T01:21:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.573+0000 7fb205813740 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5603c7857480 0x5603c7859870 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.573+0000 7fb205813740 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 0x5603c7833d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.573+0000 7fb205813740 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7833830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.573+0000 7fb205813740 1 --2- 172.21.15.69:0/1940543558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782cd30 0x5603c78332f0 secure :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4000f80 tx=0x7fb1f4009a00 comp rx=0 tx=0).stop 2024-03-19T01:21:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.573+0000 7fb205813740 1 -- 172.21.15.69:0/1940543558 >> 172.21.15.69:0/1940543558 conn(0x5603c7827460 msgr2=0x5603c78291b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.573+0000 7fb1e5600640 0 client.27760 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.574+0000 7fb205813740 1 -- 172.21.15.69:0/1940543558 shutdown_connections 2024-03-19T01:21:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.574+0000 7fb205813740 1 -- 172.21.15.69:0/1940543558 wait complete. 2024-03-19T01:21:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.574+0000 7fb205813740 20 client.27760 trim_cache size 0 max 16384 2024-03-19T01:21:30.580 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/dma.o 2024-03-19T01:21:30.591 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsrepair.o 2024-03-19T01:21:30.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.625+0000 7fb205813740 1 Processor -- start 2024-03-19T01:21:30.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.625+0000 7fb205813740 1 -- start start 2024-03-19T01:21:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.625+0000 7fb205813740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7835a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.625+0000 7fb205813740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c7836230 0x5603c7833bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.625+0000 7fb205813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7841aa0 0x5603c785c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.625+0000 7fb205813740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc1b5470 con 0x5603c782d650 2024-03-19T01:21:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.625+0000 7fb205813740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc1b6fb0 con 0x5603c7841aa0 2024-03-19T01:21:30.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.626+0000 7fb205813740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc1b5b40 con 0x5603c7836230 2024-03-19T01:21:30.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.626+0000 7fb200a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7841aa0 0x5603c785c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.626+0000 7fb1fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c7836230 0x5603c7833bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.626+0000 7fb1fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c7836230 0x5603c7833bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60934/0 (socket says 172.21.15.69:60934) 2024-03-19T01:21:30.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.626+0000 7fb1fb400640 1 -- 172.21.15.69:0/1712857959 learned_addr learned my addr 172.21.15.69:0/1712857959 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:30.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.627+0000 7fb1fbe00640 1 --2- 172.21.15.69:0/1712857959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7835a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.627+0000 7fb200a00640 1 -- 172.21.15.69:0/1712857959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c7836230 msgr2=0x5603c7833bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:30.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.627+0000 7fb200a00640 1 --2- 172.21.15.69:0/1712857959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c7836230 0x5603c7833bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.627+0000 7fb200a00640 1 -- 172.21.15.69:0/1712857959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 msgr2=0x5603c7835a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:30.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.631+0000 7fb200a00640 1 --2- 172.21.15.69:0/1712857959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7835a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.631+0000 7fb200a00640 1 -- 172.21.15.69:0/1712857959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c7464170 con 0x5603c7841aa0 2024-03-19T01:21:30.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.631+0000 7fb1fb400640 1 --2- 172.21.15.69:0/1712857959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c7836230 0x5603c7833bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:21:30.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.631+0000 7fb200a00640 1 --2- 172.21.15.69:0/1712857959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7841aa0 0x5603c785c4b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x5603c78188b0 tx=0x7fb1e8019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66c35ef9ae14d246 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:30.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.631+0000 7fb1fbe00640 1 --2- 172.21.15.69:0/1712857959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7835a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:30.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.632+0000 7fb1e7e00640 1 -- 172.21.15.69:0/1712857959 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e801c600 con 0x5603c7841aa0 2024-03-19T01:21:30.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.632+0000 7fb1e7e00640 1 -- 172.21.15.69:0/1712857959 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1e801cc80 con 0x5603c7841aa0 2024-03-19T01:21:30.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.632+0000 7fb1e7e00640 1 -- 172.21.15.69:0/1712857959 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8027d50 con 0x5603c7841aa0 2024-03-19T01:21:30.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.634+0000 7fb205813740 1 -- 172.21.15.69:0/1712857959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7841aa0 msgr2=0x5603c785c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:30.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.634+0000 7fb205813740 1 --2- 172.21.15.69:0/1712857959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7841aa0 0x5603c785c4b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x5603c78188b0 tx=0x7fb1e8019000 comp rx=0 tx=0).stop 2024-03-19T01:21:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.646+0000 7fb205813740 1 -- 172.21.15.69:0/1712857959 shutdown_connections 2024-03-19T01:21:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.646+0000 7fb205813740 1 --2- 172.21.15.69:0/1712857959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7841aa0 0x5603c785c4b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.646+0000 7fb205813740 1 --2- 172.21.15.69:0/1712857959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c7836230 0x5603c7833bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.646+0000 7fb205813740 1 --2- 172.21.15.69:0/1712857959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c782d650 0x5603c7835a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.646+0000 7fb205813740 1 -- 172.21.15.69:0/1712857959 >> 172.21.15.69:0/1712857959 conn(0x5603c7827460 msgr2=0x5603c7841390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:30.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.653+0000 7fb205813740 1 -- 172.21.15.69:0/1712857959 shutdown_connections 2024-03-19T01:21:30.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.653+0000 7fb205813740 1 -- 172.21.15.69:0/1712857959 wait complete. 2024-03-19T01:21:30.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.653+0000 7fb205813740 1 -- start start 2024-03-19T01:21:30.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.654+0000 7fb205813740 1 Processor -- start 2024-03-19T01:21:30.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.654+0000 7fb205813740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782d650 0x5603c785a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.654+0000 7fb205813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 0x5603c785a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.657+0000 7fb1fbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782d650 0x5603c785a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.657+0000 7fb1fbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782d650 0x5603c785a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60948/0 (socket says 172.21.15.69:60948) 2024-03-19T01:21:30.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.657+0000 7fb205813740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c7841aa0 0x5603c78bc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.658+0000 7fb1fbe00640 1 -- 172.21.15.69:0/1220209313 learned_addr learned my addr 172.21.15.69:0/1220209313 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.658+0000 7fb205813740 1 -- 172.21.15.69:0/1220209313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc1b68e0 con 0x5603c7841aa0 2024-03-19T01:21:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.658+0000 7fb205813740 1 -- 172.21.15.69:0/1220209313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc1b6210 con 0x5603c7836230 2024-03-19T01:21:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.658+0000 7fb205813740 1 -- 172.21.15.69:0/1220209313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc068530 con 0x5603c782d650 2024-03-19T01:21:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.658+0000 7fb205813740 10 client.0 _ll_register_callbacks cb 0x7fb1f4009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.658+0000 7fb1fb400640 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 0x5603c785a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.658+0000 7fb1fbe00640 1 -- 172.21.15.69:0/1220209313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 msgr2=0x5603c785a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.658+0000 7fb1fbe00640 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 0x5603c785a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.658+0000 7fb1fbe00640 1 -- 172.21.15.69:0/1220209313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c7841aa0 msgr2=0x5603c78bc420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:21:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.659+0000 7fb200a00640 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c7841aa0 0x5603c78bc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.659+0000 7fb1fbe00640 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c7841aa0 0x5603c78bc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.659+0000 7fb1fbe00640 1 -- 172.21.15.69:0/1220209313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c78bc960 con 0x5603c782d650 2024-03-19T01:21:30.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.659+0000 7fb200a00640 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c7841aa0 0x5603c78bc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:30.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.659+0000 7fb1fbe00640 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782d650 0x5603c785a450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009430 tx=0x7fb1f4005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd08aef1960b370c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:30.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.660+0000 7fb1e5600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:21:30.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.660+0000 7fb1e5600640 1 -- 172.21.15.69:0/1220209313 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f405c060 con 0x5603c782d650 2024-03-19T01:21:30.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.660+0000 7fb1e5600640 1 -- 172.21.15.69:0/1220209313 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fb1f4034b90 con 0x5603c782d650 2024-03-19T01:21:30.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.663+0000 7fb205813740 1 -- 172.21.15.69:0/1220209313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e801cc80 con 0x5603c782d650 2024-03-19T01:21:30.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.663+0000 7fb1e5600640 1 -- 172.21.15.69:0/1220209313 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb1f4039050 con 0x5603c782d650 2024-03-19T01:21:30.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.663+0000 7fb205813740 10 client.27766 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:30.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.663+0000 7fb205813740 1 -- 172.21.15.69:0/1220209313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5603c78bdaf0 con 0x5603c782d650 2024-03-19T01:21:30.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.666+0000 7fb1e5600640 1 -- 172.21.15.69:0/1220209313 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1e801cc80 con 0x5603c782d650 2024-03-19T01:21:30.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.666+0000 7fb1e5600640 1 -- 172.21.15.69:0/1220209313 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fb1f403c040 con 0x5603c782d650 2024-03-19T01:21:30.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.666+0000 7fb205813740 10 client.27766 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:30.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.666+0000 7fb205813740 10 client.27766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:21:30.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.666+0000 7fb205813740 20 client.27766 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:30.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.666+0000 7fb205813740 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5603c78be0e0 0x5603c78c0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:30.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.666+0000 7fb205813740 4 client.27766 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:21:30.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.666+0000 7fb205813740 1 -- 172.21.15.69:0/1220209313 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5603c7797840 con 0x5603c78be0e0 2024-03-19T01:21:30.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.666+0000 7fb1fb400640 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5603c78be0e0 0x5603c78c0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:30.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.667+0000 7fb1fb400640 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5603c78be0e0 0x5603c78c0510 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c183816dbb41c923 server_cookie=afe5621fb9873d38 in_seq=0 out_seq=0 2024-03-19T01:21:30.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.667+0000 7fb1e5600640 10 client.27766 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:30.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.668+0000 7fb1e5600640 1 -- 172.21.15.69:0/1220209313 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5603c7797840 con 0x5603c78be0e0 2024-03-19T01:21:30.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.668+0000 7fb1e5600640 10 client.27766 handle_command_reply: tid=0 2024-03-19T01:21:30.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.668+0000 7fb205813740 1 client.27766 shutdown 2024-03-19T01:21:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.668+0000 7fb205813740 1 -- 172.21.15.69:0/1220209313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782d650 msgr2=0x5603c785a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.668+0000 7fb205813740 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782d650 0x5603c785a450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009430 tx=0x7fb1f4005cc0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.669+0000 7fb205813740 1 -- 172.21.15.69:0/1220209313 shutdown_connections 2024-03-19T01:21:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.669+0000 7fb205813740 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5603c78be0e0 0x5603c78c0510 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.669+0000 7fb205813740 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603c7841aa0 0x5603c78bc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.669+0000 7fb1e5600640 0 client.27766 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.670+0000 7fb205813740 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603c7836230 0x5603c785a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.670+0000 7fb205813740 1 --2- 172.21.15.69:0/1220209313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603c782d650 0x5603c785a450 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.670+0000 7fb205813740 1 -- 172.21.15.69:0/1220209313 >> 172.21.15.69:0/1220209313 conn(0x5603c7827460 msgr2=0x5603c7831a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.670+0000 7fb205813740 1 -- 172.21.15.69:0/1220209313 shutdown_connections 2024-03-19T01:21:30.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.670+0000 7fb205813740 1 -- 172.21.15.69:0/1220209313 wait complete. 2024-03-19T01:21:30.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.670+0000 7fb205813740 20 client.27766 trim_cache size 0 max 16384 2024-03-19T01:21:30.674 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:21:30.675 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:21:30.675 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "fd79c60c-0ee4-4850-8dc0-50db38a35962", 2024-03-19T01:21:30.675 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:21:30.675 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:21:30.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.675+0000 7fb201400640 1 -- 172.21.15.69:0/1183714929 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb1e00a2a30 msgr2=0x7fb1e00a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:30.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.675+0000 7fb201400640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb1e00a2a30 0x7fb1e00a4ef0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8005fd0 tx=0x7fb1e8005cc0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.675+0000 7fb201400640 1 -- 172.21.15.69:0/1183714929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc100f10 msgr2=0x7fb1fc10d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:30.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.675+0000 7fb201400640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc100f10 0x7fb1fc10d920 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00e9f0 tx=0x7fb1ec00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.677+0000 7fb201400640 1 -- 172.21.15.69:0/1183714929 shutdown_connections 2024-03-19T01:21:30.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.677+0000 7fb201400640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1fc1059f0 0x7fb1fc10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.677+0000 7fb201400640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1fc100f10 0x7fb1fc10d920 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.677+0000 7fb201400640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb1e00a2a30 0x7fb1e00a4ef0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.677+0000 7fb201400640 1 --2- 172.21.15.69:0/1183714929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1fc1005f0 0x7fb1fc1148a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:30.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.677+0000 7fb201400640 1 -- 172.21.15.69:0/1183714929 >> 172.21.15.69:0/1183714929 conn(0x7fb1fc0772d0 msgr2=0x7fb1fc1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:30.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.679+0000 7fb201400640 1 -- 172.21.15.69:0/1183714929 shutdown_connections 2024-03-19T01:21:30.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:30.679+0000 7fb201400640 1 -- 172.21.15.69:0/1183714929 wait complete. 2024-03-19T01:21:30.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:21:30.724 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:21:30.734 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/display/built-in.a 2024-03-19T01:21:30.857 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/string_helpers.o 2024-03-19T01:21:30.892 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_tt.o 2024-03-19T01:21:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.433+0000 7f2e10800640 1 -- 172.21.15.69:0/2275907866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e0810c440 msgr2=0x7f2e0810c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.433+0000 7f2e10800640 1 --2- 172.21.15.69:0/2275907866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e0810c440 0x7f2e0810c820 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009860 tx=0x7f2e0402bb90 comp rx=0 tx=0).stop 2024-03-19T01:21:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.434+0000 7f2e10800640 1 -- 172.21.15.69:0/2275907866 shutdown_connections 2024-03-19T01:21:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.434+0000 7f2e10800640 1 --2- 172.21.15.69:0/2275907866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e08111500 0x7f2e081190d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.434+0000 7f2e10800640 1 --2- 172.21.15.69:0/2275907866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0810cdf0 0x7f2e08110e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.434+0000 7f2e10800640 1 --2- 172.21.15.69:0/2275907866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e0810c440 0x7f2e0810c820 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.434+0000 7f2e10800640 1 -- 172.21.15.69:0/2275907866 >> 172.21.15.69:0/2275907866 conn(0x7f2e0806c820 msgr2=0x7f2e0810a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:31.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.438+0000 7f2e10800640 1 -- 172.21.15.69:0/2275907866 shutdown_connections 2024-03-19T01:21:31.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.438+0000 7f2e10800640 1 -- 172.21.15.69:0/2275907866 wait complete. 2024-03-19T01:21:31.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.441+0000 7f2e10800640 1 Processor -- start 2024-03-19T01:21:31.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.442+0000 7f2e10800640 1 -- start start 2024-03-19T01:21:31.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.442+0000 7f2e10800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0810c440 0x7f2e08119020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.442+0000 7f2e10800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e08111500 0x7f2e08116110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.444+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0810c440 0x7f2e08119020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.444+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0810c440 0x7f2e08119020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51106/0 (socket says 172.21.15.69:51106) 2024-03-19T01:21:31.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.444+0000 7f2e0d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e08111500 0x7f2e08116110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.444+0000 7f2e0d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e08111500 0x7f2e08116110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60976/0 (socket says 172.21.15.69:60976) 2024-03-19T01:21:31.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.444+0000 7f2e10800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e08116650 0x7f2e08116ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.444+0000 7f2e0d400640 1 -- 172.21.15.69:0/3954609562 learned_addr learned my addr 172.21.15.69:0/3954609562 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:31.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.444+0000 7f2e10800640 1 -- 172.21.15.69:0/3954609562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0811c610 con 0x7f2e0810c440 2024-03-19T01:21:31.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.444+0000 7f2e10800640 1 -- 172.21.15.69:0/3954609562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0811c490 con 0x7f2e08116650 2024-03-19T01:21:31.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.444+0000 7f2e10800640 1 -- 172.21.15.69:0/3954609562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0811c790 con 0x7f2e08111500 2024-03-19T01:21:31.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.445+0000 7f2e0e800640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e08116650 0x7f2e08116ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.445+0000 7f2e0d400640 1 -- 172.21.15.69:0/3954609562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e08116650 msgr2=0x7f2e08116ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:31.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.445+0000 7f2e0d400640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e08116650 0x7f2e08116ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.445+0000 7f2e0d400640 1 -- 172.21.15.69:0/3954609562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0810c440 msgr2=0x7f2e08119020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:31.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.445+0000 7f2e0d400640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0810c440 0x7f2e08119020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.445+0000 7f2e0d400640 1 -- 172.21.15.69:0/3954609562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e081b35e0 con 0x7f2e08111500 2024-03-19T01:21:31.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.445+0000 7f2e0e800640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e08116650 0x7f2e08116ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:21:31.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.446+0000 7f2e0d400640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e08111500 0x7f2e08116110 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000dbf0 tx=0x7f2e0000b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:31.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.447+0000 7f2dfea00640 1 -- 172.21.15.69:0/3954609562 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00017070 con 0x7f2e08111500 2024-03-19T01:21:31.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.447+0000 7f2dfea00640 1 -- 172.21.15.69:0/3954609562 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e00003e50 con 0x7f2e08111500 2024-03-19T01:21:31.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.447+0000 7f2dfea00640 1 -- 172.21.15.69:0/3954609562 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000106c0 con 0x7f2e08111500 2024-03-19T01:21:31.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.447+0000 7f2e10800640 1 -- 172.21.15.69:0/3954609562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e081b38d0 con 0x7f2e08111500 2024-03-19T01:21:31.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.447+0000 7f2e10800640 1 -- 172.21.15.69:0/3954609562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e081b3dd0 con 0x7f2e08111500 2024-03-19T01:21:31.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.462+0000 7f2e1361a740 1 Processor -- start 2024-03-19T01:21:31.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.462+0000 7f2dfea00640 1 -- 172.21.15.69:0/3954609562 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f2e00010860 con 0x7f2e08111500 2024-03-19T01:21:31.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.465+0000 7f2e1361a740 1 -- start start 2024-03-19T01:21:31.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.466+0000 7f2e1361a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962ed40 0x55d1d962f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.466+0000 7f2e1361a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962f660 0x55d1d9637d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.466+0000 7f2e1361a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d9638240 0x55d1d963a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.466+0000 7f2e1361a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d1d95932c0 con 0x55d1d9638240 2024-03-19T01:21:31.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.466+0000 7f2e1361a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d1d953bb60 con 0x55d1d962f660 2024-03-19T01:21:31.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.466+0000 7f2e1361a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d1d958f360 con 0x55d1d962ed40 2024-03-19T01:21:31.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.468+0000 7f2dfea00640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2df40a2ab0 0x7f2df40a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.468+0000 7f2e0e800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962ed40 0x55d1d962f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.468+0000 7f2e0e800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962ed40 0x55d1d962f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:60988/0 (socket says 172.21.15.69:60988) 2024-03-19T01:21:31.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.468+0000 7f2e0e800640 1 -- 172.21.15.69:0/3799078678 learned_addr learned my addr 172.21.15.69:0/3799078678 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:31.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.469+0000 7f2dfea00640 1 -- 172.21.15.69:0/3954609562 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7f2e000cefe0 con 0x7f2e08111500 2024-03-19T01:21:31.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.469+0000 7f2e0e800640 1 -- 172.21.15.69:0/3799078678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962f660 msgr2=0x55d1d9637d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:21:31.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.469+0000 7f2e0e800640 1 --2- 172.21.15.69:0/3799078678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962f660 0x55d1d9637d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.469+0000 7f2e0e800640 1 -- 172.21.15.69:0/3799078678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d9638240 msgr2=0x55d1d963a630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:21:31.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.469+0000 7f2e0e800640 1 --2- 172.21.15.69:0/3799078678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d9638240 0x55d1d963a630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.469+0000 7f2e0e800640 1 -- 172.21.15.69:0/3799078678 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d93717d0 con 0x55d1d962ed40 2024-03-19T01:21:31.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.470+0000 7f2e0e800640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2df40a2ab0 0x7f2df40a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.470+0000 7f2e0e800640 1 --2- 172.21.15.69:0/3799078678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962ed40 0x55d1d962f120 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009d00 tx=0x7f2df8012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83b7bc8318d3238 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:31.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.473+0000 7f2df3000640 1 -- 172.21.15.69:0/3799078678 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df801c5b0 con 0x55d1d962ed40 2024-03-19T01:21:31.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.473+0000 7f2df3000640 1 -- 172.21.15.69:0/3799078678 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2df801cc30 con 0x55d1d962ed40 2024-03-19T01:21:31.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.473+0000 7f2df3000640 1 -- 172.21.15.69:0/3799078678 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df800ead0 con 0x55d1d962ed40 2024-03-19T01:21:31.475 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/delegation.o 2024-03-19T01:21:31.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.477+0000 7f2e1361a740 1 -- 172.21.15.69:0/3799078678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962ed40 msgr2=0x55d1d962f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:31.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.477+0000 7f2e1361a740 1 --2- 172.21.15.69:0/3799078678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962ed40 0x55d1d962f120 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009d00 tx=0x7f2df8012820 comp rx=0 tx=0).stop 2024-03-19T01:21:31.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.475+0000 7f2e0e800640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2df40a2ab0 0x7f2df40a4f70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2df8020000 tx=0x7f2df801d000 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:31.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.478+0000 7f2e1361a740 1 -- 172.21.15.69:0/3799078678 shutdown_connections 2024-03-19T01:21:31.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.478+0000 7f2e1361a740 1 --2- 172.21.15.69:0/3799078678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d9638240 0x55d1d963a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.478+0000 7f2e1361a740 1 --2- 172.21.15.69:0/3799078678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962f660 0x55d1d9637d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.478+0000 7f2e1361a740 1 --2- 172.21.15.69:0/3799078678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962ed40 0x55d1d962f120 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.478+0000 7f2e1361a740 1 -- 172.21.15.69:0/3799078678 >> 172.21.15.69:0/3799078678 conn(0x55d1d9629470 msgr2=0x55d1d9629850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:31.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.479+0000 7f2e1361a740 1 -- 172.21.15.69:0/3799078678 shutdown_connections 2024-03-19T01:21:31.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.480+0000 7f2e1361a740 1 -- 172.21.15.69:0/3799078678 wait complete. 2024-03-19T01:21:31.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.483+0000 7f2e1361a740 1 -- start start 2024-03-19T01:21:31.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.485+0000 7f2e1361a740 1 Processor -- start 2024-03-19T01:21:31.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.486+0000 7f2e1361a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d9635260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.488+0000 7f2e1361a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 0x55d1d96357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.488+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d9635260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.488+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d9635260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58658/0 (socket says 172.21.15.69:58658) 2024-03-19T01:21:31.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.488+0000 7f2e0de00640 1 -- 172.21.15.69:0/2987652853 learned_addr learned my addr 172.21.15.69:0/2987652853 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:31.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.488+0000 7f2e1361a740 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 0x55d1d9635ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.488+0000 7f2e1361a740 1 -- 172.21.15.69:0/2987652853 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d1d964c040 con 0x55d1d962f660 2024-03-19T01:21:31.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.488+0000 7f2e1361a740 1 -- 172.21.15.69:0/2987652853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d1d954efc0 con 0x55d1d962ed40 2024-03-19T01:21:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.488+0000 7f2e1361a740 1 -- 172.21.15.69:0/2987652853 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d1d964c1c0 con 0x55d1d9638240 2024-03-19T01:21:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.488+0000 7f2e1361a740 10 client.0 _ll_register_callbacks cb 0x55d1d954e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.489+0000 7f2e0d400640 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 0x55d1d96357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.489+0000 7f2e0e800640 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 0x55d1d9635ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.490+0000 7f2e0e800640 1 -- 172.21.15.69:0/2987652853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 msgr2=0x55d1d9635260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.490+0000 7f2e0e800640 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d9635260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.490+0000 7f2e0e800640 1 -- 172.21.15.69:0/2987652853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 msgr2=0x55d1d96357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:31.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.490+0000 7f2e0e800640 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 0x55d1d96357a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.490+0000 7f2e0e800640 1 -- 172.21.15.69:0/2987652853 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d92a73f0 con 0x55d1d9638240 2024-03-19T01:21:31.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.491+0000 7f2de7400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:21:31.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.491+0000 7f2e0e800640 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 0x55d1d9635ce0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009530 tx=0x7f2df80092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b1473170acec98c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:31.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.492+0000 7f2de7400640 1 -- 172.21.15.69:0/2987652853 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8009920 con 0x55d1d9638240 2024-03-19T01:21:31.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.492+0000 7f2de7400640 1 -- 172.21.15.69:0/2987652853 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7f2df800e850 con 0x55d1d9638240 2024-03-19T01:21:31.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.492+0000 7f2e1361a740 1 -- 172.21.15.69:0/2987652853 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df801cc30 con 0x55d1d9638240 2024-03-19T01:21:31.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.493+0000 7f2de7400640 1 -- 172.21.15.69:0/2987652853 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2df800ea50 con 0x55d1d9638240 2024-03-19T01:21:31.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.493+0000 7f2e1361a740 10 client.27778 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:31.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.493+0000 7f2e1361a740 1 -- 172.21.15.69:0/2987652853 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55d1d92670f0 con 0x55d1d9638240 2024-03-19T01:21:31.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.494+0000 7f2de7400640 1 -- 172.21.15.69:0/2987652853 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2df801cc30 con 0x55d1d9638240 2024-03-19T01:21:31.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.496+0000 7f2de7400640 1 -- 172.21.15.69:0/2987652853 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f2df8028c90 con 0x55d1d9638240 2024-03-19T01:21:31.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.496+0000 7f2e1361a740 10 client.27778 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:31.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.496+0000 7f2e1361a740 10 client.27778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:21:31.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.496+0000 7f2e1361a740 20 client.27778 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:31.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.496+0000 7f2e1361a740 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d1d96594a0 0x55d1d965b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.496+0000 7f2e1361a740 4 client.27778 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:21:31.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.496+0000 7f2e1361a740 1 -- 172.21.15.69:0/2987652853 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1d9534610 con 0x55d1d96594a0 2024-03-19T01:21:31.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.496+0000 7f2e0de00640 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d1d96594a0 0x55d1d965b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.498+0000 7f2e0de00640 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d1d96594a0 0x55d1d965b890 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a66bddf34dc43923 server_cookie=f3a8fb246760121f in_seq=0 out_seq=0 2024-03-19T01:21:31.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.498+0000 7f2de7400640 10 client.27778 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:31.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.499+0000 7f2de7400640 1 -- 172.21.15.69:0/2987652853 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d1d9534610 con 0x55d1d96594a0 2024-03-19T01:21:31.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.499+0000 7f2de7400640 10 client.27778 handle_command_reply: tid=0 2024-03-19T01:21:31.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.499+0000 7f2e1361a740 1 client.27778 shutdown 2024-03-19T01:21:31.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:31.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:31.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:31.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:31 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:31.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:31 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:31.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.508+0000 7f2e1361a740 1 -- 172.21.15.69:0/2987652853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 msgr2=0x55d1d9635ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:31.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.508+0000 7f2e1361a740 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 0x55d1d9635ce0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009530 tx=0x7f2df80092d0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.509+0000 7f2e1361a740 1 -- 172.21.15.69:0/2987652853 shutdown_connections 2024-03-19T01:21:31.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.509+0000 7f2e1361a740 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d1d96594a0 0x55d1d965b890 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.509+0000 7f2e1361a740 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 0x55d1d9635ce0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.509+0000 7f2e1361a740 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 0x55d1d96357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.509+0000 7f2e1361a740 1 --2- 172.21.15.69:0/2987652853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d9635260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.509+0000 7f2de7400640 0 client.27778 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:31.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.509+0000 7f2e1361a740 1 -- 172.21.15.69:0/2987652853 >> 172.21.15.69:0/2987652853 conn(0x55d1d9629470 msgr2=0x55d1d962b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:31.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.510+0000 7f2e1361a740 1 -- 172.21.15.69:0/2987652853 shutdown_connections 2024-03-19T01:21:31.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.513+0000 7f2e1361a740 1 -- 172.21.15.69:0/2987652853 wait complete. 2024-03-19T01:21:31.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.513+0000 7f2e1361a740 20 client.27778 trim_cache size 0 max 16384 2024-03-19T01:21:31.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:31.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:31.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:31.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:31 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:31.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:31 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:31.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:31.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:31.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:31.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:31 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:31.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:31 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:31.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.580+0000 7f2e1361a740 1 Processor -- start 2024-03-19T01:21:31.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.584+0000 7f2e1361a740 1 -- start start 2024-03-19T01:21:31.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.584+0000 7f2e1361a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d96286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.584+0000 7f2e1361a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962f660 0x55d1d9654e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.584+0000 7f2e1361a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d9638240 0x55d1d962afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.584+0000 7f2e1361a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081b5460 con 0x55d1d9638240 2024-03-19T01:21:31.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.584+0000 7f2e1361a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081b6fa0 con 0x55d1d962ed40 2024-03-19T01:21:31.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.584+0000 7f2e1361a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081b5b30 con 0x55d1d962f660 2024-03-19T01:21:31.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.585+0000 7f2e0d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962f660 0x55d1d9654e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.585+0000 7f2e0d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962f660 0x55d1d9654e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:32772/0 (socket says 172.21.15.69:32772) 2024-03-19T01:21:31.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.585+0000 7f2e0d400640 1 -- 172.21.15.69:0/2285659869 learned_addr learned my addr 172.21.15.69:0/2285659869 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:31.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.585+0000 7f2e0de00640 1 --2- 172.21.15.69:0/2285659869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d96286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.585+0000 7f2e0e800640 1 --2- 172.21.15.69:0/2285659869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d9638240 0x55d1d962afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.586+0000 7f2e0de00640 1 -- 172.21.15.69:0/2285659869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962f660 msgr2=0x55d1d9654e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:31.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.586+0000 7f2e0de00640 1 --2- 172.21.15.69:0/2285659869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962f660 0x55d1d9654e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.586+0000 7f2e0de00640 1 -- 172.21.15.69:0/2285659869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d9638240 msgr2=0x55d1d962afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:31.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.586+0000 7f2e0de00640 1 --2- 172.21.15.69:0/2285659869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d9638240 0x55d1d962afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.586+0000 7f2e0de00640 1 -- 172.21.15.69:0/2285659869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d965bc00 con 0x55d1d962ed40 2024-03-19T01:21:31.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.586+0000 7f2e0de00640 1 --2- 172.21.15.69:0/2285659869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d96286b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x55d1d965c030 tx=0x7f2e04000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b844104e1822163 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:31.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.588+0000 7f2df2600640 1 -- 172.21.15.69:0/2285659869 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04039040 con 0x55d1d962ed40 2024-03-19T01:21:31.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.588+0000 7f2df2600640 1 -- 172.21.15.69:0/2285659869 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e04049070 con 0x55d1d962ed40 2024-03-19T01:21:31.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.588+0000 7f2df2600640 1 -- 172.21.15.69:0/2285659869 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04044430 con 0x55d1d962ed40 2024-03-19T01:21:31.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.589+0000 7f2e1361a740 1 -- 172.21.15.69:0/2285659869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 msgr2=0x55d1d96286b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:31.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.589+0000 7f2e1361a740 1 --2- 172.21.15.69:0/2285659869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d96286b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x55d1d965c030 tx=0x7f2e04000bc0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.590+0000 7f2e1361a740 1 -- 172.21.15.69:0/2285659869 shutdown_connections 2024-03-19T01:21:31.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.590+0000 7f2e1361a740 1 --2- 172.21.15.69:0/2285659869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d9638240 0x55d1d962afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.590+0000 7f2e1361a740 1 --2- 172.21.15.69:0/2285659869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d962f660 0x55d1d9654e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.590+0000 7f2e1361a740 1 --2- 172.21.15.69:0/2285659869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d96286b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.590+0000 7f2e1361a740 1 -- 172.21.15.69:0/2285659869 >> 172.21.15.69:0/2285659869 conn(0x55d1d9629470 msgr2=0x55d1d962a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:31.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.591+0000 7f2e1361a740 1 -- 172.21.15.69:0/2285659869 shutdown_connections 2024-03-19T01:21:31.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.591+0000 7f2e1361a740 1 -- 172.21.15.69:0/2285659869 wait complete. 2024-03-19T01:21:31.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.592+0000 7f2e1361a740 1 -- start start 2024-03-19T01:21:31.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.592+0000 7f2e1361a740 1 Processor -- start 2024-03-19T01:21:31.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.594+0000 7f2e1361a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d9632030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.594+0000 7f2e1361a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 0x55d1d9632570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.594+0000 7f2e1361a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 0x55d1d9632ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.594+0000 7f2e1361a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081b68d0 con 0x55d1d962f660 2024-03-19T01:21:31.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.594+0000 7f2e1361a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081b6200 con 0x55d1d962ed40 2024-03-19T01:21:31.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.594+0000 7f2e1361a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081a9830 con 0x55d1d9638240 2024-03-19T01:21:31.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.594+0000 7f2e1361a740 10 client.0 _ll_register_callbacks cb 0x7f2df800e6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:21:31.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.595+0000 7f2e0d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 0x55d1d9632570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.595+0000 7f2e0d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 0x55d1d9632570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51126/0 (socket says 172.21.15.69:51126) 2024-03-19T01:21:31.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.595+0000 7f2e0d400640 1 -- 172.21.15.69:0/3027785742 learned_addr learned my addr 172.21.15.69:0/3027785742 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:21:31.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.595+0000 7f2e0de00640 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d9632030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.596+0000 7f2e0e800640 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 0x55d1d9632ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.596+0000 7f2e0de00640 1 -- 172.21.15.69:0/3027785742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 msgr2=0x55d1d9632ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:31.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.596+0000 7f2e0de00640 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 0x55d1d9632ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.596+0000 7f2e0de00640 1 -- 172.21.15.69:0/3027785742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 msgr2=0x55d1d9632570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:21:31.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.596+0000 7f2e0de00640 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 0x55d1d9632570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.596+0000 7f2e0de00640 1 -- 172.21.15.69:0/3027785742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d96be9c0 con 0x55d1d962ed40 2024-03-19T01:21:31.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.596+0000 7f2e0d400640 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 0x55d1d9632570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:21:31.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.596+0000 7f2e0e800640 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 0x55d1d9632ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:21:31.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.597+0000 7f2e0de00640 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d9632030 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2e040095f0 tx=0x7f2e04034840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=381128e1fce23423 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:21:31.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.597+0000 7f2de7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:21:31.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.597+0000 7f2de7400640 1 -- 172.21.15.69:0/3027785742 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04051070 con 0x55d1d962ed40 2024-03-19T01:21:31.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.597+0000 7f2de7400640 1 -- 172.21.15.69:0/3027785742 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7f2e04034d30 con 0x55d1d962ed40 2024-03-19T01:21:31.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.599+0000 7f2e1361a740 1 -- 172.21.15.69:0/3027785742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e04049070 con 0x55d1d962ed40 2024-03-19T01:21:31.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.600+0000 7f2de7400640 1 -- 172.21.15.69:0/3027785742 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2e0405b050 con 0x55d1d962ed40 2024-03-19T01:21:31.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.600+0000 7f2e1361a740 10 client.27818 fetch_fsmap learned FSMap version 59 2024-03-19T01:21:31.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.600+0000 7f2e1361a740 1 -- 172.21.15.69:0/3027785742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55d1d96bfc50 con 0x55d1d962ed40 2024-03-19T01:21:31.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.600+0000 7f2de7400640 1 -- 172.21.15.69:0/3027785742 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e04049070 con 0x55d1d962ed40 2024-03-19T01:21:31.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.602+0000 7f2de7400640 1 -- 172.21.15.69:0/3027785742 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f2e04004040 con 0x55d1d962ed40 2024-03-19T01:21:31.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.603+0000 7f2e1361a740 10 client.27818 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:21:31.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.603+0000 7f2e1361a740 10 client.27818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:21:31.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.603+0000 7f2e1361a740 20 client.27818 populate_metadata read hostname 'smithi069' 2024-03-19T01:21:31.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.603+0000 7f2e1361a740 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d1d96c0270 0x55d1d96c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:21:31.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.603+0000 7f2e1361a740 4 client.27818 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:21:31.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.603+0000 7f2e1361a740 1 -- 172.21.15.69:0/3027785742 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1d95a0800 con 0x55d1d96c0270 2024-03-19T01:21:31.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.603+0000 7f2e0d400640 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d1d96c0270 0x55d1d96c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:21:31.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.604+0000 7f2e0d400640 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d1d96c0270 0x55d1d96c2660 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=469a9636038e5ca1 server_cookie=961f7f958366c73f in_seq=0 out_seq=0 2024-03-19T01:21:31.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.604+0000 7f2de7400640 10 client.27818 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:31.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.604+0000 7f2de7400640 1 -- 172.21.15.69:0/3027785742 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55d1d95a0800 con 0x55d1d96c0270 2024-03-19T01:21:31.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.605+0000 7f2de7400640 10 client.27818 handle_command_reply: tid=0 2024-03-19T01:21:31.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.605+0000 7f2e1361a740 1 client.27818 shutdown 2024-03-19T01:21:31.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.606+0000 7f2e1361a740 1 -- 172.21.15.69:0/3027785742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 msgr2=0x55d1d9632030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:31.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.606+0000 7f2e1361a740 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d9632030 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2e040095f0 tx=0x7f2e04034840 comp rx=0 tx=0).stop 2024-03-19T01:21:31.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.607+0000 7f2e1361a740 1 -- 172.21.15.69:0/3027785742 shutdown_connections 2024-03-19T01:21:31.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.607+0000 7f2e1361a740 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d1d96c0270 0x55d1d96c2660 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.607+0000 7f2e1361a740 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d1d9638240 0x55d1d9632ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.607+0000 7f2e1361a740 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d1d962f660 0x55d1d9632570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.607+0000 7f2e1361a740 1 --2- 172.21.15.69:0/3027785742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1d962ed40 0x55d1d9632030 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.607+0000 7f2de7400640 0 client.27818 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:21:31.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.607+0000 7f2e1361a740 1 -- 172.21.15.69:0/3027785742 >> 172.21.15.69:0/3027785742 conn(0x55d1d9629470 msgr2=0x55d1d9635ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:31.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.607+0000 7f2e1361a740 1 -- 172.21.15.69:0/3027785742 shutdown_connections 2024-03-19T01:21:31.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.607+0000 7f2e1361a740 1 -- 172.21.15.69:0/3027785742 wait complete. 2024-03-19T01:21:31.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.609+0000 7f2e1361a740 20 client.27818 trim_cache size 0 max 16384 2024-03-19T01:21:31.611 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:21:31.612 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (11445 inodes in the stack)", 2024-03-19T01:21:31.612 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:21:31.612 INFO:teuthology.orchestra.run.smithi069.stdout: "fd79c60c-0ee4-4850-8dc0-50db38a35962": { 2024-03-19T01:21:31.612 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:21:31.612 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "fd79c60c-0ee4-4850-8dc0-50db38a35962", 2024-03-19T01:21:31.612 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:21:31.612 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:21:31.612 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:21:31.612 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:21:31.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.613+0000 7f2e10800640 1 -- 172.21.15.69:0/3954609562 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2df40a2ab0 msgr2=0x7f2df40a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:31.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.613+0000 7f2e10800640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2df40a2ab0 0x7f2df40a4f70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2df8020000 tx=0x7f2df801d000 comp rx=0 tx=0).stop 2024-03-19T01:21:31.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.613+0000 7f2e10800640 1 -- 172.21.15.69:0/3954609562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e08111500 msgr2=0x7f2e08116110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:21:31.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.613+0000 7f2e10800640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e08111500 0x7f2e08116110 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000dbf0 tx=0x7f2e0000b630 comp rx=0 tx=0).stop 2024-03-19T01:21:31.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.614+0000 7f2e10800640 1 -- 172.21.15.69:0/3954609562 shutdown_connections 2024-03-19T01:21:31.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.614+0000 7f2e10800640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2df40a2ab0 0x7f2df40a4f70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.614+0000 7f2e10800640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e08116650 0x7f2e08116ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.614+0000 7f2e10800640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e08111500 0x7f2e08116110 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.614+0000 7f2e10800640 1 --2- 172.21.15.69:0/3954609562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e0810c440 0x7f2e08119020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:21:31.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.614+0000 7f2e10800640 1 -- 172.21.15.69:0/3954609562 >> 172.21.15.69:0/3954609562 conn(0x7f2e0806c820 msgr2=0x7f2e0810f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:21:31.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.617+0000 7f2e10800640 1 -- 172.21.15.69:0/3954609562 shutdown_connections 2024-03-19T01:21:31.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:21:31.617+0000 7f2e10800640 1 -- 172.21.15.69:0/3954609562 wait complete. 2024-03-19T01:21:31.679 INFO:tasks.cephfs.filesystem:scrub status for tag:fd79c60c-0ee4-4850-8dc0-50db38a35962 - {'path': '/', 'tag': 'fd79c60c-0ee4-4850-8dc0-50db38a35962', 'options': 'recursive,force'} 2024-03-19T01:21:31.719 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/smp.o 2024-03-19T01:21:31.747 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/n_null.o 2024-03-19T01:21:31.810 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsrepair2.o 2024-03-19T01:21:31.979 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_bo.o 2024-03-19T01:21:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: pgmap v338: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:32 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:32.524 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/intel/trace.o 2024-03-19T01:21:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: pgmap v338: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-03-19T01:21:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:21:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:32 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: pgmap v338: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-03-19T01:21:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:21:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:32.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:32 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:32.601 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/hexdump.o 2024-03-19T01:21:32.676 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/intel/cap_audit.o 2024-03-19T01:21:32.905 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/pty.o 2024-03-19T01:21:32.921 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nssearch.o 2024-03-19T01:21:33.030 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/kstrtox.o 2024-03-19T01:21:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:33.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:33 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:33.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:33 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:33.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:33 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:33.749 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_bo_util.o 2024-03-19T01:21:33.782 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsutils.o 2024-03-19T01:21:33.906 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/iomap.o 2024-03-19T01:21:33.950 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/uid16.o 2024-03-19T01:21:34.241 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/pci_iomap.o 2024-03-19T01:21:34.324 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/tty_audit.o 2024-03-19T01:21:34.503 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4idmap.o 2024-03-19T01:21:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: pgmap v339: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.1 MiB/s wr, 80 op/s 2024-03-19T01:21:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:34 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:34.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: pgmap v339: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.1 MiB/s wr, 80 op/s 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:34 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: pgmap v339: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.1 MiB/s wr, 80 op/s 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:34.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:34 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.067 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/intel/perfmon.o 2024-03-19T01:21:35.145 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nswalk.o 2024-03-19T01:21:35.300 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_bo_vm.o 2024-03-19T01:21:35.351 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/kallsyms.o 2024-03-19T01:21:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:35 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:35 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:35 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:35.617 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/tty/sysrq.o 2024-03-19T01:21:35.637 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsxfeval.o 2024-03-19T01:21:35.835 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/iomap_copy.o 2024-03-19T01:21:36.221 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/connector/cn_queue.o 2024-03-19T01:21:36.434 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/devres.o 2024-03-19T01:21:36.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: pgmap v340: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 363 KiB/s wr, 52 op/s 2024-03-19T01:21:36.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:36.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:36.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:36.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:36.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:36 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: pgmap v340: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 363 KiB/s wr, 52 op/s 2024-03-19T01:21:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:36.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:36.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:36 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: pgmap v340: 129 pgs: 129 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 363 KiB/s wr, 52 op/s 2024-03-19T01:21:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:36.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:36.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:36 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:36.847 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsxfname.o 2024-03-19T01:21:36.867 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_module.o 2024-03-19T01:21:37.439 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/callback.o 2024-03-19T01:21:37.448 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/iommu/intel/built-in.a 2024-03-19T01:21:37.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:37.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:37 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:37.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:37.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:37 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:37.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:37 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:37.605 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/tty/built-in.a 2024-03-19T01:21:37.774 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/acct.o 2024-03-19T01:21:38.019 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/nsxfobj.o 2024-03-19T01:21:38.082 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/check_signature.o 2024-03-19T01:21:38.157 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/connector/connector.o 2024-03-19T01:21:38.205 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/iommu/arm/arm-smmu/built-in.a 2024-03-19T01:21:38.210 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_execbuf_util.o 2024-03-19T01:21:38.322 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/iommu/arm/arm-smmu-v3/built-in.a 2024-03-19T01:21:38.408 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/iommu/arm/built-in.a 2024-03-19T01:21:38.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:38 smithi086 ceph-mon[66829]: pgmap v341: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 358 KiB/s wr, 106 op/s 2024-03-19T01:21:38.529 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/iommu/iommufd/built-in.a 2024-03-19T01:21:38.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:38 smithi069 ceph-mon[31727]: pgmap v341: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 358 KiB/s wr, 106 op/s 2024-03-19T01:21:38.534 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/interval_tree.o 2024-03-19T01:21:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:38 smithi163 ceph-mon[67428]: pgmap v341: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 358 KiB/s wr, 106 op/s 2024-03-19T01:21:38.605 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/iommu.o 2024-03-19T01:21:38.829 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/iommu-traces.o 2024-03-19T01:21:38.861 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/psargs.o 2024-03-19T01:21:38.987 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/crash_core.o 2024-03-19T01:21:39.155 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/assoc_array.o 2024-03-19T01:21:39.470 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/callback_xdr.o 2024-03-19T01:21:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:39.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:39 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:39.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:39 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:39.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:39 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:39.571 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_range_manager.o 2024-03-19T01:21:39.982 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/psloop.o 2024-03-19T01:21:40.153 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/connector/cn_proc.o 2024-03-19T01:21:40.427 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/kexec_core.o 2024-03-19T01:21:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: pgmap v342: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 753 KiB/s wr, 324 op/s 2024-03-19T01:21:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:40.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:40 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: pgmap v342: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 753 KiB/s wr, 324 op/s 2024-03-19T01:21:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:40.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:40 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: pgmap v342: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 753 KiB/s wr, 324 op/s 2024-03-19T01:21:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:40.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:40 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:40.565 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/bitrev.o 2024-03-19T01:21:40.779 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_driver.o 2024-03-19T01:21:40.877 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_resource.o 2024-03-19T01:21:40.932 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/psobject.o 2024-03-19T01:21:41.394 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/iommu-sysfs.o 2024-03-19T01:21:41.453 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crc-ccitt.o 2024-03-19T01:21:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: pgmap v343: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 859 KiB/s wr, 477 op/s 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: pgmap v343: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 859 KiB/s wr, 477 op/s 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:41.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: pgmap v343: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 859 KiB/s wr, 477 op/s 2024-03-19T01:21:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:41.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:41.824 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/psopcode.o 2024-03-19T01:21:41.927 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/connector/built-in.a 2024-03-19T01:21:41.997 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/callback_proc.o 2024-03-19T01:21:42.282 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/vga/built-in.a 2024-03-19T01:21:42.327 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4namespace.o 2024-03-19T01:21:42.475 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_pool.o 2024-03-19T01:21:42.503 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/kexec.o 2024-03-19T01:21:42.504 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/dma-iommu.o 2024-03-19T01:21:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:42.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.515 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crc16.o 2024-03-19T01:21:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:42.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.548 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/psopinfo.o 2024-03-19T01:21:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:42.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:42.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.406 INFO:tasks.workunit.client.0.smithi069.stdout: HOSTCC lib/gen_crc32table 2024-03-19T01:21:43.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: pgmap v344: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1015 KiB/s wr, 1.16k op/s 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:43.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: pgmap v344: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1015 KiB/s wr, 1.16k op/s 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:43.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.568 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/psparse.o 2024-03-19T01:21:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: pgmap v344: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1015 KiB/s wr, 1.16k op/s 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:43.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:43.761 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/xxhash.o 2024-03-19T01:21:43.815 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/compat.o 2024-03-19T01:21:44.169 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/genalloc.o 2024-03-19T01:21:44.404 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4getroot.o 2024-03-19T01:21:44.413 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_drm_client.o 2024-03-19T01:21:44.424 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_device.o 2024-03-19T01:21:44.444 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/psscope.o 2024-03-19T01:21:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.567 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/io-pgtable.o 2024-03-19T01:21:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:44.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:44.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:44.800 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/utsname.o 2024-03-19T01:21:45.176 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/pstree.o 2024-03-19T01:21:45.238 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4client.o 2024-03-19T01:21:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:45 smithi069 ceph-mon[31727]: pgmap v345: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 903 KiB/s wr, 1.42k op/s 2024-03-19T01:21:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:45 smithi163 ceph-mon[67428]: pgmap v345: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 903 KiB/s wr, 1.42k op/s 2024-03-19T01:21:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.586 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/iommu/iova.o 2024-03-19T01:21:45.600 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/percpu_counter.o 2024-03-19T01:21:45.622 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_sys_manager.o 2024-03-19T01:21:45.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:45 smithi086 ceph-mon[66829]: pgmap v345: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 903 KiB/s wr, 1.42k op/s 2024-03-19T01:21:45.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:45.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:45.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:45.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:45.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:45.790 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/pid_namespace.o 2024-03-19T01:21:45.992 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/psutils.o 2024-03-19T01:21:46.072 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/renesas/rcar-du/built-in.a 2024-03-19T01:21:46.125 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_config.o 2024-03-19T01:21:46.151 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/renesas/built-in.a 2024-03-19T01:21:46.274 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4session.o 2024-03-19T01:21:46.452 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/syscall.o 2024-03-19T01:21:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:46.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:46.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:46.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:46.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.618 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/pswalk.o 2024-03-19T01:21:46.691 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_getparam.o 2024-03-19T01:21:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:46.769 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/ttm/ttm_agp_backend.o 2024-03-19T01:21:46.969 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/iommu/built-in.a 2024-03-19T01:21:47.233 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_ioctl.o 2024-03-19T01:21:47.251 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/stop_machine.o 2024-03-19T01:21:47.273 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/nvs.o 2024-03-19T01:21:47.368 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/psxface.o 2024-03-19T01:21:47.479 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/errname.o 2024-03-19T01:21:47.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:47.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: pgmap v346: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 903 KiB/s wr, 1.64k op/s 2024-03-19T01:21:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:47.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:47.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:47.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:47.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:47.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: pgmap v346: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 903 KiB/s wr, 1.64k op/s 2024-03-19T01:21:47.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:47.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: pgmap v346: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 903 KiB/s wr, 1.64k op/s 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:47.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.248 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/nlattr.o 2024-03-19T01:21:48.284 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rsaddr.o 2024-03-19T01:21:48.293 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/dns_resolve.o 2024-03-19T01:21:48.315 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/ttm/built-in.a 2024-03-19T01:21:48.315 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/omapdrm/built-in.a 2024-03-19T01:21:48.387 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/open.o 2024-03-19T01:21:48.420 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/audit.o 2024-03-19T01:21:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:48.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:48.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:48.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:48.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:48.843 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/wakeup.o 2024-03-19T01:21:48.895 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_irq.o 2024-03-19T01:21:48.917 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rscalc.o 2024-03-19T01:21:48.960 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/cpu_rmap.o 2024-03-19T01:21:49.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:49.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:49 smithi163 ceph-mon[67428]: pgmap v347: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 807 KiB/s wr, 1.99k op/s 2024-03-19T01:21:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:49.581 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/read_write.o 2024-03-19T01:21:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:49 smithi086 ceph-mon[66829]: pgmap v347: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 807 KiB/s wr, 1.99k op/s 2024-03-19T01:21:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:49.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:49.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:49 smithi069 ceph-mon[31727]: pgmap v347: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 807 KiB/s wr, 1.99k op/s 2024-03-19T01:21:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:49.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:49.932 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/file_table.o 2024-03-19T01:21:50.099 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4trace.o 2024-03-19T01:21:50.437 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/dynamic_queue_limits.o 2024-03-19T01:21:50.456 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rscreate.o 2024-03-19T01:21:50.714 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/auditfilter.o 2024-03-19T01:21:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:50.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:50.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:50.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:50.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:50.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:50.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:50.858 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/super.o 2024-03-19T01:21:50.991 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/glob.o 2024-03-19T01:21:51.167 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_mitigations.o 2024-03-19T01:21:51.465 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rsdumpinfo.o 2024-03-19T01:21:51.481 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/strncpy_from_user.o 2024-03-19T01:21:51.678 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/sleep.o 2024-03-19T01:21:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: pgmap v348: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 394 KiB/s wr, 1.79k op/s 2024-03-19T01:21:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: pgmap v348: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 394 KiB/s wr, 1.79k op/s 2024-03-19T01:21:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: pgmap v348: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 394 KiB/s wr, 1.79k op/s 2024-03-19T01:21:51.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:51.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:51.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:51.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.073 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/auditsc.o 2024-03-19T01:21:52.484 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/tilcdc/built-in.a 2024-03-19T01:21:52.523 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/strnlen_user.o 2024-03-19T01:21:52.531 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rsinfo.o 2024-03-19T01:21:52.559 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nfs/nfs4sysctl.o 2024-03-19T01:21:52.559 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_module.o 2024-03-19T01:21:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:52.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:52.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:52.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:52.883 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/device_sysfs.o 2024-03-19T01:21:53.396 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rsio.o 2024-03-19T01:21:53.594 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/net_utils.o 2024-03-19T01:21:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:53 smithi086 ceph-mon[66829]: pgmap v349: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 689 KiB/s wr, 1.66k op/s 2024-03-19T01:21:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:53 smithi069 ceph-mon[31727]: pgmap v349: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 689 KiB/s wr, 1.66k op/s 2024-03-19T01:21:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:53.803 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_drv.o 2024-03-19T01:21:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:53 smithi163 ceph-mon[67428]: pgmap v349: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 689 KiB/s wr, 1.66k op/s 2024-03-19T01:21:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:53.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.207 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rsirq.o 2024-03-19T01:21:54.255 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_params.o 2024-03-19T01:21:54.284 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/char_dev.o 2024-03-19T01:21:54.299 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/nfs/built-in.a 2024-03-19T01:21:54.299 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/stat.o 2024-03-19T01:21:54.569 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/power/sysfs.o 2024-03-19T01:21:54.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:54.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:54.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:21:54.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.730 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/audit_watch.o 2024-03-19T01:21:54.773 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/sg_pool.o 2024-03-19T01:21:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:21:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:21:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:21:54.830 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rslist.o 2024-03-19T01:21:55.209 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_kms.o 2024-03-19T01:21:55.378 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/power/generic_ops.o 2024-03-19T01:21:55.462 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_pci.o 2024-03-19T01:21:55.478 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rsmemory.o 2024-03-19T01:21:55.531 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/firmware_loader/builtin/main.o 2024-03-19T01:21:55.712 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/exec.o 2024-03-19T01:21:55.851 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/stackdepot.o 2024-03-19T01:21:56.006 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rsmisc.o 2024-03-19T01:21:56.009 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/base/firmware_loader/builtin/built-in.a 2024-03-19T01:21:56.072 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/audit_fsnotify.o 2024-03-19T01:21:56.084 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/firmware_loader/main.o 2024-03-19T01:21:56.168 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/power/common.o 2024-03-19T01:21:56.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:56 smithi163 ceph-mon[67428]: pgmap v350: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 681 KiB/s wr, 990 op/s 2024-03-19T01:21:56.322 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_gem.o 2024-03-19T01:21:56.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:56 smithi086 ceph-mon[66829]: pgmap v350: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 681 KiB/s wr, 990 op/s 2024-03-19T01:21:56.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:56 smithi069 ceph-mon[31727]: pgmap v350: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 681 KiB/s wr, 990 op/s 2024-03-19T01:21:56.712 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_scatterlist.o 2024-03-19T01:21:56.717 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rsserial.o 2024-03-19T01:21:56.887 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/power/qos.o 2024-03-19T01:21:56.987 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/asn1_decoder.o 2024-03-19T01:21:57.259 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/audit_tree.o 2024-03-19T01:21:57.291 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/base/firmware_loader/built-in.a 2024-03-19T01:21:57.297 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rsutils.o 2024-03-19T01:21:57.304 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_vram.o 2024-03-19T01:21:57.426 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/power/runtime.o 2024-03-19T01:21:57.788 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/pipe.o 2024-03-19T01:21:57.790 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_suspend.o 2024-03-19T01:21:57.864 INFO:tasks.workunit.client.0.smithi069.stdout: GEN lib/oid_registry_data.c 2024-03-19T01:21:57.933 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/rsxface.o 2024-03-19T01:21:58.071 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/ucs2_string.o 2024-03-19T01:21:58.214 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/sbitmap.o 2024-03-19T01:21:58.256 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_display.o 2024-03-19T01:21:58.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:21:58 smithi163 ceph-mon[67428]: pgmap v351: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 695 KiB/s wr, 721 op/s 2024-03-19T01:21:58.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:21:58 smithi069 ceph-mon[31727]: pgmap v351: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 695 KiB/s wr, 721 op/s 2024-03-19T01:21:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:21:58 smithi086 ceph-mon[66829]: pgmap v351: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 695 KiB/s wr, 721 op/s 2024-03-19T01:21:58.558 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/kprobes.o 2024-03-19T01:21:58.708 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/power/wakeirq.o 2024-03-19T01:21:58.847 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/tbdata.o 2024-03-19T01:21:58.916 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/imx/built-in.a 2024-03-19T01:21:58.971 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/seccomp.o 2024-03-19T01:21:59.279 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_switcheroo.o 2024-03-19T01:21:59.280 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/namei.o 2024-03-19T01:21:59.506 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/tbfadt.o 2024-03-19T01:21:59.510 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/group_cpus.o 2024-03-19T01:21:59.530 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/power/main.o 2024-03-19T01:21:59.655 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_vq.o 2024-03-19T01:22:00.116 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/tbfind.o 2024-03-19T01:22:00.251 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/power/wakeup.o 2024-03-19T01:22:00.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:00 smithi163 ceph-mon[67428]: pgmap v352: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 857 KiB/s wr, 489 op/s 2024-03-19T01:22:00.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:00 smithi086 ceph-mon[66829]: pgmap v352: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 857 KiB/s wr, 489 op/s 2024-03-19T01:22:00.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:00 smithi069 ceph-mon[31727]: pgmap v352: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 857 KiB/s wr, 489 op/s 2024-03-19T01:22:00.679 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/lib.a 2024-03-19T01:22:00.695 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_sysfs.o 2024-03-19T01:22:00.968 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/tbinstal.o 2024-03-19T01:22:01.032 INFO:tasks.workunit.client.0.smithi069.stdout: GEN lib/crc32table.h 2024-03-19T01:22:01.057 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/oid_registry.o 2024-03-19T01:22:01.238 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/relay.o 2024-03-19T01:22:01.263 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_fence.o 2024-03-19T01:22:01.398 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/utsname_sysctl.o 2024-03-19T01:22:01.625 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/power/wakeup_stats.o 2024-03-19T01:22:01.627 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/tbprint.o 2024-03-19T01:22:01.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:22:01.680 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:22:01.690 INFO:tasks.workunit.client.0.smithi069.stdout: CC lib/crc32.o 2024-03-19T01:22:01.869 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/delayacct.o 2024-03-19T01:22:01.881 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_utils.o 2024-03-19T01:22:02.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.063+0000 7fa4aa000640 1 -- 172.21.15.69:0/717892063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa49c09d570 msgr2=0x7fa49c0a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:02.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.063+0000 7fa4aa000640 1 --2- 172.21.15.69:0/717892063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa49c09d570 0x7fa49c0a0fa0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa498009930 tx=0x7fa49802bb90 comp rx=0 tx=0).stop 2024-03-19T01:22:02.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.063+0000 7fa4aa000640 1 -- 172.21.15.69:0/717892063 shutdown_connections 2024-03-19T01:22:02.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.063+0000 7fa4aa000640 1 --2- 172.21.15.69:0/717892063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa49c0a16d0 0x7fa49c0a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.063+0000 7fa4aa000640 1 --2- 172.21.15.69:0/717892063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa49c09d570 0x7fa49c0a0fa0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.063+0000 7fa4aa000640 1 --2- 172.21.15.69:0/717892063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa49c09cbc0 0x7fa49c09cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.063+0000 7fa4aa000640 1 -- 172.21.15.69:0/717892063 >> 172.21.15.69:0/717892063 conn(0x7fa49c009290 msgr2=0x7fa49c0096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:02.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.066+0000 7fa4aa000640 1 -- 172.21.15.69:0/717892063 shutdown_connections 2024-03-19T01:22:02.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.067+0000 7fa4aa000640 1 -- 172.21.15.69:0/717892063 wait complete. 2024-03-19T01:22:02.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.072+0000 7fa4aa000640 1 Processor -- start 2024-03-19T01:22:02.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.072+0000 7fa4aa000640 1 -- start start 2024-03-19T01:22:02.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.073+0000 7fa4aa000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa49c09cbc0 0x7fa49c138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.073+0000 7fa4aa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa49c0a16d0 0x7fa49c138da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.073+0000 7fa4aa000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa49c132a10 0x7fa49c132ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.073+0000 7fa4aa000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c0ac6f0 con 0x7fa49c132a10 2024-03-19T01:22:02.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.073+0000 7fa4aa000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c0ac570 con 0x7fa49c0a16d0 2024-03-19T01:22:02.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.073+0000 7fa4aa000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c0ac870 con 0x7fa49c09cbc0 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.073+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa49c0a16d0 0x7fa49c138da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.073+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa49c0a16d0 0x7fa49c138da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:59098/0 (socket says 172.21.15.69:59098) 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.073+0000 7fa4a3e00640 1 -- 172.21.15.69:0/2519265437 learned_addr learned my addr 172.21.15.69:0/2519265437 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.073+0000 7fa4a9600640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa49c132a10 0x7fa49c132ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.074+0000 7fa4a3e00640 1 -- 172.21.15.69:0/2519265437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa49c09cbc0 msgr2=0x7fa49c138860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.074+0000 7fa4a8c00640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa49c09cbc0 0x7fa49c138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.074+0000 7fa4a3e00640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa49c09cbc0 0x7fa49c138860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.074+0000 7fa4a3e00640 1 -- 172.21.15.69:0/2519265437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa49c132a10 msgr2=0x7fa49c132ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.074+0000 7fa4a3e00640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa49c132a10 0x7fa49c132ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.075+0000 7fa4a3e00640 1 -- 172.21.15.69:0/2519265437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49c133530 con 0x7fa49c0a16d0 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.075+0000 7fa4a8c00640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa49c09cbc0 0x7fa49c138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.075+0000 7fa4a9600640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa49c132a10 0x7fa49c132ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:22:02.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.075+0000 7fa4a3e00640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa49c0a16d0 0x7fa49c138da0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa498009900 tx=0x7fa498004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:02.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.079+0000 7fa4a1600640 1 -- 172.21.15.69:0/2519265437 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498043020 con 0x7fa49c0a16d0 2024-03-19T01:22:02.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.079+0000 7fa4a1600640 1 -- 172.21.15.69:0/2519265437 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4980054e0 con 0x7fa49c0a16d0 2024-03-19T01:22:02.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.079+0000 7fa4aa000640 1 -- 172.21.15.69:0/2519265437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49c1337c0 con 0x7fa49c0a16d0 2024-03-19T01:22:02.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.079+0000 7fa4a1600640 1 -- 172.21.15.69:0/2519265437 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498003c50 con 0x7fa49c0a16d0 2024-03-19T01:22:02.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.079+0000 7fa4aa000640 1 -- 172.21.15.69:0/2519265437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49c13fa10 con 0x7fa49c0a16d0 2024-03-19T01:22:02.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.087+0000 7fa4a1600640 1 -- 172.21.15.69:0/2519265437 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fa498046020 con 0x7fa49c0a16d0 2024-03-19T01:22:02.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.087+0000 7fa4ae3e0740 1 Processor -- start 2024-03-19T01:22:02.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.088+0000 7fa4ae3e0740 1 -- start start 2024-03-19T01:22:02.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.088+0000 7fa4ae3e0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6658f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.088+0000 7fa4ae3e0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f6658fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.088+0000 7fa4ae3e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658ff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.088+0000 7fa4ae3e0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f6658582c0 con 0x55f6658f4850 2024-03-19T01:22:02.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.088+0000 7fa4ae3e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f665800b60 con 0x55f6658fd430 2024-03-19T01:22:02.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.088+0000 7fa4ae3e0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f665854360 con 0x55f6658f3f30 2024-03-19T01:22:02.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.089+0000 7fa4a9600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6658f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.089+0000 7fa4a9600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6658f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36662/0 (socket says 172.21.15.69:36662) 2024-03-19T01:22:02.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.089+0000 7fa4a9600640 1 -- 172.21.15.69:0/3409346884 learned_addr learned my addr 172.21.15.69:0/3409346884 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:02.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.089+0000 7fa4a3e00640 1 --2- 172.21.15.69:0/3409346884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.089+0000 7fa4a8c00640 1 --2- 172.21.15.69:0/3409346884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f6658fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.089+0000 7fa4a9600640 1 -- 172.21.15.69:0/3409346884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 msgr2=0x55f6658ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:02.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.089+0000 7fa4a9600640 1 --2- 172.21.15.69:0/3409346884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.089+0000 7fa4a9600640 1 -- 172.21.15.69:0/3409346884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 msgr2=0x55f6658fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:02.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.090+0000 7fa4a9600640 1 --2- 172.21.15.69:0/3409346884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f6658fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.090+0000 7fa4a9600640 1 -- 172.21.15.69:0/3409346884 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6656367d0 con 0x55f6658f3f30 2024-03-19T01:22:02.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.090+0000 7fa4a8c00640 1 --2- 172.21.15.69:0/3409346884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f6658fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:22:02.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.090+0000 7fa4a9600640 1 --2- 172.21.15.69:0/3409346884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6658f4310 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4050e80 tx=0x7fa4a4069850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3682b506246b20f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:02.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.091+0000 7fa495c00640 1 -- 172.21.15.69:0/3409346884 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a406b7f0 con 0x55f6658f3f30 2024-03-19T01:22:02.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.091+0000 7fa495c00640 1 -- 172.21.15.69:0/3409346884 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a406be70 con 0x55f6658f3f30 2024-03-19T01:22:02.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.091+0000 7fa495c00640 1 -- 172.21.15.69:0/3409346884 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a406e620 con 0x55f6658f3f30 2024-03-19T01:22:02.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.093+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3409346884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 msgr2=0x55f6658f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:02.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.093+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3409346884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6658f4310 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4050e80 tx=0x7fa4a4069850 comp rx=0 tx=0).stop 2024-03-19T01:22:02.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.096+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3409346884 shutdown_connections 2024-03-19T01:22:02.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.096+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3409346884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.096+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3409346884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f6658fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.096+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3409346884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6658f4310 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.096+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3409346884 >> 172.21.15.69:0/3409346884 conn(0x55f6658ee660 msgr2=0x55f6658eea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:02.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.097+0000 7fa4a1600640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa48c0a2ac0 0x7fa48c0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.098+0000 7fa4a8c00640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa48c0a2ac0 0x7fa48c0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.099+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3409346884 shutdown_connections 2024-03-19T01:22:02.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.100+0000 7fa4a8c00640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa48c0a2ac0 0x7fa48c0a4f80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55f6658f3a00 tx=0x7fa490008040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:02.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.100+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3409346884 wait complete. 2024-03-19T01:22:02.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.100+0000 7fa4ae3e0740 1 -- start start 2024-03-19T01:22:02.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.101+0000 7fa4ae3e0740 1 Processor -- start 2024-03-19T01:22:02.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.101+0000 7fa4a1600640 1 -- 172.21.15.69:0/2519265437 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fa4980f17b0 con 0x7fa49c0a16d0 2024-03-19T01:22:02.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.102+0000 7fa4ae3e0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f3f30 0x55f6658fa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.103+0000 7fa4a8c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f3f30 0x55f6658fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.103+0000 7fa4a8c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f3f30 0x55f6658fa390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49780/0 (socket says 172.21.15.69:49780) 2024-03-19T01:22:02.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.103+0000 7fa4a8c00640 1 -- 172.21.15.69:0/4097458382 learned_addr learned my addr 172.21.15.69:0/4097458382 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:02.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.104+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f4850 0x55f6658fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.104+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658fae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.104+0000 7fa4a3e00640 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f4850 0x55f6658fa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.105+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/4097458382 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f665900220 con 0x55f6658f3f30 2024-03-19T01:22:02.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.105+0000 7fa4a9600640 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.105+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/4097458382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f665813fc0 con 0x55f6658fd430 2024-03-19T01:22:02.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.105+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/4097458382 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f6659003a0 con 0x55f6658f4850 2024-03-19T01:22:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.106+0000 7fa4a9600640 1 -- 172.21.15.69:0/4097458382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f4850 msgr2=0x55f6658fa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.106+0000 7fa4a9600640 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f4850 0x55f6658fa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.106+0000 7fa4a9600640 1 -- 172.21.15.69:0/4097458382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f3f30 msgr2=0x55f6658fa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.106+0000 7fa4a9600640 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f3f30 0x55f6658fa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.106+0000 7fa4a9600640 1 -- 172.21.15.69:0/4097458382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66556c3f0 con 0x55f6658fd430 2024-03-19T01:22:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.106+0000 7fa4a8c00640 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f3f30 0x55f6658fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:22:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.106+0000 7fa4a9600640 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658fae10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40674f0 tx=0x7fa4a406ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c93ecb1f4e2e703 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.107+0000 7fa48a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:22:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.107+0000 7fa48a000640 1 -- 172.21.15.69:0/4097458382 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a406b7f0 con 0x55f6658fd430 2024-03-19T01:22:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.107+0000 7fa48a000640 1 -- 172.21.15.69:0/4097458382 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fa4a408e040 con 0x55f6658fd430 2024-03-19T01:22:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.107+0000 7fa48a000640 10 client.0 inactive, discarding osd_map(104..104 src has 1..104) 2024-03-19T01:22:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.108+0000 7fa4ae3e0740 10 client.0 _ll_register_callbacks cb 0x55f665813990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.108+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/4097458382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a406be70 con 0x55f6658fd430 2024-03-19T01:22:02.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.108+0000 7fa48a000640 1 -- 172.21.15.69:0/4097458382 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa4a406b990 con 0x55f6658fd430 2024-03-19T01:22:02.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.109+0000 7fa4ae3e0740 10 client.27830 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.109+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/4097458382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=59}) -- 0x55f66552c0f0 con 0x55f6658fd430 2024-03-19T01:22:02.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.113+0000 7fa48a000640 1 -- 172.21.15.69:0/4097458382 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fa4a406bb30 con 0x55f6658fd430 2024-03-19T01:22:02.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.115+0000 7fa4ae3e0740 10 client.27830 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:02.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.115+0000 7fa4ae3e0740 10 client.27830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:22:02.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.115+0000 7fa4ae3e0740 20 client.27830 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:02.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.115+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55f66590d060 0x55f66590f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.117+0000 7fa4ae3e0740 4 client.27830 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:22:02.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.117+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/4097458382 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa4a406be70 con 0x55f66590d060 2024-03-19T01:22:02.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.119+0000 7fa4a8c00640 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55f66590d060 0x55f66590f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.120+0000 7fa4a8c00640 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55f66590d060 0x55f66590f4f0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed936b1e9c715825 server_cookie=323c0024396b6ee1 in_seq=0 out_seq=0 2024-03-19T01:22:02.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.120+0000 7fa48a000640 10 client.27830 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:02.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.120+0000 7fa48a000640 1 -- 172.21.15.69:0/4097458382 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fa4a406be70 con 0x55f66590d060 2024-03-19T01:22:02.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.120+0000 7fa48a000640 10 client.27830 handle_command_reply: tid=0 2024-03-19T01:22:02.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.120+0000 7fa4ae3e0740 1 client.27830 shutdown 2024-03-19T01:22:02.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.121+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/4097458382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 msgr2=0x55f6658fae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:02.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.121+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658fae10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40674f0 tx=0x7fa4a406ecc0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.121+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/4097458382 shutdown_connections 2024-03-19T01:22:02.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.121+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55f66590d060 0x55f66590f4f0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.121+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658fae10 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.121+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f4850 0x55f6658fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.121+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/4097458382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f3f30 0x55f6658fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.121+0000 7fa48a000640 0 client.27830 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:02.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.121+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/4097458382 >> 172.21.15.69:0/4097458382 conn(0x55f6658ee660 msgr2=0x55f6658f03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:02.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.124+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/4097458382 shutdown_connections 2024-03-19T01:22:02.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.124+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/4097458382 wait complete. 2024-03-19T01:22:02.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.124+0000 7fa4ae3e0740 20 client.27830 trim_cache size 0 max 16384 2024-03-19T01:22:02.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.165+0000 7fa4ae3e0740 1 Processor -- start 2024-03-19T01:22:02.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.165+0000 7fa4ae3e0740 1 -- start start 2024-03-19T01:22:02.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.166+0000 7fa4ae3e0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6658fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.166+0000 7fa4ae3e0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f665914e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.166+0000 7fa4ae3e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658efd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.166+0000 7fa4ae3e0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c149990 con 0x55f6658f4850 2024-03-19T01:22:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.166+0000 7fa4ae3e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c14b4d0 con 0x55f6658fd430 2024-03-19T01:22:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.166+0000 7fa4ae3e0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c14a060 con 0x55f6658f3f30 2024-03-19T01:22:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.167+0000 7fa4a9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658efd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.167+0000 7fa4a9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658efd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:59124/0 (socket says 172.21.15.69:59124) 2024-03-19T01:22:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.167+0000 7fa4a9600640 1 -- 172.21.15.69:0/3532682691 learned_addr learned my addr 172.21.15.69:0/3532682691 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.167+0000 7fa4a3e00640 1 --2- 172.21.15.69:0/3532682691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f665914e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.167+0000 7fa4a9600640 1 -- 172.21.15.69:0/3532682691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 msgr2=0x55f6658fcac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.167+0000 7fa4a9600640 1 --2- 172.21.15.69:0/3532682691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6658fcac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.167+0000 7fa4a9600640 1 -- 172.21.15.69:0/3532682691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 msgr2=0x55f665914e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.167+0000 7fa4a9600640 1 --2- 172.21.15.69:0/3532682691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f665914e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.167+0000 7fa4a9600640 1 -- 172.21.15.69:0/3532682691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6658f32b0 con 0x55f6658fd430 2024-03-19T01:22:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.168+0000 7fa4a3e00640 1 --2- 172.21.15.69:0/3532682691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f665914e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:22:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.168+0000 7fa4a9600640 1 --2- 172.21.15.69:0/3532682691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658efd70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4000b60 tx=0x7fa4a4076000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9f85901d76f1da server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.169+0000 7fa495200640 1 -- 172.21.15.69:0/3532682691 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a406b990 con 0x55f6658fd430 2024-03-19T01:22:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.169+0000 7fa495200640 1 -- 172.21.15.69:0/3532682691 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a406fab0 con 0x55f6658fd430 2024-03-19T01:22:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.169+0000 7fa495200640 1 -- 172.21.15.69:0/3532682691 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a409b940 con 0x55f6658fd430 2024-03-19T01:22:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.170+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3532682691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 msgr2=0x55f6658efd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.170+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3532682691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658efd70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4000b60 tx=0x7fa4a4076000 comp rx=0 tx=0).stop 2024-03-19T01:22:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.170+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3532682691 shutdown_connections 2024-03-19T01:22:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.170+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3532682691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f6658efd70 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.171+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3532682691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f665914e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.171+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3532682691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6658fcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.171+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3532682691 >> 172.21.15.69:0/3532682691 conn(0x55f6658ee660 msgr2=0x55f6658ef660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.171+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3532682691 shutdown_connections 2024-03-19T01:22:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.171+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3532682691 wait complete. 2024-03-19T01:22:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.172+0000 7fa4ae3e0740 1 -- start start 2024-03-19T01:22:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.172+0000 7fa4ae3e0740 1 Processor -- start 2024-03-19T01:22:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.173+0000 7fa4ae3e0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6659051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.173+0000 7fa4ae3e0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f665905720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.174+0000 7fa4a8c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6659051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.174+0000 7fa4a8c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6659051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:36696/0 (socket says 172.21.15.69:36696) 2024-03-19T01:22:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.174+0000 7fa4a8c00640 1 -- 172.21.15.69:0/3605745394 learned_addr learned my addr 172.21.15.69:0/3605745394 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.174+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f665903710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.174+0000 7fa4a3e00640 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f665905720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.175+0000 7fa4a9600640 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f665903710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.175+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3605745394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c14ae00 con 0x55f6658f4850 2024-03-19T01:22:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.175+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3605745394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c14a730 con 0x55f6658fd430 2024-03-19T01:22:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.175+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3605745394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c147930 con 0x55f6658f3f30 2024-03-19T01:22:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.176+0000 7fa4ae3e0740 10 client.0 _ll_register_callbacks cb 0x7fa4980f4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.176+0000 7fa4a9600640 1 -- 172.21.15.69:0/3605745394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 msgr2=0x55f6659051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.176+0000 7fa4a9600640 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6659051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.176+0000 7fa4a9600640 1 -- 172.21.15.69:0/3605745394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 msgr2=0x55f665905720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.176+0000 7fa4a9600640 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f665905720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.176+0000 7fa4a9600640 1 -- 172.21.15.69:0/3605745394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f665903e00 con 0x55f6658fd430 2024-03-19T01:22:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.177+0000 7fa4a9600640 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f665903710 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4050330 tx=0x7fa4a4062e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5384e203243fdf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.177+0000 7fa48a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:22:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.178+0000 7fa48a000640 1 -- 172.21.15.69:0/3605745394 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a409b470 con 0x55f6658fd430 2024-03-19T01:22:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.178+0000 7fa48a000640 1 -- 172.21.15.69:0/3605745394 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9568+0+0 (secure 0 0 0) 0x7fa4a406f7e0 con 0x55f6658fd430 2024-03-19T01:22:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.178+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3605745394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f6657f9610 con 0x55f6658fd430 2024-03-19T01:22:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.179+0000 7fa48a000640 1 -- 172.21.15.69:0/3605745394 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa4a406b990 con 0x55f6658fd430 2024-03-19T01:22:02.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.179+0000 7fa4ae3e0740 10 client.27836 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.179+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3605745394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55f665973e40 con 0x55f6658fd430 2024-03-19T01:22:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.179+0000 7fa48a000640 1 -- 172.21.15.69:0/3605745394 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f6657f9610 con 0x55f6658fd430 2024-03-19T01:22:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.179+0000 7fa48a000640 1 -- 172.21.15.69:0/3605745394 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fa4a40be430 con 0x55f6658fd430 2024-03-19T01:22:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.179+0000 7fa4ae3e0740 10 client.27836 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.179+0000 7fa4ae3e0740 10 client.27836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:22:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.179+0000 7fa4ae3e0740 20 client.27836 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.179+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55f665974560 0x55f665976950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.179+0000 7fa4ae3e0740 4 client.27836 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:22:02.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.179+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3605745394 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f665865800 con 0x55f665974560 2024-03-19T01:22:02.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.180+0000 7fa4a8c00640 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55f665974560 0x55f665976950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.182+0000 7fa4a8c00640 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55f665974560 0x55f665976950 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b198471c4312df server_cookie=b5b8e77ea5eb6da8 in_seq=0 out_seq=0 2024-03-19T01:22:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.182+0000 7fa48a000640 10 client.27836 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.182+0000 7fa48a000640 1 -- 172.21.15.69:0/3605745394 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f665865800 con 0x55f665974560 2024-03-19T01:22:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.182+0000 7fa48a000640 10 client.27836 handle_command_reply: tid=0 2024-03-19T01:22:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.182+0000 7fa4ae3e0740 1 client.27836 shutdown 2024-03-19T01:22:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.183+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3605745394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 msgr2=0x55f665903710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.183+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f665903710 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4050330 tx=0x7fa4a4062e70 comp rx=0 tx=0).stop 2024-03-19T01:22:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.183+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3605745394 shutdown_connections 2024-03-19T01:22:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.184+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55f665974560 0x55f665976950 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.184+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6658fd430 0x55f665903710 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.184+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6658f4850 0x55f665905720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.184+0000 7fa4ae3e0740 1 --2- 172.21.15.69:0/3605745394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6658f3f30 0x55f6659051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.184+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3605745394 >> 172.21.15.69:0/3605745394 conn(0x55f6658ee660 msgr2=0x55f6658efef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:02.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.184+0000 7fa48a000640 0 client.27836 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:02.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.184+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3605745394 shutdown_connections 2024-03-19T01:22:02.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.184+0000 7fa4ae3e0740 1 -- 172.21.15.69:0/3605745394 wait complete. 2024-03-19T01:22:02.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.184+0000 7fa4ae3e0740 20 client.27836 trim_cache size 0 max 16384 2024-03-19T01:22:02.187 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:22:02.187 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T01:22:02.188 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:22:02.188 INFO:teuthology.orchestra.run.smithi069.stdout: "fd79c60c-0ee4-4850-8dc0-50db38a35962": { 2024-03-19T01:22:02.188 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:22:02.188 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "fd79c60c-0ee4-4850-8dc0-50db38a35962", 2024-03-19T01:22:02.188 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:22:02.188 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:22:02.188 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:22:02.188 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:22:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.186+0000 7fa4aa000640 1 -- 172.21.15.69:0/2519265437 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa48c0a2ac0 msgr2=0x7fa48c0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.186+0000 7fa4aa000640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa48c0a2ac0 0x7fa48c0a4f80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55f6658f3a00 tx=0x7fa490008040 comp rx=0 tx=0).stop 2024-03-19T01:22:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.186+0000 7fa4aa000640 1 -- 172.21.15.69:0/2519265437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa49c0a16d0 msgr2=0x7fa49c138da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.186+0000 7fa4aa000640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa49c0a16d0 0x7fa49c138da0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa498009900 tx=0x7fa498004a90 comp rx=0 tx=0).stop 2024-03-19T01:22:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.187+0000 7fa4aa000640 1 -- 172.21.15.69:0/2519265437 shutdown_connections 2024-03-19T01:22:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.187+0000 7fa4aa000640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa49c132a10 0x7fa49c132ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.187+0000 7fa4aa000640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa48c0a2ac0 0x7fa48c0a4f80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.187+0000 7fa4aa000640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa49c0a16d0 0x7fa49c138da0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.187+0000 7fa4aa000640 1 --2- 172.21.15.69:0/2519265437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa49c09cbc0 0x7fa49c138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.187+0000 7fa4aa000640 1 -- 172.21.15.69:0/2519265437 >> 172.21.15.69:0/2519265437 conn(0x7fa49c009290 msgr2=0x7fa49c0a2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.187+0000 7fa4aa000640 1 -- 172.21.15.69:0/2519265437 shutdown_connections 2024-03-19T01:22:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:02.187+0000 7fa4aa000640 1 -- 172.21.15.69:0/2519265437 wait complete. 2024-03-19T01:22:02.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:02 smithi069 ceph-mon[31727]: pgmap v353: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 803 KiB/s wr, 80 op/s 2024-03-19T01:22:02.216 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_object.o 2024-03-19T01:22:02.219 INFO:tasks.cephfs.filesystem:scrub status for tag:fd79c60c-0ee4-4850-8dc0-50db38a35962 - {'path': '/', 'tag': 'fd79c60c-0ee4-4850-8dc0-50db38a35962', 'options': 'recursive,force'} 2024-03-19T01:22:02.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:02 smithi163 ceph-mon[67428]: pgmap v353: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 803 KiB/s wr, 80 op/s 2024-03-19T01:22:02.363 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/device_pm.o 2024-03-19T01:22:02.384 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/tbutils.o 2024-03-19T01:22:02.412 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fcntl.o 2024-03-19T01:22:02.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:02 smithi086 ceph-mon[66829]: pgmap v353: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 803 KiB/s wr, 80 op/s 2024-03-19T01:22:02.552 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/power/trace.o 2024-03-19T01:22:02.637 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/taskstats.o 2024-03-19T01:22:02.739 INFO:tasks.workunit.client.0.smithi069.stdout: AR lib/built-in.a 2024-03-19T01:22:03.034 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/tbxface.o 2024-03-19T01:22:03.044 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/tbxfload.o 2024-03-19T01:22:03.153 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/intel_clock_gating.o 2024-03-19T01:22:03.250 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_debugfs.o 2024-03-19T01:22:03.550 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/ioctl.o 2024-03-19T01:22:03.570 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/base/power/built-in.a 2024-03-19T01:22:03.637 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/block/loop.o 2024-03-19T01:22:03.701 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/misc/eeprom/built-in.a 2024-03-19T01:22:03.723 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/regmap/regmap.o 2024-03-19T01:22:03.738 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/misc/cb710/built-in.a 2024-03-19T01:22:03.752 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/tbxfroot.o 2024-03-19T01:22:03.802 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/misc/ti-st/built-in.a 2024-03-19T01:22:03.857 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/tsacct.o 2024-03-19T01:22:03.872 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/misc/lis3lv02d/built-in.a 2024-03-19T01:22:03.993 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/misc/cardreader/built-in.a 2024-03-19T01:22:04.127 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/misc/built-in.a 2024-03-19T01:22:04.244 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/tracepoint.o 2024-03-19T01:22:04.276 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_plane.o 2024-03-19T01:22:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:04 smithi163 ceph-mon[67428]: pgmap v354: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 927 KiB/s wr, 69 op/s 2024-03-19T01:22:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:04 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:22:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:04 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:22:04.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:04 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:22:04.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:04 smithi086 ceph-mon[66829]: pgmap v354: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 927 KiB/s wr, 69 op/s 2024-03-19T01:22:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:04 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:22:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:04 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:22:04.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:04 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:22:04.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:04 smithi069 ceph-mon[31727]: pgmap v354: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 927 KiB/s wr, 69 op/s 2024-03-19T01:22:04.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:04 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:22:04.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:04 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:22:04.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:04 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:22:04.592 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utaddress.o 2024-03-19T01:22:04.768 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/readdir.o 2024-03-19T01:22:05.024 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/base/test/built-in.a 2024-03-19T01:22:05.111 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/block/virtio_blk.o 2024-03-19T01:22:05.444 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/irq_work.o 2024-03-19T01:22:05.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:05 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:22:05.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:05 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:22:05.525 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utalloc.o 2024-03-19T01:22:05.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:05 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:22:05.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:05 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:22:05.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:05 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:22:05.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:05 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:22:05.631 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_ioctl.o 2024-03-19T01:22:05.714 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/intel_device_info.o 2024-03-19T01:22:05.855 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utascii.o 2024-03-19T01:22:06.333 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/select.o 2024-03-19T01:22:06.431 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/static_call.o 2024-03-19T01:22:06.431 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/proc.o 2024-03-19T01:22:06.498 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/regmap/regcache.o 2024-03-19T01:22:06.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:06 smithi086 ceph-mon[66829]: pgmap v355: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 507 KiB/s wr, 44 op/s 2024-03-19T01:22:06.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:06 smithi069 ceph-mon[31727]: pgmap v355: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 507 KiB/s wr, 44 op/s 2024-03-19T01:22:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:06 smithi163 ceph-mon[67428]: pgmap v355: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 507 KiB/s wr, 44 op/s 2024-03-19T01:22:06.644 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utbuffer.o 2024-03-19T01:22:06.867 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/block/built-in.a 2024-03-19T01:22:07.071 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/static_call_inline.o 2024-03-19T01:22:07.096 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_prime.o 2024-03-19T01:22:07.223 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/crash_dump.o 2024-03-19T01:22:07.287 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/intel_memory_region.o 2024-03-19T01:22:07.320 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utcksum.o 2024-03-19T01:22:07.422 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_trace_points.o 2024-03-19T01:22:07.596 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/regmap/regcache-rbtree.o 2024-03-19T01:22:07.969 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utcopy.o 2024-03-19T01:22:08.010 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/dcache.o 2024-03-19T01:22:08.151 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/regmap/regcache-flat.o 2024-03-19T01:22:08.276 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/jump_label.o 2024-03-19T01:22:08.468 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/mfd/built-in.a 2024-03-19T01:22:08.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:08 smithi086 ceph-mon[66829]: pgmap v356: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 634 KiB/s wr, 63 op/s 2024-03-19T01:22:08.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:08 smithi069 ceph-mon[31727]: pgmap v356: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 634 KiB/s wr, 63 op/s 2024-03-19T01:22:08.534 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/i2c/built-in.a 2024-03-19T01:22:08.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:08 smithi163 ceph-mon[67428]: pgmap v356: 129 pgs: 129 active+clean; 2.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 634 KiB/s wr, 63 op/s 2024-03-19T01:22:08.597 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/virtio/virtgpu_submit.o 2024-03-19T01:22:08.627 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/component.o 2024-03-19T01:22:08.650 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/intel_pcode.o 2024-03-19T01:22:08.653 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/bus.o 2024-03-19T01:22:08.681 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utexcep.o 2024-03-19T01:22:09.108 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/regmap/regcache-maple.o 2024-03-19T01:22:09.314 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utdebug.o 2024-03-19T01:22:09.548 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/context_tracking.o 2024-03-19T01:22:09.715 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/virtio/built-in.a 2024-03-19T01:22:09.834 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/intel_region_ttm.o 2024-03-19T01:22:09.838 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utdecode.o 2024-03-19T01:22:09.901 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/glue.o 2024-03-19T01:22:09.944 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/intel_runtime_pm.o 2024-03-19T01:22:09.978 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/iomem.o 2024-03-19T01:22:10.277 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/inode.o 2024-03-19T01:22:10.398 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/regmap/regmap-debugfs.o 2024-03-19T01:22:10.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:10 smithi086 ceph-mon[66829]: pgmap v357: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-19T01:22:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:10 smithi086 ceph-mon[66829]: osdmap e105: 12 total, 12 up, 12 in 2024-03-19T01:22:10.509 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/attr.o 2024-03-19T01:22:10.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:10 smithi069 ceph-mon[31727]: pgmap v357: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-19T01:22:10.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:10 smithi069 ceph-mon[31727]: osdmap e105: 12 total, 12 up, 12 in 2024-03-19T01:22:10.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:10 smithi163 ceph-mon[67428]: pgmap v357: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-19T01:22:10.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:10 smithi163 ceph-mon[67428]: osdmap e105: 12 total, 12 up, 12 in 2024-03-19T01:22:10.603 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utdelete.o 2024-03-19T01:22:10.826 INFO:tasks.workunit.client.0.smithi069.stdout: CC kernel/rseq.o 2024-03-19T01:22:10.877 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/panel/built-in.a 2024-03-19T01:22:10.914 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/nfc/built-in.a 2024-03-19T01:22:10.934 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/bad_inode.o 2024-03-19T01:22:11.249 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/file.o 2024-03-19T01:22:11.356 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/uterror.o 2024-03-19T01:22:11.375 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/intel_sbi.o 2024-03-19T01:22:11.391 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/base/regmap/built-in.a 2024-03-19T01:22:11.395 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/scan.o 2024-03-19T01:22:11.466 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/core.o 2024-03-19T01:22:11.824 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/bus.o 2024-03-19T01:22:11.899 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/uteval.o 2024-03-19T01:22:12.018 INFO:tasks.workunit.client.0.smithi069.stdout: AR kernel/built-in.a 2024-03-19T01:22:12.280 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/dd.o 2024-03-19T01:22:12.424 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/syscore.o 2024-03-19T01:22:12.498 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utglobal.o 2024-03-19T01:22:12.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:12 smithi086 ceph-mon[66829]: pgmap v359: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-03-19T01:22:12.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:12 smithi069 ceph-mon[31727]: pgmap v359: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-03-19T01:22:12.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:12 smithi163 ceph-mon[67428]: pgmap v359: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-03-19T01:22:12.710 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/intel_step.o 2024-03-19T01:22:12.835 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/filesystems.o 2024-03-19T01:22:12.913 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/uthex.o 2024-03-19T01:22:13.038 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/resource.o 2024-03-19T01:22:13.113 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/bridge/analogix/built-in.a 2024-03-19T01:22:13.176 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/bridge/cadence/built-in.a 2024-03-19T01:22:13.263 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/bridge/imx/built-in.a 2024-03-19T01:22:13.342 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/bridge/synopsys/built-in.a 2024-03-19T01:22:13.402 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/bridge/built-in.a 2024-03-19T01:22:13.528 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/dax/hmem/built-in.a 2024-03-19T01:22:13.555 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utids.o 2024-03-19T01:22:13.607 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/dax/built-in.a 2024-03-19T01:22:13.663 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/namespace.o 2024-03-19T01:22:13.669 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/intel_uncore.o 2024-03-19T01:22:13.684 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpi_processor.o 2024-03-19T01:22:14.043 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/hisilicon/built-in.a 2024-03-19T01:22:14.068 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma-buf/dma-buf.o 2024-03-19T01:22:14.068 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/seq_file.o 2024-03-19T01:22:14.081 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/xattr.o 2024-03-19T01:22:14.117 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/driver.o 2024-03-19T01:22:14.169 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utinit.o 2024-03-19T01:22:14.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:14 smithi086 ceph-mon[66829]: pgmap v360: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-03-19T01:22:14.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:14 smithi069 ceph-mon[31727]: pgmap v360: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-03-19T01:22:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:14 smithi163 ceph-mon[67428]: pgmap v360: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-03-19T01:22:14.829 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utlock.o 2024-03-19T01:22:14.829 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/class.o 2024-03-19T01:22:15.093 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/platform.o 2024-03-19T01:22:15.158 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utmath.o 2024-03-19T01:22:15.370 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/intel_wakeref.o 2024-03-19T01:22:15.489 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/mxsfb/built-in.a 2024-03-19T01:22:15.550 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/vlv_sideband.o 2024-03-19T01:22:15.791 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utmisc.o 2024-03-19T01:22:15.996 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma-buf/dma-fence.o 2024-03-19T01:22:16.206 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/processor_core.o 2024-03-19T01:22:16.214 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/cxl/core/built-in.a 2024-03-19T01:22:16.254 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/cxl/built-in.a 2024-03-19T01:22:16.320 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/macintosh/mac_hid.o 2024-03-19T01:22:16.421 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utmutex.o 2024-03-19T01:22:16.430 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/cpu.o 2024-03-19T01:22:16.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:16 smithi086 ceph-mon[66829]: pgmap v361: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-03-19T01:22:16.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:16 smithi069 ceph-mon[31727]: pgmap v361: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-03-19T01:22:16.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:16 smithi163 ceph-mon[67428]: pgmap v361: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-03-19T01:22:16.597 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/libfs.o 2024-03-19T01:22:16.605 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fs-writeback.o 2024-03-19T01:22:16.969 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/vlv_suspend.o 2024-03-19T01:22:17.076 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utnonansi.o 2024-03-19T01:22:17.128 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/pnode.o 2024-03-19T01:22:17.325 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma-buf/dma-fence-array.o 2024-03-19T01:22:17.669 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/firmware.o 2024-03-19T01:22:17.747 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utobject.o 2024-03-19T01:22:17.913 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/macintosh/built-in.a 2024-03-19T01:22:17.998 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/tiny/built-in.a 2024-03-19T01:22:18.018 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/soc/intel_dram.o 2024-03-19T01:22:18.029 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/processor_pdc.o 2024-03-19T01:22:18.052 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/xlnx/built-in.a 2024-03-19T01:22:18.197 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/scsi/pcmcia/built-in.a 2024-03-19T01:22:18.243 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma-buf/dma-fence-chain.o 2024-03-19T01:22:18.262 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi.o 2024-03-19T01:22:18.363 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:18 smithi163 ceph-mon[67428]: pgmap v362: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-03-19T01:22:18.376 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/init.o 2024-03-19T01:22:18.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:18 smithi086 ceph-mon[66829]: pgmap v362: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-03-19T01:22:18.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:18 smithi069 ceph-mon[31727]: pgmap v362: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-03-19T01:22:18.546 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma-buf/dma-fence-unwrap.o 2024-03-19T01:22:18.694 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utosi.o 2024-03-19T01:22:18.796 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/hosts.o 2024-03-19T01:22:18.998 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/splice.o 2024-03-19T01:22:19.010 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/gud/built-in.a 2024-03-19T01:22:19.039 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma-buf/dma-resv.o 2024-03-19T01:22:19.229 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/map.o 2024-03-19T01:22:19.425 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utownerid.o 2024-03-19T01:22:19.434 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/nvme/host/built-in.a 2024-03-19T01:22:19.515 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/nvme/target/built-in.a 2024-03-19T01:22:19.581 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/nvme/built-in.a 2024-03-19T01:22:19.668 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/libata-core.o 2024-03-19T01:22:19.737 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/soc/intel_gmch.o 2024-03-19T01:22:20.043 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_ioctl.o 2024-03-19T01:22:20.046 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/mdio-boardinfo.o 2024-03-19T01:22:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:20 smithi163 ceph-mon[67428]: pgmap v363: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 601 KiB/s wr, 55 op/s 2024-03-19T01:22:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:20 smithi163 ceph-mon[67428]: osdmap e106: 12 total, 12 up, 12 in 2024-03-19T01:22:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:20 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:22:20.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:20 smithi086 ceph-mon[66829]: pgmap v363: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 601 KiB/s wr, 55 op/s 2024-03-19T01:22:20.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:20 smithi086 ceph-mon[66829]: osdmap e106: 12 total, 12 up, 12 in 2024-03-19T01:22:20.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:20 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:22:20.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:20 smithi069 ceph-mon[31727]: pgmap v363: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 601 KiB/s wr, 55 op/s 2024-03-19T01:22:20.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:20 smithi069 ceph-mon[31727]: osdmap e106: 12 total, 12 up, 12 in 2024-03-19T01:22:20.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:20 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:22:20.561 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/devres.o 2024-03-19T01:22:20.584 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utpredef.o 2024-03-19T01:22:20.618 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/dma-buf/sync_file.o 2024-03-19T01:22:21.051 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/sync.o 2024-03-19T01:22:21.594 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsicam.o 2024-03-19T01:22:21.596 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/mdio_devres.o 2024-03-19T01:22:21.608 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/dma-buf/built-in.a 2024-03-19T01:22:21.700 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/soc/intel_pch.o 2024-03-19T01:22:21.862 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utresdecode.o 2024-03-19T01:22:21.952 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firewire/init_ohci1394_dma.o 2024-03-19T01:22:22.104 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/attribute_container.o 2024-03-19T01:22:22.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:22 smithi086 ceph-mon[66829]: pgmap v365: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 439 KiB/s wr, 41 op/s 2024-03-19T01:22:22.518 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/utimes.o 2024-03-19T01:22:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:22 smithi069 ceph-mon[31727]: pgmap v365: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 439 KiB/s wr, 41 op/s 2024-03-19T01:22:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:22 smithi163 ceph-mon[67428]: pgmap v365: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 439 KiB/s wr, 41 op/s 2024-03-19T01:22:22.827 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_error.o 2024-03-19T01:22:22.928 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utresrc.o 2024-03-19T01:22:22.964 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/phy.o 2024-03-19T01:22:23.046 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/transport_class.o 2024-03-19T01:22:23.181 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firewire/built-in.a 2024-03-19T01:22:23.286 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_memcpy.o 2024-03-19T01:22:23.298 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/d_path.o 2024-03-19T01:22:23.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:23 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:22:23.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:23 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:22:23.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:23 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:22:23.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:23 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:22:23.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:23 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:22:23.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:23 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:22:23.537 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utstate.o 2024-03-19T01:22:23.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:23 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:22:23.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:23 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:22:23.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:23 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:22:23.717 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utstring.o 2024-03-19T01:22:23.786 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/topology.o 2024-03-19T01:22:23.826 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_mm.o 2024-03-19T01:22:24.295 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/stack.o 2024-03-19T01:22:24.358 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/pse-pd/built-in.a 2024-03-19T01:22:24.365 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utstrsuppt.o 2024-03-19T01:22:24.403 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utstrtoul64.o 2024-03-19T01:22:24.406 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:24 smithi086 ceph-mon[66829]: pgmap v366: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.1 MiB/s wr, 62 op/s 2024-03-19T01:22:24.460 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/libata-scsi.o 2024-03-19T01:22:24.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:24 smithi069 ceph-mon[31727]: pgmap v366: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.1 MiB/s wr, 62 op/s 2024-03-19T01:22:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:24 smithi163 ceph-mon[67428]: pgmap v366: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.1 MiB/s wr, 62 op/s 2024-03-19T01:22:24.598 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/phy-c45.o 2024-03-19T01:22:24.668 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/container.o 2024-03-19T01:22:24.737 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_lib.o 2024-03-19T01:22:24.941 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fs_struct.o 2024-03-19T01:22:24.956 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/phy-core.o 2024-03-19T01:22:24.991 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utxface.o 2024-03-19T01:22:25.070 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_sw_fence.o 2024-03-19T01:22:25.293 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/property.o 2024-03-19T01:22:25.535 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utxfinit.o 2024-03-19T01:22:25.993 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_sw_fence_work.o 2024-03-19T01:22:26.119 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/statfs.o 2024-03-19T01:22:26.200 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utxferror.o 2024-03-19T01:22:26.200 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/cacheinfo.o 2024-03-19T01:22:26.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:26 smithi086 ceph-mon[66829]: pgmap v367: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-03-19T01:22:26.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:26 smithi069 ceph-mon[31727]: pgmap v367: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-03-19T01:22:26.557 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/phy_device.o 2024-03-19T01:22:26.558 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_syncmap.o 2024-03-19T01:22:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:26 smithi163 ceph-mon[67428]: pgmap v367: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-03-19T01:22:26.737 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/libata-eh.o 2024-03-19T01:22:26.819 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpica/utxfmutex.o 2024-03-19T01:22:26.908 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/constants.o 2024-03-19T01:22:27.099 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/swnode.o 2024-03-19T01:22:27.136 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/auxiliary.o 2024-03-19T01:22:27.353 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_user_extensions.o 2024-03-19T01:22:27.389 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fs_pin.o 2024-03-19T01:22:27.410 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/acpi/acpica/built-in.a 2024-03-19T01:22:27.872 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/ec.o 2024-03-19T01:22:27.947 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_ioc32.o 2024-03-19T01:22:28.034 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/libata-transport.o 2024-03-19T01:22:28.079 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_lib_dma.o 2024-03-19T01:22:28.129 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/nsfs.o 2024-03-19T01:22:28.326 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/devtmpfs.o 2024-03-19T01:22:28.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:28 smithi086 ceph-mon[66829]: pgmap v368: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-03-19T01:22:28.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:28 smithi069 ceph-mon[31727]: pgmap v368: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-03-19T01:22:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:28 smithi163 ceph-mon[67428]: pgmap v368: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-03-19T01:22:28.668 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/linkmode.o 2024-03-19T01:22:29.085 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_scan.o 2024-03-19T01:22:29.139 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_debugfs.o 2024-03-19T01:22:29.293 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fs_types.o 2024-03-19T01:22:29.340 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_debugfs_params.o 2024-03-19T01:22:29.423 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/libata-trace.o 2024-03-19T01:22:29.459 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/dock.o 2024-03-19T01:22:29.481 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/node.o 2024-03-19T01:22:29.644 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/mdio_bus.o 2024-03-19T01:22:30.069 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fs_context.o 2024-03-19T01:22:30.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:30 smithi069 ceph-mon[31727]: pgmap v369: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-03-19T01:22:30.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:30 smithi086 ceph-mon[66829]: pgmap v369: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-03-19T01:22:30.536 INFO:tasks.workunit.client.0.smithi069.stdout: GEN drivers/scsi/scsi_devinfo_tbl.c 2024-03-19T01:22:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:30 smithi163 ceph-mon[67428]: pgmap v369: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-03-19T01:22:30.583 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/pci_root.o 2024-03-19T01:22:30.584 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_devinfo.o 2024-03-19T01:22:30.748 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/libata-sata.o 2024-03-19T01:22:30.758 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/libata-sff.o 2024-03-19T01:22:30.976 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_display_debugfs.o 2024-03-19T01:22:31.310 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/module.o 2024-03-19T01:22:31.326 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/mdio_device.o 2024-03-19T01:22:31.541 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_sysctl.o 2024-03-19T01:22:31.560 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fs_parser.o 2024-03-19T01:22:32.112 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/platform-msi.o 2024-03-19T01:22:32.209 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/swphy.o 2024-03-19T01:22:32.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:22:32.221 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:22:32.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:32 smithi069 ceph-mon[31727]: pgmap v370: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-03-19T01:22:32.468 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_proc.o 2024-03-19T01:22:32.498 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/pci_link.o 2024-03-19T01:22:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:32 smithi086 ceph-mon[66829]: pgmap v370: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-03-19T01:22:32.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:32 smithi163 ceph-mon[67428]: pgmap v370: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-03-19T01:22:32.593 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fsopen.o 2024-03-19T01:22:32.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.656+0000 7fb6ee800640 1 -- 172.21.15.69:0/365816781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8075d50 msgr2=0x7fb6e8076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:32.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.656+0000 7fb6ee800640 1 --2- 172.21.15.69:0/365816781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8075d50 0x7fb6e8076130 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009930 tx=0x7fb6dc02bc20 comp rx=0 tx=0).stop 2024-03-19T01:22:32.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.657+0000 7fb6ee800640 1 -- 172.21.15.69:0/365816781 shutdown_connections 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.657+0000 7fb6ee800640 1 --2- 172.21.15.69:0/365816781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6e8112540 0x7fb6e8114930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.657+0000 7fb6ee800640 1 --2- 172.21.15.69:0/365816781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8076670 0x7fb6e8112000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.657+0000 7fb6ee800640 1 --2- 172.21.15.69:0/365816781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8075d50 0x7fb6e8076130 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.657+0000 7fb6ee800640 1 -- 172.21.15.69:0/365816781 >> 172.21.15.69:0/365816781 conn(0x7fb6e80ff330 msgr2=0x7fb6e8101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.657+0000 7fb6ee800640 1 -- 172.21.15.69:0/365816781 shutdown_connections 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.657+0000 7fb6ee800640 1 -- 172.21.15.69:0/365816781 wait complete. 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.658+0000 7fb6ee800640 1 Processor -- start 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.659+0000 7fb6ee800640 1 -- start start 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.659+0000 7fb6ee800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6e8075d50 0x7fb6e8072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.659+0000 7fb6ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8076670 0x7fb6e80733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.659+0000 7fb6ee800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8112540 0x7fb6e806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ee800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81180a0 con 0x7fb6e8112540 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ee800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8117f20 con 0x7fb6e8076670 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ee800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8118220 con 0x7fb6e8075d50 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ed400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6e8075d50 0x7fb6e8072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ed400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6e8075d50 0x7fb6e8072e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48016/0 (socket says 172.21.15.69:48016) 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ed400640 1 -- 172.21.15.69:0/3970666606 learned_addr learned my addr 172.21.15.69:0/3970666606 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6eca00640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8076670 0x7fb6e80733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ede00640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8112540 0x7fb6e806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ed400640 1 -- 172.21.15.69:0/3970666606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8076670 msgr2=0x7fb6e80733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ed400640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8076670 0x7fb6e80733b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ed400640 1 -- 172.21.15.69:0/3970666606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8112540 msgr2=0x7fb6e806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ed400640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8112540 0x7fb6e806bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ed400640 1 -- 172.21.15.69:0/3970666606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e806c830 con 0x7fb6e8075d50 2024-03-19T01:22:32.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.660+0000 7fb6ede00640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8112540 0x7fb6e806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:22:32.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.664+0000 7fb6ed400640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6e8075d50 0x7fb6e8072e70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc02c160 tx=0x7fb6dc004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:32.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.664+0000 7fb6e6000640 1 -- 172.21.15.69:0/3970666606 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc043020 con 0x7fb6e8075d50 2024-03-19T01:22:32.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.664+0000 7fb6ee800640 1 -- 172.21.15.69:0/3970666606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e806cac0 con 0x7fb6e8075d50 2024-03-19T01:22:32.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.664+0000 7fb6e6000640 1 -- 172.21.15.69:0/3970666606 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6dc004250 con 0x7fb6e8075d50 2024-03-19T01:22:32.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.664+0000 7fb6ee800640 1 -- 172.21.15.69:0/3970666606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e8070b70 con 0x7fb6e8075d50 2024-03-19T01:22:32.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.665+0000 7fb6e6000640 1 -- 172.21.15.69:0/3970666606 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc005220 con 0x7fb6e8075d50 2024-03-19T01:22:32.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.673+0000 7fb6e6000640 1 -- 172.21.15.69:0/3970666606 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fb6dc046020 con 0x7fb6e8075d50 2024-03-19T01:22:32.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.673+0000 7fb6f2b8b740 1 Processor -- start 2024-03-19T01:22:32.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.675+0000 7fb6e6000640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb6d00a29f0 0x7fb6d00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.676+0000 7fb6ede00640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb6d00a29f0 0x7fb6d00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.677+0000 7fb6f2b8b740 1 -- start start 2024-03-19T01:22:32.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.677+0000 7fb6f2b8b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978f2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.677+0000 7fb6e6000640 1 -- 172.21.15.69:0/3970666606 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fb6dc0bbb90 con 0x7fb6e8075d50 2024-03-19T01:22:32.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.677+0000 7fb6f2b8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978faef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.677+0000 7fb6f2b8b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978fb430 0x55ba978fd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.677+0000 7fb6f2b8b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ba978562c0 con 0x55ba978f1f30 2024-03-19T01:22:32.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.677+0000 7fb6f2b8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba977feb60 con 0x55ba978f2850 2024-03-19T01:22:32.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.677+0000 7fb6f2b8b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ba97852360 con 0x55ba978fb430 2024-03-19T01:22:32.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.678+0000 7fb6eca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978faef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.678+0000 7fb6eca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978faef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33262/0 (socket says 172.21.15.69:33262) 2024-03-19T01:22:32.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.678+0000 7fb6eca00640 1 -- 172.21.15.69:0/2346698248 learned_addr learned my addr 172.21.15.69:0/2346698248 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:32.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.678+0000 7fb6eca00640 1 -- 172.21.15.69:0/2346698248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978fb430 msgr2=0x55ba978fd820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:32.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.678+0000 7fb6eca00640 1 --2- 172.21.15.69:0/2346698248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978fb430 0x55ba978fd820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.678+0000 7fb6eca00640 1 -- 172.21.15.69:0/2346698248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 msgr2=0x55ba978f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:32.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.678+0000 7fb6eca00640 1 --2- 172.21.15.69:0/2346698248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.678+0000 7fb6eca00640 1 -- 172.21.15.69:0/2346698248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba976347d0 con 0x55ba978f2850 2024-03-19T01:22:32.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.679+0000 7fb6eca00640 1 --2- 172.21.15.69:0/2346698248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978faef0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009c20 tx=0x7fb6d4010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c256163240407 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:32.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.680+0000 7fb6da600640 1 -- 172.21.15.69:0/2346698248 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4016070 con 0x55ba978f2850 2024-03-19T01:22:32.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.680+0000 7fb6da600640 1 -- 172.21.15.69:0/2346698248 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d40035f0 con 0x55ba978f2850 2024-03-19T01:22:32.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.680+0000 7fb6da600640 1 -- 172.21.15.69:0/2346698248 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40126b0 con 0x55ba978f2850 2024-03-19T01:22:32.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.682+0000 7fb6ede00640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb6d00a29f0 0x7fb6d00a4eb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8069320 tx=0x7fb6e0009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:32.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.683+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/2346698248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 msgr2=0x55ba978faef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:32.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.683+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/2346698248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978faef0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009c20 tx=0x7fb6d4010740 comp rx=0 tx=0).stop 2024-03-19T01:22:32.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.686+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/2346698248 shutdown_connections 2024-03-19T01:22:32.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.686+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/2346698248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978fb430 0x55ba978fd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.686+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/2346698248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978faef0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.686+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/2346698248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.686+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/2346698248 >> 172.21.15.69:0/2346698248 conn(0x55ba978ec660 msgr2=0x55ba978eca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:32.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.686+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/2346698248 shutdown_connections 2024-03-19T01:22:32.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.686+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/2346698248 wait complete. 2024-03-19T01:22:32.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.687+0000 7fb6f2b8b740 1 -- start start 2024-03-19T01:22:32.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.687+0000 7fb6f2b8b740 1 Processor -- start 2024-03-19T01:22:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.691+0000 7fb6f2b8b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978f1f30 0x55ba978f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.691+0000 7fb6f2b8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.691+0000 7fb6f2b8b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978fb430 0x55ba97907720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.691+0000 7fb6f2b8b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ba978fe220 con 0x55ba978fb430 2024-03-19T01:22:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.691+0000 7fb6f2b8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba97811fc0 con 0x55ba978f2850 2024-03-19T01:22:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.691+0000 7fb6f2b8b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ba978fe3a0 con 0x55ba978f1f30 2024-03-19T01:22:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.691+0000 7fb6f2b8b740 10 client.0 _ll_register_callbacks cb 0x55ba97811990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.692+0000 7fb6ed400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978f1f30 0x55ba978f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.692+0000 7fb6ed400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978f1f30 0x55ba978f86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48032/0 (socket says 172.21.15.69:48032) 2024-03-19T01:22:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.692+0000 7fb6ed400640 1 -- 172.21.15.69:0/1090371990 learned_addr learned my addr 172.21.15.69:0/1090371990 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.692+0000 7fb6eca00640 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.692+0000 7fb6ede00640 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978fb430 0x55ba97907720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.692+0000 7fb6ed400640 1 -- 172.21.15.69:0/1090371990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 msgr2=0x55ba978f8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.692+0000 7fb6ed400640 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978f8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.692+0000 7fb6ed400640 1 -- 172.21.15.69:0/1090371990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978fb430 msgr2=0x55ba97907720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.692+0000 7fb6ed400640 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978fb430 0x55ba97907720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.692+0000 7fb6ed400640 1 -- 172.21.15.69:0/1090371990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9756a3f0 con 0x55ba978f1f30 2024-03-19T01:22:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.692+0000 7fb6ed400640 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978f1f30 0x55ba978f86a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc02c810 tx=0x7fb6dc0b85f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23965f0dbc1c0a34 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.693+0000 7fb6cea00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:22:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.693+0000 7fb6cea00640 1 -- 172.21.15.69:0/1090371990 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0ba520 con 0x55ba978f1f30 2024-03-19T01:22:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.693+0000 7fb6cea00640 1 -- 172.21.15.69:0/1090371990 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fb6dc03c350 con 0x55ba978f1f30 2024-03-19T01:22:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.693+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/1090371990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6d40035f0 con 0x55ba978f1f30 2024-03-19T01:22:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.697+0000 7fb6cea00640 1 -- 172.21.15.69:0/1090371990 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb6dc0d5030 con 0x55ba978f1f30 2024-03-19T01:22:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.700+0000 7fb6f2b8b740 10 client.27802 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.700+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/1090371990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ba9752a0f0 con 0x55ba978f1f30 2024-03-19T01:22:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.701+0000 7fb6cea00640 1 -- 172.21.15.69:0/1090371990 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d40035f0 con 0x55ba978f1f30 2024-03-19T01:22:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.701+0000 7fb6cea00640 1 -- 172.21.15.69:0/1090371990 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fb6dc0eec30 con 0x55ba978f1f30 2024-03-19T01:22:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.701+0000 7fb6f2b8b740 10 client.27802 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.701+0000 7fb6f2b8b740 10 client.27802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:22:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.701+0000 7fb6f2b8b740 20 client.27802 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.701+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba9790b9e0 0x55ba9790de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.701+0000 7fb6f2b8b740 4 client.27802 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:22:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.701+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/1090371990 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba977f7610 con 0x55ba9790b9e0 2024-03-19T01:22:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.701+0000 7fb6eca00640 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba9790b9e0 0x55ba9790de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.702+0000 7fb6eca00640 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba9790b9e0 0x55ba9790de00 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254697a065ff5dc8 server_cookie=502f2a0fd029b179 in_seq=0 out_seq=0 2024-03-19T01:22:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.702+0000 7fb6cea00640 10 client.27802 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.703+0000 7fb6cea00640 1 -- 172.21.15.69:0/1090371990 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ba977f7610 con 0x55ba9790b9e0 2024-03-19T01:22:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.703+0000 7fb6cea00640 10 client.27802 handle_command_reply: tid=0 2024-03-19T01:22:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.703+0000 7fb6f2b8b740 1 client.27802 shutdown 2024-03-19T01:22:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.704+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/1090371990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978f1f30 msgr2=0x55ba978f86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.704+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978f1f30 0x55ba978f86a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc02c810 tx=0x7fb6dc0b85f0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.704+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/1090371990 shutdown_connections 2024-03-19T01:22:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.704+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba9790b9e0 0x55ba9790de00 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.704+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978fb430 0x55ba97907720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.704+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978f8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.704+0000 7fb6cea00640 0 client.27802 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.704+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/1090371990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978f1f30 0x55ba978f86a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.704+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/1090371990 >> 172.21.15.69:0/1090371990 conn(0x55ba978ec660 msgr2=0x55ba978ee3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.704+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/1090371990 shutdown_connections 2024-03-19T01:22:32.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.708+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/1090371990 wait complete. 2024-03-19T01:22:32.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.708+0000 7fb6f2b8b740 20 client.27802 trim_cache size 0 max 16384 2024-03-19T01:22:32.717 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/physical_location.o 2024-03-19T01:22:32.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.785+0000 7fb6f2b8b740 1 Processor -- start 2024-03-19T01:22:32.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.789+0000 7fb6f2b8b740 1 -- start start 2024-03-19T01:22:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.789+0000 7fb6f2b8b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978ee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.789+0000 7fb6f2b8b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978f2850 0x55ba979136b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.789+0000 7fb6f2b8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978fb430 0x55ba97903670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.789+0000 7fb6f2b8b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c80100b0 con 0x55ba978f1f30 2024-03-19T01:22:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.789+0000 7fb6f2b8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8011bf0 con 0x55ba978fb430 2024-03-19T01:22:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.789+0000 7fb6f2b8b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8010780 con 0x55ba978f2850 2024-03-19T01:22:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.789+0000 7fb6ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978fb430 0x55ba97903670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.790+0000 7fb6ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978fb430 0x55ba97903670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33276/0 (socket says 172.21.15.69:33276) 2024-03-19T01:22:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.790+0000 7fb6ede00640 1 -- 172.21.15.69:0/607217355 learned_addr learned my addr 172.21.15.69:0/607217355 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.790+0000 7fb6ed400640 1 --2- 172.21.15.69:0/607217355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978ee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.790+0000 7fb6ede00640 1 -- 172.21.15.69:0/607217355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978f2850 msgr2=0x55ba979136b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.790+0000 7fb6ede00640 1 --2- 172.21.15.69:0/607217355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978f2850 0x55ba979136b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.790+0000 7fb6ede00640 1 -- 172.21.15.69:0/607217355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 msgr2=0x55ba978ee1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.790+0000 7fb6ede00640 1 --2- 172.21.15.69:0/607217355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978ee1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.790+0000 7fb6ede00640 1 -- 172.21.15.69:0/607217355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba978f16d0 con 0x55ba978fb430 2024-03-19T01:22:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.790+0000 7fb6ed400640 1 --2- 172.21.15.69:0/607217355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978ee1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:22:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.791+0000 7fb6ede00640 1 --2- 172.21.15.69:0/607217355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978fb430 0x55ba97903670 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0011790 tx=0x7fb6e0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8834ca70e2dcaeec server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.791+0000 7fb6d9c00640 1 -- 172.21.15.69:0/607217355 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0003970 con 0x55ba978fb430 2024-03-19T01:22:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.791+0000 7fb6d9c00640 1 -- 172.21.15.69:0/607217355 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e0023070 con 0x55ba978fb430 2024-03-19T01:22:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.791+0000 7fb6d9c00640 1 -- 172.21.15.69:0/607217355 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00043a0 con 0x55ba978fb430 2024-03-19T01:22:32.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.792+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/607217355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978fb430 msgr2=0x55ba97903670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:32.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.792+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/607217355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978fb430 0x55ba97903670 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0011790 tx=0x7fb6e0013000 comp rx=0 tx=0).stop 2024-03-19T01:22:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.792+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/607217355 shutdown_connections 2024-03-19T01:22:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.792+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/607217355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978fb430 0x55ba97903670 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.792+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/607217355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978f2850 0x55ba979136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.792+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/607217355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978ee1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.792+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/607217355 >> 172.21.15.69:0/607217355 conn(0x55ba978ec660 msgr2=0x55ba978edab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.792+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/607217355 shutdown_connections 2024-03-19T01:22:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.792+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/607217355 wait complete. 2024-03-19T01:22:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.793+0000 7fb6f2b8b740 1 -- start start 2024-03-19T01:22:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.794+0000 7fb6f2b8b740 1 Processor -- start 2024-03-19T01:22:32.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.796+0000 7fb6f2b8b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978ff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.797+0000 7fb6ed400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.797+0000 7fb6ed400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978ff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40276/0 (socket says 172.21.15.69:40276) 2024-03-19T01:22:32.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.797+0000 7fb6ed400640 1 -- 172.21.15.69:0/3031356884 learned_addr learned my addr 172.21.15.69:0/3031356884 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:32.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.797+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978ff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.798+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978fb430 0x55ba978ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.798+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/3031356884 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8011520 con 0x55ba978f1f30 2024-03-19T01:22:32.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.798+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/3031356884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8010e50 con 0x55ba978f2850 2024-03-19T01:22:32.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.798+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/3031356884 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c800e090 con 0x55ba978fb430 2024-03-19T01:22:32.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.798+0000 7fb6ede00640 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978fb430 0x55ba978ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.799+0000 7fb6ede00640 1 -- 172.21.15.69:0/3031356884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 msgr2=0x55ba978ff640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:22:32.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.799+0000 7fb6ede00640 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978ff640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.799+0000 7fb6ede00640 1 -- 172.21.15.69:0/3031356884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 msgr2=0x55ba978ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:32.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.799+0000 7fb6ede00640 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.799+0000 7fb6ede00640 1 -- 172.21.15.69:0/3031356884 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba97970e90 con 0x55ba978fb430 2024-03-19T01:22:32.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.799+0000 7fb6ed400640 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:22:32.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.800+0000 7fb6ede00640 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978fb430 0x55ba978ffb80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0011790 tx=0x7fb6e0009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8333790743fcb67 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:32.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.800+0000 7fb6f2b8b740 10 client.0 _ll_register_callbacks cb 0x7fb6dc0be3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:32.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.800+0000 7fb6cea00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:22:32.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.800+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/3031356884 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e0023070 con 0x55ba978fb430 2024-03-19T01:22:32.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.800+0000 7fb6cea00640 1 -- 172.21.15.69:0/3031356884 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0036060 con 0x55ba978fb430 2024-03-19T01:22:32.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.800+0000 7fb6cea00640 1 -- 172.21.15.69:0/3031356884 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fb6e0004c50 con 0x55ba978fb430 2024-03-19T01:22:32.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.800+0000 7fb6cea00640 1 -- 172.21.15.69:0/3031356884 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb6e0003970 con 0x55ba978fb430 2024-03-19T01:22:32.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.800+0000 7fb6f2b8b740 10 client.27805 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:32.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.800+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/3031356884 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=59}) -- 0x55ba97971dd0 con 0x55ba978fb430 2024-03-19T01:22:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.801+0000 7fb6cea00640 1 -- 172.21.15.69:0/3031356884 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fb6e0003b10 con 0x55ba978fb430 2024-03-19T01:22:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.801+0000 7fb6f2b8b740 10 client.27805 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.801+0000 7fb6f2b8b740 10 client.27805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:22:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.801+0000 7fb6f2b8b740 20 client.27805 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.801+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba9790b9e0 0x55ba979745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.801+0000 7fb6f2b8b740 4 client.27805 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:22:32.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.801+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/3031356884 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba97863800 con 0x55ba9790b9e0 2024-03-19T01:22:32.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.805+0000 7fb6ed400640 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba9790b9e0 0x55ba979745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:32.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.805+0000 7fb6ed400640 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba9790b9e0 0x55ba979745c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ab08895cd13b5d server_cookie=37b33083cba04e97 in_seq=0 out_seq=0 2024-03-19T01:22:32.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.807+0000 7fb6cea00640 10 client.27805 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:32.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.807+0000 7fb6cea00640 1 -- 172.21.15.69:0/3031356884 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ba97863800 con 0x55ba9790b9e0 2024-03-19T01:22:32.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.807+0000 7fb6cea00640 10 client.27805 handle_command_reply: tid=0 2024-03-19T01:22:32.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.807+0000 7fb6f2b8b740 1 client.27805 shutdown 2024-03-19T01:22:32.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.808+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/3031356884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978fb430 msgr2=0x55ba978ffb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:32.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.808+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978fb430 0x55ba978ffb80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0011790 tx=0x7fb6e0009b90 comp rx=0 tx=0).stop 2024-03-19T01:22:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.808+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/3031356884 shutdown_connections 2024-03-19T01:22:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.808+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ba9790b9e0 0x55ba979745c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.808+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ba978fb430 0x55ba978ffb80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.808+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba978f2850 0x55ba978ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.808+0000 7fb6f2b8b740 1 --2- 172.21.15.69:0/3031356884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba978f1f30 0x55ba978ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.808+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/3031356884 >> 172.21.15.69:0/3031356884 conn(0x55ba978ec660 msgr2=0x55ba978ee340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.808+0000 7fb6cea00640 0 client.27805 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.808+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/3031356884 shutdown_connections 2024-03-19T01:22:32.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.808+0000 7fb6f2b8b740 1 -- 172.21.15.69:0/3031356884 wait complete. 2024-03-19T01:22:32.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.812+0000 7fb6f2b8b740 20 client.27805 trim_cache size 0 max 16384 2024-03-19T01:22:32.813 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:22:32.813 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:22:32.813 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:22:32.813 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:22:32.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.814+0000 7fb6ee800640 1 -- 172.21.15.69:0/3970666606 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb6d00a29f0 msgr2=0x7fb6d00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:32.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.814+0000 7fb6ee800640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb6d00a29f0 0x7fb6d00a4eb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8069320 tx=0x7fb6e0009500 comp rx=0 tx=0).stop 2024-03-19T01:22:32.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.814+0000 7fb6ee800640 1 -- 172.21.15.69:0/3970666606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6e8075d50 msgr2=0x7fb6e8072e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:32.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.814+0000 7fb6ee800640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6e8075d50 0x7fb6e8072e70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc02c160 tx=0x7fb6dc004970 comp rx=0 tx=0).stop 2024-03-19T01:22:32.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.814+0000 7fb6ee800640 1 -- 172.21.15.69:0/3970666606 shutdown_connections 2024-03-19T01:22:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.814+0000 7fb6ee800640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8112540 0x7fb6e806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.814+0000 7fb6ee800640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb6d00a29f0 0x7fb6d00a4eb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.814+0000 7fb6ee800640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e8076670 0x7fb6e80733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.814+0000 7fb6ee800640 1 --2- 172.21.15.69:0/3970666606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6e8075d50 0x7fb6e8072e70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.814+0000 7fb6ee800640 1 -- 172.21.15.69:0/3970666606 >> 172.21.15.69:0/3970666606 conn(0x7fb6e80ff330 msgr2=0x7fb6e8101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.815+0000 7fb6ee800640 1 -- 172.21.15.69:0/3970666606 shutdown_connections 2024-03-19T01:22:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:32.815+0000 7fb6ee800640 1 -- 172.21.15.69:0/3970666606 wait complete. 2024-03-19T01:22:32.875 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:22:32.875 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:22:32.939 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cdrom/cdrom.o 2024-03-19T01:22:33.020 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/libata-pmp.o 2024-03-19T01:22:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.331+0000 7effa5a00640 1 -- 172.21.15.69:0/1075377812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa0112610 msgr2=0x7effa0114a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.331+0000 7effa5a00640 1 --2- 172.21.15.69:0/1075377812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa0112610 0x7effa0114a00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7eff90009860 tx=0x7eff9002bb70 comp rx=0 tx=0).stop 2024-03-19T01:22:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.334+0000 7effa5a00640 1 -- 172.21.15.69:0/1075377812 shutdown_connections 2024-03-19T01:22:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.334+0000 7effa5a00640 1 --2- 172.21.15.69:0/1075377812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa0112610 0x7effa0114a00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.334+0000 7effa5a00640 1 --2- 172.21.15.69:0/1075377812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effa0100210 0x7effa01120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.334+0000 7effa5a00640 1 --2- 172.21.15.69:0/1075377812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa00ff8f0 0x7effa00ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.334+0000 7effa5a00640 1 -- 172.21.15.69:0/1075377812 >> 172.21.15.69:0/1075377812 conn(0x7effa0076880 msgr2=0x7effa00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.337+0000 7effa5a00640 1 -- 172.21.15.69:0/1075377812 shutdown_connections 2024-03-19T01:22:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.337+0000 7effa5a00640 1 -- 172.21.15.69:0/1075377812 wait complete. 2024-03-19T01:22:33.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.338+0000 7effa5a00640 1 Processor -- start 2024-03-19T01:22:33.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.343+0000 7effa5a00640 1 -- start start 2024-03-19T01:22:33.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.343+0000 7effa5a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa00ff8f0 0x7effa019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:33.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.343+0000 7effa5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa0100210 0x7effa019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:33.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.343+0000 7effa5a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effa0112610 0x7effa01a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:33.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.343+0000 7effa5a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7effa0118170 con 0x7effa0112610 2024-03-19T01:22:33.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.343+0000 7effa5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7effa0117ff0 con 0x7effa0100210 2024-03-19T01:22:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.343+0000 7effa5a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7effa01182f0 con 0x7effa00ff8f0 2024-03-19T01:22:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.344+0000 7eff9ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa00ff8f0 0x7effa019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.344+0000 7eff9ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa00ff8f0 0x7effa019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48056/0 (socket says 172.21.15.69:48056) 2024-03-19T01:22:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.344+0000 7eff9ea00640 1 -- 172.21.15.69:0/3449801976 learned_addr learned my addr 172.21.15.69:0/3449801976 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.344+0000 7eff9e000640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa0100210 0x7effa019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.344+0000 7eff9ea00640 1 -- 172.21.15.69:0/3449801976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa0100210 msgr2=0x7effa019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.344+0000 7eff9ea00640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa0100210 0x7effa019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.344+0000 7eff9ea00640 1 -- 172.21.15.69:0/3449801976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effa0112610 msgr2=0x7effa01a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.344+0000 7eff9ea00640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effa0112610 0x7effa01a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.344+0000 7eff9ea00640 1 -- 172.21.15.69:0/3449801976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa01a3580 con 0x7effa00ff8f0 2024-03-19T01:22:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.345+0000 7eff9ea00640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa00ff8f0 0x7effa019e610 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7eff98009d50 tx=0x7eff9800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:33.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.345+0000 7eff97400640 1 -- 172.21.15.69:0/3449801976 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98010630 con 0x7effa00ff8f0 2024-03-19T01:22:33.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.345+0000 7eff97400640 1 -- 172.21.15.69:0/3449801976 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff98010cb0 con 0x7effa00ff8f0 2024-03-19T01:22:33.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.345+0000 7eff97400640 1 -- 172.21.15.69:0/3449801976 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98018d50 con 0x7effa00ff8f0 2024-03-19T01:22:33.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.345+0000 7effa5a00640 1 -- 172.21.15.69:0/3449801976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa01a3870 con 0x7effa00ff8f0 2024-03-19T01:22:33.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.345+0000 7effa5a00640 1 -- 172.21.15.69:0/3449801976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa01078b0 con 0x7effa00ff8f0 2024-03-19T01:22:33.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.353+0000 7effa5a00640 1 -- 172.21.15.69:0/3449801976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa0068a60 con 0x7effa00ff8f0 2024-03-19T01:22:33.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.358+0000 7eff97400640 1 -- 172.21.15.69:0/3449801976 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7eff9801f020 con 0x7effa00ff8f0 2024-03-19T01:22:33.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.360+0000 7eff97400640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7eff840a2b40 0x7eff840a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:33.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.360+0000 7eff97400640 1 -- 172.21.15.69:0/3449801976 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7eff980cf690 con 0x7effa00ff8f0 2024-03-19T01:22:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.361+0000 7eff9e000640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7eff840a2b40 0x7eff840a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:33.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.366+0000 7eff9e000640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7eff840a2b40 0x7eff840a5000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7effa01121c0 tx=0x7eff8c00b410 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:33.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:33.369+0000 7eff97400640 1 -- 172.21.15.69:0/3449801976 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7eff9809bfa0 con 0x7effa00ff8f0 2024-03-19T01:22:33.468 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_pipe_crc.o 2024-03-19T01:22:33.489 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/fixed_phy.o 2024-03-19T01:22:33.599 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/init.o 2024-03-19T01:22:33.618 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/base/trace.o 2024-03-19T01:22:33.766 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/pci_irq.o 2024-03-19T01:22:33.812 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_debugfs.o 2024-03-19T01:22:34.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.061+0000 7effa5a00640 1 -- 172.21.15.69:0/3449801976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effa00fe370 con 0x7effa00ff8f0 2024-03-19T01:22:34.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.080+0000 7eff97400640 1 -- 172.21.15.69:0/3449801976 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11930 (secure 0 0 0) 0x7eff9809b8a0 con 0x7effa00ff8f0 2024-03-19T01:22:34.102 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 59 2024-03-19T01:22:34.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.089+0000 7effa5a00640 1 -- 172.21.15.69:0/3449801976 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7eff840a2b40 msgr2=0x7eff840a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:34.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.089+0000 7effa5a00640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7eff840a2b40 0x7eff840a5000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7effa01121c0 tx=0x7eff8c00b410 comp rx=0 tx=0).stop 2024-03-19T01:22:34.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.089+0000 7effa5a00640 1 -- 172.21.15.69:0/3449801976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa00ff8f0 msgr2=0x7effa019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:34.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.089+0000 7effa5a00640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa00ff8f0 0x7effa019e610 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7eff98009d50 tx=0x7eff9800c6a0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.093+0000 7effa5a00640 1 -- 172.21.15.69:0/3449801976 shutdown_connections 2024-03-19T01:22:34.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.093+0000 7effa5a00640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effa0112610 0x7effa01a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.093+0000 7effa5a00640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7eff840a2b40 0x7eff840a5000 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.093+0000 7effa5a00640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effa0100210 0x7effa019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.093+0000 7effa5a00640 1 --2- 172.21.15.69:0/3449801976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa00ff8f0 0x7effa019e610 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.093+0000 7effa5a00640 1 -- 172.21.15.69:0/3449801976 >> 172.21.15.69:0/3449801976 conn(0x7effa0076880 msgr2=0x7effa0110f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:34.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.093+0000 7effa5a00640 1 -- 172.21.15.69:0/3449801976 shutdown_connections 2024-03-19T01:22:34.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.093+0000 7effa5a00640 1 -- 172.21.15.69:0/3449801976 wait complete. 2024-03-19T01:22:34.103 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:22:34.103 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":59,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:20:08.600901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:22:34.145 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:22:34.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:34 smithi086 ceph-mon[66829]: pgmap v371: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.2 MiB/s wr, 79 op/s 2024-03-19T01:22:34.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:34 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3449801976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:22:34.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:34 smithi069 ceph-mon[31727]: pgmap v371: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.2 MiB/s wr, 79 op/s 2024-03-19T01:22:34.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:34 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3449801976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:22:34.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:34 smithi163 ceph-mon[67428]: pgmap v371: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.2 MiB/s wr, 79 op/s 2024-03-19T01:22:34.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:34 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3449801976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:22:34.715 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_trace.o 2024-03-19T01:22:34.742 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/libata-acpi.o 2024-03-19T01:22:34.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.895+0000 7f816ce00640 1 -- 172.21.15.69:0/2720440885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f816810e070 msgr2=0x7f816810e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:34.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.895+0000 7f816ce00640 1 --2- 172.21.15.69:0/2720440885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f816810e070 0x7f816810e4f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8158009860 tx=0x7f815802bb70 comp rx=0 tx=0).stop 2024-03-19T01:22:34.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.898+0000 7f816ce00640 1 -- 172.21.15.69:0/2720440885 shutdown_connections 2024-03-19T01:22:34.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.898+0000 7f816ce00640 1 --2- 172.21.15.69:0/2720440885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f816811b000 0x7f816811d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.898+0000 7f816ce00640 1 --2- 172.21.15.69:0/2720440885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f816810e070 0x7f816810e4f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.898+0000 7f816ce00640 1 --2- 172.21.15.69:0/2720440885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8168110040 0x7f816810db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.898+0000 7f816ce00640 1 -- 172.21.15.69:0/2720440885 >> 172.21.15.69:0/2720440885 conn(0x7f816806c900 msgr2=0x7f816806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:34.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.898+0000 7f816ce00640 1 -- 172.21.15.69:0/2720440885 shutdown_connections 2024-03-19T01:22:34.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.898+0000 7f816ce00640 1 -- 172.21.15.69:0/2720440885 wait complete. 2024-03-19T01:22:34.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.899+0000 7f816ce00640 1 Processor -- start 2024-03-19T01:22:34.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.900+0000 7f816ce00640 1 -- start start 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.904+0000 7f816ce00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f816810e070 0x7f8168118630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.904+0000 7f816ce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8168110040 0x7f81681116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.904+0000 7f816ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f816811b000 0x7f8168111c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.904+0000 7f816ce00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81681209f0 con 0x7f816810e070 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.904+0000 7f816ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8168120870 con 0x7f816811b000 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.904+0000 7f816ce00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8168120b70 con 0x7f8168110040 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.904+0000 7f8165400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8168110040 0x7f81681116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.904+0000 7f8165400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8168110040 0x7f81681116e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48068/0 (socket says 172.21.15.69:48068) 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.904+0000 7f8165400640 1 -- 172.21.15.69:0/335330372 learned_addr learned my addr 172.21.15.69:0/335330372 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.905+0000 7f8165e00640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f816810e070 0x7f8168118630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.905+0000 7f8166800640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f816811b000 0x7f8168111c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.905+0000 7f8165400640 1 -- 172.21.15.69:0/335330372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f816811b000 msgr2=0x7f8168111c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:34.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.905+0000 7f8165400640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f816811b000 0x7f8168111c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.905+0000 7f8165400640 1 -- 172.21.15.69:0/335330372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f816810e070 msgr2=0x7f8168118630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:34.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.905+0000 7f8165400640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f816810e070 0x7f8168118630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.905+0000 7f8165400640 1 -- 172.21.15.69:0/335330372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8168112260 con 0x7f8168110040 2024-03-19T01:22:34.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.905+0000 7f8166800640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f816811b000 0x7f8168111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:22:34.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.905+0000 7f8165e00640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f816810e070 0x7f8168118630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:22:34.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.905+0000 7f8165400640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8168110040 0x7f81681116e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8158037040 tx=0x7f81580045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:34.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.906+0000 7f815ea00640 1 -- 172.21.15.69:0/335330372 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158041020 con 0x7f8168110040 2024-03-19T01:22:34.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.906+0000 7f815ea00640 1 -- 172.21.15.69:0/335330372 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f815802c9f0 con 0x7f8168110040 2024-03-19T01:22:34.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.906+0000 7f815ea00640 1 -- 172.21.15.69:0/335330372 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158039040 con 0x7f8168110040 2024-03-19T01:22:34.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.906+0000 7f816ce00640 1 -- 172.21.15.69:0/335330372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81681124f0 con 0x7f8168110040 2024-03-19T01:22:34.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.906+0000 7f816ce00640 1 -- 172.21.15.69:0/335330372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81681163f0 con 0x7f8168110040 2024-03-19T01:22:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.914+0000 7f816fb9a740 1 Processor -- start 2024-03-19T01:22:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.916+0000 7f815ea00640 1 -- 172.21.15.69:0/335330372 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f8158044020 con 0x7f8168110040 2024-03-19T01:22:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.917+0000 7f816fb9a740 1 -- start start 2024-03-19T01:22:34.919 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/phy/realtek.o 2024-03-19T01:22:34.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.917+0000 7f816fb9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c315a20 0x561e2c315e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.917+0000 7f816fb9a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c316340 0x561e2c31e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.917+0000 7f816fb9a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c31ef20 0x561e2c321310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.917+0000 7f816fb9a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561e2c27a380 con 0x561e2c316340 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.917+0000 7f816fb9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e2c222d20 con 0x561e2c315a20 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.917+0000 7f816fb9a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561e2c276420 con 0x561e2c31ef20 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.918+0000 7f8165e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c316340 0x561e2c31e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.918+0000 7f8165e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c316340 0x561e2c31e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40324/0 (socket says 172.21.15.69:40324) 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.918+0000 7f8165e00640 1 -- 172.21.15.69:0/79206924 learned_addr learned my addr 172.21.15.69:0/79206924 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.918+0000 7f8166800640 1 --2- 172.21.15.69:0/79206924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c315a20 0x561e2c315e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.919+0000 7f8165e00640 1 -- 172.21.15.69:0/79206924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c31ef20 msgr2=0x561e2c321310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.919+0000 7f8165e00640 1 --2- 172.21.15.69:0/79206924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c31ef20 0x561e2c321310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.919+0000 7f8165e00640 1 -- 172.21.15.69:0/79206924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c315a20 msgr2=0x561e2c315e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:34.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.919+0000 7f8165e00640 1 --2- 172.21.15.69:0/79206924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c315a20 0x561e2c315e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.919+0000 7f8165e00640 1 -- 172.21.15.69:0/79206924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e2bf8e3f0 con 0x561e2c316340 2024-03-19T01:22:34.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.919+0000 7f8165e00640 1 --2- 172.21.15.69:0/79206924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c316340 0x561e2c31e9e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8154009d00 tx=0x7f81540117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602080d46b54292 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:34.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.920+0000 7f815ea00640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f814c0a2a30 0x7f814c0a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.920+0000 7f815ea00640 1 -- 172.21.15.69:0/335330372 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7f81580f3170 con 0x7f8168110040 2024-03-19T01:22:34.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.920+0000 7f8166800640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f814c0a2a30 0x7f814c0a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.923+0000 7f8153400640 1 -- 172.21.15.69:0/79206924 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154012450 con 0x561e2c316340 2024-03-19T01:22:34.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.923+0000 7f8153400640 1 -- 172.21.15.69:0/79206924 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8154012ad0 con 0x561e2c316340 2024-03-19T01:22:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.923+0000 7f8153400640 1 -- 172.21.15.69:0/79206924 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815400fc30 con 0x561e2c316340 2024-03-19T01:22:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.924+0000 7f816fb9a740 1 -- 172.21.15.69:0/79206924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c316340 msgr2=0x561e2c31e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.924+0000 7f816fb9a740 1 --2- 172.21.15.69:0/79206924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c316340 0x561e2c31e9e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8154009d00 tx=0x7f81540117b0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.928+0000 7f8166800640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f814c0a2a30 0x7f814c0a4ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8160009370 tx=0x7f81600092e0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.928+0000 7f816fb9a740 1 -- 172.21.15.69:0/79206924 shutdown_connections 2024-03-19T01:22:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.928+0000 7f816fb9a740 1 --2- 172.21.15.69:0/79206924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c31ef20 0x561e2c321310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.928+0000 7f816fb9a740 1 --2- 172.21.15.69:0/79206924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c316340 0x561e2c31e9e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.928+0000 7f816fb9a740 1 --2- 172.21.15.69:0/79206924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c315a20 0x561e2c315e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.928+0000 7f816fb9a740 1 -- 172.21.15.69:0/79206924 >> 172.21.15.69:0/79206924 conn(0x561e2c310150 msgr2=0x561e2c310530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.929+0000 7f816fb9a740 1 -- 172.21.15.69:0/79206924 shutdown_connections 2024-03-19T01:22:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.929+0000 7f816fb9a740 1 -- 172.21.15.69:0/79206924 wait complete. 2024-03-19T01:22:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.929+0000 7f816fb9a740 1 -- start start 2024-03-19T01:22:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.930+0000 7f816fb9a740 1 Processor -- start 2024-03-19T01:22:34.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.930+0000 7f816fb9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c315a20 0x561e2c31bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.931+0000 7f816fb9a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c316340 0x561e2c31c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.931+0000 7f8165e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c315a20 0x561e2c31bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.931+0000 7f8165e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c315a20 0x561e2c31bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33348/0 (socket says 172.21.15.69:33348) 2024-03-19T01:22:34.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.931+0000 7f8165e00640 1 -- 172.21.15.69:0/3488918601 learned_addr learned my addr 172.21.15.69:0/3488918601 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:34.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.931+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c31ef20 0x561e2c31ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.931+0000 7f8165400640 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c316340 0x561e2c31c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.931+0000 7f816fb9a740 1 -- 172.21.15.69:0/3488918601 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561e2c321d10 con 0x561e2c31ef20 2024-03-19T01:22:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.931+0000 7f816fb9a740 1 -- 172.21.15.69:0/3488918601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e2c236160 con 0x561e2c315a20 2024-03-19T01:22:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.931+0000 7f816fb9a740 1 -- 172.21.15.69:0/3488918601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561e2c321e90 con 0x561e2c316340 2024-03-19T01:22:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.932+0000 7f8165400640 1 -- 172.21.15.69:0/3488918601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c315a20 msgr2=0x561e2c31bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.932+0000 7f8165400640 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c315a20 0x561e2c31bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.932+0000 7f8165400640 1 -- 172.21.15.69:0/3488918601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c31ef20 msgr2=0x561e2c31ca10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.932+0000 7f8165400640 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c31ef20 0x561e2c31ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.932+0000 7f8165400640 1 -- 172.21.15.69:0/3488918601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e2c0587d0 con 0x561e2c316340 2024-03-19T01:22:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.932+0000 7f8165400640 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c316340 0x561e2c31c4d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f81580ed0c0 tx=0x7f8158032fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46f9f9e66f247f88 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:34.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.932+0000 7f8147e00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:22:34.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.933+0000 7f8147e00640 1 -- 172.21.15.69:0/3488918601 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81580f4060 con 0x561e2c316340 2024-03-19T01:22:34.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.933+0000 7f816fb9a740 10 client.0 _ll_register_callbacks cb 0x561e2c272170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:34.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.933+0000 7f8147e00640 1 -- 172.21.15.69:0/3488918601 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7f81580b8700 con 0x561e2c316340 2024-03-19T01:22:34.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.933+0000 7f816fb9a740 1 -- 172.21.15.69:0/3488918601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8154012ad0 con 0x561e2c316340 2024-03-19T01:22:34.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.936+0000 7f8147e00640 1 -- 172.21.15.69:0/3488918601 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f81580bc790 con 0x561e2c316340 2024-03-19T01:22:34.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.936+0000 7f816fb9a740 10 client.27820 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:34.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.936+0000 7f816fb9a740 1 -- 172.21.15.69:0/3488918601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=59}) -- 0x561e2c32dab0 con 0x561e2c316340 2024-03-19T01:22:34.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.936+0000 7f8147e00640 1 -- 172.21.15.69:0/3488918601 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f815803c030 con 0x561e2c316340 2024-03-19T01:22:34.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.936+0000 7f816fb9a740 10 client.27820 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:34.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.936+0000 7f816fb9a740 10 client.27820 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:22:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.936+0000 7f816fb9a740 20 client.27820 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.937+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561e2c32f0d0 0x561e2c331590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.937+0000 7f816fb9a740 4 client.27820 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:22:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.937+0000 7f816fb9a740 1 -- 172.21.15.69:0/3488918601 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e2c2878c0 con 0x561e2c32f0d0 2024-03-19T01:22:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.937+0000 7f8165e00640 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561e2c32f0d0 0x561e2c331590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.938+0000 7f8147e00640 10 client.27820 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:22:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.939+0000 7f8165e00640 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561e2c32f0d0 0x561e2c331590 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98138da777261074 server_cookie=461e9bdf6e9dbba1 in_seq=0 out_seq=0 2024-03-19T01:22:34.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.941+0000 7f8147e00640 1 -- 172.21.15.69:0/3488918601 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561e2c2878c0 con 0x561e2c32f0d0 2024-03-19T01:22:34.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.941+0000 7f8147e00640 10 client.27820 handle_command_reply: tid=0 2024-03-19T01:22:34.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.941+0000 7f816fb9a740 1 client.27820 shutdown 2024-03-19T01:22:34.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.944+0000 7f816fb9a740 1 -- 172.21.15.69:0/3488918601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c316340 msgr2=0x561e2c31c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:34.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.944+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c316340 0x561e2c31c4d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f81580ed0c0 tx=0x7f8158032fc0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.944+0000 7f816fb9a740 1 -- 172.21.15.69:0/3488918601 shutdown_connections 2024-03-19T01:22:34.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.944+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561e2c32f0d0 0x561e2c331590 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.944+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c31ef20 0x561e2c31ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.944+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c316340 0x561e2c31c4d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.944+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3488918601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c315a20 0x561e2c31bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.944+0000 7f816fb9a740 1 -- 172.21.15.69:0/3488918601 >> 172.21.15.69:0/3488918601 conn(0x561e2c310150 msgr2=0x561e2c311ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:34.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.944+0000 7f8147e00640 0 client.27820 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:22:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.948+0000 7f816fb9a740 1 -- 172.21.15.69:0/3488918601 shutdown_connections 2024-03-19T01:22:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.948+0000 7f816fb9a740 1 -- 172.21.15.69:0/3488918601 wait complete. 2024-03-19T01:22:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.949+0000 7f816fb9a740 20 client.27820 trim_cache size 0 max 16384 2024-03-19T01:22:34.963 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/base/built-in.a 2024-03-19T01:22:34.970 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpi_lpss.o 2024-03-19T01:22:34.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.979+0000 7f816fb9a740 1 Processor -- start 2024-03-19T01:22:34.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.979+0000 7f816fb9a740 1 -- start start 2024-03-19T01:22:34.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.980+0000 7f816fb9a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c315a20 0x561e2c31e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.981+0000 7f816fb9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c316340 0x561e2c32aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.981+0000 7f816fb9a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c31ef20 0x561e2c31c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.981+0000 7f8165e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c315a20 0x561e2c31e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.981+0000 7f8165e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c315a20 0x561e2c31e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48090/0 (socket says 172.21.15.69:48090) 2024-03-19T01:22:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.981+0000 7f8165e00640 1 -- 172.21.15.69:0/3288150569 learned_addr learned my addr 172.21.15.69:0/3288150569 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.981+0000 7f816fb9a740 1 -- 172.21.15.69:0/3288150569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81681c6570 con 0x561e2c31ef20 2024-03-19T01:22:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.981+0000 7f816fb9a740 1 -- 172.21.15.69:0/3288150569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81681c80b0 con 0x561e2c316340 2024-03-19T01:22:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.981+0000 7f816fb9a740 1 -- 172.21.15.69:0/3288150569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f81681c6c40 con 0x561e2c315a20 2024-03-19T01:22:34.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.981+0000 7f8166800640 1 --2- 172.21.15.69:0/3288150569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c31ef20 0x561e2c31c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.982+0000 7f8165400640 1 --2- 172.21.15.69:0/3288150569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c316340 0x561e2c32aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.982+0000 7f8165e00640 1 -- 172.21.15.69:0/3288150569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c316340 msgr2=0x561e2c32aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:34.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.982+0000 7f8165e00640 1 --2- 172.21.15.69:0/3288150569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c316340 0x561e2c32aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.982+0000 7f8165e00640 1 -- 172.21.15.69:0/3288150569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c31ef20 msgr2=0x561e2c31c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:34.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.982+0000 7f8165e00640 1 --2- 172.21.15.69:0/3288150569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c31ef20 0x561e2c31c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.982+0000 7f8165e00640 1 -- 172.21.15.69:0/3288150569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e2c30fc80 con 0x561e2c315a20 2024-03-19T01:22:34.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.982+0000 7f8166800640 1 --2- 172.21.15.69:0/3288150569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c31ef20 0x561e2c31c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:22:34.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.983+0000 7f8165e00640 1 --2- 172.21.15.69:0/3288150569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c315a20 0x561e2c31e6c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8154009c20 tx=0x7f815400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d38b76c7842f115 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:34.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.985+0000 7f8152a00640 1 -- 172.21.15.69:0/3288150569 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815400f520 con 0x561e2c315a20 2024-03-19T01:22:34.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.985+0000 7f8152a00640 1 -- 172.21.15.69:0/3288150569 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8154031070 con 0x561e2c315a20 2024-03-19T01:22:34.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.985+0000 7f8152a00640 1 -- 172.21.15.69:0/3288150569 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154005040 con 0x561e2c315a20 2024-03-19T01:22:34.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.987+0000 7f816fb9a740 1 -- 172.21.15.69:0/3288150569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c315a20 msgr2=0x561e2c31e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.987+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3288150569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c315a20 0x561e2c31e6c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8154009c20 tx=0x7f815400f790 comp rx=0 tx=0).stop 2024-03-19T01:22:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.988+0000 7f816fb9a740 1 -- 172.21.15.69:0/3288150569 shutdown_connections 2024-03-19T01:22:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.988+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3288150569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c31ef20 0x561e2c31c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.988+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3288150569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c316340 0x561e2c32aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.988+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3288150569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c315a20 0x561e2c31e6c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.988+0000 7f816fb9a740 1 -- 172.21.15.69:0/3288150569 >> 172.21.15.69:0/3288150569 conn(0x561e2c310150 msgr2=0x561e2c316720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.988+0000 7f816fb9a740 1 -- 172.21.15.69:0/3288150569 shutdown_connections 2024-03-19T01:22:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.988+0000 7f816fb9a740 1 -- 172.21.15.69:0/3288150569 wait complete. 2024-03-19T01:22:34.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.989+0000 7f816fb9a740 1 -- start start 2024-03-19T01:22:34.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.990+0000 7f816fb9a740 1 Processor -- start 2024-03-19T01:22:34.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.991+0000 7f816fb9a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c315a20 0x561e2c327330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.992+0000 7f816fb9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c316340 0x561e2c322740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.992+0000 7f816fb9a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c31ef20 0x561e2c322c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.992+0000 7f816fb9a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81681c79e0 con 0x561e2c315a20 2024-03-19T01:22:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.992+0000 7f816fb9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81681c7310 con 0x561e2c316340 2024-03-19T01:22:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.992+0000 7f816fb9a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f81681c4550 con 0x561e2c31ef20 2024-03-19T01:22:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.992+0000 7f816fb9a740 10 client.0 _ll_register_callbacks cb 0x7f81580f2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.992+0000 7f8165e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c315a20 0x561e2c327330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.992+0000 7f8165e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c315a20 0x561e2c327330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40328/0 (socket says 172.21.15.69:40328) 2024-03-19T01:22:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.992+0000 7f8165e00640 1 -- 172.21.15.69:0/3299721001 learned_addr learned my addr 172.21.15.69:0/3299721001 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.993+0000 7f8166800640 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c31ef20 0x561e2c322c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.993+0000 7f8165400640 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c316340 0x561e2c322740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.993+0000 7f8165e00640 1 -- 172.21.15.69:0/3299721001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c31ef20 msgr2=0x561e2c322c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:34.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.993+0000 7f8165e00640 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c31ef20 0x561e2c322c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.993+0000 7f8165e00640 1 -- 172.21.15.69:0/3299721001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c316340 msgr2=0x561e2c322740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:34.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.993+0000 7f8165e00640 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c316340 0x561e2c322740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:34.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.993+0000 7f8165e00640 1 -- 172.21.15.69:0/3299721001 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e2c3234b0 con 0x561e2c315a20 2024-03-19T01:22:34.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.993+0000 7f8166800640 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c31ef20 0x561e2c322c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:22:34.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.993+0000 7f8165400640 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c316340 0x561e2c322740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:22:34.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.994+0000 7f8147e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:22:34.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.994+0000 7f8165e00640 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c315a20 0x561e2c327330 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f815400f550 tx=0x7f815400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1783cd723e3b545f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:34.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.994+0000 7f8147e00640 1 -- 172.21.15.69:0/3299721001 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815403a060 con 0x561e2c315a20 2024-03-19T01:22:34.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.994+0000 7f8147e00640 1 -- 172.21.15.69:0/3299721001 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7f815400f800 con 0x561e2c315a20 2024-03-19T01:22:34.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.995+0000 7f816fb9a740 1 -- 172.21.15.69:0/3299721001 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8154031070 con 0x561e2c315a20 2024-03-19T01:22:34.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.995+0000 7f8147e00640 1 -- 172.21.15.69:0/3299721001 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f815400fbe0 con 0x561e2c315a20 2024-03-19T01:22:34.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.995+0000 7f816fb9a740 10 client.17517 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:34.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.995+0000 7f816fb9a740 1 -- 172.21.15.69:0/3299721001 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x561e2c3954a0 con 0x561e2c315a20 2024-03-19T01:22:34.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.996+0000 7f8147e00640 1 -- 172.21.15.69:0/3299721001 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8154031070 con 0x561e2c315a20 2024-03-19T01:22:34.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.996+0000 7f8147e00640 1 -- 172.21.15.69:0/3299721001 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f8154004660 con 0x561e2c315a20 2024-03-19T01:22:34.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.996+0000 7f816fb9a740 10 client.17517 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:34.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.996+0000 7f816fb9a740 10 client.17517 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:22:34.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.996+0000 7f816fb9a740 20 client.17517 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:34.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.996+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561e2c3959d0 0x561e2c397dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:34.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.996+0000 7f816fb9a740 4 client.17517 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:22:34.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.996+0000 7f816fb9a740 1 -- 172.21.15.69:0/3299721001 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e2c281c10 con 0x561e2c3959d0 2024-03-19T01:22:34.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.996+0000 7f8165400640 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561e2c3959d0 0x561e2c397dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:34.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.997+0000 7f8165400640 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561e2c3959d0 0x561e2c397dc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e681a743fa15e43c server_cookie=b3dbb77c6c6a852b in_seq=0 out_seq=0 2024-03-19T01:22:34.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.998+0000 7f8147e00640 10 client.17517 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:22:34.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.998+0000 7f8147e00640 1 -- 172.21.15.69:0/3299721001 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e2c281c10 con 0x561e2c3959d0 2024-03-19T01:22:34.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.998+0000 7f8147e00640 10 client.17517 handle_command_reply: tid=0 2024-03-19T01:22:34.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:34.998+0000 7f816fb9a740 1 client.17517 shutdown 2024-03-19T01:22:35.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.001+0000 7f816fb9a740 1 -- 172.21.15.69:0/3299721001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c315a20 msgr2=0x561e2c327330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:35.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.001+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c315a20 0x561e2c327330 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f815400f550 tx=0x7f815400f580 comp rx=0 tx=0).stop 2024-03-19T01:22:35.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.001+0000 7f816fb9a740 1 -- 172.21.15.69:0/3299721001 shutdown_connections 2024-03-19T01:22:35.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.001+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x561e2c3959d0 0x561e2c397dc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.001+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e2c31ef20 0x561e2c322c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.001+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e2c316340 0x561e2c322740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.001+0000 7f816fb9a740 1 --2- 172.21.15.69:0/3299721001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e2c315a20 0x561e2c327330 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.001+0000 7f816fb9a740 1 -- 172.21.15.69:0/3299721001 >> 172.21.15.69:0/3299721001 conn(0x561e2c310150 msgr2=0x561e2c30e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:35.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.001+0000 7f8147e00640 0 client.17517 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:22:35.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.002+0000 7f816fb9a740 1 -- 172.21.15.69:0/3299721001 shutdown_connections 2024-03-19T01:22:35.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.003+0000 7f816fb9a740 1 -- 172.21.15.69:0/3299721001 wait complete. 2024-03-19T01:22:35.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.003+0000 7f816fb9a740 20 client.17517 trim_cache size 0 max 16384 2024-03-19T01:22:35.006 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:22:35.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.007+0000 7f816ce00640 1 -- 172.21.15.69:0/335330372 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f814c0a2a30 msgr2=0x7f814c0a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:35.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.007+0000 7f816ce00640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f814c0a2a30 0x7f814c0a4ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8160009370 tx=0x7f81600092e0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.007+0000 7f816ce00640 1 -- 172.21.15.69:0/335330372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8168110040 msgr2=0x7f81681116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:35.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.007+0000 7f816ce00640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8168110040 0x7f81681116e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8158037040 tx=0x7f81580045e0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.009+0000 7f816ce00640 1 -- 172.21.15.69:0/335330372 shutdown_connections 2024-03-19T01:22:35.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.009+0000 7f816ce00640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f814c0a2a30 0x7f814c0a4ef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.009+0000 7f816ce00640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f816811b000 0x7f8168111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.009+0000 7f816ce00640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8168110040 0x7f81681116e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.009+0000 7f816ce00640 1 --2- 172.21.15.69:0/335330372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f816810e070 0x7f8168118630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.009+0000 7f816ce00640 1 -- 172.21.15.69:0/335330372 >> 172.21.15.69:0/335330372 conn(0x7f816806c900 msgr2=0x7f81680722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:35.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.014+0000 7f816ce00640 1 -- 172.21.15.69:0/335330372 shutdown_connections 2024-03-19T01:22:35.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.014+0000 7f816ce00640 1 -- 172.21.15.69:0/335330372 wait complete. 2024-03-19T01:22:35.049 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/kernel_read_file.o 2024-03-19T01:22:35.070 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:22:35.108 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_pmu.o 2024-03-19T01:22:35.149 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/cdrom/built-in.a 2024-03-19T01:22:35.163 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/mnt_idmapping.o 2024-03-19T01:22:35.343 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/mdio/acpi_mdio.o 2024-03-19T01:22:35.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.709+0000 7fb5d0a00640 1 -- 172.21.15.69:0/351050301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc108a00 msgr2=0x7fb5cc108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:35.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.709+0000 7fb5d0a00640 1 --2- 172.21.15.69:0/351050301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc108a00 0x7fb5cc108de0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009900 tx=0x7fb5bc02bc60 comp rx=0 tx=0).stop 2024-03-19T01:22:35.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.711+0000 7fb5d0a00640 1 -- 172.21.15.69:0/351050301 shutdown_connections 2024-03-19T01:22:35.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.711+0000 7fb5d0a00640 1 --2- 172.21.15.69:0/351050301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5cc114970 0x7fb5cc116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.711+0000 7fb5d0a00640 1 --2- 172.21.15.69:0/351050301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5cc109320 0x7fb5cc114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.711+0000 7fb5d0a00640 1 --2- 172.21.15.69:0/351050301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc108a00 0x7fb5cc108de0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.711+0000 7fb5d0a00640 1 -- 172.21.15.69:0/351050301 >> 172.21.15.69:0/351050301 conn(0x7fb5cc06c900 msgr2=0x7fb5cc06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:35.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.711+0000 7fb5d0a00640 1 -- 172.21.15.69:0/351050301 shutdown_connections 2024-03-19T01:22:35.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.711+0000 7fb5d0a00640 1 -- 172.21.15.69:0/351050301 wait complete. 2024-03-19T01:22:35.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.713+0000 7fb5d0a00640 1 Processor -- start 2024-03-19T01:22:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.730+0000 7fb5d0a00640 1 -- start start 2024-03-19T01:22:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.731+0000 7fb5d0a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5cc108a00 0x7fb5cc10fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.731+0000 7fb5d0a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc109320 0x7fb5cc110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.731+0000 7fb5d0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5cc114970 0x7fb5cc10ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.731+0000 7fb5d0a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc11a4a0 con 0x7fb5cc108a00 2024-03-19T01:22:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.731+0000 7fb5d0a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc11a320 con 0x7fb5cc114970 2024-03-19T01:22:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.731+0000 7fb5d0a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc11a620 con 0x7fb5cc109320 2024-03-19T01:22:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.731+0000 7fb5ca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5cc114970 0x7fb5cc10ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.731+0000 7fb5ca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5cc114970 0x7fb5cc10ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33374/0 (socket says 172.21.15.69:33374) 2024-03-19T01:22:35.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.731+0000 7fb5ca800640 1 -- 172.21.15.69:0/4232473820 learned_addr learned my addr 172.21.15.69:0/4232473820 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:35.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.731+0000 7fb5ca800640 1 -- 172.21.15.69:0/4232473820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc109320 msgr2=0x7fb5cc110080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:22:35.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.731+0000 7fb5ca800640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc109320 0x7fb5cc110080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.732+0000 7fb5ca800640 1 -- 172.21.15.69:0/4232473820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5cc108a00 msgr2=0x7fb5cc10fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:35.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.732+0000 7fb5ca800640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5cc108a00 0x7fb5cc10fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.732+0000 7fb5ca800640 1 -- 172.21.15.69:0/4232473820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc10b510 con 0x7fb5cc114970 2024-03-19T01:22:35.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.732+0000 7fb5ca800640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5cc114970 0x7fb5cc10ac50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400b9e0 tx=0x7fb5c400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:35.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.732+0000 7fb5c2a00640 1 -- 172.21.15.69:0/4232473820 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4019070 con 0x7fb5cc114970 2024-03-19T01:22:35.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.732+0000 7fb5c2a00640 1 -- 172.21.15.69:0/4232473820 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c4014440 con 0x7fb5cc114970 2024-03-19T01:22:35.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.732+0000 7fb5c2a00640 1 -- 172.21.15.69:0/4232473820 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40125e0 con 0x7fb5cc114970 2024-03-19T01:22:35.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.733+0000 7fb5d0a00640 1 -- 172.21.15.69:0/4232473820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5cc10b800 con 0x7fb5cc114970 2024-03-19T01:22:35.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.733+0000 7fb5d0a00640 1 -- 172.21.15.69:0/4232473820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5cc113a20 con 0x7fb5cc114970 2024-03-19T01:22:35.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.749+0000 7fb5c2a00640 1 -- 172.21.15.69:0/4232473820 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fb5c401c070 con 0x7fb5cc114970 2024-03-19T01:22:35.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.749+0000 7fb5d3766740 1 Processor -- start 2024-03-19T01:22:35.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.749+0000 7fb5d3766740 1 -- start start 2024-03-19T01:22:35.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.750+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 0x55c1d13d6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.750+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d7340 0x55c1d13df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.750+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13dff20 0x55c1d13e2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.750+0000 7fb5c9400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 0x55c1d13d6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.750+0000 7fb5c9400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 0x55c1d13d6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48114/0 (socket says 172.21.15.69:48114) 2024-03-19T01:22:35.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.750+0000 7fb5d3766740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c1d133b380 con 0x55c1d13d7340 2024-03-19T01:22:35.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.750+0000 7fb5d3766740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c1d12e3d20 con 0x55c1d13dff20 2024-03-19T01:22:35.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.750+0000 7fb5d3766740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c1d1337420 con 0x55c1d13d6a20 2024-03-19T01:22:35.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.750+0000 7fb5c9400640 1 -- 172.21.15.69:0/2891692160 learned_addr learned my addr 172.21.15.69:0/2891692160 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:35.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.750+0000 7fb5c9400640 1 --2- 172.21.15.69:0/2891692160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13dff20 0x55c1d13e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.750+0000 7fb5c9e00640 1 --2- 172.21.15.69:0/2891692160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d7340 0x55c1d13df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.751+0000 7fb5c9400640 1 -- 172.21.15.69:0/2891692160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13dff20 msgr2=0x55c1d13e2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:35.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.751+0000 7fb5c9400640 1 --2- 172.21.15.69:0/2891692160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13dff20 0x55c1d13e2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.751+0000 7fb5c9400640 1 -- 172.21.15.69:0/2891692160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d7340 msgr2=0x55c1d13df9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:35.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.751+0000 7fb5c9400640 1 --2- 172.21.15.69:0/2891692160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d7340 0x55c1d13df9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.751+0000 7fb5c9400640 1 -- 172.21.15.69:0/2891692160 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d104f3f0 con 0x55c1d13d6a20 2024-03-19T01:22:35.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.751+0000 7fb5c9400640 1 --2- 172.21.15.69:0/2891692160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 0x55c1d13d6e00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009fd0 tx=0x7fb5b8009060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf5b90ffcbe115a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:35.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.751+0000 7fb5b7000640 1 -- 172.21.15.69:0/2891692160 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800c070 con 0x55c1d13d6a20 2024-03-19T01:22:35.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.751+0000 7fb5b7000640 1 -- 172.21.15.69:0/2891692160 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b8004510 con 0x55c1d13d6a20 2024-03-19T01:22:35.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.751+0000 7fb5b7000640 1 -- 172.21.15.69:0/2891692160 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800f040 con 0x55c1d13d6a20 2024-03-19T01:22:35.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.752+0000 7fb5d3766740 1 -- 172.21.15.69:0/2891692160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 msgr2=0x55c1d13d6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:35.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.752+0000 7fb5d3766740 1 --2- 172.21.15.69:0/2891692160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 0x55c1d13d6e00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009fd0 tx=0x7fb5b8009060 comp rx=0 tx=0).stop 2024-03-19T01:22:35.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.754+0000 7fb5d3766740 1 -- 172.21.15.69:0/2891692160 shutdown_connections 2024-03-19T01:22:35.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.754+0000 7fb5d3766740 1 --2- 172.21.15.69:0/2891692160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13dff20 0x55c1d13e2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.754+0000 7fb5d3766740 1 --2- 172.21.15.69:0/2891692160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d7340 0x55c1d13df9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.754+0000 7fb5d3766740 1 --2- 172.21.15.69:0/2891692160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 0x55c1d13d6e00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.754+0000 7fb5d3766740 1 -- 172.21.15.69:0/2891692160 >> 172.21.15.69:0/2891692160 conn(0x55c1d13d1150 msgr2=0x55c1d13d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:35.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.754+0000 7fb5d3766740 1 -- 172.21.15.69:0/2891692160 shutdown_connections 2024-03-19T01:22:35.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.754+0000 7fb5d3766740 1 -- 172.21.15.69:0/2891692160 wait complete. 2024-03-19T01:22:35.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.755+0000 7fb5d3766740 1 -- start start 2024-03-19T01:22:35.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.755+0000 7fb5d3766740 1 Processor -- start 2024-03-19T01:22:35.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.755+0000 7fb5c2a00640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb5b00a2a00 0x7fb5b00a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.756+0000 7fb5c9400640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb5b00a2a00 0x7fb5b00a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.757+0000 7fb5c2a00640 1 -- 172.21.15.69:0/4232473820 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fb5c40ce660 con 0x7fb5cc114970 2024-03-19T01:22:35.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.758+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d6a20 0x55c1d13dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.758+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d7340 0x55c1d13dd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.758+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 0x55c1d13dd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.758+0000 7fb5d3766740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c1d13e2cb0 con 0x55c1d13d7340 2024-03-19T01:22:35.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.758+0000 7fb5d3766740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c1d12f7160 con 0x55c1d13d6a20 2024-03-19T01:22:35.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.758+0000 7fb5d3766740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c1d13e2e30 con 0x55c1d13dff20 2024-03-19T01:22:35.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.758+0000 7fb5d3766740 10 client.0 _ll_register_callbacks cb 0x55c1d1333170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:35.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.759+0000 7fb5c9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d6a20 0x55c1d13dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.759+0000 7fb5ca800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 0x55c1d13dd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.759+0000 7fb5c9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d6a20 0x55c1d13dcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33390/0 (socket says 172.21.15.69:33390) 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.759+0000 7fb5c9e00640 1 -- 172.21.15.69:0/431234049 learned_addr learned my addr 172.21.15.69:0/431234049 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.759+0000 7fb5c9e00640 1 -- 172.21.15.69:0/431234049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 msgr2=0x55c1d13dd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.759+0000 7fb5c9e00640 1 --2- 172.21.15.69:0/431234049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 0x55c1d13dd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.759+0000 7fb5c9e00640 1 -- 172.21.15.69:0/431234049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d7340 msgr2=0x55c1d13dd420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.759+0000 7fb5c9e00640 1 --2- 172.21.15.69:0/431234049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d7340 0x55c1d13dd420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.759+0000 7fb5c9e00640 1 -- 172.21.15.69:0/431234049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d11197d0 con 0x55c1d13d6a20 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.760+0000 7fb5c9e00640 1 --2- 172.21.15.69:0/431234049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d6a20 0x55c1d13dcee0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009850 tx=0x7fb5bc009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b77cb8e1dd0f61 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.762+0000 7fb5c9400640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb5b00a2a00 0x7fb5b00a4ec0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009fd0 tx=0x7fb5b8008040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.762+0000 7fb5ab400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.762+0000 7fb5ab400640 1 -- 172.21.15.69:0/431234049 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc064070 con 0x55c1d13d6a20 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.762+0000 7fb5ab400640 1 -- 172.21.15.69:0/431234049 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fb5bc005200 con 0x55c1d13d6a20 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.762+0000 7fb5d3766740 1 -- 172.21.15.69:0/431234049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b8004510 con 0x55c1d13d6a20 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.762+0000 7fb5ab400640 1 -- 172.21.15.69:0/431234049 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb5bc005400 con 0x55c1d13d6a20 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.762+0000 7fb5d3766740 10 client.27881 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:35.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.762+0000 7fb5d3766740 1 -- 172.21.15.69:0/431234049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55c1d13eef10 con 0x55c1d13d6a20 2024-03-19T01:22:35.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.763+0000 7fb5ab400640 1 -- 172.21.15.69:0/431234049 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b8004510 con 0x55c1d13d6a20 2024-03-19T01:22:35.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.768+0000 7fb5ab400640 1 -- 172.21.15.69:0/431234049 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fb5bc0046c0 con 0x55c1d13d6a20 2024-03-19T01:22:35.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.768+0000 7fb5d3766740 10 client.27881 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:35.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.768+0000 7fb5d3766740 10 client.27881 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:22:35.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.768+0000 7fb5d3766740 20 client.27881 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:35.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.768+0000 7fb5d3766740 1 --2- 172.21.15.69:0/431234049 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c1d13f03d0 0x55c1d13f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.768+0000 7fb5d3766740 4 client.27881 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:22:35.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.768+0000 7fb5d3766740 1 -- 172.21.15.69:0/431234049 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1d13488c0 con 0x55c1d13f03d0 2024-03-19T01:22:35.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.768+0000 7fb5c9400640 1 --2- 172.21.15.69:0/431234049 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c1d13f03d0 0x55c1d13f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.769+0000 7fb5c9400640 1 --2- 172.21.15.69:0/431234049 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c1d13f03d0 0x55c1d13f27e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d8565b14b9999be server_cookie=9b211519c55ed22f in_seq=0 out_seq=0 2024-03-19T01:22:35.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.769+0000 7fb5ab400640 10 client.27881 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:22:35.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.770+0000 7fb5ab400640 1 -- 172.21.15.69:0/431234049 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c1d13488c0 con 0x55c1d13f03d0 2024-03-19T01:22:35.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.770+0000 7fb5ab400640 10 client.27881 handle_command_reply: tid=0 2024-03-19T01:22:35.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.770+0000 7fb5d3766740 1 client.27881 shutdown 2024-03-19T01:22:35.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.770+0000 7fb5d3766740 1 -- 172.21.15.69:0/431234049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d6a20 msgr2=0x55c1d13dcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:35.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.770+0000 7fb5d3766740 1 --2- 172.21.15.69:0/431234049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d6a20 0x55c1d13dcee0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009850 tx=0x7fb5bc009650 comp rx=0 tx=0).stop 2024-03-19T01:22:35.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.771+0000 7fb5d3766740 1 -- 172.21.15.69:0/431234049 shutdown_connections 2024-03-19T01:22:35.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.771+0000 7fb5d3766740 1 --2- 172.21.15.69:0/431234049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 0x55c1d13dd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.771+0000 7fb5d3766740 1 --2- 172.21.15.69:0/431234049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d7340 0x55c1d13dd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.771+0000 7fb5d3766740 1 --2- 172.21.15.69:0/431234049 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c1d13f03d0 0x55c1d13f27e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.771+0000 7fb5d3766740 1 --2- 172.21.15.69:0/431234049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d6a20 0x55c1d13dcee0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.771+0000 7fb5d3766740 1 -- 172.21.15.69:0/431234049 >> 172.21.15.69:0/431234049 conn(0x55c1d13d1150 msgr2=0x55c1d13d2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:35.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.771+0000 7fb5ab400640 0 client.27881 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:22:35.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.771+0000 7fb5d3766740 1 -- 172.21.15.69:0/431234049 shutdown_connections 2024-03-19T01:22:35.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.771+0000 7fb5d3766740 1 -- 172.21.15.69:0/431234049 wait complete. 2024-03-19T01:22:35.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.771+0000 7fb5d3766740 20 client.27881 trim_cache size 0 max 16384 2024-03-19T01:22:35.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.826+0000 7fb5d3766740 1 Processor -- start 2024-03-19T01:22:35.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.826+0000 7fb5d3766740 1 -- start start 2024-03-19T01:22:35.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.826+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d6a20 0x55c1d13ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.826+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 0x55c1d13d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.826+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 0x55c1d13f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.826+0000 7fb5d3766740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc07ccc0 con 0x55c1d13d6a20 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.826+0000 7fb5d3766740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc07e800 con 0x55c1d13d7340 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.826+0000 7fb5d3766740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc07d390 con 0x55c1d13dff20 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.827+0000 7fb5ca800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 0x55c1d13f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.827+0000 7fb5c9e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d6a20 0x55c1d13ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.827+0000 7fb5ca800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 0x55c1d13f80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48136/0 (socket says 172.21.15.69:48136) 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.827+0000 7fb5ca800640 1 -- 172.21.15.69:0/3679164681 learned_addr learned my addr 172.21.15.69:0/3679164681 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.827+0000 7fb5c9400640 1 --2- 172.21.15.69:0/3679164681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 0x55c1d13d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.827+0000 7fb5ca800640 1 -- 172.21.15.69:0/3679164681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 msgr2=0x55c1d13d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.827+0000 7fb5ca800640 1 --2- 172.21.15.69:0/3679164681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 0x55c1d13d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.827+0000 7fb5ca800640 1 -- 172.21.15.69:0/3679164681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d6a20 msgr2=0x55c1d13ebab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.828+0000 7fb5ca800640 1 --2- 172.21.15.69:0/3679164681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d6a20 0x55c1d13ebab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.828+0000 7fb5ca800640 1 -- 172.21.15.69:0/3679164681 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d13e2310 con 0x55c1d13dff20 2024-03-19T01:22:35.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.828+0000 7fb5c9e00640 1 --2- 172.21.15.69:0/3679164681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d6a20 0x55c1d13ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:22:35.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.828+0000 7fb5ca800640 1 --2- 172.21.15.69:0/3679164681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 0x55c1d13f80b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb5c409a6c0 tx=0x7fb5c400e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f7f9679e4e24f3a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:35.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.828+0000 7fb5b6600640 1 -- 172.21.15.69:0/3679164681 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4098cc0 con 0x55c1d13dff20 2024-03-19T01:22:35.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.828+0000 7fb5b6600640 1 -- 172.21.15.69:0/3679164681 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c40b3070 con 0x55c1d13dff20 2024-03-19T01:22:35.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.828+0000 7fb5b6600640 1 -- 172.21.15.69:0/3679164681 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40a2cd0 con 0x55c1d13dff20 2024-03-19T01:22:35.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.830+0000 7fb5d3766740 1 -- 172.21.15.69:0/3679164681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 msgr2=0x55c1d13f80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:35.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.830+0000 7fb5d3766740 1 --2- 172.21.15.69:0/3679164681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 0x55c1d13f80b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb5c409a6c0 tx=0x7fb5c400e040 comp rx=0 tx=0).stop 2024-03-19T01:22:35.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.830+0000 7fb5d3766740 1 -- 172.21.15.69:0/3679164681 shutdown_connections 2024-03-19T01:22:35.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.830+0000 7fb5d3766740 1 --2- 172.21.15.69:0/3679164681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13dff20 0x55c1d13f80b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.830+0000 7fb5d3766740 1 --2- 172.21.15.69:0/3679164681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 0x55c1d13d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.830+0000 7fb5d3766740 1 --2- 172.21.15.69:0/3679164681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13d6a20 0x55c1d13ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.830+0000 7fb5d3766740 1 -- 172.21.15.69:0/3679164681 >> 172.21.15.69:0/3679164681 conn(0x55c1d13d1150 msgr2=0x55c1d13d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:35.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.832+0000 7fb5d3766740 1 -- 172.21.15.69:0/3679164681 shutdown_connections 2024-03-19T01:22:35.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.832+0000 7fb5d3766740 1 -- 172.21.15.69:0/3679164681 wait complete. 2024-03-19T01:22:35.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.833+0000 7fb5d3766740 1 -- start start 2024-03-19T01:22:35.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.833+0000 7fb5d3766740 1 Processor -- start 2024-03-19T01:22:35.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.837+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 0x55c1d13e5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.837+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 0x55c1d13e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.837+0000 7fb5d3766740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13dff20 0x55c1d13e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.837+0000 7fb5d3766740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc07e130 con 0x55c1d13dff20 2024-03-19T01:22:35.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.837+0000 7fb5d3766740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc07da60 con 0x55c1d13d7340 2024-03-19T01:22:35.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.837+0000 7fb5d3766740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc07ac60 con 0x55c1d13d6a20 2024-03-19T01:22:35.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.837+0000 7fb5d3766740 10 client.0 _ll_register_callbacks cb 0x7fb5bc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:35.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.837+0000 7fb5c9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 0x55c1d13e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.837+0000 7fb5c9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 0x55c1d13e5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:33418/0 (socket says 172.21.15.69:33418) 2024-03-19T01:22:35.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.837+0000 7fb5c9400640 1 -- 172.21.15.69:0/3520881960 learned_addr learned my addr 172.21.15.69:0/3520881960 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:35.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.838+0000 7fb5ca800640 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13dff20 0x55c1d13e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.838+0000 7fb5c9400640 1 -- 172.21.15.69:0/3520881960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 msgr2=0x55c1d13e5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:35.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.838+0000 7fb5c9e00640 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 0x55c1d13e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.838+0000 7fb5c9400640 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 0x55c1d13e5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.838+0000 7fb5c9400640 1 -- 172.21.15.69:0/3520881960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13dff20 msgr2=0x55c1d13e63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:35.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.838+0000 7fb5c9400640 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13dff20 0x55c1d13e63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.838+0000 7fb5c9400640 1 -- 172.21.15.69:0/3520881960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d14556f0 con 0x55c1d13d7340 2024-03-19T01:22:35.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.838+0000 7fb5ca800640 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13dff20 0x55c1d13e63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:22:35.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.839+0000 7fb5c9400640 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 0x55c1d13e5e90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8014e30 tx=0x7fb5b8003b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c973c9fe7bd8fe32 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:35.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.839+0000 7fb5ab400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:22:35.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.839+0000 7fb5ab400640 1 -- 172.21.15.69:0/3520881960 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800f050 con 0x55c1d13d7340 2024-03-19T01:22:35.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.840+0000 7fb5d3766740 1 -- 172.21.15.69:0/3520881960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c40b3070 con 0x55c1d13d7340 2024-03-19T01:22:35.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.840+0000 7fb5ab400640 1 -- 172.21.15.69:0/3520881960 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fb5b80035f0 con 0x55c1d13d7340 2024-03-19T01:22:35.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.840+0000 7fb5ab400640 1 -- 172.21.15.69:0/3520881960 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb5b8041050 con 0x55c1d13d7340 2024-03-19T01:22:35.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.842+0000 7fb5d3766740 10 client.27887 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:35.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.842+0000 7fb5d3766740 1 -- 172.21.15.69:0/3520881960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55c1d14568f0 con 0x55c1d13d7340 2024-03-19T01:22:35.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.844+0000 7fb5ab400640 1 -- 172.21.15.69:0/3520881960 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c40b3070 con 0x55c1d13d7340 2024-03-19T01:22:35.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.844+0000 7fb5ab400640 1 -- 172.21.15.69:0/3520881960 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fb5b802f400 con 0x55c1d13d7340 2024-03-19T01:22:35.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.844+0000 7fb5d3766740 10 client.27887 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:35.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.844+0000 7fb5d3766740 10 client.27887 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:22:35.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.844+0000 7fb5d3766740 20 client.27887 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:35.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.844+0000 7fb5d3766740 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c1d1456d40 0x55c1d1459160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:35.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.844+0000 7fb5d3766740 4 client.27887 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:22:35.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.844+0000 7fb5d3766740 1 -- 172.21.15.69:0/3520881960 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1d1342c10 con 0x55c1d1456d40 2024-03-19T01:22:35.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.845+0000 7fb5c9e00640 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c1d1456d40 0x55c1d1459160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:35.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.846+0000 7fb5c9e00640 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c1d1456d40 0x55c1d1459160 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aa71a731ed6ba67a server_cookie=9688a29b6eb6f967 in_seq=0 out_seq=0 2024-03-19T01:22:35.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.846+0000 7fb5ab400640 10 client.27887 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:22:35.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.846+0000 7fb5ab400640 1 -- 172.21.15.69:0/3520881960 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1d1342c10 con 0x55c1d1456d40 2024-03-19T01:22:35.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.846+0000 7fb5ab400640 10 client.27887 handle_command_reply: tid=0 2024-03-19T01:22:35.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.846+0000 7fb5d3766740 1 client.27887 shutdown 2024-03-19T01:22:35.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.846+0000 7fb5d3766740 1 -- 172.21.15.69:0/3520881960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 msgr2=0x55c1d13e5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:35.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.846+0000 7fb5d3766740 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 0x55c1d13e5e90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8014e30 tx=0x7fb5b8003b30 comp rx=0 tx=0).stop 2024-03-19T01:22:35.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.847+0000 7fb5d3766740 1 -- 172.21.15.69:0/3520881960 shutdown_connections 2024-03-19T01:22:35.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.847+0000 7fb5d3766740 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c1d13dff20 0x55c1d13e63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.847+0000 7fb5d3766740 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55c1d1456d40 0x55c1d1459160 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.847+0000 7fb5d3766740 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1d13d7340 0x55c1d13e5e90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.847+0000 7fb5d3766740 1 --2- 172.21.15.69:0/3520881960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1d13d6a20 0x55c1d13e5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.847+0000 7fb5ab400640 0 client.27887 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:22:35.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.847+0000 7fb5d3766740 1 -- 172.21.15.69:0/3520881960 >> 172.21.15.69:0/3520881960 conn(0x55c1d13d1150 msgr2=0x55c1d13d2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:35.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.847+0000 7fb5d3766740 1 -- 172.21.15.69:0/3520881960 shutdown_connections 2024-03-19T01:22:35.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.848+0000 7fb5d3766740 1 -- 172.21.15.69:0/3520881960 wait complete. 2024-03-19T01:22:35.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.848+0000 7fb5d3766740 20 client.27887 trim_cache size 0 max 16384 2024-03-19T01:22:35.853 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:22:35.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.855+0000 7fb5d0a00640 1 -- 172.21.15.69:0/4232473820 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb5b00a2a00 msgr2=0x7fb5b00a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:35.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.855+0000 7fb5d0a00640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb5b00a2a00 0x7fb5b00a4ec0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009fd0 tx=0x7fb5b8008040 comp rx=0 tx=0).stop 2024-03-19T01:22:35.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.855+0000 7fb5d0a00640 1 -- 172.21.15.69:0/4232473820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5cc114970 msgr2=0x7fb5cc10ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:35.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.855+0000 7fb5d0a00640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5cc114970 0x7fb5cc10ac50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400b9e0 tx=0x7fb5c400bea0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.855+0000 7fb5d0a00640 1 -- 172.21.15.69:0/4232473820 shutdown_connections 2024-03-19T01:22:35.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.855+0000 7fb5d0a00640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb5b00a2a00 0x7fb5b00a4ec0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.855+0000 7fb5d0a00640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5cc114970 0x7fb5cc10ac50 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.855+0000 7fb5d0a00640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc109320 0x7fb5cc110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.855+0000 7fb5d0a00640 1 --2- 172.21.15.69:0/4232473820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5cc108a00 0x7fb5cc10fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:35.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.855+0000 7fb5d0a00640 1 -- 172.21.15.69:0/4232473820 >> 172.21.15.69:0/4232473820 conn(0x7fb5cc06c900 msgr2=0x7fb5cc071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.857+0000 7fb5d0a00640 1 -- 172.21.15.69:0/4232473820 shutdown_connections 2024-03-19T01:22:35.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:35.857+0000 7fb5d0a00640 1 -- 172.21.15.69:0/4232473820 wait complete. 2024-03-19T01:22:35.911 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:22:35.952 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/solomon/built-in.a 2024-03-19T01:22:35.990 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/mdio/fwnode_mdio.o 2024-03-19T01:22:35.999 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_logging.o 2024-03-19T01:22:36.039 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpi_apd.o 2024-03-19T01:22:36.053 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/remap_range.o 2024-03-19T01:22:36.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:36 smithi069 ceph-mon[31727]: pgmap v372: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 399 KiB/s wr, 42 op/s 2024-03-19T01:22:36.259 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/libata-pata-timings.o 2024-03-19T01:22:36.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:36 smithi086 ceph-mon[66829]: pgmap v372: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 399 KiB/s wr, 42 op/s 2024-03-19T01:22:36.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:36 smithi163 ceph-mon[67428]: pgmap v372: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 399 KiB/s wr, 42 op/s 2024-03-19T01:22:36.562 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_pm.o 2024-03-19T01:22:36.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.594+0000 7fe2a9000640 1 -- 172.21.15.69:0/3893563969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2a4108a00 msgr2=0x7fe2a4108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:36.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.594+0000 7fe2a9000640 1 --2- 172.21.15.69:0/3893563969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2a4108a00 0x7fe2a4108de0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009900 tx=0x7fe28c02bc40 comp rx=0 tx=0).stop 2024-03-19T01:22:36.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.594+0000 7fe2a9000640 1 -- 172.21.15.69:0/3893563969 shutdown_connections 2024-03-19T01:22:36.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.594+0000 7fe2a9000640 1 --2- 172.21.15.69:0/3893563969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2a4114970 0x7fe2a4116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.594+0000 7fe2a9000640 1 --2- 172.21.15.69:0/3893563969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a4109320 0x7fe2a4114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.594+0000 7fe2a9000640 1 --2- 172.21.15.69:0/3893563969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2a4108a00 0x7fe2a4108de0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.594+0000 7fe2a9000640 1 -- 172.21.15.69:0/3893563969 >> 172.21.15.69:0/3893563969 conn(0x7fe2a406c900 msgr2=0x7fe2a406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:36.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.595+0000 7fe2a9000640 1 -- 172.21.15.69:0/3893563969 shutdown_connections 2024-03-19T01:22:36.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.595+0000 7fe2a9000640 1 -- 172.21.15.69:0/3893563969 wait complete. 2024-03-19T01:22:36.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.596+0000 7fe2a9000640 1 Processor -- start 2024-03-19T01:22:36.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.599+0000 7fe2a9000640 1 -- start start 2024-03-19T01:22:36.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.599+0000 7fe2a9000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2a4108a00 0x7fe2a410c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.599+0000 7fe2a9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a4109320 0x7fe2a410c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.599+0000 7fe2a9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2a4114970 0x7fe2a410cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.599+0000 7fe2a9000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a411a3b0 con 0x7fe2a4114970 2024-03-19T01:22:36.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.600+0000 7fe2a1e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2a4108a00 0x7fe2a410c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.600+0000 7fe2a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a4109320 0x7fe2a410c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.600+0000 7fe2a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a4109320 0x7fe2a410c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45484/0 (socket says 172.21.15.69:45484) 2024-03-19T01:22:36.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.600+0000 7fe2a1400640 1 -- 172.21.15.69:0/1991608731 learned_addr learned my addr 172.21.15.69:0/1991608731 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:36.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.600+0000 7fe2a9000640 1 -- 172.21.15.69:0/1991608731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a411a230 con 0x7fe2a4109320 2024-03-19T01:22:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.600+0000 7fe2a9000640 1 -- 172.21.15.69:0/1991608731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a411a530 con 0x7fe2a4108a00 2024-03-19T01:22:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.601+0000 7fe2a2800640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2a4114970 0x7fe2a410cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.601+0000 7fe2a1400640 1 -- 172.21.15.69:0/1991608731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2a4108a00 msgr2=0x7fe2a410c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.601+0000 7fe2a1400640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2a4108a00 0x7fe2a410c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.601+0000 7fe2a1400640 1 -- 172.21.15.69:0/1991608731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2a4114970 msgr2=0x7fe2a410cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.601+0000 7fe2a1400640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2a4114970 0x7fe2a410cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.601+0000 7fe2a1400640 1 -- 172.21.15.69:0/1991608731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a41a2940 con 0x7fe2a4109320 2024-03-19T01:22:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.601+0000 7fe2a2800640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2a4114970 0x7fe2a410cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:22:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.601+0000 7fe2a1400640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a4109320 0x7fe2a410c9e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00eab0 tx=0x7fe29c00ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.602+0000 7fe29aa00640 1 -- 172.21.15.69:0/1991608731 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00ce40 con 0x7fe2a4109320 2024-03-19T01:22:36.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.602+0000 7fe29aa00640 1 -- 172.21.15.69:0/1991608731 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe29c011950 con 0x7fe2a4109320 2024-03-19T01:22:36.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.602+0000 7fe2a9000640 1 -- 172.21.15.69:0/1991608731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a41a2c30 con 0x7fe2a4109320 2024-03-19T01:22:36.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.602+0000 7fe2a9000640 1 -- 172.21.15.69:0/1991608731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a41a3180 con 0x7fe2a4109320 2024-03-19T01:22:36.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.602+0000 7fe29aa00640 1 -- 172.21.15.69:0/1991608731 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c010aa0 con 0x7fe2a4109320 2024-03-19T01:22:36.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.611+0000 7fe29aa00640 1 -- 172.21.15.69:0/1991608731 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fe29c010c40 con 0x7fe2a4109320 2024-03-19T01:22:36.617 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/gen2_engine_cs.o 2024-03-19T01:22:36.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.613+0000 7fe29aa00640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe2880a29b0 0x7fe2880a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.615+0000 7fe29aa00640 1 -- 172.21.15.69:0/1991608731 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fe29c0d09d0 con 0x7fe2a4109320 2024-03-19T01:22:36.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.616+0000 7fe2a1e00640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe2880a29b0 0x7fe2880a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.618+0000 7fe2abde3740 1 Processor -- start 2024-03-19T01:22:36.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.621+0000 7fe2a1e00640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe2880a29b0 0x7fe2880a4e70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe28c006010 tx=0x7fe28c000bc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:36.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.621+0000 7fe2abde3740 1 -- start start 2024-03-19T01:22:36.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.623+0000 7fe2abde3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 0x560c234cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.623+0000 7fe2abde3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234cf340 0x560c234d79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.624+0000 7fe2abde3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234d7f20 0x560c234da310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.624+0000 7fe2abde3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560c23433380 con 0x560c234cea20 2024-03-19T01:22:36.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.624+0000 7fe2abde3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560c233dbd20 con 0x560c234d7f20 2024-03-19T01:22:36.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.624+0000 7fe2abde3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560c2342f420 con 0x560c234cf340 2024-03-19T01:22:36.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.624+0000 7fe2a1e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 0x560c234cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.624+0000 7fe2a1e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 0x560c234cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53458/0 (socket says 172.21.15.69:53458) 2024-03-19T01:22:36.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.624+0000 7fe2a1e00640 1 -- 172.21.15.69:0/2284774012 learned_addr learned my addr 172.21.15.69:0/2284774012 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:36.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.624+0000 7fe2a2800640 1 --2- 172.21.15.69:0/2284774012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234d7f20 0x560c234da310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.625+0000 7fe2a2800640 1 -- 172.21.15.69:0/2284774012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234cf340 msgr2=0x560c234d79e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:36.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.625+0000 7fe2a2800640 1 --2- 172.21.15.69:0/2284774012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234cf340 0x560c234d79e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.625+0000 7fe2a2800640 1 -- 172.21.15.69:0/2284774012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 msgr2=0x560c234cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:36.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.625+0000 7fe2a2800640 1 --2- 172.21.15.69:0/2284774012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 0x560c234cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.625+0000 7fe2a2800640 1 -- 172.21.15.69:0/2284774012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c231473f0 con 0x560c234d7f20 2024-03-19T01:22:36.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.625+0000 7fe2a1e00640 1 --2- 172.21.15.69:0/2284774012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 0x560c234cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:22:36.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.626+0000 7fe2a2800640 1 --2- 172.21.15.69:0/2284774012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234d7f20 0x560c234da310 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b970 tx=0x7fe29400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02959b594df9772 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:36.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.628+0000 7fe293400640 1 -- 172.21.15.69:0/2284774012 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29400ddf0 con 0x560c234d7f20 2024-03-19T01:22:36.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.628+0000 7fe293400640 1 -- 172.21.15.69:0/2284774012 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe294017900 con 0x560c234d7f20 2024-03-19T01:22:36.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.628+0000 7fe293400640 1 -- 172.21.15.69:0/2284774012 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29400f9b0 con 0x560c234d7f20 2024-03-19T01:22:36.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.629+0000 7fe2abde3740 1 -- 172.21.15.69:0/2284774012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234d7f20 msgr2=0x560c234da310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:36.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.629+0000 7fe2abde3740 1 --2- 172.21.15.69:0/2284774012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234d7f20 0x560c234da310 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b970 tx=0x7fe29400be30 comp rx=0 tx=0).stop 2024-03-19T01:22:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.637+0000 7fe2abde3740 1 -- 172.21.15.69:0/2284774012 shutdown_connections 2024-03-19T01:22:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.637+0000 7fe2abde3740 1 --2- 172.21.15.69:0/2284774012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234d7f20 0x560c234da310 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.637+0000 7fe2abde3740 1 --2- 172.21.15.69:0/2284774012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234cf340 0x560c234d79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.637+0000 7fe2abde3740 1 --2- 172.21.15.69:0/2284774012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 0x560c234cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.637+0000 7fe2abde3740 1 -- 172.21.15.69:0/2284774012 >> 172.21.15.69:0/2284774012 conn(0x560c234c9150 msgr2=0x560c234c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.640+0000 7fe2abde3740 1 -- 172.21.15.69:0/2284774012 shutdown_connections 2024-03-19T01:22:36.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.643+0000 7fe2abde3740 1 -- 172.21.15.69:0/2284774012 wait complete. 2024-03-19T01:22:36.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.643+0000 7fe2abde3740 1 -- start start 2024-03-19T01:22:36.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.643+0000 7fe2abde3740 1 Processor -- start 2024-03-19T01:22:36.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.647+0000 7fe2abde3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 0x560c234d5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.647+0000 7fe2abde3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234cf340 0x560c234d5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.647+0000 7fe2abde3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 0x560c234d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.647+0000 7fe2abde3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560c234dad10 con 0x560c234cea20 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.647+0000 7fe2abde3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560c233ef160 con 0x560c234cf340 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.647+0000 7fe2abde3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560c234dae90 con 0x560c234d7f20 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.647+0000 7fe2abde3740 10 client.0 _ll_register_callbacks cb 0x560c2342b170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.648+0000 7fe2a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234cf340 0x560c234d5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.648+0000 7fe2a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234cf340 0x560c234d5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45494/0 (socket says 172.21.15.69:45494) 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.648+0000 7fe2a1400640 1 -- 172.21.15.69:0/1936223245 learned_addr learned my addr 172.21.15.69:0/1936223245 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.648+0000 7fe2a2800640 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 0x560c234d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.648+0000 7fe2a1400640 1 -- 172.21.15.69:0/1936223245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 msgr2=0x560c234d5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:36.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.648+0000 7fe2a1400640 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 0x560c234d5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.648+0000 7fe2a1400640 1 -- 172.21.15.69:0/1936223245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 msgr2=0x560c234d5010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:36.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.651+0000 7fe2a1e00640 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 0x560c234d5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.651+0000 7fe2a1400640 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 0x560c234d5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.651+0000 7fe2a1400640 1 -- 172.21.15.69:0/1936223245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c232117d0 con 0x560c234cf340 2024-03-19T01:22:36.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.651+0000 7fe2a1e00640 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 0x560c234d5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:22:36.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.652+0000 7fe2a1400640 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234cf340 0x560c234d5550 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0cdd70 tx=0x7fe29c09c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51097b82c6f31697 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:36.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.652+0000 7fe287e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:22:36.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.652+0000 7fe287e00640 1 -- 172.21.15.69:0/1936223245 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0a5130 con 0x560c234cf340 2024-03-19T01:22:36.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.652+0000 7fe2abde3740 1 -- 172.21.15.69:0/1936223245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe294017900 con 0x560c234cf340 2024-03-19T01:22:36.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.652+0000 7fe287e00640 1 -- 172.21.15.69:0/1936223245 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fe29c09c070 con 0x560c234cf340 2024-03-19T01:22:36.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.652+0000 7fe287e00640 1 -- 172.21.15.69:0/1936223245 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe29c0ba050 con 0x560c234cf340 2024-03-19T01:22:36.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.655+0000 7fe2abde3740 10 client.27899 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:36.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.655+0000 7fe2abde3740 1 -- 172.21.15.69:0/1936223245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x560c234e7150 con 0x560c234cf340 2024-03-19T01:22:36.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.655+0000 7fe287e00640 1 -- 172.21.15.69:0/1936223245 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe294017900 con 0x560c234cf340 2024-03-19T01:22:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.660+0000 7fe287e00640 1 -- 172.21.15.69:0/1936223245 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe29c0b5ce0 con 0x560c234cf340 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.660+0000 7fe2abde3740 10 client.27899 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.660+0000 7fe2abde3740 10 client.27899 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.660+0000 7fe2abde3740 20 client.27899 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.660+0000 7fe2abde3740 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x560c234e85b0 0x560c234ea9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.660+0000 7fe2abde3740 4 client.27899 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.660+0000 7fe2abde3740 1 -- 172.21.15.69:0/1936223245 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c234408c0 con 0x560c234e85b0 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.660+0000 7fe2a1e00640 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x560c234e85b0 0x560c234ea9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.661+0000 7fe287e00640 10 client.27899 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.661+0000 7fe2a1e00640 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x560c234e85b0 0x560c234ea9f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f6c621e4b8abcac server_cookie=1bf24b361086ff93 in_seq=0 out_seq=0 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.662+0000 7fe287e00640 1 -- 172.21.15.69:0/1936223245 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560c234408c0 con 0x560c234e85b0 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.662+0000 7fe287e00640 10 client.27899 handle_command_reply: tid=0 2024-03-19T01:22:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.662+0000 7fe2abde3740 1 client.27899 shutdown 2024-03-19T01:22:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.679+0000 7fe2abde3740 1 -- 172.21.15.69:0/1936223245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234cf340 msgr2=0x560c234d5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.679+0000 7fe2abde3740 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234cf340 0x560c234d5550 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0cdd70 tx=0x7fe29c09c820 comp rx=0 tx=0).stop 2024-03-19T01:22:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.679+0000 7fe2abde3740 1 -- 172.21.15.69:0/1936223245 shutdown_connections 2024-03-19T01:22:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.680+0000 7fe2abde3740 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x560c234e85b0 0x560c234ea9f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.680+0000 7fe2abde3740 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 0x560c234d5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.680+0000 7fe2abde3740 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234cf340 0x560c234d5550 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.680+0000 7fe2abde3740 1 --2- 172.21.15.69:0/1936223245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cea20 0x560c234d5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.680+0000 7fe287e00640 0 client.27899 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:22:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.680+0000 7fe2abde3740 1 -- 172.21.15.69:0/1936223245 >> 172.21.15.69:0/1936223245 conn(0x560c234c9150 msgr2=0x560c234caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.680+0000 7fe2abde3740 1 -- 172.21.15.69:0/1936223245 shutdown_connections 2024-03-19T01:22:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.680+0000 7fe2abde3740 1 -- 172.21.15.69:0/1936223245 wait complete. 2024-03-19T01:22:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.682+0000 7fe2abde3740 20 client.27899 trim_cache size 0 max 16384 2024-03-19T01:22:36.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.717+0000 7fe2abde3740 1 Processor -- start 2024-03-19T01:22:36.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.718+0000 7fe2abde3740 1 -- start start 2024-03-19T01:22:36.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.718+0000 7fe2abde3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234cea20 0x560c234d7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.718+0000 7fe2abde3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 0x560c234e3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.718+0000 7fe2abde3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 0x560c234c8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.718+0000 7fe2abde3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe280010080 con 0x560c234cf340 2024-03-19T01:22:36.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.718+0000 7fe2abde3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe280011bc0 con 0x560c234cea20 2024-03-19T01:22:36.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.718+0000 7fe2abde3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe280010750 con 0x560c234d7f20 2024-03-19T01:22:36.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.719+0000 7fe2a2800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 0x560c234c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.719+0000 7fe2a2800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 0x560c234c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50286/0 (socket says 172.21.15.69:50286) 2024-03-19T01:22:36.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.719+0000 7fe2a2800640 1 -- 172.21.15.69:0/3319275121 learned_addr learned my addr 172.21.15.69:0/3319275121 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:36.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.719+0000 7fe2a1400640 1 --2- 172.21.15.69:0/3319275121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 0x560c234e3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.720+0000 7fe2a2800640 1 -- 172.21.15.69:0/3319275121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234cea20 msgr2=0x560c234d7740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:36.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.720+0000 7fe2a2800640 1 --2- 172.21.15.69:0/3319275121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234cea20 0x560c234d7740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.720+0000 7fe2a2800640 1 -- 172.21.15.69:0/3319275121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 msgr2=0x560c234e3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:36.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.720+0000 7fe2a2800640 1 --2- 172.21.15.69:0/3319275121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 0x560c234e3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.720+0000 7fe2a2800640 1 -- 172.21.15.69:0/3319275121 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c234c8c80 con 0x560c234d7f20 2024-03-19T01:22:36.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.720+0000 7fe2a1400640 1 --2- 172.21.15.69:0/3319275121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 0x560c234e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:22:36.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.721+0000 7fe2a2800640 1 --2- 172.21.15.69:0/3319275121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 0x560c234c8050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b970 tx=0x7fe29400f4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c30088c5a427f21a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:36.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.725+0000 7fe292a00640 1 -- 172.21.15.69:0/3319275121 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29400dbf0 con 0x560c234d7f20 2024-03-19T01:22:36.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.725+0000 7fe292a00640 1 -- 172.21.15.69:0/3319275121 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe29400dd90 con 0x560c234d7f20 2024-03-19T01:22:36.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.725+0000 7fe292a00640 1 -- 172.21.15.69:0/3319275121 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2940179d0 con 0x560c234d7f20 2024-03-19T01:22:36.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.729+0000 7fe2abde3740 1 -- 172.21.15.69:0/3319275121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 msgr2=0x560c234c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:36.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.729+0000 7fe2abde3740 1 --2- 172.21.15.69:0/3319275121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 0x560c234c8050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b970 tx=0x7fe29400f4d0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.732+0000 7fe2abde3740 1 -- 172.21.15.69:0/3319275121 shutdown_connections 2024-03-19T01:22:36.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.732+0000 7fe2abde3740 1 --2- 172.21.15.69:0/3319275121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234d7f20 0x560c234c8050 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.732+0000 7fe2abde3740 1 --2- 172.21.15.69:0/3319275121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 0x560c234e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.732+0000 7fe2abde3740 1 --2- 172.21.15.69:0/3319275121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234cea20 0x560c234d7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.732+0000 7fe2abde3740 1 -- 172.21.15.69:0/3319275121 >> 172.21.15.69:0/3319275121 conn(0x560c234c9150 msgr2=0x560c234c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:36.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.740+0000 7fe2abde3740 1 -- 172.21.15.69:0/3319275121 shutdown_connections 2024-03-19T01:22:36.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.742+0000 7fe2abde3740 1 -- 172.21.15.69:0/3319275121 wait complete. 2024-03-19T01:22:36.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.742+0000 7fe2abde3740 1 -- start start 2024-03-19T01:22:36.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.743+0000 7fe2abde3740 1 Processor -- start 2024-03-19T01:22:36.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.743+0000 7fe2abde3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234cea20 0x560c234dc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.743+0000 7fe2abde3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 0x560c234dc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.744+0000 7fe2a1e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234cea20 0x560c234dc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.744+0000 7fe2a1e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234cea20 0x560c234dc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50300/0 (socket says 172.21.15.69:50300) 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.744+0000 7fe2a1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 0x560c234dc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.744+0000 7fe2a1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 0x560c234dc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53468/0 (socket says 172.21.15.69:53468) 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.744+0000 7fe2a1e00640 1 -- 172.21.15.69:0/2126753853 learned_addr learned my addr 172.21.15.69:0/2126753853 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.744+0000 7fe2abde3740 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234d7f20 0x560c2354d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.744+0000 7fe2abde3740 1 -- 172.21.15.69:0/2126753853 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe2800114f0 con 0x560c234cf340 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.744+0000 7fe2abde3740 1 -- 172.21.15.69:0/2126753853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe280010e20 con 0x560c234d7f20 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.744+0000 7fe2abde3740 1 -- 172.21.15.69:0/2126753853 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe28000e060 con 0x560c234cea20 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.745+0000 7fe2a1e00640 1 -- 172.21.15.69:0/2126753853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234d7f20 msgr2=0x560c2354d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.745+0000 7fe2a1e00640 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234d7f20 0x560c2354d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.745+0000 7fe2a1e00640 1 -- 172.21.15.69:0/2126753853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 msgr2=0x560c234dc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.745+0000 7fe2a1e00640 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 0x560c234dc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.745+0000 7fe2a1e00640 1 -- 172.21.15.69:0/2126753853 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c2354de30 con 0x560c234cea20 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.745+0000 7fe2a1400640 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 0x560c234dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.746+0000 7fe2a1e00640 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234cea20 0x560c234dc160 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009230 tx=0x7fe28c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=338458939d9798e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.746+0000 7fe2abde3740 10 client.0 _ll_register_callbacks cb 0x7fe29c010540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.746+0000 7fe2abde3740 1 -- 172.21.15.69:0/2126753853 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe29400dd90 con 0x560c234cea20 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.747+0000 7fe287e00640 10 client.27853 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.747+0000 7fe287e00640 1 -- 172.21.15.69:0/2126753853 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c005610 con 0x560c234cea20 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.747+0000 7fe287e00640 1 -- 172.21.15.69:0/2126753853 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fe28c072170 con 0x560c234cea20 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.747+0000 7fe287e00640 1 -- 172.21.15.69:0/2126753853 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe28c0a2050 con 0x560c234cea20 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.749+0000 7fe2abde3740 10 client.27853 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:36.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.749+0000 7fe2abde3740 1 -- 172.21.15.69:0/2126753853 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=59}) -- 0x560c2354efa0 con 0x560c234cea20 2024-03-19T01:22:36.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.750+0000 7fe287e00640 1 -- 172.21.15.69:0/2126753853 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe28c072370 con 0x560c234cea20 2024-03-19T01:22:36.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.750+0000 7fe2abde3740 10 client.27853 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:36.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.750+0000 7fe2abde3740 10 client.27853 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:22:36.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.750+0000 7fe2abde3740 20 client.27853 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:36.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.750+0000 7fe2abde3740 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x560c2354f520 0x560c235519c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:36.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.750+0000 7fe2abde3740 4 client.27853 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:22:36.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.750+0000 7fe2abde3740 1 -- 172.21.15.69:0/2126753853 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe29400dd90 con 0x560c2354f520 2024-03-19T01:22:36.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.750+0000 7fe2a1400640 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x560c2354f520 0x560c235519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:36.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.751+0000 7fe287e00640 10 client.27853 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:22:36.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.751+0000 7fe2a1400640 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x560c2354f520 0x560c235519c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65742bdec6093a20 server_cookie=3459882b72905812 in_seq=0 out_seq=0 2024-03-19T01:22:36.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.751+0000 7fe287e00640 1 -- 172.21.15.69:0/2126753853 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe29400dd90 con 0x560c2354f520 2024-03-19T01:22:36.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.751+0000 7fe287e00640 10 client.27853 handle_command_reply: tid=0 2024-03-19T01:22:36.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.751+0000 7fe2abde3740 1 client.27853 shutdown 2024-03-19T01:22:36.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.752+0000 7fe2abde3740 1 -- 172.21.15.69:0/2126753853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234cea20 msgr2=0x560c234dc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:36.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.752+0000 7fe2abde3740 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234cea20 0x560c234dc160 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009230 tx=0x7fe28c009260 comp rx=0 tx=0).stop 2024-03-19T01:22:36.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.753+0000 7fe2abde3740 1 -- 172.21.15.69:0/2126753853 shutdown_connections 2024-03-19T01:22:36.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.753+0000 7fe2abde3740 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x560c2354f520 0x560c235519c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.753+0000 7fe2abde3740 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c234d7f20 0x560c2354d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.753+0000 7fe287e00640 0 client.27853 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:22:36.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.753+0000 7fe2abde3740 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c234cf340 0x560c234dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.753+0000 7fe2abde3740 1 --2- 172.21.15.69:0/2126753853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560c234cea20 0x560c234dc160 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.754+0000 7fe2abde3740 1 -- 172.21.15.69:0/2126753853 >> 172.21.15.69:0/2126753853 conn(0x560c234c9150 msgr2=0x560c234cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:36.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.754+0000 7fe2abde3740 1 -- 172.21.15.69:0/2126753853 shutdown_connections 2024-03-19T01:22:36.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.754+0000 7fe2abde3740 1 -- 172.21.15.69:0/2126753853 wait complete. 2024-03-19T01:22:36.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.754+0000 7fe2abde3740 20 client.27853 trim_cache size 0 max 16384 2024-03-19T01:22:36.756 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:22:36.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.759+0000 7fe2a9000640 1 -- 172.21.15.69:0/1991608731 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe2880a29b0 msgr2=0x7fe2880a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:36.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.759+0000 7fe2a9000640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe2880a29b0 0x7fe2880a4e70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe28c006010 tx=0x7fe28c000bc0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.759+0000 7fe2a9000640 1 -- 172.21.15.69:0/1991608731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a4109320 msgr2=0x7fe2a410c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:36.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.759+0000 7fe2a9000640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a4109320 0x7fe2a410c9e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00eab0 tx=0x7fe29c00ef70 comp rx=0 tx=0).stop 2024-03-19T01:22:36.760 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpi_platform.o 2024-03-19T01:22:36.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.759+0000 7fe2a9000640 1 -- 172.21.15.69:0/1991608731 shutdown_connections 2024-03-19T01:22:36.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.759+0000 7fe2a9000640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2a4114970 0x7fe2a410cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.759+0000 7fe2a9000640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe2880a29b0 0x7fe2880a4e70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.759+0000 7fe2a9000640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a4109320 0x7fe2a410c9e0 secure :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00eab0 tx=0x7fe29c00ef70 comp rx=0 tx=0).stop 2024-03-19T01:22:36.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.759+0000 7fe2a9000640 1 --2- 172.21.15.69:0/1991608731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2a4108a00 0x7fe2a410c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:36.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.759+0000 7fe2a9000640 1 -- 172.21.15.69:0/1991608731 >> 172.21.15.69:0/1991608731 conn(0x7fe2a406c900 msgr2=0x7fe2a410a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:36.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.760+0000 7fe2a9000640 1 -- 172.21.15.69:0/1991608731 shutdown_connections 2024-03-19T01:22:36.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:36.760+0000 7fe2a9000640 1 -- 172.21.15.69:0/1991608731 wait complete. 2024-03-19T01:22:36.776 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/phy/built-in.a 2024-03-19T01:22:36.819 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:22:36.897 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/pcs/built-in.a 2024-03-19T01:22:36.970 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/auxdisplay/built-in.a 2024-03-19T01:22:37.041 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_aperture.o 2024-03-19T01:22:37.156 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/buffer.o 2024-03-19T01:22:37.389 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/ahci.o 2024-03-19T01:22:37.554 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_bsg.o 2024-03-19T01:22:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.569+0000 7fcacf400640 1 -- 172.21.15.69:0/1254476307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac8108820 msgr2=0x7fcac8108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.569+0000 7fcacf400640 1 --2- 172.21.15.69:0/1254476307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac8108820 0x7fcac8108c00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009900 tx=0x7fcab802bcc0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.569+0000 7fcacf400640 1 -- 172.21.15.69:0/1254476307 shutdown_connections 2024-03-19T01:22:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.569+0000 7fcacf400640 1 --2- 172.21.15.69:0/1254476307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac8114970 0x7fcac8116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.569+0000 7fcacf400640 1 --2- 172.21.15.69:0/1254476307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8109140 0x7fcac8114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.569+0000 7fcacf400640 1 --2- 172.21.15.69:0/1254476307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac8108820 0x7fcac8108c00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.569+0000 7fcacf400640 1 -- 172.21.15.69:0/1254476307 >> 172.21.15.69:0/1254476307 conn(0x7fcac806cd60 msgr2=0x7fcac806d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:37.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.569+0000 7fcacf400640 1 -- 172.21.15.69:0/1254476307 shutdown_connections 2024-03-19T01:22:37.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.569+0000 7fcacf400640 1 -- 172.21.15.69:0/1254476307 wait complete. 2024-03-19T01:22:37.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.571+0000 7fcacf400640 1 Processor -- start 2024-03-19T01:22:37.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.571+0000 7fcacf400640 1 -- start start 2024-03-19T01:22:37.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.572+0000 7fcacf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac8108820 0x7fcac819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.572+0000 7fcacf400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac8109140 0x7fcac819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.572+0000 7fcacf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8114970 0x7fcac81a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.572+0000 7fcacf400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81181c0 con 0x7fcac8108820 2024-03-19T01:22:37.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.572+0000 7fcacf400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8118040 con 0x7fcac8114970 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.572+0000 7fcacf400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8118340 con 0x7fcac8109140 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.572+0000 7fcacd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8114970 0x7fcac81a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.572+0000 7fcacd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8114970 0x7fcac81a2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45540/0 (socket says 172.21.15.69:45540) 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.572+0000 7fcacd400640 1 -- 172.21.15.69:0/1868212970 learned_addr learned my addr 172.21.15.69:0/1868212970 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.572+0000 7fcacca00640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac8108820 0x7fcac819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.573+0000 7fcacd400640 1 -- 172.21.15.69:0/1868212970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac8109140 msgr2=0x7fcac819eba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.573+0000 7fcacd400640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac8109140 0x7fcac819eba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.573+0000 7fcacd400640 1 -- 172.21.15.69:0/1868212970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac8108820 msgr2=0x7fcac819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.573+0000 7fcacd400640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac8108820 0x7fcac819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.573+0000 7fcacd400640 1 -- 172.21.15.69:0/1868212970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac81a35d0 con 0x7fcac8114970 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.573+0000 7fcacca00640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac8108820 0x7fcac819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:22:37.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.573+0000 7fcacd400640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8114970 0x7fcac81a2ed0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcab00091e0 tx=0x7fcab00096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:37.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.574+0000 7fcac5600640 1 -- 172.21.15.69:0/1868212970 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0004020 con 0x7fcac8114970 2024-03-19T01:22:37.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.574+0000 7fcac5600640 1 -- 172.21.15.69:0/1868212970 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab0011800 con 0x7fcac8114970 2024-03-19T01:22:37.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.574+0000 7fcac5600640 1 -- 172.21.15.69:0/1868212970 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0003c50 con 0x7fcac8114970 2024-03-19T01:22:37.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.574+0000 7fcacf400640 1 -- 172.21.15.69:0/1868212970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac81a38c0 con 0x7fcac8114970 2024-03-19T01:22:37.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.574+0000 7fcacf400640 1 -- 172.21.15.69:0/1868212970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac81ab180 con 0x7fcac8114970 2024-03-19T01:22:37.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.579+0000 7fcad205d740 1 Processor -- start 2024-03-19T01:22:37.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.580+0000 7fcad205d740 1 -- start start 2024-03-19T01:22:37.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.583+0000 7fcac5600640 1 -- 172.21.15.69:0/1868212970 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fcab0004350 con 0x7fcac8114970 2024-03-19T01:22:37.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.584+0000 7fcad205d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a605a60 0x55947a605e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.584+0000 7fcad205d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a606380 0x55947a60ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.584+0000 7fcad205d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a60ef60 0x55947a611350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.584+0000 7fcad205d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55947a56a380 con 0x55947a606380 2024-03-19T01:22:37.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.584+0000 7fcad205d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55947a512d20 con 0x55947a605a60 2024-03-19T01:22:37.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.584+0000 7fcad205d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55947a566420 con 0x55947a60ef60 2024-03-19T01:22:37.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.584+0000 7fcac7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a605a60 0x55947a605e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.584+0000 7fcac7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a605a60 0x55947a605e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45542/0 (socket says 172.21.15.69:45542) 2024-03-19T01:22:37.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.585+0000 7fcac7e00640 1 -- 172.21.15.69:0/3640190371 learned_addr learned my addr 172.21.15.69:0/3640190371 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:37.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.585+0000 7fcac7e00640 1 --2- 172.21.15.69:0/3640190371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a60ef60 0x55947a611350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.585+0000 7fcacca00640 1 --2- 172.21.15.69:0/3640190371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a606380 0x55947a60ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.585+0000 7fcac7e00640 1 -- 172.21.15.69:0/3640190371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a60ef60 msgr2=0x55947a611350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:37.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.585+0000 7fcac7e00640 1 --2- 172.21.15.69:0/3640190371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a60ef60 0x55947a611350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.585+0000 7fcac7e00640 1 -- 172.21.15.69:0/3640190371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a606380 msgr2=0x55947a60ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:37.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.585+0000 7fcac7e00640 1 --2- 172.21.15.69:0/3640190371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a606380 0x55947a60ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.585+0000 7fcac7e00640 1 -- 172.21.15.69:0/3640190371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947a27e3f0 con 0x55947a605a60 2024-03-19T01:22:37.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.585+0000 7fcacca00640 1 --2- 172.21.15.69:0/3640190371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a606380 0x55947a60ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:22:37.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.586+0000 7fcac7e00640 1 --2- 172.21.15.69:0/3640190371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a605a60 0x55947a605e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fcac0013a80 tx=0x7fcac0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953450ba66712a00 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:37.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.586+0000 7fcabdc00640 1 -- 172.21.15.69:0/3640190371 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0003b70 con 0x55947a605a60 2024-03-19T01:22:37.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.586+0000 7fcabdc00640 1 -- 172.21.15.69:0/3640190371 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac0003d10 con 0x55947a605a60 2024-03-19T01:22:37.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.586+0000 7fcabdc00640 1 -- 172.21.15.69:0/3640190371 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0004560 con 0x55947a605a60 2024-03-19T01:22:37.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.587+0000 7fcad205d740 1 -- 172.21.15.69:0/3640190371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a605a60 msgr2=0x55947a605e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:37.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.587+0000 7fcad205d740 1 --2- 172.21.15.69:0/3640190371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a605a60 0x55947a605e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fcac0013a80 tx=0x7fcac0009580 comp rx=0 tx=0).stop 2024-03-19T01:22:37.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.593+0000 7fcac5600640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaac0a29f0 0x7fcaac0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.594+0000 7fcac5600640 1 -- 172.21.15.69:0/1868212970 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fcab00f0c50 con 0x7fcac8114970 2024-03-19T01:22:37.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.595+0000 7fcac7e00640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaac0a29f0 0x7fcaac0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.600+0000 7fcac7e00640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaac0a29f0 0x7fcaac0a4eb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcac0013a80 tx=0x7fcac0004020 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.600+0000 7fcad205d740 1 -- 172.21.15.69:0/3640190371 shutdown_connections 2024-03-19T01:22:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.600+0000 7fcad205d740 1 --2- 172.21.15.69:0/3640190371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a60ef60 0x55947a611350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.600+0000 7fcad205d740 1 --2- 172.21.15.69:0/3640190371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a606380 0x55947a60ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.600+0000 7fcad205d740 1 --2- 172.21.15.69:0/3640190371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a605a60 0x55947a605e40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.600+0000 7fcad205d740 1 -- 172.21.15.69:0/3640190371 >> 172.21.15.69:0/3640190371 conn(0x55947a600190 msgr2=0x55947a600570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.600+0000 7fcad205d740 1 -- 172.21.15.69:0/3640190371 shutdown_connections 2024-03-19T01:22:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.600+0000 7fcad205d740 1 -- 172.21.15.69:0/3640190371 wait complete. 2024-03-19T01:22:37.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.601+0000 7fcad205d740 1 -- start start 2024-03-19T01:22:37.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.601+0000 7fcad205d740 1 Processor -- start 2024-03-19T01:22:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.607+0000 7fcad205d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a605a60 0x55947a60bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.607+0000 7fcad205d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a606380 0x55947a60c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.607+0000 7fcad205d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a60ef60 0x55947a60ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.607+0000 7fcad205d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55947a611d50 con 0x55947a60ef60 2024-03-19T01:22:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.607+0000 7fcad205d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55947a526160 con 0x55947a606380 2024-03-19T01:22:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.607+0000 7fcad205d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55947a611ed0 con 0x55947a605a60 2024-03-19T01:22:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.607+0000 7fcad205d740 10 client.0 _ll_register_callbacks cb 0x55947a562170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.608+0000 7fcacca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a605a60 0x55947a60bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.608+0000 7fcacca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a605a60 0x55947a60bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50340/0 (socket says 172.21.15.69:50340) 2024-03-19T01:22:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.608+0000 7fcacca00640 1 -- 172.21.15.69:0/29522843 learned_addr learned my addr 172.21.15.69:0/29522843 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:37.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.608+0000 7fcacd400640 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a60ef60 0x55947a60ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.608+0000 7fcacca00640 1 -- 172.21.15.69:0/29522843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a606380 msgr2=0x55947a60c4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:22:37.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.608+0000 7fcacca00640 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a606380 0x55947a60c4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.608+0000 7fcacca00640 1 -- 172.21.15.69:0/29522843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a60ef60 msgr2=0x55947a60ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:37.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.608+0000 7fcacca00640 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a60ef60 0x55947a60ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.608+0000 7fcacca00640 1 -- 172.21.15.69:0/29522843 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947a3487d0 con 0x55947a605a60 2024-03-19T01:22:37.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.612+0000 7fcacca00640 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a605a60 0x55947a60bf80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009900 tx=0x7fcab80045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=170656b46633c6f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:37.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.612+0000 7fcab6000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:22:37.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.612+0000 7fcab6000640 1 -- 172.21.15.69:0/29522843 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8042070 con 0x55947a605a60 2024-03-19T01:22:37.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.615+0000 7fcab6000640 1 -- 172.21.15.69:0/29522843 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fcab8005030 con 0x55947a605a60 2024-03-19T01:22:37.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.615+0000 7fcad205d740 1 -- 172.21.15.69:0/29522843 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac0003d10 con 0x55947a605a60 2024-03-19T01:22:37.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.615+0000 7fcab6000640 1 -- 172.21.15.69:0/29522843 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fcab804c050 con 0x55947a605a60 2024-03-19T01:22:37.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.615+0000 7fcad205d740 10 client.27862 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:37.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.615+0000 7fcad205d740 1 -- 172.21.15.69:0/29522843 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55947a61dfd0 con 0x55947a605a60 2024-03-19T01:22:37.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.616+0000 7fcab6000640 1 -- 172.21.15.69:0/29522843 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac0003d10 con 0x55947a605a60 2024-03-19T01:22:37.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.616+0000 7fcab6000640 1 -- 172.21.15.69:0/29522843 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fcab80035f0 con 0x55947a605a60 2024-03-19T01:22:37.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.616+0000 7fcad205d740 10 client.27862 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:37.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.616+0000 7fcad205d740 10 client.27862 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:22:37.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.616+0000 7fcad205d740 20 client.27862 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:37.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.616+0000 7fcad205d740 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55947a61f4e0 0x55947a621980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.616+0000 7fcad205d740 4 client.27862 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:22:37.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.617+0000 7fcad205d740 1 -- 172.21.15.69:0/29522843 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55947a5778c0 con 0x55947a61f4e0 2024-03-19T01:22:37.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.617+0000 7fcac7e00640 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55947a61f4e0 0x55947a621980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.618+0000 7fcab6000640 10 client.27862 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:22:37.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.618+0000 7fcac7e00640 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55947a61f4e0 0x55947a621980 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8276fe389a80c47 server_cookie=51b790d08e2eba6 in_seq=0 out_seq=0 2024-03-19T01:22:37.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.622+0000 7fcab6000640 1 -- 172.21.15.69:0/29522843 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55947a5778c0 con 0x55947a61f4e0 2024-03-19T01:22:37.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.622+0000 7fcab6000640 10 client.27862 handle_command_reply: tid=0 2024-03-19T01:22:37.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.622+0000 7fcad205d740 1 client.27862 shutdown 2024-03-19T01:22:37.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.623+0000 7fcad205d740 1 -- 172.21.15.69:0/29522843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a605a60 msgr2=0x55947a60bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:37.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.623+0000 7fcad205d740 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a605a60 0x55947a60bf80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009900 tx=0x7fcab80045a0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.624+0000 7fcad205d740 1 -- 172.21.15.69:0/29522843 shutdown_connections 2024-03-19T01:22:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.624+0000 7fcad205d740 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a60ef60 0x55947a60ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.624+0000 7fcad205d740 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55947a61f4e0 0x55947a621980 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.624+0000 7fcad205d740 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a606380 0x55947a60c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.624+0000 7fcad205d740 1 --2- 172.21.15.69:0/29522843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a605a60 0x55947a60bf80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.624+0000 7fcab6000640 0 client.27862 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:22:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.624+0000 7fcad205d740 1 -- 172.21.15.69:0/29522843 >> 172.21.15.69:0/29522843 conn(0x55947a600190 msgr2=0x55947a601ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:37.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.624+0000 7fcad205d740 1 -- 172.21.15.69:0/29522843 shutdown_connections 2024-03-19T01:22:37.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.625+0000 7fcad205d740 1 -- 172.21.15.69:0/29522843 wait complete. 2024-03-19T01:22:37.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.625+0000 7fcad205d740 20 client.27862 trim_cache size 0 max 16384 2024-03-19T01:22:37.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.688+0000 7fcad205d740 1 Processor -- start 2024-03-19T01:22:37.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.690+0000 7fcad205d740 1 -- start start 2024-03-19T01:22:37.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.690+0000 7fcad205d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 0x55947a60b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.691+0000 7fcacca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 0x55947a60b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.691+0000 7fcacca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 0x55947a60b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53516/0 (socket says 172.21.15.69:53516) 2024-03-19T01:22:37.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.691+0000 7fcad205d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 0x55947a61aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.691+0000 7fcacca00640 1 -- 172.21.15.69:0/1114762564 learned_addr learned my addr 172.21.15.69:0/1114762564 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.691+0000 7fcad205d740 1 --2- 172.21.15.69:0/1114762564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a60ef60 0x55947a627250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.691+0000 7fcac7e00640 1 --2- 172.21.15.69:0/1114762564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 0x55947a61aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.691+0000 7fcad205d740 1 -- 172.21.15.69:0/1114762564 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b53c0 con 0x55947a605a60 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.691+0000 7fcad205d740 1 -- 172.21.15.69:0/1114762564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b6f00 con 0x55947a60ef60 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.691+0000 7fcad205d740 1 -- 172.21.15.69:0/1114762564 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b5a90 con 0x55947a606380 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.692+0000 7fcacca00640 1 -- 172.21.15.69:0/1114762564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 msgr2=0x55947a61aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.692+0000 7fcacca00640 1 --2- 172.21.15.69:0/1114762564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 0x55947a61aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.692+0000 7fcacca00640 1 -- 172.21.15.69:0/1114762564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a60ef60 msgr2=0x55947a627250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.692+0000 7fcacca00640 1 --2- 172.21.15.69:0/1114762564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a60ef60 0x55947a627250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.693+0000 7fcacca00640 1 -- 172.21.15.69:0/1114762564 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947a6061b0 con 0x55947a605a60 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.693+0000 7fcacca00640 1 --2- 172.21.15.69:0/1114762564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 0x55947a60b860 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcab8000c00 tx=0x7fcab8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc127dfed3d5bd69 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.693+0000 7fcabd200640 1 -- 172.21.15.69:0/1114762564 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8056070 con 0x55947a605a60 2024-03-19T01:22:37.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.694+0000 7fcabd200640 1 -- 172.21.15.69:0/1114762564 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab8003c30 con 0x55947a605a60 2024-03-19T01:22:37.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.694+0000 7fcabd200640 1 -- 172.21.15.69:0/1114762564 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8042070 con 0x55947a605a60 2024-03-19T01:22:37.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.695+0000 7fcad205d740 1 -- 172.21.15.69:0/1114762564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 msgr2=0x55947a60b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:37.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.695+0000 7fcad205d740 1 --2- 172.21.15.69:0/1114762564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 0x55947a60b860 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcab8000c00 tx=0x7fcab8005dd0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.695+0000 7fcad205d740 1 -- 172.21.15.69:0/1114762564 shutdown_connections 2024-03-19T01:22:37.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.695+0000 7fcad205d740 1 --2- 172.21.15.69:0/1114762564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a60ef60 0x55947a627250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.695+0000 7fcad205d740 1 --2- 172.21.15.69:0/1114762564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 0x55947a61aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.695+0000 7fcad205d740 1 --2- 172.21.15.69:0/1114762564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 0x55947a60b860 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.695+0000 7fcad205d740 1 -- 172.21.15.69:0/1114762564 >> 172.21.15.69:0/1114762564 conn(0x55947a600190 msgr2=0x55947a5feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:37.697 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/mdio/built-in.a 2024-03-19T01:22:37.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.698+0000 7fcad205d740 1 -- 172.21.15.69:0/1114762564 shutdown_connections 2024-03-19T01:22:37.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.699+0000 7fcad205d740 1 -- 172.21.15.69:0/1114762564 wait complete. 2024-03-19T01:22:37.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.699+0000 7fcad205d740 1 -- start start 2024-03-19T01:22:37.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.699+0000 7fcad205d740 1 Processor -- start 2024-03-19T01:22:37.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.704+0000 7fcad205d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 0x55947a608a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.704+0000 7fcad205d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 0x55947a608f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.704+0000 7fcad205d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a60ef60 0x55947a6094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.704+0000 7fcad205d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b6830 con 0x55947a605a60 2024-03-19T01:22:37.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.704+0000 7fcad205d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b6160 con 0x55947a60ef60 2024-03-19T01:22:37.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.704+0000 7fcad205d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b33a0 con 0x55947a606380 2024-03-19T01:22:37.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.704+0000 7fcad205d740 10 client.0 _ll_register_callbacks cb 0x7fcab0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:37.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.708+0000 7fcac7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 0x55947a608f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.708+0000 7fcac7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 0x55947a608f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50366/0 (socket says 172.21.15.69:50366) 2024-03-19T01:22:37.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.708+0000 7fcac7e00640 1 -- 172.21.15.69:0/2276374015 learned_addr learned my addr 172.21.15.69:0/2276374015 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:37.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.709+0000 7fcac7e00640 1 -- 172.21.15.69:0/2276374015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a60ef60 msgr2=0x55947a6094d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:37.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.709+0000 7fcac7e00640 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a60ef60 0x55947a6094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.709+0000 7fcac7e00640 1 -- 172.21.15.69:0/2276374015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 msgr2=0x55947a608a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:37.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.709+0000 7fcacca00640 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 0x55947a608a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.709+0000 7fcac7e00640 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 0x55947a608a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.709+0000 7fcac7e00640 1 -- 172.21.15.69:0/2276374015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947a684880 con 0x55947a606380 2024-03-19T01:22:37.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.710+0000 7fcac7e00640 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 0x55947a608f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcac0013790 tx=0x7fcac0004160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d321b2bb4a21f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:37.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.710+0000 7fcacca00640 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 0x55947a608a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:22:37.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.710+0000 7fcab6000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:22:37.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.710+0000 7fcab6000640 1 -- 172.21.15.69:0/2276374015 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0004270 con 0x55947a606380 2024-03-19T01:22:37.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.710+0000 7fcab6000640 1 -- 172.21.15.69:0/2276374015 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fcac0003970 con 0x55947a606380 2024-03-19T01:22:37.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.711+0000 7fcad205d740 1 -- 172.21.15.69:0/2276374015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab8003c30 con 0x55947a606380 2024-03-19T01:22:37.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.711+0000 7fcab6000640 1 -- 172.21.15.69:0/2276374015 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fcac0044050 con 0x55947a606380 2024-03-19T01:22:37.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.711+0000 7fcad205d740 10 client.27868 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:37.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.711+0000 7fcad205d740 1 -- 172.21.15.69:0/2276374015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55947a685a80 con 0x55947a606380 2024-03-19T01:22:37.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.712+0000 7fcab6000640 1 -- 172.21.15.69:0/2276374015 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab8003c30 con 0x55947a606380 2024-03-19T01:22:37.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.712+0000 7fcab6000640 1 -- 172.21.15.69:0/2276374015 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fcac000f040 con 0x55947a606380 2024-03-19T01:22:37.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.712+0000 7fcad205d740 10 client.27868 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:37.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.712+0000 7fcad205d740 10 client.27868 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:22:37.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.712+0000 7fcad205d740 20 client.27868 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:37.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.712+0000 7fcad205d740 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55947a685ee0 0x55947a688300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:37.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.712+0000 7fcad205d740 4 client.27868 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:22:37.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.712+0000 7fcad205d740 1 -- 172.21.15.69:0/2276374015 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55947a571c10 con 0x55947a685ee0 2024-03-19T01:22:37.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.712+0000 7fcacca00640 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55947a685ee0 0x55947a688300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:37.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.717+0000 7fcab6000640 10 client.27868 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:22:37.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.717+0000 7fcacca00640 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55947a685ee0 0x55947a688300 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10a171326b97e2cf server_cookie=66211d64f79edf33 in_seq=0 out_seq=0 2024-03-19T01:22:37.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.717+0000 7fcab6000640 1 -- 172.21.15.69:0/2276374015 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55947a571c10 con 0x55947a685ee0 2024-03-19T01:22:37.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.717+0000 7fcab6000640 10 client.27868 handle_command_reply: tid=0 2024-03-19T01:22:37.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.717+0000 7fcad205d740 1 client.27868 shutdown 2024-03-19T01:22:37.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.718+0000 7fcad205d740 1 -- 172.21.15.69:0/2276374015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 msgr2=0x55947a608f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:37.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.718+0000 7fcad205d740 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 0x55947a608f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcac0013790 tx=0x7fcac0004160 comp rx=0 tx=0).stop 2024-03-19T01:22:37.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.718+0000 7fcad205d740 1 -- 172.21.15.69:0/2276374015 shutdown_connections 2024-03-19T01:22:37.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.718+0000 7fcad205d740 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55947a685ee0 0x55947a688300 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.718+0000 7fcad205d740 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55947a60ef60 0x55947a6094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.718+0000 7fcad205d740 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55947a606380 0x55947a608f90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.718+0000 7fcad205d740 1 --2- 172.21.15.69:0/2276374015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947a605a60 0x55947a608a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.718+0000 7fcad205d740 1 -- 172.21.15.69:0/2276374015 >> 172.21.15.69:0/2276374015 conn(0x55947a600190 msgr2=0x55947a60c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:37.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.718+0000 7fcab6000640 0 client.27868 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:22:37.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.719+0000 7fcad205d740 1 -- 172.21.15.69:0/2276374015 shutdown_connections 2024-03-19T01:22:37.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.719+0000 7fcad205d740 1 -- 172.21.15.69:0/2276374015 wait complete. 2024-03-19T01:22:37.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.719+0000 7fcad205d740 20 client.27868 trim_cache size 0 max 16384 2024-03-19T01:22:37.721 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:22:37.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.720+0000 7fcacf400640 1 -- 172.21.15.69:0/1868212970 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaac0a29f0 msgr2=0x7fcaac0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:37.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.720+0000 7fcacf400640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaac0a29f0 0x7fcaac0a4eb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcac0013a80 tx=0x7fcac0004020 comp rx=0 tx=0).stop 2024-03-19T01:22:37.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.720+0000 7fcacf400640 1 -- 172.21.15.69:0/1868212970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8114970 msgr2=0x7fcac81a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:37.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.720+0000 7fcacf400640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8114970 0x7fcac81a2ed0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcab00091e0 tx=0x7fcab00096a0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.722+0000 7fcacf400640 1 -- 172.21.15.69:0/1868212970 shutdown_connections 2024-03-19T01:22:37.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.722+0000 7fcacf400640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fcaac0a29f0 0x7fcaac0a4eb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.722+0000 7fcacf400640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8114970 0x7fcac81a2ed0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.722+0000 7fcacf400640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac8109140 0x7fcac819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.722+0000 7fcacf400640 1 --2- 172.21.15.69:0/1868212970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcac8108820 0x7fcac819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:37.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.722+0000 7fcacf400640 1 -- 172.21.15.69:0/1868212970 >> 172.21.15.69:0/1868212970 conn(0x7fcac806cd60 msgr2=0x7fcac80704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:37.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.722+0000 7fcacf400640 1 -- 172.21.15.69:0/1868212970 shutdown_connections 2024-03-19T01:22:37.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:37.722+0000 7fcacf400640 1 -- 172.21.15.69:0/1868212970 wait complete. 2024-03-19T01:22:37.763 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpi_pnp.o 2024-03-19T01:22:37.779 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:22:37.787 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/gen6_engine_cs.o 2024-03-19T01:22:37.926 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/3com/built-in.a 2024-03-19T01:22:37.954 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/admtek/built-in.a 2024-03-19T01:22:37.964 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/8390/built-in.a 2024-03-19T01:22:37.992 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/mpage.o 2024-03-19T01:22:38.006 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/ath/built-in.a 2024-03-19T01:22:38.027 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/adaptec/built-in.a 2024-03-19T01:22:38.083 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/atmel/built-in.a 2024-03-19T01:22:38.109 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/agere/built-in.a 2024-03-19T01:22:38.151 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/broadcom/built-in.a 2024-03-19T01:22:38.185 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/alacritech/built-in.a 2024-03-19T01:22:38.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.212+0000 7f729a200640 1 -- 172.21.15.69:0/572782319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72941144c0 msgr2=0x7f72941168d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:38.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.212+0000 7f729a200640 1 --2- 172.21.15.69:0/572782319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72941144c0 0x7f72941168d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7284009860 tx=0x7f728402bbb0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.212+0000 7f729a200640 1 -- 172.21.15.69:0/572782319 shutdown_connections 2024-03-19T01:22:38.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.212+0000 7f729a200640 1 --2- 172.21.15.69:0/572782319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72941144c0 0x7f72941168d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.212+0000 7f729a200640 1 --2- 172.21.15.69:0/572782319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7294108df0 0x7f7294109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.212+0000 7f729a200640 1 --2- 172.21.15.69:0/572782319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406dae0 0x7f7294108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.212+0000 7f729a200640 1 -- 172.21.15.69:0/572782319 >> 172.21.15.69:0/572782319 conn(0x7f729406c700 msgr2=0x7f729406cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:38.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.213+0000 7f729a200640 1 -- 172.21.15.69:0/572782319 shutdown_connections 2024-03-19T01:22:38.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.213+0000 7f729a200640 1 -- 172.21.15.69:0/572782319 wait complete. 2024-03-19T01:22:38.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.218+0000 7f729a200640 1 Processor -- start 2024-03-19T01:22:38.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.219+0000 7f729a200640 1 -- start start 2024-03-19T01:22:38.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.219+0000 7f729a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406dae0 0x7f729410a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.219+0000 7f729a200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7294108df0 0x7f729410a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.219+0000 7f729a200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72941144c0 0x7f729410ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.219+0000 7f729a200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f729411a230 con 0x7f72941144c0 2024-03-19T01:22:38.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.220+0000 7f729a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f729411a0b0 con 0x7f729406dae0 2024-03-19T01:22:38.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.220+0000 7f729a200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f729411a3b0 con 0x7f7294108df0 2024-03-19T01:22:38.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.220+0000 7f7293400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406dae0 0x7f729410a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.220+0000 7f7293400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406dae0 0x7f729410a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45580/0 (socket says 172.21.15.69:45580) 2024-03-19T01:22:38.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.220+0000 7f7292a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7294108df0 0x7f729410a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.220+0000 7f7293e00640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72941144c0 0x7f729410ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.221+0000 7f7292a00640 1 -- 172.21.15.69:0/2495335994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406dae0 msgr2=0x7f729410a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:38.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.221+0000 7f7292a00640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406dae0 0x7f729410a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.221+0000 7f7292a00640 1 -- 172.21.15.69:0/2495335994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72941144c0 msgr2=0x7f729410ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:38.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.221+0000 7f7292a00640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72941144c0 0x7f729410ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.221+0000 7f7292a00640 1 -- 172.21.15.69:0/2495335994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72941a4830 con 0x7f7294108df0 2024-03-19T01:22:38.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.221+0000 7f7292a00640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7294108df0 0x7f729410a7d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7280005fd0 tx=0x7f728000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:38.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.221+0000 7f728be00640 1 -- 172.21.15.69:0/2495335994 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728000ccc0 con 0x7f7294108df0 2024-03-19T01:22:38.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.221+0000 7f728be00640 1 -- 172.21.15.69:0/2495335994 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f728000ee70 con 0x7f7294108df0 2024-03-19T01:22:38.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.221+0000 7f728be00640 1 -- 172.21.15.69:0/2495335994 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280010430 con 0x7f7294108df0 2024-03-19T01:22:38.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.222+0000 7f7293400640 1 -- 172.21.15.69:0/2495335994 learned_addr learned my addr 172.21.15.69:0/2495335994 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:38.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.222+0000 7f7293400640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406dae0 0x7f729410a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-19T01:22:38.224 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/cisco/built-in.a 2024-03-19T01:22:38.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.224+0000 7f729a200640 1 -- 172.21.15.69:0/2495335994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72941a4b20 con 0x7f7294108df0 2024-03-19T01:22:38.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.224+0000 7f729a200640 1 -- 172.21.15.69:0/2495335994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72941a5020 con 0x7f7294108df0 2024-03-19T01:22:38.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.233+0000 7f728be00640 1 -- 172.21.15.69:0/2495335994 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f7280003fb0 con 0x7f7294108df0 2024-03-19T01:22:38.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.238+0000 7f729cf63740 1 Processor -- start 2024-03-19T01:22:38.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.238+0000 7f729cf63740 1 -- start start 2024-03-19T01:22:38.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.238+0000 7f728be00640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72780a2af0 0x7f72780a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.239+0000 7f729cf63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b6c10 0x5560db3b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.239+0000 7f728be00640 1 -- 172.21.15.69:0/2495335994 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7f72800cf4e0 con 0x7f7294108df0 2024-03-19T01:22:38.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.239+0000 7f729cf63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b7530 0x5560db3bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.239+0000 7f729cf63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.239+0000 7f729cf63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5560db31b380 con 0x5560db3b7530 2024-03-19T01:22:38.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.239+0000 7f729cf63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560db2c3d20 con 0x5560db3c0110 2024-03-19T01:22:38.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.239+0000 7f729cf63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5560db317420 con 0x5560db3b6c10 2024-03-19T01:22:38.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.239+0000 7f7293e00640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72780a2af0 0x7f72780a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.239+0000 7f7293e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.239+0000 7f7293e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3c2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45584/0 (socket says 172.21.15.69:45584) 2024-03-19T01:22:38.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.239+0000 7f7293e00640 1 -- 172.21.15.69:0/4107304530 learned_addr learned my addr 172.21.15.69:0/4107304530 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:38.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.240+0000 7f7293400640 1 --2- 172.21.15.69:0/4107304530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b6c10 0x5560db3b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.240+0000 7f7292a00640 1 --2- 172.21.15.69:0/4107304530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b7530 0x5560db3bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.241 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/alteon/built-in.a 2024-03-19T01:22:38.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.241+0000 7f7293e00640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72780a2af0 0x7f72780a4fb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f729406aa10 tx=0x7f72840323d0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:38.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.241+0000 7f7293e00640 1 -- 172.21.15.69:0/4107304530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b6c10 msgr2=0x5560db3b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:38.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.241+0000 7f7293e00640 1 --2- 172.21.15.69:0/4107304530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b6c10 0x5560db3b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.241+0000 7f7293e00640 1 -- 172.21.15.69:0/4107304530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b7530 msgr2=0x5560db3bfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:38.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.241+0000 7f7293e00640 1 --2- 172.21.15.69:0/4107304530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b7530 0x5560db3bfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.241+0000 7f7293e00640 1 -- 172.21.15.69:0/4107304530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560db02f3f0 con 0x5560db3c0110 2024-03-19T01:22:38.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.241+0000 7f7292a00640 1 --2- 172.21.15.69:0/4107304530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b7530 0x5560db3bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:22:38.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.241+0000 7f7293e00640 1 --2- 172.21.15.69:0/4107304530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3c2500 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7284036000 tx=0x7f7284048000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f476ab1f63e3ec7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:38.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.242+0000 7f727fe00640 1 -- 172.21.15.69:0/4107304530 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284039030 con 0x5560db3c0110 2024-03-19T01:22:38.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.242+0000 7f727fe00640 1 -- 172.21.15.69:0/4107304530 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f728403a040 con 0x5560db3c0110 2024-03-19T01:22:38.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.242+0000 7f727fe00640 1 -- 172.21.15.69:0/4107304530 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284005340 con 0x5560db3c0110 2024-03-19T01:22:38.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.243+0000 7f729cf63740 1 -- 172.21.15.69:0/4107304530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 msgr2=0x5560db3c2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:38.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.243+0000 7f729cf63740 1 --2- 172.21.15.69:0/4107304530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3c2500 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7284036000 tx=0x7f7284048000 comp rx=0 tx=0).stop 2024-03-19T01:22:38.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.243+0000 7f729cf63740 1 -- 172.21.15.69:0/4107304530 shutdown_connections 2024-03-19T01:22:38.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.243+0000 7f729cf63740 1 --2- 172.21.15.69:0/4107304530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3c2500 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.243+0000 7f729cf63740 1 --2- 172.21.15.69:0/4107304530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b7530 0x5560db3bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.243+0000 7f729cf63740 1 --2- 172.21.15.69:0/4107304530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b6c10 0x5560db3b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.243+0000 7f729cf63740 1 -- 172.21.15.69:0/4107304530 >> 172.21.15.69:0/4107304530 conn(0x5560db3b1340 msgr2=0x5560db3b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:38.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.244+0000 7f729cf63740 1 -- 172.21.15.69:0/4107304530 shutdown_connections 2024-03-19T01:22:38.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.246+0000 7f729cf63740 1 -- 172.21.15.69:0/4107304530 wait complete. 2024-03-19T01:22:38.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.247+0000 7f729cf63740 1 -- start start 2024-03-19T01:22:38.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.247+0000 7f729cf63740 1 Processor -- start 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.253+0000 7f729cf63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 0x5560db3bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.253+0000 7f729cf63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b7530 0x5560db3bd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.253+0000 7f729cf63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3bdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.253+0000 7f729cf63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5560db3c2ef0 con 0x5560db3b6c10 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.253+0000 7f729cf63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560db2d7160 con 0x5560db3c0110 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.253+0000 7f729cf63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5560db3c3070 con 0x5560db3b7530 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.253+0000 7f729cf63740 10 client.0 _ll_register_callbacks cb 0x5560db313170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.254+0000 7f7293400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 0x5560db3bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.254+0000 7f7293400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 0x5560db3bd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53544/0 (socket says 172.21.15.69:53544) 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.254+0000 7f7293400640 1 -- 172.21.15.69:0/1500934176 learned_addr learned my addr 172.21.15.69:0/1500934176 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.255+0000 7f7293400640 1 -- 172.21.15.69:0/1500934176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b7530 msgr2=0x5560db3bd6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.255+0000 7f7293400640 1 --2- 172.21.15.69:0/1500934176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b7530 0x5560db3bd6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.255+0000 7f7293400640 1 -- 172.21.15.69:0/1500934176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 msgr2=0x5560db3bdc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:38.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.255+0000 7f7293400640 1 --2- 172.21.15.69:0/1500934176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3bdc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.255+0000 7f7293400640 1 -- 172.21.15.69:0/1500934176 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560db0f97d0 con 0x5560db3b6c10 2024-03-19T01:22:38.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.256+0000 7f7293400640 1 --2- 172.21.15.69:0/1500934176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 0x5560db3bd1a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f728c00fdf0 tx=0x7f728c00d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24867226db475b64 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:38.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.256+0000 7f727cc00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:22:38.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.257+0000 7f727cc00640 1 -- 172.21.15.69:0/1500934176 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c01c060 con 0x5560db3b6c10 2024-03-19T01:22:38.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.257+0000 7f729cf63740 1 -- 172.21.15.69:0/1500934176 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f728403a040 con 0x5560db3b6c10 2024-03-19T01:22:38.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.257+0000 7f727cc00640 1 -- 172.21.15.69:0/1500934176 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7f728c017410 con 0x5560db3b6c10 2024-03-19T01:22:38.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.260+0000 7f727cc00640 1 -- 172.21.15.69:0/1500934176 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f728c023050 con 0x5560db3b6c10 2024-03-19T01:22:38.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.260+0000 7f729cf63740 10 client.17562 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:38.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.260+0000 7f729cf63740 1 -- 172.21.15.69:0/1500934176 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5560db3cf290 con 0x5560db3b6c10 2024-03-19T01:22:38.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.262+0000 7f727cc00640 1 -- 172.21.15.69:0/1500934176 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f728403a040 con 0x5560db3b6c10 2024-03-19T01:22:38.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.262+0000 7f727cc00640 1 -- 172.21.15.69:0/1500934176 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f728c00a550 con 0x5560db3b6c10 2024-03-19T01:22:38.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.263+0000 7f729cf63740 10 client.17562 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:38.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.263+0000 7f729cf63740 10 client.17562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:22:38.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.263+0000 7f729cf63740 20 client.17562 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:38.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.263+0000 7f729cf63740 1 --2- 172.21.15.69:0/1500934176 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560db3d06a0 0x5560db3d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.263+0000 7f729cf63740 4 client.17562 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:22:38.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.263+0000 7f729cf63740 1 -- 172.21.15.69:0/1500934176 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560db3288c0 con 0x5560db3d06a0 2024-03-19T01:22:38.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.263+0000 7f7292a00640 1 --2- 172.21.15.69:0/1500934176 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560db3d06a0 0x5560db3d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.264+0000 7f7292a00640 1 --2- 172.21.15.69:0/1500934176 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560db3d06a0 0x5560db3d2ab0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d04630421f152d0 server_cookie=d4a6ce80d3e71ee8 in_seq=0 out_seq=0 2024-03-19T01:22:38.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.264+0000 7f727cc00640 10 client.17562 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:38.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.265+0000 7f727cc00640 1 -- 172.21.15.69:0/1500934176 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5560db3288c0 con 0x5560db3d06a0 2024-03-19T01:22:38.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.265+0000 7f727cc00640 10 client.17562 handle_command_reply: tid=0 2024-03-19T01:22:38.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.265+0000 7f729cf63740 1 client.17562 shutdown 2024-03-19T01:22:38.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.266+0000 7f729cf63740 1 -- 172.21.15.69:0/1500934176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 msgr2=0x5560db3bd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:38.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.266+0000 7f729cf63740 1 --2- 172.21.15.69:0/1500934176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 0x5560db3bd1a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f728c00fdf0 tx=0x7f728c00d7a0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.267+0000 7f729cf63740 1 -- 172.21.15.69:0/1500934176 shutdown_connections 2024-03-19T01:22:38.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.267+0000 7f729cf63740 1 --2- 172.21.15.69:0/1500934176 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560db3d06a0 0x5560db3d2ab0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.267+0000 7f729cf63740 1 --2- 172.21.15.69:0/1500934176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3bdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.267+0000 7f729cf63740 1 --2- 172.21.15.69:0/1500934176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b7530 0x5560db3bd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.267+0000 7f729cf63740 1 --2- 172.21.15.69:0/1500934176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 0x5560db3bd1a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.267+0000 7f729cf63740 1 -- 172.21.15.69:0/1500934176 >> 172.21.15.69:0/1500934176 conn(0x5560db3b1340 msgr2=0x5560db3b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:38.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.267+0000 7f727cc00640 0 client.17562 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:38.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.267+0000 7f729cf63740 1 -- 172.21.15.69:0/1500934176 shutdown_connections 2024-03-19T01:22:38.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.267+0000 7f729cf63740 1 -- 172.21.15.69:0/1500934176 wait complete. 2024-03-19T01:22:38.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.267+0000 7f729cf63740 20 client.17562 trim_cache size 0 max 16384 2024-03-19T01:22:38.297 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/intel/built-in.a 2024-03-19T01:22:38.306 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/amazon/built-in.a 2024-03-19T01:22:38.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.308+0000 7f729cf63740 1 Processor -- start 2024-03-19T01:22:38.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.308+0000 7f729cf63740 1 -- start start 2024-03-19T01:22:38.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f729cf63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 0x5560db3cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f729cf63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b7530 0x5560db3d83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f729cf63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3bd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f729cf63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f72941c85b0 con 0x5560db3b6c10 2024-03-19T01:22:38.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f729cf63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72941c63a0 con 0x5560db3c0110 2024-03-19T01:22:38.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f729cf63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f72941c7ee0 con 0x5560db3b7530 2024-03-19T01:22:38.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f7293e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3bd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f7293e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3bd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45588/0 (socket says 172.21.15.69:45588) 2024-03-19T01:22:38.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f7293e00640 1 -- 172.21.15.69:0/2445567402 learned_addr learned my addr 172.21.15.69:0/2445567402 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:38.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f7293400640 1 --2- 172.21.15.69:0/2445567402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 0x5560db3cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f7293e00640 1 -- 172.21.15.69:0/2445567402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b7530 msgr2=0x5560db3d83c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:38.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f7293e00640 1 --2- 172.21.15.69:0/2445567402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b7530 0x5560db3d83c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f7293e00640 1 -- 172.21.15.69:0/2445567402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 msgr2=0x5560db3cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:38.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.309+0000 7f7293e00640 1 --2- 172.21.15.69:0/2445567402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 0x5560db3cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.310+0000 7f7293e00640 1 -- 172.21.15.69:0/2445567402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560db3ab7f0 con 0x5560db3c0110 2024-03-19T01:22:38.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.310+0000 7f7293e00640 1 --2- 172.21.15.69:0/2445567402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3bd870 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7284034850 tx=0x7f728402bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca2ecd3ae48fe3ef server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:38.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.310+0000 7f727f400640 1 -- 172.21.15.69:0/2445567402 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284039040 con 0x5560db3c0110 2024-03-19T01:22:38.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.311+0000 7f727f400640 1 -- 172.21.15.69:0/2445567402 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7284034c70 con 0x5560db3c0110 2024-03-19T01:22:38.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.311+0000 7f727f400640 1 -- 172.21.15.69:0/2445567402 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284005530 con 0x5560db3c0110 2024-03-19T01:22:38.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.314+0000 7f729cf63740 1 -- 172.21.15.69:0/2445567402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 msgr2=0x5560db3bd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:38.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.314+0000 7f729cf63740 1 --2- 172.21.15.69:0/2445567402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3bd870 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7284034850 tx=0x7f728402bbb0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.317+0000 7f729cf63740 1 -- 172.21.15.69:0/2445567402 shutdown_connections 2024-03-19T01:22:38.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.317+0000 7f729cf63740 1 --2- 172.21.15.69:0/2445567402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db3bd870 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.317+0000 7f729cf63740 1 --2- 172.21.15.69:0/2445567402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b7530 0x5560db3d83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.317+0000 7f729cf63740 1 --2- 172.21.15.69:0/2445567402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b6c10 0x5560db3cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.317+0000 7f729cf63740 1 -- 172.21.15.69:0/2445567402 >> 172.21.15.69:0/2445567402 conn(0x5560db3b1340 msgr2=0x5560db3bd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:38.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.321+0000 7f729cf63740 1 -- 172.21.15.69:0/2445567402 shutdown_connections 2024-03-19T01:22:38.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.325+0000 7f729cf63740 1 -- 172.21.15.69:0/2445567402 wait complete. 2024-03-19T01:22:38.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.325+0000 7f729cf63740 1 -- start start 2024-03-19T01:22:38.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.325+0000 7f729cf63740 1 Processor -- start 2024-03-19T01:22:38.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.326+0000 7f729cf63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b6c10 0x5560db3c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.326+0000 7f729cf63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b7530 0x5560db3c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.326+0000 7f729cf63740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db42ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.326+0000 7f729cf63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f72941c7140 con 0x5560db3b7530 2024-03-19T01:22:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.326+0000 7f729cf63740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72941c6a70 con 0x5560db3c0110 2024-03-19T01:22:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.326+0000 7f729cf63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f72941c7810 con 0x5560db3b6c10 2024-03-19T01:22:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.326+0000 7f729cf63740 10 client.0 _ll_register_callbacks cb 0x7f728c007bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.327+0000 7f7293e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db42ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.327+0000 7f7293e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db42ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45590/0 (socket says 172.21.15.69:45590) 2024-03-19T01:22:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.327+0000 7f7293e00640 1 -- 172.21.15.69:0/1011603056 learned_addr learned my addr 172.21.15.69:0/1011603056 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.327+0000 7f7293e00640 1 -- 172.21.15.69:0/1011603056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b6c10 msgr2=0x5560db3c34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.327+0000 7f7293e00640 1 --2- 172.21.15.69:0/1011603056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b6c10 0x5560db3c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.327+0000 7f7293e00640 1 -- 172.21.15.69:0/1011603056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b7530 msgr2=0x5560db3c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.327+0000 7f7293e00640 1 --2- 172.21.15.69:0/1011603056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b7530 0x5560db3c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.327+0000 7f7293e00640 1 -- 172.21.15.69:0/1011603056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560db42f010 con 0x5560db3c0110 2024-03-19T01:22:38.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.327+0000 7f7293e00640 1 --2- 172.21.15.69:0/1011603056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db42ea20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7284009780 tx=0x7f7284005600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b1deaa905bba380 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:38.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.328+0000 7f727cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:22:38.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.329+0000 7f727cc00640 1 -- 172.21.15.69:0/1011603056 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284039050 con 0x5560db3c0110 2024-03-19T01:22:38.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.329+0000 7f727cc00640 1 -- 172.21.15.69:0/1011603056 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7f7284004250 con 0x5560db3c0110 2024-03-19T01:22:38.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.331+0000 7f729cf63740 1 -- 172.21.15.69:0/1011603056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7284034c70 con 0x5560db3c0110 2024-03-19T01:22:38.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.335+0000 7f727cc00640 1 -- 172.21.15.69:0/1011603056 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f7284004450 con 0x5560db3c0110 2024-03-19T01:22:38.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.338+0000 7f729cf63740 10 client.27914 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:38.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.338+0000 7f729cf63740 1 -- 172.21.15.69:0/1011603056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5560db4363b0 con 0x5560db3c0110 2024-03-19T01:22:38.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.338+0000 7f727cc00640 1 -- 172.21.15.69:0/1011603056 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7284034c70 con 0x5560db3c0110 2024-03-19T01:22:38.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.338+0000 7f727cc00640 1 -- 172.21.15.69:0/1011603056 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f72840046c0 con 0x5560db3c0110 2024-03-19T01:22:38.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.343+0000 7f729cf63740 10 client.27914 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:38.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.343+0000 7f729cf63740 10 client.27914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:22:38.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.343+0000 7f729cf63740 20 client.27914 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:38.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.343+0000 7f729cf63740 1 --2- 172.21.15.69:0/1011603056 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560db436630 0x5560db438a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.343+0000 7f729cf63740 4 client.27914 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:22:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.343+0000 7f729cf63740 1 -- 172.21.15.69:0/1011603056 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560db322c10 con 0x5560db436630 2024-03-19T01:22:38.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.343+0000 7f7293400640 1 --2- 172.21.15.69:0/1011603056 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560db436630 0x5560db438a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:38.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.345+0000 7f7293400640 1 --2- 172.21.15.69:0/1011603056 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560db436630 0x5560db438a20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=107f0ca24831210c server_cookie=969f581b1427aed9 in_seq=0 out_seq=0 2024-03-19T01:22:38.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.345+0000 7f727cc00640 10 client.27914 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:38.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.345+0000 7f727cc00640 1 -- 172.21.15.69:0/1011603056 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560db322c10 con 0x5560db436630 2024-03-19T01:22:38.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.345+0000 7f727cc00640 10 client.27914 handle_command_reply: tid=0 2024-03-19T01:22:38.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.345+0000 7f729cf63740 1 client.27914 shutdown 2024-03-19T01:22:38.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.346+0000 7f729cf63740 1 -- 172.21.15.69:0/1011603056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 msgr2=0x5560db42ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:38.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.346+0000 7f729cf63740 1 --2- 172.21.15.69:0/1011603056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db42ea20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7284009780 tx=0x7f7284005600 comp rx=0 tx=0).stop 2024-03-19T01:22:38.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.346+0000 7f729cf63740 1 -- 172.21.15.69:0/1011603056 shutdown_connections 2024-03-19T01:22:38.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.346+0000 7f729cf63740 1 --2- 172.21.15.69:0/1011603056 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5560db436630 0x5560db438a20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.346+0000 7f729cf63740 1 --2- 172.21.15.69:0/1011603056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560db3c0110 0x5560db42ea20 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.346+0000 7f729cf63740 1 --2- 172.21.15.69:0/1011603056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5560db3b7530 0x5560db3c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.346+0000 7f729cf63740 1 --2- 172.21.15.69:0/1011603056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560db3b6c10 0x5560db3c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.346+0000 7f727cc00640 0 client.27914 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:38.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.346+0000 7f729cf63740 1 -- 172.21.15.69:0/1011603056 >> 172.21.15.69:0/1011603056 conn(0x5560db3b1340 msgr2=0x5560db3bd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:38.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.346+0000 7f729cf63740 1 -- 172.21.15.69:0/1011603056 shutdown_connections 2024-03-19T01:22:38.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.346+0000 7f729cf63740 1 -- 172.21.15.69:0/1011603056 wait complete. 2024-03-19T01:22:38.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.347+0000 7f729cf63740 20 client.27914 trim_cache size 0 max 16384 2024-03-19T01:22:38.350 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:22:38.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.354+0000 7f729a200640 1 -- 172.21.15.69:0/2495335994 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72780a2af0 msgr2=0x7f72780a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:38.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.354+0000 7f729a200640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72780a2af0 0x7f72780a4fb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f729406aa10 tx=0x7f72840323d0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.354+0000 7f729a200640 1 -- 172.21.15.69:0/2495335994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7294108df0 msgr2=0x7f729410a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:38.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.354+0000 7f729a200640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7294108df0 0x7f729410a7d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7280005fd0 tx=0x7f728000ec20 comp rx=0 tx=0).stop 2024-03-19T01:22:38.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.354+0000 7f729a200640 1 -- 172.21.15.69:0/2495335994 shutdown_connections 2024-03-19T01:22:38.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.354+0000 7f729a200640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72941144c0 0x7f729410ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.354+0000 7f729a200640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7294108df0 0x7f729410a7d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.354+0000 7f729a200640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72780a2af0 0x7f72780a4fb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.355+0000 7f729a200640 1 --2- 172.21.15.69:0/2495335994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406dae0 0x7f729410a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:38.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.355+0000 7f729a200640 1 -- 172.21.15.69:0/2495335994 >> 172.21.15.69:0/2495335994 conn(0x7f729406c700 msgr2=0x7f72940700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:38.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.358+0000 7f729a200640 1 -- 172.21.15.69:0/2495335994 shutdown_connections 2024-03-19T01:22:38.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:38.358+0000 7f729a200640 1 -- 172.21.15.69:0/2495335994 wait complete. 2024-03-19T01:22:38.393 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/intersil/built-in.a 2024-03-19T01:22:38.403 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/amd/built-in.a 2024-03-19T01:22:38.406 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:38 smithi069 ceph-mon[31727]: pgmap v373: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 306 KiB/s wr, 36 op/s 2024-03-19T01:22:38.460 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/marvell/built-in.a 2024-03-19T01:22:38.472 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/aquantia/built-in.a 2024-03-19T01:22:38.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:38 smithi086 ceph-mon[66829]: pgmap v373: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 306 KiB/s wr, 36 op/s 2024-03-19T01:22:38.518 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/arc/built-in.a 2024-03-19T01:22:38.532 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/power.o 2024-03-19T01:22:38.538 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/mediatek/built-in.a 2024-03-19T01:22:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:38 smithi163 ceph-mon[67428]: pgmap v373: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 306 KiB/s wr, 36 op/s 2024-03-19T01:22:38.573 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/asix/built-in.a 2024-03-19T01:22:38.596 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/microchip/built-in.a 2024-03-19T01:22:38.700 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/atheros/built-in.a 2024-03-19T01:22:38.706 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/purelifi/built-in.a 2024-03-19T01:22:38.757 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/quantenna/built-in.a 2024-03-19T01:22:38.757 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/cadence/built-in.a 2024-03-19T01:22:38.800 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_common.o 2024-03-19T01:22:38.813 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/ralink/built-in.a 2024-03-19T01:22:38.814 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/broadcom/tg3.o 2024-03-19T01:22:38.853 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/libahci.o 2024-03-19T01:22:38.884 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/realtek/built-in.a 2024-03-19T01:22:38.973 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/rsi/built-in.a 2024-03-19T01:22:39.013 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/gen6_ppgtt.o 2024-03-19T01:22:39.045 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/silabs/built-in.a 2024-03-19T01:22:39.124 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/st/built-in.a 2024-03-19T01:22:39.216 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/ti/built-in.a 2024-03-19T01:22:39.321 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/zydas/built-in.a 2024-03-19T01:22:39.408 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/legacy/built-in.a 2024-03-19T01:22:39.411 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_transport_spi.o 2024-03-19T01:22:39.416 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:22:39.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:22:39.416 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:22:39.463 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/brocade/built-in.a 2024-03-19T01:22:39.526 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/virtual/built-in.a 2024-03-19T01:22:39.538 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/event.o 2024-03-19T01:22:39.572 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/wireless/built-in.a 2024-03-19T01:22:39.573 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/ata_piix.o 2024-03-19T01:22:39.807 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/proc_namespace.o 2024-03-19T01:22:39.927 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/gen7_renderclear.o 2024-03-19T01:22:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.023+0000 7fb3bf600640 1 -- 172.21.15.69:0/1380436719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b8109320 msgr2=0x7fb3b810ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.023+0000 7fb3bf600640 1 --2- 172.21.15.69:0/1380436719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b8109320 0x7fb3b810ce70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009960 tx=0x7fb3a802bc30 comp rx=0 tx=0).stop 2024-03-19T01:22:40.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.023+0000 7fb3bf600640 1 -- 172.21.15.69:0/1380436719 shutdown_connections 2024-03-19T01:22:40.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.023+0000 7fb3bf600640 1 --2- 172.21.15.69:0/1380436719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b810d3b0 0x7fb3b810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.023+0000 7fb3bf600640 1 --2- 172.21.15.69:0/1380436719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b8109320 0x7fb3b810ce70 secure :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009960 tx=0x7fb3a802bc30 comp rx=0 tx=0).stop 2024-03-19T01:22:40.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.023+0000 7fb3bf600640 1 --2- 172.21.15.69:0/1380436719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b8108a00 0x7fb3b8108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.023+0000 7fb3bf600640 1 -- 172.21.15.69:0/1380436719 >> 172.21.15.69:0/1380436719 conn(0x7fb3b806cd60 msgr2=0x7fb3b806d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:40.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.025+0000 7fb3bf600640 1 -- 172.21.15.69:0/1380436719 shutdown_connections 2024-03-19T01:22:40.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.026+0000 7fb3bf600640 1 -- 172.21.15.69:0/1380436719 wait complete. 2024-03-19T01:22:40.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.027+0000 7fb3bf600640 1 Processor -- start 2024-03-19T01:22:40.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.027+0000 7fb3bf600640 1 -- start start 2024-03-19T01:22:40.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.028+0000 7fb3bf600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b8108a00 0x7fb3b81a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.028+0000 7fb3bf600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b810d3b0 0x7fb3b81a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.028+0000 7fb3bf600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b81a0a30 0x7fb3b81a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.029+0000 7fb3b7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b810d3b0 0x7fb3b81a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.029+0000 7fb3b7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b810d3b0 0x7fb3b81a6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50422/0 (socket says 172.21.15.69:50422) 2024-03-19T01:22:40.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.030+0000 7fb3b7e00640 1 -- 172.21.15.69:0/2783606141 learned_addr learned my addr 172.21.15.69:0/2783606141 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.030+0000 7fb3bf600640 1 -- 172.21.15.69:0/2783606141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b8119340 con 0x7fb3b8108a00 2024-03-19T01:22:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.030+0000 7fb3bf600640 1 -- 172.21.15.69:0/2783606141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81191c0 con 0x7fb3b81a0a30 2024-03-19T01:22:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.030+0000 7fb3bf600640 1 -- 172.21.15.69:0/2783606141 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81194c0 con 0x7fb3b810d3b0 2024-03-19T01:22:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.030+0000 7fb3bd600640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b81a0a30 0x7fb3b81a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.030+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b8108a00 0x7fb3b81a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.031+0000 7fb3b7e00640 1 -- 172.21.15.69:0/2783606141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b81a0a30 msgr2=0x7fb3b81a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.031+0000 7fb3b7e00640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b81a0a30 0x7fb3b81a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.031+0000 7fb3b7e00640 1 -- 172.21.15.69:0/2783606141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b8108a00 msgr2=0x7fb3b81a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.031+0000 7fb3b7e00640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b8108a00 0x7fb3b81a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.031+0000 7fb3b7e00640 1 -- 172.21.15.69:0/2783606141 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b81a1550 con 0x7fb3b810d3b0 2024-03-19T01:22:40.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.031+0000 7fb3b7e00640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b810d3b0 0x7fb3b81a6dc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8000c00 tx=0x7fb3a802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:40.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.032+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b8108a00 0x7fb3b81a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:22:40.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.033+0000 7fb3b5600640 1 -- 172.21.15.69:0/2783606141 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8043020 con 0x7fb3b810d3b0 2024-03-19T01:22:40.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.033+0000 7fb3b5600640 1 -- 172.21.15.69:0/2783606141 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a80054e0 con 0x7fb3b810d3b0 2024-03-19T01:22:40.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.033+0000 7fb3b5600640 1 -- 172.21.15.69:0/2783606141 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8003c30 con 0x7fb3b810d3b0 2024-03-19T01:22:40.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.034+0000 7fb3bf600640 1 -- 172.21.15.69:0/2783606141 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b81a17e0 con 0x7fb3b810d3b0 2024-03-19T01:22:40.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.034+0000 7fb3bf600640 1 -- 172.21.15.69:0/2783606141 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b8070ff0 con 0x7fb3b810d3b0 2024-03-19T01:22:40.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.043+0000 7fb3c2201740 1 Processor -- start 2024-03-19T01:22:40.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.044+0000 7fb3c2201740 1 -- start start 2024-03-19T01:22:40.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.044+0000 7fb3c2201740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e62619ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.045+0000 7fb3b5600640 1 -- 172.21.15.69:0/2783606141 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fb3a8046020 con 0x7fb3b810d3b0 2024-03-19T01:22:40.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.045+0000 7fb3bd600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e62619ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.045+0000 7fb3bd600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e62619ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53578/0 (socket says 172.21.15.69:53578) 2024-03-19T01:22:40.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.045+0000 7fb3c2201740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6261a3e0 0x562e62622a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.046+0000 7fb3c2201740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e62622fc0 0x562e626253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.046+0000 7fb3c2201740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e6257cfb0 con 0x562e62619ac0 2024-03-19T01:22:40.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.046+0000 7fb3c2201740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562e62525ab0 con 0x562e6261a3e0 2024-03-19T01:22:40.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.046+0000 7fb3c2201740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562e62579050 con 0x562e62622fc0 2024-03-19T01:22:40.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.046+0000 7fb3bd600640 1 -- 172.21.15.69:0/2626460458 learned_addr learned my addr 172.21.15.69:0/2626460458 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:40.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.046+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/2626460458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6261a3e0 0x562e62622a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.047+0000 7fb3bd600640 1 -- 172.21.15.69:0/2626460458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e62622fc0 msgr2=0x562e626253b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.047+0000 7fb3bd600640 1 --2- 172.21.15.69:0/2626460458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e62622fc0 0x562e626253b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.047+0000 7fb3bd600640 1 -- 172.21.15.69:0/2626460458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6261a3e0 msgr2=0x562e62622a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.047+0000 7fb3bd600640 1 --2- 172.21.15.69:0/2626460458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6261a3e0 0x562e62622a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.047+0000 7fb3bd600640 1 -- 172.21.15.69:0/2626460458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e62543e00 con 0x562e62619ac0 2024-03-19T01:22:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.047+0000 7fb3b5600640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb39c0a2b40 0x7fb39c0a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.047+0000 7fb3b5600640 1 -- 172.21.15.69:0/2783606141 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fb3a80f1100 con 0x7fb3b810d3b0 2024-03-19T01:22:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.047+0000 7fb3bd600640 1 --2- 172.21.15.69:0/2626460458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e62619ea0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0014730 tx=0x7fb3b0014bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc24f408eb92877e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.048+0000 7fb3adc00640 1 -- 172.21.15.69:0/2626460458 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0020030 con 0x562e62619ac0 2024-03-19T01:22:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.048+0000 7fb3adc00640 1 -- 172.21.15.69:0/2626460458 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b0014e40 con 0x562e62619ac0 2024-03-19T01:22:40.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.048+0000 7fb3adc00640 1 -- 172.21.15.69:0/2626460458 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0004240 con 0x562e62619ac0 2024-03-19T01:22:40.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.050+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb39c0a2b40 0x7fb39c0a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.051+0000 7fb3c2201740 1 -- 172.21.15.69:0/2626460458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 msgr2=0x562e62619ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.051+0000 7fb3c2201740 1 --2- 172.21.15.69:0/2626460458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e62619ea0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0014730 tx=0x7fb3b0014bf0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.051+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb39c0a2b40 0x7fb39c0a5000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x562e62619590 tx=0x7fb3a4008040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:40.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.055+0000 7fb3c2201740 1 -- 172.21.15.69:0/2626460458 shutdown_connections 2024-03-19T01:22:40.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.055+0000 7fb3c2201740 1 --2- 172.21.15.69:0/2626460458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e62622fc0 0x562e626253b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.055+0000 7fb3c2201740 1 --2- 172.21.15.69:0/2626460458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6261a3e0 0x562e62622a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.055+0000 7fb3c2201740 1 --2- 172.21.15.69:0/2626460458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e62619ea0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.055+0000 7fb3c2201740 1 -- 172.21.15.69:0/2626460458 >> 172.21.15.69:0/2626460458 conn(0x562e626141f0 msgr2=0x562e626145d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:40.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.059+0000 7fb3c2201740 1 -- 172.21.15.69:0/2626460458 shutdown_connections 2024-03-19T01:22:40.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.059+0000 7fb3c2201740 1 -- 172.21.15.69:0/2626460458 wait complete. 2024-03-19T01:22:40.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.059+0000 7fb3c2201740 1 -- start start 2024-03-19T01:22:40.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.060+0000 7fb3c2201740 1 Processor -- start 2024-03-19T01:22:40.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.061+0000 7fb3c2201740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e6261ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.061+0000 7fb3c2201740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e6261a3e0 0x562e62620480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.061+0000 7fb3c2201740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62622fc0 0x562e626209c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.061+0000 7fb3c2201740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e62625db0 con 0x562e62619ac0 2024-03-19T01:22:40.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.061+0000 7fb3c2201740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562e62538f00 con 0x562e62622fc0 2024-03-19T01:22:40.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.061+0000 7fb3c2201740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562e62625f30 con 0x562e6261a3e0 2024-03-19T01:22:40.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.061+0000 7fb3bcc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e6261ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.061+0000 7fb3bcc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e6261ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53594/0 (socket says 172.21.15.69:53594) 2024-03-19T01:22:40.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.061+0000 7fb3bcc00640 1 -- 172.21.15.69:0/714658015 learned_addr learned my addr 172.21.15.69:0/714658015 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:40.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.061+0000 7fb3c2201740 10 client.0 _ll_register_callbacks cb 0x562e625388d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:40.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.061+0000 7fb3bd600640 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62622fc0 0x562e626209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.062+0000 7fb3bcc00640 1 -- 172.21.15.69:0/714658015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e6261a3e0 msgr2=0x562e62620480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.062+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e6261a3e0 0x562e62620480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.062+0000 7fb3bcc00640 1 -- 172.21.15.69:0/714658015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62622fc0 msgr2=0x562e626209c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.062+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62622fc0 0x562e626209c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.062+0000 7fb3bcc00640 1 -- 172.21.15.69:0/714658015 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e623597e0 con 0x562e62619ac0 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.062+0000 7fb3bd600640 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62622fc0 0x562e626209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.062+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e6261ff40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4015980 tx=0x7fb3a40046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49cdfc383d9ce27d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.062+0000 7fb3a2000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.063+0000 7fb3a2000640 1 -- 172.21.15.69:0/714658015 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4023070 con 0x562e62619ac0 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.063+0000 7fb3c2201740 1 -- 172.21.15.69:0/714658015 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e6258a4f0 con 0x562e62619ac0 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.063+0000 7fb3a2000640 1 -- 172.21.15.69:0/714658015 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fb3a4013b30 con 0x562e62619ac0 2024-03-19T01:22:40.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.063+0000 7fb3a2000640 1 -- 172.21.15.69:0/714658015 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb3a4013d30 con 0x562e62619ac0 2024-03-19T01:22:40.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.064+0000 7fb3c2201740 10 client.17571 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:40.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.064+0000 7fb3c2201740 1 -- 172.21.15.69:0/714658015 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x562e623b9f70 con 0x562e62619ac0 2024-03-19T01:22:40.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.064+0000 7fb3a2000640 1 -- 172.21.15.69:0/714658015 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a402f070 con 0x562e62619ac0 2024-03-19T01:22:40.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.064+0000 7fb3a2000640 1 -- 172.21.15.69:0/714658015 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fb3a401e070 con 0x562e62619ac0 2024-03-19T01:22:40.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.065+0000 7fb3c2201740 10 client.17571 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:40.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.065+0000 7fb3c2201740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:22:40.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.065+0000 7fb3c2201740 20 client.17571 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:40.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.065+0000 7fb3c2201740 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562e62655310 0x562e626577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.065+0000 7fb3c2201740 4 client.17571 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:22:40.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.065+0000 7fb3c2201740 1 -- 172.21.15.69:0/714658015 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e62584840 con 0x562e62655310 2024-03-19T01:22:40.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.066+0000 7fb3b7e00640 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562e62655310 0x562e626577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.066+0000 7fb3b7e00640 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562e62655310 0x562e626577a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69c3b87e39fd8327 server_cookie=1be35526e449f088 in_seq=0 out_seq=0 2024-03-19T01:22:40.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.066+0000 7fb3a2000640 10 client.17571 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:40.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.067+0000 7fb3a2000640 1 -- 172.21.15.69:0/714658015 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562e62584840 con 0x562e62655310 2024-03-19T01:22:40.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.067+0000 7fb3a2000640 10 client.17571 handle_command_reply: tid=0 2024-03-19T01:22:40.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.067+0000 7fb3c2201740 1 client.17571 shutdown 2024-03-19T01:22:40.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.071+0000 7fb3c2201740 1 -- 172.21.15.69:0/714658015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 msgr2=0x562e6261ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.071+0000 7fb3c2201740 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e6261ff40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4015980 tx=0x7fb3a40046e0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.075+0000 7fb3c2201740 1 -- 172.21.15.69:0/714658015 shutdown_connections 2024-03-19T01:22:40.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.075+0000 7fb3c2201740 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562e62655310 0x562e626577a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.075+0000 7fb3c2201740 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62622fc0 0x562e626209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.075+0000 7fb3c2201740 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e6261a3e0 0x562e62620480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.075+0000 7fb3c2201740 1 --2- 172.21.15.69:0/714658015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62619ac0 0x562e6261ff40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.075+0000 7fb3a2000640 0 client.17571 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:40.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.075+0000 7fb3c2201740 1 -- 172.21.15.69:0/714658015 >> 172.21.15.69:0/714658015 conn(0x562e626141f0 msgr2=0x562e62615f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:40.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.075+0000 7fb3c2201740 1 -- 172.21.15.69:0/714658015 shutdown_connections 2024-03-19T01:22:40.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.077+0000 7fb3c2201740 1 -- 172.21.15.69:0/714658015 wait complete. 2024-03-19T01:22:40.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.077+0000 7fb3c2201740 20 client.17571 trim_cache size 0 max 16384 2024-03-19T01:22:40.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.154+0000 7fb3c2201740 1 Processor -- start 2024-03-19T01:22:40.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.154+0000 7fb3c2201740 1 -- start start 2024-03-19T01:22:40.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.155+0000 7fb3c2201740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e62619ac0 0x562e6262eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.155+0000 7fb3c2201740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6261a3e0 0x562e6265d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.155+0000 7fb3c2201740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62622fc0 0x562e6262b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.155+0000 7fb3bcc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e62619ac0 0x562e6262eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.155+0000 7fb3bcc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e62619ac0 0x562e6262eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50440/0 (socket says 172.21.15.69:50440) 2024-03-19T01:22:40.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.155+0000 7fb3bcc00640 1 -- 172.21.15.69:0/320858045 learned_addr learned my addr 172.21.15.69:0/320858045 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:40.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.155+0000 7fb3c2201740 1 -- 172.21.15.69:0/320858045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b807be10 con 0x562e62622fc0 2024-03-19T01:22:40.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.155+0000 7fb3c2201740 1 -- 172.21.15.69:0/320858045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b807d950 con 0x562e6261a3e0 2024-03-19T01:22:40.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.155+0000 7fb3c2201740 1 -- 172.21.15.69:0/320858045 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b807c4e0 con 0x562e62619ac0 2024-03-19T01:22:40.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.156+0000 7fb3bd600640 1 --2- 172.21.15.69:0/320858045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62622fc0 0x562e6262b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.156+0000 7fb3bcc00640 1 -- 172.21.15.69:0/320858045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6261a3e0 msgr2=0x562e6265d0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:40.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.156+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/320858045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6261a3e0 0x562e6265d0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.156+0000 7fb3bcc00640 1 -- 172.21.15.69:0/320858045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62622fc0 msgr2=0x562e6262b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:40.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.156+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/320858045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62622fc0 0x562e6262b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.156+0000 7fb3bcc00640 1 -- 172.21.15.69:0/320858045 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e62251170 con 0x562e62619ac0 2024-03-19T01:22:40.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.156+0000 7fb3bd600640 1 --2- 172.21.15.69:0/320858045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62622fc0 0x562e6262b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:22:40.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.157+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/320858045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e62619ac0 0x562e6262eb70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4006a50 tx=0x7fb3a4013cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=633b74479bb1e133 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:40.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.157+0000 7fb3ad200640 1 -- 172.21.15.69:0/320858045 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a402c070 con 0x562e62619ac0 2024-03-19T01:22:40.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.157+0000 7fb3ad200640 1 -- 172.21.15.69:0/320858045 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a402f040 con 0x562e62619ac0 2024-03-19T01:22:40.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.157+0000 7fb3ad200640 1 -- 172.21.15.69:0/320858045 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4023070 con 0x562e62619ac0 2024-03-19T01:22:40.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.158+0000 7fb3c2201740 1 -- 172.21.15.69:0/320858045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e62619ac0 msgr2=0x562e6262eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.158+0000 7fb3c2201740 1 --2- 172.21.15.69:0/320858045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e62619ac0 0x562e6262eb70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4006a50 tx=0x7fb3a4013cc0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.160+0000 7fb3c2201740 1 -- 172.21.15.69:0/320858045 shutdown_connections 2024-03-19T01:22:40.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.160+0000 7fb3c2201740 1 --2- 172.21.15.69:0/320858045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62622fc0 0x562e6262b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.160+0000 7fb3c2201740 1 --2- 172.21.15.69:0/320858045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e6261a3e0 0x562e6265d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.160+0000 7fb3c2201740 1 --2- 172.21.15.69:0/320858045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e62619ac0 0x562e6262eb70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.160+0000 7fb3c2201740 1 -- 172.21.15.69:0/320858045 >> 172.21.15.69:0/320858045 conn(0x562e626141f0 msgr2=0x562e6261ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.161+0000 7fb3c2201740 1 -- 172.21.15.69:0/320858045 shutdown_connections 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.161+0000 7fb3c2201740 1 -- 172.21.15.69:0/320858045 wait complete. 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.161+0000 7fb3c2201740 1 -- start start 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.161+0000 7fb3c2201740 1 Processor -- start 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.162+0000 7fb3c2201740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62619ac0 0x562e6262b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.162+0000 7fb3c2201740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e6261a3e0 0x562e62628170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.162+0000 7fb3bcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62619ac0 0x562e6262b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3bcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62619ac0 0x562e6262b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45660/0 (socket says 172.21.15.69:45660) 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3bcc00640 1 -- 172.21.15.69:0/3298153393 learned_addr learned my addr 172.21.15.69:0/3298153393 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3c2201740 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62622fc0 0x562e626286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3b7e00640 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e6261a3e0 0x562e62628170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3c2201740 1 -- 172.21.15.69:0/3298153393 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b807d280 con 0x562e62622fc0 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3c2201740 1 -- 172.21.15.69:0/3298153393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b807cbb0 con 0x562e62619ac0 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3c2201740 1 -- 172.21.15.69:0/3298153393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b8079df0 con 0x562e6261a3e0 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3c2201740 10 client.0 _ll_register_callbacks cb 0x7fb3a400fd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3b7e00640 1 -- 172.21.15.69:0/3298153393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62619ac0 msgr2=0x562e6262b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3b7e00640 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62619ac0 0x562e6262b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3b7e00640 1 -- 172.21.15.69:0/3298153393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62622fc0 msgr2=0x562e626286b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3b7e00640 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62622fc0 0x562e626286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.163+0000 7fb3b7e00640 1 -- 172.21.15.69:0/3298153393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e62628bf0 con 0x562e6261a3e0 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.164+0000 7fb3b7e00640 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e6261a3e0 0x562e62628170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80ee510 tx=0x7fb3a80bd2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf6cd459262251fb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.165+0000 7fb3a2000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.165+0000 7fb3a2000640 1 -- 172.21.15.69:0/3298153393 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80bcd80 con 0x562e6261a3e0 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.165+0000 7fb3a2000640 1 -- 172.21.15.69:0/3298153393 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fb3a802cc20 con 0x562e6261a3e0 2024-03-19T01:22:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.165+0000 7fb3c2201740 1 -- 172.21.15.69:0/3298153393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a402f040 con 0x562e6261a3e0 2024-03-19T01:22:40.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.165+0000 7fb3a2000640 1 -- 172.21.15.69:0/3298153393 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb3a80bdb20 con 0x562e6261a3e0 2024-03-19T01:22:40.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.165+0000 7fb3c2201740 10 client.27886 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:40.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.165+0000 7fb3c2201740 1 -- 172.21.15.69:0/3298153393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x562e626b43b0 con 0x562e6261a3e0 2024-03-19T01:22:40.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.166+0000 7fb3a2000640 1 -- 172.21.15.69:0/3298153393 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a402f040 con 0x562e6261a3e0 2024-03-19T01:22:40.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.167+0000 7fb3a2000640 1 -- 172.21.15.69:0/3298153393 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fb3a80b9a90 con 0x562e6261a3e0 2024-03-19T01:22:40.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.167+0000 7fb3c2201740 10 client.27886 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:40.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.167+0000 7fb3c2201740 10 client.27886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:22:40.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.167+0000 7fb3c2201740 20 client.27886 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:40.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.167+0000 7fb3c2201740 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562e626b4bb0 0x562e626b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.167+0000 7fb3c2201740 4 client.27886 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:22:40.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.167+0000 7fb3c2201740 1 -- 172.21.15.69:0/3298153393 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562e62502aa0 con 0x562e626b4bb0 2024-03-19T01:22:40.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.171+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562e626b4bb0 0x562e626b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.179+0000 7fb3a2000640 10 client.27886 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:40.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.179+0000 7fb3bcc00640 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562e626b4bb0 0x562e626b6fd0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef02fa5b9cfc750 server_cookie=e538bcfecb209a53 in_seq=0 out_seq=0 2024-03-19T01:22:40.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3a2000640 1 -- 172.21.15.69:0/3298153393 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562e62502aa0 con 0x562e626b4bb0 2024-03-19T01:22:40.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3a2000640 10 client.27886 handle_command_reply: tid=0 2024-03-19T01:22:40.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3c2201740 1 client.27886 shutdown 2024-03-19T01:22:40.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3c2201740 1 -- 172.21.15.69:0/3298153393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e6261a3e0 msgr2=0x562e62628170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3c2201740 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e6261a3e0 0x562e62628170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80ee510 tx=0x7fb3a80bd2b0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3c2201740 1 -- 172.21.15.69:0/3298153393 shutdown_connections 2024-03-19T01:22:40.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3c2201740 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562e626b4bb0 0x562e626b6fd0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3c2201740 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e62622fc0 0x562e626286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3c2201740 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e6261a3e0 0x562e62628170 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3c2201740 1 --2- 172.21.15.69:0/3298153393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562e62619ac0 0x562e6262b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3c2201740 1 -- 172.21.15.69:0/3298153393 >> 172.21.15.69:0/3298153393 conn(0x562e626141f0 msgr2=0x562e62613270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:40.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.181+0000 7fb3a2000640 0 client.27886 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:40.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.188+0000 7fb3c2201740 1 -- 172.21.15.69:0/3298153393 shutdown_connections 2024-03-19T01:22:40.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.189+0000 7fb3c2201740 1 -- 172.21.15.69:0/3298153393 wait complete. 2024-03-19T01:22:40.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.189+0000 7fb3c2201740 20 client.27886 trim_cache size 0 max 16384 2024-03-19T01:22:40.194 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:22:40.194 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:22:40.194 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "fa17395d-c264-4912-a074-3ab5cd739f56", 2024-03-19T01:22:40.194 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:22:40.194 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:22:40.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.210+0000 7fb3bf600640 1 -- 172.21.15.69:0/2783606141 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb39c0a2b40 msgr2=0x7fb39c0a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.210+0000 7fb3bf600640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb39c0a2b40 0x7fb39c0a5000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x562e62619590 tx=0x7fb3a4008040 comp rx=0 tx=0).stop 2024-03-19T01:22:40.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.211+0000 7fb3bf600640 1 -- 172.21.15.69:0/2783606141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b810d3b0 msgr2=0x7fb3b81a6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.211+0000 7fb3bf600640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b810d3b0 0x7fb3b81a6dc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8000c00 tx=0x7fb3a802c660 comp rx=0 tx=0).stop 2024-03-19T01:22:40.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.213+0000 7fb3bf600640 1 -- 172.21.15.69:0/2783606141 shutdown_connections 2024-03-19T01:22:40.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.213+0000 7fb3bf600640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb39c0a2b40 0x7fb39c0a5000 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.213+0000 7fb3bf600640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b81a0a30 0x7fb3b81a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.213+0000 7fb3bf600640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b810d3b0 0x7fb3b81a6dc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.213+0000 7fb3bf600640 1 --2- 172.21.15.69:0/2783606141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b8108a00 0x7fb3b81a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.213+0000 7fb3bf600640 1 -- 172.21.15.69:0/2783606141 >> 172.21.15.69:0/2783606141 conn(0x7fb3b806cd60 msgr2=0x7fb3b8109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:40.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.215+0000 7fb3bf600640 1 -- 172.21.15.69:0/2783606141 shutdown_connections 2024-03-19T01:22:40.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.215+0000 7fb3bf600640 1 -- 172.21.15.69:0/2783606141 wait complete. 2024-03-19T01:22:40.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:22:40.279 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:22:40.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:40 smithi086 ceph-mon[66829]: pgmap v374: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 260 KiB/s wr, 31 op/s 2024-03-19T01:22:40.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:40 smithi069 ceph-mon[31727]: pgmap v374: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 260 KiB/s wr, 31 op/s 2024-03-19T01:22:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:40 smithi163 ceph-mon[67428]: pgmap v374: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 260 KiB/s wr, 31 op/s 2024-03-19T01:22:40.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.980+0000 7fe072600640 1 -- 172.21.15.69:0/3325703185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0640b7c30 msgr2=0x7fe0640ba020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.980+0000 7fe072600640 1 --2- 172.21.15.69:0/3325703185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0640b7c30 0x7fe0640ba020 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe058009930 tx=0x7fe05802bcb0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.981+0000 7fe072600640 1 -- 172.21.15.69:0/3325703185 shutdown_connections 2024-03-19T01:22:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.981+0000 7fe072600640 1 --2- 172.21.15.69:0/3325703185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0640b7c30 0x7fe0640ba020 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.981+0000 7fe072600640 1 --2- 172.21.15.69:0/3325703185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0640a5830 0x7fe0640b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.981+0000 7fe072600640 1 --2- 172.21.15.69:0/3325703185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0640a4f10 0x7fe0640a52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.981+0000 7fe072600640 1 -- 172.21.15.69:0/3325703185 >> 172.21.15.69:0/3325703185 conn(0x7fe06401a800 msgr2=0x7fe06401ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:40.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.982+0000 7fe072600640 1 -- 172.21.15.69:0/3325703185 shutdown_connections 2024-03-19T01:22:40.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.982+0000 7fe072600640 1 -- 172.21.15.69:0/3325703185 wait complete. 2024-03-19T01:22:40.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.984+0000 7fe072600640 1 Processor -- start 2024-03-19T01:22:40.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.985+0000 7fe072600640 1 -- start start 2024-03-19T01:22:40.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.985+0000 7fe072600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0640a4f10 0x7fe0641497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.985+0000 7fe072600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0640a5830 0x7fe064149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.985+0000 7fe072600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0640b7c30 0x7fe064143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:40.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.985+0000 7fe072600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0640bd730 con 0x7fe0640a5830 2024-03-19T01:22:40.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.985+0000 7fe072600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0640bd5b0 con 0x7fe0640a4f10 2024-03-19T01:22:40.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.985+0000 7fe072600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe0640bd8b0 con 0x7fe0640b7c30 2024-03-19T01:22:40.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.987+0000 7fe071200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0640b7c30 0x7fe064143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.987+0000 7fe071200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0640b7c30 0x7fe064143990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:50462/0 (socket says 172.21.15.69:50462) 2024-03-19T01:22:40.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.987+0000 7fe071200640 1 -- 172.21.15.69:0/3649329472 learned_addr learned my addr 172.21.15.69:0/3649329472 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:40.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.987+0000 7fe06be00640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0640a4f10 0x7fe0641497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:40.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.988+0000 7fe06be00640 1 -- 172.21.15.69:0/3649329472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0640b7c30 msgr2=0x7fe064143990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.988+0000 7fe06be00640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0640b7c30 0x7fe064143990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.988+0000 7fe06be00640 1 -- 172.21.15.69:0/3649329472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0640a5830 msgr2=0x7fe064149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:40.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.988+0000 7fe06be00640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0640a5830 0x7fe064149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:40.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.988+0000 7fe06be00640 1 -- 172.21.15.69:0/3649329472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0641440e0 con 0x7fe0640a4f10 2024-03-19T01:22:40.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.988+0000 7fe06be00640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0640a4f10 0x7fe0641497e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe06000c9a0 tx=0x7fe06000ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:40.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.989+0000 7fe069600640 1 -- 172.21.15.69:0/3649329472 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060007bb0 con 0x7fe0640a4f10 2024-03-19T01:22:40.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.989+0000 7fe069600640 1 -- 172.21.15.69:0/3649329472 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe060007d50 con 0x7fe0640a4f10 2024-03-19T01:22:40.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.989+0000 7fe069600640 1 -- 172.21.15.69:0/3649329472 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0600188c0 con 0x7fe0640a4f10 2024-03-19T01:22:40.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.992+0000 7fe072600640 1 -- 172.21.15.69:0/3649329472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0641443d0 con 0x7fe0640a4f10 2024-03-19T01:22:40.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:40.992+0000 7fe072600640 1 -- 172.21.15.69:0/3649329472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0640aced0 con 0x7fe0640a4f10 2024-03-19T01:22:41.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.009+0000 7fe069600640 1 -- 172.21.15.69:0/3649329472 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fe060018a60 con 0x7fe0640a4f10 2024-03-19T01:22:41.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.012+0000 7fe076839740 1 Processor -- start 2024-03-19T01:22:41.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.014+0000 7fe069600640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe0540a2a30 0x7fe0540a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.014+0000 7fe069600640 1 -- 172.21.15.69:0/3649329472 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fe0600cf1f0 con 0x7fe0640a4f10 2024-03-19T01:22:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.016+0000 7fe071200640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe0540a2a30 0x7fe0540a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.017+0000 7fe076839740 1 -- start start 2024-03-19T01:22:41.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.018+0000 7fe076839740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078bf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.018+0000 7fe076839740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 0x55d8078c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.018+0000 7fe076839740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.018+0000 7fe076839740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d8078232c0 con 0x55d8078bef50 2024-03-19T01:22:41.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.018+0000 7fe076839740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8077cbb60 con 0x55d8078bf870 2024-03-19T01:22:41.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.018+0000 7fe076839740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d80781f360 con 0x55d8078c8450 2024-03-19T01:22:41.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.018+0000 7fe071200640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe0540a2a30 0x7fe0540a4ef0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe058006010 tx=0x7fe058004120 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:41.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.018+0000 7fe06b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 0x55d8078c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.018+0000 7fe06b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 0x55d8078c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45680/0 (socket says 172.21.15.69:45680) 2024-03-19T01:22:41.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.018+0000 7fe06b400640 1 -- 172.21.15.69:0/272051697 learned_addr learned my addr 172.21.15.69:0/272051697 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:41.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.018+0000 7fe071200640 1 --2- 172.21.15.69:0/272051697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.019+0000 7fe06b400640 1 -- 172.21.15.69:0/272051697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 msgr2=0x55d8078ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:41.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.019+0000 7fe06b400640 1 --2- 172.21.15.69:0/272051697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.019+0000 7fe06b400640 1 -- 172.21.15.69:0/272051697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 msgr2=0x55d8078bf330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:22:41.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.020+0000 7fe06be00640 1 --2- 172.21.15.69:0/272051697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078bf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.020+0000 7fe06b400640 1 --2- 172.21.15.69:0/272051697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078bf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.020+0000 7fe06b400640 1 -- 172.21.15.69:0/272051697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8076017d0 con 0x55d8078bf870 2024-03-19T01:22:41.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.020+0000 7fe06be00640 1 --2- 172.21.15.69:0/272051697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078bf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:22:41.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.021+0000 7fe06b400640 1 --2- 172.21.15.69:0/272051697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 0x55d8078c7f10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe06c04f850 tx=0x7fe06c06f5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf73023d5167838f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:41.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.022+0000 7fe05e000640 1 -- 172.21.15.69:0/272051697 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c075420 con 0x55d8078bf870 2024-03-19T01:22:41.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.022+0000 7fe05e000640 1 -- 172.21.15.69:0/272051697 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c075aa0 con 0x55d8078bf870 2024-03-19T01:22:41.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.022+0000 7fe05e000640 1 -- 172.21.15.69:0/272051697 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c074c90 con 0x55d8078bf870 2024-03-19T01:22:41.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.023+0000 7fe076839740 1 -- 172.21.15.69:0/272051697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 msgr2=0x55d8078c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:41.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.023+0000 7fe076839740 1 --2- 172.21.15.69:0/272051697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 0x55d8078c7f10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe06c04f850 tx=0x7fe06c06f5f0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.028+0000 7fe076839740 1 -- 172.21.15.69:0/272051697 shutdown_connections 2024-03-19T01:22:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.028+0000 7fe076839740 1 --2- 172.21.15.69:0/272051697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.028+0000 7fe076839740 1 --2- 172.21.15.69:0/272051697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 0x55d8078c7f10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.028+0000 7fe076839740 1 --2- 172.21.15.69:0/272051697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078bf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.028+0000 7fe076839740 1 -- 172.21.15.69:0/272051697 >> 172.21.15.69:0/272051697 conn(0x55d8078b9680 msgr2=0x55d8078b9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.028+0000 7fe076839740 1 -- 172.21.15.69:0/272051697 shutdown_connections 2024-03-19T01:22:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.028+0000 7fe076839740 1 -- 172.21.15.69:0/272051697 wait complete. 2024-03-19T01:22:41.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.030+0000 7fe076839740 1 -- start start 2024-03-19T01:22:41.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.031+0000 7fe076839740 1 Processor -- start 2024-03-19T01:22:41.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.033+0000 7fe076839740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.033+0000 7fe076839740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078bf870 0x55d8078c5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.034+0000 7fe076839740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078c8450 0x55d8078c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.034+0000 7fe076839740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d8078cb240 con 0x55d8078bef50 2024-03-19T01:22:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.034+0000 7fe071200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078c8450 0x55d8078c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.034+0000 7fe071200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078c8450 0x55d8078c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45686/0 (socket says 172.21.15.69:45686) 2024-03-19T01:22:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.034+0000 7fe071200640 1 -- 172.21.15.69:0/2979341393 learned_addr learned my addr 172.21.15.69:0/2979341393 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.034+0000 7fe076839740 1 -- 172.21.15.69:0/2979341393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8077defc0 con 0x55d8078c8450 2024-03-19T01:22:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.034+0000 7fe076839740 1 -- 172.21.15.69:0/2979341393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d8078cb3c0 con 0x55d8078bf870 2024-03-19T01:22:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.035+0000 7fe076839740 10 client.0 _ll_register_callbacks cb 0x55d8077de990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.035+0000 7fe06b400640 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078bf870 0x55d8078c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.035+0000 7fe06be00640 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.036+0000 7fe06be00640 1 -- 172.21.15.69:0/2979341393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078bf870 msgr2=0x55d8078c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.036+0000 7fe06be00640 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078bf870 0x55d8078c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.036+0000 7fe06be00640 1 -- 172.21.15.69:0/2979341393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078c8450 msgr2=0x55d8078c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.036+0000 7fe06be00640 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078c8450 0x55d8078c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.036+0000 7fe06be00640 1 -- 172.21.15.69:0/2979341393 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8075373f0 con 0x55d8078bef50 2024-03-19T01:22:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.037+0000 7fe052a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:22:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.037+0000 7fe06be00640 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078c5430 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe0600cc500 tx=0x7fe06009ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37375683c7454a18 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.038+0000 7fe052a00640 1 -- 172.21.15.69:0/2979341393 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0600a3be0 con 0x55d8078bef50 2024-03-19T01:22:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.038+0000 7fe076839740 1 -- 172.21.15.69:0/2979341393 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe06c075aa0 con 0x55d8078bef50 2024-03-19T01:22:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.038+0000 7fe052a00640 1 -- 172.21.15.69:0/2979341393 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fe0600b4060 con 0x55d8078bef50 2024-03-19T01:22:41.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.039+0000 7fe052a00640 1 -- 172.21.15.69:0/2979341393 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe0600b9050 con 0x55d8078bef50 2024-03-19T01:22:41.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.039+0000 7fe076839740 10 client.17577 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:41.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.040+0000 7fe076839740 1 -- 172.21.15.69:0/2979341393 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55d8074f70f0 con 0x55d8078bef50 2024-03-19T01:22:41.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.041+0000 7fe052a00640 1 -- 172.21.15.69:0/2979341393 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c075aa0 con 0x55d8078bef50 2024-03-19T01:22:41.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.041+0000 7fe052a00640 1 -- 172.21.15.69:0/2979341393 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe060020020 con 0x55d8078bef50 2024-03-19T01:22:41.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.041+0000 7fe076839740 10 client.17577 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:41.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.041+0000 7fe076839740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:22:41.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.041+0000 7fe076839740 20 client.17577 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:41.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.041+0000 7fe076839740 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8078d86a0 0x55d8078daab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.042+0000 7fe076839740 4 client.17577 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:22:41.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.042+0000 7fe076839740 1 -- 172.21.15.69:0/2979341393 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8077c4610 con 0x55d8078d86a0 2024-03-19T01:22:41.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.042+0000 7fe06b400640 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8078d86a0 0x55d8078daab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.043+0000 7fe06b400640 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8078d86a0 0x55d8078daab0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b08a0cb1aea0d server_cookie=372662ce15a8b56a in_seq=0 out_seq=0 2024-03-19T01:22:41.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.043+0000 7fe052a00640 10 client.17577 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:41.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.048+0000 7fe052a00640 1 -- 172.21.15.69:0/2979341393 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d8077c4610 con 0x55d8078d86a0 2024-03-19T01:22:41.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.048+0000 7fe052a00640 10 client.17577 handle_command_reply: tid=0 2024-03-19T01:22:41.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.048+0000 7fe076839740 1 client.17577 shutdown 2024-03-19T01:22:41.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.051+0000 7fe076839740 1 -- 172.21.15.69:0/2979341393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 msgr2=0x55d8078c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:41.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.051+0000 7fe076839740 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078c5430 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe0600cc500 tx=0x7fe06009ae90 comp rx=0 tx=0).stop 2024-03-19T01:22:41.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.060+0000 7fe076839740 1 -- 172.21.15.69:0/2979341393 shutdown_connections 2024-03-19T01:22:41.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.060+0000 7fe076839740 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8078d86a0 0x55d8078daab0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.060+0000 7fe076839740 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078c8450 0x55d8078c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.060+0000 7fe052a00640 0 client.17577 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:41.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.060+0000 7fe076839740 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078bf870 0x55d8078c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.060+0000 7fe076839740 1 --2- 172.21.15.69:0/2979341393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078c5430 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.060+0000 7fe076839740 1 -- 172.21.15.69:0/2979341393 >> 172.21.15.69:0/2979341393 conn(0x55d8078b9680 msgr2=0x55d8078bb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:41.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.061+0000 7fe076839740 1 -- 172.21.15.69:0/2979341393 shutdown_connections 2024-03-19T01:22:41.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.061+0000 7fe076839740 1 -- 172.21.15.69:0/2979341393 wait complete. 2024-03-19T01:22:41.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.062+0000 7fe076839740 20 client.17577 trim_cache size 0 max 16384 2024-03-19T01:22:41.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.117+0000 7fe076839740 1 Processor -- start 2024-03-19T01:22:41.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.117+0000 7fe076839740 1 -- start start 2024-03-19T01:22:41.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.118+0000 7fe076839740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.119+0000 7fe076839740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 0x55d8078d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.119+0000 7fe076839740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078b8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.119+0000 7fe076839740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe064163310 con 0x55d8078bef50 2024-03-19T01:22:41.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.119+0000 7fe06be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.119+0000 7fe071200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.119+0000 7fe076839740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe064164e50 con 0x55d8078bf870 2024-03-19T01:22:41.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.119+0000 7fe06be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53662/0 (socket says 172.21.15.69:53662) 2024-03-19T01:22:41.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.119+0000 7fe076839740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641639e0 con 0x55d8078c8450 2024-03-19T01:22:41.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.119+0000 7fe06be00640 1 -- 172.21.15.69:0/1853551361 learned_addr learned my addr 172.21.15.69:0/1853551361 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:41.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.120+0000 7fe06b400640 1 --2- 172.21.15.69:0/1853551361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 0x55d8078d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.120+0000 7fe06b400640 1 -- 172.21.15.69:0/1853551361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 msgr2=0x55d8078b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:41.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.120+0000 7fe06b400640 1 --2- 172.21.15.69:0/1853551361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078b8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.120+0000 7fe06b400640 1 -- 172.21.15.69:0/1853551361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 msgr2=0x55d8078c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:41.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.120+0000 7fe06b400640 1 --2- 172.21.15.69:0/1853551361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.121+0000 7fe06b400640 1 -- 172.21.15.69:0/1853551361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8078be300 con 0x55d8078bf870 2024-03-19T01:22:41.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.121+0000 7fe06b400640 1 --2- 172.21.15.69:0/1853551361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 0x55d8078d4070 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe06c050330 tx=0x7fe06c067ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=318eb35ad9081f70 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:41.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.122+0000 7fe071200640 1 --2- 172.21.15.69:0/1853551361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:22:41.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.122+0000 7fe05d600640 1 -- 172.21.15.69:0/1853551361 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c075420 con 0x55d8078bf870 2024-03-19T01:22:41.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.122+0000 7fe05d600640 1 -- 172.21.15.69:0/1853551361 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c075aa0 con 0x55d8078bf870 2024-03-19T01:22:41.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.122+0000 7fe05d600640 1 -- 172.21.15.69:0/1853551361 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c07f410 con 0x55d8078bf870 2024-03-19T01:22:41.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.123+0000 7fe076839740 1 -- 172.21.15.69:0/1853551361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 msgr2=0x55d8078d4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:41.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.123+0000 7fe076839740 1 --2- 172.21.15.69:0/1853551361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 0x55d8078d4070 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe06c050330 tx=0x7fe06c067ad0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.124+0000 7fe076839740 1 -- 172.21.15.69:0/1853551361 shutdown_connections 2024-03-19T01:22:41.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.124+0000 7fe076839740 1 --2- 172.21.15.69:0/1853551361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.124+0000 7fe076839740 1 --2- 172.21.15.69:0/1853551361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bf870 0x55d8078d4070 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.124+0000 7fe076839740 1 --2- 172.21.15.69:0/1853551361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bef50 0x55d8078c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.124+0000 7fe076839740 1 -- 172.21.15.69:0/1853551361 >> 172.21.15.69:0/1853551361 conn(0x55d8078b9680 msgr2=0x55d8078b8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:41.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.128+0000 7fe076839740 1 -- 172.21.15.69:0/1853551361 shutdown_connections 2024-03-19T01:22:41.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.128+0000 7fe076839740 1 -- 172.21.15.69:0/1853551361 wait complete. 2024-03-19T01:22:41.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.129+0000 7fe076839740 1 -- start start 2024-03-19T01:22:41.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.129+0000 7fe076839740 1 Processor -- start 2024-03-19T01:22:41.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.130+0000 7fe076839740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bef50 0x55d8078c2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.130+0000 7fe076839740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bf870 0x55d8078c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.130+0000 7fe06be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bef50 0x55d8078c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.131 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pcmcia/cs.o 2024-03-19T01:22:41.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.130+0000 7fe06be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bef50 0x55d8078c2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:45704/0 (socket says 172.21.15.69:45704) 2024-03-19T01:22:41.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.131+0000 7fe076839740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.131+0000 7fe076839740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe064164780 con 0x55d8078bf870 2024-03-19T01:22:41.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.131+0000 7fe076839740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641640b0 con 0x55d8078bef50 2024-03-19T01:22:41.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.131+0000 7fe076839740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe064005290 con 0x55d8078c8450 2024-03-19T01:22:41.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.131+0000 7fe076839740 10 client.0 _ll_register_callbacks cb 0x7fe06009b010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:22:41.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.131+0000 7fe06b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bf870 0x55d8078c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.131+0000 7fe06be00640 1 -- 172.21.15.69:0/143089807 learned_addr learned my addr 172.21.15.69:0/143089807 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:22:41.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.131+0000 7fe071200640 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.132+0000 7fe06be00640 1 -- 172.21.15.69:0/143089807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 msgr2=0x55d8078c2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:41.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.132+0000 7fe06be00640 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078c2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.132+0000 7fe06be00640 1 -- 172.21.15.69:0/143089807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bf870 msgr2=0x55d8078c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:22:41.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.132+0000 7fe06be00640 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bf870 0x55d8078c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.132+0000 7fe06be00640 1 -- 172.21.15.69:0/143089807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d80793da30 con 0x55d8078bef50 2024-03-19T01:22:41.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.133+0000 7fe06be00640 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bef50 0x55d8078c2060 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe060095fd0 tx=0x7fe06009ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a21de9c8ab49e0f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:22:41.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.133+0000 7fe052a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:22:41.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.134+0000 7fe052a00640 1 -- 172.21.15.69:0/143089807 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0600a3be0 con 0x55d8078bef50 2024-03-19T01:22:41.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.134+0000 7fe052a00640 1 -- 172.21.15.69:0/143089807 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9568+0+0 (secure 0 0 0) 0x7fe0600b4060 con 0x55d8078bef50 2024-03-19T01:22:41.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.136+0000 7fe076839740 1 -- 172.21.15.69:0/143089807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe06c075aa0 con 0x55d8078bef50 2024-03-19T01:22:41.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.150+0000 7fe052a00640 1 -- 172.21.15.69:0/143089807 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe0600976e0 con 0x55d8078bef50 2024-03-19T01:22:41.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.150+0000 7fe076839740 10 client.27944 fetch_fsmap learned FSMap version 59 2024-03-19T01:22:41.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.150+0000 7fe076839740 1 -- 172.21.15.69:0/143089807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55d80793da30 con 0x55d8078bef50 2024-03-19T01:22:41.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.153+0000 7fe052a00640 1 -- 172.21.15.69:0/143089807 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c075aa0 con 0x55d8078bef50 2024-03-19T01:22:41.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.153+0000 7fe052a00640 1 -- 172.21.15.69:0/143089807 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe060020070 con 0x55d8078bef50 2024-03-19T01:22:41.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.155+0000 7fe076839740 10 client.27944 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:22:41.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.155+0000 7fe076839740 10 client.27944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:22:41.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.155+0000 7fe076839740 20 client.27944 populate_metadata read hostname 'smithi069' 2024-03-19T01:22:41.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.155+0000 7fe076839740 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8078d86a0 0x55d807941140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:22:41.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.155+0000 7fe076839740 4 client.27944 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:22:41.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.155+0000 7fe076839740 1 -- 172.21.15.69:0/143089807 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d807830800 con 0x55d8078d86a0 2024-03-19T01:22:41.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.155+0000 7fe06b400640 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8078d86a0 0x55d807941140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:22:41.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.156+0000 7fe06b400640 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8078d86a0 0x55d807941140 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feadfc69a6790c04 server_cookie=f591c2e592e194de in_seq=0 out_seq=0 2024-03-19T01:22:41.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.156+0000 7fe052a00640 10 client.27944 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:41.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.156+0000 7fe052a00640 1 -- 172.21.15.69:0/143089807 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55d807830800 con 0x55d8078d86a0 2024-03-19T01:22:41.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.156+0000 7fe052a00640 10 client.27944 handle_command_reply: tid=0 2024-03-19T01:22:41.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.156+0000 7fe076839740 1 client.27944 shutdown 2024-03-19T01:22:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.164+0000 7fe076839740 1 -- 172.21.15.69:0/143089807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bef50 msgr2=0x55d8078c2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.164+0000 7fe076839740 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bef50 0x55d8078c2060 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe060095fd0 tx=0x7fe06009ada0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.164+0000 7fe076839740 1 -- 172.21.15.69:0/143089807 shutdown_connections 2024-03-19T01:22:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.164+0000 7fe076839740 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55d8078d86a0 0x55d807941140 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.164+0000 7fe076839740 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d8078c8450 0x55d8078c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.164+0000 7fe076839740 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d8078bf870 0x55d8078c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.164+0000 7fe076839740 1 --2- 172.21.15.69:0/143089807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8078bef50 0x55d8078c2060 secure :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe060095fd0 tx=0x7fe06009ada0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.164+0000 7fe076839740 1 -- 172.21.15.69:0/143089807 >> 172.21.15.69:0/143089807 conn(0x55d8078b9680 msgr2=0x55d8078c5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:41.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.164+0000 7fe052a00640 0 client.27944 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:22:41.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.165+0000 7fe076839740 1 -- 172.21.15.69:0/143089807 shutdown_connections 2024-03-19T01:22:41.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.166+0000 7fe076839740 1 -- 172.21.15.69:0/143089807 wait complete. 2024-03-19T01:22:41.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.166+0000 7fe076839740 20 client.27944 trim_cache size 0 max 16384 2024-03-19T01:22:41.174 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:22:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (11731 inodes in the stack)", 2024-03-19T01:22:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:22:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "fa17395d-c264-4912-a074-3ab5cd739f56": { 2024-03-19T01:22:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:22:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "fa17395d-c264-4912-a074-3ab5cd739f56", 2024-03-19T01:22:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:22:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:22:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:22:41.175 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:22:41.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.177+0000 7fe072600640 1 -- 172.21.15.69:0/3649329472 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe0540a2a30 msgr2=0x7fe0540a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:41.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.177+0000 7fe072600640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe0540a2a30 0x7fe0540a4ef0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe058006010 tx=0x7fe058004120 comp rx=0 tx=0).stop 2024-03-19T01:22:41.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.177+0000 7fe072600640 1 -- 172.21.15.69:0/3649329472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0640a4f10 msgr2=0x7fe0641497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:22:41.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.177+0000 7fe072600640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0640a4f10 0x7fe0641497e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe06000c9a0 tx=0x7fe06000ce60 comp rx=0 tx=0).stop 2024-03-19T01:22:41.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.178+0000 7fe072600640 1 -- 172.21.15.69:0/3649329472 shutdown_connections 2024-03-19T01:22:41.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.178+0000 7fe072600640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0640b7c30 0x7fe064143990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.178+0000 7fe072600640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0640a5830 0x7fe064149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.178+0000 7fe072600640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe0540a2a30 0x7fe0540a4ef0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.178+0000 7fe072600640 1 --2- 172.21.15.69:0/3649329472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0640a4f10 0x7fe0641497e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:22:41.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.178+0000 7fe072600640 1 -- 172.21.15.69:0/3649329472 >> 172.21.15.69:0/3649329472 conn(0x7fe06401a800 msgr2=0x7fe0640b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:22:41.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.179+0000 7fe072600640 1 -- 172.21.15.69:0/3649329472 shutdown_connections 2024-03-19T01:22:41.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:22:41.179+0000 7fe072600640 1 -- 172.21.15.69:0/3649329472 wait complete. 2024-03-19T01:22:41.216 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/direct-io.o 2024-03-19T01:22:41.230 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/evged.o 2024-03-19T01:22:41.265 INFO:tasks.cephfs.filesystem:scrub status for tag:fa17395d-c264-4912-a074-3ab5cd739f56 - {'path': '/', 'tag': 'fa17395d-c264-4912-a074-3ab5cd739f56', 'options': 'recursive,force'} 2024-03-19T01:22:41.416 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pcmcia/socket_sysfs.o 2024-03-19T01:22:41.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:41 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:22:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:41 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:41.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:41 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:41.730 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/gen8_engine_cs.o 2024-03-19T01:22:41.782 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/virtio_scsi.o 2024-03-19T01:22:42.017 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/pata_amd.o 2024-03-19T01:22:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: pgmap v375: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 222 KiB/s wr, 24 op/s 2024-03-19T01:22:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:42.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:42.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:42 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: pgmap v375: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 222 KiB/s wr, 24 op/s 2024-03-19T01:22:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:42 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: pgmap v375: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 222 KiB/s wr, 24 op/s 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:42.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:42.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:42 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:42.816 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/sysfs.o 2024-03-19T01:22:42.897 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/eventpoll.o 2024-03-19T01:22:43.226 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/gen8_ppgtt.o 2024-03-19T01:22:43.351 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pcmcia/cardbus.o 2024-03-19T01:22:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:43 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:43.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:43 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:43 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:43.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:43 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:43 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:43.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:43 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:43.582 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/sd.o 2024-03-19T01:22:43.851 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/pata_oldpiix.o 2024-03-19T01:22:44.124 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_atomic.o 2024-03-19T01:22:44.228 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/property.o 2024-03-19T01:22:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: pgmap v376: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 779 KiB/s wr, 62 op/s 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:44.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:44 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: pgmap v376: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 779 KiB/s wr, 62 op/s 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:44 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: pgmap v376: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 779 KiB/s wr, 62 op/s 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:44.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:44 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:44.904 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pcmcia/ds.o 2024-03-19T01:22:45.490 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ata/pata_sch.o 2024-03-19T01:22:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:45.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:45 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:45.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:45 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:45.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:45 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:45.674 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_breadcrumbs.o 2024-03-19T01:22:45.824 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/anon_inodes.o 2024-03-19T01:22:45.946 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpi_cmos_rtc.o 2024-03-19T01:22:46.448 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pcmcia/pcmcia_resource.o 2024-03-19T01:22:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: pgmap v377: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 871 KiB/s wr, 68 op/s 2024-03-19T01:22:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:46.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:46 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: pgmap v377: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 871 KiB/s wr, 68 op/s 2024-03-19T01:22:46.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:46.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:46 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: pgmap v377: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 871 KiB/s wr, 68 op/s 2024-03-19T01:22:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:46.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:46.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:46 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.091 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/signalfd.o 2024-03-19T01:22:47.104 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/sr.o 2024-03-19T01:22:47.126 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/x86/apple.o 2024-03-19T01:22:47.140 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/usb/built-in.a 2024-03-19T01:22:47.159 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/ata/built-in.a 2024-03-19T01:22:47.246 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/cavium/common/built-in.a 2024-03-19T01:22:47.322 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/cavium/thunder/built-in.a 2024-03-19T01:22:47.380 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/cavium/liquidio/built-in.a 2024-03-19T01:22:47.429 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_context.o 2024-03-19T01:22:47.466 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/cavium/octeon/built-in.a 2024-03-19T01:22:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:47.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:47 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:47.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:47.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:47 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.542 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/cavium/built-in.a 2024-03-19T01:22:47.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:47.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:47.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:47.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:47 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:47.705 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/timerfd.o 2024-03-19T01:22:47.819 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/sr_ioctl.o 2024-03-19T01:22:48.154 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/x86/utils.o 2024-03-19T01:22:48.293 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pcmcia/cistpl.o 2024-03-19T01:22:48.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: pgmap v378: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:48.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:48 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: pgmap v378: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-03-19T01:22:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:48.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:48 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.541 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/sr_vendor.o 2024-03-19T01:22:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: pgmap v378: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-03-19T01:22:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:48.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:48 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:48.918 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/x86/s2idle.o 2024-03-19T01:22:49.066 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_atomic_uapi.o 2024-03-19T01:22:49.481 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_context_sseu.o 2024-03-19T01:22:49.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: pgmap v379: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.3 MiB/s wr, 289 op/s 2024-03-19T01:22:49.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:49.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:49 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: pgmap v379: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.3 MiB/s wr, 289 op/s 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:49.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:49 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: pgmap v379: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1.3 MiB/s wr, 289 op/s 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:49.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:49 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:49.612 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/eventfd.o 2024-03-19T01:22:49.673 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/debugfs.o 2024-03-19T01:22:49.864 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/sg.o 2024-03-19T01:22:50.267 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pcmcia/pcmcia_cis.o 2024-03-19T01:22:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:50.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:50 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:50.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:50 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.611 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/broadcom/built-in.a 2024-03-19T01:22:50.640 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/common/common.o 2024-03-19T01:22:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:50.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:50 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:50.850 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/chelsio/built-in.a 2024-03-19T01:22:50.931 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/cisco/built-in.a 2024-03-19T01:22:50.932 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpi_lpat.o 2024-03-19T01:22:50.999 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/aio.o 2024-03-19T01:22:51.027 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/cortina/built-in.a 2024-03-19T01:22:51.186 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/dec/tulip/built-in.a 2024-03-19T01:22:51.251 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_auth.o 2024-03-19T01:22:51.258 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/dec/built-in.a 2024-03-19T01:22:51.368 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/dlink/built-in.a 2024-03-19T01:22:51.432 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_engine_cs.o 2024-03-19T01:22:51.450 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/emulex/built-in.a 2024-03-19T01:22:51.537 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/engleder/built-in.a 2024-03-19T01:22:51.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: pgmap v380: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 385 op/s 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:51.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.632 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/ezchip/built-in.a 2024-03-19T01:22:51.738 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/fujitsu/built-in.a 2024-03-19T01:22:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: pgmap v380: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 385 op/s 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:51.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: pgmap v380: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 385 op/s 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:51.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:51.850 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/fungible/built-in.a 2024-03-19T01:22:51.929 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpi_lpit.o 2024-03-19T01:22:51.951 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/google/built-in.a 2024-03-19T01:22:51.963 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pcmcia/rsrc_mgr.o 2024-03-19T01:22:51.968 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/common/debug.o 2024-03-19T01:22:52.040 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/huawei/built-in.a 2024-03-19T01:22:52.177 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000/e1000_main.o 2024-03-19T01:22:52.297 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/scsi/scsi_sysfs.o 2024-03-19T01:22:52.511 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/usb/common/built-in.a 2024-03-19T01:22:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:52.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.775 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/usb.o 2024-03-19T01:22:52.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:52.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:52.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:52.792 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000e/82571.o 2024-03-19T01:22:52.909 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/prmt.o 2024-03-19T01:22:53.255 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pcmcia/rsrc_nonstatic.o 2024-03-19T01:22:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:53 smithi163 ceph-mon[67428]: pgmap v381: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 1.39k op/s 2024-03-19T01:22:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.626 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/locks.o 2024-03-19T01:22:53.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:53 smithi086 ceph-mon[66829]: pgmap v381: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 1.39k op/s 2024-03-19T01:22:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:53 smithi069 ceph-mon[31727]: pgmap v381: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 1.39k op/s 2024-03-19T01:22:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:53.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:54.064 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/scsi/built-in.a 2024-03-19T01:22:54.165 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/hub.o 2024-03-19T01:22:54.219 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/binfmt_misc.o 2024-03-19T01:22:54.344 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpi_pcc.o 2024-03-19T01:22:54.352 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_engine_heartbeat.o 2024-03-19T01:22:54.656 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:54.656 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:54.656 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:54.656 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:54.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:54.657 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:54.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:54.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:54.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:54.880 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pcmcia/yenta_socket.o 2024-03-19T01:22:55.063 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000e/ich8lan.o 2024-03-19T01:22:55.283 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/ac.o 2024-03-19T01:22:55.717 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/mii.o 2024-03-19T01:22:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: pgmap v382: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 843 KiB/s wr, 1.61k op/s 2024-03-19T01:22:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: pgmap v382: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 843 KiB/s wr, 1.61k op/s 2024-03-19T01:22:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:55.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: pgmap v382: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 843 KiB/s wr, 1.61k op/s 2024-03-19T01:22:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:55.828 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000/e1000_hw.o 2024-03-19T01:22:56.262 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/button.o 2024-03-19T01:22:56.681 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_engine_pm.o 2024-03-19T01:22:56.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:56.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:56.772 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/binfmt_script.o 2024-03-19T01:22:56.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:56.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:56.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.034 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pcmcia/built-in.a 2024-03-19T01:22:57.134 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000/e1000_ethtool.o 2024-03-19T01:22:57.719 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/hcd.o 2024-03-19T01:22:57.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:57 smithi086 ceph-mon[66829]: pgmap v383: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 694 KiB/s wr, 1.70k op/s 2024-03-19T01:22:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:57.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:57 smithi069 ceph-mon[31727]: pgmap v383: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 694 KiB/s wr, 1.70k op/s 2024-03-19T01:22:57.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:57.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:57.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:57.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:57.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:57 smithi163 ceph-mon[67428]: pgmap v383: 129 pgs: 129 active+clean; 2.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 694 KiB/s wr, 1.70k op/s 2024-03-19T01:22:57.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:57.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:57.825 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/binfmt_elf.o 2024-03-19T01:22:57.837 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/fan_core.o 2024-03-19T01:22:57.906 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/usb/phy/built-in.a 2024-03-19T01:22:57.997 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/mon/mon_main.o 2024-03-19T01:22:58.308 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000e/80003es2lan.o 2024-03-19T01:22:58.446 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_engine_user.o 2024-03-19T01:22:58.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:58.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:58.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:58.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:58.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:58.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:58.937 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/fan_attr.o 2024-03-19T01:22:59.056 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e100.o 2024-03-19T01:22:59.241 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/mon/mon_stat.o 2024-03-19T01:22:59.730 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000/e1000_param.o 2024-03-19T01:22:59.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:59.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:59 smithi086 ceph-mon[66829]: pgmap v384: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 430 KiB/s wr, 1.76k op/s 2024-03-19T01:22:59.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:59.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:59.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:22:59.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:22:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:59.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:59 smithi069 ceph-mon[31727]: pgmap v384: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 430 KiB/s wr, 1.76k op/s 2024-03-19T01:22:59.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:22:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:22:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:59 smithi163 ceph-mon[67428]: pgmap v384: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 430 KiB/s wr, 1.76k op/s 2024-03-19T01:22:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:22:59.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:22:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:22:59.832 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpi_video.o 2024-03-19T01:23:00.039 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/compat_binfmt_elf.o 2024-03-19T01:23:00.235 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/urb.o 2024-03-19T01:23:00.278 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_execlists_submission.o 2024-03-19T01:23:00.302 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000e/mac.o 2024-03-19T01:23:00.404 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/mon/mon_text.o 2024-03-19T01:23:00.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:00.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:00.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:00.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:00.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:00.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:00.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:00.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:00.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:00.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.680 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/intel/e1000/built-in.a 2024-03-19T01:23:01.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:01.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:01.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:01 smithi086 ceph-mon[66829]: pgmap v385: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 286 KiB/s wr, 1.60k op/s 2024-03-19T01:23:01.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:01.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:01.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.771 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/video_detect.o 2024-03-19T01:23:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:01 smithi069 ceph-mon[31727]: pgmap v385: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 286 KiB/s wr, 1.60k op/s 2024-03-19T01:23:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:01.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:01.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:01.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:01 smithi163 ceph-mon[67428]: pgmap v385: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 286 KiB/s wr, 1.60k op/s 2024-03-19T01:23:01.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:01.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:01.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:01.814 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/serio/serio.o 2024-03-19T01:23:01.922 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/message.o 2024-03-19T01:23:01.957 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/mon/mon_bin.o 2024-03-19T01:23:02.095 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/i825xx/built-in.a 2024-03-19T01:23:02.184 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/loopback.o 2024-03-19T01:23:02.265 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/mbcache.o 2024-03-19T01:23:02.612 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000e/manage.o 2024-03-19T01:23:02.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:02.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:02.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:02.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:02.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:02.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.038 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/serio/i8042.o 2024-03-19T01:23:03.097 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/processor_driver.o 2024-03-19T01:23:03.323 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/posix_acl.o 2024-03-19T01:23:03.337 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_ggtt.o 2024-03-19T01:23:03.680 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/usb/mon/built-in.a 2024-03-19T01:23:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:03 smithi086 ceph-mon[66829]: pgmap v386: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 1.55k op/s 2024-03-19T01:23:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:03.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:03 smithi069 ceph-mon[31727]: pgmap v386: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 1.55k op/s 2024-03-19T01:23:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:03 smithi163 ceph-mon[67428]: pgmap v386: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 1.55k op/s 2024-03-19T01:23:03.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:03.877 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_blend.o 2024-03-19T01:23:04.014 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/keyboard/atkbd.o 2024-03-19T01:23:04.185 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/driver.o 2024-03-19T01:23:04.306 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000e/nvm.o 2024-03-19T01:23:04.437 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/processor_thermal.o 2024-03-19T01:23:04.568 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/serio/serport.o 2024-03-19T01:23:04.870 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/coredump.o 2024-03-19T01:23:04.881 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/microsoft/built-in.a 2024-03-19T01:23:04.976 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_ggtt_fencing.o 2024-03-19T01:23:05.217 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/rtc/lib.o 2024-03-19T01:23:05.318 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/input/keyboard/built-in.a 2024-03-19T01:23:05.462 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/serio/libps2.o 2024-03-19T01:23:05.472 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/config.o 2024-03-19T01:23:05.505 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/mouse/psmouse-base.o 2024-03-19T01:23:05.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:05 smithi086 ceph-mon[66829]: pgmap v387: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 962 KiB/s wr, 563 op/s 2024-03-19T01:23:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:05 smithi086 ceph-mon[66829]: osdmap e107: 12 total, 12 up, 12 in 2024-03-19T01:23:05.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:05 smithi069 ceph-mon[31727]: pgmap v387: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 962 KiB/s wr, 563 op/s 2024-03-19T01:23:05.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:05 smithi069 ceph-mon[31727]: osdmap e107: 12 total, 12 up, 12 in 2024-03-19T01:23:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:05 smithi163 ceph-mon[67428]: pgmap v387: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 962 KiB/s wr, 563 op/s 2024-03-19T01:23:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:05 smithi163 ceph-mon[67428]: osdmap e107: 12 total, 12 up, 12 in 2024-03-19T01:23:05.848 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000e/phy.o 2024-03-19T01:23:06.051 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/processor_idle.o 2024-03-19T01:23:06.118 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/rtc/class.o 2024-03-19T01:23:06.385 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/input/serio/built-in.a 2024-03-19T01:23:06.422 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt.o 2024-03-19T01:23:06.448 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/drop_caches.o 2024-03-19T01:23:06.549 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/i2c/algos/i2c-algo-bit.o 2024-03-19T01:23:06.893 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/file.o 2024-03-19T01:23:06.906 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/mouse/synaptics.o 2024-03-19T01:23:07.324 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/sysctls.o 2024-03-19T01:23:07.483 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/rtc/interface.o 2024-03-19T01:23:07.628 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/processor_throttling.o 2024-03-19T01:23:07.692 INFO:tasks.workunit.client.0.smithi069.stdout: CC fs/fhandle.o 2024-03-19T01:23:07.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:07 smithi086 ceph-mon[66829]: pgmap v389: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.1 MiB/s wr, 266 op/s 2024-03-19T01:23:07.767 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/buffer.o 2024-03-19T01:23:07.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:07 smithi069 ceph-mon[31727]: pgmap v389: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.1 MiB/s wr, 266 op/s 2024-03-19T01:23:07.810 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/i2c/algos/built-in.a 2024-03-19T01:23:07.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:07 smithi163 ceph-mon[67428]: pgmap v389: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.1 MiB/s wr, 266 op/s 2024-03-19T01:23:07.898 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/i2c/busses/i2c-i801.o 2024-03-19T01:23:07.967 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000e/param.o 2024-03-19T01:23:08.019 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.o 2024-03-19T01:23:08.543 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/mouse/focaltech.o 2024-03-19T01:23:08.686 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/processor_perflib.o 2024-03-19T01:23:08.706 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/sysfs.o 2024-03-19T01:23:08.985 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/rtc/nvmem.o 2024-03-19T01:23:08.999 INFO:tasks.workunit.client.0.smithi069.stdout: AR fs/built-in.a 2024-03-19T01:23:09.401 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/mouse/alps.o 2024-03-19T01:23:09.415 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_clock_utils.o 2024-03-19T01:23:09.420 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000e/ethtool.o 2024-03-19T01:23:09.433 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/i2c/muxes/built-in.a 2024-03-19T01:23:09.477 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_debugfs.o 2024-03-19T01:23:09.514 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/i2c/busses/built-in.a 2024-03-19T01:23:09.543 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/container.o 2024-03-19T01:23:09.588 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/i2c/i2c-boardinfo.o 2024-03-19T01:23:09.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:09 smithi086 ceph-mon[66829]: pgmap v390: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-03-19T01:23:09.760 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/rtc/dev.o 2024-03-19T01:23:09.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:09 smithi069 ceph-mon[31727]: pgmap v390: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-03-19T01:23:09.808 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/endpoint.o 2024-03-19T01:23:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:09 smithi163 ceph-mon[67428]: pgmap v390: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-03-19T01:23:10.135 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/thermal.o 2024-03-19T01:23:10.564 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/pci-quirks.o 2024-03-19T01:23:10.590 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/devio.o 2024-03-19T01:23:10.605 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_engines_debugfs.o 2024-03-19T01:23:10.983 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/i2c/i2c-core-base.o 2024-03-19T01:23:11.078 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/rtc/proc.o 2024-03-19T01:23:11.085 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/acpi_memhotplug.o 2024-03-19T01:23:11.104 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/mouse/byd.o 2024-03-19T01:23:11.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:23:11.266 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:23:11.685 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_irq.o 2024-03-19T01:23:11.687 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000e/netdev.o 2024-03-19T01:23:11.716 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/ioapic.o 2024-03-19T01:23:11.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:11 smithi086 ceph-mon[66829]: pgmap v391: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-03-19T01:23:11.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:11 smithi069 ceph-mon[31727]: pgmap v391: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-03-19T01:23:11.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:11 smithi163 ceph-mon[67428]: pgmap v391: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-03-19T01:23:11.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.842+0000 7fb03ca00640 1 -- 172.21.15.69:0/615209301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0300a5720 msgr2=0x7fb0300b7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:11.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.842+0000 7fb03ca00640 1 --2- 172.21.15.69:0/615209301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0300a5720 0x7fb0300b7550 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb0380674f0 tx=0x7fb0380a0580 comp rx=0 tx=0).stop 2024-03-19T01:23:11.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.842+0000 7fb03ca00640 1 -- 172.21.15.69:0/615209301 shutdown_connections 2024-03-19T01:23:11.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.842+0000 7fb03ca00640 1 --2- 172.21.15.69:0/615209301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0300b7a90 0x7fb0300b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.842+0000 7fb03ca00640 1 --2- 172.21.15.69:0/615209301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0300a5720 0x7fb0300b7550 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.842+0000 7fb03ca00640 1 --2- 172.21.15.69:0/615209301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0300a4e00 0x7fb0300a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.842+0000 7fb03ca00640 1 -- 172.21.15.69:0/615209301 >> 172.21.15.69:0/615209301 conn(0x7fb03001a660 msgr2=0x7fb03001aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:11.845 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/mouse/logips2pp.o 2024-03-19T01:23:11.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.850+0000 7fb03ca00640 1 -- 172.21.15.69:0/615209301 shutdown_connections 2024-03-19T01:23:11.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.850+0000 7fb03ca00640 1 -- 172.21.15.69:0/615209301 wait complete. 2024-03-19T01:23:11.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.855+0000 7fb03ca00640 1 Processor -- start 2024-03-19T01:23:11.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.858+0000 7fb03ca00640 1 -- start start 2024-03-19T01:23:11.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.858+0000 7fb03ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0300a4e00 0x7fb0300b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.858+0000 7fb03ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0300a5720 0x7fb0300b7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.858+0000 7fb03ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0300b7a90 0x7fb0300b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.858+0000 7fb03ca00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300af670 con 0x7fb0300b7a90 2024-03-19T01:23:11.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.858+0000 7fb03ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300af4f0 con 0x7fb0300a5720 2024-03-19T01:23:11.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.858+0000 7fb03ca00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300af7f0 con 0x7fb0300a4e00 2024-03-19T01:23:11.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.859+0000 7fb035400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0300a5720 0x7fb0300b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.859+0000 7fb035400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0300a5720 0x7fb0300b7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:49732/0 (socket says 172.21.15.69:49732) 2024-03-19T01:23:11.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.859+0000 7fb035400640 1 -- 172.21.15.69:0/1788145663 learned_addr learned my addr 172.21.15.69:0/1788145663 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:11.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.859+0000 7fb036800640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0300b7a90 0x7fb0300b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.859+0000 7fb035400640 1 -- 172.21.15.69:0/1788145663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0300a4e00 msgr2=0x7fb0300b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:11.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.859+0000 7fb035400640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0300a4e00 0x7fb0300b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.859+0000 7fb035400640 1 -- 172.21.15.69:0/1788145663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0300b7a90 msgr2=0x7fb0300b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:11.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.859+0000 7fb035400640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0300b7a90 0x7fb0300b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.859+0000 7fb035400640 1 -- 172.21.15.69:0/1788145663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0300b3b50 con 0x7fb0300a5720 2024-03-19T01:23:11.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.859+0000 7fb036800640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0300b7a90 0x7fb0300b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:11.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.860+0000 7fb035400640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0300a5720 0x7fb0300b7510 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb038000b60 tx=0x7fb038069930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:11.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.862+0000 7fb02ea00640 1 -- 172.21.15.69:0/1788145663 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380b5020 con 0x7fb0300a5720 2024-03-19T01:23:11.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.862+0000 7fb02ea00640 1 -- 172.21.15.69:0/1788145663 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0380a8b20 con 0x7fb0300a5720 2024-03-19T01:23:11.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.862+0000 7fb03ca00640 1 -- 172.21.15.69:0/1788145663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0300b3de0 con 0x7fb0300a5720 2024-03-19T01:23:11.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.862+0000 7fb02ea00640 1 -- 172.21.15.69:0/1788145663 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380b18b0 con 0x7fb0300a5720 2024-03-19T01:23:11.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.862+0000 7fb03ca00640 1 -- 172.21.15.69:0/1788145663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb030147d70 con 0x7fb0300a5720 2024-03-19T01:23:11.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.867+0000 7fb03f776740 1 Processor -- start 2024-03-19T01:23:11.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.870+0000 7fb02ea00640 1 -- 172.21.15.69:0/1788145663 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fb0380b8020 con 0x7fb0300a5720 2024-03-19T01:23:11.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.874+0000 7fb03f776740 1 -- start start 2024-03-19T01:23:11.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.874+0000 7fb03f776740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a773f30 0x55af4a774310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.874+0000 7fb03f776740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a77cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.874+0000 7fb03f776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 0x55af4a77f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.874+0000 7fb03f776740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55af4a6d82c0 con 0x55af4a774850 2024-03-19T01:23:11.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.874+0000 7fb03f776740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af4a680b60 con 0x55af4a77d430 2024-03-19T01:23:11.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.874+0000 7fb03f776740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55af4a6d4360 con 0x55af4a773f30 2024-03-19T01:23:11.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.875+0000 7fb036800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a773f30 0x55af4a774310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.875+0000 7fb036800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a773f30 0x55af4a774310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:39230/0 (socket says 172.21.15.69:39230) 2024-03-19T01:23:11.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.875+0000 7fb036800640 1 -- 172.21.15.69:0/1251240738 learned_addr learned my addr 172.21.15.69:0/1251240738 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:11.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.875+0000 7fb035400640 1 --2- 172.21.15.69:0/1251240738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 0x55af4a77f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.875+0000 7fb036800640 1 -- 172.21.15.69:0/1251240738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 msgr2=0x55af4a77f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:11.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.875+0000 7fb036800640 1 --2- 172.21.15.69:0/1251240738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 0x55af4a77f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.875+0000 7fb036800640 1 -- 172.21.15.69:0/1251240738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 msgr2=0x55af4a77cef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:11.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.875+0000 7fb036800640 1 --2- 172.21.15.69:0/1251240738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a77cef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.875+0000 7fb036800640 1 -- 172.21.15.69:0/1251240738 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af4a4b67d0 con 0x55af4a773f30 2024-03-19T01:23:11.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.875+0000 7fb036800640 1 --2- 172.21.15.69:0/1251240738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a773f30 0x55af4a774310 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb02800a990 tx=0x7fb02800ae50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6fa951cdec05e63 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:11.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.876+0000 7fb02ea00640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb01c0a2af0 0x7fb01c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.876+0000 7fb02ea00640 1 -- 172.21.15.69:0/1788145663 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9656+0+0 (secure 0 0 0) 0x7fb038165500 con 0x7fb0300a5720 2024-03-19T01:23:11.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.876+0000 7fb035e00640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb01c0a2af0 0x7fb01c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.877+0000 7fb023000640 1 -- 172.21.15.69:0/1251240738 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028015070 con 0x55af4a773f30 2024-03-19T01:23:11.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.877+0000 7fb023000640 1 -- 172.21.15.69:0/1251240738 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb02800cbe0 con 0x55af4a773f30 2024-03-19T01:23:11.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.877+0000 7fb023000640 1 -- 172.21.15.69:0/1251240738 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0280186d0 con 0x55af4a773f30 2024-03-19T01:23:11.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.878+0000 7fb03f776740 1 -- 172.21.15.69:0/1251240738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a773f30 msgr2=0x55af4a774310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:11.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.878+0000 7fb03f776740 1 --2- 172.21.15.69:0/1251240738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a773f30 0x55af4a774310 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb02800a990 tx=0x7fb02800ae50 comp rx=0 tx=0).stop 2024-03-19T01:23:11.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.878+0000 7fb035e00640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb01c0a2af0 0x7fb01c0a4fb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55af4a773a00 tx=0x7fb024005c70 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:11.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.878+0000 7fb03f776740 1 -- 172.21.15.69:0/1251240738 shutdown_connections 2024-03-19T01:23:11.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.878+0000 7fb03f776740 1 --2- 172.21.15.69:0/1251240738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 0x55af4a77f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.878+0000 7fb03f776740 1 --2- 172.21.15.69:0/1251240738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a77cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.878+0000 7fb03f776740 1 --2- 172.21.15.69:0/1251240738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a773f30 0x55af4a774310 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.878+0000 7fb03f776740 1 -- 172.21.15.69:0/1251240738 >> 172.21.15.69:0/1251240738 conn(0x55af4a76e660 msgr2=0x55af4a76ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:11.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.878+0000 7fb03f776740 1 -- 172.21.15.69:0/1251240738 shutdown_connections 2024-03-19T01:23:11.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.878+0000 7fb03f776740 1 -- 172.21.15.69:0/1251240738 wait complete. 2024-03-19T01:23:11.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.879+0000 7fb03f776740 1 -- start start 2024-03-19T01:23:11.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.879+0000 7fb03f776740 1 Processor -- start 2024-03-19T01:23:11.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.882+0000 7fb03f776740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a773f30 0x55af4a77a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.882+0000 7fb03f776740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a774850 0x55af4a77a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.882+0000 7fb03f776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 0x55af4a77af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.882+0000 7fb03f776740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55af4a780220 con 0x55af4a773f30 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.882+0000 7fb03f776740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af4a693fc0 con 0x55af4a77d430 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.882+0000 7fb03f776740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55af4a7803a0 con 0x55af4a774850 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.883+0000 7fb036800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 0x55af4a77af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.883+0000 7fb036800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 0x55af4a77af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:49758/0 (socket says 172.21.15.69:49758) 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.883+0000 7fb036800640 1 -- 172.21.15.69:0/2134923996 learned_addr learned my addr 172.21.15.69:0/2134923996 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.883+0000 7fb03f776740 10 client.0 _ll_register_callbacks cb 0x55af4a693990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.883+0000 7fb036800640 1 -- 172.21.15.69:0/2134923996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a774850 msgr2=0x55af4a77a9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.883+0000 7fb036800640 1 --2- 172.21.15.69:0/2134923996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a774850 0x55af4a77a9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.883+0000 7fb036800640 1 -- 172.21.15.69:0/2134923996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a773f30 msgr2=0x55af4a77a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.883+0000 7fb036800640 1 --2- 172.21.15.69:0/2134923996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a773f30 0x55af4a77a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.883+0000 7fb036800640 1 -- 172.21.15.69:0/2134923996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af4a3ec3f0 con 0x55af4a77d430 2024-03-19T01:23:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.883+0000 7fb036800640 1 --2- 172.21.15.69:0/2134923996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 0x55af4a77af00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb02800a140 tx=0x7fb02800cfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca1ca5dd1546142 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:11.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.884+0000 7fb017400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:23:11.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.884+0000 7fb017400640 1 -- 172.21.15.69:0/2134923996 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028020070 con 0x55af4a77d430 2024-03-19T01:23:11.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.884+0000 7fb017400640 1 -- 172.21.15.69:0/2134923996 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9656+0+0 (secure 0 0 0) 0x7fb02801a660 con 0x55af4a77d430 2024-03-19T01:23:11.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.884+0000 7fb03f776740 1 -- 172.21.15.69:0/2134923996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb02800cbe0 con 0x55af4a77d430 2024-03-19T01:23:11.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.884+0000 7fb017400640 1 -- 172.21.15.69:0/2134923996 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb02802f050 con 0x55af4a77d430 2024-03-19T01:23:11.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.884+0000 7fb03f776740 10 client.27956 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:11.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.884+0000 7fb03f776740 1 -- 172.21.15.69:0/2134923996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55af4a3ac0f0 con 0x55af4a77d430 2024-03-19T01:23:11.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.885+0000 7fb017400640 1 -- 172.21.15.69:0/2134923996 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb02800cbe0 con 0x55af4a77d430 2024-03-19T01:23:11.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.885+0000 7fb017400640 1 -- 172.21.15.69:0/2134923996 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fb028024970 con 0x55af4a77d430 2024-03-19T01:23:11.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.885+0000 7fb03f776740 10 client.27956 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:11.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.885+0000 7fb03f776740 10 client.27956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:23:11.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.885+0000 7fb03f776740 20 client.27956 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:11.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.885+0000 7fb03f776740 1 --2- 172.21.15.69:0/2134923996 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55af4a78d5a0 0x55af4a78f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.885+0000 7fb03f776740 4 client.27956 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:23:11.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.885+0000 7fb03f776740 1 -- 172.21.15.69:0/2134923996 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af4a679610 con 0x55af4a78d5a0 2024-03-19T01:23:11.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.886+0000 7fb035e00640 1 --2- 172.21.15.69:0/2134923996 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55af4a78d5a0 0x55af4a78f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.887+0000 7fb035e00640 1 --2- 172.21.15.69:0/2134923996 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55af4a78d5a0 0x55af4a78f990 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe180ae6408af0e7 server_cookie=a42ac9bb599bb6a3 in_seq=0 out_seq=0 2024-03-19T01:23:11.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.892+0000 7fb017400640 10 client.27956 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:11.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.892+0000 7fb017400640 1 -- 172.21.15.69:0/2134923996 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55af4a679610 con 0x55af4a78d5a0 2024-03-19T01:23:11.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.892+0000 7fb017400640 10 client.27956 handle_command_reply: tid=0 2024-03-19T01:23:11.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.892+0000 7fb03f776740 1 client.27956 shutdown 2024-03-19T01:23:11.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.897+0000 7fb03f776740 1 -- 172.21.15.69:0/2134923996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 msgr2=0x55af4a77af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:11.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.897+0000 7fb03f776740 1 --2- 172.21.15.69:0/2134923996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 0x55af4a77af00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb02800a140 tx=0x7fb02800cfc0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.900+0000 7fb03f776740 1 -- 172.21.15.69:0/2134923996 shutdown_connections 2024-03-19T01:23:11.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.900+0000 7fb03f776740 1 --2- 172.21.15.69:0/2134923996 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55af4a78d5a0 0x55af4a78f990 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.900+0000 7fb03f776740 1 --2- 172.21.15.69:0/2134923996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a77d430 0x55af4a77af00 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.900+0000 7fb03f776740 1 --2- 172.21.15.69:0/2134923996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a774850 0x55af4a77a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.900+0000 7fb03f776740 1 --2- 172.21.15.69:0/2134923996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a773f30 0x55af4a77a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.900+0000 7fb03f776740 1 -- 172.21.15.69:0/2134923996 >> 172.21.15.69:0/2134923996 conn(0x55af4a76e660 msgr2=0x55af4a7703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:11.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.900+0000 7fb017400640 0 client.27956 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:11.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.900+0000 7fb03f776740 1 -- 172.21.15.69:0/2134923996 shutdown_connections 2024-03-19T01:23:11.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.900+0000 7fb03f776740 1 -- 172.21.15.69:0/2134923996 wait complete. 2024-03-19T01:23:11.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.900+0000 7fb03f776740 20 client.27956 trim_cache size 0 max 16384 2024-03-19T01:23:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.971+0000 7fb03f776740 1 Processor -- start 2024-03-19T01:23:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.971+0000 7fb03f776740 1 -- start start 2024-03-19T01:23:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.972+0000 7fb03f776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a773f30 0x55af4a77cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.972+0000 7fb035e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a773f30 0x55af4a77cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.972+0000 7fb035e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a773f30 0x55af4a77cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:49764/0 (socket says 172.21.15.69:49764) 2024-03-19T01:23:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.972+0000 7fb035e00640 1 -- 172.21.15.69:0/1231430269 learned_addr learned my addr 172.21.15.69:0/1231430269 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.973+0000 7fb03f776740 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a7793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.974+0000 7fb035400640 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a7793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.974+0000 7fb03f776740 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 0x55af4a789020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.974+0000 7fb036800640 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 0x55af4a789020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.975+0000 7fb03f776740 1 -- 172.21.15.69:0/1231430269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb030152130 con 0x55af4a774850 2024-03-19T01:23:11.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.975+0000 7fb03f776740 1 -- 172.21.15.69:0/1231430269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb030153c70 con 0x55af4a773f30 2024-03-19T01:23:11.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.975+0000 7fb03f776740 1 -- 172.21.15.69:0/1231430269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb030152800 con 0x55af4a77d430 2024-03-19T01:23:11.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.976+0000 7fb035400640 1 -- 172.21.15.69:0/1231430269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 msgr2=0x55af4a789020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:11.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.976+0000 7fb035400640 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 0x55af4a789020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.976+0000 7fb035400640 1 -- 172.21.15.69:0/1231430269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a773f30 msgr2=0x55af4a77cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:11.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.976+0000 7fb035400640 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a773f30 0x55af4a77cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.976+0000 7fb035400640 1 -- 172.21.15.69:0/1231430269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af4a776060 con 0x55af4a774850 2024-03-19T01:23:11.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.976+0000 7fb036800640 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 0x55af4a789020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:11.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.976+0000 7fb035e00640 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a773f30 0x55af4a77cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:11.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.977+0000 7fb035400640 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a7793c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0381627e0 tx=0x7fb038168c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e6aac3e8106445 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:11.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.977+0000 7fb022600640 1 -- 172.21.15.69:0/1231430269 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038131050 con 0x55af4a774850 2024-03-19T01:23:11.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.977+0000 7fb022600640 1 -- 172.21.15.69:0/1231430269 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb03804fdd0 con 0x55af4a774850 2024-03-19T01:23:11.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.977+0000 7fb022600640 1 -- 172.21.15.69:0/1231430269 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038069b20 con 0x55af4a774850 2024-03-19T01:23:11.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.979+0000 7fb03f776740 1 -- 172.21.15.69:0/1231430269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 msgr2=0x55af4a7793c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:11.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.979+0000 7fb03f776740 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a7793c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0381627e0 tx=0x7fb038168c10 comp rx=0 tx=0).stop 2024-03-19T01:23:11.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.979+0000 7fb03f776740 1 -- 172.21.15.69:0/1231430269 shutdown_connections 2024-03-19T01:23:11.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.979+0000 7fb03f776740 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 0x55af4a789020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.979+0000 7fb03f776740 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a7793c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.979+0000 7fb03f776740 1 --2- 172.21.15.69:0/1231430269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a773f30 0x55af4a77cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.979+0000 7fb03f776740 1 -- 172.21.15.69:0/1231430269 >> 172.21.15.69:0/1231430269 conn(0x55af4a76e660 msgr2=0x55af4a76d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:11.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.980+0000 7fb03f776740 1 -- 172.21.15.69:0/1231430269 shutdown_connections 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.984+0000 7fb03f776740 1 -- 172.21.15.69:0/1231430269 wait complete. 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.984+0000 7fb03f776740 1 -- start start 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.985+0000 7fb03f776740 1 Processor -- start 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.985+0000 7fb03f776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a773f30 0x55af4a780f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.985+0000 7fb03f776740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a781440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.985+0000 7fb03f776740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 0x55af4a781980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.985+0000 7fb03f776740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0301535a0 con 0x55af4a774850 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.985+0000 7fb03f776740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb030152ed0 con 0x55af4a773f30 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.985+0000 7fb03f776740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb030150130 con 0x55af4a77d430 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.985+0000 7fb03f776740 10 client.0 _ll_register_callbacks cb 0x7fb0280119e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.986+0000 7fb035400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a781440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.987+0000 7fb036800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 0x55af4a781980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.987+0000 7fb036800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 0x55af4a781980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:39240/0 (socket says 172.21.15.69:39240) 2024-03-19T01:23:11.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.987+0000 7fb036800640 1 -- 172.21.15.69:0/1315136631 learned_addr learned my addr 172.21.15.69:0/1315136631 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:11.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.987+0000 7fb036800640 1 -- 172.21.15.69:0/1315136631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a773f30 msgr2=0x55af4a780f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:11.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.987+0000 7fb036800640 1 --2- 172.21.15.69:0/1315136631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a773f30 0x55af4a780f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.987+0000 7fb036800640 1 -- 172.21.15.69:0/1315136631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 msgr2=0x55af4a781440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:11.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.987+0000 7fb036800640 1 --2- 172.21.15.69:0/1315136631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a781440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.987+0000 7fb036800640 1 -- 172.21.15.69:0/1315136631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af4a7f2770 con 0x55af4a77d430 2024-03-19T01:23:11.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.989+0000 7fb036800640 1 --2- 172.21.15.69:0/1315136631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 0x55af4a781980 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb028024dc0 tx=0x7fb02800a920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c49594295e22992 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:11.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.989+0000 7fb017400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:23:11.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.989+0000 7fb017400640 1 -- 172.21.15.69:0/1315136631 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028038060 con 0x55af4a77d430 2024-03-19T01:23:11.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.989+0000 7fb017400640 1 -- 172.21.15.69:0/1315136631 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9656+0+0 (secure 0 0 0) 0x7fb028024400 con 0x55af4a77d430 2024-03-19T01:23:11.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.989+0000 7fb03f776740 1 -- 172.21.15.69:0/1315136631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb03804fdd0 con 0x55af4a77d430 2024-03-19T01:23:11.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.990+0000 7fb017400640 1 -- 172.21.15.69:0/1315136631 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb028020040 con 0x55af4a77d430 2024-03-19T01:23:11.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.990+0000 7fb03f776740 10 client.27904 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:11.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.990+0000 7fb03f776740 1 -- 172.21.15.69:0/1315136631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55af4a7f3930 con 0x55af4a77d430 2024-03-19T01:23:11.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.990+0000 7fb017400640 1 -- 172.21.15.69:0/1315136631 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb03804fdd0 con 0x55af4a77d430 2024-03-19T01:23:11.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.990+0000 7fb017400640 1 -- 172.21.15.69:0/1315136631 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fb02801ab00 con 0x55af4a77d430 2024-03-19T01:23:11.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.990+0000 7fb03f776740 10 client.27904 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:11.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.990+0000 7fb03f776740 10 client.27904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:23:11.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.990+0000 7fb03f776740 20 client.27904 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:11.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.990+0000 7fb03f776740 1 --2- 172.21.15.69:0/1315136631 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55af4a7f4050 0x55af4a7f6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:11.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.990+0000 7fb03f776740 4 client.27904 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:23:11.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.990+0000 7fb03f776740 1 -- 172.21.15.69:0/1315136631 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af4a6e5800 con 0x55af4a7f4050 2024-03-19T01:23:11.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.991+0000 7fb035e00640 1 --2- 172.21.15.69:0/1315136631 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55af4a7f4050 0x55af4a7f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:11.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.991+0000 7fb017400640 10 client.27904 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:11.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.991+0000 7fb035e00640 1 --2- 172.21.15.69:0/1315136631 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55af4a7f4050 0x55af4a7f6470 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d86a77b68a0c6cbb server_cookie=fb66010bde09457a in_seq=0 out_seq=0 2024-03-19T01:23:11.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.993+0000 7fb017400640 1 -- 172.21.15.69:0/1315136631 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55af4a6e5800 con 0x55af4a7f4050 2024-03-19T01:23:11.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.993+0000 7fb017400640 10 client.27904 handle_command_reply: tid=0 2024-03-19T01:23:11.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.993+0000 7fb03f776740 1 client.27904 shutdown 2024-03-19T01:23:11.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.995+0000 7fb03f776740 1 -- 172.21.15.69:0/1315136631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 msgr2=0x55af4a781980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:11.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.995+0000 7fb03f776740 1 --2- 172.21.15.69:0/1315136631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 0x55af4a781980 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb028024dc0 tx=0x7fb02800a920 comp rx=0 tx=0).stop 2024-03-19T01:23:11.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.996+0000 7fb03f776740 1 -- 172.21.15.69:0/1315136631 shutdown_connections 2024-03-19T01:23:11.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.996+0000 7fb03f776740 1 --2- 172.21.15.69:0/1315136631 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55af4a7f4050 0x55af4a7f6470 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.996+0000 7fb03f776740 1 --2- 172.21.15.69:0/1315136631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55af4a77d430 0x55af4a781980 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.996+0000 7fb03f776740 1 --2- 172.21.15.69:0/1315136631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af4a774850 0x55af4a781440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.996+0000 7fb03f776740 1 --2- 172.21.15.69:0/1315136631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af4a773f30 0x55af4a780f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:11.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.996+0000 7fb017400640 0 client.27904 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:11.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.996+0000 7fb03f776740 1 -- 172.21.15.69:0/1315136631 >> 172.21.15.69:0/1315136631 conn(0x55af4a76e660 msgr2=0x55af4a77a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:11.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.996+0000 7fb03f776740 1 -- 172.21.15.69:0/1315136631 shutdown_connections 2024-03-19T01:23:11.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.996+0000 7fb03f776740 1 -- 172.21.15.69:0/1315136631 wait complete. 2024-03-19T01:23:11.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:11.997+0000 7fb03f776740 20 client.27904 trim_cache size 0 max 16384 2024-03-19T01:23:12.002 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:23:12.002 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T01:23:12.002 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:23:12.002 INFO:teuthology.orchestra.run.smithi069.stdout: "fa17395d-c264-4912-a074-3ab5cd739f56": { 2024-03-19T01:23:12.002 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:23:12.002 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "fa17395d-c264-4912-a074-3ab5cd739f56", 2024-03-19T01:23:12.002 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:23:12.002 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:23:12.002 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:23:12.002 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:23:12.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.004+0000 7fb03ca00640 1 -- 172.21.15.69:0/1788145663 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb01c0a2af0 msgr2=0x7fb01c0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:12.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.004+0000 7fb03ca00640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb01c0a2af0 0x7fb01c0a4fb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55af4a773a00 tx=0x7fb024005c70 comp rx=0 tx=0).stop 2024-03-19T01:23:12.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.005+0000 7fb03ca00640 1 -- 172.21.15.69:0/1788145663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0300a5720 msgr2=0x7fb0300b7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:12.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.005+0000 7fb03ca00640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0300a5720 0x7fb0300b7510 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb038000b60 tx=0x7fb038069930 comp rx=0 tx=0).stop 2024-03-19T01:23:12.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.005+0000 7fb03ca00640 1 -- 172.21.15.69:0/1788145663 shutdown_connections 2024-03-19T01:23:12.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.005+0000 7fb03ca00640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0300b7a90 0x7fb0300b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:12.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.005+0000 7fb03ca00640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb01c0a2af0 0x7fb01c0a4fb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:12.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.005+0000 7fb03ca00640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0300a5720 0x7fb0300b7510 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:12.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.005+0000 7fb03ca00640 1 --2- 172.21.15.69:0/1788145663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0300a4e00 0x7fb0300b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:12.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.005+0000 7fb03ca00640 1 -- 172.21.15.69:0/1788145663 >> 172.21.15.69:0/1788145663 conn(0x7fb03001a660 msgr2=0x7fb0300b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:12.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.007+0000 7fb03ca00640 1 -- 172.21.15.69:0/1788145663 shutdown_connections 2024-03-19T01:23:12.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:12.007+0000 7fb03ca00640 1 -- 172.21.15.69:0/1788145663 wait complete. 2024-03-19T01:23:12.022 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/rtc/sysfs.o 2024-03-19T01:23:12.069 INFO:tasks.cephfs.filesystem:scrub status for tag:fa17395d-c264-4912-a074-3ab5cd739f56 - {'path': '/', 'tag': 'fa17395d-c264-4912-a074-3ab5cd739f56', 'options': 'recursive,force'} 2024-03-19T01:23:12.242 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/ehci-hcd.o 2024-03-19T01:23:12.655 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/mouse/lifebook.o 2024-03-19T01:23:12.823 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/battery.o 2024-03-19T01:23:12.957 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/rtc/rtc-mc146818-lib.o 2024-03-19T01:23:12.975 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_mcr.o 2024-03-19T01:23:13.384 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/i2c/i2c-core-smbus.o 2024-03-19T01:23:13.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:13 smithi086 ceph-mon[66829]: pgmap v392: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 272 KiB/s wr, 120 op/s 2024-03-19T01:23:13.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:13 smithi069 ceph-mon[31727]: pgmap v392: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 272 KiB/s wr, 120 op/s 2024-03-19T01:23:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:13 smithi163 ceph-mon[67428]: pgmap v392: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 272 KiB/s wr, 120 op/s 2024-03-19T01:23:14.022 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/notify.o 2024-03-19T01:23:14.119 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/mouse/trackpoint.o 2024-03-19T01:23:14.135 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/rtc/rtc-cmos.o 2024-03-19T01:23:14.315 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/bgrt.o 2024-03-19T01:23:14.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:14 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:23:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:14 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:23:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:14 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:23:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:14 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:23:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:14 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:23:14.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:14 smithi086 ceph-mon[66829]: osdmap e108: 12 total, 12 up, 12 in 2024-03-19T01:23:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:14 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:23:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:14 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:23:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:14 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:23:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:14 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:23:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:14 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:23:14.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:14 smithi069 ceph-mon[31727]: osdmap e108: 12 total, 12 up, 12 in 2024-03-19T01:23:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:14 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:23:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:14 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:23:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:14 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:23:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:14 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:23:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:14 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:23:14.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:14 smithi163 ceph-mon[67428]: osdmap e108: 12 total, 12 up, 12 in 2024-03-19T01:23:15.040 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_pm.o 2024-03-19T01:23:15.151 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/cppc_acpi.o 2024-03-19T01:23:15.172 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/generic.o 2024-03-19T01:23:15.225 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/mouse/cypress_ps2.o 2024-03-19T01:23:15.380 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/i2c/i2c-core-acpi.o 2024-03-19T01:23:15.429 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/rtc/built-in.a 2024-03-19T01:23:15.706 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/ehci-pci.o 2024-03-19T01:23:16.227 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/mouse/psmouse-smbus.o 2024-03-19T01:23:16.255 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/quirks.o 2024-03-19T01:23:16.289 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/intel/e1000e/ptp.o 2024-03-19T01:23:16.402 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/acpi/spcr.o 2024-03-19T01:23:16.442 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.o 2024-03-19T01:23:16.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:16 smithi086 ceph-mon[66829]: pgmap v394: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 403 KiB/s wr, 113 op/s 2024-03-19T01:23:16.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:16 smithi069 ceph-mon[31727]: pgmap v394: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 403 KiB/s wr, 113 op/s 2024-03-19T01:23:16.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:16 smithi163 ceph-mon[67428]: pgmap v394: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 403 KiB/s wr, 113 op/s 2024-03-19T01:23:16.759 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/litex/built-in.a 2024-03-19T01:23:16.768 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/ohci-hcd.o 2024-03-19T01:23:16.771 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/i2c/i2c-smbus.o 2024-03-19T01:23:16.881 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/input/joystick/built-in.a 2024-03-19T01:23:16.986 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/netconsole.o 2024-03-19T01:23:17.186 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/devices.o 2024-03-19T01:23:17.289 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/input/mouse/built-in.a 2024-03-19T01:23:17.391 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/acpi/built-in.a 2024-03-19T01:23:17.462 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/input/tablet/built-in.a 2024-03-19T01:23:17.615 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/input/touchscreen/built-in.a 2024-03-19T01:23:17.648 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_pm_irq.o 2024-03-19T01:23:17.691 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/input/misc/built-in.a 2024-03-19T01:23:17.734 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/input.o 2024-03-19T01:23:17.827 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/intel/e1000e/built-in.a 2024-03-19T01:23:17.924 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/intel/built-in.a 2024-03-19T01:23:17.985 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/i2c/built-in.a 2024-03-19T01:23:18.001 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/input-compat.o 2024-03-19T01:23:18.056 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_bridge.o 2024-03-19T01:23:18.061 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/marvell/octeon_ep/built-in.a 2024-03-19T01:23:18.131 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/marvell/octeontx2/built-in.a 2024-03-19T01:23:18.200 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_cache.o 2024-03-19T01:23:18.209 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/marvell/prestera/built-in.a 2024-03-19T01:23:18.229 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/phy.o 2024-03-19T01:23:18.266 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/marvell/sky2.o 2024-03-19T01:23:18.363 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:18 smithi163 ceph-mon[67428]: pgmap v395: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 510 KiB/s wr, 94 op/s 2024-03-19T01:23:18.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:18 smithi086 ceph-mon[66829]: pgmap v395: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 510 KiB/s wr, 94 op/s 2024-03-19T01:23:18.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:18 smithi069 ceph-mon[31727]: pgmap v395: 129 pgs: 129 active+clean; 2.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 510 KiB/s wr, 94 op/s 2024-03-19T01:23:18.882 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_requests.o 2024-03-19T01:23:18.992 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/port.o 2024-03-19T01:23:19.105 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/mellanox/built-in.a 2024-03-19T01:23:19.136 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_client.o 2024-03-19T01:23:19.160 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/hcd-pci.o 2024-03-19T01:23:19.187 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_sysfs.o 2024-03-19T01:23:19.473 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/ohci-pci.o 2024-03-19T01:23:19.473 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/input-mt.o 2024-03-19T01:23:20.136 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/virtio_net.o 2024-03-19T01:23:20.238 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/core/usb-acpi.o 2024-03-19T01:23:20.238 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/class/usblp.o 2024-03-19T01:23:20.425 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/i3c/built-in.a 2024-03-19T01:23:20.435 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/uhci-hcd.o 2024-03-19T01:23:20.483 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/input-poller.o 2024-03-19T01:23:20.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:20 smithi086 ceph-mon[66829]: pgmap v396: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-03-19T01:23:20.508 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gt_sysfs_pm.o 2024-03-19T01:23:20.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:20 smithi069 ceph-mon[31727]: pgmap v396: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-03-19T01:23:20.539 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gtt.o 2024-03-19T01:23:20.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:20 smithi163 ceph-mon[67428]: pgmap v396: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-03-19T01:23:21.256 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/marvell/built-in.a 2024-03-19T01:23:21.284 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/usb/core/built-in.a 2024-03-19T01:23:21.373 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/micrel/built-in.a 2024-03-19T01:23:21.388 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_client_modeset.o 2024-03-19T01:23:21.401 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/ff-core.o 2024-03-19T01:23:21.428 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/microchip/built-in.a 2024-03-19T01:23:21.475 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/mscc/built-in.a 2024-03-19T01:23:21.542 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/myricom/built-in.a 2024-03-19T01:23:21.634 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/natsemi/built-in.a 2024-03-19T01:23:21.634 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/usb/class/built-in.a 2024-03-19T01:23:21.684 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/neterion/built-in.a 2024-03-19T01:23:21.770 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/netronome/built-in.a 2024-03-19T01:23:21.793 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/touchscreen.o 2024-03-19T01:23:21.820 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/ni/built-in.a 2024-03-19T01:23:21.899 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/nvidia/forcedeth.o 2024-03-19T01:23:22.172 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/oki-semi/built-in.a 2024-03-19T01:23:22.277 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/i2c/built-in.a 2024-03-19T01:23:22.283 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/xhci.o 2024-03-19T01:23:22.345 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/tuners/built-in.a 2024-03-19T01:23:22.418 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_color_mgmt.o 2024-03-19T01:23:22.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:22 smithi086 ceph-mon[66829]: pgmap v397: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-03-19T01:23:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:22 smithi069 ceph-mon[31727]: pgmap v397: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-03-19T01:23:22.557 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/rc/keymaps/built-in.a 2024-03-19T01:23:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:22 smithi163 ceph-mon[67428]: pgmap v397: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-03-19T01:23:22.626 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/rc/built-in.a 2024-03-19T01:23:22.707 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_llc.o 2024-03-19T01:23:22.712 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/storage/scsiglue.o 2024-03-19T01:23:22.769 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/ff-memless.o 2024-03-19T01:23:22.771 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/common/b2c2/built-in.a 2024-03-19T01:23:22.819 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/common/saa7146/built-in.a 2024-03-19T01:23:22.965 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/common/siano/built-in.a 2024-03-19T01:23:23.092 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/common/v4l2-tpg/built-in.a 2024-03-19T01:23:23.163 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/common/videobuf2/built-in.a 2024-03-19T01:23:23.280 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/common/built-in.a 2024-03-19T01:23:23.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:23 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:23:23.515 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/allegro-dvt/built-in.a 2024-03-19T01:23:23.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:23 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:23:23.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:23 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:23:23.600 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/amlogic/meson-ge2d/built-in.a 2024-03-19T01:23:23.645 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/amlogic/built-in.a 2024-03-19T01:23:23.645 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_lrc.o 2024-03-19T01:23:23.762 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/amphion/built-in.a 2024-03-19T01:23:23.837 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/sparse-keymap.o 2024-03-19T01:23:23.907 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/aspeed/built-in.a 2024-03-19T01:23:24.033 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/atmel/built-in.a 2024-03-19T01:23:24.033 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/storage/protocol.o 2024-03-19T01:23:24.112 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/cadence/built-in.a 2024-03-19T01:23:24.210 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/chips-media/built-in.a 2024-03-19T01:23:24.306 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/intel/built-in.a 2024-03-19T01:23:24.329 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/marvell/built-in.a 2024-03-19T01:23:24.380 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/storage/transport.o 2024-03-19T01:23:24.407 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:24 smithi086 ceph-mon[66829]: pgmap v398: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.2 MiB/s wr, 72 op/s 2024-03-19T01:23:24.483 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/ttpci/built-in.a 2024-03-19T01:23:24.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:24 smithi069 ceph-mon[31727]: pgmap v398: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.2 MiB/s wr, 72 op/s 2024-03-19T01:23:24.552 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/b2c2/built-in.a 2024-03-19T01:23:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:24 smithi163 ceph-mon[67428]: pgmap v398: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.2 MiB/s wr, 72 op/s 2024-03-19T01:23:24.567 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/mediatek/jpeg/built-in.a 2024-03-19T01:23:24.639 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/pluto2/built-in.a 2024-03-19T01:23:24.671 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/mediatek/mdp/built-in.a 2024-03-19T01:23:24.702 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/dm1105/built-in.a 2024-03-19T01:23:24.738 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/mediatek/vcodec/built-in.a 2024-03-19T01:23:24.755 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/pt1/built-in.a 2024-03-19T01:23:24.796 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/pt3/built-in.a 2024-03-19T01:23:24.832 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/mediatek/vpu/built-in.a 2024-03-19T01:23:24.864 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/mantis/built-in.a 2024-03-19T01:23:24.878 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/mediatek/mdp3/built-in.a 2024-03-19T01:23:24.925 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/ngene/built-in.a 2024-03-19T01:23:24.925 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/mediatek/built-in.a 2024-03-19T01:23:24.951 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/vivaldi-fmap.o 2024-03-19T01:23:24.986 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/ddbridge/built-in.a 2024-03-19T01:23:25.046 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/microchip/built-in.a 2024-03-19T01:23:25.053 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/nvidia/built-in.a 2024-03-19T01:23:25.065 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/saa7146/built-in.a 2024-03-19T01:23:25.113 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/smipcie/built-in.a 2024-03-19T01:23:25.130 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/nvidia/tegra-vde/built-in.a 2024-03-19T01:23:25.161 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pps/clients/built-in.a 2024-03-19T01:23:25.174 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/netup_unidvb/built-in.a 2024-03-19T01:23:25.192 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/nvidia/built-in.a 2024-03-19T01:23:25.244 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pps/generators/built-in.a 2024-03-19T01:23:25.257 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/intel/ipu3/built-in.a 2024-03-19T01:23:25.287 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/nxp/dw100/built-in.a 2024-03-19T01:23:25.295 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pps/pps.o 2024-03-19T01:23:25.311 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/intel/built-in.a 2024-03-19T01:23:25.338 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/xhci-mem.o 2024-03-19T01:23:25.355 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/nxp/imx-jpeg/built-in.a 2024-03-19T01:23:25.370 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/packetengines/built-in.a 2024-03-19T01:23:25.392 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/pci/built-in.a 2024-03-19T01:23:25.431 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/nxp/imx8-isi/built-in.a 2024-03-19T01:23:25.446 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/qlogic/built-in.a 2024-03-19T01:23:25.477 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/nxp/built-in.a 2024-03-19T01:23:25.526 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/qualcomm/emac/built-in.a 2024-03-19T01:23:25.564 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/qualcomm/built-in.a 2024-03-19T01:23:25.568 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/input-leds.o 2024-03-19T01:23:25.584 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/qcom/camss/built-in.a 2024-03-19T01:23:25.588 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ptp/ptp_clock.o 2024-03-19T01:23:25.625 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_migrate.o 2024-03-19T01:23:25.656 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/storage/usb.o 2024-03-19T01:23:25.694 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/realtek/8139too.o 2024-03-19T01:23:25.703 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/qcom/venus/built-in.a 2024-03-19T01:23:25.797 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/qcom/built-in.a 2024-03-19T01:23:26.013 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/renesas/rcar-vin/built-in.a 2024-03-19T01:23:26.119 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/renesas/rzg2l-cru/built-in.a 2024-03-19T01:23:26.238 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/renesas/vsp1/built-in.a 2024-03-19T01:23:26.239 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pps/kapi.o 2024-03-19T01:23:26.281 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/renesas/built-in.a 2024-03-19T01:23:26.324 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/input/evdev.o 2024-03-19T01:23:26.448 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/rockchip/rga/built-in.a 2024-03-19T01:23:26.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:26 smithi086 ceph-mon[66829]: pgmap v399: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.1 MiB/s wr, 64 op/s 2024-03-19T01:23:26.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:23:26.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:23:26.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:26 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:23:26.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:26 smithi069 ceph-mon[31727]: pgmap v399: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.1 MiB/s wr, 64 op/s 2024-03-19T01:23:26.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:23:26.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:23:26.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:26 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:23:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:26 smithi163 ceph-mon[67428]: pgmap v399: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.1 MiB/s wr, 64 op/s 2024-03-19T01:23:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:23:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:23:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:26 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:23:26.607 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/rockchip/rkisp1/built-in.a 2024-03-19T01:23:26.718 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/rockchip/built-in.a 2024-03-19T01:23:26.744 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ptp/ptp_chardev.o 2024-03-19T01:23:26.919 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/samsung/exynos-gsc/built-in.a 2024-03-19T01:23:27.012 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/storage/initializers.o 2024-03-19T01:23:27.024 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/samsung/exynos4-is/built-in.a 2024-03-19T01:23:27.031 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/pps/sysfs.o 2024-03-19T01:23:27.225 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/samsung/s3c-camif/built-in.a 2024-03-19T01:23:27.304 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/samsung/s5p-g2d/built-in.a 2024-03-19T01:23:27.335 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_mocs.o 2024-03-19T01:23:27.358 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/realtek/r8169_main.o 2024-03-19T01:23:27.427 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/samsung/s5p-jpeg/built-in.a 2024-03-19T01:23:27.517 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/samsung/s5p-mfc/built-in.a 2024-03-19T01:23:27.581 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/xhci-ext-caps.o 2024-03-19T01:23:27.595 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/samsung/built-in.a 2024-03-19T01:23:27.607 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/pps/built-in.a 2024-03-19T01:23:27.650 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/input/built-in.a 2024-03-19T01:23:27.718 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/st/sti/bdisp/built-in.a 2024-03-19T01:23:27.813 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/st/sti/c8sectpfe/built-in.a 2024-03-19T01:23:27.852 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ptp/ptp_sysfs.o 2024-03-19T01:23:27.865 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/st/sti/hva/built-in.a 2024-03-19T01:23:27.887 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/st/sti/delta/built-in.a 2024-03-19T01:23:27.894 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/storage/sierra_ms.o 2024-03-19T01:23:27.907 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/xhci-hub.o 2024-03-19T01:23:27.917 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/xhci-ring.o 2024-03-19T01:23:27.949 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/st/stm32/built-in.a 2024-03-19T01:23:28.003 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/st/built-in.a 2024-03-19T01:23:28.309 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/sunxi/sun4i-csi/built-in.a 2024-03-19T01:23:28.416 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/sunxi/sun6i-csi/built-in.a 2024-03-19T01:23:28.470 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/renesas/built-in.a 2024-03-19T01:23:28.488 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/sunxi/sun6i-mipi-csi2/built-in.a 2024-03-19T01:23:28.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:28 smithi086 ceph-mon[66829]: pgmap v400: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-19T01:23:28.509 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/realtek/r8169_firmware.o 2024-03-19T01:23:28.528 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_ppgtt.o 2024-03-19T01:23:28.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:28 smithi069 ceph-mon[31727]: pgmap v400: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-19T01:23:28.550 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/built-in.a 2024-03-19T01:23:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:28 smithi163 ceph-mon[67428]: pgmap v400: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-19T01:23:28.633 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/sunxi/sun8i-di/built-in.a 2024-03-19T01:23:28.710 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/sunxi/sun8i-rotate/built-in.a 2024-03-19T01:23:28.800 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/sunxi/built-in.a 2024-03-19T01:23:28.932 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ptp/ptp_vclock.o 2024-03-19T01:23:29.056 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/storage/option_ms.o 2024-03-19T01:23:29.063 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/ti/am437x/built-in.a 2024-03-19T01:23:29.125 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/ti/cal/built-in.a 2024-03-19T01:23:29.161 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/xhci-dbg.o 2024-03-19T01:23:29.169 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/ti/vpe/built-in.a 2024-03-19T01:23:29.286 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/ti/davinci/built-in.a 2024-03-19T01:23:29.426 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/ti/omap/built-in.a 2024-03-19T01:23:29.553 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/ti/omap3isp/built-in.a 2024-03-19T01:23:29.620 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/ti/built-in.a 2024-03-19T01:23:29.677 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/rdc/built-in.a 2024-03-19T01:23:29.715 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_rc6.o 2024-03-19T01:23:29.721 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/verisilicon/built-in.a 2024-03-19T01:23:29.764 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/via/built-in.a 2024-03-19T01:23:29.819 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/xilinx/built-in.a 2024-03-19T01:23:29.870 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/platform/built-in.a 2024-03-19T01:23:29.883 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ptp/ptp_kvm_x86.o 2024-03-19T01:23:29.891 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/storage/usual-tables.o 2024-03-19T01:23:30.122 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/xhci-trace.o 2024-03-19T01:23:30.198 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/ethernet/realtek/r8169_phy_config.o 2024-03-19T01:23:30.199 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/usb/b2c2/built-in.a 2024-03-19T01:23:30.237 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/mmc/siano/built-in.a 2024-03-19T01:23:30.256 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/usb/dvb-usb/built-in.a 2024-03-19T01:23:30.280 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/mmc/built-in.a 2024-03-19T01:23:30.352 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/usb/misc/built-in.a 2024-03-19T01:23:30.363 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/usb/dvb-usb-v2/built-in.a 2024-03-19T01:23:30.412 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/power/supply/power_supply_core.o 2024-03-19T01:23:30.496 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/usb/s2255/built-in.a 2024-03-19T01:23:30.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:30 smithi086 ceph-mon[66829]: pgmap v401: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 929 KiB/s wr, 53 op/s 2024-03-19T01:23:30.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:30 smithi069 ceph-mon[31727]: pgmap v401: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 929 KiB/s wr, 53 op/s 2024-03-19T01:23:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:30 smithi163 ceph-mon[67428]: pgmap v401: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 929 KiB/s wr, 53 op/s 2024-03-19T01:23:30.570 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/usb/siano/built-in.a 2024-03-19T01:23:30.650 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/usb/ttusb-budget/built-in.a 2024-03-19T01:23:30.694 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/power/supply/power_supply_sysfs.o 2024-03-19T01:23:30.729 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/usb/ttusb-dec/built-in.a 2024-03-19T01:23:30.794 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/usb/built-in.a 2024-03-19T01:23:30.805 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/ptp/ptp_kvm_common.o 2024-03-19T01:23:30.931 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/firewire/built-in.a 2024-03-19T01:23:30.940 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/usb/storage/built-in.a 2024-03-19T01:23:31.014 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/spi/built-in.a 2024-03-19T01:23:31.090 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/power/supply/power_supply_leds.o 2024-03-19T01:23:31.097 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/test-drivers/built-in.a 2024-03-19T01:23:31.153 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/media/built-in.a 2024-03-19T01:23:31.162 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_region_lmem.o 2024-03-19T01:23:31.281 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/power/supply/power_supply_hwmon.o 2024-03-19T01:23:31.412 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hwmon/hwmon.o 2024-03-19T01:23:31.475 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/rocker/built-in.a 2024-03-19T01:23:31.553 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/xhci-debugfs.o 2024-03-19T01:23:31.696 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_connector.o 2024-03-19T01:23:31.903 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/ptp/built-in.a 2024-03-19T01:23:31.926 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/realtek/built-in.a 2024-03-19T01:23:32.042 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_renderstate.o 2024-03-19T01:23:32.061 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/samsung/built-in.a 2024-03-19T01:23:32.078 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/power/supply/built-in.a 2024-03-19T01:23:32.128 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/seeq/built-in.a 2024-03-19T01:23:32.163 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/power/built-in.a 2024-03-19T01:23:32.195 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/silan/built-in.a 2024-03-19T01:23:32.273 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/early/ehci-dbgp.o 2024-03-19T01:23:32.274 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/sis/built-in.a 2024-03-19T01:23:32.287 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/sfc/built-in.a 2024-03-19T01:23:32.331 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_reset.o 2024-03-19T01:23:32.345 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/smsc/built-in.a 2024-03-19T01:23:32.384 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_ring.o 2024-03-19T01:23:32.395 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/socionext/built-in.a 2024-03-19T01:23:32.473 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/stmicro/built-in.a 2024-03-19T01:23:32.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:32 smithi086 ceph-mon[66829]: pgmap v402: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 278 KiB/s wr, 27 op/s 2024-03-19T01:23:32.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:32 smithi069 ceph-mon[31727]: pgmap v402: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 278 KiB/s wr, 27 op/s 2024-03-19T01:23:32.541 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/sun/built-in.a 2024-03-19T01:23:32.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:32 smithi163 ceph-mon[67428]: pgmap v402: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 278 KiB/s wr, 27 op/s 2024-03-19T01:23:32.615 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/tehuti/built-in.a 2024-03-19T01:23:32.684 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/ti/built-in.a 2024-03-19T01:23:32.689 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/hwmon/built-in.a 2024-03-19T01:23:32.764 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/vertexcom/built-in.a 2024-03-19T01:23:32.821 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/via/built-in.a 2024-03-19T01:23:32.847 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/usb/host/xhci-pci.o 2024-03-19T01:23:32.866 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/wangxun/built-in.a 2024-03-19T01:23:32.927 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/wiznet/built-in.a 2024-03-19T01:23:32.934 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/net/net_failover.o 2024-03-19T01:23:33.007 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/xilinx/built-in.a 2024-03-19T01:23:33.078 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/xircom/built-in.a 2024-03-19T01:23:33.151 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/synopsys/built-in.a 2024-03-19T01:23:33.221 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_ring_submission.o 2024-03-19T01:23:33.224 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/pensando/built-in.a 2024-03-19T01:23:33.288 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/ethernet/built-in.a 2024-03-19T01:23:33.298 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_crtc.o 2024-03-19T01:23:33.354 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/usb/early/built-in.a 2024-03-19T01:23:33.490 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/thermal/broadcom/built-in.a 2024-03-19T01:23:33.540 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/thermal/samsung/built-in.a 2024-03-19T01:23:33.698 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/thermal/intel/intel_tcc.o 2024-03-19T01:23:33.735 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/thermal/intel/therm_throt.o 2024-03-19T01:23:34.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:34 smithi086 ceph-mon[66829]: pgmap v403: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 424 KiB/s wr, 33 op/s 2024-03-19T01:23:34.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:34 smithi069 ceph-mon[31727]: pgmap v403: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 424 KiB/s wr, 33 op/s 2024-03-19T01:23:34.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:34 smithi163 ceph-mon[67428]: pgmap v403: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 424 KiB/s wr, 33 op/s 2024-03-19T01:23:34.639 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/thermal/st/built-in.a 2024-03-19T01:23:35.234 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_displayid.o 2024-03-19T01:23:35.244 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/watchdog/built-in.a 2024-03-19T01:23:35.262 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/usb/host/built-in.a 2024-03-19T01:23:35.287 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_rps.o 2024-03-19T01:23:35.288 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_drv.o 2024-03-19T01:23:35.304 INFO:tasks.workunit.client.0.smithi069.stdout: CC [M] drivers/thermal/intel/x86_pkg_temp_thermal.o 2024-03-19T01:23:35.318 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/md.o 2024-03-19T01:23:35.344 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/net/built-in.a 2024-03-19T01:23:35.983 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/usb/built-in.a 2024-03-19T01:23:35.995 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_dumb_buffers.o 2024-03-19T01:23:36.009 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpufreq/cpufreq.o 2024-03-19T01:23:36.196 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_edid.o 2024-03-19T01:23:36.406 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_sa_media.o 2024-03-19T01:23:36.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:36 smithi086 ceph-mon[66829]: pgmap v404: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 399 KiB/s wr, 26 op/s 2024-03-19T01:23:36.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:36 smithi069 ceph-mon[31727]: pgmap v404: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 399 KiB/s wr, 26 op/s 2024-03-19T01:23:36.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:36 smithi163 ceph-mon[67428]: pgmap v404: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 399 KiB/s wr, 26 op/s 2024-03-19T01:23:36.834 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/thermal/intel/built-in.a 2024-03-19T01:23:36.909 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_sseu.o 2024-03-19T01:23:36.944 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/thermal/qcom/built-in.a 2024-03-19T01:23:36.985 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/thermal/tegra/built-in.a 2024-03-19T01:23:36.995 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/thermal/mediatek/built-in.a 2024-03-19T01:23:37.042 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_sseu_debugfs.o 2024-03-19T01:23:37.046 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/thermal/thermal_core.o 2024-03-19T01:23:37.500 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpufreq/freq_table.o 2024-03-19T01:23:37.598 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/md-bitmap.o 2024-03-19T01:23:38.099 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpuidle/governors/menu.o 2024-03-19T01:23:38.218 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_encoder.o 2024-03-19T01:23:38.379 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpufreq/cpufreq_performance.o 2024-03-19T01:23:38.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:38 smithi086 ceph-mon[66829]: pgmap v405: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 715 KiB/s wr, 51 op/s 2024-03-19T01:23:38.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:38 smithi069 ceph-mon[31727]: pgmap v405: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 715 KiB/s wr, 51 op/s 2024-03-19T01:23:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:38 smithi163 ceph-mon[67428]: pgmap v405: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 715 KiB/s wr, 51 op/s 2024-03-19T01:23:38.790 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpuidle/governors/haltpoll.o 2024-03-19T01:23:38.861 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/thermal/thermal_sysfs.o 2024-03-19T01:23:38.937 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_timeline.o 2024-03-19T01:23:38.977 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpufreq/cpufreq_userspace.o 2024-03-19T01:23:39.185 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/thermal/thermal_trip.o 2024-03-19T01:23:39.197 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/md-autodetect.o 2024-03-19T01:23:39.643 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpufreq/cpufreq_ondemand.o 2024-03-19T01:23:39.808 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/thermal/thermal_helpers.o 2024-03-19T01:23:39.810 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpufreq/cpufreq_governor.o 2024-03-19T01:23:39.830 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/thermal/thermal_hwmon.o 2024-03-19T01:23:39.880 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/cpuidle/governors/built-in.a 2024-03-19T01:23:40.039 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpuidle/cpuidle.o 2024-03-19T01:23:40.197 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/thermal/gov_step_wise.o 2024-03-19T01:23:40.210 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm.o 2024-03-19T01:23:40.441 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_wopcm.o 2024-03-19T01:23:40.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:40 smithi086 ceph-mon[66829]: pgmap v406: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.2 MiB/s wr, 101 op/s 2024-03-19T01:23:40.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:40 smithi069 ceph-mon[31727]: pgmap v406: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.2 MiB/s wr, 101 op/s 2024-03-19T01:23:40.546 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpufreq/cpufreq_governor_attr_set.o 2024-03-19T01:23:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:40 smithi163 ceph-mon[67428]: pgmap v406: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 1.2 MiB/s wr, 101 op/s 2024-03-19T01:23:40.571 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_file.o 2024-03-19T01:23:40.666 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_workarounds.o 2024-03-19T01:23:40.959 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/thermal/gov_user_space.o 2024-03-19T01:23:40.980 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpufreq/acpi-cpufreq.o 2024-03-19T01:23:41.335 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpufreq/intel_pstate.o 2024-03-19T01:23:41.683 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpuidle/driver.o 2024-03-19T01:23:41.843 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/thermal/built-in.a 2024-03-19T01:23:41.924 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/mmc/built-in.a 2024-03-19T01:23:41.986 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/ufs/built-in.a 2024-03-19T01:23:41.986 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpuidle/governor.o 2024-03-19T01:23:42.037 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-table.o 2024-03-19T01:23:42.042 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpuidle/sysfs.o 2024-03-19T01:23:42.058 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpuidle/poll_state.o 2024-03-19T01:23:42.069 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/shmem_utils.o 2024-03-19T01:23:42.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:23:42.071 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:23:42.497 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_fourcc.o 2024-03-19T01:23:42.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:42 smithi086 ceph-mon[66829]: pgmap v407: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-19T01:23:42.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:42 smithi069 ceph-mon[31727]: pgmap v407: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-19T01:23:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:42 smithi163 ceph-mon[67428]: pgmap v407: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-19T01:23:42.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.648+0000 7fe3f1a00640 1 -- 172.21.15.69:0/2884397529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e40a4f10 msgr2=0x7fe3e40a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:42.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.648+0000 7fe3f1a00640 1 --2- 172.21.15.69:0/2884397529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e40a4f10 0x7fe3e40a52f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009930 tx=0x7fe3dc02bcd0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.648+0000 7fe3f1a00640 1 -- 172.21.15.69:0/2884397529 shutdown_connections 2024-03-19T01:23:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.648+0000 7fe3f1a00640 1 --2- 172.21.15.69:0/2884397529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e40b7c30 0x7fe3e40ba020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.648+0000 7fe3f1a00640 1 --2- 172.21.15.69:0/2884397529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e40a5830 0x7fe3e40b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.648+0000 7fe3f1a00640 1 --2- 172.21.15.69:0/2884397529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e40a4f10 0x7fe3e40a52f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.648+0000 7fe3f1a00640 1 -- 172.21.15.69:0/2884397529 >> 172.21.15.69:0/2884397529 conn(0x7fe3e401a800 msgr2=0x7fe3e401ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.648+0000 7fe3f1a00640 1 -- 172.21.15.69:0/2884397529 shutdown_connections 2024-03-19T01:23:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.648+0000 7fe3f1a00640 1 -- 172.21.15.69:0/2884397529 wait complete. 2024-03-19T01:23:42.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.650+0000 7fe3f1a00640 1 Processor -- start 2024-03-19T01:23:42.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.652+0000 7fe3f1a00640 1 -- start start 2024-03-19T01:23:42.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.653+0000 7fe3f1a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e40a4f10 0x7fe3e41498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.653+0000 7fe3f1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e40a5830 0x7fe3e4149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.653+0000 7fe3f1a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e40b7c30 0x7fe3e4143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.653+0000 7fe3f1a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e40bd780 con 0x7fe3e40b7c30 2024-03-19T01:23:42.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.653+0000 7fe3f1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e40bd600 con 0x7fe3e40a5830 2024-03-19T01:23:42.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.653+0000 7fe3f1a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e40bd900 con 0x7fe3e40a4f10 2024-03-19T01:23:42.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.654+0000 7fe3e9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e40a5830 0x7fe3e4149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.654+0000 7fe3e9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e40a5830 0x7fe3e4149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58342/0 (socket says 172.21.15.69:58342) 2024-03-19T01:23:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.654+0000 7fe3e9600640 1 -- 172.21.15.69:0/1417614499 learned_addr learned my addr 172.21.15.69:0/1417614499 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.654+0000 7fe3eaa00640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e40b7c30 0x7fe3e4143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.654+0000 7fe3e9600640 1 -- 172.21.15.69:0/1417614499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e40a4f10 msgr2=0x7fe3e41498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.654+0000 7fe3e9600640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e40a4f10 0x7fe3e41498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.654+0000 7fe3e9600640 1 -- 172.21.15.69:0/1417614499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e40b7c30 msgr2=0x7fe3e4143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.654+0000 7fe3e9600640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e40b7c30 0x7fe3e4143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.654+0000 7fe3e9600640 1 -- 172.21.15.69:0/1417614499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e4144200 con 0x7fe3e40a5830 2024-03-19T01:23:42.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.654+0000 7fe3eaa00640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e40b7c30 0x7fe3e4143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:42.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.654+0000 7fe3e9600640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e40a5830 0x7fe3e4149e10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec066f50 tx=0x7fe3ec06ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:42.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.655+0000 7fe3e3400640 1 -- 172.21.15.69:0/1417614499 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec075420 con 0x7fe3e40a5830 2024-03-19T01:23:42.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.655+0000 7fe3e3400640 1 -- 172.21.15.69:0/1417614499 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3ec07a070 con 0x7fe3e40a5830 2024-03-19T01:23:42.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.655+0000 7fe3e3400640 1 -- 172.21.15.69:0/1417614499 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0748c0 con 0x7fe3e40a5830 2024-03-19T01:23:42.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.655+0000 7fe3f1a00640 1 -- 172.21.15.69:0/1417614499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e41444f0 con 0x7fe3e40a5830 2024-03-19T01:23:42.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.655+0000 7fe3f1a00640 1 -- 172.21.15.69:0/1417614499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e4150790 con 0x7fe3e40a5830 2024-03-19T01:23:42.657 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/leds/trigger/built-in.a 2024-03-19T01:23:42.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.664+0000 7fe3f4772740 1 Processor -- start 2024-03-19T01:23:42.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.664+0000 7fe3f4772740 1 -- start start 2024-03-19T01:23:42.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.665+0000 7fe3e3400640 1 -- 172.21.15.69:0/1417614499 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fe3ec0755c0 con 0x7fe3e40a5830 2024-03-19T01:23:42.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.665+0000 7fe3f4772740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b89310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.666+0000 7fe3f4772740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 0x559618b91ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.666+0000 7fe3eaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b89310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.666+0000 7fe3eaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b89310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58344/0 (socket says 172.21.15.69:58344) 2024-03-19T01:23:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.666+0000 7fe3eaa00640 1 -- 172.21.15.69:0/3906500428 learned_addr learned my addr 172.21.15.69:0/3906500428 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:42.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.666+0000 7fe3f4772740 1 --2- 172.21.15.69:0/3906500428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618b92430 0x559618b94820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.667+0000 7fe3f4772740 1 -- 172.21.15.69:0/3906500428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559618aed2c0 con 0x559618b92430 2024-03-19T01:23:42.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.667+0000 7fe3f4772740 1 -- 172.21.15.69:0/3906500428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559618a95b60 con 0x559618b88f30 2024-03-19T01:23:42.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.667+0000 7fe3f4772740 1 -- 172.21.15.69:0/3906500428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559618ae9360 con 0x559618b89850 2024-03-19T01:23:42.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.667+0000 7fe3e9600640 1 --2- 172.21.15.69:0/3906500428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618b92430 0x559618b94820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.667+0000 7fe3ea000640 1 --2- 172.21.15.69:0/3906500428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 0x559618b91ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.667+0000 7fe3eaa00640 1 -- 172.21.15.69:0/3906500428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 msgr2=0x559618b91ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:42.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.667+0000 7fe3eaa00640 1 --2- 172.21.15.69:0/3906500428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 0x559618b91ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.667+0000 7fe3eaa00640 1 -- 172.21.15.69:0/3906500428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618b92430 msgr2=0x559618b94820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:42.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.667+0000 7fe3eaa00640 1 --2- 172.21.15.69:0/3906500428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618b92430 0x559618b94820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.667+0000 7fe3eaa00640 1 -- 172.21.15.69:0/3906500428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596188cb7d0 con 0x559618b88f30 2024-03-19T01:23:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.668+0000 7fe3eaa00640 1 --2- 172.21.15.69:0/3906500428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b89310 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8005fd0 tx=0x7fe3d800e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf151cde4a1e62a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.669+0000 7fe3d7e00640 1 -- 172.21.15.69:0/3906500428 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8016070 con 0x559618b88f30 2024-03-19T01:23:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.669+0000 7fe3d7e00640 1 -- 172.21.15.69:0/3906500428 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d8005500 con 0x559618b88f30 2024-03-19T01:23:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.669+0000 7fe3d7e00640 1 -- 172.21.15.69:0/3906500428 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8011470 con 0x559618b88f30 2024-03-19T01:23:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.670+0000 7fe3e3400640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe3d00a29f0 0x7fe3d00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.670+0000 7fe3e3400640 1 -- 172.21.15.69:0/1417614499 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7fe3ec132320 con 0x7fe3e40a5830 2024-03-19T01:23:42.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.673+0000 7fe3f4772740 1 -- 172.21.15.69:0/3906500428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 msgr2=0x559618b89310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:42.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.673+0000 7fe3f4772740 1 --2- 172.21.15.69:0/3906500428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b89310 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8005fd0 tx=0x7fe3d800e6c0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.673+0000 7fe3ea000640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe3d00a29f0 0x7fe3d00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.674+0000 7fe3f4772740 1 -- 172.21.15.69:0/3906500428 shutdown_connections 2024-03-19T01:23:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.674+0000 7fe3f4772740 1 --2- 172.21.15.69:0/3906500428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618b92430 0x559618b94820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.674+0000 7fe3f4772740 1 --2- 172.21.15.69:0/3906500428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 0x559618b91ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.674+0000 7fe3f4772740 1 --2- 172.21.15.69:0/3906500428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b89310 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.674+0000 7fe3f4772740 1 -- 172.21.15.69:0/3906500428 >> 172.21.15.69:0/3906500428 conn(0x559618b83660 msgr2=0x559618b83a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.675+0000 7fe3ea000640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe3d00a29f0 0x7fe3d00a4eb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc02c2e0 tx=0x7fe3dc000bc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.675+0000 7fe3f4772740 1 -- 172.21.15.69:0/3906500428 shutdown_connections 2024-03-19T01:23:42.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.676+0000 7fe3f4772740 1 -- 172.21.15.69:0/3906500428 wait complete. 2024-03-19T01:23:42.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.676+0000 7fe3f4772740 1 -- start start 2024-03-19T01:23:42.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.677+0000 7fe3f4772740 1 Processor -- start 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.681+0000 7fe3f4772740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b8f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.681+0000 7fe3f4772740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 0x559618b8f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.681+0000 7fe3f4772740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618b92430 0x559618b8feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.681+0000 7fe3f4772740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559618b95220 con 0x559618b92430 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.681+0000 7fe3f4772740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559618aa8fc0 con 0x559618b88f30 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.681+0000 7fe3f4772740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559618b953a0 con 0x559618b89850 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.681+0000 7fe3f4772740 10 client.0 _ll_register_callbacks cb 0x559618aa8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.682+0000 7fe3e9600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 0x559618b8f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.682+0000 7fe3eaa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618b92430 0x559618b8feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.682+0000 7fe3e9600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 0x559618b8f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55200/0 (socket says 172.21.15.69:55200) 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.682+0000 7fe3e9600640 1 -- 172.21.15.69:0/568520420 learned_addr learned my addr 172.21.15.69:0/568520420 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.682+0000 7fe3ea000640 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b8f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.682+0000 7fe3e9600640 1 -- 172.21.15.69:0/568520420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 msgr2=0x559618b8f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.682+0000 7fe3e9600640 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b8f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.682+0000 7fe3e9600640 1 -- 172.21.15.69:0/568520420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618b92430 msgr2=0x559618b8feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.682+0000 7fe3e9600640 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618b92430 0x559618b8feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.682+0000 7fe3e9600640 1 -- 172.21.15.69:0/568520420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596188013f0 con 0x559618b89850 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.682+0000 7fe3eaa00640 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618b92430 0x559618b8feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.683+0000 7fe3e9600640 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 0x559618b8f970 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec066fc0 tx=0x7fe3ec0fe510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99bfec74277c606e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.683+0000 7fe3d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.683+0000 7fe3d4c00640 1 -- 172.21.15.69:0/568520420 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec132170 con 0x559618b89850 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.683+0000 7fe3f4772740 1 -- 172.21.15.69:0/568520420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d8005500 con 0x559618b89850 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.683+0000 7fe3d4c00640 1 -- 172.21.15.69:0/568520420 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7fe3ec0facf0 con 0x559618b89850 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.683+0000 7fe3d4c00640 1 -- 172.21.15.69:0/568520420 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe3ec0fed80 con 0x559618b89850 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.683+0000 7fe3f4772740 10 client.27907 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.683+0000 7fe3f4772740 1 -- 172.21.15.69:0/568520420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5596187c10f0 con 0x559618b89850 2024-03-19T01:23:42.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.684+0000 7fe3d4c00640 1 -- 172.21.15.69:0/568520420 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d8005500 con 0x559618b89850 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.684+0000 7fe3d4c00640 1 -- 172.21.15.69:0/568520420 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe3ec135c60 con 0x559618b89850 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.684+0000 7fe3f4772740 10 client.27907 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.684+0000 7fe3f4772740 10 client.27907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.684+0000 7fe3f4772740 20 client.27907 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.684+0000 7fe3f4772740 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559618b9fe20 0x559618ba22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.684+0000 7fe3f4772740 4 client.27907 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.684+0000 7fe3f4772740 1 -- 172.21.15.69:0/568520420 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559618a8e610 con 0x559618b9fe20 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.684+0000 7fe3ea000640 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559618b9fe20 0x559618ba22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.688+0000 7fe3ea000640 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559618b9fe20 0x559618ba22a0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8597e6ac58f5064b server_cookie=b3a669ef9652f332 in_seq=0 out_seq=0 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.688+0000 7fe3d4c00640 10 client.27907 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.689+0000 7fe3d4c00640 1 -- 172.21.15.69:0/568520420 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559618a8e610 con 0x559618b9fe20 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.689+0000 7fe3d4c00640 10 client.27907 handle_command_reply: tid=0 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.689+0000 7fe3f4772740 1 client.27907 shutdown 2024-03-19T01:23:42.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.692+0000 7fe3f4772740 1 -- 172.21.15.69:0/568520420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 msgr2=0x559618b8f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.692+0000 7fe3f4772740 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 0x559618b8f970 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec066fc0 tx=0x7fe3ec0fe510 comp rx=0 tx=0).stop 2024-03-19T01:23:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.692+0000 7fe3f4772740 1 -- 172.21.15.69:0/568520420 shutdown_connections 2024-03-19T01:23:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.692+0000 7fe3f4772740 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559618b9fe20 0x559618ba22a0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.692+0000 7fe3f4772740 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618b92430 0x559618b8feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.692+0000 7fe3f4772740 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b89850 0x559618b8f970 secure :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec066fc0 tx=0x7fe3ec0fe510 comp rx=0 tx=0).stop 2024-03-19T01:23:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.692+0000 7fe3f4772740 1 --2- 172.21.15.69:0/568520420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b8f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.693+0000 7fe3d4c00640 0 client.27907 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.693+0000 7fe3f4772740 1 -- 172.21.15.69:0/568520420 >> 172.21.15.69:0/568520420 conn(0x559618b83660 msgr2=0x559618b853b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.693+0000 7fe3f4772740 1 -- 172.21.15.69:0/568520420 shutdown_connections 2024-03-19T01:23:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.693+0000 7fe3f4772740 1 -- 172.21.15.69:0/568520420 wait complete. 2024-03-19T01:23:42.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.693+0000 7fe3f4772740 20 client.27907 trim_cache size 0 max 16384 2024-03-19T01:23:42.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.768+0000 7fe3f4772740 1 Processor -- start 2024-03-19T01:23:42.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.770+0000 7fe3f4772740 1 -- start start 2024-03-19T01:23:42.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.771+0000 7fe3f4772740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b91b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.771+0000 7fe3f4772740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b92430 0x559618b9e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.771+0000 7fe3f4772740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 0x559618b849b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.771+0000 7fe3f4772740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e4152330 con 0x559618baef30 2024-03-19T01:23:42.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.771+0000 7fe3f4772740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e4153e70 con 0x559618b88f30 2024-03-19T01:23:42.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.771+0000 7fe3f4772740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e4152a00 con 0x559618b92430 2024-03-19T01:23:42.773 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/cpuidle/cpuidle-haltpoll.o 2024-03-19T01:23:42.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.771+0000 7fe3ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b91b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.771+0000 7fe3ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b91b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58366/0 (socket says 172.21.15.69:58366) 2024-03-19T01:23:42.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.771+0000 7fe3ea000640 1 -- 172.21.15.69:0/1229439528 learned_addr learned my addr 172.21.15.69:0/1229439528 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:42.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.772+0000 7fe3eaa00640 1 --2- 172.21.15.69:0/1229439528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 0x559618b849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.772+0000 7fe3ea000640 1 -- 172.21.15.69:0/1229439528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b92430 msgr2=0x559618b9e050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:42.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.772+0000 7fe3ea000640 1 --2- 172.21.15.69:0/1229439528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b92430 0x559618b9e050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.772+0000 7fe3ea000640 1 -- 172.21.15.69:0/1229439528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 msgr2=0x559618b849b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:42.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.772+0000 7fe3ea000640 1 --2- 172.21.15.69:0/1229439528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 0x559618b849b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.772+0000 7fe3ea000640 1 -- 172.21.15.69:0/1229439528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559618b8a200 con 0x559618b88f30 2024-03-19T01:23:42.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.772+0000 7fe3eaa00640 1 --2- 172.21.15.69:0/1229439528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 0x559618b849b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:23:42.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.772+0000 7fe3ea000640 1 --2- 172.21.15.69:0/1229439528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b91b60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009930 tx=0x7fe3dc03c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e81a8cb8add4b0b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:42.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.773+0000 7fe3d7400640 1 -- 172.21.15.69:0/1229439528 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc045040 con 0x559618b88f30 2024-03-19T01:23:42.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.773+0000 7fe3d7400640 1 -- 172.21.15.69:0/1229439528 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3dc057070 con 0x559618b88f30 2024-03-19T01:23:42.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.775+0000 7fe3d7400640 1 -- 172.21.15.69:0/1229439528 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc003b50 con 0x559618b88f30 2024-03-19T01:23:42.778 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/leds/blink/built-in.a 2024-03-19T01:23:42.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.778+0000 7fe3f4772740 1 -- 172.21.15.69:0/1229439528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 msgr2=0x559618b91b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:42.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.778+0000 7fe3f4772740 1 --2- 172.21.15.69:0/1229439528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b91b60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009930 tx=0x7fe3dc03c000 comp rx=0 tx=0).stop 2024-03-19T01:23:42.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.778+0000 7fe3f4772740 1 -- 172.21.15.69:0/1229439528 shutdown_connections 2024-03-19T01:23:42.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.778+0000 7fe3f4772740 1 --2- 172.21.15.69:0/1229439528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 0x559618b849b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.778+0000 7fe3f4772740 1 --2- 172.21.15.69:0/1229439528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b92430 0x559618b9e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.778+0000 7fe3f4772740 1 --2- 172.21.15.69:0/1229439528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b88f30 0x559618b91b60 secure :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009930 tx=0x7fe3dc03c000 comp rx=0 tx=0).stop 2024-03-19T01:23:42.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.778+0000 7fe3f4772740 1 -- 172.21.15.69:0/1229439528 >> 172.21.15.69:0/1229439528 conn(0x559618b83660 msgr2=0x559618b8a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:42.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.781+0000 7fe3f4772740 1 -- 172.21.15.69:0/1229439528 shutdown_connections 2024-03-19T01:23:42.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.781+0000 7fe3f4772740 1 -- 172.21.15.69:0/1229439528 wait complete. 2024-03-19T01:23:42.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.781+0000 7fe3f4772740 1 -- start start 2024-03-19T01:23:42.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.782+0000 7fe3f4772740 1 Processor -- start 2024-03-19T01:23:42.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.782+0000 7fe3f4772740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b92430 0x559618b95640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.783+0000 7fe3f4772740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 0x559618b95b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.783+0000 7fe3ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b92430 0x559618b95640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.783+0000 7fe3ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b92430 0x559618b95640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58378/0 (socket says 172.21.15.69:58378) 2024-03-19T01:23:42.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.783+0000 7fe3e9600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 0x559618b95b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.783+0000 7fe3e9600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 0x559618b95b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43250/0 (socket says 172.21.15.69:43250) 2024-03-19T01:23:42.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.783+0000 7fe3ea000640 1 -- 172.21.15.69:0/825733920 learned_addr learned my addr 172.21.15.69:0/825733920 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:42.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.783+0000 7fe3f4772740 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b960c0 0x559618c08710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.784+0000 7fe3f4772740 1 -- 172.21.15.69:0/825733920 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e41537a0 con 0x559618baef30 2024-03-19T01:23:42.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.784+0000 7fe3f4772740 1 -- 172.21.15.69:0/825733920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e41530d0 con 0x559618b92430 2024-03-19T01:23:42.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.784+0000 7fe3f4772740 1 -- 172.21.15.69:0/825733920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e4144fb0 con 0x559618b960c0 2024-03-19T01:23:42.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.784+0000 7fe3e9600640 1 -- 172.21.15.69:0/825733920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b960c0 msgr2=0x559618c08710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:42.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.784+0000 7fe3e9600640 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b960c0 0x559618c08710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.784+0000 7fe3e9600640 1 -- 172.21.15.69:0/825733920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b92430 msgr2=0x559618b95640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:42.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.784+0000 7fe3e9600640 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b92430 0x559618b95640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.785+0000 7fe3e9600640 1 -- 172.21.15.69:0/825733920 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559618c08c50 con 0x559618baef30 2024-03-19T01:23:42.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.785+0000 7fe3e9600640 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 0x559618b95b80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x559618b904c0 tx=0x7fe3ec0fe580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b74e74aa22855a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:42.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.786+0000 7fe3d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:23:42.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.786+0000 7fe3d4c00640 1 -- 172.21.15.69:0/825733920 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0fdfe0 con 0x559618baef30 2024-03-19T01:23:42.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.786+0000 7fe3d4c00640 1 -- 172.21.15.69:0/825733920 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7fe3ec061c60 con 0x559618baef30 2024-03-19T01:23:42.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.786+0000 7fe3d4c00640 10 client.0 inactive, discarding osd_map(108..108 src has 1..108) 2024-03-19T01:23:42.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.786+0000 7fe3f4772740 10 client.0 _ll_register_callbacks cb 0x7fe3ec0fa920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:42.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.786+0000 7fe3f4772740 1 -- 172.21.15.69:0/825733920 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3dc057070 con 0x559618baef30 2024-03-19T01:23:42.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.789+0000 7fe3d4c00640 1 -- 172.21.15.69:0/825733920 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe3ec132170 con 0x559618baef30 2024-03-19T01:23:42.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.792+0000 7fe3f4772740 10 client.17604 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:42.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.792+0000 7fe3f4772740 1 -- 172.21.15.69:0/825733920 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=59}) -- 0x559618c09b90 con 0x559618baef30 2024-03-19T01:23:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.792+0000 7fe3d4c00640 1 -- 172.21.15.69:0/825733920 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe3ec1069e0 con 0x559618baef30 2024-03-19T01:23:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.792+0000 7fe3f4772740 10 client.17604 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.792+0000 7fe3f4772740 10 client.17604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:23:42.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.792+0000 7fe3f4772740 20 client.17604 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.792+0000 7fe3f4772740 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559618c0a2d0 0x559618c0c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.792+0000 7fe3f4772740 4 client.17604 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:23:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.792+0000 7fe3f4772740 1 -- 172.21.15.69:0/825733920 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe3dc057070 con 0x559618c0a2d0 2024-03-19T01:23:42.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.793+0000 7fe3ea000640 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559618c0a2d0 0x559618c0c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:42.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.794+0000 7fe3d4c00640 10 client.17604 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:42.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.794+0000 7fe3ea000640 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559618c0a2d0 0x559618c0c790 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f9e7db69e5cf35 server_cookie=c5908dad944c17cb in_seq=0 out_seq=0 2024-03-19T01:23:42.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.794+0000 7fe3d4c00640 1 -- 172.21.15.69:0/825733920 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fe3dc057070 con 0x559618c0a2d0 2024-03-19T01:23:42.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.794+0000 7fe3d4c00640 10 client.17604 handle_command_reply: tid=0 2024-03-19T01:23:42.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.794+0000 7fe3f4772740 1 client.17604 shutdown 2024-03-19T01:23:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.798+0000 7fe3f4772740 1 -- 172.21.15.69:0/825733920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 msgr2=0x559618b95b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.798+0000 7fe3f4772740 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 0x559618b95b80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x559618b904c0 tx=0x7fe3ec0fe580 comp rx=0 tx=0).stop 2024-03-19T01:23:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.798+0000 7fe3f4772740 1 -- 172.21.15.69:0/825733920 shutdown_connections 2024-03-19T01:23:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.798+0000 7fe3f4772740 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559618c0a2d0 0x559618c0c790 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.798+0000 7fe3d4c00640 0 client.17604 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.798+0000 7fe3f4772740 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559618b960c0 0x559618c08710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.798+0000 7fe3f4772740 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559618baef30 0x559618b95b80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.798+0000 7fe3f4772740 1 --2- 172.21.15.69:0/825733920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559618b92430 0x559618b95640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.798+0000 7fe3f4772740 1 -- 172.21.15.69:0/825733920 >> 172.21.15.69:0/825733920 conn(0x559618b83660 msgr2=0x559618b849b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:42.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.801+0000 7fe3f4772740 1 -- 172.21.15.69:0/825733920 shutdown_connections 2024-03-19T01:23:42.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.801+0000 7fe3f4772740 1 -- 172.21.15.69:0/825733920 wait complete. 2024-03-19T01:23:42.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.802+0000 7fe3f4772740 20 client.17604 trim_cache size 0 max 16384 2024-03-19T01:23:42.804 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:23:42.804 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:23:42.804 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:23:42.804 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:23:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 -- 172.21.15.69:0/1417614499 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe3d00a29f0 msgr2=0x7fe3d00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe3d00a29f0 0x7fe3d00a4eb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc02c2e0 tx=0x7fe3dc000bc0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 -- 172.21.15.69:0/1417614499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e40a5830 msgr2=0x7fe3e4149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e40a5830 0x7fe3e4149e10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec066f50 tx=0x7fe3ec06ff90 comp rx=0 tx=0).stop 2024-03-19T01:23:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 -- 172.21.15.69:0/1417614499 shutdown_connections 2024-03-19T01:23:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e40b7c30 0x7fe3e4143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe3d00a29f0 0x7fe3d00a4eb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3e40a5830 0x7fe3e4149e10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 --2- 172.21.15.69:0/1417614499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e40a4f10 0x7fe3e41498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 -- 172.21.15.69:0/1417614499 >> 172.21.15.69:0/1417614499 conn(0x7fe3e401a800 msgr2=0x7fe3e40b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:42.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 -- 172.21.15.69:0/1417614499 shutdown_connections 2024-03-19T01:23:42.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:42.808+0000 7fe3f1a00640 1 -- 172.21.15.69:0/1417614499 wait complete. 2024-03-19T01:23:42.870 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:23:42.870 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:23:42.883 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/leds/simple/built-in.a 2024-03-19T01:23:42.920 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/arm_ffa/built-in.a 2024-03-19T01:23:42.950 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/leds/led-core.o 2024-03-19T01:23:43.004 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/arm_scmi/built-in.a 2024-03-19T01:23:43.103 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/broadcom/built-in.a 2024-03-19T01:23:43.169 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/cirrus/built-in.a 2024-03-19T01:23:43.239 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/meson/built-in.a 2024-03-19T01:23:43.302 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/sysfs_engines.o 2024-03-19T01:23:43.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.476+0000 7ff1fce00640 1 -- 172.21.15.69:0/1655367895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f8110050 msgr2=0x7ff1f810db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:43.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.476+0000 7ff1fce00640 1 --2- 172.21.15.69:0/1655367895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f8110050 0x7ff1f810db40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009cd0 tx=0x7ff1e802c100 comp rx=0 tx=0).stop 2024-03-19T01:23:43.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.478+0000 7ff1fce00640 1 -- 172.21.15.69:0/1655367895 shutdown_connections 2024-03-19T01:23:43.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.478+0000 7ff1fce00640 1 --2- 172.21.15.69:0/1655367895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f811b000 0x7ff1f811d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:43.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.478+0000 7ff1fce00640 1 --2- 172.21.15.69:0/1655367895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f810e080 0x7ff1f810e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:43.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.478+0000 7ff1fce00640 1 --2- 172.21.15.69:0/1655367895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f8110050 0x7ff1f810db40 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:43.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.478+0000 7ff1fce00640 1 -- 172.21.15.69:0/1655367895 >> 172.21.15.69:0/1655367895 conn(0x7ff1f806c910 msgr2=0x7ff1f806cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:43.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.479+0000 7ff1fce00640 1 -- 172.21.15.69:0/1655367895 shutdown_connections 2024-03-19T01:23:43.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.479+0000 7ff1fce00640 1 -- 172.21.15.69:0/1655367895 wait complete. 2024-03-19T01:23:43.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.480+0000 7ff1fce00640 1 Processor -- start 2024-03-19T01:23:43.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.483+0000 7ff1fce00640 1 -- start start 2024-03-19T01:23:43.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.483+0000 7ff1fce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f810e080 0x7ff1f811a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:43.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.483+0000 7ff1fce00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f8110050 0x7ff1f8113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:43.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.483+0000 7ff1fce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f811b000 0x7ff1f8113cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:43.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.483+0000 7ff1fce00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8120a60 con 0x7ff1f8110050 2024-03-19T01:23:43.485 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/crypto/stm32/built-in.a 2024-03-19T01:23:43.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.483+0000 7ff1fce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81208e0 con 0x7ff1f811b000 2024-03-19T01:23:43.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.483+0000 7ff1fce00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8120be0 con 0x7ff1f810e080 2024-03-19T01:23:43.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.484+0000 7ff1f5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f810e080 0x7ff1f811a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:43.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.484+0000 7ff1f5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f810e080 0x7ff1f811a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55220/0 (socket says 172.21.15.69:55220) 2024-03-19T01:23:43.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.484+0000 7ff1f5e00640 1 -- 172.21.15.69:0/207738358 learned_addr learned my addr 172.21.15.69:0/207738358 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:43.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.484+0000 7ff1f6800640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f811b000 0x7ff1f8113cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:43.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.484+0000 7ff1f6800640 1 -- 172.21.15.69:0/207738358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f810e080 msgr2=0x7ff1f811a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:43.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.484+0000 7ff1f6800640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f810e080 0x7ff1f811a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:43.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.484+0000 7ff1f6800640 1 -- 172.21.15.69:0/207738358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f8110050 msgr2=0x7ff1f8113790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:23:43.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.484+0000 7ff1f6800640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f8110050 0x7ff1f8113790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:43.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.484+0000 7ff1f6800640 1 -- 172.21.15.69:0/207738358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f8114400 con 0x7ff1f811b000 2024-03-19T01:23:43.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.485+0000 7ff1f6800640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f811b000 0x7ff1f8113cd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000b3e0 tx=0x7ff1f000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:43.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.485+0000 7ff1eea00640 1 -- 172.21.15.69:0/207738358 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0015020 con 0x7ff1f811b000 2024-03-19T01:23:43.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.485+0000 7ff1eea00640 1 -- 172.21.15.69:0/207738358 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f0011d10 con 0x7ff1f811b000 2024-03-19T01:23:43.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.485+0000 7ff1eea00640 1 -- 172.21.15.69:0/207738358 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000f2e0 con 0x7ff1f811b000 2024-03-19T01:23:43.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.485+0000 7ff1fce00640 1 -- 172.21.15.69:0/207738358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f8118230 con 0x7ff1f811b000 2024-03-19T01:23:43.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.485+0000 7ff1fce00640 1 -- 172.21.15.69:0/207738358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f8118730 con 0x7ff1f811b000 2024-03-19T01:23:43.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.488+0000 7ff1fce00640 1 -- 172.21.15.69:0/207738358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f806dbb0 con 0x7ff1f811b000 2024-03-19T01:23:43.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.491+0000 7ff1eea00640 1 -- 172.21.15.69:0/207738358 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ff1f0018020 con 0x7ff1f811b000 2024-03-19T01:23:43.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.494+0000 7ff1eea00640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1dc0a29f0 0x7ff1dc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.494+0000 7ff1eea00640 1 -- 172.21.15.69:0/207738358 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7ff1f00ce8f0 con 0x7ff1f811b000 2024-03-19T01:23:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.494+0000 7ff1f5e00640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1dc0a29f0 0x7ff1dc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:43.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.497+0000 7ff1f5e00640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1dc0a29f0 0x7ff1dc0a4eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009a60 tx=0x7ff1e80324a0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:43.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:43.502+0000 7ff1eea00640 1 -- 172.21.15.69:0/207738358 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff1f001f020 con 0x7ff1f811b000 2024-03-19T01:23:43.565 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/crypto/xilinx/built-in.a 2024-03-19T01:23:43.570 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/cpuidle/built-in.a 2024-03-19T01:23:43.659 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/crypto/hisilicon/built-in.a 2024-03-19T01:23:43.696 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-target.o 2024-03-19T01:23:43.702 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/leds/led-class.o 2024-03-19T01:23:43.765 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/efi-stub-helper.o 2024-03-19T01:23:43.853 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/crypto/intel/keembay/built-in.a 2024-03-19T01:23:43.857 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/cpufreq/built-in.a 2024-03-19T01:23:43.907 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/crypto/intel/ixp4xx/built-in.a 2024-03-19T01:23:43.934 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/crypto/starfive/built-in.a 2024-03-19T01:23:43.963 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/crypto/intel/built-in.a 2024-03-19T01:23:43.977 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/efi-bgrt.o 2024-03-19T01:23:43.994 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/imx/built-in.a 2024-03-19T01:23:44.022 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/crypto/built-in.a 2024-03-19T01:23:44.052 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/leds/led-triggers.o 2024-03-19T01:23:44.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.069+0000 7ff1fce00640 1 -- 172.21.15.69:0/207738358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1f8114cc0 con 0x7ff1f811b000 2024-03-19T01:23:44.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.070+0000 7ff1eea00640 1 -- 172.21.15.69:0/207738358 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11930 (secure 0 0 0) 0x7ff1f009b350 con 0x7ff1f811b000 2024-03-19T01:23:44.270 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:23:44.270 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":59,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:20:08.600901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:23:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 59 2024-03-19T01:23:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.081+0000 7ff1fce00640 1 -- 172.21.15.69:0/207738358 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1dc0a29f0 msgr2=0x7ff1dc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.081+0000 7ff1fce00640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1dc0a29f0 0x7ff1dc0a4eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009a60 tx=0x7ff1e80324a0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.081+0000 7ff1fce00640 1 -- 172.21.15.69:0/207738358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f811b000 msgr2=0x7ff1f8113cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.081+0000 7ff1fce00640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f811b000 0x7ff1f8113cd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000b3e0 tx=0x7ff1f000b8a0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.081+0000 7ff1fce00640 1 -- 172.21.15.69:0/207738358 shutdown_connections 2024-03-19T01:23:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.081+0000 7ff1fce00640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1dc0a29f0 0x7ff1dc0a4eb0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009a60 tx=0x7ff1e80324a0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.081+0000 7ff1fce00640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f811b000 0x7ff1f8113cd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.081+0000 7ff1fce00640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f8110050 0x7ff1f8113790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.081+0000 7ff1fce00640 1 --2- 172.21.15.69:0/207738358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f810e080 0x7ff1f811a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.081+0000 7ff1fce00640 1 -- 172.21.15.69:0/207738358 >> 172.21.15.69:0/207738358 conn(0x7ff1f806c910 msgr2=0x7ff1f810cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:44.272 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_framebuffer.o 2024-03-19T01:23:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.083+0000 7ff1fce00640 1 -- 172.21.15.69:0/207738358 shutdown_connections 2024-03-19T01:23:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.083+0000 7ff1fce00640 1 -- 172.21.15.69:0/207738358 wait complete. 2024-03-19T01:23:44.274 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:23:44.274 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/clocksource/acpi_pm.o 2024-03-19T01:23:44.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:44 smithi086 ceph-mon[66829]: pgmap v408: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-03-19T01:23:44.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:44 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/207738358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:23:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:44 smithi069 ceph-mon[31727]: pgmap v408: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-03-19T01:23:44.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:44 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/207738358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:23:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:44 smithi163 ceph-mon[67428]: pgmap v408: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-03-19T01:23:44.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:44 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/207738358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:23:44.743 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/usbhid/hid-core.o 2024-03-19T01:23:44.793 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-linear.o 2024-03-19T01:23:44.836 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/gop.o 2024-03-19T01:23:44.862 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_ggtt_gmch.o 2024-03-19T01:23:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.887+0000 7fc59c200640 1 -- 172.21.15.69:0/720130834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc594107870 msgr2=0x7fc594107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.887+0000 7fc59c200640 1 --2- 172.21.15.69:0/720130834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc594107870 0x7fc594107c50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc580009930 tx=0x7fc58002bc60 comp rx=0 tx=0).stop 2024-03-19T01:23:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.888+0000 7fc59c200640 1 -- 172.21.15.69:0/720130834 shutdown_connections 2024-03-19T01:23:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.888+0000 7fc59c200640 1 --2- 172.21.15.69:0/720130834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5940693e0 0x7fc59410e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.888+0000 7fc59c200640 1 --2- 172.21.15.69:0/720130834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc594068a20 0x7fc594068ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.888+0000 7fc59c200640 1 --2- 172.21.15.69:0/720130834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc594107870 0x7fc594107c50 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.888+0000 7fc59c200640 1 -- 172.21.15.69:0/720130834 >> 172.21.15.69:0/720130834 conn(0x7fc594077820 msgr2=0x7fc594077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.888+0000 7fc59c200640 1 -- 172.21.15.69:0/720130834 shutdown_connections 2024-03-19T01:23:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.889+0000 7fc59c200640 1 -- 172.21.15.69:0/720130834 wait complete. 2024-03-19T01:23:44.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.890+0000 7fc59c200640 1 Processor -- start 2024-03-19T01:23:44.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.890+0000 7fc59c200640 1 -- start start 2024-03-19T01:23:44.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.891+0000 7fc59c200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc594068a20 0x7fc594100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:44.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.891+0000 7fc59c200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5940693e0 0x7fc594100c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:44.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.891+0000 7fc59c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc594107870 0x7fc594101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:44.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.891+0000 7fc59c200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc594111cd0 con 0x7fc594068a20 2024-03-19T01:23:44.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.891+0000 7fc59c200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc594111b50 con 0x7fc594107870 2024-03-19T01:23:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.891+0000 7fc59c200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc594111e50 con 0x7fc5940693e0 2024-03-19T01:23:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.891+0000 7fc598e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5940693e0 0x7fc594100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.891+0000 7fc598e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5940693e0 0x7fc594100c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55238/0 (socket says 172.21.15.69:55238) 2024-03-19T01:23:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.891+0000 7fc598e00640 1 -- 172.21.15.69:0/625121261 learned_addr learned my addr 172.21.15.69:0/625121261 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.892+0000 7fc59a200640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc594107870 0x7fc594101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.892+0000 7fc598e00640 1 -- 172.21.15.69:0/625121261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc594107870 msgr2=0x7fc594101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.892+0000 7fc598e00640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc594107870 0x7fc594101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.892+0000 7fc598e00640 1 -- 172.21.15.69:0/625121261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc594068a20 msgr2=0x7fc594100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.892+0000 7fc598e00640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc594068a20 0x7fc594100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.892+0000 7fc598e00640 1 -- 172.21.15.69:0/625121261 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5941aab70 con 0x7fc5940693e0 2024-03-19T01:23:44.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.892+0000 7fc598e00640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5940693e0 0x7fc594100c40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc58800b450 tx=0x7fc58800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:44.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.893+0000 7fc592000640 1 -- 172.21.15.69:0/625121261 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588025070 con 0x7fc5940693e0 2024-03-19T01:23:44.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.893+0000 7fc592000640 1 -- 172.21.15.69:0/625121261 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc588021ce0 con 0x7fc5940693e0 2024-03-19T01:23:44.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.893+0000 7fc592000640 1 -- 172.21.15.69:0/625121261 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588028e60 con 0x7fc5940693e0 2024-03-19T01:23:44.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.894+0000 7fc59c200640 1 -- 172.21.15.69:0/625121261 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59419c390 con 0x7fc5940693e0 2024-03-19T01:23:44.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.894+0000 7fc59c200640 1 -- 172.21.15.69:0/625121261 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59419c860 con 0x7fc5940693e0 2024-03-19T01:23:44.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.899+0000 7fc592000640 1 -- 172.21.15.69:0/625121261 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fc588028070 con 0x7fc5940693e0 2024-03-19T01:23:44.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.904+0000 7fc59eff7740 1 Processor -- start 2024-03-19T01:23:44.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.906+0000 7fc592000640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc57c0a29f0 0x7fc57c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:44.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.906+0000 7fc592000640 1 -- 172.21.15.69:0/625121261 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7fc5880df8a0 con 0x7fc5940693e0 2024-03-19T01:23:44.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.907+0000 7fc59a200640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc57c0a29f0 0x7fc57c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:44.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.908+0000 7fc59eff7740 1 -- start start 2024-03-19T01:23:44.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.908+0000 7fc59a200640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc57c0a29f0 0x7fc57c0a4eb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009ec0 tx=0x7fc58c009430 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:44.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.909+0000 7fc59eff7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x557412888e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:44.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.909+0000 7fc59eff7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x5574128919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:44.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.909+0000 7fc59eff7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 0x557412894310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:44.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.909+0000 7fc59eff7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5574127ed380 con 0x557412888a20 2024-03-19T01:23:44.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.909+0000 7fc59eff7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557412795d20 con 0x557412891f20 2024-03-19T01:23:44.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.909+0000 7fc59eff7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5574127e9420 con 0x557412889340 2024-03-19T01:23:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.910+0000 7fc599800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x557412888e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.910+0000 7fc598e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x5574128919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.910+0000 7fc598e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x5574128919e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55244/0 (socket says 172.21.15.69:55244) 2024-03-19T01:23:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.910+0000 7fc598e00640 1 -- 172.21.15.69:0/3346998708 learned_addr learned my addr 172.21.15.69:0/3346998708 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.910+0000 7fc598e00640 1 -- 172.21.15.69:0/3346998708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 msgr2=0x557412894310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.910+0000 7fc598e00640 1 --2- 172.21.15.69:0/3346998708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 0x557412894310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.910+0000 7fc598e00640 1 -- 172.21.15.69:0/3346998708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 msgr2=0x557412888e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.910+0000 7fc598e00640 1 --2- 172.21.15.69:0/3346998708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x557412888e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.910+0000 7fc598e00640 1 -- 172.21.15.69:0/3346998708 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574125013f0 con 0x557412889340 2024-03-19T01:23:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.911+0000 7fc599800640 1 --2- 172.21.15.69:0/3346998708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x557412888e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:23:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.911+0000 7fc598e00640 1 --2- 172.21.15.69:0/3346998708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x5574128919e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc5880047b0 tx=0x7fc5880ac300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=916c895a13ab22ba server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:44.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.911+0000 7fc586a00640 1 -- 172.21.15.69:0/3346998708 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5880b43a0 con 0x557412889340 2024-03-19T01:23:44.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.911+0000 7fc586a00640 1 -- 172.21.15.69:0/3346998708 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc588028480 con 0x557412889340 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.911+0000 7fc586a00640 1 -- 172.21.15.69:0/3346998708 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5880df6b0 con 0x557412889340 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.912+0000 7fc59eff7740 1 -- 172.21.15.69:0/3346998708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 msgr2=0x5574128919e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.912+0000 7fc59eff7740 1 --2- 172.21.15.69:0/3346998708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x5574128919e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc5880047b0 tx=0x7fc5880ac300 comp rx=0 tx=0).stop 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.913+0000 7fc59eff7740 1 -- 172.21.15.69:0/3346998708 shutdown_connections 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.913+0000 7fc59eff7740 1 --2- 172.21.15.69:0/3346998708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 0x557412894310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.913+0000 7fc59eff7740 1 --2- 172.21.15.69:0/3346998708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x5574128919e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.913+0000 7fc59eff7740 1 --2- 172.21.15.69:0/3346998708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x557412888e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.913+0000 7fc59eff7740 1 -- 172.21.15.69:0/3346998708 >> 172.21.15.69:0/3346998708 conn(0x557412883150 msgr2=0x557412883530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.913+0000 7fc59eff7740 1 -- 172.21.15.69:0/3346998708 shutdown_connections 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.913+0000 7fc59eff7740 1 -- 172.21.15.69:0/3346998708 wait complete. 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.913+0000 7fc59eff7740 1 -- start start 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.914+0000 7fc59eff7740 1 Processor -- start 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc59eff7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x55741288efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc59eff7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x55741288f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:44.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc59eff7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 0x55741288fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc59eff7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557412894cb0 con 0x557412888a20 2024-03-19T01:23:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc59eff7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574127a9160 con 0x557412891f20 2024-03-19T01:23:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc59eff7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557412894e30 con 0x557412889340 2024-03-19T01:23:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc598e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x55741288f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc598e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x55741288f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55256/0 (socket says 172.21.15.69:55256) 2024-03-19T01:23:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc598e00640 1 -- 172.21.15.69:0/3507891551 learned_addr learned my addr 172.21.15.69:0/3507891551 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc59eff7740 10 client.0 _ll_register_callbacks cb 0x5574127e5170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc598e00640 1 -- 172.21.15.69:0/3507891551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 msgr2=0x55741288fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc598e00640 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 0x55741288fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.915+0000 7fc598e00640 1 -- 172.21.15.69:0/3507891551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 msgr2=0x55741288efb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.916+0000 7fc599800640 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x55741288efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.916+0000 7fc598e00640 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x55741288efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.916+0000 7fc598e00640 1 -- 172.21.15.69:0/3507891551 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574125cb7d0 con 0x557412889340 2024-03-19T01:23:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.916+0000 7fc598e00640 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x55741288f4f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc5880abb90 tx=0x7fc5880dd160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46e138163ccec7a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.916+0000 7fc57b400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:23:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.916+0000 7fc599800640 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x55741288efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.917+0000 7fc57b400640 1 -- 172.21.15.69:0/3507891551 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5880c8070 con 0x557412889340 2024-03-19T01:23:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.917+0000 7fc57b400640 1 -- 172.21.15.69:0/3507891551 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7fc5880ab260 con 0x557412889340 2024-03-19T01:23:44.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.917+0000 7fc59eff7740 1 -- 172.21.15.69:0/3507891551 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc588028480 con 0x557412889340 2024-03-19T01:23:44.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.917+0000 7fc57b400640 1 -- 172.21.15.69:0/3507891551 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fc5880b43a0 con 0x557412889340 2024-03-19T01:23:44.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.921+0000 7fc59eff7740 10 client.27922 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:44.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.921+0000 7fc59eff7740 1 -- 172.21.15.69:0/3507891551 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x557412894cb0 con 0x557412889340 2024-03-19T01:23:44.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.921+0000 7fc57b400640 1 -- 172.21.15.69:0/3507891551 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc588028480 con 0x557412889340 2024-03-19T01:23:44.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.921+0000 7fc57b400640 1 -- 172.21.15.69:0/3507891551 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fc5880aaac0 con 0x557412889340 2024-03-19T01:23:44.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.923+0000 7fc59eff7740 10 client.27922 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:44.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.923+0000 7fc59eff7740 10 client.27922 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:23:44.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.923+0000 7fc59eff7740 20 client.27922 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:44.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.923+0000 7fc59eff7740 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5574128a1f00 0x5574128a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:44.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.923+0000 7fc59eff7740 4 client.27922 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:23:44.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.923+0000 7fc59eff7740 1 -- 172.21.15.69:0/3507891551 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574127fa8c0 con 0x5574128a1f00 2024-03-19T01:23:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.925+0000 7fc599800640 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5574128a1f00 0x5574128a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:44.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.926+0000 7fc599800640 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5574128a1f00 0x5574128a42f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dd95faa24c9f0e41 server_cookie=b94b428912d021db in_seq=0 out_seq=0 2024-03-19T01:23:44.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.926+0000 7fc57b400640 10 client.27922 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:23:44.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.927+0000 7fc57b400640 1 -- 172.21.15.69:0/3507891551 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5574127fa8c0 con 0x5574128a1f00 2024-03-19T01:23:44.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.927+0000 7fc57b400640 10 client.27922 handle_command_reply: tid=0 2024-03-19T01:23:44.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.927+0000 7fc59eff7740 1 client.27922 shutdown 2024-03-19T01:23:44.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.932+0000 7fc59eff7740 1 -- 172.21.15.69:0/3507891551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 msgr2=0x55741288f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.932+0000 7fc59eff7740 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x55741288f4f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc5880abb90 tx=0x7fc5880dd160 comp rx=0 tx=0).stop 2024-03-19T01:23:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.932+0000 7fc59eff7740 1 -- 172.21.15.69:0/3507891551 shutdown_connections 2024-03-19T01:23:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.932+0000 7fc59eff7740 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5574128a1f00 0x5574128a42f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.932+0000 7fc59eff7740 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 0x55741288fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.932+0000 7fc59eff7740 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x55741288f4f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.932+0000 7fc59eff7740 1 --2- 172.21.15.69:0/3507891551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x55741288efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.932+0000 7fc57b400640 0 client.27922 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:23:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.932+0000 7fc59eff7740 1 -- 172.21.15.69:0/3507891551 >> 172.21.15.69:0/3507891551 conn(0x557412883150 msgr2=0x557412884ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.933+0000 7fc59eff7740 1 -- 172.21.15.69:0/3507891551 shutdown_connections 2024-03-19T01:23:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.933+0000 7fc59eff7740 1 -- 172.21.15.69:0/3507891551 wait complete. 2024-03-19T01:23:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:44.933+0000 7fc59eff7740 20 client.27922 trim_cache size 0 max 16384 2024-03-19T01:23:45.083 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/platform/x86/amd/built-in.a 2024-03-19T01:23:45.092 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/leds/built-in.a 2024-03-19T01:23:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.102+0000 7fc59eff7740 1 Processor -- start 2024-03-19T01:23:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.102+0000 7fc59eff7740 1 -- start start 2024-03-19T01:23:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.103+0000 7fc59eff7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412888a20 0x55741289da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.103+0000 7fc59eff7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x55741288f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.103+0000 7fc599800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412888a20 0x55741289da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.103+0000 7fc599800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412888a20 0x55741289da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58440/0 (socket says 172.21.15.69:58440) 2024-03-19T01:23:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.103+0000 7fc599800640 1 -- 172.21.15.69:0/2806613570 learned_addr learned my addr 172.21.15.69:0/2806613570 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.104+0000 7fc598e00640 1 --2- 172.21.15.69:0/2806613570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x55741288f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.104+0000 7fc59eff7740 1 --2- 172.21.15.69:0/2806613570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412891f20 0x5574128a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.105+0000 7fc59eff7740 1 -- 172.21.15.69:0/2806613570 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941acda0 con 0x557412891f20 2024-03-19T01:23:45.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.105+0000 7fc59eff7740 1 -- 172.21.15.69:0/2806613570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941ae8e0 con 0x557412888a20 2024-03-19T01:23:45.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.105+0000 7fc59eff7740 1 -- 172.21.15.69:0/2806613570 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941ad470 con 0x557412889340 2024-03-19T01:23:45.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.105+0000 7fc59a200640 1 --2- 172.21.15.69:0/2806613570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412891f20 0x5574128a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.105+0000 7fc598e00640 1 -- 172.21.15.69:0/2806613570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412888a20 msgr2=0x55741289da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:45.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.105+0000 7fc598e00640 1 --2- 172.21.15.69:0/2806613570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412888a20 0x55741289da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.105+0000 7fc598e00640 1 -- 172.21.15.69:0/2806613570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412891f20 msgr2=0x5574128a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:45.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.106+0000 7fc598e00640 1 --2- 172.21.15.69:0/2806613570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412891f20 0x5574128a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.106+0000 7fc598e00640 1 -- 172.21.15.69:0/2806613570 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557412887da0 con 0x557412889340 2024-03-19T01:23:45.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.106+0000 7fc59a200640 1 --2- 172.21.15.69:0/2806613570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412891f20 0x5574128a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:45.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.106+0000 7fc598e00640 1 --2- 172.21.15.69:0/2806613570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x55741288f850 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc5880a8580 tx=0x7fc5880ac300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec04a5f01a458b9b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:45.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.107+0000 7fc586000640 1 -- 172.21.15.69:0/2806613570 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5880c8070 con 0x557412889340 2024-03-19T01:23:45.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.107+0000 7fc586000640 1 -- 172.21.15.69:0/2806613570 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc588028480 con 0x557412889340 2024-03-19T01:23:45.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.107+0000 7fc586000640 1 -- 172.21.15.69:0/2806613570 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5880aaac0 con 0x557412889340 2024-03-19T01:23:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.108+0000 7fc59eff7740 1 -- 172.21.15.69:0/2806613570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 msgr2=0x55741288f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.108+0000 7fc59eff7740 1 --2- 172.21.15.69:0/2806613570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x55741288f850 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc5880a8580 tx=0x7fc5880ac300 comp rx=0 tx=0).stop 2024-03-19T01:23:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.108+0000 7fc59eff7740 1 -- 172.21.15.69:0/2806613570 shutdown_connections 2024-03-19T01:23:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.108+0000 7fc59eff7740 1 --2- 172.21.15.69:0/2806613570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412891f20 0x5574128a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.108+0000 7fc59eff7740 1 --2- 172.21.15.69:0/2806613570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x55741288f850 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.108+0000 7fc59eff7740 1 --2- 172.21.15.69:0/2806613570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412888a20 0x55741289da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.108+0000 7fc59eff7740 1 -- 172.21.15.69:0/2806613570 >> 172.21.15.69:0/2806613570 conn(0x557412883150 msgr2=0x557412884510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:45.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.109+0000 7fc59eff7740 1 -- 172.21.15.69:0/2806613570 shutdown_connections 2024-03-19T01:23:45.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.109+0000 7fc59eff7740 1 -- 172.21.15.69:0/2806613570 wait complete. 2024-03-19T01:23:45.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.109+0000 7fc59eff7740 1 -- start start 2024-03-19T01:23:45.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.110+0000 7fc59eff7740 1 Processor -- start 2024-03-19T01:23:45.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.111+0000 7fc59eff7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x55741289a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.111+0000 7fc599800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x55741289a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.112+0000 7fc59eff7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x5574128956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.112+0000 7fc59eff7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 0x557412895c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.112+0000 7fc599800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x55741289a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43304/0 (socket says 172.21.15.69:43304) 2024-03-19T01:23:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.112+0000 7fc599800640 1 -- 172.21.15.69:0/699203062 learned_addr learned my addr 172.21.15.69:0/699203062 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.112+0000 7fc59a200640 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 0x557412895c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.113+0000 7fc59eff7740 1 -- 172.21.15.69:0/699203062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941ae210 con 0x557412888a20 2024-03-19T01:23:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.113+0000 7fc59eff7740 1 -- 172.21.15.69:0/699203062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941adb40 con 0x557412891f20 2024-03-19T01:23:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.113+0000 7fc59eff7740 1 -- 172.21.15.69:0/699203062 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941aad80 con 0x557412889340 2024-03-19T01:23:45.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.113+0000 7fc59eff7740 10 client.0 _ll_register_callbacks cb 0x7fc5880dd090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.113+0000 7fc59a200640 1 -- 172.21.15.69:0/699203062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 msgr2=0x5574128956c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:23:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.113+0000 7fc59a200640 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x5574128956c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.113+0000 7fc59a200640 1 -- 172.21.15.69:0/699203062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 msgr2=0x55741289a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.113+0000 7fc59a200640 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x55741289a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.113+0000 7fc59a200640 1 -- 172.21.15.69:0/699203062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557412896400 con 0x557412891f20 2024-03-19T01:23:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.113+0000 7fc599800640 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x55741289a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:23:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.114+0000 7fc57b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:23:45.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.114+0000 7fc59a200640 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 0x557412895c00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x557412889ab0 tx=0x7fc58c017bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da2bc946f8aaf3cd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.115+0000 7fc57b400640 1 -- 172.21.15.69:0/699203062 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c016870 con 0x557412891f20 2024-03-19T01:23:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.115+0000 7fc57b400640 1 -- 172.21.15.69:0/699203062 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7fc58c016a10 con 0x557412891f20 2024-03-19T01:23:45.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.117+0000 7fc59eff7740 1 -- 172.21.15.69:0/699203062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc588028480 con 0x557412891f20 2024-03-19T01:23:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.120+0000 7fc57b400640 1 -- 172.21.15.69:0/699203062 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fc58c016c10 con 0x557412891f20 2024-03-19T01:23:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.120+0000 7fc59eff7740 10 client.27989 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:45.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.121+0000 7fc59eff7740 1 -- 172.21.15.69:0/699203062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x557412908040 con 0x557412891f20 2024-03-19T01:23:45.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.121+0000 7fc57b400640 1 -- 172.21.15.69:0/699203062 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc588028480 con 0x557412891f20 2024-03-19T01:23:45.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.121+0000 7fc57b400640 1 -- 172.21.15.69:0/699203062 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fc58c02a030 con 0x557412891f20 2024-03-19T01:23:45.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.121+0000 7fc59eff7740 10 client.27989 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:45.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.121+0000 7fc59eff7740 10 client.27989 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:23:45.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.121+0000 7fc59eff7740 20 client.27989 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:45.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.121+0000 7fc59eff7740 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5574129087c0 0x55741290abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.122+0000 7fc59eff7740 4 client.27989 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:23:45.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.122+0000 7fc59eff7740 1 -- 172.21.15.69:0/699203062 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55741288abb0 con 0x5574129087c0 2024-03-19T01:23:45.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.123+0000 7fc599800640 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5574129087c0 0x55741290abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.127+0000 7fc599800640 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5574129087c0 0x55741290abe0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f710226eac4c90a6 server_cookie=204f7b3f030b81a7 in_seq=0 out_seq=0 2024-03-19T01:23:45.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.127+0000 7fc57b400640 10 client.27989 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:23:45.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.127+0000 7fc57b400640 1 -- 172.21.15.69:0/699203062 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55741288abb0 con 0x5574129087c0 2024-03-19T01:23:45.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.127+0000 7fc57b400640 10 client.27989 handle_command_reply: tid=0 2024-03-19T01:23:45.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.127+0000 7fc59eff7740 1 client.27989 shutdown 2024-03-19T01:23:45.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.128+0000 7fc59eff7740 1 -- 172.21.15.69:0/699203062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 msgr2=0x557412895c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.128+0000 7fc59eff7740 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 0x557412895c00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x557412889ab0 tx=0x7fc58c017bf0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.128+0000 7fc59eff7740 1 -- 172.21.15.69:0/699203062 shutdown_connections 2024-03-19T01:23:45.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.128+0000 7fc59eff7740 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x5574129087c0 0x55741290abe0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.128+0000 7fc59eff7740 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557412891f20 0x557412895c00 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.128+0000 7fc59eff7740 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557412889340 0x5574128956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.128+0000 7fc57b400640 0 client.27989 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:23:45.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.128+0000 7fc59eff7740 1 --2- 172.21.15.69:0/699203062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557412888a20 0x55741289a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.128+0000 7fc59eff7740 1 -- 172.21.15.69:0/699203062 >> 172.21.15.69:0/699203062 conn(0x557412883150 msgr2=0x557412884d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:45.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.131+0000 7fc59eff7740 1 -- 172.21.15.69:0/699203062 shutdown_connections 2024-03-19T01:23:45.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.131+0000 7fc59eff7740 1 -- 172.21.15.69:0/699203062 wait complete. 2024-03-19T01:23:45.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.131+0000 7fc59eff7740 20 client.27989 trim_cache size 0 max 16384 2024-03-19T01:23:45.132 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:23:45.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 -- 172.21.15.69:0/625121261 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc57c0a29f0 msgr2=0x7fc57c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc57c0a29f0 0x7fc57c0a4eb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009ec0 tx=0x7fc58c009430 comp rx=0 tx=0).stop 2024-03-19T01:23:45.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 -- 172.21.15.69:0/625121261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5940693e0 msgr2=0x7fc594100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5940693e0 0x7fc594100c40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc58800b450 tx=0x7fc58800b910 comp rx=0 tx=0).stop 2024-03-19T01:23:45.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 -- 172.21.15.69:0/625121261 shutdown_connections 2024-03-19T01:23:45.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc57c0a29f0 0x7fc57c0a4eb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc594107870 0x7fc594101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5940693e0 0x7fc594100c40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 --2- 172.21.15.69:0/625121261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc594068a20 0x7fc594100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 -- 172.21.15.69:0/625121261 >> 172.21.15.69:0/625121261 conn(0x7fc594077820 msgr2=0x7fc5940fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:45.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 -- 172.21.15.69:0/625121261 shutdown_connections 2024-03-19T01:23:45.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.139+0000 7fc59c200640 1 -- 172.21.15.69:0/625121261 wait complete. 2024-03-19T01:23:45.146 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/platform/x86/intel/built-in.a 2024-03-19T01:23:45.191 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/platform/x86/wmi.o 2024-03-19T01:23:45.216 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:23:45.256 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/clocksource/i8253.o 2024-03-19T01:23:45.622 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/platform/surface/built-in.a 2024-03-19T01:23:45.626 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/efi.o 2024-03-19T01:23:45.680 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/mailbox/mailbox.o 2024-03-19T01:23:45.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.701+0000 7f17aaa00640 1 -- 172.21.15.69:0/1314119799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17a410d900 msgr2=0x7f17a410dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.702+0000 7f17aaa00640 1 --2- 172.21.15.69:0/1314119799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17a410d900 0x7f17a410dd80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1790009930 tx=0x7f179002bc20 comp rx=0 tx=0).stop 2024-03-19T01:23:45.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.702+0000 7f17aaa00640 1 -- 172.21.15.69:0/1314119799 shutdown_connections 2024-03-19T01:23:45.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.702+0000 7f17aaa00640 1 --2- 172.21.15.69:0/1314119799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a410e4b0 0x7f17a411d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.702+0000 7f17aaa00640 1 --2- 172.21.15.69:0/1314119799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17a410d900 0x7f17a410dd80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.702+0000 7f17aaa00640 1 --2- 172.21.15.69:0/1314119799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a410fe30 0x7f17a4110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.702+0000 7f17aaa00640 1 -- 172.21.15.69:0/1314119799 >> 172.21.15.69:0/1314119799 conn(0x7f17a406c700 msgr2=0x7f17a406cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:45.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.703+0000 7f17aaa00640 1 -- 172.21.15.69:0/1314119799 shutdown_connections 2024-03-19T01:23:45.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.703+0000 7f17aaa00640 1 -- 172.21.15.69:0/1314119799 wait complete. 2024-03-19T01:23:45.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.704+0000 7f17aaa00640 1 Processor -- start 2024-03-19T01:23:45.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.705+0000 7f17aaa00640 1 -- start start 2024-03-19T01:23:45.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.705+0000 7f17aaa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a410d900 0x7f17a411c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.705+0000 7f17aaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a410e4b0 0x7f17a411cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.705+0000 7f17aaa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17a410fe30 0x7f17a4115870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.706+0000 7f17a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a410e4b0 0x7f17a411cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.706+0000 7f17a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a410e4b0 0x7f17a411cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58468/0 (socket says 172.21.15.69:58468) 2024-03-19T01:23:45.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.706+0000 7f17a3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a410d900 0x7f17a411c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.706+0000 7f17a8a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17a410fe30 0x7f17a4115870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.706+0000 7f17a8a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17a410fe30 0x7f17a4115870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55294/0 (socket says 172.21.15.69:55294) 2024-03-19T01:23:45.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.706+0000 7f17a3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a410d900 0x7f17a411c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43314/0 (socket says 172.21.15.69:43314) 2024-03-19T01:23:45.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.706+0000 7f17a3400640 1 -- 172.21.15.69:0/4029389337 learned_addr learned my addr 172.21.15.69:0/4029389337 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:45.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.707+0000 7f17aaa00640 1 -- 172.21.15.69:0/4029389337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4120800 con 0x7f17a410d900 2024-03-19T01:23:45.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.707+0000 7f17aaa00640 1 -- 172.21.15.69:0/4029389337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4120680 con 0x7f17a410e4b0 2024-03-19T01:23:45.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.707+0000 7f17aaa00640 1 -- 172.21.15.69:0/4029389337 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4120980 con 0x7f17a410fe30 2024-03-19T01:23:45.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.708+0000 7f17a3e00640 1 -- 172.21.15.69:0/4029389337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17a410fe30 msgr2=0x7f17a4115870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.708+0000 7f17a3e00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17a410fe30 0x7f17a4115870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.708+0000 7f17a3e00640 1 -- 172.21.15.69:0/4029389337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a410e4b0 msgr2=0x7f17a411cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.708+0000 7f17a3e00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a410e4b0 0x7f17a411cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.708+0000 7f17a3e00640 1 -- 172.21.15.69:0/4029389337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a4116130 con 0x7f17a410d900 2024-03-19T01:23:45.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.708+0000 7f17a3e00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a410d900 0x7f17a411c610 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1794005fd0 tx=0x7f179400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:45.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.709+0000 7f17a0c00640 1 -- 172.21.15.69:0/4029389337 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794009c30 con 0x7f17a410d900 2024-03-19T01:23:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.709+0000 7f17a0c00640 1 -- 172.21.15.69:0/4029389337 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179400be30 con 0x7f17a410d900 2024-03-19T01:23:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.709+0000 7f17a0c00640 1 -- 172.21.15.69:0/4029389337 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794010460 con 0x7f17a410d900 2024-03-19T01:23:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.709+0000 7f17aaa00640 1 -- 172.21.15.69:0/4029389337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a4116420 con 0x7f17a410d900 2024-03-19T01:23:45.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.710+0000 7f17aaa00640 1 -- 172.21.15.69:0/4029389337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a411a3d0 con 0x7f17a410d900 2024-03-19T01:23:45.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.717+0000 7f17ad788740 1 Processor -- start 2024-03-19T01:23:45.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.717+0000 7f17ad788740 1 -- start start 2024-03-19T01:23:45.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.718+0000 7f17ad788740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab5c10 0x55a8c8ab5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.718+0000 7f17ad788740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8ab6530 0x55a8c8abebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.718+0000 7f17ad788740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8abf110 0x55a8c8ac1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.718+0000 7f17ad788740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8c8a1a380 con 0x55a8c8abf110 2024-03-19T01:23:45.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.718+0000 7f17ad788740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8c89c2d20 con 0x55a8c8ab5c10 2024-03-19T01:23:45.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.718+0000 7f17ad788740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a8c8a16420 con 0x55a8c8ab6530 2024-03-19T01:23:45.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.719+0000 7f17a3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8abf110 0x55a8c8ac1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.724+0000 7f17a8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab5c10 0x55a8c8ab5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.724+0000 7f17a8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab5c10 0x55a8c8ab5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58470/0 (socket says 172.21.15.69:58470) 2024-03-19T01:23:45.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.724+0000 7f17a8a00640 1 -- 172.21.15.69:0/122526423 learned_addr learned my addr 172.21.15.69:0/122526423 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:45.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.724+0000 7f17a8a00640 1 -- 172.21.15.69:0/122526423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8ab6530 msgr2=0x55a8c8abebd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:45.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.724+0000 7f17a8a00640 1 --2- 172.21.15.69:0/122526423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8ab6530 0x55a8c8abebd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.724+0000 7f17a8a00640 1 -- 172.21.15.69:0/122526423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8abf110 msgr2=0x55a8c8ac1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:45.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.724+0000 7f17a8a00640 1 --2- 172.21.15.69:0/122526423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8abf110 0x55a8c8ac1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.724+0000 7f17a8a00640 1 -- 172.21.15.69:0/122526423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c872e3f0 con 0x55a8c8ab5c10 2024-03-19T01:23:45.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.724+0000 7f17a8a00640 1 --2- 172.21.15.69:0/122526423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab5c10 0x55a8c8ab5ff0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f179c011440 tx=0x7f179c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c985b2abab16c746 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:45.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.725+0000 7f1799200640 1 -- 172.21.15.69:0/122526423 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c003c30 con 0x55a8c8ab5c10 2024-03-19T01:23:45.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.725+0000 7f1799200640 1 -- 172.21.15.69:0/122526423 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179c003dd0 con 0x55a8c8ab5c10 2024-03-19T01:23:45.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.725+0000 7f1799200640 1 -- 172.21.15.69:0/122526423 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c00fc50 con 0x55a8c8ab5c10 2024-03-19T01:23:45.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.726+0000 7f17a0c00640 1 -- 172.21.15.69:0/4029389337 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f17940035f0 con 0x7f17a410d900 2024-03-19T01:23:45.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.726+0000 7f17ad788740 1 -- 172.21.15.69:0/122526423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab5c10 msgr2=0x55a8c8ab5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.726+0000 7f17ad788740 1 --2- 172.21.15.69:0/122526423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab5c10 0x55a8c8ab5ff0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f179c011440 tx=0x7f179c011900 comp rx=0 tx=0).stop 2024-03-19T01:23:45.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.729+0000 7f17ad788740 1 -- 172.21.15.69:0/122526423 shutdown_connections 2024-03-19T01:23:45.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.729+0000 7f17ad788740 1 --2- 172.21.15.69:0/122526423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8abf110 0x55a8c8ac1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.729+0000 7f17ad788740 1 --2- 172.21.15.69:0/122526423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8ab6530 0x55a8c8abebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.729+0000 7f17a0c00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f17880a29f0 0x7f17880a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.729+0000 7f17ad788740 1 --2- 172.21.15.69:0/122526423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab5c10 0x55a8c8ab5ff0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.729+0000 7f17ad788740 1 -- 172.21.15.69:0/122526423 >> 172.21.15.69:0/122526423 conn(0x55a8c8ab0340 msgr2=0x55a8c8ab0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:45.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.729+0000 7f17a0c00640 1 -- 172.21.15.69:0/4029389337 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f17940cea30 con 0x7f17a410d900 2024-03-19T01:23:45.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.732+0000 7f17a8a00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f17880a29f0 0x7f17880a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.733+0000 7f17ad788740 1 -- 172.21.15.69:0/122526423 shutdown_connections 2024-03-19T01:23:45.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.735+0000 7f17ad788740 1 -- 172.21.15.69:0/122526423 wait complete. 2024-03-19T01:23:45.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.736+0000 7f17a8a00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f17880a29f0 0x7f17880a4eb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f179c009df0 tx=0x7f179c009bf0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:45.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.736+0000 7f17ad788740 1 -- start start 2024-03-19T01:23:45.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.736+0000 7f17ad788740 1 Processor -- start 2024-03-19T01:23:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.740+0000 7f17ad788740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab5c10 0x55a8c8abc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.740+0000 7f17ad788740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab6530 0x55a8c8abc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.744+0000 7f17ad788740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8abf110 0x55a8c8abcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.744+0000 7f17ad788740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8c8ac1f00 con 0x55a8c8ab5c10 2024-03-19T01:23:45.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.744+0000 7f17ad788740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8c89d6160 con 0x55a8c8ab6530 2024-03-19T01:23:45.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.744+0000 7f17ad788740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a8c8ac2080 con 0x55a8c8abf110 2024-03-19T01:23:45.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.744+0000 7f17ad788740 10 client.0 _ll_register_callbacks cb 0x55a8c8a12170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:45.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.744+0000 7f17a3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab5c10 0x55a8c8abc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.745+0000 7f17a3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab5c10 0x55a8c8abc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43334/0 (socket says 172.21.15.69:43334) 2024-03-19T01:23:45.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.745+0000 7f17a3e00640 1 -- 172.21.15.69:0/3281080215 learned_addr learned my addr 172.21.15.69:0/3281080215 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.745+0000 7f17a3e00640 1 -- 172.21.15.69:0/3281080215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8abf110 msgr2=0x55a8c8abcad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.745+0000 7f17a3e00640 1 --2- 172.21.15.69:0/3281080215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8abf110 0x55a8c8abcad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.745+0000 7f17a3e00640 1 -- 172.21.15.69:0/3281080215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab6530 msgr2=0x55a8c8abc590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.745+0000 7f17a3e00640 1 --2- 172.21.15.69:0/3281080215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab6530 0x55a8c8abc590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.745+0000 7f17a3e00640 1 -- 172.21.15.69:0/3281080215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c87f87d0 con 0x55a8c8ab5c10 2024-03-19T01:23:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.746+0000 7f17a3e00640 1 --2- 172.21.15.69:0/3281080215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab5c10 0x55a8c8abc050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1794009610 tx=0x7f179409b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe914151ea8481b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.746+0000 7f178d600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:23:45.747 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-stripe.o 2024-03-19T01:23:45.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.748+0000 7f178d600640 1 -- 172.21.15.69:0/3281080215 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940a3420 con 0x55a8c8ab5c10 2024-03-19T01:23:45.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.748+0000 7f178d600640 1 -- 172.21.15.69:0/3281080215 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f17940972f0 con 0x55a8c8ab5c10 2024-03-19T01:23:45.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.748+0000 7f17ad788740 1 -- 172.21.15.69:0/3281080215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f179c003dd0 con 0x55a8c8ab5c10 2024-03-19T01:23:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.748+0000 7f178d600640 1 -- 172.21.15.69:0/3281080215 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f179409a0d0 con 0x55a8c8ab5c10 2024-03-19T01:23:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.748+0000 7f17ad788740 10 client.17619 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:45.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.748+0000 7f17ad788740 1 -- 172.21.15.69:0/3281080215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55a8c8ace190 con 0x55a8c8ab5c10 2024-03-19T01:23:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.749+0000 7f178d600640 1 -- 172.21.15.69:0/3281080215 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179c003dd0 con 0x55a8c8ab5c10 2024-03-19T01:23:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.749+0000 7f178d600640 1 -- 172.21.15.69:0/3281080215 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f1794099b40 con 0x55a8c8ab5c10 2024-03-19T01:23:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.749+0000 7f17ad788740 10 client.17619 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.749+0000 7f17ad788740 10 client.17619 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:23:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.749+0000 7f17ad788740 20 client.17619 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.749+0000 7f17ad788740 1 --2- 172.21.15.69:0/3281080215 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55a8c8acf5a0 0x55a8c8ad1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.749+0000 7f17ad788740 4 client.17619 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:23:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.749+0000 7f17ad788740 1 -- 172.21.15.69:0/3281080215 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8c8a278c0 con 0x55a8c8acf5a0 2024-03-19T01:23:45.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.749+0000 7f17a3400640 1 --2- 172.21.15.69:0/3281080215 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55a8c8acf5a0 0x55a8c8ad1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.751+0000 7f17a3400640 1 --2- 172.21.15.69:0/3281080215 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55a8c8acf5a0 0x55a8c8ad1a40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=922dbd0e28f93188 server_cookie=b87c2958dd0f4927 in_seq=0 out_seq=0 2024-03-19T01:23:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.751+0000 7f178d600640 10 client.17619 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:23:45.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.758+0000 7f178d600640 1 -- 172.21.15.69:0/3281080215 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a8c8a278c0 con 0x55a8c8acf5a0 2024-03-19T01:23:45.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.758+0000 7f178d600640 10 client.17619 handle_command_reply: tid=0 2024-03-19T01:23:45.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.758+0000 7f17ad788740 1 client.17619 shutdown 2024-03-19T01:23:45.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.761+0000 7f17ad788740 1 -- 172.21.15.69:0/3281080215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab5c10 msgr2=0x55a8c8abc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.761+0000 7f17ad788740 1 --2- 172.21.15.69:0/3281080215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab5c10 0x55a8c8abc050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1794009610 tx=0x7f179409b440 comp rx=0 tx=0).stop 2024-03-19T01:23:45.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.761+0000 7f17ad788740 1 -- 172.21.15.69:0/3281080215 shutdown_connections 2024-03-19T01:23:45.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.761+0000 7f17ad788740 1 --2- 172.21.15.69:0/3281080215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8abf110 0x55a8c8abcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.761+0000 7f17ad788740 1 --2- 172.21.15.69:0/3281080215 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55a8c8acf5a0 0x55a8c8ad1a40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.761+0000 7f17ad788740 1 --2- 172.21.15.69:0/3281080215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab6530 0x55a8c8abc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.761+0000 7f17ad788740 1 --2- 172.21.15.69:0/3281080215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab5c10 0x55a8c8abc050 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.761+0000 7f178d600640 0 client.17619 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:23:45.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.761+0000 7f17ad788740 1 -- 172.21.15.69:0/3281080215 >> 172.21.15.69:0/3281080215 conn(0x55a8c8ab0340 msgr2=0x55a8c8ab2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:45.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.762+0000 7f17ad788740 1 -- 172.21.15.69:0/3281080215 shutdown_connections 2024-03-19T01:23:45.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.765+0000 7f17ad788740 1 -- 172.21.15.69:0/3281080215 wait complete. 2024-03-19T01:23:45.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.765+0000 7f17ad788740 20 client.17619 trim_cache size 0 max 16384 2024-03-19T01:23:45.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.823+0000 7f17ad788740 1 Processor -- start 2024-03-19T01:23:45.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.827+0000 7f17ad788740 1 -- start start 2024-03-19T01:23:45.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.827+0000 7f17ad788740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8ab6530 0x55a8c8acad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.827+0000 7f17ad788740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8abf110 0x55a8c8abc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.827+0000 7f17ad788740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab1ea0 0x55a8c8ad7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.827+0000 7f17ad788740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1784010080 con 0x55a8c8abf110 2024-03-19T01:23:45.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.827+0000 7f17ad788740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1784011bc0 con 0x55a8c8ab1ea0 2024-03-19T01:23:45.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.827+0000 7f17ad788740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1784010750 con 0x55a8c8ab6530 2024-03-19T01:23:45.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.828+0000 7f17a3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8ab6530 0x55a8c8acad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.828+0000 7f17a3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8ab6530 0x55a8c8acad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:55300/0 (socket says 172.21.15.69:55300) 2024-03-19T01:23:45.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.828+0000 7f17a3e00640 1 -- 172.21.15.69:0/1960950514 learned_addr learned my addr 172.21.15.69:0/1960950514 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:45.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.828+0000 7f17a3400640 1 --2- 172.21.15.69:0/1960950514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8abf110 0x55a8c8abc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.828+0000 7f17a8a00640 1 --2- 172.21.15.69:0/1960950514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab1ea0 0x55a8c8ad7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.828+0000 7f17a3e00640 1 -- 172.21.15.69:0/1960950514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab1ea0 msgr2=0x55a8c8ad7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:45.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.828+0000 7f17a3e00640 1 --2- 172.21.15.69:0/1960950514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab1ea0 0x55a8c8ad7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.828+0000 7f17a3e00640 1 -- 172.21.15.69:0/1960950514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8abf110 msgr2=0x55a8c8abc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:45.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.828+0000 7f17a3e00640 1 --2- 172.21.15.69:0/1960950514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8abf110 0x55a8c8abc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.829+0000 7f17a3e00640 1 -- 172.21.15.69:0/1960950514 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c8ab0170 con 0x55a8c8ab6530 2024-03-19T01:23:45.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.829+0000 7f17a3e00640 1 --2- 172.21.15.69:0/1960950514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8ab6530 0x55a8c8acad70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f17940b5040 tx=0x7f179409ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4f51c049c3423ca server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:45.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.832+0000 7f178fe00640 1 -- 172.21.15.69:0/1960950514 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179409a270 con 0x55a8c8ab6530 2024-03-19T01:23:45.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.832+0000 7f178fe00640 1 -- 172.21.15.69:0/1960950514 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17940972f0 con 0x55a8c8ab6530 2024-03-19T01:23:45.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.832+0000 7f178fe00640 1 -- 172.21.15.69:0/1960950514 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794079aa0 con 0x55a8c8ab6530 2024-03-19T01:23:45.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.833+0000 7f17ad788740 1 -- 172.21.15.69:0/1960950514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8ab6530 msgr2=0x55a8c8acad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.833+0000 7f17ad788740 1 --2- 172.21.15.69:0/1960950514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8ab6530 0x55a8c8acad70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f17940b5040 tx=0x7f179409ef50 comp rx=0 tx=0).stop 2024-03-19T01:23:45.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.833+0000 7f17ad788740 1 -- 172.21.15.69:0/1960950514 shutdown_connections 2024-03-19T01:23:45.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.833+0000 7f17ad788740 1 --2- 172.21.15.69:0/1960950514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ab1ea0 0x55a8c8ad7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.833+0000 7f17ad788740 1 --2- 172.21.15.69:0/1960950514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8abf110 0x55a8c8abc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.833+0000 7f17ad788740 1 --2- 172.21.15.69:0/1960950514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8ab6530 0x55a8c8acad70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.833+0000 7f17ad788740 1 -- 172.21.15.69:0/1960950514 >> 172.21.15.69:0/1960950514 conn(0x55a8c8ab0340 msgr2=0x55a8c8ab1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:45.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.833+0000 7f17ad788740 1 -- 172.21.15.69:0/1960950514 shutdown_connections 2024-03-19T01:23:45.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.833+0000 7f17ad788740 1 -- 172.21.15.69:0/1960950514 wait complete. 2024-03-19T01:23:45.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.834+0000 7f17ad788740 1 -- start start 2024-03-19T01:23:45.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.834+0000 7f17ad788740 1 Processor -- start 2024-03-19T01:23:45.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.835+0000 7f17ad788740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab1ea0 0x55a8c8ac6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.835+0000 7f17ad788740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8abf110 0x55a8c8ac7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.835+0000 7f17a3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab1ea0 0x55a8c8ac6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.835+0000 7f17a3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab1ea0 0x55a8c8ac6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43350/0 (socket says 172.21.15.69:43350) 2024-03-19T01:23:45.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.838+0000 7f17a3e00640 1 -- 172.21.15.69:0/1396398588 learned_addr learned my addr 172.21.15.69:0/1396398588 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:45.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.838+0000 7f17ad788740 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ac3310 0x55a8c8ac3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.838+0000 7f17ad788740 1 -- 172.21.15.69:0/1396398588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17840114f0 con 0x55a8c8ab1ea0 2024-03-19T01:23:45.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.838+0000 7f17ad788740 1 -- 172.21.15.69:0/1396398588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1784010e20 con 0x55a8c8ac3310 2024-03-19T01:23:45.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.838+0000 7f17ad788740 1 -- 172.21.15.69:0/1396398588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f178400e060 con 0x55a8c8abf110 2024-03-19T01:23:45.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.838+0000 7f17ad788740 10 client.0 _ll_register_callbacks cb 0x7f17940038e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:45.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.838+0000 7f17a3400640 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8abf110 0x55a8c8ac7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.838+0000 7f17a8a00640 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ac3310 0x55a8c8ac3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.838+0000 7f17a3400640 1 -- 172.21.15.69:0/1396398588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ac3310 msgr2=0x55a8c8ac3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:45.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.838+0000 7f17a3400640 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ac3310 0x55a8c8ac3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.838+0000 7f17a3400640 1 -- 172.21.15.69:0/1396398588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab1ea0 msgr2=0x55a8c8ac6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:45.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.839+0000 7f17a3400640 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab1ea0 0x55a8c8ac6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.839+0000 7f17a3400640 1 -- 172.21.15.69:0/1396398588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c8aafba0 con 0x55a8c8abf110 2024-03-19T01:23:45.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.839+0000 7f17a3e00640 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab1ea0 0x55a8c8ac6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:23:45.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.840+0000 7f17a3400640 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8abf110 0x55a8c8ac7420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1790000c00 tx=0x7f1790009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2397a5ea63bd5b72 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:45.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.840+0000 7f178d600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:23:45.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.840+0000 7f178d600640 1 -- 172.21.15.69:0/1396398588 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790046d00 con 0x55a8c8abf110 2024-03-19T01:23:45.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.840+0000 7f178d600640 1 -- 172.21.15.69:0/1396398588 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f1790004040 con 0x55a8c8abf110 2024-03-19T01:23:45.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.840+0000 7f17ad788740 1 -- 172.21.15.69:0/1396398588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17940972f0 con 0x55a8c8abf110 2024-03-19T01:23:45.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.841+0000 7f178d600640 1 -- 172.21.15.69:0/1396398588 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f1790057050 con 0x55a8c8abf110 2024-03-19T01:23:45.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.841+0000 7f17ad788740 10 client.27937 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:45.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.841+0000 7f17ad788740 1 -- 172.21.15.69:0/1396398588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55a8c8b365b0 con 0x55a8c8abf110 2024-03-19T01:23:45.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.841+0000 7f178d600640 1 -- 172.21.15.69:0/1396398588 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17940972f0 con 0x55a8c8abf110 2024-03-19T01:23:45.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.841+0000 7f178d600640 1 -- 172.21.15.69:0/1396398588 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f1790046550 con 0x55a8c8abf110 2024-03-19T01:23:45.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.841+0000 7f17ad788740 10 client.27937 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:45.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.841+0000 7f17ad788740 10 client.27937 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:23:45.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.841+0000 7f17ad788740 20 client.27937 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:45.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.841+0000 7f17ad788740 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55a8c8b36db0 0x55a8c8b391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:45.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.842+0000 7f17ad788740 4 client.27937 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:23:45.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.842+0000 7f17ad788740 1 -- 172.21.15.69:0/1396398588 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8c8a21c10 con 0x55a8c8b36db0 2024-03-19T01:23:45.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.842+0000 7f17a3e00640 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55a8c8b36db0 0x55a8c8b391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:45.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.842+0000 7f17a3e00640 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55a8c8b36db0 0x55a8c8b391d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2492d8d13766483a server_cookie=9ea657e4fb11c3e in_seq=0 out_seq=0 2024-03-19T01:23:45.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.842+0000 7f178d600640 10 client.27937 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:23:45.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.843+0000 7f178d600640 1 -- 172.21.15.69:0/1396398588 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8c8a21c10 con 0x55a8c8b36db0 2024-03-19T01:23:45.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.843+0000 7f178d600640 10 client.27937 handle_command_reply: tid=0 2024-03-19T01:23:45.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.843+0000 7f17ad788740 1 client.27937 shutdown 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f17ad788740 1 -- 172.21.15.69:0/1396398588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8abf110 msgr2=0x55a8c8ac7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f17ad788740 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8abf110 0x55a8c8ac7420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1790000c00 tx=0x7f1790009650 comp rx=0 tx=0).stop 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f17ad788740 1 -- 172.21.15.69:0/1396398588 shutdown_connections 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f17ad788740 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55a8c8b36db0 0x55a8c8b391d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f17ad788740 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8c8ac3310 0x55a8c8ac3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f17ad788740 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a8c8abf110 0x55a8c8ac7420 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f17ad788740 1 --2- 172.21.15.69:0/1396398588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8c8ab1ea0 0x55a8c8ac6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f178d600640 0 client.27937 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f17ad788740 1 -- 172.21.15.69:0/1396398588 >> 172.21.15.69:0/1396398588 conn(0x55a8c8ab0340 msgr2=0x55a8c8aaf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f17ad788740 1 -- 172.21.15.69:0/1396398588 shutdown_connections 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f17ad788740 1 -- 172.21.15.69:0/1396398588 wait complete. 2024-03-19T01:23:45.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.848+0000 7f17ad788740 20 client.27937 trim_cache size 0 max 16384 2024-03-19T01:23:45.868 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:23:45.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.871+0000 7f17aaa00640 1 -- 172.21.15.69:0/4029389337 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f17880a29f0 msgr2=0x7f17880a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.871+0000 7f17aaa00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f17880a29f0 0x7f17880a4eb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f179c009df0 tx=0x7f179c009bf0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.871+0000 7f17aaa00640 1 -- 172.21.15.69:0/4029389337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a410d900 msgr2=0x7f17a411c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:45.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.871+0000 7f17aaa00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a410d900 0x7f17a411c610 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1794005fd0 tx=0x7f179400dd20 comp rx=0 tx=0).stop 2024-03-19T01:23:45.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.871+0000 7f17aaa00640 1 -- 172.21.15.69:0/4029389337 shutdown_connections 2024-03-19T01:23:45.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.871+0000 7f17aaa00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17a410fe30 0x7f17a4115870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.871+0000 7f17aaa00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f17880a29f0 0x7f17880a4eb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.871+0000 7f17aaa00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17a410e4b0 0x7f17a411cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:45.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.871+0000 7f17aaa00640 1 --2- 172.21.15.69:0/4029389337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a410d900 0x7f17a411c610 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1794005fd0 tx=0x7f179400dd20 comp rx=0 tx=0).stop 2024-03-19T01:23:45.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.871+0000 7f17aaa00640 1 -- 172.21.15.69:0/4029389337 >> 172.21.15.69:0/4029389337 conn(0x7f17a406c700 msgr2=0x7f17a4070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:45.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.873+0000 7f17aaa00640 1 -- 172.21.15.69:0/4029389337 shutdown_connections 2024-03-19T01:23:45.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:45.873+0000 7f17aaa00640 1 -- 172.21.15.69:0/4029389337 wait complete. 2024-03-19T01:23:45.904 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:23:46.010 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/clocksource/built-in.a 2024-03-19T01:23:46.054 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/secureboot.o 2024-03-19T01:23:46.127 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/usbhid/hiddev.o 2024-03-19T01:23:46.153 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/gen6_renderstate.o 2024-03-19T01:23:46.374 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/psci/built-in.a 2024-03-19T01:23:46.422 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-core.o 2024-03-19T01:23:46.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.442+0000 7f8c8a600640 1 -- 172.21.15.69:0/2646044365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c84108a00 msgr2=0x7f8c84108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:46.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.442+0000 7f8c8a600640 1 --2- 172.21.15.69:0/2646044365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c84108a00 0x7f8c84108de0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009900 tx=0x7f8c7c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:23:46.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.445+0000 7f8c8a600640 1 -- 172.21.15.69:0/2646044365 shutdown_connections 2024-03-19T01:23:46.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.445+0000 7f8c8a600640 1 --2- 172.21.15.69:0/2646044365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84114970 0x7f8c84116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.445+0000 7f8c8a600640 1 --2- 172.21.15.69:0/2646044365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c84109320 0x7f8c84114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.445+0000 7f8c8a600640 1 --2- 172.21.15.69:0/2646044365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c84108a00 0x7f8c84108de0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.445+0000 7f8c8a600640 1 -- 172.21.15.69:0/2646044365 >> 172.21.15.69:0/2646044365 conn(0x7f8c8406c900 msgr2=0x7f8c8406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:46.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.445+0000 7f8c8a600640 1 -- 172.21.15.69:0/2646044365 shutdown_connections 2024-03-19T01:23:46.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.445+0000 7f8c8a600640 1 -- 172.21.15.69:0/2646044365 wait complete. 2024-03-19T01:23:46.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.447+0000 7f8c8a600640 1 Processor -- start 2024-03-19T01:23:46.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.447+0000 7f8c8a600640 1 -- start start 2024-03-19T01:23:46.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.448+0000 7f8c8a600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c84108a00 0x7f8c8410be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.448+0000 7f8c8a600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84109320 0x7f8c8410a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.448+0000 7f8c8a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c84114970 0x7f8c8410a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.448+0000 7f8c8a600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8411a3b0 con 0x7f8c84109320 2024-03-19T01:23:46.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.448+0000 7f8c8a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8411a230 con 0x7f8c84114970 2024-03-19T01:23:46.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.448+0000 7f8c8a600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8411a530 con 0x7f8c84108a00 2024-03-19T01:23:46.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.448+0000 7f8c83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c84108a00 0x7f8c8410be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.448+0000 7f8c83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c84108a00 0x7f8c8410be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45196/0 (socket says 172.21.15.69:45196) 2024-03-19T01:23:46.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.448+0000 7f8c83400640 1 -- 172.21.15.69:0/815405107 learned_addr learned my addr 172.21.15.69:0/815405107 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:46.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.448+0000 7f8c83e00640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c84114970 0x7f8c8410a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.448+0000 7f8c82a00640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84109320 0x7f8c8410a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.449+0000 7f8c83400640 1 -- 172.21.15.69:0/815405107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c84114970 msgr2=0x7f8c8410a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:46.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.449+0000 7f8c83400640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c84114970 0x7f8c8410a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.449+0000 7f8c83400640 1 -- 172.21.15.69:0/815405107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84109320 msgr2=0x7f8c8410a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:46.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.449+0000 7f8c83400640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84109320 0x7f8c8410a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.449+0000 7f8c83400640 1 -- 172.21.15.69:0/815405107 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c841a4a50 con 0x7f8c84108a00 2024-03-19T01:23:46.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.449+0000 7f8c83e00640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c84114970 0x7f8c8410a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:23:46.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.449+0000 7f8c82a00640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84109320 0x7f8c8410a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:23:46.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.449+0000 7f8c83400640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c84108a00 0x7f8c8410be90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c02c240 tx=0x7f8c7c004390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:46.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.453+0000 7f8c7be00640 1 -- 172.21.15.69:0/815405107 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c043020 con 0x7f8c84108a00 2024-03-19T01:23:46.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.453+0000 7f8c7be00640 1 -- 172.21.15.69:0/815405107 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c0054e0 con 0x7f8c84108a00 2024-03-19T01:23:46.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.453+0000 7f8c7be00640 1 -- 172.21.15.69:0/815405107 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c003c00 con 0x7f8c84108a00 2024-03-19T01:23:46.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.453+0000 7f8c8a600640 1 -- 172.21.15.69:0/815405107 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c841a4ce0 con 0x7f8c84108a00 2024-03-19T01:23:46.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.453+0000 7f8c8a600640 1 -- 172.21.15.69:0/815405107 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c841a51e0 con 0x7f8c84108a00 2024-03-19T01:23:46.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.460+0000 7f8c7be00640 1 -- 172.21.15.69:0/815405107 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f8c7c046020 con 0x7f8c84108a00 2024-03-19T01:23:46.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.462+0000 7f8c7be00640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8c680a2a30 0x7f8c680a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.465+0000 7f8c82a00640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8c680a2a30 0x7f8c680a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.466+0000 7f8c7be00640 1 -- 172.21.15.69:0/815405107 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f8c7c0f1300 con 0x7f8c84108a00 2024-03-19T01:23:46.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.469+0000 7f8c82a00640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8c680a2a30 0x7f8c680a4ef0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8c8410ba00 tx=0x7f8c74009990 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:46.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.470+0000 7f8c8d236740 1 Processor -- start 2024-03-19T01:23:46.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.474+0000 7f8c8d236740 1 -- start start 2024-03-19T01:23:46.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.474+0000 7f8c8d236740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc57a20 0x56339dc57e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.474+0000 7f8c8d236740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc58340 0x56339dc609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.474+0000 7f8c8d236740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc60f20 0x56339dc63310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.474+0000 7f8c8d236740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56339dbbc380 con 0x56339dc58340 2024-03-19T01:23:46.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.474+0000 7f8c8d236740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56339db64d20 con 0x56339dc60f20 2024-03-19T01:23:46.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.474+0000 7f8c8d236740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56339dbb8420 con 0x56339dc57a20 2024-03-19T01:23:46.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc57a20 0x56339dc57e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc57a20 0x56339dc57e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45202/0 (socket says 172.21.15.69:45202) 2024-03-19T01:23:46.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c83400640 1 -- 172.21.15.69:0/1932889704 learned_addr learned my addr 172.21.15.69:0/1932889704 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:46.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c83e00640 1 --2- 172.21.15.69:0/1932889704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc60f20 0x56339dc63310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c82a00640 1 --2- 172.21.15.69:0/1932889704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc58340 0x56339dc609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c83400640 1 -- 172.21.15.69:0/1932889704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc60f20 msgr2=0x56339dc63310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:46.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c83400640 1 --2- 172.21.15.69:0/1932889704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc60f20 0x56339dc63310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c83400640 1 -- 172.21.15.69:0/1932889704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc58340 msgr2=0x56339dc609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:46.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c83400640 1 --2- 172.21.15.69:0/1932889704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc58340 0x56339dc609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c83400640 1 -- 172.21.15.69:0/1932889704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56339d8d03f0 con 0x56339dc57a20 2024-03-19T01:23:46.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c82a00640 1 --2- 172.21.15.69:0/1932889704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc58340 0x56339dc609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:46.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.475+0000 7f8c83400640 1 --2- 172.21.15.69:0/1932889704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc57a20 0x56339dc57e00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0c59f0 tx=0x7f8c7c0bd610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4883bb915f10fd8a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:46.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.476+0000 7f8c78c00640 1 -- 172.21.15.69:0/1932889704 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0d5020 con 0x56339dc57a20 2024-03-19T01:23:46.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.476+0000 7f8c78c00640 1 -- 172.21.15.69:0/1932889704 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c004c80 con 0x56339dc57a20 2024-03-19T01:23:46.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.476+0000 7f8c78c00640 1 -- 172.21.15.69:0/1932889704 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0c5f20 con 0x56339dc57a20 2024-03-19T01:23:46.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.477+0000 7f8c8d236740 1 -- 172.21.15.69:0/1932889704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc57a20 msgr2=0x56339dc57e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:46.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.477+0000 7f8c8d236740 1 --2- 172.21.15.69:0/1932889704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc57a20 0x56339dc57e00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0c59f0 tx=0x7f8c7c0bd610 comp rx=0 tx=0).stop 2024-03-19T01:23:46.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.477+0000 7f8c8d236740 1 -- 172.21.15.69:0/1932889704 shutdown_connections 2024-03-19T01:23:46.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.477+0000 7f8c8d236740 1 --2- 172.21.15.69:0/1932889704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc60f20 0x56339dc63310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.477+0000 7f8c8d236740 1 --2- 172.21.15.69:0/1932889704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc58340 0x56339dc609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.477+0000 7f8c8d236740 1 --2- 172.21.15.69:0/1932889704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc57a20 0x56339dc57e00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.477+0000 7f8c8d236740 1 -- 172.21.15.69:0/1932889704 >> 172.21.15.69:0/1932889704 conn(0x56339dc52150 msgr2=0x56339dc52530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:46.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.481+0000 7f8c8d236740 1 -- 172.21.15.69:0/1932889704 shutdown_connections 2024-03-19T01:23:46.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.481+0000 7f8c8d236740 1 -- 172.21.15.69:0/1932889704 wait complete. 2024-03-19T01:23:46.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.481+0000 7f8c8d236740 1 -- start start 2024-03-19T01:23:46.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.482+0000 7f8c8d236740 1 Processor -- start 2024-03-19T01:23:46.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.482+0000 7f8c8d236740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 0x56339dc5df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.483+0000 7f8c8d236740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 0x56339dc5e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.483+0000 7f8c8d236740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc60f20 0x56339dc5e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.483+0000 7f8c8d236740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56339dc63ad0 con 0x56339dc57a20 2024-03-19T01:23:46.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.483+0000 7f8c83400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 0x56339dc5df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.483+0000 7f8c83400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 0x56339dc5df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34754/0 (socket says 172.21.15.69:34754) 2024-03-19T01:23:46.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.483+0000 7f8c83400640 1 -- 172.21.15.69:0/577984809 learned_addr learned my addr 172.21.15.69:0/577984809 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:46.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.483+0000 7f8c8d236740 1 -- 172.21.15.69:0/577984809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56339db78160 con 0x56339dc58340 2024-03-19T01:23:46.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.483+0000 7f8c8d236740 1 -- 172.21.15.69:0/577984809 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56339dc63c50 con 0x56339dc60f20 2024-03-19T01:23:46.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.483+0000 7f8c8d236740 10 client.0 _ll_register_callbacks cb 0x56339dbb4170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:46.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.483+0000 7f8c83e00640 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc60f20 0x56339dc5e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.484+0000 7f8c83400640 1 -- 172.21.15.69:0/577984809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc60f20 msgr2=0x56339dc5e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.484+0000 7f8c83400640 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc60f20 0x56339dc5e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.484+0000 7f8c83400640 1 -- 172.21.15.69:0/577984809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 msgr2=0x56339dc5e4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.484+0000 7f8c82a00640 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 0x56339dc5e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.484+0000 7f8c83400640 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 0x56339dc5e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.484+0000 7f8c83400640 1 -- 172.21.15.69:0/577984809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56339d99a7d0 con 0x56339dc57a20 2024-03-19T01:23:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.484+0000 7f8c83400640 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 0x56339dc5df60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c02bc70 tx=0x7f8c7c02cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d780f32747da92a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.485+0000 7f8c6d600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:23:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.485+0000 7f8c6d600640 1 -- 172.21.15.69:0/577984809 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0d5070 con 0x56339dc57a20 2024-03-19T01:23:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.485+0000 7f8c6d600640 1 -- 172.21.15.69:0/577984809 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f8c7c0ee8c0 con 0x56339dc57a20 2024-03-19T01:23:46.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.485+0000 7f8c82a00640 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 0x56339dc5e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:46.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.485+0000 7f8c8d236740 1 -- 172.21.15.69:0/577984809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c7c004c80 con 0x56339dc57a20 2024-03-19T01:23:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.487+0000 7f8c6d600640 1 -- 172.21.15.69:0/577984809 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c0bd780 con 0x56339dc57a20 2024-03-19T01:23:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.487+0000 7f8c8d236740 10 client.17634 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.487+0000 7f8c8d236740 1 -- 172.21.15.69:0/577984809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x56339dc63ad0 con 0x56339dc57a20 2024-03-19T01:23:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.488+0000 7f8c6d600640 1 -- 172.21.15.69:0/577984809 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c004c80 con 0x56339dc57a20 2024-03-19T01:23:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.488+0000 7f8c6d600640 1 -- 172.21.15.69:0/577984809 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f8c7c0d1b60 con 0x56339dc57a20 2024-03-19T01:23:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.488+0000 7f8c8d236740 10 client.17634 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.488+0000 7f8c8d236740 10 client.17634 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:23:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.488+0000 7f8c8d236740 20 client.17634 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.488+0000 7f8c8d236740 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56339dc70e50 0x56339dc732f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.488+0000 7f8c8d236740 4 client.17634 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:23:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.488+0000 7f8c8d236740 1 -- 172.21.15.69:0/577984809 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56339dbc98c0 con 0x56339dc70e50 2024-03-19T01:23:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.489+0000 7f8c82a00640 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56339dc70e50 0x56339dc732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.490+0000 7f8c6d600640 10 client.17634 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:23:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.490+0000 7f8c82a00640 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56339dc70e50 0x56339dc732f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a62f320bb043117a server_cookie=a69ca58a520136a in_seq=0 out_seq=0 2024-03-19T01:23:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.493+0000 7f8c6d600640 1 -- 172.21.15.69:0/577984809 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56339dbc98c0 con 0x56339dc70e50 2024-03-19T01:23:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.493+0000 7f8c6d600640 10 client.17634 handle_command_reply: tid=0 2024-03-19T01:23:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.494+0000 7f8c8d236740 1 client.17634 shutdown 2024-03-19T01:23:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.498+0000 7f8c8d236740 1 -- 172.21.15.69:0/577984809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 msgr2=0x56339dc5df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.498+0000 7f8c8d236740 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 0x56339dc5df60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c02bc70 tx=0x7f8c7c02cb90 comp rx=0 tx=0).stop 2024-03-19T01:23:46.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.501+0000 7f8c8d236740 1 -- 172.21.15.69:0/577984809 shutdown_connections 2024-03-19T01:23:46.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.501+0000 7f8c8d236740 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56339dc70e50 0x56339dc732f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.501+0000 7f8c8d236740 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc60f20 0x56339dc5e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.501+0000 7f8c8d236740 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 0x56339dc5e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.501+0000 7f8c8d236740 1 --2- 172.21.15.69:0/577984809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 0x56339dc5df60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.501+0000 7f8c6d600640 0 client.17634 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:23:46.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.501+0000 7f8c8d236740 1 -- 172.21.15.69:0/577984809 >> 172.21.15.69:0/577984809 conn(0x56339dc52150 msgr2=0x56339dc53ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:46.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.503+0000 7f8c8d236740 1 -- 172.21.15.69:0/577984809 shutdown_connections 2024-03-19T01:23:46.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.503+0000 7f8c8d236740 1 -- 172.21.15.69:0/577984809 wait complete. 2024-03-19T01:23:46.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.503+0000 7f8c8d236740 20 client.17634 trim_cache size 0 max 16384 2024-03-19T01:23:46.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:46 smithi086 ceph-mon[66829]: pgmap v409: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-19T01:23:46.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:46 smithi069 ceph-mon[31727]: pgmap v409: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-19T01:23:46.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:46 smithi163 ceph-mon[67428]: pgmap v409: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-19T01:23:46.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.563+0000 7f8c8d236740 1 Processor -- start 2024-03-19T01:23:46.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.564+0000 7f8c8d236740 1 -- start start 2024-03-19T01:23:46.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.564+0000 7f8c8d236740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc57a20 0x56339dc60690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.564+0000 7f8c8d236740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc58340 0x56339dc6c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.565+0000 7f8c83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc57a20 0x56339dc60690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.565+0000 7f8c83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc57a20 0x56339dc60690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36044/0 (socket says 172.21.15.69:36044) 2024-03-19T01:23:46.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.565+0000 7f8c83400640 1 -- 172.21.15.69:0/2828384358 learned_addr learned my addr 172.21.15.69:0/2828384358 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:46.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.565+0000 7f8c8d236740 1 --2- 172.21.15.69:0/2828384358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc60f20 0x56339dc51050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.565+0000 7f8c8d236740 1 -- 172.21.15.69:0/2828384358 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841c65e0 con 0x56339dc60f20 2024-03-19T01:23:46.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.565+0000 7f8c8d236740 1 -- 172.21.15.69:0/2828384358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841c8120 con 0x56339dc57a20 2024-03-19T01:23:46.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.565+0000 7f8c8d236740 1 -- 172.21.15.69:0/2828384358 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841c6cb0 con 0x56339dc58340 2024-03-19T01:23:46.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.565+0000 7f8c82a00640 1 --2- 172.21.15.69:0/2828384358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc58340 0x56339dc6c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.566+0000 7f8c83400640 1 -- 172.21.15.69:0/2828384358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc58340 msgr2=0x56339dc6c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:46.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.566+0000 7f8c83400640 1 --2- 172.21.15.69:0/2828384358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc58340 0x56339dc6c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.566+0000 7f8c83400640 1 -- 172.21.15.69:0/2828384358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc60f20 msgr2=0x56339dc51050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:46.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.566+0000 7f8c83400640 1 --2- 172.21.15.69:0/2828384358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc60f20 0x56339dc51050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.566+0000 7f8c83400640 1 -- 172.21.15.69:0/2828384358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56339dc5c4b0 con 0x56339dc57a20 2024-03-19T01:23:46.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.566+0000 7f8c83400640 1 --2- 172.21.15.69:0/2828384358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc57a20 0x56339dc60690 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0b9b50 tx=0x7f8c7c0bb980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87e9045f86c4042 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:46.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.566+0000 7f8c6fe00640 1 -- 172.21.15.69:0/2828384358 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0d5040 con 0x56339dc57a20 2024-03-19T01:23:46.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.567+0000 7f8c6fe00640 1 -- 172.21.15.69:0/2828384358 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c004c80 con 0x56339dc57a20 2024-03-19T01:23:46.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.567+0000 7f8c6fe00640 1 -- 172.21.15.69:0/2828384358 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0c7080 con 0x56339dc57a20 2024-03-19T01:23:46.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.568+0000 7f8c8d236740 1 -- 172.21.15.69:0/2828384358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc57a20 msgr2=0x56339dc60690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:46.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.568+0000 7f8c8d236740 1 --2- 172.21.15.69:0/2828384358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc57a20 0x56339dc60690 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0b9b50 tx=0x7f8c7c0bb980 comp rx=0 tx=0).stop 2024-03-19T01:23:46.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.571+0000 7f8c8d236740 1 -- 172.21.15.69:0/2828384358 shutdown_connections 2024-03-19T01:23:46.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.571+0000 7f8c8d236740 1 --2- 172.21.15.69:0/2828384358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc60f20 0x56339dc51050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.571+0000 7f8c8d236740 1 --2- 172.21.15.69:0/2828384358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc58340 0x56339dc6c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.571+0000 7f8c8d236740 1 --2- 172.21.15.69:0/2828384358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc57a20 0x56339dc60690 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.571+0000 7f8c8d236740 1 -- 172.21.15.69:0/2828384358 >> 172.21.15.69:0/2828384358 conn(0x56339dc52150 msgr2=0x56339dc50940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:46.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.571+0000 7f8c8d236740 1 -- 172.21.15.69:0/2828384358 shutdown_connections 2024-03-19T01:23:46.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.571+0000 7f8c8d236740 1 -- 172.21.15.69:0/2828384358 wait complete. 2024-03-19T01:23:46.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.572+0000 7f8c8d236740 1 -- start start 2024-03-19T01:23:46.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.572+0000 7f8c8d236740 1 Processor -- start 2024-03-19T01:23:46.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.573+0000 7f8c8d236740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 0x56339dc64a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.573+0000 7f8c8d236740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 0x56339dc64f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.573+0000 7f8c8d236740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc60f20 0x56339dcd61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.574+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 0x56339dc64f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.574+0000 7f8c82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 0x56339dc64f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36050/0 (socket says 172.21.15.69:36050) 2024-03-19T01:23:46.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.574+0000 7f8c83400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 0x56339dc64a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.574+0000 7f8c83400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 0x56339dc64a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34772/0 (socket says 172.21.15.69:34772) 2024-03-19T01:23:46.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.574+0000 7f8c82a00640 1 -- 172.21.15.69:0/108916808 learned_addr learned my addr 172.21.15.69:0/108916808 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:46.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.574+0000 7f8c8d236740 1 -- 172.21.15.69:0/108916808 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841c7a50 con 0x56339dc57a20 2024-03-19T01:23:46.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.574+0000 7f8c8d236740 1 -- 172.21.15.69:0/108916808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841c7380 con 0x56339dc58340 2024-03-19T01:23:46.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.574+0000 7f8c8d236740 1 -- 172.21.15.69:0/108916808 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841c45e0 con 0x56339dc60f20 2024-03-19T01:23:46.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.574+0000 7f8c83e00640 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc60f20 0x56339dcd61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.575+0000 7f8c82a00640 1 -- 172.21.15.69:0/108916808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc60f20 msgr2=0x56339dcd61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:46.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.575+0000 7f8c82a00640 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc60f20 0x56339dcd61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.575+0000 7f8c8d236740 10 client.0 _ll_register_callbacks cb 0x7f8c7000b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:46.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.575+0000 7f8c82a00640 1 -- 172.21.15.69:0/108916808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 msgr2=0x56339dc64a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:46.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.575+0000 7f8c82a00640 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 0x56339dc64a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.575+0000 7f8c82a00640 1 -- 172.21.15.69:0/108916808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56339dc657f0 con 0x56339dc58340 2024-03-19T01:23:46.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.575+0000 7f8c82a00640 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 0x56339dc64f60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8c74014040 tx=0x7f8c74004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64f5492cb3873b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:46.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.575+0000 7f8c6d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:23:46.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.575+0000 7f8c6d600640 1 -- 172.21.15.69:0/108916808 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c740275c0 con 0x56339dc58340 2024-03-19T01:23:46.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.575+0000 7f8c6d600640 1 -- 172.21.15.69:0/108916808 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f8c74027760 con 0x56339dc58340 2024-03-19T01:23:46.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.579+0000 7f8c8d236740 1 -- 172.21.15.69:0/108916808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c7c004c80 con 0x56339dc58340 2024-03-19T01:23:46.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.581+0000 7f8c6d600640 1 -- 172.21.15.69:0/108916808 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f8c74003970 con 0x56339dc58340 2024-03-19T01:23:46.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.581+0000 7f8c8d236740 10 client.28019 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:46.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.581+0000 7f8c8d236740 1 -- 172.21.15.69:0/108916808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=59}) -- 0x56339dcd77b0 con 0x56339dc58340 2024-03-19T01:23:46.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.582+0000 7f8c6d600640 1 -- 172.21.15.69:0/108916808 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f8c74003b10 con 0x56339dc58340 2024-03-19T01:23:46.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.582+0000 7f8c8d236740 10 client.28019 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:46.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.582+0000 7f8c8d236740 10 client.28019 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:23:46.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.582+0000 7f8c8d236740 20 client.28019 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:46.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.582+0000 7f8c8d236740 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56339dcd7dd0 0x56339dcda1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:46.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.582+0000 7f8c8d236740 4 client.28019 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:23:46.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.582+0000 7f8c8d236740 1 -- 172.21.15.69:0/108916808 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56339dbc3c10 con 0x56339dcd7dd0 2024-03-19T01:23:46.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.585+0000 7f8c83400640 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56339dcd7dd0 0x56339dcda1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:46.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.586+0000 7f8c83400640 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56339dcd7dd0 0x56339dcda1f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8070bb68507eb35a server_cookie=cdf39f1832d8a8ae in_seq=0 out_seq=0 2024-03-19T01:23:46.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.586+0000 7f8c6d600640 10 client.28019 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:23:46.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.586+0000 7f8c6d600640 1 -- 172.21.15.69:0/108916808 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56339dbc3c10 con 0x56339dcd7dd0 2024-03-19T01:23:46.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.586+0000 7f8c6d600640 10 client.28019 handle_command_reply: tid=0 2024-03-19T01:23:46.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.586+0000 7f8c8d236740 1 client.28019 shutdown 2024-03-19T01:23:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.586+0000 7f8c8d236740 1 -- 172.21.15.69:0/108916808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 msgr2=0x56339dc64f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.586+0000 7f8c8d236740 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 0x56339dc64f60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8c74014040 tx=0x7f8c74004a40 comp rx=0 tx=0).stop 2024-03-19T01:23:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.587+0000 7f8c8d236740 1 -- 172.21.15.69:0/108916808 shutdown_connections 2024-03-19T01:23:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.587+0000 7f8c8d236740 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x56339dcd7dd0 0x56339dcda1f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.587+0000 7f8c8d236740 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56339dc60f20 0x56339dcd61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.587+0000 7f8c8d236740 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56339dc58340 0x56339dc64f60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.587+0000 7f8c8d236740 1 --2- 172.21.15.69:0/108916808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56339dc57a20 0x56339dc64a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.587+0000 7f8c8d236740 1 -- 172.21.15.69:0/108916808 >> 172.21.15.69:0/108916808 conn(0x56339dc52150 msgr2=0x56339dc511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.587+0000 7f8c6d600640 0 client.28019 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:23:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.587+0000 7f8c8d236740 1 -- 172.21.15.69:0/108916808 shutdown_connections 2024-03-19T01:23:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.587+0000 7f8c8d236740 1 -- 172.21.15.69:0/108916808 wait complete. 2024-03-19T01:23:46.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.587+0000 7f8c8d236740 20 client.28019 trim_cache size 0 max 16384 2024-03-19T01:23:46.589 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:23:46.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.591+0000 7f8c8a600640 1 -- 172.21.15.69:0/815405107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8c680a2a30 msgr2=0x7f8c680a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.591+0000 7f8c8a600640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8c680a2a30 0x7f8c680a4ef0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8c8410ba00 tx=0x7f8c74009990 comp rx=0 tx=0).stop 2024-03-19T01:23:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.591+0000 7f8c8a600640 1 -- 172.21.15.69:0/815405107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c84108a00 msgr2=0x7f8c8410be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.591+0000 7f8c8a600640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c84108a00 0x7f8c8410be90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c02c240 tx=0x7f8c7c004390 comp rx=0 tx=0).stop 2024-03-19T01:23:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.591+0000 7f8c8a600640 1 -- 172.21.15.69:0/815405107 shutdown_connections 2024-03-19T01:23:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.591+0000 7f8c8a600640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8c680a2a30 0x7f8c680a4ef0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.591+0000 7f8c8a600640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c84114970 0x7f8c8410a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.591+0000 7f8c8a600640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c84109320 0x7f8c8410a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.591+0000 7f8c8a600640 1 --2- 172.21.15.69:0/815405107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c84108a00 0x7f8c8410be90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.591+0000 7f8c8a600640 1 -- 172.21.15.69:0/815405107 >> 172.21.15.69:0/815405107 conn(0x7f8c8406c900 msgr2=0x7f8c840704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:46.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.591+0000 7f8c8a600640 1 -- 172.21.15.69:0/815405107 shutdown_connections 2024-03-19T01:23:46.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:46.596+0000 7f8c8a600640 1 -- 172.21.15.69:0/815405107 wait complete. 2024-03-19T01:23:46.662 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:23:46.699 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/platform/x86/wmi-bmof.o 2024-03-19T01:23:46.721 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/gen7_renderstate.o 2024-03-19T01:23:46.851 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-ioctl.o 2024-03-19T01:23:46.900 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/mailbox/pcc.o 2024-03-19T01:23:47.007 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/tpm.o 2024-03-19T01:23:47.048 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/smccc/built-in.a 2024-03-19T01:23:47.091 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/vars.o 2024-03-19T01:23:47.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.160+0000 7f32aa400640 1 -- 172.21.15.69:0/3922654895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f329c0a5180 msgr2=0x7f329c0a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:47.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.160+0000 7f32aa400640 1 --2- 172.21.15.69:0/3922654895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f329c0a5180 0x7f329c0a5560 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3290009900 tx=0x7f329002bc50 comp rx=0 tx=0).stop 2024-03-19T01:23:47.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.165+0000 7f32aa400640 1 -- 172.21.15.69:0/3922654895 shutdown_connections 2024-03-19T01:23:47.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.165+0000 7f32aa400640 1 --2- 172.21.15.69:0/3922654895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c0ad2a0 0x7f329c0b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.165+0000 7f32aa400640 1 --2- 172.21.15.69:0/3922654895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f329c0a5b30 0x7f329c0acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.165+0000 7f32aa400640 1 --2- 172.21.15.69:0/3922654895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f329c0a5180 0x7f329c0a5560 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.165+0000 7f32aa400640 1 -- 172.21.15.69:0/3922654895 >> 172.21.15.69:0/3922654895 conn(0x7f329c01a800 msgr2=0x7f329c01ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:47.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.168+0000 7f32aa400640 1 -- 172.21.15.69:0/3922654895 shutdown_connections 2024-03-19T01:23:47.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.168+0000 7f32aa400640 1 -- 172.21.15.69:0/3922654895 wait complete. 2024-03-19T01:23:47.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.169+0000 7f32aa400640 1 Processor -- start 2024-03-19T01:23:47.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.169+0000 7f32aa400640 1 -- start start 2024-03-19T01:23:47.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.170+0000 7f32aa400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c0a5180 0x7f329c13d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.170+0000 7f32aa400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f329c0ad2a0 0x7f329c13d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.170+0000 7f32a2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c0a5180 0x7f329c13d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.170+0000 7f32a2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c0a5180 0x7f329c13d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36062/0 (socket says 172.21.15.69:36062) 2024-03-19T01:23:47.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.174+0000 7f32aa400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f329c141c90 0x7f329c142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.174+0000 7f32aa400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f329c0b7130 con 0x7f329c141c90 2024-03-19T01:23:47.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.174+0000 7f32aa400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f329c0b6fb0 con 0x7f329c0a5180 2024-03-19T01:23:47.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.174+0000 7f32aa400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f329c0b72b0 con 0x7f329c0ad2a0 2024-03-19T01:23:47.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.174+0000 7f32a2a00640 1 -- 172.21.15.69:0/3140307010 learned_addr learned my addr 172.21.15.69:0/3140307010 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:47.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.174+0000 7f32a2000640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f329c0ad2a0 0x7f329c13d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.174+0000 7f32a3400640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f329c141c90 0x7f329c142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.175+0000 7f32a2a00640 1 -- 172.21.15.69:0/3140307010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f329c0ad2a0 msgr2=0x7f329c13d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:47.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.175+0000 7f32a2a00640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f329c0ad2a0 0x7f329c13d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.175+0000 7f32a2a00640 1 -- 172.21.15.69:0/3140307010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f329c141c90 msgr2=0x7f329c142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:47.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.175+0000 7f32a2a00640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f329c141c90 0x7f329c142140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.175+0000 7f32a2a00640 1 -- 172.21.15.69:0/3140307010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c142780 con 0x7f329c0a5180 2024-03-19T01:23:47.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.175+0000 7f32a3400640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f329c141c90 0x7f329c142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:23:47.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.175+0000 7f32a2a00640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c0a5180 0x7f329c13d3d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3290009850 tx=0x7f3290004c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:47.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.176+0000 7f3297e00640 1 -- 172.21.15.69:0/3140307010 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290042020 con 0x7f329c0a5180 2024-03-19T01:23:47.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.176+0000 7f3297e00640 1 -- 172.21.15.69:0/3140307010 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32900054e0 con 0x7f329c0a5180 2024-03-19T01:23:47.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.176+0000 7f32aa400640 1 -- 172.21.15.69:0/3140307010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c142a10 con 0x7f329c0a5180 2024-03-19T01:23:47.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.176+0000 7f3297e00640 1 -- 172.21.15.69:0/3140307010 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290003cb0 con 0x7f329c0a5180 2024-03-19T01:23:47.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.176+0000 7f32aa400640 1 -- 172.21.15.69:0/3140307010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c00c120 con 0x7f329c0a5180 2024-03-19T01:23:47.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.185+0000 7f32ad188740 1 Processor -- start 2024-03-19T01:23:47.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.185+0000 7f32ad188740 1 -- start start 2024-03-19T01:23:47.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.185+0000 7f32ad188740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f277ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.185+0000 7f32ad188740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277d030 0x5652f277d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.185+0000 7f32ad188740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f278e840 0x5652f2790c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.185+0000 7f32ad188740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5652f26e1380 con 0x5652f278e840 2024-03-19T01:23:47.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.185+0000 7f32ad188740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5652f2689d20 con 0x5652f277d030 2024-03-19T01:23:47.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.185+0000 7f32ad188740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5652f26dd420 con 0x5652f277c680 2024-03-19T01:23:47.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.186+0000 7f32a3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f277ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.186+0000 7f32a3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f277ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45266/0 (socket says 172.21.15.69:45266) 2024-03-19T01:23:47.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.186+0000 7f32a3400640 1 -- 172.21.15.69:0/39436688 learned_addr learned my addr 172.21.15.69:0/39436688 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:47.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.186+0000 7f32a3400640 1 -- 172.21.15.69:0/39436688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277d030 msgr2=0x5652f277d4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:47.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.186+0000 7f32a3400640 1 --2- 172.21.15.69:0/39436688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277d030 0x5652f277d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.186+0000 7f32a3400640 1 -- 172.21.15.69:0/39436688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f278e840 msgr2=0x5652f2790c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:47.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.186+0000 7f32a3400640 1 --2- 172.21.15.69:0/39436688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f278e840 0x5652f2790c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.186+0000 7f32a3400640 1 -- 172.21.15.69:0/39436688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f23f53f0 con 0x5652f277c680 2024-03-19T01:23:47.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.187+0000 7f32a3400640 1 --2- 172.21.15.69:0/39436688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f277ca60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f32a4050330 tx=0x7f32a4075440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83c43e9882c47449 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:47.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.187+0000 7f3297e00640 1 -- 172.21.15.69:0/3140307010 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f3290045020 con 0x7f329c0a5180 2024-03-19T01:23:47.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.190+0000 7f3297e00640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f328c0a2a30 0x7f328c0a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.190+0000 7f3297e00640 1 -- 172.21.15.69:0/3140307010 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f32900f1260 con 0x7f329c0a5180 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.190+0000 7f32a2a00640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f328c0a2a30 0x7f328c0a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.191+0000 7f3294c00640 1 -- 172.21.15.69:0/39436688 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4069390 con 0x5652f277c680 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.191+0000 7f3294c00640 1 -- 172.21.15.69:0/39436688 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a4069a10 con 0x5652f277c680 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.191+0000 7f3294c00640 1 -- 172.21.15.69:0/39436688 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40733c0 con 0x5652f277c680 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.192+0000 7f32ad188740 1 -- 172.21.15.69:0/39436688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 msgr2=0x5652f277ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.192+0000 7f32ad188740 1 --2- 172.21.15.69:0/39436688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f277ca60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f32a4050330 tx=0x7f32a4075440 comp rx=0 tx=0).stop 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.192+0000 7f32a2a00640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f328c0a2a30 0x7f328c0a4ef0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x5652f277db20 tx=0x7f3290005fd0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.192+0000 7f32ad188740 1 -- 172.21.15.69:0/39436688 shutdown_connections 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.192+0000 7f32ad188740 1 --2- 172.21.15.69:0/39436688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f278e840 0x5652f2790c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.192+0000 7f32ad188740 1 --2- 172.21.15.69:0/39436688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277d030 0x5652f277d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.192+0000 7f32ad188740 1 --2- 172.21.15.69:0/39436688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f277ca60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.192+0000 7f32ad188740 1 -- 172.21.15.69:0/39436688 >> 172.21.15.69:0/39436688 conn(0x5652f2777340 msgr2=0x5652f2777720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:47.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.192+0000 7f32ad188740 1 -- 172.21.15.69:0/39436688 shutdown_connections 2024-03-19T01:23:47.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.192+0000 7f32ad188740 1 -- 172.21.15.69:0/39436688 wait complete. 2024-03-19T01:23:47.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.193+0000 7f32ad188740 1 -- start start 2024-03-19T01:23:47.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.193+0000 7f32ad188740 1 Processor -- start 2024-03-19T01:23:47.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32ad188740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277c680 0x5652f27883e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32ad188740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 0x5652f278e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32ad188740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f278e840 0x5652f27889a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32ad188740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5652f2782320 con 0x5652f277d030 2024-03-19T01:23:47.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32ad188740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5652f269d160 con 0x5652f277c680 2024-03-19T01:23:47.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32ad188740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5652f27824a0 con 0x5652f278e840 2024-03-19T01:23:47.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32ad188740 10 client.0 _ll_register_callbacks cb 0x5652f26d9170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:47.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277c680 0x5652f27883e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277c680 0x5652f27883e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36064/0 (socket says 172.21.15.69:36064) 2024-03-19T01:23:47.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32a3400640 1 -- 172.21.15.69:0/1164144664 learned_addr learned my addr 172.21.15.69:0/1164144664 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:47.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32a2000640 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 0x5652f278e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.200+0000 7f32a3400640 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f278e840 0x5652f27889a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.201+0000 7f32a3400640 1 -- 172.21.15.69:0/1164144664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f278e840 msgr2=0x5652f27889a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:47.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.201+0000 7f32a3400640 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f278e840 0x5652f27889a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.201+0000 7f32a3400640 1 -- 172.21.15.69:0/1164144664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 msgr2=0x5652f278e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:47.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.201+0000 7f32a3400640 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 0x5652f278e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.201+0000 7f32a3400640 1 -- 172.21.15.69:0/1164144664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f24bf7d0 con 0x5652f277c680 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.201+0000 7f32a2000640 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 0x5652f278e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.201+0000 7f32a3400640 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277c680 0x5652f27883e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f32a40674f0 tx=0x7f32a407e950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3713815ce703d9d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.201+0000 7f3289600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.202+0000 7f3289600640 1 -- 172.21.15.69:0/1164144664 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4087040 con 0x5652f277c680 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.202+0000 7f3289600640 1 -- 172.21.15.69:0/1164144664 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f32a4073780 con 0x5652f277c680 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.202+0000 7f32ad188740 1 -- 172.21.15.69:0/1164144664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a4069a10 con 0x5652f277c680 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.202+0000 7f3289600640 1 -- 172.21.15.69:0/1164144664 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f32a407e740 con 0x5652f277c680 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.202+0000 7f32ad188740 10 client.28028 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.202+0000 7f32ad188740 1 -- 172.21.15.69:0/1164144664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5652f27a6060 con 0x5652f277c680 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.203+0000 7f3289600640 1 -- 172.21.15.69:0/1164144664 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a4069a10 con 0x5652f277c680 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.203+0000 7f3289600640 1 -- 172.21.15.69:0/1164144664 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f32a407c020 con 0x5652f277c680 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.203+0000 7f32ad188740 10 client.28028 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.203+0000 7f32ad188740 10 client.28028 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:23:47.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.203+0000 7f32ad188740 20 client.28028 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:47.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.203+0000 7f32ad188740 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5652f27a73b0 0x5652f27a97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.203+0000 7f32ad188740 4 client.28028 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:23:47.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.203+0000 7f32ad188740 1 -- 172.21.15.69:0/1164144664 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652f26ee8c0 con 0x5652f27a73b0 2024-03-19T01:23:47.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.204+0000 7f32a2000640 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5652f27a73b0 0x5652f27a97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.206+0000 7f32a2000640 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5652f27a73b0 0x5652f27a97a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43dedbb9a8dcbc56 server_cookie=4f13b4f6714dca77 in_seq=0 out_seq=0 2024-03-19T01:23:47.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.206+0000 7f3289600640 10 client.28028 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:23:47.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.211+0000 7f3289600640 1 -- 172.21.15.69:0/1164144664 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5652f26ee8c0 con 0x5652f27a73b0 2024-03-19T01:23:47.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.211+0000 7f3289600640 10 client.28028 handle_command_reply: tid=0 2024-03-19T01:23:47.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.211+0000 7f32ad188740 1 client.28028 shutdown 2024-03-19T01:23:47.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.211+0000 7f32ad188740 1 -- 172.21.15.69:0/1164144664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277c680 msgr2=0x5652f27883e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:47.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.212+0000 7f32ad188740 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277c680 0x5652f27883e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f32a40674f0 tx=0x7f32a407e950 comp rx=0 tx=0).stop 2024-03-19T01:23:47.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.212+0000 7f32ad188740 1 -- 172.21.15.69:0/1164144664 shutdown_connections 2024-03-19T01:23:47.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.212+0000 7f32ad188740 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f278e840 0x5652f27889a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.212+0000 7f32ad188740 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 0x5652f278e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.212+0000 7f32ad188740 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5652f27a73b0 0x5652f27a97a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.212+0000 7f32ad188740 1 --2- 172.21.15.69:0/1164144664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277c680 0x5652f27883e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.212+0000 7f32ad188740 1 -- 172.21.15.69:0/1164144664 >> 172.21.15.69:0/1164144664 conn(0x5652f2777340 msgr2=0x5652f2779090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:47.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.212+0000 7f3289600640 0 client.28028 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:23:47.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.212+0000 7f32ad188740 1 -- 172.21.15.69:0/1164144664 shutdown_connections 2024-03-19T01:23:47.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.212+0000 7f32ad188740 1 -- 172.21.15.69:0/1164144664 wait complete. 2024-03-19T01:23:47.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.213+0000 7f32ad188740 20 client.28028 trim_cache size 0 max 16384 2024-03-19T01:23:47.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.271+0000 7f32ad188740 1 Processor -- start 2024-03-19T01:23:47.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.271+0000 7f32ad188740 1 -- start start 2024-03-19T01:23:47.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.271+0000 7f32ad188740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f278dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.271+0000 7f32ad188740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277d030 0x5652f27723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.271+0000 7f32ad188740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f278e840 0x5652f27aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.271+0000 7f32ad188740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f329c013ea0 con 0x5652f278e840 2024-03-19T01:23:47.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.271+0000 7f32ad188740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f329c0159e0 con 0x5652f277d030 2024-03-19T01:23:47.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.271+0000 7f32ad188740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f329c014570 con 0x5652f277c680 2024-03-19T01:23:47.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.272+0000 7f32a3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f278dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.273+0000 7f32a2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277d030 0x5652f27723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.273+0000 7f32a2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277d030 0x5652f27723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36066/0 (socket says 172.21.15.69:36066) 2024-03-19T01:23:47.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.273+0000 7f32a2000640 1 -- 172.21.15.69:0/2459560465 learned_addr learned my addr 172.21.15.69:0/2459560465 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:47.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.273+0000 7f32a3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f278dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45288/0 (socket says 172.21.15.69:45288) 2024-03-19T01:23:47.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.273+0000 7f32a3400640 1 --2- 172.21.15.69:0/2459560465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f278e840 0x5652f27aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.273+0000 7f32a2000640 1 -- 172.21.15.69:0/2459560465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 msgr2=0x5652f278dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:47.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.273+0000 7f32a2000640 1 --2- 172.21.15.69:0/2459560465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f278dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.273+0000 7f32a2000640 1 -- 172.21.15.69:0/2459560465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f278e840 msgr2=0x5652f27aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:47.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.273+0000 7f32a2000640 1 --2- 172.21.15.69:0/2459560465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f278e840 0x5652f27aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.273+0000 7f32a2000640 1 -- 172.21.15.69:0/2459560465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f2788d10 con 0x5652f277d030 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.274+0000 7f32a3400640 1 --2- 172.21.15.69:0/2459560465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f278e840 0x5652f27aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.274+0000 7f32a2000640 1 --2- 172.21.15.69:0/2459560465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277d030 0x5652f27723b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f329c018d00 tx=0x7f3298005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47ab80097a1b7e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.275+0000 7f328be00640 1 -- 172.21.15.69:0/2459560465 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298046020 con 0x5652f277d030 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.275+0000 7f328be00640 1 -- 172.21.15.69:0/2459560465 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329801f640 con 0x5652f277d030 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.275+0000 7f328be00640 1 -- 172.21.15.69:0/2459560465 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298010cc0 con 0x5652f277d030 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.276+0000 7f32ad188740 1 -- 172.21.15.69:0/2459560465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277d030 msgr2=0x5652f27723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.276+0000 7f32ad188740 1 --2- 172.21.15.69:0/2459560465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277d030 0x5652f27723b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f329c018d00 tx=0x7f3298005d30 comp rx=0 tx=0).stop 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.277+0000 7f32ad188740 1 -- 172.21.15.69:0/2459560465 shutdown_connections 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.277+0000 7f32ad188740 1 --2- 172.21.15.69:0/2459560465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f278e840 0x5652f27aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.277+0000 7f32ad188740 1 --2- 172.21.15.69:0/2459560465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f277d030 0x5652f27723b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.277+0000 7f32ad188740 1 --2- 172.21.15.69:0/2459560465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f278dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.277+0000 7f32ad188740 1 -- 172.21.15.69:0/2459560465 >> 172.21.15.69:0/2459560465 conn(0x5652f2777340 msgr2=0x5652f2777720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:47.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.281+0000 7f32ad188740 1 -- 172.21.15.69:0/2459560465 shutdown_connections 2024-03-19T01:23:47.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.281+0000 7f32ad188740 1 -- 172.21.15.69:0/2459560465 wait complete. 2024-03-19T01:23:47.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.282+0000 7f32ad188740 1 -- start start 2024-03-19T01:23:47.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.283+0000 7f32ad188740 1 Processor -- start 2024-03-19T01:23:47.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.283+0000 7f32ad188740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f27ba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.285+0000 7f32ad188740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 0x5652f27b5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.285+0000 7f32ad188740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f278e840 0x5652f27b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.285+0000 7f32ad188740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f329c015310 con 0x5652f277d030 2024-03-19T01:23:47.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.285+0000 7f32ad188740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f329c014c40 con 0x5652f278e840 2024-03-19T01:23:47.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.285+0000 7f32ad188740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f329c011e80 con 0x5652f277c680 2024-03-19T01:23:47.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.285+0000 7f32ad188740 10 client.0 _ll_register_callbacks cb 0x7f32a4072ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:47.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.285+0000 7f32a3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f27ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.286+0000 7f32a3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f27ba310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45298/0 (socket says 172.21.15.69:45298) 2024-03-19T01:23:47.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.286+0000 7f32a3400640 1 -- 172.21.15.69:0/2797810848 learned_addr learned my addr 172.21.15.69:0/2797810848 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:47.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.286+0000 7f32a3400640 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f278e840 0x5652f27b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.286+0000 7f32a2000640 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 0x5652f27b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.286+0000 7f32a3400640 1 -- 172.21.15.69:0/2797810848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f278e840 msgr2=0x5652f27b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:47.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.286+0000 7f32a3400640 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f278e840 0x5652f27b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.286+0000 7f32a3400640 1 -- 172.21.15.69:0/2797810848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 msgr2=0x5652f27b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:47.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.286+0000 7f32a3400640 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 0x5652f27b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.286+0000 7f32a3400640 1 -- 172.21.15.69:0/2797810848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f27b6100 con 0x5652f277c680 2024-03-19T01:23:47.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.287+0000 7f32a2000640 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 0x5652f27b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:47.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.287+0000 7f32a3400640 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f27ba310 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f32a4073010 tx=0x7f32a40670a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f68171aafe992b6b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:47.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.287+0000 7f3289600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:23:47.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.287+0000 7f3289600640 1 -- 172.21.15.69:0/2797810848 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a408dbd0 con 0x5652f277c680 2024-03-19T01:23:47.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.287+0000 7f3289600640 1 -- 172.21.15.69:0/2797810848 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f32a4069760 con 0x5652f277c680 2024-03-19T01:23:47.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.290+0000 7f32ad188740 1 -- 172.21.15.69:0/2797810848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329801f640 con 0x5652f277c680 2024-03-19T01:23:47.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.291+0000 7f3289600640 1 -- 172.21.15.69:0/2797810848 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f32a40a8050 con 0x5652f277c680 2024-03-19T01:23:47.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.291+0000 7f32ad188740 10 client.27973 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:47.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.291+0000 7f32ad188740 1 -- 172.21.15.69:0/2797810848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5652f2799060 con 0x5652f277c680 2024-03-19T01:23:47.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.291+0000 7f3289600640 1 -- 172.21.15.69:0/2797810848 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329801f640 con 0x5652f277c680 2024-03-19T01:23:47.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.291+0000 7f3289600640 1 -- 172.21.15.69:0/2797810848 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f32a40ab020 con 0x5652f277c680 2024-03-19T01:23:47.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.292+0000 7f32ad188740 10 client.27973 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:47.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.292+0000 7f32ad188740 10 client.27973 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:23:47.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.292+0000 7f32ad188740 20 client.27973 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:47.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.292+0000 7f32ad188740 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5652f2799730 0x5652f279bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:47.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.292+0000 7f32ad188740 4 client.27973 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:23:47.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.292+0000 7f32ad188740 1 -- 172.21.15.69:0/2797810848 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652f26e8c10 con 0x5652f2799730 2024-03-19T01:23:47.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.294+0000 7f32a2000640 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5652f2799730 0x5652f279bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:47.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.295+0000 7f32a2000640 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5652f2799730 0x5652f279bb50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebc01c51f409e9a9 server_cookie=eaf1d53daa0c6ff in_seq=0 out_seq=0 2024-03-19T01:23:47.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.295+0000 7f3289600640 10 client.27973 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:23:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.295+0000 7f3289600640 1 -- 172.21.15.69:0/2797810848 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652f26e8c10 con 0x5652f2799730 2024-03-19T01:23:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.295+0000 7f3289600640 10 client.27973 handle_command_reply: tid=0 2024-03-19T01:23:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.296+0000 7f32ad188740 1 client.27973 shutdown 2024-03-19T01:23:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.296+0000 7f32ad188740 1 -- 172.21.15.69:0/2797810848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 msgr2=0x5652f27ba310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.296+0000 7f32ad188740 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f27ba310 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f32a4073010 tx=0x7f32a40670a0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.296+0000 7f32ad188740 1 -- 172.21.15.69:0/2797810848 shutdown_connections 2024-03-19T01:23:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.296+0000 7f32ad188740 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5652f2799730 0x5652f279bb50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.296+0000 7f32ad188740 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652f278e840 0x5652f27b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.296+0000 7f32ad188740 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f277d030 0x5652f27b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.296+0000 7f32ad188740 1 --2- 172.21.15.69:0/2797810848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5652f277c680 0x5652f27ba310 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.296+0000 7f3289600640 0 client.27973 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:23:47.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.296+0000 7f32ad188740 1 -- 172.21.15.69:0/2797810848 >> 172.21.15.69:0/2797810848 conn(0x5652f2777340 msgr2=0x5652f277df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:47.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.297+0000 7f32ad188740 1 -- 172.21.15.69:0/2797810848 shutdown_connections 2024-03-19T01:23:47.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.297+0000 7f32ad188740 1 -- 172.21.15.69:0/2797810848 wait complete. 2024-03-19T01:23:47.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.297+0000 7f32ad188740 20 client.27973 trim_cache size 0 max 16384 2024-03-19T01:23:47.299 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:23:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.300+0000 7f32aa400640 1 -- 172.21.15.69:0/3140307010 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f328c0a2a30 msgr2=0x7f328c0a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.300+0000 7f32aa400640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f328c0a2a30 0x7f328c0a4ef0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x5652f277db20 tx=0x7f3290005fd0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.300+0000 7f32aa400640 1 -- 172.21.15.69:0/3140307010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c0a5180 msgr2=0x7f329c13d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.300+0000 7f32aa400640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c0a5180 0x7f329c13d3d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3290009850 tx=0x7f3290004c00 comp rx=0 tx=0).stop 2024-03-19T01:23:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.300+0000 7f32aa400640 1 -- 172.21.15.69:0/3140307010 shutdown_connections 2024-03-19T01:23:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.300+0000 7f32aa400640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f329c141c90 0x7f329c142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.300+0000 7f32aa400640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f329c0ad2a0 0x7f329c13d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.300+0000 7f32aa400640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f328c0a2a30 0x7f328c0a4ef0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.300+0000 7f32aa400640 1 --2- 172.21.15.69:0/3140307010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c0a5180 0x7f329c13d3d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.300+0000 7f32aa400640 1 -- 172.21.15.69:0/3140307010 >> 172.21.15.69:0/3140307010 conn(0x7f329c01a800 msgr2=0x7f329c0a3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.300+0000 7f32aa400640 1 -- 172.21.15.69:0/3140307010 shutdown_connections 2024-03-19T01:23:47.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:47.301+0000 7f32aa400640 1 -- 172.21.15.69:0/3140307010 wait complete. 2024-03-19T01:23:47.359 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:23:47.441 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/gen8_renderstate.o 2024-03-19T01:23:47.466 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/usbhid/hid-pidff.o 2024-03-19T01:23:47.839 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/platform/x86/eeepc-laptop.o 2024-03-19T01:23:47.995 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/gen9_renderstate.o 2024-03-19T01:23:48.064 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/mailbox/built-in.a 2024-03-19T01:23:48.111 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-input.o 2024-03-19T01:23:48.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.119+0000 7f9e96a00640 1 -- 172.21.15.69:0/2999669713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e880a4fc0 msgr2=0x7f9e880a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:48.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.119+0000 7f9e96a00640 1 --2- 172.21.15.69:0/2999669713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e880a4fc0 0x7f9e880a53a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9e90067630 tx=0x7f9e9008f4e0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.121+0000 7f9e96a00640 1 -- 172.21.15.69:0/2999669713 shutdown_connections 2024-03-19T01:23:48.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.121+0000 7f9e96a00640 1 --2- 172.21.15.69:0/2999669713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e880ac100 0x7f9e880b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.122+0000 7f9e96a00640 1 --2- 172.21.15.69:0/2999669713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e880a5970 0x7f9e880ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.122+0000 7f9e96a00640 1 --2- 172.21.15.69:0/2999669713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e880a4fc0 0x7f9e880a53a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.122+0000 7f9e96a00640 1 -- 172.21.15.69:0/2999669713 >> 172.21.15.69:0/2999669713 conn(0x7f9e8801a640 msgr2=0x7f9e8801aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:48.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.122+0000 7f9e96a00640 1 -- 172.21.15.69:0/2999669713 shutdown_connections 2024-03-19T01:23:48.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.123+0000 7f9e96a00640 1 -- 172.21.15.69:0/2999669713 wait complete. 2024-03-19T01:23:48.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.129+0000 7f9e96a00640 1 Processor -- start 2024-03-19T01:23:48.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.130+0000 7f9e96a00640 1 -- start start 2024-03-19T01:23:48.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.130+0000 7f9e96a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e880a4fc0 0x7f9e88144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.130+0000 7f9e96a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e880ac100 0x7f9e88144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.130+0000 7f9e96a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e880b0c00 0x7f9e880b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.130+0000 7f9e94c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e880ac100 0x7f9e88144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.131+0000 7f9e94c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e880ac100 0x7f9e88144780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36100/0 (socket says 172.21.15.69:36100) 2024-03-19T01:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.132+0000 7f9e94c00640 1 -- 172.21.15.69:0/3319285594 learned_addr learned my addr 172.21.15.69:0/3319285594 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.132+0000 7f9e96a00640 1 -- 172.21.15.69:0/3319285594 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e880b70e0 con 0x7f9e880a4fc0 2024-03-19T01:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.132+0000 7f9e96a00640 1 -- 172.21.15.69:0/3319285594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e880b6f60 con 0x7f9e880ac100 2024-03-19T01:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.132+0000 7f9e96a00640 1 -- 172.21.15.69:0/3319285594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e880b7260 con 0x7f9e880b0c00 2024-03-19T01:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.132+0000 7f9e96000640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e880b0c00 0x7f9e880b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.132+0000 7f9e94c00640 1 -- 172.21.15.69:0/3319285594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e880b0c00 msgr2=0x7f9e880b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.132+0000 7f9e95600640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e880a4fc0 0x7f9e88144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.132+0000 7f9e94c00640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e880b0c00 0x7f9e880b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.132+0000 7f9e94c00640 1 -- 172.21.15.69:0/3319285594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e880a4fc0 msgr2=0x7f9e88144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.133+0000 7f9e94c00640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e880a4fc0 0x7f9e88144240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.133+0000 7f9e94c00640 1 -- 172.21.15.69:0/3319285594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e880b1950 con 0x7f9e880ac100 2024-03-19T01:23:48.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.135+0000 7f9e95600640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e880a4fc0 0x7f9e88144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:48.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.135+0000 7f9e94c00640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e880ac100 0x7f9e88144780 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400b450 tx=0x7f9e8400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:48.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.136+0000 7f9e8e000640 1 -- 172.21.15.69:0/3319285594 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84015020 con 0x7f9e880ac100 2024-03-19T01:23:48.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.136+0000 7f9e8e000640 1 -- 172.21.15.69:0/3319285594 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e84011d70 con 0x7f9e880ac100 2024-03-19T01:23:48.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.137+0000 7f9e8e000640 1 -- 172.21.15.69:0/3319285594 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400f340 con 0x7f9e880ac100 2024-03-19T01:23:48.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.139+0000 7f9e96a00640 1 -- 172.21.15.69:0/3319285594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8813fd40 con 0x7f9e880ac100 2024-03-19T01:23:48.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.139+0000 7f9e96a00640 1 -- 172.21.15.69:0/3319285594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e88140210 con 0x7f9e880ac100 2024-03-19T01:23:48.149 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/file.o 2024-03-19T01:23:48.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.155+0000 7f9e9ae30740 1 Processor -- start 2024-03-19T01:23:48.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.157+0000 7f9e8e000640 1 -- 172.21.15.69:0/3319285594 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9e84018020 con 0x7f9e880ac100 2024-03-19T01:23:48.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.157+0000 7f9e9ae30740 1 -- start start 2024-03-19T01:23:48.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.162+0000 7f9e8e000640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9e780a2a50 0x7f9e780a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.162+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 0x561cdd92bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.162+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92c530 0x561cdd934bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.162+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd937500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.162+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561cdd890380 con 0x561cdd92c530 2024-03-19T01:23:48.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.162+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561cdd838d20 con 0x561cdd92bc10 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.162+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561cdd88c420 con 0x561cdd935110 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.163+0000 7f9e95600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 0x561cdd92bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.163+0000 7f9e95600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 0x561cdd92bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36110/0 (socket says 172.21.15.69:36110) 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.163+0000 7f9e95600640 1 -- 172.21.15.69:0/154932042 learned_addr learned my addr 172.21.15.69:0/154932042 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.163+0000 7f9e96000640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9e780a2a50 0x7f9e780a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.163+0000 7f9e8e000640 1 -- 172.21.15.69:0/3319285594 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f9e840cea40 con 0x7f9e880ac100 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.164+0000 7f9e94c00640 1 --2- 172.21.15.69:0/154932042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92c530 0x561cdd934bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.164+0000 7f9e95600640 1 -- 172.21.15.69:0/154932042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 msgr2=0x561cdd937500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.164+0000 7f9e95600640 1 --2- 172.21.15.69:0/154932042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd937500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.164+0000 7f9e95600640 1 -- 172.21.15.69:0/154932042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92c530 msgr2=0x561cdd934bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.164+0000 7f9e95600640 1 --2- 172.21.15.69:0/154932042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92c530 0x561cdd934bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.164+0000 7f9e95600640 1 -- 172.21.15.69:0/154932042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cdd5a43f0 con 0x561cdd92bc10 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.164+0000 7f9e94c00640 1 --2- 172.21.15.69:0/154932042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92c530 0x561cdd934bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.164+0000 7f9e96000640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9e780a2a50 0x7f9e780a4f10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009bb0 tx=0x7f9e7c005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:48.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.167+0000 7f9e95600640 1 --2- 172.21.15.69:0/154932042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 0x561cdd92bff0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9e900507f0 tx=0x7f9e9004fdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96e9f90b5a15462b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:48.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.167+0000 7f9e82600640 1 -- 172.21.15.69:0/154932042 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900a6020 con 0x561cdd92bc10 2024-03-19T01:23:48.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.167+0000 7f9e82600640 1 -- 172.21.15.69:0/154932042 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e90096ba0 con 0x561cdd92bc10 2024-03-19T01:23:48.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.167+0000 7f9e82600640 1 -- 172.21.15.69:0/154932042 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90097b60 con 0x561cdd92bc10 2024-03-19T01:23:48.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.169+0000 7f9e9ae30740 1 -- 172.21.15.69:0/154932042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 msgr2=0x561cdd92bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:48.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.169+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/154932042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 0x561cdd92bff0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9e900507f0 tx=0x7f9e9004fdd0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.170+0000 7f9e9ae30740 1 -- 172.21.15.69:0/154932042 shutdown_connections 2024-03-19T01:23:48.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.170+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/154932042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd937500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.170+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/154932042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92c530 0x561cdd934bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.170+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/154932042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 0x561cdd92bff0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.170+0000 7f9e9ae30740 1 -- 172.21.15.69:0/154932042 >> 172.21.15.69:0/154932042 conn(0x561cdd926340 msgr2=0x561cdd926720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:48.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.170+0000 7f9e9ae30740 1 -- 172.21.15.69:0/154932042 shutdown_connections 2024-03-19T01:23:48.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.170+0000 7f9e9ae30740 1 -- 172.21.15.69:0/154932042 wait complete. 2024-03-19T01:23:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.172+0000 7f9e9ae30740 1 -- start start 2024-03-19T01:23:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.173+0000 7f9e9ae30740 1 Processor -- start 2024-03-19T01:23:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.179+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 0x561cdd9321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.179+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92c530 0x561cdd932730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.179+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd932c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.179+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561cdd937ea0 con 0x561cdd92c530 2024-03-19T01:23:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.179+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561cdd84c160 con 0x561cdd92bc10 2024-03-19T01:23:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.179+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561cdd938020 con 0x561cdd935110 2024-03-19T01:23:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.179+0000 7f9e9ae30740 10 client.0 _ll_register_callbacks cb 0x561cdd888170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:48.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.179+0000 7f9e95600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 0x561cdd9321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.180+0000 7f9e96000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd932c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.180+0000 7f9e96000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd932c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45306/0 (socket says 172.21.15.69:45306) 2024-03-19T01:23:48.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.180+0000 7f9e96000640 1 -- 172.21.15.69:0/1628091329 learned_addr learned my addr 172.21.15.69:0/1628091329 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:48.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.180+0000 7f9e94c00640 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92c530 0x561cdd932730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.180+0000 7f9e96000640 1 -- 172.21.15.69:0/1628091329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 msgr2=0x561cdd9321f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:48.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.181+0000 7f9e96000640 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 0x561cdd9321f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.181+0000 7f9e96000640 1 -- 172.21.15.69:0/1628091329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92c530 msgr2=0x561cdd932730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:48.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.181+0000 7f9e96000640 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92c530 0x561cdd932730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.181+0000 7f9e96000640 1 -- 172.21.15.69:0/1628091329 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cdd66e7d0 con 0x561cdd935110 2024-03-19T01:23:48.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.182+0000 7f9e95600640 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 0x561cdd9321f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:48.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.182+0000 7f9e96000640 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd932c70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c012010 tx=0x7f9e7c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3304be962ce690fd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:48.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.183+0000 7f9e76a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:23:48.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.183+0000 7f9e76a00640 1 -- 172.21.15.69:0/1628091329 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c00fad0 con 0x561cdd935110 2024-03-19T01:23:48.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.183+0000 7f9e76a00640 1 -- 172.21.15.69:0/1628091329 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f9e7c00fc70 con 0x561cdd935110 2024-03-19T01:23:48.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.184+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1628091329 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e90096ba0 con 0x561cdd935110 2024-03-19T01:23:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.184+0000 7f9e76a00640 1 -- 172.21.15.69:0/1628091329 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9e7c00fe70 con 0x561cdd935110 2024-03-19T01:23:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.184+0000 7f9e9ae30740 10 client.27982 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.184+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1628091329 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x561cdd944220 con 0x561cdd935110 2024-03-19T01:23:48.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.185+0000 7f9e76a00640 1 -- 172.21.15.69:0/1628091329 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e90096ba0 con 0x561cdd935110 2024-03-19T01:23:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.185+0000 7f9e76a00640 1 -- 172.21.15.69:0/1628091329 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f9e7c01f680 con 0x561cdd935110 2024-03-19T01:23:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.185+0000 7f9e9ae30740 10 client.27982 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.185+0000 7f9e9ae30740 10 client.27982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:23:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.185+0000 7f9e9ae30740 20 client.27982 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.185+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561cdd9456e0 0x561cdd947af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.185+0000 7f9e9ae30740 4 client.27982 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:23:48.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.185+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1628091329 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cdd89d8c0 con 0x561cdd9456e0 2024-03-19T01:23:48.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.186+0000 7f9e95600640 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561cdd9456e0 0x561cdd947af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.189+0000 7f9e95600640 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561cdd9456e0 0x561cdd947af0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6b3ace0c1729cb server_cookie=e69d74023be44d44 in_seq=0 out_seq=0 2024-03-19T01:23:48.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.189+0000 7f9e76a00640 10 client.27982 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:48.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.191+0000 7f9e76a00640 1 -- 172.21.15.69:0/1628091329 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561cdd89d8c0 con 0x561cdd9456e0 2024-03-19T01:23:48.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.191+0000 7f9e76a00640 10 client.27982 handle_command_reply: tid=0 2024-03-19T01:23:48.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.192+0000 7f9e9ae30740 1 client.27982 shutdown 2024-03-19T01:23:48.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.195+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1628091329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 msgr2=0x561cdd932c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:48.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.195+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd932c70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c012010 tx=0x7f9e7c004730 comp rx=0 tx=0).stop 2024-03-19T01:23:48.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.198+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1628091329 shutdown_connections 2024-03-19T01:23:48.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.198+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561cdd9456e0 0x561cdd947af0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.198+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd932c70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.198+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92c530 0x561cdd932730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.198+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/1628091329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92bc10 0x561cdd9321f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.198+0000 7f9e76a00640 0 client.27982 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:48.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.198+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1628091329 >> 172.21.15.69:0/1628091329 conn(0x561cdd926340 msgr2=0x561cdd928090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:48.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.202+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1628091329 shutdown_connections 2024-03-19T01:23:48.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.203+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1628091329 wait complete. 2024-03-19T01:23:48.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.205+0000 7f9e9ae30740 20 client.27982 trim_cache size 0 max 16384 2024-03-19T01:23:48.245 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/mem.o 2024-03-19T01:23:48.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.252+0000 7f9e9ae30740 1 Processor -- start 2024-03-19T01:23:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.253+0000 7f9e9ae30740 1 -- start start 2024-03-19T01:23:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.253+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 0x561cdd940ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.253+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92c530 0x561cdd94d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.253+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd930ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.253+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88162f00 con 0x561cdd92bc10 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.253+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88164a40 con 0x561cdd92c530 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.253+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881635d0 con 0x561cdd935110 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.254+0000 7f9e94c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92c530 0x561cdd94d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.254+0000 7f9e94c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92c530 0x561cdd94d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36130/0 (socket says 172.21.15.69:36130) 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.254+0000 7f9e94c00640 1 -- 172.21.15.69:0/1360453091 learned_addr learned my addr 172.21.15.69:0/1360453091 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.254+0000 7f9e95600640 1 --2- 172.21.15.69:0/1360453091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 0x561cdd940ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.254+0000 7f9e94c00640 1 -- 172.21.15.69:0/1360453091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 msgr2=0x561cdd930ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.254+0000 7f9e94c00640 1 --2- 172.21.15.69:0/1360453091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd930ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.254+0000 7f9e94c00640 1 -- 172.21.15.69:0/1360453091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 msgr2=0x561cdd940ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.254+0000 7f9e94c00640 1 --2- 172.21.15.69:0/1360453091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 0x561cdd940ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.254+0000 7f9e94c00640 1 -- 172.21.15.69:0/1360453091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cdd92c360 con 0x561cdd92c530 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.255+0000 7f9e95600640 1 --2- 172.21.15.69:0/1360453091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 0x561cdd940ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.255+0000 7f9e94c00640 1 --2- 172.21.15.69:0/1360453091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92c530 0x561cdd94d400 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9e840971c0 tx=0x7f9e8401c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9a27dc3af8d010 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.256+0000 7f9e81c00640 1 -- 172.21.15.69:0/1360453091 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84098e00 con 0x561cdd92c530 2024-03-19T01:23:48.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.256+0000 7f9e81c00640 1 -- 172.21.15.69:0/1360453091 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e84099060 con 0x561cdd92c530 2024-03-19T01:23:48.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.256+0000 7f9e81c00640 1 -- 172.21.15.69:0/1360453091 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84017e60 con 0x561cdd92c530 2024-03-19T01:23:48.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.257+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1360453091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92c530 msgr2=0x561cdd94d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:48.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.257+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/1360453091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92c530 0x561cdd94d400 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9e840971c0 tx=0x7f9e8401c040 comp rx=0 tx=0).stop 2024-03-19T01:23:48.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.258+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1360453091 shutdown_connections 2024-03-19T01:23:48.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.258+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/1360453091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd935110 0x561cdd930ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.258+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/1360453091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd92c530 0x561cdd94d400 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.258+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/1360453091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 0x561cdd940ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.258+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1360453091 >> 172.21.15.69:0/1360453091 conn(0x561cdd926340 msgr2=0x561cdd92d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:48.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.263+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1360453091 shutdown_connections 2024-03-19T01:23:48.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.267+0000 7f9e9ae30740 1 -- 172.21.15.69:0/1360453091 wait complete. 2024-03-19T01:23:48.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.267+0000 7f9e9ae30740 1 -- start start 2024-03-19T01:23:48.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.267+0000 7f9e9ae30740 1 Processor -- start 2024-03-19T01:23:48.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.268+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 0x561cdd93d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.268+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd92c530 0x561cdd93d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.268+0000 7f9e9ae30740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd935110 0x561cdd938580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.268+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88164370 con 0x561cdd92bc10 2024-03-19T01:23:48.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.268+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88163ca0 con 0x561cdd935110 2024-03-19T01:23:48.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.268+0000 7f9e9ae30740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88160ee0 con 0x561cdd92c530 2024-03-19T01:23:48.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.268+0000 7f9e9ae30740 10 client.0 _ll_register_callbacks cb 0x7f9e7c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:48.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.269+0000 7f9e96000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd935110 0x561cdd938580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.269+0000 7f9e96000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd935110 0x561cdd938580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36136/0 (socket says 172.21.15.69:36136) 2024-03-19T01:23:48.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.269+0000 7f9e96000640 1 -- 172.21.15.69:0/3436469139 learned_addr learned my addr 172.21.15.69:0/3436469139 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.269+0000 7f9e95600640 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 0x561cdd93d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.269+0000 7f9e96000640 1 -- 172.21.15.69:0/3436469139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd92c530 msgr2=0x561cdd93d680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.269+0000 7f9e96000640 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd92c530 0x561cdd93d680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.269+0000 7f9e96000640 1 -- 172.21.15.69:0/3436469139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 msgr2=0x561cdd93d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.269+0000 7f9e96000640 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 0x561cdd93d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.269+0000 7f9e96000640 1 -- 172.21.15.69:0/3436469139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cdd938d80 con 0x561cdd935110 2024-03-19T01:23:48.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.270+0000 7f9e95600640 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 0x561cdd93d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:23:48.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.270+0000 7f9e96000640 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd935110 0x561cdd938580 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009900 tx=0x7f9e7c009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31de35171f9e383 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:48.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.270+0000 7f9e76a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:23:48.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.270+0000 7f9e76a00640 1 -- 172.21.15.69:0/3436469139 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c03f060 con 0x561cdd935110 2024-03-19T01:23:48.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.270+0000 7f9e76a00640 1 -- 172.21.15.69:0/3436469139 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f9e7c004230 con 0x561cdd935110 2024-03-19T01:23:48.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.270+0000 7f9e9ae30740 1 -- 172.21.15.69:0/3436469139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e84099060 con 0x561cdd935110 2024-03-19T01:23:48.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.271+0000 7f9e76a00640 1 -- 172.21.15.69:0/3436469139 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9e7c046050 con 0x561cdd935110 2024-03-19T01:23:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.274+0000 7f9e9ae30740 10 client.28049 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.274+0000 7f9e9ae30740 1 -- 172.21.15.69:0/3436469139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x561cdd9ab560 con 0x561cdd935110 2024-03-19T01:23:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.274+0000 7f9e76a00640 1 -- 172.21.15.69:0/3436469139 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e7c012450 con 0x561cdd935110 2024-03-19T01:23:48.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.275+0000 7f9e76a00640 1 -- 172.21.15.69:0/3436469139 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f9e7c0038c0 con 0x561cdd935110 2024-03-19T01:23:48.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.275+0000 7f9e9ae30740 10 client.28049 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:48.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.275+0000 7f9e9ae30740 10 client.28049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:23:48.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.275+0000 7f9e9ae30740 20 client.28049 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:48.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.275+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561cdd9abc60 0x561cdd9ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:48.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.275+0000 7f9e9ae30740 4 client.28049 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:23:48.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.275+0000 7f9e9ae30740 1 -- 172.21.15.69:0/3436469139 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9e84099060 con 0x561cdd9abc60 2024-03-19T01:23:48.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.276+0000 7f9e95600640 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561cdd9abc60 0x561cdd9ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:48.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.276+0000 7f9e95600640 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561cdd9abc60 0x561cdd9ae080 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80469d1936cf8359 server_cookie=b509e7e366893927 in_seq=0 out_seq=0 2024-03-19T01:23:48.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.276+0000 7f9e76a00640 10 client.28049 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:48.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.276+0000 7f9e76a00640 1 -- 172.21.15.69:0/3436469139 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9e84099060 con 0x561cdd9abc60 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.276+0000 7f9e76a00640 10 client.28049 handle_command_reply: tid=0 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.276+0000 7f9e9ae30740 1 client.28049 shutdown 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e9ae30740 1 -- 172.21.15.69:0/3436469139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd935110 msgr2=0x561cdd938580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd935110 0x561cdd938580 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009900 tx=0x7f9e7c009390 comp rx=0 tx=0).stop 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e9ae30740 1 -- 172.21.15.69:0/3436469139 shutdown_connections 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x561cdd9abc60 0x561cdd9ae080 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cdd935110 0x561cdd938580 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561cdd92c530 0x561cdd93d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e9ae30740 1 --2- 172.21.15.69:0/3436469139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cdd92bc10 0x561cdd93d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e9ae30740 1 -- 172.21.15.69:0/3436469139 >> 172.21.15.69:0/3436469139 conn(0x561cdd926340 msgr2=0x561cdd9309f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e76a00640 0 client.28049 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e9ae30740 1 -- 172.21.15.69:0/3436469139 shutdown_connections 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e9ae30740 1 -- 172.21.15.69:0/3436469139 wait complete. 2024-03-19T01:23:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.277+0000 7f9e9ae30740 20 client.28049 trim_cache size 0 max 16384 2024-03-19T01:23:48.280 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:23:48.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.279+0000 7f9e96a00640 1 -- 172.21.15.69:0/3319285594 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9e780a2a50 msgr2=0x7f9e780a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:48.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.279+0000 7f9e96a00640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9e780a2a50 0x7f9e780a4f10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009bb0 tx=0x7f9e7c005f90 comp rx=0 tx=0).stop 2024-03-19T01:23:48.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.279+0000 7f9e96a00640 1 -- 172.21.15.69:0/3319285594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e880ac100 msgr2=0x7f9e88144780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:48.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.279+0000 7f9e96a00640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e880ac100 0x7f9e88144780 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400b450 tx=0x7f9e8400b910 comp rx=0 tx=0).stop 2024-03-19T01:23:48.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.280+0000 7f9e96a00640 1 -- 172.21.15.69:0/3319285594 shutdown_connections 2024-03-19T01:23:48.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.280+0000 7f9e96a00640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e880b0c00 0x7f9e880b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.280+0000 7f9e96a00640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9e780a2a50 0x7f9e780a4f10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.280+0000 7f9e96a00640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e880ac100 0x7f9e88144780 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.280+0000 7f9e96a00640 1 --2- 172.21.15.69:0/3319285594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e880a4fc0 0x7f9e88144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:48.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.280+0000 7f9e96a00640 1 -- 172.21.15.69:0/3319285594 >> 172.21.15.69:0/3319285594 conn(0x7f9e8801a640 msgr2=0x7f9e880a4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:48.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.280+0000 7f9e96a00640 1 -- 172.21.15.69:0/3319285594 shutdown_connections 2024-03-19T01:23:48.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:48.281+0000 7f9e96a00640 1 -- 172.21.15.69:0/3319285594 wait complete. 2024-03-19T01:23:48.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:48 smithi086 ceph-mon[66829]: pgmap v410: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-03-19T01:23:48.506 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_gem.o 2024-03-19T01:23:48.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:48 smithi069 ceph-mon[31727]: pgmap v410: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-03-19T01:23:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:48 smithi163 ceph-mon[67428]: pgmap v410: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-03-19T01:23:48.569 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-io.o 2024-03-19T01:23:48.696 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_busy.o 2024-03-19T01:23:49.214 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/tegra/built-in.a 2024-03-19T01:23:49.246 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/random.o 2024-03-19T01:23:49.262 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/randomalloc.o 2024-03-19T01:23:49.272 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/platform/x86/p2sb.o 2024-03-19T01:23:49.349 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:23:49.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:23:49.350 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:23:49.638 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/hid/usbhid/built-in.a 2024-03-19T01:23:49.789 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/xilinx/built-in.a 2024-03-19T01:23:49.853 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/pci.o 2024-03-19T01:23:49.859 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-kcopyd.o 2024-03-19T01:23:49.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.895+0000 7fe1a8000640 1 -- 172.21.15.69:0/3297968852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a0109320 msgr2=0x7fe1a0114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:49.897 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-quirks.o 2024-03-19T01:23:49.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.895+0000 7fe1a8000640 1 --2- 172.21.15.69:0/3297968852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a0109320 0x7fe1a0114430 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009900 tx=0x7fe18c02bc40 comp rx=0 tx=0).stop 2024-03-19T01:23:49.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.898+0000 7fe1a8000640 1 -- 172.21.15.69:0/3297968852 shutdown_connections 2024-03-19T01:23:49.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.898+0000 7fe1a8000640 1 --2- 172.21.15.69:0/3297968852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1a0114970 0x7fe1a0116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.898+0000 7fe1a8000640 1 --2- 172.21.15.69:0/3297968852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a0109320 0x7fe1a0114430 secure :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009900 tx=0x7fe18c02bc40 comp rx=0 tx=0).stop 2024-03-19T01:23:49.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.898+0000 7fe1a8000640 1 --2- 172.21.15.69:0/3297968852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1a0108a00 0x7fe1a0108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.898+0000 7fe1a8000640 1 -- 172.21.15.69:0/3297968852 >> 172.21.15.69:0/3297968852 conn(0x7fe1a006c900 msgr2=0x7fe1a006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:49.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.900+0000 7fe1a8000640 1 -- 172.21.15.69:0/3297968852 shutdown_connections 2024-03-19T01:23:49.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.900+0000 7fe1a8000640 1 -- 172.21.15.69:0/3297968852 wait complete. 2024-03-19T01:23:49.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.901+0000 7fe1a8000640 1 Processor -- start 2024-03-19T01:23:49.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.904+0000 7fe1a8000640 1 -- start start 2024-03-19T01:23:49.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.906+0000 7fe1a8000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1a0108a00 0x7fe1a0113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:49.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.906+0000 7fe1a8000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a0114970 0x7fe1a010b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:49.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.906+0000 7fe1a8000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1a010baa0 0x7fe1a010bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:49.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.906+0000 7fe1a8000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a011a3b0 con 0x7fe1a010baa0 2024-03-19T01:23:49.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.906+0000 7fe1a8000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a011a230 con 0x7fe1a0114970 2024-03-19T01:23:49.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.906+0000 7fe1a8000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a011a530 con 0x7fe1a0108a00 2024-03-19T01:23:49.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.906+0000 7fe1a4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a0114970 0x7fe1a010b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:49.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.906+0000 7fe1a4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a0114970 0x7fe1a010b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36158/0 (socket says 172.21.15.69:36158) 2024-03-19T01:23:49.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.906+0000 7fe1a4c00640 1 -- 172.21.15.69:0/1584473320 learned_addr learned my addr 172.21.15.69:0/1584473320 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:49.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.906+0000 7fe1a5600640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1a0108a00 0x7fe1a0113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:49.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.907+0000 7fe1a6000640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1a010baa0 0x7fe1a010bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:49.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.907+0000 7fe1a4c00640 1 -- 172.21.15.69:0/1584473320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1a0108a00 msgr2=0x7fe1a0113f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:49.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.907+0000 7fe1a4c00640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1a0108a00 0x7fe1a0113f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.907+0000 7fe1a4c00640 1 -- 172.21.15.69:0/1584473320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1a010baa0 msgr2=0x7fe1a010bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:49.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.907+0000 7fe1a4c00640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1a010baa0 0x7fe1a010bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.907+0000 7fe1a4c00640 1 -- 172.21.15.69:0/1584473320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a010fad0 con 0x7fe1a0114970 2024-03-19T01:23:49.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.907+0000 7fe1a6000640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1a010baa0 0x7fe1a010bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:49.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.907+0000 7fe1a4c00640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a0114970 0x7fe1a010b560 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe18c02c1d0 tx=0x7fe18c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:49.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.908+0000 7fe19e000640 1 -- 172.21.15.69:0/1584473320 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c041070 con 0x7fe1a0114970 2024-03-19T01:23:49.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.908+0000 7fe19e000640 1 -- 172.21.15.69:0/1584473320 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c0054e0 con 0x7fe1a0114970 2024-03-19T01:23:49.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.908+0000 7fe19e000640 1 -- 172.21.15.69:0/1584473320 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c034440 con 0x7fe1a0114970 2024-03-19T01:23:49.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.908+0000 7fe1a8000640 1 -- 172.21.15.69:0/1584473320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a010fd60 con 0x7fe1a0114970 2024-03-19T01:23:49.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.908+0000 7fe1a8000640 1 -- 172.21.15.69:0/1584473320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a0110110 con 0x7fe1a0114970 2024-03-19T01:23:49.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.923+0000 7fe1aad90740 1 Processor -- start 2024-03-19T01:23:49.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.923+0000 7fe19e000640 1 -- 172.21.15.69:0/1584473320 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fe18c044070 con 0x7fe1a0114970 2024-03-19T01:23:49.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.924+0000 7fe1aad90740 1 -- start start 2024-03-19T01:23:49.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.924+0000 7fe1aad90740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f004b00 0x55a98f004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:49.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.925+0000 7fe1aad90740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f005420 0x55a98f00dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:49.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.925+0000 7fe1a6000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f004b00 0x55a98f004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:49.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.925+0000 7fe1a6000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f004b00 0x55a98f004ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45342/0 (socket says 172.21.15.69:45342) 2024-03-19T01:23:49.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.925+0000 7fe1a6000640 1 -- 172.21.15.69:0/3668372594 learned_addr learned my addr 172.21.15.69:0/3668372594 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:49.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.925+0000 7fe1aad90740 1 --2- 172.21.15.69:0/3668372594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f00e000 0x55a98f0103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:49.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.925+0000 7fe1aad90740 1 -- 172.21.15.69:0/3668372594 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a98ef67fb0 con 0x55a98f00e000 2024-03-19T01:23:49.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.926+0000 7fe1aad90740 1 -- 172.21.15.69:0/3668372594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a98ef10ab0 con 0x55a98f005420 2024-03-19T01:23:49.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.926+0000 7fe1aad90740 1 -- 172.21.15.69:0/3668372594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a98ef64050 con 0x55a98f004b00 2024-03-19T01:23:49.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.926+0000 7fe19e000640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1880a2b80 0x7fe1880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:49.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.926+0000 7fe19e000640 1 -- 172.21.15.69:0/1584473320 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7fe18c0f0fc0 con 0x7fe1a0114970 2024-03-19T01:23:49.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.926+0000 7fe1a6000640 1 -- 172.21.15.69:0/3668372594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f005420 msgr2=0x55a98f00dac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:49.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.926+0000 7fe1a6000640 1 --2- 172.21.15.69:0/3668372594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f005420 0x55a98f00dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.926+0000 7fe1a6000640 1 -- 172.21.15.69:0/3668372594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f00e000 msgr2=0x55a98f0103f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:49.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.926+0000 7fe1a6000640 1 --2- 172.21.15.69:0/3668372594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f00e000 0x55a98f0103f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.926+0000 7fe1a6000640 1 -- 172.21.15.69:0/3668372594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98ef2ee00 con 0x55a98f004b00 2024-03-19T01:23:49.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.926+0000 7fe1a6000640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1880a2b80 0x7fe1880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:49.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.926+0000 7fe1a6000640 1 --2- 172.21.15.69:0/3668372594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f004b00 0x55a98f004ee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe19400bc50 tx=0x7fe19400d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4f31e2636b7316e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:49.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.927+0000 7fe192600640 1 -- 172.21.15.69:0/3668372594 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194011690 con 0x55a98f004b00 2024-03-19T01:23:49.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.927+0000 7fe192600640 1 -- 172.21.15.69:0/3668372594 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe194011d90 con 0x55a98f004b00 2024-03-19T01:23:49.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.927+0000 7fe192600640 1 -- 172.21.15.69:0/3668372594 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194010630 con 0x55a98f004b00 2024-03-19T01:23:49.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.928+0000 7fe1aad90740 1 -- 172.21.15.69:0/3668372594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f004b00 msgr2=0x55a98f004ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:49.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.928+0000 7fe1aad90740 1 --2- 172.21.15.69:0/3668372594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f004b00 0x55a98f004ee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe19400bc50 tx=0x7fe19400d710 comp rx=0 tx=0).stop 2024-03-19T01:23:49.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.928+0000 7fe1a6000640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1880a2b80 0x7fe1880a5040 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe19401d000 tx=0x7fe1940109f0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:49.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.928+0000 7fe1aad90740 1 -- 172.21.15.69:0/3668372594 shutdown_connections 2024-03-19T01:23:49.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.928+0000 7fe1aad90740 1 --2- 172.21.15.69:0/3668372594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f00e000 0x55a98f0103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.928+0000 7fe1aad90740 1 --2- 172.21.15.69:0/3668372594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f005420 0x55a98f00dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.928+0000 7fe1aad90740 1 --2- 172.21.15.69:0/3668372594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f004b00 0x55a98f004ee0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.928+0000 7fe1aad90740 1 -- 172.21.15.69:0/3668372594 >> 172.21.15.69:0/3668372594 conn(0x55a98efff230 msgr2=0x55a98efff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:49.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.934+0000 7fe1aad90740 1 -- 172.21.15.69:0/3668372594 shutdown_connections 2024-03-19T01:23:49.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.934+0000 7fe1aad90740 1 -- 172.21.15.69:0/3668372594 wait complete. 2024-03-19T01:23:49.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.934+0000 7fe1aad90740 1 -- start start 2024-03-19T01:23:49.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.935+0000 7fe1aad90740 1 Processor -- start 2024-03-19T01:23:49.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.939+0000 7fe1aad90740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f004b00 0x55a98f00b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:49.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.939+0000 7fe1aad90740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:49.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.939+0000 7fe1aad90740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f018290 0x55a98f01a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:49.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.939+0000 7fe1aad90740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a98f010df0 con 0x55a98f004b00 2024-03-19T01:23:49.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.939+0000 7fe1aad90740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a98ef23f00 con 0x55a98f018290 2024-03-19T01:23:49.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.939+0000 7fe1aad90740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a98f010f70 con 0x55a98f00b890 2024-03-19T01:23:49.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.939+0000 7fe1aad90740 10 client.0 _ll_register_callbacks cb 0x55a98ef238d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:49.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.940+0000 7fe1a4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:49.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.940+0000 7fe1a4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45354/0 (socket says 172.21.15.69:45354) 2024-03-19T01:23:49.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.940+0000 7fe1a4c00640 1 -- 172.21.15.69:0/1141521911 learned_addr learned my addr 172.21.15.69:0/1141521911 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:49.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.940+0000 7fe1a5600640 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f004b00 0x55a98f00b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:49.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.940+0000 7fe1a4c00640 1 -- 172.21.15.69:0/1141521911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f018290 msgr2=0x55a98f01a6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:23:49.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.940+0000 7fe1a4c00640 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f018290 0x55a98f01a6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.940+0000 7fe1a4c00640 1 -- 172.21.15.69:0/1141521911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f004b00 msgr2=0x55a98f00b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:49.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.940+0000 7fe1a4c00640 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f004b00 0x55a98f00b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.940+0000 7fe1a4c00640 1 -- 172.21.15.69:0/1141521911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98ed447e0 con 0x55a98f00b890 2024-03-19T01:23:49.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.940+0000 7fe1a5600640 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f004b00 0x55a98f00b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:49.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.940+0000 7fe1a4c00640 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00bd10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0bd270 tx=0x7fe18c0b8f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9340f2d2cd47d69 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:49.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.942+0000 7fe186a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:23:49.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.942+0000 7fe186a00640 1 -- 172.21.15.69:0/1141521911 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c04b030 con 0x55a98f00b890 2024-03-19T01:23:49.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.942+0000 7fe186a00640 1 -- 172.21.15.69:0/1141521911 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7fe18c0c1080 con 0x55a98f00b890 2024-03-19T01:23:49.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.942+0000 7fe1aad90740 1 -- 172.21.15.69:0/1141521911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe194011d90 con 0x55a98f00b890 2024-03-19T01:23:49.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.942+0000 7fe186a00640 1 -- 172.21.15.69:0/1141521911 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe18c0bcc40 con 0x55a98f00b890 2024-03-19T01:23:49.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.942+0000 7fe1aad90740 10 client.27991 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:49.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.942+0000 7fe1aad90740 1 -- 172.21.15.69:0/1141521911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55a98eda4f70 con 0x55a98f00b890 2024-03-19T01:23:49.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.943+0000 7fe186a00640 1 -- 172.21.15.69:0/1141521911 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c0bd9e0 con 0x55a98f00b890 2024-03-19T01:23:49.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.943+0000 7fe186a00640 1 -- 172.21.15.69:0/1141521911 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe18c003680 con 0x55a98f00b890 2024-03-19T01:23:49.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.943+0000 7fe1aad90740 10 client.27991 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:49.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.943+0000 7fe1aad90740 10 client.27991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:23:49.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.943+0000 7fe1aad90740 20 client.27991 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:49.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.943+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a98f01ef10 0x55a98f021330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:49.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.943+0000 7fe1aad90740 4 client.27991 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:23:49.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.943+0000 7fe1aad90740 1 -- 172.21.15.69:0/1141521911 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe194011d90 con 0x55a98f01ef10 2024-03-19T01:23:49.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.943+0000 7fe1a5600640 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a98f01ef10 0x55a98f021330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:49.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.944+0000 7fe1a5600640 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a98f01ef10 0x55a98f021330 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cbeb3a5a50fa830 server_cookie=7c7cd3c85c769965 in_seq=0 out_seq=0 2024-03-19T01:23:49.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.944+0000 7fe186a00640 10 client.27991 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:49.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.945+0000 7fe186a00640 1 -- 172.21.15.69:0/1141521911 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fe194011d90 con 0x55a98f01ef10 2024-03-19T01:23:49.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.945+0000 7fe186a00640 10 client.27991 handle_command_reply: tid=0 2024-03-19T01:23:49.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.945+0000 7fe1aad90740 1 client.27991 shutdown 2024-03-19T01:23:49.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.949+0000 7fe1aad90740 1 -- 172.21.15.69:0/1141521911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 msgr2=0x55a98f00bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:49.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.949+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00bd10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0bd270 tx=0x7fe18c0b8f20 comp rx=0 tx=0).stop 2024-03-19T01:23:49.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.949+0000 7fe1aad90740 1 -- 172.21.15.69:0/1141521911 shutdown_connections 2024-03-19T01:23:49.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.949+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a98f01ef10 0x55a98f021330 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.949+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f018290 0x55a98f01a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.949+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00bd10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.949+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1141521911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f004b00 0x55a98f00b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:49.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.949+0000 7fe186a00640 0 client.27991 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:49.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.949+0000 7fe1aad90740 1 -- 172.21.15.69:0/1141521911 >> 172.21.15.69:0/1141521911 conn(0x55a98efff230 msgr2=0x55a98f000f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:49.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.949+0000 7fe1aad90740 1 -- 172.21.15.69:0/1141521911 shutdown_connections 2024-03-19T01:23:49.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.952+0000 7fe1aad90740 1 -- 172.21.15.69:0/1141521911 wait complete. 2024-03-19T01:23:49.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:49.952+0000 7fe1aad90740 20 client.27991 trim_cache size 0 max 16384 2024-03-19T01:23:50.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.028+0000 7fe1aad90740 1 Processor -- start 2024-03-19T01:23:50.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.030+0000 7fe1aad90740 1 -- start start 2024-03-19T01:23:50.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.030+0000 7fe1aad90740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f026c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.030+0000 7fe1aad90740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.030+0000 7fe1aad90740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f018290 0x55a98f00b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.030+0000 7fe1aad90740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a01c68c0 con 0x55a98f018290 2024-03-19T01:23:50.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.030+0000 7fe1aad90740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a01c8400 con 0x55a98f004b00 2024-03-19T01:23:50.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.030+0000 7fe1aad90740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a01c6f90 con 0x55a98f00b890 2024-03-19T01:23:50.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.031+0000 7fe1a4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.031+0000 7fe1a5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f026c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.031+0000 7fe1a5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f026c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36162/0 (socket says 172.21.15.69:36162) 2024-03-19T01:23:50.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.031+0000 7fe1a4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45358/0 (socket says 172.21.15.69:45358) 2024-03-19T01:23:50.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.031+0000 7fe1a5600640 1 -- 172.21.15.69:0/3860728546 learned_addr learned my addr 172.21.15.69:0/3860728546 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:50.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.031+0000 7fe1a5600640 1 -- 172.21.15.69:0/3860728546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 msgr2=0x55a98f00ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:50.033 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_clflush.o 2024-03-19T01:23:50.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.031+0000 7fe1a5600640 1 --2- 172.21.15.69:0/3860728546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.031+0000 7fe1a5600640 1 -- 172.21.15.69:0/3860728546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f018290 msgr2=0x55a98f00b170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:50.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.032+0000 7fe1a5600640 1 --2- 172.21.15.69:0/3860728546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f018290 0x55a98f00b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.032+0000 7fe1a5600640 1 -- 172.21.15.69:0/3860728546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98ec3c170 con 0x55a98f004b00 2024-03-19T01:23:50.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.032+0000 7fe1a4c00640 1 --2- 172.21.15.69:0/3860728546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:23:50.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.032+0000 7fe1a5600640 1 --2- 172.21.15.69:0/3860728546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f026c00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55a98f005940 tx=0x7fe198004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593d3e16227ad824 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:50.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.033+0000 7fe191c00640 1 -- 172.21.15.69:0/3860728546 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19801e070 con 0x55a98f004b00 2024-03-19T01:23:50.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.033+0000 7fe191c00640 1 -- 172.21.15.69:0/3860728546 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe194011d90 con 0x55a98f004b00 2024-03-19T01:23:50.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.034+0000 7fe1aad90740 1 -- 172.21.15.69:0/3860728546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 msgr2=0x55a98f026c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:50.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.034+0000 7fe1aad90740 1 --2- 172.21.15.69:0/3860728546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f026c00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55a98f005940 tx=0x7fe198004ab0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.033+0000 7fe191c00640 1 -- 172.21.15.69:0/3860728546 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198010990 con 0x55a98f004b00 2024-03-19T01:23:50.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.038+0000 7fe1aad90740 1 -- 172.21.15.69:0/3860728546 shutdown_connections 2024-03-19T01:23:50.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.038+0000 7fe1aad90740 1 --2- 172.21.15.69:0/3860728546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f018290 0x55a98f00b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.038+0000 7fe1aad90740 1 --2- 172.21.15.69:0/3860728546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f00ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.038+0000 7fe1aad90740 1 --2- 172.21.15.69:0/3860728546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f026c00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.038+0000 7fe1aad90740 1 -- 172.21.15.69:0/3860728546 >> 172.21.15.69:0/3860728546 conn(0x55a98efff230 msgr2=0x55a98f000f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:50.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.041+0000 7fe1aad90740 1 -- 172.21.15.69:0/3860728546 shutdown_connections 2024-03-19T01:23:50.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.041+0000 7fe1aad90740 1 -- 172.21.15.69:0/3860728546 wait complete. 2024-03-19T01:23:50.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.041+0000 7fe1aad90740 1 -- start start 2024-03-19T01:23:50.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.041+0000 7fe1aad90740 1 Processor -- start 2024-03-19T01:23:50.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.042+0000 7fe1aad90740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f013270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.042+0000 7fe1a5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f013270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.042+0000 7fe1a5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f013270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36168/0 (socket says 172.21.15.69:36168) 2024-03-19T01:23:50.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.043+0000 7fe1aad90740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f0137b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.043+0000 7fe1a5600640 1 -- 172.21.15.69:0/1328551296 learned_addr learned my addr 172.21.15.69:0/1328551296 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:50.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.043+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f018290 0x55a98f013cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.043+0000 7fe1aad90740 1 -- 172.21.15.69:0/1328551296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a01c7d30 con 0x55a98f018290 2024-03-19T01:23:50.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.043+0000 7fe1aad90740 1 -- 172.21.15.69:0/1328551296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a01c7660 con 0x55a98f004b00 2024-03-19T01:23:50.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.043+0000 7fe1aad90740 1 -- 172.21.15.69:0/1328551296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a01c48a0 con 0x55a98f00b890 2024-03-19T01:23:50.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.043+0000 7fe1aad90740 10 client.0 _ll_register_callbacks cb 0x7fe18c005740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:50.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.044+0000 7fe1a6000640 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f018290 0x55a98f013cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.045+0000 7fe1a6000640 1 -- 172.21.15.69:0/1328551296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 msgr2=0x55a98f0137b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:50.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.045+0000 7fe1a6000640 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f0137b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.045+0000 7fe1a6000640 1 -- 172.21.15.69:0/1328551296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 msgr2=0x55a98f013270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:50.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.045+0000 7fe1a6000640 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f013270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.045+0000 7fe1a6000640 1 -- 172.21.15.69:0/1328551296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98f084280 con 0x55a98f018290 2024-03-19T01:23:50.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.045+0000 7fe1a5600640 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f013270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:23:50.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.046+0000 7fe1a6000640 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f018290 0x55a98f013cf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55a98f00c8f0 tx=0x7fe19400b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83da176ab730984 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:50.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.046+0000 7fe186a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:23:50.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.048+0000 7fe186a00640 1 -- 172.21.15.69:0/1328551296 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194030070 con 0x55a98f018290 2024-03-19T01:23:50.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.048+0000 7fe186a00640 1 -- 172.21.15.69:0/1328551296 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7fe194011980 con 0x55a98f018290 2024-03-19T01:23:50.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.048+0000 7fe1aad90740 1 -- 172.21.15.69:0/1328551296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe194011d90 con 0x55a98f018290 2024-03-19T01:23:50.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.049+0000 7fe186a00640 1 -- 172.21.15.69:0/1328551296 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe19400be60 con 0x55a98f018290 2024-03-19T01:23:50.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.049+0000 7fe1aad90740 10 client.17658 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:50.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.049+0000 7fe1aad90740 1 -- 172.21.15.69:0/1328551296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55a98f085610 con 0x55a98f018290 2024-03-19T01:23:50.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.049+0000 7fe186a00640 1 -- 172.21.15.69:0/1328551296 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe194011d90 con 0x55a98f018290 2024-03-19T01:23:50.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.049+0000 7fe186a00640 1 -- 172.21.15.69:0/1328551296 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fe1940047a0 con 0x55a98f018290 2024-03-19T01:23:50.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.050+0000 7fe1aad90740 10 client.17658 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:50.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.050+0000 7fe1aad90740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.050+0000 7fe1aad90740 20 client.17658 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.050+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a98f085ce0 0x55a98f088100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.050+0000 7fe1aad90740 4 client.17658 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.050+0000 7fe1aad90740 1 -- 172.21.15.69:0/1328551296 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a98ef754f0 con 0x55a98f085ce0 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.050+0000 7fe1a5600640 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a98f085ce0 0x55a98f088100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.051+0000 7fe186a00640 10 client.17658 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.051+0000 7fe1a5600640 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a98f085ce0 0x55a98f088100 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1939562ba1d48e6 server_cookie=8492c7d56eb1d48 in_seq=0 out_seq=0 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.052+0000 7fe186a00640 1 -- 172.21.15.69:0/1328551296 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a98ef754f0 con 0x55a98f085ce0 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.052+0000 7fe186a00640 10 client.17658 handle_command_reply: tid=0 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.052+0000 7fe1aad90740 1 client.17658 shutdown 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.053+0000 7fe1aad90740 1 -- 172.21.15.69:0/1328551296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f018290 msgr2=0x55a98f013cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:50.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.053+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f018290 0x55a98f013cf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55a98f00c8f0 tx=0x7fe19400b3f0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.053+0000 7fe1aad90740 1 -- 172.21.15.69:0/1328551296 shutdown_connections 2024-03-19T01:23:50.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.053+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55a98f085ce0 0x55a98f088100 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.053+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a98f018290 0x55a98f013cf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.053+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a98f00b890 0x55a98f0137b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.053+0000 7fe186a00640 0 client.17658 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:50.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.053+0000 7fe1aad90740 1 --2- 172.21.15.69:0/1328551296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a98f004b00 0x55a98f013270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.053+0000 7fe1aad90740 1 -- 172.21.15.69:0/1328551296 >> 172.21.15.69:0/1328551296 conn(0x55a98efff230 msgr2=0x55a98f00cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:50.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.054+0000 7fe1aad90740 1 -- 172.21.15.69:0/1328551296 shutdown_connections 2024-03-19T01:23:50.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.054+0000 7fe1aad90740 1 -- 172.21.15.69:0/1328551296 wait complete. 2024-03-19T01:23:50.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.054+0000 7fe1aad90740 20 client.17658 trim_cache size 0 max 16384 2024-03-19T01:23:50.065 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:23:50.066 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:23:50.066 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "3840d255-9333-4b4a-afac-4a6d116119e6", 2024-03-19T01:23:50.066 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:23:50.066 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:23:50.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 -- 172.21.15.69:0/1584473320 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1880a2b80 msgr2=0x7fe1880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:50.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1880a2b80 0x7fe1880a5040 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe19401d000 tx=0x7fe1940109f0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 -- 172.21.15.69:0/1584473320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a0114970 msgr2=0x7fe1a010b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:50.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a0114970 0x7fe1a010b560 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe18c02c1d0 tx=0x7fe18c0045e0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 -- 172.21.15.69:0/1584473320 shutdown_connections 2024-03-19T01:23:50.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1a010baa0 0x7fe1a010bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1880a2b80 0x7fe1880a5040 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1a0114970 0x7fe1a010b560 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 --2- 172.21.15.69:0/1584473320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1a0108a00 0x7fe1a0113f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 -- 172.21.15.69:0/1584473320 >> 172.21.15.69:0/1584473320 conn(0x7fe1a006c900 msgr2=0x7fe1a010a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:50.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 -- 172.21.15.69:0/1584473320 shutdown_connections 2024-03-19T01:23:50.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.070+0000 7fe1a8000640 1 -- 172.21.15.69:0/1584473320 wait complete. 2024-03-19T01:23:50.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:23:50.147 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:23:50.245 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/reboot.o 2024-03-19T01:23:50.300 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_ioctl.o 2024-03-19T01:23:50.386 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:50 smithi069 ceph-mon[31727]: pgmap v411: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-19T01:23:50.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:50 smithi086 ceph-mon[66829]: pgmap v411: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-19T01:23:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:50 smithi163 ceph-mon[67428]: pgmap v411: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-19T01:23:50.587 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/platform/x86/built-in.a 2024-03-19T01:23:50.664 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/platform/built-in.a 2024-03-19T01:23:50.857 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/dmi_scan.o 2024-03-19T01:23:50.873 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_context.o 2024-03-19T01:23:50.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.905+0000 7f02cbc00640 1 -- 172.21.15.69:0/1997378952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4114970 msgr2=0x7f02c4116d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:50.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.905+0000 7f02cbc00640 1 --2- 172.21.15.69:0/1997378952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4114970 0x7f02c4116d60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009900 tx=0x7f02b002bc60 comp rx=0 tx=0).stop 2024-03-19T01:23:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.910+0000 7f02cbc00640 1 -- 172.21.15.69:0/1997378952 shutdown_connections 2024-03-19T01:23:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.910+0000 7f02cbc00640 1 --2- 172.21.15.69:0/1997378952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4114970 0x7f02c4116d60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.910+0000 7f02cbc00640 1 --2- 172.21.15.69:0/1997378952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02c4109320 0x7f02c4114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.910+0000 7f02cbc00640 1 --2- 172.21.15.69:0/1997378952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c4108a00 0x7f02c4108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.910+0000 7f02cbc00640 1 -- 172.21.15.69:0/1997378952 >> 172.21.15.69:0/1997378952 conn(0x7f02c406c900 msgr2=0x7f02c406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.913+0000 7f02cbc00640 1 -- 172.21.15.69:0/1997378952 shutdown_connections 2024-03-19T01:23:50.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.915+0000 7f02cbc00640 1 -- 172.21.15.69:0/1997378952 wait complete. 2024-03-19T01:23:50.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.917+0000 7f02cbc00640 1 Processor -- start 2024-03-19T01:23:50.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.919+0000 7f02cbc00640 1 -- start start 2024-03-19T01:23:50.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.920+0000 7f02cbc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4108a00 0x7f02c4111730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.922+0000 7f02cbc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c4109320 0x7f02c4111c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.922+0000 7f02cbc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02c4114970 0x7f02c41121b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.922+0000 7f02cbc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02c410bf60 con 0x7f02c4114970 2024-03-19T01:23:50.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.922+0000 7f02cbc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02c410bde0 con 0x7f02c4108a00 2024-03-19T01:23:50.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.922+0000 7f02cbc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f02c410c0e0 con 0x7f02c4109320 2024-03-19T01:23:50.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.922+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c4109320 0x7f02c4111c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.922+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c4109320 0x7f02c4111c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:45378/0 (socket says 172.21.15.69:45378) 2024-03-19T01:23:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.922+0000 7f02c9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4108a00 0x7f02c4111730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.922+0000 7f02c3e00640 1 -- 172.21.15.69:0/3803386616 learned_addr learned my addr 172.21.15.69:0/3803386616 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.923+0000 7f02c9c00640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02c4114970 0x7f02c41121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.924+0000 7f02c9200640 1 -- 172.21.15.69:0/3803386616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c4109320 msgr2=0x7f02c4111c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.924+0000 7f02c9200640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c4109320 0x7f02c4111c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.924+0000 7f02c9200640 1 -- 172.21.15.69:0/3803386616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02c4114970 msgr2=0x7f02c41121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.924+0000 7f02c9200640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02c4114970 0x7f02c41121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.924+0000 7f02c9200640 1 -- 172.21.15.69:0/3803386616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c41b31b0 con 0x7f02c4108a00 2024-03-19T01:23:50.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.925+0000 7f02c9c00640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02c4114970 0x7f02c41121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:23:50.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.927+0000 7f02c9200640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4108a00 0x7f02c4111730 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00be20 tx=0x7f02bc009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.928+0000 7f02c1600640 1 -- 172.21.15.69:0/3803386616 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc019070 con 0x7f02c4108a00 2024-03-19T01:23:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.928+0000 7f02c1600640 1 -- 172.21.15.69:0/3803386616 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02bc014920 con 0x7f02c4108a00 2024-03-19T01:23:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.928+0000 7f02c1600640 1 -- 172.21.15.69:0/3803386616 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc012990 con 0x7f02c4108a00 2024-03-19T01:23:50.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.929+0000 7f02cbc00640 1 -- 172.21.15.69:0/3803386616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c41b34a0 con 0x7f02c4108a00 2024-03-19T01:23:50.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.929+0000 7f02cbc00640 1 -- 172.21.15.69:0/3803386616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c41b39a0 con 0x7f02c4108a00 2024-03-19T01:23:50.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.942+0000 7f02c1600640 1 -- 172.21.15.69:0/3803386616 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f02bc01c070 con 0x7f02c4108a00 2024-03-19T01:23:50.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.942+0000 7f02ce990740 1 Processor -- start 2024-03-19T01:23:50.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.944+0000 7f02ce990740 1 -- start start 2024-03-19T01:23:50.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.945+0000 7f02ce990740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42431d40 0x55ec42432120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.945+0000 7f02ce990740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42432660 0x55ec4243ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.945+0000 7f02ce990740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 0x55ec4243d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.945+0000 7f02ce990740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ec423962c0 con 0x55ec42431d40 2024-03-19T01:23:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.945+0000 7f02ce990740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec4233eb60 con 0x55ec42432660 2024-03-19T01:23:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.945+0000 7f02ce990740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ec42392360 con 0x55ec4243b240 2024-03-19T01:23:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.947+0000 7f02c9c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42431d40 0x55ec42432120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.947+0000 7f02c9c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42431d40 0x55ec42432120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34964/0 (socket says 172.21.15.69:34964) 2024-03-19T01:23:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.947+0000 7f02c9c00640 1 -- 172.21.15.69:0/1425817010 learned_addr learned my addr 172.21.15.69:0/1425817010 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:50.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.949+0000 7f02c9c00640 1 -- 172.21.15.69:0/1425817010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 msgr2=0x55ec4243d630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:50.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.949+0000 7f02c9c00640 1 --2- 172.21.15.69:0/1425817010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 0x55ec4243d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.949+0000 7f02c9c00640 1 -- 172.21.15.69:0/1425817010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42432660 msgr2=0x55ec4243ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:23:50.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.949+0000 7f02c9c00640 1 --2- 172.21.15.69:0/1425817010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42432660 0x55ec4243ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.949+0000 7f02c9c00640 1 -- 172.21.15.69:0/1425817010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec421747d0 con 0x55ec42431d40 2024-03-19T01:23:50.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.951+0000 7f02c1600640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02a80a2ab0 0x7f02a80a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.951+0000 7f02c1600640 1 -- 172.21.15.69:0/3803386616 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f02bc0ce930 con 0x7f02c4108a00 2024-03-19T01:23:50.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.951+0000 7f02c9c00640 1 --2- 172.21.15.69:0/1425817010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42431d40 0x55ec42432120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f02c406ac40 tx=0x7f02b002c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfb304c7454e515 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:50.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.952+0000 7f02c9c00640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02a80a2ab0 0x7f02a80a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.956+0000 7f02c9c00640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02a80a2ab0 0x7f02a80a4f70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f02b0046000 tx=0x7f02b0041000 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:50.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.956+0000 7f02ba000640 1 -- 172.21.15.69:0/1425817010 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0042070 con 0x55ec42431d40 2024-03-19T01:23:50.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.956+0000 7f02ba000640 1 -- 172.21.15.69:0/1425817010 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b00054e0 con 0x55ec42431d40 2024-03-19T01:23:50.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.956+0000 7f02ba000640 1 -- 172.21.15.69:0/1425817010 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0003c50 con 0x55ec42431d40 2024-03-19T01:23:50.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.963+0000 7f02ce990740 1 -- 172.21.15.69:0/1425817010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42431d40 msgr2=0x55ec42432120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:50.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.963+0000 7f02ce990740 1 --2- 172.21.15.69:0/1425817010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42431d40 0x55ec42432120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f02c406ac40 tx=0x7f02b002c6b0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.963+0000 7f02ce990740 1 -- 172.21.15.69:0/1425817010 shutdown_connections 2024-03-19T01:23:50.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.963+0000 7f02ce990740 1 --2- 172.21.15.69:0/1425817010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 0x55ec4243d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.963+0000 7f02ce990740 1 --2- 172.21.15.69:0/1425817010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42432660 0x55ec4243ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.963+0000 7f02ce990740 1 --2- 172.21.15.69:0/1425817010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42431d40 0x55ec42432120 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.963+0000 7f02ce990740 1 -- 172.21.15.69:0/1425817010 >> 172.21.15.69:0/1425817010 conn(0x55ec4242c470 msgr2=0x55ec4242c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:50.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.964+0000 7f02ce990740 1 -- 172.21.15.69:0/1425817010 shutdown_connections 2024-03-19T01:23:50.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.964+0000 7f02ce990740 1 -- 172.21.15.69:0/1425817010 wait complete. 2024-03-19T01:23:50.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.964+0000 7f02ce990740 1 -- start start 2024-03-19T01:23:50.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.965+0000 7f02ce990740 1 Processor -- start 2024-03-19T01:23:50.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.972+0000 7f02ce990740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec42431d40 0x55ec424382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.974+0000 7f02ce990740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42432660 0x55ec42438820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.974+0000 7f02ce990740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec4243b240 0x55ec42438d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.974+0000 7f02ce990740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ec4243dfd0 con 0x55ec4243b240 2024-03-19T01:23:50.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.974+0000 7f02ce990740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec42351fc0 con 0x55ec42432660 2024-03-19T01:23:50.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.974+0000 7f02ce990740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ec4243e150 con 0x55ec42431d40 2024-03-19T01:23:50.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.974+0000 7f02ce990740 10 client.0 _ll_register_callbacks cb 0x55ec42351990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:50.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.974+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42432660 0x55ec42438820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.974+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42432660 0x55ec42438820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36180/0 (socket says 172.21.15.69:36180) 2024-03-19T01:23:50.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.974+0000 7f02c3e00640 1 -- 172.21.15.69:0/2073165576 learned_addr learned my addr 172.21.15.69:0/2073165576 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:50.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.975+0000 7f02c9c00640 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec4243b240 0x55ec42438d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.975+0000 7f02c3e00640 1 -- 172.21.15.69:0/2073165576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec42431d40 msgr2=0x55ec424382e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:23:50.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.975+0000 7f02c3e00640 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec42431d40 0x55ec424382e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.975+0000 7f02c3e00640 1 -- 172.21.15.69:0/2073165576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec4243b240 msgr2=0x55ec42438d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:50.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.975+0000 7f02c3e00640 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec4243b240 0x55ec42438d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.975+0000 7f02c3e00640 1 -- 172.21.15.69:0/2073165576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec420aa3f0 con 0x55ec42432660 2024-03-19T01:23:50.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.975+0000 7f02c9c00640 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec4243b240 0x55ec42438d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:23:50.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.976+0000 7f02c3e00640 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42432660 0x55ec42438820 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f02b400bdb0 tx=0x7f02b400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b22657779dda32f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:50.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.976+0000 7f02aea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:23:50.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.977+0000 7f02aea00640 1 -- 172.21.15.69:0/2073165576 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4004240 con 0x55ec42432660 2024-03-19T01:23:50.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.977+0000 7f02aea00640 1 -- 172.21.15.69:0/2073165576 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f02b40043e0 con 0x55ec42432660 2024-03-19T01:23:50.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.977+0000 7f02ce990740 1 -- 172.21.15.69:0/2073165576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02b00054e0 con 0x55ec42432660 2024-03-19T01:23:50.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.977+0000 7f02aea00640 1 -- 172.21.15.69:0/2073165576 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f02b4022050 con 0x55ec42432660 2024-03-19T01:23:50.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.978+0000 7f02ce990740 10 client.28067 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:50.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.978+0000 7f02ce990740 1 -- 172.21.15.69:0/2073165576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ec4206a0f0 con 0x55ec42432660 2024-03-19T01:23:50.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.979+0000 7f02aea00640 1 -- 172.21.15.69:0/2073165576 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b00054e0 con 0x55ec42432660 2024-03-19T01:23:50.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.979+0000 7f02aea00640 1 -- 172.21.15.69:0/2073165576 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f02b4011920 con 0x55ec42432660 2024-03-19T01:23:50.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.979+0000 7f02ce990740 10 client.28067 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:50.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.979+0000 7f02ce990740 10 client.28067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:23:50.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.979+0000 7f02ce990740 20 client.28067 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:50.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.979+0000 7f02ce990740 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ec4244bb30 0x55ec4244dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:50.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.979+0000 7f02ce990740 4 client.28067 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:23:50.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.979+0000 7f02ce990740 1 -- 172.21.15.69:0/2073165576 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec42337610 con 0x55ec4244bb30 2024-03-19T01:23:50.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.980+0000 7f02c9200640 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ec4244bb30 0x55ec4244dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:50.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.983+0000 7f02c9200640 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ec4244bb30 0x55ec4244dfd0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66fd963a092ecca7 server_cookie=d3e44d8713ade7dc in_seq=0 out_seq=0 2024-03-19T01:23:50.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.983+0000 7f02aea00640 10 client.28067 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:50.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.984+0000 7f02aea00640 1 -- 172.21.15.69:0/2073165576 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ec42337610 con 0x55ec4244bb30 2024-03-19T01:23:50.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.984+0000 7f02aea00640 10 client.28067 handle_command_reply: tid=0 2024-03-19T01:23:50.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.984+0000 7f02ce990740 1 client.28067 shutdown 2024-03-19T01:23:50.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.988+0000 7f02ce990740 1 -- 172.21.15.69:0/2073165576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42432660 msgr2=0x55ec42438820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:50.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.988+0000 7f02ce990740 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42432660 0x55ec42438820 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f02b400bdb0 tx=0x7f02b400f730 comp rx=0 tx=0).stop 2024-03-19T01:23:50.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.988+0000 7f02ce990740 1 -- 172.21.15.69:0/2073165576 shutdown_connections 2024-03-19T01:23:50.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.988+0000 7f02ce990740 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ec4244bb30 0x55ec4244dfd0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.988+0000 7f02ce990740 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec4243b240 0x55ec42438d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.988+0000 7f02aea00640 0 client.28067 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:50.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.989+0000 7f02ce990740 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42432660 0x55ec42438820 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.989+0000 7f02ce990740 1 --2- 172.21.15.69:0/2073165576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec42431d40 0x55ec424382e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:50.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.989+0000 7f02ce990740 1 -- 172.21.15.69:0/2073165576 >> 172.21.15.69:0/2073165576 conn(0x55ec4242c470 msgr2=0x55ec4242e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:50.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.992+0000 7f02ce990740 1 -- 172.21.15.69:0/2073165576 shutdown_connections 2024-03-19T01:23:50.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.993+0000 7f02ce990740 1 -- 172.21.15.69:0/2073165576 wait complete. 2024-03-19T01:23:50.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:50.994+0000 7f02ce990740 20 client.28067 trim_cache size 0 max 16384 2024-03-19T01:23:51.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.108+0000 7f02ce990740 1 Processor -- start 2024-03-19T01:23:51.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.110+0000 7f02ce990740 1 -- start start 2024-03-19T01:23:51.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.111+0000 7f02ce990740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42431d40 0x55ec4243aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:51.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.111+0000 7f02ce990740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 0x55ec42446e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:51.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.111+0000 7f02ce990740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 0x55ec424538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:51.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.111+0000 7f02ce990740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02a40101a0 con 0x55ec42432660 2024-03-19T01:23:51.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.111+0000 7f02ce990740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4011ce0 con 0x55ec42431d40 2024-03-19T01:23:51.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.111+0000 7f02ce990740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4010870 con 0x55ec4243b240 2024-03-19T01:23:51.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.111+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 0x55ec42446e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:51.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.111+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 0x55ec42446e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34984/0 (socket says 172.21.15.69:34984) 2024-03-19T01:23:51.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.111+0000 7f02c3e00640 1 -- 172.21.15.69:0/3662475606 learned_addr learned my addr 172.21.15.69:0/3662475606 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:51.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.112+0000 7f02c3e00640 1 -- 172.21.15.69:0/3662475606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 msgr2=0x55ec424538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:51.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.112+0000 7f02c3e00640 1 --2- 172.21.15.69:0/3662475606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 0x55ec424538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.112+0000 7f02c3e00640 1 -- 172.21.15.69:0/3662475606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42431d40 msgr2=0x55ec4243aa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:23:51.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.112+0000 7f02c3e00640 1 --2- 172.21.15.69:0/3662475606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42431d40 0x55ec4243aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.112+0000 7f02c3e00640 1 -- 172.21.15.69:0/3662475606 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec42432480 con 0x55ec42432660 2024-03-19T01:23:51.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.115+0000 7f02c3e00640 1 --2- 172.21.15.69:0/3662475606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 0x55ec42446e00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f02b40058c0 tx=0x7f02b4005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a70eac5ecef8e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:51.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.115+0000 7f02b9600640 1 -- 172.21.15.69:0/3662475606 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b401f040 con 0x55ec42432660 2024-03-19T01:23:51.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.115+0000 7f02b9600640 1 -- 172.21.15.69:0/3662475606 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b402b070 con 0x55ec42432660 2024-03-19T01:23:51.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.115+0000 7f02b9600640 1 -- 172.21.15.69:0/3662475606 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4019a50 con 0x55ec42432660 2024-03-19T01:23:51.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.116+0000 7f02ce990740 1 -- 172.21.15.69:0/3662475606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 msgr2=0x55ec42446e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:51.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.116+0000 7f02ce990740 1 --2- 172.21.15.69:0/3662475606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 0x55ec42446e00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f02b40058c0 tx=0x7f02b4005f30 comp rx=0 tx=0).stop 2024-03-19T01:23:51.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.119+0000 7f02ce990740 1 -- 172.21.15.69:0/3662475606 shutdown_connections 2024-03-19T01:23:51.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.119+0000 7f02ce990740 1 --2- 172.21.15.69:0/3662475606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 0x55ec424538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.119+0000 7f02ce990740 1 --2- 172.21.15.69:0/3662475606 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 0x55ec42446e00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.119+0000 7f02ce990740 1 --2- 172.21.15.69:0/3662475606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42431d40 0x55ec4243aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.119+0000 7f02ce990740 1 -- 172.21.15.69:0/3662475606 >> 172.21.15.69:0/3662475606 conn(0x55ec4242c470 msgr2=0x55ec42432ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:51.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.119+0000 7f02ce990740 1 -- 172.21.15.69:0/3662475606 shutdown_connections 2024-03-19T01:23:51.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.119+0000 7f02ce990740 1 -- 172.21.15.69:0/3662475606 wait complete. 2024-03-19T01:23:51.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.120+0000 7f02ce990740 1 -- start start 2024-03-19T01:23:51.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.120+0000 7f02ce990740 1 Processor -- start 2024-03-19T01:23:51.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.128+0000 7f02ce990740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42431d40 0x55ec42443790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:51.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.128+0000 7f02ce990740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 0x55ec4243eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:51.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.128+0000 7f02ce990740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 0x55ec4243f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:51.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.128+0000 7f02ce990740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4011610 con 0x55ec42432660 2024-03-19T01:23:51.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.128+0000 7f02ce990740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4010f40 con 0x55ec42431d40 2024-03-19T01:23:51.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.128+0000 7f02ce990740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f02a400e180 con 0x55ec4243b240 2024-03-19T01:23:51.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.128+0000 7f02ce990740 10 client.0 _ll_register_callbacks cb 0x7f02b400b970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:23:51.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.129+0000 7f02c9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42431d40 0x55ec42443790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:51.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.129+0000 7f02c9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42431d40 0x55ec42443790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:36196/0 (socket says 172.21.15.69:36196) 2024-03-19T01:23:51.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.129+0000 7f02c9200640 1 -- 172.21.15.69:0/2653894019 learned_addr learned my addr 172.21.15.69:0/2653894019 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:23:51.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.130+0000 7f02c3e00640 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 0x55ec4243eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:51.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.130+0000 7f02c3e00640 1 -- 172.21.15.69:0/2653894019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 msgr2=0x55ec4243f030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:23:51.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.130+0000 7f02c3e00640 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 0x55ec4243f030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.130+0000 7f02c3e00640 1 -- 172.21.15.69:0/2653894019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42431d40 msgr2=0x55ec42443790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:23:51.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.130+0000 7f02c3e00640 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42431d40 0x55ec42443790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.130+0000 7f02c3e00640 1 -- 172.21.15.69:0/2653894019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4243f860 con 0x55ec42432660 2024-03-19T01:23:51.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.131+0000 7f02c3e00640 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 0x55ec4243eab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55ec42352070 tx=0x7f02b40045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef2969fb535e93f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:23:51.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.131+0000 7f02aea00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:23:51.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.133+0000 7f02aea00640 1 -- 172.21.15.69:0/2653894019 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4005240 con 0x55ec42432660 2024-03-19T01:23:51.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.133+0000 7f02ce990740 1 -- 172.21.15.69:0/2653894019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02b402b070 con 0x55ec42432660 2024-03-19T01:23:51.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.134+0000 7f02aea00640 1 -- 172.21.15.69:0/2653894019 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9568+0+0 (secure 0 0 0) 0x7f02b40053e0 con 0x55ec42432660 2024-03-19T01:23:51.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.134+0000 7f02aea00640 1 -- 172.21.15.69:0/2653894019 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f02b4040050 con 0x55ec42432660 2024-03-19T01:23:51.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.136+0000 7f02ce990740 10 client.17676 fetch_fsmap learned FSMap version 59 2024-03-19T01:23:51.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.136+0000 7f02ce990740 1 -- 172.21.15.69:0/2653894019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ec424b1fc0 con 0x55ec42432660 2024-03-19T01:23:51.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.139+0000 7f02aea00640 1 -- 172.21.15.69:0/2653894019 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b402b070 con 0x55ec42432660 2024-03-19T01:23:51.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.139+0000 7f02aea00640 1 -- 172.21.15.69:0/2653894019 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f02b4011830 con 0x55ec42432660 2024-03-19T01:23:51.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.140+0000 7f02ce990740 10 client.17676 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:23:51.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.140+0000 7f02ce990740 10 client.17676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:23:51.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.140+0000 7f02ce990740 20 client.17676 populate_metadata read hostname 'smithi069' 2024-03-19T01:23:51.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.140+0000 7f02ce990740 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ec424b25c0 0x55ec424b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:23:51.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.140+0000 7f02ce990740 4 client.17676 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:23:51.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.140+0000 7f02ce990740 1 -- 172.21.15.69:0/2653894019 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ec423a3800 con 0x55ec424b25c0 2024-03-19T01:23:51.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.140+0000 7f02c9200640 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ec424b25c0 0x55ec424b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:23:51.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.142+0000 7f02c9200640 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ec424b25c0 0x55ec424b49b0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a204e1085492d7b1 server_cookie=8fcf145a894890b7 in_seq=0 out_seq=0 2024-03-19T01:23:51.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.142+0000 7f02aea00640 10 client.17676 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.143+0000 7f02aea00640 1 -- 172.21.15.69:0/2653894019 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55ec423a3800 con 0x55ec424b25c0 2024-03-19T01:23:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.143+0000 7f02aea00640 10 client.17676 handle_command_reply: tid=0 2024-03-19T01:23:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.143+0000 7f02ce990740 1 client.17676 shutdown 2024-03-19T01:23:51.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.143+0000 7f02ce990740 1 -- 172.21.15.69:0/2653894019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 msgr2=0x55ec4243eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:51.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.143+0000 7f02ce990740 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 0x55ec4243eab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55ec42352070 tx=0x7f02b40045b0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.149+0000 7f02ce990740 1 -- 172.21.15.69:0/2653894019 shutdown_connections 2024-03-19T01:23:51.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.149+0000 7f02ce990740 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55ec424b25c0 0x55ec424b49b0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.149+0000 7f02ce990740 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4243b240 0x55ec4243f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.149+0000 7f02ce990740 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec42432660 0x55ec4243eab0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.149+0000 7f02ce990740 1 --2- 172.21.15.69:0/2653894019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec42431d40 0x55ec42443790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.149+0000 7f02ce990740 1 -- 172.21.15.69:0/2653894019 >> 172.21.15.69:0/2653894019 conn(0x55ec4242c470 msgr2=0x55ec4242ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:51.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.150+0000 7f02aea00640 0 client.17676 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:23:51.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.151+0000 7f02ce990740 1 -- 172.21.15.69:0/2653894019 shutdown_connections 2024-03-19T01:23:51.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.152+0000 7f02ce990740 1 -- 172.21.15.69:0/2653894019 wait complete. 2024-03-19T01:23:51.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.152+0000 7f02ce990740 20 client.17676 trim_cache size 0 max 16384 2024-03-19T01:23:51.159 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:23:51.159 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (11037 inodes in the stack)", 2024-03-19T01:23:51.159 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:23:51.159 INFO:teuthology.orchestra.run.smithi069.stdout: "3840d255-9333-4b4a-afac-4a6d116119e6": { 2024-03-19T01:23:51.159 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:23:51.159 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "3840d255-9333-4b4a-afac-4a6d116119e6", 2024-03-19T01:23:51.159 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:23:51.159 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:23:51.160 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:23:51.160 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:23:51.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.159+0000 7f02cbc00640 1 -- 172.21.15.69:0/3803386616 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02a80a2ab0 msgr2=0x7f02a80a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:51.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.159+0000 7f02cbc00640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02a80a2ab0 0x7f02a80a4f70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f02b0046000 tx=0x7f02b0041000 comp rx=0 tx=0).stop 2024-03-19T01:23:51.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.159+0000 7f02cbc00640 1 -- 172.21.15.69:0/3803386616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4108a00 msgr2=0x7f02c4111730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:23:51.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.159+0000 7f02cbc00640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4108a00 0x7f02c4111730 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00be20 tx=0x7f02bc009f70 comp rx=0 tx=0).stop 2024-03-19T01:23:51.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.160+0000 7f02cbc00640 1 -- 172.21.15.69:0/3803386616 shutdown_connections 2024-03-19T01:23:51.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.160+0000 7f02cbc00640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02c4114970 0x7f02c41121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.160+0000 7f02cbc00640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c4109320 0x7f02c4111c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.160+0000 7f02cbc00640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f02a80a2ab0 0x7f02a80a4f70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.160+0000 7f02cbc00640 1 --2- 172.21.15.69:0/3803386616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02c4108a00 0x7f02c4111730 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:23:51.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.160+0000 7f02cbc00640 1 -- 172.21.15.69:0/3803386616 >> 172.21.15.69:0/3803386616 conn(0x7f02c406c900 msgr2=0x7f02c4070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:23:51.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.163+0000 7f02cbc00640 1 -- 172.21.15.69:0/3803386616 shutdown_connections 2024-03-19T01:23:51.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:23:51.163+0000 7f02cbc00640 1 -- 172.21.15.69:0/3803386616 wait complete. 2024-03-19T01:23:51.237 INFO:tasks.cephfs.filesystem:scrub status for tag:3840d255-9333-4b4a-afac-4a6d116119e6 - {'path': '/', 'tag': '3840d255-9333-4b4a-afac-4a6d116119e6', 'options': 'recursive,force'} 2024-03-19T01:23:51.288 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/skip_spaces.o 2024-03-19T01:23:51.393 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/memattr.o 2024-03-19T01:23:51.468 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/lib-cmdline.o 2024-03-19T01:23:51.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:23:51.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:51.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:51 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:51.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:51 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:51.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:51 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:51.673 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-debug.o 2024-03-19T01:23:51.751 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/lib-ctype.o 2024-03-19T01:23:51.882 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-sysfs.o 2024-03-19T01:23:51.906 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/alignedmem.o 2024-03-19T01:23:52.234 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/relocate.o 2024-03-19T01:23:52.313 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hidraw.o 2024-03-19T01:23:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: pgmap v412: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 660 KiB/s wr, 38 op/s 2024-03-19T01:23:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:52.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:52 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: pgmap v412: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 660 KiB/s wr, 38 op/s 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:52.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:52 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: pgmap v412: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 660 KiB/s wr, 38 op/s 2024-03-19T01:23:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:52.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:52 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:52.675 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/perf/built-in.a 2024-03-19T01:23:52.769 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/hwtracing/intel_th/built-in.a 2024-03-19T01:23:52.781 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/tpm.o 2024-03-19T01:23:52.824 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_lease.o 2024-03-19T01:23:53.078 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/printk.o 2024-03-19T01:23:53.269 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-stats.o 2024-03-19T01:23:53.356 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-generic.o 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:53 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:53 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:53.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:53 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:53.713 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_create.o 2024-03-19T01:23:53.932 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/dmi-id.o 2024-03-19T01:23:54.101 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_dmabuf.o 2024-03-19T01:23:54.243 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_managed.o 2024-03-19T01:23:54.314 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/vsprintf.o 2024-03-19T01:23:54.369 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_mm.o 2024-03-19T01:23:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: pgmap v413: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 407 KiB/s wr, 50 op/s 2024-03-19T01:23:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:54 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: pgmap v413: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 407 KiB/s wr, 50 op/s 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:54 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: pgmap v413: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 407 KiB/s wr, 50 op/s 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:54.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:54 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:54.786 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/libstub/x86-stub.o 2024-03-19T01:23:54.850 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-a4tech.o 2024-03-19T01:23:55.004 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/android/built-in.a 2024-03-19T01:23:55.088 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_mode_config.o 2024-03-19T01:23:55.478 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_domain.o 2024-03-19T01:23:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:55.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:55 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:55.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:55 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.535 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-rq.o 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:55.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:55 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:55.590 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/alignedmem.stub.o 2024-03-19T01:23:55.701 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/efi-stub-helper.stub.o 2024-03-19T01:23:55.789 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/file.stub.o 2024-03-19T01:23:55.871 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-io-rewind.o 2024-03-19T01:23:56.057 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_execbuffer.o 2024-03-19T01:23:56.141 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/nvmem/layouts/built-in.a 2024-03-19T01:23:56.198 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/nvmem/core.o 2024-03-19T01:23:56.233 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-apple.o 2024-03-19T01:23:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: pgmap v414: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 320 KiB/s wr, 68 op/s 2024-03-19T01:23:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:56.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:56 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: pgmap v414: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 320 KiB/s wr, 68 op/s 2024-03-19T01:23:56.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:56.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:56 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: pgmap v414: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 320 KiB/s wr, 68 op/s 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:56.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:56 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:56.629 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/gop.stub.o 2024-03-19T01:23:56.795 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/lib-cmdline.stub.o 2024-03-19T01:23:56.810 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-belkin.o 2024-03-19T01:23:56.893 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/lib-ctype.stub.o 2024-03-19T01:23:56.985 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/mem.stub.o 2024-03-19T01:23:57.068 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/pci.stub.o 2024-03-19T01:23:57.168 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/printk.stub.o 2024-03-19T01:23:57.241 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/random.stub.o 2024-03-19T01:23:57.249 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-builtin.o 2024-03-19T01:23:57.307 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/randomalloc.stub.o 2024-03-19T01:23:57.378 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/relocate.stub.o 2024-03-19T01:23:57.411 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/memmap.o 2024-03-19T01:23:57.417 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/secureboot.stub.o 2024-03-19T01:23:57.448 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/skip_spaces.stub.o 2024-03-19T01:23:57.478 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/tpm.stub.o 2024-03-19T01:23:57.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:57.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:57 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:57.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:57 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.551 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/memmap.o 2024-03-19T01:23:57.553 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/vsprintf.stub.o 2024-03-19T01:23:57.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:57.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:57 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:57.635 INFO:tasks.workunit.client.0.smithi069.stdout: STUBCPY drivers/firmware/efi/libstub/x86-stub.stub.o 2024-03-19T01:23:57.711 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/efi/libstub/lib.a 2024-03-19T01:23:57.836 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-cherry.o 2024-03-19T01:23:57.881 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/esrt.o 2024-03-19T01:23:58.259 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-raid1.o 2024-03-19T01:23:58.266 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/nvmem/built-in.a 2024-03-19T01:23:58.502 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_mode_object.o 2024-03-19T01:23:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: pgmap v415: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 357 KiB/s wr, 207 op/s 2024-03-19T01:23:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:58.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: pgmap v415: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 357 KiB/s wr, 207 op/s 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:58.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: pgmap v415: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 357 KiB/s wr, 207 op/s 2024-03-19T01:23:58.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:58.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:58.649 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_internal.o 2024-03-19T01:23:58.795 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-log.o 2024-03-19T01:23:58.858 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-region-hash.o 2024-03-19T01:23:59.055 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-chicony.o 2024-03-19T01:23:59.516 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/runtime-wrappers.o 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: pgmap v416: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 254 KiB/s wr, 619 op/s 2024-03-19T01:23:59.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:23:59.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:23:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: pgmap v416: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 254 KiB/s wr, 619 op/s 2024-03-19T01:23:59.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:23:59.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:23:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: pgmap v416: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 254 KiB/s wr, 619 op/s 2024-03-19T01:23:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:23:59.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:23:59.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:23:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:00.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:00.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:00.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:00.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:01.065 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-cypress.o 2024-03-19T01:24:01.070 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/firmware/efi/earlycon.o 2024-03-19T01:24:01.073 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_modes.o 2024-03-19T01:24:01.094 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/md/dm-zero.o 2024-03-19T01:24:01.101 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-ezkey.o 2024-03-19T01:24:01.134 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_modeset_lock.o 2024-03-19T01:24:01.143 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_plane.o 2024-03-19T01:24:01.295 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_object.o 2024-03-19T01:24:01.560 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:01 smithi086 ceph-mon[66829]: pgmap v417: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 253 KiB/s wr, 814 op/s 2024-03-19T01:24:01.560 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:01.560 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:01.560 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:01.560 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:01.560 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:01.560 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:01 smithi163 ceph-mon[67428]: pgmap v417: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 253 KiB/s wr, 814 op/s 2024-03-19T01:24:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:01.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:01.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:01 smithi069 ceph-mon[31727]: pgmap v417: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 253 KiB/s wr, 814 op/s 2024-03-19T01:24:01.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:01.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:01.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:01.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:01.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:01.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.392 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/md/built-in.a 2024-03-19T01:24:02.541 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-gyration.o 2024-03-19T01:24:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.579 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/efi/built-in.a 2024-03-19T01:24:02.614 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_prime.o 2024-03-19T01:24:02.668 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/firmware/built-in.a 2024-03-19T01:24:02.672 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_lmem.o 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:02.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.762 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-ite.o 2024-03-19T01:24:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:02.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:02.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:02.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:02.852 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_print.o 2024-03-19T01:24:02.852 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_mman.o 2024-03-19T01:24:03.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:03 smithi163 ceph-mon[67428]: pgmap v418: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 1.75k op/s 2024-03-19T01:24:03.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:03.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:03.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:03.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:03.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.700 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_property.o 2024-03-19T01:24:03.730 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_syncobj.o 2024-03-19T01:24:03.751 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_pages.o 2024-03-19T01:24:03.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:03 smithi086 ceph-mon[66829]: pgmap v418: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 1.75k op/s 2024-03-19T01:24:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:03 smithi069 ceph-mon[31727]: pgmap v418: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 1.75k op/s 2024-03-19T01:24:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:03.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:03.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:03.939 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_phys.o 2024-03-19T01:24:03.997 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-kensington.o 2024-03-19T01:24:04.014 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-lg.o 2024-03-19T01:24:04.138 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_pm.o 2024-03-19T01:24:04.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:04.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:04.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:04.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:04.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:04.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:04.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:04.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:04.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:04.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:04.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:04.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:04.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:04.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.124 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_sysfs.o 2024-03-19T01:24:05.274 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_region.o 2024-03-19T01:24:05.341 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-lgff.o 2024-03-19T01:24:05.512 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_shmem.o 2024-03-19T01:24:05.659 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_shrinker.o 2024-03-19T01:24:05.663 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_trace_points.o 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: pgmap v419: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 1.79k op/s 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:05.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: pgmap v419: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 1.79k op/s 2024-03-19T01:24:05.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:05.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: pgmap v419: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 1.79k op/s 2024-03-19T01:24:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:05.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:05.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:05.875 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-lg4ff.o 2024-03-19T01:24:05.909 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_stolen.o 2024-03-19T01:24:06.593 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_throttle.o 2024-03-19T01:24:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.770 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-lg-g15.o 2024-03-19T01:24:06.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:06.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:06.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:06.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:06.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:06.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:06.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:06.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:07.079 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-microsoft.o 2024-03-19T01:24:07.088 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-monterey.o 2024-03-19T01:24:07.570 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_vblank.o 2024-03-19T01:24:07.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:07.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:07.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:07 smithi086 ceph-mon[66829]: pgmap v420: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 1.86k op/s 2024-03-19T01:24:07.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:07.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:07.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:07.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:07.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:07 smithi069 ceph-mon[31727]: pgmap v420: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 1.86k op/s 2024-03-19T01:24:07.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:07.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:07.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:07.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:07.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:07 smithi163 ceph-mon[67428]: pgmap v420: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 1.86k op/s 2024-03-19T01:24:07.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:07.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:08.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:08.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:08.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:08.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:08 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:08.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:08 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:08.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:08.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:08.785 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:08.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:08 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:08.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:08 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:08.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:08.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:08.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:08.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:08 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:08.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:08 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:08.834 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_tiling.o 2024-03-19T01:24:08.841 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-ntrig.o 2024-03-19T01:24:08.854 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-pl.o 2024-03-19T01:24:08.867 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-petalynx.o 2024-03-19T01:24:08.869 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_vblank_work.o 2024-03-19T01:24:08.870 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_ttm.o 2024-03-19T01:24:08.879 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_ttm_move.o 2024-03-19T01:24:09.026 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_vma_manager.o 2024-03-19T01:24:09.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:09.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:09.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:09.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: pgmap v421: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.74k op/s 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:09.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:09 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:09.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: pgmap v421: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.74k op/s 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:09.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:09 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:09.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: pgmap v421: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.74k op/s 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:09.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:09.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:09.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:09.813 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:09 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:10.126 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.o 2024-03-19T01:24:10.165 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-redragon.o 2024-03-19T01:24:10.174 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_writeback.o 2024-03-19T01:24:10.198 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_userptr.o 2024-03-19T01:24:10.385 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gem_wait.o 2024-03-19T01:24:10.639 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_ioc32.o 2024-03-19T01:24:10.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:10.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:10.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:10.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:10.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:10 smithi086 ceph-mon[66829]: osdmap e109: 12 total, 12 up, 12 in 2024-03-19T01:24:10.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:10 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:10.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:10 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:10.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:10.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:10.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:10.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:10.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:10 smithi069 ceph-mon[31727]: osdmap e109: 12 total, 12 up, 12 in 2024-03-19T01:24:10.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:10 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:10.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:10 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:10.787 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gem/i915_gemfs.o 2024-03-19T01:24:10.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:10.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:10.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:10.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:10.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:10 smithi163 ceph-mon[67428]: osdmap e109: 12 total, 12 up, 12 in 2024-03-19T01:24:10.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:10 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:10.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:10 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:11.325 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-samsung.o 2024-03-19T01:24:11.516 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_active.o 2024-03-19T01:24:11.686 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_panel.o 2024-03-19T01:24:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:11 smithi086 ceph-mon[66829]: pgmap v423: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 1.35k op/s 2024-03-19T01:24:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:11 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:11.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:11 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:11.762 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_cmd_parser.o 2024-03-19T01:24:11.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:11.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:11.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:11 smithi069 ceph-mon[31727]: pgmap v423: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 1.35k op/s 2024-03-19T01:24:11.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:11.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:11.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:11 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:11.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:11 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:11.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:11.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:11.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:11 smithi163 ceph-mon[67428]: pgmap v423: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 1.35k op/s 2024-03-19T01:24:11.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:11.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:11.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:11 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:11.812 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:11 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:11.922 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_deps.o 2024-03-19T01:24:12.066 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_pci.o 2024-03-19T01:24:12.155 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-sony.o 2024-03-19T01:24:12.265 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-sunplus.o 2024-03-19T01:24:12.527 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_debugfs.o 2024-03-19T01:24:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:12 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:12 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:12 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:12 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:12.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:12.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:12 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:12.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:12 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:12.840 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_gem_evict.o 2024-03-19T01:24:12.874 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_debugfs_crc.o 2024-03-19T01:24:12.955 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/hid/hid-topseed.o 2024-03-19T01:24:13.205 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_gem_gtt.o 2024-03-19T01:24:13.293 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_panel_orientation_quirks.o 2024-03-19T01:24:13.345 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_gem_ww.o 2024-03-19T01:24:13.634 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_gem.o 2024-03-19T01:24:13.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:13 smithi086 ceph-mon[66829]: pgmap v424: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 626 KiB/s wr, 231 op/s 2024-03-19T01:24:13.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:13 smithi069 ceph-mon[31727]: pgmap v424: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 626 KiB/s wr, 231 op/s 2024-03-19T01:24:13.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:13 smithi163 ceph-mon[67428]: pgmap v424: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 626 KiB/s wr, 231 op/s 2024-03-19T01:24:14.053 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_buddy.o 2024-03-19T01:24:14.161 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_query.o 2024-03-19T01:24:14.444 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_request.o 2024-03-19T01:24:14.490 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/hid/built-in.a 2024-03-19T01:24:14.539 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_gem_shmem_helper.o 2024-03-19T01:24:14.539 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_scheduler.o 2024-03-19T01:24:14.613 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_atomic_helper.o 2024-03-19T01:24:14.671 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_atomic_state_helper.o 2024-03-19T01:24:15.206 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_bridge_connector.o 2024-03-19T01:24:15.716 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_trace_points.o 2024-03-19T01:24:15.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:15 smithi086 ceph-mon[66829]: pgmap v425: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 518 KiB/s wr, 157 op/s 2024-03-19T01:24:15.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:15 smithi069 ceph-mon[31727]: pgmap v425: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 518 KiB/s wr, 157 op/s 2024-03-19T01:24:15.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:15 smithi163 ceph-mon[67428]: pgmap v425: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 518 KiB/s wr, 157 op/s 2024-03-19T01:24:15.866 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_crtc_helper.o 2024-03-19T01:24:15.955 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_damage_helper.o 2024-03-19T01:24:15.961 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_ttm_buddy_manager.o 2024-03-19T01:24:16.054 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_vma.o 2024-03-19T01:24:16.412 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_vma_resource.o 2024-03-19T01:24:16.736 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_encoder_slave.o 2024-03-19T01:24:17.022 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_gsc_fw.o 2024-03-19T01:24:17.038 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_gsc_proxy.o 2024-03-19T01:24:17.190 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_gsc_uc.o 2024-03-19T01:24:17.533 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_flip_work.o 2024-03-19T01:24:17.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:17 smithi086 ceph-mon[66829]: pgmap v426: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 213 KiB/s wr, 47 op/s 2024-03-19T01:24:17.759 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_gsc_uc_heci_cmd_submit.o 2024-03-19T01:24:17.765 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_format_helper.o 2024-03-19T01:24:17.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:17 smithi069 ceph-mon[31727]: pgmap v426: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 213 KiB/s wr, 47 op/s 2024-03-19T01:24:17.811 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:17 smithi163 ceph-mon[67428]: pgmap v426: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 213 KiB/s wr, 47 op/s 2024-03-19T01:24:17.891 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_gem_atomic_helper.o 2024-03-19T01:24:18.140 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc.o 2024-03-19T01:24:18.309 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_ads.o 2024-03-19T01:24:18.479 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_capture.o 2024-03-19T01:24:18.537 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_ct.o 2024-03-19T01:24:18.649 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.o 2024-03-19T01:24:19.009 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_gem_framebuffer_helper.o 2024-03-19T01:24:19.082 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_kms_helper_common.o 2024-03-19T01:24:19.386 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_fw.o 2024-03-19T01:24:19.644 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_modeset_helper.o 2024-03-19T01:24:20.058 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_plane_helper.o 2024-03-19T01:24:20.111 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_hwconfig.o 2024-03-19T01:24:20.113 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_log.o 2024-03-19T01:24:20.180 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_log_debugfs.o 2024-03-19T01:24:20.289 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_probe_helper.o 2024-03-19T01:24:20.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:20 smithi086 ceph-mon[66829]: pgmap v427: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 932 KiB/s wr, 43 op/s 2024-03-19T01:24:20.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:20 smithi086 ceph-mon[66829]: osdmap e110: 12 total, 12 up, 12 in 2024-03-19T01:24:20.514 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:20 smithi163 ceph-mon[67428]: pgmap v427: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 932 KiB/s wr, 43 op/s 2024-03-19T01:24:20.514 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:20 smithi163 ceph-mon[67428]: osdmap e110: 12 total, 12 up, 12 in 2024-03-19T01:24:20.531 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_rc.o 2024-03-19T01:24:20.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:20 smithi069 ceph-mon[31727]: pgmap v427: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 932 KiB/s wr, 43 op/s 2024-03-19T01:24:20.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:20 smithi069 ceph-mon[31727]: osdmap e110: 12 total, 12 up, 12 in 2024-03-19T01:24:20.801 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_rect.o 2024-03-19T01:24:20.992 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.o 2024-03-19T01:24:21.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:24:21.239 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:24:21.263 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_guc_submission.o 2024-03-19T01:24:21.311 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_self_refresh_helper.o 2024-03-19T01:24:21.373 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_simple_kms_helper.o 2024-03-19T01:24:21.506 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_huc.o 2024-03-19T01:24:21.528 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_huc_debugfs.o 2024-03-19T01:24:21.703 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/bridge/panel.o 2024-03-19T01:24:21.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.946+0000 7f39cf200640 1 -- 172.21.15.69:0/2943082264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d010fd40 msgr2=0x7f39d0110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:21.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.946+0000 7f39cf200640 1 --2- 172.21.15.69:0/2943082264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d010fd40 0x7f39d0110120 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009930 tx=0x7f39bc02bc40 comp rx=0 tx=0).stop 2024-03-19T01:24:21.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.948+0000 7f39cf200640 1 -- 172.21.15.69:0/2943082264 shutdown_connections 2024-03-19T01:24:21.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.948+0000 7f39cf200640 1 --2- 172.21.15.69:0/2943082264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d010e380 0x7f39d011d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.948+0000 7f39cf200640 1 --2- 172.21.15.69:0/2943082264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39d010d900 0x7f39d010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.948+0000 7f39cf200640 1 --2- 172.21.15.69:0/2943082264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d010fd40 0x7f39d0110120 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.948+0000 7f39cf200640 1 -- 172.21.15.69:0/2943082264 >> 172.21.15.69:0/2943082264 conn(0x7f39d006c6d0 msgr2=0x7f39d006cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:21.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.948+0000 7f39cf200640 1 -- 172.21.15.69:0/2943082264 shutdown_connections 2024-03-19T01:24:21.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.948+0000 7f39cf200640 1 -- 172.21.15.69:0/2943082264 wait complete. 2024-03-19T01:24:21.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.950+0000 7f39cf200640 1 Processor -- start 2024-03-19T01:24:21.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.950+0000 7f39cf200640 1 -- start start 2024-03-19T01:24:21.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cf200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d010d900 0x7f39d011c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:21.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d010e380 0x7f39d011caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:21.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cf200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39d010fd40 0x7f39d0115750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:21.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cf200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0120860 con 0x7f39d010fd40 2024-03-19T01:24:21.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01206e0 con 0x7f39d010e380 2024-03-19T01:24:21.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cf200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01209e0 con 0x7f39d010d900 2024-03-19T01:24:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d010e380 0x7f39d011caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cde00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d010d900 0x7f39d011c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d010e380 0x7f39d011caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34714/0 (socket says 172.21.15.69:34714) 2024-03-19T01:24:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cd400640 1 -- 172.21.15.69:0/2248352216 learned_addr learned my addr 172.21.15.69:0/2248352216 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39ce800640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39d010fd40 0x7f39d0115750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cd400640 1 -- 172.21.15.69:0/2248352216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d010d900 msgr2=0x7f39d011c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cd400640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d010d900 0x7f39d011c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.951+0000 7f39cd400640 1 -- 172.21.15.69:0/2248352216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39d010fd40 msgr2=0x7f39d0115750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.952+0000 7f39cd400640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39d010fd40 0x7f39d0115750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.952+0000 7f39cd400640 1 -- 172.21.15.69:0/2248352216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d0116010 con 0x7f39d010e380 2024-03-19T01:24:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.952+0000 7f39ce800640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39d010fd40 0x7f39d0115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:24:21.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.952+0000 7f39cd400640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d010e380 0x7f39d011caa0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f39c000c920 tx=0x7f39c000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:21.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.952+0000 7f39c6a00640 1 -- 172.21.15.69:0/2248352216 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0007c70 con 0x7f39d010e380 2024-03-19T01:24:21.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.952+0000 7f39c6a00640 1 -- 172.21.15.69:0/2248352216 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c0007e10 con 0x7f39d010e380 2024-03-19T01:24:21.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.952+0000 7f39cf200640 1 -- 172.21.15.69:0/2248352216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d0116300 con 0x7f39d010e380 2024-03-19T01:24:21.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.952+0000 7f39c6a00640 1 -- 172.21.15.69:0/2248352216 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c00189a0 con 0x7f39d010e380 2024-03-19T01:24:21.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.952+0000 7f39cf200640 1 -- 172.21.15.69:0/2248352216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d011a310 con 0x7f39d010e380 2024-03-19T01:24:21.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.960+0000 7f39d801f740 1 Processor -- start 2024-03-19T01:24:21.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.962+0000 7f39d801f740 1 -- start start 2024-03-19T01:24:21.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.963+0000 7f39d801f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 0x55784386e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:21.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.963+0000 7f39ce800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 0x55784386e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:21.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.963+0000 7f39ce800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 0x55784386e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46984/0 (socket says 172.21.15.69:46984) 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.963+0000 7f39ce800640 1 -- 172.21.15.69:0/3705891301 learned_addr learned my addr 172.21.15.69:0/3705891301 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.963+0000 7f39d801f740 1 --2- 172.21.15.69:0/3705891301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386e870 0x557843876f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.964+0000 7f39d801f740 1 --2- 172.21.15.69:0/3705891301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557843877450 0x557843879840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.964+0000 7f39d801f740 1 -- 172.21.15.69:0/3705891301 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5578437d22c0 con 0x55784386df50 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.964+0000 7f39d801f740 1 -- 172.21.15.69:0/3705891301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55784377ab60 con 0x55784386e870 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.964+0000 7f39d801f740 1 -- 172.21.15.69:0/3705891301 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5578437ce360 con 0x557843877450 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.964+0000 7f39cde00640 1 --2- 172.21.15.69:0/3705891301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386e870 0x557843876f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.964+0000 7f39ce800640 1 -- 172.21.15.69:0/3705891301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557843877450 msgr2=0x557843879840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.964+0000 7f39ce800640 1 --2- 172.21.15.69:0/3705891301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557843877450 0x557843879840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.964+0000 7f39ce800640 1 -- 172.21.15.69:0/3705891301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386e870 msgr2=0x557843876f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.964+0000 7f39ce800640 1 --2- 172.21.15.69:0/3705891301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386e870 0x557843876f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.964+0000 7f39ce800640 1 -- 172.21.15.69:0/3705891301 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578435b07d0 con 0x55784386df50 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.965+0000 7f39ce800640 1 --2- 172.21.15.69:0/3705891301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 0x55784386e330 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c4a0 tx=0x7f39c800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f75d52564243c73 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.966+0000 7f39bb400640 1 -- 172.21.15.69:0/3705891301 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c800e640 con 0x55784386df50 2024-03-19T01:24:21.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.966+0000 7f39bb400640 1 -- 172.21.15.69:0/3705891301 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c800ecc0 con 0x55784386df50 2024-03-19T01:24:21.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.966+0000 7f39bb400640 1 -- 172.21.15.69:0/3705891301 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8018400 con 0x55784386df50 2024-03-19T01:24:21.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.967+0000 7f39c6a00640 1 -- 172.21.15.69:0/2248352216 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f39c0018b40 con 0x7f39d010e380 2024-03-19T01:24:21.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.967+0000 7f39d801f740 1 -- 172.21.15.69:0/3705891301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 msgr2=0x55784386e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:21.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.967+0000 7f39d801f740 1 --2- 172.21.15.69:0/3705891301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 0x55784386e330 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c4a0 tx=0x7f39c800c960 comp rx=0 tx=0).stop 2024-03-19T01:24:21.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.968+0000 7f39d801f740 1 -- 172.21.15.69:0/3705891301 shutdown_connections 2024-03-19T01:24:21.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.968+0000 7f39d801f740 1 --2- 172.21.15.69:0/3705891301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557843877450 0x557843879840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.968+0000 7f39d801f740 1 --2- 172.21.15.69:0/3705891301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386e870 0x557843876f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.968+0000 7f39d801f740 1 --2- 172.21.15.69:0/3705891301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 0x55784386e330 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.968+0000 7f39d801f740 1 -- 172.21.15.69:0/3705891301 >> 172.21.15.69:0/3705891301 conn(0x557843868680 msgr2=0x557843868a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.972+0000 7f39d801f740 1 -- 172.21.15.69:0/3705891301 shutdown_connections 2024-03-19T01:24:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.972+0000 7f39d801f740 1 -- 172.21.15.69:0/3705891301 wait complete. 2024-03-19T01:24:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.973+0000 7f39d801f740 1 -- start start 2024-03-19T01:24:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.973+0000 7f39c6a00640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f39b40a29c0 0x7f39b40a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:21.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.973+0000 7f39d801f740 1 Processor -- start 2024-03-19T01:24:21.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.973+0000 7f39c6a00640 1 -- 172.21.15.69:0/2248352216 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f39c00cf3d0 con 0x7f39d010e380 2024-03-19T01:24:21.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.973+0000 7f39ce800640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f39b40a29c0 0x7f39b40a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:21.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.974+0000 7f39ce800640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f39b40a29c0 0x7f39b40a4e80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39c800ce60 tx=0x7f39c800fa50 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:21.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.974+0000 7f39d801f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386df50 0x557843874480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.974+0000 7f39d801f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 0x5578438749c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39d801f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 0x557843874f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39d801f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55784387a290 con 0x557843877450 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39d801f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55784378dfc0 con 0x55784386df50 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39d801f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55784387a410 con 0x55784386e870 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39cde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386df50 0x557843874480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39d801f740 10 client.0 _ll_register_callbacks cb 0x55784378d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39cde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386df50 0x557843874480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:34740/0 (socket says 172.21.15.69:34740) 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39cde00640 1 -- 172.21.15.69:0/1834356929 learned_addr learned my addr 172.21.15.69:0/1834356929 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39cd400640 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 0x5578438749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39cde00640 1 -- 172.21.15.69:0/1834356929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 msgr2=0x5578438749c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39cde00640 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 0x5578438749c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39cde00640 1 -- 172.21.15.69:0/1834356929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 msgr2=0x557843874f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:24:21.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39ce800640 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 0x557843874f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:21.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.975+0000 7f39cde00640 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 0x557843874f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.976+0000 7f39cde00640 1 -- 172.21.15.69:0/1834356929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578434e63f0 con 0x55784386df50 2024-03-19T01:24:21.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.976+0000 7f39cd400640 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 0x5578438749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:24:21.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.976+0000 7f39ce800640 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 0x557843874f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:24:21.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.976+0000 7f39cde00640 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386df50 0x557843874480 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f39bc02c250 tx=0x7f39bc0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71d1d1cb45038452 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:21.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.976+0000 7f39afe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:24:21.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.977+0000 7f39afe00640 1 -- 172.21.15.69:0/1834356929 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc06e060 con 0x55784386df50 2024-03-19T01:24:21.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.977+0000 7f39d801f740 1 -- 172.21.15.69:0/1834356929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c800ecc0 con 0x55784386df50 2024-03-19T01:24:21.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.977+0000 7f39afe00640 1 -- 172.21.15.69:0/1834356929 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f39bc0046c0 con 0x55784386df50 2024-03-19T01:24:21.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.977+0000 7f39afe00640 1 -- 172.21.15.69:0/1834356929 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f39bc06e060 con 0x55784386df50 2024-03-19T01:24:21.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.979+0000 7f39d801f740 10 client.28082 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:21.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.979+0000 7f39d801f740 1 -- 172.21.15.69:0/1834356929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5578434a60f0 con 0x55784386df50 2024-03-19T01:24:21.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.981+0000 7f39afe00640 1 -- 172.21.15.69:0/1834356929 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c800ecc0 con 0x55784386df50 2024-03-19T01:24:21.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.981+0000 7f39afe00640 1 -- 172.21.15.69:0/1834356929 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f39bc0035f0 con 0x55784386df50 2024-03-19T01:24:21.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.981+0000 7f39d801f740 10 client.28082 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:21.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.981+0000 7f39d801f740 10 client.28082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:24:21.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.981+0000 7f39d801f740 20 client.28082 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:21.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.981+0000 7f39d801f740 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557843887810 0x557843889c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:21.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.981+0000 7f39d801f740 4 client.28082 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:24:21.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.981+0000 7f39d801f740 1 -- 172.21.15.69:0/1834356929 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557843773610 con 0x557843887810 2024-03-19T01:24:21.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.982+0000 7f39cd400640 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557843887810 0x557843889c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:21.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.986+0000 7f39afe00640 10 client.28082 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.987+0000 7f39cd400640 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557843887810 0x557843889c00 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a897de1c75eddf server_cookie=4131735f2659d71 in_seq=0 out_seq=0 2024-03-19T01:24:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.987+0000 7f39afe00640 1 -- 172.21.15.69:0/1834356929 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557843773610 con 0x557843887810 2024-03-19T01:24:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39afe00640 10 client.28082 handle_command_reply: tid=0 2024-03-19T01:24:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39d801f740 1 client.28082 shutdown 2024-03-19T01:24:21.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39d801f740 1 -- 172.21.15.69:0/1834356929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386df50 msgr2=0x557843874480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:21.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39d801f740 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386df50 0x557843874480 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f39bc02c250 tx=0x7f39bc0045f0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39d801f740 1 -- 172.21.15.69:0/1834356929 shutdown_connections 2024-03-19T01:24:21.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39d801f740 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x557843887810 0x557843889c00 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39d801f740 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 0x557843874f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39afe00640 0 client.28082 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:21.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39d801f740 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 0x5578438749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39d801f740 1 --2- 172.21.15.69:0/1834356929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386df50 0x557843874480 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:21.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39d801f740 1 -- 172.21.15.69:0/1834356929 >> 172.21.15.69:0/1834356929 conn(0x557843868680 msgr2=0x557843867900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:21.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39d801f740 1 -- 172.21.15.69:0/1834356929 shutdown_connections 2024-03-19T01:24:21.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.988+0000 7f39d801f740 1 -- 172.21.15.69:0/1834356929 wait complete. 2024-03-19T01:24:21.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:21.989+0000 7f39d801f740 20 client.28082 trim_cache size 0 max 16384 2024-03-19T01:24:22.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.029+0000 7f39d801f740 1 Processor -- start 2024-03-19T01:24:22.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.029+0000 7f39d801f740 1 -- start start 2024-03-19T01:24:22.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.030+0000 7f39d801f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 0x557843876bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:22.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.030+0000 7f39d801f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 0x55784386a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:22.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.031+0000 7f39d801f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557843877450 0x55784388f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:22.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.031+0000 7f39d801f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01af4d0 con 0x55784386df50 2024-03-19T01:24:22.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.031+0000 7f39d801f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01b1010 con 0x557843877450 2024-03-19T01:24:22.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.031+0000 7f39d801f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01afba0 con 0x55784386e870 2024-03-19T01:24:22.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.031+0000 7f39cd400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 0x55784386a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:22.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.031+0000 7f39cd400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 0x55784386a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:39994/0 (socket says 172.21.15.69:39994) 2024-03-19T01:24:22.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.031+0000 7f39cd400640 1 -- 172.21.15.69:0/1354548993 learned_addr learned my addr 172.21.15.69:0/1354548993 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:22.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.031+0000 7f39cde00640 1 --2- 172.21.15.69:0/1354548993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 0x557843876bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:22.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.032+0000 7f39ce800640 1 --2- 172.21.15.69:0/1354548993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557843877450 0x55784388f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:22.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.032+0000 7f39cde00640 1 -- 172.21.15.69:0/1354548993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 msgr2=0x55784386a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:22.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.032+0000 7f39cde00640 1 --2- 172.21.15.69:0/1354548993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 0x55784386a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.032+0000 7f39cde00640 1 -- 172.21.15.69:0/1354548993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557843877450 msgr2=0x55784388f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:22.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.032+0000 7f39cde00640 1 --2- 172.21.15.69:0/1354548993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557843877450 0x55784388f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.032+0000 7f39cde00640 1 -- 172.21.15.69:0/1354548993 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55784388a230 con 0x55784386df50 2024-03-19T01:24:22.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.033+0000 7f39cde00640 1 --2- 172.21.15.69:0/1354548993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 0x557843876bb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009850 tx=0x7f39bc02bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e455532f2a3ed4ef server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:22.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.033+0000 7f39baa00640 1 -- 172.21.15.69:0/1354548993 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc078080 con 0x55784386df50 2024-03-19T01:24:22.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.034+0000 7f39baa00640 1 -- 172.21.15.69:0/1354548993 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39bc003be0 con 0x55784386df50 2024-03-19T01:24:22.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.034+0000 7f39d801f740 1 -- 172.21.15.69:0/1354548993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 msgr2=0x557843876bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:22.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.035+0000 7f39baa00640 1 -- 172.21.15.69:0/1354548993 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc078080 con 0x55784386df50 2024-03-19T01:24:22.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.035+0000 7f39d801f740 1 --2- 172.21.15.69:0/1354548993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 0x557843876bb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009850 tx=0x7f39bc02bc40 comp rx=0 tx=0).stop 2024-03-19T01:24:22.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.035+0000 7f39d801f740 1 -- 172.21.15.69:0/1354548993 shutdown_connections 2024-03-19T01:24:22.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.035+0000 7f39d801f740 1 --2- 172.21.15.69:0/1354548993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557843877450 0x55784388f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.035+0000 7f39d801f740 1 --2- 172.21.15.69:0/1354548993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386e870 0x55784386a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.035+0000 7f39d801f740 1 --2- 172.21.15.69:0/1354548993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55784386df50 0x557843876bb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.035+0000 7f39d801f740 1 -- 172.21.15.69:0/1354548993 >> 172.21.15.69:0/1354548993 conn(0x557843868680 msgr2=0x557843869ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:22.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.035+0000 7f39d801f740 1 -- 172.21.15.69:0/1354548993 shutdown_connections 2024-03-19T01:24:22.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.036+0000 7f39d801f740 1 -- 172.21.15.69:0/1354548993 wait complete. 2024-03-19T01:24:22.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.036+0000 7f39d801f740 1 -- start start 2024-03-19T01:24:22.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.036+0000 7f39d801f740 1 Processor -- start 2024-03-19T01:24:22.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.037+0000 7f39d801f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386df50 0x55784387d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.037+0000 7f39d801f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386e870 0x55784387db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.037+0000 7f39cde00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386df50 0x55784387d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.037+0000 7f39cde00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386df50 0x55784387d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:39998/0 (socket says 172.21.15.69:39998) 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.037+0000 7f39d801f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 0x5578438ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.038+0000 7f39d801f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01b0940 con 0x557843877450 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.038+0000 7f39d801f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01b0270 con 0x55784386e870 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.038+0000 7f39d801f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f39d01ad450 con 0x55784386df50 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.038+0000 7f39d801f740 10 client.0 _ll_register_callbacks cb 0x7f39bc0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.038+0000 7f39ce800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 0x5578438ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.038+0000 7f39cde00640 1 -- 172.21.15.69:0/3808588384 learned_addr learned my addr 172.21.15.69:0/3808588384 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.038+0000 7f39cd400640 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386e870 0x55784387db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.039+0000 7f39cde00640 1 -- 172.21.15.69:0/3808588384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386e870 msgr2=0x55784387db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.039+0000 7f39cde00640 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386e870 0x55784387db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.039+0000 7f39cde00640 1 -- 172.21.15.69:0/3808588384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 msgr2=0x5578438ecaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:22.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.039+0000 7f39cde00640 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 0x5578438ecaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.039+0000 7f39cde00640 1 -- 172.21.15.69:0/3808588384 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557843879840 con 0x55784386df50 2024-03-19T01:24:22.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.039+0000 7f39ce800640 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 0x5578438ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:22.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.040+0000 7f39cde00640 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386df50 0x55784387d610 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f39bc004480 tx=0x7f39bc004cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b88dccf3e8e0e507 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:22.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.040+0000 7f39afe00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:24:22.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.040+0000 7f39afe00640 1 -- 172.21.15.69:0/3808588384 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc081060 con 0x55784386df50 2024-03-19T01:24:22.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.040+0000 7f39afe00640 1 -- 172.21.15.69:0/3808588384 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f39bc005320 con 0x55784386df50 2024-03-19T01:24:22.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.041+0000 7f39d801f740 1 -- 172.21.15.69:0/3808588384 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39bc003be0 con 0x55784386df50 2024-03-19T01:24:22.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.041+0000 7f39afe00640 1 -- 172.21.15.69:0/3808588384 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f39bc078070 con 0x55784386df50 2024-03-19T01:24:22.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.041+0000 7f39d801f740 10 client.28006 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:22.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.041+0000 7f39d801f740 1 -- 172.21.15.69:0/3808588384 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5578438ee020 con 0x55784386df50 2024-03-19T01:24:22.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.041+0000 7f39afe00640 1 -- 172.21.15.69:0/3808588384 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39bc003be0 con 0x55784386df50 2024-03-19T01:24:22.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.041+0000 7f39afe00640 1 -- 172.21.15.69:0/3808588384 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f39bc07c9d0 con 0x55784386df50 2024-03-19T01:24:22.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.041+0000 7f39d801f740 10 client.28006 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:22.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.041+0000 7f39d801f740 10 client.28006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:24:22.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.041+0000 7f39d801f740 20 client.28006 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:22.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.041+0000 7f39d801f740 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5578438ee680 0x5578438f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:22.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.042+0000 7f39d801f740 4 client.28006 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:24:22.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.042+0000 7f39d801f740 1 -- 172.21.15.69:0/3808588384 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578437df800 con 0x5578438ee680 2024-03-19T01:24:22.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.042+0000 7f39cd400640 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5578438ee680 0x5578438f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:22.050 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_huc_fw.o 2024-03-19T01:24:22.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.072+0000 7f39cd400640 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5578438ee680 0x5578438f0aa0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c144fba82ade70d server_cookie=64e3f8a55223fe6d in_seq=0 out_seq=0 2024-03-19T01:24:22.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.072+0000 7f39afe00640 10 client.28006 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:22.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.076+0000 7f39afe00640 1 -- 172.21.15.69:0/3808588384 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5578437df800 con 0x5578438ee680 2024-03-19T01:24:22.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.076+0000 7f39afe00640 10 client.28006 handle_command_reply: tid=0 2024-03-19T01:24:22.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.076+0000 7f39d801f740 1 client.28006 shutdown 2024-03-19T01:24:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.079+0000 7f39d801f740 1 -- 172.21.15.69:0/3808588384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386df50 msgr2=0x55784387d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.079+0000 7f39d801f740 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386df50 0x55784387d610 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f39bc004480 tx=0x7f39bc004cc0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.079+0000 7f39d801f740 1 -- 172.21.15.69:0/3808588384 shutdown_connections 2024-03-19T01:24:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.079+0000 7f39d801f740 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x5578438ee680 0x5578438f0aa0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.079+0000 7f39d801f740 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557843877450 0x5578438ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.079+0000 7f39d801f740 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55784386e870 0x55784387db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.079+0000 7f39d801f740 1 --2- 172.21.15.69:0/3808588384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55784386df50 0x55784387d610 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.079+0000 7f39afe00640 0 client.28006 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:22.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.079+0000 7f39d801f740 1 -- 172.21.15.69:0/3808588384 >> 172.21.15.69:0/3808588384 conn(0x557843868680 msgr2=0x55784386a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:22.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.080+0000 7f39d801f740 1 -- 172.21.15.69:0/3808588384 shutdown_connections 2024-03-19T01:24:22.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.080+0000 7f39d801f740 1 -- 172.21.15.69:0/3808588384 wait complete. 2024-03-19T01:24:22.082 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:24:22.082 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-19T01:24:22.082 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:24:22.082 INFO:teuthology.orchestra.run.smithi069.stdout: "3840d255-9333-4b4a-afac-4a6d116119e6": { 2024-03-19T01:24:22.082 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:24:22.082 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "3840d255-9333-4b4a-afac-4a6d116119e6", 2024-03-19T01:24:22.082 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:24:22.082 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:24:22.082 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:24:22.082 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:24:22.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.080+0000 7f39d801f740 20 client.28006 trim_cache size 0 max 16384 2024-03-19T01:24:22.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.083+0000 7f39cf200640 1 -- 172.21.15.69:0/2248352216 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f39b40a29c0 msgr2=0x7f39b40a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:22.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.083+0000 7f39cf200640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f39b40a29c0 0x7f39b40a4e80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39c800ce60 tx=0x7f39c800fa50 comp rx=0 tx=0).stop 2024-03-19T01:24:22.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.083+0000 7f39cf200640 1 -- 172.21.15.69:0/2248352216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d010e380 msgr2=0x7f39d011caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:22.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.083+0000 7f39cf200640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d010e380 0x7f39d011caa0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f39c000c920 tx=0x7f39c000cde0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.084+0000 7f39cf200640 1 -- 172.21.15.69:0/2248352216 shutdown_connections 2024-03-19T01:24:22.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.084+0000 7f39cf200640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39d010fd40 0x7f39d0115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.084+0000 7f39cf200640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f39b40a29c0 0x7f39b40a4e80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.084+0000 7f39cf200640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d010e380 0x7f39d011caa0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.084+0000 7f39cf200640 1 --2- 172.21.15.69:0/2248352216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d010d900 0x7f39d011c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:22.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.084+0000 7f39cf200640 1 -- 172.21.15.69:0/2248352216 >> 172.21.15.69:0/2248352216 conn(0x7f39d006c6d0 msgr2=0x7f39d0070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:22.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.096+0000 7f39cf200640 1 -- 172.21.15.69:0/2248352216 shutdown_connections 2024-03-19T01:24:22.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:22.096+0000 7f39cf200640 1 -- 172.21.15.69:0/2248352216 wait complete. 2024-03-19T01:24:22.174 INFO:tasks.cephfs.filesystem:scrub status for tag:3840d255-9333-4b4a-afac-4a6d116119e6 - {'path': '/', 'tag': '3840d255-9333-4b4a-afac-4a6d116119e6', 'options': 'recursive,force'} 2024-03-19T01:24:22.346 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_uc.o 2024-03-19T01:24:22.444 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_uc_debugfs.o 2024-03-19T01:24:22.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:22 smithi086 ceph-mon[66829]: pgmap v429: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.0 MiB/s wr, 49 op/s 2024-03-19T01:24:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:22 smithi069 ceph-mon[31727]: pgmap v429: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.0 MiB/s wr, 49 op/s 2024-03-19T01:24:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:22 smithi163 ceph-mon[67428]: pgmap v429: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.0 MiB/s wr, 49 op/s 2024-03-19T01:24:22.565 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/uc/intel_uc_fw.o 2024-03-19T01:24:22.690 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/gt/intel_gsc.o 2024-03-19T01:24:22.965 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_hwmon.o 2024-03-19T01:24:22.985 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/drm_mipi_dsi.o 2024-03-19T01:24:23.557 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/hsw_ips.o 2024-03-19T01:24:23.757 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_atomic.o 2024-03-19T01:24:23.975 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_atomic_plane.o 2024-03-19T01:24:24.048 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_audio.o 2024-03-19T01:24:24.210 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_bios.o 2024-03-19T01:24:24.248 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_bw.o 2024-03-19T01:24:24.496 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_cdclk.o 2024-03-19T01:24:24.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:24 smithi086 ceph-mon[66829]: pgmap v430: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 54 op/s 2024-03-19T01:24:24.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:24 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:24:24.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:24 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:24:24.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:24 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:24:24.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:24 smithi069 ceph-mon[31727]: pgmap v430: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 54 op/s 2024-03-19T01:24:24.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:24 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:24:24.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:24 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:24:24.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:24 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:24:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:24 smithi163 ceph-mon[67428]: pgmap v430: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 54 op/s 2024-03-19T01:24:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:24 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:24:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:24 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:24:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:24 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:24:24.711 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_color.o 2024-03-19T01:24:25.340 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_combo_phy.o 2024-03-19T01:24:25.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:25 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:24:25.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:25 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:24:25.509 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_connector.o 2024-03-19T01:24:25.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:25 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:24:25.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:25 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:24:25.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:25 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:24:25.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:25 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:24:26.111 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_crtc.o 2024-03-19T01:24:26.119 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_crtc_state_dump.o 2024-03-19T01:24:26.425 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_cursor.o 2024-03-19T01:24:26.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:26 smithi086 ceph-mon[66829]: pgmap v431: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2024-03-19T01:24:26.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:24:26.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:26 smithi069 ceph-mon[31727]: pgmap v431: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2024-03-19T01:24:26.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:24:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:26 smithi163 ceph-mon[67428]: pgmap v431: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2024-03-19T01:24:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:24:26.788 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_display.o 2024-03-19T01:24:26.948 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_display_driver.o 2024-03-19T01:24:27.670 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_display_irq.o 2024-03-19T01:24:28.263 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_display_power.o 2024-03-19T01:24:28.405 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:28 smithi069 ceph-mon[31727]: pgmap v432: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.0 MiB/s wr, 51 op/s 2024-03-19T01:24:28.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:28 smithi086 ceph-mon[66829]: pgmap v432: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.0 MiB/s wr, 51 op/s 2024-03-19T01:24:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:28 smithi163 ceph-mon[67428]: pgmap v432: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.0 MiB/s wr, 51 op/s 2024-03-19T01:24:28.585 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_display_power_map.o 2024-03-19T01:24:28.594 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_display_power_well.o 2024-03-19T01:24:28.952 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_display_reset.o 2024-03-19T01:24:29.267 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_display_rps.o 2024-03-19T01:24:29.646 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dmc.o 2024-03-19T01:24:30.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:30 smithi086 ceph-mon[66829]: pgmap v433: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 355 KiB/s wr, 34 op/s 2024-03-19T01:24:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:30 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:24:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:30 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:24:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:30 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:24:30.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:30 smithi069 ceph-mon[31727]: pgmap v433: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 355 KiB/s wr, 34 op/s 2024-03-19T01:24:30.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:30 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:24:30.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:30 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:24:30.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:30 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:24:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:30 smithi163 ceph-mon[67428]: pgmap v433: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 355 KiB/s wr, 34 op/s 2024-03-19T01:24:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:30 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:24:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:30 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:24:30.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:30 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:24:30.566 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dpio_phy.o 2024-03-19T01:24:30.611 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dpll.o 2024-03-19T01:24:30.757 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dpll_mgr.o 2024-03-19T01:24:30.780 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dpt.o 2024-03-19T01:24:30.859 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_drrs.o 2024-03-19T01:24:31.026 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dsb.o 2024-03-19T01:24:31.515 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_fb.o 2024-03-19T01:24:32.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:32 smithi086 ceph-mon[66829]: pgmap v434: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 408 KiB/s wr, 34 op/s 2024-03-19T01:24:32.548 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:32 smithi069 ceph-mon[31727]: pgmap v434: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 408 KiB/s wr, 34 op/s 2024-03-19T01:24:32.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:32 smithi163 ceph-mon[67428]: pgmap v434: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 408 KiB/s wr, 34 op/s 2024-03-19T01:24:32.639 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_fbc.o 2024-03-19T01:24:32.639 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_fb_pin.o 2024-03-19T01:24:32.797 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_fdi.o 2024-03-19T01:24:32.820 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_fifo_underrun.o 2024-03-19T01:24:32.999 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_frontbuffer.o 2024-03-19T01:24:33.166 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_global_state.o 2024-03-19T01:24:34.139 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_hdcp.o 2024-03-19T01:24:34.183 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_hdcp_gsc.o 2024-03-19T01:24:34.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:34 smithi086 ceph-mon[66829]: pgmap v435: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 829 KiB/s wr, 50 op/s 2024-03-19T01:24:34.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:34 smithi069 ceph-mon[31727]: pgmap v435: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 829 KiB/s wr, 50 op/s 2024-03-19T01:24:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:34 smithi163 ceph-mon[67428]: pgmap v435: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 829 KiB/s wr, 50 op/s 2024-03-19T01:24:34.619 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_hotplug.o 2024-03-19T01:24:34.632 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_hotplug_irq.o 2024-03-19T01:24:34.803 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_hti.o 2024-03-19T01:24:34.806 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_load_detect.o 2024-03-19T01:24:35.016 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_lpe_audio.o 2024-03-19T01:24:35.070 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_modeset_lock.o 2024-03-19T01:24:35.753 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_modeset_verify.o 2024-03-19T01:24:36.126 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_modeset_setup.o 2024-03-19T01:24:36.290 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_overlay.o 2024-03-19T01:24:36.378 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_pch_display.o 2024-03-19T01:24:36.420 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_pch_refclk.o 2024-03-19T01:24:36.470 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_plane_initial.o 2024-03-19T01:24:36.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:36 smithi086 ceph-mon[66829]: pgmap v436: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 723 KiB/s wr, 37 op/s 2024-03-19T01:24:36.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:36 smithi069 ceph-mon[31727]: pgmap v436: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 723 KiB/s wr, 37 op/s 2024-03-19T01:24:36.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:36 smithi163 ceph-mon[67428]: pgmap v436: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 723 KiB/s wr, 37 op/s 2024-03-19T01:24:36.640 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_pmdemand.o 2024-03-19T01:24:36.666 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_psr.o 2024-03-19T01:24:37.185 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_quirks.o 2024-03-19T01:24:37.859 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_sprite.o 2024-03-19T01:24:38.120 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_sprite_uapi.o 2024-03-19T01:24:38.388 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_tc.o 2024-03-19T01:24:38.463 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_vblank.o 2024-03-19T01:24:38.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:38 smithi086 ceph-mon[66829]: pgmap v437: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 848 KiB/s wr, 42 op/s 2024-03-19T01:24:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:38 smithi069 ceph-mon[31727]: pgmap v437: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 848 KiB/s wr, 42 op/s 2024-03-19T01:24:38.541 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_vga.o 2024-03-19T01:24:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:38 smithi163 ceph-mon[67428]: pgmap v437: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 848 KiB/s wr, 42 op/s 2024-03-19T01:24:38.643 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_wm.o 2024-03-19T01:24:39.083 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/i9xx_plane.o 2024-03-19T01:24:39.553 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/i9xx_wm.o 2024-03-19T01:24:39.826 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/skl_scaler.o 2024-03-19T01:24:40.026 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/skl_universal_plane.o 2024-03-19T01:24:40.240 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/skl_watermark.o 2024-03-19T01:24:40.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:40 smithi086 ceph-mon[66829]: pgmap v438: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 863 KiB/s wr, 41 op/s 2024-03-19T01:24:40.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:40 smithi069 ceph-mon[31727]: pgmap v438: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 863 KiB/s wr, 41 op/s 2024-03-19T01:24:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:40 smithi163 ceph-mon[67428]: pgmap v438: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 863 KiB/s wr, 41 op/s 2024-03-19T01:24:40.746 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_acpi.o 2024-03-19T01:24:40.803 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_opregion.o 2024-03-19T01:24:40.984 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/dvo_ch7017.o 2024-03-19T01:24:41.411 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/dvo_ch7xxx.o 2024-03-19T01:24:42.179 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/dvo_ivch.o 2024-03-19T01:24:42.408 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/dvo_ns2501.o 2024-03-19T01:24:42.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:42 smithi086 ceph-mon[66829]: pgmap v439: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 840 KiB/s wr, 38 op/s 2024-03-19T01:24:42.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:42 smithi069 ceph-mon[31727]: pgmap v439: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 840 KiB/s wr, 38 op/s 2024-03-19T01:24:42.551 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/dvo_sil164.o 2024-03-19T01:24:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:42 smithi163 ceph-mon[67428]: pgmap v439: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 840 KiB/s wr, 38 op/s 2024-03-19T01:24:42.713 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/dvo_tfp410.o 2024-03-19T01:24:42.904 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/g4x_dp.o 2024-03-19T01:24:42.965 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/g4x_hdmi.o 2024-03-19T01:24:43.353 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/icl_dsi.o 2024-03-19T01:24:43.395 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_backlight.o 2024-03-19T01:24:43.884 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_crt.o 2024-03-19T01:24:43.986 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_cx0_phy.o 2024-03-19T01:24:44.017 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_ddi.o 2024-03-19T01:24:44.080 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_ddi_buf_trans.o 2024-03-19T01:24:44.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:44 smithi086 ceph-mon[66829]: pgmap v440: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 822 KiB/s wr, 41 op/s 2024-03-19T01:24:44.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:44 smithi069 ceph-mon[31727]: pgmap v440: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 822 KiB/s wr, 41 op/s 2024-03-19T01:24:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:44 smithi163 ceph-mon[67428]: pgmap v440: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 822 KiB/s wr, 41 op/s 2024-03-19T01:24:44.866 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_display_device.o 2024-03-19T01:24:45.782 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_display_trace.o 2024-03-19T01:24:45.807 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dkl_phy.o 2024-03-19T01:24:45.814 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dp.o 2024-03-19T01:24:45.820 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dp_aux.o 2024-03-19T01:24:46.366 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dp_aux_backlight.o 2024-03-19T01:24:46.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:46 smithi086 ceph-mon[66829]: pgmap v441: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 276 KiB/s wr, 18 op/s 2024-03-19T01:24:46.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:46 smithi069 ceph-mon[31727]: pgmap v441: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 276 KiB/s wr, 18 op/s 2024-03-19T01:24:46.577 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:46 smithi163 ceph-mon[67428]: pgmap v441: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 276 KiB/s wr, 18 op/s 2024-03-19T01:24:46.622 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dp_hdcp.o 2024-03-19T01:24:47.210 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dp_link_training.o 2024-03-19T01:24:47.457 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dp_mst.o 2024-03-19T01:24:47.688 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dsi.o 2024-03-19T01:24:47.771 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dsi_dcs_backlight.o 2024-03-19T01:24:48.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:48 smithi086 ceph-mon[66829]: pgmap v442: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 292 KiB/s wr, 21 op/s 2024-03-19T01:24:48.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:48 smithi069 ceph-mon[31727]: pgmap v442: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 292 KiB/s wr, 21 op/s 2024-03-19T01:24:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:48 smithi163 ceph-mon[67428]: pgmap v442: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 292 KiB/s wr, 21 op/s 2024-03-19T01:24:48.728 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dsi_vbt.o 2024-03-19T01:24:48.900 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_dvo.o 2024-03-19T01:24:49.020 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_gmbus.o 2024-03-19T01:24:49.274 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_hdmi.o 2024-03-19T01:24:49.527 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_lspcon.o 2024-03-19T01:24:49.547 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_lvds.o 2024-03-19T01:24:49.779 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_panel.o 2024-03-19T01:24:49.811 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_pps.o 2024-03-19T01:24:50.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:50 smithi086 ceph-mon[66829]: pgmap v443: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 224 KiB/s wr, 19 op/s 2024-03-19T01:24:50.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:50 smithi069 ceph-mon[31727]: pgmap v443: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 224 KiB/s wr, 19 op/s 2024-03-19T01:24:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:50 smithi163 ceph-mon[67428]: pgmap v443: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 224 KiB/s wr, 19 op/s 2024-03-19T01:24:50.793 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_qp_tables.o 2024-03-19T01:24:51.240 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_sdvo.o 2024-03-19T01:24:51.430 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_snps_phy.o 2024-03-19T01:24:51.514 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_tv.o 2024-03-19T01:24:51.565 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_vdsc.o 2024-03-19T01:24:51.693 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/intel_vrr.o 2024-03-19T01:24:51.729 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/vlv_dsi.o 2024-03-19T01:24:52.095 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/display/vlv_dsi_pll.o 2024-03-19T01:24:52.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:24:52.175 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:24:52.420 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_perf.o 2024-03-19T01:24:52.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:52 smithi086 ceph-mon[66829]: pgmap v444: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 251 KiB/s wr, 27 op/s 2024-03-19T01:24:52.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:52 smithi069 ceph-mon[31727]: pgmap v444: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 251 KiB/s wr, 27 op/s 2024-03-19T01:24:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:52 smithi163 ceph-mon[67428]: pgmap v444: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 251 KiB/s wr, 27 op/s 2024-03-19T01:24:53.543 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/pxp/intel_pxp.o 2024-03-19T01:24:53.600 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/pxp/intel_pxp_tee.o 2024-03-19T01:24:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.622+0000 7fac8a600640 1 -- 172.21.15.69:0/2196256184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac84109320 msgr2=0x7fac84114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.622+0000 7fac8a600640 1 --2- 172.21.15.69:0/2196256184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac84109320 0x7fac84114430 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fac74009900 tx=0x7fac7402bc30 comp rx=0 tx=0).stop 2024-03-19T01:24:53.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.624+0000 7fac8a600640 1 -- 172.21.15.69:0/2196256184 shutdown_connections 2024-03-19T01:24:53.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.624+0000 7fac8a600640 1 --2- 172.21.15.69:0/2196256184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac84114970 0x7fac84116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.624+0000 7fac8a600640 1 --2- 172.21.15.69:0/2196256184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac84109320 0x7fac84114430 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.624+0000 7fac8a600640 1 --2- 172.21.15.69:0/2196256184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac84108a00 0x7fac84108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.624+0000 7fac8a600640 1 -- 172.21.15.69:0/2196256184 >> 172.21.15.69:0/2196256184 conn(0x7fac8406c900 msgr2=0x7fac8406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:53.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.625+0000 7fac8a600640 1 -- 172.21.15.69:0/2196256184 shutdown_connections 2024-03-19T01:24:53.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.625+0000 7fac8a600640 1 -- 172.21.15.69:0/2196256184 wait complete. 2024-03-19T01:24:53.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.628+0000 7fac8a600640 1 Processor -- start 2024-03-19T01:24:53.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.629+0000 7fac8a600640 1 -- start start 2024-03-19T01:24:53.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.629+0000 7fac8a600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac84108a00 0x7fac84113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.629+0000 7fac8a600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac84109320 0x7fac84113d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.629+0000 7fac8a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac84114970 0x7fac841ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.629+0000 7fac8a600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac8411a3b0 con 0x7fac84108a00 2024-03-19T01:24:53.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.629+0000 7fac8a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac8411a230 con 0x7fac84114970 2024-03-19T01:24:53.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.629+0000 7fac8a600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fac8411a530 con 0x7fac84109320 2024-03-19T01:24:53.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.629+0000 7fac82a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac84109320 0x7fac84113d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.629+0000 7fac83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac84114970 0x7fac841ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.629+0000 7fac82a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac84109320 0x7fac84113d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:51136/0 (socket says 172.21.15.69:51136) 2024-03-19T01:24:53.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.629+0000 7fac82a00640 1 -- 172.21.15.69:0/574104450 learned_addr learned my addr 172.21.15.69:0/574104450 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:53.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.630+0000 7fac83e00640 1 -- 172.21.15.69:0/574104450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac84109320 msgr2=0x7fac84113d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:53.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.630+0000 7fac83e00640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac84109320 0x7fac84113d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.630+0000 7fac83e00640 1 -- 172.21.15.69:0/574104450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac84108a00 msgr2=0x7fac84113820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:24:53.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.633+0000 7fac83400640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac84108a00 0x7fac84113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.634+0000 7fac83e00640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac84108a00 0x7fac84113820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.634+0000 7fac83e00640 1 -- 172.21.15.69:0/574104450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac841abf70 con 0x7fac84114970 2024-03-19T01:24:53.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.634+0000 7fac83400640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac84108a00 0x7fac84113820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:24:53.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.635+0000 7fac83e00640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac84114970 0x7fac841ab870 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e970 tx=0x7fac7000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:53.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.636+0000 7fac7be00640 1 -- 172.21.15.69:0/574104450 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7000ccc0 con 0x7fac84114970 2024-03-19T01:24:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.636+0000 7fac7be00640 1 -- 172.21.15.69:0/574104450 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac7000ce60 con 0x7fac84114970 2024-03-19T01:24:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.636+0000 7fac7be00640 1 -- 172.21.15.69:0/574104450 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70010690 con 0x7fac84114970 2024-03-19T01:24:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.638+0000 7fac8a600640 1 -- 172.21.15.69:0/574104450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac841ac260 con 0x7fac84114970 2024-03-19T01:24:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.638+0000 7fac8a600640 1 -- 172.21.15.69:0/574104450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8419f880 con 0x7fac84114970 2024-03-19T01:24:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.645+0000 7fac8d40b740 1 Processor -- start 2024-03-19T01:24:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.647+0000 7fac7be00640 1 -- 172.21.15.69:0/574104450 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fac700035f0 con 0x7fac84114970 2024-03-19T01:24:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.650+0000 7fac7be00640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fac680a2af0 0x7fac680a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.650+0000 7fac8d40b740 1 -- start start 2024-03-19T01:24:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.650+0000 7fac82a00640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fac680a2af0 0x7fac680a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.650+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf7d40 0x562336cf8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.650+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf8660 0x562336d00d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.650+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336d01240 0x562336d03630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.650+0000 7fac8d40b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562336c5c2c0 con 0x562336d01240 2024-03-19T01:24:53.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.650+0000 7fac8d40b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562336c04b60 con 0x562336cf8660 2024-03-19T01:24:53.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.650+0000 7fac8d40b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562336c58360 con 0x562336cf7d40 2024-03-19T01:24:53.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.651+0000 7fac83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf7d40 0x562336cf8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.651+0000 7fac83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf7d40 0x562336cf8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:51138/0 (socket says 172.21.15.69:51138) 2024-03-19T01:24:53.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.651+0000 7fac83400640 1 -- 172.21.15.69:0/197470027 learned_addr learned my addr 172.21.15.69:0/197470027 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:53.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.652+0000 7fac83400640 1 -- 172.21.15.69:0/197470027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf8660 msgr2=0x562336d00d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:24:53.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.652+0000 7fac83400640 1 --2- 172.21.15.69:0/197470027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf8660 0x562336d00d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.652+0000 7fac83400640 1 -- 172.21.15.69:0/197470027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336d01240 msgr2=0x562336d03630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:24:53.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.654+0000 7fac7be00640 1 -- 172.21.15.69:0/574104450 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7fac700cf2d0 con 0x7fac84114970 2024-03-19T01:24:53.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.654+0000 7fac82a00640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fac680a2af0 0x7fac680a4fb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac84072660 tx=0x7fac7402df90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:53.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.654+0000 7fac83e00640 1 --2- 172.21.15.69:0/197470027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336d01240 0x562336d03630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.654+0000 7fac83400640 1 --2- 172.21.15.69:0/197470027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336d01240 0x562336d03630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.654+0000 7fac83400640 1 -- 172.21.15.69:0/197470027 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562336a3a7d0 con 0x562336cf7d40 2024-03-19T01:24:53.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.654+0000 7fac83e00640 1 --2- 172.21.15.69:0/197470027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336d01240 0x562336d03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:24:53.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.655+0000 7fac83400640 1 --2- 172.21.15.69:0/197470027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf7d40 0x562336cf8120 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009f20 tx=0x7fac7c00f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e932073e022e9342 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:53.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.655+0000 7fac78c00640 1 -- 172.21.15.69:0/197470027 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c003b70 con 0x562336cf7d40 2024-03-19T01:24:53.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.655+0000 7fac78c00640 1 -- 172.21.15.69:0/197470027 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac7c003d10 con 0x562336cf7d40 2024-03-19T01:24:53.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.655+0000 7fac78c00640 1 -- 172.21.15.69:0/197470027 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c011a30 con 0x562336cf7d40 2024-03-19T01:24:53.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.656+0000 7fac8d40b740 1 -- 172.21.15.69:0/197470027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf7d40 msgr2=0x562336cf8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:53.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.656+0000 7fac8d40b740 1 --2- 172.21.15.69:0/197470027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf7d40 0x562336cf8120 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009f20 tx=0x7fac7c00f7f0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.658+0000 7fac8d40b740 1 -- 172.21.15.69:0/197470027 shutdown_connections 2024-03-19T01:24:53.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.658+0000 7fac8d40b740 1 --2- 172.21.15.69:0/197470027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336d01240 0x562336d03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.658+0000 7fac8d40b740 1 --2- 172.21.15.69:0/197470027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf8660 0x562336d00d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.658+0000 7fac8d40b740 1 --2- 172.21.15.69:0/197470027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf7d40 0x562336cf8120 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.658+0000 7fac8d40b740 1 -- 172.21.15.69:0/197470027 >> 172.21.15.69:0/197470027 conn(0x562336cf2470 msgr2=0x562336cf2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:53.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.658+0000 7fac8d40b740 1 -- 172.21.15.69:0/197470027 shutdown_connections 2024-03-19T01:24:53.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.658+0000 7fac8d40b740 1 -- 172.21.15.69:0/197470027 wait complete. 2024-03-19T01:24:53.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.659+0000 7fac8d40b740 1 -- start start 2024-03-19T01:24:53.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.659+0000 7fac8d40b740 1 Processor -- start 2024-03-19T01:24:53.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.663+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf7d40 0x562336cfe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.663+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf8660 0x562336cfe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.663+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336d01240 0x562336cfed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.663+0000 7fac8d40b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562336d04030 con 0x562336cf8660 2024-03-19T01:24:53.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.663+0000 7fac8d40b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562336c17fc0 con 0x562336cf7d40 2024-03-19T01:24:53.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.663+0000 7fac8d40b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562336d041b0 con 0x562336d01240 2024-03-19T01:24:53.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.663+0000 7fac8d40b740 10 client.0 _ll_register_callbacks cb 0x562336c17990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:53.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.664+0000 7fac83e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336d01240 0x562336cfed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.664+0000 7fac83e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336d01240 0x562336cfed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:51148/0 (socket says 172.21.15.69:51148) 2024-03-19T01:24:53.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.664+0000 7fac83e00640 1 -- 172.21.15.69:0/3478046458 learned_addr learned my addr 172.21.15.69:0/3478046458 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:53.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.664+0000 7fac83e00640 1 -- 172.21.15.69:0/3478046458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf7d40 msgr2=0x562336cfe280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:24:53.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.664+0000 7fac83e00640 1 --2- 172.21.15.69:0/3478046458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf7d40 0x562336cfe280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.664+0000 7fac83e00640 1 -- 172.21.15.69:0/3478046458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf8660 msgr2=0x562336cfe7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:24:53.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.664+0000 7fac83e00640 1 --2- 172.21.15.69:0/3478046458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf8660 0x562336cfe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.664+0000 7fac83e00640 1 -- 172.21.15.69:0/3478046458 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623369703f0 con 0x562336d01240 2024-03-19T01:24:53.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.666+0000 7fac83e00640 1 --2- 172.21.15.69:0/3478046458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336d01240 0x562336cfed00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fac7000d010 tx=0x7fac7009bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed557261d43efbe9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:53.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.666+0000 7fac6d600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:24:53.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.666+0000 7fac6d600640 1 -- 172.21.15.69:0/3478046458 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac700b3020 con 0x562336d01240 2024-03-19T01:24:53.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.666+0000 7fac6d600640 1 -- 172.21.15.69:0/3478046458 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7fac70097b90 con 0x562336d01240 2024-03-19T01:24:53.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.666+0000 7fac8d40b740 1 -- 172.21.15.69:0/3478046458 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac7c003d10 con 0x562336d01240 2024-03-19T01:24:53.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.667+0000 7fac6d600640 1 -- 172.21.15.69:0/3478046458 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fac700a3cc0 con 0x562336d01240 2024-03-19T01:24:53.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.667+0000 7fac8d40b740 10 client.28018 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:53.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.667+0000 7fac8d40b740 1 -- 172.21.15.69:0/3478046458 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5623369300f0 con 0x562336d01240 2024-03-19T01:24:53.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.669+0000 7fac6d600640 1 -- 172.21.15.69:0/3478046458 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac7c003d10 con 0x562336d01240 2024-03-19T01:24:53.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.669+0000 7fac6d600640 1 -- 172.21.15.69:0/3478046458 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fac7009a970 con 0x562336d01240 2024-03-19T01:24:53.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.669+0000 7fac8d40b740 10 client.28018 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:53.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.669+0000 7fac8d40b740 10 client.28018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:24:53.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.669+0000 7fac8d40b740 20 client.28018 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:53.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.669+0000 7fac8d40b740 1 --2- 172.21.15.69:0/3478046458 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562336d11610 0x562336d13a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.669+0000 7fac8d40b740 4 client.28018 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:24:53.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.669+0000 7fac8d40b740 1 -- 172.21.15.69:0/3478046458 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562336bfd610 con 0x562336d11610 2024-03-19T01:24:53.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.670+0000 7fac83400640 1 --2- 172.21.15.69:0/3478046458 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562336d11610 0x562336d13a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.670+0000 7fac83400640 1 --2- 172.21.15.69:0/3478046458 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562336d11610 0x562336d13a40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39be2816ff7c1a2d server_cookie=c4690a4286a09a6 in_seq=0 out_seq=0 2024-03-19T01:24:53.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.670+0000 7fac6d600640 10 client.28018 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:53.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.671+0000 7fac6d600640 1 -- 172.21.15.69:0/3478046458 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562336bfd610 con 0x562336d11610 2024-03-19T01:24:53.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.671+0000 7fac6d600640 10 client.28018 handle_command_reply: tid=0 2024-03-19T01:24:53.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.671+0000 7fac8d40b740 1 client.28018 shutdown 2024-03-19T01:24:53.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.672+0000 7fac8d40b740 1 -- 172.21.15.69:0/3478046458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336d01240 msgr2=0x562336cfed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:53.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.672+0000 7fac8d40b740 1 --2- 172.21.15.69:0/3478046458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336d01240 0x562336cfed00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fac7000d010 tx=0x7fac7009bca0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.672+0000 7fac8d40b740 1 -- 172.21.15.69:0/3478046458 shutdown_connections 2024-03-19T01:24:53.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.672+0000 7fac8d40b740 1 --2- 172.21.15.69:0/3478046458 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562336d11610 0x562336d13a40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.672+0000 7fac8d40b740 1 --2- 172.21.15.69:0/3478046458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336d01240 0x562336cfed00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.672+0000 7fac6d600640 0 client.28018 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:53.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.672+0000 7fac8d40b740 1 --2- 172.21.15.69:0/3478046458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf8660 0x562336cfe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.672+0000 7fac8d40b740 1 --2- 172.21.15.69:0/3478046458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf7d40 0x562336cfe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.672+0000 7fac8d40b740 1 -- 172.21.15.69:0/3478046458 >> 172.21.15.69:0/3478046458 conn(0x562336cf2470 msgr2=0x562336cf41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:53.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.675+0000 7fac8d40b740 1 -- 172.21.15.69:0/3478046458 shutdown_connections 2024-03-19T01:24:53.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.675+0000 7fac8d40b740 1 -- 172.21.15.69:0/3478046458 wait complete. 2024-03-19T01:24:53.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.675+0000 7fac8d40b740 20 client.28018 trim_cache size 0 max 16384 2024-03-19T01:24:53.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.753+0000 7fac8d40b740 1 Processor -- start 2024-03-19T01:24:53.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.753+0000 7fac8d40b740 1 -- start start 2024-03-19T01:24:53.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.753+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf2470 0x562336d0ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.753+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf7d40 0x562336cf3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.754+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf8660 0x562336d19310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.754+0000 7fac82a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf7d40 0x562336cf3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.754+0000 7fac82a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf7d40 0x562336cf3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38704/0 (socket says 172.21.15.69:38704) 2024-03-19T01:24:53.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.754+0000 7fac8d40b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac64010210 con 0x562336cf7d40 2024-03-19T01:24:53.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.754+0000 7fac8d40b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac64011d50 con 0x562336cf8660 2024-03-19T01:24:53.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.754+0000 7fac8d40b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fac640108e0 con 0x562336cf2470 2024-03-19T01:24:53.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.754+0000 7fac83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf8660 0x562336d19310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.754+0000 7fac82a00640 1 -- 172.21.15.69:0/466319309 learned_addr learned my addr 172.21.15.69:0/466319309 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:53.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.755+0000 7fac83e00640 1 -- 172.21.15.69:0/466319309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf2470 msgr2=0x562336d0ce30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:24:53.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.755+0000 7fac83e00640 1 --2- 172.21.15.69:0/466319309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf2470 0x562336d0ce30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.755+0000 7fac83e00640 1 -- 172.21.15.69:0/466319309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf7d40 msgr2=0x562336cf3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:53.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.755+0000 7fac83e00640 1 --2- 172.21.15.69:0/466319309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf7d40 0x562336cf3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.755+0000 7fac83e00640 1 -- 172.21.15.69:0/466319309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562336cf22a0 con 0x562336cf8660 2024-03-19T01:24:53.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.755+0000 7fac82a00640 1 --2- 172.21.15.69:0/466319309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf7d40 0x562336cf3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:53.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.756+0000 7fac83e00640 1 --2- 172.21.15.69:0/466319309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf8660 0x562336d19310 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fac700997d0 tx=0x7fac7009b6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b880bc6fb5a692d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:53.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.757+0000 7fac6fe00640 1 -- 172.21.15.69:0/466319309 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac700a3cc0 con 0x562336cf8660 2024-03-19T01:24:53.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.757+0000 7fac6fe00640 1 -- 172.21.15.69:0/466319309 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac70097b90 con 0x562336cf8660 2024-03-19T01:24:53.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.757+0000 7fac6fe00640 1 -- 172.21.15.69:0/466319309 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7009f170 con 0x562336cf8660 2024-03-19T01:24:53.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.758+0000 7fac8d40b740 1 -- 172.21.15.69:0/466319309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf8660 msgr2=0x562336d19310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:53.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.758+0000 7fac8d40b740 1 --2- 172.21.15.69:0/466319309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf8660 0x562336d19310 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fac700997d0 tx=0x7fac7009b6a0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.763+0000 7fac8d40b740 1 -- 172.21.15.69:0/466319309 shutdown_connections 2024-03-19T01:24:53.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.763+0000 7fac8d40b740 1 --2- 172.21.15.69:0/466319309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf8660 0x562336d19310 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.763+0000 7fac8d40b740 1 --2- 172.21.15.69:0/466319309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf7d40 0x562336cf3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.763+0000 7fac8d40b740 1 --2- 172.21.15.69:0/466319309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf2470 0x562336d0ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.763+0000 7fac8d40b740 1 -- 172.21.15.69:0/466319309 >> 172.21.15.69:0/466319309 conn(0x562336cfcec0 msgr2=0x562336cf38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:53.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.763+0000 7fac8d40b740 1 -- 172.21.15.69:0/466319309 shutdown_connections 2024-03-19T01:24:53.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.763+0000 7fac8d40b740 1 -- 172.21.15.69:0/466319309 wait complete. 2024-03-19T01:24:53.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.763+0000 7fac8d40b740 1 -- start start 2024-03-19T01:24:53.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.764+0000 7fac8d40b740 1 Processor -- start 2024-03-19T01:24:53.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.767+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf2470 0x562336d05f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.767+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf7d40 0x562336d064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.768+0000 7fac8d40b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf8660 0x562336d06a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.768+0000 7fac8d40b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac64011680 con 0x562336cf8660 2024-03-19T01:24:53.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.768+0000 7fac8d40b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac64010fb0 con 0x562336cf7d40 2024-03-19T01:24:53.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.768+0000 7fac8d40b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fac6400e1b0 con 0x562336cf2470 2024-03-19T01:24:53.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.768+0000 7fac83e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf8660 0x562336d06a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.768+0000 7fac83e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf8660 0x562336d06a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38706/0 (socket says 172.21.15.69:38706) 2024-03-19T01:24:53.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.768+0000 7fac83e00640 1 -- 172.21.15.69:0/4128026982 learned_addr learned my addr 172.21.15.69:0/4128026982 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:53.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.768+0000 7fac8d40b740 10 client.0 _ll_register_callbacks cb 0x7fac7009ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:53.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.769+0000 7fac82a00640 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf7d40 0x562336d064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.769+0000 7fac83e00640 1 -- 172.21.15.69:0/4128026982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf2470 msgr2=0x562336d05f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:53.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.769+0000 7fac83e00640 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf2470 0x562336d05f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.769+0000 7fac83e00640 1 -- 172.21.15.69:0/4128026982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf7d40 msgr2=0x562336d064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:53.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.769+0000 7fac83e00640 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf7d40 0x562336d064c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.769+0000 7fac83e00640 1 -- 172.21.15.69:0/4128026982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562336d77ba0 con 0x562336cf8660 2024-03-19T01:24:53.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.770+0000 7fac83e00640 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf8660 0x562336d06a00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fac7009b4b0 tx=0x7fac70003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5fb26e09f88147 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.770+0000 7fac6d600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:24:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.770+0000 7fac6d600640 1 -- 172.21.15.69:0/4128026982 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac700a3cc0 con 0x562336cf8660 2024-03-19T01:24:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.770+0000 7fac6d600640 1 -- 172.21.15.69:0/4128026982 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7fac70097270 con 0x562336cf8660 2024-03-19T01:24:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.770+0000 7fac8d40b740 1 -- 172.21.15.69:0/4128026982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac70097b90 con 0x562336cf8660 2024-03-19T01:24:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.771+0000 7fac6d600640 1 -- 172.21.15.69:0/4128026982 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fac70097470 con 0x562336cf8660 2024-03-19T01:24:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.771+0000 7fac8d40b740 10 client.17694 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.771+0000 7fac8d40b740 1 -- 172.21.15.69:0/4128026982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x562336d78e70 con 0x562336cf8660 2024-03-19T01:24:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.771+0000 7fac6d600640 1 -- 172.21.15.69:0/4128026982 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac70097b90 con 0x562336cf8660 2024-03-19T01:24:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.771+0000 7fac6d600640 1 -- 172.21.15.69:0/4128026982 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fac700a0590 con 0x562336cf8660 2024-03-19T01:24:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.772+0000 7fac8d40b740 10 client.17694 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.772+0000 7fac8d40b740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.772+0000 7fac8d40b740 20 client.17694 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.772+0000 7fac8d40b740 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562336d01240 0x562336d7b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.772+0000 7fac8d40b740 4 client.17694 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.772+0000 7fac8d40b740 1 -- 172.21.15.69:0/4128026982 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562336c69800 con 0x562336d01240 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.772+0000 7fac83400640 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562336d01240 0x562336d7b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.773+0000 7fac6d600640 10 client.17694 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.773+0000 7fac83400640 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562336d01240 0x562336d7b390 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c94f70f9d7d0a02 server_cookie=9541cc7850a928fa in_seq=0 out_seq=0 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.773+0000 7fac6d600640 1 -- 172.21.15.69:0/4128026982 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562336c69800 con 0x562336d01240 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.773+0000 7fac6d600640 10 client.17694 handle_command_reply: tid=0 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.773+0000 7fac8d40b740 1 client.17694 shutdown 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.773+0000 7fac8d40b740 1 -- 172.21.15.69:0/4128026982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf8660 msgr2=0x562336d06a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:53.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.773+0000 7fac8d40b740 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf8660 0x562336d06a00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fac7009b4b0 tx=0x7fac70003af0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.777+0000 7fac8d40b740 1 -- 172.21.15.69:0/4128026982 shutdown_connections 2024-03-19T01:24:53.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.777+0000 7fac8d40b740 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x562336d01240 0x562336d7b390 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.777+0000 7fac8d40b740 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562336cf8660 0x562336d06a00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.777+0000 7fac8d40b740 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562336cf7d40 0x562336d064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.777+0000 7fac8d40b740 1 --2- 172.21.15.69:0/4128026982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562336cf2470 0x562336d05f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.777+0000 7fac6d600640 0 client.17694 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:53.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.777+0000 7fac8d40b740 1 -- 172.21.15.69:0/4128026982 >> 172.21.15.69:0/4128026982 conn(0x562336cfcec0 msgr2=0x562336cfead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:53.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.777+0000 7fac8d40b740 1 -- 172.21.15.69:0/4128026982 shutdown_connections 2024-03-19T01:24:53.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.777+0000 7fac8d40b740 1 -- 172.21.15.69:0/4128026982 wait complete. 2024-03-19T01:24:53.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.777+0000 7fac8d40b740 20 client.17694 trim_cache size 0 max 16384 2024-03-19T01:24:53.779 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:24:53.779 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:24:53.779 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:24:53.779 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:24:53.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.780+0000 7fac8a600640 1 -- 172.21.15.69:0/574104450 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fac680a2af0 msgr2=0x7fac680a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:53.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.780+0000 7fac8a600640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fac680a2af0 0x7fac680a4fb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac84072660 tx=0x7fac7402df90 comp rx=0 tx=0).stop 2024-03-19T01:24:53.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.780+0000 7fac8a600640 1 -- 172.21.15.69:0/574104450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac84114970 msgr2=0x7fac841ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:53.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.780+0000 7fac8a600640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac84114970 0x7fac841ab870 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e970 tx=0x7fac7000ee30 comp rx=0 tx=0).stop 2024-03-19T01:24:53.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.780+0000 7fac8a600640 1 -- 172.21.15.69:0/574104450 shutdown_connections 2024-03-19T01:24:53.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.780+0000 7fac8a600640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fac680a2af0 0x7fac680a4fb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.780+0000 7fac8a600640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac84114970 0x7fac841ab870 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.780+0000 7fac8a600640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac84109320 0x7fac84113d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.780+0000 7fac8a600640 1 --2- 172.21.15.69:0/574104450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac84108a00 0x7fac84113820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:53.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.780+0000 7fac8a600640 1 -- 172.21.15.69:0/574104450 >> 172.21.15.69:0/574104450 conn(0x7fac8406c900 msgr2=0x7fac84071800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:53.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.780+0000 7fac8a600640 1 -- 172.21.15.69:0/574104450 shutdown_connections 2024-03-19T01:24:53.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:53.784+0000 7fac8a600640 1 -- 172.21.15.69:0/574104450 wait complete. 2024-03-19T01:24:53.862 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:24:53.862 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:24:54.047 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/pxp/intel_pxp_huc.o 2024-03-19T01:24:54.080 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_gpu_error.o 2024-03-19T01:24:54.093 INFO:tasks.workunit.client.0.smithi069.stdout: CC drivers/gpu/drm/i915/i915_vgpu.o 2024-03-19T01:24:54.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.503+0000 7f9c12a00640 1 -- 172.21.15.69:0/3228446994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c10d180 msgr2=0x7f9c0c079b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:54.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.503+0000 7f9c12a00640 1 --2- 172.21.15.69:0/3228446994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c10d180 0x7f9c0c079b30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009ca0 tx=0x7f9c0402c550 comp rx=0 tx=0).stop 2024-03-19T01:24:54.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.503+0000 7f9c12a00640 1 -- 172.21.15.69:0/3228446994 shutdown_connections 2024-03-19T01:24:54.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.503+0000 7f9c12a00640 1 --2- 172.21.15.69:0/3228446994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c10d180 0x7f9c0c079b30 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:54.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.503+0000 7f9c12a00640 1 --2- 172.21.15.69:0/3228446994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c0c109440 0x7f9c0c10ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:54.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.503+0000 7f9c12a00640 1 --2- 172.21.15.69:0/3228446994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c108a90 0x7f9c0c108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:54.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.503+0000 7f9c12a00640 1 -- 172.21.15.69:0/3228446994 >> 172.21.15.69:0/3228446994 conn(0x7f9c0c06c910 msgr2=0x7f9c0c06cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:54.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.503+0000 7f9c12a00640 1 -- 172.21.15.69:0/3228446994 shutdown_connections 2024-03-19T01:24:54.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.503+0000 7f9c12a00640 1 -- 172.21.15.69:0/3228446994 wait complete. 2024-03-19T01:24:54.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.505+0000 7f9c12a00640 1 Processor -- start 2024-03-19T01:24:54.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:54 smithi086 ceph-mon[66829]: pgmap v445: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 897 KiB/s wr, 89 op/s 2024-03-19T01:24:54.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c12a00640 1 -- start start 2024-03-19T01:24:54.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c12a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c108a90 0x7f9c0c07e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:54.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c12a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c0c10d180 0x7f9c0c07ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:54.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c12a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c07f140 0x7f9c0c136cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:54.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c12a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c07d030 con 0x7f9c0c10d180 2024-03-19T01:24:54.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c12a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c07ceb0 con 0x7f9c0c07f140 2024-03-19T01:24:54.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c12a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c07d1b0 con 0x7f9c0c108a90 2024-03-19T01:24:54.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c10a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c07f140 0x7f9c0c136cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:54.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c0be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c108a90 0x7f9c0c07e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:54.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c10a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c07f140 0x7f9c0c136cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:39374/0 (socket says 172.21.15.69:39374) 2024-03-19T01:24:54.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c0be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c108a90 0x7f9c0c07e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:51170/0 (socket says 172.21.15.69:51170) 2024-03-19T01:24:54.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.507+0000 7f9c10a00640 1 -- 172.21.15.69:0/3296179704 learned_addr learned my addr 172.21.15.69:0/3296179704 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:54.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.508+0000 7f9c0b400640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c0c10d180 0x7f9c0c07ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:54.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.508+0000 7f9c10a00640 1 -- 172.21.15.69:0/3296179704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c108a90 msgr2=0x7f9c0c07e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:54.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.508+0000 7f9c10a00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c108a90 0x7f9c0c07e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:54.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.508+0000 7f9c10a00640 1 -- 172.21.15.69:0/3296179704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c0c10d180 msgr2=0x7f9c0c07ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:54.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.508+0000 7f9c10a00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c0c10d180 0x7f9c0c07ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:54.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.508+0000 7f9c10a00640 1 -- 172.21.15.69:0/3296179704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c137200 con 0x7f9c0c07f140 2024-03-19T01:24:54.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.508+0000 7f9c0be00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c108a90 0x7f9c0c07e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:54.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.508+0000 7f9c0b400640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c0c10d180 0x7f9c0c07ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:24:54.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.508+0000 7f9c10a00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c07f140 0x7f9c0c136cc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9c0402cb60 tx=0x7f9c040339c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:54.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.510+0000 7f9c08c00640 1 -- 172.21.15.69:0/3296179704 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04003d80 con 0x7f9c0c07f140 2024-03-19T01:24:54.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.510+0000 7f9c08c00640 1 -- 172.21.15.69:0/3296179704 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c040319f0 con 0x7f9c0c07f140 2024-03-19T01:24:54.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.510+0000 7f9c08c00640 1 -- 172.21.15.69:0/3296179704 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0403e420 con 0x7f9c0c07f140 2024-03-19T01:24:54.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.510+0000 7f9c12a00640 1 -- 172.21.15.69:0/3296179704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c137490 con 0x7f9c0c07f140 2024-03-19T01:24:54.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.510+0000 7f9c12a00640 1 -- 172.21.15.69:0/3296179704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c137930 con 0x7f9c0c07f140 2024-03-19T01:24:54.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.514+0000 7f9c12a00640 1 -- 172.21.15.69:0/3296179704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0c1392a0 con 0x7f9c0c07f140 2024-03-19T01:24:54.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.514+0000 7f9c08c00640 1 -- 172.21.15.69:0/3296179704 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9c0403b040 con 0x7f9c0c07f140 2024-03-19T01:24:54.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.516+0000 7f9c08c00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bf00a2a30 0x7f9bf00a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:54.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.517+0000 7f9c08c00640 1 -- 172.21.15.69:0/3296179704 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f9c040f20c0 con 0x7f9c0c07f140 2024-03-19T01:24:54.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.517+0000 7f9c0be00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bf00a2a30 0x7f9bf00a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:54.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.522+0000 7f9c0be00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bf00a2a30 0x7f9bf00a4ef0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009fc0 tx=0x7f9bf4005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:54.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:54.524+0000 7f9c08c00640 1 -- 172.21.15.69:0/3296179704 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9c04042030 con 0x7f9c0c07f140 2024-03-19T01:24:54.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:54 smithi069 ceph-mon[31727]: pgmap v445: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 897 KiB/s wr, 89 op/s 2024-03-19T01:24:54.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:54 smithi163 ceph-mon[67428]: pgmap v445: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 897 KiB/s wr, 89 op/s 2024-03-19T01:24:55.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.203+0000 7f9c12a00640 1 -- 172.21.15.69:0/3296179704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c0c07fba0 con 0x7f9c0c07f140 2024-03-19T01:24:55.205 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:24:55.205 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":59,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:20:08.600901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:24:55.205 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 59 2024-03-19T01:24:55.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.204+0000 7f9c08c00640 1 -- 172.21.15.69:0/3296179704 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11930 (secure 0 0 0) 0x7f9c040bea10 con 0x7f9c0c07f140 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.208+0000 7f9c12a00640 1 -- 172.21.15.69:0/3296179704 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bf00a2a30 msgr2=0x7f9bf00a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.208+0000 7f9c12a00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bf00a2a30 0x7f9bf00a4ef0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009fc0 tx=0x7f9bf4005f90 comp rx=0 tx=0).stop 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.208+0000 7f9c12a00640 1 -- 172.21.15.69:0/3296179704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c07f140 msgr2=0x7f9c0c136cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.208+0000 7f9c12a00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c07f140 0x7f9c0c136cc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9c0402cb60 tx=0x7f9c040339c0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.208+0000 7f9c12a00640 1 -- 172.21.15.69:0/3296179704 shutdown_connections 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.208+0000 7f9c12a00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9bf00a2a30 0x7f9bf00a4ef0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.208+0000 7f9c12a00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c0c07f140 0x7f9c0c136cc0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.208+0000 7f9c12a00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c0c10d180 0x7f9c0c07ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.208+0000 7f9c12a00640 1 --2- 172.21.15.69:0/3296179704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c108a90 0x7f9c0c07e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.208+0000 7f9c12a00640 1 -- 172.21.15.69:0/3296179704 >> 172.21.15.69:0/3296179704 conn(0x7f9c0c06c910 msgr2=0x7f9c0c071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.209+0000 7f9c12a00640 1 -- 172.21.15.69:0/3296179704 shutdown_connections 2024-03-19T01:24:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.209+0000 7f9c12a00640 1 -- 172.21.15.69:0/3296179704 wait complete. 2024-03-19T01:24:55.238 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:24:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.473+0000 7fdec9a00640 1 -- 172.21.15.69:0/1279005136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec40744f0 msgr2=0x7fdec410db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.473+0000 7fdec9a00640 1 --2- 172.21.15.69:0/1279005136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec40744f0 0x7fdec410db30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009900 tx=0x7fdeac02bcd0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.473+0000 7fdec9a00640 1 -- 172.21.15.69:0/1279005136 shutdown_connections 2024-03-19T01:24:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.473+0000 7fdec9a00640 1 --2- 172.21.15.69:0/1279005136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec410e070 0x7fdec4110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.473+0000 7fdec9a00640 1 --2- 172.21.15.69:0/1279005136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec40744f0 0x7fdec410db30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.473+0000 7fdec9a00640 1 --2- 172.21.15.69:0/1279005136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec4073bd0 0x7fdec4073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.473+0000 7fdec9a00640 1 -- 172.21.15.69:0/1279005136 >> 172.21.15.69:0/1279005136 conn(0x7fdec4078c20 msgr2=0x7fdec4079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.473+0000 7fdec9a00640 1 -- 172.21.15.69:0/1279005136 shutdown_connections 2024-03-19T01:24:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.473+0000 7fdec9a00640 1 -- 172.21.15.69:0/1279005136 wait complete. 2024-03-19T01:24:55.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.474+0000 7fdec9a00640 1 Processor -- start 2024-03-19T01:24:55.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.474+0000 7fdec9a00640 1 -- start start 2024-03-19T01:24:55.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.475+0000 7fdec9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec4073bd0 0x7fdec419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.475+0000 7fdec9a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec40744f0 0x7fdec419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.475+0000 7fdec9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec410e070 0x7fdec419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.475+0000 7fdec9a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4113de0 con 0x7fdec40744f0 2024-03-19T01:24:55.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.475+0000 7fdec9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4113c60 con 0x7fdec410e070 2024-03-19T01:24:55.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.475+0000 7fdec9a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4113f60 con 0x7fdec4073bd0 2024-03-19T01:24:55.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.475+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec4073bd0 0x7fdec419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.475+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec4073bd0 0x7fdec419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:51186/0 (socket says 172.21.15.69:51186) 2024-03-19T01:24:55.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.475+0000 7fdec2a00640 1 -- 172.21.15.69:0/3456517377 learned_addr learned my addr 172.21.15.69:0/3456517377 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:55.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.475+0000 7fdec3400640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec410e070 0x7fdec419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.476+0000 7fdec2a00640 1 -- 172.21.15.69:0/3456517377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec410e070 msgr2=0x7fdec419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.476+0000 7fdec2a00640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec410e070 0x7fdec419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.476+0000 7fdec2a00640 1 -- 172.21.15.69:0/3456517377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec40744f0 msgr2=0x7fdec419a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:24:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.477+0000 7fdebbe00640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec40744f0 0x7fdec419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.477+0000 7fdec2a00640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec40744f0 0x7fdec419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.477+0000 7fdec2a00640 1 -- 172.21.15.69:0/3456517377 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec419f290 con 0x7fdec4073bd0 2024-03-19T01:24:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.477+0000 7fdebbe00640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec40744f0 0x7fdec419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:24:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.477+0000 7fdec2a00640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec4073bd0 0x7fdec419a2a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400d950 tx=0x7fdeb400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.477+0000 7fdebb400640 1 -- 172.21.15.69:0/3456517377 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4017070 con 0x7fdec4073bd0 2024-03-19T01:24:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.477+0000 7fdebb400640 1 -- 172.21.15.69:0/3456517377 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb4009c30 con 0x7fdec4073bd0 2024-03-19T01:24:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.478+0000 7fdec9a00640 1 -- 172.21.15.69:0/3456517377 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec419f580 con 0x7fdec4073bd0 2024-03-19T01:24:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.478+0000 7fdebb400640 1 -- 172.21.15.69:0/3456517377 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400f6b0 con 0x7fdec4073bd0 2024-03-19T01:24:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.478+0000 7fdec9a00640 1 -- 172.21.15.69:0/3456517377 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec41a6e10 con 0x7fdec4073bd0 2024-03-19T01:24:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.482+0000 7fdecc7f1740 1 Processor -- start 2024-03-19T01:24:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.482+0000 7fdecc7f1740 1 -- start start 2024-03-19T01:24:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.482+0000 7fdebb400640 1 -- 172.21.15.69:0/3456517377 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fdeb400f850 con 0x7fdec4073bd0 2024-03-19T01:24:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.483+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8ca20 0x55ad93a8ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.483+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8d340 0x55ad93a959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.483+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a98310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.483+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ad939f1380 con 0x55ad93a8ca20 2024-03-19T01:24:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.483+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad93999d20 con 0x55ad93a8d340 2024-03-19T01:24:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.483+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ad939ed420 con 0x55ad93a95f20 2024-03-19T01:24:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.483+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8d340 0x55ad93a959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.483+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8d340 0x55ad93a959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:39416/0 (socket says 172.21.15.69:39416) 2024-03-19T01:24:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.483+0000 7fdec2a00640 1 -- 172.21.15.69:0/2876796844 learned_addr learned my addr 172.21.15.69:0/2876796844 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.483+0000 7fdec3400640 1 --2- 172.21.15.69:0/2876796844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8ca20 0x55ad93a8ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.484+0000 7fdec2a00640 1 -- 172.21.15.69:0/2876796844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 msgr2=0x55ad93a98310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:24:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.484+0000 7fdec2a00640 1 --2- 172.21.15.69:0/2876796844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a98310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.484+0000 7fdec2a00640 1 -- 172.21.15.69:0/2876796844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8ca20 msgr2=0x55ad93a8ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.484+0000 7fdec2a00640 1 --2- 172.21.15.69:0/2876796844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8ca20 0x55ad93a8ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.484+0000 7fdec2a00640 1 -- 172.21.15.69:0/2876796844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad937053f0 con 0x55ad93a8d340 2024-03-19T01:24:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.484+0000 7fdec2a00640 1 --2- 172.21.15.69:0/2876796844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8d340 0x55ad93a959e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40cc0a0 tx=0x7fdeb40d2f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=781d3226a91c37ba server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.484+0000 7fdeb3e00640 1 -- 172.21.15.69:0/2876796844 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40e0430 con 0x55ad93a8d340 2024-03-19T01:24:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.485+0000 7fdeb3e00640 1 -- 172.21.15.69:0/2876796844 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb40d3910 con 0x55ad93a8d340 2024-03-19T01:24:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.485+0000 7fdebb400640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdea80a2a70 0x7fdea80a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.485+0000 7fdeb3e00640 1 -- 172.21.15.69:0/2876796844 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40df8b0 con 0x55ad93a8d340 2024-03-19T01:24:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.485+0000 7fdebb400640 1 -- 172.21.15.69:0/3456517377 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7fdeb40cee60 con 0x7fdec4073bd0 2024-03-19T01:24:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.485+0000 7fdec3400640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdea80a2a70 0x7fdea80a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.486+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2876796844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8d340 msgr2=0x55ad93a959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.486+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2876796844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8d340 0x55ad93a959e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40cc0a0 tx=0x7fdeb40d2f90 comp rx=0 tx=0).stop 2024-03-19T01:24:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.486+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2876796844 shutdown_connections 2024-03-19T01:24:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.486+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2876796844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a98310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.486+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2876796844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8d340 0x55ad93a959e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.486+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2876796844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8ca20 0x55ad93a8ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.486+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2876796844 >> 172.21.15.69:0/2876796844 conn(0x55ad93a87150 msgr2=0x55ad93a87530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.486+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2876796844 shutdown_connections 2024-03-19T01:24:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.486+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2876796844 wait complete. 2024-03-19T01:24:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.487+0000 7fdec3400640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdea80a2a70 0x7fdea80a4f30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005fd0 tx=0x7fdebc0096a0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.487+0000 7fdecc7f1740 1 -- start start 2024-03-19T01:24:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.487+0000 7fdecc7f1740 1 Processor -- start 2024-03-19T01:24:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.488+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8ca20 0x55ad93a92f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.488+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a8d340 0x55ad93a93470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.488+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a95f20 0x55ad93a939b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.488+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ad93a98d10 con 0x55ad93a8ca20 2024-03-19T01:24:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.488+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad939ad160 con 0x55ad93a95f20 2024-03-19T01:24:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.488+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ad93a98e90 con 0x55ad93a8d340 2024-03-19T01:24:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.488+0000 7fdecc7f1740 10 client.0 _ll_register_callbacks cb 0x55ad939e9170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.488+0000 7fdec3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a95f20 0x55ad93a939b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.488+0000 7fdec3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a95f20 0x55ad93a939b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:39422/0 (socket says 172.21.15.69:39422) 2024-03-19T01:24:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.488+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8ca20 0x55ad93a92f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.488+0000 7fdec3400640 1 -- 172.21.15.69:0/2481271679 learned_addr learned my addr 172.21.15.69:0/2481271679 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.489+0000 7fdebbe00640 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a8d340 0x55ad93a93470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.489+0000 7fdec3400640 1 -- 172.21.15.69:0/2481271679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a8d340 msgr2=0x55ad93a93470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.489+0000 7fdec3400640 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a8d340 0x55ad93a93470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.489+0000 7fdec3400640 1 -- 172.21.15.69:0/2481271679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8ca20 msgr2=0x55ad93a92f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.489+0000 7fdec3400640 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8ca20 0x55ad93a92f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.489+0000 7fdec3400640 1 -- 172.21.15.69:0/2481271679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad937cf7d0 con 0x55ad93a95f20 2024-03-19T01:24:55.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.490+0000 7fdec3400640 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a95f20 0x55ad93a939b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdebc010990 tx=0x7fdebc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f42e5e37cfa1b0cb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.490+0000 7fdeb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:24:55.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.490+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2481271679 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc01e790 con 0x55ad93a95f20 2024-03-19T01:24:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.491+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2481271679 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7fdebc01e930 con 0x55ad93a95f20 2024-03-19T01:24:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.491+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2481271679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb40d3910 con 0x55ad93a95f20 2024-03-19T01:24:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.491+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2481271679 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fdebc01eb30 con 0x55ad93a95f20 2024-03-19T01:24:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.491+0000 7fdecc7f1740 10 client.28115 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.491+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2481271679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ad93aa50a0 con 0x55ad93a95f20 2024-03-19T01:24:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.491+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2481271679 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb40d3910 con 0x55ad93a95f20 2024-03-19T01:24:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.492+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2481271679 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fdebc027430 con 0x55ad93a95f20 2024-03-19T01:24:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.492+0000 7fdecc7f1740 10 client.28115 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.492+0000 7fdecc7f1740 10 client.28115 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:24:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.492+0000 7fdecc7f1740 20 client.28115 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.492+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55ad93aa64b0 0x55ad93aa88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.492+0000 7fdecc7f1740 4 client.28115 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:24:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.492+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2481271679 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad939fe8c0 con 0x55ad93aa64b0 2024-03-19T01:24:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.492+0000 7fdec2a00640 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55ad93aa64b0 0x55ad93aa88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.493+0000 7fdec2a00640 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55ad93aa64b0 0x55ad93aa88e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34e028bc7da4f586 server_cookie=43329eee53f70652 in_seq=0 out_seq=0 2024-03-19T01:24:55.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.493+0000 7fdeb0c00640 10 client.28115 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:24:55.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.494+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2481271679 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ad939fe8c0 con 0x55ad93aa64b0 2024-03-19T01:24:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.494+0000 7fdeb0c00640 10 client.28115 handle_command_reply: tid=0 2024-03-19T01:24:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.494+0000 7fdecc7f1740 1 client.28115 shutdown 2024-03-19T01:24:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.495+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2481271679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a95f20 msgr2=0x55ad93a939b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.495+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a95f20 0x55ad93a939b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdebc010990 tx=0x7fdebc004a40 comp rx=0 tx=0).stop 2024-03-19T01:24:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.495+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2481271679 shutdown_connections 2024-03-19T01:24:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.495+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55ad93aa64b0 0x55ad93aa88e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.495+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a95f20 0x55ad93a939b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.495+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a8d340 0x55ad93a93470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.495+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2481271679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8ca20 0x55ad93a92f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.495+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2481271679 >> 172.21.15.69:0/2481271679 conn(0x55ad93a87150 msgr2=0x55ad93a88ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.495+0000 7fdeb0c00640 0 client.28115 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:24:55.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.495+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2481271679 shutdown_connections 2024-03-19T01:24:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.495+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2481271679 wait complete. 2024-03-19T01:24:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.496+0000 7fdecc7f1740 20 client.28115 trim_cache size 0 max 16384 2024-03-19T01:24:55.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:55 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3296179704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:24:55.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.525+0000 7fdecc7f1740 1 Processor -- start 2024-03-19T01:24:55.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.525+0000 7fdecc7f1740 1 -- start start 2024-03-19T01:24:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.525+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8ca20 0x55ad93a95660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.525+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8d340 0x55ad93a937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.526+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a86050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.526+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41b11a0 con 0x55ad93a8d340 2024-03-19T01:24:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.526+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41b2ce0 con 0x55ad93a8ca20 2024-03-19T01:24:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.526+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41b1870 con 0x55ad93a95f20 2024-03-19T01:24:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.526+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8ca20 0x55ad93a95660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.526+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8ca20 0x55ad93a95660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:39426/0 (socket says 172.21.15.69:39426) 2024-03-19T01:24:55.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.526+0000 7fdec2a00640 1 -- 172.21.15.69:0/3096209232 learned_addr learned my addr 172.21.15.69:0/3096209232 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:55.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.526+0000 7fdec3400640 1 --2- 172.21.15.69:0/3096209232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a86050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.526+0000 7fdebbe00640 1 --2- 172.21.15.69:0/3096209232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8d340 0x55ad93a937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.526+0000 7fdec3400640 1 -- 172.21.15.69:0/3096209232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8ca20 msgr2=0x55ad93a95660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.526+0000 7fdec3400640 1 --2- 172.21.15.69:0/3096209232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8ca20 0x55ad93a95660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.527+0000 7fdec3400640 1 -- 172.21.15.69:0/3096209232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8d340 msgr2=0x55ad93a937d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.527+0000 7fdec3400640 1 --2- 172.21.15.69:0/3096209232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8d340 0x55ad93a937d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.527+0000 7fdec3400640 1 -- 172.21.15.69:0/3096209232 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad93a98310 con 0x55ad93a95f20 2024-03-19T01:24:55.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.527+0000 7fdebbe00640 1 --2- 172.21.15.69:0/3096209232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8d340 0x55ad93a937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:55.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.527+0000 7fdec3400640 1 --2- 172.21.15.69:0/3096209232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a86050 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdebc027880 tx=0x7fdebc009930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2562373190a2f274 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.527+0000 7fdeb3400640 1 -- 172.21.15.69:0/3096209232 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc027430 con 0x55ad93a95f20 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.527+0000 7fdeb3400640 1 -- 172.21.15.69:0/3096209232 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdebc0275d0 con 0x55ad93a95f20 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.527+0000 7fdeb3400640 1 -- 172.21.15.69:0/3096209232 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0052a0 con 0x55ad93a95f20 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.528+0000 7fdecc7f1740 1 -- 172.21.15.69:0/3096209232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 msgr2=0x55ad93a86050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.528+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/3096209232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a86050 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdebc027880 tx=0x7fdebc009930 comp rx=0 tx=0).stop 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.528+0000 7fdecc7f1740 1 -- 172.21.15.69:0/3096209232 shutdown_connections 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.528+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/3096209232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a86050 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.528+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/3096209232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8d340 0x55ad93a937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.528+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/3096209232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8ca20 0x55ad93a95660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.528+0000 7fdecc7f1740 1 -- 172.21.15.69:0/3096209232 >> 172.21.15.69:0/3096209232 conn(0x55ad93a87150 msgr2=0x55ad93a85940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.529+0000 7fdecc7f1740 1 -- 172.21.15.69:0/3096209232 shutdown_connections 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.529+0000 7fdecc7f1740 1 -- 172.21.15.69:0/3096209232 wait complete. 2024-03-19T01:24:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.529+0000 7fdecc7f1740 1 -- start start 2024-03-19T01:24:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.529+0000 7fdecc7f1740 1 Processor -- start 2024-03-19T01:24:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.530+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8ca20 0x55ad93a8fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.530+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8d340 0x55ad93a902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.530+0000 7fdecc7f1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.530+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41b2610 con 0x55ad93a8d340 2024-03-19T01:24:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.530+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41b1f40 con 0x55ad93a8ca20 2024-03-19T01:24:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.530+0000 7fdecc7f1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41af180 con 0x55ad93a95f20 2024-03-19T01:24:55.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.530+0000 7fdecc7f1740 10 client.0 _ll_register_callbacks cb 0x7fdebc009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:55.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:55 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3296179704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.531+0000 7fdec3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.531+0000 7fdec3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a907e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:51216/0 (socket says 172.21.15.69:51216) 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.531+0000 7fdec3400640 1 -- 172.21.15.69:0/2183857381 learned_addr learned my addr 172.21.15.69:0/2183857381 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.531+0000 7fdebbe00640 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8d340 0x55ad93a902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.532+0000 7fdec2a00640 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8ca20 0x55ad93a8fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.532+0000 7fdec3400640 1 -- 172.21.15.69:0/2183857381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8ca20 msgr2=0x55ad93a8fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.532+0000 7fdec3400640 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8ca20 0x55ad93a8fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.532+0000 7fdec3400640 1 -- 172.21.15.69:0/2183857381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8d340 msgr2=0x55ad93a902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.532+0000 7fdec3400640 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8d340 0x55ad93a902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.532+0000 7fdec3400640 1 -- 172.21.15.69:0/2183857381 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad93b14290 con 0x55ad93a95f20 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.532+0000 7fdec3400640 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a907e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdebc027920 tx=0x7fdebc0274b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bf0ec511c92c4d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.532+0000 7fdeb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.532+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2183857381 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0049c0 con 0x55ad93a95f20 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.533+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2183857381 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7fdebc004b60 con 0x55ad93a95f20 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.533+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2183857381 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdebc0275d0 con 0x55ad93a95f20 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.533+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2183857381 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fdebc04d050 con 0x55ad93a95f20 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.533+0000 7fdecc7f1740 10 client.28039 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.533+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2183857381 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ad93b15590 con 0x55ad93a95f20 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.533+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2183857381 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdebc0275d0 con 0x55ad93a95f20 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.533+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2183857381 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fdebc005040 con 0x55ad93a95f20 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.534+0000 7fdecc7f1740 10 client.28039 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.534+0000 7fdecc7f1740 10 client.28039 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.534+0000 7fdecc7f1740 20 client.28039 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:55.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.534+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55ad93b15be0 0x55ad93b18000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.534+0000 7fdecc7f1740 4 client.28039 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:24:55.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.534+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2183857381 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad939f8c10 con 0x55ad93b15be0 2024-03-19T01:24:55.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.534+0000 7fdec2a00640 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55ad93b15be0 0x55ad93b18000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.535+0000 7fdec2a00640 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55ad93b15be0 0x55ad93b18000 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=156545fd021709bb server_cookie=89085dbce46527bd in_seq=0 out_seq=0 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.535+0000 7fdeb0c00640 10 client.28039 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.535+0000 7fdeb0c00640 1 -- 172.21.15.69:0/2183857381 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad939f8c10 con 0x55ad93b15be0 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.535+0000 7fdeb0c00640 10 client.28039 handle_command_reply: tid=0 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.535+0000 7fdecc7f1740 1 client.28039 shutdown 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.535+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2183857381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 msgr2=0x55ad93a907e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.535+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a907e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdebc027920 tx=0x7fdebc0274b0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.536+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2183857381 shutdown_connections 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.536+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55ad93b15be0 0x55ad93b18000 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.536+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad93a95f20 0x55ad93a907e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.536+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad93a8d340 0x55ad93a902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.536+0000 7fdecc7f1740 1 --2- 172.21.15.69:0/2183857381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad93a8ca20 0x55ad93a8fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.536+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2183857381 >> 172.21.15.69:0/2183857381 conn(0x55ad93a87150 msgr2=0x55ad93a85940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.536+0000 7fdeb0c00640 0 client.28039 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:24:55.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.536+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2183857381 shutdown_connections 2024-03-19T01:24:55.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.536+0000 7fdecc7f1740 1 -- 172.21.15.69:0/2183857381 wait complete. 2024-03-19T01:24:55.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.537+0000 7fdecc7f1740 20 client.28039 trim_cache size 0 max 16384 2024-03-19T01:24:55.539 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:24:55.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.537+0000 7fdec9a00640 1 -- 172.21.15.69:0/3456517377 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdea80a2a70 msgr2=0x7fdea80a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.537+0000 7fdec9a00640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdea80a2a70 0x7fdea80a4f30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005fd0 tx=0x7fdebc0096a0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.537+0000 7fdec9a00640 1 -- 172.21.15.69:0/3456517377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec4073bd0 msgr2=0x7fdec419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.537+0000 7fdec9a00640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec4073bd0 0x7fdec419a2a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400d950 tx=0x7fdeb400de10 comp rx=0 tx=0).stop 2024-03-19T01:24:55.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.538+0000 7fdec9a00640 1 -- 172.21.15.69:0/3456517377 shutdown_connections 2024-03-19T01:24:55.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.538+0000 7fdec9a00640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdea80a2a70 0x7fdea80a4f30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.538+0000 7fdec9a00640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec410e070 0x7fdec419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.538+0000 7fdec9a00640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdec40744f0 0x7fdec419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.539+0000 7fdec9a00640 1 --2- 172.21.15.69:0/3456517377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec4073bd0 0x7fdec419a2a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.539+0000 7fdec9a00640 1 -- 172.21.15.69:0/3456517377 >> 172.21.15.69:0/3456517377 conn(0x7fdec4078c20 msgr2=0x7fdec40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.541+0000 7fdec9a00640 1 -- 172.21.15.69:0/3456517377 shutdown_connections 2024-03-19T01:24:55.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.541+0000 7fdec9a00640 1 -- 172.21.15.69:0/3456517377 wait complete. 2024-03-19T01:24:55.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:55 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3296179704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:24:55.566 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:24:55.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.796+0000 7f8805800640 1 -- 172.21.15.69:0/3544981321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880010be30 msgr2=0x7f880010e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.796+0000 7f8805800640 1 --2- 172.21.15.69:0/3544981321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880010be30 0x7f880010e260 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009ca0 tx=0x7f87f802c190 comp rx=0 tx=0).stop 2024-03-19T01:24:55.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.797+0000 7f8805800640 1 -- 172.21.15.69:0/3544981321 shutdown_connections 2024-03-19T01:24:55.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.797+0000 7f8805800640 1 --2- 172.21.15.69:0/3544981321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880010be30 0x7f880010e260 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.797+0000 7f8805800640 1 --2- 172.21.15.69:0/3544981321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8800103380 0x7f880010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.797+0000 7f8805800640 1 --2- 172.21.15.69:0/3544981321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8800102a60 0x7f8800102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.797+0000 7f8805800640 1 -- 172.21.15.69:0/3544981321 >> 172.21.15.69:0/3544981321 conn(0x7f8800076aa0 msgr2=0x7f8800076eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.797+0000 7f8805800640 1 -- 172.21.15.69:0/3544981321 shutdown_connections 2024-03-19T01:24:55.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.797+0000 7f8805800640 1 -- 172.21.15.69:0/3544981321 wait complete. 2024-03-19T01:24:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.798+0000 7f8805800640 1 Processor -- start 2024-03-19T01:24:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.798+0000 7f8805800640 1 -- start start 2024-03-19T01:24:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f8805800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8800102a60 0x7f880019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f8805800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8800103380 0x7f880019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f8805800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880010be30 0x7f880019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f8805800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8800111bc0 con 0x7f8800103380 2024-03-19T01:24:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f8805800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8800111a40 con 0x7f880010be30 2024-03-19T01:24:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f8805800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8800111d40 con 0x7f8800102a60 2024-03-19T01:24:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f87fea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8800102a60 0x7f880019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f87fea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8800102a60 0x7f880019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:51232/0 (socket says 172.21.15.69:51232) 2024-03-19T01:24:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f87fea00640 1 -- 172.21.15.69:0/3642019694 learned_addr learned my addr 172.21.15.69:0/3642019694 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f87ff400640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880010be30 0x7f880019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f87fe000640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8800103380 0x7f880019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.799+0000 7f87fea00640 1 -- 172.21.15.69:0/3642019694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880010be30 msgr2=0x7f880019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f87fea00640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880010be30 0x7f880019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f87fea00640 1 -- 172.21.15.69:0/3642019694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8800103380 msgr2=0x7f880019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f87fea00640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8800103380 0x7f880019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f87fea00640 1 -- 172.21.15.69:0/3642019694 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880019f230 con 0x7f8800102a60 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f87fe000640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8800103380 0x7f880019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f87ff400640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880010be30 0x7f880019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f87fea00640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8800102a60 0x7f880019a240 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e9f0 tx=0x7f87f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f87f7400640 1 -- 172.21.15.69:0/3642019694 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f000cd80 con 0x7f8800102a60 2024-03-19T01:24:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f87f7400640 1 -- 172.21.15.69:0/3642019694 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f0010910 con 0x7f8800102a60 2024-03-19T01:24:55.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f8805800640 1 -- 172.21.15.69:0/3642019694 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880019f520 con 0x7f8800102a60 2024-03-19T01:24:55.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f87f7400640 1 -- 172.21.15.69:0/3642019694 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0018990 con 0x7f8800102a60 2024-03-19T01:24:55.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.800+0000 7f8805800640 1 -- 172.21.15.69:0/3642019694 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88001a6db0 con 0x7f8800102a60 2024-03-19T01:24:55.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.803+0000 7f880859d740 1 Processor -- start 2024-03-19T01:24:55.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f880859d740 1 -- start start 2024-03-19T01:24:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f880859d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 0x5561f3bf9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f880859d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 0x5561f3c029e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f880859d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3c02f20 0x5561f3c05310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f880859d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5561f3b5e380 con 0x5561f3bf9a20 2024-03-19T01:24:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f880859d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561f3b06d20 con 0x5561f3bfa340 2024-03-19T01:24:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f880859d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5561f3b5a420 con 0x5561f3c02f20 2024-03-19T01:24:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f87f7400640 1 -- 172.21.15.69:0/3642019694 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f87f0010ac0 con 0x7f8800102a60 2024-03-19T01:24:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f87ff400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 0x5561f3bf9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f87fe000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3c02f20 0x5561f3c05310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f87ff400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 0x5561f3bf9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38792/0 (socket says 172.21.15.69:38792) 2024-03-19T01:24:55.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.804+0000 7f87ff400640 1 -- 172.21.15.69:0/3888854833 learned_addr learned my addr 172.21.15.69:0/3888854833 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.805+0000 7f87fea00640 1 --2- 172.21.15.69:0/3888854833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 0x5561f3c029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.805+0000 7f87ff400640 1 -- 172.21.15.69:0/3888854833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3c02f20 msgr2=0x5561f3c05310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.805+0000 7f87ff400640 1 --2- 172.21.15.69:0/3888854833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3c02f20 0x5561f3c05310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.805+0000 7f87ff400640 1 -- 172.21.15.69:0/3888854833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 msgr2=0x5561f3c029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.805+0000 7f87ff400640 1 --2- 172.21.15.69:0/3888854833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 0x5561f3c029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.805+0000 7f87ff400640 1 -- 172.21.15.69:0/3888854833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f38723f0 con 0x5561f3bf9a20 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.805+0000 7f87fea00640 1 --2- 172.21.15.69:0/3888854833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 0x5561f3c029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.805+0000 7f87ff400640 1 --2- 172.21.15.69:0/3888854833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 0x5561f3bf9e00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f880006aed0 tx=0x7f87f8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662402f9f3f07b48 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.806+0000 7f87ebe00640 1 -- 172.21.15.69:0/3888854833 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8043070 con 0x5561f3bf9a20 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.806+0000 7f87ebe00640 1 -- 172.21.15.69:0/3888854833 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f8003e50 con 0x5561f3bf9a20 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.806+0000 7f87ebe00640 1 -- 172.21.15.69:0/3888854833 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8039040 con 0x5561f3bf9a20 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.806+0000 7f87f7400640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f87e40a29f0 0x7f87e40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.806+0000 7f87f7400640 1 -- 172.21.15.69:0/3642019694 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f87f00cf5c0 con 0x7f8800102a60 2024-03-19T01:24:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.807+0000 7f87fea00640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f87e40a29f0 0x7f87e40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.807+0000 7f880859d740 1 -- 172.21.15.69:0/3888854833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 msgr2=0x5561f3bf9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.807+0000 7f880859d740 1 --2- 172.21.15.69:0/3888854833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 0x5561f3bf9e00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f880006aed0 tx=0x7f87f8009900 comp rx=0 tx=0).stop 2024-03-19T01:24:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.807+0000 7f880859d740 1 -- 172.21.15.69:0/3888854833 shutdown_connections 2024-03-19T01:24:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.807+0000 7f880859d740 1 --2- 172.21.15.69:0/3888854833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3c02f20 0x5561f3c05310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.807+0000 7f880859d740 1 --2- 172.21.15.69:0/3888854833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 0x5561f3c029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.807+0000 7f880859d740 1 --2- 172.21.15.69:0/3888854833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 0x5561f3bf9e00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.807+0000 7f880859d740 1 -- 172.21.15.69:0/3888854833 >> 172.21.15.69:0/3888854833 conn(0x5561f3bf4150 msgr2=0x5561f3bf4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.807+0000 7f880859d740 1 -- 172.21.15.69:0/3888854833 shutdown_connections 2024-03-19T01:24:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.807+0000 7f880859d740 1 -- 172.21.15.69:0/3888854833 wait complete. 2024-03-19T01:24:55.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.807+0000 7f87fea00640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f87e40a29f0 0x7f87e40a4eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x5561f3bf94f0 tx=0x7f87f0000bc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.808+0000 7f880859d740 1 -- start start 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.808+0000 7f880859d740 1 Processor -- start 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.808+0000 7f880859d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bf9a20 0x5561f3bfffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.808+0000 7f880859d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bfa340 0x5561f3c00500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.808+0000 7f880859d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3c02f20 0x5561f3c00a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.808+0000 7f880859d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5561f3c05cb0 con 0x5561f3bfa340 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.808+0000 7f880859d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561f3b1a160 con 0x5561f3c02f20 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.808+0000 7f880859d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5561f3c05e30 con 0x5561f3bf9a20 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.808+0000 7f880859d740 10 client.0 _ll_register_callbacks cb 0x5561f3b56170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.809+0000 7f87fe000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bfa340 0x5561f3c00500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.809+0000 7f87ff400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3c02f20 0x5561f3c00a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.809+0000 7f87fe000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bfa340 0x5561f3c00500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38806/0 (socket says 172.21.15.69:38806) 2024-03-19T01:24:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.809+0000 7f87fe000640 1 -- 172.21.15.69:0/3703544509 learned_addr learned my addr 172.21.15.69:0/3703544509 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.809+0000 7f87ff400640 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bf9a20 0x5561f3bfffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.809+0000 7f87fe000640 1 -- 172.21.15.69:0/3703544509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bf9a20 msgr2=0x5561f3bfffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.809+0000 7f87fe000640 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bf9a20 0x5561f3bfffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.809+0000 7f87fe000640 1 -- 172.21.15.69:0/3703544509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3c02f20 msgr2=0x5561f3c00a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.809+0000 7f87fe000640 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3c02f20 0x5561f3c00a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.809+0000 7f87fe000640 1 -- 172.21.15.69:0/3703544509 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f393c7d0 con 0x5561f3bfa340 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.809+0000 7f87ff400640 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bf9a20 0x5561f3bfffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.810+0000 7f87fe000640 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bfa340 0x5561f3c00500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0126b0 tx=0x7f87ec012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec89342c958ab41 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.810+0000 7f87e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.810+0000 7f87e8c00640 1 -- 172.21.15.69:0/3703544509 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec004260 con 0x5561f3bfa340 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.810+0000 7f87e8c00640 1 -- 172.21.15.69:0/3703544509 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f87ec004400 con 0x5561f3bfa340 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.810+0000 7f880859d740 1 -- 172.21.15.69:0/3703544509 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f8003e50 con 0x5561f3bfa340 2024-03-19T01:24:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.810+0000 7f87e8c00640 1 -- 172.21.15.69:0/3703544509 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f87ec0119a0 con 0x5561f3bfa340 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.810+0000 7f880859d740 10 client.17724 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.810+0000 7f880859d740 1 -- 172.21.15.69:0/3703544509 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5561f3c12010 con 0x5561f3bfa340 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.811+0000 7f87e8c00640 1 -- 172.21.15.69:0/3703544509 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f8003e50 con 0x5561f3bfa340 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.811+0000 7f87e8c00640 1 -- 172.21.15.69:0/3703544509 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f87ec00f440 con 0x5561f3bfa340 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.811+0000 7f880859d740 10 client.17724 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.811+0000 7f880859d740 10 client.17724 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.811+0000 7f880859d740 20 client.17724 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.811+0000 7f880859d740 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5561f3c13520 0x5561f3c15960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.811+0000 7f880859d740 4 client.17724 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.811+0000 7f880859d740 1 -- 172.21.15.69:0/3703544509 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561f3b6b8c0 con 0x5561f3c13520 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.811+0000 7f87ff400640 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5561f3c13520 0x5561f3c15960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.812+0000 7f87e8c00640 10 client.17724 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:24:55.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.812+0000 7f87ff400640 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5561f3c13520 0x5561f3c15960 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e00b4d688be491a8 server_cookie=e8cc8cd2f7346eec in_seq=0 out_seq=0 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f87e8c00640 1 -- 172.21.15.69:0/3703544509 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5561f3b6b8c0 con 0x5561f3c13520 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f87e8c00640 10 client.17724 handle_command_reply: tid=0 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f880859d740 1 client.17724 shutdown 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f880859d740 1 -- 172.21.15.69:0/3703544509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bfa340 msgr2=0x5561f3c00500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f880859d740 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bfa340 0x5561f3c00500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0126b0 tx=0x7f87ec012b70 comp rx=0 tx=0).stop 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f880859d740 1 -- 172.21.15.69:0/3703544509 shutdown_connections 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f880859d740 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5561f3c13520 0x5561f3c15960 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f880859d740 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3c02f20 0x5561f3c00a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f880859d740 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bfa340 0x5561f3c00500 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f880859d740 1 --2- 172.21.15.69:0/3703544509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bf9a20 0x5561f3bfffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f880859d740 1 -- 172.21.15.69:0/3703544509 >> 172.21.15.69:0/3703544509 conn(0x5561f3bf4150 msgr2=0x5561f3bf5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f87e8c00640 0 client.17724 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:24:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f880859d740 1 -- 172.21.15.69:0/3703544509 shutdown_connections 2024-03-19T01:24:55.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.813+0000 7f880859d740 1 -- 172.21.15.69:0/3703544509 wait complete. 2024-03-19T01:24:55.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.814+0000 7f880859d740 20 client.17724 trim_cache size 0 max 16384 2024-03-19T01:24:55.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.841+0000 7f880859d740 1 Processor -- start 2024-03-19T01:24:55.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.842+0000 7f880859d740 1 -- start start 2024-03-19T01:24:55.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.842+0000 7f880859d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 0x5561f3c026f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.842+0000 7f880859d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bfa340 0x5561f3c0ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.842+0000 7f880859d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3c02f20 0x5561f3c1b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.842+0000 7f880859d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88001b1040 con 0x5561f3bf9a20 2024-03-19T01:24:55.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.842+0000 7f880859d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88001b2b80 con 0x5561f3c02f20 2024-03-19T01:24:55.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.842+0000 7f880859d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f88001b1710 con 0x5561f3bfa340 2024-03-19T01:24:55.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.842+0000 7f87ff400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 0x5561f3c026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.842+0000 7f87fe000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bfa340 0x5561f3c0ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.842+0000 7f87fe000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bfa340 0x5561f3c0ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:51258/0 (socket says 172.21.15.69:51258) 2024-03-19T01:24:55.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.842+0000 7f87fe000640 1 -- 172.21.15.69:0/1497556060 learned_addr learned my addr 172.21.15.69:0/1497556060 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:55.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.843+0000 7f87ff400640 1 --2- 172.21.15.69:0/1497556060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3c02f20 0x5561f3c1b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.843+0000 7f87fe000640 1 -- 172.21.15.69:0/1497556060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3c02f20 msgr2=0x5561f3c1b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.843+0000 7f87fe000640 1 --2- 172.21.15.69:0/1497556060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3c02f20 0x5561f3c1b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.843+0000 7f87fe000640 1 -- 172.21.15.69:0/1497556060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 msgr2=0x5561f3c026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.843+0000 7f87fe000640 1 --2- 172.21.15.69:0/1497556060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 0x5561f3c026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.843+0000 7f87fe000640 1 -- 172.21.15.69:0/1497556060 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f3befa20 con 0x5561f3bfa340 2024-03-19T01:24:55.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.843+0000 7f87ff400640 1 --2- 172.21.15.69:0/1497556060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 0x5561f3c026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:55.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.844+0000 7f87fe000640 1 --2- 172.21.15.69:0/1497556060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bfa340 0x5561f3c0ea50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f87ec011910 tx=0x7f87ec009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7236c41f8a328d01 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.844+0000 7f87eb400640 1 -- 172.21.15.69:0/1497556060 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec02e020 con 0x5561f3bfa340 2024-03-19T01:24:55.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.844+0000 7f87eb400640 1 -- 172.21.15.69:0/1497556060 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87ec00f440 con 0x5561f3bfa340 2024-03-19T01:24:55.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.844+0000 7f87eb400640 1 -- 172.21.15.69:0/1497556060 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0035f0 con 0x5561f3bfa340 2024-03-19T01:24:55.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.845+0000 7f880859d740 1 -- 172.21.15.69:0/1497556060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bfa340 msgr2=0x5561f3c0ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.845+0000 7f880859d740 1 --2- 172.21.15.69:0/1497556060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bfa340 0x5561f3c0ea50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f87ec011910 tx=0x7f87ec009770 comp rx=0 tx=0).stop 2024-03-19T01:24:55.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.845+0000 7f880859d740 1 -- 172.21.15.69:0/1497556060 shutdown_connections 2024-03-19T01:24:55.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.845+0000 7f880859d740 1 --2- 172.21.15.69:0/1497556060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3c02f20 0x5561f3c1b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.845+0000 7f880859d740 1 --2- 172.21.15.69:0/1497556060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bfa340 0x5561f3c0ea50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.845+0000 7f880859d740 1 --2- 172.21.15.69:0/1497556060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3bf9a20 0x5561f3c026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.845+0000 7f880859d740 1 -- 172.21.15.69:0/1497556060 >> 172.21.15.69:0/1497556060 conn(0x5561f3bf4150 msgr2=0x5561f3bf2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.845+0000 7f880859d740 1 -- 172.21.15.69:0/1497556060 shutdown_connections 2024-03-19T01:24:55.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.845+0000 7f880859d740 1 -- 172.21.15.69:0/1497556060 wait complete. 2024-03-19T01:24:55.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.846+0000 7f880859d740 1 -- start start 2024-03-19T01:24:55.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.846+0000 7f880859d740 1 Processor -- start 2024-03-19T01:24:55.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.846+0000 7f880859d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bf9a20 0x5561f3c06b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.846+0000 7f880859d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 0x5561f3c07060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f880859d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3c02f20 0x5561f3c075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f880859d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88001b24b0 con 0x5561f3c02f20 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f880859d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88001b1de0 con 0x5561f3bfa340 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f880859d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f88001af020 con 0x5561f3bf9a20 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f880859d740 10 client.0 _ll_register_callbacks cb 0x7f87ec0097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f87fe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 0x5561f3c07060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f87ff400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bf9a20 0x5561f3c06b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f87fe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 0x5561f3c07060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:47856/0 (socket says 172.21.15.69:47856) 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f87fe000640 1 -- 172.21.15.69:0/1477023328 learned_addr learned my addr 172.21.15.69:0/1477023328 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f87ff400640 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3c02f20 0x5561f3c075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f87fe000640 1 -- 172.21.15.69:0/1477023328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bf9a20 msgr2=0x5561f3c06b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f87fe000640 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bf9a20 0x5561f3c06b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f87fe000640 1 -- 172.21.15.69:0/1477023328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3c02f20 msgr2=0x5561f3c075a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:55.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f87fe000640 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3c02f20 0x5561f3c075a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.847+0000 7f87fe000640 1 -- 172.21.15.69:0/1477023328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f3c81280 con 0x5561f3bfa340 2024-03-19T01:24:55.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.848+0000 7f87fe000640 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 0x5561f3c07060 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00f980 tx=0x7f87ec00fda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab2ef66383aae00b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:55.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.848+0000 7f87e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:24:55.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.848+0000 7f87e8c00640 1 -- 172.21.15.69:0/1477023328 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0052e0 con 0x5561f3bfa340 2024-03-19T01:24:55.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.848+0000 7f87e8c00640 1 -- 172.21.15.69:0/1477023328 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f87ec005480 con 0x5561f3bfa340 2024-03-19T01:24:55.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.848+0000 7f880859d740 1 -- 172.21.15.69:0/1477023328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ec00f440 con 0x5561f3bfa340 2024-03-19T01:24:55.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.848+0000 7f87e8c00640 1 -- 172.21.15.69:0/1477023328 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f87ec044050 con 0x5561f3bfa340 2024-03-19T01:24:55.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.848+0000 7f880859d740 10 client.28136 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:55.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.848+0000 7f880859d740 1 -- 172.21.15.69:0/1477023328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5561f3c82540 con 0x5561f3bfa340 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.849+0000 7f87e8c00640 1 -- 172.21.15.69:0/1477023328 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87ec00f440 con 0x5561f3bfa340 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.849+0000 7f87e8c00640 1 -- 172.21.15.69:0/1477023328 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f87ec02a4c0 con 0x5561f3bfa340 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.849+0000 7f880859d740 10 client.28136 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.849+0000 7f880859d740 10 client.28136 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.849+0000 7f880859d740 20 client.28136 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.849+0000 7f880859d740 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5561f3c82bd0 0x5561f3c84ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.849+0000 7f880859d740 4 client.28136 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.849+0000 7f880859d740 1 -- 172.21.15.69:0/1477023328 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561f3b65c10 con 0x5561f3c82bd0 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.849+0000 7f87ff400640 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5561f3c82bd0 0x5561f3c84ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f87ff400640 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5561f3c82bd0 0x5561f3c84ff0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab628acdf7864689 server_cookie=92afd1f25f44299b in_seq=0 out_seq=0 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f87e8c00640 10 client.28136 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f87e8c00640 1 -- 172.21.15.69:0/1477023328 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561f3b65c10 con 0x5561f3c82bd0 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f87e8c00640 10 client.28136 handle_command_reply: tid=0 2024-03-19T01:24:55.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f880859d740 1 client.28136 shutdown 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f880859d740 1 -- 172.21.15.69:0/1477023328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 msgr2=0x5561f3c07060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f880859d740 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 0x5561f3c07060 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00f980 tx=0x7f87ec00fda0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f880859d740 1 -- 172.21.15.69:0/1477023328 shutdown_connections 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f880859d740 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f3c02f20 0x5561f3c075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f880859d740 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x5561f3c82bd0 0x5561f3c84ff0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f880859d740 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561f3bfa340 0x5561f3c07060 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f87e8c00640 0 client.28136 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f880859d740 1 --2- 172.21.15.69:0/1477023328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5561f3bf9a20 0x5561f3c06b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f880859d740 1 -- 172.21.15.69:0/1477023328 >> 172.21.15.69:0/1477023328 conn(0x5561f3bf4150 msgr2=0x5561f3c00810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f880859d740 1 -- 172.21.15.69:0/1477023328 shutdown_connections 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.850+0000 7f880859d740 1 -- 172.21.15.69:0/1477023328 wait complete. 2024-03-19T01:24:55.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.851+0000 7f880859d740 20 client.28136 trim_cache size 0 max 16384 2024-03-19T01:24:55.855 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:24:55.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.851+0000 7f8805800640 1 -- 172.21.15.69:0/3642019694 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f87e40a29f0 msgr2=0x7f87e40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.851+0000 7f8805800640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f87e40a29f0 0x7f87e40a4eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x5561f3bf94f0 tx=0x7f87f0000bc0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.851+0000 7f8805800640 1 -- 172.21.15.69:0/3642019694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8800102a60 msgr2=0x7f880019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:55.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.851+0000 7f8805800640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8800102a60 0x7f880019a240 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e9f0 tx=0x7f87f000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.852+0000 7f8805800640 1 -- 172.21.15.69:0/3642019694 shutdown_connections 2024-03-19T01:24:55.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.852+0000 7f8805800640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f87e40a29f0 0x7f87e40a4eb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.852+0000 7f8805800640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880010be30 0x7f880019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.852+0000 7f8805800640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8800103380 0x7f880019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.852+0000 7f8805800640 1 --2- 172.21.15.69:0/3642019694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8800102a60 0x7f880019a240 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:55.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.852+0000 7f8805800640 1 -- 172.21.15.69:0/3642019694 >> 172.21.15.69:0/3642019694 conn(0x7f8800076aa0 msgr2=0x7f88000ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:55.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.852+0000 7f8805800640 1 -- 172.21.15.69:0/3642019694 shutdown_connections 2024-03-19T01:24:55.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:55.852+0000 7f8805800640 1 -- 172.21.15.69:0/3642019694 wait complete. 2024-03-19T01:24:55.874 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:24:56.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.092+0000 7fa93b400640 1 -- 172.21.15.69:0/2915676205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9341039c0 msgr2=0x7fa934103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.092+0000 7fa93b400640 1 --2- 172.21.15.69:0/2915676205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9341039c0 0x7fa934103e40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa920009900 tx=0x7fa92002bc30 comp rx=0 tx=0).stop 2024-03-19T01:24:56.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.092+0000 7fa93b400640 1 -- 172.21.15.69:0/2915676205 shutdown_connections 2024-03-19T01:24:56.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.092+0000 7fa93b400640 1 --2- 172.21.15.69:0/2915676205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa934104380 0x7fa934110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.092+0000 7fa93b400640 1 --2- 172.21.15.69:0/2915676205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9341039c0 0x7fa934103e40 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.092+0000 7fa93b400640 1 --2- 172.21.15.69:0/2915676205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9341099d0 0x7fa934109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.092+0000 7fa93b400640 1 -- 172.21.15.69:0/2915676205 >> 172.21.15.69:0/2915676205 conn(0x7fa9340ff490 msgr2=0x7fa9341018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.093+0000 7fa93b400640 1 -- 172.21.15.69:0/2915676205 shutdown_connections 2024-03-19T01:24:56.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.093+0000 7fa93b400640 1 -- 172.21.15.69:0/2915676205 wait complete. 2024-03-19T01:24:56.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.093+0000 7fa93b400640 1 Processor -- start 2024-03-19T01:24:56.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa93b400640 1 -- start start 2024-03-19T01:24:56.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa93b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9341039c0 0x7fa93419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa93b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa934104380 0x7fa93419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa93b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9341099d0 0x7fa93419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa93b400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa934113e30 con 0x7fa9341039c0 2024-03-19T01:24:56.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa93b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa934113cb0 con 0x7fa9341099d0 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa93b400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa934113fb0 con 0x7fa934104380 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa938a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9341039c0 0x7fa93419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa933e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa934104380 0x7fa93419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa938a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9341039c0 0x7fa93419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44372/0 (socket says 172.21.15.69:44372) 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa938a00640 1 -- 172.21.15.69:0/586253302 learned_addr learned my addr 172.21.15.69:0/586253302 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.094+0000 7fa939400640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9341099d0 0x7fa93419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa938a00640 1 -- 172.21.15.69:0/586253302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa934104380 msgr2=0x7fa93419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa938a00640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa934104380 0x7fa93419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa938a00640 1 -- 172.21.15.69:0/586253302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9341099d0 msgr2=0x7fa93419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa938a00640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9341099d0 0x7fa93419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa938a00640 1 -- 172.21.15.69:0/586253302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93419f340 con 0x7fa9341039c0 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa939400640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9341099d0 0x7fa93419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa933e00640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa934104380 0x7fa93419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:56.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa938a00640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9341039c0 0x7fa93419a350 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa92800e9f0 tx=0x7fa92800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa931600640 1 -- 172.21.15.69:0/586253302 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92800cd80 con 0x7fa9341039c0 2024-03-19T01:24:56.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa931600640 1 -- 172.21.15.69:0/586253302 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa928005500 con 0x7fa9341039c0 2024-03-19T01:24:56.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa93b400640 1 -- 172.21.15.69:0/586253302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93419f630 con 0x7fa9341039c0 2024-03-19T01:24:56.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.095+0000 7fa931600640 1 -- 172.21.15.69:0/586253302 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928010690 con 0x7fa9341039c0 2024-03-19T01:24:56.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.096+0000 7fa93b400640 1 -- 172.21.15.69:0/586253302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9341a6ec0 con 0x7fa9341039c0 2024-03-19T01:24:56.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.099+0000 7fa93e03e740 1 Processor -- start 2024-03-19T01:24:56.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.099+0000 7fa93e03e740 1 -- start start 2024-03-19T01:24:56.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.100+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78bfa20 0x555ea78bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.100+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78c0340 0x555ea78c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.100+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c8f20 0x555ea78cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.100+0000 7fa93e03e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555ea7824380 con 0x555ea78bfa20 2024-03-19T01:24:56.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.100+0000 7fa93e03e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ea77ccd20 con 0x555ea78c0340 2024-03-19T01:24:56.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.100+0000 7fa93e03e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555ea7820420 con 0x555ea78c8f20 2024-03-19T01:24:56.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.100+0000 7fa931600640 1 -- 172.21.15.69:0/586253302 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fa9280046c0 con 0x7fa9341039c0 2024-03-19T01:24:56.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.100+0000 7fa939400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78bfa20 0x555ea78bfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.100+0000 7fa933e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c8f20 0x555ea78cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.100+0000 7fa939400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78bfa20 0x555ea78bfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44382/0 (socket says 172.21.15.69:44382) 2024-03-19T01:24:56.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.100+0000 7fa939400640 1 -- 172.21.15.69:0/3747009265 learned_addr learned my addr 172.21.15.69:0/3747009265 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.101+0000 7fa939400640 1 -- 172.21.15.69:0/3747009265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c8f20 msgr2=0x555ea78cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.101+0000 7fa939400640 1 --2- 172.21.15.69:0/3747009265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c8f20 0x555ea78cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.101+0000 7fa939400640 1 -- 172.21.15.69:0/3747009265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78c0340 msgr2=0x555ea78c89e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:24:56.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.101+0000 7fa939400640 1 --2- 172.21.15.69:0/3747009265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78c0340 0x555ea78c89e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.101+0000 7fa939400640 1 -- 172.21.15.69:0/3747009265 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea75383f0 con 0x555ea78bfa20 2024-03-19T01:24:56.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.101+0000 7fa933e00640 1 --2- 172.21.15.69:0/3747009265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c8f20 0x555ea78cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:56.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.101+0000 7fa939400640 1 --2- 172.21.15.69:0/3747009265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78bfa20 0x555ea78bfe00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0107e0 tx=0x7fa92c010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26809706ee74e7e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.101+0000 7fa925c00640 1 -- 172.21.15.69:0/3747009265 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c003b70 con 0x555ea78bfa20 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.102+0000 7fa925c00640 1 -- 172.21.15.69:0/3747009265 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92c00ee10 con 0x555ea78bfa20 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.102+0000 7fa925c00640 1 -- 172.21.15.69:0/3747009265 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c004770 con 0x555ea78bfa20 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.102+0000 7fa93e03e740 1 -- 172.21.15.69:0/3747009265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78bfa20 msgr2=0x555ea78bfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.102+0000 7fa93e03e740 1 --2- 172.21.15.69:0/3747009265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78bfa20 0x555ea78bfe00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0107e0 tx=0x7fa92c010ca0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.103+0000 7fa93e03e740 1 -- 172.21.15.69:0/3747009265 shutdown_connections 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.103+0000 7fa93e03e740 1 --2- 172.21.15.69:0/3747009265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c8f20 0x555ea78cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.103+0000 7fa93e03e740 1 --2- 172.21.15.69:0/3747009265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78c0340 0x555ea78c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.103+0000 7fa93e03e740 1 --2- 172.21.15.69:0/3747009265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78bfa20 0x555ea78bfe00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.103+0000 7fa93e03e740 1 -- 172.21.15.69:0/3747009265 >> 172.21.15.69:0/3747009265 conn(0x555ea78ba150 msgr2=0x555ea78ba530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.103+0000 7fa931600640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa91c0a2a70 0x7fa91c0a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.103+0000 7fa93e03e740 1 -- 172.21.15.69:0/3747009265 shutdown_connections 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.103+0000 7fa93e03e740 1 -- 172.21.15.69:0/3747009265 wait complete. 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.103+0000 7fa931600640 1 -- 172.21.15.69:0/586253302 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7fa9280cf590 con 0x7fa9341039c0 2024-03-19T01:24:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.103+0000 7fa939400640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa91c0a2a70 0x7fa91c0a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.104+0000 7fa93e03e740 1 -- start start 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.104+0000 7fa93e03e740 1 Processor -- start 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.104+0000 7fa939400640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa91c0a2a70 0x7fa91c0a4f30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0043d0 tx=0x7fa92c009920 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.104+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 0x555ea78c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.104+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c0340 0x555ea78c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c8f20 0x555ea78c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa93e03e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555ea78cbd10 con 0x555ea78c8f20 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa93e03e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ea77e0160 con 0x555ea78bfa20 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa93e03e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555ea78cbe90 con 0x555ea78c0340 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa938a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 0x555ea78c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa93e03e740 10 client.0 _ll_register_callbacks cb 0x555ea781c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa933e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c0340 0x555ea78c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa933e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c0340 0x555ea78c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46478/0 (socket says 172.21.15.69:46478) 2024-03-19T01:24:56.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa933e00640 1 -- 172.21.15.69:0/1463045561 learned_addr learned my addr 172.21.15.69:0/1463045561 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa939400640 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c8f20 0x555ea78c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa933e00640 1 -- 172.21.15.69:0/1463045561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 msgr2=0x555ea78c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa933e00640 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 0x555ea78c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa933e00640 1 -- 172.21.15.69:0/1463045561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c8f20 msgr2=0x555ea78c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa933e00640 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c8f20 0x555ea78c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa933e00640 1 -- 172.21.15.69:0/1463045561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea76027d0 con 0x555ea78c0340 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.105+0000 7fa938a00640 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 0x555ea78c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.106+0000 7fa933e00640 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c0340 0x555ea78c6540 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa920009900 tx=0x7fa920005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e08e97d7e0f79b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.106+0000 7fa91a000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.106+0000 7fa91a000640 1 -- 172.21.15.69:0/1463045561 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa920004340 con 0x555ea78c0340 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.106+0000 7fa93e03e740 1 -- 172.21.15.69:0/1463045561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa92c00ee10 con 0x555ea78c0340 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.106+0000 7fa91a000640 1 -- 172.21.15.69:0/1463045561 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7fa92002c8b0 con 0x555ea78c0340 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.106+0000 7fa91a000640 1 -- 172.21.15.69:0/1463045561 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa9200044e0 con 0x555ea78c0340 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.106+0000 7fa93e03e740 10 client.28069 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:56.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.106+0000 7fa93e03e740 1 -- 172.21.15.69:0/1463045561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x555ea78d8070 con 0x555ea78c0340 2024-03-19T01:24:56.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.106+0000 7fa91a000640 1 -- 172.21.15.69:0/1463045561 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92c00ee10 con 0x555ea78c0340 2024-03-19T01:24:56.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.107+0000 7fa91a000640 1 -- 172.21.15.69:0/1463045561 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fa9200037b0 con 0x555ea78c0340 2024-03-19T01:24:56.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.107+0000 7fa93e03e740 10 client.28069 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.107+0000 7fa93e03e740 10 client.28069 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.107+0000 7fa93e03e740 20 client.28069 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.107+0000 7fa93e03e740 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x555ea78d95d0 0x555ea78db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.107+0000 7fa93e03e740 4 client.28069 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.107+0000 7fa93e03e740 1 -- 172.21.15.69:0/1463045561 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ea78318c0 con 0x555ea78d95d0 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.107+0000 7fa938a00640 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x555ea78d95d0 0x555ea78db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.108+0000 7fa938a00640 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x555ea78d95d0 0x555ea78db9f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f26b8b0e9b42687d server_cookie=3bc29b545bbde486 in_seq=0 out_seq=0 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.108+0000 7fa91a000640 10 client.28069 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.108+0000 7fa91a000640 1 -- 172.21.15.69:0/1463045561 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555ea78318c0 con 0x555ea78d95d0 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.108+0000 7fa91a000640 10 client.28069 handle_command_reply: tid=0 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.108+0000 7fa93e03e740 1 client.28069 shutdown 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa93e03e740 1 -- 172.21.15.69:0/1463045561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c0340 msgr2=0x555ea78c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa93e03e740 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c0340 0x555ea78c6540 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa920009900 tx=0x7fa920005e70 comp rx=0 tx=0).stop 2024-03-19T01:24:56.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa93e03e740 1 -- 172.21.15.69:0/1463045561 shutdown_connections 2024-03-19T01:24:56.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa93e03e740 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x555ea78d95d0 0x555ea78db9f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa93e03e740 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c8f20 0x555ea78c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa93e03e740 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c0340 0x555ea78c6540 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa91a000640 0 client.28069 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:24:56.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa93e03e740 1 --2- 172.21.15.69:0/1463045561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 0x555ea78c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa93e03e740 1 -- 172.21.15.69:0/1463045561 >> 172.21.15.69:0/1463045561 conn(0x555ea78ba150 msgr2=0x555ea78bbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa93e03e740 1 -- 172.21.15.69:0/1463045561 shutdown_connections 2024-03-19T01:24:56.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa93e03e740 1 -- 172.21.15.69:0/1463045561 wait complete. 2024-03-19T01:24:56.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.109+0000 7fa93e03e740 20 client.28069 trim_cache size 0 max 16384 2024-03-19T01:24:56.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.137+0000 7fa93e03e740 1 Processor -- start 2024-03-19T01:24:56.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.137+0000 7fa93e03e740 1 -- start start 2024-03-19T01:24:56.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.137+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 0x555ea78d4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.137+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c0340 0x555ea78c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.137+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c8f20 0x555ea78e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.137+0000 7fa93e03e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341b1090 con 0x555ea78c0340 2024-03-19T01:24:56.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.137+0000 7fa93e03e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341b2bd0 con 0x555ea78bfa20 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.137+0000 7fa93e03e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341b1760 con 0x555ea78c8f20 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.137+0000 7fa938a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 0x555ea78d4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa938a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 0x555ea78d4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:47888/0 (socket says 172.21.15.69:47888) 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa938a00640 1 -- 172.21.15.69:0/2515429865 learned_addr learned my addr 172.21.15.69:0/2515429865 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa939400640 1 --2- 172.21.15.69:0/2515429865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c8f20 0x555ea78e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa933e00640 1 --2- 172.21.15.69:0/2515429865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c0340 0x555ea78c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa938a00640 1 -- 172.21.15.69:0/2515429865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c8f20 msgr2=0x555ea78e12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa938a00640 1 --2- 172.21.15.69:0/2515429865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c8f20 0x555ea78e12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa938a00640 1 -- 172.21.15.69:0/2515429865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c0340 msgr2=0x555ea78c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa938a00640 1 --2- 172.21.15.69:0/2515429865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c0340 0x555ea78c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa938a00640 1 -- 172.21.15.69:0/2515429865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea78d4ff0 con 0x555ea78bfa20 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa933e00640 1 --2- 172.21.15.69:0/2515429865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c0340 0x555ea78c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:56.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa938a00640 1 --2- 172.21.15.69:0/2515429865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 0x555ea78d4ab0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa928004c00 tx=0x7fa928086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d40dba975276e04c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa925200640 1 -- 172.21.15.69:0/2515429865 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9280b3020 con 0x555ea78bfa20 2024-03-19T01:24:56.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa925200640 1 -- 172.21.15.69:0/2515429865 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92809b7a0 con 0x555ea78bfa20 2024-03-19T01:24:56.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.138+0000 7fa925200640 1 -- 172.21.15.69:0/2515429865 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9280a3ab0 con 0x555ea78bfa20 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.139+0000 7fa93e03e740 1 -- 172.21.15.69:0/2515429865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 msgr2=0x555ea78d4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.139+0000 7fa93e03e740 1 --2- 172.21.15.69:0/2515429865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 0x555ea78d4ab0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa928004c00 tx=0x7fa928086040 comp rx=0 tx=0).stop 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.140+0000 7fa93e03e740 1 -- 172.21.15.69:0/2515429865 shutdown_connections 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.140+0000 7fa93e03e740 1 --2- 172.21.15.69:0/2515429865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78c8f20 0x555ea78e12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.140+0000 7fa93e03e740 1 --2- 172.21.15.69:0/2515429865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c0340 0x555ea78c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.140+0000 7fa93e03e740 1 --2- 172.21.15.69:0/2515429865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78bfa20 0x555ea78d4ab0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.140+0000 7fa93e03e740 1 -- 172.21.15.69:0/2515429865 >> 172.21.15.69:0/2515429865 conn(0x555ea78ba150 msgr2=0x555ea78b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.140+0000 7fa93e03e740 1 -- 172.21.15.69:0/2515429865 shutdown_connections 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.140+0000 7fa93e03e740 1 -- 172.21.15.69:0/2515429865 wait complete. 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.140+0000 7fa93e03e740 1 -- start start 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.140+0000 7fa93e03e740 1 Processor -- start 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78bfa20 0x555ea78c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c0340 0x555ea78c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa93e03e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78c8f20 0x555ea793e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa93e03e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341b2500 con 0x555ea78c0340 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa93e03e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341b1e30 con 0x555ea78c8f20 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa93e03e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341af070 con 0x555ea78bfa20 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa93e03e740 10 client.0 _ll_register_callbacks cb 0x7fa920009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa933e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c0340 0x555ea78c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa938a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78bfa20 0x555ea78c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa939400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78c8f20 0x555ea793e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa938a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78bfa20 0x555ea78c3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46490/0 (socket says 172.21.15.69:46490) 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa938a00640 1 -- 172.21.15.69:0/672370008 learned_addr learned my addr 172.21.15.69:0/672370008 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa938a00640 1 -- 172.21.15.69:0/672370008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78c8f20 msgr2=0x555ea793e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa938a00640 1 --2- 172.21.15.69:0/672370008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78c8f20 0x555ea793e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa938a00640 1 -- 172.21.15.69:0/672370008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c0340 msgr2=0x555ea78c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa938a00640 1 --2- 172.21.15.69:0/672370008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c0340 0x555ea78c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.141+0000 7fa938a00640 1 -- 172.21.15.69:0/672370008 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea793ee30 con 0x555ea78bfa20 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.142+0000 7fa938a00640 1 --2- 172.21.15.69:0/672370008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78bfa20 0x555ea78c3270 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa92809b4a0 tx=0x7fa92809bfe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b221d2414c610f58 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.142+0000 7fa91a000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.142+0000 7fa91a000640 1 -- 172.21.15.69:0/672370008 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9280b3040 con 0x555ea78bfa20 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.142+0000 7fa91a000640 1 -- 172.21.15.69:0/672370008 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7fa9280ccb10 con 0x555ea78bfa20 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.142+0000 7fa93e03e740 1 -- 172.21.15.69:0/672370008 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa92809b7a0 con 0x555ea78bfa20 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.142+0000 7fa91a000640 1 -- 172.21.15.69:0/672370008 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa9280ccd10 con 0x555ea78bfa20 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.142+0000 7fa93e03e740 10 client.28075 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.142+0000 7fa93e03e740 1 -- 172.21.15.69:0/672370008 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x555ea79400a0 con 0x555ea78bfa20 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.142+0000 7fa91a000640 1 -- 172.21.15.69:0/672370008 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92809b7a0 con 0x555ea78bfa20 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.142+0000 7fa91a000640 1 -- 172.21.15.69:0/672370008 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7fa928084ce0 con 0x555ea78bfa20 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.142+0000 7fa93e03e740 10 client.28075 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.143+0000 7fa93e03e740 10 client.28075 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.143+0000 7fa93e03e740 20 client.28075 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:56.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.143+0000 7fa93e03e740 1 --2- 172.21.15.69:0/672370008 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x555ea7940760 0x555ea7942b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.143+0000 7fa93e03e740 4 client.28075 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.143+0000 7fa93e03e740 1 -- 172.21.15.69:0/672370008 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ea782bc10 con 0x555ea7940760 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.143+0000 7fa933e00640 1 --2- 172.21.15.69:0/672370008 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x555ea7940760 0x555ea7942b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.143+0000 7fa933e00640 1 --2- 172.21.15.69:0/672370008 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x555ea7940760 0x555ea7942b50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a5869597b77ef1b server_cookie=6f9952e3b7be1afa in_seq=0 out_seq=0 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.143+0000 7fa91a000640 10 client.28075 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa91a000640 1 -- 172.21.15.69:0/672370008 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ea782bc10 con 0x555ea7940760 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa91a000640 10 client.28075 handle_command_reply: tid=0 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 1 client.28075 shutdown 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 1 -- 172.21.15.69:0/672370008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78bfa20 msgr2=0x555ea78c3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 1 --2- 172.21.15.69:0/672370008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78bfa20 0x555ea78c3270 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa92809b4a0 tx=0x7fa92809bfe0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 1 -- 172.21.15.69:0/672370008 shutdown_connections 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 1 --2- 172.21.15.69:0/672370008 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x555ea7940760 0x555ea7942b50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 1 --2- 172.21.15.69:0/672370008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ea78c8f20 0x555ea793e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 1 --2- 172.21.15.69:0/672370008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ea78c0340 0x555ea78c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa91a000640 0 client.28075 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:24:56.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 1 --2- 172.21.15.69:0/672370008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555ea78bfa20 0x555ea78c3270 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 1 -- 172.21.15.69:0/672370008 >> 172.21.15.69:0/672370008 conn(0x555ea78ba150 msgr2=0x555ea78bb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 1 -- 172.21.15.69:0/672370008 shutdown_connections 2024-03-19T01:24:56.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 1 -- 172.21.15.69:0/672370008 wait complete. 2024-03-19T01:24:56.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.144+0000 7fa93e03e740 20 client.28075 trim_cache size 0 max 16384 2024-03-19T01:24:56.149 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:24:56.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.145+0000 7fa93b400640 1 -- 172.21.15.69:0/586253302 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa91c0a2a70 msgr2=0x7fa91c0a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.145+0000 7fa93b400640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa91c0a2a70 0x7fa91c0a4f30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0043d0 tx=0x7fa92c009920 comp rx=0 tx=0).stop 2024-03-19T01:24:56.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.145+0000 7fa93b400640 1 -- 172.21.15.69:0/586253302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9341039c0 msgr2=0x7fa93419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.145+0000 7fa93b400640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9341039c0 0x7fa93419a350 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa92800e9f0 tx=0x7fa92800edf0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.145+0000 7fa93b400640 1 -- 172.21.15.69:0/586253302 shutdown_connections 2024-03-19T01:24:56.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.145+0000 7fa93b400640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fa91c0a2a70 0x7fa91c0a4f30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.145+0000 7fa93b400640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9341099d0 0x7fa93419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.145+0000 7fa93b400640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa934104380 0x7fa93419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.145+0000 7fa93b400640 1 --2- 172.21.15.69:0/586253302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9341039c0 0x7fa93419a350 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.145+0000 7fa93b400640 1 -- 172.21.15.69:0/586253302 >> 172.21.15.69:0/586253302 conn(0x7fa9340ff490 msgr2=0x7fa9340ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.146+0000 7fa93b400640 1 -- 172.21.15.69:0/586253302 shutdown_connections 2024-03-19T01:24:56.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.146+0000 7fa93b400640 1 -- 172.21.15.69:0/586253302 wait complete. 2024-03-19T01:24:56.153 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/i915/built-in.a 2024-03-19T01:24:56.167 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:24:56.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.378+0000 7f75a4a00640 1 -- 172.21.15.69:0/729683122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a00744f0 msgr2=0x7f75a010db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.378+0000 7f75a4a00640 1 --2- 172.21.15.69:0/729683122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a00744f0 0x7f75a010db30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f758c009900 tx=0x7f758c02bcd0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.379+0000 7f75a4a00640 1 -- 172.21.15.69:0/729683122 shutdown_connections 2024-03-19T01:24:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.379+0000 7f75a4a00640 1 --2- 172.21.15.69:0/729683122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75a010e070 0x7f75a0110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.379+0000 7f75a4a00640 1 --2- 172.21.15.69:0/729683122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a00744f0 0x7f75a010db30 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.379+0000 7f75a4a00640 1 --2- 172.21.15.69:0/729683122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75a0073bd0 0x7f75a0073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.379+0000 7f75a4a00640 1 -- 172.21.15.69:0/729683122 >> 172.21.15.69:0/729683122 conn(0x7f75a0078c20 msgr2=0x7f75a0079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.379+0000 7f75a4a00640 1 -- 172.21.15.69:0/729683122 shutdown_connections 2024-03-19T01:24:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.379+0000 7f75a4a00640 1 -- 172.21.15.69:0/729683122 wait complete. 2024-03-19T01:24:56.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.380+0000 7f75a4a00640 1 Processor -- start 2024-03-19T01:24:56.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.380+0000 7f75a4a00640 1 -- start start 2024-03-19T01:24:56.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.380+0000 7f75a4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0073bd0 0x7f75a019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.380+0000 7f75a4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75a00744f0 0x7f75a019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.380+0000 7f75a4a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75a010e070 0x7f75a019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.380+0000 7f75a4a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0113db0 con 0x7f75a010e070 2024-03-19T01:24:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.380+0000 7f75a4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0113c30 con 0x7f75a0073bd0 2024-03-19T01:24:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.380+0000 7f75a4a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0113f30 con 0x7f75a00744f0 2024-03-19T01:24:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.380+0000 7f759de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0073bd0 0x7f75a019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f759de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0073bd0 0x7f75a019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:47910/0 (socket says 172.21.15.69:47910) 2024-03-19T01:24:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f759d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75a00744f0 0x7f75a019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f759de00640 1 -- 172.21.15.69:0/1548466981 learned_addr learned my addr 172.21.15.69:0/1548466981 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f759e800640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75a010e070 0x7f75a019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f759de00640 1 -- 172.21.15.69:0/1548466981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75a00744f0 msgr2=0x7f75a019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f759de00640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75a00744f0 0x7f75a019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f759de00640 1 -- 172.21.15.69:0/1548466981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75a010e070 msgr2=0x7f75a019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f759de00640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75a010e070 0x7f75a019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f759de00640 1 -- 172.21.15.69:0/1548466981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a019f2a0 con 0x7f75a0073bd0 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f759e800640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75a010e070 0x7f75a019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f759de00640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0073bd0 0x7f75a019a2b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f759000ea60 tx=0x7f759000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f7596a00640 1 -- 172.21.15.69:0/1548466981 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759000cde0 con 0x7f75a0073bd0 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.381+0000 7f7596a00640 1 -- 172.21.15.69:0/1548466981 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75900035f0 con 0x7f75a0073bd0 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.382+0000 7f75a4a00640 1 -- 172.21.15.69:0/1548466981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a019f590 con 0x7f75a0073bd0 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.382+0000 7f7596a00640 1 -- 172.21.15.69:0/1548466981 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590010670 con 0x7f75a0073bd0 2024-03-19T01:24:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.382+0000 7f75a4a00640 1 -- 172.21.15.69:0/1548466981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a01a6e20 con 0x7f75a0073bd0 2024-03-19T01:24:56.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.385+0000 7f75a77a0740 1 Processor -- start 2024-03-19T01:24:56.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f75a77a0740 1 -- start start 2024-03-19T01:24:56.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395370a20 0x55c395370e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395371340 0x55c3953799e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395379f20 0x55c39537c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f75a77a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c3952d5380 con 0x55c395370a20 2024-03-19T01:24:56.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f75a77a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c39527dd20 con 0x55c395371340 2024-03-19T01:24:56.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f75a77a0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c3952d1420 con 0x55c395379f20 2024-03-19T01:24:56.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f7596a00640 1 -- 172.21.15.69:0/1548466981 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f75900037a0 con 0x7f75a0073bd0 2024-03-19T01:24:56.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f759d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395379f20 0x55c39537c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f759e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395370a20 0x55c395370e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f759d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395379f20 0x55c39537c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46524/0 (socket says 172.21.15.69:46524) 2024-03-19T01:24:56.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.386+0000 7f759d400640 1 -- 172.21.15.69:0/1068430818 learned_addr learned my addr 172.21.15.69:0/1068430818 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.387+0000 7f759de00640 1 --2- 172.21.15.69:0/1068430818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395371340 0x55c3953799e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.387+0000 7f759d400640 1 -- 172.21.15.69:0/1068430818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395371340 msgr2=0x55c3953799e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.387+0000 7f759d400640 1 --2- 172.21.15.69:0/1068430818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395371340 0x55c3953799e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.387+0000 7f759d400640 1 -- 172.21.15.69:0/1068430818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395370a20 msgr2=0x55c395370e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.387+0000 7f759d400640 1 --2- 172.21.15.69:0/1068430818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395370a20 0x55c395370e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.387+0000 7f759d400640 1 -- 172.21.15.69:0/1068430818 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c394fe93f0 con 0x55c395379f20 2024-03-19T01:24:56.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.387+0000 7f759d400640 1 --2- 172.21.15.69:0/1068430818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395379f20 0x55c39537c310 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f758c000c00 tx=0x7f758c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf12730fac6d942 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.388+0000 7f758b000640 1 -- 172.21.15.69:0/1068430818 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c043070 con 0x55c395379f20 2024-03-19T01:24:56.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.388+0000 7f758b000640 1 -- 172.21.15.69:0/1068430818 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c02c6f0 con 0x55c395379f20 2024-03-19T01:24:56.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.388+0000 7f758b000640 1 -- 172.21.15.69:0/1068430818 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c004a60 con 0x55c395379f20 2024-03-19T01:24:56.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.388+0000 7f7596a00640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f75840a29f0 0x7f75840a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.388+0000 7f75a77a0740 1 -- 172.21.15.69:0/1068430818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395379f20 msgr2=0x55c39537c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.388+0000 7f75a77a0740 1 --2- 172.21.15.69:0/1068430818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395379f20 0x55c39537c310 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f758c000c00 tx=0x7f758c009560 comp rx=0 tx=0).stop 2024-03-19T01:24:56.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.389+0000 7f7596a00640 1 -- 172.21.15.69:0/1548466981 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f75900cf590 con 0x7f75a0073bd0 2024-03-19T01:24:56.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.389+0000 7f75a77a0740 1 -- 172.21.15.69:0/1068430818 shutdown_connections 2024-03-19T01:24:56.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.389+0000 7f75a77a0740 1 --2- 172.21.15.69:0/1068430818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395379f20 0x55c39537c310 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.389+0000 7f75a77a0740 1 --2- 172.21.15.69:0/1068430818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395371340 0x55c3953799e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.389+0000 7f75a77a0740 1 --2- 172.21.15.69:0/1068430818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395370a20 0x55c395370e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.389+0000 7f75a77a0740 1 -- 172.21.15.69:0/1068430818 >> 172.21.15.69:0/1068430818 conn(0x55c39536b150 msgr2=0x55c39536b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.389+0000 7f75a77a0740 1 -- 172.21.15.69:0/1068430818 shutdown_connections 2024-03-19T01:24:56.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.389+0000 7f759e800640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f75840a29f0 0x7f75840a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.389+0000 7f75a77a0740 1 -- 172.21.15.69:0/1068430818 wait complete. 2024-03-19T01:24:56.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.390+0000 7f75a77a0740 1 -- start start 2024-03-19T01:24:56.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.390+0000 7f759e800640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f75840a29f0 0x7f75840a4eb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7598005fd0 tx=0x7f7598009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.390+0000 7f75a77a0740 1 Processor -- start 2024-03-19T01:24:56.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395376f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c395377460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395379f20 0x55c3953779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f75a77a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c39537ccb0 con 0x55c395379f20 2024-03-19T01:24:56.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f75a77a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c395291160 con 0x55c395370a20 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f75a77a0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c39537ce30 con 0x55c395371340 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f75a77a0740 10 client.0 _ll_register_callbacks cb 0x55c3952cd170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f759de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395376f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f759de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395376f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:47932/0 (socket says 172.21.15.69:47932) 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f759de00640 1 -- 172.21.15.69:0/2504683985 learned_addr learned my addr 172.21.15.69:0/2504683985 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f759d400640 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c395377460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.391+0000 7f759e800640 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395379f20 0x55c3953779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f759de00640 1 -- 172.21.15.69:0/2504683985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 msgr2=0x55c395377460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f759de00640 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c395377460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f759de00640 1 -- 172.21.15.69:0/2504683985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395379f20 msgr2=0x55c3953779a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f759de00640 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395379f20 0x55c3953779a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f759de00640 1 -- 172.21.15.69:0/2504683985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3950b37d0 con 0x55c395370a20 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f759d400640 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c395377460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:56.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f759de00640 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395376f20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f759000ea30 tx=0x7f7590097a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ec675edb3ef8be server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f757f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:24:56.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f757f400640 1 -- 172.21.15.69:0/2504683985 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759009b140 con 0x55c395370a20 2024-03-19T01:24:56.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f757f400640 1 -- 172.21.15.69:0/2504683985 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f7590097e50 con 0x55c395370a20 2024-03-19T01:24:56.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f75a77a0740 1 -- 172.21.15.69:0/2504683985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f758c02c6f0 con 0x55c395370a20 2024-03-19T01:24:56.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.392+0000 7f757f400640 1 -- 172.21.15.69:0/2504683985 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f759009bee0 con 0x55c395370a20 2024-03-19T01:24:56.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.393+0000 7f75a77a0740 10 client.28163 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:56.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.393+0000 7f75a77a0740 1 -- 172.21.15.69:0/2504683985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55c395388fa0 con 0x55c395370a20 2024-03-19T01:24:56.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.393+0000 7f757f400640 1 -- 172.21.15.69:0/2504683985 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c02c6f0 con 0x55c395370a20 2024-03-19T01:24:56.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.393+0000 7f757f400640 1 -- 172.21.15.69:0/2504683985 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f75900ccb50 con 0x55c395370a20 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.393+0000 7f75a77a0740 10 client.28163 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.393+0000 7f75a77a0740 10 client.28163 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.393+0000 7f75a77a0740 20 client.28163 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.393+0000 7f75a77a0740 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c39538a460 0x55c39538c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.393+0000 7f75a77a0740 4 client.28163 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.393+0000 7f75a77a0740 1 -- 172.21.15.69:0/2504683985 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3952e28c0 con 0x55c39538a460 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.393+0000 7f759d400640 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c39538a460 0x55c39538c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.394+0000 7f757f400640 10 client.28163 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.394+0000 7f759d400640 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c39538a460 0x55c39538c850 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68a87e6c7967c476 server_cookie=4ca48acd5a989d1e in_seq=0 out_seq=0 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f757f400640 1 -- 172.21.15.69:0/2504683985 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c3952e28c0 con 0x55c39538a460 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f757f400640 10 client.28163 handle_command_reply: tid=0 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f75a77a0740 1 client.28163 shutdown 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f75a77a0740 1 -- 172.21.15.69:0/2504683985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 msgr2=0x55c395376f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f75a77a0740 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395376f20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f759000ea30 tx=0x7f7590097a80 comp rx=0 tx=0).stop 2024-03-19T01:24:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f75a77a0740 1 -- 172.21.15.69:0/2504683985 shutdown_connections 2024-03-19T01:24:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f75a77a0740 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395379f20 0x55c3953779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f75a77a0740 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c395377460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f75a77a0740 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c39538a460 0x55c39538c850 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f75a77a0740 1 --2- 172.21.15.69:0/2504683985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395376f20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f757f400640 0 client.28163 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:24:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f75a77a0740 1 -- 172.21.15.69:0/2504683985 >> 172.21.15.69:0/2504683985 conn(0x55c39536b150 msgr2=0x55c39536cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f75a77a0740 1 -- 172.21.15.69:0/2504683985 shutdown_connections 2024-03-19T01:24:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.395+0000 7f75a77a0740 1 -- 172.21.15.69:0/2504683985 wait complete. 2024-03-19T01:24:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.396+0000 7f75a77a0740 20 client.28163 trim_cache size 0 max 16384 2024-03-19T01:24:56.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.412+0000 7f75a77a0740 1 Processor -- start 2024-03-19T01:24:56.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.412+0000 7f75a77a0740 1 -- start start 2024-03-19T01:24:56.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.412+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395370a20 0x55c395385ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.412+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c395377790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.412+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395379f20 0x55c3953920b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.412+0000 7f75a77a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01b0f40 con 0x55c395370a20 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.412+0000 7f75a77a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01b2a80 con 0x55c395379f20 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.412+0000 7f75a77a0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01b1610 con 0x55c395371340 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c395377790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c395377790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46544/0 (socket says 172.21.15.69:46544) 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759d400640 1 -- 172.21.15.69:0/2466567374 learned_addr learned my addr 172.21.15.69:0/2466567374 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759e800640 1 --2- 172.21.15.69:0/2466567374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395379f20 0x55c3953920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759de00640 1 --2- 172.21.15.69:0/2466567374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395370a20 0x55c395385ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759d400640 1 -- 172.21.15.69:0/2466567374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395379f20 msgr2=0x55c3953920b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759d400640 1 --2- 172.21.15.69:0/2466567374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395379f20 0x55c3953920b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759d400640 1 -- 172.21.15.69:0/2466567374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395370a20 msgr2=0x55c395385ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759d400640 1 --2- 172.21.15.69:0/2466567374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395370a20 0x55c395385ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759d400640 1 -- 172.21.15.69:0/2466567374 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c39538ce80 con 0x55c395371340 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759de00640 1 --2- 172.21.15.69:0/2466567374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395370a20 0x55c395385ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:56.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.413+0000 7f759d400640 1 --2- 172.21.15.69:0/2466567374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c395377790 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55c39536c2e0 tx=0x7f758c02bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adf6d3ad97adb849 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.414+0000 7f758a600640 1 -- 172.21.15.69:0/2466567374 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c043040 con 0x55c395371340 2024-03-19T01:24:56.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.414+0000 7f758a600640 1 -- 172.21.15.69:0/2466567374 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c004040 con 0x55c395371340 2024-03-19T01:24:56.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.414+0000 7f758a600640 1 -- 172.21.15.69:0/2466567374 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c0054a0 con 0x55c395371340 2024-03-19T01:24:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.414+0000 7f75a77a0740 1 -- 172.21.15.69:0/2466567374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 msgr2=0x55c395377790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.414+0000 7f75a77a0740 1 --2- 172.21.15.69:0/2466567374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c395377790 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55c39536c2e0 tx=0x7f758c02bcd0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.415+0000 7f75a77a0740 1 -- 172.21.15.69:0/2466567374 shutdown_connections 2024-03-19T01:24:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.415+0000 7f75a77a0740 1 --2- 172.21.15.69:0/2466567374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395379f20 0x55c3953920b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.415+0000 7f75a77a0740 1 --2- 172.21.15.69:0/2466567374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c395377790 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.415+0000 7f75a77a0740 1 --2- 172.21.15.69:0/2466567374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395370a20 0x55c395385ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.415+0000 7f75a77a0740 1 -- 172.21.15.69:0/2466567374 >> 172.21.15.69:0/2466567374 conn(0x55c39536b150 msgr2=0x55c39536a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.415+0000 7f75a77a0740 1 -- 172.21.15.69:0/2466567374 shutdown_connections 2024-03-19T01:24:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.415+0000 7f75a77a0740 1 -- 172.21.15.69:0/2466567374 wait complete. 2024-03-19T01:24:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.415+0000 7f75a77a0740 1 -- start start 2024-03-19T01:24:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.415+0000 7f75a77a0740 1 Processor -- start 2024-03-19T01:24:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395382550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c39537d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f75a77a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395379f20 0x55c39537dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f75a77a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01b23b0 con 0x55c395379f20 2024-03-19T01:24:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f75a77a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01b1ce0 con 0x55c395370a20 2024-03-19T01:24:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f75a77a0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f75a01aef20 con 0x55c395371340 2024-03-19T01:24:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f75a77a0740 10 client.0 _ll_register_callbacks cb 0x7f7590003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f759de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395382550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f759de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395382550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:47956/0 (socket says 172.21.15.69:47956) 2024-03-19T01:24:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f759de00640 1 -- 172.21.15.69:0/172251946 learned_addr learned my addr 172.21.15.69:0/172251946 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f759e800640 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395379f20 0x55c39537dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f759de00640 1 -- 172.21.15.69:0/172251946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 msgr2=0x55c39537d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:24:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f759de00640 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c39537d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f759de00640 1 -- 172.21.15.69:0/172251946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395379f20 msgr2=0x55c39537dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f759de00640 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395379f20 0x55c39537dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f759de00640 1 -- 172.21.15.69:0/172251946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c39536af80 con 0x55c395370a20 2024-03-19T01:24:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.416+0000 7f759e800640 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395379f20 0x55c39537dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.417+0000 7f759de00640 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395382550 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f759001f000 tx=0x7f759009ac30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6efbd962750bf80f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.417+0000 7f757f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.417+0000 7f757f400640 1 -- 172.21.15.69:0/172251946 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759009b140 con 0x55c395370a20 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.417+0000 7f757f400640 1 -- 172.21.15.69:0/172251946 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f75900b6e10 con 0x55c395370a20 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.417+0000 7f75a77a0740 1 -- 172.21.15.69:0/172251946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f758c004040 con 0x55c395370a20 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.417+0000 7f757f400640 1 -- 172.21.15.69:0/172251946 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f759009b5b0 con 0x55c395370a20 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.417+0000 7f75a77a0740 10 client.28169 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.417+0000 7f75a77a0740 1 -- 172.21.15.69:0/172251946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55c3953e9670 con 0x55c395370a20 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.418+0000 7f757f400640 1 -- 172.21.15.69:0/172251946 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c004040 con 0x55c395370a20 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.418+0000 7f757f400640 1 -- 172.21.15.69:0/172251946 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f75900ccb50 con 0x55c395370a20 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.418+0000 7f75a77a0740 10 client.28169 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.418+0000 7f75a77a0740 10 client.28169 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:24:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.418+0000 7f75a77a0740 20 client.28169 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:56.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.418+0000 7f75a77a0740 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c3953e9d40 0x55c3953ec180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.418+0000 7f75a77a0740 4 client.28169 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:24:56.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.418+0000 7f75a77a0740 1 -- 172.21.15.69:0/172251946 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3952dcc10 con 0x55c3953e9d40 2024-03-19T01:24:56.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.420+0000 7f759d400640 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c3953e9d40 0x55c3953ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.420+0000 7f757f400640 10 client.28169 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:24:56.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.420+0000 7f759d400640 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c3953e9d40 0x55c3953ec180 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73c9f4055041191b server_cookie=6355807877767a78 in_seq=0 out_seq=0 2024-03-19T01:24:56.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f757f400640 1 -- 172.21.15.69:0/172251946 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3952dcc10 con 0x55c3953e9d40 2024-03-19T01:24:56.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f757f400640 10 client.28169 handle_command_reply: tid=0 2024-03-19T01:24:56.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 1 client.28169 shutdown 2024-03-19T01:24:56.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 1 -- 172.21.15.69:0/172251946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 msgr2=0x55c395382550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395382550 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f759001f000 tx=0x7f759009ac30 comp rx=0 tx=0).stop 2024-03-19T01:24:56.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 1 -- 172.21.15.69:0/172251946 shutdown_connections 2024-03-19T01:24:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c395379f20 0x55c39537dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c395371340 0x55c39537d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55c3953e9d40 0x55c3953ec180 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 1 --2- 172.21.15.69:0/172251946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395370a20 0x55c395382550 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f757f400640 0 client.28169 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:24:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 1 -- 172.21.15.69:0/172251946 >> 172.21.15.69:0/172251946 conn(0x55c39536b150 msgr2=0x55c39536c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 1 -- 172.21.15.69:0/172251946 shutdown_connections 2024-03-19T01:24:56.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 1 -- 172.21.15.69:0/172251946 wait complete. 2024-03-19T01:24:56.426 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:24:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.421+0000 7f75a77a0740 20 client.28169 trim_cache size 0 max 16384 2024-03-19T01:24:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 -- 172.21.15.69:0/1548466981 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f75840a29f0 msgr2=0x7f75840a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f75840a29f0 0x7f75840a4eb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7598005fd0 tx=0x7f7598009500 comp rx=0 tx=0).stop 2024-03-19T01:24:56.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 -- 172.21.15.69:0/1548466981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0073bd0 msgr2=0x7f75a019a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0073bd0 0x7f75a019a2b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f759000ea60 tx=0x7f759000ef20 comp rx=0 tx=0).stop 2024-03-19T01:24:56.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 -- 172.21.15.69:0/1548466981 shutdown_connections 2024-03-19T01:24:56.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75a010e070 0x7f75a019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75a00744f0 0x7f75a019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f75840a29f0 0x7f75840a4eb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 --2- 172.21.15.69:0/1548466981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a0073bd0 0x7f75a019a2b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 -- 172.21.15.69:0/1548466981 >> 172.21.15.69:0/1548466981 conn(0x7f75a0078c20 msgr2=0x7f75a00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 -- 172.21.15.69:0/1548466981 shutdown_connections 2024-03-19T01:24:56.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.423+0000 7f75a4a00640 1 -- 172.21.15.69:0/1548466981 wait complete. 2024-03-19T01:24:56.428 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/drm/built-in.a 2024-03-19T01:24:56.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:56 smithi069 ceph-mon[31727]: pgmap v446: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 836 KiB/s wr, 88 op/s 2024-03-19T01:24:56.445 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:24:56.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:56 smithi086 ceph-mon[66829]: pgmap v446: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 836 KiB/s wr, 88 op/s 2024-03-19T01:24:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:56 smithi163 ceph-mon[67428]: pgmap v446: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 836 KiB/s wr, 88 op/s 2024-03-19T01:24:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.661+0000 7f9eda400640 1 -- 172.21.15.69:0/455903732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed4073bf0 msgr2=0x7f9ed4073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.661+0000 7f9eda400640 1 --2- 172.21.15.69:0/455903732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed4073bf0 0x7f9ed4073fd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009900 tx=0x7f9ec802bcd0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.661+0000 7f9eda400640 1 -- 172.21.15.69:0/455903732 shutdown_connections 2024-03-19T01:24:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.661+0000 7f9eda400640 1 --2- 172.21.15.69:0/455903732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ed410df30 0x7f9ed4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.661+0000 7f9eda400640 1 --2- 172.21.15.69:0/455903732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed4074510 0x7f9ed410d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.661+0000 7f9eda400640 1 --2- 172.21.15.69:0/455903732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed4073bf0 0x7f9ed4073fd0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.661+0000 7f9eda400640 1 -- 172.21.15.69:0/455903732 >> 172.21.15.69:0/455903732 conn(0x7f9ed40fd1c0 msgr2=0x7f9ed40ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.661+0000 7f9eda400640 1 -- 172.21.15.69:0/455903732 shutdown_connections 2024-03-19T01:24:56.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.661+0000 7f9eda400640 1 -- 172.21.15.69:0/455903732 wait complete. 2024-03-19T01:24:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.662+0000 7f9eda400640 1 Processor -- start 2024-03-19T01:24:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.662+0000 7f9eda400640 1 -- start start 2024-03-19T01:24:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.663+0000 7f9eda400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed4073bf0 0x7f9ed419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.663+0000 7f9eda400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed4074510 0x7f9ed419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.663+0000 7f9eda400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ed410df30 0x7f9ed419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.663+0000 7f9eda400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed4113ce0 con 0x7f9ed4073bf0 2024-03-19T01:24:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.663+0000 7f9eda400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed4113b60 con 0x7f9ed4074510 2024-03-19T01:24:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.663+0000 7f9eda400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed4113e60 con 0x7f9ed410df30 2024-03-19T01:24:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.663+0000 7f9ed3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed4074510 0x7f9ed419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.663+0000 7f9ed9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed4073bf0 0x7f9ed419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.663+0000 7f9ed3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed4074510 0x7f9ed419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:47970/0 (socket says 172.21.15.69:47970) 2024-03-19T01:24:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.663+0000 7f9ed3e00640 1 -- 172.21.15.69:0/1041333928 learned_addr learned my addr 172.21.15.69:0/1041333928 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9ed3e00640 1 -- 172.21.15.69:0/1041333928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ed410df30 msgr2=0x7f9ed419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9ed9a00640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ed410df30 0x7f9ed419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ed410df30 0x7f9ed419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9ed3e00640 1 -- 172.21.15.69:0/1041333928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed4073bf0 msgr2=0x7f9ed419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed4073bf0 0x7f9ed419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9ed3e00640 1 -- 172.21.15.69:0/1041333928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed419f1b0 con 0x7f9ed4074510 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9ed9a00640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ed410df30 0x7f9ed419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed4074510 0x7f9ed419a700 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000e9f0 tx=0x7f9ec000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9ed1600640 1 -- 172.21.15.69:0/1041333928 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000cd80 con 0x7f9ed4074510 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9ed1600640 1 -- 172.21.15.69:0/1041333928 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec0010910 con 0x7f9ed4074510 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9eda400640 1 -- 172.21.15.69:0/1041333928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed419f4a0 con 0x7f9ed4074510 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9ed1600640 1 -- 172.21.15.69:0/1041333928 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0018990 con 0x7f9ed4074510 2024-03-19T01:24:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.664+0000 7f9eda400640 1 -- 172.21.15.69:0/1041333928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed41a6d30 con 0x7f9ed4074510 2024-03-19T01:24:56.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.668+0000 7f9ed1600640 1 -- 172.21.15.69:0/1041333928 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9ec0010ac0 con 0x7f9ed4074510 2024-03-19T01:24:56.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.669+0000 7f9ede7df740 1 Processor -- start 2024-03-19T01:24:56.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.669+0000 7f9ede7df740 1 -- start start 2024-03-19T01:24:56.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.669+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 0x55815fe8cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.669+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fe95bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.669+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe960f0 0x55815fe984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.669+0000 7f9ede7df740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55815fdf1380 con 0x55815fe960f0 2024-03-19T01:24:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.669+0000 7f9ede7df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55815fd99d20 con 0x55815fe8d510 2024-03-19T01:24:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.669+0000 7f9ede7df740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55815fded420 con 0x55815fe8cbf0 2024-03-19T01:24:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.669+0000 7f9ed9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fe95bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.669+0000 7f9ed9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fe95bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:47978/0 (socket says 172.21.15.69:47978) 2024-03-19T01:24:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.669+0000 7f9ed9000640 1 -- 172.21.15.69:0/3650422843 learned_addr learned my addr 172.21.15.69:0/3650422843 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.670+0000 7f9ed9a00640 1 --2- 172.21.15.69:0/3650422843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 0x55815fe8cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.670+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/3650422843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe960f0 0x55815fe984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.670+0000 7f9ed9000640 1 -- 172.21.15.69:0/3650422843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 msgr2=0x55815fe8cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.670+0000 7f9ed9000640 1 --2- 172.21.15.69:0/3650422843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 0x55815fe8cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.670+0000 7f9ed9000640 1 -- 172.21.15.69:0/3650422843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe960f0 msgr2=0x55815fe984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.670+0000 7f9ed9000640 1 --2- 172.21.15.69:0/3650422843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe960f0 0x55815fe984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.670+0000 7f9ed9000640 1 -- 172.21.15.69:0/3650422843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55815fb053f0 con 0x55815fe8d510 2024-03-19T01:24:56.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.670+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/3650422843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe960f0 0x55815fe984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:24:56.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.670+0000 7f9ed1600640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9ebc0a29f0 0x7f9ebc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.670+0000 7f9ed9000640 1 --2- 172.21.15.69:0/3650422843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fe95bb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=279292d571825e1b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.671+0000 7f9ed1600640 1 -- 172.21.15.69:0/1041333928 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f9ec00cf6d0 con 0x7f9ed4074510 2024-03-19T01:24:56.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.671+0000 7f9ec6000640 1 -- 172.21.15.69:0/3650422843 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8043070 con 0x55815fe8d510 2024-03-19T01:24:56.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.671+0000 7f9ec6000640 1 -- 172.21.15.69:0/3650422843 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec802c6f0 con 0x55815fe8d510 2024-03-19T01:24:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.671+0000 7f9ed9a00640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9ebc0a29f0 0x7f9ebc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.671+0000 7f9ec6000640 1 -- 172.21.15.69:0/3650422843 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8004a60 con 0x55815fe8d510 2024-03-19T01:24:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.671+0000 7f9ede7df740 1 -- 172.21.15.69:0/3650422843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 msgr2=0x55815fe95bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.671+0000 7f9ede7df740 1 --2- 172.21.15.69:0/3650422843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fe95bb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8009560 comp rx=0 tx=0).stop 2024-03-19T01:24:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.672+0000 7f9ed9a00640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9ebc0a29f0 0x7f9ebc0a4eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4069550 tx=0x7f9ecc009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.672+0000 7f9ede7df740 1 -- 172.21.15.69:0/3650422843 shutdown_connections 2024-03-19T01:24:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.672+0000 7f9ede7df740 1 --2- 172.21.15.69:0/3650422843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe960f0 0x55815fe984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.672+0000 7f9ede7df740 1 --2- 172.21.15.69:0/3650422843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fe95bb0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.672+0000 7f9ede7df740 1 --2- 172.21.15.69:0/3650422843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 0x55815fe8cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.672+0000 7f9ede7df740 1 -- 172.21.15.69:0/3650422843 >> 172.21.15.69:0/3650422843 conn(0x55815fe87320 msgr2=0x55815fe87700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.672+0000 7f9ede7df740 1 -- 172.21.15.69:0/3650422843 shutdown_connections 2024-03-19T01:24:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.673+0000 7f9ede7df740 1 -- 172.21.15.69:0/3650422843 wait complete. 2024-03-19T01:24:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.673+0000 7f9ede7df740 1 -- start start 2024-03-19T01:24:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.673+0000 7f9ede7df740 1 Processor -- start 2024-03-19T01:24:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe8cbf0 0x55815fe930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8d510 0x55815fe93610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe960f0 0x55815fe93b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ede7df740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55815fe98e80 con 0x55815fe8cbf0 2024-03-19T01:24:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ede7df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55815fdad160 con 0x55815fe960f0 2024-03-19T01:24:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ede7df740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55815fe99000 con 0x55815fe8d510 2024-03-19T01:24:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ed9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe960f0 0x55815fe93b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ede7df740 10 client.0 _ll_register_callbacks cb 0x55815fde9170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:56.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ed9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe8cbf0 0x55815fe930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ed3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8d510 0x55815fe93610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ed3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8d510 0x55815fe93610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46606/0 (socket says 172.21.15.69:46606) 2024-03-19T01:24:56.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ed9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe960f0 0x55815fe93b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:47990/0 (socket says 172.21.15.69:47990) 2024-03-19T01:24:56.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.674+0000 7f9ed3e00640 1 -- 172.21.15.69:0/2981312079 learned_addr learned my addr 172.21.15.69:0/2981312079 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.675+0000 7f9ed3e00640 1 -- 172.21.15.69:0/2981312079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe960f0 msgr2=0x55815fe93b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.675+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/2981312079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe960f0 0x55815fe93b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.675+0000 7f9ed3e00640 1 -- 172.21.15.69:0/2981312079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe8cbf0 msgr2=0x55815fe930d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.675+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/2981312079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe8cbf0 0x55815fe930d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.675+0000 7f9ed3e00640 1 -- 172.21.15.69:0/2981312079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55815fbcf7d0 con 0x55815fe8d510 2024-03-19T01:24:56.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.675+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/2981312079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8d510 0x55815fe93610 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000e9c0 tx=0x7f9ec009c120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1166cbf122139a2a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.675+0000 7f9ebaa00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:24:56.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.675+0000 7f9ebaa00640 1 -- 172.21.15.69:0/2981312079 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00cce50 con 0x55815fe8d510 2024-03-19T01:24:56.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.675+0000 7f9ebaa00640 1 -- 172.21.15.69:0/2981312079 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f9ec00cf220 con 0x55815fe8d510 2024-03-19T01:24:56.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.675+0000 7f9ede7df740 1 -- 172.21.15.69:0/2981312079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec802c6f0 con 0x55815fe8d510 2024-03-19T01:24:56.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.676+0000 7f9ebaa00640 1 -- 172.21.15.69:0/2981312079 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9ec00cf420 con 0x55815fe8d510 2024-03-19T01:24:56.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.676+0000 7f9ede7df740 10 client.28099 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:56.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.676+0000 7f9ede7df740 1 -- 172.21.15.69:0/2981312079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55815fea5150 con 0x55815fe8d510 2024-03-19T01:24:56.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.676+0000 7f9ebaa00640 1 -- 172.21.15.69:0/2981312079 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec802c6f0 con 0x55815fe8d510 2024-03-19T01:24:56.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.676+0000 7f9ebaa00640 1 -- 172.21.15.69:0/2981312079 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f9ec009ae50 con 0x55815fe8d510 2024-03-19T01:24:56.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.676+0000 7f9ede7df740 10 client.28099 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:56.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.676+0000 7f9ede7df740 10 client.28099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:24:56.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.676+0000 7f9ede7df740 20 client.28099 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:56.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.676+0000 7f9ede7df740 1 --2- 172.21.15.69:0/2981312079 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55815fea6560 0x55815fea8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.676+0000 7f9ede7df740 4 client.28099 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:24:56.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.676+0000 7f9ede7df740 1 -- 172.21.15.69:0/2981312079 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55815fdfe8c0 con 0x55815fea6560 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.677+0000 7f9ed9000640 1 --2- 172.21.15.69:0/2981312079 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55815fea6560 0x55815fea8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.677+0000 7f9ed9000640 1 --2- 172.21.15.69:0/2981312079 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55815fea6560 0x55815fea8980 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3e8f34035db0b0 server_cookie=d7952c12a6afe41e in_seq=0 out_seq=0 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.677+0000 7f9ebaa00640 10 client.28099 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ebaa00640 1 -- 172.21.15.69:0/2981312079 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55815fdfe8c0 con 0x55815fea6560 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ebaa00640 10 client.28099 handle_command_reply: tid=0 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ede7df740 1 client.28099 shutdown 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ede7df740 1 -- 172.21.15.69:0/2981312079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8d510 msgr2=0x55815fe93610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ede7df740 1 --2- 172.21.15.69:0/2981312079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8d510 0x55815fe93610 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000e9c0 tx=0x7f9ec009c120 comp rx=0 tx=0).stop 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ede7df740 1 -- 172.21.15.69:0/2981312079 shutdown_connections 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ede7df740 1 --2- 172.21.15.69:0/2981312079 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55815fea6560 0x55815fea8980 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ede7df740 1 --2- 172.21.15.69:0/2981312079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe960f0 0x55815fe93b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ede7df740 1 --2- 172.21.15.69:0/2981312079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8d510 0x55815fe93610 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ede7df740 1 --2- 172.21.15.69:0/2981312079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe8cbf0 0x55815fe930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ede7df740 1 -- 172.21.15.69:0/2981312079 >> 172.21.15.69:0/2981312079 conn(0x55815fe87320 msgr2=0x55815fe89070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.678+0000 7f9ebaa00640 0 client.28099 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:56.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.679+0000 7f9ede7df740 1 -- 172.21.15.69:0/2981312079 shutdown_connections 2024-03-19T01:24:56.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.679+0000 7f9ede7df740 1 -- 172.21.15.69:0/2981312079 wait complete. 2024-03-19T01:24:56.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.679+0000 7f9ede7df740 20 client.28099 trim_cache size 0 max 16384 2024-03-19T01:24:56.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.706+0000 7f9ede7df740 1 Processor -- start 2024-03-19T01:24:56.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.706+0000 7f9ede7df740 1 -- start start 2024-03-19T01:24:56.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.706+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 0x55815fe95800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.706+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fea1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.706+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe960f0 0x55815feae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.706+0000 7f9ede7df740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed41b0e20 con 0x55815fe960f0 2024-03-19T01:24:56.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.706+0000 7f9ede7df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed41b2960 con 0x55815fe8d510 2024-03-19T01:24:56.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.706+0000 7f9ede7df740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed41b14f0 con 0x55815fe8cbf0 2024-03-19T01:24:56.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed9000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 0x55815fe95800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed9000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 0x55815fe95800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46614/0 (socket says 172.21.15.69:46614) 2024-03-19T01:24:56.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed9000640 1 -- 172.21.15.69:0/3661633159 learned_addr learned my addr 172.21.15.69:0/3661633159 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fea1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed9a00640 1 --2- 172.21.15.69:0/3661633159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe960f0 0x55815feae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed9000640 1 -- 172.21.15.69:0/3661633159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 msgr2=0x55815fea1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed9000640 1 --2- 172.21.15.69:0/3661633159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fea1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed9000640 1 -- 172.21.15.69:0/3661633159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe960f0 msgr2=0x55815feae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed9000640 1 --2- 172.21.15.69:0/3661633159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe960f0 0x55815feae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed9000640 1 -- 172.21.15.69:0/3661633159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55815fe984e0 con 0x55815fe8cbf0 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/3661633159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fea1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ed9000640 1 --2- 172.21.15.69:0/3661633159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 0x55815fe95800 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec802bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f890392fa7005f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ec5600640 1 -- 172.21.15.69:0/3661633159 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8043040 con 0x55815fe8cbf0 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ec5600640 1 -- 172.21.15.69:0/3661633159 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec8004c10 con 0x55815fe8cbf0 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.707+0000 7f9ec5600640 1 -- 172.21.15.69:0/3661633159 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8048450 con 0x55815fe8cbf0 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.708+0000 7f9ede7df740 1 -- 172.21.15.69:0/3661633159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 msgr2=0x55815fe95800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.708+0000 7f9ede7df740 1 --2- 172.21.15.69:0/3661633159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 0x55815fe95800 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec802bcd0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.708+0000 7f9ede7df740 1 -- 172.21.15.69:0/3661633159 shutdown_connections 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.708+0000 7f9ede7df740 1 --2- 172.21.15.69:0/3661633159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe960f0 0x55815feae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.708+0000 7f9ede7df740 1 --2- 172.21.15.69:0/3661633159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fea1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.708+0000 7f9ede7df740 1 --2- 172.21.15.69:0/3661633159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe8cbf0 0x55815fe95800 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.708+0000 7f9ede7df740 1 -- 172.21.15.69:0/3661633159 >> 172.21.15.69:0/3661633159 conn(0x55815fe87320 msgr2=0x55815fe85aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.708+0000 7f9ede7df740 1 -- 172.21.15.69:0/3661633159 shutdown_connections 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.709+0000 7f9ede7df740 1 -- 172.21.15.69:0/3661633159 wait complete. 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.709+0000 7f9ede7df740 1 -- start start 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.709+0000 7f9ede7df740 1 Processor -- start 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe8cbf0 0x55815fe8ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fe904e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ede7df740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe960f0 0x55815fe90a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ede7df740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed41b2290 con 0x55815fe8cbf0 2024-03-19T01:24:56.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ede7df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed41b1bc0 con 0x55815fe8d510 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ede7df740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed41aee00 con 0x55815fe960f0 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ede7df740 10 client.0 _ll_register_callbacks cb 0x7f9ec0097b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ed3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fe904e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ed9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe8cbf0 0x55815fe8ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ed9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe8cbf0 0x55815fe8ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44538/0 (socket says 172.21.15.69:44538) 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ed9000640 1 -- 172.21.15.69:0/1287193890 learned_addr learned my addr 172.21.15.69:0/1287193890 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ed9a00640 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe960f0 0x55815fe90a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ed3e00640 1 -- 172.21.15.69:0/1287193890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe960f0 msgr2=0x55815fe90a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe960f0 0x55815fe90a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ed3e00640 1 -- 172.21.15.69:0/1287193890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe8cbf0 msgr2=0x55815fe8ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe8cbf0 0x55815fe8ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.710+0000 7f9ed3e00640 1 -- 172.21.15.69:0/1287193890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55815ff0bb70 con 0x55815fe8d510 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.711+0000 7f9ed9a00640 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe960f0 0x55815fe90a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:56.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.711+0000 7f9ed3e00640 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fe904e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9ec009b5e0 tx=0x7f9ec000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650de7dd53acccc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.711+0000 7f9ebaa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.711+0000 7f9ebaa00640 1 -- 172.21.15.69:0/1287193890 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec009b240 con 0x55815fe8d510 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.711+0000 7f9ede7df740 1 -- 172.21.15.69:0/1287193890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec8004c10 con 0x55815fe8d510 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.711+0000 7f9ebaa00640 1 -- 172.21.15.69:0/1287193890 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f9ec0097950 con 0x55815fe8d510 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.711+0000 7f9ebaa00640 1 -- 172.21.15.69:0/1287193890 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f9ec00ac050 con 0x55815fe8d510 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.711+0000 7f9ede7df740 10 client.28187 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.711+0000 7f9ede7df740 1 -- 172.21.15.69:0/1287193890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55815ff0ce70 con 0x55815fe8d510 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.711+0000 7f9ebaa00640 1 -- 172.21.15.69:0/1287193890 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec8004c10 con 0x55815fe8d510 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.712+0000 7f9ebaa00640 1 -- 172.21.15.69:0/1287193890 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f9ec009a7a0 con 0x55815fe8d510 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.712+0000 7f9ede7df740 10 client.28187 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.712+0000 7f9ede7df740 10 client.28187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.712+0000 7f9ede7df740 20 client.28187 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.712+0000 7f9ede7df740 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55815ff0d4b0 0x55815ff0f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.712+0000 7f9ede7df740 4 client.28187 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:24:56.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.712+0000 7f9ede7df740 1 -- 172.21.15.69:0/1287193890 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55815fdf8c10 con 0x55815ff0d4b0 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.712+0000 7f9ed9000640 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55815ff0d4b0 0x55815ff0f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.712+0000 7f9ed9000640 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55815ff0d4b0 0x55815ff0f8d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6378336a430fc8ea server_cookie=9fb887c598d4009d in_seq=0 out_seq=0 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.712+0000 7f9ebaa00640 10 client.28187 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.712+0000 7f9ebaa00640 1 -- 172.21.15.69:0/1287193890 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55815fdf8c10 con 0x55815ff0d4b0 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ebaa00640 10 client.28187 handle_command_reply: tid=0 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 1 client.28187 shutdown 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 1 -- 172.21.15.69:0/1287193890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 msgr2=0x55815fe904e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fe904e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9ec009b5e0 tx=0x7f9ec000cf90 comp rx=0 tx=0).stop 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 1 -- 172.21.15.69:0/1287193890 shutdown_connections 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55815ff0d4b0 0x55815ff0f8d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55815fe960f0 0x55815fe90a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ebaa00640 0 client.28187 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55815fe8d510 0x55815fe904e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 1 --2- 172.21.15.69:0/1287193890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55815fe8cbf0 0x55815fe8ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 1 -- 172.21.15.69:0/1287193890 >> 172.21.15.69:0/1287193890 conn(0x55815fe87320 msgr2=0x55815fe93900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 1 -- 172.21.15.69:0/1287193890 shutdown_connections 2024-03-19T01:24:56.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 1 -- 172.21.15.69:0/1287193890 wait complete. 2024-03-19T01:24:56.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.713+0000 7f9ede7df740 20 client.28187 trim_cache size 0 max 16384 2024-03-19T01:24:56.719 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 -- 172.21.15.69:0/1041333928 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9ebc0a29f0 msgr2=0x7f9ebc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9ebc0a29f0 0x7f9ebc0a4eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4069550 tx=0x7f9ecc009500 comp rx=0 tx=0).stop 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 -- 172.21.15.69:0/1041333928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed4074510 msgr2=0x7f9ed419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed4074510 0x7f9ed419a700 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000e9f0 tx=0x7f9ec000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 -- 172.21.15.69:0/1041333928 shutdown_connections 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ed410df30 0x7f9ed419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9ebc0a29f0 0x7f9ebc0a4eb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ed4074510 0x7f9ed419a700 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 --2- 172.21.15.69:0/1041333928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ed4073bf0 0x7f9ed419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 -- 172.21.15.69:0/1041333928 >> 172.21.15.69:0/1041333928 conn(0x7f9ed40fd1c0 msgr2=0x7f9ed40fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 -- 172.21.15.69:0/1041333928 shutdown_connections 2024-03-19T01:24:56.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:56.714+0000 7f9eda400640 1 -- 172.21.15.69:0/1041333928 wait complete. 2024-03-19T01:24:56.762 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/gpu/built-in.a 2024-03-19T01:24:56.934 INFO:tasks.workunit.client.0.smithi069.stdout: AR drivers/built-in.a 2024-03-19T01:24:57.551 INFO:tasks.workunit.client.0.smithi069.stderr:make[1]: *** [/home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/Makefile:2034: .] Error 2 2024-03-19T01:24:57.551 INFO:tasks.workunit.client.0.smithi069.stderr:make: *** [Makefile:234: __sub-make] Error 2 2024-03-19T01:24:57.553 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-19T01:24:57.553 INFO:tasks.workunit:Stopping ['kernel_untar_build.sh'] on client.0... 2024-03-19T01:24:57.553 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-19T01:24:57.731 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-19T01:24:57.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:24:57.732 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-19T01:24:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.935+0000 7f2d86a00640 1 -- 172.21.15.69:0/3613018000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d801039c0 msgr2=0x7f2d80103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.935+0000 7f2d86a00640 1 --2- 172.21.15.69:0/3613018000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d801039c0 0x7f2d80103e40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009930 tx=0x7f2d6c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:24:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.936+0000 7f2d86a00640 1 -- 172.21.15.69:0/3613018000 shutdown_connections 2024-03-19T01:24:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.936+0000 7f2d86a00640 1 --2- 172.21.15.69:0/3613018000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d80104380 0x7f2d80110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.936+0000 7f2d86a00640 1 --2- 172.21.15.69:0/3613018000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d801039c0 0x7f2d80103e40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.936+0000 7f2d86a00640 1 --2- 172.21.15.69:0/3613018000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d801099d0 0x7f2d80109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.936+0000 7f2d86a00640 1 -- 172.21.15.69:0/3613018000 >> 172.21.15.69:0/3613018000 conn(0x7f2d800ff4d0 msgr2=0x7f2d801018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.936+0000 7f2d86a00640 1 -- 172.21.15.69:0/3613018000 shutdown_connections 2024-03-19T01:24:57.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.936+0000 7f2d86a00640 1 -- 172.21.15.69:0/3613018000 wait complete. 2024-03-19T01:24:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.937+0000 7f2d86a00640 1 Processor -- start 2024-03-19T01:24:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.937+0000 7f2d86a00640 1 -- start start 2024-03-19T01:24:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.938+0000 7f2d86a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d801039c0 0x7f2d8019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.938+0000 7f2d86a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d80104380 0x7f2d8019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.938+0000 7f2d86a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d801099d0 0x7f2d8019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.938+0000 7f2d86a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d80113e30 con 0x7f2d801039c0 2024-03-19T01:24:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.938+0000 7f2d86a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d80113cb0 con 0x7f2d80104380 2024-03-19T01:24:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.938+0000 7f2d86a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2d80113fb0 con 0x7f2d801099d0 2024-03-19T01:24:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.938+0000 7f2d7f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d80104380 0x7f2d8019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.938+0000 7f2d84a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d801099d0 0x7f2d8019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.938+0000 7f2d7fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d801039c0 0x7f2d8019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.938+0000 7f2d7f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d80104380 0x7f2d8019a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:48010/0 (socket says 172.21.15.69:48010) 2024-03-19T01:24:57.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.938+0000 7f2d7f400640 1 -- 172.21.15.69:0/1545412126 learned_addr learned my addr 172.21.15.69:0/1545412126 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.939+0000 7f2d7f400640 1 -- 172.21.15.69:0/1545412126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d801099d0 msgr2=0x7f2d8019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.939+0000 7f2d7f400640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d801099d0 0x7f2d8019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.939+0000 7f2d7f400640 1 -- 172.21.15.69:0/1545412126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d801039c0 msgr2=0x7f2d8019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.939+0000 7f2d7f400640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d801039c0 0x7f2d8019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.939+0000 7f2d7f400640 1 -- 172.21.15.69:0/1545412126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8019f330 con 0x7f2d80104380 2024-03-19T01:24:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.939+0000 7f2d7f400640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d80104380 0x7f2d8019a8d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009900 tx=0x7f2d6c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.939+0000 7f2d7cc00640 1 -- 172.21.15.69:0/1545412126 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c043020 con 0x7f2d80104380 2024-03-19T01:24:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.939+0000 7f2d86a00640 1 -- 172.21.15.69:0/1545412126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8019f5c0 con 0x7f2d80104380 2024-03-19T01:24:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.939+0000 7f2d7cc00640 1 -- 172.21.15.69:0/1545412126 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d6c0054e0 con 0x7f2d80104380 2024-03-19T01:24:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.939+0000 7f2d7cc00640 1 -- 172.21.15.69:0/1545412126 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c003bb0 con 0x7f2d80104380 2024-03-19T01:24:57.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.940+0000 7f2d86a00640 1 -- 172.21.15.69:0/1545412126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d801a6eb0 con 0x7f2d80104380 2024-03-19T01:24:57.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.943+0000 7f2d7cc00640 1 -- 172.21.15.69:0/1545412126 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f2d6c046020 con 0x7f2d80104380 2024-03-19T01:24:57.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.944+0000 7f2d897b4740 1 Processor -- start 2024-03-19T01:24:57.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.944+0000 7f2d897b4740 1 -- start start 2024-03-19T01:24:57.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d035ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d0413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d897b4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563b4cf98fb0 con 0x563b4d035b00 2024-03-19T01:24:57.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d897b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b4cf41ab0 con 0x563b4d036420 2024-03-19T01:24:57.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d897b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563b4cf95050 con 0x563b4d03f000 2024-03-19T01:24:57.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d7fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d7fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:48018/0 (socket says 172.21.15.69:48018) 2024-03-19T01:24:57.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d84a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d035ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d7fe00640 1 -- 172.21.15.69:0/726927198 learned_addr learned my addr 172.21.15.69:0/726927198 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:57.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d7f400640 1 --2- 172.21.15.69:0/726927198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d0413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d7fe00640 1 -- 172.21.15.69:0/726927198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 msgr2=0x563b4d0413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:57.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/726927198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d0413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.945+0000 7f2d7fe00640 1 -- 172.21.15.69:0/726927198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 msgr2=0x563b4d035ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:57.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.946+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/726927198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d035ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.946+0000 7f2d7fe00640 1 -- 172.21.15.69:0/726927198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4cf5fe00 con 0x563b4d036420 2024-03-19T01:24:57.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.946+0000 7f2d7f400640 1 --2- 172.21.15.69:0/726927198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d0413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:57.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.946+0000 7f2d7cc00640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2d640a29f0 0x7f2d640a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.946+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/726927198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03eac0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009c70 tx=0x7f2d70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72a1dd4cf70a1d16 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:57.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.946+0000 7f2d7cc00640 1 -- 172.21.15.69:0/1545412126 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f2d6c0f1100 con 0x7f2d80104380 2024-03-19T01:24:57.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.946+0000 7f2d75200640 1 -- 172.21.15.69:0/726927198 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70004230 con 0x563b4d036420 2024-03-19T01:24:57.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.946+0000 7f2d75200640 1 -- 172.21.15.69:0/726927198 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d700043d0 con 0x563b4d036420 2024-03-19T01:24:57.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.946+0000 7f2d75200640 1 -- 172.21.15.69:0/726927198 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70019820 con 0x563b4d036420 2024-03-19T01:24:57.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.946+0000 7f2d84a00640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2d640a29f0 0x7f2d640a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.947+0000 7f2d897b4740 1 -- 172.21.15.69:0/726927198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 msgr2=0x563b4d03eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.947+0000 7f2d897b4740 1 --2- 172.21.15.69:0/726927198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03eac0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009c70 tx=0x7f2d70011810 comp rx=0 tx=0).stop 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.947+0000 7f2d897b4740 1 -- 172.21.15.69:0/726927198 shutdown_connections 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.947+0000 7f2d897b4740 1 --2- 172.21.15.69:0/726927198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d0413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.947+0000 7f2d897b4740 1 --2- 172.21.15.69:0/726927198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03eac0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.947+0000 7f2d897b4740 1 --2- 172.21.15.69:0/726927198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d035ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.947+0000 7f2d897b4740 1 -- 172.21.15.69:0/726927198 >> 172.21.15.69:0/726927198 conn(0x563b4d030230 msgr2=0x563b4d030610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.947+0000 7f2d84a00640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2d640a29f0 0x7f2d640a4eb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d78005fd0 tx=0x7f2d78009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.947+0000 7f2d897b4740 1 -- 172.21.15.69:0/726927198 shutdown_connections 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.947+0000 7f2d897b4740 1 -- 172.21.15.69:0/726927198 wait complete. 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.948+0000 7f2d897b4740 1 -- start start 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.948+0000 7f2d897b4740 1 Processor -- start 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d03bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d03ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d897b4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563b4d041df0 con 0x563b4d035b00 2024-03-19T01:24:57.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d897b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b4cf54f00 con 0x563b4d036420 2024-03-19T01:24:57.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d897b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563b4d041f70 con 0x563b4d03f000 2024-03-19T01:24:57.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d897b4740 10 client.0 _ll_register_callbacks cb 0x563b4cf548d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:57.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d7f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d7f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:48026/0 (socket says 172.21.15.69:48026) 2024-03-19T01:24:57.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d7f400640 1 -- 172.21.15.69:0/265194445 learned_addr learned my addr 172.21.15.69:0/265194445 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:57.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d84a00640 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d03ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d03bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d7f400640 1 -- 172.21.15.69:0/265194445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 msgr2=0x563b4d03ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:57.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d7f400640 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d03ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d7f400640 1 -- 172.21.15.69:0/265194445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 msgr2=0x563b4d03bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d7f400640 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d03bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.949+0000 7f2d7f400640 1 -- 172.21.15.69:0/265194445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4cd757e0 con 0x563b4d036420 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.950+0000 7f2d7f400640 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03c520 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c000c00 tx=0x7f2d6c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d8efd3796e9eeb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.950+0000 7f2d69600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.950+0000 7f2d69600640 1 -- 172.21.15.69:0/265194445 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0bcd80 con 0x563b4d036420 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.950+0000 7f2d897b4740 1 -- 172.21.15.69:0/265194445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d700043d0 con 0x563b4d036420 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.950+0000 7f2d69600640 1 -- 172.21.15.69:0/265194445 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f2d6c0ee680 con 0x563b4d036420 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.950+0000 7f2d69600640 1 -- 172.21.15.69:0/265194445 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2d6c004c80 con 0x563b4d036420 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.950+0000 7f2d897b4740 10 client.28199 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.950+0000 7f2d897b4740 1 -- 172.21.15.69:0/265194445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x563b4cdd5f70 con 0x563b4d036420 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.950+0000 7f2d69600640 1 -- 172.21.15.69:0/265194445 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d700043d0 con 0x563b4d036420 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.951+0000 7f2d69600640 1 -- 172.21.15.69:0/265194445 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f2d6c0c1100 con 0x563b4d036420 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.951+0000 7f2d897b4740 10 client.28199 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:57.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.951+0000 7f2d897b4740 10 client.28199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.951+0000 7f2d897b4740 20 client.28199 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.951+0000 7f2d897b4740 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563b4d04f2c0 0x563b4d051730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.951+0000 7f2d897b4740 4 client.28199 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.951+0000 7f2d897b4740 1 -- 172.21.15.69:0/265194445 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b4cfa64f0 con 0x563b4d04f2c0 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.951+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563b4d04f2c0 0x563b4d051730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.952+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563b4d04f2c0 0x563b4d051730 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86586691920c6fe8 server_cookie=b8609320b73734db in_seq=0 out_seq=0 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.952+0000 7f2d69600640 10 client.28199 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.952+0000 7f2d69600640 1 -- 172.21.15.69:0/265194445 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563b4cfa64f0 con 0x563b4d04f2c0 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.952+0000 7f2d69600640 10 client.28199 handle_command_reply: tid=0 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.952+0000 7f2d897b4740 1 client.28199 shutdown 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d897b4740 1 -- 172.21.15.69:0/265194445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 msgr2=0x563b4d03c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d897b4740 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03c520 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c000c00 tx=0x7f2d6c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:24:57.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d897b4740 1 -- 172.21.15.69:0/265194445 shutdown_connections 2024-03-19T01:24:57.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d897b4740 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563b4d04f2c0 0x563b4d051730 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d897b4740 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d03ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d897b4740 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d036420 0x563b4d03c520 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d69600640 0 client.28199 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:57.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d897b4740 1 --2- 172.21.15.69:0/265194445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d03bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d897b4740 1 -- 172.21.15.69:0/265194445 >> 172.21.15.69:0/265194445 conn(0x563b4d030230 msgr2=0x563b4d031f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:57.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d897b4740 1 -- 172.21.15.69:0/265194445 shutdown_connections 2024-03-19T01:24:57.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d897b4740 1 -- 172.21.15.69:0/265194445 wait complete. 2024-03-19T01:24:57.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.953+0000 7f2d897b4740 20 client.28199 trim_cache size 0 max 16384 2024-03-19T01:24:57.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.972+0000 7f2d897b4740 1 Processor -- start 2024-03-19T01:24:57.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.972+0000 7f2d897b4740 1 -- start start 2024-03-19T01:24:57.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d035b00 0x563b4d03e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d036420 0x563b4d04ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d02f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d897b4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801b10b0 con 0x563b4d036420 2024-03-19T01:24:57.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d897b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801b2bf0 con 0x563b4d035b00 2024-03-19T01:24:57.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d897b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801b1780 con 0x563b4d03f000 2024-03-19T01:24:57.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d7fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d035b00 0x563b4d03e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d7f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d036420 0x563b4d04ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d84a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d02f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d7fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d035b00 0x563b4d03e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:48032/0 (socket says 172.21.15.69:48032) 2024-03-19T01:24:57.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d7fe00640 1 -- 172.21.15.69:0/1030971495 learned_addr learned my addr 172.21.15.69:0/1030971495 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:57.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d7fe00640 1 -- 172.21.15.69:0/1030971495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 msgr2=0x563b4d02f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:57.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/1030971495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d02f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d7fe00640 1 -- 172.21.15.69:0/1030971495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d036420 msgr2=0x563b4d04ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:57.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/1030971495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d036420 0x563b4d04ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.973+0000 7f2d7fe00640 1 -- 172.21.15.69:0/1030971495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4cc6d170 con 0x563b4d035b00 2024-03-19T01:24:57.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.974+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/1030971495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d035b00 0x563b4d03e710 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2d70003f30 tx=0x7f2d70005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd538e979c222165 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:57.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.974+0000 7f2d6be00640 1 -- 172.21.15.69:0/1030971495 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70030070 con 0x563b4d035b00 2024-03-19T01:24:57.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.974+0000 7f2d6be00640 1 -- 172.21.15.69:0/1030971495 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d70005c50 con 0x563b4d035b00 2024-03-19T01:24:57.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.974+0000 7f2d6be00640 1 -- 172.21.15.69:0/1030971495 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d700052a0 con 0x563b4d035b00 2024-03-19T01:24:57.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.975+0000 7f2d897b4740 1 -- 172.21.15.69:0/1030971495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d035b00 msgr2=0x563b4d03e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:57.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.975+0000 7f2d897b4740 1 --2- 172.21.15.69:0/1030971495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d035b00 0x563b4d03e710 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2d70003f30 tx=0x7f2d70005f30 comp rx=0 tx=0).stop 2024-03-19T01:24:57.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.975+0000 7f2d897b4740 1 -- 172.21.15.69:0/1030971495 shutdown_connections 2024-03-19T01:24:57.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.975+0000 7f2d897b4740 1 --2- 172.21.15.69:0/1030971495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d03f000 0x563b4d02f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.975+0000 7f2d897b4740 1 --2- 172.21.15.69:0/1030971495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d036420 0x563b4d04ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.975+0000 7f2d897b4740 1 --2- 172.21.15.69:0/1030971495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d035b00 0x563b4d03e710 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.975+0000 7f2d897b4740 1 -- 172.21.15.69:0/1030971495 >> 172.21.15.69:0/1030971495 conn(0x563b4d030230 msgr2=0x563b4d02ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.975+0000 7f2d897b4740 1 -- 172.21.15.69:0/1030971495 shutdown_connections 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.975+0000 7f2d897b4740 1 -- 172.21.15.69:0/1030971495 wait complete. 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.975+0000 7f2d897b4740 1 -- start start 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.975+0000 7f2d897b4740 1 Processor -- start 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d0ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d036420 0x563b4d0adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d897b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d03f000 0x563b4d043460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d897b4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801b2520 con 0x563b4d035b00 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d897b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801b1e50 con 0x563b4d03f000 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d897b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801af090 con 0x563b4d036420 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d897b4740 10 client.0 _ll_register_callbacks cb 0x7f2d6c02ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d7f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d036420 0x563b4d0adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d7f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d036420 0x563b4d0adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46676/0 (socket says 172.21.15.69:46676) 2024-03-19T01:24:57.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d7f400640 1 -- 172.21.15.69:0/3930970758 learned_addr learned my addr 172.21.15.69:0/3930970758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:57.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d84a00640 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d03f000 0x563b4d043460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d0ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d7f400640 1 -- 172.21.15.69:0/3930970758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d03f000 msgr2=0x563b4d043460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:57.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d7f400640 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d03f000 0x563b4d043460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d7f400640 1 -- 172.21.15.69:0/3930970758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 msgr2=0x563b4d0ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:57.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d7f400640 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d0ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.976+0000 7f2d7f400640 1 -- 172.21.15.69:0/3930970758 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4d043c60 con 0x563b4d036420 2024-03-19T01:24:57.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.977+0000 7f2d84a00640 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d03f000 0x563b4d043460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:57.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.977+0000 7f2d7f400640 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d036420 0x563b4d0adeb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0bb6d0 tx=0x7f2d6c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=156fec1cf82d6f06 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:57.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.977+0000 7f2d69600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:24:57.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.977+0000 7f2d69600640 1 -- 172.21.15.69:0/3930970758 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c004c80 con 0x563b4d036420 2024-03-19T01:24:57.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.977+0000 7f2d69600640 1 -- 172.21.15.69:0/3930970758 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f2d6c0c5730 con 0x563b4d036420 2024-03-19T01:24:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.977+0000 7f2d897b4740 1 -- 172.21.15.69:0/3930970758 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d70005c50 con 0x563b4d036420 2024-03-19T01:24:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.977+0000 7f2d69600640 1 -- 172.21.15.69:0/3930970758 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f2d6c005690 con 0x563b4d036420 2024-03-19T01:24:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.977+0000 7f2d897b4740 10 client.28123 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.977+0000 7f2d897b4740 1 -- 172.21.15.69:0/3930970758 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x563b4d0b6240 con 0x563b4d036420 2024-03-19T01:24:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.977+0000 7f2d69600640 1 -- 172.21.15.69:0/3930970758 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d70005c50 con 0x563b4d036420 2024-03-19T01:24:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.978+0000 7f2d69600640 1 -- 172.21.15.69:0/3930970758 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f2d6c0eeb50 con 0x563b4d036420 2024-03-19T01:24:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.978+0000 7f2d897b4740 10 client.28123 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.978+0000 7f2d897b4740 10 client.28123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:24:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.978+0000 7f2d897b4740 20 client.28123 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.978+0000 7f2d897b4740 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563b4d0b6830 0x563b4d0b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.978+0000 7f2d897b4740 4 client.28123 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.978+0000 7f2d897b4740 1 -- 172.21.15.69:0/3930970758 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563b4cfa0840 con 0x563b4d0b6830 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.978+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563b4d0b6830 0x563b4d0b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.978+0000 7f2d7fe00640 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563b4d0b6830 0x563b4d0b8c50 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca9d655d1c7ae91 server_cookie=877d97c1eef87a9a in_seq=0 out_seq=0 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.978+0000 7f2d69600640 10 client.28123 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.979+0000 7f2d69600640 1 -- 172.21.15.69:0/3930970758 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563b4cfa0840 con 0x563b4d0b6830 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.979+0000 7f2d69600640 10 client.28123 handle_command_reply: tid=0 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.979+0000 7f2d897b4740 1 client.28123 shutdown 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d897b4740 1 -- 172.21.15.69:0/3930970758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d036420 msgr2=0x563b4d0adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d897b4740 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d036420 0x563b4d0adeb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0bb6d0 tx=0x7f2d6c004610 comp rx=0 tx=0).stop 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d897b4740 1 -- 172.21.15.69:0/3930970758 shutdown_connections 2024-03-19T01:24:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d897b4740 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x563b4d0b6830 0x563b4d0b8c50 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d897b4740 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b4d03f000 0x563b4d043460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d897b4740 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b4d036420 0x563b4d0adeb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d897b4740 1 --2- 172.21.15.69:0/3930970758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563b4d035b00 0x563b4d0ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d897b4740 1 -- 172.21.15.69:0/3930970758 >> 172.21.15.69:0/3930970758 conn(0x563b4d030230 msgr2=0x563b4d02f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d69600640 0 client.28123 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d897b4740 1 -- 172.21.15.69:0/3930970758 shutdown_connections 2024-03-19T01:24:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d897b4740 1 -- 172.21.15.69:0/3930970758 wait complete. 2024-03-19T01:24:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.980+0000 7f2d897b4740 20 client.28123 trim_cache size 0 max 16384 2024-03-19T01:24:57.984 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:24:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-03-19T01:24:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "f9b09e23-b574-4a40-9990-2a062806c126", 2024-03-19T01:24:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-03-19T01:24:57.985 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:24:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.981+0000 7f2d86a00640 1 -- 172.21.15.69:0/1545412126 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2d640a29f0 msgr2=0x7f2d640a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.981+0000 7f2d86a00640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2d640a29f0 0x7f2d640a4eb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d78005fd0 tx=0x7f2d78009500 comp rx=0 tx=0).stop 2024-03-19T01:24:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.981+0000 7f2d86a00640 1 -- 172.21.15.69:0/1545412126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d80104380 msgr2=0x7f2d8019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.981+0000 7f2d86a00640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d80104380 0x7f2d8019a8d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009900 tx=0x7f2d6c004360 comp rx=0 tx=0).stop 2024-03-19T01:24:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.981+0000 7f2d86a00640 1 -- 172.21.15.69:0/1545412126 shutdown_connections 2024-03-19T01:24:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.981+0000 7f2d86a00640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d801099d0 0x7f2d8019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.981+0000 7f2d86a00640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2d640a29f0 0x7f2d640a4eb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.981+0000 7f2d86a00640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d80104380 0x7f2d8019a8d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.981+0000 7f2d86a00640 1 --2- 172.21.15.69:0/1545412126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d801039c0 0x7f2d8019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.981+0000 7f2d86a00640 1 -- 172.21.15.69:0/1545412126 >> 172.21.15.69:0/1545412126 conn(0x7f2d800ff4d0 msgr2=0x7f2d800ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.982+0000 7f2d86a00640 1 -- 172.21.15.69:0/1545412126 shutdown_connections 2024-03-19T01:24:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:57.982+0000 7f2d86a00640 1 -- 172.21.15.69:0/1545412126 wait complete. 2024-03-19T01:24:58.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:24:58.005 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:24:58.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:58 smithi069 ceph-mon[31727]: pgmap v447: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 837 KiB/s wr, 93 op/s 2024-03-19T01:24:58.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:58 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:58.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:58 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:58.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.310+0000 7f7f6f400640 1 -- 172.21.15.69:0/1896561374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f68110040 msgr2=0x7f7f6810db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:58.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.310+0000 7f7f6f400640 1 --2- 172.21.15.69:0/1896561374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f68110040 0x7f7f6810db30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009860 tx=0x7f7f5802bb70 comp rx=0 tx=0).stop 2024-03-19T01:24:58.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.310+0000 7f7f6f400640 1 -- 172.21.15.69:0/1896561374 shutdown_connections 2024-03-19T01:24:58.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.310+0000 7f7f6f400640 1 --2- 172.21.15.69:0/1896561374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f6811b000 0x7f7f6811d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.310+0000 7f7f6f400640 1 --2- 172.21.15.69:0/1896561374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f6810e070 0x7f7f6810e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.310+0000 7f7f6f400640 1 --2- 172.21.15.69:0/1896561374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f68110040 0x7f7f6810db30 secure :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009860 tx=0x7f7f5802bb70 comp rx=0 tx=0).stop 2024-03-19T01:24:58.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.310+0000 7f7f6f400640 1 -- 172.21.15.69:0/1896561374 >> 172.21.15.69:0/1896561374 conn(0x7f7f6806c900 msgr2=0x7f7f6806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:58.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.310+0000 7f7f6f400640 1 -- 172.21.15.69:0/1896561374 shutdown_connections 2024-03-19T01:24:58.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.310+0000 7f7f6f400640 1 -- 172.21.15.69:0/1896561374 wait complete. 2024-03-19T01:24:58.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.312+0000 7f7f6f400640 1 Processor -- start 2024-03-19T01:24:58.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.312+0000 7f7f6f400640 1 -- start start 2024-03-19T01:24:58.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.312+0000 7f7f6f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f6810e070 0x7f7f681a7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.312+0000 7f7f6f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6811b000 0x7f7f681a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.312+0000 7f7f6f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f681ab910 0x7f7f681abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f6f400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f68120960 con 0x7f7f6810e070 2024-03-19T01:24:58.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f6f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681207e0 con 0x7f7f6811b000 2024-03-19T01:24:58.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f6f400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7f68120ae0 con 0x7f7f681ab910 2024-03-19T01:24:58.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f6d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f681ab910 0x7f7f681abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f6d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f681ab910 0x7f7f681abdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46704/0 (socket says 172.21.15.69:46704) 2024-03-19T01:24:58.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f6d400640 1 -- 172.21.15.69:0/662335195 learned_addr learned my addr 172.21.15.69:0/662335195 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:58.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f67e00640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6811b000 0x7f7f681a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f6d400640 1 -- 172.21.15.69:0/662335195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6811b000 msgr2=0x7f7f681a7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:58.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f6d400640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6811b000 0x7f7f681a7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f6d400640 1 -- 172.21.15.69:0/662335195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f6810e070 msgr2=0x7f7f681a7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:58.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f6d400640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f6810e070 0x7f7f681a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.313+0000 7f7f6d400640 1 -- 172.21.15.69:0/662335195 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f681ac400 con 0x7f7f681ab910 2024-03-19T01:24:58.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.314+0000 7f7f6d400640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f681ab910 0x7f7f681abdc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009fd0 tx=0x7f7f6000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:58.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.314+0000 7f7f65600640 1 -- 172.21.15.69:0/662335195 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60004b20 con 0x7f7f681ab910 2024-03-19T01:24:58.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.314+0000 7f7f65600640 1 -- 172.21.15.69:0/662335195 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f60003e50 con 0x7f7f681ab910 2024-03-19T01:24:58.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.314+0000 7f7f65600640 1 -- 172.21.15.69:0/662335195 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6000b630 con 0x7f7f681ab910 2024-03-19T01:24:58.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.314+0000 7f7f6f400640 1 -- 172.21.15.69:0/662335195 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f681ac6f0 con 0x7f7f681ab910 2024-03-19T01:24:58.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.314+0000 7f7f6f400640 1 -- 172.21.15.69:0/662335195 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f681b3e90 con 0x7f7f681ab910 2024-03-19T01:24:58.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.320+0000 7f7f72222740 1 Processor -- start 2024-03-19T01:24:58.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.322+0000 7f7f65600640 1 -- 172.21.15.69:0/662335195 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f7f6001d020 con 0x7f7f681ab910 2024-03-19T01:24:58.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.322+0000 7f7f72222740 1 -- start start 2024-03-19T01:24:58.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.323+0000 7f7f72222740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 0x559e16d67120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.323+0000 7f7f72222740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 0x559e16d6fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.323+0000 7f7f72222740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d72630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.323+0000 7f7f72222740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559e16ccb2c0 con 0x559e16d67660 2024-03-19T01:24:58.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.323+0000 7f7f72222740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e16c73b60 con 0x559e16d70240 2024-03-19T01:24:58.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.323+0000 7f7f72222740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559e16cc7360 con 0x559e16d66d40 2024-03-19T01:24:58.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.324+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 0x559e16d67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.324+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 0x559e16d67120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46708/0 (socket says 172.21.15.69:46708) 2024-03-19T01:24:58.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.324+0000 7f7f67e00640 1 -- 172.21.15.69:0/1460045900 learned_addr learned my addr 172.21.15.69:0/1460045900 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:58.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.324+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/1460045900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 0x559e16d6fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.324+0000 7f7f67e00640 1 --2- 172.21.15.69:0/1460045900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d72630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.325+0000 7f7f6ca00640 1 -- 172.21.15.69:0/1460045900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 msgr2=0x559e16d67120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:58.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.325+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/1460045900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 0x559e16d67120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.325+0000 7f7f6ca00640 1 -- 172.21.15.69:0/1460045900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 msgr2=0x559e16d72630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:58.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.325+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/1460045900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d72630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.325+0000 7f7f6ca00640 1 -- 172.21.15.69:0/1460045900 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e16aa97d0 con 0x559e16d67660 2024-03-19T01:24:58.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.325+0000 7f7f65600640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f500a29f0 0x7f7f500a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.326+0000 7f7f67e00640 1 --2- 172.21.15.69:0/1460045900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d72630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:24:58.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.326+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/1460045900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 0x559e16d6fd00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7f58008000 tx=0x7f7f58009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149eb21eab056f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:58.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.326+0000 7f7f65600640 1 -- 172.21.15.69:0/662335195 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f7f600cf350 con 0x7f7f681ab910 2024-03-19T01:24:58.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.327+0000 7f7f67e00640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f500a29f0 0x7f7f500a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.327+0000 7f7f55c00640 1 -- 172.21.15.69:0/1460045900 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58041020 con 0x559e16d67660 2024-03-19T01:24:58.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.327+0000 7f7f55c00640 1 -- 172.21.15.69:0/1460045900 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f58034cb0 con 0x559e16d67660 2024-03-19T01:24:58.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.327+0000 7f7f55c00640 1 -- 172.21.15.69:0/1460045900 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58039040 con 0x559e16d67660 2024-03-19T01:24:58.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.328+0000 7f7f72222740 1 -- 172.21.15.69:0/1460045900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 msgr2=0x559e16d6fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:58.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.328+0000 7f7f72222740 1 --2- 172.21.15.69:0/1460045900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 0x559e16d6fd00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7f58008000 tx=0x7f7f58009560 comp rx=0 tx=0).stop 2024-03-19T01:24:58.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.329+0000 7f7f67e00640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f500a29f0 0x7f7f500a4eb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x559e16d62600 tx=0x7f7f5c004730 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:58.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.329+0000 7f7f72222740 1 -- 172.21.15.69:0/1460045900 shutdown_connections 2024-03-19T01:24:58.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.329+0000 7f7f72222740 1 --2- 172.21.15.69:0/1460045900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d72630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.329+0000 7f7f72222740 1 --2- 172.21.15.69:0/1460045900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 0x559e16d6fd00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.329+0000 7f7f72222740 1 --2- 172.21.15.69:0/1460045900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 0x559e16d67120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.329+0000 7f7f72222740 1 -- 172.21.15.69:0/1460045900 >> 172.21.15.69:0/1460045900 conn(0x559e16d61470 msgr2=0x559e16d61850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:58.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.329+0000 7f7f72222740 1 -- 172.21.15.69:0/1460045900 shutdown_connections 2024-03-19T01:24:58.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.330+0000 7f7f72222740 1 -- 172.21.15.69:0/1460045900 wait complete. 2024-03-19T01:24:58.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.330+0000 7f7f72222740 1 -- start start 2024-03-19T01:24:58.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.332+0000 7f7f72222740 1 Processor -- start 2024-03-19T01:24:58.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.332+0000 7f7f72222740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d66d40 0x559e16d6d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.333+0000 7f7f72222740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d67660 0x559e16d6d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.333+0000 7f7f72222740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d6dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.333+0000 7f7f72222740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559e16d72fd0 con 0x559e16d66d40 2024-03-19T01:24:58.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.333+0000 7f7f72222740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e16c86fc0 con 0x559e16d70240 2024-03-19T01:24:58.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.333+0000 7f7f72222740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559e16d73150 con 0x559e16d67660 2024-03-19T01:24:58.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.333+0000 7f7f72222740 10 client.0 _ll_register_callbacks cb 0x559e16c86990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.333+0000 7f7f6d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d6dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.333+0000 7f7f6ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d66d40 0x559e16d6d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.333+0000 7f7f6d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d6dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:48084/0 (socket says 172.21.15.69:48084) 2024-03-19T01:24:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.333+0000 7f7f6d400640 1 -- 172.21.15.69:0/2532362024 learned_addr learned my addr 172.21.15.69:0/2532362024 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.334+0000 7f7f6ca00640 1 -- 172.21.15.69:0/2532362024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d67660 msgr2=0x559e16d6d820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:24:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.334+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/2532362024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d67660 0x559e16d6d820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.334+0000 7f7f6ca00640 1 -- 172.21.15.69:0/2532362024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 msgr2=0x559e16d6dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.334+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/2532362024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d6dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.334+0000 7f7f6ca00640 1 -- 172.21.15.69:0/2532362024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e169df3f0 con 0x559e16d66d40 2024-03-19T01:24:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.334+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/2532362024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d66d40 0x559e16d6d2e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7f58037040 tx=0x7f7f58034830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977256f0c98cdb8e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.334+0000 7f7f4a000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:24:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.335+0000 7f7f4a000640 1 -- 172.21.15.69:0/2532362024 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5804a020 con 0x559e16d66d40 2024-03-19T01:24:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.335+0000 7f7f4a000640 1 -- 172.21.15.69:0/2532362024 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f7f5802c720 con 0x559e16d66d40 2024-03-19T01:24:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.335+0000 7f7f72222740 1 -- 172.21.15.69:0/2532362024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f58034cb0 con 0x559e16d66d40 2024-03-19T01:24:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.335+0000 7f7f4a000640 1 -- 172.21.15.69:0/2532362024 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f7f58039050 con 0x559e16d66d40 2024-03-19T01:24:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.335+0000 7f7f72222740 10 client.17808 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:58.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.336+0000 7f7f72222740 1 -- 172.21.15.69:0/2532362024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x559e1699f0f0 con 0x559e16d66d40 2024-03-19T01:24:58.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.336+0000 7f7f4a000640 1 -- 172.21.15.69:0/2532362024 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f58034cb0 con 0x559e16d66d40 2024-03-19T01:24:58.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.337+0000 7f7f4a000640 1 -- 172.21.15.69:0/2532362024 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f7f5803c070 con 0x559e16d66d40 2024-03-19T01:24:58.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.337+0000 7f7f72222740 10 client.17808 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:58.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.337+0000 7f7f72222740 10 client.17808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:24:58.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.337+0000 7f7f72222740 20 client.17808 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:58.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.337+0000 7f7f72222740 1 --2- 172.21.15.69:0/2532362024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559e16d80450 0x559e16d82840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.337+0000 7f7f72222740 4 client.17808 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:24:58.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.337+0000 7f7f72222740 1 -- 172.21.15.69:0/2532362024 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e16c6c610 con 0x559e16d80450 2024-03-19T01:24:58.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.339+0000 7f7f67e00640 1 --2- 172.21.15.69:0/2532362024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559e16d80450 0x559e16d82840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.341+0000 7f7f67e00640 1 --2- 172.21.15.69:0/2532362024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559e16d80450 0x559e16d82840 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f5c1ecf34667ee5 server_cookie=2ffd76c2930a903f in_seq=0 out_seq=0 2024-03-19T01:24:58.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.342+0000 7f7f4a000640 10 client.17808 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:58.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.343+0000 7f7f4a000640 1 -- 172.21.15.69:0/2532362024 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559e16c6c610 con 0x559e16d80450 2024-03-19T01:24:58.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.343+0000 7f7f4a000640 10 client.17808 handle_command_reply: tid=0 2024-03-19T01:24:58.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.343+0000 7f7f72222740 1 client.17808 shutdown 2024-03-19T01:24:58.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.343+0000 7f7f72222740 1 -- 172.21.15.69:0/2532362024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d66d40 msgr2=0x559e16d6d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:58.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.343+0000 7f7f72222740 1 --2- 172.21.15.69:0/2532362024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d66d40 0x559e16d6d2e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7f58037040 tx=0x7f7f58034830 comp rx=0 tx=0).stop 2024-03-19T01:24:58.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.344+0000 7f7f72222740 1 -- 172.21.15.69:0/2532362024 shutdown_connections 2024-03-19T01:24:58.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.344+0000 7f7f72222740 1 --2- 172.21.15.69:0/2532362024 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559e16d80450 0x559e16d82840 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.344+0000 7f7f72222740 1 --2- 172.21.15.69:0/2532362024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d6dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.344+0000 7f7f72222740 1 --2- 172.21.15.69:0/2532362024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d67660 0x559e16d6d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.344+0000 7f7f72222740 1 --2- 172.21.15.69:0/2532362024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d66d40 0x559e16d6d2e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.344+0000 7f7f72222740 1 -- 172.21.15.69:0/2532362024 >> 172.21.15.69:0/2532362024 conn(0x559e16d61470 msgr2=0x559e16d631c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:58.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.344+0000 7f7f4a000640 0 client.17808 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:58.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.344+0000 7f7f72222740 1 -- 172.21.15.69:0/2532362024 shutdown_connections 2024-03-19T01:24:58.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.345+0000 7f7f72222740 1 -- 172.21.15.69:0/2532362024 wait complete. 2024-03-19T01:24:58.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.345+0000 7f7f72222740 20 client.17808 trim_cache size 0 max 16384 2024-03-19T01:24:58.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.392+0000 7f7f72222740 1 Processor -- start 2024-03-19T01:24:58.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.392+0000 7f7f72222740 1 -- start start 2024-03-19T01:24:58.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.393+0000 7f7f72222740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 0x559e16d6fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.393+0000 7f7f72222740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 0x559e16d6c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.393+0000 7f7f72222740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d85860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.393+0000 7f7f72222740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681be130 con 0x559e16d67660 2024-03-19T01:24:58.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.393+0000 7f7f72222740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681bfc70 con 0x559e16d70240 2024-03-19T01:24:58.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.393+0000 7f7f72222740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681be800 con 0x559e16d66d40 2024-03-19T01:24:58.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.394+0000 7f7f6ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 0x559e16d6fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.394+0000 7f7f6ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 0x559e16d6fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46720/0 (socket says 172.21.15.69:46720) 2024-03-19T01:24:58.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.394+0000 7f7f6ca00640 1 -- 172.21.15.69:0/30373775 learned_addr learned my addr 172.21.15.69:0/30373775 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:58.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.394+0000 7f7f67e00640 1 --2- 172.21.15.69:0/30373775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 0x559e16d6c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.394+0000 7f7f6d400640 1 --2- 172.21.15.69:0/30373775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d85860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.394+0000 7f7f6ca00640 1 -- 172.21.15.69:0/30373775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 msgr2=0x559e16d85860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:58.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.394+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/30373775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d85860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.394+0000 7f7f6ca00640 1 -- 172.21.15.69:0/30373775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 msgr2=0x559e16d6c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:58.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.394+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/30373775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 0x559e16d6c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.394+0000 7f7f6ca00640 1 -- 172.21.15.69:0/30373775 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e16d6b830 con 0x559e16d66d40 2024-03-19T01:24:58.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.394+0000 7f7f67e00640 1 --2- 172.21.15.69:0/30373775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 0x559e16d6c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:58.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.395+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/30373775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 0x559e16d6fa10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009860 tx=0x7f7f58009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=184443407225ccb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:58.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.395+0000 7f7f55200640 1 -- 172.21.15.69:0/30373775 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5804a070 con 0x559e16d66d40 2024-03-19T01:24:58.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.395+0000 7f7f55200640 1 -- 172.21.15.69:0/30373775 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f580348a0 con 0x559e16d66d40 2024-03-19T01:24:58.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.396+0000 7f7f55200640 1 -- 172.21.15.69:0/30373775 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58041040 con 0x559e16d66d40 2024-03-19T01:24:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.397+0000 7f7f72222740 1 -- 172.21.15.69:0/30373775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 msgr2=0x559e16d6fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.397+0000 7f7f72222740 1 --2- 172.21.15.69:0/30373775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 0x559e16d6fa10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009860 tx=0x7f7f58009540 comp rx=0 tx=0).stop 2024-03-19T01:24:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.397+0000 7f7f72222740 1 -- 172.21.15.69:0/30373775 shutdown_connections 2024-03-19T01:24:58.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.397+0000 7f7f72222740 1 --2- 172.21.15.69:0/30373775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d70240 0x559e16d85860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.397+0000 7f7f72222740 1 --2- 172.21.15.69:0/30373775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d67660 0x559e16d6c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.397+0000 7f7f72222740 1 --2- 172.21.15.69:0/30373775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d66d40 0x559e16d6fa10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.397+0000 7f7f72222740 1 -- 172.21.15.69:0/30373775 >> 172.21.15.69:0/30373775 conn(0x559e16d61470 msgr2=0x559e16d6ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:58.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.398+0000 7f7f72222740 1 -- 172.21.15.69:0/30373775 shutdown_connections 2024-03-19T01:24:58.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.398+0000 7f7f72222740 1 -- 172.21.15.69:0/30373775 wait complete. 2024-03-19T01:24:58.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.398+0000 7f7f72222740 1 -- start start 2024-03-19T01:24:58.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.399+0000 7f7f72222740 1 Processor -- start 2024-03-19T01:24:58.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.400+0000 7f7f72222740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d66d40 0x559e16d85ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.400+0000 7f7f72222740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d67660 0x559e16d83070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.400+0000 7f7f72222740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d70240 0x559e16d837b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.400+0000 7f7f72222740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681bf5a0 con 0x559e16d70240 2024-03-19T01:24:58.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.400+0000 7f7f72222740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681beed0 con 0x559e16d66d40 2024-03-19T01:24:58.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.400+0000 7f7f72222740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681bc110 con 0x559e16d67660 2024-03-19T01:24:58.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.401+0000 7f7f72222740 10 client.0 _ll_register_callbacks cb 0x7f7f60009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:24:58.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.401+0000 7f7f6d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d70240 0x559e16d837b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.401+0000 7f7f6ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d66d40 0x559e16d85ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.401+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d67660 0x559e16d83070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.402+0000 7f7f6d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d70240 0x559e16d837b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44638/0 (socket says 172.21.15.69:44638) 2024-03-19T01:24:58.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.402+0000 7f7f6d400640 1 -- 172.21.15.69:0/3245969887 learned_addr learned my addr 172.21.15.69:0/3245969887 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:24:58.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.402+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d67660 0x559e16d83070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:46734/0 (socket says 172.21.15.69:46734) 2024-03-19T01:24:58.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.401+0000 7f7f6ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d66d40 0x559e16d85ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:48100/0 (socket says 172.21.15.69:48100) 2024-03-19T01:24:58.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.402+0000 7f7f6d400640 1 -- 172.21.15.69:0/3245969887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d67660 msgr2=0x559e16d83070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:58.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.402+0000 7f7f6d400640 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d67660 0x559e16d83070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.402+0000 7f7f6d400640 1 -- 172.21.15.69:0/3245969887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d66d40 msgr2=0x559e16d85ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:24:58.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.402+0000 7f7f6d400640 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d66d40 0x559e16d85ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.402+0000 7f7f6d400640 1 -- 172.21.15.69:0/3245969887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e16de53f0 con 0x559e16d70240 2024-03-19T01:24:58.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.402+0000 7f7f67e00640 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d67660 0x559e16d83070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:24:58.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.403+0000 7f7f6d400640 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d70240 0x559e16d837b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7f60004360 tx=0x7f7f6009b160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef74e35588223ef9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:24:58.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.403+0000 7f7f4a000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:24:58.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.404+0000 7f7f4a000640 1 -- 172.21.15.69:0/3245969887 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6001a040 con 0x559e16d70240 2024-03-19T01:24:58.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.404+0000 7f7f72222740 1 -- 172.21.15.69:0/3245969887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f580348a0 con 0x559e16d70240 2024-03-19T01:24:58.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.405+0000 7f7f4a000640 1 -- 172.21.15.69:0/3245969887 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f7f600b4220 con 0x559e16d70240 2024-03-19T01:24:58.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.405+0000 7f7f4a000640 1 -- 172.21.15.69:0/3245969887 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f7f600b4420 con 0x559e16d70240 2024-03-19T01:24:58.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.406+0000 7f7f72222740 10 client.17814 fetch_fsmap learned FSMap version 59 2024-03-19T01:24:58.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.406+0000 7f7f72222740 1 -- 172.21.15.69:0/3245969887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x559e16de65c0 con 0x559e16d70240 2024-03-19T01:24:58.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.406+0000 7f7f4a000640 1 -- 172.21.15.69:0/3245969887 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f580348a0 con 0x559e16d70240 2024-03-19T01:24:58.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.406+0000 7f7f4a000640 1 -- 172.21.15.69:0/3245969887 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f7f6000b040 con 0x559e16d70240 2024-03-19T01:24:58.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.406+0000 7f7f72222740 10 client.17814 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:24:58.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.406+0000 7f7f72222740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:24:58.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.406+0000 7f7f72222740 20 client.17814 populate_metadata read hostname 'smithi069' 2024-03-19T01:24:58.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.406+0000 7f7f72222740 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559e16de6d10 0x559e16de9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:24:58.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.406+0000 7f7f72222740 4 client.17814 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:24:58.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.406+0000 7f7f72222740 1 -- 172.21.15.69:0/3245969887 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e16cd8800 con 0x559e16de6d10 2024-03-19T01:24:58.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.407+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559e16de6d10 0x559e16de9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:24:58.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.408+0000 7f7f6ca00640 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559e16de6d10 0x559e16de9130 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b7020292edf9ad4 server_cookie=93a87bf36623d1e4 in_seq=0 out_seq=0 2024-03-19T01:24:58.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.408+0000 7f7f4a000640 10 client.17814 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:58.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.410+0000 7f7f4a000640 1 -- 172.21.15.69:0/3245969887 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x559e16cd8800 con 0x559e16de6d10 2024-03-19T01:24:58.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.410+0000 7f7f4a000640 10 client.17814 handle_command_reply: tid=0 2024-03-19T01:24:58.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.410+0000 7f7f72222740 1 client.17814 shutdown 2024-03-19T01:24:58.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.411+0000 7f7f72222740 1 -- 172.21.15.69:0/3245969887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d70240 msgr2=0x559e16d837b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:58.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.411+0000 7f7f72222740 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d70240 0x559e16d837b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7f60004360 tx=0x7f7f6009b160 comp rx=0 tx=0).stop 2024-03-19T01:24:58.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.412+0000 7f7f72222740 1 -- 172.21.15.69:0/3245969887 shutdown_connections 2024-03-19T01:24:58.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.412+0000 7f7f72222740 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x559e16de6d10 0x559e16de9130 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.412+0000 7f7f72222740 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559e16d70240 0x559e16d837b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.412+0000 7f7f72222740 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e16d67660 0x559e16d83070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.412+0000 7f7f72222740 1 --2- 172.21.15.69:0/3245969887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e16d66d40 0x559e16d85ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.412+0000 7f7f72222740 1 -- 172.21.15.69:0/3245969887 >> 172.21.15.69:0/3245969887 conn(0x559e16d61470 msgr2=0x559e16d7bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:58.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.412+0000 7f7f4a000640 0 client.17814 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:24:58.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.413+0000 7f7f72222740 1 -- 172.21.15.69:0/3245969887 shutdown_connections 2024-03-19T01:24:58.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.413+0000 7f7f72222740 1 -- 172.21.15.69:0/3245969887 wait complete. 2024-03-19T01:24:58.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.413+0000 7f7f72222740 20 client.17814 trim_cache size 0 max 16384 2024-03-19T01:24:58.414 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:24:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (8933 inodes in the stack)", 2024-03-19T01:24:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-03-19T01:24:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "f9b09e23-b574-4a40-9990-2a062806c126": { 2024-03-19T01:24:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-03-19T01:24:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "f9b09e23-b574-4a40-9990-2a062806c126", 2024-03-19T01:24:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-03-19T01:24:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:24:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:24:58.415 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:24:58.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.415+0000 7f7f6f400640 1 -- 172.21.15.69:0/662335195 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f500a29f0 msgr2=0x7f7f500a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:58.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.415+0000 7f7f6f400640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f500a29f0 0x7f7f500a4eb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x559e16d62600 tx=0x7f7f5c004730 comp rx=0 tx=0).stop 2024-03-19T01:24:58.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.415+0000 7f7f6f400640 1 -- 172.21.15.69:0/662335195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f681ab910 msgr2=0x7f7f681abdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:24:58.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.415+0000 7f7f6f400640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f681ab910 0x7f7f681abdc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009fd0 tx=0x7f7f6000c800 comp rx=0 tx=0).stop 2024-03-19T01:24:58.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.416+0000 7f7f6f400640 1 -- 172.21.15.69:0/662335195 shutdown_connections 2024-03-19T01:24:58.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.416+0000 7f7f6f400640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f681ab910 0x7f7f681abdc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.416+0000 7f7f6f400640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f7f500a29f0 0x7f7f500a4eb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.416+0000 7f7f6f400640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6811b000 0x7f7f681a7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.416+0000 7f7f6f400640 1 --2- 172.21.15.69:0/662335195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f6810e070 0x7f7f681a7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:24:58.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.416+0000 7f7f6f400640 1 -- 172.21.15.69:0/662335195 >> 172.21.15.69:0/662335195 conn(0x7f7f6806c900 msgr2=0x7f7f68119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:24:58.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.416+0000 7f7f6f400640 1 -- 172.21.15.69:0/662335195 shutdown_connections 2024-03-19T01:24:58.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:24:58.416+0000 7f7f6f400640 1 -- 172.21.15.69:0/662335195 wait complete. 2024-03-19T01:24:58.446 INFO:tasks.cephfs.filesystem:scrub status for tag:f9b09e23-b574-4a40-9990-2a062806c126 - {'path': '/', 'tag': 'f9b09e23-b574-4a40-9990-2a062806c126', 'options': 'recursive,force'} 2024-03-19T01:24:58.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:58 smithi086 ceph-mon[66829]: pgmap v447: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 837 KiB/s wr, 93 op/s 2024-03-19T01:24:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:58 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:58.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:58 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:58.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:58 smithi163 ceph-mon[67428]: pgmap v447: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 837 KiB/s wr, 93 op/s 2024-03-19T01:24:58.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:58 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:58.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:58 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub queued for path: / 2024-03-19T01:24:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:24:59.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:24:59 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.527 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub queued for path: / 2024-03-19T01:24:59.527 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.527 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.528 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:24:59.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:24:59 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.534 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test kernel_untar_build.sh) on smithi069 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e48b1f6c0608b2615e1890cc46286c3877d6e910 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2024-03-19T01:24:59.771 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b2e443ee9ae9439eb8014606c2d0fbcc Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test kernel_untar_build.sh) on smithi069 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e48b1f6c0608b2615e1890cc46286c3877d6e910 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2024-03-19T01:24:59.774 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-19T01:24:59.784 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-19T01:24:59.785 INFO:teuthology.task.exec:Executing custom commands... 2024-03-19T01:24:59.785 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-03-19T01:24:59.785 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-19T01:24:59.790 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub queued for path: / 2024-03-19T01:24:59.790 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.790 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.790 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.791 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.791 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.791 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.791 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.791 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.791 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.791 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.791 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.792 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.792 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.792 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.792 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.792 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.792 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.792 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.792 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.792 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.792 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.793 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.793 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.793 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.793 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.793 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.793 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.793 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.793 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.794 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.794 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.794 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.794 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.794 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.794 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.794 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.794 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.799 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.799 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.799 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.799 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.799 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.799 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.799 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.799 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:24:59.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:24:59 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:24:59.903 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-19T01:25:00.041 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-19T01:25:00.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.311+0000 7f8ea4c00640 1 -- 172.21.15.69:0/2221493540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ea0109a00 msgr2=0x7f8ea0109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:00.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.311+0000 7f8ea4c00640 1 --2- 172.21.15.69:0/2221493540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ea0109a00 0x7f8ea0109de0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009930 tx=0x7f8e8802bc20 comp rx=0 tx=0).stop 2024-03-19T01:25:00.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.313+0000 7f8ea4c00640 1 -- 172.21.15.69:0/2221493540 shutdown_connections 2024-03-19T01:25:00.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.313+0000 7f8ea4c00640 1 --2- 172.21.15.69:0/2221493540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea01043b0 0x7f8ea01108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:00.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.313+0000 7f8ea4c00640 1 --2- 172.21.15.69:0/2221493540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ea01039f0 0x7f8ea0103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:00.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.313+0000 7f8ea4c00640 1 --2- 172.21.15.69:0/2221493540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ea0109a00 0x7f8ea0109de0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:00.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.313+0000 7f8ea4c00640 1 -- 172.21.15.69:0/2221493540 >> 172.21.15.69:0/2221493540 conn(0x7f8ea00ff4e0 msgr2=0x7f8ea0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:00.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.313+0000 7f8ea4c00640 1 -- 172.21.15.69:0/2221493540 shutdown_connections 2024-03-19T01:25:00.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.313+0000 7f8ea4c00640 1 -- 172.21.15.69:0/2221493540 wait complete. 2024-03-19T01:25:00.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.314+0000 7f8ea4c00640 1 Processor -- start 2024-03-19T01:25:00.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.314+0000 7f8ea4c00640 1 -- start start 2024-03-19T01:25:00.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.315+0000 7f8ea4c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ea01039f0 0x7f8ea0075d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:00.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.315+0000 7f8ea4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea01043b0 0x7f8ea0076270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:00.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.315+0000 7f8ea4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ea0109a00 0x7f8ea00767b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:00.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.315+0000 7f8ea4c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0113e30 con 0x7f8ea01039f0 2024-03-19T01:25:00.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.315+0000 7f8ea4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0113cb0 con 0x7f8ea01043b0 2024-03-19T01:25:00.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.315+0000 7f8ea4c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0113fb0 con 0x7f8ea0109a00 2024-03-19T01:25:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.315+0000 7f8e9d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea01043b0 0x7f8ea0076270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.315+0000 7f8e9e800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ea0109a00 0x7f8ea00767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.315+0000 7f8e9d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea01043b0 0x7f8ea0076270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:48122/0 (socket says 172.21.15.69:48122) 2024-03-19T01:25:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.316+0000 7f8e9d400640 1 -- 172.21.15.69:0/1673037882 learned_addr learned my addr 172.21.15.69:0/1673037882 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.316+0000 7f8e9de00640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ea01039f0 0x7f8ea0075d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.316+0000 7f8e9d400640 1 -- 172.21.15.69:0/1673037882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ea0109a00 msgr2=0x7f8ea00767b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.316+0000 7f8e9d400640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ea0109a00 0x7f8ea00767b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.316+0000 7f8e9d400640 1 -- 172.21.15.69:0/1673037882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ea01039f0 msgr2=0x7f8ea0075d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.316+0000 7f8e9d400640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ea01039f0 0x7f8ea0075d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.316+0000 7f8e9d400640 1 -- 172.21.15.69:0/1673037882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea01aa8e0 con 0x7f8ea01043b0 2024-03-19T01:25:00.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.316+0000 7f8e9d400640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea01043b0 0x7f8ea0076270 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000d980 tx=0x7f8e9000de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:00.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.317+0000 7f8e96a00640 1 -- 172.21.15.69:0/1673037882 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90017070 con 0x7f8ea01043b0 2024-03-19T01:25:00.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.317+0000 7f8e96a00640 1 -- 172.21.15.69:0/1673037882 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e90009c30 con 0x7f8ea01043b0 2024-03-19T01:25:00.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.317+0000 7f8e96a00640 1 -- 172.21.15.69:0/1673037882 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9000f620 con 0x7f8ea01043b0 2024-03-19T01:25:00.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.317+0000 7f8ea4c00640 1 -- 172.21.15.69:0/1673037882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea01aabd0 con 0x7f8ea01043b0 2024-03-19T01:25:00.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.317+0000 7f8ea4c00640 1 -- 172.21.15.69:0/1673037882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea01ab010 con 0x7f8ea01043b0 2024-03-19T01:25:00.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.319+0000 7f8ea4c00640 1 -- 172.21.15.69:0/1673037882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea0105130 con 0x7f8ea01043b0 2024-03-19T01:25:00.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.322+0000 7f8e96a00640 1 -- 172.21.15.69:0/1673037882 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f8e90010460 con 0x7f8ea01043b0 2024-03-19T01:25:00.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.324+0000 7f8e96a00640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8e840a2b60 0x7f8e840a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:00.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.324+0000 7f8e96a00640 1 -- 172.21.15.69:0/1673037882 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7f8e900cedc0 con 0x7f8ea01043b0 2024-03-19T01:25:00.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.325+0000 7f8e9de00640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8e840a2b60 0x7f8e840a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:00.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.326+0000 7f8e9de00640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8e840a2b60 0x7f8e840a5020 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009900 tx=0x7f8e88005fd0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:00.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.332+0000 7f8e96a00640 1 -- 172.21.15.69:0/1673037882 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8e90009de0 con 0x7f8ea01043b0 2024-03-19T01:25:00.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: pgmap v448: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 948 KiB/s wr, 97 op/s 2024-03-19T01:25:00.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:00.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:00 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: pgmap v448: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 948 KiB/s wr, 97 op/s 2024-03-19T01:25:00.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:00.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:00 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: pgmap v448: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 948 KiB/s wr, 97 op/s 2024-03-19T01:25:00.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:00.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:00 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:00.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.745+0000 7f8ea4c00640 1 -- 172.21.15.69:0/1673037882 --> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f8ea0100200 con 0x7f8e840a2b60 2024-03-19T01:25:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.859+0000 7f8e96a00640 1 -- 172.21.15.69:0/1673037882 <== mgr.25645 v2:172.21.15.69:6800/2931658273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f8ea0100200 con 0x7f8e840a2b60 2024-03-19T01:25:00.861 INFO:teuthology.orchestra.run.smithi069.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-19T00:00:00", "created": "2024-03-19T01:10:41", "first": "2024-03-19T01:11:00", "last": "2024-03-19T01:25:00", "last_pruned": "2024-03-19T01:24:00", "created_count": 15, "pruned_count": 8, "active": true} 2024-03-19T01:25:00.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.863+0000 7f8ea4c00640 1 -- 172.21.15.69:0/1673037882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8e840a2b60 msgr2=0x7f8e840a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:00.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.863+0000 7f8ea4c00640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8e840a2b60 0x7f8e840a5020 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009900 tx=0x7f8e88005fd0 comp rx=0 tx=0).stop 2024-03-19T01:25:00.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.863+0000 7f8ea4c00640 1 -- 172.21.15.69:0/1673037882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea01043b0 msgr2=0x7f8ea0076270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:00.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.863+0000 7f8ea4c00640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea01043b0 0x7f8ea0076270 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000d980 tx=0x7f8e9000de40 comp rx=0 tx=0).stop 2024-03-19T01:25:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.864+0000 7f8ea4c00640 1 -- 172.21.15.69:0/1673037882 shutdown_connections 2024-03-19T01:25:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.864+0000 7f8ea4c00640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ea0109a00 0x7f8ea00767b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.864+0000 7f8ea4c00640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f8e840a2b60 0x7f8e840a5020 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.864+0000 7f8ea4c00640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea01043b0 0x7f8ea0076270 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.864+0000 7f8ea4c00640 1 --2- 172.21.15.69:0/1673037882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ea01039f0 0x7f8ea0075d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.864+0000 7f8ea4c00640 1 -- 172.21.15.69:0/1673037882 >> 172.21.15.69:0/1673037882 conn(0x7f8ea00ff4e0 msgr2=0x7f8ea00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.864+0000 7f8ea4c00640 1 -- 172.21.15.69:0/1673037882 shutdown_connections 2024-03-19T01:25:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:00.864+0000 7f8ea4c00640 1 -- 172.21.15.69:0/1673037882 wait complete. 2024-03-19T01:25:00.891 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-19T01:25:01.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.181+0000 7ff2e9a00640 1 -- 172.21.15.69:0/136062989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2e4100d90 msgr2=0x7ff2e410d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:01.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.181+0000 7ff2e9a00640 1 --2- 172.21.15.69:0/136062989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2e4100d90 0x7ff2e410d9c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009900 tx=0x7ff2cc02bc40 comp rx=0 tx=0).stop 2024-03-19T01:25:01.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.182+0000 7ff2e9a00640 1 -- 172.21.15.69:0/136062989 shutdown_connections 2024-03-19T01:25:01.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.182+0000 7ff2e9a00640 1 --2- 172.21.15.69:0/136062989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e410df90 0x7ff2e4110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:01.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.182+0000 7ff2e9a00640 1 --2- 172.21.15.69:0/136062989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2e4100d90 0x7ff2e410d9c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:01.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.182+0000 7ff2e9a00640 1 --2- 172.21.15.69:0/136062989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e41003e0 0x7ff2e41007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:01.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.182+0000 7ff2e9a00640 1 -- 172.21.15.69:0/136062989 >> 172.21.15.69:0/136062989 conn(0x7ff2e40fc030 msgr2=0x7ff2e40fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:01.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.182+0000 7ff2e9a00640 1 -- 172.21.15.69:0/136062989 shutdown_connections 2024-03-19T01:25:01.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.182+0000 7ff2e9a00640 1 -- 172.21.15.69:0/136062989 wait complete. 2024-03-19T01:25:01.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.183+0000 7ff2e9a00640 1 Processor -- start 2024-03-19T01:25:01.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.184+0000 7ff2e9a00640 1 -- start start 2024-03-19T01:25:01.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.184+0000 7ff2e9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e41003e0 0x7ff2e419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:01.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.184+0000 7ff2e9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4100d90 0x7ff2e419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:01.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.184+0000 7ff2e9a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2e410df90 0x7ff2e4195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:01.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.185+0000 7ff2e9a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e4104f40 con 0x7ff2e410df90 2024-03-19T01:25:01.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.185+0000 7ff2e9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e4104dc0 con 0x7ff2e4100d90 2024-03-19T01:25:01.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.185+0000 7ff2e9a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41050c0 con 0x7ff2e41003e0 2024-03-19T01:25:01.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.185+0000 7ff2dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4100d90 0x7ff2e419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:01.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.185+0000 7ff2dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4100d90 0x7ff2e419bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:48140/0 (socket says 172.21.15.69:48140) 2024-03-19T01:25:01.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.185+0000 7ff2dbe00640 1 -- 172.21.15.69:0/184922052 learned_addr learned my addr 172.21.15.69:0/184922052 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:01.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.185+0000 7ff2e3e00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e41003e0 0x7ff2e419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:01.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.185+0000 7ff2e9000640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2e410df90 0x7ff2e4195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:01.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.186+0000 7ff2e3e00640 1 -- 172.21.15.69:0/184922052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4100d90 msgr2=0x7ff2e419bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:01.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.186+0000 7ff2e3e00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4100d90 0x7ff2e419bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:01.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.186+0000 7ff2e3e00640 1 -- 172.21.15.69:0/184922052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2e410df90 msgr2=0x7ff2e4195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:01.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.186+0000 7ff2e3e00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2e410df90 0x7ff2e4195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:01.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.186+0000 7ff2e3e00640 1 -- 172.21.15.69:0/184922052 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e41967b0 con 0x7ff2e41003e0 2024-03-19T01:25:01.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.186+0000 7ff2dbe00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4100d90 0x7ff2e419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:01.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.187+0000 7ff2e3e00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e41003e0 0x7ff2e419ba60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400e9f0 tx=0x7ff2d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:01.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.187+0000 7ff2e1600640 1 -- 172.21.15.69:0/184922052 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400ccc0 con 0x7ff2e41003e0 2024-03-19T01:25:01.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.187+0000 7ff2e9a00640 1 -- 172.21.15.69:0/184922052 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e4196aa0 con 0x7ff2e41003e0 2024-03-19T01:25:01.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.187+0000 7ff2e1600640 1 -- 172.21.15.69:0/184922052 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2d4010910 con 0x7ff2e41003e0 2024-03-19T01:25:01.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.187+0000 7ff2e1600640 1 -- 172.21.15.69:0/184922052 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4018960 con 0x7ff2e41003e0 2024-03-19T01:25:01.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.187+0000 7ff2e9a00640 1 -- 172.21.15.69:0/184922052 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e41a29e0 con 0x7ff2e41003e0 2024-03-19T01:25:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.192+0000 7ff2e9a00640 1 -- 172.21.15.69:0/184922052 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c0005290 con 0x7ff2e41003e0 2024-03-19T01:25:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.194+0000 7ff2e1600640 1 -- 172.21.15.69:0/184922052 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ff2d4010ac0 con 0x7ff2e41003e0 2024-03-19T01:25:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.196+0000 7ff2e1600640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2c80a29b0 0x7ff2c80a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.197+0000 7ff2e1600640 1 -- 172.21.15.69:0/184922052 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9568+0+0 (secure 0 0 0) 0x7ff2d40cf580 con 0x7ff2e41003e0 2024-03-19T01:25:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.197+0000 7ff2dbe00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2c80a29b0 0x7ff2c80a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:01.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.198+0000 7ff2dbe00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2c80a29b0 0x7ff2c80a4e70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4197300 tx=0x7ff2cc02cf90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:01.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.200+0000 7ff2e1600640 1 -- 172.21.15.69:0/184922052 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff2d409bed0 con 0x7ff2e41003e0 2024-03-19T01:25:01.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:01.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.516 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:01 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:01.542 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.542 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:01 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:01.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:01 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.613+0000 7ff2e9a00640 1 -- 172.21.15.69:0/184922052 --> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff2c0000bc0 con 0x7ff2c80a29b0 2024-03-19T01:25:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.627+0000 7ff2e1600640 1 -- 172.21.15.69:0/184922052 <== mgr.25645 v2:172.21.15.69:6800/2931658273 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff2c0000bc0 con 0x7ff2c80a29b0 2024-03-19T01:25:01.628 INFO:teuthology.orchestra.run.smithi069.stdout:/ 1m 6m3h 2024-03-19T01:25:01.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.630+0000 7ff2e9a00640 1 -- 172.21.15.69:0/184922052 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2c80a29b0 msgr2=0x7ff2c80a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:01.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.630+0000 7ff2e9a00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2c80a29b0 0x7ff2c80a4e70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4197300 tx=0x7ff2cc02cf90 comp rx=0 tx=0).stop 2024-03-19T01:25:01.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.630+0000 7ff2e9a00640 1 -- 172.21.15.69:0/184922052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e41003e0 msgr2=0x7ff2e419ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:01.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.630+0000 7ff2e9a00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e41003e0 0x7ff2e419ba60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400e9f0 tx=0x7ff2d400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:25:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.631+0000 7ff2e9a00640 1 -- 172.21.15.69:0/184922052 shutdown_connections 2024-03-19T01:25:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.631+0000 7ff2e9a00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2e410df90 0x7ff2e4195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.631+0000 7ff2e9a00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2c80a29b0 0x7ff2c80a4e70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.631+0000 7ff2e9a00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e4100d90 0x7ff2e419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.631+0000 7ff2e9a00640 1 --2- 172.21.15.69:0/184922052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e41003e0 0x7ff2e419ba60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.631+0000 7ff2e9a00640 1 -- 172.21.15.69:0/184922052 >> 172.21.15.69:0/184922052 conn(0x7ff2e40fc030 msgr2=0x7ff2e410f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:01.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.631+0000 7ff2e9a00640 1 -- 172.21.15.69:0/184922052 shutdown_connections 2024-03-19T01:25:01.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:01.631+0000 7ff2e9a00640 1 -- 172.21.15.69:0/184922052 wait complete. 2024-03-19T01:25:01.655 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-19T01:25:01.666 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-19T01:25:02.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: pgmap v449: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 891 KiB/s wr, 97 op/s 2024-03-19T01:25:02.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: from='client.28229 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:25:02.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.511 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:02.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:02 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: pgmap v449: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 891 KiB/s wr, 97 op/s 2024-03-19T01:25:02.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: from='client.28229 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:25:02.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:02.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:02 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: pgmap v449: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 891 KiB/s wr, 97 op/s 2024-03-19T01:25:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: from='client.28229 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:25:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:02.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:02.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:02 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: from='client.28150 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:25:03.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:03.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:03 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: from='client.28150 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:25:03.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:03.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:03 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: from='client.28150 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:25:03.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:03.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:03.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:03 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: pgmap v450: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 899 KiB/s wr, 306 op/s 2024-03-19T01:25:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:04.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:04 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: pgmap v450: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 899 KiB/s wr, 306 op/s 2024-03-19T01:25:04.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:04.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:04 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: pgmap v450: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 899 KiB/s wr, 306 op/s 2024-03-19T01:25:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:04.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:04.764 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:04 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: pgmap v451: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 253 KiB/s wr, 275 op/s 2024-03-19T01:25:05.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:05.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:05 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: pgmap v451: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 253 KiB/s wr, 275 op/s 2024-03-19T01:25:05.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.566 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:05.567 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:05 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: pgmap v451: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 253 KiB/s wr, 275 op/s 2024-03-19T01:25:05.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.761 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:05.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:05.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:05 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:06.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:06 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:06.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:06 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.757 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:06.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:06.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:06 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:07.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:07 smithi163 ceph-mon[67428]: scrub summary: active paths [/] 2024-03-19T01:25:07.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:07 smithi163 ceph-mon[67428]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:07.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:07 smithi163 ceph-mon[67428]: pgmap v452: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 431 KiB/s wr, 837 op/s 2024-03-19T01:25:07.755 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:07 smithi086 ceph-mon[66829]: scrub summary: active paths [/] 2024-03-19T01:25:07.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:07.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:07 smithi086 ceph-mon[66829]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:07.756 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:07 smithi086 ceph-mon[66829]: pgmap v452: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 431 KiB/s wr, 837 op/s 2024-03-19T01:25:07.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:07 smithi069 ceph-mon[31727]: scrub summary: active paths [/] 2024-03-19T01:25:07.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:07.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:07 smithi069 ceph-mon[31727]: scrub summary: idle+waiting paths [/] 2024-03-19T01:25:07.783 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:07 smithi069 ceph-mon[31727]: pgmap v452: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 431 KiB/s wr, 837 op/s 2024-03-19T01:25:10.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:10 smithi086 ceph-mon[66829]: pgmap v453: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 688 KiB/s wr, 1.77k op/s 2024-03-19T01:25:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:10 smithi086 ceph-mon[66829]: osdmap e111: 12 total, 12 up, 12 in 2024-03-19T01:25:10.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:10 smithi069 ceph-mon[31727]: pgmap v453: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 688 KiB/s wr, 1.77k op/s 2024-03-19T01:25:10.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:10 smithi069 ceph-mon[31727]: osdmap e111: 12 total, 12 up, 12 in 2024-03-19T01:25:10.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:10 smithi163 ceph-mon[67428]: pgmap v453: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 688 KiB/s wr, 1.77k op/s 2024-03-19T01:25:10.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:10 smithi163 ceph-mon[67428]: osdmap e111: 12 total, 12 up, 12 in 2024-03-19T01:25:12.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:12 smithi086 ceph-mon[66829]: pgmap v455: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 659 KiB/s wr, 2.24k op/s 2024-03-19T01:25:12.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:12 smithi069 ceph-mon[31727]: pgmap v455: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 659 KiB/s wr, 2.24k op/s 2024-03-19T01:25:12.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:12 smithi163 ceph-mon[67428]: pgmap v455: 129 pgs: 129 active+clean; 2.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 659 KiB/s wr, 2.24k op/s 2024-03-19T01:25:14.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:14 smithi086 ceph-mon[66829]: pgmap v456: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 600 KiB/s wr, 1.99k op/s 2024-03-19T01:25:14.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:14 smithi069 ceph-mon[31727]: pgmap v456: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 600 KiB/s wr, 1.99k op/s 2024-03-19T01:25:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:14 smithi163 ceph-mon[67428]: pgmap v456: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 600 KiB/s wr, 1.99k op/s 2024-03-19T01:25:16.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:16 smithi086 ceph-mon[66829]: pgmap v457: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 600 KiB/s wr, 1.95k op/s 2024-03-19T01:25:16.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:16 smithi069 ceph-mon[31727]: pgmap v457: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 600 KiB/s wr, 1.95k op/s 2024-03-19T01:25:16.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:16 smithi163 ceph-mon[67428]: pgmap v457: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 600 KiB/s wr, 1.95k op/s 2024-03-19T01:25:18.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:18 smithi086 ceph-mon[66829]: pgmap v458: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 380 KiB/s wr, 1.27k op/s 2024-03-19T01:25:18.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:18 smithi069 ceph-mon[31727]: pgmap v458: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 380 KiB/s wr, 1.27k op/s 2024-03-19T01:25:18.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:18 smithi163 ceph-mon[67428]: pgmap v458: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 380 KiB/s wr, 1.27k op/s 2024-03-19T01:25:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:19 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:25:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:19 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:25:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:19 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:25:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:19 smithi086 ceph-mon[66829]: osdmap e112: 12 total, 12 up, 12 in 2024-03-19T01:25:19.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:19 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:25:19.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:19 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:25:19.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:19 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:25:19.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:19 smithi069 ceph-mon[31727]: osdmap e112: 12 total, 12 up, 12 in 2024-03-19T01:25:19.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:19 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:25:19.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:19 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:25:19.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:19 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:25:19.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:19 smithi163 ceph-mon[67428]: osdmap e112: 12 total, 12 up, 12 in 2024-03-19T01:25:20.462 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:20 smithi163 ceph-mon[67428]: pgmap v459: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 79 KiB/s wr, 147 op/s 2024-03-19T01:25:20.463 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:20 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:25:20.463 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:20 smithi163 ceph-mon[67428]: scrub summary: idle 2024-03-19T01:25:20.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:20 smithi086 ceph-mon[66829]: pgmap v459: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 79 KiB/s wr, 147 op/s 2024-03-19T01:25:20.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:20 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:25:20.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:20 smithi086 ceph-mon[66829]: scrub summary: idle 2024-03-19T01:25:20.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:20 smithi069 ceph-mon[31727]: pgmap v459: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 79 KiB/s wr, 147 op/s 2024-03-19T01:25:20.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:20 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:25:20.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:20 smithi069 ceph-mon[31727]: scrub summary: idle 2024-03-19T01:25:22.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:22 smithi086 ceph-mon[66829]: pgmap v461: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 15 op/s 2024-03-19T01:25:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:22 smithi069 ceph-mon[31727]: pgmap v461: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 15 op/s 2024-03-19T01:25:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:22 smithi163 ceph-mon[67428]: pgmap v461: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 15 op/s 2024-03-19T01:25:24.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:24 smithi086 ceph-mon[66829]: pgmap v462: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-03-19T01:25:24.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:24 smithi069 ceph-mon[31727]: pgmap v462: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-03-19T01:25:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:24 smithi163 ceph-mon[67428]: pgmap v462: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-03-19T01:25:26.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:26 smithi086 ceph-mon[66829]: pgmap v463: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-03-19T01:25:26.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:26 smithi069 ceph-mon[31727]: pgmap v463: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-03-19T01:25:26.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:26 smithi163 ceph-mon[67428]: pgmap v463: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-03-19T01:25:28.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-19T01:25:28.448 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-19T01:25:28.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:28 smithi086 ceph-mon[66829]: pgmap v464: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-03-19T01:25:28.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:28 smithi069 ceph-mon[31727]: pgmap v464: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-03-19T01:25:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:28 smithi163 ceph-mon[67428]: pgmap v464: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-03-19T01:25:28.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.664+0000 7ff3ad800640 1 -- 172.21.15.69:0/4252962352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a81077c0 msgr2=0x7ff3a8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.664+0000 7ff3ad800640 1 --2- 172.21.15.69:0/4252962352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a81077c0 0x7ff3a8107ba0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff398009900 tx=0x7ff39802bc30 comp rx=0 tx=0).stop 2024-03-19T01:25:28.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.664+0000 7ff3ad800640 1 -- 172.21.15.69:0/4252962352 shutdown_connections 2024-03-19T01:25:28.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.664+0000 7ff3ad800640 1 --2- 172.21.15.69:0/4252962352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a8102170 0x7ff3a810e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.664+0000 7ff3ad800640 1 --2- 172.21.15.69:0/4252962352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81017b0 0x7ff3a8101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.664+0000 7ff3ad800640 1 --2- 172.21.15.69:0/4252962352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a81077c0 0x7ff3a8107ba0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.664+0000 7ff3ad800640 1 -- 172.21.15.69:0/4252962352 >> 172.21.15.69:0/4252962352 conn(0x7ff3a8078c20 msgr2=0x7ff3a8079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:28.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.664+0000 7ff3ad800640 1 -- 172.21.15.69:0/4252962352 shutdown_connections 2024-03-19T01:25:28.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.664+0000 7ff3ad800640 1 -- 172.21.15.69:0/4252962352 wait complete. 2024-03-19T01:25:28.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.665+0000 7ff3ad800640 1 Processor -- start 2024-03-19T01:25:28.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3ad800640 1 -- start start 2024-03-19T01:25:28.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3ad800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81017b0 0x7ff3a819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3ad800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8102170 0x7ff3a819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3ad800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a81077c0 0x7ff3a819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3ad800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8111b90 con 0x7ff3a81017b0 2024-03-19T01:25:28.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3ad800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8111a10 con 0x7ff3a8102170 2024-03-19T01:25:28.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3ad800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8111d10 con 0x7ff3a81077c0 2024-03-19T01:25:28.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3a7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a81077c0 0x7ff3a819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3a6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8102170 0x7ff3a819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3a7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a81077c0 0x7ff3a819ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33102/0 (socket says 172.21.15.69:33102) 2024-03-19T01:25:28.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3a7400640 1 -- 172.21.15.69:0/2526774710 learned_addr learned my addr 172.21.15.69:0/2526774710 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:28.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.666+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81017b0 0x7ff3a819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff3a7400640 1 -- 172.21.15.69:0/2526774710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8102170 msgr2=0x7ff3a819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff3a7400640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8102170 0x7ff3a819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff3a7400640 1 -- 172.21.15.69:0/2526774710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81017b0 msgr2=0x7ff3a819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff3a7400640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81017b0 0x7ff3a819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff3a7400640 1 -- 172.21.15.69:0/2526774710 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a819f3c0 con 0x7ff3a81077c0 2024-03-19T01:25:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81017b0 0x7ff3a819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:25:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff3a7400640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a81077c0 0x7ff3a819ec90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000eb10 tx=0x7ff3a000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff39f400640 1 -- 172.21.15.69:0/2526774710 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0004b20 con 0x7ff3a81077c0 2024-03-19T01:25:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff39f400640 1 -- 172.21.15.69:0/2526774710 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a0004cc0 con 0x7ff3a81077c0 2024-03-19T01:25:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff3ad800640 1 -- 172.21.15.69:0/2526774710 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a819f6b0 con 0x7ff3a81077c0 2024-03-19T01:25:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff39f400640 1 -- 172.21.15.69:0/2526774710 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0010760 con 0x7ff3a81077c0 2024-03-19T01:25:28.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.667+0000 7ff3ad800640 1 -- 172.21.15.69:0/2526774710 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a81a6f40 con 0x7ff3a81077c0 2024-03-19T01:25:28.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.670+0000 7ff39f400640 1 -- 172.21.15.69:0/2526774710 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ff3a0005020 con 0x7ff3a81077c0 2024-03-19T01:25:28.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.670+0000 7ff3b05cf740 1 Processor -- start 2024-03-19T01:25:28.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.671+0000 7ff3b05cf740 1 -- start start 2024-03-19T01:25:28.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.671+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.671+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 0x564146eb4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.671+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eb5240 0x564146eb7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.671+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564146e102c0 con 0x564146eb5240 2024-03-19T01:25:28.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.671+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564146db8b60 con 0x564146eabd40 2024-03-19T01:25:28.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.671+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564146e0c360 con 0x564146eac660 2024-03-19T01:25:28.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.671+0000 7ff3a6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 0x564146eb4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 0x564146eb4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33118/0 (socket says 172.21.15.69:33118) 2024-03-19T01:25:28.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6a00640 1 -- 172.21.15.69:0/2818775556 learned_addr learned my addr 172.21.15.69:0/2818775556 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:28.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eac120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53868/0 (socket says 172.21.15.69:53868) 2024-03-19T01:25:28.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6000640 1 --2- 172.21.15.69:0/2818775556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eb5240 0x564146eb7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6a00640 1 -- 172.21.15.69:0/2818775556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 msgr2=0x564146eac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:28.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/2818775556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eac120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6a00640 1 -- 172.21.15.69:0/2818775556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eb5240 msgr2=0x564146eb7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:28.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/2818775556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eb5240 0x564146eb7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6a00640 1 -- 172.21.15.69:0/2818775556 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564146bee7d0 con 0x564146eac660 2024-03-19T01:25:28.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6000640 1 --2- 172.21.15.69:0/2818775556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eb5240 0x564146eb7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:25:28.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff39f400640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff38c0a2970 0x7ff38c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.672+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/2818775556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 0x564146eb4d00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff398000c00 tx=0x7ff398005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=366fc1c36c75c826 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:28.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.673+0000 7ff39f400640 1 -- 172.21.15.69:0/2526774710 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7ff3a00cf870 con 0x7ff3a81077c0 2024-03-19T01:25:28.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.673+0000 7ff3a6000640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff38c0a2970 0x7ff38c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.673+0000 7ff393e00640 1 -- 172.21.15.69:0/2818775556 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398042070 con 0x564146eac660 2024-03-19T01:25:28.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.673+0000 7ff393e00640 1 -- 172.21.15.69:0/2818775556 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff398004240 con 0x564146eac660 2024-03-19T01:25:28.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.673+0000 7ff393e00640 1 -- 172.21.15.69:0/2818775556 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3980046c0 con 0x564146eac660 2024-03-19T01:25:28.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3b05cf740 1 -- 172.21.15.69:0/2818775556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 msgr2=0x564146eb4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/2818775556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 0x564146eb4d00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff398000c00 tx=0x7ff398005e50 comp rx=0 tx=0).stop 2024-03-19T01:25:28.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3a6000640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff38c0a2970 0x7ff38c0a4e30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x564146ea7600 tx=0x7ff394013040 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:28.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3b05cf740 1 -- 172.21.15.69:0/2818775556 shutdown_connections 2024-03-19T01:25:28.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/2818775556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eb5240 0x564146eb7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/2818775556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 0x564146eb4d00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/2818775556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3b05cf740 1 -- 172.21.15.69:0/2818775556 >> 172.21.15.69:0/2818775556 conn(0x564146ea6470 msgr2=0x564146ea6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:28.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3b05cf740 1 -- 172.21.15.69:0/2818775556 shutdown_connections 2024-03-19T01:25:28.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3b05cf740 1 -- 172.21.15.69:0/2818775556 wait complete. 2024-03-19T01:25:28.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3b05cf740 1 -- start start 2024-03-19T01:25:28.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.674+0000 7ff3b05cf740 1 Processor -- start 2024-03-19T01:25:28.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 0x564146eb27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eb5240 0x564146eb2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564146eb7fd0 con 0x564146eb5240 2024-03-19T01:25:28.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564146dcbfc0 con 0x564146eabd40 2024-03-19T01:25:28.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564146eb8150 con 0x564146eac660 2024-03-19T01:25:28.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3b05cf740 10 client.0 _ll_register_callbacks cb 0x564146dcb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:28.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3a6000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 0x564146eb27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3a6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3a6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53876/0 (socket says 172.21.15.69:53876) 2024-03-19T01:25:28.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3a6000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 0x564146eb27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33134/0 (socket says 172.21.15.69:33134) 2024-03-19T01:25:28.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3a6a00640 1 -- 172.21.15.69:0/1927638601 learned_addr learned my addr 172.21.15.69:0/1927638601 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:28.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.675+0000 7ff3a7400640 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eb5240 0x564146eb2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.676+0000 7ff3a6a00640 1 -- 172.21.15.69:0/1927638601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 msgr2=0x564146eb27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:28.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.676+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 0x564146eb27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.676+0000 7ff3a6a00640 1 -- 172.21.15.69:0/1927638601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eb5240 msgr2=0x564146eb2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:28.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.676+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eb5240 0x564146eb2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.676+0000 7ff3a6a00640 1 -- 172.21.15.69:0/1927638601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564146b243f0 con 0x564146eabd40 2024-03-19T01:25:28.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.676+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb22b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff398006010 tx=0x7ff39802c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d7132908fce14d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:28.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.676+0000 7ff390c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:25:28.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.676+0000 7ff390c00640 1 -- 172.21.15.69:0/1927638601 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39804f060 con 0x564146eabd40 2024-03-19T01:25:28.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.676+0000 7ff390c00640 1 -- 172.21.15.69:0/1927638601 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7ff398047650 con 0x564146eabd40 2024-03-19T01:25:28.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.676+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1927638601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff398004240 con 0x564146eabd40 2024-03-19T01:25:28.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.677+0000 7ff390c00640 1 -- 172.21.15.69:0/1927638601 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff398042070 con 0x564146eabd40 2024-03-19T01:25:28.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.677+0000 7ff3b05cf740 10 client.28247 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:28.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.677+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1927638601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564146ae40f0 con 0x564146eabd40 2024-03-19T01:25:28.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.677+0000 7ff390c00640 1 -- 172.21.15.69:0/1927638601 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff398004240 con 0x564146eabd40 2024-03-19T01:25:28.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.677+0000 7ff390c00640 1 -- 172.21.15.69:0/1927638601 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7ff398034930 con 0x564146eabd40 2024-03-19T01:25:28.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.677+0000 7ff3b05cf740 10 client.28247 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:28.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.677+0000 7ff3b05cf740 10 client.28247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:25:28.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.677+0000 7ff3b05cf740 20 client.28247 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:28.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.677+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564146ec5440 0x564146ec7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.677+0000 7ff3b05cf740 4 client.28247 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:25:28.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.677+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1927638601 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564146db1610 con 0x564146ec5440 2024-03-19T01:25:28.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.678+0000 7ff3a6000640 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564146ec5440 0x564146ec7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.678+0000 7ff390c00640 10 client.28247 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:25:28.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.678+0000 7ff3a6000640 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564146ec5440 0x564146ec7830 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=626d0538bd311909 server_cookie=4a3e479cd4a71b7d in_seq=0 out_seq=0 2024-03-19T01:25:28.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff390c00640 1 -- 172.21.15.69:0/1927638601 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564146db1610 con 0x564146ec5440 2024-03-19T01:25:28.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff390c00640 10 client.28247 handle_command_reply: tid=0 2024-03-19T01:25:28.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff3b05cf740 1 client.28247 shutdown 2024-03-19T01:25:28.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1927638601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 msgr2=0x564146eb22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb22b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff398006010 tx=0x7ff39802c750 comp rx=0 tx=0).stop 2024-03-19T01:25:28.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1927638601 shutdown_connections 2024-03-19T01:25:28.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564146ec5440 0x564146ec7830 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eb5240 0x564146eb2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eac660 0x564146eb27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/1927638601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb22b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1927638601 >> 172.21.15.69:0/1927638601 conn(0x564146ea6470 msgr2=0x564146ea81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:28.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff390c00640 0 client.28247 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:25:28.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.679+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1927638601 shutdown_connections 2024-03-19T01:25:28.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.680+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1927638601 wait complete. 2024-03-19T01:25:28.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.680+0000 7ff3b05cf740 20 client.28247 trim_cache size 0 max 16384 2024-03-19T01:25:28.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.698+0000 7ff3b05cf740 1 Processor -- start 2024-03-19T01:25:28.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.698+0000 7ff3b05cf740 1 -- start start 2024-03-19T01:25:28.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eac660 0x564146ec0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eb5240 0x564146ea5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81b0fd0 con 0x564146eac660 2024-03-19T01:25:28.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81b2b10 con 0x564146eabd40 2024-03-19T01:25:28.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81b16a0 con 0x564146eb5240 2024-03-19T01:25:28.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3a6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3a6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eac660 0x564146ec0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3a6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53890/0 (socket says 172.21.15.69:53890) 2024-03-19T01:25:28.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3a6a00640 1 -- 172.21.15.69:0/1397058734 learned_addr learned my addr 172.21.15.69:0/1397058734 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:28.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3a7400640 1 --2- 172.21.15.69:0/1397058734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eb5240 0x564146ea5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3a6a00640 1 -- 172.21.15.69:0/1397058734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eb5240 msgr2=0x564146ea5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:28.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.699+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/1397058734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eb5240 0x564146ea5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.700+0000 7ff3a6a00640 1 -- 172.21.15.69:0/1397058734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eac660 msgr2=0x564146ec0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:28.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.700+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/1397058734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eac660 0x564146ec0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.700+0000 7ff3a6a00640 1 -- 172.21.15.69:0/1397058734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564146ec7ba0 con 0x564146eabd40 2024-03-19T01:25:28.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.700+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/1397058734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb49e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff39802c170 tx=0x7ff39802dfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f9da130ad16bb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:28.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.700+0000 7ff393400640 1 -- 172.21.15.69:0/1397058734 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39804f070 con 0x564146eabd40 2024-03-19T01:25:28.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.700+0000 7ff393400640 1 -- 172.21.15.69:0/1397058734 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff398034920 con 0x564146eabd40 2024-03-19T01:25:28.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.700+0000 7ff393400640 1 -- 172.21.15.69:0/1397058734 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39803ddb0 con 0x564146eabd40 2024-03-19T01:25:28.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.701+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1397058734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 msgr2=0x564146eb49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.701+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/1397058734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb49e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff39802c170 tx=0x7ff39802dfc0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.701+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1397058734 shutdown_connections 2024-03-19T01:25:28.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.701+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/1397058734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eb5240 0x564146ea5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.701+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/1397058734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eac660 0x564146ec0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.701+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/1397058734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eabd40 0x564146eb49e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.701+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1397058734 >> 172.21.15.69:0/1397058734 conn(0x564146ea6470 msgr2=0x564146ea4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:28.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.701+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1397058734 shutdown_connections 2024-03-19T01:25:28.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.701+0000 7ff3b05cf740 1 -- 172.21.15.69:0/1397058734 wait complete. 2024-03-19T01:25:28.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.702+0000 7ff3b05cf740 1 -- start start 2024-03-19T01:25:28.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.702+0000 7ff3b05cf740 1 Processor -- start 2024-03-19T01:25:28.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eabd40 0x564146eb8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eac660 0x564146eb9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3b05cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eb5240 0x564146eb9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81b2440 con 0x564146eac660 2024-03-19T01:25:28.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81b1d70 con 0x564146eb5240 2024-03-19T01:25:28.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3b05cf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81aefb0 con 0x564146eabd40 2024-03-19T01:25:28.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3b05cf740 10 client.0 _ll_register_callbacks cb 0x7ff398009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:28.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3a6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eabd40 0x564146eb8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3a6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eabd40 0x564146eb8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33156/0 (socket says 172.21.15.69:33156) 2024-03-19T01:25:28.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3a6a00640 1 -- 172.21.15.69:0/635275688 learned_addr learned my addr 172.21.15.69:0/635275688 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:28.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3a7400640 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eb5240 0x564146eb9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3a6000640 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eac660 0x564146eb9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3a6a00640 1 -- 172.21.15.69:0/635275688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eb5240 msgr2=0x564146eb9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:28.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eb5240 0x564146eb9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3a6a00640 1 -- 172.21.15.69:0/635275688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eac660 msgr2=0x564146eb9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:28.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eac660 0x564146eb9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3a6a00640 1 -- 172.21.15.69:0/635275688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564146f2a400 con 0x564146eabd40 2024-03-19T01:25:28.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.703+0000 7ff3a7400640 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eb5240 0x564146eb9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:28.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.704+0000 7ff3a6a00640 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eabd40 0x564146eb8c00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff3980057c0 tx=0x7ff39803d4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d8f1c3fc6d34a79 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:28.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.704+0000 7ff390c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:25:28.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.704+0000 7ff390c00640 1 -- 172.21.15.69:0/635275688 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39804f040 con 0x564146eabd40 2024-03-19T01:25:28.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.704+0000 7ff390c00640 1 -- 172.21.15.69:0/635275688 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7ff39803dab0 con 0x564146eabd40 2024-03-19T01:25:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.704+0000 7ff3b05cf740 1 -- 172.21.15.69:0/635275688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff398034920 con 0x564146eabd40 2024-03-19T01:25:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.704+0000 7ff390c00640 1 -- 172.21.15.69:0/635275688 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff39806a050 con 0x564146eabd40 2024-03-19T01:25:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.704+0000 7ff3b05cf740 10 client.28168 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.704+0000 7ff3b05cf740 1 -- 172.21.15.69:0/635275688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564146f2b650 con 0x564146eabd40 2024-03-19T01:25:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.705+0000 7ff390c00640 1 -- 172.21.15.69:0/635275688 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff398034920 con 0x564146eabd40 2024-03-19T01:25:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.705+0000 7ff390c00640 1 -- 172.21.15.69:0/635275688 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7ff398047ac0 con 0x564146eabd40 2024-03-19T01:25:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.705+0000 7ff3b05cf740 10 client.28168 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.705+0000 7ff3b05cf740 10 client.28168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:25:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.705+0000 7ff3b05cf740 20 client.28168 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.705+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564146f2bad0 0x564146f2dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.705+0000 7ff3b05cf740 4 client.28168 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-19T01:25:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.705+0000 7ff3b05cf740 1 -- 172.21.15.69:0/635275688 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564146e1d800 con 0x564146f2bad0 2024-03-19T01:25:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.705+0000 7ff3a6000640 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564146f2bad0 0x564146f2dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff3a6000640 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564146f2bad0 0x564146f2dec0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71aaf89658a29bd8 server_cookie=614ff25a8a2209b9 in_seq=0 out_seq=0 2024-03-19T01:25:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff390c00640 10 client.28168 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:25:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff390c00640 1 -- 172.21.15.69:0/635275688 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564146e1d800 con 0x564146f2bad0 2024-03-19T01:25:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff390c00640 10 client.28168 handle_command_reply: tid=0 2024-03-19T01:25:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff3b05cf740 1 client.28168 shutdown 2024-03-19T01:25:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff3b05cf740 1 -- 172.21.15.69:0/635275688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eabd40 msgr2=0x564146eb8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eabd40 0x564146eb8c00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff3980057c0 tx=0x7ff39803d4e0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff3b05cf740 1 -- 172.21.15.69:0/635275688 shutdown_connections 2024-03-19T01:25:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x564146f2bad0 0x564146f2dec0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564146eb5240 0x564146eb9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564146eac660 0x564146eb9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff3b05cf740 1 --2- 172.21.15.69:0/635275688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564146eabd40 0x564146eb8c00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff3b05cf740 1 -- 172.21.15.69:0/635275688 >> 172.21.15.69:0/635275688 conn(0x564146ea6470 msgr2=0x564146ea56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:28.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.706+0000 7ff390c00640 0 client.28168 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:25:28.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.707+0000 7ff3b05cf740 1 -- 172.21.15.69:0/635275688 shutdown_connections 2024-03-19T01:25:28.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.707+0000 7ff3b05cf740 1 -- 172.21.15.69:0/635275688 wait complete. 2024-03-19T01:25:28.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.707+0000 7ff3b05cf740 20 client.28168 trim_cache size 0 max 16384 2024-03-19T01:25:28.715 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:25:28.715 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-03-19T01:25:28.715 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-03-19T01:25:28.716 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:25:28.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.707+0000 7ff3ad800640 1 -- 172.21.15.69:0/2526774710 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff38c0a2970 msgr2=0x7ff38c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.707+0000 7ff3ad800640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff38c0a2970 0x7ff38c0a4e30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x564146ea7600 tx=0x7ff394013040 comp rx=0 tx=0).stop 2024-03-19T01:25:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.707+0000 7ff3ad800640 1 -- 172.21.15.69:0/2526774710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a81077c0 msgr2=0x7ff3a819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.707+0000 7ff3ad800640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a81077c0 0x7ff3a819ec90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000eb10 tx=0x7ff3a000c2d0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.708+0000 7ff3ad800640 1 -- 172.21.15.69:0/2526774710 shutdown_connections 2024-03-19T01:25:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.708+0000 7ff3ad800640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a81077c0 0x7ff3a819ec90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.708+0000 7ff3ad800640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff38c0a2970 0x7ff38c0a4e30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.708+0000 7ff3ad800640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8102170 0x7ff3a819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.708+0000 7ff3ad800640 1 --2- 172.21.15.69:0/2526774710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81017b0 0x7ff3a819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.708+0000 7ff3ad800640 1 -- 172.21.15.69:0/2526774710 >> 172.21.15.69:0/2526774710 conn(0x7ff3a8078c20 msgr2=0x7ff3a810cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:28.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.708+0000 7ff3ad800640 1 -- 172.21.15.69:0/2526774710 shutdown_connections 2024-03-19T01:25:28.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.708+0000 7ff3ad800640 1 -- 172.21.15.69:0/2526774710 wait complete. 2024-03-19T01:25:28.727 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-19T01:25:28.727 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:25:28.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.935+0000 7fb27e200640 1 -- 172.21.15.69:0/2413270839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb278101fd0 msgr2=0x7fb27810e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.935+0000 7fb27e200640 1 --2- 172.21.15.69:0/2413270839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb278101fd0 0x7fb27810e4e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb270009ca0 tx=0x7fb27002c100 comp rx=0 tx=0).stop 2024-03-19T01:25:28.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.935+0000 7fb27e200640 1 -- 172.21.15.69:0/2413270839 shutdown_connections 2024-03-19T01:25:28.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.935+0000 7fb27e200640 1 --2- 172.21.15.69:0/2413270839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb278101fd0 0x7fb27810e4e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.935+0000 7fb27e200640 1 --2- 172.21.15.69:0/2413270839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb278101610 0x7fb278101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.935+0000 7fb27e200640 1 --2- 172.21.15.69:0/2413270839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2781076b0 0x7fb278107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.935+0000 7fb27e200640 1 -- 172.21.15.69:0/2413270839 >> 172.21.15.69:0/2413270839 conn(0x7fb2780fd100 msgr2=0x7fb2780ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:28.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.935+0000 7fb27e200640 1 -- 172.21.15.69:0/2413270839 shutdown_connections 2024-03-19T01:25:28.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.935+0000 7fb27e200640 1 -- 172.21.15.69:0/2413270839 wait complete. 2024-03-19T01:25:28.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.936+0000 7fb27e200640 1 Processor -- start 2024-03-19T01:25:28.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb27e200640 1 -- start start 2024-03-19T01:25:28.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb27e200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb278101610 0x7fb278199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb27e200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb278101fd0 0x7fb27819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb27e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2781076b0 0x7fb27819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb27e200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb278111a30 con 0x7fb278101610 2024-03-19T01:25:28.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb27e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781118b0 con 0x7fb2781076b0 2024-03-19T01:25:28.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb27e200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb278111bb0 con 0x7fb278101fd0 2024-03-19T01:25:28.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb27ce00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb278101610 0x7fb278199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb27d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2781076b0 0x7fb27819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb27ce00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb278101610 0x7fb278199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46432/0 (socket says 172.21.15.69:46432) 2024-03-19T01:25:28.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb27ce00640 1 -- 172.21.15.69:0/2493095786 learned_addr learned my addr 172.21.15.69:0/2493095786 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:28.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.937+0000 7fb277e00640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb278101fd0 0x7fb27819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.938+0000 7fb27d800640 1 -- 172.21.15.69:0/2493095786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb278101fd0 msgr2=0x7fb27819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.938+0000 7fb27d800640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb278101fd0 0x7fb27819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.938+0000 7fb27d800640 1 -- 172.21.15.69:0/2493095786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb278101610 msgr2=0x7fb278199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:28.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.938+0000 7fb27d800640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb278101610 0x7fb278199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:28.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.938+0000 7fb27d800640 1 -- 172.21.15.69:0/2493095786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27819efc0 con 0x7fb2781076b0 2024-03-19T01:25:28.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.938+0000 7fb27ce00640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb278101610 0x7fb278199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:28.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.938+0000 7fb27d800640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2781076b0 0x7fb27819e890 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb270009bf0 tx=0x7fb270034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.938+0000 7fb275600640 1 -- 172.21.15.69:0/2493095786 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270039050 con 0x7fb2781076b0 2024-03-19T01:25:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.938+0000 7fb275600640 1 -- 172.21.15.69:0/2493095786 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb270045070 con 0x7fb2781076b0 2024-03-19T01:25:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.938+0000 7fb275600640 1 -- 172.21.15.69:0/2493095786 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2700044c0 con 0x7fb2781076b0 2024-03-19T01:25:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.938+0000 7fb27e200640 1 -- 172.21.15.69:0/2493095786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27819f250 con 0x7fb2781076b0 2024-03-19T01:25:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.939+0000 7fb27e200640 1 -- 172.21.15.69:0/2493095786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2781a6b40 con 0x7fb2781076b0 2024-03-19T01:25:28.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.940+0000 7fb27e200640 1 -- 172.21.15.69:0/2493095786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb278068530 con 0x7fb2781076b0 2024-03-19T01:25:28.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.943+0000 7fb275600640 1 -- 172.21.15.69:0/2493095786 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fb270004660 con 0x7fb2781076b0 2024-03-19T01:25:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.944+0000 7fb275600640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb2600a29f0 0x7fb2600a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.945+0000 7fb275600640 1 -- 172.21.15.69:0/2493095786 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7fb2700f2420 con 0x7fb2781076b0 2024-03-19T01:25:28.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.945+0000 7fb27ce00640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb2600a29f0 0x7fb2600a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.945+0000 7fb275600640 1 -- 172.21.15.69:0/2493095786 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb2700beec0 con 0x7fb2781076b0 2024-03-19T01:25:28.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:28.946+0000 7fb27ce00640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb2600a29f0 0x7fb2600a4eb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0058c0 tx=0x7fb26c005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.194+0000 7fb27e200640 1 -- 172.21.15.69:0/2493095786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2781a6d10 con 0x7fb2781076b0 2024-03-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.195+0000 7fb275600640 1 -- 172.21.15.69:0/2493095786 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11930 (secure 0 0 0) 0x7fb2700be7c0 con 0x7fb2781076b0 2024-03-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":59,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:20:08.600901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 59 2024-03-19T01:25:29.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.197+0000 7fb27e200640 1 -- 172.21.15.69:0/2493095786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb2600a29f0 msgr2=0x7fb2600a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.197+0000 7fb27e200640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb2600a29f0 0x7fb2600a4eb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0058c0 tx=0x7fb26c005f90 comp rx=0 tx=0).stop 2024-03-19T01:25:29.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.197+0000 7fb27e200640 1 -- 172.21.15.69:0/2493095786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2781076b0 msgr2=0x7fb27819e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.197+0000 7fb27e200640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2781076b0 0x7fb27819e890 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb270009bf0 tx=0x7fb270034670 comp rx=0 tx=0).stop 2024-03-19T01:25:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.198+0000 7fb27e200640 1 -- 172.21.15.69:0/2493095786 shutdown_connections 2024-03-19T01:25:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.198+0000 7fb27e200640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb2600a29f0 0x7fb2600a4eb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.198+0000 7fb27e200640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2781076b0 0x7fb27819e890 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.198+0000 7fb27e200640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb278101fd0 0x7fb27819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.198+0000 7fb27e200640 1 --2- 172.21.15.69:0/2493095786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb278101610 0x7fb278199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.198+0000 7fb27e200640 1 -- 172.21.15.69:0/2493095786 >> 172.21.15.69:0/2493095786 conn(0x7fb2780fd100 msgr2=0x7fb2780fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.198+0000 7fb27e200640 1 -- 172.21.15.69:0/2493095786 shutdown_connections 2024-03-19T01:25:29.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.198+0000 7fb27e200640 1 -- 172.21.15.69:0/2493095786 wait complete. 2024-03-19T01:25:29.218 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-19T01:25:29.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.424+0000 7f88d1c00640 1 -- 172.21.15.69:0/730382282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88cc101790 msgr2=0x7f88cc101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.424+0000 7f88d1c00640 1 --2- 172.21.15.69:0/730382282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88cc101790 0x7f88cc101c10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009900 tx=0x7f88b402bc30 comp rx=0 tx=0).stop 2024-03-19T01:25:29.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.424+0000 7f88d1c00640 1 -- 172.21.15.69:0/730382282 shutdown_connections 2024-03-19T01:25:29.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.424+0000 7f88d1c00640 1 --2- 172.21.15.69:0/730382282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88cc102150 0x7f88cc10e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.424+0000 7f88d1c00640 1 --2- 172.21.15.69:0/730382282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88cc101790 0x7f88cc101c10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.424+0000 7f88d1c00640 1 --2- 172.21.15.69:0/730382282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88cc1077a0 0x7f88cc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.424+0000 7f88d1c00640 1 -- 172.21.15.69:0/730382282 >> 172.21.15.69:0/730382282 conn(0x7f88cc0fd280 msgr2=0x7f88cc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.425+0000 7f88d1c00640 1 -- 172.21.15.69:0/730382282 shutdown_connections 2024-03-19T01:25:29.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.425+0000 7f88d1c00640 1 -- 172.21.15.69:0/730382282 wait complete. 2024-03-19T01:25:29.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.425+0000 7f88d1c00640 1 Processor -- start 2024-03-19T01:25:29.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88d1c00640 1 -- start start 2024-03-19T01:25:29.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88d1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88cc101790 0x7f88cc1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88d1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88cc102150 0x7f88cc198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88d1c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88cc1077a0 0x7f88cc19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88d1c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc111c00 con 0x7f88cc1077a0 2024-03-19T01:25:29.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88d1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc111a80 con 0x7f88cc102150 2024-03-19T01:25:29.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88d1c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc111d80 con 0x7f88cc101790 2024-03-19T01:25:29.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88cc102150 0x7f88cc198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88caa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88cc101790 0x7f88cc1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88cc102150 0x7f88cc198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53950/0 (socket says 172.21.15.69:53950) 2024-03-19T01:25:29.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88c3e00640 1 -- 172.21.15.69:0/2682932186 learned_addr learned my addr 172.21.15.69:0/2682932186 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.426+0000 7f88cb400640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88cc1077a0 0x7f88cc19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.427+0000 7f88caa00640 1 -- 172.21.15.69:0/2682932186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88cc102150 msgr2=0x7f88cc198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.427+0000 7f88caa00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88cc102150 0x7f88cc198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.427+0000 7f88caa00640 1 -- 172.21.15.69:0/2682932186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88cc1077a0 msgr2=0x7f88cc19c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.427+0000 7f88caa00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88cc1077a0 0x7f88cc19c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.427+0000 7f88caa00640 1 -- 172.21.15.69:0/2682932186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc19d0d0 con 0x7f88cc101790 2024-03-19T01:25:29.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.427+0000 7f88c3e00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88cc102150 0x7f88cc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:29.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.427+0000 7f88caa00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88cc101790 0x7f88cc1980e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009b50 tx=0x7f88bc00b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.427+0000 7f88c3400640 1 -- 172.21.15.69:0/2682932186 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc009e50 con 0x7f88cc101790 2024-03-19T01:25:29.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.427+0000 7f88d1c00640 1 -- 172.21.15.69:0/2682932186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc19d3c0 con 0x7f88cc101790 2024-03-19T01:25:29.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.427+0000 7f88c3400640 1 -- 172.21.15.69:0/2682932186 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88bc005500 con 0x7f88cc101790 2024-03-19T01:25:29.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.427+0000 7f88c3400640 1 -- 172.21.15.69:0/2682932186 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc019670 con 0x7f88cc101790 2024-03-19T01:25:29.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.428+0000 7f88d1c00640 1 -- 172.21.15.69:0/2682932186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc073a50 con 0x7f88cc101790 2024-03-19T01:25:29.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.432+0000 7f88c3400640 1 -- 172.21.15.69:0/2682932186 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f88bc020020 con 0x7f88cc101790 2024-03-19T01:25:29.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.432+0000 7f88d487d740 1 Processor -- start 2024-03-19T01:25:29.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.432+0000 7f88d487d740 1 -- start start 2024-03-19T01:25:29.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.432+0000 7f88d487d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 0x55bbe25c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.432+0000 7f88d487d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.432+0000 7f88d487d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.432+0000 7f88d487d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bbe2529380 con 0x55bbe25c5340 2024-03-19T01:25:29.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.432+0000 7f88d487d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bbe24d1d20 con 0x55bbe25c4a20 2024-03-19T01:25:29.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.432+0000 7f88d487d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bbe2525420 con 0x55bbe25cdf20 2024-03-19T01:25:29.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 0x55bbe25c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 0x55bbe25c4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53952/0 (socket says 172.21.15.69:53952) 2024-03-19T01:25:29.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33198/0 (socket says 172.21.15.69:33198) 2024-03-19T01:25:29.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88cb400640 1 -- 172.21.15.69:0/2892526903 learned_addr learned my addr 172.21.15.69:0/2892526903 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88caa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88cb400640 1 -- 172.21.15.69:0/2892526903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 msgr2=0x55bbe25d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88cb400640 1 --2- 172.21.15.69:0/2892526903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88cb400640 1 -- 172.21.15.69:0/2892526903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 msgr2=0x55bbe25cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88cb400640 1 --2- 172.21.15.69:0/2892526903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88cb400640 1 -- 172.21.15.69:0/2892526903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe223d3f0 con 0x55bbe25c4a20 2024-03-19T01:25:29.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.433+0000 7f88cb400640 1 --2- 172.21.15.69:0/2892526903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 0x55bbe25c4e00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009fd0 tx=0x7f88c40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db0fe460f479183b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.434+0000 7f88c3400640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88b00a2b10 0x7f88b00a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.434+0000 7f88c3400640 1 -- 172.21.15.69:0/2682932186 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7f88bc0d2a70 con 0x7f88cc101790 2024-03-19T01:25:29.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.434+0000 7f88bbe00640 1 -- 172.21.15.69:0/2892526903 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4004d20 con 0x55bbe25c4a20 2024-03-19T01:25:29.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.434+0000 7f88bbe00640 1 -- 172.21.15.69:0/2892526903 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c4004050 con 0x55bbe25c4a20 2024-03-19T01:25:29.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.434+0000 7f88bbe00640 1 -- 172.21.15.69:0/2892526903 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4005260 con 0x55bbe25c4a20 2024-03-19T01:25:29.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.434+0000 7f88caa00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88b00a2b10 0x7f88b00a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.435+0000 7f88d487d740 1 -- 172.21.15.69:0/2892526903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 msgr2=0x55bbe25c4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.435+0000 7f88d487d740 1 --2- 172.21.15.69:0/2892526903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 0x55bbe25c4e00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009fd0 tx=0x7f88c40118d0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.435+0000 7f88d487d740 1 -- 172.21.15.69:0/2892526903 shutdown_connections 2024-03-19T01:25:29.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.435+0000 7f88d487d740 1 --2- 172.21.15.69:0/2892526903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.435+0000 7f88d487d740 1 --2- 172.21.15.69:0/2892526903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.435+0000 7f88d487d740 1 --2- 172.21.15.69:0/2892526903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 0x55bbe25c4e00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.435+0000 7f88d487d740 1 -- 172.21.15.69:0/2892526903 >> 172.21.15.69:0/2892526903 conn(0x55bbe25bf150 msgr2=0x55bbe25bf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.435+0000 7f88caa00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88b00a2b10 0x7f88b00a4fd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55bbe25c44f0 tx=0x7f88bc004ba0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.435+0000 7f88d487d740 1 -- 172.21.15.69:0/2892526903 shutdown_connections 2024-03-19T01:25:29.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.435+0000 7f88d487d740 1 -- 172.21.15.69:0/2892526903 wait complete. 2024-03-19T01:25:29.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.436+0000 7f88d487d740 1 -- start start 2024-03-19T01:25:29.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.436+0000 7f88d487d740 1 Processor -- start 2024-03-19T01:25:29.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88d487d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 0x55bbe25cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88d487d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88d487d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88d487d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bbe25d0d10 con 0x55bbe25c5340 2024-03-19T01:25:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88d487d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bbe24e5160 con 0x55bbe25c4a20 2024-03-19T01:25:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88d487d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55bbe25d0e90 con 0x55bbe25cdf20 2024-03-19T01:25:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88d487d740 10 client.0 _ll_register_callbacks cb 0x55bbe2521170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 0x55bbe25cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 0x55bbe25cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53968/0 (socket says 172.21.15.69:53968) 2024-03-19T01:25:29.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88cb400640 1 -- 172.21.15.69:0/822672019 learned_addr learned my addr 172.21.15.69:0/822672019 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.437+0000 7f88cb400640 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.438+0000 7f88c3e00640 1 -- 172.21.15.69:0/822672019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 msgr2=0x55bbe25cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.438+0000 7f88c3e00640 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.438+0000 7f88c3e00640 1 -- 172.21.15.69:0/822672019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 msgr2=0x55bbe25cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.438+0000 7f88c3e00640 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 0x55bbe25cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.438+0000 7f88c3e00640 1 -- 172.21.15.69:0/822672019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe23077d0 con 0x55bbe25c5340 2024-03-19T01:25:29.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.438+0000 7f88cb400640 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:25:29.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.438+0000 7f88c3e00640 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25cb540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f88b4000c00 tx=0x7f88b4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5479b7b0342fc7c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.438+0000 7f88b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:25:29.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.438+0000 7f88b8c00640 1 -- 172.21.15.69:0/822672019 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b402cab0 con 0x55bbe25c5340 2024-03-19T01:25:29.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.438+0000 7f88b8c00640 1 -- 172.21.15.69:0/822672019 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7f88b402cc50 con 0x55bbe25c5340 2024-03-19T01:25:29.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.438+0000 7f88d487d740 1 -- 172.21.15.69:0/822672019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c4004050 con 0x55bbe25c5340 2024-03-19T01:25:29.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.439+0000 7f88b8c00640 1 -- 172.21.15.69:0/822672019 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f88b4004f40 con 0x55bbe25c5340 2024-03-19T01:25:29.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.439+0000 7f88d487d740 10 client.17862 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:29.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.439+0000 7f88d487d740 1 -- 172.21.15.69:0/822672019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55bbe25dd070 con 0x55bbe25c5340 2024-03-19T01:25:29.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.439+0000 7f88b8c00640 1 -- 172.21.15.69:0/822672019 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c4004050 con 0x55bbe25c5340 2024-03-19T01:25:29.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.439+0000 7f88b8c00640 1 -- 172.21.15.69:0/822672019 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f88b4046070 con 0x55bbe25c5340 2024-03-19T01:25:29.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.439+0000 7f88d487d740 10 client.17862 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:29.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.439+0000 7f88d487d740 10 client.17862 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:25:29.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.439+0000 7f88d487d740 20 client.17862 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:29.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.439+0000 7f88d487d740 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55bbe25de480 0x55bbe25e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.439+0000 7f88d487d740 4 client.17862 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:25:29.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.439+0000 7f88d487d740 1 -- 172.21.15.69:0/822672019 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbe25368c0 con 0x55bbe25de480 2024-03-19T01:25:29.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.440+0000 7f88cb400640 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55bbe25de480 0x55bbe25e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.440+0000 7f88cb400640 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55bbe25de480 0x55bbe25e08d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10c0dc31886d4da5 server_cookie=a33b3779fc842be4 in_seq=0 out_seq=0 2024-03-19T01:25:29.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.440+0000 7f88b8c00640 10 client.17862 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:25:29.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.441+0000 7f88b8c00640 1 -- 172.21.15.69:0/822672019 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bbe25368c0 con 0x55bbe25de480 2024-03-19T01:25:29.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.441+0000 7f88b8c00640 10 client.17862 handle_command_reply: tid=0 2024-03-19T01:25:29.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.441+0000 7f88d487d740 1 client.17862 shutdown 2024-03-19T01:25:29.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.441+0000 7f88d487d740 1 -- 172.21.15.69:0/822672019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 msgr2=0x55bbe25cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.441+0000 7f88d487d740 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25cb540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f88b4000c00 tx=0x7f88b4004040 comp rx=0 tx=0).stop 2024-03-19T01:25:29.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.442+0000 7f88d487d740 1 -- 172.21.15.69:0/822672019 shutdown_connections 2024-03-19T01:25:29.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.442+0000 7f88d487d740 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55bbe25de480 0x55bbe25e08d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.442+0000 7f88d487d740 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.442+0000 7f88d487d740 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25cb540 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.442+0000 7f88b8c00640 0 client.17862 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:25:29.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.442+0000 7f88d487d740 1 --2- 172.21.15.69:0/822672019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c4a20 0x55bbe25cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.442+0000 7f88d487d740 1 -- 172.21.15.69:0/822672019 >> 172.21.15.69:0/822672019 conn(0x55bbe25bf150 msgr2=0x55bbe25c0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.442+0000 7f88d487d740 1 -- 172.21.15.69:0/822672019 shutdown_connections 2024-03-19T01:25:29.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.442+0000 7f88d487d740 1 -- 172.21.15.69:0/822672019 wait complete. 2024-03-19T01:25:29.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.442+0000 7f88d487d740 20 client.17862 trim_cache size 0 max 16384 2024-03-19T01:25:29.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.462+0000 7f88d487d740 1 Processor -- start 2024-03-19T01:25:29.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.462+0000 7f88d487d740 1 -- start start 2024-03-19T01:25:29.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.462+0000 7f88d487d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25c4a20 0x55bbe25cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.462+0000 7f88d487d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25d9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.462+0000 7f88d487d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25cdf20 0x55bbe25e61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.462+0000 7f88d487d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1acc60 con 0x55bbe25c5340 2024-03-19T01:25:29.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.462+0000 7f88d487d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1ae7a0 con 0x55bbe25cdf20 2024-03-19T01:25:29.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.462+0000 7f88d487d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1ad330 con 0x55bbe25c4a20 2024-03-19T01:25:29.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25d9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25cdf20 0x55bbe25e61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25cdf20 0x55bbe25e61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:53970/0 (socket says 172.21.15.69:53970) 2024-03-19T01:25:29.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88cb400640 1 -- 172.21.15.69:0/3148474864 learned_addr learned my addr 172.21.15.69:0/3148474864 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88c3e00640 1 --2- 172.21.15.69:0/3148474864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25d9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46480/0 (socket says 172.21.15.69:46480) 2024-03-19T01:25:29.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88cb400640 1 --2- 172.21.15.69:0/3148474864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25c4a20 0x55bbe25cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88c3e00640 1 -- 172.21.15.69:0/3148474864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25c4a20 msgr2=0x55bbe25cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88c3e00640 1 --2- 172.21.15.69:0/3148474864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25c4a20 0x55bbe25cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88c3e00640 1 -- 172.21.15.69:0/3148474864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25cdf20 msgr2=0x55bbe25e61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88c3e00640 1 --2- 172.21.15.69:0/3148474864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25cdf20 0x55bbe25e61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88c3e00640 1 -- 172.21.15.69:0/3148474864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe25d9ff0 con 0x55bbe25c5340 2024-03-19T01:25:29.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88cb400640 1 --2- 172.21.15.69:0/3148474864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25cdf20 0x55bbe25e61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:29.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.463+0000 7f88c3e00640 1 --2- 172.21.15.69:0/3148474864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25d9ab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f88b402c850 tx=0x7f88b402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc33aab4774d17a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.464+0000 7f88bb400640 1 -- 172.21.15.69:0/3148474864 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b40035f0 con 0x55bbe25c5340 2024-03-19T01:25:29.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.464+0000 7f88bb400640 1 -- 172.21.15.69:0/3148474864 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b4003790 con 0x55bbe25c5340 2024-03-19T01:25:29.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.464+0000 7f88bb400640 1 -- 172.21.15.69:0/3148474864 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4005340 con 0x55bbe25c5340 2024-03-19T01:25:29.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.464+0000 7f88d487d740 1 -- 172.21.15.69:0/3148474864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 msgr2=0x55bbe25d9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.464+0000 7f88d487d740 1 --2- 172.21.15.69:0/3148474864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25d9ab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f88b402c850 tx=0x7f88b402bc30 comp rx=0 tx=0).stop 2024-03-19T01:25:29.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.464+0000 7f88d487d740 1 -- 172.21.15.69:0/3148474864 shutdown_connections 2024-03-19T01:25:29.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.464+0000 7f88d487d740 1 --2- 172.21.15.69:0/3148474864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25cdf20 0x55bbe25e61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.464+0000 7f88d487d740 1 --2- 172.21.15.69:0/3148474864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c5340 0x55bbe25d9ab0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.464+0000 7f88d487d740 1 --2- 172.21.15.69:0/3148474864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25c4a20 0x55bbe25cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.464+0000 7f88d487d740 1 -- 172.21.15.69:0/3148474864 >> 172.21.15.69:0/3148474864 conn(0x55bbe25bf150 msgr2=0x55bbe25cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.464+0000 7f88d487d740 1 -- 172.21.15.69:0/3148474864 shutdown_connections 2024-03-19T01:25:29.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.465+0000 7f88d487d740 1 -- 172.21.15.69:0/3148474864 wait complete. 2024-03-19T01:25:29.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.465+0000 7f88d487d740 1 -- start start 2024-03-19T01:25:29.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.465+0000 7f88d487d740 1 Processor -- start 2024-03-19T01:25:29.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.465+0000 7f88d487d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c4a20 0x55bbe25d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.465+0000 7f88d487d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c5340 0x55bbe25d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.465+0000 7f88d487d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.465+0000 7f88d487d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1ae0d0 con 0x55bbe25c4a20 2024-03-19T01:25:29.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.465+0000 7f88d487d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1ada00 con 0x55bbe25c5340 2024-03-19T01:25:29.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.465+0000 7f88d487d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1aac40 con 0x55bbe25cdf20 2024-03-19T01:25:29.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.465+0000 7f88d487d740 10 client.0 _ll_register_callbacks cb 0x7f88b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:29.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88cb400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c4a20 0x55bbe25d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c5340 0x55bbe25d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88cb400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c4a20 0x55bbe25d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46486/0 (socket says 172.21.15.69:46486) 2024-03-19T01:25:29.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88cb400640 1 -- 172.21.15.69:0/981848662 learned_addr learned my addr 172.21.15.69:0/981848662 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88cb400640 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88c3e00640 1 -- 172.21.15.69:0/981848662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 msgr2=0x55bbe25d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88c3e00640 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88c3e00640 1 -- 172.21.15.69:0/981848662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c4a20 msgr2=0x55bbe25d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88c3e00640 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c4a20 0x55bbe25d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88c3e00640 1 -- 172.21.15.69:0/981848662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe2643a20 con 0x55bbe25c5340 2024-03-19T01:25:29.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88c3e00640 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c5340 0x55bbe25d2160 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x55bbe25e0ed0 tx=0x7f88b4005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ece94b65dd123fa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.466+0000 7f88b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:25:29.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88b8c00640 1 -- 172.21.15.69:0/981848662 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b402ccc0 con 0x55bbe25c5340 2024-03-19T01:25:29.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88b8c00640 1 -- 172.21.15.69:0/981848662 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7f88b40050e0 con 0x55bbe25c5340 2024-03-19T01:25:29.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88d487d740 1 -- 172.21.15.69:0/981848662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b4003790 con 0x55bbe25c5340 2024-03-19T01:25:29.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88b8c00640 1 -- 172.21.15.69:0/981848662 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f88b4067050 con 0x55bbe25c5340 2024-03-19T01:25:29.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88d487d740 10 client.28277 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:29.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88d487d740 1 -- 172.21.15.69:0/981848662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55bbe2644d00 con 0x55bbe25c5340 2024-03-19T01:25:29.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88b8c00640 1 -- 172.21.15.69:0/981848662 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b4003790 con 0x55bbe25c5340 2024-03-19T01:25:29.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88b8c00640 1 -- 172.21.15.69:0/981848662 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f88b4046040 con 0x55bbe25c5340 2024-03-19T01:25:29.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88d487d740 10 client.28277 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:29.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88d487d740 10 client.28277 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.d 2024-03-19T01:25:29.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88d487d740 20 client.28277 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:29.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88d487d740 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55bbe2645390 0x55bbe26477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.467+0000 7f88d487d740 4 client.28277 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:25:29.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.468+0000 7f88d487d740 1 -- 172.21.15.69:0/981848662 --> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbe2530c10 con 0x55bbe2645390 2024-03-19T01:25:29.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.468+0000 7f88cb400640 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55bbe2645390 0x55bbe26477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.469+0000 7f88cb400640 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55bbe2645390 0x55bbe26477b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7b59f3caafc468c server_cookie=cce3567c8efc1824 in_seq=0 out_seq=0 2024-03-19T01:25:29.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.469+0000 7f88b8c00640 10 client.28277 ms_handle_connect on v2:172.21.15.69:6836/2154698833 2024-03-19T01:25:29.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.469+0000 7f88b8c00640 1 -- 172.21.15.69:0/981848662 <== mds.4 v2:172.21.15.69:6836/2154698833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbe2530c10 con 0x55bbe2645390 2024-03-19T01:25:29.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.469+0000 7f88b8c00640 10 client.28277 handle_command_reply: tid=0 2024-03-19T01:25:29.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 1 client.28277 shutdown 2024-03-19T01:25:29.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 1 -- 172.21.15.69:0/981848662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c5340 msgr2=0x55bbe25d2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c5340 0x55bbe25d2160 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x55bbe25e0ed0 tx=0x7f88b4005e50 comp rx=0 tx=0).stop 2024-03-19T01:25:29.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 1 -- 172.21.15.69:0/981848662 shutdown_connections 2024-03-19T01:25:29.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.69:6836/2154698833,v1:172.21.15.69:6837/2154698833] conn(0x55bbe2645390 0x55bbe26477b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bbe25cdf20 0x55bbe25d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbe25c5340 0x55bbe25d2160 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88b8c00640 0 client.28277 ms_handle_reset on v2:172.21.15.69:6836/2154698833 2024-03-19T01:25:29.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 1 --2- 172.21.15.69:0/981848662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbe25c4a20 0x55bbe25d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 1 -- 172.21.15.69:0/981848662 >> 172.21.15.69:0/981848662 conn(0x55bbe25bf150 msgr2=0x55bbe25cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 1 -- 172.21.15.69:0/981848662 shutdown_connections 2024-03-19T01:25:29.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 1 -- 172.21.15.69:0/981848662 wait complete. 2024-03-19T01:25:29.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.470+0000 7f88d487d740 20 client.28277 trim_cache size 0 max 16384 2024-03-19T01:25:29.478 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:25:29.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 -- 172.21.15.69:0/2682932186 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88b00a2b10 msgr2=0x7f88b00a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88b00a2b10 0x7f88b00a4fd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55bbe25c44f0 tx=0x7f88bc004ba0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 -- 172.21.15.69:0/2682932186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88cc101790 msgr2=0x7f88cc1980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88cc101790 0x7f88cc1980e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009b50 tx=0x7f88bc00b690 comp rx=0 tx=0).stop 2024-03-19T01:25:29.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 -- 172.21.15.69:0/2682932186 shutdown_connections 2024-03-19T01:25:29.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88cc1077a0 0x7f88cc19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88b00a2b10 0x7f88b00a4fd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88cc102150 0x7f88cc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 --2- 172.21.15.69:0/2682932186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88cc101790 0x7f88cc1980e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 -- 172.21.15.69:0/2682932186 >> 172.21.15.69:0/2682932186 conn(0x7f88cc0fd280 msgr2=0x7f88cc0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 -- 172.21.15.69:0/2682932186 shutdown_connections 2024-03-19T01:25:29.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.471+0000 7f88d1c00640 1 -- 172.21.15.69:0/2682932186 wait complete. 2024-03-19T01:25:29.495 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-19T01:25:29.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.700+0000 7f1e89800640 1 -- 172.21.15.69:0/2715602294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84104380 msgr2=0x7f1e84110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.700+0000 7f1e89800640 1 --2- 172.21.15.69:0/2715602294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84104380 0x7f1e84110890 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009ca0 tx=0x7f1e7c02c100 comp rx=0 tx=0).stop 2024-03-19T01:25:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.701+0000 7f1e89800640 1 -- 172.21.15.69:0/2715602294 shutdown_connections 2024-03-19T01:25:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.701+0000 7f1e89800640 1 --2- 172.21.15.69:0/2715602294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84104380 0x7f1e84110890 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.701+0000 7f1e89800640 1 --2- 172.21.15.69:0/2715602294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e841039c0 0x7f1e84103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.701+0000 7f1e89800640 1 --2- 172.21.15.69:0/2715602294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e841099d0 0x7f1e84109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.701+0000 7f1e89800640 1 -- 172.21.15.69:0/2715602294 >> 172.21.15.69:0/2715602294 conn(0x7f1e840ff490 msgr2=0x7f1e841018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.701+0000 7f1e89800640 1 -- 172.21.15.69:0/2715602294 shutdown_connections 2024-03-19T01:25:29.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.701+0000 7f1e89800640 1 -- 172.21.15.69:0/2715602294 wait complete. 2024-03-19T01:25:29.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.702+0000 7f1e89800640 1 Processor -- start 2024-03-19T01:25:29.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.703+0000 7f1e89800640 1 -- start start 2024-03-19T01:25:29.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.703+0000 7f1e89800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e841039c0 0x7f1e8419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.703+0000 7f1e89800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84104380 0x7f1e8419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.703+0000 7f1e89800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e841099d0 0x7f1e8419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.703+0000 7f1e89800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84113dd0 con 0x7f1e841039c0 2024-03-19T01:25:29.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.703+0000 7f1e89800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84113c50 con 0x7f1e84104380 2024-03-19T01:25:29.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.703+0000 7f1e89800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84113f50 con 0x7f1e841099d0 2024-03-19T01:25:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.703+0000 7f1e82000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84104380 0x7f1e8419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.703+0000 7f1e83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e841099d0 0x7f1e8419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.703+0000 7f1e82a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e841039c0 0x7f1e8419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.703+0000 7f1e82000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84104380 0x7f1e8419a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54000/0 (socket says 172.21.15.69:54000) 2024-03-19T01:25:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.704+0000 7f1e82000640 1 -- 172.21.15.69:0/1574034305 learned_addr learned my addr 172.21.15.69:0/1574034305 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.704+0000 7f1e82000640 1 -- 172.21.15.69:0/1574034305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e841099d0 msgr2=0x7f1e8419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.704+0000 7f1e82000640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e841099d0 0x7f1e8419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.704+0000 7f1e82000640 1 -- 172.21.15.69:0/1574034305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e841039c0 msgr2=0x7f1e8419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.704+0000 7f1e82000640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e841039c0 0x7f1e8419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.704+0000 7f1e82000640 1 -- 172.21.15.69:0/1574034305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8419f250 con 0x7f1e84104380 2024-03-19T01:25:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.704+0000 7f1e83400640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e841099d0 0x7f1e8419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:29.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.704+0000 7f1e82000640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84104380 0x7f1e8419a7f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000e9f0 tx=0x7f1e7000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.704+0000 7f1e7b400640 1 -- 172.21.15.69:0/1574034305 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000cd80 con 0x7f1e84104380 2024-03-19T01:25:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.705+0000 7f1e7b400640 1 -- 172.21.15.69:0/1574034305 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e70010910 con 0x7f1e84104380 2024-03-19T01:25:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.705+0000 7f1e89800640 1 -- 172.21.15.69:0/1574034305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8419f540 con 0x7f1e84104380 2024-03-19T01:25:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.705+0000 7f1e7b400640 1 -- 172.21.15.69:0/1574034305 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70018990 con 0x7f1e84104380 2024-03-19T01:25:29.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.705+0000 7f1e89800640 1 -- 172.21.15.69:0/1574034305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e841a6dd0 con 0x7f1e84104380 2024-03-19T01:25:29.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.709+0000 7f1e7b400640 1 -- 172.21.15.69:0/1574034305 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f1e70010ac0 con 0x7f1e84104380 2024-03-19T01:25:29.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.709+0000 7f1e8c575740 1 Processor -- start 2024-03-19T01:25:29.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.710+0000 7f1e8c575740 1 -- start start 2024-03-19T01:25:29.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.710+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a031ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.710+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 0x55f6a03239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.710+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a0323f20 0x55f6a0326310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.710+0000 7f1e8c575740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f6a027f380 con 0x55f6a031b340 2024-03-19T01:25:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.710+0000 7f1e8c575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f6a0227d20 con 0x55f6a0323f20 2024-03-19T01:25:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.710+0000 7f1e8c575740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f6a027b420 con 0x55f6a031aa20 2024-03-19T01:25:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.710+0000 7f1e83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a031ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.710+0000 7f1e82a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 0x55f6a03239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e82a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 0x55f6a03239e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46534/0 (socket says 172.21.15.69:46534) 2024-03-19T01:25:29.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e82a00640 1 -- 172.21.15.69:0/2495198015 learned_addr learned my addr 172.21.15.69:0/2495198015 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.710+0000 7f1e83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a031ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33254/0 (socket says 172.21.15.69:33254) 2024-03-19T01:25:29.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e82000640 1 --2- 172.21.15.69:0/2495198015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a0323f20 0x55f6a0326310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e83400640 1 -- 172.21.15.69:0/2495198015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a0323f20 msgr2=0x55f6a0326310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e83400640 1 --2- 172.21.15.69:0/2495198015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a0323f20 0x55f6a0326310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e83400640 1 -- 172.21.15.69:0/2495198015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 msgr2=0x55f6a03239e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e83400640 1 --2- 172.21.15.69:0/2495198015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 0x55f6a03239e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e83400640 1 -- 172.21.15.69:0/2495198015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f69ff933f0 con 0x55f6a031aa20 2024-03-19T01:25:29.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e7b400640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e680a29f0 0x7f1e680a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e83400640 1 --2- 172.21.15.69:0/2495198015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a031ae00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009bf0 tx=0x7f1e7c005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e3486135bcc85d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e7b400640 1 -- 172.21.15.69:0/1574034305 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7f1e700cf550 con 0x7f1e84104380 2024-03-19T01:25:29.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e6fe00640 1 -- 172.21.15.69:0/2495198015 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c045020 con 0x55f6a031aa20 2024-03-19T01:25:29.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.711+0000 7f1e82a00640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e680a29f0 0x7f1e680a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.712+0000 7f1e6fe00640 1 -- 172.21.15.69:0/2495198015 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7c03a040 con 0x55f6a031aa20 2024-03-19T01:25:29.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.712+0000 7f1e6fe00640 1 -- 172.21.15.69:0/2495198015 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c039040 con 0x55f6a031aa20 2024-03-19T01:25:29.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.712+0000 7f1e8c575740 1 -- 172.21.15.69:0/2495198015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 msgr2=0x55f6a031ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.712+0000 7f1e8c575740 1 --2- 172.21.15.69:0/2495198015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a031ae00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009bf0 tx=0x7f1e7c005fa0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.712+0000 7f1e82a00640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e680a29f0 0x7f1e680a4eb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55f6a031a4f0 tx=0x7f1e74005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.712+0000 7f1e8c575740 1 -- 172.21.15.69:0/2495198015 shutdown_connections 2024-03-19T01:25:29.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.712+0000 7f1e8c575740 1 --2- 172.21.15.69:0/2495198015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a0323f20 0x55f6a0326310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.712+0000 7f1e8c575740 1 --2- 172.21.15.69:0/2495198015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 0x55f6a03239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.712+0000 7f1e8c575740 1 --2- 172.21.15.69:0/2495198015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a031ae00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.713+0000 7f1e8c575740 1 -- 172.21.15.69:0/2495198015 >> 172.21.15.69:0/2495198015 conn(0x55f6a0315150 msgr2=0x55f6a0315530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.713+0000 7f1e8c575740 1 -- 172.21.15.69:0/2495198015 shutdown_connections 2024-03-19T01:25:29.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.713+0000 7f1e8c575740 1 -- 172.21.15.69:0/2495198015 wait complete. 2024-03-19T01:25:29.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.713+0000 7f1e8c575740 1 -- start start 2024-03-19T01:25:29.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.713+0000 7f1e8c575740 1 Processor -- start 2024-03-19T01:25:29.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031aa20 0x55f6a0321010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031b340 0x55f6a0321550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a0323f20 0x55f6a0321a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e8c575740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f6a0326cb0 con 0x55f6a0323f20 2024-03-19T01:25:29.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e8c575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f6a023b160 con 0x55f6a031aa20 2024-03-19T01:25:29.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e8c575740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f6a0326e30 con 0x55f6a031b340 2024-03-19T01:25:29.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e8c575740 10 client.0 _ll_register_callbacks cb 0x55f6a0277170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:29.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031aa20 0x55f6a0321010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031aa20 0x55f6a0321010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54022/0 (socket says 172.21.15.69:54022) 2024-03-19T01:25:29.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e82a00640 1 -- 172.21.15.69:0/1568527470 learned_addr learned my addr 172.21.15.69:0/1568527470 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e82000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031b340 0x55f6a0321550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e83400640 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a0323f20 0x55f6a0321a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.714+0000 7f1e82000640 1 -- 172.21.15.69:0/1568527470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031aa20 msgr2=0x55f6a0321010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e82000640 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031aa20 0x55f6a0321010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e82000640 1 -- 172.21.15.69:0/1568527470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a0323f20 msgr2=0x55f6a0321a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e82000640 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a0323f20 0x55f6a0321a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e82000640 1 -- 172.21.15.69:0/1568527470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a005d7d0 con 0x55f6a031b340 2024-03-19T01:25:29.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e82a00640 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031aa20 0x55f6a0321010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:29.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e82000640 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031b340 0x55f6a0321550 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1e700cc850 tx=0x7f1e7009b6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae992746d77a916 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:25:29.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e6cc00640 1 -- 172.21.15.69:0/1568527470 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e700cccd0 con 0x55f6a031b340 2024-03-19T01:25:29.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e6cc00640 1 -- 172.21.15.69:0/1568527470 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7f1e700cf170 con 0x55f6a031b340 2024-03-19T01:25:29.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e8c575740 1 -- 172.21.15.69:0/1568527470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e7c03a040 con 0x55f6a031b340 2024-03-19T01:25:29.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e6cc00640 1 -- 172.21.15.69:0/1568527470 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f1e700cf370 con 0x55f6a031b340 2024-03-19T01:25:29.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e8c575740 10 client.28204 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:29.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e8c575740 1 -- 172.21.15.69:0/1568527470 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55f6a0333060 con 0x55f6a031b340 2024-03-19T01:25:29.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.715+0000 7f1e6cc00640 1 -- 172.21.15.69:0/1568527470 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7c03a040 con 0x55f6a031b340 2024-03-19T01:25:29.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.716+0000 7f1e6cc00640 1 -- 172.21.15.69:0/1568527470 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f1e70018b30 con 0x55f6a031b340 2024-03-19T01:25:29.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.716+0000 7f1e8c575740 10 client.28204 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:29.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.716+0000 7f1e8c575740 10 client.28204 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:25:29.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.716+0000 7f1e8c575740 20 client.28204 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:29.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.716+0000 7f1e8c575740 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f6a0334520 0x55f6a0336910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.716+0000 7f1e8c575740 4 client.28204 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:25:29.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.716+0000 7f1e8c575740 1 -- 172.21.15.69:0/1568527470 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6a028c8c0 con 0x55f6a0334520 2024-03-19T01:25:29.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.716+0000 7f1e82a00640 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f6a0334520 0x55f6a0336910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.717+0000 7f1e82a00640 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f6a0334520 0x55f6a0336910 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e27b5d1e98554e0 server_cookie=bdca9c6c8a9fce61 in_seq=0 out_seq=0 2024-03-19T01:25:29.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.717+0000 7f1e6cc00640 10 client.28204 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:25:29.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e6cc00640 1 -- 172.21.15.69:0/1568527470 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f6a028c8c0 con 0x55f6a0334520 2024-03-19T01:25:29.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e6cc00640 10 client.28204 handle_command_reply: tid=0 2024-03-19T01:25:29.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 1 client.28204 shutdown 2024-03-19T01:25:29.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 1 -- 172.21.15.69:0/1568527470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031b340 msgr2=0x55f6a0321550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031b340 0x55f6a0321550 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1e700cc850 tx=0x7f1e7009b6a0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 1 -- 172.21.15.69:0/1568527470 shutdown_connections 2024-03-19T01:25:29.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a0323f20 0x55f6a0321a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031b340 0x55f6a0321550 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f6a0334520 0x55f6a0336910 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 1 --2- 172.21.15.69:0/1568527470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031aa20 0x55f6a0321010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e6cc00640 0 client.28204 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:25:29.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 1 -- 172.21.15.69:0/1568527470 >> 172.21.15.69:0/1568527470 conn(0x55f6a0315150 msgr2=0x55f6a0316ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 1 -- 172.21.15.69:0/1568527470 shutdown_connections 2024-03-19T01:25:29.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 1 -- 172.21.15.69:0/1568527470 wait complete. 2024-03-19T01:25:29.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.718+0000 7f1e8c575740 20 client.28204 trim_cache size 0 max 16384 2024-03-19T01:25:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e8c575740 1 Processor -- start 2024-03-19T01:25:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e8c575740 1 -- start start 2024-03-19T01:25:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a0323740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 0x55f6a032fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a0323f20 0x55f6a0316c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e8c575740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841b1000 con 0x55f6a031b340 2024-03-19T01:25:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e8c575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841b2b40 con 0x55f6a0323f20 2024-03-19T01:25:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e8c575740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841b16d0 con 0x55f6a031aa20 2024-03-19T01:25:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e82a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a0323740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e82a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a0323740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33276/0 (socket says 172.21.15.69:33276) 2024-03-19T01:25:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e82a00640 1 -- 172.21.15.69:0/3881530269 learned_addr learned my addr 172.21.15.69:0/3881530269 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e82000640 1 --2- 172.21.15.69:0/3881530269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 0x55f6a032fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.733+0000 7f1e83400640 1 --2- 172.21.15.69:0/3881530269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a0323f20 0x55f6a0316c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.734+0000 7f1e82a00640 1 -- 172.21.15.69:0/3881530269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a0323f20 msgr2=0x55f6a0316c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.734+0000 7f1e82a00640 1 --2- 172.21.15.69:0/3881530269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a0323f20 0x55f6a0316c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.734+0000 7f1e82a00640 1 -- 172.21.15.69:0/3881530269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 msgr2=0x55f6a032fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.734+0000 7f1e82a00640 1 --2- 172.21.15.69:0/3881530269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 0x55f6a032fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.734+0000 7f1e82a00640 1 -- 172.21.15.69:0/3881530269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6a032ff90 con 0x55f6a031aa20 2024-03-19T01:25:29.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.734+0000 7f1e83400640 1 --2- 172.21.15.69:0/3881530269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a0323f20 0x55f6a0316c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.734+0000 7f1e82000640 1 --2- 172.21.15.69:0/3881530269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 0x55f6a032fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.734+0000 7f1e82a00640 1 --2- 172.21.15.69:0/3881530269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a0323740 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55f6a03370e0 tx=0x7f1e7401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=425e08b2cda325c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.734+0000 7f1e6f400640 1 -- 172.21.15.69:0/3881530269 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7402b4e0 con 0x55f6a031aa20 2024-03-19T01:25:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.735+0000 7f1e6f400640 1 -- 172.21.15.69:0/3881530269 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e74030070 con 0x55f6a031aa20 2024-03-19T01:25:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.735+0000 7f1e6f400640 1 -- 172.21.15.69:0/3881530269 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7401e680 con 0x55f6a031aa20 2024-03-19T01:25:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.735+0000 7f1e8c575740 1 -- 172.21.15.69:0/3881530269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 msgr2=0x55f6a0323740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.735+0000 7f1e8c575740 1 --2- 172.21.15.69:0/3881530269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a0323740 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55f6a03370e0 tx=0x7f1e7401b000 comp rx=0 tx=0).stop 2024-03-19T01:25:29.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.735+0000 7f1e8c575740 1 -- 172.21.15.69:0/3881530269 shutdown_connections 2024-03-19T01:25:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.735+0000 7f1e8c575740 1 --2- 172.21.15.69:0/3881530269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a0323f20 0x55f6a0316c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.735+0000 7f1e8c575740 1 --2- 172.21.15.69:0/3881530269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031b340 0x55f6a032fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.735+0000 7f1e8c575740 1 --2- 172.21.15.69:0/3881530269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a031aa20 0x55f6a0323740 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.735+0000 7f1e8c575740 1 -- 172.21.15.69:0/3881530269 >> 172.21.15.69:0/3881530269 conn(0x55f6a0315150 msgr2=0x55f6a0316530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.735+0000 7f1e8c575740 1 -- 172.21.15.69:0/3881530269 shutdown_connections 2024-03-19T01:25:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.735+0000 7f1e8c575740 1 -- 172.21.15.69:0/3881530269 wait complete. 2024-03-19T01:25:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.736+0000 7f1e8c575740 1 -- start start 2024-03-19T01:25:29.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.736+0000 7f1e8c575740 1 Processor -- start 2024-03-19T01:25:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.736+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031aa20 0x55f6a031e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.736+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031b340 0x55f6a031e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.736+0000 7f1e8c575740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a0323f20 0x55f6a03997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.736+0000 7f1e8c575740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841b2470 con 0x55f6a031aa20 2024-03-19T01:25:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.736+0000 7f1e8c575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841b1da0 con 0x55f6a031b340 2024-03-19T01:25:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.736+0000 7f1e8c575740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841aefe0 con 0x55f6a0323f20 2024-03-19T01:25:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.736+0000 7f1e8c575740 10 client.0 _ll_register_callbacks cb 0x7f1e7000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e82000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031b340 0x55f6a031e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a0323f20 0x55f6a03997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e83400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a0323f20 0x55f6a03997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33278/0 (socket says 172.21.15.69:33278) 2024-03-19T01:25:29.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e83400640 1 -- 172.21.15.69:0/3754966109 learned_addr learned my addr 172.21.15.69:0/3754966109 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e82000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031b340 0x55f6a031e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54046/0 (socket says 172.21.15.69:54046) 2024-03-19T01:25:29.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e82a00640 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031aa20 0x55f6a031e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e83400640 1 -- 172.21.15.69:0/3754966109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031b340 msgr2=0x55f6a031e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e83400640 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031b340 0x55f6a031e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e83400640 1 -- 172.21.15.69:0/3754966109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031aa20 msgr2=0x55f6a031e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e83400640 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031aa20 0x55f6a031e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e83400640 1 -- 172.21.15.69:0/3754966109 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a0399ce0 con 0x55f6a0323f20 2024-03-19T01:25:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e82000640 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031b340 0x55f6a031e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e83400640 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a0323f20 0x55f6a03997a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009870 tx=0x7f1e7c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6b82ab42f19d05b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.737+0000 7f1e6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:25:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e6cc00640 1 -- 172.21.15.69:0/3754966109 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c045040 con 0x55f6a0323f20 2024-03-19T01:25:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e6cc00640 1 -- 172.21.15.69:0/3754966109 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7f1e7c032790 con 0x55f6a0323f20 2024-03-19T01:25:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e8c575740 1 -- 172.21.15.69:0/3754966109 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e74030070 con 0x55f6a0323f20 2024-03-19T01:25:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e6cc00640 1 -- 172.21.15.69:0/3754966109 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f1e7c039040 con 0x55f6a0323f20 2024-03-19T01:25:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e8c575740 10 client.28210 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e8c575740 1 -- 172.21.15.69:0/3754966109 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55f6a039af10 con 0x55f6a0323f20 2024-03-19T01:25:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e6cc00640 1 -- 172.21.15.69:0/3754966109 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e74030070 con 0x55f6a0323f20 2024-03-19T01:25:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e6cc00640 1 -- 172.21.15.69:0/3754966109 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f1e7c003c70 con 0x55f6a0323f20 2024-03-19T01:25:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e8c575740 10 client.28210 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e8c575740 10 client.28210 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-03-19T01:25:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e8c575740 20 client.28210 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e8c575740 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f6a039b2a0 0x55f6a039d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e8c575740 4 client.28210 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:25:29.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.738+0000 7f1e8c575740 1 -- 172.21.15.69:0/3754966109 --> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6a0286c10 con 0x55f6a039b2a0 2024-03-19T01:25:29.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.739+0000 7f1e82a00640 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f6a039b2a0 0x55f6a039d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.739+0000 7f1e82a00640 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f6a039b2a0 0x55f6a039d6c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8abbe99342da1757 server_cookie=609d51446e11ba0 in_seq=0 out_seq=0 2024-03-19T01:25:29.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.739+0000 7f1e6cc00640 10 client.28210 ms_handle_connect on v2:172.21.15.86:6834/786469397 2024-03-19T01:25:29.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.739+0000 7f1e6cc00640 1 -- 172.21.15.69:0/3754966109 <== mds.3 v2:172.21.15.86:6834/786469397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6a0286c10 con 0x55f6a039b2a0 2024-03-19T01:25:29.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.739+0000 7f1e6cc00640 10 client.28210 handle_command_reply: tid=0 2024-03-19T01:25:29.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.739+0000 7f1e8c575740 1 client.28210 shutdown 2024-03-19T01:25:29.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.739+0000 7f1e8c575740 1 -- 172.21.15.69:0/3754966109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a0323f20 msgr2=0x55f6a03997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.739+0000 7f1e8c575740 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a0323f20 0x55f6a03997a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009870 tx=0x7f1e7c005bd0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.740+0000 7f1e8c575740 1 -- 172.21.15.69:0/3754966109 shutdown_connections 2024-03-19T01:25:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.740+0000 7f1e8c575740 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f6a0323f20 0x55f6a03997a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.740+0000 7f1e8c575740 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.86:6834/786469397,v1:172.21.15.86:6835/786469397] conn(0x55f6a039b2a0 0x55f6a039d6c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.740+0000 7f1e8c575740 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6a031b340 0x55f6a031e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.740+0000 7f1e8c575740 1 --2- 172.21.15.69:0/3754966109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f6a031aa20 0x55f6a031e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.740+0000 7f1e6cc00640 0 client.28210 ms_handle_reset on v2:172.21.15.86:6834/786469397 2024-03-19T01:25:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.740+0000 7f1e8c575740 1 -- 172.21.15.69:0/3754966109 >> 172.21.15.69:0/3754966109 conn(0x55f6a0315150 msgr2=0x55f6a03141d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.740+0000 7f1e8c575740 1 -- 172.21.15.69:0/3754966109 shutdown_connections 2024-03-19T01:25:29.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.740+0000 7f1e8c575740 1 -- 172.21.15.69:0/3754966109 wait complete. 2024-03-19T01:25:29.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.740+0000 7f1e8c575740 20 client.28210 trim_cache size 0 max 16384 2024-03-19T01:25:29.750 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:25:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 -- 172.21.15.69:0/1574034305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e680a29f0 msgr2=0x7f1e680a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e680a29f0 0x7f1e680a4eb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55f6a031a4f0 tx=0x7f1e74005f90 comp rx=0 tx=0).stop 2024-03-19T01:25:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 -- 172.21.15.69:0/1574034305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84104380 msgr2=0x7f1e8419a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84104380 0x7f1e8419a7f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000e9f0 tx=0x7f1e7000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 -- 172.21.15.69:0/1574034305 shutdown_connections 2024-03-19T01:25:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e841099d0 0x7f1e8419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1e680a29f0 0x7f1e680a4eb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e84104380 0x7f1e8419a7f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 --2- 172.21.15.69:0/1574034305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e841039c0 0x7f1e8419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 -- 172.21.15.69:0/1574034305 >> 172.21.15.69:0/1574034305 conn(0x7f1e840ff490 msgr2=0x7f1e840ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 -- 172.21.15.69:0/1574034305 shutdown_connections 2024-03-19T01:25:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.741+0000 7f1e89800640 1 -- 172.21.15.69:0/1574034305 wait complete. 2024-03-19T01:25:29.759 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-19T01:25:29.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.959+0000 7ff21f600640 1 -- 172.21.15.69:0/143040428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2181099d0 msgr2=0x7ff218109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.959+0000 7ff21f600640 1 --2- 172.21.15.69:0/143040428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2181099d0 0x7ff218109db0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff208009860 tx=0x7ff20802bb70 comp rx=0 tx=0).stop 2024-03-19T01:25:29.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.959+0000 7ff21f600640 1 -- 172.21.15.69:0/143040428 shutdown_connections 2024-03-19T01:25:29.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.959+0000 7ff21f600640 1 --2- 172.21.15.69:0/143040428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff218104380 0x7ff218110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.959+0000 7ff21f600640 1 --2- 172.21.15.69:0/143040428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181039c0 0x7ff218103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.959+0000 7ff21f600640 1 --2- 172.21.15.69:0/143040428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2181099d0 0x7ff218109db0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.959+0000 7ff21f600640 1 -- 172.21.15.69:0/143040428 >> 172.21.15.69:0/143040428 conn(0x7ff2180ff490 msgr2=0x7ff2181018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.959+0000 7ff21f600640 1 -- 172.21.15.69:0/143040428 shutdown_connections 2024-03-19T01:25:29.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.959+0000 7ff21f600640 1 -- 172.21.15.69:0/143040428 wait complete. 2024-03-19T01:25:29.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.961+0000 7ff21f600640 1 Processor -- start 2024-03-19T01:25:29.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.961+0000 7ff21f600640 1 -- start start 2024-03-19T01:25:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.961+0000 7ff21f600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181039c0 0x7ff21819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.961+0000 7ff21f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff218104380 0x7ff21819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.961+0000 7ff21f600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2181099d0 0x7ff21819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.961+0000 7ff21f600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff218113db0 con 0x7ff2181039c0 2024-03-19T01:25:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff21f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff218113c30 con 0x7ff218104380 2024-03-19T01:25:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff21f600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff218113f30 con 0x7ff2181099d0 2024-03-19T01:25:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff20fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff218104380 0x7ff21819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff21cc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181039c0 0x7ff21819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff20fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff218104380 0x7ff21819a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54060/0 (socket says 172.21.15.69:54060) 2024-03-19T01:25:29.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff21cc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181039c0 0x7ff21819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46576/0 (socket says 172.21.15.69:46576) 2024-03-19T01:25:29.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff20fe00640 1 -- 172.21.15.69:0/2426231816 learned_addr learned my addr 172.21.15.69:0/2426231816 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff21d600640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2181099d0 0x7ff21819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff20fe00640 1 -- 172.21.15.69:0/2426231816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2181099d0 msgr2=0x7ff21819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff20fe00640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2181099d0 0x7ff21819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff20fe00640 1 -- 172.21.15.69:0/2426231816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181039c0 msgr2=0x7ff21819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff20fe00640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181039c0 0x7ff21819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff20fe00640 1 -- 172.21.15.69:0/2426231816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21819f300 con 0x7ff218104380 2024-03-19T01:25:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff21d600640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2181099d0 0x7ff21819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.962+0000 7ff21cc00640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181039c0 0x7ff21819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.963+0000 7ff20fe00640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff218104380 0x7ff21819a850 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff20000e9f0 tx=0x7ff20000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.963+0000 7ff216000640 1 -- 172.21.15.69:0/2426231816 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20000cd80 con 0x7ff218104380 2024-03-19T01:25:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.963+0000 7ff216000640 1 -- 172.21.15.69:0/2426231816 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff200010910 con 0x7ff218104380 2024-03-19T01:25:29.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.963+0000 7ff216000640 1 -- 172.21.15.69:0/2426231816 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200018990 con 0x7ff218104380 2024-03-19T01:25:29.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.963+0000 7ff21f600640 1 -- 172.21.15.69:0/2426231816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21819f5f0 con 0x7ff218104380 2024-03-19T01:25:29.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.963+0000 7ff21f600640 1 -- 172.21.15.69:0/2426231816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2181a6e80 con 0x7ff218104380 2024-03-19T01:25:29.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.967+0000 7ff216000640 1 -- 172.21.15.69:0/2426231816 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ff200010ac0 con 0x7ff218104380 2024-03-19T01:25:29.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.967+0000 7ff2223b3740 1 Processor -- start 2024-03-19T01:25:29.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.968+0000 7ff2223b3740 1 -- start start 2024-03-19T01:25:29.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.968+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1da20 0x5584b5b1de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.968+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 0x5584b5b269e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.968+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b29310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.968+0000 7ff2223b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5584b5a82380 con 0x5584b5b1da20 2024-03-19T01:25:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.968+0000 7ff2223b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5584b5a2ad20 con 0x5584b5b26f20 2024-03-19T01:25:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.968+0000 7ff2223b3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5584b5a7e420 con 0x5584b5b1e340 2024-03-19T01:25:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.968+0000 7ff21d600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1da20 0x5584b5b1de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.968+0000 7ff21cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 0x5584b5b269e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.968+0000 7ff21cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 0x5584b5b269e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33302/0 (socket says 172.21.15.69:33302) 2024-03-19T01:25:29.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.968+0000 7ff21cc00640 1 -- 172.21.15.69:0/2392888191 learned_addr learned my addr 172.21.15.69:0/2392888191 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.969+0000 7ff20fe00640 1 --2- 172.21.15.69:0/2392888191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b29310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.969+0000 7ff21cc00640 1 -- 172.21.15.69:0/2392888191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 msgr2=0x5584b5b29310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.969+0000 7ff21cc00640 1 --2- 172.21.15.69:0/2392888191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b29310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.969+0000 7ff21cc00640 1 -- 172.21.15.69:0/2392888191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1da20 msgr2=0x5584b5b1de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.969+0000 7ff21cc00640 1 --2- 172.21.15.69:0/2392888191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1da20 0x5584b5b1de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.969+0000 7ff21cc00640 1 -- 172.21.15.69:0/2392888191 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584b57963f0 con 0x5584b5b1e340 2024-03-19T01:25:29.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.969+0000 7ff21cc00640 1 --2- 172.21.15.69:0/2392888191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 0x5584b5b269e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff20802c0b0 tx=0x7ff2080058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ce5725743d84046 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.969+0000 7ff216000640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1fc0a29f0 0x7ff1fc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.969+0000 7ff20dc00640 1 -- 172.21.15.69:0/2392888191 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208039040 con 0x5584b5b1e340 2024-03-19T01:25:29.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.970+0000 7ff20dc00640 1 -- 172.21.15.69:0/2392888191 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff208004230 con 0x5584b5b1e340 2024-03-19T01:25:29.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.970+0000 7ff20dc00640 1 -- 172.21.15.69:0/2392888191 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20802c9e0 con 0x5584b5b1e340 2024-03-19T01:25:29.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.970+0000 7ff216000640 1 -- 172.21.15.69:0/2426231816 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7ff2000cf550 con 0x7ff218104380 2024-03-19T01:25:29.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.970+0000 7ff21d600640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1fc0a29f0 0x7ff1fc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.970+0000 7ff2223b3740 1 -- 172.21.15.69:0/2392888191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 msgr2=0x5584b5b269e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.970+0000 7ff2223b3740 1 --2- 172.21.15.69:0/2392888191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 0x5584b5b269e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff20802c0b0 tx=0x7ff2080058d0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.970+0000 7ff2223b3740 1 -- 172.21.15.69:0/2392888191 shutdown_connections 2024-03-19T01:25:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.970+0000 7ff2223b3740 1 --2- 172.21.15.69:0/2392888191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b29310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.970+0000 7ff2223b3740 1 --2- 172.21.15.69:0/2392888191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 0x5584b5b269e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.970+0000 7ff2223b3740 1 --2- 172.21.15.69:0/2392888191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1da20 0x5584b5b1de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.970+0000 7ff2223b3740 1 -- 172.21.15.69:0/2392888191 >> 172.21.15.69:0/2392888191 conn(0x5584b5b18150 msgr2=0x5584b5b18530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.971+0000 7ff21d600640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1fc0a29f0 0x7ff1fc0a4eb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff210005fd0 tx=0x7ff2100095c0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.971+0000 7ff2223b3740 1 -- 172.21.15.69:0/2392888191 shutdown_connections 2024-03-19T01:25:29.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.971+0000 7ff2223b3740 1 -- 172.21.15.69:0/2392888191 wait complete. 2024-03-19T01:25:29.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.971+0000 7ff2223b3740 1 -- start start 2024-03-19T01:25:29.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.972+0000 7ff2223b3740 1 Processor -- start 2024-03-19T01:25:29.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.972+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1da20 0x5584b5b23ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.972+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 0x5584b5b24400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.972+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b24940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.972+0000 7ff2223b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5584b5b29cb0 con 0x5584b5b1da20 2024-03-19T01:25:29.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.972+0000 7ff2223b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5584b5a3e160 con 0x5584b5b26f20 2024-03-19T01:25:29.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.972+0000 7ff2223b3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5584b5b29e30 con 0x5584b5b1e340 2024-03-19T01:25:29.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.972+0000 7ff2223b3740 10 client.0 _ll_register_callbacks cb 0x5584b5a7a170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:29.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff21d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b24940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff21d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b24940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54086/0 (socket says 172.21.15.69:54086) 2024-03-19T01:25:29.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff21d600640 1 -- 172.21.15.69:0/2583777980 learned_addr learned my addr 172.21.15.69:0/2583777980 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff20fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 0x5584b5b24400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff21cc00640 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1da20 0x5584b5b23ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff20fe00640 1 -- 172.21.15.69:0/2583777980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 msgr2=0x5584b5b24940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff20fe00640 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b24940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff20fe00640 1 -- 172.21.15.69:0/2583777980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1da20 msgr2=0x5584b5b23ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff20fe00640 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1da20 0x5584b5b23ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff20fe00640 1 -- 172.21.15.69:0/2583777980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584b58607d0 con 0x5584b5b1e340 2024-03-19T01:25:29.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff21cc00640 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1da20 0x5584b5b23ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:25:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff20fe00640 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 0x5584b5b24400 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff2000cc850 tx=0x7ff200009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc106a09633c078 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff206000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:25:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.973+0000 7ff206000640 1 -- 172.21.15.69:0/2583777980 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20009b170 con 0x5584b5b1e340 2024-03-19T01:25:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff206000640 1 -- 172.21.15.69:0/2583777980 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7ff2000cf150 con 0x5584b5b1e340 2024-03-19T01:25:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff2223b3740 1 -- 172.21.15.69:0/2583777980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff208004230 con 0x5584b5b1e340 2024-03-19T01:25:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff206000640 1 -- 172.21.15.69:0/2583777980 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff2000cccd0 con 0x5584b5b1e340 2024-03-19T01:25:29.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff2223b3740 10 client.28222 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff2223b3740 1 -- 172.21.15.69:0/2583777980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5584b5b35f40 con 0x5584b5b1e340 2024-03-19T01:25:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff206000640 1 -- 172.21.15.69:0/2583777980 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff208004230 con 0x5584b5b1e340 2024-03-19T01:25:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff206000640 1 -- 172.21.15.69:0/2583777980 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7ff2000afbd0 con 0x5584b5b1e340 2024-03-19T01:25:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff2223b3740 10 client.28222 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff2223b3740 10 client.28222 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:25:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff2223b3740 20 client.28222 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff2223b3740 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x5584b5b37290 0x5584b5b39700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff2223b3740 4 client.28222 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:25:29.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.974+0000 7ff2223b3740 1 -- 172.21.15.69:0/2583777980 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584b5a8f8c0 con 0x5584b5b37290 2024-03-19T01:25:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.975+0000 7ff21cc00640 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x5584b5b37290 0x5584b5b39700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.975+0000 7ff21cc00640 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x5584b5b37290 0x5584b5b39700 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ced4cce8ae6a0fc server_cookie=3366ef098813ad45 in_seq=0 out_seq=0 2024-03-19T01:25:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.975+0000 7ff206000640 10 client.28222 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:25:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff206000640 1 -- 172.21.15.69:0/2583777980 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5584b5a8f8c0 con 0x5584b5b37290 2024-03-19T01:25:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff206000640 10 client.28222 handle_command_reply: tid=0 2024-03-19T01:25:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff2223b3740 1 client.28222 shutdown 2024-03-19T01:25:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff2223b3740 1 -- 172.21.15.69:0/2583777980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 msgr2=0x5584b5b24400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff2223b3740 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 0x5584b5b24400 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff2000cc850 tx=0x7ff200009770 comp rx=0 tx=0).stop 2024-03-19T01:25:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff2223b3740 1 -- 172.21.15.69:0/2583777980 shutdown_connections 2024-03-19T01:25:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff2223b3740 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x5584b5b37290 0x5584b5b39700 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff2223b3740 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b24940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff2223b3740 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1e340 0x5584b5b24400 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff2223b3740 1 --2- 172.21.15.69:0/2583777980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1da20 0x5584b5b23ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff206000640 0 client.28222 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:25:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff2223b3740 1 -- 172.21.15.69:0/2583777980 >> 172.21.15.69:0/2583777980 conn(0x5584b5b18150 msgr2=0x5584b5b19ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:29.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.976+0000 7ff2223b3740 1 -- 172.21.15.69:0/2583777980 shutdown_connections 2024-03-19T01:25:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.977+0000 7ff2223b3740 1 -- 172.21.15.69:0/2583777980 wait complete. 2024-03-19T01:25:29.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.977+0000 7ff2223b3740 20 client.28222 trim_cache size 0 max 16384 2024-03-19T01:25:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.994+0000 7ff2223b3740 1 Processor -- start 2024-03-19T01:25:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.994+0000 7ff2223b3740 1 -- start start 2024-03-19T01:25:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.994+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b1da20 0x5584b5b265f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.994+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1e340 0x5584b5b32ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.994+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b26f20 0x5584b5b3efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.994+0000 7ff2223b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181b10e0 con 0x5584b5b1e340 2024-03-19T01:25:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.994+0000 7ff2223b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181b2c20 con 0x5584b5b1da20 2024-03-19T01:25:29.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.994+0000 7ff2223b3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181b17b0 con 0x5584b5b26f20 2024-03-19T01:25:29.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff21d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b26f20 0x5584b5b3efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff21cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b1da20 0x5584b5b265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff21d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b26f20 0x5584b5b3efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33312/0 (socket says 172.21.15.69:33312) 2024-03-19T01:25:29.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff21d600640 1 -- 172.21.15.69:0/209998500 learned_addr learned my addr 172.21.15.69:0/209998500 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:29.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff20fe00640 1 --2- 172.21.15.69:0/209998500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1e340 0x5584b5b32ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:29.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff21cc00640 1 -- 172.21.15.69:0/209998500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b26f20 msgr2=0x5584b5b3efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff21cc00640 1 --2- 172.21.15.69:0/209998500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b26f20 0x5584b5b3efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff21cc00640 1 -- 172.21.15.69:0/209998500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1e340 msgr2=0x5584b5b32ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:29.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff21cc00640 1 --2- 172.21.15.69:0/209998500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1e340 0x5584b5b32ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:29.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff21cc00640 1 -- 172.21.15.69:0/209998500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584b5b29680 con 0x5584b5b1da20 2024-03-19T01:25:29.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff21cc00640 1 --2- 172.21.15.69:0/209998500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b1da20 0x5584b5b265f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff20802c180 tx=0x7ff208041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3900e12aed7ad533 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:29.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.995+0000 7ff20d200640 1 -- 172.21.15.69:0/209998500 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208039040 con 0x5584b5b1da20 2024-03-19T01:25:29.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.996+0000 7ff20d200640 1 -- 172.21.15.69:0/209998500 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff208034440 con 0x5584b5b1da20 2024-03-19T01:25:29.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.996+0000 7ff20d200640 1 -- 172.21.15.69:0/209998500 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20802cbb0 con 0x5584b5b1da20 2024-03-19T01:25:29.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.996+0000 7ff2223b3740 1 -- 172.21.15.69:0/209998500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b1da20 msgr2=0x5584b5b265f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:29.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.996+0000 7ff2223b3740 1 --2- 172.21.15.69:0/209998500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b1da20 0x5584b5b265f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff20802c180 tx=0x7ff208041040 comp rx=0 tx=0).stop 2024-03-19T01:25:29.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.996+0000 7ff2223b3740 1 -- 172.21.15.69:0/209998500 shutdown_connections 2024-03-19T01:25:30.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.996+0000 7ff2223b3740 1 --2- 172.21.15.69:0/209998500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b26f20 0x5584b5b3efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.996+0000 7ff2223b3740 1 --2- 172.21.15.69:0/209998500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1e340 0x5584b5b32ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.996+0000 7ff2223b3740 1 --2- 172.21.15.69:0/209998500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b1da20 0x5584b5b265f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.996+0000 7ff2223b3740 1 -- 172.21.15.69:0/209998500 >> 172.21.15.69:0/209998500 conn(0x5584b5b18150 msgr2=0x5584b5b24710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.997+0000 7ff2223b3740 1 -- 172.21.15.69:0/209998500 shutdown_connections 2024-03-19T01:25:30.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.997+0000 7ff2223b3740 1 -- 172.21.15.69:0/209998500 wait complete. 2024-03-19T01:25:30.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.997+0000 7ff2223b3740 1 -- start start 2024-03-19T01:25:30.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.997+0000 7ff2223b3740 1 Processor -- start 2024-03-19T01:25:30.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1da20 0x5584b5b2f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1e340 0x5584b5b2c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff2223b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b2c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff2223b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181b2550 con 0x5584b5b1e340 2024-03-19T01:25:30.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff2223b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181b1e80 con 0x5584b5b26f20 2024-03-19T01:25:30.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff2223b3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181af0c0 con 0x5584b5b1da20 2024-03-19T01:25:30.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff2223b3740 10 client.0 _ll_register_callbacks cb 0x7ff20000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:30.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff21cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1da20 0x5584b5b2f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff20fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1e340 0x5584b5b2c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff21cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1da20 0x5584b5b2f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33326/0 (socket says 172.21.15.69:33326) 2024-03-19T01:25:30.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff21cc00640 1 -- 172.21.15.69:0/1496625309 learned_addr learned my addr 172.21.15.69:0/1496625309 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:30.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.998+0000 7ff21d600640 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b2c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.999+0000 7ff21cc00640 1 -- 172.21.15.69:0/1496625309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 msgr2=0x5584b5b2c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.999+0000 7ff21cc00640 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b2c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.999+0000 7ff21cc00640 1 -- 172.21.15.69:0/1496625309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1e340 msgr2=0x5584b5b2c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.999+0000 7ff21cc00640 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1e340 0x5584b5b2c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.999+0000 7ff21cc00640 1 -- 172.21.15.69:0/1496625309 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584b5b2f700 con 0x5584b5b1da20 2024-03-19T01:25:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.999+0000 7ff21cc00640 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1da20 0x5584b5b2f110 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff208009450 tx=0x7ff208009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d88cf638bb992f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.999+0000 7ff206000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:25:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.999+0000 7ff206000640 1 -- 172.21.15.69:0/1496625309 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208039040 con 0x5584b5b1da20 2024-03-19T01:25:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.999+0000 7ff2223b3740 1 -- 172.21.15.69:0/1496625309 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff208034440 con 0x5584b5b1da20 2024-03-19T01:25:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:29.999+0000 7ff206000640 1 -- 172.21.15.69:0/1496625309 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7ff2080345f0 con 0x5584b5b1da20 2024-03-19T01:25:30.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.000+0000 7ff206000640 1 -- 172.21.15.69:0/1496625309 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff2080347f0 con 0x5584b5b1da20 2024-03-19T01:25:30.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.000+0000 7ff2223b3740 10 client.28228 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:30.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.000+0000 7ff2223b3740 1 -- 172.21.15.69:0/1496625309 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5584b5b9dfd0 con 0x5584b5b1da20 2024-03-19T01:25:30.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.000+0000 7ff206000640 1 -- 172.21.15.69:0/1496625309 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff208034440 con 0x5584b5b1da20 2024-03-19T01:25:30.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.000+0000 7ff206000640 1 -- 172.21.15.69:0/1496625309 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7ff208032cf0 con 0x5584b5b1da20 2024-03-19T01:25:30.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.000+0000 7ff2223b3740 10 client.28228 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:30.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.000+0000 7ff2223b3740 10 client.28228 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-19T01:25:30.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.000+0000 7ff2223b3740 20 client.28228 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:30.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.000+0000 7ff2223b3740 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x5584b5b9e250 0x5584b5ba0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.000+0000 7ff2223b3740 4 client.28228 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:25:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.000+0000 7ff2223b3740 1 -- 172.21.15.69:0/1496625309 --> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584b5a89c10 con 0x5584b5b9e250 2024-03-19T01:25:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.001+0000 7ff20fe00640 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x5584b5b9e250 0x5584b5ba0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.001+0000 7ff206000640 10 client.28228 ms_handle_connect on v2:172.21.15.163:6838/2045902254 2024-03-19T01:25:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.001+0000 7ff20fe00640 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x5584b5b9e250 0x5584b5ba0640 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0444fdac89e963e server_cookie=a411126da96011a2 in_seq=0 out_seq=0 2024-03-19T01:25:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.001+0000 7ff206000640 1 -- 172.21.15.69:0/1496625309 <== mds.2 v2:172.21.15.163:6838/2045902254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584b5a89c10 con 0x5584b5b9e250 2024-03-19T01:25:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.001+0000 7ff206000640 10 client.28228 handle_command_reply: tid=0 2024-03-19T01:25:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.001+0000 7ff2223b3740 1 client.28228 shutdown 2024-03-19T01:25:30.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff2223b3740 1 -- 172.21.15.69:0/1496625309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1da20 msgr2=0x5584b5b2f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff2223b3740 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1da20 0x5584b5b2f110 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff208009450 tx=0x7ff208009480 comp rx=0 tx=0).stop 2024-03-19T01:25:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff2223b3740 1 -- 172.21.15.69:0/1496625309 shutdown_connections 2024-03-19T01:25:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff2223b3740 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.163:6838/2045902254,v1:172.21.15.163:6839/2045902254] conn(0x5584b5b9e250 0x5584b5ba0640 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff2223b3740 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584b5b26f20 0x5584b5b2c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff2223b3740 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5584b5b1e340 0x5584b5b2c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff206000640 0 client.28228 ms_handle_reset on v2:172.21.15.163:6838/2045902254 2024-03-19T01:25:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff2223b3740 1 --2- 172.21.15.69:0/1496625309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5584b5b1da20 0x5584b5b2f110 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff2223b3740 1 -- 172.21.15.69:0/1496625309 >> 172.21.15.69:0/1496625309 conn(0x5584b5b18150 msgr2=0x5584b5b24710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff2223b3740 1 -- 172.21.15.69:0/1496625309 shutdown_connections 2024-03-19T01:25:30.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff2223b3740 1 -- 172.21.15.69:0/1496625309 wait complete. 2024-03-19T01:25:30.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.002+0000 7ff2223b3740 20 client.28228 trim_cache size 0 max 16384 2024-03-19T01:25:30.010 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:25:30.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 -- 172.21.15.69:0/2426231816 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1fc0a29f0 msgr2=0x7ff1fc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1fc0a29f0 0x7ff1fc0a4eb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff210005fd0 tx=0x7ff2100095c0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 -- 172.21.15.69:0/2426231816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff218104380 msgr2=0x7ff21819a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff218104380 0x7ff21819a850 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff20000e9f0 tx=0x7ff20000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 -- 172.21.15.69:0/2426231816 shutdown_connections 2024-03-19T01:25:30.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2181099d0 0x7ff21819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1fc0a29f0 0x7ff1fc0a4eb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff218104380 0x7ff21819a850 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 --2- 172.21.15.69:0/2426231816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2181039c0 0x7ff21819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 -- 172.21.15.69:0/2426231816 >> 172.21.15.69:0/2426231816 conn(0x7ff2180ff490 msgr2=0x7ff2180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 -- 172.21.15.69:0/2426231816 shutdown_connections 2024-03-19T01:25:30.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.003+0000 7ff21f600640 1 -- 172.21.15.69:0/2426231816 wait complete. 2024-03-19T01:25:30.023 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-19T01:25:30.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.230+0000 7f6662a00640 1 -- 172.21.15.69:0/2950027080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c1038a0 msgr2=0x7f665c103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.230+0000 7f6662a00640 1 --2- 172.21.15.69:0/2950027080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c1038a0 0x7f665c103d20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6648009930 tx=0x7f664802bc30 comp rx=0 tx=0).stop 2024-03-19T01:25:30.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.230+0000 7f6662a00640 1 -- 172.21.15.69:0/2950027080 shutdown_connections 2024-03-19T01:25:30.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.230+0000 7f6662a00640 1 --2- 172.21.15.69:0/2950027080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f665c104260 0x7f665c110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.230+0000 7f6662a00640 1 --2- 172.21.15.69:0/2950027080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c1038a0 0x7f665c103d20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.230+0000 7f6662a00640 1 --2- 172.21.15.69:0/2950027080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f665c1098b0 0x7f665c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.230+0000 7f6662a00640 1 -- 172.21.15.69:0/2950027080 >> 172.21.15.69:0/2950027080 conn(0x7f665c0ff330 msgr2=0x7f665c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.231+0000 7f6662a00640 1 -- 172.21.15.69:0/2950027080 shutdown_connections 2024-03-19T01:25:30.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.231+0000 7f6662a00640 1 -- 172.21.15.69:0/2950027080 wait complete. 2024-03-19T01:25:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.232+0000 7f6662a00640 1 Processor -- start 2024-03-19T01:25:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.232+0000 7f6662a00640 1 -- start start 2024-03-19T01:25:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.232+0000 7f6662a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f665c1038a0 0x7f665c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.232+0000 7f6662a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f665c104260 0x7f665c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.232+0000 7f6662a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c1098b0 0x7f665c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.232+0000 7f6662a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f665c113d10 con 0x7f665c104260 2024-03-19T01:25:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.232+0000 7f6662a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f665c113b90 con 0x7f665c1038a0 2024-03-19T01:25:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.232+0000 7f6662a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f665c113e90 con 0x7f665c1098b0 2024-03-19T01:25:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.233+0000 7f6661600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f665c1038a0 0x7f665c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.233+0000 7f6662000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c1098b0 0x7f665c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.233+0000 7f6660c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f665c104260 0x7f665c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.233+0000 7f6661600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f665c1038a0 0x7f665c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54110/0 (socket says 172.21.15.69:54110) 2024-03-19T01:25:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.233+0000 7f6661600640 1 -- 172.21.15.69:0/478095330 learned_addr learned my addr 172.21.15.69:0/478095330 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:30.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.233+0000 7f6661600640 1 -- 172.21.15.69:0/478095330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c1098b0 msgr2=0x7f665c19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.233+0000 7f6661600640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c1098b0 0x7f665c19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.233+0000 7f6661600640 1 -- 172.21.15.69:0/478095330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f665c104260 msgr2=0x7f665c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.233+0000 7f6661600640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f665c104260 0x7f665c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.233+0000 7f6661600640 1 -- 172.21.15.69:0/478095330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c19f520 con 0x7f665c1038a0 2024-03-19T01:25:30.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.233+0000 7f6661600640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f665c1038a0 0x7f665c19a530 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f665000e9f0 tx=0x7f665000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.234+0000 7f665a000640 1 -- 172.21.15.69:0/478095330 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665000cd80 con 0x7f665c1038a0 2024-03-19T01:25:30.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.234+0000 7f665a000640 1 -- 172.21.15.69:0/478095330 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6650010910 con 0x7f665c1038a0 2024-03-19T01:25:30.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.234+0000 7f6662a00640 1 -- 172.21.15.69:0/478095330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c19f810 con 0x7f665c1038a0 2024-03-19T01:25:30.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.234+0000 7f665a000640 1 -- 172.21.15.69:0/478095330 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650018990 con 0x7f665c1038a0 2024-03-19T01:25:30.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.234+0000 7f6662a00640 1 -- 172.21.15.69:0/478095330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c1a70a0 con 0x7f665c1038a0 2024-03-19T01:25:30.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.237+0000 7f665a000640 1 -- 172.21.15.69:0/478095330 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f6650010ac0 con 0x7f665c1038a0 2024-03-19T01:25:30.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.237+0000 7f6666e36740 1 Processor -- start 2024-03-19T01:25:30.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.237+0000 7f6666e36740 1 -- start start 2024-03-19T01:25:30.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6666e36740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 0x55570840cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6666e36740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570840d510 0x555708415bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6666e36740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557084160f0 0x5557084184e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6666e36740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555708371380 con 0x55570840d510 2024-03-19T01:25:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6666e36740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555708319d20 con 0x5557084160f0 2024-03-19T01:25:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6666e36740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55570836d420 con 0x55570840cbf0 2024-03-19T01:25:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6662000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 0x55570840cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6662000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 0x55570840cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33338/0 (socket says 172.21.15.69:33338) 2024-03-19T01:25:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6662000640 1 -- 172.21.15.69:0/2942814176 learned_addr learned my addr 172.21.15.69:0/2942814176 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6661600640 1 --2- 172.21.15.69:0/2942814176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570840d510 0x555708415bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6660c00640 1 --2- 172.21.15.69:0/2942814176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557084160f0 0x5557084184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6662000640 1 -- 172.21.15.69:0/2942814176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557084160f0 msgr2=0x5557084184e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.238+0000 7f6662000640 1 --2- 172.21.15.69:0/2942814176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557084160f0 0x5557084184e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f6662000640 1 -- 172.21.15.69:0/2942814176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570840d510 msgr2=0x555708415bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f6662000640 1 --2- 172.21.15.69:0/2942814176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570840d510 0x555708415bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f6662000640 1 -- 172.21.15.69:0/2942814176 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557080853f0 con 0x55570840cbf0 2024-03-19T01:25:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f6660c00640 1 --2- 172.21.15.69:0/2942814176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557084160f0 0x5557084184e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:25:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f6661600640 1 --2- 172.21.15.69:0/2942814176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570840d510 0x555708415bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:25:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f665a000640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f66440a29f0 0x7f66440a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f6662000640 1 --2- 172.21.15.69:0/2942814176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 0x55570840cfd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6654009fd0 tx=0x7f66540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=775922829c6e83e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f665a000640 1 -- 172.21.15.69:0/478095330 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7f66500cf4c0 con 0x7f665c1038a0 2024-03-19T01:25:30.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f6661600640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f66440a29f0 0x7f66440a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f664e600640 1 -- 172.21.15.69:0/2942814176 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654004d20 con 0x55570840cbf0 2024-03-19T01:25:30.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f664e600640 1 -- 172.21.15.69:0/2942814176 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6654004050 con 0x55570840cbf0 2024-03-19T01:25:30.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.239+0000 7f664e600640 1 -- 172.21.15.69:0/2942814176 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654005220 con 0x55570840cbf0 2024-03-19T01:25:30.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.240+0000 7f6666e36740 1 -- 172.21.15.69:0/2942814176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 msgr2=0x55570840cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.240+0000 7f6666e36740 1 --2- 172.21.15.69:0/2942814176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 0x55570840cfd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6654009fd0 tx=0x7f66540118d0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.240+0000 7f6661600640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f66440a29f0 0x7f66440a4eb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55570840c6c0 tx=0x7f6650000bc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.240+0000 7f6666e36740 1 -- 172.21.15.69:0/2942814176 shutdown_connections 2024-03-19T01:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.240+0000 7f6666e36740 1 --2- 172.21.15.69:0/2942814176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557084160f0 0x5557084184e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.240+0000 7f6666e36740 1 --2- 172.21.15.69:0/2942814176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570840d510 0x555708415bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.240+0000 7f6666e36740 1 --2- 172.21.15.69:0/2942814176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 0x55570840cfd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.240+0000 7f6666e36740 1 -- 172.21.15.69:0/2942814176 >> 172.21.15.69:0/2942814176 conn(0x555708407320 msgr2=0x555708407700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.240+0000 7f6666e36740 1 -- 172.21.15.69:0/2942814176 shutdown_connections 2024-03-19T01:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.240+0000 7f6666e36740 1 -- 172.21.15.69:0/2942814176 wait complete. 2024-03-19T01:25:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.241+0000 7f6666e36740 1 -- start start 2024-03-19T01:25:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.241+0000 7f6666e36740 1 Processor -- start 2024-03-19T01:25:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.241+0000 7f6666e36740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 0x555708413080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.241+0000 7f6666e36740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840d510 0x5557084135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.241+0000 7f6666e36740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557084160f0 0x555708413b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.241+0000 7f6666e36740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555708418ee0 con 0x5557084160f0 2024-03-19T01:25:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.241+0000 7f6666e36740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55570832d160 con 0x55570840d510 2024-03-19T01:25:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.241+0000 7f6666e36740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555708419060 con 0x55570840cbf0 2024-03-19T01:25:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6666e36740 10 client.0 _ll_register_callbacks cb 0x555708369170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6660c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840d510 0x5557084135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6660c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840d510 0x5557084135c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54116/0 (socket says 172.21.15.69:54116) 2024-03-19T01:25:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6660c00640 1 -- 172.21.15.69:0/81855015 learned_addr learned my addr 172.21.15.69:0/81855015 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6662000640 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 0x555708413080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6662000640 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557084160f0 0x555708413b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6660c00640 1 -- 172.21.15.69:0/81855015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 msgr2=0x555708413080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6660c00640 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 0x555708413080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6660c00640 1 -- 172.21.15.69:0/81855015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557084160f0 msgr2=0x555708413b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6660c00640 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557084160f0 0x555708413b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6660c00640 1 -- 172.21.15.69:0/81855015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570814f7d0 con 0x55570840d510 2024-03-19T01:25:30.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.242+0000 7f6662000640 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557084160f0 0x555708413b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:25:30.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.243+0000 7f6660c00640 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840d510 0x5557084135c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6648000c00 tx=0x7f6648005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7232db5808f32cd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.243+0000 7f6642a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:25:30.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.243+0000 7f6642a00640 1 -- 172.21.15.69:0/81855015 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664802ca90 con 0x55570840d510 2024-03-19T01:25:30.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.243+0000 7f6666e36740 1 -- 172.21.15.69:0/81855015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6654004050 con 0x55570840d510 2024-03-19T01:25:30.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.243+0000 7f6642a00640 1 -- 172.21.15.69:0/81855015 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7f664802cc30 con 0x55570840d510 2024-03-19T01:25:30.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.243+0000 7f6642a00640 1 -- 172.21.15.69:0/81855015 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6648052050 con 0x55570840d510 2024-03-19T01:25:30.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.243+0000 7f6666e36740 10 client.28322 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:30.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.243+0000 7f6666e36740 1 -- 172.21.15.69:0/81855015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5557084250d0 con 0x55570840d510 2024-03-19T01:25:30.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.243+0000 7f6642a00640 1 -- 172.21.15.69:0/81855015 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6654004050 con 0x55570840d510 2024-03-19T01:25:30.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.244+0000 7f6642a00640 1 -- 172.21.15.69:0/81855015 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f66480035f0 con 0x55570840d510 2024-03-19T01:25:30.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.244+0000 7f6666e36740 10 client.28322 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:30.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.244+0000 7f6666e36740 10 client.28322 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:25:30.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.244+0000 7f6666e36740 20 client.28322 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:30.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.244+0000 7f6666e36740 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557084265e0 0x555708428a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.244+0000 7f6666e36740 4 client.28322 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:25:30.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.244+0000 7f6666e36740 1 -- 172.21.15.69:0/81855015 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55570837e8c0 con 0x5557084265e0 2024-03-19T01:25:30.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.244+0000 7f6662000640 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557084265e0 0x555708428a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.245+0000 7f6662000640 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557084265e0 0x555708428a00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3500bb39320544ca server_cookie=28c836ffbe05517c in_seq=0 out_seq=0 2024-03-19T01:25:30.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.245+0000 7f6642a00640 10 client.28322 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:25:30.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6642a00640 1 -- 172.21.15.69:0/81855015 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55570837e8c0 con 0x5557084265e0 2024-03-19T01:25:30.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6642a00640 10 client.28322 handle_command_reply: tid=0 2024-03-19T01:25:30.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6666e36740 1 client.28322 shutdown 2024-03-19T01:25:30.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6666e36740 1 -- 172.21.15.69:0/81855015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840d510 msgr2=0x5557084135c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6666e36740 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840d510 0x5557084135c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6648000c00 tx=0x7f6648005e50 comp rx=0 tx=0).stop 2024-03-19T01:25:30.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6666e36740 1 -- 172.21.15.69:0/81855015 shutdown_connections 2024-03-19T01:25:30.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6666e36740 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557084160f0 0x555708413b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6666e36740 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x5557084265e0 0x555708428a00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6666e36740 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840d510 0x5557084135c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6666e36740 1 --2- 172.21.15.69:0/81855015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840cbf0 0x555708413080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6642a00640 0 client.28322 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:25:30.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.246+0000 7f6666e36740 1 -- 172.21.15.69:0/81855015 >> 172.21.15.69:0/81855015 conn(0x555708407320 msgr2=0x555708409070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.247+0000 7f6666e36740 1 -- 172.21.15.69:0/81855015 shutdown_connections 2024-03-19T01:25:30.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.247+0000 7f6666e36740 1 -- 172.21.15.69:0/81855015 wait complete. 2024-03-19T01:25:30.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.247+0000 7f6666e36740 20 client.28322 trim_cache size 0 max 16384 2024-03-19T01:25:30.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.261+0000 7f6666e36740 1 Processor -- start 2024-03-19T01:25:30.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.261+0000 7f6666e36740 1 -- start start 2024-03-19T01:25:30.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.261+0000 7f6666e36740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 0x5557084157b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.261+0000 7f6666e36740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570840d510 0x555708421cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.261+0000 7f6666e36740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557084160f0 0x55570842e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.261+0000 7f6666e36740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1b0de0 con 0x55570840d510 2024-03-19T01:25:30.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.261+0000 7f6666e36740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1b2920 con 0x55570840cbf0 2024-03-19T01:25:30.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.261+0000 7f6666e36740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1b14b0 con 0x5557084160f0 2024-03-19T01:25:30.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.262+0000 7f6662000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 0x5557084157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.262+0000 7f6662000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 0x5557084157b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54120/0 (socket says 172.21.15.69:54120) 2024-03-19T01:25:30.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.262+0000 7f6662000640 1 -- 172.21.15.69:0/3062958203 learned_addr learned my addr 172.21.15.69:0/3062958203 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:30.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.262+0000 7f6660c00640 1 --2- 172.21.15.69:0/3062958203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570840d510 0x555708421cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.262+0000 7f6662000640 1 --2- 172.21.15.69:0/3062958203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557084160f0 0x55570842e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.262+0000 7f6662000640 1 -- 172.21.15.69:0/3062958203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557084160f0 msgr2=0x55570842e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.262+0000 7f6662000640 1 --2- 172.21.15.69:0/3062958203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557084160f0 0x55570842e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.262+0000 7f6662000640 1 -- 172.21.15.69:0/3062958203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570840d510 msgr2=0x555708421cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.262+0000 7f6662000640 1 --2- 172.21.15.69:0/3062958203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570840d510 0x555708421cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.262+0000 7f6662000640 1 -- 172.21.15.69:0/3062958203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555708406b60 con 0x55570840cbf0 2024-03-19T01:25:30.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.262+0000 7f6662000640 1 --2- 172.21.15.69:0/3062958203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 0x5557084157b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6654009fa0 tx=0x7f6654009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85196e8bfcedec8a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.263+0000 7f664dc00640 1 -- 172.21.15.69:0/3062958203 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654032070 con 0x55570840cbf0 2024-03-19T01:25:30.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.263+0000 7f664dc00640 1 -- 172.21.15.69:0/3062958203 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665402c520 con 0x55570840cbf0 2024-03-19T01:25:30.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.263+0000 7f664dc00640 1 -- 172.21.15.69:0/3062958203 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665400f6e0 con 0x55570840cbf0 2024-03-19T01:25:30.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.264+0000 7f6666e36740 1 -- 172.21.15.69:0/3062958203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 msgr2=0x5557084157b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.264+0000 7f6666e36740 1 --2- 172.21.15.69:0/3062958203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 0x5557084157b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6654009fa0 tx=0x7f6654009500 comp rx=0 tx=0).stop 2024-03-19T01:25:30.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.264+0000 7f6666e36740 1 -- 172.21.15.69:0/3062958203 shutdown_connections 2024-03-19T01:25:30.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.264+0000 7f6666e36740 1 --2- 172.21.15.69:0/3062958203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5557084160f0 0x55570842e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.264+0000 7f6666e36740 1 --2- 172.21.15.69:0/3062958203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55570840d510 0x555708421cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.264+0000 7f6666e36740 1 --2- 172.21.15.69:0/3062958203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 0x5557084157b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.264+0000 7f6666e36740 1 -- 172.21.15.69:0/3062958203 >> 172.21.15.69:0/3062958203 conn(0x555708407320 msgr2=0x55570840e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.264+0000 7f6666e36740 1 -- 172.21.15.69:0/3062958203 shutdown_connections 2024-03-19T01:25:30.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.264+0000 7f6666e36740 1 -- 172.21.15.69:0/3062958203 wait complete. 2024-03-19T01:25:30.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.264+0000 7f6666e36740 1 -- start start 2024-03-19T01:25:30.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.265+0000 7f6666e36740 1 Processor -- start 2024-03-19T01:25:30.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.265+0000 7f6666e36740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 0x555708419fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.265+0000 7f6666e36740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840d510 0x55570841a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.265+0000 7f6666e36740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557084160f0 0x55570848b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.265+0000 7f6666e36740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1b2250 con 0x5557084160f0 2024-03-19T01:25:30.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.265+0000 7f6666e36740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1b1b80 con 0x55570840cbf0 2024-03-19T01:25:30.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.265+0000 7f6666e36740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1aedc0 con 0x55570840d510 2024-03-19T01:25:30.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.265+0000 7f6666e36740 10 client.0 _ll_register_callbacks cb 0x7f66540093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:30.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.265+0000 7f6660c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840d510 0x55570841a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.265+0000 7f6662000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 0x555708419fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6662000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 0x555708419fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54136/0 (socket says 172.21.15.69:54136) 2024-03-19T01:25:30.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6662000640 1 -- 172.21.15.69:0/626381830 learned_addr learned my addr 172.21.15.69:0/626381830 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:30.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6660c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840d510 0x55570841a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33376/0 (socket says 172.21.15.69:33376) 2024-03-19T01:25:30.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6662000640 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557084160f0 0x55570848b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6660c00640 1 -- 172.21.15.69:0/626381830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 msgr2=0x555708419fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6660c00640 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 0x555708419fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6660c00640 1 -- 172.21.15.69:0/626381830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557084160f0 msgr2=0x55570848b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6660c00640 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557084160f0 0x55570848b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6660c00640 1 -- 172.21.15.69:0/626381830 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570848be40 con 0x55570840d510 2024-03-19T01:25:30.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6660c00640 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840d510 0x55570841a4f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6648003ac0 tx=0x7f6648009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=250fa846b9a36131 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6642a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-03-19T01:25:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6642a00640 1 -- 172.21.15.69:0/626381830 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664805a020 con 0x55570840d510 2024-03-19T01:25:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.266+0000 7f6642a00640 1 -- 172.21.15.69:0/626381830 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7f66480035f0 con 0x55570840d510 2024-03-19T01:25:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6666e36740 1 -- 172.21.15.69:0/626381830 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665402c520 con 0x55570840d510 2024-03-19T01:25:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6642a00640 1 -- 172.21.15.69:0/626381830 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6648065050 con 0x55570840d510 2024-03-19T01:25:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6666e36740 10 client.28246 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6666e36740 1 -- 172.21.15.69:0/626381830 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55570848cfe0 con 0x55570840d510 2024-03-19T01:25:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6642a00640 1 -- 172.21.15.69:0/626381830 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665402c520 con 0x55570840d510 2024-03-19T01:25:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6642a00640 1 -- 172.21.15.69:0/626381830 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7f6648004ee0 con 0x55570840d510 2024-03-19T01:25:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6666e36740 10 client.28246 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6666e36740 10 client.28246 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-19T01:25:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6666e36740 20 client.28246 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6666e36740 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55570848d7e0 0x55570848fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6666e36740 4 client.28246 mds_command: new command op to 24580 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:25:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.267+0000 7f6666e36740 1 -- 172.21.15.69:0/626381830 --> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555708378c10 con 0x55570848d7e0 2024-03-19T01:25:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.268+0000 7f6662000640 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55570848d7e0 0x55570848fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.268+0000 7f6662000640 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55570848d7e0 0x55570848fc00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c294bdbe5c1a6b00 server_cookie=36d723fc719b580a in_seq=0 out_seq=0 2024-03-19T01:25:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.268+0000 7f6642a00640 10 client.28246 ms_handle_connect on v2:172.21.15.69:6840/4121471703 2024-03-19T01:25:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.268+0000 7f6642a00640 1 -- 172.21.15.69:0/626381830 <== mds.1 v2:172.21.15.69:6840/4121471703 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555708378c10 con 0x55570848d7e0 2024-03-19T01:25:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.268+0000 7f6642a00640 10 client.28246 handle_command_reply: tid=0 2024-03-19T01:25:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.268+0000 7f6666e36740 1 client.28246 shutdown 2024-03-19T01:25:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6666e36740 1 -- 172.21.15.69:0/626381830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840d510 msgr2=0x55570841a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6666e36740 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840d510 0x55570841a4f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6648003ac0 tx=0x7f6648009520 comp rx=0 tx=0).stop 2024-03-19T01:25:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6666e36740 1 -- 172.21.15.69:0/626381830 shutdown_connections 2024-03-19T01:25:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6666e36740 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557084160f0 0x55570848b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6666e36740 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55570840d510 0x55570841a4f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6666e36740 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.69:6840/4121471703,v1:172.21.15.69:6841/4121471703] conn(0x55570848d7e0 0x55570848fc00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6666e36740 1 --2- 172.21.15.69:0/626381830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55570840cbf0 0x555708419fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6642a00640 0 client.28246 ms_handle_reset on v2:172.21.15.69:6840/4121471703 2024-03-19T01:25:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6666e36740 1 -- 172.21.15.69:0/626381830 >> 172.21.15.69:0/626381830 conn(0x555708407320 msgr2=0x555708408550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6666e36740 1 -- 172.21.15.69:0/626381830 shutdown_connections 2024-03-19T01:25:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6666e36740 1 -- 172.21.15.69:0/626381830 wait complete. 2024-03-19T01:25:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.269+0000 7f6666e36740 20 client.28246 trim_cache size 0 max 16384 2024-03-19T01:25:30.278 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:25:30.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 -- 172.21.15.69:0/478095330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f66440a29f0 msgr2=0x7f66440a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f66440a29f0 0x7f66440a4eb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55570840c6c0 tx=0x7f6650000bc0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 -- 172.21.15.69:0/478095330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f665c1038a0 msgr2=0x7f665c19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f665c1038a0 0x7f665c19a530 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f665000e9f0 tx=0x7f665000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 -- 172.21.15.69:0/478095330 shutdown_connections 2024-03-19T01:25:30.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c1098b0 0x7f665c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f665c104260 0x7f665c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f66440a29f0 0x7f66440a4eb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 --2- 172.21.15.69:0/478095330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f665c1038a0 0x7f665c19a530 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 -- 172.21.15.69:0/478095330 >> 172.21.15.69:0/478095330 conn(0x7f665c0ff330 msgr2=0x7f665c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 -- 172.21.15.69:0/478095330 shutdown_connections 2024-03-19T01:25:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.270+0000 7f6662a00640 1 -- 172.21.15.69:0/478095330 wait complete. 2024-03-19T01:25:30.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:30 smithi069 ceph-mon[31727]: pgmap v465: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-03-19T01:25:30.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:30 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2493095786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:25:30.291 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-19T01:25:30.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:30 smithi086 ceph-mon[66829]: pgmap v465: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-03-19T01:25:30.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:30 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2493095786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:25:30.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.530+0000 7ff30a000640 1 -- 172.21.15.69:0/3548996545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3041099d0 msgr2=0x7ff304109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.530+0000 7ff30a000640 1 --2- 172.21.15.69:0/3548996545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3041099d0 0x7ff304109db0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009930 tx=0x7ff2f402bc30 comp rx=0 tx=0).stop 2024-03-19T01:25:30.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.530+0000 7ff30a000640 1 -- 172.21.15.69:0/3548996545 shutdown_connections 2024-03-19T01:25:30.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.530+0000 7ff30a000640 1 --2- 172.21.15.69:0/3548996545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff304104380 0x7ff304110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.530+0000 7ff30a000640 1 --2- 172.21.15.69:0/3548996545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3041039c0 0x7ff304103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.530+0000 7ff30a000640 1 --2- 172.21.15.69:0/3548996545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3041099d0 0x7ff304109db0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.530+0000 7ff30a000640 1 -- 172.21.15.69:0/3548996545 >> 172.21.15.69:0/3548996545 conn(0x7ff3040ff4d0 msgr2=0x7ff3041018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.530+0000 7ff30a000640 1 -- 172.21.15.69:0/3548996545 shutdown_connections 2024-03-19T01:25:30.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.530+0000 7ff30a000640 1 -- 172.21.15.69:0/3548996545 wait complete. 2024-03-19T01:25:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.531+0000 7ff30a000640 1 Processor -- start 2024-03-19T01:25:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.531+0000 7ff30a000640 1 -- start start 2024-03-19T01:25:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.532+0000 7ff30a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3041039c0 0x7ff30419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.532+0000 7ff30a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff304104380 0x7ff30419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.532+0000 7ff30a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3041099d0 0x7ff30419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.532+0000 7ff30a000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff304113e30 con 0x7ff3041039c0 2024-03-19T01:25:30.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.532+0000 7ff30a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff304113cb0 con 0x7ff304104380 2024-03-19T01:25:30.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.532+0000 7ff30a000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff304113fb0 con 0x7ff3041099d0 2024-03-19T01:25:30.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.532+0000 7ff303e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3041099d0 0x7ff30419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.532+0000 7ff303400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3041039c0 0x7ff30419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.532+0000 7ff303e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3041099d0 0x7ff30419ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33388/0 (socket says 172.21.15.69:33388) 2024-03-19T01:25:30.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.532+0000 7ff303e00640 1 -- 172.21.15.69:0/1689040727 learned_addr learned my addr 172.21.15.69:0/1689040727 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:30.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.532+0000 7ff302a00640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff304104380 0x7ff30419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.533+0000 7ff303400640 1 -- 172.21.15.69:0/1689040727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3041099d0 msgr2=0x7ff30419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.533+0000 7ff303400640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3041099d0 0x7ff30419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.533+0000 7ff303400640 1 -- 172.21.15.69:0/1689040727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff304104380 msgr2=0x7ff30419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.533+0000 7ff303400640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff304104380 0x7ff30419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.533+0000 7ff303400640 1 -- 172.21.15.69:0/1689040727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30419f380 con 0x7ff3041039c0 2024-03-19T01:25:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.533+0000 7ff303e00640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3041099d0 0x7ff30419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:25:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.533+0000 7ff303400640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3041039c0 0x7ff30419a390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4036040 tx=0x7ff2f4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.533+0000 7ff2fbe00640 1 -- 172.21.15.69:0/1689040727 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4043020 con 0x7ff3041039c0 2024-03-19T01:25:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.533+0000 7ff30a000640 1 -- 172.21.15.69:0/1689040727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30419f610 con 0x7ff3041039c0 2024-03-19T01:25:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.534+0000 7ff2fbe00640 1 -- 172.21.15.69:0/1689040727 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f4005000 con 0x7ff3041039c0 2024-03-19T01:25:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.534+0000 7ff2fbe00640 1 -- 172.21.15.69:0/1689040727 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4003af0 con 0x7ff3041039c0 2024-03-19T01:25:30.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.534+0000 7ff30a000640 1 -- 172.21.15.69:0/1689040727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3041a6f00 con 0x7ff3041039c0 2024-03-19T01:25:30.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.537+0000 7ff2fbe00640 1 -- 172.21.15.69:0/1689040727 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ff2f4046020 con 0x7ff3041039c0 2024-03-19T01:25:30.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.538+0000 7ff30cde7740 1 Processor -- start 2024-03-19T01:25:30.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.538+0000 7ff30cde7740 1 -- start start 2024-03-19T01:25:30.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.538+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be6a20 0x55fcd3be6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.538+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be7340 0x55fcd3bef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.538+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3beff20 0x55fcd3bf2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.538+0000 7ff30cde7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fcd3b4b380 con 0x55fcd3be6a20 2024-03-19T01:25:30.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.538+0000 7ff30cde7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fcd3af3d20 con 0x55fcd3beff20 2024-03-19T01:25:30.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.538+0000 7ff30cde7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fcd3b47420 con 0x55fcd3be7340 2024-03-19T01:25:30.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.538+0000 7ff303400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be7340 0x55fcd3bef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.538+0000 7ff303400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be7340 0x55fcd3bef9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33396/0 (socket says 172.21.15.69:33396) 2024-03-19T01:25:30.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.539+0000 7ff303400640 1 -- 172.21.15.69:0/707911829 learned_addr learned my addr 172.21.15.69:0/707911829 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:30.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.539+0000 7ff303e00640 1 --2- 172.21.15.69:0/707911829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be6a20 0x55fcd3be6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.539+0000 7ff302a00640 1 --2- 172.21.15.69:0/707911829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3beff20 0x55fcd3bf2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.539+0000 7ff303400640 1 -- 172.21.15.69:0/707911829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3beff20 msgr2=0x55fcd3bf2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.539+0000 7ff303400640 1 --2- 172.21.15.69:0/707911829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3beff20 0x55fcd3bf2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.539+0000 7ff303400640 1 -- 172.21.15.69:0/707911829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be6a20 msgr2=0x55fcd3be6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.539+0000 7ff303400640 1 --2- 172.21.15.69:0/707911829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be6a20 0x55fcd3be6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.539+0000 7ff303400640 1 -- 172.21.15.69:0/707911829 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcd385f3f0 con 0x55fcd3be7340 2024-03-19T01:25:30.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.539+0000 7ff303400640 1 --2- 172.21.15.69:0/707911829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be7340 0x55fcd3bef9e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40ee4a0 tx=0x7ff2f40f9270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9945e5449288bc10 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.540+0000 7ff2fbe00640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2e80a2970 0x7ff2e80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.540+0000 7ff2efe00640 1 -- 172.21.15.69:0/707911829 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40b9710 con 0x55fcd3be7340 2024-03-19T01:25:30.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.540+0000 7ff2efe00640 1 -- 172.21.15.69:0/707911829 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f40fe030 con 0x55fcd3be7340 2024-03-19T01:25:30.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.540+0000 7ff2efe00640 1 -- 172.21.15.69:0/707911829 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40bc8c0 con 0x55fcd3be7340 2024-03-19T01:25:30.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.540+0000 7ff2fbe00640 1 -- 172.21.15.69:0/1689040727 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7ff2f40f1150 con 0x7ff3041039c0 2024-03-19T01:25:30.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.540+0000 7ff303e00640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2e80a2970 0x7ff2e80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.541+0000 7ff30cde7740 1 -- 172.21.15.69:0/707911829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be7340 msgr2=0x55fcd3bef9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.541+0000 7ff30cde7740 1 --2- 172.21.15.69:0/707911829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be7340 0x55fcd3bef9e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40ee4a0 tx=0x7ff2f40f9270 comp rx=0 tx=0).stop 2024-03-19T01:25:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.541+0000 7ff303e00640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2e80a2970 0x7ff2e80a4e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0049d0 tx=0x7ff2fc009500 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.541+0000 7ff30cde7740 1 -- 172.21.15.69:0/707911829 shutdown_connections 2024-03-19T01:25:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.541+0000 7ff30cde7740 1 --2- 172.21.15.69:0/707911829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3beff20 0x55fcd3bf2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.541+0000 7ff30cde7740 1 --2- 172.21.15.69:0/707911829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be7340 0x55fcd3bef9e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.541+0000 7ff30cde7740 1 --2- 172.21.15.69:0/707911829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be6a20 0x55fcd3be6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.541+0000 7ff30cde7740 1 -- 172.21.15.69:0/707911829 >> 172.21.15.69:0/707911829 conn(0x55fcd3be1150 msgr2=0x55fcd3be1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.541+0000 7ff30cde7740 1 -- 172.21.15.69:0/707911829 shutdown_connections 2024-03-19T01:25:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.541+0000 7ff30cde7740 1 -- 172.21.15.69:0/707911829 wait complete. 2024-03-19T01:25:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.542+0000 7ff30cde7740 1 -- start start 2024-03-19T01:25:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.542+0000 7ff30cde7740 1 Processor -- start 2024-03-19T01:25:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3becf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bed4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3beff20 0x55fcd3beda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff30cde7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fcd3bf2ad0 con 0x55fcd3be7340 2024-03-19T01:25:30.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff30cde7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fcd3b07160 con 0x55fcd3be6a20 2024-03-19T01:25:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff30cde7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fcd3bf2c50 con 0x55fcd3beff20 2024-03-19T01:25:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff30cde7740 10 client.0 _ll_register_callbacks cb 0x55fcd3b43170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff303400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3becf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff302a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bed4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff303400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3becf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54178/0 (socket says 172.21.15.69:54178) 2024-03-19T01:25:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff303400640 1 -- 172.21.15.69:0/3170152694 learned_addr learned my addr 172.21.15.69:0/3170152694 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.543+0000 7ff303e00640 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3beff20 0x55fcd3beda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff303400640 1 -- 172.21.15.69:0/3170152694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3beff20 msgr2=0x55fcd3beda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff303400640 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3beff20 0x55fcd3beda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff303400640 1 -- 172.21.15.69:0/3170152694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 msgr2=0x55fcd3bed4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff303400640 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bed4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff303400640 1 -- 172.21.15.69:0/3170152694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcd39297d0 con 0x55fcd3be6a20 2024-03-19T01:25:30.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff303400640 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3becf80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40b98d0 tx=0x7ff2f4009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a519efa3ff22e7f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff2ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-19T01:25:30.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff2ecc00640 1 -- 172.21.15.69:0/3170152694 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40bcbf0 con 0x55fcd3be6a20 2024-03-19T01:25:30.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff2ecc00640 1 -- 172.21.15.69:0/3170152694 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7ff2f40aad80 con 0x55fcd3be6a20 2024-03-19T01:25:30.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff30cde7740 1 -- 172.21.15.69:0/3170152694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f40fe030 con 0x55fcd3be6a20 2024-03-19T01:25:30.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff2ecc00640 1 -- 172.21.15.69:0/3170152694 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff2f40bcdd0 con 0x55fcd3be6a20 2024-03-19T01:25:30.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.544+0000 7ff30cde7740 10 client.28340 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:30.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.545+0000 7ff30cde7740 1 -- 172.21.15.69:0/3170152694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55fcd3bf2ad0 con 0x55fcd3be6a20 2024-03-19T01:25:30.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.545+0000 7ff2ecc00640 1 -- 172.21.15.69:0/3170152694 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f40fe030 con 0x55fcd3be6a20 2024-03-19T01:25:30.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.545+0000 7ff2ecc00640 1 -- 172.21.15.69:0/3170152694 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7ff2f40ac070 con 0x55fcd3be6a20 2024-03-19T01:25:30.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.545+0000 7ff30cde7740 10 client.28340 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:30.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.545+0000 7ff30cde7740 10 client.28340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:25:30.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.545+0000 7ff30cde7740 20 client.28340 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:30.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.545+0000 7ff30cde7740 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fcd3bfff30 0x55fcd3c023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.545+0000 7ff30cde7740 4 client.28340 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:25:30.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.545+0000 7ff30cde7740 1 -- 172.21.15.69:0/3170152694 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcd3b588c0 con 0x55fcd3bfff30 2024-03-19T01:25:30.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.546+0000 7ff302a00640 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fcd3bfff30 0x55fcd3c023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.546+0000 7ff2ecc00640 10 client.28340 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:25:30.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.546+0000 7ff302a00640 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fcd3bfff30 0x55fcd3c023a0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac38dab4e06132e server_cookie=dd9d86492b43e18f in_seq=0 out_seq=0 2024-03-19T01:25:30.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.547+0000 7ff2ecc00640 1 -- 172.21.15.69:0/3170152694 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fcd3b588c0 con 0x55fcd3bfff30 2024-03-19T01:25:30.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.547+0000 7ff2ecc00640 10 client.28340 handle_command_reply: tid=0 2024-03-19T01:25:30.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.547+0000 7ff30cde7740 1 client.28340 shutdown 2024-03-19T01:25:30.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff30cde7740 1 -- 172.21.15.69:0/3170152694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 msgr2=0x55fcd3becf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff30cde7740 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3becf80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40b98d0 tx=0x7ff2f4009300 comp rx=0 tx=0).stop 2024-03-19T01:25:30.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff30cde7740 1 -- 172.21.15.69:0/3170152694 shutdown_connections 2024-03-19T01:25:30.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff30cde7740 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fcd3bfff30 0x55fcd3c023a0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff30cde7740 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3beff20 0x55fcd3beda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff30cde7740 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff30cde7740 1 --2- 172.21.15.69:0/3170152694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3becf80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff2ecc00640 0 client.28340 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:25:30.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff30cde7740 1 -- 172.21.15.69:0/3170152694 >> 172.21.15.69:0/3170152694 conn(0x55fcd3be1150 msgr2=0x55fcd3be2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff30cde7740 1 -- 172.21.15.69:0/3170152694 shutdown_connections 2024-03-19T01:25:30.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff30cde7740 1 -- 172.21.15.69:0/3170152694 wait complete. 2024-03-19T01:25:30.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.548+0000 7ff30cde7740 20 client.28340 trim_cache size 0 max 16384 2024-03-19T01:25:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:30 smithi163 ceph-mon[67428]: pgmap v465: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-03-19T01:25:30.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:30 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2493095786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:25:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.562+0000 7ff30cde7740 1 Processor -- start 2024-03-19T01:25:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.562+0000 7ff30cde7740 1 -- start start 2024-03-19T01:25:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3bef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bfb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3beff20 0x55fcd3c07c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff30cde7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041b1090 con 0x55fcd3be7340 2024-03-19T01:25:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff30cde7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041b2bd0 con 0x55fcd3be6a20 2024-03-19T01:25:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff30cde7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041b1760 con 0x55fcd3beff20 2024-03-19T01:25:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff303400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3bef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff303400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3bef6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54194/0 (socket says 172.21.15.69:54194) 2024-03-19T01:25:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff303400640 1 -- 172.21.15.69:0/384073311 learned_addr learned my addr 172.21.15.69:0/384073311 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff302a00640 1 --2- 172.21.15.69:0/384073311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bfb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff303e00640 1 --2- 172.21.15.69:0/384073311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3beff20 0x55fcd3c07c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff303400640 1 -- 172.21.15.69:0/384073311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3beff20 msgr2=0x55fcd3c07c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff303400640 1 --2- 172.21.15.69:0/384073311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3beff20 0x55fcd3c07c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff303400640 1 -- 172.21.15.69:0/384073311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 msgr2=0x55fcd3bfb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff303400640 1 --2- 172.21.15.69:0/384073311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bfb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.563+0000 7ff303400640 1 -- 172.21.15.69:0/384073311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcd3be7170 con 0x55fcd3be6a20 2024-03-19T01:25:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.564+0000 7ff303400640 1 --2- 172.21.15.69:0/384073311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3bef6b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009930 tx=0x7ff2f402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a858afc40bb2869 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.564+0000 7ff2ef400640 1 -- 172.21.15.69:0/384073311 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40bcfb0 con 0x55fcd3be6a20 2024-03-19T01:25:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.564+0000 7ff2ef400640 1 -- 172.21.15.69:0/384073311 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f40fe070 con 0x55fcd3be6a20 2024-03-19T01:25:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.564+0000 7ff2ef400640 1 -- 172.21.15.69:0/384073311 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40bd240 con 0x55fcd3be6a20 2024-03-19T01:25:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.565+0000 7ff30cde7740 1 -- 172.21.15.69:0/384073311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 msgr2=0x55fcd3bef6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.565+0000 7ff30cde7740 1 --2- 172.21.15.69:0/384073311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3bef6b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009930 tx=0x7ff2f402bc30 comp rx=0 tx=0).stop 2024-03-19T01:25:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.565+0000 7ff30cde7740 1 -- 172.21.15.69:0/384073311 shutdown_connections 2024-03-19T01:25:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.565+0000 7ff30cde7740 1 --2- 172.21.15.69:0/384073311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3beff20 0x55fcd3c07c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.565+0000 7ff30cde7740 1 --2- 172.21.15.69:0/384073311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bfb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.565+0000 7ff30cde7740 1 --2- 172.21.15.69:0/384073311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3be6a20 0x55fcd3bef6b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.565+0000 7ff30cde7740 1 -- 172.21.15.69:0/384073311 >> 172.21.15.69:0/384073311 conn(0x55fcd3be1150 msgr2=0x55fcd3be01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.565+0000 7ff30cde7740 1 -- 172.21.15.69:0/384073311 shutdown_connections 2024-03-19T01:25:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.565+0000 7ff30cde7740 1 -- 172.21.15.69:0/384073311 wait complete. 2024-03-19T01:25:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.565+0000 7ff30cde7740 1 -- start start 2024-03-19T01:25:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.566+0000 7ff30cde7740 1 Processor -- start 2024-03-19T01:25:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.566+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be6a20 0x55fcd3be9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.566+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bea200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.566+0000 7ff30cde7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3beff20 0x55fcd3bea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.566+0000 7ff30cde7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041b2500 con 0x55fcd3be7340 2024-03-19T01:25:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.566+0000 7ff30cde7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041b1e30 con 0x55fcd3beff20 2024-03-19T01:25:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.566+0000 7ff30cde7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041af070 con 0x55fcd3be6a20 2024-03-19T01:25:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.566+0000 7ff30cde7740 10 client.0 _ll_register_callbacks cb 0x7ff2fc0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:25:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.566+0000 7ff303400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be6a20 0x55fcd3be9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.567+0000 7ff303400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be6a20 0x55fcd3be9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33436/0 (socket says 172.21.15.69:33436) 2024-03-19T01:25:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.567+0000 7ff302a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bea200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.567+0000 7ff303400640 1 -- 172.21.15.69:0/2896128616 learned_addr learned my addr 172.21.15.69:0/2896128616 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.567+0000 7ff303e00640 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3beff20 0x55fcd3bea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.567+0000 7ff302a00640 1 -- 172.21.15.69:0/2896128616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be6a20 msgr2=0x55fcd3be9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.567+0000 7ff302a00640 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be6a20 0x55fcd3be9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.567+0000 7ff302a00640 1 -- 172.21.15.69:0/2896128616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3beff20 msgr2=0x55fcd3bea740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:25:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.567+0000 7ff302a00640 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3beff20 0x55fcd3bea740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.567+0000 7ff302a00640 1 -- 172.21.15.69:0/2896128616 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcd3c65400 con 0x55fcd3be7340 2024-03-19T01:25:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.567+0000 7ff302a00640 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bea200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x55fcd3c02be0 tx=0x7ff2f0004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8308e3c5ed22398a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.567+0000 7ff2ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-03-19T01:25:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.568+0000 7ff2ecc00640 1 -- 172.21.15.69:0/2896128616 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0023070 con 0x55fcd3be7340 2024-03-19T01:25:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.568+0000 7ff2ecc00640 1 -- 172.21.15.69:0/2896128616 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7ff2f001f600 con 0x55fcd3be7340 2024-03-19T01:25:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.568+0000 7ff30cde7740 1 -- 172.21.15.69:0/2896128616 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f40fe070 con 0x55fcd3be7340 2024-03-19T01:25:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.568+0000 7ff2ecc00640 1 -- 172.21.15.69:0/2896128616 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff2f002f050 con 0x55fcd3be7340 2024-03-19T01:25:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.568+0000 7ff30cde7740 10 client.17937 fetch_fsmap learned FSMap version 59 2024-03-19T01:25:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.568+0000 7ff30cde7740 1 -- 172.21.15.69:0/2896128616 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55fcd3c66720 con 0x55fcd3be7340 2024-03-19T01:25:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.568+0000 7ff2ecc00640 1 -- 172.21.15.69:0/2896128616 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f40fe070 con 0x55fcd3be7340 2024-03-19T01:25:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.569+0000 7ff2ecc00640 1 -- 172.21.15.69:0/2896128616 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 59) ==== 8362+0+0 (secure 0 0 0) 0x7ff2f0010430 con 0x55fcd3be7340 2024-03-19T01:25:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.569+0000 7ff30cde7740 10 client.17937 fetch_fsmap finished waiting for FSMap version 59 2024-03-19T01:25:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.569+0000 7ff30cde7740 10 client.17937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-19T01:25:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.569+0000 7ff30cde7740 20 client.17937 populate_metadata read hostname 'smithi069' 2024-03-19T01:25:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.569+0000 7ff30cde7740 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fcd3c66e50 0x55fcd3c69270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.569+0000 7ff30cde7740 4 client.17937 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-19T01:25:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.569+0000 7ff30cde7740 1 -- 172.21.15.69:0/2896128616 --> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fcd3b52c10 con 0x55fcd3c66e50 2024-03-19T01:25:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.569+0000 7ff303400640 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fcd3c66e50 0x55fcd3c69270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:30.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff2ecc00640 10 client.17937 ms_handle_connect on v2:172.21.15.86:6838/621579864 2024-03-19T01:25:30.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff303400640 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fcd3c66e50 0x55fcd3c69270 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=317f834c0f0c8e0a server_cookie=8a553dbb9b3d2423 in_seq=0 out_seq=0 2024-03-19T01:25:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff2ecc00640 1 -- 172.21.15.69:0/2896128616 <== mds.0 v2:172.21.15.86:6838/621579864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fcd3b52c10 con 0x55fcd3c66e50 2024-03-19T01:25:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff2ecc00640 10 client.17937 handle_command_reply: tid=0 2024-03-19T01:25:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff30cde7740 1 client.17937 shutdown 2024-03-19T01:25:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff30cde7740 1 -- 172.21.15.69:0/2896128616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 msgr2=0x55fcd3bea200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff30cde7740 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bea200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x55fcd3c02be0 tx=0x7ff2f0004560 comp rx=0 tx=0).stop 2024-03-19T01:25:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff30cde7740 1 -- 172.21.15.69:0/2896128616 shutdown_connections 2024-03-19T01:25:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff30cde7740 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.86:6838/621579864,v1:172.21.15.86:6839/621579864] conn(0x55fcd3c66e50 0x55fcd3c69270 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff30cde7740 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fcd3beff20 0x55fcd3bea740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff30cde7740 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fcd3be7340 0x55fcd3bea200 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff30cde7740 1 --2- 172.21.15.69:0/2896128616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fcd3be6a20 0x55fcd3be9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff30cde7740 1 -- 172.21.15.69:0/2896128616 >> 172.21.15.69:0/2896128616 conn(0x55fcd3be1150 msgr2=0x55fcd3be24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.570+0000 7ff2ecc00640 0 client.17937 ms_handle_reset on v2:172.21.15.86:6838/621579864 2024-03-19T01:25:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.572+0000 7ff30cde7740 1 -- 172.21.15.69:0/2896128616 shutdown_connections 2024-03-19T01:25:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.572+0000 7ff30cde7740 1 -- 172.21.15.69:0/2896128616 wait complete. 2024-03-19T01:25:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.572+0000 7ff30cde7740 20 client.17937 trim_cache size 0 max 16384 2024-03-19T01:25:30.579 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-03-19T01:25:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.572+0000 7ff30a000640 1 -- 172.21.15.69:0/1689040727 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2e80a2970 msgr2=0x7ff2e80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.572+0000 7ff30a000640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2e80a2970 0x7ff2e80a4e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0049d0 tx=0x7ff2fc009500 comp rx=0 tx=0).stop 2024-03-19T01:25:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.572+0000 7ff30a000640 1 -- 172.21.15.69:0/1689040727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3041039c0 msgr2=0x7ff30419a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.573+0000 7ff30a000640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3041039c0 0x7ff30419a390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4036040 tx=0x7ff2f4005e80 comp rx=0 tx=0).stop 2024-03-19T01:25:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.573+0000 7ff30a000640 1 -- 172.21.15.69:0/1689040727 shutdown_connections 2024-03-19T01:25:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.573+0000 7ff30a000640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3041099d0 0x7ff30419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.573+0000 7ff30a000640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff2e80a2970 0x7ff2e80a4e30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.573+0000 7ff30a000640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff304104380 0x7ff30419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.573+0000 7ff30a000640 1 --2- 172.21.15.69:0/1689040727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3041039c0 0x7ff30419a390 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.573+0000 7ff30a000640 1 -- 172.21.15.69:0/1689040727 >> 172.21.15.69:0/1689040727 conn(0x7ff3040ff4d0 msgr2=0x7ff3040ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.573+0000 7ff30a000640 1 -- 172.21.15.69:0/1689040727 shutdown_connections 2024-03-19T01:25:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:30.573+0000 7ff30a000640 1 -- 172.21.15.69:0/1689040727 wait complete. 2024-03-19T01:25:31.517 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:31 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:25:31.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:31 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:25:31.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:31 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:25:31.592 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-19T01:25:31.593 INFO:tasks.fwd_scrub:done joining 2024-03-19T01:25:31.593 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-19T01:25:31.604 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:25:31.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.817+0000 7ff217400640 1 -- 172.21.15.69:0/229863653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff210073bf0 msgr2=0x7ff210073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:31.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.817+0000 7ff217400640 1 --2- 172.21.15.69:0/229863653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff210073bf0 0x7ff210073fd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff204009900 tx=0x7ff20402bcc0 comp rx=0 tx=0).stop 2024-03-19T01:25:31.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.817+0000 7ff217400640 1 -- 172.21.15.69:0/229863653 shutdown_connections 2024-03-19T01:25:31.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.817+0000 7ff217400640 1 --2- 172.21.15.69:0/229863653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff21010e050 0x7ff210110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:31.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.817+0000 7ff217400640 1 --2- 172.21.15.69:0/229863653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff210074510 0x7ff21010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:31.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.817+0000 7ff217400640 1 --2- 172.21.15.69:0/229863653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff210073bf0 0x7ff210073fd0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:31.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.817+0000 7ff217400640 1 -- 172.21.15.69:0/229863653 >> 172.21.15.69:0/229863653 conn(0x7ff2100fd2c0 msgr2=0x7ff2100ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:31.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.817+0000 7ff217400640 1 -- 172.21.15.69:0/229863653 shutdown_connections 2024-03-19T01:25:31.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.817+0000 7ff217400640 1 -- 172.21.15.69:0/229863653 wait complete. 2024-03-19T01:25:31.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.818+0000 7ff217400640 1 Processor -- start 2024-03-19T01:25:31.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.818+0000 7ff217400640 1 -- start start 2024-03-19T01:25:31.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.819+0000 7ff217400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff210073bf0 0x7ff21019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:31.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.819+0000 7ff217400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff210074510 0x7ff21019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:31.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.819+0000 7ff217400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21010e050 0x7ff21019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:31.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.819+0000 7ff217400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff210113e00 con 0x7ff210074510 2024-03-19T01:25:31.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.819+0000 7ff217400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff210113c80 con 0x7ff210073bf0 2024-03-19T01:25:31.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.819+0000 7ff217400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff210113f80 con 0x7ff21010e050 2024-03-19T01:25:31.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.819+0000 7ff214a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff210073bf0 0x7ff21019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:31.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.819+0000 7ff214a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff210073bf0 0x7ff21019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54234/0 (socket says 172.21.15.69:54234) 2024-03-19T01:25:31.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.819+0000 7ff214a00640 1 -- 172.21.15.69:0/622770681 learned_addr learned my addr 172.21.15.69:0/622770681 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:31.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.819+0000 7ff215400640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21010e050 0x7ff21019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:31.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.819+0000 7ff20fe00640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff210074510 0x7ff21019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:31.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.820+0000 7ff214a00640 1 -- 172.21.15.69:0/622770681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21010e050 msgr2=0x7ff21019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:31.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.820+0000 7ff214a00640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21010e050 0x7ff21019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:31.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.820+0000 7ff214a00640 1 -- 172.21.15.69:0/622770681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff210074510 msgr2=0x7ff21019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:31.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.820+0000 7ff214a00640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff210074510 0x7ff21019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:31.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.820+0000 7ff214a00640 1 -- 172.21.15.69:0/622770681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21019f320 con 0x7ff210073bf0 2024-03-19T01:25:31.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.820+0000 7ff215400640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21010e050 0x7ff21019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:31.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.820+0000 7ff214a00640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff210073bf0 0x7ff21019a380 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff204000c00 tx=0x7ff2040043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:31.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.820+0000 7ff20d600640 1 -- 172.21.15.69:0/622770681 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204042020 con 0x7ff210073bf0 2024-03-19T01:25:31.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.820+0000 7ff20d600640 1 -- 172.21.15.69:0/622770681 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff204005000 con 0x7ff210073bf0 2024-03-19T01:25:31.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.820+0000 7ff217400640 1 -- 172.21.15.69:0/622770681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21019f5b0 con 0x7ff210073bf0 2024-03-19T01:25:31.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.820+0000 7ff20d600640 1 -- 172.21.15.69:0/622770681 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204003b20 con 0x7ff210073bf0 2024-03-19T01:25:31.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.821+0000 7ff217400640 1 -- 172.21.15.69:0/622770681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2101a6ea0 con 0x7ff210073bf0 2024-03-19T01:25:31.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.822+0000 7ff217400640 1 -- 172.21.15.69:0/622770681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff210068a60 con 0x7ff210073bf0 2024-03-19T01:25:31.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.824+0000 7ff20d600640 1 -- 172.21.15.69:0/622770681 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ff204045020 con 0x7ff210073bf0 2024-03-19T01:25:31.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.826+0000 7ff20d600640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1f80a29f0 0x7ff1f80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:31.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.826+0000 7ff20d600640 1 -- 172.21.15.69:0/622770681 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7ff2040f0df0 con 0x7ff210073bf0 2024-03-19T01:25:31.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.826+0000 7ff20fe00640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1f80a29f0 0x7ff1f80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:31.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.827+0000 7ff20fe00640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1f80a29f0 0x7ff1f80a4eb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff2100ff950 tx=0x7ff1fc00b390 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:31.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:31.830+0000 7ff20d600640 1 -- 172.21.15.69:0/622770681 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff20404c020 con 0x7ff210073bf0 2024-03-19T01:25:32.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.035+0000 7ff217400640 1 -- 172.21.15.69:0/622770681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff21019b680 con 0x7ff210073bf0 2024-03-19T01:25:32.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.037+0000 7ff20d600640 1 -- 172.21.15.69:0/622770681 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+25364 (secure 0 0 0) 0x7ff2040bd810 con 0x7ff210073bf0 2024-03-19T01:25:32.038 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:25:32.039 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":112,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:25:19.113550+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3960674803":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/3562156769":"2024-03-20T01:10:40.006231+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6801/3562156769":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2535372856":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2291738694":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2266394714":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2207308634":"2024-03-20T01:10:40.006231+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]},{"pool":4,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:25:32.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.040+0000 7ff217400640 1 -- 172.21.15.69:0/622770681 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1f80a29f0 msgr2=0x7ff1f80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:32.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.040+0000 7ff217400640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1f80a29f0 0x7ff1f80a4eb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff2100ff950 tx=0x7ff1fc00b390 comp rx=0 tx=0).stop 2024-03-19T01:25:32.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.040+0000 7ff217400640 1 -- 172.21.15.69:0/622770681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff210073bf0 msgr2=0x7ff21019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:32.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.040+0000 7ff217400640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff210073bf0 0x7ff21019a380 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff204000c00 tx=0x7ff2040043c0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.040+0000 7ff217400640 1 -- 172.21.15.69:0/622770681 shutdown_connections 2024-03-19T01:25:32.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.040+0000 7ff217400640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21010e050 0x7ff21019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.040+0000 7ff217400640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff210074510 0x7ff21019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.040+0000 7ff217400640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff1f80a29f0 0x7ff1f80a4eb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.041+0000 7ff217400640 1 --2- 172.21.15.69:0/622770681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff210073bf0 0x7ff21019a380 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.041+0000 7ff217400640 1 -- 172.21.15.69:0/622770681 >> 172.21.15.69:0/622770681 conn(0x7ff2100fd2c0 msgr2=0x7ff2100feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:32.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.041+0000 7ff217400640 1 -- 172.21.15.69:0/622770681 shutdown_connections 2024-03-19T01:25:32.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.041+0000 7ff217400640 1 -- 172.21.15.69:0/622770681 wait complete. 2024-03-19T01:25:32.063 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:07:13.368130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:07:14.444206+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 111, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T01:07:18.189146+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 111, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T01:25:32.064 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:25:32.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.266+0000 7fdf34000640 1 -- 172.21.15.69:0/3293066906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2c1067f0 msgr2=0x7fdf2c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:32.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.266+0000 7fdf34000640 1 --2- 172.21.15.69:0/3293066906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2c1067f0 0x7fdf2c106bd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009930 tx=0x7fdf2002bc20 comp rx=0 tx=0).stop 2024-03-19T01:25:32.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.267+0000 7fdf34000640 1 -- 172.21.15.69:0/3293066906 shutdown_connections 2024-03-19T01:25:32.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.267+0000 7fdf34000640 1 --2- 172.21.15.69:0/3293066906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf2c100ad0 0x7fdf2c10e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.267+0000 7fdf34000640 1 --2- 172.21.15.69:0/3293066906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf2c100110 0x7fdf2c100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.267+0000 7fdf34000640 1 --2- 172.21.15.69:0/3293066906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2c1067f0 0x7fdf2c106bd0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.267+0000 7fdf34000640 1 -- 172.21.15.69:0/3293066906 >> 172.21.15.69:0/3293066906 conn(0x7fdf2c075a70 msgr2=0x7fdf2c075e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:32.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.267+0000 7fdf34000640 1 -- 172.21.15.69:0/3293066906 shutdown_connections 2024-03-19T01:25:32.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.267+0000 7fdf34000640 1 -- 172.21.15.69:0/3293066906 wait complete. 2024-03-19T01:25:32.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.268+0000 7fdf34000640 1 Processor -- start 2024-03-19T01:25:32.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.268+0000 7fdf34000640 1 -- start start 2024-03-19T01:25:32.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.268+0000 7fdf34000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2c100110 0x7fdf2c1012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:32.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.268+0000 7fdf34000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf2c100ad0 0x7fdf2c101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:32.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.268+0000 7fdf34000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf2c1067f0 0x7fdf2c101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:32.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.268+0000 7fdf34000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c111cc0 con 0x7fdf2c1067f0 2024-03-19T01:25:32.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.268+0000 7fdf34000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c111b40 con 0x7fdf2c100ad0 2024-03-19T01:25:32.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.268+0000 7fdf34000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c111e40 con 0x7fdf2c100110 2024-03-19T01:25:32.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.268+0000 7fdf31600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2c100110 0x7fdf2c1012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:32.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.268+0000 7fdf31600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2c100110 0x7fdf2c1012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33466/0 (socket says 172.21.15.69:33466) 2024-03-19T01:25:32.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.269+0000 7fdf31600640 1 -- 172.21.15.69:0/3974963227 learned_addr learned my addr 172.21.15.69:0/3974963227 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:32.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.269+0000 7fdf30c00640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf2c100ad0 0x7fdf2c101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:32.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.269+0000 7fdf32000640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf2c1067f0 0x7fdf2c101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:32.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.269+0000 7fdf31600640 1 -- 172.21.15.69:0/3974963227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf2c100ad0 msgr2=0x7fdf2c101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:32.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.269+0000 7fdf31600640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf2c100ad0 0x7fdf2c101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.269+0000 7fdf31600640 1 -- 172.21.15.69:0/3974963227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf2c1067f0 msgr2=0x7fdf2c101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:32.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.269+0000 7fdf31600640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf2c1067f0 0x7fdf2c101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.269+0000 7fdf31600640 1 -- 172.21.15.69:0/3974963227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c1a66e0 con 0x7fdf2c100110 2024-03-19T01:25:32.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.269+0000 7fdf31600640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2c100110 0x7fdf2c1012f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009430 tx=0x7fdf2002cb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:32.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.270+0000 7fdf2a000640 1 -- 172.21.15.69:0/3974963227 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20041070 con 0x7fdf2c100110 2024-03-19T01:25:32.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.270+0000 7fdf2a000640 1 -- 172.21.15.69:0/3974963227 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf20004450 con 0x7fdf2c100110 2024-03-19T01:25:32.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.270+0000 7fdf2a000640 1 -- 172.21.15.69:0/3974963227 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20003dc0 con 0x7fdf2c100110 2024-03-19T01:25:32.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.270+0000 7fdf34000640 1 -- 172.21.15.69:0/3974963227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2c1a6970 con 0x7fdf2c100110 2024-03-19T01:25:32.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.270+0000 7fdf34000640 1 -- 172.21.15.69:0/3974963227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2c1a6e10 con 0x7fdf2c100110 2024-03-19T01:25:32.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.271+0000 7fdf34000640 1 -- 172.21.15.69:0/3974963227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf2c068a60 con 0x7fdf2c100110 2024-03-19T01:25:32.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.274+0000 7fdf2a000640 1 -- 172.21.15.69:0/3974963227 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fdf20044070 con 0x7fdf2c100110 2024-03-19T01:25:32.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.275+0000 7fdf2a000640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdf140a29f0 0x7fdf140a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:32.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.276+0000 7fdf2a000640 1 -- 172.21.15.69:0/3974963227 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7fdf200f0b70 con 0x7fdf2c100110 2024-03-19T01:25:32.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.276+0000 7fdf30c00640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdf140a29f0 0x7fdf140a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:32.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.276+0000 7fdf30c00640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdf140a29f0 0x7fdf140a4eb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c102d80 tx=0x7fdf18005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:32.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.280+0000 7fdf2a000640 1 -- 172.21.15.69:0/3974963227 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdf200bd5d0 con 0x7fdf2c100110 2024-03-19T01:25:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:32 smithi086 ceph-mon[66829]: pgmap v466: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-03-19T01:25:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:32 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:25:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:32 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:25:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:32 smithi086 ceph-mon[66829]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:25:32.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:32 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/622770681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:25:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:32 smithi069 ceph-mon[31727]: pgmap v466: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-03-19T01:25:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:32 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:25:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:32 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:25:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:32 smithi069 ceph-mon[31727]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:25:32.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:32 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/622770681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:25:32.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.560+0000 7fdf34000640 1 -- 172.21.15.69:0/3974963227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf2c109cd0 con 0x7fdf2c100110 2024-03-19T01:25:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:32 smithi163 ceph-mon[67428]: pgmap v466: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-03-19T01:25:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:32 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:25:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:32 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:25:32.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:32 smithi163 ceph-mon[67428]: from='mgr.25645 ' entity='mgr.x' 2024-03-19T01:25:32.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:32 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/622770681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:25:32.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.561+0000 7fdf2a000640 1 -- 172.21.15.69:0/3974963227 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7fdf200bced0 con 0x7fdf2c100110 2024-03-19T01:25:32.563 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:25:32.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.563+0000 7fdf34000640 1 -- 172.21.15.69:0/3974963227 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdf140a29f0 msgr2=0x7fdf140a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:32.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.563+0000 7fdf34000640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdf140a29f0 0x7fdf140a4eb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c102d80 tx=0x7fdf18005f90 comp rx=0 tx=0).stop 2024-03-19T01:25:32.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.563+0000 7fdf34000640 1 -- 172.21.15.69:0/3974963227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2c100110 msgr2=0x7fdf2c1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:32.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.563+0000 7fdf34000640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2c100110 0x7fdf2c1012f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009430 tx=0x7fdf2002cb10 comp rx=0 tx=0).stop 2024-03-19T01:25:32.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.564+0000 7fdf34000640 1 -- 172.21.15.69:0/3974963227 shutdown_connections 2024-03-19T01:25:32.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.564+0000 7fdf34000640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf2c1067f0 0x7fdf2c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.564+0000 7fdf34000640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fdf140a29f0 0x7fdf140a4eb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.564+0000 7fdf34000640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf2c100ad0 0x7fdf2c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.564+0000 7fdf34000640 1 --2- 172.21.15.69:0/3974963227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2c100110 0x7fdf2c1012f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.564+0000 7fdf34000640 1 -- 172.21.15.69:0/3974963227 >> 172.21.15.69:0/3974963227 conn(0x7fdf2c075a70 msgr2=0x7fdf2c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:32.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.564+0000 7fdf34000640 1 -- 172.21.15.69:0/3974963227 shutdown_connections 2024-03-19T01:25:32.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.564+0000 7fdf34000640 1 -- 172.21.15.69:0/3974963227 wait complete. 2024-03-19T01:25:32.590 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:25:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.789+0000 7ffbe0800640 1 -- 172.21.15.69:0/3788279633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbd8103380 msgr2=0x7ffbd810b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.789+0000 7ffbe0800640 1 --2- 172.21.15.69:0/3788279633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbd8103380 0x7ffbd810b860 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009900 tx=0x7ffbc802bcc0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.789+0000 7ffbe0800640 1 -- 172.21.15.69:0/3788279633 shutdown_connections 2024-03-19T01:25:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.789+0000 7ffbe0800640 1 --2- 172.21.15.69:0/3788279633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd810be30 0x7ffbd810e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.789+0000 7ffbe0800640 1 --2- 172.21.15.69:0/3788279633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbd8103380 0x7ffbd810b860 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.789+0000 7ffbe0800640 1 --2- 172.21.15.69:0/3788279633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8102a60 0x7ffbd8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.789+0000 7ffbe0800640 1 -- 172.21.15.69:0/3788279633 >> 172.21.15.69:0/3788279633 conn(0x7ffbd8076aa0 msgr2=0x7ffbd8076eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.789+0000 7ffbe0800640 1 -- 172.21.15.69:0/3788279633 shutdown_connections 2024-03-19T01:25:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.789+0000 7ffbe0800640 1 -- 172.21.15.69:0/3788279633 wait complete. 2024-03-19T01:25:32.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.790+0000 7ffbe0800640 1 Processor -- start 2024-03-19T01:25:32.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.791+0000 7ffbe0800640 1 -- start start 2024-03-19T01:25:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.791+0000 7ffbe0800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd8102a60 0x7ffbd819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.791+0000 7ffbe0800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8103380 0x7ffbd819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.791+0000 7ffbe0800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbd810be30 0x7ffbd819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.791+0000 7ffbe0800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8111c10 con 0x7ffbd8102a60 2024-03-19T01:25:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.791+0000 7ffbe0800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8111a90 con 0x7ffbd8103380 2024-03-19T01:25:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.791+0000 7ffbe0800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8111d90 con 0x7ffbd810be30 2024-03-19T01:25:32.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.791+0000 7ffbdd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8103380 0x7ffbd819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:32.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.791+0000 7ffbdde00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd8102a60 0x7ffbd819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:32.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.791+0000 7ffbde800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbd810be30 0x7ffbd819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:32.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.792+0000 7ffbdd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8103380 0x7ffbd819a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54252/0 (socket says 172.21.15.69:54252) 2024-03-19T01:25:32.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.792+0000 7ffbde800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbd810be30 0x7ffbd819ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33480/0 (socket says 172.21.15.69:33480) 2024-03-19T01:25:32.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.792+0000 7ffbdd400640 1 -- 172.21.15.69:0/2939067547 learned_addr learned my addr 172.21.15.69:0/2939067547 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:32.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.792+0000 7ffbdd400640 1 -- 172.21.15.69:0/2939067547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbd810be30 msgr2=0x7ffbd819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:32.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.792+0000 7ffbdd400640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbd810be30 0x7ffbd819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.792+0000 7ffbdd400640 1 -- 172.21.15.69:0/2939067547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd8102a60 msgr2=0x7ffbd819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:32.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.792+0000 7ffbdd400640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd8102a60 0x7ffbd819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:32.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.792+0000 7ffbdd400640 1 -- 172.21.15.69:0/2939067547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd819f2f0 con 0x7ffbd8103380 2024-03-19T01:25:32.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.792+0000 7ffbde800640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbd810be30 0x7ffbd819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:32.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.792+0000 7ffbdd400640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8103380 0x7ffbd819a840 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8000c00 tx=0x7ffbc8004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:32.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.792+0000 7ffbd2a00640 1 -- 172.21.15.69:0/2939067547 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8042020 con 0x7ffbd8103380 2024-03-19T01:25:32.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.793+0000 7ffbd2a00640 1 -- 172.21.15.69:0/2939067547 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc8005000 con 0x7ffbd8103380 2024-03-19T01:25:32.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.793+0000 7ffbe0800640 1 -- 172.21.15.69:0/2939067547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd819f580 con 0x7ffbd8103380 2024-03-19T01:25:32.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.793+0000 7ffbd2a00640 1 -- 172.21.15.69:0/2939067547 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8003b20 con 0x7ffbd8103380 2024-03-19T01:25:32.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.793+0000 7ffbe0800640 1 -- 172.21.15.69:0/2939067547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd81a6e70 con 0x7ffbd8103380 2024-03-19T01:25:32.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.794+0000 7ffbe0800640 1 -- 172.21.15.69:0/2939067547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd806a610 con 0x7ffbd8103380 2024-03-19T01:25:32.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.796+0000 7ffbd2a00640 1 -- 172.21.15.69:0/2939067547 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ffbc8045020 con 0x7ffbd8103380 2024-03-19T01:25:32.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.797+0000 7ffbd2a00640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffbc00a29f0 0x7ffbc00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:32.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.797+0000 7ffbd2a00640 1 -- 172.21.15.69:0/2939067547 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7ffbc80f0c80 con 0x7ffbd8103380 2024-03-19T01:25:32.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.797+0000 7ffbdde00640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffbc00a29f0 0x7ffbc00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:32.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.798+0000 7ffbdde00640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffbc00a29f0 0x7ffbc00a4eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009bd0 tx=0x7ffbcc005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:32.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:32.801+0000 7ffbd2a00640 1 -- 172.21.15.69:0/2939067547 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffbc804c020 con 0x7ffbd8103380 2024-03-19T01:25:33.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.047+0000 7ffbe0800640 1 -- 172.21.15.69:0/2939067547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffbd819b650 con 0x7ffbd8103380 2024-03-19T01:25:33.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.047+0000 7ffbd2a00640 1 -- 172.21.15.69:0/2939067547 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7ffbc80bd6a0 con 0x7ffbd8103380 2024-03-19T01:25:33.049 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:25:33.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 -- 172.21.15.69:0/2939067547 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffbc00a29f0 msgr2=0x7ffbc00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffbc00a29f0 0x7ffbc00a4eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009bd0 tx=0x7ffbcc005f90 comp rx=0 tx=0).stop 2024-03-19T01:25:33.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 -- 172.21.15.69:0/2939067547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8103380 msgr2=0x7ffbd819a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8103380 0x7ffbd819a840 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8000c00 tx=0x7ffbc8004360 comp rx=0 tx=0).stop 2024-03-19T01:25:33.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 -- 172.21.15.69:0/2939067547 shutdown_connections 2024-03-19T01:25:33.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbd810be30 0x7ffbd819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffbc00a29f0 0x7ffbc00a4eb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8103380 0x7ffbd819a840 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 --2- 172.21.15.69:0/2939067547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd8102a60 0x7ffbd819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 -- 172.21.15.69:0/2939067547 >> 172.21.15.69:0/2939067547 conn(0x7ffbd8076aa0 msgr2=0x7ffbd81017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:33.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 -- 172.21.15.69:0/2939067547 shutdown_connections 2024-03-19T01:25:33.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.050+0000 7ffbe0800640 1 -- 172.21.15.69:0/2939067547 wait complete. 2024-03-19T01:25:33.071 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:25:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.264+0000 7fb09d800640 1 -- 172.21.15.69:0/3525940331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0981039c0 msgr2=0x7fb098103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.264+0000 7fb09d800640 1 --2- 172.21.15.69:0/3525940331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0981039c0 0x7fb098103e40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb080009930 tx=0x7fb08002bc30 comp rx=0 tx=0).stop 2024-03-19T01:25:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.264+0000 7fb09d800640 1 -- 172.21.15.69:0/3525940331 shutdown_connections 2024-03-19T01:25:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.264+0000 7fb09d800640 1 --2- 172.21.15.69:0/3525940331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb098104380 0x7fb098110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.264+0000 7fb09d800640 1 --2- 172.21.15.69:0/3525940331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0981039c0 0x7fb098103e40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.264+0000 7fb09d800640 1 --2- 172.21.15.69:0/3525940331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0981099d0 0x7fb098109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.264+0000 7fb09d800640 1 -- 172.21.15.69:0/3525940331 >> 172.21.15.69:0/3525940331 conn(0x7fb0980ff490 msgr2=0x7fb0981018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.264+0000 7fb09d800640 1 -- 172.21.15.69:0/3525940331 shutdown_connections 2024-03-19T01:25:33.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.264+0000 7fb09d800640 1 -- 172.21.15.69:0/3525940331 wait complete. 2024-03-19T01:25:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.265+0000 7fb09d800640 1 Processor -- start 2024-03-19T01:25:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.265+0000 7fb09d800640 1 -- start start 2024-03-19T01:25:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.265+0000 7fb09d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0981039c0 0x7fb09819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:33.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.265+0000 7fb09d800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb098104380 0x7fb09819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb09d800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0981099d0 0x7fb09819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb09d800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb098113e30 con 0x7fb098104380 2024-03-19T01:25:33.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb09d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb098113cb0 con 0x7fb0981039c0 2024-03-19T01:25:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb09d800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb098113fb0 con 0x7fb0981099d0 2024-03-19T01:25:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb096a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0981039c0 0x7fb09819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb096a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0981039c0 0x7fb09819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54264/0 (socket says 172.21.15.69:54264) 2024-03-19T01:25:33.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb097400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0981099d0 0x7fb09819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb096000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb098104380 0x7fb09819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb096a00640 1 -- 172.21.15.69:0/3246216330 learned_addr learned my addr 172.21.15.69:0/3246216330 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:33.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb096a00640 1 -- 172.21.15.69:0/3246216330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0981099d0 msgr2=0x7fb09819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb096a00640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0981099d0 0x7fb09819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb096a00640 1 -- 172.21.15.69:0/3246216330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb098104380 msgr2=0x7fb09819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb096a00640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb098104380 0x7fb09819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb096a00640 1 -- 172.21.15.69:0/3246216330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09819f330 con 0x7fb0981039c0 2024-03-19T01:25:33.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.266+0000 7fb097400640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0981099d0 0x7fb09819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:33.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.267+0000 7fb096a00640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0981039c0 0x7fb09819a390 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb08800eb90 tx=0x7fb08800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:33.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.267+0000 7fb08f400640 1 -- 172.21.15.69:0/3246216330 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088004d20 con 0x7fb0981039c0 2024-03-19T01:25:33.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.267+0000 7fb08f400640 1 -- 172.21.15.69:0/3246216330 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb08800ee90 con 0x7fb0981039c0 2024-03-19T01:25:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.267+0000 7fb09d800640 1 -- 172.21.15.69:0/3246216330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09819f620 con 0x7fb0981039c0 2024-03-19T01:25:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.267+0000 7fb08f400640 1 -- 172.21.15.69:0/3246216330 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088010830 con 0x7fb0981039c0 2024-03-19T01:25:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.267+0000 7fb09d800640 1 -- 172.21.15.69:0/3246216330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0981a6eb0 con 0x7fb0981039c0 2024-03-19T01:25:33.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.268+0000 7fb09d800640 1 -- 172.21.15.69:0/3246216330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb098068a60 con 0x7fb0981039c0 2024-03-19T01:25:33.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.270+0000 7fb08f400640 1 -- 172.21.15.69:0/3246216330 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fb088005020 con 0x7fb0981039c0 2024-03-19T01:25:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.271+0000 7fb08f400640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb07c0a29f0 0x7fb07c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.271+0000 7fb08f400640 1 -- 172.21.15.69:0/3246216330 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7fb0880cf790 con 0x7fb0981039c0 2024-03-19T01:25:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.271+0000 7fb096000640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb07c0a29f0 0x7fb07c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:33.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.272+0000 7fb096000640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb07c0a29f0 0x7fb07c0a4eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb080009900 tx=0x7fb08002df90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:33.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.275+0000 7fb08f400640 1 -- 172.21.15.69:0/3246216330 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb08809c0e0 con 0x7fb0981039c0 2024-03-19T01:25:33.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.487+0000 7fb09d800640 1 -- 172.21.15.69:0/3246216330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb09802d1f0 con 0x7fb0981039c0 2024-03-19T01:25:33.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.488+0000 7fb08f400640 1 -- 172.21.15.69:0/3246216330 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) ==== 100+0+11 (secure 0 0 0) 0x7fb08809b9e0 con 0x7fb0981039c0 2024-03-19T01:25:33.489 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-03-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.490+0000 7fb09d800640 1 -- 172.21.15.69:0/3246216330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb07c0a29f0 msgr2=0x7fb07c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.490+0000 7fb09d800640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb07c0a29f0 0x7fb07c0a4eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb080009900 tx=0x7fb08002df90 comp rx=0 tx=0).stop 2024-03-19T01:25:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.490+0000 7fb09d800640 1 -- 172.21.15.69:0/3246216330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0981039c0 msgr2=0x7fb09819a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.490+0000 7fb09d800640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0981039c0 0x7fb09819a390 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb08800eb90 tx=0x7fb08800c6a0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.490+0000 7fb09d800640 1 -- 172.21.15.69:0/3246216330 shutdown_connections 2024-03-19T01:25:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.490+0000 7fb09d800640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0981099d0 0x7fb09819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.490+0000 7fb09d800640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb098104380 0x7fb09819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.490+0000 7fb09d800640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fb07c0a29f0 0x7fb07c0a4eb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.490+0000 7fb09d800640 1 --2- 172.21.15.69:0/3246216330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0981039c0 0x7fb09819a390 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.490+0000 7fb09d800640 1 -- 172.21.15.69:0/3246216330 >> 172.21.15.69:0/3246216330 conn(0x7fb0980ff490 msgr2=0x7fb0980ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.490+0000 7fb09d800640 1 -- 172.21.15.69:0/3246216330 shutdown_connections 2024-03-19T01:25:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.491+0000 7fb09d800640 1 -- 172.21.15.69:0/3246216330 wait complete. 2024-03-19T01:25:33.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:33 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3974963227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:25:33.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:33 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2939067547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:25:33.506 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:33 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3974963227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:25:33.506 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:33 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2939067547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:25:33.509 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T01:25:33.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:33 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3974963227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:25:33.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:33 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2939067547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:25:33.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.711+0000 7fe1f3e00640 1 -- 172.21.15.69:0/474069507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f4100460 msgr2=0x7fe1f4100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.711+0000 7fe1f3e00640 1 --2- 172.21.15.69:0/474069507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f4100460 0x7fe1f4100840 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009be0 tx=0x7fe1e402bef0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.711+0000 7fe1f3e00640 1 -- 172.21.15.69:0/474069507 shutdown_connections 2024-03-19T01:25:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.711+0000 7fe1f3e00640 1 --2- 172.21.15.69:0/474069507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f410e010 0x7fe1f4110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.711+0000 7fe1f3e00640 1 --2- 172.21.15.69:0/474069507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f4100e10 0x7fe1f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.711+0000 7fe1f3e00640 1 --2- 172.21.15.69:0/474069507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f4100460 0x7fe1f4100840 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.711+0000 7fe1f3e00640 1 -- 172.21.15.69:0/474069507 >> 172.21.15.69:0/474069507 conn(0x7fe1f40fc070 msgr2=0x7fe1f40fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.712+0000 7fe1f3e00640 1 -- 172.21.15.69:0/474069507 shutdown_connections 2024-03-19T01:25:33.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.712+0000 7fe1f3e00640 1 -- 172.21.15.69:0/474069507 wait complete. 2024-03-19T01:25:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.712+0000 7fe1f3e00640 1 Processor -- start 2024-03-19T01:25:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1f3e00640 1 -- start start 2024-03-19T01:25:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1f3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f4100460 0x7fe1f419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1f3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f4100e10 0x7fe1f419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f410e010 0x7fe1f419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1f3e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4113c80 con 0x7fe1f4100460 2024-03-19T01:25:33.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1f3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4113b00 con 0x7fe1f410e010 2024-03-19T01:25:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1f3e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4113e00 con 0x7fe1f4100e10 2024-03-19T01:25:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1f2a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f4100460 0x7fe1f419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1ea000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f4100e10 0x7fe1f419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1ea000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f4100e10 0x7fe1f419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:33504/0 (socket says 172.21.15.69:33504) 2024-03-19T01:25:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1ea000640 1 -- 172.21.15.69:0/232286882 learned_addr learned my addr 172.21.15.69:0/232286882 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.713+0000 7fe1f3400640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f410e010 0x7fe1f419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.714+0000 7fe1f2a00640 1 -- 172.21.15.69:0/232286882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f4100e10 msgr2=0x7fe1f419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.714+0000 7fe1f2a00640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f4100e10 0x7fe1f419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.714+0000 7fe1f2a00640 1 -- 172.21.15.69:0/232286882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f410e010 msgr2=0x7fe1f419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.714+0000 7fe1f2a00640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f410e010 0x7fe1f419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.714+0000 7fe1f2a00640 1 -- 172.21.15.69:0/232286882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f419f1b0 con 0x7fe1f4100460 2024-03-19T01:25:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.714+0000 7fe1ea000640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f4100e10 0x7fe1f419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.714+0000 7fe1f3400640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f410e010 0x7fe1f419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:25:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.714+0000 7fe1f2a00640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f4100460 0x7fe1f419a1c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4000f80 tx=0x7fe1e4033730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.715+0000 7fe1ebe00640 1 -- 172.21.15.69:0/232286882 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4043020 con 0x7fe1f4100460 2024-03-19T01:25:33.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.715+0000 7fe1ebe00640 1 -- 172.21.15.69:0/232286882 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1e402ca60 con 0x7fe1f4100460 2024-03-19T01:25:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.715+0000 7fe1f3e00640 1 -- 172.21.15.69:0/232286882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f419f440 con 0x7fe1f4100460 2024-03-19T01:25:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.715+0000 7fe1ebe00640 1 -- 172.21.15.69:0/232286882 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e403e6b0 con 0x7fe1f4100460 2024-03-19T01:25:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.715+0000 7fe1f3e00640 1 -- 172.21.15.69:0/232286882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f41a6d30 con 0x7fe1f4100460 2024-03-19T01:25:33.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.716+0000 7fe1f3e00640 1 -- 172.21.15.69:0/232286882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f4068530 con 0x7fe1f4100460 2024-03-19T01:25:33.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.718+0000 7fe1ebe00640 1 -- 172.21.15.69:0/232286882 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fe1e4046020 con 0x7fe1f4100460 2024-03-19T01:25:33.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.720+0000 7fe1ebe00640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1d80a29f0 0x7fe1d80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:33.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.720+0000 7fe1ebe00640 1 -- 172.21.15.69:0/232286882 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7fe1e40f1580 con 0x7fe1f4100460 2024-03-19T01:25:33.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.720+0000 7fe1ea000640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1d80a29f0 0x7fe1d80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:33.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.721+0000 7fe1ea000640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1d80a29f0 0x7fe1d80a4eb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40ff640 tx=0x7fe1dc009260 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:33.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.725+0000 7fe1ebe00640 1 -- 172.21.15.69:0/232286882 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe1e404d020 con 0x7fe1f4100460 2024-03-19T01:25:33.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.953+0000 7fe1f3e00640 1 -- 172.21.15.69:0/232286882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe1f4109110 con 0x7fe1f4100460 2024-03-19T01:25:33.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.954+0000 7fe1ebe00640 1 -- 172.21.15.69:0/232286882 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v112) ==== 103+0+11 (secure 0 0 0) 0x7fe1e40bdfe0 con 0x7fe1f4100460 2024-03-19T01:25:33.955 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:25:33.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.957+0000 7fe1f3e00640 1 -- 172.21.15.69:0/232286882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1d80a29f0 msgr2=0x7fe1d80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.957+0000 7fe1f3e00640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1d80a29f0 0x7fe1d80a4eb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40ff640 tx=0x7fe1dc009260 comp rx=0 tx=0).stop 2024-03-19T01:25:33.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.957+0000 7fe1f3e00640 1 -- 172.21.15.69:0/232286882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f4100460 msgr2=0x7fe1f419a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:33.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.957+0000 7fe1f3e00640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f4100460 0x7fe1f419a1c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4000f80 tx=0x7fe1e4033730 comp rx=0 tx=0).stop 2024-03-19T01:25:33.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.958+0000 7fe1f3e00640 1 -- 172.21.15.69:0/232286882 shutdown_connections 2024-03-19T01:25:33.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.958+0000 7fe1f3e00640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fe1d80a29f0 0x7fe1d80a4eb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.958+0000 7fe1f3e00640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f410e010 0x7fe1f419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.958+0000 7fe1f3e00640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f4100e10 0x7fe1f419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.958+0000 7fe1f3e00640 1 --2- 172.21.15.69:0/232286882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f4100460 0x7fe1f419a1c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:33.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.958+0000 7fe1f3e00640 1 -- 172.21.15.69:0/232286882 >> 172.21.15.69:0/232286882 conn(0x7fe1f40fc070 msgr2=0x7fe1f40fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:33.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.958+0000 7fe1f3e00640 1 -- 172.21.15.69:0/232286882 shutdown_connections 2024-03-19T01:25:33.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:33.958+0000 7fe1f3e00640 1 -- 172.21.15.69:0/232286882 wait complete. 2024-03-19T01:25:33.977 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-19T01:25:34.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.166+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3012890377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a41076b0 msgr2=0x7fc8a4107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:34.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.166+0000 7fc8a8c00640 1 --2- 172.21.15.69:0/3012890377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a41076b0 0x7fc8a4107a90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc894009930 tx=0x7fc89402bc20 comp rx=0 tx=0).stop 2024-03-19T01:25:34.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.166+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3012890377 shutdown_connections 2024-03-19T01:25:34.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.166+0000 7fc8a8c00640 1 --2- 172.21.15.69:0/3012890377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4101fd0 0x7fc8a410e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:34.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.166+0000 7fc8a8c00640 1 --2- 172.21.15.69:0/3012890377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8a4101610 0x7fc8a4101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:34.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.166+0000 7fc8a8c00640 1 --2- 172.21.15.69:0/3012890377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a41076b0 0x7fc8a4107a90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:34.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.166+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3012890377 >> 172.21.15.69:0/3012890377 conn(0x7fc8a40fd100 msgr2=0x7fc8a40ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:34.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.166+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3012890377 shutdown_connections 2024-03-19T01:25:34.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.166+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3012890377 wait complete. 2024-03-19T01:25:34.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.167+0000 7fc8a8c00640 1 Processor -- start 2024-03-19T01:25:34.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.167+0000 7fc8a8c00640 1 -- start start 2024-03-19T01:25:34.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a8c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8a4101610 0x7fc8a419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:34.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4101fd0 0x7fc8a419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:34.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a8c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a41076b0 0x7fc8a419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:34.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a8c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41119f0 con 0x7fc8a4101610 2024-03-19T01:25:34.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4111870 con 0x7fc8a4101fd0 2024-03-19T01:25:34.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a8c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4111b70 con 0x7fc8a41076b0 2024-03-19T01:25:34.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4101fd0 0x7fc8a419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:34.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4101fd0 0x7fc8a419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:54284/0 (socket says 172.21.15.69:54284) 2024-03-19T01:25:34.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a2a00640 1 -- 172.21.15.69:0/3211233041 learned_addr learned my addr 172.21.15.69:0/3211233041 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:25:34.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a3400640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8a4101610 0x7fc8a419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:34.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a3e00640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a41076b0 0x7fc8a419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:34.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a2a00640 1 -- 172.21.15.69:0/3211233041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a41076b0 msgr2=0x7fc8a419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:34.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a2a00640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a41076b0 0x7fc8a419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:34.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a2a00640 1 -- 172.21.15.69:0/3211233041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8a4101610 msgr2=0x7fc8a419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:34.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a2a00640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8a4101610 0x7fc8a419a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:34.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.168+0000 7fc8a2a00640 1 -- 172.21.15.69:0/3211233041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a419f090 con 0x7fc8a4101fd0 2024-03-19T01:25:34.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.169+0000 7fc8a2a00640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4101fd0 0x7fc8a419a5e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9f0 tx=0x7fc89000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:34.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.169+0000 7fc89be00640 1 -- 172.21.15.69:0/3211233041 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89000cd80 con 0x7fc8a4101fd0 2024-03-19T01:25:34.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.169+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3211233041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a419f380 con 0x7fc8a4101fd0 2024-03-19T01:25:34.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.169+0000 7fc89be00640 1 -- 172.21.15.69:0/3211233041 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc890010910 con 0x7fc8a4101fd0 2024-03-19T01:25:34.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.169+0000 7fc89be00640 1 -- 172.21.15.69:0/3211233041 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890018990 con 0x7fc8a4101fd0 2024-03-19T01:25:34.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.169+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3211233041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a41a6c10 con 0x7fc8a4101fd0 2024-03-19T01:25:34.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.170+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3211233041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a4102d50 con 0x7fc8a4101fd0 2024-03-19T01:25:34.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.173+0000 7fc89be00640 1 -- 172.21.15.69:0/3211233041 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fc890010ac0 con 0x7fc8a4101fd0 2024-03-19T01:25:34.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.174+0000 7fc89be00640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc8880a2970 0x7fc8880a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:25:34.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.174+0000 7fc89be00640 1 -- 172.21.15.69:0/3211233041 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9568+0+0 (secure 0 0 0) 0x7fc8900cf5b0 con 0x7fc8a4101fd0 2024-03-19T01:25:34.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.174+0000 7fc8a3400640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc8880a2970 0x7fc8880a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:25:34.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.175+0000 7fc8a3400640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc8880a2970 0x7fc8880a4e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc894005ab0 tx=0x7fc894000bc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:25:34.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.178+0000 7fc89be00640 1 -- 172.21.15.69:0/3211233041 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc89009b8b0 con 0x7fc8a4101fd0 2024-03-19T01:25:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:34 smithi069 ceph-mon[31727]: pgmap v467: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-03-19T01:25:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:34 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3246216330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:25:34.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:34 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/232286882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.464+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3211233041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fc8a41a6e80 con 0x7fc8a4101fd0 2024-03-19T01:25:34.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.479+0000 7fc89be00640 1 -- 172.21.15.69:0/3211233041 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v28) ==== 81+0+740361 (secure 0 0 0) 0x7fc89009f950 con 0x7fc8a4101fd0 2024-03-19T01:25:34.493 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 28, 2024-03-19T01:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "active_gid": 25645, 2024-03-19T01:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "active_name": "x", 2024-03-19T01:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addrs": { 2024-03-19T01:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-03-19T01:25:34.495 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.495 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-03-19T01:25:34.495 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6800", 2024-03-19T01:25:34.495 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2931658273 2024-03-19T01:25:34.495 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.495 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.495 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-03-19T01:25:34.495 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6801", 2024-03-19T01:25:34.495 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2931658273 2024-03-19T01:25:34.496 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.496 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T01:25:34.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.496 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addr": "172.21.15.69:6801/2931658273", 2024-03-19T01:25:34.496 INFO:teuthology.orchestra.run.smithi069.stdout: "active_change": "2024-03-19T01:10:40.006572+0000", 2024-03-19T01:25:34.496 INFO:teuthology.orchestra.run.smithi069.stdout: "active_mgr_features": 4540701547738038271, 2024-03-19T01:25:34.496 INFO:teuthology.orchestra.run.smithi069.stdout: "available": true, 2024-03-19T01:25:34.497 INFO:teuthology.orchestra.run.smithi069.stdout: "standbys": [ 2024-03-19T01:25:34.497 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.497 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 25628, 2024-03-19T01:25:34.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "y", 2024-03-19T01:25:34.497 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540701547738038271, 2024-03-19T01:25:34.497 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-03-19T01:25:34.497 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-03-19T01:25:34.497 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.498 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.498 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.498 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:34.498 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:34.498 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.498 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.498 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.498 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:34.499 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.499 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.499 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-03-19T01:25:34.499 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.499 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.499 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.500 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.500 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.500 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.500 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.500 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.500 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.500 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.500 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.500 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.501 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.501 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.501 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.501 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.501 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.501 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.501 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.501 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.502 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.502 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.502 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.502 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.502 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.502 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.502 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.502 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.502 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.503 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.503 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.503 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.503 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.503 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.503 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.504 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.504 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.504 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.504 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.504 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.504 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.504 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.504 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.505 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.505 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.505 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.505 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.505 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.505 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.505 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.505 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.505 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.506 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.506 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.506 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.506 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.506 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.506 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.506 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.507 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.507 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.507 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.507 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.507 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.507 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.507 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.508 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.508 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-03-19T01:25:34.508 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-03-19T01:25:34.508 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.509 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.509 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.509 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.509 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.509 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-03-19T01:25:34.509 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.510 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.510 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.510 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.510 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-03-19T01:25:34.510 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-03-19T01:25:34.510 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-03-19T01:25:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-03-19T01:25:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-03-19T01:25:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-03-19T01:25:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-03-19T01:25:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-03-19T01:25:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-03-19T01:25:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-03-19T01:25:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-03-19T01:25:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-03-19T01:25:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-03-19T01:25:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-03-19T01:25:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-03-19T01:25:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-03-19T01:25:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-03-19T01:25:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-03-19T01:25:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-03-19T01:25:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-19T01:25:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-03-19T01:25:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-03-19T01:25:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-03-19T01:25:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-03-19T01:25:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-03-19T01:25:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-03-19T01:25:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-03-19T01:25:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-03-19T01:25:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-03-19T01:25:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-03-19T01:25:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-03-19T01:25:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T01:25:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-03-19T01:25:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-03-19T01:25:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-03-19T01:25:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-03-19T01:25:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-19T01:25:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T01:25:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-03-19T01:25:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-03-19T01:25:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-03-19T01:25:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-03-19T01:25:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-19T01:25:34.533 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.533 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.533 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.534 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.534 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-03-19T01:25:34.534 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-03-19T01:25:34.535 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.535 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.535 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.535 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-03-19T01:25:34.535 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.535 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.536 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.536 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-19T01:25:34.536 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-19T01:25:34.536 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.536 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.536 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.537 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-03-19T01:25:34.537 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-03-19T01:25:34.537 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:34.537 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.537 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.537 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-03-19T01:25:34.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-03-19T01:25:34.538 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-03-19T01:25:34.538 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.538 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-03-19T01:25:34.539 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-19T01:25:34.539 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.539 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.539 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-03-19T01:25:34.540 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-03-19T01:25:34.540 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.540 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.540 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.540 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-03-19T01:25:34.541 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.541 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.541 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.541 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-03-19T01:25:34.541 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T01:25:34.541 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.542 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.542 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.542 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-03-19T01:25:34.542 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-03-19T01:25:34.542 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:34.542 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.543 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.543 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:34.543 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-03-19T01:25:34.543 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-03-19T01:25:34.543 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.543 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-19T01:25:34.544 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T01:25:34.544 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.544 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.544 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.544 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.545 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.545 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.545 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.545 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.545 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.546 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.546 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.546 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.546 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.546 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.546 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.547 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.547 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.547 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.547 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.547 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.547 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.548 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.548 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.548 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.548 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.548 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.548 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.549 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.549 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.549 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.549 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.549 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.549 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.549 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.550 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.550 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.550 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.550 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.550 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.550 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.550 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.550 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.551 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.551 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.551 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.551 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.551 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.551 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.551 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.551 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.551 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.552 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.552 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.552 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.552 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.552 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.552 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.552 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.552 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.552 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.553 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.553 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.553 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.553 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.553 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.553 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-03-19T01:25:34.553 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-03-19T01:25:34.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:34.554 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.554 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.554 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:34.554 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.554 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.554 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.554 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-19T01:25:34.554 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.555 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.555 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.555 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-03-19T01:25:34.555 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-03-19T01:25:34.555 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.555 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.555 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.555 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-03-19T01:25:34.556 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.556 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.556 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.556 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-03-19T01:25:34.556 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-03-19T01:25:34.556 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-03-19T01:25:34.556 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-03-19T01:25:34.556 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-03-19T01:25:34.556 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.557 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-03-19T01:25:34.557 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.557 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.557 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.557 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.557 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-03-19T01:25:34.557 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-03-19T01:25:34.557 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.558 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.558 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.558 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.558 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.559 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-19T01:25:34.559 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.559 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.559 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.559 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.559 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:34.559 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:34.560 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.560 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.560 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.560 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:34.560 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.560 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.560 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.561 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-19T01:25:34.561 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.561 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.561 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.561 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-03-19T01:25:34.562 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-03-19T01:25:34.562 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.562 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.562 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.562 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:34.562 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:34.562 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.563 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-19T01:25:34.563 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-19T01:25:34.563 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.563 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.563 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.564 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-03-19T01:25:34.564 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-03-19T01:25:34.564 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:34.564 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.564 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.564 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:34.565 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.565 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.565 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.565 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-19T01:25:34.565 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.565 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.566 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.566 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.566 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.566 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.567 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-03-19T01:25:34.567 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.567 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.567 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.567 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-03-19T01:25:34.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-03-19T01:25:34.568 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:34.568 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.568 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.568 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-03-19T01:25:34.569 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-19T01:25:34.569 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.569 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.570 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.570 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-03-19T01:25:34.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-03-19T01:25:34.570 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.571 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.571 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.571 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-03-19T01:25:34.571 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.571 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.572 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.572 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-19T01:25:34.572 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.572 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.572 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.572 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.572 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-03-19T01:25:34.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-03-19T01:25:34.573 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.573 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.573 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.573 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-03-19T01:25:34.573 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.573 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.573 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.574 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-19T01:25:34.574 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.574 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.574 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.574 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.575 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-03-19T01:25:34.575 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-03-19T01:25:34.575 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.575 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.575 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.575 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.575 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.576 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.576 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.576 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-19T01:25:34.576 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-19T01:25:34.576 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.576 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.576 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-03-19T01:25:34.577 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-03-19T01:25:34.577 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.577 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.577 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.577 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:34.577 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.577 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.578 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.578 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-19T01:25:34.578 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.578 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.578 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.578 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.578 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-03-19T01:25:34.579 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-03-19T01:25:34.579 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:34.579 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.579 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.579 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-03-19T01:25:34.579 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.579 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.579 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.579 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-19T01:25:34.580 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.580 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.580 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.580 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.580 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-03-19T01:25:34.580 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-03-19T01:25:34.580 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.580 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.581 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.581 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.581 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.581 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.581 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.581 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-03-19T01:25:34.581 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-03-19T01:25:34.581 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-03-19T01:25:34.582 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.582 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-19T01:25:34.582 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.582 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.582 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.582 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.582 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-03-19T01:25:34.582 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-03-19T01:25:34.583 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.583 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.583 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.583 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.583 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.583 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.583 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.583 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-19T01:25:34.583 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.584 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.584 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.584 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.584 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-03-19T01:25:34.584 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-03-19T01:25:34.584 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.584 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.584 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.584 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.584 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.585 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.585 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.585 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-19T01:25:34.585 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.585 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.585 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.585 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.585 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-03-19T01:25:34.586 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-03-19T01:25:34.586 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.586 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.586 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.586 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.586 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.586 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.586 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.587 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-19T01:25:34.587 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.587 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.587 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.587 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.587 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-03-19T01:25:34.587 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-03-19T01:25:34.587 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.588 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.588 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-19T01:25:34.588 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.588 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.588 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.588 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:34.588 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.589 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.589 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.589 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-03-19T01:25:34.589 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-03-19T01:25:34.589 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.589 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.589 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.589 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-19T01:25:34.590 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.590 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.590 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-03-19T01:25:34.590 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.590 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.590 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.590 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.591 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-03-19T01:25:34.591 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-03-19T01:25:34.591 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.591 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.591 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.591 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-19T01:25:34.591 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.592 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.592 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.592 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-03-19T01:25:34.592 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.592 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.592 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.592 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.593 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-03-19T01:25:34.593 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-03-19T01:25:34.593 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.593 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.593 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.593 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-19T01:25:34.593 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.594 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.594 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.594 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:34.594 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.594 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.594 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.594 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.595 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-03-19T01:25:34.595 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-03-19T01:25:34.595 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.595 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.595 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.595 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-19T01:25:34.596 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.596 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.596 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.596 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-03-19T01:25:34.596 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.596 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.596 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.597 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.597 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-03-19T01:25:34.597 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-03-19T01:25:34.597 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.597 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.597 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.597 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-19T01:25:34.598 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.598 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.598 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.598 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-03-19T01:25:34.598 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.598 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.598 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.599 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.599 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-03-19T01:25:34.599 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-03-19T01:25:34.599 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.599 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.599 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.600 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-19T01:25:34.600 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.600 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.600 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.600 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-03-19T01:25:34.600 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.600 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.601 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.601 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.601 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-03-19T01:25:34.601 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-03-19T01:25:34.601 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.601 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.601 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.602 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-19T01:25:34.602 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.602 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.602 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.602 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-03-19T01:25:34.602 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.602 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.603 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.603 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.603 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-03-19T01:25:34.603 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-03-19T01:25:34.603 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.603 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.604 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.604 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-19T01:25:34.604 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.604 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.604 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.604 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-03-19T01:25:34.604 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.605 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.605 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.605 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.605 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-03-19T01:25:34.605 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-03-19T01:25:34.605 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.605 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.606 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.606 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-19T01:25:34.606 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.606 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.606 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.606 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-03-19T01:25:34.606 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.607 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.607 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.607 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-03-19T01:25:34.607 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-03-19T01:25:34.607 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.608 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.608 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.608 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-19T01:25:34.608 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.608 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.608 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.608 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:34.609 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.609 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.609 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.609 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.609 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-03-19T01:25:34.609 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-03-19T01:25:34.609 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.609 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.609 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-19T01:25:34.610 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.610 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.610 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.610 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-03-19T01:25:34.610 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.610 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.610 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.611 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.611 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-03-19T01:25:34.611 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-03-19T01:25:34.611 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.611 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.611 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.611 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-19T01:25:34.611 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.612 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.612 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.612 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:34.612 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.612 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.612 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.612 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.612 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-03-19T01:25:34.612 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-03-19T01:25:34.613 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.613 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.613 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.613 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-19T01:25:34.613 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.613 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.613 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.613 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-03-19T01:25:34.614 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.614 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.614 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.614 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.614 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-03-19T01:25:34.614 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-03-19T01:25:34.615 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.615 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.615 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.615 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-19T01:25:34.615 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.615 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.615 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.615 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-03-19T01:25:34.616 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.616 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.616 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.616 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.616 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-03-19T01:25:34.616 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-03-19T01:25:34.616 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.616 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.616 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.617 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.617 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.617 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.617 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.617 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-03-19T01:25:34.617 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.617 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.617 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.618 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.618 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-03-19T01:25:34.618 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-03-19T01:25:34.618 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.618 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.618 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.618 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:34.618 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.619 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.619 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.619 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-19T01:25:34.619 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.619 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.619 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.619 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.620 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-03-19T01:25:34.620 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-03-19T01:25:34.620 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.620 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.620 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.620 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-03-19T01:25:34.620 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.621 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.621 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.621 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-19T01:25:34.621 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.621 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.621 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.621 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.622 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-03-19T01:25:34.622 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-03-19T01:25:34.622 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.622 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.622 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.622 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-03-19T01:25:34.622 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.623 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.623 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.623 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-19T01:25:34.623 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.623 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.623 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.623 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.624 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-03-19T01:25:34.624 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-03-19T01:25:34.624 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.624 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.624 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-03-19T01:25:34.625 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.625 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.625 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.625 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-03-19T01:25:34.625 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.625 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.625 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.626 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.626 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-03-19T01:25:34.626 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-03-19T01:25:34.626 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.626 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.626 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.626 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.627 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.627 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.627 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.627 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-19T01:25:34.627 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.627 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.627 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.627 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.628 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-03-19T01:25:34.628 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-03-19T01:25:34.628 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.628 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.628 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.628 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:34.628 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.629 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.629 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.629 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-03-19T01:25:34.629 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.629 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.629 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.629 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.630 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-03-19T01:25:34.630 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-03-19T01:25:34.630 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.630 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.630 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.630 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:34.630 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.631 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.631 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.631 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-03-19T01:25:34.631 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.631 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.631 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.631 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.632 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-03-19T01:25:34.632 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-03-19T01:25:34.632 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.632 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.632 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.632 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.632 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.633 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.633 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.633 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-19T01:25:34.633 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.633 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.633 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.633 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.633 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-03-19T01:25:34.634 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-03-19T01:25:34.634 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.634 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.634 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.634 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.634 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.634 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.635 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-19T01:25:34.635 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.635 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.635 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.635 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.635 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.635 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.635 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.636 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.636 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.636 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.636 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.636 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.636 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.637 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.637 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.637 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.637 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.637 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.637 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.638 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.638 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.638 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.638 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.638 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.638 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.638 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-03-19T01:25:34.638 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-03-19T01:25:34.639 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.639 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.639 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.639 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.639 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.639 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.639 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.639 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-19T01:25:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-19T01:25:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.642 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.642 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.642 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.642 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.642 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.642 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.642 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.643 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.643 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.643 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.643 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.643 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.643 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.643 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.643 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.643 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.644 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.644 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.644 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.644 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.644 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.644 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.644 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.644 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.645 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.645 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.645 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.645 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.645 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.645 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.645 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.645 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-19T01:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-19T01:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-19T01:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.648 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.648 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.648 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-19T01:25:34.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-19T01:25:34.648 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.648 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.648 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.648 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-03-19T01:25:34.649 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.649 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.649 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.649 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-19T01:25:34.649 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.649 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.649 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.650 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.650 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-03-19T01:25:34.650 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-03-19T01:25:34.650 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.650 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.650 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.650 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:34.650 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.650 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.651 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.651 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-03-19T01:25:34.651 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.651 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.651 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.651 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.651 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-03-19T01:25:34.651 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-03-19T01:25:34.652 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.652 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.652 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.652 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:34.652 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.652 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.652 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.653 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-19T01:25:34.653 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.653 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.653 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.653 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.653 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-03-19T01:25:34.653 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-03-19T01:25:34.654 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.654 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.654 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.654 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.654 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.654 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.654 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.655 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-03-19T01:25:34.655 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.655 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.655 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.655 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.655 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-03-19T01:25:34.655 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-03-19T01:25:34.656 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.656 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.656 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.656 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-03-19T01:25:34.656 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.656 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.657 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.657 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-03-19T01:25:34.657 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-03-19T01:25:34.657 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.657 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-03-19T01:25:34.657 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.657 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.658 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.658 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.658 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-03-19T01:25:34.658 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-03-19T01:25:34.658 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.658 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.658 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-03-19T01:25:34.659 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.659 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.659 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.659 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-19T01:25:34.659 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.659 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.660 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.660 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.660 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-03-19T01:25:34.660 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-03-19T01:25:34.660 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.660 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.661 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.661 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-19T01:25:34.661 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.661 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.661 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.661 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-19T01:25:34.661 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.662 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.662 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.662 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.662 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-03-19T01:25:34.662 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-03-19T01:25:34.663 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.663 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.663 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.663 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.663 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.663 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.663 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.664 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-19T01:25:34.664 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.664 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.664 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.664 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-03-19T01:25:34.665 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-03-19T01:25:34.665 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.665 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.665 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.665 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.665 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.666 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.666 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.666 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-19T01:25:34.666 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.666 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.666 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.666 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.667 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-03-19T01:25:34.667 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-03-19T01:25:34.667 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.667 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.667 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.667 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.668 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.668 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.668 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.668 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-19T01:25:34.668 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.668 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.669 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.669 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.669 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-03-19T01:25:34.669 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-03-19T01:25:34.669 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.669 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.669 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.670 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.670 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.670 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.670 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.670 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-19T01:25:34.670 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.671 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.671 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.671 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-03-19T01:25:34.671 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-03-19T01:25:34.671 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.671 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.671 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.671 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.671 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.672 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.672 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.672 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-19T01:25:34.672 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.672 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.672 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.672 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.672 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-03-19T01:25:34.673 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-03-19T01:25:34.673 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.673 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.673 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.673 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-03-19T01:25:34.673 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.673 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.673 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.673 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-03-19T01:25:34.674 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.674 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.674 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.674 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.674 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-03-19T01:25:34.674 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-03-19T01:25:34.674 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.674 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.675 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.675 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.675 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.675 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.675 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.675 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-19T01:25:34.675 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.675 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.676 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.676 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-03-19T01:25:34.676 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-03-19T01:25:34.676 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.676 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.676 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.676 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.676 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.677 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.677 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.677 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-19T01:25:34.677 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.677 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.677 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.677 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.677 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-03-19T01:25:34.678 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-03-19T01:25:34.678 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.678 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.678 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.678 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.678 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.678 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.678 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.679 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-19T01:25:34.679 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.679 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.679 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.679 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.679 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-03-19T01:25:34.679 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-03-19T01:25:34.679 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.680 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.680 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.680 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.680 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.680 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.680 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.680 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-03-19T01:25:34.681 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.681 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.681 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.681 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.681 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-03-19T01:25:34.681 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-03-19T01:25:34.681 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.682 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.682 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.682 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.682 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.682 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.682 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.682 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-19T01:25:34.683 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.683 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.683 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.683 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-03-19T01:25:34.683 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-03-19T01:25:34.683 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.684 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.684 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.684 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.684 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.684 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.684 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.684 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-19T01:25:34.685 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.685 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.685 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.685 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.685 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.685 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.685 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-03-19T01:25:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.686 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.687 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.688 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.689 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.690 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.691 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.692 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.693 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.694 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.695 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.696 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-03-19T01:25:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-03-19T01:25:34.697 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-03-19T01:25:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.698 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-19T01:25:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-03-19T01:25:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-03-19T01:25:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.699 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-03-19T01:25:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-19T01:25:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.700 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-03-19T01:25:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.701 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-19T01:25:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-19T01:25:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.702 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-03-19T01:25:34.703 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-19T01:25:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.704 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-19T01:25:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-19T01:25:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.705 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.706 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-03-19T01:25:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-03-19T01:25:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.707 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.708 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-19T01:25:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-19T01:25:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.709 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.710 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-03-19T01:25:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-19T01:25:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.711 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-19T01:25:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-19T01:25:34.712 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.713 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-19T01:25:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-19T01:25:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.714 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.715 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.718 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.718 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.718 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.718 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.719 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.719 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-19T01:25:34.719 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-19T01:25:34.719 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.719 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.719 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.720 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.720 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.720 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.720 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.720 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.721 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.721 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.721 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-19T01:25:34.721 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-19T01:25:34.721 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.721 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.721 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.722 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.722 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.722 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.722 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.722 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.722 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.722 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.723 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.723 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.723 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-19T01:25:34.723 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-19T01:25:34.723 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.723 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.724 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.724 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.724 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.724 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.724 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.724 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.724 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.725 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.725 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.725 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.725 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-19T01:25:34.725 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-19T01:25:34.725 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.725 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.726 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.726 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.726 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.726 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.726 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.726 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.727 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.727 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.727 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.727 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.727 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-19T01:25:34.727 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-19T01:25:34.727 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.728 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.728 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.728 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.728 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.728 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.728 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-19T01:25:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-19T01:25:34.730 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.730 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.730 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.730 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.730 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.730 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.731 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.731 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.731 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.731 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.731 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.731 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.731 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-03-19T01:25:34.732 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-19T01:25:34.732 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.732 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.732 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.732 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-03-19T01:25:34.732 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.732 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.733 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.733 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.733 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.733 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.733 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.733 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.733 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-19T01:25:34.733 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-19T01:25:34.733 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.734 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.734 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.734 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.734 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.734 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.734 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.734 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.734 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.735 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.735 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.735 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.735 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-03-19T01:25:34.735 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-03-19T01:25:34.735 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.735 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.735 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.736 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-03-19T01:25:34.737 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-19T01:25:34.737 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.737 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.737 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.737 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-03-19T01:25:34.737 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.737 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.737 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.737 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.737 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.738 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.738 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.738 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.738 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-19T01:25:34.738 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-19T01:25:34.738 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.738 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.739 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.739 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.739 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.739 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.739 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.739 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.739 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.739 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.739 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.740 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.740 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-19T01:25:34.740 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-19T01:25:34.740 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.740 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.740 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.740 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.740 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.740 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.741 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.741 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.741 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.741 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.741 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.741 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.741 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-19T01:25:34.741 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-19T01:25:34.742 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.742 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.742 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.742 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.742 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.742 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.742 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.743 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.743 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.743 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.743 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.743 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.743 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-19T01:25:34.743 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-19T01:25:34.744 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.744 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.744 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.744 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.744 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.744 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.745 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.745 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.745 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.745 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.745 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.745 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.745 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-03-19T01:25:34.746 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-19T01:25:34.746 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.746 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.746 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.746 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-03-19T01:25:34.746 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.746 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.747 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.747 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.747 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.747 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.747 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.747 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.747 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-03-19T01:25:34.748 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-19T01:25:34.748 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.748 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.748 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.748 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.748 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.748 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.749 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.749 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.749 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.749 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.749 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.749 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.750 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-19T01:25:34.750 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-19T01:25:34.750 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.750 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.750 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.751 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.751 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.751 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.751 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.751 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.751 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.751 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.752 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.752 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-19T01:25:34.752 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-19T01:25:34.752 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.752 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.752 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.752 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.753 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.753 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.753 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.753 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.753 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.753 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.754 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.754 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.754 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-19T01:25:34.754 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-19T01:25:34.754 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.754 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.754 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.755 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.755 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.755 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.755 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.755 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.755 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.755 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.756 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.756 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.756 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-19T01:25:34.756 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-19T01:25:34.756 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.756 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.756 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.757 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.757 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.757 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.757 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.757 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.757 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.757 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.758 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.758 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.758 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-19T01:25:34.758 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-19T01:25:34.758 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.758 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.758 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.759 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.759 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.759 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.759 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.759 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.759 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.759 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.760 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.760 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.760 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-19T01:25:34.760 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-19T01:25:34.760 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.760 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.761 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.761 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.761 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.761 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.761 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.761 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.761 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.761 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.762 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.762 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.762 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-19T01:25:34.762 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-19T01:25:34.762 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.762 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.762 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.762 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.763 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.763 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.763 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.763 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.763 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.763 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.763 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.764 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-19T01:25:34.764 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-19T01:25:34.764 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.764 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.764 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.764 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.764 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.764 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.765 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.765 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.765 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.765 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.765 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.765 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-03-19T01:25:34.765 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-19T01:25:34.765 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.766 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.766 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.766 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.766 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.766 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.766 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.766 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.767 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.767 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.767 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.767 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.767 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-19T01:25:34.767 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-19T01:25:34.767 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.767 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.767 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.768 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-19T01:25:34.768 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.768 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.768 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.768 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.768 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.768 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.768 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.769 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.769 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-19T01:25:34.769 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-19T01:25:34.769 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.769 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.769 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.769 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:34.770 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.770 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.770 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.770 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.770 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.770 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.770 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.770 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.770 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-19T01:25:34.771 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-19T01:25:34.771 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.771 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.771 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.771 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-03-19T01:25:34.771 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.771 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.771 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.772 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.772 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.772 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.772 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.772 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.772 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-19T01:25:34.773 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-19T01:25:34.773 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.773 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.773 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.773 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-03-19T01:25:34.773 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.773 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.773 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.774 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.774 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.774 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.774 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.774 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.774 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-03-19T01:25:34.774 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-19T01:25:34.775 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.775 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.775 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.775 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.775 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.776 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.776 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.776 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.776 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.776 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.776 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.776 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-19T01:25:34.776 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-19T01:25:34.776 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.777 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.777 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.777 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-03-19T01:25:34.777 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.777 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.777 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.777 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.777 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.778 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.778 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.778 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.778 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-03-19T01:25:34.778 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-19T01:25:34.778 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.778 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.778 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.779 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.779 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.779 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.779 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.779 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.779 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.779 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.780 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.780 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.780 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-03-19T01:25:34.780 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-19T01:25:34.780 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.780 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.780 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.780 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.781 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.781 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.781 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.781 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.781 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.781 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.782 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.782 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.782 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-03-19T01:25:34.782 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-19T01:25:34.782 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.782 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.783 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.783 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.783 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.783 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.783 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.783 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.783 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.784 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.784 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.784 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.784 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-19T01:25:34.784 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-19T01:25:34.784 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.784 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.785 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.785 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:34.785 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.785 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.785 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.785 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.785 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.786 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.786 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.786 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.786 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-19T01:25:34.786 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-19T01:25:34.786 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.787 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.787 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.787 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:34.787 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.787 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.787 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.787 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.788 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.788 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.788 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.788 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.788 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-19T01:25:34.788 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-19T01:25:34.789 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.789 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.789 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.789 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:34.789 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.789 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.789 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.790 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.790 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.790 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.790 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.790 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.790 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-03-19T01:25:34.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-03-19T01:25:34.791 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.791 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.791 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.791 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.791 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.791 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.792 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.792 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.792 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.792 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.792 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.792 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.792 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-03-19T01:25:34.793 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-03-19T01:25:34.793 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.793 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.793 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.793 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.793 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.793 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.793 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.794 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.794 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.794 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.794 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.794 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.794 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-03-19T01:25:34.795 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-03-19T01:25:34.795 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.795 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.795 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.795 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.795 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.795 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.796 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.796 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-03-19T01:25:34.796 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.796 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.796 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.796 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-03-19T01:25:34.796 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-03-19T01:25:34.797 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.797 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.797 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.797 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-03-19T01:25:34.797 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.797 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.797 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.798 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.798 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.798 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.798 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.798 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-03-19T01:25:34.798 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-03-19T01:25:34.798 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.798 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.799 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.799 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.799 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.799 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.799 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.799 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.799 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.799 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.800 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.800 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.800 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.800 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.800 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.800 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.800 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.800 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.804 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.804 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.804 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.804 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.804 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.805 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.805 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.805 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.805 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.805 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.805 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.806 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.806 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.806 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.806 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.806 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.806 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.806 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.807 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.807 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.807 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.807 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.807 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.807 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.807 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.808 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.808 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.808 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.808 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.808 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.808 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.808 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.809 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.809 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.809 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.809 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.809 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.809 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-03-19T01:25:34.809 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-03-19T01:25:34.810 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.810 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.810 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.810 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.810 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.810 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.810 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.811 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-03-19T01:25:34.811 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.811 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.811 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.811 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.811 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-03-19T01:25:34.812 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-03-19T01:25:34.812 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.812 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.812 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.812 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.812 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.813 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.813 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.813 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.813 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.813 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.813 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.814 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-03-19T01:25:34.814 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-03-19T01:25:34.814 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.814 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.814 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.814 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-03-19T01:25:34.815 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.815 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.815 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.815 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.815 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.815 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.815 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.816 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.816 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-03-19T01:25:34.816 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-03-19T01:25:34.816 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.816 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.816 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.816 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-03-19T01:25:34.817 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.817 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.817 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.817 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.817 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.817 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.817 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.818 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-03-19T01:25:34.818 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-03-19T01:25:34.818 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.818 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.818 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.818 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.819 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.819 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.819 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.819 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.819 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.819 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.820 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.820 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.820 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-03-19T01:25:34.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-03-19T01:25:34.820 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.820 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.820 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.821 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-03-19T01:25:34.821 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.821 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.821 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.821 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.821 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.821 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.822 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.822 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.822 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-03-19T01:25:34.822 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-03-19T01:25:34.822 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.822 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.823 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.823 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-03-19T01:25:34.823 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.823 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.823 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.823 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.823 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-03-19T01:25:34.823 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.824 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.824 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.824 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.824 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.824 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.824 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-03-19T01:25:34.824 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-03-19T01:25:34.824 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.824 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.825 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.825 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-03-19T01:25:34.825 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-03-19T01:25:34.825 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-03-19T01:25:34.825 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.825 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.825 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.825 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.826 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.826 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.826 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-03-19T01:25:34.826 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-03-19T01:25:34.826 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.826 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.826 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.826 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.827 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.827 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.827 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.827 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.827 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.827 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.827 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.827 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.828 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.828 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.828 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-03-19T01:25:34.828 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.828 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.828 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.828 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-03-19T01:25:34.828 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-03-19T01:25:34.829 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.829 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.829 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.829 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.829 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.829 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.829 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.829 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-03-19T01:25:34.830 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.830 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.830 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.830 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.830 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.830 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.830 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.830 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.830 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.831 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.831 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.831 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.831 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.831 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.831 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.831 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.831 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.832 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.832 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.832 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.832 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.832 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.832 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.832 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.832 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.832 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.833 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.833 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.833 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.833 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.833 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.833 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.833 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.833 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.834 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.834 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.834 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.834 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.834 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.834 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.835 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.835 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.835 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.835 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.835 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.835 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.835 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.835 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.836 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.836 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.836 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.836 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.836 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.836 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.837 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.837 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.837 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.837 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.837 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.837 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.838 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.838 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.838 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.838 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.838 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.838 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.838 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.839 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.839 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.839 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.839 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.839 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.839 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.839 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-03-19T01:25:34.840 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-03-19T01:25:34.840 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.840 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.840 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.840 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-03-19T01:25:34.840 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.840 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.841 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.841 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-19T01:25:34.841 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.841 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.841 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.841 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.841 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-03-19T01:25:34.841 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-03-19T01:25:34.842 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.842 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.842 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.842 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-03-19T01:25:34.842 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.842 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.842 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.842 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-19T01:25:34.843 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.843 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.843 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.843 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.843 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-03-19T01:25:34.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-03-19T01:25:34.843 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.843 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.844 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.844 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-03-19T01:25:34.844 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.844 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.844 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-03-19T01:25:34.844 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.844 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.845 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.845 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.845 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-03-19T01:25:34.845 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-03-19T01:25:34.845 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.845 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.845 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.845 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-03-19T01:25:34.845 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.846 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.846 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.846 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-19T01:25:34.846 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.846 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.846 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.847 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.847 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-03-19T01:25:34.847 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-03-19T01:25:34.847 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.847 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.847 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.847 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:34.848 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.848 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.848 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.848 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-19T01:25:34.848 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.848 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.848 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.849 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.849 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:34.849 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:34.849 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.849 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.849 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.849 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:34.850 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.850 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.850 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.850 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-03-19T01:25:34.850 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.850 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.851 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.851 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.851 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-03-19T01:25:34.851 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-03-19T01:25:34.851 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.851 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.852 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.852 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-03-19T01:25:34.852 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.852 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.852 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.852 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-19T01:25:34.852 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.853 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.853 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.853 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.853 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.853 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.853 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.853 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-03-19T01:25:34.854 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.854 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.854 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.854 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.854 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.854 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.854 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.855 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.855 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.855 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.855 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.855 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.856 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.856 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.856 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.856 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.856 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.856 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.856 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.856 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.856 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.857 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.857 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.857 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.857 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.857 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.857 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.857 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.858 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.858 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.858 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.858 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.858 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.858 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.858 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.858 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.859 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.859 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.859 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.859 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.859 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.859 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.859 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.860 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.860 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.860 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.860 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.860 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.860 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.860 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.860 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.861 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.861 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.861 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.861 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.861 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.861 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.861 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.861 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.861 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.862 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.862 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.862 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.862 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.862 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.862 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.862 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.862 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.863 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.863 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.863 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.863 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.863 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-03-19T01:25:34.863 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-03-19T01:25:34.863 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.863 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.864 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.864 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-03-19T01:25:34.864 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.864 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.864 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.864 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.864 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.864 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.865 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.865 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-03-19T01:25:34.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-03-19T01:25:34.865 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.865 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.865 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.865 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-03-19T01:25:34.866 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.866 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.866 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.866 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.866 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.866 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.866 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.866 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.867 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:34.867 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:34.867 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.867 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.867 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.867 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:34.867 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.868 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.868 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.868 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.868 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.868 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.868 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.868 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.869 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.869 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.869 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.869 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-03-19T01:25:34.869 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-03-19T01:25:34.869 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-03-19T01:25:34.869 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.870 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-03-19T01:25:34.870 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-03-19T01:25:34.870 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.870 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.870 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.870 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-03-19T01:25:34.871 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.871 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.871 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.871 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-19T01:25:34.871 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.871 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.871 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.872 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.872 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-03-19T01:25:34.872 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-03-19T01:25:34.872 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.872 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.872 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.873 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-03-19T01:25:34.873 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.873 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.873 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.873 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-19T01:25:34.873 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.873 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.874 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.874 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.874 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-03-19T01:25:34.874 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-03-19T01:25:34.874 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.874 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.875 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.875 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.875 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.875 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.875 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.875 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-03-19T01:25:34.875 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.875 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.876 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.876 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.876 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:34.876 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:34.876 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:34.876 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.876 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.877 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-03-19T01:25:34.877 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-03-19T01:25:34.877 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.877 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.877 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-19T01:25:34.877 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.878 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.878 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.878 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.878 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.878 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.878 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.878 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.879 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.879 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.879 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.879 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.879 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.879 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.880 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.880 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.880 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.880 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.880 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.880 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.880 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.881 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.881 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.881 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.881 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.881 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.881 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.881 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.882 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.882 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.882 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.882 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.882 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.882 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.883 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.883 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.883 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.883 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.883 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.883 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.884 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.884 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.884 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.884 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.884 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.884 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.885 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.885 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.885 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.885 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.885 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.885 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.885 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.885 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.886 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.886 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.886 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.886 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.886 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.886 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.886 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.886 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.886 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.887 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.887 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.887 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.887 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.887 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.887 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.887 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.887 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.888 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.888 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.888 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-03-19T01:25:34.888 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-03-19T01:25:34.888 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.888 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.888 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.888 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.889 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.889 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.889 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.889 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-03-19T01:25:34.889 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.889 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.889 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.890 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-03-19T01:25:34.890 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-03-19T01:25:34.890 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.890 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.890 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.890 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-03-19T01:25:34.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.891 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.891 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-03-19T01:25:34.891 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.891 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.891 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.891 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.891 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-03-19T01:25:34.891 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-03-19T01:25:34.892 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.892 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.892 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.892 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-03-19T01:25:34.892 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.892 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.892 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.893 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T01:25:34.893 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.893 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.893 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.893 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.893 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-03-19T01:25:34.893 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-03-19T01:25:34.894 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.894 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.894 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.894 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:34.894 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:34.894 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-03-19T01:25:34.894 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.894 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-19T01:25:34.895 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.895 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.895 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.895 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.895 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-03-19T01:25:34.895 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-03-19T01:25:34.895 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.896 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.896 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.896 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.896 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.896 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.896 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.896 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-03-19T01:25:34.897 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.897 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.897 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.897 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-03-19T01:25:34.897 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-03-19T01:25:34.897 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.898 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.898 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.898 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-03-19T01:25:34.898 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.898 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.898 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.899 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T01:25:34.899 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.899 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.899 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.899 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.900 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.900 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.900 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-03-19T01:25:34.900 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.900 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.901 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.901 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.901 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.901 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.901 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.901 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.902 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.902 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.902 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.902 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.902 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.902 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.903 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.903 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.903 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.903 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.903 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.903 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.903 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.904 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.904 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.904 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.904 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.904 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.904 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.905 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.905 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.905 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.905 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.905 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.905 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.905 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.906 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.906 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.906 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.906 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.906 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.906 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.907 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.907 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.907 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.907 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.907 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.907 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.907 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.907 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.907 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.908 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.908 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.908 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.908 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.908 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.908 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.908 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.909 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.909 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.909 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.909 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.909 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.909 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.909 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.910 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.910 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.910 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.910 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.910 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.910 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.911 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.911 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.911 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-03-19T01:25:34.911 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.911 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.911 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.912 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.912 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.912 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.912 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.912 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.912 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.912 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.913 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.913 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.913 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.913 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.913 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.913 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.913 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.914 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.914 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.914 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.914 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.914 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.914 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.915 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.915 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.915 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.915 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.915 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.916 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.916 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.916 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.916 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.916 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.916 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.917 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.917 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.917 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.917 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.917 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.917 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.917 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.918 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.918 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.918 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.918 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.918 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.918 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.918 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.918 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.919 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.919 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.919 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.919 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.919 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.919 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.919 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.919 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.920 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.920 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.920 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.920 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.920 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.920 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.920 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.920 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.921 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.921 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.921 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.921 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.921 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.921 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.921 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.921 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.922 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-03-19T01:25:34.922 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.922 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.922 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.922 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-03-19T01:25:34.922 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-03-19T01:25:34.922 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.922 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.923 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.923 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-03-19T01:25:34.923 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.923 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.923 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.926 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-19T01:25:34.926 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.927 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.927 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.927 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-03-19T01:25:34.927 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-03-19T01:25:34.927 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.927 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.927 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.928 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:34.928 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-03-19T01:25:34.928 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.928 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.928 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-19T01:25:34.929 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.929 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.929 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.929 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.929 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.930 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.930 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.930 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.931 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.931 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.931 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.931 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.931 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.932 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.932 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.932 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.932 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.933 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.933 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.933 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.933 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.933 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.933 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.934 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.934 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.934 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.934 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.934 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.934 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.934 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.935 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.935 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.935 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.935 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.935 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.935 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.936 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.936 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.936 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.936 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.936 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.936 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.936 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.937 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.937 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.937 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.937 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.937 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.937 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.937 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.938 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.938 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.938 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.938 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.938 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.938 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.939 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.939 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.939 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.939 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.939 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.939 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.940 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.940 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.940 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.940 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.940 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.940 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.940 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.941 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.941 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.941 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.941 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-03-19T01:25:34.941 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.941 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.942 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.942 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-03-19T01:25:34.942 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-03-19T01:25:34.942 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.942 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.942 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.942 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-03-19T01:25:34.943 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.943 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.943 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.943 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-03-19T01:25:34.943 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-19T01:25:34.943 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.943 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.944 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.944 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.944 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.944 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.944 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.944 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.945 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.945 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.945 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.945 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.945 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.945 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.945 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.946 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.946 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.946 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.946 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.946 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.946 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.946 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.947 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.947 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.947 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.947 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.947 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.947 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.947 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.948 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.948 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.948 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.948 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.948 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.948 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.948 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.949 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.949 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.949 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.949 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.949 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.949 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.949 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.950 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.950 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.950 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.950 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.950 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.950 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.950 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.951 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.951 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.951 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.951 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.951 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.951 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.951 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.952 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.952 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.952 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.952 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.952 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.952 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.952 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.952 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.953 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.953 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.953 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.953 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.953 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.953 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-03-19T01:25:34.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-03-19T01:25:34.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.954 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.954 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.954 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.954 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.954 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.954 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-03-19T01:25:34.954 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-19T01:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-03-19T01:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-03-19T01:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-03-19T01:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-03-19T01:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.957 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.957 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.957 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-03-19T01:25:34.957 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-03-19T01:25:34.957 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:34.957 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.957 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.957 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-03-19T01:25:34.958 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.958 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.958 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.958 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-03-19T01:25:34.958 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.958 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.958 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.958 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.959 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-03-19T01:25:34.959 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-03-19T01:25:34.959 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.959 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.959 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.959 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.959 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.959 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.960 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.960 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-03-19T01:25:34.960 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.960 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.960 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.960 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.960 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-03-19T01:25:34.961 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-03-19T01:25:34.961 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.961 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.961 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.961 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-03-19T01:25:34.961 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.961 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.962 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.962 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-19T01:25:34.962 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-19T01:25:34.962 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.962 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.962 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.962 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.963 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.963 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.963 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-03-19T01:25:34.963 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.963 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.963 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.963 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.963 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.964 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.964 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.964 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.964 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.964 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.964 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.965 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.965 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.965 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.965 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.965 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.965 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.965 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.966 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.966 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.966 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.966 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.966 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.966 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.967 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.967 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.967 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.967 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.967 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.967 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.967 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.968 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.968 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.968 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.968 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.968 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.968 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.968 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.969 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.969 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.969 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.969 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.969 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.969 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.969 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.970 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.970 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.970 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.970 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.970 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.970 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.971 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.971 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.971 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.971 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.971 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.971 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.972 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.972 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.972 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.972 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.972 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.972 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.973 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.973 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.973 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.973 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.973 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.973 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.973 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.973 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.974 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.974 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.974 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.974 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.974 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-03-19T01:25:34.974 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.974 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.974 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.975 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.975 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.975 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.975 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.975 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.975 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.975 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.975 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.975 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.976 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.976 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.976 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.976 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.976 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.976 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.976 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.977 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.977 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.977 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.977 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.977 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.977 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.977 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.978 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.978 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.978 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.978 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.978 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.978 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.978 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.979 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.979 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.979 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.979 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.979 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.979 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.980 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.980 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.980 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.980 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.980 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.980 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.980 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.980 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.981 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.981 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.981 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.981 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.981 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.981 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.981 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.981 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.982 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.982 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.982 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.982 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.982 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.982 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.982 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.982 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.982 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.983 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.983 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.983 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.983 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.983 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.983 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.984 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.984 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.984 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.984 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.984 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.984 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.984 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-03-19T01:25:34.984 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.985 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.985 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.985 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.985 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.985 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.985 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.985 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.985 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.986 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.986 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.986 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.986 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.986 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.986 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.986 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.986 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.987 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.987 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.987 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.987 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.987 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.987 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.987 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.987 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:34.988 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:34.988 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.988 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.988 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.988 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.988 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.988 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.988 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.989 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.989 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.989 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.989 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.989 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.989 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:34.989 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:34.989 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.989 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.990 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.990 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:34.990 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.990 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.990 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.990 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.990 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.991 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.991 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.991 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.991 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.991 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.991 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.991 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.991 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.991 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.992 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.992 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:34.992 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:34.992 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.992 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.992 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.992 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.993 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.993 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.993 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.993 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.993 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.993 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.993 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.994 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.994 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:34.994 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.994 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:34.994 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-03-19T01:25:34.994 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:34.994 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:34.995 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:34.995 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-03-19T01:25:34.995 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-03-19T01:25:34.995 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:34.995 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.995 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:34.996 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:34.996 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.996 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.996 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:34.996 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-19T01:25:34.996 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:34.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:34.997 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:34.997 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:34.997 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:34.997 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:34.997 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:34.997 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:34.997 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:34.998 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:34.998 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:34.998 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:34.998 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:34.998 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:34.998 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:34.999 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:34.999 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:34.999 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:34.999 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:34.999 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:34.999 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:34.999 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.000 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.000 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.000 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.000 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.000 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.000 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.001 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.001 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.001 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.001 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.001 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.001 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.001 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.001 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.002 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.002 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.002 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.002 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.002 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.003 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.003 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.003 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.003 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.003 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.003 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.004 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.004 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.004 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.004 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.004 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.004 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.004 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.005 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.005 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.005 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.005 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.005 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.005 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.006 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.006 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.006 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.006 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.006 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.006 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.007 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.007 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.007 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.007 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.007 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.007 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.007 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.008 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-03-19T01:25:35.008 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-03-19T01:25:35.008 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.008 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.008 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.008 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.008 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.009 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.009 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.009 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-03-19T01:25:35.009 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-03-19T01:25:35.009 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-03-19T01:25:35.009 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.009 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-03-19T01:25:35.010 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.010 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.010 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.010 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.010 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.010 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.011 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.011 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-03-19T01:25:35.011 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.011 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.011 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.011 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.011 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.011 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.012 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.012 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.012 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.012 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.012 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.012 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.012 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.013 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.013 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.013 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.013 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.013 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.013 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.013 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.013 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.014 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.014 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.014 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.014 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.014 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.014 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.014 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.014 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.015 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.015 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.015 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.015 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.015 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.015 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.015 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.016 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.016 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.016 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.016 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.016 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.016 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.016 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.016 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.017 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.017 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.017 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.017 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.017 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.017 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.017 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.017 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.018 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.018 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.018 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.018 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.018 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.018 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.018 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.018 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.019 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.019 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.019 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.019 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.019 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.019 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.019 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.019 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.020 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.020 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.020 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.020 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.020 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.020 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.020 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.020 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-03-19T01:25:35.021 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.021 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.021 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.021 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.021 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.022 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.022 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.022 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.022 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.022 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.022 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.023 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.023 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.023 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.023 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.023 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.023 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.023 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.024 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.024 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.024 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.024 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.024 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.024 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.024 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.025 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.025 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.025 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.025 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.025 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.025 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.026 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.026 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.026 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.026 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.026 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.026 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.026 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.027 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.027 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.027 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.027 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.027 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.027 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.027 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.028 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.028 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.028 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.028 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.028 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.028 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.028 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.029 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.029 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.029 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.029 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.029 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.030 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.030 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.030 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.030 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.030 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.031 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.031 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.031 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.031 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.031 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.031 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.031 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.032 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.032 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.032 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.032 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.032 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.032 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-03-19T01:25:35.033 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.033 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.033 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.033 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.033 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.033 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.033 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.034 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.034 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.034 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.034 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.034 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.034 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.034 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.035 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.036 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.037 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.037 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.037 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.037 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.037 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.037 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.038 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.038 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.038 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.038 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.038 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.038 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.039 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.039 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.039 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.039 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.041 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.042 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.042 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.042 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.042 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.042 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.042 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.043 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.043 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.043 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.043 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.043 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.043 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.043 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.043 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.044 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.044 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.044 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.044 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.044 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.044 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.044 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.044 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.045 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.045 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.045 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.045 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.045 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.045 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:35.045 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:35.045 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.045 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.046 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.046 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:35.046 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.046 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.046 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.046 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.046 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.047 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.047 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.047 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.047 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-03-19T01:25:35.047 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-03-19T01:25:35.047 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:35.047 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.047 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.047 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-03-19T01:25:35.048 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-03-19T01:25:35.048 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.048 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.048 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-03-19T01:25:35.048 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-19T01:25:35.048 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.048 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.049 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.049 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.049 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.049 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-03-19T01:25:35.049 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.049 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.049 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.050 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-03-19T01:25:35.050 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-03-19T01:25:35.050 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.050 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.050 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.050 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.050 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.050 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.050 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.051 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-19T01:25:35.051 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.051 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.051 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.051 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-03-19T01:25:35.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-03-19T01:25:35.051 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.052 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.052 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.052 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.052 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.052 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.052 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.052 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.053 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.053 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.053 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.053 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.053 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.054 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.054 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.054 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.054 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.054 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.054 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.055 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.055 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.055 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.055 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.055 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.055 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.055 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.055 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.056 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.056 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.056 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.056 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.056 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.057 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.057 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.057 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.057 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.057 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.057 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.058 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.058 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.058 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.058 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.058 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.059 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.059 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.059 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.059 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.059 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.059 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.059 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.060 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.060 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.060 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.060 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.060 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.060 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.060 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.061 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.061 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.061 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.061 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.061 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.061 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.062 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.062 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.062 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.062 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.062 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.062 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.062 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.063 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.063 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.063 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.063 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.063 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.064 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.064 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.064 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.064 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-03-19T01:25:35.064 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-03-19T01:25:35.064 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.065 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.065 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.065 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-03-19T01:25:35.065 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.066 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.066 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.066 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-03-19T01:25:35.066 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.066 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.066 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.067 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.067 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:35.067 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:35.067 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.067 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.068 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.068 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:35.068 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.068 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.068 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.069 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-03-19T01:25:35.069 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.069 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.069 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.070 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.070 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.070 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.070 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-03-19T01:25:35.071 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.071 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.071 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.071 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-03-19T01:25:35.071 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-03-19T01:25:35.071 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.071 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.072 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.072 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.072 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.072 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.072 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.072 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.072 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-03-19T01:25:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-03-19T01:25:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.073 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-19T01:25:35.074 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-19T01:25:35.075 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.075 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.075 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.075 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.075 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.075 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.075 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.075 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.076 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.076 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.076 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.076 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.076 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.076 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.076 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.076 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.077 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.077 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.077 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.077 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.077 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.077 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.077 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.077 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.078 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.078 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.078 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.078 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.078 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.078 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.078 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.079 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.079 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.079 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.079 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.079 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.079 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.079 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.079 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.080 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.080 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.080 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.080 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.080 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.080 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.080 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.080 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.081 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.081 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.081 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.081 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.081 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.081 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.081 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.081 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.081 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.082 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.082 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.082 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.082 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.082 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.082 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.082 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.082 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.083 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.083 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.083 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.083 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.083 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.083 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.083 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.084 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-03-19T01:25:35.084 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-03-19T01:25:35.084 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.084 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.084 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.084 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.084 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.084 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.085 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.085 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.085 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.085 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.085 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.085 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.085 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-19T01:25:35.086 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-19T01:25:35.086 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.086 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.086 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.086 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-03-19T01:25:35.086 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.086 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.087 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.087 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.087 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.087 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.087 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.087 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.087 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-03-19T01:25:35.088 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-03-19T01:25:35.088 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:35.088 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.088 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.088 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-03-19T01:25:35.088 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.088 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.089 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.089 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.089 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.089 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.089 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.089 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.089 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-03-19T01:25:35.089 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-03-19T01:25:35.090 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.090 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.090 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.090 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-03-19T01:25:35.090 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.090 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.091 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.091 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-19T01:25:35.091 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.091 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.091 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.091 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.092 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-03-19T01:25:35.092 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-03-19T01:25:35.092 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.092 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.093 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.093 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-03-19T01:25:35.093 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.093 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.093 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.093 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-19T01:25:35.093 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.094 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.094 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.094 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.094 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-03-19T01:25:35.094 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-03-19T01:25:35.095 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.095 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.095 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.095 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-03-19T01:25:35.095 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.096 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.096 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.096 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.096 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.096 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.096 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.097 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.097 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-03-19T01:25:35.097 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-03-19T01:25:35.097 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.097 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.098 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.098 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-03-19T01:25:35.098 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.098 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.098 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.099 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-03-19T01:25:35.099 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-03-19T01:25:35.099 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.099 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.099 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.100 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.100 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.100 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.100 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-03-19T01:25:35.100 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-03-19T01:25:35.100 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.101 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.101 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.101 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-03-19T01:25:35.101 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-03-19T01:25:35.101 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-03-19T01:25:35.102 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.102 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.102 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.102 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.102 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.103 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.103 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.103 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.103 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.103 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-03-19T01:25:35.103 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.104 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.104 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.104 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.104 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.104 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.104 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.104 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.105 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.105 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.105 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.105 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.105 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.105 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.105 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.105 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.105 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.106 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.106 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.106 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.106 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.106 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.106 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.106 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.106 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.107 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.107 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.107 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.107 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.107 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.107 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.107 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.107 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.108 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.108 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.108 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.108 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.108 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.108 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.108 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.108 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.110 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.110 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.110 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.110 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.110 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.110 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.110 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.110 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.110 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.111 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.111 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.111 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.111 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.111 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.111 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.111 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.111 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.112 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.112 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.112 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.112 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.112 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.112 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-03-19T01:25:35.112 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-03-19T01:25:35.112 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.113 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.113 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.113 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:35.113 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.113 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.113 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.113 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.113 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.114 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.114 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.114 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.114 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-03-19T01:25:35.114 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-03-19T01:25:35.114 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.114 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.115 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.115 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.115 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.115 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.115 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.115 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.115 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.115 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.116 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.116 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.116 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-03-19T01:25:35.116 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-03-19T01:25:35.116 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.116 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.116 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.116 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.116 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.117 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.117 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.117 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.117 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.117 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.117 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.117 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.117 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.118 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.118 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.118 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-03-19T01:25:35.118 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.118 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.118 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.118 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-03-19T01:25:35.118 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-03-19T01:25:35.119 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.119 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.119 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.119 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.119 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.119 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.119 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.119 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.120 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.120 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.120 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.120 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.120 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-03-19T01:25:35.120 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-03-19T01:25:35.120 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.121 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.121 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.121 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.121 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.121 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.121 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.122 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.122 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.122 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.122 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.122 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.123 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.123 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.123 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.123 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.123 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.124 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.124 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.124 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.124 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.124 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.124 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.124 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.125 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.125 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.125 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.125 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.125 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.125 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.126 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.126 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.126 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.126 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.126 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.126 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.126 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.127 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.127 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.127 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.127 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.127 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.127 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.127 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.128 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.128 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.128 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.128 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.128 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.128 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.128 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.129 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.129 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.129 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.129 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.129 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.129 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.129 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.130 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.130 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.130 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.130 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.130 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.130 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.130 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.131 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.131 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.131 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.131 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.131 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.131 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.132 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.132 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.132 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.132 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.132 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.132 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.132 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.133 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.133 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.133 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-03-19T01:25:35.133 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-03-19T01:25:35.133 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.133 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.133 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.134 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.134 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.134 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.134 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.134 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.134 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.135 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.135 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.135 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.135 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-03-19T01:25:35.135 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-03-19T01:25:35.135 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.135 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.136 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.136 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.136 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.136 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.136 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.136 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.136 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.136 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.137 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.137 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.137 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.137 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.137 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.137 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-03-19T01:25:35.137 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.137 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.138 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.138 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.138 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.138 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.138 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.138 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.138 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.138 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.139 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.139 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.139 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.139 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.139 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.139 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.139 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.139 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.139 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.140 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.140 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.140 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.140 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.140 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.140 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.140 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.140 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.141 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.141 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.141 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.141 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.141 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.141 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.141 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.141 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.141 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.142 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.142 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.142 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.142 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.142 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.142 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.142 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.142 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.143 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.143 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.143 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.143 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.143 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.143 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.143 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.143 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.144 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.144 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.144 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.144 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.144 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.144 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.144 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.144 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.144 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.145 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.145 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.145 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.145 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.145 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.145 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.145 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.145 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.146 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.146 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.146 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.146 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.146 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.146 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.146 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-03-19T01:25:35.147 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.147 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.147 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.147 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.147 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.147 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.147 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.147 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.148 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.148 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.148 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.148 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.148 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.148 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.149 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.149 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.149 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.149 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.149 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.149 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.149 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.150 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.150 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.150 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.150 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.150 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.150 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.150 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.151 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.151 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.151 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.151 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.151 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.151 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.151 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.152 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.152 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.152 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.152 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.152 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.152 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.153 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.153 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.153 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.153 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.153 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.153 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.154 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.154 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.154 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.154 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.154 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.154 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.155 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.155 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.155 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.155 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.155 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.155 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.155 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.156 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.156 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.156 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.156 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.156 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.156 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.157 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.157 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.157 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.157 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.157 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.158 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.158 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-03-19T01:25:35.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-03-19T01:25:35.158 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.158 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.158 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.159 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-03-19T01:25:35.159 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.159 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.159 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.159 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-19T01:25:35.159 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.160 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.160 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.160 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.160 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.160 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.160 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.161 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-03-19T01:25:35.161 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.161 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.161 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.161 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.161 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.162 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.162 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.162 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.162 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.162 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.162 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.162 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.163 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.163 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.163 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.163 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.163 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.163 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.164 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.164 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.164 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.164 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.164 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.164 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.164 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.165 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.165 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.165 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.165 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.165 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.165 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.165 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.165 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.166 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.166 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.166 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.166 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.166 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.166 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.166 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.166 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.167 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.167 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.167 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.167 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.167 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.167 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.167 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.168 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.168 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.168 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.168 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.168 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.168 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.168 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.168 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.169 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.169 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.169 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.169 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.169 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.169 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.169 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.169 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.170 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.170 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.170 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.170 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.170 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.170 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.170 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.170 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.171 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.171 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-03-19T01:25:35.171 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-03-19T01:25:35.171 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.171 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.171 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.171 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.171 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.172 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.172 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.172 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.172 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.172 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.172 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.172 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-03-19T01:25:35.173 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-03-19T01:25:35.173 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.173 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.173 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.173 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-03-19T01:25:35.173 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.173 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.173 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.173 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.174 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.174 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.174 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.174 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-03-19T01:25:35.174 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-03-19T01:25:35.174 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.174 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.175 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.175 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.175 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.175 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.175 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.175 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.175 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.175 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.176 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.176 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.176 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-03-19T01:25:35.176 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-03-19T01:25:35.176 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.176 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.177 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.177 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.177 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.177 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.177 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.177 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.177 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.178 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.178 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.178 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.178 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-03-19T01:25:35.178 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-03-19T01:25:35.178 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:35.178 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.179 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.179 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.179 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.179 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.179 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.179 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.180 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.180 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.180 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.180 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.180 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-03-19T01:25:35.180 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-03-19T01:25:35.180 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.181 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.181 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.181 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.181 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.181 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.181 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.181 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.182 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.182 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.182 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.182 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.182 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-03-19T01:25:35.182 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-03-19T01:25:35.182 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.183 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.183 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.183 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.183 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.183 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.183 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.183 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.183 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.183 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.184 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.184 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.184 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-03-19T01:25:35.184 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-03-19T01:25:35.184 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.184 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.184 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.184 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.185 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.185 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.185 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.185 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.185 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.185 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.186 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.186 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.186 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-03-19T01:25:35.186 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-03-19T01:25:35.186 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.186 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.187 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.187 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.187 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:35.187 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-03-19T01:25:35.187 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.188 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.188 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.188 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.188 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.188 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.188 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-03-19T01:25:35.188 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-03-19T01:25:35.189 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.189 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.189 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.189 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.189 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.189 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.189 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.190 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.190 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.190 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.190 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.190 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.190 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-03-19T01:25:35.190 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-03-19T01:25:35.191 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.191 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.191 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.191 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.191 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.191 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.192 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.192 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.192 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.192 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.192 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.192 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.192 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-03-19T01:25:35.193 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-03-19T01:25:35.193 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.193 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.193 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.193 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.193 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.193 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.194 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.194 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.194 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.194 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.194 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.194 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.195 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.195 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.195 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.195 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-03-19T01:25:35.195 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.195 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.195 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.196 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-03-19T01:25:35.196 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-03-19T01:25:35.196 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.196 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.196 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.196 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.196 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.197 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.197 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.197 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-03-19T01:25:35.197 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.197 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.197 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.197 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.198 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-03-19T01:25:35.198 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-03-19T01:25:35.198 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.198 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.198 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.198 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.199 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.199 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.199 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.199 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-03-19T01:25:35.199 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.199 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.199 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.199 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.199 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.200 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.200 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.200 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.200 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.200 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.200 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.200 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.201 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.201 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.201 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.201 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.201 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.201 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.201 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.201 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.201 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.202 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.202 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.202 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.202 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.202 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.202 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.202 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.203 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.203 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.203 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.203 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.203 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.203 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.203 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.203 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.203 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.204 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.204 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.204 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.204 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.204 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.204 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.204 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.204 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.205 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.205 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.205 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.205 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.205 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.205 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.205 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.205 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.206 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.206 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.206 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.206 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.206 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.206 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.206 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.206 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.207 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.207 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.207 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.207 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.207 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.207 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.207 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.207 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.207 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.208 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.208 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.208 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.208 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.208 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.208 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.208 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.208 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-03-19T01:25:35.209 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.209 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.209 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.209 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.209 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.209 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.209 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.210 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.210 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.210 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.210 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.210 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.210 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.210 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.211 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.211 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.211 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.211 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.211 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.211 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.211 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.211 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.212 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.212 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.212 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.212 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.212 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.212 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.213 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.213 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.213 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.213 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.213 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.213 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.214 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.214 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.214 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.214 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.214 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.214 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.214 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.215 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.215 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.215 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.215 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.215 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.215 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.215 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.216 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.216 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.216 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.216 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.216 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.216 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.216 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.217 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.217 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.217 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.217 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.217 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.218 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.218 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.218 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.218 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.218 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.218 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.218 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.219 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.219 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.219 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.219 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.219 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.219 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.219 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-03-19T01:25:35.220 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.220 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.220 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.220 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.220 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.220 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.220 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.221 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.221 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.221 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.221 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.221 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.221 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.221 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.222 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.222 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.222 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.222 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.222 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.222 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.222 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.223 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.223 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.223 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.223 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.223 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.223 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.223 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.224 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.224 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.224 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.224 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.224 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.224 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.224 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.225 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.225 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.225 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.225 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.225 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.225 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.226 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.226 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.226 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.226 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.226 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.226 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.226 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.227 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.227 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.227 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.227 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.227 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.227 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.227 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.227 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.228 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.228 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.228 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.228 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.228 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.228 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.228 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.228 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.228 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.229 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.229 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.229 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.229 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.229 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.229 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.229 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.229 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.230 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.230 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.230 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-03-19T01:25:35.230 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.230 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.230 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.230 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-03-19T01:25:35.230 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-03-19T01:25:35.230 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-19T01:25:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.231 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.232 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.233 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-03-19T01:25:35.233 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.233 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.233 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.233 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.233 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.233 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.233 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.234 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.234 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.234 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.234 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.234 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.234 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.234 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.234 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.235 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.235 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.235 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.235 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.235 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.235 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.235 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.235 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.236 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.236 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.236 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.236 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.236 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.236 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.236 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.236 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.237 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.237 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.237 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.237 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.237 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.237 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.237 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.238 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.238 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.238 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.238 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.238 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.238 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.238 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.239 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.239 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.239 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.239 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.239 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.239 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.239 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.239 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.240 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.240 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.240 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.240 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.240 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.240 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.240 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.241 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.241 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.241 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.241 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.241 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.241 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.241 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.242 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.242 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.242 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.242 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.242 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.242 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.243 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.243 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.243 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.243 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.243 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.243 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.244 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-03-19T01:25:35.244 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.244 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.244 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.244 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-03-19T01:25:35.244 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-03-19T01:25:35.245 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.245 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.245 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.245 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.245 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.245 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.245 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.246 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-19T01:25:35.246 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.246 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.246 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.246 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.246 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-03-19T01:25:35.246 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-03-19T01:25:35.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.247 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.247 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.247 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.247 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.247 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.247 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.247 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-19T01:25:35.248 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.248 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.248 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.248 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.248 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-03-19T01:25:35.248 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-03-19T01:25:35.248 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.249 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.249 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.249 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.249 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.249 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.249 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.249 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-19T01:25:35.250 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.250 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.250 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.250 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.250 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-03-19T01:25:35.250 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-03-19T01:25:35.250 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.251 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.251 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.251 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.251 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.251 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.251 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.251 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-19T01:25:35.252 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.252 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.252 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.252 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.252 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-03-19T01:25:35.252 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-03-19T01:25:35.253 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.253 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.253 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.253 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.253 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.253 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.253 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.254 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-19T01:25:35.254 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.254 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.254 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.254 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.254 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-03-19T01:25:35.254 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-03-19T01:25:35.255 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.255 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.255 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.255 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.255 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.255 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.255 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.256 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.256 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.256 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.256 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.256 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.256 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-03-19T01:25:35.257 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-03-19T01:25:35.257 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.257 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.257 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.257 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.257 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.258 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.258 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.258 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.258 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.258 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.258 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.258 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.259 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-03-19T01:25:35.259 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-03-19T01:25:35.259 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.259 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.259 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.259 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-19T01:25:35.259 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.259 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.259 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.260 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.260 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.260 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.260 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.260 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.260 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-03-19T01:25:35.260 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-03-19T01:25:35.260 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.261 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.261 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.261 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.261 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.261 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.261 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.261 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.261 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.261 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.262 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.262 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:35.262 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:35.262 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.262 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.262 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.262 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-03-19T01:25:35.263 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-03-19T01:25:35.263 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.263 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.263 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.263 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.263 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.263 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.263 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-03-19T01:25:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-03-19T01:25:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-03-19T01:25:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.264 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.265 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.265 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.265 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.265 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.265 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-03-19T01:25:35.265 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-03-19T01:25:35.265 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.266 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.266 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.266 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.266 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.266 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.266 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.266 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.266 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.267 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.267 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.267 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-03-19T01:25:35.267 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-03-19T01:25:35.267 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.267 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.267 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.268 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.268 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.268 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.268 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.268 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.268 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.268 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.268 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.268 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.269 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.269 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.269 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.269 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.269 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.269 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.270 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.270 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.270 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.270 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.270 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.270 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.271 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.271 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.271 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.271 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.271 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.271 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.272 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.272 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.272 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.272 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.272 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.272 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.272 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.272 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.273 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.273 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.273 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.273 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.273 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.273 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.274 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.274 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.274 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.274 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.274 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.274 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.274 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.275 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.275 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.275 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.275 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.275 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.275 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.275 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.276 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.276 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.276 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.276 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.276 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.276 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.276 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.277 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.277 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.277 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.277 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.277 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.277 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.278 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.278 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.278 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.278 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.278 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.278 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.279 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.279 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-03-19T01:25:35.279 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-03-19T01:25:35.279 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.279 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.279 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.280 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.280 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.280 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.280 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.280 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.280 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.281 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.281 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.281 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-03-19T01:25:35.281 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-03-19T01:25:35.282 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.282 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.282 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.282 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.282 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.283 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.283 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.283 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.283 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.283 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.284 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.284 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-03-19T01:25:35.284 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-03-19T01:25:35.284 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.284 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.284 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.285 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-19T01:25:35.285 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.285 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.285 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.285 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.286 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.286 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.286 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.286 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.286 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.286 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.287 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.287 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-03-19T01:25:35.287 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.287 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.287 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.287 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.288 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.288 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.288 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.288 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.288 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.289 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.289 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.289 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.289 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.289 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.289 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.290 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.290 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.290 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.290 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.290 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.290 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.291 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.291 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.291 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.291 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.291 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.291 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.291 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.291 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.292 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.292 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.292 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.292 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.292 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.292 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.292 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.293 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.293 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.293 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.293 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.293 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.293 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.294 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.294 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.294 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.294 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.294 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.294 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.294 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.295 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.295 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.295 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.295 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.295 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.295 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.295 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.295 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.296 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.296 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.296 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.296 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.296 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.296 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.296 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.297 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.297 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.297 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.297 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.297 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.297 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.297 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.297 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.298 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.298 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-03-19T01:25:35.298 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.298 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.298 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.298 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.299 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.299 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.299 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.299 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.299 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.299 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.299 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.300 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.300 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.300 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.300 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.300 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.300 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.300 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.301 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.301 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.301 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.301 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.302 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.302 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.302 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.302 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.302 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.302 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.303 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.303 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.303 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.303 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.303 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.303 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.303 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.304 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.304 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.304 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.304 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.304 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.305 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.305 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.305 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.305 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.305 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.305 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.305 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.306 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.306 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.306 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.306 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.306 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.306 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.307 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.307 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.307 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.307 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.307 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.307 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.308 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.308 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.308 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.308 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.308 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.308 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.309 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.309 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.309 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.309 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-03-19T01:25:35.309 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-03-19T01:25:35.309 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.309 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.310 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.310 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-03-19T01:25:35.310 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.310 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.310 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.310 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-19T01:25:35.310 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.310 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.311 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.311 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-03-19T01:25:35.311 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-03-19T01:25:35.311 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.311 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.311 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.312 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.312 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.312 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.312 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-03-19T01:25:35.312 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.313 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.313 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.313 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-03-19T01:25:35.313 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-03-19T01:25:35.313 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.314 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.314 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.314 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:35.314 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.314 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.314 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.314 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-19T01:25:35.315 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.315 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.315 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.315 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.315 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-03-19T01:25:35.315 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-03-19T01:25:35.315 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.316 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.316 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.316 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.316 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.316 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.317 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.317 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-19T01:25:35.317 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.317 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.317 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.317 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.317 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.318 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.318 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.318 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-03-19T01:25:35.318 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.318 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.318 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.318 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-03-19T01:25:35.319 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-03-19T01:25:35.319 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:35.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.319 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.319 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-03-19T01:25:35.319 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.320 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.320 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.320 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.320 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.320 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.320 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.320 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-03-19T01:25:35.321 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-03-19T01:25:35.321 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.321 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.321 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.321 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.322 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.322 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.322 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.322 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.322 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:35.323 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:35.323 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.323 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.323 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:35.323 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.324 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.324 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.324 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.324 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.324 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.324 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.324 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.325 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.325 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.325 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.325 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.325 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.325 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.326 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.326 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.326 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.326 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.326 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.326 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.326 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.327 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.327 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.327 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.327 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.327 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.327 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.328 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.328 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.328 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.328 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.328 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.328 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.329 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.329 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.329 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.329 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.329 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.330 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.330 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.330 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.330 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.330 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.330 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.331 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.331 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.335 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.335 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.335 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.335 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.335 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.335 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.336 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.336 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.336 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.336 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.336 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.336 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.336 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.337 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.337 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.337 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.337 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.337 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.338 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.338 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.338 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.338 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.338 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.338 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.339 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.339 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.339 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.339 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.339 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-03-19T01:25:35.340 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-03-19T01:25:35.340 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.340 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.340 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.340 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.340 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.340 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.341 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.341 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.341 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.341 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.341 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.341 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-03-19T01:25:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-03-19T01:25:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-03-19T01:25:35.342 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.343 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.344 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.344 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-03-19T01:25:35.344 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-03-19T01:25:35.344 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.344 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.344 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-19T01:25:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.345 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.346 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.346 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.346 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.346 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.346 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T01:25:35.347 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.347 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.347 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 25642, 2024-03-19T01:25:35.347 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "z", 2024-03-19T01:25:35.347 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540701547738038271, 2024-03-19T01:25:35.347 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-03-19T01:25:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-03-19T01:25:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.348 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:35.349 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:35.349 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.349 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.349 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.349 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:35.349 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.350 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.350 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.350 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-03-19T01:25:35.350 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.350 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.350 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.350 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.351 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.351 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.351 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.351 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.351 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.351 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.351 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.352 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.352 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.352 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.352 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.352 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.352 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.352 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.352 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.352 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.353 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.353 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.353 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.353 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.353 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.353 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.353 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.353 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.354 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.354 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.354 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.354 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.354 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.355 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.355 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.355 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.355 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.355 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.355 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.355 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.355 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.356 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.356 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.356 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.356 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.356 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.356 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.356 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.357 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.357 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.357 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.357 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.357 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.357 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.357 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.358 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.358 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.358 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.358 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.358 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.358 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.358 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.359 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.359 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.359 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.359 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.359 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.359 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.359 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.360 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.360 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.360 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.360 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.360 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.360 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-03-19T01:25:35.360 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-03-19T01:25:35.361 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.361 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.361 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.361 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.361 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.361 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.362 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.362 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-03-19T01:25:35.362 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.362 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.362 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.363 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.363 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-03-19T01:25:35.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-03-19T01:25:35.363 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.363 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.363 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.364 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-03-19T01:25:35.364 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.364 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.364 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.365 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-03-19T01:25:35.365 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.365 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.365 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.365 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.366 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-03-19T01:25:35.366 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-03-19T01:25:35.366 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.366 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.367 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.367 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.367 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.367 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.367 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.368 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-03-19T01:25:35.368 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.368 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.368 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.368 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.368 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-03-19T01:25:35.369 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-03-19T01:25:35.369 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.369 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.369 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.369 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.369 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.369 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.370 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.370 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-03-19T01:25:35.370 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.370 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.370 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.370 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.371 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-03-19T01:25:35.371 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-03-19T01:25:35.371 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.371 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.371 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.371 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-03-19T01:25:35.371 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.372 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.372 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.372 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-03-19T01:25:35.372 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.372 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.372 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.373 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.373 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-03-19T01:25:35.373 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-03-19T01:25:35.373 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.373 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.373 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.373 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.373 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.374 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.374 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.374 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-03-19T01:25:35.374 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.374 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.374 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.374 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-03-19T01:25:35.374 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-03-19T01:25:35.375 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.375 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.375 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.375 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.375 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.375 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.375 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.376 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-19T01:25:35.376 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.376 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.376 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.376 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.376 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-03-19T01:25:35.377 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-03-19T01:25:35.377 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.377 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.377 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.377 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.377 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.378 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.378 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.378 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-03-19T01:25:35.378 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.378 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.378 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.378 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.379 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.379 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.379 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.379 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-03-19T01:25:35.379 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.379 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.380 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.380 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-03-19T01:25:35.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-03-19T01:25:35.380 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.380 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.380 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.380 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.380 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.380 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.381 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.381 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-03-19T01:25:35.381 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.381 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.381 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.381 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.381 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-03-19T01:25:35.381 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-03-19T01:25:35.382 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.382 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.382 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.382 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-03-19T01:25:35.382 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.382 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.382 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.382 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-03-19T01:25:35.383 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T01:25:35.383 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.383 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.383 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.383 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-03-19T01:25:35.383 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-03-19T01:25:35.383 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:35.383 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.384 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.384 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:35.384 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-03-19T01:25:35.384 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-03-19T01:25:35.384 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.384 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-19T01:25:35.384 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T01:25:35.384 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.384 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.385 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.385 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-03-19T01:25:35.385 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-03-19T01:25:35.385 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:35.385 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.385 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.385 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-03-19T01:25:35.385 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:35.385 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-03-19T01:25:35.386 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.386 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-19T01:25:35.386 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.386 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.386 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.386 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.386 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-03-19T01:25:35.386 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-03-19T01:25:35.387 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.387 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.387 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.387 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-03-19T01:25:35.387 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.387 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.387 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.387 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-19T01:25:35.387 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-19T01:25:35.388 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.388 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.388 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.388 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-03-19T01:25:35.388 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-03-19T01:25:35.388 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:35.388 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.388 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.388 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-03-19T01:25:35.389 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-03-19T01:25:35.389 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-03-19T01:25:35.389 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.389 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-03-19T01:25:35.389 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-19T01:25:35.389 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.389 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.389 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.390 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-03-19T01:25:35.390 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-03-19T01:25:35.390 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.390 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.390 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.390 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-03-19T01:25:35.390 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.391 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.391 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.391 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-03-19T01:25:35.391 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T01:25:35.391 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.391 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.391 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.391 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-03-19T01:25:35.391 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-03-19T01:25:35.392 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:35.392 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.392 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.392 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:35.392 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-03-19T01:25:35.393 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-03-19T01:25:35.393 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.393 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-19T01:25:35.393 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T01:25:35.393 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.393 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.393 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.394 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.394 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.394 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.394 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.394 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.395 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.395 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.395 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.395 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.395 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.395 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.396 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.396 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.396 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.396 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.396 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.397 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.397 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.397 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.397 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.397 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.397 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.398 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.398 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.398 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.398 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.398 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.399 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.399 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.399 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.399 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.399 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.399 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.400 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.400 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.400 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.400 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.400 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.401 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.401 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.401 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.401 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.401 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.402 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.402 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.402 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.402 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.402 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.403 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.403 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.403 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.403 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.406 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.406 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.406 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.406 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.406 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.406 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.407 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-03-19T01:25:35.407 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-03-19T01:25:35.407 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:35.407 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.407 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.407 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:35.407 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.407 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.408 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.408 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-19T01:25:35.408 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.408 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.408 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.409 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-03-19T01:25:35.409 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-03-19T01:25:35.409 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.409 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.409 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.409 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-03-19T01:25:35.410 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.410 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.410 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.410 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-03-19T01:25:35.410 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-03-19T01:25:35.410 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-03-19T01:25:35.410 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-03-19T01:25:35.411 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-03-19T01:25:35.411 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.411 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-03-19T01:25:35.411 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.411 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.411 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.411 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.411 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-03-19T01:25:35.412 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-03-19T01:25:35.412 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.412 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.412 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.412 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.412 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.412 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.412 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-19T01:25:35.413 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.413 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.413 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.413 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.413 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:35.413 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:35.413 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.413 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.414 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.414 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:35.414 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.414 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.414 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.414 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-19T01:25:35.414 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.414 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.415 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-03-19T01:25:35.415 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-03-19T01:25:35.415 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.415 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.415 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.415 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:35.415 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:35.416 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.416 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.416 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-19T01:25:35.416 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-19T01:25:35.416 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.416 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.416 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-03-19T01:25:35.416 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-03-19T01:25:35.417 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:35.417 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.417 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.417 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:35.417 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.417 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.417 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.417 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-19T01:25:35.417 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.418 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.418 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.418 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.418 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.418 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.418 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.418 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-03-19T01:25:35.419 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.419 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.419 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.419 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-03-19T01:25:35.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-03-19T01:25:35.419 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:35.419 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.419 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-03-19T01:25:35.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.420 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.420 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-19T01:25:35.420 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.420 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.420 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.421 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-03-19T01:25:35.421 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-03-19T01:25:35.421 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.421 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.421 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.421 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-03-19T01:25:35.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.422 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.422 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-19T01:25:35.422 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.422 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.422 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.423 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.423 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-03-19T01:25:35.423 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-03-19T01:25:35.423 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.423 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.423 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.423 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-03-19T01:25:35.424 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.424 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.424 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.424 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-19T01:25:35.424 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.424 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.425 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.425 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-03-19T01:25:35.425 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-03-19T01:25:35.425 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.425 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.425 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.426 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.426 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.426 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.426 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.426 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-19T01:25:35.426 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-19T01:25:35.427 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.427 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.427 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-03-19T01:25:35.428 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-03-19T01:25:35.428 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.428 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.428 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.428 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:35.428 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.428 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.429 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.429 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-19T01:25:35.429 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.429 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.429 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.429 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.430 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-03-19T01:25:35.430 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-03-19T01:25:35.430 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:35.430 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.430 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.431 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-03-19T01:25:35.431 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.431 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.431 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.431 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-19T01:25:35.431 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.432 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.432 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.432 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.432 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-03-19T01:25:35.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-03-19T01:25:35.433 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.433 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.433 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.433 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.433 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.433 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.434 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.434 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-03-19T01:25:35.434 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-03-19T01:25:35.434 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-03-19T01:25:35.435 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.435 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-19T01:25:35.435 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.435 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.435 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.435 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-03-19T01:25:35.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-03-19T01:25:35.436 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.436 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.436 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.436 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.436 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.437 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.437 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.437 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-19T01:25:35.437 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.437 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.437 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.438 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-03-19T01:25:35.438 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-03-19T01:25:35.438 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.438 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.438 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.438 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.439 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.439 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.439 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.439 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-19T01:25:35.439 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.439 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.439 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.439 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.440 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-03-19T01:25:35.440 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-03-19T01:25:35.440 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.440 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.440 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.440 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.440 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.440 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.440 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.441 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-19T01:25:35.441 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.441 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.441 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.441 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-03-19T01:25:35.441 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-03-19T01:25:35.441 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.441 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.442 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-19T01:25:35.442 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.442 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.442 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.442 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:35.442 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.443 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.443 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.443 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.443 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-03-19T01:25:35.443 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-03-19T01:25:35.443 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.443 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.443 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.443 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-19T01:25:35.444 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.444 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.444 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-03-19T01:25:35.444 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.444 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.444 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.444 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.445 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-03-19T01:25:35.445 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-03-19T01:25:35.445 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.445 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.445 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.445 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-19T01:25:35.445 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.445 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.445 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.446 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-03-19T01:25:35.446 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.446 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.446 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.446 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.446 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-03-19T01:25:35.446 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-03-19T01:25:35.446 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.447 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.447 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.447 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-19T01:25:35.447 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.447 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.447 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.447 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:35.447 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.447 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.448 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.448 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.448 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-03-19T01:25:35.448 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-03-19T01:25:35.448 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.448 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.448 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.448 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-19T01:25:35.448 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.449 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.449 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.449 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-03-19T01:25:35.449 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.449 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.449 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.449 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.449 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-03-19T01:25:35.450 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-03-19T01:25:35.450 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.450 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.450 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.450 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-19T01:25:35.450 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.451 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-03-19T01:25:35.451 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.451 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.451 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.451 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.451 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-03-19T01:25:35.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-03-19T01:25:35.451 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.452 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.452 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.452 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-19T01:25:35.452 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.452 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.452 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-03-19T01:25:35.453 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.453 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.453 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.453 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.453 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-03-19T01:25:35.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-03-19T01:25:35.453 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.454 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.454 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.454 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-19T01:25:35.454 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.454 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.454 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.454 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-03-19T01:25:35.454 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.455 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.455 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.455 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.455 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-03-19T01:25:35.455 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-03-19T01:25:35.455 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.455 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.456 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.456 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-19T01:25:35.456 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.456 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.456 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.456 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-03-19T01:25:35.456 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.457 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.457 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.457 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-03-19T01:25:35.457 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-03-19T01:25:35.457 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.458 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.458 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.458 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-19T01:25:35.458 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.458 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.458 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.458 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-03-19T01:25:35.459 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.459 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.459 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.459 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.459 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-03-19T01:25:35.459 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-03-19T01:25:35.459 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.460 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.460 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.460 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-19T01:25:35.460 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.460 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.460 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.460 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:35.461 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.461 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.461 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.461 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-03-19T01:25:35.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-03-19T01:25:35.461 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.461 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.462 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-19T01:25:35.462 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.462 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.462 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-03-19T01:25:35.463 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.463 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.463 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.463 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-03-19T01:25:35.463 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-03-19T01:25:35.463 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.464 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.464 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.464 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-19T01:25:35.464 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.464 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.464 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.464 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:35.465 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.465 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.465 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.465 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.465 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-03-19T01:25:35.465 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-03-19T01:25:35.465 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.466 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.466 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.466 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-19T01:25:35.466 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.466 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.466 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-03-19T01:25:35.467 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.467 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.467 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.467 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-03-19T01:25:35.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-03-19T01:25:35.467 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.468 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.468 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-19T01:25:35.468 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.468 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.468 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.468 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-03-19T01:25:35.469 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.469 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.469 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.469 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-03-19T01:25:35.469 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-03-19T01:25:35.469 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.470 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.470 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.470 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.470 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.470 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-03-19T01:25:35.470 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.471 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.471 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.471 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.471 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-03-19T01:25:35.471 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-03-19T01:25:35.471 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.471 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.471 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.472 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:35.472 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.472 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.472 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.472 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-19T01:25:35.472 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.472 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.473 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.473 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-03-19T01:25:35.473 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-03-19T01:25:35.473 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.473 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.473 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.473 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-03-19T01:25:35.473 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.474 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.474 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.474 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-19T01:25:35.474 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.474 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.474 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.474 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-03-19T01:25:35.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-03-19T01:25:35.475 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.475 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.475 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.475 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-03-19T01:25:35.475 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.475 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-19T01:25:35.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.476 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.476 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.476 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.476 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-03-19T01:25:35.476 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-03-19T01:25:35.476 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.476 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.477 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.477 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-03-19T01:25:35.477 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.477 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.477 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.477 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-03-19T01:25:35.477 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.477 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.478 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.478 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-03-19T01:25:35.478 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-03-19T01:25:35.478 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.478 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.478 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.478 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.478 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.479 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.479 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.479 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-19T01:25:35.479 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.479 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.479 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.479 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.479 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-03-19T01:25:35.480 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-03-19T01:25:35.480 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.480 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.480 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.480 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:35.480 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.480 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.480 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.481 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-03-19T01:25:35.481 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.481 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.481 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.481 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.481 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-03-19T01:25:35.481 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-03-19T01:25:35.482 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.482 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.482 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.482 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:35.482 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.482 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.482 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.482 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-03-19T01:25:35.483 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.483 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.483 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.483 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-03-19T01:25:35.483 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-03-19T01:25:35.483 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.484 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.484 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.484 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.484 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.484 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-19T01:25:35.485 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.485 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.485 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.485 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-03-19T01:25:35.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-03-19T01:25:35.486 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.486 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.486 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.486 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.486 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.486 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.487 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-19T01:25:35.487 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.487 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.487 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.487 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.487 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.488 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.488 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.488 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.488 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.488 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.488 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.488 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.489 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.489 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.489 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.489 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.489 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.489 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.489 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.490 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.490 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.490 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.490 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.490 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.490 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.491 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-03-19T01:25:35.491 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-03-19T01:25:35.491 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.491 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.491 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.491 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.491 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.492 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.492 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.492 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-19T01:25:35.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.492 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.492 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.493 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.493 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.493 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.493 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.493 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.493 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.493 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.494 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.494 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.494 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.494 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-19T01:25:35.494 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.494 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.494 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.495 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.495 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.495 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.495 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.495 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.496 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.496 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.496 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.496 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.496 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.496 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.496 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.497 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.497 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.497 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.497 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.497 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.497 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.497 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.498 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.498 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.498 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.498 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.498 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.498 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.498 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.499 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.499 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.499 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.499 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.499 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.499 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.499 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.500 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.500 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.500 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-19T01:25:35.500 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-19T01:25:35.500 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.500 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.500 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.500 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.500 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.501 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.501 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.501 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-19T01:25:35.501 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.501 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.501 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.501 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-19T01:25:35.502 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-19T01:25:35.502 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.502 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.502 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.502 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-03-19T01:25:35.502 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.502 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.502 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.503 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-19T01:25:35.503 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.503 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.503 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.503 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.503 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-03-19T01:25:35.503 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-03-19T01:25:35.503 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.504 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.504 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.504 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:35.504 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.504 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.504 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.504 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-03-19T01:25:35.504 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.505 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.505 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.505 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-03-19T01:25:35.505 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-03-19T01:25:35.505 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.505 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.506 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:35.506 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.506 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.506 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-19T01:25:35.506 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.506 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.506 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.506 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.507 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-03-19T01:25:35.507 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-03-19T01:25:35.507 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.507 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.507 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.507 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.507 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.507 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.508 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-03-19T01:25:35.508 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.508 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-03-19T01:25:35.508 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-03-19T01:25:35.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.509 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.509 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.509 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-03-19T01:25:35.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.509 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.510 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-03-19T01:25:35.510 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-03-19T01:25:35.510 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.510 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-03-19T01:25:35.510 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.510 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.511 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.511 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.511 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-03-19T01:25:35.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-03-19T01:25:35.511 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.511 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.511 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.511 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-03-19T01:25:35.512 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.512 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.512 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.512 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-19T01:25:35.512 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.512 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.512 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.512 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.513 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-03-19T01:25:35.513 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-03-19T01:25:35.513 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.513 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.513 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.513 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-19T01:25:35.513 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.513 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.514 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.514 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-19T01:25:35.514 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.514 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.514 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.514 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.514 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-03-19T01:25:35.514 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-03-19T01:25:35.515 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.515 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.515 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.515 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.515 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.515 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.515 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.516 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-19T01:25:35.516 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.516 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.516 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.516 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.516 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-03-19T01:25:35.516 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-03-19T01:25:35.517 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.517 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.517 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.517 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.517 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.517 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.517 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.518 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-19T01:25:35.518 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.518 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.518 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.518 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.518 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-03-19T01:25:35.518 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-03-19T01:25:35.519 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.519 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.519 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.519 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.519 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.519 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.519 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.520 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-19T01:25:35.520 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.520 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.520 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.520 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.520 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-03-19T01:25:35.520 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-03-19T01:25:35.521 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.521 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.521 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.522 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.522 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-19T01:25:35.522 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.522 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.522 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.522 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.523 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-03-19T01:25:35.523 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-03-19T01:25:35.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.523 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.523 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.523 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.524 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.524 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.524 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-19T01:25:35.524 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.524 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.525 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-03-19T01:25:35.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-03-19T01:25:35.525 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.525 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.525 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.526 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-03-19T01:25:35.526 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.526 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.526 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.526 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-03-19T01:25:35.526 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.526 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.527 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.527 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.527 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-03-19T01:25:35.527 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-03-19T01:25:35.527 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.527 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.527 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.528 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.528 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.528 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.528 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.528 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-19T01:25:35.528 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.529 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.529 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.529 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.529 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-03-19T01:25:35.529 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-03-19T01:25:35.529 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.529 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.530 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.530 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.530 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.530 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.530 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-19T01:25:35.530 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.531 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.531 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.531 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.531 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-03-19T01:25:35.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-03-19T01:25:35.531 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.531 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.531 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.532 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.532 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.532 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.532 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.532 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-19T01:25:35.532 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.532 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.532 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.532 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.533 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-03-19T01:25:35.533 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-03-19T01:25:35.533 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.533 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.533 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.533 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.533 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.533 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.533 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.534 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-03-19T01:25:35.534 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.534 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.534 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.534 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.538 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-03-19T01:25:35.538 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-03-19T01:25:35.538 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.538 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.538 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.539 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.539 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.539 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.539 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-19T01:25:35.539 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.539 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.539 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.540 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-03-19T01:25:35.540 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-03-19T01:25:35.540 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.540 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.540 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.540 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.540 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.541 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.541 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.541 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-19T01:25:35.541 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.541 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.541 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.541 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.541 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.541 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.542 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.542 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-03-19T01:25:35.542 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.542 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.542 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.542 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.542 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.542 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.543 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.543 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.543 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.543 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.543 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.543 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.543 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.543 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.544 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.544 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.544 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.544 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.544 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.544 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.544 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.544 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.545 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.545 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.545 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.545 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.545 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.545 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.545 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.546 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.546 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.546 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.546 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.546 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.546 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.547 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.547 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.547 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.547 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.547 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.547 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.547 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.548 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.548 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.548 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.548 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.548 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.549 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.549 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.549 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.549 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.549 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.549 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.549 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.550 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.550 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.550 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.550 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.550 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.550 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.550 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.551 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.551 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.551 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.551 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.551 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.551 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.551 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.552 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.552 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-03-19T01:25:35.552 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-03-19T01:25:35.552 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.552 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.552 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.553 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-03-19T01:25:35.553 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.553 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.553 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-19T01:25:35.553 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.553 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.554 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.554 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.554 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-03-19T01:25:35.554 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-03-19T01:25:35.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.554 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.554 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.555 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-03-19T01:25:35.555 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.555 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.555 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.555 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-19T01:25:35.555 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.555 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.556 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.556 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.556 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.556 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.556 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.556 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-03-19T01:25:35.556 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.557 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.557 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.557 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-19T01:25:35.557 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-19T01:25:35.557 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.557 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.557 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.558 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:35.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.558 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.558 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.558 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.558 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.559 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.559 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.559 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.559 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-03-19T01:25:35.559 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-19T01:25:35.559 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.559 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.559 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.560 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.560 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.560 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.560 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.560 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.560 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.560 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.561 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.561 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-19T01:25:35.561 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-19T01:25:35.561 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.561 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.562 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.562 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.562 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.562 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.562 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.562 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.562 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.563 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.563 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.563 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.563 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-03-19T01:25:35.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-03-19T01:25:35.563 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.563 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.563 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.564 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.564 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.564 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.564 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.564 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.564 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.564 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.564 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.565 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.565 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-19T01:25:35.565 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-19T01:25:35.565 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.565 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.565 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.565 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.566 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.566 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.566 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.566 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.566 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.566 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.566 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.567 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-03-19T01:25:35.567 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-19T01:25:35.567 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.567 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.567 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.567 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.567 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.567 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.568 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.568 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.568 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.568 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.568 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.568 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-19T01:25:35.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-19T01:25:35.569 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.569 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.569 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.569 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.569 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.569 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.570 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.570 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.570 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-19T01:25:35.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-19T01:25:35.570 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.570 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.571 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.571 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.571 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.571 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.571 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.571 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.571 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.571 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.572 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.572 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.572 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-19T01:25:35.572 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-19T01:25:35.572 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.572 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.572 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.573 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.573 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.573 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.573 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.573 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.573 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.573 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.574 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-19T01:25:35.574 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-19T01:25:35.574 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.574 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.574 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.574 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.575 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.575 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.575 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.575 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.575 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.575 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.575 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.576 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-19T01:25:35.576 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-19T01:25:35.576 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.576 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.576 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.576 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.577 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.577 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.577 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.577 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.577 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.577 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.577 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.578 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.578 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-19T01:25:35.578 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-19T01:25:35.578 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.578 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.578 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.578 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.579 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.579 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.579 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.579 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.579 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.579 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.579 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.580 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.580 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-19T01:25:35.580 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-19T01:25:35.580 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.580 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.580 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.580 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.580 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.581 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.581 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.581 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.581 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.581 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.581 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.581 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.581 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-19T01:25:35.582 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-19T01:25:35.582 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.582 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.582 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.582 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.582 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.582 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.583 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.583 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.583 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.583 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.583 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.583 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.583 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-03-19T01:25:35.584 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-19T01:25:35.584 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.584 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.584 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.584 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-03-19T01:25:35.584 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.584 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.585 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.585 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.585 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.585 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.585 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.585 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.585 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-19T01:25:35.585 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-19T01:25:35.586 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.586 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.586 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.586 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.586 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.586 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.587 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.587 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.587 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.587 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.587 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.587 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.587 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-03-19T01:25:35.588 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-03-19T01:25:35.588 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.588 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.588 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.588 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.588 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.589 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.589 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.589 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.589 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.589 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.589 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-03-19T01:25:35.590 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-19T01:25:35.590 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.590 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.590 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.590 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-03-19T01:25:35.590 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.591 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.591 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.591 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.591 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.591 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.591 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.591 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.592 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-19T01:25:35.592 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-19T01:25:35.592 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.592 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.592 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.592 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.592 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.593 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.593 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.593 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.593 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.593 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.593 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.593 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.594 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-19T01:25:35.594 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-19T01:25:35.594 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.594 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.594 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.594 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.594 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.595 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.595 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.595 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.595 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.595 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.595 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.595 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.595 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-19T01:25:35.595 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-19T01:25:35.596 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.596 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.596 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.596 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.596 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.596 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.596 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.596 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.596 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.597 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.597 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.597 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.597 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-19T01:25:35.597 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-19T01:25:35.597 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.597 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.597 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.598 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.598 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.598 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.598 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.598 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.598 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.598 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.598 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.599 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.599 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-03-19T01:25:35.599 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-19T01:25:35.599 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.599 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.599 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.599 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-03-19T01:25:35.599 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.600 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.600 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.600 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.600 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.600 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.600 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.600 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.600 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-03-19T01:25:35.600 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-19T01:25:35.601 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.601 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.601 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.601 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.601 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.601 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.601 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.601 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.601 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.602 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.602 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.602 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.602 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-19T01:25:35.602 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-19T01:25:35.602 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.602 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.602 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.603 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.603 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.603 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.603 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.603 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.603 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.603 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.603 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.604 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.604 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-19T01:25:35.604 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-19T01:25:35.604 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.604 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.604 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.604 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.605 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.605 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.605 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.605 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.605 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.605 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.605 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.606 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.606 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-19T01:25:35.606 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-19T01:25:35.606 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.606 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.606 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.606 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.606 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.607 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.607 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.607 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.607 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.607 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.607 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.608 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-19T01:25:35.608 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-19T01:25:35.608 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.608 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.608 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.608 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.608 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.609 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.609 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.609 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.609 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.609 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.609 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.609 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.610 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-19T01:25:35.610 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-19T01:25:35.610 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.610 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.610 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.611 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.611 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.611 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.611 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.611 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.611 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.611 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.612 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-19T01:25:35.612 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-19T01:25:35.612 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.612 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.612 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.612 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.612 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.613 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.613 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.613 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.613 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.613 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.613 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.613 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.614 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-19T01:25:35.614 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-19T01:25:35.614 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.614 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.614 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.614 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.614 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.615 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.615 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.615 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.615 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.615 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.615 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.615 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.616 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-19T01:25:35.616 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-19T01:25:35.616 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.616 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.616 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.616 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.617 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.617 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.617 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.617 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.617 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.617 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.617 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.618 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.618 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-03-19T01:25:35.618 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-19T01:25:35.618 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.618 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.618 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.618 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.619 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.619 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.619 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.619 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.619 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.619 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.619 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.619 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.620 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-19T01:25:35.620 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-19T01:25:35.620 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.620 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.620 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.620 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-19T01:25:35.620 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.621 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.621 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.621 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.621 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.621 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.621 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.621 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.622 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-19T01:25:35.622 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-19T01:25:35.622 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.622 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.622 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.622 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:35.622 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.623 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.623 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.623 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.623 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.623 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.623 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.623 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.623 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-19T01:25:35.624 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-19T01:25:35.624 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.624 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.624 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-03-19T01:25:35.624 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.624 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.624 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.625 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.625 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.625 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.625 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.625 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.625 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-19T01:25:35.625 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-19T01:25:35.625 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.626 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.626 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.626 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-03-19T01:25:35.626 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.626 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.626 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.626 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.626 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.627 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.627 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.627 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.627 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-03-19T01:25:35.627 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-19T01:25:35.627 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.627 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.627 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.628 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.628 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.628 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.628 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.628 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.628 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.628 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.628 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.629 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.629 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-19T01:25:35.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-19T01:25:35.629 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.629 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.629 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.629 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-03-19T01:25:35.629 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.629 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.630 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.630 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.630 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.630 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.630 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.630 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.630 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-03-19T01:25:35.630 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-19T01:25:35.631 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.631 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.631 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.631 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.631 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.631 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.631 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.631 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.632 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.632 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.632 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.632 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.632 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-03-19T01:25:35.632 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-19T01:25:35.632 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.632 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.633 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.633 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.633 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.633 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.633 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.633 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.633 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.634 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.634 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.634 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.634 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-03-19T01:25:35.634 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-19T01:25:35.634 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.634 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.635 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.635 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.635 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.635 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.635 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.635 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.635 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.636 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.636 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.636 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.636 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-19T01:25:35.636 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-19T01:25:35.636 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.636 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.636 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:35.637 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.637 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.637 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.637 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.637 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.637 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.638 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.638 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.638 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-19T01:25:35.638 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-19T01:25:35.638 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.638 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.638 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.639 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:35.639 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.639 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.639 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.639 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.639 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.640 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.640 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.640 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.640 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-19T01:25:35.640 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-19T01:25:35.640 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.640 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.641 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.641 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:35.641 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.641 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.641 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.641 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.641 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.641 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.642 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.642 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.642 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-03-19T01:25:35.642 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-03-19T01:25:35.642 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.642 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.642 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.642 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.643 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.643 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.643 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.643 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.643 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.643 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.643 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.643 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.644 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-03-19T01:25:35.644 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-03-19T01:25:35.644 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.644 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.644 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.644 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.645 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.645 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.645 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.645 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.645 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.645 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.645 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.646 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.646 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-03-19T01:25:35.646 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-03-19T01:25:35.646 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.646 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.646 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.646 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.647 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.647 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.647 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.647 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-03-19T01:25:35.647 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.647 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.647 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.648 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-03-19T01:25:35.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-03-19T01:25:35.648 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.648 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.648 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.648 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-03-19T01:25:35.649 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.649 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.649 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.649 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.649 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.649 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.650 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.650 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.650 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-03-19T01:25:35.650 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-03-19T01:25:35.650 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.650 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.650 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.651 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.651 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.651 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.651 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.651 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.651 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.652 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.652 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.652 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.652 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.652 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.652 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.652 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.653 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.653 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.653 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.653 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.653 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.653 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.653 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.653 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.654 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.654 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.654 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.654 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.654 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.654 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.654 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.654 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.654 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.655 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.655 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.655 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.655 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.655 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.655 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.655 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.655 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.656 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.656 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.656 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.656 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.656 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.656 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.656 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.656 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.656 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.657 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.657 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.657 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.657 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.657 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.657 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.657 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.657 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.658 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.658 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.658 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.658 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.658 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.658 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.658 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.658 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.659 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.659 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.659 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.659 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.659 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.659 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.659 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.659 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.660 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.660 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.660 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.660 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.660 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.660 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.660 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.660 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-03-19T01:25:35.661 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-03-19T01:25:35.661 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.661 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.661 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.661 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.661 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.661 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.661 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.662 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-03-19T01:25:35.662 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.662 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.662 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.662 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.662 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-03-19T01:25:35.662 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-03-19T01:25:35.662 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.663 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.663 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.663 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.663 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.663 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.663 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.663 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.663 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.663 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.664 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.664 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-03-19T01:25:35.664 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-03-19T01:25:35.664 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.664 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.665 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.665 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-03-19T01:25:35.665 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.665 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.665 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.665 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.666 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.666 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.666 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.666 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.666 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-03-19T01:25:35.666 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-03-19T01:25:35.667 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.667 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.667 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.667 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-03-19T01:25:35.667 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.668 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.668 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.668 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.668 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.668 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.668 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.669 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.669 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-03-19T01:25:35.669 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-03-19T01:25:35.669 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.669 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.670 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.670 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.670 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.670 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.670 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.670 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.671 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.671 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.671 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.671 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.671 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-03-19T01:25:35.672 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-03-19T01:25:35.672 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.672 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.672 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.672 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-03-19T01:25:35.672 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.673 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.673 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.673 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.673 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.673 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.674 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.674 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.674 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-03-19T01:25:35.674 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-03-19T01:25:35.674 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.674 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.675 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.675 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-03-19T01:25:35.675 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.675 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.675 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.676 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.676 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-03-19T01:25:35.676 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.676 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.676 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.676 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.676 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.677 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.677 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-03-19T01:25:35.677 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-03-19T01:25:35.677 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.677 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.678 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.678 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-03-19T01:25:35.678 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-03-19T01:25:35.678 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-03-19T01:25:35.678 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.678 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.678 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.679 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.679 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.679 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.679 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-03-19T01:25:35.679 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-03-19T01:25:35.679 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.679 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.680 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.680 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.680 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.680 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.680 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.680 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.681 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.681 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.681 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.681 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.681 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.681 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.681 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.682 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-03-19T01:25:35.682 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.682 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.682 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.682 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-03-19T01:25:35.682 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-03-19T01:25:35.682 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.683 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.683 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.683 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.683 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.683 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.683 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.683 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-03-19T01:25:35.683 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.684 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.684 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.684 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.684 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.684 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.684 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.684 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.684 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.685 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.685 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.685 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.685 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.685 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.685 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.685 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.685 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.686 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.686 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.686 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.686 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.686 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.686 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.686 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.686 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.687 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.687 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.687 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.687 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.687 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.687 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.687 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.687 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.688 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.688 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.688 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.688 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.688 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.688 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.688 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.688 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.689 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.689 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.689 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.689 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.689 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.689 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.689 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.689 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.689 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.690 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.690 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.690 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.690 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.690 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.690 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.690 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.691 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.691 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.691 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.691 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.691 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.691 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.691 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.691 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.691 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.692 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.692 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.692 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.692 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.692 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.692 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.693 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.693 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.693 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-03-19T01:25:35.693 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-03-19T01:25:35.693 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.693 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.693 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.694 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-03-19T01:25:35.694 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.694 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.694 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.694 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-19T01:25:35.694 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.694 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.695 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.695 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.695 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-03-19T01:25:35.695 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-03-19T01:25:35.695 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.695 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.695 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.696 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-03-19T01:25:35.696 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.696 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.696 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.696 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-19T01:25:35.696 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.696 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.696 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.697 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.697 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-03-19T01:25:35.697 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-03-19T01:25:35.697 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.697 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.697 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.697 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-03-19T01:25:35.698 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.698 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.698 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.698 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-03-19T01:25:35.698 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.698 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.699 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.699 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.699 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-03-19T01:25:35.699 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-03-19T01:25:35.699 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.699 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.699 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.700 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-03-19T01:25:35.700 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.700 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.700 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.700 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-19T01:25:35.700 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.700 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.701 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.701 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.701 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-03-19T01:25:35.701 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-03-19T01:25:35.701 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.701 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.701 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.702 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.702 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.702 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.702 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.702 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-19T01:25:35.702 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.702 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.703 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.703 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.703 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:35.703 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:35.703 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.703 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.703 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.704 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:35.704 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.704 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.704 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.704 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-03-19T01:25:35.704 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.704 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.704 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.705 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.705 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-03-19T01:25:35.705 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-03-19T01:25:35.705 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.705 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.706 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.706 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-03-19T01:25:35.706 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.706 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.706 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.706 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-19T01:25:35.706 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.707 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.707 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.707 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.707 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.707 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.707 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.707 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-03-19T01:25:35.708 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.708 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.708 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.708 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.708 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.708 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.708 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.708 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.709 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.709 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.709 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.709 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.709 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.709 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.710 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.710 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.710 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.710 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.710 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.710 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.711 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.711 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.711 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.711 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.711 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.711 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.711 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.712 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.712 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.712 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.712 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.712 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.712 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.712 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.712 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.712 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.713 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.713 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.713 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.713 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.713 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.713 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.713 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.713 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.714 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.714 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.714 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.714 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.714 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.714 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.714 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.714 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.715 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.715 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.715 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.715 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.715 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.715 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.715 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.715 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.716 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.716 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.716 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.716 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.716 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.716 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.716 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.716 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.717 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.717 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.717 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.717 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.717 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.717 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-03-19T01:25:35.717 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-03-19T01:25:35.717 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.717 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.718 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.718 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-03-19T01:25:35.718 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.718 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.718 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.718 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.718 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.719 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.719 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.719 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.719 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-03-19T01:25:35.719 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-03-19T01:25:35.719 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.719 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.719 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-03-19T01:25:35.720 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.720 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.720 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.720 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.720 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.720 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.721 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.721 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:35.721 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:35.721 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.721 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.721 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.721 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:35.721 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.722 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.723 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.723 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.723 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-03-19T01:25:35.723 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-03-19T01:25:35.723 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-03-19T01:25:35.723 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.723 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-03-19T01:25:35.724 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-03-19T01:25:35.724 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.724 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.724 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.724 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-03-19T01:25:35.724 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.724 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.725 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.725 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-19T01:25:35.725 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.725 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.725 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.725 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.725 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-03-19T01:25:35.725 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-03-19T01:25:35.726 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.726 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.726 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.726 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-03-19T01:25:35.726 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.726 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.726 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.727 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-19T01:25:35.727 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.727 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.727 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.727 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.727 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-03-19T01:25:35.727 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-03-19T01:25:35.728 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.728 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.728 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.728 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.728 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.728 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.729 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.729 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-03-19T01:25:35.729 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.729 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.729 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.729 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.729 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:35.730 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:35.730 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.730 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.730 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.730 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-03-19T01:25:35.730 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-03-19T01:25:35.730 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.731 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.731 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-19T01:25:35.731 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.731 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.731 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.731 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.731 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.732 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.732 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.732 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.732 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.732 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.732 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.732 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.733 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.733 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.733 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.733 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.733 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.733 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.733 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.734 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.734 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.734 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.734 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.734 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.734 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.734 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.735 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.735 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.735 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.735 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.735 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.735 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.735 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.736 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.736 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.736 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.736 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.736 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.736 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.737 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.737 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.737 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.737 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.737 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.737 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.738 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.738 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.738 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.738 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.738 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.738 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.738 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.739 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.739 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.739 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.739 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.742 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.742 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.742 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.742 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.742 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.742 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.743 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.743 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.743 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.743 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.743 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.743 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.743 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.743 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.744 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.744 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.744 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.744 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-03-19T01:25:35.744 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-03-19T01:25:35.744 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.745 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.745 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.745 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.745 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.745 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.746 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.746 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-03-19T01:25:35.746 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.746 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.746 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.746 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.746 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-03-19T01:25:35.746 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-03-19T01:25:35.747 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.747 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.747 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-03-19T01:25:35.747 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.747 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.747 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.747 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-03-19T01:25:35.747 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.748 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.748 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.748 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.748 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-03-19T01:25:35.748 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-03-19T01:25:35.748 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.748 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.748 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.749 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-03-19T01:25:35.749 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.749 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.749 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.749 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T01:25:35.749 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.749 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.749 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.750 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.750 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-03-19T01:25:35.750 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-03-19T01:25:35.750 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.750 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.750 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:35.750 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:35.751 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-03-19T01:25:35.751 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.751 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-19T01:25:35.751 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.751 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.751 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.751 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.751 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-03-19T01:25:35.751 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-03-19T01:25:35.752 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.752 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.752 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.752 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.752 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.752 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.752 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.752 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-03-19T01:25:35.753 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.753 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.753 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.753 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-03-19T01:25:35.753 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-03-19T01:25:35.753 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.753 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.753 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.754 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-03-19T01:25:35.754 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.754 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.754 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.754 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T01:25:35.754 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.754 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.755 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.755 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.755 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.755 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.755 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.755 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-03-19T01:25:35.756 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.756 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.756 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.756 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.756 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.756 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.756 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.757 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.757 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.757 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.757 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.757 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.757 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.757 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.758 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.758 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.758 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.758 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.758 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.758 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.759 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.759 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.759 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.759 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.759 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.759 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.759 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.760 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.760 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.760 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.760 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.760 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.760 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.761 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.761 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.761 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.761 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.761 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.761 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.762 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.762 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.762 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.762 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.762 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.762 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.763 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.763 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.763 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.763 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.763 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.763 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.763 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.764 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.764 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.764 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.764 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.764 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.764 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.765 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.765 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.765 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.765 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.765 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.765 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.766 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.766 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.766 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.766 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.766 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.766 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.766 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.767 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.767 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.767 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.767 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-03-19T01:25:35.767 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.767 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.767 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.767 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.768 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.768 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.768 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.768 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.768 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.768 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.768 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.768 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.769 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.769 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.769 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.769 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.769 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.769 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.769 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.770 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.770 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.770 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.770 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.770 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.770 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.771 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.771 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.771 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.771 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.771 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.771 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.771 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.772 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.772 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.772 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.772 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.772 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.772 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.772 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.773 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.773 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.773 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.773 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.773 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.773 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.773 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.773 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.774 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.774 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.774 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.774 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.774 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.774 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.774 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.774 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.774 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.775 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.775 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.775 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.775 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.775 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.775 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.775 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.775 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.776 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.776 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.776 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.776 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.776 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.776 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.776 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.777 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.777 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.777 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.777 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.777 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.777 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-03-19T01:25:35.777 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.777 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.777 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.778 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-03-19T01:25:35.778 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-03-19T01:25:35.778 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.778 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.778 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.778 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-03-19T01:25:35.778 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.778 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.779 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.779 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-19T01:25:35.779 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.779 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.779 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.779 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.779 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-03-19T01:25:35.779 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-03-19T01:25:35.779 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.780 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.780 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.780 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:35.780 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-03-19T01:25:35.780 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.780 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.780 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-19T01:25:35.780 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.781 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.781 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.781 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.781 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.781 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.781 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.781 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.781 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.781 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.782 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.782 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.782 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.782 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.782 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.782 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.782 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.782 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.783 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.783 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.783 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.783 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.783 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.783 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.783 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.783 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.783 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.784 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.784 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.784 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.784 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.784 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.784 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.784 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.784 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.785 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.785 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.785 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.785 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.785 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.785 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.785 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.785 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.785 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.786 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.786 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.786 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.786 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.786 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.786 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.786 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.787 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.787 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.787 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.787 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.787 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.787 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.787 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.788 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.788 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.788 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.788 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.788 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.788 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.788 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.788 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.789 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.789 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.789 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.789 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.789 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.789 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.789 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.790 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.790 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.790 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.790 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.790 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-03-19T01:25:35.790 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.790 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.791 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.791 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-03-19T01:25:35.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-03-19T01:25:35.791 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.791 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.791 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.791 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-03-19T01:25:35.792 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.792 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.792 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.792 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-03-19T01:25:35.792 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-19T01:25:35.792 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.792 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.793 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.793 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.793 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.793 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.793 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.793 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.793 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.794 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.794 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.794 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.794 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.794 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.794 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.794 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.795 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.795 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.795 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.795 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.795 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.795 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.795 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.796 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.796 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.796 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.796 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.796 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.796 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.797 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.797 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.797 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.797 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.797 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.797 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.798 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.798 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.798 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.798 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.798 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.798 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.798 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.799 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.799 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.799 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.799 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.799 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.799 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.799 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.800 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.800 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.800 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.800 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.800 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.800 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.800 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.801 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.801 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.801 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.801 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.801 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.801 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.801 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.802 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.802 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.802 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.802 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.802 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.802 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.802 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.803 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.803 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-03-19T01:25:35.803 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-03-19T01:25:35.803 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.803 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.803 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.803 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.804 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.804 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.804 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-03-19T01:25:35.804 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-19T01:25:35.804 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.805 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-03-19T01:25:35.805 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-03-19T01:25:35.805 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.805 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.805 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-03-19T01:25:35.805 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.805 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.805 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.806 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-03-19T01:25:35.806 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.806 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.806 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.806 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.806 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-03-19T01:25:35.806 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-03-19T01:25:35.806 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.807 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.807 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.807 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-03-19T01:25:35.807 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.807 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.807 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.807 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-03-19T01:25:35.807 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.808 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.808 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.808 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.808 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-03-19T01:25:35.808 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-03-19T01:25:35.809 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.809 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.809 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.809 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.809 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.809 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.809 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.809 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-03-19T01:25:35.809 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.810 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.810 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.810 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.810 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-03-19T01:25:35.810 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-03-19T01:25:35.810 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.810 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.810 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.811 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-03-19T01:25:35.811 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.811 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.811 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.811 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-19T01:25:35.811 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-19T01:25:35.811 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.811 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.812 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.812 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.812 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.812 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.812 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-03-19T01:25:35.812 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.812 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.812 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.812 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.813 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.813 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.813 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.813 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.813 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.813 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.813 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.813 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.814 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.814 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.814 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.814 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.814 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.814 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.814 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.815 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.815 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.815 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.815 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.815 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.815 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.815 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.816 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.816 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.816 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.816 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.816 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.816 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.816 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.816 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.817 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.817 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.817 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.817 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.817 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.817 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.818 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.818 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.818 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.818 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.818 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.818 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.818 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.819 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.819 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.819 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.819 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.819 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.819 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.820 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.820 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.820 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.820 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.820 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.820 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.821 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.821 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.821 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.821 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.821 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.821 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.822 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.822 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.822 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.822 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.822 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.822 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.822 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.823 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.823 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.823 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-03-19T01:25:35.823 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.823 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.823 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.823 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.824 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.824 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.824 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.824 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.824 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.824 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.824 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.825 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.825 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.825 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.825 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.825 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.825 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.825 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.826 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.826 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.826 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.826 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.826 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.826 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.827 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.827 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.827 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.827 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.827 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.827 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.827 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.828 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.828 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.828 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.828 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.828 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.828 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.828 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.829 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.829 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.829 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.829 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.829 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.829 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.830 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.830 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.830 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.830 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.830 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.830 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.830 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.831 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.831 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.831 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.831 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.831 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.831 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.831 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.832 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.832 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.832 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.832 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.832 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.832 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.832 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.832 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.833 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.833 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.833 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.833 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.833 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.833 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.833 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.833 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.833 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-03-19T01:25:35.834 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.834 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.834 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.834 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.834 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.834 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.835 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.835 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.835 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.835 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.835 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.835 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.835 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.836 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.836 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.836 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.836 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.836 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.836 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.836 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.836 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.836 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.837 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.837 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.837 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.837 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.837 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.837 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.837 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.837 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.838 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.838 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.838 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.838 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.838 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.838 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.838 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.838 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.839 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.839 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.839 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.839 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.839 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.839 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.839 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.839 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.839 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.840 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.840 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.840 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.840 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.840 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.840 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.840 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.840 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.841 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.841 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.841 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.841 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.841 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.841 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.841 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.841 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.841 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.842 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.842 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.842 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.842 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.842 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.842 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.842 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.842 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.842 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.843 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.843 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-03-19T01:25:35.843 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.843 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.843 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.843 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-03-19T01:25:35.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-03-19T01:25:35.844 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.844 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.844 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.844 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.844 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.844 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.844 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-19T01:25:35.845 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.845 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.845 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.845 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.845 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.845 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.845 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.846 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.846 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.846 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.846 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.846 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.846 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.846 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.847 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.847 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.847 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.847 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.847 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.847 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.847 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.848 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.848 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.848 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.848 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.848 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.848 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.848 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.849 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.849 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.849 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.849 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.849 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.849 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.849 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.849 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.850 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.850 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.850 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.850 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.850 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.850 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.851 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.851 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.851 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.851 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.851 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.851 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.851 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.852 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.852 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.852 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.852 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.852 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.852 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.852 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.852 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.853 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.853 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.853 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.853 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.853 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.853 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.854 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.854 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.854 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.854 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.854 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.854 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.854 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.855 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.855 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.855 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.855 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.855 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-03-19T01:25:35.855 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-03-19T01:25:35.855 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.856 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.856 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.856 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.856 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.856 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.856 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.856 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-03-19T01:25:35.857 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-03-19T01:25:35.857 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-03-19T01:25:35.857 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.857 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-03-19T01:25:35.857 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.857 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.857 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.858 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.858 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.858 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.858 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.858 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-03-19T01:25:35.858 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.858 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.859 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.859 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.859 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.859 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.859 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.859 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.859 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.860 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.860 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.860 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.860 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.860 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.860 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.860 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.861 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.861 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.861 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.861 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.861 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.861 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.862 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.862 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.862 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.862 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.862 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.862 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.862 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.863 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.863 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.863 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.863 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.863 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.863 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.863 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.863 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.864 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.864 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.864 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.864 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.864 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.864 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.864 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.864 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.864 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.865 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.865 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.865 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.865 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.865 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.865 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.865 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.865 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.866 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.866 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.866 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.866 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.866 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.866 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.866 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.867 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.868 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.868 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.868 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.868 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.868 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.868 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.868 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-03-19T01:25:35.868 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.869 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.869 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.869 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.869 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.869 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.869 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.869 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.869 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.870 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.870 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.870 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.870 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.870 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.870 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.870 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.870 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.870 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.871 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.871 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.871 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.871 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.871 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.871 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.871 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.871 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.871 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.872 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.872 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.872 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.872 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.872 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.872 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.872 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.873 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.873 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.873 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.873 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.873 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.873 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.873 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.874 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.874 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.874 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.874 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.874 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.874 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.874 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.874 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.875 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.875 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.875 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.875 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.875 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.875 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.875 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.876 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.876 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.876 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.876 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.876 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.876 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.877 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.877 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.877 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.877 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.877 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.877 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.877 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.878 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.878 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.878 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.878 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.878 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.878 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.878 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.879 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-03-19T01:25:35.879 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.879 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.879 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.879 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.879 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.879 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.880 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.880 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.880 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.880 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.880 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.880 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.880 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.881 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.881 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.881 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.881 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.881 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.881 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.881 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.882 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.882 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.882 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.882 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.882 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.882 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.882 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.883 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.883 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.883 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.883 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.883 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.883 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.883 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.884 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.884 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.884 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.884 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.884 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.884 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.884 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.885 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.885 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.885 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.885 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.885 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.885 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.885 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.885 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.886 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.886 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.886 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.886 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.886 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.886 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.887 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.887 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.887 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.887 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.887 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.887 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.887 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.888 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.888 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.888 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.888 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.888 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.888 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.888 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.889 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.889 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.889 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.889 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:35.889 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:35.889 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.890 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.890 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.890 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:35.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.890 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.890 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.891 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.891 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.891 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.891 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.891 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-03-19T01:25:35.891 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-03-19T01:25:35.891 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:35.891 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.892 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.892 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-03-19T01:25:35.892 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-03-19T01:25:35.892 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.892 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.892 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-03-19T01:25:35.892 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-19T01:25:35.892 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.893 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.893 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.893 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.893 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.893 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.893 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-03-19T01:25:35.893 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.893 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.894 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.894 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-03-19T01:25:35.894 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-03-19T01:25:35.894 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.894 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.894 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.894 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.894 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.895 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.895 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.895 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-19T01:25:35.895 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.895 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.895 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.895 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.895 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-03-19T01:25:35.895 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-03-19T01:25:35.896 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.896 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.896 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.896 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.896 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.896 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.896 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.896 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.897 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.897 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.897 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.897 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.897 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.897 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.897 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.897 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.897 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.898 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.898 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.898 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.898 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.898 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.898 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.898 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.898 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.899 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.899 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.899 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.899 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.899 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.899 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.899 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.900 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.900 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.900 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.900 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.900 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.900 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.900 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.900 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.900 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.901 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.901 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.901 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.901 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.901 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.901 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.902 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.902 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.902 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.902 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.902 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.902 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.903 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.903 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.903 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.903 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.903 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.903 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.903 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.903 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.904 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.904 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.904 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.904 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.904 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.904 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.904 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.904 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.905 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.905 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.905 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.905 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.905 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.905 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.905 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.905 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-03-19T01:25:35.906 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-03-19T01:25:35.906 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.906 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.906 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.906 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-03-19T01:25:35.906 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.906 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.906 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.906 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-03-19T01:25:35.907 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.907 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.907 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.907 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:35.907 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:35.907 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:35.908 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.908 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.908 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:35.908 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.908 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.908 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.908 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-03-19T01:25:35.909 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.909 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.909 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.909 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.909 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.910 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.910 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-03-19T01:25:35.910 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.910 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.910 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.910 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-03-19T01:25:35.911 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-03-19T01:25:35.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.911 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.911 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.911 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.911 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.911 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.912 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.912 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.912 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.912 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.912 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.913 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-03-19T01:25:35.913 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-03-19T01:25:35.913 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.913 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.913 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.913 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.913 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.914 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.914 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.914 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-19T01:25:35.914 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-19T01:25:35.914 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.914 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.915 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.915 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.915 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.915 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.915 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.915 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.916 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.916 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.916 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.916 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.916 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.916 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.916 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.917 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.917 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.917 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.917 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.917 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.917 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.918 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.918 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.918 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.918 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.918 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.918 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.919 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.919 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.919 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.919 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.919 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.919 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.919 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.920 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.920 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.920 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.920 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.920 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.920 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.921 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.921 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.921 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.921 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.921 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.922 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.922 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.922 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.922 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.922 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.922 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.922 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.923 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.923 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.923 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.923 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.923 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.923 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.924 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.924 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.924 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.924 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.924 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.924 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.924 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.925 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.925 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.925 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-03-19T01:25:35.925 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-03-19T01:25:35.925 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.925 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.925 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.925 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.926 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.926 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.926 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.926 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.926 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.926 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.926 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.926 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.927 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-19T01:25:35.927 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-19T01:25:35.927 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.927 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.927 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.927 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-03-19T01:25:35.927 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.927 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.928 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.928 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.928 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.928 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.928 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.928 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-03-19T01:25:35.928 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-03-19T01:25:35.928 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:35.929 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.929 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.929 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-03-19T01:25:35.929 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.929 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.929 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.929 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.929 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.930 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.930 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.930 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.930 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-03-19T01:25:35.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-03-19T01:25:35.930 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.930 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.931 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-03-19T01:25:35.931 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.931 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.931 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.931 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-19T01:25:35.931 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.931 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.932 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.932 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-03-19T01:25:35.932 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-03-19T01:25:35.932 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.932 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.932 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.932 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-03-19T01:25:35.933 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.933 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.933 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.933 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-19T01:25:35.933 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.933 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.933 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.934 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-03-19T01:25:35.934 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-03-19T01:25:35.934 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.934 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.934 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.934 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-03-19T01:25:35.934 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.935 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.935 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.935 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.935 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.935 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.935 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.936 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-03-19T01:25:35.936 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-03-19T01:25:35.936 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.936 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.936 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.936 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-03-19T01:25:35.936 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.937 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.937 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.937 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-03-19T01:25:35.937 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-03-19T01:25:35.937 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.937 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.938 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.938 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.938 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.938 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-03-19T01:25:35.938 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-03-19T01:25:35.938 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.939 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.939 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.939 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-03-19T01:25:35.939 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-03-19T01:25:35.939 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-03-19T01:25:35.939 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.939 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.940 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.940 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.940 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.940 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.940 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.940 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.941 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.941 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-03-19T01:25:35.941 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.941 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.941 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.941 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.942 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.942 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.942 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.942 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.942 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.942 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.943 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.943 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.943 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.943 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.943 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.943 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.943 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.944 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.944 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.944 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.944 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.944 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.944 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.944 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.945 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.945 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.945 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.945 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.945 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.945 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.945 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.946 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.946 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.946 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.946 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.946 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.946 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.946 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.947 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.947 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.947 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.947 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.947 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.947 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.947 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.948 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.948 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.948 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.948 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.948 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.948 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.949 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.949 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.949 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.949 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.949 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.949 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.949 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.950 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.950 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.950 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.950 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.950 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.950 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.950 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.951 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.951 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.951 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.951 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.951 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.951 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.952 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.952 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-03-19T01:25:35.952 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-03-19T01:25:35.952 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:35.952 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.952 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.952 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:35.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.953 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.953 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.953 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.953 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.953 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.953 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.953 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-03-19T01:25:35.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-03-19T01:25:35.954 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.954 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.954 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.954 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.954 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.954 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.954 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.955 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.955 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.955 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.955 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-03-19T01:25:35.955 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-03-19T01:25:35.955 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.955 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.956 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.956 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.956 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.956 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.956 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.956 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.956 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.956 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.957 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.957 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.957 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.957 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.957 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.957 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-03-19T01:25:35.957 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.957 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.957 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.958 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-03-19T01:25:35.958 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-03-19T01:25:35.958 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.958 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.958 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.958 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:35.958 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.958 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.959 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.959 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.959 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.959 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.959 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.959 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.959 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-03-19T01:25:35.959 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-03-19T01:25:35.959 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.960 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.960 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.960 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.960 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.960 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.960 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.960 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.960 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.961 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.961 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.961 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.961 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.961 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.961 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.961 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.961 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.962 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.962 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.962 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.962 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.962 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.962 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.962 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.963 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.964 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.964 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.964 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.964 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.964 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.964 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.964 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.965 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.965 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.965 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.965 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.965 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.965 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.966 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.966 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.966 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.966 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.966 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.966 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.966 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.966 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.967 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.967 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.967 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.967 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.967 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.967 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.967 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.968 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.968 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.968 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.968 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.968 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.968 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.968 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.968 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.969 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.969 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.969 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.969 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.969 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.969 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.969 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.969 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.970 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.970 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.970 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.970 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.970 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.970 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.970 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.970 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-03-19T01:25:35.971 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-03-19T01:25:35.971 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.971 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.971 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.971 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.971 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.971 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.971 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.971 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.972 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.972 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.972 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.972 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.972 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-03-19T01:25:35.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-03-19T01:25:35.972 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.973 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.973 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.973 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.973 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.973 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.973 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.973 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.974 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.974 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.974 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.974 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.974 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.974 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.974 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.975 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-03-19T01:25:35.975 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.975 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.975 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.975 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.975 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.975 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.976 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.976 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.976 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.976 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.976 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.976 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.976 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.976 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.977 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.977 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.977 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.977 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.977 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.977 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.977 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.977 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.978 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.978 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.978 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.978 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.978 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.978 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.979 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.979 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.979 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.979 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.979 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.979 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.979 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.980 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.980 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.980 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.980 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.980 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.980 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.980 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.981 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.981 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.981 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.981 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.981 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.981 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.982 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.982 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.982 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.982 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.982 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.982 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.983 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.983 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.983 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.983 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.983 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.983 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.984 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.984 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.984 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.984 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.984 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.984 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.984 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.984 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.985 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.985 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.985 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.985 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.985 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.985 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.985 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.985 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.986 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-03-19T01:25:35.986 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.986 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.986 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.986 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.986 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.986 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.986 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.987 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.987 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.987 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.987 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.987 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.987 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.987 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.987 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.987 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.988 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.988 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.988 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.988 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.988 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.988 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.988 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.988 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.988 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:35.989 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:35.989 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.989 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.989 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.989 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.989 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.989 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.989 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.990 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.990 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.990 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.990 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.990 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.990 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:35.990 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:35.991 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.991 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.991 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.991 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:35.991 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.991 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.992 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.992 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.992 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.992 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.992 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.992 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:35.992 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:35.992 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:35.992 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.993 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.993 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.993 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.993 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.993 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:35.993 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:35.993 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:35.993 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.994 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.994 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:35.994 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.994 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.994 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.994 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:35.994 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.994 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.994 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.995 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.995 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-03-19T01:25:35.995 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-03-19T01:25:35.995 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.995 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.995 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:35.995 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-03-19T01:25:35.996 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.996 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.996 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:35.996 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-19T01:25:35.996 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:35.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:35.996 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:35.996 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.997 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:35.997 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:35.997 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:35.997 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-03-19T01:25:35.997 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:35.997 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:35.997 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:35.998 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:35.998 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:35.998 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:35.998 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:35.998 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:35.998 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:35.998 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:35.998 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:35.999 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:35.999 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:35.999 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:35.999 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:35.999 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:35.999 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.000 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.000 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.000 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.000 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.000 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.000 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.000 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.000 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.001 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.001 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.001 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.001 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.001 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.001 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.001 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.002 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.002 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.002 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.002 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.002 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.003 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.003 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.003 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.003 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.003 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.003 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.003 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.004 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.004 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.004 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.004 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.004 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.004 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.004 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.005 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.005 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.005 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.005 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.005 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.005 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.005 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.006 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.006 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.006 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.006 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.006 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.006 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.007 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.007 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.007 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.007 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.007 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.007 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.007 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.007 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.008 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-03-19T01:25:36.008 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-03-19T01:25:36.008 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.008 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.008 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.008 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.009 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.009 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.009 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.009 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.009 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.009 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.010 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.010 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.010 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-03-19T01:25:36.010 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-03-19T01:25:36.010 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.010 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.011 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.011 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-03-19T01:25:36.011 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.011 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.011 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.011 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.011 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.012 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.012 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.012 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.012 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-03-19T01:25:36.012 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-03-19T01:25:36.012 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.012 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.013 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.013 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.013 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.013 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.013 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.013 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.013 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.014 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.014 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.014 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.014 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-03-19T01:25:36.014 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-03-19T01:25:36.014 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.015 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.015 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.015 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.015 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.015 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.015 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.015 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.015 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.016 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.016 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.016 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.016 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-03-19T01:25:36.016 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-03-19T01:25:36.016 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:36.016 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.016 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.017 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.017 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.017 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.017 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.017 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.017 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.017 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.018 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.018 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.018 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-03-19T01:25:36.018 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-03-19T01:25:36.018 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.019 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.019 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.019 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.019 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.019 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.020 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.020 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.020 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.020 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.020 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.021 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.021 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-03-19T01:25:36.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-03-19T01:25:36.021 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.021 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.022 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.022 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.022 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.022 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.023 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.023 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.023 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.023 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.023 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.024 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-03-19T01:25:36.024 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-03-19T01:25:36.024 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.024 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.024 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.025 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.025 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.025 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.025 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.026 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.026 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.026 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.026 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.027 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.027 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-03-19T01:25:36.027 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-03-19T01:25:36.027 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.028 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.028 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.028 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.028 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:36.029 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-03-19T01:25:36.029 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.029 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.030 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.030 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.030 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.030 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.031 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-03-19T01:25:36.031 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-03-19T01:25:36.031 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.031 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.032 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.032 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.032 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.032 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.032 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.032 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.033 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.033 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.033 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.033 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.033 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-03-19T01:25:36.034 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-03-19T01:25:36.034 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.034 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.034 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.034 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.034 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.035 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.035 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.035 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.035 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.035 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.036 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.036 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.036 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-03-19T01:25:36.036 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-03-19T01:25:36.036 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.036 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.037 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.037 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.037 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.037 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.037 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.038 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.038 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.038 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.038 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.038 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.039 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.039 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.039 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.039 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-03-19T01:25:36.039 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.039 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.039 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.040 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-03-19T01:25:36.040 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-03-19T01:25:36.040 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.040 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.040 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.040 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.040 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.041 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.041 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.041 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-03-19T01:25:36.041 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.041 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.041 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.042 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.042 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-03-19T01:25:36.042 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-03-19T01:25:36.042 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.042 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.042 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.042 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.043 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.043 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.043 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.043 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-03-19T01:25:36.043 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.043 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.043 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.044 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.044 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.044 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.044 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.044 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.044 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.044 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.044 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.045 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.045 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.045 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.045 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.045 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.045 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.045 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.045 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.046 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.046 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.046 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.046 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.046 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.046 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.047 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.047 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.047 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.047 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.047 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.047 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.047 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.047 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.048 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.048 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.048 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.048 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.048 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.048 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.048 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.049 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.049 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.049 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.049 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.049 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.049 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.049 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.049 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.050 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.050 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.050 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.050 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.050 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.050 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.050 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.050 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.051 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.051 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.051 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.051 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.051 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.051 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.052 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.052 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.052 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.052 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.052 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.052 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.052 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.053 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.053 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.053 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-03-19T01:25:36.053 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.053 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.054 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.054 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.054 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.054 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.054 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.054 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.054 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.055 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.055 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.055 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.055 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.055 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.055 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.055 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.056 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.056 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.056 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.056 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.056 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.056 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.057 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.057 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.057 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.057 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.057 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.057 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.057 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.057 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.058 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.058 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.058 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.058 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.058 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.058 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.058 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.058 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.059 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.059 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.059 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.059 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.059 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.059 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.059 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.059 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.060 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.060 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.060 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.060 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.060 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.060 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.061 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.061 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.061 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.061 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.061 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.061 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.061 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.061 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.062 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.062 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.062 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.062 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.062 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.062 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.062 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.063 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.063 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.063 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.063 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.063 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.063 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.063 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.064 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.064 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-03-19T01:25:36.064 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.064 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.064 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.064 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.064 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.065 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.065 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.065 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.065 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.065 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.065 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.065 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.066 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.066 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.066 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.066 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.066 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.066 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.067 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.067 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.067 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.067 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.067 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.067 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.067 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.068 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.068 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.068 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.068 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.068 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.068 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.069 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.069 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.069 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.069 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.069 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.069 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.069 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.070 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.070 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.070 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.070 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.070 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.070 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.071 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.071 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.071 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.071 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.071 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.071 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.072 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.072 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.072 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.072 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.072 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.072 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.073 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.073 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.073 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.073 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.073 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.074 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.074 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.074 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.074 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.074 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.074 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.075 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.075 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.075 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.075 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.075 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.076 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.076 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.076 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.076 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.076 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-03-19T01:25:36.076 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.076 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.076 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.077 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-03-19T01:25:36.077 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-03-19T01:25:36.077 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.077 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.077 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.077 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-19T01:25:36.077 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.077 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.077 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.078 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.078 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.078 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.078 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.078 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.078 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:36.078 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:36.078 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.079 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.079 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.079 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-03-19T01:25:36.079 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.079 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.079 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.079 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.079 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.080 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.080 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.080 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.080 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.080 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.080 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.080 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.080 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.081 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.081 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.081 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.081 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.081 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.081 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.081 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.081 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.081 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.082 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.082 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.082 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.082 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.082 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.082 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.082 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.082 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.083 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.083 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.083 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.083 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.083 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.083 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.083 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.083 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.084 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.084 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.084 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.084 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.084 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.084 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.084 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.084 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.084 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.085 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.085 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.085 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.085 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.085 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.085 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.085 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.085 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.086 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.086 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.086 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.086 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.086 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.086 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.087 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.087 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.087 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.087 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.087 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.087 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.088 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.088 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.088 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.088 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.088 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.088 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.088 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.089 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.089 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.089 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.089 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.089 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.089 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.090 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.090 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-03-19T01:25:36.090 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.090 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.090 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.090 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-03-19T01:25:36.091 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-03-19T01:25:36.091 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.091 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.091 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.091 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.092 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.092 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.092 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.092 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-19T01:25:36.092 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.092 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.093 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.093 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.093 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-03-19T01:25:36.093 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-03-19T01:25:36.093 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.093 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.094 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.094 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.094 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.094 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.094 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.095 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-19T01:25:36.095 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.095 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.095 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.095 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.095 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-03-19T01:25:36.096 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-03-19T01:25:36.096 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.096 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.096 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.096 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.096 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.096 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.097 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.097 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-19T01:25:36.097 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.097 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.097 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.097 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.097 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-03-19T01:25:36.098 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-03-19T01:25:36.098 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.098 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.098 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.098 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.098 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.099 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.099 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.099 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-19T01:25:36.099 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.099 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.099 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.099 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.100 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-03-19T01:25:36.100 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-03-19T01:25:36.100 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.100 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.100 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.100 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.100 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.101 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.101 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.101 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-19T01:25:36.101 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.101 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.101 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.102 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.102 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-03-19T01:25:36.102 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-03-19T01:25:36.102 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.102 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.102 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.103 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.103 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.103 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.103 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.103 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.103 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.103 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.104 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.104 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.104 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-03-19T01:25:36.104 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-03-19T01:25:36.104 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.104 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.104 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.104 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.105 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.105 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.105 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.105 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.105 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.105 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.105 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.105 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.105 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-03-19T01:25:36.106 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-03-19T01:25:36.106 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.106 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.106 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.106 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-19T01:25:36.106 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.106 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.106 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.107 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.107 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.107 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.107 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.107 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-03-19T01:25:36.107 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-03-19T01:25:36.107 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.108 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.108 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.108 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.108 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.108 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.108 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.108 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.108 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.109 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.109 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.109 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.109 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:36.109 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:36.109 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.109 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.109 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.109 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-03-19T01:25:36.110 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-03-19T01:25:36.110 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.110 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.110 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.110 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.110 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.110 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.110 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.110 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-03-19T01:25:36.111 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-03-19T01:25:36.111 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.111 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.111 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.111 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-03-19T01:25:36.111 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.111 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.111 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.112 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.112 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.112 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.112 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.112 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.112 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-03-19T01:25:36.112 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-03-19T01:25:36.112 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.113 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.113 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.113 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.113 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.113 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.113 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.113 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.113 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.114 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.114 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.114 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.114 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-03-19T01:25:36.114 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-03-19T01:25:36.114 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.115 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.115 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.115 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.115 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.115 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.115 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.116 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.116 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.116 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.116 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.116 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.116 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.116 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.116 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.117 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.117 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.117 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.117 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.117 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.117 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.118 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.118 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.118 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.118 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.118 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.118 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.119 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.119 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.119 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.119 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.119 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.120 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.120 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.120 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.120 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.120 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.120 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.121 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.121 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.121 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.121 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.121 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.122 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.122 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.122 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.122 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.122 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.122 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.123 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.123 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.123 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.123 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.123 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.123 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.123 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.123 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.124 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.124 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.124 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.124 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.124 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.124 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.124 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.124 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.125 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.125 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.125 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.125 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.125 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.125 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.126 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.126 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.126 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.126 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.126 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.126 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.127 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.127 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.127 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.127 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-03-19T01:25:36.127 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-03-19T01:25:36.127 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.128 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.128 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.128 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.128 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.128 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.129 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.129 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.129 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.129 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.129 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.130 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.130 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-03-19T01:25:36.130 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-03-19T01:25:36.130 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.130 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.131 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.131 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.131 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.131 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.131 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.131 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.132 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.132 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.132 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.132 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.132 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-03-19T01:25:36.132 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-03-19T01:25:36.132 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.133 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.133 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.133 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-19T01:25:36.133 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.133 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.133 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.133 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.134 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.134 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.134 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.134 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.134 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.134 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.135 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.135 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-03-19T01:25:36.135 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.135 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.135 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.135 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.135 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.135 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.136 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.136 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.136 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.136 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.136 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.136 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.136 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.136 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.136 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.137 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.137 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.137 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.137 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.137 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.137 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.137 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.137 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.138 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.138 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.138 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.138 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.138 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.138 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.138 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.138 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.139 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.139 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.139 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.139 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.139 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.139 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.139 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.139 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.140 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.140 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.140 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.140 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.140 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.140 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.140 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.140 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.140 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.140 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.141 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.141 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.141 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.141 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.141 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.141 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.141 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.142 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.142 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.142 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.142 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.142 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.142 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.142 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.142 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.143 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.143 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.143 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.143 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.143 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.143 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.143 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.143 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.143 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.144 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.144 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.144 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.144 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-03-19T01:25:36.144 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.144 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.144 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.144 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.145 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.145 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.145 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.145 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.145 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.145 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.146 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.146 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.146 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.150 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.150 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.150 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.151 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.151 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.151 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.151 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.151 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.151 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.151 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.151 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.152 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.152 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.152 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.152 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.152 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.152 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.153 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.153 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.153 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.153 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.153 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.153 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.153 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.154 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.154 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.154 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.154 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.154 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.154 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.155 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.155 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.155 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.155 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.155 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.155 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.155 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.156 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.156 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.156 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.156 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.156 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.156 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.156 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.157 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.157 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.157 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.157 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.157 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.157 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.158 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.158 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.158 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.158 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.158 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.158 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.158 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.159 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.159 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.159 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.159 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-03-19T01:25:36.159 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-03-19T01:25:36.159 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.159 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.160 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.160 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-03-19T01:25:36.160 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.160 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.160 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.160 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-19T01:25:36.160 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.161 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.161 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.161 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.161 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-03-19T01:25:36.161 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-03-19T01:25:36.161 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.161 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.161 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.162 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.162 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.162 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.162 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.162 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-03-19T01:25:36.162 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.163 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.163 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.163 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.163 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-03-19T01:25:36.163 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-03-19T01:25:36.163 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.163 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.164 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.164 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:36.164 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.164 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.164 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.164 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-19T01:25:36.164 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.165 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.165 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.165 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.165 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-03-19T01:25:36.165 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-03-19T01:25:36.165 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.165 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.166 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.166 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.166 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.166 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.166 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.166 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-19T01:25:36.166 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.166 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.167 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.167 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.167 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.167 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.167 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.167 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-03-19T01:25:36.167 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.167 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.168 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.168 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-03-19T01:25:36.168 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-03-19T01:25:36.168 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:36.168 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.168 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.168 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-03-19T01:25:36.168 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.169 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.169 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.169 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.169 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.169 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.169 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.169 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.169 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-03-19T01:25:36.170 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-03-19T01:25:36.170 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.170 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.170 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.170 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.170 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.170 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.170 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.171 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.171 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.171 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.171 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.171 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.171 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:36.171 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:36.171 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.171 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.172 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.172 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:36.172 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.172 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.172 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.172 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.172 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.172 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.173 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.173 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.173 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.173 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.173 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.173 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.173 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.173 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.173 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.174 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.175 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.175 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.175 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.175 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.175 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.175 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.175 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.175 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.176 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.176 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.176 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.176 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.176 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.176 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.176 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.176 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.177 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.177 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.177 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.177 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.177 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.177 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.177 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.178 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.178 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.178 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.178 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.178 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.178 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.178 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.178 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.179 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.179 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.179 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.179 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.179 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.179 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.179 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.180 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.180 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.180 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.180 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.180 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.180 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.181 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.181 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.181 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.181 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.181 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.181 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.181 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.182 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.182 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.182 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.182 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-03-19T01:25:36.182 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-03-19T01:25:36.182 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.182 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.182 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.183 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.183 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.183 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.183 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.183 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.183 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.183 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.184 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.184 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.184 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-03-19T01:25:36.184 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-03-19T01:25:36.184 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.184 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.185 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.185 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-03-19T01:25:36.185 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.185 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.185 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.185 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.185 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.186 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.186 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.186 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.186 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-03-19T01:25:36.186 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-03-19T01:25:36.186 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.186 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.186 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.187 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-19T01:25:36.187 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.187 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.187 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.187 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.187 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.187 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.187 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.187 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.188 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.188 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.188 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T01:25:36.188 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.188 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.188 INFO:teuthology.orchestra.run.smithi069.stdout: "modules": [ 2024-03-19T01:25:36.188 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-03-19T01:25:36.188 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard", 2024-03-19T01:25:36.189 INFO:teuthology.orchestra.run.smithi069.stdout: "iostat", 2024-03-19T01:25:36.189 INFO:teuthology.orchestra.run.smithi069.stdout: "nfs", 2024-03-19T01:25:36.189 INFO:teuthology.orchestra.run.smithi069.stdout: "restful", 2024-03-19T01:25:36.189 INFO:teuthology.orchestra.run.smithi069.stdout: "snap_schedule" 2024-03-19T01:25:36.189 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.189 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-03-19T01:25:36.190 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.190 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-03-19T01:25:36.190 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.190 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.190 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.190 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:36.190 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:36.191 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.191 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.191 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.191 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:36.191 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.191 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.192 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.192 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-03-19T01:25:36.192 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.192 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.192 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.192 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.193 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.193 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.193 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.193 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.193 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.193 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.193 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.194 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.194 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.194 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.194 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.194 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.194 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.195 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.195 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.195 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.195 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.195 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.195 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.195 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.196 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.196 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.196 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.196 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.196 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.196 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.196 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.196 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.197 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.197 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.197 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.197 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.197 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.197 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.197 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.197 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.198 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.198 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.198 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.198 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.198 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.198 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.198 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.198 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.199 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.199 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.199 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.199 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.199 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.199 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.199 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.199 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.199 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.200 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.200 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.200 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.200 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.200 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.200 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.200 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.200 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.201 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.201 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.201 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.201 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.201 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.201 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.201 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.201 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.202 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.202 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-03-19T01:25:36.202 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-03-19T01:25:36.202 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.202 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.202 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.202 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.202 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.203 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.203 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.203 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-03-19T01:25:36.203 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.203 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.203 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.203 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.203 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-03-19T01:25:36.204 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-03-19T01:25:36.204 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.204 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.204 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.204 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-03-19T01:25:36.204 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.204 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.204 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.205 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-03-19T01:25:36.205 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.205 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.205 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.205 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.205 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-03-19T01:25:36.205 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-03-19T01:25:36.205 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.206 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.206 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.206 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.206 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.206 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.206 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.206 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-03-19T01:25:36.206 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.207 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.207 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.207 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.207 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-03-19T01:25:36.207 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-03-19T01:25:36.207 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.207 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.208 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.208 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.208 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.208 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.208 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.208 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-03-19T01:25:36.208 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.209 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.209 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.209 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.209 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-03-19T01:25:36.209 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-03-19T01:25:36.209 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.209 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.209 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.210 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-03-19T01:25:36.210 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.210 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.210 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.210 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-03-19T01:25:36.210 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.210 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.211 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.211 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.211 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-03-19T01:25:36.211 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-03-19T01:25:36.211 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.211 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.211 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.212 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.212 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.212 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.212 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.212 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-03-19T01:25:36.212 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.212 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.213 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.213 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-03-19T01:25:36.213 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-03-19T01:25:36.213 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.213 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.213 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.214 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.214 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.214 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.214 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.214 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-19T01:25:36.214 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.214 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.215 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.215 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.215 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-03-19T01:25:36.215 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-03-19T01:25:36.215 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.215 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.216 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.216 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.216 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.216 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.216 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.216 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-03-19T01:25:36.216 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.217 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.217 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.217 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.217 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.217 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-03-19T01:25:36.218 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.218 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.218 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.218 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-03-19T01:25:36.218 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-03-19T01:25:36.218 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.218 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.219 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.219 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.219 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.219 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.219 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.219 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-03-19T01:25:36.220 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.220 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.220 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.220 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.220 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-03-19T01:25:36.220 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-03-19T01:25:36.220 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.221 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.221 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.221 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-03-19T01:25:36.221 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.221 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.221 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.221 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-03-19T01:25:36.222 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T01:25:36.222 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.222 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.222 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.222 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-03-19T01:25:36.222 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-03-19T01:25:36.222 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:36.223 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.223 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.223 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:36.223 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-03-19T01:25:36.223 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-03-19T01:25:36.223 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.223 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-19T01:25:36.224 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T01:25:36.224 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.224 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.224 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.224 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-03-19T01:25:36.224 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-03-19T01:25:36.225 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:36.225 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.225 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.225 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-03-19T01:25:36.225 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:36.225 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-03-19T01:25:36.225 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.226 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-19T01:25:36.226 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.226 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.226 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.226 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-03-19T01:25:36.226 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-03-19T01:25:36.226 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.227 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.227 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.227 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-03-19T01:25:36.227 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.227 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.227 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.227 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-19T01:25:36.227 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-19T01:25:36.227 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.228 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.228 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.228 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-03-19T01:25:36.228 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-03-19T01:25:36.228 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:36.228 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.228 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.228 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-03-19T01:25:36.229 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-03-19T01:25:36.229 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-03-19T01:25:36.229 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.229 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-03-19T01:25:36.229 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-19T01:25:36.229 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.229 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.229 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.229 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-03-19T01:25:36.230 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-03-19T01:25:36.230 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.230 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.230 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.230 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-03-19T01:25:36.230 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.230 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.230 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.230 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-03-19T01:25:36.231 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T01:25:36.231 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.231 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.231 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.231 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-03-19T01:25:36.231 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-03-19T01:25:36.231 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:36.231 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.232 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.232 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:36.232 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-03-19T01:25:36.232 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-03-19T01:25:36.232 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.232 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-19T01:25:36.232 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T01:25:36.232 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.232 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.233 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.233 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.233 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.233 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.233 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.233 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.233 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.234 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.234 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.234 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.234 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.234 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.234 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.234 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.234 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.234 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.235 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.235 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.235 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.235 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.235 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.235 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.235 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.235 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.236 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.236 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.236 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.236 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.236 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.236 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.236 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.237 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.237 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.237 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.237 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.237 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.237 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.237 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.237 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.238 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.238 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.238 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.238 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.238 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.238 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.238 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.239 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.239 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.239 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.239 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.239 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.239 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.239 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.240 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.240 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.240 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.240 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.240 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.240 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.240 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.241 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.241 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.241 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.241 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.241 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.241 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.241 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.242 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.242 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.242 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.242 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-03-19T01:25:36.242 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-03-19T01:25:36.242 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:36.242 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.243 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.243 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:36.243 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.243 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.243 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.243 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-19T01:25:36.243 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.244 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.244 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.244 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.244 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-03-19T01:25:36.244 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-03-19T01:25:36.244 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.244 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.245 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.245 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-03-19T01:25:36.245 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.245 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.245 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.245 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-03-19T01:25:36.246 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-03-19T01:25:36.246 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-03-19T01:25:36.246 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-03-19T01:25:36.246 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-03-19T01:25:36.246 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.246 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-03-19T01:25:36.247 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.247 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.247 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.247 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.247 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-03-19T01:25:36.247 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-03-19T01:25:36.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.248 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.248 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.248 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.248 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.248 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.248 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.249 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-19T01:25:36.249 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.249 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.249 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.249 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.249 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:36.250 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:36.250 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.250 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.250 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.250 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:36.250 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.250 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.251 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.251 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-19T01:25:36.251 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.251 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.251 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.251 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.251 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-03-19T01:25:36.251 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-03-19T01:25:36.252 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.252 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.252 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.252 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:36.252 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:36.252 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.252 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.252 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-19T01:25:36.253 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-19T01:25:36.253 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.253 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.253 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.253 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-03-19T01:25:36.253 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-03-19T01:25:36.253 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:36.253 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.254 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.254 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:36.254 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.254 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.254 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.254 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-19T01:25:36.254 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.255 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.255 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.255 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.255 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.255 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.255 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.255 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-03-19T01:25:36.256 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.256 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.256 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.256 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-03-19T01:25:36.256 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-03-19T01:25:36.256 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:36.257 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.257 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.257 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-03-19T01:25:36.257 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.257 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.257 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.258 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-19T01:25:36.258 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.258 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.258 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.258 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.258 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-03-19T01:25:36.258 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-03-19T01:25:36.259 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.259 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.259 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.259 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-03-19T01:25:36.259 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.259 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.259 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.259 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-19T01:25:36.259 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.260 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.260 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.260 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.260 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-03-19T01:25:36.260 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-03-19T01:25:36.260 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.260 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.260 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.261 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-03-19T01:25:36.261 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.261 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.261 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.261 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-19T01:25:36.261 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.261 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.261 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.261 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.262 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-03-19T01:25:36.262 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-03-19T01:25:36.262 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.262 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.262 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.262 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.262 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.262 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.263 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.263 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-19T01:25:36.263 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-19T01:25:36.263 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.263 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.263 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.263 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-03-19T01:25:36.263 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-03-19T01:25:36.264 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.264 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.264 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.264 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:36.264 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.264 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.264 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.264 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-19T01:25:36.265 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.265 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.265 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.265 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-03-19T01:25:36.265 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-03-19T01:25:36.265 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:36.265 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.265 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.266 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-03-19T01:25:36.266 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.266 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.266 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.266 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-19T01:25:36.266 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.266 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.267 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.267 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-03-19T01:25:36.267 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-03-19T01:25:36.267 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.267 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.267 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.267 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.267 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.268 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.268 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.268 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-03-19T01:25:36.268 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-03-19T01:25:36.268 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-03-19T01:25:36.268 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.268 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-19T01:25:36.268 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.268 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.269 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.269 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.269 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-03-19T01:25:36.269 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-03-19T01:25:36.269 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.269 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.269 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.270 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.270 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.270 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.270 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-19T01:25:36.270 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.270 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.270 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.270 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.271 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-03-19T01:25:36.271 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-03-19T01:25:36.271 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.271 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.271 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.272 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.272 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.272 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.272 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.272 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-19T01:25:36.273 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.273 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.273 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.273 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-03-19T01:25:36.273 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-03-19T01:25:36.273 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.273 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.274 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.274 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.274 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.274 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.275 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.275 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-19T01:25:36.275 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.275 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.275 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.275 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.275 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-03-19T01:25:36.276 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-03-19T01:25:36.276 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.276 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.276 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.276 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-19T01:25:36.276 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.276 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.277 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.277 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:36.277 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.277 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.277 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.277 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-03-19T01:25:36.278 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-03-19T01:25:36.278 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.278 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.278 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.278 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-19T01:25:36.278 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.279 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.279 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.279 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-03-19T01:25:36.279 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.279 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.279 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.280 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-03-19T01:25:36.280 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-03-19T01:25:36.280 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.280 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.280 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.280 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-19T01:25:36.281 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.281 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.281 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.281 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-03-19T01:25:36.281 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.281 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.281 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.282 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-03-19T01:25:36.282 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-03-19T01:25:36.282 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.282 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.283 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-19T01:25:36.283 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.283 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.283 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.283 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:36.283 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.283 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.284 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.284 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.284 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-03-19T01:25:36.284 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-03-19T01:25:36.284 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.284 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.285 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.285 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-19T01:25:36.285 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.285 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.285 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.285 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-03-19T01:25:36.285 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.286 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.286 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.286 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.286 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-03-19T01:25:36.286 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-03-19T01:25:36.286 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.287 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.287 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.287 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-19T01:25:36.287 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.287 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.287 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-03-19T01:25:36.288 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.288 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.288 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.288 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-03-19T01:25:36.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-03-19T01:25:36.289 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.289 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.289 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.289 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-19T01:25:36.289 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.289 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.289 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.289 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-03-19T01:25:36.290 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.290 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.290 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.290 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-03-19T01:25:36.290 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-03-19T01:25:36.290 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.290 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.291 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.291 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-19T01:25:36.291 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.291 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.291 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.291 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-03-19T01:25:36.291 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.291 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.292 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.292 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-03-19T01:25:36.292 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-03-19T01:25:36.292 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.292 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.292 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.292 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-19T01:25:36.293 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.293 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.293 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.293 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-03-19T01:25:36.293 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.293 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.293 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.294 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.294 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-03-19T01:25:36.294 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-03-19T01:25:36.294 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.294 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.294 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.294 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-19T01:25:36.294 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.295 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.295 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.295 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-03-19T01:25:36.295 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.295 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.295 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.295 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.295 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-03-19T01:25:36.296 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-03-19T01:25:36.296 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.296 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.296 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-19T01:25:36.296 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.296 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.296 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.297 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:36.297 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.297 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.297 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.297 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.297 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-03-19T01:25:36.297 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-03-19T01:25:36.297 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.298 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.298 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.298 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-19T01:25:36.298 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.298 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.298 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.298 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-03-19T01:25:36.298 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.299 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.299 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.299 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.299 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-03-19T01:25:36.299 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-03-19T01:25:36.299 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.299 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-19T01:25:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-03-19T01:25:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-03-19T01:25:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-03-19T01:25:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.302 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-19T01:25:36.302 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.302 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.302 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.302 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-03-19T01:25:36.302 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.303 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.303 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.303 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.303 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-03-19T01:25:36.303 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-03-19T01:25:36.303 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.303 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.304 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.304 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-19T01:25:36.304 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.304 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.304 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.304 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-03-19T01:25:36.305 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.305 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.305 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.305 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.305 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-03-19T01:25:36.305 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-03-19T01:25:36.305 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.306 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.306 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.306 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.306 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.306 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.306 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.307 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-03-19T01:25:36.307 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.307 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.307 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.307 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-03-19T01:25:36.307 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-03-19T01:25:36.308 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.308 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.308 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.308 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:36.308 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.308 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.309 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.309 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-19T01:25:36.309 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.309 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.309 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.310 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-03-19T01:25:36.310 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-03-19T01:25:36.310 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.310 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.310 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.310 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-03-19T01:25:36.310 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.311 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.311 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.311 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-19T01:25:36.311 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.311 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.311 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.312 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-03-19T01:25:36.312 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-03-19T01:25:36.312 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.312 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.312 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.312 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-03-19T01:25:36.313 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.313 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.313 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.313 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-19T01:25:36.313 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.313 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.313 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.313 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-03-19T01:25:36.314 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-03-19T01:25:36.314 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.314 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.314 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.314 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-03-19T01:25:36.314 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.314 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.314 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.315 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-03-19T01:25:36.315 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.315 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.315 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.315 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.316 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-03-19T01:25:36.316 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-03-19T01:25:36.316 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.316 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.316 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.316 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.316 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.317 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.317 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.317 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-19T01:25:36.317 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.317 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.317 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.317 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.317 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-03-19T01:25:36.318 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-03-19T01:25:36.318 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.318 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.318 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.318 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:36.318 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.318 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.318 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.319 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-03-19T01:25:36.319 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.319 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.319 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.319 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.319 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-03-19T01:25:36.319 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-03-19T01:25:36.319 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.320 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.320 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:36.320 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.320 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.320 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-03-19T01:25:36.320 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.320 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.320 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.321 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.321 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-03-19T01:25:36.321 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-03-19T01:25:36.321 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.321 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.321 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.321 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.322 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.322 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.322 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-19T01:25:36.322 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.322 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-03-19T01:25:36.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-03-19T01:25:36.323 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.323 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.323 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.323 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.323 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.323 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-19T01:25:36.323 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.324 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.324 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.324 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.324 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.324 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.324 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.324 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.325 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.325 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.325 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.325 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.325 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.325 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.325 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.325 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.325 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.326 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.326 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.326 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.326 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.326 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.326 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.326 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.326 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-03-19T01:25:36.327 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-03-19T01:25:36.327 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.327 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.327 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.327 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.327 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.327 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.327 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.328 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-19T01:25:36.328 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.328 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.328 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.328 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.328 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.328 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.328 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.329 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.329 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.329 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.329 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.329 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.329 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.329 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-19T01:25:36.329 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.330 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.330 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.330 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.330 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.330 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.330 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.330 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.330 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.331 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.331 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.331 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.331 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.331 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.331 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.331 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.332 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.332 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.332 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.332 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.332 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.332 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.332 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.333 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.333 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.333 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.333 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.333 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.333 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.333 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.334 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.334 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.334 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.334 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.334 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.334 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.335 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.335 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.335 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.335 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-19T01:25:36.335 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-19T01:25:36.335 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.335 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.335 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.336 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.336 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.336 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.336 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.336 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-19T01:25:36.336 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.337 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.337 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.337 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-19T01:25:36.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-19T01:25:36.337 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.338 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.338 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.338 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-03-19T01:25:36.338 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.338 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.338 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.338 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-19T01:25:36.339 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.339 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.339 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.339 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-03-19T01:25:36.339 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-03-19T01:25:36.340 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.340 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.340 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.340 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:36.340 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.340 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.340 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.341 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-03-19T01:25:36.341 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.341 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.341 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.341 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.341 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-03-19T01:25:36.341 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-03-19T01:25:36.342 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.342 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.342 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.342 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:36.342 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.342 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.342 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.343 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-19T01:25:36.343 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.343 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.343 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.343 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.343 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-03-19T01:25:36.343 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-03-19T01:25:36.344 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.344 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.344 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.344 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.344 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.344 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.344 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.345 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-03-19T01:25:36.345 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.345 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.345 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.345 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-03-19T01:25:36.345 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-03-19T01:25:36.346 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.346 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.346 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.346 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-03-19T01:25:36.346 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.346 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.347 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.347 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-03-19T01:25:36.347 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-03-19T01:25:36.347 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.347 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-03-19T01:25:36.347 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.347 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.347 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.348 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.348 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-03-19T01:25:36.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-03-19T01:25:36.348 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.348 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.348 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.348 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-03-19T01:25:36.349 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.349 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.349 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.349 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-19T01:25:36.349 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.349 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.349 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.349 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.350 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-03-19T01:25:36.350 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-03-19T01:25:36.350 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.350 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.350 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.350 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-19T01:25:36.350 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.350 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.350 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.351 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-19T01:25:36.351 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.351 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.351 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.351 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.351 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-03-19T01:25:36.351 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-03-19T01:25:36.351 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.352 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.352 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.352 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.352 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.352 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.352 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.352 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-19T01:25:36.352 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.352 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.353 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.353 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.353 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-03-19T01:25:36.353 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-03-19T01:25:36.353 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.353 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.353 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.353 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.353 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.354 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.354 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.354 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-19T01:25:36.354 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.354 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.354 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.354 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.355 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-03-19T01:25:36.355 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-03-19T01:25:36.355 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.355 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.355 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.355 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.355 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.356 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.356 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-19T01:25:36.356 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.356 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.356 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.356 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.356 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-03-19T01:25:36.356 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-03-19T01:25:36.357 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.357 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.357 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.357 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.357 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.357 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.357 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.357 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-19T01:25:36.358 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.358 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.358 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.358 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.358 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-03-19T01:25:36.358 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-03-19T01:25:36.358 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.358 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.359 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.359 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.359 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.359 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.359 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.359 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-19T01:25:36.359 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.360 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.360 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.360 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.360 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-03-19T01:25:36.360 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-03-19T01:25:36.360 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.360 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.360 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.361 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-03-19T01:25:36.361 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.361 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.361 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.361 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-03-19T01:25:36.361 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.361 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.362 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.362 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.362 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-03-19T01:25:36.362 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-03-19T01:25:36.362 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.362 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.362 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.363 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.363 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.363 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.363 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.363 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-19T01:25:36.363 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.363 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.364 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.364 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.364 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-03-19T01:25:36.364 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-03-19T01:25:36.364 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.364 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.365 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.365 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.365 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.365 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.365 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.365 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-19T01:25:36.365 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.366 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.366 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.366 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.366 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-03-19T01:25:36.366 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-03-19T01:25:36.366 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.366 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.367 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.367 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.367 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.367 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.367 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.367 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-19T01:25:36.368 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.368 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.368 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.368 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.368 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-03-19T01:25:36.368 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-03-19T01:25:36.368 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.369 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.369 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.369 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.369 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.369 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.369 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.369 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-03-19T01:25:36.370 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.370 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.370 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.370 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.370 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-03-19T01:25:36.370 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-03-19T01:25:36.371 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.371 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.371 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.371 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.371 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.371 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.371 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.372 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-19T01:25:36.372 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.372 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.372 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.372 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.372 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-03-19T01:25:36.372 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-03-19T01:25:36.373 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.373 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.373 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.373 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.373 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.373 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.373 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.374 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-19T01:25:36.374 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.374 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.374 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.374 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.374 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.375 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.375 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-03-19T01:25:36.375 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.375 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.375 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.375 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.375 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.376 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.376 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.376 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.376 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.376 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.376 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.376 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.377 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.377 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.377 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.377 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.377 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.377 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.377 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.377 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.378 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.378 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.378 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.378 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.378 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.378 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.378 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.378 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.379 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.379 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.379 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.379 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.379 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.379 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.379 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.379 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.379 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.380 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.380 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.380 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.380 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.380 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.380 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.381 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.381 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.381 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.381 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.381 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.381 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.381 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.381 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.381 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.382 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.382 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.382 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.382 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.382 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.382 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.382 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.383 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.383 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.383 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.383 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.383 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.383 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.383 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.383 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.384 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.384 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.384 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.384 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-03-19T01:25:36.384 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-03-19T01:25:36.384 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.384 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.384 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.385 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-03-19T01:25:36.385 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.385 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.385 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.385 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-19T01:25:36.385 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.385 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.385 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.386 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.386 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-03-19T01:25:36.386 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-03-19T01:25:36.386 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.386 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.386 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.386 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-03-19T01:25:36.387 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.387 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.387 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.387 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-19T01:25:36.387 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.387 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.387 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.387 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.387 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.388 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.388 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.388 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-03-19T01:25:36.388 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.388 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.388 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.388 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-19T01:25:36.388 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-19T01:25:36.389 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.389 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.389 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.389 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:36.389 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.389 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.389 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.389 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.389 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.390 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.390 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.390 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.390 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-03-19T01:25:36.390 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-19T01:25:36.390 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.390 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.391 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.391 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.391 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.391 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.391 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.391 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.391 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.391 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.391 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.392 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.392 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-19T01:25:36.392 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-19T01:25:36.392 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.392 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.392 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.392 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.392 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.393 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.393 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.393 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.393 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.393 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.393 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.393 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.393 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-03-19T01:25:36.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-03-19T01:25:36.394 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.394 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.394 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.394 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.394 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.395 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.395 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.395 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.395 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.395 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.395 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.395 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.396 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-19T01:25:36.396 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-19T01:25:36.396 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.396 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.396 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.396 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.396 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.397 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.397 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.397 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.397 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.397 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.397 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.397 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.398 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-03-19T01:25:36.398 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-19T01:25:36.398 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.398 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.398 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.398 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.398 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.399 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.399 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.399 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.399 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.399 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.399 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.400 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.400 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-19T01:25:36.400 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-19T01:25:36.400 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.400 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.400 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.401 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.401 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.401 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.401 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.401 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.401 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.402 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.402 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.402 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.402 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-19T01:25:36.402 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-19T01:25:36.402 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.402 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.403 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.403 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.403 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.403 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.403 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.403 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.403 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.404 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.404 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.404 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.404 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-19T01:25:36.404 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-19T01:25:36.404 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.405 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.405 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.405 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.405 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.405 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.405 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.405 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.406 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.406 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.406 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.406 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.406 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-19T01:25:36.406 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-19T01:25:36.406 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.407 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.407 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.407 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.407 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.407 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.407 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.407 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.408 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.408 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.408 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.408 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-19T01:25:36.408 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-19T01:25:36.409 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.409 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.409 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.409 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.409 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.409 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.409 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.409 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.409 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.410 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.410 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.410 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.410 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-19T01:25:36.410 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-19T01:25:36.410 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.410 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.410 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.411 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.411 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.411 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.411 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.411 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.411 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.411 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.411 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.411 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.412 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-19T01:25:36.412 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-19T01:25:36.412 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.412 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.412 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.412 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.412 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.413 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.413 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.413 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.413 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.413 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.413 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.413 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-19T01:25:36.413 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-19T01:25:36.413 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.414 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.414 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.414 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.414 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.414 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.414 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.414 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.414 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.415 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.415 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.415 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-03-19T01:25:36.415 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-19T01:25:36.415 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.415 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.415 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.416 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-03-19T01:25:36.416 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.416 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.416 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.416 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.416 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.416 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.417 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.417 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.417 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-19T01:25:36.417 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-19T01:25:36.417 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.417 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.418 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.418 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.418 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.418 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.418 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.418 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.419 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.419 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.419 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.419 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-03-19T01:25:36.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-03-19T01:25:36.420 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.420 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.420 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.421 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.421 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.421 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.421 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.421 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.422 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.422 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-03-19T01:25:36.422 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-19T01:25:36.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.422 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.422 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.423 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-03-19T01:25:36.423 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.423 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.423 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.423 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.424 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.424 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.424 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-19T01:25:36.424 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-19T01:25:36.425 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.425 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.425 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.425 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.425 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.425 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.425 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.426 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.426 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.426 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.426 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.426 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.426 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-19T01:25:36.427 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-19T01:25:36.427 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.427 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.427 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.427 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.427 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.428 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.428 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.428 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.428 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.428 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.428 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.428 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.429 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-19T01:25:36.429 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-19T01:25:36.429 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.429 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.429 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.429 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.430 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.430 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.430 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.430 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.430 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.430 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.430 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.431 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-19T01:25:36.431 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-19T01:25:36.431 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.431 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.431 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.432 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.432 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.432 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.432 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.432 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.432 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.432 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.433 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.433 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.433 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-03-19T01:25:36.433 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-19T01:25:36.433 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.433 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.433 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.434 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-03-19T01:25:36.434 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.434 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.434 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.434 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.434 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.434 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.435 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.435 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-03-19T01:25:36.435 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-19T01:25:36.435 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.435 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.435 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.436 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.436 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.436 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.436 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.436 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.436 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.437 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.437 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.437 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-19T01:25:36.437 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-19T01:25:36.437 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.437 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.437 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.437 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.438 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.438 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.438 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.438 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.438 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.438 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.438 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-19T01:25:36.439 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-19T01:25:36.439 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.439 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.439 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.439 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.439 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.439 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.439 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.439 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.440 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.440 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.440 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.440 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.440 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-19T01:25:36.440 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-19T01:25:36.440 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.440 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.440 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.441 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.441 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.441 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.441 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.441 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.441 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.441 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.441 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.442 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-19T01:25:36.442 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-19T01:25:36.442 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.442 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.442 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.442 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.442 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.443 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.443 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.443 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.443 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.443 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.443 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.443 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-19T01:25:36.443 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-19T01:25:36.444 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.444 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.444 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.444 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.444 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.444 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.445 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.445 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.445 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.445 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-19T01:25:36.445 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-19T01:25:36.445 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.445 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.445 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.446 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.446 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.446 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.446 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.446 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.446 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.446 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.446 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.447 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.447 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-19T01:25:36.447 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-19T01:25:36.447 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.447 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.447 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.447 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.448 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.448 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.448 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.448 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.448 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.448 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.448 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.449 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.449 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-19T01:25:36.449 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-19T01:25:36.449 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.449 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.449 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.449 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.450 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.450 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.450 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.450 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.450 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.451 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.451 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-03-19T01:25:36.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-19T01:25:36.451 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.451 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.451 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.451 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.452 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.452 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.452 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.452 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.452 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.452 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.452 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-19T01:25:36.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-19T01:25:36.453 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.453 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.453 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.453 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-19T01:25:36.453 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.453 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.453 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.454 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.454 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.454 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.454 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.454 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.454 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-19T01:25:36.454 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-19T01:25:36.454 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.455 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.455 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.455 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:36.455 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.455 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.455 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.455 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.455 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.456 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.456 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.456 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-19T01:25:36.456 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-19T01:25:36.456 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.457 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.457 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.457 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-03-19T01:25:36.457 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.457 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.457 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.458 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.458 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.458 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.458 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-19T01:25:36.458 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-19T01:25:36.459 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.459 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.459 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.459 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-03-19T01:25:36.459 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.459 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.459 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.460 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.460 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.460 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.460 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.460 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.460 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-03-19T01:25:36.460 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-19T01:25:36.461 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.461 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.461 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.461 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.461 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.461 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.461 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.462 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.462 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.462 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.462 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.462 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.462 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-19T01:25:36.462 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-19T01:25:36.463 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.463 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.463 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.463 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-03-19T01:25:36.463 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.463 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.463 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.464 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.464 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.464 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.464 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-03-19T01:25:36.465 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-19T01:25:36.465 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.465 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.465 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.465 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.465 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.465 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.466 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.466 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.466 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.466 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.466 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.466 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-03-19T01:25:36.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-19T01:25:36.467 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.467 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.467 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.467 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.467 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.467 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.468 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.468 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.468 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.468 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.468 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.468 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.468 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-03-19T01:25:36.469 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-19T01:25:36.469 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.469 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.469 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.469 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.469 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.469 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.469 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.470 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.470 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.470 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.470 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.470 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.470 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-19T01:25:36.470 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-19T01:25:36.470 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.470 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.471 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.471 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:36.471 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.471 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.471 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.471 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.471 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.471 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.472 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.472 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.472 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-19T01:25:36.472 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-19T01:25:36.472 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.472 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.472 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.472 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:36.472 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.473 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.473 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.473 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.473 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.473 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.473 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.473 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-19T01:25:36.473 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-19T01:25:36.474 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.474 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.474 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.474 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:36.474 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.474 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.474 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.474 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.475 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.475 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.475 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.475 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.475 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-03-19T01:25:36.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-03-19T01:25:36.475 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.475 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.475 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.476 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.476 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.476 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.476 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.476 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.477 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.477 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.477 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-03-19T01:25:36.477 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-03-19T01:25:36.477 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.477 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.477 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.478 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.478 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.478 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.478 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.478 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.478 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.478 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.479 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-03-19T01:25:36.479 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-03-19T01:25:36.479 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.479 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.479 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.479 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.479 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.479 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.480 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.480 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-03-19T01:25:36.480 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.480 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.480 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.481 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-03-19T01:25:36.481 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-03-19T01:25:36.481 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.481 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.481 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.481 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-03-19T01:25:36.481 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.481 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.482 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.482 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.482 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.482 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.482 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.482 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.482 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-03-19T01:25:36.483 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-03-19T01:25:36.483 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.483 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.483 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.483 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.483 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.483 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.484 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.484 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.484 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.484 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.485 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.485 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.485 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.485 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.485 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.485 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.486 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.486 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.486 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.486 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.486 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.486 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.487 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.487 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.487 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.487 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.487 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.487 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.487 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.487 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.488 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.488 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.488 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.488 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.488 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.488 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.488 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.489 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.489 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.489 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.489 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.489 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.489 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.489 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.489 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.490 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.490 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.490 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.490 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.490 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.490 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.490 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.490 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.491 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.491 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.491 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.491 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.491 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.491 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.491 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.491 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.492 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.492 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.492 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.492 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.492 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.492 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.492 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.492 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.493 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.493 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.493 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.493 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.493 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.493 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.493 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.493 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.494 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.494 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-03-19T01:25:36.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-03-19T01:25:36.494 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.494 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.494 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.494 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.494 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.495 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.495 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.495 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-03-19T01:25:36.495 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.495 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.495 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.495 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.495 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-03-19T01:25:36.496 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-03-19T01:25:36.496 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.496 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.496 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.496 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.496 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.496 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.496 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.497 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.497 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.497 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.497 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.497 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-03-19T01:25:36.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-03-19T01:25:36.497 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.498 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.498 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.498 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-03-19T01:25:36.498 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.498 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.498 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.498 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.498 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.499 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.499 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.499 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-03-19T01:25:36.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-03-19T01:25:36.499 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.499 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.499 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.500 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-03-19T01:25:36.500 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.500 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.500 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.500 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.500 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.501 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.501 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.501 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-03-19T01:25:36.501 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-03-19T01:25:36.501 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.501 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.501 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.501 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.502 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.502 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.502 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.502 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.502 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.502 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.502 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.503 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.503 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-03-19T01:25:36.503 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-03-19T01:25:36.503 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.503 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.503 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.503 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-03-19T01:25:36.503 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.503 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.504 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.504 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.504 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.504 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.504 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.504 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-03-19T01:25:36.504 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-03-19T01:25:36.505 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.505 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.505 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-03-19T01:25:36.505 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.505 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.505 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.505 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.506 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-03-19T01:25:36.506 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.506 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.506 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.506 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.506 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.507 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-03-19T01:25:36.507 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-03-19T01:25:36.507 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.507 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.507 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.507 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-03-19T01:25:36.507 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-03-19T01:25:36.507 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-03-19T01:25:36.507 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.508 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.509 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-03-19T01:25:36.509 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-03-19T01:25:36.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.509 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.509 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.509 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.510 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.510 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.510 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.510 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.510 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.510 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.510 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.510 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-03-19T01:25:36.511 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.511 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.511 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.511 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-03-19T01:25:36.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-03-19T01:25:36.511 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.512 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.512 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.512 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.512 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.512 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.512 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.512 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-03-19T01:25:36.512 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.513 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.513 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.513 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.513 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.513 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.513 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.513 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.513 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.514 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.514 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.514 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.514 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.514 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.514 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.514 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.514 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.515 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.515 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.515 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.515 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.515 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.515 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.515 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.515 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.516 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.516 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.516 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.516 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.516 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.516 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.516 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.517 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.517 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.517 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.517 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.517 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.517 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.517 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.518 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.518 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.518 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.518 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.518 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.518 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.519 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.519 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.519 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.519 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.519 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.519 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.519 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.520 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.520 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.520 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.520 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.520 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.520 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.520 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.520 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.521 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.521 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.521 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.521 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.521 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.521 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.522 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.522 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-03-19T01:25:36.522 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-03-19T01:25:36.522 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.522 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.522 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.522 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-03-19T01:25:36.523 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.523 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.523 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.523 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-19T01:25:36.523 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.523 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.523 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.523 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.524 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-03-19T01:25:36.524 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-03-19T01:25:36.524 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.524 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.524 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.524 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-03-19T01:25:36.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.524 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.525 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.525 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-19T01:25:36.525 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.525 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.525 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-03-19T01:25:36.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-03-19T01:25:36.526 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.526 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.526 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.526 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-03-19T01:25:36.526 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.526 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.526 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.527 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-03-19T01:25:36.527 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.527 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.527 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.527 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.527 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-03-19T01:25:36.527 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-03-19T01:25:36.527 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.528 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.528 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.528 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-03-19T01:25:36.528 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.528 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.528 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.528 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-19T01:25:36.528 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.529 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.529 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.529 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.529 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-03-19T01:25:36.529 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-03-19T01:25:36.529 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.529 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.529 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.530 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.530 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.530 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.530 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-19T01:25:36.530 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.530 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.530 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.531 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.531 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:36.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:36.531 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.531 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.531 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.531 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:36.531 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.532 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.532 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.532 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-03-19T01:25:36.532 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.532 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.532 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.532 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.532 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-03-19T01:25:36.533 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-03-19T01:25:36.533 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.533 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.533 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.533 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-03-19T01:25:36.533 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.533 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.533 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.534 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-19T01:25:36.534 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.534 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.534 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.534 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.534 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.534 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.534 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-03-19T01:25:36.535 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.535 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.535 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.535 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.535 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.535 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.536 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.536 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.536 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.536 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.536 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.536 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.536 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.536 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.537 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.537 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.537 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.537 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.537 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.537 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.537 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.537 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.538 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.538 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.538 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.538 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.538 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.538 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.539 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.539 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.539 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.539 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.539 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.539 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.539 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.539 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.540 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.540 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.540 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.540 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.540 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.540 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.540 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.540 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.541 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.541 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.541 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.541 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.541 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.541 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.541 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.541 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.541 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.542 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.542 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.542 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.542 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.542 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.542 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.542 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.542 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.543 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.543 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.543 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.543 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.543 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.543 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.543 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.543 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-03-19T01:25:36.544 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-03-19T01:25:36.544 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.544 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.544 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.544 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-03-19T01:25:36.544 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.544 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.544 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.545 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.545 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.545 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.545 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.545 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-03-19T01:25:36.545 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-03-19T01:25:36.545 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.545 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.546 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.546 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-03-19T01:25:36.546 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.546 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.546 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.547 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.547 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.547 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.547 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:36.547 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:36.547 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.547 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.547 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.548 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-03-19T01:25:36.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.548 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.548 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.548 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.548 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.548 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.548 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.549 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.549 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.549 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.549 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-03-19T01:25:36.549 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-03-19T01:25:36.549 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-03-19T01:25:36.549 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.550 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-03-19T01:25:36.550 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-03-19T01:25:36.550 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.550 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.550 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.550 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-03-19T01:25:36.550 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.550 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.551 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.551 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-19T01:25:36.551 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.551 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.551 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.551 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.551 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-03-19T01:25:36.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-03-19T01:25:36.552 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.552 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.552 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.552 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-03-19T01:25:36.552 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.552 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.552 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.552 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-19T01:25:36.553 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.553 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.553 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.553 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-03-19T01:25:36.553 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-03-19T01:25:36.553 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.553 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.554 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.554 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.554 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.554 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.554 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.554 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-03-19T01:25:36.554 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.554 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.555 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.555 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:36.555 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:36.558 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.558 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.558 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-03-19T01:25:36.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-03-19T01:25:36.558 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.558 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.559 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-19T01:25:36.559 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.559 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.559 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.559 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.559 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.559 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.559 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.560 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.560 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.560 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.560 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.560 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.560 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.560 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.560 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.560 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.561 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.561 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.561 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.561 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.561 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.561 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.561 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.562 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.562 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.562 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.562 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.562 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.562 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.562 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.562 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.564 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.564 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.564 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.564 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.564 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.564 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.564 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.564 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.565 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.565 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.565 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.565 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.565 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.565 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.565 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.565 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.566 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.566 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.566 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.566 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.566 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.566 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.566 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.567 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.567 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.567 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.567 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.567 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.567 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.567 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.567 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.568 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.568 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-03-19T01:25:36.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-03-19T01:25:36.568 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.568 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.568 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.568 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.568 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-03-19T01:25:36.569 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.569 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.569 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.569 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.569 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-03-19T01:25:36.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-03-19T01:25:36.570 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.570 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.570 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.570 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-03-19T01:25:36.570 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.570 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.570 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.571 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-03-19T01:25:36.571 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.571 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.571 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.571 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-03-19T01:25:36.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-03-19T01:25:36.571 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.571 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.572 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-03-19T01:25:36.572 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.572 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.572 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.572 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T01:25:36.572 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.572 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.572 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.573 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-03-19T01:25:36.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-03-19T01:25:36.573 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.573 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.573 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.573 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:36.573 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:36.574 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-03-19T01:25:36.574 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.574 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-19T01:25:36.574 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.574 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.574 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.574 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.574 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-03-19T01:25:36.574 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-03-19T01:25:36.575 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.575 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.575 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.575 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.575 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.575 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.575 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.575 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-03-19T01:25:36.576 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.576 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.576 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.576 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-03-19T01:25:36.576 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-03-19T01:25:36.576 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.576 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.577 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.577 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-03-19T01:25:36.577 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.577 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.577 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.577 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T01:25:36.577 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.577 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.577 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.578 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.578 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.578 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.578 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.578 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-03-19T01:25:36.578 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.578 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.578 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.579 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.579 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.579 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.579 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.579 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.579 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.579 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.579 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.579 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.580 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.580 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.580 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.580 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.580 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.580 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.580 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.580 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.580 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.581 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.581 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.581 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.581 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.581 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.581 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.581 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.581 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.582 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.582 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.582 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.582 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.582 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.582 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.582 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.582 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.582 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.583 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.583 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.583 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.583 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.583 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.583 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.583 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.583 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.584 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.584 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.584 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.584 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.584 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.584 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.584 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.584 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.584 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.585 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.585 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.585 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.585 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.585 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.585 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.585 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.585 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.585 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.586 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.586 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.586 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.586 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.586 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.586 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.586 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.586 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.587 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.587 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.587 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.587 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.587 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-03-19T01:25:36.587 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.587 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.587 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.587 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.588 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.588 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.588 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.588 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.588 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.588 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.588 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.589 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.589 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.589 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.589 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.589 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.589 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.589 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.589 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.589 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.590 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.590 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.590 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.590 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.590 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.590 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.590 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.591 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.591 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.591 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.591 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.591 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.591 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.591 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.591 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.591 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.592 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.592 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.592 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.592 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.592 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.592 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.592 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.592 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.593 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.593 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.593 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.593 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.593 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.593 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.593 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.593 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.593 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.594 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.594 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.594 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.594 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.594 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.594 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.594 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.594 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.595 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.595 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.595 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.595 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.595 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.595 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.595 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.595 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.596 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.596 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.596 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.596 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.596 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.596 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.596 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-03-19T01:25:36.596 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.596 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.597 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.597 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-03-19T01:25:36.597 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-03-19T01:25:36.597 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.597 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.597 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.597 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-03-19T01:25:36.597 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.598 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.598 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.598 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-19T01:25:36.598 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.598 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.598 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.598 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.598 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-03-19T01:25:36.599 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-03-19T01:25:36.599 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.599 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.599 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.599 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:36.599 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-03-19T01:25:36.599 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.599 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.600 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-19T01:25:36.600 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.600 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.600 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.600 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.600 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.600 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.600 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.601 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.601 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.601 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.601 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.601 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.601 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.601 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.601 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.601 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.602 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.602 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.602 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.602 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.602 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.602 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.602 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.603 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.603 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.603 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.603 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.603 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.603 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.603 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.603 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.604 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.604 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.604 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.604 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.604 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.604 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.604 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.604 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.604 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.605 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.605 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.605 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.605 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.605 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.605 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.605 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.605 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.606 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.606 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.606 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.606 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.606 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.606 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.606 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.606 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.607 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.607 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.607 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.607 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.607 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.607 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.607 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.607 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.608 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.608 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.608 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.608 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.608 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.608 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.609 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.609 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.609 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.609 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.609 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.609 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.609 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-03-19T01:25:36.609 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.610 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.610 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.610 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-03-19T01:25:36.610 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-03-19T01:25:36.610 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.610 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-03-19T01:25:36.611 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.611 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.611 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.611 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-03-19T01:25:36.611 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-19T01:25:36.611 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.611 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.611 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.611 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.612 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.612 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.612 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.612 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.612 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.612 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.612 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.612 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.613 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.613 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.613 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.613 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.613 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.613 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.613 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.613 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.613 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.614 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.614 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.614 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.614 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.614 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.614 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.614 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.614 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.615 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.615 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.615 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.615 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.615 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.615 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.615 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.615 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.615 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.616 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.616 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.616 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.616 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.616 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.616 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.616 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.616 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.616 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.617 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.617 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.617 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.617 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.617 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.617 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.617 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.617 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.618 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.618 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.618 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.618 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.618 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.618 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.618 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.618 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.619 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.619 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.619 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.619 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.619 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.619 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.619 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.619 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.620 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.620 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.620 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-03-19T01:25:36.620 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-03-19T01:25:36.620 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.620 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.620 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.621 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.621 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.621 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.621 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.621 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-03-19T01:25:36.621 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-19T01:25:36.621 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.621 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.621 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.622 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-03-19T01:25:36.622 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-03-19T01:25:36.622 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.622 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.622 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.622 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-03-19T01:25:36.622 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.622 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.623 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.623 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-03-19T01:25:36.623 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.623 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.623 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.623 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.623 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-03-19T01:25:36.623 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-03-19T01:25:36.623 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.624 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.624 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-03-19T01:25:36.624 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.624 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.624 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.624 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-03-19T01:25:36.624 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.624 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.625 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.625 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.625 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-03-19T01:25:36.625 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-03-19T01:25:36.625 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.625 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.625 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.625 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.626 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.626 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.626 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.626 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-03-19T01:25:36.626 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.626 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.626 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.626 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.626 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-03-19T01:25:36.627 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-03-19T01:25:36.627 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.627 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.627 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.627 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-03-19T01:25:36.627 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.627 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.628 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.628 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-19T01:25:36.628 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-19T01:25:36.628 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.628 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.628 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.628 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.628 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.628 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-03-19T01:25:36.629 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.629 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.629 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.629 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.629 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.630 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.630 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.630 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.630 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.630 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.630 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.630 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.630 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.631 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.631 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.631 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.631 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.631 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.631 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.631 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.631 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.631 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.632 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.632 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.632 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.632 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.632 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.632 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.632 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.632 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.633 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.633 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.633 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.633 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.633 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.633 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.633 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.633 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.634 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.634 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.634 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.634 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.634 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.634 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.635 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.635 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.635 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.635 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.635 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.635 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.635 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.635 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.635 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.636 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.636 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.636 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.636 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.636 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.636 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.636 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.636 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.637 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.637 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.637 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.637 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.637 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.637 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.637 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.638 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.638 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.638 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.638 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.638 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.638 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-03-19T01:25:36.638 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.638 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.638 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.639 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.639 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.639 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.639 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.639 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.639 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.639 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.639 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.640 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.640 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.640 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.640 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.640 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.640 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.640 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.640 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.640 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.641 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.641 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.641 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.641 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.641 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.641 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.641 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.641 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.642 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.642 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.642 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.642 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.642 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.642 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.642 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.643 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.643 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.643 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.643 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.643 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.643 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.644 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.644 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.644 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.644 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.644 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.644 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.644 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.644 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.644 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.645 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.645 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.645 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.645 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.645 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.646 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.646 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.646 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.646 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.646 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.646 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.646 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.646 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.646 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.647 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.647 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.647 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.647 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.647 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.648 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.648 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.648 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.648 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.648 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.648 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-03-19T01:25:36.649 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.649 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.649 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.649 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.649 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.649 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.650 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.650 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.650 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.650 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.650 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.650 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.650 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.651 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.651 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.651 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.651 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.651 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.651 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.651 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.652 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.652 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.652 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.652 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.652 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.652 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.652 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.653 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.653 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.653 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.653 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.653 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.653 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.653 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.654 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.654 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.654 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.654 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.654 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.654 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.655 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.655 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.655 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.655 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.655 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.655 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.655 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.656 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.656 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.656 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.656 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.656 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.656 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.657 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.657 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.657 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.657 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.657 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.657 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.657 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.657 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.658 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.658 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.658 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.658 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.658 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.658 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.658 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.658 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.659 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.659 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.659 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.659 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.659 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.659 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.659 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.659 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-03-19T01:25:36.660 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.660 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.660 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.660 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-03-19T01:25:36.660 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-03-19T01:25:36.660 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.660 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.660 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.661 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.661 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.661 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.661 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.661 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-19T01:25:36.661 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.661 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.661 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.662 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.662 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.662 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.662 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.662 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.662 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.662 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.662 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.662 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.663 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.663 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.663 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.663 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.663 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.663 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.663 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.663 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.664 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.664 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.664 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.664 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.664 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.664 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.664 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.665 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.665 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.665 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.665 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.665 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.665 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.665 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.665 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.666 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.666 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.666 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.666 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.666 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.666 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.666 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.666 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.667 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.667 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.667 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.667 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.667 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.667 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.667 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.667 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.668 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.668 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.668 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.668 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.668 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.668 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.668 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.668 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.668 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.669 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.669 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.669 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.669 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.669 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.669 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.669 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.669 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.669 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.670 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.670 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.670 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.670 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-03-19T01:25:36.670 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-03-19T01:25:36.670 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.670 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.671 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.671 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.671 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.671 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.671 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.671 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-03-19T01:25:36.671 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-03-19T01:25:36.671 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-03-19T01:25:36.672 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.672 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-03-19T01:25:36.672 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.672 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.672 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.672 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.672 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.672 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.673 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.673 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-03-19T01:25:36.673 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.673 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.673 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.673 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.673 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.673 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.674 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.674 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.674 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.674 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.674 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.674 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.674 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.674 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.675 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.675 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.675 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.675 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.675 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.675 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.675 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.675 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.675 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.676 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.676 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.676 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.676 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.676 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.676 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.676 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.677 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.677 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.677 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.677 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.677 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.677 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.677 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.677 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.678 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.678 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.678 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.678 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.678 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.678 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.678 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.678 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.679 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.679 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.679 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.679 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.679 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.679 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.679 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.679 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.679 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.680 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.680 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.680 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.680 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.680 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.680 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.680 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.680 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.681 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.681 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.681 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.681 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.681 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.681 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.681 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.682 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.682 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.682 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.682 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.682 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.682 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-03-19T01:25:36.682 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.682 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.683 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.683 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.683 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.683 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.683 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.683 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.683 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.683 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.684 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.684 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.684 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.684 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.684 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.684 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.685 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.685 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.685 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.685 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.685 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.685 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.685 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.685 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.686 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.686 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.686 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.686 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.686 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.686 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.687 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.687 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.687 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.687 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.687 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.687 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.687 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.687 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.688 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.688 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.688 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.688 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.688 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.688 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.688 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.689 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.689 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.689 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.689 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.689 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.689 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.689 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.690 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.690 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.690 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.690 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.690 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.690 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.691 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.691 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.691 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.691 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.691 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.691 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.691 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.692 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.692 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.692 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.692 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.692 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.692 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.692 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.692 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.693 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.693 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.693 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.693 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-03-19T01:25:36.693 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.693 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.693 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.693 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.693 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.694 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.694 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.694 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.694 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.694 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.694 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.694 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.694 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.695 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.695 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.695 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.695 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.695 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.695 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.695 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.695 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.696 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.696 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.696 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.696 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.696 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.696 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.696 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.696 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.697 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.697 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.697 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.697 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.697 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.697 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.697 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.697 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.697 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.698 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.698 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.698 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.698 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.698 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.698 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.698 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.698 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.699 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.699 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.699 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.699 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.699 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.699 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.699 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.699 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.699 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.700 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.700 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.700 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.700 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.700 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.700 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.700 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.700 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.701 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.701 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.701 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.701 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.701 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.701 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.701 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.701 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.701 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.702 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.702 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:36.702 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:36.702 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.702 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.702 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.702 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:36.702 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.703 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.703 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.703 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.703 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.703 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.703 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.703 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.703 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-03-19T01:25:36.703 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-03-19T01:25:36.704 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:36.704 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.704 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.704 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-03-19T01:25:36.704 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-03-19T01:25:36.704 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.704 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.704 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-03-19T01:25:36.705 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-19T01:25:36.705 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.705 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.705 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.705 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.705 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.705 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.705 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-03-19T01:25:36.705 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.706 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.706 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.706 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-03-19T01:25:36.706 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-03-19T01:25:36.706 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.706 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.706 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.706 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.707 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.707 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.707 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.707 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-19T01:25:36.707 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.707 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.707 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.707 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.707 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-03-19T01:25:36.708 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-03-19T01:25:36.708 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.708 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.708 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.708 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.708 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.708 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.708 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.709 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.709 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.709 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.709 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.709 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.709 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.709 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.709 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.709 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.710 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.710 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.710 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.710 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.710 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.710 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.710 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.711 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.711 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.711 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.711 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.711 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.711 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.711 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.711 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.712 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.712 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.712 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.712 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.712 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.712 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.712 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.712 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.713 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.713 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.713 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.713 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.713 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.713 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.713 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.713 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.714 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.714 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.714 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.714 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.714 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.714 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.714 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.714 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.714 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.715 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.715 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.715 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.715 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.715 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.715 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.715 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.715 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.716 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.716 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.716 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.716 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.716 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.716 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.716 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.716 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.717 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.717 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.717 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.717 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.717 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.717 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.717 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.717 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.718 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.718 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.718 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-03-19T01:25:36.718 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-03-19T01:25:36.718 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.718 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.718 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.718 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-03-19T01:25:36.719 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.719 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.719 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.719 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-03-19T01:25:36.719 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.719 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.719 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.720 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.720 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-03-19T01:25:36.720 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-03-19T01:25:36.720 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.720 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.720 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-03-19T01:25:36.720 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.721 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.721 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.721 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-03-19T01:25:36.721 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.721 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.721 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.721 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.721 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.722 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.722 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.722 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-03-19T01:25:36.722 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.722 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.722 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.722 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-03-19T01:25:36.722 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-03-19T01:25:36.723 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.723 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.723 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.723 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.723 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.723 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.723 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.723 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.724 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.724 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.724 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.724 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.724 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-03-19T01:25:36.724 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-03-19T01:25:36.724 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.724 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.724 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.725 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.725 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.725 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.725 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.725 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-19T01:25:36.725 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-19T01:25:36.725 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.725 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.726 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.726 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.726 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.726 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.726 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.726 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.726 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.726 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.727 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.727 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.727 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.727 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.727 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.727 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.727 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.727 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.727 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.728 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.728 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.728 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.728 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.728 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.728 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.728 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.728 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.729 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.729 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.729 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.729 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.729 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.729 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.729 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.729 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.730 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.730 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.730 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.730 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.730 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.730 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.730 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.730 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.730 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.731 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.731 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.731 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.731 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.731 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.731 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.731 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.731 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.732 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.732 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.732 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.732 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.732 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.732 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.732 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.732 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.733 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.733 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.733 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.733 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.733 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.733 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.733 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.733 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.734 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.734 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.734 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.734 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.734 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.734 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-03-19T01:25:36.734 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-03-19T01:25:36.734 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.734 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.735 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.735 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.735 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.735 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.735 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.735 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.735 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.736 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.736 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.736 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-19T01:25:36.736 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-19T01:25:36.736 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.736 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.736 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.736 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-03-19T01:25:36.737 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.737 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.737 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.737 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.737 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.737 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.737 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.737 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.738 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-03-19T01:25:36.738 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-03-19T01:25:36.738 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:36.738 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.738 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.738 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-03-19T01:25:36.738 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.738 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.739 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.739 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.739 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.739 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.739 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.739 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.739 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-03-19T01:25:36.739 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-03-19T01:25:36.740 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.740 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.740 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.740 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-03-19T01:25:36.740 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.740 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.740 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.740 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-19T01:25:36.740 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.741 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.741 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.741 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.741 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-03-19T01:25:36.741 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-03-19T01:25:36.741 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.741 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.741 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.742 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-03-19T01:25:36.742 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.742 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.742 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.742 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-19T01:25:36.742 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.742 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.742 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.743 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.743 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-03-19T01:25:36.743 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-03-19T01:25:36.743 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.743 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.743 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.743 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-03-19T01:25:36.743 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.744 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.744 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.744 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.744 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.744 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.744 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.744 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.744 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-03-19T01:25:36.745 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-03-19T01:25:36.745 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.745 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.745 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.745 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-03-19T01:25:36.745 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.745 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.745 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.745 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-03-19T01:25:36.746 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-03-19T01:25:36.746 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.746 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.746 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.746 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.746 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.746 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.746 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-03-19T01:25:36.747 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-03-19T01:25:36.747 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.747 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.747 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-03-19T01:25:36.747 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-03-19T01:25:36.747 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-03-19T01:25:36.747 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.748 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.748 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.748 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.748 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.748 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.748 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.748 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.748 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.748 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-03-19T01:25:36.749 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.749 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.749 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.749 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.749 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.750 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.750 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.750 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.750 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.750 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.750 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.750 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.751 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.751 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.751 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.751 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.751 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.751 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.751 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.751 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.752 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.752 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.752 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.752 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.752 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.752 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.752 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.753 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.753 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.753 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.753 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.753 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.753 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.753 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.753 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.754 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.754 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.754 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.754 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.754 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.754 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.754 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.755 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.755 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.755 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.755 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.755 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.755 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.755 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.755 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.755 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.756 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.756 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.756 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.756 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.756 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.756 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.756 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.756 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.757 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.757 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.757 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.757 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.757 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.757 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.757 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.757 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.758 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.758 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.758 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.758 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-03-19T01:25:36.758 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-03-19T01:25:36.758 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.758 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.758 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.759 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-03-19T01:25:36.759 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.759 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.759 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.759 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.759 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.759 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.759 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.760 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.760 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-03-19T01:25:36.760 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-03-19T01:25:36.760 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.760 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.760 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.762 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.762 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.762 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.762 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.762 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.762 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.762 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.763 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.763 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-03-19T01:25:36.763 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-03-19T01:25:36.763 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.763 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.763 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.763 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.764 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.764 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.764 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.764 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.764 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.764 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.764 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.764 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.765 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.765 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.765 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-03-19T01:25:36.765 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.765 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.765 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.765 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-03-19T01:25:36.765 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-03-19T01:25:36.766 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.766 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.766 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.766 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.766 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.766 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.766 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.766 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.767 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.767 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.767 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.767 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.767 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-03-19T01:25:36.767 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-03-19T01:25:36.767 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.767 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.767 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.768 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.768 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.768 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.768 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.768 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.768 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.768 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.768 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.768 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.769 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.769 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.769 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.769 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.769 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.769 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.769 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.770 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.770 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.770 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.770 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.770 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.771 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.771 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.771 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.771 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.771 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.771 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.772 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.772 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.772 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.772 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.772 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.772 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.773 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.773 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.773 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.773 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.773 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.773 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.774 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.774 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.774 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.774 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.774 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.775 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.775 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.775 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.775 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.775 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.775 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.776 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.776 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.776 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.776 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.776 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.776 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.777 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.777 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.777 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.777 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.777 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.777 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.778 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.778 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.778 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.778 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.778 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.778 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.779 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.779 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.779 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.779 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.779 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.779 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.779 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.779 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.780 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.780 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.780 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.780 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-03-19T01:25:36.780 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-03-19T01:25:36.780 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.780 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.780 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.781 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.781 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.781 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.781 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.781 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.781 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.781 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.781 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.781 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.782 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-03-19T01:25:36.782 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-03-19T01:25:36.782 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.782 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.782 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.782 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.782 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.783 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.783 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.783 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.783 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.783 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.783 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.783 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.783 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.783 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.784 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.784 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-03-19T01:25:36.784 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.784 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.784 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.784 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.784 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.785 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.785 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.785 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.785 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.785 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.785 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.785 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.785 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.785 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.786 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.786 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.786 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.786 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.786 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.786 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.786 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.786 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.787 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.787 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.787 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.787 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.787 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.787 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.787 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.787 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.787 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.787 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.788 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.788 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.788 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.788 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.788 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.788 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.788 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.788 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.789 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.789 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.789 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.789 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.789 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.789 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.789 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.790 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.790 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.790 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.790 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.790 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.790 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.790 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.790 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.791 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.791 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.791 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.791 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.791 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.791 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.791 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.792 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.792 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.792 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.792 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.792 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.792 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.792 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.792 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.792 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.793 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.793 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.793 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.793 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.793 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-03-19T01:25:36.793 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.793 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.793 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.793 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.794 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.794 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.794 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.794 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.794 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.794 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.794 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.794 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.795 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.795 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.795 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.795 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.795 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.795 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.795 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.796 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.796 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.796 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.796 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.796 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.796 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.796 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.796 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.797 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.797 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.797 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.797 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.797 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.797 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.797 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.798 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.798 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.798 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.798 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.798 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.798 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.798 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.798 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.798 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.799 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.799 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.799 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.799 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.799 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.799 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.799 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.799 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.800 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.800 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.800 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.800 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.800 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.800 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.800 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.800 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.801 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.801 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.801 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.801 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.801 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.801 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.801 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.802 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.802 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.802 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.802 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.802 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-03-19T01:25:36.802 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-03-19T01:25:36.802 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.802 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.802 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.803 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-03-19T01:25:36.803 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.803 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.803 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.803 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-19T01:25:36.803 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.803 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.804 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.804 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.804 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.804 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-03-19T01:25:36.804 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.804 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.805 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.805 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.805 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.805 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.805 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.805 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.805 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.805 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.806 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.806 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.806 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.806 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.806 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.806 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.806 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.806 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.807 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.807 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.807 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.807 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.807 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.807 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.807 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.807 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.807 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.808 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.808 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.808 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.808 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.808 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.808 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.808 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.808 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.808 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.809 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.809 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.809 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.809 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.809 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.809 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.809 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.809 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.810 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.810 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.810 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.810 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.810 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.810 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.810 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.810 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.811 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.811 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.811 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.811 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.811 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.811 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.811 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.811 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.811 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.812 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.812 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.812 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.812 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.812 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.812 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.812 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.812 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.813 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.813 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.813 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.813 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-03-19T01:25:36.813 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-03-19T01:25:36.813 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.813 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.813 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.813 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.814 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.814 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.814 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.814 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.814 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.814 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.814 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.815 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-03-19T01:25:36.815 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-03-19T01:25:36.815 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.815 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.815 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.815 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-03-19T01:25:36.815 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.816 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.816 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.816 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.816 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.816 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.816 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.816 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-03-19T01:25:36.816 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-03-19T01:25:36.817 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.817 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.817 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.817 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.817 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.817 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.817 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.817 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.818 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.818 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.818 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.818 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-03-19T01:25:36.818 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-03-19T01:25:36.818 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.818 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.818 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.819 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.819 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.819 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.819 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.819 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.819 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.819 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.819 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.820 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.820 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-03-19T01:25:36.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-03-19T01:25:36.820 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-03-19T01:25:36.820 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.820 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.820 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.820 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.820 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.821 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.821 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.821 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.821 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.821 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.821 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.821 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-03-19T01:25:36.821 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-03-19T01:25:36.821 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.822 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.822 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.822 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.822 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.822 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.822 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.822 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.822 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.823 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.823 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.823 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.823 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-03-19T01:25:36.823 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-03-19T01:25:36.823 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.823 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.823 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.824 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.824 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.824 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.824 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.824 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.824 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.824 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.824 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.824 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.825 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-03-19T01:25:36.825 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-03-19T01:25:36.825 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.825 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.825 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.825 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.825 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.825 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.825 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.826 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.826 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.826 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.826 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.826 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.826 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-03-19T01:25:36.826 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-03-19T01:25:36.826 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.827 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.827 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.827 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.827 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-03-19T01:25:36.827 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-03-19T01:25:36.827 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.827 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.827 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.827 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.828 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.828 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-03-19T01:25:36.828 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-03-19T01:25:36.828 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.828 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.828 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.828 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.829 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.829 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.829 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.829 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.829 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.829 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.829 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.829 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.829 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-03-19T01:25:36.830 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-03-19T01:25:36.830 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.830 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.830 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.830 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.830 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.830 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.830 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.831 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.831 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.831 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.831 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.831 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.831 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-03-19T01:25:36.831 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-03-19T01:25:36.831 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.832 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.832 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.832 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.832 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.832 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.832 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.832 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.832 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.832 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.833 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.833 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.833 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.833 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.833 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.833 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-03-19T01:25:36.833 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.833 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.833 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.834 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-03-19T01:25:36.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-03-19T01:25:36.834 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.834 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.834 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.834 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.834 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.834 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.835 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.835 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-03-19T01:25:36.835 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.835 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.835 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.835 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.835 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-03-19T01:25:36.835 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-03-19T01:25:36.836 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.836 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.836 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.836 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.836 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.836 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.836 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.836 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-03-19T01:25:36.836 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.837 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.837 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.837 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.837 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.837 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.837 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.837 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.838 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.838 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.838 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.838 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.838 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.838 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.838 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.838 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.838 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.839 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.839 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.839 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.839 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.839 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.839 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.839 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.839 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.840 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.840 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.840 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.840 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.840 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.840 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.840 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.840 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.840 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.841 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.841 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.841 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.841 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.841 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.841 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.841 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.841 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.843 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.843 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.843 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.843 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.843 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.843 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.843 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.843 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.844 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.844 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.844 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.844 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.844 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.844 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.844 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.844 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.845 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.845 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.845 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.845 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.845 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.845 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.845 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.845 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.846 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-03-19T01:25:36.846 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.846 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.846 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.846 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.846 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.846 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.846 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.847 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.847 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.847 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.847 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.847 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.847 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.847 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.847 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.848 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.848 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.848 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.848 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.848 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.848 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.848 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.848 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.848 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.849 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.849 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.849 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.849 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.849 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.849 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.849 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.849 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.850 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.850 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.850 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.850 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.850 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.850 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.850 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.850 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.851 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.851 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.851 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.851 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.851 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.851 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.851 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.851 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.851 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.852 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.852 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.852 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.852 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.852 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.852 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.852 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.852 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.853 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.853 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.853 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.853 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.853 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.853 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.853 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.853 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.854 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.854 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.854 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.854 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.854 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.854 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.854 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.854 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.854 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.855 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.855 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.855 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-03-19T01:25:36.855 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.855 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.855 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.855 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.855 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.856 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.856 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.856 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.856 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.856 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.856 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.856 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.856 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.856 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.857 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.857 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.857 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.857 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.857 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.857 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.857 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.858 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.858 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.858 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.858 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.858 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.858 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.858 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.859 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.859 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.859 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.859 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.859 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.859 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.859 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.859 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.859 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.860 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.860 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.860 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.860 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.860 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.860 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.860 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.860 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.861 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.861 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.861 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.861 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.861 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.861 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.861 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.861 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.862 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.862 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.862 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.862 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.862 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.862 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.862 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.862 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.863 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.863 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.863 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.863 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.863 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.863 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.863 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.863 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.863 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.864 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.864 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.864 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.864 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.864 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.864 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-03-19T01:25:36.864 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.864 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.864 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.865 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-03-19T01:25:36.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-03-19T01:25:36.865 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.865 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.865 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.865 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-19T01:25:36.865 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.865 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.866 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.866 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.866 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.866 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.866 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.866 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.866 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:36.866 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:36.866 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.867 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.867 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.867 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-03-19T01:25:36.867 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.867 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.867 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.867 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.867 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-03-19T01:25:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-03-19T01:25:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-03-19T01:25:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-19T01:25:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-03-19T01:25:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-03-19T01:25:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.880 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.880 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.880 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-19T01:25:36.880 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.880 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.880 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.880 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.880 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-03-19T01:25:36.880 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-03-19T01:25:36.881 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.881 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.881 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.881 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.881 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.881 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.881 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.881 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-19T01:25:36.882 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.882 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.882 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.882 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.882 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-03-19T01:25:36.882 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-03-19T01:25:36.882 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.882 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.883 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.883 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.883 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.883 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.883 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.883 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-19T01:25:36.883 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.883 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.883 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.884 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.884 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-03-19T01:25:36.884 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-03-19T01:25:36.884 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.884 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.884 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.884 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.884 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.885 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.885 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.885 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-19T01:25:36.885 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.885 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.885 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.885 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-03-19T01:25:36.886 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-03-19T01:25:36.886 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.886 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.886 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.886 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.886 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.886 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.886 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.887 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.887 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.887 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.887 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.887 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.887 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-03-19T01:25:36.887 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-03-19T01:25:36.887 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.888 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.888 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.888 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.888 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.888 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.888 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.888 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.888 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.889 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.889 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.889 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-03-19T01:25:36.889 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-03-19T01:25:36.889 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.889 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.889 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.889 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-19T01:25:36.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.890 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.890 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.890 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.890 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.890 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.890 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.891 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-03-19T01:25:36.891 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-03-19T01:25:36.891 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.891 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.891 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.891 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.891 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.891 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.892 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.892 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.892 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.892 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.892 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.892 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.892 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:36.892 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:36.893 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.893 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.893 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.893 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-03-19T01:25:36.893 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-03-19T01:25:36.893 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.893 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.893 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.894 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.894 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.894 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.894 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.894 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-03-19T01:25:36.894 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-03-19T01:25:36.894 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.894 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.895 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.895 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-03-19T01:25:36.895 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.895 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.895 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.895 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.895 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.895 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.895 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.896 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.896 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-03-19T01:25:36.896 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-03-19T01:25:36.896 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.896 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.896 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.896 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.896 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.897 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.897 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.897 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.897 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.897 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.897 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.897 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-03-19T01:25:36.897 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-03-19T01:25:36.898 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.898 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.898 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.898 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.898 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.898 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.898 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.898 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.899 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.899 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.899 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.899 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.899 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.899 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.899 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.899 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.900 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.900 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.900 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.900 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.900 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.900 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.900 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.900 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.901 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.901 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.901 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.901 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.901 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.901 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.901 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.901 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.902 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.902 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.902 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.902 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.902 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.902 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.902 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.902 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.903 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.903 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.903 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.903 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.903 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.903 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.903 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.903 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.903 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.904 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.904 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.904 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.904 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.904 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.904 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.904 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.904 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.904 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.905 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.905 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.905 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.905 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.905 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.905 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.905 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.906 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.906 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.906 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.906 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.906 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.906 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.906 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.906 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.907 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.907 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.907 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.907 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.907 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.907 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-03-19T01:25:36.907 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-03-19T01:25:36.907 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.908 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.908 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.908 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.908 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.908 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.908 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.908 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.908 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.908 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.909 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.909 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-03-19T01:25:36.909 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-03-19T01:25:36.909 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.909 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.909 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.909 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.910 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.910 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.910 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.910 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.910 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.910 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.910 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.910 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.911 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-03-19T01:25:36.911 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-03-19T01:25:36.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.911 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.911 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.911 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-19T01:25:36.911 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.911 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.911 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.912 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.912 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.912 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.912 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.912 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.912 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.912 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.912 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-03-19T01:25:36.913 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.913 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.913 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.913 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.913 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.913 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.913 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.913 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.914 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.914 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.914 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.914 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.914 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.914 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.914 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.914 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.915 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.915 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.915 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.915 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.915 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.915 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.915 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.916 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.916 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.916 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.916 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.916 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.916 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.916 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.916 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.917 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.917 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.917 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.917 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.917 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.917 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.917 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.917 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.917 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.918 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.918 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.918 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.918 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.918 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.918 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.918 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.918 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.919 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.919 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.919 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.919 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.919 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.919 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.919 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.919 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.920 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.920 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.920 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.920 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.920 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.920 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.920 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.921 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.921 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.921 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.921 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.921 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.921 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.921 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.922 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.922 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.922 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-03-19T01:25:36.922 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.922 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.922 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.922 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.922 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.923 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.923 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.923 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.923 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.923 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.923 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.923 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.923 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.924 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.924 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.924 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.924 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.924 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.924 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.924 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.924 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.925 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.925 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.925 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.925 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.925 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.925 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.925 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.926 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.926 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.926 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.926 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.926 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.926 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.926 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.927 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.927 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.927 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.927 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.927 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.927 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.927 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.927 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.928 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.928 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.928 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.928 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.928 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.928 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.928 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.928 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.929 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.929 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.929 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.929 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.929 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.929 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.929 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.930 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.930 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.930 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.930 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.930 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.930 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.931 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.931 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.931 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.931 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.931 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-03-19T01:25:36.931 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-03-19T01:25:36.932 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.932 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.932 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.932 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-03-19T01:25:36.932 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.932 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.932 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.932 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-19T01:25:36.933 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.933 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.933 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.933 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-03-19T01:25:36.933 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-03-19T01:25:36.933 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.933 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.934 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.934 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.934 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.934 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.934 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-03-19T01:25:36.934 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.934 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.935 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.935 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-03-19T01:25:36.935 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-03-19T01:25:36.935 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.935 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.935 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.935 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-03-19T01:25:36.935 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.936 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.936 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.936 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-19T01:25:36.936 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.936 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.936 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.936 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.936 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-03-19T01:25:36.937 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-03-19T01:25:36.937 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.937 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.937 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.937 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-03-19T01:25:36.937 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.937 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.937 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.937 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-19T01:25:36.938 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.938 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.938 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.938 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.938 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.938 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.938 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-03-19T01:25:36.939 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-03-19T01:25:36.939 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-03-19T01:25:36.939 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-03-19T01:25:36.939 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-03-19T01:25:36.939 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-03-19T01:25:36.939 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-03-19T01:25:36.939 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.939 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.940 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-03-19T01:25:36.940 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.940 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.940 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.940 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.940 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.940 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.940 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.940 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.941 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-03-19T01:25:36.941 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-03-19T01:25:36.941 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.941 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.941 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.941 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.941 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.942 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.942 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.942 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.942 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.942 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.942 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.942 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.943 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-03-19T01:25:36.943 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-03-19T01:25:36.943 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-03-19T01:25:36.943 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.943 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.943 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-03-19T01:25:36.943 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.944 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.944 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.944 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.944 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.944 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.944 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.944 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.944 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-03-19T01:25:36.944 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-03-19T01:25:36.945 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.945 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.945 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.945 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.945 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.945 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.945 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.945 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.946 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.946 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.946 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.946 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.946 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.946 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.946 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.946 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.947 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.947 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.947 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.947 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-03-19T01:25:36.947 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-03-19T01:25:36.947 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.947 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.947 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.947 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.948 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.948 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.948 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.948 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.948 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.948 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.948 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.948 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.949 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-03-19T01:25:36.949 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-03-19T01:25:36.949 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.949 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.949 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.949 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-03-19T01:25:36.949 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.949 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.949 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-03-19T01:25:36.950 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-03-19T01:25:36.950 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-03-19T01:25:36.950 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-03-19T01:25:36.950 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-03-19T01:25:36.950 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-03-19T01:25:36.950 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-03-19T01:25:36.950 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.950 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.951 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.951 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.951 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-03-19T01:25:36.951 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-03-19T01:25:36.951 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-03-19T01:25:36.951 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.951 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-03-19T01:25:36.952 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-03-19T01:25:36.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.952 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.952 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.952 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.952 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.952 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.952 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.952 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.953 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-03-19T01:25:36.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-03-19T01:25:36.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.953 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.953 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.953 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-03-19T01:25:36.953 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.953 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.954 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.954 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.954 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.954 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.954 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.954 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-03-19T01:25:36.954 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-03-19T01:25:36.955 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-03-19T01:25:36.955 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.955 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.955 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-03-19T01:25:36.955 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.955 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.955 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.955 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.955 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.956 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.956 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.956 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.956 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-03-19T01:25:36.956 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-03-19T01:25:36.956 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-03-19T01:25:36.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-03-19T01:25:36.957 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-03-19T01:25:36.957 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-19T01:25:36.957 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-03-19T01:25:36.957 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-03-19T01:25:36.957 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-03-19T01:25:36.957 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-03-19T01:25:36.957 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-03-19T01:25:36.957 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-03-19T01:25:36.957 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-03-19T01:25:36.958 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.958 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.958 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.958 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.958 INFO:teuthology.orchestra.run.smithi069.stdout: "services": { 2024-03-19T01:25:36.958 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard": "https://172.21.15.69:8443/" 2024-03-19T01:25:36.958 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.958 INFO:teuthology.orchestra.run.smithi069.stdout: "always_on_modules": { 2024-03-19T01:25:36.959 INFO:teuthology.orchestra.run.smithi069.stdout: "octopus": [ 2024-03-19T01:25:36.959 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-03-19T01:25:36.959 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-03-19T01:25:36.959 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-03-19T01:25:36.959 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-03-19T01:25:36.959 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-03-19T01:25:36.959 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-03-19T01:25:36.959 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-03-19T01:25:36.960 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-03-19T01:25:36.960 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-03-19T01:25:36.960 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-03-19T01:25:36.960 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.960 INFO:teuthology.orchestra.run.smithi069.stdout: "pacific": [ 2024-03-19T01:25:36.960 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-03-19T01:25:36.960 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-03-19T01:25:36.960 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-03-19T01:25:36.961 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-03-19T01:25:36.961 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-03-19T01:25:36.961 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-03-19T01:25:36.961 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-03-19T01:25:36.961 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-03-19T01:25:36.961 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-03-19T01:25:36.961 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-03-19T01:25:36.961 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.961 INFO:teuthology.orchestra.run.smithi069.stdout: "quincy": [ 2024-03-19T01:25:36.962 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-03-19T01:25:36.962 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-03-19T01:25:36.962 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-03-19T01:25:36.962 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-03-19T01:25:36.962 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-03-19T01:25:36.962 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-03-19T01:25:36.962 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-03-19T01:25:36.962 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-03-19T01:25:36.963 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-03-19T01:25:36.963 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-03-19T01:25:36.963 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.963 INFO:teuthology.orchestra.run.smithi069.stdout: "reef": [ 2024-03-19T01:25:36.963 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-03-19T01:25:36.963 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-03-19T01:25:36.963 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-03-19T01:25:36.963 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-03-19T01:25:36.964 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-03-19T01:25:36.964 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-03-19T01:25:36.964 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-03-19T01:25:36.964 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-03-19T01:25:36.964 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-03-19T01:25:36.964 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-03-19T01:25:36.964 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-03-19T01:25:36.964 INFO:teuthology.orchestra.run.smithi069.stdout: "squid": [ 2024-03-19T01:25:36.964 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-03-19T01:25:36.965 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-03-19T01:25:36.965 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-03-19T01:25:36.965 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-03-19T01:25:36.965 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-03-19T01:25:36.965 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-03-19T01:25:36.965 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-03-19T01:25:36.965 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-03-19T01:25:36.965 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-03-19T01:25:36.966 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-03-19T01:25:36.966 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T01:25:36.966 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.966 INFO:teuthology.orchestra.run.smithi069.stdout: "last_failure_osd_epoch": 94, 2024-03-19T01:25:36.966 INFO:teuthology.orchestra.run.smithi069.stdout: "active_clients": [ 2024-03-19T01:25:36.966 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.966 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-03-19T01:25:36.966 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-03-19T01:25:36.966 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.967 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-03-19T01:25:36.967 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-03-19T01:25:36.967 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1502161019 2024-03-19T01:25:36.967 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.967 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T01:25:36.967 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.967 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.967 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "libcephsqlite", 2024-03-19T01:25:36.968 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-03-19T01:25:36.968 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.968 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-03-19T01:25:36.968 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-03-19T01:25:36.968 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1373409245 2024-03-19T01:25:36.968 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.968 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T01:25:36.968 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.968 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.969 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-03-19T01:25:36.969 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-03-19T01:25:36.969 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.969 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-03-19T01:25:36.969 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-03-19T01:25:36.969 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 993446402 2024-03-19T01:25:36.969 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.969 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T01:25:36.969 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.970 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.970 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-03-19T01:25:36.970 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-03-19T01:25:36.970 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.970 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-03-19T01:25:36.970 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-03-19T01:25:36.970 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 660990571 2024-03-19T01:25:36.970 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.971 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T01:25:36.971 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.971 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.971 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-03-19T01:25:36.971 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-03-19T01:25:36.971 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.971 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-03-19T01:25:36.972 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-03-19T01:25:36.972 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2924953471 2024-03-19T01:25:36.972 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.972 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T01:25:36.972 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:36.972 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-03-19T01:25:36.972 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-03-19T01:25:36.973 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-03-19T01:25:36.973 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-03-19T01:25:36.973 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-03-19T01:25:36.973 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1077830426 2024-03-19T01:25:36.973 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.973 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T01:25:36.973 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:36.973 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-19T01:25:36.974 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:25:36.976 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:34 smithi163 ceph-mon[67428]: pgmap v467: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-03-19T01:25:36.976 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:34 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3246216330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:25:36.976 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:34 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/232286882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:25:36.976 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:35 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3211233041' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-19T01:25:36.976 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:36 smithi163 ceph-mon[67428]: pgmap v468: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-19T01:25:36.976 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:34 smithi086 ceph-mon[66829]: pgmap v467: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-03-19T01:25:36.976 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:34 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3246216330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:25:36.977 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:34 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/232286882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:25:36.977 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:35 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3211233041' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-19T01:25:36.977 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:36 smithi086 ceph-mon[66829]: pgmap v468: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-19T01:25:36.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3211233041 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc8880a2970 msgr2=0x7fc8880a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc8880a2970 0x7fc8880a4e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc894005ab0 tx=0x7fc894000bc0 comp rx=0 tx=0).stop 2024-03-19T01:25:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3211233041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4101fd0 msgr2=0x7fc8a419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:25:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4101fd0 0x7fc8a419a5e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9f0 tx=0x7fc89000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:25:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3211233041 shutdown_connections 2024-03-19T01:25:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a41076b0 0x7fc8a419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fc8880a2970 0x7fc8880a4e30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4101fd0 0x7fc8a419a5e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 --2- 172.21.15.69:0/3211233041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8a4101610 0x7fc8a419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:25:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3211233041 >> 172.21.15.69:0/3211233041 conn(0x7fc8a40fd100 msgr2=0x7fc8a40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:25:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3211233041 shutdown_connections 2024-03-19T01:25:36.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:25:34.490+0000 7fc8a8c00640 1 -- 172.21.15.69:0/3211233041 wait complete. 2024-03-19T01:25:36.987 DEBUG:tasks.check_counter:Getting stats from a 2024-03-19T01:25:36.988 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-19T01:25:36.988 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:35 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3211233041' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-19T01:25:36.988 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:36 smithi069 ceph-mon[31727]: pgmap v468: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-19T01:25:37.501 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:25:38.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:38 smithi086 ceph-mon[66829]: pgmap v469: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-03-19T01:25:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:38 smithi069 ceph-mon[31727]: pgmap v469: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-03-19T01:25:38.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:38 smithi163 ceph-mon[67428]: pgmap v469: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-03-19T01:25:40.230 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:25:40.231 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:25:40.231 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 36195, 2024-03-19T01:25:40.231 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 41741, 2024-03-19T01:25:40.231 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 13191444, 2024-03-19T01:25:40.231 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 36791256, 2024-03-19T01:25:40.231 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 12, 2024-03-19T01:25:40.231 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 4, 2024-03-19T01:25:40.231 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 8.771346034, 2024-03-19T01:25:40.232 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 5.171515706, 2024-03-19T01:25:40.232 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 5.361948907, 2024-03-19T01:25:40.232 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.756000811, 2024-03-19T01:25:40.232 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:40.232 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 41741, 2024-03-19T01:25:40.232 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 18.520134060, 2024-03-19T01:25:40.232 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000443691 2024-03-19T01:25:40.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.232 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:40.233 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 35560, 2024-03-19T01:25:40.233 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.015605078, 2024-03-19T01:25:40.233 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000438 2024-03-19T01:25:40.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.233 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T01:25:40.233 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T01:25:40.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.233 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:25:40.234 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 23617, 2024-03-19T01:25:40.234 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 23658, 2024-03-19T01:25:40.234 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 6137634, 2024-03-19T01:25:40.234 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 12429131, 2024-03-19T01:25:40.234 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 24, 2024-03-19T01:25:40.234 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 7, 2024-03-19T01:25:40.234 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 4.386184417, 2024-03-19T01:25:40.234 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 1.565434567, 2024-03-19T01:25:40.234 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.649163078, 2024-03-19T01:25:40.235 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 1.363818445, 2024-03-19T01:25:40.235 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:40.235 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 23656, 2024-03-19T01:25:40.235 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 11.883774750, 2024-03-19T01:25:40.235 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000502357 2024-03-19T01:25:40.235 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.235 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:40.235 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 726, 2024-03-19T01:25:40.235 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000595322, 2024-03-19T01:25:40.236 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000820 2024-03-19T01:25:40.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.236 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 5040, 2024-03-19T01:25:40.236 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 150288 2024-03-19T01:25:40.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.236 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:25:40.236 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 31598, 2024-03-19T01:25:40.236 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 30637, 2024-03-19T01:25:40.236 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 12213127, 2024-03-19T01:25:40.237 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 18850869, 2024-03-19T01:25:40.237 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 38, 2024-03-19T01:25:40.237 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 11, 2024-03-19T01:25:40.237 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 5.797401064, 2024-03-19T01:25:40.237 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 2.125749479, 2024-03-19T01:25:40.237 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 1.407794953, 2024-03-19T01:25:40.238 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 1.497901477, 2024-03-19T01:25:40.238 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:40.238 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 30634, 2024-03-19T01:25:40.238 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 14.589410423, 2024-03-19T01:25:40.242 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000476248 2024-03-19T01:25:40.242 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.242 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:40.242 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 8356, 2024-03-19T01:25:40.243 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.005388387, 2024-03-19T01:25:40.243 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000644 2024-03-19T01:25:40.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.243 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2197120, 2024-03-19T01:25:40.243 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 951616 2024-03-19T01:25:40.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.243 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-03-19T01:25:40.243 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-03-19T01:25:40.243 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-03-19T01:25:40.244 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.244 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-03-19T01:25:40.244 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-03-19T01:25:40.244 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-03-19T01:25:40.244 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 33198, 2024-03-19T01:25:40.244 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 32.281676303, 2024-03-19T01:25:40.244 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000972398 2024-03-19T01:25:40.244 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:40.244 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.245 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-03-19T01:25:40.245 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-03-19T01:25:40.245 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-03-19T01:25:40.245 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 405, 2024-03-19T01:25:40.245 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.091659959, 2024-03-19T01:25:40.245 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000226320 2024-03-19T01:25:40.245 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:40.245 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.245 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-03-19T01:25:40.246 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 4105, 2024-03-19T01:25:40.246 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 3913, 2024-03-19T01:25:40.246 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-03-19T01:25:40.246 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3913, 2024-03-19T01:25:40.246 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 11.566182738, 2024-03-19T01:25:40.246 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002955835 2024-03-19T01:25:40.246 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.246 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-03-19T01:25:40.247 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 192, 2024-03-19T01:25:40.247 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2024-03-19T01:25:40.247 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2024-03-19T01:25:40.247 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 38, 2024-03-19T01:25:40.247 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 3, 2024-03-19T01:25:40.247 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-03-19T01:25:40.247 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 6691, 2024-03-19T01:25:40.248 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 203, 2024-03-19T01:25:40.248 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 6035, 2024-03-19T01:25:40.248 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 453, 2024-03-19T01:25:40.248 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 6400, 2024-03-19T01:25:40.248 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 627, 2024-03-19T01:25:40.248 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 4530, 2024-03-19T01:25:40.248 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 4530, 2024-03-19T01:25:40.249 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 218, 2024-03-19T01:25:40.249 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 5235, 2024-03-19T01:25:40.249 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 3827, 2024-03-19T01:25:40.249 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 114, 2024-03-19T01:25:40.249 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 183, 2024-03-19T01:25:40.249 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-03-19T01:25:40.249 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-03-19T01:25:40.249 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 1, 2024-03-19T01:25:40.249 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2024-03-19T01:25:40.250 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-03-19T01:25:40.250 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 20, 2024-03-19T01:25:40.250 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 632, 2024-03-19T01:25:40.250 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 162, 2024-03-19T01:25:40.250 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 2953, 2024-03-19T01:25:40.250 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-03-19T01:25:40.250 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 59497, 2024-03-19T01:25:40.250 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 1, 2024-03-19T01:25:40.251 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 89514, 2024-03-19T01:25:40.251 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 1636611686, 2024-03-19T01:25:40.251 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 6, 2024-03-19T01:25:40.251 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 59497, 2024-03-19T01:25:40.251 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-03-19T01:25:40.251 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T01:25:40.251 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-03-19T01:25:40.252 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 5070, 2024-03-19T01:25:40.252 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T01:25:40.252 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 5070, 2024-03-19T01:25:40.252 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 54427, 2024-03-19T01:25:40.252 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2024-03-19T01:25:40.252 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 506, 2024-03-19T01:25:40.252 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 3291, 2024-03-19T01:25:40.252 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 11, 2024-03-19T01:25:40.253 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T01:25:40.253 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T01:25:40.253 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 7838, 2024-03-19T01:25:40.253 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 7231, 2024-03-19T01:25:40.253 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 173, 2024-03-19T01:25:40.253 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 1077 2024-03-19T01:25:40.253 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.253 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-03-19T01:25:40.253 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2024-03-19T01:25:40.254 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 0, 2024-03-19T01:25:40.254 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 0, 2024-03-19T01:25:40.254 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 170, 2024-03-19T01:25:40.254 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 40, 2024-03-19T01:25:40.254 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2024-03-19T01:25:40.254 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-03-19T01:25:40.254 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-03-19T01:25:40.254 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 129, 2024-03-19T01:25:40.255 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 129, 2024-03-19T01:25:40.255 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-03-19T01:25:40.255 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-03-19T01:25:40.255 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-03-19T01:25:40.255 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-03-19T01:25:40.255 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-03-19T01:25:40.255 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-03-19T01:25:40.255 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-03-19T01:25:40.256 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 0, 2024-03-19T01:25:40.256 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 0, 2024-03-19T01:25:40.256 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T01:25:40.256 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 20, 2024-03-19T01:25:40.256 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-03-19T01:25:40.256 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 3, 2024-03-19T01:25:40.256 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-03-19T01:25:40.256 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-03-19T01:25:40.257 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.257 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-03-19T01:25:40.257 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2024-03-19T01:25:40.257 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 11963, 2024-03-19T01:25:40.257 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2024-03-19T01:25:40.257 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2024-03-19T01:25:40.257 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 11963, 2024-03-19T01:25:40.257 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-03-19T01:25:40.257 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2024-03-19T01:25:40.258 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 17, 2024-03-19T01:25:40.258 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2024-03-19T01:25:40.258 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2024-03-19T01:25:40.258 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 17, 2024-03-19T01:25:40.258 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 2, 2024-03-19T01:25:40.258 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-03-19T01:25:40.258 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2024-03-19T01:25:40.258 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 4194304, 2024-03-19T01:25:40.259 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 36484072, 2024-03-19T01:25:40.259 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 0, 2024-03-19T01:25:40.259 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-03-19T01:25:40.259 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1027, 2024-03-19T01:25:40.259 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 22.998570002, 2024-03-19T01:25:40.259 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.022393933 2024-03-19T01:25:40.259 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.259 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 0 2024-03-19T01:25:40.259 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.260 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-03-19T01:25:40.260 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 6384, 2024-03-19T01:25:40.260 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 7138, 2024-03-19T01:25:40.260 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 754, 2024-03-19T01:25:40.260 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 817, 2024-03-19T01:25:40.260 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 845, 2024-03-19T01:25:40.260 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 28, 2024-03-19T01:25:40.260 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 6691, 2024-03-19T01:25:40.260 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 7500, 2024-03-19T01:25:40.261 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 809, 2024-03-19T01:25:40.261 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 4530, 2024-03-19T01:25:40.261 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 7454, 2024-03-19T01:25:40.261 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 2924, 2024-03-19T01:25:40.261 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 98008, 2024-03-19T01:25:40.261 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 223612 2024-03-19T01:25:40.261 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.261 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-03-19T01:25:40.262 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 4968, 2024-03-19T01:25:40.262 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 14, 2024-03-19T01:25:40.262 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 4105, 2024-03-19T01:25:40.262 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 36, 2024-03-19T01:25:40.262 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 492, 2024-03-19T01:25:40.262 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-03-19T01:25:40.262 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2459, 2024-03-19T01:25:40.262 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 7.662875718, 2024-03-19T01:25:40.263 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.003116256 2024-03-19T01:25:40.263 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.263 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-03-19T01:25:40.263 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2, 2024-03-19T01:25:40.263 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.001723445, 2024-03-19T01:25:40.263 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000861722 2024-03-19T01:25:40.263 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.263 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-03-19T01:25:40.263 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.264 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.264 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.264 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.264 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-03-19T01:25:40.264 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.264 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.264 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.264 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.264 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-03-19T01:25:40.265 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 430, 2024-03-19T01:25:40.265 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.424749941, 2024-03-19T01:25:40.265 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000987790 2024-03-19T01:25:40.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.265 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-03-19T01:25:40.265 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.265 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.265 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.266 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.266 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-03-19T01:25:40.266 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1, 2024-03-19T01:25:40.266 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000091745, 2024-03-19T01:25:40.266 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000091745 2024-03-19T01:25:40.266 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.266 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-03-19T01:25:40.266 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.266 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.267 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.267 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-03-19T01:25:40.267 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.267 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.267 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.267 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-03-19T01:25:40.268 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.268 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.268 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.268 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.269 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-03-19T01:25:40.269 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.269 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.269 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.269 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.269 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-03-19T01:25:40.269 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 179, 2024-03-19T01:25:40.269 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.321161732, 2024-03-19T01:25:40.270 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001794199 2024-03-19T01:25:40.270 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.270 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-03-19T01:25:40.270 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.270 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.270 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.270 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.270 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-03-19T01:25:40.271 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.271 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.271 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.271 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-03-19T01:25:40.271 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 204, 2024-03-19T01:25:40.271 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.192694844, 2024-03-19T01:25:40.271 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000944582 2024-03-19T01:25:40.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.272 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-03-19T01:25:40.272 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 496, 2024-03-19T01:25:40.272 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 1.581603971, 2024-03-19T01:25:40.272 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.003188717 2024-03-19T01:25:40.272 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.272 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-03-19T01:25:40.272 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1, 2024-03-19T01:25:40.272 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.003088380, 2024-03-19T01:25:40.273 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.003088380 2024-03-19T01:25:40.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.273 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-03-19T01:25:40.273 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.273 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.273 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.273 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-03-19T01:25:40.274 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.274 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.274 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.274 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.274 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-03-19T01:25:40.274 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.274 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.274 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.274 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.275 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-03-19T01:25:40.275 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.275 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.275 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.275 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.275 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-03-19T01:25:40.275 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.275 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.276 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.276 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.276 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-03-19T01:25:40.276 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 12, 2024-03-19T01:25:40.276 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.558512284, 2024-03-19T01:25:40.276 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.046542690 2024-03-19T01:25:40.276 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.276 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-03-19T01:25:40.276 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.277 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.277 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.277 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-03-19T01:25:40.277 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.277 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.277 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.277 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-03-19T01:25:40.278 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.278 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.278 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.278 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.278 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-03-19T01:25:40.278 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.278 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.278 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.279 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-03-19T01:25:40.279 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.279 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.279 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.279 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-03-19T01:25:40.279 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 129, 2024-03-19T01:25:40.279 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.819680678, 2024-03-19T01:25:40.280 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.006354113 2024-03-19T01:25:40.280 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.280 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-03-19T01:25:40.280 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-03-19T01:25:40.280 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-03-19T01:25:40.280 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.280 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.280 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.281 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:40.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.281 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-03-19T01:25:40.281 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 1, 2024-03-19T01:25:40.281 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 2, 2024-03-19T01:25:40.281 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 1, 2024-03-19T01:25:40.281 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 1, 2024-03-19T01:25:40.281 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-03-19T01:25:40.281 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 168, 2024-03-19T01:25:40.282 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 168, 2024-03-19T01:25:40.282 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 892, 2024-03-19T01:25:40.282 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-03-19T01:25:40.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.282 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-03-19T01:25:40.282 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:25:40.282 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-03-19T01:25:40.282 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:25:40.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:25:40.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:25:40.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:25:40.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:25:40.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:25:40.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:25:40.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:25:40.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:25:40.283 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:25:40.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:25:40.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:25:40.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:25:40.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-03-19T01:25:40.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:25:40.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-03-19T01:25:40.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:25:40.284 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:25:40.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:25:40.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:25:40.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:25:40.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:25:40.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:25:40.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-03-19T01:25:40.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:25:40.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:25:40.285 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:25:40.286 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-03-19T01:25:40.286 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-03-19T01:25:40.286 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-03-19T01:25:40.286 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:25:40.286 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:25:40.286 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:25:40.286 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:25:40.286 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 491362, 2024-03-19T01:25:40.287 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 5240, 2024-03-19T01:25:40.287 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 1320, 2024-03-19T01:25:40.287 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 15, 2024-03-19T01:25:40.287 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-03-19T01:25:40.287 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-03-19T01:25:40.287 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:25:40.287 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-03-19T01:25:40.287 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:25:40.287 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-03-19T01:25:40.288 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 9832, 2024-03-19T01:25:40.288 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 205, 2024-03-19T01:25:40.288 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:25:40.288 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:25:40.288 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-03-19T01:25:40.288 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-03-19T01:25:40.288 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 23497207, 2024-03-19T01:25:40.288 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 292133, 2024-03-19T01:25:40.289 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-03-19T01:25:40.289 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-03-19T01:25:40.289 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-03-19T01:25:40.289 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-03-19T01:25:40.289 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.289 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-03-19T01:25:40.289 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-03-19T01:25:40.289 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-03-19T01:25:40.289 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 61341, 2024-03-19T01:25:40.290 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 33722408, 2024-03-19T01:25:40.290 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 30, 2024-03-19T01:25:40.290 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 61341, 2024-03-19T01:25:40.290 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-03-19T01:25:40.290 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 61341, 2024-03-19T01:25:40.290 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 129.174193825, 2024-03-19T01:25:40.290 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002105837 2024-03-19T01:25:40.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.291 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-03-19T01:25:40.291 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-03-19T01:25:40.291 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 61341, 2024-03-19T01:25:40.291 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 61751 2024-03-19T01:25:40.291 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.291 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 61341, 2024-03-19T01:25:40.291 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 59540, 2024-03-19T01:25:40.291 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 1801, 2024-03-19T01:25:40.291 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-03-19T01:25:40.292 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-03-19T01:25:40.292 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 0, 2024-03-19T01:25:40.292 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 10, 2024-03-19T01:25:40.292 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 43, 2024-03-19T01:25:40.292 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 1070, 2024-03-19T01:25:40.292 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 86, 2024-03-19T01:25:40.292 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-03-19T01:25:40.292 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-03-19T01:25:40.293 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-03-19T01:25:40.293 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 3, 2024-03-19T01:25:40.293 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 277, 2024-03-19T01:25:40.293 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-03-19T01:25:40.293 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-03-19T01:25:40.293 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-03-19T01:25:40.293 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 59497, 2024-03-19T01:25:40.293 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 20, 2024-03-19T01:25:40.293 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-03-19T01:25:40.294 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-03-19T01:25:40.294 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-03-19T01:25:40.294 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-03-19T01:25:40.294 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-03-19T01:25:40.294 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-03-19T01:25:40.294 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T01:25:40.294 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-03-19T01:25:40.294 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-03-19T01:25:40.295 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 11, 2024-03-19T01:25:40.295 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-03-19T01:25:40.295 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-03-19T01:25:40.295 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-03-19T01:25:40.295 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-03-19T01:25:40.295 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-03-19T01:25:40.295 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-03-19T01:25:40.295 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-03-19T01:25:40.295 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-03-19T01:25:40.296 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-03-19T01:25:40.296 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-03-19T01:25:40.296 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-03-19T01:25:40.296 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-03-19T01:25:40.296 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-03-19T01:25:40.296 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-03-19T01:25:40.296 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-03-19T01:25:40.296 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-03-19T01:25:40.296 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 112, 2024-03-19T01:25:40.297 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-03-19T01:25:40.297 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 23, 2024-03-19T01:25:40.297 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 12, 2024-03-19T01:25:40.297 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 12, 2024-03-19T01:25:40.297 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 0, 2024-03-19T01:25:40.297 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-03-19T01:25:40.297 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 594, 2024-03-19T01:25:40.297 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 0, 2024-03-19T01:25:40.297 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 140 2024-03-19T01:25:40.298 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.298 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-03-19T01:25:40.298 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 1, 2024-03-19T01:25:40.298 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-03-19T01:25:40.298 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-03-19T01:25:40.298 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-03-19T01:25:40.298 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.298 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-03-19T01:25:40.299 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2024-03-19T01:25:40.299 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.299 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-03-19T01:25:40.299 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-03-19T01:25:40.299 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 10, 2024-03-19T01:25:40.299 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-03-19T01:25:40.299 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 5, 2024-03-19T01:25:40.299 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 258, 2024-03-19T01:25:40.299 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 129, 2024-03-19T01:25:40.300 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-03-19T01:25:40.300 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.300 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:25:40.300 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:40.300 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-03-19T01:25:40.300 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:40.300 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 91406, 2024-03-19T01:25:40.300 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 23476288, 2024-03-19T01:25:40.300 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:40.301 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 91406, 2024-03-19T01:25:40.301 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-03-19T01:25:40.301 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-03-19T01:25:40.301 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 91406, 2024-03-19T01:25:40.301 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 23476288, 2024-03-19T01:25:40.301 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:40.301 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.301 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.301 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.302 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:40.302 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.302 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-03-19T01:25:40.302 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:40.302 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-03-19T01:25:40.302 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:40.302 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-03-19T01:25:40.302 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-03-19T01:25:40.303 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:40.303 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-03-19T01:25:40.303 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 61341, 2024-03-19T01:25:40.303 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 33736469, 2024-03-19T01:25:40.303 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 61341, 2024-03-19T01:25:40.303 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 33736469, 2024-03-19T01:25:40.303 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:40.303 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.303 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.304 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.304 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:40.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.304 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-03-19T01:25:40.304 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:40.304 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-03-19T01:25:40.304 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:40.304 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-03-19T01:25:40.304 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-03-19T01:25:40.305 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:40.305 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-03-19T01:25:40.305 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 61341, 2024-03-19T01:25:40.305 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 61341, 2024-03-19T01:25:40.305 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 61341, 2024-03-19T01:25:40.305 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 61341, 2024-03-19T01:25:40.305 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:40.305 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.305 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.306 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.306 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:40.306 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.306 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-03-19T01:25:40.306 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:40.306 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-03-19T01:25:40.306 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:40.306 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 129, 2024-03-19T01:25:40.307 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 14061, 2024-03-19T01:25:40.307 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:40.307 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 129, 2024-03-19T01:25:40.307 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-03-19T01:25:40.307 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-03-19T01:25:40.307 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 43, 2024-03-19T01:25:40.307 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 14061, 2024-03-19T01:25:40.307 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:40.308 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.308 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.308 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.308 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:40.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:40.308 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x564c2b56c500": { 2024-03-19T01:25:40.308 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:40.308 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-03-19T01:25:40.308 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:40.309 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 11963, 2024-03-19T01:25:40.309 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 32289768, 2024-03-19T01:25:40.309 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:40.309 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 11963, 2024-03-19T01:25:40.309 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-03-19T01:25:40.309 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-03-19T01:25:40.309 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 1027, 2024-03-19T01:25:40.309 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 32289768, 2024-03-19T01:25:40.310 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:40.310 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:40.310 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:40.310 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:40.310 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:40.310 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:40.310 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:25:40.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:40 smithi086 ceph-mon[66829]: pgmap v470: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-03-19T01:25:40.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:40 smithi069 ceph-mon[31727]: pgmap v470: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-03-19T01:25:40.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:40 smithi163 ceph-mon[67428]: pgmap v470: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-03-19T01:25:41.181 INFO:tasks.check_counter:Daemon mds.a mds.exported=20 2024-03-19T01:25:41.181 INFO:tasks.check_counter:Daemon mds.a mds.imported=162 2024-03-19T01:25:41.182 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-03-19T01:25:41.182 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-19T01:25:41.182 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-19T01:25:41.182 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=3 2024-03-19T01:25:41.182 DEBUG:tasks.check_counter:Getting stats from d 2024-03-19T01:25:41.182 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-19T01:25:41.637 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:25:42.351 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:42 smithi069 ceph-mon[31727]: pgmap v471: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-03-19T01:25:42.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:42 smithi086 ceph-mon[66829]: pgmap v471: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-03-19T01:25:42.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:42 smithi163 ceph-mon[67428]: pgmap v471: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-03-19T01:25:44.041 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:25:44.041 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:25:44.041 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 69862, 2024-03-19T01:25:44.041 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 69143, 2024-03-19T01:25:44.041 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 22400775, 2024-03-19T01:25:44.042 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 90651467, 2024-03-19T01:25:44.042 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 13, 2024-03-19T01:25:44.042 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 6, 2024-03-19T01:25:44.042 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 14.739754562, 2024-03-19T01:25:44.042 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 7.743974750, 2024-03-19T01:25:44.042 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 3.196426879, 2024-03-19T01:25:44.042 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 2.836963606, 2024-03-19T01:25:44.042 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:44.042 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 69142, 2024-03-19T01:25:44.043 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 22.390442591, 2024-03-19T01:25:44.043 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000323832 2024-03-19T01:25:44.043 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.043 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:44.043 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 33108, 2024-03-19T01:25:44.043 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.015384676, 2024-03-19T01:25:44.043 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000464 2024-03-19T01:25:44.043 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.044 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 259424, 2024-03-19T01:25:44.044 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 498448 2024-03-19T01:25:44.044 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.044 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:25:44.044 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 245225, 2024-03-19T01:25:44.044 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 324565, 2024-03-19T01:25:44.044 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 72666237, 2024-03-19T01:25:44.044 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 201145290, 2024-03-19T01:25:44.044 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 38, 2024-03-19T01:25:44.045 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 8, 2024-03-19T01:25:44.045 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 54.530318831, 2024-03-19T01:25:44.045 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 33.374986801, 2024-03-19T01:25:44.045 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 297.859867268, 2024-03-19T01:25:44.045 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 7.024888998, 2024-03-19T01:25:44.045 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:44.045 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 324563, 2024-03-19T01:25:44.045 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 86.811418855, 2024-03-19T01:25:44.045 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000267471 2024-03-19T01:25:44.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.046 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:44.046 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 162355, 2024-03-19T01:25:44.046 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.065631795, 2024-03-19T01:25:44.046 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000404 2024-03-19T01:25:44.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.046 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2037856, 2024-03-19T01:25:44.046 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 172880 2024-03-19T01:25:44.047 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.047 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:25:44.047 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 182691, 2024-03-19T01:25:44.047 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 184489, 2024-03-19T01:25:44.047 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 46819913, 2024-03-19T01:25:44.047 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 174436819, 2024-03-19T01:25:44.047 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 28, 2024-03-19T01:25:44.047 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 6, 2024-03-19T01:25:44.047 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 25.380167735, 2024-03-19T01:25:44.048 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 8.311645812, 2024-03-19T01:25:44.048 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 4.397300132, 2024-03-19T01:25:44.048 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 8.149774171, 2024-03-19T01:25:44.048 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:44.048 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 184485, 2024-03-19T01:25:44.048 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 63.212906744, 2024-03-19T01:25:44.048 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000342645 2024-03-19T01:25:44.048 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.048 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:44.049 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 8107, 2024-03-19T01:25:44.049 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.003616414, 2024-03-19T01:25:44.049 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000446 2024-03-19T01:25:44.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.049 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-03-19T01:25:44.049 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 148576 2024-03-19T01:25:44.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.049 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-03-19T01:25:44.050 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-03-19T01:25:44.050 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-03-19T01:25:44.050 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.050 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-03-19T01:25:44.050 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-03-19T01:25:44.050 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-03-19T01:25:44.050 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 207819, 2024-03-19T01:25:44.050 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 221.475606967, 2024-03-19T01:25:44.051 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001065713 2024-03-19T01:25:44.051 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:44.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.051 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-03-19T01:25:44.051 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-03-19T01:25:44.051 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-03-19T01:25:44.051 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2342, 2024-03-19T01:25:44.051 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 1.338025304, 2024-03-19T01:25:44.052 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000571317 2024-03-19T01:25:44.052 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:44.052 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.052 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-03-19T01:25:44.052 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 40939, 2024-03-19T01:25:44.052 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 40603, 2024-03-19T01:25:44.052 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-03-19T01:25:44.052 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 40603, 2024-03-19T01:25:44.052 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 162.144239121, 2024-03-19T01:25:44.053 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.003993405 2024-03-19T01:25:44.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.053 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-03-19T01:25:44.053 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 336, 2024-03-19T01:25:44.053 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 22, 2024-03-19T01:25:44.053 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2024-03-19T01:25:44.053 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 515, 2024-03-19T01:25:44.053 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 56, 2024-03-19T01:25:44.053 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-03-19T01:25:44.054 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 39277, 2024-03-19T01:25:44.054 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 812, 2024-03-19T01:25:44.054 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 38297, 2024-03-19T01:25:44.054 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 168, 2024-03-19T01:25:44.054 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 38116, 2024-03-19T01:25:44.054 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 4373, 2024-03-19T01:25:44.054 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 32752, 2024-03-19T01:25:44.054 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 32752, 2024-03-19T01:25:44.055 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 292, 2024-03-19T01:25:44.055 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 42813, 2024-03-19T01:25:44.055 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 40565, 2024-03-19T01:25:44.055 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 291, 2024-03-19T01:25:44.055 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 142, 2024-03-19T01:25:44.055 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-03-19T01:25:44.055 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-03-19T01:25:44.055 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2024-03-19T01:25:44.055 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2024-03-19T01:25:44.055 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-03-19T01:25:44.056 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 238, 2024-03-19T01:25:44.056 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 4373, 2024-03-19T01:25:44.056 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 106, 2024-03-19T01:25:44.056 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 2337, 2024-03-19T01:25:44.056 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-03-19T01:25:44.056 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 372846, 2024-03-19T01:25:44.056 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2024-03-19T01:25:44.056 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 89514, 2024-03-19T01:25:44.057 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 1636611686, 2024-03-19T01:25:44.057 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 6, 2024-03-19T01:25:44.057 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 372846, 2024-03-19T01:25:44.057 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-03-19T01:25:44.057 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T01:25:44.057 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-03-19T01:25:44.057 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 19370, 2024-03-19T01:25:44.057 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T01:25:44.057 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 19370, 2024-03-19T01:25:44.058 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 353476, 2024-03-19T01:25:44.058 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 34, 2024-03-19T01:25:44.058 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 2016, 2024-03-19T01:25:44.058 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 40357, 2024-03-19T01:25:44.058 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 89, 2024-03-19T01:25:44.058 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T01:25:44.058 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T01:25:44.058 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 50720, 2024-03-19T01:25:44.059 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 43411, 2024-03-19T01:25:44.059 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 946, 2024-03-19T01:25:44.059 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 5463 2024-03-19T01:25:44.059 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.059 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-03-19T01:25:44.059 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2024-03-19T01:25:44.059 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 0, 2024-03-19T01:25:44.059 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 0, 2024-03-19T01:25:44.060 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 125, 2024-03-19T01:25:44.060 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 260, 2024-03-19T01:25:44.060 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2024-03-19T01:25:44.060 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-03-19T01:25:44.060 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-03-19T01:25:44.060 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 1244, 2024-03-19T01:25:44.060 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 1244, 2024-03-19T01:25:44.060 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-03-19T01:25:44.061 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-03-19T01:25:44.061 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-03-19T01:25:44.061 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-03-19T01:25:44.061 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-03-19T01:25:44.061 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-03-19T01:25:44.061 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-03-19T01:25:44.061 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 0, 2024-03-19T01:25:44.061 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 0, 2024-03-19T01:25:44.061 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T01:25:44.062 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 238, 2024-03-19T01:25:44.062 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-03-19T01:25:44.062 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 57, 2024-03-19T01:25:44.062 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-03-19T01:25:44.062 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-03-19T01:25:44.062 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.062 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-03-19T01:25:44.062 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 2, 2024-03-19T01:25:44.062 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 91417, 2024-03-19T01:25:44.063 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2024-03-19T01:25:44.063 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2024-03-19T01:25:44.063 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 91417, 2024-03-19T01:25:44.063 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-03-19T01:25:44.063 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2024-03-19T01:25:44.063 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 113, 2024-03-19T01:25:44.063 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2024-03-19T01:25:44.063 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2024-03-19T01:25:44.063 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 113, 2024-03-19T01:25:44.064 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 9, 2024-03-19T01:25:44.064 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-03-19T01:25:44.064 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2024-03-19T01:25:44.064 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 4194304, 2024-03-19T01:25:44.064 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 247893344, 2024-03-19T01:25:44.064 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 0, 2024-03-19T01:25:44.064 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-03-19T01:25:44.064 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 4356, 2024-03-19T01:25:44.064 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 377.958503736, 2024-03-19T01:25:44.065 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.086767333 2024-03-19T01:25:44.065 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.065 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 0 2024-03-19T01:25:44.065 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.065 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-03-19T01:25:44.065 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 37896, 2024-03-19T01:25:44.065 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 43513, 2024-03-19T01:25:44.065 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 5617, 2024-03-19T01:25:44.065 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 2283, 2024-03-19T01:25:44.066 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 2698, 2024-03-19T01:25:44.066 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 415, 2024-03-19T01:25:44.066 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 39277, 2024-03-19T01:25:44.066 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 45018, 2024-03-19T01:25:44.066 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 5741, 2024-03-19T01:25:44.067 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 32752, 2024-03-19T01:25:44.067 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 40919, 2024-03-19T01:25:44.067 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 8167, 2024-03-19T01:25:44.067 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 249824, 2024-03-19T01:25:44.067 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 223612 2024-03-19T01:25:44.067 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.067 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-03-19T01:25:44.067 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 43465, 2024-03-19T01:25:44.068 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 551, 2024-03-19T01:25:44.068 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 40939, 2024-03-19T01:25:44.068 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 261, 2024-03-19T01:25:44.068 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 1839, 2024-03-19T01:25:44.068 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-03-19T01:25:44.068 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 34453, 2024-03-19T01:25:44.068 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 129.301695076, 2024-03-19T01:25:44.068 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.003752987 2024-03-19T01:25:44.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.069 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-03-19T01:25:44.069 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2, 2024-03-19T01:25:44.069 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.068877518, 2024-03-19T01:25:44.069 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.034438759 2024-03-19T01:25:44.069 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.069 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-03-19T01:25:44.069 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.070 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.070 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.070 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-03-19T01:25:44.070 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.070 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.070 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.071 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-03-19T01:25:44.071 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 882, 2024-03-19T01:25:44.071 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 1.472302243, 2024-03-19T01:25:44.071 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001669276 2024-03-19T01:25:44.071 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.071 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-03-19T01:25:44.071 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.071 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.072 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.072 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.072 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-03-19T01:25:44.072 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.072 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.072 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.072 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.073 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-03-19T01:25:44.073 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.073 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.073 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.073 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.073 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-03-19T01:25:44.073 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.074 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.074 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.074 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.074 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-03-19T01:25:44.075 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.075 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.075 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.075 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.075 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-03-19T01:25:44.075 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.076 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.076 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.076 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.076 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-03-19T01:25:44.076 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1824, 2024-03-19T01:25:44.076 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 4.389613538, 2024-03-19T01:25:44.076 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002406586 2024-03-19T01:25:44.076 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.077 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-03-19T01:25:44.077 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.077 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.077 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.077 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.077 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-03-19T01:25:44.077 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.077 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.078 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.078 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.078 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-03-19T01:25:44.078 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 222, 2024-03-19T01:25:44.078 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 1.323329161, 2024-03-19T01:25:44.078 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.005960942 2024-03-19T01:25:44.078 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.078 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-03-19T01:25:44.079 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1884, 2024-03-19T01:25:44.079 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 7.793155572, 2024-03-19T01:25:44.079 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.004136494 2024-03-19T01:25:44.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.079 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-03-19T01:25:44.079 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.079 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.079 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.080 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.080 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-03-19T01:25:44.080 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.080 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.080 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.080 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.080 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-03-19T01:25:44.080 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.081 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.081 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.081 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.081 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-03-19T01:25:44.081 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.081 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.081 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.081 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.082 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-03-19T01:25:44.082 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.082 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.082 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.082 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.082 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-03-19T01:25:44.082 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.083 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.083 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.083 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-03-19T01:25:44.083 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 92, 2024-03-19T01:25:44.083 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 9.284810599, 2024-03-19T01:25:44.083 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.100921854 2024-03-19T01:25:44.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.084 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-03-19T01:25:44.084 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.084 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.084 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.084 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.084 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-03-19T01:25:44.084 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.084 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.085 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.085 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.085 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-03-19T01:25:44.085 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.085 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.085 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.085 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.085 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-03-19T01:25:44.086 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.086 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.086 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.086 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.086 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-03-19T01:25:44.086 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.086 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.086 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.087 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.087 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-03-19T01:25:44.087 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1244, 2024-03-19T01:25:44.087 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 8.510455414, 2024-03-19T01:25:44.087 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.006841202 2024-03-19T01:25:44.087 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.087 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-03-19T01:25:44.087 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-03-19T01:25:44.088 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-03-19T01:25:44.088 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.088 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.088 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.088 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:44.088 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.088 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-03-19T01:25:44.088 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 1, 2024-03-19T01:25:44.089 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 4, 2024-03-19T01:25:44.089 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 3, 2024-03-19T01:25:44.089 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 1, 2024-03-19T01:25:44.089 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-03-19T01:25:44.089 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 1969, 2024-03-19T01:25:44.089 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 1969, 2024-03-19T01:25:44.089 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 902, 2024-03-19T01:25:44.089 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-03-19T01:25:44.089 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.090 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-03-19T01:25:44.090 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:25:44.090 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-03-19T01:25:44.090 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:25:44.090 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:25:44.090 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:25:44.090 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:25:44.090 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:25:44.091 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:25:44.091 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:25:44.091 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:25:44.091 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:25:44.091 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:25:44.091 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:25:44.091 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:25:44.091 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:25:44.091 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-03-19T01:25:44.092 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:25:44.092 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-03-19T01:25:44.092 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:25:44.092 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:25:44.092 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:25:44.092 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:25:44.092 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:25:44.092 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:25:44.093 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:25:44.093 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-03-19T01:25:44.093 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:25:44.093 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:25:44.093 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:25:44.093 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-03-19T01:25:44.093 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-03-19T01:25:44.093 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-03-19T01:25:44.093 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:25:44.094 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:25:44.094 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:25:44.094 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:25:44.094 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 1549968, 2024-03-19T01:25:44.094 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 33480, 2024-03-19T01:25:44.094 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 968, 2024-03-19T01:25:44.094 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 11, 2024-03-19T01:25:44.094 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-03-19T01:25:44.095 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-03-19T01:25:44.095 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:25:44.095 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-03-19T01:25:44.095 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:25:44.095 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-03-19T01:25:44.095 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 9832, 2024-03-19T01:25:44.095 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 205, 2024-03-19T01:25:44.095 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:25:44.096 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:25:44.096 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-03-19T01:25:44.096 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-03-19T01:25:44.096 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 136164563, 2024-03-19T01:25:44.096 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 1647847, 2024-03-19T01:25:44.096 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-03-19T01:25:44.096 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-03-19T01:25:44.096 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-03-19T01:25:44.097 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-03-19T01:25:44.097 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.097 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-03-19T01:25:44.097 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-03-19T01:25:44.097 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-03-19T01:25:44.097 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 381423, 2024-03-19T01:25:44.097 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 258795655, 2024-03-19T01:25:44.097 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 33, 2024-03-19T01:25:44.098 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 381423, 2024-03-19T01:25:44.098 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-03-19T01:25:44.098 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 381423, 2024-03-19T01:25:44.098 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 676.060314570, 2024-03-19T01:25:44.098 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001772468 2024-03-19T01:25:44.098 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.098 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-03-19T01:25:44.098 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-03-19T01:25:44.098 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 381423, 2024-03-19T01:25:44.099 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 382732 2024-03-19T01:25:44.099 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.099 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 381423, 2024-03-19T01:25:44.099 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 373001, 2024-03-19T01:25:44.099 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 8422, 2024-03-19T01:25:44.099 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-03-19T01:25:44.099 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-03-19T01:25:44.099 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 0, 2024-03-19T01:25:44.100 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 10, 2024-03-19T01:25:44.100 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 133, 2024-03-19T01:25:44.100 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 4489, 2024-03-19T01:25:44.100 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 129, 2024-03-19T01:25:44.100 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-03-19T01:25:44.100 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-03-19T01:25:44.100 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-03-19T01:25:44.100 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 56, 2024-03-19T01:25:44.101 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 2558, 2024-03-19T01:25:44.101 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-03-19T01:25:44.101 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-03-19T01:25:44.101 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-03-19T01:25:44.101 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 372846, 2024-03-19T01:25:44.101 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 20, 2024-03-19T01:25:44.101 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-03-19T01:25:44.101 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-03-19T01:25:44.102 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-03-19T01:25:44.102 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-03-19T01:25:44.102 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-03-19T01:25:44.102 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-03-19T01:25:44.102 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T01:25:44.102 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-03-19T01:25:44.102 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-03-19T01:25:44.102 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 89, 2024-03-19T01:25:44.103 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-03-19T01:25:44.103 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-03-19T01:25:44.103 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-03-19T01:25:44.103 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-03-19T01:25:44.103 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-03-19T01:25:44.103 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-03-19T01:25:44.103 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-03-19T01:25:44.103 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-03-19T01:25:44.104 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-03-19T01:25:44.104 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-03-19T01:25:44.104 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-03-19T01:25:44.104 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-03-19T01:25:44.104 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-03-19T01:25:44.104 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-03-19T01:25:44.104 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-03-19T01:25:44.104 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-03-19T01:25:44.104 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 112, 2024-03-19T01:25:44.105 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-03-19T01:25:44.105 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 23, 2024-03-19T01:25:44.105 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 12, 2024-03-19T01:25:44.105 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 12, 2024-03-19T01:25:44.105 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 0, 2024-03-19T01:25:44.105 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-03-19T01:25:44.105 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 2088, 2024-03-19T01:25:44.105 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 44, 2024-03-19T01:25:44.106 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 270 2024-03-19T01:25:44.106 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.106 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-03-19T01:25:44.106 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 1, 2024-03-19T01:25:44.106 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-03-19T01:25:44.106 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-03-19T01:25:44.106 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-03-19T01:25:44.106 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.107 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-03-19T01:25:44.107 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2024-03-19T01:25:44.107 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.107 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-03-19T01:25:44.107 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-03-19T01:25:44.107 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 10, 2024-03-19T01:25:44.107 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-03-19T01:25:44.107 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 5, 2024-03-19T01:25:44.107 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 2488, 2024-03-19T01:25:44.108 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 1244, 2024-03-19T01:25:44.108 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-03-19T01:25:44.108 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.108 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:25:44.108 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:44.108 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-03-19T01:25:44.108 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:44.108 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 497774, 2024-03-19T01:25:44.109 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 98536183, 2024-03-19T01:25:44.109 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:44.109 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 497774, 2024-03-19T01:25:44.109 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-03-19T01:25:44.109 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-03-19T01:25:44.109 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 497774, 2024-03-19T01:25:44.109 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 98536183, 2024-03-19T01:25:44.109 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:44.110 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.110 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.110 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.110 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:44.110 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.110 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-03-19T01:25:44.110 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:44.110 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-03-19T01:25:44.111 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:44.111 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-03-19T01:25:44.111 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-03-19T01:25:44.111 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:44.111 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-03-19T01:25:44.111 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 381423, 2024-03-19T01:25:44.111 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 258930915, 2024-03-19T01:25:44.111 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 381423, 2024-03-19T01:25:44.112 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 258930915, 2024-03-19T01:25:44.112 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:44.112 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.112 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.112 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.112 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:44.112 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.112 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-03-19T01:25:44.113 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:44.113 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-03-19T01:25:44.113 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:44.113 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-03-19T01:25:44.113 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-03-19T01:25:44.113 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:44.113 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-03-19T01:25:44.113 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 381423, 2024-03-19T01:25:44.113 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 381423, 2024-03-19T01:25:44.114 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 381423, 2024-03-19T01:25:44.114 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 381423, 2024-03-19T01:25:44.114 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:44.114 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.114 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.115 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.115 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:44.115 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.115 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-03-19T01:25:44.115 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:44.115 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-03-19T01:25:44.115 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:44.115 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 1244, 2024-03-19T01:25:44.115 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 135612, 2024-03-19T01:25:44.116 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:44.116 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 1244, 2024-03-19T01:25:44.116 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-03-19T01:25:44.116 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-03-19T01:25:44.116 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 133, 2024-03-19T01:25:44.116 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 135612, 2024-03-19T01:25:44.116 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:44.116 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.117 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.117 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.117 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:44.117 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:44.117 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x558915a82500": { 2024-03-19T01:25:44.117 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:44.117 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-03-19T01:25:44.117 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:44.117 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 91417, 2024-03-19T01:25:44.118 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 243699040, 2024-03-19T01:25:44.118 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:44.118 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 91417, 2024-03-19T01:25:44.118 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-03-19T01:25:44.118 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-03-19T01:25:44.122 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 4356, 2024-03-19T01:25:44.123 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 243699040, 2024-03-19T01:25:44.123 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:44.123 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:44.123 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:44.123 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:44.123 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:44.123 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:44.123 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:25:44.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:44 smithi086 ceph-mon[66829]: pgmap v472: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-03-19T01:25:44.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:44 smithi069 ceph-mon[31727]: pgmap v472: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-03-19T01:25:44.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:44 smithi163 ceph-mon[67428]: pgmap v472: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-03-19T01:25:45.028 INFO:tasks.check_counter:Daemon mds.d mds.exported=238 2024-03-19T01:25:45.028 INFO:tasks.check_counter:Daemon mds.d mds.imported=106 2024-03-19T01:25:45.028 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=6 2024-03-19T01:25:45.028 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-03-19T01:25:45.029 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-03-19T01:25:45.029 INFO:tasks.check_counter:Daemon mds.d mds.dir_split=56 2024-03-19T01:25:45.029 DEBUG:tasks.check_counter:Getting stats from g 2024-03-19T01:25:45.029 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-19T01:25:45.504 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:25:46.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:46 smithi069 ceph-mon[31727]: pgmap v473: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-19T01:25:46.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:46 smithi086 ceph-mon[66829]: pgmap v473: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-19T01:25:46.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:46 smithi163 ceph-mon[67428]: pgmap v473: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-19T01:25:47.752 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:25:47.753 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:25:47.753 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 313, 2024-03-19T01:25:47.753 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 300, 2024-03-19T01:25:47.753 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1875472, 2024-03-19T01:25:47.753 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 185360, 2024-03-19T01:25:47.753 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-03-19T01:25:47.753 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-03-19T01:25:47.754 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.105932671, 2024-03-19T01:25:47.754 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.041978483, 2024-03-19T01:25:47.754 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.056862572, 2024-03-19T01:25:47.754 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.014346780, 2024-03-19T01:25:47.754 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:47.754 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 299, 2024-03-19T01:25:47.754 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.045747500, 2024-03-19T01:25:47.754 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000153001 2024-03-19T01:25:47.754 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:47.755 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:47.755 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:47.755 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:47.755 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:47.755 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:47.755 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1875472, 2024-03-19T01:25:47.755 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 185360 2024-03-19T01:25:47.755 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:47.755 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:25:47.756 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 6, 2024-03-19T01:25:47.756 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 231, 2024-03-19T01:25:47.756 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 3344, 2024-03-19T01:25:47.756 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 196416, 2024-03-19T01:25:47.756 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-03-19T01:25:47.756 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-03-19T01:25:47.756 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.034093443, 2024-03-19T01:25:47.756 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.025208223, 2024-03-19T01:25:47.756 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.004006176, 2024-03-19T01:25:47.757 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T01:25:47.757 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:47.757 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 229, 2024-03-19T01:25:47.757 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.048828425, 2024-03-19T01:25:47.757 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000213224 2024-03-19T01:25:47.757 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:47.757 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:47.757 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:47.757 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:47.758 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:47.758 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:47.758 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-19T01:25:47.758 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 196416 2024-03-19T01:25:47.758 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:47.758 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:25:47.758 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-03-19T01:25:47.758 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-03-19T01:25:47.759 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-03-19T01:25:47.759 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-03-19T01:25:47.759 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-03-19T01:25:47.759 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-03-19T01:25:47.759 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.000928470, 2024-03-19T01:25:47.759 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-03-19T01:25:47.759 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000766458, 2024-03-19T01:25:47.759 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T01:25:47.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:47.760 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:47.760 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:47.760 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:47.760 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:47.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:47.760 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:47.760 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:47.760 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:47.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:47.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T01:25:47.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T01:25:47.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:47.761 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-03-19T01:25:47.761 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-03-19T01:25:47.761 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-03-19T01:25:47.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:47.762 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-03-19T01:25:47.762 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:25:47.762 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-03-19T01:25:47.762 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:25:47.762 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:25:47.762 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:25:47.762 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:25:47.762 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:25:47.762 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:25:47.763 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:25:47.763 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:25:47.763 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:25:47.763 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:25:47.763 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:25:47.763 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:25:47.763 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:25:47.763 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-03-19T01:25:47.763 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:25:47.764 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-03-19T01:25:47.764 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:25:47.764 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:25:47.764 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:25:47.764 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:25:47.764 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:25:47.764 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:25:47.764 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:25:47.764 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-03-19T01:25:47.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:25:47.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:25:47.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:25:47.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-03-19T01:25:47.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-03-19T01:25:47.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-03-19T01:25:47.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:25:47.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:25:47.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:25:47.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:25:47.766 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148754, 2024-03-19T01:25:47.766 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 31, 2024-03-19T01:25:47.766 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-03-19T01:25:47.766 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-03-19T01:25:47.766 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-03-19T01:25:47.766 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-03-19T01:25:47.767 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:25:47.767 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-03-19T01:25:47.767 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:25:47.767 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-03-19T01:25:47.767 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-03-19T01:25:47.767 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-03-19T01:25:47.767 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:25:47.767 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:25:47.767 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-03-19T01:25:47.768 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-03-19T01:25:47.768 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-03-19T01:25:47.768 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-03-19T01:25:47.768 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-03-19T01:25:47.768 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-03-19T01:25:47.768 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-03-19T01:25:47.768 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-03-19T01:25:47.768 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:47.768 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:25:47.769 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:47.769 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-03-19T01:25:47.769 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:47.769 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 315, 2024-03-19T01:25:47.769 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1831342, 2024-03-19T01:25:47.769 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:47.769 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 315, 2024-03-19T01:25:47.769 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-03-19T01:25:47.769 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-03-19T01:25:47.770 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 315, 2024-03-19T01:25:47.770 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1831342, 2024-03-19T01:25:47.770 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:47.770 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:47.770 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:47.770 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:47.770 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:47.770 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:47.770 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:25:48.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:48 smithi086 ceph-mon[66829]: pgmap v474: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-03-19T01:25:48.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:48 smithi069 ceph-mon[31727]: pgmap v474: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-03-19T01:25:48.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:48 smithi163 ceph-mon[67428]: pgmap v474: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-03-19T01:25:48.596 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-19T01:25:48.596 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-19T01:25:48.596 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-19T01:25:48.596 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-19T01:25:48.597 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-19T01:25:48.597 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-03-19T01:25:48.597 DEBUG:tasks.check_counter:Getting stats from j 2024-03-19T01:25:48.597 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-19T01:25:49.067 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:25:50.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:50 smithi069 ceph-mon[31727]: pgmap v475: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-19T01:25:50.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:50 smithi086 ceph-mon[66829]: pgmap v475: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-19T01:25:50.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:50 smithi163 ceph-mon[67428]: pgmap v475: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-19T01:25:51.215 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-19T01:25:51.216 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:25:51.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-03-19T01:25:51.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-03-19T01:25:51.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-03-19T01:25:51.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-03-19T01:25:51.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-03-19T01:25:51.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-03-19T01:25:51.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.003575782, 2024-03-19T01:25:51.217 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-03-19T01:25:51.217 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000762538, 2024-03-19T01:25:51.217 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T01:25:51.217 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:51.217 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:51.217 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:51.217 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:51.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T01:25:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T01:25:51.219 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:51.219 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:25:51.219 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 323, 2024-03-19T01:25:51.219 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 308, 2024-03-19T01:25:51.219 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1880128, 2024-03-19T01:25:51.219 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 189632, 2024-03-19T01:25:51.219 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-03-19T01:25:51.219 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-03-19T01:25:51.220 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.104098212, 2024-03-19T01:25:51.220 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.040821773, 2024-03-19T01:25:51.220 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.063691201, 2024-03-19T01:25:51.220 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.013310258, 2024-03-19T01:25:51.220 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:51.220 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 308, 2024-03-19T01:25:51.220 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.054787358, 2024-03-19T01:25:51.220 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000177881 2024-03-19T01:25:51.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:51.221 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:51.221 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:51.221 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:51.221 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:51.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:51.221 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1880128, 2024-03-19T01:25:51.221 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 189632 2024-03-19T01:25:51.222 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:51.222 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:25:51.222 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2, 2024-03-19T01:25:51.222 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 235, 2024-03-19T01:25:51.223 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 320, 2024-03-19T01:25:51.223 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 197024, 2024-03-19T01:25:51.223 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-03-19T01:25:51.223 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-03-19T01:25:51.223 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.034312339, 2024-03-19T01:25:51.223 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.026594513, 2024-03-19T01:25:51.223 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.003096807, 2024-03-19T01:25:51.224 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T01:25:51.224 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:51.224 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 232, 2024-03-19T01:25:51.224 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.045982827, 2024-03-19T01:25:51.224 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000198201 2024-03-19T01:25:51.224 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:51.224 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:51.224 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:51.225 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:51.225 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:51.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:51.225 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-19T01:25:51.225 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 197024 2024-03-19T01:25:51.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:51.225 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-03-19T01:25:51.225 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-03-19T01:25:51.226 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-03-19T01:25:51.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:51.226 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-03-19T01:25:51.226 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:25:51.226 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-03-19T01:25:51.226 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:25:51.226 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:25:51.226 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:25:51.227 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:25:51.227 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:25:51.227 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:25:51.227 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:25:51.227 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:25:51.227 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:25:51.227 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:25:51.227 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:25:51.227 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:25:51.228 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:25:51.228 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-03-19T01:25:51.228 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:25:51.228 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-03-19T01:25:51.228 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:25:51.228 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:25:51.228 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:25:51.228 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:25:51.229 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:25:51.229 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:25:51.229 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:25:51.229 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-03-19T01:25:51.229 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:25:51.229 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:25:51.229 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:25:51.229 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-03-19T01:25:51.229 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-03-19T01:25:51.230 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-03-19T01:25:51.230 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:25:51.230 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:25:51.230 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:25:51.230 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:25:51.230 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148754, 2024-03-19T01:25:51.230 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 31, 2024-03-19T01:25:51.230 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-03-19T01:25:51.231 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-03-19T01:25:51.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-03-19T01:25:51.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-03-19T01:25:51.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:25:51.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-03-19T01:25:51.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:25:51.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-03-19T01:25:51.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-03-19T01:25:51.232 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-03-19T01:25:51.232 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:25:51.232 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:25:51.232 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-03-19T01:25:51.232 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-03-19T01:25:51.232 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-03-19T01:25:51.232 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-03-19T01:25:51.232 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-03-19T01:25:51.233 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-03-19T01:25:51.233 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-03-19T01:25:51.233 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-03-19T01:25:51.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-03-19T01:25:51.233 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:25:51.233 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-03-19T01:25:51.233 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-03-19T01:25:51.233 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-03-19T01:25:51.233 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 321, 2024-03-19T01:25:51.234 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1832122, 2024-03-19T01:25:51.234 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:51.234 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 321, 2024-03-19T01:25:51.234 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-03-19T01:25:51.234 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-03-19T01:25:51.234 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 321, 2024-03-19T01:25:51.234 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1832122, 2024-03-19T01:25:51.234 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-03-19T01:25:51.235 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-03-19T01:25:51.235 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-03-19T01:25:51.235 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-03-19T01:25:51.235 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:51.235 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-03-19T01:25:51.235 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-19T01:25:52.052 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-19T01:25:52.053 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-19T01:25:52.053 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-19T01:25:52.053 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-19T01:25:52.053 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-19T01:25:52.053 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-03-19T01:25:52.053 DEBUG:tasks.check_counter:Getting stats from b 2024-03-19T01:25:52.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-19T01:25:52.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:52 smithi069 ceph-mon[31727]: pgmap v476: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-19T01:25:52.386 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:25:52.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:52 smithi086 ceph-mon[66829]: pgmap v476: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-19T01:25:52.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:52 smithi163 ceph-mon[67428]: pgmap v476: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-19T01:25:54.244 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-19T01:25:54.245 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:25:54.245 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 275377, 2024-03-19T01:25:54.245 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 276790, 2024-03-19T01:25:54.245 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 112335978, 2024-03-19T01:25:54.245 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 252275948, 2024-03-19T01:25:54.245 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 18, 2024-03-19T01:25:54.245 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-03-19T01:25:54.246 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 14.233780947, 2024-03-19T01:25:54.246 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 4.464219925, 2024-03-19T01:25:54.246 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 3.616979408, 2024-03-19T01:25:54.246 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 5.197656764, 2024-03-19T01:25:54.246 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:54.246 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 276787, 2024-03-19T01:25:54.246 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 52.473014439, 2024-03-19T01:25:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000189579 2024-03-19T01:25:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 21866, 2024-03-19T01:25:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.003848468, 2024-03-19T01:25:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000176 2024-03-19T01:25:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2667168, 2024-03-19T01:25:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 416720 2024-03-19T01:25:54.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.248 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:25:54.248 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 326154, 2024-03-19T01:25:54.248 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 453116, 2024-03-19T01:25:54.248 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 123281081, 2024-03-19T01:25:54.248 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 333191402, 2024-03-19T01:25:54.248 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 185, 2024-03-19T01:25:54.248 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 9, 2024-03-19T01:25:54.248 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 17.480315032, 2024-03-19T01:25:54.248 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 7.184705228, 2024-03-19T01:25:54.249 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 9.455918866, 2024-03-19T01:25:54.249 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 3.657611485, 2024-03-19T01:25:54.249 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:54.249 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 453112, 2024-03-19T01:25:54.249 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 60.389788670, 2024-03-19T01:25:54.249 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000133277 2024-03-19T01:25:54.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.249 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:54.249 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 243256, 2024-03-19T01:25:54.250 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.050128713, 2024-03-19T01:25:54.250 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000206 2024-03-19T01:25:54.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.250 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 89440, 2024-03-19T01:25:54.250 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 270432 2024-03-19T01:25:54.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.250 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:25:54.250 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 243821, 2024-03-19T01:25:54.251 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 251958, 2024-03-19T01:25:54.251 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 98012523, 2024-03-19T01:25:54.251 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 247546954, 2024-03-19T01:25:54.251 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 13, 2024-03-19T01:25:54.251 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-03-19T01:25:54.251 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 15.248612918, 2024-03-19T01:25:54.251 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 6.087020545, 2024-03-19T01:25:54.251 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 3.614631550, 2024-03-19T01:25:54.251 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 4.701330604, 2024-03-19T01:25:54.252 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:54.252 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 251959, 2024-03-19T01:25:54.252 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 36.050095049, 2024-03-19T01:25:54.252 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000143079 2024-03-19T01:25:54.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.252 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:54.252 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 32269, 2024-03-19T01:25:54.252 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.006015895, 2024-03-19T01:25:54.252 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000186 2024-03-19T01:25:54.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.253 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 6368, 2024-03-19T01:25:54.253 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 424032 2024-03-19T01:25:54.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.253 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-19T01:25:54.253 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-03-19T01:25:54.253 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-19T01:25:54.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.253 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-03-19T01:25:54.253 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-19T01:25:54.254 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-19T01:25:54.254 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 462563, 2024-03-19T01:25:54.254 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 211.907012696, 2024-03-19T01:25:54.254 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000458114 2024-03-19T01:25:54.254 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:54.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.254 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-03-19T01:25:54.254 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-19T01:25:54.255 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-19T01:25:54.255 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2697, 2024-03-19T01:25:54.255 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.224072613, 2024-03-19T01:25:54.255 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000083082 2024-03-19T01:25:54.255 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:54.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.255 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-03-19T01:25:54.255 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 63323, 2024-03-19T01:25:54.255 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 62554, 2024-03-19T01:25:54.256 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-03-19T01:25:54.256 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 62554, 2024-03-19T01:25:54.256 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 183.345837704, 2024-03-19T01:25:54.256 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002931001 2024-03-19T01:25:54.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.256 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-03-19T01:25:54.256 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 769, 2024-03-19T01:25:54.256 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 166, 2024-03-19T01:25:54.256 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 2, 2024-03-19T01:25:54.257 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 2783, 2024-03-19T01:25:54.257 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 161, 2024-03-19T01:25:54.257 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-03-19T01:25:54.257 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 65290, 2024-03-19T01:25:54.257 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 7194, 2024-03-19T01:25:54.257 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 58075, 2024-03-19T01:25:54.257 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 21, 2024-03-19T01:25:54.257 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 55013, 2024-03-19T01:25:54.258 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 8611, 2024-03-19T01:25:54.258 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 46645, 2024-03-19T01:25:54.258 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 46645, 2024-03-19T01:25:54.258 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 530, 2024-03-19T01:25:54.258 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 65092, 2024-03-19T01:25:54.258 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 62028, 2024-03-19T01:25:54.258 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 497, 2024-03-19T01:25:54.258 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 83, 2024-03-19T01:25:54.259 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-03-19T01:25:54.259 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-03-19T01:25:54.259 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-03-19T01:25:54.259 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-03-19T01:25:54.259 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-03-19T01:25:54.259 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 375, 2024-03-19T01:25:54.259 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 8607, 2024-03-19T01:25:54.259 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 66, 2024-03-19T01:25:54.259 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 1436, 2024-03-19T01:25:54.260 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-03-19T01:25:54.260 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 611768, 2024-03-19T01:25:54.260 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-03-19T01:25:54.260 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 89590, 2024-03-19T01:25:54.260 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 1645705921, 2024-03-19T01:25:54.260 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 6, 2024-03-19T01:25:54.260 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 611768, 2024-03-19T01:25:54.260 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-03-19T01:25:54.260 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T01:25:54.261 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-03-19T01:25:54.261 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 37031, 2024-03-19T01:25:54.261 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 20, 2024-03-19T01:25:54.261 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 37011, 2024-03-19T01:25:54.261 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 574737, 2024-03-19T01:25:54.261 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 11, 2024-03-19T01:25:54.261 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 4152, 2024-03-19T01:25:54.261 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 59068, 2024-03-19T01:25:54.262 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 74, 2024-03-19T01:25:54.262 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T01:25:54.262 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T01:25:54.262 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 74103, 2024-03-19T01:25:54.262 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 70764, 2024-03-19T01:25:54.262 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 865, 2024-03-19T01:25:54.262 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 8777 2024-03-19T01:25:54.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.262 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-03-19T01:25:54.263 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-03-19T01:25:54.263 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-03-19T01:25:54.263 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-03-19T01:25:54.263 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 97, 2024-03-19T01:25:54.263 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 399, 2024-03-19T01:25:54.263 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-03-19T01:25:54.263 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-03-19T01:25:54.263 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-03-19T01:25:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 1212, 2024-03-19T01:25:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 1212, 2024-03-19T01:25:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-03-19T01:25:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-03-19T01:25:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-03-19T01:25:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-03-19T01:25:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-03-19T01:25:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-03-19T01:25:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-03-19T01:25:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-03-19T01:25:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-03-19T01:25:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 20, 2024-03-19T01:25:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 375, 2024-03-19T01:25:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-03-19T01:25:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 161, 2024-03-19T01:25:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-03-19T01:25:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-03-19T01:25:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-03-19T01:25:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 6, 2024-03-19T01:25:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 135902, 2024-03-19T01:25:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 43942, 2024-03-19T01:25:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 36871, 2024-03-19T01:25:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 99031, 2024-03-19T01:25:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-03-19T01:25:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 7071, 2024-03-19T01:25:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 181, 2024-03-19T01:25:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 53, 2024-03-19T01:25:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 44, 2024-03-19T01:25:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 137, 2024-03-19T01:25:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 9, 2024-03-19T01:25:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-03-19T01:25:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 9, 2024-03-19T01:25:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 89238282, 2024-03-19T01:25:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 412557600, 2024-03-19T01:25:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-03-19T01:25:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-03-19T01:25:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 8128, 2024-03-19T01:25:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 779.965744566, 2024-03-19T01:25:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.095960352 2024-03-19T01:25:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-03-19T01:25:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-03-19T01:25:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 63347, 2024-03-19T01:25:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 73182, 2024-03-19T01:25:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 9835, 2024-03-19T01:25:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 4136, 2024-03-19T01:25:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 4763, 2024-03-19T01:25:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 627, 2024-03-19T01:25:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 65290, 2024-03-19T01:25:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 75508, 2024-03-19T01:25:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 10218, 2024-03-19T01:25:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 46645, 2024-03-19T01:25:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 58538, 2024-03-19T01:25:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 11893, 2024-03-19T01:25:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 368152, 2024-03-19T01:25:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 231804 2024-03-19T01:25:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-03-19T01:25:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 68404, 2024-03-19T01:25:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 1000, 2024-03-19T01:25:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 63323, 2024-03-19T01:25:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 416, 2024-03-19T01:25:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 2804, 2024-03-19T01:25:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-03-19T01:25:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 52307, 2024-03-19T01:25:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 144.547424656, 2024-03-19T01:25:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002763443 2024-03-19T01:25:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-03-19T01:25:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 346, 2024-03-19T01:25:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.696898822, 2024-03-19T01:25:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002014158 2024-03-19T01:25:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-03-19T01:25:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-03-19T01:25:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-03-19T01:25:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 386, 2024-03-19T01:25:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.280322012, 2024-03-19T01:25:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.005907569 2024-03-19T01:25:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-03-19T01:25:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-03-19T01:25:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1, 2024-03-19T01:25:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000129603, 2024-03-19T01:25:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000129603 2024-03-19T01:25:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-03-19T01:25:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-03-19T01:25:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.277 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-03-19T01:25:54.277 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.277 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.277 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.278 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-03-19T01:25:54.278 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 15, 2024-03-19T01:25:54.278 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.008520909, 2024-03-19T01:25:54.278 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000568060 2024-03-19T01:25:54.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.278 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-03-19T01:25:54.279 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3183, 2024-03-19T01:25:54.279 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 9.815744105, 2024-03-19T01:25:54.279 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.003083802 2024-03-19T01:25:54.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.279 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-03-19T01:25:54.279 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.279 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.279 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.280 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-03-19T01:25:54.280 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 15, 2024-03-19T01:25:54.280 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.943377365, 2024-03-19T01:25:54.280 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.062891824 2024-03-19T01:25:54.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.280 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-03-19T01:25:54.280 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1145, 2024-03-19T01:25:54.280 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.631929304, 2024-03-19T01:25:54.280 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000551903 2024-03-19T01:25:54.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.281 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-03-19T01:25:54.281 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3763, 2024-03-19T01:25:54.281 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 17.477840221, 2024-03-19T01:25:54.281 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.004644655 2024-03-19T01:25:54.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.281 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-03-19T01:25:54.281 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 29, 2024-03-19T01:25:54.281 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.094998857, 2024-03-19T01:25:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.003275822 2024-03-19T01:25:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-03-19T01:25:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-03-19T01:25:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.283 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.283 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.283 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-03-19T01:25:54.283 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 124, 2024-03-19T01:25:54.283 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.945196661, 2024-03-19T01:25:54.283 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.007622553 2024-03-19T01:25:54.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.284 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-03-19T01:25:54.284 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 9, 2024-03-19T01:25:54.284 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.993609874, 2024-03-19T01:25:54.284 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.110401097 2024-03-19T01:25:54.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.284 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-03-19T01:25:54.284 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.284 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.284 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.285 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-03-19T01:25:54.285 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 108, 2024-03-19T01:25:54.285 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.241027240, 2024-03-19T01:25:54.285 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.020750252 2024-03-19T01:25:54.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.285 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-03-19T01:25:54.285 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.285 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.286 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.286 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-03-19T01:25:54.286 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.286 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.286 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.286 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-03-19T01:25:54.286 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.287 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.287 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.287 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-03-19T01:25:54.287 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 9, 2024-03-19T01:25:54.287 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.021945650, 2024-03-19T01:25:54.287 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002438405 2024-03-19T01:25:54.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.287 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-03-19T01:25:54.288 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 26, 2024-03-19T01:25:54.288 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.041387273, 2024-03-19T01:25:54.288 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001591818 2024-03-19T01:25:54.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.288 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-03-19T01:25:54.288 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1088, 2024-03-19T01:25:54.288 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.605485152, 2024-03-19T01:25:54.288 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002394747 2024-03-19T01:25:54.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.289 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-03-19T01:25:54.289 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-03-19T01:25:54.289 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-03-19T01:25:54.289 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.289 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.289 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.289 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:54.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.289 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-03-19T01:25:54.290 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 2, 2024-03-19T01:25:54.290 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 6, 2024-03-19T01:25:54.290 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 4, 2024-03-19T01:25:54.290 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 2, 2024-03-19T01:25:54.290 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-03-19T01:25:54.290 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 555, 2024-03-19T01:25:54.290 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 277, 2024-03-19T01:25:54.290 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 915, 2024-03-19T01:25:54.291 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-03-19T01:25:54.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.291 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-19T01:25:54.291 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:25:54.291 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-19T01:25:54.291 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:25:54.291 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:25:54.291 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:25:54.291 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:25:54.292 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:25:54.292 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:25:54.292 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:25:54.292 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:25:54.292 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:25:54.292 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:25:54.292 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:25:54.292 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:25:54.292 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:25:54.293 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-19T01:25:54.293 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:25:54.293 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-19T01:25:54.293 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:25:54.293 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:25:54.293 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:25:54.293 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:25:54.293 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:25:54.293 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:25:54.294 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:25:54.294 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-19T01:25:54.294 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:25:54.294 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:25:54.294 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:25:54.294 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-19T01:25:54.294 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-19T01:25:54.294 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-19T01:25:54.294 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:25:54.295 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:25:54.295 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:25:54.295 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:25:54.295 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 2109576, 2024-03-19T01:25:54.295 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 48625, 2024-03-19T01:25:54.295 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 704, 2024-03-19T01:25:54.295 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 8, 2024-03-19T01:25:54.295 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-19T01:25:54.295 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-19T01:25:54.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:25:54.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-19T01:25:54.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:25:54.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-19T01:25:54.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9832, 2024-03-19T01:25:54.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 205, 2024-03-19T01:25:54.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:25:54.296 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:25:54.297 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-19T01:25:54.297 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-19T01:25:54.297 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 225088612, 2024-03-19T01:25:54.297 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2766139, 2024-03-19T01:25:54.297 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-19T01:25:54.297 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-19T01:25:54.297 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-19T01:25:54.297 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-19T01:25:54.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.298 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-03-19T01:25:54.298 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-03-19T01:25:54.298 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-03-19T01:25:54.298 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 669535, 2024-03-19T01:25:54.298 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 476108066, 2024-03-19T01:25:54.298 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 37, 2024-03-19T01:25:54.298 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 669535, 2024-03-19T01:25:54.298 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-03-19T01:25:54.299 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 669535, 2024-03-19T01:25:54.299 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 18404.313008529, 2024-03-19T01:25:54.299 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.027488201 2024-03-19T01:25:54.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.299 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-03-19T01:25:54.299 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-03-19T01:25:54.299 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 669535, 2024-03-19T01:25:54.299 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 737940 2024-03-19T01:25:54.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.300 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 669535, 2024-03-19T01:25:54.300 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 612110, 2024-03-19T01:25:54.300 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 57425, 2024-03-19T01:25:54.300 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-03-19T01:25:54.300 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-03-19T01:25:54.300 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 209, 2024-03-19T01:25:54.300 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 42437, 2024-03-19T01:25:54.300 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 154, 2024-03-19T01:25:54.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 8262, 2024-03-19T01:25:54.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 155, 2024-03-19T01:25:54.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-03-19T01:25:54.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-03-19T01:25:54.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-03-19T01:25:54.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 115, 2024-03-19T01:25:54.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 2975, 2024-03-19T01:25:54.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-03-19T01:25:54.301 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-03-19T01:25:54.302 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-03-19T01:25:54.302 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 611821, 2024-03-19T01:25:54.302 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 64327, 2024-03-19T01:25:54.302 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-03-19T01:25:54.302 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-03-19T01:25:54.302 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-03-19T01:25:54.302 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-03-19T01:25:54.302 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-03-19T01:25:54.303 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-03-19T01:25:54.303 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T01:25:54.303 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-03-19T01:25:54.303 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-03-19T01:25:54.303 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 74, 2024-03-19T01:25:54.303 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-03-19T01:25:54.303 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-03-19T01:25:54.303 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-03-19T01:25:54.303 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-03-19T01:25:54.303 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-03-19T01:25:54.304 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-03-19T01:25:54.304 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-03-19T01:25:54.304 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-03-19T01:25:54.304 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-03-19T01:25:54.304 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-03-19T01:25:54.304 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-03-19T01:25:54.304 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-03-19T01:25:54.304 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-03-19T01:25:54.304 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-03-19T01:25:54.305 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-03-19T01:25:54.305 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-03-19T01:25:54.305 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 112, 2024-03-19T01:25:54.305 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-03-19T01:25:54.305 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 28, 2024-03-19T01:25:54.305 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 12, 2024-03-19T01:25:54.305 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 12, 2024-03-19T01:25:54.305 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-03-19T01:25:54.305 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-03-19T01:25:54.305 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 6716, 2024-03-19T01:25:54.306 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 336, 2024-03-19T01:25:54.306 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 359 2024-03-19T01:25:54.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.306 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-03-19T01:25:54.306 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-03-19T01:25:54.306 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-03-19T01:25:54.306 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-03-19T01:25:54.306 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-03-19T01:25:54.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.307 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-03-19T01:25:54.307 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 2 2024-03-19T01:25:54.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.307 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-03-19T01:25:54.307 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-03-19T01:25:54.307 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 10, 2024-03-19T01:25:54.307 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-03-19T01:25:54.307 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 9, 2024-03-19T01:25:54.308 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 2300, 2024-03-19T01:25:54.308 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 1212, 2024-03-19T01:25:54.308 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-03-19T01:25:54.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.308 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:25:54.308 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:25:54.308 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-19T01:25:54.308 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:25:54.308 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 845348, 2024-03-19T01:25:54.309 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 259686601, 2024-03-19T01:25:54.309 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:54.309 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 845348, 2024-03-19T01:25:54.309 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:25:54.309 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:25:54.309 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 845348, 2024-03-19T01:25:54.309 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 259686601, 2024-03-19T01:25:54.309 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:25:54.309 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.310 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.310 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.310 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:54.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.310 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-03-19T01:25:54.310 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:25:54.310 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-19T01:25:54.310 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:25:54.310 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:25:54.311 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:25:54.311 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:54.311 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:25:54.311 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 669535, 2024-03-19T01:25:54.311 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 476236686, 2024-03-19T01:25:54.311 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 669535, 2024-03-19T01:25:54.311 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 476236686, 2024-03-19T01:25:54.311 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:25:54.311 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.312 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.312 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.312 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:54.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.312 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-03-19T01:25:54.312 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:25:54.312 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-03-19T01:25:54.312 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:25:54.312 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:25:54.313 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:25:54.313 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:54.313 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:25:54.313 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 669535, 2024-03-19T01:25:54.313 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 669535, 2024-03-19T01:25:54.313 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 669535, 2024-03-19T01:25:54.313 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 669535, 2024-03-19T01:25:54.313 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:25:54.313 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.314 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.314 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.314 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:54.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.314 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-03-19T01:25:54.314 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:25:54.314 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-19T01:25:54.314 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:25:54.314 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 1212, 2024-03-19T01:25:54.315 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 131284, 2024-03-19T01:25:54.315 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:54.315 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 1212, 2024-03-19T01:25:54.315 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:25:54.315 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:25:54.315 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 134, 2024-03-19T01:25:54.315 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 131284, 2024-03-19T01:25:54.315 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:25:54.315 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.316 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.316 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.316 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:54.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:54.316 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x5621fc7fe500": { 2024-03-19T01:25:54.316 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:25:54.316 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-19T01:25:54.316 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:25:54.317 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 135902, 2024-03-19T01:25:54.317 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 408363296, 2024-03-19T01:25:54.317 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:54.317 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 135902, 2024-03-19T01:25:54.317 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:25:54.317 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:25:54.317 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 8128, 2024-03-19T01:25:54.317 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 408363296, 2024-03-19T01:25:54.317 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:25:54.317 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:54.318 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:54.318 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:54.318 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:54.318 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:54.318 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-19T01:25:54.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:54 smithi086 ceph-mon[66829]: pgmap v477: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-03-19T01:25:54.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:54 smithi069 ceph-mon[31727]: pgmap v477: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-03-19T01:25:54.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:54 smithi163 ceph-mon[67428]: pgmap v477: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-03-19T01:25:55.483 INFO:tasks.check_counter:Daemon mds.b mds.exported=375 2024-03-19T01:25:55.483 INFO:tasks.check_counter:Daemon mds.b mds.imported=66 2024-03-19T01:25:55.483 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-19T01:25:55.483 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=15 2024-03-19T01:25:55.483 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=9 2024-03-19T01:25:55.483 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=161 2024-03-19T01:25:55.483 DEBUG:tasks.check_counter:Getting stats from e 2024-03-19T01:25:55.484 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-19T01:25:55.773 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:25:56.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:56 smithi069 ceph-mon[31727]: pgmap v478: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-03-19T01:25:56.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:56 smithi086 ceph-mon[66829]: pgmap v478: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-03-19T01:25:56.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:56 smithi163 ceph-mon[67428]: pgmap v478: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-03-19T01:25:58.131 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-19T01:25:58.131 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:25:58.131 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 19423, 2024-03-19T01:25:58.131 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 21929, 2024-03-19T01:25:58.131 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 8455656, 2024-03-19T01:25:58.132 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 17703236, 2024-03-19T01:25:58.132 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 20, 2024-03-19T01:25:58.132 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-03-19T01:25:58.132 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 1.121773798, 2024-03-19T01:25:58.132 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.412426980, 2024-03-19T01:25:58.132 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.590491957, 2024-03-19T01:25:58.132 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.229951019, 2024-03-19T01:25:58.132 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:58.133 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 21929, 2024-03-19T01:25:58.133 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.132334983, 2024-03-19T01:25:58.133 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000051636 2024-03-19T01:25:58.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.133 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:58.133 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 11062, 2024-03-19T01:25:58.133 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.003260017, 2024-03-19T01:25:58.133 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000294 2024-03-19T01:25:58.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.133 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2189728, 2024-03-19T01:25:58.134 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 587520 2024-03-19T01:25:58.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.134 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:25:58.134 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 20800, 2024-03-19T01:25:58.134 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 21037, 2024-03-19T01:25:58.134 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 5407045, 2024-03-19T01:25:58.134 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 12055648, 2024-03-19T01:25:58.135 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 13, 2024-03-19T01:25:58.135 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 7, 2024-03-19T01:25:58.135 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 1.224284672, 2024-03-19T01:25:58.135 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.397324955, 2024-03-19T01:25:58.135 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.291134367, 2024-03-19T01:25:58.135 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.435141729, 2024-03-19T01:25:58.135 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:58.135 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 21034, 2024-03-19T01:25:58.135 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.662960921, 2024-03-19T01:25:58.136 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000031518 2024-03-19T01:25:58.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.136 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:58.136 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 689, 2024-03-19T01:25:58.136 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000162297, 2024-03-19T01:25:58.136 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000235 2024-03-19T01:25:58.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.136 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-19T01:25:58.137 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 567344 2024-03-19T01:25:58.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.137 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:25:58.137 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 19803, 2024-03-19T01:25:58.137 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 15758, 2024-03-19T01:25:58.137 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 9019481, 2024-03-19T01:25:58.137 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 7411316, 2024-03-19T01:25:58.137 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 40, 2024-03-19T01:25:58.137 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 9, 2024-03-19T01:25:58.138 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 1.117174857, 2024-03-19T01:25:58.138 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.423208770, 2024-03-19T01:25:58.138 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.570699811, 2024-03-19T01:25:58.138 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.194592450, 2024-03-19T01:25:58.138 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:25:58.138 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 15756, 2024-03-19T01:25:58.138 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.496980235, 2024-03-19T01:25:58.138 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000031542 2024-03-19T01:25:58.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.139 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:25:58.139 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 17541, 2024-03-19T01:25:58.139 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.002979358, 2024-03-19T01:25:58.139 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000169 2024-03-19T01:25:58.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.139 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T01:25:58.139 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T01:25:58.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.140 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-19T01:25:58.140 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-03-19T01:25:58.140 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-19T01:25:58.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.140 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-03-19T01:25:58.140 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-19T01:25:58.140 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-19T01:25:58.141 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 31125, 2024-03-19T01:25:58.141 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 7.351421399, 2024-03-19T01:25:58.141 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000236190 2024-03-19T01:25:58.141 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:58.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.141 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-03-19T01:25:58.141 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-19T01:25:58.141 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-19T01:25:58.142 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2, 2024-03-19T01:25:58.142 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000004974, 2024-03-19T01:25:58.142 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000002487 2024-03-19T01:25:58.142 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:58.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.142 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-03-19T01:25:58.142 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 930, 2024-03-19T01:25:58.142 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 728, 2024-03-19T01:25:58.143 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-03-19T01:25:58.143 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 728, 2024-03-19T01:25:58.143 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.307465866, 2024-03-19T01:25:58.143 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001795969 2024-03-19T01:25:58.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.143 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-03-19T01:25:58.143 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 202, 2024-03-19T01:25:58.143 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-03-19T01:25:58.143 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-03-19T01:25:58.144 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 29, 2024-03-19T01:25:58.144 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 2, 2024-03-19T01:25:58.144 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-03-19T01:25:58.144 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 3864, 2024-03-19T01:25:58.144 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 1, 2024-03-19T01:25:58.144 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 3863, 2024-03-19T01:25:58.144 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-03-19T01:25:58.144 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 3862, 2024-03-19T01:25:58.145 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 42, 2024-03-19T01:25:58.145 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 3222, 2024-03-19T01:25:58.145 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 3222, 2024-03-19T01:25:58.145 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 153, 2024-03-19T01:25:58.145 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 1256, 2024-03-19T01:25:58.145 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 790, 2024-03-19T01:25:58.145 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 179, 2024-03-19T01:25:58.145 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 178, 2024-03-19T01:25:58.145 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-03-19T01:25:58.146 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-03-19T01:25:58.146 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-03-19T01:25:58.146 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-03-19T01:25:58.146 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-03-19T01:25:58.146 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 4, 2024-03-19T01:25:58.146 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 42, 2024-03-19T01:25:58.146 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 158, 2024-03-19T01:25:58.146 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 2978, 2024-03-19T01:25:58.146 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-03-19T01:25:58.147 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 38405, 2024-03-19T01:25:58.147 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-03-19T01:25:58.147 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 89514, 2024-03-19T01:25:58.147 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 1636611686, 2024-03-19T01:25:58.147 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 6, 2024-03-19T01:25:58.147 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 38405, 2024-03-19T01:25:58.147 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-03-19T01:25:58.147 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T01:25:58.147 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-03-19T01:25:58.148 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 2048, 2024-03-19T01:25:58.148 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T01:25:58.148 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 2048, 2024-03-19T01:25:58.148 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 36357, 2024-03-19T01:25:58.148 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-03-19T01:25:58.148 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 518, 2024-03-19T01:25:58.148 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 546, 2024-03-19T01:25:58.148 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T01:25:58.148 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T01:25:58.149 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T01:25:58.149 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 4220, 2024-03-19T01:25:58.149 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 3790, 2024-03-19T01:25:58.149 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 45, 2024-03-19T01:25:58.149 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 280 2024-03-19T01:25:58.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.149 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-03-19T01:25:58.149 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-03-19T01:25:58.149 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-03-19T01:25:58.149 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-03-19T01:25:58.150 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 191, 2024-03-19T01:25:58.150 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 9, 2024-03-19T01:25:58.150 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-03-19T01:25:58.150 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-03-19T01:25:58.150 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-03-19T01:25:58.150 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-03-19T01:25:58.150 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-03-19T01:25:58.150 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-03-19T01:25:58.151 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-03-19T01:25:58.151 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-03-19T01:25:58.151 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-03-19T01:25:58.151 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-03-19T01:25:58.151 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-03-19T01:25:58.151 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-03-19T01:25:58.151 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-03-19T01:25:58.151 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-03-19T01:25:58.151 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T01:25:58.151 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 4, 2024-03-19T01:25:58.152 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-03-19T01:25:58.152 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 3, 2024-03-19T01:25:58.152 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-03-19T01:25:58.152 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-03-19T01:25:58.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.152 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-03-19T01:25:58.152 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-03-19T01:25:58.152 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 5221, 2024-03-19T01:25:58.152 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-03-19T01:25:58.153 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-03-19T01:25:58.153 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 5221, 2024-03-19T01:25:58.153 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-03-19T01:25:58.153 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-03-19T01:25:58.153 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 8, 2024-03-19T01:25:58.153 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-03-19T01:25:58.153 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-03-19T01:25:58.153 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 8, 2024-03-19T01:25:58.153 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-03-19T01:25:58.153 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-03-19T01:25:58.154 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-03-19T01:25:58.154 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-03-19T01:25:58.154 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 18690131, 2024-03-19T01:25:58.154 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-03-19T01:25:58.154 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-03-19T01:25:58.154 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 473, 2024-03-19T01:25:58.154 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 20.625049323, 2024-03-19T01:25:58.154 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.043604755 2024-03-19T01:25:58.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.155 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-03-19T01:25:58.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.155 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-03-19T01:25:58.155 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3865, 2024-03-19T01:25:58.155 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 3907, 2024-03-19T01:25:58.155 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 42, 2024-03-19T01:25:58.155 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 574, 2024-03-19T01:25:58.155 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 603, 2024-03-19T01:25:58.155 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 29, 2024-03-19T01:25:58.156 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 3864, 2024-03-19T01:25:58.156 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 3920, 2024-03-19T01:25:58.156 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 56, 2024-03-19T01:25:58.156 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 3222, 2024-03-19T01:25:58.156 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 4756, 2024-03-19T01:25:58.156 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 1534, 2024-03-19T01:25:58.156 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 80752, 2024-03-19T01:25:58.156 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 223612 2024-03-19T01:25:58.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.157 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-03-19T01:25:58.157 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 1100, 2024-03-19T01:25:58.157 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 14, 2024-03-19T01:25:58.157 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 930, 2024-03-19T01:25:58.157 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 16, 2024-03-19T01:25:58.157 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 54, 2024-03-19T01:25:58.157 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-03-19T01:25:58.157 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 278, 2024-03-19T01:25:58.157 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.341499390, 2024-03-19T01:25:58.157 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001228415 2024-03-19T01:25:58.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.158 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-03-19T01:25:58.158 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.158 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.158 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.158 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-03-19T01:25:58.158 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.158 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.159 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.159 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-03-19T01:25:58.159 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.159 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.159 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.159 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-03-19T01:25:58.159 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 113, 2024-03-19T01:25:58.160 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.039032544, 2024-03-19T01:25:58.160 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000345420 2024-03-19T01:25:58.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.160 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-03-19T01:25:58.160 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.160 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.160 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.160 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-03-19T01:25:58.161 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.161 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.161 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.161 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-03-19T01:25:58.161 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.161 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.161 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.162 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-03-19T01:25:58.162 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.162 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.162 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.162 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-03-19T01:25:58.162 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-03-19T01:25:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.164 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-03-19T01:25:58.164 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 17, 2024-03-19T01:25:58.164 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.019291548, 2024-03-19T01:25:58.164 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001134796 2024-03-19T01:25:58.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.164 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-03-19T01:25:58.164 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.164 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.165 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.165 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-03-19T01:25:58.165 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.165 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.165 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.165 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-03-19T01:25:58.165 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 22, 2024-03-19T01:25:58.166 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.009243715, 2024-03-19T01:25:58.166 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000420168 2024-03-19T01:25:58.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.166 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-03-19T01:25:58.166 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 297, 2024-03-19T01:25:58.166 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.888193846, 2024-03-19T01:25:58.166 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002990551 2024-03-19T01:25:58.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.167 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-03-19T01:25:58.167 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.167 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.167 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.167 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-03-19T01:25:58.167 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.167 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.168 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.168 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-03-19T01:25:58.168 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.168 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.168 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.168 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-03-19T01:25:58.169 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.169 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.169 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.169 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-03-19T01:25:58.169 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.169 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.169 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.170 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-03-19T01:25:58.170 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.170 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.170 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.170 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-03-19T01:25:58.171 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1, 2024-03-19T01:25:58.171 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.010204823, 2024-03-19T01:25:58.171 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.010204823 2024-03-19T01:25:58.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.171 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-03-19T01:25:58.171 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.171 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.171 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.172 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-03-19T01:25:58.172 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.172 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.172 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.172 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-03-19T01:25:58.172 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.172 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.172 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.173 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-03-19T01:25:58.173 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.173 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.173 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.173 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-03-19T01:25:58.173 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.174 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.174 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.174 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-03-19T01:25:58.174 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.174 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.174 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.174 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-03-19T01:25:58.175 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-03-19T01:25:58.175 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-03-19T01:25:58.175 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.175 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.175 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.175 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:58.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.175 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-03-19T01:25:58.175 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-03-19T01:25:58.176 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 1, 2024-03-19T01:25:58.176 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 0, 2024-03-19T01:25:58.176 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-03-19T01:25:58.176 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-03-19T01:25:58.176 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 18, 2024-03-19T01:25:58.176 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 18, 2024-03-19T01:25:58.176 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 4791, 2024-03-19T01:25:58.176 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-03-19T01:25:58.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.177 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-19T01:25:58.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:25:58.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-19T01:25:58.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:25:58.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:25:58.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:25:58.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:25:58.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:25:58.177 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:25:58.178 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:25:58.178 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:25:58.178 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:25:58.178 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:25:58.178 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:25:58.178 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:25:58.178 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:25:58.178 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-19T01:25:58.179 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:25:58.179 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-19T01:25:58.179 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:25:58.179 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:25:58.179 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:25:58.179 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:25:58.179 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:25:58.179 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:25:58.179 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:25:58.180 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-19T01:25:58.180 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:25:58.180 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:25:58.180 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:25:58.180 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-19T01:25:58.180 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-19T01:25:58.180 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-19T01:25:58.180 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:25:58.180 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:25:58.181 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:25:58.181 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:25:58.181 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 430812, 2024-03-19T01:25:58.181 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 3622, 2024-03-19T01:25:58.181 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 1760, 2024-03-19T01:25:58.181 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 20, 2024-03-19T01:25:58.181 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-19T01:25:58.181 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-19T01:25:58.181 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:25:58.181 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-19T01:25:58.182 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:25:58.182 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-19T01:25:58.182 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9832, 2024-03-19T01:25:58.182 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 205, 2024-03-19T01:25:58.182 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:25:58.182 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:25:58.182 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-19T01:25:58.182 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-19T01:25:58.183 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 14154967, 2024-03-19T01:25:58.183 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 164504, 2024-03-19T01:25:58.183 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-19T01:25:58.183 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-19T01:25:58.183 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-19T01:25:58.183 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-19T01:25:58.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.183 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-03-19T01:25:58.183 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-03-19T01:25:58.184 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-03-19T01:25:58.184 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 39142, 2024-03-19T01:25:58.184 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 15314803, 2024-03-19T01:25:58.184 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 33, 2024-03-19T01:25:58.184 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 39142, 2024-03-19T01:25:58.184 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-03-19T01:25:58.184 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 39142, 2024-03-19T01:25:58.184 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 65.958601476, 2024-03-19T01:25:58.184 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001685110 2024-03-19T01:25:58.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.185 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-03-19T01:25:58.185 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-03-19T01:25:58.185 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 39142, 2024-03-19T01:25:58.185 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 39381 2024-03-19T01:25:58.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.185 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 39142, 2024-03-19T01:25:58.185 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 38405, 2024-03-19T01:25:58.186 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 737, 2024-03-19T01:25:58.186 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-03-19T01:25:58.186 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-03-19T01:25:58.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 0, 2024-03-19T01:25:58.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 10, 2024-03-19T01:25:58.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 0, 2024-03-19T01:25:58.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 473, 2024-03-19T01:25:58.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 35, 2024-03-19T01:25:58.187 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-03-19T01:25:58.187 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-03-19T01:25:58.187 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-03-19T01:25:58.187 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 2, 2024-03-19T01:25:58.187 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 9, 2024-03-19T01:25:58.187 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-03-19T01:25:58.188 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-03-19T01:25:58.188 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-03-19T01:25:58.188 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 38405, 2024-03-19T01:25:58.188 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 20, 2024-03-19T01:25:58.188 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-03-19T01:25:58.188 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-03-19T01:25:58.188 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-03-19T01:25:58.188 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-03-19T01:25:58.189 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-03-19T01:25:58.189 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-03-19T01:25:58.189 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T01:25:58.189 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-03-19T01:25:58.189 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-03-19T01:25:58.189 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-03-19T01:25:58.189 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-03-19T01:25:58.189 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-03-19T01:25:58.189 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-03-19T01:25:58.190 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-03-19T01:25:58.190 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-03-19T01:25:58.190 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-03-19T01:25:58.190 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-03-19T01:25:58.190 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-03-19T01:25:58.190 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-03-19T01:25:58.190 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-03-19T01:25:58.190 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-03-19T01:25:58.190 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-03-19T01:25:58.191 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-03-19T01:25:58.191 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-03-19T01:25:58.191 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-03-19T01:25:58.191 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-03-19T01:25:58.191 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 112, 2024-03-19T01:25:58.191 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-03-19T01:25:58.191 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 23, 2024-03-19T01:25:58.191 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 12, 2024-03-19T01:25:58.191 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 12, 2024-03-19T01:25:58.192 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-03-19T01:25:58.192 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-03-19T01:25:58.192 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 329, 2024-03-19T01:25:58.192 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 0, 2024-03-19T01:25:58.192 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 98 2024-03-19T01:25:58.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.192 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-03-19T01:25:58.192 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-03-19T01:25:58.192 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-03-19T01:25:58.193 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-03-19T01:25:58.193 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-03-19T01:25:58.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.193 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-03-19T01:25:58.193 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-03-19T01:25:58.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.193 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-03-19T01:25:58.193 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-03-19T01:25:58.193 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T01:25:58.194 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-03-19T01:25:58.194 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-03-19T01:25:58.194 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-03-19T01:25:58.194 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-03-19T01:25:58.194 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-03-19T01:25:58.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.194 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:25:58.194 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:25:58.194 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-19T01:25:58.195 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:25:58.195 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 60022, 2024-03-19T01:25:58.195 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 17529848, 2024-03-19T01:25:58.195 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:58.195 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 60022, 2024-03-19T01:25:58.195 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:25:58.195 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:25:58.195 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 60022, 2024-03-19T01:25:58.196 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 17529848, 2024-03-19T01:25:58.197 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:25:58.197 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.197 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.197 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.198 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:58.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.198 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-03-19T01:25:58.198 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:25:58.198 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-19T01:25:58.198 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:25:58.198 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:25:58.198 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:25:58.199 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:58.199 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:25:58.199 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 39142, 2024-03-19T01:25:58.199 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 15314803, 2024-03-19T01:25:58.199 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 39142, 2024-03-19T01:25:58.199 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 15314803, 2024-03-19T01:25:58.199 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:25:58.199 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.199 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.200 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.200 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:58.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.200 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-03-19T01:25:58.200 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:25:58.200 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-03-19T01:25:58.200 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:25:58.200 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:25:58.200 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:25:58.201 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:58.201 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:25:58.201 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 39142, 2024-03-19T01:25:58.201 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 39142, 2024-03-19T01:25:58.201 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 39142, 2024-03-19T01:25:58.201 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 39142, 2024-03-19T01:25:58.201 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:25:58.201 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.201 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.202 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.202 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:58.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.202 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-03-19T01:25:58.202 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:25:58.202 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-19T01:25:58.202 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:25:58.202 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:25:58.203 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:25:58.203 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:58.203 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:25:58.203 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:25:58.203 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:25:58.203 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-03-19T01:25:58.203 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-03-19T01:25:58.203 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:25:58.203 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.203 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.204 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.204 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:58.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:25:58.204 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55df0a604500": { 2024-03-19T01:25:58.204 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:25:58.204 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-19T01:25:58.204 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:25:58.204 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 5221, 2024-03-19T01:25:58.204 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 14495827, 2024-03-19T01:25:58.205 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:25:58.205 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 5221, 2024-03-19T01:25:58.205 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:25:58.205 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:25:58.205 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 473, 2024-03-19T01:25:58.205 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 14495827, 2024-03-19T01:25:58.205 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:25:58.205 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:25:58.205 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:25:58.205 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:25:58.206 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:58.206 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:25:58.206 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-19T01:25:58.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:25:58 smithi086 ceph-mon[66829]: pgmap v479: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-03-19T01:25:58.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:25:58 smithi069 ceph-mon[31727]: pgmap v479: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-03-19T01:25:58.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:25:58 smithi163 ceph-mon[67428]: pgmap v479: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-03-19T01:25:58.824 INFO:tasks.check_counter:Daemon mds.e mds.exported=4 2024-03-19T01:25:58.825 INFO:tasks.check_counter:Daemon mds.e mds.imported=158 2024-03-19T01:25:58.825 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-03-19T01:25:58.825 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-19T01:25:58.825 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-19T01:25:58.825 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=2 2024-03-19T01:25:58.825 DEBUG:tasks.check_counter:Getting stats from h 2024-03-19T01:25:58.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-19T01:25:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:26:00.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:00 smithi069 ceph-mon[31727]: pgmap v480: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-03-19T01:26:00.541 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:00 smithi086 ceph-mon[66829]: pgmap v480: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-03-19T01:26:00.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:00 smithi163 ceph-mon[67428]: pgmap v480: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-03-19T01:26:01.892 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-19T01:26:01.892 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:26:01.892 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2813, 2024-03-19T01:26:01.892 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2815, 2024-03-19T01:26:01.892 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 20634123, 2024-03-19T01:26:01.893 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 894541, 2024-03-19T01:26:01.893 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 7, 2024-03-19T01:26:01.893 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-03-19T01:26:01.893 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.183978845, 2024-03-19T01:26:01.893 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.054141680, 2024-03-19T01:26:01.893 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.054517249, 2024-03-19T01:26:01.894 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.061775325, 2024-03-19T01:26:01.894 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:01.894 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2813, 2024-03-19T01:26:01.894 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.059321810, 2024-03-19T01:26:01.894 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000021088 2024-03-19T01:26:01.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.894 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:01.895 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.895 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.895 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.895 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T01:26:01.895 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T01:26:01.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.895 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:26:01.896 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 739, 2024-03-19T01:26:01.896 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 690, 2024-03-19T01:26:01.896 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 10636896, 2024-03-19T01:26:01.896 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 307028, 2024-03-19T01:26:01.896 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-03-19T01:26:01.896 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-03-19T01:26:01.896 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.075751323, 2024-03-19T01:26:01.896 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.020432931, 2024-03-19T01:26:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.052902056, 2024-03-19T01:26:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.013520878, 2024-03-19T01:26:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 689, 2024-03-19T01:26:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.021398466, 2024-03-19T01:26:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000031057 2024-03-19T01:26:01.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.898 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:01.898 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.898 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.898 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.898 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2147792, 2024-03-19T01:26:01.898 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 193440 2024-03-19T01:26:01.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.899 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:26:01.899 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 164, 2024-03-19T01:26:01.899 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 393, 2024-03-19T01:26:01.899 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 4240236, 2024-03-19T01:26:01.899 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 703830, 2024-03-19T01:26:01.899 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 9, 2024-03-19T01:26:01.900 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-03-19T01:26:01.900 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.024826906, 2024-03-19T01:26:01.900 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.010280440, 2024-03-19T01:26:01.900 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.008209566, 2024-03-19T01:26:01.900 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.003748380, 2024-03-19T01:26:01.900 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:01.900 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 391, 2024-03-19T01:26:01.901 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.035588413, 2024-03-19T01:26:01.901 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000091018 2024-03-19T01:26:01.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.901 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:01.901 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.901 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.901 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.902 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-03-19T01:26:01.902 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 653904 2024-03-19T01:26:01.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.902 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-19T01:26:01.902 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-03-19T01:26:01.902 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-19T01:26:01.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.903 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-03-19T01:26:01.903 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-19T01:26:01.903 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-19T01:26:01.903 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5714, 2024-03-19T01:26:01.903 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.127231248, 2024-03-19T01:26:01.903 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000022266 2024-03-19T01:26:01.903 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:01.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.904 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-03-19T01:26:01.904 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-19T01:26:01.904 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-19T01:26:01.904 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.904 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.904 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.904 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:01.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.905 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-03-19T01:26:01.905 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-03-19T01:26:01.905 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-03-19T01:26:01.905 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-03-19T01:26:01.905 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.905 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.906 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.906 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-03-19T01:26:01.906 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-03-19T01:26:01.906 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-03-19T01:26:01.906 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-03-19T01:26:01.906 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-03-19T01:26:01.907 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-03-19T01:26:01.907 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-03-19T01:26:01.907 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 6629, 2024-03-19T01:26:01.907 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-03-19T01:26:01.907 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 6629, 2024-03-19T01:26:01.907 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-03-19T01:26:01.907 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 6629, 2024-03-19T01:26:01.907 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-03-19T01:26:01.908 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-03-19T01:26:01.908 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-03-19T01:26:01.908 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 202, 2024-03-19T01:26:01.908 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-03-19T01:26:01.908 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-03-19T01:26:01.908 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-03-19T01:26:01.908 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-03-19T01:26:01.908 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-03-19T01:26:01.909 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-03-19T01:26:01.909 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-03-19T01:26:01.909 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-03-19T01:26:01.909 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-03-19T01:26:01.909 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-03-19T01:26:01.909 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-03-19T01:26:01.909 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-03-19T01:26:01.909 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-03-19T01:26:01.910 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-03-19T01:26:01.910 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-03-19T01:26:01.910 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-03-19T01:26:01.910 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-03-19T01:26:01.910 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-03-19T01:26:01.910 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-03-19T01:26:01.910 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T01:26:01.910 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-03-19T01:26:01.911 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T01:26:01.911 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-03-19T01:26:01.911 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-03-19T01:26:01.911 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T01:26:01.911 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T01:26:01.911 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-03-19T01:26:01.911 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-03-19T01:26:01.912 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T01:26:01.912 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-03-19T01:26:01.912 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T01:26:01.912 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T01:26:01.912 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T01:26:01.912 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-03-19T01:26:01.912 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-03-19T01:26:01.913 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-03-19T01:26:01.913 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-03-19T01:26:01.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.913 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-03-19T01:26:01.913 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-03-19T01:26:01.913 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-03-19T01:26:01.913 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-03-19T01:26:01.913 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-03-19T01:26:01.914 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-03-19T01:26:01.914 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-03-19T01:26:01.914 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-03-19T01:26:01.914 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-03-19T01:26:01.914 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-03-19T01:26:01.914 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-03-19T01:26:01.914 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-03-19T01:26:01.914 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-03-19T01:26:01.915 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-03-19T01:26:01.915 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-03-19T01:26:01.915 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-03-19T01:26:01.915 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-03-19T01:26:01.915 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-03-19T01:26:01.915 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-03-19T01:26:01.915 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-03-19T01:26:01.915 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T01:26:01.916 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-03-19T01:26:01.916 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-03-19T01:26:01.916 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-03-19T01:26:01.916 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-03-19T01:26:01.916 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-03-19T01:26:01.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.916 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-03-19T01:26:01.916 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-03-19T01:26:01.916 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-03-19T01:26:01.917 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-03-19T01:26:01.917 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-03-19T01:26:01.917 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 11994, 2024-03-19T01:26:01.917 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-03-19T01:26:01.917 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-03-19T01:26:01.917 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-03-19T01:26:01.917 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-03-19T01:26:01.917 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-03-19T01:26:01.917 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 17, 2024-03-19T01:26:01.918 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-03-19T01:26:01.918 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-03-19T01:26:01.918 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-03-19T01:26:01.918 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-03-19T01:26:01.918 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 36608120, 2024-03-19T01:26:01.918 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 36603359, 2024-03-19T01:26:01.918 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-03-19T01:26:01.918 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.919 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.919 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.919 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 11994 2024-03-19T01:26:01.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.919 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-03-19T01:26:01.919 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 6372, 2024-03-19T01:26:01.920 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 7132, 2024-03-19T01:26:01.920 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 760, 2024-03-19T01:26:01.920 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 736, 2024-03-19T01:26:01.920 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 779, 2024-03-19T01:26:01.920 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 43, 2024-03-19T01:26:01.920 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 6629, 2024-03-19T01:26:01.920 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 7260, 2024-03-19T01:26:01.920 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 631, 2024-03-19T01:26:01.921 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-03-19T01:26:01.921 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-03-19T01:26:01.921 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-03-19T01:26:01.921 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 78208, 2024-03-19T01:26:01.921 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215420 2024-03-19T01:26:01.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.921 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-03-19T01:26:01.922 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-03-19T01:26:01.922 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-03-19T01:26:01.922 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-03-19T01:26:01.922 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-03-19T01:26:01.922 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-03-19T01:26:01.922 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-03-19T01:26:01.922 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.923 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.923 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.923 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-03-19T01:26:01.923 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.923 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.923 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.924 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-03-19T01:26:01.924 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.924 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.924 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.924 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-03-19T01:26:01.925 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.925 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.925 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.925 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-03-19T01:26:01.925 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.925 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.925 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.926 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-03-19T01:26:01.926 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.926 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.926 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.926 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-03-19T01:26:01.926 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.927 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.927 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.927 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-03-19T01:26:01.927 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.927 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.927 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.928 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-03-19T01:26:01.928 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.928 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.928 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.928 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-03-19T01:26:01.929 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.929 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.929 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.929 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-03-19T01:26:01.929 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.929 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.930 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.930 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-03-19T01:26:01.931 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.931 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.931 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.931 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-03-19T01:26:01.931 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.932 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.932 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.932 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-03-19T01:26:01.932 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.932 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.932 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.933 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-03-19T01:26:01.933 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.933 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.933 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.933 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-03-19T01:26:01.933 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.933 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.934 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.934 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-03-19T01:26:01.934 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.934 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.934 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.935 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-03-19T01:26:01.935 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.935 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.935 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.936 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-03-19T01:26:01.936 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.936 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.936 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.936 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-03-19T01:26:01.937 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.937 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.937 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.937 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-03-19T01:26:01.937 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.937 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.937 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.938 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-03-19T01:26:01.938 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.938 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.938 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.938 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-03-19T01:26:01.938 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.939 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.939 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.939 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-03-19T01:26:01.939 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.939 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.939 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.940 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-03-19T01:26:01.940 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.940 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.940 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.940 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-03-19T01:26:01.940 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.941 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.941 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.941 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-03-19T01:26:01.941 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.941 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.941 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.942 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-03-19T01:26:01.942 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.942 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.942 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.942 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-03-19T01:26:01.943 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.943 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.943 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.943 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-03-19T01:26:01.943 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-03-19T01:26:01.943 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-03-19T01:26:01.944 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.944 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.944 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.944 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:01.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.944 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-03-19T01:26:01.944 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-03-19T01:26:01.945 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 2, 2024-03-19T01:26:01.945 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 1, 2024-03-19T01:26:01.945 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 2, 2024-03-19T01:26:01.945 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-03-19T01:26:01.945 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-03-19T01:26:01.945 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-03-19T01:26:01.945 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 8671, 2024-03-19T01:26:01.945 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-03-19T01:26:01.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.946 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-19T01:26:01.946 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:26:01.946 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-19T01:26:01.946 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:26:01.946 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:26:01.946 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:26:01.947 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:26:01.947 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:26:01.947 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:26:01.947 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:26:01.947 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:26:01.947 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:26:01.947 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:26:01.948 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:26:01.948 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:26:01.948 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:26:01.948 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-19T01:26:01.948 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:26:01.948 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-19T01:26:01.948 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:26:01.949 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:26:01.949 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:26:01.949 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:26:01.949 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:26:01.949 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:26:01.949 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:26:01.949 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-19T01:26:01.950 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:26:01.950 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:26:01.950 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:26:01.950 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-19T01:26:01.950 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-19T01:26:01.950 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-19T01:26:01.950 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:26:01.951 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:26:01.951 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:26:01.951 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:26:01.951 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 485848, 2024-03-19T01:26:01.951 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 7577, 2024-03-19T01:26:01.951 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-03-19T01:26:01.951 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-03-19T01:26:01.952 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-19T01:26:01.952 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-19T01:26:01.952 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:26:01.952 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-19T01:26:01.952 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:26:01.952 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-19T01:26:01.952 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9832, 2024-03-19T01:26:01.952 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 205, 2024-03-19T01:26:01.953 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:26:01.953 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:26:01.953 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-19T01:26:01.953 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-19T01:26:01.953 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 21466959, 2024-03-19T01:26:01.953 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 253576, 2024-03-19T01:26:01.953 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-19T01:26:01.954 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-19T01:26:01.954 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-19T01:26:01.954 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-19T01:26:01.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.954 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-03-19T01:26:01.954 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-03-19T01:26:01.955 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-03-19T01:26:01.955 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 3327, 2024-03-19T01:26:01.955 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-03-19T01:26:01.955 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-03-19T01:26:01.955 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 3327, 2024-03-19T01:26:01.955 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-03-19T01:26:01.955 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3327, 2024-03-19T01:26:01.955 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 14.755686807, 2024-03-19T01:26:01.956 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.004435132 2024-03-19T01:26:01.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.956 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-03-19T01:26:01.956 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-03-19T01:26:01.956 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3327, 2024-03-19T01:26:01.956 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3328 2024-03-19T01:26:01.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.956 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 3327, 2024-03-19T01:26:01.957 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 3327, 2024-03-19T01:26:01.957 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-03-19T01:26:01.957 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-03-19T01:26:01.957 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-03-19T01:26:01.958 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 1989, 2024-03-19T01:26:01.958 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-03-19T01:26:01.958 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 1336, 2024-03-19T01:26:01.958 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-03-19T01:26:01.958 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-03-19T01:26:01.958 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-03-19T01:26:01.958 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-03-19T01:26:01.959 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-03-19T01:26:01.959 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-03-19T01:26:01.959 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-03-19T01:26:01.959 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-03-19T01:26:01.959 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-03-19T01:26:01.959 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-03-19T01:26:01.960 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-03-19T01:26:01.960 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-03-19T01:26:01.960 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-03-19T01:26:01.960 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-03-19T01:26:01.960 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-03-19T01:26:01.960 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-03-19T01:26:01.960 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-03-19T01:26:01.961 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-03-19T01:26:01.961 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T01:26:01.961 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-03-19T01:26:01.961 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-03-19T01:26:01.961 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-03-19T01:26:01.962 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-03-19T01:26:01.962 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-03-19T01:26:01.962 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-03-19T01:26:01.962 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-03-19T01:26:01.962 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-03-19T01:26:01.962 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-03-19T01:26:01.962 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-03-19T01:26:01.963 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-03-19T01:26:01.963 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-03-19T01:26:01.963 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-03-19T01:26:01.963 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-03-19T01:26:01.963 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-03-19T01:26:01.964 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-03-19T01:26:01.964 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-03-19T01:26:01.964 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-03-19T01:26:01.964 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-03-19T01:26:01.964 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 112, 2024-03-19T01:26:01.964 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-03-19T01:26:01.965 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 23, 2024-03-19T01:26:01.965 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 7, 2024-03-19T01:26:01.965 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 8, 2024-03-19T01:26:01.965 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 1, 2024-03-19T01:26:01.965 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-03-19T01:26:01.965 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-03-19T01:26:01.966 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-03-19T01:26:01.966 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-03-19T01:26:01.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.966 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-03-19T01:26:01.966 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-03-19T01:26:01.966 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-03-19T01:26:01.967 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-03-19T01:26:01.967 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-03-19T01:26:01.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.967 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-03-19T01:26:01.967 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-03-19T01:26:01.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.968 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-03-19T01:26:01.968 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-03-19T01:26:01.968 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T01:26:01.968 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-03-19T01:26:01.969 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-03-19T01:26:01.969 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-03-19T01:26:01.969 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-03-19T01:26:01.969 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-03-19T01:26:01.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.969 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:26:01.970 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:26:01.970 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-19T01:26:01.970 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:26:01.970 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 3712, 2024-03-19T01:26:01.970 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 35154763, 2024-03-19T01:26:01.970 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:01.970 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 3712, 2024-03-19T01:26:01.971 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:26:01.971 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:26:01.971 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3712, 2024-03-19T01:26:01.971 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 35154763, 2024-03-19T01:26:01.971 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:26:01.971 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.971 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.971 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.972 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:01.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.972 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-03-19T01:26:01.972 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:26:01.972 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-19T01:26:01.972 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:26:01.972 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:26:01.973 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:26:01.973 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:01.973 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:01.973 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3327, 2024-03-19T01:26:01.973 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 32425559, 2024-03-19T01:26:01.973 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3327, 2024-03-19T01:26:01.973 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 32425559, 2024-03-19T01:26:01.974 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:26:01.974 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.974 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.974 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.974 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:01.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.974 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-03-19T01:26:01.975 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:26:01.975 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-03-19T01:26:01.975 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:26:01.975 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:26:01.975 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:26:01.975 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:01.975 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:01.975 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3327, 2024-03-19T01:26:01.976 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 3327, 2024-03-19T01:26:01.976 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3327, 2024-03-19T01:26:01.976 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 3327, 2024-03-19T01:26:01.976 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:26:01.976 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.976 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.976 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:01.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.977 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-03-19T01:26:01.977 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:26:01.977 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-19T01:26:01.977 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:26:01.977 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:26:01.977 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:26:01.978 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:01.978 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:01.978 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:26:01.978 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:26:01.978 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-03-19T01:26:01.978 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-03-19T01:26:01.978 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:26:01.978 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.979 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.979 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:01.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:01.979 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55c5d1b366e0": { 2024-03-19T01:26:01.979 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:26:01.979 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-19T01:26:01.979 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:26:01.980 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:26:01.980 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:26:01.980 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:01.980 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:01.980 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:26:01.980 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:26:01.980 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-03-19T01:26:01.980 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-03-19T01:26:01.981 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:26:01.981 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:01.981 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:01.981 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:01.981 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:01.981 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:01.981 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-19T01:26:02.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:02 smithi086 ceph-mon[66829]: pgmap v481: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-03-19T01:26:02.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:02 smithi069 ceph-mon[31727]: pgmap v481: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-03-19T01:26:02.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:02 smithi163 ceph-mon[67428]: pgmap v481: 129 pgs: 129 active+clean; 2.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-03-19T01:26:02.656 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-19T01:26:02.656 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-19T01:26:02.656 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-03-19T01:26:02.656 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-19T01:26:02.656 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-19T01:26:02.656 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-03-19T01:26:02.656 DEBUG:tasks.check_counter:Getting stats from k 2024-03-19T01:26:02.657 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-19T01:26:02.959 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:26:04.465 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:04 smithi086 ceph-mon[66829]: pgmap v482: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.2 MiB/s wr, 41 op/s 2024-03-19T01:26:04.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:04 smithi069 ceph-mon[31727]: pgmap v482: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.2 MiB/s wr, 41 op/s 2024-03-19T01:26:04.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:04 smithi163 ceph-mon[67428]: pgmap v482: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.2 MiB/s wr, 41 op/s 2024-03-19T01:26:04.741 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-19T01:26:04.741 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:26:04.741 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1958, 2024-03-19T01:26:04.742 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1914, 2024-03-19T01:26:04.742 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 2608726, 2024-03-19T01:26:04.742 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 697988, 2024-03-19T01:26:04.742 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-03-19T01:26:04.742 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-03-19T01:26:04.742 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.163504814, 2024-03-19T01:26:04.742 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.049562466, 2024-03-19T01:26:04.742 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.073961764, 2024-03-19T01:26:04.743 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.044190806, 2024-03-19T01:26:04.743 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:04.743 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1913, 2024-03-19T01:26:04.743 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.056181068, 2024-03-19T01:26:04.743 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000029368 2024-03-19T01:26:04.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.743 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:04.744 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.744 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.744 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.744 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2125312, 2024-03-19T01:26:04.744 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 197456 2024-03-19T01:26:04.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.744 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:26:04.744 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 904, 2024-03-19T01:26:04.745 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1138, 2024-03-19T01:26:04.745 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 6614730, 2024-03-19T01:26:04.745 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 853194, 2024-03-19T01:26:04.745 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 7, 2024-03-19T01:26:04.745 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-03-19T01:26:04.745 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.069144898, 2024-03-19T01:26:04.745 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.023362724, 2024-03-19T01:26:04.745 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.020044818, 2024-03-19T01:26:04.745 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.018931336, 2024-03-19T01:26:04.746 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:04.746 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1135, 2024-03-19T01:26:04.746 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.039492541, 2024-03-19T01:26:04.746 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000034795 2024-03-19T01:26:04.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.746 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:04.746 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.746 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.747 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.747 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-19T01:26:04.747 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 566608 2024-03-19T01:26:04.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.747 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:26:04.747 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 653, 2024-03-19T01:26:04.747 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 652, 2024-03-19T01:26:04.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 8559227, 2024-03-19T01:26:04.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 206513, 2024-03-19T01:26:04.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-03-19T01:26:04.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-03-19T01:26:04.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.047280956, 2024-03-19T01:26:04.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.011573995, 2024-03-19T01:26:04.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.019022083, 2024-03-19T01:26:04.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.013192163, 2024-03-19T01:26:04.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:04.749 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 651, 2024-03-19T01:26:04.749 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.015690522, 2024-03-19T01:26:04.749 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000024102 2024-03-19T01:26:04.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.749 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:04.749 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.749 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.749 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.750 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-19T01:26:04.750 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 448 2024-03-19T01:26:04.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.750 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-19T01:26:04.750 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-03-19T01:26:04.750 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-19T01:26:04.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.751 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-03-19T01:26:04.751 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-19T01:26:04.751 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-19T01:26:04.751 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5394, 2024-03-19T01:26:04.751 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.121669234, 2024-03-19T01:26:04.751 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000022556 2024-03-19T01:26:04.751 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:04.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.752 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-03-19T01:26:04.752 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-19T01:26:04.752 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-19T01:26:04.752 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.752 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.752 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.752 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:04.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.752 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-03-19T01:26:04.753 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-03-19T01:26:04.753 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-03-19T01:26:04.753 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-03-19T01:26:04.753 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.753 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.753 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.753 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-03-19T01:26:04.754 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-03-19T01:26:04.754 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-03-19T01:26:04.754 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-03-19T01:26:04.754 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-03-19T01:26:04.754 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-03-19T01:26:04.754 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-03-19T01:26:04.754 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 3852, 2024-03-19T01:26:04.755 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-03-19T01:26:04.755 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 3852, 2024-03-19T01:26:04.755 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-03-19T01:26:04.755 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 3852, 2024-03-19T01:26:04.755 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-03-19T01:26:04.755 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-03-19T01:26:04.755 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-03-19T01:26:04.755 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 138, 2024-03-19T01:26:04.756 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-03-19T01:26:04.756 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-03-19T01:26:04.756 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-03-19T01:26:04.756 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-03-19T01:26:04.756 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-03-19T01:26:04.756 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-03-19T01:26:04.756 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-03-19T01:26:04.756 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-03-19T01:26:04.757 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-03-19T01:26:04.757 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-03-19T01:26:04.757 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-03-19T01:26:04.757 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-03-19T01:26:04.757 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-03-19T01:26:04.757 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-03-19T01:26:04.757 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-03-19T01:26:04.757 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-03-19T01:26:04.757 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-03-19T01:26:04.758 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-03-19T01:26:04.758 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-03-19T01:26:04.758 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T01:26:04.758 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-03-19T01:26:04.758 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T01:26:04.758 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-03-19T01:26:04.758 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-03-19T01:26:04.759 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T01:26:04.759 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T01:26:04.759 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-03-19T01:26:04.759 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-03-19T01:26:04.759 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T01:26:04.759 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-03-19T01:26:04.759 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T01:26:04.759 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T01:26:04.760 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T01:26:04.760 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-03-19T01:26:04.760 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-03-19T01:26:04.760 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-03-19T01:26:04.760 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-03-19T01:26:04.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.760 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-03-19T01:26:04.760 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-03-19T01:26:04.760 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-03-19T01:26:04.761 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-03-19T01:26:04.761 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-03-19T01:26:04.761 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-03-19T01:26:04.761 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-03-19T01:26:04.761 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-03-19T01:26:04.761 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-03-19T01:26:04.761 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-03-19T01:26:04.761 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-03-19T01:26:04.762 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-03-19T01:26:04.762 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-03-19T01:26:04.762 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-03-19T01:26:04.762 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-03-19T01:26:04.762 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-03-19T01:26:04.762 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-03-19T01:26:04.762 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-03-19T01:26:04.762 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-03-19T01:26:04.763 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-03-19T01:26:04.763 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T01:26:04.763 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-03-19T01:26:04.763 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-03-19T01:26:04.763 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-03-19T01:26:04.763 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-03-19T01:26:04.763 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-03-19T01:26:04.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.763 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-03-19T01:26:04.764 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-03-19T01:26:04.764 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-03-19T01:26:04.764 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-03-19T01:26:04.764 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-03-19T01:26:04.764 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 5287, 2024-03-19T01:26:04.764 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-03-19T01:26:04.764 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-03-19T01:26:04.764 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-03-19T01:26:04.764 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-03-19T01:26:04.765 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-03-19T01:26:04.765 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 8, 2024-03-19T01:26:04.765 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-03-19T01:26:04.765 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-03-19T01:26:04.765 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-03-19T01:26:04.765 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-03-19T01:26:04.765 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 18964326, 2024-03-19T01:26:04.765 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 18960107, 2024-03-19T01:26:04.766 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-03-19T01:26:04.766 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.766 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.766 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.766 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 5287 2024-03-19T01:26:04.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.766 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-03-19T01:26:04.767 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3855, 2024-03-19T01:26:04.767 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 3897, 2024-03-19T01:26:04.767 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 42, 2024-03-19T01:26:04.767 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 464, 2024-03-19T01:26:04.767 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 507, 2024-03-19T01:26:04.767 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 43, 2024-03-19T01:26:04.767 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 3852, 2024-03-19T01:26:04.767 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 3894, 2024-03-19T01:26:04.767 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 42, 2024-03-19T01:26:04.768 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-03-19T01:26:04.768 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-03-19T01:26:04.768 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-03-19T01:26:04.768 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 65000, 2024-03-19T01:26:04.768 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215420 2024-03-19T01:26:04.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.768 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-03-19T01:26:04.768 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-03-19T01:26:04.768 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-03-19T01:26:04.769 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-03-19T01:26:04.769 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-03-19T01:26:04.769 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-03-19T01:26:04.769 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-03-19T01:26:04.769 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.769 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.769 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.769 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-03-19T01:26:04.770 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.770 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.770 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.770 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-03-19T01:26:04.770 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.770 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.770 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.771 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-03-19T01:26:04.771 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.771 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.771 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.771 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-03-19T01:26:04.771 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.772 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.772 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.772 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-03-19T01:26:04.772 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.772 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.772 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.772 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-03-19T01:26:04.773 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.773 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.773 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.773 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-03-19T01:26:04.773 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.773 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.773 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.774 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-03-19T01:26:04.774 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.774 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.774 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.774 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-03-19T01:26:04.774 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.774 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.775 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.775 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-03-19T01:26:04.775 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.775 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.775 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.775 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-03-19T01:26:04.776 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.776 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.776 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.777 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-03-19T01:26:04.777 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.777 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.777 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.777 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-03-19T01:26:04.777 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.778 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.778 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.778 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-03-19T01:26:04.778 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.778 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.778 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.779 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-03-19T01:26:04.779 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.779 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.779 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.779 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-03-19T01:26:04.779 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.779 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.780 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.780 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-03-19T01:26:04.780 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.780 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.780 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.780 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-03-19T01:26:04.780 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.781 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.781 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.781 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-03-19T01:26:04.781 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.781 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.781 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.782 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-03-19T01:26:04.782 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.782 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.782 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.783 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-03-19T01:26:04.783 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.783 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.783 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.784 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-03-19T01:26:04.784 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.784 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.784 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.784 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-03-19T01:26:04.785 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.785 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.785 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.785 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-03-19T01:26:04.785 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.785 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.786 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.786 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-03-19T01:26:04.786 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.786 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.786 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.786 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-03-19T01:26:04.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.787 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.787 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-03-19T01:26:04.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.787 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.788 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-03-19T01:26:04.788 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.788 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.788 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.788 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-03-19T01:26:04.788 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-03-19T01:26:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-03-19T01:26:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-03-19T01:26:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-03-19T01:26:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 1, 2024-03-19T01:26:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 0, 2024-03-19T01:26:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-03-19T01:26:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-03-19T01:26:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-03-19T01:26:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-03-19T01:26:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 4795, 2024-03-19T01:26:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-03-19T01:26:04.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-19T01:26:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:26:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-19T01:26:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:26:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:26:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:26:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:26:04.791 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:26:04.792 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:26:04.792 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:26:04.792 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:26:04.792 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:26:04.792 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:26:04.792 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:26:04.792 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:26:04.792 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:26:04.793 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-19T01:26:04.793 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:26:04.793 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-19T01:26:04.793 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:26:04.793 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:26:04.793 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:26:04.793 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:26:04.793 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:26:04.793 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:26:04.794 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:26:04.794 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-19T01:26:04.794 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:26:04.794 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:26:04.794 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:26:04.794 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-19T01:26:04.794 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-19T01:26:04.794 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-19T01:26:04.795 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:26:04.795 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:26:04.795 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:26:04.795 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:26:04.795 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 364754, 2024-03-19T01:26:04.795 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 4285, 2024-03-19T01:26:04.795 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-03-19T01:26:04.796 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-03-19T01:26:04.796 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-19T01:26:04.796 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-19T01:26:04.796 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:26:04.796 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-19T01:26:04.796 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:26:04.796 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-19T01:26:04.796 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9832, 2024-03-19T01:26:04.796 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 205, 2024-03-19T01:26:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:26:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:26:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-19T01:26:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-19T01:26:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 12612371, 2024-03-19T01:26:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 136398, 2024-03-19T01:26:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-19T01:26:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-19T01:26:04.798 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-19T01:26:04.798 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-19T01:26:04.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.798 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-03-19T01:26:04.798 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-03-19T01:26:04.798 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-03-19T01:26:04.798 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 3123, 2024-03-19T01:26:04.798 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-03-19T01:26:04.799 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-03-19T01:26:04.799 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 3123, 2024-03-19T01:26:04.799 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-03-19T01:26:04.799 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3123, 2024-03-19T01:26:04.799 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 26.189444507, 2024-03-19T01:26:04.799 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.008385989 2024-03-19T01:26:04.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.799 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-03-19T01:26:04.800 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-03-19T01:26:04.800 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3123, 2024-03-19T01:26:04.800 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3124 2024-03-19T01:26:04.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.800 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 3123, 2024-03-19T01:26:04.800 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 3123, 2024-03-19T01:26:04.800 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-03-19T01:26:04.800 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-03-19T01:26:04.801 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-03-19T01:26:04.801 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 1885, 2024-03-19T01:26:04.801 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-03-19T01:26:04.801 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 1236, 2024-03-19T01:26:04.801 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-03-19T01:26:04.801 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-03-19T01:26:04.801 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-03-19T01:26:04.801 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-03-19T01:26:04.801 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-03-19T01:26:04.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-03-19T01:26:04.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-03-19T01:26:04.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-03-19T01:26:04.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-03-19T01:26:04.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-03-19T01:26:04.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-03-19T01:26:04.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-03-19T01:26:04.802 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-03-19T01:26:04.803 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-03-19T01:26:04.803 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-03-19T01:26:04.803 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-03-19T01:26:04.803 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-03-19T01:26:04.803 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-03-19T01:26:04.803 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T01:26:04.803 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-03-19T01:26:04.803 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-03-19T01:26:04.803 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-03-19T01:26:04.804 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-03-19T01:26:04.804 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-03-19T01:26:04.804 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-03-19T01:26:04.804 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-03-19T01:26:04.804 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-03-19T01:26:04.804 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-03-19T01:26:04.804 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-03-19T01:26:04.804 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-03-19T01:26:04.804 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-03-19T01:26:04.805 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-03-19T01:26:04.805 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-03-19T01:26:04.805 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-03-19T01:26:04.805 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-03-19T01:26:04.805 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-03-19T01:26:04.805 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-03-19T01:26:04.805 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-03-19T01:26:04.805 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 113, 2024-03-19T01:26:04.806 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-03-19T01:26:04.806 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 24, 2024-03-19T01:26:04.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 7, 2024-03-19T01:26:04.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 8, 2024-03-19T01:26:04.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 1, 2024-03-19T01:26:04.806 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-03-19T01:26:04.806 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-03-19T01:26:04.806 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-03-19T01:26:04.807 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-03-19T01:26:04.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.807 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-03-19T01:26:04.807 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-03-19T01:26:04.807 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-03-19T01:26:04.807 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-03-19T01:26:04.807 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-03-19T01:26:04.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.807 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-03-19T01:26:04.808 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-03-19T01:26:04.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.808 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-03-19T01:26:04.808 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-03-19T01:26:04.808 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T01:26:04.808 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-03-19T01:26:04.808 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-03-19T01:26:04.808 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-03-19T01:26:04.808 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-03-19T01:26:04.809 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-03-19T01:26:04.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.809 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:26:04.809 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:26:04.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-19T01:26:04.809 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:26:04.809 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 3511, 2024-03-19T01:26:04.809 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 17444189, 2024-03-19T01:26:04.810 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:04.810 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 3511, 2024-03-19T01:26:04.810 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:26:04.810 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:26:04.810 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3511, 2024-03-19T01:26:04.810 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 17444189, 2024-03-19T01:26:04.811 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:26:04.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.811 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.811 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:04.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.811 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-03-19T01:26:04.811 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:26:04.812 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-19T01:26:04.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:26:04.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:26:04.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:26:04.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:04.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:04.813 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3123, 2024-03-19T01:26:04.813 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 14770451, 2024-03-19T01:26:04.813 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3123, 2024-03-19T01:26:04.813 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 14770451, 2024-03-19T01:26:04.813 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:26:04.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.813 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:04.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.814 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-03-19T01:26:04.814 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:26:04.814 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-03-19T01:26:04.814 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:26:04.814 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:26:04.814 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:26:04.815 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:04.815 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:04.815 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3123, 2024-03-19T01:26:04.815 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 3123, 2024-03-19T01:26:04.815 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3123, 2024-03-19T01:26:04.815 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 3123, 2024-03-19T01:26:04.815 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:26:04.815 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.815 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.816 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.816 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:04.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.816 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-03-19T01:26:04.816 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:26:04.816 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-19T01:26:04.816 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:26:04.816 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:26:04.817 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:26:04.817 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:04.817 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:04.817 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:26:04.817 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:26:04.817 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-03-19T01:26:04.817 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-03-19T01:26:04.817 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:26:04.818 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.818 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.818 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.818 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:04.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-19T01:26:04.818 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55e3297de6e0": { 2024-03-19T01:26:04.818 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-19T01:26:04.818 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-19T01:26:04.818 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-19T01:26:04.819 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-19T01:26:04.819 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-19T01:26:04.819 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:04.819 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:04.819 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-19T01:26:04.819 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-19T01:26:04.819 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-03-19T01:26:04.819 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-03-19T01:26:04.820 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-19T01:26:04.820 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-19T01:26:04.820 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-19T01:26:04.820 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-19T01:26:04.820 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:04.820 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-19T01:26:04.820 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-19T01:26:05.494 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:05 smithi086 ceph-mon[66829]: osdmap e113: 12 total, 12 up, 12 in 2024-03-19T01:26:05.495 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:05 smithi086 ceph-mon[66829]: pgmap v484: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 44 op/s 2024-03-19T01:26:05.520 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-19T01:26:05.521 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-19T01:26:05.521 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-03-19T01:26:05.521 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-19T01:26:05.521 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-19T01:26:05.521 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-03-19T01:26:05.521 DEBUG:tasks.check_counter:Getting stats from c 2024-03-19T01:26:05.521 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-19T01:26:05.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:05 smithi163 ceph-mon[67428]: osdmap e113: 12 total, 12 up, 12 in 2024-03-19T01:26:05.548 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:05 smithi163 ceph-mon[67428]: pgmap v484: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 44 op/s 2024-03-19T01:26:05.773 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:26:05.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:05 smithi069 ceph-mon[31727]: osdmap e113: 12 total, 12 up, 12 in 2024-03-19T01:26:05.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:05 smithi069 ceph-mon[31727]: pgmap v484: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 44 op/s 2024-03-19T01:26:07.659 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:07 smithi163 ceph-mon[67428]: pgmap v485: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 50 op/s 2024-03-19T01:26:07.688 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-03-19T01:26:07.689 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:26:07.689 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 2615, 2024-03-19T01:26:07.689 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 2566, 2024-03-19T01:26:07.689 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 137036526, 2024-03-19T01:26:07.689 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 903882, 2024-03-19T01:26:07.689 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 7, 2024-03-19T01:26:07.690 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 3, 2024-03-19T01:26:07.690 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.252783568, 2024-03-19T01:26:07.690 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.054148566, 2024-03-19T01:26:07.690 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.147545660, 2024-03-19T01:26:07.690 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.052571398, 2024-03-19T01:26:07.690 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:07.690 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2565, 2024-03-19T01:26:07.691 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.061079530, 2024-03-19T01:26:07.691 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000023812 2024-03-19T01:26:07.691 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.691 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:07.691 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.691 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.691 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.691 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.692 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 2149808, 2024-03-19T01:26:07.692 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 195056 2024-03-19T01:26:07.692 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.692 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:26:07.692 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 727, 2024-03-19T01:26:07.692 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 959, 2024-03-19T01:26:07.692 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 144247820, 2024-03-19T01:26:07.692 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 875794, 2024-03-19T01:26:07.692 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 10, 2024-03-19T01:26:07.693 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 7, 2024-03-19T01:26:07.693 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.123841255, 2024-03-19T01:26:07.693 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.019206388, 2024-03-19T01:26:07.693 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.083646286, 2024-03-19T01:26:07.693 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.015314122, 2024-03-19T01:26:07.693 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:07.693 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 957, 2024-03-19T01:26:07.693 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.043635482, 2024-03-19T01:26:07.694 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000045596 2024-03-19T01:26:07.694 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.694 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:07.694 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.694 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.694 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.694 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.694 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-03-19T01:26:07.695 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 646768 2024-03-19T01:26:07.695 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.695 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:26:07.695 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1257, 2024-03-19T01:26:07.695 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1257, 2024-03-19T01:26:07.695 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 138819747, 2024-03-19T01:26:07.695 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 399097, 2024-03-19T01:26:07.695 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 6, 2024-03-19T01:26:07.695 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 5, 2024-03-19T01:26:07.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.142405434, 2024-03-19T01:26:07.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.025424317, 2024-03-19T01:26:07.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.082852955, 2024-03-19T01:26:07.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.027614674, 2024-03-19T01:26:07.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:07.696 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1255, 2024-03-19T01:26:07.696 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.031606368, 2024-03-19T01:26:07.696 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000025184 2024-03-19T01:26:07.697 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.697 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:07.697 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.697 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.697 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.697 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.697 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T01:26:07.697 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T01:26:07.698 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.698 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-03-19T01:26:07.698 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2024-03-19T01:26:07.698 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-03-19T01:26:07.698 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.698 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2024-03-19T01:26:07.698 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-03-19T01:26:07.699 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-03-19T01:26:07.699 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 6787, 2024-03-19T01:26:07.699 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.261458063, 2024-03-19T01:26:07.699 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000038523 2024-03-19T01:26:07.699 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:07.699 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.699 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2024-03-19T01:26:07.699 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-03-19T01:26:07.700 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-03-19T01:26:07.700 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.700 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.700 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.700 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:07.700 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.700 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2024-03-19T01:26:07.700 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 0, 2024-03-19T01:26:07.700 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 0, 2024-03-19T01:26:07.701 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2024-03-19T01:26:07.701 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.701 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.701 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.701 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.701 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2024-03-19T01:26:07.701 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 0, 2024-03-19T01:26:07.701 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 0, 2024-03-19T01:26:07.702 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 0, 2024-03-19T01:26:07.702 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 0, 2024-03-19T01:26:07.702 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 0, 2024-03-19T01:26:07.702 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2024-03-19T01:26:07.702 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 65090, 2024-03-19T01:26:07.702 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 8723, 2024-03-19T01:26:07.702 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 56367, 2024-03-19T01:26:07.702 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 0, 2024-03-19T01:26:07.703 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 52630, 2024-03-19T01:26:07.703 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 0, 2024-03-19T01:26:07.703 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 0, 2024-03-19T01:26:07.703 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 0, 2024-03-19T01:26:07.703 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 444, 2024-03-19T01:26:07.703 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 0, 2024-03-19T01:26:07.703 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 0, 2024-03-19T01:26:07.703 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 0, 2024-03-19T01:26:07.703 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 0, 2024-03-19T01:26:07.704 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2024-03-19T01:26:07.704 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2024-03-19T01:26:07.704 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 0, 2024-03-19T01:26:07.704 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 0, 2024-03-19T01:26:07.704 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2024-03-19T01:26:07.704 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 0, 2024-03-19T01:26:07.704 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 0, 2024-03-19T01:26:07.704 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 0, 2024-03-19T01:26:07.705 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 0, 2024-03-19T01:26:07.705 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2024-03-19T01:26:07.705 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 0, 2024-03-19T01:26:07.705 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2024-03-19T01:26:07.705 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 89590, 2024-03-19T01:26:07.705 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 1645705921, 2024-03-19T01:26:07.705 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 6, 2024-03-19T01:26:07.705 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T01:26:07.705 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2024-03-19T01:26:07.706 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T01:26:07.706 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2024-03-19T01:26:07.706 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 0, 2024-03-19T01:26:07.706 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T01:26:07.706 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T01:26:07.706 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 0, 2024-03-19T01:26:07.706 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 0, 2024-03-19T01:26:07.706 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T01:26:07.707 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 0, 2024-03-19T01:26:07.707 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T01:26:07.707 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T01:26:07.707 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T01:26:07.707 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 0, 2024-03-19T01:26:07.707 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 0, 2024-03-19T01:26:07.707 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 0, 2024-03-19T01:26:07.707 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 0 2024-03-19T01:26:07.708 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.708 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2024-03-19T01:26:07.708 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 0, 2024-03-19T01:26:07.708 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2024-03-19T01:26:07.708 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2024-03-19T01:26:07.708 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 0, 2024-03-19T01:26:07.708 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 0, 2024-03-19T01:26:07.708 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2024-03-19T01:26:07.708 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2024-03-19T01:26:07.709 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2024-03-19T01:26:07.709 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 0, 2024-03-19T01:26:07.709 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 0, 2024-03-19T01:26:07.709 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 0, 2024-03-19T01:26:07.709 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2024-03-19T01:26:07.709 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2024-03-19T01:26:07.709 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2024-03-19T01:26:07.709 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2024-03-19T01:26:07.710 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2024-03-19T01:26:07.710 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2024-03-19T01:26:07.710 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_path": 0, 2024-03-19T01:26:07.710 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_inode": 0, 2024-03-19T01:26:07.710 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T01:26:07.710 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 0, 2024-03-19T01:26:07.710 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2024-03-19T01:26:07.710 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 0, 2024-03-19T01:26:07.711 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2024-03-19T01:26:07.711 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2024-03-19T01:26:07.711 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.711 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2024-03-19T01:26:07.711 INFO:teuthology.orchestra.run.smithi163.stdout: "evlrg": 0, 2024-03-19T01:26:07.711 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 0, 2024-03-19T01:26:07.711 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 0, 2024-03-19T01:26:07.711 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 0, 2024-03-19T01:26:07.712 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 138254, 2024-03-19T01:26:07.712 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2024-03-19T01:26:07.712 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 0, 2024-03-19T01:26:07.712 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 0, 2024-03-19T01:26:07.712 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 0, 2024-03-19T01:26:07.712 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 0, 2024-03-19T01:26:07.712 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 140, 2024-03-19T01:26:07.712 INFO:teuthology.orchestra.run.smithi163.stdout: "segmjr": 13, 2024-03-19T01:26:07.712 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2024-03-19T01:26:07.713 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 0, 2024-03-19T01:26:07.713 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 89238282, 2024-03-19T01:26:07.713 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 420851967, 2024-03-19T01:26:07.713 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 420842857, 2024-03-19T01:26:07.713 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2024-03-19T01:26:07.713 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.713 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.714 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.714 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.714 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 138254 2024-03-19T01:26:07.714 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.714 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2024-03-19T01:26:07.714 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 62747, 2024-03-19T01:26:07.714 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 72551, 2024-03-19T01:26:07.714 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 9804, 2024-03-19T01:26:07.715 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 4071, 2024-03-19T01:26:07.715 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 4706, 2024-03-19T01:26:07.715 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 635, 2024-03-19T01:26:07.715 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 65090, 2024-03-19T01:26:07.715 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 73773, 2024-03-19T01:26:07.715 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 8683, 2024-03-19T01:26:07.715 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 0, 2024-03-19T01:26:07.715 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 0, 2024-03-19T01:26:07.715 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 0, 2024-03-19T01:26:07.716 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 298808, 2024-03-19T01:26:07.716 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 215420 2024-03-19T01:26:07.716 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.716 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2024-03-19T01:26:07.716 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 0, 2024-03-19T01:26:07.716 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 0, 2024-03-19T01:26:07.716 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 0, 2024-03-19T01:26:07.716 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 0, 2024-03-19T01:26:07.716 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 0, 2024-03-19T01:26:07.717 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2024-03-19T01:26:07.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.717 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.717 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.717 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2024-03-19T01:26:07.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.717 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.718 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.718 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2024-03-19T01:26:07.718 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.718 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.718 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.718 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.718 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2024-03-19T01:26:07.718 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.719 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.719 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.719 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.719 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2024-03-19T01:26:07.719 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.719 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.719 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.719 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.720 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2024-03-19T01:26:07.720 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.720 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.720 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.720 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.720 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2024-03-19T01:26:07.720 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.720 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.720 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.721 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.721 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2024-03-19T01:26:07.721 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.721 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.721 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.721 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.721 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2024-03-19T01:26:07.721 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.722 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.722 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.722 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2024-03-19T01:26:07.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.722 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.722 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.723 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2024-03-19T01:26:07.723 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.723 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.724 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.724 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.724 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2024-03-19T01:26:07.724 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.724 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.724 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.724 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.724 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2024-03-19T01:26:07.725 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.725 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.725 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.725 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.725 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2024-03-19T01:26:07.725 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.725 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.725 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.726 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.726 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2024-03-19T01:26:07.726 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.726 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.726 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.726 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.726 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2024-03-19T01:26:07.726 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.727 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.727 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.727 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.727 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2024-03-19T01:26:07.727 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.727 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.727 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.727 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.728 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2024-03-19T01:26:07.728 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.728 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.728 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.728 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.728 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2024-03-19T01:26:07.728 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.728 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.729 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.729 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.729 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2024-03-19T01:26:07.729 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.729 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.729 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.729 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.729 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2024-03-19T01:26:07.729 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.730 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.730 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.730 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.730 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2024-03-19T01:26:07.730 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.730 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.730 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.730 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.731 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2024-03-19T01:26:07.731 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.731 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.731 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.731 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.731 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2024-03-19T01:26:07.731 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.731 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.732 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.732 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.732 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2024-03-19T01:26:07.732 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.732 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.732 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.732 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.732 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2024-03-19T01:26:07.732 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.733 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.733 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.733 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.733 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2024-03-19T01:26:07.733 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.733 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.733 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.733 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.734 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2024-03-19T01:26:07.734 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.734 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.734 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.734 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.734 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2024-03-19T01:26:07.734 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.734 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.735 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.735 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.735 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2024-03-19T01:26:07.735 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2024-03-19T01:26:07.735 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2024-03-19T01:26:07.735 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.735 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.735 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.735 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:07.736 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.736 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2024-03-19T01:26:07.736 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 2, 2024-03-19T01:26:07.736 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 6, 2024-03-19T01:26:07.736 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 4, 2024-03-19T01:26:07.736 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 3, 2024-03-19T01:26:07.736 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2024-03-19T01:26:07.736 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 0, 2024-03-19T01:26:07.736 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 0, 2024-03-19T01:26:07.737 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 12383, 2024-03-19T01:26:07.737 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2024-03-19T01:26:07.737 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.737 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-03-19T01:26:07.737 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:26:07.737 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-03-19T01:26:07.737 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:26:07.737 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:26:07.737 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:26:07.738 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:26:07.738 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:26:07.738 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:26:07.738 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:26:07.738 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:26:07.738 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:26:07.738 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:26:07.738 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:26:07.738 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:26:07.739 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:26:07.739 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-03-19T01:26:07.739 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:26:07.739 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-03-19T01:26:07.739 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:26:07.739 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:26:07.739 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:26:07.740 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:26:07.740 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:26:07.740 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:26:07.740 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:26:07.740 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-03-19T01:26:07.740 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:26:07.740 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:26:07.740 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:26:07.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-03-19T01:26:07.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-03-19T01:26:07.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-03-19T01:26:07.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:26:07.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:26:07.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:26:07.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:26:07.741 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 2909120, 2024-03-19T01:26:07.741 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 72233, 2024-03-19T01:26:07.742 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2024-03-19T01:26:07.742 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2024-03-19T01:26:07.742 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-03-19T01:26:07.742 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-03-19T01:26:07.742 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:26:07.742 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-03-19T01:26:07.742 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:26:07.742 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-03-19T01:26:07.742 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 10072, 2024-03-19T01:26:07.743 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 211, 2024-03-19T01:26:07.743 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:26:07.743 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:26:07.743 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-03-19T01:26:07.743 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-03-19T01:26:07.743 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 204951843, 2024-03-19T01:26:07.743 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 2388674, 2024-03-19T01:26:07.743 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-03-19T01:26:07.743 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-03-19T01:26:07.744 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-03-19T01:26:07.744 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-03-19T01:26:07.744 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.744 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2024-03-19T01:26:07.744 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2024-03-19T01:26:07.744 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2024-03-19T01:26:07.744 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 4204, 2024-03-19T01:26:07.744 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 26, 2024-03-19T01:26:07.745 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 4, 2024-03-19T01:26:07.745 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 4204, 2024-03-19T01:26:07.745 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2024-03-19T01:26:07.745 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 4204, 2024-03-19T01:26:07.745 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 20.287522335, 2024-03-19T01:26:07.745 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.004825766 2024-03-19T01:26:07.745 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.745 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2024-03-19T01:26:07.745 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2024-03-19T01:26:07.746 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 4204, 2024-03-19T01:26:07.746 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 4205 2024-03-19T01:26:07.746 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.746 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 4204, 2024-03-19T01:26:07.746 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 4204, 2024-03-19T01:26:07.746 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 0, 2024-03-19T01:26:07.746 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2024-03-19T01:26:07.746 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2024-03-19T01:26:07.746 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 2522, 2024-03-19T01:26:07.747 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 0, 2024-03-19T01:26:07.747 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 1679, 2024-03-19T01:26:07.747 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 0, 2024-03-19T01:26:07.747 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 0, 2024-03-19T01:26:07.747 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2024-03-19T01:26:07.747 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2024-03-19T01:26:07.747 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2024-03-19T01:26:07.747 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 0, 2024-03-19T01:26:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 0, 2024-03-19T01:26:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2024-03-19T01:26:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2024-03-19T01:26:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2024-03-19T01:26:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 2, 2024-03-19T01:26:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 0, 2024-03-19T01:26:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2024-03-19T01:26:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2024-03-19T01:26:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2024-03-19T01:26:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2024-03-19T01:26:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2024-03-19T01:26:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2024-03-19T01:26:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T01:26:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2024-03-19T01:26:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2024-03-19T01:26:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 0, 2024-03-19T01:26:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2024-03-19T01:26:07.750 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2024-03-19T01:26:07.750 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2024-03-19T01:26:07.750 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2024-03-19T01:26:07.750 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2024-03-19T01:26:07.750 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2024-03-19T01:26:07.750 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2024-03-19T01:26:07.750 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2024-03-19T01:26:07.750 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2024-03-19T01:26:07.751 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2024-03-19T01:26:07.751 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2024-03-19T01:26:07.751 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2024-03-19T01:26:07.751 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2024-03-19T01:26:07.751 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2024-03-19T01:26:07.751 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2024-03-19T01:26:07.751 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2024-03-19T01:26:07.751 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 113, 2024-03-19T01:26:07.751 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2024-03-19T01:26:07.752 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 24, 2024-03-19T01:26:07.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 11, 2024-03-19T01:26:07.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 11, 2024-03-19T01:26:07.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 0, 2024-03-19T01:26:07.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2024-03-19T01:26:07.752 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 0, 2024-03-19T01:26:07.752 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 2, 2024-03-19T01:26:07.752 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 0 2024-03-19T01:26:07.752 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.753 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2024-03-19T01:26:07.753 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 0, 2024-03-19T01:26:07.753 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2024-03-19T01:26:07.753 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2024-03-19T01:26:07.753 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2024-03-19T01:26:07.753 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.753 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2024-03-19T01:26:07.753 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 0 2024-03-19T01:26:07.753 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.754 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2024-03-19T01:26:07.754 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2024-03-19T01:26:07.754 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T01:26:07.754 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2024-03-19T01:26:07.754 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 0, 2024-03-19T01:26:07.754 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed_ops": 0, 2024-03-19T01:26:07.754 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 0, 2024-03-19T01:26:07.754 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2024-03-19T01:26:07.755 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.755 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:26:07.755 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:07.755 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-03-19T01:26:07.755 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:07.755 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 4595, 2024-03-19T01:26:07.755 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 419667931, 2024-03-19T01:26:07.755 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:07.755 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 4595, 2024-03-19T01:26:07.756 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:07.756 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:07.756 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 4595, 2024-03-19T01:26:07.756 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 419667931, 2024-03-19T01:26:07.756 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:07.756 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.756 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.756 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.757 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:07.757 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.757 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2024-03-19T01:26:07.757 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:07.757 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-03-19T01:26:07.757 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:07.757 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:07.757 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:07.758 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:07.758 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:07.758 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 4204, 2024-03-19T01:26:07.758 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 416789431, 2024-03-19T01:26:07.758 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 4204, 2024-03-19T01:26:07.758 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 416789431, 2024-03-19T01:26:07.758 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:07.758 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.759 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.759 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.759 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:07.759 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.759 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2024-03-19T01:26:07.759 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:07.759 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2024-03-19T01:26:07.759 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:07.759 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:07.760 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:07.760 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:07.760 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:07.760 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 4204, 2024-03-19T01:26:07.760 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 4204, 2024-03-19T01:26:07.760 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 4204, 2024-03-19T01:26:07.760 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 4204, 2024-03-19T01:26:07.760 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:07.761 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.761 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.761 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.761 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:07.761 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.761 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2024-03-19T01:26:07.761 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:07.761 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-03-19T01:26:07.762 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:07.762 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:07.762 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:07.762 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:07.762 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:07.762 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:07.763 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:07.763 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-03-19T01:26:07.763 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-03-19T01:26:07.763 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:07.763 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.763 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.763 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.763 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:07.763 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:07.764 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x560cf4b6e6e0": { 2024-03-19T01:26:07.764 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:07.764 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-03-19T01:26:07.764 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:07.764 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:07.764 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:07.764 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:07.764 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:07.765 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:07.765 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:07.765 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-03-19T01:26:07.765 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-03-19T01:26:07.765 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:07.765 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:07.765 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:07.765 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:07.766 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:07.766 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:07.766 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-03-19T01:26:07.766 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:07 smithi086 ceph-mon[66829]: pgmap v485: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 50 op/s 2024-03-19T01:26:07.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:07 smithi069 ceph-mon[31727]: pgmap v485: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 50 op/s 2024-03-19T01:26:08.537 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-19T01:26:08.537 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-19T01:26:08.537 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-03-19T01:26:08.537 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-19T01:26:08.537 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-19T01:26:08.537 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-03-19T01:26:08.538 DEBUG:tasks.check_counter:Getting stats from f 2024-03-19T01:26:08.538 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-19T01:26:08.833 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:26:10.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:10 smithi163 ceph-mon[67428]: pgmap v486: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 51 op/s 2024-03-19T01:26:10.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:10 smithi163 ceph-mon[67428]: osdmap e114: 12 total, 12 up, 12 in 2024-03-19T01:26:10.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:10 smithi086 ceph-mon[66829]: pgmap v486: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 51 op/s 2024-03-19T01:26:10.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:10 smithi086 ceph-mon[66829]: osdmap e114: 12 total, 12 up, 12 in 2024-03-19T01:26:10.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:10 smithi069 ceph-mon[31727]: pgmap v486: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 51 op/s 2024-03-19T01:26:10.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:10 smithi069 ceph-mon[31727]: osdmap e114: 12 total, 12 up, 12 in 2024-03-19T01:26:10.714 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-03-19T01:26:10.714 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:26:10.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 548, 2024-03-19T01:26:10.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 548, 2024-03-19T01:26:10.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 12730766, 2024-03-19T01:26:10.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 173504, 2024-03-19T01:26:10.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 8, 2024-03-19T01:26:10.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 3, 2024-03-19T01:26:10.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.045539667, 2024-03-19T01:26:10.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.009790411, 2024-03-19T01:26:10.716 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.020553189, 2024-03-19T01:26:10.716 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.011147890, 2024-03-19T01:26:10.716 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:10.716 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 547, 2024-03-19T01:26:10.716 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.024342290, 2024-03-19T01:26:10.716 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000044501 2024-03-19T01:26:10.716 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.716 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:10.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.717 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.717 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.717 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-19T01:26:10.717 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 448 2024-03-19T01:26:10.717 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.717 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:26:10.717 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1038, 2024-03-19T01:26:10.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1272, 2024-03-19T01:26:10.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 16620065, 2024-03-19T01:26:10.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 899117, 2024-03-19T01:26:10.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 8, 2024-03-19T01:26:10.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 6, 2024-03-19T01:26:10.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.088132519, 2024-03-19T01:26:10.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.028031202, 2024-03-19T01:26:10.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.027011526, 2024-03-19T01:26:10.719 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.025273855, 2024-03-19T01:26:10.719 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:10.719 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1269, 2024-03-19T01:26:10.719 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.055593226, 2024-03-19T01:26:10.719 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000043808 2024-03-19T01:26:10.719 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.719 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:10.719 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.719 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.720 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.720 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.720 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-19T01:26:10.720 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 569664 2024-03-19T01:26:10.720 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.720 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:26:10.720 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 2287, 2024-03-19T01:26:10.720 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 2238, 2024-03-19T01:26:10.720 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 11113518, 2024-03-19T01:26:10.721 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 800234, 2024-03-19T01:26:10.721 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 5, 2024-03-19T01:26:10.721 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 4, 2024-03-19T01:26:10.721 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.185296949, 2024-03-19T01:26:10.721 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.048727146, 2024-03-19T01:26:10.721 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.090935974, 2024-03-19T01:26:10.721 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.049832682, 2024-03-19T01:26:10.721 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:10.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2237, 2024-03-19T01:26:10.722 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.063622603, 2024-03-19T01:26:10.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000028441 2024-03-19T01:26:10.722 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.722 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:10.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.722 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.722 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.723 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 2150704, 2024-03-19T01:26:10.723 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 196352 2024-03-19T01:26:10.723 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.723 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-03-19T01:26:10.723 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2024-03-19T01:26:10.723 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-03-19T01:26:10.723 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.723 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2024-03-19T01:26:10.723 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-03-19T01:26:10.724 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-03-19T01:26:10.724 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 5871, 2024-03-19T01:26:10.724 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.125894155, 2024-03-19T01:26:10.724 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000021443 2024-03-19T01:26:10.724 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:10.724 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.725 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2024-03-19T01:26:10.725 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-03-19T01:26:10.725 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-03-19T01:26:10.725 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.725 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.725 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.725 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:10.725 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.725 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2024-03-19T01:26:10.726 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 0, 2024-03-19T01:26:10.726 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 0, 2024-03-19T01:26:10.726 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2024-03-19T01:26:10.726 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.726 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.726 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.726 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.726 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2024-03-19T01:26:10.727 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 0, 2024-03-19T01:26:10.727 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 0, 2024-03-19T01:26:10.727 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 0, 2024-03-19T01:26:10.727 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 0, 2024-03-19T01:26:10.727 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 0, 2024-03-19T01:26:10.727 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2024-03-19T01:26:10.727 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 8622, 2024-03-19T01:26:10.727 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 0, 2024-03-19T01:26:10.727 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 8622, 2024-03-19T01:26:10.728 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 0, 2024-03-19T01:26:10.728 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 8622, 2024-03-19T01:26:10.728 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 0, 2024-03-19T01:26:10.728 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 0, 2024-03-19T01:26:10.728 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 0, 2024-03-19T01:26:10.728 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 150, 2024-03-19T01:26:10.728 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 0, 2024-03-19T01:26:10.728 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 0, 2024-03-19T01:26:10.728 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 0, 2024-03-19T01:26:10.729 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 0, 2024-03-19T01:26:10.729 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2024-03-19T01:26:10.729 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2024-03-19T01:26:10.729 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 0, 2024-03-19T01:26:10.729 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 0, 2024-03-19T01:26:10.729 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2024-03-19T01:26:10.729 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 0, 2024-03-19T01:26:10.729 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 0, 2024-03-19T01:26:10.729 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 0, 2024-03-19T01:26:10.730 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 0, 2024-03-19T01:26:10.730 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2024-03-19T01:26:10.730 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 0, 2024-03-19T01:26:10.730 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2024-03-19T01:26:10.730 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 0, 2024-03-19T01:26:10.730 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 0, 2024-03-19T01:26:10.730 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 0, 2024-03-19T01:26:10.730 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T01:26:10.731 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2024-03-19T01:26:10.731 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T01:26:10.731 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2024-03-19T01:26:10.731 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 0, 2024-03-19T01:26:10.731 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T01:26:10.731 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T01:26:10.731 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 0, 2024-03-19T01:26:10.731 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 0, 2024-03-19T01:26:10.731 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T01:26:10.732 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 0, 2024-03-19T01:26:10.732 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T01:26:10.732 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T01:26:10.732 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T01:26:10.732 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 0, 2024-03-19T01:26:10.732 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 0, 2024-03-19T01:26:10.732 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 0, 2024-03-19T01:26:10.732 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 0 2024-03-19T01:26:10.732 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.733 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2024-03-19T01:26:10.733 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 0, 2024-03-19T01:26:10.733 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2024-03-19T01:26:10.733 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2024-03-19T01:26:10.733 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 0, 2024-03-19T01:26:10.733 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 0, 2024-03-19T01:26:10.733 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2024-03-19T01:26:10.733 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2024-03-19T01:26:10.733 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2024-03-19T01:26:10.734 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 0, 2024-03-19T01:26:10.734 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 0, 2024-03-19T01:26:10.734 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 0, 2024-03-19T01:26:10.734 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2024-03-19T01:26:10.734 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2024-03-19T01:26:10.734 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2024-03-19T01:26:10.734 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2024-03-19T01:26:10.734 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2024-03-19T01:26:10.735 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2024-03-19T01:26:10.735 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_path": 0, 2024-03-19T01:26:10.735 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_inode": 0, 2024-03-19T01:26:10.735 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T01:26:10.735 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 0, 2024-03-19T01:26:10.735 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2024-03-19T01:26:10.735 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 0, 2024-03-19T01:26:10.735 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2024-03-19T01:26:10.735 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2024-03-19T01:26:10.735 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.736 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2024-03-19T01:26:10.736 INFO:teuthology.orchestra.run.smithi163.stdout: "evlrg": 0, 2024-03-19T01:26:10.736 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 0, 2024-03-19T01:26:10.736 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 0, 2024-03-19T01:26:10.736 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 0, 2024-03-19T01:26:10.736 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 13115, 2024-03-19T01:26:10.736 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2024-03-19T01:26:10.736 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 0, 2024-03-19T01:26:10.736 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 0, 2024-03-19T01:26:10.737 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 0, 2024-03-19T01:26:10.737 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 0, 2024-03-19T01:26:10.737 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 18, 2024-03-19T01:26:10.737 INFO:teuthology.orchestra.run.smithi163.stdout: "segmjr": 3, 2024-03-19T01:26:10.737 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2024-03-19T01:26:10.737 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 0, 2024-03-19T01:26:10.737 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 4194304, 2024-03-19T01:26:10.737 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 41527059, 2024-03-19T01:26:10.737 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 41507650, 2024-03-19T01:26:10.738 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2024-03-19T01:26:10.738 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.738 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.738 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.738 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.738 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 13115 2024-03-19T01:26:10.738 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.739 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2024-03-19T01:26:10.739 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 8438, 2024-03-19T01:26:10.739 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 8931, 2024-03-19T01:26:10.739 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 493, 2024-03-19T01:26:10.739 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 693, 2024-03-19T01:26:10.739 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 748, 2024-03-19T01:26:10.739 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 55, 2024-03-19T01:26:10.739 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 8622, 2024-03-19T01:26:10.739 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 8995, 2024-03-19T01:26:10.740 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 373, 2024-03-19T01:26:10.740 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 0, 2024-03-19T01:26:10.740 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 0, 2024-03-19T01:26:10.740 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 0, 2024-03-19T01:26:10.740 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 85104, 2024-03-19T01:26:10.740 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 215420 2024-03-19T01:26:10.740 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.740 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2024-03-19T01:26:10.741 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 0, 2024-03-19T01:26:10.741 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 0, 2024-03-19T01:26:10.741 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 0, 2024-03-19T01:26:10.741 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 0, 2024-03-19T01:26:10.741 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 0, 2024-03-19T01:26:10.741 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2024-03-19T01:26:10.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.741 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.742 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.742 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2024-03-19T01:26:10.742 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.742 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.742 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.742 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.742 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2024-03-19T01:26:10.742 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.742 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.742 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.743 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.743 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2024-03-19T01:26:10.743 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.743 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.743 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.743 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.743 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2024-03-19T01:26:10.743 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.744 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.744 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.744 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.744 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2024-03-19T01:26:10.744 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.744 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.744 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.744 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.744 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2024-03-19T01:26:10.745 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.745 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.745 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.745 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.745 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2024-03-19T01:26:10.745 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.745 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.745 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.745 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.746 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2024-03-19T01:26:10.746 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.746 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.746 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.746 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.746 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2024-03-19T01:26:10.746 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.746 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.747 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.747 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.747 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2024-03-19T01:26:10.747 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.747 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.747 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.747 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.748 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2024-03-19T01:26:10.748 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.748 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.748 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.748 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.748 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2024-03-19T01:26:10.749 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.749 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.749 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.749 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.749 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2024-03-19T01:26:10.749 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.749 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.749 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.749 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.750 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2024-03-19T01:26:10.750 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.750 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.750 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.750 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.750 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2024-03-19T01:26:10.750 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.750 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.750 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.751 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.751 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2024-03-19T01:26:10.751 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.751 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.751 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.751 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.751 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2024-03-19T01:26:10.751 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.751 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.752 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.752 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.752 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2024-03-19T01:26:10.752 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.752 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.752 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.752 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.752 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2024-03-19T01:26:10.753 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.753 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.753 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.753 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.753 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2024-03-19T01:26:10.753 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.753 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.754 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.754 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.754 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2024-03-19T01:26:10.754 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.754 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.754 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.754 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.754 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2024-03-19T01:26:10.755 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.755 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.755 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.755 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.755 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2024-03-19T01:26:10.755 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.755 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.755 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.755 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.756 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2024-03-19T01:26:10.756 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.756 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.756 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.756 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.756 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2024-03-19T01:26:10.756 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.756 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.756 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.757 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.757 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2024-03-19T01:26:10.757 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.757 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.757 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.757 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.757 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2024-03-19T01:26:10.757 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.758 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.758 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.758 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.758 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2024-03-19T01:26:10.758 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.758 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.758 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.759 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.759 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2024-03-19T01:26:10.759 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2024-03-19T01:26:10.759 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2024-03-19T01:26:10.759 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.759 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.759 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.759 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:10.759 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.760 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2024-03-19T01:26:10.760 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 2, 2024-03-19T01:26:10.760 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 5, 2024-03-19T01:26:10.760 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 3, 2024-03-19T01:26:10.760 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 3, 2024-03-19T01:26:10.760 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2024-03-19T01:26:10.760 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 0, 2024-03-19T01:26:10.760 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 0, 2024-03-19T01:26:10.760 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 12386, 2024-03-19T01:26:10.761 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2024-03-19T01:26:10.761 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.761 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-03-19T01:26:10.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:26:10.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-03-19T01:26:10.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:26:10.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:26:10.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:26:10.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:26:10.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:26:10.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:26:10.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:26:10.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:26:10.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:26:10.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:26:10.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:26:10.763 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:26:10.763 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:26:10.763 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-03-19T01:26:10.763 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:26:10.763 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-03-19T01:26:10.763 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:26:10.763 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:26:10.763 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:26:10.763 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:26:10.764 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:26:10.764 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:26:10.764 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:26:10.764 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-03-19T01:26:10.764 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:26:10.764 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:26:10.764 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:26:10.764 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-03-19T01:26:10.764 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-03-19T01:26:10.765 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-03-19T01:26:10.765 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:26:10.765 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:26:10.765 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:26:10.765 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:26:10.765 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 566469, 2024-03-19T01:26:10.765 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 9308, 2024-03-19T01:26:10.766 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2024-03-19T01:26:10.766 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2024-03-19T01:26:10.766 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-03-19T01:26:10.766 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-03-19T01:26:10.766 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:26:10.766 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-03-19T01:26:10.767 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:26:10.767 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-03-19T01:26:10.767 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 10072, 2024-03-19T01:26:10.767 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 211, 2024-03-19T01:26:10.767 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:26:10.767 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:26:10.767 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-03-19T01:26:10.768 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-03-19T01:26:10.768 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 27302260, 2024-03-19T01:26:10.768 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 314310, 2024-03-19T01:26:10.768 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-03-19T01:26:10.768 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-03-19T01:26:10.768 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-03-19T01:26:10.768 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-03-19T01:26:10.768 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.768 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2024-03-19T01:26:10.769 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2024-03-19T01:26:10.769 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2024-03-19T01:26:10.769 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 3479, 2024-03-19T01:26:10.769 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 21, 2024-03-19T01:26:10.769 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 3, 2024-03-19T01:26:10.769 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 3479, 2024-03-19T01:26:10.769 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2024-03-19T01:26:10.769 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3479, 2024-03-19T01:26:10.769 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 8.694795815, 2024-03-19T01:26:10.770 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.002499222 2024-03-19T01:26:10.770 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.770 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2024-03-19T01:26:10.770 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2024-03-19T01:26:10.770 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3479, 2024-03-19T01:26:10.770 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 3480 2024-03-19T01:26:10.770 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.771 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 3479, 2024-03-19T01:26:10.771 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 3479, 2024-03-19T01:26:10.771 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 0, 2024-03-19T01:26:10.771 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2024-03-19T01:26:10.771 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2024-03-19T01:26:10.771 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 2132, 2024-03-19T01:26:10.771 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 0, 2024-03-19T01:26:10.771 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 1345, 2024-03-19T01:26:10.771 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 0, 2024-03-19T01:26:10.772 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 0, 2024-03-19T01:26:10.772 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2024-03-19T01:26:10.772 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2024-03-19T01:26:10.772 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2024-03-19T01:26:10.772 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 0, 2024-03-19T01:26:10.772 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 0, 2024-03-19T01:26:10.772 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2024-03-19T01:26:10.772 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2024-03-19T01:26:10.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2024-03-19T01:26:10.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 1, 2024-03-19T01:26:10.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 0, 2024-03-19T01:26:10.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2024-03-19T01:26:10.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2024-03-19T01:26:10.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2024-03-19T01:26:10.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2024-03-19T01:26:10.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2024-03-19T01:26:10.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2024-03-19T01:26:10.774 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T01:26:10.774 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2024-03-19T01:26:10.774 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2024-03-19T01:26:10.774 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 0, 2024-03-19T01:26:10.774 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2024-03-19T01:26:10.774 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2024-03-19T01:26:10.774 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2024-03-19T01:26:10.774 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2024-03-19T01:26:10.775 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2024-03-19T01:26:10.775 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2024-03-19T01:26:10.775 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2024-03-19T01:26:10.775 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2024-03-19T01:26:10.775 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2024-03-19T01:26:10.775 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2024-03-19T01:26:10.775 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2024-03-19T01:26:10.775 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2024-03-19T01:26:10.776 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2024-03-19T01:26:10.776 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2024-03-19T01:26:10.776 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2024-03-19T01:26:10.776 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2024-03-19T01:26:10.776 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 114, 2024-03-19T01:26:10.776 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2024-03-19T01:26:10.776 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 25, 2024-03-19T01:26:10.776 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 9, 2024-03-19T01:26:10.776 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 10, 2024-03-19T01:26:10.777 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 1, 2024-03-19T01:26:10.777 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2024-03-19T01:26:10.777 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 0, 2024-03-19T01:26:10.777 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 2, 2024-03-19T01:26:10.777 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 0 2024-03-19T01:26:10.777 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.777 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2024-03-19T01:26:10.777 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 0, 2024-03-19T01:26:10.777 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2024-03-19T01:26:10.778 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2024-03-19T01:26:10.778 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2024-03-19T01:26:10.778 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.778 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2024-03-19T01:26:10.778 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 0 2024-03-19T01:26:10.778 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.778 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2024-03-19T01:26:10.778 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2024-03-19T01:26:10.779 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T01:26:10.779 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2024-03-19T01:26:10.779 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 0, 2024-03-19T01:26:10.779 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed_ops": 0, 2024-03-19T01:26:10.779 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 0, 2024-03-19T01:26:10.779 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2024-03-19T01:26:10.779 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.779 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:26:10.780 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:10.780 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-03-19T01:26:10.780 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:10.780 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 3869, 2024-03-19T01:26:10.780 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 40093466, 2024-03-19T01:26:10.780 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:10.780 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 3869, 2024-03-19T01:26:10.780 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:10.780 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:10.781 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3869, 2024-03-19T01:26:10.781 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 40093466, 2024-03-19T01:26:10.781 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:10.781 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.781 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.781 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.781 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:10.781 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.782 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2024-03-19T01:26:10.782 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:10.782 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-03-19T01:26:10.782 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:10.782 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:10.782 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:10.782 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:10.782 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:10.783 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 3479, 2024-03-19T01:26:10.783 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 37336939, 2024-03-19T01:26:10.783 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3479, 2024-03-19T01:26:10.783 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 37336939, 2024-03-19T01:26:10.783 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:10.783 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.783 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.783 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.783 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:10.784 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.784 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2024-03-19T01:26:10.784 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:10.784 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2024-03-19T01:26:10.784 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:10.784 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:10.784 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:10.784 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:10.784 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:10.785 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 3479, 2024-03-19T01:26:10.785 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 3479, 2024-03-19T01:26:10.785 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3479, 2024-03-19T01:26:10.785 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 3479, 2024-03-19T01:26:10.785 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:10.785 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.785 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.785 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2024-03-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-03-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-03-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-03-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x55e2c492c6e0": { 2024-03-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-03-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-03-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-03-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-03-19T01:26:11.585 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-19T01:26:11.586 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-19T01:26:11.586 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-19T01:26:11.586 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-19T01:26:11.586 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-19T01:26:11.586 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-03-19T01:26:11.586 DEBUG:tasks.check_counter:Getting stats from i 2024-03-19T01:26:11.586 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-19T01:26:11.847 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:26:12.271 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:12 smithi163 ceph-mon[67428]: pgmap v488: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 610 KiB/s wr, 15 op/s 2024-03-19T01:26:12.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:12 smithi086 ceph-mon[66829]: pgmap v488: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 610 KiB/s wr, 15 op/s 2024-03-19T01:26:12.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:12 smithi069 ceph-mon[31727]: pgmap v488: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 610 KiB/s wr, 15 op/s 2024-03-19T01:26:13.711 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-03-19T01:26:13.712 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:26:13.712 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 47634, 2024-03-19T01:26:13.712 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 56709, 2024-03-19T01:26:13.712 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 17210924, 2024-03-19T01:26:13.712 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 33978918, 2024-03-19T01:26:13.712 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 49, 2024-03-19T01:26:13.712 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 7, 2024-03-19T01:26:13.713 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 2.587937532, 2024-03-19T01:26:13.713 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.934713283, 2024-03-19T01:26:13.713 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 1.210701175, 2024-03-19T01:26:13.713 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.594316422, 2024-03-19T01:26:13.713 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:13.713 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 56706, 2024-03-19T01:26:13.713 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 2.382614087, 2024-03-19T01:26:13.713 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000042016 2024-03-19T01:26:13.714 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.714 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:13.714 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 25851, 2024-03-19T01:26:13.714 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.006101217, 2024-03-19T01:26:13.714 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000236 2024-03-19T01:26:13.714 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.714 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 2194288, 2024-03-19T01:26:13.714 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 591584 2024-03-19T01:26:13.714 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.715 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:26:13.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 35742, 2024-03-19T01:26:13.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 36039, 2024-03-19T01:26:13.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 9278846, 2024-03-19T01:26:13.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 30016879, 2024-03-19T01:26:13.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 14, 2024-03-19T01:26:13.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 7, 2024-03-19T01:26:13.715 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 2.482713518, 2024-03-19T01:26:13.716 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 1.070022359, 2024-03-19T01:26:13.716 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.515697611, 2024-03-19T01:26:13.716 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.747812391, 2024-03-19T01:26:13.716 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:13.716 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 36036, 2024-03-19T01:26:13.716 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 1.056910163, 2024-03-19T01:26:13.716 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000029329 2024-03-19T01:26:13.716 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.716 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:13.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1522, 2024-03-19T01:26:13.717 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000253551, 2024-03-19T01:26:13.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000166 2024-03-19T01:26:13.717 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.717 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-03-19T01:26:13.717 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 663296 2024-03-19T01:26:13.717 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.717 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:26:13.717 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 25127, 2024-03-19T01:26:13.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 22786, 2024-03-19T01:26:13.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 10885067, 2024-03-19T01:26:13.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 15715208, 2024-03-19T01:26:13.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 15, 2024-03-19T01:26:13.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 6, 2024-03-19T01:26:13.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 1.294293256, 2024-03-19T01:26:13.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.450035807, 2024-03-19T01:26:13.718 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.629868481, 2024-03-19T01:26:13.719 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.265784122, 2024-03-19T01:26:13.719 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:13.719 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 22787, 2024-03-19T01:26:13.719 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.703104295, 2024-03-19T01:26:13.719 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000030855 2024-03-19T01:26:13.719 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.719 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:13.719 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 22068, 2024-03-19T01:26:13.720 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.003457226, 2024-03-19T01:26:13.720 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000156 2024-03-19T01:26:13.720 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.720 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 7152, 2024-03-19T01:26:13.720 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 18864 2024-03-19T01:26:13.720 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.720 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-03-19T01:26:13.720 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2024-03-19T01:26:13.720 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-03-19T01:26:13.721 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.721 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2024-03-19T01:26:13.721 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-03-19T01:26:13.721 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-03-19T01:26:13.721 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 61080, 2024-03-19T01:26:13.721 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 17.082083785, 2024-03-19T01:26:13.721 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000279667 2024-03-19T01:26:13.721 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:13.722 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.722 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2024-03-19T01:26:13.722 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-03-19T01:26:13.722 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-03-19T01:26:13.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 341, 2024-03-19T01:26:13.722 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.019626985, 2024-03-19T01:26:13.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000057557 2024-03-19T01:26:13.722 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:13.723 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.723 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2024-03-19T01:26:13.723 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 4183, 2024-03-19T01:26:13.723 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 3993, 2024-03-19T01:26:13.723 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2024-03-19T01:26:13.723 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3993, 2024-03-19T01:26:13.723 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 8.902437003, 2024-03-19T01:26:13.723 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.002229510 2024-03-19T01:26:13.724 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.724 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2024-03-19T01:26:13.724 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 190, 2024-03-19T01:26:13.724 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 7, 2024-03-19T01:26:13.724 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 0, 2024-03-19T01:26:13.724 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 83, 2024-03-19T01:26:13.724 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 8, 2024-03-19T01:26:13.724 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2024-03-19T01:26:13.725 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 8537, 2024-03-19T01:26:13.725 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 159, 2024-03-19T01:26:13.725 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 7621, 2024-03-19T01:26:13.725 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 757, 2024-03-19T01:26:13.725 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 8310, 2024-03-19T01:26:13.725 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 374, 2024-03-19T01:26:13.725 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 6342, 2024-03-19T01:26:13.725 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 6342, 2024-03-19T01:26:13.725 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 172, 2024-03-19T01:26:13.726 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 4908, 2024-03-19T01:26:13.726 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 4013, 2024-03-19T01:26:13.726 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 169, 2024-03-19T01:26:13.726 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 189, 2024-03-19T01:26:13.726 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2024-03-19T01:26:13.726 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2024-03-19T01:26:13.726 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 0, 2024-03-19T01:26:13.726 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 0, 2024-03-19T01:26:13.727 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2024-03-19T01:26:13.727 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 25, 2024-03-19T01:26:13.727 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 374, 2024-03-19T01:26:13.727 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 170, 2024-03-19T01:26:13.727 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 4324, 2024-03-19T01:26:13.727 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2024-03-19T01:26:13.727 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 73856, 2024-03-19T01:26:13.727 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2024-03-19T01:26:13.727 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 89514, 2024-03-19T01:26:13.728 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 1636611686, 2024-03-19T01:26:13.728 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 6, 2024-03-19T01:26:13.728 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 73856, 2024-03-19T01:26:13.728 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2024-03-19T01:26:13.728 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T01:26:13.728 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2024-03-19T01:26:13.728 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 3607, 2024-03-19T01:26:13.728 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T01:26:13.728 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 3607, 2024-03-19T01:26:13.729 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 70249, 2024-03-19T01:26:13.729 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 19, 2024-03-19T01:26:13.729 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 406, 2024-03-19T01:26:13.729 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 3275, 2024-03-19T01:26:13.729 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 11, 2024-03-19T01:26:13.729 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T01:26:13.729 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T01:26:13.729 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 9082, 2024-03-19T01:26:13.729 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 8613, 2024-03-19T01:26:13.730 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 126, 2024-03-19T01:26:13.730 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 935 2024-03-19T01:26:13.730 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.730 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2024-03-19T01:26:13.730 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 0, 2024-03-19T01:26:13.730 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2024-03-19T01:26:13.730 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2024-03-19T01:26:13.730 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 173, 2024-03-19T01:26:13.731 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 48, 2024-03-19T01:26:13.731 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2024-03-19T01:26:13.731 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2024-03-19T01:26:13.731 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2024-03-19T01:26:13.731 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 120, 2024-03-19T01:26:13.731 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 120, 2024-03-19T01:26:13.731 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 0, 2024-03-19T01:26:13.731 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2024-03-19T01:26:13.732 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2024-03-19T01:26:13.732 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2024-03-19T01:26:13.732 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2024-03-19T01:26:13.732 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2024-03-19T01:26:13.732 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2024-03-19T01:26:13.732 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_path": 0, 2024-03-19T01:26:13.732 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_inode": 0, 2024-03-19T01:26:13.732 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T01:26:13.732 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 25, 2024-03-19T01:26:13.733 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2024-03-19T01:26:13.733 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 9, 2024-03-19T01:26:13.733 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2024-03-19T01:26:13.733 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2024-03-19T01:26:13.733 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.733 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2024-03-19T01:26:13.733 INFO:teuthology.orchestra.run.smithi163.stdout: "evlrg": 0, 2024-03-19T01:26:13.733 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 13123, 2024-03-19T01:26:13.734 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 0, 2024-03-19T01:26:13.734 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 0, 2024-03-19T01:26:13.734 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 13123, 2024-03-19T01:26:13.734 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2024-03-19T01:26:13.734 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 0, 2024-03-19T01:26:13.734 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 18, 2024-03-19T01:26:13.734 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 0, 2024-03-19T01:26:13.734 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 0, 2024-03-19T01:26:13.734 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 18, 2024-03-19T01:26:13.735 INFO:teuthology.orchestra.run.smithi163.stdout: "segmjr": 3, 2024-03-19T01:26:13.735 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2024-03-19T01:26:13.735 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 0, 2024-03-19T01:26:13.735 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 4194304, 2024-03-19T01:26:13.735 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 41541988, 2024-03-19T01:26:13.735 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 0, 2024-03-19T01:26:13.735 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2024-03-19T01:26:13.735 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 965, 2024-03-19T01:26:13.735 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 33.237935292, 2024-03-19T01:26:13.736 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.034443456 2024-03-19T01:26:13.736 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.736 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 0 2024-03-19T01:26:13.736 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.736 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2024-03-19T01:26:13.736 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 8456, 2024-03-19T01:26:13.736 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 8950, 2024-03-19T01:26:13.736 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 494, 2024-03-19T01:26:13.736 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 783, 2024-03-19T01:26:13.737 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 831, 2024-03-19T01:26:13.737 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 48, 2024-03-19T01:26:13.737 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 8537, 2024-03-19T01:26:13.737 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 9031, 2024-03-19T01:26:13.737 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 494, 2024-03-19T01:26:13.737 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 6342, 2024-03-19T01:26:13.737 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 9946, 2024-03-19T01:26:13.737 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 3604, 2024-03-19T01:26:13.738 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 112564, 2024-03-19T01:26:13.738 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 223612 2024-03-19T01:26:13.738 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.738 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2024-03-19T01:26:13.738 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 4714, 2024-03-19T01:26:13.738 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 603, 2024-03-19T01:26:13.738 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 4183, 2024-03-19T01:26:13.738 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 61, 2024-03-19T01:26:13.739 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 1357, 2024-03-19T01:26:13.739 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2024-03-19T01:26:13.739 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2612, 2024-03-19T01:26:13.739 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 4.564458378, 2024-03-19T01:26:13.739 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001747495 2024-03-19T01:26:13.739 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.739 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2024-03-19T01:26:13.739 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 81, 2024-03-19T01:26:13.740 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.355947203, 2024-03-19T01:26:13.740 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.004394409 2024-03-19T01:26:13.740 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.740 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2024-03-19T01:26:13.740 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.740 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.740 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.740 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.740 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2024-03-19T01:26:13.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.741 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.741 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.741 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2024-03-19T01:26:13.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 269, 2024-03-19T01:26:13.741 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.183441025, 2024-03-19T01:26:13.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000681936 2024-03-19T01:26:13.741 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.742 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2024-03-19T01:26:13.742 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.742 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.742 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.742 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.742 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2024-03-19T01:26:13.742 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.742 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.742 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.743 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.743 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2024-03-19T01:26:13.743 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.743 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.743 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.743 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.743 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2024-03-19T01:26:13.743 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.744 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.744 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.744 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.744 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2024-03-19T01:26:13.744 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.744 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.744 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2024-03-19T01:26:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2024-03-19T01:26:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 220, 2024-03-19T01:26:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.530722965, 2024-03-19T01:26:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.002412377 2024-03-19T01:26:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2024-03-19T01:26:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2024-03-19T01:26:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2024-03-19T01:26:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 97, 2024-03-19T01:26:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.048674417, 2024-03-19T01:26:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000501798 2024-03-19T01:26:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2024-03-19T01:26:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 509, 2024-03-19T01:26:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 1.440520424, 2024-03-19T01:26:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.002830099 2024-03-19T01:26:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2024-03-19T01:26:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2024-03-19T01:26:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2024-03-19T01:26:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2024-03-19T01:26:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2024-03-19T01:26:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2024-03-19T01:26:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2024-03-19T01:26:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 49, 2024-03-19T01:26:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.341038131, 2024-03-19T01:26:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.006959961 2024-03-19T01:26:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2024-03-19T01:26:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2024-03-19T01:26:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2024-03-19T01:26:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2024-03-19T01:26:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2024-03-19T01:26:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 36, 2024-03-19T01:26:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.245770251, 2024-03-19T01:26:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.006826951 2024-03-19T01:26:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2024-03-19T01:26:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 120, 2024-03-19T01:26:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 1.191864209, 2024-03-19T01:26:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.009932201 2024-03-19T01:26:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2024-03-19T01:26:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2024-03-19T01:26:13.757 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2024-03-19T01:26:13.757 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.757 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.757 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.757 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:13.757 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.757 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2024-03-19T01:26:13.757 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 2, 2024-03-19T01:26:13.757 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 5, 2024-03-19T01:26:13.758 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 3, 2024-03-19T01:26:13.758 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 2, 2024-03-19T01:26:13.758 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2024-03-19T01:26:13.758 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 261, 2024-03-19T01:26:13.758 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 130, 2024-03-19T01:26:13.758 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 934, 2024-03-19T01:26:13.758 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2024-03-19T01:26:13.758 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.758 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-03-19T01:26:13.759 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:26:13.759 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-03-19T01:26:13.759 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:26:13.759 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:26:13.759 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:26:13.759 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:26:13.759 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:26:13.759 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:26:13.760 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:26:13.760 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:26:13.760 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:26:13.760 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:26:13.760 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:26:13.760 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:26:13.760 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:26:13.760 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-03-19T01:26:13.760 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:26:13.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-03-19T01:26:13.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:26:13.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:26:13.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:26:13.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:26:13.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:26:13.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:26:13.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:26:13.761 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-03-19T01:26:13.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:26:13.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:26:13.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:26:13.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-03-19T01:26:13.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-03-19T01:26:13.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-03-19T01:26:13.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:26:13.762 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:26:13.763 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:26:13.763 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:26:13.763 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 621861, 2024-03-19T01:26:13.763 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 7184, 2024-03-19T01:26:13.763 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 1760, 2024-03-19T01:26:13.763 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 20, 2024-03-19T01:26:13.763 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-03-19T01:26:13.763 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-03-19T01:26:13.763 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:26:13.764 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-03-19T01:26:13.764 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:26:13.764 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-03-19T01:26:13.764 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 9832, 2024-03-19T01:26:13.764 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 205, 2024-03-19T01:26:13.764 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:26:13.764 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:26:13.764 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-03-19T01:26:13.764 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-03-19T01:26:13.765 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 30040907, 2024-03-19T01:26:13.765 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 365773, 2024-03-19T01:26:13.765 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-03-19T01:26:13.765 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-03-19T01:26:13.765 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-03-19T01:26:13.765 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-03-19T01:26:13.765 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.765 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2024-03-19T01:26:13.765 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2024-03-19T01:26:13.766 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2024-03-19T01:26:13.766 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 75600, 2024-03-19T01:26:13.766 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 39886524, 2024-03-19T01:26:13.766 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 29, 2024-03-19T01:26:13.766 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 75600, 2024-03-19T01:26:13.766 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2024-03-19T01:26:13.766 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 75600, 2024-03-19T01:26:13.766 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 124.791085901, 2024-03-19T01:26:13.767 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001650675 2024-03-19T01:26:13.767 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.767 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2024-03-19T01:26:13.767 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2024-03-19T01:26:13.767 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 75600, 2024-03-19T01:26:13.767 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 76012 2024-03-19T01:26:13.767 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.767 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 75600, 2024-03-19T01:26:13.767 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 73890, 2024-03-19T01:26:13.768 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 1710, 2024-03-19T01:26:13.768 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2024-03-19T01:26:13.768 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2024-03-19T01:26:13.768 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 0, 2024-03-19T01:26:13.768 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 10, 2024-03-19T01:26:13.768 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 27, 2024-03-19T01:26:13.768 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 992, 2024-03-19T01:26:13.768 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 79, 2024-03-19T01:26:13.769 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2024-03-19T01:26:13.769 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2024-03-19T01:26:13.769 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2024-03-19T01:26:13.769 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 8, 2024-03-19T01:26:13.769 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 260, 2024-03-19T01:26:13.769 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2024-03-19T01:26:13.769 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2024-03-19T01:26:13.769 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2024-03-19T01:26:13.769 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 73856, 2024-03-19T01:26:13.770 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 20, 2024-03-19T01:26:13.770 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2024-03-19T01:26:13.770 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2024-03-19T01:26:13.770 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2024-03-19T01:26:13.770 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2024-03-19T01:26:13.770 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2024-03-19T01:26:13.770 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2024-03-19T01:26:13.770 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T01:26:13.771 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2024-03-19T01:26:13.771 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2024-03-19T01:26:13.771 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 11, 2024-03-19T01:26:13.771 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2024-03-19T01:26:13.771 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2024-03-19T01:26:13.771 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2024-03-19T01:26:13.771 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2024-03-19T01:26:13.771 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2024-03-19T01:26:13.771 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2024-03-19T01:26:13.772 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2024-03-19T01:26:13.772 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2024-03-19T01:26:13.772 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2024-03-19T01:26:13.772 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2024-03-19T01:26:13.772 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2024-03-19T01:26:13.772 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2024-03-19T01:26:13.772 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2024-03-19T01:26:13.772 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2024-03-19T01:26:13.773 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2024-03-19T01:26:13.773 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2024-03-19T01:26:13.773 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 114, 2024-03-19T01:26:13.773 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2024-03-19T01:26:13.773 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 25, 2024-03-19T01:26:13.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 12, 2024-03-19T01:26:13.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 12, 2024-03-19T01:26:13.773 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 0, 2024-03-19T01:26:13.774 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2024-03-19T01:26:13.774 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 596, 2024-03-19T01:26:13.774 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 14, 2024-03-19T01:26:13.774 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 139 2024-03-19T01:26:13.774 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.774 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2024-03-19T01:26:13.774 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 1, 2024-03-19T01:26:13.774 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2024-03-19T01:26:13.774 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2024-03-19T01:26:13.775 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2024-03-19T01:26:13.775 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.775 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2024-03-19T01:26:13.775 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 0 2024-03-19T01:26:13.775 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.775 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2024-03-19T01:26:13.775 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2024-03-19T01:26:13.775 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 10, 2024-03-19T01:26:13.776 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2024-03-19T01:26:13.776 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 5, 2024-03-19T01:26:13.776 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed_ops": 240, 2024-03-19T01:26:13.776 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 120, 2024-03-19T01:26:13.776 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2024-03-19T01:26:13.776 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.776 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:26:13.776 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:13.776 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-03-19T01:26:13.777 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:13.777 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 108499, 2024-03-19T01:26:13.777 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 27790137, 2024-03-19T01:26:13.777 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:13.777 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 108499, 2024-03-19T01:26:13.777 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:13.777 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:13.777 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 108499, 2024-03-19T01:26:13.777 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 27790137, 2024-03-19T01:26:13.778 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:13.778 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.778 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.778 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.778 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:13.778 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.778 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2024-03-19T01:26:13.778 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:13.779 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-03-19T01:26:13.779 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:13.779 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:13.779 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:13.779 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:13.779 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:13.779 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 75600, 2024-03-19T01:26:13.779 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 39899524, 2024-03-19T01:26:13.779 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 75600, 2024-03-19T01:26:13.780 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 39899524, 2024-03-19T01:26:13.780 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:13.780 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.780 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.780 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.780 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:13.780 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.780 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2024-03-19T01:26:13.781 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:13.781 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2024-03-19T01:26:13.781 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:13.781 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:13.781 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:13.781 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:13.781 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:13.781 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 75600, 2024-03-19T01:26:13.781 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 75600, 2024-03-19T01:26:13.782 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 75600, 2024-03-19T01:26:13.782 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 75600, 2024-03-19T01:26:13.782 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:13.782 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.782 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.782 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.782 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:13.782 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.782 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2024-03-19T01:26:13.783 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:13.783 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-03-19T01:26:13.783 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:13.783 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 120, 2024-03-19T01:26:13.783 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 13112, 2024-03-19T01:26:13.783 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:13.783 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 120, 2024-03-19T01:26:13.783 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:13.783 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:13.784 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 27, 2024-03-19T01:26:13.784 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 13112, 2024-03-19T01:26:13.784 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:13.784 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.784 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.784 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.784 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:13.784 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:13.784 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x55562d4f2500": { 2024-03-19T01:26:13.785 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:13.785 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-03-19T01:26:13.785 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:13.785 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 13123, 2024-03-19T01:26:13.785 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 37347684, 2024-03-19T01:26:13.785 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:13.785 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 13123, 2024-03-19T01:26:13.785 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:13.786 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:13.786 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 965, 2024-03-19T01:26:13.786 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 37347684, 2024-03-19T01:26:13.786 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:13.786 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:13.786 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:13.786 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:13.786 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:13.786 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:13.787 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-03-19T01:26:14.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:14 smithi086 ceph-mon[66829]: pgmap v489: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 742 KiB/s wr, 33 op/s 2024-03-19T01:26:14.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:14 smithi069 ceph-mon[31727]: pgmap v489: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 742 KiB/s wr, 33 op/s 2024-03-19T01:26:14.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:14 smithi163 ceph-mon[67428]: pgmap v489: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 742 KiB/s wr, 33 op/s 2024-03-19T01:26:14.653 INFO:tasks.check_counter:Daemon mds.i mds.exported=25 2024-03-19T01:26:14.653 INFO:tasks.check_counter:Daemon mds.i mds.imported=170 2024-03-19T01:26:14.653 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-19T01:26:14.654 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-19T01:26:14.654 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-19T01:26:14.654 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=8 2024-03-19T01:26:14.654 DEBUG:tasks.check_counter:Getting stats from l 2024-03-19T01:26:14.654 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-19T01:26:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:26:16.412 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:16 smithi163 ceph-mon[67428]: pgmap v490: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 606 KiB/s wr, 27 op/s 2024-03-19T01:26:16.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:16 smithi086 ceph-mon[66829]: pgmap v490: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 606 KiB/s wr, 27 op/s 2024-03-19T01:26:16.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:16 smithi069 ceph-mon[31727]: pgmap v490: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 606 KiB/s wr, 27 op/s 2024-03-19T01:26:16.690 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-03-19T01:26:16.690 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T01:26:16.690 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 2000, 2024-03-19T01:26:16.690 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1998, 2024-03-19T01:26:16.691 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 85504310, 2024-03-19T01:26:16.691 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 634541, 2024-03-19T01:26:16.691 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 8, 2024-03-19T01:26:16.691 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 4, 2024-03-19T01:26:16.692 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.177543194, 2024-03-19T01:26:16.692 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.036399504, 2024-03-19T01:26:16.692 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.083732493, 2024-03-19T01:26:16.692 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.045575754, 2024-03-19T01:26:16.693 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:16.693 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1997, 2024-03-19T01:26:16.693 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.050401552, 2024-03-19T01:26:16.693 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000025238 2024-03-19T01:26:16.693 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.694 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:16.694 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.694 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.694 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.695 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.695 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-19T01:26:16.695 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 448 2024-03-19T01:26:16.695 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.695 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T01:26:16.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1562, 2024-03-19T01:26:16.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1518, 2024-03-19T01:26:16.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 82136654, 2024-03-19T01:26:16.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 573263, 2024-03-19T01:26:16.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 6, 2024-03-19T01:26:16.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 5, 2024-03-19T01:26:16.696 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.155124229, 2024-03-19T01:26:16.697 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.033613873, 2024-03-19T01:26:16.697 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.103938829, 2024-03-19T01:26:16.697 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.026675931, 2024-03-19T01:26:16.697 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:16.697 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1517, 2024-03-19T01:26:16.697 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.037547115, 2024-03-19T01:26:16.697 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000024750 2024-03-19T01:26:16.697 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.697 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:16.698 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.698 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.698 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.698 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.698 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 2126912, 2024-03-19T01:26:16.699 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 200192 2024-03-19T01:26:16.699 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.699 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T01:26:16.699 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 540, 2024-03-19T01:26:16.699 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 779, 2024-03-19T01:26:16.699 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 88253993, 2024-03-19T01:26:16.700 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 743255, 2024-03-19T01:26:16.700 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 8, 2024-03-19T01:26:16.700 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 6, 2024-03-19T01:26:16.700 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.081814368, 2024-03-19T01:26:16.700 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.017128356, 2024-03-19T01:26:16.700 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.049401846, 2024-03-19T01:26:16.700 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.010402383, 2024-03-19T01:26:16.700 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T01:26:16.700 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 776, 2024-03-19T01:26:16.701 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.041641841, 2024-03-19T01:26:16.701 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000053662 2024-03-19T01:26:16.701 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.701 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-03-19T01:26:16.701 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.701 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.701 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.701 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.701 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-19T01:26:16.702 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 572160 2024-03-19T01:26:16.702 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.702 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-03-19T01:26:16.702 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2024-03-19T01:26:16.702 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-03-19T01:26:16.702 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.702 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2024-03-19T01:26:16.702 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-03-19T01:26:16.703 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-03-19T01:26:16.703 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 6186, 2024-03-19T01:26:16.703 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.127024376, 2024-03-19T01:26:16.703 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000020534 2024-03-19T01:26:16.703 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:16.703 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.703 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2024-03-19T01:26:16.703 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-03-19T01:26:16.703 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-03-19T01:26:16.704 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.704 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.704 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.704 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:16.704 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.704 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2024-03-19T01:26:16.704 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 0, 2024-03-19T01:26:16.704 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 0, 2024-03-19T01:26:16.704 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2024-03-19T01:26:16.705 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.705 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.705 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.705 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.705 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2024-03-19T01:26:16.705 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 0, 2024-03-19T01:26:16.705 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 0, 2024-03-19T01:26:16.705 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 0, 2024-03-19T01:26:16.706 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 0, 2024-03-19T01:26:16.706 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 0, 2024-03-19T01:26:16.706 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2024-03-19T01:26:16.706 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 40315, 2024-03-19T01:26:16.706 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 0, 2024-03-19T01:26:16.706 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 40315, 2024-03-19T01:26:16.706 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 0, 2024-03-19T01:26:16.706 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 40315, 2024-03-19T01:26:16.707 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 0, 2024-03-19T01:26:16.707 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 0, 2024-03-19T01:26:16.707 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 0, 2024-03-19T01:26:16.707 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 256, 2024-03-19T01:26:16.707 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 0, 2024-03-19T01:26:16.707 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 0, 2024-03-19T01:26:16.707 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 0, 2024-03-19T01:26:16.707 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 0, 2024-03-19T01:26:16.707 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2024-03-19T01:26:16.708 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2024-03-19T01:26:16.708 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 0, 2024-03-19T01:26:16.708 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 0, 2024-03-19T01:26:16.708 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2024-03-19T01:26:16.708 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 0, 2024-03-19T01:26:16.708 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 0, 2024-03-19T01:26:16.708 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 0, 2024-03-19T01:26:16.708 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 0, 2024-03-19T01:26:16.709 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2024-03-19T01:26:16.709 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 0, 2024-03-19T01:26:16.709 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2024-03-19T01:26:16.709 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 0, 2024-03-19T01:26:16.709 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 0, 2024-03-19T01:26:16.709 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 0, 2024-03-19T01:26:16.709 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T01:26:16.709 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2024-03-19T01:26:16.709 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T01:26:16.710 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2024-03-19T01:26:16.710 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 0, 2024-03-19T01:26:16.710 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T01:26:16.710 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T01:26:16.710 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 0, 2024-03-19T01:26:16.710 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 0, 2024-03-19T01:26:16.710 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T01:26:16.710 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 0, 2024-03-19T01:26:16.711 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T01:26:16.711 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T01:26:16.711 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T01:26:16.711 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 0, 2024-03-19T01:26:16.711 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 0, 2024-03-19T01:26:16.711 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 0, 2024-03-19T01:26:16.711 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 0 2024-03-19T01:26:16.711 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2024-03-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 0, 2024-03-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2024-03-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2024-03-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 0, 2024-03-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 0, 2024-03-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2024-03-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2024-03-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2024-03-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 0, 2024-03-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 0, 2024-03-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 0, 2024-03-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2024-03-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2024-03-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2024-03-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2024-03-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2024-03-19T01:26:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2024-03-19T01:26:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_path": 0, 2024-03-19T01:26:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_inode": 0, 2024-03-19T01:26:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T01:26:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 0, 2024-03-19T01:26:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2024-03-19T01:26:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 0, 2024-03-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2024-03-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2024-03-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2024-03-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: "evlrg": 0, 2024-03-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 0, 2024-03-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 0, 2024-03-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 0, 2024-03-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 93178, 2024-03-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2024-03-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 0, 2024-03-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 0, 2024-03-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 0, 2024-03-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 0, 2024-03-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 115, 2024-03-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi163.stdout: "segmjr": 9, 2024-03-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2024-03-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 0, 2024-03-19T01:26:16.717 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 4194304, 2024-03-19T01:26:16.717 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 256843200, 2024-03-19T01:26:16.717 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 256839893, 2024-03-19T01:26:16.717 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2024-03-19T01:26:16.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.717 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.717 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.718 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 93178 2024-03-19T01:26:16.718 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.718 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2024-03-19T01:26:16.718 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 37830, 2024-03-19T01:26:16.718 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 43436, 2024-03-19T01:26:16.718 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 5606, 2024-03-19T01:26:16.718 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 2176, 2024-03-19T01:26:16.718 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 2579, 2024-03-19T01:26:16.719 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 403, 2024-03-19T01:26:16.719 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 40315, 2024-03-19T01:26:16.719 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 44677, 2024-03-19T01:26:16.719 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 4362, 2024-03-19T01:26:16.719 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 0, 2024-03-19T01:26:16.719 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 0, 2024-03-19T01:26:16.719 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 0, 2024-03-19T01:26:16.719 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 193904, 2024-03-19T01:26:16.719 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 215420 2024-03-19T01:26:16.720 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.720 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2024-03-19T01:26:16.720 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 0, 2024-03-19T01:26:16.720 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 0, 2024-03-19T01:26:16.720 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 0, 2024-03-19T01:26:16.720 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 0, 2024-03-19T01:26:16.720 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 0, 2024-03-19T01:26:16.720 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2024-03-19T01:26:16.720 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.721 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.721 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.721 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.721 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2024-03-19T01:26:16.721 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.721 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.721 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.721 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.722 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2024-03-19T01:26:16.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.722 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.722 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.722 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2024-03-19T01:26:16.722 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.722 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.723 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.723 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.723 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2024-03-19T01:26:16.723 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.723 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.723 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.723 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.723 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2024-03-19T01:26:16.724 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.724 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.724 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.724 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.724 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2024-03-19T01:26:16.724 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.724 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.724 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.724 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.725 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2024-03-19T01:26:16.725 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.725 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.725 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.725 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.725 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2024-03-19T01:26:16.725 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.725 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.725 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.726 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.726 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2024-03-19T01:26:16.726 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.726 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.726 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.726 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.726 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2024-03-19T01:26:16.726 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.727 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.727 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.727 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.728 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2024-03-19T01:26:16.728 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.728 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.728 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.728 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.728 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2024-03-19T01:26:16.728 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.728 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.729 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.729 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.729 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2024-03-19T01:26:16.729 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.729 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.729 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.729 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.729 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2024-03-19T01:26:16.730 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.730 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.730 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.730 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.730 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2024-03-19T01:26:16.730 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.730 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.731 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.731 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.731 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2024-03-19T01:26:16.731 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.731 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.731 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.731 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.731 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2024-03-19T01:26:16.731 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.732 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.732 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.732 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.732 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2024-03-19T01:26:16.732 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.732 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.732 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.732 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.733 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2024-03-19T01:26:16.733 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.733 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.733 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.733 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.733 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2024-03-19T01:26:16.733 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.733 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.733 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.734 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.734 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2024-03-19T01:26:16.734 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.734 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.734 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.734 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.734 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2024-03-19T01:26:16.734 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.735 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.735 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.735 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.735 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2024-03-19T01:26:16.735 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.735 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.735 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.735 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.736 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2024-03-19T01:26:16.736 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.736 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.736 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.736 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.736 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2024-03-19T01:26:16.736 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.736 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.736 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2024-03-19T01:26:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2024-03-19T01:26:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2024-03-19T01:26:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2024-03-19T01:26:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2024-03-19T01:26:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2024-03-19T01:26:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2024-03-19T01:26:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 1, 2024-03-19T01:26:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 4, 2024-03-19T01:26:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 3, 2024-03-19T01:26:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 3, 2024-03-19T01:26:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2024-03-19T01:26:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 0, 2024-03-19T01:26:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 0, 2024-03-19T01:26:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 12422, 2024-03-19T01:26:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2024-03-19T01:26:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-03-19T01:26:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-03-19T01:26:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-03-19T01:26:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T01:26:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-03-19T01:26:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T01:26:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-03-19T01:26:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T01:26:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T01:26:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T01:26:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T01:26:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T01:26:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-03-19T01:26:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T01:26:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T01:26:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-03-19T01:26:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-03-19T01:26:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-03-19T01:26:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-03-19T01:26:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T01:26:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T01:26:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T01:26:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T01:26:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T01:26:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-03-19T01:26:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-03-19T01:26:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-03-19T01:26:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T01:26:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T01:26:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-03-19T01:26:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-03-19T01:26:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-03-19T01:26:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-03-19T01:26:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T01:26:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-03-19T01:26:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T01:26:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-03-19T01:26:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 1806590, 2024-03-19T01:26:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 42435, 2024-03-19T01:26:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2024-03-19T01:26:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2024-03-19T01:26:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-03-19T01:26:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-03-19T01:26:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-03-19T01:26:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-03-19T01:26:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-03-19T01:26:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-03-19T01:26:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 9832, 2024-03-19T01:26:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 205, 2024-03-19T01:26:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T01:26:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-03-19T01:26:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-03-19T01:26:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-03-19T01:26:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 122813853, 2024-03-19T01:26:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 1319749, 2024-03-19T01:26:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-03-19T01:26:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-03-19T01:26:16.748 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-03-19T01:26:16.748 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-03-19T01:26:16.748 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.748 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2024-03-19T01:26:16.748 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2024-03-19T01:26:16.748 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2024-03-19T01:26:16.748 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 3705, 2024-03-19T01:26:16.748 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 21, 2024-03-19T01:26:16.748 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 3, 2024-03-19T01:26:16.749 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 3705, 2024-03-19T01:26:16.749 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2024-03-19T01:26:16.749 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3705, 2024-03-19T01:26:16.749 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 44.858946609, 2024-03-19T01:26:16.749 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.012107677 2024-03-19T01:26:16.749 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.749 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2024-03-19T01:26:16.749 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2024-03-19T01:26:16.750 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3705, 2024-03-19T01:26:16.750 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 3706 2024-03-19T01:26:16.750 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.750 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 3705, 2024-03-19T01:26:16.750 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 3705, 2024-03-19T01:26:16.750 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 0, 2024-03-19T01:26:16.750 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2024-03-19T01:26:16.751 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2024-03-19T01:26:16.751 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 2161, 2024-03-19T01:26:16.751 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 0, 2024-03-19T01:26:16.751 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 1542, 2024-03-19T01:26:16.751 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 0, 2024-03-19T01:26:16.751 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 0, 2024-03-19T01:26:16.751 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2024-03-19T01:26:16.751 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2024-03-19T01:26:16.751 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2024-03-19T01:26:16.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 0, 2024-03-19T01:26:16.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 0, 2024-03-19T01:26:16.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2024-03-19T01:26:16.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2024-03-19T01:26:16.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2024-03-19T01:26:16.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 1, 2024-03-19T01:26:16.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 0, 2024-03-19T01:26:16.752 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2024-03-19T01:26:16.753 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2024-03-19T01:26:16.753 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2024-03-19T01:26:16.753 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2024-03-19T01:26:16.753 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2024-03-19T01:26:16.753 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2024-03-19T01:26:16.753 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T01:26:16.753 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2024-03-19T01:26:16.753 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2024-03-19T01:26:16.754 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 0, 2024-03-19T01:26:16.754 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2024-03-19T01:26:16.754 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2024-03-19T01:26:16.754 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2024-03-19T01:26:16.754 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2024-03-19T01:26:16.754 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2024-03-19T01:26:16.754 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2024-03-19T01:26:16.754 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2024-03-19T01:26:16.754 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2024-03-19T01:26:16.755 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2024-03-19T01:26:16.755 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2024-03-19T01:26:16.755 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2024-03-19T01:26:16.755 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2024-03-19T01:26:16.755 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2024-03-19T01:26:16.755 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2024-03-19T01:26:16.755 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2024-03-19T01:26:16.755 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2024-03-19T01:26:16.756 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 114, 2024-03-19T01:26:16.756 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2024-03-19T01:26:16.756 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 25, 2024-03-19T01:26:16.756 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 11, 2024-03-19T01:26:16.756 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 11, 2024-03-19T01:26:16.756 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 0, 2024-03-19T01:26:16.756 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2024-03-19T01:26:16.756 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 0, 2024-03-19T01:26:16.756 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 2, 2024-03-19T01:26:16.757 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 0 2024-03-19T01:26:16.757 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.757 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2024-03-19T01:26:16.757 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 0, 2024-03-19T01:26:16.757 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2024-03-19T01:26:16.757 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2024-03-19T01:26:16.757 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2024-03-19T01:26:16.757 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.757 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2024-03-19T01:26:16.758 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 0 2024-03-19T01:26:16.758 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.758 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2024-03-19T01:26:16.758 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2024-03-19T01:26:16.758 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T01:26:16.758 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2024-03-19T01:26:16.758 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 0, 2024-03-19T01:26:16.758 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed_ops": 0, 2024-03-19T01:26:16.759 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 0, 2024-03-19T01:26:16.759 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2024-03-19T01:26:16.759 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.759 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T01:26:16.759 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:16.759 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-03-19T01:26:16.759 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:16.759 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 4098, 2024-03-19T01:26:16.759 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 255503379, 2024-03-19T01:26:16.760 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:16.760 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 4098, 2024-03-19T01:26:16.760 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:16.760 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:16.760 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 4098, 2024-03-19T01:26:16.760 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 255503379, 2024-03-19T01:26:16.760 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:16.760 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.760 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.761 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.761 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:16.761 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.761 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2024-03-19T01:26:16.761 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:16.761 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-03-19T01:26:16.762 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:16.762 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:16.762 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:16.762 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:16.762 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:16.762 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 3705, 2024-03-19T01:26:16.762 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 252731381, 2024-03-19T01:26:16.762 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3705, 2024-03-19T01:26:16.763 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 252731381, 2024-03-19T01:26:16.763 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:16.763 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.763 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.763 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.763 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:16.763 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.763 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2024-03-19T01:26:16.764 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:16.764 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2024-03-19T01:26:16.764 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:16.764 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:16.764 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:16.764 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:16.764 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:16.764 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 3705, 2024-03-19T01:26:16.764 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 3705, 2024-03-19T01:26:16.765 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 3705, 2024-03-19T01:26:16.765 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 3705, 2024-03-19T01:26:16.765 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:16.765 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.765 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.765 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.765 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:16.765 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.765 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2024-03-19T01:26:16.766 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:16.766 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-03-19T01:26:16.766 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:16.766 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:16.766 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:16.766 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:16.766 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:16.766 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:16.767 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:16.767 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-03-19T01:26:16.767 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-03-19T01:26:16.767 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:16.767 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.767 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.767 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.767 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:16.767 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-03-19T01:26:16.768 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x563999dd06e0": { 2024-03-19T01:26:16.768 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-03-19T01:26:16.768 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-03-19T01:26:16.768 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-03-19T01:26:16.768 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-03-19T01:26:16.768 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-03-19T01:26:16.768 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-03-19T01:26:16.768 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-03-19T01:26:16.768 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-03-19T01:26:16.769 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-03-19T01:26:16.769 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-03-19T01:26:16.769 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-03-19T01:26:16.769 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-03-19T01:26:16.769 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-03-19T01:26:16.769 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-03-19T01:26:16.769 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-03-19T01:26:16.769 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:16.769 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-03-19T01:26:16.770 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-03-19T01:26:17.545 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-19T01:26:17.545 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-19T01:26:17.546 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-19T01:26:17.546 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-19T01:26:17.546 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-19T01:26:17.546 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-03-19T01:26:17.546 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-19T01:26:17.557 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-19T01:26:17.557 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T01:26:17.557 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T01:26:17.665 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-19T01:26:17.665 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T01:26:17.721 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:26:17.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.931+0000 7f6c6fc00640 1 -- 172.21.15.69:0/3565185288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c6810e070 msgr2=0x7f6c68110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:17.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.931+0000 7f6c6fc00640 1 --2- 172.21.15.69:0/3565185288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c6810e070 0x7f6c68110460 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009ca0 tx=0x7f6c6002c180 comp rx=0 tx=0).stop 2024-03-19T01:26:17.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.932+0000 7f6c6fc00640 1 -- 172.21.15.69:0/3565185288 shutdown_connections 2024-03-19T01:26:17.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.932+0000 7f6c6fc00640 1 --2- 172.21.15.69:0/3565185288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c6810e070 0x7f6c68110460 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:17.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.932+0000 7f6c6fc00640 1 --2- 172.21.15.69:0/3565185288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c680744f0 0x7f6c6810db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:17.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.932+0000 7f6c6fc00640 1 --2- 172.21.15.69:0/3565185288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c68073bd0 0x7f6c68073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:17.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.932+0000 7f6c6fc00640 1 -- 172.21.15.69:0/3565185288 >> 172.21.15.69:0/3565185288 conn(0x7f6c680fd2c0 msgr2=0x7f6c680ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:17.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.932+0000 7f6c6fc00640 1 -- 172.21.15.69:0/3565185288 shutdown_connections 2024-03-19T01:26:17.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.932+0000 7f6c6fc00640 1 -- 172.21.15.69:0/3565185288 wait complete. 2024-03-19T01:26:17.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.933+0000 7f6c6fc00640 1 Processor -- start 2024-03-19T01:26:17.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.933+0000 7f6c6fc00640 1 -- start start 2024-03-19T01:26:17.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.934+0000 7f6c6fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c68073bd0 0x7f6c6819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:17.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.934+0000 7f6c6fc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c680744f0 0x7f6c6819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:17.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.934+0000 7f6c6fc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c6810e070 0x7f6c6819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:17.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.934+0000 7f6c6fc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68113d90 con 0x7f6c680744f0 2024-03-19T01:26:17.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.934+0000 7f6c6fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68113c10 con 0x7f6c68073bd0 2024-03-19T01:26:17.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.934+0000 7f6c6fc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68113f10 con 0x7f6c6810e070 2024-03-19T01:26:17.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.934+0000 7f6c6d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c68073bd0 0x7f6c6819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:17.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.934+0000 7f6c67e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c680744f0 0x7f6c6819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:17.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.934+0000 7f6c6d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c68073bd0 0x7f6c6819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58662/0 (socket says 172.21.15.69:58662) 2024-03-19T01:26:17.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.934+0000 7f6c6d200640 1 -- 172.21.15.69:0/337087215 learned_addr learned my addr 172.21.15.69:0/337087215 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:17.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.934+0000 7f6c6dc00640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c6810e070 0x7f6c6819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:17.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.935+0000 7f6c6d200640 1 -- 172.21.15.69:0/337087215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c6810e070 msgr2=0x7f6c6819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:17.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.935+0000 7f6c6d200640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c6810e070 0x7f6c6819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:17.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.935+0000 7f6c6d200640 1 -- 172.21.15.69:0/337087215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c680744f0 msgr2=0x7f6c6819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:17.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.935+0000 7f6c6d200640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c680744f0 0x7f6c6819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:17.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.935+0000 7f6c6d200640 1 -- 172.21.15.69:0/337087215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6819f2b0 con 0x7f6c68073bd0 2024-03-19T01:26:17.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.935+0000 7f6c6d200640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c68073bd0 0x7f6c6819a310 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00e9f0 tx=0x7f6c5c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:17.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.935+0000 7f6c65600640 1 -- 172.21.15.69:0/337087215 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c00cd80 con 0x7f6c68073bd0 2024-03-19T01:26:17.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.935+0000 7f6c65600640 1 -- 172.21.15.69:0/337087215 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c5c005500 con 0x7f6c68073bd0 2024-03-19T01:26:17.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.935+0000 7f6c6fc00640 1 -- 172.21.15.69:0/337087215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6819f5a0 con 0x7f6c68073bd0 2024-03-19T01:26:17.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.935+0000 7f6c65600640 1 -- 172.21.15.69:0/337087215 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c010690 con 0x7f6c68073bd0 2024-03-19T01:26:17.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.936+0000 7f6c6fc00640 1 -- 172.21.15.69:0/337087215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c681a6e30 con 0x7f6c68073bd0 2024-03-19T01:26:17.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.937+0000 7f6c6fc00640 1 -- 172.21.15.69:0/337087215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c68068a60 con 0x7f6c68073bd0 2024-03-19T01:26:17.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.940+0000 7f6c65600640 1 -- 172.21.15.69:0/337087215 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f6c5c0046c0 con 0x7f6c68073bd0 2024-03-19T01:26:17.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.941+0000 7f6c65600640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6c500a29f0 0x7f6c500a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:17.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.941+0000 7f6c65600640 1 -- 172.21.15.69:0/337087215 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9568+0+0 (secure 0 0 0) 0x7f6c5c0cf620 con 0x7f6c68073bd0 2024-03-19T01:26:17.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.942+0000 7f6c67e00640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6c500a29f0 0x7f6c500a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:17.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.942+0000 7f6c67e00640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6c500a29f0 0x7f6c500a4eb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c680ff950 tx=0x7f6c54005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:17.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:17.945+0000 7f6c65600640 1 -- 172.21.15.69:0/337087215 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6c5c09bf30 con 0x7f6c68073bd0 2024-03-19T01:26:18.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.154+0000 7f6c6fc00640 1 -- 172.21.15.69:0/337087215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c68109250 con 0x7f6c68073bd0 2024-03-19T01:26:18.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.161+0000 7f6c65600640 1 -- 172.21.15.69:0/337087215 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+25364 (secure 0 0 0) 0x7f6c5c09b830 con 0x7f6c68073bd0 2024-03-19T01:26:18.162 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:26:18.162 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":114,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:26:09.130305+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3960674803":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/3562156769":"2024-03-20T01:10:40.006231+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6801/3562156769":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2535372856":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2291738694":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2266394714":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2207308634":"2024-03-20T01:10:40.006231+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:26:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.163+0000 7f6c6fc00640 1 -- 172.21.15.69:0/337087215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6c500a29f0 msgr2=0x7f6c500a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.163+0000 7f6c6fc00640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6c500a29f0 0x7f6c500a4eb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c680ff950 tx=0x7f6c54005f90 comp rx=0 tx=0).stop 2024-03-19T01:26:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.163+0000 7f6c6fc00640 1 -- 172.21.15.69:0/337087215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c68073bd0 msgr2=0x7f6c6819a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.163+0000 7f6c6fc00640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c68073bd0 0x7f6c6819a310 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00e9f0 tx=0x7f6c5c00edf0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.163+0000 7f6c6fc00640 1 -- 172.21.15.69:0/337087215 shutdown_connections 2024-03-19T01:26:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.163+0000 7f6c6fc00640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c6810e070 0x7f6c6819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.163+0000 7f6c6fc00640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c680744f0 0x7f6c6819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.163+0000 7f6c6fc00640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f6c500a29f0 0x7f6c500a4eb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.163+0000 7f6c6fc00640 1 --2- 172.21.15.69:0/337087215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c68073bd0 0x7f6c6819a310 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.163+0000 7f6c6fc00640 1 -- 172.21.15.69:0/337087215 >> 172.21.15.69:0/337087215 conn(0x7f6c680fd2c0 msgr2=0x7f6c680feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.164+0000 7f6c6fc00640 1 -- 172.21.15.69:0/337087215 shutdown_connections 2024-03-19T01:26:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.164+0000 7f6c6fc00640 1 -- 172.21.15.69:0/337087215 wait complete. 2024-03-19T01:26:18.186 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:02:06.167894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:07:13.368130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:07:14.444206+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-19T01:07:18.189146+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-19T01:26:18.186 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:26:18.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.402+0000 7f7281800640 1 -- 172.21.15.69:0/2839410106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f727c100460 msgr2=0x7f727c100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:18.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.402+0000 7f7281800640 1 --2- 172.21.15.69:0/2839410106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f727c100460 0x7f727c100840 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f726c009900 tx=0x7f726c02bc50 comp rx=0 tx=0).stop 2024-03-19T01:26:18.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.402+0000 7f7281800640 1 -- 172.21.15.69:0/2839410106 shutdown_connections 2024-03-19T01:26:18.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.402+0000 7f7281800640 1 --2- 172.21.15.69:0/2839410106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727c10e010 0x7f727c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.402+0000 7f7281800640 1 --2- 172.21.15.69:0/2839410106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c100e10 0x7f727c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.402+0000 7f7281800640 1 --2- 172.21.15.69:0/2839410106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f727c100460 0x7f727c100840 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.402+0000 7f7281800640 1 -- 172.21.15.69:0/2839410106 >> 172.21.15.69:0/2839410106 conn(0x7f727c0fc070 msgr2=0x7f727c0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:18.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.402+0000 7f7281800640 1 -- 172.21.15.69:0/2839410106 shutdown_connections 2024-03-19T01:26:18.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.402+0000 7f7281800640 1 -- 172.21.15.69:0/2839410106 wait complete. 2024-03-19T01:26:18.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.404+0000 7f7281800640 1 Processor -- start 2024-03-19T01:26:18.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.404+0000 7f7281800640 1 -- start start 2024-03-19T01:26:18.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.404+0000 7f7281800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f727c100460 0x7f727c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:18.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.404+0000 7f7281800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c100e10 0x7f727c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:18.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.404+0000 7f7281800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727c10e010 0x7f727c19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:18.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.404+0000 7f7281800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f727c113ca0 con 0x7f727c100460 2024-03-19T01:26:18.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.404+0000 7f7281800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f727c113b20 con 0x7f727c100e10 2024-03-19T01:26:18.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.404+0000 7f7281800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f727c113e20 con 0x7f727c10e010 2024-03-19T01:26:18.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f727b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c100e10 0x7f727c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:18.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f7280e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727c10e010 0x7f727c19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:18.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f727be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f727c100460 0x7f727c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:18.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f7280e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727c10e010 0x7f727c19eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48274/0 (socket says 172.21.15.69:48274) 2024-03-19T01:26:18.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f7280e00640 1 -- 172.21.15.69:0/3171302511 learned_addr learned my addr 172.21.15.69:0/3171302511 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:18.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f727b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c100e10 0x7f727c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58688/0 (socket says 172.21.15.69:58688) 2024-03-19T01:26:18.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f7280e00640 1 -- 172.21.15.69:0/3171302511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c100e10 msgr2=0x7f727c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:18.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f7280e00640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c100e10 0x7f727c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f7280e00640 1 -- 172.21.15.69:0/3171302511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f727c100460 msgr2=0x7f727c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:18.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f7280e00640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f727c100460 0x7f727c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f7280e00640 1 -- 172.21.15.69:0/3171302511 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c19f1e0 con 0x7f727c10e010 2024-03-19T01:26:18.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f727b400640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c100e10 0x7f727c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:26:18.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.405+0000 7f7280e00640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727c10e010 0x7f727c19eab0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f727400eb10 tx=0x7f727400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:18.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.406+0000 7f7278c00640 1 -- 172.21.15.69:0/3171302511 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274004b20 con 0x7f727c10e010 2024-03-19T01:26:18.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.406+0000 7f7278c00640 1 -- 172.21.15.69:0/3171302511 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7274004cc0 con 0x7f727c10e010 2024-03-19T01:26:18.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.406+0000 7f7278c00640 1 -- 172.21.15.69:0/3171302511 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274010790 con 0x7f727c10e010 2024-03-19T01:26:18.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.406+0000 7f7281800640 1 -- 172.21.15.69:0/3171302511 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c19f4d0 con 0x7f727c10e010 2024-03-19T01:26:18.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.406+0000 7f7281800640 1 -- 172.21.15.69:0/3171302511 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c1a6d60 con 0x7f727c10e010 2024-03-19T01:26:18.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.407+0000 7f7281800640 1 -- 172.21.15.69:0/3171302511 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727c068530 con 0x7f727c10e010 2024-03-19T01:26:18.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.409+0000 7f7278c00640 1 -- 172.21.15.69:0/3171302511 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f7274005020 con 0x7f727c10e010 2024-03-19T01:26:18.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.410+0000 7f7278c00640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72600a29f0 0x7f72600a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:18.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.410+0000 7f7278c00640 1 -- 172.21.15.69:0/3171302511 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9568+0+0 (secure 0 0 0) 0x7f72740cf900 con 0x7f727c10e010 2024-03-19T01:26:18.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.410+0000 7f727be00640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72600a29f0 0x7f72600a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:18.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.411+0000 7f727be00640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72600a29f0 0x7f72600a4eb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f726c006010 tx=0x7f726c005b50 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:18.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.414+0000 7f7278c00640 1 -- 172.21.15.69:0/3171302511 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f727409c210 con 0x7f727c10e010 2024-03-19T01:26:18.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:18 smithi086 ceph-mon[66829]: pgmap v491: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 162 KiB/s wr, 23 op/s 2024-03-19T01:26:18.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:18 smithi069 ceph-mon[31727]: pgmap v491: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 162 KiB/s wr, 23 op/s 2024-03-19T01:26:18.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:18 smithi163 ceph-mon[67428]: pgmap v491: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 162 KiB/s wr, 23 op/s 2024-03-19T01:26:18.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.645+0000 7f7281800640 1 -- 172.21.15.69:0/3171302511 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f727c109140 con 0x7f727c10e010 2024-03-19T01:26:18.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.646+0000 7f7278c00640 1 -- 172.21.15.69:0/3171302511 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7f727409bb10 con 0x7f727c10e010 2024-03-19T01:26:18.646 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-03-19T01:26:18.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.649+0000 7f7281800640 1 -- 172.21.15.69:0/3171302511 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72600a29f0 msgr2=0x7f72600a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:18.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.649+0000 7f7281800640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72600a29f0 0x7f72600a4eb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f726c006010 tx=0x7f726c005b50 comp rx=0 tx=0).stop 2024-03-19T01:26:18.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.649+0000 7f7281800640 1 -- 172.21.15.69:0/3171302511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727c10e010 msgr2=0x7f727c19eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:18.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.649+0000 7f7281800640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727c10e010 0x7f727c19eab0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f727400eb10 tx=0x7f727400c2d0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.649+0000 7f7281800640 1 -- 172.21.15.69:0/3171302511 shutdown_connections 2024-03-19T01:26:18.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.649+0000 7f7281800640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727c10e010 0x7f727c19eab0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.649+0000 7f7281800640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f72600a29f0 0x7f72600a4eb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.649+0000 7f7281800640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f727c100e10 0x7f727c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.649+0000 7f7281800640 1 --2- 172.21.15.69:0/3171302511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f727c100460 0x7f727c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.649+0000 7f7281800640 1 -- 172.21.15.69:0/3171302511 >> 172.21.15.69:0/3171302511 conn(0x7f727c0fc070 msgr2=0x7f727c10f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:18.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.650+0000 7f7281800640 1 -- 172.21.15.69:0/3171302511 shutdown_connections 2024-03-19T01:26:18.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.650+0000 7f7281800640 1 -- 172.21.15.69:0/3171302511 wait complete. 2024-03-19T01:26:18.672 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:26:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.874+0000 7f8845e00640 1 -- 172.21.15.69:0/493661205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8840101790 msgr2=0x7f8840101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.874+0000 7f8845e00640 1 --2- 172.21.15.69:0/493661205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8840101790 0x7f8840101c10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f882c009930 tx=0x7f882c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:26:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.874+0000 7f8845e00640 1 -- 172.21.15.69:0/493661205 shutdown_connections 2024-03-19T01:26:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.874+0000 7f8845e00640 1 --2- 172.21.15.69:0/493661205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8840102150 0x7f884010e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.874+0000 7f8845e00640 1 --2- 172.21.15.69:0/493661205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8840101790 0x7f8840101c10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.874+0000 7f8845e00640 1 --2- 172.21.15.69:0/493661205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88401077a0 0x7f8840107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.874+0000 7f8845e00640 1 -- 172.21.15.69:0/493661205 >> 172.21.15.69:0/493661205 conn(0x7f88400fd280 msgr2=0x7f88400ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:18.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.874+0000 7f8845e00640 1 -- 172.21.15.69:0/493661205 shutdown_connections 2024-03-19T01:26:18.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.874+0000 7f8845e00640 1 -- 172.21.15.69:0/493661205 wait complete. 2024-03-19T01:26:18.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.875+0000 7f8845e00640 1 Processor -- start 2024-03-19T01:26:18.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.875+0000 7f8845e00640 1 -- start start 2024-03-19T01:26:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f8845e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8840101790 0x7f884019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f8845e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8840102150 0x7f884019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f8845e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88401077a0 0x7f884019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f8845e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8840111c00 con 0x7f8840101790 2024-03-19T01:26:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f8845e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8840111a80 con 0x7f88401077a0 2024-03-19T01:26:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f8845e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8840111d80 con 0x7f8840102150 2024-03-19T01:26:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f883e000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8840102150 0x7f884019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:18.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f883ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8840101790 0x7f884019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f883e000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8840102150 0x7f884019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48292/0 (socket says 172.21.15.69:48292) 2024-03-19T01:26:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f883e000640 1 -- 172.21.15.69:0/222496546 learned_addr learned my addr 172.21.15.69:0/222496546 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f883f400640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88401077a0 0x7f884019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f883e000640 1 -- 172.21.15.69:0/222496546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88401077a0 msgr2=0x7f884019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f883e000640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88401077a0 0x7f884019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f883e000640 1 -- 172.21.15.69:0/222496546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8840101790 msgr2=0x7f884019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f883e000640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8840101790 0x7f884019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:18.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.876+0000 7f883e000640 1 -- 172.21.15.69:0/222496546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884019f370 con 0x7f8840102150 2024-03-19T01:26:18.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.877+0000 7f883e000640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8840102150 0x7f884019a8c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f882c009900 tx=0x7f882c004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:18.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.877+0000 7f8844a00640 1 -- 172.21.15.69:0/222496546 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c043020 con 0x7f8840102150 2024-03-19T01:26:18.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.877+0000 7f8844a00640 1 -- 172.21.15.69:0/222496546 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882c0054e0 con 0x7f8840102150 2024-03-19T01:26:18.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.877+0000 7f8845e00640 1 -- 172.21.15.69:0/222496546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884019f600 con 0x7f8840102150 2024-03-19T01:26:18.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.877+0000 7f8844a00640 1 -- 172.21.15.69:0/222496546 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c003bb0 con 0x7f8840102150 2024-03-19T01:26:18.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.877+0000 7f8845e00640 1 -- 172.21.15.69:0/222496546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88401a6ef0 con 0x7f8840102150 2024-03-19T01:26:18.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.878+0000 7f8845e00640 1 -- 172.21.15.69:0/222496546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8840068a60 con 0x7f8840102150 2024-03-19T01:26:18.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.881+0000 7f8844a00640 1 -- 172.21.15.69:0/222496546 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f882c046020 con 0x7f8840102150 2024-03-19T01:26:18.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.882+0000 7f8844a00640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88240a29f0 0x7f88240a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:18.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.883+0000 7f8844a00640 1 -- 172.21.15.69:0/222496546 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9568+0+0 (secure 0 0 0) 0x7f882c0f10c0 con 0x7f8840102150 2024-03-19T01:26:18.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.883+0000 7f883ea00640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88240a29f0 0x7f88240a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:18.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.883+0000 7f883ea00640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88240a29f0 0x7f88240a4eb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8830009b50 tx=0x7f8830005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:18.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:18.886+0000 7f8844a00640 1 -- 172.21.15.69:0/222496546 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f882c04d020 con 0x7f8840102150 2024-03-19T01:26:19.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.087+0000 7f8845e00640 1 -- 172.21.15.69:0/222496546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f88401a7150 con 0x7f8840102150 2024-03-19T01:26:19.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.088+0000 7f8844a00640 1 -- 172.21.15.69:0/222496546 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) ==== 104+0+11 (secure 0 0 0) 0x7f882c0bdb20 con 0x7f8840102150 2024-03-19T01:26:19.088 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:26:19.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.091+0000 7f8845e00640 1 -- 172.21.15.69:0/222496546 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88240a29f0 msgr2=0x7f88240a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.091+0000 7f8845e00640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88240a29f0 0x7f88240a4eb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8830009b50 tx=0x7f8830005f90 comp rx=0 tx=0).stop 2024-03-19T01:26:19.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.091+0000 7f8845e00640 1 -- 172.21.15.69:0/222496546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8840102150 msgr2=0x7f884019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.091+0000 7f8845e00640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8840102150 0x7f884019a8c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f882c009900 tx=0x7f882c004360 comp rx=0 tx=0).stop 2024-03-19T01:26:19.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.092+0000 7f8845e00640 1 -- 172.21.15.69:0/222496546 shutdown_connections 2024-03-19T01:26:19.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.092+0000 7f8845e00640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f88240a29f0 0x7f88240a4eb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.092+0000 7f8845e00640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88401077a0 0x7f884019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.092+0000 7f8845e00640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8840102150 0x7f884019a8c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.092+0000 7f8845e00640 1 --2- 172.21.15.69:0/222496546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8840101790 0x7f884019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.092+0000 7f8845e00640 1 -- 172.21.15.69:0/222496546 >> 172.21.15.69:0/222496546 conn(0x7f88400fd280 msgr2=0x7f884010ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:19.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.092+0000 7f8845e00640 1 -- 172.21.15.69:0/222496546 shutdown_connections 2024-03-19T01:26:19.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.092+0000 7f8845e00640 1 -- 172.21.15.69:0/222496546 wait complete. 2024-03-19T01:26:19.110 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:26:19.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.321+0000 7f9bb7600640 1 -- 172.21.15.69:0/2067019457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb0075d50 msgr2=0x7f9bb0076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.321+0000 7f9bb7600640 1 --2- 172.21.15.69:0/2067019457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb0075d50 0x7f9bb0076130 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009930 tx=0x7f9ba402bc20 comp rx=0 tx=0).stop 2024-03-19T01:26:19.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.321+0000 7f9bb7600640 1 -- 172.21.15.69:0/2067019457 shutdown_connections 2024-03-19T01:26:19.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.321+0000 7f9bb7600640 1 --2- 172.21.15.69:0/2067019457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bb0112660 0x7f9bb0114a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.321+0000 7f9bb7600640 1 --2- 172.21.15.69:0/2067019457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb0076670 0x7f9bb0112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.321+0000 7f9bb7600640 1 --2- 172.21.15.69:0/2067019457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb0075d50 0x7f9bb0076130 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.321+0000 7f9bb7600640 1 -- 172.21.15.69:0/2067019457 >> 172.21.15.69:0/2067019457 conn(0x7f9bb00ff490 msgr2=0x7f9bb01018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:19.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.321+0000 7f9bb7600640 1 -- 172.21.15.69:0/2067019457 shutdown_connections 2024-03-19T01:26:19.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.321+0000 7f9bb7600640 1 -- 172.21.15.69:0/2067019457 wait complete. 2024-03-19T01:26:19.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.322+0000 7f9bb7600640 1 Processor -- start 2024-03-19T01:26:19.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bb7600640 1 -- start start 2024-03-19T01:26:19.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bb7600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb0075d50 0x7f9bb019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:19.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bb7600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bb0076670 0x7f9bb019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bb7600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb0112660 0x7f9bb01a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bb7600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb0118220 con 0x7f9bb0076670 2024-03-19T01:26:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bb7600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb01180a0 con 0x7f9bb0075d50 2024-03-19T01:26:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bb7600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb01183a0 con 0x7f9bb0112660 2024-03-19T01:26:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bb4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb0075d50 0x7f9bb019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bb4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb0075d50 0x7f9bb019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58730/0 (socket says 172.21.15.69:58730) 2024-03-19T01:26:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bb4c00640 1 -- 172.21.15.69:0/3126323442 learned_addr learned my addr 172.21.15.69:0/3126323442 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bafe00640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bb0076670 0x7f9bb019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:19.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.323+0000 7f9bb5600640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb0112660 0x7f9bb01a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.324+0000 7f9bb4c00640 1 -- 172.21.15.69:0/3126323442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb0112660 msgr2=0x7f9bb01a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.324+0000 7f9bb4c00640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb0112660 0x7f9bb01a2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.324+0000 7f9bb4c00640 1 -- 172.21.15.69:0/3126323442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bb0076670 msgr2=0x7f9bb019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.324+0000 7f9bb4c00640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bb0076670 0x7f9bb019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.324+0000 7f9bb4c00640 1 -- 172.21.15.69:0/3126323442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb01a3720 con 0x7f9bb0075d50 2024-03-19T01:26:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.324+0000 7f9bb4c00640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb0075d50 0x7f9bb019e780 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4005ab0 tx=0x7f9ba4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.324+0000 7f9bad600640 1 -- 172.21.15.69:0/3126323442 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4043020 con 0x7f9bb0075d50 2024-03-19T01:26:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.324+0000 7f9bad600640 1 -- 172.21.15.69:0/3126323442 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ba4005000 con 0x7f9bb0075d50 2024-03-19T01:26:19.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.324+0000 7f9bb7600640 1 -- 172.21.15.69:0/3126323442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb01a39b0 con 0x7f9bb0075d50 2024-03-19T01:26:19.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.324+0000 7f9bad600640 1 -- 172.21.15.69:0/3126323442 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4003a80 con 0x7f9bb0075d50 2024-03-19T01:26:19.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.324+0000 7f9bb7600640 1 -- 172.21.15.69:0/3126323442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb01ab2a0 con 0x7f9bb0075d50 2024-03-19T01:26:19.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.325+0000 7f9bb7600640 1 -- 172.21.15.69:0/3126323442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb0068a60 con 0x7f9bb0075d50 2024-03-19T01:26:19.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.327+0000 7f9bad600640 1 -- 172.21.15.69:0/3126323442 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f9ba4046020 con 0x7f9bb0075d50 2024-03-19T01:26:19.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.329+0000 7f9bad600640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9b980a29f0 0x7f9b980a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:19.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.329+0000 7f9bad600640 1 -- 172.21.15.69:0/3126323442 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9568+0+0 (secure 0 0 0) 0x7f9ba40f1510 con 0x7f9bb0075d50 2024-03-19T01:26:19.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.329+0000 7f9bafe00640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9b980a29f0 0x7f9b980a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:19.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.330+0000 7f9bafe00640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9b980a29f0 0x7f9b980a4eb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0101960 tx=0x7f9b9c005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:19.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.331+0000 7f9bad600640 1 -- 172.21.15.69:0/3126323442 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9ba404d020 con 0x7f9bb0075d50 2024-03-19T01:26:19.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:19 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/337087215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:26:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:19 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3171302511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:26:19.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:19 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/222496546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:26:19.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:19 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/337087215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:26:19.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:19 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3171302511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:26:19.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:19 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/222496546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:26:19.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.535+0000 7f9bb7600640 1 -- 172.21.15.69:0/3126323442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9bb010d7f0 con 0x7f9bb0075d50 2024-03-19T01:26:19.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.536+0000 7f9bad600640 1 -- 172.21.15.69:0/3126323442 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) ==== 100+0+11 (secure 0 0 0) 0x7f9ba40bdf30 con 0x7f9bb0075d50 2024-03-19T01:26:19.537 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-03-19T01:26:19.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.538+0000 7f9bb7600640 1 -- 172.21.15.69:0/3126323442 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9b980a29f0 msgr2=0x7f9b980a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.538+0000 7f9bb7600640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9b980a29f0 0x7f9b980a4eb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0101960 tx=0x7f9b9c005f90 comp rx=0 tx=0).stop 2024-03-19T01:26:19.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.538+0000 7f9bb7600640 1 -- 172.21.15.69:0/3126323442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb0075d50 msgr2=0x7f9bb019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.538+0000 7f9bb7600640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb0075d50 0x7f9bb019e780 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4005ab0 tx=0x7f9ba4005e80 comp rx=0 tx=0).stop 2024-03-19T01:26:19.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.538+0000 7f9bb7600640 1 -- 172.21.15.69:0/3126323442 shutdown_connections 2024-03-19T01:26:19.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.538+0000 7f9bb7600640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb0112660 0x7f9bb01a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.538+0000 7f9bb7600640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bb0076670 0x7f9bb019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.538+0000 7f9bb7600640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f9b980a29f0 0x7f9b980a4eb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.538+0000 7f9bb7600640 1 --2- 172.21.15.69:0/3126323442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb0075d50 0x7f9bb019e780 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.538+0000 7f9bb7600640 1 -- 172.21.15.69:0/3126323442 >> 172.21.15.69:0/3126323442 conn(0x7f9bb00ff490 msgr2=0x7f9bb0113d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:19.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.539+0000 7f9bb7600640 1 -- 172.21.15.69:0/3126323442 shutdown_connections 2024-03-19T01:26:19.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.539+0000 7f9bb7600640 1 -- 172.21.15.69:0/3126323442 wait complete. 2024-03-19T01:26:19.560 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-19T01:26:19.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:19 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/337087215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:26:19.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:19 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3171302511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:26:19.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:19 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/222496546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:26:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.779+0000 7f2be4a00640 1 -- 172.21.15.69:0/2529428960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2be010bc70 msgr2=0x7f2be010e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.779+0000 7f2be4a00640 1 --2- 172.21.15.69:0/2529428960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2be010bc70 0x7f2be010e110 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009be0 tx=0x7f2bd802bf80 comp rx=0 tx=0).stop 2024-03-19T01:26:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.779+0000 7f2be4a00640 1 -- 172.21.15.69:0/2529428960 shutdown_connections 2024-03-19T01:26:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.779+0000 7f2be4a00640 1 --2- 172.21.15.69:0/2529428960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2be010bc70 0x7f2be010e110 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.779+0000 7f2be4a00640 1 --2- 172.21.15.69:0/2529428960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be0103170 0x7f2be010b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.779+0000 7f2be4a00640 1 --2- 172.21.15.69:0/2529428960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be0102850 0x7f2be0102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.779+0000 7f2be4a00640 1 -- 172.21.15.69:0/2529428960 >> 172.21.15.69:0/2529428960 conn(0x7f2be00fc070 msgr2=0x7f2be00fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.779+0000 7f2be4a00640 1 -- 172.21.15.69:0/2529428960 shutdown_connections 2024-03-19T01:26:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.779+0000 7f2be4a00640 1 -- 172.21.15.69:0/2529428960 wait complete. 2024-03-19T01:26:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.781+0000 7f2be4a00640 1 Processor -- start 2024-03-19T01:26:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.781+0000 7f2be4a00640 1 -- start start 2024-03-19T01:26:19.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.781+0000 7f2be4a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be0102850 0x7f2be019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:19.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.781+0000 7f2be4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be0103170 0x7f2be019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:19.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.781+0000 7f2be4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2be010bc70 0x7f2be019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:19.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.781+0000 7f2be4a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0111ac0 con 0x7f2be0102850 2024-03-19T01:26:19.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.781+0000 7f2be4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0111940 con 0x7f2be0103170 2024-03-19T01:26:19.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.781+0000 7f2be4a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0111c40 con 0x7f2be010bc70 2024-03-19T01:26:19.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.782+0000 7f2bde800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be0102850 0x7f2be019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:19.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.782+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be0103170 0x7f2be019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:19.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.782+0000 7f2bdf200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2be010bc70 0x7f2be019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:19.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.782+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be0103170 0x7f2be019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58744/0 (socket says 172.21.15.69:58744) 2024-03-19T01:26:19.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.782+0000 7f2bd5e00640 1 -- 172.21.15.69:0/2053355107 learned_addr learned my addr 172.21.15.69:0/2053355107 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:19.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.782+0000 7f2bd5e00640 1 -- 172.21.15.69:0/2053355107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2be010bc70 msgr2=0x7f2be019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.782+0000 7f2bd5e00640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2be010bc70 0x7f2be019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.782+0000 7f2bd5e00640 1 -- 172.21.15.69:0/2053355107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be0102850 msgr2=0x7f2be019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.782+0000 7f2bd5e00640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be0102850 0x7f2be019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.782+0000 7f2bd5e00640 1 -- 172.21.15.69:0/2053355107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be019f090 con 0x7f2be0103170 2024-03-19T01:26:19.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.782+0000 7f2bd5e00640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be0103170 0x7f2be019a5e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800e9f0 tx=0x7f2bc800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:19.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.783+0000 7f2bd7e00640 1 -- 172.21.15.69:0/2053355107 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800cab0 con 0x7f2be0103170 2024-03-19T01:26:19.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.783+0000 7f2bd7e00640 1 -- 172.21.15.69:0/2053355107 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc800cc50 con 0x7f2be0103170 2024-03-19T01:26:19.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.783+0000 7f2bd7e00640 1 -- 172.21.15.69:0/2053355107 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80187e0 con 0x7f2be0103170 2024-03-19T01:26:19.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.783+0000 7f2be4a00640 1 -- 172.21.15.69:0/2053355107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be019f380 con 0x7f2be0103170 2024-03-19T01:26:19.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.783+0000 7f2be4a00640 1 -- 172.21.15.69:0/2053355107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be01a6c10 con 0x7f2be0103170 2024-03-19T01:26:19.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.784+0000 7f2be4a00640 1 -- 172.21.15.69:0/2053355107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be0068530 con 0x7f2be0103170 2024-03-19T01:26:19.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.786+0000 7f2bd7e00640 1 -- 172.21.15.69:0/2053355107 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f2bc8018b90 con 0x7f2be0103170 2024-03-19T01:26:19.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.787+0000 7f2bd7e00640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2bc40a29f0 0x7f2bc40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:19.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.787+0000 7f2bd7e00640 1 -- 172.21.15.69:0/2053355107 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9568+0+0 (secure 0 0 0) 0x7f2bc80cf630 con 0x7f2be0103170 2024-03-19T01:26:19.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.787+0000 7f2bde800640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2bc40a29f0 0x7f2bc40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:19.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.788+0000 7f2bde800640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2bc40a29f0 0x7f2bc40a4eb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0005fd0 tx=0x7f2bd0009260 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:19.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.791+0000 7f2bd7e00640 1 -- 172.21.15.69:0/2053355107 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2bc809bf40 con 0x7f2be0103170 2024-03-19T01:26:19.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.990+0000 7f2be4a00640 1 -- 172.21.15.69:0/2053355107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2be01a6e70 con 0x7f2be0103170 2024-03-19T01:26:19.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.991+0000 7f2bd7e00640 1 -- 172.21.15.69:0/2053355107 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v114) ==== 103+0+11 (secure 0 0 0) 0x7f2bc809b840 con 0x7f2be0103170 2024-03-19T01:26:19.991 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-03-19T01:26:19.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 -- 172.21.15.69:0/2053355107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2bc40a29f0 msgr2=0x7f2bc40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2bc40a29f0 0x7f2bc40a4eb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0005fd0 tx=0x7f2bd0009260 comp rx=0 tx=0).stop 2024-03-19T01:26:19.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 -- 172.21.15.69:0/2053355107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be0103170 msgr2=0x7f2be019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:19.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be0103170 0x7f2be019a5e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800e9f0 tx=0x7f2bc800edf0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 -- 172.21.15.69:0/2053355107 shutdown_connections 2024-03-19T01:26:19.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2be010bc70 0x7f2be019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f2bc40a29f0 0x7f2bc40a4eb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be0103170 0x7f2be019a5e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 --2- 172.21.15.69:0/2053355107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be0102850 0x7f2be019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:19.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 -- 172.21.15.69:0/2053355107 >> 172.21.15.69:0/2053355107 conn(0x7f2be00fc070 msgr2=0x7f2be010dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:19.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 -- 172.21.15.69:0/2053355107 shutdown_connections 2024-03-19T01:26:19.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:19.994+0000 7f2be4a00640 1 -- 172.21.15.69:0/2053355107 wait complete. 2024-03-19T01:26:20.013 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-19T01:26:20.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.212+0000 7fbbaf200640 1 -- 172.21.15.69:0/3805661143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba81097f0 msgr2=0x7fbba8109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:20.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.212+0000 7fbbaf200640 1 --2- 172.21.15.69:0/3805661143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba81097f0 0x7fbba8109bd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009930 tx=0x7fbb9c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:26:20.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.213+0000 7fbbaf200640 1 -- 172.21.15.69:0/3805661143 shutdown_connections 2024-03-19T01:26:20.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.213+0000 7fbbaf200640 1 --2- 172.21.15.69:0/3805661143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba81041a0 0x7fbba81106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:20.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.213+0000 7fbbaf200640 1 --2- 172.21.15.69:0/3805661143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba81037e0 0x7fbba8103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:20.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.213+0000 7fbbaf200640 1 --2- 172.21.15.69:0/3805661143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba81097f0 0x7fbba8109bd0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:20.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.213+0000 7fbbaf200640 1 -- 172.21.15.69:0/3805661143 >> 172.21.15.69:0/3805661143 conn(0x7fbba80ff250 msgr2=0x7fbba8101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:20.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.213+0000 7fbbaf200640 1 -- 172.21.15.69:0/3805661143 shutdown_connections 2024-03-19T01:26:20.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.213+0000 7fbbaf200640 1 -- 172.21.15.69:0/3805661143 wait complete. 2024-03-19T01:26:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.214+0000 7fbbaf200640 1 Processor -- start 2024-03-19T01:26:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.214+0000 7fbbaf200640 1 -- start start 2024-03-19T01:26:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.215+0000 7fbbaf200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba81037e0 0x7fbba819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.215+0000 7fbbaf200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba81041a0 0x7fbba819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:20.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.215+0000 7fbbaf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba81097f0 0x7fbba819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.215+0000 7fbbaf200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8113c50 con 0x7fbba81041a0 2024-03-19T01:26:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.215+0000 7fbbaf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8113ad0 con 0x7fbba81097f0 2024-03-19T01:26:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.215+0000 7fbbaf200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8113dd0 con 0x7fbba81037e0 2024-03-19T01:26:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.215+0000 7fbbade00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba81037e0 0x7fbba819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.215+0000 7fbbade00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba81037e0 0x7fbba819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48364/0 (socket says 172.21.15.69:48364) 2024-03-19T01:26:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.215+0000 7fbbade00640 1 -- 172.21.15.69:0/213056444 learned_addr learned my addr 172.21.15.69:0/213056444 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.215+0000 7fbbad400640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba81041a0 0x7fbba819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:20.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.215+0000 7fbbae800640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba81097f0 0x7fbba819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbbade00640 1 -- 172.21.15.69:0/213056444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba81097f0 msgr2=0x7fbba819ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbbade00640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba81097f0 0x7fbba819ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbbade00640 1 -- 172.21.15.69:0/213056444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba81041a0 msgr2=0x7fbba819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbbade00640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba81041a0 0x7fbba819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbbade00640 1 -- 172.21.15.69:0/213056444 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba819f130 con 0x7fbba81037e0 2024-03-19T01:26:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbbad400640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba81041a0 0x7fbba819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:26:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbbade00640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba81037e0 0x7fbba819a190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009900 tx=0x7fbb9c004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbba2a00640 1 -- 172.21.15.69:0/213056444 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c043020 con 0x7fbba81037e0 2024-03-19T01:26:20.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbba2a00640 1 -- 172.21.15.69:0/213056444 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb9c0054e0 con 0x7fbba81037e0 2024-03-19T01:26:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbbaf200640 1 -- 172.21.15.69:0/213056444 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba819f3c0 con 0x7fbba81037e0 2024-03-19T01:26:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbba2a00640 1 -- 172.21.15.69:0/213056444 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c003bb0 con 0x7fbba81037e0 2024-03-19T01:26:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.216+0000 7fbbaf200640 1 -- 172.21.15.69:0/213056444 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba81a6cb0 con 0x7fbba81037e0 2024-03-19T01:26:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.217+0000 7fbbaf200640 1 -- 172.21.15.69:0/213056444 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba8068980 con 0x7fbba81037e0 2024-03-19T01:26:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.219+0000 7fbba2a00640 1 -- 172.21.15.69:0/213056444 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fbb9c046020 con 0x7fbba81037e0 2024-03-19T01:26:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.220+0000 7fbba2a00640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbb900a2970 0x7fbb900a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.221+0000 7fbba2a00640 1 -- 172.21.15.69:0/213056444 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9568+0+0 (secure 0 0 0) 0x7fbb9c0f11f0 con 0x7fbba81037e0 2024-03-19T01:26:20.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.221+0000 7fbbad400640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbb900a2970 0x7fbb900a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:20.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.222+0000 7fbbad400640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbb900a2970 0x7fbb900a4e30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbba8108c20 tx=0x7fbb98005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:20.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.224+0000 7fbba2a00640 1 -- 172.21.15.69:0/213056444 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbb9c04d020 con 0x7fbba81037e0 2024-03-19T01:26:20.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:20 smithi069 ceph-mon[31727]: pgmap v492: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 120 KiB/s wr, 22 op/s 2024-03-19T01:26:20.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:20 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3126323442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:26:20.282 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:20 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2053355107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:26:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:20 smithi163 ceph-mon[67428]: pgmap v492: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 120 KiB/s wr, 22 op/s 2024-03-19T01:26:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:20 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3126323442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:26:20.311 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:20 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2053355107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:26:20.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.450+0000 7fbbaf200640 1 -- 172.21.15.69:0/213056444 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbba810bd50 con 0x7fbba81037e0 2024-03-19T01:26:20.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.451+0000 7fbba2a00640 1 -- 172.21.15.69:0/213056444 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v114) ==== 81+0+1003 (secure 0 0 0) 0x7fbb9c0bdc10 con 0x7fbba81037e0 2024-03-19T01:26:20.452 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3960674803 2024-03-20T01:10:40.006231+0000 2024-03-19T01:26:20.452 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3077206365 2024-03-20T00:58:49.440150+0000 2024-03-19T01:26:20.452 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1824937012 2024-03-20T00:59:23.503145+0000 2024-03-19T01:26:20.452 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3562156769 2024-03-20T01:10:40.006231+0000 2024-03-19T01:26:20.452 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/2422305615 2024-03-20T00:58:49.440150+0000 2024-03-19T01:26:20.452 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/697036895 2024-03-20T00:58:49.440150+0000 2024-03-19T01:26:20.453 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4228526145 2024-03-20T00:58:49.440150+0000 2024-03-19T01:26:20.453 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/2422305615 2024-03-20T00:58:49.440150+0000 2024-03-19T01:26:20.453 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2948479024 2024-03-20T00:59:23.503145+0000 2024-03-19T01:26:20.453 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3562156769 2024-03-20T01:10:40.006231+0000 2024-03-19T01:26:20.453 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2580938130 2024-03-20T00:59:23.503145+0000 2024-03-19T01:26:20.453 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/4073584559 2024-03-20T00:59:23.503145+0000 2024-03-19T01:26:20.453 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/4073584559 2024-03-20T00:59:23.503145+0000 2024-03-19T01:26:20.453 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2535372856 2024-03-20T01:10:40.006231+0000 2024-03-19T01:26:20.453 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2291738694 2024-03-20T01:10:40.006231+0000 2024-03-19T01:26:20.454 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2266394714 2024-03-20T01:10:40.006231+0000 2024-03-19T01:26:20.454 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2207308634 2024-03-20T01:10:40.006231+0000 2024-03-19T01:26:20.455 INFO:teuthology.orchestra.run.smithi069.stderr:listed 17 entries 2024-03-19T01:26:20.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.453+0000 7fbbaf200640 1 -- 172.21.15.69:0/213056444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbb900a2970 msgr2=0x7fbb900a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:20.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.454+0000 7fbbaf200640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbb900a2970 0x7fbb900a4e30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbba8108c20 tx=0x7fbb98005f90 comp rx=0 tx=0).stop 2024-03-19T01:26:20.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.454+0000 7fbbaf200640 1 -- 172.21.15.69:0/213056444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba81037e0 msgr2=0x7fbba819a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:20.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.454+0000 7fbbaf200640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba81037e0 0x7fbba819a190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009900 tx=0x7fbb9c004360 comp rx=0 tx=0).stop 2024-03-19T01:26:20.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.454+0000 7fbbaf200640 1 -- 172.21.15.69:0/213056444 shutdown_connections 2024-03-19T01:26:20.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.454+0000 7fbbaf200640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fbb900a2970 0x7fbb900a4e30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:20.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.454+0000 7fbbaf200640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba81097f0 0x7fbba819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:20.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.454+0000 7fbbaf200640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba81041a0 0x7fbba819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:20.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.454+0000 7fbbaf200640 1 --2- 172.21.15.69:0/213056444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbba81037e0 0x7fbba819a190 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:20.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.454+0000 7fbbaf200640 1 -- 172.21.15.69:0/213056444 >> 172.21.15.69:0/213056444 conn(0x7fbba80ff250 msgr2=0x7fbba80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:20.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.454+0000 7fbbaf200640 1 -- 172.21.15.69:0/213056444 shutdown_connections 2024-03-19T01:26:20.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:20.454+0000 7fbbaf200640 1 -- 172.21.15.69:0/213056444 wait complete. 2024-03-19T01:26:20.475 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-19T01:26:20.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:26:20.475 DEBUG:teuthology.orchestra.run.smithi069:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-19T01:26:20.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:20 smithi086 ceph-mon[66829]: pgmap v492: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 120 KiB/s wr, 22 op/s 2024-03-19T01:26:20.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:20 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3126323442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:26:20.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:20 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2053355107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-19T01:26:21.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:21 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/213056444' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T01:26:21.527 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:21 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/213056444' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T01:26:21.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:21 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/213056444' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T01:26:22.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:22 smithi086 ceph-mon[66829]: pgmap v493: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 106 KiB/s wr, 20 op/s 2024-03-19T01:26:22.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:22 smithi069 ceph-mon[31727]: pgmap v493: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 106 KiB/s wr, 20 op/s 2024-03-19T01:26:22.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:22 smithi163 ceph-mon[67428]: pgmap v493: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 106 KiB/s wr, 20 op/s 2024-03-19T01:26:22.758 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-03-19T01:26:22.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:26:22.758 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-19T01:26:22.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:26:22.814 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-03-19T01:26:22.879 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-19T01:26:22.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:26:22.881 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-19T01:26:22.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:26:22.953 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-03-19T01:26:23.066 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-19T01:26:23.078 INFO:tasks.cephadm:Teardown begin 2024-03-19T01:26:23.078 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-19T01:26:23.078 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:26:23.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.270+0000 7f0c7ac00640 1 -- 172.21.15.69:0/3270251784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c740744f0 msgr2=0x7f0c7410dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:23.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.270+0000 7f0c7ac00640 1 --2- 172.21.15.69:0/3270251784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c740744f0 0x7f0c7410dbc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009900 tx=0x7f0c6002bcd0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.270+0000 7f0c7ac00640 1 -- 172.21.15.69:0/3270251784 shutdown_connections 2024-03-19T01:26:23.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.270+0000 7f0c7ac00640 1 --2- 172.21.15.69:0/3270251784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7410e100 0x7f0c741104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.270+0000 7f0c7ac00640 1 --2- 172.21.15.69:0/3270251784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c740744f0 0x7f0c7410dbc0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.270+0000 7f0c7ac00640 1 --2- 172.21.15.69:0/3270251784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c74073bd0 0x7f0c74073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.270+0000 7f0c7ac00640 1 -- 172.21.15.69:0/3270251784 >> 172.21.15.69:0/3270251784 conn(0x7f0c740fd2f0 msgr2=0x7f0c740ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:23.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.270+0000 7f0c7ac00640 1 -- 172.21.15.69:0/3270251784 shutdown_connections 2024-03-19T01:26:23.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.270+0000 7f0c7ac00640 1 -- 172.21.15.69:0/3270251784 wait complete. 2024-03-19T01:26:23.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.271+0000 7f0c7ac00640 1 Processor -- start 2024-03-19T01:26:23.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.271+0000 7f0c7ac00640 1 -- start start 2024-03-19T01:26:23.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.272+0000 7f0c7ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c74073bd0 0x7f0c7419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:23.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.272+0000 7f0c7ac00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c740744f0 0x7f0c7419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:23.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.272+0000 7f0c7ac00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7410e100 0x7f0c7419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:23.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.272+0000 7f0c7ac00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0c74113e70 con 0x7f0c7410e100 2024-03-19T01:26:23.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.272+0000 7f0c7ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c74113cf0 con 0x7f0c74073bd0 2024-03-19T01:26:23.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.272+0000 7f0c7ac00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0c74113ff0 con 0x7f0c740744f0 2024-03-19T01:26:23.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.272+0000 7f0c73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c74073bd0 0x7f0c7419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:23.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.272+0000 7f0c73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c74073bd0 0x7f0c7419a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58792/0 (socket says 172.21.15.69:58792) 2024-03-19T01:26:23.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.272+0000 7f0c73e00640 1 -- 172.21.15.69:0/1281067764 learned_addr learned my addr 172.21.15.69:0/1281067764 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:23.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.272+0000 7f0c78c00640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7410e100 0x7f0c7419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:23.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.272+0000 7f0c73400640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c740744f0 0x7f0c7419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:23.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.273+0000 7f0c73e00640 1 -- 172.21.15.69:0/1281067764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c740744f0 msgr2=0x7f0c7419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:23.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.273+0000 7f0c73e00640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c740744f0 0x7f0c7419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.273+0000 7f0c73e00640 1 -- 172.21.15.69:0/1281067764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7410e100 msgr2=0x7f0c7419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:23.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.273+0000 7f0c73e00640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7410e100 0x7f0c7419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.273+0000 7f0c73e00640 1 -- 172.21.15.69:0/1281067764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7419f300 con 0x7f0c74073bd0 2024-03-19T01:26:23.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.273+0000 7f0c73e00640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c74073bd0 0x7f0c7419a310 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400e9f0 tx=0x7f0c6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:23.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.273+0000 7f0c70c00640 1 -- 172.21.15.69:0/1281067764 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400cd80 con 0x7f0c74073bd0 2024-03-19T01:26:23.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.273+0000 7f0c70c00640 1 -- 172.21.15.69:0/1281067764 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c64010910 con 0x7f0c74073bd0 2024-03-19T01:26:23.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.273+0000 7f0c70c00640 1 -- 172.21.15.69:0/1281067764 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64018990 con 0x7f0c74073bd0 2024-03-19T01:26:23.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.273+0000 7f0c7ac00640 1 -- 172.21.15.69:0/1281067764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7419f5f0 con 0x7f0c74073bd0 2024-03-19T01:26:23.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.273+0000 7f0c7ac00640 1 -- 172.21.15.69:0/1281067764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c741a6e80 con 0x7f0c74073bd0 2024-03-19T01:26:23.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.274+0000 7f0c7ac00640 1 -- 172.21.15.69:0/1281067764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c74075280 con 0x7f0c74073bd0 2024-03-19T01:26:23.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.276+0000 7f0c70c00640 1 -- 172.21.15.69:0/1281067764 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f0c64010ac0 con 0x7f0c74073bd0 2024-03-19T01:26:23.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.277+0000 7f0c70c00640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0c580a29f0 0x7f0c580a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:23.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.277+0000 7f0c70c00640 1 -- 172.21.15.69:0/1281067764 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9568+0+0 (secure 0 0 0) 0x7f0c640cf550 con 0x7f0c74073bd0 2024-03-19T01:26:23.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.278+0000 7f0c73400640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0c580a29f0 0x7f0c580a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:23.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.278+0000 7f0c73400640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0c580a29f0 0x7f0c580a4eb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0c60005ab0 tx=0x7f0c60005a20 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:23.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.281+0000 7f0c70c00640 1 -- 172.21.15.69:0/1281067764 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0c6409b810 con 0x7f0c74073bd0 2024-03-19T01:26:23.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.525+0000 7f0c7ac00640 1 -- 172.21.15.69:0/1281067764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c74067d80 con 0x7f0c74073bd0 2024-03-19T01:26:23.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.526+0000 7f0c70c00640 1 -- 172.21.15.69:0/1281067764 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11930 (secure 0 0 0) 0x7f0c6409f8b0 con 0x7f0c74073bd0 2024-03-19T01:26:23.527 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:26:23.527 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":59,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:20:08.600901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:26:23.527 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 59 2024-03-19T01:26:23.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.528+0000 7f0c7ac00640 1 -- 172.21.15.69:0/1281067764 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0c580a29f0 msgr2=0x7f0c580a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:23.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.528+0000 7f0c7ac00640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0c580a29f0 0x7f0c580a4eb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0c60005ab0 tx=0x7f0c60005a20 comp rx=0 tx=0).stop 2024-03-19T01:26:23.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.528+0000 7f0c7ac00640 1 -- 172.21.15.69:0/1281067764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c74073bd0 msgr2=0x7f0c7419a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:23.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.528+0000 7f0c7ac00640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c74073bd0 0x7f0c7419a310 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400e9f0 tx=0x7f0c6400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.528+0000 7f0c7ac00640 1 -- 172.21.15.69:0/1281067764 shutdown_connections 2024-03-19T01:26:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.528+0000 7f0c7ac00640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7410e100 0x7f0c7419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.528+0000 7f0c7ac00640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c740744f0 0x7f0c7419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.528+0000 7f0c7ac00640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f0c580a29f0 0x7f0c580a4eb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.528+0000 7f0c7ac00640 1 --2- 172.21.15.69:0/1281067764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c74073bd0 0x7f0c7419a310 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.528+0000 7f0c7ac00640 1 -- 172.21.15.69:0/1281067764 >> 172.21.15.69:0/1281067764 conn(0x7f0c740fd2f0 msgr2=0x7f0c740fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.529+0000 7f0c7ac00640 1 -- 172.21.15.69:0/1281067764 shutdown_connections 2024-03-19T01:26:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.529+0000 7f0c7ac00640 1 -- 172.21.15.69:0/1281067764 wait complete. 2024-03-19T01:26:23.553 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:26:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.752+0000 7ff88e400640 1 -- 172.21.15.69:0/2915582103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888101610 msgr2=0x7ff888101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.752+0000 7ff88e400640 1 --2- 172.21.15.69:0/2915582103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888101610 0x7ff888101a90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff874009900 tx=0x7ff87402bc30 comp rx=0 tx=0).stop 2024-03-19T01:26:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.752+0000 7ff88e400640 1 -- 172.21.15.69:0/2915582103 shutdown_connections 2024-03-19T01:26:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.752+0000 7ff88e400640 1 --2- 172.21.15.69:0/2915582103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff888101fd0 0x7ff88810e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.752+0000 7ff88e400640 1 --2- 172.21.15.69:0/2915582103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888101610 0x7ff888101a90 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.752+0000 7ff88e400640 1 --2- 172.21.15.69:0/2915582103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8881076b0 0x7ff888107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.752+0000 7ff88e400640 1 -- 172.21.15.69:0/2915582103 >> 172.21.15.69:0/2915582103 conn(0x7ff8880fd100 msgr2=0x7ff8880ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.753+0000 7ff88e400640 1 -- 172.21.15.69:0/2915582103 shutdown_connections 2024-03-19T01:26:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.753+0000 7ff88e400640 1 -- 172.21.15.69:0/2915582103 wait complete. 2024-03-19T01:26:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.754+0000 7ff88e400640 1 Processor -- start 2024-03-19T01:26:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.754+0000 7ff88e400640 1 -- start start 2024-03-19T01:26:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888101610 0x7ff88819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88e400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff888101fd0 0x7ff88819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88e400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8881076b0 0x7ff88819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88e400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff888111a40 con 0x7ff888101fd0 2024-03-19T01:26:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881118c0 con 0x7ff888101610 2024-03-19T01:26:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88e400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff888111bc0 con 0x7ff8881076b0 2024-03-19T01:26:23.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888101610 0x7ff88819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:23.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888101610 0x7ff88819a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58818/0 (socket says 172.21.15.69:58818) 2024-03-19T01:26:23.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88d000640 1 -- 172.21.15.69:0/1576901444 learned_addr learned my addr 172.21.15.69:0/1576901444 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:23.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff887e00640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff888101fd0 0x7ff88819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:23.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88da00640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8881076b0 0x7ff88819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:23.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88d000640 1 -- 172.21.15.69:0/1576901444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8881076b0 msgr2=0x7ff88819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:23.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88d000640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8881076b0 0x7ff88819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88d000640 1 -- 172.21.15.69:0/1576901444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff888101fd0 msgr2=0x7ff88819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88d000640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff888101fd0 0x7ff88819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88d000640 1 -- 172.21.15.69:0/1576901444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88819f080 con 0x7ff888101610 2024-03-19T01:26:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.755+0000 7ff88da00640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8881076b0 0x7ff88819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:26:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.756+0000 7ff88d000640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888101610 0x7ff88819a090 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e9c0 tx=0x7ff87c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.756+0000 7ff885600640 1 -- 172.21.15.69:0/1576901444 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00cd80 con 0x7ff888101610 2024-03-19T01:26:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.756+0000 7ff885600640 1 -- 172.21.15.69:0/1576901444 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff87c0035f0 con 0x7ff888101610 2024-03-19T01:26:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.756+0000 7ff885600640 1 -- 172.21.15.69:0/1576901444 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c010690 con 0x7ff888101610 2024-03-19T01:26:23.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.756+0000 7ff88e400640 1 -- 172.21.15.69:0/1576901444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88819f370 con 0x7ff888101610 2024-03-19T01:26:23.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.756+0000 7ff88e400640 1 -- 172.21.15.69:0/1576901444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8881a6c00 con 0x7ff888101610 2024-03-19T01:26:23.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.758+0000 7ff88e400640 1 -- 172.21.15.69:0/1576901444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff888068530 con 0x7ff888101610 2024-03-19T01:26:23.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.759+0000 7ff885600640 1 -- 172.21.15.69:0/1576901444 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ff87c0037a0 con 0x7ff888101610 2024-03-19T01:26:23.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.761+0000 7ff885600640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff8700a29f0 0x7ff8700a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:23.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.761+0000 7ff885600640 1 -- 172.21.15.69:0/1576901444 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9568+0+0 (secure 0 0 0) 0x7ff87c0cf590 con 0x7ff888101610 2024-03-19T01:26:23.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.761+0000 7ff887e00640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff8700a29f0 0x7ff8700a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:23.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.762+0000 7ff887e00640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff8700a29f0 0x7ff8700a4eb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff87402efd0 tx=0x7ff874000bc0 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:23.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:23.765+0000 7ff885600640 1 -- 172.21.15.69:0/1576901444 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff87c09bee0 con 0x7ff888101610 2024-03-19T01:26:24.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.018+0000 7ff88e400640 1 -- 172.21.15.69:0/1576901444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8881a6e70 con 0x7ff888101610 2024-03-19T01:26:24.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.019+0000 7ff885600640 1 -- 172.21.15.69:0/1576901444 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11930 (secure 0 0 0) 0x7ff87c09b7e0 con 0x7ff888101610 2024-03-19T01:26:24.020 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:26:24.020 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/184915179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":184915179},{"type":"v1","addr":"172.21.15.69:6835","nonce":184915179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1997100038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1997100038},{"type":"v1","addr":"172.21.15.69:6839","nonce":1997100038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":59,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:07:14.928258+0000","modified":"2024-03-19T01:20:08.600901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24580,"mds_2":24550,"mds_3":24494,"mds_4":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6833/1468679776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1468679776},{"type":"v1","addr":"172.21.15.163:6833","nonce":1468679776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3047552839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3047552839},{"type":"v1","addr":"172.21.15.86:6833","nonce":3047552839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"d","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.69:6837/2154698833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2154698833},{"type":"v1","addr":"172.21.15.69:6837","nonce":2154698833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.86:6835/786469397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":786469397},{"type":"v1","addr":"172.21.15.86:6835","nonce":786469397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6835/2175201817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2175201817},{"type":"v1","addr":"172.21.15.163:6835","nonce":2175201817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/1996224781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1996224781},{"type":"v1","addr":"172.21.15.86:6837","nonce":1996224781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1677890136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1677890136},{"type":"v1","addr":"172.21.15.163:6837","nonce":1677890136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.163:6839/2045902254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":2045902254},{"type":"v1","addr":"172.21.15.163:6839","nonce":2045902254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24580":{"gid":24580,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6841/4121471703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":4121471703},{"type":"v1","addr":"172.21.15.69:6841","nonce":4121471703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/621579864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":621579864},{"type":"v1","addr":"172.21.15.86:6839","nonce":621579864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24494,24550,24580,24476,24581]},"id":1}]} 2024-03-19T01:26:24.020 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 59 2024-03-19T01:26:24.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.021+0000 7ff88e400640 1 -- 172.21.15.69:0/1576901444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff8700a29f0 msgr2=0x7ff8700a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:24.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.021+0000 7ff88e400640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff8700a29f0 0x7ff8700a4eb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff87402efd0 tx=0x7ff874000bc0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.021+0000 7ff88e400640 1 -- 172.21.15.69:0/1576901444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888101610 msgr2=0x7ff88819a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:24.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.021+0000 7ff88e400640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888101610 0x7ff88819a090 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e9c0 tx=0x7ff87c00ee80 comp rx=0 tx=0).stop 2024-03-19T01:26:24.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.022+0000 7ff88e400640 1 -- 172.21.15.69:0/1576901444 shutdown_connections 2024-03-19T01:26:24.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.022+0000 7ff88e400640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8881076b0 0x7ff88819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.022+0000 7ff88e400640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff888101fd0 0x7ff88819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.022+0000 7ff88e400640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ff8700a29f0 0x7ff8700a4eb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.022+0000 7ff88e400640 1 --2- 172.21.15.69:0/1576901444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff888101610 0x7ff88819a090 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.022+0000 7ff88e400640 1 -- 172.21.15.69:0/1576901444 >> 172.21.15.69:0/1576901444 conn(0x7ff8880fd100 msgr2=0x7ff8880fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:24.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.022+0000 7ff88e400640 1 -- 172.21.15.69:0/1576901444 shutdown_connections 2024-03-19T01:26:24.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.022+0000 7ff88e400640 1 -- 172.21.15.69:0/1576901444 wait complete. 2024-03-19T01:26:24.040 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:26:24.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.236+0000 7fd0b8a00640 1 -- 172.21.15.69:0/3719372479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b4073bd0 msgr2=0x7fd0b4073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:24.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.236+0000 7fd0b8a00640 1 --2- 172.21.15.69:0/3719372479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b4073bd0 0x7fd0b4073fb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009900 tx=0x7fd0a402bcc0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.236+0000 7fd0b8a00640 1 -- 172.21.15.69:0/3719372479 shutdown_connections 2024-03-19T01:26:24.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.236+0000 7fd0b8a00640 1 --2- 172.21.15.69:0/3719372479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b410ded0 0x7fd0b41102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.236+0000 7fd0b8a00640 1 --2- 172.21.15.69:0/3719372479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b40744f0 0x7fd0b410d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.236+0000 7fd0b8a00640 1 --2- 172.21.15.69:0/3719372479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b4073bd0 0x7fd0b4073fb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.236+0000 7fd0b8a00640 1 -- 172.21.15.69:0/3719372479 >> 172.21.15.69:0/3719372479 conn(0x7fd0b40fd120 msgr2=0x7fd0b40ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:24.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.236+0000 7fd0b8a00640 1 -- 172.21.15.69:0/3719372479 shutdown_connections 2024-03-19T01:26:24.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.236+0000 7fd0b8a00640 1 -- 172.21.15.69:0/3719372479 wait complete. 2024-03-19T01:26:24.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.237+0000 7fd0b8a00640 1 Processor -- start 2024-03-19T01:26:24.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.237+0000 7fd0b8a00640 1 -- start start 2024-03-19T01:26:24.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b8a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b4073bd0 0x7fd0b419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:24.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40744f0 0x7fd0b419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:24.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b8a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b410ded0 0x7fd0b419e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:24.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b8a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113c00 con 0x7fd0b410ded0 2024-03-19T01:26:24.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b8a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113a80 con 0x7fd0b40744f0 2024-03-19T01:26:24.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b8a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113d80 con 0x7fd0b4073bd0 2024-03-19T01:26:24.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40744f0 0x7fd0b419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:24.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b2800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b4073bd0 0x7fd0b419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:24.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40744f0 0x7fd0b419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58846/0 (socket says 172.21.15.69:58846) 2024-03-19T01:26:24.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b1e00640 1 -- 172.21.15.69:0/2868422189 learned_addr learned my addr 172.21.15.69:0/2868422189 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:24.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b3200640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b410ded0 0x7fd0b419e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b2800640 1 -- 172.21.15.69:0/2868422189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40744f0 msgr2=0x7fd0b419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b2800640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40744f0 0x7fd0b419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.238+0000 7fd0b2800640 1 -- 172.21.15.69:0/2868422189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b410ded0 msgr2=0x7fd0b419e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.239+0000 7fd0b2800640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b410ded0 0x7fd0b419e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.239+0000 7fd0b2800640 1 -- 172.21.15.69:0/2868422189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b419f110 con 0x7fd0b4073bd0 2024-03-19T01:26:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.239+0000 7fd0b1e00640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40744f0 0x7fd0b419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:26:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.239+0000 7fd0b3200640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b410ded0 0x7fd0b419e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:26:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.239+0000 7fd0b2800640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b4073bd0 0x7fd0b419a120 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd0a402c200 tx=0x7fd0a4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.239+0000 7fd0ab400640 1 -- 172.21.15.69:0/2868422189 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4042020 con 0x7fd0b4073bd0 2024-03-19T01:26:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.239+0000 7fd0b8a00640 1 -- 172.21.15.69:0/2868422189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b419f3a0 con 0x7fd0b4073bd0 2024-03-19T01:26:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.239+0000 7fd0ab400640 1 -- 172.21.15.69:0/2868422189 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0a402cdf0 con 0x7fd0b4073bd0 2024-03-19T01:26:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.239+0000 7fd0ab400640 1 -- 172.21.15.69:0/2868422189 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4003a50 con 0x7fd0b4073bd0 2024-03-19T01:26:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.239+0000 7fd0b8a00640 1 -- 172.21.15.69:0/2868422189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b41a6c90 con 0x7fd0b4073bd0 2024-03-19T01:26:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.241+0000 7fd0b8a00640 1 -- 172.21.15.69:0/2868422189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b4068530 con 0x7fd0b4073bd0 2024-03-19T01:26:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.242+0000 7fd0ab400640 1 -- 172.21.15.69:0/2868422189 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7fd0a4045020 con 0x7fd0b4073bd0 2024-03-19T01:26:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.243+0000 7fd0ab400640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0980a2970 0x7fd0980a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.244+0000 7fd0ab400640 1 -- 172.21.15.69:0/2868422189 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9568+0+0 (secure 0 0 0) 0x7fd0a40f0c60 con 0x7fd0b4073bd0 2024-03-19T01:26:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.244+0000 7fd0b1e00640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0980a2970 0x7fd0980a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:24.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.244+0000 7fd0b1e00640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0980a2970 0x7fd0980a4e30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0b410f180 tx=0x7fd0a0005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:24.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.247+0000 7fd0ab400640 1 -- 172.21.15.69:0/2868422189 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd0a404c020 con 0x7fd0b4073bd0 2024-03-19T01:26:24.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:24 smithi069 ceph-mon[31727]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 126 KiB/s wr, 24 op/s 2024-03-19T01:26:24.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:24 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1281067764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:26:24.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:24 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1576901444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:26:24.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.498+0000 7fd0b8a00640 1 -- 172.21.15.69:0/2868422189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0b4109050 con 0x7fd0b4073bd0 2024-03-19T01:26:24.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.500+0000 7fd0ab400640 1 -- 172.21.15.69:0/2868422189 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+25364 (secure 0 0 0) 0x7fd0a40bd6c0 con 0x7fd0b4073bd0 2024-03-19T01:26:24.501 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-19T01:26:24.501 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":114,"fsid":"934de1c2-e58b-11ee-95c9-87774f69a715","created":"2024-03-19T00:58:09.792298+0000","modified":"2024-03-19T01:26:09.130305+0000","last_up_change":"2024-03-19T01:06:17.736066+0000","last_in_change":"2024-03-19T01:05:57.638241+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:02:06.167894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:07:13.368130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:07:14.444206+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-19T01:07:18.189146+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"53d5a7ae-973f-48a0-a4f5-904f7be2b0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6803","nonce":3647912729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6805","nonce":3647912729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6809","nonce":3647912729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3647912729},{"type":"v1","addr":"172.21.15.69:6807","nonce":3647912729}]},"public_addr":"172.21.15.69:6803/3647912729","cluster_addr":"172.21.15.69:6805/3647912729","heartbeat_back_addr":"172.21.15.69:6809/3647912729","heartbeat_front_addr":"172.21.15.69:6807/3647912729","state":["exists","up"]},{"osd":1,"uuid":"a7a78610-164b-454c-b229-91059a95c647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6801","nonce":2662600946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6803","nonce":2662600946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6807","nonce":2662600946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2662600946},{"type":"v1","addr":"172.21.15.86:6805","nonce":2662600946}]},"public_addr":"172.21.15.86:6801/2662600946","cluster_addr":"172.21.15.86:6803/2662600946","heartbeat_back_addr":"172.21.15.86:6807/2662600946","heartbeat_front_addr":"172.21.15.86:6805/2662600946","state":["exists","up"]},{"osd":2,"uuid":"ef693645-672d-4f7c-bfe9-1ba357b7fd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6801","nonce":3542508503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6803","nonce":3542508503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6807","nonce":3542508503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3542508503},{"type":"v1","addr":"172.21.15.163:6805","nonce":3542508503}]},"public_addr":"172.21.15.163:6801/3542508503","cluster_addr":"172.21.15.163:6803/3542508503","heartbeat_back_addr":"172.21.15.163:6807/3542508503","heartbeat_front_addr":"172.21.15.163:6805/3542508503","state":["exists","up"]},{"osd":3,"uuid":"5c06aaad-6a1f-4e27-b991-248826c99079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6811","nonce":2038881914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6813","nonce":2038881914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6817","nonce":2038881914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2038881914},{"type":"v1","addr":"172.21.15.69:6815","nonce":2038881914}]},"public_addr":"172.21.15.69:6811/2038881914","cluster_addr":"172.21.15.69:6813/2038881914","heartbeat_back_addr":"172.21.15.69:6817/2038881914","heartbeat_front_addr":"172.21.15.69:6815/2038881914","state":["exists","up"]},{"osd":4,"uuid":"6cc3d6e9-ff8f-4cf1-b245-83311dc7564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6809","nonce":1709421684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6811","nonce":1709421684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6815","nonce":1709421684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1709421684},{"type":"v1","addr":"172.21.15.86:6813","nonce":1709421684}]},"public_addr":"172.21.15.86:6809/1709421684","cluster_addr":"172.21.15.86:6811/1709421684","heartbeat_back_addr":"172.21.15.86:6815/1709421684","heartbeat_front_addr":"172.21.15.86:6813/1709421684","state":["exists","up"]},{"osd":5,"uuid":"7808c335-63a3-46f1-a05d-8bf41c048ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6809","nonce":95931076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6811","nonce":95931076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6815","nonce":95931076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":95931076},{"type":"v1","addr":"172.21.15.163:6813","nonce":95931076}]},"public_addr":"172.21.15.163:6809/95931076","cluster_addr":"172.21.15.163:6811/95931076","heartbeat_back_addr":"172.21.15.163:6815/95931076","heartbeat_front_addr":"172.21.15.163:6813/95931076","state":["exists","up"]},{"osd":6,"uuid":"92498308-b815-42a5-acbd-433c52723dd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6819","nonce":2819834341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6821","nonce":2819834341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6825","nonce":2819834341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2819834341},{"type":"v1","addr":"172.21.15.69:6823","nonce":2819834341}]},"public_addr":"172.21.15.69:6819/2819834341","cluster_addr":"172.21.15.69:6821/2819834341","heartbeat_back_addr":"172.21.15.69:6825/2819834341","heartbeat_front_addr":"172.21.15.69:6823/2819834341","state":["exists","up"]},{"osd":7,"uuid":"9f4bc368-4502-4d9d-898d-641c779f7511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6817","nonce":866804392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6819","nonce":866804392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6823","nonce":866804392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":866804392},{"type":"v1","addr":"172.21.15.86:6821","nonce":866804392}]},"public_addr":"172.21.15.86:6817/866804392","cluster_addr":"172.21.15.86:6819/866804392","heartbeat_back_addr":"172.21.15.86:6823/866804392","heartbeat_front_addr":"172.21.15.86:6821/866804392","state":["exists","up"]},{"osd":8,"uuid":"81553561-de57-4fb8-894b-d4c63532dc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6817","nonce":171535778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6819","nonce":171535778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6823","nonce":171535778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":171535778},{"type":"v1","addr":"172.21.15.163:6821","nonce":171535778}]},"public_addr":"172.21.15.163:6817/171535778","cluster_addr":"172.21.15.163:6819/171535778","heartbeat_back_addr":"172.21.15.163:6823/171535778","heartbeat_front_addr":"172.21.15.163:6821/171535778","state":["exists","up"]},{"osd":9,"uuid":"83e5279d-25d1-4acb-8fe9-47f4b32ad1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6827","nonce":3178049735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6829","nonce":3178049735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6833","nonce":3178049735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":3178049735},{"type":"v1","addr":"172.21.15.69:6831","nonce":3178049735}]},"public_addr":"172.21.15.69:6827/3178049735","cluster_addr":"172.21.15.69:6829/3178049735","heartbeat_back_addr":"172.21.15.69:6833/3178049735","heartbeat_front_addr":"172.21.15.69:6831/3178049735","state":["exists","up"]},{"osd":10,"uuid":"41af0156-b4aa-43f3-9e40-6396608fbfa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6825","nonce":4217204109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6827","nonce":4217204109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6831","nonce":4217204109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4217204109},{"type":"v1","addr":"172.21.15.86:6829","nonce":4217204109}]},"public_addr":"172.21.15.86:6825/4217204109","cluster_addr":"172.21.15.86:6827/4217204109","heartbeat_back_addr":"172.21.15.86:6831/4217204109","heartbeat_front_addr":"172.21.15.86:6829/4217204109","state":["exists","up"]},{"osd":11,"uuid":"bf43d278-aaee-4d11-9fda-3b72869c045c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6825","nonce":2918196559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6827","nonce":2918196559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6831","nonce":2918196559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2918196559},{"type":"v1","addr":"172.21.15.163:6829","nonce":2918196559}]},"public_addr":"172.21.15.163:6825/2918196559","cluster_addr":"172.21.15.163:6827/2918196559","heartbeat_back_addr":"172.21.15.163:6831/2918196559","heartbeat_front_addr":"172.21.15.163:6829/2918196559","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:14.552117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:01:39.287180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:02.677691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:02:34.386863+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:00.568056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:25.742277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:03:59.333737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:26.006497+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:04:51.035539+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:24.450737+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:05:50.530392+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:06:15.863026+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3960674803":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/3077206365":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/1824937012":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/3562156769":"2024-03-20T01:10:40.006231+0000","172.21.15.69:6801/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/697036895":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/4228526145":"2024-03-20T00:58:49.440150+0000","172.21.15.69:6800/2422305615":"2024-03-20T00:58:49.440150+0000","172.21.15.69:0/2948479024":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6801/3562156769":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2580938130":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6800/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:6801/4073584559":"2024-03-20T00:59:23.503145+0000","172.21.15.69:0/2535372856":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2291738694":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2266394714":"2024-03-20T01:10:40.006231+0000","172.21.15.69:0/2207308634":"2024-03-20T01:10:40.006231+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:26:24.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 -- 172.21.15.69:0/2868422189 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0980a2970 msgr2=0x7fd0980a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:24.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0980a2970 0x7fd0980a4e30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0b410f180 tx=0x7fd0a0005f90 comp rx=0 tx=0).stop 2024-03-19T01:26:24.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 -- 172.21.15.69:0/2868422189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b4073bd0 msgr2=0x7fd0b419a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:24.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b4073bd0 0x7fd0b419a120 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd0a402c200 tx=0x7fd0a4005e80 comp rx=0 tx=0).stop 2024-03-19T01:26:24.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 -- 172.21.15.69:0/2868422189 shutdown_connections 2024-03-19T01:26:24.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0b410ded0 0x7fd0b419e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7fd0980a2970 0x7fd0980a4e30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0b40744f0 0x7fd0b419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 --2- 172.21.15.69:0/2868422189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b4073bd0 0x7fd0b419a120 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 -- 172.21.15.69:0/2868422189 >> 172.21.15.69:0/2868422189 conn(0x7fd0b40fd120 msgr2=0x7fd0b40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:24.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 -- 172.21.15.69:0/2868422189 shutdown_connections 2024-03-19T01:26:24.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.503+0000 7fd0b8a00640 1 -- 172.21.15.69:0/2868422189 wait complete. 2024-03-19T01:26:24.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:24 smithi086 ceph-mon[66829]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 126 KiB/s wr, 24 op/s 2024-03-19T01:26:24.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:24 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1281067764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:26:24.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:24 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1576901444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:26:24.526 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-19T01:26:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:24 smithi163 ceph-mon[67428]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 126 KiB/s wr, 24 op/s 2024-03-19T01:26:24.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:24 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1281067764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:26:24.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:24 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1576901444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:26:24.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.724+0000 7ffadbe00640 1 -- 172.21.15.69:0/2578821153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc100e60 msgr2=0x7ffadc10d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.724+0000 7ffadbe00640 1 --2- 172.21.15.69:0/2578821153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc100e60 0x7ffadc10d320 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009cd0 tx=0x7ffad402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.724+0000 7ffadbe00640 1 -- 172.21.15.69:0/2578821153 shutdown_connections 2024-03-19T01:26:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.724+0000 7ffadbe00640 1 --2- 172.21.15.69:0/2578821153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc100e60 0x7ffadc10d320 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.724+0000 7ffadbe00640 1 --2- 172.21.15.69:0/2578821153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffadc1004a0 0x7ffadc100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.724+0000 7ffadbe00640 1 --2- 172.21.15.69:0/2578821153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffadc106560 0x7ffadc106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.724+0000 7ffadbe00640 1 -- 172.21.15.69:0/2578821153 >> 172.21.15.69:0/2578821153 conn(0x7ffadc0fbf50 msgr2=0x7ffadc0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:24.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.724+0000 7ffadbe00640 1 -- 172.21.15.69:0/2578821153 shutdown_connections 2024-03-19T01:26:24.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.724+0000 7ffadbe00640 1 -- 172.21.15.69:0/2578821153 wait complete. 2024-03-19T01:26:24.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.726+0000 7ffadbe00640 1 Processor -- start 2024-03-19T01:26:24.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.726+0000 7ffadbe00640 1 -- start start 2024-03-19T01:26:24.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.726+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc1004a0 0x7ffadc19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:24.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffadc100e60 0x7ffadc19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:24.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffadc106560 0x7ffadc19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:24.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadbe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc110820 con 0x7ffadc106560 2024-03-19T01:26:24.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc1106a0 con 0x7ffadc1004a0 2024-03-19T01:26:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadbe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc1109a0 con 0x7ffadc100e60 2024-03-19T01:26:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffada000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffadc100e60 0x7ffadc19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc1004a0 0x7ffadc19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffada000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffadc100e60 0x7ffadc19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48420/0 (socket says 172.21.15.69:48420) 2024-03-19T01:26:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc1004a0 0x7ffadc19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:58868/0 (socket says 172.21.15.69:58868) 2024-03-19T01:26:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffada000640 1 -- 172.21.15.69:0/3473463691 learned_addr learned my addr 172.21.15.69:0/3473463691 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadb400640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffadc106560 0x7ffadc19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadaa00640 1 -- 172.21.15.69:0/3473463691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffadc100e60 msgr2=0x7ffadc19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadaa00640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffadc100e60 0x7ffadc19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadaa00640 1 -- 172.21.15.69:0/3473463691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffadc106560 msgr2=0x7ffadc19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadaa00640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffadc106560 0x7ffadc19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffadaa00640 1 -- 172.21.15.69:0/3473463691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffadc19f040 con 0x7ffadc1004a0 2024-03-19T01:26:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.727+0000 7ffada000640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffadc100e60 0x7ffadc19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:26:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.728+0000 7ffadaa00640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc1004a0 0x7ffadc19a2e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00e9f0 tx=0x7ffacc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.728+0000 7ffad3400640 1 -- 172.21.15.69:0/3473463691 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00ccc0 con 0x7ffadc1004a0 2024-03-19T01:26:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.728+0000 7ffad3400640 1 -- 172.21.15.69:0/3473463691 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffacc010910 con 0x7ffadc1004a0 2024-03-19T01:26:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.728+0000 7ffadbe00640 1 -- 172.21.15.69:0/3473463691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffadc19f330 con 0x7ffadc1004a0 2024-03-19T01:26:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.728+0000 7ffad3400640 1 -- 172.21.15.69:0/3473463691 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc018960 con 0x7ffadc1004a0 2024-03-19T01:26:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.728+0000 7ffadbe00640 1 -- 172.21.15.69:0/3473463691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffadc1a6b80 con 0x7ffadc1004a0 2024-03-19T01:26:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.729+0000 7ffadbe00640 1 -- 172.21.15.69:0/3473463691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffadc068530 con 0x7ffadc1004a0 2024-03-19T01:26:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.731+0000 7ffad3400640 1 -- 172.21.15.69:0/3473463691 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7ffacc010ac0 con 0x7ffadc1004a0 2024-03-19T01:26:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.732+0000 7ffad3400640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffac00a2970 0x7ffac00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.732+0000 7ffad3400640 1 -- 172.21.15.69:0/3473463691 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9568+0+0 (secure 0 0 0) 0x7ffacc0cf670 con 0x7ffadc1004a0 2024-03-19T01:26:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.732+0000 7ffada000640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffac00a2970 0x7ffac00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.733+0000 7ffada000640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffac00a2970 0x7ffac00a4e30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0ff420 tx=0x7ffac8005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:24.737+0000 7ffad3400640 1 -- 172.21.15.69:0/3473463691 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffacc09b930 con 0x7ffadc1004a0 2024-03-19T01:26:25.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.005+0000 7ffadbe00640 1 -- 172.21.15.69:0/3473463691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ffadc1a6eb0 con 0x7ffadc1004a0 2024-03-19T01:26:25.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.015+0000 7ffad3400640 1 -- 172.21.15.69:0/3473463691 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) ==== 161+0+0 (secure 0 0 0) 0x7ffacc09f9d0 con 0x7ffadc1004a0 2024-03-19T01:26:25.018 INFO:teuthology.orchestra.run.smithi069.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-19T01:26:25.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.021+0000 7ffadbe00640 1 -- 172.21.15.69:0/3473463691 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffac00a2970 msgr2=0x7ffac00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:25.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.021+0000 7ffadbe00640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffac00a2970 0x7ffac00a4e30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0ff420 tx=0x7ffac8005f90 comp rx=0 tx=0).stop 2024-03-19T01:26:25.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.021+0000 7ffadbe00640 1 -- 172.21.15.69:0/3473463691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc1004a0 msgr2=0x7ffadc19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:25.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.021+0000 7ffadbe00640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc1004a0 0x7ffadc19a2e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00e9f0 tx=0x7ffacc00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:26:25.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.021+0000 7ffadbe00640 1 -- 172.21.15.69:0/3473463691 shutdown_connections 2024-03-19T01:26:25.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.021+0000 7ffadbe00640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffadc106560 0x7ffadc19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:25.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.021+0000 7ffadbe00640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffadc100e60 0x7ffadc19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:25.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.021+0000 7ffadbe00640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7ffac00a2970 0x7ffac00a4e30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:25.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.021+0000 7ffadbe00640 1 --2- 172.21.15.69:0/3473463691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc1004a0 0x7ffadc19a2e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:25.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.022+0000 7ffadbe00640 1 -- 172.21.15.69:0/3473463691 >> 172.21.15.69:0/3473463691 conn(0x7ffadc0fbf50 msgr2=0x7ffadc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:25.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.022+0000 7ffadbe00640 1 -- 172.21.15.69:0/3473463691 shutdown_connections 2024-03-19T01:26:25.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.022+0000 7ffadbe00640 1 -- 172.21.15.69:0/3473463691 wait complete. 2024-03-19T01:26:25.042 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-19T01:26:25.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.230+0000 7f597d200640 1 -- 172.21.15.69:0/1290911012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59780744e0 msgr2=0x7f5978112260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:25.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.230+0000 7f597d200640 1 --2- 172.21.15.69:0/1290911012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59780744e0 0x7f5978112260 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5964009900 tx=0x7f596402bcc0 comp rx=0 tx=0).stop 2024-03-19T01:26:25.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.230+0000 7f597d200640 1 -- 172.21.15.69:0/1290911012 shutdown_connections 2024-03-19T01:26:25.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.230+0000 7f597d200640 1 --2- 172.21.15.69:0/1290911012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59781127a0 0x7f5978114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:25.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.230+0000 7f597d200640 1 --2- 172.21.15.69:0/1290911012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59780744e0 0x7f5978112260 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:25.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.230+0000 7f597d200640 1 --2- 172.21.15.69:0/1290911012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5978073bc0 0x7f5978073fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:25.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.230+0000 7f597d200640 1 -- 172.21.15.69:0/1290911012 >> 172.21.15.69:0/1290911012 conn(0x7f59780fd280 msgr2=0x7f59780ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:25.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.230+0000 7f597d200640 1 -- 172.21.15.69:0/1290911012 shutdown_connections 2024-03-19T01:26:25.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.230+0000 7f597d200640 1 -- 172.21.15.69:0/1290911012 wait complete. 2024-03-19T01:26:25.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.231+0000 7f597d200640 1 Processor -- start 2024-03-19T01:26:25.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.231+0000 7f597d200640 1 -- start start 2024-03-19T01:26:25.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.231+0000 7f597d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5978073bc0 0x7f597819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:25.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.231+0000 7f597d200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59780744e0 0x7f597819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:25.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.231+0000 7f597d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59781127a0 0x7f59781a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:25.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.231+0000 7f597d200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59781181d0 con 0x7f5978073bc0 2024-03-19T01:26:25.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.231+0000 7f597d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5978118050 con 0x7f59781127a0 2024-03-19T01:26:25.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.231+0000 7f597d200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5978118350 con 0x7f59780744e0 2024-03-19T01:26:25.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5975e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5978073bc0 0x7f597819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:25.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5975400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59780744e0 0x7f597819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:25.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5975400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59780744e0 0x7f597819ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.69:48474/0 (socket says 172.21.15.69:48474) 2024-03-19T01:26:25.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5975400640 1 -- 172.21.15.69:0/1890989147 learned_addr learned my addr 172.21.15.69:0/1890989147 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:25.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5976800640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59781127a0 0x7f59781a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:25.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5975400640 1 -- 172.21.15.69:0/1890989147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59781127a0 msgr2=0x7f59781a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:25.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5975400640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59781127a0 0x7f59781a2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5975400640 1 -- 172.21.15.69:0/1890989147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5978073bc0 msgr2=0x7f597819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5975400640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5978073bc0 0x7f597819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5975400640 1 -- 172.21.15.69:0/1890989147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59781a3690 con 0x7f59780744e0 2024-03-19T01:26:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5976800640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59781127a0 0x7f59781a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:26:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5975e00640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5978073bc0 0x7f597819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:26:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.232+0000 7f5975400640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59780744e0 0x7f597819ec30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5964000c00 tx=0x7f5964004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.233+0000 7f596ea00640 1 -- 172.21.15.69:0/1890989147 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964042020 con 0x7f59780744e0 2024-03-19T01:26:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.233+0000 7f597d200640 1 -- 172.21.15.69:0/1890989147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59781a3920 con 0x7f59780744e0 2024-03-19T01:26:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.233+0000 7f596ea00640 1 -- 172.21.15.69:0/1890989147 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5964005000 con 0x7f59780744e0 2024-03-19T01:26:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.233+0000 7f596ea00640 1 -- 172.21.15.69:0/1890989147 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964003b20 con 0x7f59780744e0 2024-03-19T01:26:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.233+0000 7f597d200640 1 -- 172.21.15.69:0/1890989147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59781ab210 con 0x7f59780744e0 2024-03-19T01:26:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.234+0000 7f597d200640 1 -- 172.21.15.69:0/1890989147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5978075300 con 0x7f59780744e0 2024-03-19T01:26:25.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.236+0000 7f596ea00640 1 -- 172.21.15.69:0/1890989147 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f5964045020 con 0x7f59780744e0 2024-03-19T01:26:25.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.237+0000 7f596ea00640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f595c0a29f0 0x7f595c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:25.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.237+0000 7f596ea00640 1 -- 172.21.15.69:0/1890989147 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9910+0+0 (secure 0 0 0) 0x7f59640f0e90 con 0x7f59780744e0 2024-03-19T01:26:25.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.237+0000 7f5975e00640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f595c0a29f0 0x7f595c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:25.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.238+0000 7f5975e00640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f595c0a29f0 0x7f595c0a4eb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59680095f0 tx=0x7f5968005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:25.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.241+0000 7f596ea00640 1 -- 172.21.15.69:0/1890989147 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f596404c020 con 0x7f59780744e0 2024-03-19T01:26:25.282 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d[69820]: ignoring --setuser ceph since I am not root 2024-03-19T01:26:25.282 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d[69820]: ignoring --setgroup ceph since I am not root 2024-03-19T01:26:25.283 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-mds[69844]: -- 172.21.15.69:0/4273245442 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cccaf05a40 con 0x55cccaed9000 2024-03-19T01:26:25.283 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d[69820]: starting mds.d at 2024-03-19T01:26:25.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a[31723]: 2024-03-19T01:26:25.008+0000 7f3e82e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T01:26:25.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2868422189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:26:25.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3473463691' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T01:26:25.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T01:26:25.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-mon[31727]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-19T01:26:25.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-mon[31727]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T01:26:25.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-19T01:26:25.284 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-mon[31727]: osdmap e115: 12 total, 12 up, 12 in 2024-03-19T01:26:25.285 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-mon[31727]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-19T01:26:25.285 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a[70809]: ignoring --setuser ceph since I am not root 2024-03-19T01:26:25.285 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a[70809]: ignoring --setgroup ceph since I am not root 2024-03-19T01:26:25.285 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-mds[70829]: -- 172.21.15.69:0/3790896970 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55931d4d1a40 con 0x55931d4a4800 2024-03-19T01:26:25.285 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:26:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a[70809]: starting mds.a at 2024-03-19T01:26:25.311 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c[98581]: ignoring --setuser ceph since I am not root 2024-03-19T01:26:25.312 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c[98581]: ignoring --setgroup ceph since I am not root 2024-03-19T01:26:25.312 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mds[98585]: -- 172.21.15.163:0/223918641 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555778b89a40 con 0x555778b5d000 2024-03-19T01:26:25.312 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c[98581]: starting mds.c at 2024-03-19T01:26:25.313 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i[98884]: ignoring --setuser ceph since I am not root 2024-03-19T01:26:25.313 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i[98884]: ignoring --setgroup ceph since I am not root 2024-03-19T01:26:25.313 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mds[98888]: -- 172.21.15.163:0/2041835541 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633751b9a40 con 0x56337518c800 2024-03-19T01:26:25.313 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i[98884]: starting mds.i at 2024-03-19T01:26:25.314 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f[98273]: ignoring --setuser ceph since I am not root 2024-03-19T01:26:25.314 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f[98273]: ignoring --setgroup ceph since I am not root 2024-03-19T01:26:25.314 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mds[98277]: -- 172.21.15.163:0/2450907378 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ddc7303a40 con 0x55ddc72d7000 2024-03-19T01:26:25.314 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f[98273]: starting mds.f at 2024-03-19T01:26:25.315 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l[97948]: ignoring --setuser ceph since I am not root 2024-03-19T01:26:25.315 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l[97948]: ignoring --setgroup ceph since I am not root 2024-03-19T01:26:25.315 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mds[97952]: -- 172.21.15.163:0/4157086537 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56180fb29a40 con 0x56180fafc800 2024-03-19T01:26:25.315 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l[97948]: starting mds.l at 2024-03-19T01:26:25.315 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2868422189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:26:25.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3473463691' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T01:26:25.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T01:26:25.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mon[67428]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-19T01:26:25.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mon[67428]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T01:26:25.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-19T01:26:25.316 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mon[67428]: osdmap e115: 12 total, 12 up, 12 in 2024-03-19T01:26:25.317 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:25 smithi163 ceph-mon[67428]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-19T01:26:25.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:25.485+0000 7f597d200640 1 -- 172.21.15.69:0/1890989147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f597819f9f0 con 0x7f59780744e0 2024-03-19T01:26:25.506 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e[97001]: ignoring --setuser ceph since I am not root 2024-03-19T01:26:25.506 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e[97001]: ignoring --setgroup ceph since I am not root 2024-03-19T01:26:25.506 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mds[97005]: -- 172.21.15.86:0/2900296923 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d71dd3fa40 con 0x55d71dd12c00 2024-03-19T01:26:25.506 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e[97001]: starting mds.e at 2024-03-19T01:26:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2868422189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:26:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3473463691' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T01:26:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T01:26:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mon[66829]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-19T01:26:25.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mon[66829]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T01:26:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-19T01:26:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mon[66829]: osdmap e115: 12 total, 12 up, 12 in 2024-03-19T01:26:25.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mon[66829]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-19T01:26:25.508 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b[97627]: ignoring --setuser ceph since I am not root 2024-03-19T01:26:25.509 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b[97627]: ignoring --setgroup ceph since I am not root 2024-03-19T01:26:25.509 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mds[97631]: -- 172.21.15.86:0/2592841193 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f9c16da40 con 0x562f9c141000 2024-03-19T01:26:25.509 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b[97627]: starting mds.b at 2024-03-19T01:26:25.509 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h[97318]: ignoring --setuser ceph since I am not root 2024-03-19T01:26:25.509 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h[97318]: ignoring --setgroup ceph since I am not root 2024-03-19T01:26:25.510 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mds[97322]: -- 172.21.15.86:0/218951506 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fbe2b23a40 con 0x55fbe2af6800 2024-03-19T01:26:25.510 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h[97318]: starting mds.h at 2024-03-19T01:26:25.510 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-k[96680]: ignoring --setuser ceph since I am not root 2024-03-19T01:26:25.510 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-k[96680]: ignoring --setgroup ceph since I am not root 2024-03-19T01:26:25.511 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mds[96684]: -- 172.21.15.86:0/2494607007 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b8c862e4e0 con 0x55b8c8604800 2024-03-19T01:26:25.511 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-mds[96684]: -- 172.21.15.86:0/2494607007 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b8c8631a40 con 0x55b8c8604800 2024-03-19T01:26:25.511 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:26:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-k[96680]: starting mds.k at 2024-03-19T01:26:26.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.017+0000 7f596ea00640 1 -- 172.21.15.69:0/1890989147 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) ==== 104+0+0 (secure 0 0 0) 0x7f59640bd730 con 0x7f59780744e0 2024-03-19T01:26:26.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.021+0000 7f597d200640 1 -- 172.21.15.69:0/1890989147 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f595c0a29f0 msgr2=0x7f595c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:26.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.021+0000 7f597d200640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f595c0a29f0 0x7f595c0a4eb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59680095f0 tx=0x7f5968005f90 comp rx=0 tx=0).stop 2024-03-19T01:26:26.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.021+0000 7f597d200640 1 -- 172.21.15.69:0/1890989147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59780744e0 msgr2=0x7f597819ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:26.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.021+0000 7f597d200640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59780744e0 0x7f597819ec30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5964000c00 tx=0x7f5964004360 comp rx=0 tx=0).stop 2024-03-19T01:26:26.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.021+0000 7f597d200640 1 -- 172.21.15.69:0/1890989147 shutdown_connections 2024-03-19T01:26:26.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.022+0000 7f597d200640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f595c0a29f0 0x7f595c0a4eb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:26.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.022+0000 7f597d200640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59781127a0 0x7f59781a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:26.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.022+0000 7f597d200640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59780744e0 0x7f597819ec30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:26.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.022+0000 7f597d200640 1 --2- 172.21.15.69:0/1890989147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5978073bc0 0x7f597819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:26.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.022+0000 7f597d200640 1 -- 172.21.15.69:0/1890989147 >> 172.21.15.69:0/1890989147 conn(0x7f59780fd280 msgr2=0x7f59780fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:26.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.022+0000 7f597d200640 1 -- 172.21.15.69:0/1890989147 shutdown_connections 2024-03-19T01:26:26.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.022+0000 7f597d200640 1 -- 172.21.15.69:0/1890989147 wait complete. 2024-03-19T01:26:26.046 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-19T01:26:26.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.249+0000 7f195f800640 1 -- 172.21.15.69:0/1944386763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19581089e0 msgr2=0x7f195810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:26.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.249+0000 7f195f800640 1 --2- 172.21.15.69:0/1944386763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19581089e0 0x7f195810ca20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1948009900 tx=0x7f194802bc30 comp rx=0 tx=0).stop 2024-03-19T01:26:26.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.249+0000 7f195f800640 1 -- 172.21.15.69:0/1944386763 shutdown_connections 2024-03-19T01:26:26.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.249+0000 7f195f800640 1 --2- 172.21.15.69:0/1944386763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f195810d100 0x7f1958114cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:26.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.249+0000 7f195f800640 1 --2- 172.21.15.69:0/1944386763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19581089e0 0x7f195810ca20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:26.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.249+0000 7f195f800640 1 --2- 172.21.15.69:0/1944386763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1958108030 0x7f1958108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:26.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.249+0000 7f195f800640 1 -- 172.21.15.69:0/1944386763 >> 172.21.15.69:0/1944386763 conn(0x7f1958075d60 msgr2=0x7f1958078180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:26.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.249+0000 7f195f800640 1 -- 172.21.15.69:0/1944386763 shutdown_connections 2024-03-19T01:26:26.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.249+0000 7f195f800640 1 -- 172.21.15.69:0/1944386763 wait complete. 2024-03-19T01:26:26.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.250+0000 7f195f800640 1 Processor -- start 2024-03-19T01:26:26.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.250+0000 7f195f800640 1 -- start start 2024-03-19T01:26:26.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.250+0000 7f195f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1958108030 0x7f195819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:26.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.250+0000 7f195f800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f195810d100 0x7f195819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:26.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.250+0000 7f195f800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19581a2fe0 0x7f19581a3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:26.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.250+0000 7f195f800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19581181e0 con 0x7f195810d100 2024-03-19T01:26:26.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.250+0000 7f195f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1958118060 con 0x7f1958108030 2024-03-19T01:26:26.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.250+0000 7f195f800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1958118360 con 0x7f19581a2fe0 2024-03-19T01:26:26.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.250+0000 7f194fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f195810d100 0x7f195819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:26.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f194fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f195810d100 0x7f195819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56986/0 (socket says 172.21.15.69:56986) 2024-03-19T01:26:26.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f194fe00640 1 -- 172.21.15.69:0/3758839633 learned_addr learned my addr 172.21.15.69:0/3758839633 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:26.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f195ce00640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1958108030 0x7f195819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:26.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f195d800640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19581a2fe0 0x7f19581a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:26.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f194fe00640 1 -- 172.21.15.69:0/3758839633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19581a2fe0 msgr2=0x7f19581a3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:26.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f194fe00640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19581a2fe0 0x7f19581a3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:26.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f194fe00640 1 -- 172.21.15.69:0/3758839633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1958108030 msgr2=0x7f195819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:26.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f194fe00640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1958108030 0x7f195819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:26.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f194fe00640 1 -- 172.21.15.69:0/3758839633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19581a3ad0 con 0x7f195810d100 2024-03-19T01:26:26.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f194fe00640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f195810d100 0x7f195819ec60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f194802c240 tx=0x7f1948004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:26.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f1956000640 1 -- 172.21.15.69:0/3758839633 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948043020 con 0x7f195810d100 2024-03-19T01:26:26.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f1956000640 1 -- 172.21.15.69:0/3758839633 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19480054e0 con 0x7f195810d100 2024-03-19T01:26:26.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f1956000640 1 -- 172.21.15.69:0/3758839633 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948003bb0 con 0x7f195810d100 2024-03-19T01:26:26.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.251+0000 7f195f800640 1 -- 172.21.15.69:0/3758839633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19581a3d60 con 0x7f195810d100 2024-03-19T01:26:26.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.252+0000 7f195f800640 1 -- 172.21.15.69:0/3758839633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19581ab5b0 con 0x7f195810d100 2024-03-19T01:26:26.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.253+0000 7f195f800640 1 -- 172.21.15.69:0/3758839633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1958068a60 con 0x7f195810d100 2024-03-19T01:26:26.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.255+0000 7f1956000640 1 -- 172.21.15.69:0/3758839633 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f1948046020 con 0x7f195810d100 2024-03-19T01:26:26.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.256+0000 7f1956000640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f19380a29f0 0x7f19380a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:26.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.256+0000 7f1956000640 1 -- 172.21.15.69:0/3758839633 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9910+0+0 (secure 0 0 0) 0x7f19480f1280 con 0x7f195810d100 2024-03-19T01:26:26.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.256+0000 7f195ce00640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f19380a29f0 0x7f19380a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:26.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.257+0000 7f195ce00640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f19380a29f0 0x7f19380a4eb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f19400095f0 tx=0x7f1940005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:26.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.261+0000 7f1956000640 1 -- 172.21.15.69:0/3758839633 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f194804d020 con 0x7f195810d100 2024-03-19T01:26:26.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:26.444+0000 7f195f800640 1 -- 172.21.15.69:0/3758839633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19581089e0 con 0x7f195810d100 2024-03-19T01:26:26.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-03-19T01:26:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/1890989147' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:26:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:26:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-19T01:26:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-19T01:26:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: Cluster is now healthy 2024-03-19T01:26:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-19T01:26:26.507 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:26:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.86:6834/1195979437,v1:172.21.15.86:6835/1195979437] up:boot 2024-03-19T01:26:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:26:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.163:6834/3253258272,v1:172.21.15.163:6835/3253258272] up:boot 2024-03-19T01:26:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.163:6836/295659844,v1:172.21.15.163:6837/295659844] up:boot 2024-03-19T01:26:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.163:6832/1100936380,v1:172.21.15.163:6833/1100936380] up:boot 2024-03-19T01:26:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.86:6836/3024999710,v1:172.21.15.86:6837/3024999710] up:boot 2024-03-19T01:26:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:26:26.508 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.86:6832/3035142927,v1:172.21.15.86:6833/3035142927] up:boot 2024-03-19T01:26:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.86:6838/2035028912,v1:172.21.15.86:6839/2035028912] up:boot 2024-03-19T01:26:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.163:6838/691974545,v1:172.21.15.163:6839/691974545] up:boot 2024-03-19T01:26:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.69:6836/2963919636,v1:172.21.15.69:6837/2963919636] up:boot 2024-03-19T01:26:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: mds.? [v2:172.21.15.69:6840/4243480509,v1:172.21.15.69:6841/4243480509] up:boot 2024-03-19T01:26:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: fsmap 12 up:standby 2024-03-19T01:26:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:26:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:26:26.509 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:26:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:26:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:26:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:26:26.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:26 smithi086 ceph-mon[66829]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:26:26.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-03-19T01:26:26.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/1890989147' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:26:26.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:26:26.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-19T01:26:26.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-19T01:26:26.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: Cluster is now healthy 2024-03-19T01:26:26.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-19T01:26:26.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:26:26.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.86:6834/1195979437,v1:172.21.15.86:6835/1195979437] up:boot 2024-03-19T01:26:26.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:26:26.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.163:6834/3253258272,v1:172.21.15.163:6835/3253258272] up:boot 2024-03-19T01:26:26.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.163:6836/295659844,v1:172.21.15.163:6837/295659844] up:boot 2024-03-19T01:26:26.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.163:6832/1100936380,v1:172.21.15.163:6833/1100936380] up:boot 2024-03-19T01:26:26.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.86:6836/3024999710,v1:172.21.15.86:6837/3024999710] up:boot 2024-03-19T01:26:26.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:26:26.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.86:6832/3035142927,v1:172.21.15.86:6833/3035142927] up:boot 2024-03-19T01:26:26.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.86:6838/2035028912,v1:172.21.15.86:6839/2035028912] up:boot 2024-03-19T01:26:26.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.163:6838/691974545,v1:172.21.15.163:6839/691974545] up:boot 2024-03-19T01:26:26.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.69:6836/2963919636,v1:172.21.15.69:6837/2963919636] up:boot 2024-03-19T01:26:26.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: mds.? [v2:172.21.15.69:6840/4243480509,v1:172.21.15.69:6841/4243480509] up:boot 2024-03-19T01:26:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: fsmap 12 up:standby 2024-03-19T01:26:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:26:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:26:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:26:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:26:26.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:26:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:26:26.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:26 smithi069 ceph-mon[31727]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:26:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-03-19T01:26:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/1890989147' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:26:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:26:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-19T01:26:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-19T01:26:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: Cluster is now healthy 2024-03-19T01:26:26.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-19T01:26:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:26:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.86:6834/1195979437,v1:172.21.15.86:6835/1195979437] up:boot 2024-03-19T01:26:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:26:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.163:6834/3253258272,v1:172.21.15.163:6835/3253258272] up:boot 2024-03-19T01:26:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.163:6836/295659844,v1:172.21.15.163:6837/295659844] up:boot 2024-03-19T01:26:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.163:6832/1100936380,v1:172.21.15.163:6833/1100936380] up:boot 2024-03-19T01:26:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.86:6836/3024999710,v1:172.21.15.86:6837/3024999710] up:boot 2024-03-19T01:26:26.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:26:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.86:6832/3035142927,v1:172.21.15.86:6833/3035142927] up:boot 2024-03-19T01:26:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.86:6838/2035028912,v1:172.21.15.86:6839/2035028912] up:boot 2024-03-19T01:26:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.163:6838/691974545,v1:172.21.15.163:6839/691974545] up:boot 2024-03-19T01:26:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.69:6836/2963919636,v1:172.21.15.69:6837/2963919636] up:boot 2024-03-19T01:26:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: mds.? [v2:172.21.15.69:6840/4243480509,v1:172.21.15.69:6841/4243480509] up:boot 2024-03-19T01:26:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: fsmap 12 up:standby 2024-03-19T01:26:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:26:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:26:26.564 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:26:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:26:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:26:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:26:26.565 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:26 smithi163 ceph-mon[67428]: from='mgr.25645 172.21.15.69:0/432063000' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:26:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.203+0000 7f1956000640 1 -- 172.21.15.69:0/3758839633 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) ==== 181+0+0 (secure 0 0 0) 0x7f19480bdb20 con 0x7f195810d100 2024-03-19T01:26:27.205 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' removed 2024-03-19T01:26:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 -- 172.21.15.69:0/3758839633 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f19380a29f0 msgr2=0x7f19380a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f19380a29f0 0x7f19380a4eb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f19400095f0 tx=0x7f1940005f90 comp rx=0 tx=0).stop 2024-03-19T01:26:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 -- 172.21.15.69:0/3758839633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f195810d100 msgr2=0x7f195819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f195810d100 0x7f195819ec60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f194802c240 tx=0x7f1948004360 comp rx=0 tx=0).stop 2024-03-19T01:26:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 -- 172.21.15.69:0/3758839633 shutdown_connections 2024-03-19T01:26:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19581a2fe0 0x7f19581a3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f195810d100 0x7f195819ec60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f19380a29f0 0x7f19380a4eb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 --2- 172.21.15.69:0/3758839633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1958108030 0x7f195819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 -- 172.21.15.69:0/3758839633 >> 172.21.15.69:0/3758839633 conn(0x7f1958075d60 msgr2=0x7f1958078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 -- 172.21.15.69:0/3758839633 shutdown_connections 2024-03-19T01:26:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.209+0000 7f195f800640 1 -- 172.21.15.69:0/3758839633 wait complete. 2024-03-19T01:26:27.236 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-19T01:26:27.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.481+0000 7f5462800640 1 -- 172.21.15.69:0/3228304721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c1099d0 msgr2=0x7f545c109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:27.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.481+0000 7f5462800640 1 --2- 172.21.15.69:0/3228304721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c1099d0 0x7f545c109db0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f544c009930 tx=0x7f544c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:26:27.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.481+0000 7f5462800640 1 -- 172.21.15.69:0/3228304721 shutdown_connections 2024-03-19T01:26:27.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.481+0000 7f5462800640 1 --2- 172.21.15.69:0/3228304721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f545c104380 0x7f545c110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:27.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.481+0000 7f5462800640 1 --2- 172.21.15.69:0/3228304721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f545c1039c0 0x7f545c103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:27.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.481+0000 7f5462800640 1 --2- 172.21.15.69:0/3228304721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c1099d0 0x7f545c109db0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:27.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.481+0000 7f5462800640 1 -- 172.21.15.69:0/3228304721 >> 172.21.15.69:0/3228304721 conn(0x7f545c0ff470 msgr2=0x7f545c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:27.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.481+0000 7f5462800640 1 -- 172.21.15.69:0/3228304721 shutdown_connections 2024-03-19T01:26:27.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.481+0000 7f5462800640 1 -- 172.21.15.69:0/3228304721 wait complete. 2024-03-19T01:26:27.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.482+0000 7f5462800640 1 Processor -- start 2024-03-19T01:26:27.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.482+0000 7f5462800640 1 -- start start 2024-03-19T01:26:27.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f5462800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f545c1039c0 0x7f545c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:27.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f5462800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c104380 0x7f545c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:27.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f5462800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f545c1099d0 0x7f545c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:27.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f5462800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f545c113e00 con 0x7f545c1039c0 2024-03-19T01:26:27.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f5462800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f545c113c80 con 0x7f545c1099d0 2024-03-19T01:26:27.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f5462800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f545c113f80 con 0x7f545c104380 2024-03-19T01:26:27.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f545b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f545c1039c0 0x7f545c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:27.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f545b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f545c1039c0 0x7f545c19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57004/0 (socket says 172.21.15.69:57004) 2024-03-19T01:26:27.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f545b400640 1 -- 172.21.15.69:0/641879918 learned_addr learned my addr 172.21.15.69:0/641879918 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:27.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f545be00640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f545c1099d0 0x7f545c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:27.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f5452a00640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c104380 0x7f545c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:27.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f545b400640 1 -- 172.21.15.69:0/641879918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c104380 msgr2=0x7f545c19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:27.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f545b400640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c104380 0x7f545c19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:27.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f545b400640 1 -- 172.21.15.69:0/641879918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f545c1099d0 msgr2=0x7f545c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:27.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f545b400640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f545c1099d0 0x7f545c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:27.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f545b400640 1 -- 172.21.15.69:0/641879918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c19f2d0 con 0x7f545c1039c0 2024-03-19T01:26:27.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.483+0000 7f5452a00640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c104380 0x7f545c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:26:27.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.484+0000 7f545b400640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f545c1039c0 0x7f545c19a2e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f544c009900 tx=0x7f544c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:27.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.484+0000 7f5458c00640 1 -- 172.21.15.69:0/641879918 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c043020 con 0x7f545c1039c0 2024-03-19T01:26:27.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.484+0000 7f5462800640 1 -- 172.21.15.69:0/641879918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c19f560 con 0x7f545c1039c0 2024-03-19T01:26:27.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.484+0000 7f5458c00640 1 -- 172.21.15.69:0/641879918 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544c0054e0 con 0x7f545c1039c0 2024-03-19T01:26:27.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.484+0000 7f5458c00640 1 -- 172.21.15.69:0/641879918 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c003bb0 con 0x7f545c1039c0 2024-03-19T01:26:27.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.484+0000 7f5462800640 1 -- 172.21.15.69:0/641879918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c1a6e50 con 0x7f545c1039c0 2024-03-19T01:26:27.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.485+0000 7f5462800640 1 -- 172.21.15.69:0/641879918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545c068a60 con 0x7f545c1039c0 2024-03-19T01:26:27.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.487+0000 7f5458c00640 1 -- 172.21.15.69:0/641879918 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f544c046020 con 0x7f545c1039c0 2024-03-19T01:26:27.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.488+0000 7f5458c00640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54400a2970 0x7f54400a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:27.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.488+0000 7f5458c00640 1 -- 172.21.15.69:0/641879918 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9385+0+0 (secure 0 0 0) 0x7f544c0f11a0 con 0x7f545c1039c0 2024-03-19T01:26:27.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.489+0000 7f5452a00640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54400a2970 0x7f54400a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:27.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.489+0000 7f5452a00640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54400a2970 0x7f54400a4e30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f545c101af0 tx=0x7f5444005f90 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:27.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.492+0000 7f5458c00640 1 -- 172.21.15.69:0/641879918 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f544c04d020 con 0x7f545c1039c0 2024-03-19T01:26:27.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:27 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3758839633' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T01:26:27.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:27 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3758839633' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T01:26:27.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:27 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3758839633' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T01:26:27.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:27.682+0000 7f5462800640 1 -- 172.21.15.69:0/641879918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f545c10bf30 con 0x7f545c1039c0 2024-03-19T01:26:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.206+0000 7f5458c00640 1 -- 172.21.15.69:0/641879918 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v117) ==== 169+0+0 (secure 0 0 0) 0x7f544c0bdc50 con 0x7f545c1039c0 2024-03-19T01:26:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' removed 2024-03-19T01:26:28.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.209+0000 7f5462800640 1 -- 172.21.15.69:0/641879918 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54400a2970 msgr2=0x7f54400a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:28.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.209+0000 7f5462800640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54400a2970 0x7f54400a4e30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f545c101af0 tx=0x7f5444005f90 comp rx=0 tx=0).stop 2024-03-19T01:26:28.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.209+0000 7f5462800640 1 -- 172.21.15.69:0/641879918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f545c1039c0 msgr2=0x7f545c19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:28.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.209+0000 7f5462800640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f545c1039c0 0x7f545c19a2e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f544c009900 tx=0x7f544c004360 comp rx=0 tx=0).stop 2024-03-19T01:26:28.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.211+0000 7f5462800640 1 -- 172.21.15.69:0/641879918 shutdown_connections 2024-03-19T01:26:28.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.211+0000 7f5462800640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f54400a2970 0x7f54400a4e30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:28.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.211+0000 7f5462800640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f545c1099d0 0x7f545c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:28.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.211+0000 7f5462800640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c104380 0x7f545c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:28.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.211+0000 7f5462800640 1 --2- 172.21.15.69:0/641879918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f545c1039c0 0x7f545c19a2e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:28.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.211+0000 7f5462800640 1 -- 172.21.15.69:0/641879918 >> 172.21.15.69:0/641879918 conn(0x7f545c0ff470 msgr2=0x7f545c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:28.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.212+0000 7f5462800640 1 -- 172.21.15.69:0/641879918 shutdown_connections 2024-03-19T01:26:28.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.212+0000 7f5462800640 1 -- 172.21.15.69:0/641879918 wait complete. 2024-03-19T01:26:28.237 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-19T01:26:28.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.434+0000 7f1b78c00640 1 -- 172.21.15.69:0/2041444029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b741089e0 msgr2=0x7f1b7410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:28.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.434+0000 7f1b78c00640 1 --2- 172.21.15.69:0/2041444029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b741089e0 0x7f1b7410ca20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009900 tx=0x7f1b6002bc40 comp rx=0 tx=0).stop 2024-03-19T01:26:28.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.434+0000 7f1b78c00640 1 -- 172.21.15.69:0/2041444029 shutdown_connections 2024-03-19T01:26:28.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.434+0000 7f1b78c00640 1 --2- 172.21.15.69:0/2041444029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b7410d100 0x7f1b74114cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:28.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.434+0000 7f1b78c00640 1 --2- 172.21.15.69:0/2041444029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b741089e0 0x7f1b7410ca20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:28.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.434+0000 7f1b78c00640 1 --2- 172.21.15.69:0/2041444029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b74108030 0x7f1b74108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:28.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.434+0000 7f1b78c00640 1 -- 172.21.15.69:0/2041444029 >> 172.21.15.69:0/2041444029 conn(0x7f1b74075d60 msgr2=0x7f1b74078180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:28.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.434+0000 7f1b78c00640 1 -- 172.21.15.69:0/2041444029 shutdown_connections 2024-03-19T01:26:28.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.434+0000 7f1b78c00640 1 -- 172.21.15.69:0/2041444029 wait complete. 2024-03-19T01:26:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.435+0000 7f1b78c00640 1 Processor -- start 2024-03-19T01:26:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.435+0000 7f1b78c00640 1 -- start start 2024-03-19T01:26:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.435+0000 7f1b78c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b74108030 0x7f1b7419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.435+0000 7f1b78c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b7410d100 0x7f1b7419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:28.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.435+0000 7f1b78c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b741a2fc0 0x7f1b741a3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.435+0000 7f1b78c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741181c0 con 0x7f1b7410d100 2024-03-19T01:26:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.435+0000 7f1b78c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74118040 con 0x7f1b74108030 2024-03-19T01:26:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.435+0000 7f1b78c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74118340 con 0x7f1b741a2fc0 2024-03-19T01:26:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b7410d100 0x7f1b7419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b74108030 0x7f1b7419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b7410d100 0x7f1b7419ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57026/0 (socket says 172.21.15.69:57026) 2024-03-19T01:26:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b72800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b741a2fc0 0x7f1b741a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b74108030 0x7f1b7419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.69:38458/0 (socket says 172.21.15.69:38458) 2024-03-19T01:26:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71400640 1 -- 172.21.15.69:0/2070202620 learned_addr learned my addr 172.21.15.69:0/2070202620 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-19T01:26:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71400640 1 -- 172.21.15.69:0/2070202620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b741a2fc0 msgr2=0x7f1b741a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71400640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b741a2fc0 0x7f1b741a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71400640 1 -- 172.21.15.69:0/2070202620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b74108030 msgr2=0x7f1b7419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71400640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b74108030 0x7f1b7419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71400640 1 -- 172.21.15.69:0/2070202620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b741a3ab0 con 0x7f1b7410d100 2024-03-19T01:26:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71e00640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b74108030 0x7f1b7419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:26:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.436+0000 7f1b71400640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b7410d100 0x7f1b7419ec40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1b6002c250 tx=0x7f1b600043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:28.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.437+0000 7f1b6aa00640 1 -- 172.21.15.69:0/2070202620 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60041070 con 0x7f1b7410d100 2024-03-19T01:26:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.437+0000 7f1b6aa00640 1 -- 172.21.15.69:0/2070202620 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b6002cca0 con 0x7f1b7410d100 2024-03-19T01:26:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.437+0000 7f1b6aa00640 1 -- 172.21.15.69:0/2070202620 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60003e00 con 0x7f1b7410d100 2024-03-19T01:26:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.437+0000 7f1b78c00640 1 -- 172.21.15.69:0/2070202620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b741a3d40 con 0x7f1b7410d100 2024-03-19T01:26:28.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.437+0000 7f1b78c00640 1 -- 172.21.15.69:0/2070202620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b741ab500 con 0x7f1b7410d100 2024-03-19T01:26:28.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.438+0000 7f1b78c00640 1 -- 172.21.15.69:0/2070202620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b74068a60 con 0x7f1b7410d100 2024-03-19T01:26:28.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.440+0000 7f1b6aa00640 1 -- 172.21.15.69:0/2070202620 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 28) ==== 137421+0+0 (secure 0 0 0) 0x7f1b60044070 con 0x7f1b7410d100 2024-03-19T01:26:28.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.441+0000 7f1b6aa00640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1b580a2970 0x7f1b580a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:26:28.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.441+0000 7f1b6aa00640 1 -- 172.21.15.69:0/2070202620 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8797+0+0 (secure 0 0 0) 0x7f1b600f0c60 con 0x7f1b7410d100 2024-03-19T01:26:28.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.442+0000 7f1b71e00640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1b580a2970 0x7f1b580a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:26:28.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.442+0000 7f1b71e00640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1b580a2970 0x7f1b580a4e30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400bfd0 tx=0x7f1b6400b390 comp rx=0 tx=0).ready entity=mgr.25645 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:26:28.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.445+0000 7f1b6aa00640 1 -- 172.21.15.69:0/2070202620 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b600bd310 con 0x7f1b7410d100 2024-03-19T01:26:28.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:28 smithi086 ceph-mon[66829]: pgmap v497: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-03-19T01:26:28.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:28 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/3758839633' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T01:26:28.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:28 smithi086 ceph-mon[66829]: osdmap e116: 12 total, 12 up, 12 in 2024-03-19T01:26:28.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:28 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/641879918' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T01:26:28.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:28 smithi069 ceph-mon[31727]: pgmap v497: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-03-19T01:26:28.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:28 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/3758839633' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T01:26:28.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:28 smithi069 ceph-mon[31727]: osdmap e116: 12 total, 12 up, 12 in 2024-03-19T01:26:28.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:28 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/641879918' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T01:26:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:28 smithi163 ceph-mon[67428]: pgmap v497: 129 pgs: 129 active+clean; 2.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-03-19T01:26:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:28 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/3758839633' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T01:26:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:28 smithi163 ceph-mon[67428]: osdmap e116: 12 total, 12 up, 12 in 2024-03-19T01:26:28.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:28 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/641879918' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T01:26:28.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:28.651+0000 7f1b78c00640 1 -- 172.21.15.69:0/2070202620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1b741089e0 con 0x7f1b7410d100 2024-03-19T01:26:29.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.228+0000 7f1b6aa00640 1 -- 172.21.15.69:0/2070202620 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v118) ==== 178+0+0 (secure 0 0 0) 0x7f1b600c13b0 con 0x7f1b7410d100 2024-03-19T01:26:29.229 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data_ec' removed 2024-03-19T01:26:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.235+0000 7f1b78c00640 1 -- 172.21.15.69:0/2070202620 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1b580a2970 msgr2=0x7f1b580a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.235+0000 7f1b78c00640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1b580a2970 0x7f1b580a4e30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400bfd0 tx=0x7f1b6400b390 comp rx=0 tx=0).stop 2024-03-19T01:26:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.235+0000 7f1b78c00640 1 -- 172.21.15.69:0/2070202620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b7410d100 msgr2=0x7f1b7419ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:26:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.235+0000 7f1b78c00640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b7410d100 0x7f1b7419ec40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1b6002c250 tx=0x7f1b600043b0 comp rx=0 tx=0).stop 2024-03-19T01:26:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.236+0000 7f1b78c00640 1 -- 172.21.15.69:0/2070202620 shutdown_connections 2024-03-19T01:26:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.236+0000 7f1b78c00640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b741a2fc0 0x7f1b741a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.236+0000 7f1b78c00640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b7410d100 0x7f1b7419ec40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:29.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.236+0000 7f1b78c00640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.69:6800/2931658273,v1:172.21.15.69:6801/2931658273] conn(0x7f1b580a2970 0x7f1b580a4e30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:29.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.236+0000 7f1b78c00640 1 --2- 172.21.15.69:0/2070202620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b74108030 0x7f1b7419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:26:29.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.236+0000 7f1b78c00640 1 -- 172.21.15.69:0/2070202620 >> 172.21.15.69:0/2070202620 conn(0x7f1b74075d60 msgr2=0x7f1b74077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:26:29.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.236+0000 7f1b78c00640 1 -- 172.21.15.69:0/2070202620 shutdown_connections 2024-03-19T01:26:29.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-19T01:26:29.236+0000 7f1b78c00640 1 -- 172.21.15.69:0/2070202620 wait complete. 2024-03-19T01:26:29.266 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T01:26:29.372 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T01:26:29.431 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T01:26:29.466 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-19T01:26:29.466 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-19T01:26:29.505 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:29 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/641879918' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T01:26:29.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:29 smithi086 ceph-mon[66829]: osdmap e117: 12 total, 12 up, 12 in 2024-03-19T01:26:29.506 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:29 smithi086 ceph-mon[66829]: from='client.? 172.21.15.69:0/2070202620' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T01:26:29.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:29 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/641879918' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T01:26:29.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:29 smithi069 ceph-mon[31727]: osdmap e117: 12 total, 12 up, 12 in 2024-03-19T01:26:29.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:29 smithi069 ceph-mon[31727]: from='client.? 172.21.15.69:0/2070202620' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T01:26:29.533 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-19T01:26:29.549 DEBUG:teuthology.orchestra.run.smithi163:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-19T01:26:29.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:29 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/641879918' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T01:26:29.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:29 smithi163 ceph-mon[67428]: osdmap e117: 12 total, 12 up, 12 in 2024-03-19T01:26:29.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:29 smithi163 ceph-mon[67428]: from='client.? 172.21.15.69:0/2070202620' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T01:26:29.566 INFO:tasks.cephadm:Stopping all daemons... 2024-03-19T01:26:29.566 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-19T01:26:29.566 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.a 2024-03-19T01:26:30.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:29 smithi069 systemd[1]: Stopping Ceph mon.a for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:26:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a[31723]: 2024-03-19T01:26:30.165+0000 7f3e89c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:26:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a[31723]: 2024-03-19T01:26:30.165+0000 7f3e89c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-19T01:26:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:30 smithi069 podman[126419]: 2024-03-19 01:26:30.350950709 +0000 UTC m=+0.343210284 container stop d1aee4c67e5b47beb54e6b513c8b123c7ff4ffa556a525e687e5856cfd7f573f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-19T01:26:30.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:30 smithi069 podman[126419]: 2024-03-19 01:26:30.376710448 +0000 UTC m=+0.368970016 container died d1aee4c67e5b47beb54e6b513c8b123c7ff4ffa556a525e687e5856cfd7f573f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-19T01:26:31.866 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:31 smithi069 podman[126419]: 2024-03-19 01:26:31.60977893 +0000 UTC m=+1.602038505 container cleanup d1aee4c67e5b47beb54e6b513c8b123c7ff4ffa556a525e687e5856cfd7f573f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:26:31.867 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:31 smithi069 bash[126419]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a 2024-03-19T01:26:32.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Mar 19 01:26:31 smithi069 podman[126431]: 2024-03-19 01:26:31.866125156 +0000 UTC m=+1.499511733 container remove d1aee4c67e5b47beb54e6b513c8b123c7ff4ffa556a525e687e5856cfd7f573f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-a, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:26:32.215 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.a.service' 2024-03-19T01:26:32.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:26:32.339 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-19T01:26:32.339 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-19T01:26:32.339 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.b 2024-03-19T01:26:32.717 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:32 smithi086 systemd[1]: Stopping Ceph mon.b for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:26:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-b[66825]: 2024-03-19T01:26:32.715+0000 7f1d45e1a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:26:33.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-b[66825]: 2024-03-19T01:26:32.715+0000 7f1d45e1a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-19T01:26:33.006 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:32 smithi086 podman[104572]: 2024-03-19 01:26:32.808638672 +0000 UTC m=+0.243425312 container died e818e2c8a5e84a182706c2dd741ed7f33f10fbc176223285e134ef1d75514c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-03-19T01:26:33.416 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:33 smithi086 podman[104572]: 2024-03-19 01:26:33.155874413 +0000 UTC m=+0.590661079 container cleanup e818e2c8a5e84a182706c2dd741ed7f33f10fbc176223285e134ef1d75514c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-b, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-19T01:26:33.416 INFO:journalctl@ceph.mon.b.smithi086.stdout:Mar 19 01:26:33 smithi086 bash[104572]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-b 2024-03-19T01:26:33.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.b.service' 2024-03-19T01:26:33.688 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:26:33.688 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-19T01:26:33.688 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-19T01:26:33.689 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.c 2024-03-19T01:26:34.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:33 smithi163 systemd[1]: Stopping Ceph mon.c for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:26:34.561 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:34 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-c[67424]: 2024-03-19T01:26:34.065+0000 7fc32bf9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:26:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:34 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-c[67424]: 2024-03-19T01:26:34.065+0000 7fc32bf9d700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-19T01:26:34.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:34 smithi163 podman[106632]: 2024-03-19 01:26:34.270512588 +0000 UTC m=+0.384784918 container died 5e9f2948f746f4df094eb32109fdf9457cdba128ea7127d2293ff61de7bdfac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T01:26:35.061 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:34 smithi163 podman[106632]: 2024-03-19 01:26:34.674595192 +0000 UTC m=+0.788867532 container cleanup 5e9f2948f746f4df094eb32109fdf9457cdba128ea7127d2293ff61de7bdfac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=) 2024-03-19T01:26:35.062 INFO:journalctl@ceph.mon.c.smithi163.stdout:Mar 19 01:26:34 smithi163 bash[106632]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mon-c 2024-03-19T01:26:35.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mon.c.service' 2024-03-19T01:26:35.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:26:35.350 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-19T01:26:35.350 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-19T01:26:35.350 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.x 2024-03-19T01:26:35.783 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:26:35 smithi069 systemd[1]: Stopping Ceph mgr.x for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:26:36.282 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:26:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:26:35.918+0000 7f61ad000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:26:36.283 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:26:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x[31952]: 2024-03-19T01:26:35.918+0000 7f61ad000700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-19T01:26:36.283 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:26:36 smithi069 podman[126630]: 2024-03-19 01:26:36.021760238 +0000 UTC m=+0.290884247 container died 1507fbc37617fcd688631aefa50718b9a927a1c425a7b365bb70e2a23aa0a8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-19T01:26:38.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:26:38 smithi069 podman[126630]: 2024-03-19 01:26:38.583325686 +0000 UTC m=+2.852449701 container cleanup 1507fbc37617fcd688631aefa50718b9a927a1c425a7b365bb70e2a23aa0a8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:26:38.782 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:26:38 smithi069 bash[126630]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x 2024-03-19T01:26:39.281 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Mar 19 01:26:38 smithi069 podman[126644]: 2024-03-19 01:26:38.870132574 +0000 UTC m=+2.055105785 container remove 1507fbc37617fcd688631aefa50718b9a927a1c425a7b365bb70e2a23aa0a8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-19T01:26:39.333 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-19T01:26:39.431 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:26:39.432 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-19T01:26:39.432 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-19T01:26:39.432 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.y 2024-03-19T01:26:39.756 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:26:39 smithi086 systemd[1]: Stopping Ceph mgr.y for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:26:40.103 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:26:39 smithi086 podman[104697]: 2024-03-19 01:26:39.772641816 +0000 UTC m=+0.143941311 container died 1d9b5a97ff38a92dbda8d130af67dddac4fb25e7dc638809eebb5b44fe6f6138 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:26:40.506 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:26:40 smithi086 podman[104697]: 2024-03-19 01:26:40.201869184 +0000 UTC m=+0.573168688 container cleanup 1d9b5a97ff38a92dbda8d130af67dddac4fb25e7dc638809eebb5b44fe6f6138 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-19T01:26:40.506 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Mar 19 01:26:40 smithi086 bash[104697]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-y 2024-03-19T01:26:40.657 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-19T01:26:40.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:26:40.695 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-19T01:26:40.696 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-19T01:26:40.696 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.z 2024-03-19T01:26:41.061 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:26:40 smithi163 systemd[1]: Stopping Ceph mgr.z for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:26:41.366 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:26:41 smithi163 podman[106754]: 2024-03-19 01:26:41.067900265 +0000 UTC m=+0.177095178 container died 52de883900050730f6a3359f4937d846d2d5bef1b706f6271f68f43b395a63fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-03-19T01:26:41.667 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:26:41 smithi163 podman[106754]: 2024-03-19 01:26:41.473211441 +0000 UTC m=+0.582406336 container cleanup 52de883900050730f6a3359f4937d846d2d5bef1b706f6271f68f43b395a63fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-19T01:26:41.667 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Mar 19 01:26:41 smithi163 bash[106754]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mgr-z 2024-03-19T01:26:41.891 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mgr.z.service' 2024-03-19T01:26:41.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:26:41.928 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-19T01:26:41.928 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-19T01:26:41.928 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.0 2024-03-19T01:26:42.282 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:42 smithi069 systemd[1]: Stopping Ceph osd.0 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:26:43.032 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0[44087]: 2024-03-19T01:26:42.543+0000 7f684e800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:26:43.033 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0[44087]: 2024-03-19T01:26:42.543+0000 7f684e800700 -1 osd.0 118 *** Got signal Terminated *** 2024-03-19T01:26:43.033 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0[44087]: 2024-03-19T01:26:42.543+0000 7f684e800700 -1 osd.0 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:26:48.444 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:48 smithi069 podman[126752]: 2024-03-19 01:26:48.142202064 +0000 UTC m=+5.745865244 container died 29c8d2dbdd4fbe8d8c5af4038fa2eecd7274a72d09ceed741f8a63c80456cadb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:26:48.782 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:48 smithi069 podman[126752]: 2024-03-19 01:26:48.579671203 +0000 UTC m=+6.183334421 container cleanup 29c8d2dbdd4fbe8d8c5af4038fa2eecd7274a72d09ceed741f8a63c80456cadb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-19T01:26:48.782 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:48 smithi069 bash[126752]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0 2024-03-19T01:26:49.282 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:48 smithi069 podman[126771]: 2024-03-19 01:26:48.839455627 +0000 UTC m=+0.677263164 container remove 29c8d2dbdd4fbe8d8c5af4038fa2eecd7274a72d09ceed741f8a63c80456cadb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-03-19T01:26:49.945 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:49 smithi069 podman[126857]: 2024-03-19 01:26:49.488045459 +0000 UTC m=+0.031821822 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:26:49.946 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:49 smithi069 podman[126857]: 2024-03-19 01:26:49.598726423 +0000 UTC m=+0.142502823 container create 77a1485f65bfd9dd76b3f08c8a74ab97f9432c0c65ce924c1b7d944e433fd3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-19T01:26:49.946 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:49 smithi069 podman[126857]: 2024-03-19 01:26:49.944378698 +0000 UTC m=+0.488155105 container init 77a1485f65bfd9dd76b3f08c8a74ab97f9432c0c65ce924c1b7d944e433fd3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-19T01:26:50.282 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:49 smithi069 podman[126857]: 2024-03-19 01:26:49.955914902 +0000 UTC m=+0.499691322 container start 77a1485f65bfd9dd76b3f08c8a74ab97f9432c0c65ce924c1b7d944e433fd3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T01:26:50.282 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:50 smithi069 podman[126857]: 2024-03-19 01:26:50.051498664 +0000 UTC m=+0.595275114 container attach 77a1485f65bfd9dd76b3f08c8a74ab97f9432c0c65ce924c1b7d944e433fd3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-19T01:26:50.782 INFO:journalctl@ceph.osd.0.smithi069.stdout:Mar 19 01:26:50 smithi069 podman[126857]: 2024-03-19 01:26:50.506887648 +0000 UTC m=+1.050664058 container died 77a1485f65bfd9dd76b3f08c8a74ab97f9432c0c65ce924c1b7d944e433fd3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:26:51.297 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.0.service' 2024-03-19T01:26:51.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:26:51.419 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-19T01:26:51.419 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-19T01:26:51.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.1 2024-03-19T01:26:51.723 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:51 smithi086 systemd[1]: Stopping Ceph osd.1 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:26:52.006 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1[71463]: 2024-03-19T01:26:51.721+0000 7fb30e0e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:26:52.006 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1[71463]: 2024-03-19T01:26:51.721+0000 7fb30e0e0700 -1 osd.1 118 *** Got signal Terminated *** 2024-03-19T01:26:52.006 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1[71463]: 2024-03-19T01:26:51.721+0000 7fb30e0e0700 -1 osd.1 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:26:57.506 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:57 smithi086 podman[104821]: 2024-03-19 01:26:57.245434517 +0000 UTC m=+5.634932114 container died 514b793e949d987bfe48a5d4799d886dee5c04d2ac88239cba699584fd826342 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307) 2024-03-19T01:26:58.045 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:57 smithi086 podman[104821]: 2024-03-19 01:26:57.87264699 +0000 UTC m=+6.262144580 container cleanup 514b793e949d987bfe48a5d4799d886dee5c04d2ac88239cba699584fd826342 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-19T01:26:58.045 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:57 smithi086 bash[104821]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1 2024-03-19T01:26:58.505 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:58 smithi086 podman[104837]: 2024-03-19 01:26:58.274182819 +0000 UTC m=+1.026448100 container remove 514b793e949d987bfe48a5d4799d886dee5c04d2ac88239cba699584fd826342 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS) 2024-03-19T01:26:59.006 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:58 smithi086 podman[104936]: 2024-03-19 01:26:58.650218525 +0000 UTC m=+0.019514968 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:26:59.006 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:58 smithi086 podman[104936]: 2024-03-19 01:26:58.766680076 +0000 UTC m=+0.135976502 container create c0b51ebf671923104b362c83031648c6f83f7fe6c0649710a5c3576c3c2dd3a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-19T01:26:59.390 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:59 smithi086 podman[104936]: 2024-03-19 01:26:59.01079084 +0000 UTC m=+0.380087291 container init c0b51ebf671923104b362c83031648c6f83f7fe6c0649710a5c3576c3c2dd3a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:26:59.391 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:59 smithi086 podman[104936]: 2024-03-19 01:26:59.014044981 +0000 UTC m=+0.383341406 container start c0b51ebf671923104b362c83031648c6f83f7fe6c0649710a5c3576c3c2dd3a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-19T01:26:59.391 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:59 smithi086 podman[104936]: 2024-03-19 01:26:59.102597471 +0000 UTC m=+0.471893917 container attach c0b51ebf671923104b362c83031648c6f83f7fe6c0649710a5c3576c3c2dd3a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True) 2024-03-19T01:26:59.747 INFO:journalctl@ceph.osd.1.smithi086.stdout:Mar 19 01:26:59 smithi086 podman[104936]: 2024-03-19 01:26:59.390798105 +0000 UTC m=+0.760094530 container died c0b51ebf671923104b362c83031648c6f83f7fe6c0649710a5c3576c3c2dd3a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307) 2024-03-19T01:27:00.070 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.1.service' 2024-03-19T01:27:00.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:27:00.112 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-19T01:27:00.113 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-19T01:27:00.113 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.2 2024-03-19T01:27:00.403 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:00 smithi163 systemd[1]: Stopping Ceph osd.2 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:27:00.671 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:00 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2[72574]: 2024-03-19T01:27:00.402+0000 7f645ff13700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:27:00.671 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:00 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2[72574]: 2024-03-19T01:27:00.402+0000 7f645ff13700 -1 osd.2 118 *** Got signal Terminated *** 2024-03-19T01:27:00.671 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:00 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2[72574]: 2024-03-19T01:27:00.402+0000 7f645ff13700 -1 osd.2 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:27:06.105 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:05 smithi163 podman[106880]: 2024-03-19 01:27:05.828033071 +0000 UTC m=+5.525762748 container died 742f2eea0dc47d8c41b86a606197b2916856cbbeaf303c9d0876a3ab5ef0a442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-19T01:27:06.472 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:06 smithi163 podman[106880]: 2024-03-19 01:27:06.20324507 +0000 UTC m=+5.900974739 container cleanup 742f2eea0dc47d8c41b86a606197b2916856cbbeaf303c9d0876a3ab5ef0a442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True) 2024-03-19T01:27:06.472 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:06 smithi163 bash[106880]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2 2024-03-19T01:27:06.811 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:06 smithi163 podman[106909]: 2024-03-19 01:27:06.471921264 +0000 UTC m=+0.645407626 container remove 742f2eea0dc47d8c41b86a606197b2916856cbbeaf303c9d0876a3ab5ef0a442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:27:07.168 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:06 smithi163 podman[107003]: 2024-03-19 01:27:06.716139785 +0000 UTC m=+0.017500805 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:27:07.169 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:06 smithi163 podman[107003]: 2024-03-19 01:27:06.814745992 +0000 UTC m=+0.116106991 container create b35a378f6062c2dc5be357151bb42f8056d96ea4aafd421e22cc68983fa50bed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-19T01:27:07.169 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:07 smithi163 podman[107003]: 2024-03-19 01:27:07.059838999 +0000 UTC m=+0.361200006 container init b35a378f6062c2dc5be357151bb42f8056d96ea4aafd421e22cc68983fa50bed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-19T01:27:07.169 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:07 smithi163 podman[107003]: 2024-03-19 01:27:07.062952146 +0000 UTC m=+0.364313155 container start b35a378f6062c2dc5be357151bb42f8056d96ea4aafd421e22cc68983fa50bed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-19T01:27:07.169 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:07 smithi163 podman[107003]: 2024-03-19 01:27:07.168241584 +0000 UTC m=+0.469602602 container attach b35a378f6062c2dc5be357151bb42f8056d96ea4aafd421e22cc68983fa50bed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-19T01:27:07.561 INFO:journalctl@ceph.osd.2.smithi163.stdout:Mar 19 01:27:07 smithi163 podman[107003]: 2024-03-19 01:27:07.358836969 +0000 UTC m=+0.660197994 container died b35a378f6062c2dc5be357151bb42f8056d96ea4aafd421e22cc68983fa50bed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-19T01:27:08.034 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.2.service' 2024-03-19T01:27:08.076 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:27:08.076 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-19T01:27:08.076 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-19T01:27:08.076 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.3 2024-03-19T01:27:08.477 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:08 smithi069 systemd[1]: Stopping Ceph osd.3 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:27:08.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:08 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:08.475+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:08.783 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:08 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3[50584]: 2024-03-19T01:27:08.713+0000 7f1f85800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:27:08.783 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:08 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3[50584]: 2024-03-19T01:27:08.713+0000 7f1f85800700 -1 osd.3 118 *** Got signal Terminated *** 2024-03-19T01:27:08.783 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:08 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3[50584]: 2024-03-19T01:27:08.713+0000 7f1f85800700 -1 osd.3 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:27:09.311 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:09 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:09.037+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:09.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:08 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:08.851+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:09.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:09 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:09.461+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:10.311 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:09 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:09.996+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:10.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:09 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:09.816+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:10.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:10 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:10.454+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:10.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:10 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:10.470+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:11.311 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:11 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:11.017+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:11.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:10 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:10.815+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:11.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:11 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:11.419+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:11.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:11 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:11.505+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:12.320 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:11 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:11.999+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:12.321 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:11 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:11.864+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:12.493 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:12 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3[50584]: 2024-03-19T01:27:12.126+0000 7f1f7c000700 -1 osd.3 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.618905+0000 front 2024-03-19T01:26:46.619395+0000 (oldest deadline 2024-03-19T01:27:11.319077+0000) 2024-03-19T01:27:12.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:12 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:12.385+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:12.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:12 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:12.493+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:13.311 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:12 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:12.956+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:13.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:12 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:12.830+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:13.479 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:13 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3[50584]: 2024-03-19T01:27:13.149+0000 7f1f7c000700 -1 osd.3 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.618905+0000 front 2024-03-19T01:26:46.619395+0000 (oldest deadline 2024-03-19T01:27:11.319077+0000) 2024-03-19T01:27:13.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:13 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:13.435+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:13.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:13 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:13.478+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:14.311 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:13 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:13.927+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:14.311 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:13 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:13.845+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:14.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:14 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:14.521+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:14.608 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:14 smithi069 podman[126962]: 2024-03-19 01:27:14.293369169 +0000 UTC m=+5.735141651 container stop 9a9438706daf65b301429e1d32a6487381986d90805b7fae7e3f94327fd403e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-19T01:27:14.608 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:14 smithi069 podman[126962]: 2024-03-19 01:27:14.294440368 +0000 UTC m=+5.736212893 container died 9a9438706daf65b301429e1d32a6487381986d90805b7fae7e3f94327fd403e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-19T01:27:14.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:14 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:14.414+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:15.032 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:14 smithi069 podman[126962]: 2024-03-19 01:27:14.791208221 +0000 UTC m=+6.232980739 container cleanup 9a9438706daf65b301429e1d32a6487381986d90805b7fae7e3f94327fd403e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-19T01:27:15.032 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:14 smithi069 bash[126962]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3 2024-03-19T01:27:15.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:14 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:14.799+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:15.062 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:14 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:14.910+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:15.532 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:15 smithi069 podman[126992]: 2024-03-19 01:27:15.126431312 +0000 UTC m=+0.804893636 container remove 9a9438706daf65b301429e1d32a6487381986d90805b7fae7e3f94327fd403e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-19T01:27:15.755 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:15 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:15.371+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:15.944 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:15 smithi069 podman[127082]: 2024-03-19 01:27:15.676348005 +0000 UTC m=+0.034965118 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:27:15.944 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:15 smithi069 podman[127082]: 2024-03-19 01:27:15.793888916 +0000 UTC m=+0.152506063 container create a7ad99c456a5cd6fd0def71002cc518d872ad476ad4f548170e3d82a9bf010b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T01:27:15.944 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:15 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:15.570+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:16.061 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:15.958+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:16.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:15.802+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:16.282 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:16 smithi069 podman[127082]: 2024-03-19 01:27:16.063863179 +0000 UTC m=+0.422480333 container init a7ad99c456a5cd6fd0def71002cc518d872ad476ad4f548170e3d82a9bf010b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:27:16.283 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:16 smithi069 podman[127082]: 2024-03-19 01:27:16.077039446 +0000 UTC m=+0.435656564 container start a7ad99c456a5cd6fd0def71002cc518d872ad476ad4f548170e3d82a9bf010b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-19T01:27:16.283 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:16 smithi069 podman[127082]: 2024-03-19 01:27:16.179705153 +0000 UTC m=+0.538322274 container attach a7ad99c456a5cd6fd0def71002cc518d872ad476ad4f548170e3d82a9bf010b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T01:27:16.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:16 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:16.361+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:16.783 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 19 01:27:16 smithi069 podman[127082]: 2024-03-19 01:27:16.510788329 +0000 UTC m=+0.869405523 container died a7ad99c456a5cd6fd0def71002cc518d872ad476ad4f548170e3d82a9bf010b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-19T01:27:16.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:16 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:16.558+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:17.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:16.795+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:17.062 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:16.982+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:17.324 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.3.service' 2024-03-19T01:27:17.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:27:17.448 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-19T01:27:17.448 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-19T01:27:17.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.4 2024-03-19T01:27:17.726 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:17 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:17.393+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:17.727 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:17 smithi086 systemd[1]: Stopping Ceph osd.4 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:27:18.006 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:17 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4[78002]: 2024-03-19T01:27:17.725+0000 7f2f97c83700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:27:18.006 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:17 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4[78002]: 2024-03-19T01:27:17.725+0000 7f2f97c83700 -1 osd.4 118 *** Got signal Terminated *** 2024-03-19T01:27:18.006 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:17 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4[78002]: 2024-03-19T01:27:17.725+0000 7f2f97c83700 -1 osd.4 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:27:18.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:17 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:17.545+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:18.061 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:17 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:17.932+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:18.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:17 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:17.777+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:18.755 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:18 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:18.391+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:19.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:18 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:18.578+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:19.061 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:18 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:18.926+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:19.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:18 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:18.769+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:19.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:19 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:19.394+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:19.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:19 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:19.394+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:20.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:19 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:19.610+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:20.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:19 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:19.610+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:20.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:19 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:19.804+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:20.061 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:19 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:19.912+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:20.256 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:19 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:19.951+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:20.756 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:20 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4[78002]: 2024-03-19T01:27:20.595+0000 7f2f9026f700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:54.694343+0000 front 2024-03-19T01:26:54.694369+0000 (oldest deadline 2024-03-19T01:27:20.594470+0000) 2024-03-19T01:27:20.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:20 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:20.411+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:20.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:20 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:20.411+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:21.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:20 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:20.571+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:21.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:20 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:20.571+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:21.061 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:20 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:20.900+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:21.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:20 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:20.769+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:21.255 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:20 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:20.926+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:21.756 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:21 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4[78002]: 2024-03-19T01:27:21.570+0000 7f2f9026f700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:54.694343+0000 front 2024-03-19T01:26:54.694369+0000 (oldest deadline 2024-03-19T01:27:20.594470+0000) 2024-03-19T01:27:21.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:21 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:21.452+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:21.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:21 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:21.452+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:22.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:21 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:21.572+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:22.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:21 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:21.572+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:22.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:21 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:21.712+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:22.061 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:21 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:21.897+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:22.061 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:21 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:21.897+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.294247+0000 front 2024-03-19T01:26:55.294147+0000 (oldest deadline 2024-03-19T01:27:21.194198+0000) 2024-03-19T01:27:22.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:21 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:21.807+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:22.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:21 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:21.807+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:22.256 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:21 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:21.931+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:22.756 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:22 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4[78002]: 2024-03-19T01:27:22.604+0000 7f2f9026f700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:54.694343+0000 front 2024-03-19T01:26:54.694369+0000 (oldest deadline 2024-03-19T01:27:20.594470+0000) 2024-03-19T01:27:22.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:22 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:22.489+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:22.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:22 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:22.489+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:22.811 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:22 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:22.333+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:23.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:22 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:22.678+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:23.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:22 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:22.594+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:23.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:22 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:22.594+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:23.187 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:22 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:22.929+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:23.288 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:22 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:22.888+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:23.288 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:22 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:22.888+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.294247+0000 front 2024-03-19T01:26:55.294147+0000 (oldest deadline 2024-03-19T01:27:21.194198+0000) 2024-03-19T01:27:23.288 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:22 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:22.829+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:23.289 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:22 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:22.829+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:23.443 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:23 smithi086 podman[105042]: 2024-03-19 01:27:23.18686706 +0000 UTC m=+5.547782423 container died f9066765700148353f6c73c640622a5c4b520a4524aeb7a85693960b66de5c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:27:23.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:23 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:23.287+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:23.756 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:23 smithi086 podman[105042]: 2024-03-19 01:27:23.560485611 +0000 UTC m=+5.921400956 container cleanup f9066765700148353f6c73c640622a5c4b520a4524aeb7a85693960b66de5c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:27:23.756 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:23 smithi086 bash[105042]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4 2024-03-19T01:27:23.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:23 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:23.442+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:23.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:23 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:23.442+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:24.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:23 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:23.658+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:24.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:23 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:23.596+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:24.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:23 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:23.596+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:24.170 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:23 smithi086 podman[105059]: 2024-03-19 01:27:23.854863603 +0000 UTC m=+0.670085321 container remove f9066765700148353f6c73c640622a5c4b520a4524aeb7a85693960b66de5c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True) 2024-03-19T01:27:24.170 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:23 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:23.978+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:24.250 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:23 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:23.817+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:24.250 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:23 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:23.817+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:24.250 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:23 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:23.884+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:24.250 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:23 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:23.884+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.294247+0000 front 2024-03-19T01:26:55.294147+0000 (oldest deadline 2024-03-19T01:27:21.194198+0000) 2024-03-19T01:27:24.441 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:24 smithi086 podman[105154]: 2024-03-19 01:27:24.071655511 +0000 UTC m=+0.017510237 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:27:24.442 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:24 smithi086 podman[105154]: 2024-03-19 01:27:24.188953339 +0000 UTC m=+0.134808075 container create 65408d8409109a644e45d71992760b886109715799cdc25cef4e8ee153c2bdaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-19T01:27:24.442 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:24 smithi086 podman[105154]: 2024-03-19 01:27:24.4413228 +0000 UTC m=+0.387177512 container init 65408d8409109a644e45d71992760b886109715799cdc25cef4e8ee153c2bdaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True) 2024-03-19T01:27:24.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:24 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:24.249+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:24.745 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:24 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:24.447+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:24.745 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:24 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:24.447+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:24.746 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:24 smithi086 podman[105154]: 2024-03-19 01:27:24.444564285 +0000 UTC m=+0.390419006 container start 65408d8409109a644e45d71992760b886109715799cdc25cef4e8ee153c2bdaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-03-19T01:27:24.746 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:24 smithi086 podman[105154]: 2024-03-19 01:27:24.533015988 +0000 UTC m=+0.478870726 container attach 65408d8409109a644e45d71992760b886109715799cdc25cef4e8ee153c2bdaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-19T01:27:25.005 INFO:journalctl@ceph.osd.4.smithi086.stdout:Mar 19 01:27:24 smithi086 podman[105154]: 2024-03-19 01:27:24.745423233 +0000 UTC m=+0.691277960 container died 65408d8409109a644e45d71992760b886109715799cdc25cef4e8ee153c2bdaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-19T01:27:25.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:24 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:24.627+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:25.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:24 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:24.603+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:25.033 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:24 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:24.603+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:25.246 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:24 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:24.852+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:25.247 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:24 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:24.852+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:25.247 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:24 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:24.922+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:25.247 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:24 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:24.922+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.294247+0000 front 2024-03-19T01:26:55.294147+0000 (oldest deadline 2024-03-19T01:27:21.194198+0000) 2024-03-19T01:27:25.353 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:25.014+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:25.385 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.4.service' 2024-03-19T01:27:25.425 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:27:25.426 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-19T01:27:25.426 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-19T01:27:25.426 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.5 2024-03-19T01:27:25.561 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:25 smithi163 systemd[1]: Stopping Ceph osd.5 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:27:25.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:25.245+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:25.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:25.450+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:25.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:25 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:25.450+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:26.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:25.586+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:26.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:25.629+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:26.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:25 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:25.629+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:26.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:25.867+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:26.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:25.867+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:26.062 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:25.725+0000 7f6cd12c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:27:26.062 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:25.725+0000 7f6cd12c5700 -1 osd.5 118 *** Got signal Terminated *** 2024-03-19T01:27:26.062 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:25.725+0000 7f6cd12c5700 -1 osd.5 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:27:26.062 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:25.887+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:26.062 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:25 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:25.887+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.294247+0000 front 2024-03-19T01:26:55.294147+0000 (oldest deadline 2024-03-19T01:27:21.194198+0000) 2024-03-19T01:27:26.256 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:26 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:26.001+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:26.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:26 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:26.222+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:26.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:26 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:26.485+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:26.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:26 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:26.485+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:27.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:26 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:26.635+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:27.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:26 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:26.644+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:27.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:26 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:26.644+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:27.198 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:26 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:26.822+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:27.198 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:26 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:26.822+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:27.198 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:26 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:26.822+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:27.198 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:26 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:26.890+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:27.199 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:26 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:26.890+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.294247+0000 front 2024-03-19T01:26:55.294147+0000 (oldest deadline 2024-03-19T01:27:21.194198+0000) 2024-03-19T01:27:27.505 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:27 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:27.024+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:27.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:27 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:27.196+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:27.778 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:27 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:27.531+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:27.779 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:27 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:27.531+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:28.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:27 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:27.636+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:28.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:27 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:27.621+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:28.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:27 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:27.621+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:28.162 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:27 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:27.871+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:28.162 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:27 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:27.871+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:28.163 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:27 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:27.872+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:28.163 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:27 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:27.903+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:28.163 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:27 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:27.903+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.294247+0000 front 2024-03-19T01:26:55.294147+0000 (oldest deadline 2024-03-19T01:27:21.194198+0000) 2024-03-19T01:27:28.163 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:27 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:27.903+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.795515+0000 front 2024-03-19T01:27:02.795495+0000 (oldest deadline 2024-03-19T01:27:27.495659+0000) 2024-03-19T01:27:28.502 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:28 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:28.054+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:28.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:28 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:28.161+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:28.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:28 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:28.501+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:28.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:28 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:28.501+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:29.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:28 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:28.623+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:29.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:28 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:28.623+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.117324+0000 front 2024-03-19T01:27:03.117365+0000 (oldest deadline 2024-03-19T01:27:27.817506+0000) 2024-03-19T01:27:29.033 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:28 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:28.575+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:29.033 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:28 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:28.575+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:29.133 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:28 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:28.843+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:29.134 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:28 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:28.843+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:29.134 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:28 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:28.843+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:29.134 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:28 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:28.929+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:29.134 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:28 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:28.929+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.294247+0000 front 2024-03-19T01:26:55.294147+0000 (oldest deadline 2024-03-19T01:27:21.194198+0000) 2024-03-19T01:27:29.135 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:28 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:28.929+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.795515+0000 front 2024-03-19T01:27:02.795495+0000 (oldest deadline 2024-03-19T01:27:27.495659+0000) 2024-03-19T01:27:29.473 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:29 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:29.016+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:29.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:29 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:29.132+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:29.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:29 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:29.472+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:29.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:29 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:29.472+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:30.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:29 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:29.651+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:30.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:29 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:29.651+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.117324+0000 front 2024-03-19T01:27:03.117365+0000 (oldest deadline 2024-03-19T01:27:27.817506+0000) 2024-03-19T01:27:30.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:29 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:29.569+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:30.033 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:29 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:29.569+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:30.033 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:29 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:29.569+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:30.255 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:29 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:29.979+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:30.311 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:30 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:30.121+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:30.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:29 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:29.892+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:30.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:29 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:29.892+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:30.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:29 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:29.892+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:30.312 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:29 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:29.949+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:46.691965+0000 front 2024-03-19T01:26:46.692022+0000 (oldest deadline 2024-03-19T01:27:08.991970+0000) 2024-03-19T01:27:30.312 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:29 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:29.949+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.294247+0000 front 2024-03-19T01:26:55.294147+0000 (oldest deadline 2024-03-19T01:27:21.194198+0000) 2024-03-19T01:27:30.313 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:29 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5[79145]: 2024-03-19T01:27:29.949+0000 7f6cc98b1700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.795515+0000 front 2024-03-19T01:27:02.795495+0000 (oldest deadline 2024-03-19T01:27:27.495659+0000) 2024-03-19T01:27:30.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:30 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:30.424+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:30.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:30 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:30.424+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:30.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:30 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:30.424+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:31.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:30.631+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:31.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:30.631+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.117324+0000 front 2024-03-19T01:27:03.117365+0000 (oldest deadline 2024-03-19T01:27:27.817506+0000) 2024-03-19T01:27:31.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:30.533+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:31.033 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:30.533+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:31.033 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:30 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:30.533+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:31.292 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:31 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:31.082+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:31.292 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:30 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:30.935+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:31.292 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:30 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:30.935+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:31.292 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:30 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:30.935+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:31.456 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:31 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:31.022+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:31.456 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:31 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:31.022+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:31.546 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:31 smithi163 podman[107108]: 2024-03-19 01:27:31.291605502 +0000 UTC m=+5.671836939 container died 9f818c46f6d4fe8cacf09fe793145aa3a79124b69825eba8406858ef594d3d9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-19T01:27:31.755 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:31 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:31.455+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:31.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:31 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:31.455+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:31.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:31 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:31.455+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:31.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:31.602+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:31.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:31.602+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.117324+0000 front 2024-03-19T01:27:03.117365+0000 (oldest deadline 2024-03-19T01:27:27.817506+0000) 2024-03-19T01:27:31.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:31.491+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:31.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:31.491+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:31.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:31.491+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:31.811 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:31 smithi163 podman[107108]: 2024-03-19 01:27:31.709240275 +0000 UTC m=+6.089471714 container cleanup 9f818c46f6d4fe8cacf09fe793145aa3a79124b69825eba8406858ef594d3d9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_CLEAN=True) 2024-03-19T01:27:31.811 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:31 smithi163 bash[107108]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5 2024-03-19T01:27:32.311 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:31 smithi163 podman[107134]: 2024-03-19 01:27:31.919488637 +0000 UTC m=+0.625315360 container remove 9f818c46f6d4fe8cacf09fe793145aa3a79124b69825eba8406858ef594d3d9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-19T01:27:32.312 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:32 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:32.100+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:32.312 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:32 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:32.100+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:32.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:31 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:31.887+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:32.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:31 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:31.887+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:32.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:31 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:31.887+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:32.464 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:32.024+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:32.464 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:32.024+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:32.590 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:32 smithi163 podman[107233]: 2024-03-19 01:27:32.213810698 +0000 UTC m=+0.017621243 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:27:32.590 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:32 smithi163 podman[107233]: 2024-03-19 01:27:32.347686595 +0000 UTC m=+0.151497123 container create 60634ee09ccac22ab522c71d0ae6510571434f74679a18567f073ac6a0e1427c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-19T01:27:32.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:32.463+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:32.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:32.463+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:32.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:32 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:32.463+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:32.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:32.581+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:32.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:32.581+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.117324+0000 front 2024-03-19T01:27:03.117365+0000 (oldest deadline 2024-03-19T01:27:27.817506+0000) 2024-03-19T01:27:32.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:32.466+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:32.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:32.466+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:32.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:32 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:32.466+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:32.864 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:32 smithi163 podman[107233]: 2024-03-19 01:27:32.590168233 +0000 UTC m=+0.393978771 container init 60634ee09ccac22ab522c71d0ae6510571434f74679a18567f073ac6a0e1427c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:27:32.864 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:32 smithi163 podman[107233]: 2024-03-19 01:27:32.59316564 +0000 UTC m=+0.396976182 container start 60634ee09ccac22ab522c71d0ae6510571434f74679a18567f073ac6a0e1427c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-19T01:27:32.864 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:32 smithi163 podman[107233]: 2024-03-19 01:27:32.69024938 +0000 UTC m=+0.494059923 container attach 60634ee09ccac22ab522c71d0ae6510571434f74679a18567f073ac6a0e1427c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:27:33.149 INFO:journalctl@ceph.osd.5.smithi163.stdout:Mar 19 01:27:32 smithi163 podman[107233]: 2024-03-19 01:27:32.883601234 +0000 UTC m=+0.687411763 container died 60634ee09ccac22ab522c71d0ae6510571434f74679a18567f073ac6a0e1427c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-5-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:27:33.150 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:32 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:32.862+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:33.150 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:32 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:32.862+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:33.150 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:32 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:32.862+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:33.463 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:33 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:33.043+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:33.464 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:33 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:33.043+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:33.495 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:33 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:33.148+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:33.495 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:33 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:33.148+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:33.517 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.5.service' 2024-03-19T01:27:33.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:27:33.559 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-19T01:27:33.560 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-19T01:27:33.560 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.6 2024-03-19T01:27:33.684 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:33 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:33.586+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:33.684 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:33 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:33.586+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.117324+0000 front 2024-03-19T01:27:03.117365+0000 (oldest deadline 2024-03-19T01:27:27.817506+0000) 2024-03-19T01:27:33.685 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:33 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:33.424+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:33.685 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:33 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:33.424+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:33.685 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:33 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:33.424+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:33.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:33 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:33.462+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:33.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:33 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:33.462+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:33.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:33 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:33.462+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:34.032 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:33 smithi069 systemd[1]: Stopping Ceph osd.6 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:27:34.311 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:34 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:34.101+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:34.312 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:34 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:34.101+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:34.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:33 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:33.862+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:34.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:33 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:33.862+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:34.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:33 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:33.862+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:34.448 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:34 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:34.011+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:34.449 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:34 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:34.011+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:34.473 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:34 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:34.158+0000 7fd450400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:27:34.473 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:34 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:34.158+0000 7fd450400700 -1 osd.6 118 *** Got signal Terminated *** 2024-03-19T01:27:34.473 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:34 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:34.158+0000 7fd450400700 -1 osd.6 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:27:34.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:34 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:34.447+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:34.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:34 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:34.447+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:34.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:34 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:34.447+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:34.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:34 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:34.447+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:34.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:34 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:34.543+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:34.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:34 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:34.543+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.117324+0000 front 2024-03-19T01:27:03.117365+0000 (oldest deadline 2024-03-19T01:27:27.817506+0000) 2024-03-19T01:27:34.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:34 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:34.471+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:34.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:34 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:34.471+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:34.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:34 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:34.472+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:35.256 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:35 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:35.002+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:35.256 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:35 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:35.002+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:35.256 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:35 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:35.002+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:35.311 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:35 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:35.062+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:35.311 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:35 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:35.062+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:35.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:34 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:34.860+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:35.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:34 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:34.860+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:35.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:34 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:34.860+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:35.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:35 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:35.493+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:35.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:35 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:35.493+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:35.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:35 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:35.493+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:35.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:35 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:35.493+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:35.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:35.515+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:35.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:35.515+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.117324+0000 front 2024-03-19T01:27:03.117365+0000 (oldest deadline 2024-03-19T01:27:27.817506+0000) 2024-03-19T01:27:35.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:35.515+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.319882+0000 front 2024-03-19T01:27:11.320019+0000 (oldest deadline 2024-03-19T01:27:35.419707+0000) 2024-03-19T01:27:35.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:35.472+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:35.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:35.472+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:35.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:35.472+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:35.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:35.472+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:36.311 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:36 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:36.046+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:36.312 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:36 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:36.046+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:36.312 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:36 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:36.046+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:36.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:35 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:35.898+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:36.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:35 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:35.898+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:36.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:35 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:35.898+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:36.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:36 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:36.049+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:36.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:36 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:36.049+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:36.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:36 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:36.049+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:36.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:36 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:36.531+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:36.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:36 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:36.531+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.117324+0000 front 2024-03-19T01:27:03.117365+0000 (oldest deadline 2024-03-19T01:27:27.817506+0000) 2024-03-19T01:27:36.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:36 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:36.531+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.319882+0000 front 2024-03-19T01:27:11.320019+0000 (oldest deadline 2024-03-19T01:27:35.419707+0000) 2024-03-19T01:27:36.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:36 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:36.432+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:36.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:36 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:36.432+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:36.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:36 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:36.432+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:36.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:36 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:36.432+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:37.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:36 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:36.510+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:37.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:36 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:36.510+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:37.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:36 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:36.510+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:37.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:36 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:36.510+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:37.311 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:37 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:37.073+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:37.312 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:37 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:37.073+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:37.312 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:37 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:37.073+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:37.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:36 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:36.856+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:37.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:36 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:36.856+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:37.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:36 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:36.856+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:37.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:36 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:36.856+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:37.501 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:37 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:37.041+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:37.502 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:37 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:37.041+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:37.502 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:37 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:37.041+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:37.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:37 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:37.500+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:37.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:37 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:37.500+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:37.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:37 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:37.500+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:37.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:37 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:37.500+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:37.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:37 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:37.539+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:37.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:37 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:37.539+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.117324+0000 front 2024-03-19T01:27:03.117365+0000 (oldest deadline 2024-03-19T01:27:27.817506+0000) 2024-03-19T01:27:37.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:37 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:37.539+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.319882+0000 front 2024-03-19T01:27:11.320019+0000 (oldest deadline 2024-03-19T01:27:35.419707+0000) 2024-03-19T01:27:37.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:37 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:37.446+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:37.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:37 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:37.446+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:37.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:37 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:37.446+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:37.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:37 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:37.446+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:38.311 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:38 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:38.112+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:38.311 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:38 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:38.112+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:38.312 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:38 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:38.112+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:38.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:37 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:37.884+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:38.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:37 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:37.884+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:38.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:37 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:37.884+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:38.312 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:37 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:37.884+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:38.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:38 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:38.029+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:38.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:38 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:38.029+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:38.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:38 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:38.029+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:38.684 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:38 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:38.527+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.515050+0000 front 2024-03-19T01:26:55.514938+0000 (oldest deadline 2024-03-19T01:27:21.415175+0000) 2024-03-19T01:27:38.684 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:38 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:38.527+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.117324+0000 front 2024-03-19T01:27:03.117365+0000 (oldest deadline 2024-03-19T01:27:27.817506+0000) 2024-03-19T01:27:38.684 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:38 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6[57262]: 2024-03-19T01:27:38.527+0000 7fd446c00700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.319882+0000 front 2024-03-19T01:27:11.320019+0000 (oldest deadline 2024-03-19T01:27:35.419707+0000) 2024-03-19T01:27:38.685 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:38 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:38.414+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:38.685 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:38 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:38.414+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:38.685 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:38 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:38.414+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:38.685 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:38 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:38.414+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:38.997 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:38 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:38.530+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:38.998 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:38 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:38.530+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:38.998 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:38 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:38.530+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:38.998 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:38 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:38.530+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:39.127 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:38 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:38.871+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:39.127 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:38 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:38.871+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:39.127 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:38 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:38.871+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:39.127 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:38 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:38.871+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:39.256 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:38 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:38.996+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:39.256 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:38 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:38.996+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:39.256 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:38 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:38.996+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:39.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:39 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:39.126+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:39.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:39 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:39.126+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:39.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:39 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:39.126+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:39.685 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:39 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:39.428+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:39.685 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:39 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:39.428+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:39.685 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:39 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:39.428+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:39.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:39 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:39.428+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:39.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:39 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:39.495+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:39.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:39 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:39.495+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:39.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:39 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:39.495+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:39.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:39 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:39.495+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:40.012 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:39 smithi069 podman[127187]: 2024-03-19 01:27:39.684802171 +0000 UTC m=+5.660529148 container died 3ed75fe455dcb426ba45e84bd8438789d41b0bfb79ba7907c522789e39e4560c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T01:27:40.171 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:39 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:39.856+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:40.171 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:39 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:39.856+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:40.171 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:39 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:39.856+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:40.171 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:39 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:39.856+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:40.282 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:40 smithi069 podman[127187]: 2024-03-19 01:27:40.144464611 +0000 UTC m=+6.120191578 container cleanup 3ed75fe455dcb426ba45e84bd8438789d41b0bfb79ba7907c522789e39e4560c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-19T01:27:40.282 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:40 smithi069 bash[127187]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6 2024-03-19T01:27:40.505 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:40 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:40.021+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:40.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:40 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:40.021+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:40.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:40 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:40.021+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:40.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:40 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:40.170+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:40.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:40 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:40.170+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:40.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:40 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:40.170+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:40.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:40 smithi069 podman[127207]: 2024-03-19 01:27:40.445733395 +0000 UTC m=+0.726878233 container remove 3ed75fe455dcb426ba45e84bd8438789d41b0bfb79ba7907c522789e39e4560c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:27:40.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:40.414+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:40.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:40.414+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:40.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:40.414+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:40.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:40 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:40.414+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:41.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:40 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:40.523+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:41.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:40 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:40.523+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:41.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:40 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:40.523+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:41.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:40 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:40.523+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:41.176 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:40 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:40.831+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:41.176 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:40 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:40.831+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:41.176 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:40 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:40.831+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:41.176 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:40 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:40.831+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:41.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:41 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:41.030+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:41.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:41 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:41.030+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:41.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:41 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:41.030+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:41.509 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:41 smithi069 podman[127304]: 2024-03-19 01:27:41.133945114 +0000 UTC m=+0.030876742 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:27:41.509 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:41 smithi069 podman[127304]: 2024-03-19 01:27:41.229658934 +0000 UTC m=+0.126590582 container create 0887b324059dcf59e86808f6c347ac4680cf0b1cee8d0a3433fb01f76f28b7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:27:41.509 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:41 smithi069 podman[127304]: 2024-03-19 01:27:41.508486925 +0000 UTC m=+0.405418580 container init 0887b324059dcf59e86808f6c347ac4680cf0b1cee8d0a3433fb01f76f28b7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:27:41.510 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:41 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:41.402+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:41.510 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:41 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:41.402+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:41.510 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:41 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:41.402+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:41.510 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:41 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:41.402+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:41.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:41 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:41.175+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:41.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:41 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:41.175+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:41.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:41 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:41.175+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:41.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:41 smithi069 podman[127304]: 2024-03-19 01:27:41.520397188 +0000 UTC m=+0.417328843 container start 0887b324059dcf59e86808f6c347ac4680cf0b1cee8d0a3433fb01f76f28b7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-19T01:27:41.782 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:41 smithi069 podman[127304]: 2024-03-19 01:27:41.640728675 +0000 UTC m=+0.537660380 container attach 0887b324059dcf59e86808f6c347ac4680cf0b1cee8d0a3433fb01f76f28b7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:27:42.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:41 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:41.515+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:42.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:41 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:41.516+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:42.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:41 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:41.516+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:42.006 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:41 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:41.516+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:42.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:41 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:41.821+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:42.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:41 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:41.821+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:42.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:41 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:41.821+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:42.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:41 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:41.821+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:42.274 INFO:journalctl@ceph.osd.6.smithi069.stdout:Mar 19 01:27:41 smithi069 podman[127304]: 2024-03-19 01:27:41.946791856 +0000 UTC m=+0.843723508 container died 0887b324059dcf59e86808f6c347ac4680cf0b1cee8d0a3433fb01f76f28b7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-19T01:27:42.505 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:42 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:42.013+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:42.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:42 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:42.013+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:42.506 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:42 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:42.013+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:42.532 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:42.375+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:42.533 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:42.375+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:42.533 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:42.375+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:42.533 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:42 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:42.375+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:42.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:42 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:42.142+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:42.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:42 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:42.142+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:42.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:42 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:42.142+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:42.677 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.6.service' 2024-03-19T01:27:42.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:27:42.791 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-19T01:27:42.791 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-19T01:27:42.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.7 2024-03-19T01:27:42.822 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:42 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:42.515+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:42.822 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:42 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:42.515+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:42.822 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:42 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:42.515+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:42.823 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:42 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:42.515+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:43.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:42 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:42.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:43.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:42 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:42.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:43.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:42 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:42.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:43.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:42 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:42.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:43.106 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:42 smithi086 systemd[1]: Stopping Ceph osd.7 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:27:43.106 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:43 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:43.057+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:43.107 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:43 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:43.057+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:43.107 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:43 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:43.057+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:43.107 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:43 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:43.105+0000 7f4d0d2f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:27:43.107 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:43 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:43.105+0000 7f4d0d2f1700 -1 osd.7 118 *** Got signal Terminated *** 2024-03-19T01:27:43.107 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:43 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:43.105+0000 7f4d0d2f1700 -1 osd.7 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:27:43.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:43 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:43.179+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:43.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:43 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:43.179+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:43.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:43 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:43.179+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:43.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:43 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:43.489+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:43.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:43 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:43.489+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:43.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:43 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:43.489+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:43.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:43 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:43.489+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:43.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:43 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:43.351+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:43.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:43 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:43.351+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:43.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:43 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:43.351+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:43.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:43 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:43.351+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:44.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:43 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:43.760+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:44.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:43 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:43.760+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:44.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:43 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:43.760+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:44.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:43 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:43.760+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:44.456 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:44 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:44.068+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:44.457 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:44 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:44.068+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:44.457 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:44 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:44.068+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:44.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:44.222+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:44.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:44.222+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:44.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:44.222+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:44.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:44.222+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:44.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:44 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:44.455+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:44.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:44 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:44.455+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:44.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:44 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:44.455+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:44.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:44 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:44.455+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:44.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:44 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:44.389+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:44.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:44 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:44.389+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:44.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:44 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:44.389+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:44.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:44 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:44.389+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:44.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:44 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:44.389+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:45.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:44.793+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:45.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:44.793+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:45.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:44.793+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:45.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:44 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:44.793+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:45.438 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:45 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:45.077+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:45.439 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:45 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:45.077+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:45.439 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:45 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:45.077+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:45.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:45 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:45.183+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:45.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:45 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:45.183+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:45.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:45 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:45.183+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:45.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:45 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:45.183+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:45.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:45 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:45.436+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:45.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:45 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:45.436+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:45.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:45 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:45.436+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:45.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:45 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:45.436+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:45.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:45 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:45.431+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:45.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:45 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:45.431+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:45.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:45 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:45.431+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:45.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:45 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:45.431+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:45.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:45 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:45.431+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:46.195 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:45 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:45.816+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:46.195 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:45 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:45.816+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:46.195 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:45 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:45.816+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:46.196 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:45 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:45.816+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:46.389 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:46.034+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:46.389 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:46.034+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:46.390 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:46.034+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:46.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:46.194+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:46.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:46.194+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:46.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:46.194+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:46.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:46.194+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:46.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:46.388+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:46.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:46.388+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:46.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:46.388+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:46.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:46.388+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:46.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:46 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:46.463+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:46.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:46 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:46.463+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:46.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:46 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:46.463+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:46.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:46 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:46.463+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:46.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:46 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:46.463+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:47.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:46.781+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:47.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:46.781+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:47.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:46.781+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:47.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:46 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:46.781+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:47.431 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:47.053+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:47.431 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:47.053+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:47.432 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:47.053+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:47.432 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:47.053+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.500061+0000 front 2024-03-19T01:27:22.499920+0000 (oldest deadline 2024-03-19T01:27:46.600057+0000) 2024-03-19T01:27:47.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:47.207+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:47.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:47.207+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:47.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:47.207+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:47.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:47.207+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:47.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:47.430+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:47.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:47.430+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:47.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:47.430+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:47.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:47 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:47.430+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:47.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:47 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:47.452+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:47.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:47 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:47.452+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:47.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:47 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:47.452+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:47.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:47 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:47.452+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:47.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:47 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:47.452+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:48.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:47.805+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:48.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:47.805+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:48.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:47.805+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:48.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:47 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:47.805+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:48.453 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:48 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:48.038+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:53.696531+0000 front 2024-03-19T01:26:53.696563+0000 (oldest deadline 2024-03-19T01:27:19.596623+0000) 2024-03-19T01:27:48.453 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:48 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:48.038+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:26:59.597171+0000 front 2024-03-19T01:26:59.597371+0000 (oldest deadline 2024-03-19T01:27:30.797757+0000) 2024-03-19T01:27:48.454 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:48 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:48.038+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.798538+0000 front 2024-03-19T01:27:10.798670+0000 (oldest deadline 2024-03-19T01:27:34.898444+0000) 2024-03-19T01:27:48.454 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:48 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7[85037]: 2024-03-19T01:27:48.038+0000 7f4d098e5700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.500061+0000 front 2024-03-19T01:27:22.499920+0000 (oldest deadline 2024-03-19T01:27:46.600057+0000) 2024-03-19T01:27:48.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:48 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:48.237+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:48.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:48 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:48.237+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:48.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:48 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:48.237+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:48.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:48 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:48.237+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:48.756 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:48 smithi086 podman[105271]: 2024-03-19 01:27:48.565877546 +0000 UTC m=+5.582378506 container stop 4133f42b837780a381e57b9b59bc64cb4a24f5d3f269c29263c03cf36d78e948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD) 2024-03-19T01:27:48.756 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:48 smithi086 podman[105271]: 2024-03-19 01:27:48.591458417 +0000 UTC m=+5.607959378 container died 4133f42b837780a381e57b9b59bc64cb4a24f5d3f269c29263c03cf36d78e948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2) 2024-03-19T01:27:48.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:48 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:48.452+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:48.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:48 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:48.452+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:48.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:48 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:48.452+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:48.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:48 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:48.452+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:48.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:48 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:48.445+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:48.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:48 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:48.445+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:48.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:48 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:48.445+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:48.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:48 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:48.445+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:48.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:48 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:48.445+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:49.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:48 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:48.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:49.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:48 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:48.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:49.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:48 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:48.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:49.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:48 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:48.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:49.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:48 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:48.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:27:49.134 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:48 smithi086 podman[105271]: 2024-03-19 01:27:48.913987367 +0000 UTC m=+5.930488319 container cleanup 4133f42b837780a381e57b9b59bc64cb4a24f5d3f269c29263c03cf36d78e948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:27:49.134 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:48 smithi086 bash[105271]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7 2024-03-19T01:27:49.418 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:49 smithi086 podman[105288]: 2024-03-19 01:27:49.133878533 +0000 UTC m=+0.561746938 container remove 4133f42b837780a381e57b9b59bc64cb4a24f5d3f269c29263c03cf36d78e948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-19T01:27:49.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:49.254+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:49.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:49.254+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:49.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:49.254+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:49.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:49.254+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:49.745 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:49 smithi086 podman[105379]: 2024-03-19 01:27:49.42550779 +0000 UTC m=+0.017538914 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:27:49.745 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:49 smithi086 podman[105379]: 2024-03-19 01:27:49.525477626 +0000 UTC m=+0.117508741 container create b78a3066e17e2ed1b2846f4ca089627d6535cc1787aa5d4b65b9ab16372c056f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:27:49.746 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:49 smithi086 podman[105379]: 2024-03-19 01:27:49.745181757 +0000 UTC m=+0.337212881 container init b78a3066e17e2ed1b2846f4ca089627d6535cc1787aa5d4b65b9ab16372c056f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:27:49.746 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:49.417+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:49.746 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:49.417+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:49.746 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:49.417+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:49.747 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:49.417+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:49.747 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:49 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:49.417+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:27:49.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:49 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:49.397+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:49.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:49 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:49.397+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:49.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:49 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:49.397+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:49.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:49 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:49.397+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:49.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:49 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:49.397+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:50.006 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:49 smithi086 podman[105379]: 2024-03-19 01:27:49.748510214 +0000 UTC m=+0.340541330 container start b78a3066e17e2ed1b2846f4ca089627d6535cc1787aa5d4b65b9ab16372c056f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-03-19T01:27:50.006 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:49 smithi086 podman[105379]: 2024-03-19 01:27:49.81926646 +0000 UTC m=+0.411297593 container attach b78a3066e17e2ed1b2846f4ca089627d6535cc1787aa5d4b65b9ab16372c056f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-19T01:27:50.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:49.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:50.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:49.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:50.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:49.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:50.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:49.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:50.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:49 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:49.782+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:27:50.299 INFO:journalctl@ceph.osd.7.smithi086.stdout:Mar 19 01:27:50 smithi086 podman[105379]: 2024-03-19 01:27:50.031338357 +0000 UTC m=+0.623369479 container died b78a3066e17e2ed1b2846f4ca089627d6535cc1787aa5d4b65b9ab16372c056f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:27:50.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:50 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:50.229+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:50.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:50 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:50.229+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:50.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:50 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:50.229+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:50.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:50 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:50.229+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:50.665 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:50 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:50.377+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:50.665 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:50 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:50.377+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:50.665 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:50 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:50.377+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:50.665 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:50 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:50.377+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:50.665 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:50 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:50.377+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:27:50.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.7.service' 2024-03-19T01:27:50.727 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:50 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:50.444+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:50.727 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:50 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:50.444+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:50.728 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:50 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:50.444+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:50.728 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:50 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:50.444+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:50.728 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:50 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:50.444+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:50.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:27:50.768 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-19T01:27:50.769 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-19T01:27:50.769 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.8 2024-03-19T01:27:51.061 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:50 smithi163 systemd[1]: Stopping Ceph osd.8 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:27:51.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:50 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:50.802+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:51.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:50 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:50.802+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:51.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:50 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:50.802+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:51.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:50 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:50.802+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:51.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:50 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:50.802+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:27:51.506 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:51.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:51.506 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:51.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:51.506 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:51.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:51.506 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:51.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:51.506 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:51 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:51.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:27:51.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:51.088+0000 7f08ff5fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:27:51.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:51.088+0000 7f08ff5fb700 -1 osd.8 118 *** Got signal Terminated *** 2024-03-19T01:27:51.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:51.088+0000 7f08ff5fb700 -1 osd.8 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:27:51.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:51.231+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:51.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:51.231+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:51.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:51.231+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:51.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:51.231+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:51.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:51 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:51.493+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:51.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:51 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:51.493+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:51.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:51 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:51.493+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:51.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:51 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:51.493+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:51.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:51 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:51.493+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:52.228 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:51.826+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:52.228 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:51.826+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:52.228 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:51.826+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:52.228 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:51.826+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:52.228 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:51 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:51.826+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:27:52.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:52 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:52.227+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:52.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:52 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:52.227+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:52.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:52 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:52.227+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:52.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:52 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:52.227+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:52.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:52 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:52.391+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:52.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:52 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:52.391+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:52.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:52 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:52.391+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:52.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:52 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:52.391+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:52.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:52 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:52.391+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:27:52.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:52 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:52.391+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:27:52.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:52 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:52.486+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:52.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:52 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:52.486+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:52.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:52 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:52.486+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:52.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:52 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:52.486+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:52.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:52 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:52.486+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:53.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:52 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:52.797+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:53.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:52 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:52.797+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:53.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:52 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:52.797+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:53.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:52 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:52.797+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:53.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:52 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:52.797+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:27:53.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:53 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:53.271+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:53.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:53 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:53.271+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:53.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:53 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:53.271+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:53.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:53 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:53.271+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:53.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:53 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:53.381+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:53.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:53 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:53.381+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:53.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:53 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:53.381+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:53.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:53 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:53.381+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:53.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:53 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:53.381+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:27:53.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:53 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:53.381+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:27:53.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:53 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:53.451+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:53.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:53 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:53.451+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:53.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:53 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:53.451+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:53.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:53 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:53.451+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:53.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:53 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:53.451+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:54.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:53 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:53.776+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:54.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:53 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:53.776+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:54.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:53 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:53.776+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:54.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:53 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:53.776+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:54.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:53 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:53.776+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:27:54.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:53 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:53.776+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:27:54.561 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:54 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:54.297+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:54.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:54 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:54.297+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:54.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:54 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:54.297+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:54.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:54 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:54.297+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:54.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:54 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:54.344+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:54.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:54 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:54.344+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:54.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:54 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:54.344+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:54.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:54 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:54.344+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:54.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:54 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:54.344+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:27:54.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:54 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:54.344+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:27:54.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:54 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:54.498+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:54.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:54 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:54.498+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:54.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:54 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:54.498+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:54.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:54 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:54.498+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:54.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:54 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:54.498+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:55.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:54 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:54.783+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:55.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:54 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:54.783+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:55.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:54 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:54.783+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:55.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:54 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:54.783+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:55.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:54 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:54.783+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:27:55.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:54 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:54.783+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:27:55.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:55 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:55.278+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.780231+0000 front 2024-03-19T01:26:55.780400+0000 (oldest deadline 2024-03-19T01:27:21.580695+0000) 2024-03-19T01:27:55.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:55 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:55.278+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:01.580990+0000 front 2024-03-19T01:27:01.581097+0000 (oldest deadline 2024-03-19T01:27:31.581845+0000) 2024-03-19T01:27:55.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:55 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:55.278+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:11.582515+0000 front 2024-03-19T01:27:11.582681+0000 (oldest deadline 2024-03-19T01:27:35.682454+0000) 2024-03-19T01:27:55.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:55 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:55.278+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.083935+0000 front 2024-03-19T01:27:22.083835+0000 (oldest deadline 2024-03-19T01:27:43.183878+0000) 2024-03-19T01:27:55.562 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:55 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8[86180]: 2024-03-19T01:27:55.278+0000 7f08f7be7700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.485020+0000 front 2024-03-19T01:27:28.485041+0000 (oldest deadline 2024-03-19T01:27:54.385015+0000) 2024-03-19T01:27:55.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:55 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:55.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:55.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:55 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:55.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:55.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:55 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:55.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:55.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:55 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:55.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:55.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:55 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:55.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:27:55.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:55 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:55.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:27:55.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:55.512+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:55.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:55.512+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:55.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:55.512+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:55.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:55.512+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:55.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:55 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:55.512+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:56.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:55 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:55.791+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:56.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:55 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:55.791+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:56.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:55 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:55.791+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:56.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:55 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:55.791+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:56.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:55 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:55.791+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:27:56.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:55 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:55.791+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:27:56.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:56 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:56.336+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:56.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:56 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:56.336+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:56.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:56 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:56.336+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:56.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:56 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:56.336+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:56.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:56 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:56.336+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:27:56.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:56 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:56.336+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:27:56.811 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:56 smithi163 podman[107340]: 2024-03-19 01:27:56.520158462 +0000 UTC m=+5.556816099 container died 3cdcb8e9565b0f72f99d35b5ece9db0b0525180e3d4138a2ea69392ddea020b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=) 2024-03-19T01:27:57.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:56 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:56.551+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:57.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:56 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:56.551+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:57.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:56 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:56.551+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:57.033 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:56 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:56.551+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:57.033 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:56 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:56.551+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:57.166 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:56 smithi163 podman[107340]: 2024-03-19 01:27:56.922256912 +0000 UTC m=+5.958914557 container cleanup 3cdcb8e9565b0f72f99d35b5ece9db0b0525180e3d4138a2ea69392ddea020b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-19T01:27:57.166 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:56 smithi163 bash[107340]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8 2024-03-19T01:27:57.166 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:56 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:56.834+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:57.166 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:56 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:56.834+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:57.167 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:56 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:56.834+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:57.167 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:56 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:56.834+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:57.167 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:56 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:56.834+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:27:57.167 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:56 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:56.834+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:27:57.478 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:57 smithi163 podman[107357]: 2024-03-19 01:27:57.165667729 +0000 UTC m=+0.641521093 container remove 3cdcb8e9565b0f72f99d35b5ece9db0b0525180e3d4138a2ea69392ddea020b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-19T01:27:57.657 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:57 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:57.312+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:57.658 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:57 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:57.312+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:57.658 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:57 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:57.312+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:57.658 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:57 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:57.312+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:57.658 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:57 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:57.312+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:27:57.658 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:57 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:57.312+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:27:57.731 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:57 smithi163 podman[107450]: 2024-03-19 01:27:57.381219868 +0000 UTC m=+0.018044692 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:27:57.731 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:57 smithi163 podman[107450]: 2024-03-19 01:27:57.483344595 +0000 UTC m=+0.120169448 container create e4ef088fd181f469eb74b91e748d857768d20e96ed34944305e91d951e75abd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8-deactivate, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-19T01:27:57.731 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:57 smithi163 podman[107450]: 2024-03-19 01:27:57.727677374 +0000 UTC m=+0.364502195 container init e4ef088fd181f469eb74b91e748d857768d20e96ed34944305e91d951e75abd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-19T01:27:57.731 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:57 smithi163 podman[107450]: 2024-03-19 01:27:57.730786003 +0000 UTC m=+0.367610818 container start e4ef088fd181f469eb74b91e748d857768d20e96ed34944305e91d951e75abd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:27:58.021 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:57 smithi163 podman[107450]: 2024-03-19 01:27:57.827855237 +0000 UTC m=+0.464680060 container attach e4ef088fd181f469eb74b91e748d857768d20e96ed34944305e91d951e75abd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307) 2024-03-19T01:27:58.022 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:57 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:57.829+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:58.022 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:57 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:57.829+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:58.022 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:57 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:57.829+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:58.022 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:57 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:57.829+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:58.022 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:57 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:57.829+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:27:58.022 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:57 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:57.829+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:27:58.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:57 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:57.537+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:58.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:57 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:57.537+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:58.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:57 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:57.537+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:58.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:57 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:57.537+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:58.033 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:57 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:57.537+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:58.033 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:57 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:57.537+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:30.692595+0000 front 2024-03-19T01:27:30.692645+0000 (oldest deadline 2024-03-19T01:27:56.592842+0000) 2024-03-19T01:27:58.294 INFO:journalctl@ceph.osd.8.smithi163.stdout:Mar 19 01:27:58 smithi163 podman[107450]: 2024-03-19 01:27:58.021604569 +0000 UTC m=+0.658429393 container died e4ef088fd181f469eb74b91e748d857768d20e96ed34944305e91d951e75abd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-19T01:27:58.665 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.8.service' 2024-03-19T01:27:58.668 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:58 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:58.326+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:58.668 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:58 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:58.326+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:58.668 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:58 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:58.326+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:58.668 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:58 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:58.326+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:58.668 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:58 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:58.326+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:27:58.669 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:58 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:58.326+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:27:58.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:27:58.707 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-19T01:27:58.707 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-19T01:27:58.707 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.9 2024-03-19T01:27:58.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:58 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:58.489+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:58.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:58 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:58.489+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:58.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:58 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:58.489+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:58.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:58 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:58.489+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:58.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:58 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:58.489+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:58.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:58 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:58.489+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:30.692595+0000 front 2024-03-19T01:27:30.692645+0000 (oldest deadline 2024-03-19T01:27:56.592842+0000) 2024-03-19T01:27:59.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:58 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:58.781+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:27:59.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:58 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:58.781+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:27:59.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:58 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:58.781+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:27:59.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:58 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:58.781+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:27:59.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:58 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:58.781+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:27:59.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:58 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:58.781+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:27:59.281 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:58 smithi069 systemd[1]: Stopping Ceph osd.9 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:27:59.759 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:59 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:59.341+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:27:59.759 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:59 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:59.342+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:27:59.759 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:59 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:59.342+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:27:59.760 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:59 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:59.342+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:27:59.760 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:59 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:59.342+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:27:59.760 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:27:59 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:27:59.342+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:27:59.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:59 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:59.294+0000 7f8353a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:27:59.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:59 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:59.294+0000 7f8353a00700 -1 osd.9 118 *** Got signal Terminated *** 2024-03-19T01:27:59.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:59 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:59.294+0000 7f8353a00700 -1 osd.9 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:27:59.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:59 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:59.477+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:27:59.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:59 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:59.477+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:27:59.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:59 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:59.477+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:27:59.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:59 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:59.477+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:27:59.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:59 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:59.477+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:27:59.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:27:59 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:27:59.477+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:30.692595+0000 front 2024-03-19T01:27:30.692645+0000 (oldest deadline 2024-03-19T01:27:56.592842+0000) 2024-03-19T01:28:00.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:59 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:59.749+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:00.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:59 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:59.749+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:00.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:59 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:59.749+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:00.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:59 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:59.749+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:00.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:59 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:59.749+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:00.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:27:59 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:27:59.749+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:00.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:00 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:00.297+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:00.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:00 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:00.297+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:00.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:00 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:00.297+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:00.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:00 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:00.297+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:00.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:00 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:00.297+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:00.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:00 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:00.297+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:00.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:00.478+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:28:00.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:00.478+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:28:00.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:00.478+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:28:00.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:00.478+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:28:00.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:00.478+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:28:00.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:00 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:00.478+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:30.692595+0000 front 2024-03-19T01:27:30.692645+0000 (oldest deadline 2024-03-19T01:27:56.592842+0000) 2024-03-19T01:28:01.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:00 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:00.763+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:01.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:00 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:00.763+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:01.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:00 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:00.763+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:01.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:00 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:00.763+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:01.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:00 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:00.763+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:01.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:00 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:00.763+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:01.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:01 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:01.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:01.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:01 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:01.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:01.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:01 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:01.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:01.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:01 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:01.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:01.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:01 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:01.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:01.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:01 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:01.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:01.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:01 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:01.324+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:01.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:01 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:01.473+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:28:01.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:01 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:01.473+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:28:01.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:01 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:01.473+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:28:01.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:01 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:01.473+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:28:01.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:01 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:01.473+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:28:01.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:01 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:01.473+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:30.692595+0000 front 2024-03-19T01:27:30.692645+0000 (oldest deadline 2024-03-19T01:27:56.592842+0000) 2024-03-19T01:28:02.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:01 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:01.741+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:02.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:01 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:01.741+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:02.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:01 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:01.741+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:02.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:01 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:01.741+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:02.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:01 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:01.741+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:02.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:01 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:01.741+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:02.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:02 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:02.360+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:02.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:02 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:02.360+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:02.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:02 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:02.360+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:02.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:02 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:02.360+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:02.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:02 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:02.360+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:02.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:02 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:02.360+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:02.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:02 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:02.360+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:02.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:02 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:02.491+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:28:02.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:02 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:02.491+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:28:02.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:02 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:02.491+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:28:02.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:02 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:02.491+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:28:02.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:02 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:02.491+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:28:02.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:02 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:02.491+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:30.692595+0000 front 2024-03-19T01:27:30.692645+0000 (oldest deadline 2024-03-19T01:27:56.592842+0000) 2024-03-19T01:28:03.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:02 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:02.732+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:03.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:02 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:02.732+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:03.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:02 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:02.732+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:03.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:02 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:02.732+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:03.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:02 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:02.732+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:03.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:02 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:02.732+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:03.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:03 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:03.339+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:03.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:03 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:03.340+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:03.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:03 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:03.340+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:03.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:03 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:03.340+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:03.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:03 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:03.340+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:03.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:03 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:03.340+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:03.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:03 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:03.340+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:03.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:03 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:03.451+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:44.875577+0000 front 2024-03-19T01:26:44.875735+0000 (oldest deadline 2024-03-19T01:27:08.374711+0000) 2024-03-19T01:28:03.782 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:03 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:03.451+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:52.977775+0000 front 2024-03-19T01:26:52.978124+0000 (oldest deadline 2024-03-19T01:27:18.778846+0000) 2024-03-19T01:28:03.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:03 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:03.451+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:03.982230+0000 front 2024-03-19T01:27:03.982373+0000 (oldest deadline 2024-03-19T01:27:29.183464+0000) 2024-03-19T01:28:03.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:03 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:03.451+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:09.184230+0000 front 2024-03-19T01:27:09.184803+0000 (oldest deadline 2024-03-19T01:27:35.084268+0000) 2024-03-19T01:28:03.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:03 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:03.451+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:21.988792+0000 front 2024-03-19T01:27:21.989210+0000 (oldest deadline 2024-03-19T01:27:43.689037+0000) 2024-03-19T01:28:03.783 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:03 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9[63855]: 2024-03-19T01:28:03.451+0000 7f834a200700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:30.692595+0000 front 2024-03-19T01:27:30.692645+0000 (oldest deadline 2024-03-19T01:27:56.592842+0000) 2024-03-19T01:28:04.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:03 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:03.725+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:04.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:03 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:03.725+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:04.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:03 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:03.725+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:04.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:03 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:03.725+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:04.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:03 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:03.725+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:04.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:03 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:03.725+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:04.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:04 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:04.348+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:04.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:04 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:04.348+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:04.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:04 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:04.348+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:04.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:04 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:04.348+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:04.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:04 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:04.348+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:04.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:04 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:04.348+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:04.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:04 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:04.348+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:05.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:04 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:04.695+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:05.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:04 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:04.695+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:05.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:04 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:04.695+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:05.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:04 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:04.695+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:05.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:04 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:04.695+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:05.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:04 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:04.695+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:05.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:04 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:04.695+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:05.189 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:04 smithi069 podman[127411]: 2024-03-19 01:28:04.89138158 +0000 UTC m=+5.752949048 container died d1cbed7bce70f0074bf6e693814a82664011833a964144a3739df0bee6d68709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-19T01:28:05.531 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:05 smithi069 podman[127411]: 2024-03-19 01:28:05.329564264 +0000 UTC m=+6.191131669 container cleanup d1cbed7bce70f0074bf6e693814a82664011833a964144a3739df0bee6d68709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:28:05.532 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:05 smithi069 bash[127411]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9 2024-03-19T01:28:05.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:05 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:05.347+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:05.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:05 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:05.347+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:05.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:05 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:05.347+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:05.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:05 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:05.347+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:05.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:05 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:05.347+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:05.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:05 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:05.347+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:05.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:05 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:05.347+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:05.781 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:05 smithi069 podman[127431]: 2024-03-19 01:28:05.530666649 +0000 UTC m=+0.603915456 container remove d1cbed7bce70f0074bf6e693814a82664011833a964144a3739df0bee6d68709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-19T01:28:06.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:05 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:05.679+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:06.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:05 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:05.679+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:06.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:05 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:05.679+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:06.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:05 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:05.679+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:06.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:05 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:05.679+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:06.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:05 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:05.679+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:06.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:05 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:05.679+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:06.644 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:06 smithi069 podman[127521]: 2024-03-19 01:28:06.253690582 +0000 UTC m=+0.033920115 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:28:06.645 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:06 smithi069 podman[127521]: 2024-03-19 01:28:06.373160326 +0000 UTC m=+0.153389883 container create bffddf9eff40016263fa29c9bd0bd93faafd0c52f28ae6a2cd5509d526626792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-19T01:28:06.645 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:06 smithi069 podman[127521]: 2024-03-19 01:28:06.64349624 +0000 UTC m=+0.423725787 container init bffddf9eff40016263fa29c9bd0bd93faafd0c52f28ae6a2cd5509d526626792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-19T01:28:06.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:06 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:06.361+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:06.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:06 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:06.361+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:06.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:06 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:06.361+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:06.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:06 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:06.361+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:06.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:06 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:06.361+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:06.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:06 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:06.361+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:06.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:06 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:06.361+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:07.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:06 smithi069 podman[127521]: 2024-03-19 01:28:06.654360081 +0000 UTC m=+0.434589625 container start bffddf9eff40016263fa29c9bd0bd93faafd0c52f28ae6a2cd5509d526626792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:28:07.032 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:06 smithi069 podman[127521]: 2024-03-19 01:28:06.733667787 +0000 UTC m=+0.513897368 container attach bffddf9eff40016263fa29c9bd0bd93faafd0c52f28ae6a2cd5509d526626792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-03-19T01:28:07.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:06 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:06.641+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:07.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:06 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:06.641+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:07.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:06 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:06.641+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:07.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:06 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:06.641+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:07.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:06 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:06.641+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:07.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:06 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:06.641+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:07.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:06 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:06.641+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:07.424 INFO:journalctl@ceph.osd.9.smithi069.stdout:Mar 19 01:28:07 smithi069 podman[127521]: 2024-03-19 01:28:07.094642936 +0000 UTC m=+0.874872500 container died bffddf9eff40016263fa29c9bd0bd93faafd0c52f28ae6a2cd5509d526626792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-9-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-19T01:28:07.733 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.9.service' 2024-03-19T01:28:07.758 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:07 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:07.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:07.758 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:07 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:07.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:07.758 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:07 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:07.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:07.759 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:07 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:07.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:07.759 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:07 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:07.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:07.759 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:07 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:07.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:07.759 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:07 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:07.356+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:07.830 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:28:07.831 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-19T01:28:07.831 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-19T01:28:07.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.10 2024-03-19T01:28:08.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:07 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:07.631+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:08.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:07 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:07.631+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:08.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:07 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:07.631+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:08.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:07 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:07.631+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:08.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:07 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:07.631+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:08.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:07 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:07.631+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:08.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:07 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:07.631+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:08.177 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:07 smithi086 systemd[1]: Stopping Ceph osd.10 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:28:08.506 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:08 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:08.176+0000 7f7cd9d05700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:28:08.506 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:08 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:08.176+0000 7f7cd9d05700 -1 osd.10 118 *** Got signal Terminated *** 2024-03-19T01:28:08.507 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:08 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:08.176+0000 7f7cd9d05700 -1 osd.10 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:28:08.507 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:08 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:08.397+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:08.507 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:08 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:08.397+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:08.507 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:08 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:08.397+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:08.508 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:08 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:08.397+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:08.508 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:08 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:08.397+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:08.508 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:08 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:08.397+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:08.508 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:08 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:08.397+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:09.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:08 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:08.588+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:09.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:08 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:08.588+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:09.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:08 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:08.588+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:09.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:08 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:08.588+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:09.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:08 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:08.588+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:09.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:08 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:08.588+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:09.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:08 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:08.588+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:09.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:09 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:09.354+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:09.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:09 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:09.354+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:09.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:09 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:09.354+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:09.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:09 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:09.354+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:09.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:09 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:09.354+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:09.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:09 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:09.354+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:09.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:09 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:09.354+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:10.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:09 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:09.610+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:10.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:09 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:09.610+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:10.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:09 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:09.610+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:10.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:09 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:09.610+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:10.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:09 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:09.610+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:10.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:09 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:09.610+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:10.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:09 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:09.610+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:10.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:10 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:10.329+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:10.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:10 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:10.329+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:10.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:10 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:10.329+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:10.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:10 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:10.329+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:10.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:10 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:10.329+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:10.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:10 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:10.329+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:10.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:10 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:10.329+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:11.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:10 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:10.613+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:11.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:10 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:10.613+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:11.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:10 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:10.613+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:11.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:10 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:10.613+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:11.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:10 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:10.613+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:11.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:10 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:10.613+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:11.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:10 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:10.613+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:11.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:11 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:11.358+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:11.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:11 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:11.358+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:11.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:11 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:11.358+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:11.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:11 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:11.358+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:11.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:11 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:11.358+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:11.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:11 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:11.358+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:11.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:11 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:11.358+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:12.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:11 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:11.567+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:12.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:11 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:11.567+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:12.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:11 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:11.567+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:12.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:11 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:11.567+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:12.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:11 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:11.567+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:12.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:11 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:11.567+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:12.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:11 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:11.567+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:12.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:12 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:12.341+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:43.598913+0000 front 2024-03-19T01:26:43.599037+0000 (oldest deadline 2024-03-19T01:27:09.999448+0000) 2024-03-19T01:28:12.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:12 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:12.341+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.900241+0000 front 2024-03-19T01:26:55.900297+0000 (oldest deadline 2024-03-19T01:27:19.300838+0000) 2024-03-19T01:28:12.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:12 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:12.341+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:05.202095+0000 front 2024-03-19T01:27:05.201998+0000 (oldest deadline 2024-03-19T01:27:29.802277+0000) 2024-03-19T01:28:12.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:12 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:12.341+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.903335+0000 front 2024-03-19T01:27:10.903238+0000 (oldest deadline 2024-03-19T01:27:34.403142+0000) 2024-03-19T01:28:12.756 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:12 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:12.341+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.604522+0000 front 2024-03-19T01:27:22.604562+0000 (oldest deadline 2024-03-19T01:27:48.504604+0000) 2024-03-19T01:28:12.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:12 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:12.341+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.505201+0000 front 2024-03-19T01:27:28.505156+0000 (oldest deadline 2024-03-19T01:27:51.905481+0000) 2024-03-19T01:28:12.757 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:12 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10[92082]: 2024-03-19T01:28:12.341+0000 7f7cd22f1700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.307240+0000 front 2024-03-19T01:27:38.307336+0000 (oldest deadline 2024-03-19T01:28:00.607029+0000) 2024-03-19T01:28:13.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:12 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:12.605+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:13.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:12 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:12.605+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:13.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:12 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:12.605+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:13.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:12 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:12.605+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:13.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:12 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:12.605+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:13.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:12 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:12.605+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:13.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:12 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:12.605+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:13.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:12 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:12.605+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-03-19T01:27:47.976273+0000 front 2024-03-19T01:27:47.976294+0000 (oldest deadline 2024-03-19T01:28:12.576738+0000) 2024-03-19T01:28:13.935 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:13 smithi086 podman[105497]: 2024-03-19 01:28:13.680540818 +0000 UTC m=+5.659305709 container died 6a1ba4d29e85422689cd9aba866bc9bc89aaa10c67115d3f9053049abe1d620d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2) 2024-03-19T01:28:14.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:13 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:13.591+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:14.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:13 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:13.591+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:14.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:13 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:13.591+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:14.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:13 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:13.591+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:14.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:13 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:13.591+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:14.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:13 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:13.591+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:14.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:13 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:13.591+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:14.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:13 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:13.591+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-03-19T01:27:47.976273+0000 front 2024-03-19T01:27:47.976294+0000 (oldest deadline 2024-03-19T01:28:12.576738+0000) 2024-03-19T01:28:14.202 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:14 smithi086 podman[105497]: 2024-03-19 01:28:14.025604982 +0000 UTC m=+6.004369870 container cleanup 6a1ba4d29e85422689cd9aba866bc9bc89aaa10c67115d3f9053049abe1d620d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-19T01:28:14.202 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:14 smithi086 bash[105497]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10 2024-03-19T01:28:14.505 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:14 smithi086 podman[105515]: 2024-03-19 01:28:14.201547841 +0000 UTC m=+0.516550694 container remove 6a1ba4d29e85422689cd9aba866bc9bc89aaa10c67115d3f9053049abe1d620d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:28:14.811 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:14 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:14.543+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:14.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:14 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:14.543+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:14.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:14 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:14.543+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:14.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:14 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:14.543+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:14.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:14 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:14.543+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:14.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:14 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:14.543+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:14.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:14 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:14.543+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:14.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:14 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:14.543+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-03-19T01:27:47.976273+0000 front 2024-03-19T01:27:47.976294+0000 (oldest deadline 2024-03-19T01:28:12.576738+0000) 2024-03-19T01:28:14.905 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:14 smithi086 podman[105604]: 2024-03-19 01:28:14.52923614 +0000 UTC m=+0.017865868 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:28:14.905 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:14 smithi086 podman[105604]: 2024-03-19 01:28:14.627225102 +0000 UTC m=+0.115854818 container create dbd4f7a0259f224af156911f7c250ed8b1ecaaf7b4e3c20fcaf8603433f5d905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-19T01:28:14.905 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:14 smithi086 podman[105604]: 2024-03-19 01:28:14.838414313 +0000 UTC m=+0.327044029 container init dbd4f7a0259f224af156911f7c250ed8b1ecaaf7b4e3c20fcaf8603433f5d905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-19T01:28:14.905 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:14 smithi086 podman[105604]: 2024-03-19 01:28:14.842333125 +0000 UTC m=+0.330962842 container start dbd4f7a0259f224af156911f7c250ed8b1ecaaf7b4e3c20fcaf8603433f5d905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD) 2024-03-19T01:28:14.905 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:14 smithi086 podman[105604]: 2024-03-19 01:28:14.90478711 +0000 UTC m=+0.393416829 container attach dbd4f7a0259f224af156911f7c250ed8b1ecaaf7b4e3c20fcaf8603433f5d905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-03-19T01:28:15.255 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:15 smithi086 podman[105604]: 2024-03-19 01:28:15.130800084 +0000 UTC m=+0.619429827 container died dbd4f7a0259f224af156911f7c250ed8b1ecaaf7b4e3c20fcaf8603433f5d905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-19T01:28:15.617 INFO:journalctl@ceph.osd.10.smithi086.stdout:Mar 19 01:28:15 smithi086 podman[105630]: 2024-03-19 01:28:15.607450569 +0000 UTC m=+0.471341184 container remove dbd4f7a0259f224af156911f7c250ed8b1ecaaf7b4e3c20fcaf8603433f5d905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-10-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-19T01:28:15.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.10.service' 2024-03-19T01:28:15.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:28:15.663 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-19T01:28:15.663 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-19T01:28:15.663 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.11 2024-03-19T01:28:15.811 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:15.532+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:15.811 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:15.532+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:15.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:15.532+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:15.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:15.532+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:15.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:15.532+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:15.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:15.532+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:15.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:15.532+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:15.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:15.532+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-03-19T01:27:47.976273+0000 front 2024-03-19T01:27:47.976294+0000 (oldest deadline 2024-03-19T01:28:12.576738+0000) 2024-03-19T01:28:15.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 systemd[1]: Stopping Ceph osd.11 for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:28:16.311 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:15.975+0000 7f612bbde700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:28:16.311 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:15.975+0000 7f612bbde700 -1 osd.11 118 *** Got signal Terminated *** 2024-03-19T01:28:16.311 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:15 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:15.975+0000 7f612bbde700 -1 osd.11 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T01:28:16.811 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:16.535+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:16.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:16.535+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:16.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:16.535+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:16.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:16.535+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:16.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:16.535+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:16.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:16.535+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:16.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:16.535+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:16.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:16 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:16.535+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-03-19T01:27:47.976273+0000 front 2024-03-19T01:27:47.976294+0000 (oldest deadline 2024-03-19T01:28:12.576738+0000) 2024-03-19T01:28:18.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:17 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:17.560+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:18.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:17 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:17.560+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:18.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:17 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:17.560+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:18.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:17 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:17.560+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:18.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:17 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:17.560+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:18.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:17 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:17.560+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:18.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:17 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:17.560+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:18.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:17 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:17.560+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-03-19T01:27:47.976273+0000 front 2024-03-19T01:27:47.976294+0000 (oldest deadline 2024-03-19T01:28:12.576738+0000) 2024-03-19T01:28:19.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:18 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:18.569+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:19.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:18 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:18.569+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:19.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:18 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:18.569+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:19.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:18 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:18.569+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:19.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:18 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:18.569+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:19.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:18 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:18.569+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:19.062 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:18 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:18.569+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:19.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:18 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:18.569+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-03-19T01:27:47.976273+0000 front 2024-03-19T01:27:47.976294+0000 (oldest deadline 2024-03-19T01:28:12.576738+0000) 2024-03-19T01:28:19.063 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:18 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:18.569+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6820 osd.8 since back 2024-03-19T01:27:52.577291+0000 front 2024-03-19T01:27:52.577341+0000 (oldest deadline 2024-03-19T01:28:18.477373+0000) 2024-03-19T01:28:19.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:19 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:19.556+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:19.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:19 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:19.556+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:19.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:19 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:19.556+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:19.813 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:19 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:19.556+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:19.813 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:19 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:19.556+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:19.813 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:19 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:19.556+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:19.813 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:19 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:19.556+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:19.813 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:19 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:19.556+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-03-19T01:27:47.976273+0000 front 2024-03-19T01:27:47.976294+0000 (oldest deadline 2024-03-19T01:28:12.576738+0000) 2024-03-19T01:28:19.814 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:19 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:19.556+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6820 osd.8 since back 2024-03-19T01:27:52.577291+0000 front 2024-03-19T01:27:52.577341+0000 (oldest deadline 2024-03-19T01:28:18.477373+0000) 2024-03-19T01:28:20.811 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:20 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:20.516+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-03-19T01:26:42.864556+0000 front 2024-03-19T01:26:42.864885+0000 (oldest deadline 2024-03-19T01:27:08.164408+0000) 2024-03-19T01:28:20.811 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:20 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:20.516+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-03-19T01:26:55.666890+0000 front 2024-03-19T01:26:55.667000+0000 (oldest deadline 2024-03-19T01:27:21.566659+0000) 2024-03-19T01:28:20.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:20 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:20.516+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-03-19T01:27:02.568404+0000 front 2024-03-19T01:27:02.568198+0000 (oldest deadline 2024-03-19T01:27:26.068428+0000) 2024-03-19T01:28:20.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:20 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:20.516+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-03-19T01:27:10.670619+0000 front 2024-03-19T01:27:10.670690+0000 (oldest deadline 2024-03-19T01:27:36.570414+0000) 2024-03-19T01:28:20.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:20 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:20.516+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-03-19T01:27:22.372107+0000 front 2024-03-19T01:27:22.372310+0000 (oldest deadline 2024-03-19T01:27:48.271849+0000) 2024-03-19T01:28:20.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:20 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:20.516+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6812 osd.5 since back 2024-03-19T01:27:28.772869+0000 front 2024-03-19T01:27:28.773065+0000 (oldest deadline 2024-03-19T01:27:53.472993+0000) 2024-03-19T01:28:20.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:20 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:20.516+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-03-19T01:27:38.074768+0000 front 2024-03-19T01:27:38.074850+0000 (oldest deadline 2024-03-19T01:28:03.974520+0000) 2024-03-19T01:28:20.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:20 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:20.516+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-03-19T01:27:47.976273+0000 front 2024-03-19T01:27:47.976294+0000 (oldest deadline 2024-03-19T01:28:12.576738+0000) 2024-03-19T01:28:20.812 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:20 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11[93243]: 2024-03-19T01:28:20.516+0000 7f61281d2700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.163:6820 osd.8 since back 2024-03-19T01:27:52.577291+0000 front 2024-03-19T01:27:52.577341+0000 (oldest deadline 2024-03-19T01:28:18.477373+0000) 2024-03-19T01:28:21.642 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:21 smithi163 podman[107567]: 2024-03-19 01:28:21.311773336 +0000 UTC m=+5.452976028 container died 507b831e9fe1b762f5aa793517a67a72ef9f7a7302f71ee5db73160f090eef60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-19T01:28:21.642 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:21 smithi163 podman[107567]: 2024-03-19 01:28:21.642254252 +0000 UTC m=+5.783456943 container cleanup 507b831e9fe1b762f5aa793517a67a72ef9f7a7302f71ee5db73160f090eef60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-19T01:28:22.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:21 smithi163 bash[107567]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11 2024-03-19T01:28:22.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:21 smithi163 podman[107585]: 2024-03-19 01:28:21.865238691 +0000 UTC m=+0.547970808 container remove 507b831e9fe1b762f5aa793517a67a72ef9f7a7302f71ee5db73160f090eef60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-19T01:28:22.562 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:22 smithi163 podman[107681]: 2024-03-19 01:28:22.269522974 +0000 UTC m=+0.082557490 container create a7c0ab2406429acb3b5b37cb3c88f69c8eb7cf9e3b4d53c703cb9933ebfe6a72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-19T01:28:22.562 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:22 smithi163 podman[107681]: 2024-03-19 01:28:22.204911254 +0000 UTC m=+0.017945794 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:28:22.562 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:22 smithi163 podman[107681]: 2024-03-19 01:28:22.497546651 +0000 UTC m=+0.310581180 container init a7c0ab2406429acb3b5b37cb3c88f69c8eb7cf9e3b4d53c703cb9933ebfe6a72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-19T01:28:22.562 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:22 smithi163 podman[107681]: 2024-03-19 01:28:22.500909245 +0000 UTC m=+0.313943776 container start a7c0ab2406429acb3b5b37cb3c88f69c8eb7cf9e3b4d53c703cb9933ebfe6a72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-19T01:28:23.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:22 smithi163 podman[107681]: 2024-03-19 01:28:22.588597515 +0000 UTC m=+0.401632055 container attach a7c0ab2406429acb3b5b37cb3c88f69c8eb7cf9e3b4d53c703cb9933ebfe6a72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-19T01:28:23.061 INFO:journalctl@ceph.osd.11.smithi163.stdout:Mar 19 01:28:22 smithi163 podman[107681]: 2024-03-19 01:28:22.789878744 +0000 UTC m=+0.602913263 container died a7c0ab2406429acb3b5b37cb3c88f69c8eb7cf9e3b4d53c703cb9933ebfe6a72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:28:23.360 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@osd.11.service' 2024-03-19T01:28:23.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:28:23.401 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-19T01:28:23.401 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-19T01:28:23.402 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.a 2024-03-19T01:28:23.782 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:28:23 smithi069 systemd[1]: Stopping Ceph mds.a for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:28:24.282 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:28:23 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a[70809]: 2024-03-19T01:28:23.944+0000 7f5747000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:28:24.282 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:28:23 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a[70809]: 2024-03-19T01:28:23.944+0000 7f5747000700 -1 mds.a *** got signal Terminated *** 2024-03-19T01:28:25.460 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:28:25 smithi069 podman[127638]: 2024-03-19 01:28:25.151302536 +0000 UTC m=+1.329914511 container died e20f4f9f69ca57d47970edaf30bcec3b65cb2254f4eeb4532f7b33839d296052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-19T01:28:25.782 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:28:25 smithi069 podman[127638]: 2024-03-19 01:28:25.571420131 +0000 UTC m=+1.750032097 container cleanup e20f4f9f69ca57d47970edaf30bcec3b65cb2254f4eeb4532f7b33839d296052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-03-19T01:28:25.782 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:28:25 smithi069 bash[127638]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a 2024-03-19T01:28:26.282 INFO:journalctl@ceph.mds.a.smithi069.stdout:Mar 19 01:28:25 smithi069 podman[127651]: 2024-03-19 01:28:25.880523989 +0000 UTC m=+0.713336773 container remove e20f4f9f69ca57d47970edaf30bcec3b65cb2254f4eeb4532f7b33839d296052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-a, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-19T01:28:26.368 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.a.service' 2024-03-19T01:28:26.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:28:26.485 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-19T01:28:26.485 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-19T01:28:26.485 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.d 2024-03-19T01:28:27.032 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:28:26 smithi069 systemd[1]: Stopping Ceph mds.d for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:28:27.532 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:28:27 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d[69820]: 2024-03-19T01:28:27.089+0000 7f74f5400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:28:27.532 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:28:27 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d[69820]: 2024-03-19T01:28:27.089+0000 7f74f5400700 -1 mds.d *** got signal Terminated *** 2024-03-19T01:28:29.480 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:28:29 smithi069 podman[127761]: 2024-03-19 01:28:29.20379342 +0000 UTC m=+2.304160864 container died c66cfdc41c180d37c8f7c4cad0d0c5459fa002ba6bc246a49d16aae689c57974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-19T01:28:29.782 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:28:29 smithi069 podman[127761]: 2024-03-19 01:28:29.658431041 +0000 UTC m=+2.758798504 container cleanup c66cfdc41c180d37c8f7c4cad0d0c5459fa002ba6bc246a49d16aae689c57974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-19T01:28:29.782 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:28:29 smithi069 bash[127761]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d 2024-03-19T01:28:30.274 INFO:journalctl@ceph.mds.d.smithi069.stdout:Mar 19 01:28:29 smithi069 podman[127775]: 2024-03-19 01:28:29.826229113 +0000 UTC m=+0.606207246 container remove c66cfdc41c180d37c8f7c4cad0d0c5459fa002ba6bc246a49d16aae689c57974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T01:28:30.309 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.d.service' 2024-03-19T01:28:30.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:28:30.427 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-19T01:28:30.427 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-19T01:28:30.428 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.g 2024-03-19T01:28:31.027 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:28:30 smithi069 systemd[1]: Stopping Ceph mds.g for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:28:31.282 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:28:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-g[70464]: 2024-03-19T01:28:31.025+0000 7f82f3000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:28:31.282 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:28:31 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-g[70464]: 2024-03-19T01:28:31.025+0000 7f82f3000700 -1 mds.g *** got signal Terminated *** 2024-03-19T01:28:34.142 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:28:33 smithi069 podman[127881]: 2024-03-19 01:28:33.831732889 +0000 UTC m=+2.925142157 container died 7adea5bd5206be3bc2b21c80569326c52472ae36b9561609b34aee3fb36066da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-g, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS) 2024-03-19T01:28:34.454 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:28:34 smithi069 podman[127881]: 2024-03-19 01:28:34.253130405 +0000 UTC m=+3.346539646 container cleanup 7adea5bd5206be3bc2b21c80569326c52472ae36b9561609b34aee3fb36066da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-19T01:28:34.455 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:28:34 smithi069 bash[127881]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-g 2024-03-19T01:28:34.782 INFO:journalctl@ceph.mds.g.smithi069.stdout:Mar 19 01:28:34 smithi069 podman[127895]: 2024-03-19 01:28:34.454059827 +0000 UTC m=+0.603717289 container remove 7adea5bd5206be3bc2b21c80569326c52472ae36b9561609b34aee3fb36066da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-19T01:28:34.937 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.g.service' 2024-03-19T01:28:35.046 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:28:35.046 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-19T01:28:35.047 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-19T01:28:35.047 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.j 2024-03-19T01:28:35.282 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:28:35 smithi069 systemd[1]: Stopping Ceph mds.j for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:28:36.032 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:28:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-j[69473]: 2024-03-19T01:28:35.612+0000 7fb9abe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:28:36.032 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:28:35 smithi069 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-j[69473]: 2024-03-19T01:28:35.612+0000 7fb9abe00700 -1 mds.j *** got signal Terminated *** 2024-03-19T01:28:39.630 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:28:39 smithi069 podman[128000]: 2024-03-19 01:28:39.377968225 +0000 UTC m=+3.894936486 container died 65388de9d2b5c7888854986dbdf5b13a7273b17d046d4d6a95407a3f8e7b6878 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-j, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-19T01:28:39.992 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:28:39 smithi069 podman[128000]: 2024-03-19 01:28:39.809229725 +0000 UTC m=+4.326197958 container cleanup 65388de9d2b5c7888854986dbdf5b13a7273b17d046d4d6a95407a3f8e7b6878 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-19T01:28:39.992 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:28:39 smithi069 bash[128000]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-j 2024-03-19T01:28:40.281 INFO:journalctl@ceph.mds.j.smithi069.stdout:Mar 19 01:28:39 smithi069 podman[128013]: 2024-03-19 01:28:39.990888284 +0000 UTC m=+0.602292964 container remove 65388de9d2b5c7888854986dbdf5b13a7273b17d046d4d6a95407a3f8e7b6878 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-j, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:28:40.465 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.j.service' 2024-03-19T01:28:40.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:28:40.581 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-19T01:28:40.582 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-19T01:28:40.582 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.b 2024-03-19T01:28:40.875 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:28:40 smithi086 systemd[1]: Stopping Ceph mds.b for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:28:41.138 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:28:40 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b[97627]: 2024-03-19T01:28:40.874+0000 7f95b9f4c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:28:41.139 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:28:40 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b[97627]: 2024-03-19T01:28:40.874+0000 7f95b9f4c700 -1 mds.b *** got signal Terminated *** 2024-03-19T01:28:45.208 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:28:44 smithi086 podman[105709]: 2024-03-19 01:28:44.884358163 +0000 UTC m=+4.103924481 container died f8a2b62ff9d5110359c45a6d4780a417ebf192dd64616544baa01c1a23e5468e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-19T01:28:45.208 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:28:45 smithi086 podman[105709]: 2024-03-19 01:28:45.208054322 +0000 UTC m=+4.427620629 container cleanup f8a2b62ff9d5110359c45a6d4780a417ebf192dd64616544baa01c1a23e5468e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-19T01:28:45.506 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:28:45 smithi086 bash[105709]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b 2024-03-19T01:28:45.506 INFO:journalctl@ceph.mds.b.smithi086.stdout:Mar 19 01:28:45 smithi086 podman[105735]: 2024-03-19 01:28:45.436421074 +0000 UTC m=+0.544877851 container remove f8a2b62ff9d5110359c45a6d4780a417ebf192dd64616544baa01c1a23e5468e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:28:45.646 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.b.service' 2024-03-19T01:28:45.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:28:45.684 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-19T01:28:45.685 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-19T01:28:45.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.e 2024-03-19T01:28:46.006 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:28:45 smithi086 systemd[1]: Stopping Ceph mds.e for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:28:46.506 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:28:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e[97001]: 2024-03-19T01:28:46.011+0000 7fb57d71c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:28:46.506 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:28:46 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e[97001]: 2024-03-19T01:28:46.011+0000 7fb57d71c700 -1 mds.e *** got signal Terminated *** 2024-03-19T01:28:49.444 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:28:49 smithi086 podman[105842]: 2024-03-19 01:28:49.096424753 +0000 UTC m=+3.198653630 container died 26cfff715229157fcfa9d50cb293bc10d9877882871cd3163bda0dce6ac16c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-19T01:28:49.445 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:28:49 smithi086 podman[105842]: 2024-03-19 01:28:49.444240264 +0000 UTC m=+3.546469135 container cleanup 26cfff715229157fcfa9d50cb293bc10d9877882871cd3163bda0dce6ac16c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:28:49.720 INFO:journalctl@ceph.mds.e.smithi086.stdout:Mar 19 01:28:49 smithi086 bash[105842]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-e 2024-03-19T01:28:49.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.e.service' 2024-03-19T01:28:49.894 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:28:49.894 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-19T01:28:49.895 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-19T01:28:49.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.h 2024-03-19T01:28:50.223 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:28:49 smithi086 systemd[1]: Stopping Ceph mds.h for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:28:50.506 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:28:50 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h[97318]: 2024-03-19T01:28:50.222+0000 7fc494061700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:28:50.506 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:28:50 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h[97318]: 2024-03-19T01:28:50.222+0000 7fc494061700 -1 mds.h *** got signal Terminated *** 2024-03-19T01:28:53.349 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:28:53 smithi086 podman[105961]: 2024-03-19 01:28:53.081472081 +0000 UTC m=+2.973691738 container died ed84330be6840d4dd6c6076573ead809970d59520c5c3ca32dfd1027d87e41c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-19T01:28:53.756 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:28:53 smithi086 podman[105961]: 2024-03-19 01:28:53.430806381 +0000 UTC m=+3.323026046 container cleanup ed84330be6840d4dd6c6076573ead809970d59520c5c3ca32dfd1027d87e41c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T01:28:53.756 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:28:53 smithi086 bash[105961]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h 2024-03-19T01:28:53.756 INFO:journalctl@ceph.mds.h.smithi086.stdout:Mar 19 01:28:53 smithi086 podman[105975]: 2024-03-19 01:28:53.598711913 +0000 UTC m=+0.510337382 container remove ed84330be6840d4dd6c6076573ead809970d59520c5c3ca32dfd1027d87e41c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-19T01:28:53.807 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.h.service' 2024-03-19T01:28:53.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:28:53.842 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-19T01:28:53.842 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-19T01:28:53.842 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.k 2024-03-19T01:28:54.151 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:28:53 smithi086 systemd[1]: Stopping Ceph mds.k for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:28:54.505 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:28:54 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-k[96680]: 2024-03-19T01:28:54.150+0000 7f0251924700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:28:54.506 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:28:54 smithi086 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-k[96680]: 2024-03-19T01:28:54.150+0000 7f0251924700 -1 mds.k *** got signal Terminated *** 2024-03-19T01:28:57.384 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:28:57 smithi086 podman[106081]: 2024-03-19 01:28:57.091181127 +0000 UTC m=+3.039793166 container died d18bf0781d2cabbe0bfc487d78e4e59e23eea40d54548905477bb404805f07f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-19T01:28:57.705 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:28:57 smithi086 podman[106081]: 2024-03-19 01:28:57.383832799 +0000 UTC m=+3.332444815 container cleanup d18bf0781d2cabbe0bfc487d78e4e59e23eea40d54548905477bb404805f07f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-k, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, ceph=True) 2024-03-19T01:28:57.705 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:28:57 smithi086 bash[106081]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-k 2024-03-19T01:28:57.705 INFO:journalctl@ceph.mds.k.smithi086.stdout:Mar 19 01:28:57 smithi086 podman[106095]: 2024-03-19 01:28:57.6096723 +0000 UTC m=+0.521074485 container remove d18bf0781d2cabbe0bfc487d78e4e59e23eea40d54548905477bb404805f07f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-k, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:28:57.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.k.service' 2024-03-19T01:28:57.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:28:57.750 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-19T01:28:57.750 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-19T01:28:57.750 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.c 2024-03-19T01:28:58.049 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:28:57 smithi163 systemd[1]: Stopping Ceph mds.c for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:28:58.314 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:28:58 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c[98581]: 2024-03-19T01:28:58.133+0000 7f0970efd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:28:58.314 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:28:58 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c[98581]: 2024-03-19T01:28:58.133+0000 7f0970efd700 -1 mds.c *** got signal Terminated *** 2024-03-19T01:29:01.376 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:29:01 smithi163 podman[107787]: 2024-03-19 01:29:01.10418092 +0000 UTC m=+3.161359816 container died 0c95bfbc9d13fc50e62edb6fab9b49409482dd8b09855b18617cd298e1a5f213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:29:01.726 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:29:01 smithi163 podman[107787]: 2024-03-19 01:29:01.474530399 +0000 UTC m=+3.531709286 container cleanup 0c95bfbc9d13fc50e62edb6fab9b49409482dd8b09855b18617cd298e1a5f213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-19T01:29:01.726 INFO:journalctl@ceph.mds.c.smithi163.stdout:Mar 19 01:29:01 smithi163 bash[107787]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-c 2024-03-19T01:29:01.853 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.c.service' 2024-03-19T01:29:01.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:29:01.891 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-19T01:29:01.892 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-19T01:29:01.892 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.f 2024-03-19T01:29:02.061 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:29:01 smithi163 systemd[1]: Stopping Ceph mds.f for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:29:02.561 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:29:02 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f[98273]: 2024-03-19T01:29:02.211+0000 7f19da967700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:29:02.562 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:29:02 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f[98273]: 2024-03-19T01:29:02.211+0000 7f19da967700 -1 mds.f *** got signal Terminated *** 2024-03-19T01:29:05.339 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:29:05 smithi163 podman[107920]: 2024-03-19 01:29:05.083121691 +0000 UTC m=+2.976733443 container died 274807e936ebf9b8b294db696d65626cb987b890e9747ec1b28156d4d161a5df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:29:05.598 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:29:05 smithi163 podman[107920]: 2024-03-19 01:29:05.42126822 +0000 UTC m=+3.314879980 container cleanup 274807e936ebf9b8b294db696d65626cb987b890e9747ec1b28156d4d161a5df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:29:05.599 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:29:05 smithi163 bash[107920]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f 2024-03-19T01:29:05.827 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.f.service' 2024-03-19T01:29:05.851 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:29:05 smithi163 podman[107935]: 2024-03-19 01:29:05.597363405 +0000 UTC m=+0.514005800 container remove 274807e936ebf9b8b294db696d65626cb987b890e9747ec1b28156d4d161a5df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-19T01:29:05.851 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:29:05 smithi163 systemd[1]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.f.service: Deactivated successfully. 2024-03-19T01:29:05.851 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:29:05 smithi163 systemd[1]: Stopped Ceph mds.f for 934de1c2-e58b-11ee-95c9-87774f69a715. 2024-03-19T01:29:05.851 INFO:journalctl@ceph.mds.f.smithi163.stdout:Mar 19 01:29:05 smithi163 systemd[1]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.f.service: Consumed 5.002s CPU time. 2024-03-19T01:29:05.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:29:05.864 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-19T01:29:05.864 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-19T01:29:05.864 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.i 2024-03-19T01:29:06.184 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:29:05 smithi163 systemd[1]: Stopping Ceph mds.i for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:29:06.561 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:29:06 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i[98884]: 2024-03-19T01:29:06.182+0000 7f1317a08700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:29:06.561 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:29:06 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i[98884]: 2024-03-19T01:29:06.182+0000 7f1317a08700 -1 mds.i *** got signal Terminated *** 2024-03-19T01:29:09.561 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:29:09 smithi163 podman[108041]: 2024-03-19 01:29:09.094188089 +0000 UTC m=+3.017365716 container died f72b47305772aaa4edb9aeda4a72a921b8f336ba138aa79a028b12e02e12a6eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS) 2024-03-19T01:29:09.942 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:29:09 smithi163 podman[108041]: 2024-03-19 01:29:09.674731769 +0000 UTC m=+3.597909407 container cleanup f72b47305772aaa4edb9aeda4a72a921b8f336ba138aa79a028b12e02e12a6eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:29:09.943 INFO:journalctl@ceph.mds.i.smithi163.stdout:Mar 19 01:29:09 smithi163 bash[108041]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-i 2024-03-19T01:29:10.048 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.i.service' 2024-03-19T01:29:10.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:29:10.083 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-19T01:29:10.083 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-19T01:29:10.083 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.l 2024-03-19T01:29:10.311 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:29:10 smithi163 systemd[1]: Stopping Ceph mds.l for 934de1c2-e58b-11ee-95c9-87774f69a715... 2024-03-19T01:29:10.811 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:29:10 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l[97948]: 2024-03-19T01:29:10.403+0000 7f3f5f41f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:29:10.811 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:29:10 smithi163 ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l[97948]: 2024-03-19T01:29:10.403+0000 7f3f5f41f700 -1 mds.l *** got signal Terminated *** 2024-03-19T01:29:13.337 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:29:13 smithi163 podman[108162]: 2024-03-19 01:29:13.086516362 +0000 UTC m=+2.790693600 container died d46aed8c7f24f75d1d0b06217a6bfa6aeb08910df7aa2ebafbd764ad3122fd11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-19T01:29:13.338 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:29:13 smithi163 podman[108162]: 2024-03-19 01:29:13.337523287 +0000 UTC m=+3.041700575 container cleanup d46aed8c7f24f75d1d0b06217a6bfa6aeb08910df7aa2ebafbd764ad3122fd11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-19T01:29:13.630 INFO:journalctl@ceph.mds.l.smithi163.stdout:Mar 19 01:29:13 smithi163 bash[108162]: ceph-934de1c2-e58b-11ee-95c9-87774f69a715-mds-l 2024-03-19T01:29:13.729 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-934de1c2-e58b-11ee-95c9-87774f69a715@mds.l.service' 2024-03-19T01:29:13.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T01:29:13.763 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-19T01:29:13.764 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-19T01:29:14.088 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:29:18.980 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-19T01:29:19.169 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:29:21.772 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-19T01:29:21.962 INFO:teuthology.orchestra.run.smithi163.stdout:Deleting cluster with fsid: 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:29:24.401 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T01:29:24.501 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T01:29:24.535 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T01:29:24.564 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-19T01:29:24.567 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245/remote/smithi069/crash 2024-03-19T01:29:24.569 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/crash -- . 2024-03-19T01:29:24.662 INFO:teuthology.orchestra.run.smithi069.stderr:tar: /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-19T01:29:24.663 INFO:teuthology.orchestra.run.smithi069.stderr:tar: Error is not recoverable: exiting now 2024-03-19T01:29:24.667 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245/remote/smithi086/crash 2024-03-19T01:29:24.668 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/crash -- . 2024-03-19T01:29:24.714 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-19T01:29:24.715 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-03-19T01:29:24.717 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245/remote/smithi163/crash 2024-03-19T01:29:24.718 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/crash -- . 2024-03-19T01:29:24.762 INFO:teuthology.orchestra.run.smithi163.stderr:tar: /var/lib/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-19T01:29:24.762 INFO:teuthology.orchestra.run.smithi163.stderr:tar: Error is not recoverable: exiting now 2024-03-19T01:29:24.763 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-19T01:29:24.763 DEBUG:teuthology.orchestra.run.smithi069:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-19T01:29:24.893 INFO:teuthology.orchestra.run.smithi069.stdout:2024-03-19T01:00:31.689368+0000 mon.a (mon.0) 210 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T01:29:24.894 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-19T01:29:24.894 INFO:tasks.cephadm:Compressing logs... 2024-03-19T01:29:24.894 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-19T01:29:24.898 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-19T01:29:24.901 DEBUG:teuthology.orchestra.run.smithi163:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-19T01:29:24.933 INFO:teuthology.orchestra.run.smithi163.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-03-19T01:29:24.933 INFO:teuthology.orchestra.run.smithi163.stderr: /var/log/ceph/cephadm.log 2024-03-19T01:29:24.934 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.l.log 2024-03-19T01:29:24.934 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-19T01:29:24.934 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.i.log 2024-03-19T01:29:24.934 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.11.log 2024-03-19T01:29:24.934 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mgr.z.log 2024-03-19T01:29:24.934 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.i.log: /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.8.log 2024-03-19T01:29:24.934 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mgr.z.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-19T01:29:24.935 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-19T01:29:24.935 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.f.log 2024-03-19T01:29:24.935 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-19T01:29:24.935 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-19T01:29:24.935 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.log 2024-03-19T01:29:24.935 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-19T01:29:24.935 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-19T01:29:24.936 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-19T01:29:24.941 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.log: 92.2% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mgr.z.log.gz 2024-03-19T01:29:24.941 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.c.log 2024-03-19T01:29:24.942 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-19T01:29:24.944 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.10.log 2024-03-19T01:29:24.944 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-19T01:29:24.944 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.b.log 2024-03-19T01:29:24.944 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-19T01:29:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.audit.log: /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-19T01:29:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.k.log 2024-03-19T01:29:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.h.log 2024-03-19T01:29:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.7.log 2024-03-19T01:29:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.e.log 2024-03-19T01:29:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mgr.y.log 2024-03-19T01:29:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-19T01:29:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.log 2024-03-19T01:29:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-19T01:29:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mon.b.log 2024-03-19T01:29:24.953 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.2.log: 89.5% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-19T01:29:24.953 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-19T01:29:24.954 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mon.b.log: 92.1% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mgr.y.log.gz 2024-03-19T01:29:24.959 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mon.c.log 2024-03-19T01:29:24.966 INFO:teuthology.orchestra.run.smithi069.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-19T01:29:24.967 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-19T01:29:24.968 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-19T01:29:24.970 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.6.log 2024-03-19T01:29:24.970 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mon.a.log 2024-03-19T01:29:24.971 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-19T01:29:24.972 INFO:teuthology.orchestra.run.smithi069.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-19T01:29:24.972 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.g.log 2024-03-19T01:29:24.973 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.j.log 2024-03-19T01:29:24.974 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-19T01:29:24.975 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mgr.x.log 2024-03-19T01:29:24.976 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-19T01:29:24.977 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-19T01:29:24.978 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.log 2024-03-19T01:29:24.978 INFO:teuthology.orchestra.run.smithi069.stderr: 94.5% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.g.log.gz 2024-03-19T01:29:24.979 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-19T01:29:24.980 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.9.log 2024-03-19T01:29:24.980 INFO:teuthology.orchestra.run.smithi069.stderr: 94.4% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.j.log.gz 2024-03-19T01:29:24.981 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.d.log 2024-03-19T01:29:24.981 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.a.log 2024-03-19T01:29:24.985 INFO:teuthology.orchestra.run.smithi086.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-19T01:29:24.986 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-19T01:29:24.991 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.d.log: /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.a.log: 89.5% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-19T01:29:25.015 INFO:teuthology.orchestra.run.smithi069.stderr: 83.9% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-19T01:29:25.029 INFO:teuthology.orchestra.run.smithi069.stderr: 90.2% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-19T01:29:25.101 INFO:teuthology.orchestra.run.smithi069.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-19T01:29:25.110 INFO:teuthology.orchestra.run.smithi069.stderr: 90.8% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-19T01:29:25.157 INFO:teuthology.orchestra.run.smithi086.stderr: 83.6% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-19T01:29:25.158 INFO:teuthology.orchestra.run.smithi086.stderr: 90.3% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-19T01:29:25.164 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-19T01:29:25.316 INFO:teuthology.orchestra.run.smithi069.stderr: 90.5% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mgr.x.log.gz 2024-03-19T01:29:25.469 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-19T01:29:25.559 INFO:teuthology.orchestra.run.smithi086.stderr: 94.5% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.k.log.gz 2024-03-19T01:29:25.667 INFO:teuthology.orchestra.run.smithi086.stderr: 91.7% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mon.b.log.gz 2024-03-19T01:29:25.935 INFO:teuthology.orchestra.run.smithi163.stderr: 91.5% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mon.c.log.gz 2024-03-19T01:29:25.968 INFO:teuthology.orchestra.run.smithi086.stderr: 94.9% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.h.log.gz 2024-03-19T01:29:26.186 INFO:teuthology.orchestra.run.smithi163.stderr: 94.8% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.f.log.gz 2024-03-19T01:29:26.512 INFO:teuthology.orchestra.run.smithi069.stderr: 90.5% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mon.a.log.gz 2024-03-19T01:29:31.658 INFO:teuthology.orchestra.run.smithi163.stderr: 95.2% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.l.log.gz 2024-03-19T01:29:35.544 INFO:teuthology.orchestra.run.smithi163.stderr: 95.3% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.c.log.gz 2024-03-19T01:29:37.051 INFO:teuthology.orchestra.run.smithi086.stderr: 94.5% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.e.log.gz 2024-03-19T01:29:39.914 INFO:teuthology.orchestra.run.smithi086.stderr: 92.1% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.10.log.gz 2024-03-19T01:29:44.237 INFO:teuthology.orchestra.run.smithi163.stderr: 94.0% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.i.log.gz 2024-03-19T01:29:45.116 INFO:teuthology.orchestra.run.smithi069.stderr: 93.9% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.a.log.gz 2024-03-19T01:29:45.465 INFO:teuthology.orchestra.run.smithi086.stderr: 92.7% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-19T01:29:46.459 INFO:teuthology.orchestra.run.smithi069.stderr: 93.0% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-19T01:29:48.262 INFO:teuthology.orchestra.run.smithi163.stderr: 93.3% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.11.log.gz 2024-03-19T01:29:49.559 INFO:teuthology.orchestra.run.smithi163.stderr: 92.9% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.8.log.gz 2024-03-19T01:29:50.753 INFO:teuthology.orchestra.run.smithi086.stderr: 92.3% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.7.log.gz 2024-03-19T01:29:54.914 INFO:teuthology.orchestra.run.smithi069.stderr: 93.2% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.6.log.gz 2024-03-19T01:29:56.430 INFO:teuthology.orchestra.run.smithi163.stderr: 93.3% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-19T01:29:57.147 INFO:teuthology.orchestra.run.smithi069.stderr: 92.9% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-19T01:30:02.867 INFO:teuthology.orchestra.run.smithi069.stderr: 93.1% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.9.log.gz 2024-03-19T01:30:05.898 INFO:teuthology.orchestra.run.smithi163.stderr: 93.0% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-19T01:30:05.900 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-03-19T01:30:05.900 INFO:teuthology.orchestra.run.smithi163.stderr:real 0m40.985s 2024-03-19T01:30:05.900 INFO:teuthology.orchestra.run.smithi163.stderr:user 2m33.092s 2024-03-19T01:30:05.901 INFO:teuthology.orchestra.run.smithi163.stderr:sys 0m4.943s 2024-03-19T01:30:22.298 INFO:teuthology.orchestra.run.smithi086.stderr: 93.5% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-19T01:30:59.553 INFO:teuthology.orchestra.run.smithi069.stderr: 93.2% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.d.log.gz 2024-03-19T01:30:59.557 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-03-19T01:30:59.557 INFO:teuthology.orchestra.run.smithi069.stderr:real 1m34.625s 2024-03-19T01:30:59.558 INFO:teuthology.orchestra.run.smithi069.stderr:user 3m29.175s 2024-03-19T01:30:59.558 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m16.596s 2024-03-19T01:31:49.691 INFO:teuthology.orchestra.run.smithi086.stderr: 93.4% -- replaced with /var/log/ceph/934de1c2-e58b-11ee-95c9-87774f69a715/ceph-mds.b.log.gz 2024-03-19T01:31:49.694 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-03-19T01:31:49.694 INFO:teuthology.orchestra.run.smithi086.stderr:real 2m24.777s 2024-03-19T01:31:49.694 INFO:teuthology.orchestra.run.smithi086.stderr:user 3m36.874s 2024-03-19T01:31:49.694 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m7.347s 2024-03-19T01:31:49.695 INFO:tasks.cephadm:Archiving logs... 2024-03-19T01:31:49.696 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245/remote/smithi069/log 2024-03-19T01:31:49.697 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-19T01:32:19.317 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245/remote/smithi086/log 2024-03-19T01:32:19.318 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-19T01:32:53.861 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245/remote/smithi163/log 2024-03-19T01:32:53.862 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-19T01:33:14.479 INFO:tasks.cephadm:Removing cluster... 2024-03-19T01:33:14.480 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 --force 2024-03-19T01:33:14.808 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:33:15.963 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 --force 2024-03-19T01:33:16.154 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:33:16.737 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 934de1c2-e58b-11ee-95c9-87774f69a715 --force 2024-03-19T01:33:16.932 INFO:teuthology.orchestra.run.smithi163.stdout:Deleting cluster with fsid: 934de1c2-e58b-11ee-95c9-87774f69a715 2024-03-19T01:33:17.357 INFO:tasks.cephadm:Removing cephadm ... 2024-03-19T01:33:17.357 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-19T01:33:17.414 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-19T01:33:17.430 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-19T01:33:17.445 INFO:tasks.cephadm:Teardown complete 2024-03-19T01:33:17.446 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-19T01:33:17.457 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-19T01:33:17.458 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-19T01:33:17.465 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-19T01:33:17.473 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-19T01:33:17.683 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-19T01:33:17.722 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-19T01:33:17.723 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T01:33:17.732 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T01:33:17.735 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T01:33:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-03-19T01:33:17.750 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-03-19T01:33:17.755 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T01:33:17.755 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-03-19T01:33:17.755 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 +1651us[+1651us] +/- 176ms 2024-03-19T01:33:17.755 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv02.front.sepia.ceph.com 3 6 377 28 -1024us[-1010us] +/- 77ms 2024-03-19T01:33:17.755 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +331us[ +331us] +/- 67ms 2024-03-19T01:33:17.755 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:33:17.779 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-03-19T01:33:17.788 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T01:33:17.789 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-03-19T01:33:17.789 INFO:teuthology.orchestra.run.smithi069.stdout:^- hv01.front.sepia.ceph.com 2 6 377 30 +1921us[+1956us] +/- 176ms 2024-03-19T01:33:17.789 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv02.front.sepia.ceph.com 3 6 377 28 -720us[ -684us] +/- 77ms 2024-03-19T01:33:17.789 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +633us[ +669us] +/- 67ms 2024-03-19T01:33:17.789 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:33:17.822 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T01:33:17.823 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-03-19T01:33:17.823 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 94 +1607us[+1649us] +/- 175ms 2024-03-19T01:33:17.823 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv02.front.sepia.ceph.com 3 6 377 28 -1061us[-1038us] +/- 77ms 2024-03-19T01:33:17.823 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 +289us[ +313us] +/- 67ms 2024-03-19T01:33:17.823 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:33:17.830 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-19T01:33:17.952 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-19T01:33:17.953 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-19T01:33:18.014 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-19T01:33:18.279 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-19T01:33:18.369 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-19T01:33:18.445 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T01:33:18.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:33:18.551 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T01:33:18.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:33:18.584 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T01:33:18.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:33:18.614 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-19T01:33:18.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-19T01:33:18.673 INFO:teuthology.task.internal:Duration was 4268.077689 seconds 2024-03-19T01:33:18.674 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-19T01:33:18.686 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-19T01:33:18.686 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-19T01:33:18.694 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-19T01:33:18.697 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-19T01:33:18.741 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T01:33:18.755 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T01:33:18.885 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T01:33:19.414 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-19T01:33:19.414 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2024-03-19T01:33:19.414 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-19T01:33:19.471 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-03-19T01:33:19.472 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-19T01:33:19.499 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2024-03-19T01:33:19.499 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-19T01:33:19.531 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-19T01:33:19.532 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-19T01:33:19.539 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-19T01:33:19.542 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-19T01:33:19.624 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-19T01:33:19.625 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-19T01:33:19.631 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-19T01:33:19.633 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-19T01:33:20.225 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-19T01:33:20.236 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-19T01:33:20.236 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-19T01:33:20.344 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-19T01:33:20.383 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-19T01:33:20.411 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-19T01:33:20.425 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-19T01:33:20.433 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-19T01:33:20.436 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-19T01:33:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-03-19T01:33:20.477 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2024-03-19T01:33:20.554 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2024-03-19T01:33:20.608 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-19T01:33:20.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:33:20.658 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-19T01:33:20.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:33:20.673 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-19T01:33:20.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:33:20.688 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-19T01:33:20.700 INFO:teuthology.task.internal:Transferring archived files... 2024-03-19T01:33:20.702 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245/remote/smithi069 2024-03-19T01:33:20.702 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-19T01:33:20.813 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245/remote/smithi086 2024-03-19T01:33:20.813 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-19T01:33:20.853 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245/remote/smithi163 2024-03-19T01:33:20.854 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-19T01:33:20.894 INFO:teuthology.task.internal:Removing archive directory... 2024-03-19T01:33:20.895 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-19T01:33:20.902 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-19T01:33:20.905 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-19T01:33:20.952 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-19T01:33:20.969 INFO:teuthology.task.internal:Not uploading archives. 2024-03-19T01:33:20.969 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-19T01:33:21.010 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-19T01:33:21.010 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-19T01:33:21.018 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-19T01:33:21.021 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-19T01:33:21.033 INFO:teuthology.orchestra.run.smithi086.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 19 01:33 /home/ubuntu/cephtest 2024-03-19T01:33:21.034 INFO:teuthology.orchestra.run.smithi163.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 19 01:33 /home/ubuntu/cephtest 2024-03-19T01:33:21.068 INFO:teuthology.orchestra.run.smithi069.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 19 01:33 /home/ubuntu/cephtest 2024-03-19T01:33:21.071 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-19T01:33:21.083 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-19T01:33:21.108 INFO:teuthology.nuke:Checking targets against current locks 2024-03-19T01:33:21.130 DEBUG:teuthology.nuke:shortname: smithi069 2024-03-19T01:33:21.131 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-19T01:33:21.153 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:01:51.146747', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGSTS8WjOSgm25/wh31ZgyFJookYsiny1sF8uKwrPy8zX0WPys5NAlc6S7eWESl8+Cc5hGCNUuy/06arND/TEmg='} 2024-03-19T01:33:21.158 DEBUG:teuthology.nuke:shortname: smithi086 2024-03-19T01:33:21.159 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-19T01:33:21.181 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:01:51.150374', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFmS9BN+w4hdh7UG5GLfF38yw5sMZ3EGVaDjQ7r/TUz/28kmdUjUYE0Pv+4n1b6H4hRBGlSKvSTZHY5igkRmjE4='} 2024-03-19T01:33:21.187 DEBUG:teuthology.nuke:shortname: smithi163 2024-03-19T01:33:21.187 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-19T01:33:21.218 INFO:teuthology.orchestra.console.smithi069:Power off 2024-03-19T01:33:21.219 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-19T01:33:21.229 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609245', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:01:51.148665', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKxq9gaGaFTQoAjNVdlBop7VwVaMDGxPMpyddTH+j+v06Ke7MxGbUQu30X9yjirPsvbJd8U/rv4PKhkcvil8d+0='} 2024-03-19T01:33:21.245 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2024-03-19T01:33:21.245 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:21.266 INFO:teuthology.orchestra.console.smithi086:Power off 2024-03-19T01:33:21.266 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-19T01:33:21.279 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-03-19T01:33:21.289 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2024-03-19T01:33:21.290 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:21.310 INFO:teuthology.orchestra.console.smithi163:Power off 2024-03-19T01:33:21.311 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-19T01:33:21.321 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-03-19T01:33:21.336 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2024-03-19T01:33:21.337 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:21.361 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-03-19T01:33:25.280 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:25.323 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:25.362 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:25.391 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-03-19T01:33:25.436 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-03-19T01:33:25.474 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-03-19T01:33:29.394 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:29.437 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:29.475 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:29.504 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-03-19T01:33:29.547 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-03-19T01:33:29.585 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-03-19T01:33:33.507 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:33.549 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:33.587 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T01:33:33.617 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2024-03-19T01:33:33.659 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-03-19T01:33:33.697 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2024-03-19T01:33:33.718 INFO:teuthology.orchestra.console.smithi069:Power off completed 2024-03-19T01:33:33.760 INFO:teuthology.orchestra.console.smithi086:Power off completed 2024-03-19T01:33:33.798 INFO:teuthology.orchestra.console.smithi163:Power off completed 2024-03-19T01:33:33.904 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 73069ad93affa1ccfbc13c95e629925210a73624 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/kernel_untar_build}} duration: 4268.077689409256 failure_reason: 'Command failed (workunit test kernel_untar_build.sh) on smithi069 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e48b1f6c0608b2615e1890cc46286c3877d6e910 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b2e443ee9ae9439eb8014606c2d0fbcc status: fail success: false 2024-03-19T01:33:33.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-19T01:33:33.997 INFO:teuthology.run:FAIL